LLM engineering · Easy · 10 min

An OpenAI-compatible chat request

Ollama, vLLM and LiteLLM accept the same format as the OpenAI API. Build the right URL and body and validate the input before sending anything.

Write buildChatRequest(opts) that returns { url, body } for a local OpenAI-compatible server. opts has model, system (optional), user, temperature (default 0.7), maxTokens (default 512), stream (default false) and baseUrl (default http://localhost:11434).

The URL is baseUrl without a trailing slash plus /v1/chat/completions. The body uses the API's field names: model, messages (first { role: 'system', content } when there is a system, then { role: 'user', content }), temperature, max_tokens and stream.

Throw an error when model or user is empty, when temperature is not between 0 and 2, or when maxTokens is not a positive integer.

Challenges 0/4

Uses the /v1/chat/completions URL, also with a trailing slash in baseUrl
Puts the system message before the user message
Uses max_tokens and stream, with the defaults
Rejects an empty model, an out-of-range temperature and an invalid maxTokens

function buildChatRequest({ model, system, user, temperature = 0.7, maxTokens = 512, stream = false, baseUrl = 'http://localhost:11434' } = {}) {
  // 1. validate model, user, temperature (0 to 2) and maxTokens (positive integer)
  // 2. url: baseUrl without trailing slash + '/v1/chat/completions'
  // 3. body: model, messages, temperature, max_tokens, stream
  return {
    url: baseUrl + '/api/chat',
    body: { model, messages: [{ role: 'user', content: user }], temperature, maxTokens }
  };
}

console.log(buildChatRequest({ model: 'qwen2.5:7b', system: 'Responde en una frase.', user: '¿Qué es un GGUF?' }));

Console output appears here (console.log).

Go deeper: the Ollama reference →

This in production, with your data? Let's talk for 15 minutes →

Challenges 0/4

VORLUX AI