Chat API

POST /chat/completions runs a prompt through the input filters, calls the provider, runs the reply through the output filters, and returns the result as a Server-Sent Events (SSE) stream.

Endpoint

Request body

Validation runs with whitelisting: a field this table does not list is stripped from the body without an error. A misspelled temperture produces a successful call at the default temperature.

Message format

role is system, user or assistant. content is either a string or an array of content parts.

{ "role": "user", "content": "Message content" }

{
  "role": "user",
  "content": [
    { "type": "text", "text": "What is in this image?" },
    { "type": "image_url", "image_url": { "url": "data:image/png;base64,iVBORw0KG…", "detail": "auto" } }
  ]
}

{
  "role": "user",
  "content": [
    { "type": "text", "text": "Summarise this contract" },
    { "type": "document", "document": { "data": "JVBERi0xLjQK…", "type": "pdf", "name": "contract.pdf" } }
  ]
}

A document part takes type from pdf, docx, txt, csv or xlsx. Image and document content is inspected and redacted before it reaches the model, like text.

Example request

curl -N -sS -X POST "https://<appliance-fqdn>/chat/completions" \
  -H "Authorization: Bearer cid_key_0123456789abcdef…" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "provider": "openai",
    "messages": [
      { "role": "system", "content": "You are a helpful customer service agent." },
      { "role": "user", "content": "My name is John Smith and my email is john@example.com" }
    ]
  }'

What the client receives — the whole exchange, in order:

data: {"type":"token_usage","usage":{"prompt_tokens":320,"completion_tokens":12,"total_tokens":332}}

data: {"id":"filtered-response","content":"Hello. How can I help you today?","finish_reason":"stop","filtered":false,"entities_masked":0}

data: [DONE]

The name and email were masked before the provider saw them, but the event that reports that masking is not forwarded on this endpoint. To see what was detected, open Detection & Filtering → All Detections in the dashboard, or send the same text to POST /api/v1/guardrails/detect, which returns the detections synchronously.

The SSE contract

The provider streams token deltas. CID222 consumes every one of them, accumulates the text, runs output filtering over the complete reply, and only then writes the answer to your connection. A client therefore sees: optional control events, then silence for the length of the generation, then one content event, then data: [DONE].

Each line is data: followed by one JSON object, and blank-line separated. The stream always ends with data: [DONE].

Events, in the order they can appear

Events 1 and 2 come from the LLM-review lock gate and appear only when a review has locked the user or left a pending warning. Events 4 to 6 appear only when model routing or prompt compression is enabled. Event 3 appears only when the request carried documents. Exactly one of 9a and 9b occurs; 9c can follow 9b.

Note the ordering of events 8 and 9: token_usage is forwarded while the reply is still being buffered, so it arrives before the answer, not after it.

Reading it

The one event that carries model text is identified by "id":"filtered-response", not by a type. Match on that:

if event.get("id") == "filtered-response":
    answer = event["content"]      # the whole reply, exactly once

Do not accumulate content across events. There is only one.

token_usage variants

usage always carries prompt_tokens, completion_tokens and total_tokens. Prompt-cache figures are added by the providers that report them, under provider-specific names: cached_input_tokens on OpenAI, cache_read_input_tokens and cache_creation_input_tokens on Anthropic. Read them defensively.

Events that do not exist

Do not write a client that waits for any of these — nothing emits them:

• content_block_delta, delta.text, message_stop, chatcmpl-… incremental ids, and any OpenAI or Anthropic streaming frame.
• input_rejected. The input-rejection event is content_rejected.
• hallucination. The retrieval-grounding check runs in the background after the reply has been returned. It writes a detection record and a session warning; it never reaches the stream.

Events that exist but are not forwarded here

• user_message_processed, which reports what was masked in the prompt, is emitted internally and swallowed by the orchestrator on /chat/completions. It is forwarded on POST /sessions/:id/messages.
• Every per-token provider chunk, the provider's own finish_reason chunk, and the provider's own [DONE].

Blocked content

A policy block is not an HTTP error. The response status is 200 and the block arrives as an event, so a client that checks only the status code sees success.

Blocked input — the provider is never called and no provider tokens are spent:

data: {"error":"Content contains prohibited information","type":"content_rejected","entities":[{"type":"jailbreak","action":"reject","confidence":0.96}]}

data: [DONE]

Blocked output — the model generated a reply, but the output filter refused to return it:

data: {"type":"token_usage","usage":{"prompt_tokens":88,"completion_tokens":140,"total_tokens":228}}

data: {"type":"output_content_rejected","severity":"critical","message":"🚫 Response blocked due to policy violations","reason":"Credit card detected in response","entities_detected":1,"actions_applied":["reject"]}

data: [DONE]

On an unhandled server-side failure the controller writes data: {"error":"An error occurred"} and closes the stream.

Supported models

model accepts any model your tenant holds an active provider credential for. Call GET /models for the live list. The seeded catalogue is:

openai and azure_openai both register gpt-4o, gpt-4o-mini and gpt-4-turbo. Send provider alongside model to say which one you mean. Azure's GPT-3.5 deployment is gpt-35-turbo, without the dot.

Errors

There is no 429 on this endpoint: /chat/completions is not rate limited.

Other chat endpoints

Related

• API overview — base paths, credentials and error shapes across surfaces.
• Sessions API — the same pipeline with server-side conversation state.
• Content detection — scan text without calling a provider.