SetCloud
✦ Ask AI Get started

Streaming

A chat completion doesn’t have to arrive all at once. Set stream: true and the Gateway sends it back token by token, as server-sent events, so your app can start rendering before the model has finished.

Enabling streaming

Add "stream": true to a POST /v1/chat/completions body and the response switches from a single JSON object to a text/event-stream:

curl https://setcloud.com/v1/chat/completions \
  -H "Authorization: Bearer $SETCLOUD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nova-lite",
    "messages": [{ "role": "user", "content": "Explain DNS propagation." }],
    "stream": true
  }'

Consuming server-sent events

Each event is a chat.completion.chunk. The first carries the assistant role, subsequent chunks carry content fragments in delta.content, and the final chunk sets finish_reason. The stream ends with a literal data: [DONE] line — read the body as a stream and append each chunk’s content as it arrives rather than waiting for the connection to close.

How streamed usage is billed

Streaming doesn’t change what you’re billed for — it changes when you see the total. Add "stream_options": { "include_usage": true } and one extra chunk arrives just before [DONE] carrying the full usage object for the request, the same token counts you’d get back from a non-streamed call.

Next