SetCloud
✦ Ask AI Get started

API reference

The AI Gateway supports the chat-completions, embeddings, and model-list endpoints documented here. Configure a compatible client with the base URL and a gateway key; other endpoints and SDK features are not implied.

Base URL: https://setcloud.com/v1

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.SETCLOUD_API_KEY,
  baseURL: 'https://setcloud.com/v1',
});

Authentication

Every request needs a bearer token:

Authorization: Bearer sk-sc-...

Create keys in the dashboard under AI Gateway → API Keys. The full key is shown once at creation and never again.

  • Up to 25 active keys per account. Revoke unused keys to free a slot.
  • Each key can be restricted to specific models and carries its own rate limits, set at creation.
  • Requests bill token costs against your prepaid credit balance.

Missing or invalid keys return 401:

{ "error": { "message": "Invalid API key", "type": "authentication_error", "code": "invalid_api_key" } }

POST /v1/chat/completions

Create a chat completion.

Request body

Field Type Required Notes
model string yes A model id from GET /v1/models — registry models or your own fine-tuned custom-* models
messages array yes At least one { "role": "system" | "user" | "assistant" | "tool", "content": string }. developer is accepted as an alias of system. Assistant messages may carry tool_calls (with content null); tool messages answer one via tool_call_id
temperature number no Sampling temperature
max_tokens integer no Output cap; bounded by the model’s maximum
max_completion_tokens integer no Accepted as an alias of max_tokens
top_p number no Nucleus sampling
stop string | array no A stop sequence, or an array of them
stream boolean no true switches the response to server-sent events
stream_options object no { "include_usage": true } adds a final usage chunk to streams
tools array no Function definitions the model may call — see Tool calling
tool_choice string | object no "auto", "none", "required", or { "type": "function", "function": { "name": "..." } }. Ignored when tools is absent

These are the only fields the API honors. Unrecognized fields are ignored, and tool_choice without tools is ignored rather than rejected.

Example — curl

curl https://setcloud.com/v1/chat/completions \
  -H "Authorization: Bearer $SETCLOUD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nova-lite",
    "messages": [
      { "role": "system", "content": "You are terse." },
      { "role": "user", "content": "What is a CNAME record?" }
    ],
    "max_tokens": 200
  }'

Example — JavaScript (fetch)

const res = await fetch('https://setcloud.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.SETCLOUD_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'nova-micro',
    messages: [{ role: 'user', content: 'Summarize this release note in one line.' }],
  }),
});
const data = await res.json();
console.log(data.choices[0].message.content);

Response

{
  "id": "chatcmpl-9f3c2a1b7d",
  "object": "chat.completion",
  "created": 1718064000,
  "model": "nova-micro",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "An alias pointing one hostname at another." },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 18,
    "completion_tokens": 11,
    "total_tokens": 29
  }
}

usage is what you’re billed for — token costs are deducted from your credit balance per request, with a minimum charge of $0.01 per request (this minimum applies to every /v1 endpoint).

finish_reason is "stop" when the model finished naturally, "length" when it hit the token cap, "tool_calls" when it wants you to run a tool, or "content_filter".

Tool calling

Pass tools and the model can ask your code to run a function. When it does, the assistant message carries tool_calls and finish_reason is "tool_calls". Its content may contain accompanying text or be null. Use the returned assistant message unchanged when continuing the conversation.

curl https://setcloud.com/v1/chat/completions \
  -H "Authorization: Bearer $SETCLOUD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nova-lite",
    "messages": [{ "role": "user", "content": "What is the weather in Lisbon?" }],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Current weather for a city",
          "parameters": {
            "type": "object",
            "properties": { "city": { "type": "string" } },
            "required": ["city"]
          }
        }
      }
    ]
  }'
{
  "id": "chatcmpl-4b91d2c8ae",
  "object": "chat.completion",
  "created": 1751328000,
  "model": "nova-lite",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_a3f8c91b2d",
            "type": "function",
            "function": { "name": "get_weather", "arguments": "{\"city\":\"Lisbon\"}" }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": { "prompt_tokens": 312, "completion_tokens": 43, "total_tokens": 355 }
}

Run the function, append the assistant message and a tool message with the result, and send the conversation back for the final answer:

{ "role": "tool", "tool_call_id": "call_a3f8c91b2d", "content": "{\"temp_c\":21,\"sky\":\"clear\"}" }

Steer it with tool_choice: "auto" (the default) lets the model decide, "none" blocks tool use, "required" forces a call, and { "type": "function", "function": { "name": "get_weather" } } forces one specific function.

Some models support automatic tool selection only. An unsupported forced choice is rejected before inference; use "auto" with those models.

When a model needs private continuation state, SetCloud keeps it encrypted for up to one hour and returns ordinary opaque tool-call IDs. Keep the original messages, tool-call IDs, model, system prompt and tool definitions intact, and submit tool results within that hour using the same account and organization. No extra request field is needed. Private reasoning and signatures are not included in the API response.

Up to 64 saved continuations can remain active for one user, organization and model at a time. A continuation supports at most 128 tool calls and 512 KiB of saved assistant content. A full store blocks new tool conversations, but an existing conversation with verified tool results can still finish. Any additional round of tool calls requires storage capacity. A full or unavailable continuation store returns continuation_unavailable; wait for capacity to recover. Invalid, changed or expired state returns continuation_invalid. Start a new conversation with a summary when state expires, and do not repeat tool actions that already completed. If inference finishes but its continuation cannot be saved, measured model usage is still charged and no tool calls are released.

Streaming: with "stream": true, tool calls arrive as delta.tool_calls fragments — the first fragment of each call carries its id and function.name, later fragments append to function.arguments. The final chunk sets finish_reason: "tool_calls".

Streaming

Set "stream": true and the response becomes text/event-stream. Each event is a chat.completion.chunk; the first chunk carries delta.role, content arrives in delta.content (tool calls in delta.tool_calls), the final chunk sets finish_reason ("stop", "length", or "tool_calls"), and the stream ends with data: [DONE]. Add "stream_options": { "include_usage": true } and one extra chunk before [DONE] reports token usage.

data: {"id":"chatcmpl-9f3c2a1b7d","object":"chat.completion.chunk","created":1718064000,"model":"nova-micro","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-9f3c2a1b7d","object":"chat.completion.chunk","created":1718064000,"model":"nova-micro","choices":[{"index":0,"delta":{"content":"An alias"},"finish_reason":null}]}

data: {"id":"chatcmpl-9f3c2a1b7d","object":"chat.completion.chunk","created":1718064000,"model":"nova-micro","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]
const res = await fetch('https://setcloud.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.SETCLOUD_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'nova-lite',
    messages: [{ role: 'user', content: 'Explain DNS propagation.' }],
    stream: true,
  }),
});

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split('\n');
  buffer = lines.pop() ?? '';
  for (const line of lines) {
    if (!line.startsWith('data: ')) continue;
    const payload = line.slice(6);
    if (payload === '[DONE]') continue;
    const chunk = JSON.parse(payload);
    process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
  }
}

If the stream is interrupted server-side, the last event is an error payload: {"error":{"message":"Stream interrupted","type":"server_error","code":"stream_error"}}.

POST /v1/embeddings

Turn text into vectors for search, clustering, and retrieval.

Request body

Field Type Required Notes
model string yes embed-text-v2
input string | array yes One string, or an array of 1–96 strings, each up to 50,000 characters. Token arrays are not supported — send text
dimensions integer no 256, 512, or 1024 (the default)
encoding_format string no "float" (the default) or "base64"

Request bodies are capped at 1 MB.

Example — curl

curl https://setcloud.com/v1/embeddings \
  -H "Authorization: Bearer $SETCLOUD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "embed-text-v2",
    "input": ["How do CNAME records work?", "DNS propagation explained"],
    "dimensions": 512
  }'

Response

{
  "object": "list",
  "data": [
    { "object": "embedding", "index": 0, "embedding": [0.0132, -0.0208, ...] },
    { "object": "embedding", "index": 1, "embedding": [0.0087, 0.0154, ...] }
  ],
  "model": "embed-text-v2",
  "usage": { "prompt_tokens": 14, "total_tokens": 14 }
}

With "encoding_format": "base64", each embedding is instead a base64 string encoding the float32 vector — this is what the OpenAI Python SDK requests by default, and it decodes it back to floats for you.

Embeddings bill input tokens only, deducted from your credit balance — the $0.01-per-request minimum applies here too. usage.prompt_tokens is what you’re billed for; see the pricing page for current per-token rates.

LangChain: OpenAIEmbeddings defaults to sending pre-tokenized input (token arrays), which the API doesn’t accept. Set check_embedding_ctx_length=False so it sends plain text, and keep chunk_size at 96 or below:

import os
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    model="embed-text-v2",
    base_url="https://setcloud.com/v1",
    api_key=os.environ["SETCLOUD_API_KEY"],
    check_embedding_ctx_length=False,  # send text, not token arrays
    chunk_size=96,
)

GET /v1/models

List the models your key can use. The response below is illustrative, not a complete or guaranteed catalog.

curl https://setcloud.com/v1/models \
  -H "Authorization: Bearer $SETCLOUD_API_KEY"
{
  "object": "list",
  "data": [
    { "id": "nova-pro", "object": "model", "created": 1700000000, "owned_by": "setcloud" },
    { "id": "nova-lite", "object": "model", "created": 1700000000, "owned_by": "setcloud" },
    { "id": "nova-micro", "object": "model", "created": 1700000000, "owned_by": "setcloud" },
    { "id": "embed-text-v2", "object": "model", "created": 1700000000, "owned_by": "setcloud" }
  ]
}

Your fine-tuned models appear here too, as custom-* ids with owned_by: "user", visible only to your account. If the key has a model allow-list, the list is filtered to it.

Per-model rates are on the pricing page and in your dashboard.

Errors

Errors use the OpenAI format:

{ "error": { "message": "...", "type": "...", "code": "..." } }
Status Type Code When
400 invalid_request_error invalid_request Malformed body — missing model, messages, or input; bad tools/tool_choice shapes; or a request the model itself rejects
400 invalid_request_error model_not_found Unknown model id
401 authentication_error missing_api_key No Authorization header
401 authentication_error invalid_api_key Key doesn’t validate
402 insufficient_quota insufficient_credits Credit balance is empty — deposit credits in your dashboard
402 spend_budget_reached spend_budget_reached Your account’s spend cap has paused AI Gateway usage — raise it or disable auto-pause in your dashboard
402 budget_exceeded budget_exceeded Your organization’s monthly AI budget is reached — raise it in your dashboard or wait for next month
403 invalid_request_error model_not_allowed Key’s allow-list excludes the requested model
404 invalid_request_error model_not_supported The model doesn’t support the endpoint — e.g. a chat completion with embed-text-v2
409 invalid_request_error continuation_invalid Saved tool context expired or does not match the original conversation; do not repeat completed tool effects
413 invalid_request_error payload_too_large Request body over 1 MB
429 rate_limit_error rate_limit_exceeded Per-key RPM or TPM limit hit, or the model is briefly over capacity — wait Retry-After seconds
500 server_error internal_error, model_error Something failed on our side — retry
503 server_error continuation_unavailable Tool continuation capacity or storage is unavailable; measured inference usage, if any, remains charged

Spend caps apply to /v1 traffic the same way they apply to the rest of the AI Gateway — see spend caps and limits for what pausing does and doesn’t affect.

Rate limits

Limits apply per key, per minute:

  • Requests: 60 RPM by default.
  • Tokens: 100,000 TPM by default.

Both are configurable per key when you create it in the dashboard. A 429 response includes a Retry-After header with the seconds to wait.

Batched embeddings requests weigh more: each input item in a POST /v1/embeddings call counts as one request against the RPM limit (20 inputs use 20 request units).