Making requests

This page describes the exact wire format the gateway expects. The gateway is OpenAI Chat Completions-compatible, so most OpenAI SDKs work without changes — you only need to point the base URL at the gateway and pass the gateway API key.

Minimal request

{
  "model": "smart-router",
  "messages": [
    {"role": "user", "content": "Hello, who are you?"}
  ]
}

Fields used by the gateway:

Field	Type	Notes
model	string	Alias. See Model aliases.
messages	array	OpenAI chat completion messages.
stream	bool	Optional. Enables SSE.
metadata	object	Optional. Gateway-specific routing metadata (see below).
temperature, top_p, max_tokens, stop, tools, tool_choice, response_format, seed, user, …	various	Forwarded to OpenRouter unchanged.

metadata fields read by the gateway:

Key	Type	Default	Effect
routing	string	auto	Hint, not a control directive. pro/flash/free are honored only if they don't conflict with the privacy gate.
reasoning	string	auto	high forces Pro unless blocked by the budget soft cap.
sensitive	bool	true	If true, the privacy gate applies.
session_id	string	—	Equivalent to X-Smart-Gateway-Session.
goal_id	string	—	Equivalent to X-Smart-Gateway-Goal.
task_id	string	—	Equivalent to X-Task-Id.

The same fields can be sent as HTTP headers (preferred for control directives, see Authentication).

Example: ask the default router

curl -sS "$SMART_GATEWAY_BASE_URL/v1/chat/completions" \
  -H "Authorization: Bearer $SMART_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "smart-router",
        "messages": [{"role": "user", "content": "What does HTTP 429 mean?"}],
        "max_tokens": 60
      }'

Example: force Pro

curl -sS "$SMART_GATEWAY_BASE_URL/v1/chat/completions" \
  -H "Authorization: Bearer $SMART_GATEWAY_API_KEY" \
  -H "X-Pro: true" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "smart-router",
        "messages": [{"role": "user", "content": "Sketch a design for a per-process semaphore."}],
        "max_tokens": 400
      }'

X-Pro: true and the smart-router-pro alias are equivalent. The header is convenient when a client cannot set model dynamically.

Example: streaming

Add "stream": true and parse the SSE response. See Streaming for the exact chunk shape and a worked example.

Example: multimodal is rejected

The gateway does not implement vision in v1.0.0. Sending an image content part returns HTTP 400:

{
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "What is in this image?"},
        {"type": "image_url", "image_url": {"url": "https://example.com/x.png"}}
      ]
    }
  ]
}

Returns:

HTTP/1.1 400 Bad Request

If you need vision, use OpenRouter directly until v1.x adds it. The policy layer explicitly rejects multimodal in v1 (frozen-v3/00-architecture-decisions.md).

Example: dry-run a routing decision

If you want to know which model the gateway would pick without making the upstream call, use /v1/route/decision:

curl -sS "$SMART_GATEWAY_BASE_URL/v1/route/decision" \
  -H "Authorization: Bearer $SMART_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "task": {
          "task_id": "t1",
          "goal_id": "g1",
          "kind": "DEBUG",
          "instruction": "Find the race in the auth refresh path.",
          "sensitive": true,
          "risk_level": "HIGH",
          "blast_radius": "HIGH",
          "reasoning_requirement": 0.8
        },
        "runtime": {
          "attempts": 1,
          "same_error_count": 1,
          "tool_depth": 9,
          "files_touched": 8
        }
      }'

The response is a RoutingDecision JSON object. See API reference → Route decision.

Timeouts, retries

The gateway has its own retry policy against OpenRouter (exponential backoff, up to 3 attempts, only for 429/5xx and network errors — see src/smart_gateway/clients/openrouter.py). It does not retry 4xx errors.

Client-side, the standard OpenAI SDK retry advice applies: retry on 429 and 5xx, never on 4xx, and use a max of 3 attempts with exponential backoff. Streaming responses should be reissued only on connection errors, not on partial completions.

The gateway honors proxy_read_timeout 300s from the production nginx vhost, so requests that take more than 5 minutes will be terminated by the proxy. If you need longer, talk to the operator.

Rate limits

The gateway itself does not enforce rate limits. OpenRouter rate limits still apply on the upstream. If you hit a 429 from OpenRouter, the gateway logs it as FALLBACK or provider_errors and the underlying metrics increment with status="429" on the sgw_requests_total counter.

Logging and what is preserved

Every request produces a structured JSON log line with:

• task_id
• decision.action (ROUTE/SCOUT/STOP_FOR_HUMAN)
• decision.model
• decision.cause
• level, message, logger, timestamp

No request body, no response body, no message content, and no secrets are logged. See Security → Sensitive data for the redaction policy.