Skip to content

Concepts

A glossary-style summary of the building blocks. The full reference is in Reference → Glossary; this page is the short version.

Alias vs model

  • An alias is what you put in the model field of a request: smart-router, smart-router-flash, smart-router-pro, smart-router-free.
  • A model is the real OpenRouter identifier the gateway forwards to: deepseek/deepseek-v4-flash-0731, deepseek/deepseek-v4-pro-0813, nvidia/nemotron-3-ultra-550b-a55b:free, or the emergency fallback z-ai/glm-5.2.

You should always send aliases. The real model ID is an implementation detail. The gateway translates aliases into model IDs using the frozen registry in frozen-v3/config/models.yaml.

Roles

  • Worker / default. DeepSeek V4 Flash. Cheap, fast, used for almost everything that is not explicitly hard.
  • Senior solver. DeepSeek V4 Pro. Used when risk, complexity, or repeated failure makes Flash the wrong tool.
  • Scout / critic. Nemotron 3 Ultra. Used only for non-sensitive gray-zone decisions. The Scout does not author your answer; it recommends Pro or Flash, and the gateway may still override based on policy.
  • Emergency fallback. GLM-5.2. Only used if a call to Pro fails end-to-end after the gateway's own retries.

Routing decision

A RoutingDecision is the structured object the gateway produces internally. It records:

  • which model was chosen,
  • why it was chosen (cause),
  • the action (ROUTE, SCOUT, STOP_FOR_HUMAN, DENY),
  • whether a hard rule fired,
  • and (when the answer was a Scout) which downstream model the Scout recommended.

The dry-run endpoint POST /v1/route/decision returns the same structure without forwarding the request to OpenRouter. It is the right tool for debugging routing behavior.

Task vs request

A request is a single HTTP call to /v1/chat/completions. A task is the abstract unit of work the client is trying to accomplish; it can span many requests (e.g. an Aider session that tries an edit, sees a test fail, retries with extra context, then asks the model again). Counters such as "same error twice" only make sense across requests inside the same task.

If the client does not send a stable task ID, the gateway treats every request as a new task. It still routes correctly, but the runtime-evidence layer (e.g. "escalate after the same error twice") collapses to per-request only.

Session, goal, task

  • Session — A logical coding session, usually a single editor process. Conveyed by X-Smart-Gateway-Session (preferred), the metadata.session_id field, or the OpenAI user field.
  • Goal — A larger unit of work inside a session ("fix the auth bug"). Conveyed by X-Smart-Gateway-Goal or metadata.goal_id.
  • Task — A single request to the chat completions endpoint, or a small bounded exchange of requests. Conveyed by X-Task-Id or metadata.task_id.

The gateway uses these to scope budget counters and runtime evidence. They never appear in the upstream OpenRouter request.

Privacy tiers

  • Sensitive — Default. Any request that touches private code, internal documentation, customer data, or anything that should not appear in a public model provider's logs.
  • Non-sensitive — Explicitly declared. Public documentation, open-source code you own, benchmark prompts with no proprietary content.

The default is sensitive = true. If you set metadata.sensitive = false you are asserting that the request is safe to send to the free Nemotron endpoint, and the gateway will accept that assertion. The gateway also runs a content-level redaction pass and will silently upgrade a request to sensitive = true if it detects a likely secret in the message body (see Security → Sensitive data).

Budget

A per-goal budget with soft and hard caps:

Cap Default Behavior on exceed
Per-goal soft $2.00 Pro is no longer used; Flash and Scout are still allowed.
Per-goal hard $5.00 STOP_FOR_HUMAN. The gateway refuses the request and asks the operator.
Per-day soft $10.00 Same as soft cap, scoped to a calendar day.
Per-day hard $25.00 STOP_FOR_HUMAN, scoped to a calendar day.

In production these caps are enforced by a PostgreSQL-backed budget ledger with optimistic concurrency control. If the database is unreachable, the gateway enters SAFE_DEGRADED and the policy refuses to silently downgrade. See Architecture → Budget for the full state machine; the operator playbook for PostgreSQL recovery is in docs-internal/operations/postgres-recovery.md (operator-only).

Health states

Each upstream model has a small state machine:

  • HEALTHY — normal operation.
  • DEGRADED — elevated latency or error rate, prefer other models.
  • UNHEALTHY — high error rate, refuse to send.
  • HALF_OPEN — in cooldown, allow a small number of probes.
  • DISABLED — administratively disabled.
  • STALE — no recent data, only limited probes.
  • QUARANTINE — failed quality regression, manual clearance required.

If your selected model is UNHEALTHY and no fallback is healthy, the gateway returns STOP_FOR_HUMAN rather than failing the request silently.

Phases and tags

  • SGW-BLUEPRINT-2026-08-18-01 — the routing policy freeze ID. The gateway refuses to start with a different value.
  • SGW-BLUEPRINT-2026-08-18-02 — the council protocol freeze ID. Affects governance, not runtime behavior.
  • SGW-V3-DESIGN-FREEZE-2026-08-20-01 — the architecture freeze that produced v1.0.0.
  • v1.0.0 — the first production release tag.
  • evidence-complete-v1.0.0 — the post-release evidence tag (POST-RELEASE-E3 + POST-RELEASE-E4 + final review).

What "v1.0.0" actually freezes

The release v1.0.0 freezes:

  • The set of model IDs and their roles in frozen-v3/config/models.yaml.
  • The routing rule chain and all thresholds in frozen-v3/config/routing-policy.yaml.
  • The OpenAI-compatible surface (/v1/chat/completions, /v1/route/decision, /healthz, /metrics).
  • The privacy gate's fail-closed behavior.
  • The deployment topology (gateway on loopback, host nginx terminating TLS, isolated PostgreSQL network).

It does not freeze:

  • The OpenRouter upstream pricing or quota behavior.
  • Let's Encrypt certificate lifetimes.
  • The exact Aider/IDE versions used during evidence runs.
  • The internal evidence-review tooling (scripts/e4_real/, artifacts/).