Skip to content

Routing troubleshooting

When the routing decision is unexpected, work through this page. Operator-only actions are summarized briefly here; the full daily operations playbook is at docs-internal/operations/daily-operations.md (operator-only).

Tools

The fastest way to debug a routing decision is the dry-run endpoint:

curl -sS "$BASE/v1/route/decision" \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d @request-body.json

The response includes cause, evidence, score, hard_rule, and sensitive_block_applied. Map these to the rule chain in Architecture → Routing pipeline.

Decision did not match my expectation

Symptom: I asked for Pro but got Flash.

Possible causes:

  • Soft budget cap fired. Pro is disabled when the per-goal or per-day soft cap is exceeded. The decision's cause will be score_flash (not score_pro) and hard_rule will be false.
  • Hard rule did not fire. Your instruction may not have a high-reasoning marker, and the task kind may not match a hard Pro rule. Inspect evidence.
  • The score is in the gray zone. Score is between flash_max and pro_min. If non-sensitive, Scout is invoked. If sensitive, the midpoint rule decides.
  • Pro was unhealthy. The model-health registry substituted a fallback. Look for FALLBACK_TRY in the routing events.

Symptom: I asked for Flash but got Pro.

Possible causes:

  • A hard Pro rule fired. The task kind, risk level, or blast radius matched. The decision's evidence array lists which.
  • High-reasoning marker present. Look at your instruction for /pro, [reasoning:high], "think deeply", etc.
  • Structural failure or same-error-twice. Runtime signals triggered hard Pro.

Symptom: I asked for the free model but got 403.

This is by design. The free model cannot receive sensitive content. See Security → Free-model policy.

To use the free model, mark the request as metadata.sensitive: false and verify the content is genuinely public.

Symptom: The same prompt sometimes routes Flash and sometimes Pro.

Possible causes:

  • Gray-zone flapping. Score near the band edges. Tighten the band by adjusting flash_max / pro_min — but that requires a release.
  • Health-driven substitution. Flash and Pro have different health states at different times. Look at the health registry state via metrics.
  • High-reasoning marker variation. If your prompt template sometimes includes "think deeply" and sometimes does not, the routing will flap.

Decision contradicts the dry-run

If /v1/route/decision says Pro but /v1/chat/completions returns Flash (or vice versa), the most likely cause is the model health registry:

  • /v1/route/decision does not consult the health registry.
  • /v1/chat/completions does.

So a model that is HEALTHY at dry-run time may be UNHEALTHY at request time, and the gateway substitutes.

Degraded mode

If you see STOP_FOR_HUMAN with cause=degraded_mode, the gateway has lost its PostgreSQL connection. There is no client-side fix; the operator must investigate. The PostgreSQL recovery playbook is at docs-internal/operations/postgres-recovery.md (operator-only).

Score anomalies

The complexity score is computed from eight dimensions, each with a weight in routing-policy.yaml::score.weights. The default weights are:

Dimension Weight
reasoning_requirement 20
ambiguity 10
blast_radius 15
dependency_depth 10
tool_depth 10
verification_difficulty 10
runtime_failure 15
business_risk 10

If the score is consistently wrong for your workload, the inputs are wrong. The runtime extractor normalizes errors but does not read your mind about complexity. Send accurate reasoning_requirement, dependency_depth, etc. in metadata if you want the score to reflect your judgment.

Forced behavior

If the decision is STOP_FOR_HUMAN and the detail names a specific cause, the action is intentional. Common causes:

Cause Meaning
free_route_blocked_by_privacy Sensitive + free. Set sensitive=false or use a paid tier.
budget_hard_cap_exceeded Budget hard cap fired. Wait for the next period, or escalate to the operator.
all_models_unhealthy No upstream model is healthy. Wait or escalate.
degraded_mode PostgreSQL is down. The gateway cannot safely make a decision.

See also