Skip to content

Free-model policy

The free model is nvidia/nemotron-3-ultra-550b-a55b:free, served by OpenRouter. The gateway's relationship to it is narrow and conservative.

Why the policy is conservative

OpenRouter's terms for the Nemotron free endpoint explicitly warn against confidential or personal data. The gateway treats that as a hard architectural boundary, not a cost preference. Sending private code, customer data, or PII to a free public model is not acceptable in this deployment, regardless of how cheap it would be.

What the free model is allowed to do

Two narrow jobs, both gated by task.sensitive=false:

  1. Scout in the gray zone. The free model is asked a single question: "PRO or FLASH?" Its recommendation is advisory; the policy can still override it.
  2. Direct execution for SUMMARIZE, CLASSIFY, and PUBLIC_REVIEW tasks that are non-sensitive and (if repo context is present) explicitly opted in.

Anything else goes to a paid model (Flash or Pro).

What the free model is forbidden from doing

  • Receiving sensitive content.
  • Receiving private repository content unless the client set repo_free_opt_in=true (an explicit opt-in).
  • Receiving any request that was forced free (smart-router-free, X-Scout: true, metadata.routing="free").

The enforcement

# in src/smart_gateway/policy.py::decide

# 1) Explicit route to NEMOTRON:
if task.explicit_route == Route.NEMOTRON:
    if _free_allowed(task, policy):
        return RoutingDecision(... model=Route.NEMOTRON ...)
    return RoutingDecision(
        action=Action.STOP_FOR_HUMAN,
        cause="free_route_blocked_by_privacy",
        sensitive_block_applied=True,
        ...
    )

The STOP_FOR_HUMAN decision becomes an HTTP 403 with the standard privacy error. There is no setting in v1.0.0 that turns this off.

The "ALLOW_FREE_MODEL_FOR_PRIVATE_REPOS" environment knob

The deployment has a single environment variable that the operator can flip:

# env.production.template
ALLOW_FREE_MODEL_FOR_PRIVATE_REPOS=false

The v1.0.0 production deployment sets this to false. Flipping it requires a privacy review and a release. The change does not bypass the per-request privacy gate; it only changes the operator's standing policy.

How to ask for free-tier work safely

Use the default router and mark sensitive=false:

curl -sS "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "smart-router",
        "messages": [{"role":"user","content":"<public doc>"}],
        "metadata": {"sensitive": false}
      }'

The policy will pick Flash for most tasks. The free model is invoked only on SUMMARIZE / CLASSIFY / PUBLIC_REVIEW or in the gray zone. Either way, you stay in control.

See also