Smart Routing

Routing Modes

The proxy supports four routing modes, controlled by routing.mode in config:

• auto, Always classify and route by complexity, even when the client sends a specific model name. Recommended for maximum savings.
• cascade, Start with a cheap model, escalate if the response shows uncertainty or refusal.
• standard, Use learned routing rules from the core engine.
• passthrough, No routing; forward requests as-is to the specified provider.

Complexity Classification

The classifier analyzes only the last user message (not system prompts) and assigns a numeric score:

Score mapping: <2 = simple, 2-3 = moderate, 4-15 = complex, ≥ 16 = elite

Elite Tier (score ≥ 16)

The elite tier targets requests that demand frontier reasoning: multi-system architecture design, long-horizon research synthesis, or any task where score accumulates to 16 or above. By default, auto-routing does not promote a request to elite. You must opt in explicitly.

Three ways to reach elite:

• Explicit model alias, request rp:reason and RelayPlane routes to the configured elite model.
• Auto opt-in, set routing.allow_elite_auto: true in config and requests that score ≥ 16 are automatically promoted.
• Suffix routing, append :quality to any model name, which resolves to the highest configured tier.

The elite model defaults to claude-fable-5 (Anthropic) or gpt-5.5 (OpenAI), and is configurable via routing.complexity.elite in config.

Task Type Inference

Separately from complexity, the core engine infers one of 9 task types from the prompt text. This is used for telemetry categorization and learned routing rules.

Cascade Escalation

In cascade mode, the proxy detects when a cheaper model's response is insufficient:

• Uncertainty, phrases like "I'm not sure", "it's hard to say", "I can't definitively"
• Refusal, phrases like "I can't help with that", "as an AI"
• Error, provider returns an error

When escalation triggers, the request is re-sent to the next model in the cascade chain.

Model Resolution

The proxy resolves models in this order:

1. Check modelOverrides in config
2. Check RelayPlane aliases (relayplane:auto → rp:balanced)
3. Check smart aliases (rp:best, rp:fast, etc.)
4. Check model mapping (sonnet → claude-sonnet-5)
5. Auto-detect by prefix (claude-* → Anthropic, gpt-* → OpenAI, etc.)
6. Try provider/model format (anthropic/claude-3-5-sonnet-latest)

Response Headers

Every proxied response includes routing metadata headers:

• x-relayplane-routed-model, Actual model used
• x-relayplane-requested-model, Original model requested
• x-relayplane-complexity, Inferred complexity level
• x-relayplane-provider, Provider used
• x-relayplane-routing-mode, Which routing mode was applied

Suffix Routing

Append a routing suffix to any model name to override the routing strategy for that request:

• claude-sonnet-5:cost, Force cost optimization
• gpt-4o:quality, Force highest quality
• claude-3-5-haiku:fast, Force fastest model

Credential Quarantine on Auth Failure

When routing requests across a credential pool, the proxy auto-quarantines any credential that returns two consecutive 401 auth-failure responses. The quarantined credential is held out of the pool for one hour, and the router falls back to the next credential in the pool so traffic keeps flowing. This prevents a single revoked or rate-limited key from breaking every subsequent request routed through it.