Routing and fallback

Choosing the model

For each gateway request the model is, in order:

1. the model in the request, when it names a catalog model;
2. the key's (or project's) default_model, when the request has no model or sends "default";
3. a routed choice, when the request sends "auto".

The chosen model must be in the key's allowed_models (an empty list allows every catalog model); otherwise the request fails with 403 forbidden.

Auto routing

Send "model": "auto" and the gateway picks among the routing candidates. Set them in the project's settings (Settings → Gateway in the console) or a key's rules:

{
  "routing": {
    "mode": "auto",
    "objective": "cost",
    "candidates": ["gpt-4o-mini", "claude-3-5-haiku-latest", "gemini-2.5-flash"]
  }
}

• objective is cost (cheapest billed price), latency (fastest recent responses) or balanced.
• candidates: [] means every allowed model.
• With mode: "auto", requests that don't name a model are routed the same way; with mode: "fixed" they use default_model.

GET /v1/models lists auto when routing is possible for the key. The model that answered is in the X-Scalaix-Model header and in the response's model field.

Fallback

fallback_models is an ordered list. When the model returns an error or times out, the gateway tries the next model in the list with the same request, until one answers or the list ends.

{ "fallback_models": ["claude-3-5-haiku-latest", "gemini-2.5-flash"] }

• X-Scalaix-Fallbacks tells you how many models were tried before the answer (0 when the first one worked).
• A provider that keeps failing is skipped for a short while (a circuit breaker), so you don't wait for its timeouts on every request.
• If every model fails you get 502 upstream_error or 504 timeout.
• Usage records the requested model, the model that answered and the number of fallbacks, so the usage report shows how often fallback saved a request.

Endpoints have their own fallback list in their definition; see Collections.