Skip to main content
To get automatic failover, create a routing rule with the Fallback chain strategy. to11 tries the targets in the order you declare them and moves to the next one when a target fails with a connection error or a 5xx response.

​
Before you start

  • Two or more targets for the same model — different providers, or different stored credentials for the same provider.
  • Each provider connected under Project → AI Gateway → Providers whose access policy allows the model you want. See Connecting a provider.

​
Create a Fallback chain rule

  1. Open Project → AI Gateway → Routing.
  2. Start a new rule with the create-rule flow.
  3. Choose the Fallback chain strategy.
  4. Give the rule a Name, for example gpt-4o-failover. Your application calls it as route::gpt-4o-failover.
  5. Add the targets in priority order — the first is the primary, the rest are tried in turn. Each target names a Provider, a Model, and a stored credential.
  6. Save the rule.

​
How fallback works

to11 tries the targets in declared order. If the current target fails (a connection error or an HTTP 5xx), the next target is attempted immediately.

​
Retries within a target

Each target attempt is retried with exponential backoff before to11 moves on to the next target. The default policy is up to 2 retries with a 500ms base backoff. A routing rule can override this default when a model needs different behavior.
Retries handle a transient blip on a single target; the fallback chain handles a target that stays down. Together they give you per-target resilience plus failover across targets.

​
Send a request

Call the route by name. to11 handles the failover transparently:
When the primary target is down, the response comes from the next target in the chain. To confirm which target served the request — and to see the failover attempts — open Projects → Traces in Observe. Each attempt is recorded with its target and attempt number, so you can reconstruct the full chain.

​
Next steps

Weighted split

Split traffic across targets by weight.

Direct routing

Route a model to a single managed target.

Routing overview

The three-layer model and how a request picks its layer.