Skip to main content
Every request that reaches the to11 gateway is resolved through a three-layer routing model. Each layer is independently useful, and you opt into the higher ones as your project needs them. You configure all of this in the dashboard under Project → AI Gateway — there is nothing to install and no file to edit.

​
Three-layer routing model

L1 is the default. Any model a connected provider serves can be called straight away as <provider-slug>::<model> — see Naming the provider in a request. L2 adds a route — a name mapped to a strategy (Direct routing, Weighted split, Fallback chain, or A/B experiment) and a set of targets, each carrying a stored credential — so callers no longer need to supply a provider key. L3 is task-level dispatch: a function resolves a task name to a route or model. Functions are managed by to11; the way you set up task-level routing yourself is a named route, called as route::<name> — see Route by a stable name.

​
Resolution flow

The gateway reads the model field and decides from its prefix alone. It never searches your providers or routes for a bare model name.
Once the provider is known — from the prefix, the header, or the target a route selected — the gateway checks the provider’s access policy in the request’s environment. A model it does not serve returns 404; a route skips such a target and returns 404 only when no target is left.

​
Namespace prefixes

A prefix goes in the model field of the request body:
The provider slug is the routing identifier you gave the provider when you connected it; openai here assumes an OpenAI provider with that identifier. A bare name like summarize never reaches a route — call it as route::summarize.

​
Routing strategies

A route selects among its targets using one of these strategies. In the dashboard’s Routing create-rule flow you pick a strategy and fill in its fields. You also choose which stored credential each target uses when you add it.

​
Failover and retry

The fallback strategy tries targets in declared order: if a target fails with a connection error or a 5xx response, the next target is attempted immediately. Within a single attempt, the gateway retries with exponential backoff — by default up to 2 retries with a 500ms base backoff. A route or function can override this default.

​
Scope and limitations

  • No model remapping by default. The model after the <provider-slug>:: prefix is forwarded to the upstream provider as sent, unless a route’s target names a different upstream model.
  • Weighted routing is not health-aware. A weighted target keeps receiving its proportional share of traffic even when it is erroring. Use a Fallback chain when you need automatic failover.
To confirm which provider, route, and target actually served a request, open Projects → Traces in Observe — each request records its routing path, strategy, and resolved target.

​
Next steps

Passthrough

How L1 passthrough and BYOK requests resolve.

Direct routing

Route a model to a single managed target.

Weighted split

Split traffic across targets by weight.

Fallback chain

Fail over to the next target on error.