Error Handling

The gateway sits between your SDK and the upstream LLM provider. It sends a request only to a provider that speaks your SDK’s wire format, so a provider’s error arrives in a shape close to what your SDK expects. Depending on the route, the gateway forwards the error body as is or rewrites it into your SDK’s error envelope, and it always adds headers that classify the error. This page covers what the gateway guarantees about error responses, when it rewrites an error body, and how to inspect the gateway’s classification headers for richer error handling.

Guarantees at a glance

Surface	What you see
`x-to11-error-code` header	Stable cross-provider classification — same value regardless of upstream or SDK.
`x-to11-upstream-provider` header	Provider slug for the upstream that returned the error.
Response body (a provider other than your SDK’s own, in the same wire format)	Rewritten into your SDK’s error envelope so SDK retry logic triggers.
Response body (same-provider routes)	Forwarded verbatim from the upstream.
Response body (native passthrough surfaces)	Forwarded byte-for-byte.
Response body (5xx errors)	Message redacted to `provider returned status N`.
HTTP status	Verbatim from the upstream.
Wire-format mismatch	`400 Bad Request` with `x-to11-error-code: cross_dialect`; no provider is contacted. See Routing errors.
`x-should-retry` / `retry-after-ms` (retryable gateway `INFRA_UNAVAILABLE` 503s)	Whether a gateway infrastructure failure is worth retrying, and how soon. Fail-closed gateway 503s omit both — see below.

Headers always classify

Every error that originates at a provider — or at the gateway’s auth layer — carries two stable headers, regardless of which provider, which SDK surface, or which routing strategy produced it:

x-to11-error-code: rate_limit
x-to11-upstream-provider: anthropic

The x-to11-error-code taxonomy is the canonical way to classify these errors. Values are stable across all combinations of upstream provider and surface SDK:

`x-to11-error-code`	Meaning
`auth`	Credentials missing, invalid, or revoked (401).
`forbidden`	Authenticated but not permitted (403).
`bad_request`	Malformed request, including context-length and other 4xx conditions.
`quota`	Account-level quota exhausted; retry will not clear.
`rate_limit`	Throttled — retry with backoff will likely succeed.
`overloaded`	Upstream under capacity pressure; retry with backoff.
`content_policy`	Upstream content / safety filter blocked the request or response.
`model_not_found`	Model name did not resolve at the upstream (404).
`org_verification_required`	Gated-model verification flow required.
`upstream`	Generic 5xx upstream condition or network fault (body redacted).
`feature_disabled`	The requested routing surface is disabled for this project (403).
`cross_dialect`	The request’s wire format differs from the provider’s; rejected before any provider is contacted (400).

Build retry, alerting, and metrics dashboards on these headers — not on response body text. The header value is the same whichever provider produced the error and whichever SDK you called through. Apart from feature_disabled and cross_dialect, errors raised by the gateway’s own routing and infrastructure layer — before any provider is contacted — carry no x-to11-error-code, because the taxonomy above classifies upstream conditions. Classify those by HTTP status plus the error.code field in the body (INFRA_UNAVAILABLE, SNAPSHOT_DECRYPT_FAILED, ENVIRONMENT_NOT_FOUND, ROUTING_NOT_CONFIGURED). See Gateway-side failures state their own retry hint.

Bodies from another provider match your SDK

Your SDK reaches a provider other than its own only when that provider speaks the same wire format — the OpenAI SDK calling Groq, Mistral, or Google Gemini through its OpenAI-compatible endpoint, for example. When such a provider returns an error the gateway recognizes, the gateway classifies it and re-emits it in your SDK’s error envelope, so your SDK’s built-in retry policy fires as it would against its own provider. On the OpenAI envelope, upstream overload and rate limiting both render as "type": "rate_limit_error" with "code": "rate_limit_exceeded", because OpenAI has no overload error type; this keeps the OpenAI SDK’s automatic retry working. The x-to11-error-code header still tells them apart (overloaded or rate_limit).

Retry-After is threaded through

When the upstream response carries a Retry-After header (integer-seconds form), the gateway re-emits it on its response. SDKs that honor Retry-After (the official OpenAI and Anthropic SDKs both do) back off accordingly. When the upstream omits the header, the gateway does not synthesize a default — Retry-After is absent on the gateway response too, and the SDK falls back to its own backoff policy.

Gateway-side failures state their own retry hint

The section above covers errors that originate at the provider. A failure inside the gateway’s own infrastructure — a 503 whose body carries "code": "INFRA_UNAVAILABLE" — states its retry hint explicitly instead of leaving your client to infer one from the status code:

HTTP/1.1 503 Service Unavailable
x-should-retry: true
retry-after-ms: 250
Retry-After: 1
Content-Type: application/json

{"error":{"type":"infra_unavailable","code":"INFRA_UNAVAILABLE",
          "message":"Environment registry is temporarily unavailable. Please retry shortly."}}

x-should-retry: true — a positive signal that this request is safe to send again. The official OpenAI and Anthropic SDKs honor it.
retry-after-ms — the same delay at millisecond precision, for clients that read it. These failures typically clear within a round trip, so a whole-second value would overstate how long the condition lasts.
Retry-After: 1 — the standard header, floored to one second, for plain HTTP clients and proxies that only understand seconds.

The request never reached a provider, so retrying costs you nothing but the round trip and no tokens are billed. These responses carry no x-to11-error-code — that header classifies upstream provider failures, and no provider was contacted here. Branch on the status plus x-should-retry, or on error.code in the body. Not every gateway 503 is retryable this way. A rejected routing snapshot ("code": "SNAPSHOT_DECRYPT_FAILED") fails closed and only clears once the project’s configuration is republished — it deliberately omits x-should-retry and the sub-second hint, because retrying cannot resolve it. Read the headers rather than assuming every 503 behaves the same way.

Same-provider routes remain verbatim

When the surface SDK and the upstream provider match (OpenAI SDK → OpenAI upstream, Anthropic SDK → Anthropic upstream), the gateway forwards the upstream error body verbatim. This preserves any provider-specific signal — header fields, param indicators, request IDs — that the SDK might inspect beyond the standard envelope shape. The classification headers (x-to11-error-code, x-to11-upstream-provider) are still attached. The HTTP status is still verbatim.

Native passthrough remains byte-for-byte

Provider-native passthrough routes forward the upstream response byte-for-byte on both the success and error paths, giving you bit-identical compatibility with a provider’s native API.

5xx bodies are redacted

For an upstream 5xx, the response body’s message is rewritten to a conservative provider returned status N form. The upstream’s full message text never reaches the client, even if it would have carried useful debugging signal — that signal is preserved on the trace for the request, which you can inspect in the to11 dashboard (Projects → Traces), not in the response body. A generic upstream 5xx is surfaced to the client as 502 Bad Gateway. The one exception is capacity overload, which is surfaced as 529 (the status Anthropic uses) and classified as overloaded. Either way the x-to11-error-code header reaches the client, so your SDK can classify and retry without depending on body text.

Routing errors

These errors come from the gateway’s routing step, before any provider is contacted. Retrying the same request does not clear them; change the request or the provider’s configuration.

Status	Cause	Response
`400 Bad Request`	The `model` has no `::` prefix and the request carries no `x-to11-provider-id` header, so it names no provider.	Message: invalid request: model `<model>` does not name a provider: send `<provider-slug>::<model>` or set the `x-to11-provider-id` header
`400 Bad Request`	The request’s wire format differs from the format of the provider it resolves to — an OpenAI-format request to an Anthropic provider, for example. The gateway does not translate between formats.	Header `x-to11-error-code: cross_dialect`. The message names both formats and the provider; see the example below.
`404 Not Found`	The `<provider-slug>::` prefix or the `x-to11-provider-id` header names no provider active in the request’s environment.	Body `error.code`: `PROVIDER_NOT_FOUND`, with the slug or id and the environment
`404 Not Found`	The provider does not serve the model in the request’s environment: a blocked pattern matches it under Allow by default, or it is not allowed under Deny by default.	Message names the model and the environment (`no matching function: <model> (env <env>)`), or, when the model is a route’s target, the route and the reason
`410 Gone`	A later turn of a chained Responses API conversation asks for a model that the provider which served the earlier turn no longer serves.	Message names the response id and the reason

See Naming the provider in a request and Access policy.

Wire-format mismatch

The gateway forwards a request to its provider as written, so the provider must speak the request’s wire format. An OpenAI SDK request to /v1/chat/completions that names an Anthropic provider returns:

HTTP/1.1 400 Bad Request
x-to11-error-code: cross_dialect
Content-Type: application/json

{"error":{"message":"request arrived in the `openai.chat` wire format but model `<model>` resolves to provider `anthropic`, which serves the `anthropic.messages` wire format. The gateway does not translate between wire formats. Call this model with an SDK that speaks `anthropic.messages`, or choose a model served by a provider that speaks `openai.chat`."}}

The body has this {"error": {"message": ...}} shape whichever SDK sent the request, so branch on the status and the x-to11-error-code header. The wire formats are openai.chat (/v1/chat/completions), openai.responses (/v1/responses), and anthropic.messages (/v1/messages); which providers speak each is listed in Which SDK to call a provider with.

Examples

Inspecting `x-to11-error-code` for richer classification

When you need to differentiate quota exhaustion (retry will not clear) from rate limiting (retry with backoff clears) or from upstream overload, read the x-to11-error-code header. The body cannot always tell them apart: a body rewritten into the OpenAI envelope renders overload and rate limiting alike as rate_limit_exceeded, and a body forwarded verbatim carries whatever the provider put there.

import httpx

resp = httpx.post(
    "https://gw.to11.ai/v1/chat/completions",
    json={
        "model": "openai::gpt-4o",
        "messages": [{"role": "user", "content": "Hello"}],
    },
    headers={
        "x-to11-authorization": "Bearer <your-to11-api-key>",
        "x-to11-project-id": "<your-project-id>",
    },
)

if resp.status_code >= 400:
    code = resp.headers.get("x-to11-error-code")
    upstream = resp.headers.get("x-to11-upstream-provider")
    if code == "quota":
        # Don't retry — account-level exhaustion.
        raise RuntimeError(f"Quota exhausted on {upstream}")
    elif code in ("rate_limit", "overloaded"):
        # Backoff and retry per Retry-After.
        ...
    elif code == "auth":
        # Credentials are wrong — fail fast.
        ...

curl -i https://gw.to11.ai/v1/chat/completions \
  -H "x-to11-authorization: Bearer $TO11_API_KEY" \
  -H "x-to11-project-id: $TO11_PROJECT_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai::gpt-4o",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

# Example error response:
# HTTP/1.1 429 Too Many Requests
# x-to11-error-code: rate_limit
# x-to11-upstream-provider: openai
# Content-Type: application/json
#
# {"error":{ ...the provider's error, forwarded as is... }}

Unrecognized envelopes

When an upstream returns an error shape the gateway does not recognize (an empty body, malformed JSON, or a new upstream error kind), it falls back to a generic error body rather than one in your SDK’s envelope. The x-to11-error-code header still classifies the error correctly, so retry and alerting logic built on the header keeps working regardless.

Mid-stream SSE error frames

When an upstream returns 200 OK for a streaming request and then emits an error partway through the stream, the gateway delivers that error as an SSE error frame in the shape your SDK expects, and marks the stream aborted so no false success terminator follows:

OpenAI surface — data: {"error":{...}} (the OpenAI SDK parses the top-level error key into the appropriate exception class).
Anthropic surface — event: error followed by data: {"type":"error","error":{...}} (the Anthropic SDK dispatches by SSE event name).

Same-provider and native-passthrough routes forward the upstream’s error frame verbatim; a frame from another provider in the same wire format is rewritten into your SDK’s shape, as on the status-time path. The HTTP status line stays at 200 OK — the error lives in the response body, not on the status line.

API Reference — endpoints, status codes, request/response shapes.
Telemetry — how error classification shows up in spans and metrics.
Streaming — error-handling on the streaming path.

​Guarantees at a glance

​Headers always classify

​Bodies from another provider match your SDK

​Retry-After is threaded through

​Gateway-side failures state their own retry hint

​Same-provider routes remain verbatim

​Native passthrough remains byte-for-byte

​5xx bodies are redacted

​Routing errors

​Wire-format mismatch

​Examples

​Inspecting x-to11-error-code for richer classification

​Unrecognized envelopes

​Mid-stream SSE error frames

​Related