> ## Documentation Index
> Fetch the complete documentation index at: https://to11.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Providers
> Supported LLM providers, the wire format each one speaks, and how model routing works.
The to11 AI Gateway routes requests to LLM providers on your behalf. **to11 operates the gateway for you** — you add providers, configure credentials, and define routing in the dashboard (**project → Gateway**), not by editing any configuration file directly. This page lists the providers the gateway supports, describes how model routing works, and shows which SDK reaches each provider.
For the conceptual model, see [Providers](/docs/deploy/concepts/providers). For the underlying configuration format behind the dashboard, see [Configuration](/docs/reference/configuration).
## Supported providers
The gateway supports the following providers. The **canonical id** is the identifier the gateway uses internally for each provider; you will see it in the dashboard and in routing prefixes. The **format** indicates whether the gateway reads the provider's responses and errors through a dedicated native adapter, or treats it as an OpenAI-compatible endpoint. Either way, the gateway forwards your request body as written; see [Which SDK to call a provider with](#which-sdk-to-call-a-provider-with). All providers support streaming.
Some providers are in beta; their error handling and parameter coverage may be less complete than providers with native adapters.
### Native-format adapters
These providers have a dedicated adapter for their own response and error shapes.
| Provider | Canonical id | Format |
| - | - | - |
| OpenAI | `openai` | Native |
| Anthropic | `anthropic` | Native |
| Google Gemini | `google-gemini` | Native or OpenAI-compatible |
| Google Vertex AI | `gcp-vertex-ai` | Native or OpenAI-compatible |
| AWS Bedrock | `aws-bedrock` | Native |
Google Gemini and Google Vertex AI each serve two wire formats: their native one and an OpenAI-compatible one. The gateway uses the OpenAI-compatible one unless the provider is set to native mode; see [Which SDK to call a provider with](#which-sdk-to-call-a-provider-with).
**Cohere chat uses Cohere's Compatibility API.** Chat requests are sent to Cohere's OpenAI-compatible surface, so you call Cohere models with an OpenAI-shaped request and the gateway forwards it unchanged. Cohere-only chat features are not available through that surface — `connectors`, `documents`, `citation_options`, and the citations they return. Embeddings and rerank are unaffected and still use Cohere's own `/v2/embed` and `/v2/rerank`.
### OpenAI-compatible providers
These providers speak the OpenAI `/chat/completions` format.
| Provider | Canonical id | Format |
| - | - | - |
| Cohere | `cohere` | OpenAI-compatible (chat) |
| xAI | `xai` | OpenAI-compatible |
| Mistral | `mistral` | OpenAI-compatible |
| Groq | `groq` | OpenAI-compatible |
| Together AI | `together` | OpenAI-compatible |
| Fireworks AI | `fireworks` | OpenAI-compatible |
| DeepSeek | `deepseek` | OpenAI-compatible |
| Perplexity | `perplexity` | OpenAI-compatible |
| Azure OpenAI | `azure-openai` | OpenAI-compatible |
| OpenRouter | `openrouter` | OpenAI-compatible |
| Cerebras | `cerebras` | OpenAI-compatible |
| SambaNova | `sambanova` | OpenAI-compatible |
| AI21 | `ai21` | OpenAI-compatible |
| Databricks | `databricks` | OpenAI-compatible |
| HuggingFace | `huggingface` | OpenAI-compatible |
| DeepInfra | `deepinfra` | OpenAI-compatible |
| NVIDIA NIM | `nvidia-nim` | OpenAI-compatible |
| Cloudflare Workers AI | `cloudflare-ai` | OpenAI-compatible |
| Novita AI | `novita-ai` | OpenAI-compatible |
| Nebius | `nebius` | OpenAI-compatible |
| Moonshot | `moonshot` | OpenAI-compatible |
| Ollama | `ollama` | OpenAI-compatible |
| vLLM | `vllm` | OpenAI-compatible |
## Configuring a provider
You add and configure providers in the dashboard under **project → Gateway**. For each provider you select the provider, supply credentials, give it a routing identifier, and choose its access policy: **Allow by default** (every model except blocked patterns, the default) or **Deny by default** (only the models you allow). See [Providers](/docs/deploy/concepts/providers) for the conceptual walkthrough and [Models](/docs/deploy/concepts/models#access-policy) for access policies.
Behind the dashboard, each provider is represented as a named entry in the gateway's configuration. The shape of a single provider entry is:
```toml theme={null}
[providers.openai]
base_url = "https://api.openai.com/v1"
models = ["..."]
credential = "env::OPENAI_API_KEY"
timeout_ms = 300000
```
You do not edit this format directly in the hosted product; the dashboard manages it for you. See [Configuration](/docs/reference/configuration) for the full configuration reference.
## Model routing
The gateway routes each request based on the `model` field in the request body. The field can take several forms.
| Form | Example | Behavior |
| - | - | - |
| `::model` | `anthropic::claude-sonnet-4-6` | Sends the request to the connected provider with that routing identifier |
| Unprefixed model | `claude-sonnet-4-6` | Sends the request to the provider named by the `x-to11-provider-id` header; rejected with `400 Bad Request` without it |
| `function::name` | `function::summarize` | Dispatches to a named function |
| `route::name` | `route::balanced-gpt4o` | Dispatches to a named route |
The prefix separator is `::` (two colons); everything after the first `::` is the model id sent upstream. The examples assume providers whose routing identifiers are `openai` and `anthropic`.
The gateway never infers a provider from an unprefixed model name. Whichever form names the provider, the provider's [access policy](/docs/deploy/concepts/models#access-policy) in the request's environment decides whether the model is served; a model it does not serve returns `404 Not Found`.
The `x-genai-provider` request header names a provider type, not one of your connected providers, and does not choose where a request goes. If it is present alongside a prefix that is also a provider type name, such as `openai::`, and the two disagree, the request is rejected with `400 Bad Request`.
### Pinning a provider by id
`x-to11-provider-id` names one provider by its id, for a caller that cannot put a pin in the model string — a coding agent picks its own model, and rewriting it to route the request would change what the agent asked for.
It is a **fallback**, not an override: it is read only for an unprefixed model. A `::model`, `function::` or `route::` model keeps deciding and the header is ignored. A model is chosen per request, while a header is often fixed for a whole session by the environment a client runs in, so the model is the more specific statement.
When the header names a provider that is not activated in the request's environment, the request is rejected with `404`. The provider id appears in the address of the provider's page in the dashboard.
The [to11 CLI](/docs/reference/cli/commands#to11-code) sends this header, from the provider named in [`code.claude.yaml`](/docs/reference/cli/configuration#codeclaudeyaml).
## Which SDK to call a provider with
The gateway does not translate between wire formats. It forwards your request to the provider as written, and the provider's response comes back in that same format. The endpoint you call sets your request's wire format, and the provider the request resolves to — named in the `model` field or selected as a route's target — must speak that format.
| Endpoint | SDK call | Wire format | Providers that speak it |
| - | - | - | - |
| `/v1/chat/completions` | OpenAI SDK `chat.completions` | `openai.chat` | OpenAI, xAI, and every provider under [OpenAI-compatible providers](#openai-compatible-providers), including Cohere. Google Gemini and Google Vertex AI in OpenAI-compatible mode, which is the default. AWS Bedrock when its base URL is Bedrock's OpenAI-compatible endpoint (a path ending in `/v1` or `/openai/v1`). |
| `/v1/responses` | OpenAI SDK `responses` | `openai.responses` | OpenAI, xAI |
| `/v1/messages` | Anthropic SDK `messages` | `anthropic.messages` | Anthropic. Claude models on Google Vertex AI in native mode. |
Two formats are served on provider-native routes that mirror the provider's own API, not on the endpoints above. The gateway forwards these bodies as written too:
* `bedrock.converse` — AWS Bedrock on its `bedrock-runtime` endpoint, which is a Bedrock provider's default base URL. Call `/v1/bedrock/runtime/model/{modelId}/converse` or `/v1/bedrock/runtime/model/{modelId}/converse-stream`, the paths of the AWS Bedrock Runtime Converse API under the gateway.
* `gemini.generate` — Gemini models on Google Vertex AI in native mode. Call `/v1/vertex/projects/{project}/locations/{region}/publishers/{publisher}/models/{model}:generateContent` or `:streamGenerateContent`, the paths of the Vertex AI REST API under the gateway. Google Gemini in native mode, outside Vertex AI, has no route on the gateway; use it in OpenAI-compatible mode.
A request whose wire format differs from the provider's is rejected with `400 Bad Request` and the `x-to11-error-code: cross_dialect` header before anything is sent to the provider. The message names both formats and the provider. See [Routing errors](/docs/reference/error-handling#routing-errors).
The same applies to [routes](/docs/deploy/concepts/routes): every target of a route must be on a provider that speaks the format your application sends. A request that lands on a target speaking another format is rejected the same way.
### What reaches the provider
The gateway does not rebuild the request. Three edits to the body are its own:
* `model` is replaced by the upstream model id: your model without its `::` prefix, or the model of the route target that serves the request.
* Keys that the gateway reads for its own telemetry, and that no provider defines, are removed.
* On a streaming `/v1/chat/completions` request, `stream_options.include_usage` is set to `true` so the provider reports token usage, for every provider except Cohere. The stream then includes a usage chunk whose `choices` array is empty.
A successful response comes back as the provider sent it; a stream may gain the two additions listed in [Streaming](/docs/reference/streaming#sse-passthrough). For error bodies, see [Error handling](/docs/reference/error-handling).
### Extended thinking
Anthropic extended thinking blocks (the `thinking` content type) reach your client as Anthropic sent them, because the response is not rebuilt.
## Credentials
A project's provider credentials can be supplied in two ways:
* **Managed (server-side).** You store the provider credential in the dashboard. to11 holds the credential and attaches it to requests routed to that provider.
* **Caller-supplied (BYOK).** The caller passes the provider credential on the request, and the gateway forwards it to the upstream provider.
These determine how the gateway authenticates to the upstream provider. They are distinct from to11 platform authentication, which uses the `x-to11-authorization` and `x-to11-project-id` headers; see the [API reference](/docs/reference/api).
## Provider limitations
Every provider serves chat on the endpoint that matches its wire format; see [Which SDK to call a provider with](#which-sdk-to-call-a-provider-with), including the provider-native routes for AWS Bedrock and Google Vertex AI. Other endpoints are available only for specific provider families.
| Feature | OpenAI-family | Anthropic |
| - | :-: | :-: |
| Chat | `/v1/chat/completions` | `/v1/messages` |
| Embeddings | Yes | No |
| Image generation | Yes | No |
| Audio transcription | Yes | No |
| Audio speech (TTS) | Yes | No |
| Files API | Yes | No |
| Token counting | No | Yes |
Requests for an unsupported feature return `400 Bad Request`.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.