Skip to main content
The to11 gateway is a managed LLM proxy. It sits between your application and your model providers with minimal added overhead, routing requests, applying inline security, and emitting OpenTelemetry traces. You point your SDK at it — there’s nothing to deploy or run yourself. You connect providers and define routing in the dashboard’s AI Gateway section.

​
Architecture

​
Routing

Each request names where it goes in its model field: a provider, as <provider-slug>::<model>, or a named route, as route::<name>. A bare model name needs the x-to11-provider-id header to say which provider; without it the request is rejected. Functions are managed by to11, so the two modes you set up yourself are passthrough and named routes. You opt into managed routing as you need it. See the Routing overview for the full resolution flow, and Concepts for providers, models, targets, and routes.

​
Key capabilities

​
One wire format, end to end

The gateway accepts requests in the OpenAI or Anthropic wire format and forwards each one to its provider as written; the provider’s response comes back the same way. It does not translate between formats, so you call each provider with the SDK whose format it speaks — the Anthropic SDK for Anthropic, the OpenAI SDK for OpenAI and the OpenAI-compatible providers. A request in a format the provider does not speak is rejected with 400 before it reaches the provider. See Which SDK to call a provider with. Forwarding the request as written is what lets provider features the gateway has no model of — prompt-cache breakpoints, server tools, extended thinking — reach the provider intact.

​
Streaming

Streamed events are forwarded as they arrive, without being re-serialized, and telemetry is read from the stream as it passes. See Streaming.

​
Inline security

A security pipeline runs in the request path before anything is forwarded upstream: blocklist matching and PII detection today, with ML-based prompt-injection and content moderation coming. Blocked requests are rejected and never reach the provider.

​
Telemetry

Every call produces an OpenTelemetry GenAI span — model, tokens, latency, finish reason, time-to-first-token for streams, and optional prompt and completion content. See Instrument and Observe.

​
Next steps

Routing

Passthrough, direct, weighted, fallback, and A/B experiments.

Concepts

Providers, models, targets, and routes.

Environments

Separate production from staging.

Observe

See what each request did in production.