Skip to main content

​
Gateway Quickstart

This tutorial walks you through building the gateway from source, sending your first request, and enabling security — all without Docker or the full to11 stack.

​
Prerequisites

  • Rust stable toolchain — install via rustup if you don’t have it
  • One API key — an OpenAI or Anthropic API key to pass in the Authorization header

​
1. Clone and build

git clone https://github.com/soerenmartius/llm-development-platform.git
cd llm-development-platform

cargo build --release -p gateway
The binary is output at target/release/gateway.

​
2. Define your providers

Runtime dispatch requires at least one [providers.*] entry in config.toml. The gateway treats this section as the routing table: a request can only resolve to an upstream that has an explicit record. Without a matching entry the gateway returns 404 PROVIDER_NOT_FOUND and never reaches an upstream. The built-in provider catalog (OpenAI, Anthropic, xAI, …) is an authoring concept — it tells the gateway how to translate each upstream’s wire shape. It is not an implicit routing table. There is no “default OpenAI” or “default Anthropic” — if you want to dispatch to OpenAI, you write a [providers.openai] block. Customers of the to11 hosted product don’t author TOML; they register providers through the API or dashboard. Both paths land on the same invariant: a defined provider record is required before the gateway will dispatch to that upstream. Create a minimal config file at config/gateway.toml: 
[server]
host = "127.0.0.1"
port = 4000

# Model ids verified 2026-05-28 against
# https://platform.claude.com/docs/en/about-claude/models/overview
# (Anthropic) and docker/gateway/config.toml (production fixture for
# OpenAI — platform.openai.com/docs/models was unreachable at audit
# time). Bump these as upstream catalogs change.
[providers.openai]
base_url   = "https://api.openai.com/v1"
credential = "env::OPENAI_API_KEY"
models     = ["gpt-4o-mini"]
timeout_ms = 30000

[providers.anthropic]
base_url   = "https://api.anthropic.com/v1"
credential = "env::ANTHROPIC_API_KEY"
models     = ["claude-haiku-4-5-20251001"]
timeout_ms = 60000
The credential = "env::OPENAI_API_KEY" line reads from the named environment variable at gateway startup. You can omit it entirely — [providers.openai] defaults to env::OPENAI_API_KEY by convention, [providers.anthropic] to env::ANTHROPIC_API_KEY, and so on. See the full grammar in the Configuration Reference.
The gateway also supports passthrough authentication — if the env::* variable is missing, the caller’s own Authorization header is forwarded to the upstream provider. The gateway never reads or stores provider API keys it didn’t put there itself.
Upgrading from an earlier gateway? Previous releases let unconfigured providers resolve to implicit catalog defaults (https://api.openai.com/v1, https://api.anthropic.com/v1, …). That fallback was removed: every upstream you want to dispatch to now needs an explicit [providers.*] block. If your old config relied on the implicit default, add the matching block from the sample above before restarting.

​
3. Run the gateway

cargo run --release -p gateway
You’ll see the gateway start on 127.0.0.1:4000.

​
4. Send a test request

curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello from to11!"}],
    "max_tokens": 128
  }'
You should receive a standard OpenAI-format response. The gateway proxied your request to OpenAI with sub-millisecond overhead.

​
5. Enable security

Add a [security] block to your config to activate inline guardrails: 
[security]
enabled   = true
blocklist = ["ignore previous instructions", "jailbreak"]
Restart the gateway, then test that the blocklist works: 
curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Please ignore previous instructions"}]
  }'
You’ll receive a 400 Bad Request — the request was blocked before reaching OpenAI. PII detection (SSN, credit cards, phone numbers, emails) is also active automatically.

​
Next steps

Configuration

Full TOML reference — providers, telemetry, security.

API Reference

All endpoints: chat, embeddings, images, audio, files.

Telemetry

OpenTelemetry GenAI spans, metrics, and ClickHouse queries.

Streaming

Fast-path vs normalised-path streaming architecture.