Skip to main content

Documentation Index

Fetch the complete documentation index at: https://to11.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

​
Functions

A function is a named entry in [functions.*] that maps a task name to a routing strategy with targets or models. Functions sit at the top of the gateway’s routing hierarchy (L3), providing a layer of indirection between what your application wants to do and which model fulfils that intent.

​
The problem functions solve

Without functions, the model field in a request does double duty: it describes both what the caller wants (“generate a summary”) and how the gateway routes (“send to OpenAI’s gpt-4o”). This coupling creates two problems:
  1. Implicit model swaps. When a route maps gpt-4o to a fallback target running Claude, the caller asked for one model but received another. The indirection is hidden.
  2. Application-config coupling. Changing which model serves a task requires updating application code, not just gateway configuration.
Functions make the indirection explicit. Your application sends model: "function::summarise" and the gateway routes to whichever model the operator has configured for that task. Swapping from gpt-4o to claude-sonnet-4-6 is a config change, not a code change.

​
How callers invoke a function

Prefix the function name with function:: in the model field: 
{
  "model": "function::summarise",
  "messages": [
    { "role": "user", "content": "Summarise this document..." }
  ]
}
The gateway strips the prefix, looks up the function by name, and routes according to its strategy and targets.

​
Three configuration patterns

Functions support three mutually exclusive ways to specify where requests should go. A function must use exactly one of models, targets, or steps.

​
Inline models shorthand

List model names directly. The gateway creates ephemeral targets internally, inheriting credentials from the corresponding providers: 
[functions.summarise]
endpoint = "chat"
strategy = "fallback"
models   = ["gpt-4o", "claude-sonnet-4-6"]
This is the simplest pattern — useful when you do not need custom weights, per-target timeouts, or separate credentials.

​
Named targets reference

Reference pre-defined targets by name, giving you full control over weights, credentials, and timeouts: 
[targets.openai-primary]
model      = "gpt-4o"
credential = "env::OPENAI_API_KEY_PRIMARY"
weight     = 80

[targets.openai-secondary]
model      = "gpt-4o"
credential = "env::OPENAI_API_KEY_SECONDARY"
weight     = 20

[functions.summarise]
endpoint = "chat"
strategy = "weighted"
targets  = ["openai-primary", "openai-secondary"]

​
Multi-step fallback

For complex failover scenarios, functions support steps — identical to route steps. Each step has its own strategy and targets: 
[functions.extract]
endpoint = "chat"
strategy = "fallback"

[[functions.extract.steps]]
strategy = "weighted"
targets = ["openai-primary", "openai-secondary"]

[[functions.extract.steps]]
strategy = "single"
targets = ["azure-fallback"]
Step 1 distributes across two OpenAI keys. If both fail, step 2 falls back to Azure.
A function must use exactly one of models, targets, or steps. The gateway validates this at configuration load time and rejects configs that specify multiple fields.

​
Per-function retry

Each function can declare its own retry configuration, overriding the global [routing.retry] setting: 
[functions.classify]
endpoint = "chat"
strategy = "fallback"
models   = ["gpt-4o", "claude-sonnet-4-6"]

[functions.classify.retry]
max_retries     = 4
backoff_base_ms = 200
When retry is absent from a function, the global [routing.retry] configuration applies. When that is also absent, the defaults are max_retries = 2 and backoff_base_ms = 500.

​
Endpoint type

Every function declares which endpoint it serves. This is required — the gateway rejects functions without it. 
[functions.embed]
endpoint = "embeddings"   # Required: chat | embeddings | audio_speech | audio_transcription | image_generation
strategy = "single"
models = ["text-embedding-3-small"]

​
Routing strategies

Functions support four strategies: single, weighted, fallback, and experiment. The first three work the same as routes. The experiment strategy enables A/B testing with weighted variants — see Experiment Routing.

​
No fallthrough

If a function matches the request but all of its targets fail (after retries), the gateway returns an error to the caller. It does not fall through to L2 routes or L1 passthrough. This is intentional — functions represent explicit operator intent, and silently falling back to an unrelated routing layer would undermine that.

​
How functions fit into the routing hierarchy

Functions are the highest-priority routing layer: 
Request (model = "function::summarise")
  |
  v
L3: Function match?  --> [functions.summarise]
  |  no
  v
L2: Route match?     --> [routes.*]
  |  no
  v
L1: Provider lookup  --> [providers.*]
  |  no
  v
404 Unknown Model
A request with the function:: prefix is always resolved at L3. An unprefixed request is resolved top-down: L3 (function match) → L2 (route match) → L1 (provider match). If the name matches a function, it is dispatched there — the explicit prefix simply bypasses the top-down scan.

​
Next steps

Routes

How routes map model names to strategies across targets.

Routing Overview

Full routing system design: managed vs passthrough, resolution flow.

Configuration Reference

Full TOML configuration reference.