Documentation Index

Fetch the complete documentation index at: https://to11.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

​

Span Attributes

Every GenAI span emitted by the gateway follows the OpenTelemetry GenAI semantic conventions. The span name follows the pattern {operation} {model} (e.g. chat gpt-4o, embeddings text-embedding-3-small). This page is the complete catalogue of every span attribute the gateway may emit. Attributes are grouped by category, mirroring the product architecture: core GenAI semantics, routing, experimentation, client context, and caching. For configuration of the telemetry pipeline itself, see Telemetry Overview.

​

Conventions used in this reference

• Type column uses OTel attribute types: string, int, float, bool.
• string (JSON) denotes a string attribute whose value is a JSON-encoded structure.
• string (JSON array) denotes a string attribute whose value is a JSON array.
• “When set” means the attribute is omitted entirely when the source value is absent; it is never recorded as empty or zero.
• Attributes in the gen_ai.* namespace follow the OTel GenAI semantic conventions. Attributes in the gateway.* namespace are gateway-specific extensions.

​

Endpoint instrumentation matrix

Not every endpoint produces GenAI telemetry. The table below shows which endpoints emit GenAI spans and metrics versus plain HTTP spans only. Endpoints marked “No” under GenAI Span still produce standard HTTP spans via the application telemetry exporter. Media endpoints (images/generations, audio/speech, audio/transcriptions) do not report token usage metrics. Only gen_ai.client.operation.duration is recorded for those endpoints.

​

Always present

These four attributes appear on every GenAI span regardless of endpoint, provider, or request shape. They form the minimum set required for basic filtering and grouping in any trace backend.

​

Request parameters (when set)

These attributes are recorded only when the corresponding parameter is present in the request body. Absent parameters produce no attribute rather than a zero or null value.

​

Response attributes

Populated after the upstream provider returns a response (or after the final streaming chunk). For streaming requests, token counts and finish reasons are extracted from the terminal usage chunk when the provider includes one, or accumulated from individual deltas otherwise. The gen_ai.response.model attribute is particularly useful when the request specifies an alias (e.g. gpt-4o) and the provider resolves it to a specific snapshot (e.g. gpt-4o-2024-08-06).

​

Advanced token types

Provider-specific token breakdowns beyond standard input/output counts. These attributes supplement gen_ai.usage.input_tokens and gen_ai.usage.output_tokens with finer-grained detail where the upstream provider supports it. When Anthropic cache tokens are present, the relationship between token counts is:

• gen_ai.usage.input_tokens = total input tokens (includes both cached and non-cached)
• gen_ai.usage.cache_read.input_tokens = subset of input tokens served from cache
• gen_ai.usage.cache_creation.input_tokens = tokens newly written to cache (aggregate)
• gen_ai.usage.cache_creation_5m.input_tokens = 5-minute TTL portion of cache creation (when Anthropic sends TTL breakdown)
• gen_ai.usage.cache_creation_1h.input_tokens = 1-hour TTL portion of cache creation (when Anthropic sends TTL breakdown)

When both the aggregate and TTL-specific breakdown are present, cache_creation.input_tokens equals the sum of cache_creation_5m.input_tokens + cache_creation_1h.input_tokens. The gateway passes all fields through as-is from the provider. For OpenAI reasoning models, gen_ai.usage.reasoning_tokens is separate from gen_ai.usage.output_tokens. Reasoning tokens represent internal chain-of-thought computation and are billed separately by the provider.

​

Streaming-only attributes

Present exclusively on spans where gen_ai.request.stream is true. These attributes provide per-request performance observability for streaming workloads. For non-streaming requests, none of these attributes are present. The overall request duration is captured by the span’s own start and end timestamps.

​

Output type

Indicates the modality of the response produced by the operation. The value is determined by the endpoint, not by the response content. The mapping from endpoint to output type is fixed: Audio transcription endpoints produce text as their output type, since the response is transcribed text rather than audio.

​

Server address

Identifies the upstream provider endpoint the gateway connected to. These follow the OTel server.* namespace conventions.

​

Error attributes

Present when the upstream call fails or times out. A span with error attributes always has its OTel status set accordingly. The span StatusCode is set to ERROR and StatusMessage contains the error detail string. On successful requests, StatusCode remains UNSET and no error.type attribute is recorded. Error type values: Provider-returned error bodies are not captured as span attributes. The StatusMessage contains a summary suitable for filtering and alerting.

​

Session and user metadata

Populated from request headers or body fields. All attributes in this group are optional and appear only when the corresponding header or field is present. These attributes enable grouping spans by user, session, or conversation for analytics and debugging. When both the x-to11-user-id header and the user body field are present, the header takes precedence. The gen_ai.user.id attribute enables per-user analytics queries in ClickHouse without requiring application-level joins.

​

Tool call child spans

When a model response includes tool calls, the gateway emits child spans nested under the GenAI operation span. Each tool call produces one child span. The parent-child relationship is visible in any trace viewer that supports nested spans. Span name: tool_call.{function_name} (e.g. tool_call.get_weather) The gen_ai.tool.definitions attribute (string, JSON array) appears on the parent span, not on tool call child spans. It contains the tool/function definitions sent in the request when capture_content is enabled. Multiple tool calls within a single response each produce their own child span. The child spans share the same parent span ID as the GenAI operation span. If a response contains no tool calls, no child spans are created.

​

Timing behaviour

In the non-streaming case, the gateway cannot observe when individual tool calls were generated. The synthetic timestamp distribution ensures that tool call spans are evenly spaced within the parent span’s duration, providing a reasonable approximation for visualisation in trace viewers.

​

Gateway routing attributes

Present on spans that traverse the gateway’s managed routing layer. These attributes use the gateway.* namespace, which is specific to the to11 gateway and not part of the OTel semantic conventions. Passthrough requests carry gateway.routing.path = "passthrough" and omit the remaining routing attributes. For fallback strategies, the gateway.routing.attempt attribute increments with each retry. The first attempt is 1. Combined with gateway.routing.target, this allows reconstruction of the full fallback chain from span data.

​

Experiment and variant attributes

Present when the request is routed through a function with multiple variants. These attributes record which variant was selected and the weight distribution at the time of the request. For functions with a single variant, the variant attributes are still emitted but gateway.experiment.total_variants is 1. Variant selection is determined at request time based on the configured weights. The gateway.variant.weight attribute records the weight of the selected variant, not the probability. To compute the selection probability, divide by the sum of all variant weights (derivable from the total variants and their individual weights across spans).

​

Client-context operation attributes

These attributes appear on sibling spans created via x-to11-context-* headers. Each header value generates a separate span adjacent to the main GenAI operation span, sharing the same parent trace context. The span name matches the operation type (e.g. execute_tool, retrieval, invoke_agent, create_agent).

​

execute_tool spans

​

retrieval spans

​

invoke_agent / create_agent spans

The invoke_agent and create_agent span types differ in their attribute sets. gen_ai.agent.step tracks the execution step within an agent invocation, while gen_ai.agent.description records the agent definition at creation time. Both share gen_ai.agent.id and gen_ai.agent.name.

​

Content capture attributes

Recorded when content capture is enabled via the capture_content configuration flag or the GATEWAY_GENAI_CAPTURE_CONTENT=true environment variable. These attributes contain the full prompt and completion text as serialised JSON. Streaming responses are accumulated up to a 64 KB limit (MAX_ACCUMULATED_CONTENT_BYTES); non-streaming responses are captured in full. For media endpoints, content capture behaviour varies:

​

Cache status attributes

Present when the gateway’s response cache is active for the request. The skip status indicates that caching was enabled but the request was not eligible (e.g. streaming requests, or requests with temperature > 0). On a cache hit, the gateway skips the upstream provider call entirely. The span still records gen_ai.response.model and token usage attributes from the cached response. The gen_ai.cache.age_seconds attribute indicates staleness, allowing cache freshness analysis across requests. Cache status values: