Skip to main content
The gateway instruments LLM calls automatically. For custom spans — non-LLM operations, your own pipeline steps, or enriching traces from your own services — you can send OpenTelemetry data directly to to11. Anything the gateway never sees, you can still put on the record.

​
When to use direct ingestion

  • Custom spans for non-LLM operations — data preprocessing, post-processing, or any application logic you want visible alongside LLM calls.
  • Enriching gateway traces with application-side timing — capture your full RAG pipeline end to end, including database queries and retrieval steps.
  • Agent lifecycle spans — operations like create_agent or session management that are better emitted from your client than from the gateway.

​
Prerequisites

  • A to11 API key with otel:write scope.
  • An OpenTelemetry SDK in your language of choice.

​
Exchange an API key for a collector token

Direct ingestion authenticates with a short-lived token. Exchange your API key for one before configuring your SDK: 
curl -X POST https://api.to11.ai/v1/ingest-token/exchange \
  -H "Content-Type: application/json" \
  -d '{
    "credential": "your-api-key"
  }'
Response: 
{
  "token": "eyJ...",
  "projectId": "proj_abc123",
  "scopeClass": "project",
  "scopes": ["otel:write"],
  "expiresAt": "2026-03-25T17:30:00Z"
}
The returned projectId is the project the token is scoped to. to11 binds every ingested span to this project, so you do not pass it explicitly when sending spans. If your API key is org-scoped rather than project-scoped, include requestedProjectId in the request body to select the target project. The token is rejected if the key is not authorized for that project.
Tokens expire after 15 minutes. Refresh by calling the exchange endpoint again before expiry.

​
Configure your OTel SDK

to11 accepts OTLP on two public endpoints:
ProtocolEndpoint
gRPCcollector.to11.ai:443
HTTPhttps://collector.to11.ai/v1/traces
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource

resource = Resource.create({
    "service.name": "my-rag-pipeline",
})

exporter = OTLPSpanExporter(
    endpoint="collector.to11.ai:443",
    headers={"authorization": f"Bearer {token}"},
    # TLS is on by default
)

provider = TracerProvider(resource=resource)
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)
You do not set project.id yourself. to11 binds the spans to your project automatically — the project the token was issued for — and overwrites any value you send.

​
Send spans

tracer = trace.get_tracer("my-rag-pipeline")

with tracer.start_as_current_span("document-preprocessing") as span:
    span.set_attribute("pipeline.stage", "preprocessing")
    span.set_attribute("document.count", 42)
    # ... your processing code ...

​
Correlate with gateway traces

To place your custom spans in the same trace as gateway LLM calls, propagate the W3C traceparent header. See Distributed Tracing for details. 
from opentelemetry.propagate import inject

headers = {}
inject(headers)  # Adds traceparent to headers dict

# Pass these headers to the gateway
response = requests.post(
    "https://gw.to11.ai/v1/chat/completions",
    headers={
        **headers,
        "x-to11-authorization": f"Bearer {to11_api_key}",
        "Authorization": f"Bearer {provider_api_key}",
        "Content-Type": "application/json",
    },
    json={"model": "gpt-4o", "messages": [...]},
)