OpenTelemetry and the AI Gateway

The AI Gateway is in development

The AI Gateway is in development as a way to connect a project, so you can't choose it on the platform yet; OpenTelemetry is ready to use. The gateway's endpoints stay on: projects already connected this way keep working, and a local test runner on a machine without an OpenAI key uses them as before.

Two endpoints send traces to a project's Logs without the SDK: an OTLP receiver for apps instrumented with OpenTelemetry, and the AI Gateway, which your OpenAI or Anthropic SDK calls instead of the provider. Both take a project token with the upload scope (pj_…, made under Automations → API tokens with "Allow uploading logs and experiments" ticked), and both write to that token's project.

OpenTelemetry (OTLP)

POST https://<platform address>/otel/v1/traces

Protocol	OTLP over HTTP: application/x-protobuf (or application/protobuf), or application/json
Compression	gzip or none
Auth	Authorization: Bearer pj_… or x-api-key: pj_…
Size	Up to 8 MB per request, before and after gunzip (413 otherwise: send fewer spans per request)
Answer	200 with {} when every span was taken, or {"partialSuccess": {"rejectedSpans": "N", "errorMessage": "…"}}: a span that can't be read is refused on its own, never the whole request. Errors are a google.rpc.Status (401 no token, 403 no upload scope, 415 another content type)

The usual exporter variables:

OTEL_EXPORTER_OTLP_ENDPOINT=https://<platform address>/otel
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer%20pj_your_token_here
OTEL_RESOURCE_ATTRIBUTES=deployment.environment.name=prod,service.name=support-agent

What it reads. Each span is read by the first convention it carries: the Vercel AI SDK (ai.*), OpenInference (openinference.span.kind, input.value, llm.*), OpenLLMetry / Traceloop (traceloop.*, gen_ai.prompt.N.*), then the OpenTelemetry GenAI semantic conventions (gen_ai.operation.name, gen_ai.request.model, gen_ai.response.model, gen_ai.usage.*, gen_ai.input.messages, gen_ai.output.messages, tool calls and their events). A span with none of them is kept as kind other with its attributes. Per span: up to 256 attributes, strings of 32 KB, 32 events.

• Kind and cost: model and embedding spans get their model, tokens and a cost worked out from the model's list price; a span whose model has no known price is counted as unpriced in the trace's metadata.
• Environment: from the resource's deployment.environment.name (or deployment.environment): dev/development/local → dev, staging/stage → staging, prod/production → prod, ci → ci; other values are dropped. Without it, the header x-preworld-environment (set it in OTEL_EXPORTER_OTLP_HEADERS).
• Version: service.version shows as the trace's version.
• Linking to an experiment: span attributes preworld.experiment_id and preworld.row_key (the SDK sets them inside an Eval). An experiment of another project is ignored.

There is no request rate limit on OTLP today. What you send is metered as "Traces uploaded" and isn't charged for now.

The AI Gateway

Endpoint	Shape
POST /gw/openai/v1/chat/completions	OpenAI chat completions, streaming included
POST /gw/openai/v1/responses	OpenAI responses
POST /gw/openai/v1/embeddings	OpenAI embeddings
GET /gw/openai/v1/models	The platform's models (an OpenAI list)
POST /gw/anthropic/v1/messages	Anthropic messages, streaming included

Set your SDK's base URL to https://<platform address>/gw/openai/v1 (OpenAI) or https://<platform address>/gw/anthropic (Anthropic), and its API key to the project token: Authorization: Bearer pj_… or x-api-key: pj_… both work.

Platform models or your own key

	Platform models	Your own provider key
How	Ask for a model the platform offers (GET /gw/openai/v1/models): Gemini 4 Argon, GPT-6.1 Sol, Gemini 3.8 Flash, GPT-6 Luna; embeddings on gemini-embedding-001 where enabled. Name it as the list does (gemini-3.8-flash) or with the platform's prefix (vertex:gemini-3.8-flash, azure:gpt-6.1-sol, as the project's settings write it): both work	Add the header x-provider-key: <your key>
Who answers	The platform's model, through its own Azure or Google Cloud account	The provider: Anthropic for the Anthropic shape; for the OpenAI shape, Google for gemini… / google/… models, Anthropic for claude… models, OpenAI otherwise
What you pay	The tokens at the model's list price plus 30%, on the organization's bill as "Platform model (AI under test)"	Nothing to us for the call; the provider bills you. The trace still shows the call's cost at list price
Credit	Checked before each call: 402 when the credit is used up, 403 org_suspended	Not needed

The provider key is passed on for that call only, and never stored, logged or traced. An Anthropic-shape request on a platform model is translated; features the platform's models don't have (server tools, document blocks) answer 400 unsupported_on_platform_model.

Headers

Header	Direction	Meaning
x-provider-key	Request	Your provider key (8 to 512 printable characters)
x-preworld-environment	Request	dev, staging, prod or ci, for the trace
anthropic-version, anthropic-beta, openai-organization, openai-project	Request	Passed to the provider
x-preworld-trace-id	Answer	The trace this call became
retry-after, x-ratelimit-*, anthropic-ratelimit-*, x-request-id	Answer	Passed back from the provider

What it records

Each call is one trace (source "Gateway") with one span: chat <model> or embeddings <model>, its tokens (cached ones too), cost, the request body as input and the reply as output, and gen_ai.* attributes. A stream is recorded whole once it ends. A caller that closes it once it has the stream's last word ([DONE], a finish reason, message_stop), as OpenAI's Python SDK does, has the whole reply: that call isn't broken off. One the caller closes before its last word is "Cancelled by caller" (status cancelled, preworld.gateway.broken_off_reason caller_closed): not an error. One that really broke off is marked as an error with why (preworld.gateway.broken_off_reason): the model sent nothing for 120 seconds, the model's answer broke, or the stream ended before its last word. A failed call says why on its span, in error.type and otel.status_description: the provider's error message, or why the stream broke off. The Logs list shows a call's last user message as its "Input" and the reply's text as its "Output"; the whole request and reply are in the trace's panel. Online scoring rules score these traces like any other, except one cancelled by its caller.

Limits and errors

• 600 calls a minute per organization by default (429 rate_limited, with Retry-After).
• Request body up to 4 MB; max_tokens (or max_completion_tokens, max_output_tokens) up to 32,000.
• 120 seconds without a byte from the model: 504 upstream_timeout.
• Errors come in the shape of the endpoint: OpenAI's {"error": {"message", "type", "code"}} or Anthropic's {"type": "error", "error": {"type", "message"}}. Codes include unauthorized, invalid_api_key, token_scope_missing, model_not_found (not a platform model: send your key in x-provider-key), platform_models_off, max_tokens_too_large, invalid_provider_key, upstream_unreachable. A provider's own error comes back with its status in the endpoint's shape, with the provider's own message (Vertex's [{"error": …}] list, for one, becomes OpenAI's {"error": {"message", "type", "code": "invalid_argument"}}); one already in that shape is passed back as it came.

Billing

See Pricing: platform-model calls are billed by usage at list price plus 30%; calls with your own key and OTLP traces aren't charged; trace volume is metered and shown on the usage page.