Project secrets

Project secrets hold the values a cloud run needs and review.yaml must never contain, such as your agent's model key, a test account's token or a registry sign-in. This page covers which ones to add, how they are protected and where they end up. An agent written for OpenAI's API can also run with no key of yours (see your agent's model). Values that aren't secret, such as a test account of your repository's own test backend, can go in review.yaml instead (see secrets or runner_env).

Project secrets are for GitHub repository and your own image projects. A local test runner reads the values from its own machine, and a chat AI keeps its key on its own connection.

Add a secret

1. Open the project's Targets and click "Secrets" on the repository's row (or open the way's wizard).
2. In the "Secrets" section of the wizard, type the name (for example OPENAI_API_KEY) and the value.
3. Click "Add".

The list shows each secret's name, its last four characters ("Ends in") and when it was updated. To replace a value, add the same name again. To remove one, click the bin beside it. Only a person signed in in the browser can add or delete secrets; CLI tokens and the assistant can't.

Which names to add

Add a secret for every variable the run reads, unless it isn't secret and review.yaml sets it in runner_env (see secrets or runner_env). You needn't check this table yourself: a sandbox project's "Secrets" section lists the names the platform found in review.yaml and your confirmed tests, who gives each, and which are missing.

What needs it	Where the name is
Your agent (its model key, API settings)	agent.env_passthrough in review.yaml. Without OPENAI_API_KEY, the platform's model answers (see your agent's model)
Your state server	backend.state.reader.base_url_env
A SQLite database for resets or data reads	url_env in backend.reset or backend.state.reader
The test users' tokens	The env:NAME each test names for its user
Install commands that need a value (a private package registry, say)	Whatever environment.setup reads
A judge model the platform doesn't provide	See judge model keys

attach.base_url_env needs no secret: the test runner sets it to the tool proxy for each attempt.

Secrets or runner_env

Not every variable a run reads is a secret. Values that aren't can be written in review.yaml under runner_env, so the repository brings them along and nobody has to add them on the platform.

Put it in	When	For example
Project secrets	The value opens something: a model key, an API token, a database password, an account on a shared test system	OPENAI_API_KEY, PAYMENTS_API_KEY
runner_env in review.yaml	The value is no secret: the address of a service the repository starts itself, a test account of the repository's own test backend	EXPENSE_STATE_BASE: http://127.0.0.1:47591, EXPENSE_TOKEN_ALICE: tok-e_alice

• A project secret with the same name wins, so you can override a repository's value without changing the repository.
• Anyone who can read the repository sees the values in runner_env, and they appear unmasked in logs and reports.
• A value in runner_env that looks like a real credential (a key shape such as sk-… or ghp_…, a JWT, a private key, or a long random string) stops review.yaml from loading, and the error names the variable. Add such a value as a project secret instead.

Naming rules

• Capital letters, digits and underscores only, starting with a letter, at most 64 characters. review.yaml can only name variables written this way.
• A value is 1 to 4,000 characters.
• Use values of at least 8 characters: shorter ones are not masked in logs (below).

How secrets are sealed and delivered

1. The value travels to the platform over TLS and is sealed as it arrives: encrypted with AES-256-GCM under a fresh key, and that key wrapped with RSA-OAEP (SHA-256) under the platform's public key.
2. The API stores only the sealed value and the last four characters. It holds no private key, so nobody can read the value back through it, you included.
3. When a run starts, its sandbox exchanges a one-time bootstrap token for a session token that works for that job only. It shows that token to the platform executor, the one component with the private key, which opens the project's secrets and hands them to that sandbox.
4. In the sandbox, the values are environment variables of the test runner. The agent itself receives only the ones in agent.env_passthrough, plus the platform model's address and placeholder key when there is no OPENAI_API_KEY. Install commands, resets and the judge see the rest too. The session token stays with the test runner, and the agent never receives it.
5. The virtual machine is deleted when the run ends, and the values with it.

See encryption in the security overview.

Redaction in logs and reports

• In the logs, every line a sandbox streams back (from the test runner, your services and your agent) has each secret value of 8 characters or longer replaced with a placeholder before it leaves the virtual machine.
• In the run bundle's traces and reports, the test users' tokens are replaced with [REDACTED:NAME]. So are the values (8 characters or longer) of env_passthrough variables whose names contain KEY, TOKEN, SECRET, PASSWORD, PASSWD, CREDENTIAL or AUTH, and strings shaped like keys (Bearer …, sk-…, ghp_…, Slack tokens, JWTs). Name secret variables so they say what they are, such as PAYMENTS_API_KEY.
• Before a run bundle is uploaded, it is scanned one last time. If a file still contains one of those values or a key-shaped string, the upload stops and nothing is sent.

The run bundle records variable names, never secret values. Values written in runner_env aren't secrets: they stay as written, in the run log and in the bundle's copy of review.yaml.

Registry sign-in

A private image needs a sign-in for its registry. In an image project, fill in "Registry sign-in" (registry, user name and a read-only token) and click "Save sign-in". The platform keeps it as a project secret of kind registry, named after the registry (for example REGISTRY_GHCR_IO) and sealed like any other. Its value is JSON:

{ "server": "ghcr.io", "username": "acme-bot", "password": "ghp_…" }

server is the host name and port only, without https:// or a path. The page shows the registry, the user name and the token's last four characters. The platform executor uses it to read the image's manifest, and the sandbox signs in with it before pulling and signs out at the end. A public image needs none.

A GitHub repository needs no secret to be cloned: each sandbox gets a GitHub installation token that lasts one hour and covers only the repositories you selected.

Your agent's model

A repository or image project doesn't need your model key. When its secrets have no OPENAI_API_KEY, the platform answers your agent's model calls and bills them as usage:

1. The sandbox gives your agent OPENAI_BASE_URL, an address on the sandbox itself, and OPENAI_API_KEY set to platform-model, a placeholder that opens nothing. It does so whatever agent.env_passthrough lists.
2. An agent that reads those two variables (any OpenAI SDK does) works unchanged: chat completions and the Responses API, streamed or not, with tool calls.
3. The test runner relays each call to the platform with the run's own session token, which the agent never sees. The platform calls the model with its own account; its keys never enter the virtual machine.

Which model answers:

The agent asks for	What answers
One of the platform's models by name: gemini-4-argon, gpt-6.1-sol, gemini-3.8-flash or gpt-6-luna	That model
Any other model	The project's model, chosen in the project's header (models)

When a model is replaced, the run log says so, and the report's setup check "Does the agent use a model to decide" records which models answered.

GPT-6.1 Sol and GPT-6 Luna reject some request settings, so the platform adjusts them:

• It sends max_tokens as max_completion_tokens, with room for the model's thinking when the agent asked for another model.
• It leaves out a temperature or top_p other than 1, and other sampling settings those models reject.
• It applies stop to the reply itself, so an agent that cuts at Observation: reads the text up to it as it asked.

When the model that answers isn't the one the agent asked for, the platform also adapts what that model doesn't take (since 20 October 2026). The openai-agents SDK, for example, asks gpt-5.2 for the reasoning effort none, which GPT-6.1 Sol refuses:

• A reasoning effort the model refuses (reasoning.effort, or reasoning_effort on chat completions) is sent as the closest one above it that the model takes: none becomes minimal, or low where minimal is refused too. It is never lowered.
• A setting the model doesn't take at all, such as verbosity, is left out. Only such settings are ever changed, never the messages, the tools or the reply's format.
• What a model refuses is learned from its own error and not sent to it again. The run log says each change once, for example "The platform model gpt-6.1-sol doesn't take reasoning.effort=none from the agent's request: the platform sent low instead (the closest above, never lower)".
• A model the agent asked for by name gets the request as the agent sent it: a value it refuses is the agent's own.

When the model still refuses the request, or answers with a rate limit or a server error, that is the platform's problem, not your agent's: the attempt is never judged as the agent's, the platform re-runs it once, and it isn't charged (see re-runs by the platform). A rate limit's retry-after is passed on to your agent, so its SDK waits as long as the model asks.

The Responses API (/v1/responses, the openai-agents SDK's default) works with every model. Gemini 3.8 Flash has none of its own, so with it as the project's model the platform answers such a call through Gemini's chat completions and translates the request and the reply, streamed or not:

• instructions, text and image input, function tools with their calls and results, tool_choice, parallel_tool_calls, structured output (text.format), max_output_tokens, temperature and top_p work as on a GPT model.
• previous_response_id works for 6 hours after the response it names, unless the platform restarts meanwhile. A response sent with store: false isn't kept. The openai-agents SDK sends the whole conversation anyway.
• OpenAI's reasoning items an agent sends back are left out: Gemini thinks for itself.
• When the agent asked for another model, a reasoning effort below low (none, minimal) is sent as low, and text.verbosity, which Gemini doesn't take, is left out. The run log says so, as for the GPT models.
• When Gemini's safety filters block the request or the reply, the response ends incomplete with the reason content_filter, as OpenAI's content filter does.
• What only OpenAI's servers have gets an HTTP 400 error that names it: hosted tools (web search, file search, code interpreter, computer use, image generation, remote MCP), files by file_id, stored prompts, conversation and background mode.

What it costs and where it stops:

• Billed as "Platform model (AI under test)" usage, at the model prices. How much an agent will use can't be known before the run.
• Each run has a spending limit of about $6.50. The limit is lower when the plan's credit and the credit balance, the spending cap or the prepaid balance have less than that left. Once the limit is reached, further model calls get an HTTP 402 error with the code agent_model_budget_exhausted, and the run stops after the attempt in progress with a partial report.
• Only POST /v1/chat/completions and POST /v1/responses are answered. Other endpoints, such as embeddings, answer 404, so an agent that needs them needs its own key.
• When a GPT model answers, Azure's default content filter applies, and a reply it blocks is reported as such (content filters).

To use your own key instead, add OPENAI_API_KEY as a project secret and list it in agent.env_passthrough. Your provider then bills you, and nothing goes through the platform. The wizard's "Secrets" section says which of the two a run will use. With the platform's models, that section, the checks before "Start test" and the cost ceiling all name the models that answer, as in the table above, rather than one model for every call. An agent on another provider (Anthropic, its own Azure endpoint) needs that provider's key as before.

Judge model keys

The judge model grades replies for the checks that need one. Whether it needs a key of yours depends on where the run happens:

Where the run happens	Judge model	Key
Sandbox (repository or image)	The project's model (models), when judge.model names a model the platform provides	None. The sandbox reaches it through the platform, whose key never enters the virtual machine. Billed as judge model usage.
Sandbox (repository or image)	Any other model in judge.model	Its provider's variables as project secrets: OPENAI_API_KEY (openai:), ANTHROPIC_API_KEY (anthropic:), OPENAI_COMPAT_BASE_URL and OPENAI_COMPAT_API_KEY (openai-compatible:), AZURE_AI_BASE_URL and AZURE_AI_API_KEY (azure:). You pay the provider directly.
Local test runner	With the judge model's key on the machine, any model in judge.model; without one, the project's model when judge.model names a model the platform provides	With a key: the same variables, set on the test machine or in the .env next to review.yaml. Without: none. The test runner calls the platform's judge proxy with its own token, the platform's key never leaves the platform, billed as judge model usage.
Chat AI over HTTP	The project's model	None: the platform's judge.

The judge's key never reaches the agent: an OPENAI_API_KEY you pass through stays the agent's own.

Platform judge spending

A platform-provided judge is capped per run (see spending controls). Once the cap is reached, the rest isn't judged. What isn't judged doesn't count as passed, so the verdict is never a pass.

Next steps

• review.yaml reference: the fields that name variables.
• Sandboxes and limits: what else a sandbox can reach.
• Security overview: encryption, isolation and retention.