Security overview

Preworld Review runs code you give it and sends it attacks on purpose, so isolation is the core of the design. This page says what protects you and your data, and marks each item: In place works today; Planned is on our roadmap and not yet available. For the questions a vendor-security review usually asks, see the security questionnaire.

Isolation of cloud runs

Applies when you connect a GitHub repository or bring your own image.

• One fresh virtual machine per run In place. Every run and every pre-run gets its own virtual machine, created from our machine image (Ubuntu 24.04 LTS) and deleted when the run ends. It runs on Google Compute Engine in London, or on Microsoft Azure for a project whose sandbox cloud is set to Azure (Sweden Central for now, UK South in London once it is available to us). A virtual machine is never reused and never shared between runs or customers.
• Maximum run time In place. Your organization's limit (120 minutes by default) is enforced. When 5 minutes are left, the run stops after the attempt in progress and reports what ran. A run still going at the limit is stopped and its virtual machine deleted. As a backstop, every virtual machine on Google Cloud is created with a maximum run duration of the limit plus 10 minutes, after which Google deletes it regardless. Azure has no such setting. Every virtual machine there gets an auto-shutdown schedule 15 minutes after the limit instead; at that time Azure stops and deallocates it regardless, and the platform then deletes it. Because that schedule is a time of day, a project runs on Azure only while its organization's limit is at most 1,420 minutes (23 hours 40 minutes).
• A separate network In place. Sandbox virtual machines live in their own VPC network (in production also in their own Google Cloud project), with no external IP address. All inbound traffic is denied; outbound traffic is allowed on TCP ports 443 and 80 only, through Cloud NAT on one fixed address. On Azure the same rules hold in a virtual network and resource group of their own, through a NAT gateway on a second fixed address. Port 80 is there for package mirrors during image builds; your agent itself is held to 443 (next point).
• No cloud identity In place. Sandbox virtual machines have no service account, so the metadata server hands out no Google Cloud credentials; on Azure they have no managed identity, and VM extensions are switched off. The start-up script comes from the machine image, not from metadata.
• Your agent runs as its own unprivileged user In place. The test runner and your agent are different Linux users. The agent user is not in the Docker group (the Docker socket is equivalent to root), has no sudo, and cannot read the test runner's files or home directory. Per-user firewall rules stop it from reaching the metadata server (on Azure also the platform's host address, except for name lookups), the control plane and the container network. It can reach your services on localhost, and the internet on port 443.
• Each attempt in its own resource-limited scope In place. Each attempt starts your agent in a separate systemd scope with a memory limit (the class's memory minus 1 GB) and a limit of 512 tasks, in its own working directory. When the attempt ends, the whole scope is killed, including any background processes the agent started.
• Single-use tokens In place. A virtual machine starts with a bootstrap token that is valid for 10 minutes and works once; the test runner exchanges it for a session token that is valid only for that one job and is revoked when the job ends.
• No automatic re-runs In place. If a virtual machine disappears mid-run, the run is marked incomplete rather than retried, because your agent may already have changed data in your test backend.

Automated isolation checks

We run 27 automated checks on a virtual machine booted from each new machine image, on both architectures (x86 and ARM). The current images passed 27 of 27 on 29 September 2026. The Azure image (x86 only) gets four more, 31 in all: the agent and its containers cannot reach Azure's host agent address (WireServer, ports 80 and 32526), and root cannot get a managed-identity token. Result: 31/31 (2026-10-01).

Area	What is checked
Network	The agent cannot read the metadata server; cannot reach the control plane; an HTTP redirect to the metadata server is blocked as well; plain HTTP (port 80) is blocked. The agent can reach the internet on port 443 and resolve names; the test runner can reach the control plane.
Local privileges	The agent cannot use Docker, cannot use sudo, cannot read the test runner's home directory and cannot list the test runner's working directory.
Services	The agent can reach a service published on localhost, but cannot reach the container network directly.
Per-attempt scope	The test runner starts the agent as the agent user with the job's environment; the agent and a background (nohup) child both run inside the scope; killing the scope leaves no agent process behind; going over the memory limit kills the attempt; the task limit stops a fork bomb.

Other connection methods

• Local test runner: your agent runs on your own machine as an ordinary process, with only the environment variables your review.yaml passes to it. There is no container isolation there, so use a dedicated test machine. The test runner only connects outbound; the platform can choose which confirmed tests to run but cannot send commands.
• Chat AI over HTTP: the platform executor only sends HTTP requests and never runs your code. It checks and pins every address a URL resolves to and refuses cloud metadata addresses and private or local addresses. Tool calls your AI asks for are recorded, never executed.

GitHub App permissions

The App is installed only on the repositories you select, and asks for these permissions (GitHub requests an App's permissions as one set, so an installer can't leave one out):

• Repository contents: Read: clone the commit being tested into the sandbox.
• Metadata: Read: list your repositories and branches in the wizard.
• Pull requests: Read and write: run on pull requests and, on projects with checks switched on, keep one comment with the release gate's table on the pull request. An installation made before the App asked for write keeps Read until an owner accepts the permission update; until then there is no comment, only the check.
• Checks: Write: used only on projects with checks switched on, to post one check run per automatic run (the verdict and a link to the report). An installation made before the App asked for it needs an owner of the GitHub organization to accept the permission update on GitHub; until then runs go on as before, with no check.

Code access is read-only. The App never pushes code or opens pull requests; it writes only the check run and that one pull request comment. Installation tokens are valid for one hour and cover only the selected repositories.

Encryption

• In transit In place. The web app and API are served only over HTTPS (TLS 1.2 or 1.3, certificates from Let's Encrypt, HTTP Strict Transport Security). Test runners and sandboxes connect to the platform over HTTPS, outbound only.
• Secrets you store In place. Project secrets and chat AI keys are sealed as they arrive: encrypted with AES-256-GCM under a fresh key, and that key wrapped with RSA-OAEP (SHA-256) using the platform's public key. The API holds only the public key, so it stores what you entered but can never read it back. You see only the last four characters. Only the platform executor holds the private key. A sandbox gets its project's values from the executor by presenting its session token, and secret values are replaced with a placeholder in the logs it uploads.
• At rest In place. Google Cloud encrypts all stored data by default: the database, object storage, backups and disks. Microsoft Azure does the same for the disks of sandbox virtual machines that run there. Customer-managed encryption keys are not offered.
• Platform credentials In place. The platform's own credentials (database password, signing keys, model API keys) are kept in Google Secret Manager, replicated only within London, and read at start-up into memory-backed files that only the service can read.

Data residency

Everything we store is in Google Cloud's London region (europe-west2): the database, run bundles and evidence, test files, backups, secrets and logs. Database backups stay in the same region. The sandbox virtual machines run there too, unless a project chooses Azure as its sandbox cloud. That project's runs then use virtual machines in Microsoft Azure's Sweden Central region (EU), moving to UK South (London) once it is available to us. Those virtual machines hold the run's code, test data and output only while the run lasts, and the run bundle is stored in London. The exceptions are the clouds' own records of changes to our cloud resources. Google Cloud keeps the Admin Activity and System Event audit logs of our cloud projects (who changed which cloud resource) in its global log storage, and Microsoft Azure keeps the activity log of our Azure subscription itself.

Data leaves the UK only where a service we use sits elsewhere:

• The platform's models judge, run the assistant and the platform AI, and answer a platform-model chat AI, so they receive the text they grade or answer. They are Microsoft Azure's Global Standard deployments (GPT-6.1 Sol, GPT-6 Luna) and Google's Vertex AI global endpoint (Gemini 3.8 Flash), which may process it in any of their regions. The embeddings for similarity checks stay in London.
• OpenAI or Anthropic in the United States, only where a setup names one of their models.
• A project set to run its sandboxes on Azure runs in Sweden Central until UK South is available.
• A visitor who accepts analytics cookies sends Microsoft Clarity clicks, scrolling and page layout. Every text of the web app is hidden, and nothing is sent from pages with reports, logs, tests or projects (cookies).
• Payments will go through Stripe.

See sub-processors.

Your agent's own model calls go wherever your agent sends them, with your keys. A repository or image project with no OPENAI_API_KEY sends them to the project's model instead (Microsoft Azure or Google Cloud, as above), which receives what your agent sends it (your agent's model).

Data boundary by connection method

Connecting a repository means source code enters our cloud. The wizard says so and asks you to confirm before it saves the binding.

Connection	What enters our cloud	What stays after the run
GitHub repository	The source code at the chosen commit (cloned into the sandbox), your .reviewctl/ test files, project secrets (sealed), the dependency cache	The run bundle (evidence, redacted traces, logs) and the dependency cache (dependencies only, 30 days). The source code is deleted with the virtual machine.
Repository with a devcontainer or Dockerfile In place	As above, plus the built image archive	As above, plus the image archive (per commit, deletable in settings)
Your own container image In place	The image (pulled into the sandbox), test files, project secrets (sealed)	The run bundle. The image is deleted with the virtual machine.
Chat AI over HTTP	The conversations (questions and replies) and the chat AI's key (sealed)	The run bundle
Local test runner	Only run bundles and test files. Source code, data and credentials stay on your machine.	The run bundle
SDK In place	What your code uploads: each row's input, expected output, output and scores, and the spans of the calls you trace. Your agent runs in your process; your model keys never reach us.	The experiment and its scores; its traces for the retention period
OpenTelemetry In place	The spans your exporter sends, with their attributes, inputs and outputs	The traces, for the retention period
AI Gateway In place	Each model call your app makes through it: the request and the reply. A provider key sent in x-provider-key is passed on for that call only, never stored or logged	One trace per call, for the retention period

Run and evaluate only In place. Preworld Review never runs a coding AI (such as Claude Code or Codex) on your code and never pushes changes to your repository. A sandbox builds and runs your agent for the tests, and its machine image has no coding AI installed. A coding AI you connect yourself (the "dev AI") runs on your machine with your own key. The platform hands it a prompt and receives the draft tests it writes; it cannot confirm tests or start a test run.

No connection method is meant for production data: our terms require test environments and test credentials.

Retention and deletion

Data	How long	Status
Source code cloned into a sandbox	Deleted with the virtual machine when the run ends	In place
Dependency cache	30 days	In place
Evidence files and sandbox logs	90 days by default; organization owners can set 1 to 3,650 days	In place: the setting, and the nightly deletion of expired evidence files and logs
Reports and verdicts	Kept for as long as your account exists	In place
A deleted project	Hidden at once and restorable; permanently deleted 30 days later	In place: hiding, restoring, and the permanent deletion 30 days later
Deletion on written request	Deleted immediately on a written request from your organization	In place: an operator runs the admin deletion for the project or the whole organization; it is recorded in the audit log
Database backups	Point-in-time recovery for 7 days, 14 daily backups, daily exports kept 90 days	In place
Control-plane logs	30 days	In place

Deleted data can remain in backups until those expire (at most 90 days); backups are not used for anything but restoring the service.

Audit log

• Only people make sensitive changes In place. Confirming tests, setting pass criteria, starting tests, pairing test runners, storing secrets, binding repositories and changing organization policy can only be done by a signed-in person in the browser. Command-line tokens can't, and the in-app assistant can only prepare a card that a person clicks.
• Assistant tool calls In place. Every tool call the in-app assistant makes is recorded.
• Organization audit log In place. A log of every managed action (who, what, when, from which IP address) that the organization's owners can read in the web app (Organization → Audit log).

Access

• Tenant isolation In place. Every query is filtered by organization; another organization's resources answer "not found".
• Least-privilege service accounts In place. The control-plane virtual machine's service account can write logs and metrics, read container images, read its own secrets, read and write objects in its own three buckets, and create and delete virtual machines only in the sandbox project. On Azure the control plane signs in with a federated credential, so no key is stored. The identity it signs in as can only create, read and delete sandbox virtual machines (with their disks, network interfaces and shutdown schedules) in the sandbox resource group, and read the machine images. It cannot create public IP addresses, run commands or extensions, identities or role assignments. Sandbox virtual machines have no service account or managed identity at all.
• Administrative access In place. SSH to the control plane goes only through Google's Identity-Aware Proxy (port 22 is closed to the internet), with OS Login, so there are no SSH keys to manage. The database has a private IP address only.
• Tokens and sessions In place. Sessions, API tokens and test runner tokens are stored only as SHA-256 hashes. The session cookie is HttpOnly, SameSite=Lax and Secure. A test runner's token works only for its own project's runner protocol, and you can revoke it at any time. A CLI token stops working after 90 days unused and a year after sign-in; reviewctl logout revokes it, and each person can revoke their own under Organization → My CLI tokens. Removing a member revokes the test runners, tokens and sessions they had. Signing in with GitHub or Google uses the authorization code flow with PKCE and a one-time state tied to the browser for ten minutes. Google's ID token is checked (signature, issuer, audience, expiry and nonce). Only email addresses the provider has verified can match an account, and the provider's access token is used once and never stored.
• Single sign-on (SAML / OIDC) is not offered yet.

Monitoring and continuity

• Uptime monitoring In place. The health endpoint is checked continuously from outside; failures alert us by email, as do budget thresholds.
• Backups and restore drills In place. Automated daily database backups with 7 days of point-in-time recovery, a second daily export to a separate bucket, and versioned object storage. We rehearse a restore every quarter; the last point-in-time restore drill passed on 28 September 2026.
• More alerts Planned: error rate, response time, database storage, sandbox start failures, leftover virtual machines and long queues.
• Public status page Planned. See status.

Independent assurance

• External penetration test Planned: before our first enterprise contract.
• Certifications: Preworld Review is not SOC 2 or ISO 27001 certified. The underlying infrastructure is Google Cloud's and, for sandboxes on Azure, Microsoft Azure's; both hold those certifications.

Responsible disclosure

If you think you have found a security problem, email huanting.w@nice2.io with what you found and how to reproduce it. Please test only against your own organization, don't access other customers' data, don't degrade the service for others, and give us reasonable time to fix the problem before you disclose it. We won't take action against good-faith research that follows these rules.