Command line (reviewctl)

reviewctl signs in to the platform, runs a local test runner, runs tests on your own machine, uploads runs and connects a coding AI. This page lists its commands and options.

Install

You need Node.js 22.5 or later. The platform serves the install package itself, with a version number, at https://<platform address>/downloads/:

curl -fsS https://<platform address>/downloads/latest.json
# {"version": "0.1.20260930120509", "file": "reviewctl-0.1.20260930120509.tgz", "sha256": "…", "size": …, "node": ">=22.5", "built_at": "…"}
curl -fsSO https://<platform address>/downloads/reviewctl-<version>.tgz
curl -fsSO https://<platform address>/downloads/reviewctl-<version>.tgz.sha256
shasum -a 256 -c reviewctl-<version>.tgz.sha256    # on Linux: sha256sum -c …
npm install -g ./reviewctl-<version>.tgz
reviewctl --version                                 # e.g. 0.1.20260930120509+2e690dd

npm fetches the CLI's open-source dependencies from its registry. Each platform release comes with a new version; the last number is when it was built. A test runner's row under Targets → "Test runners" says when a newer version is available and gives the upgrade command.

reviewctl keeps its files in ~/.reviewctl/; set REVIEWCTL_HOME to move them. There you'll find credentials.json from login, runners/<project id>.json from runner register and dev-links/<repository>-<hash>.json from dev connect, one per repository, each readable only by you.

Before any command runs, it loads the .env next to --config, or else the one in the current folder. Variables already set in the shell win.

Language

reviewctl speaks English unless the system language (LC_ALL, LC_MESSAGES or LANG, the first one set) is Chinese. Set REVIEWCTL_LANG=en or REVIEWCTL_LANG=zh in the shell or in the .env to choose. Only the words for people change: JSON output, error codes and exit codes stay the same.

Output and exit codes

• --json makes a command print one JSON object on standard output, and nothing else there.
• An error is one line on standard error (Error: …). With --json it is one JSON object, {"error": {"code": "…", "message": "…"}}, on the last line of standard error. Either way the exit code is 3.

Exit code	Test commands (review --wait, report, retest --wait, eval)	doctor
0	Passed (including a pass on the replies only)	Everything is ready
1	Not passed (eval: also below --min-pass-rate)
2	Didn't finish: no verdict	Some checks have gaps
3	Error: no valid report	review.yaml is not valid

Sign in

reviewctl login --api https://<platform address>

It prints an address and a code such as ABCD-EFGH. Open the address and sign in; with GitHub or Google you come straight back to the code. Enter the code under "Approve CLI sign-in" and click "Approve". The code is valid for 10 minutes. The CLI then saves a CLI token. --label <name> sets the name it is listed under (default reviewctl@<host>).

A CLI token is yours, for one organization. It stops working after 90 days unused or a year after sign-in, and you can revoke it under "Organization" → "My CLI tokens". It can read and upload runs, and run reviewctl connect --ai, which pre-runs its checks on your own test runner. Confirming tests and starting runs on the platform stay with a person in the browser.

reviewctl logout      # revokes this machine's CLI token on the platform, then deletes the saved file

In CI, set REVIEWCTL_API_URL and REVIEWCTL_TOKEN (a CLI token from reviewctl login) instead of a saved sign-in.

Set up a project

reviewctl init --project <project id> --domain <domain>
reviewctl doctor

Command	Options
init	--project <id>: the project's id from the web page (prj_…), which runner register needs; without it, a new id. --domain <name>: orders-refunds (the default, with the built-in mock backend), expense, generic or any lowercase name. --backend fixture|own. --dir <path> (default .). --with-templates: copy the domain's example scenarios. --force: overwrite an existing review.yaml. --answers <json>: answers without questions. --json.
doctor	Checks what the connection can do: reset, tool observation, fault injection, reading results, judge ≠ agent, scenarios and pass criteria. --config <path> (default review.yaml). --probe: run one task with no side effects (the tool proxy denies every write) to prove the tool calls are seen. --json.

init looks at the project (frameworks, MCP configuration, model key variable names, prompt files) and writes review.yaml and a .review/ folder. Fields are in the review.yaml reference.

Run a test runner

reviewctl runner register --api https://<platform address> --code ABCD-EFGH --config review.yaml
reviewctl runner start --config review.yaml

Command	Options
runner register	--api <url> and --code <code> are required; the code comes from "Create pairing code" in the "Local test runner" wizard under Targets, works once, within 10 minutes. --config <path> (default review.yaml): its project.id must be the project's. --label <name> (default runner@<host>). --json. It reports the agent's tool names, data query names and identity variable names, never contents or values.
runner start	--config <path>. --once: take one job if there is one, then exit. --poll-ms <n>: wait between claims while idle (default 5000). --allow-auth-ref <names...>: extra identity variables (env:NAME) tests from the platform may use. --no-collect-files: don't copy test files from the repository into the scenario folder. --self-update: on "Update and restart" from the web, download the platform's latest package, check its SHA-256 and install it with npm install -g before restarting (see have "Restart" update it too).

runner start keeps running, asks the platform for work every few seconds and reports a heartbeat. Press Ctrl-C once to stop after the current attempt and upload a partial report, twice to stop at once. "Restart" on the test runner's row on the web restarts it with the reviewctl now on disk. Run it under systemd or launchd so it survives reboots. The options --job, --bootstrap-file, --api and --work-dir of runner start are for sandboxes only.

Let AI finish and check

reviewctl connect --ai --project <project id> [--code ABCD-EFGH]

Run it in your agent's repository after reviewctl login. The platform's model reads the repository, writes the test connection files under .reviewctl/ and pre-runs a check on this machine. When the check fails, it revises the files and checks again, up to 10 checks. Local test runner: let AI finish and check says what it reads and sends, and how it writes the files.

Before it reads, sends or registers anything, the command asks the platform whether it may start: on a project connected through a GitHub repository or its own image it stops there, without registering this machine or using the pairing code. When the project already has one running, it says which machine that is on and when it last answered, and asks whether to cancel it and start again (--yes never cancels one for you). Before sending, it lists what this machine needs ready (the repository's own dependencies, the keys the agent calls its model with; the judge model is the platform's, billed by usage); checks and tests run in a temporary copy of the repository. When the platform has a newer reviewctl, it says how to install it when it starts.

Option	What it does
--ai	Required: use the platform's model to write the test connection files and check them (its calls count against the assistant's daily allowance).
--project <id>	Required: the project's id on the web page (prj_…).
--code <pairing code>	Needed when this machine isn't a test runner of the project yet, to register it (click "Create pairing code" under "Local test runner" on the web).
--dir <path>	The repository root; default: the nearest folder with .git from here up.
--yes	Don't ask: send, write the files and check.
--timeout <minutes>	How long to wait at most (default 120).
--json	Print one JSON object at the end (the progress still goes to standard error).

Exit codes: 0 the connection works; 1 it stopped, with the reason in the output (a secret missing, a test to confirm, can't be tested this way, the agent's own problem, 10 checks failed); 2 not finished (you said no, cancelled, or it timed out); 3 an error.

Run tests on your machine

These run the scenarios in scenarios.dir against your agent, on this machine, with the gate in review.yaml. Nothing goes to the platform unless you push it.

Command	Options
review	Starts a run in the background and prints its id. --wait: wait and print the report, with the exit codes above. --json: the full report.json. --summary: a short JSON summary (verdict, counts, problems, the report's path). --repeats <n>. --scenarios <ids> and --conditions <ids or normal,fault,attack>, comma-separated. --label <text>: a version label. --estimate: planned attempts and model call limits only; runs nothing. --config <path>.
report	--run <id> is required. --format markdown|detail|json|summary (default markdown). --finding <id>: one problem as JSON, with its evidence and fix.
retest	--run <baseline id> is required. Reruns that run's scenarios against the current agent and compares. --wait, --json, --summary, --repeats <n> (default 3), --label <text>, --config <path>.
cancel	--run <id> is required. Stops after the current attempt; a report is still written. --reason <text>, --json.
scenarios validate <dir>	Checks every scenario in the folder against state_queries.yaml. --json.
scenarios confirm <file>	--by <name> is required. Marks a scenario confirmed; changing it afterwards fails validation.
scenarios draft	--template <id> --id <slug> --task <text> are required; --order <id>, --actor <user_id>, --no-record, --config <path>, --json. Drafts a scenario from a template and one recording.

Each local run keeps its files in .review/runs/<run id>/ next to review.yaml, among them report.json, report.md and fixes.md, the fix notes to hand to a coding AI.

Upload a run

reviewctl push --run <run id>

Uploads a local run to the platform and writes its report link back into the local report. Uploading the same run again changes nothing. It needs reviewctl login (or REVIEWCTL_API_URL and REVIEWCTL_TOKEN). The address is --api <url>, else cloud.api_url in review.yaml, else the one you signed in to. --config <path>, --json. Set cloud.auto_push: true to upload every local run when it ends.

Evaluate a test set in CI

reviewctl eval --set support-faq.yaml --target target.yaml --min-pass-rate 90% -o results.xml -o summary.md

Runs a test set against a chat AI on this machine, for CI. Two options are required. --set <file> is the test set; test sets on the platform can be exported to this file. --target <file> is the AI's connection: kind, base_url, model, system_prompt or system_prompt_file, and key_env, the variable holding its key.

Option	What it does
--gate <file>	Pass criteria; without them every question must pass, or --min-pass-rate decides.
--min-pass-rate <rate>	The lowest pass rate on normal questions: 0.9 or 90%. REVIEWCTL_PASS_RATE_THRESHOLD works too.
--repeats <n>	How many times to ask each question.
-o, --output <path>	Write the results; the extension picks the format: .json, .jsonl, .csv, .html, .md, or .xml for JUnit. Repeatable.
--baseline <ref>	Compare with a report.json, a run id on the platform, or latest (needs --project).
--project <id>, --push	Upload the run to that project on the platform.
--label <text>, --title <text>	A version label, and the summary's title.
--judge <model>	The judge model; default JUDGE_MODEL, else openai:gpt-4.1-mini. Without its key, only rule checks run.
--json	Verdict, pass rate, exit code and the files written, as JSON.

Set REVIEWCTL_STRIP_OUTPUT=1 to leave the AI's replies out of what eval writes.

Launch the project's tests and wait for the release gate

REVIEWCTL_API_URL=https://<platform address> REVIEWCTL_TOKEN=pj_your_token_here \
  reviewctl eval --launch --project prj_01J8ZK3Q9MXYZ0000000000000 --commit "$GIT_SHA" --wait 60 -o summary.md

--launch starts every confirmed test of the project on the platform (POST /v1/projects/<id>/jobs, with the project's API token), waits for the release gate of the experiment the round becomes, prints its table and exits 0 (passed), 1 (blocked), 2 (not judged: no rules, not finished, or not final within --wait) or 3 (an error). It takes --project (required), --commit, --image, --scope all|functional|security|failed, --repeats, --wait <minutes> (default 60), --idempotency-key, --title, -o <file.md|file.json> and --json; --set, --target, --gate, --baseline, --push and --min-pass-rate don't go with it. See CI and the GitHub Action.

The SDKs have a command of their own, preworld eval, for *.eval.ts and eval_*.py files: see SDKs.

Connect a dev AI

reviewctl dev connects the coding AI on a developer's machine to one project and lets it go through the whole loop: fetch its prompt, hand in tests, change the test connection, pre-run, start formal tests, read reports and fix notes. How far it may go is up to the permissions ticked for the connection on the web: confirming tests, starting formal tests, changing the test connection or the pass criteria stay with a person on the web unless the connection is allowed them. How to make a connection, what each permission does and the daily cap are in Using the platform from a coding AI.

reviewctl dev connect <code> --api https://<platform address> --register claude

The code comes from "Get connect command" in "Settings" → "Dev AI" on the web, and works once, within 10 minutes; one the page replaced with "Get a new code" is refused, saying it was voided. The connection is saved per repository under ~/.reviewctl/dev-links/ and lasts 30 days: one machine can be connected to several projects at once, and each command uses the connection of the repository it runs in (the ~/.reviewctl/dev-link.json an older version saved is still used in a repository with none of its own).

dev connect option	What it does
--api <url>	Required: the platform address, in the command the web page gives.
--register <tool>	Which coding tool to register the MCP server reviewctl-dev (reviewctl dev mcp) with: claude (claude mcp add when claude is installed, else the repository's .mcp.json), cursor (.cursor/mcp.json), codex (codex mcp add when codex is installed, else only the lines to add to config.toml) or none. Left out, it asks at a terminal, and elsewhere only prints how to register by hand. A different reviewctl-dev already there is replaced only on a y at a terminal.
--yes	Don't ask: without --register, take the default (claude when it is installed). A different reviewctl-dev already there is still left alone.
--runner	Register the MCP server with --runner; see dev mcp below.
--json	Print one JSON object only.

Command	What it does	Permission
dev status	Drafts, missing test files, todos, whether the test runner is online, what this connection may do, and what to do next.
dev prompt	The prompt the platform wrote for this repository: which tests, files and todos to write, and how to hand them in. It carries the test plan approved for the project before, if there is one.
dev submit <file>	Hand in the JSON written (- reads standard input); lists the problems and exits 1 when it doesn't pass.	Write test drafts and upload files
dev upload <repo_path>	Upload a test file a draft is missing. --name <name>: its name in the tests.	Write test drafts and upload files
dev wiring	The project's test connection files on the platform: each one's size, hash and who saved it last. --show prints their text too.
dev push <files...>	Save files under the repository's .reviewctl/ on the platform, checked by the same rules as saving on the web.	Change the test connection files
dev assist	Have the platform's AI finish the test connection and check it (a project on a GitHub repository or its own image; for one on a local test runner it says to run reviewctl connect --ai in the repository). --status only shows how far it got.	Have the platform's AI finish the connection (to start it)
dev tests	Tests and drafts: id, AI review, last result, and the pass criteria.
dev test <id>	One test's rules (YAML): by the test's id its draft (or its latest confirmed version when it has none), by a version's id that version.
dev review	Have the platform's AI review the drafts again: where each is, why any didn't pass, which wait for a person.	Write test drafts and upload files
dev confirm <id>	Confirm a test's draft, checked as when a person confirms it on the web.	Confirm tests
dev setup	The latest pre-run: whether the tests reach the agent, and fix notes when not; exits 1 when it didn't pass.
dev retry	Pre-run again after a fix and wait: 0 passed, 1 not passed, 2 not finished yet. --no-wait, --timeout <seconds> (default 600).	Pre-run
dev run	Start a formal test, as "Start test" on the web: a pre-run first, then the test once it passes; held to the connection's daily cap (with no daily limit, to what the organization has left). --scenario <ids...> (default: every confirmed test), --condition <ids...> (with --scenario; faults alone, with no normal condition and no attack, need pass criteria that name them), --repeats <n>, --wait (0 done, 1 didn't run through, 2 not finished yet), --timeout <seconds> (default 600).	Start formal tests
dev job [job_id]	Where a job is: status, the pre-run's result, each attempt's result and what next; default: the latest formal test this connection started. --wait, --timeout <seconds>.
dev cancel <job_id>	Cancel a job this connection started (it stops after the attempt in progress).
dev runs	The latest formal tests: run_id, verdict, problems and report link. --limit <n> (default 10).
dev report [run_id]	A formal test's conclusion (Markdown written for an AI; data with --json); default: the latest.
dev diagnosis [run_id]	The platform AI's diagnosis: for what failed or couldn't be decided, whether the cause is the agent, the test connection or the test itself, and what to do.
dev fixes	The fix notes for AI from the latest report, or --run <id>.
dev gate <file>	Save the pass criteria in the file as the project's (JSON or YAML, in the format of gate in dev tests --json).	Change the pass criteria
dev tasks	The tasks sent to it through the API.
dev done <task_id>	Mark a task done; --failed --note <text> hands it back.
dev pull	Bring the project's test connection files (.reviewctl/, as changed on the platform) into the repository, listing what was added, changed or unchanged; it only writes the files, for you to review and commit. --dir <path>.
dev disconnect	Disconnect this repository: its connection is closed on the platform (even if it was already closed or expired) and the one saved here for it deleted; other repositories' stay. Remove the registered MCP server yourself; the command says how.
dev mcp	Serve all 40 tools as an MCP stdio server (those of the commands above, plus 6 for the test plan, 5 of them removed and only saying so, and 9 for experiments, logs and the release gate), see the loop and the tools. --runner: for a project on a local test runner, once the server is up, start the project's test runner in the background (reviewctl runner start; this machine has to be registered as its test runner, with none online on the platform), and stop it when the server exits; its log is ~/.reviewctl/logs/runner-<project id>.log. --config <path>: the review.yaml for --runner; default: the one in this folder, else .reviewctl/review.yaml.

All of them take --json. A step the connection isn't allowed isn't done, and the command says to ask a person to "Edit permissions" on the connection in "Settings → Dev AI".

For a project on a local test runner, dev retry (try_run in MCP) and dev run run on the test runner, which has to be online; when it isn't, nothing starts, and dev status says what to do.

MCP servers and skill

There are two MCP servers, for different jobs:

Server	How it starts	What it does
reviewctl-dev	reviewctl dev mcp (after reviewctl dev connect), or, with no reviewctl installed, the platform's MCP over HTTP at https://<platform address>/v1/dev/mcp	Works on one project on the platform through the loop of connection, tests, review, pre-run, formal test and report, and reads its experiments, logs and release gate: 40 tools (over HTTP, all but upload_file, which reads this machine's files). How to register it: Using the platform from a coding AI.
Local tests	reviewctl mcp --config review.yaml	Runs tests on this machine from review.yaml, without the platform: check_capabilities, review_project, get_report, retest and cancel_run.

reviewctl skill install --host claude-code|codex installs the product-review skill into a coding assistant (--project for this project's .claude/skills/, --dest <dir> for another folder). reviewctl skill where prints where each host keeps skills.

Environment variables

Variable	What it does
REVIEWCTL_LANG	en or zh: the language of messages.
REVIEWCTL_HOME	Where reviewctl keeps its files (default ~/.reviewctl).
REVIEWCTL_API_URL, REVIEWCTL_TOKEN	The platform address and a CLI token, in place of a saved sign-in (CI); for eval --launch, the project's API token pj_….
REVIEWCTL_PASS_RATE_THRESHOLD	eval's --min-pass-rate.
REVIEWCTL_STRIP_OUTPUT	Leave replies out of eval's output files.
JUDGE_MODEL	eval's default judge model.
JUDGE_SECOND_MODEL	eval's second judge model: a reply the first couldn't decide is read again by it, with the same prompt. Its key has to be on this machine; the same model as the first isn't used. Unset, nothing is read again, and the results say so. When the two read something differently, the stronger one decides, by the platform's model catalog, with any model outside it ranked below; when both are outside it, it stays undecided (see the second judge model).
REVIEWCTL_GIT	The git to read the code version with.

Next steps

• Local test runner: install, register and start step by step.
• review.yaml reference: what init writes and doctor checks.
• API and tokens: start runs from CI without the CLI.