Filter language

One small language filters traces, experiments, experiment rows and dataset rows, on the web (Logs, Experiments, an experiment's rows, a dataset, online scoring rules, review queues' daily fill) and in the API's filter= parameter.

field op value [AND|OR field op value …]

Grammar

• A clause is field op value, for example score.factuality < 0.5.
• Join clauses with AND and OR (any case). AND binds tighter than OR; use parentheses to group: (tag = a OR tag = b) AND score.x >= 1.
• There is no NOT; use !=.
• An empty filter matches everything. A filter is at most 2,000 characters.

Operators

Operator	Meaning	Works on
= !=	Equal, not equal (exact, case-sensitive)	Every field
> >= < <=	Compare numbers	score.*, delta.*
~	Contains, case-insensitive	Text fields (not score.* or delta.*)

A missing or undecided value matches only !=. On a field with several values (the models, tools or tags of a trace), = matches when any value is equal, != when none is, and ~ when any contains the text.

Values

• A bare word runs until a space or one of ( ) " ' = ! < > ~, so builtin:task_done, gpt-4.1 and sc_01J9…@2 need no quotes.
• Quote anything else with "…" or '…'; a backslash escapes the next character: input ~ "say \"hi\"".
• Quote a value spelled and or or, which would otherwise read as a keyword.
• Numbers (0.5, -0.25, .5) count as numbers only for score.* and delta.*.

Fields

Field	Traces (Logs)	Experiments (list)	Experiment rows	Dataset rows
score.<scorer>	The trace's online or human score	The scorer's mean	The row's score (mean of its decided trials)	—
delta.<scorer>	—	—	The row's score minus the baseline row's	—
status	ok, error, cancelled	running, done, cancelled, failed, invalid	decided, undecided	—
source	otlp, sdk, gateway, sandbox	round, run, sdk, playground	run (from a sandbox or runner report), native	manual, import, log, experiment, review, ai, scenario, test_set
environment	dev, staging, prod, ci	The same	—	—
model	Every model the trace called	The target's model	The judge models	—
tool	Every tool span's name	—	—	—
tag	The trace's metadata.tags	The experiment's metadata.tags	The row's tags	The row's tags
condition	—	—	normal, fault, attack, or finer: fault:<tool>:<fault>, attack:<vector>[:<plugin>]	The same
input, output	The trace's input and output (the first 500 characters)	—	The row's input and output	The row's input and its expected output

<scorer> is a scorer id: builtin:<slug>, or a project scorer sc_…. On experiment rows, sc_… matches the scorer at any version; on traces, a project scorer is written with its version, sc_…@2.

Field names are case-insensitive; the <scorer> part keeps its case. A field that doesn't apply (marked —) has no value, so only != matches it.

Examples

Filter	Finds
score.factuality < 0.5	Rows (or experiments) whose factuality is below 0.5
delta.builtin:task_done <= -0.25	Rows that got clearly worse than the baseline
output ~ "not found"	Outputs that mention "not found"
model != 'gpt-4.1'	Traces that never called gpt-4.1
tool = refund_create AND status = error	Traces where the refund tool was called and the trace errored
(tag = billing OR tag = refunds) AND score.correctness >= 0.8	Rows tagged billing or refunds that scored well
condition = attack AND score.builtin:attack_success > 0	Attacks that got through
environment = prod AND score.builtin:correctness < 0.5	Production traces online scoring marked low

Errors

A filter that doesn't read is underlined on the web where it goes wrong. The API answers 400 bad_filter, with the 0-based character position. For score.factuality ~ high:

{ "error": { "code": "bad_filter", "message": "The filter isn't right: score is a number: \"~\" compares words (at 17)", "position": 17 } }

A filter longer than 2,000 characters is 400 invalid_body. The list of datasets and the list of scorers accept filter= but don't apply it yet.