CLI reference
etalon is the console script. python3 -m etalon is the same program.
etalon --version prints etalon 0.0.1. etalon --help lists the subcommands.
etalon <command> --help. Behaviour that the help text does
not spell out is taken from the runner.
A command that fails before it can finish (a pack qualify will not load, a
missing file, or a bad value) prints the error to stderr and exits 1. Each
command below states the exit codes it uses when it does finish. Signature
custody is in Signing.
etalon pack init
etalon pack init PATH --from demo copies the bundled synthetic demo to a new
editable directory. An existing destination is rejected. No repository clone,
network request, or judge service is required.
etalon pack validate
path is the directory that contains pack.yaml.
On success the text report prints the pack label (
id@version),
content_sha256, qualification, calibration, and challenge case counts, and
result: pack matches the schema and pack rules. Exit 0 means the pack
matched. Exit 1 means it did not. Diagnostics are included in the text report
and in the JSON object.
etalon qualify
--output.
Required flags are --endpoint, --pack, --output, and --operator.
The client does not read proxy variables. A bearer token, when set, comes
from
ETALON_API_KEY or else OPENAI_API_KEY. There is no API-key flag.
On a finished command the process prints:
QUALIFIED, NOT_QUALIFIED, or INDETERMINATE. Exit 2 means one or
more requests failed after retries. A bundle is still written, the capture is
marked incomplete, and the decision is INDETERMINATE.
How the decision is chosen is in How qualification works.
etalon report
path is a run directory.
With no --output, the command prints <path>/report.html and, when that
file exists, the coverage-model line. It does not rewrite report.html,
report.json, or hashes.json. Exit 1 if report.html is missing.
--output writes a new HTML file from the bundle. The bundle is left
unchanged. --show-sensitive or --redaction-policy requires --output.
Without it the command exits 1 and prints that a derivative path is required.
etalon inspect
valid_until, whether hashes
match, signature status, and each requirement row. When metrics.json
records an executed judge, the summary adds the judge model, prompt hash,
rubric version, overlay agreement, calibration agreement, provenance, known
failures, uncertain cases, and overlay disagreements.
--json prints manifest, decision, integrity, and coverage_model.
Exit 0 when the bundle hashes match. Exit 1 when they do not. The summary is
still printed.
etalon verify
Exit 0 prints
bundle hashes match and a signature line (unsigned,
matches supplied public key, or matches the public key embedded in signature.json).
Exit 1 prints the errors. Unsigned bundles verify unless --public-key or
--require-signature is set. A signature that does not match the bytes of
hashes.json fails verification.
etalon keygen
--private is the seed file, created mode 0600.
--public is the public key file. Both are JSON. The command refuses to
overwrite an existing file. The seed is not encrypted. Keep it offline and
out of the bundle. Custody is described in Signing.
etalon sign
hashes.json in the run directory path. --key is
the private key JSON. The command refuses a bundle that fails hash,
fingerprint, or manifest checks. It writes signature.json next to the
bundle and replaces an existing signature file. Exit 0 prints the payload
name, payload hash, and public key.
etalon export
The package contains
auditor-report.html and manifest.json. The HTML
states the decision, fingerprint, coverage limit, judge and calibration
summary when a judge ran, hashes, and signature status. It does not copy raw
case inputs or outputs. The manifest includes the SHA-256 of the HTML file.
Exit 0 when the bundle integrity check passed. Exit 1 when it did not. The
package is still written and records the failure. The statement on the
package is qualification evidence for a person to read. It does not determine
that a regulation, contract, or internal control has been met.
etalon check
--endpoint is required.
The other flags are the inputs to re-supply. A flag you omit is not
re-observed.
Human output starts with the exit-code legend, then rows:
- fired — the re-supplied value differs
- not re-observed — a captured value was not re-supplied; this changes the exit by that trigger’s severity
- unverified gaps — the stored field is
unavailable; printed at the trigger severity; does not by itself change the exit
etalon check <run> --endpoint URL alone does not exit 0. Weight files are
never hashed as one blob. Pass --model-index with per-file hashes if you
have them. --json prints the same comparison as objects.
--extra-body replaces request_extras (use {} to clear). It does not merge
the way qualify --extra-body does, and {} on qualify does not wipe pack
extras. Check refuses bare --extra-body unless sampling is re-observed in
the same command (--pack, a serving config with a sampling mapping, or a
sampling flag). Serving-config request_extras may be top-level or under
sampling. See Packs.
A live value that was not re-supplied is printed as not_observed. Each row
prints the severity, the trigger name, the stored projection, and the live
projection.
etalon compare
always_requalify, review_required, or
no_material_impact. The line about whether qualification evidence changed
is the decision, the intervals, the critical-event count, and realised
coverage.
--json prints the same comparison, including each interval’s lower and
upper bounds and coverage_model. Exit 0 when the report is printed. A
missing bundle file exits 1.
Additional request controls and checkpoints
qualify --extra-body '{"reasoning_effort":"none"}' supplies backend-specific
JSON parameters, merged over pack request_extras by key. Reserved request
fields and credentials are rejected. Effective extras are fingerprinted and
compared as sampling parameters. check --extra-body replaces the extras object;
resupply the pack or full serving sampling configuration as well.
During capture, progress prints roughly every five seconds and completed
responses are fsynced to RUN_ID.inprogress/captures.jsonl. The checkpoint is
separate from the finalized evidence directory and remains after interruption.
It is not a qualification result. Automatic resume is not available yet.