> ## Documentation Index > Fetch the complete documentation index at: https://docs.etalon.sh/llms.txt > Use this file to discover all available pages before exploring further. # CLI reference > Subcommands and flags for the etalon command. # CLI reference `etalon` is the console script. `python3 -m etalon` is the same program. `etalon --version` prints `etalon 0.0.1`. `etalon --help` lists the subcommands. ```text theme={null} etalon [-h] [--version] {pack,qualify,report,inspect,verify,keygen,sign,export,check,compare} ... ``` Flags below match `etalon --help`. Behaviour that the help text does not spell out is taken from the runner. A command that fails before it can finish (a pack `qualify` will not load, a missing file, or a bad value) prints the error to stderr and exits 1. Each command below states the exit codes it uses when it does finish. Signature custody is in [Signing](/docs/signing.md). ## `etalon pack init` `etalon pack init PATH --from demo` copies the bundled synthetic demo to a new editable directory. An existing destination is rejected. No repository clone, network request, or judge service is required. ## `etalon pack validate` ```text theme={null} etalon pack validate [-h] [--json] path ``` Check a pack directory against the schema and the runner's pack rules. `path` is the directory that contains `pack.yaml`. | Flag | Meaning | | -------- | ----------------------------------------------- | | `--json` | Print a JSON object instead of the text report. | On success the text report prints the pack label (`id@version`), `content_sha256`, qualification, calibration, and challenge case counts, and `result: pack matches the schema and pack rules`. Exit 0 means the pack matched. Exit 1 means it did not. Diagnostics are included in the text report and in the JSON object. ## `etalon qualify` ```text theme={null} etalon qualify [-h] --endpoint ENDPOINT --pack PACK --output OUTPUT --operator OPERATOR [--serving-config SERVING_CONFIG] [--system-prompt SYSTEM_PROMPT] [--chat-template CHAT_TEMPLATE] [--model-index MODEL_INDEX] [--model MODEL] [--concurrency CONCURRENCY] [--max-retries MAX_RETRIES] [--timeout TIMEOUT] [--temperature TEMPERATURE] [--top-p TOP_P] [--seed SEED] [--max-tokens MAX_TOKENS] [--extra-body EXTRA_BODY] [--run-id RUN_ID] [--redaction-policy REDACTION_POLICY] [--show-sensitive] [--strip-reasoning] [--quiet] [--judge-endpoint JUDGE_ENDPOINT] [--judge-model JUDGE_MODEL] ``` Run a pack against an endpoint and write an evidence bundle under `--output`. Required flags are `--endpoint`, `--pack`, `--output`, and `--operator`. | Flag | Meaning | | ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--endpoint` | Base URL of the OpenAI-compatible endpoint. A value that ends in `/v1` is called at `/v1/chat/completions`. A value that already ends in `/chat/completions` is used as given. Any other value is joined with the pack interface path. | | `--pack` | Pack directory. | | `--output` | Directory that will contain `/`. | | `--operator` | Recorded on the manifest and the fingerprint. | | `--serving-config` | YAML file of engine, revision, quantization, and related serving fields. Omitted fields stay `unavailable`. | | `--system-prompt` | Prompt file. The fingerprint stores its hash. The text is sent as the system message. | | `--chat-template` | Template file. The fingerprint stores its hash. | | `--model-index` | JSON or YAML map of per-file weight hashes under `files` or `file_hashes`. Each digest is `sha256:` and 64 hex characters. | | `--model` | Model id sent on each request. If omitted, the endpoint must list exactly one model id. | | `--concurrency` | Parallel requests. Default `4`. | | `--max-retries` | Extra attempts after the first request, on a retryable HTTP status or a transport error. Default `2` (three attempts in total). | | `--timeout` | Per-request timeout in seconds. Default `60`. | | `--temperature`, `--top-p`, `--seed`, `--max-tokens` | Override the pack sampling values for this run. | | `--extra-body` | JSON object merged by key over pack `request_extras`. Nested objects are replaced, not deep-merged. `{}` leaves pack extras unchanged. | | `--run-id` | Run directory name. Refused if that directory already exists. | | `--redaction-policy` | YAML with `redact_sensitive_inputs`, `redact_sensitive_outputs`, `strip_reasoning`, and `placeholder`. This changes the report written into the bundle. `strip_reasoning: true` also replaces reasoning traces in `outputs.jsonl`. | | `--show-sensitive` | Turn case redaction off for the report written into the bundle. Secret exclusion still applies. | | `--strip-reasoning` | Replace `reasoning`, `reasoning_content`, and `thinking` in `outputs.jsonl` with `[reasoning-stripped]`. Secret exclusion still runs on the original text first. | | `--quiet` | Do not print capture progress or the sampling-mismatch warning. | | `--judge-endpoint` | OpenAI-compatible endpoint for a pack judge. Defaults to `--endpoint`. Used by the `endpoint` judge transport. | | `--judge-model` | Override the pack judge model id. A chat model name, or a TypeSafe id such as `jev-latest`. | The client does not read proxy variables. A bearer token, when set, comes from `ETALON_API_KEY` or else `OPENAI_API_KEY`. There is no API-key flag. On a finished command the process prints: ```text theme={null} run: / decision: QUALIFIED | NOT_QUALIFIED | INDETERMINATE coverage: (value-presence; not a cross-product of coverage cells) report: //report.html ``` Exit 0 means every request completed and a bundle was written. The decision may be `QUALIFIED`, `NOT_QUALIFIED`, or `INDETERMINATE`. Exit 2 means one or more requests failed after retries. A bundle is still written, the capture is marked incomplete, and the decision is `INDETERMINATE`. How the decision is chosen is in [How qualification works](/docs/how-qualification-works.md). ## `etalon report` ```text theme={null} etalon report [-h] [--output OUTPUT] [--redaction-policy REDACTION_POLICY] [--show-sensitive] path ``` `path` is a run directory. With no `--output`, the command prints `/report.html` and, when that file exists, the coverage-model line. It does not rewrite `report.html`, `report.json`, or `hashes.json`. Exit 1 if `report.html` is missing. `--output` writes a new HTML file from the bundle. The bundle is left unchanged. `--show-sensitive` or `--redaction-policy` requires `--output`. Without it the command exits 1 and prints that a derivative path is required. ## `etalon inspect` ```text theme={null} etalon inspect [-h] [--json] path ``` Print a bundle summary: run id, pack, decision, realised coverage with the value-presence limit, critical-event count, `valid_until`, whether hashes match, signature status, and each requirement row. When `metrics.json` records an executed judge, the summary adds the judge model, prompt hash, rubric version, overlay agreement, calibration agreement, provenance, known failures, uncertain cases, and overlay disagreements. `--json` prints `manifest`, `decision`, `integrity`, and `coverage_model`. Exit 0 when the bundle hashes match. Exit 1 when they do not. The summary is still printed. ## `etalon verify` ```text theme={null} etalon verify [-h] [--public-key PUBLIC_KEY] [--require-signature] [--json] path ``` Check bundle hashes, structure, the fingerprint, and any signature. | Flag | Meaning | | --------------------- | ------------------------------------------------------------------------------------------------ | | `--public-key` | Ed25519 public key JSON. Requires a matching signature. A file that contains a seed is rejected. | | `--require-signature` | Fail when `signature.json` is absent. | | `--json` | Print the verification object. | Exit 0 prints `bundle hashes match` and a signature line (`unsigned`, `matches supplied public key`, or `matches the public key embedded in signature.json`). Exit 1 prints the errors. Unsigned bundles verify unless `--public-key` or `--require-signature` is set. A signature that does not match the bytes of `hashes.json` fails verification. ## `etalon keygen` ```text theme={null} etalon keygen [-h] --private PRIVATE --public PUBLIC ``` Write an Ed25519 key pair. `--private` is the seed file, created mode `0600`. `--public` is the public key file. Both are JSON. The command refuses to overwrite an existing file. The seed is not encrypted. Keep it offline and out of the bundle. Custody is described in [Signing](/docs/signing.md). ## `etalon sign` ```text theme={null} etalon sign [-h] --key KEY path ``` Sign the raw bytes of `hashes.json` in the run directory `path`. `--key` is the private key JSON. The command refuses a bundle that fails hash, fingerprint, or manifest checks. It writes `signature.json` next to the bundle and replaces an existing signature file. Exit 0 prints the payload name, payload hash, and public key. ## `etalon export` ```text theme={null} etalon export [-h] --output OUTPUT [--public-key PUBLIC_KEY] path ``` Write a review package from a bundle. The bundle is not modified. | Flag | Meaning | | -------------- | --------------------------------------------------- | | `--output` | Directory to create the package in. Required. | | `--public-key` | Check the bundle signature against this public key. | The package contains `auditor-report.html` and `manifest.json`. The HTML states the decision, fingerprint, coverage limit, judge and calibration summary when a judge ran, hashes, and signature status. It does not copy raw case inputs or outputs. The manifest includes the SHA-256 of the HTML file. Exit 0 when the bundle integrity check passed. Exit 1 when it did not. The package is still written and records the failure. The statement on the package is qualification evidence for a person to read. It does not determine that a regulation, contract, or internal control has been met. ## `etalon check` ```text theme={null} etalon check [-h] --endpoint ENDPOINT [--pack PACK] [--serving-config SERVING_CONFIG] [--system-prompt SYSTEM_PROMPT] [--chat-template CHAT_TEMPLATE] [--model-index MODEL_INDEX] [--model MODEL] [--operator OPERATOR] [--temperature TEMPERATURE] [--top-p TOP_P] [--seed SEED] [--max-tokens MAX_TOKENS] [--extra-body EXTRA_BODY] [--json] path ``` Compare the bundle fingerprint with a live reading. `--endpoint` is required. The other flags are the inputs to re-supply. A flag you omit is not re-observed. Human output starts with the exit-code legend, then rows: * fired — the re-supplied value differs * not re-observed — a captured value was not re-supplied; this changes the exit by that trigger's severity * unverified gaps — the stored field is `unavailable`; printed at the trigger severity; does not by itself change the exit An ambiguous fingerprint (a mapped field is missing, or an envelope contradicts itself) is listed on its own. It is not an unavailable gap, and the check cannot exit 0. | Exit | Meaning | | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 0 | Every captured trigger value was re-supplied and matches. `no_material_impact` may still be listed. A stored `unavailable` gap does not by itself change this. | | 1 | A `review_required` trigger fired or its captured value was not re-supplied, and nothing forces 2. | | 2 | An `always_requalify` trigger fired or its captured value was not re-supplied. | `etalon check --endpoint URL` alone does not exit 0. Weight files are never hashed as one blob. Pass `--model-index` with per-file hashes if you have them. `--json` prints the same comparison as objects. `--extra-body` replaces `request_extras` (use `{}` to clear). It does not merge the way `qualify --extra-body` does, and `{}` on qualify does not wipe pack extras. Check refuses bare `--extra-body` unless sampling is re-observed in the same command (`--pack`, a serving config with a `sampling` mapping, or a sampling flag). Serving-config `request_extras` may be top-level or under `sampling`. See [Packs](/docs/packs.md). A live value that was not re-supplied is printed as `not_observed`. Each row prints the severity, the trigger name, the stored projection, and the live projection. ## `etalon compare` ```text theme={null} etalon compare [-h] [--json] left right ``` Compare two run directories. The text report prints both decision statuses, each metric's point estimate and confidence interval, critical-event counts, and realised coverage. The coverage line states value-presence. Fingerprint fields that differ are mapped to `always_requalify`, `review_required`, or `no_material_impact`. The line about whether qualification evidence changed is the decision, the intervals, the critical-event count, and realised coverage. `--json` prints the same comparison, including each interval's lower and upper bounds and `coverage_model`. Exit 0 when the report is printed. A missing bundle file exits 1. ## Additional request controls and checkpoints `qualify --extra-body '{"reasoning_effort":"none"}'` supplies backend-specific JSON parameters, merged over pack `request_extras` by key. Reserved request fields and credentials are rejected. Effective extras are fingerprinted and compared as sampling parameters. `check --extra-body` replaces the extras object; resupply the pack or full serving sampling configuration as well. During capture, progress prints roughly every five seconds and completed responses are fsynced to `RUN_ID.inprogress/captures.jsonl`. The checkpoint is separate from the finalized evidence directory and remains after interruption. It is not a qualification result. Automatic resume is not available yet.