> ## Documentation Index > Fetch the complete documentation index at: https://docs.etalon.sh/llms.txt > Use this file to discover all available pages before exploring further. # Overview > Decision, coverage, check, and compare rules on one page. # Etalon Operator steps, the command reference, and the public fixtures are in the [documentation index](/docs/introduction.md). This page states how a decision is reached. [How qualification works](/docs/how-qualification-works.md) follows the same rules through the files a run writes. Etalon produces qualification evidence for one self-hosted model deployment. A pack is the specification: corpus, rubrics, evaluators, thresholds, decision policy, and requalification triggers. The runner executes that pack inside the customer's network, records the system fingerprint with explicit metadata states, and writes an evidence bundle plus a static HTML report. The report says what was evaluated and under which recorded configuration. It does not determine that a regulation or internal control has been met. The packs checked in here are public fixtures only. `examples/demo-pack` is a synthetic public fixture for tests and the offline demo. `examples/contact-routing` is a second public fixture, with a judge overlay. See [Packs](/docs/packs.md). ## Layout ``` etalon/ runner package and JSON Schemas examples/demo-pack/ synthetic pack, no judge examples/contact-routing/ public fixture with a judge overlay examples/mock_endpoint.py examples/sample-run/ prebuilt QUALIFIED bundle schema -> etalon/schema ``` ## Decisions `QUALIFIED`, `NOT_QUALIFIED`, and `INDETERMINATE` come only from the pack policy. Proportion requirements use the 95% Wilson interval. A critical-failure maximum of zero fails on a single observed event. A completed capture with any hard miss is `NOT_QUALIFIED` even when another metric straddles its threshold. An incomplete capture is `INDETERMINATE` even when critical events were already counted. The decision summary states that critical-event count whenever it was produced. `INDETERMINATE` is also a normal outcome when the interval crosses a threshold, coverage is short, variance is high, a required evaluator did not score, or mandatory fingerprint fields are missing. Qualification stays current until a declared trigger changes. `valid_until` is that sentence, never a date. ## Judge and calibration A pack may declare a rubric judge. The runner records the judge model, the prompt hash, the prompt version, and the rubric version. Every judge verdict records uncertainty. A missing or unusable verdict is a judge failure and the run is `INDETERMINATE`. When the pack lists `judge_uncertainty`, uncertainty at or above the pack maximum is also `INDETERMINATE`, unless a hard miss on a completed capture is already `NOT_QUALIFIED`. That uncertainty is still written on the decision. `typesafe` (alias `jev`) is the TypeSafe Jev transport. Uncertainty there is the Noul probability that the verdict is uncertain. Choice confidence is evidence and does not fill a missing Noul. A live run needs `TYPESAFE_API_KEY`. See `docs/typesafe-judge.md`. Critical dimensions stay on reference evaluators. Calibration cases are not sent to the system under test. They carry a human score and a candidate output. The runner reports judge–referee agreement and the known-failure cases. Agreement below the declared record is `calibration_invalid`. Reference evaluators remain the qualification metrics. The report, `inspect`, and `compare` show the reference route and the judge together when a judge ran. Failure classes for this route are critical misroute, soft ambiguity, abstain miss, and judge failure. ## Coverage v0.1 realised coverage is value-presence. A declared dimension value counts when it appears at least once on a qualification case. The realised figure is the unweighted mean of those fractions. This version does not expand a cross-product of coverage cells, and it does not weight a value by how often it appears. `etalon qualify`, `etalon inspect`, `etalon compare`, and the HTML report state that limit next to the number. A pack that declares another coverage model is rejected. A fraction of 1 means every declared value appeared at least once on its own dimension. It does not mean every combination of values was measured. ## Check and compare `etalon check` prints a short legend for exit 0, 1, and 2, then three kinds of row: * fired — the re-supplied value differs from the stored projection * not re-observed — a captured value was not re-supplied; this changes the exit by that trigger's severity * unverified gaps — the stored field is `unavailable`; the row is printed at that trigger's severity and does not by itself change the exit A live value that was not re-supplied is `not_observed`. Each row prints the severity, the trigger name, the stored projection, and the live projection. An ambiguous fingerprint (a trigger field is missing, or an envelope contradicts itself) is listed on its own. It is not treated as an unavailable gap, and it prevents exit 0. `etalon compare` prints both decision statuses, each metric's point estimate and confidence interval, critical-event counts, and realised coverage, and it states that realised coverage is value-presence. Fingerprint fields that differ are mapped to `always_requalify`, `review_required`, or `no_material_impact`. The line `qualification evidence changed` is about the decision, the intervals, the critical-event count, and realised coverage. ## Signatures, gates, and review exports An operator can sign `hashes.json` with an Ed25519 seed held offline. `signature.json` sits beside the bundle and is not one of the hashed files. Unsigned bundles from earlier versions still verify. Verification with a separately held public key fails when the evidence bytes change, including when `hashes.json` is rewritten to match the changed bytes. `examples/verify_bundle.py` repeats that check without installing Etalon. A GitHub Action runs the same script and fails the job when the bundle is missing, altered, unsigned, or signed by a different key. `etalon export` writes an HTML review file and a manifest for a person. That package is qualification evidence. It does not determine that a regulation, contract, or internal control has been met. Key generation, custody, and the gate are described in `docs/signing.md`.