Getting started

This page qualifies the synthetic demo pack against the local fixture. The fixture is a script, not model weights. A baseline run records decision QUALIFIED. Python 3.11 or newer is required. An air-gapped install is a different procedure: Offline install.

Install from a checkout

python3 -m venv .venv
source .venv/bin/activate
pip install -e .

pip install -e ".[dev]" adds pytest. pip install 'etalon[typesafe]' adds the TypeSafe SDK. The demo pack does not declare a judge, so the base install is enough here. etalon --version prints etalon 0.0.1. python3 -m etalon is the same entry point when the etalon script is not on PATH.

Open a report without installing

The repository includes two finished bundles. Open either file in a browser:

examples/sample-run/report.html — QUALIFIED
examples/sample-run-broken/report.html — NOT_QUALIFIED

After install, etalon verify examples/sample-run recomputes that bundle’s hashes. The command prints bundle hashes match and signature: unsigned.

Run the local fixture

In one terminal:

python3 examples/mock_endpoint.py --port 8000

The process listens on 127.0.0.1 only and prints listening on http://127.0.0.1:8000/v1. Leave it running. In a second terminal, with the virtual environment active:

etalon pack validate examples/demo-pack

etalon qualify \
  --endpoint http://127.0.0.1:8000/v1 \
  --pack examples/demo-pack \
  --serving-config examples/demo-pack/serving.yaml \
  --system-prompt examples/demo-pack/system_prompt.txt \
  --model etalon-demo-baseline \
  --operator "Demo Operator" \
  --output runs/

qualify prints the run directory, the decision, the realised coverage, and the report path. Coverage is value-presence. It is not a cross-product of coverage cells. Exit 0 means the capture finished and a bundle was written. The decision may still be NOT_QUALIFIED or INDETERMINATE. Exit 2 means the endpoint did not complete the capture. A bundle is still written, and the decision is INDETERMINATE. Replace <run-id> below with the directory name qualify printed.

etalon inspect runs/<run-id>
etalon verify runs/<run-id>
etalon report runs/<run-id>

inspect prints the pack, the decision, coverage, critical-event count, valid_until, hash integrity, and each requirement row. verify checks the bundle bytes. report with no --output prints the existing report.html path and does not rewrite the bundle. Open that file in a browser.

Check whether the fingerprint is still current

check re-reads the endpoint and the inputs you pass. It does not re-score the corpus. Re-supply every recorded input. Exit 0 means each captured trigger value was supplied again and matches:

etalon check runs/<run-id> \
  --endpoint http://127.0.0.1:8000/v1 \
  --pack examples/demo-pack \
  --serving-config examples/demo-pack/serving.yaml \
  --system-prompt examples/demo-pack/system_prompt.txt \
  --model etalon-demo-baseline \
  --operator "Demo Operator"

etalon check runs/<run-id> --endpoint http://127.0.0.1:8000/v1 alone does not exit 0. Recorded trigger values that were not supplied again are not re-observed, and that changes the exit by the trigger’s severity. Exit codes and the other flags are in the CLI reference.

Other fixture models

--model selects a scripted profile. The repository README lists each model id and the decision it produces. Examples:

Model id	Decision on a completed capture
`etalon-demo-baseline`	`QUALIFIED`
`etalon-demo-quantized`	`NOT_QUALIFIED`
`etalon-demo-error`	`INDETERMINATE` (`endpoint_errors`), and `qualify` exits 2

Contact-routing fixture

examples/contact-routing is a second public fixture. It adds a TypeSafe judge. Start the same mock process with --pack examples/contact-routing, install etalon[typesafe], and export TYPESAFE_API_KEY before qualify. Without the key the capture can still finish and the decision is INDETERMINATE. See Judges and TypeSafe judge.

Start from an installed wheel

pip install etalon
etalon pack init ./my-demo --from demo
etalon pack validate ./my-demo
etalon qualify --pack ./my-demo --endpoint http://localhost:11434/v1 \
  --model YOUR_LOCAL_MODEL --system-prompt ./my-demo/system_prompt.txt \
  --operator "Local evaluator" --output ./runs --max-tokens 2048

Choose the token budget and any --extra-body controls for the deployment you intend to test. serving.yaml in the demo describes a mock endpoint; do not use it to describe Ollama or a real serving stack. Missing weight identity is reported honestly. The demo is deterministic and needs no judge or external service.

Progress and interrupted captures

During capture, the CLI prints progress at least every five seconds, including elapsed time and an approximate ETA once responses arrive. A slow first request still produces a heartbeat. A separate RUN_ID.inprogress/ directory contains a fingerprint, state, and an fsynced captures.jsonl journal. Each completed response is appended before progress advances, with the existing secret scrubber. The journal contains case data and belongs inside your deployment boundary. It is a recovery checkpoint, not finalized qualification evidence. On success, RUN_ID/ contains the normal verifiable bundle and the journal is marked finalized. Interrupted checkpoints are retained. Automatic resume is not implemented; a fresh run needs a new run ID. Do not combine captures across configurations.

​Getting started

​Install from a checkout

​Open a report without installing

​Run the local fixture

​Check whether the fingerprint is still current

​Other fixture models

​Contact-routing fixture

​Start from an installed wheel

​Progress and interrupted captures