Acceptance Criteria¶

Section 11 of the specification, and where each item is implemented and checked. A criterion with no test is a claim, not a property.

Criterion	Implementation	Checked by
The Judge implements exactly the Section 1.3 matrix; a `fail_closed_catch` can never be counted or exported as a bypass	`harness/judge.py`, `harness/status.py`	`tests/test_judge.py` (every status, both directions; unknown severity and unknown gap type raise `UnknownVerdictError`)
Every attempt runs against a restored, canary-verified database; `accumulate` campaigns record per-attempt hashes	`Environment.restore` is called before every TrustSight analysis; `Environment.check_canary` runs after every non-accumulate restore; `Trace.db_hash`	`tests/test_environment.py::test_every_restore_is_canary_verified`, `tests/test_environment.py::test_a_drifted_canary_aborts`
`config_fingerprint` is verified on every attempt	`Environment.check_fingerprint`, called per attempt in `harness/campaign.py` and `harness/regression.py`	`tests/test_environment.py::test_a_changed_fingerprint_aborts_mid_campaign`
`trustsight_version: "latest"` is rejected; the resolved version is recorded	`Environment.resolve`	`tests/test_environment.py::test_latest_is_not_a_version`
Forbidden techniques without checkers are rejected; violations are recorded	`validators/constraints.py`, `CampaignConfig`	`tests/test_validators.py::test_a_forbidden_technique_needs_a_checker`
The behavior validator's calibration suite gates fixture export; bypass rates are labelled lower bounds with Wilson intervals	`Exporter.__init__` refuses export unless calibration is `passed`; `harness/stats.py` computes Wilson intervals	`tests/test_pipeline.py::test_export_is_refused_without_calibration`, `tests/test_validators.py`
LLM campaigns refuse to start without `max_cost_usd` and abort at the ceiling with a complete record	`generators/llm.py`; `CostCeilingReached` ends the loop before the record is built	`tests/test_generators.py`
Traces contain only pipeline-produced data	`harness/recorder.py`; every trace field is copied from a measured object	inspection of `Trace.to_dict`
Deduplication and known-bypass recognition work across campaigns and versions, with `patch_status` recorded	`harness/dedup.py`, `Status.KNOWN_BYPASS_MATCH`, patch-verification logic in `harness/campaign.py`	`tests/test_pipeline.py`, `tests/test_regression_gate.py`
The regression gate replays committed bypasses plus canary and API/CLI parity checks; exit codes are 0/2 only	`harness/regression.py`, `harness/__main__.py`	`tests/test_regression_gate.py`
Harness self-security gates run in CI and fail the build	`tests/test_self_security.py`, `scripts/scan_secrets.py`, `.github/workflows/ci.yml`	the gates themselves and the CI workflow
The CLI and the MCP server expose the same operations and call the same implementations	`harness/surface.py` is the one list; the CLI dispatches every operation name and the MCP server registers a tool for each, both resolving to the function in `harness/mcp/server.py`	`tests/test_surface_parity.py`
`campaign.yml` is committed before execution; the record links the commit	`_git_commit` reads `.git` directly (no subprocess); `record.campaign_commit`	inspection of `record.json`
Every process the harness spawns runs in a kernel-enforced namespace and carries kernel resource limits; without unprivileged user namespaces it refuses by name	`harness/sandbox.py` (the one clone site) applies the namespace flags and `RLIMIT_AS/FSIZE/NOFILE/CPU`, `PR_SET_PDEATHSIG` and `PR_SET_NO_NEW_PRIVS`; `validators/syntax.py`, `harness/runner.py`, `harness/pool.py` exec into it	`tests/test_sandbox.py` (the child reports its own limits and namespaces), `tests/test_runner_worker.py`, `tests/test_pool.py`, `tests/test_self_security.py` (the clone site is the sandbox; the socket probe)
All previously known bypasses are reproducible from committed inputs	`campaigns/known-bypasses-manual/` (`inputs.yml`)	run the campaign; the migration gate re-renders every cell and requires the recorded `diff_sha256`
Untested rules are countable, not inferred; a probe cannot name a rule that does not exist	`harness/coverage.py`, `python -m harness coverage`	`tests/test_coverage.py`
A benign diff that is flagged is measured, and an unflagged one is not counted as a bypass	`harness/benign.py`, `python -m harness benign`	`tests/test_benign.py`
An untrusted `campaign.yml` cannot read outside its roots through the generator	`_within` in `harness/__main__.py`	`tests/test_mcp.py` (generator path escape)
A campaign can assert the federation IOC layer without touching the verdict	`prompt.expected_iocs`, `environment.ioc_baseline`, `Trace.trustsight["ioc_expectations"]`	`tests/test_config.py`, the `wave3-*` campaign traces
CI and the container image pin the same TrustSight commit	`.github/trustsight-commit`, read by `ci.yml` and `docker.yml`	the two workflows, which assert the checked-out SHA
The published image measures the pinned build and runs the tool	`Dockerfile`, `.github/workflows/docker.yml`	the workflow smoke test (`--help`, `coverage`, `regression`)

Where the validator stops¶

Two campaign inputs are discarded as behavior_lost while containing a live chain, and both are kept deliberately: a record that shows the instrument's limits is more useful than one that hides them.

/usr/bin/c?rl -s … | bash - the validator does not resolve globs, so it cannot prove c?rl names a fetch client. Refusing is correct: the alternative is guessing what a glob would have matched on a filesystem the harness never looks at.
A config written for a daemon (dnsmasq --conf-file=…) - the execution is performed later, by a program the validator does not model.

Both are recall failures, which Section 1.5 permits and which is why every published count is a lower bound. Precision failures would not be acceptable, and the calibration suite exists to catch them.