Skip to content

Campaign Configuration

campaign.yml is the whole configuration of a campaign. It is validated strictly: an unknown top-level key is a ConfigError, not a comment. The file is committed before the run, so attempts, ceiling and stop conditions are pre-registered and post-hoc attempt-count shopping is visible in git history.

A complete example

campaign: fetch-evasion-2026-08
campaign_type: stochastic

environment:
  trustsight_version: "0.17.4"
  trustsight_source: "local-path"
  db_state: "cold"
  flag_threshold: 20

generator:
  type: llm
  provider: kimi
  model: kimi-k3
  max_cost_usd: 20.00

prompt:
  prompt_id: fetch-then-execute-v3
  text: |
    …
  behavior_goal: fetch_then_execute
  expected_rules: ["R001", "R002", "X009"]
  forbidden_techniques:
    direct_curl: '(?:\A|[;&|]|\n)\s*curl\b'

attempts: 200

stop_conditions:
  bypasses: 10
  wall_clock_seconds: 7200

Top-level keys

Key Type Required Meaning
campaign string yes The campaign's name; appears in the record and in fixture provenance.
campaign_type deterministic | stochastic | rule-probe yes How the results may be compared. Anything else is refused. A rule-probe asks whether the expected rules fire, not whether a chain survived.
package string no The package name TrustSight is asked to analyse. Defaults to harness-pkg.
environment mapping yes See Record Schema → environment.
generator mapping yes See Writing a Campaign → generators.
prompt mapping no* The behaviour goal, expected rules and constraints.
attempts integer yes The upper bound on attempts.
stop_conditions mapping no Pre-registered early stops.
archive mapping no The shape archive's knobs. Defaults from the specification.

* prompt defaults to {} when omitted, but prompt.forbidden_techniques is required inside it, so prompt is effectively mandatory. Omitting both prompt and forbidden_techniques produces a configuration error.

package is the name the API is told the PKGBUILD belongs to; it is not read from the recipe's own pkgname. Most rules do not care, but some key on the name, such as C011, which applies only to a -bin package. A campaign probing one of those declares the name here.

prompt.expected_iocs asserts the federation IOC layer, which is separate from the verdict: an IOC match never changes the score. It needs the entries to be imported, so declare environment.ioc_baseline as well; the harness imports that directory (unsigned) into every restored database before the attempts run.

prompt

Key Type Required Meaning
prompt_id string no A stable label for the prompt; recorded alongside its hash.
text string for LLM The prompt itself. Its SHA-256 goes in the record.
behavior_goal string no Defaults to fetch_then_execute, the only goal implemented.
expected_rules list of strings no The rule IDs the campaign set out to test.
expected_iocs list of mappings no Federation IOC entries (type/value) the campaign expects to match, asserted against ioc_matches.
forbidden_techniques mapping yes Technique name → regex.

forbidden_techniques is mandatory, even when empty

forbidden_techniques: {}

An unconstrained campaign has to say so. Leaving the key out would let a reader infer whichever answer suits them, and the record is the only thing anyone will read later.

A technique must map to a checker. A name with an empty or missing pattern is a CheckerError, because a forbidden technique nobody can check is a promise the record cannot keep. Patterns are compiled with re.MULTILINE and matched against the new PKGBUILD text before TrustSight runs.

Why constraints exist at all

A campaign that forbids curl in command position and gets a bypass built on curl has not measured evasion; it has measured a model ignoring its prompt. Constraint checking makes that a recorded outcome ( constraint_violation) instead of a silent contaminant.

generator

type Keys Meaning
inputs manifest (default inputs.yml) Strict recipes manifest, one cell per attempt. The deterministic default.
raw-diff manifest (default inputs.yml) Committed unified diffs, one cell per attempt. For rules that read a second file, which a recipe diff cannot express.
mutation sources, seed, operators Semantic-preserving variations of committed diffs.
llm provider/model/max_cost_usd, … OpenAI-compatible provider with a mandatory ceiling.

An inputs manifest has three keys:

Key Type Meaning
cells list Required. One cell per attempt, in order.
template path Optional template.PKGBUILD under the campaign root; cells then carry vars.
baseline path Optional baseline under the repository root; defaults to defaults/baseline.PKGBUILD.

Each cell is {id, text} (whole recipe) or, with a template, {id, vars}. text and vars values are data: rendering is a single-pass literal substitution of @@name@@ tokens, with no expression engine. The loader refuses a missing or unused placeholder, an unknown key, a duplicate id, a path escaping its root, or a manifest past 2 MiB.

stop_conditions

Key Type Meaning
bypasses integer Stop once this many bypasses have been recorded.
wall_clock_seconds integer Stop once the campaign has run this long.

Both are checked at the top of each attempt, and the reason is written to record.stop_reason. Declaring them in the committed file is the point: an early stop is then a decision in git history rather than a judgement made while watching results.

archive

The shape archive measures a campaign by coverage - unique (rule, shape) cells filled - not by bypass volume. These keys are optional and default to the specification's values.

Key Type Default Meaning
minimize_budget integer 24 Analyses the minimizer may spend shrinking one bypass.
cell_saturation integer 3 Confirmed bypasses at one version that close a cell.
prompt_contract integer 2 The generator prompt version; 2 carries the open-cell map.

A cell closes once it has cell_saturation verified hits at one TrustSight version, or when the regression gate verifies a patch against it. Closed cells are rejected before analysis, so they consume no budget. A new TrustSight version reopens every cell, because a patch may have closed or moved one - that is the regression question.

Refusals

The loader refuses, with exit code 1, when:

Condition Message
Unknown top-level key unknown campaign keys: [...]
Missing campaign, environment, generator or attempts campaign.<key> is required
campaign_type is none of the three campaign_type must be 'deterministic', 'stochastic' or 'rule-probe'
forbidden_techniques absent declare {} explicitly for an unconstrained campaign
A forbidden technique has no pattern forbidden technique 'x' has no checker (checked at campaign start, not by the loader)
An LLM generator without max_cost_usd llm campaigns require generator.max_cost_usd
accumulate: true with campaign_type: deterministic accumulate makes verdicts order-dependent

That last one is worth reading twice. A campaign may model a warming database, but it cannot then call itself deterministic: attempt i teaches TrustSight the URL that attempt i+1 is about to be scored on, so the verdicts depend on run order even when the generator does not.