External harness adapter¶
iteron-eval exposes a public, serde-serializable adapter contract for harnesses that want to
evaluate an Iteron tunables profile without running iteron-evolve. The current contract admits
exactly terminal-bench version 2.1; a missing, newer, older, or differently formatted version
is refused.
Lifecycle¶
- Export Iteron's optimization surface and construct a valid profile outside the adapter.
- Create a bounded
TerminalBenchRequestwith exact task/trial identity, profile and registry digests, Iteron revision, absolute artifact paths, and time/turn/output/evidence/memory ceilings. - Call
TerminalBenchRequest::command(). Execute the returned program and argv with its cleared, fixed public environment. Credential names may be inherited directly by the launcher from the six built-in provider variables; values never enter the request, command document, or evidence envelope. - Bound stdout/stderr and wall time exactly as specified. Save stdout at
stdout_path, retain the emitted effective profile, and retrieve the run record named by Iteron's machine result. - Hash those bytes, have the external verifier write its score evidence, then serialize an
ExternalHarnessResult.parse_external_harness_resultrejects duplicate keys, oversized input, identity drift, invalid digests/paths, and evidence or timing beyond the request bounds.
The command uses Iteron's ordinary --tunables-profile plus
--tunables-profile-digest path. --emit-tunables-profile provides the exact effective-profile
artifact, while JSON stdout supplies the terminal result and run id. The adapter has no dependency
on iteron-evolve, cannot select a winner, and has no authority to promote a candidate.
Path validation is lexical and must occur before execution. The launcher is additionally responsible for opening/canonicalizing inputs and artifacts without following an unsafe symlink, and for verifying every recorded byte count and SHA-256 against the file it actually retained.
Passing contract tests or running a one-task smoke establishes adapter/surface acceptance only. It is not a Terminal-Bench campaign, qualification, leaderboard submission, or score claim. Those claims require the pinned upstream dataset, its prescribed trials/resources, external verifier evidence, and complete failed/timeout accounting.
Research implementation activation¶
The CLI also has a separate, operator-owned process-implementation path. It is intentionally
hidden from ordinary help and requires both
--implementation-candidate <PATH> --implementation-candidate-digest <64-lowercase-hex> with
--harness-profile research. The file is read once as a bounded regular file without symlinks and
is verified directly through the marketplace activation registry. This research path does not
consult HOME, ITERON_CONFIG_HOME, the installed plugin store, or composition winners. Every
source still has an absolute canonical catalog and artifact root, exact manifest and executable
digests, a regular executable, and capabilities intersected with the explicit CLI host ceiling.
Project configuration cannot name or activate an implementation.
Activation is atomic and every source resolves through the marketplace's verified launch-plan API.
For research, the paired hidden CLI arguments are the process-execution authorization; declared
decision authority is still intersected with the explicit host and caller ceilings. The ordinary
installed-plugin path remains separate: an implementation there must retain code_executing,
equal a composition-winning binding, and remain inside the winning plugin's fixed catalog/artifact
layout. All 28 optimization modules have an independent public module identity, lifecycle, and
consumption-evidence row. Nine typed production consumers execute these as deterministic ordered
module-stage chains; sharing a consumer does not merge implementation identity or permit one
implementation to stand in for a sibling module.
Each module stage launches a bounded direct child, loads it, starts one correlated run, consumes a terminal schema-checked decision, then stops and reaps it. A stage can return a typed decision or explicitly inherit the prior stage (the compiled baseline for the first stage). The decision's authority is intersected with the registry-minted plan, every prior stage, and the caller ceiling. Any lifecycle, protocol, schema, deadline, receipt, or identity failure returns the port's typed unknown/refusal decision; it never silently skips the failed stage.
Fresh genesis records the exact implementation, manifest, artifact, candidate, and activation
identity in the immutable bundle receipt. Resume requires the same candidate and reconstructs the
same receipt byte-for-byte; omission or drift is refused. Children inherit the same compiled slot
objects. Actual process consumption is reported at
<runs_dir>/.iteron-implementation-<bare-activation-sha256>-consumption.json with schema
iteron-implementation-consumption/1, the prefixed candidate digest, bare activation digest, CLI
run id, and ordered per-module loaded/started/terminal/stopped truth flags. Flags start false and
change only after the corresponding operation succeeds.
Neither path is a trainer or promotion protocol. They do not define reward, trajectory, checkpoint exchange, or authorize a candidate to select itself.