Sigil: Experiment Harness

• initializing a development harness for a spell or sigil,
• selecting the next missing example prompt,
• running one bounded native skill/subagent or explicit legacy runtime example,
• running a live stability loop for one regime,
• validating fixtures, expected outputs, live outputs, and reports,
• writing a run report,
• emitting signal-observer-compatible telemetry from a run report,
• making the same harness pattern usable in external repositories.

• mode: init, next, run, loop, validate, report, or observe,
• artifact path,
• artifact type for init: spell or sigil,
• optional profile for init: generic-spell, spellcraft, generic-sigil, or sigil-development,
• optional example ID or --all,
• optional RERUN=1 when overwriting existing example outputs is intentional,
• optional legacy runtime binary such as CODEX_BIN when explicitly testing a CLI adapter.

• <quality-bar> defines the acceptance criteria that classify the output as pass, partial, fail, or not_checked.
• <anti-patterns> defines known false-success boundaries that become anti_pattern_hits.
• The first implementation uses structured section and keyword checks; semantic judging can be layered into the observer later.
• Report machine fields must include QUALITY_BAR_STATUS, ANTI_PATTERN_HITS_JSON, and WORKFLOW_GAPS_JSON when findings exist.
• Profile-aware validation must emit PROFILE_ID, LIFECYCLE_OWNER, ARTIFACT_TYPE, CONTRACT_PATH, PROMPT_SET, REGIME_SET, and PROFILE_VALIDATION.

• reports become safe invocation envelopes,
• envelopes are appended to .arcanum/observability/signals/sigil-invocations.jsonl,
• observer hook activity is recorded under .arcanum/observability/hooks/,
• per-sigil and per-capability lookup indexes are rebuilt from the central ledger,
• reflection counters are updated in .arcanum/observability/reflection-state.json,
• configured reflection thresholds are evaluated during observation and emitted as usage-threshold, output-threshold, gap-threshold, or severe-gap with recommendation reflect-now,
• dedupe prevents repeated observer emissions for the same report and observer version,
• telemetry write failures never block the primary validation result.

• live Codex regimes are primary promotion evidence,
• deterministic fixtures remain controls,
• a loop passes after two consecutive successful attempts,
• failed attempts require robot-talks improvement reasoning before auto-improvement,
• improvements must be reversible and rolled back when the next attempt is worse.

See development/ARCHITECTURE.md and development/IMPLEMENTATION-LAYERING.md.

Legacy Codex CLI example execution is explicit adapter evidence only and uses this command shape:

codex exec \
  -C <repository-root> \
  --sandbox workspace-write \
  --output-last-message <artifact-folder>/development/example-runs/<run-id>.last-message.md \
  "$(cat <artifact-folder>/development/example-prompts/<task-id>.md)"

The runner then chooses the evidence body:

1. Prefer <artifact-folder>/development/example-outputs/<task-id>.output.md when the nested agent wrote a valid artifact result body there.
2. Otherwise copy the sidecar last-message file into the output path only when the sidecar is itself a valid artifact result body.
3. Block when both paths are empty, save-summaries, or lack a recognizable result heading.

Use CODEX_BIN when provided. Otherwise discover codex from PATH or known local extension paths.

• create the standard harness layout for new reusable spells and sigils,
• create and validate development/EXPERIMENT-PROFILE.md for every initialized harness,
• preserve existing harness files unless overwrite is explicit,
• select exactly one prompt for normal runs,
• require explicit --all for batch model calls,
• save the real artifact response rather than a save-summary,
• preserve artifact-file output when the nested agent writes it before returning a final summary,
• write raw run logs and timestamped reports,
• emit one observer-compatible telemetry event when repository observability is available,
• report pass, flag, or block honestly,
• remain usable from external repositories through runtime command adapters.

• treating a well-written contract as validation evidence without examples,
• silently running every prompt,
• overwriting outputs without RERUN=1,
• making artifact-local wrappers authoritative over the canonical sigil,
• validating only markdown presence while ignoring output shape,
• accepting a legacy harness without profile metadata as valid,
• embedding invoke-specific assumptions in the generic harness.

## Experiment Harness Result

- Mode: init | next | run | loop | validate | report | observe
- Artifact: <path>
- Artifact type: spell | sigil | unknown
- Profile: <profile-id | unknown>
- Selection: <regime-id | task-id | none | not applicable>
- Output: <path | none | not applicable>
- Report: <path | none | not applicable>
- Validation: pass | flag | block | not run
- Profile validation: pass | flag | block | not run
- Observation: recorded | skipped | failed
- Next unrun: <task-id | none | unknown>