Release Benchmark Publishing

Use this skill only in the Odylith product repo when preparing a release or refreshing the public benchmark story in the repo-root README.md.

Do not use this skill in consumer repos.

Maintainer Mode

• Use pinned dogfood when proving the shipped benchmark and release story.
• Use detached source-local only when current unreleased src/odylith/* changes need to execute before you return to pinned dogfood for proof.
• Source-tree-only helpers such as the benchmark graph generator or source-tree pytest commands may run on the product repo toolchain with PYTHONPATH=src; that is a maintainer dev detail, not a consumer contract.

Goal

• Recover and maintain the governed release slice while refreshing the public benchmark story.
• Refresh the current published proof-host benchmark snapshot.
• Regenerate the canonical README benchmark SVGs.
• Update the repo-root README.md benchmark numbers and wording from the same benchmark report.
• Preserve the current benchmark graph style and tone across releases.
• Publish only the conservative --profile proof view for the current proof host, not the default quick lane or the most flattering single-profile report.
• Treat proof as the governing product benchmark and diagnostic as a secondary tuning surface. A diagnostic gain does not count if it hurts the live proof result.

Canonical Files

• Benchmark report: .odylith/runtime/odylith-benchmarks/latest.v1.json
• Release planning registry: odylith/radar/source/releases/releases.v1.json
• Release planning history: odylith/radar/source/releases/release-assignment-events.v1.jsonl
• README: README.md
• Publication snapshot writer: src/odylith/runtime/evaluation/odylith_benchmark_publication.py
• Graph generator: src/odylith/runtime/evaluation/odylith_benchmark_graphs.py
• Graph outputs:
  • docs/benchmarks/proof/odylith-benchmark-family-heatmap.svg
  • docs/benchmarks/proof/odylith-benchmark-quality-frontier.svg
  • docs/benchmarks/proof/odylith-benchmark-frontier.svg
  • docs/benchmarks/proof/odylith-benchmark-operating-posture.svg
  • docs/benchmarks/diagnostic/odylith-benchmark-family-heatmap.svg
  • docs/benchmarks/diagnostic/odylith-benchmark-quality-frontier.svg
  • docs/benchmarks/diagnostic/odylith-benchmark-frontier.svg
  • docs/benchmarks/diagnostic/odylith-benchmark-operating-posture.svg
• Maintainer rule: odylith/maintainer/agents-guidelines/RELEASE_BENCHMARKS.md

Canonical Flow

1. Recover the current governed slice with Odylith first: search the active workstream, bound plan, related bugs, related components, related diagrams, and recent Compass/session context; extend or reopen existing truth before creating new records. Treat repo-local release planning as part of that same governed slice: inspect odylith release list --repo-root . and odylith release show current --repo-root . when the benchmark story is preparing a release-bound candidate.
2. If the benchmark-publication slice is genuinely new, create the missing workstream and bound plan before non-trivial publishing changes; if the work spans multiple release lanes, split it with child workstreams or execution waves.
3. Reconcile release-planning truth for the candidate version before changing public copy:
   • ensure the candidate release record carries the correct version or tag
   • if the matching authored release note already exists, leave name empty to inherit the note title or set that exact same title
   • move any carry-forward workstreams to the follow-on release instead of leaving them attached to the shipped line in public-facing planning truth
4. Run the release-proof benchmark corpus: ./.odylith/bin/odylith benchmark --repo-root . --profile proof The default quick lane is developer signal only. Use proof so latest.v1.json covers both warm and cold for the live odylith_on versus odylith_off pair. If internal tuning needs isolated packet or prompt measurements, run --profile diagnostic separately and do not publish that as the product comparison. If the proof run needs to be split operationally, use deterministic --family or shard flags and publish only from the completed full proof report. For full-corpus shard splits, merge the shard history reports with PYTHONPATH=src python3 -m odylith.runtime.evaluation.odylith_benchmark_shard_merge --repo-root . <report-id>... before regenerating graphs or README copy. If maintainers activate a version-scoped benchmark override instead, stop the in-flight proof run, record the override in odylith/runtime/source/release-maintainer-overrides.v1.json, and update Casebook plus the bound release workstream and plan before continuing the release lane.
5. Refresh the benchmark snapshot docs, tracked latest-summary JSON, and the README-linked proof/ and diagnostic/ SVGs from that same selected report set: PYTHONPATH=src python -m odylith.runtime.evaluation.odylith_benchmark_publication --repo-root . This is the canonical path for:
   • docs/benchmarks/LIVE_BENCHMARK_SNAPSHOT.md
   • docs/benchmarks/GROUNDING_BENCHMARK_SNAPSHOT.md
   • docs/benchmarks/BENCHMARK_TABLES.md
   • docs/benchmarks/latest-summary.v1.json
   • docs/benchmarks/proof/*.svg
   • docs/benchmarks/diagnostic/*.svg Do not hand-edit those snapshot files when the publication writer can derive them from the validated report. The lower-level graph renderer remains available for explicit graph-only or single-report refreshes: PYTHONPATH=src python -m odylith.runtime.evaluation.odylith_benchmark_graphs --repo-root . --out-dir docs/benchmarks --profiles proof diagnostic
6. Update the repo-root README.md benchmark snapshot from that same report. Make the README explicit that the current published snapshot is Codex-host-scoped and conservative across the published cache profiles, that odylith_off is the raw host CLI lane, that raw_agent_baseline is only the internal report alias, and that odylith_repo_scan_baseline is only a secondary scaffold control kept in the tracked report rather than the headline public table. The README should also stay explicit that the public pair uses the same live proof-host CLI model and reasoning contract with an isolated temporary host home and stripped repo-authored guidance surfaces, that published timing is benchmark time to valid outcome rather than solo-user latency, and that published token cost is full-session spend rather than the initial prompt size. If the report came from detached source-local, say that explicitly and keep the first-release baseline warning visible until docs/benchmarks/release-baselines.v1.json records a shipped proof. When a proof slice is intentionally strict-bounded, keep that contract visible in the narrative: supplemental docs and implementation anchors may be suppressed when the truthful required surface is already the listed anchor set, and validator-only tests or generated artifacts do not count as approved first-pass reads unless a focused contradiction points directly there. If the closeout would benefit from naming Odylith directly, keep it to one short end-of-work Odylith Assist: line backed by measured proof, and follow the detailed wording contract in Odylith Chatter instead of expanding the benchmark lane's own prompt tax.
7. Keep the README graph order and tone unchanged unless the product is intentionally adding benchmark-marketing cuts from the same report:
   • family heatmap
   • quality frontier
   • frontier
   • operating posture
8. Keep the README graph block in this exact order:
   • odylith-benchmark-family-heatmap.svg
   • odylith-benchmark-quality-frontier.svg
   • odylith-benchmark-frontier.svg
   • odylith-benchmark-operating-posture.svg
9. If the benchmark slice surfaced a named failure mode, stale diagram, or thin component boundary, update Casebook, Atlas, Registry, and Compass in the same change.
10. Run: PYTHONPATH=src pytest -q tests/unit/runtime/test_odylith_benchmark_graphs.py

Non-Negotiables

• The README benchmark section must stay explicit about the current proof host.
• The README benchmark section must describe the conservative published view, not primary_comparison when they differ.
• Benchmark status semantics must stay explicit:
  • correctness, grounding, validation/fit, and robustness are the hard gate
  • tighter-budget behavior is the live-proof secondary guardrail
  • time to valid outcome and full-session token spend stay published as live-proof diagnostics, not solo-user or initial-prompt claims
  • prompt/payload efficiency thresholds belong to the diagnostic lane
  • packet coverage, widening frequency, route posture, and similar mechanism signals stay published as advisory context
  • current live-proof guardrail is within_budget_rate >= 0.80
• Benchmark publication does not waive Odylith governance upkeep: do not leave the workstream, plan, bug memory, component specs, diagrams, or Compass context stale after changing the public benchmark story.
• When the benchmark story is release-bound, do not let public benchmark copy drift from repo-local release planning truth. Candidate release naming, carried work, and current or next alias ownership must agree with the release note and the public launch framing.
• Do not substitute screenshots, ad hoc chart exports, or different filenames.
• Do not hand-edit the generated SVGs.
• Preserve the current beige/paper visual direction, red baseline marks, teal Odylith marks, and the current benchmark-graph narrative unless the product is making an intentional benchmark-style redesign.
• odylith_off in README, graphs, and review framing is the raw host CLI lane. raw_agent_baseline is only the internal report alias for that lane.
• odylith_repo_scan_baseline is the repo-scan scaffold control, not the honest no-scaffold baseline, and it should stay out of the primary public README table.
• The public pair must hold the same live proof-host CLI model and reasoning contract on both lanes, and it must run with an isolated temporary host home plus stripped repo-authored guidance surfaces instead of inheriting local skills, plugins, MCP config, or AGENTS.md state by accident.
• Post-run adoption-proof sampling is supplementary. It must be bounded and degrade cleanly on timeout or transport loss instead of blocking persistence of a completed full proof report.
• Any Odylith-by-name closeout or visible-intervention fallback note must stay evidence-backed, soulful, friendly, authentic, and factual, and it must point to concrete measured proof rather than generic product praise. Follow the canonical closeout contract in Odylith Chatter.
• If the style changes intentionally, update the generator, README, and graph tests in the same change.