Evidence Ledger — the spine every auditor reads

Infrastructure skill, not an auditor. It produces the only structure the auditor skills are allowed to reason over, so they don't each re-read the PDF and hallucinate a different table and a different list of numbers. It emits no findings and no verdict — only artifact_manifest.json + claims.json. See references/integrity-forensics-contract.md §"The pipeline" (stages [1]–[2]).

Build the ledger for: $ARGUMENTS

🔁 Not verdict-bearing — but not a polling skill either. The deterministic backbone (Steps 1–2) is a pure function of the hashed sources: same source bytes → byte-identical claims.json. Re-run it only when the sources change, never on a wall-clock timer. The only non-deterministic part is the optional enrichment pass (Step 3), which is additive and skippable. Do not wrap this skill in /loop / /schedule / CronCreate; there is no verdict to re-fire and no external event to wait on.

Why this exists

Five language-model auditors each independently parsing a PDF = five different hallucinated tables and five different number lists, none reproducible — and the obvious dismissal, "an LLM grading another LLM's paper is just slop." The structural answer is one deterministic pass that turns the paper into:

• artifact_manifest.json — what was observable; this fixes the observability level L, the ceiling on every downstream finding's severity, and
• claims.json — a list of span-anchored, hashed, checkable claims (schemas/claims.schema.json).

Every downstream finding must cite a claim_id from this ledger and quote a verbatim span of it. No ledger claim → no finding (the single most important integrity rule of the repo, enforced again by tools/adjudicate_findings.py). That is what makes the difference between "a model said so" and "here is the exact sentence, its file, and its content hash" (DESIGN.md §2).

Role in the pipeline (what this skill does and does NOT do)

Stage	Skill / tool	Emits	Judges?
[1]–[2] ledger	evidence-ledger (this skill) + tools/build_manifest.py, tools/build_claim_ledger.py	artifact_manifest.json + claims.json	No. States what the paper says.
[3] auditors	consistency-audit, citation-forensics, baseline-comparison-audit, experiment-forensics	<skill>.findings.json (read the ledger; quote its spans)	Propose findings — not the verdict.
[3] surface	presentation-signals	capped-at-minor surface findings (auxiliary)	Never a standalone verdict.
[3] memo	adversarial-case-builder	an evidence-bound memo	No verdict weight.
[4] verdict	tools/adjudicate_findings.py	report.json + REPORT.md	Yes — the ONLY verdict, by fixed rules, no model in the loop.

This skill is stage [1]–[2] only. It states what the paper says, never whether it is right. The finding.schema.json skill enum technically lists evidence-ledger for completeness, but this skill never writes a finding object. If you came here for a PASS/FAIL, you want /anti-autoresearch (the orchestrator), not this skill.

Core principle

Deterministic first; the model may only add, never invent, never judge.

1. The numeric/citation/table backbone comes from code (tools/), not a model — that is what makes the whole pipeline reproducible and defensible.
2. The optional LLM pass is an additive claim-extractor: it may add a span-anchored semantic claim whose text_span is a verbatim substring of a hashed source file; it may never introduce a number, alter an extracted value, propose a finding, or assign a severity/verdict. The executor validates every added span (Step 4) and rejects anything it cannot locate. Its surface is deliberately broad — beyond method/scope/baseline it also captures theorem statements (with their assumptions), definitions, proof/derivation steps, equations, conclusions, the motivation span, and reproducibility-artifact references — but every one of those rides on an existing claims.schema.json type (scope/method/comparison/artifact_ref); enrichment widens what content is anchored, never the type vocabulary, the Step-4 gate, or the deterministic backbone.
3. No span → no claim. Every text_span in the ledger must be locatable in a hashed source. This is the same gate the adjudicator enforces on findings, applied one stage upstream.

Constants

• LEDGER_VERSION = 0.1 — stamped into claims.json by build_claim_ledger.py; never hand-edit.
• TAXONOMY_VERSION = 0.5 — the ledger is taxonomy-agnostic (it tags no pattern_id); patterns are applied post-hoc by auditors (references/hack-pattern-taxonomy.md, now 46 integrity patterns A–H + 13 AIS + 2 advisory). Never tag a claim with a pattern_id here. v0.5 migrated the pure-style patterns to the zero-weight AIS track; the ledger stays untagged, but its enrichment (Step 3) now surfaces the anchors those auditors quote.
• OBSERVABILITY = derived — L0 (PDF/text only) · L1 (LaTeX, no results) · L2 (repo + results). L3 is never emitted in v0 (we never promise reproduction). The rule is deterministic (references/observability-levels.md).
• EMITS_FINDINGS = false · EMITS_VERDICT = false — load-bearing. This skill produces a ledger, not judgments.
• DETECT_ONLY = true — never edits the audited paper; only reads sources and writes its own outputs (this is why Edit is absent from allowed-tools).
• ENRICH = true (default) — run the additive semantic pass (Step 3), which surfaces the semantic, proof/derivation, and structure spans the regex backbone misses (theorem statements, assumptions, definitions, proof steps, equations, conclusions, the motivation span, reproducibility-artifact references). Set false (or pass — enrich: false) to ship the deterministic backbone alone. Enrichment is non-blocking: if the Codex MCP is unavailable it is skipped and the deterministic ledger is the canonical output.
• REVIEWER (enrichment only) — model gpt-5.6-sol, model_reasoning_effort: xhigh, sandbox: read-only, different model family from the executor (references/reviewer-independence.md Layer 1). CONTEXT_POLICY = fresh: a new mcp__codex__codex thread per run, never mcp__codex__codex-reply. Told only source paths + the existing ledger, never the executor's opinions or any prior finding.
• OUTPUTS — artifact_manifest.json, claims.json (+ enrichment trace under .aris/traces/evidence-ledger/<date>_run<NN>/ when Step 3 runs), all written into the paper directory.

Resolve the repo root once and reuse it for every tool call: ROOT=$(git rev-parse --show-toplevel 2>/dev/null || pwd). Shell variables do not persist between separate Bash calls, so Step 0 persists the resolved values to a run.env; every later block re-sources it (or run a Step's commands in one Bash call). Always use absolute paths.

---

Step 0 — Resolve the input & set up the run

$ARGUMENTS is a paper directory, a PDF path, or an arXiv id. Resolve it to one absolute PAPER_DIR + a stable PAPER_ID, extract PDF text if there is no LaTeX, and persist the values. Never fabricate inputs; if no source text can be produced, stop (a fake claims.json is worse than none).

ROOT=$(git rev-parse --show-toplevel 2>/dev/null || pwd)
ARG="$ARGUMENTS"

if [ -d "$ARG" ]; then                                   # (a) paper directory
    PAPER_DIR="$(cd "$ARG" && pwd)"; PAPER_ID="$(basename "$PAPER_DIR")"
elif [ -f "$ARG" ] && printf '%s' "$ARG" | grep -qiE '\.pdf$'; then   # (b) lone PDF
    PAPER_ID="$(basename "${ARG%.*}")"
    PAPER_DIR="$(pwd)/.aa_work/$PAPER_ID"; mkdir -p "$PAPER_DIR"; cp "$ARG" "$PAPER_DIR/paper.pdf"
elif printf '%s' "$ARG" | grep -qE '^[0-9]{4}\.[0-9]{4,5}(v[0-9]+)?$'; then   # (c) arXiv id
    PAPER_ID="${ARG//./_}"
    PAPER_DIR="$(pwd)/.aa_work/$PAPER_ID"; mkdir -p "$PAPER_DIR"
    ( curl -fsSL "https://arxiv.org/e-print/$ARG" -o "$PAPER_DIR/src.tar" \
        && tar -xf "$PAPER_DIR/src.tar" -C "$PAPER_DIR" 2>/dev/null ) \
      || curl -fsSL "https://arxiv.org/pdf/$ARG.pdf" -o "$PAPER_DIR/paper.pdf"   # source preferred (better spans)
else
    echo "ERROR: cannot resolve '$ARG' (need a dir, a .pdf path, or an arXiv id like 2401.01234)"; exit 1
fi

# No LaTeX? Extract PDF text now (best spans available at L0). -layout preserves table columns.
if ! find "$PAPER_DIR" -name '*.tex' -not -path '*/.aris/*' | grep -q .; then
    # Deterministic primary-PDF pick (issue #11): asset-looking PDFs (figures/,
    # images/, supplement*/ dirs or fig-like basenames) demoted, paper-like basenames
    # promoted. A dir holding ONLY figure/supplement PDFs yields NO pick (exit 1) —
    # the run then stops honestly at "no source text" instead of building the whole
    # ledger from a figure PDF. Only exit 1 is tolerated; a selector CRASH must not
    # silently degrade the run to the stale-.txt fallback.
    PDF=$(python3 "$ROOT/tools/select_primary_pdf.py" "$PAPER_DIR"); rc=$?
    [ "$rc" -le 1 ] || { echo "ERROR: select_primary_pdf.py failed (rc=$rc)"; exit 1; }
    [ -n "$PDF" ] && { pdftotext -layout "$PDF" "$PAPER_DIR/paper.txt" 2>/dev/null \
      || mutool draw -F txt -o "$PAPER_DIR/paper.txt" "$PDF" 2>/dev/null \
      || python3 -c 'import sys,fitz;open(sys.argv[2],"w").write("\n".join(p.get_text() for p in fitz.open(sys.argv[1])))' "$PDF" "$PAPER_DIR/paper.txt" 2>/dev/null; }
    # extraction yielded nothing but a pre-extracted *.txt exists? adopt it as the L0 source
    if [ ! -s "$PAPER_DIR/paper.txt" ]; then
        TXT=$(find "$PAPER_DIR" -maxdepth 2 -name '*.txt' -not -path '*/.aris/*' ! -name paper.txt | head -n1)
        [ -n "$TXT" ] && cp "$TXT" "$PAPER_DIR/paper.txt"
    fi
fi

mkdir -p "$PAPER_DIR/.aris/evidence-ledger"
cat > "$PAPER_DIR/.aris/evidence-ledger/run.env" <<EOF
ROOT="$ROOT"
PAPER_DIR="$PAPER_DIR"
PAPER_ID="$PAPER_ID"
EOF
echo "PAPER_DIR=$PAPER_DIR  PAPER_ID=$PAPER_ID  ROOT=$ROOT"
find "$PAPER_DIR" \( -name '*.tex' -o -name '*.pdf' -o -name '*.txt' \) -not -path '*/.aris/*' | sort

Every later Bash block begins with source "<PAPER_DIR>/.aris/evidence-ledger/run.env" — substitute the absolute PAPER_DIR printed above.

Validation gate. "$PAPER_DIR" must now contain at least one *.tex or a non-empty paper.txt (the exact L0 source Step 2 Branch B reads). A bare *.pdf whose text never extracted is not enough — there are no spans to anchor.

Failure handling.

• No *.tex and no non-empty paper.txt (e.g. a PDF every extractor failed on) → STOP: report exactly what was searched; with no source text there are no spans to anchor, so there can be no ledger.
• arXiv curl failed (network/proxy) → STOP: report the exit and ask the caller for a local paper-dir or PDF. Do not fabricate a ledger.
• A near-empty / garbled paper.txt (scanned image, heavy math) — concretely, wc -c < "$PAPER_DIR/paper.txt" is implausibly small for the page count (rule of thumb: under ~1000 bytes for a multi-page paper) or is mostly non-alphanumeric → say so explicitly and treat the run as L0 with confidence: low throughout; do not silently proceed as if you had clean text.

Step 1 — Artifact manifest + observability level (deterministic)

Inventory what is available and derive L by the fixed rule — a tool, not a manual judgment. The level caps every downstream finding's severity, so it must be honest.

source "<PAPER_DIR>/.aris/evidence-ledger/run.env"
MAN_ARGS=(--paper-id "$PAPER_ID" --dir "$PAPER_DIR" --out "$PAPER_DIR/artifact_manifest.json")
[ -f "$PAPER_DIR/paper.txt" ] && MAN_ARGS+=(--pdf-text "$PAPER_DIR/paper.txt")
python3 "$ROOT/tools/build_manifest.py" "${MAN_ARGS[@]}"
# -> manifest: observability L1 (latex=1 pdf=0 bib=0 repo=False results=False) -> .../artifact_manifest.json

L=$(python3 -c 'import json,sys;print(json.load(open(sys.argv[1]))["observability_level"])' "$PAPER_DIR/artifact_manifest.json")
case "$L" in 0|1|2) : ;; *) echo "ERROR: unexpected level '$L' (v0 operates only at L0/L1/L2)"; exit 1;; esac
echo "L=$L" >> "$PAPER_DIR/.aris/evidence-ledger/run.env"   # carry L forward to Step 2
echo "observability level = L$L"

The derivation rule (references/observability-levels.md; build_manifest.py implements exactly this):

Present	Level	Means
repo (code/,src/,repo/ or loose *.py/*.ipynb) and results (*.json/*.csv under results/,outputs/,logs/)	L2	code + results checkable: fake GT, self-norm, phantom results, paper↔result match
LaTeX present, no results	L1	source-level checks on stable spans (file:line, real cells, real .bib)
PDF / text only	L0	internal self-consistency, arithmetic, citation existence/context only

Validation gate — never over-state the level. If you only have a PDF, L MUST be 0; do not hand --observability-level 2 to Step 2 because a repo "exists somewhere else." build_manifest.py never sets repo.rerunnable: true (no L3 in v0) — do not edit it to true. Edge cases the rule handles correctly (L2 requires both a repo and result data files): an empty results/ dir (no *.json/*.csv) does not reach L2, and a repo present without result files does not reach L2 either — each stays at whatever the source gives (L1 if LaTeX is present, else L0).

Failure handling. build_manifest.py non-zero exit or empty/invalid JSON → STOP: without a derived level you cannot legally cap severity downstream; do not guess a level.

Step 2 — Extract the deterministic ledger (no LLM)

The numeric/citation/table backbone comes from code, not a model. LaTeX-first (stable spans + real line numbers); the PDF-text path is a lower-confidence fallback. Pass the same L derived in Step 1.

Pick the branch: if any *.tex exist use Branch A; otherwise use Branch B. The TEX array below makes that test explicit and survives spaces in paths.

Branch A — LaTeX present (L1/L2; preferred):

source "<PAPER_DIR>/.aris/evidence-ledger/run.env"
TEX=()   # space-safe + deterministic: one path per line, sorted for reproducible order
while IFS= read -r f; do [ -n "$f" ] && TEX+=("$f"); done \
  < <(find "$PAPER_DIR" -type f -name '*.tex' -not -path '*/.aris/*' | LC_ALL=C sort)
[ ${#TEX[@]} -gt 0 ] || { echo "no .tex found — use Branch B"; exit 1; }
python3 "$ROOT/tools/build_claim_ledger.py" --paper-id "$PAPER_ID" \
    --latex "${TEX[@]}" \
    --observability-level "$L" \
    --out "$PAPER_DIR/claims.json"
# -> ledger: 13 claims {'caption': 1, 'citation': 3, 'number': 7, 'table_cell': 2} -> .../claims.json

Branch B — no LaTeX (text-only spans; paper.txt from Step 0). Usually L0, but L2 when a repo + result files exist without any .tex, so pass the derived $L — never a hardcoded 0 (the gate below asserts the ledger level equals the manifest $L):

source "<PAPER_DIR>/.aris/evidence-ledger/run.env"
test -s "$PAPER_DIR/paper.txt" || { echo "ERROR: no extracted text"; exit 1; }
python3 "$ROOT/tools/build_claim_ledger.py" --paper-id "$PAPER_ID" \
    --pdf-text "$PAPER_DIR/paper.txt" --observability-level "$L" \
    --out "$PAPER_DIR/claims.json"

What it extracts (each claim carries claim_id, type, verbatim text_span, location{file,line,section}, evidence_anchor = sha256 of the source text, extractor, confidence; numeric claims also carry a parsed value):

type	From	extractor	confidence
table_cell	numbers inside tabular	table_parser	medium
number	numeric prose (%, points, x, or near a metric word)	latex_regex	high
citation	\cite{...} (keys in refs[])	latex_regex	high
scope	scope/SOTA language (comprehensive, robust, outperform…, first to, …)	latex_regex	high
caption	\caption{...}	latex_regex	medium
(PDF path) number / scope	sentence text from paper.txt	pdf_text	low

The section tracker labels each span abstract | intro | method | experiments | table:N | figure:N | appendix | body (section names lowercased from \section{...}); the PDF-text path cannot track sections or lines and labels them unknown. Numeric claims carry value{raw,normalized,unit,metric,direction,aggregation} so the deterministic layer can do arithmetic (delta/coherence checks) downstream without a model. Omit --generated-at for byte-reproducible output (the eval harness does this); pass --generated-at "$(date -u +%Y-%m-%dT%H:%M:%SZ)" for run provenance — the claims are identical either way.

Worked example (clean fixture eval/fixtures/clean/sample_paper.tex as main.tex) — the stdout above, then two real claims (location.file mirrors the path you pass to --latex):

{ "claim_id": "C001", "type": "table_cell",
  "text_span": "Baseline \\cite{smith2024bar} & 73.1 \\\\",
  "location": {"file": "main.tex", "line": 36, "section": "table:1"},
  "value": {"raw":"73.1","normalized":73.1,"unit":null,"metric":null,"direction":"unknown","aggregation":"unspecified"},
  "evidence_anchor": "e6186efa…0460", "extractor": "table_parser", "confidence": "medium" }

{ "claim_id": "C003", "type": "number",
  "text_span": "FooNet reaches 78.0\\% accuracy, improving from a 73.1\\% baseline to 78.0\\% accuracy, a 6.7\\% relative improvement.",
  "location": {"file": "main.tex", "line": 9, "section": "abstract"},
  "value": {"raw":"78.0","normalized":78.0,"unit":"%","metric":"accuracy","direction":"unknown","aggregation":"unspecified"},
  "evidence_anchor": "e6186efa…0460", "extractor": "latex_regex", "confidence": "high" }

The ledger states, it does not judge. Run the extractor on the corrupted eval/fixtures/synthetic_corruptions/delta_inflate.tex (abstract says "16.7% relative improvement") and you get an identical 13-claim shape — only C003's verbatim text changes. Spotting that 16.7% contradicts 73.1→78.0 is consistency-audit's job (HP-DELTA-ERROR), not the ledger's; the ledger just captures the span faithfully.

Validation gate. Confirm the ledger is well-formed, the level matches, and report the claim mix:

source "<PAPER_DIR>/.aris/evidence-ledger/run.env"
python3 - "$PAPER_DIR/claims.json" "$L" <<'PY'
import json,sys
d=json.load(open(sys.argv[1],encoding="utf-8")); L=int(sys.argv[2])
for k in ("ledger_version","paper_id","observability_level","source_files","claims"):
    assert k in d, f"ledger missing top-level key: {k}"
assert d["observability_level"]==L, f"level drift: ledger={d['observability_level']} manifest={L}"
assert d["source_files"] and all(s.get("sha256") for s in d["source_files"]), "missing source hash"
for c in d["claims"]:
    assert {"claim_id","type","text_span","location"} <= c.keys(), f"claim {c.get('claim_id')} missing field"
    assert c["location"].get("file"), f"claim {c['claim_id']} has no location.file"
by={t:sum(1 for c in d["claims"] if c["type"]==t) for t in sorted({c['type'] for c in d['claims']})}
print(f"OK ledger L{L}: {len(d['claims'])} claims {by}")
PY

Failure handling.

• build_claim_ledger.py errors with "provide at least one --latex or --pdf-text" → your file glob matched nothing; re-check the branch (use Branch B when no *.tex were found, i.e. the TEX array is empty).
• 0 claims on a paper that visibly has numbers/citations → the wrong files were passed or the .tex is a stub. Re-inspect inputs (read the head of the .tex/.txt) and re-run; do not fabricate claims. A genuinely claim-free paper is rare — ship the empty ledger only after confirming inputs.
• A single malformed .tex can crash the extractor (build_claim_ledger.py has no per-file try/except) → drop that one path from the TEX array and re-run rather than abandoning the whole paper; note the dropped file.

Step 3 — Optional additive semantic enrichment (cross-model, fresh thread)

Skip entirely when ENRICH = false. This step adds claims; it never edits or removes a deterministic claim, never adds a number, never proposes a finding.

The regex backbone has high recall on the numeric/citation surface but misses the semantic, proof/derivation, and structure spans auditors need: the method-definition span, theorem statements with their assumptions, explicit scope sentences, the baseline list, comparison framings — and, for the families, the spans that families B (argument-chain / causal-leap), D (reproducibility) and G (proof & derivation) anchor to: definitions, proof / derivation steps, formulas / equations, stated assumptions, load-bearing conclusions, the motivation span, and reproducibility-artifact references (does the paper ship / promise code, prompts, configs?). These feed consistency-audit (HP-METHOD-DRIFT, HP-THEOREM-SCOPE-DRIFT, HP-ARGUMENT-CHAIN-BREAK, HP-CAUSAL-EVIDENCE-LEAP), experiment-forensics (HP-MISSING-REPRO-ARTIFACT), and proof-derivation-forensics (HP-PROOF-OBLIGATION-GAP, HP-PROOF-CIRCULARITY, HP-DERIVATION-INVALID, HP-SYMBOL-SEMANTIC-DRIFT, HP-ASSUMPTION-SMUGGLE). Make one cross-model call to add such span-anchored candidate claims. This is extraction help, not a review: the model proposes candidate claims (verbatim spans); the executor's deterministic substring gate (Step 4) decides what is admitted — nothing here is a finding or a verdict (references/reviewer-independence.md).

No new type vocabulary — broadened content on the existing schema types. Every new span rides on a claims.schema.json type the deterministic layer and Step 4 already allow, so the anti-hallucination gate is unchanged and claims.json stays schema-valid. The mapping — what new content the enrichment surfaces → which existing type carries it → which family/pattern anchors to it:

New span the enrichment surfaces	Carrying type	Anchors for (family · pattern)
theorem / lemma / proposition statement (incl. its stated assumptions)	scope	B · HP-THEOREM-SCOPE-DRIFT · G · HP-PROOF-OBLIGATION-GAP
stated assumption / hypothesis (standalone)	scope	G · HP-ASSUMPTION-SMUGGLE
definition of a symbol / operator / construct	method	G · HP-SYMBOL-SEMANTIC-DRIFT
proof step / derivation transition (symbolic)	method	G · HP-DERIVATION-INVALID, HP-PROOF-CIRCULARITY
formula / equation (symbolic, non-numeric)	method	G · HP-DERIVATION-INVALID, HP-SYMBOL-SEMANTIC-DRIFT
load-bearing conclusion (causal / equivalence / relational)	comparison	B · HP-CAUSAL-EVIDENCE-LEAP
the motivation / problem-framing span	scope	B · HP-ARGUMENT-CHAIN-BREAK
reproducibility-artifact reference (code / prompt / config present or "will release")	artifact_ref	D · HP-MISSING-REPRO-ARTIFACT

These are anchors, not findings. The ledger never says a proof is circular, an assumption is smuggled, a chain is broken, or an artifact is missing — it only captures the verbatim span so the family-B/D/G reviewer has a claim_id to quote. The judgment stays in the auditor; the verdict stays in tools/adjudicate_findings.py. Family-G recall is highest at L1: equation and theorem-statement spans carry stable line numbers, which lets proof-derivation-forensics scaffold per-theorem anchor candidates by line window — so the prompt below asks the reviewer to include line whenever it extracts from LaTeX.