prompt-miner

Mine the harness's own session history to learn which prompt traits produce the best sessions, and feed those learnings back into identity. This skill is the judgment layer on top of the deterministic mine-traces.mjs engine: the engine collects, scores, and ranks; this skill correlates prompt features against outcome, mines falsifiable markers, and proposes durable lessons for approval.

It is a cross-session, data-driven cousin of /retro. Where /retro reflects on the current conversation, /prompt-miner reflects on the corpus of past sessions across both harnesses.

disable-model-invocation: true suppresses auto-invocation only — the model will not fire this skill on its own. A user-typed /prompt-miner still runs the full body below, including the Step-3 LLM marker synthesis. There is no conflict: the deterministic engine produces an objective dataset; the LLM step interprets it.

Privacy contract

This skill reads real session transcripts, which can contain secrets and private content. The contract is non-negotiable:

• Default output is feature vectors + metadata only — never raw prompt text. The engine omits promptText unless --include-prompt-text is passed.
• --include-prompt-text applies a redaction pass (line-level token patterns + block-level key bodies) and prints a WARNING banner. Use it only when you must read the prompt wording, and never commit the result.
• All artifacts land in ephemeral scratch under $TMPDIR, outside the repo. Never stage, commit, or paste a transcript or an --include-prompt-text report.
• The engine never writes into the repository. Its only outputs are the scratch artifacts under $TMPDIR.

When to use

• /prompt-miner invoked to mine the corpus for prompt-quality markers.
• Periodically (the daily crons/prompt-miner.md cron drives the unattended path; this skill is the interactive path).

When NOT to use

• /retro — reflects on the current conversation, not the historical corpus. Use /retro to close a session; use /prompt-miner to learn across sessions.
• /audit context / /audit skills / /wiki lint — those score harness artifacts (context budget, skills, wiki). /prompt-miner scores prompts.

Result tag

Announce exactly one human result tag at the end of the run:

RESULT: MINING-COMPLETE | DRY-RUN | NO-SESSIONS | NO-CORPUS

Tag	Meaning
MINING-COMPLETE	The engine ran, markers were mined, and the propose-then-confirm gate ran.
DRY-RUN	--dry-run was passed: dataset computed and printed; nothing written; no gate.
NO-SESSIONS	The engine found zero in-window sessions (sessionsScanned == 0). Stop after Step 1.
NO-CORPUS	Sessions exist, but no session-type stratum reaches the sessions_supporting ≥ 10 floor (see references/markers.md). Report and stop before proposing markers.

NO-CORPUS is distinct from "no markers crossed the bar" (NO-CANDIDATE): the former means the corpus is too small to mine anything reliably.

Steps

Step 1 — Run the engine

Run the deterministic engine, forwarding the user's arguments verbatim. Use the array form so a --weights '{...}' JSON argument stays a single token rather than being word-split:

args=($ARGUMENTS)
node "${CLAUDE_SKILL_DIR}/scripts/mine-traces.mjs" "${args[@]}"

The engine writes prompt-miner-<UTC-date>.json + .md to --out (default $TMPDIR/oh-prompt-miner/<UTC-date>/), unless --dry-run was passed (it prints the JSON dataset to stdout and writes nothing). The flag surface (defaults in parens):

• --harness all|claude|pi (all), --since/--until (YYYY-MM-DD), --hours N (precedence over --since), --last-n N, --min-turns N (2), --top N (15), --attribution first|all (first).
• --include-prompt-text (off — see Privacy contract), --no-git (stub ground-truth bonus to 0), --weights '<json>', --out <dir>, --report-only, --dry-run, --max-file-mb N (50).

If manifest.sessionsScanned == 0: announce RESULT: NO-SESSIONS and stop. If --dry-run was passed: read the printed dataset, optionally summarize the top/bottom ranked sessions, announce RESULT: DRY-RUN, and stop (no marker proposals, no identity writes).

Step 2 — Read the dataset

Read the emitted prompt-miner-<UTC-date>.json (or the --dry-run stdout). The shape is documented in references/report-schema.md:

• manifest — sessionsScanned, sessionsRanked, toolErrorsTotal, malformedLines, skippedFiles, weights, window, scoreModel, ceilingSaturation (per-stratum { atCeiling, total } census over the rankable population — how much of each session type sits on the clamp ceiling).
• sessions[] — ranked (score desc), each with score, scoreUncapped (the same value before the 0..100 clamp — the correlation scale for Step 3), scoreBreakdown, sessionType, and a features vector (the 13 markerFeatureKeys).
• unranked[] — noHumanPrompt / below-minTurns sessions (kept, not ranked).
• weaknesses[] — metadata-only WH-<NNN> harness-weakness records clustering repeated failure signals across the corpus; deterministic, and never carries prompt text (supporting_traces = session-id metadata only). See references/report-schema.md.

Verify every score is reconstructable from its scoreBreakdown before trusting the ranking — the score is a heuristic proxy, not a verdict (see references/scoring.md; correctionDensity is the highest-variance signal).

Step 3 — Mine markers (stratified by session type)

For each session-type stratum (impl, retro, query, audit, cron, other) — never pooled across types (pooling manufactures Simpson's-paradox artifacts; see references/markers.md) — correlate each feature in markerFeatureKeys against the session scoreUncapped — not score, which is censored above 100 (see references/markers.md). Emit each marker in the exact falsifiable schema from references/markers.md:

{
  "feature": "hasAcceptanceCriteria",
  "direction": "positive",
  "threshold": true,
  "sessions_supporting": 14,
  "sessions_contradicting": 3,
  "effect_size": 0.41,
  "effect_size_capped": 0.36
}

A marker is reportable only when, within a single stratum, sessions_supporting ≥ 10 and effect_size ≥ 0.3.

Stability guard. Compute effect_size_capped — the same statistic against the clamped score — for every marker. A marker whose two scales disagree on sign, or where exactly one of |effect_size| ≥ 0.3 and |effect_size_capped| ≥ 0.3 holds, is UNSTABLE: report it with both values and the reason, but do not carry it into Step 4 — it is not promotable on either scale, earns no identity proposal, and files no issue. See references/markers.md § Stability guard.

Corpus-size gate. If no session type reaches the sessions_supporting ≥ 10 floor: announce RESULT: NO-CORPUS, report that the corpus is too small to mine reliably, and stop — do not propose markers from a thin corpus. If strata are large enough but nothing clears both thresholds this run, report NO-CANDIDATE and stop (no identity proposals).

Step 4 — Propose-then-confirm (mirrors /retro)

Only run this step when reportable markers exist and --report-only / --dry-run were not passed. Translate each reportable marker into a candidate lesson, then gate it exactly like /retro (.claude/skills/retro/SKILL.md § 6):

1. Qualify filter. Drop any candidate that is a secret, raw command output, a step-by-step plan, or anything re-derivable in under a minute.

2. Dedup against existing probes. For each surviving candidate, grep .agro/evals/probes/ for a probe that already asserts the same invariant; if it is already captured, link or skip — never double-write (this is the same dedup /retro performs in its qualify filter).

3. Promotability. A marker that is merely descriptive ("this corpus shows X prompt trait correlates with better <type> sessions") is reported, not promoted — say it in the report and stop there. Only a marker that has generalized across many sessions into a prescriptive principle ("always include acceptance criteria") earns a proposed probe under .agro/evals/probes/ — and a probe is never auto-written.

4. Propose, then wait. Present the block and stop until the user responds:

   Proposed probe(s) under .agro/evals/probes/:
   - <probe name> asserts <prescriptive principle> [prompt-miner · <stratum>] — basis: <one clause>
   
   Type APPROVE to record, SKIP to discard any item, or EDIT <n> <new text> to revise.
   

5. Record approved items. On APPROVE, add each approved probe proposal to the run report in $TMPDIR, beside the weakness records and the ranked marker table. /prompt-miner writes no tracked file; an approved proposal becomes a real probe only through /spec, which builds and gates it. --report-only and --dry-run skip this step entirely.

Announce RESULT: MINING-COMPLETE once the gate has run.

Anti-patterns

• Pooling across session types. Markers MUST be stratified; a marker that holds only after pooling is not reportable.
• Mining a thin corpus. Below the sessions_supporting ≥ 10 floor, emit NO-CORPUS and stop. Do not manufacture noise-driven markers.
• Committing transcripts. Artifacts are gitignored; never stage them, and never commit --include-prompt-text output.
• Auto-promoting a marker. Step 4 is propose-then-confirm. Never record a probe proposal without an explicit APPROVE, and never from a single run's evidence.
• Word-splitting --weights. Always invoke the engine via the args=($ARGUMENTS) array form so the JSON stays one token.
• Inventing a file to save a marker in. A descriptive marker that does not generalize is reported and dropped. Do not create a ledger or a dated note to hold it.

References

• references/scoring.md — the friction + ground-truth scoring model.
• references/markers.md — the feature taxonomy + falsifiable marker schema + thresholds.
• references/report-schema.md — the emitted JSON/MD dataset shape.