/quiz-me — quiz the user on what was just built

Adapted from the MIT-licensed no-numb plugin (github.com/Ciucky/no-numb); see .claude/skills/ATTRIBUTION-no-numb.md. Runs host-side.

Overview

After you write code for someone, they usually understand the result at a high level — what it does, its inputs and outputs — but not the internals or the why. That understanding "vaporizes." This skill has you administer a short comprehension quiz on what you just did this turn, so the understanding sticks. Being tested is itself the learning (retrieval practice); the quiz is not just a check.

Core principle: quiz what this turn produced, from your own memory of what you just did — not generic trivia, not the whole codebase, not what the app does at a high level.

When to run

• Right after a turn in which you edited or wrote files, before finishing.
• When the user runs /quiz-me.
• When the no-numb Stop hook (.claude/hooks/nonumb-gate.sh) blocks your turn and asks you to quiz. (That gate is DEFAULT-OFF; it fires only when the user has opted in via DEUS_NONUMB or ~/.config/deus/nonumb.json.)

Step 1 — Is it worth quizzing?

Look at what you changed this turn. Bias toward quizzing. Only skip when the change is genuinely cosmetic: formatting / whitespace, a variable rename, a CSS color tweak, a typo fix, a version bump. Everything else → quiz. When in doubt, quiz. If you do skip, say so in one line and stop — don't quiz on nothing.

Step 2 — Read the settings

Read ~/.config/deus/nonumb.json. Keys (all optional, defaults in parentheses):

Key	Meaning
depth ("standard")	"standard" / "deep" / "principle" — the retrieval-difficulty dial below.
grader ("independent")	"independent" = a blind GPT/codex backend authors the quiz (Step 2b); "self" = you author it from your own memory of the turn (skip Step 2b).
grader_model (codex default)	optional faster codex model id for the author backend.
grader_reasoning ("low")	codex reasoning effort for authoring — low keeps the gate snappy.
grader_timeout_s (120)	author backend timeout.
cards.enabled (true)	persist a learning card after a pass (Step 7).
cards.dir ("Learning-Cards")	vault-relative card directory.

Note whether this was a code change or a non-code change (docs/prose/config) — it changes how the depth dial applies. The optional "principle" depth is the most advanced tier (below).

Step 2b — Independent authoring (P4), when grader == "independent"

The default is to NOT author the quiz yourself — you wrote the code, so you'd write softball questions and flatter your own choices. Instead, hand authoring to a blind backend that has not seen your reasoning:

python3 ~/deus/scripts/nonumb.py author --repo "$PWD" --depth <depth> --json

(Use the directory whose change you're quizzing on as --repo.) author reads grader_model / grader_reasoning / grader_timeout_s from nonumb.json itself, so you do not pass them. On success (ok:true) it returns questions[] — each with axis, depth, stem, exactly four length-balanced options, an integer correct_slot, and a why. Deliver THOSE questions verbatim in Step 4 and grade each answer by comparing the chosen slot to correct_slot — a pure integer comparison. Do not rewrite, soften, or re-key them, and do not offer the AskUserQuestion "Other" free-text box for these questions (judging a typed answer against the key would reintroduce exactly the self-grading bias this step removes). Remember grader_source: "codex" for the card.

Fail open. If the command returns ok:false (codex unavailable, timeout, empty/ non-git diff, or a non-conforming quiz), say in one line: "independent quiz unavailable ({reason}) — self-authored," then fall back to authoring the quiz yourself per Steps 3–4. Record grader_source: "self" so the fallback is visible in the card.

Step 3 — The two dials: depth × axis

A good quiz crosses two independent dials. Decide each per question.

Dial A — depth (how hard to retrieve the answer)

The single test for standard vs deep:

Do you need to read the code to answer it? Yes → deep. No → standard.

• standard (default) — answerable WITHOUT opening a file, from understanding the decisions. Do not reference specific files, lines, or function names. "Why does X break if that guard is removed?"
• deep — only answerable BY reading the code. Always point them to where to look (file + function/region). Favor reasoning over recall. "In the doorbell check in gate.sh, what edit would slip past it?"
• principle (advanced, one tier up) — strip the specifics and ask the transferable lesson: what general class of bug/decision is this, and where else does it apply? "This is an instance of what general failure pattern, and where else would it bite?"

Dial B — comprehension axis (which facet you probe)

Sample across these five axes — don't ask five of the same kind. The number of questions scales to the change, with a minimum floor so a quiz can't be trivially passed with one softball: at least 2 questions for any non-cosmetic change, and at least 4 for a new file or a substantial refactor. A one-line fix → 2; a new module → 4+. Don't pad past what the change warrants, and don't force all five axes on a tiny change.

Axis	Probes
what changed	name the real edit + new behavior
why this shape	the tradeoff chosen over the obvious alternative
what would break	predict a regression if a key line changed
how was it verified	name the actual test/command run — or admit there wasn't one
what to review later	one remaining uncertainty/risk worth flagging

The how-was-it-verified axis is the highest-value one: it forces you to confront whether you actually ran anything. If the honest answer is "it wasn't verified," say so plainly — that surfaces the gap to the user before they move on. In gate-triggered mode you (the model) already know what commands you ran this turn and the user does not — so don't quiz them on a fact only you hold. Instead, disclose it: state what was actually run (or that nothing was), and if you want a question on this axis, ask what the verification would need to cover, not "what did Claude run."

Non-code changes (docs/prose): the read-the-code test doesn't apply — ask about what changed and treat depth as a difficulty knob. Config files are code-like enough that deep/principle work normally.

Stay below the awareness line (every mode): never ask "what does this app do" or "what are its inputs and outputs." That's the high-level understanding the user already has — testing it teaches nothing. Aim at the internals, the why, and the transferable lesson — the part that vaporizes.

Step 4 — Deliver as multiple choice, one at a time

Deliver with AskUserQuestion, one question per call, with four options each.

• Plausible distractors. The wrong options must be the misconceptions someone would actually hold if they didn't understand this code. Obvious-dummy options turn it back into trivia and let them pass by elimination.
• Length-balanced options. Write all four options at roughly the same length and specificity. Never let the correct answer be the longest or most detailed — that is a tell that lets the user pattern-match the answer without understanding it. (Equalize: if the correct answer needs a clause, give the distractors one too.)
• Vary where the correct answer sits. Decide the correct option's slot (1st–4th) before you write the options, and rotate it across questions (e.g. 2nd, 4th, 1st, 3rd). Never default the correct answer to the first slot — a quiz whose answer is always option 1 is passed by reflex.

Multiple choice is required, not a style preference. AskUserQuestion is a tool call, so your turn stays alive and the gate can hold. A plain-text question would force you to end your turn and wait for a reply — which releases the gate. Do not switch to prose questions during a gated quiz. (When self-authoring, the "Other" free-text box is fine for the rare question that needs a typed answer, since it stays inside the tool call. When delivering an independently-authored quiz (Step 2b), do NOT offer "Other" — grading there is a pure integer slot comparison against the external key.)

Step 5 — Grade and explain

After each answer: if correct, confirm in a sentence. If wrong, show the correct option and explain why — the specific low-level detail they missed. This explanation is where the learning actually lands, so make it real, not a restatement.

Step 6 — Retake on any miss

Any wrong answer means the user retakes the full quiz. Regenerate it with light rewording and reordering so they can't just memorize the answer key. Stay in this turn and keep going until they pass — do not end your turn, summarize, or move on to other work until every answer is correct (or the user deliberately interrupts). On a pass, you're done; just conclude normally.

Step 7 — Persist the learning card (P2), when cards.enabled

After the user passes, persist one lean learning card to the vault so missed concepts can resurface later (spaced repetition). Pipe a JSON card to:

echo '<card-json>' | python3 ~/deus/scripts/nonumb.py record --json

The card JSON fields:

Field	Value
description (required)	one-line summary of what was learned/missed — this is what memory_tree embeds, so make it a real, searchable sentence.
turn_summary (required)	what the turn changed, in a phrase.
depth (required)	the depth you quizzed at.
grader_source (required)	"codex" if Step 2b authored it, "self" if you fell back.
diff_hash	from the author output (ties the card to the change).
axes_covered	the axes you sampled, e.g. ["what_changed","how_verified"].
missed_concepts	the specific things they got wrong (empty if first-try pass).
verification_command	the actual test/command this change was verified with, or "NONE — flagged".
review_later	the one remaining uncertainty worth flagging.
repo	the repo path.

Use real newlines in values, not \n. record reads cards.{enabled,dir} from nonumb.json itself — so you do not pass --cards-dir; a user-set directory is honored automatically, and if cards.enabled is false record is a no-op skip. The card is written to the gitignored personal vault (never the repo) and indexed immediately (a fast incremental memory_tree build), so it's queryable at once. A record failure must not block your turn — note it in one line and finish.

Quick reference

Depth	The test	References files/lines?
standard	answer it without reading code	no
deep	must read the code to answer	yes — point them there
principle	the transferable lesson, specifics stripped	sometimes

Common mistakes

• Quizzing the whole repo. Only quiz what this turn changed.
• Trivia distractors. Make the wrong answers tempting, not silly.
• The longest option is the answer. Length-balance every question.
• All one axis. Sample across the five; don't skip how-was-it-verified.
• Drifting too high. "What does the app do" tests what they didn't lose.
• Standard that references code, or deep that doesn't point to it.
• Switching to prose in a gated turn. Keep it AskUserQuestion MC.
• Skipping too eagerly. Only genuinely cosmetic changes skip.
• Self-authoring when grader: independent. Run Step 2b first; only self-author on its ok:false fallback.
• Re-keying or grading "Other" on an independent quiz. Deliver verbatim; grade by integer slot only.
• Forgetting the card. On a pass with cards.enabled, persist it (Step 7) so misses can resurface.
• Correct answer always first. Vary its slot every question.