formulate-hypothesis

You are formulating one optimization hypothesis for the autoresearch loop. The output of this skill MUST be a structured proposal that the master agent reviews before dispatching. Do not skip steps.

Step 1 — Gather inputs from caller / session context

The caller (typically the loop in /start-experiment Step 9, step 2(c)(i)) provides:

• <model> + <lane> — e.g. <model> + jax
• <variant> — current variant being optimized (e.g. 3B/v5p-16, 24B/v5p-32)
• <mode> (optional, default frontier) — one of:
  • frontier (default): variant has a measured-best baseline and regression would be costly. Full ceremony. Pick this when the model page's variant matrix shows a non-trivial Current best for the variant.
  • exploration: variant is early-stage, stuck, or being probed. Lighter ceremony, hunches allowed. Pick for stalled variants where the ladder has saturated, or when probing a hypothesis category the wiki doesn't yet cover.
  • bootstrap: brand-new model with thin/no per-model state (no per-model refuted-patterns.md, no retrospectives, variant matrix has < 3 rows or no Current best). Even lighter ceremony — Steps 4, 5 degrade gracefully; first-principles reasoning from index + blueprint is the primary input. See the "Bootstrap mode" section below for the auto-detection rule and the full mode-specific override behavior.
  • user-override: the caller has specified an exact candidate (passed in <user_specified_candidate>). Skill skips hypothesis generation and just validates the candidate (refuted-pattern check + topic fit + semantics-preservation check), then emits the proposal.
• <user_specified_candidate> (optional, REQUIRED if <mode> is user-override) — the exact hypothesis the caller wants validated (one sentence, falsifiable form).
• <topic_hint> (optional) — caller's best guess at the topic. If not provided, infer from recent profile / frontier-shift conversation.
• <recent_profile_summary> (optional but strongly preferred) — bucket attribution from the most recent xprof. If absent, the proposal should recommend capturing a profile first rather than committing to a candidate.
• <current_frontier> — MFU + experiment ID of the variant's current frontier
• <last_2_3_experiments> — what was just tried, with verdicts
• <in_flight_v_ids> — workload v-IDs other tracks are running, so the proposal steers clear of duplicates

If any input is missing AND not derivable from session context, ask the caller for it before proceeding. Do not invent.

Step 1.5 — Describe the profile signal in your own words BEFORE consulting the TLDR

Before you open the index, write a 2-3 sentence description of what <recent_profile_summary> actually shows: which buckets dominate, by what percentage, what changed vs the prior experiment, whether the dominant bucket has a clear name (convolution fusion, custom-call, data formatting, etc.) or is unusual.

This is the profile-first anchor. The TLDR table maps known signatures to topics; without grounding in your own reading of the profile first, you'll be tempted to round the actual signal into the nearest TLDR row even when it doesn't quite fit. Reading the profile first lets you spot mismatches and novel signatures.

If no recent profile is available, state that explicitly and recommend the caller capture a profile before this skill commits to a hypothesis — except in exploration mode, where a profile-less hunch is allowed (see Step 7's exploration template).

Cost visibility — what this skill consumes per invocation

Full-ceremony invocations (frontier mode, no short-circuit) read:

• Index: ~25k tokens
• Blueprint: ~18k tokens
• Catalog-cited depth pages for 1-3 implicated topics: ~30-150k
• Model page + last 3 experiments + 1 retrospective: ~25-50k
• Per-model refuted-patterns.md: <5k

Total: ~100-260k tokens per invocation = ~$0.50-2.00 at current Opus 4.7 input pricing.

This is the right cost when frontier-hardening, exploring a non-trivial candidate, or when a regression would be expensive. It is NOT the right cost for:

• One-off interactive questions ("what's the current frontier?") — just read the model page directly
• Quick triage during a chat ("what should we try next?") — caller can name a candidate and use user-override mode for a ~5k validation
• Loop iterations where the previous hypothesis just refuted and the next-best from the open-hypothesis list is obvious — caller can specify it via user-override

If you're invoking this skill repeatedly within a 5-minute window, the prompt cache will absorb most of the cost. Across multi-hour loop iterations, expect re-reads on cache eviction.

Bootstrap mode — handling new models with thin state

If <mode> is bootstrap (or autodetected: model page has < 3 variant matrix rows AND no per-model refuted-patterns.md AND no retrospective in wiki/analyses/ matching the model):

• Steps 4 + 5 degrade gracefully: per-model refuted-patterns.md likely doesn't exist; retrospectives don't exist; model page is thin. Note these absences explicitly in the proposal.
• Step 3 (depth pages from index) becomes the primary precedent source — substitute for per-model history.
• First-principles reasoning from the index's hardware envelope + the blueprint's Phase 0 (substrate sanity) and Phase 1 (cheap memory + precision) is what shapes the early proposals. Bootstrap candidates often live at Phase 0 or Phase 1, but novel-phase candidates are also allowed.
• Refuted-pattern check (Step 6) uses Layer A (generic principles) only — Layer B (per-model refutations) is empty by definition.
• Proposal flagged as Mode: bootstrap so the master/reviewer knows the precedent grounding is generic-only.

Use bootstrap mode when:

• A new model just entered the autoresearch loop and is being shaken down
• The variant has just been added to the matrix and there's no measured baseline yet
• All earlier ladder rungs (Phase 0, Phase 1) are unvalidated for this variant

Kernel families — this skill does NOT apply

Kernel experiment families (wiki/kernel_experiments/, lane: pallas) use the self-contained /formulate-kernel-hypothesis skill instead — the kernel lane has its own catalog (kernel-optimization-index.md), no blueprint, and different mandatory fields (intervention class, op-point provenance, candidate plan, parity-gated bar vs the frontier). If you were invoked for a kernel family, STOP and load that skill; this one's model machinery (blueprint phases, variant matrices, MFU targets) does not transfer.

User-override mode — caller-specified candidate

If <mode> is user-override, the caller has already decided what to try and passed <user_specified_candidate>. The skill's job is validation, not formulation:

1. Read index + blueprint (Step 2, 2b) — but only to determine which topic the candidate lives in (or to confirm it's novel).
2. Skip Step 3 (depth-page reads) unless the candidate is in a topic with a documented refuted pattern adjacent to the candidate — then read the specific refutation observation page.
3. Check the candidate against:
   • Generic refuted-pattern principles for its topic (Layer A)
   • Per-model refuted patterns IF the per-model refuted-patterns.md exists (Layer B)
   • Semantics-preservation (program.md "What you CANNOT do" — math changes, dropping below bf16, etc.)
   • Phase ordering (is the candidate's phase reachable given saturation of earlier phases?)
4. Emit proposal as Variant E: user-override (see Step 7).

If the candidate violates a hard constraint (semantics change, structurally refuted pattern verbatim), surface the violation in the proposal — but the caller's override stands. Recommend an alternative for the caller's consideration; don't auto-abandon.

Total tokens: ~5-15k (vs ~100-260k for full ceremony). Latency: ~1-2 turns of master context.

Exploration mode — lighter ceremony for hunches + stalled variants + novel directions

If <mode> was set to exploration in Step 1, the following apply throughout the skill:

• Steps 3 + 4 are optional, not mandatory. The catalog-cited depth-page reads and per-model refuted-patterns check can be skipped if the candidate is genuinely exploratory (you're trying X because you have a hunch from a recent profile observation, not because the catalog points at it).
• Step 6 refuted-pattern check is advisory, not blocking. If the candidate matches a refuted pattern, surface the match in the proposal — don't auto-abandon. Exploration mode tolerates retrying refuted patterns when the op-point has materially shifted.
• Profile-less hunches allowed. If no recent profile exists, exploration mode lets you propose anyway with the proposal explicitly flagged as Profile-less: yes.
• Novel directions are first-class. Phase and topic fields can be novel without justification ceiling — exploration mode exists precisely to probe ideas the catalog doesn't yet cover.
• Proposal template is 5-line minimum (see Step 7's exploration variant): hypothesis, what triggered the hunch, what's being explored, falsification criterion, recommended dispatch.
• Flag the proposal as Mode: exploration so the master/reviewer can weight differently (e.g., accept higher refutation rate in exchange for surface-area coverage).

Use exploration mode when:

• The variant has no measured-best baseline yet (early-stage)
• The variant has stalled at a plateau and the standard ladder has saturated
• You're probing a hypothesis category the wiki doesn't yet cover
• You have a novel direction that doesn't fit any cataloged topic or blueprint phase

Do NOT use exploration mode when:

• The variant has a non-trivial Current best and a regression would be costly (frontier mode is the default for a reason)
• A clear profile-driven candidate exists from the catalog (use the standard path)

---

Step 2 — Read wiki/model-optimization-index.md in full

Read the entire wiki/model-optimization-index.md. Don't skim — read it.

Rationale: the file's cardinal rule enforces that every line is hypothesis-generation material — TPU envelope constants, topic mechanism citations, generic principles, refuted-pattern principles, Pallas kernel catalog, reference stacks. There is nothing in it to skip. The downside of selective reads (missing cross-references the TLDR didn't telegraph, tunnel-visioning on TLDR-implied topics) is exactly the v391y-class failure mode this skill exists to prevent.

Reading the full file once at hypothesis time costs ~25k tokens; the master context is 200k+. Cheap relative to the safety it buys.

After reading, identify the 1–3 topic sections the candidate hypothesis most directly touches, and note any unexpected cross-references to other topics (these are exactly the connections the TLDR table won't surface). Capture the topic anchor IDs (e.g. #ce--softmax-loss, #moe-expert-dispatch) for the proposal.

Handling novel profile signatures

If no TLDR row matches the profile signal you described in Step 1.5: do not punt to the caller. Instead:

1. Describe the signal precisely (which bucket, what %, what's unusual about it).
2. Reason from first principles using the index's hardware envelope (HBM BW, VMEM cap, MXU shape, bytes/flop) + the most-relevant existing topic sections.
3. Propose a hypothesis grounded in that reasoning.
4. Flag the proposal as Novel-signature: yes so the master/reviewer knows the precedent path didn't apply, and file a new TLDR row recommendation in the proposal — one-line "profile-signature → topic → first-line hypothesis class" — for the next regeneration of the index to incorporate.

The catalog isn't a ceiling; it's a starting set. Novel signatures are where new optimizations come from. Reasoning from first principles + filing the row back is how the catalog grows.

Step 2b — Read wiki/model-optimization-blueprint.md in full

Read the entire wiki/model-optimization-blueprint.md. Same rationale as Step 2 — it's lean, every line is operational, and you need full context to apply the phase-ordering rules.

The index and the blueprint are intentionally complementary:

• Index answers "for topic X (CE / sharding / VMEM / flags / …), what's the mechanism and what's been refuted?" — topic-keyed lookup.
• Blueprint answers "in what order do I apply optimizations, what compounds, what trap will bite me?" — ladder-keyed procedure.

Use the blueprint's ladder (Phase 0 → 6) to identify:

• The phase the candidate sits at — one of:
  • Phase 0 (substrate sanity)
  • Phase 1 (cheap memory + precision)
  • Phase 2 (known-good high-yield flag bundle)
  • Phase 3 (kernel / structural code work — biggest wins, 2-30×)
  • Phase 4 (batch + sequence amortization)
  • Phase 5 (cross-variant + cross-generation transfer)
  • Phase 6 (exploratory flag sweep — last resort)
• Phase saturation: have all earlier phases been applied for this variant? If Phase 0-1 substrate-sanity / cheap-memory levers haven't been verified, those should usually come BEFORE a Phase 3 kernel rewrite — even when the profile signal points at the kernel. Surface this to the caller.
• Compounding awareness: the blueprint documents which earlier phases unlock or remove blocking constraints for later ones (e.g., xla_tpu_scoped_vmem_limit_kib unlocks larger Pallas block sizes). If the candidate depends on a not-yet-applied earlier phase, sequence accordingly.
• Trap awareness: the blueprint flags specific traps per phase (e.g. "flag sweep last" rule, "silent no-op audit" requirement, "verify the change took effect before drawing perf conclusions"). The candidate must not violate the phase's traps.

If the candidate is genuinely Phase 6 (exploratory flag sweep) but Phase 1–5 are not yet saturated on this variant: the blueprint's "flag sweep last" rule is advisory — it reflects the empirical observation that Phase 6 has typically < 1 pp yield. The agent can still propose Phase 6 with a one-sentence justification (e.g., "new libtpu version unlocks this flag, want to validate before committing to Phase 3 kernel work"). The proposal must explicitly note the saturation gap.

Phase doesn't fit any of 0-6? Use Blueprint phase: novel in the proposal with a one-sentence mechanism description. The ladder is curated from past lessons; novel mechanisms are how it grows.

Step 3 — Read the depth pages cited under the implicated topics

Trivial-case short-circuit (skip Step 3 + 4 reads when this fires)

If ALL of the following are true:

• The candidate is a single flag flip that already appears verbatim in the index's "Universal multi-host stack" or a named MaxText preset (e.g., CF_FOR_ALL_GATHER, DISABLE_COLLECTIVE_MATMUL, HOST_OFFLOAD_FLAGS), AND
• The profile signal directly implicates the flag (e.g., dominant all-reduce bucket + you're proposing the AllReduce-related CF flag), AND
• You're in frontier mode (exploration mode also short-circuits — see "Exploration mode" below)

Then you can skip the depth-page reads in Step 3 and the per-model layer in Step 4, and produce a 3-line proposal instead of the full template. The 3 lines must include: hypothesis (the flag + value), profile signal (1 line), falsification criterion (1 line). Flag the proposal as Short-circuit: yes.

Rationale: known-good flag bundles are documented precisely because they're low-risk + low-novelty. Walking the full ceremony for "set the flag the index says to set" is pure overhead.

Otherwise — read the cited depth pages

For each of the 1–3 implicated topics identified in Step 2, the topic's Mechanism subsection in the index lists specific cited concept + observation + source pages. Read those pages directly — they are the depth content for the topic.

Use the Read tool for each cited path. The catalog already did the filtering work: by listing them under the topic's Mechanism, the index is telling you these are the relevant pages for this topic. No further filtering is needed; no subagent indirection is needed.

Selectivity guidance within a topic's citation list (so context doesn't balloon unnecessarily):

• Always read any observation page in the citation list — observations document failure modes specific to this project where exact wording matters most (e.g., the precise canonical pattern, the exact deadlock condition).
• Always read any concept page named as canonical (e.g., the index will mark "canonical" pages explicitly, or they'll be the only concept page for a load-bearing primitive like shard_map, segment-matmul-pallas-kernel, etc.).
• Read by default concept pages that the index's one-line description marks as load-bearing for the candidate's mechanism. If the description tells you the page isn't load-bearing for your specific candidate (e.g., a v6e-only nuance when you're on v5p), skip it.
• Do not cascade — when a depth page references further depth pages, do NOT follow those references. The catalog's curation stops at one level.

Source pages (papers, docs) — read these for foundational understanding when the candidate involves a kernel/architecture variant the wiki has ingested as a source (NSA, FA3, TyphoonMLA, etc.). The catalog will tell you when.

For each cited page read, note: principles, refuted patterns, cross-references to other topics, specific code patterns. Carry forward into the proposal in Step 7.

Context-cost honest expectation: a typical topic's Mechanism cites 5-10 pages; 1-3 topics × ~5-10 pages × ~5k tokens each = ~30-150k tokens read into master context per hypothesis. This is the right cost — the agent needs source material to formulate a quality hypothesis; the catalog already filtered to the most relevant material; adding a summarizer middleman would only reduce fidelity without reducing real cost.

Step 4 — Read the per-model refuted-patterns layer (if it exists)

Read wiki/experiments/<model>_autoresearch_optimization/refuted-patterns.md if the file exists.

This is the per-model layer — it lists specific experiment IDs (e.g. v391y on 3B/v5p-16) that refuted a pattern with the variant + measured regression. If the file doesn't exist for this model, that's fine — note this and continue.

If the file exists but doesn't have an entry for the topic in scope: that's also fine — the topic may genuinely have no prior refutations on this model.

Step 5 — Read the lane log + most recent retrospective FIRST + model page + last 3 experiments

Read in this order. The retrospective is the highest-density single input — read it FIRST when fresh.

Step 5a — Most recent retrospective for THIS (model, lane) (read FIRST when fresh)

Search wiki/analyses/ for *<model>*<lane>*retrospective*.md, take the newest. If it exists AND its updated: (or filename) date is within the last 6 hours, it is fresh; in that case:

• Read it in full
• Extract the ## Recommended next directions table. The top 3 entries are candidate proposals for THIS hypothesis-formulation invocation. Add them to your candidate set with source: retrospective:<filename> tagging.
• Extract the ## Experiment ledger table. Use it to skip redundant per-experiment reads (the retrospective already categorized + verdict-tagged everything).
• Note the retrospective's Search-strategy assessment (premature-exhaustion signals): if the retrospective flagged "single-axis-heavy: ratio > 5:1" or "frontier stuck for N+ experiments", bias the candidate ranking toward kernel/structural work over more single-axis probing. The retrospective's diagnosis IS evidence.

If the retrospective is stale (>6 hours OR not present), proceed normally — read it for context but don't treat it as primary input.

Step 3 shortcut: if the retrospective is fresh AND covered the topics your candidate touches, you may SKIP Step 3's depth-page reads for those topics — the retrospective is the synthesized view of that topic-space. Skip only when the retrospective explicitly cites the same depth pages you'd otherwise read (which it typically does in its Sources section).

Step 5b — Lane log + model page + recent experiments

After the retrospective (or skipping to here if no retrospective), read:

1. Last 50 lines of the lane's log: wiki/experiments/<model>_autoresearch_optimization/<lane>/log.md. Per SCHEMA's two-tier log convention, the lane's log carries every recent loop-iteration + start / stop markers + manual ops on this lane. If the file doesn't exist, this is a new lane — skip and proceed.

2. wiki/models/<model>-<lane>.md — focus on the variant matrix row for <variant>, the "Cross-variant open hypotheses", and "Retired hypotheses" sections.

3. The last 3 experiment pages under wiki/experiments/<model>_autoresearch_optimization/<lane>/ — what was just tried with verdicts. If a fresh retrospective ALREADY covered these in its ledger, you may skip the deep read and use the retrospective's summary (cite which retrospective row).

4. Fallback for "most recent retrospective" search — if Step 5a didn't find one for THIS lane, search wider:

   ls wiki/analyses/ | grep -E "<model>.*<lane>.*(retrospective|synthesis|day-summary|session-summary)" | sort -r | head -1
   

   If that returns nothing, also try the cross-lane retrospectives matching the model only (these capture lessons that span lanes — e.g. *<model>*cross-lane*retrospective.md). Read just the single most recent retrospective found. The newest retrospective is the freshest synthesis and typically references earlier ones if relevant — no need to load multiple. If none exist for this model+lane, note that and continue.

   Why this matters: retrospectives are the highest-density distillation of hard-won lessons across a multi-day or multi-experiment chunk of optimization work. They surface compounding patterns, near-miss refutations, and cross-cutting traps that the model page's variant matrix + per-experiment pages don't capture in one place. The blueprint's ladder is partly derived from retrospectives; reading the latest one catches model-specific nuances the blueprint generalized away.

These are the per-model state (vs the index/blueprint generic state). Together with Step 4's per-model refuted-patterns.md, they give the master the model-specific picture.

Optional: if the topic-keyword search of wiki/observations/ surfaces an observation page the index didn't cite (the index isn't exhaustive), read that page too. Don't go beyond 1-2 such uncited observations.

Step 6 — Check candidate against the two-layer refuted set

Compare the candidate hypothesis against:

Layer A — Generic refuted-pattern principles from the topic section(s) in Step 3:

• These are model-agnostic traps (e.g. "Reasoning about kernel cost from FLOPs alone"; "Auto-partitioning a Mosaic kernel"; "Same-axis nested collectives").
• If the candidate violates any of these principles → abandon the candidate. Pick a different angle.

Layer B — Per-model refuted experiments from Step 4 (refuted-patterns.md) + the model page's "Retired hypotheses" section + the last-3-experiments' verdicts read in Step 5:

• If the candidate matches a per-model refuted experiment verbatim → abandon the candidate.
• If the candidate is a variant of a refuted pattern (different op-point, different scale, different hardware row) → either explicitly cite why the prior refutation should not apply at the current op-point, OR pick a different candidate.
• If the candidate is genuinely orthogonal → proceed.

This step is the single biggest safeguard against repeat-failure-mode regressions. Do not skip.

Step 7 — Produce the structured proposal