test-optimization

Purpose: Systematic test-performance workflow for any sub-project (backend, frontend, cli, mcp-server, E2E).

Output: Optimized tests with per-slice commits + summary ending with ## TEST OPTIMIZATION COMPLETE.

When --resolve is given, skip every other step and go straight to resolve_candidates. Read resolving-candidates.md in full first.

Git does not use the Nix prefix. All other repo tooling does: CURSOR_DEV=true nix develop -c …

Non-negotiable rules (every optimization pass):

1. Remove or simplify redundant tests first — merge overlapping scenarios, drop duplicate setup, delete tests that only repeat coverage elsewhere.
2. No fixed-time waits — no sleep, no cy.wait(ms) without an assertion, no arbitrary setTimeout / debounce-timeout polling in unit tests. Use assertions, intercept aliases, fake timers, or API/testability setup.
3. Flaky is failure — re-run touched tests until stable; fix root cause, do not mask with retries.

Execution model: After writing the plan, always use dough-execute-plan (.agents/skills/dough-execute-plan/SKILL.md). Coordinator delegates each group to a fresh sub-agent and applies dough-execute-plan's coordinator-owned wrap-up. Do not accumulate context across slices in one agent.

E2E skip tag: @skipOptimizationDueToKnownNecessarySlowness on a Scenario or Feature marks known-necessary slowness. Profile runs exclude it via --expose tags=… (see profile). Adding the tag is a developer decision (Jidoka) — propose only; do not add it yourself.

Candidates: .planning/test-optimization-blacklist.md holds Candidates from optimization runs (proposals only). Keep that section; do not invent a Skip list there.

Do not commit raw profile JSON (large, machine-specific). Gitignored paths: e2e_test/reports/, .planning/*-profile-results.json, .planning/quick/*-profile-results.json, ongoing/*-profile-results.json.

Scope	Profile command	Parse durations from
E2E	CURSOR_DEV=true nix develop -c pnpm cy:run-on-sut --reporter json --expose tags='not @ignore and not @skipOptimizationDueToKnownNecessarySlowness' (SUT up; pnpm sut:healthcheck)	JSON blocks in stdout — tee to /tmp/e2e-profile.log
Frontend	CURSOR_DEV=true nix develop -c pnpm -C frontend exec vitest run --reporter=json	Vitest JSON testResults[].assertionResults[].duration
CLI	cd cli && CURSOR_DEV=true nix develop -c pnpm exec vitest run --reporter=json	Same as Vitest
Backend	CURSOR_DEV=true nix develop -c pnpm backend:test_only then parse	backend/build/test-results/test/TEST-*.xml → testcase@time
MCP server	CURSOR_DEV=true nix develop -c pnpm -C mcp-server exec vitest run --reporter=json	Vitest JSON

E2E tags: Always pass --expose tags='not @ignore and not @skipOptimizationDueToKnownNecessarySlowness' for profile (and re-profile) so tagged scenarios/features are not run and do not enter the top 10%. CI default tags still apply for normal runs; this override is profile-only. In CI, also keep excluding @wip if you mirror CI: not @ignore and not @wip and not @skipOptimizationDueToKnownNecessarySlowness. Use --expose / -x, not --env — preprocessor v27 on Cypress >= 15.17 ignores env.tags.

Frontend note: frontend:test runs Vitest browser mode (--browser=chromium). Profile uses plain vitest run for duration data; verify changes with frontend:test.

E2E JSON: Cypress --reporter-options output=… may not write a file; tee stdout and parse { "tests": [ { "title", "duration" } ] } blocks between spec runs.

Store baseline locally (e.g. .planning/quick/<scope>-profile-results.json with "do not commit" note). Record baseline wall time and test count in the plan.

Sort eligible by duration descending; take first n_slow. Record in plan: rank, seconds/ms, file/spec, test/scenario name.

Strategy	How
By file	One group per file containing ≥1 slow test
Batches of 3	Consecutive slow tests in rank order, 3 per group (last may be smaller)

Choose the strategy with fewer groups. Tie-break: prefer by file.

Read sub-project rules when editing tests: frontend.mdc / frontend-testing.mdc, backend.mdc / backend-testing.mdc, e2e-authoring.mdc, cli.mdc.

Each group slice (sub-agent):

1. Optimize only tests in that group (see optimize_tactics).
2. Verify with focused commands (see verify).
3. Return control for dough-execute-plan's required refactor, formatting, plan update, perf(<scope>): … commit, and push sequence.

Hard-to-improve → Candidates / skip tag: If no meaningful speedup after serious attempt, or would need product/design trade-off:

1. Do not force a weak change.
2. Append under Candidates in .planning/test-optimization-blacklist.md: file, test/scenario, duration, why hard, date (YYYY-MM-DD).
3. Mark slice done (or Jidoka-stop if value decision required).

Promoting a Candidate to permanent skip is a developer decision (Jidoka) — propose tagging the Scenario or Feature with @skipOptimizationDueToKnownNecessarySlowness; do not add the tag yourself.

Read and follow verification.md.

• Re-run same profile command as baseline (same --expose tags=… for E2E).
• Record: test count, suite wall, top-10 table, top-10% total CPU (Vitest) or sum of slow scenarios (E2E).
• Note any new Candidates proposed.
• If full E2E re-profile is red (e.g. Bad Gateway on cleanDB), document that; use per-spec timings + CI for authoritative "after" — do not fake a green wall time.

Set plan Status: done; clean up spent plan history (see planning_cleanup). Keep the Candidates blacklist file.

1. Do not keep two plans for the same scope — merge or delete duplicates.
2. Optionally keep one-line note in STATE/ROADMAP; otherwise delete spent quick/ PLAN and SUMMARY noise (see planning.mdc history cleanup).
3. Never commit profile JSON.
4. Leave .planning/test-optimization-blacklist.md and active GSD milestone artifacts untouched.

If user asks only to clean up: remove completed test-opt plans; do not delete blacklist or unrelated GSD dirs in progress.

<success_criteria> Optimize mode:

• Full-suite profile captured with E2E skip tag excluded via --expose tags
• Top 10% selected from eligible (profiled) tests
• Plan written and executed via dough-execute-plan (commit + push per group)
• Non-negotiable rules applied (no redundant tests left, no fixed waits, no flaky)
• Re-profile recorded; plan marked done; spent history cleaned
• Final output includes ## TEST OPTIMIZATION COMPLETE

Resolve mode (--resolve):

• Every Candidate resolved by exactly one of tag / plan / ask
• At most one plan created for all replacements
• Resolved Candidates deleted from the blacklist (no "Resolved" archive kept)
• No profiling or optimization performed
• Final output includes ## CANDIDATES RESOLVED </success_criteria>

1. Scope and baseline vs after metrics.
2. Groups optimized and commits made.
3. Candidates proposed (if any).
4. Planning cleanup performed.

## TEST OPTIMIZATION COMPLETE

Resolve mode (--resolve) — report per Candidate: the decision (tag / plan / ask) and its one-line rationale; the plan location if one was created; confirm the blacklist Candidates list was pruned.

## CANDIDATES RESOLVED

<out_of_scope>

• Do not optimize in the coordinator agent after plan is written.
• Do not commit profile JSON.
• In optimize mode, do not add @skipOptimizationDueToKnownNecessarySlowness without developer Jidoka (in --resolve mode you may tag directly per the resolve_candidates step).
• Do not add @focus / @only in committed code.
• Do not run full E2E suite for per-slice verify unless shared helpers require it. </out_of_scope>