Pipeline Context

Read these files before proceeding with pipeline work:

1. memory/pipeline-ops.md — Emergency controls, worker architecture, operational details
2. memory/lessons-learned.md — Operational postmortems and patterns
3. .claude/docs/pipeline.md — Full processing pipeline (states, crons, prompts, costs)
4. .claude/docs/worker-architecture.md — Lambda worker details
5. .claude/docs/page-lifecycle.md — Page processing states

Critical Rules

• Model selection: prefer getModelForBook(book) from src/lib/types/ai-models.ts over hardcoding. It routes BPH books and non-Latin-script languages to gemini-3-flash-preview (full quality) and everything else to gemini-3.1-flash-lite-preview (50% cheaper, comparable quality on Latin-script). Enrich-worker uses gemini-3.1-flash-lite-preview for all phases (summary+index, chapters, quality scoring, collection assignment). Never use anything below Gemini v3 — gemini-2.x is deprecated.
• NEVER use Gemini Batch API for translation — use Lambda workers (SQS FIFO)
• Any script overwriting ocr.data or translation.data MUST call createRevision() first
• MongoDB Atlas saturates at ~40 concurrent Lambda jobs — global backpressure limit
• Emergency stop: system_config._id: 'processing_control', set paused: true

Audit Trail

All AI calls logged to gemini_usage collection via logGeminiCall() in src/lib/gemini-logger.ts.

• Book history timeline: GET /api/books/[id]/history (assembles from 6 collections)
• Dashboard: GET /api/admin/processing-dashboard?provider=ia
• Error classification: src/lib/errors.ts → classifyError(error)
• cost_tracking collection is DEPRECATED — use gemini_usage for all cost queries

Staleness Check

After reading the memory files above, flag anything that contradicts what you observe in the codebase:

• File paths or function names that no longer exist
• Behavioral claims that don't match the current code
• Stats or counts with dates older than 14 days — note as potentially stale
• If you find contradictions, update the memory file immediately and tell the user what changed.

Also Relevant

• Batch processing (Gemini Batch API): .claude/docs/batch-processing.md
• Observability & audit trail: .claude/docs/observability.md
• First translation identification: .claude/docs/first-translation-system.md