CI Forensics

SkillMonitoring & ops

This skill should be used when reading CI logs and diagnosing failures, fetching Actions logs through the GitHub MCP, classifying failures by cause, and distinguishing flaky tests from real regressions.

Available today. Use it from your connected AI after setup.

No other account needed.

Connect ahel once, and every AI you use reads what you have installed.

Then ask your AI: use the CI Forensics skill

What this skill tells your AI

The instructions your AI receives, as published by thelobbi/claude in plugins/delivery-orchestrator/skills/ci-forensics/SKILL.md and read by ahel’s review.

Fetching logs — the correct sequence

1. mcp__github__actions_list   method: "list_workflow_runs", branch: <head ref>
2. mcp__github__get_job_logs   run_id: <run>, failed_only: true
3. mcp__github__get_job_logs   job_id: <job>, return_content: true, tail_lines: 200

Known pitfalls in these tools:

actions_list and actions_get are multiplexed behind a method discriminator. Calling either without method errors.
actions_get also needs resource_id (not run_id) and returns metadata, not logs. It is the wrong tool for log triage.
get_check_run fetches one check run by checkRunId; it has no list mode. For "is CI green on this sha", use actions_list → list_workflow_runs and filter by head_sha.
list_workflow_runs can exceed the token limit and be spilled to a file. Parse that file with node -e, do not read it inline.

Failure taxonomy

Classify before changing anything. Fixing an infra flake by editing source code produces a red PR with extra churn.

Class	Signals	Action
Real regression	Traces to a file/line in the diff; reproduces locally; the test is new or newly touched	Fix + regression test
Flaky	Same commit sha, different result; timing/ordering/port/network in the trace	Confirm across ≥3 runs, then quarantine
Infra	Runner OOM, disk full, image pull failure, registry 5xx, action download timeout	Re-run. Escalate after 2. Never edit code
Base broken	Same job red on the base at a commit predating the branch point	Report once, wait for recovery
Config drift	Failure in workflow YAML, action inputs, or a missing secret	Fix the workflow
Dependency	Appeared with no source change; a transitive package published	Pin or roll back

Establishing "base broken"

Do not infer it from error text. Check the base branch's own runs for the same job at a commit predating this branch's point of divergence. Red there too → base breakage. Otherwise it is yours.

Flaky vs real

Evidence	Strength
Same commit sha, different results	Conclusive
Passed on a neighbouring run, no relevant change	Strong
Fails only under parallel execution	Strong — ordering or shared state
Fails only on one OS or runtime version	Medium — environment dependence
"It passed when I re-ran it"	Weak — never sufficient alone

A test that has never passed is not flaky. It is broken.

Failure rate hints at mechanism: ~50% is usually an async race; ~2% is usually resource pressure or a real-time dependency.

Sources of nondeterminism

Real time · unseeded randomness · test ordering and shared module state · fixed ports · shared database rows · reused temp paths · missing awaits · real network calls · resource pressure.

Never

Never disable, skip, or .only a test to make CI pass. .only silently disables the rest of the file.
Never re-run a job to make a failure disappear without recording why it failed.
Never claim CI is fixed without a green run to point at.

Signals

GitHub stars: 21
Forks: 2
Last commit: Sep 2026

Advanced

Catalog kind: skill
Gateway key: ci-forensics
Source: github.com/thelobbi/claude

github.com/thelobbi/claude

More in Monitoring & ops

internal-comms
Skill · anthropics agent-eval
Skill · affaan-m architecture-decision-records
Skill · affaan-m dashboard-builder
Skill · affaan-m django-celery
Skill · affaan-m healthcare-cdss-patterns
Skill · affaan-m