GateTest

One gate. 122 modules. Self-healing CI.

AI-powered code quality. Pay per scan via Stripe.

---

The 30-second pitch

GateTest is a single CLI plus a composite GitHub Action that runs 122 static-analysis modules against any codebase, then uses an AI fix engine to repair the findings it can. It replaces SonarQube, Snyk, ESLint, Cypress, Lighthouse, axe, pa11y, and twenty-plus other tools with one config, one gate decision, and one report.

It is different because the cost trends to zero. Deterministic AST and rule-based layers run first — these are free and ship the fix in milliseconds. The AI layer only runs on patterns nothing else has seen. Every AI win is distilled into a reusable recipe, so the next time the same pattern appears anywhere in the network it is handled for free. The longer you run GateTest, the less of it is paid work.

What you get depends on the tier. A pull request with the fixes, regression tests pinned to each fix, an architecture-shape critique, a cross-finding attack-chain analysis, and a CTO-readable executive summary — in whichever combination the tier you bought includes. One-time payment per scan via Stripe at checkout. No subscription, no auto-renew.

---

Install & Usage — 30 seconds

GitHub Action — recommended for most users

Drop this in .github/workflows/gatetest.yml:

name: GateTest Quality Gate
on: [push, pull_request]
jobs:
  gate:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      # Optional: powers the PR summary comment, inline suggestions and
      # auto-repair PRs. Without it the gate still runs and blocks on
      # findings — the comment and suggestions are skipped, with a warning
      # in the log, and no PR opens even if auto-fix finds something to fix.
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
      - uses: crclabs-hq/GateTest@v1
        with:
          suite: full
          auto-fix: ${{ github.event_name == 'pull_request' }}
        env:
          # Optional: unlocks auto-fix and AI review. Without it the gate
          # still runs and blocks on findings — CI just doesn't open a fix PR.
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

The action is a composite — no Docker pull, no container build. It installs GateTest, runs the gate, and if auto-fix: true and ANTHROPIC_API_KEY is set, runs the AI repair loop on a blocking gate. See action.yml for every input.

The action authenticates with the workflow's own token by default (github-token input, ${{ github.token }}), so the permissions: block above is all it needs: without pull-requests: write the summary comment and suggestions are skipped, with a warning in the log. Add issues: write if you turn on track-non-fixable: true, and security-events: write (plus actions: read) if you upload the --sarif report to the Security tab with github/codeql-action/upload-sarif (see Wire it into CI below).

Your first full run passes. Turning a gate on against an existing codebase would otherwise fail on years of backlog nobody wrote this week, so a full-repo run that finds no .gatetest/baseline.json snapshots what is already there and exits green. Commit that file and every run after it fails on new findings only — pull requests are judged on the files they change from the very first run. Details under baseline mode.

Editor — VS Code, Cursor, Windsurf, VSCodium

The whole engine in your Problems panel, before you commit. Every finding lands on the line that caused it, with the fix. Free, no account, and nothing leaves your machine.

• VS Code: search GateTest in the Extensions view, or install from the Visual Studio Marketplace.
• Cursor, Windsurf, VSCodium, Gitpod, Eclipse Theia: search GateTest in the Extensions view, or install from Open VSX — the registry those editors read. Same build, published on the same run.

Source and the full command reference: vscode-extension/.

CLI — local development

# Install from npm:
npm install -g @gatetest/cli
gatetest --suite quick

# Or run against the current directory with no install:
npx --yes @gatetest/cli --suite quick

# Or clone and run from source:
git clone https://github.com/crclabs-hq/GateTest
cd GateTest && npm install
node bin/gatetest.js --suite quick

Pre-push sweep

Run the full pre-merge sweep locally in one command:

npm run sweep          # ~30-60s — tests + build + gate + secrets + self-scan

This runs the same seven checks that block a merge in CI. Verdict is green or red. Exit code is 0 or 1, matching CI exactly.

Fast path during iteration:

npm run sweep -- --fast    # skip tests + build, gate-only, ~3-5s

See gatetest sweep --help for every flag.

Silencing a false positive — 10 seconds

Every scanner gets it wrong sometimes. When GateTest flags something you've judged safe, add one line to a .gatetestignore file at your repo root:

# Silence one rule from one module:
secrets:generic-api-key

# Silence a whole module:
deadCode

# Silence a rule everywhere it fires:
*:trailing-whitespace

# Scope a suppression to a path:
secrets:generic-api-key@tests/fixtures/**

# Skip a path entirely:
vendor/**

Suppressed findings are excluded from the gate decision and every failure count, but stay visible in a suppressedChecks list — nothing is silently hidden. Two more controls:

• gatetest --noise — ranks your noisiest modules and prints the exact ignore line to copy. The same signal, aggregated across every opted-in scan, is published rule by rule at gatetest.io/noise.
• Auto-softening — a module you chronically dismiss stops blocking the gate on its own (never on thin evidence: it takes repeated dismissals at a high fire-rate).

Accepting a real risk — recorded, expiring

.gatetestignore is for a false positive: "this rule is wrong about my repo." It is silent (no reason, no expiry) and permanent — the right tool when the finding shouldn't exist at all.

An accepted risk is for the other case: the finding is real, you accept it anyway, and you want that decision on the record with a reason and a review date — not a bypass nobody can see:

gatetest --accept-risk secrets:apiKey:src/legacy-client.js:42 \
  --reason "internal tool behind VPN, rotation ticketed for Q1" \
  --until 2026-12-31 --by craig --persist

--accept-risk <finding-id> takes the same id --format json prints as issues[].id (<module>:<check>); repeat the whole group for more than one finding. --reason is required — without it the override is a loud warning (exit 2 under --strict or in CI) and is never applied. --until is the review date: past it, the finding blocks again, with a message naming the expired override — an accepted risk that nobody revisits is exactly the UNRECORDED bypass this feature exists to prevent. --persist writes the override into .gatetest/accepted-risks.json, an array of { id, reason, by, until, created } committed and reviewed like any other file; without --persist the override applies to this run only.

An accepted risk never disappears from the report the way a .gatetestignore suppression does — it stops blocking, but stays visible in a separate overrides array in the JSON report, in the PR comment's "Accepted risks" section, and as a SARIF suppressions entry (so GitHub's Security tab shows it dismissed with your reason, not silently absent).

	.gatetestignore	Accepted risk
The finding is	wrong (false positive)	real, accepted on purpose
Reason required?	no	yes
Expires?	never	optional --until
Visible after applied?	in suppressedChecks only	in the report, PR comment, and SARIF

The policy is reviewed as policy. .gatetest.json and .gatetestignore are what every later PR is judged by, so a PR that changes them says so: a suppression added to .gatetestignore, a module disabled, the gate set to report-only or the block threshold raised in .gatetest.json each produce a Gate policy changed warning on that PR — reported, never blocking, quiet on comments and on tightening. Every signed report records the SHA-256 of both files (gatetest verify-report prints them), so two reports that disagree can be told apart by policy, not only by engine.

Project-wide options live in .gatetest.json (suites, per-module config, severity overrides) — run gatetest --init to scaffold one.

Deterministic vs. model-judged findings

GateTest runs two kinds of checks. Most modules (secrets, syntax, lint, crossFileTaint, and the rest of the deterministic majority) are rules: given the same input they always produce the same verdict. A handful of modules (aiReview, agentic, architectureDrift, intentVerification, regressionPredictor, and the AI-engine half of fakeFixDetector) ask a model to judge the code — a different kind of evidence, and one that shouldn't be weighted the same as a rule firing.

Every finding carries a verdictSource: deterministic, model, or mixed. A model-judged finding never blocks the gate by default — it's reported as a warning, with wouldBlock: true preserved on the finding so you can see what a stricter policy would have decided. Opt model-judged findings INTO blocking with:

gatetest --suite full --model-verdicts-block

or .gatetest.json:

{ "gate": { "modelVerdictsBlock": true } }

or the environment variable GATETEST_MODEL_VERDICTS_BLOCK=1. Every surface shows the split: the console summary prints deterministic vs. model-judged blocking counts, the JSON report tags each finding's verdictSource, SARIF carries it under properties.verdictSource for the GitHub Security tab, and the PR comment labels model-judged findings so a reviewer knows which alerts are a rule and which are an opinion.

Reporting a false positive

Silencing a finding hides it from your gate; reporting it is how the rule itself gets fixed for everyone. gatetest report-fp <module:rule> --reason "<text>" prints a prefilled GitHub issue URL — it never sends anything on its own, you review and submit it. The same form is at .github/ISSUE_TEMPLATE/false-positive.yml if you'd rather open it directly. Every retraction ships with a control-pair test pinning the exact line that should stay quiet, and the running count — reported, retracted, median hours to a fix — is published at gatetest.io/trust.

Onboarding a mature repo — baseline mode

Turning a scanner on against a large existing codebase usually means drowning in a backlog you didn't write. GateTest's baseline mode grandfathers everything that exists today so the gate only ever fails on new findings — "clean as you code."

# Guided first run: scans, captures .gatetest/baseline.json, and prints the
# per-module count, the .gatetest.json snippet for a grace period, and the
# exact CI line — everything below, without piecing it together yourself.
gatetest baseline --init

# Or the plain flag, if you just want the snapshot with no recap:
gatetest --baseline

# From now on, normal runs pass on the pre-existing findings and only
# block on NEW ones. Baselined findings stay visible, never hidden.
gatetest --suite full

Fix a baselined finding and it's gone for good; the count is tracked per file, so adding a second secret to a file that already had one baselined re-blocks the gate (you can't sneak a new problem in behind an old one). Refresh the snapshot after paying down debt with gatetest --baseline (or re-run the wizard); delete .gatetest/baseline.json to see everything again.

A grace period while the team triages — --report-only-until

--report-only (below) is advisory forever — useful locally, a bad CI default, since nobody comes back to remove it and a gate that can never turn red is not a gate. --report-only-until <YYYY-MM-DD> is the time-boxed version: advisory up to (not including) that UTC date, then automatically ignored — the gate starts enforcing on its own, no second PR to flip a flag.

gatetest --suite full --report-only-until 2026-10-15

or .gatetest.json (the flag wins when both are set):

{ "reportOnlyUntil": "2026-10-15" }

--strict wins over either. Every surface says which mode ran and why: the console line ("report-only until 2026-10-15 (19 days left) — findings are shown, nothing blocks"), --format json's enforcing / reportOnlyUntil fields, and the PR comment, whose grade badge never wears the same green tick as a real pass when a finding would have blocked under enforcement. An invalid date is a usage error (exit 2, UTC ISO dates only); a date that has already passed prints one warning and enforces. gatetest baseline --init suggests a date 14 days out.

Testing pages behind a login — authenticated crawl

The live crawler can carry a session so it reaches authed areas (/dashboard/*, account pages) instead of bouncing off the login redirect:

# A header (repeatable), a cookie, or an exported browser session —
# values support ${ENV_VAR} so secrets stay out of committed config:
gatetest --crawl https://app.example.com --crawl-header "Authorization: Bearer ${TOKEN}"
gatetest --crawl https://app.example.com --crawl-cookie "session=${SESSION}"
gatetest --crawl https://app.example.com --crawl-storage-state state.json

Session material is only ever sent to the target's own origin — never to third-party links, assets, or cross-origin redirects. Without a session, a crawl that hits a login wall tells you exactly which flag to add rather than silently skipping the protected pages. The hosted scanner at gatetest.io accepts the same session auth.

Claude Code / MCP — give your agent eyes, ears & hands

Connect GateTest directly to Claude Code (or any MCP-compatible AI) in one command:

claude mcp add gatetest -- npx -y @gatetest/mcp-server

24 tools across five families:

Family	Tools	What it gives the agent
Engine	scan_local, run_module, fix_issue, verify_fix, …	Scan + fix local code
👁 Eyes	capture_screenshot, get_visual_diff	See the rendered page as a real image
👂 Ears	get_production_errors, run_live_checks	Hear Sentry/Datadog/Rollbar errors + localhost runtime failures
🤝 Hands	verify_fix	Hard ✅/❌ — prove the fix actually worked
🔬 Root Cause	resolve_stack_trace, blame_regression	Resolve a minified stack trace to original file:line via source maps; find the git commit that introduced a specific line. Same engines are also CLI subcommands (gatetest trace, gatetest blame) — one implementation, both entry points

Works with Claude Code, Cursor, Windsurf, Continue, and Cline. See packages/mcp-server/ for the full tool reference and example prompts.

Website — no install at all

Visit gatetest.io/web and paste any URL. You get a free preview and a paid full report. For WordPress sites use gatetest.io/wp.

Wire it into CI — GitHub, GitLab, or CircleCI

Don't hand-write the pipeline. One command scaffolds a complete, conventional config:

gatetest --ci-init github     # .github/workflows/gatetest.yml
gatetest --ci-init gitlab     # .gitlab-ci.yml
gatetest --ci-init circleci   # .circleci/config.yml

Each generated config gates the right thing at the right time rather than running everything everywhere: a quick, diff-scoped scan on merge requests and pull requests, a full scan on the main branch, and a separate security stage. JUnit and SARIF are emitted to .gatetest/reports/ and wired into the platform's native test-reporting and artifact storage, so failures show up in the UI instead of only in the log.

On any other CI — Jenkins, Buildkite, Bitbucket, Drone — the CLI is the whole integration:

npx --yes @gatetest/cli --suite full --junit --sarif

Onboarding an existing codebase? Pair this with baseline mode above so the gate only fails on new findings.

Merge queues and monorepos

Merge queues. The GitHub Action and the drop-in workflow handle the merge_group event: each group is scanned diff-scoped against the queue's base (the event payload's base_sha, which the engine resolves through one shared base resolver — the same one --pr, prSize and the fake-fix detector use, so no module measures a different diff from another). Add merge_group: under on: in your workflow and nothing else changes.

Path filters. In a monorepo, scope the gate to the packages it owns in .gatetest.json:

{ "paths": { "include": ["packages/api", "packages/shared/**"], "exclude": ["**/fixtures/**"] } }

A bare directory means everything under it; * is one segment, ** any depth; exclude wins. The filter applies at the one file walk every module shares, findings from modules with their own lookups are dropped at the runner, and every report says so — Scope: .gatetest.json paths — include packages/api (3 finding(s) outside it not shown) — and carries it in the signed provenance. No paths key, no filter.

Replay a failing CI run locally

Reproduce any failing GitHub Actions run on your laptop in seconds:

gatetest replay https://github.com/<owner>/<repo>/actions/runs/<run-id>

This fetches the run, identifies which steps failed, and runs them locally against your current working tree. Output tells you whether the failure reproduces, doesn't reproduce (flaky CI), or hits a different error.

Authentication is optional — if you have a GITHUB_TOKEN set or gh CLI installed, replay can read private repo runs. Otherwise it uses the unauthenticated rate limit (60 req/hour, fine for a few replays).

When a gate is blocked inside GitHub Actions, the log and the checks tab already carry this command with the run's URL filled in.

Self-hosted and air-gapped

The engine is an npm package with four runtime dependencies that reads your tree and writes to .gatetest/. The only thing that can leave the machine is the anonymized telemetry flush (module and rule ids with integer counts, never code, paths or repo names). Switch it with GATETEST_TELEMETRY=1|0 or "telemetry": true|false in .gatetest.json (GATETEST_NO_TELEMETRY=1 still works as an alias for off); gatetest --telemetry-status prints the current setting, which switch decided it and the host. The first run prints exactly what is sent, once. Uploads go only to gatetest.io: a GATETEST_TELEMETRY_URL (or a base-URL override) on any other host is refused unless you set GATETEST_TELEMETRY_ALLOW_HOST=1, which is how a self-hoster points it at their own ingest. The AI-backed fix paths are opt-in and need ANTHROPIC_API_KEY. For an air-gapped runner, make that a stated promise:

gatetest --suite full --offline        # or GATETEST_OFFLINE=1

Under --offline nothing leaves the machine: no telemetry upload, no AI calls (--fix / --auto-pr are refused with a message, gatetest fix exits 2), no live API ping from --doctor. The console prints the mode, the summary carries offline: true, and the signed provenance records it — so a report produced inside the perimeter can be verified outside it with gatetest verify-report and the key. There is no licence server and no account; nothing expires.

The JSON report is a versioned contract

Every JSON report carries a top-level schemaVersion; the stable fields, the version rules and the deprecation policy are in docs/api/report-schema.md, pinned by tests/report-schema-contract.test.js.

Redirecting or disabling report output

Every scan writes reports (.gatetest/reports/) and two memory stores (.gatetest/memory.json, .gatetest/memory/) into the scanned checkout by default — fine for a normal dev machine, not for a CI runner, a monorepo, or anyone scanning a read-only tree, where it leaves git status dirty with no way to opt out.

# Redirect everything (reports + memory) to one path instead:
gatetest --suite full --report-dir /tmp/gatetest-out    # or GATETEST_REPORT_DIR=/tmp/gatetest-out

# Or write nothing to disk at all — the console summary and --format json's
# stdout document are unaffected, exit codes unchanged:
gatetest --suite full --no-artifacts                    # or GATETEST_NO_ARTIFACTS=1

--report-dir wins over the env var, which wins over .gatetest.json's reporting.outputDir, which wins over the default. If you keep the default location, add .gatetest/ to that repo's .gitignore — GateTest prints a one-line stderr hint after the summary the first time it notices that repo's own .gitignore doesn't cover it (never in --format json mode, never with --no-artifacts).

Flaky tests: measured, quarantined, and they expire

The flakyTests module reads test source for the shapes that flake. The engine also measures it: each time it runs your tests (unitTests, when the runner is node --test — TAP or spec output), it records every test's pass/fail per run in .gatetest/memory.json (same consent switch as telemetry; test names are hashed, never stored or uploaded). A test that passed and failed on the same commit, or flipped 2 or more times in its last 10 runs, is flaky.

A flaky test's failure is reported as a warning — quarantined flaky test (flipped 3 of last 10 runs) — instead of blocking, and is listed in the console, the JSON report (flaky[], summary.flake) and the PR comment. The quarantine lasts 14 days from the first time the test was flagged; after that the failure blocks again unless the flake is fixed. A test that fails on every run is never quarantined — that is a real failure.

gatetest --suite standard --no-quarantine   # block on every failing test again

Tune it in .gatetest.json: "flaky": { "flips": 2, "window": 10, "quarantineDays": 14, "quarantine": true }. The README badge gets a last segment, flake N%, once a run is on record, and flake not measured before that.

Gitignored paths and build output