CompatLab

CompatLab tests published npm artifacts across pinned JavaScript runtimes. Reports distinguish observed loading behavior, coverage, and environment limits from broader claims of compatibility.

Production domain: compatlab.me. Read the methodology before interpreting a passing report.

Agents can read existing evidence through the anonymous API or connect a remote MCP client to https://compatlab.me/mcp using Streamable HTTP. The read-only check_package and get_report tools return exact versions, runtime pins, coverage and report links; they never submit scans. See MCP setup and validation. Missing evidence is not a compatibility verdict.

The local engine resolves and prepares public npm artifacts with scripts disabled, seals their dependency tree, and probes them across pinned Node, Bun, and Deno runtimes. The CLI supports bounded checks and explicit snapshot reuse or lock-based rebuilds. The worker includes host ownership, capacity reservations, recovery and hostile-code qualification. The PostgreSQL catalog and private control service provide transactional admission, durable leases, authenticated result ingestion and snapshot-local dispatch. The anonymous website supports discovery, explicit scan requests, durable progress, report matrices, evidence and reproduction downloads. SSH operator controls, deployment packaging, encrypted off-host backups, retention and release-qualification gates are included; public admission defaults to disabled. See the operations runbook and qualification record. See website setup, reports, orchestration, catalog and admission, worker lifecycle, local execution, preparation, and runtime profiles for limits and prerequisites.

Development

Named assertions and CI artifacts extend the maintainer workflow. Assertions use commit-pinned offline fixtures and separate behavioral evidence. The Linux/runsc CLI can check a pre-publication archive with a distinct source identity and caller-supplied provenance.

The public maintainer section is coming soon. Its optional account implementation uses GitHub App login, encrypted OAuth tokens, live repository authority checks, revocation and account quotas, but sign-in and release monitoring stay disabled in this deployment. Public reports remain anonymous.

Use Node.js 24.21.0 from .node-version and pnpm 12.8.1 from package.json.

corepack enable
pnpm install --frozen-lockfile
pnpm check
pnpm cli --help
pnpm cli doctor --json

pnpm check runs formatting/lint checks, a strict workspace build, test type checking, and unit/CLI tests. Build before running the CLI or tests directly. Development checks work on macOS and Linux; a Docker daemon is needed for doctor, sandbox tests and the local PostgreSQL test server. See database qualification for the separate integration gate.

doctor reports whether a Linux amd64 Docker server has a registered runsc runtime. It returns exit code 1 when prerequisites are missing and never executes package code. Passing it does not qualify a host for untrusted execution. There is no fallback to running packages on the developer's machine.

On a prepared Linux amd64 host with runsc:

pnpm build
pnpm test:sandbox

The smoke test builds a digest-pinned fixture image and checks ESM/CommonJS completion, module failure, fake stdout success, missing results, and abnormal exit. It uses authored fixtures only. CI installs a checksum-pinned gVisor release on a disposable GitHub-hosted runner; that installer is not for development or production machines.

Repository structure

Path	Implemented responsibility
apps/cli	Prerequisites, checks, CI archives, reproduction, SSH administration and backup encryption
apps/web	Anonymous discovery, scan requests, durable progress and report pages
packages/engine	Registry resolution, analysis, planning and bounded probe orchestration
packages/catalog	PostgreSQL identities, admission, leases, result validation, recovery and cache lookup
packages/contracts	Canonical vocabulary and bounded completion validation
services/worker	Preparation, sealed storage, runtime supervision and local evidence storage
services/control	Private WireGuard-bound worker API and reconciliation
harnesses	Automatic loading and separate named assertion completion protocols
services/maintainer	Release reconciliation, comparison and independently retried email
runtime-images	Digest-pinned minimal runtime image recipe
fixtures	Authored module/protocol and preparation archive fixtures
infra	Pinned service packaging, worker provisioning, backup and recovery configuration
scripts	Containment, corpus, deployment and recovery qualification
docs	Architecture, delivery sequence, research, and decision records

Delivery and contribution

The fourteen-PR delivery plan covers the public MVP and gated maintainer workflows. The architecture plan, PRD v1.1, and research notes define the design. The original PRD is preserved as historical reference.

All project changes use feature branches and pull requests. See CONTRIBUTING.md for checks and review expectations, and SECURITY.md for reporting security issues. The repository is maintained by siddiksawani and licensed under MIT.

For operating the public site, see production operations and search discovery. The deferred maintainer release-monitoring workflow is described in monitoring and deployment.