E2E Test Runner

Architecture overview

• Tests live in e2e/*.spec.ts, config in playwright.config.ts
• Page objects live in e2e/pages/<feature>/ — every spec uses them; conventions in docs/dev/e2e-page-objects.md, gotchas in docs/dev/e2e-page-objects-migration.md
• Global setup (e2e/global-setup.ts) builds the app (skipped when no build input changed since the last e2e build — tracked via .e2e-build-marker), creates/migrates per-worker databases (via scripts/ensure-test-db.ts: pending migrations are applied, drifted databases are rebuilt), and starts one server per worker
• Port calculation: E2E_BASE_PORT = PORT (from .env) + 500. Worker N uses port base+N, except ports on the WHATWG fetch bad port list (e.g. 6679) are skipped — see e2eWorkerPort in e2e/helpers/playwright.ts
• Worker count: E2E_WORKERS env, defaulting to min(8, max(4, cores - 2))
• Worker databases: db-test-e2e-<N>.sqlite3 in the project root; every test starts from a wiped database holding only the admin (Sendou) and N-ZAP users, and builds its own data with the factories fixture
• SeaweedFS (S3-compatible storage) is started via Docker Compose if not already running

Pre-flight checks (run before every test execution)

Before running tests, check for these common issues:

1. Port conflicts — Check if anything is already listening on the e2e ports (base port + worker index):

   lsof -i :4501-4508 2>/dev/null
   

   If ports are occupied by leftover e2e servers, kill them. If occupied by something else, warn the user.

2. Docker running — SeaweedFS requires Docker. Check with docker info if there are storage-related failures.

Stale worker databases (db-test-e2e-*.sqlite3) are handled automatically: global setup applies pending migrations and rebuilds databases whose migration history has drifted.

Running tests

Run all tests

pnpm run test:e2e

Run a specific test file

pnpm exec playwright test e2e/<name>.spec.ts

Batch multiple files into one invocation — every invocation pays global setup.

Flaky detection (repeats each test 10 times, stops on first failure)

pnpm run test:e2e:flaky-detect

Force a rebuild of the app

E2E_FORCE_BUILD=true pnpm run test:e2e

Global setup reuses the previous build when nothing under app/, public/, the lockfile, or the vite/react-router configs changed. Use this override if you suspect a stale build (e.g. after changing env-dependent build behavior).

Debugging failures

Follow this funnel when tests fail:

Step 1: Read the error output

• Look for the actual assertion or timeout that failed
• Check if it's an infrastructure error (server didn't start, migration failed) vs. a test logic error

Step 2: Check infrastructure issues

Common infrastructure errors and fixes:

• "table already exists" → Should not happen anymore (global setup rebuilds drifted worker DBs); if it does, rm -f db-test-e2e-*.sqlite3 and investigate scripts/ensure-test-db.ts
• "Server on port X did not start within timeout" → Port conflict or app build error. Check ports with lsof -i :<port> and check for build errors
• "SeaweedFS failed to start" → Docker not running or compose issue. Check docker info
• "Test ended with database writes the server never saw" → A factory call was not followed by a helper that talks to the server; add a navigate/impersonate after the writes

Step 3: Reduce to single debug worker

If the error is unclear, re-run with debug output and a single worker to see server logs:

E2E_DEBUG=true E2E_WORKERS=1 pnpm exec playwright test e2e/<failing-test>.spec.ts

This shows stdout/stderr from the test server, which is hidden by default.

Step 4: Examine trace artifacts

Playwright is configured with trace: { mode: "retain-on-failure", snapshots: false } — failure traces carry screenshots, network and action logs but no DOM snapshots (recording those cost ~17% of total suite time). After a failure, check test-results/<test-folder>/error-context.md for the page's accessibility snapshot at failure time, or view the trace:

pnpm exec playwright show-trace test-results/<test-folder>/trace.zip

When a failure needs full DOM snapshots to understand, re-run just that test with --trace retain-on-failure (the CLI flag records complete traces).

Re-render races

Live events run on an in-process event bus per worker server (SSE, see app/features/events), so cross-worker crosstalk cannot cause flakes. Google Fonts are also blocked at the context level so font swaps never reflow the page mid-test. Re-renders from the test's own action revalidations can still swallow a React Aria press (press start registers, press end never fires — no POST); waitForPOSTResponse retries for this, so route flows through it rather than adding sleeps.

Other known sources, each handled once in the helpers or page objects:

• YouTube is stubbed at the context level (a fake iframe_api, everything else aborted): the real player loads from the internet at its own pace and, arriving mid-test, closed an open select on the VoD form
• dnd-kit stops every click in the document for 50ms after a drop; drag helpers end with waitForDropToSettle so the next click lands
• A tab or link clicked before hydration (after a raw page.reload(), say) loads the target as a new document and the click after it is lost; reload through the page object's reload() which waits for hydration
• Popovers close on navigation in a passive effect, so for a frame the old panel and the new page both show the same names; scope locators to main where a name can appear in both
• React Router drops a fetcher's redirect when a navigation (a revalidation included) started after the submission; broadcast revalidations are jittered up to 1.5s after a live event, so one could land mid-submission and the action silently did nothing. holdRevalidationsDuring (used by useActionSubmit and SendouForm) defers them until the submission settles

Test pattern reference

Every test builds its own data with factories and drives the UI through page objects:

import { NZAP_TEST_ID } from "~/db/seed/constants";
import { expect, impersonate, test } from "./helpers/playwright";
import { BuildsPage } from "./pages/builds/builds-page";

test.describe("Feature", () => {
	test("does something", async ({ page, factories }) => {
		await factories.BuildFactory.create({ ownerId: NZAP_TEST_ID });

		await impersonate(page, NZAP_TEST_ID);
		const builds = new BuildsPage(page);
		await builds.goto();
		// ... interact via page object methods, assert in the spec ...
	});
});

Key rules:

• The database starts each test holding only the admin and N-ZAP; the factories worker fixture (see e2e/helpers/factories.ts for the registry) creates everything else
• Locators live in page objects under e2e/pages/ — specs contain no raw getByTestId/getByRole calls; see docs/dev/e2e-page-objects.md
• Use navigate() instead of page.goto() — it waits for hydration (page objects' goto() methods wrap it)
• Use submit() instead of clicking submit buttons directly — it waits for the POST response
• Use impersonate(page, userId?) to authenticate. Default is admin (ADMIN_ID); prefer N-ZAP (NZAP_TEST_ID) when the flow doesn't need admin rights
• Avoid page.waitForTimeout — use assertions or waitFor patterns instead (the one exception is waitForDropToSettle, dnd-kit's post-drop window has nothing observable to wait on)
• Import test from ./helpers/playwright (not from @playwright/test) — it includes worker port fixtures and the database reset
• Factory writes must be followed by a helper that talks to the server (navigate, impersonate, submit) or the test fails with "writes the server never saw"

Environment variables

Variable	Purpose	Default
E2E_WORKERS	Number of parallel workers	min(8, max(4, cores − 2))
E2E_DEBUG	Show server stdout/stderr when "true"	unset
E2E_FORCE_BUILD	Rebuild the app even when inputs look unchanged	unset
PORT	Base port for dev server (e2e adds 500)	5173