@parcel/watcher integration

@parcel/watcher is Kolu's default filesystem watcher. Reach for it instead of chokidar, raw fs.watch, or hand-rolled polling whenever a feature needs to observe a directory subtree. Today's only consumer is the working-tree watcher (packages/integrations/git/src/working-tree-watcher.ts) — the git-dir watchers (head-watcher, reflog-watcher, index-watcher) use plain fs.watch via kolu-io's refcounted-dir-watcher.ts because they target a single known file inside .git/, where parcel-watcher's recursive model would be overkill. New fs-monitoring code should default to parcel-watcher unless it has a similarly narrow target.

Backend dispatch

Source: node_modules/.pnpm/@parcel+watcher@2.5.6/node_modules/@parcel/watcher/src/Backend.cc:30-69.

Kolu pins the OS-native backend explicitly (inotify / fs-events / windows, chosen off process.platform — see PARCEL_BACKEND in working-tree-watcher.ts). It does not use backend: "default". The pin skips the auto-dispatch order below — most importantly the WatchmanBackend probe, whose per-subscribe popen leaks a zombie on watchman-less hosts (see next section). If you write a new parcel consumer, pin the backend the same way.

backend: "default" would select in this order, first match wins:

1. FSEvents on macOS — native recursive, one stream per repo.
2. WatchmanBackend if WatchmanBackend::checkAvailable() returns true.
3. WindowsBackend on Windows — native recursive ReadDirectoryChangesW.
4. InotifyBackend on Linux — one inotify slot per non-ignored directory.
5. KqueueBackend on BSD.
6. BruteForceBackend — periodic full-tree stat; the fallback fallback.

So under "default" on Linux, watchman is probed (and preferred if reachable) before inotify; on macOS watchman is never used (FSEvents wins). The explicit pin lands directly on step 1/3/4 for the platform and never runs step 2's probe.

How parcel invokes watchman

Source: src/watchman/WatchmanBackend.cc.

1. checkAvailable() (line 107) just calls watchmanConnect().

2. getSockPath() (line 43) does the only watchman binary invocation:

   popen("watchman --output-encoding=bser get-sockname 2>/dev/null", "r")
   

   then parses BSER output for the sockname field. If WATCHMAN_SOCK env var is set, that wins and the binary isn't run at all.

   ⚠ Zombie leak — why Kolu pins the backend (juspay/kolu#1691). On a host without watchman, this popen forks /bin/sh, the watchman exec fails, and parcel's error path returns without pclose() — so the sh is never wait()ed. popen bypasses child_process, so Node's libuv can't reap it either: it lingers as a zombie forever, one per subscribe. Free on CPU/RAM but a long-lived daemon (a remote-bound padi serving a full e2e run) accumulates dozens and the event loop drags. Pinning the native backend (above) skips checkAvailable() entirely, so the popen never runs. Verified: per-subscribe leak → 0.

3. From there it's a Unix-domain socket carrying BSER-encoded JSON. No more subprocess spawns.

Commands (BSER arrays sent over the socket):

Source	Command	Purpose
watchmanWatch (line 100)	["watch", "/abs/dir"]	start tracking
clock (line 230)	["clock", "/abs/dir"]	get baseline clock token
subscribe (line 281)	["subscribe", "/abs/dir", "parcel-<ptr>", {fields,since,expression}]	start receiving events
unsubscribe (line 330)	["unsubscribe", "/abs/dir", id]	tear down

Subscription IDs are "parcel-" + hex(watcher-pointer). Event mapping in handleFiles (line 137):

• new && exists → create
• exists && !S_ISDIR(mode) → update
• !new && !exists → remove

Ignore globs are translated to a watchman expression ["not", ["anyof", ["dirname", rel], ...]] only for ignores that are direct subpaths of the watched root (line 300). Glob-style ignores like **/node_modules are filtered client-side after events arrive — pass them in the ignore option but don't expect watchman-side filtering.

Cookie files (.watchman-cookie-*)

Written by the watchman daemon, not parcel-watcher. The daemon drops these files inside watched dirs to verify it can observe its own writes (watch/query commands trigger the dance). Normally ephemeral. Stragglers mean the daemon was killed mid-handshake. Add .watchman-cookie-* to .gitignore if watchman is in use.

Kolu's runtime status

As of #788, Kolu does not ship watchman with the production binary. The nix run wrapper (default.nix:156) only adds nodejs git gh to PATH — so even under backend: "default" the watchman probe would always fail and parcel would fall through to inotify on Linux / FSEvents on macOS. Since #1692, Kolu doesn't rely on that fall-through: it pins the native backend, so checkAvailable() (and its leaking popen) never runs at all. Issue #788 tracks the watchman integration work; if it lands, the pin is where you'd re-enable watchman deliberately (with a WATCHMAN_SOCK that avoids the probe).

Kolu's wrapper invariants

packages/integrations/git/src/working-tree-watcher.ts:

• Refcounted shared singleton per repoRoot — N callers → one parcel subscription, listener-side filtering by optional filePath.
• Trailing-edge debounce at WATCHER_DEBOUNCE_MS (150ms) coalesces bursts into one fire per listener.
• Hard-coded IGNORE_GLOBS — .git, node_modules, dist, build, target, .next, .turbo, .cache, .parcel-cache, .kolu-dev, .kolu-state, .DS_Store. Not gitignore-aware. Over-firing on user build outputs is absorbed by the snapshot-equality check in streamSnapshots(...) upstream.
• Post-install reconciliation tick (line 176) — parcelSubscribe is async. Filesystem mutations between subscribe() call and parcel resolving are invisible to parcel; the streaming endpoint already yielded its initial snapshot. The reconciliation fires every current listener once parcel is ready, so consumers re-read state and catch the missed window. Without this the client sees a stale view that no future event corrects.
• cancelled guard on the .then — if the last subscriber unsubscribed before parcel resolved, late-unsubscribe the AsyncSubscription instead of storing it.
• Per-repoRoot call chain (parcelCallChains) — every subscribe and unsubscribe for a repo runs one at a time, in issue order. Never let two parcel calls for the same directory overlap; see the hazard below.

What Kolu logs

Watcher lifecycle through Logger (kolu-shared). Grep these strings to verify the watcher came up in production:

• info git: working-tree watcher installed ← parcel resolved successfully
• info git: working-tree watcher retired ← last subscriber gone
• error git: working-tree watcher install failed ← parcel subscribe() rejected
• error git: working-tree watcher callback error ← parcel reported event-stream error
• error git: working-tree late-unsubscribe failed
• error git: working-tree unsubscribe failed
• error git: working-tree listener threw

Individual filesystem events are not logged — too noisy. The bucket-and- debounce path swallows event paths silently.

Failure modes worth knowing

1. Container/WSL2 bind mounts — inotify/FSEvents may be unavailable. Nuance since the backend pin (#1692): BruteForceBackend (~1s polling) is parcel's compile-time-last-resort backend — under "default" it's reached only when no earlier backend is compiled in, not when a compiled one fails to construct at runtime. On Kolu's Linux build INOTIFY is compiled, so "default" selects InotifyBackend (after the failed watchman probe), and a bind mount that can't inotify makes both "default" and the pinned "inotify" fail identically (error git: working-tree watcher install failed) — the pin forgoes no polling path there. On the three targets the pin is behaviorally identical to "default"'s post-probe outcome, minus the leak. On an untargeted platform (not linux/darwin/win32) the pin throws at module init (unsupported platform …) rather than degrade to a silent poller — kolu targets exactly those three, and the repo's fail-fast philosophy forbids a graceful-degradation fallback. (If a BSD/other target ever became real, map it to its native backend — kqueue etc. — not to a poller.) Net: leak-free everywhere, and no platform silently degrades.

2. Linux inotify slot exhaustion — kernel default is fs.inotify.max_user_watches=8192. A typical Kolu repo uses ~500–2000 slots; multiple worktrees compound. Watchman amortizes this across one daemon (#788).

3. detect-libc glibc/musl picker runs on require('@parcel/watcher') (index.js:5). The Nix build needs the matching native binary in node_modules — confirm @parcel/watcher-linux-x64-glibc is present under node_modules/.pnpm/.

4. Overlapping subscribe/unsubscribe for the same key silently kills the watcher (juspay/kolu#2065) — the worst failure mode here, because nothing reports it. Parcel keys its process-global Watcher registry (Watcher::getShared) AND its backend subscription set (Backend::watch / Backend::unwatch) on (dir, ignorePaths, ignoreGlobs), and runs each call's backend half on a libuv threadpool thread. Two overlapping calls for the same key therefore land in arbitrary order, and one order loses: the new subscribe finds the retiring watcher still registered (equal by key) and installs no OS watches, then the retiring unsubscribe tears the existing ones down. The new subscription resolves, holds its callbacks, logs watcher installed, and receives nothing, forever.

   Measured on @parcel/watcher@2.5.6: an un-awaited unsubscribe() followed by an immediate re-subscribe() of the same dir + ignore set produced a dead watcher in 18/25 idle runs, and 10/10 when the libuv pool was saturated (at rebuild delays from 15ms to 200ms). Awaiting the teardown first: 0/25 and 0/10. On an idle process the teardown usually wins on its own, which is why this hides in single-scenario runs and only bites a loaded parallel suite.

   Rule: serialize a directory's parcel calls (kolu does this in sequenceParcelCall). Pinned by working-tree-watcher.churn.test.ts, which loads the threadpool to force the lossy ordering — on linux/inotify only, for the reason below.

5. macOS: delivery can run ~15s behind, and parcel's stream is the reason. Parcel's startStream (src/macos/FSEventsBackend.cc) creates the stream without kFSEventStreamCreateFlagNoDefer, so the first event's delivery is deferred rather than immediate. On a volume under constant churn the defer window collapses to the daemon's maximum coalescing window. Measured on kolu's darwin CI box (ci@petit, macOS 26.5.2, juspay/kolu#2175): first event 14.2s after the write, then batches every 15.0s, 53/53 events delivered — nothing lost, everything late. fs.watch in the same process, on the same directory, answered in 1.3s; libuv passes NoDefer. That box's fseventsd was also 21 days up, 6.5 GB resident, ~190% CPU — worth checking before concluding anything about a healthy mac.

   Two consequences. A test may not gate on a parcel event arriving on darwin (fs-watch-delivery.testlib.ts is the one flag that says so; #2175 also records a rarer case where a post-churn subscription delivered nothing at all for 60s+, reproduced with no kolu code in the loop). And on a macOS host whose filesystem is this busy, kolu's own Code tab is that far behind — worth remembering before chasing a "stale git status" report as a kolu bug.

6. dontFixup = true in default.nix skips patchELF on the native .node binary. Today the @parcel/watcher binary loads its own libstdc++ via fallback paths and works, but if a future parcel-watcher version pulls in a harder dynamic-link requirement, expect to revisit this.

Quick references

• Backend dispatch: node_modules/.pnpm/@parcel+watcher@2.5.6/node_modules/@parcel/watcher/src/Backend.cc:30
• Watchman invocation: node_modules/.pnpm/@parcel+watcher@2.5.6/node_modules/@parcel/watcher/src/watchman/WatchmanBackend.cc:43
• Kolu wrapper: packages/integrations/git/src/working-tree-watcher.ts
• Composed watcher API: packages/integrations/git/src/repo-change.ts
• Watchman integration issue: juspay/kolu#788
• Streaming endpoints that consume the watcher: packages/server/src/router.ts (onStatusChange, onDiffChange, onListAllChange, onReadFileChange)