verifying-merged-work

The decisions in this file are also code

Run it, do not retype it:

node scripts/verify-merge.mjs <pr-number>

It reads the tip from git ls-remote rather than the API's cached head, refuses when the branch cannot answer, names every blocking job rather than counting them, and reports a second reviewer that never ran as distinct from one that found nothing. A gate typed out again by hand is a second implementation of the same question, which this repository has a rule about.

It answers a different question before and after the merge, because the two questions have different subjects. Open, it judges the branch tip — the thing being proposed. Merged, it judges merge_commit_sha: a squash commit is main plus the change rather than the tree CI ran on, and the two disagree. Measured on one merged pull request here, the branch head reported the CI job and one integration leg as success while the merge commit had them queued.

A merge commit whose jobs are all skipped was superseded, and that is a pass. A push to main that a newer push has already overtaken skips its jobs rather than repeating work: the newer run's verdict includes this commit. skipped already passes the gate, and here it is literally true. Nothing is cancelled by this, so a cancelled job still means what it says and still blocks. To see what actually tested the commit, read the newest run of that workflow on main.

Know its range before trusting it. The script's module header lists what it does not cover, and the four worth carrying in your head are: it snapshots threads and checks once rather than holding them still; REQUIRED_CHECKS is a floor listing workflows whose absence is known to mean no coverage, not every workflow; refs/pull/N/merge is resolved rather than pinned; and the landed-whole range screens without certifying. The project runs this advisory, so those windows are closed by the merge precondition below, not by the script.

Exit status distinguishes three outcomes, and the third is not a softer second. 0 passed, 1 blocked, 2 did not get to answer — a rewritten history, or a candidate list from the landed-whole screen that nobody has settled by content yet. A caller may retry or escalate a 2; it must never read one as a pass.

scripts/verify-merge.mjs implements the judgements below as pure functions — whether a branch can answer the question at all, whether a job counts as passing, whether a check was ever DUE to report given the paths the pull request touches, whether a review verdict belongs to the revision being merged, and whether a second reviewer looked. scripts/verify-merge.test.mjs runs them against the inputs that produced this repository's actual false cleans, and pnpm test:scripts runs it in CI.

Prefer them to re-deriving the logic in shell. The snippets here stay because the reasoning is worth reading, but a snippet is not a control: every one of them was wrong at least once — a count computed and never read, an exit status swallowed by the pipeline that consumed it, a comparison against a base that moves — and each was found by a person executing it mentally rather than by anything running it. When a rule and the script disagree, the script is the one with tests.

A squash merge makes every ancestry check unsound

Merging squashes the branch into one new commit, so the branch head is never an ancestor of main. git branch --merged, git log | grep <sha> and "is this commit in main" therefore answer confidently and wrongly. Verify by CONTENT.

Which marker you grep for is load-bearing. The commonest shape is a lost TAIL: commits land on the branch after GitHub computed the merge, or the merge runs from a stale head, so the END of the branch goes missing and a marker taken from an early commit passes cleanly on a PR that dropped its last three.

That heuristic says where to look FIRST; it is not what makes a check sufficient. A branch that was rebased, amended or force-pushed after the merge was computed diverges differently — rewriting an EARLIER commit while leaving the final patch text unchanged means the stale merge still contains your final-commit marker, and the check passes over content that was rewritten underneath it. When history was rewritten, compare the delta (step 3) rather than trusting any single marker.

None of the steps below can detect a lost tail

They all read headRefOid, and headRefOid is the MERGED head — the snapshot GitHub took when it computed the merge, not the branch's current tip. A commit pushed after that snapshot is outside the procedure entirely: absent from the merge, absent from headRefOid, and therefore absent from both sides of every comparison here. Each step then confirms that everything which merged, merged.

The branch is the only place a stranded commit still exists, so it is where to LOOK. It is not a place that can certify, and that limit is structural rather than a gap to be patched.

A branch cannot prove a negative. It is mutable, it is remote, and every observation of it is a separate round trip. Each of these erases a tail and leaves the range empty, indistinguishable from a branch that never had one:

• a force-push resetting B back to merged head A;
• deleting the branch and recreating the same ref at A;
• any reset landing between the timeline read and the ls-remote — the two are separate requests and nothing holds the ref still between them.

Enumerating those is not a fix. Three were found by patching this block three times, and the fourth is whatever nobody has thought of, because the property being asked for — "no commit ever existed here that is not in the merge" — is not a property a mutable ref can answer.

So use it as a SCREEN and take the verdict from CONTENT. Any commit it names is worth confirming against the merge commit; an empty result means only that this cheap look found nothing, never that nothing was lost.

Settle a named candidate against the CANDIDATE, not against the PR head. The numbered steps below are written for the PR as a whole, and step 3's PR side ends at <headRefOid> — which excludes a stranded commit by definition, so its two deltas come out identical whether or not that commit's change landed. When the candidate has a unique marker, grep for it as in step 2. When it does not — a binary, a rename, a mode change — compare its own patch:

git diff "$CAND^..$CAND" -- "$PATHS"          # what the stranded commit did
git diff "$MERGE^..$MERGE" -- "$PATHS"        # what the squash contains

Its hunks appearing in the second is the evidence. Their absence is the loss — unless a later candidate cancels it.

Judge the tail's NET effect, not each commit alone. When the screen names several commits and a later one reverts or supersedes an earlier one, neither individual patch need appear in the squash, and checking them one at a time reports an intentionally cancelled intermediate as lost work. That is the same correction step 4 already makes for a pull request as a whole, applied to the candidate list:

FIRST=$(git log --format=%H "$GH..$TIP" | tail -1)
git diff "$GH..$TIP" -- "$PATHS"              # what the tail did, on net

Compare THAT against the squash. A tail whose net effect is empty lost nothing, however many commits the screen listed. When it matters — a release, an incident, a PR whose tail you have reason to doubt — verify the commits you intended to land by content, per the numbered steps below, and do not let an empty range stand in for that.

The guards are inside the block rather than beside it, because the failures they prevent all look exactly like a pass. BASE_REMOTE holds main and the merge commit; HEAD_REMOTE holds the PR's branch, and for a fork that is a different repository entirely:

set -euo pipefail
PR=<number>
# Captured to a variable first. `read < <(...)` reports the status of `read`,
# not of the command inside, so a failed `gh` there sets empty fields and the
# script carries on with them under `set -e`.
META=$(gh pr view "$PR" \
  --json isCrossRepository,headRepositoryOwner,headRepository,headRefName,headRefOid,mergeCommit \
  --jq '[.isCrossRepository,.headRepositoryOwner.login,.headRepository.name,
         .headRefName,.headRefOid,.mergeCommit.oid]|@tsv') || {
  echo "PR#$PR: metadata query failed — NOT CHECKABLE, which is not clean" >&2
  exit 2
}
IFS=$'\t' read -r CROSS OWNER REPO BR GH MERGE <<<"$META"
BASE_REMOTE=origin                        # must point at the BASE repository
HEAD_REMOTE=origin
[ "$CROSS" = true ] && HEAD_REMOTE="https://github.com/$OWNER/$REPO.git"

# History rewritten? Then the range below cannot certify anything, so this
# EXITS rather than annotating. `--paginate` because the timeline is paged at
# 100 and long PRs here run to three pages: an unpaginated query reads page one
# and answers zero, which is the reassuring direction. It emits one count per
# page, hence the sum.
# Deletion and recreation erases a tail exactly as a force-push does, and the
# recreated ref reads as ordinary — so both events disqualify. This list is a
# floor, not a proof: see the note above on why enumeration cannot close this.
PAGES=$(gh api --paginate "repos/nextlyhq/nextly/issues/$PR/timeline?per_page=100" \
  --jq '[.[]|select(.event=="head_ref_force_pushed" or .event=="head_ref_deleted"
                    or .event=="head_ref_restored")]|length') || {
  echo "PR#$PR: timeline query failed — NOT CHECKABLE, which is not clean" >&2
  exit 2
}
FORCED=$(printf '%s\n' "$PAGES" | awk '{s+=$1} END{print s+0}')
if [ "${FORCED:-1}" -gt 0 ]; then
  echo "PR#$PR: $FORCED history-rewrite event(s) — NOT CHECKABLE, not clean" >&2
  exit 2
fi

TIP=$(git ls-remote "$HEAD_REMOTE" "refs/heads/$BR" | cut -f1)
if [ -z "$TIP" ]; then
  echo "PR#$PR: no such ref on $HEAD_REMOTE — NOT CHECKABLE, which is not clean" >&2
  exit 2
fi
# Unchecked, these leave `git log` reading whatever objects happen to be local
# already — a stale answer wearing the same shape as a fresh one.
git fetch "$HEAD_REMOTE" "$TIP" --quiet || {
  echo "PR#$PR: could not fetch $TIP — NOT CHECKABLE, which is not clean" >&2
  exit 2
}
git fetch "$BASE_REMOTE" "$MERGE" --quiet || {
  echo "PR#$PR: could not fetch $MERGE — NOT CHECKABLE, which is not clean" >&2
  exit 2
}
git log --oneline "$GH..$TIP"          # candidates: commits absent from the merge

Fetch each object from the remote that HAS it. The head commit of a fork PR is not on origin, so fetching it from there fails and the procedure stops before the content checks — reading as a broken verification rather than as a lookup pointed at the wrong repository.

An empty TIP degenerates the range and the check reports clean without having looked. Three things produce it, and only the first is unanswerable:

• the branch was deleted after merging — NOT CHECKABLE;
• the PR came from a FORK, so its branch was never on origin at all. This one is the trap, because the empty result is indistinguishable from deletion and invites exactly the wrong conclusion. isCrossRepository is what settles it, so query the head repository rather than the base;
• $BR names no ref, because it was typed from memory or from a task file rather than read from headRefName. Measured here: a lane checked a PR that HAD stranded a commit, used a branch name one word off from the real head ref, got an empty tip, and concluded the branch had been auto-deleted. The branch existed the whole time, and the correct query reports the stranded commit.

Derive every field in the same command that uses it, and treat an empty tip as a refusal to answer.

Every step in that block either produces an answer or exits 2, and it is worth stating as the block's rule rather than leaving it to be rediscovered. Three separate ways of failing open have now been found in these few lines: a branch name that resolved to nothing, a force-push count that was computed and never read, and an API failure whose exit status was swallowed by the awk that summed its output — each one turning "I could not look" into "nothing to see". A pipeline is the specific trap there, since its status is the LAST command's: capture the query's output in its own substitution, check it, and sum afterwards. When a third instance of one shape appears, the shape is a property of the design rather than of the instances.

A force-push can erase the evidence, and the range then reports clean. If merged head A was followed by stranded commit B, and the branch was later reset back to A, the range is A..A — empty, and indistinguishable from a branch that never had B. Nothing local can recover B, because the ref that pointed at it is gone.

So FORCED is not decoration. A non-zero count means the tip you are comparing against is not the history that was pushed, and this check CANNOT certify the PR — say NOT CHECKABLE and fall back to content, per-commit, from whatever record of the intended commits exists. The timeline reports the event (head_ref_force_pushed, with the actor and the resulting commit_id) but not the history it replaced, so detection is all it offers.

Do not reach for the timeline's committed entries to recover them: measured on a PR with two force-pushes, they number exactly what pulls/N/commits returns — both describe the current head's history, neither the erased one.

Output is a CANDIDATE LIST, not a verdict. The range says only "absent from the merged head", and a surviving branch collects commits for other reasons: it was force-pushed or rebased, it was reused for follow-up work, or someone kept pushing after the merge. Each is legitimately absent from the squash and none is a lost tail. Screen with this, then confirm each named commit by CONTENT against the merge commit — a marker unique to it, scoped to the path it changed — before calling anything lost. Cheap in both directions: the screen costs one command and the confirmation is what you can put in a claim.

The symmetry is the point. Reading the range as a verdict OVER-reports; every instrument in the table below UNDER-reports. Only the pairing answers.

Measured against a PR known to have lost two commits and one known intact:

instrument	lost 2 commits	intact
headRefOid's date predates the merge	"clean"	"clean"
step 3's delta comparison	IDENTICAL	IDENTICAL
gh api pulls/N/commits	2 commits only	complete
git log <headRefOid>..<ls-remote tip>	2 STRANDED	empty

The first three are not weak checks; they cannot see this class of defect at all, because each derives from the snapshot being audited. That is worth stating because the delta comparison is the most rigorous-looking option on offer, and its thoroughness is what earns the trust it does not deserve here. The commits API is the most tempting of the three — it is named as though it lists what the branch contained, and it lists what merged.

Three PRs lost tails on a single day in this repository, one of them carrying a P1 and one leaving main red, and none was found by the procedure below.

Three different questions, and content answers only one

They are easy to run together and none substitutes for another:

• Did my code land? — content, from the final commit. What the numbered steps below answer.
• Did EVERY commit land? — no single command answers this. The ls-remote screen above NAMES candidates and cannot certify their absence; each candidate is then settled by content. Content taken from a commit that merged can never reach a commit that did not, so the screen is what supplies the list to check.
• Was the job green? — the merge commit's own check-runs, asserted as success. A PR merged here with two Integration jobs failing and left main red for hours; its author had verified the content correctly and that check had nothing to say about the failure.

1. Confirm what was actually merged, then FETCH the object before probing it:

   gh pr view N --json headRefOid,mergeCommit
   git fetch origin <mergeCommit> <headRefOid>   # gh reports; it does not fetch
   

   gh pr view prints PR information and adds nothing to the local object database, so probing a reported SHA without this exits with unable to resolve revision — which reads as a failed verification rather than as a missing object.

   Fetch BOTH. A squash commit does not have the PR head as an ancestor, so fetching only the merge commit leaves headRefOid unresolvable — and step 3 dereferences it. Outside the PR worktree, or after the branch is deleted, that is where the procedure stops.

   Probe that merge commit, never origin/main. Run before a fetch and origin/main is still the pre-merge ref, so every check reports loss falsely; run after main advances and a later commit can make omitted content look present.

2. Take the check from the final commit, in whichever direction it changed things. A marker only proves anything if it is UNIQUE to that commit and the search is SCOPED to the path it changed — a string that also occurs elsewhere answers the same way whether or not the commit landed. Match it as a FIXED string: a marker containing ., [ or * is otherwise a pattern, and can match text it was never taken from.

   The marker must fit on ONE line. git grep matches per line, so a marker spanning a wrapped comment or a formatted call finds nothing in a merge commit that contains it — which reads exactly like a lost tail, in the alarming direction. Measured here while verifying a merge by content: the control is what separated the two, because the same marker was absent from the BRANCH HEAD as well, and content the branch does not have cannot have been lost by the merge. Pick a marker from a single line, and when the search comes back empty run it against the head before believing it.

   • it ADDED content → git grep -F -e "$marker" <mergeCommit> -- <path>; expect a hit. The -e is not optional: a marker beginning with -, which a Markdown list item usually does, is otherwise parsed as an option and exits 129 without checking anything.

   • it only REMOVED content → same command; expect NO hit. Grepping for ADDED text here finds nothing whether or not the commit landed, which reads as failure either way and proves nothing.

     An absent marker is only evidence once you have shown the search CAN find it. git grep exits 1 for "no lines selected" and for "the pathspec matched no files" alike, so a mistyped or since-renamed <path> certifies the removal without ever reading the file. Run the same command against <headRefOid>^ and require a hit: that proves the path resolves and the marker is real.

     But that control validates the INSTRUMENT, not the outcome, and for a removal it cannot validate the outcome at all. If the text was added earlier in the same PR and removed later, <mergeCommit>^ — the state main was in before the merge — never contained it, so its absence afterwards is guaranteed whether or not the removal landed. Neither preimage separates "the removal merged" from "the text was never there". Absence is simply not a witness here.

     So for a removal, use the control to prove the search works, then prove the outcome with the DELTA in step 3: the removal hunk must appear in git diff <mergeCommit>^..<mergeCommit> -- <path>. That is a positive observation of the change landing rather than an inference from nothing being found.

   • it changed a file mode, a binary, or a rename → text search cannot see it.

3. When nothing is unique to the commit, or the change is a mode/binary/rename, compare the PR's delta — not the whole object. Diffing the merged path entry against the branch-head entry (git ls-tree, blob ids) is wrong as soon as main changed ANOTHER hunk of the same file after the branch point: a correct squash contains both changes, the blobs legitimately differ, and the check reports a loss that did not happen. Compare what the PR itself changed: git diff <mergeBase>..<headRefOid> -- <path> against git diff <mergeCommit>^..<mergeCommit> -- <path>, expecting the PR's hunks in the second. The squash side is diffed from its OWN parent: a squash commit's parent IS main at the moment of the merge, so that range is exactly the squash patch. Using <mergeBase> there sweeps in every commit main gained after the branch point, and the PR's hunks disappear among them. Whole-object equality is sound only when main never touched the path. --stat is never sound: it reports only that a path was touched, which any earlier commit in the same PR already guarantees.

   This step answers "was the squash rewritten", never "did every commit land". The two sides are different objects — the PR side is bounded by <headRefOid>, the squash side is main's own history at <mergeCommit>^..<mergeCommit> — but neither can contain a commit that never merged, so a commit pushed after the merge was computed is outside both and the comparison returns IDENTICAL. Run the ls-remote check above first; this one is not a substitute for it.

4. If the final commit is a pure revert of an earlier one in the same PR, check the NET effect, not the last hunk.

The danger window is push-a-fix-then-merge-immediately, which is what everyone does once CI is green and threads are cleared. A PR has already merged here missing its last commit, reading as complete with every thread resolved.

Detection is cleanup; the gate is the fix. Everything else in this file runs after something has already shipped. Merge with the head you verified as a PRECONDITION, so the merge itself refuses when the branch has moved:

This runs BEFORE a merge exists, so it cannot borrow from the block above: that one reads mergeCommit and exits when there is none, which is every open PR.

Pass in the revision you VERIFIED. Do not re-read the head here. Re-reading returns whatever is newest, which after a push is exactly the unverified revision the flag exists to reject — so a fresh read turns the precondition into a rubber stamp for the newest commit. The value to pass is the one the green checks and the clean review belong to:

set -euo pipefail
PR=<number>
VERIFIED=<the sha whose checks were green and whose review was clean>

# Confirms the branch has not moved since, and refuses rather than merging on.
NOW=$(gh pr view "$PR" --json headRefOid --jq .headRefOid) || {
  echo "PR#$PR: head query failed — do not merge" >&2; exit 2
}
[ "$NOW" = "$VERIFIED" ] || {
  echo "PR#$PR: head moved $VERIFIED -> $NOW; re-run the gate" >&2; exit 2
}
gh pr merge "$PR" --squash --match-head-commit "$VERIFIED"

The comparison is a courtesy that yields a readable message; --match-head-commit is what makes it safe, because it is the server that refuses. Both take $VERIFIED, never $NOW. Bind it to that variable rather than retyping a SHA: a merge precondition naming the wrong revision either refuses a correct merge or, worse, permits the one it was added to stop.