gh-write-verification: the landing check's result is read, not just its command by EdbertChan · Pull Request #494 · EdbertChan/catstack

EdbertChan · 2026-09-12T18:12:20Z

Summary

A merge safeguard checks that a merged change reached the main branch before the workflow falls quiet.

It previously treated running the landing check as proof, even when the check reported failure or could not check.

The safeguard read the command that launched the check but not the result it returned.

It now requires the paired result to say OK, while failed, uncertain, or missing results continue to flag the merge.

Review Claim

A failed or uncertain landing check still flags the merge, while a successful landing check keeps the workflow quiet.

Review Lane

behavior

Review Unit

engine-runtime

Safety Invariant

No merge shape that is silent today becomes noisy: a merge followed by a landing check printing OK stays silent, and only FAIL, UNCHECKED, or a missing result newly flags.

Slice Rationale

This slice changes one landing detector and its tests in one hook directory. It shares no file with the rest of the sweep stack, so it runs independently.

Non-goals

Does not change the verifier script's exit contract.
Does not change which commands count as merges.
Does not widen the hook to other tools.
Does not change skill prose.

Architecture

Before

graph TD
    A["landing command"] --> B["detector reads command text"]
    B --> C["merge treated as proven"]

After

graph TD
    A["landing command"] --> B["detector pairs command with tool result"]
    B --> C["detector requires OK result"]
    C --> D["merge is proven or remains flagged"]

Test Plan

python3 -m unittest discover -s engine/hooks/gh-write-verification/tests -v — 52 tests passed.
python3 engine/skills/make-pr/scripts/preflight.py --base origin/main — preflight passed; hook coverage passed.
python3 scripts/check_hook_test_coverage.py engine/hooks/gh-write-verification — 1 hook checked.
bash scripts/scrub-handoff-artifacts.sh — no handoff artifacts remain.

Revert Plan

Safe to revert? Yes.
Revert command: git revert <sha>.
Post-revert steps: Rerun the landing-detector test suite.
Data migration? No.

Note

Medium Risk
Tightens Stop-hook behavior for gh pr merge turns: agents that ran a failing or inconclusive verifier will now be blocked until a successful OK result appears.

Overview
The unverified landing stop-hook no longer treats “ran a landing check” as proof. It pairs each Bash command in the turn transcript with its tool result and only counts a later check as proof when the output contains a passing OK: line (via LANDING_OK_RE).

bash_commands_this_turn now walks JSONL for tool_use / tool_result by tool_use_id and returns (command, result) records; _proves_landing and merges_missing_landing_proof use those pairs. Merges stay blocked when the verifier was never run, has no paired result, or prints FAIL / UNCHECKED instead of OK.

Tests extend the synthetic transcript helper with optional tool results and add cases for missing output, failed checks, and unchecked runs.

^{Reviewed by Cursor Bugbot for commit f70ccad. Bugbot is set up for automated code reviews on this repo. Configure here.}

cursor · 2026-09-12T18:12:27Z

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_909fe4b7-e4be-4459-957b-ec155048c9a7)

mergify · 2026-09-13T03:47:50Z

Queued — the merge queue status continues in this comment ↓.

EdbertChan · 2026-09-13T08:08:48Z

Mergify repair stopped: missing required check validate

EdbertChan · 2026-09-13T15:55:24Z

Invoker repair dispatch was not acknowledged. The request was recorded as pending, then closed without consuming the code-repair retry budget; automatic retry remains enabled for current head 533f8ff.

EdbertChan · 2026-09-13T18:18:24Z

Mergify repair stopped: required check failed: test. The retry cap was reached for current head 533f8ff.

… the landing-proof detector require the check's OK result, not merely its invocation. Review claim: A landing check that printed FAIL or UNCHECKED does not satisfy the landing-proof hook. Review lane: behavior Safety invariant: No merge shape that is silent today becomes noisy: a merge followed by a landing check printing OK stays silent, and only FAIL, UNCHECKED, or a missing result newly flags. Effectiveness measurement: Three fixtures: a landing check printing OK stays silent, one printing FAIL flags, one printing UNCHECKED flags. The current detector passes only the first, so the suite fails before the change and passes after. Slice rationale: One detector and its tests in one hook directory; shares no file with the sweep-skill chain, so it runs independently rather than waiting on it. Architectural effect: The detector gains access to each command's paired tool result, so the hook judges outcomes rather than invocations. Goal: Change engine/hooks/gh-write-verification/detect.py so _proves_landing requires the paired tool result to report OK, and add tests for the FAIL and UNCHECKED cases. Motivation: The hook's own purpose is to stop a merge claim that outran its landing check. Because it reads only the command text, a check that ran and reported failure clears it exactly as well as one that reported success, so the failure mode it exists to catch is the one it cannot see. Alternative considerations: Parsing the exit code alone was rejected because the script prints its verdict on stdout and an exit code is not available for every recorded command shape. Leaving the detector as-is and adding a prose rule was rejected: the detector is the mechanism, so prose would be a second copy of the same rule with no enforcement. Implementation details: Extend bash_commands_this_turn (detect.py lines 369-391) to carry each command's paired tool result alongside its text rather than discarding non-assistant records. Change _proves_landing (detect.py lines 291-302) to require an OK marker in that result, treating a missing result as not proving landing. Update merges_missing_landing_proof (detect.py line 306) to pass the results through. Add tests asserting a FAIL result and an UNCHECKED result both still flag, and that an OK result stays silent. Non-goals: Does not change the verifier script's exit contract, does not change which commands count as merges, does not widen the hook to other tools, and does not touch any skill prose. Layer: domain Feature state: active Files: engine/hooks/gh-write-verification/detect.py, engine/hooks/gh-write-verification/tests/test_hooks.py Change types: - engine/hooks/gh-write-verification/detect.py: modify - engine/hooks/gh-write-verification/tests/test_hooks.py: modify Acceptance criteria: - `python3 -m unittest discover -s engine/hooks/gh-write-verification/tests -v` exits 0. - A landing check whose result printed FAIL still flags the merge. - A landing check whose result printed UNCHECKED still flags the merge. - A landing check whose result printed OK stays silent. Solution: Make the landing-proof detector require the check's OK result, not merely its invocation. Review claim: A landing check that printed FAIL or UNCHECKED does not satisfy the landing-proof hook. Review lane: behavior Safety invariant: No merge shape that is silent today becomes noisy: a merge followed by a landing check printing OK stays silent, and only FAIL, UNCHECKED, or a missing result newly flags. Effectiveness measurement: Three fixtures: a landing check printing OK stays silent, one printing FAIL flags, one printing UNCHECKED flags. The current detector passes only the first, so the suite fails before the change and passes after. Slice rationale: One detector and its tests in one hook directory; shares no file with the sweep-skill chain, so it runs independently rather than waiting on it. Architectural effect: The detector gains access to each command's paired tool result, so the hook judges outcomes rather than invocations. Goal: Change engine/hooks/gh-write-verification/detect.py so _proves_landing requires the paired tool result to report OK, and add tests for the FAIL and UNCHECKED cases. Motivation: The hook's own purpose is to stop a merge claim that outran its landing check. Because it reads only the command text, a check that ran and reported failure clears it exactly as well as one that reported success, so the failure mode it exists to catch is the one it cannot see. Alternative considerations: Parsing the exit code alone was rejected because the script prints its verdict on stdout and an exit code is not available for every recorded command shape. Leaving the detector as-is and adding a prose rule was rejected: the detector is the mechanism, so prose would be a second copy of the same rule with no enforcement. Implementation details: Extend bash_commands_this_turn (detect.py lines 369-391) to carry each command's paired tool result alongside its text rather than discarding non-assistant records. Change _proves_landing (detect.py lines 291-302) to require an OK marker in that result, treating a missing result as not proving landing. Update merges_missing_landing_proof (detect.py line 306) to pass the results through. Add tests asserting a FAIL result and an UNCHECKED result both still flag, and that an OK result stays silent. Non-goals: Does not change the verifier script's exit contract, does not change which commands count as merges, does not widen the hook to other tools, and does not touch any skill prose. Layer: domain Feature state: active Files: engine/hooks/gh-write-verification/detect.py, engine/hooks/gh-write-verification/tests/test_hooks.py Change types: - engine/hooks/gh-write-verification/detect.py: modify - engine/hooks/gh-write-verification/tests/test_hooks.py: modify Acceptance criteria: - `python3 -m unittest discover -s engine/hooks/gh-write-verification/tests -v` exits 0. - A landing check whose result printed FAIL still flags the merge. - A landing check whose result printed UNCHECKED still flags the merge. - A landing check whose result printed OK stays silent. Invoker-Finalize-Id: 275f6067-457a-4dde-8dbf-8542a8fff971

… deterministic proof for the landing detector's result reading. Review claim: The hook's tests assert a failed landing check still flags the merge. Review lane: proof Safety invariant: Proof-only; adds no product behavior. Effectiveness measurement: The suite fails if a FAIL or UNCHECKED landing result clears the hook. Slice rationale: One proof slice for this step. Architectural effect: None; verification only. Goal: Prove the landing detector's result reading deterministically. Motivation: The defect is a detector that reads the wrong input, so the proof drives it with results rather than commands. Alternative considerations: Manual verification was rejected as non-deterministic. Implementation details: Execute the command below as the terminal proof. Non-goals: No product edits here; proof only. Layer: app_regression Feature state: active Exit code: 0 Invoker-Finalize-Id: 1e117326-8ae6-4b5c-9862-bf9319098010

… that no ephemeral inter-task handoff files remain. Review claim: The workflow leaves no scratch handoff artifacts behind. Review lane: cleanup Safety invariant: Read-only; never deletes files, alters the index, or commits caller work. Effectiveness measurement: The gate fails when a plans/invoker-handoff.* or lens-*.json file is still present after the leaf tasks complete. Slice rationale: One terminal hygiene gate for the workflow. Architectural effect: None; check only. Goal: Confirm no ephemeral handoff files survive the run. Motivation: Inter-task scratch files leak into diffs and read as part of the work. Alternative considerations: Deleting them automatically was rejected; the gate reports, it does not mutate. Implementation details: Run scripts/scrub-handoff-artifacts.sh without --apply. Non-goals: No deletion, no index changes, no commits. Layer: app_regression Feature state: active Exit code: 0 Invoker-Finalize-Id: 657b1e05-5809-43b8-a2ea-e4ef235c90a4

…thon 3.9 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HQ9TkZeqARxv3e7W7xesYy

cursor · 2026-09-14T04:12:13Z

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_d28eee2c-7363-4aaa-b155-762f4bfa6781)

EdbertChan · 2026-09-14T04:17:45Z

@Mergifyio queue

mergify · 2026-09-14T04:18:39Z

Merge Queue Status

✅ Entered queue — 2026-09-14 04:18 UTC · Rule: admin-bypass · triggered by @EdbertChan with the @mergifyio queue command
✅ Checks passed · on draft merge queue: checking #494 on main (344657b), stacked on 4 pull requests ahead #589
✅ Merged — 2026-09-14 04:25 UTC · at 2b78558d04acc09d8080c215d050b8c7fd1b1f8a · squash

This pull request spent 6 minutes 43 seconds in the queue, including 4 minutes 53 seconds running CI.

Required conditions to merge

check-success = lint
check-success = test
check-success = validate

EdbertChan added the admin-bypass label Sep 13, 2026

Invoker Bot and others added 4 commits September 13, 2026 20:58

gh-write-verification: keep the command record alias importable on Py…

f70ccad

…thon 3.9 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HQ9TkZeqARxv3e7W7xesYy

EdbertChan force-pushed the plan/gh-write-verification-the-landing-check-s-result-is-read-not-just-its-command branch from 533f8ff to f70ccad Compare September 14, 2026 04:12

mergify Bot added the queued label Sep 14, 2026

mergify Bot mentioned this pull request Sep 14, 2026

merge queue: checking #494 on main (344657b), stacked on 4 pull requests ahead #589

Closed

6 tasks

mergify Bot merged commit 2b78558 into main Sep 14, 2026
6 checks passed

mergify Bot deleted the plan/gh-write-verification-the-landing-check-s-result-is-read-not-just-its-command branch September 14, 2026 04:25

mergify Bot removed the queued label Sep 14, 2026

Navigation Menu

gh-write-verification: the landing check's result is read, not just its command - #494

gh-write-verification: the landing check's result is read, not just its command#494
mergify[bot] merged 4 commits into
mainfrom
plan/gh-write-verification-the-landing-check-s-result-is-read-not-just-its-command

EdbertChan commented Sep 12, 2026 •

edited by cursor Bot

Loading

Uh oh!

cursor Bot commented Sep 12, 2026

Uh oh!

mergify Bot commented Sep 13, 2026 •

edited

Loading

Uh oh!

EdbertChan commented Sep 13, 2026

Uh oh!

EdbertChan commented Sep 13, 2026

Uh oh!

EdbertChan commented Sep 13, 2026

Uh oh!

cursor Bot commented Sep 14, 2026

Uh oh!

EdbertChan commented Sep 14, 2026

Uh oh!

mergify Bot commented Sep 14, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

EdbertChan commented Sep 12, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Review Claim

Review Lane

Review Unit

Safety Invariant

Slice Rationale

Non-goals

Architecture

Before

After

Test Plan

Revert Plan

Uh oh!

cursor Bot commented Sep 12, 2026

Bugbot couldn't run - usage limit reached

Uh oh!

mergify Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EdbertChan commented Sep 13, 2026

Uh oh!

EdbertChan commented Sep 13, 2026

Uh oh!

EdbertChan commented Sep 13, 2026

Uh oh!

cursor Bot commented Sep 14, 2026

Bugbot couldn't run - usage limit reached

Uh oh!

EdbertChan commented Sep 14, 2026

Uh oh!

mergify Bot commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Queue Status

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

EdbertChan commented Sep 12, 2026 •

edited by cursor Bot

Loading

mergify Bot commented Sep 13, 2026 •

edited

Loading

mergify Bot commented Sep 14, 2026 •

edited

Loading