Skip to content

merge queue: checking #540 on main (344657b), stacked on #519 and #525 - #587

Closed
mergify[bot] wants to merge 16 commits into
mainfrom
mergify/merge-queue/e5ccf6eb30
Closed

merge queue: checking #540 on main (344657b), stacked on #519 and #525#587
mergify[bot] wants to merge 16 commits into
mainfrom
mergify/merge-queue/e5ccf6eb30

Conversation

@mergify

@mergify mergify Bot commented Sep 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🎉 This pull request has been checked successfully and will be merged soon. 🎉

#540 is queued for merge on branch main (344657b).

Stacked behind 2 pull requests queued ahead of this batch, not part of it. These checks run on a tip that also carries their commits, so a failure here can come from them as much as from #540.

Queued ahead of this batch:

This pull request has been created by Mergify to speculatively check the mergeability of #540.
You don't need to do anything. Mergify will close this pull request automatically when it is complete.

Required conditions of queue rule admin-bypass for merge:

  • check-success = lint
  • check-success = test
  • check-success = validate

Required conditions to stay in the queue:

---
checking_base_sha: dce75f9ae07ad92289eefffb88ef63d587b81882
previous_check_retries: []
previous_failed_batches: []
pull_requests:
  - number: 540
    scopes: []
scopes: []
...

EdbertChan and others added 16 commits September 13, 2026 20:58
@EdbertChan @claude
[Hooks Cannot See] (2) Require the output to entail the claim, not me…
eaa9504 
…rely agree

The session behind slice (1) ran one command against one container image and
proved that image had no corepack. That was written up as corepack having been
removed from Node 25+, a version-boundary claim no command in the session ever
checked. The output was true and the sentence was wider than the output.

prove-it already covers a missing check and a hedge. It did not cover a check
that ran, came back narrower than the claim, and got filed under the wider
heading anyway -- which is the common case, because the cheap check is always
the reachable instance.

It binds hardest in a correction: restating the original overclaim while
pasting a narrower proof relabels the mistake as a fix, which is what happened
here before the rule existed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011ZqCxSQVpmM9Rx37C7irgf
Change-Id: Ibc32514be641d37277115faf6068e75ea02e0083
Invoker Bot @EdbertChan
invoker: wf-1789279240280-10/verify-judge-investigate-tests — Proof s…
a6c7201 
…tep 1 for the llm-judge investigate mode.

Review claim: The investigate-mode unit tests run and pass.
Review lane: proof
Safety invariant: Proof-only; adds no product behavior.
Effectiveness measurement: The run exits 0 and reports at least one test; Python exits 5 when -k matches nothing.
Slice rationale: One proof step for this workflow's single change.
Architectural effect: None; verification only.
Goal: Prove the investigate runner argv and time limit deterministically.
Motivation: Each slice carries its own executable proof.
Alternative considerations: Manual inspection was rejected as non-deterministic.
Implementation details: Run the filtered unittest discovery once.
Non-goals: No product edits here; proof only.
Layer: app_regression
Feature state: active

Exit code: 0
Invoker-Finalize-Id: 964f7535-7395-4bb3-b657-37a77b357f44
Invoker Bot @EdbertChan
invoker: wf-1789279240280-10/verify-judge-full-suite — Proof step 2 f…
b182d29 
…or the llm-judge investigate mode.

Review claim: Every llm-judge test still passes after the change.
Review lane: proof
Safety invariant: Proof-only; adds no product behavior.
Effectiveness measurement: The whole llm-judge test folder exits 0.
Slice rationale: One proof step guarding current judge callers.
Architectural effect: None; verification only.
Goal: Prove default jobs behave as before.
Motivation: wrong-check-reflect and diu-stop already depend on the judge.
Alternative considerations: Running only the new tests was rejected; it would miss a regression in the default runner set.
Implementation details: Run the whole llm-judge test folder.
Non-goals: No product edits here; proof only.
Layer: app_regression
Feature state: active

Exit code: 0
Invoker-Finalize-Id: c94db032-2d49-4594-b6e7-b3ac43e7b56b
Invoker Bot @EdbertChan
invoker: wf-1789279240280-10/verify-judge-no-new-comments — Proof ste…
538767d 
…p 3 for the llm-judge investigate mode.

Review claim: The change adds no explanatory code comments.
Review lane: proof
Safety invariant: Proof-only; adds no product behavior.
Effectiveness measurement: The comment gate exits 0 against origin/main.
Slice rationale: One proof step for catstack's required comment gate.
Architectural effect: None; verification only.
Goal: Keep the CI comment gate green.
Motivation: CI runs this gate on every pull request.
Alternative considerations: Waiting for CI was rejected; it is slower to find.
Implementation details: Run the comment gate against origin/main.
Non-goals: No product edits here; proof only.
Layer: app_regression
Feature state: active

Exit code: 0
Invoker-Finalize-Id: 492bf3e7-0cff-46b3-b0d1-89687e47b6b7
Invoker Bot @EdbertChan
invoker: wf-1789279240280-10/scrub-handoff-artifacts — Check that no …
a711d7d 
…ephemeral inter-task handoff files remain before the merge gate.

Review claim: Only ephemeral handoff files are checked; no product or test file is touched.
Review lane: cleanup
Safety invariant: The scrub script runs read-only and never touches the home Invoker ledger.
Effectiveness measurement: The scrub script exits 0 and reports no handoff path.
Slice rationale: One cleanup slice required on every implementation plan that opens a pull request, kept separate from behavior and proof work.
Architectural effect: None.
Goal: Leave a clean worktree for the pull request.
Motivation: Ephemeral inter-task files must not leak into the reviewed diff.
Alternative considerations: Leaving this step out was rejected; it is a hard requirement for every plan that opens a pull request.
Implementation details: Run the repository's handoff-scrub script without --apply.
Non-goals: No product edits in this task.
Layer: docs
Feature state: active

Exit code: 0
Invoker-Finalize-Id: 2aafd01f-794a-46f1-8763-0a04ed358458
Invoker Bot @EdbertChan
invoker: wf-1789279268766-11/implement-inbox-report-line — Append the…
3ca9331 
… judge answer's report string to a hit message in the llm-judge inbox.

Review claim: inbox.messages() returns on_hit followed by a space and answer["report"] when the verdict is a
hit and report is a non-empty string, clipped to 600 characters; every other verdict produces the same text
as before.
Review lane: behavior
Safety invariant: A verdict whose answer carries no "report" string yields byte-for-byte the same message as
today, so wrong-check-reflect and diu-stop messages are unchanged. Clean verdicts still produce nothing.
Effectiveness measurement: Unit tests prove a hit with a report ends with that report, a hit lacking one
equals on_hit exactly, a report over 600 characters is clipped, and a non-string report is ignored.
Slice rationale: One behavior slice in the inbox, dormant because no current judge prompt asks for a report
key; the hook that asks for it lands next.
Architectural effect: The inbox can carry per-verdict evidence text from the model to the agent.
Goal: Let a judge tell the agent what it found, not just that it found something.
Motivation: The planned unverified-tag check must report whether a "cannot verify" blocker and its claim were
true or false, with the file or quote that shows it. A fixed on_hit sentence cannot carry that.
Alternative considerations: Building the message inside the hook at enqueue time was rejected because the
answer does not exist until the background run ends. A new verdict field was rejected as more surface than
one optional answer key.
Implementation details: In engine/hooks/llm-judge/inbox.py, in messages(), after picking the hit text, read
item.get("answer"); when it is a dict whose "report" value is a str with non-blank content, append " " plus
that value stripped and clipped to REPORT_LIMIT = 600 characters. Add tests to
engine/hooks/llm-judge/tests/test_inbox.py.
Non-goals: No change to judge.py, to unchecked or clean handling, or to any hook. No README edit in this task.
Layer: domain
Feature state: dormant
Files:
- engine/hooks/llm-judge/inbox.py
- engine/hooks/llm-judge/tests/test_inbox.py
Change types:
- engine/hooks/llm-judge/inbox.py: modify
- engine/hooks/llm-judge/tests/test_inbox.py: modify
Acceptance criteria:
- Tests whose names contain "report" pass.
- The whole llm-judge test folder passes.
- python3 scripts/check_no_new_comments.py --base origin/main exits 0.

Solution:
  Append the judge answer's report string to a hit message in the llm-judge inbox.
Review claim: inbox.messages() returns on_hit followed by a space and answer["report"] when the verdict is a
hit and report is a non-empty string, clipped to 600 characters; every other verdict produces the same text
as before.
Review lane: behavior
Safety invariant: A verdict whose answer carries no "report" string yields byte-for-byte the same message as
today, so wrong-check-reflect and diu-stop messages are unchanged. Clean verdicts still produce nothing.
Effectiveness measurement: Unit tests prove a hit with a report ends with that report, a hit lacking one
equals on_hit exactly, a report over 600 characters is clipped, and a non-string report is ignored.
Slice rationale: One behavior slice in the inbox, dormant because no current judge prompt asks for a report
key; the hook that asks for it lands next.
Architectural effect: The inbox can carry per-verdict evidence text from the model to the agent.
Goal: Let a judge tell the agent what it found, not just that it found something.
Motivation: The planned unverified-tag check must report whether a "cannot verify" blocker and its claim were
true or false, with the file or quote that shows it. A fixed on_hit sentence cannot carry that.
Alternative considerations: Building the message inside the hook at enqueue time was rejected because the
answer does not exist until the background run ends. A new verdict field was rejected as more surface than
one optional answer key.
Implementation details: In engine/hooks/llm-judge/inbox.py, in messages(), after picking the hit text, read
item.get("answer"); when it is a dict whose "report" value is a str with non-blank content, append " " plus
that value stripped and clipped to REPORT_LIMIT = 600 characters. Add tests to
engine/hooks/llm-judge/tests/test_inbox.py.
Non-goals: No change to judge.py, to unchecked or clean handling, or to any hook. No README edit in this task.
Layer: domain
Feature state: dormant
Files:
- engine/hooks/llm-judge/inbox.py
- engine/hooks/llm-judge/tests/test_inbox.py
Change types:
- engine/hooks/llm-judge/inbox.py: modify
- engine/hooks/llm-judge/tests/test_inbox.py: modify
Acceptance criteria:
- Tests whose names contain "report" pass.
- The whole llm-judge test folder passes.
- python3 scripts/check_no_new_comments.py --base origin/main exits 0.

Invoker-Finalize-Id: c48ed200-f1d8-4426-b7f9-e0b2f09a285c
Invoker Bot @EdbertChan
invoker: wf-1789279268766-11/verify-inbox-report-tests — Proof step 1…
f12cf32 
… for the inbox report line.

Review claim: The report-line unit tests run and pass.
Review lane: proof
Safety invariant: Proof-only; adds no product behavior.
Effectiveness measurement: The run exits 0 and reports at least one test; Python exits 5 when -k matches nothing.
Slice rationale: One proof step for this workflow's single change.
Architectural effect: None; verification only.
Goal: Prove the report line deterministically.
Motivation: Each slice carries its own executable proof.
Alternative considerations: Manual inspection was rejected as non-deterministic.
Implementation details: Run the filtered unittest discovery once.
Non-goals: No product edits here; proof only.
Layer: app_regression
Feature state: active

Exit code: 0
Invoker-Finalize-Id: 7ffa5643-18c9-4676-875e-a4105c1be37b
Invoker Bot @EdbertChan
invoker: wf-1789279268766-11/verify-inbox-full-suite — Proof step 2 f…
34ddc99 
…or the inbox report line.

Review claim: Every llm-judge test still passes after the change.
Review lane: proof
Safety invariant: Proof-only; adds no product behavior.
Effectiveness measurement: The whole llm-judge test folder exits 0.
Slice rationale: One proof step guarding current inbox callers.
Architectural effect: None; verification only.
Goal: Prove messages for current callers are unchanged.
Motivation: wrong-check-reflect and diu-stop already deliver through the inbox.
Alternative considerations: Running only the new tests was rejected; it would miss a regression for current callers.
Implementation details: Run the whole llm-judge test folder.
Non-goals: No product edits here; proof only.
Layer: app_regression
Feature state: active

Exit code: 0
Invoker-Finalize-Id: 4c6eee23-2412-44e4-bb8e-665509f2f540
Invoker Bot @EdbertChan
invoker: wf-1789279268766-11/verify-inbox-no-new-comments — Proof ste…
80c5064 
…p 3 for the inbox report line.

Review claim: The change adds no explanatory code comments.
Review lane: proof
Safety invariant: Proof-only; adds no product behavior.
Effectiveness measurement: The comment gate exits 0 against origin/main.
Slice rationale: One proof step for catstack's required comment gate.
Architectural effect: None; verification only.
Goal: Keep the CI comment gate green.
Motivation: CI runs this gate on every pull request.
Alternative considerations: Waiting for CI was rejected; it is slower to find.
Implementation details: Run the comment gate against origin/main.
Non-goals: No product edits here; proof only.
Layer: app_regression
Feature state: active

Exit code: 0
Invoker-Finalize-Id: 49853f19-8166-4bc3-89a1-dbf73ed9688c
Invoker Bot @EdbertChan
invoker: wf-1789279268766-11/document-inbox-report-line — Describe th…
e3d91c9 
…e report line in the llm-judge README's inbox section.

Review claim: The "Delivery: the inbox" section of engine/hooks/llm-judge/README.md says a hit whose answer carries a report string ends with that report, clipped to 600 characters.
Review lane: docs
Safety invariant: Only engine/hooks/llm-judge/README.md changes; no code or test file is edited.
Effectiveness measurement: grep finds the word report and the number 600 in the README inbox section.
Slice rationale: The README line for the behavior in this same pull request, which catstack's ship-a-detector playbook requires to ship together.
Architectural effect: None; README only.
Goal: Keep the inbox contract in the README accurate.
Motivation: catstack keeps every hook's contract in its README.
Alternative considerations: A later README-only pull request was rejected; the playbook counts that as a known repair pattern.
Implementation details: Edit the hit bullet under "## Delivery: the inbox" in engine/hooks/llm-judge/README.md.
Non-goals: No code or test edits.
Layer: docs
Feature state: active
Files:
- engine/hooks/llm-judge/README.md
Change types:
- engine/hooks/llm-judge/README.md: modify
Acceptance criteria:
- `grep -n "report" engine/hooks/llm-judge/README.md` exits 0.

Context:
  wf-1789279268766-11/implement-inbox-report-line (0d98bfa): Append the judge answer's report string to a hit message in the llm-judge inbox.
Review claim: inbox.messages() returns on_hit followed by a space and answer["report"] when the verdict is a
hit and report is a non-empty string, clipped to 600 characters; every other verdict produces the same text
as before.
Review lane: behavior
Safety invariant: A verdict whose answer carries no "report" string yields byte-for-byte the same message as
today, so wrong-check-reflect and diu-stop messages are unchanged. Clean verdicts still produce nothing.
Effectiveness measurement: Unit tests prove a hit with a report ends with that report, a hit lacking one
equals on_hit exactly, a report over 600 characters is clipped, and a non-string report is ignored.
Slice rationale: One behavior slice in the inbox, dormant because no current judge prompt asks for a report
key; the hook that asks for it lands next.
Architectural effect: The inbox can carry per-verdict evidence text from the model to the agent.
Goal: Let a judge tell the agent what it found, not just that it found something.
Motivation: The planned unverified-tag check must report whether a "cannot verify" blocker and its claim were
true or false, with the file or quote that shows it. A fixed on_hit sentence cannot carry that.
Alternative considerations: Building the message inside the hook at enqueue time was rejected because the
answer does not exist until the background run ends. A new verdict field was rejected as more surface than
one optional answer key.
Implementation details: In engine/hooks/llm-judge/inbox.py, in messages(), after picking the hit text, read
item.get("answer"); when it is a dict whose "report" value is a str with non-blank content, append " " plus
that value stripped and clipped to REPORT_LIMIT = 600 characters. Add tests to
engine/hooks/llm-judge/tests/test_inbox.py.
Non-goals: No change to judge.py, to unchecked or clean handling, or to any hook. No README edit in this task.
Layer: domain
Feature state: dormant
Files:
- engine/hooks/llm-judge/inbox.py
- engine/hooks/llm-judge/tests/test_inbox.py
Change types:
- engine/hooks/llm-judge/inbox.py: modify
- engine/hooks/llm-judge/tests/test_inbox.py: modify
Acceptance criteria:
- Tests whose names contain "report" pass.
- The whole llm-judge test folder passes.
- python3 scripts/check_no_new_comments.py --base origin/main exits 0.

Solution:
  Describe the report line in the llm-judge README's inbox section.
Review claim: The "Delivery: the inbox" section of engine/hooks/llm-judge/README.md says a hit whose answer carries a report string ends with that report, clipped to 600 characters.
Review lane: docs
Safety invariant: Only engine/hooks/llm-judge/README.md changes; no code or test file is edited.
Effectiveness measurement: grep finds the word report and the number 600 in the README inbox section.
Slice rationale: The README line for the behavior in this same pull request, which catstack's ship-a-detector playbook requires to ship together.
Architectural effect: None; README only.
Goal: Keep the inbox contract in the README accurate.
Motivation: catstack keeps every hook's contract in its README.
Alternative considerations: A later README-only pull request was rejected; the playbook counts that as a known repair pattern.
Implementation details: Edit the hit bullet under "## Delivery: the inbox" in engine/hooks/llm-judge/README.md.
Non-goals: No code or test edits.
Layer: docs
Feature state: active
Files:
- engine/hooks/llm-judge/README.md
Change types:
- engine/hooks/llm-judge/README.md: modify
Acceptance criteria:
- `grep -n "report" engine/hooks/llm-judge/README.md` exits 0.

Invoker-Finalize-Id: c105fd49-67e1-44b6-827f-95db211a0555
Invoker Bot @EdbertChan
invoker: wf-1789279268766-11/scrub-handoff-artifacts — Check that no …
ef227f4 
…ephemeral inter-task handoff files remain before the merge gate.

Review claim: Only ephemeral handoff files are checked; no product or test file is touched.
Review lane: cleanup
Safety invariant: The scrub script runs read-only and never touches the home Invoker ledger.
Effectiveness measurement: The scrub script exits 0 and reports no handoff path.
Slice rationale: One cleanup slice required on every implementation plan that opens a pull request, kept separate from behavior and proof work.
Architectural effect: None.
Goal: Leave a clean worktree for the pull request.
Motivation: Ephemeral inter-task files must not leak into the reviewed diff.
Alternative considerations: Leaving this step out was rejected; it is a hard requirement for every plan that opens a pull request.
Implementation details: Run the repository's handoff-scrub script without --apply.
Non-goals: No product edits in this task.
Layer: docs
Feature state: active

Exit code: 0
Invoker-Finalize-Id: 007250b8-44a0-4a48-a256-7fd291266e8f
@EdbertChan @claude
principle-prove-it: example where the pasted output is narrower than …
2371e0a 
…the claim

The skill test coverage gate requires a test change with any rule-shaped
SKILL.md change. The new entailment rule gets a fires_ example, matching how
the subagent-contradiction rule got fires_subagent_contradiction.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HQ9TkZeqARxv3e7W7xesYy
Change-Id: Ia60c5c6a87b249bb186b83a5be5b0717603536bd
@EdbertChan @claude
test(llm-judge): the shared test base runs only a local stub judge
1aaedf3 
Tests on the base can no longer fall through to codex/claude/cursor, and
enqueue is pinned to the temporary state folder.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013smxPCr4XeE7R77zVtN1u4
Change-Id: I7b2356d2fda21cbe253900540139827fcc4d8827
@mergify
Merge of #519
bd76330
@mergify
Merge of #525
dce75f9
@mergify
Merge of #540
7ccd6ec
@mergify mergify Bot mentioned this pull request Sep 14, 2026
Merged
4 tasks
@mergify mergify Bot closed this Sep 14, 2026
@mergify
mergify Bot deleted the mergify/merge-queue/e5ccf6eb30 branch September 14, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@EdbertChan