From ae160c2ab983cb08dcc0f3047a635e0862c4895d Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sat, 12 Sep 2026 18:08:37 +0000 Subject: [PATCH 01/12] =?UTF-8?q?invoker:=20wf-1789236076827-17/implement-?= =?UTF-8?q?sweep-step5a-wiring=20=E2=80=94=20Rewrite=20the=20Step=205a=20p?= =?UTF-8?q?assage=20to=20call=20the=20shipped=20probe.=20Review=20claim:?= =?UTF-8?q?=20Step=205a=20names=20the=20probe=20and=20all=20three=20of=20i?= =?UTF-8?q?ts=20outcomes.=20Review=20lane:=20behavior=20Safety=20invariant?= =?UTF-8?q?:=20The=20blocked-pull-request=20policy=20in=20Step=205=20is=20?= =?UTF-8?q?unchanged;=20only=20the=20passage=20describing=20how=20to=20tes?= =?UTF-8?q?t=20is=20rewritten.=20Effectiveness=20measurement:=20A=20fixtur?= =?UTF-8?q?e=20asserts=20the=20Step=205a=20passage=20names=20the=20probe?= =?UTF-8?q?=20and=20all=20three=20outcomes,=20and=20that=20the=20words=20`?= =?UTF-8?q?rm=20-rf`=20no=20longer=20appear=20in=20it.=20Slice=20rationale?= =?UTF-8?q?:=20One=20passage=20of=20one=20document,=20stacked=20on=20the?= =?UTF-8?q?=20step=20that=20ships=20the=20executable=20it=20names.=20Archi?= =?UTF-8?q?tectural=20effect:=20The=20written=20procedure=20points=20at=20?= =?UTF-8?q?a=20shipped=20executable=20instead=20of=20an=20inline=20recipe.?= =?UTF-8?q?=20Goal:=20Rewrite=20the=20Step=205a=20passage=20of=20product/s?= =?UTF-8?q?kills/admin-bypass-sweep/SKILL.md=20to=20call=20scripts/probe?= =?UTF-8?q?=5Fbranch=5Frebase.sh=20and=20describe=20its=20three=20outcomes?= =?UTF-8?q?.=20Motivation:=20The=20passage=20already=20told=20the=20reader?= =?UTF-8?q?=20to=20tear=20down=20with=20the=20safe=20teardown=20command,?= =?UTF-8?q?=20and=20that=20wording=20did=20not=20prevent=20an=20improvised?= =?UTF-8?q?=20removal=20that=20corrupted=20the=20next=20reading.=20Naming?= =?UTF-8?q?=20one=20executable=20removes=20the=20chance=20to=20improvise.?= =?UTF-8?q?=20Alternative=20considerations:=20Adding=20another=20sentence?= =?UTF-8?q?=20of=20guidance=20was=20rejected,=20since=20equivalent=20guida?= =?UTF-8?q?nce=20is=20already=20present=20and=20did=20not=20hold.=20Implem?= =?UTF-8?q?entation=20details:=20Replace=20the=20fenced=20passage=20in=20S?= =?UTF-8?q?tep=205a=20with=20a=20call=20to=20scripts/probe=5Fbranch=5Freba?= =?UTF-8?q?se.sh=20and=20a=20short=20list=20of=20its=20three=20outcomes.?= =?UTF-8?q?=20Add=20a=20fixture=20under=20the=20skill's=20tests=20folder?= =?UTF-8?q?=20asserting=20the=20passage=20names=20the=20probe,=20names=20a?= =?UTF-8?q?ll=20three=20outcomes,=20and=20no=20longer=20contains=20an=20im?= =?UTF-8?q?provised=20removal.=20Non-goals:=20Does=20not=20change=20Step?= =?UTF-8?q?=205's=20policy=20for=20a=20blocked=20pull=20request,=20does=20?= =?UTF-8?q?not=20touch=20the=20merge=20passage,=20and=20does=20not=20edit?= =?UTF-8?q?=20any=20other=20written=20procedure.=20Layer:=20domain=20Featu?= =?UTF-8?q?re=20state:=20active=20Files:=20product/skills/admin-bypass-swe?= =?UTF-8?q?ep/SKILL.md,=20product/skills/admin-bypass-sweep/tests/fires=5F?= =?UTF-8?q?step5a=5Fnames=5Fprobe.md=20Change=20types:=20-=20product/skill?= =?UTF-8?q?s/admin-bypass-sweep/SKILL.md:=20docs-only=20-=20product/skills?= =?UTF-8?q?/admin-bypass-sweep/tests/fires=5Fstep5a=5Fnames=5Fprobe.md:=20?= =?UTF-8?q?create=20Acceptance=20criteria:=20-=20`python3=20-m=20unittest?= =?UTF-8?q?=20discover=20-s=20product/skills/admin-bypass-sweep/tests=20-v?= =?UTF-8?q?`=20exits=200.=20-=20The=20Step=205a=20passage=20names=20script?= =?UTF-8?q?s/probe=5Fbranch=5Frebase.sh.=20-=20The=20Step=205a=20passage?= =?UTF-8?q?=20lists=20all=20three=20outcomes.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Solution: Rewrite the Step 5a passage to call the shipped probe. Review claim: Step 5a names the probe and all three of its outcomes. Review lane: behavior Safety invariant: The blocked-pull-request policy in Step 5 is unchanged; only the passage describing how to test is rewritten. Effectiveness measurement: A fixture asserts the Step 5a passage names the probe and all three outcomes, and that the words `rm -rf` no longer appear in it. Slice rationale: One passage of one document, stacked on the step that ships the executable it names. Architectural effect: The written procedure points at a shipped executable instead of an inline recipe. Goal: Rewrite the Step 5a passage of product/skills/admin-bypass-sweep/SKILL.md to call scripts/probe_branch_rebase.sh and describe its three outcomes. Motivation: The passage already told the reader to tear down with the safe teardown command, and that wording did not prevent an improvised removal that corrupted the next reading. Naming one executable removes the chance to improvise. Alternative considerations: Adding another sentence of guidance was rejected, since equivalent guidance is already present and did not hold. Implementation details: Replace the fenced passage in Step 5a with a call to scripts/probe_branch_rebase.sh and a short list of its three outcomes. Add a fixture under the skill's tests folder asserting the passage names the probe, names all three outcomes, and no longer contains an improvised removal. Non-goals: Does not change Step 5's policy for a blocked pull request, does not touch the merge passage, and does not edit any other written procedure. Layer: domain Feature state: active Files: product/skills/admin-bypass-sweep/SKILL.md, product/skills/admin-bypass-sweep/tests/fires_step5a_names_probe.md Change types: - product/skills/admin-bypass-sweep/SKILL.md: docs-only - product/skills/admin-bypass-sweep/tests/fires_step5a_names_probe.md: create Acceptance criteria: - `python3 -m unittest discover -s product/skills/admin-bypass-sweep/tests -v` exits 0. - The Step 5a passage names scripts/probe_branch_rebase.sh. - The Step 5a passage lists all three outcomes. Invoker-Finalize-Id: e859b330-f6c2-4da7-9aaf-2356607f423a --- product/skills/admin-bypass-sweep/SKILL.md | 37 ++++++++---------- .../tests/fires_step5a_names_probe.md | 4 ++ .../tests/test_step5a_probe_passage.py | 38 +++++++++++++++++++ 3 files changed, 58 insertions(+), 21 deletions(-) create mode 100644 product/skills/admin-bypass-sweep/tests/fires_step5a_names_probe.md create mode 100644 product/skills/admin-bypass-sweep/tests/test_step5a_probe_passage.py diff --git a/product/skills/admin-bypass-sweep/SKILL.md b/product/skills/admin-bypass-sweep/SKILL.md index becb9df3..44aa6969 100644 --- a/product/skills/admin-bypass-sweep/SKILL.md +++ b/product/skills/admin-bypass-sweep/SKILL.md @@ -193,32 +193,27 @@ human: These are mechanically distinguishable, and only the first one is the "real conflict" Step 5 means. Before recording a `CONFLICTING` PR as -blocked, check which case it is, in a disposable worktree outside the -human's main checkout — never touch their primary working tree's branch or -uncommitted state to do this: +blocked, run the rebase probe outside the human's main checkout — never +touch their primary working tree's branch or uncommitted state to do this: ```bash git fetch origin master -git worktree add /tmp//pr- origin/ -cd /tmp//pr- -git checkout -b fix/pr--rebase -git rebase origin/master +scripts/probe_branch_rebase.sh origin/ origin/master ``` -- **Rebase applies clean** (no conflict markers, `git status` clean) — this - was stale mergeability, not a real conflict. Force-push the rebased - branch back to the PR's head with `--force-with-lease` pinned to the - known old SHA, wait for GitHub to recompute (`sleep 5`), confirm - `mergeable` now reads `MERGEABLE`, then continue this PR (and its - children) through Step 4 as normal. -- **Rebase stops with conflict markers** — this is Step 5's real-conflict - case. Run `git rebase --abort`, remove the scratch worktree, and follow - Step 5 as written: stop the chain, record as blocked, move on. Do not - attempt to resolve the markers by picking a side — that part of Step 5 - still applies. - -Remove the scratch worktree (`git worktree remove --force`) once the PR's -fate — merged or genuinely blocked — is decided. +- **Exit 0, `OK`** — the PR rebases cleanly onto current master, so this was + stale mergeability rather than Step 5's real-conflict case. Wait for + GitHub to recompute (`sleep 5`), confirm `mergeable` now reads + `MERGEABLE`, then continue this PR (and its children) through Step 4 as + normal. +- **Exit 1, `FAIL`** — the probe ran and found a real content conflict. + Follow Step 5 as written: stop the chain, record as blocked, and move on. + Do not attempt to resolve the markers by picking a side — that part of + Step 5 still applies. +- **Exit 3, `UNCHECKED`** — the probe could not run, so nothing is proven. + Do not treat this as stale mergeability and do not treat it as Step 5's + real-conflict case; report the unchecked PR separately for manual retry + or setup repair. ## Step 6: Prove the final state diff --git a/product/skills/admin-bypass-sweep/tests/fires_step5a_names_probe.md b/product/skills/admin-bypass-sweep/tests/fires_step5a_names_probe.md new file mode 100644 index 00000000..f5767180 --- /dev/null +++ b/product/skills/admin-bypass-sweep/tests/fires_step5a_names_probe.md @@ -0,0 +1,4 @@ +Fixture assertion: the Step 5a passage in +`product/skills/admin-bypass-sweep/SKILL.md` must name +`scripts/probe_branch_rebase.sh`, must name all three probe outcomes +`OK`, `FAIL`, and `UNCHECKED`, and must not contain `rm -rf`. diff --git a/product/skills/admin-bypass-sweep/tests/test_step5a_probe_passage.py b/product/skills/admin-bypass-sweep/tests/test_step5a_probe_passage.py new file mode 100644 index 00000000..ba7aabef --- /dev/null +++ b/product/skills/admin-bypass-sweep/tests/test_step5a_probe_passage.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import re +import unittest +from pathlib import Path + +SKILL_DIR = Path(__file__).resolve().parents[1] +SKILL = (SKILL_DIR / "SKILL.md").read_text(encoding="utf-8") +FIXTURE = (SKILL_DIR / "tests" / "fires_step5a_names_probe.md").read_text(encoding="utf-8") + + +def step5a_passage() -> str: + match = re.search(r"## Step 5a:.*?(?=\n## Step 6:)", SKILL, re.S) + if not match: + raise AssertionError("Step 5a passage not found") + return match.group(0) + + +class TestStep5aProbePassage(unittest.TestCase): + def test_fixture_names_the_assertions(self): + self.assertIn("scripts/probe_branch_rebase.sh", FIXTURE) + for outcome in ("OK", "FAIL", "UNCHECKED"): + self.assertIn(outcome, FIXTURE) + self.assertIn("rm -rf", FIXTURE) + + def test_step5a_names_probe_and_all_three_outcomes(self): + passage = step5a_passage() + self.assertIn("scripts/probe_branch_rebase.sh", passage) + for outcome in ("OK", "FAIL", "UNCHECKED"): + self.assertRegex(passage, rf"`{outcome}`") + + def test_step5a_no_longer_contains_improvised_removal(self): + self.assertNotIn("rm -rf", step5a_passage()) + + +if __name__ == "__main__": + unittest.main() From 01d01bce74b5aa8ecc24aae01af93d78bbc6de44 Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sat, 12 Sep 2026 18:09:28 +0000 Subject: [PATCH 02/12] =?UTF-8?q?invoker:=20wf-1789236076827-17/verify-swe?= =?UTF-8?q?ep-step5a-wiring=20=E2=80=94=20Run=20the=20deterministic=20proo?= =?UTF-8?q?f=20for=20the=20Step=205a=20wiring.=20Review=20claim:=20A=20fix?= =?UTF-8?q?ture=20asserts=20the=20passage=20names=20the=20probe.=20Review?= =?UTF-8?q?=20lane:=20proof=20Safety=20invariant:=20Proof-only;=20adds=20n?= =?UTF-8?q?o=20product=20behavior.=20Effectiveness=20measurement:=20The=20?= =?UTF-8?q?suite=20fails=20if=20the=20passage=20stops=20naming=20the=20pro?= =?UTF-8?q?be=20or=20drops=20an=20outcome.=20Slice=20rationale:=20One=20pr?= =?UTF-8?q?oof=20slice=20for=20this=20step.=20Architectural=20effect:=20No?= =?UTF-8?q?ne;=20verification=20only.=20Goal:=20Prove=20the=20Step=205a=20?= =?UTF-8?q?wiring=20deterministically.=20Motivation:=20The=20change=20is?= =?UTF-8?q?=20to=20one=20written=20passage,=20so=20the=20proof=20reads=20t?= =?UTF-8?q?hat=20passage.=20Alternative=20considerations:=20Manual=20verif?= =?UTF-8?q?ication=20was=20rejected=20as=20non-deterministic.=20Implementa?= =?UTF-8?q?tion=20details:=20Execute=20the=20command=20below=20as=20the=20?= =?UTF-8?q?terminal=20proof.=20Non-goals:=20No=20product=20edits=20here;?= =?UTF-8?q?=20proof=20only.=20Layer:=20app=5Fregression=20Feature=20state:?= =?UTF-8?q?=20active?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: 3aa932b8-5cc2-4fba-bc5c-115d7a475b4f From 05c285d9f24e30bba85cf0c822a152d354be1acd Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sat, 12 Sep 2026 18:10:15 +0000 Subject: [PATCH 03/12] =?UTF-8?q?invoker:=20wf-1789236076827-17/scrub-hand?= =?UTF-8?q?off-artifacts=20=E2=80=94=20Terminal=20check=20that=20no=20ephe?= =?UTF-8?q?meral=20inter-task=20handoff=20files=20remain.=20Review=20claim?= =?UTF-8?q?:=20The=20workflow=20leaves=20no=20scratch=20handoff=20artifact?= =?UTF-8?q?s=20behind.=20Review=20lane:=20cleanup=20Safety=20invariant:=20?= =?UTF-8?q?Read-only;=20never=20deletes=20files,=20alters=20the=20index,?= =?UTF-8?q?=20or=20commits=20caller=20work.=20Effectiveness=20measurement:?= =?UTF-8?q?=20The=20gate=20fails=20when=20a=20plans/invoker-handoff.*=20or?= =?UTF-8?q?=20lens-*.json=20file=20is=20still=20present=20after=20the=20le?= =?UTF-8?q?af=20tasks=20complete.=20Slice=20rationale:=20One=20terminal=20?= =?UTF-8?q?hygiene=20gate=20for=20the=20workflow.=20Architectural=20effect?= =?UTF-8?q?:=20None;=20check=20only.=20Goal:=20Confirm=20no=20ephemeral=20?= =?UTF-8?q?handoff=20files=20survive=20the=20run.=20Motivation:=20Inter-ta?= =?UTF-8?q?sk=20scratch=20files=20leak=20into=20diffs=20and=20read=20as=20?= =?UTF-8?q?part=20of=20the=20work.=20Alternative=20considerations:=20Delet?= =?UTF-8?q?ing=20them=20automatically=20was=20rejected;=20the=20gate=20rep?= =?UTF-8?q?orts,=20it=20does=20not=20mutate.=20Implementation=20details:?= =?UTF-8?q?=20Run=20scripts/scrub-handoff-artifacts.sh=20without=20--apply?= =?UTF-8?q?.=20Non-goals:=20No=20deletion,=20no=20index=20changes,=20no=20?= =?UTF-8?q?commits.=20Layer:=20app=5Fregression=20Feature=20state:=20activ?= =?UTF-8?q?e?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: 2c57922e-973b-4c94-a350-bde1591b77ee From 535aa396ea9809154396018cb97102c5129e32e9 Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sat, 12 Sep 2026 18:24:13 +0000 Subject: [PATCH 04/12] =?UTF-8?q?invoker:=20wf-1789236110851-19/implement-?= =?UTF-8?q?sweep-diff-read-gate=20=E2=80=94=20Require=20a=20full=20diff=20?= =?UTF-8?q?read=20per=20PR=20before=20an=20admin=20merge,=20and=20mark=20a?= =?UTF-8?q?=20narrowed=20read=20unchecked.=20Review=20claim:=20A=20PR=20wh?= =?UTF-8?q?ose=20diff=20was=20read=20through=20a=20truncating=20filter=20i?= =?UTF-8?q?s=20unchecked,=20never=20reviewed.=20Review=20lane:=20behavior?= =?UTF-8?q?=20Safety=20invariant:=20A=20truncated=20or=20filtered=20diff?= =?UTF-8?q?=20read=20marks=20that=20PR=20unchecked,=20never=20reviewed;=20?= =?UTF-8?q?no=20PR=20that=20is=20reviewable=20today=20becomes=20unmergeabl?= =?UTF-8?q?e,=20because=20the=20gate=20reports=20rather=20than=20blocks=20?= =?UTF-8?q?the=20merge=20command.=20Effectiveness=20measurement:=20A=20fix?= =?UTF-8?q?ture=20review=20command=20piped=20through=20`head`=20marks=20it?= =?UTF-8?q?s=20PR=20unchecked,=20and=20the=20same=20command=20without=20th?= =?UTF-8?q?e=20pipe=20marks=20it=20reviewed.=20Both=20asserted=20in=20the?= =?UTF-8?q?=20skill's=20test=20directory.=20Slice=20rationale:=20One=20ste?= =?UTF-8?q?p=20of=20the=20sweep=20skill's=20review=20procedure;=20stacked?= =?UTF-8?q?=20behind=20the=20tally=20step=20because=20both=20edit=20the=20?= =?UTF-8?q?same=20SKILL.md.=20Architectural=20effect:=20Step=204=20gains?= =?UTF-8?q?=20an=20explicit=20read-completeness=20precondition=20expressed?= =?UTF-8?q?=20as=20the=20same=20three-outcome=20contract=20the=20rest=20of?= =?UTF-8?q?=20the=20stack=20uses.=20Goal:=20Change=20Step=204=20of=20produ?= =?UTF-8?q?ct/skills/admin-bypass-sweep/SKILL.md=20so=20each=20PR's=20diff?= =?UTF-8?q?=20is=20written=20to=20a=20file,=20fully=20read,=20and=20assert?= =?UTF-8?q?ed=20line-complete=20before=20`gh=20pr=20merge=20--admin`,=20wi?= =?UTF-8?q?th=20a=20narrowed=20read=20reported=20as=20unchecked.=20Motivat?= =?UTF-8?q?ion:=20A=20sweep=20reported=20'Reviewed=20all=2019'=20after=20t?= =?UTF-8?q?wo=20commands=20that=20between=20them=20truncated=20one=20PR=20?= =?UTF-8?q?entirely=20and=20filtered=20every=20test=20file=20out=20of=20fi?= =?UTF-8?q?ve=20others.=20The=20largest=20diff=20in=20the=20sweep,=20touch?= =?UTF-8?q?ing=20six=20hook=20detectors,=20was=20cleared=20for=20merge=20o?= =?UTF-8?q?n=20about=2021%=20of=20its=20non-test=20changes.=20Alternative?= =?UTF-8?q?=20considerations:=20Telling=20the=20operator=20not=20to=20pipe?= =?UTF-8?q?=20through=20head=20was=20rejected=20as=20unenforceable=20prose?= =?UTF-8?q?.=20Blocking=20the=20merge=20outright=20on=20a=20narrowed=20rea?= =?UTF-8?q?d=20was=20rejected=20for=20this=20slice:=20the=20skill's=20own?= =?UTF-8?q?=20consent=20model=20puts=20the=20merge=20decision=20with=20the?= =?UTF-8?q?=20human,=20so=20the=20gate=20reports=20unchecked=20and=20the?= =?UTF-8?q?=20operator=20decides.=20Implementation=20details:=20In=20Step?= =?UTF-8?q?=204,=20replace=20the=20ad=20hoc=20`gh=20pr=20diff`=20skim=20wi?= =?UTF-8?q?th:=20write=20each=20PR's=20diff=20to=20a=20file,=20record=20`w?= =?UTF-8?q?c=20-l`,=20read=20the=20whole=20file,=20and=20assert=20lines=5F?= =?UTF-8?q?read=20equals=20the=20recorded=20total=20before=20merging=20tha?= =?UTF-8?q?t=20PR.=20Any=20read=20through=20head,=20tail,=20grep,=20awk,?= =?UTF-8?q?=20or=20sed=20marks=20that=20PR=20unchecked.=20Use=20the=20same?= =?UTF-8?q?=20three=20outcomes=20as=20the=20rest=20of=20the=20stack:=20rev?= =?UTF-8?q?iewed,=20flagged,=20unchecked.=20Add=20fixtures=20under=20the?= =?UTF-8?q?=20skill's=20tests=20directory=20for=20a=20full=20read=20and=20?= =?UTF-8?q?a=20truncated=20read.=20Non-goals:=20Does=20not=20change=20what?= =?UTF-8?q?=20counts=20as=20a=20review=20finding,=20does=20not=20add=20an?= =?UTF-8?q?=20automated=20code=20review,=20does=20not=20alter=20the=20cons?= =?UTF-8?q?ent=20requirements=20in=20the=20STOP=20section,=20and=20does=20?= =?UTF-8?q?not=20touch=20any=20other=20skill.=20Layer:=20domain=20Feature?= =?UTF-8?q?=20state:=20active=20Files:=20product/skills/admin-bypass-sweep?= =?UTF-8?q?/SKILL.md,=20product/skills/admin-bypass-sweep/tests/=20Change?= =?UTF-8?q?=20types:=20-=20product/skills/admin-bypass-sweep/SKILL.md:=20d?= =?UTF-8?q?ocs-only=20-=20product/skills/admin-bypass-sweep/tests/fires=5F?= =?UTF-8?q?truncated=5Fdiff=5Fread.md:=20create=20-=20product/skills/admin?= =?UTF-8?q?-bypass-sweep/tests/stays=5Fsilent=5Ffull=5Fdiff=5Fread.md:=20c?= =?UTF-8?q?reate=20Acceptance=20criteria:=20-=20`python3=20-m=20unittest?= =?UTF-8?q?=20discover=20-s=20product/skills/admin-bypass-sweep/tests=20-v?= =?UTF-8?q?`=20exits=200.=20-=20Step=204=20requires=20a=20recorded=20line?= =?UTF-8?q?=20count=20and=20a=20full=20read=20before=20`gh=20pr=20merge=20?= =?UTF-8?q?--admin`.=20-=20A=20read=20through=20head,=20tail,=20grep,=20aw?= =?UTF-8?q?k,=20or=20sed=20is=20documented=20as=20unchecked.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Solution: Require a full diff read per PR before an admin merge, and mark a narrowed read unchecked. Review claim: A PR whose diff was read through a truncating filter is unchecked, never reviewed. Review lane: behavior Safety invariant: A truncated or filtered diff read marks that PR unchecked, never reviewed; no PR that is reviewable today becomes unmergeable, because the gate reports rather than blocks the merge command. Effectiveness measurement: A fixture review command piped through `head` marks its PR unchecked, and the same command without the pipe marks it reviewed. Both asserted in the skill's test directory. Slice rationale: One step of the sweep skill's review procedure; stacked behind the tally step because both edit the same SKILL.md. Architectural effect: Step 4 gains an explicit read-completeness precondition expressed as the same three-outcome contract the rest of the stack uses. Goal: Change Step 4 of product/skills/admin-bypass-sweep/SKILL.md so each PR's diff is written to a file, fully read, and asserted line-complete before `gh pr merge --admin`, with a narrowed read reported as unchecked. Motivation: A sweep reported 'Reviewed all 19' after two commands that between them truncated one PR entirely and filtered every test file out of five others. The largest diff in the sweep, touching six hook detectors, was cleared for merge on about 21% of its non-test changes. Alternative considerations: Telling the operator not to pipe through head was rejected as unenforceable prose. Blocking the merge outright on a narrowed read was rejected for this slice: the skill's own consent model puts the merge decision with the human, so the gate reports unchecked and the operator decides. Implementation details: In Step 4, replace the ad hoc `gh pr diff` skim with: write each PR's diff to a file, record `wc -l`, read the whole file, and assert lines_read equals the recorded total before merging that PR. Any read through head, tail, grep, awk, or sed marks that PR unchecked. Use the same three outcomes as the rest of the stack: reviewed, flagged, unchecked. Add fixtures under the skill's tests directory for a full read and a truncated read. Non-goals: Does not change what counts as a review finding, does not add an automated code review, does not alter the consent requirements in the STOP section, and does not touch any other skill. Layer: domain Feature state: active Files: product/skills/admin-bypass-sweep/SKILL.md, product/skills/admin-bypass-sweep/tests/ Change types: - product/skills/admin-bypass-sweep/SKILL.md: docs-only - product/skills/admin-bypass-sweep/tests/fires_truncated_diff_read.md: create - product/skills/admin-bypass-sweep/tests/stays_silent_full_diff_read.md: create Acceptance criteria: - `python3 -m unittest discover -s product/skills/admin-bypass-sweep/tests -v` exits 0. - Step 4 requires a recorded line count and a full read before `gh pr merge --admin`. - A read through head, tail, grep, awk, or sed is documented as unchecked. Invoker-Finalize-Id: 86232251-f80f-4a4b-a379-36417182163d --- product/skills/admin-bypass-sweep/SKILL.md | 41 +++++++++++-- .../tests/fixture_complete_diff_read.md | 20 +++++++ .../tests/fixture_truncated_diff_read.md | 23 +++++++ .../tests/test_diff_read_gate.py | 60 +++++++++++++++++++ 4 files changed, 140 insertions(+), 4 deletions(-) create mode 100644 product/skills/admin-bypass-sweep/tests/fixture_complete_diff_read.md create mode 100644 product/skills/admin-bypass-sweep/tests/fixture_truncated_diff_read.md create mode 100644 product/skills/admin-bypass-sweep/tests/test_diff_read_gate.py diff --git a/product/skills/admin-bypass-sweep/SKILL.md b/product/skills/admin-bypass-sweep/SKILL.md index becb9df3..1558da55 100644 --- a/product/skills/admin-bypass-sweep/SKILL.md +++ b/product/skills/admin-bypass-sweep/SKILL.md @@ -141,10 +141,43 @@ diffs; stop and re-derive the safe approach first. Both requirements in the STOP section must be satisfied before this step runs. -Skim `gh pr diff ` for each PR before merging it, even under consent — -this is the only review most of these PRs get, since the merge bypasses -required checks entirely. The human's consent authorizes bypassing CI; it -does not stand in for having actually looked at what's being merged. +Before any `gh pr merge --admin`, write that PR's diff to a file, record +the file's total line count with `wc -l`, read the whole file, and assert +that the number of lines read equals the recorded total. This is the only +review most of these PRs get, since the merge bypasses required checks +entirely. The human's consent authorizes bypassing CI; it does not stand in +for having actually looked at what's being merged. + +Use this read-completeness precondition for each PR: + +```bash +pr= +diff_file="$(mktemp -t admin-bypass-pr-${pr}.diff.XXXXXX)" +gh pr diff "$pr" --repo / > "$diff_file" +diff_lines="$(wc -l < "$diff_file" | tr -d ' ')" +nl -ba "$diff_file" +lines_read= +test "$lines_read" = "$diff_lines" +``` + +Classify the PR before merging it, using the same three outcomes as the +rest of this procedure: + +- `reviewed` — the diff file was read line-complete, `lines_read` equals + `diff_lines`, and no review finding was found. +- `flagged` — the diff file was read line-complete, `lines_read` equals + `diff_lines`, and one or more review findings were recorded for the + human. +- `unchecked` — the diff read was narrowed, filtered, or truncated, so + line completeness was not established. + +A read through `head`, `tail`, `grep`, `awk`, or `sed` is a narrowed read: +mark that PR `unchecked`, never `reviewed`, even if the visible lines look +fine. The operator may not report such a PR as reviewed. This check reports +the PR's review state; it does not create a new authorization path or block +the human from deciding to proceed with the admin merge anyway. If an +`unchecked` PR is merged, the final report must still list it as +`unchecked`, not reviewed. For a single-PR stack: diff --git a/product/skills/admin-bypass-sweep/tests/fixture_complete_diff_read.md b/product/skills/admin-bypass-sweep/tests/fixture_complete_diff_read.md new file mode 100644 index 00000000..b3baba78 --- /dev/null +++ b/product/skills/admin-bypass-sweep/tests/fixture_complete_diff_read.md @@ -0,0 +1,20 @@ +User invokes `/admin-bypass-sweep` with the required consent sentence and the +operator reaches Step 4 for PR 42. + +The operator writes the full diff to a file: + +```bash +pr=42 +diff_file="$(mktemp -t admin-bypass-pr-${pr}.diff.XXXXXX)" +gh pr diff "$pr" --repo neko/example > "$diff_file" +diff_lines="$(wc -l < "$diff_file" | tr -d ' ')" +nl -ba "$diff_file" +lines_read=184 +test "$lines_read" = "$diff_lines" +``` + +The recorded `diff_lines` value is 184, and the complete `nl -ba` output ends +at line 184. No review finding is found. + +Expected outcome: PR 42 may be reported as `reviewed` before the operator runs +`gh pr merge 42 --repo neko/example --admin --squash`. diff --git a/product/skills/admin-bypass-sweep/tests/fixture_truncated_diff_read.md b/product/skills/admin-bypass-sweep/tests/fixture_truncated_diff_read.md new file mode 100644 index 00000000..3c1a1646 --- /dev/null +++ b/product/skills/admin-bypass-sweep/tests/fixture_truncated_diff_read.md @@ -0,0 +1,23 @@ +User invokes `/admin-bypass-sweep` with the required consent sentence and the +operator reaches Step 4 for PR 77. + +The operator writes the diff to a file, but reads it through a truncating +filter: + +```bash +pr=77 +diff_file="$(mktemp -t admin-bypass-pr-${pr}.diff.XXXXXX)" +gh pr diff "$pr" --repo neko/example > "$diff_file" +diff_lines="$(wc -l < "$diff_file" | tr -d ' ')" +head -200 "$diff_file" +lines_read=200 +test "$lines_read" = "$diff_lines" +``` + +The recorded `diff_lines` value is 913, and only the first 200 lines were read +through `head`. + +Expected outcome: PR 77 is `unchecked`, not `reviewed`. The operator may not +report PR 77 as reviewed, even if the visible lines look fine. If the human +decides to proceed with `gh pr merge 77 --repo neko/example --admin --squash`, +the final report must still list PR 77 as `unchecked`. diff --git a/product/skills/admin-bypass-sweep/tests/test_diff_read_gate.py b/product/skills/admin-bypass-sweep/tests/test_diff_read_gate.py new file mode 100644 index 00000000..254c27e1 --- /dev/null +++ b/product/skills/admin-bypass-sweep/tests/test_diff_read_gate.py @@ -0,0 +1,60 @@ +from pathlib import Path +import re +import unittest + + +ROOT = Path(__file__).resolve().parents[1] +SKILL = ROOT / "SKILL.md" +TESTS = ROOT / "tests" + + +class DiffReadGateTests(unittest.TestCase): + def setUp(self): + self.skill = SKILL.read_text() + step4_match = re.search( + r"## Step 4: Merge each stack, bottom-up(?P.*?)## Step 5:", + self.skill, + re.S, + ) + self.assertIsNotNone(step4_match, "Step 4 section is present") + self.step4 = step4_match.group("body") + + def test_step4_requires_recorded_line_count_and_full_read_before_admin_merge(self): + merge_index = self.step4.index("gh pr merge ") + pre_merge = self.step4[:merge_index] + + self.assertIn("Before any `gh pr merge --admin`", pre_merge) + self.assertIn("gh pr diff", pre_merge) + self.assertIn("> \"$diff_file\"", pre_merge) + self.assertIn("wc -l", pre_merge) + self.assertIn("nl -ba \"$diff_file\"", pre_merge) + self.assertIn('test "$lines_read" = "$diff_lines"', pre_merge) + self.assertIn("number of lines read equals the recorded total", pre_merge) + + def test_step4_documents_narrowed_reads_as_unchecked_not_reviewed(self): + for command in ("head", "tail", "grep", "awk", "sed"): + self.assertIn(f"`{command}`", self.step4) + + self.assertIn("mark that PR `unchecked`, never `reviewed`", self.step4) + self.assertIn("may not report such a PR as reviewed", self.step4) + + def test_complete_read_fixture_marks_pr_reviewed(self): + fixture = (TESTS / "fixture_complete_diff_read.md").read_text() + + self.assertIn("wc -l", fixture) + self.assertIn("nl -ba", fixture) + self.assertIn("lines_read=184", fixture) + self.assertIn("Expected outcome: PR 42 may be reported as `reviewed`", fixture) + + def test_truncated_read_fixture_marks_pr_unchecked(self): + fixture = (TESTS / "fixture_truncated_diff_read.md").read_text() + fixture_lower = fixture.lower() + + self.assertIn("head -200", fixture) + self.assertIn("diff_lines` value is 913", fixture) + self.assertIn("PR 77 is `unchecked`, not `reviewed`", fixture) + self.assertIn("operator may not\nreport pr 77 as reviewed", fixture_lower) + + +if __name__ == "__main__": + unittest.main() From df1cf2dc4a123a0198de29eace777496ffff49b4 Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sat, 12 Sep 2026 18:24:57 +0000 Subject: [PATCH 05/12] =?UTF-8?q?invoker:=20wf-1789236110851-19/verify-swe?= =?UTF-8?q?ep-diff-read-gate=20=E2=80=94=20Run=20the=20deterministic=20pro?= =?UTF-8?q?of=20for=20the=20full-diff-read=20gate.=20Review=20claim:=20The?= =?UTF-8?q?=20skill's=20fixtures=20assert=20a=20truncated=20diff=20read=20?= =?UTF-8?q?is=20reported=20unchecked.=20Review=20lane:=20proof=20Safety=20?= =?UTF-8?q?invariant:=20Proof-only;=20adds=20no=20product=20behavior.=20Ef?= =?UTF-8?q?fectiveness=20measurement:=20The=20suite=20fails=20if=20the=20t?= =?UTF-8?q?runcated-read=20fixture=20is=20reported=20as=20reviewed.=20Slic?= =?UTF-8?q?e=20rationale:=20One=20proof=20slice=20for=20this=20step.=20Arc?= =?UTF-8?q?hitectural=20effect:=20None;=20verification=20only.=20Goal:=20P?= =?UTF-8?q?rove=20the=20full-diff-read=20gate=20deterministically.=20Motiv?= =?UTF-8?q?ation:=20The=20defect=20is=20an=20over-claimed=20review,=20so?= =?UTF-8?q?=20the=20proof=20asserts=20the=20narrowed=20case=20is=20named?= =?UTF-8?q?=20unchecked.=20Alternative=20considerations:=20Manual=20verifi?= =?UTF-8?q?cation=20was=20rejected=20as=20non-deterministic.=20Implementat?= =?UTF-8?q?ion=20details:=20Execute=20the=20command=20below=20as=20the=20t?= =?UTF-8?q?erminal=20proof.=20Non-goals:=20No=20product=20edits=20here;=20?= =?UTF-8?q?proof=20only.=20Layer:=20app=5Fregression=20Feature=20state:=20?= =?UTF-8?q?active?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: 4cde49f2-d3fe-41d7-ad5f-4c24fb36f491 From 6cd1a053b2c52546d09671366102d18f196fc57c Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sat, 12 Sep 2026 18:25:46 +0000 Subject: [PATCH 06/12] =?UTF-8?q?invoker:=20wf-1789236110851-19/scrub-hand?= =?UTF-8?q?off-artifacts=20=E2=80=94=20Terminal=20check=20that=20no=20ephe?= =?UTF-8?q?meral=20inter-task=20handoff=20files=20remain.=20Review=20claim?= =?UTF-8?q?:=20The=20workflow=20leaves=20no=20scratch=20handoff=20artifact?= =?UTF-8?q?s=20behind.=20Review=20lane:=20cleanup=20Safety=20invariant:=20?= =?UTF-8?q?Read-only;=20never=20deletes=20files,=20alters=20the=20index,?= =?UTF-8?q?=20or=20commits=20caller=20work.=20Effectiveness=20measurement:?= =?UTF-8?q?=20The=20gate=20fails=20when=20a=20plans/invoker-handoff.*=20or?= =?UTF-8?q?=20lens-*.json=20file=20is=20still=20present=20after=20the=20le?= =?UTF-8?q?af=20tasks=20complete.=20Slice=20rationale:=20One=20terminal=20?= =?UTF-8?q?hygiene=20gate=20for=20the=20workflow.=20Architectural=20effect?= =?UTF-8?q?:=20None;=20check=20only.=20Goal:=20Confirm=20no=20ephemeral=20?= =?UTF-8?q?handoff=20files=20survive=20the=20run.=20Motivation:=20Inter-ta?= =?UTF-8?q?sk=20scratch=20files=20leak=20into=20diffs=20and=20read=20as=20?= =?UTF-8?q?part=20of=20the=20work.=20Alternative=20considerations:=20Delet?= =?UTF-8?q?ing=20them=20automatically=20was=20rejected;=20the=20gate=20rep?= =?UTF-8?q?orts,=20it=20does=20not=20mutate.=20Implementation=20details:?= =?UTF-8?q?=20Run=20scripts/scrub-handoff-artifacts.sh=20without=20--apply?= =?UTF-8?q?.=20Non-goals:=20No=20deletion,=20no=20index=20changes,=20no=20?= =?UTF-8?q?commits.=20Layer:=20app=5Fregression=20Feature=20state:=20activ?= =?UTF-8?q?e?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: 401825e5-c8a7-488b-9472-5b8212a9ca68 From f2435ab7e581130a870ecc0dfde53d7dca912b0b Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sun, 13 Sep 2026 06:04:17 +0000 Subject: [PATCH 07/12] =?UTF-8?q?invoker:=20wf-1789279240280-10/implement-?= =?UTF-8?q?judge-investigate-mode=20=E2=80=94=20Let=20an=20llm-judge=20job?= =?UTF-8?q?=20opt=20into=20a=20read-only=20investigate=20mode=20with=20its?= =?UTF-8?q?=20own=20time=20limit=20and=20working=20folder.=20Review=20clai?= =?UTF-8?q?m:=20A=20job=20carrying=20"mode":=20"investigate"=20runs=20on?= =?UTF-8?q?=20a=20second=20runner=20set=20whose=20Claude=20argv=20grants?= =?UTF-8?q?=20only=20the=20Read,=20Grep=20and=20Glob=20tools=20and=20forbi?= =?UTF-8?q?ds=20Write,=20Edit,=20NotebookEdit=20and=20Bash,=20with=20the?= =?UTF-8?q?=20job's=20own=20timeout=5Fseconds=20and=20cwd;=20a=20job=20lac?= =?UTF-8?q?king=20that=20key=20runs=20exactly=20as=20today.=20Review=20lan?= =?UTF-8?q?e:=20behavior=20Safety=20invariant:=20An=20investigate=20job=20?= =?UTF-8?q?never=20receives=20a=20tool=20that=20can=20write,=20edit,=20or?= =?UTF-8?q?=20execute=20shell=20text.=20The=20Claude=20runner=20argv=20alw?= =?UTF-8?q?ays=20carries=20--allowedTools=20Read=20Grep=20Glob=20and=20--d?= =?UTF-8?q?isallowedTools=20Write=20Edit=20NotebookEdit=20Bash,=20the=20Co?= =?UTF-8?q?dex=20runner=20always=20carries=20--sandbox=20read-only,=20and?= =?UTF-8?q?=20cursor-agent=20is=20never=20used=20for=20investigate=20jobs?= =?UTF-8?q?=20because=20it=20has=20no=20read-only=20switch.=20Effectivenes?= =?UTF-8?q?s=20measurement:=20New=20unit=20tests=20pin=20the=20exact=20arg?= =?UTF-8?q?v=20of=20each=20investigate=20runner=20and=20prove=20a=20defaul?= =?UTF-8?q?t=20job=20still=20gets=2060=20seconds=20in=20an=20empty=20temp?= =?UTF-8?q?=20folder.=20The=20PR=20body=20pastes=20the=20live=20probe=20in?= =?UTF-8?q?=20which=20claude=20-p=20with=20this=20argv=20was=20asked=20to?= =?UTF-8?q?=20Write=20a=20file=20and=20to=20touch=20one=20through=20Bash,?= =?UTF-8?q?=20answered=20{"wrote":false,"read":true,"bash":false},=20and?= =?UTF-8?q?=20neither=20target=20file=20existed=20afterwards.=20Slice=20ra?= =?UTF-8?q?tionale:=20One=20behavior=20slice=20inside=20the=20judge,=20dor?= =?UTF-8?q?mant=20until=20a=20hook=20asks=20for=20an=20investigate=20job,?= =?UTF-8?q?=20so=20a=20reviewer=20can=20judge=20the=20permission=20argv=20?= =?UTF-8?q?on=20its=20own.=20Architectural=20effect:=20llm-judge=20gains?= =?UTF-8?q?=20a=20second,=20read-only=20runner=20set=20chosen=20per=20job;?= =?UTF-8?q?=20the=20default=20runner=20set=20and=20every=20current=20calle?= =?UTF-8?q?r=20stay=20unchanged.=20Goal:=20Give=20later=20hooks=20a=20safe?= =?UTF-8?q?=20way=20to=20ask=20a=20model=20to=20check=20a=20claim=20by=20r?= =?UTF-8?q?eading=20files=20on=20this=20machine.=20Motivation:=20A=20backt?= =?UTF-8?q?est=20of=2044=20unverified=20tags=20from=20the=20last=2014=20da?= =?UTF-8?q?ys=20found=2033=20whose=20stated=20blocker=20was=20false;=20mos?= =?UTF-8?q?t=20could=20have=20been=20checked=20by=20reading=20a=20log,=20a?= =?UTF-8?q?=20transcript,=20or=20a=20source=20file=20already=20on=20disk.?= =?UTF-8?q?=20Today=20each=20judge=20runner=20gets=2060=20seconds=20in=20a?= =?UTF-8?q?n=20empty=20temp=20folder=20with=20no=20tool=20grants,=20so=20i?= =?UTF-8?q?t=20cannot=20read=20anything.=20Alternative=20considerations:?= =?UTF-8?q?=20claude=20--permission-mode=20plan=20was=20rejected=20after?= =?UTF-8?q?=20a=20live=20probe=20wrote=20a=20plan=20file=20under=20~/.clau?= =?UTF-8?q?de/plans.=20A=20Bash=20prefix=20allowlist=20such=20as=20Bash(gi?= =?UTF-8?q?t=20log:*)=20was=20rejected=20after=20a=20live=20probe=20blocke?= =?UTF-8?q?d=20even=20plain=20git=20log,=20so=20shell=20access=20stays=20o?= =?UTF-8?q?ut=20of=20this=20slice.=20Implementation=20details:=20In=20engi?= =?UTF-8?q?ne/hooks/llm-judge/judge.py=20add=20INVESTIGATE=5FRUNNERS=20bes?= =?UTF-8?q?ide=20DEFAULT=5FRUNNERS=20holding=20the=20codex=20and=20claude?= =?UTF-8?q?=20argv=20given=20under=20Runner=20argv.=20Make=20runners()=20t?= =?UTF-8?q?ake=20the=20job=20mode=20and=20pick=20INVESTIGATE=5FRUNNERS=20f?= =?UTF-8?q?or=20"investigate";=20CATSTACK=5FLLM=5FJUDGE=5FRUNNERS=20still?= =?UTF-8?q?=20replaces=20whichever=20set=20is=20picked.=20Make=20run=5Frun?= =?UTF-8?q?ner=20accept=20timeout=5Fseconds=20(default=20TIMEOUT=5FSECONDS?= =?UTF-8?q?,=20capped=20at=20INVESTIGATE=5FTIMEOUT=5FCAP=20=3D=20600)=20an?= =?UTF-8?q?d=20cwd=20(used=20only=20when=20it=20is=20an=20absolute=20path?= =?UTF-8?q?=20to=20a=20folder=20that=20exists,=20else=20the=20temp=20folde?= =?UTF-8?q?r=20plus=20a=20judge.log=20line).=20Thread=20mode,=20timeout=5F?= =?UTF-8?q?seconds=20and=20cwd=20from=20the=20job=20through=20ask()=20and?= =?UTF-8?q?=20run=5Fjob().=20Runner=20argv:=20codex=20exec=20--skip-git-re?= =?UTF-8?q?po-check=20--sandbox=20read-only=20-c=20notify=3D[]=20PROMPT,?= =?UTF-8?q?=20and=20claude=20-p=20--model=20haiku=20--settings=20'{"disabl?= =?UTF-8?q?eAllHooks":=20true}'=20--allowedTools=20Read=20Grep=20Glob=20--?= =?UTF-8?q?disallowedTools=20Write=20Edit=20NotebookEdit=20Bash=20--=20PRO?= =?UTF-8?q?MPT.=20Non-goals:=20No=20hook=20asks=20for=20an=20investigate?= =?UTF-8?q?=20job=20in=20this=20slice.=20No=20inbox=20edit,=20no=20change?= =?UTF-8?q?=20to=20DEFAULT=5FRUNNERS,=20wrong-check-reflect=20or=20diu-sto?= =?UTF-8?q?p.=20No=20shell=20or=20network=20access=20for=20the=20investiga?= =?UTF-8?q?tor.=20Layer:=20domain=20Feature=20state:=20dormant=20Files:=20?= =?UTF-8?q?-=20engine/hooks/llm-judge/judge.py=20-=20engine/hooks/llm-judg?= =?UTF-8?q?e/tests/test=5Fjudge.py=20Change=20types:=20-=20engine/hooks/ll?= =?UTF-8?q?m-judge/judge.py:=20modify=20-=20engine/hooks/llm-judge/tests/t?= =?UTF-8?q?est=5Fjudge.py:=20modify=20Acceptance=20criteria:=20-=20Tests?= =?UTF-8?q?=20whose=20names=20contain=20"investigate"=20pass=20and=20pin?= =?UTF-8?q?=20both=20investigate=20runner=20argv=20exactly.=20-=20A=20test?= =?UTF-8?q?=20proves=20a=20job=20lacking=20a=20mode=20key=20still=20runs?= =?UTF-8?q?=20DEFAULT=5FRUNNERS=20with=20a=2060=20second=20limit.=20-=20A?= =?UTF-8?q?=20test=20proves=20timeout=5Fseconds=20above=20600=20is=20cappe?= =?UTF-8?q?d=20at=20600.=20-=20A=20test=20whose=20name=20contains=20"missi?= =?UTF-8?q?ng"=20proves=20a=20cwd=20that=20does=20not=20exist=20falls=20ba?= =?UTF-8?q?ck=20to=20a=20temp=20folder=20and=20logs=20it.=20-=20python3=20?= =?UTF-8?q?scripts/check=5Fno=5Fnew=5Fcomments.py=20--base=20origin/main?= =?UTF-8?q?=20exits=200.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Solution: Let an llm-judge job opt into a read-only investigate mode with its own time limit and working folder. Review claim: A job carrying "mode": "investigate" runs on a second runner set whose Claude argv grants only the Read, Grep and Glob tools and forbids Write, Edit, NotebookEdit and Bash, with the job's own timeout_seconds and cwd; a job lacking that key runs exactly as today. Review lane: behavior Safety invariant: An investigate job never receives a tool that can write, edit, or execute shell text. The Claude runner argv always carries --allowedTools Read Grep Glob and --disallowedTools Write Edit NotebookEdit Bash, the Codex runner always carries --sandbox read-only, and cursor-agent is never used for investigate jobs because it has no read-only switch. Effectiveness measurement: New unit tests pin the exact argv of each investigate runner and prove a default job still gets 60 seconds in an empty temp folder. The PR body pastes the live probe in which claude -p with this argv was asked to Write a file and to touch one through Bash, answered {"wrote":false,"read":true,"bash":false}, and neither target file existed afterwards. Slice rationale: One behavior slice inside the judge, dormant until a hook asks for an investigate job, so a reviewer can judge the permission argv on its own. Architectural effect: llm-judge gains a second, read-only runner set chosen per job; the default runner set and every current caller stay unchanged. Goal: Give later hooks a safe way to ask a model to check a claim by reading files on this machine. Motivation: A backtest of 44 unverified tags from the last 14 days found 33 whose stated blocker was false; most could have been checked by reading a log, a transcript, or a source file already on disk. Today each judge runner gets 60 seconds in an empty temp folder with no tool grants, so it cannot read anything. Alternative considerations: claude --permission-mode plan was rejected after a live probe wrote a plan file under ~/.claude/plans. A Bash prefix allowlist such as Bash(git log:*) was rejected after a live probe blocked even plain git log, so shell access stays out of this slice. Implementation details: In engine/hooks/llm-judge/judge.py add INVESTIGATE_RUNNERS beside DEFAULT_RUNNERS holding the codex and claude argv given under Runner argv. Make runners() take the job mode and pick INVESTIGATE_RUNNERS for "investigate"; CATSTACK_LLM_JUDGE_RUNNERS still replaces whichever set is picked. Make run_runner accept timeout_seconds (default TIMEOUT_SECONDS, capped at INVESTIGATE_TIMEOUT_CAP = 600) and cwd (used only when it is an absolute path to a folder that exists, else the temp folder plus a judge.log line). Thread mode, timeout_seconds and cwd from the job through ask() and run_job(). Runner argv: codex exec --skip-git-repo-check --sandbox read-only -c notify=[] PROMPT, and claude -p --model haiku --settings '{"disableAllHooks": true}' --allowedTools Read Grep Glob --disallowedTools Write Edit NotebookEdit Bash -- PROMPT. Non-goals: No hook asks for an investigate job in this slice. No inbox edit, no change to DEFAULT_RUNNERS, wrong-check-reflect or diu-stop. No shell or network access for the investigator. Layer: domain Feature state: dormant Files: - engine/hooks/llm-judge/judge.py - engine/hooks/llm-judge/tests/test_judge.py Change types: - engine/hooks/llm-judge/judge.py: modify - engine/hooks/llm-judge/tests/test_judge.py: modify Acceptance criteria: - Tests whose names contain "investigate" pass and pin both investigate runner argv exactly. - A test proves a job lacking a mode key still runs DEFAULT_RUNNERS with a 60 second limit. - A test proves timeout_seconds above 600 is capped at 600. - A test whose name contains "missing" proves a cwd that does not exist falls back to a temp folder and logs it. - python3 scripts/check_no_new_comments.py --base origin/main exits 0. Invoker-Finalize-Id: 890bf8a3-d647-4273-855b-8ca21b411f55 --- engine/hooks/llm-judge/judge.py | 43 ++++++++--- engine/hooks/llm-judge/tests/test_judge.py | 84 ++++++++++++++++++++++ 2 files changed, 116 insertions(+), 11 deletions(-) diff --git a/engine/hooks/llm-judge/judge.py b/engine/hooks/llm-judge/judge.py index a81f19b7..12696205 100644 --- a/engine/hooks/llm-judge/judge.py +++ b/engine/hooks/llm-judge/judge.py @@ -14,6 +14,7 @@ import uuid TIMEOUT_SECONDS = 60 +INVESTIGATE_TIMEOUT_CAP = 600 KILL_GRACE_SECONDS = 5 REASON_LIMIT = 300 PROMPT_SLOT = "{prompt}" @@ -25,6 +26,10 @@ ("claude", ["claude", "-p", "--model", "haiku", "--settings", '{"disableAllHooks": true}', PROMPT_SLOT]), ("cursor", ["cursor-agent", "-p", "--output-format", "text", PROMPT_SLOT]), ) +INVESTIGATE_RUNNERS = ( + ("codex", ["codex", "exec", "--skip-git-repo-check", "--sandbox", "read-only", "-c", "notify=[]", PROMPT_SLOT]), + ("claude", ["claude", "-p", "--model", "haiku", "--settings", '{"disableAllHooks": true}', "--allowedTools", "Read", "Grep", "Glob", "--disallowedTools", "Write", "Edit", "NotebookEdit", "Bash", "--", PROMPT_SLOT]), +) def state_root() -> str: @@ -49,10 +54,11 @@ def valid_runner(entry: object) -> bool: ) -def runners() -> list[tuple[str, list[str]]]: +def runners(mode: object = None) -> list[tuple[str, list[str]]]: + default = INVESTIGATE_RUNNERS if mode == "investigate" else DEFAULT_RUNNERS raw = os.environ.get(RUNNERS_ENV) if not raw: - return [(name, list(argv)) for name, argv in DEFAULT_RUNNERS] + return [(name, list(argv)) for name, argv in default] try: parsed = json.loads(raw) except ValueError as exc: @@ -96,13 +102,26 @@ def stop_group(proc: subprocess.Popen) -> str: return stderr or "" -def run_runner(name: str, argv: list[str], prompt: str) -> tuple[dict, dict | None]: +def bounded_timeout(timeout_seconds: object) -> int | float: + if isinstance(timeout_seconds, bool) or not isinstance(timeout_seconds, (int, float)): + timeout_seconds = TIMEOUT_SECONDS + return min(timeout_seconds, INVESTIGATE_TIMEOUT_CAP) + + +def run_runner(name: str, argv: list[str], prompt: str, timeout_seconds: object = TIMEOUT_SECONDS, cwd: object = None) -> tuple[dict, dict | None]: if shutil.which(argv[0]) is None: return failed(name, "not installed"), None command = [prompt if item == PROMPT_SLOT else item for item in argv] env = dict(os.environ) env[CHILD_ENV] = "1" - with tempfile.TemporaryDirectory(prefix="llm-judge-") as cwd: + timeout = bounded_timeout(timeout_seconds) + with tempfile.TemporaryDirectory(prefix="llm-judge-") as temp_cwd: + runner_cwd = temp_cwd + if cwd is not None: + if isinstance(cwd, str) and os.path.isabs(cwd) and os.path.isdir(cwd): + runner_cwd = cwd + else: + log(f"runner {name}: refused cwd {cwd!r}") try: proc = subprocess.Popen( command, @@ -110,16 +129,16 @@ def run_runner(name: str, argv: list[str], prompt: str) -> tuple[dict, dict | No stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, - cwd=cwd, + cwd=runner_cwd, env=env, start_new_session=True, ) except OSError as exc: return failed(name, clip(type(exc).__name__, str(exc))), None try: - stdout, stderr = proc.communicate(timeout=TIMEOUT_SECONDS) + stdout, stderr = proc.communicate(timeout=timeout) except subprocess.TimeoutExpired: - return failed(name, clip(f"timed out after {TIMEOUT_SECONDS}s", stop_group(proc))), None + return failed(name, clip(f"timed out after {timeout}s", stop_group(proc))), None if proc.returncode != 0: return failed(name, clip(f"exit {proc.returncode}", stderr)), None answer = last_json_object(stdout) @@ -128,10 +147,12 @@ def run_runner(name: str, argv: list[str], prompt: str) -> tuple[dict, dict | No return {"runner": name, "ok": True, "reason": "answered"}, answer -def ask(prompt: str) -> dict: +def ask(prompt: str, mode: object = None, timeout_seconds: object = None, cwd: object = None) -> dict: + if timeout_seconds is None: + timeout_seconds = TIMEOUT_SECONDS attempts = [] - for name, argv in runners(): - attempt, answer = run_runner(name, argv, prompt) + for name, argv in runners(mode): + attempt, answer = run_runner(name, argv, prompt, timeout_seconds=timeout_seconds, cwd=cwd) attempts.append(attempt) if answer is not None: return {"outcome": "answered", "runner": name, "answer": answer, "attempts": attempts} @@ -216,7 +237,7 @@ def run_job(path: str) -> dict: raise ValueError(f"job file holds a JSON {type(loaded).__name__}, not an object") job = dict(loaded) job.setdefault("id", stem) - result = verdict(job, ask(str(job["prompt"]))) + result = verdict(job, ask(str(job["prompt"]), mode=job.get("mode"), timeout_seconds=job.get("timeout_seconds", TIMEOUT_SECONDS), cwd=job.get("cwd"))) except Exception as exc: print(f"catstack-hook-error llm-judge: {type(exc).__name__}: {exc}", file=sys.stderr) log(f"job {job.get('id')} failed: {type(exc).__name__}: {exc}\n{traceback.format_exc()}") diff --git a/engine/hooks/llm-judge/tests/test_judge.py b/engine/hooks/llm-judge/tests/test_judge.py index 08d61c0d..8465f0b2 100644 --- a/engine/hooks/llm-judge/tests/test_judge.py +++ b/engine/hooks/llm-judge/tests/test_judge.py @@ -124,6 +124,90 @@ def test_malformed_runners_env_refuses_instead_of_running_defaults(self): def test_default_runner_order_is_codex_then_claude_then_cursor(self): self.assertEqual([name for name, _ in judge.runners()], ["codex", "claude", "cursor"]) + def test_investigate_runner_argv_is_read_only_and_excludes_cursor(self): + self.assertEqual( + judge.runners("investigate"), + [ + ( + "codex", + [ + "codex", + "exec", + "--skip-git-repo-check", + "--sandbox", + "read-only", + "-c", + "notify=[]", + judge.PROMPT_SLOT, + ], + ), + ( + "claude", + [ + "claude", + "-p", + "--model", + "haiku", + "--settings", + '{"disableAllHooks": true}', + "--allowedTools", + "Read", + "Grep", + "Glob", + "--disallowedTools", + "Write", + "Edit", + "NotebookEdit", + "Bash", + "--", + judge.PROMPT_SLOT, + ], + ), + ], + ) + + def test_investigate_runners_env_replaces_investigate_defaults(self): + custom = ["probe", [PY, "-c", "print('{}')", "{prompt}"]] + self.use_runners(custom) + self.assertEqual(judge.runners("investigate"), [("probe", custom[1])]) + + def test_investigate_job_threads_timeout_and_cwd_to_runner(self): + path = os.path.join(self.state.name, "jobs", "investigate-job.json") + with tempfile.TemporaryDirectory() as cwd: + judge.write_json_atomic(path, self.job(id="investigate-job", mode="investigate", timeout_seconds=123, cwd=cwd)) + calls = [] + + def capture(name, argv, prompt, timeout_seconds=judge.TIMEOUT_SECONDS, cwd=None): + calls.append((name, timeout_seconds, cwd)) + return {"runner": name, "ok": True, "reason": "answered"}, {"match": True} + + with patch.object(judge, "run_runner", side_effect=capture): + result = judge.run_job(path) + + self.assertEqual(result["outcome"], "hit") + self.assertEqual(calls, [("codex", 123, cwd)]) + + def test_investigate_timeout_is_capped_at_600_seconds(self): + self.assertEqual(judge.bounded_timeout(999), 600) + + def test_non_investigate_job_still_gets_default_timeout_and_empty_temp_cwd(self): + self.use_runners(runner("env", "import json, os; print(json.dumps({'cwd': os.getcwd(), 'entries': os.listdir('.')}))")) + path = os.path.join(self.state.name, "jobs", "default-job.json") + judge.write_json_atomic(path, self.job(id="default-job")) + calls = [] + original = judge.run_runner + + def capture(name, argv, prompt, timeout_seconds=judge.TIMEOUT_SECONDS, cwd=None): + calls.append((timeout_seconds, cwd)) + return original(name, argv, prompt, timeout_seconds=timeout_seconds, cwd=cwd) + + with patch.object(judge, "run_runner", side_effect=capture): + result = judge.run_job(path) + + self.assertEqual(calls, [(judge.TIMEOUT_SECONDS, None)]) + self.assertEqual(result["answer"]["entries"], []) + self.assertFalse(os.path.exists(result["answer"]["cwd"])) + class TestVerdict(JudgeTestCase): def test_hit_when_every_hit_key_is_true(self): From 06053d2205c59605a67687e87b6831effbda81aa Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sun, 13 Sep 2026 06:04:58 +0000 Subject: [PATCH 08/12] =?UTF-8?q?invoker:=20wf-1789279240280-10/verify-jud?= =?UTF-8?q?ge-investigate-tests=20=E2=80=94=20Proof=20step=201=20for=20the?= =?UTF-8?q?=20llm-judge=20investigate=20mode.=20Review=20claim:=20The=20in?= =?UTF-8?q?vestigate-mode=20unit=20tests=20run=20and=20pass.=20Review=20la?= =?UTF-8?q?ne:=20proof=20Safety=20invariant:=20Proof-only;=20adds=20no=20p?= =?UTF-8?q?roduct=20behavior.=20Effectiveness=20measurement:=20The=20run?= =?UTF-8?q?=20exits=200=20and=20reports=20at=20least=20one=20test;=20Pytho?= =?UTF-8?q?n=20exits=205=20when=20-k=20matches=20nothing.=20Slice=20ration?= =?UTF-8?q?ale:=20One=20proof=20step=20for=20this=20workflow's=20single=20?= =?UTF-8?q?change.=20Architectural=20effect:=20None;=20verification=20only?= =?UTF-8?q?.=20Goal:=20Prove=20the=20investigate=20runner=20argv=20and=20t?= =?UTF-8?q?ime=20limit=20deterministically.=20Motivation:=20Each=20slice?= =?UTF-8?q?=20carries=20its=20own=20executable=20proof.=20Alternative=20co?= =?UTF-8?q?nsiderations:=20Manual=20inspection=20was=20rejected=20as=20non?= =?UTF-8?q?-deterministic.=20Implementation=20details:=20Run=20the=20filte?= =?UTF-8?q?red=20unittest=20discovery=20once.=20Non-goals:=20No=20product?= =?UTF-8?q?=20edits=20here;=20proof=20only.=20Layer:=20app=5Fregression=20?= =?UTF-8?q?Feature=20state:=20active?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: 964f7535-7395-4bb3-b657-37a77b357f44 From 361358e396a2c4b124360db295ad35e856b5ecb7 Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sun, 13 Sep 2026 06:05:27 +0000 Subject: [PATCH 09/12] =?UTF-8?q?invoker:=20wf-1789279240280-10/document-j?= =?UTF-8?q?udge-investigate-mode=20=E2=80=94=20Describe=20the=20llm-judge?= =?UTF-8?q?=20investigate=20mode=20in=20the=20llm-judge=20README.=20Review?= =?UTF-8?q?=20claim:=20engine/hooks/llm-judge/README.md=20gains=20an=20Inv?= =?UTF-8?q?estigate=20mode=20section=20that=20matches=20judge.py.=20Review?= =?UTF-8?q?=20lane:=20docs=20Safety=20invariant:=20Only=20engine/hooks/llm?= =?UTF-8?q?-judge/README.md=20changes;=20no=20code=20or=20test=20file=20is?= =?UTF-8?q?=20edited.=20Effectiveness=20measurement:=20grep=20finds=20the?= =?UTF-8?q?=20section=20heading=20and=20each=20named=20flag=20in=20the=20R?= =?UTF-8?q?EADME.=20Slice=20rationale:=20The=20README=20section=20for=20th?= =?UTF-8?q?e=20behavior=20in=20this=20same=20pull=20request,=20which=20cat?= =?UTF-8?q?stack's=20ship-a-detector=20playbook=20requires=20to=20ship=20t?= =?UTF-8?q?ogether.=20Architectural=20effect:=20None;=20README=20only.=20G?= =?UTF-8?q?oal:=20Let=20the=20next=20hook=20author=20find=20the=20investig?= =?UTF-8?q?ate=20mode=20and=20its=20limits.=20Motivation:=20catstack=20kee?= =?UTF-8?q?ps=20every=20hook's=20contract=20in=20its=20README.=20Alternati?= =?UTF-8?q?ve=20considerations:=20A=20later=20README-only=20pull=20request?= =?UTF-8?q?=20was=20rejected;=20catstack's=20playbook=20counts=20a=20READM?= =?UTF-8?q?E=20that=20lands=20later=20as=20a=20known=20repair=20pattern.?= =?UTF-8?q?=20Implementation=20details:=20Add=20one=20"##=20Investigate=20?= =?UTF-8?q?mode"=20section=20to=20engine/hooks/llm-judge/README.md=20after?= =?UTF-8?q?=20"##=20Runner=20order".=20Non-goals:=20No=20code=20or=20test?= =?UTF-8?q?=20edits.=20Layer:=20docs=20Feature=20state:=20active=20Files:?= =?UTF-8?q?=20-=20engine/hooks/llm-judge/README.md=20Change=20types:=20-?= =?UTF-8?q?=20engine/hooks/llm-judge/README.md:=20modify=20Acceptance=20cr?= =?UTF-8?q?iteria:=20-=20`grep=20-n=20"##=20Investigate=20mode"=20engine/h?= =?UTF-8?q?ooks/llm-judge/README.md`=20exits=200.=20-=20The=20section=20na?= =?UTF-8?q?mes=20--sandbox=20read-only,=20--allowedTools=20Read=20Grep=20G?= =?UTF-8?q?lob,=20--disallowedTools=20Write=20Edit=20NotebookEdit=20Bash,?= =?UTF-8?q?=20and=20600.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Context: wf-1789279240280-10/implement-judge-investigate-mode (f2435ab): Let an llm-judge job opt into a read-only investigate mode with its own time limit and working folder. Review claim: A job carrying "mode": "investigate" runs on a second runner set whose Claude argv grants only the Read, Grep and Glob tools and forbids Write, Edit, NotebookEdit and Bash, with the job's own timeout_seconds and cwd; a job lacking that key runs exactly as today. Review lane: behavior Safety invariant: An investigate job never receives a tool that can write, edit, or execute shell text. The Claude runner argv always carries --allowedTools Read Grep Glob and --disallowedTools Write Edit NotebookEdit Bash, the Codex runner always carries --sandbox read-only, and cursor-agent is never used for investigate jobs because it has no read-only switch. Effectiveness measurement: New unit tests pin the exact argv of each investigate runner and prove a default job still gets 60 seconds in an empty temp folder. The PR body pastes the live probe in which claude -p with this argv was asked to Write a file and to touch one through Bash, answered {"wrote":false,"read":true,"bash":false}, and neither target file existed afterwards. Slice rationale: One behavior slice inside the judge, dormant until a hook asks for an investigate job, so a reviewer can judge the permission argv on its own. Architectural effect: llm-judge gains a second, read-only runner set chosen per job; the default runner set and every current caller stay unchanged. Goal: Give later hooks a safe way to ask a model to check a claim by reading files on this machine. Motivation: A backtest of 44 unverified tags from the last 14 days found 33 whose stated blocker was false; most could have been checked by reading a log, a transcript, or a source file already on disk. Today each judge runner gets 60 seconds in an empty temp folder with no tool grants, so it cannot read anything. Alternative considerations: claude --permission-mode plan was rejected after a live probe wrote a plan file under ~/.claude/plans. A Bash prefix allowlist such as Bash(git log:*) was rejected after a live probe blocked even plain git log, so shell access stays out of this slice. Implementation details: In engine/hooks/llm-judge/judge.py add INVESTIGATE_RUNNERS beside DEFAULT_RUNNERS holding the codex and claude argv given under Runner argv. Make runners() take the job mode and pick INVESTIGATE_RUNNERS for "investigate"; CATSTACK_LLM_JUDGE_RUNNERS still replaces whichever set is picked. Make run_runner accept timeout_seconds (default TIMEOUT_SECONDS, capped at INVESTIGATE_TIMEOUT_CAP = 600) and cwd (used only when it is an absolute path to a folder that exists, else the temp folder plus a judge.log line). Thread mode, timeout_seconds and cwd from the job through ask() and run_job(). Runner argv: codex exec --skip-git-repo-check --sandbox read-only -c notify=[] PROMPT, and claude -p --model haiku --settings '{"disableAllHooks": true}' --allowedTools Read Grep Glob --disallowedTools Write Edit NotebookEdit Bash -- PROMPT. Non-goals: No hook asks for an investigate job in this slice. No inbox edit, no change to DEFAULT_RUNNERS, wrong-check-reflect or diu-stop. No shell or network access for the investigator. Layer: domain Feature state: dormant Files: - engine/hooks/llm-judge/judge.py - engine/hooks/llm-judge/tests/test_judge.py Change types: - engine/hooks/llm-judge/judge.py: modify - engine/hooks/llm-judge/tests/test_judge.py: modify Acceptance criteria: - Tests whose names contain "investigate" pass and pin both investigate runner argv exactly. - A test proves a job lacking a mode key still runs DEFAULT_RUNNERS with a 60 second limit. - A test proves timeout_seconds above 600 is capped at 600. - A test whose name contains "missing" proves a cwd that does not exist falls back to a temp folder and logs it. - python3 scripts/check_no_new_comments.py --base origin/main exits 0. Solution: Describe the llm-judge investigate mode in the llm-judge README. Review claim: engine/hooks/llm-judge/README.md gains an Investigate mode section that matches judge.py. Review lane: docs Safety invariant: Only engine/hooks/llm-judge/README.md changes; no code or test file is edited. Effectiveness measurement: grep finds the section heading and each named flag in the README. Slice rationale: The README section for the behavior in this same pull request, which catstack's ship-a-detector playbook requires to ship together. Architectural effect: None; README only. Goal: Let the next hook author find the investigate mode and its limits. Motivation: catstack keeps every hook's contract in its README. Alternative considerations: A later README-only pull request was rejected; catstack's playbook counts a README that lands later as a known repair pattern. Implementation details: Add one "## Investigate mode" section to engine/hooks/llm-judge/README.md after "## Runner order". Non-goals: No code or test edits. Layer: docs Feature state: active Files: - engine/hooks/llm-judge/README.md Change types: - engine/hooks/llm-judge/README.md: modify Acceptance criteria: - `grep -n "## Investigate mode" engine/hooks/llm-judge/README.md` exits 0. - The section names --sandbox read-only, --allowedTools Read Grep Glob, --disallowedTools Write Edit NotebookEdit Bash, and 600. Invoker-Finalize-Id: c7674a0c-7e5f-4540-b149-295bbac2133e --- engine/hooks/llm-judge/README.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/engine/hooks/llm-judge/README.md b/engine/hooks/llm-judge/README.md index fe5ee99b..31eba814 100644 --- a/engine/hooks/llm-judge/README.md +++ b/engine/hooks/llm-judge/README.md @@ -94,6 +94,25 @@ most 300 characters, taken from the end of stderr or the error text. Tests use it to plug in small fake runners. If it is set but not that shape, `ask` raises `ValueError` instead of quietly falling back to the real runners. +## Investigate mode + +A job opts in with `"mode": "investigate"`. It uses a read-only runner set: + +1. **codex**: `codex exec --skip-git-repo-check --sandbox read-only -c notify=[] PROMPT` +2. **claude**: `claude -p --model haiku --settings '{"disableAllHooks": true}' --allowedTools Read Grep Glob --disallowedTools Write Edit NotebookEdit Bash -- PROMPT` + +`cursor-agent` is not used because it has no read-only switch. + +An investigate job may carry `timeout_seconds`. The judge caps it at 600 +seconds. If it is missing or not a number, the runner gets 60 seconds. + +An investigate job may carry `cwd`. The judge uses it only when it is an +absolute path to a folder that exists. Otherwise the runner uses a fresh temp +folder, and `judge.log` gets a line saying the cwd was refused. + +`CATSTACK_LLM_JUDGE_RUNNERS` still replaces the selected runner set. A job that +gets no answer from any runner still comes back `unchecked`. + ## Three outcomes `verdict(job, result)` turns an `ask` result into one of: From 68035e29614ec95dd9816d1045a45b0a87e31e99 Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sun, 13 Sep 2026 06:05:38 +0000 Subject: [PATCH 10/12] =?UTF-8?q?invoker:=20wf-1789279240280-10/verify-jud?= =?UTF-8?q?ge-full-suite=20=E2=80=94=20Proof=20step=202=20for=20the=20llm-?= =?UTF-8?q?judge=20investigate=20mode.=20Review=20claim:=20Every=20llm-jud?= =?UTF-8?q?ge=20test=20still=20passes=20after=20the=20change.=20Review=20l?= =?UTF-8?q?ane:=20proof=20Safety=20invariant:=20Proof-only;=20adds=20no=20?= =?UTF-8?q?product=20behavior.=20Effectiveness=20measurement:=20The=20whol?= =?UTF-8?q?e=20llm-judge=20test=20folder=20exits=200.=20Slice=20rationale:?= =?UTF-8?q?=20One=20proof=20step=20guarding=20current=20judge=20callers.?= =?UTF-8?q?=20Architectural=20effect:=20None;=20verification=20only.=20Goa?= =?UTF-8?q?l:=20Prove=20default=20jobs=20behave=20as=20before.=20Motivatio?= =?UTF-8?q?n:=20wrong-check-reflect=20and=20diu-stop=20already=20depend=20?= =?UTF-8?q?on=20the=20judge.=20Alternative=20considerations:=20Running=20o?= =?UTF-8?q?nly=20the=20new=20tests=20was=20rejected;=20it=20would=20miss?= =?UTF-8?q?=20a=20regression=20in=20the=20default=20runner=20set.=20Implem?= =?UTF-8?q?entation=20details:=20Run=20the=20whole=20llm-judge=20test=20fo?= =?UTF-8?q?lder.=20Non-goals:=20No=20product=20edits=20here;=20proof=20onl?= =?UTF-8?q?y.=20Layer:=20app=5Fregression=20Feature=20state:=20active?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: c94db032-2d49-4594-b6e7-b3ac43e7b56b From 258b69e0e624433f98541b73039937e86d5c7599 Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sun, 13 Sep 2026 06:06:18 +0000 Subject: [PATCH 11/12] =?UTF-8?q?invoker:=20wf-1789279240280-10/verify-jud?= =?UTF-8?q?ge-no-new-comments=20=E2=80=94=20Proof=20step=203=20for=20the?= =?UTF-8?q?=20llm-judge=20investigate=20mode.=20Review=20claim:=20The=20ch?= =?UTF-8?q?ange=20adds=20no=20explanatory=20code=20comments.=20Review=20la?= =?UTF-8?q?ne:=20proof=20Safety=20invariant:=20Proof-only;=20adds=20no=20p?= =?UTF-8?q?roduct=20behavior.=20Effectiveness=20measurement:=20The=20comme?= =?UTF-8?q?nt=20gate=20exits=200=20against=20origin/main.=20Slice=20ration?= =?UTF-8?q?ale:=20One=20proof=20step=20for=20catstack's=20required=20comme?= =?UTF-8?q?nt=20gate.=20Architectural=20effect:=20None;=20verification=20o?= =?UTF-8?q?nly.=20Goal:=20Keep=20the=20CI=20comment=20gate=20green.=20Moti?= =?UTF-8?q?vation:=20CI=20runs=20this=20gate=20on=20every=20pull=20request?= =?UTF-8?q?.=20Alternative=20considerations:=20Waiting=20for=20CI=20was=20?= =?UTF-8?q?rejected;=20it=20is=20slower=20to=20find.=20Implementation=20de?= =?UTF-8?q?tails:=20Run=20the=20comment=20gate=20against=20origin/main.=20?= =?UTF-8?q?Non-goals:=20No=20product=20edits=20here;=20proof=20only.=20Lay?= =?UTF-8?q?er:=20app=5Fregression=20Feature=20state:=20active?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: 492bf3e7-0cff-46b3-b0d1-89687e47b6b7 From 94a1361c12138f32edd5867a664feb2726982f2c Mon Sep 17 00:00:00 2001 From: Invoker Bot Date: Sun, 13 Sep 2026 06:06:59 +0000 Subject: [PATCH 12/12] =?UTF-8?q?invoker:=20wf-1789279240280-10/scrub-hand?= =?UTF-8?q?off-artifacts=20=E2=80=94=20Check=20that=20no=20ephemeral=20int?= =?UTF-8?q?er-task=20handoff=20files=20remain=20before=20the=20merge=20gat?= =?UTF-8?q?e.=20Review=20claim:=20Only=20ephemeral=20handoff=20files=20are?= =?UTF-8?q?=20checked;=20no=20product=20or=20test=20file=20is=20touched.?= =?UTF-8?q?=20Review=20lane:=20cleanup=20Safety=20invariant:=20The=20scrub?= =?UTF-8?q?=20script=20runs=20read-only=20and=20never=20touches=20the=20ho?= =?UTF-8?q?me=20Invoker=20ledger.=20Effectiveness=20measurement:=20The=20s?= =?UTF-8?q?crub=20script=20exits=200=20and=20reports=20no=20handoff=20path?= =?UTF-8?q?.=20Slice=20rationale:=20One=20cleanup=20slice=20required=20on?= =?UTF-8?q?=20every=20implementation=20plan=20that=20opens=20a=20pull=20re?= =?UTF-8?q?quest,=20kept=20separate=20from=20behavior=20and=20proof=20work?= =?UTF-8?q?.=20Architectural=20effect:=20None.=20Goal:=20Leave=20a=20clean?= =?UTF-8?q?=20worktree=20for=20the=20pull=20request.=20Motivation:=20Ephem?= =?UTF-8?q?eral=20inter-task=20files=20must=20not=20leak=20into=20the=20re?= =?UTF-8?q?viewed=20diff.=20Alternative=20considerations:=20Leaving=20this?= =?UTF-8?q?=20step=20out=20was=20rejected;=20it=20is=20a=20hard=20requirem?= =?UTF-8?q?ent=20for=20every=20plan=20that=20opens=20a=20pull=20request.?= =?UTF-8?q?=20Implementation=20details:=20Run=20the=20repository's=20hando?= =?UTF-8?q?ff-scrub=20script=20without=20--apply.=20Non-goals:=20No=20prod?= =?UTF-8?q?uct=20edits=20in=20this=20task.=20Layer:=20docs=20Feature=20sta?= =?UTF-8?q?te:=20active?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Exit code: 0 Invoker-Finalize-Id: 2aafd01f-794a-46f1-8763-0a04ed358458