Teach two hooks the checks they could not see - #486
Open
EdbertChan wants to merge 1 commit into
Open
Conversation
Three findings from one reflect pass over a session that built a sandboxed install e2e for Invoker. Each is applied with a fail-before/pass-after test. narrow-the-scope fired four times in that session and was wrong all four. Its VERIFY_RE knew pytest, npm test, tsc and friends, but nothing about `bash scripts/test-foo.sh` or `docker build` -- so a shell-and-container project looks exactly like an edit streak with no verification in it. A detector that cries wolf trains the reader to skip it, so this adds those patterns and one more reset rule: a Bash command naming the basename of a file in the current streak clears that file, because running the script you just edited is how this kind of work gets checked and no regex can enumerate every project's entry point. tests/fixtures/shell_verified_streak_2026-09-11.json is the verbatim sequence, silent with the verification row and firing without it. hook-freshness watched the checkout behind ~/.claude/hooks for staleness but never checked that the registered hook scripts still resolve. In that same session ~/.claude/hooks/split-scope pointed into a deleted worktree: the hook was registered, could not run, and said nothing for the whole session, which is the "a check that could not run is not a pass" rule breaking on the gates themselves. The sweep reads settings.json, expands the paths and names every one that does not resolve. It returns three outcomes, not two -- an unreadable or unparseable settings.json reports unchecked rather than clean, which the first draft of this change got wrong until explicit-failures caught it. install.sh now warns when it is linking out of a git worktree instead of silently producing links that die with it. It deliberately does not redirect to the main checkout: tests/test_install.py:290 specifies that install.sh links against its own checkout, because installing from a worktree is how a branch gets tested. An earlier draft of this commit did redirect, and that test caught it. hook-freshness's sweep is what catches the link after the worktree is gone. The prove-it principle gains the rule the session actually broke: the pasted output has to entail the sentence, not merely agree with it. A run on one image proved that image lacked corepack; it was written up as corepack being removed from Node 25+, which no command in the session ever checked. Filing a narrow result under a wide heading is the error even when every word of the output is true, and it binds hardest in a correction, where restating the original overclaim relabels the mistake as a fix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ZqCxSQVpmM9Rx37C7irgf Change-Id: I7474c0682a2b17524b99f8f0e3f46782ae56e547
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_404ad159-b2d6-4f2c-819a-d40e2e9c4f37) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three findings from one reflect pass over a session that built a sandboxed
install e2e for Invoker. Each is applied with a fail-before/pass-after test.
narrow-the-scope fired four times in that session and was wrong all four. Its
VERIFY_RE knew pytest, npm test, tsc and friends, but nothing about
bash scripts/test-foo.shordocker build-- so a shell-and-containerproject looks exactly like an edit streak with no verification in it. A
detector that cries wolf trains the reader to skip it, so this adds those
patterns and one more reset rule: a Bash command naming the basename of a file
in the current streak clears that file, because running the script you just
edited is how this kind of work gets checked and no regex can enumerate every
project's entry point. tests/fixtures/shell_verified_streak_2026-09-11.json is
the verbatim sequence, silent with the verification row and firing without it.
hook-freshness watched the checkout behind ~/.claude/hooks for staleness but
never checked that the registered hook scripts still resolve. In that same
session ~/.claude/hooks/split-scope pointed into a deleted worktree: the hook
was registered, could not run, and said nothing for the whole session, which is
the "a check that could not run is not a pass" rule breaking on the gates
themselves. The sweep reads settings.json, expands the paths and names every
one that does not resolve. It returns three outcomes, not two -- an unreadable
or unparseable settings.json reports unchecked rather than clean, which the
first draft of this change got wrong until explicit-failures caught it.
install.sh now warns when it is linking out of a git worktree instead of
silently producing links that die with it. It deliberately does not redirect to
the main checkout: tests/test_install.py:290 specifies that install.sh links
against its own checkout, because installing from a worktree is how a branch
gets tested. An earlier draft of this commit did redirect, and that test caught
it. hook-freshness's sweep is what catches the link after the worktree is gone.
The prove-it principle gains the rule the session actually broke: the pasted
output has to entail the sentence, not merely agree with it. A run on one
image proved that image lacked corepack; it was written up as corepack being
removed from Node 25+, which no command in the session ever checked. Filing a
narrow result under a wide heading is the error even when every word of the
output is true, and it binds hardest in a correction, where restating the
original overclaim relabels the mistake as a fix.
Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_011ZqCxSQVpmM9Rx37C7irgf
Note
Low Risk
Changes are advisory-only hook behavior, install warnings, and documentation; no auth or production runtime paths. Slight risk of false negatives on basename-based streak reset if unrelated commands mention the same filename.
Overview
Addresses three session failures: hooks that missed real verification, hooks that silently failed when paths broke, and over-broad “verified” claims.
narrow-the-scope now treats shell/container work as verification:
VERIFY_REcoversbash …test*.sh,docker build/compose, andnode --check; a Bash command that mentions the basename of a file in the current edit streak clears only that file’s count (so running the script you edited counts without resetting unrelated streaks). Fixture-backed tests pin the prior four false positives.hook-freshness adds a sweep of
~/.claude/settings.jsonhook commands for script paths that do not exist; unreadable/missing settings are reported as unchecked, not healthy. That advisory can combine with the existing checkout-staleness line in one session message.install.sh prints a loud warning when run from a git worktree (links die with the worktree) without redirecting install to main checkout.
principle-prove-it adds a gate rule: pasted output must entail the claim (narrow witness ≠ wide generalization), especially in corrections.
Reviewed by Cursor Bugbot for commit 332b5e2. Bugbot is set up for automated code reviews on this repo. Configure here.