Install check: assert every hook entrypoint is registered for its event - #490
Open
EdbertChan wants to merge 7 commits into
Open
Conversation
…aim: scripts/check_install_effective.py fails when a declared hook entrypoint is missing from settings.json for its own event, and treats a generated skill directory carrying .catstack-generated as installed rather than shadowed.
Review lane: policy
Safety invariant: Read-only gate. It writes nothing, installs nothing, and reports an unreadable or unparsable settings file as UNCHECKED rather than as a pass.
Effectiveness measurement: The next task's fixtures, run against this change, must pass, and the same fixtures must fail against the gate as it stands today.
Slice rationale: The gate itself, alone. Its fixtures are the next task because the atomicity lint keeps repo-root scripts and repo-root tests in separate tasks.
Architectural effect: The gate's input changes from one file name plus a substring search to the parsed contents of every claude*.hook.json compared against the parsed settings file.
Goal: Replace the folder-name substring test with a parsed event-and-command comparison, and exempt a directory that carries the installer's own marker.
Motivation: Eleven declared entrypoints were dead on a real machine while this gate passed, so a merged hook did nothing and the operator had no signal.
Alternative considerations: Asserting only that each event name appears was rejected: the event can be present for another hook while this hook's command is absent. Dropping the shadow check entirely was rejected: a hand-made directory that is not the installer's own output still hides the repo's copy.
Implementation details: In scripts/check_install_effective.py, rewrite check_hooks_registered (currently at lines 229-240) to glob every claude*.hook.json in each engine/hooks/* directory, read its "hooks" object, and for each event and each command entry assert that settings.json's parsed hooks hold an entry for that event whose command string matches; report each missing pair naming the hook, the file and the event. Keep the existing behavior for a missing settings file and add the same treatment for a settings file that does not parse. In the shadow check (currently lines 154-158), skip an installed path that is a directory containing a .catstack-generated file, because install.sh writes that marker itself.
Non-goals: No change to install.sh, to any hook, or to any other check in this file. No test edits here, no new gate, and no CI wiring.
Layer: domain
Feature state: active
Files:
- scripts/check_install_effective.py
Change types:
- scripts/check_install_effective.py: modify
Acceptance criteria:
- `python3 scripts/check_no_new_comments.py --base origin/main` exits 0.
- `python3 -c "import ast,sys; ast.parse(open('scripts/check_install_effective.py').read())"` exits 0.
Exit code: 0
Invoker-Finalize-Id: 7900229a-d0ad-4c69-b9fc-2ac738cba100
…: tests pin that the installer verifier names an unregistered event-and-command pair and stays silent on the installer's own generated directory. Review lane: proof Safety invariant: Test files only. No product or gate code changes in this task. Effectiveness measurement: The new cases pass on this branch and fail when the gate is reverted to its previous shape. Slice rationale: Fixtures for the gate changed in the previous task, kept separate because the atomicity lint classes repo-root tests as proof files. Architectural effect: None; coverage only. Goal: Pin both behaviors with fixtures. Motivation: The gate's untested branches are how eleven dead entrypoints passed unnoticed. Alternative considerations: Asserting only the happy path was rejected: the defect was a false pass, so the missing-pair case is the one that matters. Implementation details: Extend tests/test_install_effective.py with a temporary HOME and a settings fixture: one case omits a declared event and asserts the hook and event are named; one case has the event with a different command and asserts it is still reported; one case is fully registered and asserts silence; one case has a generated directory carrying .catstack-generated and asserts silence; one case has a real directory without the marker and asserts it is still reported. Follow the existing style in that file for pointing the module at a temporary HOME. Non-goals: No change to the gate, to install.sh, or to any other test file. Layer: app_regression Feature state: active Files: - tests/test_install_effective.py Change types: - tests/test_install_effective.py: modify Acceptance criteria: - `python3 -m unittest discover -s tests -p 'test_install_effective*' -v` exits 0 and runs the new cases. - `python3 scripts/check_no_new_comments.py --base origin/main` exits 0. Exit code: 0 Invoker-Finalize-Id: d9859789-e795-42e7-a22b-1d123a452446
…: the installer-verifier tests, including the new registration and generated-directory cases, pass. Review lane: proof Safety invariant: Verification is read-only and changes no file. Effectiveness measurement: The unittest run is the measurement. Slice rationale: One proof for one claim. Architectural effect: None; verification only. Goal: Prove the gate's tests pass. Motivation: A gate with untested branches is how this defect survived. Alternative considerations: The whole suite was rejected as slower without adding evidence for this claim. Implementation details: Run the installer-verifier tests. Non-goals: No edits. Layer: app_regression Feature state: active Acceptance criteria: - Exits 0 only when every test in that file passes. Exit code: 0 Invoker-Finalize-Id: f7f3f5df-dab1-4b09-bb8b-a1b062807446
…no ephemeral handoff files remain in the worktree. Review lane: cleanup Safety invariant: The scrub script only checks for known handoff artifact names and never touches source or tests. Effectiveness measurement: The script exits non-zero if any handoff artifact remains. Slice rationale: Required terminal scrub for every implementation workflow. Architectural effect: None; hygiene only. Goal: Leave the branch free of handoff artifacts. Motivation: Handoff files must not reach the PR. Alternative considerations: Manual cleanup was rejected as non-deterministic. Implementation details: Run scripts/scrub-handoff-artifacts.sh. Non-goals: No product edits. Layer: app_regression Feature state: active Acceptance criteria: - `bash scripts/scrub-handoff-artifacts.sh` exits 0. Exit code: 0 Invoker-Finalize-Id: ec522a8e-4769-4249-b0d0-d439b832abb6
…ac431566a-99d44a22 — Review claim: no ephemeral handoff files remain in the worktree. Review lane: cleanup Safety invariant: The scrub script only checks for known handoff artifact names and never touches source or tests. Effectiveness measurement: The script exits non-zero if any handoff artifact remains. Slice rationale: Required terminal scrub for every implementation workflow. Architectural effect: None; hygiene only. Goal: Leave the branch free of handoff artifacts. Motivation: Handoff files must not reach the PR. Alternative considerations: Manual cleanup was rejected as non-deterministic. Implementation details: Run scripts/scrub-handoff-artifacts.sh. Non-goals: No product edits. Layer: app_regression Feature state: active Acceptance criteria: - `bash scripts/scrub-handoff-artifacts.sh` exits 0.
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_6cf1de49-8f81-45ff-84ea-2dce1a02c9aa) |
Contributor
|
Tick the box to add this pull request to the merge queue (same as
|
Owner
Author
|
Mergify repair stopped: missing required check validate |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The install check tells people when their local agent setup does not match this project.
Some hook files were skipped. A folder name anywhere in the settings also counted as a match.
The check now reads all hook files. It matches each event and command with the local settings.
It also trusts skill folders made by the installer. It still reports hand-made folders that hide project skills.
Review Claim
Approve a check that finds each missing hook and accepts skill folders made by the installer.
Review Lane
behavior
Review Unit
engine-runtime
Safety Invariant
The verifier only reads repository declarations and local settings. It does not write files or install software.
Unreadable or invalid settings produce an unchecked result, not a pass.
Assumptions: The user has not confirmed this safety invariant in this session.
Slice Rationale
The verifier and its regression coverage form one review claim. The coverage proves the failure and success paths of the same read-only check.
Non-goals
Architecture
Before
graph TD A["One default hook declaration per folder"] --> B["Search the raw settings text for the folder name"] B --> C["Report missing folder names"] D["Installed skill directory"] --> E["Always report a shadow conflict"]After
graph TD A["Every hook declaration file"] --> B["Parse event and command pairs"] C["Local settings"] --> D["Parse registered event and command pairs"] B --> E["Compare exact pairs"] D --> E E --> F["Report each missing registration"] G["Installed skill directory"] --> H["Check the generated marker"] H --> I["Accept generated directories or report hand-made conflicts"]Test Plan
Test Plan
python3 engine/skills/make-pr/scripts/preflight.py --base origin/mainpython3 -m unittest discover -s tests -p 'test_install_effective*' -v: 22 tests passed.python3 scripts/check_no_new_comments.py --base origin/main: no new comments found.python3 -c "import ast; ast.parse(open('scripts/check_install_effective.py').read())": exited 0.bash scripts/scrub-handoff-artifacts.sh: reportedscrub-handoff-artifacts-ok.Revert Plan
Revert Plan
git revert <merge-commit-sha>Note
Low Risk
Changes only affect a read-only verification script and unit tests; no installer, hook runtime, or settings writes.
Overview
Tightens the read-only install verifier so hook drift is detected precisely and installer-created skill layouts are not false positives.
Hook registration no longer treats a hook folder name appearing anywhere in
~/.claude/settings.jsonas registered. It parses JSON from settings and from everyengine/hooks/*/claude*.hook.json, then flags each declared event + command that is missing from settings. Unreadable or invalid settings yield UNCHECKED notes instead of a silent pass.Skills: real directories under
~/.claude/skills/<name>are still drift unless they contain a.catstack-generatedmarker, in which case the checker skips them.Tests add helpers for synthetic hook declarations and settings, cover wrong event, wrong command, and correct registration, plus generated vs hand-made skill directories.
Reviewed by Cursor Bugbot for commit 1c518bb. Bugbot is set up for automated code reviews on this repo. Configure here.