Skip to content

Install check: assert every hook entrypoint is registered for its event - #490

Open
EdbertChan wants to merge 7 commits into
mainfrom
plan/install-check-assert-every-hook-entrypoint-is-registered-for-its-event
Open

Install check: assert every hook entrypoint is registered for its event#490
EdbertChan wants to merge 7 commits into
mainfrom
plan/install-check-assert-every-hook-entrypoint-is-registered-for-its-event

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Summary

The install check tells people when their local agent setup does not match this project.

Some hook files were skipped. A folder name anywhere in the settings also counted as a match.

The check now reads all hook files. It matches each event and command with the local settings.

It also trusts skill folders made by the installer. It still reports hand-made folders that hide project skills.

Review Claim

Approve a check that finds each missing hook and accepts skill folders made by the installer.

Review Lane

behavior

Review Unit

engine-runtime

Safety Invariant

The verifier only reads repository declarations and local settings. It does not write files or install software.

Unreadable or invalid settings produce an unchecked result, not a pass.

Assumptions: The user has not confirmed this safety invariant in this session.

Slice Rationale

The verifier and its regression coverage form one review claim. The coverage proves the failure and success paths of the same read-only check.

Non-goals

  • No installer behavior changes.
  • No hook implementation changes.
  • No changes to other installation checks or CI wiring.

Architecture

Before

graph TD
    A["One default hook declaration per folder"] --> B["Search the raw settings text for the folder name"]
    B --> C["Report missing folder names"]
    D["Installed skill directory"] --> E["Always report a shadow conflict"]
Loading

After

graph TD
    A["Every hook declaration file"] --> B["Parse event and command pairs"]
    C["Local settings"] --> D["Parse registered event and command pairs"]
    B --> E["Compare exact pairs"]
    D --> E
    E --> F["Report each missing registration"]
    G["Installed skill directory"] --> H["Check the generated marker"]
    H --> I["Accept generated directories or report hand-made conflicts"]
Loading

Test Plan

Test Plan
  • python3 engine/skills/make-pr/scripts/preflight.py --base origin/main

    unit    engine-runtime: 1 file(s)
    neutral 1 file(s): tests/test_install_effective.py
    declare Review Unit: engine-runtime
    gates   none required for these paths
    ok      preflight passed
    
  • python3 -m unittest discover -s tests -p 'test_install_effective*' -v: 22 tests passed.

  • python3 scripts/check_no_new_comments.py --base origin/main: no new comments found.

  • python3 -c "import ast; ast.parse(open('scripts/check_install_effective.py').read())": exited 0.

  • bash scripts/scrub-handoff-artifacts.sh: reported scrub-handoff-artifacts-ok.

Revert Plan

Revert Plan
  • Safe to revert? Yes. The change only affects a read-only verifier and its tests.
  • Revert command: git revert <merge-commit-sha>
  • Post-revert steps: None.
  • Data migration? No.

Note

Low Risk
Changes only affect a read-only verification script and unit tests; no installer, hook runtime, or settings writes.

Overview
Tightens the read-only install verifier so hook drift is detected precisely and installer-created skill layouts are not false positives.

Hook registration no longer treats a hook folder name appearing anywhere in ~/.claude/settings.json as registered. It parses JSON from settings and from every engine/hooks/*/claude*.hook.json, then flags each declared event + command that is missing from settings. Unreadable or invalid settings yield UNCHECKED notes instead of a silent pass.

Skills: real directories under ~/.claude/skills/<name> are still drift unless they contain a .catstack-generated marker, in which case the checker skips them.

Tests add helpers for synthetic hook declarations and settings, cover wrong event, wrong command, and correct registration, plus generated vs hand-made skill directories.

Reviewed by Cursor Bugbot for commit 1c518bb. Bugbot is set up for automated code reviews on this repo. Configure here.

CI Bot and others added 7 commits September 12, 2026 07:03
…aim: scripts/check_install_effective.py fails when a declared hook entrypoint is missing from settings.json for its own event, and treats a generated skill directory carrying .catstack-generated as installed rather than shadowed.

Review lane: policy
Safety invariant: Read-only gate. It writes nothing, installs nothing, and reports an unreadable or unparsable settings file as UNCHECKED rather than as a pass.
Effectiveness measurement: The next task's fixtures, run against this change, must pass, and the same fixtures must fail against the gate as it stands today.
Slice rationale: The gate itself, alone. Its fixtures are the next task because the atomicity lint keeps repo-root scripts and repo-root tests in separate tasks.
Architectural effect: The gate's input changes from one file name plus a substring search to the parsed contents of every claude*.hook.json compared against the parsed settings file.
Goal: Replace the folder-name substring test with a parsed event-and-command comparison, and exempt a directory that carries the installer's own marker.
Motivation: Eleven declared entrypoints were dead on a real machine while this gate passed, so a merged hook did nothing and the operator had no signal.
Alternative considerations: Asserting only that each event name appears was rejected: the event can be present for another hook while this hook's command is absent. Dropping the shadow check entirely was rejected: a hand-made directory that is not the installer's own output still hides the repo's copy.
Implementation details: In scripts/check_install_effective.py, rewrite check_hooks_registered (currently at lines 229-240) to glob every claude*.hook.json in each engine/hooks/* directory, read its "hooks" object, and for each event and each command entry assert that settings.json's parsed hooks hold an entry for that event whose command string matches; report each missing pair naming the hook, the file and the event. Keep the existing behavior for a missing settings file and add the same treatment for a settings file that does not parse. In the shadow check (currently lines 154-158), skip an installed path that is a directory containing a .catstack-generated file, because install.sh writes that marker itself.
Non-goals: No change to install.sh, to any hook, or to any other check in this file. No test edits here, no new gate, and no CI wiring.
Layer: domain
Feature state: active
Files:
- scripts/check_install_effective.py
Change types:
- scripts/check_install_effective.py: modify
Acceptance criteria:
- `python3 scripts/check_no_new_comments.py --base origin/main` exits 0.
- `python3 -c "import ast,sys; ast.parse(open('scripts/check_install_effective.py').read())"` exits 0.

Exit code: 0
Invoker-Finalize-Id: 7900229a-d0ad-4c69-b9fc-2ac738cba100
…: tests pin that the installer verifier names an unregistered event-and-command pair and stays silent on the installer's own generated directory.

Review lane: proof
Safety invariant: Test files only. No product or gate code changes in this task.
Effectiveness measurement: The new cases pass on this branch and fail when the gate is reverted to its previous shape.
Slice rationale: Fixtures for the gate changed in the previous task, kept separate because the atomicity lint classes repo-root tests as proof files.
Architectural effect: None; coverage only.
Goal: Pin both behaviors with fixtures.
Motivation: The gate's untested branches are how eleven dead entrypoints passed unnoticed.
Alternative considerations: Asserting only the happy path was rejected: the defect was a false pass, so the missing-pair case is the one that matters.
Implementation details: Extend tests/test_install_effective.py with a temporary HOME and a settings fixture: one case omits a declared event and asserts the hook and event are named; one case has the event with a different command and asserts it is still reported; one case is fully registered and asserts silence; one case has a generated directory carrying .catstack-generated and asserts silence; one case has a real directory without the marker and asserts it is still reported. Follow the existing style in that file for pointing the module at a temporary HOME.
Non-goals: No change to the gate, to install.sh, or to any other test file.
Layer: app_regression
Feature state: active
Files:
- tests/test_install_effective.py
Change types:
- tests/test_install_effective.py: modify
Acceptance criteria:
- `python3 -m unittest discover -s tests -p 'test_install_effective*' -v` exits 0 and runs the new cases.
- `python3 scripts/check_no_new_comments.py --base origin/main` exits 0.

Exit code: 0
Invoker-Finalize-Id: d9859789-e795-42e7-a22b-1d123a452446
…: the installer-verifier tests, including the new registration and generated-directory cases, pass.

Review lane: proof
Safety invariant: Verification is read-only and changes no file.
Effectiveness measurement: The unittest run is the measurement.
Slice rationale: One proof for one claim.
Architectural effect: None; verification only.
Goal: Prove the gate's tests pass.
Motivation: A gate with untested branches is how this defect survived.
Alternative considerations: The whole suite was rejected as slower without adding evidence for this claim.
Implementation details: Run the installer-verifier tests.
Non-goals: No edits.
Layer: app_regression
Feature state: active
Acceptance criteria:
- Exits 0 only when every test in that file passes.

Exit code: 0
Invoker-Finalize-Id: f7f3f5df-dab1-4b09-bb8b-a1b062807446
…no ephemeral handoff files remain in the worktree.

Review lane: cleanup
Safety invariant: The scrub script only checks for known handoff artifact names and never touches source or tests.
Effectiveness measurement: The script exits non-zero if any handoff artifact remains.
Slice rationale: Required terminal scrub for every implementation workflow.
Architectural effect: None; hygiene only.
Goal: Leave the branch free of handoff artifacts.
Motivation: Handoff files must not reach the PR.
Alternative considerations: Manual cleanup was rejected as non-deterministic.
Implementation details: Run scripts/scrub-handoff-artifacts.sh.
Non-goals: No product edits.
Layer: app_regression
Feature state: active
Acceptance criteria:
- `bash scripts/scrub-handoff-artifacts.sh` exits 0.

Exit code: 0
Invoker-Finalize-Id: ec522a8e-4769-4249-b0d0-d439b832abb6
…ac431566a-99d44a22 — Review claim: no ephemeral handoff files remain in the worktree.

Review lane: cleanup
Safety invariant: The scrub script only checks for known handoff artifact names and never touches source or tests.
Effectiveness measurement: The script exits non-zero if any handoff artifact remains.
Slice rationale: Required terminal scrub for every implementation workflow.
Architectural effect: None; hygiene only.
Goal: Leave the branch free of handoff artifacts.
Motivation: Handoff files must not reach the PR.
Alternative considerations: Manual cleanup was rejected as non-deterministic.
Implementation details: Run scripts/scrub-handoff-artifacts.sh.
Non-goals: No product edits.
Layer: app_regression
Feature state: active
Acceptance criteria:
- `bash scripts/scrub-handoff-artifacts.sh` exits 0.
@cursor

cursor Bot commented Sep 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_6cf1de49-8f81-45ff-84ea-2dce1a02c9aa)

@mergify

mergify Bot commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@EdbertChan

Copy link
Copy Markdown
Owner Author

Mergify repair stopped: missing required check validate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant