Skip to content

audit(token_audit): count naming the shortcut, and typing the fix command - #385

Merged
mergify[bot] merged 1 commit into
mainfrom
stack/EdbertChan/reflect/ui-input-guard-hook-freshness-20260908/count-naming-shortcut-typing-fix-cmd--fbb2b979
Sep 12, 2026
Merged

audit(token_audit): count naming the shortcut, and typing the fix command#385
mergify[bot] merged 1 commit into
mainfrom
stack/EdbertChan/reflect/ui-input-guard-hook-freshness-20260908/count-naming-shortcut-typing-fix-cmd--fbb2b979

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Sep 10, 2026

Copy link
Copy Markdown
Owner

Summary

The same-type intervention detector matched only told-you, accusation, agent-blame, restated-ask, and proof-challenge.

So a session where the user twice named the shortcut the agent had taken, and typed the fix-the-process command, scored zero and reported that no automate-me was owed. The audit said no while the user was asking for exactly that.

Two kinds join the set. One catches naming the shortcut ("cheap way out", "why would you", "straight up", "that's weird"), the other catches a typed /automate-me, /reflect, or /thrash.

Reaching for the fix-the-process command is itself the intervention.

Before and After

A two-message session where the user says "cheap way out", then "why would you ... straight up ... thats wierd", and types /reflect, run through python3 engine/skills/reflect/scripts/token_audit.py codex session.jsonl on the base (54530d6) and on this PR (e8454f8):

Before:
  [1] 2026-09-09T05:46:08.000Z ['multi-question-marks']: 'why would you submit a rule that would ignore files straight up??? thats wierd'
frustration-flagged user messages: 1/2; interruptions: 0; peak window 2026-09-09T05:46:08.000Z -> 2026-09-09T05:46:08.000Z
intervention-must-automate: no (count=0) no repeated intervention class (one correction is not automate-me)

After:
  [0] 2026-09-09T04:40:47.000Z ['cheap-way-out', 'explicit-invocation']: 'since you took the cheap way out and needed a stop hook, add that hook and /reflect'
  [1] 2026-09-09T05:46:08.000Z ['cheap-way-out', 'multi-question-marks']: 'why would you submit a rule that would ignore files straight up??? thats wierd'
frustration-flagged user messages: 2/2; interruptions: 0; peak window 2026-09-09T04:40:47.000Z -> 2026-09-09T05:46:08.000Z
intervention-must-automate: yes (count=3) same-type complaint / iteration: cheap-way-outx2, cheap-way-out+explicit-invocation

Review Claim

Naming the shortcut twice, or typing the fix-the-process command, counts toward the same-type intervention that makes automate-me mandatory.

Review Lane

behavior

Review Unit

tooling-policy

Safety Invariant

Two entries are appended to the pattern list and two names to the kind set; nothing existing is edited, so every previously flagged session flags the same way. Product blame and plain questions still score zero, asserted by a negative fixture alongside the positive one.

Slice Rationale

The detector change is separable from the skill prose in the slice below it, and either can be reverted alone.

Non-goals

Does not change what the flag causes to happen, only when it fires. Does not touch the frustration score itself or the cost audit.

Test Plan

Test Plan
  • python3 -m unittest discover -s engine/skills/reflect/scripts/tests — 172 tests, OK (2 new: the real shape fires, product blame and a plain question stay silent)
  • Re-audited the session that motivated this: intervention-must-automate: yes (count=3) where it previously read no (count=0)
  • python3 scripts/check_no_new_comments.py --base origin/main — ok

Revert Plan

Revert Plan
  • Safe to revert? Yes
  • Revert command: git revert <sha>
  • Post-revert steps: None
  • Data migration? No

🤖 Generated with Claude Code

https://claude.ai/code/session_01KU2pPKob4MJ1NqjsfTNyYJ


Note

Medium Risk
Changes live PreToolUse/Stop routing for subagent publishing and how CAT-UNVERIFIED tags behave; mis-tuned command parsing or liveness caching could block or allow the wrong path, but tests and fail-open/UNCHECKED branches are documented.

Overview
Replaces agent-routing-guard (PreToolUse on Agent prompt regex) with publish-act-guard, which blocks actual publishing shell commands (git push, gh pr create|merge|ready, mutating gh api on pulls, mergify stack push, create-pr.mjs, safe-stack-push.mjs --execute) only when the caller is a subagent and invoker-cli query capacity says a live owner is reachable. Liveness failures are UNCHECKED and allow the command with a named reason; install merges the new hook and strips the retired Agent hook entry.

Adds unverified-tag-ledger: Stop records well-formed {{CAT-UNVERIFIED: …}} tags, refuses a turn that tags without running a verification tool (unless stop_hook_active), and UserPromptSubmit reminds on outstanding claims; it opts out of SubagentStop inheritance. pr-schema-gate no longer treats validator exit 0 as clean when output says UNCHECKED/SKIPPED/not installed.

Policy and reflect: cat-mode and named-constraints add hook/classifier block is a stop, not a puzzle; CLAUDE.core.md tightens evidence for cache/offline reads and stale readings after restarts. reflect step 5 requires fanout_complete.py (complete / incomplete / unchecked exit codes) before applying Accepted findings. token_audit counts cheap way out / why would you and typed /reflect|automate-me|thrash toward intervention-must-automate.

Reviewed by Cursor Bugbot for commit 0d53f86. Bugbot is set up for automated code reviews on this repo. Configure here.

@EdbertChan

Copy link
Copy Markdown
Owner Author

This pull request is part of a Mergify stack:

# Pull Request Link
1 cat-mode: rejoin wrapped prose so the file is back under its own cap #382
2 hook(diu-stop): scope the unverified marker to its own paragraph #383
3 skill(reflect): do not apply findings from a fan-out that never returned #384
4 audit(token_audit): count naming the shortcut, and typing the fix command #385 👈

@cursor

cursor Bot commented Sep 10, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_e37bedbb-23a6-4755-98f3-4a91bd38efcd)

@EdbertChan
EdbertChan force-pushed the stack/EdbertChan/reflect/ui-input-guard-hook-freshness-20260908/apply-findings-fan-never-returned--8cc26dd3 branch from 54530d6 to 41708da Compare September 12, 2026 06:37
@EdbertChan
EdbertChan force-pushed the stack/EdbertChan/reflect/ui-input-guard-hook-freshness-20260908/count-naming-shortcut-typing-fix-cmd--fbb2b979 branch from e8454f8 to 9e1b549 Compare September 12, 2026 07:11
@cursor

cursor Bot commented Sep 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_ce859959-1e3a-4f4c-9afc-5742617a75de)

…mand

The same-type intervention detector matched only told-you, accusation, and
agent-blame, so a session where the user twice named the shortcut the agent
took, and typed /automate-me and /reflect, scored zero and reported that no
automate-me was owed.

Two kinds join the set: cheap-way-out ("cheap way out", "why would you",
"straight up", "that's weird") and explicit-invocation (a typed
/automate-me, /reflect, or /thrash). Reaching for the fix-the-process
command is itself the intervention. Product blame and plain questions still
score zero, with fixtures both ways.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KU2pPKob4MJ1NqjsfTNyYJ
Change-Id: Ifbb2b9797be1017833d9f5e2a45b3bd06ffdad61
@EdbertChan
EdbertChan force-pushed the stack/EdbertChan/reflect/ui-input-guard-hook-freshness-20260908/count-naming-shortcut-typing-fix-cmd--fbb2b979 branch from 9e1b549 to 0d53f86 Compare September 12, 2026 18:08
@cursor

cursor Bot commented Sep 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_d7e45ac9-b31a-4fea-932a-5a7336830c0d)

@EdbertChan
EdbertChan changed the base branch from stack/EdbertChan/reflect/ui-input-guard-hook-freshness-20260908/apply-findings-fan-never-returned--8cc26dd3 to main September 12, 2026 18:08
@mergify

mergify Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Queued — the merge queue status continues in this comment ↓.

@EdbertChan

Copy link
Copy Markdown
Owner Author

@Mergifyio queue

@mergify

mergify Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

This pull request spent 5 minutes 16 seconds in the queue, including 4 minutes 28 seconds running CI.

Required conditions to merge
  • check-success = lint
  • check-success = test

@mergify mergify Bot added the queued label Sep 12, 2026
@mergify
mergify Bot merged commit 814e208 into main Sep 12, 2026
4 checks passed
@mergify
mergify Bot deleted the stack/EdbertChan/reflect/ui-input-guard-hook-freshness-20260908/count-naming-shortcut-typing-fix-cmd--fbb2b979 branch September 12, 2026 18:22
@mergify mergify Bot removed the queued label Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant