Skip to content

[CAT-UNVERIFIED tag](3) The engine rules tell authors to write the tag - #417

Merged
EdbertChan merged 1 commit into
mainfrom
stack/EdbertChan/cat-unverified-tag/3-engine-prose
Sep 12, 2026
Merged

[CAT-UNVERIFIED tag](3) The engine rules tell authors to write the tag#417
EdbertChan merged 1 commit into
mainfrom
stack/EdbertChan/cat-unverified-tag/3-engine-prose

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Summary

engine/CLAUDE.core.md now tells an agent to run the check first and use the tag only when the check cannot run.

It still said to write bare UNVERIFIED:, which the hooks now reject. An agent following it would be blocked for obeying it.

The rewrite also changes the order. The old rule made the marker the default answer to missing evidence. The new rule puts the check first.

Review Claim

Approve the engine rules naming {{CAT-UNVERIFIED: <claim> -- cannot verify: <reason>}} instead of bare UNVERIFIED:, plus a helper that finds lines still instructing the old marker.

Review Lane

behavior

Review Unit

engine-runtime

Safety Invariant

Prose plus one read-only scan helper; no hook, install or runtime path changes.

Slice Rationale

Rules come after the behavior they describe. Corpus skills, the docs files and the product skill are separate review units, so they are the next three PRs.

Non-goals

  • No hook or install change.
  • Lines where the word means something else are left alone: "UNVERIFIED schema" (Codex hooks.json) and "UNVERIFIED end-to-end" (a status cell). So are recorded transcripts and fixtures.

Test Plan

Test Plan
  • python3 engine/skills/make-pr/scripts/preflight.py --base <this PR's base>
unit    engine-runtime: 4 file(s)
neutral 1 file(s): tests/escape_hatch_vocab.py
declare Review Unit: engine-runtime
gate    python3 scripts/check_codify_has_code.py --base stack/EdbertChan/cat-unverified-tag/2-retry-checks-evidence
        ok      codify-has-code
gate    python3 scripts/check_no_dated_provenance.py --base stack/EdbertChan/cat-unverified-tag/2-retry-checks-evidence
        ok      no dated provenance
gate    python3 scripts/check_hook_test_coverage.py engine/hooks/wrong-check-reflect
        check_hook_test_coverage: OK (1 hook(s) checked)
gate    python3 scripts/check_skills_three_harnesses.py
        ok      skills three-harness check
gate    python3 scripts/check_ecosystem_boundaries.py
        ok      ecosystem boundaries
gate    python3 scripts/check_skill_file_refs.py
        ok      skill file refs
gate    python3 scripts/check_skill_test_coverage.py --base stack/EdbertChan/cat-unverified-tag/2-retry-checks-evidence --head HEAD
        ok      skill test coverage
gate    python3 scripts/check_skill_trigger_mechanism.py
        ok      skill trigger mechanism
gate    python3 scripts/check_skill_trigger_policy.py
        ok	skill trigger policy
gate    python3 scripts/check_subagent_scope_contract.py
        ok	subagent scope contract
gate    python3 scripts/run_skill_scenarios.py
        ok  	subagent-fact-attributed
        ok  	stale-green-caught-without-any-admission
        ok  	stale-green-already-owned-by-the-reply
        ok  	red-then-green-is-a-fix-not-a-stale-claim
        
        ok	all 19 scenario(s) behaved as declared
ok      preflight passed
  • bash scripts/run_all_tests.sh -- exit 0, 45 suites OK, 0 failed, 1676 tests
  • tests/escape_hatch_vocab.py flags 30 instructing lines on 509b9cd and 0 with the whole stack applied
  • python3 -m unittest discover -s engine/skills/make-pr/tests

Revert Plan

Revert Plan
  • Safe to revert? Yes
  • Revert command: git revert <sha>
  • Post-revert steps: None
  • Data migration? No

🤖 Generated with Claude Code

https://claude.ai/code/session_01VKsvxJk65w6q7KnPSRYvNg


Note

Low Risk
Documentation and a vocabulary-scan helper only; no hook, install, or runtime behavior changes in this slice.

Overview
Engine prose and the make-pr fixture-vs-live gate now match what evidence hooks enforce: run the check first, and only when verification truly cannot run, mark claims with {{CAT-UNVERIFIED: <claim> -- cannot verify: <reason>}}. Bare UNVERIFIED: is explicitly retired in engine/CLAUDE.core.md, evidence-check.mdc, and make-pr/SKILL.md so agents are not told to use a marker hooks reject.

A new read-only helper tests/escape_hatch_vocab.py scans instructional files for lines that still tell authors to write bare UNVERIFIED: (ignoring “retired” mentions and non-marker phrases like “UNVERIFIED schema”). make-pr adds unit tests that the skill names the CAT tag and passes that scan.

Reviewed by Cursor Bugbot for commit 6a541b6. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor

cursor Bot commented Sep 11, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_35c1ed16-a96c-49dd-86ba-039bac5e7c6a)

@EdbertChan

Copy link
Copy Markdown
Owner Author

Mergify repair stopped: lowest open stack PR #417 is based on stack/EdbertChan/cat-unverified-tag/2-retry-checks-evidence, which still belongs to open PR(s) #416 outside this stack; leaving it alone to avoid dropping dependency changes

@EdbertChan
EdbertChan force-pushed the stack/EdbertChan/cat-unverified-tag/2-retry-checks-evidence branch from ae767ea to e34b32f Compare September 12, 2026 06:18
@EdbertChan
EdbertChan changed the base branch from stack/EdbertChan/cat-unverified-tag/2-retry-checks-evidence to main September 12, 2026 06:19
The hooks clear on the tag and nothing else, but engine/CLAUDE.core.md
still said to write bare `UNVERIFIED:`. An agent following them was blocked for doing
what they said.

The core rule changes meaning, not only spelling. "If you have no
evidence, write UNVERIFIED:" made the marker the default answer to missing
evidence. It now says: run the check; if it genuinely cannot run, tag it
and name the blocker.

Also updated: the make-pr fixture-vs-live gate and the wrong-check-reflect
Cursor rule.

tests/escape_hatch_vocab.py finds any line that still instructs the
retired marker. On 509b9cd it flags 30 lines across the repo; with the
whole stack applied it flags 0. make-pr's new test_skill_text.py uses it.

Left alone on purpose, since the word means something else there:
"UNVERIFIED schema" (Codex hooks.json) and "UNVERIFIED end-to-end" (a
status cell), plus recorded transcripts and fixtures, which are history.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VKsvxJk65w6q7KnPSRYvNg
@mergify

mergify Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@EdbertChan
EdbertChan force-pushed the stack/EdbertChan/cat-unverified-tag/3-engine-prose branch from 4829e39 to 6a541b6 Compare September 12, 2026 06:19
@cursor

cursor Bot commented Sep 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_c274087d-4c83-421b-9da3-b0c06ad587df)

@EdbertChan
EdbertChan merged commit f06abf9 into main Sep 12, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants