Skip to content

check-tenant-audit-census --self-test exits 1 on main: the page states 298 declared objects, the live census counts 300, and the self-test needs the exact string the main gate deliberately ignores #17437

Description

@claude

The failing probe

node scripts/check-tenant-audit-census.mjs --self-test

exits 1 on origin/main, with:

  ✗ an unenforced prose claim reworded off the page IS a finding
✗ check-tenant-audit-census self-test: 1 of 19 case(s) failed.

Its PR-verdict sibling — node scripts/check-tenant-audit-census.mjs, with no --self-test — exits 0. So the gate reports green on every PR while its own health check is red, and the workflow runs both (pnpm check:tenant-audit-census chains them), which means this is CI-visible and not merely latent.

What is actually wrong

The failing case is at scripts/check-tenant-audit-census.mjs:845:

t('an unenforced prose claim reworded off the page IS a finding',
  check(page.replace(`Across ${census.declaredObjects} declared objects`, 'Across the declared objects'))
    .some((p) => p.startsWith('[unenforced-prose-missing]')));

census is the LIVE census (runCensus()), page is the real
content/docs/permissions/tenant-audit-census.mdx. Measured on origin/main:

reading value
live declaredObjects (runCensus()) 300
the number written on the page (line 87, and again in the counts table at line 233) 298

The two disagree, so the .replace() finds no such substring, is a no-op, the unchanged page produces no [unenforced-prose-missing] finding, and the case fails.

Why the main gate stays green while this happens

Deliberately — the sibling case two lines up records it:

// The unenforced PROSE figure: value free, sentence required.
t('⭐ a stale unenforced prose number is NOT a finding', )

The gate checks that the SENTENCE is present, never that the NUMBER is current. That is a reasonable rule for a dated, explicitly unenforced figure. The problem is that the self-test then reaches for the exact string built from the LIVE count, so the same drift the main gate tolerates makes the self-test fail. The two halves disagree about whether the number matters.

⇒ Whichever way it is settled, it should be settled once: either the self-test builds its fixture from the page's own number (so it stays value-free like the gate), or the page's number is re-measured and the drift becomes a real finding. Both readings are defensible and this card does not pick one.

Provenance — not caused by the branch that found it

Found while running the derived gate list for #15972 (PR #17436). That diff adds zero object declarations:

git diff origin/main...HEAD -- '*.ts' | grep -c '^+.*ObjectSchema.create'   →  0

so it cannot move declaredObjects. The drift of 2 predates the branch. It is filed rather than fixed because the fix is a choice between two contracts (see above), and because the page is documentation owned by another lane.

The mechanism, measured (added after PR #17436 root-caused it)

The drift is not mysterious and the trigger is cheap to hit. declaredObjects() in scripts/tenant-audit-census.mjs walks every *.object.ts and counts each object literal carrying a snake_case name: string literal — it does not distinguish an object declaration from a nested one:

if (nm && /^[a-z][a-z0-9_]*$/.test(nm) && !objects.has(nm)) objects.set(nm, { file: rel, tenancyDisabled: disabled });

So the four actions[] names on sys_position (activate_position, deactivate_position, set_default_position, clone_position) are already in the tally. ⇒ the figure was never a count of objects, and it moves whenever anyone adds a nested snake_case name: to any *.object.ts — an action, a list view, an action param, or (as in #17436) a validations[] rule, whose name packages/spec requires to be snake_case.

Measured in ONE worktree with ONE node_modules, switching only HEAD: at origin/main (ab56ea3a1) the census reports 298 and the self-test exits 0; with two validation rules added it reports 300 and the self-test exits 1.

⇒ The two halves are worth separating:

  1. The counter over-matches. Whether declaredObjects should count only the top-level object literal is a real question, and answering it would change the published figure again (probably downward).
  2. The self-test is coupled to a figure its own gate declares unenforced. This is the half that turns a tolerated drift into a red. The gate is value-free by design («a stale unenforced prose number is NOT a finding»), but the self-test builds its .replace() argument from the LIVE count, so the mutation silently becomes a no-op the moment the page and the tree disagree — and the case then fails for the page rather than for the classifier it exists to pin. Either half could be fixed independently.

⛔ Regenerating the page (what #17436 did, under node scripts/tenant-audit-census.mjs --write) clears the symptom for one commit. It does not close this card: the next nested snake_case name: re-breaks it.

A second trap in the same family: the gate has no pnpm spelling

pnpm check:tenant-audit-census answers ERR_PNPM_RECURSIVE_EXEC_FIRST_FAIL / exit 254 — there is no such script in the root package.json. The workflow invokes the gate directly (.github/workflows/lint.yml: node scripts/check-tenant-audit-census.mjs --self-test). Every sibling gate in that job has a pnpm check:* alias, so the obvious spelling is the wrong one here, and a 254 reads like a failure to anyone who does not open the log. ⛔ It is script-not-found, never a gate verdict — the same NOT-MEASURED class as this gate's own exit 3.

Repro

pnpm install
node scripts/check-tenant-audit-census.mjs           # exit 0
node scripts/check-tenant-audit-census.mjs --self-test  # exit 1

Capture the exit code before any pipe — | head closes the read end early and rewrites the producer's status.


Generated by Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions