Found while verifying an unrelated PR (#17435) on macOS: two --self-test runs fail on a
developer machine and are green on CI. Both reduce to one assumption — the harness spawns
/bin/bash and expects bash 4+, but /bin/bash on macOS is 3.2.57, which is the very
floor this repo defends.
Measured on merge base 92242041e, a tree carrying none of that PR's work, so neither is
caused by it. Both reproduce there identically.
Instance 1 — check-step-collectors --self-test: 30 failures, all got 127
The harness reproduces GitHub's run: invocation faithfully (spawnSync('bash', ['-e', script]),
scripts/check-step-collectors.mjs:515), which is exactly the point of it. But the workflow
blocks it replays use mapfile, a bash 4 builtin:
mapfile -t selftests < <(find .claude/hooks -type f -name '*.selftest.sh' | sort)
On macOS that line exits 127 before the block does anything, so every assertion about the
collector's behaviour fails — including the ablation legs, which then "cannot reproduce the
defect" and say so. The step under test is fine; the harness cannot run it here.
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin25)
$ /bin/bash -c 'type mapfile'
/bin/bash: line 0: type: mapfile: not found
Instance 2 — check-bash32-floor --self-test: 8 of 153 failures
The one that matters, because of what it is defending. The failing case is:
the simulated-3.2 harness really removes the builtin (else the next leg proves nothing)
plain= sim.out= sim.err=...: line 1: mapfile: command not found
The harness simulates a 3.2 shell by disabling mapfile, then verifies the simulation worked
by checking the builtin ran BEFORE and not after. On a real 3.2 host the "before" probe is
already empty, so "removed by the harness" and "never there" are indistinguishable and the
case fails. The gate that defends the bash 3.2 floor cannot self-test on bash 3.2.
Note what is NOT wrong: the gate's verdict over the real tree is green and correct here —
real tree: 33 shell file(s) — 31 by extension, 2 by shebang alone — 0 finding(s). Only the
self-test's harness-verification leg fails. This is not #17141, which rules on the gate's
BOUNDARY (userland-binary differences being a different axis) and leaves the harness alone.
Why this is worth a card rather than a shrug
- Both gates are in the derived family set for ordinary changes, so every agent and every
human running the local gate sweep on a Mac meets two reds that take real time to
characterize as environmental. I spent that time this round; the next one will too.
- A self-test that cannot pass on a platform is a self-test whose verdict nobody reads on
that platform, which is how a genuinely red one gets waved through.
- For instance 2 specifically, the simulation leg exists precisely so the following legs mean
something. On the floor platform it is the leg that fails.
Not proposing the fix
Deliberately unassigned and ungraded. Sketches only, for triage to weigh: instance 1 could
resolve the interpreter once and skip with a stated reason when it is not 4+ (a skip that is
loud, not silent, or #4690 applies); instance 2 needs the harness to detect that the builtin
is natively absent and take that as the simulation rather than failing to create it. Both
touch the "an empty population is a refusal, not a quiet pass" rule, so neither should be
written as a quiet skip.
Found while verifying an unrelated PR (#17435) on macOS: two
--self-testruns fail on adeveloper machine and are green on CI. Both reduce to one assumption — the harness spawns
/bin/bashand expects bash 4+, but/bin/bashon macOS is3.2.57, which is the veryfloor this repo defends.
Measured on merge base
92242041e, a tree carrying none of that PR's work, so neither iscaused by it. Both reproduce there identically.
Instance 1 —
check-step-collectors --self-test: 30 failures, allgot 127The harness reproduces GitHub's
run:invocation faithfully (spawnSync('bash', ['-e', script]),scripts/check-step-collectors.mjs:515), which is exactly the point of it. But the workflowblocks it replays use
mapfile, a bash 4 builtin:On macOS that line exits 127 before the block does anything, so every assertion about the
collector's behaviour fails — including the ablation legs, which then "cannot reproduce the
defect" and say so. The step under test is fine; the harness cannot run it here.
Instance 2 —
check-bash32-floor --self-test: 8 of 153 failuresThe one that matters, because of what it is defending. The failing case is:
The harness simulates a 3.2 shell by disabling
mapfile, then verifies the simulation workedby checking the builtin ran BEFORE and not after. On a real 3.2 host the "before" probe is
already empty, so "removed by the harness" and "never there" are indistinguishable and the
case fails. The gate that defends the bash 3.2 floor cannot self-test on bash 3.2.
Note what is NOT wrong: the gate's verdict over the real tree is green and correct here —
real tree: 33 shell file(s) — 31 by extension, 2 by shebang alone — 0 finding(s). Only theself-test's harness-verification leg fails. This is not #17141, which rules on the gate's
BOUNDARY (userland-binary differences being a different axis) and leaves the harness alone.
Why this is worth a card rather than a shrug
human running the local gate sweep on a Mac meets two reds that take real time to
characterize as environmental. I spent that time this round; the next one will too.
that platform, which is how a genuinely red one gets waved through.
something. On the floor platform it is the leg that fails.
Not proposing the fix
Deliberately unassigned and ungraded. Sketches only, for triage to weigh: instance 1 could
resolve the interpreter once and skip with a stated reason when it is not 4+ (a skip that is
loud, not silent, or #4690 applies); instance 2 needs the harness to detect that the builtin
is natively absent and take that as the simulation rather than failing to create it. Both
touch the "an empty population is a refusal, not a quiet pass" rule, so neither should be
written as a quiet skip.