ci(coverage): give the instrumented lane its own per-test budget so the coverage gate can run - #9474
Merged
Merged
Conversation
`vitest.config.mts` sets `testTimeout: 15000`, calibrated for the uninstrumented lane. The coverage lane runs the same files under v8 instrumentation, where the repo-sweeping rows under `scripts/__tests__/` take several times longer. Measured on this tree with the coverage flag set as the only variable: zero of the 146 rows in the two heaviest sweep files exceed 15s uninstrumented, five exceed it instrumented. A timed-out row is not a failed gate, it is no gate. `coverage-report` gates every merge step on `needs.test-coverage.result == 'success'`, so one dead row skips the merge and the thresholds are never evaluated. The uninstrumented lane keeps its 15s budget unchanged, so nothing that guards day-to-day work is relaxed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013VGeMu3p6qEFWR6K6GGLaW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #9271. ⛔ Deliberately NOT
Fixes— this closes the proximate half only; the structural half named at the bottom stays open on the card.What was measured
⭐ The blind window is 116 consecutive
ci.ymlpush runs onmain— runs 17230 → 17738,2026-09-11T23:17:11Z→2026-09-14T07:09:07Z, 2 days 9 h 42 m.Test (coverage)isfailurein 116 of 116. Last run whose coverage lane was green: run 17227 (a94e4d073, 2026-09-11T21:27:02Z), which also published the onlycoverage-reportartifact in the window.⛔ objectui#8479's 66-run streak is NOT this streak. Population: the 500 most recent
branch=main, event=pushruns of workflow 223145786 (runs 15611 → 17738, from 2026-09-07T06:45Z). It holds 221 successes, the newest being run 17227 — so the 2026-09-07 streak healed and this is a second, later one. ⇒ the card's written re-grade trigger ("longer than roughly a week") does not fire;priority:p2stands on duration.The break in the chain, read rather than inferred
⭐ The blobs are not missing. On red run 17738 all four
coverage-blob-Nartifacts exist and are live (7.8–8.7 MB each); the green run 17227 carries the same four pluscoverage-report.blobs=skippedin the sentinel is the counting step being skipped, not an absent blob.The break is that
coverage-reportgates every step onneeds.test-coverage.result == 'success'. One timed-out row reds a shard, so checkout, download, blob-count and merge are all skipped — and the merge is the only placecoverage.thresholdsis enforced, because the shard legs override all four to0. Verbatim from run 17738:Instrumentation isolated as the single variable
The same-commit CI comparison could never settle this:
test-coverageisif: github.event_name == 'push'andtestisif: github.event_name != 'push', so instrumentation and trigger event are confounded by construction. Isolated locally instead — same tree, same container, same command,--coveragethe only difference:db6aa19a9The five rows that blow the budget under instrumentation are exactly the five CI sites in those files, and the ordering corroborates: the marginal row (15.0 s locally) is the one that fires 11× in CI, while the 24–25 s rows fire 64–94×.
Same instrumented command, same container, two trees, rows paired by name:
check-i18n-dead-keysrows present on BOTH treesa94e4d073(last green)db6aa19a9(today)⇒ the shared rows did not slow down at all. All three of that file's timeout sites are rows that did not exist at the last green run, added 2026-09-12; and the single most frequent site overall,
check-object-metadata-write-doors.test.ts, is an entire file that landed 2026-09-12T04:00Z. The tracked-file corpus grew only 3.2% (7492 → 7733) across the window, nowhere near enough to matter.The repair, and what it does not weaken
A lane-scoped
--testTimeout=60000on the instrumented shard command — 4× the uninstrumented budget, 2.4× the heaviest row observed under instrumentation. ⛔ Not a per-row bump: the wandering site is exactly why it is applied to the lane.vitest.config.mtskeepstestTimeout: 15000, so the uninstrumented lane — the one that actually runs before a merge — is untouched.gates_weakened: NONE.test-coveragehastimeout-minutes: 40against observed shard wall clocks of 698–1053 s.Evidence the gate is restored
⭐ A green shard is not evidence a report was produced, so here is the report and the thresholds. CI cannot host this demonstration — the coverage lane runs only on
pushto[main, develop], never on a PR — so it is reproduced locally over the five CI timeout-site files:--testTimeout=15000--testTimeout=60000The merged report from the passing leg, produced by
ci.yml's own merge command:⭐ Firing control that the thresholds are genuinely enforced and not merely printed — same blobs, same command, floor raised above the measured value:
⛔ The half this PR does NOT fix — it needs a decision above this card
The coverage lane runs only on push to
main/develop. An author adding a 6-second repo-sweep row cannot see this lane before merging; the first signal is after the row is already onmain, on a red that blocks nothing. That is why the class recurs — this file's own comment records objectui#8545 pricing two earlier episodes at 84 and 87 consecutive unevaluated pushes, and this one ran to 116.⇒ A budget alone postpones the next episode, it does not prevent it. Making the lane visible pre-merge is the structural fix, and the card rules out making it blocking — so the fork is the maintainer's, not this PR's.
Verification
pnpm exec vitest run --project unit scripts/__tests__/— 157 files / 4625 tests passed, exit 0 (this is the surface holding every gate that readsci.yml).node scripts/check-changeset-presence.mjs— exit 0, "no changeset is owed".node scripts/check-governed-queue-guard.mjs --test .github/workflows/ci.yml— NOT GOVERNED (control:AGENTS.mdreturns GOVERNED, exit 3).🤖 Generated with Claude Code
https://claude.ai/code/session_013VGeMu3p6qEFWR6K6GGLaW
Generated by Claude Code