Conversation
…ted by measurement
The user asked for an honest review of whether the codebase needs a new north
star. Measured rather than opined, v0.42.0 -> v0.57.0 (15 releases):
workspace source 130,658 -> 174,578 lines (+34%)
instruction_selector.rs 24,909 -> 29,616 lines (+19%)
code (pre-test-module) 18,481
wildcard `_ =>` arms 105
selector churn since v0.42 5,515 added / 808 deleted (6.8 : 1)
VCR-SEL-001 verified rules 40 -> 50 FLAT since v0.50
files saying "mirror" 57
files saying "hand-maintained" 11
files saying "single source of truth" 1
The finding: epic #242's GOAL is right and its STRATEGY is not working. The
thing it exists to replace grew 4,707 lines while the replacement gained ten
rules and then stalled for seven releases. We are not replacing the
patch-accreting selector — we are building a verified path ALONGSIDE it, and
the unverified one is winning on volume.
The mechanism is visible in v0.57. VCR-SEL-001 made 50 ops Rocq-proved via a
BYTE-INVISIBLE FLIP: the proofs are real, the shipped path did change, and the
hand-written arms STAYED. So each release adds a proof, a checker for it, a doc
claim about the checker, and a ledger pin for the doc — every one
hand-maintained. v0.57 found 5 of its 10 defects were in CHECKERS and three doc
claims had rotted behind a green 43/43. The verification machinery is now its
own defect surface, and it grows because nothing is retired.
CORRECTION, not replacement (recorded in CLAUDE.md's North Star):
A rule is not done when it is proven. It is done when the hand-written arm
it replaces is DELETED.
That makes the epic's thesis a metric that CAN GO THE WRONG WAY — this repo's
own standard for whether a gate is worth anything. It is currently going the
wrong way and nothing notices, because nothing measures it.
v0.58 — ten artifacts, subtraction-first:
RQ-58-METRIC pin the numbers (ceilings that must FALL, floor that must
RISE); adding an arm without deleting one must red the gate
RQ-58-RETIRE delete the hand-written arms for the 50 DSL-covered ops —
must be BYTE-IDENTICAL, since the DSL already emits them
RQ-58-WILDCARD the selector's 105 `_ =>` arms, same class as #946 but with
miscompile consequence; rank, convert, state the denominator
RQ-58-MIRRORS generate-don't-mirror over the 57; fixing the FEATURE_MATRIX
freshness gate (render-vs-template, never template-vs-code)
is the highest-leverage item because it hid three others
RQ-58-SELDSL un-stall 50 -> 70 rules, each landing WITH its deletion
RQ-58-SPLIT split the 29,616-line file along the #197 seam, byte-identical
RQ-58-SELECT973 #973 ARM select miscompile AND the CI gap that hid it
RQ-58-VERIFYWIRE #981 shift-decline wiring
RQ-58-OBJECT #938 breaking 0.x-minor on the crate differentials depend on
RQ-58-FLAKE #977, whose silent direction is reading a STALE ELF and passing
Explicitly NOT "clean up the codebase": refactoring 29k lines of selector with
no per-step execution oracle is how you inject the miscompiles this project
exists to prevent — and v0.57 supplies the cautionary case, since the gpio
differential could not have caught its own miscompile. Every subtraction lane
is gated on byte-identity or an execution differential.
rivet: 50 errors before AND after (unchanged); warnings +20, the standard pair
every artifact carries × 10. claim_check 43/43.
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Asked for an honest review of whether the codebase needs a new north star. Measured rather than opined.
The finding: the goal is right, the strategy is not working
v0.42.0 → v0.57.0, fifteen releases:
instruction_selector.rsEpic #242 exists to replace the patch-accreting code generator. The thing it replaces grew 4,707 lines while the replacement gained ten rules and then stalled for seven releases. We are not replacing it — we are building a verified path alongside it, and the unverified one is winning on volume.
Supporting evidence: 105 wildcard
_ =>arms in the selector; 57 files say "mirror", 11 "hand-maintained", exactly 1 "single source of truth".The mechanism, visible in v0.57
VCR-SEL-001 made 50 ops Rocq-proved through a byte-invisible flip — the proofs are real and the shipped path did change, but the hand-written arms stayed. So every release adds a proof, plus a checker for the proof, plus a doc claim about the checker, plus a ledger pin for the doc. All hand-maintained.
v0.57 then found 5 of its 10 defects were in checkers, and three doc claims had rotted behind a green 43/43 — including a capability that release shipped still listed as a loud decline. The verification machinery has become its own defect surface, and it grows because nothing is ever retired.
The correction — not a replacement
That turns the epic's thesis into a metric that can go the wrong way — this repo's own standard for whether a gate is worth anything. It is currently going the wrong way and nothing notices, because nothing measures it. Recorded in
CLAUDE.md's North Star section.v0.58 — ten artifacts, subtraction first
_ =>arms — same class as #946 but with miscompile consequence; rank, convert, state the denominatorselectmiscompile and the CI gap that hid it — ARM fixtures are never compiled in CIWhat this deliberately is not
Not "clean up the codebase." Refactoring 29k lines of instruction selector without a per-step execution oracle is how you inject the miscompiles this project exists to prevent — and v0.57 supplies the cautionary case, since the gpio differential could not have caught its own miscompile. Every subtraction lane is gated on byte-identity or an execution differential; a deletion that moves emitted bytes without an oracle proving the new bytes correct is refused, not explained.
Gates
rivet 50 errors before and after (unchanged); warnings +20 — the standard pair every artifact carries × 10.
claim_check43/43.Refs #242
🤖 Generated with Claude Code
https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L