ci(lint): reject raw NUL bytes in tracked sources#3135
Merged
Conversation
A single literal U+0000 byte makes grep/ripgrep classify the whole file as binary and silently return zero matches: `grep -n saveMetaItem packages/metadata-protocol/src/protocol.ts` reported nothing despite 16 real hits, hiding a core protocol file from code search and from every grep-based lint -- with no error to say so. Nothing caught this. git decides binary-ness from the first 8000 bytes only, and protocol.ts carried its NUL at offset 147230, so it kept diffing as ordinary text through review. That blind spot is how six separate files accumulated the same defect before #3127 fixed them. Add scripts/check-nul-bytes.mjs, wired into the lint workflow beside the other guards. It scans tracked .ts/.tsx/.js/.jsx/.mjs/.cjs/.cts/.mts sources (generated and vendored output excluded), reports each offender as file:line:column plus the byte offset -- the location grep cannot give you -- and points the author at the \u0000 escape form and the existing convention at packages/rest/src/rest-server.ts:1065. Verified by replaying the real defect rather than only a synthetic one: restoring protocol.ts to its pre-#3127 state turns the guard red at 3054:72 (offset 147230, confirmed against an independent oracle) while git still diffs that file as ordinary text. The fixed tree is green across 2261 files in ~0.2s. A tracked file under an excluded dir stays ignored even when force-added, so the exclusion is doing real work rather than riding on .gitignore. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
scripts/check-nul-bytes.mjsand wires it into thelintjob, so a raw NUL (0x00) in a tracked source file fails CI.Why
A single literal U+0000 byte makes grep/ripgrep classify the whole file as binary and silently return zero matches.
grep -n saveMetaItem packages/metadata-protocol/src/protocol.tsreported nothing despite 16 real hits — a core protocol file invisible to code search and to every grep-based lint, with no error to say so.Nothing caught this. git decides binary-ness from the first 8000 bytes only, and protocol.ts carried its NUL at offset 147230, so it kept diffing as ordinary text through review. That blind spot is how six separate files accumulated the same defect before #3127 fixed them. This guard is what keeps them from coming back.
What it does
Scans tracked
.ts/.tsx/.js/.jsx/.mjs/.cjs/.cts/.mtsfiles (git ls-files; generated and vendored output excluded). On a hit it reportsfile:line:columnplus the byte offset — the location grep fundamentally cannot give you — and points the author at the\u0000escape form and the existing convention atpackages/rest/src/rest-server.ts:1065.Verification
Verified by replaying the real defect, not just a synthetic one. Restoring
protocol.tsto its pre-#3127 state reproduces the original exactly and shows why it survived:grep -n saveMetaItemgrep -a)git diff --numstatprotocol.ts:3054:72, byte offset 147230The reported
3054:72was confirmed against an independent oracle, and the offending line is exactly the composite-key separator. Also checked: a NEW file with 2 NULs is caught (plural wording correct); a.tsxfile is caught; a file using the correct escape form passes; an untracked file is ignored; and a tracked file under an excluded dir stays ignored even when force-added — so the exclusion does real work rather than riding on.gitignore.Clean tree: green across 2261 files in ~0.2s. ESLint clean.
Notes
.jsx/.cjsare the same class of hand-authored source and match zero files today, so including them closes a future gap at no cost.\u0000into a real NUL. It happened twice while writing this PR — aBashcall was rejected for containing control characters because the sequence in my own comments had already become real NULs. Build such strings programmatically (chr(92) + 'u0000') rather than typing them.skip-changeset).🤖 Generated with Claude Code