fix(repo): escape raw NUL bytes that hid six source files from grep#3127
Merged
Conversation
Six source files embedded a literal U+0000 byte as a composite-key separator instead of the `\u0000` escape. grep/ripgrep scan the whole buffer, classify such a file as binary, and silently return no matches: `grep -n saveMetaItem protocol.ts` reported nothing despite 16 hits, making a core protocol file invisible to code search and to any grep-based lint tooling. This survived review because git only scans the first 8000 bytes for a NUL, and protocol.ts carried its NUL at offset 147230 - so the file kept diffing as ordinary text while grep had already given up on it. Replace each raw NUL with `\u0000`, matching the existing convention in rest-server.ts. That form is chosen over `\0`, which becomes a legacy octal escape error if it is ever followed by a digit. The resulting strings are byte-identical, so behaviour is unchanged: getMetaItemCached still emits the same per-locale ETags as before (en-US 68f978ed / zh-CN 50289f3b / no-locale 75dad1ef), still returns 304 on a same-locale match, and still declines 304 across locales (issue #1319). No changeset: pure fix, no user-facing change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
📓 Docs Drift CheckThis PR changes 5 package(s): 30 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
os-zhuang
added a commit
that referenced
this pull request
Jul 17, 2026
A single literal U+0000 byte makes grep/ripgrep classify the whole file as binary and silently return zero matches: `grep -n saveMetaItem packages/metadata-protocol/src/protocol.ts` reported nothing despite 16 real hits, hiding a core protocol file from code search and from every grep-based lint -- with no error to say so. Nothing caught this. git decides binary-ness from the first 8000 bytes only, and protocol.ts carried its NUL at offset 147230, so it kept diffing as ordinary text through review. That blind spot is how six separate files accumulated the same defect before #3127 fixed them. Add scripts/check-nul-bytes.mjs, wired into the lint workflow beside the other guards. It scans tracked .ts/.tsx/.js/.jsx/.mjs/.cjs/.cts/.mts sources (generated and vendored output excluded), reports each offender as file:line:column plus the byte offset -- the location grep cannot give you -- and points the author at the \u0000 escape form and the existing convention at packages/rest/src/rest-server.ts:1065. Verified by replaying the real defect rather than only a synthetic one: restoring protocol.ts to its pre-#3127 state turns the guard red at 3054:72 (offset 147230, confirmed against an independent oracle) while git still diffs that file as ordinary text. The fixed tree is green across 2261 files in ~0.2s. A tracked file under an excluded dir stays ignored even when force-added, so the exclusion is doing real work rather than riding on .gitignore. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Six source files embedded a literal U+0000 byte as a composite-key separator instead of the
\u0000escape.grep/ripgrepscan the whole buffer, classify such a file as binary, and silently return no matches:...despite 16 real hits. A core protocol file was invisible to code search and to any grep-based lint tooling.
metadata-protocol/src/protocol.ts:3054cli/src/utils/lint-view-refs.ts:148lint/src/build-access-matrix.ts:90plugins/plugin-security/src/suggested-audience-bindings.ts:108services/service-storage/src/attachment-access-hooks.ts:179services/service-analytics/src/__tests__/objectql-strategy-boolean-filter.test.ts:63Why it survived review
git only scans the first 8000 bytes for a NUL.
protocol.tscarried its NUL at offset 147230 — past that window — so git kept diffing it as ordinary text while grep had already given up on it. The defect was invisible from both directions.Fix
Replace each raw NUL with
\u0000, matching the existing convention atrest-server.ts:1065('\u0000default'), which uses the escape for exactly this kind of key sentinel. Chosen over\0, which degrades into a legacy-octal escape error if ever followed by a digit.Verification
The changed line in
protocol.tslives ingetMetaItemCached, which no test executes — it is only ever mocked. So a green suite would not have proven anything here. Verified directly instead, by driving the real method and comparing against an independent oracle that reproduces the pre-fix raw-NUL expression:en-US68f978ed68f978edzh-CN50289f3b50289f3b75dad1ef75dad1efByte-identical. Locale-varying ETags still hold (issue #1319): same-locale match returns 304, cross-locale correctly does not.
Also: zero raw NULs repo-wide;
grep -c saveMetaItem protocol.tsnow returns 16; emitteddist/contains zero NULs; 1383 tests green across all six affected packages (metadata-protocol 34, lint 225, service-analytics 164, service-storage 99, plugin-security 481, cli 380); builds green.No changeset — pure fix, no user-facing behaviour change (AGENTS.md post-task checklist #3).
Follow-up
Nothing prevents this from recurring; six files accumulated it silently. A CI guard rejecting raw NULs in source files would close it — happy to add in a separate PR.
Generated with Claude Code