Skip to content

http: address review feedback for packed headers - #20

Draft
anonrig wants to merge 38 commits into
mainfrom
cursor/http-http2-perf-reviews-47de
Draft

http: address review feedback for packed headers#20
anonrig wants to merge 38 commits into
mainfrom
cursor/http-http2-perf-reviews-47de

Conversation

@anonrig

@anonrig anonrig commented Aug 17, 2026

Copy link
Copy Markdown
Owner

This addresses the review comments on nodejs/node#65332 (CHANGES_REQUESTED from @jasnell, plus the Copilot end() callback note). The same commits have been pushed to cursor/http-http2-perf-f24c so that PR is updated.

Review fixes

  • Endianness (s390x / AIX CI): packed incoming-header integers are now written and read as little-endian. The previous memcpy of host-endian uint32_t disagreed with JS, which always reconstructed count/flags from individual bytes as LE. That matches the Jenkins failures on rhel*-s390x and AIX.
  • Export layout constants: magic, flag bits, and offsets live in C++ and are exported on HTTPParser so JS does not keep a second copy.
  • rawHeaders stays an own property: IncomingMessage still constructs this.rawHeaders = []. Packed headers install an own accessor that materializes into an own data property on first read, so this is not a prototype-getter / semver-major change.
  • end(chunk, cb) race: the user callback is queued before write_(), so a synchronous socket.write() callback (combined chunked-end via assignSocket()) cannot mark the message finished first and then report ERR_STREAM_ALREADY_FINISHED.
  • HTTP/2 toLowerCase micro-opts removed: measured HTTP/2 header work was within noise; the duplicated skip-toLowerCase helpers are gone. buildNgHeaderString still uses the hoisted processor / SafeSet cleanup.
  • All-caps header special cases removed from the materialized _hasBodyHeaders() fallback; Title-Case, lowercase, and toLowerCase() remain.
  • unpackHeaderList export removed from _http_common (it was only used by the parser binding test, which now materializes via internalBinding).

Tests

  • Packed-buffer LE magic (NHDR bytes) in test-http-parser.js.
  • Binding helpers, exported constants, own-property rawHeaders, Host / Expect / TE / body-header lazy checks, and materialization fallbacks in test-http-native-headers.js.
  • Sync end(chunk, cb) with assignSocket() + custom Writable.
  • // Flags: --expose-internals on tests that require('internal/test/binding').

Locally, 780/780 test-http* / test-https* / test-http2* (parallel + sequential + pummel + async-hooks) passed on a release build. ESLint on the touched JS files and cpplint on src/node_http_parser.cc are clean.

The previous x86_64-darwin shared-library job on the original PR failed installing Cachix (cachix.org DNS), not compiling this change. Regular macOS CI passed.

Open in Web Open in Cursor 

nodejs-github-bot and others added 30 commits August 14, 2026 18:33
PR-URL: nodejs#65114
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Signed-off-by: ulofiai <monsterking@tutamail.com>
PR-URL: nodejs#65118
Fixes: nodejs#63638
Refs: libuv/libuv#5152
Refs: libuv/libuv@e640dc9
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: nodejs#65250
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: René <contact.9a5d6388@renegade334.me.uk>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Stewart X Addison <sxa@redhat.com>
Reviewed-By: Mike McCready <66998419+MikeMcC399@users.noreply.github.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Beth Griggs <bethanyngriggs@gmail.com>
Use undefined as the no-error sentinel when cancelling broadcast and
share consumers. This ensures that 0, an empty string, false, and null
are propagated instead of being converted to clean completion.

Make sync share surface cancellation reasons before handling detached
consumers, and add regression coverage for async and sync consumers.

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: codex:gpt-5.6-sol
PR-URL: nodejs#64705
Fixes: nodejs#64704
Reviewed-By: James M Snell <jasnell@gmail.com>
SQLite requires that an authorizer callback not modify the connection
that invoked it, and counts sqlite3_prepare_v2() and sqlite3_step() as
modifications. node:sqlite let the callback call prepare(), exec(), the
statement execution methods, and other connection-mutating APIs on the
same DatabaseSync. Track authorizer depth on DatabaseSync with an RAII
guard around the callback and throw ERR_INVALID_STATE from the affected
entry points while it is on the stack. Covering every authorizer
invocation, including the re-prepare that SQLite can run during
sqlite3_step(), exposed a second and distinct hazard: reentering a
statement that is currently being stepped is a use-after-free rather
than a contract violation, since finalizing it frees the virtual machine
under sqlite3_step() and re-running it resets that machine
mid-execution. Any callback SQLite invokes during execution can reach
it, so a user-defined function is enough. Track the statements currently
being stepped and reject reentry into only those, which leaves a
user-defined function free to prepare, run, and finalize its own helper
statements.

Signed-off-by: Trevor Burnham <trevorburnham@gmail.com>
Fixes: nodejs#63207
Assisted-by: claude:opus-5
PR-URL: nodejs#65156
Reviewed-By: Xuguang Mei <meixuguang@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Signed-off-by: Guilherme Araújo <arauujogui@gmail.com>
PR-URL: nodejs#62757
Reviewed-By: Xuguang Mei <meixuguang@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Edy Silva <edigleyssonsilva@gmail.com>
Replace the async generator backing Symbol.asyncIterator with a
hand-rolled iterator. The generator machinery costs several extra
promise allocations and microtask hops per chunk: yield awaits the
yielded value and resolves the pending request through separate
promises. Buffered chunks are now delivered as an already-resolved
promise, one microtask sooner than before.

Thenable chunks are still awaited before delivery, requests received
while a next() is outstanding are queued, and return()/throw() before
the first next() complete the iterator without touching the stream.

The earlier delivery is observable by code racing an abort against
the first chunk. The flatMap AbortSignal test relied on such a race;
it is reworked to abort deterministically while two mappers are in
flight, asserting the concurrency limit, in-flight cancellation and
rejection, without depending on delivery timing or timers.

streams/readable-async-iterator.js sync='yes': +32.59% (***)
streams/readable-async-iterator.js sync='no': +9.84% (***)

Assisted-by: Claude Fable 5
Signed-off-by: Matteo Collina <matteo.collina@gmail.com>
PR-URL: nodejs#64447
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Mattias Buelens <mattias@buelens.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
The highWaterMark values were passed as properties of the underlying
source and sink dictionaries, where they are ignored: a queuing
strategy's highWaterMark is read from the constructors' second argument.
Every configuration therefore measured the identical workload at the
default highWaterMark of 1, which also explains the historically high
run-to-run variance of this benchmark family.

Pass the strategies as the constructors' second argument and cover the
default (1) alongside buffered (1024, 4096) configurations.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: nodejs#65138
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Three related reductions on the per-chunk paths:

Wrap user sink.write and source.pull callbacks without coercing their
result into a promise. When the callback returns a non-thenable (the
common synchronous case), fulfillment is guaranteed and no then() lookup
is observable, so the fulfilled reaction is enqueued through a single
shared resolved promise at the exact microtask position the coerced
promise's reaction would have had, skipping the implicit async-wrapper
promise per chunk. Thenable results go through PromiseResolve(), which
matches the spec's "a promise resolved with" conversion (identity for
native promises).

Park pipeTo's pump on backpressure by installing a record that
duck-types the writer's lazily-materialized [[readyPromise]] record and
whose resolve function is the pump continuation itself. Backpressure
clearing then resumes the pump directly instead of materializing a fresh
promise record plus reaction per flip, and the pump no longer schedules
a microtask per batch. writableStreamUpdateBackpressure publishes the
new backpressure state before resolving the ready record so the pump
observes the updated value.

Replace queueMicrotask() on the pipeTo and tee chunk-forwarding paths
with a reaction on the shared resolved promise, which enqueues the
continuation at the same position without the per-call scheduling
overhead.

pipe-to improves by 8-14% across all benchmark configurations, with
readable-read and tee also improving in spot runs.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: nodejs#65138
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
The start, pull, and write non-op algorithms are all raw callbacks with
an identical empty body now, so a single shared nonOpCallback replaces
nonOpStart, nonOpPull, and nonOpWrite.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: nodejs#65138
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Signed-off-by: greenhead <shren0812@gmail.com>
PR-URL: nodejs#65265
Refs: https://www.sqlite.org/session/c_changeset_abort.html
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
The pointer range test creates multiple closures from the same function
literals and explicitly requests synchronous optimization. V8 can also
schedule concurrent recompilation for those closures.

Wait for background optimization before closing the dynamic library so
compiler jobs cannot outlive the fast FFI metadata they reference.

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: codex:gpt-5.6-sol
PR-URL: nodejs#65300
Refs: https://github.com/nodejs/reliability/issues?q=sort%3Aupdated-desc%20%22test-ffi-fast-integer-validation%22
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
The quic implementation calls setWakeUp with
the assumption, that it is only executed once
per event loop cycle.
This assumption is wrong.
Only setImmediate will guarantee, that the
execution is delayed to later in the event loop
and happening once in the event loop.

Fixes: nodejs#64035
Signed-off-by: Marten Richter <marten.richter@freenet.de>
PR-URL: nodejs#64044
Reviewed-By: James M Snell <jasnell@gmail.com>
Signed-off-by: Felix P. <devfep@gmail.com>
PR-URL: nodejs#65268
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Signed-off-by: Guilherme Araújo <arauujogui@gmail.com>
PR-URL: nodejs#62241
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: René <contact.9a5d6388@renegade334.me.uk>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
Co-authored-by: Aviv Keller <me@aviv.sh>
Signed-off-by: Aviv Keller <me@aviv.sh>
PR-URL: nodejs#64894
Fixes: nodejs#43583
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
To hopefully get to the bottom of WPT crashes that have no traces.

Signed-off-by: Filip Skokan <panva.ip@gmail.com>
Co-authored-by: Aviv Keller <me@aviv.sh>
Signed-off-by: Aviv Keller <me@aviv.sh>
PR-URL: nodejs#64894
Fixes: nodejs#43583
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Signed-off-by: Aviv Keller <me@aviv.sh>
PR-URL: nodejs#64894
Fixes: nodejs#43583
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
Co-authored-by: Aviv Keller <me@aviv.sh>
Signed-off-by: Aviv Keller <me@aviv.sh>
PR-URL: nodejs#64894
Fixes: nodejs#43583
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Signed-off-by: Aviv Keller <me@aviv.sh>
PR-URL: nodejs#64894
Fixes: nodejs#43583
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Signed-off-by: NIxxy25 <tellaoyinkansola25@gmail.com>
PR-URL: nodejs#65271
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
Signed-off-by: Aviv Keller <me@aviv.sh>
PR-URL: nodejs#64986
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Replace `Array.prototype.forEach()` with `for...of` loops across 17
tests in `test/parallel`, so each loop body reads as a plain statement
rather than an arrow callback.

None of the iterated values are sparse arrays, the one case where
`forEach` and `for...of` genuinely differ, so both constructs visit the
same elements in the same order. No callback relied on `this`, an early
return, or async behaviour, and the number of assertions run in each
file is unchanged.

Signed-off-by: Phillip Markert <phillip@ephisys.com>
PR-URL: nodejs#65272
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Signed-off-by: freida-code <150387862+freida-code@users.noreply.github.com>
PR-URL: nodejs#65270
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
PR-URL: nodejs#65224
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Signed-off-by: Lazizbek Ergashev <lazerg2@gmail.com>
PR-URL: nodejs#65283
Fixes: nodejs#65280
Refs: nodejs/node-v0.x-archive#853
Refs: nodejs@3935adc
Refs: nodejs#18297
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
the binary-upload target uses $(TARNAME)-$(OSTYPE)-$(ARCH).tar.xz as the
name to upload whereas it is created by the $(BINARYTAR) target as
$(BINARYNAME). Since BINARYNAME includes the optional VARIATION when
present this gets missed out int he binary-upload target, for example
during a release build for Alpine/musl. This commit changes the
binary-upload target to use the same variable for the tarball that is
used when the file is created.

Signed-off-by: Stewart X Addison <sxa@ibm.com>
PR-URL: nodejs#65282
Reviewed-By: Richard Lau <richard.lau@ibm.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Replace regex-based header token/value checks with byte lookup
tables, cache OutgoingMessage lenient-validation, and coalesce
headers with small Buffer bodies into a single socket write.

Scan IncomingMessage rawHeaders for Content-Length and
Transfer-Encoding so optimizeEmptyRequests does not force
req.headers construction.

On the HTTP/2 path, skip toLowerCase for already-lowercase names,
use a Set for sensitive/single-value header checks, and reserve
outgoing session storage to avoid reallocs while gathering
nghttp2 frames.

Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
Send headers, the last chunk, and the chunked terminator in a single
write() when res.end() is used with Transfer-Encoding: chunked.

Reuse prebuilt HTTP/1.1 status lines for default reason phrases, skip
toLowerCase() on common outgoing header names, and hoist HTTP/2 header
serialization off the per-call closure.

Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
The chunked end() fast path concatenated headers with the body and
wrote the result using the body encoding. That re-encoded obs-text
header values as UTF-8 and reduced corked res.end() to a single
socket.write(), which broke test-http-server-non-utf8-header and
test-http-response-cork.

Copy headers as latin1 into the combined buffer, and accept a single
write after uncork.

Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
anonrig and others added 8 commits August 16, 2026 20:16
JS lookup tables were slower than V8's regex for header values and
for token names longer than ~10 bytes. Restore the regex path for
those cases, and use the table for names of length <= 10 so
Connection / Keep-Alive stay on the faster path.

Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
Pass NativeHttpHeaders from the parser instead of a JS string
array. IncomingMessage materializes rawHeaders/headers only when
read. Host/Expect/body-header checks use C++ has/get.
Skip Buffer::Copy for dumped bodies via parser.setSkipBody().

Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
Store header name/value bytes in a packed Buffer instead of a
per-request native BaseObject or JS string array. IncomingMessage
materializes rawHeaders only when read. Host/Expect/body-header
checks use flag bits so the default server path never creates
header strings. Dumped bodies skip Buffer::Copy via setSkipBody().

Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
Drop the extra util-inl.h include, wrap lines to 80 columns, and
apply clang-format so format-cpp / lint-cpp pass.

Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
Write packed incoming-header integers as little-endian so s390x and
AIX agree with JS. Export the layout constants from C++ instead of
duplicating them. Keep IncomingMessage.rawHeaders as an own property
to avoid a semver-major prototype change. Queue end() callbacks
before the combined write so sync socket writes still succeed. Drop
the HTTP/2 toLowerCase micro-optimizations and the unused
unpackHeaderList export.

Co-authored-by: Yagiz Nizipli <anonrig@users.noreply.github.com>
Restore the buildNgHeaderString JSDoc, wrap the 400-response assertion
in mustCall, and use mustSucceed for the sync end() callback test.

Co-authored-by: Yagiz Nizipli <anonrig@users.noreply.github.com>
These files require internal/test/binding, which is only available when
Node is started with --expose-internals.

Co-authored-by: Yagiz Nizipli <anonrig@users.noreply.github.com>
Exercise _hasHeader/_getHeader/_hasBodyHeaders after rawHeaders is
materialized, plus empty name/value binding helpers, so the JS scan
paths stay covered once the C++ buffer is dropped.

Co-authored-by: Yagiz Nizipli <anonrig@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.