Streaming support for smithy4s-ndjson

[kubukoz]

Adds codegen support for org.polyvariant.ndjson#ndjsonRestJson, so operations can stream their request body, their response body, or both.

The model

Smithy restricts @streaming to blob/union, and the protocol requires such a member to carry @httpPayload. That gives exactly two framings, applied identically in both directions:

Smithy	Wire	TypeScript
@streaming blob	application/octet-stream	AsyncIterable<Uint8Array>
@streaming union	application/x-ndjson	AsyncIterable<TheUnion>

All four in/out combinations fall out of that.

What it looks like

A streamed member is surfaced as an AsyncIterable on the generated type itself, so signatures stay ordinary — no Omit<…> at call sites:

const feed = new FeedClient(transport, streamTransport)

const { events } = await feed.watch({ id })
for await (const event of events) { /* typed as FeedEvent */ }

await feed.upload({ id, body: chunks /* AsyncIterable<Uint8Array> */ })

The member is left out of the zod object (validating it would mean consuming the stream), and ndjson elements are schema-checked one at a time as they are pulled — a bad element throws StreamDecodeError at that element instead of truncating the stream.

Framing is the transport's job: StreamTransport.requestStream receives explicit requestStreamEncoding / responseStreamEncoding rather than inferring from a content type. Mocks mirror the client, so a story implements a streaming operation as an async generator.

Only the protocol module is a dependency — the codegen keys off @streaming members, so nothing Scala-specific from smithy4s-ndjson is involved.

Backward compatibility

Verified by diffing a non-streaming model's output before and after: byte-for-byte identical. Getting there caught a member-ordering regression that the substring tests missed.

Typechecking the generated TypeScript

The Scala tests assert on substrings, which cannot catch a type error. This adds a nix flake that runs the real tsc over a committed sample (typecheck/ — a model covering every construct, plus a consumer-side usage.ts exercising clients, streams and mocks as a caller would), under strict + erasableSyntaxOnly.

It immediately earned its keep, catching two pre-existing bugs unrelated to streaming:

• a @httpQuery timestamp was cast to string | number | boolean, which a Date does not satisfy — now serialised per target shape;
• a structure whose only member streams intersected with z.infer of an empty z.object (Record<string, never>), making every property never.

CI installs nix, runs nix flake check, and fails if the committed sample has drifted from the model.

Testing

• 18 unit tests (was 16)
• sbtPlugin/scripted, extended with a streaming service
• nix flake check — verified it fails on an introduced type error, not just that it passes

🤖 Generated with Claude Code

https://claude.ai/code/session_01MGq82T8PTfFTS6s2tH5Xs9