Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions .github/workflows/npm-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Publishes `runtime/` (@polyvariant/smithy-ts-runtime) to npm on a `v*` tag.
#
# Hand-written, unlike ci.yml: that file is generated by sbt-typelevel
# (`sbt githubWorkflowGenerate`) and CI fails if it drifts, so the npm side
# lives here where it won't be clobbered.
#
# The tag is the single source of version truth — `v0.2.3` publishes 0.2.3,
# matching what sbt-typelevel derives for the JVM artifacts, so a tag ships
# both halves at the same version. package.json carries a placeholder 0.0.0
# that is overwritten here rather than committed per release.
name: Publish npm package

on:
push:
tags: [v*]
# Publishing is irreversible for a given version, so it can also be driven by
# hand — for a re-run after a transient failure.
workflow_dispatch:
inputs:
version:
description: Version to publish (without the leading `v`)
required: true

concurrency:
group: npm-publish-${{ github.ref }}

jobs:
publish:
name: Publish @polyvariant/smithy-ts-runtime
runs-on: ubuntu-22.04
permissions:
contents: read
# Trusted publishing: npm exchanges this workflow's OIDC identity for a
# short-lived publish credential, so there is no token to store or
# rotate. It also backs the provenance attestation, which lets consumers
# verify the tarball was built from this commit by this workflow.
id-token: write
steps:
- name: Checkout
uses: actions/checkout@v6

- name: Setup pnpm
uses: pnpm/action-setup@v4

- name: Setup node
uses: actions/setup-node@v4
with:
node-version: 22
cache: pnpm
registry-url: https://registry.npmjs.org

- name: Resolve version from the tag
id: version
run: |
if [ -n "${{ inputs.version }}" ]; then
version="${{ inputs.version }}"
else
version="${GITHUB_REF_NAME#v}"
fi
echo "version=$version" >> "$GITHUB_OUTPUT"
echo "Publishing $version"

- name: Install
run: pnpm install --frozen-lockfile

# The same check `nix flake check` runs: build, typecheck, unit tests, and
# the generated-sample typecheck that pins the transport types together.
# A tag should never publish something that doesn't pass it.
- name: Check
run: pnpm check

- name: Set the version
working-directory: runtime
run: pnpm version "${{ steps.version.outputs.version }}" --no-git-tag-version --allow-same-version

# `npm publish`, not `pnpm publish`: the OIDC exchange that backs trusted
# publishing is implemented in the npm CLI. `--provenance` is implied by
# trusted publishing, but stays explicit so a fallback to a token still
# attests. npm >= 11.5.1 is required for OIDC.
- name: Publish
working-directory: runtime
run: |
npm install -g npm@latest
npm publish --access public --provenance
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,7 @@ node_modules/
.metals/
.bloop/
metals.sbt

# nix build output
result
result-*
85 changes: 73 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@ From one model, into a single `generated.ts`:
`.brand<'Xxx'>()`-ed for nominal typing).
2. **Error classes** — one `XxxError extends Error` per `@error` shape, carrying the declared
HTTP status and error-type name.
3. **Transport** — a small `Transport` interface plus request/response types you implement once
(over `fetch`, axios, …). The generated clients are transport-agnostic. Models with
streaming operations also get a `StreamTransport` (see [Streaming](#streaming)).
3. **Transport** — a small `Transport` interface plus request/response types. The generated
clients are transport-agnostic; a ready-made `fetch` implementation ships separately in
[`@polyvariant/smithy-ts-runtime`](runtime/). Models with streaming operations also get a
`StreamTransport` (see [Streaming](#streaming)).
4. **Service clients** — one `XxxClient` class per service, one `async` method per operation.
Each method walks the operation's `@http` trait to build the request (URI labels, query,
headers, body) and parses the response with the generated output schema, dispatching declared
Expand Down Expand Up @@ -135,7 +136,8 @@ streamed response commits its status before the first element, **a mid-stream fa
be an HTTP status** — model it as a member of the streamed union (the protocol expects a
terminal member such as `completed` / `failed`).

Framing itself is the transport's job: implement `StreamTransport.requestStream`, which
Framing itself is the transport's job: `@polyvariant/smithy-ts-runtime` does it for you, and
a hand-rolled transport implements `StreamTransport.requestStream`, which
receives `requestStreamEncoding` / `responseStreamEncoding` telling it which framing to
apply, so it never has to guess from a content type. The generated code adds the per-element
schema on top. Services with streaming operations take both halves —
Expand All @@ -155,9 +157,42 @@ mockService(WatcherMock, {
})
```

> **TODO:** the `Transport` / `StreamTransport` implementations are currently written by hand
> in each consuming project. They should be published as a library (a `fetch`-based transport
> with ndjson/binary framing) so consumers don't re-implement the same framing logic.
Implementations of both halves ship in
[`@polyvariant/smithy-ts-runtime`](runtime/) — see [Transports](#transports).

## Transports

The codegen emits the `Transport` / `StreamTransport` *interfaces*; the
implementation lives in [`@polyvariant/smithy-ts-runtime`](runtime/), published
separately so a generated file stays dependency-free.

```sh
pnpm add @polyvariant/smithy-ts-runtime
```

```ts
import { chain, fetchTransport, withHeaders } from '@polyvariant/smithy-ts-runtime'
import { DirectoryClient, FeedClient } from './generated.js'

const transport = chain(
fetchTransport({ baseUrl: '/api' }),
withHeaders(() => ({ authorization: `Bearer ${token()}` })),
)

const directory = new DirectoryClient(transport)
const feed = new FeedClient(transport, transport) // streaming ops take both
```

It covers the whole contract — unary requests, ndjson and binary framing in both
directions, 401 handling, and a middleware seam (`chain` / `around` / `tap` /
`interceptorStack`) for tracing, auth headers and error reporting. The framing
primitives are exported on their own for transports it doesn't ship.

The library imports nothing from generated code: it declares structural copies of
the transport types, which TypeScript matches by shape. `typecheck/src/runtimeUsage.ts`
compiles the two against each other, so the pairing can't drift silently.

See [runtime/README.md](runtime/README.md) for the full API.

## Conventions & limits

Expand All @@ -177,21 +212,47 @@ mockService(WatcherMock, {
sbt test # unit tests
sbt sbtPlugin/scripted # the sbt plugin, end to end
sbt tsCodegenSample # regenerate typecheck/src/generated.ts
nix flake check # type-check the generated TypeScript with tsc
nix flake check # type-check + test the TypeScript side
pnpm check # the same, without nix (needs `pnpm install` first)
```

`typecheck/` holds a model (`model.smithy`) exercising every construct the codegen emits, its
committed output (`src/generated.ts`), and a consumer-side `src/usage.ts` that uses the
clients, streams and mocks the way a caller would. `nix flake check` runs the real `tsc` over
both under `strict` + `erasableSyntaxOnly`.
The TypeScript lives in a pnpm workspace of two packages:

- `runtime/` — the published transport library. `pnpm --filter @polyvariant/smithy-ts-runtime
run check` builds it, type-checks it and runs its `node:test` suite (framing round-trips,
the transport against a `fetch` double, middleware ordering).
- `typecheck/` — a model (`model.smithy`) exercising every construct the codegen emits, its
committed output (`src/generated.ts`), a consumer-side `src/usage.ts` that uses the clients,
streams and mocks the way a caller would, and `src/runtimeUsage.ts`, which drives those same
clients with the *library's* transport. That last file is what pins the library's structural
transport types to the ones the codegen emits — change one without the other and it stops
compiling.

`nix flake check` runs both under `strict` + `erasableSyntaxOnly`.

This matters because the Scala tests assert on substrings of the emitted file, which cannot
catch a type error — a generator declared as `AsyncIterable`, an intersection with an empty
`z.object`, a `Date` cast to a query value. After changing the generator, run
`sbt tsCodegenSample` and commit the result; CI fails if it drifts.

Changing anything under `runtime/` or `typecheck/` that moves the lockfile means updating
`pnpmDeps.hash` in `nix/typecheck.nix` — build once, and nix prints the hash it wanted.
Note that `nix build` only sees git-tracked files, so `git add` new files before running it.

A `nix develop` shell provides node, pnpm, sbt and a JDK.

### Releasing

A `v*` tag ships both halves at the same version: sbt-typelevel publishes the JVM artifacts
from the generated `ci.yml`, and `.github/workflows/npm-publish.yml` publishes
`@polyvariant/smithy-ts-runtime` to npm. The tag is the only source of version truth —
`runtime/package.json` keeps a placeholder `0.0.0` that the workflow overwrites, so there is no
version to bump by hand.

`ci.yml` is generated (`sbt githubWorkflowGenerate`) and CI fails if it drifts; the npm
workflow is hand-written for that reason. Publishing needs an `NPM_TOKEN` secret with publish
rights on the `@polyvariant` scope.

## Dependencies

`smithy-build`, `smithy-codegen-core`, `smithy-model`, `alloy-core`, `smithy4s-protocol`, and
Expand Down
68 changes: 51 additions & 17 deletions nix/typecheck.nix
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,22 @@
pnpmConfigHook,
}:

# `tsc` over the committed sample in typecheck/ — the only check that proves the
# generated TypeScript is valid TypeScript.
# `tsc` over the TypeScript in this repo — the only check that proves the
# generated code, and the runtime library that serves it, are valid TypeScript.
#
# Hermetic: the .ts is committed rather than generated here, so no JVM or sbt is
# needed in the sandbox, and node_modules comes from a pnpm FOD. Regenerate the
# sample with `sbt tsCodegenSample`; CI runs `sbt tsCodegenSampleCheck` to fail
# if it drifts from the model.
# Two workspace packages:
#
# runtime/ the published transport library: built (so its .d.ts exist),
# typechecked, and unit-tested with node:test.
# typecheck/ the committed sample generated.ts, a consumer-side usage file,
# and runtimeUsage.ts — which is what proves the library's
# structural Transport types still match the ones the codegen
# emits.
#
# Hermetic: the .ts sample is committed rather than generated here, so no JVM or
# sbt is needed in the sandbox, and node_modules comes from a pnpm FOD.
# Regenerate the sample with `sbt tsCodegenSample`; CI runs
# `sbt tsCodegenSampleCheck` to fail if it drifts from the model.

let
pnpm = pnpm_10;
Expand All @@ -23,15 +32,32 @@ stdenvNoCC.mkDerivation (finalAttrs: {
pname = "smithy-ts-codegen-typecheck";
version = "0.0.0";

src = lib.cleanSourceWith {
src = lib.cleanSource ../typecheck;
filter =
path: type:
let
rel = lib.removePrefix (toString ../typecheck + "/") (toString path);
in
!(lib.hasPrefix "node_modules" rel);
};
# The workspace root: both packages plus the lockfile and workspace manifest.
# Only the JS side of the repo, so a change to the Scala tree doesn't
# invalidate this derivation.
src =
let
root = ../.;
keepTop = [
"package.json"
"pnpm-lock.yaml"
"pnpm-workspace.yaml"
"runtime"
"typecheck"
];
in
lib.cleanSourceWith {
src = lib.cleanSource root;
filter =
path: _type:
let
rel = lib.removePrefix (toString root + "/") (toString path);
top = lib.head (lib.splitString "/" rel);
in
builtins.elem top keepTop
&& !(lib.hasInfix "node_modules" rel)
&& !(lib.hasInfix "dist" rel);
};

nativeBuildInputs = [
nodejs
Expand All @@ -43,12 +69,20 @@ stdenvNoCC.mkDerivation (finalAttrs: {
inherit pnpm;
inherit (finalAttrs) pname version src;
fetcherVersion = 3;
hash = "sha256-Ey7ZO5qu7Nnc0udSu0v1e+2FPQmb53KLtxo64jKD0hc=";
hash = "sha256-X8nNnf+hpzKtNFvXbtPx/g9rimlit2/t3AGpSrjw8rU=";
};

buildPhase = ''
runHook preBuild
pnpm run typecheck

# The library first: typecheck/ imports its built .d.ts, and its own
# `typecheck` script builds as a prerequisite.
pnpm --filter @polyvariant/smithy-ts-runtime run check

# Then the generated sample + the consumer-side usage, including the file
# that pins the library's transport types to the generated ones.
pnpm --filter smithy-ts-codegen-typecheck run typecheck

runHook postBuild
'';

Expand Down
10 changes: 10 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"name": "smithy-ts-codegen-workspace",
"private": true,
"version": "0.0.0",
"type": "module",
"packageManager": "pnpm@10.33.0",
"scripts": {
"check": "pnpm --filter @polyvariant/smithy-ts-runtime run check && pnpm --filter smithy-ts-codegen-typecheck run typecheck"
}
}
59 changes: 59 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions pnpm-workspace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
packages:
- runtime
- typecheck
2 changes: 2 additions & 0 deletions runtime/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules/
dist/
Loading
Loading