Skip to content

ci: smoke-check a local preview of the packaged Worker before merge - #278

Merged
os-bill merged 3 commits into
mainfrom
claude/issue-274-pre-merge-render-gate
Sep 8, 2026
Merged

ci: smoke-check a local preview of the packaged Worker before merge#278
os-bill merged 3 commits into
mainfrom
claude/issue-274-pre-merge-render-gate

Conversation

@os-bill

@os-bill os-bill commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Fixes #274

A rendering defect currently has exactly one detector: production. #269 built the
recovery around it — the deploy is gated on CI, the published artifact is the one CI
tested, the live site is smoke-checked after deploying, a bad deploy auto-rolls back —
but the chain deploy succeeds -> smoke fails -> rollback fires has never once executed
end to end. A check that fires before merge costs a red pull request; the same check
firing after merge costs a live outage plus a recovery path nobody has seen run.

This adds one step to the build job: .github/scripts/smoke-docs.mjs — the same
script deploy-docs.yml runs against https://docs.objectos.ai, unmodified — pointed
with --base at a local opennextjs-cloudflare preview of the Worker this job already
packages.

The card was cheaper than it was written, and this is why

It was filed when the build job packaged a Worker only on a push to main. #262 removed
that condition, so on every pull request a .open-next package already exists by the
time this step runs. opennextjs-cloudflare preview does not build: it populates the
incremental cache and runs wrangler dev on the package that is already there. No
opennextjs-cloudflare build, no second next build, no Cloudflare credentials — the
Worker is served locally under real workerd.

Measured on a GitHub runner, this step start to finish:

run step 15 boot to Ready on
34250422860 10 s
34250966221 11 s 9 s

For scale, in the same job turbo run build took 69 s and packaging the Worker 16 s. The
whole build job went from about 100 s to 131 s.

Why the step runs last, after the artifact upload

preview copies .open-next/cache into .open-next/assets/cdn-cgi. Measured on this
tree, .open-next goes from 386 MB to 653 MB the moment the preview boots.
opennextjs-cloudflare deploy makes that same copy in the deploy job from the
.open-next/cache the artifact already carries, so booting the preview before the upload
would add 268 MB to every main artifact, both ways across the wire, to ship a copy the
deploy remakes anyway. Ordered last, the uploaded bundle is byte-for-byte what it was
before this step existed, and the gating is unchanged: deploy-docs needs the whole
build job, so a red here keeps a bad render off production exactly as a red anywhere
above it does.

The gate is demonstrated red, three ways

1. The 09-04 defect, reproduced offline. Ablated apps/docs/open-next.config.ts by
removing the incrementalCache override — the configuration that caused the outage:
dummy cache plus dynamicParams = false means every page is answered by the prerendered
_not-found route. Mutation confirmed on disk (git hash-object moved from
315674e7 to d70daefc, injected marker present once, removed line absent) and confirmed
to have reached the artifact the preview serves (the compiled
.open-next/.build/open-next.config.mjs lost its static-assets-incremental-cache
reference), then re-packaged before judging anything. The step exited 1 with

✗ smoke: 21 finding(s) against http://127.0.0.1:8792

all four targets 404, [status] [final-path] [error-shell] [too-little-text] [h1-mismatch] [few-links] — the same 21 the card recorded for main at d6f5dda.
Restored by git checkout HEAD on the absolute path, verified byte-for-byte (hash back to
315674e7, git diff HEAD empty), re-packaged, and the ablation marker confirmed absent
from the compiled config again.

2. A preview that never becomes ready is NOT MEASURED, not a pass. With the boot
budget cut to 5 s against a healthy preview, the step exits 1: "the 5s boot budget ran
out with the preview still starting"
. This is the failure shape the card warns about —
"no findings" from a dead server is indistinguishable from a rendered site, and this lane
logged four probes of that shape in one day.

3. A preview that cannot start at all. With the compiled OpenNext config removed the
preview dies in about two seconds; the step exits 1 with "the preview process exited
before it was ready"
and prints the preview log (ERROR Could not find compiled Open Next config) into the step summary.

Green in CI, step 15 of run 34250966221, its own record:

preview ready: [wrangler:info] Ready on http://127.0.0.1:8792
preview pid 3100 in process group 3100, step in 2034
✓ /                             http 200  final /docs   4640 visible chars  21 same-site links
✓ /en/docs                      http 200  final /docs   4640 visible chars  21 same-site links
✓ /docs/quickstart              http 200                9414 visible chars  14 same-site links
✓ /docs/build/interface/views   http 200                7772 visible chars  22 same-site links
✓ (control, expected red) /docs/objectos-smoke-negative-control-269
    control tripped [status error-shell too-little-text few-links]
✓ smoke: 4 page(s) rendered against http://127.0.0.1:8792, negative control demonstrated red

The negative control is not decoration and is carried along unchanged: every run also
fetches a /docs/ slug no page claims and fails on negative-control-passed if it comes
back clean.

What the first CI run caught, and the fix

Run 34250422860 was green and still left evidence of a defect in this step: the runner's
own post-job sweeper reported Terminate orphan process for esbuild and two workerd
processes. The preview is a chain of six processes — pnpm, node, sh, pnpm, wrangler,
workerd — and a SIGTERM to the pnpm wrapper at the top does not reach the rest, so it is
started with setsid and the cleanup signals the whole process group.

setsid only moves the child into its own group once the forked child has exec'd it, so
reading that group with ps immediately after & is a race that can still return the
step's own group — and the guard that stops the step from signalling itself then falls
back to the pnpm wrapper alone. The same code cleaned up completely in this repo's
container on every run, which is how a race presents. The group is now resolved inside the
cleanup function, at kill time. Run 34250966221 prints preview pid 3100 in process group 3100, step in 2034 and its Cleaning up orphan processes line is followed by nothing at
all. A preview that genuinely shares the step's group now emits a warning instead of
leaving a silent orphan.

Scope

.github/workflows/ci.yml only. .github/scripts/smoke-docs.mjs is untouched — it needed
no change to accept a local base, --base was already there. deploy-docs.yml and
rollback-docs.yml are untouched. No changeset: nothing is published from a package here.

Verification on 815e881, the head of this branch: CI run 34250966221 green with step 15
The docs site renders — smoke-check a local preview recorded success; the step body
extracted from the committed YAML and executed verbatim locally, green plus both NOT
MEASURED shapes red; node .github/scripts/smoke-docs.mjs --self-test exit 0 (11 response
cases, 2 run cases, all 12 rules demonstrated able to fail); the workflow parses to 14
build steps and the file carries no control bytes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ChPQM8jamxLUfUAxwFpJ8S

os-bill and others added 3 commits September 8, 2026 16:13
The 09-04 outage was detected in production. #269 built detection and
recovery around the deploy, but the only environment in which a rendering
defect is DETECTED is still production, and the chain `deploy succeeds ->
smoke fails -> rollback fires` has never executed end to end.

Runs `.github/scripts/smoke-docs.mjs` — the same script `deploy-docs.yml`
runs against the live site, unmodified — with `--base` pointing at a local
`opennextjs-cloudflare preview` of the `.open-next` package the `build` job
already produces on every pull request since #262. No second build, no
Cloudflare credentials.

Readiness is asserted from wrangler's own `Ready on` line before anything is
judged, because "no findings" from a server that never started is
indistinguishable from a rendered site.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ChPQM8jamxLUfUAxwFpJ8S
…ASURED

A crashed preview and an exhausted boot budget call for different fixes — a
broken bundle versus a slow runner — and the log printed underneath is the
same either way. Both demonstrated: with the budget cut to 5s the step reports
the budget ran out; with the compiled OpenNext config removed it reports the
process exited. Both exit 1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ChPQM8jamxLUfUAxwFpJ8S
`setsid` only moves the child into its own group once the forked child has
exec'd it, so reading the group with `ps` immediately after `&` is a race that
can still return the step's own group — and the guard that stops the step from
signalling itself then falls back to signalling the pnpm wrapper alone, which
ignores SIGTERM.

Measured on run 34250422860: the step went green and the runner's own orphan
sweeper had to terminate esbuild and two workerd processes after the job. The
identical code cleaned up completely in this repo's container on every run,
which is how a race presents.

Resolving the group inside the cleanup function removes the window. A preview
that genuinely shares the step's group now says so as a warning instead of
leaving a silent orphan, and the ready line records both group ids.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ChPQM8jamxLUfUAxwFpJ8S
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI can now smoke-check a rendered page BEFORE merge, and nothing wires it up — prevention is still missing from the deploy pipeline

1 participant