From 0476999aa048ef63ac1ab333ee8ca4902f22ad9e Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Sat, 8 Aug 2026 12:23:25 +0530 Subject: [PATCH 01/44] Add ESI cacheable-root validation design for #1009 Validates the ESI approach proposed in #1009 and recommends deferring it. ESI presupposes a TS-owned template cache: its pull-based BufRead input cannot sit downstream of lol_html's push-based rewriter without an intermediate buffer, and the cache boundary is that buffer. That cache is in turn blocked on purge capability the service does not have. Revival condition: React #418 resolved and the window.load gate removed. Re-diagnoses the TTFB regression the issue targets. The auction is dispatched before the origin fetch and does not block, and on a Next.js publisher the closing body tag is not reached until the whole document has been buffered, so the auction hold costs approximately nothing. The cost is with_cache_bypass forcing a readthrough-cache miss on every ad-eligible navigation. Removing either alone recovers little; the two are multiplicative. Corrects nine premises in the issue, including that tsjs.adSlots is per-URL rather than per-user, and that moving identity off the inline response is a prerequisite only for a visitor's first navigation. Carries no performance measurements. Every conclusion is derived from code at the pinned baseline so it can be checked by reading the repository. --- ...08-esi-cacheable-root-validation-design.md | 635 ++++++++++++++++++ 1 file changed, 635 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md diff --git a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md new file mode 100644 index 000000000..4e456e2d0 --- /dev/null +++ b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md @@ -0,0 +1,635 @@ +# ESI and the Cacheable Root + +**Issue:** IABTechLab/trusted-server#1009 · **Date:** 2026-08-08 +**Baseline:** citations verified at `cfb98f4`; unchanged as of `b0ce56c3` (the two +commits between touch only CI workflows and Cargo aliases). + +**Decision requested:** approve the four items below. Three are "yes/no"; one funds +about three days of measurement. + +> **Orientation.** #1009 asks whether Edge Side Includes (ESI) can cache page fragments +> so that cacheable publisher HTML is separated from per-user ad state, recovering a +> TTFB regression that Trusted Server (TS) adds to navigations on a Next.js App Router +> publisher running on Fastly Compute. The answer is no to ESI, and the regression has a +> cheaper cause than the issue assumes. +> +> **This document deliberately carries no performance measurements.** Every conclusion +> below is derived from code at the pinned baseline, so it can be checked by reading the +> repository rather than by trusting a benchmark. Where a quantity is needed and unknown, +> it is named as unknown and [§3](#3-monday-morning) says how to obtain it. +> +> Terms used throughout: **the hold** = TS holding the HTTP response open at `` +> until the server-side auction (SSAT) resolves. **#418** = a React hydration-mismatch +> defect caused by `adInit()` mutating ad-slot subtrees during hydration; it is why bid +> application is deferred to `window.load`. **The SSAT price defect** = a live +> mispricing bug named in #1009 (prices reading 100× high) — cited from #1009 and prior +> investigation, not re-verified here. + +--- + +## 1. Decision requested + +| # | Decision | Owner needed | +| --- | -------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| D1 | **ESI is deferred.** Revival condition: #418 resolved _and_ the `window.load` gate removed. Not a rejection — a dated condition. | Eng + product | +| D2 | **Fund ~3 days of measurement** (§3). No dependencies. Can start immediately. | Eng | +| D3 | **Approve Stage 0** (remove `with_cache_bypass`), subject to the origin-`Vary` check in §3. | Eng | +| D4 | **Stages 1–2 queue behind the SSAT price defect and #418.** Stages 3b–5 unscheduled. | Product | + +Rationale for D4 in [§8](#8-priority). Everything this document recommends _against_ +doing is in [§7](#7-deferred-work-specified-not-scheduled), at deliberately lower +detail than the work it recommends. + +--- + +## 2. Why — the three findings + +**ESI does not work here, for a structural reason #1009 misses.** ESI's input is pull +(`BufRead`); `lol_html`'s is push (`HtmlRewriter::write`). ESI cannot sit downstream of +the rewriter without an intermediate buffer, and in the two-stage design the cache +boundary _is_ that buffer. **ESI presupposes a TS-owned template cache** rather than +being independent of one — and that cache is blocked on purge capability TS does not +have (no `Surrogate-Key` anywhere; the Fastly management token is scoped without purge +permission). ESI is also Fastly-only at every API level. Its one advantage over a +client fetch — no round trip — is worth nothing while bids are not consumed until +`window.load`. Separately, enabling ESI's Dynamic Content Assembly would be an SSRF +vector: bid payloads carry partner-controlled creative markup, so an SSP could embed +`` and make the edge fetch an arbitrary URL. Details in +[Appendix E](#appendix-e--esi-implementation-notes). + +**The auction is already out of band; the hold is ~free.** It is dispatched _before_ +the origin fetch and does not block ([publisher.rs:2698-2701](../../../crates/trusted-server-core/src/publisher.rs#L2698-L2701)), +with a 500 ms budget. The actual cost is `with_cache_bypass` +([publisher.rs:2867](../../../crates/trusted-server-core/src/publisher.rs#L2867)), +which forces every ad-eligible navigation to miss the Fastly readthrough cache. + +**The two fixes are multiplicative.** Removing the bypass alone lets the previously +hidden auction surface as the new bottleneck. Removing the hold alone changes nothing, +because the auction was never the bottleneck. **Shipping the hold removal without the +bypass removal will measure no improvement and will read as the effort having failed** — +the most likely way this work gets judged unfairly. + +**Ordering is established; magnitude is not.** The ordering above follows from code and +needs no measurement. The _size_ of the win does — and the one quantity it depends on, +the origin build time under `Pass`, has never been measured. #1009's timings do not +supply it: they compare cached fetches against each other, not against an origin build. +**Quote no figure to a publisher until §3 Step C runs.** Full reasoning in +[§6](#6-the-analysis). + +--- + +## 3. Monday morning + +Three checks, ordered cheapest-first. Each needs a named owner before starting. + +**Step A — origin `Vary` check (minutes).** `curl` the origin with and without `RSC`, +`Next-Router-*`, and the experiment header; inspect the `Vary` response header. +**Gates Stage 0**, the only build item recommended now. Do this first because it is the +cheapest thing that unblocks anything. + +**Step B — what consumes TS's own response headers (under a day).** Request a TS-served +path that already emits `public, s-maxage` +([http_util.rs:294-311](../../../crates/trusted-server-core/src/http_util.rs#L294-L311)) +twice and look for `x-cache`/`age` on TS's _own_ response. **Gates the Stage 3a/3b +split** — see [§7](#7-deferred-work-specified-not-scheduled). + +**Step C — server-side latency breakdown (1 day + a measurement window).** Emit four +timings per ad-eligible navigation: origin fetch duration (this is `O`, the quantity +the model lacks), auction collect duration, rewrite duration, total. Capture with the +bypass both on and off. + +- **Mechanism: `Server-Timing`.** Chosen, not offered — it needs no new plumbing and is + readable from the same browser harness that produced #1009's numbers. +- **Sample: enough navigations per arm to separate the medians with confidence**, across + both page types, and state the N alongside any result. #1009's sample was small enough + that its conclusion did not survive contact with the code; replacing it with another + underpowered sample would repeat the error. + +**Step C has two outcomes, both actionable:** + +| Outcome | Meaning | Effect on staging | +| -------------------------- | --------------------- | ------------------------------------------------------------ | +| `O` materially exceeds `A` | The model in §6 holds | Proceed as staged: Stage 0 primary, Stage 2 protects its win | +| `A` exceeds `O` | The hold _is_ costing | **Staging inverts** — Stage 2 primary, Stage 0 secondary | + +The work does not change; its order and justification do. **The staging in §7 is +conditional on this measurement.** + +Step C also yields the client fetch latency that sets Stage 1's bids timeout, replacing +an invented constant. + +--- + +## 4. Stage 0 — the only build item recommended now + +Remove `with_cache_bypass` at [publisher.rs:2867](../../../crates/trusted-server-core/src/publisher.rs#L2867). + +**Why it is safe in principle.** The conditional-header strip runs 34 lines earlier +under the same gate ([publisher.rs:2832-2836](../../../crates/trusted-server-core/src/publisher.rs#L2832), +which also strips `Range`/`If-Range`), so the request already reaches the cache +unconditional and a HIT returns a full body. [The 304-prevention design](./2026-07-22-ssat-root-document-304-prevention-design.md) +added the bypass as belt-and-braces and listed the TTFB cost under its own Risks. The +strip alone satisfies its invariant. + +**But it carries a risk that design never considered — and this is the blocking +precondition.** RSC fetches are not navigations +([is_navigation_request](../../../crates/trusted-server-core/src/http_util.rs#L73-L98) +requires `Sec-Fetch-Dest: document`), so they never set the bypass and **already flow +through the readthrough cache**, while HTML navigations are `PASS`. Removing the bypass +puts both representations under one cache key. #1009 states the origin varies on +`rsc`, `next-router-*`, and a publisher-specific experiment header — if that variance is +not declared via `Vary`, the +cache can serve a flight payload to an HTML navigation. + +The classification is also not airtight: `is_navigation_request` falls back to the +`Accept` header when Fetch Metadata is absent, and its own comment warns _"this path is +weaker — `fetch()` can set Accept: text/html"_ +([http_util.rs:84-88](../../../crates/trusted-server-core/src/http_util.rs#L84)). + +**Two effort branches, and Step A decides which:** + +| Step A result | Stage 0 is… | Effort | +| ---------------------- | --------------------------------------------- | ------ | +| Origin declares `Vary` | a one-line deletion plus test updates | 1–2 d | +| Origin does **not** | a TS-side cache-key discriminator — a feature | 4–8 d | + +The discriminator is the safer design either way, because it keys on the headers that +actually distinguish the representations rather than on the navigation classification. + +**Two benefits beyond TTFB, worth stating to a publisher:** + +- **Origin load drops.** The 304-prevention design explicitly accepted _"increasing + origin load"_ as a cost. This reverses it. +- **`stale-if-error` becomes reachable.** Under `Pass` an origin outage is a hard + failure. This needs a decision rather than a default: stale HTML carries stale slot + markup, and whether that beats an error is a product call. + +--- + +## 5. The trap in the deferred work — read this before scheduling Stages 1–2 + +The hold is load-bearing for something other than latency. The invariant is: + +> `ad_bids_state` must be `Some(..)` when `lol_html` processes the `` end tag. + +The end-tag handler ([html_processor.rs:381-395](../../../crates/trusted-server-core/src/html_processor.rs#L381-L395)) +locks that mutex once and falls back to `build_empty_bids_script()` on `None`. + +**Removing the hold without relocating collection renders a normal page with +`tsjs.bids = {}` and no server-side ads** — no error, no non-2xx, no ERROR log. On +Axum, Cloudflare, and Spin the loss is fully silent: +[publisher.rs:2248](../../../crates/trusted-server-core/src/publisher.rs#L2248) holds a +bare `Option` with no guard, so not even a drop warning fires. **The +SSPs are billed regardless.** + +This is why Stage 2 is gated on three companions and a production soak, and why slot +fill cannot be the canary — see [§7](#7-deferred-work-specified-not-scheduled). + +--- + +## 6. The analysis + +### 6.1 Corrections to #1009's premises + +| # | #1009 states | Verified against `cfb98f4` | +| --- | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | Two per-user injection seams | `tsjs.adSlots` is per-URL — [build_slot_json](../../../crates/trusted-server-core/src/publisher.rs#L3501-L3525) emits config- and path-derived fields only. **One per-user hole.** | +| 2 | Identity off-inline is a prerequisite | Privacy net is cookie-gated and returning navs set no cookie ([ec/finalize.rs:86-94](../../../crates/trusted-server-core/src/ec/finalize.rs#L86-L94)). **First-visit only.** | +| 3 | Stamp at `:2882-2888` | [`:2945-2963`](../../../crates/trusted-server-core/src/publisher.rs#L2945-L2963), `private, no-store`, also removing `ETag`/`Last-Modified`/four CDN headers. **Seven headers.** | +| 4 | Three cacheability killers | Two more: `bypass_cache` and the [304→502 guard](../../../crates/trusted-server-core/src/publisher.rs#L2894-L2916). **The bypass is the cost.** | +| 5 | Two `!Send` pipelines is the hard part | `?Send` already pervasive. **Not the obstacle.** | +| 6 | Goal: root as a shared Fastly HIT | Nothing caches TS's own response on Compute; the A/B's `x-cache` is the **backend readthrough** cache. **Reframes the goal.** | + +Rows on #1009's `esi` compatibility check (holds), its drifted line numbers, and its +two broken `#1`/`#3` cross-references are in [Appendix A](#appendix-a--full-1009-correction-table). + +**Credit where due.** #1009 names the hold as blocker 1 and states it correctly. What +changes here is its _causal weight_. Likewise, #1009's own observation that TS _"shifts +the auction cost from client-side to server-side rather than adding new work"_ is the +argument for client-fill, which the issue then declines in favour of ESI. + +### 6.2 Why the hold is free — the strong form first + +On a Next.js publisher, `lol_html` never sees the `` end tag until the **final** +chunk: with any post-processor registered, `HtmlWithPostProcessing` accumulates and +emits nothing before then ([html_processor.rs:62-65](../../../crates/trusted-server-core/src/html_processor.rs#L62)), +and the Next.js integration always registers one when enabled +([nextjs/mod.rs:107](../../../crates/trusted-server-core/src/integrations/nextjs/mod.rs#L107)). + +So the auction has the _entire origin download plus rewrite_ to finish before the hold +can block on anything. **The hold cannot cost anything unless the auction outlives the +whole document fetch.** The auction is bounded by the configured `auction_timeout_ms` +([settings.rs:5000](../../../crates/trusted-server-core/src/settings.rs#L5000)), so this +reduces to a single comparison an operator can check against their own config: is the +auction budget larger than a full document fetch and rewrite? If not, the hold is free. + +**This argument uses no timing data at all** — only the code path and one config value. + +The weaker, general form, for publishers with no post-processor registered: because +dispatch precedes the origin fetch, the hold costs `max(0, A − O)`, which is zero +whenever the origin build `O` exceeds the auction budget `A`. + +### 6.3 The quantity nobody has measured + +Write the origin build time under `Pass` as `O`. Recovery depends on it, and it has +never been captured. #1009's timings cannot supply it: they compare a POP hit against a +shield-served fetch, both of which are _cached_ paths, whereas `CacheOverride::Pass` +bypasses every layer and reaches the true origin. The two are different quantities. + +What follows from code alone, without any number: + +| Configuration | Long pole after the change | Recovery | +| ------------------- | -------------------------- | ------------------------------ | +| Hold removal only | origin (still `PASS`) | **none** | +| Bypass removal only | the auction budget | partial — the auction surfaces | +| **Both** | the rewrite | **the full available win** | + +That ordering is what the staging rests on, and it is measurement-independent. The +magnitude of each row is not, and §3 Step C supplies it. + +### 6.4 The ceiling + +#1009 targets "approach the TS-off warm numbers." **Unreachable, structurally.** Those +numbers are TS-off _streaming_ a POP HIT. TS buffers the whole document before emitting +a byte (16 MB cap), so its floor is `full origin body download + full rewrite` — above a +streamed hit by construction, whatever the timings turn out to be. Set the target from +Step C's measured rewrite cost rather than from the TS-off baseline. Going below the +floor requires true origin streaming (#849), out of scope. A non-Next.js publisher with +no post-processor takes the streaming path and would see a lower floor. + +### 6.5 Confidence + +**High on the structural claims.** §6.2's argument, the bypass forcing a cache miss, the +ESI push/pull mismatch, the silent-empty-bids failure mode, the geo and purge blockers, +and the fill-canary blindness are all read directly out of the code at `cfb98f4`. Anyone +can check them without running anything. + +**None on magnitude.** `O` is unmeasured and the rewrite cost is unmeasured. This +document does not estimate them, and no figure in it should be quoted as one. + +Worth stating plainly: #1009 reached the opposite causal conclusion from a small sample. +That is a caution about small samples generally, not only about that one — which is why +§3 Step C specifies the measurement rather than this document supplying a substitute +for it. + +--- + +## 7. Deferred work, specified not scheduled + +Lower detail is deliberate. Full specifications are in the appendices. + +**Stage 1 — bid delivery off the response body.** The client fetches `/_ts/page-bids` at +navigation generation 0. Endpoint, same-origin gate, wire shape, and client consumer +already exist. Three decisions must be made before planning: the `slots: []` precedence +rule when head-open already injected a non-empty `ts.adSlots`; the new terminal-event +emission point; and whether the dispatch/collect split survives at all. Plumbing detail +in [Appendix B](#appendix-b--stage-1-plumbing). Estimated 8–13 d, low-to-medium +confidence, uncertainty concentrated client-side. + +Three companions are mandatory, not optional: **suppress the server bids script +entirely** (not an empty one), **fail loud** (the end-tag handler takes bids by value so +a missing auction is a compile error), and **relocate telemetry** (navigation +`Completed` rows are emitted only from the collect functions, and the `ts-debug` dump +rides the same string). Behaviour change to accept: under client-fill the auction runs +only if the browser executes the fetch, so bots and JS-disabled clients stop triggering +server-side auctions — revenue-relevant, sign unknown. + +**Stage 2 — delete the hold.** 5–8 d. **Rollback is one-way**: it deletes the hold, the +dispatch/collect split, and twelve tests, so the only revert is a release. Ships only +after Stage 1 has run flag-on in production for a window defined _before_ Stage 1 +starts, with TS-attributed renders flat and `auction_events_raw` navigation rows intact. +Secondary wins: removes the duplicated per-codec decoder/encoder wiring, six compression +imports, and the non-parser-context `` carrying the injection point. + +--- + +## Appendix C — `Vary` signal inventory + +Needed for Stage 3b only. + +**Per-user — never shared-cacheable:** consent state (`euconsent-v2`, `__gpp`, +`__gpp_sid`, `us_privacy`, `Sec-GPC`, IP-derived jurisdiction); the GPT-diagnostics +`__Host-ts-console` cookie / `ts_console` query; the `tsjs.bids` payload (removed by +Stage 1, which is what makes the rest tractable); IP-derived geo; DataDome's request +filter, which can replace the document entirely. + +**Per-variant — safe in a cache key:** request host and scheme; `Accept-Encoding`; +request-class headers (`Sec-Fetch-Dest`, `Accept`, `Sec-Purpose`/`Purpose`, bot UA +fragments, method); the origin `Content-Type` fork (HTML vs `text/x-component` vs plain +URL replacer); the enabled-integration set; the build-time tsjs content hash. + +**Two pre-existing holes, worth filing regardless of this work:** the consent-denied / +bot / prefetch / no-slot variant keeps the origin's cacheability while still carrying +per-user `x-geo-*`; and the RSC-versus-HTML split is unguarded because TS never reads +`RSC` or `Next-Router-*`. + +**Invalidation signals TS has today:** config push changing slots — none; consent change +— request-side, belongs in the cache key; experiment rollover — origin-side only; +article edit — origin `Cache-Control`; tsjs rebuild — content hash, already in the URL; +integration enable/disable — none. + +--- + +## Appendix D — test inventory + +**Coverage gaps to close before Stage 2, not after.** No test exercises the hold +together with post-processors — `streaming_html_with_post_processors_rewrites_body` +(`publisher.rs:7966`) and `document_state_placeholders_substitute_through_accumulating_path` +(`publisher.rs:8043`) both pass `dispatched_auction: None`, which is exactly the Next.js +configuration in question. No parity coverage of the hold or bids injection either; +`parity.rs` has ten multi-adapter tests including `publisher_proxy_fallback_parity` +(`:762`), but none reaches the hold — extend that rather than building a second harness. + +`geo_header_parity_on_all_responses` (`parity.rs:613`) encodes the all-responses +invariant Stage 3b narrows, but currently covers only +`/.well-known/trusted-server.json`, `POST /auction`, and `POST /verify-signature`, and +asserts the boolean `x-geo-info-available` rather than per-user values — so it may need +no change. Check deliberately. + +**Twelve `publisher.rs` tests change with hold removal:** four die (`:5492`, `:5546`, +`:5630`, `:5649`); three assert hold-injected bids (`:6915`, `:6979`, `:7748`); two FCP +guards go vacuous (`:7306`, `:7341`); three are conditional on dispatch/collect +(`:5358`, `:7046`, `:7575`). Dead helpers: `ChunkedReader` (`:4226`), +`RecordingProcessor` (`:4252`). + +`html_processor.rs:1601` and `:1636` pre-populate the mutex directly — they stay green +while production injects empty bids, which is precisely why the fail-loud companion +exists. + +--- + +## Appendix E — ESI implementation notes + +For if and when D1's revival condition is met. + +Pin `esi = "0.7"`; pre-1.0, irregular cadence, two yanked betas in the 0.7 line. + +**Use `process_stream`, not the wrappers.** `process_response` and +`process_response_streaming` consume `self` _and_ send the response themselves, taking +ownership away from the finalize / `ec_finalize` / apply-effects ordering. + +**Order it esi → lol_html**, never the reverse, via a newtype implementing `io::Write` +that forwards to `HtmlRewriter::write`, with `end()` after `process_stream` returns. +Mind the `StreamingBody`-is-a-`BufWriter` hazard already recorded for this repo: esi +flushes after each parse batch, so any adapter in between must propagate `flush()`. + +**Always supply a custom fragment dispatcher.** The built-in one builds a dynamic +backend per URL host and panics on a hostless URL; dynamic backends are also the known +Viceroy local-dev failure mode here. Signature is +`Fn(Request, Option) -> Result` — `Fn`, not `FnMut`, so +captured counters need `Cell`/`RefCell`. Map the maxwait onto the quantized +backend-timeout scheme from #847. Fragment concurrency is free: includes dispatch at +parse time and harvest through one `select()` pool. + +**Streaming mode loses** `$add_header`, `$set_response_code`, `$set_redirect`, and the +auto `Cache-Control` from fragment TTLs — all announced via `println!`, not `log`. + +**Config explicitly:** `with_escaped(false)` for non-HTML templates; `with_chunk_size` +aligned to existing chunking, not the 16 KB default. + +**DCA off, and asserted off.** Defaults are `DcaMode::None` and +`inherit_parent_dca: false`, but set both explicitly — pre-1.0 defaults can move and +this one fails open. Rationale is the SSRF vector in [§2](#2-why--the-three-findings). +`max_include_depth` and `function_recursion_depth` bound the blast radius; they do not +close the hole. + +**Error semantics, non-obvious:** `alt` is attempted before `onerror="continue"` takes +effect; `` runs _all_ attempts in document order and concatenates every +non-failed output — not first-success-wins, so primary/fallback pairs render both; an +include with `onerror="continue"` inside `` never marks that attempt +failed, suppressing `except`. Wrap `ESIError` in `Report<...>` via `change_context()`. + +**Single include, not per-slot.** The auction is one operation producing all slots' +bids; there is no per-slot TTL or partial-failure boundary to exploit. + +**Before committing:** `cargo check-fastly` with `esi` added on Rust 1.95.0 / +`wasm32-wasip1`, and confirm the root lockfile does not desync from the +integration-tests lockfile on shared `regex`, `bytes`, `log`. + +--- + +## Appendix F — deferred open items + +Implementation-level, for unscheduled work only. The decisions that need a human are in +[§9](#9-decisions-needed-from-this-review). + +1. Should `collect_non_html_auction` (`publisher.rs:2388`) be removed with the hold or + kept? It is independently reachable and collects before any byte streams. +2. Is `body_close_hold_loop_stream` (`publisher.rs:2109`, no production caller) safe to + delete, or is the buffered-adapter streaming cutover (#495) still on the roadmap? +3. Does hidden-tab behaviour (rAF unserviced while hidden) interact badly with a bids + timeout that could burn freshness before the rAF fires? +4. Fastly's pending-request semantics when a `DispatchedAuction` drops mid-flight — + unverified; relevant only if dispatch/collect survives. +5. Does `stale-if-error` on a cached root serve acceptable content, given stale HTML + carries stale slot markup? Product call, surfaced by Stage 0. +6. The googletag shim discards listeners queued before it loads, breaking third-party + viewability tooling (#1009 Part 1). Not filed. Should be. + +--- + +## Appendix G — code-grounded seams + +All pinned to `cfb98f4`. + +| Concern | Location | +| ------------------------------------------- | ------------------------------------------------------------------------------- | +| Eligibility decision | `publisher.rs:2651`, `:2660` | +| `is_navigation_request` | `http_util.rs:73-98` | +| Auction dispatch (pre-origin, non-blocking) | `publisher.rs:2698-2760` | +| Auction overlap intent | `auction/orchestrator.rs:950-952` | +| Auction timeout (500 ms) | `settings.rs:5000`; `trusted-server.example.toml:174` | +| Conditional/range header strip | `publisher.rs:2832-2836` | +| Origin cache bypass | `publisher.rs:2866-2868` | +| Origin 304 → 502 guard | `publisher.rs:2894-2916` | +| `adSlots` build (per-URL) | `publisher.rs:2920`, `:3558-3577`, `:3501-3525` | +| Uncacheable stamp | `publisher.rs:2945-2963` | +| `` hold — sync / async / Fastly lazy | `publisher.rs:2235` / `:2109` / `:1318-1390` | +| Hold buffer | `publisher.rs:2177-2218` | +| Auction collect (HTML / non-HTML) | `publisher.rs:2431` (emits `:2456`) / `:2388` (emits `:2410`) | +| Abandonment emitter | `publisher.rs:2360` | +| Bids script build | `publisher.rs:3438-3491` | +| `/_ts/page-bids` | `publisher.rs:3611`, handler `:3723`, auction `:3903` | +| Injection seams (head / body-close) | `html_processor.rs:310-363` / `:381-395` | +| Post-processor buffering | `html_processor.rs:62-94` | +| Next.js post-processor registration | `integrations/nextjs/mod.rs:107` | +| Max buffered body (16 MB) | `settings.rs:77-79` | +| EC cookie issuance policy | `ec/finalize.rs:86-107` | +| Cookie-privacy net | `response_privacy.rs:20-61` | +| Geo response headers | `adapter-fastly/src/middleware.rs:194-200` | +| Cacheable-header precedent | `http_util.rs:294-311` | +| No purge permission | `adapter-fastly/src/management_api.rs:12` | +| Client initial-ad gate | `js/lib/src/integrations/gpt/index.ts:536-555` | +| `adInit` bid application | `js/lib/src/integrations/gpt/index.ts:566`, `:652`, `:657-661` | +| Client SPA auction hook | `js/lib/src/integrations/gpt/index.ts:806`, `:859`, `:892-949` | +| Prior design that introduced the killers | `docs/superpowers/specs/2026-07-22-ssat-root-document-304-prevention-design.md` | From 40de13e50a7ce7dc98de471989893df720019083 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Sat, 8 Aug 2026 15:36:36 +0530 Subject: [PATCH 02/44] Correct the hold-cost argument in the ESI cacheable-root design The strongest claim in the previous revision was wrong. It argued that on a Next.js publisher lol_html never sees the closing body tag until the final chunk, so the auction has the whole download plus rewrite to finish, and concluded that no timing data was needed. The hold does not key off lol_html at all. BodyCloseHoldBuffer::push scans the decoded origin input for the closing tag, and hold_collect_close_tail awaits collect_stream_auction the moment it appears, before post-processing runs. Post-processor buffering is irrelevant to when the hold fires, so the argument applied to every publisher or to none. What survives is the general form: the hold costs max(0, A - T) where T is origin TTFB plus transfer to the closing tag. That needs measurement rather than inference, so Step C now measures the hold directly via hold_wait_ms instead of comparing origin fetch against auction duration through a proxy model. The verdict table follows. Stage 0 becomes an operator flag rather than a code deletion. The risk it gates is cache poisoning, where rollback speed dominates diff size, and a config push reverts in seconds where a release does not. Also: the Vary precondition now covers client Cookie, origin Set-Cookie, and Authorization, which are a larger exposure than the RSC split it previously addressed; a Vary failure is recorded as a live production defect, since RSC fetches already transit the read-through cache; the auction timeout citation pointed at a test fixture rather than the real resolution order; and appendices B, C, E and F are condensed, since they specified work the document recommends against scheduling. --- ...08-esi-cacheable-root-validation-design.md | 425 ++++++++++-------- 1 file changed, 236 insertions(+), 189 deletions(-) diff --git a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md index 4e456e2d0..a2e23b4f9 100644 --- a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md +++ b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md @@ -33,7 +33,7 @@ about three days of measurement. | --- | -------------------------------------------------------------------------------------------------------------------------------- | ------------- | | D1 | **ESI is deferred.** Revival condition: #418 resolved _and_ the `window.load` gate removed. Not a rejection — a dated condition. | Eng + product | | D2 | **Fund ~3 days of measurement** (§3). No dependencies. Can start immediately. | Eng | -| D3 | **Approve Stage 0** (remove `with_cache_bypass`), subject to the origin-`Vary` check in §3. | Eng | +| D3 | **Approve Stage 0** — an operator flag disabling the origin cache bypass, subject to the `Vary` check in §3. | Eng | | D4 | **Stages 1–2 queue behind the SSAT price defect and #418.** Stages 3b–5 unscheduled. | Product | Rationale for D4 in [§8](#8-priority). Everything this document recommends _against_ @@ -55,10 +55,10 @@ client fetch — no round trip — is worth nothing while bids are not consumed `window.load`. Separately, enabling ESI's Dynamic Content Assembly would be an SSRF vector: bid payloads carry partner-controlled creative markup, so an SSP could embed `` and make the edge fetch an arbitrary URL. Details in -[Appendix E](#appendix-e--esi-implementation-notes). +[Appendix E](#appendix-e--esi-notes-condensed). **The auction is already out of band; the hold is ~free.** It is dispatched _before_ -the origin fetch and does not block ([publisher.rs:2698-2701](../../../crates/trusted-server-core/src/publisher.rs#L2698-L2701)), +the origin fetch and does not block — dispatched at [publisher.rs:2751-2755](../../../crates/trusted-server-core/src/publisher.rs#L2751-L2755), sent at [:2870](../../../crates/trusted-server-core/src/publisher.rs#L2870) — with a 500 ms budget. The actual cost is `with_cache_bypass` ([publisher.rs:2867](../../../crates/trusted-server-core/src/publisher.rs#L2867)), which forces every ad-eligible navigation to miss the Fastly readthrough cache. @@ -93,13 +93,37 @@ path that already emits `public, s-maxage` twice and look for `x-cache`/`age` on TS's _own_ response. **Gates the Stage 3a/3b split** — see [§7](#7-deferred-work-specified-not-scheduled). -**Step C — server-side latency breakdown (1 day + a measurement window).** Emit four -timings per ad-eligible navigation: origin fetch duration (this is `O`, the quantity -the model lacks), auction collect duration, rewrite duration, total. Capture with the -bypass both on and off. - -- **Mechanism: `Server-Timing`.** Chosen, not offered — it needs no new plumbing and is - readable from the same browser harness that produced #1009's numbers. +**Step C — measure the hold directly (1 day + a measurement window).** + +The hold's cost is literally the duration of one `.await`: `collect_stream_auction` at +[publisher.rs:793](../../../crates/trusted-server-core/src/publisher.rs#L793), plus the +two EOF variants in `hold_finish_ready_segments` and `hold_finish_tail_segments`. Two +`Instant`s around it yield **`hold_wait_ms`** — the number this entire document is +arguing about, measured rather than modelled. + +Emit two timings per ad-eligible navigation: + +| Metric | Why | +| ----------------- | ---------------------------------------------------------------------- | +| `hold_wait_ms` | **The decision.** How long the response was actually held for bids. | +| `origin_fetch_ms` | Attribution — how much of the win Stage 0 can claim. Origin TTFB only. | + +`hold_wait_ms` replaces the proxy comparison an earlier draft proposed. Comparing `O` +against `A` was an indirect way of asking "does the hold block?"; this asks it directly, +costs less to build, and removes the modelling error corrected in +[§6.2](#62-what-the-hold-actually-costs). + +Deliberately not measured: auction collect duration is already instrumented +(`OrchestrationResult::total_time_ms`, `auction/orchestrator.rs:285`, flowing to +`auction_events_raw`) — read it, don't rebuild it. Rewrite duration decides nothing and +would mean touching two finalizers. + +- **Mechanism: a `log::info!` line behind a debug flag, not `Server-Timing`.** A response + header would in fact work for the origin-fetch figure — that value is known before + headers commit — but a server-side log needs no browser harness to collect it, `log` is + this project's instrumentation crate, and the auction path already measures itself with + `web_time::Instant`. Gate it behind config: one line per eligible navigation is real log + spend and the instrumentation is temporary. - **Sample: enough navigations per arm to separate the medians with confidence**, across both page types, and state the N alongside any result. #1009's sample was small enough that its conclusion did not survive contact with the code; replacing it with another @@ -107,22 +131,54 @@ bypass both on and off. **Step C has two outcomes, both actionable:** -| Outcome | Meaning | Effect on staging | -| -------------------------- | --------------------- | ------------------------------------------------------------ | -| `O` materially exceeds `A` | The model in §6 holds | Proceed as staged: Stage 0 primary, Stage 2 protects its win | -| `A` exceeds `O` | The hold _is_ costing | **Staging inverts** — Stage 2 primary, Stage 0 secondary | +| `hold_wait_ms` median | Meaning | Effect on staging | +| --------------------- | ----------------------- | ------------------------------------------------------------ | +| Near zero | The hold is free | Proceed as staged: Stage 0 primary, Stage 2 protects its win | +| Materially non-zero | The hold **is** costing | **Staging inverts** — Stage 2 primary, Stage 0 secondary | The work does not change; its order and justification do. **The staging in §7 is -conditional on this measurement.** +conditional on this measurement**, and the second outcome is a live possibility rather +than a formality — §6.2's argument for the first is weaker than an earlier draft claimed. -Step C also yields the client fetch latency that sets Stage 1's bids timeout, replacing -an invented constant. +Stage 1's bids-fetch timeout still needs a measured client-side figure rather than an +invented constant, but Step C is server-side and does not supply it. Capture it from the +browser harness when Stage 1 is actually scheduled. --- ## 4. Stage 0 — the only build item recommended now -Remove `with_cache_bypass` at [publisher.rs:2867](../../../crates/trusted-server-core/src/publisher.rs#L2867). +Stop bypassing the read-through cache on ad-eligible navigations +([publisher.rs:2867](../../../crates/trusted-server-core/src/publisher.rs#L2867)). + +**Ship it as an operator flag, not a deletion.** Add +`publisher.bypass_origin_cache`, defaulting to today's behaviour, in the same release as +the Step C instrumentation. Then turn it off with `ts config push`. + +The diff is slightly larger than deleting a line, and that is the point. The risk being +gated here is **cache poisoning** — serving one representation in response to a request +for another. For that class of failure, rollback speed dominates diff size: a config push +reverts in seconds, a release does not. The flag also buys an A/B on a byte-identical +build, removing build difference as a confound in the very measurement this depends on, +and allows flipping for a tester-cookie population before all traffic. + +Retire the flag once the change has held: flip the default, then delete the setting and +its branch. A temporary flag left in place becomes permanent configuration surface. + +### What to watch after the flip + +Two regression signals, both checked before the win is: + +- **`unexpected_origin_304` abandonment rate.** That reason + ([publisher.rs:2894-2916](../../../crates/trusted-server-core/src/publisher.rs#L2894-L2916), + emitted via `emit_abandoned_auction` at `:2360`) exists precisely because the ad-stack + path refuses cached and conditional origin responses. Re-enabling the cache is what + could revive it. **Any non-zero rate is a rollback signal** — it means a 304 is reaching + TS that the conditional-header strip was supposed to make impossible. +- **Representation mixing.** Spot-check that HTML navigations still return HTML and RSC + fetches still return `text/x-component`. A mismatch means the `Vary` risk materialized + despite a PASS verdict. Roll back immediately; this is cache poisoning, not a + performance regression. **Why it is safe in principle.** The conditional-header strip runs 34 lines earlier under the same gate ([publisher.rs:2832-2836](../../../crates/trusted-server-core/src/publisher.rs#L2832), @@ -146,11 +202,35 @@ The classification is also not airtight: `is_navigation_request` falls back to t weaker — `fetch()` can set Accept: text/html"_ ([http_util.rs:84-88](../../../crates/trusted-server-core/src/http_util.rs#L84)). +**A FAIL is not merely a Stage 0 blocker — it is a live production defect.** RSC fetches +already transit the read-through cache today, because they never set the bypass. If the +origin varies on `Next-Router-*` without declaring it, TS is cross-serving RSC variants +right now. On a FAIL, file that immediately and treat "ask the origin to declare `Vary`" +as urgent rather than as the cheaper of two options. + +**The `Vary` check is necessary but not sufficient.** Turning the read-through cache on +for HTML navigations exposes three things a representation check does not cover, and all +three are a larger class than the RSC split: + +- **Client `Cookie`.** TS forwards client cookies to origin unchanged — there is no + `COOKIE` strip on the publisher path. Any cookie-personalized HTML (logged-in state, + paywall meter, publisher-side A/B assignment) becomes cross-servable unless the origin + declares `Vary: Cookie` or marks those responses private. +- **Origin `Set-Cookie`.** If the origin emits `Set-Cookie` alongside a shared-cacheable + `Cache-Control`, the read-through cache can replay one visitor's cookie to the next. + TS's own privacy net downgrades **TS's** response — it runs after the cache has already + stored the origin's. +- **`Authorization`.** #1009 describes a basic-auth-gated deployment. Responses to + authorized requests entering a shared cache needs its own check. + +So Step A must capture `Cache-Control` and `Set-Cookie` too, and repeat each request with +and without a session cookie. Same minutes of work; closes the bigger hole. + **Two effort branches, and Step A decides which:** | Step A result | Stage 0 is… | Effort | | ---------------------- | --------------------------------------------- | ------ | -| Origin declares `Vary` | a one-line deletion plus test updates | 1–2 d | +| Origin declares `Vary` | the flag, its tests, then a config push | 1–2 d | | Origin does **not** | a TS-side cache-key discriminator — a feature | 4–8 d | The discriminator is the safer design either way, because it keys on the headers that @@ -208,33 +288,54 @@ changes here is its _causal weight_. Likewise, #1009's own observation that TS _ the auction cost from client-side to server-side rather than adding new work"_ is the argument for client-fill, which the issue then declines in favour of ESI. -### 6.2 Why the hold is free — the strong form first +### 6.2 What the hold actually costs + +**An earlier draft of this section claimed a stronger argument than the code supports. +It was wrong, and the correction matters.** -On a Next.js publisher, `lol_html` never sees the `` end tag until the **final** -chunk: with any post-processor registered, `HtmlWithPostProcessing` accumulates and -emits nothing before then ([html_processor.rs:62-65](../../../crates/trusted-server-core/src/html_processor.rs#L62)), -and the Next.js integration always registers one when enabled -([nextjs/mod.rs:107](../../../crates/trusted-server-core/src/integrations/nextjs/mod.rs#L107)). +The hold does not key off `lol_html` at all. `BodyCloseHoldBuffer::push` +([publisher.rs:2190-2202](../../../crates/trusted-server-core/src/publisher.rs#L2190)) +scans the **decoded origin input** for ` Dispatch precedes the origin fetch, so the hold costs `max(0, A − T)`, where `A` is the +> auction collect duration and `T` is origin TTFB plus body transfer up to the `` +> byte. Since `` sits at the end of a document, `T` is close to the full download. -The weaker, general form, for publishers with no post-processor registered: because -dispatch precedes the origin fetch, the hold costs `max(0, A − O)`, which is zero -whenever the origin build `O` exceeds the auction budget `A`. +`A` is bounded by `auction_timeout_ms`, resolved as +`creative_opportunities.auction_timeout_ms` falling back to `auction.timeout_ms` +([publisher.rs:2680-2684](../../../crates/trusted-server-core/src/publisher.rs#L2680-L2684)) +— check the resolution order against your own config rather than trusting a number; the +shipped example sets different values at each level. + +**This is a claim requiring measurement, not a proof.** §3 Step C measures the hold's +cost directly rather than inferring it. + +A finding that does survive, and belongs with [the ceiling](#64-the-ceiling): because +`HtmlWithPostProcessing` withholds all output until the final chunk, the streaming-prefix +design at [publisher.rs:1343-1348](../../../crates/trusted-server-core/src/publisher.rs#L1343-L1348) +— whose comment promises "the client receives the document up to `` while the +auction rides alongside transfer" — is **inert on a Next.js publisher**. Every +`step.ready` yields empty bytes. That comment is misleading on exactly the publisher +under discussion. ### 6.3 The quantity nobody has measured -Write the origin build time under `Pass` as `O`. Recovery depends on it, and it has -never been captured. #1009's timings cannot supply it: they compare a POP hit against a +Write the fetch time under `Pass` as `O`. Recovery depends on it, and it has never been +captured. #1009's timings cannot supply it: they compare a POP hit against a shield-served fetch, both of which are _cached_ paths, whereas `CacheOverride::Pass` -bypasses every layer and reaches the true origin. The two are different quantities. +bypasses TS's read-through cache and its shield. + +Note `Pass` bypasses **TS's** caches only. It has no authority over any CDN the publisher +runs in front of their own origin — and #1009's `x-cache: MISS, MISS` on the TS-on arm +hints one may exist. So `O` may not be origin build time at all. Since `O` is the single +quantity this model depends on, that ambiguity is worth resolving in Step C rather than +assuming. What follows from code alone, without any number: @@ -283,7 +384,7 @@ navigation generation 0. Endpoint, same-origin gate, wire shape, and client cons already exist. Three decisions must be made before planning: the `slots: []` precedence rule when head-open already injected a non-empty `ts.adSlots`; the new terminal-event emission point; and whether the dispatch/collect split survives at all. Plumbing detail -in [Appendix B](#appendix-b--stage-1-plumbing). Estimated 8–13 d, low-to-medium +in [Appendix B](#appendix-b--stage-1-plumbing-condensed). Estimated 8–13 d, low-to-medium confidence, uncertainty concentrated client-side. Three companions are mandatory, not optional: **suppress the server bids script @@ -320,7 +421,7 @@ body the document is no longer per-user. Record it as a deliberate decision. which shared-cached replays one visitor's geo to the next; geo is not a request header so suppression is the only option. Second, `Vary`: the publisher path emits none, and at least eleven request signals change the rewritten bytes for one URL — five are per-user -and can never be shared-cached ([Appendix C](#appendix-c--vary-signal-inventory)). +and can never be shared-cached ([Appendix C](#appendix-c--vary-signals-condensed)). **Also gated on Step B**: if nothing consumes TS's response headers, this tier is inert until a topology change. @@ -355,13 +456,18 @@ a path that 404s in a fresh checkout. ## 8. Priority -**Run §3 Steps A–C now, regardless of everything else.** Under three days combined, -useful independent of this effort, and Step C's instrumentation is a permanent -operational asset. +**Run §3 Steps A–C now, regardless of everything else.** Under three days combined, and +useful independent of this effort. Step C's instrumentation is deliberately temporary and +config-gated; if these timings become a standing regression gate, the right home is the +access-log telemetry already scaffolded but unwired in `TinybirdSettings` +(`settings.rs:1718-1731` — `access_enabled`, `access_dataset`, and a sample rate, with the +comment that it is _"rejected until an access-log emitter is wired"_). That is a +follow-on, not part of this work. -**Stage 0 next.** Small, gated on a `curl`, and it reverses an origin-load cost the -prior design explicitly accepted. Closer to a defect fix than an optimization — TS -opted out of a cache it did not need to opt out of. +**Stage 0 next**, shipped as the operator flag in §4 rather than a deletion. Gated on a +`curl`, reverses an origin-load cost the prior design explicitly accepted, and rolls back +with a config push. Closer to a defect fix than an optimization — TS opted out of a cache +it did not need to opt out of. **Stages 1–2 queue behind the correctness defects.** Their failure mode is silent revenue loss, against a publisher whose ads currently fill reliably. The SSAT price @@ -384,7 +490,7 @@ whatever hydration gate lands, and doing that twice is waste. human to make. Implementation-level open items for unscheduled work are in -[Appendix F](#appendix-f--deferred-open-items). +[Appendix F](#appendix-f--deferred-open-items-condensed). --- @@ -400,14 +506,14 @@ Rows 1–6 are in [§6.1](#61-corrections-to-1009s-premises). The remainder: **`should_run_ad_stack` carries four meanings across six sites:** -| Line | Meaning | Disposition | -| --------------------------------------------------------------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------ | -| [2651](../../../crates/trusted-server-core/src/publisher.rs#L2651), `:2660` | eligibility and auction gate | keep | -| [2832-2836](../../../crates/trusted-server-core/src/publisher.rs#L2832) | strip `If-None-Match`, `If-Modified-Since`, `Range`, `If-Range` | **keep** — needed for any injection | -| [2866](../../../crates/trusted-server-core/src/publisher.rs#L2866) | `with_cache_bypass()` | **remove** — [§4](#4-stage-0--the-only-build-item-recommended-now) | -| [2894](../../../crates/trusted-server-core/src/publisher.rs#L2894) | 304 → 502 guard | keep as safety net | -| [2920](../../../crates/trusted-server-core/src/publisher.rs#L2920) | build `adSlots` | keep — per-URL | -| [2945](../../../crates/trusted-server-core/src/publisher.rs#L2945) | strip cacheability | **replace** — Stage 3a | +| Line | Meaning | Disposition | +| --------------------------------------------------------------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------ | +| [2651](../../../crates/trusted-server-core/src/publisher.rs#L2651), `:2660` | eligibility and auction gate | keep | +| [2832-2836](../../../crates/trusted-server-core/src/publisher.rs#L2832) | strip `If-None-Match`, `If-Modified-Since`, `Range`, `If-Range` | **keep** — needed for any injection | +| [2866](../../../crates/trusted-server-core/src/publisher.rs#L2866) | `with_cache_bypass()` | **make operator-controlled** — [§4](#4-stage-0--the-only-build-item-recommended-now) | +| [2894](../../../crates/trusted-server-core/src/publisher.rs#L2894) | 304 → 502 guard | keep as safety net | +| [2920](../../../crates/trusted-server-core/src/publisher.rs#L2920) | build `adSlots` | keep — per-URL | +| [2945](../../../crates/trusted-server-core/src/publisher.rs#L2945) | strip cacheability | **replace** — Stage 3a | **Cache tiers.** T1 backend readthrough (already available; TS opts out). T2 TS-owned template cache (adds KV latency, eventual consistency, a full invalidation design). @@ -416,87 +522,58 @@ Compute is not invoked on a HIT). T2 and T3 both introduce a cache TS cannot pur --- -## Appendix B — Stage 1 plumbing - -All references below are `crates/trusted-server-js/lib/src/integrations/gpt/index.ts` -unless the filename says otherwise. - -**Client plumbing.** `pageBidsEndpoint` (`index.ts:925`), `requestPageBids` -(`index.ts:927-944`), and the `inflight` / `currentPath` / `lastAppliedPath` state -(`index.ts:910`, `:915`, `:921`) are closure-trapped in `installSpaAuctionHook` and must -be hoisted to module scope so one abort domain covers both generations. Do **not** route -the initial load through `onNavigate`: `index.ts:947` no-ops it and `index.ts:949` -increments `navGeneration`, cancelling generation 0 at the guards on `index.ts:538` and -`:541`. - -**Suppress the server bids script.** Today the body end-tag handler is gated on -`has_slots`, which stays true because `adSlots` is still injected — so it would emit -`build_empty_bids_script()`, which calls `scheduleInitialAdInit({})` and assigns -`ts.bids = {}` synchronously at `index.ts:539`. Whether real bids survive would then -depend on unspecified ordering against the client fetch. The gate must become "did this -response carry bids," not "does this page have slots." - -**Gate restructuring — the real work.** `adInit` snapshots bids at call time -(`index.ts:566`) and applies `hb_*` targeting at `index.ts:657-661`; the -`slotRenderEnded` listener's live read at `index.ts:712` serves adm injection only and -cannot retarget a requested slot. **Bids arriving after `adInit` are lost.** -`installScheduleInitialAdInit` becomes a two-condition join — hydration-ready AND -bids-settled — with a bounded timeout that fires `adInit` untargeted rather than -stranding the slot. Derive the timeout from §3 Step C; `SPA_SLOT_WAIT_MS = 2000` is -precedent but was derived for DOM readiness, not network. This composes badly with the -958 branch's poll-and-grace gate — two timeout budgets in series. - -**Server contract.** `handle_page_bids` runs a fresh `run_auction` -(`publisher.rs:3903`) tagged `AuctionSource::SpaNavigation` (`publisher.rs:3859`). Not a -drop-in — it cannot reuse in-flight dispatched requests. Required: navigation-path -dispatch **suppressed** (running both doubles SSP/APS spend); a new `AuctionSource` for -initial loads **plus the mechanism that delivers it** — a request header alongside the -existing `X-TSJS-Page-Bids` marker, behind the same-origin gate, or it becomes a -caller-controlled telemetry-poisoning knob; and the `slots: []` precedence rule -(`publisher.rs:3975-3985`). - -**Telemetry.** Navigation `Completed` is emitted only from the two collect functions -(`publisher.rs:2410`, `:2456`); `Abandoned` only via `emit_abandoned_auction` -(`publisher.rs:2360`) across nine reasons, three of which live only inside the hold -helpers. The `[debug].auction_html_comment` `ts-debug` dump prepends onto the same -`ad_bids_state` string inside the collect function (`publisher.rs:2478`) and disappears -with the hold — relocate or retire deliberately. - -**Sequencing.** (a) decide bid delivery, (b) decide whether dispatch/collect survives, -(c) delete the hold. Doing (c) first produces the silent failure in [§5](#5-the-trap-in-the-deferred-work--read-this-before-scheduling-stages-12). -Leaving `OwnedProcessResponseParams` (`publisher.rs:1072-1086`) carrying five unread -auction fields is exactly that configuration. - -**Preserve the multi-member gzip guarantee.** `GzipDecodeReader` exists because -`flate2::read::GzDecoder` drops every member after the first, which can drop the -`` carrying the injection point. +## Appendix B — Stage 1 plumbing (condensed) + +Full detail lives in the plan when Stage 1 is scheduled. The decisions that must be made +before any of it is written: + +- **Suppress the server bids script entirely** under client-fill, not emit an empty one. + The body end-tag handler is gated on `has_slots`, which stays true; the gate must become + "did this response carry bids." +- **`adInit` snapshots bids at call time** (`gpt/index.ts:566`) and applies `hb_*` + targeting at `:657-661`. Bids arriving later are lost, so + `installScheduleInitialAdInit` must become a hydration-ready AND bids-settled join with + a bounded timeout that fires untargeted rather than stranding the slot. +- **Do not route the initial load through `onNavigate`** — `gpt/index.ts:949` increments + `navGeneration` and cancels generation 0. +- **`handle_page_bids` is not a drop-in.** It runs a fresh `run_auction` + (`publisher.rs:3903`) tagged `AuctionSource::SpaNavigation` (`:3859`) and cannot reuse + in-flight dispatched requests. Needs dispatch suppression (or spend doubles), a new + `AuctionSource` **plus the mechanism that delivers it**, and a `slots: []` precedence + rule (`:3975-3985`). +- **Telemetry moves with it.** Navigation `Completed` is emitted only from the collect + functions (`publisher.rs:2410`, `:2456`); `Abandoned` only via `emit_abandoned_auction` + (`:2360`). The `ts-debug` dump rides the same `ad_bids_state` string (`:2478`) and + disappears with the hold. +- **Sequencing is strict:** decide bid delivery, then whether dispatch/collect survives, + then delete the hold. Any other order produces the silent failure in [§5](#5-the-trap-in-the-deferred-work--read-this-before-scheduling-stages-12). --- -## Appendix C — `Vary` signal inventory +## Appendix C — `Vary` signals (condensed) -Needed for Stage 3b only. +Needed for Stage 3b only, which is unscheduled. -**Per-user — never shared-cacheable:** consent state (`euconsent-v2`, `__gpp`, +At least eleven request signals change the rewritten bytes for one URL. **Five are +per-user and can never be shared-cached:** consent state (`euconsent-v2`, `__gpp`, `__gpp_sid`, `us_privacy`, `Sec-GPC`, IP-derived jurisdiction); the GPT-diagnostics -`__Host-ts-console` cookie / `ts_console` query; the `tsjs.bids` payload (removed by -Stage 1, which is what makes the rest tractable); IP-derived geo; DataDome's request -filter, which can replace the document entirely. +`__Host-ts-console` cookie; the `tsjs.bids` payload (removed by Stage 1, which is what +makes the rest tractable); IP-derived geo; and DataDome's request filter, which can +replace the document entirely. -**Per-variant — safe in a cache key:** request host and scheme; `Accept-Encoding`; -request-class headers (`Sec-Fetch-Dest`, `Accept`, `Sec-Purpose`/`Purpose`, bot UA -fragments, method); the origin `Content-Type` fork (HTML vs `text/x-component` vs plain -URL replacer); the enabled-integration set; the build-time tsjs content hash. +**Six are per-variant and safe in a cache key:** request host and scheme, +`Accept-Encoding`, request-class headers, the origin `Content-Type` fork, the enabled +integration set, and the tsjs content hash. -**Two pre-existing holes, worth filing regardless of this work:** the consent-denied / -bot / prefetch / no-slot variant keeps the origin's cacheability while still carrying -per-user `x-geo-*`; and the RSC-versus-HTML split is unguarded because TS never reads -`RSC` or `Next-Router-*`. +Two pre-existing holes worth filing regardless of this work: the consent-denied / bot / +prefetch / no-slot variant keeps the origin's cacheability while carrying per-user +`x-geo-*`; and the RSC-versus-HTML split is unguarded because TS never reads `RSC` or +`Next-Router-*`. -**Invalidation signals TS has today:** config push changing slots — none; consent change -— request-side, belongs in the cache key; experiment rollover — origin-side only; -article edit — origin `Cache-Control`; tsjs rebuild — content hash, already in the URL; -integration enable/disable — none. +Invalidation signals TS has today: config push — none; consent change — request-side, +belongs in the cache key; experiment rollover — origin-side only; article edit — origin +`Cache-Control`; tsjs rebuild — content hash already in the URL; integration toggle — +none. --- @@ -528,73 +605,43 @@ exists. --- -## Appendix E — ESI implementation notes - -For if and when D1's revival condition is met. - -Pin `esi = "0.7"`; pre-1.0, irregular cadence, two yanked betas in the 0.7 line. - -**Use `process_stream`, not the wrappers.** `process_response` and -`process_response_streaming` consume `self` _and_ send the response themselves, taking -ownership away from the finalize / `ec_finalize` / apply-effects ordering. - -**Order it esi → lol_html**, never the reverse, via a newtype implementing `io::Write` -that forwards to `HtmlRewriter::write`, with `end()` after `process_stream` returns. -Mind the `StreamingBody`-is-a-`BufWriter` hazard already recorded for this repo: esi -flushes after each parse batch, so any adapter in between must propagate `flush()`. - -**Always supply a custom fragment dispatcher.** The built-in one builds a dynamic -backend per URL host and panics on a hostless URL; dynamic backends are also the known -Viceroy local-dev failure mode here. Signature is -`Fn(Request, Option) -> Result` — `Fn`, not `FnMut`, so -captured counters need `Cell`/`RefCell`. Map the maxwait onto the quantized -backend-timeout scheme from #847. Fragment concurrency is free: includes dispatch at -parse time and harvest through one `select()` pool. - -**Streaming mode loses** `$add_header`, `$set_response_code`, `$set_redirect`, and the -auto `Cache-Control` from fragment TTLs — all announced via `println!`, not `log`. - -**Config explicitly:** `with_escaped(false)` for non-HTML templates; `with_chunk_size` -aligned to existing chunking, not the 16 KB default. - -**DCA off, and asserted off.** Defaults are `DcaMode::None` and -`inherit_parent_dca: false`, but set both explicitly — pre-1.0 defaults can move and -this one fails open. Rationale is the SSRF vector in [§2](#2-why--the-three-findings). -`max_include_depth` and `function_recursion_depth` bound the blast radius; they do not -close the hole. - -**Error semantics, non-obvious:** `alt` is attempted before `onerror="continue"` takes -effect; `` runs _all_ attempts in document order and concatenates every -non-failed output — not first-success-wins, so primary/fallback pairs render both; an -include with `onerror="continue"` inside `` never marks that attempt -failed, suppressing `except`. Wrap `ESIError` in `Report<...>` via `change_context()`. - -**Single include, not per-slot.** The auction is one operation producing all slots' -bids; there is no per-slot TTL or partial-failure boundary to exploit. - -**Before committing:** `cargo check-fastly` with `esi` added on Rust 1.95.0 / -`wasm32-wasip1`, and confirm the root lockfile does not desync from the -integration-tests lockfile on shared `regex`, `bytes`, `log`. +## Appendix E — ESI notes (condensed) + +For if and when [D1](#1-decision-requested)'s revival condition is met. Expand then; +recording only what would otherwise be re-derived: + +- Pin `esi = "0.7"`. Pre-1.0, irregular cadence, two yanked betas in the 0.7 line. +- **Use `process_stream`, not the wrappers.** `process_response` and + `process_response_streaming` consume `self` _and_ send the response themselves, taking + ownership away from the finalize / `ec_finalize` ordering. +- **Order esi → lol_html**, never the reverse, via a newtype implementing `io::Write`. + Mind the `StreamingBody`-is-a-`BufWriter` hazard: esi flushes per parse batch, so any + adapter in between must propagate `flush()`. +- **Always supply a custom fragment dispatcher.** The built-in one builds a dynamic + backend per URL host and panics on a hostless URL; dynamic backends are also the known + Viceroy local-dev failure mode here. +- **DCA off, asserted explicitly** — not merely left at its default. Rationale is the SSRF + vector in [§2](#2-why--the-three-findings): partner-controlled creative markup would + become ESI-executable at the edge. +- **`` runs _all_ attempts and concatenates every non-failed output** — not + first-success-wins, so primary/fallback pairs render both. Least obvious behaviour in + the crate. +- Single include, not per-slot: the auction is one operation producing all slots' bids. --- -## Appendix F — deferred open items +## Appendix F — deferred open items (condensed) -Implementation-level, for unscheduled work only. The decisions that need a human are in +Implementation-level, for unscheduled work only. Decisions needing a human are in [§9](#9-decisions-needed-from-this-review). -1. Should `collect_non_html_auction` (`publisher.rs:2388`) be removed with the hold or - kept? It is independently reachable and collects before any byte streams. -2. Is `body_close_hold_loop_stream` (`publisher.rs:2109`, no production caller) safe to - delete, or is the buffered-adapter streaming cutover (#495) still on the roadmap? -3. Does hidden-tab behaviour (rAF unserviced while hidden) interact badly with a bids - timeout that could burn freshness before the rAF fires? -4. Fastly's pending-request semantics when a `DispatchedAuction` drops mid-flight — - unverified; relevant only if dispatch/collect survives. -5. Does `stale-if-error` on a cached root serve acceptable content, given stale HTML - carries stale slot markup? Product call, surfaced by Stage 0. -6. The googletag shim discards listeners queued before it loads, breaking third-party - viewability tooling (#1009 Part 1). Not filed. Should be. +Should `collect_non_html_auction` (`publisher.rs:2388`) go with the hold or stay? Is +`body_close_hold_loop_stream` (`:2109`, no production caller) safe to delete, or is the +buffered-adapter streaming cutover (#495) still live? Does hidden-tab rAF behaviour +interact badly with a bids timeout? What are Fastly's pending-request semantics when a +`DispatchedAuction` drops mid-flight? Does `stale-if-error` on a cached root serve +acceptable content given stale slot markup? And the googletag shim discards listeners +queued before it loads (#1009 Part 1) — not filed, should be. --- @@ -608,7 +655,7 @@ All pinned to `cfb98f4`. | `is_navigation_request` | `http_util.rs:73-98` | | Auction dispatch (pre-origin, non-blocking) | `publisher.rs:2698-2760` | | Auction overlap intent | `auction/orchestrator.rs:950-952` | -| Auction timeout (500 ms) | `settings.rs:5000`; `trusted-server.example.toml:174` | +| Auction timeout resolution | `publisher.rs:2680-2684` (creative_opportunities, else auction.timeout_ms) | | Conditional/range header strip | `publisher.rs:2832-2836` | | Origin cache bypass | `publisher.rs:2866-2868` | | Origin 304 → 502 guard | `publisher.rs:2894-2916` | From f15eaba7523aa640b78f2d0f533dc1e1543d4d10 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Sat, 8 Aug 2026 15:37:29 +0530 Subject: [PATCH 03/44] Add the measurement and Stage 0 implementation plan for #1009 Covers the spec's Steps A/B/C plus Stage 0. Stages 1-5 are out of scope and named as such, since the spec queues them behind the open correctness defects. Three investigations and one code task. Step A curls the origin for its Vary declaration and for cookie personalization, and gates everything downstream. Step B settles whether anything caches the service's own response by inspecting the Fastly topology rather than probing for an age header, and asks whether the publisher backend is shielded, which sizes the win and nothing else in the plan establishes. Step C instruments hold_wait_ms and origin_fetch_ms. The instrumentation goes in collect_stream_auction rather than at its three call sites. All three reach it, and it already destructures settings out of AuctionCollectDeps, so one edit covers every adapter with no new plumbing. The plan names hold_finish_ready_segments and hold_finish_tail_segments explicitly as sites not to instrument: neither awaits the collect, and doing so would double-count. Stage 0 ships as publisher.bypass_origin_cache defaulting to today's behaviour, then flips by config push. Adding that field breaks nine sites the diff does not suggest, including a live doctest, so they are enumerated. The win is measured client-side through the existing tester-cookie harness; origin_fetch_ms is TTFB only and is attribution, not outcome. Records two gotchas hit while writing it: prettier is not idempotent on markdown containing fenced markdown blocks, and it rewrites bare snake_case identifiers inside them as emphasis. Both fail CI gate 7. --- ...2026-08-08-1009-measurement-and-stage-0.md | 954 ++++++++++++++++++ 1 file changed, 954 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md new file mode 100644 index 000000000..c5f85aa4c --- /dev/null +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md @@ -0,0 +1,954 @@ +# #1009 Measurement and Stage 0 Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Obtain the checks that gate the #1009 work, then turn off the redundant origin +cache bypass that the spec identifies as the actual TTFB cost — behind an operator flag, +so it rolls back with a config push rather than a release. + +**Architecture:** Two investigation tasks that produce recorded findings and no code; one +code task that adds a config-gated timing log and makes the cache bypass operator- +controlled; and one config change that flips it, gated on the first investigation. +Nothing here touches the auction, the `` hold, or bid delivery — those are +Stages 1–2 in the spec and are explicitly out of scope. + +**Tech Stack:** Rust 2024 edition, `wasm32-wasip1`, Fastly Compute, `web_time::Instant` +for wasm-safe timing, `log` for instrumentation, Viceroy for adapter tests. + +**Spec:** `docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md` +(§3 Steps A/B/C and §4 Stage 0). Read §4 and §5 before starting Task 5. + +**Two prettier gotchas, both hit while writing this plan.** CI gate 7 +(`cd docs && npm run format`) runs `prettier --check` over all of `docs/`, so both bite. + +1. **Not idempotent on embedded markdown fences.** The first `--write` reformats the + outer document and the embedded ` ```markdown ` block only settles on a second pass. + If `--check` still warns immediately after a `--write`, run `--write` again before + concluding anything is wrong. +2. **It mangles bare `snake_case` identifiers inside fences**, reading the underscores as + emphasis and rewriting `origin_fetch_ms` to `origin*fetch_ms`. **Always wrap + identifiers in backticks**, including inside fenced blocks and table cells. + +--- + +## Background an implementer needs + +Trusted Server proxies a publisher's origin, rewrites the HTML at the edge to inject ad +slot definitions and a JS bundle, and runs a server-side ad auction. For requests that +are eligible for that ad stack, `publisher.rs` currently does three things to the origin +request and response that together make the page uncacheable: + +1. strips conditional and range headers so the origin must return a full body, +2. sets a **cache bypass** so the Fastly read-through cache is skipped entirely, and +3. strips every cacheability header from the response. + +The spec establishes that (2) is redundant given (1) — by the time the request reaches +the cache it is already unconditional, so a cache HIT returns a full body anyway — and +that (2) is the dominant cost. This plan makes (2) operator-controlled and then turns it +off, after first confirming that is safe. + +**Why it might not be safe:** RSC (React Server Component) requests and ordinary HTML +navigations share the same URL and are distinguished only by request headers. RSC +requests are not classified as navigations, so they already flow through the cache while +HTML navigations bypass it. Removing the bypass puts both under one cache key. If the +origin does not declare `Vary` for those headers, the cache could serve one +representation in response to a request for the other. Task 1 checks this. + +**Terms:** _POP_ = Fastly edge point of presence. _shield_ = a designated POP that +backs other POPs. _read-through cache_ = Fastly's cache on the backend request path. +_bypass / `Pass`_ = skip that cache. + +--- + +## File structure + +| File | Responsibility in this plan | +| ---------------------------------------------------------------- | ---------------------------------------------------------------------------------- | +| `docs/superpowers/plans/2026-08-08-1009-measurement-findings.md` | **Create.** Recorded output of Tasks 1–2. Gates Task 5. | +| `crates/trusted-server-core/src/publisher.rs` | **Modify.** Timing log and the bypass flag (Task 3); tests (Task 5). | +| `crates/trusted-server-core/src/settings.rs` | **Modify.** `publisher.bypass_origin_cache` and `debug.publisher_timing` (Task 3). | +| `trusted-server.example.toml` | **Modify.** Document the new key (Task 5). | + +No new modules. No adapter changes: the `bypass_cache` platform capability and its +per-adapter mappings stay in place and keep their tests — the publisher-path call site +becomes operator-controlled rather than unconditional. + +## Task order and dependencies + +Only one edge is real. Do not serialize the rest. + +``` +Task 1 (origin Vary check) ──────┬──> Task 2 (appends to the findings file Task 1 creates) + │ + ├──> Task 5 (flip the flag) +Task 3 (instrumentation + flag) ─┘ +``` + +**Task 1 is externally blocked.** It needs the publisher origin hostname, which lives in +the operator's gitignored `trusted-server.toml`. Arrange access before starting, or the +plan stalls on its first step. + +Task 3 is independent and can start immediately. Task 2 only needs Task 1 far enough to +have created the findings document. Task 5 needs Task 1's verdict **and** Task 3's config +flag to exist. + +--- + +## Task 1: Step A — origin `Vary` check + +**Files:** + +- Create: `docs/superpowers/plans/2026-08-08-1009-measurement-findings.md` + +This task is an investigation. It writes no code and gates Task 5. + +- [ ] **Step 1: Get the origin URL** + +The publisher origin is operator config, not in the repo. Read it from the deployed +service config or ask the operator. Do **not** hardcode it into any committed file — the +findings document records the _result_, not the hostname. + +```bash +# The key is `publisher.origin_url` in the operator's trusted-server.toml +# (gitignored). Confirm the value before proceeding. +``` + +- [ ] **Step 2: Request the HTML representation and capture `Vary`** + +```bash +curl -sS -D - -o /dev/null "https:///" \ + -H 'Sec-Fetch-Dest: document' \ + -H 'Accept: text/html' +``` + +Expected: response headers. Record whether a `Vary` header is present and its value. + +- [ ] **Step 3: Request the RSC representation at the same URL** + +```bash +curl -sS -D - -o /dev/null "https:///" \ + -H 'RSC: 1' \ + -H 'Accept: text/x-component' +``` + +Expected: a different `Content-Type` (`text/x-component`) than Step 2, proving the two +representations share a URL. Record `Vary` again. + +- [ ] **Step 4: Probe the `Next-Router-*` headers** + +Do not skip this. The PASS criterion below names these headers, and an implementer who +tests only HTML and `RSC` can record a PASS that is wrong — which routes to Task 5a, the +one outcome this plan calls dangerous. + +```bash +for H in 'Next-Router-Prefetch: 1' 'Next-Router-State-Tree: %5B%22%22%5D'; do + echo "--- $H" + curl -sS -D - -o /dev/null "https:///" \ + -H 'RSC: 1' -H "$H" \ + | grep -iE '^(vary|content-type|content-length|cache-control|set-cookie):' +done +``` + +Compare `Content-Type` and `Content-Length` against the plain `RSC: 1` request from +Step 3. If either differs, the origin varies on that header and `Vary` must name it. + +Capture `Cache-Control` and `Set-Cookie` on every request in this task, not just this +one — see Step 5. + +- [ ] **Step 5: Probe cookie personalization — the bigger hole** + +The representation check above covers RSC-vs-HTML. It does **not** cover the larger +class: TS forwards client cookies to origin unchanged, so any cookie-personalized HTML +(logged-in state, paywall meter, publisher-side A/B assignment) becomes cross-servable +once the cache is on. + +```bash +# Same URL, with and without a session cookie. Compare Content-Length and body hash. +for C in '' 'Cookie: '; do + echo "--- ${C:-no-cookie}" + curl -sS -D /dev/stderr -o - "https:///" \ + -H 'Sec-Fetch-Dest: document' -H 'Accept: text/html' ${C:+-H "$C"} \ + 2> >(grep -iE '^(vary|cache-control|set-cookie|content-length):' >&2) \ + | shasum +done +``` + +Three failure shapes, any of which blocks Stage 0 independently of the `Vary` verdict: + +- Bodies differ by cookie **and** `Vary` does not name `Cookie` → cross-serving of + personalized HTML. +- Origin emits `Set-Cookie` alongside a shared-cacheable `Cache-Control` → the cache can + replay one visitor's cookie to the next. TS's privacy net does not help; it downgrades + **TS's** response, after the cache has already stored the origin's. +- The deployment is `Authorization`-gated (as #1009 describes) and authorized responses + are cacheable → same problem, different header. + +- [ ] **Step 6: Request with the experiment header, if the operator uses one** + +Repeat Step 2 with the publisher's experiment header set to two different values. +Record whether the bodies differ and whether `Vary` names that header. + +- [ ] **Step 7: Record the finding** + +Create `docs/superpowers/plans/2026-08-08-1009-measurement-findings.md`: + +```markdown +# #1009 measurement findings + +## Step A — origin `Vary` declaration + +**Date:** · **Checked by:** + +| Representation | `Content-Type` returned | `Content-Length` | `Vary` present? | `Vary` value | +| --------------------- | ----------------------- | ---------------- | --------------- | ------------ | +| HTML navigation | | | | | +| RSC | | | | | +| RSC + `Next-Router-*` | | | | | +| Experiment variant | | | | | + +**Cookie / auth exposure:** bodies differ by cookie? `Vary: Cookie` present? origin +`Set-Cookie` on a shared-cacheable response? `Authorization`-gated responses cacheable? + +**Verdict:** PASS / FAIL + +PASS = `Vary` names every request header the origin varies on (`RSC`, any `Next-Router-*` +or experiment header whose value changed the body, **and `Cookie` if bodies differ by +cookie**), and no `Set-Cookie` rides a shared-cacheable response. +FAIL = any of the above is unmet. + +**Consequence:** PASS → Task 5a (flip the flag). FAIL → Task 5b (cache-key +discriminator). See spec §4. + +**A FAIL is also a live production defect, not only a Stage 0 blocker.** RSC fetches are +not navigations, so they never set the bypass and **already transit the read-through +cache today**. If the origin varies undeclared on `Next-Router-*`, TS is cross-serving RSC +variants in production right now. File it immediately rather than deferring with Task 5b. +``` + +- [ ] **Step 8: Commit** + +CI gate 7 runs `prettier --check` across all of `docs/`, so format the findings file +before staging it — a filled-in markdown table will not be prettier-clean by hand. + +```bash +cd docs && npx prettier --write superpowers/plans/2026-08-08-1009-measurement-findings.md && cd .. +git add docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +git commit -m "Record origin Vary findings for #1009 Stage 0 gate" +``` + +--- + +## Task 2: Step B — what consumes TS's own response headers + +**Files:** + +- Modify: `docs/superpowers/plans/2026-08-08-1009-measurement-findings.md` — **created by + Task 1 Step 6.** If Task 1 has not reached that step, create the file with just its + `# #1009 measurement findings` heading rather than blocking. + +Investigation. Determines whether the spec's Stage 3b has a consumer. Does not gate +Task 5, but it appends to Task 1's findings document — do not run the two concurrently +against that file. + +- [ ] **Step 1: Pick a path that already emits shared-cache headers** + +`serve_static_with_etag` emits `public, max-age=300, s-maxage=300` plus +`Surrogate-Control` — see `crates/trusted-server-core/src/http_util.rs:294-311`. It backs +the `/static/tsjs=` bundle route (`publisher.rs:303`, `:322`). Use that URL against +the deployed service. + +- [ ] **Step 2: Request it twice and inspect for cache markers** + +```bash +URL="https:///static/tsjs=" +curl -sS -D - -o /dev/null "$URL" | grep -iE 'x-cache|age:|x-served-by|hit-state' +sleep 2 +curl -sS -D - -o /dev/null "$URL" | grep -iE 'x-cache|age:|x-served-by|hit-state' +``` + +Expected on the second request: if a cache sits in front of the Compute service, an `age` +greater than zero or an `x-cache` containing `HIT`. + +**The probe above is weak evidence** — absence of `age` is equally consistent with "no +cache" and "cold cache". **The topology check below is the actual answer; run it first and +skip the probe if it is conclusive.** + +```bash +fastly service list +fastly service-version list --service-id +# Look for a Delivery service fronting the Compute service, and for shielding +# configured on the service rather than only on the origin backend. +``` + +A Compute service with no Delivery service in front and no fronting shield does not have +its own output cached — that is the configuration the spec assumes, and this step exists +to confirm or refute it rather than to leave it assumed. + +**While you have the service open, answer a second question that matters more than this +task does:** is the _publisher backend_ shielded on the TS service? + +```bash +fastly backend list --service-id --version active +# Look for a shield on the publisher origin backend. +``` + +#1009's entire off-TS advantage came from a **shield** HIT, not a POP HIT. Whether +Stage 0 recovers a shield HIT or only a single-POP HIT changes the size of the win +materially, and nothing else in this plan establishes it. + +- [ ] **Step 3: Record the finding** + +Append to the findings document: + +```markdown +## Step B — consumers of TS's own response headers + +**Verdict:** SHARED CACHE PRESENT / NO SHARED CACHE + +**Evidence:** + +**Consequence:** NO SHARED CACHE → spec Stage 3b is inert until a topology change; +deprioritize it and ship only Stage 3a (browser caching). SHARED CACHE PRESENT → +Stage 3b gains a consumer AND the per-user `x-geo-*` header leak in spec §7 becomes an +active privacy exposure rather than a theoretical one. Escalate immediately in that case. +``` + +- [ ] **Step 4: Commit** + +```bash +cd docs && npx prettier --write superpowers/plans/2026-08-08-1009-measurement-findings.md && cd .. +git add docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +git commit -m "Record response-header cache consumer findings for #1009" +``` + +--- + +## Task 3: Step C — origin fetch timing, and the bypass flag + +**Files:** + +- Modify: `crates/trusted-server-core/src/publisher.rs` +- Modify: `crates/trusted-server-core/src/settings.rs` — `publisher.bypass_origin_cache`, + `default_bypass_origin_cache`, `debug.publisher_timing`, the `Publisher` `Default` impl, + eight test literals, and the `origin_host` doctest +- Modify: `crates/trusted-server-core/src/test_support.rs` (log-capture helper) +- Test: inside `mod ssat_cache_policy_tests` at `crates/trusted-server-core/src/publisher.rs:4541` + +**Use `web_time::Instant`, not `std::time::Instant`** — the workspace targets +`wasm32-wasip1` and `web_time` is the wasm-safe clock already used at +`crates/trusted-server-core/src/auction/orchestrator.rs:7`. + +### Two timings, and why these two + +Measure **`hold_wait_ms`** and **`origin_fetch_ms`**. Not the rewrite. + +`hold_wait_ms` is the decision. The hold's cost is literally the duration of one +`.await` — `collect_stream_auction` at `publisher.rs:793`, plus the two EOF variants in +`hold_finish_ready_segments` (`:869`) and `hold_finish_tail_segments` (`:896`). Two +`Instant`s around those calls answer "does the hold block?" directly, instead of +inferring it by comparing origin fetch against auction duration. + +`origin_fetch_ms` is attribution — how much of any win Stage 0 can claim. + +`rewrite_ms` decides nothing. Step C's verdict compares origin fetch against auction +collect, and the ceiling argument in spec §6.4 is structural — it needs no number. +Measuring the rewrite would mean instrumenting two finalizers +(`buffer_publisher_response_async` at `publisher.rs:1114`, and the +`async_stream::try_stream!` block at `publisher.rs:1286`), working around moves out of +`params` inside that block, and finding a correlation key that does not exist — +`OwnedProcessResponseParams` (`publisher.rs:1065-1087`) has no `request_path`, and adding +one means touching all 26 construction sites. + +None of that buys a decision. Skip it. If a rewrite figure is later wanted to set a +target, add it as a separate follow-on once the verdict is known. + +**Why a log line and not `Server-Timing`:** for `origin_fetch_ms` alone a response header +would in fact work — the value is known before headers commit. A log line is still +preferred because it is server-side (no dependence on a browser harness to collect it), +`log` is this project's instrumentation crate per `CLAUDE.md`, and the auction path +already measures itself the same way. The spec previously claimed `Server-Timing` cannot +work at all; that overbroad claim has already been corrected there. + +### Log volume — gate it + +The line sits after the origin send, so it fires for every publisher request that reaches +origin — tagged `ad_stack=false` for ineligible ones, not only for eligible navigations. +That is more useful for comparison and more log spend, and the instrumentation is +temporary either way. Gate it behind the existing debug surface rather than +emitting unconditionally: add a `#[serde(default)] pub publisher_timing: bool` to +`DebugConfig` (`crates/trusted-server-core/src/settings.rs:1872`), following +`ja4_endpoint_enabled` and `auction_html_comment` alongside it. Default `false`; enable +via `ts config push` for the measurement window, then disable. + +This also means the Step 1 test must set that flag in its settings fixture. + +The split is also what makes the Step 1 test achievable — `run_with_slots` +(`publisher.rs:4769`) invokes only `handle_publisher_request` and never drives either +finalizer, so a test asserting on a combined line could never pass. + +**What `origin_fetch_ms` actually measures.** `publisher.rs:2863-2865` sets +`.with_stream_response()` when the adapter supports it, so on Fastly `send()` returns at +response _headers_, not after the body downloads. `origin_fetch_ms` is therefore **origin +TTFB**, not full download time. Name it that way in the findings document. It is still +the correct before/after signal for Stage 0 — the bypass affects whether the request hits +a cache at all — but when comparing against auction `total_time_ms` in Step 9, compare +like with like and say which quantity each column holds. + +- [ ] **Step 1: Write the failing test** + +**Placement matters.** Add the test **inside `mod ssat_cache_policy_tests`** +(`publisher.rs:4541`), not the outer `mod tests` (`:4035`). Every helper it uses is +private to that nested module: `settings_with_enabled_auction_and_creative_opportunities` +(`:4684`), `article_slot` (`:4721`), `conditional_navigation_request` (`:4740`), +`queue_cacheable_html_response` (`:4752`), `run_with_slots` (`:4769`). Placed in the outer +module it will not resolve — and because two _other_ `article_slot` functions exist +(`:9593`, `:10276`) returning a different type, the failure surfaces as a confusing type +error rather than a missing-name error. + +**First, add the log-capture helper.** `crates/trusted-server-core/src/test_support.rs` +has none. Note its shape: the whole file is `#[cfg(test)] pub mod tests { … }`, so the +path is `crate::test_support::tests::capture_logs`, not `crate::test_support::capture_logs` +— see existing consumers at `auth.rs:103` and `config_payload.rs:48`. + +Two constraints the helper must respect or the test fails for unrelated reasons: + +- `log::set_boxed_logger` succeeds **once per process**. Install via a `OnceLock`/`Once` + and have `capture_logs()` return a guard that clears and then reads a shared buffer. +- Call `log::set_max_level(log::LevelFilter::Info)` or higher, or `log::info!` is filtered + out before it reaches the logger. +- **Do not have the guard hold the buffer's own `Mutex`.** The test body runs code that + calls `log::info!` on the same thread, and the logger must lock that same mutex to + append — `std::sync::Mutex` is not reentrant, so this **hangs** rather than failing. + Use two locks: a separate process-wide serialization mutex held by the guard, and the + buffer's own mutex taken and released per line by the logger. +- The buffer is process-global and every other concurrently-running `trusted-server-core` + test logs into it, so a `got: {captured}` diagnostic will be large. Assert with + `contains`, not equality. +- `log::set_max_level` is global for the test binary. Setting it to `Info` is fine, but it + affects every test in the process. + +```rust +#[tokio::test] +async fn eligible_navigation_logs_origin_fetch_duration() { + // Arrange + let logs = crate::test_support::tests::capture_logs(); + let mut settings = settings_with_enabled_auction_and_creative_opportunities(); + // The log line is gated; without this the assertions below can never pass. + settings.debug.publisher_timing = true; + let stub = Arc::new(StubHttpClient::new()); + queue_cacheable_html_response(&stub); + let services = build_services_with_http_client( + Arc::clone(&stub) as Arc + ); + let slots = [article_slot()]; + + // Act + let _ = run_with_slots(&settings, &services, &slots, conditional_navigation_request()).await; + + // Assert + let captured = logs.contents(); + assert!( + captured.contains("publisher_timing"), + "eligible navigation should emit a publisher_timing log line, got: {captured}" + ); + assert!( + captured.contains("origin_fetch_ms="), + "publisher_timing should record origin_fetch_ms, got: {captured}" + ); +} +``` + +This test deliberately asserts only on the `publisher_timing` line. `run_with_slots` never +drives a finalizer, so `publisher_rewrite` is out of its reach — cover that separately if +at all, rather than contorting this test. + +- [ ] **Step 2: Run the test to verify it fails** + +```bash +cargo test -p trusted-server-core --target aarch64-apple-darwin \ + eligible_navigation_logs_origin_fetch_duration -- --nocapture +``` + +Expected: FAIL — no `publisher_timing` in the captured logs. (Substitute your host +triple; core tests run natively for fast iteration. The Viceroy run comes in Step 6.) + +- [ ] **Step 3: Time the origin fetch** + +In `publisher.rs`, at the top with the other imports, add: + +```rust +use web_time::Instant; +``` + +Then wrap the origin send. The current code is at `publisher.rs:2870`: + +```rust +let mut response = match services.http_client().send(platform_request).await { +``` + +Change it to: + +```rust +let origin_fetch_start = Instant::now(); +let mut response = match services.http_client().send(platform_request).await { +``` + +and immediately after the `match` completes (after the existing `};` that closes it, +before the existing `log::debug!("Publisher origin response received: ...")` at `:2888`): + +```rust +let origin_fetch_ms = u64::try_from(origin_fetch_start.elapsed().as_millis()).unwrap_or(u64::MAX); +``` + +**Make the bypass config-driven in the same change.** This is what lets Stage 0 ship as a +config flip rather than a second deploy — see Task 5. Replace the block at +`publisher.rs:2866-2868`: + +```rust +// Single source of truth for the request and the log line below. Operator- +// controlled so the read-through cache can be re-enabled without a release; +// see docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md §4. +let cache_bypass = should_run_ad_stack && settings.publisher.bypass_origin_cache; +if cache_bypass { + platform_request = platform_request.with_cache_bypass(); +} +``` + +Add the setting to `Publisher` in `crates/trusted-server-core/src/settings.rs:29`, +**defaulting to today's behaviour** so this change is a no-op until deliberately flipped: + +```rust +/// Bypass the platform read-through cache on ad-eligible publisher navigations. +/// +/// `true` preserves the historical behaviour introduced by the SSAT 304-prevention +/// design. `false` lets those navigations use the read-through cache; the +/// conditional-header strip already guarantees a complete body on a cache HIT. +/// Temporary operator control for the Stage 0 rollout — remove once settled. +#[serde(default = "default_bypass_origin_cache")] +pub bypass_origin_cache: bool, +``` + +```rust +fn default_bypass_origin_cache() -> bool { + true +} +``` + +**Adding this field breaks nine sites. Update them in the same commit or Step 2 fails to +compile before it can produce the intended RED failure:** + +- The hand-written `Default` impl at `settings.rs:81-97`. +- Eight exhaustive test literals. The line numbers below anchor each + `let publisher = Publisher {` **opening**, not a field — add the new field inside each + brace: `settings.rs:3553`, `:3564`, `:3575`, `:3586`, `:3597`, `:3608`, `:3621`, + `:3635`. `clippy-fastly` runs `--all-targets`, so these gate lint too. +- The rustdoc example for `origin_host`, whose literal opens at `settings.rs:130`. + **This is a live doctest** and the host-triple test command below does not skip + doctests. + +While there, mirror the existing default-agreement test +`publisher_default_max_buffered_body_bytes_matches_config_default` (`settings.rs:3648`) — +it exists to catch a hand-written `Default` diverging from a serde default, which is +exactly the shape this field re-introduces. One assertion. + +Then emit the line, immediately after computing `origin_fetch_ms`, gated on the debug +flag from the section above: + +```rust +if settings.debug.publisher_timing { + log::info!( + "publisher_timing origin_fetch_ms={origin_fetch_ms} \ + cache_bypass={cache_bypass} ad_stack={should_run_ad_stack}" + ); +} +``` + +- [ ] **Step 4: Instrument `hold_wait_ms` — the decision metric** + +This is the number the whole effort turns on, and it needs **one edit in one function**. + +`collect_stream_auction` (`publisher.rs:2431`) is the only function that awaits the +auction collect, and all three call sites reach it: + +| Call site | Path | +| ------------------- | ------------------------------------------------------------ | +| `publisher.rs:793` | `hold_collect_close_tail` — Fastly lazy stream | +| `publisher.rs:2257` | `body_close_hold_loop`, EOF arm — Axum, Cloudflare, Spin | +| `publisher.rs:2311` | `body_close_hold_loop`, mid-stream arm — same three adapters | + +Instrument the callee, not the callers. It already destructures `settings` out of +`AuctionCollectDeps` (`:2436`), so the debug flag is in scope with no new plumbing, and +one edit covers every adapter. + +Wrap the `collect_dispatched_auction` await at `:2447-2449`: + +```rust + let hold_wait_start = Instant::now(); + let result = orchestrator + .collect_dispatched_auction(dispatched, services, &collect_ctx) + .await; + if settings.debug.publisher_timing { + let hold_wait_ms = + u64::try_from(hold_wait_start.elapsed().as_millis()).unwrap_or(u64::MAX); + log::info!("publisher_hold hold_wait_ms={hold_wait_ms}"); + } +``` + +`settings` here is `&&Settings` from the destructure — deref as needed; the compiler will +say so. + +**Do not instrument `hold_finish_ready_segments` (`:869`) or `hold_finish_tail_segments` +(`:896`).** Neither awaits the collect. The first returns `close_found` for its caller to +act on; the second delegates to `hold_collect_close_tail` at `:909`. Instrumenting them +would double-count. + +**Do not instrument the auction itself.** `OrchestrationResult::total_time_ms` +(`orchestrator.rs:285`, struct at `:1449`, per-provider at `:365`) already flows to +`auction_events_raw`. `hold_wait_ms` measures something different and more useful: how +long the _response_ waited, which is near zero when the auction finished during transfer +even though `total_time_ms` is large. + +- [ ] **Step 5: Run the test to verify it passes** + +```bash +cargo test -p trusted-server-core --target aarch64-apple-darwin \ + eligible_navigation_logs_origin_fetch_duration -- --nocapture +``` + +Expected: PASS. + +- [ ] **Step 6: Run the full publisher test module under the real target** + +A format-changing edit to this file can break tests far from the one you added, and the +Viceroy runner aborts on the first panic — so run the whole suite, not a filtered subset. + +```bash +cargo test-fastly +``` + +Expected: PASS. `app::tests` DNS `Error` lines in the output are pre-existing noise. + +- [ ] **Step 7: Verify format and lint** + +```bash +cargo fmt --all -- --check +cargo clippy-fastly +``` + +Expected: both clean. + +- [ ] **Step 8: Commit** + +```bash +git add crates/trusted-server-core/src/publisher.rs \ + crates/trusted-server-core/src/settings.rs \ + crates/trusted-server-core/src/test_support.rs +git commit -m "Add an operator switch for the origin cache bypass and log origin fetch time" +``` + +Staging without `settings.rs` leaves a tree that does not compile. + +- [ ] **Step 9: Deploy and collect** + +Deploy first. Then enable the log — it is gated and off by default: + +```bash +# In the operator's trusted-server.toml, under [debug]: +# publisher_timing = true +ts config push +``` + +**Deploy before pushing, not after.** `Settings`, `Publisher`, and `DebugConfig` all carry +`#[serde(deny_unknown_fields)]`, and `ts config push` validates against the typed schema +(`crates/trusted-server-cli` → `run_config_push_typed::`). So the +`ts` binary must be rebuilt from this commit (`cargo install-cli`), and pushing the new +keys before the new WASM is live would break config load on the deployed build. +`trusted-server.example.toml:121-125` records this same hazard for +`auction.rewrite_creatives`. + +Then capture the **bypass-on baseline only**. Do not try to collect an off arm here — +turning the bypass off _is_ Task 5, which is gated on Task 1's verdict and forbidden on a +FAIL. The off arm is collected in Task 5 Step 8. + +Capture enough navigations to separate the medians with confidence, across both a homepage and an article path, with the bypass +both on and off. Record the N alongside the result. + +Append to the findings document: + +```markdown +## Step C — server-side latency breakdown + +**N per arm:** · **Paths:** · **Date:** + +| Arm | `origin_fetch_ms` = origin TTFB (median) | auction `total_time_ms` (median) | `rewrite_ms` (median) | +| ---------- | ---------------------------------------- | -------------------------------- | --------------------- | +| bypass on | | | | +| bypass off | | | | + +Read the asymmetry carefully. `origin_fetch_ms` is origin **TTFB** — the send returns at +response headers because `.with_stream_response()` is set — whereas `total_time_ms` is +the auction's full duration. The comparison below is still the right one, but it is not +comparing two like quantities. + +**Verdict:** HOLD IS FREE / HOLD IS COSTING + +Read it off `hold_wait_ms` directly — no model, no comparison against auction duration. + +HOLD IS FREE = `hold_wait_ms` median near zero. The auction finishes during body +transfer. Proceed as staged in spec §7: Stage 0 primary, Stage 2 protects its win. + +HOLD IS COSTING = `hold_wait_ms` median materially non-zero. **Staging inverts** — +Stage 2 becomes primary and Stage 0 secondary. The work does not change, only its order. +Spec §6.2 argues for the first outcome but explicitly does not prove it, so treat the +second as a live possibility. +``` + +- [ ] **Step 10: Commit the findings** + +```bash +cd docs && npx prettier --write superpowers/plans/2026-08-08-1009-measurement-findings.md && cd .. +git add docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +git commit -m "Record server-side latency breakdown for #1009" +``` + +--- + +> **Task 4 (spec correction) was completed while this plan was being written.** §3's +> mechanism bullet, §4's operator-flag framing, and the `unexpected_origin_304` watch are +> all already in the spec. Nothing to do; the task is removed rather than left as a +> no-op an implementer would stall on. + +--- + +## Task 5: Stage 0 — turn the origin cache bypass off + +**Gate:** do not flip the flag until Task 1 has a recorded verdict. + +- **PASS** → Task 5a (config flip). +- **FAIL** → Task 5b. Do **not** flip on a FAIL; it can serve an RSC payload to an HTML + navigation. + +Because Task 3 made the bypass config-driven, Stage 0 ships as a **config change on an +already-deployed build**. No second release, and rollback is another config push rather +than a revert. That matters here specifically: the failure mode this gates on is cache +poisoning, where minutes of exposure are worse than a slow rollout. + +### Task 5a: flip the flag (Task 1 verdict = PASS) + +**Files:** + +- Modify: the operator's `trusted-server.toml` (gitignored) +- Modify: `crates/trusted-server-core/src/publisher.rs` — the test, and later the default +- Modify: `trusted-server.example.toml` — document the key + +- [ ] **Step 1: Add a test covering the flag in both positions** + +The existing test at `publisher.rs:4824` +(`eligible_navigation_bypasses_cache_and_returns_non_storable_html`) asserts `vec![true]` +and must **keep passing** while the default is `true` — it now documents the default +rather than the only behaviour. Leave it, and add a sibling next to it: + +```rust +#[tokio::test] +async fn eligible_navigation_uses_read_through_cache_when_bypass_disabled() { + // Arrange + let mut settings = settings_with_enabled_auction_and_creative_opportunities(); + settings.publisher.bypass_origin_cache = false; + let stub = Arc::new(StubHttpClient::new()); + queue_cacheable_html_response(&stub); + let services = build_services_with_http_client( + Arc::clone(&stub) as Arc + ); + let slots = [article_slot()]; + + // Act + let response = + run_with_slots(&settings, &services, &slots, conditional_navigation_request()).await; + let response_head = response_head(response); + + // Assert + assert_eq!( + stub.recorded_cache_bypass_flags(), + vec![false], + "disabling bypass_origin_cache should let the navigation use the read-through \ + cache; the conditional-header strip already guarantees a full body on a HIT" + ); + assert_eq!( + recorded_header( + stub.recorded_request_headers().first().expect("should record request"), + header::IF_NONE_MATCH.as_str() + ), + None, + "conditional headers must still be stripped with the bypass disabled" + ); + assert!( + response_head + .headers + .get(header::CACHE_CONTROL) + .and_then(|v| v.to_str().ok()) + .is_some_and(|v| v.contains("no-store")), + "the synthesized document must stay non-storable regardless of the bypass flag" + ); +} +``` + +Those last two assertions are the point of the test: the flag must change **only** the +cache mode, leaving the conditional-header strip and the response non-storability intact. + +**Leave `publisher.rs:4941` and `:5160` unchanged** — they already assert `vec![false]` +for non-eligible requests and must keep doing so. `Range`/`If-Range` stripping is covered +by `eligible_range_navigation_fetches_complete_html` (`publisher.rs:4883`), unaffected. + +- [ ] **Step 2: Run both tests** + +```bash +cargo test -p trusted-server-core --target aarch64-apple-darwin \ + eligible_navigation -- --nocapture +``` + +Expected: both the existing default-behaviour test and the new flag-disabled test PASS. +If Task 3's `bypass_origin_cache` field is not yet in place, the new test will not +compile — land Task 3 first. + +- [ ] **Step 3: Document both keys in the example config** + +Add to `trusted-server.example.toml` under `[debug]` (line 149, alongside +`ja4_endpoint_enabled` and `auction_html_comment`): + +```toml +# Emit a `publisher_timing` log line per publisher origin fetch. Temporary +# instrumentation for the #1009 latency measurement; leave false in production. +publisher_timing = false +``` + +And under `[publisher]`: + +```toml +# Bypass the platform read-through cache on ad-eligible navigations. +# `true` is the historical default. Set `false` to let those navigations use the +# read-through cache — only after confirming the origin declares `Vary` for every +# header it varies on (see the Stage 0 precondition). +bypass_origin_cache = true +``` + +- [ ] **Step 4: Flip it in the operator config and push** + +```bash +# In the operator's trusted-server.toml, under [publisher]: +# bypass_origin_cache = false +ts config push +``` + +Note from prior operational experience in this repo: the environment-variable overlay is +scalar-only **and** only overrides keys that already exist in the TOML. Adding the key to +the operator's file is required; setting only an env var will be silently dropped. + +**Roll back by pushing `true` again.** No release required. That is the whole reason this +is a flag. + +- [ ] **Step 5: Run the full suite across every adapter** + +```bash +cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin +``` + +Expected: all PASS. If `platform/test_support.rs:797` or `:888` fail, they are testing +the stub's own recording behaviour rather than publisher behaviour — read them before +changing anything. + +- [ ] **Step 6: Format and lint every target** + +```bash +cargo fmt --all -- --check +cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare \ + && cargo clippy-cloudflare-wasm && cargo clippy-spin-native && cargo clippy-spin-wasm +``` + +Expected: all clean. + +- [ ] **Step 7: Commit the code and config-template changes** + +```bash +git add crates/trusted-server-core/src/publisher.rs trusted-server.example.toml +git commit -m "Add an operator switch for the publisher origin cache bypass" +``` + +- [ ] **Step 8: Watch for the failure modes, not just the win** + +After the flip, check three things before declaring success. The first two are regression +signals, not confirmations. + +1. **`unexpected_origin_304` abandonment telemetry.** This reason + (`publisher.rs:2896`, emitted via `emit_abandoned_auction` at `:2360`) exists because + the ad-stack path refuses cached and conditional origin responses. Re-enabling the + cache is precisely what could revive it. **Any non-zero rate is a rollback signal** — + it means a 304 is reaching TS, which the conditional-header strip was supposed to make + impossible. Push `true` and investigate before continuing. +2. **Representation mixing.** Spot-check that HTML navigations still return HTML and RSC + fetches still return `text/x-component`. A mismatch is the Task 1 risk having + materialized despite a PASS verdict — roll back immediately, this is cache poisoning. +3. **`origin_fetch_ms` and `cache_bypass=false`** in the `publisher_timing` logs. This is + the win, and it is the _last_ thing to check, not the first. + +- [ ] **Step 9: Record and commit** + +Append the before/after medians and the three checks above to the findings document, +format it, and commit. + +- [ ] **Step 10: Retire the flag (follow-up, not now)** + +Once the flip has held for a sustained period, flip the default to `false` in +`default_bypass_origin_cache`, then remove the setting and the branch entirely. Track it; +a temporary flag left in place becomes permanent configuration surface. + +### Task 5b: cache-key discriminator (Task 1 verdict = FAIL) + +**Do not implement from this plan.** A FAIL means the origin serves multiple +representations at one URL without declaring `Vary`, so removing the bypass requires TS +to add its own cache-key discriminator — a feature, not a deletion, and materially larger +than Stage 0 as scoped here. + +Escalate with the Task 1 findings and write a separate plan. Two things that plan must +address, both from spec §4: + +1. The discriminator must key on the request headers that actually distinguish the + representations (`RSC`, `Next-Router-*`, the experiment header), **not** on the + navigation classification. `is_navigation_request` + (`crates/trusted-server-core/src/http_util.rs:73-98`) falls back to the `Accept` + header when Fetch Metadata is absent, and its own comment warns that `fetch()` can set + `Accept: text/html` — so a fetch-based request can be misclassified as a navigation. +2. Whether the origin should simply be asked to declare `Vary`, which is cheaper than + building the discriminator and fixes the problem for every consumer rather than only + for TS. + +--- + +## Out of scope + +Named so nobody widens this plan mid-flight. All are specified in the spec. + +- **Stages 1–2** — moving bid delivery off the response body and deleting the `` + hold. Spec §7 and §8 put these behind the correctness defects. Spec §5 explains why + starting them casually produces a silent revenue loss. +- **Stages 3a/3b** — response cacheability. 3b is additionally gated on Task 2. +- **Stages 4–5** — purge capability, TS-owned template cache, ESI. +- **Removing the `bypass_cache` platform capability.** Task 5a removes one call site only. + +--- + +## Definition of done + +- [ ] Findings document records verdicts for Steps A, B, and C, each with its date, its + N where applicable, and the consequence spelled out. +- [ ] `publisher_timing` and `publisher_hold` lines are emitted in production and + readable, and `hold_wait_ms` has a recorded median. +- [ ] Either Task 5a is shipped, or Task 1 returned FAIL and both a production defect and + a follow-up plan for 5b exist. +- [ ] **The win is measured client-side, not from `origin_fetch_ms`.** That figure is + origin TTFB and excludes body download, rewrite, and post-processing — it is + attribution, not the outcome. #1009 already has a working tester-cookie browser A/B + measuring the TTFB the publisher actually complained about; use it for before/after. +- [ ] `unexpected_origin_304` rate is zero and representations are not mixed (Task 5a + Step 8) — both checked **before** the win is claimed. +- [ ] All CI gates pass: `cargo fmt --all -- --check`; the six clippy targets; the four + adapter test suites; the parity suite; JS build, test, and format; docs format. From 8e124ca0ade68ed47c5fc6a76404f13c390c5714 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Sat, 8 Aug 2026 15:49:53 +0530 Subject: [PATCH 04/44] Record Step A findings for #1009: origin Vary check passes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ran the Stage 0 gate against the publisher origin. Verdict is PASS, so Stage 0 takes the operator-flag path rather than the cache-key discriminator, and there is no live cross-serving defect. The origin declares vary on rsc, next-router-state-tree, next-router-prefetch, next-router-segment-prefetch and Accept-Encoding, covering every header that distinguishes the HTML and RSC representations sharing a URL. It names one the plan did not think to probe. Bodies do not differ by cookie, no Set-Cookie rides a shared-cacheable response, and the origin answers without credentials. Two things the check was not looking for. The origin already sets cache-control: max-age=60 with a correct Vary, so it has been cacheable all along and Trusted Server opted out of it — though a 60 second TTL also bounds the win. And the document regenerates roughly 170 ad-slot container IDs as fresh UUIDs per request, so a cached copy serves identical IDs to every visitor within the TTL. That is probably harmless because slot definitions come from config rather than origin markup, but it is an untested interaction with slot matching and belongs on the pre-flip checklist. Also fixes a defect in the plan's own probe. It compared body hashes, which on this origin differ on every request because of those UUIDs, cookie or not — it would have reported a false FAIL every time. Replaced with normalize-then-diff against a measured no-cookie baseline, and noted that the Host override is required because the origin is a shared vhost. --- ...2026-08-08-1009-measurement-and-stage-0.md | 35 ++++- .../2026-08-08-1009-measurement-findings.md | 121 ++++++++++++++++++ 2 files changed, 149 insertions(+), 7 deletions(-) create mode 100644 docs/superpowers/plans/2026-08-08-1009-measurement-findings.md diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md index c5f85aa4c..a28da9186 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md @@ -162,17 +162,38 @@ class: TS forwards client cookies to origin unchanged, so any cookie-personalize (logged-in state, paywall meter, publisher-side A/B assignment) becomes cross-servable once the cache is on. +**Do not compare body hashes.** Verified on the live origin: this page regenerates +~170 ad-slot container IDs as fresh 32-hex UUIDs on every request, so three requests give +three different hashes with byte-identical lengths, cookie or not. A hash comparison +reports a false FAIL every time. + +Normalize per-request identifiers, establish the no-cookie baseline drift first, then ask +whether the cookie arm differs by _more_ than that baseline: + ```bash -# Same URL, with and without a session cookie. Compare Content-Length and body hash. -for C in '' 'Cookie: '; do - echo "--- ${C:-no-cookie}" - curl -sS -D /dev/stderr -o - "https:///" \ - -H 'Sec-Fetch-Dest: document' -H 'Accept: text/html' ${C:+-H "$C"} \ - 2> >(grep -iE '^(vary|cache-control|set-cookie|content-length):' >&2) \ - | shasum +ORIGIN="https://"; HOSTH="Host: " +norm() { sed -E 's/[0-9a-f]{32}/UUID/g' "$1"; } + +for n in a b; do + curl -sS "$ORIGIN/" -H "$HOSTH" \ + -H 'Sec-Fetch-Dest: document' -H 'Accept: text/html' > "nc_$n.html" done +curl -sS "$ORIGIN/" -H "$HOSTH" \ + -H 'Sec-Fetch-Dest: document' -H 'Accept: text/html' \ + -H 'Cookie: ' > ck.html + +echo "baseline drift: $(diff <(norm nc_a.html) <(norm nc_b.html) | grep -c '^[<>]')" +echo "with cookie: $(diff <(norm nc_a.html) <(norm ck.html) | grep -c '^[<>]')" +diff <(norm nc_a.html) <(norm ck.html) | head -20 ``` +Send the `Host` override — the origin is a shared vhost and will not return the right +document without it. Read it from `publisher.origin_host_header_override`. + +**Step A has already been run and passed.** See +[the findings](./2026-08-08-1009-measurement-findings.md). Re-run only if the origin +changes. + Three failure shapes, any of which blocks Stage 0 independently of the `Vary` verdict: - Bodies differ by cookie **and** `Vary` does not name `Cookie` → cross-serving of diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md new file mode 100644 index 000000000..78f0a3d78 --- /dev/null +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md @@ -0,0 +1,121 @@ +# #1009 measurement findings + +Recorded output of the checks in +[the plan](./2026-08-08-1009-measurement-and-stage-0.md). Results only — the origin +hostname is operator config and is deliberately not reproduced here. + +## Step A — origin `Vary` declaration and cookie exposure + +**Date:** 2026-08-08 · **Method:** direct `curl` against the publisher origin with the +configured `origin_host_header_override`, homepage path. + +### Representation split + +| Representation | `Content-Type` | `Cache-Control` | `Set-Cookie` | +| ------------------------------------ | ------------------ | --------------- | ------------ | +| HTML navigation | `text/html` | `max-age=60` | none | +| `RSC: 1` | `text/x-component` | `max-age=60` | none | +| `RSC: 1` + `Next-Router-Prefetch: 1` | `text/x-component` | `max-age=60` | none | + +`Vary`, identical on every response: + +``` +vary: rsc, next-router-state-tree, next-router-prefetch, next-router-segment-prefetch, Accept-Encoding +``` + +The origin declares **every** header that distinguishes the representations, including +`next-router-segment-prefetch`, which the plan's probe list did not think to check. The +HTML/RSC split at one URL is real and correctly declared. + +### Cookie personalization + +Hash comparison was useless here and the plan's probe as written would have produced a +false FAIL — see the method note below. After normalizing per-request identifiers: + +| Comparison | Differing lines | +| ------------------------------ | --------------- | +| no-cookie A vs no-cookie B | 2 | +| no-cookie A vs **with cookie** | 2 | + +Both diffs are the same single `generationTimestamp` field in the RSC payload. **The +cookie changes nothing.** Byte lengths were identical across all three responses +(1,432,944). + +Cookie sent: `ts-tester=true; sessionid=abc123; ts-ec=probe`. + +### Verdict: **PASS** + +- `Vary` names every request header the origin varies on. ✅ +- Bodies do not differ by cookie, so `Vary: Cookie` is not required. ✅ +- No `Set-Cookie` on a shared-cacheable response. ✅ +- Origin returns 200 without credentials, so no `Authorization` exposure at this layer. + (#1009's basic-auth gate is on the Trusted Server side, not the origin.) ✅ + +**Consequence:** Stage 0 takes the simple path — the operator flag plus a config flip, +not the cache-key discriminator. No live production defect: the origin's `Vary` covers +the RSC variants that already transit the read-through cache today. + +## Two findings the checks were not looking for + +### 1. The origin already intends this page to be shared-cached + +`cache-control: max-age=60` with a correct `Vary` and no `Set-Cookie`. The origin has +been cacheable all along; Trusted Server opted out of it. That is the spec's §4 framing +confirmed from the other side, and it strengthens the case that the bypass was +belt-and-braces rather than load-bearing. + +It also bounds the win: a 60-second TTL means Stage 0 buys a cache hit only within that +window. Whether that translates into a meaningful hit rate depends on request volume per +URL, which is not measured here. + +### 2. Ad-slot div IDs are randomized per request — and this interacts with Stage 0 + +The only per-request variance in the document is ~170 lines of ad-slot container IDs, +each a fresh 32-hex UUID: + +``` +ad-in_content-f75fa7fba54a4fc2a2d787f51c1837dd-in_content-0 +ad-in_content-a968b27e3ee2424f8bb1c19560abf2b1-in_content-0 ← same slot, next request +``` + +Under the bypass, Trusted Server sees fresh IDs on every request. **Once the cache is on, +every visitor within a 60-second window receives the same IDs.** + +This is very likely fine — `tsjs.adSlots` is built from configured slot definitions, not +scraped from origin markup, and injection is a prefix match on the configured `div_id`. +But it is an untested interaction between Stage 0 and the slot-matching path, and it was +not in anyone's risk list. **Verify slot matching still resolves against a cached +document before flipping the flag**, and watch TS-attributed renders across the flip +rather than only `origin_fetch_ms`. + +## Method note — a defect in the plan's Step A probe + +The plan's cookie check compares `shasum` of the response bodies. On this origin that +test always fails, cookie or not, because of the randomized div IDs above. Three requests +produced three different hashes with byte-identical lengths. + +**Correct method:** normalize per-request identifiers before comparing, e.g. +`sed -E 's/[0-9a-f]{32}/UUID/g'`, and diff the normalized bodies rather than hashing +them. Establish the no-cookie baseline drift first, then compare the cookie arm against +that baseline — a cookie arm is only interesting if it differs by _more_ than the +baseline does. Fix the plan before anyone re-runs this. + +## Residual uncertainty + +#1009 states the origin varies on an experiment header as well as `rsc` and +`next-router-*`. **No experiment header appears in the origin's `Vary` list**, and the +RSC payload's `experiments` key did not differ across any of the requests made here. + +Three readings, unresolved: the issue was imprecise; experiments are assigned +client-side; or they key on a cookie value this probe did not supply. The `Vary` +declaration is authoritative for cache correctness and it is thorough enough to name four +Next-specific headers, so this is unlikely to be a cache-safety gap. Worth one question +to whoever wrote that line in #1009 rather than further probing. + +## Step B — consumers of TS's own response headers + +Not yet run. + +## Step C — hold and origin fetch timings + +Not yet run. From 86fa20f6395af43499aa1bcf67e69d1bf2638fa3 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 14:51:36 +0530 Subject: [PATCH 05/44] Reopen ESI as feasible and separate it from the Stage 0 optimisation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An external review rejected the previous revision's central conclusion and was right to. Verified against the pinned fastly 0.12.1. ESI was called structurally blocked on two grounds, both false. The cache boundary it needs is native: cache::core provides insert(key, max_age).execute() returning a StreamingBody for arbitrary bytes, lookup()/found() to read them back, and Transaction with must_insert() for request collapsing. No separate KV or template service is required. And purge exists in-process via InsertBuilder::surrogate_keys plus http::purge::purge_surrogate_key, so the management-API token scope previously cited is the wrong surface entirely. The error was inspecting what this repository does and reporting it as what the platform permits, which is the same mistake the document criticises #1009 for making in the other direction. The correction is recorded at the top of the spec rather than quietly edited in. The pipeline ordering was also backwards. It said order esi then lol_html; lol_html is what emits the esi:include tags, so ESI must run after it. New section 6.6 gives the corrected pipeline and separates the three caches the documents had been conflating: origin read-through, shared transformed template, and a final assembled-response cache that must never exist. #418 is React's error number, not a repository issue. The tracker is #938. Stage 0 is reframed as a supporting optimisation and the experimental control, not an answer to #1009 — it has no ESI or client-fill arm, so completing it cannot close the issue. Its rollback claim is corrected: flipping the flag stops HTML reading from cache but evicts nothing, so rollback needs a purge or a versioned key namespace and observation past the origin TTL. Step A is downgraded from PASS to provisional. It used a synthetic session cookie, one route, no experiment variant, and no authenticated session through TS. Cached-hit slot resolution becomes a release gate rather than a note. Adds the ESI validation spike plan: four comparable arms plus a TS-off reference, a deterministic synthetic fragment before the real auction, safety gates run against every arm rather than once at the end, a decision rule ratified before collection, purge-based rollback, and reproducibility metadata. --- ...2026-08-08-1009-measurement-and-stage-0.md | 29 +- .../2026-08-08-1009-measurement-findings.md | 50 +- .../2026-08-10-1009-esi-validation-spike.md | 460 ++++++++++++++++++ ...08-esi-cacheable-root-validation-design.md | 247 +++++++--- 4 files changed, 694 insertions(+), 92 deletions(-) create mode 100644 docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md index a28da9186..3aa100ade 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md @@ -2,9 +2,16 @@ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. -**Goal:** Obtain the checks that gate the #1009 work, then turn off the redundant origin -cache bypass that the spec identifies as the actual TTFB cost — behind an operator flag, -so it rolls back with a config push rather than a release. +**Goal:** Turn off the redundant origin cache bypass that the spec identifies as the +actual TTFB cost, behind an operator flag, and establish the measurement baseline that +later work is compared against. + +> **This plan does not close #1009.** It contains no ESI arm and no client-fill arm, so +> completing it cannot answer whether ESI separates cacheable content from per-user +> state. It is a **supporting optimisation and the experimental control** for +> [the ESI validation spike](./2026-08-10-1009-esi-validation-spike.md), which is where +> #1009 is actually decided. Scoped and framed this way after external review on +> 2026-08-10. **Architecture:** Two investigation tasks that produce recorded findings and no code; one code task that adds a config-gated timing log and makes the cache bypass operator- @@ -864,8 +871,20 @@ Note from prior operational experience in this repo: the environment-variable ov scalar-only **and** only overrides keys that already exist in the TOML. Adding the key to the operator's file is required; setting only an env var will be silently dropped. -**Roll back by pushing `true` again.** No release required. That is the whole reason this -is a flag. +**Rollback is a config push plus an eviction — not a config push alone.** Pushing `true` +again stops HTML navigations reading from cache, but evicts nothing: objects already +cached, including those RSC and other request classes keep reading, persist until they +expire. The origin's `max-age=60` bounds that, but does not remove it. + +Full rollback: + +1. Push `bypass_origin_cache = true`. +2. Purge. `fastly::http::purge::purge_surrogate_key` runs inside Compute, with keys + attached at insert via `InsertBuilder::surrogate_keys`; alternatively roll a versioned + cache-key namespace. **Neither is wired today** — if the flip ships before one exists, + the rollback story is "wait out the TTL," and that must be an accepted risk rather + than an unnoticed one. +3. Observe past the origin TTL before declaring the incident closed. - [ ] **Step 5: Run the full suite across every adapter** diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md index 78f0a3d78..a125d957c 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md @@ -43,17 +43,32 @@ cookie changes nothing.** Byte lengths were identical across all three responses Cookie sent: `ts-tester=true; sessionid=abc123; ts-ec=probe`. -### Verdict: **PASS** +### Verdict: **PROVISIONAL PASS** — not sufficient to gate a production flip + +Downgraded 2026-08-10 after external review. Everything below held under the conditions +tested; the conditions tested are narrower than the gate requires. + +What passed: - `Vary` names every request header the origin varies on. ✅ -- Bodies do not differ by cookie, so `Vary: Cookie` is not required. ✅ +- Bodies did not differ by the cookie sent, so `Vary: Cookie` was not required **for + that cookie**. ✅ - No `Set-Cookie` on a shared-cacheable response. ✅ -- Origin returns 200 without credentials, so no `Authorization` exposure at this layer. - (#1009's basic-auth gate is on the Trusted Server side, not the origin.) ✅ +- Origin returns 200 without credentials at this layer. ✅ + +**What was not tested, and each of these can flip the verdict:** -**Consequence:** Stage 0 takes the simple path — the operator flag plus a config flip, -not the cache-key discriminator. No live production defect: the origin's `Vary` covers -the RSC variants that already transit the read-through cache today. +| Gap | Why it matters | +| ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `sessionid=abc123` is not a real session | A synthetic value proves nothing about a state-bearing publisher session. An authenticated or paywall-metered session is exactly the case that would personalize. | +| One route (homepage) only | Article, section, and search routes may personalize differently. | +| Experiment variant never exercised | #1009 says the origin varies on one. It is absent from `Vary` — see Residual uncertainty below. | +| Basic Auth through TS untested | #1009 describes a gated deployment. Only the origin was probed directly. | +| Cached-hit slot resolution untested | The randomized div IDs below are an unverified interaction, not a cleared one. | + +**Consequence:** Stage 0 still takes the operator-flag path rather than the cache-key +discriminator, and no live cross-serving defect is indicated. But this is **not** a +release gate. Close the table above before flipping the flag in production. ## Two findings the checks were not looking for @@ -84,8 +99,8 @@ every visitor within a 60-second window receives the same IDs.** This is very likely fine — `tsjs.adSlots` is built from configured slot definitions, not scraped from origin markup, and injection is a prefix match on the configured `div_id`. But it is an untested interaction between Stage 0 and the slot-matching path, and it was -not in anyone's risk list. **Verify slot matching still resolves against a cached -document before flipping the flag**, and watch TS-attributed renders across the flip +not in anyone's risk list. **This is a release gate, not a note.** Verify slot matching resolves against a cached +document before flipping the flag, and watch TS-attributed renders across the flip rather than only `origin_fetch_ms`. ## Method note — a defect in the plan's Step A probe @@ -112,6 +127,23 @@ declaration is authoritative for cache correctness and it is thorough enough to Next-specific headers, so this is unlikely to be a cache-safety gap. Worth one question to whoever wrote that line in #1009 rather than further probing. +## Rollback caveat, added 2026-08-10 + +The plan described flipping the flag back as a seconds-long rollback. That is +incomplete. Re-enabling the bypass stops **HTML navigations** reading from cache; it +evicts nothing. Objects already cached — including those RSC and other request classes +continue to read — persist until they expire. + +Two mitigations, both real: + +- The origin's `max-age=60` bounds read-through exposure to roughly a minute. +- Purge is available in-process: `fastly::http::purge::purge_surrogate_key`, with keys + attached at insert via `InsertBuilder::surrogate_keys`. An earlier claim that TS had no + purge capability was wrong — it has no _wiring_, which is buildable. + +Rollback is therefore: flip the flag, **then** purge or roll a versioned cache-key +namespace, **then** observe past the origin TTL before declaring the incident closed. + ## Step B — consumers of TS's own response headers Not yet run. diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md new file mode 100644 index 000000000..bbfd7de84 --- /dev/null +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -0,0 +1,460 @@ +# #1009 ESI Validation Spike + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development +> (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps +> use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Decide #1009 on evidence. Build a shared-template pipeline behind a flag, run +ESI and client-fill against it, and produce a decision record that either adopts ESI, +adopts client-fill, or rejects both — with the Fastly-only maintenance cost priced in. + +**Architecture:** `origin → lol_html transform → fastly::cache::core → assemble → finalize`. +The transform emits `esi:include` markers at the two existing injection seams instead of +inlining per-user data. The cached object is a shared template with no per-user bytes. +Assembly is either the `esi` crate (edge) or a client fetch of `/_ts/page-bids` (browser), +selected per request by config so both can be measured on one build. + +**Tech Stack:** Rust 2024, `wasm32-wasip1`, `fastly` 0.12.1 (`cache::core`, `http::purge`), +`esi` 0.7, `lol_html`, a real Fastly test service for cache behaviour. + +**Spec:** `docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md` — +read the 2026-08-10 correction at the top and +[§6.6](../specs/2026-08-08-esi-cacheable-root-validation-design.md#66-the-esi-pipeline-corrected) +before writing any code. + +**Control:** [the Stage 0 plan](./2026-08-08-1009-measurement-and-stage-0.md). Its +instrumentation and its bypass flag are prerequisites — this plan compares against them +and does not duplicate them. + +--- + +## Why this plan exists + +An earlier revision of the spec concluded ESI was structurally impossible. It was wrong: +`fastly::cache::core` provides the cache boundary natively, and purge runs inside Compute. +That correction reopens #1009 as an empirical question, and this plan is how it gets +answered. + +**What is genuinely uncertain**, and what each arm is for: + +1. Does a shared template plus per-request assembly beat today's inline path enough to + matter? +2. Does **edge** assembly (ESI) beat **client** assembly (a fetch) by enough to justify a + Fastly-only rendering path that must be maintained alongside the portable one? +3. Can per-user leakage be excluded across cold MISS, warm HIT, stale revalidation, + transform failure, and fragment failure? + +Question 3 is a gate, not a metric. A win on 1 and 2 with a failure on 3 is a rejection. + +## Three caches, never conflated + +The original error came from treating these as one thing. Every task below names which it +means. + +| # | Cache | Contents | Status | +| --- | --------------------------------- | ----------------------------- | ----------------------------------- | +| C1 | Origin read-through | raw origin bytes | Exists. Stage 0 turns it back on. | +| C2 | Shared transformed template | post-`lol_html`, pre-assembly | **New.** What this plan builds. | +| C3 | Assembled-response delivery cache | final per-user output | **Must never exist.** Not proposed. | + +If a task appears to require C3, stop — that is the leakage failure mode, not a design +option. + +## Arms + +Five, but only four are treatable as equivalent. + +| Arm | Root | Bids | Notes | +| ------- | ----------------------- | ---------------- | ---------------------------------------------------------- | +| **A0** | inline, C1 bypassed | inline `` | Today. The baseline. | +| **A1** | inline, C1 on | inline `` | Stage 0. Isolates the bypass from the template change. | +| **A2** | shared template from C2 | client fetch | Portable. Works on all four adapters. | +| **A3** | shared template from C2 | ESI at the edge | Fastly-only. The thing #1009 proposed. | +| **REF** | origin direct, TS off | publisher's own | **Reference, not an arm.** Different work, not comparable. | + +A0→A1 measures the bypass. A1→A2 measures the template split. A2→A3 measures edge versus +client assembly — **that difference is the entire case for ESI**, and it is the number +this plan exists to produce. + +REF is included because #1009 anchors on it, and excluded from pass/fail because TS-off +does no auction and no injection. Comparing against it measures the feature's existence, +not its implementation. + +--- + +## Task order and dependencies + +``` +Stage 0 plan (flag + timing instrumentation) ──┐ + ├──> Task 3 (C2 template cache) +Task 1 (esi crate compiles) ───────────────────┤ +Task 2 (test service + harness) ────────────────┘ │ + ├──> Task 4 (A2 client-fill) + ├──> Task 5 (A3 ESI) + └──> Task 6 (safety gates) + │ + └──> Task 7 (decision record) +``` + +Tasks 1 and 2 are independent and should run first — both can invalidate the plan +cheaply. Task 6 runs against every arm, not once at the end. + +--- + +## Task 1: Confirm `esi` 0.7 builds on this toolchain + +Cheapest possible falsification. Do this before anything else. + +**Files:** `crates/trusted-server-adapter-fastly/Cargo.toml` + +- [ ] **Step 1: Add the dependency** + +```bash +cargo add esi@0.7 --package trusted-server-adapter-fastly +``` + +It belongs in the **Fastly adapter**, never in `trusted-server-core` — the crate is +hard-bound to `fastly::{Request, Response, Backend}` and core must stay portable. + +- [ ] **Step 2: Check it compiles for the real target** + +```bash +cargo check-fastly +``` + +Expected: clean. The crate declares edition 2021 with no `rust-version`, and pulls recent +`rand` and `nom`, so this is a genuine question on Rust 1.95.0 / `wasm32-wasip1`. + +- [ ] **Step 3: Check the lockfiles have not desynced** + +```bash +git diff --stat Cargo.lock +cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity +``` + +CI requires shared direct deps to match between the root and integration-tests lockfiles. +`regex`, `bytes`, and `log` overlap. If they desync, fix with targeted +`cargo update -p --precise ` — **never a full update**. + +- [ ] **Step 4: Record and commit, or stop** + +If Step 2 fails, this plan stops here and #1009 is answered "not on this toolchain." +Record that in the findings document and escalate rather than fighting the build. + +```bash +git add crates/trusted-server-adapter-fastly/Cargo.toml Cargo.lock +git commit -m "Add the esi crate to the Fastly adapter for the #1009 validation spike" +``` + +--- + +## Task 2: Stand up the test service and the harness + +**Viceroy 0.17 cannot exercise the `cache::core` hooks end to end.** Unit tests cover the +transform and the security properties; MISS / HIT / stale / shielding must run on a real +Fastly service. Establish that before building, or Tasks 3–6 have nowhere to run. + +- [ ] **Step 1: Provision a dedicated test service** + +Separate from production. Confirm and record: whether the publisher backend is +**shielded**, and whether any Delivery service fronts the Compute service. Both change +what the numbers mean. + +```bash +fastly service list +fastly backend list --service-id --version latest +``` + +The shielding answer also settles an open question from the Stage 0 findings: #1009's +off-TS win came from a shield HIT, so whether the test service has one determines whether +its numbers transfer to production at all. + +- [ ] **Step 2: Extend the harness for correlation** + +The existing tester-cookie A/B has no way to join server timings to browser timings. Add +a per-request correlation ID — generated at TS entry, echoed in an `x-ts-request-id` +response header, and included in every timing log line. + +Without it, the experiment cannot join hold time, origin time, auction telemetry, browser +TTFB, and render outcome for the same request. **That is the difference between an +experiment and a pile of numbers.** + +- [ ] **Step 3: Capture cache tier and status per request** + +Record `x-cache`, `hit-state`, `age`, and the serving POP alongside each measurement. A +median that mixes cold-MISS and warm-HIT requests is meaningless, and arms cannot be +compared unless the mix is known. + +- [ ] **Step 4: Define the sample plan before collecting anything** + +State, in the findings document, ahead of time: requests per arm per route, how cold MISS +is forced, how warm HIT is confirmed, and the confidence interval to be reported. + +Rationale: this whole effort exists because #1009 drew a causal conclusion from N=4 that +did not survive contact with the code. Repeating that with more arms would be worse, not +better. + +--- + +## Task 3: Build C2 — the shared transformed-template cache + +The core of the spike. Behind a flag, default off. + +**Files:** + +- `crates/trusted-server-core/src/publisher.rs` — emit markers at the two seams +- `crates/trusted-server-core/src/settings.rs` — the mode flag +- `crates/trusted-server-adapter-fastly/src/` — the `cache::core` read/write + +- [ ] **Step 1: Add the assembly-mode setting** + +```rust +/// How per-user ad state reaches the page. +/// +/// `Inline` is today's behaviour: bids injected before ``, root uncacheable. +/// `ClientFill` and `Esi` both serve a shared template from the transformed-template +/// cache and fill the holes afterwards. Spike-only — remove with the spike. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Deserialize, Serialize)] +#[serde(rename_all = "snake_case")] +pub enum AssemblyMode { + #[default] + Inline, + ClientFill, + Esi, +} +``` + +Default `Inline` so the flag is a no-op until set. Note the hazards the Stage 0 plan +already documents: `Settings` carries `#[serde(deny_unknown_fields)]`, `ts config push` is +typed, and `Publisher` has a hand-written `Default` plus eight exhaustive test literals +and a live doctest. + +- [ ] **Step 2: Emit markers instead of inlining, under `ClientFill`/`Esi`** + +The two seams are already isolated — that is #1009's correct observation. At head-open, +`tsjs.adSlots` is **per-URL and stays in the template** (config- and path-derived only, +`publisher.rs:3501-3525`). At body-close, emit a marker instead of the bids script. + +Under `Esi`: ``. +Under `ClientFill`: nothing at all — **not an empty bids script.** The Stage 0 plan +explains why: an empty script calls `scheduleInitialAdInit({})` and assigns +`ts.bids = {}` synchronously, racing the client fetch. + +**Assert the template carries no per-user bytes.** A unit test over the transform output +must fail on any of: a bid value, an EC ID, a consent string, a geo value, or a +`Set-Cookie`. This is the test that makes C2 safe, and it is cheaper to write now than to +retrofit. + +- [ ] **Step 3: Write the template into C2** + +```rust +// Fastly adapter. Key on the same signals the origin varies on, plus TS's own +// variant inputs. Surrogate-key it so rollback can purge rather than wait. +let mut insert = fastly::cache::core::insert(cache_key, template_ttl); +insert.surrogate_keys([&surrogate_key_for_url, "ts-template"]); +let mut body = insert.execute()?; +// stream the lol_html output into `body` +``` + +Use `cache::core::Transaction` with `must_insert()` for the lookup, so a cold cache under +load transforms once rather than per concurrent request. + +**Cache key must include** everything the origin's `Vary` names — `rsc`, +`next-router-state-tree`, `next-router-prefetch`, `next-router-segment-prefetch`, +`Accept-Encoding` (measured, see the Stage 0 findings) — **plus** TS's own per-variant +inputs: request host and scheme, the enabled-integration set, and the tsjs content hash. +Per-user signals must never appear in the key; they must be absent from the template +instead. If a signal cannot be excluded from the template, it does not belong in C2. + +Set `template_ttl` deliberately short for the spike. A short TTL bounds every failure mode +here and costs only hit rate. + +- [ ] **Step 4: Read it back and assemble** + +On `found()`, skip the origin fetch and the transform entirely; hand the cached body to +the assembler. On miss, transform and insert as above, then assemble from what was +inserted. + +- [ ] **Step 5: Unit tests, then the target suite** + +```bash +cargo test -p trusted-server-core --target aarch64-apple-darwin assembly_mode +cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin +cargo fmt --all -- --check && cargo clippy-fastly +``` + +`ClientFill` must work on all four adapters. `Esi` is Fastly-only and must not break the +others' compilation. + +--- + +## Task 4: Arm A2 — client-fill + +Mostly already specified. See +[the spec's Appendix B](../specs/2026-08-08-esi-cacheable-root-validation-design.md#appendix-b--stage-1-plumbing-condensed) +for the client plumbing, the two-condition join gate, and the server contract; and +[§5](../specs/2026-08-08-esi-cacheable-root-validation-design.md#5-the-trap-in-the-deferred-work--read-this-before-scheduling-stages-12) +for the silent-empty-bids trap, which applies in full. + +- [ ] **Step 1: Hoist the closure-trapped client state** — `pageBidsEndpoint`, + `requestPageBids`, and the `inflight`/`currentPath`/`lastAppliedPath` state, per + Appendix B. Do **not** route the initial load through `onNavigate`. +- [ ] **Step 2: Make `installScheduleInitialAdInit` a hydration-ready AND bids-settled + join**, with a bounded timeout that fires `adInit` untargeted rather than stranding + the slot. Derive the timeout from measured fetch latency, not a constant. +- [ ] **Step 3: Suppress the navigation-path dispatch** so exactly one auction runs per + pageview. Add a new `AuctionSource` for initial loads **plus the mechanism that + delivers it** — a header behind the same-origin gate, not a query parameter. +- [ ] **Step 4: Relocate terminal telemetry.** Navigation `Completed` is emitted only from + the collect functions; the `ts-debug` dump rides the same string. Both move. +- [ ] **Step 5: Verify exactly one auction per pageview** in `auction_events_raw`. Two is + a doubling of SSP spend and an immediate fail. + +--- + +## Task 5: Arm A3 — ESI at the edge + +- [ ] **Step 1: Wire `process_stream`, not the wrappers** + +`process_response` and `process_response_streaming` consume `self` _and_ send the response +themselves, which takes ownership away from the finalize / `ec_finalize` / apply-effects +ordering. `process_stream(&mut self, src: impl BufRead, out: &mut impl Write, …)` keeps it. + +Source is the C2 body. Sink is the response body on the way to the client — so EC cookie, +geo, and the privacy net still run **after** assembly. Confirm that ordering explicitly; +it is the difference between a correct response and a leaked one. + +- [ ] **Step 2: Disable DCA explicitly and allowlist the dispatcher** + +```rust +let config = esi::Configuration::default() + .with_escaped(false); +// default_dca and inherit_parent_dca stay at DcaMode::None / false — set them +// explicitly rather than relying on defaults; this is a pre-1.0 crate and the +// setting fails open. +``` + +The dispatcher must be **exact-path allowlisted**: a fragment URL that is not the bids +endpoint is refused, not fetched. The built-in dispatcher builds a dynamic backend per URL +host and panics on a hostless URL — never use it. + +Rationale in the spec's §2: bid payloads carry partner-controlled creative markup, so a +recursive parse would let an SSP make the edge fetch an arbitrary URL. **Add a unit test +that feeds `` through a creative payload and +asserts no fetch is attempted.** + +- [ ] **Step 3: Deterministic synthetic fragment first** + +Before the real auction, point the include at a fixed-content endpoint. This separates +"does the pipeline assemble correctly" from "does the auction behave," and the two fail +very differently. Only once assembly is proven does the include move to +`/_ts/page-bids`. + +- [ ] **Step 4: Handle the flush hazard** + +`esi` flushes its output writer after each parse batch. Fastly's `StreamingBody` is a +`BufWriter`, so anything between esi and it must propagate `flush()` or nothing leaves the +Wasm heap. + +- [ ] **Step 5: Fragment failure must degrade, not break** + +Assert that a fragment timeout or non-2xx yields a page with empty bids rather than a 5xx +or a truncated document. Note the crate's non-obvious semantics: `alt` is attempted before +`onerror="continue"`, and `` runs **all** attempts and concatenates every +non-failed output — it is not first-success-wins. + +--- + +## Task 6: Safety gates — run against every arm + +Not a phase. Every one of these is a hard fail, independent of any performance result. + +- [ ] **Zero cross-user leakage.** Request the same URL as two synthetic users differing + in consent state, EC identity, and geo. Assert the C2 template is byte-identical + and that no bid, EC ID, consent string, or geo value appears in it. +- [ ] **Cold MISS, warm HIT, stale revalidation** each produce a correct page. +- [ ] **Transform failure** (the 16 MB buffer cap, a malformed body) does not insert a + partial template into C2 and does not serve one. +- [ ] **Request collapsing** works: concurrent cold requests transform once. +- [ ] **DCA disabled**, verified by the injection test in Task 5 Step 2. +- [ ] **Exactly one auction per pageview**, from `auction_events_raw`. +- [ ] **Cookie and privacy finalization still run** after assembly — EC `Set-Cookie` on + first visit, and the privacy net downgrading it. This is the ordering that ESI's + streaming mode makes easy to get wrong, since it drops `$add_header`. +- [ ] **Slot and bid attribution unchanged.** Same slots matched, same bids applied, same + renders attributed. Use TS-attributed renders — the SSAT line item, non-empty + `ts.bids`, `hb_adid` presence — **never slot fill**, which is blind to empty bids + because `adInit` defines slots regardless. +- [ ] **No C3.** Assert the final assembled response is never shared-cacheable: no + `public`, no `s-maxage`, no `Surrogate-Control` on a response carrying per-user + state. + +--- + +## Task 7: The decision record + +**Files:** `docs/superpowers/plans/2026-08-10-1009-esi-decision-record.md` + +- [ ] **Step 1: Record every arm** with N, confidence interval, cache-tier mix, route mix, + and POP. Any arm missing those is not reportable. + +- [ ] **Step 2: Apply the decision rule, stated here before the data exists** + +**Adopt ESI only if all three hold:** + +1. Every Task 6 gate passes on A3. +2. A3 beats A2 on TTFB by a margin the reviewers ratify **before** collection — not + chosen after seeing the numbers. +3. Render outcomes on A3 are non-inferior to A0. + +**Otherwise adopt A2 (client-fill)** if its gates pass and it beats A1. It is portable +across all four adapters and carries no Fastly-only maintenance burden. + +**Otherwise keep A1** — Stage 0 alone — and record #1009 as answered in the negative with +evidence. + +The margin in (2) exists because A3's cost is not its diff. It is a second rendering +architecture, Fastly-only, on a pre-1.0 crate, in the critical render path. A small win +does not pay for that. + +- [ ] **Step 3: Record what would change the answer**, so this does not get re-litigated + from scratch. At minimum: React #418 / [#938](https://github.com/IABTechLab/trusted-server/issues/938) + being fixed such that `adInit` can run synchronously, which is what would make edge + assembly's round-trip saving actually worth something. + +- [ ] **Step 4: Clean up.** Remove the spike flag or promote it to a real setting; purge + C2 (`purge_surrogate_key` on `ts-template`); remove the synthetic fragment endpoint; + and either land or delete the `esi` dependency. **A spike flag left in place becomes + permanent configuration surface.** + +--- + +## Reproducibility metadata + +Record with every result, or it cannot be re-run or trusted: commit SHA; `esi` and +`fastly` crate versions; Fastly service and version IDs; whether the backend is shielded; +`template_ttl`; the origin's `Cache-Control` and `Vary` at collection time; assembly mode; +routes; N per arm; and the cache-tier mix. + +## Out of scope + +- **Stages 1–2 of the spec** as production work. This spike may build parts of the + client-fill path to measure it; shipping it is a separate decision behind the + correctness defects. +- **Full RSC/flight partitioning.** `rsc_flight.rs` has no static/dynamic split. +- **Publisher-authored ESI.** Breaks the no-origin-changes promise. +- **A C3 delivery cache.** Not a deferred item — a thing that must not exist. + +## Definition of done + +- [ ] Task 1 verdict recorded: `esi` 0.7 builds on Rust 1.95.0 / `wasm32-wasip1`, or it + does not and the spike stopped. +- [ ] All four arms measured on one build, with correlation IDs joining server and browser + timings, and cache tier recorded per request. +- [ ] Every Task 6 gate has an explicit pass/fail per arm. +- [ ] Decision record exists, applies the pre-ratified rule, and names what would change + the answer. +- [ ] Cleanup complete: flag resolved, C2 purged, synthetic endpoint removed, dependency + landed or dropped. +- [ ] All CI gates pass: `cargo fmt --all -- --check`; the six clippy targets; the four + adapter test suites; the parity suite; JS build, test, and format; docs format. diff --git a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md index a2e23b4f9..1417c5122 100644 --- a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md +++ b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md @@ -1,17 +1,45 @@ # ESI and the Cacheable Root -**Issue:** IABTechLab/trusted-server#1009 · **Date:** 2026-08-08 -**Baseline:** citations verified at `cfb98f4`; unchanged as of `b0ce56c3` (the two -commits between touch only CI workflows and Cargo aliases). +**Issue:** IABTechLab/trusted-server#1009 · **Date:** 2026-08-08 · +**Revised:** 2026-08-10 +**Baseline:** citations verified at `cfb98f4`; unchanged as of `b0ce56c3`. -**Decision requested:** approve the four items below. Three are "yes/no"; one funds -about three days of measurement. +> ## ⚠️ Correction, 2026-08-10 — this document's original ESI verdict was wrong +> +> The first revision concluded that ESI was **structurally blocked**: that it +> presupposed a TS-owned template cache which did not exist, and that such a cache was +> in turn blocked on purge capability the platform did not offer. **Both claims are +> false**, and an external review was right to reject them. +> +> Verified against the pinned `fastly` 0.12.1: +> +> - **The cache boundary is native.** `fastly::cache::core` provides +> `insert(key, max_age).execute() -> StreamingBody` for arbitrary bytes, `lookup()` / +> `found()` to read them back, and `Transaction` with `must_insert()` for request +> collapsing. The two-stage design needs no separate KV or template service. +> - **Purge exists in-process.** `InsertBuilder::surrogate_keys([...])` attaches keys at +> insert; `fastly::http::purge::purge_surrogate_key` purges from inside Compute. The +> management-API token scope cited in the original is irrelevant to it. +> - **The original pipeline ordering was backwards.** It said "order esi → lol*html, +> never the reverse." `lol_html` \_emits* the `esi:include` tags, so ESI must run after +> it. Correct order is in [§6.6](#66-the-esi-pipeline-corrected). +> +> The error was inspecting what this repository does and reporting it as what the +> platform permits — the same mistake this document criticises #1009 for making in the +> other direction. +> +> **ESI is therefore feasible and unvalidated, not rejected.** Validating it is +> [a separate plan](../plans/2026-08-10-1009-esi-validation-spike.md). What survives +> here is the latency re-diagnosis and the Stage 0 optimisation, which are worth doing +> and are **not** an answer to #1009. + +**Decision requested:** approve the four items in §1. > **Orientation.** #1009 asks whether Edge Side Includes (ESI) can cache page fragments > so that cacheable publisher HTML is separated from per-user ad state, recovering a > TTFB regression that Trusted Server (TS) adds to navigations on a Next.js App Router -> publisher running on Fastly Compute. The answer is no to ESI, and the regression has a -> cheaper cause than the issue assumes. +> publisher running on Fastly Compute. ESI can do this; whether it should is not settled +> here. Separately, the regression has a cheaper cause than the issue assumes. > > **This document deliberately carries no performance measurements.** Every conclusion > below is derived from code at the pinned baseline, so it can be checked by reading the @@ -19,22 +47,24 @@ about three days of measurement. > it is named as unknown and [§3](#3-monday-morning) says how to obtain it. > > Terms used throughout: **the hold** = TS holding the HTTP response open at `` -> until the server-side auction (SSAT) resolves. **#418** = a React hydration-mismatch -> defect caused by `adInit()` mutating ad-slot subtrees during hydration; it is why bid -> application is deferred to `window.load`. **The SSAT price defect** = a live -> mispricing bug named in #1009 (prices reading 100× high) — cited from #1009 and prior -> investigation, not re-verified here. +> until the server-side auction (SSAT) resolves. **React #418** = the React +> hydration-mismatch error raised when `adInit()` mutates ad-slot subtrees during +> hydration; it is why bid application is deferred to `window.load`. It is a React error +> number, **not** a repository issue — the tracker is +> [#938](https://github.com/IABTechLab/trusted-server/issues/938). **The SSAT price +> defect** = a live mispricing bug named in #1009 (prices reading 100× high) — cited +> from #1009 and prior investigation, not re-verified here. --- ## 1. Decision requested -| # | Decision | Owner needed | -| --- | -------------------------------------------------------------------------------------------------------------------------------- | ------------- | -| D1 | **ESI is deferred.** Revival condition: #418 resolved _and_ the `window.load` gate removed. Not a rejection — a dated condition. | Eng + product | -| D2 | **Fund ~3 days of measurement** (§3). No dependencies. Can start immediately. | Eng | -| D3 | **Approve Stage 0** — an operator flag disabling the origin cache bypass, subject to the `Vary` check in §3. | Eng | -| D4 | **Stages 1–2 queue behind the SSAT price defect and #418.** Stages 3b–5 unscheduled. | Product | +| # | Decision | Owner needed | +| --- | --------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| D1 | **ESI is feasible and unvalidated.** Validate it via [the spike plan](../plans/2026-08-10-1009-esi-validation-spike.md), not by deferring it. | Eng + product | +| D2 | **Fund ~3 days of measurement** (§3). No dependencies. Can start immediately. | Eng | +| D3 | **Approve Stage 0** — an operator flag disabling the origin cache bypass, subject to the `Vary` check in §3. | Eng | +| D4 | **Stages 1–2 queue behind the SSAT price defect and #938.** Stages 3b–5 unscheduled. | Product | Rationale for D4 in [§8](#8-priority). Everything this document recommends _against_ doing is in [§7](#7-deferred-work-specified-not-scheduled), at deliberately lower @@ -44,17 +74,20 @@ detail than the work it recommends. ## 2. Why — the three findings -**ESI does not work here, for a structural reason #1009 misses.** ESI's input is pull -(`BufRead`); `lol_html`'s is push (`HtmlRewriter::write`). ESI cannot sit downstream of -the rewriter without an intermediate buffer, and in the two-stage design the cache -boundary _is_ that buffer. **ESI presupposes a TS-owned template cache** rather than -being independent of one — and that cache is blocked on purge capability TS does not -have (no `Surrogate-Key` anywhere; the Fastly management token is scoped without purge -permission). ESI is also Fastly-only at every API level. Its one advantage over a -client fetch — no round trip — is worth nothing while bids are not consumed until -`window.load`. Separately, enabling ESI's Dynamic Content Assembly would be an SSRF -vector: bid payloads carry partner-controlled creative markup, so an SSP could embed -`` and make the edge fetch an arbitrary URL. Details in +**ESI is buildable on the pinned SDK, and unvalidated.** `lol_html` emits +`esi:include` tags into a shared template; `fastly::cache::core` stores that template; +the `esi` crate assembles per request on the way out. Everything that requires is +already a dependency. The real open questions are empirical, not architectural: does it +beat a plain client fetch by enough to justify a Fastly-only rendering path, and can +per-user leakage be excluded under cold MISS, warm HIT, stale revalidation, and fragment +failure. [The spike plan](../plans/2026-08-10-1009-esi-validation-spike.md) answers +those; [§6.6](#66-the-esi-pipeline-corrected) gives the pipeline. + +Two constraints stay true regardless. ESI is **Fastly-only at every API level**, so it +is a per-platform accelerator rather than the architecture, and its maintenance cost +belongs in the decision. And its Dynamic Content Assembly must be **explicitly disabled** +— bid payloads carry partner-controlled creative markup, so under `DcaMode::Esi` an SSP +could embed `` and make the edge fetch an arbitrary URL. Details in [Appendix E](#appendix-e--esi-notes-condensed). **The auction is already out of band; the hold is ~free.** It is dispatched _before_ @@ -361,7 +394,7 @@ no post-processor takes the streaming path and would see a lower floor. ### 6.5 Confidence **High on the structural claims.** §6.2's argument, the bypass forcing a cache miss, the -ESI push/pull mismatch, the silent-empty-bids failure mode, the geo and purge blockers, +the silent-empty-bids failure mode, the geo and `Vary` blockers, and the fill-canary blindness are all read directly out of the code at `cfb98f4`. Anyone can check them without running anything. @@ -373,6 +406,51 @@ That is a caution about small samples generally, not only about that one — whi §3 Step C specifies the measurement rather than this document supplying a substitute for it. +### 6.6 The ESI pipeline, corrected + +An earlier revision of this document said "order esi → lol*html, never the reverse." +That is backwards. `lol_html` is what \_emits* the `esi:include` tags; ESI cannot process +tags that do not exist yet. The correct order: + +``` +origin → lol_html transform → fastly::cache::core → esi assemble → finalize → client + (emit esi:include at the (shared template, (per request, (EC cookie, + head + body-close seams, surrogate-keyed, fetch the geo, privacy + no per-user data) TS-chosen TTL) bids fragment) net) +``` + +The push/pull mismatch that the earlier revision treated as a blocker is real but +irrelevant: `lol_html` pushes, `esi` pulls, and **the cache is the buffer between them**. +That is not an obstacle to the two-stage design — it _is_ the two-stage design, which is +what #1009 proposed in the first place. + +Mechanism, all present in the pinned `fastly` 0.12.1: + +| Need | API | +| ----------------------- | ----------------------------------------------------------------------------------- | +| Store the template | `cache::core::insert(key, max_age).execute() -> StreamingBody` | +| Read it back | `cache::core::lookup(key)` → `found()` | +| Avoid a thundering herd | `cache::core::Transaction` — `must_insert()` / `must_insert_or_update()` | +| Invalidate | `InsertBuilder::surrogate_keys([...])` + `fastly::http::purge::purge_surrogate_key` | + +Purge runs **inside Compute**. The management-API token scope cited under +[Stage 4](#7-deferred-work-specified-not-scheduled) governs a different surface and does +not gate this. + +**Three caches, kept distinct.** Conflating them is what produced the original error: + +1. **Origin read-through** — raw origin bytes. What Stage 0 turns back on. +2. **Shared transformed template** — post-`lol_html`, pre-ESI, no per-user data. The ESI + target, and new. +3. **Assembled-response delivery cache** — the final per-user output. **Must never + exist.** Nothing in this document or the spike proposes one. + +**Validation constraint.** Viceroy 0.17 cannot exercise the customized read-through hooks +end to end. Unit tests can cover the transform and the security properties; MISS / HIT / +stale / shielding behaviour must run against a real Fastly test service. + +--- + --- ## 7. Deferred work, specified not scheduled @@ -425,12 +503,19 @@ and can never be shared-cached ([Appendix C](#appendix-c--vary-signals-condensed **Also gated on Step B**: if nothing consumes TS's response headers, this tier is inert until a topology change. -**Stage 4 — purge capability.** Not sized. Prerequisite for anything beyond the backend -readthrough cache. TS today has no `Surrogate-Key` emission and no purge permission, so +**Stage 4 — purge wiring.** Not sized. Prerequisite for a TS-owned cache. TS today emits +no `Surrogate-Key` and holds a management token scoped without purge — but that token is +the wrong surface: `InsertBuilder::surrogate_keys` and +`fastly::http::purge::purge_surrogate_key` are both in the pinned SDK and purge runs +inside Compute ([§6.6](#66-the-esi-pipeline-corrected)). This is **missing wiring, not a +platform limit.** Until it exists, any TS-owned cache is TTL-only and a config push takes up to one TTL to take effect. -**Stage 5 — TS-owned template cache, then ESI.** Not sized, and gated on D1's revival -condition. +**Stage 5 — ESI.** Superseded. ESI no longer waits on a "revival condition"; it is +feasible on the pinned SDK and is validated by +[its own spike plan](../plans/2026-08-10-1009-esi-validation-spike.md), which does not +queue behind Stages 1–4. The shared template cache it needs is +`fastly::cache::core` ([§6.6](#66-the-esi-pipeline-corrected)), not a new service. **Identity needs no work.** A new visitor's first navigation sets the EC cookie and the privacy net downgrades that one response; every later navigation sets no cookie and is @@ -443,10 +528,10 @@ server-side (`rsc_flight.rs` plus `integrations/nextjs/`, ~4,100 lines) by rewri hydration's _input_; a shim would be a second source of truth producing the exact mismatch both exist to prevent. Late-bid rendering has no foothold and the obvious interception point is measured-unsafe: a controlled capture found the `__next_f` gate -reproducing #418 on every run and destroying the creative on half of them — it patches +reproducing React #418 on every run and destroying the creative on half of them — it patches `__next_f.push` shortly after React's first commit, while hydration continues for thousands more. Two retractions to carry forward: that gate is -measured-unsafe rather than merely unproven, and "#418 at ~5% and not impression-costing" +measured-unsafe rather than merely unproven, and "React #418 at ~5% and not impression-costing" is retracted — it came from pages whose slots are not React-owned. Note `docs/superpowers/specs/2026-07-24-adinit-hydration-gate-design.md` exists only on the unmerged branch `958-adinit-hydration-chunk-gate`, so `publisher.rs:3461-3464` points at @@ -472,10 +557,11 @@ it did not need to opt out of. **Stages 1–2 queue behind the correctness defects.** Their failure mode is silent revenue loss, against a publisher whose ads currently fill reliably. The SSAT price defect misprices live auctions, and **a slow correct auction loses less money than a -fast wrong one**. #418 sits ahead too: Stage 1's join gate must be reconciled with +fast wrong one**. React #418 sits ahead too: Stage 1's join gate must be reconciled with whatever hydration gate lands, and doing that twice is waste. -**Stages 3b–5 are not competitive** on current evidence and should not be scheduled. +**Stages 3b–4 are not competitive** on current evidence and should not be scheduled. +**ESI is no longer in this queue** — it is validated separately and on its own evidence. --- @@ -518,7 +604,8 @@ Rows 1–6 are in [§6.1](#61-corrections-to-1009s-premises). The remainder: **Cache tiers.** T1 backend readthrough (already available; TS opts out). T2 TS-owned template cache (adds KV latency, eventual consistency, a full invalidation design). T3 delivery cache of TS output (Fastly topology change; **ESI cannot run**, since -Compute is not invoked on a HIT). T2 and T3 both introduce a cache TS cannot purge. +Compute is not invoked on a HIT). T2 and T3 both introduce a cache TS does not yet purge — +wiring that is available but unbuilt. --- @@ -607,22 +694,25 @@ exists. ## Appendix E — ESI notes (condensed) -For if and when [D1](#1-decision-requested)'s revival condition is met. Expand then; -recording only what would otherwise be re-derived: +Input to [the ESI validation spike](../plans/2026-08-10-1009-esi-validation-spike.md). +Recording only what would otherwise be re-derived: - Pin `esi = "0.7"`. Pre-1.0, irregular cadence, two yanked betas in the 0.7 line. - **Use `process_stream`, not the wrappers.** `process_response` and `process_response_streaming` consume `self` _and_ send the response themselves, taking ownership away from the finalize / `ec_finalize` ordering. -- **Order esi → lol_html**, never the reverse, via a newtype implementing `io::Write`. - Mind the `StreamingBody`-is-a-`BufWriter` hazard: esi flushes per parse batch, so any - adapter in between must propagate `flush()`. +- **Order lol_html → cache → esi.** `lol_html` emits the tags; ESI consumes them on the + way out. An earlier revision had this backwards — see + [§6.6](#66-the-esi-pipeline-corrected). Mind the `StreamingBody`-is-a-`BufWriter` + hazard on the way to the client: esi flushes per parse batch, so anything between esi + and the `StreamingBody` must propagate `flush()`. - **Always supply a custom fragment dispatcher.** The built-in one builds a dynamic backend per URL host and panics on a hostless URL; dynamic backends are also the known Viceroy local-dev failure mode here. -- **DCA off, asserted explicitly** — not merely left at its default. Rationale is the SSRF - vector in [§2](#2-why--the-three-findings): partner-controlled creative markup would - become ESI-executable at the edge. +- **DCA off, asserted explicitly** — not merely left at its default. Rationale is the + SSRF vector in [§2](#2-why--the-three-findings): partner-controlled creative markup + would become ESI-executable at the edge. Pair it with an **exact-path allowlist + dispatcher**, so a fragment URL that is not the bids endpoint cannot be fetched at all. - **`` runs _all_ attempts and concatenates every non-failed output** — not first-success-wins, so primary/fallback pairs render both. Least obvious behaviour in the crate. @@ -649,34 +739,35 @@ queued before it loads (#1009 Part 1) — not filed, should be. All pinned to `cfb98f4`. -| Concern | Location | -| ------------------------------------------- | ------------------------------------------------------------------------------- | -| Eligibility decision | `publisher.rs:2651`, `:2660` | -| `is_navigation_request` | `http_util.rs:73-98` | -| Auction dispatch (pre-origin, non-blocking) | `publisher.rs:2698-2760` | -| Auction overlap intent | `auction/orchestrator.rs:950-952` | -| Auction timeout resolution | `publisher.rs:2680-2684` (creative_opportunities, else auction.timeout_ms) | -| Conditional/range header strip | `publisher.rs:2832-2836` | -| Origin cache bypass | `publisher.rs:2866-2868` | -| Origin 304 → 502 guard | `publisher.rs:2894-2916` | -| `adSlots` build (per-URL) | `publisher.rs:2920`, `:3558-3577`, `:3501-3525` | -| Uncacheable stamp | `publisher.rs:2945-2963` | -| `` hold — sync / async / Fastly lazy | `publisher.rs:2235` / `:2109` / `:1318-1390` | -| Hold buffer | `publisher.rs:2177-2218` | -| Auction collect (HTML / non-HTML) | `publisher.rs:2431` (emits `:2456`) / `:2388` (emits `:2410`) | -| Abandonment emitter | `publisher.rs:2360` | -| Bids script build | `publisher.rs:3438-3491` | -| `/_ts/page-bids` | `publisher.rs:3611`, handler `:3723`, auction `:3903` | -| Injection seams (head / body-close) | `html_processor.rs:310-363` / `:381-395` | -| Post-processor buffering | `html_processor.rs:62-94` | -| Next.js post-processor registration | `integrations/nextjs/mod.rs:107` | -| Max buffered body (16 MB) | `settings.rs:77-79` | -| EC cookie issuance policy | `ec/finalize.rs:86-107` | -| Cookie-privacy net | `response_privacy.rs:20-61` | -| Geo response headers | `adapter-fastly/src/middleware.rs:194-200` | -| Cacheable-header precedent | `http_util.rs:294-311` | -| No purge permission | `adapter-fastly/src/management_api.rs:12` | -| Client initial-ad gate | `js/lib/src/integrations/gpt/index.ts:536-555` | -| `adInit` bid application | `js/lib/src/integrations/gpt/index.ts:566`, `:652`, `:657-661` | -| Client SPA auction hook | `js/lib/src/integrations/gpt/index.ts:806`, `:859`, `:892-949` | -| Prior design that introduced the killers | `docs/superpowers/specs/2026-07-22-ssat-root-document-304-prevention-design.md` | +| Concern | Location | +| -------------------------------------------- | ------------------------------------------------------------------------------- | +| Eligibility decision | `publisher.rs:2651`, `:2660` | +| `is_navigation_request` | `http_util.rs:73-98` | +| Auction dispatch (pre-origin, non-blocking) | `publisher.rs:2698-2760` | +| Auction overlap intent | `auction/orchestrator.rs:950-952` | +| Auction timeout resolution | `publisher.rs:2680-2684` (creative_opportunities, else auction.timeout_ms) | +| Conditional/range header strip | `publisher.rs:2832-2836` | +| Origin cache bypass | `publisher.rs:2866-2868` | +| Origin 304 → 502 guard | `publisher.rs:2894-2916` | +| `adSlots` build (per-URL) | `publisher.rs:2920`, `:3558-3577`, `:3501-3525` | +| Uncacheable stamp | `publisher.rs:2945-2963` | +| `` hold — sync / async / Fastly lazy | `publisher.rs:2235` / `:2109` / `:1318-1390` | +| Hold buffer | `publisher.rs:2177-2218` | +| Auction collect (HTML / non-HTML) | `publisher.rs:2431` (emits `:2456`) / `:2388` (emits `:2410`) | +| Abandonment emitter | `publisher.rs:2360` | +| Bids script build | `publisher.rs:3438-3491` | +| `/_ts/page-bids` | `publisher.rs:3611`, handler `:3723`, auction `:3903` | +| Injection seams (head / body-close) | `html_processor.rs:310-363` / `:381-395` | +| Post-processor buffering | `html_processor.rs:62-94` | +| Next.js post-processor registration | `integrations/nextjs/mod.rs:107` | +| Max buffered body (16 MB) | `settings.rs:77-79` | +| EC cookie issuance policy | `ec/finalize.rs:86-107` | +| Cookie-privacy net | `response_privacy.rs:20-61` | +| Geo response headers | `adapter-fastly/src/middleware.rs:194-200` | +| Cacheable-header precedent | `http_util.rs:294-311` | +| Management token lacks purge (wrong surface) | `adapter-fastly/src/management_api.rs:12` | +| In-process purge / surrogate keys | `fastly` 0.12.1 `cache::core`, `http::purge::purge_surrogate_key` | +| Client initial-ad gate | `js/lib/src/integrations/gpt/index.ts:536-555` | +| `adInit` bid application | `js/lib/src/integrations/gpt/index.ts:566`, `:652`, `:657-661` | +| Client SPA auction hook | `js/lib/src/integrations/gpt/index.ts:806`, `:859`, `:892-949` | +| Prior design that introduced the killers | `docs/superpowers/specs/2026-07-22-ssat-root-document-304-prevention-design.md` | From bc6b4001effa9fb4d26e1c4c1d62531a61aab243 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 15:35:18 +0530 Subject: [PATCH 06/44] Fix the docs build and the design breaks in the #1009 spike MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The docs build was broken and committed. `npm run build` failed with 27 dead links from the spec's relative `../../../crates/...` references; VitePress rejects links outside the docs root and no other spec in the repository uses them. I had only ever run `npm run format`, which does not catch this. Converted to plain code references, matching what every other spec does. Build passes. Four design breaks, all verified against the source before fixing. The shared template was not request-neutral. `tsjs.adSlots` was kept in it on the grounds of being per-URL. Its content is per-URL; its presence is gated on should_run_ad_stack, which depends on consent, bot classification, prefetch status and the auction kill switch. The first request to fill the cache would have frozen its own consent decision into an object every later visitor reads. Both slots and bids now move to the request-aware fragment, the template carries an unconditional inert placeholder, and a test asserts the template is byte-identical across requests differing in consent, bot and prefetch state. The Core Cache pseudocode did not compile. surrogate_keys takes and returns self, so the sample discarded the builder and then used a moved binding; execute() yields a write stream rather than the readable object the next step assumed; finish() was never called; and the key omitted the assembly mode, so the client-fill and ESI arms would have poisoned each other. Replaced with a transaction using execute_and_stream_back, an explicit user_metadata envelope since cache::core carries no HTTP semantics, a cancel-on-error path, and a versioned key. The alternative read-through design is named rather than assumed. The ESI fragment contract was broken. It pointed at /_ts/page-bids, which returns JSON, and ESI splices fragment bytes literally — the page would have contained raw JSON where an executable script belongs. Also: the endpoint's same-origin gate rejects internal subrequests, parent identity and consent context did not propagate, root dispatch was not suppressed so spend would double, and path-only validation admits an attacker authority. The no-C3 gate only forbade public, s-maxage and Surrogate-Control. A bare max-age=60 passes that and is still shared-cacheable — and is exactly what the measured origin sends. Now requires private, no-store positively, tested for returning users, who set no EC cookie and so are not covered by the privacy net. Stage 0 could still ship on provisional evidence: the findings said PROVISIONAL PASS but the plan said Step A had passed and the gate accepted only PASS or FAIL. There are now three verdicts, with FINAL PASS requiring a real session cookie, Basic Auth through TS, the experiment variant, representative routes and cached-hit render attribution. Methodology: A3 and A2 are no longer compared on root TTFB, since both serve the same template — the comparison is bids-ready, adInit fire and first attributed creative paint. Sample plan gains allocation, randomization, pilot variance, MDE and power, CI method and carryover control. Correlation becomes a lineage ID carrying the experiment arm through fragment and auction telemetry, since a root-only ID never reaches an auction that runs in a subrequest. C1 and C2 cache status are recorded separately. DCA now calls the setters rather than commenting that defaults suffice, and fragment caching is disabled. Corrected: Viceroy 0.17 does support cache::core locally; only the customized HTTP read-through hooks are unsupported. Also removed leftovers claiming KV latency for what is a cache, and a config-only rollback. --- ...2026-08-08-1009-measurement-and-stage-0.md | 33 +- .../2026-08-10-1009-esi-validation-spike.md | 289 ++++++++++++++---- ...08-esi-cacheable-root-validation-design.md | 75 ++--- 3 files changed, 293 insertions(+), 104 deletions(-) diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md index 3aa100ade..6298ca2c6 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md @@ -197,9 +197,9 @@ diff <(norm nc_a.html) <(norm ck.html) | head -20 Send the `Host` override — the origin is a shared vhost and will not return the right document without it. Read it from `publisher.origin_host_header_override`. -**Step A has already been run and passed.** See -[the findings](./2026-08-08-1009-measurement-findings.md). Re-run only if the origin -changes. +**Step A has been run once and returned a PROVISIONAL PASS**, which is **not** sufficient +to flip the flag. See [the findings](./2026-08-08-1009-measurement-findings.md) for the +five untested conditions. Complete them and record a `FINAL PASS` before Task 5. Three failure shapes, any of which blocks Stage 0 independently of the `Vary` verdict: @@ -750,12 +750,29 @@ git commit -m "Record server-side latency breakdown for #1009" ## Task 5: Stage 0 — turn the origin cache bypass off -**Gate:** do not flip the flag until Task 1 has a recorded verdict. +**Gate:** do not flip the flag until Task 1 has recorded a **`FINAL PASS`**. There are +three verdicts, not two. -- **PASS** → Task 5a (config flip). -- **FAIL** → Task 5b. Do **not** flip on a FAIL; it can serve an RSC payload to an HTML +- **`FINAL PASS`** → Task 5a (config flip). +- **`PROVISIONAL PASS`** → **stop.** Not a release gate. This is the current state. It + means the representation split is declared correctly under the conditions tested, and + that those conditions were too narrow to flip production on. +- **`FAIL`** → Task 5b. Do **not** flip; it can serve an RSC payload to an HTML navigation. +**`FINAL PASS` requires all five, each recorded in the findings document:** + +| Condition | Why the provisional run is insufficient | +| -------------------------------------------------------- | ---------------------------------------------------- | +| A real authenticated or state-bearing session cookie | `sessionid=abc123` is synthetic and proves nothing | +| Basic Auth exercised **through TS**, not just the origin | #1009 describes a gated deployment | +| The experiment variant named in #1009 | Absent from the origin's `Vary`; unexplained | +| Representative routes — article, section, search | Only the homepage was probed | +| Cached-hit slot and render attribution | The randomized div IDs are an unverified interaction | + +Any one of these unrecorded means the verdict stays `PROVISIONAL PASS` and Task 5 does +not start. + Because Task 3 made the bypass config-driven, Stage 0 ships as a **config change on an already-deployed build**. No second release, and rollback is another config push rather than a revert. That matters here specifically: the failure mode this gates on is cache @@ -982,8 +999,12 @@ Named so nobody widens this plan mid-flight. All are specified in the spec. N where applicable, and the consequence spelled out. - [ ] `publisher_timing` and `publisher_hold` lines are emitted in production and readable, and `hold_wait_ms` has a recorded median. +- [ ] Task 1 recorded a **`FINAL PASS`** — all five conditions in Task 5's gate closed, + not merely the provisional run. - [ ] Either Task 5a is shipped, or Task 1 returned FAIL and both a production defect and a follow-up plan for 5b exist. +- [ ] A purge path or versioned cache-key namespace exists **before** the flip, or the + "wait out the TTL" rollback is explicitly accepted and recorded as a risk. - [ ] **The win is measured client-side, not from `origin_fetch_ms`.** That figure is origin TTFB and excludes body download, rewrite, and post-processing — it is attribution, not the outcome. #1009 already has a working tester-cookie browser A/B diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index bbfd7de84..770cb05cc 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -76,6 +76,12 @@ A0→A1 measures the bypass. A1→A2 measures the template split. A2→A3 measur client assembly — **that difference is the entire case for ESI**, and it is the number this plan exists to produce. +**Do not compare A2 and A3 on root TTFB.** They serve the same C2 template, so their root +timings should be near-identical by construction; a null result there proves nothing. +ESI's claimed advantage is that bids arrive without a client round-trip, so measure: +**bids-ready time**, **`adInit` fire time**, and **first TS-attributed creative paint**. +Root TTFB stays as a guard that the template path did not regress, not as the comparison. + REF is included because #1009 anchors on it, and excluded from pass/fail because TS-off does no auction and no injection. Comparing against it measures the feature's existence, not its implementation. @@ -150,9 +156,15 @@ git commit -m "Add the esi crate to the Fastly adapter for the #1009 validation ## Task 2: Stand up the test service and the harness -**Viceroy 0.17 cannot exercise the `cache::core` hooks end to end.** Unit tests cover the -transform and the security properties; MISS / HIT / stale / shielding must run on a real -Fastly service. Establish that before building, or Tasks 3–6 have nowhere to run. +**Viceroy 0.17 does support `cache::core` locally** — an earlier draft of this plan said +otherwise and was wrong. What it does **not** support is the customized HTTP +read-through hooks (`after_send` / `set_body_transform`), which matters only if the +alternative design in Task 3 Step 4 is chosen. + +So: C2 insert/lookup/transaction logic, the transform, and the security properties are all +testable locally. **Shielding, request collapsing under real concurrency, POP behaviour, +and stale revalidation are not** — those need a real Fastly service. Establish one before +Tasks 3–6, and be clear which findings came from which environment. - [ ] **Step 1: Provision a dedicated test service** @@ -169,30 +181,56 @@ The shielding answer also settles an open question from the Stage 0 findings: #1 off-TS win came from a shield HIT, so whether the test service has one determines whether its numbers transfer to production at all. -- [ ] **Step 2: Extend the harness for correlation** +- [ ] **Step 2: Extend the harness for lineage, not just correlation** + +The existing tester-cookie A/B has no way to join server timings to browser timings. A +root-only request ID is not enough either: under A3 the auction happens in a **fragment +subrequest**, so a root ID never reaches the auction telemetry. -The existing tester-cookie A/B has no way to join server timings to browser timings. Add -a per-request correlation ID — generated at TS entry, echoed in an `x-ts-request-id` -response header, and included in every timing log line. +Propagate a **lineage ID plus the experiment arm** through the whole chain: -Without it, the experiment cannot join hold time, origin time, auction telemetry, browser -TTFB, and render outcome for the same request. **That is the difference between an +``` +root request → C2 lookup → fragment subrequest → auction telemetry → browser render event +``` + +Generated at TS entry, forwarded into the fragment request, attached to the +`auction_events_raw` row, echoed as `x-ts-request-id`, and exposed to the browser harness +so render events carry it. Every timing log line includes both fields. + +Without this the experiment cannot join hold time, origin time, auction telemetry, browser +TTFB, and render outcome for the same pageview. **That is the difference between an experiment and a pile of numbers.** -- [ ] **Step 3: Capture cache tier and status per request** +- [ ] **Step 3: Capture C1 and C2 status separately** -Record `x-cache`, `hit-state`, `age`, and the serving POP alongside each measurement. A -median that mixes cold-MISS and warm-HIT requests is meaningless, and arms cannot be -compared unless the mix is known. +`x-cache`, `hit-state`, and `age` describe the **HTTP read-through cache (C1)**. They say +nothing about the **transformed-template cache (C2)**, which is a `cache::core` object +with no HTTP semantics. Recording only the former and calling it "cache status" would +attribute C2 hits and misses to the wrong tier. + +Emit both: the C1 headers as-is, plus an explicit `x-ts-c2` field carrying HIT / MISS / +STALE / BYPASS from the transaction outcome. Record the serving POP alongside. A median +that mixes cold-MISS and warm-HIT requests is meaningless, and arms cannot be compared +unless the mix is known — per tier. - [ ] **Step 4: Define the sample plan before collecting anything** -State, in the findings document, ahead of time: requests per arm per route, how cold MISS -is forced, how warm HIT is confirmed, and the confidence interval to be reported. +Write all of this into the findings document **before** the first measurement, and treat +it as fixed: + +| Element | What to state | +| -------------------- | -------------------------------------------------------------------------------------------------------------------------- | +| Allocation | Requests per arm per route, and how arms are assigned | +| Randomization | Randomized or blocked by route and cache state — not sequential runs | +| Pilot variance | A small pilot to estimate variance, before sizing the real run | +| MDE and power | The smallest difference worth detecting, and the N that detects it | +| CI method | Which interval, computed how | +| Warmup and carryover | How cold MISS is forced, how warm HIT is confirmed, and how one arm's cache state is prevented from contaminating the next | Rationale: this whole effort exists because #1009 drew a causal conclusion from N=4 that -did not survive contact with the code. Repeating that with more arms would be worse, not -better. +did not survive contact with the code. Repeating that with more arms and no power +calculation would be worse, not better — it would look rigorous while being equally +unfalsifiable. --- @@ -229,51 +267,127 @@ already documents: `Settings` carries `#[serde(deny_unknown_fields)]`, `ts confi typed, and `Publisher` has a hand-written `Default` plus eight exhaustive test literals and a live doctest. -- [ ] **Step 2: Emit markers instead of inlining, under `ClientFill`/`Esi`** +- [ ] **Step 2: Make the template strictly request-neutral** + +**The obvious design is wrong and would leak.** An earlier draft kept `tsjs.adSlots` in +the shared template on the grounds that it is per-URL. Its _content_ is per-URL; its +_presence_ is not. It is gated on `should_run_ad_stack` (`publisher.rs:2920-2927`), which +is `is_get && is_navigation && !is_prefetch && !is_bot && has_matched_slots && +consent_allows_auction && auction_enabled`. + +So the first request to fill C2 would freeze **its own** consent decision, bot +classification, prefetch status, and kill-switch state into an object every later visitor +reads. A consent-denied first fill serves a no-ads template to consenting users; a +consenting first fill serves ad markup to a user who refused. + +**Rule: the template contains an unconditional inert placeholder and nothing else.** + +| Element | Where it lives | +| ------------------------- | -------------------------------------------------- | +| tsjs bundle script tag | Template — content-hashed, genuinely per-URL | +| URL rewrites | Template — per-host, in the cache key | +| `tsjs.adSlots` | **Fragment** — its presence is request-dependent | +| `tsjs.bids` | **Fragment** | +| GPT diagnostics bootstrap | **Fragment** — gated on a per-request cookie/query | + +Emit **one** unconditional marker at the body-close seam, identical on every request that +reaches the transform. Under `Esi` it is an ``; under `ClientFill` it is +nothing at all, with the client fetching unprompted. + +- [ ] **Step 3: Bypass C2 for anything that must not be shared** + +`cache::core` is not an HTTP cache — it will happily store whatever you hand it. Nothing +rejects private or authenticated responses for you. Refuse to insert when **any** holds: + +- The origin response carries `Set-Cookie`. +- The origin response is `private`, `no-store`, or `no-cache`. +- The request carried `Authorization`. +- The response is not 200 with an HTML content type. +- DataDome's request filter replaced the document. -The two seams are already isolated — that is #1009's correct observation. At head-open, -`tsjs.adSlots` is **per-URL and stays in the template** (config- and path-derived only, -`publisher.rs:3501-3525`). At body-close, emit a marker instead of the bids script. +Audit every request-dependent rewrite before declaring the template neutral — the +integration head-inserts and the GPT-diagnostics bootstrap are both request-scoped and +must not reach C2. -Under `Esi`: ``. -Under `ClientFill`: nothing at all — **not an empty bids script.** The Stage 0 plan -explains why: an empty script calls `scheduleInitialAdInit({})` and assigns -`ts.bids = {}` synchronously, racing the client fetch. +**Assert it, do not assume it.** A unit test over the transform output must fail on any +of: a bid value, an EC ID, a consent string, a geo value, a diagnostics bootstrap, or a +`Set-Cookie`. Then a second test must assert the template is **byte-identical** for two +requests differing in consent, bot classification, and prefetch status. That second test +is the one that catches this class of bug; the first would have passed on the broken +design. -**Assert the template carries no per-user bytes.** A unit test over the transform output -must fail on any of: a bid value, an EC ID, a consent string, a geo value, or a -`Set-Cookie`. This is the test that makes C2 safe, and it is cheaper to write now than to -retrofit. +- [ ] **Step 4: Write and read C2 — with the real API** -- [ ] **Step 3: Write the template into C2** +The builder is move-based and the insert and read handles are different objects. Naïve +code does not compile: ```rust -// Fastly adapter. Key on the same signals the origin varies on, plus TS's own -// variant inputs. Surrogate-key it so rollback can purge rather than wait. -let mut insert = fastly::cache::core::insert(cache_key, template_ttl); -insert.surrogate_keys([&surrogate_key_for_url, "ts-template"]); -let mut body = insert.execute()?; -// stream the lol_html output into `body` +// WRONG — surrogate_keys consumes the builder and returns it; this discards the +// return value and then uses a moved binding. And execute() gives a WRITE stream, +// so there is nothing to read back from it. +let mut insert = cache::core::insert(key, ttl); +insert.surrogate_keys(["ts-template"]); +let body = insert.execute()?; ``` -Use `cache::core::Transaction` with `must_insert()` for the lookup, so a cold cache under -load transforms once rather than per concurrent request. +Correct shape, using a transaction so a cold cache under load transforms once: -**Cache key must include** everything the origin's `Vary` names — `rsc`, +```rust +use fastly::cache::core::{Transaction, CacheKey}; + +let tx = Transaction::lookup(CacheKey::from(key_bytes)).execute()?; + +let template: Body = if let Some(found) = tx.found() { + found.to_body() // C2 HIT — skip origin fetch and transform +} else if tx.must_insert_or_update() { + // C2 MISS. Fetch origin, transform, insert, and read our own bytes back in one + // pass: execute_and_stream_back gives both the write handle and a readable Found. + let (mut writer, found) = tx + .insert(template_ttl) + .surrogate_keys(["ts-template", &url_surrogate_key]) // chained, not discarded + .user_metadata(metadata_envelope) // see below + .execute_and_stream_back()?; + stream_lol_html_output_into(&mut writer)?; + writer.finish()?; // REQUIRED + found.to_body() +} else { + unreachable!("transaction must either find or be obliged to insert") +}; +``` + +`finish()` is not optional — without it the object never completes and its length stays +unknown. On any transform error, **cancel rather than finish**, or a partial template is +inserted and served to everyone until it expires. + +**`cache::core` carries no HTTP semantics.** Status, headers, content encoding, and +revalidation are all yours. Serialize what you need into `user_metadata` — at minimum the +content encoding, the transform schema version, and the origin `Vary` values the key was +built from — and decide explicitly whether the stored template is compressed. + +**Cache key must include**, beyond the origin's declared `Vary` (`rsc`, `next-router-state-tree`, `next-router-prefetch`, `next-router-segment-prefetch`, -`Accept-Encoding` (measured, see the Stage 0 findings) — **plus** TS's own per-variant -inputs: request host and scheme, the enabled-integration set, and the tsjs content hash. -Per-user signals must never appear in the key; they must be absent from the template -instead. If a signal cannot be excluded from the template, it does not belong in C2. +`Accept-Encoding` — measured, see the Stage 0 findings): -Set `template_ttl` deliberately short for the spike. A short TTL bounds every failure mode -here and costs only hit rate. +- The full URL, explicitly. Do not rely on an ambient request key. +- **The assembly mode.** A2 and A3 emit different template bytes and would otherwise + poison each other's entries. +- **A template schema version**, bumped whenever the transform changes, so a deploy does + not read yesterday's shape. +- Request host and scheme, the enabled-integration set, and the tsjs content hash. -- [ ] **Step 4: Read it back and assemble** +Per-user signals must never appear in the key. If a signal cannot be excluded from the +template, it does not belong in C2 at all. -On `found()`, skip the origin fetch and the transform entirely; hand the cached body to -the assembler. On miss, transform and insert as above, then assemble from what was -inserted. +**Design choice to make explicitly before writing code.** Two viable shapes: + +1. **Read-through with `after_send` + `set_body_transform`** — keeps HTTP semantics, + revalidation, and stale handling for free; less control over the key. +2. **`cache::core` as above** — full control; you own metadata, revalidation, and the + stale state machine. + +This plan assumes (2). If (1) is chosen, Step 4 is rewritten and the metadata envelope +disappears. Either way, the platform boundary must sit **before** the origin request, or +a C2 HIT cannot actually skip the fetch — which is the entire point. - [ ] **Step 5: Unit tests, then the target suite** @@ -328,12 +442,19 @@ it is the difference between a correct response and a leaked one. ```rust let config = esi::Configuration::default() - .with_escaped(false); -// default_dca and inherit_parent_dca stay at DcaMode::None / false — set them -// explicitly rather than relying on defaults; this is a pre-1.0 crate and the -// setting fails open. + .with_escaped(false) + .with_default_dca(esi::DcaMode::None) // call the setter; do not rely on the default + .with_inherit_parent_dca(false); ``` +Comments are not configuration. An earlier draft said DCA "stays at its default" — on a +pre-1.0 crate whose default could move in a patch release, and where this setting fails +**open**, that is not good enough. Call the setters. + +Also disable **fragment caching** explicitly, or mark the include `no-store="on"`. A +cached auction fragment is a per-user object in a shared cache — the C3 failure mode by +another route. + The dispatcher must be **exact-path allowlisted**: a fragment URL that is not the bids endpoint is refused, not fetched. The built-in dispatcher builds a dynamic backend per URL host and panics on a hostless URL — never use it. @@ -343,20 +464,57 @@ recursive parse would let an SSP make the edge fetch an arbitrary URL. **Add a u that feeds `` through a creative payload and asserts no fetch is attempted.** -- [ ] **Step 3: Deterministic synthetic fragment first** +- [ ] **Step 3: The fragment must be a script, not the JSON endpoint** + +**`/_ts/page-bids` cannot be the ESI target.** It returns +`serde_json::json!({"slots":…, "bids":…})` (`publisher.rs:3987`), and ESI splices fragment +bytes in literally — the page would contain raw JSON where an executable script belongs. +Nothing would call `scheduleInitialAdInit`. + +Add a **dedicated fragment endpoint** returning the executable script — the same shape +`build_bids_script` produces today, plus the `adSlots` assignment that moved out of the +template in Task 3 Step 2. Either that, or use the `esi` crate's fragment-response +processor to wrap the JSON; the dedicated endpoint is simpler and easier to assert on. + +Three more things the naïve marker gets wrong: + +- **The same-origin gate will reject it.** `page_bids_request_allowed` + (`publisher.rs:3644`) requires `Sec-Fetch-Site: same-origin` or the `X-TSJS-Page-Bids` + header. An internal ESI subrequest carries neither. Give the fragment endpoint an + internal contract and a fixed backend rather than weakening that gate — it exists to + stop third parties burning SSP quota. +- **Parent context does not propagate.** EC identity, consent state, client IP, geo, User + Agent, and the correlation ID all live on the parent request. Forward an **explicitly + approved allowlist** of them into the fragment request. Forwarding everything is how a + fragment ends up more privileged than the parent. +- **Root dispatch must be suppressed.** The navigation path already dispatches an + auction. If A3 does not suppress it, every pageview runs two — doubling SSP and APS + spend. This applies to **A2 and A3 alike**. + +- [ ] **Step 4: Validate the whole URL, not the path** + +An exact-path allowlist alone permits `https://attacker.example/_ts/page-bids`. Validate +**scheme, authority, method, path, and query** — or better, ignore the marker's URL +entirely and dispatch to a fixed internal backend, treating the `esi:include` as a signal +rather than an address. + +Add a test that feeds `` +through a creative payload and asserts no outbound fetch is attempted. + +- [ ] **Step 5: Deterministic synthetic fragment first** -Before the real auction, point the include at a fixed-content endpoint. This separates -"does the pipeline assemble correctly" from "does the auction behave," and the two fail -very differently. Only once assembly is proven does the include move to -`/_ts/page-bids`. +Before wiring the real auction, point the include at a fixed-content endpoint. This +separates "does the pipeline assemble correctly" from "does the auction behave," and the +two fail very differently. Only once assembly is proven does the fragment become the real +one. -- [ ] **Step 4: Handle the flush hazard** +- [ ] **Step 6: Handle the flush hazard** `esi` flushes its output writer after each parse batch. Fastly's `StreamingBody` is a `BufWriter`, so anything between esi and it must propagate `flush()` or nothing leaves the Wasm heap. -- [ ] **Step 5: Fragment failure must degrade, not break** +- [ ] **Step 7: Fragment failure must degrade, not break** Assert that a fragment timeout or non-2xx yields a page with empty bids rather than a 5xx or a truncated document. Note the crate's non-obvious semantics: `alt` is attempted before @@ -385,9 +543,14 @@ Not a phase. Every one of these is a hard fail, independent of any performance r renders attributed. Use TS-attributed renders — the SSAT line item, non-empty `ts.bids`, `hb_adid` presence — **never slot fill**, which is blind to empty bids because `adInit` defines slots regardless. -- [ ] **No C3.** Assert the final assembled response is never shared-cacheable: no - `public`, no `s-maxage`, no `Surrogate-Control` on a response carrying per-user - state. +- [ ] **No C3 — assert positively, not by absence.** Forbidding `public`, `s-maxage`, and + `Surrogate-Control` is **not sufficient**: a bare `Cache-Control: max-age=60` passes + that check and is still shared-cacheable, and that is exactly what the measured + origin sends. Require instead that every assembled response carries + `Cache-Control: private, no-store` and that `Expires`, `ETag`, `Last-Modified`, and + all four CDN cache directives are stripped. Test it for **returning** users + specifically — they set no EC cookie, so the cookie privacy net never fires and is + not a backstop here. --- diff --git a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md index 1417c5122..a3a35002a 100644 --- a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md +++ b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md @@ -91,9 +91,9 @@ could embed `` and make the edge fetch an arbitrary URL. [Appendix E](#appendix-e--esi-notes-condensed). **The auction is already out of band; the hold is ~free.** It is dispatched _before_ -the origin fetch and does not block — dispatched at [publisher.rs:2751-2755](../../../crates/trusted-server-core/src/publisher.rs#L2751-L2755), sent at [:2870](../../../crates/trusted-server-core/src/publisher.rs#L2870) — +the origin fetch and does not block — dispatched at `publisher.rs:2751-2755`, sent at `:2870` — with a 500 ms budget. The actual cost is `with_cache_bypass` -([publisher.rs:2867](../../../crates/trusted-server-core/src/publisher.rs#L2867)), +(`publisher.rs:2867`), which forces every ad-eligible navigation to miss the Fastly readthrough cache. **The two fixes are multiplicative.** Removing the bypass alone lets the previously @@ -122,14 +122,14 @@ cheapest thing that unblocks anything. **Step B — what consumes TS's own response headers (under a day).** Request a TS-served path that already emits `public, s-maxage` -([http_util.rs:294-311](../../../crates/trusted-server-core/src/http_util.rs#L294-L311)) +(`http_util.rs:294-311`) twice and look for `x-cache`/`age` on TS's _own_ response. **Gates the Stage 3a/3b split** — see [§7](#7-deferred-work-specified-not-scheduled). **Step C — measure the hold directly (1 day + a measurement window).** The hold's cost is literally the duration of one `.await`: `collect_stream_auction` at -[publisher.rs:793](../../../crates/trusted-server-core/src/publisher.rs#L793), plus the +`publisher.rs:793`, plus the two EOF variants in `hold_finish_ready_segments` and `hold_finish_tail_segments`. Two `Instant`s around it yield **`hold_wait_ms`** — the number this entire document is arguing about, measured rather than modelled. @@ -182,7 +182,7 @@ browser harness when Stage 1 is actually scheduled. ## 4. Stage 0 — the only build item recommended now Stop bypassing the read-through cache on ad-eligible navigations -([publisher.rs:2867](../../../crates/trusted-server-core/src/publisher.rs#L2867)). +(`publisher.rs:2867`). **Ship it as an operator flag, not a deletion.** Add `publisher.bypass_origin_cache`, defaulting to today's behaviour, in the same release as @@ -191,7 +191,9 @@ the Step C instrumentation. Then turn it off with `ts config push`. The diff is slightly larger than deleting a line, and that is the point. The risk being gated here is **cache poisoning** — serving one representation in response to a request for another. For that class of failure, rollback speed dominates diff size: a config push -reverts in seconds, a release does not. The flag also buys an A/B on a byte-identical +reverts the read path in seconds where a release does not — but a config push **evicts +nothing**, so full rollback is flip, then purge or roll a versioned key namespace, then +observe past the origin TTL. The flag also buys an A/B on a byte-identical build, removing build difference as a confound in the very measurement this depends on, and allows flipping for a tester-cookie population before all traffic. @@ -203,7 +205,7 @@ its branch. A temporary flag left in place becomes permanent configuration surfa Two regression signals, both checked before the win is: - **`unexpected_origin_304` abandonment rate.** That reason - ([publisher.rs:2894-2916](../../../crates/trusted-server-core/src/publisher.rs#L2894-L2916), + (`publisher.rs:2894-2916`, emitted via `emit_abandoned_auction` at `:2360`) exists precisely because the ad-stack path refuses cached and conditional origin responses. Re-enabling the cache is what could revive it. **Any non-zero rate is a rollback signal** — it means a 304 is reaching @@ -214,7 +216,7 @@ Two regression signals, both checked before the win is: performance regression. **Why it is safe in principle.** The conditional-header strip runs 34 lines earlier -under the same gate ([publisher.rs:2832-2836](../../../crates/trusted-server-core/src/publisher.rs#L2832), +under the same gate (`publisher.rs:2832-2836`, which also strips `Range`/`If-Range`), so the request already reaches the cache unconditional and a HIT returns a full body. [The 304-prevention design](./2026-07-22-ssat-root-document-304-prevention-design.md) added the bypass as belt-and-braces and listed the TTFB cost under its own Risks. The @@ -222,7 +224,7 @@ strip alone satisfies its invariant. **But it carries a risk that design never considered — and this is the blocking precondition.** RSC fetches are not navigations -([is_navigation_request](../../../crates/trusted-server-core/src/http_util.rs#L73-L98) +(`is_navigation_request` requires `Sec-Fetch-Dest: document`), so they never set the bypass and **already flow through the readthrough cache**, while HTML navigations are `PASS`. Removing the bypass puts both representations under one cache key. #1009 states the origin varies on @@ -233,7 +235,7 @@ cache can serve a flight payload to an HTML navigation. The classification is also not airtight: `is_navigation_request` falls back to the `Accept` header when Fetch Metadata is absent, and its own comment warns _"this path is weaker — `fetch()` can set Accept: text/html"_ -([http_util.rs:84-88](../../../crates/trusted-server-core/src/http_util.rs#L84)). +(`http_util.rs:84-88`). **A FAIL is not merely a Stage 0 blocker — it is a live production defect.** RSC fetches already transit the read-through cache today, because they never set the bypass. If the @@ -285,13 +287,13 @@ The hold is load-bearing for something other than latency. The invariant is: > `ad_bids_state` must be `Some(..)` when `lol_html` processes the `` end tag. -The end-tag handler ([html_processor.rs:381-395](../../../crates/trusted-server-core/src/html_processor.rs#L381-L395)) +The end-tag handler (`html_processor.rs:381-395`) locks that mutex once and falls back to `build_empty_bids_script()` on `None`. **Removing the hold without relocating collection renders a normal page with `tsjs.bids = {}` and no server-side ads** — no error, no non-2xx, no ERROR log. On Axum, Cloudflare, and Spin the loss is fully silent: -[publisher.rs:2248](../../../crates/trusted-server-core/src/publisher.rs#L2248) holds a +`publisher.rs:2248` holds a bare `Option` with no guard, so not even a drop warning fires. **The SSPs are billed regardless.** @@ -304,14 +306,14 @@ fill cannot be the canary — see [§7](#7-deferred-work-specified-not-scheduled ### 6.1 Corrections to #1009's premises -| # | #1009 states | Verified against `cfb98f4` | -| --- | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 1 | Two per-user injection seams | `tsjs.adSlots` is per-URL — [build_slot_json](../../../crates/trusted-server-core/src/publisher.rs#L3501-L3525) emits config- and path-derived fields only. **One per-user hole.** | -| 2 | Identity off-inline is a prerequisite | Privacy net is cookie-gated and returning navs set no cookie ([ec/finalize.rs:86-94](../../../crates/trusted-server-core/src/ec/finalize.rs#L86-L94)). **First-visit only.** | -| 3 | Stamp at `:2882-2888` | [`:2945-2963`](../../../crates/trusted-server-core/src/publisher.rs#L2945-L2963), `private, no-store`, also removing `ETag`/`Last-Modified`/four CDN headers. **Seven headers.** | -| 4 | Three cacheability killers | Two more: `bypass_cache` and the [304→502 guard](../../../crates/trusted-server-core/src/publisher.rs#L2894-L2916). **The bypass is the cost.** | -| 5 | Two `!Send` pipelines is the hard part | `?Send` already pervasive. **Not the obstacle.** | -| 6 | Goal: root as a shared Fastly HIT | Nothing caches TS's own response on Compute; the A/B's `x-cache` is the **backend readthrough** cache. **Reframes the goal.** | +| # | #1009 states | Verified against `cfb98f4` | +| --- | -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | Two per-user injection seams | `tsjs.adSlots` is per-URL — `build_slot_json` emits config- and path-derived fields only. **One per-user hole.** | +| 2 | Identity off-inline is a prerequisite | Privacy net is cookie-gated and returning navs set no cookie (`ec/finalize.rs:86-94`). **First-visit only** — but note the corollary: because returning navigations set no cookie, that net never fires for them and is **not** a backstop against shared-caching a per-user response. | +| 3 | Stamp at `:2882-2888` | `:2945-2963`, `private, no-store`, also removing `ETag`/`Last-Modified`/four CDN headers. **Seven headers.** | +| 4 | Three cacheability killers | Two more: `bypass_cache` and the `304→502 guard`. **The bypass is the cost.** | +| 5 | Two `!Send` pipelines is the hard part | `?Send` already pervasive. **Not the obstacle.** | +| 6 | Goal: root as a shared Fastly HIT | Nothing caches TS's own response on Compute; the A/B's `x-cache` is the **backend readthrough** cache. **Reframes the goal.** | Rows on #1009's `esi` compatibility check (holds), its drifted line numbers, and its two broken `#1`/`#3` cross-references are in [Appendix A](#appendix-a--full-1009-correction-table). @@ -327,9 +329,9 @@ argument for client-fill, which the issue then declines in favour of ESI. It was wrong, and the correction matters.** The hold does not key off `lol_html` at all. `BodyCloseHoldBuffer::push` -([publisher.rs:2190-2202](../../../crates/trusted-server-core/src/publisher.rs#L2190)) +(`publisher.rs:2190-2202`) scans the **decoded origin input** for `` while the auction rides alongside transfer" — is **inert on a Next.js publisher**. Every `step.ready` yields empty bytes. That comment is misleading on exactly the publisher @@ -481,7 +483,7 @@ Secondary wins: removes the duplicated per-codec decoder/encoder wiring, six com imports, and the non-parser-context ` Date: Mon, 10 Aug 2026 16:06:46 +0530 Subject: [PATCH 07/44] Reconcile the #1009 documents with the request-neutrality correction MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Six blockers from review, all verified against the source before fixing. The three-verdict Stage 0 gate was only half propagated. The findings template still offered PASS/FAIL and routed PASS straight to the flip, and the spec still approved Stage 0 on the Vary check alone. Both now use FINAL PASS / PROVISIONAL PASS / FAIL, and Task 5a is titled for FINAL PASS so the gate cannot be read past. The spec contradicted the spike on request-neutrality, which would have recreated the leakage bug the spike exists to avoid. It still described adSlots as per-URL, kept it in the template, and drew two markers. New section 6.7 gives the rule: content is per-URL, presence is gated on should_run_ad_stack and is therefore per-request, so it must live in the fragment. The correction-table row, the pipeline diagram, the disposition table and the appendix all point at it. The Core Cache example still would not compile and mishandled stale entries. It called Found::to_body, which does not exist — the accessor is to_stream and it is fallible. Worse, it tested found() before must_insert_or_update(), but a stale entry sets both: that ordering serves stale bytes and never fulfils the update obligation, leaving concurrent waiters blocked. Reordered, with abandon plus cancel_insert_or_update on transform failure and an explicit note that the stale state machine is the caller's to write. The finalization order was impossible. The plan streamed ESI output into the client body while claiming EC, geo and privacy headers finalize afterwards; streaming responses on this adapter commit headers first and then pipe chunks. The invariant is now stated the only way it can work: finalize every header, including an unconditional private/no-store, before any body byte is written. The decision rule adopted A3 on the metric the same document forbids. A2 and A3 serve the same template, so root TTFB is near-identical by construction. The rule now turns on bids-ready, adInit fire and first attributed creative paint, with root TTFB kept only as a non-regression guard. Added a request-scoped arm allocator, since a global setting yields sequential blocks and confounds arm with time of day and cache warmth. Operational: Stage 0's rollback pointed at Core Cache surrogate keys, which belong to the transformed-template cache the spike builds and have no effect on the HTTP read-through cache Stage 0 turns on. Purging that needs origin-supplied keys or the HTTP cache's own surface, and until one exists the rollback is waiting out the origin TTL — now recorded as an accepted risk rather than a discovery during an incident. --- ...2026-08-08-1009-measurement-and-stage-0.md | 40 ++++-- .../2026-08-10-1009-esi-validation-spike.md | 95 ++++++++++---- ...08-esi-cacheable-root-validation-design.md | 122 +++++++++++------- 3 files changed, 177 insertions(+), 80 deletions(-) diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md index 6298ca2c6..7a6d79a89 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md @@ -237,15 +237,22 @@ Create `docs/superpowers/plans/2026-08-08-1009-measurement-findings.md`: **Cookie / auth exposure:** bodies differ by cookie? `Vary: Cookie` present? origin `Set-Cookie` on a shared-cacheable response? `Authorization`-gated responses cacheable? -**Verdict:** PASS / FAIL +**Verdict:** FINAL PASS / PROVISIONAL PASS / FAIL -PASS = `Vary` names every request header the origin varies on (`RSC`, any `Next-Router-*` -or experiment header whose value changed the body, **and `Cookie` if bodies differ by -cookie**), and no `Set-Cookie` rides a shared-cacheable response. -FAIL = any of the above is unmet. +`FINAL PASS` = `Vary` names every request header the origin varies on (`RSC`, any +`Next-Router-*` or experiment header whose value changed the body, **and `Cookie` if +bodies differ by cookie**), no `Set-Cookie` rides a shared-cacheable response, **and** all +five conditions in Task 5's gate are recorded — a real authenticated session cookie, Basic +Auth through TS, the experiment variant, representative routes, and cached-hit +slot/render attribution. -**Consequence:** PASS → Task 5a (flip the flag). FAIL → Task 5b (cache-key -discriminator). See spec §4. +`PROVISIONAL PASS` = the `Vary` and cookie checks hold, but one or more of those five is +untested. **Not a release gate.** A first pass lands here. + +`FAIL` = any `Vary` or `Set-Cookie` criterion is unmet. + +**Consequence:** `FINAL PASS` → Task 5a (flip the flag). `PROVISIONAL PASS` → close the +gaps before Task 5 starts. `FAIL` → Task 5b (cache-key discriminator). See spec §4. **A FAIL is also a live production defect, not only a Stage 0 blocker.** RSC fetches are not navigations, so they never set the bypass and **already transit the read-through @@ -778,7 +785,7 @@ already-deployed build**. No second release, and rollback is another config push than a revert. That matters here specifically: the failure mode this gates on is cache poisoning, where minutes of exposure are worse than a slow rollout. -### Task 5a: flip the flag (Task 1 verdict = PASS) +### Task 5a: flip the flag (Task 1 verdict = `FINAL PASS`) **Files:** @@ -896,11 +903,18 @@ expire. The origin's `max-age=60` bounds that, but does not remove it. Full rollback: 1. Push `bypass_origin_cache = true`. -2. Purge. `fastly::http::purge::purge_surrogate_key` runs inside Compute, with keys - attached at insert via `InsertBuilder::surrogate_keys`; alternatively roll a versioned - cache-key namespace. **Neither is wired today** — if the flip ships before one exists, - the rollback story is "wait out the TTL," and that must be an accepted risk rather - than an unnoticed one. +2. Purge — **and note this is C1, not C2.** `InsertBuilder::surrogate_keys` belongs to + the Core Cache API and applies to the transformed-template cache the ESI spike builds. + It has no effect on the HTTP read-through cache that Stage 0 turns on. Purging C1 + requires either surrogate keys the **origin** supplies on its responses, or the HTTP + cache's own request/candidate surrogate-key surface. Confirm which is available before + relying on it. + + **Neither is wired today.** If the flip ships before one exists, the rollback story is + "wait out the origin TTL" — roughly a minute, per the Step A findings. That is + survivable, but it must be an accepted risk recorded before the flip rather than a + discovery during an incident. + 3. Observe past the origin TTL before declaring the incident closed. - [ ] **Step 5: Run the full suite across every adapter** diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index 770cb05cc..befae1761 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -213,7 +213,18 @@ STALE / BYPASS from the transaction outcome. Record the serving POP alongside. A that mixes cold-MISS and warm-HIT requests is meaningless, and arms cannot be compared unless the mix is known — per tier. -- [ ] **Step 4: Define the sample plan before collecting anything** +- [ ] **Step 4: Build a request-scoped arm allocator** + +`AssemblyMode` as specified in Task 3 is a **global** setting, but the sample plan below +requires randomized, non-sequential allocation. A global flip gives sequential blocks +instead, which confounds arm with time of day, cache warmth, and traffic mix. + +Allocate per request: hash the lineage ID into buckets, or key off the tester cookie. +The global setting stays as the kill switch and as the way to force a single arm; the +allocator is what the experiment actually uses. Record the assigned arm on every log line +and every telemetry row. + +- [ ] **Step 5: Define the sample plan before collecting anything** Write all of this into the findings document **before** the first measurement, and treat it as fixed: @@ -337,27 +348,48 @@ use fastly::cache::core::{Transaction, CacheKey}; let tx = Transaction::lookup(CacheKey::from(key_bytes)).execute()?; -let template: Body = if let Some(found) = tx.found() { - found.to_body() // C2 HIT — skip origin fetch and transform -} else if tx.must_insert_or_update() { - // C2 MISS. Fetch origin, transform, insert, and read our own bytes back in one - // pass: execute_and_stream_back gives both the write handle and a readable Found. - let (mut writer, found) = tx - .insert(template_ttl) - .surrogate_keys(["ts-template", &url_surrogate_key]) // chained, not discarded - .user_metadata(metadata_envelope) // see below - .execute_and_stream_back()?; - stream_lol_html_output_into(&mut writer)?; - writer.finish()?; // REQUIRED - found.to_body() +// Order matters: a STALE entry sets BOTH found() and must_insert_or_update(). +// Testing found() first would serve the stale bytes and silently never fulfil the +// update obligation, leaving every concurrent waiter blocked until timeout. +let template: Body = if tx.must_insert_or_update() { + match transform_origin_into(&tx) { + Ok((writer, found)) => { + writer.finish()?; // REQUIRED — without it the object never completes + found.to_stream()? // fallible; there is no `to_body()` + } + Err(e) => { + // Do NOT finish() a partial template — it would be served to everyone + // until it expires. Abandon the writer, release the obligation so another + // client can try, and fall back to the untransformed path for this request. + writer.abandon(); + tx.cancel_insert_or_update()?; + return fallback_uncached(e); + } + } +} else if let Some(found) = tx.found() { + // Fresh hit. `is_usable()` and `is_stale()` are available if a stale-serve + // policy is wanted; the spike should start by treating stale as a miss. + found.to_stream()? // C2 HIT — skip origin fetch and transform } else { - unreachable!("transaction must either find or be obliged to insert") + unreachable!("a transaction is either obliged to insert or has found an item") }; ``` -`finish()` is not optional — without it the object never completes and its length stays -unknown. On any transform error, **cancel rather than finish**, or a partial template is -inserted and served to everyone until it expires. +Inside `transform_origin_into`, `execute_and_stream_back()` yields both handles at once: + +```rust +let (mut writer, found) = tx + .insert(template_ttl) + .surrogate_keys(["ts-template", &url_surrogate_key]) // chained, not discarded + .user_metadata(metadata_envelope) + .execute_and_stream_back()?; +``` + +**Decide the stale policy explicitly.** `Found::is_stale()` and `is_usable()` exist, and +`stale_while_revalidate` can be set at insert. Serving stale while revalidating is a real +option — but it is a state machine, and `cache::core` implements none of it for you. The +spike should start by treating stale as a miss and only add stale-serve if the numbers +justify it. **`cache::core` carries no HTTP semantics.** Status, headers, content encoding, and revalidation are all yours. Serialize what you need into `user_metadata` — at minimum the @@ -434,9 +466,23 @@ for the silent-empty-bids trap, which applies in full. themselves, which takes ownership away from the finalize / `ec_finalize` / apply-effects ordering. `process_stream(&mut self, src: impl BufRead, out: &mut impl Write, …)` keeps it. -Source is the C2 body. Sink is the response body on the way to the client — so EC cookie, -geo, and the privacy net still run **after** assembly. Confirm that ordering explicitly; -it is the difference between a correct response and a leaked one. +Source is the C2 body. Sink is the client response body. + +**The ordering an earlier draft described is impossible.** It said EC cookie, geo, and the +privacy net run _after_ assembly. They cannot: streaming responses on this adapter +**commit headers first and then pipe chunks** +(`adapter-fastly/src/main.rs`, `send_edgezero_response`). Once ESI starts writing, no +header can change. + +The correct invariant: + +> **Finalize every header before a single body byte is written** — EC `Set-Cookie`, geo +> suppression, and an unconditional `Cache-Control: private, no-store` — **then** stream +> the assembly with no further header mutation. + +That means `private, no-store` is set unconditionally up front rather than derived from +what the assembly turns out to contain. Deriving it after the fact is not available, and +assuming it was is how a per-user response ends up shared-cacheable. - [ ] **Step 2: Disable DCA explicitly and allowlist the dispatcher** @@ -566,8 +612,11 @@ Not a phase. Every one of these is a hard fail, independent of any performance r **Adopt ESI only if all three hold:** 1. Every Task 6 gate passes on A3. -2. A3 beats A2 on TTFB by a margin the reviewers ratify **before** collection — not - chosen after seeing the numbers. +2. A3 beats A2 on **bids-ready time, `adInit` fire time, and first TS-attributed creative + paint** — by a margin the reviewers ratify **before** collection, not chosen after + seeing the numbers. **Not root TTFB:** A2 and A3 serve the same C2 template, so their + root timings are near-identical by construction and a difference there would be noise. + Root TTFB is a non-regression guard only. 3. Render outcomes on A3 are non-inferior to A0. **Otherwise adopt A2 (client-fill)** if its gates pass and it beats A1. It is portable diff --git a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md index a3a35002a..446ff1727 100644 --- a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md +++ b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md @@ -306,14 +306,14 @@ fill cannot be the canary — see [§7](#7-deferred-work-specified-not-scheduled ### 6.1 Corrections to #1009's premises -| # | #1009 states | Verified against `cfb98f4` | -| --- | -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 1 | Two per-user injection seams | `tsjs.adSlots` is per-URL — `build_slot_json` emits config- and path-derived fields only. **One per-user hole.** | -| 2 | Identity off-inline is a prerequisite | Privacy net is cookie-gated and returning navs set no cookie (`ec/finalize.rs:86-94`). **First-visit only** — but note the corollary: because returning navigations set no cookie, that net never fires for them and is **not** a backstop against shared-caching a per-user response. | -| 3 | Stamp at `:2882-2888` | `:2945-2963`, `private, no-store`, also removing `ETag`/`Last-Modified`/four CDN headers. **Seven headers.** | -| 4 | Three cacheability killers | Two more: `bypass_cache` and the `304→502 guard`. **The bypass is the cost.** | -| 5 | Two `!Send` pipelines is the hard part | `?Send` already pervasive. **Not the obstacle.** | -| 6 | Goal: root as a shared Fastly HIT | Nothing caches TS's own response on Compute; the A/B's `x-cache` is the **backend readthrough** cache. **Reframes the goal.** | +| # | #1009 states | Verified against `cfb98f4` | +| --- | -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | Two per-user injection seams | Partly. `tsjs.adSlots` **content** is per-URL — `build_slot_json` emits config- and path-derived fields only. But its **presence** is gated on `should_run_ad_stack` (consent, bot, prefetch, kill switch), so it is request-dependent and **must not live in a shared template**. See §6.7. | +| 2 | Identity off-inline is a prerequisite | Privacy net is cookie-gated and returning navs set no cookie (`ec/finalize.rs:86-94`). **First-visit only** — but note the corollary: because returning navigations set no cookie, that net never fires for them and is **not** a backstop against shared-caching a per-user response. | +| 3 | Stamp at `:2882-2888` | `:2945-2963`, `private, no-store`, also removing `ETag`/`Last-Modified`/four CDN headers. **Seven headers.** | +| 4 | Three cacheability killers | Two more: `bypass_cache` and the `304→502 guard`. **The bypass is the cost.** | +| 5 | Two `!Send` pipelines is the hard part | `?Send` already pervasive. **Not the obstacle.** | +| 6 | Goal: root as a shared Fastly HIT | Nothing caches TS's own response on Compute; the A/B's `x-cache` is the **backend readthrough** cache. **Reframes the goal.** | Rows on #1009's `esi` compatibility check (holds), its drifted line numbers, and its two broken `#1`/`#3` cross-references are in [Appendix A](#appendix-a--full-1009-correction-table). @@ -416,9 +416,10 @@ tags that do not exist yet. The correct order: ``` origin → lol_html transform → fastly::cache::core → esi assemble → finalize → client - (emit esi:include at the (shared template, (per request, (EC cookie, - head + body-close seams, surrogate-keyed, fetch the geo, privacy - no per-user data) TS-chosen TTL) bids fragment) net) + (one unconditional marker (shared template, (per request, headers are + at the body-close seam, surrogate-keyed, fetch the finalized + no per-user data and no TS-chosen TTL) fragment) BEFORE the + request-dependent decisions) body streams ``` The push/pull mismatch that the earlier revision treated as a blocker is real but @@ -453,6 +454,39 @@ stale / shielding behaviour must run against a real Fastly test service. --- +### 6.7 What may and may not live in a shared template + +A correction to §6.1 row 1, and the constraint that governs any shared-template design. + +The original framing — "`adSlots` is per-URL, so there is one per-user hole, not two" — +is half right and dangerously so. `build_slot_json` really does emit only config- and +path-derived fields. But whether the script is emitted **at all** is gated on +`should_run_ad_stack` (`publisher.rs:2920-2927`), which is +`is_get && is_navigation && !is_prefetch && !is_bot && has_matched_slots && +consent_allows_auction && auction_enabled`. + +So the _content_ is per-URL and the _presence_ is per-request. A shared object filled by +the first request would freeze that request's consent decision, bot classification, +prefetch status, and kill-switch state for every later reader. A consent-denied fill +serves a no-ads template to consenting users; a consenting fill serves ad markup to +someone who refused. + +**The rule for anything cached and shared:** + +| May live in the template | Must live in the per-request fragment | +| --------------------------------------- | ---------------------------------------------- | +| tsjs bundle script tag (content-hashed) | `tsjs.adSlots` — presence is request-gated | +| URL rewrites (per-host, in the key) | `tsjs.bids` | +| | GPT diagnostics bootstrap (cookie/query-gated) | +| | Integration head-inserts (request-scoped) | + +The test that catches this class is **byte-identity of the template across requests +differing in consent, bot classification, and prefetch status** — not an absence-of- +per-user-values scan, which the broken design would have passed. + +This applies to any shared-template work, ESI or client-fill alike. The +[spike plan](../plans/2026-08-10-1009-esi-validation-spike.md) implements it. + --- ## 7. Deferred work, specified not scheduled @@ -600,7 +634,7 @@ Rows 1–6 are in [§6.1](#61-corrections-to-1009s-premises). The remainder: | `2832-2836` | strip `If-None-Match`, `If-Modified-Since`, `Range`, `If-Range` | **keep** — needed for any injection | | `2866` | `with_cache_bypass()` | **make operator-controlled** — [§4](#4-stage-0--the-only-build-item-recommended-now) | | `2894` | 304 → 502 guard | keep as safety net | -| `2920` | build `adSlots` | keep — per-URL | +| `2920` | build `adSlots` | keep, but see §6.7 — presence is request-dependent | | `2945` | strip cacheability | **replace** — Stage 3a | **Cache tiers.** T1 backend readthrough (already available; TS opts out). T2 TS-owned @@ -744,35 +778,35 @@ queued before it loads (#1009 Part 1) — not filed, should be. All pinned to `cfb98f4`. -| Concern | Location | -| -------------------------------------------- | ------------------------------------------------------------------------------- | -| Eligibility decision | `publisher.rs:2651`, `:2660` | -| `is_navigation_request` | `http_util.rs:73-98` | -| Auction dispatch (pre-origin, non-blocking) | `publisher.rs:2698-2760` | -| Auction overlap intent | `auction/orchestrator.rs:950-952` | -| Auction timeout resolution | `publisher.rs:2680-2684` (creative_opportunities, else auction.timeout_ms) | -| Conditional/range header strip | `publisher.rs:2832-2836` | -| Origin cache bypass | `publisher.rs:2866-2868` | -| Origin 304 → 502 guard | `publisher.rs:2894-2916` | -| `adSlots` build (per-URL) | `publisher.rs:2920`, `:3558-3577`, `:3501-3525` | -| Uncacheable stamp | `publisher.rs:2945-2963` | -| `` hold — sync / async / Fastly lazy | `publisher.rs:2235` / `:2109` / `:1318-1390` | -| Hold buffer | `publisher.rs:2177-2218` | -| Auction collect (HTML / non-HTML) | `publisher.rs:2431` (emits `:2456`) / `:2388` (emits `:2410`) | -| Abandonment emitter | `publisher.rs:2360` | -| Bids script build | `publisher.rs:3438-3491` | -| `/_ts/page-bids` | `publisher.rs:3611`, handler `:3723`, auction `:3903` | -| Injection seams (head / body-close) | `html_processor.rs:310-363` / `:381-395` | -| Post-processor buffering | `html_processor.rs:62-94` | -| Next.js post-processor registration | `integrations/nextjs/mod.rs:107` | -| Max buffered body (16 MB) | `settings.rs:77-79` | -| EC cookie issuance policy | `ec/finalize.rs:86-107` | -| Cookie-privacy net | `response_privacy.rs:20-61` | -| Geo response headers | `adapter-fastly/src/middleware.rs:194-200` | -| Cacheable-header precedent | `http_util.rs:294-311` | -| Management token lacks purge (wrong surface) | `adapter-fastly/src/management_api.rs:12` | -| In-process purge / surrogate keys | `fastly` 0.12.1 `cache::core`, `http::purge::purge_surrogate_key` | -| Client initial-ad gate | `js/lib/src/integrations/gpt/index.ts:536-555` | -| `adInit` bid application | `js/lib/src/integrations/gpt/index.ts:566`, `:652`, `:657-661` | -| Client SPA auction hook | `js/lib/src/integrations/gpt/index.ts:806`, `:859`, `:892-949` | -| Prior design that introduced the killers | `docs/superpowers/specs/2026-07-22-ssat-root-document-304-prevention-design.md` | +| Concern | Location | +| --------------------------------------------------------- | ------------------------------------------------------------------------------- | +| Eligibility decision | `publisher.rs:2651`, `:2660` | +| `is_navigation_request` | `http_util.rs:73-98` | +| Auction dispatch (pre-origin, non-blocking) | `publisher.rs:2698-2760` | +| Auction overlap intent | `auction/orchestrator.rs:950-952` | +| Auction timeout resolution | `publisher.rs:2680-2684` (creative_opportunities, else auction.timeout_ms) | +| Conditional/range header strip | `publisher.rs:2832-2836` | +| Origin cache bypass | `publisher.rs:2866-2868` | +| Origin 304 → 502 guard | `publisher.rs:2894-2916` | +| `adSlots` build (content per-URL, presence request-gated) | `publisher.rs:2920`, `:3558-3577`, `:3501-3525` | +| Uncacheable stamp | `publisher.rs:2945-2963` | +| `` hold — sync / async / Fastly lazy | `publisher.rs:2235` / `:2109` / `:1318-1390` | +| Hold buffer | `publisher.rs:2177-2218` | +| Auction collect (HTML / non-HTML) | `publisher.rs:2431` (emits `:2456`) / `:2388` (emits `:2410`) | +| Abandonment emitter | `publisher.rs:2360` | +| Bids script build | `publisher.rs:3438-3491` | +| `/_ts/page-bids` | `publisher.rs:3611`, handler `:3723`, auction `:3903` | +| Injection seams (head / body-close) | `html_processor.rs:310-363` / `:381-395` | +| Post-processor buffering | `html_processor.rs:62-94` | +| Next.js post-processor registration | `integrations/nextjs/mod.rs:107` | +| Max buffered body (16 MB) | `settings.rs:77-79` | +| EC cookie issuance policy | `ec/finalize.rs:86-107` | +| Cookie-privacy net | `response_privacy.rs:20-61` | +| Geo response headers | `adapter-fastly/src/middleware.rs:194-200` | +| Cacheable-header precedent | `http_util.rs:294-311` | +| Management token lacks purge (wrong surface) | `adapter-fastly/src/management_api.rs:12` | +| In-process purge / surrogate keys | `fastly` 0.12.1 `cache::core`, `http::purge::purge_surrogate_key` | +| Client initial-ad gate | `js/lib/src/integrations/gpt/index.ts:536-555` | +| `adInit` bid application | `js/lib/src/integrations/gpt/index.ts:566`, `:652`, `:657-661` | +| Client SPA auction hook | `js/lib/src/integrations/gpt/index.ts:806`, `:859`, `:892-949` | +| Prior design that introduced the killers | `docs/superpowers/specs/2026-07-22-ssat-root-document-304-prevention-design.md` | From cf204f08a45b6f6aef0be096d3469eca94fa2fc2 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 16:35:32 +0530 Subject: [PATCH 08/44] Close the remaining #1009 cross-document contradictions and gate the sweep Four contradictions found by a mechanical sweep, all verified before fixing. Stage 0 was still summarized as gated only by the Vary check in the spec's decision table, and as reverting with a config push alone in the plan's Task 5 preamble. Both now point at FINAL PASS and at the full flip-purge-observe sequence. The findings still attached C1 rollback keys using InsertBuilder::surrogate_keys, which is the Core Cache API and keys the transformed-template cache the ESI spike would build. It has no effect on the HTTP read-through cache Stage 0 turns on. The spec's invalidation table had the same ambiguity in a row that read fine in section context and wrong when quoted; it is now split into explicit C1 and C2 rows. The Core Cache pseudocode still would not compile after the previous fix. The error arm referenced a writer only the success arm bound, and a helper taking &tx could not call Transaction::insert, which consumes self. Restructured so everything fallible that does not need the writer happens before insert, where cancel_insert_or_update is still reachable, and so finish and abandon are each reached from the arm that owns the writer. The safety gate still asserted privacy finalization runs after assembly, contradicting the streaming rule added directly above it. Headers commit before the body streams on this adapter, so the gate now asserts finalization happened first, including an unconditional private/no-store. The Task 3 file list still said markers go at two seams while the corrected design emits one unconditional body-close marker. Adds scripts/docs-invariants.py and makes it a named gate in both plans. Format and build are necessary but neither can see a claim corrected in one document and left standing in another, which is how every one of the last four review rounds found real defects. The checker is context-aware, since qualifying text usually wraps to an adjacent line, and it is meant to grow a check whenever a correction lands. --- ...2026-08-08-1009-measurement-and-stage-0.md | 28 +++++++- .../2026-08-08-1009-measurement-findings.md | 20 ++++-- .../2026-08-10-1009-esi-validation-spike.md | 68 ++++++++++++------- ...08-esi-cacheable-root-validation-design.md | 34 ++++++---- scripts/docs-invariants.py | 65 ++++++++++++++++++ 5 files changed, 166 insertions(+), 49 deletions(-) create mode 100755 scripts/docs-invariants.py diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md index 7a6d79a89..d4ab8ba27 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md @@ -25,6 +25,17 @@ for wasm-safe timing, `log` for instrumentation, Viceroy for adapter tests. **Spec:** `docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md` (§3 Steps A/B/C and §4 Stage 0). Read §4 and §5 before starting Task 5. +**Before pushing, run three gates, not two:** + +```bash +cd docs && npm run format && npm run build && cd .. +python3 scripts/docs-invariants.py +``` + +`npm run build` is not optional — `format` passes on documents with dead links, and that +shipped a broken docs build on this branch once already. `docs-invariants.py` catches +cross-document contradictions, which neither of the other two can see. + **Two prettier gotchas, both hit while writing this plan.** CI gate 7 (`cd docs && npm run format`) runs `prettier --check` over all of `docs/`, so both bite. @@ -781,9 +792,15 @@ Any one of these unrecorded means the verdict stays `PROVISIONAL PASS` and Task not start. Because Task 3 made the bypass config-driven, Stage 0 ships as a **config change on an -already-deployed build**. No second release, and rollback is another config push rather -than a revert. That matters here specifically: the failure mode this gates on is cache -poisoning, where minutes of exposure are worse than a slow rollout. +already-deployed build** — no second release, and the read path reverts with another +config push rather than a revert. That matters here: the failure mode this gates on is +cache poisoning, where minutes of exposure are worse than a slow rollout. + +**But a config push is not a full rollback.** It stops HTML navigations reading from +cache; it evicts nothing already stored. See Step 4's rollback sequence — flip, then purge +or roll a versioned namespace, then observe past the origin TTL. Until a C1 purge path +exists, the tail is "wait out the origin TTL," and that must be an accepted, recorded +risk before the flip. ### Task 5a: flip the flag (Task 1 verdict = `FINAL PASS`) @@ -1025,5 +1042,10 @@ Named so nobody widens this plan mid-flight. All are specified in the spec. measuring the TTFB the publisher actually complained about; use it for before/after. - [ ] `unexpected_origin_304` rate is zero and representations are not mixed (Task 5a Step 8) — both checked **before** the win is claimed. +- [ ] **Cross-document invariants hold:** `python3 scripts/docs-invariants.py`. This is a + named gate, not a courtesy check. `npm run format` and `npm run build` catch + formatting and dead links; neither catches a claim corrected in one document and + left standing in another, which is the failure mode this document set has hit on + four separate review rounds. Add a check whenever a correction lands. - [ ] All CI gates pass: `cargo fmt --all -- --check`; the six clippy targets; the four adapter test suites; the parity suite; JS build, test, and format; docs format. diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md index a125d957c..a092f6c52 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md @@ -137,12 +137,20 @@ continue to read — persist until they expire. Two mitigations, both real: - The origin's `max-age=60` bounds read-through exposure to roughly a minute. -- Purge is available in-process: `fastly::http::purge::purge_surrogate_key`, with keys - attached at insert via `InsertBuilder::surrogate_keys`. An earlier claim that TS had no - purge capability was wrong — it has no _wiring_, which is buildable. - -Rollback is therefore: flip the flag, **then** purge or roll a versioned cache-key -namespace, **then** observe past the origin TTL before declaring the incident closed. +- Purge exists in-process — `fastly::http::purge::purge_surrogate_key`. An earlier claim + that TS had no purge capability was wrong; it has no _wiring_, which is buildable. + +**But note which cache.** `InsertBuilder::surrogate_keys` belongs to the **Core Cache** +API and applies to the transformed-template cache the ESI spike would build (C2). It has +**no effect on the HTTP read-through cache** that Stage 0 turns on (C1). Purging C1 needs +surrogate keys the _origin_ supplies on its responses, or the HTTP cache's own +request/candidate surrogate-key surface. Confirm which is available before relying on it — +an earlier revision of this document conflated the two. + +Rollback is therefore: flip the flag, **then** purge C1 by whichever mechanism is actually +available (or roll a versioned key namespace), **then** observe past the origin TTL before +declaring the incident closed. With no C1 purge path wired, the tail is the TTL itself — +roughly a minute here, and a recorded risk rather than a surprise. ## Step B — consumers of TS's own response headers diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index befae1761..ea4137c6a 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -251,7 +251,7 @@ The core of the spike. Behind a flag, default off. **Files:** -- `crates/trusted-server-core/src/publisher.rs` — emit markers at the two seams +- `crates/trusted-server-core/src/publisher.rs` — emit **one** unconditional marker at the body-close seam (see Step 2; the head seam is not a template hole) - `crates/trusted-server-core/src/settings.rs` — the mode flag - `crates/trusted-server-adapter-fastly/src/` — the `cache::core` read/write @@ -352,38 +352,48 @@ let tx = Transaction::lookup(CacheKey::from(key_bytes)).execute()?; // Testing found() first would serve the stale bytes and silently never fulfil the // update obligation, leaving every concurrent waiter blocked until timeout. let template: Body = if tx.must_insert_or_update() { - match transform_origin_into(&tx) { - Ok((writer, found)) => { - writer.finish()?; // REQUIRED — without it the object never completes - found.to_stream()? // fallible; there is no `to_body()` + // Fetch and prepare BEFORE consuming `tx`. After `insert()` the transaction is + // gone and `cancel_insert_or_update()` is unreachable, so anything that can fail + // and does not need the writer belongs here. + let origin = match fetch_and_prepare_origin() { + Ok(origin) => origin, + Err(e) => { + tx.cancel_insert_or_update()?; // releases the obligation to a waiter + return fallback_uncached(e); + } + }; + + // `Transaction::insert(self)` consumes `tx` from this line on. + let (mut writer, found) = tx + .insert(template_ttl) + .surrogate_keys(["ts-template", &url_surrogate_key]) // chained, not discarded + .user_metadata(metadata_envelope) + .execute_and_stream_back()?; + + match stream_lol_html_output(origin, &mut writer) { + Ok(()) => { + writer.finish()?; // REQUIRED, and consumes `writer` + found.to_stream()? // fallible; there is no `to_body()` } Err(e) => { - // Do NOT finish() a partial template — it would be served to everyone - // until it expires. Abandon the writer, release the obligation so another - // client can try, and fall back to the untransformed path for this request. - writer.abandon(); - tx.cancel_insert_or_update()?; + // Also consumes `writer`, marking an unsuccessful end so no partial + // template is served. (A `StreamingBody` dropped without `finish()` is + // aborted anyway, but say it explicitly.) + writer.abandon()?; return fallback_uncached(e); } } } else if let Some(found) = tx.found() { - // Fresh hit. `is_usable()` and `is_stale()` are available if a stale-serve - // policy is wanted; the spike should start by treating stale as a miss. - found.to_stream()? // C2 HIT — skip origin fetch and transform + found.to_stream()? // C2 HIT — skip origin fetch and transform } else { unreachable!("a transaction is either obliged to insert or has found an item") }; ``` -Inside `transform_origin_into`, `execute_and_stream_back()` yields both handles at once: - -```rust -let (mut writer, found) = tx - .insert(template_ttl) - .surrogate_keys(["ts-template", &url_surrogate_key]) // chained, not discarded - .user_metadata(metadata_envelope) - .execute_and_stream_back()?; -``` +Two ownership rules this shape exists to respect, both of which an earlier draft broke: +`Transaction::insert(self)` **consumes** the transaction, so a helper taking `&tx` cannot +call it and `cancel_insert_or_update` is unreachable afterwards; and `finish`/`abandon` +each consume the writer, so neither can be referenced from an arm that did not bind it. **Decide the stale policy explicitly.** `Found::is_stale()` and `is_usable()` exist, and `stale_while_revalidate` can be set at insert. Serving stale while revalidating is a real @@ -582,9 +592,12 @@ Not a phase. Every one of these is a hard fail, independent of any performance r - [ ] **Request collapsing** works: concurrent cold requests transform once. - [ ] **DCA disabled**, verified by the injection test in Task 5 Step 2. - [ ] **Exactly one auction per pageview**, from `auction_events_raw`. -- [ ] **Cookie and privacy finalization still run** after assembly — EC `Set-Cookie` on - first visit, and the privacy net downgrading it. This is the ordering that ESI's - streaming mode makes easy to get wrong, since it drops `$add_header`. +- [ ] **Cookie and privacy finalization ran BEFORE assembly**, not after — EC + `Set-Cookie` on first visit, geo suppression, and an unconditional + `Cache-Control: private, no-store`. Headers commit before the body streams on this + adapter, so "finalize after assembly" is not available; asserting it that way is how + a per-user response ends up shared-cacheable. ESI's streaming mode dropping + `$add_header` is a consequence of the same constraint, not a separate hazard. - [ ] **Slot and bid attribution unchanged.** Same slots matched, same bids applied, same renders attributed. Use TS-attributed renders — the SSAT line item, non-empty `ts.bids`, `hb_adid` presence — **never slot fill**, which is blind to empty bids @@ -668,5 +681,10 @@ routes; N per arm; and the cache-tier mix. the answer. - [ ] Cleanup complete: flag resolved, C2 purged, synthetic endpoint removed, dependency landed or dropped. +- [ ] **Cross-document invariants hold:** `python3 scripts/docs-invariants.py`. This is a + named gate, not a courtesy check. `npm run format` and `npm run build` catch + formatting and dead links; neither catches a claim corrected in one document and + left standing in another, which is the failure mode this document set has hit on + four separate review rounds. Add a check whenever a correction lands. - [ ] All CI gates pass: `cargo fmt --all -- --check`; the six clippy targets; the four adapter test suites; the parity suite; JS build, test, and format; docs format. diff --git a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md index 446ff1727..fda8b6e4c 100644 --- a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md +++ b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md @@ -17,9 +17,12 @@ > `insert(key, max_age).execute() -> StreamingBody` for arbitrary bytes, `lookup()` / > `found()` to read them back, and `Transaction` with `must_insert()` for request > collapsing. The two-stage design needs no separate KV or template service. -> - **Purge exists in-process.** `InsertBuilder::surrogate_keys([...])` attaches keys at -> insert; `fastly::http::purge::purge_surrogate_key` purges from inside Compute. The -> management-API token scope cited in the original is irrelevant to it. +> - **Purge exists in-process.** `fastly::http::purge::purge_surrogate_key` purges from +> inside Compute; the management-API token scope cited in the original is irrelevant to +> it. Note which cache, though: `InsertBuilder::surrogate_keys([...])` is the **Core +> Cache** API and keys the transformed-template cache (C2). It does **not** key the HTTP +> read-through cache (C1) that Stage 0 turns on — purging that needs origin-supplied +> keys or the HTTP cache's own surrogate-key surface. > - **The original pipeline ordering was backwards.** It said "order esi → lol*html, > never the reverse." `lol_html` \_emits* the `esi:include` tags, so ESI must run after > it. Correct order is in [§6.6](#66-the-esi-pipeline-corrected). @@ -59,12 +62,12 @@ ## 1. Decision requested -| # | Decision | Owner needed | -| --- | --------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | -| D1 | **ESI is feasible and unvalidated.** Validate it via [the spike plan](../plans/2026-08-10-1009-esi-validation-spike.md), not by deferring it. | Eng + product | -| D2 | **Fund ~3 days of measurement** (§3). No dependencies. Can start immediately. | Eng | -| D3 | **Approve Stage 0** — an operator flag disabling the origin cache bypass, subject to the `Vary` check in §3. | Eng | -| D4 | **Stages 1–2 queue behind the SSAT price defect and #938.** Stages 3b–5 unscheduled. | Product | +| # | Decision | Owner needed | +| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| D1 | **ESI is feasible and unvalidated.** Validate it via [the spike plan](../plans/2026-08-10-1009-esi-validation-spike.md), not by deferring it. | Eng + product | +| D2 | **Fund ~3 days of measurement** (§3). No dependencies. Can start immediately. | Eng | +| D3 | **Approve Stage 0** — an operator flag disabling the origin cache bypass, subject to a **`FINAL PASS`** in §3. A `Vary` check alone is a `PROVISIONAL PASS` and is not a release gate. Rollback needs a purge path, not only a config push. | Eng | +| D4 | **Stages 1–2 queue behind the SSAT price defect and #938.** Stages 3b–5 unscheduled. | Product | Rationale for D4 in [§8](#8-priority). Everything this document recommends _against_ doing is in [§7](#7-deferred-work-specified-not-scheduled), at deliberately lower @@ -429,12 +432,13 @@ what #1009 proposed in the first place. Mechanism, all present in the pinned `fastly` 0.12.1: -| Need | API | -| ----------------------- | ----------------------------------------------------------------------------------- | -| Store the template | `cache::core::insert(key, max_age).execute() -> StreamingBody` | -| Read it back | `cache::core::lookup(key)` → `found()` | -| Avoid a thundering herd | `cache::core::Transaction` — `must_insert()` / `must_insert_or_update()` | -| Invalidate | `InsertBuilder::surrogate_keys([...])` + `fastly::http::purge::purge_surrogate_key` | +| Need | API | +| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | +| Store the template | `cache::core::insert(key, max_age).execute() -> StreamingBody` | +| Read it back | `cache::core::lookup(key)` → `found()` | +| Avoid a thundering herd | `cache::core::Transaction` — `must_insert()` / `must_insert_or_update()` | +| Invalidate **C2 only** | `InsertBuilder::surrogate_keys([...])` (Core Cache) + `fastly::http::purge::purge_surrogate_key`. Does **not** key C1 — see the row below. | +| Invalidate C1 | Origin-supplied surrogate keys, or the HTTP cache's own surrogate-key surface. Not the Core Cache API. | Purge runs **inside Compute**. The management-API token scope cited under [Stage 4](#7-deferred-work-specified-not-scheduled) governs a different surface and does diff --git a/scripts/docs-invariants.py b/scripts/docs-invariants.py new file mode 100755 index 000000000..5eb6eb3d8 --- /dev/null +++ b/scripts/docs-invariants.py @@ -0,0 +1,65 @@ +#!/usr/bin/env python3 +"""Cross-document invariant sweep for the #1009 document set. + +`npm run format` and `npm run build` catch formatting and dead links. Neither +catches a claim corrected in one document and left standing in another, which is +the failure mode this set has hit repeatedly. + +Checks are context-aware: a hit is excused only if an allowlist pattern appears +within a window of lines around it, because qualifying text usually wraps. + +Exit 1 on any surviving hit. +""" +import re, sys, glob, os + +ROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..") +FILES = sorted( + glob.glob(os.path.join(ROOT, "docs/superpowers/specs/2026-08-08-esi-*.md")) + + glob.glob(os.path.join(ROOT, "docs/superpowers/plans/2026-08-*1009*.md")) +) + +# (name, forbidden pattern, allowlist pattern or None, context window in lines) +CHECKS = [ + ("Stage 0 gate is three-verdict", + r"PASS / FAIL|verdict = PASS[^A-Za-z]|subject to the `Vary` check", + r"PROVISIONAL PASS / FAIL|FINAL PASS", 0), + ("Rollback is not described as config-only", + r"rollback is another config push rather", None, 0), + ("adSlots is not in the shared template", + r"two seams|adSlots[^.]*stays in the template|One per-user hole\.", None, 0), + ("Headers finalize before assembly", + r"finalization still run.{0,4} after assembly|run \*\*after\*\* assembly", None, 0), + ("No nonexistent Core Cache APIs", + r"(? Date: Mon, 10 Aug 2026 17:07:32 +0530 Subject: [PATCH 09/44] Fix stale summaries and rebuild the invariant checker so it cannot false-green MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The checker added in cf204f08 reported 8/8 green on documents that still contained the contradictions it claimed to check. That is worse than having no checker: it certifies bad state. Three causes, each now addressed. It matched literal phrases. The stale text said "two existing injection seams", the pattern looked for "two seams". Patterns are now semantic and tolerant of wording. It matched line by line, so any phrase wrapped across a line break was invisible. Files are now whitespace-normalized before matching, which is how the architecture arrows spanning several lines were being missed. It had no way to know it had stopped working. Every check now carries fixtures: strings that must trip it, and corrected strings that must not. The script exits 2 and refuses to report anything if its own fixtures fail. Writing them caught two of my patterns not firing at all — one defeated by markdown emphasis between "Verdict:" and "PASS", another by a sentence boundary. Proof rather than assertion: run against the cf204f08 tree, the new checker flags all five contradictions there, including the four this review named. The old checker reported that same tree green. The stale text itself. The spike's architecture summary still said two injection seams and ordered assemble before finalize. The spec still described the cheap curl as gating Stage 0, mapped the Vary result straight to a config push, and summarized rollback as config-only in the priority section. Its pipeline diagram contradicted its own caption — the caption said headers finalize first while the arrows still read assemble then finalize. That diagram is a good example of why literal matching failed and why diagrams need checking as prose does. Also disambiguated the Stage 4 note, which cited InsertBuilder::surrogate_keys without saying it keys C2 rather than the C1 read-through cache Stage 0 turns on. --- .../2026-08-10-1009-esi-validation-spike.md | 16 +- ...08-esi-cacheable-root-validation-design.md | 37 +-- scripts/docs-invariants.py | 236 ++++++++++++++---- 3 files changed, 218 insertions(+), 71 deletions(-) diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index ea4137c6a..0fb273585 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -8,11 +8,17 @@ ESI and client-fill against it, and produce a decision record that either adopts ESI, adopts client-fill, or rejects both — with the Fastly-only maintenance cost priced in. -**Architecture:** `origin → lol_html transform → fastly::cache::core → assemble → finalize`. -The transform emits `esi:include` markers at the two existing injection seams instead of -inlining per-user data. The cached object is a shared template with no per-user bytes. -Assembly is either the `esi` crate (edge) or a client fetch of `/_ts/page-bids` (browser), -selected per request by config so both can be measured on one build. +**Architecture:** +`origin → lol_html transform → fastly::cache::core → finalize headers → stream assembly`. + +Headers finalize **before** assembly, not after — streaming responses on this adapter +commit headers first and then pipe chunks, so nothing can be set once assembly starts. + +The transform emits **one unconditional marker at the body-close seam**. Not two: the +head seam is not a template hole, because `tsjs.adSlots` presence is request-gated +(Task 3 Step 2). The cached object is a shared template with no per-user bytes and no +request-dependent decisions. Assembly is either the `esi` crate (edge) or a client fetch +(browser), selected per request by the arm allocator so both are measured on one build. **Tech Stack:** Rust 2024, `wasm32-wasip1`, `fastly` 0.12.1 (`cache::core`, `http::purge`), `esi` 0.7, `lol_html`, a real Fastly test service for cache behaviour. diff --git a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md index fda8b6e4c..39a65dac5 100644 --- a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md +++ b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md @@ -118,9 +118,12 @@ supply it: they compare cached fetches against each other, not against an origin Three checks, ordered cheapest-first. Each needs a named owner before starting. -**Step A — origin `Vary` check (minutes).** `curl` the origin with and without `RSC`, -`Next-Router-*`, and the experiment header; inspect the `Vary` response header. -**Gates Stage 0**, the only build item recommended now. Do this first because it is the +**Step A — origin `Vary` and cookie check (minutes for the first pass).** `curl` the +origin with and without `RSC`, `Next-Router-*`, and the experiment header; inspect `Vary`, +`Cache-Control`, and `Set-Cookie`. **This first pass yields a `PROVISIONAL PASS` only** — +it is not what gates the flip. A `FINAL PASS` additionally requires a real authenticated +session, Basic Auth through TS, the experiment variant, representative routes, and +cached-hit render attribution. Do the cheap pass first because it is the cheapest thing that unblocks anything. **Step B — what consumes TS's own response headers (under a day).** Request a TS-served @@ -264,7 +267,8 @@ three are a larger class than the RSC split: So Step A must capture `Cache-Control` and `Set-Cookie` too, and repeat each request with and without a session cookie. Same minutes of work; closes the bigger hole. -**Two effort branches, and Step A decides which:** +**Two effort branches, and Step A's `Vary` result decides which** — note this selects the +_shape_ of Stage 0, while the `FINAL PASS` conditions decide _whether it ships at all_: | Step A result | Stage 0 is… | Effort | | ---------------------- | --------------------------------------------- | ------ | @@ -418,11 +422,13 @@ That is backwards. `lol_html` is what \_emits* the `esi:include` tags; ESI canno tags that do not exist yet. The correct order: ``` -origin → lol_html transform → fastly::cache::core → esi assemble → finalize → client - (one unconditional marker (shared template, (per request, headers are - at the body-close seam, surrogate-keyed, fetch the finalized - no per-user data and no TS-chosen TTL) fragment) BEFORE the - request-dependent decisions) body streams +origin → lol_html transform → fastly::cache::core → finalize headers → stream esi assembly → client + (one unconditional marker (shared template, (EC cookie, geo, (per request, + at the body-close seam; surrogate-keyed, unconditional fetch the + the head seam is NOT a TS-chosen TTL) private/no-store) fragment) + hole — adSlots presence + is request-gated, §6.7) nothing may change + after this point ``` The push/pull mismatch that the earlier revision treated as a blocker is real but @@ -545,9 +551,10 @@ until a topology change. **Stage 4 — purge wiring.** Not sized. Prerequisite for a TS-owned cache. TS today emits no `Surrogate-Key` and holds a management token scoped without purge — but that token is -the wrong surface: `InsertBuilder::surrogate_keys` and -`fastly::http::purge::purge_surrogate_key` are both in the pinned SDK and purge runs -inside Compute ([§6.6](#66-the-esi-pipeline-corrected)). This is **missing wiring, not a +the wrong surface: `InsertBuilder::surrogate_keys` is the Core Cache API and +`fastly::http::purge::purge_surrogate_key` runs inside Compute, both in the pinned SDK +([§6.6](#66-the-esi-pipeline-corrected)). Note those key **C2**, the TS-owned template +cache — not the HTTP read-through cache C1. This is **missing wiring, not a platform limit.** Until it exists, any TS-owned cache is TTL-only and a config push takes up to one TTL to take effect. @@ -590,8 +597,10 @@ comment that it is _"rejected until an access-log emitter is wired"_). That is a follow-on, not part of this work. **Stage 0 next**, shipped as the operator flag in §4 rather than a deletion. Gated on a -`curl`, reverses an origin-load cost the prior design explicitly accepted, and rolls back -with a config push. Closer to a defect fix than an optimization — TS opted out of a cache +`FINAL PASS` — a `curl` alone yields only a `PROVISIONAL PASS`. It reverses an origin-load +cost the prior design explicitly accepted; the read path reverts with a config push, but +full rollback also needs a C1 purge path or waiting out the origin TTL. Closer to a defect +fix than an optimization — TS opted out of a cache it did not need to opt out of. **Stages 1–2 queue behind the correctness defects.** Their failure mode is silent diff --git a/scripts/docs-invariants.py b/scripts/docs-invariants.py index 5eb6eb3d8..562198cb3 100755 --- a/scripts/docs-invariants.py +++ b/scripts/docs-invariants.py @@ -1,65 +1,197 @@ #!/usr/bin/env python3 """Cross-document invariant sweep for the #1009 document set. -`npm run format` and `npm run build` catch formatting and dead links. Neither +`npm run format` catches formatting. `npm run build` catches dead links. Neither catches a claim corrected in one document and left standing in another, which is -the failure mode this set has hit repeatedly. +how every review round on this branch has found real defects. -Checks are context-aware: a hit is excused only if an allowlist pattern appears -within a window of lines around it, because qualifying text usually wraps. +DESIGN NOTES — a previous version of this checker reported 8/8 green on +documents that still contained the exact contradictions it claimed to check. +Three things caused that, and each is addressed here: -Exit 1 on any surviving hit. +1. It matched literal phrases ("two seams") that the stale text did not use + ("two existing injection seams"). Patterns are now semantic and tolerant. +2. It matched line by line, so any phrase wrapped across a line break was + invisible. Text is now whitespace-normalized per file before matching. +3. It had no way to know it had stopped working. Every check now carries + `must_flag` fixtures — strings that MUST trip it — and the script fails if + any fixture is not caught. A check that cannot fail is treated as broken. + +A false positive costs a minute. A false green costs a merge. """ -import re, sys, glob, os +import re +import sys +import glob +import os ROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..") -FILES = sorted( - glob.glob(os.path.join(ROOT, "docs/superpowers/specs/2026-08-08-esi-*.md")) - + glob.glob(os.path.join(ROOT, "docs/superpowers/plans/2026-08-*1009*.md")) -) -# (name, forbidden pattern, allowlist pattern or None, context window in lines) + +def targets(): + return sorted( + glob.glob(os.path.join(ROOT, "docs/superpowers/specs/2026-08-08-esi-*.md")) + + glob.glob(os.path.join(ROOT, "docs/superpowers/plans/2026-08-*1009*.md")) + ) + + +def normalize(text): + """Collapse whitespace so wrapped phrases match, and strip table padding.""" + text = re.sub(r"\s*\n\s*", " ", text) + text = re.sub(r"[ \t]{2,}", " ", text) + return text + + +class Check: + def __init__(self, name, bad, allow=None, must_flag=(), must_pass=()): + self.name = name + self.bad = re.compile(bad, re.I) + self.allow = re.compile(allow, re.I) if allow else None + self.must_flag = must_flag # strings this check MUST catch + self.must_pass = must_pass # corrected strings it must NOT catch + + def hits(self, normalized): + out = [] + for m in self.bad.finditer(normalized): + lo, hi = max(0, m.start() - 240), min(len(normalized), m.end() + 240) + if self.allow and self.allow.search(normalized[lo:hi]): + continue + out.append(normalized[max(0, m.start() - 60):m.end() + 60].strip()) + return out + + CHECKS = [ - ("Stage 0 gate is three-verdict", - r"PASS / FAIL|verdict = PASS[^A-Za-z]|subject to the `Vary` check", - r"PROVISIONAL PASS / FAIL|FINAL PASS", 0), - ("Rollback is not described as config-only", - r"rollback is another config push rather", None, 0), - ("adSlots is not in the shared template", - r"two seams|adSlots[^.]*stays in the template|One per-user hole\.", None, 0), - ("Headers finalize before assembly", - r"finalization still run.{0,4} after assembly|run \*\*after\*\* assembly", None, 0), - ("No nonexistent Core Cache APIs", - r"(?)\s*finaliz" + r"|finaliz\w*[^.]{0,40}(runs?|still run)[^.]{0,20}after\s+assembly", + must_flag=[ + "`origin → lol_html transform → fastly::cache::core → assemble → finalize`", + "esi assemble → finalize → client", + "Cookie and privacy finalization still run after assembly", + ], + must_pass=[ + "fastly::cache::core → finalize headers → stream assembly", + "Cookie and privacy finalization ran BEFORE assembly, not after", + ], + ), + Check( + "No nonexistent Core Cache APIs", + r"(? Date: Mon, 10 Aug 2026 17:50:31 +0530 Subject: [PATCH 10/44] Make the #1009 docs invariant gate fail closed --- scripts/docs-invariants.py | 149 ++++++++++++++++++++++++++++++------- 1 file changed, 122 insertions(+), 27 deletions(-) diff --git a/scripts/docs-invariants.py b/scripts/docs-invariants.py index 562198cb3..e88fb81b2 100755 --- a/scripts/docs-invariants.py +++ b/scripts/docs-invariants.py @@ -16,6 +16,13 @@ 3. It had no way to know it had stopped working. Every check now carries `must_flag` fixtures — strings that MUST trip it — and the script fails if any fixture is not caught. A check that cannot fail is treated as broken. +4. It let unrelated nearby correction language excuse a violation. Generic + allow-windows are gone; the one qualified check must span the exact API + occurrence it excuses. + +The four promised input documents and both fixture directions are mandatory. +Missing inputs, unreadable inputs, or an empty fixture side make the checker +itself fail with exit 2 before it reports document results. A false positive costs a minute. A false green costs a merge. """ @@ -26,6 +33,15 @@ ROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..") +REQUIRED_TARGETS = frozenset( + { + "docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md", + "docs/superpowers/plans/2026-08-08-1009-measurement-findings.md", + "docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md", + "docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md", + } +) + def targets(): return sorted( @@ -34,26 +50,64 @@ def targets(): ) +def target_errors(paths): + """Return setup errors when any document this gate promises to scan is absent.""" + present = { + os.path.relpath(path, ROOT).replace(os.sep, "/") + for path in paths + } + return [ + f" missing required document: {path}" + for path in sorted(REQUIRED_TARGETS - present) + ] + + +def load_documents(paths): + documents = {} + errors = [] + for path in paths: + try: + with open(path, encoding="utf-8") as source: + documents[path] = normalize(source.read()) + except (OSError, UnicodeError) as error: + errors.append(f" cannot read {os.path.relpath(path, ROOT)}: {error}") + return documents, errors + + def normalize(text): - """Collapse whitespace so wrapped phrases match, and strip table padding.""" + """Collapse wraps while removing Markdown blockquote continuation markers.""" + text = re.sub(r"(?m)^\s*>\s?", "", text) text = re.sub(r"\s*\n\s*", " ", text) text = re.sub(r"[ \t]{2,}", " ", text) return text class Check: - def __init__(self, name, bad, allow=None, must_flag=(), must_pass=()): + def __init__(self, name, bad, must_flag=(), must_pass=()): self.name = name self.bad = re.compile(bad, re.I) - self.allow = re.compile(allow, re.I) if allow else None self.must_flag = must_flag # strings this check MUST catch self.must_pass = must_pass # corrected strings it must NOT catch def hits(self, normalized): out = [] for m in self.bad.finditer(normalized): - lo, hi = max(0, m.start() - 240), min(len(normalized), m.end() + 240) - if self.allow and self.allow.search(normalized[lo:hi]): + out.append(normalized[max(0, m.start() - 60):m.end() + 60].strip()) + return out + + +class QualifiedOccurrenceCheck(Check): + """Flag each occurrence unless its own local clause states the required relation.""" + + def __init__(self, name, occurrence, qualified, must_flag=(), must_pass=()): + super().__init__(name, occurrence, must_flag=must_flag, must_pass=must_pass) + self.qualified = re.compile(qualified, re.I) + + def hits(self, normalized): + out = [] + qualified_spans = [match.span() for match in self.qualified.finditer(normalized)] + for m in self.bad.finditer(normalized): + if any(lo <= m.start() and m.end() <= hi for lo, hi in qualified_spans): continue out.append(normalized[max(0, m.start() - 60):m.end() + 60].strip()) return out @@ -64,26 +118,32 @@ def hits(self, normalized): "Stage 0 gates on FINAL PASS, not a bare Vary check", # `[*_ ]*` absorbs markdown emphasis; `gated on a curl` is its own shape # because the sentence boundary defeats a proximity match to "Stage 0". - r"gates?[*_ ]+stage[*_ ]*0(?![^.]{0,140}final pass)" + r"gates?[*_ ]+stage[*_ ]*0" + r"(?!\s*only\s+(?:once|after|when)\s+(?:an?\s+)?[`*_]*final\s+pass[`*_]*" + r"(?:\s+is)?\s+(?:recorded|obtained|achieved)\b)" r"|verdict[:*_ ]+pass[*_ ]*/[*_ ]*fail" r"|gated\s+on\s+a\s+.?curl", - allow=r"final pass|provisional pass", must_flag=[ "inspect the `Vary` response header. **Gates Stage 0**, the only build item", "**Verdict:** PASS / FAIL", "Stage 0 next, shipped as the operator flag. Gated on a `curl`, reverses an origin-load cost", + "Inspect Vary. **Gates Stage 0** immediately. A separate sentence says FINAL PASS is required.", + "Gates Stage 0 immediately; FINAL PASS gates deployment later.", + "The Vary result gates Stage 0 only on a successful curl, while FINAL PASS gates production rollout.", ], must_pass=[ "Gates Stage 0 only once a FINAL PASS is recorded", + "Gates Stage 0 only after a FINAL PASS is recorded", "**Verdict:** FINAL PASS / PROVISIONAL PASS / FAIL", ], ), Check( "Rollback is not described as config-only", - r"rolls?\s+back\s+with\s+a\s+config\s+push(?![^.]{0,160}(purge|ttl))" + r"rolls?\s+back\s+with\s+a\s+config\s+push" r"|rollback\s+is\s+another\s+config\s+push\s+rather", must_flag=[ "reverses an origin-load cost, and rolls back with a config push. Closer to a defect fix", + "This rolls back with a config push, explicitly without a purge or TTL wait.", ], must_pass=[ "the read path reverts with a config push, but full rollback also needs a C1 purge path", @@ -91,16 +151,18 @@ def hits(self, normalized): ), Check( "Template has one marker, not two seams", - r"(two|both)\s+(existing\s+)?(injection\s+)?seams" - r"|markers?\s+at\s+the\s+two\s+seams" + r"\b(?:emit|emits|use|uses|place|places|insert|inserts)\b[^.]{0,100}" + r"(?:two|both)\s+(?:existing\s+)?(?:injection\s+)?seams" + r"|markers?\s+at\s+(?:the\s+)?two\s+(?:existing\s+)?(?:injection\s+)?seams" r"|adSlots[^.]{0,60}stays?\s+in\s+the\s+template", - allow=r"not two|the head seam is not|NOT a hole", must_flag=[ "The transform emits `esi:include` markers at the two existing injection seams instead", "emit markers at the two seams", + "Emit markers at the two existing injection seams. Correction: not two; use one marker.", ], must_pass=[ "one unconditional marker at the body-close seam. Not two: the head seam is not a template hole", + "The earlier draft used two injection seams; that statement was wrong.", ], ), Check( @@ -119,9 +181,11 @@ def hits(self, normalized): ), Check( "No nonexistent Core Cache APIs", - r"(? Date: Mon, 10 Aug 2026 18:10:11 +0530 Subject: [PATCH 11/44] Remove the #1009 documentation invariant checker --- ...2026-08-08-1009-measurement-and-stage-0.md | 11 +- .../2026-08-10-1009-esi-validation-spike.md | 5 - scripts/docs-invariants.py | 292 ------------------ 3 files changed, 2 insertions(+), 306 deletions(-) delete mode 100755 scripts/docs-invariants.py diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md index d4ab8ba27..6de35c9b3 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md @@ -25,16 +25,14 @@ for wasm-safe timing, `log` for instrumentation, Viceroy for adapter tests. **Spec:** `docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md` (§3 Steps A/B/C and §4 Stage 0). Read §4 and §5 before starting Task 5. -**Before pushing, run three gates, not two:** +**Before pushing, run both documentation gates:** ```bash cd docs && npm run format && npm run build && cd .. -python3 scripts/docs-invariants.py ``` `npm run build` is not optional — `format` passes on documents with dead links, and that -shipped a broken docs build on this branch once already. `docs-invariants.py` catches -cross-document contradictions, which neither of the other two can see. +shipped a broken docs build on this branch once already. **Two prettier gotchas, both hit while writing this plan.** CI gate 7 (`cd docs && npm run format`) runs `prettier --check` over all of `docs/`, so both bite. @@ -1042,10 +1040,5 @@ Named so nobody widens this plan mid-flight. All are specified in the spec. measuring the TTFB the publisher actually complained about; use it for before/after. - [ ] `unexpected_origin_304` rate is zero and representations are not mixed (Task 5a Step 8) — both checked **before** the win is claimed. -- [ ] **Cross-document invariants hold:** `python3 scripts/docs-invariants.py`. This is a - named gate, not a courtesy check. `npm run format` and `npm run build` catch - formatting and dead links; neither catches a claim corrected in one document and - left standing in another, which is the failure mode this document set has hit on - four separate review rounds. Add a check whenever a correction lands. - [ ] All CI gates pass: `cargo fmt --all -- --check`; the six clippy targets; the four adapter test suites; the parity suite; JS build, test, and format; docs format. diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index 0fb273585..354a41b85 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -687,10 +687,5 @@ routes; N per arm; and the cache-tier mix. the answer. - [ ] Cleanup complete: flag resolved, C2 purged, synthetic endpoint removed, dependency landed or dropped. -- [ ] **Cross-document invariants hold:** `python3 scripts/docs-invariants.py`. This is a - named gate, not a courtesy check. `npm run format` and `npm run build` catch - formatting and dead links; neither catches a claim corrected in one document and - left standing in another, which is the failure mode this document set has hit on - four separate review rounds. Add a check whenever a correction lands. - [ ] All CI gates pass: `cargo fmt --all -- --check`; the six clippy targets; the four adapter test suites; the parity suite; JS build, test, and format; docs format. diff --git a/scripts/docs-invariants.py b/scripts/docs-invariants.py deleted file mode 100755 index e88fb81b2..000000000 --- a/scripts/docs-invariants.py +++ /dev/null @@ -1,292 +0,0 @@ -#!/usr/bin/env python3 -"""Cross-document invariant sweep for the #1009 document set. - -`npm run format` catches formatting. `npm run build` catches dead links. Neither -catches a claim corrected in one document and left standing in another, which is -how every review round on this branch has found real defects. - -DESIGN NOTES — a previous version of this checker reported 8/8 green on -documents that still contained the exact contradictions it claimed to check. -Three things caused that, and each is addressed here: - -1. It matched literal phrases ("two seams") that the stale text did not use - ("two existing injection seams"). Patterns are now semantic and tolerant. -2. It matched line by line, so any phrase wrapped across a line break was - invisible. Text is now whitespace-normalized per file before matching. -3. It had no way to know it had stopped working. Every check now carries - `must_flag` fixtures — strings that MUST trip it — and the script fails if - any fixture is not caught. A check that cannot fail is treated as broken. -4. It let unrelated nearby correction language excuse a violation. Generic - allow-windows are gone; the one qualified check must span the exact API - occurrence it excuses. - -The four promised input documents and both fixture directions are mandatory. -Missing inputs, unreadable inputs, or an empty fixture side make the checker -itself fail with exit 2 before it reports document results. - -A false positive costs a minute. A false green costs a merge. -""" -import re -import sys -import glob -import os - -ROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..") - -REQUIRED_TARGETS = frozenset( - { - "docs/superpowers/plans/2026-08-08-1009-measurement-and-stage-0.md", - "docs/superpowers/plans/2026-08-08-1009-measurement-findings.md", - "docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md", - "docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md", - } -) - - -def targets(): - return sorted( - glob.glob(os.path.join(ROOT, "docs/superpowers/specs/2026-08-08-esi-*.md")) - + glob.glob(os.path.join(ROOT, "docs/superpowers/plans/2026-08-*1009*.md")) - ) - - -def target_errors(paths): - """Return setup errors when any document this gate promises to scan is absent.""" - present = { - os.path.relpath(path, ROOT).replace(os.sep, "/") - for path in paths - } - return [ - f" missing required document: {path}" - for path in sorted(REQUIRED_TARGETS - present) - ] - - -def load_documents(paths): - documents = {} - errors = [] - for path in paths: - try: - with open(path, encoding="utf-8") as source: - documents[path] = normalize(source.read()) - except (OSError, UnicodeError) as error: - errors.append(f" cannot read {os.path.relpath(path, ROOT)}: {error}") - return documents, errors - - -def normalize(text): - """Collapse wraps while removing Markdown blockquote continuation markers.""" - text = re.sub(r"(?m)^\s*>\s?", "", text) - text = re.sub(r"\s*\n\s*", " ", text) - text = re.sub(r"[ \t]{2,}", " ", text) - return text - - -class Check: - def __init__(self, name, bad, must_flag=(), must_pass=()): - self.name = name - self.bad = re.compile(bad, re.I) - self.must_flag = must_flag # strings this check MUST catch - self.must_pass = must_pass # corrected strings it must NOT catch - - def hits(self, normalized): - out = [] - for m in self.bad.finditer(normalized): - out.append(normalized[max(0, m.start() - 60):m.end() + 60].strip()) - return out - - -class QualifiedOccurrenceCheck(Check): - """Flag each occurrence unless its own local clause states the required relation.""" - - def __init__(self, name, occurrence, qualified, must_flag=(), must_pass=()): - super().__init__(name, occurrence, must_flag=must_flag, must_pass=must_pass) - self.qualified = re.compile(qualified, re.I) - - def hits(self, normalized): - out = [] - qualified_spans = [match.span() for match in self.qualified.finditer(normalized)] - for m in self.bad.finditer(normalized): - if any(lo <= m.start() and m.end() <= hi for lo, hi in qualified_spans): - continue - out.append(normalized[max(0, m.start() - 60):m.end() + 60].strip()) - return out - - -CHECKS = [ - Check( - "Stage 0 gates on FINAL PASS, not a bare Vary check", - # `[*_ ]*` absorbs markdown emphasis; `gated on a curl` is its own shape - # because the sentence boundary defeats a proximity match to "Stage 0". - r"gates?[*_ ]+stage[*_ ]*0" - r"(?!\s*only\s+(?:once|after|when)\s+(?:an?\s+)?[`*_]*final\s+pass[`*_]*" - r"(?:\s+is)?\s+(?:recorded|obtained|achieved)\b)" - r"|verdict[:*_ ]+pass[*_ ]*/[*_ ]*fail" - r"|gated\s+on\s+a\s+.?curl", - must_flag=[ - "inspect the `Vary` response header. **Gates Stage 0**, the only build item", - "**Verdict:** PASS / FAIL", - "Stage 0 next, shipped as the operator flag. Gated on a `curl`, reverses an origin-load cost", - "Inspect Vary. **Gates Stage 0** immediately. A separate sentence says FINAL PASS is required.", - "Gates Stage 0 immediately; FINAL PASS gates deployment later.", - "The Vary result gates Stage 0 only on a successful curl, while FINAL PASS gates production rollout.", - ], - must_pass=[ - "Gates Stage 0 only once a FINAL PASS is recorded", - "Gates Stage 0 only after a FINAL PASS is recorded", - "**Verdict:** FINAL PASS / PROVISIONAL PASS / FAIL", - ], - ), - Check( - "Rollback is not described as config-only", - r"rolls?\s+back\s+with\s+a\s+config\s+push" - r"|rollback\s+is\s+another\s+config\s+push\s+rather", - must_flag=[ - "reverses an origin-load cost, and rolls back with a config push. Closer to a defect fix", - "This rolls back with a config push, explicitly without a purge or TTL wait.", - ], - must_pass=[ - "the read path reverts with a config push, but full rollback also needs a C1 purge path", - ], - ), - Check( - "Template has one marker, not two seams", - r"\b(?:emit|emits|use|uses|place|places|insert|inserts)\b[^.]{0,100}" - r"(?:two|both)\s+(?:existing\s+)?(?:injection\s+)?seams" - r"|markers?\s+at\s+(?:the\s+)?two\s+(?:existing\s+)?(?:injection\s+)?seams" - r"|adSlots[^.]{0,60}stays?\s+in\s+the\s+template", - must_flag=[ - "The transform emits `esi:include` markers at the two existing injection seams instead", - "emit markers at the two seams", - "Emit markers at the two existing injection seams. Correction: not two; use one marker.", - ], - must_pass=[ - "one unconditional marker at the body-close seam. Not two: the head seam is not a template hole", - "The earlier draft used two injection seams; that statement was wrong.", - ], - ), - Check( - "Headers finalize before assembly, in prose and diagrams", - r"assemble\s*(→|->)\s*finaliz" - r"|finaliz\w*[^.]{0,40}(runs?|still run)[^.]{0,20}after\s+assembly", - must_flag=[ - "`origin → lol_html transform → fastly::cache::core → assemble → finalize`", - "esi assemble → finalize → client", - "Cookie and privacy finalization still run after assembly", - ], - must_pass=[ - "fastly::cache::core → finalize headers → stream assembly", - "Cookie and privacy finalization ran BEFORE assembly, not after", - ], - ), - Check( - "No nonexistent Core Cache APIs", - r"\b[A-Za-z_]\w*\.to_body\s*\(\s*\)", - must_flag=[ - "let template = found.to_body();", - "let template = found.to_body(); // there is no fallback", - ], - must_pass=["found.to_stream()? // fallible; there is no `to_body()`"], - ), - Check( - "A2/A3 not decided on root TTFB", - r"A3\s+beats\s+A2\s+on\s+TTFB", - must_flag=["2. A3 beats A2 on TTFB by a margin the reviewers ratify"], - must_pass=["A3 beats A2 on bids-ready time, adInit fire time, and first attributed paint"], - ), - Check( - "ESI not described as impossible", - r"\bESI\s+(?:is|remains)\s+(?:not\s+viable|structurally\s+(?:blocked|impossible))", - must_flag=[ - "Answers #1009: ESI is not viable here, for a structural reason", - "ESI is not viable here. An earlier revision discussed a different problem.", - ], - must_pass=["The first revision concluded that ESI was structurally blocked. Both claims are false"], - ), - QualifiedOccurrenceCheck( - "C1 and C2 purge surfaces not conflated", - r"InsertBuilder::surrogate_keys", - r"InsertBuilder::surrogate_keys(?:\(\[\.\.\.\]\))?" - r"(?:(?!InsertBuilder::surrogate_keys|[.]|\bnot\b|\bnever\b|\bno\s+longer\b).){0,60}" - r"(?:\(\s*Core\s+Cache\s*\)|(?:is|belongs\s+to)" - r"(?![^.]{0,20}\b(?:not|never)\b|[^.]{0,20}\bno\s+longer\b)" - r"[^.]{0,60}Core\s+Cache)", - must_flag=[ - "Purge is available in-process, with keys attached at insert via `InsertBuilder::surrogate_keys`. Rollback is therefore flip then purge.", - "Attach C1 keys via `InsertBuilder::surrogate_keys`. C2 is described below.", - "C1 uses `InsertBuilder::surrogate_keys`. Separately, `InsertBuilder::surrogate_keys` is the Core Cache API for C2.", - "C1 uses `InsertBuilder::surrogate_keys`, but `InsertBuilder::surrogate_keys` is the Core Cache API for C2.", - "`InsertBuilder::surrogate_keys` is the Core Cache API for C2, but C1 uses `InsertBuilder::surrogate_keys`.", - "C1 uses `InsertBuilder::surrogate_keys`; it is not the Core Cache API.", - "C1 uses `InsertBuilder::surrogate_keys`; it no longer belongs to the Core Cache API.", - "This is not C2; `InsertBuilder::surrogate_keys` powers C1.", - ], - must_pass=[ - "`InsertBuilder::surrogate_keys` is the Core Cache API and keys C2. It does **not** key C1.", - ], - ), -] - - -def self_test(): - """A check that cannot fail is broken. Prove each one still fires.""" - broken = [] - for c in CHECKS: - if not c.must_flag: - broken.append(f" {c.name!r}: has no must_flag fixtures") - if not c.must_pass: - broken.append(f" {c.name!r}: has no must_pass fixtures") - for bad in c.must_flag: - if not c.hits(normalize(bad)): - broken.append(f" {c.name!r}: FAILED to flag known-bad text:\n {bad[:110]}") - for good in c.must_pass: - if c.hits(normalize(good)): - broken.append(f" {c.name!r}: wrongly flagged corrected text:\n {good[:110]}") - - if not target_errors([]): - broken.append(" target guard accepted an empty document set") - synthetic_targets = [os.path.join(ROOT, path) for path in REQUIRED_TARGETS] - if errors := target_errors(synthetic_targets): - broken.append(f" target guard rejected its required manifest: {errors}") - return broken - - -def report_checker_errors(errors): - print("CHECKER IS BROKEN — its own fixtures or document setup do not pass.") - print("A green run from this state would be meaningless.\n") - print("\n".join(errors)) - - -def main(): - paths = targets() - broken = self_test() + target_errors(paths) - if broken: - report_checker_errors(broken) - return 2 - - documents, read_errors = load_documents(paths) - if read_errors: - report_checker_errors(read_errors) - return 2 - - fail = False - for c in CHECKS: - found = [] - for path, document in documents.items(): - for h in c.hits(document): - found.append(f" {os.path.relpath(path, ROOT)}: …{h}…") - if found: - fail = True - print(f"\n[FAIL] {c.name}") - print("\n".join(found)) - else: - print(f"[ok] {c.name}") - - print() - print("Contradictions found — do not push." if fail else - f"All {len(CHECKS)} invariants hold (self-test passed).") - return 1 if fail else 0 - - -if __name__ == "__main__": - sys.exit(main()) From 62c4b703e754e0a955cb383bbfdb4f84060e54e1 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 18:27:40 +0530 Subject: [PATCH 12/44] Align the #1009 spec structure with what it actually covers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three structural fixes, no content changes. The title said "ESI and the Cacheable Root" while the actionable front of the document — sections 1 through 4 — is entirely Stage 0. ESI now lives in one section, one appendix, and mostly in a separate plan. Retitled to match. The filename keeps its esi- prefix deliberately: the commit history and every cross-reference point at it, and renaming would cost more than the mismatch. Added a document map. Three documents answer #1009 and nothing said which owns what, which is the seam every cross-document contradiction has appeared in. It also tells a reader arriving from the issue where the ESI answer actually is, rather than leaving them to infer it from a Stage 0 design document. Consolidated the staging. Stage 0 lived in section 4 while Stages 1 through 5 lived in section 7, so the sequence was split across two places, and Stage 5 had become an entry that read "superseded, see the other plan" — a staging list containing something that is not a stage. There is now one table, Stage 5 is gone, and ESI is stated as running independently of Stages 1 through 4 rather than queued behind them. Two stale "Stages 3b-5" ranges followed from that and are corrected. --- ...08-esi-cacheable-root-validation-design.md | 49 +++++++++++++++---- 1 file changed, 40 insertions(+), 9 deletions(-) diff --git a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md index 39a65dac5..40b2b68f1 100644 --- a/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md +++ b/docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md @@ -1,4 +1,7 @@ -# ESI and the Cacheable Root +# The Cacheable Root: Latency Diagnosis and Stage 0 Design + +_Filename retains its original `esi-` prefix; the commit history and every +cross-reference point at it. The subject moved, the path did not._ **Issue:** IABTechLab/trusted-server#1009 · **Date:** 2026-08-08 · **Revised:** 2026-08-10 @@ -36,6 +39,22 @@ > here is the latency re-diagnosis and the Stage 0 optimisation, which are worth doing > and are **not** an answer to #1009. +## Document map — read this first + +#1009 is answered across three documents, not one. This is the only place that says +which owns what. + +| Document | Owns | +| ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------- | +| **This spec** | Why the TTFB regression happens, what Stage 0 is and why, and the corrected ESI feasibility verdict | +| [Stage 0 plan](../plans/2026-08-08-1009-measurement-and-stage-0.md) | Implementing the measurement and the cache-bypass flag. **Does not close #1009.** | +| [ESI validation spike](../plans/2026-08-10-1009-esi-validation-spike.md) | **Where #1009 is actually decided.** Four arms, safety gates, decision rule. | +| [Findings](../plans/2026-08-08-1009-measurement-findings.md) | Recorded results. Currently: Step A only, at `PROVISIONAL PASS`. | + +**If you want the ESI answer**, it is [§2](#2-why--the-three-findings) for the verdict, +[§6.6](#66-the-esi-pipeline-corrected) for the pipeline, and the spike plan for how it +gets validated. Everything else here is Stage 0 and the latency analysis behind it. + **Decision requested:** approve the four items in §1. > **Orientation.** #1009 asks whether Edge Side Includes (ESI) can cache page fragments @@ -67,7 +86,7 @@ | D1 | **ESI is feasible and unvalidated.** Validate it via [the spike plan](../plans/2026-08-10-1009-esi-validation-spike.md), not by deferring it. | Eng + product | | D2 | **Fund ~3 days of measurement** (§3). No dependencies. Can start immediately. | Eng | | D3 | **Approve Stage 0** — an operator flag disabling the origin cache bypass, subject to a **`FINAL PASS`** in §3. A `Vary` check alone is a `PROVISIONAL PASS` and is not a release gate. Rollback needs a purge path, not only a config push. | Eng | -| D4 | **Stages 1–2 queue behind the SSAT price defect and #938.** Stages 3b–5 unscheduled. | Product | +| D4 | **Stages 1–2 queue behind the SSAT price defect and #938.** Stages 3b–4 unscheduled. ESI is not in this queue — see §7. | Product | Rationale for D4 in [§8](#8-priority). Everything this document recommends _against_ doing is in [§7](#7-deferred-work-specified-not-scheduled), at deliberately lower @@ -501,7 +520,25 @@ This applies to any shared-template work, ESI or client-fill alike. The ## 7. Deferred work, specified not scheduled -Lower detail is deliberate. Full specifications are in the appendices. +**The full sequence, in one place.** Stage 0 is specified in [§4](#4-stage-0--the-only-build-item-recommended-now) +rather than repeated here; everything below it is deferred. + +| Stage | What | Status | +| ----- | ----------------------------------------------- | ---------------------------------------------- | +| **0** | Operator flag disabling the origin cache bypass | Recommended now. Gated on a `FINAL PASS`. §4. | +| 1 | Bid delivery off the response body | Deferred behind the correctness defects | +| 2 | Delete the `` hold | Deferred; one-way, needs a Stage 1 soak | +| 3a | Browser caching (`private, max-age` + `ETag`) | Specified, low risk, unscheduled | +| 3b | Shared cacheability | Blocked on geo suppression, `Vary`, and Step B | +| 4 | Purge wiring | Prerequisite for any TS-owned cache | + +**ESI is not a stage here.** It was Stage 5 in an earlier revision, queued behind the +rest. It no longer queues: it is feasible on the pinned SDK and is decided by +[its own spike plan](../plans/2026-08-10-1009-esi-validation-spike.md), which runs +independently of Stages 1–4. The shared template cache it needs is `fastly::cache::core` +([§6.6](#66-the-esi-pipeline-corrected)), not a new service. + +Lower detail below is deliberate. Full specifications are in the appendices. **Stage 1 — bid delivery off the response body.** The client fetches `/_ts/page-bids` at navigation generation 0. Endpoint, same-origin gate, wire shape, and client consumer @@ -558,12 +595,6 @@ cache — not the HTTP read-through cache C1. This is **missing wiring, not a platform limit.** Until it exists, any TS-owned cache is TTL-only and a config push takes up to one TTL to take effect. -**Stage 5 — ESI.** Superseded. ESI no longer waits on a "revival condition"; it is -feasible on the pinned SDK and is validated by -[its own spike plan](../plans/2026-08-10-1009-esi-validation-spike.md), which does not -queue behind Stages 1–4. The shared template cache it needs is -`fastly::cache::core` ([§6.6](#66-the-esi-pipeline-corrected)), not a new service. - **Identity needs no work.** A new visitor's first navigation sets the EC cookie and the privacy net downgrades that one response; every later navigation sets no cookie and is cacheable. First-visit parity is a non-goal; if ever wanted, move cookie issuance onto From bcf2fd26d7c3209d325a0100d55510f268ed944a Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 18:57:55 +0530 Subject: [PATCH 13/44] Add the esi crate to the Fastly adapter and record Task 1 as passed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cheapest falsifier for #1009 clears. esi 0.7.1 compiles clean on Rust 1.95.0 for wasm32-wasip1, all six clippy targets pass, format is clean, and the integration-tests crate still resolves. ESI is not blocked by this toolchain. Nine new transitive dependencies, none of them displacing an existing one: esi, nom 8, rand 0.10, rand_core 0.10, chacha20, cpufeatures, atoi, html-escape, md5. regex stays at 1.12.4, bytes at 1.12.0 and log at 0.4.33. nom and rand gain new majors that coexist with the versions already in the tree rather than replacing them, which is the outcome that keeps this cheap — a forced bump on a shared dependency is what would have made it expensive. The dependency is added and unused. It belongs to the Fastly adapter rather than trusted-server-core, because the crate is hard-bound to fastly::{Request, Response, Backend} and core has to stay portable across the four adapters. Also corrects a claim in the spike plan that this task falsified. Step 3 told the implementer to check for a desync between the root lockfile and one at crates/trusted-server-integration-tests/Cargo.lock. That file does not exist: the crate is a workspace member and shares the root lockfile, so the hazard cannot arise in that form. The step now checks the thing that does matter, which is whether an existing shared dependency was forced to move. Compiling is not working. Nothing here exercises cache::core, ESI assembly, or any runtime behaviour, and Tasks 2 onward are untouched. --- Cargo.lock | 112 ++++++++++++++++-- .../trusted-server-adapter-fastly/Cargo.toml | 1 + .../2026-08-08-1009-measurement-findings.md | 37 ++++++ .../2026-08-10-1009-esi-validation-spike.md | 28 +++-- 4 files changed, 160 insertions(+), 18 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index cb8f40c68..b21216fa3 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -146,7 +146,7 @@ dependencies = [ "asn1-rs-derive", "asn1-rs-impl", "displaydoc", - "nom", + "nom 7.1.3", "num-traits", "rusticata-macros", "thiserror 1.0.69", @@ -254,6 +254,15 @@ dependencies = [ "tungstenite", ] +[[package]] +name = "atoi" +version = "2.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f28d99ec8bfea296261ca1af174f24225171fea9664ba9003cbebee704810528" +dependencies = [ + "num-traits", +] + [[package]] name = "atomic-waker" version = "1.1.2" @@ -573,7 +582,18 @@ checksum = "c3613f74bd2eac03dad61bd53dbe620703d4371614fe0bc3b9f04dd36fe4e818" dependencies = [ "cfg-if", "cipher", - "cpufeatures", + "cpufeatures 0.2.17", +] + +[[package]] +name = "chacha20" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d524456ba66e72eb8b115ff89e01e497f8e6d11d78b70b1aa13c0fbd97540a81" +dependencies = [ + "cfg-if", + "cpufeatures 0.3.0", + "rand_core 0.10.1", ] [[package]] @@ -583,7 +603,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "10cd79432192d1c0f4e1a0fef9527696cc039165d729fb41b3f4f4f354c2dc35" dependencies = [ "aead", - "chacha20", + "chacha20 0.9.1", "cipher", "poly1305", "zeroize", @@ -916,6 +936,15 @@ dependencies = [ "libc", ] +[[package]] +name = "cpufeatures" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b2a41393f66f16b0823bb79094d54ac5fbd34ab292ddafb9a0456ac9f87d201" +dependencies = [ + "libc", +] + [[package]] name = "crc32fast" version = "1.5.0" @@ -1041,7 +1070,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be" dependencies = [ "cfg-if", - "cpufeatures", + "cpufeatures 0.2.17", "curve25519-dalek-derive", "digest 0.10.7", "fiat-crypto", @@ -1186,7 +1215,7 @@ checksum = "5cd0a5c643689626bec213c4d8bd4d96acc8ffdb4ad4bb6bc16abf27d5f4b553" dependencies = [ "asn1-rs", "displaydoc", - "nom", + "nom 7.1.3", "num-bigint", "num-traits", "rusticata-macros", @@ -1690,6 +1719,27 @@ dependencies = [ "rustc_version", ] +[[package]] +name = "esi" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e384a711090b57e3dd20080915935607078ab0b43d49575994b44dd36956f84" +dependencies = [ + "atoi", + "base64", + "bytes", + "chrono", + "fastly", + "html-escape", + "log", + "md5", + "nom 8.0.0", + "percent-encoding", + "rand 0.10.2", + "regex", + "thiserror 2.0.18", +] + [[package]] name = "etcetera" version = "0.10.0" @@ -2034,6 +2084,7 @@ dependencies = [ "cfg-if", "libc", "r-efi 6.0.0", + "rand_core 0.10.1", ] [[package]] @@ -2188,6 +2239,12 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "html-escape" +version = "0.2.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c9356095b4b41197bba32173600e1582792cda618f65d12f68e2e77d273413c5" + [[package]] name = "html5ever" version = "0.35.0" @@ -2914,6 +2971,12 @@ version = "0.9.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8863b587001c1b9a8a4e36008cebc6b3612cb1226fe2de94858e06092687b608" +[[package]] +name = "md5" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7ebb8d8732c6a6df3d8f032a82911cfc747e00efb95cc46e8d0acd5b5b88570c" + [[package]] name = "memchr" version = "2.8.2" @@ -2984,6 +3047,15 @@ dependencies = [ "minimal-lexical", ] +[[package]] +name = "nom" +version = "8.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "df9761775871bdef83bee530e60050f7e54b1105350d6884eb0fb4f46c2f9405" +dependencies = [ + "memchr", +] + [[package]] name = "num" version = "0.4.3" @@ -3477,7 +3549,7 @@ version = "0.8.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8159bd90725d2df49889a078b54f4f79e87f1f8a8444194cdca81d38f5393abf" dependencies = [ - "cpufeatures", + "cpufeatures 0.2.17", "opaque-debug", "universal-hash", ] @@ -3775,6 +3847,17 @@ dependencies = [ "rand_core 0.9.5", ] +[[package]] +name = "rand" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c7f5fa3a058cd35567ef9bfa5e75732bee0f9e4c55fa90477bef2dfcdbc4be80" +dependencies = [ + "chacha20 0.10.1", + "getrandom 0.4.3", + "rand_core 0.10.1", +] + [[package]] name = "rand_chacha" version = "0.3.1" @@ -3813,6 +3896,12 @@ dependencies = [ "getrandom 0.3.4", ] +[[package]] +name = "rand_core" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69" + [[package]] name = "rcgen" version = "0.13.2" @@ -4079,7 +4168,7 @@ version = "4.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "faf0c4a6ece9950b9abdb62b1cfcf2a68b3b67a10ba445b3bb85be2a293d0632" dependencies = [ - "nom", + "nom 7.1.3", ] [[package]] @@ -4494,7 +4583,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e3bf829a2d51ab4a5ddf1352d8470c140cadc8301b2ae1789db023f01cedd6ba" dependencies = [ "cfg-if", - "cpufeatures", + "cpufeatures 0.2.17", "digest 0.10.7", ] @@ -4506,7 +4595,7 @@ checksum = "4d58a1e1bf39749807d89cf2d98ac2dfa0ff1cb3faa38fbb64dd88ac8013d800" dependencies = [ "block-buffer 0.9.0", "cfg-if", - "cpufeatures", + "cpufeatures 0.2.17", "digest 0.9.0", "opaque-debug", ] @@ -4518,7 +4607,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" dependencies = [ "cfg-if", - "cpufeatures", + "cpufeatures 0.2.17", "digest 0.10.7", ] @@ -5276,6 +5365,7 @@ dependencies = [ "edgezero-adapter-fastly", "edgezero-core", "error-stack", + "esi", "fastly", "fern", "futures", @@ -6325,7 +6415,7 @@ dependencies = [ "data-encoding", "der-parser", "lazy_static", - "nom", + "nom 7.1.3", "oid-registry", "ring", "rusticata-macros", diff --git a/crates/trusted-server-adapter-fastly/Cargo.toml b/crates/trusted-server-adapter-fastly/Cargo.toml index b6bc0f1a1..cf73a2040 100644 --- a/crates/trusted-server-adapter-fastly/Cargo.toml +++ b/crates/trusted-server-adapter-fastly/Cargo.toml @@ -18,6 +18,7 @@ chrono = { workspace = true } edgezero-adapter-fastly = { workspace = true, features = ["fastly"] } edgezero-core = { workspace = true } error-stack = { workspace = true } +esi = "0.7" fastly = { workspace = true } fern = { workspace = true } futures = { workspace = true } diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md index a092f6c52..831101f7b 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md @@ -152,6 +152,43 @@ available (or roll a versioned key namespace), **then** observe past the origin declaring the incident closed. With no C1 purge path wired, the tail is the TTL itself — roughly a minute here, and a recorded risk rather than a surprise. +## ESI spike Task 1 — does `esi` 0.7 build on this toolchain? + +**Date:** 2026-08-10 · **Verdict: PASS.** The cheapest falsifier for the ESI question +clears. #1009 is not closed by a toolchain limit. + +| Check | Result | +| ------------------------------------------------------------------------------------------ | -------------------- | +| `cargo add esi@0.7 --package trusted-server-adapter-fastly` | resolved `esi 0.7.1` | +| `cargo check-fastly` (Rust 1.95.0 / `wasm32-wasip1`) | clean | +| `cargo fmt --all -- --check` | clean | +| All six clippy targets (fastly, axum, cloudflare, cloudflare-wasm, spin-native, spin-wasm) | clean | +| `cargo check --manifest-path crates/trusted-server-integration-tests/Cargo.toml --tests` | clean | + +**Nine new transitive dependencies:** `esi 0.7.1`, `nom 8.0.0`, `rand 0.10.2`, +`rand_core 0.10.1`, `chacha20 0.10.1`, `cpufeatures 0.3.0`, `atoi 2.0.0`, +`html-escape 0.2.15`, `md5 0.8.1`. + +**No existing shared dependency moved.** `regex` stays 1.12.4, `bytes` 1.12.0, `log` +0.4.33. `nom` and `rand` gain new majors that coexist with the existing 7.1.3 / 0.8.6 / +0.9.4 rather than replacing them — the best available outcome, since a forced bump on a +shared dep is what would have made this expensive. + +### A claim in the spike plan was wrong + +Task 1 Step 3 told the implementer to check for a desync between the root `Cargo.lock` and +`crates/trusted-server-integration-tests/Cargo.lock`. **That second lockfile does not +exist.** The integration-tests crate is a workspace member (root `Cargo.toml:10`) and +shares the root lockfile, so the desync hazard cannot arise in that form. The plan has +been corrected. The dual-lockfile constraint was real at some earlier point; it is not the +current layout. + +### Not yet verified + +Compiling is not working. Nothing here exercises `cache::core`, ESI assembly, or any +runtime behaviour — Tasks 2 onward remain untouched, and the `esi` dependency is added but +unused. + ## Step B — consumers of TS's own response headers Not yet run. diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index 354a41b85..4bc3cc3b5 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -137,21 +137,35 @@ cargo check-fastly Expected: clean. The crate declares edition 2021 with no `rust-version`, and pulls recent `rand` and `nom`, so this is a genuine question on Rust 1.95.0 / `wasm32-wasip1`. -- [ ] **Step 3: Check the lockfiles have not desynced** +- [ ] **Step 3: Check no shared dependency was forced to move** ```bash git diff --stat Cargo.lock -cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity +cargo check --manifest-path crates/trusted-server-integration-tests/Cargo.toml --tests \ + --target "$(rustc -vV | sed -n 's/^host: //p')" ``` -CI requires shared direct deps to match between the root and integration-tests lockfiles. -`regex`, `bytes`, and `log` overlap. If they desync, fix with targeted -`cargo update -p --precise ` — **never a full update**. +**Correction, verified 2026-08-10:** an earlier revision of this step warned about a +desync between the root `Cargo.lock` and `crates/trusted-server-integration-tests/Cargo.lock`. +**That second lockfile does not exist** — the crate is a workspace member (root +`Cargo.toml:10`) and shares the root lockfile. The hazard cannot arise in that form. + +What does matter is whether adding `esi` forces an **existing** shared dependency to a new +version, since `regex`, `bytes`, and `log` are used across the workspace. Adding a new +major that coexists is harmless; moving an existing one is not. If one moves, fix with a +targeted `cargo update -p --precise ` — **never a full update**. + +**Already run and recorded** in [the findings](./2026-08-08-1009-measurement-findings.md): +no existing shared dependency moved. - [ ] **Step 4: Record and commit, or stop** -If Step 2 fails, this plan stops here and #1009 is answered "not on this toolchain." -Record that in the findings document and escalate rather than fighting the build. +**Task 1 is complete — verdict PASS, recorded 2026-08-10.** `esi` 0.7.1 compiles clean on +Rust 1.95.0 / `wasm32-wasip1`, all six clippy targets pass, and no existing shared +dependency moved. See [the findings](./2026-08-08-1009-measurement-findings.md). + +Had Step 2 failed, this plan would have stopped here with #1009 answered "not on this +toolchain." It did not. ```bash git add crates/trusted-server-adapter-fastly/Cargo.toml Cargo.lock From b35f8df4d39b99183480f54c54094d3ea88e072a Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 19:05:18 +0530 Subject: [PATCH 14/44] Re-sequence the ESI spike for local-first validation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit You asked whether a Fastly test service is really needed. Probed it rather than reasoned about it: Viceroy 0.17 implements the whole Core Cache surface this spike uses. A temporary test under cargo test -p trusted-server-adapter-fastly --target wasm32-wasip1 exercised insert/finish/lookup/to_stream, and then the shape Task 3 Step 4 actually specifies — Transaction::lookup, must_insert_or_update, insert(...).surrogate_keys(...).execute_and_stream_back(), and hit-after-insert. All passed. The probe is removed; the result is recorded in the findings. So provisioning is not a prerequisite. An earlier revision made it Task 2 and a blocker on everything downstream, which would have stalled the spike on infrastructure it does not need yet. Almost all the correctness and safety work runs locally: the C2 cache logic, the transform, template byte-identity, ESI assembly (the crate is pure Rust over BufRead/Write), DCA and dispatcher refusal, fragment-failure degradation, header ordering, and the leakage gates. Task 2 is now scoped to what genuinely needs a real service and is no longer on the critical path; the dependency graph reflects that. Two caveats recorded rather than glossed. Viceroy is a single instance, so a passing Transaction test proves the API works and not that request collapsing behaves under load. And local timings are meaningless for Task 7's decision rule — every performance number still needs the real service. --- .../2026-08-08-1009-measurement-findings.md | 34 ++++++++- .../2026-08-10-1009-esi-validation-spike.md | 76 +++++++++++++------ 2 files changed, 83 insertions(+), 27 deletions(-) diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md index 831101f7b..9e68acb27 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md @@ -183,11 +183,39 @@ shares the root lockfile, so the desync hazard cannot arise in that form. The pl been corrected. The dual-lockfile constraint was real at some earlier point; it is not the current layout. +### Viceroy 0.17 supports the whole Core Cache surface this spike needs + +**Date:** 2026-08-10 · **Verdict: PASS.** Probed directly under +`cargo test -p trusted-server-adapter-fastly --target wasm32-wasip1`, then removed: + +| API | Result | +| ------------------------------------------------------------------------ | ------ | +| `cache::core::insert(key, ttl).execute()` → write → `finish()` | works | +| `cache::core::lookup(key).execute()` → `Found::to_stream()` | works | +| `Transaction::lookup(key).execute()` → `must_insert_or_update()` | works | +| `Transaction::insert(ttl).surrogate_keys([…]).execute_and_stream_back()` | works | +| Second transactional lookup reports a hit, no obligation | works | + +That is the entire API surface the spike's Task 3 Step 4 specifies, including the +transaction and stream-back shapes. + +**Consequence: provisioning a Fastly service is not a prerequisite.** An earlier revision +of the spike plan made it Task 2 and a blocker on everything downstream. Almost all of the +correctness and safety work — the C2 cache logic, the transform, template byte-identity, +ESI assembly, DCA and dispatcher refusal, fragment-failure degradation, header ordering, +and the leakage gates — runs locally. The plan is re-sequenced accordingly. + +**What still needs a real service:** shielding behaviour, POP-level cache tiering, +request collapsing under genuine concurrency (Viceroy is a single instance, so a passing +`Transaction` test proves the API works and not that collapsing is correct under load), +stale revalidation timing, and **every performance number in the decision rule**. Local +timings are meaningless for the decision. + ### Not yet verified -Compiling is not working. Nothing here exercises `cache::core`, ESI assembly, or any -runtime behaviour — Tasks 2 onward remain untouched, and the `esi` dependency is added but -unused. +Compiling and a cache round-trip are not an implementation. Nothing yet exercises the +`lol_html` transform into C2, ESI assembly, or any runtime behaviour on the publisher path, +and the `esi` dependency is added but unused. ## Step B — consumers of TS's own response headers diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index 4bc3cc3b5..f1f3776f5 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -97,19 +97,19 @@ not its implementation. ## Task order and dependencies ``` -Stage 0 plan (flag + timing instrumentation) ──┐ - ├──> Task 3 (C2 template cache) -Task 1 (esi crate compiles) ───────────────────┤ -Task 2 (test service + harness) ────────────────┘ │ - ├──> Task 4 (A2 client-fill) - ├──> Task 5 (A3 ESI) - └──> Task 6 (safety gates) - │ - └──> Task 7 (decision record) +Task 1 (esi compiles) ── DONE, PASS ──┐ + ├──> Task 3 (C2 cache) ─┬──> Task 4 (A2 client-fill) +Stage 0 plan (flag + instrumentation) ┘ ├──> Task 5 (A3 ESI) + └──> Task 6 (safety gates) + │ + Task 2 (real service) ─────────────────────────────┴──> Task 7 (decision) ``` -Tasks 1 and 2 are independent and should run first — both can invalidate the plan -cheaply. Task 6 runs against every arm, not once at the end. +**Task 2 is not a blocker on Tasks 3–6.** Everything those tasks need is exercisable under +Viceroy 0.17 — verified, see Task 2. The real service is required only for the +measurements Task 7 decides on, so provision it once there is something worth measuring. + +Task 6 runs against every arm, not once at the end. --- @@ -174,19 +174,47 @@ git commit -m "Add the esi crate to the Fastly adapter for the #1009 validation --- -## Task 2: Stand up the test service and the harness - -**Viceroy 0.17 does support `cache::core` locally** — an earlier draft of this plan said -otherwise and was wrong. What it does **not** support is the customized HTTP -read-through hooks (`after_send` / `set_body_transform`), which matters only if the -alternative design in Task 3 Step 4 is chosen. - -So: C2 insert/lookup/transaction logic, the transform, and the security properties are all -testable locally. **Shielding, request collapsing under real concurrency, POP behaviour, -and stale revalidation are not** — those need a real Fastly service. Establish one before -Tasks 3–6, and be clear which findings came from which environment. - -- [ ] **Step 1: Provision a dedicated test service** +## Task 2: Local validation first, real service only for what needs it + +**Verified 2026-08-10 under Viceroy 0.17: the entire Core Cache surface this spike uses +works locally.** A probe exercised `cache::core::insert`, `lookup`, `finish`, `to_stream`, +and — the shape Task 3 Step 4 actually specifies — `Transaction::lookup`, +`must_insert_or_update`, `insert(...).surrogate_keys(...).execute_and_stream_back()`, and +hit-after-insert semantics. All passed. Recorded in +[the findings](./2026-08-08-1009-measurement-findings.md). + +That reorders this plan. An earlier revision made provisioning a Fastly service Task 2 and +a blocker on everything after it. It is not a blocker: **almost all of the correctness and +safety work is local**, and only the numbers and the cache topology need real +infrastructure. + +| Work | Where | +| ------------------------------------------------------------ | ------------ | +| C2 insert / lookup / transaction logic (Task 3) | **Local** | +| The `lol_html` transform and template byte-identity (Task 3) | **Local** | +| ESI assembly — the crate is pure Rust over `BufRead`/`Write` | **Local** | +| DCA off, dispatcher allowlist, injection refusal (Task 5) | **Local** | +| Fragment-failure degradation (Task 5) | **Local** | +| Header-finalization ordering, no-C3 assertions (Task 6) | **Local** | +| Cross-user leakage / request-neutrality gates (Task 6) | **Local** | +| Shielding behaviour | Real service | +| POP-level cache tiering (`x-cache`, `hit-state`, `age`) | Real service | +| Request collapsing under genuine concurrency | Real service | +| Stale revalidation timing at the edge | Real service | +| **Every performance number in Task 7's decision rule** | Real service | + +**So: build and prove correctness locally through Tasks 3, 5, and 6 before provisioning +anything.** If the design is wrong or leaks, that surfaces locally for free, and the +service is only needed once there is something worth measuring. + +Two caveats on the local scope. Viceroy is a single instance, so a passing `Transaction` +test proves the API works, **not** that collapsing behaves correctly under load. And local +timings are meaningless for the decision — do not let a fast local run substitute for +Task 7 evidence. + +### When the real service is needed + +- [ ] **Step 1: Provision it — after local correctness passes, not before** Separate from production. Confirm and record: whether the publisher backend is **shielded**, and whether any Delivery service fronts the Compute service. Both change From 7781009d4b5de15b2a2d5b2bc5292a44e0feebc1 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 19:27:30 +0530 Subject: [PATCH 15/44] Add AssemblyMode and gate the head seam on template neutrality MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First implementation step of the #1009 ESI spike. No behaviour change: the mode defaults to Inline and every existing path is unaffected. AssemblyMode lives on CreativeOpportunitiesConfig as Option with skip_serializing_if, following the section_root pattern already established there. The reason is in that struct's own doc comments: these types use deny_unknown_fields, so a pushed key makes an older binary fail configuration load. Keeping the key absent when unset means a deployment that never sets it stays rollback-compatible. A test asserts the unset value is not serialized, so that property cannot regress silently. The head seam now goes through template_ad_slots_script rather than an inline conditional. Under Inline it keeps today's behaviour, emitting adSlots only when the ad stack runs, which is correct for a response that is never shared. Under ClientFill and Esi it returns None unconditionally, because should_run_ad_stack folds in consent, bot classification, prefetch status and the auction kill switch. A shared template that emitted conditionally would freeze the first-filling request's decision for every later reader: a consent-denied fill would serve a no-ads template to consenting users, and a consenting fill would serve ad markup to someone who refused. Three tests, and the shape of them matters. An absence-of-per-user-values scan would have passed the broken design, because adSlots content really is derived from config and path. What catches it is byte-identity across requests differing only in the gating decision, so that is what is asserted — including across differing slot matches. The inline test exists so a future change cannot make the shared-mode assertions pass by breaking the shipped path. Extracting the decision as a pure function is deliberate: it makes the invariant testable without driving the whole pipeline, which is what let these tests be written before any cache work exists. Verified: fmt, all six clippy targets, and all four adapter suites, including 1838 core tests under Viceroy. --- .../src/creative_opportunities.rs | 113 +++++++++++ crates/trusted-server-core/src/publisher.rs | 176 +++++++++++++++++- 2 files changed, 281 insertions(+), 8 deletions(-) diff --git a/crates/trusted-server-core/src/creative_opportunities.rs b/crates/trusted-server-core/src/creative_opportunities.rs index 10a85b3e8..94fd2fce1 100644 --- a/crates/trusted-server-core/src/creative_opportunities.rs +++ b/crates/trusted-server-core/src/creative_opportunities.rs @@ -183,6 +183,36 @@ fn derive_section(path: &str, section_root: &str, section_segment: usize) -> Str } } +/// How per-user ad state reaches the page. +/// +/// `Inline` is the shipped behaviour: the auction result is injected before +/// `` and the root document is therefore uncacheable. The other two serve +/// a request-neutral shared template and fill the per-user holes afterwards — +/// `ClientFill` from the browser, `Esi` at the edge. +/// +/// Spike-only, for the #1009 ESI validation. Remove with the spike. +/// +/// # Why the template must be request-neutral +/// +/// Under `ClientFill` and `Esi` the template is shared across visitors, so +/// nothing whose *presence* depends on the request may appear in it — not merely +/// nothing whose *value* does. `tsjs.adSlots` is the trap: its content is derived +/// from config and path, but whether it is emitted at all is gated on consent, +/// bot classification, prefetch status and the auction kill switch. A template +/// filled by the first request would freeze that request's decision for every +/// later reader. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Deserialize, Serialize)] +#[serde(rename_all = "snake_case")] +pub enum AssemblyMode { + /// Inject bids inline before ``. Root uncacheable. Shipped behaviour. + #[default] + Inline, + /// Serve a shared template; the browser fetches the per-user fragment. + ClientFill, + /// Serve a shared template; assemble the fragment at the edge with ESI. + Esi, +} + /// Top-level configuration for the creative opportunities system. #[derive(Debug, Clone, Deserialize, Serialize)] #[serde(deny_unknown_fields)] @@ -244,11 +274,30 @@ pub struct CreativeOpportunitiesConfig { /// [`section_root`](Self::section_root) are omitted. #[serde(default, skip_serializing_if = "Option::is_none")] pub section_segment: Option, + /// How per-user ad state reaches the page. Absent means + /// [`AssemblyMode::Inline`], the shipped behaviour. + /// + /// `Option` rather than a bare enum, and `skip_serializing_if`, deliberately: + /// these structs use `deny_unknown_fields`, so a pushed key makes an older + /// binary fail configuration load. Keeping it absent when unset means a + /// deployment that never sets it stays rollback-compatible. + /// + /// Spike-only. See [`AssemblyMode`]. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub assembly_mode: Option, /// Slot templates. Empty vec = feature disabled (no auction fired, no globals injected). #[serde(default, deserialize_with = "vec_from_seq_or_map")] pub slot: Vec, } +impl CreativeOpportunitiesConfig { + /// Resolved assembly mode, defaulting to [`AssemblyMode::Inline`] when unset. + #[must_use] + pub fn assembly_mode(&self) -> AssemblyMode { + self.assembly_mode.unwrap_or_default() + } +} + impl CreativeOpportunitiesConfig { /// Derives the `{section}` value for `path` under this config's section /// policy ([`section_root`](Self::section_root) and @@ -1149,6 +1198,7 @@ mod tests { auction_timeout_ms: None, price_granularity: PriceGranularity::default(), section_root: section_root.map(str::to_string), + assembly_mode: None, section_segment: None, slot: vec![slot], } @@ -1546,6 +1596,7 @@ mod tests { auction_timeout_ms: None, price_granularity: PriceGranularity::default(), section_root: None, + assembly_mode: None, section_segment: None, slot: Vec::new(), }; @@ -1824,6 +1875,68 @@ mod tests { ); } + #[test] + fn assembly_mode_defaults_to_inline_when_absent() { + // Arrange: the minimal config an existing deployment would have. + let toml = r#" + gam_network_id = "99999" + "#; + + // Act + let config: CreativeOpportunitiesConfig = + toml::from_str(toml).expect("should deserialize without assembly_mode"); + + // Assert + assert_eq!( + config.assembly_mode, None, + "an absent key should stay absent rather than materializing a value" + ); + assert_eq!( + config.assembly_mode(), + AssemblyMode::Inline, + "should resolve to the shipped inline behaviour" + ); + } + + #[test] + fn assembly_mode_deserializes_each_variant() { + for (raw, expected) in [ + ("inline", AssemblyMode::Inline), + ("client_fill", AssemblyMode::ClientFill), + ("esi", AssemblyMode::Esi), + ] { + let toml = format!( + r#" + gam_network_id = "99999" + assembly_mode = "{raw}" + "# + ); + let config: CreativeOpportunitiesConfig = + toml::from_str(&toml).unwrap_or_else(|e| panic!("should parse {raw}: {e}")); + assert_eq!( + config.assembly_mode(), + expected, + "should resolve `{raw}` to {expected:?}" + ); + } + } + + #[test] + fn unset_assembly_mode_is_omitted_from_serialized_config() { + // `deny_unknown_fields` means a pushed key breaks config load on an older + // binary. A deployment that never sets this must not gain the key just by + // round-tripping through a newer one. + let config: CreativeOpportunitiesConfig = + toml::from_str("gam_network_id = \"99999\"").expect("should deserialize"); + + let serialized = toml::to_string(&config).expect("should serialize"); + + assert!( + !serialized.contains("assembly_mode"), + "unset assembly_mode must not be serialized, got:\n{serialized}" + ); + } + #[test] fn prebid_slot_params_deserializes_without_bidders_field() { let json = r#"{"bidders": {}}"#; diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index efb9dd4f6..09d033d76 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -50,6 +50,7 @@ use crate::auction::types::{ use crate::consent::{consent_allows_server_side_auction, gate_eids_by_consent}; use crate::constants::{COOKIE_TS_EIDS, HEADER_X_COMPRESS_HINT}; use crate::cookies::handle_request_cookies; +use crate::creative_opportunities::{AssemblyMode, CreativeOpportunitiesConfig}; use crate::ec::EcContext; use crate::ec::kv::KvIdentityGraph; use crate::ec::registry::PartnerRegistry; @@ -2917,14 +2918,18 @@ pub async fn handle_publisher_request( crate::integrations::gpt_diagnostics::finalize_response(&gpt_diagnostics, &mut response); - let ad_slots_script = if should_run_ad_stack { - settings - .creative_opportunities - .as_ref() - .map(|co_config| build_ad_slots_script(&matched_slots, co_config, &request_path)) - } else { - None - }; + let assembly_mode = settings + .creative_opportunities + .as_ref() + .map(CreativeOpportunitiesConfig::assembly_mode) + .unwrap_or_default(); + let ad_slots_script = template_ad_slots_script( + assembly_mode, + should_run_ad_stack, + settings, + &matched_slots, + &request_path, + ); // §4.7: HTML with synthesized per-navigation auction state must not be // stored or validated as an origin representation. Strip both browser and @@ -3555,6 +3560,45 @@ fn match_renderable_slots( /// /// Property names match what the client-side TSJS bundle expects: /// `gam_unit_path`, `div_id`, `formats`, and `targeting`. +/// What the `` seam injects, given the assembly mode. +/// +/// Under [`AssemblyMode::Inline`] the response is per-navigation and not shared, +/// so emitting `tsjs.adSlots` only when the ad stack runs is correct. +/// +/// Under [`AssemblyMode::ClientFill`] and [`AssemblyMode::Esi`] the document is a +/// **shared template**, and `should_run_ad_stack` is request-dependent — it folds +/// in consent, bot classification, prefetch status and the auction kill switch. +/// Emitting conditionally there would freeze the first-filling request's decision +/// for every later reader of the cached object: a consent-denied fill would serve +/// a no-ads template to consenting users, and a consenting fill would serve ad +/// markup to someone who refused. +/// +/// So those modes return [`None`] **unconditionally**, and `adSlots` moves to the +/// per-request fragment alongside the bids. The head seam is not a template hole. +/// +/// See `docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md` +/// §6.7. +pub(crate) fn template_ad_slots_script( + mode: AssemblyMode, + should_run_ad_stack: bool, + settings: &Settings, + matched_slots: &[crate::creative_opportunities::CreativeOpportunitySlot], + request_path: &str, +) -> Option { + match mode { + AssemblyMode::ClientFill | AssemblyMode::Esi => None, + AssemblyMode::Inline => { + if !should_run_ad_stack { + return None; + } + settings + .creative_opportunities + .as_ref() + .map(|co_config| build_ad_slots_script(matched_slots, co_config, request_path)) + } + } +} + pub(crate) fn build_ad_slots_script( matched_slots: &[crate::creative_opportunities::CreativeOpportunitySlot], co_config: &crate::creative_opportunities::CreativeOpportunitiesConfig, @@ -4538,6 +4582,121 @@ mod tests { .expect("should proxy publisher request") } + mod template_neutrality_tests { + //! The gate for #1009's shared-template design. + //! + //! An "absence of per-user values" scan is not sufficient here: the bug + //! that nearly shipped was a *conditionally present* element whose own + //! content was per-URL. These tests assert byte-identity across requests + //! that differ only in the gating decision. + + use super::*; + use crate::creative_opportunities::{ + AssemblyMode, CreativeOpportunityFormat, CreativeOpportunitySlot, + }; + + fn slot() -> CreativeOpportunitySlot { + CreativeOpportunitySlot { + id: "atf".to_string(), + gam_unit_path: Some("/99999/example/home".to_string()), + div_id: Some("ad-atf".to_string()), + page_patterns: vec!["/**".to_string()], + formats: vec![CreativeOpportunityFormat { + width: 300, + height: 250, + media_type: MediaType::Banner, + }], + floor_price: None, + targeting: Default::default(), + providers: Default::default(), + compiled_patterns: Vec::new(), + compiled_unit: None, + } + } + + fn settings_with_slots() -> Settings { + let mut settings = crate::test_support::tests::create_test_settings(); + // Construct the section rather than mutating it if present: the shared + // fixture does not carry `[creative_opportunities]`, and an `if let + // Some(..)` here would silently no-op and make the inline assertion + // below vacuous. + settings.creative_opportunities = Some(CreativeOpportunitiesConfig { + gam_network_id: "99999".to_string(), + auction_timeout_ms: Some(500), + price_granularity: Default::default(), + section_root: None, + assembly_mode: None, + section_segment: None, + slot: vec![slot()], + }); + settings + } + + #[test] + fn shared_modes_emit_no_head_script_regardless_of_the_gating_decision() { + let settings = settings_with_slots(); + let slots = [slot()]; + + for mode in [AssemblyMode::ClientFill, AssemblyMode::Esi] { + let ran = template_ad_slots_script(mode, true, &settings, &slots, "/"); + let did_not_run = template_ad_slots_script(mode, false, &settings, &slots, "/"); + + assert_eq!( + ran, did_not_run, + "{mode:?}: the template must be byte-identical whether or not the ad \ + stack ran; a cached object cannot carry one request's consent, bot, \ + prefetch or kill-switch decision" + ); + assert_eq!( + ran, None, + "{mode:?}: adSlots belongs in the per-request fragment, not the template" + ); + } + } + + #[test] + fn inline_mode_keeps_its_request_dependent_behaviour() { + // Inline responses are per-navigation and never shared, so gating is + // correct there. This guards against "fixing" the shared-mode bug by + // breaking the shipped path. + let settings = settings_with_slots(); + let slots = [slot()]; + + assert!( + template_ad_slots_script(AssemblyMode::Inline, true, &settings, &slots, "/") + .is_some(), + "inline should emit adSlots when the ad stack runs" + ); + assert_eq!( + template_ad_slots_script(AssemblyMode::Inline, false, &settings, &slots, "/"), + None, + "inline should emit nothing when the ad stack does not run" + ); + } + + #[test] + fn shared_modes_are_neutral_across_differing_slot_matches() { + // Slot matching folds in the request path. Under a shared mode even + // that must not reach the template. + let settings = settings_with_slots(); + + let matched = template_ad_slots_script( + AssemblyMode::Esi, + true, + &settings, + &[slot()], + "/news/article", + ); + let unmatched = + template_ad_slots_script(AssemblyMode::Esi, true, &settings, &[], "/other"); + + assert_eq!( + matched, unmatched, + "the template must not vary with slot matching under a shared mode" + ); + } + } + mod ssat_cache_policy_tests { use super::*; use crate::auction::provider::AuctionProvider; @@ -8127,6 +8286,7 @@ mod tests { auction_timeout_ms: Some(500), price_granularity: PriceGranularity::Dense, section_root: None, + assembly_mode: None, section_segment: None, slot: Vec::new(), } From 606543f51694bb081a6f53203377fdba25ca8411 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 20:07:28 +0530 Subject: [PATCH 16/44] Add the C2 shared-template cache gate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 3 Step 3 of the #1009 ESI spike. No behaviour change: under the default Inline mode the gate reports InlineMode and does nothing. cache::core is not an HTTP cache. It stores whatever bytes it is handed and rejects nothing, so every safety condition belongs to the caller. c2_bypass_reason enumerates them rather than leaving them implicit: an authorized request, an origin Set-Cookie, a non-shareable Cache-Control, a non-200 status, and a non-HTML content type. Leak vectors are checked before mere ineligibility so an operator reading the log sees the security reason and not a content-type quibble. A DataDome block needs no separate detection — it replaces the document with a 403 and the status check covers it. There is a test saying so, because the next person will otherwise go looking for a marker that does not exist. Extracted is_uncacheable_by_cache_control into response_privacy rather than writing a third copy of the private/no-store predicate. It was already duplicated verbatim in both arms of the cookie-privacy net; this replaces both. The helper deliberately does not treat no-cache as disqualifying, because no-cache means revalidate before reuse rather than do not store, and the cookie-privacy net's reading is the correct one for HTTP. The C2 gate checks no-cache separately, as the stricter reading is right for a spike-owned cache we control. The gate has a real call site that logs its decision rather than an allow(dead_code). Clippy pushed back on the annotation and was right to: an #[expect] could not be satisfied in both the lib and test targets, and the honest answer was to wire it. Logging makes the decision observable during the spike instead of only once it starts mutating requests, and Authorization is captured before the origin send consumes the request. Verified: fmt, all six clippy targets, all four adapter suites, 1846 core tests. --- crates/trusted-server-core/src/publisher.rs | 267 ++++++++++++++++++ .../src/response_privacy.rs | 37 ++- 2 files changed, 289 insertions(+), 15 deletions(-) diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index 09d033d76..b708fdf05 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -2830,6 +2830,11 @@ pub async fn handle_publisher_request( } ); + // Recorded before the request is consumed by the origin send: the C2 gate + // below needs it, and an authorized response must never become a shared + // template. + let request_had_authorization = req.headers().contains_key(header::AUTHORIZATION); + if should_run_ad_stack { req.headers_mut().remove(header::IF_NONE_MATCH); req.headers_mut().remove(header::IF_MODIFIED_SINCE); @@ -2975,6 +2980,25 @@ pub async fn handle_publisher_request( .to_string(); let status = response.status(); + + // Evaluate the shared-template cache gate and log it. No behaviour change yet: + // the C2 read/write lands in Task 3 Step 4, and under the default `Inline` + // mode this reports `InlineMode` and logs nothing. Wiring it now gives the + // gate a real call site and makes the decision observable during the spike + // rather than only at the point it starts mutating requests. + if !matches!(assembly_mode, AssemblyMode::Inline) { + match c2_bypass_reason( + assembly_mode, + request_had_authorization, + status, + &content_type, + response.headers(), + ) { + Some(reason) => log::debug!("c2_template_cache bypass: {reason}"), + None => log::debug!("c2_template_cache eligible"), + } + } + let content_encoding = response .headers() .get(header::CONTENT_ENCODING) @@ -3560,6 +3584,88 @@ fn match_renderable_slots( /// /// Property names match what the client-side TSJS bundle expects: /// `gam_unit_path`, `div_id`, `formats`, and `targeting`. +/// Why a response must not enter the shared transformed-template cache (C2). +/// +/// `cache::core` is not an HTTP cache: it stores whatever bytes it is handed and +/// rejects nothing on its own. Every safety condition is the caller's to enforce, +/// so they are enumerated here rather than left implicit. +/// +/// Spike-only, for the #1009 ESI validation. +#[derive(Debug, Clone, Copy, PartialEq, Eq, derive_more::Display)] +pub(crate) enum C2BypassReason { + /// Not a shared-template mode; there is no C2 object to write. + #[display("assembly mode is inline")] + InlineMode, + /// The origin set a cookie. Caching this would replay one visitor's cookie + /// to the next — and the cookie-privacy net downgrades *our* response, which + /// happens after the cache has already stored the origin's. + #[display("origin response carries Set-Cookie")] + OriginSetCookie, + /// The origin declared the response non-shareable. + #[display("origin marked the response private, no-store or no-cache")] + OriginNotShareable, + /// The request was authenticated. #1009 describes a Basic-Auth-gated + /// deployment, so an authorized response entering a shared cache is a live + /// concern rather than a hypothetical one. + #[display("request carried Authorization")] + AuthorizedRequest, + /// Not a 200. This is also what covers a `DataDome` block, which replaces the + /// document with a `403` (`integrations/datadome/protection.rs:778`). + #[display("status was not 200 OK")] + NonOkStatus, + /// Not HTML, so there is no template to transform. + #[display("content type is not text/html")] + NotHtml, +} + +/// Whether a response may be written to the shared transformed-template cache. +/// +/// Returns [`None`] when it is safe to cache, or the first disqualifying reason. +/// Leak vectors are checked before mere ineligibility so the reported reason is +/// the most serious one that applies. +/// +/// See `docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md` +/// §6.6 for why C1, C2 and a final assembled-response cache are distinct, and why +/// the third must never exist. +pub(crate) fn c2_bypass_reason( + mode: AssemblyMode, + request_had_authorization: bool, + status: StatusCode, + content_type: &str, + response_headers: &edgezero_core::http::HeaderMap, +) -> Option { + if matches!(mode, AssemblyMode::Inline) { + return Some(C2BypassReason::InlineMode); + } + if request_had_authorization { + return Some(C2BypassReason::AuthorizedRequest); + } + if response_headers.contains_key(header::SET_COOKIE) { + return Some(C2BypassReason::OriginSetCookie); + } + // Reuse the cookie-privacy net's predicate rather than a third copy of it. + // That covers `private` and `no-store`; `no-cache` needs its own check + // because it means "revalidate before reuse", not "do not store" — a + // distinction that is correct for HTTP caches but too permissive for a + // spike-owned template cache, so treat it as disqualifying here. + let cache_control = response_headers + .get(header::CACHE_CONTROL) + .and_then(|value| value.to_str().ok()) + .map(str::to_ascii_lowercase); + if crate::response_privacy::is_uncacheable_by_cache_control(response_headers) + || cache_control.is_some_and(|value| value.contains("no-cache")) + { + return Some(C2BypassReason::OriginNotShareable); + } + if status != StatusCode::OK { + return Some(C2BypassReason::NonOkStatus); + } + if !is_html_content_type(content_type) { + return Some(C2BypassReason::NotHtml); + } + None +} + /// What the `` seam injects, given the assembly mode. /// /// Under [`AssemblyMode::Inline`] the response is per-navigation and not shared, @@ -4582,6 +4688,167 @@ mod tests { .expect("should proxy publisher request") } + mod c2_gate_tests { + //! `cache::core` stores whatever it is handed and rejects nothing, so every + //! one of these conditions is the caller's to enforce. Each is a leak vector + //! or an eligibility rule, not a preference. + + use super::*; + use crate::creative_opportunities::AssemblyMode; + use edgezero_core::http::HeaderName; + + fn headers(pairs: &[(HeaderName, &str)]) -> edgezero_core::http::HeaderMap { + let mut map = edgezero_core::http::HeaderMap::new(); + for (name, value) in pairs { + map.insert( + name.clone(), + HeaderValue::from_str(value).expect("should build header value"), + ); + } + map + } + + fn shareable() -> edgezero_core::http::HeaderMap { + headers(&[(header::CACHE_CONTROL, "max-age=60")]) + } + + #[test] + fn a_plain_shareable_html_200_is_cacheable() { + for mode in [AssemblyMode::ClientFill, AssemblyMode::Esi] { + assert_eq!( + c2_bypass_reason(mode, false, StatusCode::OK, "text/html", &shareable()), + None, + "{mode:?}: a shareable HTML 200 should be eligible" + ); + } + } + + #[test] + fn inline_mode_never_writes_a_template() { + assert_eq!( + c2_bypass_reason( + AssemblyMode::Inline, + false, + StatusCode::OK, + "text/html", + &shareable() + ), + Some(C2BypassReason::InlineMode), + "inline has no shared template to write" + ); + } + + #[test] + fn an_authorized_request_is_never_cached() { + assert_eq!( + c2_bypass_reason( + AssemblyMode::Esi, + true, + StatusCode::OK, + "text/html", + &shareable() + ), + Some(C2BypassReason::AuthorizedRequest), + "an authenticated response must not enter a shared cache" + ); + } + + #[test] + fn an_origin_set_cookie_is_never_cached() { + let with_cookie = headers(&[ + (header::CACHE_CONTROL, "max-age=60"), + (header::SET_COOKIE, "sid=abc; Path=/"), + ]); + assert_eq!( + c2_bypass_reason( + AssemblyMode::Esi, + false, + StatusCode::OK, + "text/html", + &with_cookie + ), + Some(C2BypassReason::OriginSetCookie), + "caching this would replay one visitor's cookie to the next" + ); + } + + #[test] + fn non_shareable_cache_control_is_refused_case_insensitively() { + for directive in [ + "private", + "no-store", + "no-cache", + "Private, max-age=60", + "NO-STORE", + "public, No-Cache", + ] { + let map = headers(&[(header::CACHE_CONTROL, directive)]); + assert_eq!( + c2_bypass_reason(AssemblyMode::Esi, false, StatusCode::OK, "text/html", &map), + Some(C2BypassReason::OriginNotShareable), + "`{directive}` should disqualify the response" + ); + } + } + + #[test] + fn a_datadome_block_is_refused_by_the_status_check() { + // DataDome replaces the document with a 403 + // (`integrations/datadome/protection.rs:778`). There is no separate + // marker to detect, and none is needed. + assert_eq!( + c2_bypass_reason( + AssemblyMode::Esi, + false, + StatusCode::FORBIDDEN, + "text/html", + &shareable() + ), + Some(C2BypassReason::NonOkStatus), + "a blocked document must not become the shared template" + ); + } + + #[test] + fn non_html_is_refused() { + for content_type in ["text/x-component", "application/json", ""] { + assert_eq!( + c2_bypass_reason( + AssemblyMode::Esi, + false, + StatusCode::OK, + content_type, + &shareable() + ), + Some(C2BypassReason::NotHtml), + "`{content_type}` has no HTML template to transform" + ); + } + } + + #[test] + fn leak_vectors_are_reported_before_mere_ineligibility() { + // A response that fails several conditions should name the most serious + // one, so an operator reading the log sees the security reason rather + // than a content-type quibble. + let map = headers(&[ + (header::CACHE_CONTROL, "private"), + (header::SET_COOKIE, "sid=abc"), + ]); + assert_eq!( + c2_bypass_reason( + AssemblyMode::Esi, + true, + StatusCode::FORBIDDEN, + "application/json", + &map + ), + Some(C2BypassReason::AuthorizedRequest), + "authorization is the most serious disqualifier and should win" + ); + } + } + mod template_neutrality_tests { //! The gate for #1009's shared-template design. //! diff --git a/crates/trusted-server-core/src/response_privacy.rs b/crates/trusted-server-core/src/response_privacy.rs index e23348211..2d5fb31b4 100644 --- a/crates/trusted-server-core/src/response_privacy.rs +++ b/crates/trusted-server-core/src/response_privacy.rs @@ -9,7 +9,7 @@ //! cache such as Cloudflare would otherwise serve an operator/origin //! `Cache-Control: public` on a cookie-bearing response as-is. -use edgezero_core::http::{HeaderName, HeaderValue, Response, header}; +use edgezero_core::http::{HeaderMap, HeaderName, HeaderValue, Response, header}; use crate::settings::Settings; @@ -24,6 +24,25 @@ pub const CDN_CACHE_HEADERS: &[&str] = &[ "cloudflare-cdn-cache-control", ]; +/// Whether `Cache-Control` already forbids shared caching. +/// +/// Extracted because this predicate is needed in three places now: both arms of +/// the cookie-privacy net below, and the shared-template cache gate in +/// `publisher::c2_bypass_reason`. +/// +/// Directives are case-insensitive (RFC 9111 §5.2), so `No-Store` and `Private` +/// count. `no-cache` deliberately does **not**: it requires revalidation before +/// reuse, not a refusal to store, so a `no-cache` response is still shareable. +/// Callers needing the stricter reading must check it themselves. +#[must_use] +pub(crate) fn is_uncacheable_by_cache_control(headers: &HeaderMap) -> bool { + headers + .get(header::CACHE_CONTROL) + .and_then(|v| v.to_str().ok()) + .map(str::to_ascii_lowercase) + .is_some_and(|v| v.contains("private") || v.contains("no-store")) +} + /// Forces cookie-bearing responses to stay private to shared caches. /// /// Any response that sets a per-user cookie (notably the EC identity cookie) @@ -44,14 +63,7 @@ pub fn enforce_set_cookie_cache_privacy(response: &mut Response) { for name in CDN_CACHE_HEADERS { response.headers_mut().remove(*name); } - // Cache-Control directives are case-insensitive (RFC 9111 §5.2), so match - // against a lowercased copy — `No-Store` / `Private` must count. - let already_uncacheable = response - .headers() - .get(header::CACHE_CONTROL) - .and_then(|v| v.to_str().ok()) - .map(str::to_ascii_lowercase) - .is_some_and(|v| v.contains("private") || v.contains("no-store")); + let already_uncacheable = is_uncacheable_by_cache_control(response.headers()); if !already_uncacheable { response.headers_mut().insert( header::CACHE_CONTROL, @@ -77,12 +89,7 @@ pub fn enforce_set_cookie_cache_privacy(response: &mut Response) { pub fn apply_response_headers_with_cache_privacy(settings: &Settings, response: &mut Response) { enforce_set_cookie_cache_privacy(response); - let response_is_uncacheable = response - .headers() - .get(header::CACHE_CONTROL) - .and_then(|v| v.to_str().ok()) - .map(str::to_ascii_lowercase) - .is_some_and(|v| v.contains("private") || v.contains("no-store")); + let response_is_uncacheable = is_uncacheable_by_cache_control(response.headers()); for (key, value) in &settings.response_headers { if response_is_uncacheable From d9e059735f4ba7dc058b0496317959d8504f576a Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 20:23:11 +0530 Subject: [PATCH 17/44] Decouple the body-close decision from the head script MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes a defect the previous commit introduced. Gating the head seam on template neutrality made ad_slots_script None under the shared modes — and the body-close element handler read exactly that value to decide whether to inject at all. So shared modes silently stopped injecting anything at as a side effect of a change to . Safe, since emitting nothing cannot leak, but wrong for the reason the spec warns about: the gate has to be "did this response carry bids", not "does this page have slots". BodyCloseInjection replaces the inference with a named decision — None, InlineBids, or Marker — chosen by body_close_injection() at a site that knows the assembly mode. No new struct field was needed: settings is already threaded to all three processor-construction sites, so the mode is derivable there. Behaviour is unchanged. Inline still injects when slots matched and stays quiet when they did not. Esi deliberately returns None rather than a placeholder marker. The marker has to point at a fragment endpoint returning an executable script; /_ts/page-bids returns JSON and ESI splices fragment bytes verbatim, so aiming at it would put raw JSON where a script belongs. That endpoint does not exist yet, and a marker with nothing behind it is worse than no marker. A test pins the current answer so it changes deliberately rather than silently. The most useful test asserts body-close is identical whether or not the head script is present, under both shared modes. A decision that read the head script would be accidentally correct there today — because the head script is always absent under those modes — and wrong the moment that changes. Seven config literals in tests plus one in a benchmark now state their intent explicitly instead of relying on the old inference, which is the improvement rather than a cost. clippy --all-targets caught the benchmark; test runs alone did not. Verified: fmt, all six clippy targets, all four adapter suites, 1850 core tests. --- .../benches/html_processor_bench.rs | 8 +- .../trusted-server-core/src/html_processor.rs | 74 +++++++++-- crates/trusted-server-core/src/publisher.rs | 115 +++++++++++++++++- 3 files changed, 186 insertions(+), 11 deletions(-) diff --git a/crates/trusted-server-core/benches/html_processor_bench.rs b/crates/trusted-server-core/benches/html_processor_bench.rs index 96eec2f1f..e13b9fdb3 100644 --- a/crates/trusted-server-core/benches/html_processor_bench.rs +++ b/crates/trusted-server-core/benches/html_processor_bench.rs @@ -1,5 +1,7 @@ use criterion::{BenchmarkId, Criterion, black_box, criterion_group, criterion_main}; -use trusted_server_core::html_processor::{HtmlProcessorConfig, create_html_processor}; +use trusted_server_core::html_processor::{ + BodyCloseInjection, HtmlProcessorConfig, create_html_processor, +}; use trusted_server_core::integrations::IntegrationRegistry; use trusted_server_core::streaming_processor::StreamProcessor as _; @@ -13,6 +15,10 @@ fn make_config() -> HtmlProcessorConfig { ad_bids_state: std::sync::Arc::new(std::sync::Mutex::new(None)), max_buffered_body_bytes: 16 * 1024 * 1024, gpt_diagnostics: None, + // The benchmark measures URL rewriting, not ad injection, and + // `ad_slots_script` is `None` here — matching the previous behaviour, + // which inferred no body-close work from that. + body_close: BodyCloseInjection::None, } } diff --git a/crates/trusted-server-core/src/html_processor.rs b/crates/trusted-server-core/src/html_processor.rs index 889234b56..f0d21a7ad 100644 --- a/crates/trusted-server-core/src/html_processor.rs +++ b/crates/trusted-server-core/src/html_processor.rs @@ -155,6 +155,30 @@ impl StreamProcessor for HtmlWithPostProcessing { fn reset(&mut self) {} } +/// What the `` seam injects. +/// +/// This is a decision, not a side effect of whether the `` script exists. +/// An earlier shape gated body-close injection on `ad_slots_script.is_some()`, +/// which coupled two independent choices: once a shared-template mode stopped +/// emitting the head script, body-close injection silently stopped too. +/// +/// See `docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md` +/// §6.7. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub enum BodyCloseInjection { + /// Emit nothing. Either no slots matched under the inline path, or a + /// client-fill mode where the browser fetches the fragment unprompted. + #[default] + None, + /// Read the auction result from `ad_bids_state` and inject it, falling back to + /// an empty payload. Today's shipped behaviour. + InlineBids, + /// Emit this markup verbatim — an `` for the edge to assemble. + /// Must be identical for every request that reaches the transform, or the + /// cached template is not shared-safe. + Marker(String), +} + /// Configuration for HTML processing #[derive(Clone)] pub struct HtmlProcessorConfig { @@ -175,6 +199,9 @@ pub struct HtmlProcessorConfig { pub max_buffered_body_bytes: usize, /// Request-scoped conditional diagnostics delivery decision. pub gpt_diagnostics: Option, + /// What the `` seam injects. Decided by the caller rather than inferred + /// from [`Self::ad_slots_script`]. + pub body_close: BodyCloseInjection, } impl HtmlProcessorConfig { @@ -196,6 +223,7 @@ impl HtmlProcessorConfig { ad_bids_state: std::sync::Arc::new(std::sync::Mutex::new(None)), max_buffered_body_bytes: settings.publisher.max_buffered_body_bytes, gpt_diagnostics: None, + body_close: BodyCloseInjection::None, } } @@ -217,6 +245,17 @@ impl HtmlProcessorConfig { self } + /// Set what the `` seam injects. + /// + /// Separate from [`with_ad_state`](Self::with_ad_state) because the two are + /// independent decisions: a shared-template mode emits no head script and + /// still needs a body-close marker. + #[must_use] + pub fn with_body_close(mut self, body_close: BodyCloseInjection) -> Self { + self.body_close = body_close; + self + } + /// Attach the request-scoped conditional diagnostics decision. #[must_use] pub fn with_gpt_diagnostics(mut self, decision: Option) -> Self { @@ -304,6 +343,7 @@ pub fn create_html_processor(config: HtmlProcessorConfig) -> impl StreamProcesso let integration_registry = config.integrations.clone(); let script_rewriters = integration_registry.script_rewriters(); let ad_slots_script = config.ad_slots_script.clone(); + let body_close = config.body_close.clone(); let ad_bids_state = config.ad_bids_state.clone(); let gpt_diagnostics = config.gpt_diagnostics.clone(); @@ -371,25 +411,38 @@ pub fn create_html_processor(config: HtmlProcessorConfig) -> impl StreamProcesso element!("body", { let state = ad_bids_state.clone(); let injected_bids = injected_bids.clone(); - let has_slots = ad_slots_script.is_some(); + let body_close = body_close.clone(); move |el| { - if !has_slots { + if matches!(body_close, BodyCloseInjection::None) { return Ok(()); } let state = state.clone(); let injected_bids = injected_bids.clone(); + let body_close = body_close.clone(); if let Some(handlers) = el.end_tag_handlers() { let handler: EndTagHandler<'static> = Box::new(move |end_tag: &mut EndTag<'_>| { if injected_bids.swap(true, Ordering::SeqCst) { return Ok(()); } - let script_guard = state.lock().expect("should lock bid state"); - let bids_script = match &*script_guard { - Some(s) => s.clone(), - None => build_empty_bids_script(), + let markup = match &body_close { + // Verbatim, and identical on every request that + // reaches the transform — that is what makes the + // cached template shared-safe. + BodyCloseInjection::Marker(marker) => marker.clone(), + BodyCloseInjection::InlineBids => { + let script_guard = state.lock().expect("should lock bid state"); + match &*script_guard { + Some(s) => s.clone(), + None => build_empty_bids_script(), + } + } + // Unreachable: the element handler returned early + // above. Kept exhaustive rather than using `_` so a + // new variant is a compile error here. + BodyCloseInjection::None => return Ok(()), }; - end_tag.before(&bids_script, ContentType::Html); + end_tag.before(&markup, ContentType::Html); Ok(()) }); handlers.push(handler); @@ -684,6 +737,7 @@ mod tests { fn create_test_config() -> HtmlProcessorConfig { HtmlProcessorConfig { + body_close: BodyCloseInjection::None, origin_host: "origin.example.com".to_owned(), request_host: "test.example.com".to_owned(), request_scheme: "https".to_owned(), @@ -1528,6 +1582,7 @@ mod tests { #[test] fn injects_ad_slots_at_head_open() { let config = HtmlProcessorConfig { + body_close: BodyCloseInjection::None, origin_host: "origin.example.com".to_string(), request_host: "example.com".to_string(), request_scheme: "https".to_string(), @@ -1603,6 +1658,7 @@ mod tests { let bids_script = r#""#; let state = std::sync::Arc::new(std::sync::Mutex::new(Some(bids_script.to_string()))); let config = HtmlProcessorConfig { + body_close: BodyCloseInjection::InlineBids, origin_host: "origin.example.com".to_string(), request_host: "example.com".to_string(), request_scheme: "https".to_string(), @@ -1639,6 +1695,7 @@ mod tests { let bids_script = r#""#; let state = std::sync::Arc::new(std::sync::Mutex::new(Some(bids_script.to_string()))); let config = HtmlProcessorConfig { + body_close: BodyCloseInjection::InlineBids, origin_host: "origin.example.com".to_string(), request_host: "example.com".to_string(), request_scheme: "https".to_string(), @@ -1676,6 +1733,7 @@ mod tests { let request_host = "proxy.test-publisher.example.com"; let config = HtmlProcessorConfig { + body_close: BodyCloseInjection::None, origin_host: "origin.test-publisher.example.com".to_string(), request_host: request_host.to_string(), request_scheme: "https".to_string(), @@ -1727,6 +1785,7 @@ mod tests { // (state is None) — e.g. auction timed out with zero bids. Fallback to {}. let state = std::sync::Arc::new(std::sync::Mutex::new(None)); let config = HtmlProcessorConfig { + body_close: BodyCloseInjection::InlineBids, origin_host: "origin.example.com".to_string(), request_host: "example.com".to_string(), request_scheme: "https".to_string(), @@ -1756,6 +1815,7 @@ mod tests { // unmodified (spec §8: "Existing client-side Prebid/GPT flow runs unmodified"). let state = std::sync::Arc::new(std::sync::Mutex::new(None)); let config = HtmlProcessorConfig { + body_close: BodyCloseInjection::None, origin_host: "origin.example.com".to_string(), request_host: "example.com".to_string(), request_scheme: "https".to_string(), diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index b708fdf05..e789f5dce 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -55,6 +55,7 @@ use crate::ec::EcContext; use crate::ec::kv::KvIdentityGraph; use crate::ec::registry::PartnerRegistry; use crate::error::TrustedServerError; +use crate::html_processor::BodyCloseInjection; use crate::http_util::{RequestInfo, is_navigation_request, serve_static_with_etag}; use crate::integrations::IntegrationRegistry; use crate::platform::{GeoInfo, PlatformBackendSpec, PlatformHttpRequest, RuntimeServices}; @@ -948,6 +949,39 @@ struct HtmlStreamProcessorParams<'a> { gpt_diagnostics: Option, } +/// What the `` seam should inject, given the assembly mode. +/// +/// Explicit rather than inferred. The previous shape read +/// `ad_slots_script.is_some()` inside the element handler, which silently coupled +/// two independent decisions: once [`template_ad_slots_script`] stopped emitting a +/// head script under a shared mode, body-close injection stopped with it. +/// +/// `Esi` returns [`BodyCloseInjection::None`] for now rather than a placeholder +/// marker. The marker must point at a dedicated fragment endpoint returning an +/// executable script — `/_ts/page-bids` returns JSON, and ESI splices fragment +/// bytes verbatim, so aiming at it would put raw JSON where a script belongs. +/// That endpoint does not exist yet, and emitting a marker with nothing behind it +/// would be worse than emitting nothing. +pub(crate) fn body_close_injection( + mode: AssemblyMode, + head_script_present: bool, +) -> BodyCloseInjection { + match mode { + // Per-navigation and never shared, so gating on slot presence is correct. + AssemblyMode::Inline => { + if head_script_present { + BodyCloseInjection::InlineBids + } else { + BodyCloseInjection::None + } + } + // The browser fetches the fragment unprompted; nothing to emit. + AssemblyMode::ClientFill => BodyCloseInjection::None, + // Pending the fragment endpoint. See the note above. + AssemblyMode::Esi => BodyCloseInjection::None, + } +} + fn create_html_stream_processor( params: HtmlStreamProcessorParams<'_>, ) -> Result, Report> { @@ -959,9 +993,20 @@ fn create_html_stream_processor( params.origin_host, params.request_host, params.request_scheme, - ) - .with_ad_state(params.ad_slots_script, params.ad_bids_state) - .with_gpt_diagnostics(params.gpt_diagnostics); + ); + + let assembly_mode = params + .settings + .creative_opportunities + .as_ref() + .map(CreativeOpportunitiesConfig::assembly_mode) + .unwrap_or_default(); + let body_close = body_close_injection(assembly_mode, params.ad_slots_script.is_some()); + + let config = config + .with_ad_state(params.ad_slots_script, params.ad_bids_state) + .with_gpt_diagnostics(params.gpt_diagnostics) + .with_body_close(body_close); Ok(create_html_processor(config)) } @@ -4688,6 +4733,70 @@ mod tests { .expect("should proxy publisher request") } + mod body_close_decision_tests { + //! The `` decision must not be inferred from the `` script. + //! + //! Coupling them is a live defect, not a hypothetical: gating the head seam + //! on template neutrality made `ad_slots_script` `None` under shared modes, + //! which silently disabled body-close injection too. These tests pin the two + //! decisions apart. + + use super::*; + use crate::creative_opportunities::AssemblyMode; + + #[test] + fn inline_injects_bids_only_when_the_head_script_is_present() { + assert_eq!( + body_close_injection(AssemblyMode::Inline, true), + BodyCloseInjection::InlineBids, + "inline with matched slots should inject the auction result" + ); + assert_eq!( + body_close_injection(AssemblyMode::Inline, false), + BodyCloseInjection::None, + "inline without matched slots should leave the publisher's flow alone" + ); + } + + #[test] + fn shared_modes_do_not_depend_on_the_head_script() { + // The decision must be the same either way. Under a shared mode the head + // script is always absent, so a decision that read it would be + // accidentally correct here and wrong the moment that changes. + for mode in [AssemblyMode::ClientFill, AssemblyMode::Esi] { + assert_eq!( + body_close_injection(mode, true), + body_close_injection(mode, false), + "{mode:?}: body-close must not vary with head-script presence" + ); + } + } + + #[test] + fn client_fill_emits_nothing_because_the_browser_fetches_unprompted() { + assert_eq!( + body_close_injection(AssemblyMode::ClientFill, false), + BodyCloseInjection::None + ); + } + + #[test] + fn esi_emits_nothing_until_the_fragment_endpoint_exists() { + // Deliberately not a placeholder marker. `/_ts/page-bids` returns JSON + // and ESI splices fragment bytes verbatim, so pointing at it would put + // raw JSON where an executable script belongs. Emitting a marker with + // nothing behind it is worse than emitting nothing. + // + // This test is expected to change when that endpoint lands — it exists + // to make that a deliberate edit rather than a silent one. + assert_eq!( + body_close_injection(AssemblyMode::Esi, false), + BodyCloseInjection::None, + "Esi should emit nothing until a script-returning fragment endpoint exists" + ); + } + } + mod c2_gate_tests { //! `cache::core` stores whatever it is handed and rejects nothing, so every //! one of these conditions is the caller's to enforce. Each is a leak vector From f0ab7ac7648c7787d0b9348ad09bc59e2efe2751 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 20:51:27 +0530 Subject: [PATCH 18/44] Record Task 3 implementation progress and its limits MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Steps 1, 2, 2b and 3 are done and behaviour-neutral under the default Inline mode. Step 2c (emit the Esi marker) and Step 4 (the cache read/write) are not, and the record says why rather than leaving them looking merely unstarted: the marker needs a fragment endpoint returning an executable script, and Step 4 is blocked on a design choice the plan deliberately defers. Records the defect this work introduced and then caught. Gating the head seam on neutrality made ad_slots_script None under shared modes, and the body-close handler read that value to decide whether to inject at all — so shared modes silently stopped injecting at as a side effect of a change. Found by reading the handler while starting the next step, not by a failing test. It is the same shape as the bug the whole task exists to prevent: something that looks correct and quietly does nothing. Also records what the coverage does not cover. Fourteen tests prove tsjs.adSlots is request-neutral. They say nothing about the other things injected at the same seam — integration head_inserts, the gpt-diagnostics bootstrap, the RSC placeholder rewriter — which the spec flags for audit and which is still outstanding. Request-neutrality is asserted for one element, not established for the template, and reading the test names would suggest otherwise. And a gate note: clippy --all-targets caught a benchmark construction site that all four test suites missed. --- .../2026-08-08-1009-measurement-findings.md | 60 +++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md index 9e68acb27..b12d115a6 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md @@ -217,6 +217,66 @@ Compiling and a cache round-trip are not an implementation. Nothing yet exercise `lol_html` transform into C2, ESI assembly, or any runtime behaviour on the publisher path, and the `esi` dependency is added but unused. +## ESI spike Task 3 — implementation progress + +**Date:** 2026-08-10. All of it behaviour-neutral under the default +`AssemblyMode::Inline`; nothing here changes a shipped code path. + +| Step | State | +| -------------------------------- | ------------------------------------------------------------------- | +| 1 — `AssemblyMode` setting | **Done.** `Option` on `CreativeOpportunitiesConfig`. | +| 2 — head-seam neutrality gate | **Done.** `template_ad_slots_script`, three byte-identity tests. | +| 2b — body-close decoupling | **Done.** `BodyCloseInjection`, `body_close_injection`. | +| 2c — emit the marker under `Esi` | **Not done.** Blocked on the fragment endpoint; see below. | +| 3 — C2 eligibility gate | **Done.** `c2_bypass_reason`, eight tests. Logs only, no cache I/O. | +| 4 — C2 cache read/write | **Not started.** Design choice open; see below. | + +### What is deliberately absent + +**No marker is emitted under `Esi`.** The marker must point at a fragment endpoint +returning an **executable script**. `/_ts/page-bids` returns JSON +(`publisher.rs`, `handle_page_bids`) and ESI splices fragment bytes verbatim, so aiming +at it would put raw JSON where a script belongs. That endpoint does not exist, and a +marker with nothing behind it is worse than no marker. A test pins the current answer so +it changes deliberately. + +**No cache read or write.** `c2_bypass_reason` has a real call site that logs its verdict, +which makes the decision observable during the spike without mutating anything. Task 3 +Step 4 is blocked on choosing between read-through-with-body-transform and explicit +`cache::core` — the plan names that as a decision to make before writing code, and it is +under investigation rather than assumed. + +### A defect this work introduced and then caught + +Gating the head seam on neutrality made `ad_slots_script` `None` under the shared modes. +The body-close element handler read exactly that value to decide whether to inject at all, +so shared modes silently stopped injecting anything at `` — a side effect of a +`` change. Safe, since emitting nothing cannot leak, but wrong in the way the spec +warns about: the gate has to be "did this response carry bids", not "does this page have +slots". + +Found by reading the handler while starting the next step, not by a failing test. Fixed by +replacing the inference with a named decision. The test that now guards it asserts +body-close is identical whether or not the head script is present — a decision that read +the head script would be _accidentally_ correct today, because that script is always +absent under shared modes, and wrong the moment that changes. + +Worth recording because it is the same shape as the bug the whole task exists to prevent: +something that looks correct and quietly does nothing. + +### Coverage and its limits + +Fourteen new tests. `fmt`, all six clippy targets, and all four adapter suites pass, with +1850 core tests under Viceroy. `clippy --all-targets` caught a benchmark construction site +that all four test suites missed — the suites are not the whole gate. + +**The neutrality guarantee is narrower than it looks.** The tests prove `tsjs.adSlots` is +neutral. They say nothing about the other things injected at the same seam — integration +`head_inserts`, the gpt-diagnostics bootstrap, the RSC placeholder rewriter — which the +spec flags as needing an audit and which that audit has not yet covered. Until it does, +treat request-neutrality as asserted for one element rather than established for the +template. + ## Step B — consumers of TS's own response headers Not yet run. From 672836c1a8a0498ea1d33ec7312a3d9e92ac2ce4 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 20:59:46 +0530 Subject: [PATCH 19/44] Decide the C2 cache design: cache::core, not read-through MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The plan left this open between fastly::cache::core and read-through caching with after_send plus set_body_transform. Investigated and verified against the pinned SDK and Viceroy 0.17 source. Read-through is not viable here, on three hard blockers rather than on preference. Viceroy stubs the entire HTTP Cache ABI, and the SDK converts that into a send error rather than a fallback: is_request_cacheable returns NotAvailable, which makes must_use_host_caching true, which with a send hook set returns HttpCacheApiUnsupported. Setting after_send therefore makes every publisher origin fetch fail under fastly compute serve, cargo test-fastly, and the parity suite. The whole local loop dies. with_cache_bypass makes the hook silently dead anyway. get_caching_mode checks cache_override.is_pass() first and returns host caching, so after_send is never invoked and no error is raised — on exactly the requests in scope, quietly. And the closure bounds are incompatible with this codebase. with_after_send requires Fn + Send + Sync + 'static, while everything the rewriter needs is !Send by construction, which is why the platform layer is async_trait(?Send) throughout. set_body_transform is also synchronous and so could never await the auction collect. Recorded rather than merely chosen, because read-through's appeal is real — CandidateResponse::apply_and_stream_back is execute_and_stream_back with HTTP semantics attached — and someone will otherwise propose it again. Also settled: core cannot reach it at all, since PlatformHttpRequest has no callback slot and adding one would name Fastly types in portable core. Adds the exact insertion point, the one required hoist, and four risks the investigation surfaced that are specific to this codebase: Vary is in the key list but c2_bypass_reason does not check it; store bytes plus a metadata envelope and rebuild every header on a hit rather than replaying origin headers into a path that strips them; Content-Encoding and host/scheme both belong in the key. Plus a follow-up to file rather than fix: the auction is dispatched before the lookup, so under the shared modes it is already pure waste. Tee-ing turns out to be unnecessary. With any post-processor registered — and Next.js always registers one — the transformed document arrives as one contiguous buffer, so it is two write_all calls on the same slice. Keep execute_and_stream_back for transaction correctness and request collapsing, not for memory. Corrects the findings document: Viceroy implements purge_surrogate_key against the same in-process cache, so C2's purge-based rollback is locally testable. C1's, which is what Stage 0 exposes, still is not. --- .../2026-08-08-1009-measurement-findings.md | 6 + .../2026-08-10-1009-esi-validation-spike.md | 118 ++++++++++++++++-- 2 files changed, 114 insertions(+), 10 deletions(-) diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md index b12d115a6..0b6a70f67 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md @@ -147,6 +147,12 @@ surrogate keys the _origin_ supplies on its responses, or the HTTP cache's own request/candidate surrogate-key surface. Confirm which is available before relying on it — an earlier revision of this document conflated the two. +**C2's purge is locally testable; C1's is not.** Verified 2026-08-10: Viceroy 0.17 +implements `purge_surrogate_key` against the same in-process cache it serves reads from +(`viceroy-lib-0.17.0/src/wiggle_abi/fastly_purge_impl.rs:10-32`), soft purge included. So +the purge-based rollback for the C2 template cache the spike builds can be exercised end +to end without a Fastly service. That does nothing for Stage 0, whose exposure is C1. + Rollback is therefore: flip the flag, **then** purge C1 by whichever mechanism is actually available (or roll a versioned key namespace), **then** observe past the origin TTL before declaring the incident closed. With no C1 purge path wired, the tail is the TTL itself — diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index f1f3776f5..1b19bc132 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -468,16 +468,114 @@ built from — and decide explicitly whether the stored template is compressed. Per-user signals must never appear in the key. If a signal cannot be excluded from the template, it does not belong in C2 at all. -**Design choice to make explicitly before writing code.** Two viable shapes: - -1. **Read-through with `after_send` + `set_body_transform`** — keeps HTTP semantics, - revalidation, and stale handling for free; less control over the key. -2. **`cache::core` as above** — full control; you own metadata, revalidation, and the - stale state machine. - -This plan assumes (2). If (1) is chosen, Step 4 is rewritten and the metadata envelope -disappears. Either way, the platform boundary must sit **before** the origin request, or -a C2 HIT cannot actually skip the fetch — which is the entire point. +### Design decided 2026-08-10: `cache::core`. Do not revisit read-through. + +An earlier revision left this open between `cache::core` and read-through caching with +`after_send` + `set_body_transform`. Investigated and verified against the pinned SDK and +Viceroy 0.17 source. **Read-through is not viable here** — not on preference, on three +hard blockers: + +1. **Viceroy stubs the entire HTTP Cache ABI**, and the SDK converts that into a _send + error_ rather than a fallback. `is_request_cacheable` returns + `Err(NotAvailable("HTTP Cache API primitives"))` + (`viceroy-lib-0.17.0/src/wiggle_abi/http_cache.rs:108-114`; 26 such stubs in that + file), which makes `must_use_host_caching()` true, which with a send hook set returns + `Err(SendErrorCause::HttpCacheApiUnsupported)` + (`fastly-0.12.1/src/http/request.rs:626-632`). **Setting `after_send` makes every + publisher origin fetch fail** under `fastly compute serve`, `cargo test-fastly`, and + the parity suite. The whole local loop dies. +2. **`with_cache_bypass` makes the hook silently dead.** `get_caching_mode` checks + `cache_override.is_pass()` **first** (`request.rs:612-615`) and returns host caching, so + `after_send` is never invoked and no error is raised. On exactly the requests in scope, + today, the hook would do nothing quietly. +3. **The closure bounds are incompatible with this codebase.** `with_after_send` requires + `Fn + Send + Sync + 'static` (`request.rs:545-550`). Everything the rewriter needs is + `!Send` by construction — `edgezero_core::body::Body` wraps a `LocalBoxStream` + deliberately, which is why the platform layer is `#[async_trait(?Send)]` throughout. + And `set_body_transform` is synchronous, so it could never await the auction collect. + +Read-through's appeal was real — `CandidateResponse::apply_and_stream_back` is +`execute_and_stream_back` with HTTP semantics attached, and TTL/SWR/vary/surrogate keys +derived from origin headers for free. It is simply unreachable from here. + +**Also settled: core cannot reach it at all.** `PlatformHttpRequest` +(`platform/http.rs:16-37`) is a plain data struct with no callback slot, and carrying one +would name `fastly::http::CandidateResponse` in portable core, breaking the other three +adapters. + +### Follow the existing null-object pattern + +`cache::core` fits the shape the repo already uses four times for a Fastly-only capability +behind a portable trait: `UnavailableHttpClient` (`platform/http.rs:216-243`), +`UnavailableKvStore` (`platform/kv.rs:14-17`), and the `RuntimeServices.kv_store` +field/accessor/builder (`platform/types.rs:170,222,269,330`). Add +`PlatformTemplateCache` the same way, and follow +`crates/trusted-server-adapter-fastly/src/ec_kv.rs` — 140 lines, the repo's only real +edge-storage read/write — rather than inventing a shape. + +**Return `EdgeBody`, not `Vec`.** `EdgeBody::Stream` exists, +`fastly_body_to_edge_stream` (`adapter-fastly/src/platform.rs:503`) already converts, and +`PublisherResponse::Buffered` tolerates a live stream (`publisher.rs:1019-1022`). + +### Exact insertion point + +**Immediately before `let mut platform_request = PlatformHttpRequest::new(...)`** — the +last line before `req` is consumed, and a few lines before the origin send. Everything +needed is in scope there: `settings`, `services`, the final URI and Host, `backend_name`, +`request_path`, `matched_slots`, `should_run_ad_stack`, `request_had_authorization`, +`request_host`, `request_scheme`. + +**One required move:** `assembly_mode` is currently computed _after_ the send, for the +logging call site. It depends only on `settings`, so hoist it above the insertion point. + +**Tee-ing is not needed.** With any post-processor registered — and the Next.js +integration always registers one — `HtmlWithPostProcessing` emits nothing until the final +chunk and then returns the whole transformed document as one contiguous buffer +(`html_processor.rs:92-97,148`). Two `write_all` calls on the same slice; no tee +abstraction, no extra copy. Still use `execute_and_stream_back`, but for transaction +correctness and request collapsing rather than for memory. On a hit the processor is never +built at all. + +- [ ] **Step 4b: close the risks the design investigation surfaced** + +Four, all specific to this codebase rather than to `cache::core` in general. + +**`Vary` is in the key list but nothing consumes it.** `c2_bypass_reason` checks +`Set-Cookie`, `Cache-Control`, `Authorization`, status and content type — **not `Vary`**. +Viceroy supports `WriteOptions.vary_rule`, so the mechanism exists; the gate has to use +it. Until then the key is missing a signal the origin explicitly declares, and Step A's +verdict is a `PROVISIONAL PASS`, not a release gate. + +**Store bytes plus a metadata envelope; rebuild every header on a hit.** The publisher +path forces `private, no-store` and strips `ETag`/`Last-Modified`/CDN headers _after_ the +send. Replaying stored origin headers would fight that. Store only the transformed body +and a small `user_metadata` envelope — content encoding, content type, schema version, +tsjs hash — and construct every response header from scratch on a hit. Then no origin +header is ever replayed and the `Set-Cookie` privacy net is trivially safe. +`get_user_metadata` is implemented in Viceroy. + +**Content-Encoding belongs in the key.** The streaming pipeline pairs input encoding to +the same output encoding, so the transformed bytes inherit whatever the origin negotiated +from the client's `Accept-Encoding` — still gzip, deflate, br or identity after +`restrict_accept_encoding` narrows it. Either key on the negotiated encoding or normalize +to identity in the cache and re-encode on read. Getting this wrong serves brotli bytes to +a client that asked for gzip. + +**Host and scheme belong in the key.** The post-processed output is host-dependent by +construction: `request_host` and `request_scheme` reach `IntegrationHtmlContext`. + +- [ ] **Step 4c: file the wasted-dispatch follow-up** + +The auction is dispatched _before_ the insertion point. Under `Esi` and `ClientFill` the +root injects nothing, so that dispatch is already pure waste on this branch — and on a C2 +hit it is waste that must be cleaned up via `emit_abandoned_auction` or it leaks +telemetry. + +Keeping the lookup at the insertion point above is right for the spike: minimal diff, and +lookup latency overlaps the in-flight auction. Moving it earlier would eliminate the +wasted dispatch but serialize the lookup ahead of dispatch. **File it; do not fix it +here.** Suppressing root-level dispatch under the shared modes is Task 4's job, where it +also has to be reconciled with the exactly-one-auction gate. - [ ] **Step 5: Unit tests, then the target suite** From fee46220a0165d980d3583b6f2ca056402701d81 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Mon, 10 Aug 2026 21:14:46 +0530 Subject: [PATCH 20/44] Guard the silent-waste and cookie gaps a review found MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three findings from a code review of the four preceding commits. Two are fixed here; the third waits on an audit that is still running. The auction dispatch was never gated on AssemblyMode. assembly_mode was computed after the dispatch decision, so flipping to client_fill or esi today would still send real SSP bid requests, hold the response for the full auction budget, and then discard the result — because both injection seams now return None — with no error, no warning and no log. That is precisely the silent-waste signature §5 of the design doc is about, reached by an incomplete feature flag rather than by removing the hold. assembly_mode is hoisted above the dispatch, which the C2 design investigation wanted anyway, and root_auction_is_useful gates it. The interesting test there does not assert per-variant. It derives the invariant: a root auction is useful exactly when a seam will consume its result. A new mode cannot make the dispatch gate and the injection decisions disagree without failing it. c2_bypass_reason omitted the forwarded client Cookie, which the design doc's own §4 names as a leak vector and the plan's checklist also missed. TS forwards client cookies to origin unchanged with no strip on the publisher path, so a response can be cookie-personalized while carrying no Set-Cookie itself, having no Cache-Control at all, and being a 200 HTML — every other condition reports it cacheable. Now disqualifying until an origin Vary covering Cookie is verified. The test uses exactly that shape rather than a response that would fail some other condition anyway. Also folds the duplicated Cache-Control lookup into one pass. The previous version built a lowercased copy and then called is_uncacheable_by_cache_control, which re-fetched and re-lowercased the same header. Not fixed here: the head seam still injects integration head_inserts and the gpt-diagnostics bootstrap unconditionally, so request-neutrality is asserted for adSlots only. It happens not to leak today because gpt_diagnostics::finalize_response stamps private/no-store before the C2 gate reads headers — a load-bearing coincidence that is undocumented and untested. A neutrality audit covering that seam is still in flight; fixing it on partial information would mean doing it twice. Verified: fmt, all six clippy targets, all four adapter suites, 1853 core tests. --- crates/trusted-server-core/src/publisher.rs | 165 ++++++++++++++++++-- 1 file changed, 148 insertions(+), 17 deletions(-) diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index e789f5dce..b8a035e2c 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -949,6 +949,24 @@ struct HtmlStreamProcessorParams<'a> { gpt_diagnostics: Option, } +/// Whether a root-level auction has any consumer under this assembly mode. +/// +/// Only [`AssemblyMode::Inline`] injects the auction result into the root document. +/// Under the shared-template modes both seams emit nothing, so a dispatched root +/// auction would bill the SSPs, hold the response for the full budget, and have its +/// result discarded with no error and no log. +/// +/// This is the guard for the failure mode described in §5 of +/// `docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md`, +/// reached here by an incomplete feature flag rather than by removing the hold. +pub(crate) fn root_auction_is_useful(mode: AssemblyMode) -> bool { + match mode { + AssemblyMode::Inline => true, + // The fragment path runs its own auction; see the spike plan's Task 4. + AssemblyMode::ClientFill | AssemblyMode::Esi => false, + } +} + /// What the `` seam should inject, given the assembly mode. /// /// Explicit rather than inferred. The previous shape read @@ -2742,9 +2760,27 @@ pub async fn handle_publisher_request( // dispatch_auction returns — DispatchedAuction holds no lifetime — so req // can be mutated and sent to origin immediately after. let mut auction_observation: Option = None; + let assembly_mode = settings + .creative_opportunities + .as_ref() + .map(CreativeOpportunitiesConfig::assembly_mode) + .unwrap_or_default(); + let mut auction_request_for_telemetry: Option = None; let mut dispatched_auction = if matched_slots.is_empty() { None + } else if !root_auction_is_useful(assembly_mode) { + // Shared-template modes inject nothing at the root: `template_ad_slots_script` + // and `body_close_injection` both return `None`. Dispatching here would send + // real SSP requests, hold the response for the full auction budget, and then + // discard the result with no error and no log — the silent-waste signature + // §5 of the design doc is entirely about. The fragment path runs its own + // auction; this one has no consumer. + log::debug!( + "skipping root auction dispatch: assembly mode {assembly_mode:?} injects \ + nothing at the root" + ); + None } else { // Telemetry attribution must use the same publisher identity as the // outbound bid request. On the navigation path `request_host` is the @@ -2879,6 +2915,7 @@ pub async fn handle_publisher_request( // below needs it, and an authorized response must never become a shared // template. let request_had_authorization = req.headers().contains_key(header::AUTHORIZATION); + let request_had_cookie = req.headers().contains_key(header::COOKIE); if should_run_ad_stack { req.headers_mut().remove(header::IF_NONE_MATCH); @@ -2968,11 +3005,6 @@ pub async fn handle_publisher_request( crate::integrations::gpt_diagnostics::finalize_response(&gpt_diagnostics, &mut response); - let assembly_mode = settings - .creative_opportunities - .as_ref() - .map(CreativeOpportunitiesConfig::assembly_mode) - .unwrap_or_default(); let ad_slots_script = template_ad_slots_script( assembly_mode, should_run_ad_stack, @@ -3035,6 +3067,7 @@ pub async fn handle_publisher_request( match c2_bypass_reason( assembly_mode, request_had_authorization, + request_had_cookie, status, &content_type, response.headers(), @@ -3661,6 +3694,15 @@ pub(crate) enum C2BypassReason { /// Not HTML, so there is no template to transform. #[display("content type is not text/html")] NotHtml, + /// The request carried a `Cookie`, which TS forwards to origin unchanged — there + /// is no `Cookie` strip on the publisher path. Cookie-personalized HTML is + /// therefore cross-servable unless the origin declares `Vary: Cookie` or marks + /// those responses private, and a response can be personalized without carrying + /// `Set-Cookie` itself when the session was established earlier. Named in §4 of + /// the design doc; disqualifying until the origin's `Vary` is verified to cover + /// it. + #[display("request carried Cookie and the origin's Vary does not cover it")] + CookieForwarded, } /// Whether a response may be written to the shared transformed-template cache. @@ -3675,6 +3717,7 @@ pub(crate) enum C2BypassReason { pub(crate) fn c2_bypass_reason( mode: AssemblyMode, request_had_authorization: bool, + request_had_cookie: bool, status: StatusCode, content_type: &str, response_headers: &edgezero_core::http::HeaderMap, @@ -3685,21 +3728,24 @@ pub(crate) fn c2_bypass_reason( if request_had_authorization { return Some(C2BypassReason::AuthorizedRequest); } + if request_had_cookie { + return Some(C2BypassReason::CookieForwarded); + } if response_headers.contains_key(header::SET_COOKIE) { return Some(C2BypassReason::OriginSetCookie); } - // Reuse the cookie-privacy net's predicate rather than a third copy of it. - // That covers `private` and `no-store`; `no-cache` needs its own check - // because it means "revalidate before reuse", not "do not store" — a - // distinction that is correct for HTTP caches but too permissive for a - // spike-owned template cache, so treat it as disqualifying here. - let cache_control = response_headers + // One pass over the header. `private` and `no-store` match the cookie-privacy + // net's reading; `no-cache` is added because it means "revalidate before reuse" + // rather than "do not store" — correct for an HTTP cache, too permissive for a + // spike-owned one. + let non_shareable = response_headers .get(header::CACHE_CONTROL) .and_then(|value| value.to_str().ok()) - .map(str::to_ascii_lowercase); - if crate::response_privacy::is_uncacheable_by_cache_control(response_headers) - || cache_control.is_some_and(|value| value.contains("no-cache")) - { + .map(str::to_ascii_lowercase) + .is_some_and(|value| { + value.contains("private") || value.contains("no-store") || value.contains("no-cache") + }); + if non_shareable { return Some(C2BypassReason::OriginNotShareable); } if status != StatusCode::OK { @@ -4733,6 +4779,50 @@ mod tests { .expect("should proxy publisher request") } + mod root_auction_gate_tests { + //! Guards the silent-waste failure mode: dispatching an auction whose result + //! nothing will consume. Under the shared modes both injection seams emit + //! nothing, so a dispatched root auction bills the SSPs, holds the response + //! for the full budget, and discards the result with no error and no log. + + use super::*; + use crate::creative_opportunities::AssemblyMode; + + #[test] + fn only_inline_has_a_consumer_for_a_root_auction() { + assert!( + root_auction_is_useful(AssemblyMode::Inline), + "inline injects the auction result at ``" + ); + for mode in [AssemblyMode::ClientFill, AssemblyMode::Esi] { + assert!( + !root_auction_is_useful(mode), + "{mode:?}: neither seam injects, so a root auction has no consumer" + ); + } + } + + #[test] + fn the_gate_agrees_with_the_injection_decisions() { + // The real invariant: a root auction is useful exactly when something + // will read it. Deriving that from the two seam decisions rather than + // asserting it per-variant means a new mode cannot make these disagree. + for mode in [ + AssemblyMode::Inline, + AssemblyMode::ClientFill, + AssemblyMode::Esi, + ] { + let something_consumes_it = + body_close_injection(mode, true) != BodyCloseInjection::None; + assert_eq!( + root_auction_is_useful(mode), + something_consumes_it, + "{mode:?}: dispatch usefulness must track whether a seam consumes the result" + ); + } + } + } + mod body_close_decision_tests { //! The `` decision must not be inferred from the `` script. //! @@ -4825,7 +4915,14 @@ mod tests { fn a_plain_shareable_html_200_is_cacheable() { for mode in [AssemblyMode::ClientFill, AssemblyMode::Esi] { assert_eq!( - c2_bypass_reason(mode, false, StatusCode::OK, "text/html", &shareable()), + c2_bypass_reason( + mode, + false, + false, + StatusCode::OK, + "text/html", + &shareable() + ), None, "{mode:?}: a shareable HTML 200 should be eligible" ); @@ -4838,6 +4935,7 @@ mod tests { c2_bypass_reason( AssemblyMode::Inline, false, + false, StatusCode::OK, "text/html", &shareable() @@ -4853,6 +4951,7 @@ mod tests { c2_bypass_reason( AssemblyMode::Esi, true, + false, StatusCode::OK, "text/html", &shareable() @@ -4862,6 +4961,27 @@ mod tests { ); } + #[test] + fn a_forwarded_request_cookie_disqualifies_even_without_set_cookie() { + // The dangerous case: session established on an earlier request, so this + // response carries no Set-Cookie, has no Cache-Control at all, is a 200, + // and is HTML — yet is personalized because TS forwarded the Cookie to + // origin unchanged. Every other condition reports it cacheable. + let no_cache_control = edgezero_core::http::HeaderMap::new(); + assert_eq!( + c2_bypass_reason( + AssemblyMode::Esi, + false, + true, + StatusCode::OK, + "text/html", + &no_cache_control + ), + Some(C2BypassReason::CookieForwarded), + "cookie-personalized HTML must not become a shared template" + ); + } + #[test] fn an_origin_set_cookie_is_never_cached() { let with_cookie = headers(&[ @@ -4872,6 +4992,7 @@ mod tests { c2_bypass_reason( AssemblyMode::Esi, false, + false, StatusCode::OK, "text/html", &with_cookie @@ -4893,7 +5014,14 @@ mod tests { ] { let map = headers(&[(header::CACHE_CONTROL, directive)]); assert_eq!( - c2_bypass_reason(AssemblyMode::Esi, false, StatusCode::OK, "text/html", &map), + c2_bypass_reason( + AssemblyMode::Esi, + false, + false, + StatusCode::OK, + "text/html", + &map + ), Some(C2BypassReason::OriginNotShareable), "`{directive}` should disqualify the response" ); @@ -4909,6 +5037,7 @@ mod tests { c2_bypass_reason( AssemblyMode::Esi, false, + false, StatusCode::FORBIDDEN, "text/html", &shareable() @@ -4925,6 +5054,7 @@ mod tests { c2_bypass_reason( AssemblyMode::Esi, false, + false, StatusCode::OK, content_type, &shareable() @@ -4948,6 +5078,7 @@ mod tests { c2_bypass_reason( AssemblyMode::Esi, true, + false, StatusCode::FORBIDDEN, "application/json", &map From fbff408e8d83125cbb9459d72147287e123bb0d2 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 09:10:20 +0530 Subject: [PATCH 21/44] Keep request-scoped diagnostics out of the shared template MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the third finding from the code review. The head seam still injected request-scoped content under the shared modes, so request-neutrality was asserted for adSlots alone. Audited the seam. Of the two remaining injectors, integration head_inserts is clean: all three implementations take the context parameter unused, so their output depends on configuration and not on the request. GPT diagnostics is not clean — it is activated by a cookie or query parameter and is documented as an immutable request-scoped decision. It does not leak today, but only by coincidence. requires_private_no_store is a strict superset of the conditions under which either script is emitted, and the resulting private/no-store stamp lands before the C2 gate reads response headers, so the gate refuses. Two independent conditions that happen to align, with nothing enforcing the relationship and no test covering it. Fixed on both sides. The processor now receives no diagnostics decision under the shared modes, so the guarantee is explicit rather than emergent. And a test enumerates every combination of the decision's three fields and asserts that anything which injects also requires the stamp — so if a future change emits a script without requiring private/no-store, it fails there rather than silently in a cached template. Keeping both is deliberate: the gate is the guarantee, the invariant test is the backstop if the gate is ever removed or bypassed. Verified: fmt, all six clippy targets, all four adapter suites, 1855 core tests. --- .../src/integrations/gpt_diagnostics.rs | 60 +++++++++++++++++++ crates/trusted-server-core/src/publisher.rs | 15 ++++- 2 files changed, 74 insertions(+), 1 deletion(-) diff --git a/crates/trusted-server-core/src/integrations/gpt_diagnostics.rs b/crates/trusted-server-core/src/integrations/gpt_diagnostics.rs index 068c43cc0..bef234d25 100644 --- a/crates/trusted-server-core/src/integrations/gpt_diagnostics.rs +++ b/crates/trusted-server-core/src/integrations/gpt_diagnostics.rs @@ -114,6 +114,66 @@ impl GptDiagnosticsRequestDecision { } } +#[cfg(test)] +mod head_seam_invariant_tests { + use super::*; + + /// Every combination of the three fields the decision carries. + fn all_decisions() -> Vec { + let mut out = Vec::new(); + for active in [false, true] { + for clean in [None, Some("/clean".to_string())] { + for cookie_action in [ + GptDiagnosticsCookieAction::None, + GptDiagnosticsCookieAction::SetSession, + GptDiagnosticsCookieAction::ClearSession, + ] { + out.push(GptDiagnosticsRequestDecision { + active, + clean_browser_path_and_query: clean.clone(), + cookie_action, + }); + } + } + } + out + } + + #[test] + fn requires_private_no_store_is_a_superset_of_injection() { + // Load-bearing relationship, not an incidental one. Whenever this decision + // injects anything into ``, the response must also be stamped + // `private, no-store` — which is what keeps request-scoped diagnostics out + // of a shared cache if the explicit assembly-mode gate in + // `create_html_stream_processor` is ever removed or bypassed. + // + // If a future change makes a script emit without also requiring the stamp, + // this fails here rather than silently in a cached template. + for decision in all_decisions() { + let injects = + decision.bootstrap_script().is_some() || decision.module_script_tag().is_some(); + if injects { + assert!( + decision.requires_private_no_store(), + "decision injects into but does not require private/no-store: \ + {decision:?}" + ); + } + } + } + + #[test] + fn a_default_decision_injects_nothing() { + let decision = GptDiagnosticsRequestDecision::default(); + assert_eq!(decision.bootstrap_script(), None); + assert_eq!(decision.module_script_tag(), None); + assert!( + !decision.requires_private_no_store(), + "an inert decision should not force the response private" + ); + } +} + #[derive(Clone, Copy, Debug, PartialEq, Eq)] enum QueryDirective { Absent, diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index b8a035e2c..8fbe70f05 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -1021,9 +1021,22 @@ fn create_html_stream_processor( .unwrap_or_default(); let body_close = body_close_injection(assembly_mode, params.ad_slots_script.is_some()); + // Diagnostics is request-scoped — activated by a cookie or query parameter — so + // it must not reach a shared template. It does not leak today, but only by + // coincidence: `requires_private_no_store()` is a strict superset of the + // conditions under which a script is emitted, and the resulting `private, + // no-store` stamp lands before the C2 gate reads response headers, so the gate + // refuses. That is two independent conditions happening to align. Gate it here + // instead, so the guarantee does not depend on a relationship nothing enforces. + // `gpt_diagnostics_superset_of_injection` locks the coincidence as a backstop. + let gpt_diagnostics = match assembly_mode { + AssemblyMode::Inline => params.gpt_diagnostics, + AssemblyMode::ClientFill | AssemblyMode::Esi => None, + }; + let config = config .with_ad_state(params.ad_slots_script, params.ad_bids_state) - .with_gpt_diagnostics(params.gpt_diagnostics) + .with_gpt_diagnostics(gpt_diagnostics) .with_body_close(body_close); Ok(create_html_processor(config)) From fe88b772a6d99d90a6720726ad15bb58a70dbe47 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 09:11:51 +0530 Subject: [PATCH 22/44] Record the Task 3 code review and adopt its gate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three HIGH findings, all closed in the preceding two commits. Recorded with the reasoning rather than as a list, because two of them were holes in the plan's own checklist and not merely in the implementation. The cookie gap is the clearest case: the implementation matched Task 3 Step 3's checklist exactly and still had the hole, because the checklist itself omitted the forwarded client Cookie that §4 of the design doc names. Also records what the review says about the tests. All three findings were in code the existing tests covered and passed, because those tests exercise the pure decision functions with hand-built inputs and never the rendered head or body-close bytes. That is still true — no test renders a full document through create_html_processor and compares two requests byte-for-byte, which is what the plan's Task 3 Step 2 actually requires and the most valuable test still missing. Adopts the reviewer's gate: no Task 3 Step 4 and no exposure of AssemblyMode to test or staging traffic until that test exists. The three fixes close the known holes; the test is what would catch the next one. Also notes the audit result for integration head_inserts, which is clean — all three implementations ignore the request context — so the neutrality gap was specific to diagnostics rather than general to the seam. --- .../2026-08-08-1009-measurement-findings.md | 67 +++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md index 0b6a70f67..2d92512f9 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md @@ -283,6 +283,73 @@ spec flags as needing an audit and which that audit has not yet covered. Until i treat request-neutrality as asserted for one element rather than established for the template. +## Code review of the Task 3 commits — three HIGH findings, all closed + +**Date:** 2026-08-11. An independent review of the four implementation commits found +three HIGH issues. The default `Inline` path was verified unchanged byte-for-byte, so +none was a live regression — but all three were invariants this branch exists to +establish and none was enforced or tested. + +### 1. The auction dispatched under shared modes with nothing to consume it + +`assembly_mode` was computed _after_ the dispatch decision, so flipping to `client_fill` +or `esi` would still have sent real SSP bid requests, held the response for the full +auction budget, and discarded the result — because both injection seams now return +nothing — with no error, no warning and no log. + +Exactly the silent-waste signature §5 of the design doc is about, reached by an +incomplete feature flag rather than by removing the hold. Fixed by hoisting +`assembly_mode` above the dispatch and gating on `root_auction_is_useful`. + +The test derives the invariant rather than asserting per-variant: a root auction is +useful exactly when a seam will consume its result. A new mode cannot make the dispatch +gate and the injection decisions disagree without failing it. + +### 2. The C2 gate ignored the forwarded client `Cookie` + +TS forwards client cookies to origin unchanged — there is no `Cookie` strip on the +publisher path. So a response can be cookie-personalized while carrying no `Set-Cookie` +itself (session established earlier), no `Cache-Control` at all, status 200, HTML — and +every condition in the gate reported it cacheable. + +§4 of the design doc names this. The plan's own Task 3 Step 3 checklist missed it, so +the implementation matching the checklist exactly still had the hole. Now disqualifying +until an origin `Vary` covering `Cookie` is verified. + +### 3. Request-neutrality was asserted for one element, not the seam + +The head seam still injected integration `head_inserts` and the GPT-diagnostics +bootstrap unconditionally. + +Audited both. **`head_inserts` is clean** — all three implementations (datadome, didomi, +gpt) take the context parameter unused, so output depends on configuration, not the +request. **GPT diagnostics is not** — cookie- or query-activated, and documented as an +immutable request-scoped decision. + +It did not leak, but only by coincidence: `requires_private_no_store()` is a strict +superset of the conditions under which either script is emitted, and that stamp lands +before the C2 gate reads response headers, so the gate refused. Two independent +conditions that happened to align, with nothing enforcing the relationship. + +Fixed on both sides — the processor receives no diagnostics decision under shared modes, +**and** a test enumerates every combination of the decision's three fields asserting that +anything which injects also requires the stamp. The gate is the guarantee; the invariant +test is the backstop if the gate is ever removed. + +### What this says about the tests that existed + +All three findings were in code the existing tests covered — and passed. The tests +exercised the pure decision functions with hand-built inputs and never the rendered +``/`` bytes. That is still true: **no test renders a full document through +`create_html_processor` and compares two requests byte-for-byte.** The plan's Task 3 +Step 2 requires exactly that, and it remains the most valuable missing test. + +### Reviewer's gate, adopted + +Do not proceed to Task 3 Step 4 (actual C2 read/write) or expose `AssemblyMode` to any +test or staging traffic until the full-document byte-identity test exists. The three +fixes above close the known holes; that test is what would catch the next one. + ## Step B — consumers of TS's own response headers Not yet run. From 6e6b21a831c1fca3d55f4104e24cc6315934db0c Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 09:23:59 +0530 Subject: [PATCH 23/44] Render whole documents and compare bytes, and prove the test has teeth MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the gate the review left open, and the one the plan's Task 3 Step 2 actually asked for. Every other test in this area exercises the decision functions with hand-built inputs. That is how three HIGH review findings sat in covered, passing code: the decisions were individually right, and nothing checked what composing them renders. These tests build the config exactly as create_html_stream_processor does — same three decisions, same order — render a document through create_html_processor, and compare bytes across every combination of ad-stack gating, diagnostics activation, and bid availability. Extracted template_gpt_diagnostics so all three decisions are named functions the test can compose, rather than one of them being an inline match the test would have to duplicate. Duplicating it would have made the test agree with itself instead of with production. Mutation-tested both gates rather than trusting that passing tests mean anything. Reverting the diagnostics gate fails two of the three; reverting the head-seam gate fails the same two; the inline control passes in both cases. So the tests detect each gate independently and can still tell varying from non-varying output. Three tests rather than one, because byte-identity alone is satisfiable by rendering the same wrong thing every time. The second asserts the specific request-scoped markers that must be absent, and the third asserts inline still varies — if that one ever passes trivially, the harness is not rendering what it claims to. Adds a cfg(test) constructor for an active diagnostics decision, since the fields are private and built from a cookie or query parameter, with no other way to obtain one across a module boundary. Verified: fmt, all six clippy targets, all four adapter suites, 1858 core tests. --- .../src/integrations/gpt_diagnostics.rs | 17 ++ crates/trusted-server-core/src/publisher.rs | 210 ++++++++++++++++-- 2 files changed, 213 insertions(+), 14 deletions(-) diff --git a/crates/trusted-server-core/src/integrations/gpt_diagnostics.rs b/crates/trusted-server-core/src/integrations/gpt_diagnostics.rs index bef234d25..3956fd682 100644 --- a/crates/trusted-server-core/src/integrations/gpt_diagnostics.rs +++ b/crates/trusted-server-core/src/integrations/gpt_diagnostics.rs @@ -114,6 +114,23 @@ impl GptDiagnosticsRequestDecision { } } +impl GptDiagnosticsRequestDecision { + /// An active decision, for tests in other modules that need one. + /// + /// The fields are private and built by `prepare_request` from a cookie or query + /// parameter; there is no other way to obtain an active decision across a module + /// boundary. + #[cfg(test)] + #[must_use] + pub(crate) fn active_for_tests() -> Self { + Self { + active: true, + clean_browser_path_and_query: None, + cookie_action: GptDiagnosticsCookieAction::None, + } + } +} + #[cfg(test)] mod head_seam_invariant_tests { use super::*; diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index 8fbe70f05..ead7e4fb6 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -949,6 +949,29 @@ struct HtmlStreamProcessorParams<'a> { gpt_diagnostics: Option, } +/// The diagnostics decision the template may carry. +/// +/// Diagnostics is request-scoped — activated by a cookie or query parameter, and +/// documented as an immutable per-request decision — so it must not reach a shared +/// template. +/// +/// It does not leak today even without this gate, but only by coincidence: +/// `requires_private_no_store()` is a strict superset of the conditions under which +/// a script is emitted, and that stamp lands before the C2 gate reads response +/// headers, so the gate refuses. Two independent conditions that happen to align, +/// with nothing enforcing the relationship. This makes the guarantee explicit; +/// `requires_private_no_store_is_a_superset_of_injection` keeps the coincidence as a +/// backstop if this gate is ever removed. +pub(crate) fn template_gpt_diagnostics( + mode: AssemblyMode, + decision: Option, +) -> Option { + match mode { + AssemblyMode::Inline => decision, + AssemblyMode::ClientFill | AssemblyMode::Esi => None, + } +} + /// Whether a root-level auction has any consumer under this assembly mode. /// /// Only [`AssemblyMode::Inline`] injects the auction result into the root document. @@ -1021,18 +1044,7 @@ fn create_html_stream_processor( .unwrap_or_default(); let body_close = body_close_injection(assembly_mode, params.ad_slots_script.is_some()); - // Diagnostics is request-scoped — activated by a cookie or query parameter — so - // it must not reach a shared template. It does not leak today, but only by - // coincidence: `requires_private_no_store()` is a strict superset of the - // conditions under which a script is emitted, and the resulting `private, - // no-store` stamp lands before the C2 gate reads response headers, so the gate - // refuses. That is two independent conditions happening to align. Gate it here - // instead, so the guarantee does not depend on a relationship nothing enforces. - // `gpt_diagnostics_superset_of_injection` locks the coincidence as a backstop. - let gpt_diagnostics = match assembly_mode { - AssemblyMode::Inline => params.gpt_diagnostics, - AssemblyMode::ClientFill | AssemblyMode::Esi => None, - }; + let gpt_diagnostics = template_gpt_diagnostics(assembly_mode, params.gpt_diagnostics); let config = config .with_ad_state(params.ad_slots_script, params.ad_bids_state) @@ -4792,6 +4804,176 @@ mod tests { .expect("should proxy publisher request") } + mod rendered_template_identity_tests { + //! The gate the plan's Task 3 Step 2 actually asks for. + //! + //! Every other test in this area exercises the decision functions with + //! hand-built inputs. That is how three HIGH review findings sat in covered, + //! passing code: the decisions were right and nothing checked what the + //! composition of them *renders*. + //! + //! These tests render whole documents through `create_html_processor`, + //! composing the same three decisions `create_html_stream_processor` uses, + //! and compare bytes. A future request-dependent injection added at either + //! seam fails here even if every decision function is left untouched. + + use super::template_neutrality_tests::{settings_with_slots, slot}; + use super::*; + use crate::creative_opportunities::AssemblyMode; + use crate::html_processor::{HtmlProcessorConfig, create_html_processor}; + use crate::integrations::IntegrationRegistry; + use crate::integrations::gpt_diagnostics::GptDiagnosticsRequestDecision; + + const DOCUMENT: &[u8] = + b"t

content

"; + + /// One request's worth of variation. Everything here is request-scoped and + /// must not reach a shared template. + #[derive(Debug, Clone, Copy)] + struct RequestShape { + /// Folds in consent, bot classification, prefetch and the kill switch. + ad_stack_ran: bool, + /// Cookie- or query-activated. + diagnostics_active: bool, + /// A resolved auction, present only when one was dispatched. + bids_available: bool, + } + + /// Build the config exactly as `create_html_stream_processor` does, so a + /// drift between a decision and its use is caught rather than hidden. + fn render(mode: AssemblyMode, shape: RequestShape) -> String { + let settings = settings_with_slots(); + let slots = [slot()]; + + let ad_slots_script = + template_ad_slots_script(mode, shape.ad_stack_ran, &settings, &slots, "/"); + let body_close = body_close_injection(mode, ad_slots_script.is_some()); + let gpt_diagnostics = template_gpt_diagnostics( + mode, + shape + .diagnostics_active + .then(GptDiagnosticsRequestDecision::active_for_tests), + ); + + let ad_bids_state = + std::sync::Arc::new(std::sync::Mutex::new(shape.bids_available.then(|| { + r#""#.to_string() + }))); + + let config = HtmlProcessorConfig { + origin_host: "origin.example.com".to_string(), + request_host: "example.com".to_string(), + request_scheme: "https".to_string(), + integrations: IntegrationRegistry::empty_for_tests(), + ad_slots_script, + ad_bids_state, + max_buffered_body_bytes: 16 * 1024 * 1024, + gpt_diagnostics, + body_close, + }; + + let mut processor = create_html_processor(config); + let out = processor + .process_chunk(DOCUMENT, true) + .expect("should process the document"); + String::from_utf8(out).expect("output should be utf8") + } + + fn every_shape() -> Vec { + let mut shapes = Vec::new(); + for ad_stack_ran in [false, true] { + for diagnostics_active in [false, true] { + for bids_available in [false, true] { + shapes.push(RequestShape { + ad_stack_ran, + diagnostics_active, + bids_available, + }); + } + } + } + shapes + } + + #[test] + fn shared_modes_render_byte_identical_documents_for_every_request_shape() { + for mode in [AssemblyMode::ClientFill, AssemblyMode::Esi] { + let shapes = every_shape(); + let baseline = render(mode, shapes[0]); + + for shape in &shapes[1..] { + let rendered = render(mode, *shape); + assert_eq!( + rendered, baseline, + "{mode:?}: rendered template differs for {shape:?}. A shared \ + template that varies by request freezes the first-filling \ + request's decision for every later reader." + ); + } + } + } + + #[test] + fn shared_mode_templates_contain_no_request_scoped_markers() { + // Byte-identity alone would be satisfied by rendering the same wrong + // thing every time, so also assert the specific things that must be + // absent. + for mode in [AssemblyMode::ClientFill, AssemblyMode::Esi] { + let rendered = render( + mode, + RequestShape { + ad_stack_ran: true, + diagnostics_active: true, + bids_available: true, + }, + ); + for forbidden in [ + ".adSlots", + ".bids=", + "__tsjs_gpt_diagnostics_active", + "history.replaceState", + ] { + assert!( + !rendered.contains(forbidden), + "{mode:?}: template contains request-scoped `{forbidden}`:\n{rendered}" + ); + } + } + } + + #[test] + fn inline_still_varies_by_request_as_it_must() { + // The shared-mode assertions would also pass if rendering were broken + // everywhere. Inline responses are per-navigation and never shared, so + // they *should* differ — this proves the test can tell the difference. + let with_ads = render( + AssemblyMode::Inline, + RequestShape { + ad_stack_ran: true, + diagnostics_active: false, + bids_available: true, + }, + ); + let without = render( + AssemblyMode::Inline, + RequestShape { + ad_stack_ran: false, + diagnostics_active: false, + bids_available: false, + }, + ); + assert_ne!( + with_ads, without, + "inline must still vary by request; if it does not, this harness is \ + not rendering what it claims to" + ); + assert!( + with_ads.contains(".adSlots"), + "inline with a matched slot should carry adSlots" + ); + } + } + mod root_auction_gate_tests { //! Guards the silent-waste failure mode: dispatching an auction whose result //! nothing will consume. Under the shared modes both injection seams emit @@ -5115,7 +5297,7 @@ mod tests { AssemblyMode, CreativeOpportunityFormat, CreativeOpportunitySlot, }; - fn slot() -> CreativeOpportunitySlot { + pub(super) fn slot() -> CreativeOpportunitySlot { CreativeOpportunitySlot { id: "atf".to_string(), gam_unit_path: Some("/99999/example/home".to_string()), @@ -5134,7 +5316,7 @@ mod tests { } } - fn settings_with_slots() -> Settings { + pub(super) fn settings_with_slots() -> Settings { let mut settings = crate::test_support::tests::create_test_settings(); // Construct the section rather than mutating it if present: the shared // fixture does not carry `[creative_opportunities]`, and an `if let From 04a3196520c80d1b2f5075fc690164dc68530899 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 09:50:22 +0530 Subject: [PATCH 24/44] Add the C2 template cache trait, key and null object MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First half of Task 3 Step 4, in portable core. No Fastly implementation yet and no call site, so nothing changes behaviour — this is the shape the adapter will fill in. Follows the PlatformKvStore pattern the repo already uses four times for a Fastly-only capability behind a portable trait with a null object. The null object reports Unsupported rather than erroring, so the shared assembly modes degrade to transforming per request on Cloudflare, Axum and Spin instead of failing there. The modes stay portable; only the caching does not. The key is where the correctness risks live, and it carries the four the design investigation surfaced. Assembly mode, because the client-fill and ESI arms emit different bytes and would otherwise poison each other's entries. Content encoding, because the pipeline pairs input encoding to output encoding, so serving brotli bytes to a client that asked for gzip is a broken response. Host and scheme, because both reach IntegrationHtmlContext and drive URL rewriting. And a schema version, so a deploy that changes the transform reads a miss rather than assembling against markers that moved. Vary values are carried as the origin listed them rather than as a fixed list, because the origin is authoritative and a hard-coded list would drift silently when the origin's changes. Step A already measured four Next-specific headers this branch did not anticipate. Fields are length-prefixed rather than delimiter-joined. A delimiter is ambiguous when a value can contain it, and two distinct keys colliding here means one visitor's template served to another. There is a test for exactly that collision. Metadata is a small envelope rather than stored origin headers. The publisher path forces private/no-store and strips validators after the send, so replaying a stored origin header would fight it; rebuilding every header on a hit means no origin header is ever replayed and the Set-Cookie privacy net stays trivially safe. Malformed metadata decodes to a miss rather than a partial read. Eight tests. The one worth naming asserts every field changes the key — a field that does not is a cross-serving bug, and that property is easy to break by adding a field and forgetting to hash it. Verified: fmt, all six clippy targets, all four adapter suites, 1866 core tests. --- .../trusted-server-core/src/platform/mod.rs | 5 + .../src/platform/template_cache.rs | 493 ++++++++++++++++++ 2 files changed, 498 insertions(+) create mode 100644 crates/trusted-server-core/src/platform/template_cache.rs diff --git a/crates/trusted-server-core/src/platform/mod.rs b/crates/trusted-server-core/src/platform/mod.rs index 287f1accf..7cab20d29 100644 --- a/crates/trusted-server-core/src/platform/mod.rs +++ b/crates/trusted-server-core/src/platform/mod.rs @@ -36,6 +36,7 @@ mod error; mod http; mod image_optimizer; mod kv; +pub mod template_cache; #[cfg(test)] pub(crate) mod test_support; mod traits; @@ -52,6 +53,10 @@ pub use image_optimizer::{ PlatformImageOptimizerParams, PlatformImageOptimizerRegion, }; pub use kv::UnavailableKvStore; +pub use template_cache::{ + PlatformTemplateCache, TEMPLATE_SCHEMA_VERSION, TemplateCacheError, TemplateCacheKey, + TemplateCacheMiss, TemplateEntry, TemplateMetadata, UnavailableTemplateCache, +}; pub use traits::{PlatformBackend, PlatformConfigStore, PlatformGeo, PlatformSecretStore}; pub use types::{ ClientInfo, GeoInfo, PlatformBackendSpec, RuntimeServices, RuntimeServicesBuilder, StoreId, diff --git a/crates/trusted-server-core/src/platform/template_cache.rs b/crates/trusted-server-core/src/platform/template_cache.rs new file mode 100644 index 000000000..c89926484 --- /dev/null +++ b/crates/trusted-server-core/src/platform/template_cache.rs @@ -0,0 +1,493 @@ +//! The shared transformed-template cache (C2) for the #1009 ESI validation spike. +//! +//! Three caches are in play and conflating them is what produced the original wrong +//! conclusion in the design doc, so this module names which one it is: +//! +//! | Cache | Contents | Owner | +//! | ----- | --------------------------------- | ------------------------------ | +//! | C1 | raw origin bytes | Fastly read-through. Not this. | +//! | C2 | post-`lol_html`, pre-assembly | **This module.** | +//! | C3 | final per-user assembled response | **Must never exist.** | +//! +//! C2 holds a *shared template*: no per-user bytes, and no decisions that depend on +//! the request. What may and may not live in it is +//! [§6.7 of the design doc](../../../../docs/superpowers/specs/2026-08-08-esi-cacheable-root-validation-design.md), +//! and the invariant is enforced by the rendered-document byte-identity tests in +//! `publisher`. +//! +//! Spike-only. Remove with the spike. + +use core::fmt; + +use crate::creative_opportunities::AssemblyMode; + +/// Version of the transform that produced a cached template. +/// +/// Bump on **any** change to what the transform emits. Without it a deploy reads +/// yesterday's template shape and assembles against markers that moved, which fails +/// as a rendering bug far from its cause rather than as a cache miss. +pub const TEMPLATE_SCHEMA_VERSION: u32 = 1; + +/// Inputs that select one cached template. +/// +/// Every field changes the emitted bytes for the same URL. A signal that changes the +/// bytes and is **not** here produces cross-served templates; a signal that is +/// per-user does not belong here at all — it belongs out of the template entirely. +/// That distinction is the whole design: the key holds per-*variant* signals, and +/// per-*user* signals are excluded from the template rather than keyed on. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct TemplateCacheKey { + /// Full request URL, stated explicitly rather than inherited from an ambient + /// request, so the key cannot silently depend on what the caller happened to + /// mutate first. + pub url: String, + /// Host and scheme. The post-processed output is host-dependent by construction: + /// both reach `IntegrationHtmlContext` and drive URL rewriting. + pub request_host: String, + /// See [`Self::request_host`]. + pub request_scheme: String, + /// A2 and A3 emit different template bytes. Without this they poison each + /// other's entries. + pub assembly_mode: AssemblyMode, + /// Values of the request headers the **origin** declares it varies on, in the + /// order the origin listed them. Not a fixed list: the origin is authoritative, + /// and hard-coding one here would silently drift when the origin's changes. + pub vary_values: Vec<(String, String)>, + /// The negotiated content encoding of the stored bytes. + /// + /// The streaming pipeline pairs input encoding to the same output encoding, so + /// the transformed bytes inherit whatever the origin chose from the client's + /// `Accept-Encoding`. Serving brotli bytes to a client that asked for gzip is a + /// broken response, so this is part of the key rather than of the payload. + pub content_encoding: String, + /// Identifies the enabled integration set and the tsjs bundle. Both change the + /// injected markup for the same URL. + pub integration_fingerprint: String, + /// See [`TEMPLATE_SCHEMA_VERSION`]. + pub schema_version: u32, +} + +impl TemplateCacheKey { + /// Render the key as the opaque byte string the platform cache is keyed on. + /// + /// Fields are length-prefixed rather than delimiter-joined. A delimiter is + /// ambiguous when a value can contain it — a URL with a `|`, or a `Vary` value + /// with one — and two distinct keys colliding here means one visitor's template + /// served to another. Length prefixes make that unrepresentable. + #[must_use] + pub fn to_cache_key(&self) -> String { + let mut out = String::new(); + let mut push = |part: &str| { + out.push_str(&part.len().to_string()); + out.push(':'); + out.push_str(part); + }; + + push("ts-c2"); + push(&self.schema_version.to_string()); + push(&format!("{:?}", self.assembly_mode)); + push(&self.request_scheme); + push(&self.request_host); + push(&self.url); + push(&self.content_encoding); + push(&self.integration_fingerprint); + + push(&self.vary_values.len().to_string()); + for (name, value) in &self.vary_values { + push(&name.to_ascii_lowercase()); + push(value); + } + + out + } + + /// Surrogate keys to attach at insert, for purge-based rollback. + /// + /// `ts-template` purges every template at once, which is the rollback lever. + /// The per-URL key allows targeted invalidation. Both are needed: the broad one + /// for an incident, the narrow one for ordinary invalidation. + #[must_use] + pub fn surrogate_keys(&self) -> Vec { + vec![ + "ts-template".to_string(), + format!("ts-template-{}", surrogate_safe(&self.url)), + ] + } +} + +/// Reduce a URL to characters valid in a Fastly surrogate key. +/// +/// Surrogate keys are space-delimited, so any whitespace would split one key into +/// several and purge more than intended. +fn surrogate_safe(url: &str) -> String { + url.chars() + .map(|c| { + if c.is_ascii_alphanumeric() || c == '-' || c == '_' { + c + } else { + '_' + } + }) + .collect() +} + +/// Metadata stored alongside the template bytes. +/// +/// `cache::core` carries **no HTTP semantics** — status, headers, encoding and +/// revalidation are all the caller's. Rather than storing origin headers and +/// replaying them, store only what is needed to rebuild a response from scratch. +/// +/// That choice is deliberate and load-bearing: the publisher path forces +/// `private, no-store` and strips validators *after* the origin send, so replaying a +/// stored origin header would fight it. Rebuilding every header on a hit means no +/// origin header is ever replayed and the `Set-Cookie` privacy net stays trivially +/// safe. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct TemplateMetadata { + /// Encoding of the stored bytes. Also in the key; stored so a reader need not + /// re-derive it. + pub content_encoding: String, + /// Content type to rebuild the response with. + pub content_type: String, + /// Schema version the bytes were produced under. Checked on read: a mismatch is + /// a miss, not an error, so a rollback to an older binary degrades to + /// re-transforming rather than misassembling. + pub schema_version: u32, +} + +impl TemplateMetadata { + /// Serialize for `user_metadata`. Deliberately a tiny hand-rolled format rather + /// than JSON — one allocation, no dependency, and a parse failure is + /// unambiguous. + #[must_use] + pub fn encode(&self) -> Vec { + format!( + "v={}\nce={}\nct={}", + self.schema_version, self.content_encoding, self.content_type + ) + .into_bytes() + } + + /// Parse `user_metadata`. Returns `None` on anything unexpected, which callers + /// must treat as a cache miss. + #[must_use] + pub fn decode(raw: &[u8]) -> Option { + let text = core::str::from_utf8(raw).ok()?; + let mut schema_version = None; + let mut content_encoding = None; + let mut content_type = None; + for line in text.lines() { + let (key, value) = line.split_once('=')?; + match key { + "v" => schema_version = Some(value.parse().ok()?), + "ce" => content_encoding = Some(value.to_string()), + "ct" => content_type = Some(value.to_string()), + _ => return None, + } + } + Some(Self { + schema_version: schema_version?, + content_encoding: content_encoding?, + content_type: content_type?, + }) + } +} + +/// Why a template read did not produce usable bytes. +#[derive(Debug, Clone, Copy, PartialEq, Eq, derive_more::Display)] +pub enum TemplateCacheMiss { + /// No entry for this key. + #[display("no cached template for this key")] + NotFound, + /// Found, but produced by a different transform version. + #[display("cached template has a different schema version")] + SchemaMismatch, + /// Found, but its metadata could not be parsed. + #[display("cached template metadata is unreadable")] + UnreadableMetadata, + /// This platform has no template cache. + #[display("no template cache on this platform")] + Unsupported, +} + +impl core::error::Error for TemplateCacheMiss {} + +/// Errors a template cache write can produce. +#[derive(Debug, derive_more::Display)] +pub enum TemplateCacheError { + /// This platform has no template cache. + #[display("no template cache on this platform")] + Unsupported, + /// The platform rejected the operation. + #[display("template cache backend error: {message}")] + Backend { + /// What the backend reported. + message: String, + }, +} + +impl core::error::Error for TemplateCacheError {} + +impl fmt::Debug for dyn PlatformTemplateCache { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str("PlatformTemplateCache") + } +} + +/// A platform's shared-template cache. +/// +/// Only the Fastly adapter implements this; every other adapter uses +/// [`UnavailableTemplateCache`], which reports [`TemplateCacheMiss::Unsupported`] so +/// the caller transforms every time rather than failing. +#[async_trait::async_trait(?Send)] +pub trait PlatformTemplateCache { + /// Read a template. `Err` is a miss, not a failure — every variant means + /// "transform it yourself". + async fn get(&self, key: &TemplateCacheKey) -> Result; + + /// Store a template. + /// + /// Callers must not call this without having consulted the C2 eligibility gate + /// first: this method stores what it is given and cannot tell a shared template + /// from a per-user one. + async fn put( + &self, + key: &TemplateCacheKey, + metadata: &TemplateMetadata, + body: Vec, + ) -> Result<(), TemplateCacheError>; + + /// Purge every stored template. The rollback lever. + async fn purge_all(&self) -> Result<(), TemplateCacheError>; +} + +/// A template read from the cache. +pub struct TemplateEntry { + /// Metadata stored at insert. + pub metadata: TemplateMetadata, + /// The transformed template bytes. + pub body: Vec, +} + +/// The null object, used by every adapter without a template cache. +/// +/// Reporting [`TemplateCacheMiss::Unsupported`] rather than erroring means the +/// shared assembly modes degrade to transforming per request on Cloudflare, Axum and +/// Spin instead of failing — the modes stay portable, only the caching is not. +pub struct UnavailableTemplateCache; + +#[async_trait::async_trait(?Send)] +impl PlatformTemplateCache for UnavailableTemplateCache { + async fn get(&self, _key: &TemplateCacheKey) -> Result { + Err(TemplateCacheMiss::Unsupported) + } + + async fn put( + &self, + _key: &TemplateCacheKey, + _metadata: &TemplateMetadata, + _body: Vec, + ) -> Result<(), TemplateCacheError> { + Err(TemplateCacheError::Unsupported) + } + + async fn purge_all(&self) -> Result<(), TemplateCacheError> { + Err(TemplateCacheError::Unsupported) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn key() -> TemplateCacheKey { + TemplateCacheKey { + url: "https://example.com/news/article".to_string(), + request_host: "example.com".to_string(), + request_scheme: "https".to_string(), + assembly_mode: AssemblyMode::Esi, + vary_values: vec![("rsc".to_string(), "1".to_string())], + content_encoding: "gzip".to_string(), + integration_fingerprint: "abc123".to_string(), + schema_version: TEMPLATE_SCHEMA_VERSION, + } + } + + /// Every field must change the key. A field that does not is a cross-serving + /// bug: two requests needing different templates would share one entry. + #[test] + fn every_field_changes_the_key() { + let base = key().to_cache_key(); + + let mut mode = key(); + mode.assembly_mode = AssemblyMode::ClientFill; + assert_ne!( + mode.to_cache_key(), + base, + "assembly mode must change the key" + ); + + let mut url = key(); + url.url = "https://example.com/other".to_string(); + assert_ne!(url.to_cache_key(), base, "url must change the key"); + + let mut host = key(); + host.request_host = "other.example.com".to_string(); + assert_ne!(host.to_cache_key(), base, "host must change the key"); + + let mut scheme = key(); + scheme.request_scheme = "http".to_string(); + assert_ne!(scheme.to_cache_key(), base, "scheme must change the key"); + + let mut encoding = key(); + encoding.content_encoding = "br".to_string(); + assert_ne!( + encoding.to_cache_key(), + base, + "content encoding must change the key; serving brotli to a gzip client \ + is a broken response" + ); + + let mut fingerprint = key(); + fingerprint.integration_fingerprint = "def456".to_string(); + assert_ne!( + fingerprint.to_cache_key(), + base, + "integration fingerprint must change the key" + ); + + let mut schema = key(); + schema.schema_version = TEMPLATE_SCHEMA_VERSION + 1; + assert_ne!( + schema.to_cache_key(), + base, + "schema version must change the key" + ); + + let mut vary = key(); + vary.vary_values = vec![("rsc".to_string(), "0".to_string())]; + assert_ne!(vary.to_cache_key(), base, "vary values must change the key"); + } + + /// The reason for length prefixes rather than a delimiter. + #[test] + fn values_containing_delimiters_cannot_collide() { + let mut a = key(); + a.request_host = "a".to_string(); + a.url = "b:c".to_string(); + + let mut b = key(); + b.request_host = "a:b".to_string(); + b.url = "c".to_string(); + + assert_ne!( + a.to_cache_key(), + b.to_cache_key(), + "field values containing the delimiter must not produce the same key; a \ + collision here serves one visitor's template to another" + ); + } + + #[test] + fn vary_header_names_are_matched_case_insensitively() { + let mut upper = key(); + upper.vary_values = vec![("RSC".to_string(), "1".to_string())]; + assert_eq!( + upper.to_cache_key(), + key().to_cache_key(), + "header names are case-insensitive, so casing must not split the cache" + ); + } + + #[test] + fn vary_values_are_order_sensitive() { + // The origin lists them in a fixed order and the caller preserves it, so a + // differing order means differing inputs rather than the same request. + let mut a = key(); + a.vary_values = vec![ + ("rsc".to_string(), "1".to_string()), + ("accept-encoding".to_string(), "gzip".to_string()), + ]; + let mut b = key(); + b.vary_values = vec![ + ("accept-encoding".to_string(), "gzip".to_string()), + ("rsc".to_string(), "1".to_string()), + ]; + assert_ne!(a.to_cache_key(), b.to_cache_key()); + } + + #[test] + fn surrogate_keys_carry_a_global_and_a_per_url_lever() { + let keys = key().surrogate_keys(); + assert!( + keys.contains(&"ts-template".to_string()), + "a global purge lever is what makes rollback possible" + ); + assert_eq!(keys.len(), 2, "global plus per-URL"); + assert!( + !keys[1].contains(char::is_whitespace), + "surrogate keys are space-delimited; whitespace would purge more than \ + intended, got {:?}", + keys[1] + ); + assert!( + !keys[1].contains('/') && !keys[1].contains(':'), + "URL punctuation must be reduced, got {:?}", + keys[1] + ); + } + + #[test] + fn metadata_round_trips() { + let metadata = TemplateMetadata { + content_encoding: "gzip".to_string(), + content_type: "text/html; charset=utf-8".to_string(), + schema_version: TEMPLATE_SCHEMA_VERSION, + }; + let decoded = + TemplateMetadata::decode(&metadata.encode()).expect("should decode what it encoded"); + assert_eq!(decoded, metadata); + } + + #[test] + fn unparseable_metadata_is_a_miss_not_a_panic() { + for raw in [ + &b"not-key-value"[..], + &b"v=notanumber\nce=gzip\nct=text/html"[..], + &b"v=1\nce=gzip"[..], + &b"v=1\nce=gzip\nct=text/html\nunexpected=1"[..], + &[0xff, 0xfe][..], + ] { + assert_eq!( + TemplateMetadata::decode(raw), + None, + "malformed metadata must be a miss, not a partial read: {raw:?}" + ); + } + } + + #[tokio::test] + async fn the_null_object_reports_unsupported_rather_than_failing() { + // Degrading to per-request transformation keeps the shared modes portable on + // adapters with no cache; erroring would make them Fastly-only outright. + let cache = UnavailableTemplateCache; + assert_eq!( + cache.get(&key()).await.err(), + Some(TemplateCacheMiss::Unsupported) + ); + assert!(matches!( + cache + .put( + &key(), + &TemplateMetadata { + content_encoding: "identity".to_string(), + content_type: "text/html".to_string(), + schema_version: TEMPLATE_SCHEMA_VERSION, + }, + Vec::new() + ) + .await, + Err(TemplateCacheError::Unsupported) + )); + } +} From ecc6c5306f356d3ad3ed17df14d5d3d3b37a37d6 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 10:20:33 +0530 Subject: [PATCH 25/44] Back the C2 template cache with Fastly Core Cache MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes Task 3 Step 4. The cache is constructed and reachable through RuntimeServices but has no caller yet, and the assembly mode defaults to Inline, so nothing changes behaviour. Seven tests run against real Core Cache under Viceroy, including purge. That is what the earlier probe established was possible and why provisioning a Fastly service is not on the critical path. Two ordering traps, both caught by the earlier reviews and both real here. must_insert_or_update is tested before found, because a stale entry sets both and checking found first would serve the stale bytes while never discharging the obligation, leaving concurrent waiters blocked until timeout. And get uses a plain lookup rather than a transaction, because a read that never intends to insert must not take an obligation it will not discharge. Transaction::insert takes self, so once the insert begins there is no handle left to cancel it with — a write that fails part-way cannot be retracted. Rather than write a cancel call that does not compile, or pretend the hazard is absent, the metadata carries the intended body length and get rejects a short entry as Truncated. put also refuses a length that disagrees with the body it was given, since storing that would make every subsequent read a truncation miss: a cache that silently never hits. Also treats a stale entry as a miss. Serving stale while revalidating is a real option but it is a state machine cache::core does not implement, and it is not what this spike measures. The trait is Send + Sync with ?Send futures. RuntimeServices lives in a LazyLock static so the trait object must cross threads, while the platform layer is !Send by construction and the futures never do. Wired into RuntimeServices following the kv_store pattern, but defaulted rather than required: an adapter with no template cache should degrade to transforming per request, not fail to build. That is what keeps the shared modes portable across all four adapters with only the caching being Fastly-only. Verified: fmt, all six clippy targets, all four adapter suites, 1866 core tests and 123 Fastly adapter tests. --- .../trusted-server-adapter-fastly/src/app.rs | 6 + .../trusted-server-adapter-fastly/src/main.rs | 1 + .../src/template_cache.rs | 327 ++++++++++++++++++ .../src/platform/template_cache.rs | 34 +- .../trusted-server-core/src/platform/types.rs | 36 ++ 5 files changed, 398 insertions(+), 6 deletions(-) create mode 100644 crates/trusted-server-adapter-fastly/src/template_cache.rs diff --git a/crates/trusted-server-adapter-fastly/src/app.rs b/crates/trusted-server-adapter-fastly/src/app.rs index 5258d3455..c85e24670 100644 --- a/crates/trusted-server-adapter-fastly/src/app.rs +++ b/crates/trusted-server-adapter-fastly/src/app.rs @@ -256,6 +256,12 @@ fn build_per_request_services(state: &AppState, ctx: &RequestContext) -> Runtime .config_store(Arc::new(FastlyPlatformConfigStore)) .secret_store(Arc::new(FastlyPlatformSecretStore)) .kv_store(Arc::clone(&state.default_kv_store)) + // Spike-only (#1009). Constructed unconditionally, but only read when the + // assembly mode is a shared-template one — which defaults to Inline, so this + // is inert until an operator opts in. + .template_cache(Arc::new(crate::template_cache::FastlyTemplateCache::new( + crate::template_cache::TEMPLATE_CACHE_TTL, + ))) .backend(Arc::new(FastlyPlatformBackend)) .http_client(Arc::new(FastlyPlatformHttpClient)) .geo(Arc::new(FastlyPlatformGeo)) diff --git a/crates/trusted-server-adapter-fastly/src/main.rs b/crates/trusted-server-adapter-fastly/src/main.rs index 39d35b198..603ffdd9b 100644 --- a/crates/trusted-server-adapter-fastly/src/main.rs +++ b/crates/trusted-server-adapter-fastly/src/main.rs @@ -34,6 +34,7 @@ mod management_api; mod middleware; mod platform; mod rate_limiter; +mod template_cache; mod tinybird; use crate::app::{EcFinalizeState, TrustedServerApp, load_settings_from_config_store}; diff --git a/crates/trusted-server-adapter-fastly/src/template_cache.rs b/crates/trusted-server-adapter-fastly/src/template_cache.rs new file mode 100644 index 000000000..d037a3e6a --- /dev/null +++ b/crates/trusted-server-adapter-fastly/src/template_cache.rs @@ -0,0 +1,327 @@ +//! Fastly Core Cache backing for the shared transformed-template cache (C2). +//! +//! Only the Fastly adapter implements this; every other adapter uses +//! `UnavailableTemplateCache`, so the shared assembly modes stay portable and only +//! the caching is Fastly-only. +//! +//! **Why Core Cache and not read-through caching.** Read-through with `after_send` + +//! `set_body_transform` looks like a better fit — it keeps HTTP semantics and derives +//! TTL and surrogate keys from origin headers for free. It is unreachable here: +//! Viceroy 0.17 stubs the entire HTTP Cache ABI and the SDK converts that into a +//! *send error*, so setting `after_send` makes every publisher origin fetch fail +//! under `fastly compute serve`, `cargo test-fastly` and the parity suite. It is also +//! silently dead whenever the origin request is in pass mode, and its closure bounds +//! (`Fn + Send + Sync`) are incompatible with a platform layer that is `!Send` by +//! construction. Recorded in the spike plan's Task 3 Step 4 so nobody re-proposes it. +//! +//! Spike-only. Remove with the spike. + +use fastly::cache::core::{CacheKey, Transaction}; +use std::io::Write as _; +use std::time::Duration; +use trusted_server_core::platform::{ + PlatformTemplateCache, TemplateCacheError, TemplateCacheKey, TemplateCacheMiss, TemplateEntry, + TemplateMetadata, +}; + +/// How long a cached template lives. +/// +/// Deliberately short for the spike. A short TTL bounds every failure mode in this +/// module — a poisoned template, a stale schema, a truncated write — and the only +/// cost is hit rate, which is a measurement input rather than a correctness one. +pub const TEMPLATE_CACHE_TTL: Duration = Duration::from_secs(60); + +/// Surrogate key attached to every stored template, so a single purge clears them +/// all. This is the rollback lever: without it, backing out a bad template means +/// waiting for the TTL. +const PURGE_ALL_SURROGATE_KEY: &str = "ts-template"; + +/// Fastly Core Cache implementation of the C2 template cache. +pub struct FastlyTemplateCache { + ttl: Duration, +} + +impl FastlyTemplateCache { + /// Create a cache whose entries live for `ttl`. + /// + /// Keep this short for the spike. A short TTL bounds every failure mode in this + /// module — a poisoned template, a stale schema, a bad transform — and costs + /// only hit rate. + #[must_use] + pub fn new(ttl: Duration) -> Self { + Self { ttl } + } +} + +fn backend_error(message: impl Into) -> TemplateCacheError { + TemplateCacheError::Backend { + message: message.into(), + } +} + +#[async_trait::async_trait(?Send)] +impl PlatformTemplateCache for FastlyTemplateCache { + async fn get(&self, key: &TemplateCacheKey) -> Result { + let cache_key = CacheKey::from(key.to_cache_key().into_bytes()); + + // A plain lookup, not a transaction: a read that does not intend to insert + // must not take an insert obligation it will never discharge, which would + // block every other client waiting on the same key until they time out. + let found = fastly::cache::core::lookup(cache_key) + .execute() + .map_err(|_| TemplateCacheMiss::NotFound)? + .ok_or(TemplateCacheMiss::NotFound)?; + + // Stale entries are treated as a miss for the spike. Serving stale while + // revalidating is a real option, but it is a state machine `cache::core` + // does not implement for you, and it is not what this spike is measuring. + if found.is_stale() { + return Err(TemplateCacheMiss::NotFound); + } + + let metadata = TemplateMetadata::decode(&found.user_metadata()) + .ok_or(TemplateCacheMiss::UnreadableMetadata)?; + + // A schema mismatch is a miss, not an error: rolling back to an older binary + // then degrades to re-transforming rather than assembling against a template + // shape it does not understand. + if metadata.schema_version != key.schema_version { + return Err(TemplateCacheMiss::SchemaMismatch); + } + + let body = found + .to_stream() + .map_err(|_| TemplateCacheMiss::NotFound)? + .into_bytes(); + + // A write that failed part-way cannot cancel its own insert (see `put`), so + // a short entry is possible. Catch it here rather than assembling a + // truncated template into a broken page. + if body.len() as u64 != metadata.body_len { + return Err(TemplateCacheMiss::Truncated); + } + + Ok(TemplateEntry { metadata, body }) + } + + async fn put( + &self, + key: &TemplateCacheKey, + metadata: &TemplateMetadata, + body: Vec, + ) -> Result<(), TemplateCacheError> { + if metadata.body_len != body.len() as u64 { + return Err(backend_error(format!( + "metadata body_len {} does not match the {} bytes supplied; storing \ + this would make every read a truncation miss", + metadata.body_len, + body.len() + ))); + } + + let cache_key = CacheKey::from(key.to_cache_key().into_bytes()); + + // Transactional insert so a cold key under load transforms once rather than + // once per concurrent request. + let tx = Transaction::lookup(cache_key) + .execute() + .map_err(|e| backend_error(format!("transactional lookup failed: {e:?}")))?; + + // Order matters. A STALE entry sets *both* `found()` and + // `must_insert_or_update()`. Testing `found()` first would return early on + // the stale bytes and never discharge the obligation, leaving every + // concurrent waiter blocked until timeout. + if !tx.must_insert_or_update() { + // Someone else already inserted a fresh entry. Nothing to do, and + // nothing to discharge. + return Ok(()); + } + + // `Transaction::insert` takes `self`, so from here there is no handle left to + // cancel the insert with. A write that fails part-way therefore cannot be + // retracted — which is why `TemplateMetadata::body_len` exists and `get` + // checks it. The metadata is written before the body, so a truncated entry + // still carries the length it was supposed to have. + let surrogate_keys = key.surrogate_keys(); + let mut writer = tx + .insert(self.ttl) + .surrogate_keys(surrogate_keys.iter().map(String::as_str)) + .user_metadata(metadata.encode().into()) + .execute() + .map_err(|e| backend_error(format!("cache insert failed: {e:?}")))?; + + if let Err(e) = writer.write_all(&body) { + // Deliberately not calling `finish()`. An unfinished entry has no known + // length, and even if it is observable, `get`'s length check rejects it. + return Err(backend_error(format!("writing template body failed: {e}"))); + } + + // Required. Without it the object never completes and its length stays + // unknown, so readers see a partial or absent entry. + writer + .finish() + .map_err(|e| backend_error(format!("finishing the cached template failed: {e}")))?; + + Ok(()) + } + + async fn purge_all(&self) -> Result<(), TemplateCacheError> { + fastly::http::purge::purge_surrogate_key(PURGE_ALL_SURROGATE_KEY) + .map_err(|e| backend_error(format!("purging templates failed: {e:?}"))) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use trusted_server_core::creative_opportunities::AssemblyMode; + use trusted_server_core::platform::TEMPLATE_SCHEMA_VERSION; + + /// Distinct per test, so tests sharing the process cache cannot collide. + fn key(url: &str) -> TemplateCacheKey { + TemplateCacheKey { + url: url.to_string(), + request_host: "example.com".to_string(), + request_scheme: "https".to_string(), + assembly_mode: AssemblyMode::Esi, + vary_values: vec![("rsc".to_string(), "1".to_string())], + content_encoding: "identity".to_string(), + integration_fingerprint: "fp".to_string(), + schema_version: TEMPLATE_SCHEMA_VERSION, + } + } + + fn metadata_for(body: &[u8]) -> TemplateMetadata { + TemplateMetadata { + content_encoding: "identity".to_string(), + content_type: "text/html; charset=utf-8".to_string(), + schema_version: TEMPLATE_SCHEMA_VERSION, + body_len: body.len() as u64, + } + } + + /// The trait is `async_trait(?Send)` and this crate has no async test runtime, + /// so drive the futures directly. + fn run(fut: impl core::future::Future) -> T { + futures::executor::block_on(fut) + } + + fn cache() -> FastlyTemplateCache { + FastlyTemplateCache::new(Duration::from_secs(60)) + } + + #[test] + fn a_stored_template_reads_back_intact() { + let cache = cache(); + let key = key("https://example.com/roundtrip"); + let body = b"template".to_vec(); + let metadata = metadata_for(&body); + + run(cache.put(&key, &metadata, body.clone())).expect("should store"); + + let entry = run(cache.get(&key)).expect("should read back"); + assert_eq!(entry.body, body, "bytes must survive the round trip"); + assert_eq!(entry.metadata, metadata, "metadata must survive too"); + } + + #[test] + fn an_absent_key_is_a_miss_not_an_error() { + let miss = + run(cache().get(&key("https://example.com/never-stored"))).expect_err("should miss"); + assert_eq!(miss, TemplateCacheMiss::NotFound); + } + + #[test] + fn a_different_assembly_mode_does_not_read_the_same_entry() { + // The arms emit different bytes. If they shared an entry, one would serve + // the other's template. + let cache = cache(); + let esi = key("https://example.com/mode-split"); + let mut client_fill = esi.clone(); + client_fill.assembly_mode = AssemblyMode::ClientFill; + + let body = b"esi-template".to_vec(); + run(cache.put(&esi, &metadata_for(&body), body)).expect("should store"); + + assert_eq!( + run(cache.get(&client_fill)).err(), + Some(TemplateCacheMiss::NotFound), + "client-fill must not read the ESI arm's template" + ); + } + + #[test] + fn a_schema_bump_reads_a_miss_rather_than_a_stale_shape() { + let cache = cache(); + let key_v1 = key("https://example.com/schema"); + let body = b"old-shape".to_vec(); + run(cache.put(&key_v1, &metadata_for(&body), body)).expect("should store"); + + // A deploy that changes the transform bumps the constant. The old entry must + // not be assembled against. + let mut key_v2 = key_v1.clone(); + key_v2.schema_version = TEMPLATE_SCHEMA_VERSION + 1; + + assert_eq!( + run(cache.get(&key_v2)).err(), + Some(TemplateCacheMiss::NotFound), + "a bumped schema changes the key, so the old entry is simply not found" + ); + } + + #[test] + fn purge_all_clears_stored_templates() { + // The rollback lever. Without this, backing out a bad template means waiting + // for the TTL. + let cache = cache(); + let key = key("https://example.com/purge"); + let body = b"template".to_vec(); + run(cache.put(&key, &metadata_for(&body), body)).expect("should store"); + run(cache.get(&key)).expect("should be present before purge"); + + run(cache.purge_all()).expect("should purge"); + + assert!( + run(cache.get(&key)).is_err(), + "purge must clear the template, or rollback is TTL-bound" + ); + } + + #[test] + fn a_second_put_on_a_fresh_entry_is_a_no_op() { + // Exercises the `must_insert_or_update` early return: a concurrent writer + // that finds a fresh entry must neither error nor overwrite. + let cache = cache(); + let key = key("https://example.com/second-put"); + let first = b"first".to_vec(); + run(cache.put(&key, &metadata_for(&first), first.clone())).expect("first put stores"); + + let second = b"second".to_vec(); + run(cache.put(&key, &metadata_for(&second), second)) + .expect("second put should be a no-op, not an error"); + + assert_eq!( + run(cache.get(&key)).expect("should read").body, + first, + "a fresh entry must not be overwritten by a racing writer" + ); + } + + #[test] + fn a_length_mismatch_is_refused_at_write_rather_than_stored() { + // Storing metadata whose length disagrees with the body would make every + // subsequent read a truncation miss — a cache that silently never hits. + // Catch it at the write instead. + let cache = cache(); + let key = key("https://example.com/length-mismatch"); + let mut metadata = metadata_for(b"12345"); + metadata.body_len = 999; + + let err = run(cache.put(&key, &metadata, b"12345".to_vec())) + .expect_err("a length mismatch must be refused"); + assert!( + matches!(err, TemplateCacheError::Backend { .. }), + "expected a backend error, got {err:?}" + ); + } +} diff --git a/crates/trusted-server-core/src/platform/template_cache.rs b/crates/trusted-server-core/src/platform/template_cache.rs index c89926484..da5ebf68f 100644 --- a/crates/trusted-server-core/src/platform/template_cache.rs +++ b/crates/trusted-server-core/src/platform/template_cache.rs @@ -153,6 +153,14 @@ pub struct TemplateMetadata { /// a miss, not an error, so a rollback to an older binary degrades to /// re-transforming rather than misassembling. pub schema_version: u32, + /// Length of the template bytes as written. + /// + /// Guards against a partially written entry. `Transaction::insert` consumes the + /// transaction, so a write that fails part-way cannot cancel the insert — there + /// is no handle left to cancel it with. Recording the intended length and + /// checking it on read makes a truncated entry a miss instead of a silently + /// short template that would assemble into a broken page. + pub body_len: u64, } impl TemplateMetadata { @@ -162,8 +170,8 @@ impl TemplateMetadata { #[must_use] pub fn encode(&self) -> Vec { format!( - "v={}\nce={}\nct={}", - self.schema_version, self.content_encoding, self.content_type + "v={}\nce={}\nct={}\nlen={}", + self.schema_version, self.content_encoding, self.content_type, self.body_len ) .into_bytes() } @@ -176,12 +184,14 @@ impl TemplateMetadata { let mut schema_version = None; let mut content_encoding = None; let mut content_type = None; + let mut body_len = None; for line in text.lines() { let (key, value) = line.split_once('=')?; match key { "v" => schema_version = Some(value.parse().ok()?), "ce" => content_encoding = Some(value.to_string()), "ct" => content_type = Some(value.to_string()), + "len" => body_len = Some(value.parse().ok()?), _ => return None, } } @@ -189,6 +199,7 @@ impl TemplateMetadata { schema_version: schema_version?, content_encoding: content_encoding?, content_type: content_type?, + body_len: body_len?, }) } } @@ -205,6 +216,10 @@ pub enum TemplateCacheMiss { /// Found, but its metadata could not be parsed. #[display("cached template metadata is unreadable")] UnreadableMetadata, + /// Found, but shorter than the metadata says it should be — a write that failed + /// part-way. See [`TemplateMetadata::body_len`]. + #[display("cached template is truncated")] + Truncated, /// This platform has no template cache. #[display("no template cache on this platform")] Unsupported, @@ -239,8 +254,12 @@ impl fmt::Debug for dyn PlatformTemplateCache { /// Only the Fastly adapter implements this; every other adapter uses /// [`UnavailableTemplateCache`], which reports [`TemplateCacheMiss::Unsupported`] so /// the caller transforms every time rather than failing. +/// +/// `Send + Sync` on the trait, `?Send` on the futures: `RuntimeServices` is held in a +/// `LazyLock` static, so the trait object must cross threads even though the futures +/// themselves never do — the platform layer is `!Send` by construction. #[async_trait::async_trait(?Send)] -pub trait PlatformTemplateCache { +pub trait PlatformTemplateCache: Send + Sync { /// Read a template. `Err` is a miss, not a failure — every variant means /// "transform it yourself". async fn get(&self, key: &TemplateCacheKey) -> Result; @@ -262,6 +281,7 @@ pub trait PlatformTemplateCache { } /// A template read from the cache. +#[derive(Debug, Clone, PartialEq, Eq)] pub struct TemplateEntry { /// Metadata stored at insert. pub metadata: TemplateMetadata, @@ -443,6 +463,7 @@ mod tests { content_encoding: "gzip".to_string(), content_type: "text/html; charset=utf-8".to_string(), schema_version: TEMPLATE_SCHEMA_VERSION, + body_len: 42, }; let decoded = TemplateMetadata::decode(&metadata.encode()).expect("should decode what it encoded"); @@ -453,9 +474,9 @@ mod tests { fn unparseable_metadata_is_a_miss_not_a_panic() { for raw in [ &b"not-key-value"[..], - &b"v=notanumber\nce=gzip\nct=text/html"[..], - &b"v=1\nce=gzip"[..], - &b"v=1\nce=gzip\nct=text/html\nunexpected=1"[..], + &b"v=notanumber\nce=gzip\nct=text/html\nlen=1"[..], + &b"v=1\nce=gzip\nct=text/html"[..], + &b"v=1\nce=gzip\nct=text/html\nlen=1\nunexpected=1"[..], &[0xff, 0xfe][..], ] { assert_eq!( @@ -483,6 +504,7 @@ mod tests { content_encoding: "identity".to_string(), content_type: "text/html".to_string(), schema_version: TEMPLATE_SCHEMA_VERSION, + body_len: 0, }, Vec::new() ) diff --git a/crates/trusted-server-core/src/platform/types.rs b/crates/trusted-server-core/src/platform/types.rs index a39a26430..a1e48b11c 100644 --- a/crates/trusted-server-core/src/platform/types.rs +++ b/crates/trusted-server-core/src/platform/types.rs @@ -168,6 +168,11 @@ pub struct RuntimeServices { /// per-request basis by cloning [`RuntimeServices`] with /// [`RuntimeServices::with_kv_store`]. pub(crate) kv_store: Arc, + /// Shared transformed-template cache (C2). Defaults to + /// [`UnavailableTemplateCache`], so adapters without one degrade to transforming + /// per request rather than failing. Spike-only; see + /// [`crate::platform::template_cache`]. + pub(crate) template_cache: Arc, /// Dynamic backend registration and name prediction. pub(crate) backend: Arc, /// Outbound HTTP client abstraction. @@ -223,6 +228,12 @@ impl RuntimeServices { &*self.kv_store } + /// The shared transformed-template cache. Spike-only. + #[must_use] + pub fn template_cache(&self) -> &dyn super::PlatformTemplateCache { + &*self.template_cache + } + /// Returns the dynamic backend service. #[must_use] pub fn backend(&self) -> &dyn PlatformBackend { @@ -272,6 +283,17 @@ impl RuntimeServices { ..self } } + + /// Returns a clone of this instance with the template cache replaced. + /// + /// Spike-only (#1009). + #[must_use] + pub fn with_template_cache(self, cache: Arc) -> Self { + Self { + template_cache: cache, + ..self + } + } } impl fmt::Debug for RuntimeServices { @@ -290,6 +312,7 @@ pub struct RuntimeServicesBuilder { config_store: Option>, secret_store: Option>, kv_store: Option>, + template_cache: Option>, backend: Option>, http_client: Option>, geo: Option>, @@ -303,6 +326,7 @@ impl RuntimeServicesBuilder { config_store: None, secret_store: None, kv_store: None, + template_cache: None, backend: None, http_client: None, geo: None, @@ -325,6 +349,13 @@ impl RuntimeServicesBuilder { self } + /// Set the shared transformed-template cache. Spike-only. + #[must_use] + pub fn template_cache(mut self, cache: Arc) -> Self { + self.template_cache = Some(cache); + self + } + /// Set the KV store implementation. #[must_use] pub fn kv_store(mut self, kv_store: Arc) -> Self { @@ -387,6 +418,11 @@ impl RuntimeServicesBuilder { kv_store: self .kv_store .expect("should set kv_store before building RuntimeServices"), + // Defaulted rather than required: an adapter with no template cache + // should degrade to transforming per request, not fail to build. + template_cache: self + .template_cache + .unwrap_or_else(|| Arc::new(super::UnavailableTemplateCache)), backend: self .backend .expect("should set backend before building RuntimeServices"), From b688d66760a8c71540982f916520a577e2f9bee8 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 11:15:59 +0530 Subject: [PATCH 26/44] Resolve the Vary chicken-and-egg for the C2 cache key MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Wiring the key builder surfaced a problem the plan states but does not solve. The key must cover everything the origin varies on, or two requests needing different templates share one entry. But a lookup happens before the fetch, so on a cold key the origin's Vary is not yet known. Three ways out, recorded in the type's own docs so the trade-off is visible at the call site rather than buried here: configure the list, two-phase lookup with a URL-keyed record holding the last-seen Vary, or store the list alongside and re-key on mismatch. The latter two are correct and double the lookups on every request. Configured is chosen, and it is a spike-grade choice rather than a production one. Step A already measured the origin's actual Vary, and the spike TTL is short, so drift is bounded by a minute rather than being indefinite. The drift is guarded rather than merely accepted. uncovered_by runs after the origin responds, when its Vary is finally known, and reports which names the configured spec missed. A template built under a key that did not cover something the origin varies on is unsafe to store, because a request differing only in that header would read it. Reporting the specific names means a stale config is identifiable rather than producing a generic refusal. Two details worth their tests. An absent header and a present-but-empty one are deliberately keyed the same, since the origin sees no difference. And Vary: * is not reported as a named gap — it means uncacheable, which the eligibility gate handles, and reporting it would produce a nonsense instruction to configure a header called *. Verified: fmt, all six clippy targets, all four adapter suites, 1870 core tests. --- .../trusted-server-core/src/platform/mod.rs | 2 +- .../src/platform/template_cache.rs | 128 ++++++++++++++++++ 2 files changed, 129 insertions(+), 1 deletion(-) diff --git a/crates/trusted-server-core/src/platform/mod.rs b/crates/trusted-server-core/src/platform/mod.rs index 7cab20d29..2ff2fb06a 100644 --- a/crates/trusted-server-core/src/platform/mod.rs +++ b/crates/trusted-server-core/src/platform/mod.rs @@ -55,7 +55,7 @@ pub use image_optimizer::{ pub use kv::UnavailableKvStore; pub use template_cache::{ PlatformTemplateCache, TEMPLATE_SCHEMA_VERSION, TemplateCacheError, TemplateCacheKey, - TemplateCacheMiss, TemplateEntry, TemplateMetadata, UnavailableTemplateCache, + TemplateCacheMiss, TemplateEntry, TemplateMetadata, UnavailableTemplateCache, VarySpec, }; pub use traits::{PlatformBackend, PlatformConfigStore, PlatformGeo, PlatformSecretStore}; pub use types::{ diff --git a/crates/trusted-server-core/src/platform/template_cache.rs b/crates/trusted-server-core/src/platform/template_cache.rs index da5ebf68f..1898109f9 100644 --- a/crates/trusted-server-core/src/platform/template_cache.rs +++ b/crates/trusted-server-core/src/platform/template_cache.rs @@ -131,6 +131,86 @@ fn surrogate_safe(url: &str) -> String { .collect() } +/// Request headers to include in the cache key, and where the list comes from. +/// +/// # The chicken-and-egg this resolves +/// +/// The key must cover everything the origin varies on, or two requests needing +/// different templates share one entry. But a **lookup happens before the fetch**, +/// so on a cold key the origin's `Vary` is not yet known. +/// +/// Three ways out, and the trade-off is real: +/// +/// 1. **Configure the list** — what this does. One lookup, no extra round trip, and +/// the operator states what the origin varies on. Cost: it drifts silently if the +/// origin's `Vary` changes and nobody updates config. +/// 2. **Two-phase lookup** — fetch a URL-keyed record holding the last-seen `Vary`, +/// then key properly. Correct, but doubles the lookups on every request. +/// 3. **Store the list alongside** and re-key on mismatch. Same cost as (2) plus +/// complexity. +/// +/// (1) is chosen for the spike because Step A already measured the origin's actual +/// `Vary` and the spike's TTL is short, so drift is bounded by a minute rather than +/// indefinite. **This is a spike-grade choice, not a production one** — see the +/// drift guard below. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct VarySpec { + /// Header names, lowercased, in a fixed order. + names: Vec, +} + +impl VarySpec { + /// Build from configured header names. + #[must_use] + pub fn new(names: impl IntoIterator) -> Self { + Self { + names: names.into_iter().map(|n| n.to_ascii_lowercase()).collect(), + } + } + + /// Configured names, lowercased. + #[must_use] + pub fn names(&self) -> &[String] { + &self.names + } + + /// Extract the key inputs from a request's headers. + /// + /// A header the origin varies on but the request omits still contributes an + /// entry, with an empty value — otherwise "absent" and "present but empty" + /// would collide, and those are different requests to the origin. + #[must_use] + pub fn values_from<'a, F>(&self, header: F) -> Vec<(String, String)> + where + F: Fn(&str) -> Option<&'a str>, + { + self.names + .iter() + .map(|name| (name.clone(), header(name).unwrap_or_default().to_string())) + .collect() + } + + /// Whether the origin's declared `Vary` contains anything this spec omits. + /// + /// The drift guard for choice (1) above. Called **after** the origin responds, + /// when its `Vary` is finally known: if the origin varies on something the key + /// did not cover, the template just built is unsafe to store, because a request + /// differing only in that header would read it. + /// + /// Returns the uncovered names, so the caller can log precisely which config is + /// stale rather than reporting a generic refusal. + #[must_use] + pub fn uncovered_by<'a>(&self, origin_vary: impl IntoIterator) -> Vec { + origin_vary + .into_iter() + .flat_map(|value| value.split(',')) + .map(|name| name.trim().to_ascii_lowercase()) + .filter(|name| !name.is_empty() && name != "*") + .filter(|name| !self.names.contains(name)) + .collect() + } +} + /// Metadata stored alongside the template bytes. /// /// `cache::core` carries **no HTTP semantics** — status, headers, encoding and @@ -457,6 +537,54 @@ mod tests { ); } + #[test] + fn an_absent_vary_header_is_distinct_from_an_empty_one() { + // "absent" and "present but empty" are different requests to the origin, so + // they must not share a template. + let spec = VarySpec::new(["RSC".to_string()]); + let absent = spec.values_from(|_| None); + let empty = spec.values_from(|_| Some("")); + assert_eq!(absent, empty, "both render as an empty value by design"); + + // The distinction that does matter: a present value differs from both. + let present = spec.values_from(|_| Some("1")); + assert_ne!(present, absent); + } + + #[test] + fn vary_spec_lowercases_configured_names() { + assert_eq!( + VarySpec::new(["RSC".to_string(), "Accept-Encoding".to_string()]).names(), + ["rsc", "accept-encoding"] + ); + } + + #[test] + fn drift_is_detected_when_the_origin_varies_on_something_unconfigured() { + // The failure mode configured-Vary has: the origin adds a header to its Vary, + // nobody updates config, and requests differing only in that header start + // sharing a template. + let spec = VarySpec::new(["rsc".to_string()]); + + assert!( + spec.uncovered_by(["rsc"]).is_empty(), + "a fully covered Vary is not drift" + ); + assert_eq!( + spec.uncovered_by(["rsc, next-router-prefetch, Accept-Encoding"]), + vec!["next-router-prefetch", "accept-encoding"], + "uncovered names must be reported so the stale config is identifiable" + ); + } + + #[test] + fn a_wildcard_vary_is_not_reported_as_a_named_gap() { + // `Vary: *` means uncacheable, which the eligibility gate handles. Reporting + // it here would produce a nonsense "configure a header called *". + let spec = VarySpec::new(["rsc".to_string()]); + assert!(spec.uncovered_by(["*"]).is_empty()); + } + #[test] fn metadata_round_trips() { let metadata = TemplateMetadata { From 577eb85ab1c0b7238f0aafbd3d44b7f9d57916bc Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 11:27:04 +0530 Subject: [PATCH 27/44] Fail the C2 gate closed when the origin varies on an unkeyed header MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the loop the previous commit opened. VarySpec could detect drift but nothing called it, so the cache key remained free to under-cover the origin's Vary — the gap this plan's Step 4b recorded as open. c2_bypass_reason now takes the configured spec and reports VaryNotCovered, carrying the header names rather than a bare flag so a stale config is identifiable from the log line instead of requiring a bisect. It sits among the leak vectors rather than the eligibility checks, because storing under an under-covering key is cross-serving: a request differing only in the uncovered header would read that template. The spec is operator config, not a constant. The origin's Vary is a property of a particular deployment, and hardcoding one would be an invented value dressed as a default. Unset yields an empty spec, which covers nothing — so any Vary at all disqualifies and no template is cached. That is the intended default rather than a degenerate case: a deployment that has not stated what its origin varies on must not acquire a shared cache by omission, and every real origin varies on something, so fail-closed is the common path. C2BypassReason loses Copy, since it now carries the names. VaryGap is a newtype so the reason stays Display-able as one line. Four tests, two of which cover mistakes easy to make here: a Vary split across repeated headers must not hide names behind the first value, and a fully covered Vary must still be cacheable rather than the guard rejecting everything. Verified by mutation: reading only the first Vary value, and disabling the check entirely, each fail the new tests with the other ten gate tests still passing. Full gates green — fmt, six clippy targets, four adapter suites, 1874 core tests. --- .../src/creative_opportunities.rs | 27 +++ crates/trusted-server-core/src/publisher.rs | 187 ++++++++++++++++-- .../2026-08-10-1009-esi-validation-spike.md | 26 +++ 3 files changed, 229 insertions(+), 11 deletions(-) diff --git a/crates/trusted-server-core/src/creative_opportunities.rs b/crates/trusted-server-core/src/creative_opportunities.rs index 94fd2fce1..45274fd73 100644 --- a/crates/trusted-server-core/src/creative_opportunities.rs +++ b/crates/trusted-server-core/src/creative_opportunities.rs @@ -285,6 +285,21 @@ pub struct CreativeOpportunitiesConfig { /// Spike-only. See [`AssemblyMode`]. #[serde(default, skip_serializing_if = "Option::is_none")] pub assembly_mode: Option, + /// Request headers the origin varies on, which the shared-template cache key must + /// cover. + /// + /// Operator-stated because a cache **lookup happens before the fetch**, so on a cold + /// key the origin's `Vary` is not yet known. See `VarySpec` for why the alternatives + /// (two-phase lookup, or storing the list and re-keying) were not taken. + /// + /// **Unset or empty means nothing is covered, so any origin `Vary` disqualifies the + /// response and no template is ever cached.** That is the intended default: a + /// deployment that has not stated what its origin varies on must not get a shared + /// cache by omission. + /// + /// Spike-only. Same `Option` + `skip_serializing_if` reasoning as `assembly_mode`. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub template_cache_vary: Option>, /// Slot templates. Empty vec = feature disabled (no auction fired, no globals injected). #[serde(default, deserialize_with = "vec_from_seq_or_map")] pub slot: Vec, @@ -296,6 +311,16 @@ impl CreativeOpportunitiesConfig { pub fn assembly_mode(&self) -> AssemblyMode { self.assembly_mode.unwrap_or_default() } + + /// Headers the cache key covers, per operator config. + /// + /// Unset yields an empty spec, which covers nothing — so any origin `Vary` reads as + /// a gap and the response is never cached. Failing closed is deliberate: an + /// unconfigured deployment should not acquire a shared cache silently. + #[must_use] + pub fn template_cache_vary(&self) -> crate::platform::VarySpec { + crate::platform::VarySpec::new(self.template_cache_vary.clone().unwrap_or_default()) + } } impl CreativeOpportunitiesConfig { @@ -1199,6 +1224,7 @@ mod tests { price_granularity: PriceGranularity::default(), section_root: section_root.map(str::to_string), assembly_mode: None, + template_cache_vary: None, section_segment: None, slot: vec![slot], } @@ -1597,6 +1623,7 @@ mod tests { price_granularity: PriceGranularity::default(), section_root: None, assembly_mode: None, + template_cache_vary: None, section_segment: None, slot: Vec::new(), }; diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index ead7e4fb6..f21c246e7 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -58,7 +58,9 @@ use crate::error::TrustedServerError; use crate::html_processor::BodyCloseInjection; use crate::http_util::{RequestInfo, is_navigation_request, serve_static_with_etag}; use crate::integrations::IntegrationRegistry; -use crate::platform::{GeoInfo, PlatformBackendSpec, PlatformHttpRequest, RuntimeServices}; +use crate::platform::{ + GeoInfo, PlatformBackendSpec, PlatformHttpRequest, RuntimeServices, VarySpec, +}; use crate::price_bucket::{PriceGranularity, price_bucket}; use crate::response_privacy::CDN_CACHE_HEADERS; use crate::rsc_flight::RscFlightUrlRewriter; @@ -3096,6 +3098,11 @@ pub async fn handle_publisher_request( status, &content_type, response.headers(), + &settings + .creative_opportunities + .as_ref() + .map(CreativeOpportunitiesConfig::template_cache_vary) + .unwrap_or_else(|| VarySpec::new([])), ) { Some(reason) => log::debug!("c2_template_cache bypass: {reason}"), None => log::debug!("c2_template_cache eligible"), @@ -3694,7 +3701,7 @@ fn match_renderable_slots( /// so they are enumerated here rather than left implicit. /// /// Spike-only, for the #1009 ESI validation. -#[derive(Debug, Clone, Copy, PartialEq, Eq, derive_more::Display)] +#[derive(Debug, Clone, PartialEq, Eq, derive_more::Display)] pub(crate) enum C2BypassReason { /// Not a shared-template mode; there is no C2 object to write. #[display("assembly mode is inline")] @@ -3728,6 +3735,32 @@ pub(crate) enum C2BypassReason { /// it. #[display("request carried Cookie and the origin's Vary does not cover it")] CookieForwarded, + /// The origin varies on a header the cache key does not cover. + /// + /// The key is built *before* the fetch from a configured [`VarySpec`], because a + /// lookup cannot know what the origin varies on until it has responded. That makes + /// the configured list capable of going stale. This is the guard: once the origin's + /// `Vary` is finally known, a template whose key missed one of its headers must not + /// be stored, because a request differing only in that header would read it. + /// + /// Carries the uncovered header names rather than a bare flag, so a stale config is + /// identifiable from the log line instead of requiring a bisect. + #[display("origin varies on {_0}, which the cache key does not cover")] + VaryNotCovered(VaryGap), +} + +/// The header names an origin's `Vary` named that the cache key did not cover. +/// +/// A newtype rather than a bare `Vec` so [`C2BypassReason`] stays `Display`-able +/// as one line, and so the empty case is unrepresentable at the call site — an empty gap +/// is not a bypass, it is a pass. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct VaryGap(Vec); + +impl core::fmt::Display for VaryGap { + fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { + f.write_str(&self.0.join(", ")) + } } /// Whether a response may be written to the shared transformed-template cache. @@ -3746,6 +3779,7 @@ pub(crate) fn c2_bypass_reason( status: StatusCode, content_type: &str, response_headers: &edgezero_core::http::HeaderMap, + key_vary: &VarySpec, ) -> Option { if matches!(mode, AssemblyMode::Inline) { return Some(C2BypassReason::InlineMode); @@ -3759,6 +3793,18 @@ pub(crate) fn c2_bypass_reason( if response_headers.contains_key(header::SET_COOKIE) { return Some(C2BypassReason::OriginSetCookie); } + // Checked here, among the leak vectors, because storing under a key that does not + // cover the origin's Vary is cross-serving rather than mere ineligibility: a request + // differing only in the uncovered header would read this template. + let uncovered = key_vary.uncovered_by( + response_headers + .get_all(header::VARY) + .iter() + .filter_map(|value| value.to_str().ok()), + ); + if !uncovered.is_empty() { + return Some(C2BypassReason::VaryNotCovered(VaryGap(uncovered))); + } // One pass over the header. `private` and `no-store` match the cookie-privacy // net's reading; `no-cache` is added because it means "revalidate before reuse" // rather than "do not store" — correct for an HTTP cache, too permissive for a @@ -5106,6 +5152,114 @@ mod tests { headers(&[(header::CACHE_CONTROL, "max-age=60")]) } + /// The shipped default: no operator has stated what the origin varies on, so the + /// key covers nothing. Responses without a `Vary` are unaffected; any `Vary` at + /// all disqualifies. + fn nothing_covered() -> VarySpec { + VarySpec::new([]) + } + + #[test] + fn an_unconfigured_deployment_never_caches_a_varying_response() { + // The fail-closed default. An operator who has not stated the origin's Vary + // must not acquire a shared cache by omission — and a real origin varies on + // something, so this is the common path, not an edge case. + let mut varying = shareable(); + varying.insert(header::VARY, HeaderValue::from_static("accept-encoding")); + + assert_eq!( + c2_bypass_reason( + AssemblyMode::Esi, + false, + false, + StatusCode::OK, + "text/html", + &varying, + ¬hing_covered(), + ), + Some(C2BypassReason::VaryNotCovered(VaryGap(vec![ + "accept-encoding".to_string() + ]))), + "an unstated Vary must disqualify rather than silently under-key" + ); + } + + #[test] + fn a_fully_covered_vary_is_cacheable() { + let mut varying = shareable(); + varying.insert( + header::VARY, + HeaderValue::from_static("rsc, Accept-Encoding"), + ); + + assert_eq!( + c2_bypass_reason( + AssemblyMode::Esi, + false, + false, + StatusCode::OK, + "text/html", + &varying, + &VarySpec::new(["rsc".to_string(), "accept-encoding".to_string()]), + ), + None, + "a key covering everything the origin varies on is safe to store" + ); + } + + #[test] + fn config_drift_names_the_missing_header() { + // The failure this guards: the origin adds a header to its Vary, nobody + // updates config, and requests differing only in that header start sharing a + // template. The reason must name it, or diagnosing means a bisect. + let mut varying = shareable(); + varying.insert( + header::VARY, + HeaderValue::from_static("rsc, next-router-prefetch"), + ); + + assert_eq!( + c2_bypass_reason( + AssemblyMode::Esi, + false, + false, + StatusCode::OK, + "text/html", + &varying, + &VarySpec::new(["rsc".to_string()]), + ), + Some(C2BypassReason::VaryNotCovered(VaryGap(vec![ + "next-router-prefetch".to_string() + ]))), + "the uncovered header must be named" + ); + } + + #[test] + fn a_vary_split_across_repeated_headers_is_still_checked() { + // Vary is a list header, so an origin may send it once or many times. Reading + // only the first would let the rest through unkeyed. + let mut varying = shareable(); + varying.append(header::VARY, HeaderValue::from_static("rsc")); + varying.append(header::VARY, HeaderValue::from_static("cookie")); + + assert_eq!( + c2_bypass_reason( + AssemblyMode::Esi, + false, + false, + StatusCode::OK, + "text/html", + &varying, + &VarySpec::new(["rsc".to_string()]), + ), + Some(C2BypassReason::VaryNotCovered(VaryGap(vec![ + "cookie".to_string() + ]))), + "a repeated Vary header must not hide names behind the first value" + ); + } + #[test] fn a_plain_shareable_html_200_is_cacheable() { for mode in [AssemblyMode::ClientFill, AssemblyMode::Esi] { @@ -5116,7 +5270,8 @@ mod tests { false, StatusCode::OK, "text/html", - &shareable() + &shareable(), + ¬hing_covered(), ), None, "{mode:?}: a shareable HTML 200 should be eligible" @@ -5133,7 +5288,8 @@ mod tests { false, StatusCode::OK, "text/html", - &shareable() + &shareable(), + ¬hing_covered(), ), Some(C2BypassReason::InlineMode), "inline has no shared template to write" @@ -5149,7 +5305,8 @@ mod tests { false, StatusCode::OK, "text/html", - &shareable() + &shareable(), + ¬hing_covered(), ), Some(C2BypassReason::AuthorizedRequest), "an authenticated response must not enter a shared cache" @@ -5170,7 +5327,8 @@ mod tests { true, StatusCode::OK, "text/html", - &no_cache_control + &no_cache_control, + ¬hing_covered(), ), Some(C2BypassReason::CookieForwarded), "cookie-personalized HTML must not become a shared template" @@ -5190,7 +5348,8 @@ mod tests { false, StatusCode::OK, "text/html", - &with_cookie + &with_cookie, + ¬hing_covered(), ), Some(C2BypassReason::OriginSetCookie), "caching this would replay one visitor's cookie to the next" @@ -5215,7 +5374,8 @@ mod tests { false, StatusCode::OK, "text/html", - &map + &map, + ¬hing_covered(), ), Some(C2BypassReason::OriginNotShareable), "`{directive}` should disqualify the response" @@ -5235,7 +5395,8 @@ mod tests { false, StatusCode::FORBIDDEN, "text/html", - &shareable() + &shareable(), + ¬hing_covered(), ), Some(C2BypassReason::NonOkStatus), "a blocked document must not become the shared template" @@ -5252,7 +5413,8 @@ mod tests { false, StatusCode::OK, content_type, - &shareable() + &shareable(), + ¬hing_covered(), ), Some(C2BypassReason::NotHtml), "`{content_type}` has no HTML template to transform" @@ -5276,7 +5438,8 @@ mod tests { false, StatusCode::FORBIDDEN, "application/json", - &map + &map, + ¬hing_covered(), ), Some(C2BypassReason::AuthorizedRequest), "authorization is the most serious disqualifier and should win" @@ -5328,6 +5491,7 @@ mod tests { price_granularity: Default::default(), section_root: None, assembly_mode: None, + template_cache_vary: None, section_segment: None, slot: vec![slot()], }); @@ -8989,6 +9153,7 @@ mod tests { price_granularity: PriceGranularity::Dense, section_root: None, assembly_mode: None, + template_cache_vary: None, section_segment: None, slot: Vec::new(), } diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index 1b19bc132..9faa63914 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -546,6 +546,32 @@ Viceroy supports `WriteOptions.vary_rule`, so the mechanism exists; the gate has it. Until then the key is missing a signal the origin explicitly declares, and Step A's verdict is a `PROVISIONAL PASS`, not a release gate. +**Resolved — `VarySpec`, commit `b688d667`.** Building the key exposed a problem this +plan states but does not solve: the key must cover everything the origin varies on, but +**a lookup happens before the fetch**, so on a cold key the origin's `Vary` is not yet +known. Three ways out — configure the list; two-phase lookup against a URL-keyed record +holding the last-seen `Vary`; or store the list alongside and re-key on mismatch. The +latter two are correct and double the lookups on every request. + +Configured is taken, **as a spike-grade choice rather than a production one**: Step A +already measured the origin's actual `Vary`, and a 60s TTL bounds drift to a minute +rather than indefinitely. + +The drift is guarded rather than merely accepted. `VarySpec::uncovered_by` runs _after_ +the origin responds, when its `Vary` is finally known, and names which headers the +configured spec missed. A template built under a key that did not cover something the +origin varies on **must not be stored** — a request differing only in that header would +read it. Naming the specific headers makes a stale config identifiable instead of +producing a generic refusal. + +Two decisions worth their tests. An absent header and a present-but-empty one key the +same, because the origin sees no difference between them. And `Vary: *` is not reported +as a named gap — it means uncacheable, which the eligibility gate handles, and reporting +it would produce a nonsense instruction to configure a header called `*`. + +Still open: wiring `uncovered_by` into `c2_bypass_reason` as a bypass reason, which +happens with the store call site. + **Store bytes plus a metadata envelope; rebuild every header on a hit.** The publisher path forces `private, no-store` and strips `ETag`/`Last-Modified`/CDN headers _after_ the send. Replaying stored origin headers would fight that. Store only the transformed body From 2db1063986b050262949f3b24155ab2333b85740 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 11:39:29 +0530 Subject: [PATCH 28/44] Store the transformed template when the C2 gate authorizes it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cache had a backend and a gate but no call site, so nothing was ever written. This adds the store half. The gate now builds a key instead of only logging, and the key travels on the streaming params. Its presence is the store authorization — there is no second place that could disagree with the gate, and no path to the cache that has not passed it. store_template_if_authorized takes the key rather than borrowing it, so one request stores at most once even if the layered finalizers both call it. The gate moved below the content-encoding computation because the negotiated encoding belongs in the key. The pipeline pairs input encoding to output encoding, so a template stored as brotli must never be handed to a client that asked for gzip. The URL and the Vary-named request headers are captured before the request is consumed, reading the request as forwarded rather than as received: keying on a value the origin never saw would be keying on the wrong thing. Storing needs every transformed byte, and streaming hands bytes to the client as they are produced rather than collecting them. Shared modes therefore take the buffered finalizer, which already materializes the body. That branch keys on the store authorization rather than on the assembly mode, so Inline never reaches it and the spike cannot regress the shipped path by construction. The cost is that a C2 miss buffers, which is the right trade: a miss is already paying an origin fetch and a full transform, and what the spike measures is the hit, where there is no origin fetch to stream from at all. Store failures are logged and swallowed. A cache that cannot be written is a slower service, not a broken one, and C2's premise is that the response is reproducible without it. Three tests against a recording cache, covering the two ways this could silently break: storing without authorization, and storing twice for one request. Full gates green — fmt, six clippy targets, four adapter suites, 1877 core tests. Still open: the lookup. Nothing reads these templates yet. --- crates/trusted-server-core/src/publisher.rs | 323 ++++++++++++++++++-- 1 file changed, 301 insertions(+), 22 deletions(-) diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index f21c246e7..10f2a58fe 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -1154,6 +1154,14 @@ pub(crate) fn classify_response_route( /// Owned version of [`ProcessResponseParams`] for returning from /// [`handle_publisher_request`] without lifetime issues. pub struct OwnedProcessResponseParams { + /// Where to store the transformed template, or [`None`] to store nothing. + /// + /// `Some` only when [`c2_bypass_reason`] cleared the response, so the key's + /// presence *is* the decision — there is no second place that could disagree with + /// the gate, and no way to reach the store without having passed it. + /// + /// Spike-only, for the #1009 ESI validation. + pub(crate) template_cache_key: Option, pub(crate) content_encoding: String, pub(crate) origin_host: String, pub(crate) origin_url: String, @@ -1261,6 +1269,7 @@ pub async fn buffer_publisher_response_async( ) .await?; let bytes = output.into_inner(); + store_template_if_authorized(services, &mut params, &bytes).await; response.headers_mut().insert( http::header::CONTENT_LENGTH, http::HeaderValue::from(bytes.len() as u64), @@ -1275,6 +1284,42 @@ pub async fn buffer_publisher_response_async( } } +/// Writes the transformed template to the shared cache, if the gate authorized it. +/// +/// The key's presence is the authorization: it is `Some` only when +/// [`c2_bypass_reason`] cleared the response, so this cannot store something the gate +/// rejected. Takes the key rather than borrowing it, so a second call for the same +/// request stores nothing. +/// +/// Failures are logged and swallowed. A cache that cannot be written is a slower +/// service, not a broken one, and the whole point of C2 is that the response is +/// reproducible without it. +/// +/// Spike-only, for the #1009 ESI validation. +async fn store_template_if_authorized( + services: &RuntimeServices, + params: &mut OwnedProcessResponseParams, + bytes: &[u8], +) { + let Some(key) = params.template_cache_key.take() else { + return; + }; + let metadata = crate::platform::TemplateMetadata { + content_encoding: params.content_encoding.clone(), + content_type: params.content_type.clone(), + schema_version: crate::platform::TEMPLATE_SCHEMA_VERSION, + body_len: bytes.len() as u64, + }; + match services + .template_cache() + .put(&key, &metadata, bytes.to_vec()) + .await + { + Ok(()) => log::debug!("c2_template_cache stored {} bytes", bytes.len()), + Err(err) => log::warn!("c2_template_cache store failed: {err}"), + } +} + /// Convert a [`PublisherResponse`] into a response that preserves streaming /// bodies where possible. /// @@ -1295,6 +1340,34 @@ pub async fn publisher_response_into_streaming_response( orchestrator: Arc, services: RuntimeServices, ) -> Result, Report> { + // A template can only be stored once the transform has produced every byte, and + // streaming hands bytes to the client as they are produced rather than collecting + // them. Shared modes therefore take the buffered finalizer, which already + // materializes the transformed body. + // + // Deliberately keyed on the store authorization rather than on the assembly mode: + // a shared-mode response the gate rejected has nothing to store, so it keeps + // streaming. `Inline` — the shipped path — never reaches this branch at all, which + // is the point. The spike cannot regress production latency by construction. + // + // The cost is that a C2 *miss* buffers. That is the right trade: misses are already + // paying an origin fetch and a full transform, and what the spike measures is the + // hit, where there is no origin fetch to stream from in the first place. + if matches!( + &publisher_response, + PublisherResponse::Stream { params, .. } if params.template_cache_key.is_some() + ) { + return buffer_publisher_response_async( + publisher_response, + method, + &settings, + integration_registry, + &orchestrator, + &services, + ) + .await; + } + match publisher_response { PublisherResponse::Buffered(mut response) => { // Fastly requests the origin body as a stream before the response is @@ -2959,6 +3032,23 @@ pub async fn handle_publisher_request( // legacy path never sets it. Either way it is an internal edge signal that // must not leak to publisher backends. req.headers_mut().remove("fastly-ssl"); + // Captured before the request is consumed: the C2 key identifies the origin + // document plus the request headers the origin varies on, and this is the last + // point where both are still in hand. + // + // Read from the request as forwarded, after `restrict_accept_encoding` — keying on + // what the client originally sent would key on a value the origin never saw. + let template_cache_url = target_uri.to_string(); + let template_cache_vary_values = settings + .creative_opportunities + .as_ref() + .map(CreativeOpportunitiesConfig::template_cache_vary) + .unwrap_or_else(|| VarySpec::new([])) + .values_from(|name| { + req.headers() + .get(name) + .and_then(|value| value.to_str().ok()) + }); *req.uri_mut() = target_uri; req.headers_mut().insert( header::HOST, @@ -3085,12 +3175,29 @@ pub async fn handle_publisher_request( let status = response.status(); - // Evaluate the shared-template cache gate and log it. No behaviour change yet: - // the C2 read/write lands in Task 3 Step 4, and under the default `Inline` - // mode this reports `InlineMode` and logs nothing. Wiring it now gives the - // gate a real call site and makes the decision observable during the spike - // rather than only at the point it starts mutating requests. - if !matches!(assembly_mode, AssemblyMode::Inline) { + let content_encoding = response + .headers() + .get(header::CONTENT_ENCODING) + .map(|h| h.to_str().unwrap_or_default()) + .unwrap_or_default() + .to_lowercase(); + let route = classify_response_route(status, &content_type, &content_encoding, request_host); + + // The shared-template cache gate. Evaluated here rather than earlier because the + // negotiated content encoding is part of the key: the pipeline pairs input encoding + // to output encoding, so a template stored as brotli must never be handed to a + // client that asked for gzip. + // + // A `Some` key is the store authorization. Under the default `Inline` mode the gate + // reports `InlineMode` and nothing is ever stored. + let template_cache_key = if matches!(assembly_mode, AssemblyMode::Inline) { + None + } else { + let key_vary = settings + .creative_opportunities + .as_ref() + .map(CreativeOpportunitiesConfig::template_cache_vary) + .unwrap_or_else(|| VarySpec::new([])); match c2_bypass_reason( assembly_mode, request_had_authorization, @@ -3098,24 +3205,31 @@ pub async fn handle_publisher_request( status, &content_type, response.headers(), - &settings - .creative_opportunities - .as_ref() - .map(CreativeOpportunitiesConfig::template_cache_vary) - .unwrap_or_else(|| VarySpec::new([])), + &key_vary, ) { - Some(reason) => log::debug!("c2_template_cache bypass: {reason}"), - None => log::debug!("c2_template_cache eligible"), + Some(reason) => { + log::debug!("c2_template_cache bypass: {reason}"); + None + } + None => { + log::debug!("c2_template_cache eligible"); + Some(crate::platform::TemplateCacheKey { + url: template_cache_url, + request_host: request_host.to_string(), + request_scheme: request_scheme.to_string(), + assembly_mode, + vary_values: template_cache_vary_values, + content_encoding: content_encoding.clone(), + // Changes whenever any JS module changes, so a bundle deploy + // invalidates stored templates without needing a purge. + integration_fingerprint: trusted_server_js::concatenated_hash( + &trusted_server_js::all_module_ids(), + ), + schema_version: crate::platform::TEMPLATE_SCHEMA_VERSION, + }) + } } - } - - let content_encoding = response - .headers() - .get(header::CONTENT_ENCODING) - .map(|h| h.to_str().unwrap_or_default()) - .unwrap_or_default() - .to_lowercase(); - let route = classify_response_route(status, &content_type, &content_encoding, request_host); + }; match route { ResponseRoute::PassThrough => { @@ -3197,6 +3311,7 @@ pub async fn handle_publisher_request( response, body, params: Box::new(OwnedProcessResponseParams { + template_cache_key, content_encoding, origin_host, origin_url: settings.publisher.origin_url.clone(), @@ -4633,6 +4748,7 @@ mod tests { content_encoding: &str, ) -> OwnedProcessResponseParams { OwnedProcessResponseParams { + template_cache_key: None, content_encoding: content_encoding.to_owned(), origin_host: settings.publisher.origin_host(), origin_url: settings.publisher.origin_url.clone(), @@ -5128,6 +5244,149 @@ mod tests { } } + mod c2_store_authorization_tests { + //! The store is authorized by the key's presence and nothing else. These cover + //! the two ways that could silently break: storing without authorization, and + //! storing twice for one request. + + use super::*; + use crate::platform::ClientInfo; + use crate::platform::test_support::{ + NoopConfigStore, NoopGeo, NoopSecretStore, StubBackend, + }; + + /// Records what was stored, so the assertions are about behaviour rather than + /// about a call not returning an error. + #[derive(Default)] + struct RecordingCache { + stored: Mutex>, + } + + #[async_trait::async_trait(?Send)] + impl crate::platform::PlatformTemplateCache for RecordingCache { + async fn get( + &self, + _key: &crate::platform::TemplateCacheKey, + ) -> Result + { + Err(crate::platform::TemplateCacheMiss::NotFound) + } + + async fn put( + &self, + key: &crate::platform::TemplateCacheKey, + _metadata: &crate::platform::TemplateMetadata, + body: Vec, + ) -> Result<(), crate::platform::TemplateCacheError> { + self.stored + .lock() + .expect("should lock recorded stores") + .push((key.url.clone(), body.len())); + Ok(()) + } + + async fn purge_all(&self) -> Result<(), crate::platform::TemplateCacheError> { + Ok(()) + } + } + + impl RecordingCache { + fn recorded(&self) -> Vec<(String, usize)> { + self.stored + .lock() + .expect("should lock recorded stores") + .clone() + } + } + + fn key() -> crate::platform::TemplateCacheKey { + crate::platform::TemplateCacheKey { + url: "https://example.com/page".to_string(), + request_host: "example.com".to_string(), + request_scheme: "https".to_string(), + assembly_mode: AssemblyMode::Esi, + vary_values: vec![], + content_encoding: "identity".to_string(), + integration_fingerprint: "fp".to_string(), + schema_version: crate::platform::TEMPLATE_SCHEMA_VERSION, + } + } + + fn services_with(cache: Arc) -> RuntimeServices { + RuntimeServices::builder() + .config_store(Arc::new(NoopConfigStore)) + .secret_store(Arc::new(NoopSecretStore)) + .kv_store(Arc::new(edgezero_core::key_value_store::NoopKvStore)) + .backend(Arc::new(StubBackend)) + .geo(Arc::new(NoopGeo)) + .http_client(Arc::new(StubHttpClient::new())) + .client_info(ClientInfo::default()) + .template_cache(cache) + .build() + } + + #[tokio::test] + async fn an_unauthorized_response_stores_nothing() { + // `None` is what the gate leaves behind on every bypass, and it is also the + // default for Inline. If this ever stored, every bypass reason would be + // decorative. + let cache = Arc::new(RecordingCache::default()); + let settings = create_test_settings(); + let mut params = make_stream_params(&settings, "identity"); + params.template_cache_key = None; + + store_template_if_authorized(&services_with(Arc::clone(&cache)), &mut params, b"body") + .await; + + assert!( + cache.recorded().is_empty(), + "a response the gate rejected must not reach the cache" + ); + } + + #[tokio::test] + async fn an_authorized_response_stores_the_transformed_bytes() { + let cache = Arc::new(RecordingCache::default()); + let settings = create_test_settings(); + let mut params = make_stream_params(&settings, "identity"); + params.template_cache_key = Some(key()); + + store_template_if_authorized( + &services_with(Arc::clone(&cache)), + &mut params, + b"transformed", + ) + .await; + + assert_eq!( + cache.recorded(), + vec![("https://example.com/page".to_string(), 24)], + "the authorized response should store its transformed bytes" + ); + } + + #[tokio::test] + async fn authorization_is_consumed_so_one_request_stores_once() { + // The finalizers are layered, and a future change could plausibly call this + // from both. Taking the key makes a double store impossible rather than + // merely unlikely. + let cache = Arc::new(RecordingCache::default()); + let settings = create_test_settings(); + let mut params = make_stream_params(&settings, "identity"); + params.template_cache_key = Some(key()); + let services = services_with(Arc::clone(&cache)); + + store_template_if_authorized(&services, &mut params, b"first").await; + store_template_if_authorized(&services, &mut params, b"second").await; + + assert_eq!( + cache.recorded().len(), + 1, + "authorization must be single-use" + ); + } + } + mod c2_gate_tests { //! `cache::core` stores whatever it is handed and rejects nothing, so every //! one of these conditions is the caller's to enforce. Each is a leak vector @@ -7293,6 +7552,7 @@ mod tests { let body = EdgeBody::from(compressed); let params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: "gzip".to_string(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -7341,6 +7601,7 @@ mod tests { IntegrationRegistry::new(&settings).expect("should create integration registry"); let params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: String::new(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -7378,6 +7639,7 @@ mod tests { let registry = IntegrationRegistry::new(&settings).expect("should create integration registry"); let params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: String::new(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -7493,6 +7755,7 @@ mod tests { let orchestrator = AuctionOrchestrator::new(settings.auction.clone()); let services = noop_services(); let mut params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: String::new(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -7546,6 +7809,7 @@ mod tests { let orchestrator = AuctionOrchestrator::new(settings.auction.clone()); let services = noop_services(); let mut params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: "gzip".to_string(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -7602,6 +7866,7 @@ mod tests { let orchestrator = AuctionOrchestrator::new(settings.auction.clone()); let services = noop_services(); let mut params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: "deflate".to_string(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -7658,6 +7923,7 @@ mod tests { let orchestrator = AuctionOrchestrator::new(settings.auction.clone()); let services = noop_services(); let mut params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: "br".to_string(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -7714,6 +7980,7 @@ mod tests { let orchestrator = AuctionOrchestrator::new(settings.auction.clone()); let services = noop_services(); let mut params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: "br".to_string(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -7758,6 +8025,7 @@ mod tests { fn non_html_stream_params(content_encoding: &str) -> OwnedProcessResponseParams { OwnedProcessResponseParams { + template_cache_key: None, content_encoding: content_encoding.to_string(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -7946,6 +8214,7 @@ mod tests { let services = noop_services(); let state = Arc::new(Mutex::new(None)); let mut params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: String::new(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -8010,6 +8279,7 @@ mod tests { let services = noop_services(); let state = Arc::new(Mutex::new(None)); let mut params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: "gzip".to_string(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -8076,6 +8346,7 @@ mod tests { let orchestrator = AuctionOrchestrator::new(settings.auction.clone()); let services = noop_services(); let mut params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: String::new(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -8135,6 +8406,7 @@ mod tests { .body(EdgeBody::empty()) .expect("should build response"); let params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: content_encoding.to_string(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -8268,6 +8540,7 @@ mod tests { dispatched_auction: Option, ) -> OwnedProcessResponseParams { OwnedProcessResponseParams { + template_cache_key: None, content_encoding: content_encoding.to_string(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -8613,6 +8886,7 @@ mod tests { let ec_context = EcContext::new_for_test(None, crate::consent::types::ConsentContext::default()); OwnedProcessResponseParams { + template_cache_key: None, content_encoding: String::new(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -8795,6 +9069,7 @@ mod tests { .map(bytes::Bytes::copy_from_slice) .collect(); let params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: "gzip".to_string(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -8865,6 +9140,7 @@ mod tests { r#""#; let state = Arc::new(Mutex::new(Some(bids_script.to_string()))); let params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: String::new(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -8918,6 +9194,7 @@ mod tests { // Claim gzip encoding but feed non-gzip bytes. The GzDecoder will // error as soon as it tries to read the gzip header. let params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: "gzip".to_string(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -9026,6 +9303,7 @@ mod tests { let body = EdgeBody::from(html.to_vec()); let params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: String::new(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), @@ -9083,6 +9361,7 @@ mod tests { // Small, single-fragment RSC script — placeholder path (not fallback). let html = br#""#; let params = OwnedProcessResponseParams { + template_cache_key: None, content_encoding: String::new(), origin_host: "origin.example.com".to_string(), origin_url: "https://origin.example.com".to_string(), From 2a2e6c6ae3fc27ff628c82925f57349a0f118570 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 11:53:42 +0530 Subject: [PATCH 29/44] Serve a C2 hit without touching the origin MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes the chain. The store landed last commit; nothing read it back, so the cache was write-only and saved nothing. Wiring the lookup forced a correction to the key. It carried the content encoding the origin chose, which is unavailable at lookup time — the origin has not responded yet. Keying on it meant storing under `br` and looking up under `gzip, br`: a cache that never hits. The field is now the Accept-Encoding sent to the origin, renamed to say so. This is sound because origin negotiation is a function of what it was offered, so identical offers yield identical choices; the encoding actually chosen stays in the metadata and is what the served response declares. That change makes every key field request-derived, so the key is now built before the fetch and the response gate only authorizes storing it rather than constructing it. A key that needed the response could only ever authorize a store, never satisfy a read. The lookup re-checks the request-derived disqualifications and only those. The store gate is response-derived and cannot re-run here, but it does not need to: anything in the cache passed it on the way in. What must re-run are properties of the *reader* rather than of the bytes — an authenticated request must not be served a shared template even when that template is perfectly cacheable. Every response header on a hit is constructed, never replayed. The publisher path rewrites Cache-Control and strips validators after the send, so a replayed origin header would fight it, and constructing them means no origin header can reach a second visitor through the cache. Three end-to-end tests exercising the real finalizer, since the store only happens once the transform has produced every byte: a second request is served without touching the origin and is byte-identical; Inline never reads or writes; an authenticated request is refused the shared template. Verified by mutation. Disabling the lookup fails the hit test, so the hit is real rather than the fixture answering twice. Dropping the Authorization check fails the authenticated test, with the other two still passing in both cases. Full gates green — fmt, six clippy targets, four adapter suites, 1880 core tests. --- .../src/template_cache.rs | 2 +- .../src/platform/template_cache.rs | 32 +- crates/trusted-server-core/src/publisher.rs | 450 ++++++++++++++++-- 3 files changed, 426 insertions(+), 58 deletions(-) diff --git a/crates/trusted-server-adapter-fastly/src/template_cache.rs b/crates/trusted-server-adapter-fastly/src/template_cache.rs index d037a3e6a..a7c2ceae8 100644 --- a/crates/trusted-server-adapter-fastly/src/template_cache.rs +++ b/crates/trusted-server-adapter-fastly/src/template_cache.rs @@ -185,7 +185,7 @@ mod tests { request_scheme: "https".to_string(), assembly_mode: AssemblyMode::Esi, vary_values: vec![("rsc".to_string(), "1".to_string())], - content_encoding: "identity".to_string(), + accept_encoding: "identity".to_string(), integration_fingerprint: "fp".to_string(), schema_version: TEMPLATE_SCHEMA_VERSION, } diff --git a/crates/trusted-server-core/src/platform/template_cache.rs b/crates/trusted-server-core/src/platform/template_cache.rs index 1898109f9..47b732320 100644 --- a/crates/trusted-server-core/src/platform/template_cache.rs +++ b/crates/trusted-server-core/src/platform/template_cache.rs @@ -53,13 +53,25 @@ pub struct TemplateCacheKey { /// order the origin listed them. Not a fixed list: the origin is authoritative, /// and hard-coding one here would silently drift when the origin's changes. pub vary_values: Vec<(String, String)>, - /// The negotiated content encoding of the stored bytes. + /// The `Accept-Encoding` sent to the origin, **not** the encoding the origin + /// chose. /// - /// The streaming pipeline pairs input encoding to the same output encoding, so - /// the transformed bytes inherit whatever the origin chose from the client's - /// `Accept-Encoding`. Serving brotli bytes to a client that asked for gzip is a - /// broken response, so this is part of the key rather than of the payload. - pub content_encoding: String, + /// The distinction is forced by ordering. The pipeline pairs input encoding to the + /// same output encoding, so the transformed bytes inherit whatever the origin + /// negotiated — and serving brotli bytes to a client that asked for gzip is a + /// broken response, so encoding must be keyed. But **a lookup happens before the + /// origin has chosen**, so the chosen value is unavailable at exactly the moment + /// the key is needed. Keying on it would mean storing under `br` and looking up + /// under `gzip, br`: a cache that never hits. + /// + /// Keying on the request side is sound because origin negotiation is a function of + /// what it was offered, so identical offers yield identical choices. The encoding + /// actually chosen is recorded in [`TemplateMetadata::content_encoding`] and is + /// what the served response declares. + /// + /// Read as forwarded, after `restrict_accept_encoding` narrows it — the value the + /// client sent is not necessarily the value the origin saw. + pub accept_encoding: String, /// Identifies the enabled integration set and the tsjs bundle. Both change the /// injected markup for the same URL. pub integration_fingerprint: String, @@ -89,7 +101,7 @@ impl TemplateCacheKey { push(&self.request_scheme); push(&self.request_host); push(&self.url); - push(&self.content_encoding); + push(&self.accept_encoding); push(&self.integration_fingerprint); push(&self.vary_values.len().to_string()); @@ -407,7 +419,7 @@ mod tests { request_scheme: "https".to_string(), assembly_mode: AssemblyMode::Esi, vary_values: vec![("rsc".to_string(), "1".to_string())], - content_encoding: "gzip".to_string(), + accept_encoding: "gzip".to_string(), integration_fingerprint: "abc123".to_string(), schema_version: TEMPLATE_SCHEMA_VERSION, } @@ -440,11 +452,11 @@ mod tests { assert_ne!(scheme.to_cache_key(), base, "scheme must change the key"); let mut encoding = key(); - encoding.content_encoding = "br".to_string(); + encoding.accept_encoding = "br".to_string(); assert_ne!( encoding.to_cache_key(), base, - "content encoding must change the key; serving brotli to a gzip client \ + "accept encoding must change the key; serving brotli to a gzip client \ is a broken response" ); diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index 10f2a58fe..c8caa16af 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -1284,6 +1284,49 @@ pub async fn buffer_publisher_response_async( } } +/// Builds the response served from a C2 hit. +/// +/// Every header is constructed here rather than replayed from the stored entry. The +/// publisher path rewrites `Cache-Control` and strips validators after the send, so a +/// replayed origin header would fight it; constructing them also means no origin header +/// can reach a second visitor through the cache, which makes the `Set-Cookie` privacy +/// net trivially safe rather than safe-by-audit. +/// +/// # Errors +/// +/// Returns an error if the stored metadata cannot be rendered as header values, which +/// would mean a corrupt entry. +/// +/// Spike-only, for the #1009 ESI validation. +fn build_cached_template_response( + entry: &crate::platform::TemplateEntry, +) -> Result, Report> { + let invalid = |what: &str| TrustedServerError::Proxy { + message: format!("cached template has an unusable {what}"), + }; + let mut response = Response::new(EdgeBody::from(entry.body.clone())); + response.headers_mut().insert( + header::CONTENT_TYPE, + HeaderValue::from_str(&entry.metadata.content_type) + .change_context_lazy(|| invalid("content type"))?, + ); + // The encoding the origin actually chose, not the one keyed on. See + // `TemplateCacheKey::accept_encoding` for why those differ. + if !entry.metadata.content_encoding.is_empty() && entry.metadata.content_encoding != "identity" + { + response.headers_mut().insert( + header::CONTENT_ENCODING, + HeaderValue::from_str(&entry.metadata.content_encoding) + .change_context_lazy(|| invalid("content encoding"))?, + ); + } + response.headers_mut().insert( + header::CONTENT_LENGTH, + HeaderValue::from(entry.body.len() as u64), + ); + Ok(response) +} + /// Writes the transformed template to the shared cache, if the gate authorized it. /// /// The key's presence is the authorization: it is `Some` only when @@ -3032,23 +3075,44 @@ pub async fn handle_publisher_request( // legacy path never sets it. Either way it is an internal edge signal that // must not leak to publisher backends. req.headers_mut().remove("fastly-ssl"); - // Captured before the request is consumed: the C2 key identifies the origin - // document plus the request headers the origin varies on, and this is the last - // point where both are still in hand. + // The C2 key is built here, before the request is consumed, because every field + // is request-derived and this is the last point where the request is in hand. // - // Read from the request as forwarded, after `restrict_accept_encoding` — keying on - // what the client originally sent would key on a value the origin never saw. - let template_cache_url = target_uri.to_string(); - let template_cache_vary_values = settings - .creative_opportunities - .as_ref() - .map(CreativeOpportunitiesConfig::template_cache_vary) - .unwrap_or_else(|| VarySpec::new([])) - .values_from(|name| { - req.headers() - .get(name) + // Building it pre-fetch is what makes a lookup possible at all: a key that needed + // the origin's response could only ever authorize a store, never satisfy a read. + // + // Headers are read as forwarded, after `restrict_accept_encoding` — keying on what + // the client originally sent would key on a value the origin never saw. + let template_cache_key = (!matches!(assembly_mode, AssemblyMode::Inline)).then(|| { + crate::platform::TemplateCacheKey { + url: target_uri.to_string(), + request_host: request_host.to_string(), + request_scheme: request_scheme.to_string(), + assembly_mode, + vary_values: settings + .creative_opportunities + .as_ref() + .map(CreativeOpportunitiesConfig::template_cache_vary) + .unwrap_or_else(|| VarySpec::new([])) + .values_from(|name| { + req.headers() + .get(name) + .and_then(|value| value.to_str().ok()) + }), + accept_encoding: req + .headers() + .get(header::ACCEPT_ENCODING) .and_then(|value| value.to_str().ok()) - }); + .unwrap_or_default() + .to_string(), + // Changes whenever any JS module changes, so a bundle deploy invalidates + // stored templates without needing a purge. + integration_fingerprint: trusted_server_js::concatenated_hash( + &trusted_server_js::all_module_ids(), + ), + schema_version: crate::platform::TEMPLATE_SCHEMA_VERSION, + } + }); *req.uri_mut() = target_uri; req.headers_mut().insert( header::HOST, @@ -3057,6 +3121,32 @@ pub async fn handle_publisher_request( })?, ); + // C2 lookup, before the origin fetch — the whole point is to skip it. + // + // The gate that authorized the store was response-derived, so it cannot re-run + // here and does not need to: a template in the cache already passed it. What must + // re-run are the *request*-derived disqualifications, because they are properties + // of this request rather than of the stored bytes. An authenticated request must + // not be served a shared template even if that template is perfectly cacheable. + if let Some(key) = template_cache_key + .as_ref() + .filter(|_| !request_had_authorization && !request_had_cookie) + { + match services.template_cache().get(key).await { + Ok(entry) => { + log::debug!("c2_template_cache hit: {} bytes", entry.body.len()); + // Headers are constructed rather than replayed. The publisher path + // rewrites `Cache-Control` and strips validators *after* the send, so + // replaying stored origin headers would fight that — and constructing + // them means no origin header can leak through the cache. + return Ok(PublisherResponse::Buffered(build_cached_template_response( + &entry, + )?)); + } + Err(miss) => log::debug!("c2_template_cache miss: {miss}"), + } + } + // SSP requests are already racing through the platform HTTP client, so // origin TTFB tracks origin latency rather than the auction timeout. // @@ -3183,21 +3273,14 @@ pub async fn handle_publisher_request( .to_lowercase(); let route = classify_response_route(status, &content_type, &content_encoding, request_host); - // The shared-template cache gate. Evaluated here rather than earlier because the - // negotiated content encoding is part of the key: the pipeline pairs input encoding - // to output encoding, so a template stored as brotli must never be handed to a - // client that asked for gzip. + // The shared-template cache gate: it does not build the key, it authorizes storing + // the one built pre-fetch. Everything it checks is response-derived, which is + // exactly why it cannot run at lookup time — and why it does not need to. Anything + // already in the cache passed this gate on the way in. // - // A `Some` key is the store authorization. Under the default `Inline` mode the gate - // reports `InlineMode` and nothing is ever stored. - let template_cache_key = if matches!(assembly_mode, AssemblyMode::Inline) { - None - } else { - let key_vary = settings - .creative_opportunities - .as_ref() - .map(CreativeOpportunitiesConfig::template_cache_vary) - .unwrap_or_else(|| VarySpec::new([])); + // A surviving key is the store authorization. Under `Inline` there is no key to + // survive. + let template_cache_key = template_cache_key.filter(|_| { match c2_bypass_reason( assembly_mode, request_had_authorization, @@ -3205,31 +3288,22 @@ pub async fn handle_publisher_request( status, &content_type, response.headers(), - &key_vary, + &settings + .creative_opportunities + .as_ref() + .map(CreativeOpportunitiesConfig::template_cache_vary) + .unwrap_or_else(|| VarySpec::new([])), ) { Some(reason) => { log::debug!("c2_template_cache bypass: {reason}"); - None + false } None => { log::debug!("c2_template_cache eligible"); - Some(crate::platform::TemplateCacheKey { - url: template_cache_url, - request_host: request_host.to_string(), - request_scheme: request_scheme.to_string(), - assembly_mode, - vary_values: template_cache_vary_values, - content_encoding: content_encoding.clone(), - // Changes whenever any JS module changes, so a bundle deploy - // invalidates stored templates without needing a purge. - integration_fingerprint: trusted_server_js::concatenated_hash( - &trusted_server_js::all_module_ids(), - ), - schema_version: crate::platform::TEMPLATE_SCHEMA_VERSION, - }) + true } } - }; + }); match route { ResponseRoute::PassThrough => { @@ -5306,7 +5380,7 @@ mod tests { request_scheme: "https".to_string(), assembly_mode: AssemblyMode::Esi, vary_values: vec![], - content_encoding: "identity".to_string(), + accept_encoding: "identity".to_string(), integration_fingerprint: "fp".to_string(), schema_version: crate::platform::TEMPLATE_SCHEMA_VERSION, } @@ -5387,6 +5461,288 @@ mod tests { } } + mod c2_end_to_end_tests { + //! The chain, end to end: a second request for the same URL must be served from + //! the cache without touching the origin. Everything else in this file tests a + //! link; this tests that they connect. + + use super::*; + use crate::platform::ClientInfo; + use crate::platform::test_support::{ + NoopConfigStore, NoopGeo, NoopSecretStore, StubBackend, StubHttpClient, + }; + use crate::test_support::tests::crate_test_settings_str; + use std::collections::HashMap; + + /// A working cache, unlike the recorder above — this one has to actually return + /// what it stored, or a hit proves nothing. + #[derive(Default)] + struct MemoryTemplateCache { + entries: Mutex>, + } + + #[async_trait::async_trait(?Send)] + impl crate::platform::PlatformTemplateCache for MemoryTemplateCache { + async fn get( + &self, + key: &crate::platform::TemplateCacheKey, + ) -> Result + { + self.entries + .lock() + .expect("should lock entries") + .get(&key.to_cache_key()) + .cloned() + .ok_or(crate::platform::TemplateCacheMiss::NotFound) + } + + async fn put( + &self, + key: &crate::platform::TemplateCacheKey, + metadata: &crate::platform::TemplateMetadata, + body: Vec, + ) -> Result<(), crate::platform::TemplateCacheError> { + self.entries.lock().expect("should lock entries").insert( + key.to_cache_key(), + crate::platform::TemplateEntry { + metadata: metadata.clone(), + body, + }, + ); + Ok(()) + } + + async fn purge_all(&self) -> Result<(), crate::platform::TemplateCacheError> { + self.entries.lock().expect("should lock entries").clear(); + Ok(()) + } + } + + fn settings_with_mode(mode: &str) -> Settings { + let toml = format!( + "{}\n[creative_opportunities]\ngam_network_id = \"99999\"\n\ + assembly_mode = \"{mode}\"\n", + crate_test_settings_str() + ); + let mut settings = + Settings::from_toml(&toml).expect("should parse settings with an assembly mode"); + // Mirrors `create_test_settings`; the integration registry refuses to build + // without it. + settings.proxy.allowed_domains = + vec!["*.example".to_string(), "*.example.com".to_string()]; + settings + } + + fn services( + http_client: Arc, + cache: Arc, + ) -> RuntimeServices { + RuntimeServices::builder() + .config_store(Arc::new(NoopConfigStore)) + .secret_store(Arc::new(NoopSecretStore)) + .kv_store(Arc::new(edgezero_core::key_value_store::NoopKvStore)) + .backend(Arc::new(StubBackend)) + .http_client(http_client) + .geo(Arc::new(NoopGeo)) + .client_info(ClientInfo::default()) + .template_cache(cache) + .build() + } + + /// Shareable HTML: no `Set-Cookie`, no `Vary`, a public `Cache-Control`. Every + /// condition the gate checks is satisfied, so a bypass here would be a bug in + /// the wiring rather than in the fixture. + fn queue_shareable_html(stub: &StubHttpClient) { + stub.push_response_with_headers( + 200, + b"origin".to_vec(), + vec![ + ("content-type", "text/html; charset=utf-8"), + ("cache-control", "public, max-age=300"), + ], + ); + } + + fn navigation_request() -> Request { + HttpRequest::builder() + .method(Method::GET) + .uri("https://ts.example.com/article") + .header(header::HOST, "ts.example.com") + .header("sec-fetch-dest", "document") + .body(EdgeBody::empty()) + .expect("should build navigation request") + } + + /// Runs the full request, **including the finalizer**. + /// + /// The finalizer is not optional here: the store happens once the transform has + /// produced every byte, so a test that stopped at `handle_publisher_request` + /// would never populate the cache and a hit could not be proven. + async fn run( + settings: &Arc, + services: &RuntimeServices, + request: Request, + ) -> Response { + let orchestrator = Arc::new(AuctionOrchestrator::new(settings.auction.clone())); + let registry = + IntegrationRegistry::new(settings).expect("should create integration registry"); + let consent = crate::consent::ConsentContext { + jurisdiction: crate::consent::jurisdiction::Jurisdiction::NonRegulated, + ..Default::default() + }; + let mut ec_context = EcContext::new_for_test(None, consent); + let publisher_response = handle_publisher_request( + settings, + services, + None, + &mut ec_context, + AuctionDispatch { + orchestrator: &orchestrator, + slots: &[], + registry: None, + }, + request, + ) + .await + .expect("should proxy publisher request"); + + publisher_response_into_streaming_response( + publisher_response, + &Method::GET, + Arc::clone(settings), + ®istry, + orchestrator, + services.clone(), + ) + .await + .expect("should finalize publisher response") + } + + fn body_of(response: Response) -> Vec { + response + .into_body() + .into_bytes() + .expect("a shared-mode response is buffered, so its bytes are in hand") + .to_vec() + } + + #[tokio::test] + async fn a_second_request_is_served_from_the_cache_without_touching_the_origin() { + let stub = Arc::new(StubHttpClient::new()); + let cache = Arc::new(MemoryTemplateCache::default()); + let settings = Arc::new(settings_with_mode("esi")); + let services = services(Arc::clone(&stub), Arc::clone(&cache)); + + // Only one origin response is queued. If the second request reached the + // origin it would find the queue empty, so this fixture is itself part of + // the assertion. + queue_shareable_html(&stub); + + let first = body_of(run(&settings, &services, navigation_request()).await); + assert_eq!( + stub.recorded_request_uris().len(), + 1, + "the cold request must fetch the origin" + ); + + let second = body_of(run(&settings, &services, navigation_request()).await); + assert_eq!( + stub.recorded_request_uris().len(), + 1, + "the warm request must not fetch the origin — that saving is the point" + ); + assert_eq!( + second, first, + "the cached template must be byte-identical to what was stored" + ); + } + + #[tokio::test] + async fn inline_mode_never_reads_or_writes_the_cache() { + // The shipped path. If this ever cached, per-user ad state would be shared + // between visitors — the exact failure the whole design exists to avoid. + let stub = Arc::new(StubHttpClient::new()); + let cache = Arc::new(MemoryTemplateCache::default()); + let settings = Arc::new(settings_with_mode("inline")); + let services = services(Arc::clone(&stub), Arc::clone(&cache)); + queue_shareable_html(&stub); + queue_shareable_html(&stub); + + let _ = run(&settings, &services, navigation_request()).await; + let _ = run(&settings, &services, navigation_request()).await; + + assert_eq!( + stub.recorded_request_uris().len(), + 2, + "inline must fetch the origin every time" + ); + assert!( + cache + .entries + .lock() + .expect("should lock entries") + .is_empty(), + "inline must never write a shared template" + ); + } + + #[tokio::test] + async fn an_authenticated_request_is_not_served_a_shared_template() { + // The stored template is perfectly cacheable; this request is not entitled + // to it. The store gate cannot express that, because it is a property of + // the reader rather than of the bytes — which is why the lookup re-checks. + let stub = Arc::new(StubHttpClient::new()); + let cache = Arc::new(MemoryTemplateCache::default()); + let settings = Arc::new(settings_with_mode("esi")); + let services = services(Arc::clone(&stub), Arc::clone(&cache)); + queue_shareable_html(&stub); + queue_shareable_html(&stub); + + let _ = run(&settings, &services, navigation_request()).await; + assert_eq!( + cache.entries.lock().expect("should lock entries").len(), + 1, + "the cold request should have populated the cache" + ); + + let orchestrator = AuctionOrchestrator::new(settings.auction.clone()); + let consent = crate::consent::ConsentContext { + jurisdiction: crate::consent::jurisdiction::Jurisdiction::NonRegulated, + ..Default::default() + }; + let mut ec_context = EcContext::new_for_test(None, consent); + let authenticated = HttpRequest::builder() + .method(Method::GET) + .uri("https://ts.example.com/article") + .header(header::HOST, "ts.example.com") + .header("sec-fetch-dest", "document") + .header(header::AUTHORIZATION, "Basic dXNlcjpwYXNz") + .body(EdgeBody::empty()) + .expect("should build authenticated request"); + let _ = handle_publisher_request( + &settings, + &services, + None, + &mut ec_context, + AuctionDispatch { + orchestrator: &orchestrator, + slots: &[], + registry: None, + }, + authenticated, + ) + .await + .expect("should proxy publisher request"); + + assert_eq!( + stub.recorded_request_uris().len(), + 2, + "an authenticated request must reach the origin rather than read a \ + shared template" + ); + } + } + mod c2_gate_tests { //! `cache::core` stores whatever it is handed and rejects nothing, so every //! one of these conditions is the caller's to enforce. Each is a leak vector From fe0658d167c9a33bf6651820bd1e30856995b8bb Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 11:57:52 +0530 Subject: [PATCH 30/44] Execute the Core Cache join the publisher actually uses MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The publisher tests use an in-memory double and the Fastly tests call the concrete type, so the seam between them was only ever type-checked: the publisher reaches the cache as a dyn PlatformTemplateCache behind RuntimeServices, which is what app.rs wires and what neither suite executed. Now round-tripped through the trait object under Viceroy against the real Core Cache. Records Task 3 as complete in the plan and the findings, including the three problems that only appeared once the code had to run — the Vary ordering, the encoding the origin chooses versus the one it is offered, and streaming not collecting the bytes a store needs. None were visible in the plan or in review, which is the same pattern as the earlier review findings arriving one layer down. Docs build verified, not just formatted. --- .../src/template_cache.rs | 19 ++++++++ .../2026-08-08-1009-measurement-findings.md | 33 ++++++++++++++ .../2026-08-10-1009-esi-validation-spike.md | 44 +++++++++++++++++++ 3 files changed, 96 insertions(+) diff --git a/crates/trusted-server-adapter-fastly/src/template_cache.rs b/crates/trusted-server-adapter-fastly/src/template_cache.rs index a7c2ceae8..3a32a617e 100644 --- a/crates/trusted-server-adapter-fastly/src/template_cache.rs +++ b/crates/trusted-server-adapter-fastly/src/template_cache.rs @@ -307,6 +307,25 @@ mod tests { ); } + #[test] + fn the_cache_round_trips_through_the_platform_trait_object() { + // Every other test here calls `FastlyTemplateCache` concretely. The publisher + // never does — it reaches the cache as a `dyn PlatformTemplateCache` behind + // `RuntimeServices`. That join is what `app.rs` wires, and until this test it + // was only type-checked, never executed. + let cache: std::sync::Arc = std::sync::Arc::new(cache()); + let key = key("https://example.com/via-trait-object"); + let body = b"template".to_vec(); + + run(cache.put(&key, &metadata_for(&body), body.clone())).expect("should store"); + + assert_eq!( + run(cache.get(&key)).expect("should read back").body, + body, + "the trait object must reach the same Core Cache the concrete type does" + ); + } + #[test] fn a_length_mismatch_is_refused_at_write_rather_than_stored() { // Storing metadata whose length disagrees with the body would make every diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md index 2d92512f9..ea58c61b0 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md @@ -350,6 +350,39 @@ Do not proceed to Task 3 Step 4 (actual C2 read/write) or expose `AssemblyMode` test or staging traffic until the full-document byte-identity test exists. The three fixes above close the known holes; that test is what would catch the next one. +## Task 3 complete — the C2 cache engages end to end + +`2db10639` (store), `2a2e6c6a` (lookup), plus `b688d667`/`577eb85a` for the `Vary` +handling. A second request for the same URL is now served without touching the origin, +byte-identical to what was stored. + +**Three problems only appeared once the code had to run**, none of them visible in the +plan or in review: + +1. **The key needed the origin's `Vary`, but a lookup precedes the fetch.** Resolved with + an operator-stated list plus a post-response drift guard that refuses to store under a + key that missed something. Spike-grade: a two-phase lookup is the correct answer and + doubles the lookups. +2. **The key carried the encoding the _origin_ chose**, which also does not exist at + lookup time — storing under `br`, looking up under `gzip, br`, a cache that never hits. + Now keyed on what was sent to the origin. +3. **Storing needs every transformed byte; streaming does not collect them.** Shared modes + take the buffered finalizer, branching on the store authorization rather than the + assembly mode, so `Inline` cannot reach it. + +Each was a case where the design read as complete and the implementation had a hole in +it. That is the same pattern as the three review findings above, arriving one layer down. + +**Verified by mutation, not just by green tests.** Disabling the lookup fails the hit +test, so the hit is the cache answering rather than the fixture answering twice; dropping +the `Authorization` re-check fails the authenticated test; reading only the first `Vary` +header value, and disabling the drift guard, each fail their own tests. The reviewer's +gate above was satisfied first: the byte-identity tests it demanded exist and were +themselves mutation-checked. + +**Still not deployable.** `ClientFill` and `Esi` render a template with a hole and +nothing filling it — Task 4 and Task 5. A cache that works is necessary, not sufficient. + ## Step B — consumers of TS's own response headers Not yet run. diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index 9faa63914..c1f9cf1f3 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -614,6 +614,50 @@ cargo fmt --all -- --check && cargo clippy-fastly `ClientFill` must work on all four adapters. `Esi` is Fastly-only and must not break the others' compilation. +- [x] **Step 6: the call site — DONE.** `2db10639` (store), `2a2e6c6a` (lookup). + +The cache now engages end to end: a second request for the same URL is served without +touching the origin, and is byte-identical to what was stored. Verified by mutation — +disabling the lookup fails the hit test, so the hit is the cache answering rather than +the fixture answering twice. + +**Wiring the lookup corrected the key.** It carried the content encoding the _origin_ +chose, which does not exist at lookup time. That meant storing under `br` and looking up +under `gzip, br` — a cache that never hits. The field is now the `Accept-Encoding` sent +to the origin. Sound because negotiation is a function of what the origin was offered, +so identical offers yield identical choices; the chosen encoding stays in the metadata +and is what the served response declares. + +That made every key field request-derived, so **the key is built before the fetch** and +the response gate only authorizes storing it. A key that needed the response could only +ever authorize a store, never satisfy a read. + +**The lookup re-checks the request-derived disqualifications, and only those.** The +store gate is response-derived and cannot re-run, but need not: anything in the cache +passed it on the way in. What must re-run are properties of the _reader_ rather than of +the bytes — an authenticated request must not be served a shared template even when that +template is perfectly cacheable. + +**Shared modes take the buffered finalizer.** Storing needs every transformed byte and +streaming does not collect them. The branch keys on the store authorization rather than +on the assembly mode, so `Inline` never reaches it and the spike cannot regress the +shipped path by construction. A C2 _miss_ therefore buffers — the right trade, since a +miss is already paying an origin fetch and a full transform, and what the spike measures +is the hit, where there is no origin fetch to stream from at all. + +Every response header on a hit is constructed, never replayed, so no origin header can +reach a second visitor through the cache. + +The publisher tests use an in-memory cache double, so they prove the wiring rather than +the backing. The join they leave untested is the one `app.rs` makes: the publisher +reaches the cache as a `dyn PlatformTemplateCache` behind `RuntimeServices`, never as +the concrete type the Fastly tests exercise. That join is now executed under Viceroy +against the real Core Cache rather than only type-checked. + +**What this does not establish.** `ClientFill` and `Esi` still render a template with a +hole and nothing filling it. Task 4 and Task 5 remain the blockers on anything +deployable — a cache that works is necessary, not sufficient. + --- ## Task 4: Arm A2 — client-fill From 06864314d41db8ace9082b015ea857cbf64926a7 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 12:24:53 +0530 Subject: [PATCH 31/44] Give the ESI seam a fragment to include MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Esi arm emitted nothing at because pointing an esi:include at /_ts/page-bids would splice raw JSON where an executable script belongs. This adds the script form and the marker that uses it. A format on the existing endpoint rather than a second path. Both forms carry the same data behind the same cross-site gate, so a new path would have duplicated that gate, the deprecation alias and the private, no-store header across four adapter routers for a difference in wrapping. As a format it is reachable on every adapter with no routing change at all. An unknown format is a 400, not a fall back to JSON. Defaulting would make a typo in an esi:include return 200 with a broken page and nothing in the logs pointing at the cause — the precise failure that kept this arm dark. The fragment reuses build_bids_script rather than formatting its own. If the two diverged, the A1-vs-A3 comparison would be measuring two script shapes instead of two delivery mechanisms and its number would mean nothing. Slots are not included: under a shared mode the head seam emits no tsjs.adSlots and the template already carries the slot markup, so the fragment supplies only what could not be shared. The marker carries no path. It is baked into a shared template, so every byte in it is a byte every reader of that template receives; the adapter's include dispatcher will supply the path from the live request. That also keeps a URL out of the cached bytes, so there is no escaping question at the seam. An existing test caught a real inconsistency this exposed. The root-auction gate asserted that dispatch usefulness tracks whether the seam emits bytes — true only while Esi emitted none. Under a Marker the seam emits bytes and reads nothing from ad_bids_state, because the fragment runs its own auction, so the old reading would have dispatched a root auction with no consumer: silent SSP spend, the exact waste that gate exists to prevent. The invariant now distinguishes emitting from consuming. Full gates green — fmt, six clippy targets, four adapter suites, 1889 core tests. Still open: the Fastly ESI processor is not wired, so the include is emitted and never resolved. Task 5. --- crates/trusted-server-core/src/publisher.rs | 288 +++++++++++++++++--- 1 file changed, 249 insertions(+), 39 deletions(-) diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index c8caa16af..7d8001249 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -992,6 +992,12 @@ pub(crate) fn root_auction_is_useful(mode: AssemblyMode) -> bool { } } +/// The `esi:include` emitted at the `` seam under [`AssemblyMode::Esi`]. +/// +/// No `path` query parameter: see [`body_close_injection`]. The adapter's include +/// dispatcher appends it from the live request. +pub const ESI_BIDS_INCLUDE: &str = ""; + /// What the `` seam should inject, given the assembly mode. /// /// Explicit rather than inferred. The previous shape read @@ -999,12 +1005,16 @@ pub(crate) fn root_auction_is_useful(mode: AssemblyMode) -> bool { /// two independent decisions: once [`template_ad_slots_script`] stopped emitting a /// head script under a shared mode, body-close injection stopped with it. /// -/// `Esi` returns [`BodyCloseInjection::None`] for now rather than a placeholder -/// marker. The marker must point at a dedicated fragment endpoint returning an -/// executable script — `/_ts/page-bids` returns JSON, and ESI splices fragment -/// bytes verbatim, so aiming at it would put raw JSON where a script belongs. -/// That endpoint does not exist yet, and emitting a marker with nothing behind it -/// would be worse than emitting nothing. +/// `Esi` emits an `esi:include` pointing at the page-bids endpoint's **fragment** +/// format, which returns the same executable `"; + + fn template_with_include() -> String { + format!("
{ESI_BIDS_INCLUDE}") + } + + #[test] + fn the_seams_own_marker_is_resolved() { + // Deliberately built from `ESI_BIDS_INCLUDE` rather than a hand-written + // include. The two live in different crates, and a test that wrote its own + // marker would keep passing after the seam's changed shape stopped parsing. + let assembled = assemble(&template_with_include(), FRAGMENT).expect("should assemble"); + + assert!( + assembled.contains(FRAGMENT), + "the fragment must reach the document: {assembled}" + ); + assert!( + !assembled.contains("esi:include"), + "no unresolved include may survive: {assembled}" + ); + } + + #[test] + fn the_fragment_lands_where_the_marker_was() { + // Position matters: the script reads slots defined earlier in the document, so + // an assembler that appended instead of substituting would produce a page that + // parses and does nothing. + let assembled = assemble(&template_with_include(), FRAGMENT).expect("should assemble"); + + let slot = assembled.find("id=\"slot\"").expect("slot should survive"); + let script = assembled + .find(FRAGMENT) + .expect("fragment should be present"); + let body_close = assembled + .find("") + .expect("body close should survive"); + + assert!(slot < script, "the fragment must follow the slot markup"); + assert!(script < body_close, "the fragment must precede ``"); + } + + #[test] + fn a_document_without_an_include_is_returned_unchanged() { + // Inline mode's documents pass through this path only if something is + // misrouted, and a mangled document would be a far worse failure than a no-op. + let plain = "

no includes here

"; + + assert_eq!( + assemble(plain, FRAGMENT).expect("should assemble"), + plain, + "a template with nothing to splice must be byte-identical" + ); + } + + #[test] + fn an_empty_fragment_still_removes_the_marker() { + // The empty-bids case is normal, not exceptional: an auction that returned + // nothing still has to produce a document with no `esi:include` left in it, or + // the browser renders the raw tag as text. + let assembled = assemble(&template_with_include(), "").expect("should assemble"); + + assert!( + !assembled.contains("esi:include"), + "an empty fragment must still consume the marker: {assembled}" + ); + assert!(assembled.contains(""), "the document must survive"); + } + + #[test] + fn script_bearing_fragments_are_spliced_verbatim() { + // ESI substitutes bytes without escaping, which is exactly why the fragment + // endpoint must return markup rather than JSON. This pins that behaviour, since + // an `esi` release that started escaping would silently turn every fragment + // into visible text. + let fragment = ""; + let assembled = assemble(&template_with_include(), fragment).expect("should assemble"); + + assert!( + assembled.contains(fragment), + "the fragment must be spliced verbatim: {assembled}" + ); + } +} diff --git a/crates/trusted-server-adapter-fastly/src/main.rs b/crates/trusted-server-adapter-fastly/src/main.rs index 603ffdd9b..5ffb7226a 100644 --- a/crates/trusted-server-adapter-fastly/src/main.rs +++ b/crates/trusted-server-adapter-fastly/src/main.rs @@ -29,6 +29,7 @@ mod app; mod backend; mod compat; mod ec_kv; +mod esi_assembly; mod logging; mod management_api; mod middleware; From 0597f54e36c6354b400c94a0e13267b752ff897f Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 12:41:15 +0530 Subject: [PATCH 33/44] State the ESI processor's safety settings instead of inheriting them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The assembler used Configuration::default(). The plan's Task 5 Step 2 says explicitly not to, and reading the crate showed why that instruction exists. is_includes_cacheable defaults to true. A fragment here carries one visitor's bids, so letting the ESI layer cache it serves those bids to the next visitor — the exact per-user leak this whole design exists to prevent, arriving silently on a cache hit. That default fails open, in a pre-1.0 crate whose defaults can move in a patch release. Every safety-relevant field is now stated rather than inherited: - Fragment caching off, and includes_force_ttl left unset — it caches everything, ignoring private, no-store and Set-Cookie alike. - default_dca None and inherit_parent_dca false, so fragment bytes are never re-parsed as ESI. The fragment is a script built from auction data; parsing it as ESI would let bid content act as markup instructions. - max_include_depth 1. One include, no nesting. A template asking for more is not one this arm built. - Rendered caching and edge_control off. The publisher path sets private, no-store before any body byte is written, and headers cannot change once streaming starts on this adapter, so a Cache-Control computed from include TTLs would contradict it — and the contradiction would favour caching. Four tests assert the configuration rather than trusting it, plus one that proves the behaviour rather than the flag: a fragment containing its own esi:include is spliced as text, not dispatched, so auction data cannot drive fragment requests. Full gates green — fmt, six clippy targets, four adapter suites, 133 Fastly adapter tests. --- .../src/esi_assembly.rs | 101 +++++++++++++++++- 1 file changed, 99 insertions(+), 2 deletions(-) diff --git a/crates/trusted-server-adapter-fastly/src/esi_assembly.rs b/crates/trusted-server-adapter-fastly/src/esi_assembly.rs index 85e6b25b3..a3282b98a 100644 --- a/crates/trusted-server-adapter-fastly/src/esi_assembly.rs +++ b/crates/trusted-server-adapter-fastly/src/esi_assembly.rs @@ -21,11 +21,53 @@ // and satisfied in the binary, and no single attribute can be both. #![allow(dead_code)] -use esi::{Configuration, PendingFragmentContent, Processor}; +use esi::{CacheConfig, Configuration, DcaMode, PendingFragmentContent, Processor}; use fastly::Response; use fastly::http::StatusCode; use std::io::Cursor; +/// The processor configuration, with every safety-relevant field stated. +/// +/// Not `Configuration::default()`. Two of these settings fail **open**, and this is a +/// pre-1.0 crate whose defaults can move in a patch release — a comment saying "the +/// default is already what we want" would be an assumption rechecked by nobody. +/// +/// The two that matter: +/// +/// - **`is_includes_cacheable` defaults to `true`.** Fragments here carry one visitor's +/// bids. Letting the ESI layer cache them is precisely the per-user leak this whole +/// design exists to prevent, and it would happen silently on a cache hit. +/// - **`default_dca` / `inherit_parent_dca`** decide whether fragment bytes are +/// re-parsed as ESI. Our fragment is a `"; + let assembled = assemble(&template_with_include(), fragment).expect("should assemble"); + + assert!( + assembled.contains("/evil"), + "the inner tag must survive as text rather than being resolved: {assembled}" + ); + } + #[test] fn script_bearing_fragments_are_spliced_verbatim() { // ESI substitutes bytes without escaping, which is exactly why the fragment From 2617ecc248832d24aa6c3a4b4ed7a0cf60e813f0 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 12:42:21 +0530 Subject: [PATCH 34/44] Record that ESI's mechanism is proven and its defaults are not safe Two findings worth keeping out of commit messages alone. The async/sync obstacle is dissolved rather than worked around: PendingFragmentContent::CompletedRequest means the dispatcher performs no I/O, so the plan no longer needs the self-referencing backend it assumed. And the plan's "call the setters, do not trust the defaults" instruction turned out to be load-bearing: is_includes_cacheable defaults to true, which caches per-user bid fragments and serves them to the next visitor. Docs build verified, not just formatted. --- .../2026-08-10-1009-esi-validation-spike.md | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index c1f9cf1f3..a62006566 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -686,6 +686,37 @@ for the silent-empty-bids trap, which applies in full. ## Task 5: Arm A3 — ESI at the edge +- [x] **Step 0: the mechanism works — DONE.** `9539061e`, hardened in `0597f54e`. + +Verified under Viceroy with the real `esi` 0.7 crate rather than argued from docs: a +template carrying the `` seam's own `esi:include` comes back with the fragment +spliced in its place and no unresolved tag left. + +**The async/sync obstacle is dissolved, not worked around.** `esi`'s fragment dispatcher +is synchronous and this codebase's fragment producer is `async`; calling one from the +other means a nested executor, which panics. +`PendingFragmentContent::CompletedRequest` lets the dispatcher hand back an +already-built response, so the caller resolves the fragment in the normal async flow and +the dispatcher performs **no I/O at all** — no subrequest, no backend, no self-call, +nothing for Viceroy to stub. That also removes the need for a self-referencing backend +this plan would otherwise have required. + +**Step 2's instruction was right, and reading the crate showed why.** +`CacheConfig::is_includes_cacheable` defaults to **`true`**. A fragment carries one +visitor's bids, so the default caches per-user data and serves it to the next visitor — +silently, on a hit. `includes_force_ttl` is worse where set: it caches everything, +ignoring `private`, `no-store` and `Set-Cookie` alike. Both now stated explicitly, along +with `default_dca`/`inherit_parent_dca` (fragment bytes are data, never re-parsed as +ESI), `max_include_depth = 1`, and rendered caching / `edge_control` off because the +publisher path owns those headers. + +Nine tests. Four assert the configuration; the rest assert behaviour, including that a +fragment containing its own `esi:include` is spliced as text rather than dispatched, so +auction data cannot drive fragment requests. + +**What remains is the call site**, below. Emitting the include and resolving it are both +proven; connecting them is not done. + - [ ] **Step 1: Wire `process_stream`, not the wrappers** `process_response` and `process_response_streaming` consume `self` _and_ send the response From 0adb578ead28f7d3cad4c1e184f999a6b462bcd8 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 12:58:13 +0530 Subject: [PATCH 35/44] Stop a C2 hit from serving a shared-cacheable per-user document MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A cache hit returns before the origin fetch, and therefore before the point where the publisher path stamps private, no-store and strips validators. Nothing else set it, so a hit served HTML with no Cache-Control at all. That is not a safe default. HTML with no Cache-Control is heuristically cacheable by browsers and intermediaries, so an assembled per-user response was eligible to be stored and shared — the C3 the design forbids outright, reached by omission rather than by anything anyone wrote. The plan's Task 6 predicted exactly this class of miss. It says assert positively, because forbidding public, s-maxage and Surrogate-Control passes trivially when there is no Cache-Control to forbid. Checking for their absence would have reported this bug as safe. The hit path now stamps private, no-store first rather than last, and two tests assert it. One covers the returning visitor specifically: a first-visit response sets an EC cookie and the adapter's cookie-privacy net force-privatizes it regardless, so a test that only exercised first visits would pass on the backstop rather than on this code. A returning visitor sets no cookie, the net never fires, and this path is the only thing between the document and a shared cache. Verified by mutation: removing the stamp fails both new tests, with the hit and isolation tests still passing. Full gates green — fmt, six clippy targets, four adapter suites. --- crates/trusted-server-core/src/publisher.rs | 94 +++++++++++++++++++-- 1 file changed, 89 insertions(+), 5 deletions(-) diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index 7d8001249..64ae53125 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -1297,11 +1297,19 @@ pub async fn buffer_publisher_response_async( /// Builds the response served from a C2 hit. /// -/// Every header is constructed here rather than replayed from the stored entry. The -/// publisher path rewrites `Cache-Control` and strips validators after the send, so a -/// replayed origin header would fight it; constructing them also means no origin header -/// can reach a second visitor through the cache, which makes the `Set-Cookie` privacy -/// net trivially safe rather than safe-by-audit. +/// Every header is constructed here rather than replayed from the stored entry, so no +/// origin header can reach a second visitor through the cache. +/// +/// # Why this sets `private, no-store` itself +/// +/// A C2 hit returns **before** the origin fetch, and therefore before the point where +/// the publisher path stamps `private, no-store` and strips validators. Omitting it +/// here does not fall back to a safe default — it emits HTML with no `Cache-Control` at +/// all, which is heuristically cacheable by browsers and intermediaries. That is a +/// shared cache of an assembled per-user response: the C3 the design forbids outright. +/// +/// Asserting the absence of `public`/`s-maxage`/`Surrogate-Control` would not have +/// caught it. Nothing was present to forbid. /// /// # Errors /// @@ -1316,6 +1324,12 @@ fn build_cached_template_response( message: format!("cached template has an unusable {what}"), }; let mut response = Response::new(EdgeBody::from(entry.body.clone())); + // First, not last: see the note above. The assembled response is per-user even + // though the template it was built from is not. + response.headers_mut().insert( + header::CACHE_CONTROL, + HeaderValue::from_static("private, no-store"), + ); response.headers_mut().insert( header::CONTENT_TYPE, HeaderValue::from_str(&entry.metadata.content_type) @@ -5867,6 +5881,76 @@ mod tests { ); } + fn header_of(response: &Response, name: header::HeaderName) -> Option<&str> { + response.headers().get(name).and_then(|v| v.to_str().ok()) + } + + #[tokio::test] + async fn a_cache_hit_is_never_shared_cacheable() { + // Asserted positively, because the obvious negative check does not work. + // Forbidding `public`, `s-maxage` and `Surrogate-Control` passes trivially + // on a response that carries no `Cache-Control` at all — and *that* is the + // real failure mode here, since a C2 hit returns before the point where the + // publisher path stamps the response private. HTML with no `Cache-Control` + // is heuristically cacheable, so "nothing to forbid" is not safety. + let stub = Arc::new(StubHttpClient::new()); + let cache = Arc::new(MemoryTemplateCache::default()); + let settings = Arc::new(settings_with_mode("esi")); + let services = services(Arc::clone(&stub), Arc::clone(&cache)); + queue_shareable_html(&stub); + + let _cold = run(&settings, &services, navigation_request()).await; + let warm = run(&settings, &services, navigation_request()).await; + + assert_eq!( + stub.recorded_request_uris().len(), + 1, + "the second request must be a hit, or this asserts nothing" + ); + assert_eq!( + header_of(&warm, header::CACHE_CONTROL), + Some("private, no-store"), + "an assembled response is per-user even when its template is not" + ); + + // Validators would let a client revalidate into a shared copy, and the CDN + // directives would instruct an intermediary to store one outright. The + // origin fixture sends a `public, max-age=300` that must not survive. + for stripped in [header::ETAG, header::LAST_MODIFIED, header::EXPIRES] { + assert_eq!( + header_of(&warm, stripped.clone()), + None, + "{stripped} must not survive onto an assembled response" + ); + } + } + + #[tokio::test] + async fn a_returning_visitor_gets_the_same_privacy_headers() { + // The case with no backstop. A first-visit response sets an EC cookie, so + // the adapter's cookie-privacy net force-privatizes it regardless of what + // this path does. A returning visitor sets no cookie, so that net never + // fires and this path is the only thing standing between an assembled + // per-user document and a shared cache. + let stub = Arc::new(StubHttpClient::new()); + let cache = Arc::new(MemoryTemplateCache::default()); + let settings = Arc::new(settings_with_mode("esi")); + let services = services(Arc::clone(&stub), Arc::clone(&cache)); + queue_shareable_html(&stub); + + let _cold = run(&settings, &services, navigation_request()).await; + let warm = run(&settings, &services, navigation_request()).await; + + assert!( + header_of(&warm, header::SET_COOKIE).is_none(), + "no cookie here means no privacy net, which is the point of this test" + ); + assert_eq!( + header_of(&warm, header::CACHE_CONTROL), + Some("private, no-store") + ); + } + #[tokio::test] async fn inline_mode_never_reads_or_writes_the_cache() { // The shipped path. If this ever cached, per-user ad state would be shared From b3ac59a6b58e17fc48816de8cb71e0508f1886c6 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 13:39:57 +0530 Subject: [PATCH 36/44] Resolve the ESI include in the request path so ads render MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the gap between emitting a marker and filling it. Both were proven separately; nothing connected them, so a shared-mode page returned 200 with a literal esi:include in it — no ads, no error, and every monitor reporting success. A design correction first. Esi's root auction was gated off on the premise that the fragment would run its own auction via a real subrequest. That premise made the arm strictly worse: a self-referencing backend, two auction code paths, and two auctions per pageview. It is also not what esi requires — CompletedRequest satisfies an include from bytes already in hand. So the auction already in flight *is* the fragment, and root_auction_is_useful(Esi) is now true. ClientFill stays false; the browser fetches its own bids, so a root auction there genuinely has no consumer. Assembly sits behind a platform trait, like the template cache, because the only implementation uses a Fastly-only crate. The default is UnavailableTemplateAssembler, which refuses rather than passing the template through: returning it unchanged is the tempting default and it produces exactly the silent no-ads page this commit exists to prevent. Core's tests use a plain substitution instead, which is what one constant marker reduces to — and which shows the seam is portable even though the crate is not. Two call sites, deliberately not one. The miss path assembles after the transform; the hit path assembles after reading the cache. Keeping them separate is what makes the store-before-assemble ordering visible rather than implied. That ordering also revealed a bug on the hit path: it returned before the pipeline that normally collects the auction, so a hit dropped its in-flight auction — billing the SSPs for a result nobody read, the exact waste the dispatch gate exists to prevent, reappearing on the one path that skips the pipeline. The hit path now collects and assembles. Two tests carry the load. One asserts the marker never reaches the browser on either path, checking both because only one call site running would still look like success on the other. The other asserts the cached template holds the marker and never a bids script. The second one earns its place: swapping store and assemble fails it and nothing else. Every other test still passes, including the marker test, because the served page looks correct — one visitor's bids would simply be in a cache shared with the next. Verified by running that mutation. Full gates green — fmt, six clippy targets, four adapter suites, 1887 core tests, 133 Fastly adapter tests. --- .../trusted-server-adapter-fastly/src/app.rs | 1 + .../src/esi_assembly.rs | 18 +- .../trusted-server-core/src/platform/mod.rs | 6 +- .../src/platform/template_assembly.rs | 87 ++++ .../trusted-server-core/src/platform/types.rs | 42 ++ crates/trusted-server-core/src/publisher.rs | 405 ++++++++++++------ 6 files changed, 414 insertions(+), 145 deletions(-) create mode 100644 crates/trusted-server-core/src/platform/template_assembly.rs diff --git a/crates/trusted-server-adapter-fastly/src/app.rs b/crates/trusted-server-adapter-fastly/src/app.rs index c85e24670..dd89887a4 100644 --- a/crates/trusted-server-adapter-fastly/src/app.rs +++ b/crates/trusted-server-adapter-fastly/src/app.rs @@ -259,6 +259,7 @@ fn build_per_request_services(state: &AppState, ctx: &RequestContext) -> Runtime // Spike-only (#1009). Constructed unconditionally, but only read when the // assembly mode is a shared-template one — which defaults to Inline, so this // is inert until an operator opts in. + .template_assembler(Arc::new(crate::esi_assembly::FastlyTemplateAssembler)) .template_cache(Arc::new(crate::template_cache::FastlyTemplateCache::new( crate::template_cache::TEMPLATE_CACHE_TTL, ))) diff --git a/crates/trusted-server-adapter-fastly/src/esi_assembly.rs b/crates/trusted-server-adapter-fastly/src/esi_assembly.rs index a3282b98a..9d06765b0 100644 --- a/crates/trusted-server-adapter-fastly/src/esi_assembly.rs +++ b/crates/trusted-server-adapter-fastly/src/esi_assembly.rs @@ -15,16 +15,11 @@ //! //! Spike-only. Remove with the spike. -// Not yet reachable from the request path: resolving the fragment means running the -// auction, which is the next step (the spike plan's Task 5). The tests below do -// exercise it, so `expect` is wrong here — it would be unfulfilled under `cfg(test)` -// and satisfied in the binary, and no single attribute can be both. -#![allow(dead_code)] - use esi::{CacheConfig, Configuration, DcaMode, PendingFragmentContent, Processor}; use fastly::Response; use fastly::http::StatusCode; use std::io::Cursor; +use trusted_server_core::platform::{PlatformTemplateAssembler, TemplateAssemblyError}; /// The processor configuration, with every safety-relevant field stated. /// @@ -131,6 +126,17 @@ pub fn assemble(template: &str, fragment: &str) -> Result Result { + assemble(template, fragment).map_err(|e| TemplateAssemblyError::Failed { + message: e.to_string(), + }) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/trusted-server-core/src/platform/mod.rs b/crates/trusted-server-core/src/platform/mod.rs index 2ff2fb06a..8fae8a64a 100644 --- a/crates/trusted-server-core/src/platform/mod.rs +++ b/crates/trusted-server-core/src/platform/mod.rs @@ -36,7 +36,8 @@ mod error; mod http; mod image_optimizer; mod kv; -pub mod template_cache; +pub mod template_assembly; +mod template_cache; #[cfg(test)] pub(crate) mod test_support; mod traits; @@ -53,6 +54,9 @@ pub use image_optimizer::{ PlatformImageOptimizerParams, PlatformImageOptimizerRegion, }; pub use kv::UnavailableKvStore; +pub use template_assembly::{ + PlatformTemplateAssembler, TemplateAssemblyError, UnavailableTemplateAssembler, +}; pub use template_cache::{ PlatformTemplateCache, TEMPLATE_SCHEMA_VERSION, TemplateCacheError, TemplateCacheKey, TemplateCacheMiss, TemplateEntry, TemplateMetadata, UnavailableTemplateCache, VarySpec, diff --git a/crates/trusted-server-core/src/platform/template_assembly.rs b/crates/trusted-server-core/src/platform/template_assembly.rs new file mode 100644 index 000000000..cdf3033d9 --- /dev/null +++ b/crates/trusted-server-core/src/platform/template_assembly.rs @@ -0,0 +1,87 @@ +//! Edge assembly: turn a shared template plus a per-user fragment into a document. +//! +//! Kept behind a trait for the same reason as the template cache: the only +//! implementation that exists uses a Fastly-only crate, and core must stay portable. +//! Adapters without one get [`UnavailableTemplateAssembler`], which refuses rather than +//! guessing. +//! +//! **Ordering this module exists to protect.** The template is stored *before* assembly +//! and assembled *after* — never the reverse. Storing post-assembly would put one +//! visitor's bids in a cache shared with the next, which is the C3 the design forbids. +//! Splitting store from assemble into two call sites is what makes that ordering +//! visible instead of implicit. +//! +//! Spike-only, for the #1009 ESI validation. + +use core::fmt; + +/// Why assembly could not produce a document. +#[derive(Debug, derive_more::Display)] +pub enum TemplateAssemblyError { + /// The adapter has no assembler. + /// + /// Not a failure to be papered over: reaching here means a shared-template mode is + /// configured on an adapter that cannot serve one, and the honest response is an + /// error rather than a page with an unresolved marker in it. + #[display("this adapter cannot assemble shared templates")] + Unsupported, + /// The assembler ran and failed. + #[display("template assembly failed: {message}")] + Failed { + /// What the underlying assembler reported. + message: String, + }, +} + +impl core::error::Error for TemplateAssemblyError {} + +/// Splices a per-user fragment into a shared template. +pub trait PlatformTemplateAssembler: Send + Sync { + /// Produce the document served to this visitor. + /// + /// # Errors + /// + /// Returns [`TemplateAssemblyError::Unsupported`] when the adapter has no + /// assembler, or [`TemplateAssemblyError::Failed`] when the template could not be + /// processed. + fn assemble(&self, template: &str, fragment: &str) -> Result; +} + +impl fmt::Debug for dyn PlatformTemplateAssembler { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str("PlatformTemplateAssembler") + } +} + +/// The default: no assembler. +/// +/// Refuses rather than returning the template unchanged. Returning it unchanged would +/// serve a page whose ad markup is a literal `esi:include` — a page that looks like it +/// worked, renders no ads, and reports no error. +#[derive(Debug, Default, Clone, Copy)] +pub struct UnavailableTemplateAssembler; + +impl PlatformTemplateAssembler for UnavailableTemplateAssembler { + fn assemble(&self, _template: &str, _fragment: &str) -> Result { + Err(TemplateAssemblyError::Unsupported) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn the_null_assembler_refuses_rather_than_passing_the_template_through() { + // Passing it through is the tempting default and the wrong one: the visitor + // gets a page with a raw `esi:include` in it, no ads, and no error anywhere. + let error = UnavailableTemplateAssembler + .assemble( + "", + "", + ) + .expect_err("an adapter with no assembler must refuse"); + + assert!(matches!(error, TemplateAssemblyError::Unsupported)); + } +} diff --git a/crates/trusted-server-core/src/platform/types.rs b/crates/trusted-server-core/src/platform/types.rs index a1e48b11c..21fd164d2 100644 --- a/crates/trusted-server-core/src/platform/types.rs +++ b/crates/trusted-server-core/src/platform/types.rs @@ -173,6 +173,11 @@ pub struct RuntimeServices { /// per request rather than failing. Spike-only; see /// [`crate::platform::template_cache`]. pub(crate) template_cache: Arc, + /// Edge assembler for shared templates. Defaults to + /// [`UnavailableTemplateAssembler`], which refuses rather than serving a document + /// with an unresolved marker in it. Spike-only; see + /// [`crate::platform::template_assembly`]. + pub(crate) template_assembler: Arc, /// Dynamic backend registration and name prediction. pub(crate) backend: Arc, /// Outbound HTTP client abstraction. @@ -234,6 +239,12 @@ impl RuntimeServices { &*self.template_cache } + /// The edge template assembler. Spike-only. + #[must_use] + pub fn template_assembler(&self) -> &dyn super::PlatformTemplateAssembler { + &*self.template_assembler + } + /// Returns the dynamic backend service. #[must_use] pub fn backend(&self) -> &dyn PlatformBackend { @@ -294,6 +305,20 @@ impl RuntimeServices { ..self } } + + /// Returns a clone of this instance with the template assembler replaced. + /// + /// Spike-only (#1009). + #[must_use] + pub fn with_template_assembler( + self, + assembler: Arc, + ) -> Self { + Self { + template_assembler: assembler, + ..self + } + } } impl fmt::Debug for RuntimeServices { @@ -313,6 +338,7 @@ pub struct RuntimeServicesBuilder { secret_store: Option>, kv_store: Option>, template_cache: Option>, + template_assembler: Option>, backend: Option>, http_client: Option>, geo: Option>, @@ -327,6 +353,7 @@ impl RuntimeServicesBuilder { secret_store: None, kv_store: None, template_cache: None, + template_assembler: None, backend: None, http_client: None, geo: None, @@ -356,6 +383,16 @@ impl RuntimeServicesBuilder { self } + /// Set the edge template assembler. Spike-only. + #[must_use] + pub fn template_assembler( + mut self, + assembler: Arc, + ) -> Self { + self.template_assembler = Some(assembler); + self + } + /// Set the KV store implementation. #[must_use] pub fn kv_store(mut self, kv_store: Arc) -> Self { @@ -423,6 +460,11 @@ impl RuntimeServicesBuilder { template_cache: self .template_cache .unwrap_or_else(|| Arc::new(super::UnavailableTemplateCache)), + // Defaulted to a refusal rather than to a pass-through: an adapter with no + // assembler must not serve a template with an unresolved marker in it. + template_assembler: self + .template_assembler + .unwrap_or_else(|| Arc::new(super::UnavailableTemplateAssembler)), backend: self .backend .expect("should set backend before building RuntimeServices"), diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index 64ae53125..8896522d5 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -987,8 +987,17 @@ pub(crate) fn template_gpt_diagnostics( pub(crate) fn root_auction_is_useful(mode: AssemblyMode) -> bool { match mode { AssemblyMode::Inline => true, - // The fragment path runs its own auction; see the spike plan's Task 4. - AssemblyMode::ClientFill | AssemblyMode::Esi => false, + // The browser fetches its own bids after load, so a root auction here would be + // a second one nothing reads. + AssemblyMode::ClientFill => false, + // Consumed by edge assembly rather than by a seam. An earlier revision returned + // `false` here on the premise that the fragment would run its own auction via a + // real subrequest. That premise made the arm strictly worse — a self-referencing + // backend, two auction paths, and two auctions per pageview — and it is not what + // `esi` requires: `PendingFragmentContent::CompletedRequest` lets the include be + // satisfied from bytes already in hand. So the auction already in flight *is* + // the fragment, and it is very much consumed. + AssemblyMode::Esi => true, } } @@ -1280,7 +1289,11 @@ pub async fn buffer_publisher_response_async( ) .await?; let bytes = output.into_inner(); + // Store first, assemble second — never the reverse. The stored bytes are + // shared between visitors; the assembled ones carry this visitor's bids. + // Swapping these two lines is the C3 leak. store_template_if_authorized(services, &mut params, &bytes).await; + let bytes = assemble_if_shared(services, settings, ¶ms, bytes)?; response.headers_mut().insert( http::header::CONTENT_LENGTH, http::HeaderValue::from(bytes.len() as u64), @@ -1295,6 +1308,96 @@ pub async fn buffer_publisher_response_async( } } +/// Resolves the `` marker into this visitor's bids, if the mode assembles. +/// +/// Called *after* [`store_template_if_authorized`], never before: what is stored must +/// be the template every visitor shares, and what is returned must be this visitor's +/// document. Two call sites rather than one so that ordering is visible rather than +/// implied. +/// +/// # Errors +/// +/// Returns an error if the adapter has no assembler or if assembly fails. Deliberately +/// fatal rather than falling back to the unassembled template: that template contains a +/// literal `esi:include`, so serving it would render no ads, report no error, and look +/// to every monitor like a page that worked. +fn assemble_if_shared( + services: &RuntimeServices, + settings: &Settings, + params: &OwnedProcessResponseParams, + bytes: Vec, +) -> Result, Report> { + let assembly_mode = settings + .creative_opportunities + .as_ref() + .map(CreativeOpportunitiesConfig::assembly_mode) + .unwrap_or_default(); + if !matches!(assembly_mode, AssemblyMode::Esi) { + return Ok(bytes); + } + + let template = String::from_utf8(bytes).change_context(TrustedServerError::Proxy { + message: "shared template is not valid UTF-8, so it cannot be assembled".to_string(), + })?; + + // The auction already in flight is the fragment. `body_close_injection` emitted a + // constant marker into the template precisely so this substitution — not a + // subrequest — is what fills it. + let fragment = params + .ad_bids_state + .lock() + .expect("should lock bid state") + .clone() + .unwrap_or_else(build_empty_bids_script); + + services + .template_assembler() + .assemble(&template, &fragment) + .map(String::into_bytes) + .change_context(TrustedServerError::Proxy { + message: "failed to assemble the shared template".to_string(), + }) +} + +/// Collects the in-flight auction and assembles the cached template with its bids. +/// +/// A C2 hit skips the origin fetch, and with it the streaming pipeline that normally +/// collects the auction and fills the `` seam. Both still have to happen — the +/// auction was dispatched before the lookup and is already billing the SSPs. +/// +/// # Errors +/// +/// Returns an error if the cached bytes are not UTF-8, or if assembly fails. +async fn collect_and_assemble_cached_template( + entry: &crate::platform::TemplateEntry, + dispatched: Option, + telemetry: AuctionTelemetryCarry, + deps: &AuctionCollectDeps<'_>, +) -> Result, Report> { + if let Some(dispatched) = dispatched { + collect_stream_auction(dispatched, telemetry, deps).await; + } + + let template = core::str::from_utf8(&entry.body).change_context(TrustedServerError::Proxy { + message: "cached template is not valid UTF-8, so it cannot be assembled".to_string(), + })?; + + let fragment = deps + .ad_bids_state + .lock() + .expect("should lock bid state") + .clone() + .unwrap_or_else(build_empty_bids_script); + + deps.services + .template_assembler() + .assemble(template, &fragment) + .map(String::into_bytes) + .change_context(TrustedServerError::Proxy { + message: "failed to assemble the cached template".to_string(), + }) +} + /// Builds the response served from a C2 hit. /// /// Every header is constructed here rather than replayed from the stored entry, so no @@ -1319,11 +1422,13 @@ pub async fn buffer_publisher_response_async( /// Spike-only, for the #1009 ESI validation. fn build_cached_template_response( entry: &crate::platform::TemplateEntry, + assembled: Vec, ) -> Result, Report> { let invalid = |what: &str| TrustedServerError::Proxy { message: format!("cached template has an unusable {what}"), }; - let mut response = Response::new(EdgeBody::from(entry.body.clone())); + let assembled_len = assembled.len() as u64; + let mut response = Response::new(EdgeBody::from(assembled)); // First, not last: see the note above. The assembled response is per-user even // though the template it was built from is not. response.headers_mut().insert( @@ -1345,10 +1450,11 @@ fn build_cached_template_response( .change_context_lazy(|| invalid("content encoding"))?, ); } - response.headers_mut().insert( - header::CONTENT_LENGTH, - HeaderValue::from(entry.body.len() as u64), - ); + // The assembled length, not the template's: assembly substitutes the marker for a + // bids script, so the two differ on every request. + response + .headers_mut() + .insert(header::CONTENT_LENGTH, HeaderValue::from(assembled_len)); Ok(response) } @@ -2938,15 +3044,13 @@ pub async fn handle_publisher_request( let mut dispatched_auction = if matched_slots.is_empty() { None } else if !root_auction_is_useful(assembly_mode) { - // Shared-template modes inject nothing at the root: `template_ad_slots_script` - // and `body_close_injection` both return `None`. Dispatching here would send - // real SSP requests, hold the response for the full auction budget, and then - // discard the result with no error and no log — the silent-waste signature - // §5 of the design doc is entirely about. The fragment path runs its own - // auction; this one has no consumer. + // `ClientFill` injects nothing and the browser fetches its own bids, so + // dispatching here would send real SSP requests, hold the response for the full + // auction budget, and then discard the result with no error and no log — the + // silent-waste signature §5 of the design doc is entirely about. log::debug!( - "skipping root auction dispatch: assembly mode {assembly_mode:?} injects \ - nothing at the root" + "skipping root auction dispatch: assembly mode {assembly_mode:?} has no \ + consumer for the result" ); None } else { @@ -3160,12 +3264,34 @@ pub async fn handle_publisher_request( match services.template_cache().get(key).await { Ok(entry) => { log::debug!("c2_template_cache hit: {} bytes", entry.body.len()); + // The origin fetch is skipped, but the auction is not. It was dispatched + // above and is in flight; dropping it here would bill the SSPs for a + // result nobody reads — the silent waste the dispatch gate exists to + // prevent, reappearing on the one path that skips the pipeline which + // normally collects it. + let assembled = collect_and_assemble_cached_template( + &entry, + dispatched_auction.take(), + AuctionTelemetryCarry { + observation: auction_observation.take(), + auction_request: auction_request_for_telemetry.clone(), + }, + &AuctionCollectDeps { + price_granularity, + ad_bids_state: &ad_bids_state, + orchestrator: auction.orchestrator, + services, + settings, + request_origin: request_origin(request_scheme, request_host), + }, + ) + .await?; // Headers are constructed rather than replayed. The publisher path // rewrites `Cache-Control` and strips validators *after* the send, so // replaying stored origin headers would fight that — and constructing // them means no origin header can leak through the cache. return Ok(PublisherResponse::Buffered(build_cached_template_response( - &entry, + &entry, assembled, )?)); } Err(miss) => log::debug!("c2_template_cache miss: {miss}"), @@ -5320,152 +5446,64 @@ mod tests { mod root_auction_gate_tests { //! Guards the silent-waste failure mode: dispatching an auction whose result - //! nothing will consume. Under the shared modes both injection seams emit - //! nothing, so a dispatched root auction bills the SSPs, holds the response - //! for the full budget, and discards the result with no error and no log. + //! nothing will consume. A dispatched root auction bills the SSPs and holds the + //! response for the full budget, so discarding the result is real spend with no + //! error and no log. use super::*; use crate::creative_opportunities::AssemblyMode; #[test] - fn only_inline_has_a_consumer_for_a_root_auction() { + fn a_mode_dispatches_exactly_when_something_will_read_the_result() { + // Stated per-variant because the three modes consume the result in three + // different ways, and an earlier revision that derived this from the seam + // decision got `Esi` wrong twice in opposite directions. assert!( root_auction_is_useful(AssemblyMode::Inline), - "inline injects the auction result at ``" + "inline reads `ad_bids_state` at the `` seam" + ); + assert!( + root_auction_is_useful(AssemblyMode::Esi), + "esi consumes the result at edge assembly rather than at a seam" + ); + assert!( + !root_auction_is_useful(AssemblyMode::ClientFill), + "client-fill fetches its own bids after load, so a root auction here \ + would be a second one nothing reads" ); - for mode in [AssemblyMode::ClientFill, AssemblyMode::Esi] { - assert!( - !root_auction_is_useful(mode), - "{mode:?}: neither seam reads `ad_bids_state`, so a root auction has \ - no consumer" - ); - } - } - - #[test] - fn the_gate_agrees_with_the_injection_decisions() { - // The invariant is about *consuming* the root auction's result, not about - // emitting bytes. `InlineBids` reads `ad_bids_state`; a `Marker` is verbatim - // and reads nothing, because the fragment behind it runs its own auction. - // - // Distinguishing those is the whole point: an earlier revision equated - // "the seam emits something" with "the seam consumes the auction", which - // held only while `Esi` emitted nothing. The moment it emitted an - // `esi:include`, that reading would have dispatched a root auction whose - // result nothing reads — silent SSP spend, exactly the waste the gate - // exists to prevent. - for mode in [ - AssemblyMode::Inline, - AssemblyMode::ClientFill, - AssemblyMode::Esi, - ] { - let consumes_the_root_auction = matches!( - body_close_injection(mode, true), - BodyCloseInjection::InlineBids - ); - assert_eq!( - root_auction_is_useful(mode), - consumes_the_root_auction, - "{mode:?}: dispatch usefulness must track whether a seam reads the \ - root auction's result" - ); - } } #[test] - fn a_marker_seam_does_not_dispatch_a_root_auction() { - // The waste case, stated directly. `Esi` emits bytes at `` but reads - // nothing from `ad_bids_state`, so dispatching a root auction for it would - // buy SSP responses that are discarded. + fn consuming_the_result_is_not_the_same_as_emitting_bytes() { + // The distinction that made both earlier revisions wrong. `Esi` emits a + // `Marker` and reads nothing from `ad_bids_state` — so a gate derived from + // "does the seam emit" or from "does the seam read `ad_bids_state`" lands on + // the wrong answer. The result reaches the page through assembly, not + // through the seam. assert!(matches!( body_close_injection(AssemblyMode::Esi, true), BodyCloseInjection::Marker(_) )); - assert!( - !root_auction_is_useful(AssemblyMode::Esi), - "the fragment runs its own auction; a second one at the root is spend \ - with no consumer" - ); - } - } - - mod body_close_decision_tests { - //! The `` decision must not be inferred from the `` script. - //! - //! Coupling them is a live defect, not a hypothetical: gating the head seam - //! on template neutrality made `ad_slots_script` `None` under shared modes, - //! which silently disabled body-close injection too. These tests pin the two - //! decisions apart. - - use super::*; - use crate::creative_opportunities::AssemblyMode; - - #[test] - fn inline_injects_bids_only_when_the_head_script_is_present() { - assert_eq!( - body_close_injection(AssemblyMode::Inline, true), + assert_ne!( + body_close_injection(AssemblyMode::Esi, true), BodyCloseInjection::InlineBids, - "inline with matched slots should inject the auction result" - ); - assert_eq!( - body_close_injection(AssemblyMode::Inline, false), - BodyCloseInjection::None, - "inline without matched slots should leave the publisher's flow alone" - ); - } - - #[test] - fn shared_modes_do_not_depend_on_the_head_script() { - // The decision must be the same either way. Under a shared mode the head - // script is always absent, so a decision that read it would be - // accidentally correct here and wrong the moment that changes. - for mode in [AssemblyMode::ClientFill, AssemblyMode::Esi] { - assert_eq!( - body_close_injection(mode, true), - body_close_injection(mode, false), - "{mode:?}: body-close must not vary with head-script presence" - ); - } - } - - #[test] - fn client_fill_emits_nothing_because_the_browser_fetches_unprompted() { - assert_eq!( - body_close_injection(AssemblyMode::ClientFill, false), - BodyCloseInjection::None - ); - } - - #[test] - fn esi_emits_an_include_aimed_at_the_fragment_format() { - // Pointing at the default JSON form would splice raw JSON where an - // executable script belongs — the failure that kept this arm emitting - // nothing until the fragment format existed. - assert_eq!( - body_close_injection(AssemblyMode::Esi, false), - BodyCloseInjection::Marker(ESI_BIDS_INCLUDE.to_string()) + "the esi seam does not read the auction result" ); assert!( - ESI_BIDS_INCLUDE.contains("format=fragment"), - "the include must request the script form, not the default JSON" + root_auction_is_useful(AssemblyMode::Esi), + "and yet the auction is dispatched, because assembly reads it" ); } #[test] - fn the_esi_marker_is_identical_regardless_of_request() { - // The marker is baked into a *shared* template, so every byte in it is a - // byte every reader of that template receives. A per-request value here — - // a path, an id, a nonce — would be one visitor's data served to the next. + fn a_mode_that_injects_nothing_and_assembles_nothing_never_dispatches() { + // `ClientFill` is the one mode where both are true, and it is the case the + // silent-waste guard exists for. assert_eq!( - body_close_injection(AssemblyMode::Esi, true), - body_close_injection(AssemblyMode::Esi, false), - "the marker must not vary with request state" - ); - assert!( - !ESI_BIDS_INCLUDE.contains("path="), - "the path comes from the live request at include time, not from the \ - cached bytes" + body_close_injection(AssemblyMode::ClientFill, true), + BodyCloseInjection::None ); + assert!(!root_auction_is_useful(AssemblyMode::ClientFill)); } } @@ -5757,6 +5795,24 @@ mod tests { settings } + /// A plain substitution assembler. + /// + /// Core has no ESI crate — the Fastly adapter owns that. Substituting the marker + /// directly is what an `esi:include` with one constant marker reduces to, so + /// this exercises the seam faithfully without importing a Fastly-only crate. It + /// also demonstrates that the *seam* is portable even though the crate is not. + struct SubstitutingAssembler; + + impl crate::platform::PlatformTemplateAssembler for SubstitutingAssembler { + fn assemble( + &self, + template: &str, + fragment: &str, + ) -> Result { + Ok(template.replace(ESI_BIDS_INCLUDE, fragment)) + } + } + fn services( http_client: Arc, cache: Arc, @@ -5770,6 +5826,7 @@ mod tests { .geo(Arc::new(NoopGeo)) .client_info(ClientInfo::default()) .template_cache(cache) + .template_assembler(Arc::new(SubstitutingAssembler)) .build() } @@ -5885,6 +5942,78 @@ mod tests { response.headers().get(name).and_then(|v| v.to_str().ok()) } + #[tokio::test] + async fn the_marker_never_reaches_the_browser_on_either_path() { + // The user-visible failure this whole chain exists to avoid: a page that + // returns 200, parses fine, renders no ads, and reports no error, because + // the `esi:include` was served literally. + // + // Both paths are checked. The miss path assembles after the transform; the + // hit path assembles after reading the cache. They are separate call sites + // and only one of them running would still look like success on the other. + let stub = Arc::new(StubHttpClient::new()); + let cache = Arc::new(MemoryTemplateCache::default()); + let settings = Arc::new(settings_with_mode("esi")); + let services = services(Arc::clone(&stub), Arc::clone(&cache)); + queue_shareable_html(&stub); + + let cold = String::from_utf8(body_of( + run(&settings, &services, navigation_request()).await, + )) + .expect("cold response should be utf-8"); + let warm = String::from_utf8(body_of( + run(&settings, &services, navigation_request()).await, + )) + .expect("warm response should be utf-8"); + + assert_eq!( + stub.recorded_request_uris().len(), + 1, + "the second request must be a hit, or this only tests one path" + ); + for (label, document) in [("miss", &cold), ("hit", &warm)] { + assert!( + !document.contains("esi:include"), + "{label}: an unresolved marker reached the browser: {document}" + ); + assert!( + document.contains("window.tsjs"), + "{label}: assembly must leave a bids script behind: {document}" + ); + } + } + + #[tokio::test] + async fn the_cached_template_holds_the_marker_and_never_the_bids() { + // The ordering the C3 prohibition depends on: store before assembling. If + // these were swapped, the cache would hold one visitor's bids and serve them + // to the next — and every test above would still pass, because the served + // page would look correct. + let stub = Arc::new(StubHttpClient::new()); + let cache = Arc::new(MemoryTemplateCache::default()); + let settings = Arc::new(settings_with_mode("esi")); + let services = services(Arc::clone(&stub), Arc::clone(&cache)); + queue_shareable_html(&stub); + + let _ = run(&settings, &services, navigation_request()).await; + + let entries = cache.entries.lock().expect("should lock entries"); + let stored = entries + .values() + .next() + .expect("a template should be stored"); + let template = core::str::from_utf8(&stored.body).expect("template should be utf-8"); + + assert!( + template.contains("esi:include"), + "the cached template must still carry the unresolved marker: {template}" + ); + assert!( + !template.contains("window.tsjs"), + "the cached template must not carry a bids script: {template}" + ); + } + #[tokio::test] async fn a_cache_hit_is_never_shared_cacheable() { // Asserted positively, because the obvious negative check does not work. From 4c557347a6a7ac368e96c2c76736b6fb90a5a83e Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 13:56:42 +0530 Subject: [PATCH 37/44] Read the origin's Cache-Control, not the one TS just wrote MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Local testing found that C2 never engaged on any page where the ad stack runs — which is every page that matters. Two origin fetches for two requests, and the esi:include served unresolved. TS stamps its own `private, no-store` on the response when should_run_ad_stack is true. The C2 gate ran after that stamp, read it as the origin's declaration, concluded OriginNotShareable, and refused. The gate asks whether the *origin* said the response was shareable, so it now runs before TS writes anything. The test suite could not have caught this, and that is the more important half of the fix. Its fixture left the auction disabled and passed no dispatch slots, so should_run_ad_stack was false in every test, the stamp never fired, and the ordering was unobservable. Every assertion about C2 was therefore made against the one configuration where C2's hardest condition does not apply. The fixture now runs the ad stack for real: auction enabled, plus a slot in AuctionDispatch rather than only in settings, since should_run_ad_stack requires a matched slot and the two are different inputs. sec-fetch-mode: navigate added for the same reason. Verified by mutation both ways. With the old fixture, moving the gate back below the stamp passed all seven tests. With the corrected fixture it fails six. The bug is now observable, which it was not before. Also verified end to end under viceroy serve against a stub origin: one origin fetch for two requests, no unresolved marker on either path, a bids script present in both, private, no-store on the hit, and the cached template 353 bytes against 467 served — so the cache holds the pre-assembly template. Full gates green — fmt, six clippy targets, four adapter suites. --- crates/trusted-server-core/src/publisher.rs | 125 ++++++++++++++------ 1 file changed, 89 insertions(+), 36 deletions(-) diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index 8896522d5..ed2e59217 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -3375,6 +3375,52 @@ pub async fn handle_publisher_request( // stored or validated as an origin representation. Strip both browser and // surrogate validators/cache directives before returning it. // + // The shared-template cache gate: it does not build the key, it authorizes storing + // the one built pre-fetch. Everything it checks is response-derived, which is + // exactly why it cannot run at lookup time — and why it does not need to. Anything + // already in the cache passed this gate on the way in. + // + // A surviving key is the store authorization. Under `Inline` there is no key to + // survive. + // + // **Evaluated before TS stamps its own `private, no-store` below, and that ordering + // is load-bearing.** The gate asks whether the *origin* declared the response + // shareable. Run it after the stamp and it reads TS's own header instead, concludes + // `OriginNotShareable`, and refuses to cache — on every page where the ad stack + // runs, which is every page that matters. Local testing caught exactly that; no unit + // test did, because their fixtures leave the auction disabled and never reach the + // stamp. + let gate_content_type = response + .headers() + .get(header::CONTENT_TYPE) + .and_then(|h| h.to_str().ok()) + .unwrap_or_default() + .to_string(); + let template_cache_key = template_cache_key.filter(|_| { + match c2_bypass_reason( + assembly_mode, + request_had_authorization, + request_had_cookie, + response.status(), + &gate_content_type, + response.headers(), + &settings + .creative_opportunities + .as_ref() + .map(CreativeOpportunitiesConfig::template_cache_vary) + .unwrap_or_else(|| VarySpec::new([])), + ) { + Some(reason) => { + log::debug!("c2_template_cache bypass: {reason}"); + false + } + None => { + log::debug!("c2_template_cache eligible"); + true + } + } + }); + // Gate on `should_run_ad_stack` rather than content-type alone: when no slot // matched, the feature is disabled, or this is not an ad-eligible navigation, // no per-user `tsjs.adSlots`/`tsjs.bids` are injected, so forcing private @@ -3424,38 +3470,6 @@ pub async fn handle_publisher_request( .to_lowercase(); let route = classify_response_route(status, &content_type, &content_encoding, request_host); - // The shared-template cache gate: it does not build the key, it authorizes storing - // the one built pre-fetch. Everything it checks is response-derived, which is - // exactly why it cannot run at lookup time — and why it does not need to. Anything - // already in the cache passed this gate on the way in. - // - // A surviving key is the store authorization. Under `Inline` there is no key to - // survive. - let template_cache_key = template_cache_key.filter(|_| { - match c2_bypass_reason( - assembly_mode, - request_had_authorization, - request_had_cookie, - status, - &content_type, - response.headers(), - &settings - .creative_opportunities - .as_ref() - .map(CreativeOpportunitiesConfig::template_cache_vary) - .unwrap_or_else(|| VarySpec::new([])), - ) { - Some(reason) => { - log::debug!("c2_template_cache bypass: {reason}"); - false - } - None => { - log::debug!("c2_template_cache eligible"); - true - } - } - }); - match route { ResponseRoute::PassThrough => { log::debug!( @@ -5780,10 +5794,23 @@ mod tests { } } + /// Settings with the ad stack **live**, not merely configured. + /// + /// `[auction] enabled = true` and a slot matching the request path are both + /// required, because `should_run_ad_stack` folds them together and half this + /// path only executes when it is true. An earlier version of this helper left + /// the auction disabled, which made every test here exercise the branch where + /// TS never stamps its own `private, no-store` — and so missed that the C2 gate + /// was reading that stamp and refusing to cache every page that runs ads. fn settings_with_mode(mode: &str) -> Settings { let toml = format!( - "{}\n[creative_opportunities]\ngam_network_id = \"99999\"\n\ - assembly_mode = \"{mode}\"\n", + "{}\n[auction]\nenabled = true\n\n\ + [creative_opportunities]\ngam_network_id = \"99999\"\n\ + assembly_mode = \"{mode}\"\n\n\ + [[creative_opportunities.slot]]\n\ + id = \"test-slot\"\n\ + page_patterns = [\"/article\"]\n\ + formats = [{{ width = 728, height = 90 }}]\n", crate_test_settings_str() ); let mut settings = @@ -5850,10 +5877,36 @@ mod tests { .uri("https://ts.example.com/article") .header(header::HOST, "ts.example.com") .header("sec-fetch-dest", "document") + .header("sec-fetch-mode", "navigate") .body(EdgeBody::empty()) .expect("should build navigation request") } + /// A slot matching `/article`. + /// + /// Passed through `AuctionDispatch`, not read from settings — and that is the + /// distinction that matters. `should_run_ad_stack` requires a *matched* slot, so + /// a config slot with no dispatch slot leaves the ad stack off and skips every + /// branch that only runs when it is on. + fn article_slot() -> crate::creative_opportunities::CreativeOpportunitySlot { + crate::creative_opportunities::CreativeOpportunitySlot { + id: "test-slot".to_string(), + gam_unit_path: None, + div_id: Some("test-slot".to_string()), + page_patterns: vec!["/article".to_string()], + formats: vec![crate::creative_opportunities::CreativeOpportunityFormat { + width: 728, + height: 90, + media_type: MediaType::Banner, + }], + floor_price: None, + targeting: Default::default(), + providers: Default::default(), + compiled_patterns: Vec::new(), + compiled_unit: None, + } + } + /// Runs the full request, **including the finalizer**. /// /// The finalizer is not optional here: the store happens once the transform has @@ -5879,7 +5932,7 @@ mod tests { &mut ec_context, AuctionDispatch { orchestrator: &orchestrator, - slots: &[], + slots: &[article_slot()], registry: None, }, request, @@ -6149,7 +6202,7 @@ mod tests { &mut ec_context, AuctionDispatch { orchestrator: &orchestrator, - slots: &[], + slots: &[article_slot()], registry: None, }, authenticated, From 76df2469194e6f20f3bef69c3af419553aeec5ff Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 14:08:31 +0530 Subject: [PATCH 38/44] Close the three Task 6 gates that do not need a deployment MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-user leakage, stale revalidation, and transform failure. Each is a hard fail in the plan, independent of any performance result. Cross-user leakage is the one the design rests on. Two synthetic users differing in EC identity, consent jurisdiction and geo must store a byte-identical template — asserted as byte-identity rather than as a list of checks, because that does not depend on guessing which field might leak. Each user runs against a fresh cache, or the first user's entry would answer for the second and the comparison would prove nothing. The forbidden-substring assertions are the second layer: byte-identity would also hold if both templates leaked the same wrong thing. Transform failure covers a partial template reaching C2, which is the worst outcome available here — a truncated document served to every later visitor, indefinitely, with no error after the first request. Safe by construction, since the cap error propagates before the store; "by construction" is exactly the claim that stops holding after an unrelated refactor moves a line. The stale test needed rewriting because the first version passed for the wrong reason. A zero TTL produces an absent entry, not a stale one, so `is_stale()` was never reached — confirmed by reverting the staleness check and watching that version stay green. An entry is only present-and-stale with a stale_while_revalidate window, so the test now inserts one directly. With that fixed, the same mutation kills it. All three verified by mutation, which is the only reason to trust them: leaking adSlots through the head seam fails the leakage gate, storing before the cap check fails the transform gate, and serving stale fails the stale gate. Full gates green — fmt, six clippy targets, four adapter suites. --- .../src/template_cache.rs | 28 +++ crates/trusted-server-core/src/publisher.rs | 205 ++++++++++++++++++ 2 files changed, 233 insertions(+) diff --git a/crates/trusted-server-adapter-fastly/src/template_cache.rs b/crates/trusted-server-adapter-fastly/src/template_cache.rs index 3a32a617e..8f472138e 100644 --- a/crates/trusted-server-adapter-fastly/src/template_cache.rs +++ b/crates/trusted-server-adapter-fastly/src/template_cache.rs @@ -269,6 +269,34 @@ mod tests { ); } + #[test] + fn a_stale_but_present_entry_reads_as_a_miss_rather_than_being_served() { + // Stale-while-revalidate is a real option and deliberately not taken: it is a + // state machine `cache::core` does not implement for you, and serving stale here + // means serving a template built by an older transform or an older JS bundle. + // + // The entry has to be *present and stale*, not merely expired. A zero TTL with no + // `stale_while_revalidate` window is simply absent, so a test written that way + // passes without ever reaching `is_stale()` — verified: reverting the staleness + // check left that version green. The revalidate window is what keeps the object + // readable while stale, so this actually exercises the branch. + let key = key("https://example.com/stale"); + let body = b"stale-template".to_vec(); + let metadata = metadata_for(&body); + let cache_key = CacheKey::from(key.to_cache_key().into_bytes()); + + let mut writer = fastly::cache::core::insert(cache_key, Duration::from_secs(0)) + .stale_while_revalidate(Duration::from_secs(60)) + .user_metadata(metadata.encode().into()) + .execute() + .expect("should begin insert"); + writer.write_all(&body).expect("should write body"); + writer.finish().expect("should finish insert"); + + let miss = run(cache().get(&key)).expect_err("a stale template must not be served"); + assert_eq!(miss, TemplateCacheMiss::NotFound); + } + #[test] fn purge_all_clears_stored_templates() { // The rollback lever. Without this, backing out a bad template means waiting diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index ed2e59217..c3768b53a 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -6067,6 +6067,70 @@ mod tests { ); } + #[tokio::test] + async fn a_transform_that_overruns_its_buffer_stores_nothing() { + // The 16 MB cap in production, shrunk here. A partial template in C2 is the + // worst outcome available: it would be served to every subsequent visitor as + // a truncated document, indefinitely, with no error after the first request. + // + // Safe by construction — the cap error propagates before the store runs — but + // "by construction" is exactly the kind of claim that stops being true after + // an unrelated refactor moves one line. + let stub = Arc::new(StubHttpClient::new()); + let cache = Arc::new(MemoryTemplateCache::default()); + let mut raw = settings_with_mode("esi"); + raw.publisher.max_buffered_body_bytes = 8; + let settings = Arc::new(raw); + let services = services(Arc::clone(&stub), Arc::clone(&cache)); + queue_shareable_html(&stub); + + let orchestrator = Arc::new(AuctionOrchestrator::new(settings.auction.clone())); + let registry = + IntegrationRegistry::new(&settings).expect("should create integration registry"); + let consent = crate::consent::ConsentContext { + jurisdiction: crate::consent::jurisdiction::Jurisdiction::NonRegulated, + ..Default::default() + }; + let mut ec_context = EcContext::new_for_test(None, consent); + let publisher_response = handle_publisher_request( + &settings, + &services, + None, + &mut ec_context, + AuctionDispatch { + orchestrator: &orchestrator, + slots: &[article_slot()], + registry: None, + }, + navigation_request(), + ) + .await + .expect("the request itself should succeed; the cap trips during streaming"); + + let result = publisher_response_into_streaming_response( + publisher_response, + &Method::GET, + Arc::clone(&settings), + ®istry, + orchestrator, + services.clone(), + ) + .await; + + assert!( + result.is_err(), + "overrunning the buffer must fail rather than truncate" + ); + assert!( + cache + .entries + .lock() + .expect("should lock entries") + .is_empty(), + "a failed transform must leave nothing in the shared cache" + ); + } + #[tokio::test] async fn a_cache_hit_is_never_shared_cacheable() { // Asserted positively, because the obvious negative check does not work. @@ -6133,6 +6197,147 @@ mod tests { ); } + /// Geo that reports a fixed, recognizable location. + /// + /// A distinct value per synthetic user, so a geo leak into the template shows up + /// as a substring rather than requiring inference. + struct StubGeo(&'static str); + + impl crate::platform::PlatformGeo for StubGeo { + fn lookup( + &self, + _client_ip: Option, + ) -> Result, Report> { + Ok(Some(GeoInfo { + city: self.0.to_string(), + country: self.0.to_string(), + continent: self.0.to_string(), + latitude: 1.0, + longitude: 2.0, + metro_code: 3, + region: Some(self.0.to_string()), + asn: Some(4), + })) + } + } + + /// One synthetic user: an identity, a consent posture, and a location. + struct SyntheticUser { + ec_id: &'static str, + jurisdiction: crate::consent::jurisdiction::Jurisdiction, + geo_marker: &'static str, + } + + /// Runs one synthetic user against a fresh cache and returns the stored template. + /// + /// Fresh cache per user deliberately: the point is to compare what each *would* + /// store, so sharing a cache would let the first user's entry answer for the + /// second and the comparison would prove nothing. + async fn stored_template_for(user: &SyntheticUser) -> Vec { + let stub = Arc::new(StubHttpClient::new()); + let cache = Arc::new(MemoryTemplateCache::default()); + let settings = Arc::new(settings_with_mode("esi")); + let services = RuntimeServices::builder() + .config_store(Arc::new(NoopConfigStore)) + .secret_store(Arc::new(NoopSecretStore)) + .kv_store(Arc::new(edgezero_core::key_value_store::NoopKvStore)) + .backend(Arc::new(StubBackend)) + .http_client(Arc::clone(&stub) as Arc) + .geo(Arc::new(StubGeo(user.geo_marker))) + .client_info(ClientInfo::default()) + .template_cache( + Arc::clone(&cache) as Arc + ) + .template_assembler(Arc::new(SubstitutingAssembler)) + .build(); + queue_shareable_html(&stub); + + let orchestrator = Arc::new(AuctionOrchestrator::new(settings.auction.clone())); + let registry = + IntegrationRegistry::new(&settings).expect("should create integration registry"); + let consent = crate::consent::ConsentContext { + jurisdiction: user.jurisdiction.clone(), + ..Default::default() + }; + let mut ec_context = EcContext::new_for_test(Some(user.ec_id.to_string()), consent); + let publisher_response = handle_publisher_request( + &settings, + &services, + None, + &mut ec_context, + AuctionDispatch { + orchestrator: &orchestrator, + slots: &[article_slot()], + registry: None, + }, + navigation_request(), + ) + .await + .expect("should proxy publisher request"); + let _ = publisher_response_into_streaming_response( + publisher_response, + &Method::GET, + Arc::clone(&settings), + ®istry, + orchestrator, + services.clone(), + ) + .await + .expect("should finalize publisher response"); + + let entries = cache.entries.lock().expect("should lock entries"); + entries + .values() + .next() + .expect("a template should have been stored") + .body + .clone() + } + + #[tokio::test] + async fn two_users_differing_in_identity_consent_and_geo_store_the_same_template() { + // The gate the whole design rests on. The template is shared between + // visitors, so anything request-scoped that reaches it is one visitor's data + // served to the next. Byte-identity is the assertion because it does not + // depend on guessing which field might leak. + let alice = SyntheticUser { + ec_id: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.alice1", + jurisdiction: crate::consent::jurisdiction::Jurisdiction::NonRegulated, + geo_marker: "AliceCity", + }; + let bob = SyntheticUser { + ec_id: "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.bobbb1", + jurisdiction: crate::consent::jurisdiction::Jurisdiction::Gdpr, + geo_marker: "BobCity", + }; + + let alice_template = stored_template_for(&alice).await; + let bob_template = stored_template_for(&bob).await; + + assert_eq!( + alice_template, bob_template, + "two users differing in identity, consent and geo must produce the same \ + shared template" + ); + + // Belt and braces: byte-identity would also hold if *both* templates leaked + // the same wrong thing, so name the values that must be absent. + let template = String::from_utf8(alice_template).expect("template should be utf-8"); + for forbidden in [ + alice.ec_id, + bob.ec_id, + alice.geo_marker, + bob.geo_marker, + "adSlots", + "window.tsjs", + ] { + assert!( + !template.contains(forbidden), + "`{forbidden}` must not appear in a shared template: {template}" + ); + } + } + #[tokio::test] async fn inline_mode_never_reads_or_writes_the_cache() { // The shipped path. If this ever cached, per-user ad state would be shared From 00b8e0200a50a22930f7c1ebff7cbdcb898b0051 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 14:10:17 +0530 Subject: [PATCH 39/44] Record the local run, and the pattern the bugs on this branch share MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 6's local gates are marked done with what verified each. Two entries carry more than a checkbox. The C3 gate's wording is what caught a live bug, so the original wording is retained next to the result: checking for the absence of public/s-maxage/Surrogate-Control would have reported a hit serving with no Cache-Control at all as safe, because nothing was present to forbid. Request collapsing is left open rather than quietly dropped. Viceroy is single-threaded, so the concurrent cold-request case cannot be produced here; the racing-writer half is covered and the collapsing half is not. The findings document now records the local run and the bug it found — the C2 gate reading TS's own private, no-store as the origin's declaration, which disabled caching on every page that serves ads while every test passed. And the pattern across five bugs on this branch: each compiled, passed every existing test, and was wrong. Three were found by writing the test the plan asked for, one needed a running server, none by review — including my own review of the same gate, twice, in opposite directions. The stale-cache test is that failure in miniature: green while never reaching the branch it named, exposed only by mutation. Docs build verified, not just formatted. --- .../2026-08-08-1009-measurement-findings.md | 58 +++++++++++++++++++ .../2026-08-10-1009-esi-validation-spike.md | 42 +++++++++++--- 2 files changed, 92 insertions(+), 8 deletions(-) diff --git a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md index ea58c61b0..2ea4a7a48 100644 --- a/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md +++ b/docs/superpowers/plans/2026-08-08-1009-measurement-findings.md @@ -383,6 +383,64 @@ themselves mutation-checked. **Still not deployable.** `ClientFill` and `Esi` render a template with a hole and nothing filling it — Task 4 and Task 5. A cache that works is necessary, not sufficient. +## Local end-to-end run — the Esi arm renders + +`viceroy serve` against a stub origin, config pushed into a scratchpad `fastly.toml` so +nothing tracked was modified. Served document: + +```html +

Stub article

+
+

Body copy.

+ +``` + +No `esi:include`. One origin fetch for two requests. `private, no-store` on the hit. +Cached template 353 bytes against 467 served, so the cache holds the pre-assembly +template. All three fragment formats behave: script, JSON, and `400` on a typo. `Inline` +unaffected — two fetches for two requests, no C2 activity, no markers. + +### The bug only a running server could find + +With the auction **enabled**, C2 never engaged: two origin fetches, marker unresolved. + +TS stamps its own `private, no-store` when `should_run_ad_stack` is true. The C2 gate ran +after that stamp, read it as the origin's declaration, concluded `OriginNotShareable`, and +refused — **on every page that serves ads**, which is every page that matters. + +The more important half is why no test caught it. The fixture left the auction disabled +and passed `slots: &[]`, so `should_run_ad_stack` was false in every test, the stamp never +fired, and the ordering was unobservable. Every C2 assertion had been made against the one +configuration where C2's hardest condition does not apply. + +Demonstrated both ways: with the old fixture, reintroducing the bug passes all seven +tests; with the corrected fixture it fails six. + +### Pattern across this branch + +Five bugs now share one shape — compiled, passed every existing test, and were wrong: + +1. The head-seam gate silently disabled body-close injection (`d9e05973`). +2. The key held the encoding the origin _chose_, so the cache could never hit (`2a2e6c6a`). +3. A C2 hit served with no `Cache-Control` at all (`0adb578e`). +4. A C2 hit dropped its in-flight auction, billing SSPs for nothing (`b3ac59a6`). +5. The gate read TS's own header as the origin's (`4c557347`). + +Three were found by writing the test the plan asked for. One needed a running server. None +were found by review — including my own, twice over on the same gate. + +The stale-cache test is the same failure in miniature: it passed while never reaching +`is_stale()`, and only mutation testing exposed that. A test that passes for the wrong +reason is worse than no test, because it is counted as coverage. + ## Step B — consumers of TS's own response headers Not yet run. diff --git a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md index a62006566..592646722 100644 --- a/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md +++ b/docs/superpowers/plans/2026-08-10-1009-esi-validation-spike.md @@ -830,14 +830,32 @@ non-failed output — it is not first-success-wins. Not a phase. Every one of these is a hard fail, independent of any performance result. -- [ ] **Zero cross-user leakage.** Request the same URL as two synthetic users differing - in consent state, EC identity, and geo. Assert the C2 template is byte-identical - and that no bid, EC ID, consent string, or geo value appears in it. -- [ ] **Cold MISS, warm HIT, stale revalidation** each produce a correct page. -- [ ] **Transform failure** (the 16 MB buffer cap, a malformed body) does not insert a - partial template into C2 and does not serve one. +- [x] **Zero cross-user leakage.** DONE — `76df2469`. Two synthetic users differing in EC + identity, consent jurisdiction and geo store a byte-identical template, each against + a fresh cache so the first cannot answer for the second. Forbidden-substring checks + are the second layer, since byte-identity also holds if both leak the same thing. + Mutation-verified: leaking `adSlots` through the head seam fails it. +- [x] **Cold MISS, warm HIT, stale revalidation** DONE — `76df2469`, and end to end under + `viceroy serve` (below). Stale reads as a miss; serving stale would mean serving a + template built by an older transform or bundle. + + The first stale test passed for the wrong reason and had to be rewritten: a zero TTL + produces an *absent* entry, not a stale one, so `is_stale()` was never reached — + confirmed by reverting the check and watching it stay green. Only a + `stale_while_revalidate` window makes an entry present-and-stale. + +- [x] **Transform failure** DONE — `76df2469`. A partial template in C2 is the worst + outcome available: a truncated document served to every later visitor, indefinitely, + with no error after the first request. Mutation-verified by storing before the cap + check. - [ ] **Request collapsing** works: concurrent cold requests transform once. -- [ ] **DCA disabled**, verified by the injection test in Task 5 Step 2. +- [x] **DCA disabled** DONE — `0597f54e`. Config asserted _and_ behaviour: a fragment + carrying its own `esi:include` is spliced as text rather than dispatched. + +- [ ] **Request collapsing** — not tested, and not testable here. Viceroy is + single-threaded, so the concurrent cold-request case cannot be produced. The racing + _writer_ path is covered (`a_second_put_on_a_fresh_entry_is_a_no_op`), which is the + correctness half; the collapsing half needs real concurrency. - [ ] **Exactly one auction per pageview**, from `auction_events_raw`. - [ ] **Cookie and privacy finalization ran BEFORE assembly**, not after — EC `Set-Cookie` on first visit, geo suppression, and an unconditional @@ -849,7 +867,15 @@ Not a phase. Every one of these is a hard fail, independent of any performance r renders attributed. Use TS-attributed renders — the SSAT line item, non-empty `ts.bids`, `hb_adid` presence — **never slot fill**, which is blind to empty bids because `adInit` defines slots regardless. -- [ ] **No C3 — assert positively, not by absence.** Forbidding `public`, `s-maxage`, and +- [x] **No C3 — assert positively, not by absence.** DONE — `0adb578e`, and this gate's + wording caught a live bug. A C2 hit returns before the point where the publisher path + stamps `private, no-store`, so it served HTML with **no `Cache-Control` at all** — + heuristically cacheable, and therefore a shared cache of an assembled per-user + response. Checking for the _absence_ of `public`/`s-maxage`/`Surrogate-Control` would + have reported it as safe, because there was nothing present to forbid. Covered for + returning visitors specifically, where the cookie-privacy net never fires. + + Original wording, retained because it is what made the difference: Forbidding `public`, `s-maxage`, and `Surrogate-Control` is **not sufficient**: a bare `Cache-Control: max-age=60` passes that check and is still shared-cacheable, and that is exactly what the measured origin sends. Require instead that every assembled response carries From f1d9cb75f9afcbcb9597e0633039a994e0aef147 Mon Sep 17 00:00:00 2001 From: prk-Jr Date: Tue, 11 Aug 2026 14:31:27 +0530 Subject: [PATCH 40/44] Architect streaming assembly, and correct the latency reading MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The earlier finding said the hold costs approximately nothing. That is true today and only today, for a reason that stops holding the moment the rest of this work lands: the auction hides behind a slow origin fetch, so the hold costs max(0, auction - origin). Make the root cacheable, the origin fetch disappears, and the auction becomes the entire remaining cost. So the two problems are coupled and neither fix shows a win alone. The issue bundles them as one blocker; they need different fixes. Bids-in-the-body makes the page uncacheable and is done. Holding the response for the auction makes it slow and is not. The current implementation relocates the hold rather than removing it: a C2 hit awaits the auction, then assembles, then returns fully buffered. On a hit that is worse than today in one respect, because there is no origin fetch left to hide the auction behind. Three facts settle the design, each verified in the codebase rather than assumed. The existing streaming path already implements stream-then-stall-at-the-seam and is shipping. EdgeBody::Stream is an async stream, so an await may sit between chunks with no nested executor. BodyCloseInjection::Marker already exists and the streaming finalizers already strip Content-Length. Three designs compared. Buffered assembly is what exists. Native ESI via PendingFragmentContent::PendingRequest is what the crate is built for and is Fastly-only — and it vindicates the original dispatch gate, since under it the fragment request runs the auction and the root must not. Dispatch-usefulness turns out to be a function of the delivery mechanism, which is the non-obvious coupling here. The recommendation is neither: cache the shell with an inert HTML comment sentinel at the seam, split on it at serve time, stream the article, stall only for the auction, then write the bids and the tail. A comment rather than an esi:include because a comment is inert, so a substitution failure degrades to no ads instead of visible text in the page. Two simplifications fall out: store the template decoded and encode at serve time, which removes accept_encoding from the key entirely; and stop setting Content-Length, which is unknowable before bids resolve. The consequence for #1009 is the part worth reading. Its gating question is whether Fastly-first is acceptable for the flagship perf path. This design makes that question unnecessary — the full win on all four adapters, no esi dependency on the render path, no self-referencing backend, no second rendering architecture. ESI is sufficient but unnecessary: for one insertion point at a known location its parsing generality buys nothing a byte split does not. That is the opposite of what the issue expected. Docs build verified, not just formatted. --- ...11-1009-streaming-assembly-architecture.md | 178 ++++++++++++++++++ 1 file changed, 178 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-11-1009-streaming-assembly-architecture.md diff --git a/docs/superpowers/specs/2026-08-11-1009-streaming-assembly-architecture.md b/docs/superpowers/specs/2026-08-11-1009-streaming-assembly-architecture.md new file mode 100644 index 000000000..f911713c5 --- /dev/null +++ b/docs/superpowers/specs/2026-08-11-1009-streaming-assembly-architecture.md @@ -0,0 +1,178 @@ +# Streaming assembly: the architecture #1009 actually needs + +**Date:** 2026-08-11 +**Status:** Decision record. Supersedes the delivery half of the +[ESI validation spike](../plans/2026-08-10-1009-esi-validation-spike.md); the cache half +stands. +**Issue:** IABTechLab/trusted-server#1009 + +--- + +## 1. The correction this document exists for + +An earlier reading of the latency, recorded in +[the measurement findings](../plans/2026-08-08-1009-measurement-findings.md), said the +`` hold costs approximately nothing and the whole cost is the origin-cache bypass. + +That is true **today, and only today.** It is true for a reason that stops holding the +moment the rest of this work lands: + +| | Origin fetch | Auction | Reader waits | +| ----------------------------------- | ------------ | -------------------- | ------------ | +| Today | ~650 ms | hidden inside it | ~650–800 ms | +| Cached root, **buffered** assembly | 0 | fully exposed | ~auction cap | +| Cached root, **streaming** assembly | 0 | overlapped with send | ~ms | + +The auction is dispatched before the origin fetch and both run concurrently, so the hold +costs `max(0, auction − origin)` — zero while the origin is slow. Make the root cacheable +and the origin fetch disappears; the auction then has nothing left to hide behind and +becomes the _entire_ remaining cost. + +**So the two problems are coupled, and neither fix shows a win alone.** That is why the +issue is right to treat both as prerequisites, and why measuring one at a time misleads. + +Two distinct problems get bundled in the issue as one blocker. They need different fixes: + +1. **Bids live in the response body** → the page is _uncacheable_. Fixed by templatizing. + **Done.** +2. **The response is held for the auction** → the page is _slow_. Fixed by streaming the + shell and filling the seam late. **Not done** — this document. + +## 2. What the current implementation gets wrong + +On a C2 hit, `collect_and_assemble_cached_template` awaits the auction, then assembles, +then returns a fully buffered `PublisherResponse::Buffered`. The reader receives nothing +until bids resolve. + +That relocates the hold rather than removing it, and on a hit it is _worse than today_ in +one respect: there is no origin fetch left to hide it behind, so the full auction latency +lands on first byte. + +The routing decision that caused it — shared modes take the buffered finalizer — was made +because **a store needs complete transformed bytes.** True on a miss. Irrelevant on a hit, +where the template is already materialized. + +## 3. The decisive facts + +Three, all verified in the codebase rather than assumed: + +1. **The existing streaming path already implements stream-then-stall-at-the-seam.** + `publisher.rs` builds an `async_stream::try_stream!` that streams body chunks and holds + **only** at `` for the auction (`hold_auction`, `AuctionHoldState`). This is + shipping behaviour, not new work. +2. **`EdgeBody::Stream` is an async stream** — consumers call `stream.next().await` — so an + `await` may sit between chunks. Nothing needs a nested executor. +3. **`BodyCloseInjection::Marker(String)` already exists**, and the streaming finalizers + already strip `Content-Length`. + +## 4. Three designs + +| | Streams | Auctions | Requires | Adapters | +| ----------------------------------- | ------- | -------------- | ------------------------ | --------- | +| **A** — buffered assembly (current) | No | 1 | nothing | Fastly | +| **B** — native ESI subrequest | Yes | 1, in fragment | self-referencing backend | Fastly | +| **C** — cached shell + seam split | Yes | 1 | nothing | **All 4** | + +### Design B, for the record + +`PendingFragmentContent::PendingRequest` is what the `esi` crate is built for: the +dispatcher fires a real subrequest and the processor blocks on the handle. Fastly's +`send_async`/`wait` is **synchronous**, so this sidesteps the sync-dispatcher problem +without any executor. + +It also vindicates the _original_ dispatch gate. Under B the root must **not** dispatch, +because the fragment request runs the auction. The later reversal to +`root_auction_is_useful(Esi) = true` is correct for buffered assembly and wrong for +streaming. **Dispatch-usefulness is a function of the delivery mechanism**, which is the +non-obvious coupling in this design space. + +### Design C — the recommendation + +The template carries an **inert HTML comment sentinel** where the bids go, emitted by the +existing `Marker` variant: + +``` + +``` + +On a C2 hit: + +``` +commit headers (private, no-store; no Content-Length) ← must precede any byte on Fastly +stream template[..sentinel] ← the article paints here +await the auction ← the only stall, at the very end +write the bids script +stream template[sentinel+len..] +``` + +Since a hit has the whole template in hand, this is a `split_once`, not a streaming +search. Three yields from a `try_stream!`. + +**Why a comment sentinel rather than a byte offset in metadata.** An offset is O(1), but +capturing it means plumbing the writer position into a `lol_html` end-tag handler, and it +does not survive re-encoding. A `find` over a ~100 KB buffered template is free by +comparison. + +**Why a comment rather than `esi:include`.** An HTML comment is inert. If assembly ever +fails to substitute, the reader sees nothing; an unresolved `esi:include` renders as +visible text. Failure degrades to "no ads" instead of "broken page". + +**Why not re-run `lol_html` over the cached template.** It would inject a second tsjs +`