perf/test: cache library previews and serve E2E with workers - #303
Merged
HugoFara merged 2 commits intoAug 30, 2026
Merged
Conversation
The suggestion panels on /texts/new and the home page request a difficulty preview for every book they list, and each preview downloaded the whole book again: one call measured at 5.4 s, all of it the fetch, against milliseconds for the tokenizing and vocabulary lookup that follow. Nothing was cached, so opening the page refetched the same handful of books every time and pinned a PHP worker per download for the duration. Adds a small file-backed FileCache and puts the fetched source document behind it, keyed by URL with a 24 h TTL matching the Gutenberg suggestion cache. Same call now: 5.4 s cold, 0.02 s warm. Only the document is shared — coverage and difficulty are still computed per user on top of it, so the figures stay personal. FileCache degrades to a miss on every failure it can hit: an unwritable temp directory, an unreadable entry, an expired one. The cost of a miss is a refetch, and a cache that can throw on a page load would be worse than no cache at all. Writes go to a temp file and rename, so a reader never sees a half-written entry and two writers cannot interleave.
Three specs in 05-texts.cy.ts failed, and the symptoms pointed at the
application: text creation "did not redirect", and /text/edit failed to load
with ESOCKETTIMEDOUT. Neither was true. The POST succeeded, the redirect to
/text/{id}/read was issued, and the browser was already requesting it — the
request was simply queued behind other work and arrived after Cypress's 10 s
assertion timeout. Measured on the documented setup: a page that takes 11 ms
on an idle server took 28.5 s while difficulty previews were in flight.
The cause is `php -S`, which serves one request at a time unless
PHP_CLI_SERVER_WORKERS is set. Any slow request — a text import, an outbound
fetch — stalls the whole suite behind it, and the failure surfaces somewhere
unrelated. Adds `npm run serve` as the one way to start the dev server, with
workers, and points CLAUDE.md and both copies of the contributing guide at it
with the reason.
With that and the source-document cache, the full suite goes from 4 failures
in 6:49 to 304 passing in 3:03 from a cold cache.
HugoFara
force-pushed
the
test/e2e-reliability
branch
from
August 30, 2026 22:14
5d2d17c to
f87c35a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack 6 of 6. Base: #302. Fixes three E2E failures that were not
application bugs, plus the production performance problem underneath them.
Two specs in
05-texts.cy.tsreported that text creation "did not redirect",and a third failed with
ESOCKETTIMEDOUToncy.visit('/text/edit'). None ofthat was true. The
POSTreturned 200, the redirect to/text/{id}/readwasissued, and the browser was already requesting it — the request was simply
queued and arrived after Cypress's 10 s assertion timeout. Measured on the
documented setup: a page taking 11 ms on an idle server took 28.5 s
while difficulty previews were in flight.
First commit — the real cost (this one matters in production)
The suggestion panels on
/texts/newand the home page request a difficultypreview for every book they list, three at a time, walking the whole list. Each
preview downloaded the entire book from Gutenberg — one call measured at
5.4 s, essentially all of it network, against milliseconds for the tokenizing
and vocabulary lookup that follow. Nothing was cached, so every page load
refetched the same books and pinned a PHP worker per download.
Adds a small file-backed
FileCacheand puts the fetched document behind it,keyed by URL, 24 h TTL matching the existing Gutenberg suggestion cache. Same
call: 5.4 s cold → 0.02 s warm. Only the document is shared — coverage and
difficulty are still computed per user on top of it, so the figures stay
personal.
FileCachedegrades to a miss on every failure it can hit: unwritable tempdirectory, unreadable entry, expired entry. The cost of a miss is a refetch, and
a cache that can throw on a page load would be worse than no cache. Writes go to
a temp file and rename, so a reader never sees a half-written entry. 11 unit
tests, including that an empty value is not stored (caching a failed fetch would
pin the failure for the whole TTL) and that a key like
../../escapecannotsteer the write out of the directory.
Second commit — the dev server
php -Sserves one request at a time unlessPHP_CLI_SERVER_WORKERSisset. That is what turned one slow request into a suite-wide stall, and why the
failure surfaced somewhere unrelated to its cause. Adds
npm run serveas theone way to start the dev server, and points
CLAUDE.mdand both copies of thecontributing guide at it with the reason.
This is dev-only: a real deployment runs Apache or php-fpm, which is
multi-process already.
Result
Each half was verified alone: caching alone still failed (3 failures);
workers alone passed. Both are needed — the cache for the real cost, workers
so one slow request cannot starve everything.
Full suite, cold cache: 304/304 passing, 23/23 specs, 3:03. Previously 4
failures in 6:49 —
99-screenshots.cy.tsalso failed and now clears.Stack: #298 → #299 → #300 → #301 → #302 → 6