Skip to content

perf/test: cache library previews and serve E2E with workers - #303

Merged
HugoFara merged 2 commits into
refactor/client-side-annotated-displayfrom
test/e2e-reliability
Aug 30, 2026
Merged

perf/test: cache library previews and serve E2E with workers#303
HugoFara merged 2 commits into
refactor/client-side-annotated-displayfrom
test/e2e-reliability

Conversation

@HugoFara

Copy link
Copy Markdown
Owner

Stack 6 of 6. Base: #302. Fixes three E2E failures that were not
application bugs
, plus the production performance problem underneath them.

Two specs in 05-texts.cy.ts reported that text creation "did not redirect",
and a third failed with ESOCKETTIMEDOUT on cy.visit('/text/edit'). None of
that was true. The POST returned 200, the redirect to /text/{id}/read was
issued, and the browser was already requesting it — the request was simply
queued and arrived after Cypress's 10 s assertion timeout. Measured on the
documented setup: a page taking 11 ms on an idle server took 28.5 s
while difficulty previews were in flight.

First commit — the real cost (this one matters in production)

The suggestion panels on /texts/new and the home page request a difficulty
preview for every book they list, three at a time, walking the whole list. Each
preview downloaded the entire book from Gutenberg — one call measured at
5.4 s, essentially all of it network, against milliseconds for the tokenizing
and vocabulary lookup that follow. Nothing was cached, so every page load
refetched the same books and pinned a PHP worker per download.

Adds a small file-backed FileCache and puts the fetched document behind it,
keyed by URL, 24 h TTL matching the existing Gutenberg suggestion cache. Same
call: 5.4 s cold → 0.02 s warm. Only the document is shared — coverage and
difficulty are still computed per user on top of it, so the figures stay
personal.

FileCache degrades to a miss on every failure it can hit: unwritable temp
directory, unreadable entry, expired entry. The cost of a miss is a refetch, and
a cache that can throw on a page load would be worse than no cache. Writes go to
a temp file and rename, so a reader never sees a half-written entry. 11 unit
tests, including that an empty value is not stored (caching a failed fetch would
pin the failure for the whole TTL) and that a key like ../../escape cannot
steer the write out of the directory.

Second commit — the dev server

php -S serves one request at a time unless PHP_CLI_SERVER_WORKERS is
set. That is what turned one slow request into a suite-wide stall, and why the
failure surfaced somewhere unrelated to its cause. Adds npm run serve as the
one way to start the dev server, and points CLAUDE.md and both copies of the
contributing guide at it with the reason.

This is dev-only: a real deployment runs Apache or php-fpm, which is
multi-process already.

Result

Each half was verified alone: caching alone still failed (3 failures);
workers alone passed. Both are needed — the cache for the real cost, workers
so one slow request cannot starve everything.

Full suite, cold cache: 304/304 passing, 23/23 specs, 3:03. Previously 4
failures in 6:49 — 99-screenshots.cy.ts also failed and now clears.

Stack: #298#299#300#301#3026

The suggestion panels on /texts/new and the home page request a difficulty
preview for every book they list, and each preview downloaded the whole book
again: one call measured at 5.4 s, all of it the fetch, against milliseconds
for the tokenizing and vocabulary lookup that follow. Nothing was cached, so
opening the page refetched the same handful of books every time and pinned a
PHP worker per download for the duration.

Adds a small file-backed FileCache and puts the fetched source document behind
it, keyed by URL with a 24 h TTL matching the Gutenberg suggestion cache. Same
call now: 5.4 s cold, 0.02 s warm. Only the document is shared — coverage and
difficulty are still computed per user on top of it, so the figures stay
personal.

FileCache degrades to a miss on every failure it can hit: an unwritable temp
directory, an unreadable entry, an expired one. The cost of a miss is a
refetch, and a cache that can throw on a page load would be worse than no
cache at all. Writes go to a temp file and rename, so a reader never sees a
half-written entry and two writers cannot interleave.
Three specs in 05-texts.cy.ts failed, and the symptoms pointed at the
application: text creation "did not redirect", and /text/edit failed to load
with ESOCKETTIMEDOUT. Neither was true. The POST succeeded, the redirect to
/text/{id}/read was issued, and the browser was already requesting it — the
request was simply queued behind other work and arrived after Cypress's 10 s
assertion timeout. Measured on the documented setup: a page that takes 11 ms
on an idle server took 28.5 s while difficulty previews were in flight.

The cause is `php -S`, which serves one request at a time unless
PHP_CLI_SERVER_WORKERS is set. Any slow request — a text import, an outbound
fetch — stalls the whole suite behind it, and the failure surfaces somewhere
unrelated. Adds `npm run serve` as the one way to start the dev server, with
workers, and points CLAUDE.md and both copies of the contributing guide at it
with the reason.

With that and the source-document cache, the full suite goes from 4 failures
in 6:49 to 304 passing in 3:03 from a cold cache.
@HugoFara
HugoFara force-pushed the test/e2e-reliability branch from 5d2d17c to f87c35a Compare August 30, 2026 22:14
@HugoFara
HugoFara merged commit 27ac193 into develop Aug 30, 2026
14 checks passed
HugoFara added a commit that referenced this pull request Aug 30, 2026
Covers #298-#303: the annotated display view moving to the client, Globals
losing the members that did nothing, every PHP-to-JS config blob going through
one emitter, and the preview caching that also fixes the E2E timeouts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant