Skip to content

Projection pipeline: make projector dispatch and ordering explicit, and stop paying for projectors that ignore an event #5856

Description

@cheruvian

Follow-up to #5719 / #5855. This is a proposal to test appetite before anyone reviews a large diff — happy to close it if the direction isn't wanted.

What I ran into

While fixing #5719 I needed to prove that narrowing the thread shell-summary refresh was safe on replay. Two properties of ProjectionPipeline.ts made that harder than expected, and both look worth addressing on their own merits.

1. The run order is load-bearing but undeclared.

The threads projector derives its shell summary by counting rows that the message, activity, proposed-plan and approval projections write for the same event. It is correct only because it sits last in the projectors array. Nothing states that, and nothing checks it. Reordering that array, or making the loop concurrent, silently produces stale derived state rather than failing.

2. Every event pays for all nine projectors.

runProjectorForEvent runs each projector inside a transaction and writes its cursor row, whether or not that projector handles the event type. For thread.message-sent, 3 of 9 projectors have a case; the other 6 open a savepoint and write a projection_state row to do nothing. The checkpoints projector handles nothing at all — its state is actually projected by the turns projector — and still costs a cursor write per event.

After #5855, roughly two-thirds of the remaining per-delta database work is this bookkeeping.

Proposed direction

Step 1 — declarations. Each projector declares an on map keyed by event type, and a reads list naming the projections it queries. Dispatch comes from the key set; run order comes from a topological sort of reads, which throws at construction on a cycle, an unknown dependency or a duplicate name. The key set is the subscription, so it cannot drift from the implementation the way a separate handles: [...] array would. Purely structural — no behavior change.

Step 2 — stop paying for non-handlers. With dispatch derived, skip projectors that do not handle an event, collapse the nine per-event cursor writes into one statement, and drop the inner withTransaction on the live path (it is a savepoint nested inside the command transaction, and a projector failure rolls the whole command back anyway, so it does no work there — it matters only during bootstrap, where there is no enclosing transaction).

Step 3 — optional. One module per projector. Mostly a readability change, but it makes each projector's repository dependencies visible instead of inferred from a shared closure over eleven repositories.

Why this order

Step 2 is where the measurable win is, and it is only safe to write once dispatch can be derived from declarations rather than guessed. Step 1 is the enabling change and is independently useful, since it converts a silent-corruption failure mode into a startup error.

Measurement instrument

#5855 adds a test-only SQL client decorator that records every statement the pipeline issues, so these claims are falsifiable rather than architectural taste. It currently pins 112 statements and zero full-thread scans for eight streamed deltas, and equal counts against a 4-activity and a 200-activity thread. The same harness is the natural acceptance gate for steps 1 and 2.

Status

Steps 1 and 3 are written and green (305 tests, typecheck and lint clean, statement counts identical before and after, so no behavior change). I have deliberately not opened PRs for them, because they restructure a core file and that is a bigger ask than a bug fix. If the direction is welcome I will send them as reviewable stacked PRs; if you would rather keep the switches, #5855 stands on its own and this can be closed.

Model and harness: Claude Opus 5 (1M context) in Claude Code.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions