Raised in docs/presentations/se.md §9.6. Filed prerequisite-gated, in the style of #333/#335/#341: not work to start, but a named trigger so the question is answered before it is forced.
The current shape, and why it is right today
Persistence is SQLite in WAL mode via a hand-rolled Repository — no ORM, plain SQL, hand-written mapping. That is a good fit for what keel is:
- One writer. One account, one venue, one cycle per interval, one process. The agent is synchronous by design (see
se.md §10 — concurrency buys nothing here and costs testability).
- The audit trail is the schema. Every order, veto and state transition is a row the engine reads back to make its next decision. Hand-written SQL keeps that inspectable.
- The database is a file. It can be copied, diffed, attached to a bug report, and backed up before a migration — which
keel update does automatically (*.db.bak-before-<version>-<stamp>).
None of that is accidental, and this issue is not a proposal to change it.
What would actually strain it
Recording the shapes, so the trigger is concrete rather than a vague "when we scale":
- Concurrent writers. Today's deployment runs three profiles — live, paper-daily, paper-hourly — but each has its own database file, so there is still one writer per file. A design that had two processes writing one file (e.g. a supervisor alongside the agent, or a web layer that writes) is the first genuine break. WAL permits concurrent readers with one writer; it does not make two writers safe.
- Cycle frequency. The live wrapper trades once per UTC day; the hourly evidence profile cycles hourly. Neither is close to write-bound. A sub-minute cycle with per-tick persistence would be a different question.
- Candle volume. The deployment's
keel.db already holds ~129MB, mostly candles, and grows with every fetched product. Read performance on the median-volume statistic and backtests is the thing to watch, not write contention.
- Long-running transactions during a migration.
keel migrate runs per database on update; a schema change over a large candle table is the one operation with a real stall risk.
The trigger — revisit when any of these fires
- A second process needs to write the same database file.
- A deployment profile moves to a cycle faster than hourly.
- A migration on the candle tables takes long enough to threaten a cycle window.
- Read latency on the liquidity/backtest paths becomes a measured complaint, not a suspicion.
Until one fires, the answer is "SQLite, and here is why" — and that answer should be written down so it stops being re-litigated by each new reader.
Acceptance criteria
Notes
Raised in
docs/presentations/se.md§9.6. Filed prerequisite-gated, in the style of #333/#335/#341: not work to start, but a named trigger so the question is answered before it is forced.The current shape, and why it is right today
Persistence is SQLite in WAL mode via a hand-rolled
Repository— no ORM, plain SQL, hand-written mapping. That is a good fit for what keel is:se.md§10 — concurrency buys nothing here and costs testability).keel updatedoes automatically (*.db.bak-before-<version>-<stamp>).None of that is accidental, and this issue is not a proposal to change it.
What would actually strain it
Recording the shapes, so the trigger is concrete rather than a vague "when we scale":
keel.dbalready holds ~129MB, mostly candles, and grows with every fetched product. Read performance on the median-volume statistic and backtests is the thing to watch, not write contention.keel migrateruns per database on update; a schema change over a large candle table is the one operation with a real stall risk.The trigger — revisit when any of these fires
Until one fires, the answer is "SQLite, and here is why" — and that answer should be written down so it stops being re-litigated by each new reader.
Acceptance criteria
Notes
keel/data/db.py::connect("not a tuning preference"), which is the right precedent for how this decision should be recorded.