fix: flaky unit tests from virtual-time timeouts racing real datastore i/o

[jvsena42]

Three unit tests fail intermittently on CI and never locally. They share one root cause: a withTimeoutOrNull measured on runTest's virtual clock, racing work that crosses a real dispatcher.

Affected tests

Test	Location
QuickPayRepoTest > stale failure event does not settle a dispatch the ldk row shows pending	QuickPayRepoTest.kt:714
QuickPayRepoTest > reconcile during live dispatched op does not steal completion	QuickPayRepoTest.kt:1008
LightningServiceTest > stop destroys the node handle when it is already not running	LightningServiceTest.kt:126

Observed occurrences

Not tied to any one branch — all three have failed on master itself:

• CI 33082462001 (master, 2026-08-27) — reconcile during live dispatched op
• CI 32989065879 (master, 2026-08-26) — stop destroys the node handle when it is already not running
• CI 33170079347 (PR fix: restore androidtest hilt bindings #1192, commit bd66a2142) — failed on three consecutive attempts, each with a different subset:
  • attempt 1: stale failure event does not settle a dispatch...
  • attempt 2: reconcile during live dispatched op... and stop destroys the node handle...
  • attempt 3: reconcile during live dispatched op...

Roughly 6 of the last 20 CI runs across branches failed this way.

Root cause

QuickPayRepoTest builds a real, file-backed DataStore:

private val cacheStore = CacheStore(context)            // QuickPayRepoTest.kt:110

private val Context.appCacheDataStore by dataStore(     // CacheStore.kt:25
    fileName = "app_cache.json",
    serializer = AppCacheSerializer,
)

The AndroidX dataStore delegate defaults its scope to Dispatchers.IO, so every cacheStore.data.first() and cacheStore.update {} is real file I/O on a real dispatcher, executed inside a virtual-time runTest.

While the test coroutine is parked on that I/O, the test scheduler has no runnable task, so runTest fast-forwards virtual time to the next scheduled task — which is the armed timeout in QuickPayRepo.awaitCompletionOrPending:

withTimeoutOrNull(LightningRepo.SEND_LN_TIMEOUT) { current.settled.await() }   // QuickPayRepo.kt:350
mutex.withLock { opsByKey[invoiceHash]?.let { emitPendingLocked(it) } }

The timeout fires, Pending is emitted, and op.emitted is a one-shot latch — so the subsequent signalCompletion hits emitSuccessLocked, sees emitted == true, and returns without emitting:

private fun emitSuccessLocked(op: InFlightOp, feePaidMsat: ULong?): Boolean {
    if (op.emitted) { op.settled.complete(Unit); return false }   // QuickPayRepo.kt:677

awaitItem() therefore observes Pending where the test asserts Success — matching the CI output exactly:

Expected value to be of type <QuickPaySessionEvent.Success>, actual <QuickPaySessionEvent$Pending>.

LightningServiceTest is the same shape: stop() hops through ServiceQueue.LDK.background, which is withContext on a real single-thread executor (ServiceQueue.kt), while releaseHandle joins under withTimeoutOrNull(NODE_RELEASE_TIMEOUT) with NODE_RELEASE_TIMEOUT = 1.seconds. The real hop lets virtual time jump past 1s, the join is abandoned, and node.destroy() never runs:

Wanted but not invoked: node.destroy();
However, there was exactly 1 interaction with this mock: node.stop()

Why it is CI-only

The jump only happens if the scheduler actually idles during the real I/O. On a fast local SSD the DataStore write typically completes without yielding; on CI's slower and contended I/O it idles reliably.

Not reproducible locally: ~12 executions on an M-series Mac (7 full-suite runs, 3 QuickPayRepoTest-only runs, 2 two-class runs under 8-way CPU contention) produced zero failures.

Note for whoever picks this up

Raising the timeout does not fix it. runTest advances to whichever delayed task is next, so a larger value just produces a larger jump and the timeout still fires.

The fix is to remove the real I/O from these tests — an in-memory CacheStore for unit tests. That needs an extracted interface or an injected DataStore, since CacheStore is currently a concrete class wrapping a Context delegate. Reordering assertions is not sufficient: reconcileAgainstLdk() itself touches the store inside the racy window.

Verification will have to be done on CI across repeated runs, since the failure does not appear locally.