fix: Retry after partial file reads. - #218
Conversation
f3aaa08 to
d1f90f1
Compare
The two reload tests asserted an exact ExpectedDataSetForSegmentOnlyFile(2), which encoded an assumption of exactly two LoadAll attempts. With the file watcher firing on truncate-then-write plus the new JSON-parse retry, the successful attempt's version can be 3+ and the strict JSON comparison times out waiting for a version-2 event that never comes. Switch both tests to ExpectPredicate with a shared structural matcher and refactor the existing predicate in ModifiedFileIsReloadedIfAutoUpdateIsOn to use the same helper. Also tighten the retry path in FileDataSource: drop the trailing TODO, short-circuit LoadAll once Dispose() has been called, and skip the post- delay LoadAll if disposal raced the retry.
… autoUpdate. Review round-1 fixes: - Reset the retry budget when a failure is observed on an externally triggered load (Start or file-change notification), so a later partial-write episode still gets retries after an earlier episode exhausted the counter. - Treat any parser failure as retryable, not only JsonException, so alternate parsers (e.g. YAML) get the same partial-read handling. - Keep at most one pending retry, and skip it if an intervening load already succeeded, to avoid redundant Inits and spurious change events. - Do not retry when autoUpdate is off, preserving the documented load-once semantics. - Log the parse exception summary on retry and full detail on give-up; catch exceptions escaping the retry task so they are not unobserved. - Add deterministic IFileReader-driven tests for retry recovery, the attempt cap, episode reset, YAML parsing, autoUpdate(false), and dispose cancellation; episode reset, YAML, and autoUpdate(false) tests fail on the previous implementation.
… external loads. Review round-2 fixes: - Externally triggered loads clear the per-path failure counts up front, and give-up clears them as well, so a stale count from a dead retry chain can never be charged to a later episode (which previously caused an immediate give-up with zero retries on multi-path configs). - When one path's give-up ends the chain, other paths that were promised retries get a terminal error log naming them. - Remove a path's failure count as soon as it parses, before the merge step, so a duplicate-key configuration error cannot strand it. - Log the exception stack trace at debug level on each retry. - Correct the AutoUpdate doc: retries are bounded per detected change, not unconditional. - Tests: deterministic multi-path episode tests (fresh budget after another path's give-up; terminal log for abandoned paths) that fail on the previous implementation; a redundant-reload guard test; waits now fail loudly on timeout; the dispose test no longer depends on running within the retry delay.
…sing. Addresses Bugbot review: a transient read error (for example, a writer replacing the file) during a retry previously ended the episode with budget remaining and no terminal log, recreating the stuck-stale-data state this PR targets. Such failures are now charged to the same per-path budget: the chain schedules the next retry until the budget is exhausted, then ends the episode with terminal logs. Externally triggered loads keep the existing non-retrying behavior for non-parse errors. New test fails on the previous implementation.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 64cb514. Configure here.
| // file-change notification is not guaranteed. Charge the failure to | ||
| // the same per-path budget and continue the chain. | ||
| HandleRetryLoadFailure(path); | ||
| } |
There was a problem hiding this comment.
Error logged before retriable failure
Low Severity
LogHelpers.LogException always runs before HandleRetryLoadFailure, so every retriable read failure during a retry episode is logged at Error even though a Warn retry follows. That diverges from HandleParseFailure, which keeps retriable failures at Warn and reserves Error for giving up, and can falsely look like a terminal load failure to operators and monitors.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 64cb514. Configure here.


When we detect a file has been written there isn't any promise that the file content itself will actually be complete, or that a subsequent notification for that file will be received when it is.
Currently sometimes the file update will trigger, and the JSON parsing will fail, and then it won't get re-triggered. Which causes some tests to fail.
This introduces a retry for any failed JSON parsing up to a maximum number of times.
Ideally we would extract the file watcher and abstract the file system API so we could simulate these conditions.
Note
Overview
When auto-update is enabled,
FileDataSourcenow retries loads after parse failures (e.g. reading a file mid-write), up to 5 attempts with a 600 ms delay between tries, instead of stopping until another file-change event.Retries are scoped to per-episode failure budgets: a new
Start()or watcher reload clears counts and starts fresh. Auto-update off keeps the documented one-shot load—no background retries. Multi-path loads share a single retry chain; if one path exhausts its budget, other paths that were waiting on retries get terminal error logs and wait for the next change.Dispose cancels pending retries; a successful external reload before a scheduled retry avoids redundant
Initand spurious change events. Transient read errors during a retry episode are charged to the same per-path budget as parse failures.Builder docs for
AutoUpdatedescribe the bounded retry behavior. Tests use scripted file readers and relaxed reload assertions where watcher timing makes versions non-deterministic.Reviewed by Cursor Bugbot for commit 64cb514. Bugbot is set up for automated code reviews on this repo. Configure here.