From 70277cf4e187a75c5cf7366d2370247886e1bff7 Mon Sep 17 00:00:00 2001 From: Alexey Zimarev Date: Wed, 12 Aug 2026 18:49:54 +0200 Subject: [PATCH 1/2] docs: streaming reads and ReadStreamToEnd Document the memory semantics of IEventReader reads (KurrentDB streams, relational stores buffer up to count) and the new ReadStreamToEnd extension for reading a whole stream with bounded memory, replacing the count: int.MaxValue idiom. Matches Eventuous/eventuous#568. Co-Authored-By: Claude Fable 5 --- .../dotnet-next/persistence/event-store.md | 37 ++++++++++++++----- src/content/docs/dotnet-next/whats-new.mdx | 14 +++++++ 2 files changed, 42 insertions(+), 9 deletions(-) diff --git a/src/content/docs/dotnet-next/persistence/event-store.md b/src/content/docs/dotnet-next/persistence/event-store.md index ec53fac..af05de5 100644 --- a/src/content/docs/dotnet-next/persistence/event-store.md +++ b/src/content/docs/dotnet-next/persistence/event-store.md @@ -29,14 +29,15 @@ All of those are immutable records. The event store provides the following operations: -| Function | What's it for | -|------------------|------------------------------------------------------------------| -| `AppendEvents` | Append one or more events to a given stream. | -| `AppendEvents` | Append events to multiple streams in a single operation. | -| `ReadEvents` | Read events from a stream forwards, from a given start position. | -| `StreamExists` | Check if a stream exists. | -| `TruncateStream` | Remove events from a stream up to a given position. | -| `DeleteStream` | Delete a stream entirely. | +| Function | What's it for | +|----------------------|-------------------------------------------------------------------| +| `AppendEvents` | Append one or more events to a given stream. | +| `AppendEvents` | Append events to multiple streams in a single operation. | +| `ReadEvents` | Read events from a stream forwards, from a given start position. | +| `ReadEventsBackwards`| Read events from a stream backwards, from a given start position. | +| `StreamExists` | Check if a stream exists. | +| `TruncateStream` | Remove events from a stream up to a given position. | +| `DeleteStream` | Delete a stream entirely. | ## Usage examples @@ -124,7 +125,25 @@ foreach (var evt in events) { } ``` -The `ReadStream` extension method handles pagination automatically and returns all events: +The `IEventReader.ReadEvents` interface method returns `IAsyncEnumerable`. `KurrentDBEventStore` streams events as they arrive from the server, holding at most one deserialized event at a time. Relational stores buffer up to `count` events per call before yielding, so memory usage can grow with `count` — keep the count bounded when reading from those stores. + +### Reading a whole stream + +To read a stream to the end, use the `ReadStreamToEnd` extension method instead of calling `ReadEvents` with `int.MaxValue` as the count. It reads events in pages (500 by default) and yields them as they arrive, so memory use stays bounded by the page size no matter how large the stream is: + +```csharp +await foreach (var evt in eventStore.ReadStreamToEnd( + streamName, + StreamReadPosition.Start, + cancellationToken: cancellationToken +)) { + // Process one event at a time with bounded memory use +} +``` + +Use the `pageSize` parameter to tune the page size, and `failIfNotFound: false` to get an empty sequence instead of a `StreamNotFound` exception when the stream doesn't exist. + +If you need the whole stream as an array, the `ReadStream` extension method pages through the stream the same way and collects all events: ```csharp var allEvents = await eventStore.ReadStream( diff --git a/src/content/docs/dotnet-next/whats-new.mdx b/src/content/docs/dotnet-next/whats-new.mdx index d4b40ac..24c7c4e 100644 --- a/src/content/docs/dotnet-next/whats-new.mdx +++ b/src/content/docs/dotnet-next/whats-new.mdx @@ -52,8 +52,22 @@ await foreach (var evt in eventReader.ReadEvents(stream, start, count, ct)) { } ``` +`KurrentDBEventStore` yields each event as it arrives from the server, holding at most one deserialized event at a time regardless of the requested count. Relational stores buffer up to `count` events per call, so memory usage can grow with `count` — keep the count bounded when reading from those stores. + The previous signatures (with `failIfNotFound`, returning `Task`) are preserved as extension methods in `StoreFunctions`, so **all existing callers compile and work unchanged**. +### Read a whole stream with ReadStreamToEnd + +To read a stream to the end, use the new `ReadStreamToEnd` extension method instead of calling `ReadEvents` with `int.MaxValue` as the count. It reads events in pages (500 by default) and yields them as they arrive, so memory use stays bounded by the page size no matter how large the stream is: + +```csharp +await foreach (var evt in eventReader.ReadStreamToEnd(stream, StreamReadPosition.Start, cancellationToken: ct)) { + // Process one event at a time with bounded memory use +} +``` + +Use `pageSize` to tune the page size, and `failIfNotFound: false` to get an empty sequence instead of a `StreamNotFound` exception when the stream doesn't exist. The `ReadStream` extension method pages through the stream the same way and collects the result into an array. + ### SignalR subscription gateway Two new packages enable real-time event streaming to browser UIs, mobile apps, or other remote clients via SignalR: From f03241990300c0631bd7ff900dfd0b9ed3c2223e Mon Sep 17 00:00:00 2001 From: Alexey Zimarev Date: Wed, 19 Aug 2026 17:35:04 +0200 Subject: [PATCH 2/2] docs: reader contract, tiered/Redis changes from the review rounds Document the enforced read contract (exact count unless stream end, past-end reads return empty), the KurrentDB system-event compensation, the tiered reader fixes, the pageSize validation, and the Redis inclusive position semantics with the legacy-stream rejection caveat. Matches Eventuous/eventuous#568 as hardened by its review. Co-Authored-By: Claude Fable 5 --- .../docs/dotnet-next/persistence/event-store.md | 9 +++++++-- src/content/docs/dotnet-next/whats-new.mdx | 13 +++++++++++-- 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/src/content/docs/dotnet-next/persistence/event-store.md b/src/content/docs/dotnet-next/persistence/event-store.md index af05de5..212dd7e 100644 --- a/src/content/docs/dotnet-next/persistence/event-store.md +++ b/src/content/docs/dotnet-next/persistence/event-store.md @@ -125,7 +125,12 @@ foreach (var evt in events) { } ``` -The `IEventReader.ReadEvents` interface method returns `IAsyncEnumerable`. `KurrentDBEventStore` streams events as they arrive from the server, holding at most one deserialized event at a time. Relational stores buffer up to `count` events per call before yielding, so memory usage can grow with `count` — keep the count bounded when reading from those stores. +The `IEventReader.ReadEvents` interface method returns `IAsyncEnumerable`. `KurrentDBEventStore` streams events as they arrive from the server, holding at most one deserialized event at a time. Relational stores buffer events in an amount proportional to `count` per call before yielding, so memory usage can grow with `count` — keep the count bounded when reading from those stores. + +The reader contract holds for all stores: + +- A read yields exactly `count` events unless the end of the stream is reached, so a short read means there is nothing left to read. +- Reading past the end of an existing stream returns an empty sequence; only a missing stream throws `StreamNotFound`. ### Reading a whole stream @@ -141,7 +146,7 @@ await foreach (var evt in eventStore.ReadStreamToEnd( } ``` -Use the `pageSize` parameter to tune the page size, and `failIfNotFound: false` to get an empty sequence instead of a `StreamNotFound` exception when the stream doesn't exist. +Use the `pageSize` parameter to tune the page size (it must be positive), and `failIfNotFound: false` to get an empty sequence instead of a `StreamNotFound` exception when the stream doesn't exist. If you need the whole stream as an array, the `ReadStream` extension method pages through the stream the same way and collects all events: diff --git a/src/content/docs/dotnet-next/whats-new.mdx b/src/content/docs/dotnet-next/whats-new.mdx index 24c7c4e..48652b1 100644 --- a/src/content/docs/dotnet-next/whats-new.mdx +++ b/src/content/docs/dotnet-next/whats-new.mdx @@ -52,7 +52,12 @@ await foreach (var evt in eventReader.ReadEvents(stream, start, count, ct)) { } ``` -`KurrentDBEventStore` yields each event as it arrives from the server, holding at most one deserialized event at a time regardless of the requested count. Relational stores buffer up to `count` events per call, so memory usage can grow with `count` — keep the count bounded when reading from those stores. +`KurrentDBEventStore` yields each event as it arrives from the server, holding at most one deserialized event at a time regardless of the requested count. Relational stores buffer events in an amount proportional to `count` per call, so memory usage can grow with `count` — keep the count bounded when reading from those stores. + +The read contract is now explicit and enforced across all stores: + +- A read yields exactly `count` events unless the end of the stream is reached, so a short read reliably means the stream end. `KurrentDBEventStore` skips non-deserializable `$`-prefixed system events and issues follow-up reads to still deliver the requested count. +- Reading past the end of an existing stream returns an empty sequence; only a missing stream throws `StreamNotFound`. This also holds for `TieredEventReader`, which previously threw on past-end reads, and now caps its output at the requested count when combining hot and archive tiers. The previous signatures (with `failIfNotFound`, returning `Task`) are preserved as extension methods in `StoreFunctions`, so **all existing callers compile and work unchanged**. @@ -66,7 +71,11 @@ await foreach (var evt in eventReader.ReadStreamToEnd(stream, StreamReadPosition } ``` -Use `pageSize` to tune the page size, and `failIfNotFound: false` to get an empty sequence instead of a `StreamNotFound` exception when the stream doesn't exist. The `ReadStream` extension method pages through the stream the same way and collects the result into an array. +Use `pageSize` to tune the page size (it must be positive, otherwise the call throws `ArgumentOutOfRangeException`), and `failIfNotFound: false` to get an empty sequence instead of a `StreamNotFound` exception when the stream doesn't exist. The `ReadStream` extension method pages through the stream the same way and collects the result into an array. + +### Redis store position semantics + +The Redis store now reads with positions inclusive of the start position, matching every other store, and assigns explicit stream entry IDs on append so that positions always round-trip — previously, paged reads could silently skip events at page boundaries for same-millisecond append bursts. Streams containing entries written by earlier versions with auto-generated IDs the position encoding can't represent (sequence numbers above 9) are rejected for reads from a non-zero position with `NotSupportedException`; read such streams from the start, which identifies the offending entry, and migrate them. ### SignalR subscription gateway