Ultra-low-latency SPMC/MPMC pub/sub using stamped ring buffers.
Photon Ring is a zero-allocation pub/sub crate for Rust built around pre-allocated ring buffers, per-slot stamp validation, and T: Pod payloads. It targets the part of concurrent systems where queueing overhead dominates: market data, telemetry fanout, staged pipelines, and other hot-path broadcast workloads where every subscriber should see every message.
By default, slots use a volatile-based seqlock for maximum performance. With the atomic-slots feature, the same stamp protocol operates over AtomicU64 stripes — free of data races under the Rust abstract machine for padding-free payloads, with zero performance regression on x86-64, and verified under Miri in CI.
It is no_std compatible with alloc, supports named-topic buses and typed buses, and includes a pipeline builder for multi-stage thread topologies on supported desktop/server platforms.
Important
The default channel() is lossy on overflow: the publisher never blocks, and slow subscribers detect drops via TryRecvError::Lagged. If lossless delivery matters more than raw latency, use channel_bounded().
use photon_ring::{channel, channel_mpmc, Photon};
// SPMC channel (single producer, multiple consumers)
let (mut pub_, subs) = channel::<u64>(1024);
let mut sub = subs.subscribe();
pub_.publish(42);
assert_eq!(sub.try_recv(), Ok(42));
// MPMC channel (multiple producers, clone-able)
let (mp_pub, subs) = channel_mpmc::<u64>(1024);
let mp_pub2 = mp_pub.clone();
mp_pub.publish(1);
mp_pub2.publish(2);
// Named-topic bus
let bus = Photon::<u64>::new(1024);
let mut p = bus.publisher("prices");
let mut s = bus.subscribe("prices");
p.publish(100);
assert_eq!(s.try_recv(), Ok(100));[dependencies]
photon-ring = "2.5.0"Optional features:
derive: enables#[derive(photon_ring::DerivePod)]for user-definedPodtypes.hugepages: enables Linux memory controls such asmlock,prefault, and NUMA helpers.atomic-slots: enables a data-race-free slot implementation that decomposes the payload intoAtomicU64stripes (stepping down throughAtomicU32/U16/U8for a trailing partial stripe) instead ofwrite_volatile/read_volatile. Zero performance cost on x86-64. On ARM64 both paths pay the same reader-side acquire fence, soatomic-slotscosts nothing extra there either. Eliminates the formal undefined behavior the default path carries under the Rust abstract machine, for payloads with no padding bytes. Its multi-threaded tests run under Miri in CI (miri (atomic-slots)), so the claim is machine-checked rather than asserted. Padding is the remaining gap: a type such as(u8, u64)has 7 uninitialized bytes, and reading those as part of an atomic word is itself undefined. Use#[repr(C)]with explicit padding fields (as the examples do) so every byte is initialized.
Rust 1.94+ is supported. For best performance, compile with -C target-cpu=native to enable PREFETCHW and other CPU-specific optimizations.
Inter-thread communication is often the dominant cost in concurrent systems. Traditional messaging designs usually pay for at least one expensive property on the hot path:
| Approach | Write cost | Read cost | Allocation |
|---|---|---|---|
std::sync::mpsc |
Lock + CAS | Lock + CAS | Per-message |
Mutex<VecDeque> |
Lock acquisition | Lock acquisition | Dynamic growth |
crossbeam-channel |
CAS on head | CAS on tail | None |
| LMAX Disruptor pattern | Sequence claim + barrier | Sequence barrier spin | None |
The Disruptor showed that pre-allocated rings can remove allocator overhead and drive very low latency, but its shared sequence barriers still create cache-line contention between producers and consumers.
Photon Ring moves synchronization into each slot. Every slot carries its own seqlock stamp beside the payload, so readers validate the data they just loaded instead of bouncing a shared barrier cache line.
64 bytes (one cache line)
+-----------------------------------------------------+
| stamp: AtomicU64 | value: T |
| (seqlock) | (Pod - all bit patterns valid)|
+-----------------------------------------------------+
For T <= 56 bytes, stamp and value share one cache line.
Larger T spills to additional lines (still correct, slightly slower).
1. stamp = seq * 2 + 1 (odd = write in progress)
2. fence(Release) (stamp visible before data)
3. write_volatile(slot.value, data)
4. stamp = seq * 2 + 2 (even = write complete, Release)
5. cursor = seq (Release - consumers can proceed)
1. s1 = stamp.load(Acquire)
2. if odd -> spin
3. if s1 < expected -> Empty
4. if s1 > expected -> Lagged
5. value = read_volatile(slot) (direct read, T: Pod)
6. fence(Acquire) (payload read completes before re-check)
7. s2 = stamp.load(Relaxed)
8. if s1 == s2 -> return
9. else -> retry
- No shared mutable state on the read path. Each subscriber keeps its own local cursor. Readers do not publish progress into a shared hot cache line unless bounded backpressure tracking is enabled.
- Stamp and payload are co-located. For
T <= 56bytes, the stamp check and payload read hit the same cache line. - No allocation on publish or receive. The ring is fixed at construction time, and hot-path operations are direct slot reads and writes.
T: Podmakes torn reads safe to reject. Every bit pattern is valid, so an optimistic torn read is harmless and discarded by the stamp re-check.- Single-producer SPMC avoids write-side contention.
Publisher::publishtakes&mut self, so the type system enforces one producer without CAS.MpPublisheradds an MPMC path when you need multiple concurrent writers.
Measured with Criterion on an Intel i7-10700KF (8C/16T, 3.80 GHz, Linux 6.8, Rust 1.93.1) and Apple M1 Pro (8C, macOS 26.3, Rust 1.92.0), --release, 100 samples, no core pinning unless stated.
Note
These numbers use Pod payloads and compare concrete implementations, not abstract algorithms. Scheduler noise, pinning, CPU generation, and payload layout all matter, so treat them as reproducible snapshots rather than universal constants.
i7-10700KF M1 Pro
────────── ──────
Publish only 2.8 ns 2.4 ns
Roundtrip (1 sub, same thread) 2.7 ns 8.8 ns
Fanout (10 independent subs) 17.0 ns 27.7 ns
SubscriberGroup (any N, O(1)) 2.6 ns 8.8 ns
MPMC (1 pub, 1 sub) 12.1 ns 10.6 ns
Empty poll 0.9 ns 1.1 ns
Batch 64 + drain 158 ns 282 ns
Struct roundtrip (24B Pod) 4.8 ns 9.3 ns
Cross-thread roundtrip 95 ns 130 ns
One-way latency (RDTSC) 48 ns p50 —
- Sustained throughput: about 300M msg/s on Intel and 88M msg/s on M1 Pro
- Payload scaling: at cache-line-sized payloads the copy is a few percent of latency — cross-core cache-coherence transfer dominates. The copy only becomes co-dominant in the KiB range; see
docs/payload-scaling.md
Backpressure exists so a consumer that must not lose messages can stop the publisher. But two things should never stop the world: a consumer that is only observing, and a consumer that has died.
Because each slot carries its own stamp, subscribers need no shared barrier — so a single ring can carry different delivery contracts per consumer:
let (mut pub_, subs) = channel_bounded::<Order>(1024, 0);
let mut risk = subs.subscribe(); // gates the publisher, loses nothing
let mut telemetry = subs.subscribe_lossy(); // never gates it, drops when behindrisk keeps its no-loss guarantee. telemetry is invisible to the publisher's
backpressure scan, so however slow it gets it cannot stall order flow; when it
falls behind it observes Lagged { skipped } with an exact count, and
receive_ratio() reports what it sampled. Both read the same sequence numbers,
so an observation can be correlated with the message the risk engine processed.
A consumer that dies also releases the ring, provided its Subscriber drops
with it — which is automatic when the consumer thread owns the subscriber, as
the thread unwinds and Drop removes it from the backpressure set. A subscriber
parked in long-lived shared state (an Arc'd registry, a supervisor struct)
outlives its consumer and keeps gating the publisher, so don't do that with a
tracked subscriber. A merely wedged consumer still applies backpressure — that
is the guarantee working as intended.
The no-loss guarantee is a property of each tracked subscriber's lifetime, not of the ring: if the last tracked subscriber goes away while lossy ones remain, nothing gates the publisher any more and the bounded ring behaves like a lossy one.
Subscribers can also attach to a ring that is already running, so a lossy debug tap can be added and removed on a live system without perturbing it.
cargo run --release --example degradation demonstrates both scenarios;
tests/degradation.rs asserts them.
| Capability | Photon Ring |
|---|---|
| Delivery | Broadcast — every subscriber sees every message |
| Publish | 2.8 ns lossy, 7.96 ns bounded with a live consumer |
| Cross-thread roundtrip | 95 ns |
| Throughput | ~300M msg/s sustained |
| Per-consumer contracts | Gating and non-gating subscribers on one ring |
| Consumer failure | A dead consumer releases the publisher |
| Hot attach | Subscribe to and detach from a running ring |
| Topologies | Pipelines, fan-out, managed terminal consumers |
| Topic bus | Named topics, and a typed bus for per-topic payload types |
| Backpressure | Optional, per subscriber |
no_std |
Yes, with alloc |
| Multi-producer | Yes |
Photon Ring is for streams where every subscriber should observe every message with minimal coordination between them. If instead each message should be owned by exactly one receiver, a work queue is the better shape.
Channels are the lowest-level interface. channel::<T>(capacity) creates the fastest single-producer path and returns a Publisher<T> plus a cloneable Subscribable<T>. channel_bounded::<T>(capacity, watermark) adds optional backpressure; Publisher::try_publish returns PublishError::Full(value) instead of overwriting unread slots. channel_mpmc::<T>(capacity) returns MpPublisher<T>, which is Clone + Send + Sync and uses atomic sequence claiming for concurrent producers. On the write side, the important APIs are publish, publish_with for in-place construction, publish_batch on Publisher, and published/capacity for lightweight counters.
Subscribers are independent and contention-free by default. Subscribable::subscribe() starts from future messages only, while subscribe_from_oldest() starts at the oldest message still retained in the ring. Subscriber<T> exposes try_recv, recv, recv_with, latest, pending, recv_batch, and drain, plus observability counters through total_received, total_lagged, and receive_ratio. If multiple logical consumers are polled on the same thread, subscribe_group::<N>() creates a SubscriberGroup<T, N> that performs one ring read for the whole group instead of one per logical subscriber.
For topic routing, Photon<T> provides a string-keyed bus where all topics share the same payload type, and TypedBus allows a different T: Pod per topic. Both lazily create topics and expose publisher, try_publisher, subscribe, and subscribable. publisher() will panic if the publisher for that topic was already taken, and TypedBus also panics on type mismatches for an existing topic.
Pipelines build dedicated-thread processing graphs on supported OS targets. topology::Pipeline::builder().capacity(...).input::<T>() returns an input publisher plus a typed builder; .then(...) chains stages, .fan_out(...) creates a diamond, .then_a(...) and .then_b(...) extend either branch, and .build() returns the final subscriber plus a Pipeline handle. then_with(f, WaitStrategy) (and then_a_with, then_b_with) lets you configure the wait strategy for each pipeline stage. The handle supports shutdown, join, panicked_stages, is_healthy, and stage_count. For manual shutdown outside topology, use Shutdown.
The #[derive(photon_ring::DeriveMessage)] macro supports a #[photon(as_enum)] attribute for fields whose types are #[repr(u8)] enums. Unrecognized types without this attribute now produce a compile error instead of being silently assumed to be enums.
Ring capacity accepts any integer >= 2. Power-of-two capacities use bitwise seq & mask for zero-overhead indexing; arbitrary capacities use Lemire reciprocal-multiply fastmod (~1.5 ns).
Wait behavior is explicit. recv_with accepts WaitStrategy::BusySpin, YieldSpin, BackoffSpin, Adaptive, MonitorWait, or MonitorWaitFallback depending on whether you want the absolute lowest wakeup latency or better core sharing. MonitorWait uses Intel UMONITOR/UMWAIT (Alder Lake+) for near-zero power wakeup (~30 ns), with automatic fallback to PAUSE on older x86 or WFE on ARM; construct it safely via WaitStrategy::monitor_wait(&stamp). MonitorWaitFallback uses TPAUSE without requiring an address. On supported platforms, the crate also includes affinity helpers for CPU pinning; with the hugepages feature on Linux, you can use Publisher::mlock, Publisher::prefault, and mem::{set_numa_preferred, reset_numa_policy} to reduce page-fault and NUMA noise.
photon-ring-async— Runtime-agnostic async wrappers.AsyncSubscriberandAsyncSubscriberGroupwith yield-based polling and configurable spin budget. Works with tokio, smol, embassy, or any executor.photon-ring-metrics— Observability wrappers withSubscriberMetrics(snapshot/delta tracking) andPublisherMetrics. Framework-agnostic — bring your own prometheus/opentelemetry.
| Constraint | Rationale |
|---|---|
T: Pod |
Every bit pattern must be valid, which makes optimistic torn reads safe to reject. |
| Capacity >= 2 | Any capacity works. Power-of-two uses seq & mask; arbitrary capacity uses Lemire fastmod (~1.5 ns, zero-division). |
| Single producer by default | The fastest path relies on &mut self rather than write-side atomics. |
| Lossy overflow by default | The publisher never blocks; subscribers detect drops through Lagged. |
| 64-bit atomics required | The core algorithm depends on AtomicU64. |
| 64-bit sequence numbers | Stamp encoding seq * 2 + 2 overflows at u64::MAX / 2 (~9.2 × 10^18 messages). At 1 billion msg/s this would take ~292 years. |
| Platform | Core ring | Affinity | Topology | Hugepages |
|---|---|---|---|---|
| x86_64 Linux | Yes | Yes | Yes | Yes |
| x86_64 macOS / Windows | Yes | Yes | Yes | No |
| aarch64 Linux | Yes | Yes | Yes | Yes |
| aarch64 macOS (Apple Silicon) | Yes | Yes | Yes | No |
| wasm32 | Yes | No | No | No |
| FreeBSD / NetBSD / Android | Yes | Yes | Yes | No |
| 32-bit ARM (Cortex-M) | No | No | No | No |
The Pod trait means more than Copy: every possible bit pattern of the payload must be valid. This is required because the stamp-based read protocol may speculatively read bytes from a slot while a writer is updating it. If a torn bit pattern could be invalid for T, the read would be undefined behavior before the stamp check could discard it.
Primitive numerics, arrays of Pod, and tuples of Pod are already supported. For your own structs, use #[repr(C)], stick to Pod fields, and implement Pod manually or via the derive feature when appropriate.
| Type | Why it is not Pod |
Use instead |
|---|---|---|
bool |
Only 0 and 1 are valid |
u8 |
char |
Must be a valid Unicode scalar | u32 |
NonZero<u32> |
0 is invalid |
u32 |
Option<T> |
The discriminant has invalid patterns | Sentinel integer |
Rust enum |
Only declared variants are valid | u8 or u32 |
&T, &str |
Pointers must be valid | Value types only |
String, Vec<_> |
Heap-owning, has Drop |
Fixed [u8; N] buffer |
Photon Ring offers two slot implementations, selectable at compile time:
| Default (volatile) | atomic-slots feature |
|
|---|---|---|
| Mechanism | write_volatile / read_volatile |
AtomicU64::store/load(Relaxed) stripes |
| Formal status | Data race under Rust abstract machine (practical UB) | Formally sound — no data races |
| Miri | Flags multi-threaded tests | Passes, enforced in CI |
| x86-64 cost | Baseline | Zero — identical MOV instructions |
| ARM64 cost | One DMB ISHLD reader fence (both paths) |
Same fence — no additional cost |
| Precedent | Same pattern as Linux kernel seqlocks (20+ years) | Per-word atomic decomposition, as in atomic-memcpy |
Note
Both implementations place an Acquire fence between the payload read and the
stamp re-check — the same barrier the Linux kernel's read_seqcount_retry()
carries as smp_rmb(). Without it an acquire load is one-way and a weakly
ordered CPU may satisfy the payload read after the re-check has validated,
which would return data from a later overwrite. On x86 the fence emits no
instruction — TSO already orders load-load — but it is still a compiler
barrier, and measured at roughly +1.2 ns on the same-thread roundtrip
microbenchmark because it forbids reordering the optimiser was otherwise
free to do. That is the price of the guarantee, on every architecture.
With that fence in place, the default volatile implementation produces correct
results on real hardware; what remains is that it is a data race under Rust's
abstract machine, which Miri reports and no compiler has yet exploited. Enable
atomic-slots for a build free of that race — machine-checked in CI, and free
on x86-64.
Tip
Keep rich domain types at the edges and publish compact Pod messages in the middle. Convert enums, Option, booleans, and strings into explicit numeric fields or fixed-size buffers before calling publish.
Examples of safe boundary conversions:
bool->u8(0 = false,1 = true)enum Side { Buy, Sell }->u8(0 = Buy,1 = Sell)Option<u32>->u32(0 = None, nonzero =Some)String/&str->[u8; N]
cargo test
cargo bench
cargo bench --bench payload_scaling
cargo +nightly miri test --test correctness -- --test-threads=1
RUSTFLAGS="--cfg loom" cargo test --release --test loom_mpmc --test loom_backpressure # exhaustive interleaving checks
cargo run --release --example market_data
cargo run --release --example pipeline
cargo run --release --example backpressureLicensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE)
- MIT license (LICENSE-MIT)
at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.