feat(moq-net): usage stats in the model (BroadcastInfo-carried, origin-attributed) by kixelated · Pull Request #1895 · moq-dev/moq

kixelated · 2026-06-23T22:17:00Z

Implements the design in rs/moq-net/DESIGN-stats.md (PR #1894), which supersedes the with_meter / set_meter approach on #1873. Per-broadcast usage sinks live in BroadcastInfo, are set at construction, and ride the immutable Arc<BroadcastInfo> down to every track, group, and frame. Usage is atomics, so the model bumps through a shared &Arc<Usage> with no mutation, no setter, and no Arc::make_mut.

This takes over #1873 (which should be closed in favor of this).

What changed

The design's three phases, each landed as its own commit:

The model carries usage stats. New model/usage.rs: Usage (groups/frames/bytes payload + opened/closed lifecycle atomics) and BroadcastStats (one Usage per direction). BroadcastInfo gains a stats field; Arc<BroadcastInfo> + Arc<TrackInfo> are threaded through Broadcast/Track/Group; payload bumps move into the model (ingress at create_group/append_frame, egress as the consumer reads). The stats-layer BroadcastStats is renamed BroadcastHandle.
Origin attributes egress, ingress baked at construction. The lite/IETF subscriber loops build the broadcast with stats.producer already set. The per-session OriginConsumer::with_egress provider stamps each BroadcastConsumer it yields (announce stream, request_broadcast, dynamic accept) with that session's egress sink, so per-tier egress survives with zero mutation. The handler/gateway per-frame/byte/group bumps are deleted; the stats layer reads the shared Arc<Usage> instead of vending payload guards.
Model-tracked live viewer/publisher counts. A Live refcount over each sink: a BroadcastConsumer is one live viewer while it has ≥1 outstanding TrackConsumer (the token rides into the TrackSubscriber, so a subscription counts for its whole life); a broadcast with ≥1 live TrackProducer is one publisher (the publisher token rides the shared broadcast state, so the relay's dynamic-accept ingress path is covered). SessionBroadcasts is gone; the publish loop maps opened - closed onto the existing broadcasts / broadcasts_closed fields, so the published schema is unchanged.

Semantics

Egress is per-consumer (N viewers count N times); ingress is single-writer (counts once). A relay counts both directions without double-counting either.
The lite fetch path is metered through the model: a fetch re-serves an existing group, so it bumps frames/bytes but not groups.
The model counts raw (decompressed) bytes vs. the old post-compression wire bytes. Only the hang catalog track sets compress, so the difference is noise; a follow-up will cache compressed bytes so the count is the wire size again. (Confirmed acceptable.)

Testing

cargo test -p moq-net passes (400 lib + 4 integration), including new model tests for ingress-once/egress-per-viewer, origin egress-stamping, and live viewer/publisher counts. clippy/doc/fmt clean; dependent crates (moq-mux, hang, moq-relay, moq-native, moq-srt, moq-json, moq-ffi) compile.

Note: this environment's egress policy blocks the kixelated/nvidia-video-codec-sdk git fork and the sandbox has no IPv6, so the full workspace just check and the moq-native end-to-end relay suite can't run here. CI's nix run is the source of truth for those.

Cross-package sync

js/net: not mirrored. The stats aggregator / usage sinks are relay-side Rust only; the browser client has no equivalent. No doc/concept wire change (this is API, not wire).

Breaking changes (target `dev`)

Public API in rs/moq-net: BroadcastInfo gains stats; Broadcast{Producer,Consumer,Dynamic} carry Arc<BroadcastInfo>; GroupProducer::new takes Arc<TrackInfo> + Arc<BroadcastInfo> (was Option<Timescale>); the stats-layer BroadcastStats is renamed BroadcastHandle; SessionBroadcasts / BroadcastSubscription and the publisher_broadcasts / subscriber_broadcasts / per-track frame/bytes/group methods are removed; Counters swaps its payload + broadcasts atoms for a shared Arc<Usage>. TrackProducer::new(name, info) keeps its signature (standalone tracks get a shared no-op broadcast).

Follow-ups

Wire moq-rtc (WHIP/WHEP) and moq-rtmp through the same origin egress / ingress-at-construction seam (the mechanism exists; their origins just need with_egress and an ingress-baked broadcast).
Cache compressed bytes in the model so the byte count is the wire size for compressed tracks.

🤖 Generated with Claude Code

(Written by Claude)

Move per-broadcast usage counting into the transport-agnostic model, the foundation for metering every transport (moq-lite, IETF, and the non-MoQ gateways) uniformly without re-instrumenting each data path. New `model/usage.rs`: a `Usage` struct (groups/frames/bytes payload counters plus opened/closed lifecycle counters, all atomics) and a `BroadcastStats` pair (one `Usage` per direction). `BroadcastInfo` gains a `stats` field, so the immutable broadcast handle carries the sinks down to every track, group, and frame through a shared `Arc<BroadcastInfo>`. The group also carries an `Arc<TrackInfo>`, so `timescale` is read from there instead of threaded separately. Producer-side handles bump the ingress sink (groups at create_group/ append_group, frames+bytes at append_frame); consumer-side handles bump the egress sink as a subscriber receives groups and reads frames. A fetch re-serves an existing group, so it bumps frames/bytes but not groups. Behavior-preserving: the sinks default to unreferenced no-op atomics, so a standalone broadcast is unmetered and the existing stats-layer wiring is untouched. The stats-layer `BroadcastStats` is renamed `BroadcastHandle` to free the name for the new model type. `TrackProducer::new` keeps its signature (standalone tracks get the shared no-op broadcast). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011tHLbSdh7JuEL8Uyahsd3d

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011tHLbSdh7JuEL8Uyahsd3d

Wire the model's usage sinks to the stats layer and meter every transport uniformly, removing the per-handler payload counting. - Stats layer: each per-(tier, role) `Counters` holds an `Arc<Usage>` shared with the model (the same `Arc` baked into the broadcast), and the snapshot reads payload from it. `BroadcastHandle` vends the ingress/egress `Usage` for baking/stamping. The per-track `frame()`/`bytes()`/`group()` guard methods are gone; the track guards now track only the `subscriptions` lifecycle. - Ingress baked at construction: the lite/IETF subscriber loops build the broadcast with `stats.producer` already set, so create_group/create_frame meter ingress. The handler's per-frame/byte/group bumps are removed. - Egress attributed by the origin: `OriginConsumer::with_egress` carries a per-session sink provider; `get_broadcast`, the dynamic `request_broadcast` path, and `AnnounceConsumer` delivery stamp each `BroadcastConsumer`'s `stats.consumer` with the session's egress sink. The lite/IETF sessions attach it. The publisher's per-frame/byte/group bumps are removed; the model meters as the consumer reads (live via next_frame, fetch via get_frame; a fetch counts frames/bytes but not groups). Egress is per-consumer (N viewers count N times) and ingress is single-writer. Per the design, the model counts raw (decompressed) bytes; only the catalog track compresses, so the difference is noise (a follow-up will cache compressed bytes). Tests: model-level ingress-once/egress-per-viewer and origin egress-stamping; all 400 moq-net lib tests pass. Dependent crates compile; clippy/doc/fmt clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011tHLbSdh7JuEL8Uyahsd3d

Move the live viewer/publisher count into the model and retire the stats-layer `SessionBroadcasts` sentinel. The published `broadcasts` / `broadcasts_closed` schema is unchanged; it's now driven from a cleaner place. - `usage.rs`: a `Live` refcount over a `Usage` sink. The `0 -> 1` transition bumps `opened`, the last `1 -> 0` drop bumps `closed`, so `opened - closed` is the live count. Clones of a handle share one `Live` (one logical viewer/publisher). - Consumer side: `BroadcastConsumer` carries a viewers `Live` over its egress sink (re-keyed when the origin stamps the session sink). Each `TrackConsumer` from `track()` holds a token; it propagates to the `TrackSubscriber` so a subscription counts as a viewer for its whole life, not just while the `TrackConsumer` handle exists. N tracks on one consumer is still one viewer. - Producer side: a publishers `Live` over the ingress sink lives on the shared broadcast state, so every track-creation path (`create_track`, `reserve_track`, and the dynamic handler's `requested_track` -> `accept`, which is the relay's ingress path) takes a token. A broadcast with >= 1 live track is one publisher. - `stats.rs`: `Counters` drops its `broadcasts`/`broadcasts_closed` atoms and reads them from the model's `Usage.opened/closed` in the snapshot. `SessionBroadcasts` / `BroadcastSubscription` and the `publisher_broadcasts` / `subscriber_broadcasts` handle methods are removed. - handlers: drop the `SessionBroadcasts` plumbing; the model counts viewers (egress) and publishers (ingress) automatically. Tests: model-level live-count test (publisher per broadcast, viewer dedup across tracks) plus the reworked snapshot-mapping test. 400 moq-net lib tests pass; clippy/doc/fmt clean; dependent crates compile. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011tHLbSdh7JuEL8Uyahsd3d

kixelated · 2026-06-24T04:23:03Z

-		track_stats: std::sync::Arc<crate::PublisherTrack>,
+		// Held for the group's lifetime so it counts as one subscription; payload
+		// is metered by the model when the frames are read.
+		_track_stats: std::sync::Arc<crate::PublisherTrack>,


This should be in model now right?

kixelated · 2026-06-24T04:23:38Z

+		let publish = {
+			let stats = stats.clone();
+			publish.with_egress(std::sync::Arc::new(move |abs: &str| {
+				stats.broadcast(abs).egress_usage()


Why isn't this done in origin?

kixelated · 2026-06-24T04:25:17Z

+
+/// Shared no-op broadcast metadata for tracks created outside a broadcast.
+///
+/// A standalone [`TrackProducer::new`] track has no broadcast to inherit from, so


Delete TrackProducer::new and delete this. All tracks must be created/owned by a broadcast.

kixelated · 2026-06-24T04:27:19Z

-			state: Default::default(),
+		let info = Arc::new(info);
+		let state = kio::Producer::<BroadcastState>::default();
+		if let Ok(mut s) = state.write() {


Don't make a default state then grab the lock. Set the init state instead

kixelated · 2026-06-24T04:27:47Z

-		let track = TrackProducer::new(name, info);
 		let mut state = BroadcastState::modify(&self.state)?;
+		let publisher = state.publishers.enter();
+		let track = TrackProducer::for_broadcast(name, info, self.info.clone(), Some(publisher));


rename to new and make pub(super)

kixelated · 2026-06-24T04:29:58Z

 	}

 	/// The parent track's negotiated timescale, or `None` for untimed tracks.
 	pub fn timescale(&self) -> Option<Timescale> {


Should delete this

kixelated · 2026-06-24T04:30:20Z

@@ -342,7 +359,18 @@ impl std::ops::Deref for GroupConsumer {
 impl GroupConsumer {
 	/// The parent track's negotiated timescale, or `None` for untimed tracks.
 	pub fn timescale(&self) -> Option<Timescale> {


kixelated · 2026-06-24T04:31:54Z

+	// When set, stamp the resolved (dynamically served) consumer with this
+	// session's egress sink for the absolute path. The already-announced path
+	// (`Ready`) is stamped by `get_broadcast` instead.
+	stamp: Option<(EgressSink, PathOwned)>,


Maybe we have Arc instead? IDK

kixelated · 2026-06-24T04:32:43Z

+
+impl Usage {
+	/// Record one group.
+	pub(crate) fn add_group(&self) {


Maybe all should be pub for custom stats.

kixelated · 2026-06-24T04:34:54Z

+pub(crate) struct Live {
+	usage: Arc<Usage>,
+	count: AtomicUsize,
+}


Maybe use Weak? If it fails to upgrade, we make a new LiveToken?

- `BroadcastProducer::new`: build `BroadcastState` with its publisher count set up front instead of default-then-lock. - Drop `GroupProducer::timescale` (unused) and `GroupConsumer::timescale`; the lite fetch path now reads the timescale from the track's `TrackInfo` (the same `info` it already fetches for compression), so the group carries `Arc<TrackInfo>` purely for the model's own use. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011tHLbSdh7JuEL8Uyahsd3d

kixelated · 2026-06-24T04:43:59Z

Thanks for the thorough pass. Pushed 27730d3 for the clear-cut ones:

BroadcastProducer::new (default-then-lock) — now builds BroadcastState with the publisher count set up front.
GroupProducer::timescale / GroupConsumer::timescale — both deleted. The one caller (lite fetch) now reads the timescale from track.info() (the TrackInfo it already fetches for compression), so the group keeps Arc<TrackInfo> only for the model's own use.

On the bigger ones I'd like your call before I sink the effort, since a couple ripple beyond moq-net:

Delete TrackProducer::new / noop_broadcast (all tracks broadcast-owned). I'm on board with the principle, but it's ~88 call sites across moq-mux (container producer/consumer), moq-json, and moq-net, and most of those create standalone tracks for muxing — making them broadcast-owned is a moq-mux design change, not a mechanical rename. It also reverses the design doc's "keep new so the ~80 deferred sites are untouched". Do it in this PR, or land this and do the broadcast-owned migration as a follow-up? (And I read broadcast.rs:212 as "rename for_broadcast → new, pub(super)" once standalone new is gone — confirm?)
Subscriptions in the model (ietf/publisher.rs:294). Agreed — subscriptions/subscriptions_closed can move to the model the same way broadcasts did (a token on a subscriptions Usage), which lets me drop the publisher_track/subscriber_track guards entirely. This PR or a follow-up?
Egress wiring "why not in origin?" (ietf/session.rs:36). The closure is stats.broadcast(abs).egress_usage() — the model deliberately can't depend on the stats layer, so the stats→usage mapping has to be supplied from outside. I put it in session::start because that's where the StatsHandle already lives, but it's duplicated lite/ietf. I can instead attach it in moq-relay when it scopes the per-session origin (pulling the stats coupling out of moq-net's session entirely), or factor a shared helper. Preference?
"set publisher stuff is sus" (broadcast.rs:361). The producer token rides the shared BroadcastState so the relay's dynamic-accept ingress path (poll_requested_track) is covered as well as create_track/reserve_track. What's bugging you — the multiple attach points, the Default no-op Live, or the whole approach? Happy to reshape if you have a shape in mind.

Exploratory ones I'll treat as follow-ups unless you want them now: make Usage's bump methods pub for custom stats (usage.rs:32), Weak<LiveToken> (usage.rs:97), Arc<OriginInfo> (origin.rs:1185), and the set_consumer_sink clone-to-swap (broadcast.rs:447).

(Written by Claude)

Generated by Claude Code

A frame writer needs the track's timescale to decide whether to attach a per-frame timestamp; `hang::container::Frame::encode` relies on it. Only `GroupConsumer::timescale` was redundant (its lone caller now reads the timescale from `TrackInfo`). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011tHLbSdh7JuEL8Uyahsd3d

kixelated · 2026-06-24T04:48:07Z

Correction on the timescale deletions: CI caught that hang::container::Frame::encode uses GroupProducer::timescale (to decide whether to attach a per-frame timestamp), so I kept that one and deleted only GroupConsumer::timescale (3ab234f). Fully removing GroupProducer::timescale means threading the timescale into hang's Frame::encode (and its moq-mux callers) instead — happy to do that if you want it gone, just flagging it's a cross-crate change rather than a local delete.

(Written by Claude)

Generated by Claude Code

claude added 5 commits June 23, 2026 22:16

style(moq-net): rustfmt import ordering in group.rs

2169ad1

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011tHLbSdh7JuEL8Uyahsd3d

docs(moq-net): avoid private intra-doc link in Usage::closed

184948c

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011tHLbSdh7JuEL8Uyahsd3d

kixelated changed the title ~~feat(moq-net): usage stats in the model (BroadcastInfo-carried)~~ feat(moq-net): usage stats in the model (BroadcastInfo-carried, origin-attributed) Jun 23, 2026

kixelated commented Jun 24, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(moq-net): usage stats in the model (BroadcastInfo-carried, origin-attributed)#1895

feat(moq-net): usage stats in the model (BroadcastInfo-carried, origin-attributed)#1895
kixelated wants to merge 7 commits into
devfrom
claude/vigilant-shannon-t539t1

kixelated commented Jun 23, 2026 •

edited

Loading

Uh oh!

kixelated Jun 24, 2026

Uh oh!

kixelated Jun 24, 2026

Uh oh!

kixelated Jun 24, 2026

Uh oh!

kixelated Jun 24, 2026

Uh oh!

kixelated Jun 24, 2026

Uh oh!

kixelated Jun 24, 2026

Uh oh!

kixelated Jun 24, 2026

Uh oh!

kixelated Jun 24, 2026

Uh oh!

kixelated Jun 24, 2026

Uh oh!

kixelated Jun 24, 2026

Uh oh!

kixelated commented Jun 24, 2026

Uh oh!

kixelated commented Jun 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

kixelated commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

Semantics

Testing

Cross-package sync

Breaking changes (target dev)

Follow-ups

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kixelated commented Jun 24, 2026

Uh oh!

kixelated commented Jun 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kixelated commented Jun 23, 2026 •

edited

Loading

Breaking changes (target `dev`)