Skip to content

feat(peer): answer RLY-009 so relay.dig.net/dht can show the content layer - #136

Merged
MichaelTaylor3d merged 1 commit into
mainfrom
feat/dht-records-reader
Aug 2, 2026
Merged

feat(peer): answer RLY-009 so relay.dig.net/dht can show the content layer#136
MichaelTaylor3d merged 1 commit into
mainfrom
feat/dht-records-reader

Conversation

@MichaelTaylor3d

Copy link
Copy Markdown
Contributor

Fixes DIG-Network/dig_ecosystem#1935. The last piece — with this, relay.dig.net/dht starts reporting real records.

What it does

Registers the reader that answers RLY-009. The relay is not a DHT node and holds no provider records, but it keeps a live reservation to every registered peer — and because a Kademlia node stores records for keys near its OWN peer_id, what this answers describes many other peers' content, not what this node caches. The union across connected nodes is a broad slice of the real DHT.

  • Registered only when the DHT is actually up. With no reader the node stays silent, which on the wire is indistinguishable from a pre-RLY-009 node — the right default for a feature that publishes what this node knows about the network.
  • Counts, never identities. A provider record is a (peer_id, content_key) pair, exactly the linkage /map refuses to publish.
  • The snapshot runs under block_in_place so the relay's request cannot stall the reservation socket's own poll. This node depends on that socket for its reachability; an observability feature must never be able to cost it.

The dig-download 0.15 migration, which was not a version bump

verify_module_anchor became async and now takes a borrowed &dyn ModuleReader instead of &[u8].

The verifier still materializes the bytes, deliberately: extract_data_section_blob and DataView::parse both parse the whole container, and the admitted digest is over the whole module. Verifying from a prefix would mean teaching those parsers to stream — a real change to the trust core, not something to fold into a dependency bump. The working set is the same one the old &[u8] API forced on the engine, just owned on this side of the call.

A staging read failure maps to Unavailable, never NotAnchored. That distinction is the whole point of the Rejection split: NotAnchored is evidence against the holder and earns it a durable demotion, while a failure to read our own staging area is this node's fault. Collapsing them would demote an honest peer for our local I/O error. New test covers exactly that path.

The cascade behind it

dig-node needs dig-nat 0.18 and the whole peer stack in ONE dependency graph — two dig-nat instances do not unify. So: dig-nat 0.18, dig-dht 0.11.1, dig-download 0.15, dig-peer 0.9, dig-peer-selector 0.9, dig-gossip main, all released first.

dig-nat 0.18 also made RelayMessage #[non_exhaustive], so the next wire addition ships as a patch instead of forcing this cascade again.

Verified

483 dig-node-core lib tests green, workspace builds, fmt / clippy --all-targets -D warnings clean.

Minor bump: 0.72.5 -> 0.73.0.

Live verification on the #1927 fleet follows the release — /dht.json should go from reporting_peers: 0 to the fleet's real records.

Co-Authored-By: Claude noreply@anthropic.com

…layer

Registers the reader the relay's /dht view is built from. The relay is not a DHT node
and holds no provider records, but it keeps a live reservation to every registered
peer — and because a Kademlia node stores records for keys near its OWN peer_id, what
this answers describes MANY OTHER peers' content, not what this node caches. The union
across connected nodes is a broad slice of the real DHT.

Registered only when the DHT is actually up. With no reader the node stays silent,
which on the wire is indistinguishable from a pre-RLY-009 node — the right default for
a feature that publishes what this node knows about the network.

Counts, never identities: a provider record is a (peer_id, content_key) pair, exactly
the linkage the relay's /map refuses to publish.

The snapshot is taken under block_in_place so the relay's request cannot stall the
reservation socket's own poll — this node depends on that socket for its reachability,
and an observability feature must never be able to cost it.

Also migrates ChainAnchoredModuleVerifier to dig-download 0.15, whose
verify_module_anchor became async and takes a borrowed ModuleReader instead of a slice.
The verifier still materializes the bytes: extract_data_section_blob and
DataView::parse both parse the whole container, and the admitted digest is over the
whole module, so verifying from a prefix would mean teaching those parsers to stream —
a real change to the trust core, not something to fold into a dep bump. A staging read
failure maps to Unavailable, never NotAnchored, because it is THIS node failing to read
its own bytes and must not durably demote an honest holder; there is a new test for
exactly that.

Peer stack moves together to the single-instance graph this needs: dig-nat 0.18,
dig-dht 0.11.1, dig-download 0.15, dig-peer 0.9, dig-peer-selector 0.9, dig-gossip main.

Refs DIG-Network/dig_ecosystem#1935, #1933

Co-Authored-By: Claude <noreply@anthropic.com>
@MichaelTaylor3d
MichaelTaylor3d force-pushed the feat/dht-records-reader branch from 16fab3d to c49272d Compare August 2, 2026 05:46
@MichaelTaylor3d
MichaelTaylor3d merged commit f92ba47 into main Aug 2, 2026
15 checks passed
@MichaelTaylor3d
MichaelTaylor3d deleted the feat/dht-records-reader branch August 2, 2026 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant