fix(cache): collapse the double whole-capsule pull into one shared single-flight gate (#1614)

Cargo.toml

@@ -32,7 +32,7 @@ edition = "2021"
 # the ROOT manifest (`[workspace.package].version`), so it MUST be set here for a
 # release to fire (§3.6). The library crates (dig-node-core/dig-runtime/dig-wallet)
 # keep their own independent versions — only the released binary tracks the workspace version.
-version = "0.75.0"
+version = "0.75.1"
 
 # Release hardening, matching digstore: keep integer-overflow checks ON in release.
 # The node parses untrusted serialized input and does offset/length arithmetic over

DEVELOPMENT_LOG.md

@@ -4,6 +4,27 @@ High-signal realizations from debugging/development: non-obvious cross-system co
 sharp edges, and gotchas. Concise durable facts with context — NOT a change diary. See
 `CLAUDE.md` §4.5 for the maintenance contract (a curator periodically re-verifies + prunes).
 
+## §21 backfill and the #1576 reshare are TWO transports for the same capsule — one gate, not two (#1614)
+
+A read miss can pull the SAME `(store, root)` whole `.dig` down two independent legs, and for a long
+time each had its OWN in-flight set, blind to the other:
+
+- **Leg A — §21 backfill** (`maybe_backfill_capsule` → `gap_fill_generation` → `cache_fetch_and_cache`):
+  the authenticated whole-store sync from the RPC upstream. This is the PEERLESS-network fallback — it
+  can acquire a capsule when NO peer serves it, so it must never be suppressed away.
+- **Leg B — #1576 reshare warm** (`spawn_capsule_reshare` → `CapsuleWarmer::warm`): the P2P pull that
+  makes a reader a discoverable HOLDER. It has NO upstream fallback — with no providers it just refuses.
+
+Because Leg A used `Node::backfilling` (a `HashSet<String>`) and Leg B used a separate `WarmRegistry`,
+a single read fired BOTH — 2× bandwidth/disk/CPU for one artifact. The fix is one shared single-flight
+gate: `Node::capsule_acquisition` is the ONE `Arc<WarmRegistry>`, and the reshare warmer is wired with a
+CLONE of that same Arc (`wire_capsule_reshare(..., node.capsule_acquisition_gate())`), so both legs
+test-and-set one registry keyed `"{store}:{root}"`. The keys were ALREADY byte-identical across the two
+sites (`CapsuleKey::Display` == `maybe_backfill`'s `format!` == `WarmRegistry`'s key), which is what made
+the collapse race-free — one mutex, one key space. Load-bearing ordering: each leg's origin/config/held
+gates run BEFORE it claims the gate, so a gated-out read (a Peer origin, a cross-site read, an already-held
+capsule) never consumes a slot. Keep BOTH legs — dropping Leg A would lose the peerless fallback.
+
 ## Root has NO systemd `--user` bus, so a user-scope service install is impossible under `sudo` (#526)
 
 `systemctl --user` (and everything `service-manager`'s systemd backend does at `ServiceLevel::User`)

SPEC.md

@@ -2313,8 +2313,11 @@ a root OTHER than the confirmed root MUST be rejected (never cached or served).
   *actively seeks other nodes to pull missing generations* rather than only reacting to reads.
   (b) **Backfill-on-miss** — when a read is satisfied from another node or the upstream rather than
   from local disk, the node background-backfills the whole capsule so the NEXT read of that resource is
-  served locally (deduplicated: a backfill already in flight for `store:root` is not started twice).
-  Enabled by default; toggle with the `DIG_NODE_BACKFILL_ON_MISS` environment variable.
+  served locally (deduplicated: a backfill already in flight for `store:root` is not started twice). This
+  dedup spans BOTH whole-capsule transports: the §21 backfill here and the §21.3 P2P reshare warm claim
+  ONE shared single-flight gate keyed `(store, root)`, so a read starts at most one whole-capsule pull no
+  matter which leg wins, and the shared cap bounds concurrent distinct-generation acquisitions across the
+  two legs together. Enabled by default; toggle with the `DIG_NODE_BACKFILL_ON_MISS` environment variable.
 - **Fail-closed.** Gap-fill never pulls against an unconfirmable root (the §14.2 decision gates it).
 - **Verification invariant.** Every served module is verified against the chain-anchored root at SERVE,
   no matter how it arrived — a client read, a §21 whole-store sync, or a proactive/backfill gap-fill.
@@ -3913,9 +3916,17 @@ read's latency is user-facing. Serving a module range is paced by the SAME FCFS
 `dig.fetchRange` uses (§17) — a whole-capsule pull is the largest thing a node serves, so exempting it
 would leave the biggest transfer as the one path able to starve every other peer.
 
-At most ONE warm per generation runs at a time, so a burst of reads across a capsule's resources cannot
-start N concurrent pulls of the same module. A store-granularity read starts no warm: it does not name
-WHICH generation to pull, and guessing would reshare a capsule nobody asked for.
+A single read triggers AT MOST ONE whole-capsule acquisition for a `(store, root)` generation, regardless
+of tier. Two transports can pull the SAME capsule down — the §21 authenticated whole-store backfill
+(`maybe_backfill_capsule` → `gap_fill_generation`, the peerless-network fallback that acquires from the
+RPC upstream when no peer serves) and the §21.3 P2P reshare warm — and they are two routes to the same
+artifact, so they claim ONE shared single-flight gate keyed `(store, root)`. Whichever leg claims the key
+first runs the pull; the other, and any further read of the same not-yet-held capsule, is refused. A
+burst of reads across a capsule's resources therefore cannot start N concurrent pulls of the same module,
+and the two legs cannot double-pull it between them. The shared gate ALSO bounds how many DISTINCT
+generations may acquire concurrently (a fixed cap across BOTH legs, not each in isolation): claims beyond
+the cap are SKIPPED, not queued — the next read simply tries again. A store-granularity read starts no
+warm: it does not name WHICH generation to pull, and guessing would reshare a capsule nobody asked for.
 
 ### 21.5. Dial path (MUST)
 
@@ -3943,7 +3954,9 @@ A read can trigger background legs that spend this node's bandwidth and disk and
 advertises network-wide: the whole-capsule warm (`maybe_backfill_capsule`, §21.4) and the reshare pull
 plus holder-announce (§21.3). Those legs MUST be reachable only from a read this node's OWN OPERATOR
 made. A read that arrived from the network is SERVED normally and effects NOTHING: it starts no warm, no
-reshare, no promotion, and no announce.
+reshare, no promotion, and no announce. These two legs share ONE single-flight acquisition gate (§21.4),
+so the origin/config/held gates on EACH leg run BEFORE it can claim that gate — a gated-out read never
+consumes a slot, and the two legs dedup against each other rather than each pulling the capsule.
 
 The rule is not optional hardening. Every peer-facing read surface is unauthenticated in the sense that
 matters here — any well-formed self-signed mTLS leaf is accepted (§13), and the plaintext `/s/` surface

crates/dig-node-core/src/download.rs

@@ -1184,6 +1184,10 @@ impl NodeContent {
         anchor_resolver: Arc<dyn crate::shared::AnchoredRootResolver>,
         announce: Arc<dyn crate::seams::dig_peer::AnnounceHolder>,
         cache_dir: &Path,
+        // The node's SHARED single-flight acquisition gate (#1614). Passed in — NOT freshly created —
+        // so this reshare warm claims the SAME registry the §21 backfill leg does; the two transports
+        // for one capsule then dedup against each other instead of each pulling the whole `.dig`.
+        capsule_acquisition: Arc<crate::seams::dig_peer::WarmRegistry>,
     ) {
         let transport = Arc::new(crate::seams::dig_peer::NatModuleTransport::new(
             node_cert,
@@ -1205,7 +1209,7 @@ impl NodeContent {
                 cache_dir: cache_dir.to_path_buf(),
             },
             announce,
-            Arc::new(crate::seams::dig_peer::WarmRegistry::new()),
+            capsule_acquisition,
             dig_download::ModuleDownloadConfig::default(),
         ));
     }