Skip to content

design: path-local persistent bundle derivation #86

Description

@flyingrobots

Problem

Applications with immutable sharded indexes can read one bundle member in bounded work, but changing a few members requires callers to enumerate and rebuild the complete bundle. This preserves payload deduplication but makes incremental materialization O(total members), repeats direct-edge validation, and prevents the storage layer from reusing unchanged fanout nodes.

The immediate evidence is git-warp incremental materialization: adding five nodes to a retained 100-node corpus still emitted 166 blob writes, 220 object metadata checks, and 399 targeted tree lookups. The current flat-property profile rebuilds every shard rather than deriving a new root from the old root.

Required design

Specify a lawful git-cas operation such as bundles.derive() or bundles.update() that accepts a retained base bundle plus ordered puts/deletes and returns a staged immutable bundle. Existing validated support should be reused without weakening integrity; only changed fanout paths should be rewritten when the persisted layout permits it.

Acceptance criteria

  • Define base-retention and witness preconditions.
  • Define puts, replacements, deletes, insertion, and no-op semantics.
  • Preserve canonical ordering, limits, descriptor summaries, and support validation.
  • Bound memory by fanout depth plus update window rather than total members.
  • Prove unchanged fanout subtrees are structurally shared.
  • Publish command-count benchmarks for one update in a bundle much larger than memory.
  • Expose no raw Git OIDs or mutable-ref authority to callers.

Downstream

Needed for truly sublinear retained-index updates in git-warp and other git-cas consumers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:runtimeRuntime portability or public API behaviorarea:storageStorage, Git objects, manifests, or persistencestatus:needs-designRequires a design before implementationtype:designDesign doc or proof-plan worktype:sliceTurn-sized proof work under a goalpost

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions