Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions BEARING.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,11 @@ What exists now:
current cache and derived-state objects under `refs/cas/rootsets/*`. Updates
use parentless commits and compare-and-swap ref writes so removed entries can
become prunable instead of staying reachable through storage history.
- **Opaque application assets.** `ContentAddressableStore.assets`, `retention`,
and `publications` now compose streaming asset storage, canonical handles,
current-generation retention, and allowlisted compare-and-swap application
refs. Staged results and immutable witnesses keep content identity separate
from retention claims.
- **Migration script.** `scripts/migrate-encryption.js` upgrades legacy v1/v2
manifests to the current scheme identifiers.

Expand Down
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
storage, and repository/cache diagnostics as the `v6.2.0` implementation
boundary. Invariant I-002 makes `git-cas` the owner of physical CAS and cache
lifecycle while applications retain domain and causal semantics.
- **Opaque asset lifecycle API** — `cas.assets.put()` and `assets.open()` stream
through the existing CAS pipeline using immutable, canonical `AssetHandle`
values. Staged results explicitly report that the operation created no root,
while `cas.retention.retain()` returns generation-scoped evidence for the
exact RootSet tree edge that made the graph reachable.
- **Generic application publication** — `cas.publications.commit()` validates
handle graphs, bounded ordered commit parents, commit messages, explicit ref
allowlists, reserved Git/CAS namespaces, and expected heads before
compare-and-swap publication. Success returns a `RetentionWitness`; stale
heads report structured expected, observed, and attempted-commit evidence.
- **Portable handle validation** — canonical SHA-1 and SHA-256 handle tokens
contain no repository location, survive clone and mirror transfer, and fail
explicitly when their referenced graph is absent or has the wrong codec or
Git object type.

### Changed

- **Ordered commit parents** — `GitRefPort.createCommit()` now accepts an
additive `parentOids` array while retaining the existing single-`parentOid`
compatibility input. Create-only ref updates derive the Git null OID width
from the repository's commit ID instead of assuming SHA-1.

## [6.1.0] — 2026-07-11

Expand Down
95 changes: 82 additions & 13 deletions GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ limits, and port contracts live in the advanced guide.
| Feature Area | Covered Here | Advanced Coverage |
| ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Facade lifecycle, JSON/CBOR factories, full constructor | [Library Quick Start](#library-quick-start), [Configuration Reference](#configuration-reference) | [Direct CasService and Custom Port Contracts](./ADVANCED_GUIDE.md#direct-casservice-and-custom-port-contracts) |
| Opaque asset handles, retention evidence, application publication | [Library Quick Start](#library-quick-start), [Application Storage Lifecycle](#application-storage-lifecycle) | [Application Storage API](./docs/API.md#application-storage) |
| Direct `CasService` construction | [Configuration Reference](#configuration-reference) notes the facade/direct split | [Direct CasService and Custom Port Contracts](./ADVANCED_GUIDE.md#direct-casservice-and-custom-port-contracts) |
| File, stream, tree, vault-safe store workflows | [Store Operations](#store-operations), [Vault Management](#vault-management) | [Store/Restore Pipeline](./docs/STORE_RESTORE_PIPELINE.md), [Manifest Integrity Hash](./ADVANCED_GUIDE.md#manifest-integrity-hash), [Merkle Manifests](./ADVANCED_GUIDE.md#merkle-manifests) |
| Restore modes and bounded memory behavior | [Restore Modes](#restore-modes) | [Store/Restore Pipeline](./docs/STORE_RESTORE_PIPELINE.md), [Parallel Chunk Restore](./ADVANCED_GUIDE.md#parallel-chunk-restore), [Streaming Decompression](./ADVANCED_GUIDE.md#streaming-decompression) |
Expand All @@ -70,31 +71,99 @@ limits, and port contracts live in the advanced guide.

## Library Quick Start

A complete init-store-tree-restore cycle:
A complete stage-retain-open cycle:

```js
import ContentAddressableStore from '@git-stunts/git-cas';
import { createReadStream } from 'node:fs';
import ContentAddressableStore, { AssetHandle } from '@git-stunts/git-cas';

// 1. Initialize
const cas = await ContentAddressableStore.open({ cwd: '/path/to/repo' });

// 2. Store a file
const manifest = await cas.storeFile({
filePath: './photo.jpg',
// Stage bytes and receive an immutable content handle. This creates no root.
const staged = await cas.assets.put({
source: createReadStream('./photo.jpg'),
slug: 'photos/vacation',
filename: 'photo.jpg',
});

// 3. Create a Git tree (persists the manifest + chunks as a tree object)
const treeOid = await cas.createTree({ manifest });
// Make the graph Git-reachable through a current-generation RootSet.
const retained = await cas.retention.retain({
handle: staged.handle,
root: { ref: 'refs/cas/rootsets/photos', name: 'vacation' },
policy: 'pinned',
});

// 4. Add to the vault index (GC-safe ref)
await cas.addToVault({ slug: 'photos/vacation', treeOid });
// Stream the payload without exposing its manifest or chunk OIDs.
for await (const chunk of cas.assets.open({ handle: staged.handle })) {
consume(chunk);
}

// 5. Restore later
const readBack = await cas.readManifest({ treeOid });
const { buffer } = await cas.restore({ manifest: readBack });
console.log(retained.witness.root.generation);
```

Node's file stream is an `AsyncIterable<Uint8Array>`; Bun and Deno callers can
supply their native async byte streams through the same contract.

## Application Storage Lifecycle

The ordinary application boundary separates content identity from retention:

1. `cas.assets.put()` streams bytes through the existing chunking, compression,
encryption, and deduplication pipeline. It returns a `StagedAsset` whose
immutable `AssetHandle` locates the resulting manifest tree.
2. A staged result says this operation created no reachability root. It does
not claim that globally deduplicated objects are unreachable through every
other ref.
3. `cas.retention.retain()` installs the handle under a RootSet and returns a
`RetentionWitness` naming the exact generation and tree path observed.
4. `cas.publications.commit()` validates the handle graph, creates an
application commit with caller-controlled ordered parents, and updates an
explicitly allowed application ref by compare-and-swap.

Handles have a canonical, repository-location-independent token form:

```js
const token = staged.handle.toString();
const sameHandle = AssetHandle.parse(token);
```

The token is portable only with its referenced Git object graph. Opening it in
a clone or mirror succeeds after the graph is transferred; opening it in a
repository without that graph fails with `HANDLE_TARGET_MISSING`.

### Generic Application Publication

Publication is disabled until the facade receives an explicit application-ref
allowlist. The `refs/cas/*` namespace is always reserved.

```js
const cas = await ContentAddressableStore.open({
cwd: '/path/to/repo',
applicationRefPrefixes: ['refs/my-app/'],
});

const publication = await cas.publications.commit({
root: staged.handle,
commit: {
message: 'Publish vacation photo',
parents: previousCommitId === null ? [] : [previousCommitId],
},
ref: {
name: 'refs/my-app/assets',
expected: previousCommitId,
},
});
```

`expected` is mandatory, including `null` for create-only publication. A stale
head fails with `PUBLICATION_CONFLICT` and includes `expected`, `observed`, and
the attempted commit ID. The returned witness is generation-scoped: it proves
what the ref reached when publication succeeded, not that a mutable ref still
points there later.

`pinned` is a retention-policy term. It means ordinary cache eviction must not
release the entry; it does not create or imply a Git pack `.keep` file.

### Factory Methods

| Factory | Codec | Use Case |
Expand Down
24 changes: 23 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ Unlike traditional LFS which moves files to external servers, `git-cas` treats t
- **Mutable Root Sets**: Cache and derived-state trees can be anchored under
`refs/cas/rootsets/*` while live, then become collectible after eviction,
without vault history retaining every prior generation.
- **Opaque Application Handles**: Applications stage and stream assets through
immutable handles, then retain or publish them with generation-scoped
evidence instead of managing manifests, trees, or payload OIDs directly.
- **Key Lifecycle**: Envelope encryption separates DEKs from KEKs. Rotate passphrases across an entire vault without re-encrypting data blobs. Privacy mode HMAC-hashes slug names to prevent metadata discovery.
- **Runtime-Adaptive**: A single core supports Node.js 22+, Bun, and Deno through a strict hexagonal port architecture with runtime-specific crypto adapters.

Expand Down Expand Up @@ -57,11 +60,26 @@ git-cas vault dashboard
Integrate managed blob storage directly into your TypeScript or JavaScript application.

```js
import { createReadStream } from 'node:fs';
import ContentAddressableStore from '@git-stunts/git-cas';

const cas = await ContentAddressableStore.open({ cwd: '.' });
const manifest = await cas.storeFile({ filePath: './asset.bin', slug: 'app/asset' });
const staged = await cas.assets.put({
source: createReadStream('./asset.bin'),
slug: 'app/asset',
filename: 'asset.bin',
});
const retained = await cas.retention.retain({
handle: staged.handle,
root: { ref: 'refs/cas/rootsets/my-app', name: 'app/asset' },
policy: 'pinned',
});
```

`staged.handle` is a content locator, not a durability promise. The returned
retention witness identifies the exact Git generation and tree edge that made
the asset reachable.

## Capability Map

The README is the front door. Detailed mechanics live in the guide set:
Expand All @@ -73,6 +91,7 @@ The README is the front door. Detailed mechanics live in the guide set:
| Encryption scheme selection | [Encryption Modes](./docs/ENCRYPTION_MODES.md) |
| CDC internals, Merkle manifests, KDF policy, and tuning | [Advanced Guide](./ADVANCED_GUIDE.md) |
| Ports, adapters, and collaborator boundaries | [Architecture](./ARCHITECTURE.md) |
| Opaque handles, retention, and application publication | [Application storage](./docs/API.md#application-storage) |
| GC retention for caches and derived state | [Root Sets](./docs/API.md#root-sets) |
| v5 to v6 migration | [Upgrading](./UPGRADING.md) |

Expand All @@ -91,6 +110,9 @@ Core capabilities:
- **Root-set retention**: current cache entries live under caller-owned
`refs/cas/rootsets/*` refs. Entries are Git-reachable while present, and old
set generations are not retained as commit history.
- **Application storage**: `assets`, `retention`, and `publications` compose
streaming CAS writes, validated handles, reachability roots, compare-and-swap
refs, and immutable lifecycle evidence.
- **Envelope recipients**: multi-recipient key wrapping and recipient rotation
avoid re-encrypting data blobs.
- **Operational diagnostics**: `git-cas doctor` validates vault health and
Expand Down
Loading
Loading