diff --git a/BEARING.md b/BEARING.md index 227cb66c..66ee2a41 100644 --- a/BEARING.md +++ b/BEARING.md @@ -62,6 +62,11 @@ What exists now: current cache and derived-state objects under `refs/cas/rootsets/*`. Updates use parentless commits and compare-and-swap ref writes so removed entries can become prunable instead of staying reachable through storage history. +- **Opaque application assets.** `ContentAddressableStore.assets`, `retention`, + and `publications` now compose streaming asset storage, canonical handles, + current-generation retention, and allowlisted compare-and-swap application + refs. Staged results and immutable witnesses keep content identity separate + from retention claims. - **Migration script.** `scripts/migrate-encryption.js` upgrades legacy v1/v2 manifests to the current scheme identifiers. diff --git a/CHANGELOG.md b/CHANGELOG.md index 97cd5663..14b1fd25 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 storage, and repository/cache diagnostics as the `v6.2.0` implementation boundary. Invariant I-002 makes `git-cas` the owner of physical CAS and cache lifecycle while applications retain domain and causal semantics. +- **Opaque asset lifecycle API** — `cas.assets.put()` and `assets.open()` stream + through the existing CAS pipeline using immutable, canonical `AssetHandle` + values. Staged results explicitly report that the operation created no root, + while `cas.retention.retain()` returns generation-scoped evidence for the + exact RootSet tree edge that made the graph reachable. +- **Generic application publication** — `cas.publications.commit()` validates + handle graphs, bounded ordered commit parents, commit messages, explicit ref + allowlists, reserved Git/CAS namespaces, and expected heads before + compare-and-swap publication. Success returns a `RetentionWitness`; stale + heads report structured expected, observed, and attempted-commit evidence. +- **Portable handle validation** — canonical SHA-1 and SHA-256 handle tokens + contain no repository location, survive clone and mirror transfer, and fail + explicitly when their referenced graph is absent or has the wrong codec or + Git object type. + +### Changed + +- **Ordered commit parents** — `GitRefPort.createCommit()` now accepts an + additive `parentOids` array while retaining the existing single-`parentOid` + compatibility input. Create-only ref updates derive the Git null OID width + from the repository's commit ID instead of assuming SHA-1. ## [6.1.0] — 2026-07-11 diff --git a/GUIDE.md b/GUIDE.md index cca293d4..44a000f2 100644 --- a/GUIDE.md +++ b/GUIDE.md @@ -51,6 +51,7 @@ limits, and port contracts live in the advanced guide. | Feature Area | Covered Here | Advanced Coverage | | ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Facade lifecycle, JSON/CBOR factories, full constructor | [Library Quick Start](#library-quick-start), [Configuration Reference](#configuration-reference) | [Direct CasService and Custom Port Contracts](./ADVANCED_GUIDE.md#direct-casservice-and-custom-port-contracts) | +| Opaque asset handles, retention evidence, application publication | [Library Quick Start](#library-quick-start), [Application Storage Lifecycle](#application-storage-lifecycle) | [Application Storage API](./docs/API.md#application-storage) | | Direct `CasService` construction | [Configuration Reference](#configuration-reference) notes the facade/direct split | [Direct CasService and Custom Port Contracts](./ADVANCED_GUIDE.md#direct-casservice-and-custom-port-contracts) | | File, stream, tree, vault-safe store workflows | [Store Operations](#store-operations), [Vault Management](#vault-management) | [Store/Restore Pipeline](./docs/STORE_RESTORE_PIPELINE.md), [Manifest Integrity Hash](./ADVANCED_GUIDE.md#manifest-integrity-hash), [Merkle Manifests](./ADVANCED_GUIDE.md#merkle-manifests) | | Restore modes and bounded memory behavior | [Restore Modes](#restore-modes) | [Store/Restore Pipeline](./docs/STORE_RESTORE_PIPELINE.md), [Parallel Chunk Restore](./ADVANCED_GUIDE.md#parallel-chunk-restore), [Streaming Decompression](./ADVANCED_GUIDE.md#streaming-decompression) | @@ -70,31 +71,99 @@ limits, and port contracts live in the advanced guide. ## Library Quick Start -A complete init-store-tree-restore cycle: +A complete stage-retain-open cycle: ```js -import ContentAddressableStore from '@git-stunts/git-cas'; +import { createReadStream } from 'node:fs'; +import ContentAddressableStore, { AssetHandle } from '@git-stunts/git-cas'; -// 1. Initialize const cas = await ContentAddressableStore.open({ cwd: '/path/to/repo' }); -// 2. Store a file -const manifest = await cas.storeFile({ - filePath: './photo.jpg', +// Stage bytes and receive an immutable content handle. This creates no root. +const staged = await cas.assets.put({ + source: createReadStream('./photo.jpg'), slug: 'photos/vacation', + filename: 'photo.jpg', }); -// 3. Create a Git tree (persists the manifest + chunks as a tree object) -const treeOid = await cas.createTree({ manifest }); +// Make the graph Git-reachable through a current-generation RootSet. +const retained = await cas.retention.retain({ + handle: staged.handle, + root: { ref: 'refs/cas/rootsets/photos', name: 'vacation' }, + policy: 'pinned', +}); -// 4. Add to the vault index (GC-safe ref) -await cas.addToVault({ slug: 'photos/vacation', treeOid }); +// Stream the payload without exposing its manifest or chunk OIDs. +for await (const chunk of cas.assets.open({ handle: staged.handle })) { + consume(chunk); +} -// 5. Restore later -const readBack = await cas.readManifest({ treeOid }); -const { buffer } = await cas.restore({ manifest: readBack }); +console.log(retained.witness.root.generation); +``` + +Node's file stream is an `AsyncIterable`; Bun and Deno callers can +supply their native async byte streams through the same contract. + +## Application Storage Lifecycle + +The ordinary application boundary separates content identity from retention: + +1. `cas.assets.put()` streams bytes through the existing chunking, compression, + encryption, and deduplication pipeline. It returns a `StagedAsset` whose + immutable `AssetHandle` locates the resulting manifest tree. +2. A staged result says this operation created no reachability root. It does + not claim that globally deduplicated objects are unreachable through every + other ref. +3. `cas.retention.retain()` installs the handle under a RootSet and returns a + `RetentionWitness` naming the exact generation and tree path observed. +4. `cas.publications.commit()` validates the handle graph, creates an + application commit with caller-controlled ordered parents, and updates an + explicitly allowed application ref by compare-and-swap. + +Handles have a canonical, repository-location-independent token form: + +```js +const token = staged.handle.toString(); +const sameHandle = AssetHandle.parse(token); +``` + +The token is portable only with its referenced Git object graph. Opening it in +a clone or mirror succeeds after the graph is transferred; opening it in a +repository without that graph fails with `HANDLE_TARGET_MISSING`. + +### Generic Application Publication + +Publication is disabled until the facade receives an explicit application-ref +allowlist. The `refs/cas/*` namespace is always reserved. + +```js +const cas = await ContentAddressableStore.open({ + cwd: '/path/to/repo', + applicationRefPrefixes: ['refs/my-app/'], +}); + +const publication = await cas.publications.commit({ + root: staged.handle, + commit: { + message: 'Publish vacation photo', + parents: previousCommitId === null ? [] : [previousCommitId], + }, + ref: { + name: 'refs/my-app/assets', + expected: previousCommitId, + }, +}); ``` +`expected` is mandatory, including `null` for create-only publication. A stale +head fails with `PUBLICATION_CONFLICT` and includes `expected`, `observed`, and +the attempted commit ID. The returned witness is generation-scoped: it proves +what the ref reached when publication succeeded, not that a mutable ref still +points there later. + +`pinned` is a retention-policy term. It means ordinary cache eviction must not +release the entry; it does not create or imply a Git pack `.keep` file. + ### Factory Methods | Factory | Codec | Use Case | diff --git a/README.md b/README.md index e8aadbae..e067d5d9 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,9 @@ Unlike traditional LFS which moves files to external servers, `git-cas` treats t - **Mutable Root Sets**: Cache and derived-state trees can be anchored under `refs/cas/rootsets/*` while live, then become collectible after eviction, without vault history retaining every prior generation. +- **Opaque Application Handles**: Applications stage and stream assets through + immutable handles, then retain or publish them with generation-scoped + evidence instead of managing manifests, trees, or payload OIDs directly. - **Key Lifecycle**: Envelope encryption separates DEKs from KEKs. Rotate passphrases across an entire vault without re-encrypting data blobs. Privacy mode HMAC-hashes slug names to prevent metadata discovery. - **Runtime-Adaptive**: A single core supports Node.js 22+, Bun, and Deno through a strict hexagonal port architecture with runtime-specific crypto adapters. @@ -57,11 +60,26 @@ git-cas vault dashboard Integrate managed blob storage directly into your TypeScript or JavaScript application. ```js +import { createReadStream } from 'node:fs'; import ContentAddressableStore from '@git-stunts/git-cas'; + const cas = await ContentAddressableStore.open({ cwd: '.' }); -const manifest = await cas.storeFile({ filePath: './asset.bin', slug: 'app/asset' }); +const staged = await cas.assets.put({ + source: createReadStream('./asset.bin'), + slug: 'app/asset', + filename: 'asset.bin', +}); +const retained = await cas.retention.retain({ + handle: staged.handle, + root: { ref: 'refs/cas/rootsets/my-app', name: 'app/asset' }, + policy: 'pinned', +}); ``` +`staged.handle` is a content locator, not a durability promise. The returned +retention witness identifies the exact Git generation and tree edge that made +the asset reachable. + ## Capability Map The README is the front door. Detailed mechanics live in the guide set: @@ -73,6 +91,7 @@ The README is the front door. Detailed mechanics live in the guide set: | Encryption scheme selection | [Encryption Modes](./docs/ENCRYPTION_MODES.md) | | CDC internals, Merkle manifests, KDF policy, and tuning | [Advanced Guide](./ADVANCED_GUIDE.md) | | Ports, adapters, and collaborator boundaries | [Architecture](./ARCHITECTURE.md) | +| Opaque handles, retention, and application publication | [Application storage](./docs/API.md#application-storage) | | GC retention for caches and derived state | [Root Sets](./docs/API.md#root-sets) | | v5 to v6 migration | [Upgrading](./UPGRADING.md) | @@ -91,6 +110,9 @@ Core capabilities: - **Root-set retention**: current cache entries live under caller-owned `refs/cas/rootsets/*` refs. Entries are Git-reachable while present, and old set generations are not retained as commit history. +- **Application storage**: `assets`, `retention`, and `publications` compose + streaming CAS writes, validated handles, reachability roots, compare-and-swap + refs, and immutable lifecycle evidence. - **Envelope recipients**: multi-recipient key wrapping and recipient rotation avoid re-encrypting data blobs. - **Operational diagnostics**: `git-cas doctor` validates vault health and diff --git a/docs/API.md b/docs/API.md index d4b0ea3f..8da7e461 100644 --- a/docs/API.md +++ b/docs/API.md @@ -15,14 +15,15 @@ should treat restored data, chunker output, codec output, and keys as ## Table of Contents 1. [ContentAddressableStore](#contentaddressablestore) -2. [Root Sets](#root-sets) -3. [Vault](#vault) -4. [CasService](#casservice) -5. [Events](#events) -6. [Value Objects](#value-objects) -7. [Ports](#ports) -8. [Codecs](#codecs) -9. [Error Codes](#error-codes) +2. [Application Storage](#application-storage) +3. [Root Sets](#root-sets) +4. [Vault](#vault) +5. [CasService](#casservice) +6. [Events](#events) +7. [Value Objects](#value-objects) +8. [Ports](#ports) +9. [Codecs](#codecs) +10. [Error Codes](#error-codes) ## ContentAddressableStore @@ -75,6 +76,8 @@ new ContentAddressableStore(options); - `options.maxRestoreBufferSize` (optional): Max bytes for buffered encrypted/compressed restore (default: 536870912 / 512 MiB) - `options.maxBlobSize` (optional): Max bytes for metadata blob reads (default: 10485760 / 10 MiB) - `options.compressionAdapter` (optional): CompressionPort implementation (default: NodeCompressionAdapter) +- `options.applicationRefPrefixes` (optional): Explicit application-owned ref prefixes allowed for generic publication; publication is disabled when omitted, and Git/CAS-managed namespaces are always reserved +- `options.clock` (optional): `{ now(): Date }` clock used for deterministic staged results and retention witnesses **Example:** @@ -905,6 +908,215 @@ Returns the configured chunk size in bytes. console.log(cas.chunkSize); // 262144 ``` +## Application Storage + +The high-level application boundary exposes immutable content handles and +explicit lifecycle operations. It composes the lower-level store, manifest, +tree, RootSet, commit, and ref operations without requiring callers to manage +payload OIDs. + +### `assets.put()` + +```javascript +const staged = await cas.assets.put({ + source, + slug, + filename, + encryptionKey, + passphrase, + encryption, + kdfOptions, + compression, + recipients, + merkleThreshold, + chunking, +}); +``` + +Streams an `AsyncIterable` through the existing CAS pipeline, +creates its manifest tree, and returns an immutable `StagedAsset`. Payload +bytes are consumed through the configured chunker and are not returned in the +result. Manifest metadata remains subject to the existing manifest and Merkle +limits documented in the streaming matrix. + +`filename` defaults to `slug`. All other storage options have the same meaning +as `store()`. + +**Returns:** `Promise` + +```javascript +{ + version: 1, + state: 'staged', + handle: AssetHandle, + asset: { slug, filename, size }, + retention: { + policy: null, + reachability: 'unanchored', + protection: 'not-established', + }, + observedAt: '2026-07-13T10:00:00.000Z', +} +``` + +`unanchored` means this operation created no reachability root. It is not a +global assertion that a deduplicated object graph is unreachable through every +other Git ref. `not-established` likewise means this operation made no +protection claim. Call `retention.retain()` or `publications.commit()` before +relying on a new handle beyond Git's unreachable-object grace period. + +### `assets.open()` + +```javascript +for await (const chunk of cas.assets.open({ + handle, + encryptionKey, + passphrase, +})) { + consume(chunk); +} +``` + +Validates the canonical handle, codec, root tree, manifest, and referenced +chunk graph, then returns restored bytes as an `AsyncIterable`. +Encrypted restore behavior follows the existing streaming matrix; explicit +`whole` encryption may use its documented bounded compatibility buffer. + +Missing transferred objects fail with `HANDLE_TARGET_MISSING`. A handle created +for a different manifest codec fails with `HANDLE_CODEC_MISMATCH`. + +### `assets.adopt()` + +```javascript +const staged = await cas.assets.adopt({ treeOid }); +``` + +Validates an existing git-cas manifest tree and wraps it in the same staged +result returned by `assets.put()`. This is a migration bridge for callers that +already persisted raw tree OIDs; new application code should exchange handles. + +### `AssetHandle` + +`AssetHandle` is an immutable, versioned content locator. Its canonical token +has this shape: + +```text +git-cas:1:asset:manifest-tree::: +``` + +```javascript +import { AssetHandle } from '@git-stunts/git-cas'; + +const token = staged.handle.toString(); +const parsed = AssetHandle.parse(token); +const data = parsed.toJSON(); +``` + +The token contains no filesystem or repository location. It works in another +clone or mirror only after the referenced Git object graph has been transferred. +"Opaque" is an ownership boundary, not encryption: callers may serialize and +compare the token, while `git-cas` owns interpretation and object traversal. + +### `retention.retain()` + +```javascript +const result = await cas.retention.retain({ + handle, + root: { + ref: 'refs/cas/rootsets/my-app-cache', + name: 'coordinate-184', + }, + policy: 'evictable', +}); +``` + +Validates the handle graph, installs its tree in the named current-generation +RootSet, and returns: + +```javascript +{ + changed: true, + witness: RetentionWitness, +} +``` + +The witness records `policy`, observed `reachability`, timestamp, RootSet ref, +generation commit, and the exact tree path that retained the handle. The +default policy is `pinned`; `evictable` allows higher-level cache policy to +release the entry. Neither policy creates a Git pack `.keep` file. + +### `publications.commit()` + +```javascript +const result = await cas.publications.commit({ + root: handle, + commit: { + message: 'Publish application state', + parents: previous === null ? [] : [previous], + }, + ref: { + name: 'refs/my-app/state', + expected: previous, + }, +}); +``` + +Generic publication is available only below a constructor-configured +`applicationRefPrefixes` allowlist. Even a broad allowlist cannot publish below +`refs/bisect/*`, `refs/cas/*`, `refs/heads/*`, `refs/notes/*`, +`refs/remotes/*`, `refs/replace/*`, `refs/rewritten/*`, `refs/stash`, +`refs/tags/*`, or `refs/worktree/*`. The method: + +1. validates the complete supported handle graph; +2. validates at most 64 ordered parent commit IDs and a commit message of at + most 1 MiB; +3. creates a commit whose tree is the validated handle root; and +4. updates the application ref with compare-and-swap semantics. + +`ref.expected` is mandatory. Use `null` to require that the ref not exist. +A stale expectation fails with `PUBLICATION_CONFLICT`; error metadata includes +`expected`, `observed`, and `attemptedCommitId`. Because Git objects are +immutable, a failed ref update can leave the attempted commit unreachable until +normal Git pruning; it cannot publish a partial ref state. + +**Returns:** + +```javascript +{ + operation: 'publication', + commitId, + ref, + root: AssetHandle, + witness: RetentionWitness, +} +``` + +### `RetentionWitness` + +A `RetentionWitness` is immutable evidence for one observed retaining +generation. It does not promise that a mutable ref still points to that +generation later. + +```javascript +{ + version: 1, + handle: AssetHandle, + policy: 'pinned' | 'evictable', + reachability: 'anchored' | 'orphaned' | 'volatile', + root: { + kind: 'root-set' | 'publication' | 'cache-set' | 'expiring-set', + namespace: string, + ref: string, + generation: string, + path: string, + }, + observedAt: string, +} +``` + +`toJSON()` serializes `handle` to its canonical token and copies the root +evidence fields. + ## Root Sets Root sets retain a mutable current set of Git blobs or trees. They are intended @@ -2334,15 +2546,25 @@ new CasError({ message, code, meta, documentationUrl }); | `GIT_ERROR` | Underlying Git plumbing command failed | `readManifest()`, `inspectAsset()`, `collectReferencedChunks()` | | `GIT_REF_NOT_FOUND` | Git ref lookup found no ref; vault reads normalize this to empty state | `GitRefAdapter`, `VaultPersistence` | | `INVALID_OPTIONS` | Mutually exclusive options provided or unsupported option value | `store()`, `restore()` | +| `HANDLE_INVALID` | Handle object or canonical token is malformed or unsupported | `AssetHandle`, `StagedAsset` | +| `HANDLE_KIND_MISMATCH` | A handle has a valid envelope but the wrong content kind | Handle parsing and application storage | +| `HANDLE_CODEC_MISMATCH` | Handle manifest codec differs from the active CAS codec | `assets.open()`, `assets.adopt()`, retention, publication | +| `HANDLE_TARGET_MISSING` | The repository does not contain the complete object graph referenced by a handle | `assets.open()`, `assets.adopt()`, retention, publication | +| `HANDLE_TARGET_TYPE_MISMATCH` | A handle root or referenced object has the wrong Git object type | Application storage validation | +| `RETENTION_WITNESS_INVALID` | Witness policy, reachability, root evidence, generation, or timestamp is invalid | `RetentionWitness` | +| `PUBLICATION_INVALID` | Publication message, parent list, expected head, dependency, or clock input is invalid | `publications.commit()` | +| `PUBLICATION_REF_FORBIDDEN` | Target ref is invalid, reserved, or outside configured application namespaces | `publications.commit()` | +| `PUBLICATION_CONFLICT` | Application ref head differs from the explicit expected head | `publications.commit()` | +| `PUBLICATION_REF_UPDATE_FAILED` | Application ref update or post-failure head observation failed for a non-conflict reason | `publications.commit()` | | `INVALID_SLUG` | Slug fails validation (empty, control chars, `..` segments, etc.) | `addToVault()` | | `ROOT_SET_REF_INVALID` | Ref is outside `refs/cas/rootsets/*` or is not a safe Git ref | `rootSets.open()` | | `ROOT_SET_ENTRY_INVALID` | Entry name, OID, type, retention, or duplicate-name validation failed | `put()`, `replace()`, `mutate()`, `repair()` | | `ROOT_SET_TARGET_MISSING` | A requested target OID does not exist | `put()`, `replace()`, `mutate()`, `repair()` | -| `ROOT_SET_TARGET_UNREADABLE` | Git could not inspect a target, but did not prove that it was missing | Root-set mutation and doctor | +| `ROOT_SET_TARGET_UNREADABLE` | Git could not inspect a target, but did not prove that it was missing | Root-set mutation and doctor | | `ROOT_SET_TARGET_TYPE_MISMATCH` | Declared blob/tree type does not match the Git object | `put()`, `replace()`, `mutate()`, `repair()` | | `ROOT_SET_CONFLICT` | Concurrent root-set update exhausted bounded compare-and-swap retries | Root-set mutations and repair | | `ROOT_SET_HEAD_INVALID` | Root-set ref cannot resolve to a readable commit tree | `read()`, `list()`, `doctor()` | -| `ROOT_SET_METADATA_INVALID` | `.rootset.json` is malformed, non-canonical, or belongs to another ref | `read()`, `list()`, `doctor()` | +| `ROOT_SET_METADATA_INVALID` | `.rootset.json` is malformed, non-canonical, or belongs to another ref | `read()`, `list()`, `doctor()` | | `ROOT_SET_TREE_INVALID` | Metadata and the Git tree's actual reachability edges disagree | `read()`, `list()`, `doctor()` | | `ROOT_SET_REF_UPDATE_FAILED` | Root-set ref update failed for a non-conflict reason | Root-set mutations and repair | | `VAULT_ENTRY_NOT_FOUND` | Slug does not exist in vault | `removeFromVault()`, `resolveVaultEntry()` | diff --git a/docs/design/0047-application-storage-cache-boundary/application-storage-cache-boundary.md b/docs/design/0047-application-storage-cache-boundary/application-storage-cache-boundary.md index d8fbbacc..95fe834c 100644 --- a/docs/design/0047-application-storage-cache-boundary/application-storage-cache-boundary.md +++ b/docs/design/0047-application-storage-cache-boundary/application-storage-cache-boundary.md @@ -178,7 +178,7 @@ const cas = new ContentAddressableStore(options); const avatar = await cas.assets.put({ source: avatarByteStream, slug: 'alice-avatar', - mime: 'image/png', + filename: 'alice-avatar.png', }); const materialization = await cas.bundles.put({ @@ -794,6 +794,9 @@ its own committed witness under its cycle directory. Design review evidence is recorded in [witness/design-review.md](./witness/design-review.md). +Opaque asset handle, retention, and publication implementation evidence is +recorded in [witness/asset-handles.md](./witness/asset-handles.md). + ## Risks - The facade could become a new monolith. Mitigation: capability groups delegate diff --git a/docs/design/0047-application-storage-cache-boundary/witness/asset-handles.md b/docs/design/0047-application-storage-cache-boundary/witness/asset-handles.md new file mode 100644 index 00000000..c97bc9e7 --- /dev/null +++ b/docs/design/0047-application-storage-cache-boundary/witness/asset-handles.md @@ -0,0 +1,171 @@ +# API-0047 Opaque Asset Handle Witness + +Implementation slice: [#54](https://github.com/git-stunts/git-cas/issues/54) + +Implementation commit: `a9e47ef3025d68b98c752fb1abb826bde856ad83` + +Review correction commits: + +- `3d6eaa49479fbbc3f6cf6cf191343bc0532783dd` +- `18cd397b24087d232311f973a6cb69d335bb0114` + +## Claim + +Applications can stream asset bytes into and out of `git-cas`, exchange a +canonical immutable handle, make the referenced graph reachable through a +managed root, or publish it through an allowlisted application ref without +sequencing manifests, trees, payload OIDs, commits, and ref updates themselves. + +## Source Evidence + +- The facade exposes frozen `assets`, `retention`, and `publications` + capabilities while preserving the existing low-level surface. + [cite: `index.js#153-168@a9e47ef3025d68b98c752fb1abb826bde856ad83`] +- `AssetHandle` validates version, kind, format, codec, hash algorithm, and OID, + and produces one canonical repository-location-independent token. + [cite: `src/domain/value-objects/AssetHandle.js#25-118@a9e47ef3025d68b98c752fb1abb826bde856ad83`] +- `assets.put()` composes the existing streaming store and manifest-tree path; + `assets.open()` delegates restored output to `restoreStream()` after handle + graph validation. + [cite: `src/domain/services/AssetService.js#26-101@a9e47ef3025d68b98c752fb1abb826bde856ad83`] +- A staged result explicitly says the operation created no retention root. It + does not mislabel globally deduplicated objects as proven unreachable. + [cite: `src/domain/value-objects/StagedAsset.js#17-53@fd7e6e5d1b540ad879730adf526d187441155e59`] +- `retention.retain()` validates the handle, installs the tree through RootSet, + and reports the exact committed generation and evidence path. + [cite: `src/domain/services/RetentionService.js#37-61@a9e47ef3025d68b98c752fb1abb826bde856ad83`] +- `publications.commit()` validates ordered commit parents and the handle graph, + creates the application commit, performs an expected-head ref update, and + returns generation-scoped evidence. + [cite: `src/domain/services/PublicationService.js#43-134@a9e47ef3025d68b98c752fb1abb826bde856ad83`] +- Witness construction validates policy, observed reachability, root kind, + fully qualified ref, generation OID, evidence path, and canonical timestamp. + [cite: `src/domain/value-objects/RetentionWitness.js#22-110@a9e47ef3025d68b98c752fb1abb826bde856ad83`] + +## Real Git Evidence + +The Docker-gated integration test creates a bare repository and checks Git's +own object and ref behavior: + +- a fresh staged tree appears in `git prune -n --expire=now` output; +- RootSet retention removes that tree from the prune candidate set; +- the reported generation and path resolve to the retained tree; +- replacement makes the prior unique tree prunable while retaining the winner; +- application publication preserves caller-controlled parent order; +- a tree OID is rejected as a commit parent; +- a stale expected head reports a conflict without replacing the winning ref; +- one canonical handle opens after mirror transfer and fails explicitly in an + empty repository. + +[cite: `test/integration/application-storage.test.js#102-199@a9e47ef3025d68b98c752fb1abb826bde856ad83`] + +## Self-Review + +The implementation was reviewed against #54 and API-0047 for ownership, +durability claims, concurrency, compatibility, and runtime-neutral byte +contracts. + +- Existing `store`, `restore`, `createTree`, vault, RootSet, and single-parent + `GitRefPort.createCommit()` callers remain compatible. +- New facade capabilities are additive and lazily share the existing adapters + and service initialization. +- Publication is disabled without an explicit application ref prefix and + always rejects Git/CAS-managed ref namespaces, even under a broad `refs/` + allowlist. +- `expected` is mandatory, including `null` for create-only publication. +- Handles expose content identity but make no durability promise. +- Publication commit IDs remain public application identities; payload object + traversal remains owned by `git-cas`. +- Complete handle-graph validation deduplicates chunk OIDs and bounds concurrent + object-type reads by the configured CAS concurrency. + +## Code Lawyer Review + +### CL-001: Tree peeling did not prove that a parent was a commit + +The first draft used `resolveTree(parent)`. Git's `^{tree}` syntax also accepts +a tree OID, so that check could admit a non-commit parent and defer failure to +`commit-tree`. Parent admission now uses `resolveParents()`, and the real-Git +test supplies a tree OID to prove structured rejection. +[cite: `src/domain/services/PublicationService.js#80-99@a9e47ef3025d68b98c752fb1abb826bde856ad83`] + +### CL-002: Create-only ref updates assumed SHA-1 + +The existing adapter used forty zeroes for Git's null OID. It now derives the +null OID width from the new commit ID, preserving SHA-1 behavior and supporting +SHA-256 repositories. +[cite: `src/infrastructure/adapters/GitRefAdapter.js#106-121@a9e47ef3025d68b98c752fb1abb826bde856ad83`] + +### CL-003: Overlapping publication allowlists obscured provenance + +Allowlisted prefixes are sorted most-specific-first, so a witness for +`refs/warp/cache/events` records `refs/warp/cache/` rather than the broader +`refs/warp/` namespace. +[cite: `src/domain/services/PublicationService.js#188-211@a9e47ef3025d68b98c752fb1abb826bde856ad83`] + +### CL-004: Ref observation could mask the original update failure + +If the ref update and the subsequent head observation both fail, the service +now reports `PUBLICATION_REF_UPDATE_FAILED` with both causes rather than leaking +an unclassified observation error. +[cite: `src/domain/services/PublicationService.js#101-134@a9e47ef3025d68b98c752fb1abb826bde856ad83`] + +### CL-005: Staged reachability cannot be inferred globally under deduplication + +The result uses `unanchored` and `not-established` to state that this operation +created no root or protection claim. It does not call the graph `orphaned`, +because identical objects may already be reachable through another ref. The +real-Git prune assertion is deliberately made against fresh unique test +content. +[cite: `src/domain/value-objects/StagedAsset.js#29-39@fd7e6e5d1b540ad879730adf526d187441155e59`] + +### CL-006: Sequential validation made large handle graphs latency-bound + +Complete handle validation now deduplicates chunk OIDs in one pass and checks +them through a fixed worker pool capped by the CAS concurrency. This avoids one +serial Git round trip per unique chunk without creating one promise per chunk. +The test instruments active object-type reads and proves the configured bound. +[cite: `src/domain/services/AssetService.js#103-121@3d6eaa49479fbbc3f6cf6cf191343bc0532783dd`] +[cite: `test/unit/domain/services/AssetService.test.js#93-122@3d6eaa49479fbbc3f6cf6cf191343bc0532783dd`] + +### CL-007: A broad allowlist could authorize Git-managed refs + +Publication now hard-blocks core Git and CAS namespaces independently of the +application allowlist. Both reserved prefix configuration and publication below +a reserved namespace fail before commit construction or ref mutation. +[cite: `src/domain/services/PublicationService.js#11-22@3d6eaa49479fbbc3f6cf6cf191343bc0532783dd`] +[cite: `src/domain/services/PublicationService.js#159-168@3d6eaa49479fbbc3f6cf6cf191343bc0532783dd`] +[cite: `src/domain/services/PublicationService.js#200-242@3d6eaa49479fbbc3f6cf6cf191343bc0532783dd`] +[cite: `test/unit/domain/services/PublicationService.test.js#145-184@3d6eaa49479fbbc3f6cf6cf191343bc0532783dd`] + +### CL-008: Retention validated RootSet refs after handle resolution + +Retention now validates the RootSet namespace before resolving the asset graph. +The negative test proves that a malformed ref performs neither handle +resolution nor ref mutation. +[cite: `src/domain/services/RetentionService.js#37-40@3d6eaa49479fbbc3f6cf6cf191343bc0532783dd`] +[cite: `src/domain/services/RetentionService.js#87-101@3d6eaa49479fbbc3f6cf6cf191343bc0532783dd`] +[cite: `test/unit/domain/services/RetentionService.test.js#93-108@3d6eaa49479fbbc3f6cf6cf191343bc0532783dd`] + +## Residual Constraints + +- Asset payload ingress and plaintext/framed/convergent egress use the existing + streaming pipeline. Manifest metadata is still materialized by the existing + `Manifest` model, and explicit `whole` encryption retains its documented + bounded compatibility-buffer behavior. This slice does not claim constant + metadata residency. +- A retention witness proves one observed generation. A mutable ref may move + afterward. +- This slice publishes `AssetHandle`. Bundle and page handle publication is + added by #51 through the same resolver boundary. +- A failed compare-and-swap can leave its immutable attempted commit + unreachable. That object is ordinary Git prune material, not partial + publication. + +## Validation + +- `pnpm lint` +- `pnpm test`: 198 files passed; 1,746 tests passed; 2 skipped +- `pnpm test:integration:node`: 7 files and 161 tests passed +- Docker Bun application-storage integration: 3 tests passed +- Docker Deno application-storage integration: 3 tests passed diff --git a/index.d.ts b/index.d.ts index 922ff4f6..88937355 100644 --- a/index.d.ts +++ b/index.d.ts @@ -41,6 +41,115 @@ export type { RecipientEntry, EncryptionScheme, }; + +export interface AssetHandleData { + version: 1; + kind: 'asset'; + format: 'manifest-tree'; + codec: string; + hashAlgorithm: 'sha1' | 'sha256'; + oid: string; +} + +export interface AssetHandleInit { + version?: 1; + kind?: 'asset'; + format?: 'manifest-tree'; + codec: string; + hashAlgorithm?: 'sha1' | 'sha256'; + oid: string; +} + +export type AssetHandleInput = AssetHandle | AssetHandleData | AssetHandleInit | string; + +/** Repository-independent locator for one validated git-cas asset graph. */ +export declare class AssetHandle implements AssetHandleData { + readonly version: 1; + readonly kind: 'asset'; + readonly format: 'manifest-tree'; + readonly codec: string; + readonly hashAlgorithm: 'sha1' | 'sha256'; + readonly oid: string; + constructor(value: AssetHandleInit); + static from(value: AssetHandleInput): AssetHandle; + static parse(token: string): AssetHandle; + toString(): string; + toJSON(): AssetHandleData; +} + +export interface StagedAssetData { + version: 1; + state: 'staged'; + handle: string; + asset: { slug: string; filename: string; size: number }; + retention: { + policy: null; + reachability: 'unanchored'; + protection: 'not-established'; + }; + observedAt: string; +} + +/** Result for an asset graph written without a reachability root. */ +export declare class StagedAsset { + readonly version: 1; + readonly state: 'staged'; + readonly handle: AssetHandle; + readonly asset: Readonly<{ slug: string; filename: string; size: number }>; + readonly retention: Readonly<{ + policy: null; + reachability: 'unanchored'; + protection: 'not-established'; + }>; + readonly observedAt: string; + constructor(value: { + handle: AssetHandleInput; + slug: string; + filename: string; + size: number; + observedAt: string; + }); + toJSON(): StagedAssetData; +} + +export type RetentionPolicy = 'pinned' | 'evictable'; +export type RetentionReachability = 'anchored' | 'orphaned' | 'volatile'; +export type RetentionRootKind = 'root-set' | 'publication' | 'cache-set' | 'expiring-set'; + +export interface RetentionRoot { + kind: RetentionRootKind; + namespace: string; + ref: string; + generation: string; + path: string; +} + +export interface RetentionWitnessData { + version: 1; + handle: string; + policy: RetentionPolicy; + reachability: RetentionReachability; + root: RetentionRoot; + observedAt: string; +} + +/** Immutable evidence for one observed retaining Git generation. */ +export declare class RetentionWitness { + readonly version: 1; + readonly handle: AssetHandle; + readonly policy: RetentionPolicy; + readonly reachability: RetentionReachability; + readonly root: Readonly; + readonly observedAt: string; + constructor(value: { + handle: AssetHandleInput; + policy: RetentionPolicy; + reachability: RetentionReachability; + root: RetentionRoot; + observedAt: string; + }); + toJSON(): RetentionWitnessData; +} export type { CryptoPort, CodecPort, @@ -162,6 +271,7 @@ export declare class GitRefPortBase { createCommit(options: { treeOid: string; parentOid?: string | null; + parentOids?: string[]; message: string; }): Promise; updateRef(options: { @@ -264,6 +374,10 @@ export interface ContentAddressableStoreOptions { maxRestoreBufferSize?: number; /** Safety limit for readBlob metadata in bytes. @default 10485760 (10 MiB) */ maxBlobSize?: number; + /** Application-owned ref prefixes permitted for generic publication. */ + applicationRefPrefixes?: string[]; + /** Injectable clock used for deterministic lifecycle evidence. */ + clock?: { now(): Date }; } /** Options for {@link ContentAddressableStore.open}. */ @@ -290,7 +404,7 @@ export interface VaultEntry { } export type RootSetEntryType = 'blob' | 'tree'; -export type RootSetRetention = 'pinned' | 'evictable'; +export type RootSetRetention = RetentionPolicy; /** One Git object retained by a root set while the entry is present. */ export interface RootSetEntry { @@ -525,6 +639,59 @@ export const SCHEME_FRAMED: 'framed'; /** Encryption scheme constant for convergent (dedup-preserving) encryption. */ export const SCHEME_CONVERGENT: 'convergent'; +export interface AssetPutOptions { + source: AsyncIterable; + slug: string; + filename?: string; + encryptionKey?: Uint8Array; + passphrase?: string; + encryption?: StoreEncryptionOptions; + kdfOptions?: Omit; + compression?: { algorithm: 'gzip' }; + recipients?: Array<{ label: string; key: Uint8Array }>; + merkleThreshold?: number; + chunking?: ChunkingConfig; +} + +export interface AssetCapability { + put(options: AssetPutOptions): Promise; + adopt(options: { treeOid: string }): Promise; + open(options: { + handle: AssetHandleInput; + encryptionKey?: Uint8Array; + passphrase?: string; + }): AsyncIterable; +} + +export interface RetentionResult { + readonly changed: boolean; + readonly witness: RetentionWitness; +} + +export interface RetentionCapability { + retain(options: { + handle: AssetHandleInput; + root: { ref: string; name: string }; + policy?: RetentionPolicy; + }): Promise; +} + +export interface PublicationResult { + readonly operation: 'publication'; + readonly commitId: string; + readonly ref: string; + readonly root: AssetHandle; + readonly witness: RetentionWitness; +} + +export interface PublicationCapability { + commit(options: { + root: AssetHandleInput; + commit: { message: string; parents?: string[] }; + ref: { name: string; expected: string | null }; + }): Promise; +} + /** * High-level facade for the Content Addressable Store library. * @@ -543,6 +710,10 @@ export default class ContentAddressableStore { }): Promise; }; + readonly assets: AssetCapability; + readonly retention: RetentionCapability; + readonly publications: PublicationCapability; + getService(): Promise; getVaultService(): Promise; getRootSetRegistry(): Promise; diff --git a/index.js b/index.js index 3005e0ce..9f10dc17 100644 --- a/index.js +++ b/index.js @@ -10,6 +10,9 @@ import CasService from './src/domain/services/CasService.js'; import VaultService from './src/domain/services/VaultService.js'; import RootSet from './src/domain/services/RootSet.js'; import RootSetRegistry from './src/domain/services/RootSetRegistry.js'; +import AssetService from './src/domain/services/AssetService.js'; +import PublicationService from './src/domain/services/PublicationService.js'; +import RetentionService from './src/domain/services/RetentionService.js'; import rotateVaultPassphrase from './src/domain/services/rotateVaultPassphrase.js'; import GitPersistenceAdapter from './src/infrastructure/adapters/GitPersistenceAdapter.js'; import GitRefAdapter from './src/infrastructure/adapters/GitRefAdapter.js'; @@ -71,6 +74,9 @@ export { default as ChunkingPort } from './src/ports/ChunkingPort.js'; export { default as ObservabilityPort } from './src/ports/ObservabilityPort.js'; export { default as Manifest } from './src/domain/value-objects/Manifest.js'; export { default as Chunk } from './src/domain/value-objects/Chunk.js'; +export { default as AssetHandle } from './src/domain/value-objects/AssetHandle.js'; +export { default as StagedAsset } from './src/domain/value-objects/StagedAsset.js'; +export { default as RetentionWitness } from './src/domain/value-objects/RetentionWitness.js'; export { default as EventEmitterObserver } from './src/infrastructure/adapters/EventEmitterObserver.js'; export { default as StatsCollector } from './src/infrastructure/adapters/StatsCollector.js'; export { default as FixedChunker } from './src/infrastructure/chunkers/FixedChunker.js'; @@ -102,6 +108,8 @@ export default class ContentAddressableStore { * @param {number} [options.maxRestoreBufferSize=536870912] - Max buffered restore size in bytes for encrypted/compressed restores (default 512 MiB). * @param {number} [options.maxBlobSize=10485760] - Safety limit for readBlob metadata in bytes (default 10 MiB). * @param {import('./src/ports/CompressionPort.js').default} [options.compressionAdapter] - Compression adapter (default NodeCompressionAdapter). + * @param {string[]} [options.applicationRefPrefixes] - Explicit application ref namespaces allowed for generic publication. + * @param {{ now(): Date }} [options.clock] - Injectable clock for deterministic evidence. */ constructor({ plumbing, @@ -117,6 +125,8 @@ export default class ContentAddressableStore { maxRestoreBufferSize, maxBlobSize, compressionAdapter, + applicationRefPrefixes, + clock, }) { this.#config = { plumbing, @@ -132,16 +142,39 @@ export default class ContentAddressableStore { maxRestoreBufferSize, maxBlobSize, compressionAdapter, + applicationRefPrefixes, + clock, }; this.service = null; this.#servicePromise = null; + this.#installCapabilities(); + } + + #installCapabilities() { this.rootSets = Object.freeze({ open: async (options) => (await this.#getRootSetRegistry()).open(options), }); + this.assets = Object.freeze({ + put: async (options) => (await this.#getAssetService()).put(options), + adopt: async (options) => (await this.#getAssetService()).adopt(options), + open: (options) => this.#openAsset(options), + }); + this.retention = Object.freeze({ + retain: async (options) => (await this.#getRetentionService()).retain(options), + }); + this.publications = Object.freeze({ + commit: async (options) => (await this.#getPublicationService()).commit(options), + }); } - /** @type {{ plumbing: *, chunkSize?: number, codec?: *, policy?: *, crypto?: *, observability?: *, merkleThreshold?: number, concurrency?: number, chunking?: *, chunker?: *, maxRestoreBufferSize?: number, maxBlobSize?: number, compressionAdapter?: * }} */ + /** @type {{ plumbing: *, chunkSize?: number, codec?: *, policy?: *, crypto?: *, observability?: *, merkleThreshold?: number, concurrency?: number, chunking?: *, chunker?: *, maxRestoreBufferSize?: number, maxBlobSize?: number, compressionAdapter?: *, applicationRefPrefixes?: string[], clock?: { now(): Date } }} */ #config; + /** @type {AssetService|null} */ + #assetService = null; + /** @type {PublicationService|null} */ + #publicationService = null; + /** @type {RetentionService|null} */ + #retentionService = null; /** @type {VaultService|null} */ #vault = null; /** @type {RootSetRegistry|null} */ @@ -202,10 +235,27 @@ export default class ContentAddressableStore { observability: this.service.observability, }); this.#rootSetRegistry = new RootSetRegistry({ persistence, ref }); + this.#initApplicationServices({ ref, cfg }); return this.service; } + #initApplicationServices({ ref, cfg }) { + this.#assetService = new AssetService({ cas: this.service, clock: cfg.clock }); + const resolveRoot = (handle) => this.#assetService.resolveRoot(handle); + this.#retentionService = new RetentionService({ + rootSets: this.#rootSetRegistry, + resolveRoot, + clock: cfg.clock, + }); + this.#publicationService = new PublicationService({ + ref, + resolveRoot, + applicationRefPrefixes: cfg.applicationRefPrefixes, + clock: cfg.clock, + }); + } + /** * Lazily initializes and returns the underlying {@link VaultService}. * @private @@ -226,6 +276,30 @@ export default class ContentAddressableStore { return this.#rootSetRegistry; } + /** @returns {Promise} */ + async #getAssetService() { + await this.#getService(); + return this.#assetService; + } + + /** @returns {Promise} */ + async #getRetentionService() { + await this.#getService(); + return this.#retentionService; + } + + /** @returns {Promise} */ + async #getPublicationService() { + await this.#getService(); + return this.#publicationService; + } + + /** @returns {AsyncIterable} */ + async *#openAsset(options) { + const assets = await this.#getAssetService(); + yield* assets.open(options); + } + /** * Lazily initializes and returns the underlying {@link CasService}. * @returns {Promise} diff --git a/src/domain/errors/Codes.js b/src/domain/errors/Codes.js index 84d2463b..ab34a23b 100644 --- a/src/domain/errors/Codes.js +++ b/src/domain/errors/Codes.js @@ -7,6 +7,11 @@ const ErrorCodes = Object.freeze({ GIT_OBJECT_NOT_FOUND: 'GIT_OBJECT_NOT_FOUND', GIT_PLUMBING_INITIALIZATION_FAILED: 'GIT_PLUMBING_INITIALIZATION_FAILED', GIT_REF_NOT_FOUND: 'GIT_REF_NOT_FOUND', + HANDLE_CODEC_MISMATCH: 'HANDLE_CODEC_MISMATCH', + HANDLE_INVALID: 'HANDLE_INVALID', + HANDLE_KIND_MISMATCH: 'HANDLE_KIND_MISMATCH', + HANDLE_TARGET_MISSING: 'HANDLE_TARGET_MISSING', + HANDLE_TARGET_TYPE_MISMATCH: 'HANDLE_TARGET_TYPE_MISMATCH', INTEGRITY_ERROR: 'INTEGRITY_ERROR', INVALID_CHUNKING_STRATEGY: 'INVALID_CHUNKING_STRATEGY', INVALID_ENCRYPTION_SCHEME: 'INVALID_ENCRYPTION_SCHEME', @@ -25,9 +30,14 @@ const ErrorCodes = Object.freeze({ NO_MATCHING_RECIPIENT: 'NO_MATCHING_RECIPIENT', PERSISTENCE_CAPABILITY_REQUIRED: 'PERSISTENCE_CAPABILITY_REQUIRED', PORT_NOT_IMPLEMENTED: 'PORT_NOT_IMPLEMENTED', + PUBLICATION_CONFLICT: 'PUBLICATION_CONFLICT', + PUBLICATION_INVALID: 'PUBLICATION_INVALID', + PUBLICATION_REF_FORBIDDEN: 'PUBLICATION_REF_FORBIDDEN', + PUBLICATION_REF_UPDATE_FAILED: 'PUBLICATION_REF_UPDATE_FAILED', RECIPIENT_ALREADY_EXISTS: 'RECIPIENT_ALREADY_EXISTS', RECIPIENT_NOT_FOUND: 'RECIPIENT_NOT_FOUND', RESTORE_TOO_LARGE: 'RESTORE_TOO_LARGE', + RETENTION_WITNESS_INVALID: 'RETENTION_WITNESS_INVALID', ROOT_SET_CONFLICT: 'ROOT_SET_CONFLICT', ROOT_SET_DEPENDENCY_INVALID: 'ROOT_SET_DEPENDENCY_INVALID', ROOT_SET_ENTRY_INVALID: 'ROOT_SET_ENTRY_INVALID', diff --git a/src/domain/helpers/assertCanonicalTimestamp.js b/src/domain/helpers/assertCanonicalTimestamp.js new file mode 100644 index 00000000..a7afdc52 --- /dev/null +++ b/src/domain/helpers/assertCanonicalTimestamp.js @@ -0,0 +1,15 @@ +/** + * @param {unknown} value + * @param {object} options + * @param {(message: string, meta: object) => Error} options.invalid + * @param {string} options.message + */ +export default function assertCanonicalTimestamp(value, { invalid, message }) { + if ( + typeof value !== 'string' || + Number.isNaN(Date.parse(value)) || + new Date(value).toISOString() !== value + ) { + throw invalid(message, { observedAt: value }); + } +} diff --git a/src/domain/services/AssetService.js b/src/domain/services/AssetService.js new file mode 100644 index 00000000..55a2bfe5 --- /dev/null +++ b/src/domain/services/AssetService.js @@ -0,0 +1,213 @@ +import createCasError from '../errors/createCasError.js'; +import { ErrorCodes } from '../errors/index.js'; +import AssetHandle from '../value-objects/AssetHandle.js'; +import StagedAsset from '../value-objects/StagedAsset.js'; + +const DEFAULT_CLOCK = Object.freeze({ now: () => new Date() }); + +/** + * High-level asset handle boundary over the existing streaming CAS pipeline. + */ +export default class AssetService { + #cas; + #clock; + + /** + * @param {object} options + * @param {import('./CasService.js').default} options.cas + * @param {{ now(): Date }} [options.clock] + */ + constructor({ cas, clock = DEFAULT_CLOCK }) { + AssetService.#assertDependencies(cas, clock); + this.#cas = cas; + this.#clock = clock; + } + + /** + * Streams one asset into CAS and returns an explicitly unanchored handle. + * + * @param {object} options + * @returns {Promise} + */ + async put(options) { + const filename = options?.filename ?? options?.slug; + const manifest = await this.#cas.store({ ...options, filename }); + const oid = await this.#cas.createTree({ + manifest, + merkleThreshold: options?.merkleThreshold, + }); + return this.#staged({ manifest, oid }); + } + + /** + * Adopts a validated existing git-cas manifest tree into an asset handle. + * + * @param {{ treeOid: string }} options + * @returns {Promise} + */ + async adopt({ treeOid }) { + const handle = new AssetHandle({ codec: this.#cas.codec.extension, oid: treeOid }); + const root = await this.resolveRoot(handle); + return this.#staged({ manifest: root.manifest, oid: root.oid }); + } + + /** + * Streams plaintext bytes from a validated asset handle. + * + * @param {object} options + * @param {AssetHandle|string|object} options.handle + * @param {Uint8Array} [options.encryptionKey] + * @param {string} [options.passphrase] + * @returns {AsyncIterable} + */ + async *open({ handle: value, encryptionKey, passphrase }) { + const root = await this.resolveRoot(value); + try { + yield* this.#cas.restoreStream({ + manifest: root.manifest, + encryptionKey, + passphrase, + }); + } catch (error) { + throw this.#mapTargetError(error, root.handle); + } + } + + /** + * Validates the handle and its complete manifest/chunk object graph. + * + * @param {AssetHandle|string|object} value + * @returns {Promise} + */ + async resolveRoot(value) { + const handle = AssetHandle.from(value); + this.#assertCodec(handle); + await this.#assertObjectType(handle, handle.oid, 'tree'); + + let manifest; + try { + manifest = await this.#cas.readManifest({ treeOid: handle.oid }); + } catch (error) { + throw this.#mapTargetError(error, handle); + } + await this.#assertChunkGraph(handle, manifest); + return Object.freeze({ + handle, + oid: handle.oid, + type: 'tree', + size: manifest.size, + manifest, + }); + } + + async #assertChunkGraph(handle, manifest) { + const blobs = new Set(); + for (const chunk of manifest.chunks) { + blobs.add(chunk.blob); + } + const iterator = blobs.values(); + const workerCount = Math.min(this.#cas.concurrency, blobs.size); + + const validateNext = async () => { + while (true) { + const next = iterator.next(); + if (next.done) { + return; + } + await this.#assertObjectType(handle, next.value, 'blob'); + } + }; + + await Promise.all(Array.from({ length: workerCount }, () => validateNext())); + } + + async #assertObjectType(handle, oid, expectedType) { + let actualType; + try { + actualType = await this.#cas.persistence.readObjectType(oid); + } catch (error) { + throw this.#mapTargetError(error, handle, oid); + } + if (actualType !== expectedType) { + throw createCasError( + 'Asset handle target has the wrong Git object type', + ErrorCodes.HANDLE_TARGET_TYPE_MISMATCH, + { handle: handle.toString(), targetOid: oid, expectedType, actualType } + ); + } + } + + #assertCodec(handle) { + const expectedCodec = this.#cas.codec.extension; + if (handle.codec !== expectedCodec) { + throw createCasError( + 'Asset handle codec does not match this CAS instance', + ErrorCodes.HANDLE_CODEC_MISMATCH, + { + handle: handle.toString(), + expectedCodec, + actualCodec: handle.codec, + } + ); + } + } + + #mapTargetError(error, handle, targetOid = handle.oid) { + if (error?.code === ErrorCodes.HANDLE_TARGET_MISSING) { + return error; + } + const missing = + error?.code === ErrorCodes.GIT_OBJECT_NOT_FOUND || + /(?:object|blob|tree) not found/iu.test( + error instanceof Error ? error.message : String(error) + ); + if (!missing) { + return error; + } + return createCasError( + 'Asset handle target graph is missing from this repository', + ErrorCodes.HANDLE_TARGET_MISSING, + { handle: handle.toString(), targetOid, originalError: error } + ); + } + + #staged({ manifest, oid }) { + return new StagedAsset({ + handle: new AssetHandle({ codec: this.#cas.codec.extension, oid }), + slug: manifest.slug, + filename: manifest.filename, + size: manifest.size, + observedAt: this.#observedAt(), + }); + } + + #observedAt() { + const now = this.#clock.now(); + if (!(now instanceof Date) || Number.isNaN(now.getTime())) { + throw createCasError( + 'AssetService clock returned an invalid Date', + ErrorCodes.INVALID_OPTIONS + ); + } + return now.toISOString(); + } + + static #assertDependencies(cas, clock) { + const methods = ['store', 'createTree', 'readManifest', 'restoreStream']; + if (!cas || methods.some((method) => typeof cas[method] !== 'function')) { + throw createCasError( + 'AssetService requires a complete CasService', + ErrorCodes.INVALID_OPTIONS + ); + } + if (!cas.persistence || typeof cas.persistence.readObjectType !== 'function') { + throw createCasError( + 'AssetService requires object-type inspection', + ErrorCodes.INVALID_OPTIONS + ); + } + if (!clock || typeof clock.now !== 'function') { + throw createCasError('AssetService clock must provide now()', ErrorCodes.INVALID_OPTIONS); + } + } +} diff --git a/src/domain/services/PublicationService.js b/src/domain/services/PublicationService.js new file mode 100644 index 00000000..ef07a3f8 --- /dev/null +++ b/src/domain/services/PublicationService.js @@ -0,0 +1,315 @@ +import createCasError from '../errors/createCasError.js'; +import { ErrorCodes } from '../errors/index.js'; +import { utf8Encode } from '../encoding/utf8.js'; +import Oid from '../value-objects/Oid.js'; +import RetentionWitness from '../value-objects/RetentionWitness.js'; + +const DEFAULT_CLOCK = Object.freeze({ now: () => new Date() }); +const FORBIDDEN_REF_CHARS = '~^:?*[\\'; +const MAX_COMMIT_MESSAGE_BYTES = 1024 * 1024; +const MAX_PARENTS = 64; +const RESERVED_REF_NAMESPACES = Object.freeze([ + 'refs/bisect', + 'refs/cas', + 'refs/heads', + 'refs/notes', + 'refs/remotes', + 'refs/replace', + 'refs/rewritten', + 'refs/stash', + 'refs/tags', + 'refs/worktree', +]); + +/** + * Publishes validated content roots through application-owned Git refs. + */ +export default class PublicationService { + #applicationRefPrefixes; + #clock; + #ref; + #resolveRoot; + + /** + * @param {object} options + * @param {import('../../ports/GitRefPort.js').default} options.ref + * @param {(handle: unknown) => Promise} options.resolveRoot + * @param {string[]} [options.applicationRefPrefixes] + * @param {{ now(): Date }} [options.clock] + */ + constructor({ ref, resolveRoot, applicationRefPrefixes = [], clock = DEFAULT_CLOCK }) { + PublicationService.#assertDependencies(ref, resolveRoot, clock); + this.#applicationRefPrefixes = PublicationService.#normalizePrefixes(applicationRefPrefixes); + this.#ref = ref; + this.#resolveRoot = resolveRoot; + this.#clock = clock; + } + + /** + * @param {object} options + * @param {unknown} options.root + * @param {{ message: string, parents?: string[] }} options.commit + * @param {{ name: string, expected: string|null }} options.ref + * @returns {Promise} + */ + async commit({ root: handle, commit, ref }) { + const namespace = this.#authorizeRef(ref?.name); + const expected = PublicationService.#normalizeExpected(ref); + const message = PublicationService.#normalizeMessage(commit?.message); + const parentOids = await this.#normalizeParents(commit?.parents ?? []); + const target = await this.#resolveRoot(handle); + PublicationService.#assertTreeRoot(target); + + const commitId = await this.#ref.createCommit({ + treeOid: target.oid, + parentOids, + message, + }); + await this.#updateRef({ ref: ref.name, commitId, expected }); + + const witness = new RetentionWitness({ + handle: target.handle, + policy: 'pinned', + reachability: 'anchored', + root: { + kind: 'publication', + namespace, + ref: ref.name, + generation: commitId, + path: '/', + }, + observedAt: this.#observedAt(), + }); + return Object.freeze({ + operation: 'publication', + commitId, + ref: ref.name, + root: target.handle, + witness, + }); + } + + async #normalizeParents(values) { + if (!Array.isArray(values) || values.length > MAX_PARENTS) { + throw PublicationService.#invalid('Publication parents must be a bounded array', { + parentCount: values?.length, + maxParents: MAX_PARENTS, + }); + } + const parents = values.map((value) => PublicationService.#normalizeOid(value, 'parent')); + for (const parent of parents) { + try { + await this.#ref.resolveParents(parent); + } catch (error) { + throw PublicationService.#invalid('Publication parent is not a readable commit', { + parent, + originalError: error, + }); + } + } + return parents; + } + + async #updateRef({ ref, commitId, expected }) { + try { + await this.#ref.updateRef({ + ref, + newOid: commitId, + expectedOldOid: expected, + }); + } catch (error) { + let observed; + try { + observed = await this.#resolveObserved(ref); + } catch (observationError) { + throw createCasError( + 'Application publication ref update failed and the current head could not be observed', + ErrorCodes.PUBLICATION_REF_UPDATE_FAILED, + { ref, expected, attemptedCommitId: commitId, originalError: error, observationError } + ); + } + if (observed !== expected) { + throw createCasError('Application publication conflict', ErrorCodes.PUBLICATION_CONFLICT, { + ref, + expected, + observed, + attemptedCommitId: commitId, + originalError: error, + }); + } + throw createCasError( + 'Application publication ref update failed', + ErrorCodes.PUBLICATION_REF_UPDATE_FAILED, + { ref, expected, observed, attemptedCommitId: commitId, originalError: error } + ); + } + } + + async #resolveObserved(ref) { + try { + return await this.#ref.resolveRef(ref); + } catch (error) { + if (error?.code === ErrorCodes.GIT_REF_NOT_FOUND) { + return null; + } + throw error; + } + } + + #authorizeRef(value) { + if (!PublicationService.#isValidRef(value) || PublicationService.#isReservedRef(value)) { + throw PublicationService.#forbidden(value); + } + const namespace = this.#applicationRefPrefixes.find((prefix) => value.startsWith(prefix)); + if (!namespace) { + throw PublicationService.#forbidden(value); + } + return namespace; + } + + #observedAt() { + const now = this.#clock.now(); + if (!(now instanceof Date) || Number.isNaN(now.getTime())) { + throw PublicationService.#invalid('PublicationService clock returned an invalid Date'); + } + return now.toISOString(); + } + + static #normalizeExpected(ref) { + if (!ref || !Object.hasOwn(ref, 'expected')) { + throw PublicationService.#invalid('Publication ref requires an explicit expected head', { + ref, + }); + } + return ref.expected === null + ? null + : PublicationService.#normalizeOid(ref.expected, 'expected'); + } + + static #normalizeMessage(value) { + const size = typeof value === 'string' ? utf8Encode(value).length : -1; + if (size < 1 || size > MAX_COMMIT_MESSAGE_BYTES || value.includes('\0')) { + throw PublicationService.#invalid('Publication commit message is invalid', { + messageBytes: size, + maxMessageBytes: MAX_COMMIT_MESSAGE_BYTES, + }); + } + return value; + } + + static #normalizePrefixes(prefixes) { + if (!Array.isArray(prefixes)) { + throw PublicationService.#invalid('Application ref prefixes must be an array', { prefixes }); + } + const normalized = [...new Set(prefixes)]; + for (const prefix of normalized) { + if ( + !PublicationService.#isValidRef(`${prefix}placeholder`) || + !prefix.endsWith('/') || + PublicationService.#isReservedRef(prefix.slice(0, -1)) + ) { + throw PublicationService.#invalid('Application ref prefix is invalid or reserved', { + prefix, + }); + } + } + normalized.sort((left, right) => { + const lengthOrder = right.length - left.length; + if (lengthOrder !== 0) { + return lengthOrder; + } + return left < right ? -1 : left > right ? 1 : 0; + }); + return Object.freeze(normalized); + } + + static #isValidRef(value) { + if ( + PublicationService.#hasInvalidRefShape(value) || + PublicationService.#hasForbiddenRefCharacter(value) + ) { + return false; + } + return value + .split('/') + .every((part) => part.length > 0 && part !== '.' && part !== '..' && !part.endsWith('.lock')); + } + + static #isReservedRef(value) { + return RESERVED_REF_NAMESPACES.some( + (namespace) => value === namespace || value.startsWith(`${namespace}/`) + ); + } + + static #hasInvalidRefShape(value) { + return ( + typeof value !== 'string' || + !value.startsWith('refs/') || + value.endsWith('/') || + value.endsWith('.') || + value.includes('//') || + value.includes('..') || + value.includes('@{') + ); + } + + static #hasForbiddenRefCharacter(value) { + if (typeof value !== 'string') { + return true; + } + for (const character of value) { + const codePoint = character.codePointAt(0); + if (codePoint <= 0x20 || codePoint === 0x7f || FORBIDDEN_REF_CHARS.includes(character)) { + return true; + } + } + return false; + } + + static #normalizeOid(value, field) { + try { + return Oid.from(value).toString(); + } catch (error) { + throw PublicationService.#invalid(`Publication ${field} is not a valid object identifier`, { + field, + value, + originalError: error, + }); + } + } + + static #assertTreeRoot(target) { + if (!target || target.type !== 'tree') { + throw createCasError( + 'Publication root must resolve to a Git tree', + ErrorCodes.HANDLE_TARGET_TYPE_MISMATCH, + { expectedType: 'tree', actualType: target?.type ?? null } + ); + } + } + + static #assertDependencies(ref, resolveRoot, clock) { + const methods = ['resolveRef', 'resolveParents', 'createCommit', 'updateRef']; + if (!ref || methods.some((method) => typeof ref[method] !== 'function')) { + throw PublicationService.#invalid('PublicationService requires a complete Git ref port'); + } + if (typeof resolveRoot !== 'function') { + throw PublicationService.#invalid('PublicationService requires a handle resolver'); + } + if (!clock || typeof clock.now !== 'function') { + throw PublicationService.#invalid('PublicationService clock must provide now()'); + } + } + + static #forbidden(ref) { + return createCasError( + 'Application publication ref is invalid, reserved, or not configured', + ErrorCodes.PUBLICATION_REF_FORBIDDEN, + { ref } + ); + } + + static #invalid(message, meta = {}) { + return createCasError(message, ErrorCodes.PUBLICATION_INVALID, meta); + } +} diff --git a/src/domain/services/RetentionService.js b/src/domain/services/RetentionService.js new file mode 100644 index 00000000..fd9404a4 --- /dev/null +++ b/src/domain/services/RetentionService.js @@ -0,0 +1,120 @@ +import createCasError from '../errors/createCasError.js'; +import { ErrorCodes } from '../errors/index.js'; +import RootSetMetadataCodec from './RootSetMetadataCodec.js'; +import RootSetRef, { ROOT_SET_REF_PREFIX } from '../value-objects/RootSetRef.js'; +import RetentionWitness from '../value-objects/RetentionWitness.js'; + +const DEFAULT_CLOCK = Object.freeze({ now: () => new Date() }); + +/** + * Converts a validated content handle into a RootSet reachability edge. + */ +export default class RetentionService { + #clock; + #resolveRoot; + #rootSets; + + /** + * @param {object} options + * @param {{ open(options: object): import('./RootSet.js').default }} options.rootSets + * @param {(handle: unknown) => Promise} options.resolveRoot + * @param {{ now(): Date }} [options.clock] + */ + constructor({ rootSets, resolveRoot, clock = DEFAULT_CLOCK }) { + RetentionService.#assertDependencies(rootSets, resolveRoot, clock); + this.#rootSets = rootSets; + this.#resolveRoot = resolveRoot; + this.#clock = clock; + } + + /** + * @param {object} options + * @param {unknown} options.handle + * @param {{ ref: string, name: string }} options.root + * @param {'pinned'|'evictable'} [options.policy] + * @returns {Promise<{ changed: boolean, witness: RetentionWitness }>} + */ + async retain({ handle, root, policy = 'pinned' }) { + RetentionService.#assertRoot(root); + const target = await this.#resolveRoot(handle); + const rootSet = this.#rootSets.open({ ref: root.ref }); + const result = await rootSet.put({ + name: root.name, + oid: target.oid, + type: target.type, + retention: policy, + }); + const path = RetentionService.#evidencePath(result.entries, root.name); + const witness = new RetentionWitness({ + handle: target.handle, + policy, + reachability: 'anchored', + root: { + kind: 'root-set', + namespace: root.ref.slice(ROOT_SET_REF_PREFIX.length), + ref: root.ref, + generation: result.commitOid, + path, + }, + observedAt: this.#observedAt(), + }); + return Object.freeze({ changed: result.changed, witness }); + } + + #observedAt() { + const now = this.#clock.now(); + if (!(now instanceof Date) || Number.isNaN(now.getTime())) { + throw createCasError( + 'RetentionService clock returned an invalid Date', + ErrorCodes.INVALID_OPTIONS + ); + } + return now.toISOString(); + } + + static #evidencePath(entries, name) { + const index = entries.findIndex((entry) => entry.name === name); + if (index === -1) { + throw createCasError( + 'Retained entry is absent from the committed root generation', + ErrorCodes.ROOT_SET_TREE_INVALID, + { name } + ); + } + return RootSetMetadataCodec.slotFor(index); + } + + static #assertRoot(root) { + if (!root || typeof root !== 'object' || Array.isArray(root)) { + throw createCasError('Retention root must be an object', ErrorCodes.INVALID_OPTIONS, { + root, + }); + } + if (typeof root.ref !== 'string' || typeof root.name !== 'string' || root.name.length === 0) { + throw createCasError( + 'Retention root requires ref and name strings', + ErrorCodes.INVALID_OPTIONS, + { root } + ); + } + RootSetRef.from(root.ref); + } + + static #assertDependencies(rootSets, resolveRoot, clock) { + if (!rootSets || typeof rootSets.open !== 'function') { + throw createCasError( + 'RetentionService requires a RootSet registry', + ErrorCodes.INVALID_OPTIONS + ); + } + if (typeof resolveRoot !== 'function') { + throw createCasError( + 'RetentionService requires a handle resolver', + ErrorCodes.INVALID_OPTIONS + ); + } + if (!clock || typeof clock.now !== 'function') { + throw createCasError('RetentionService clock must provide now()', ErrorCodes.INVALID_OPTIONS); + } + } +} diff --git a/src/domain/value-objects/AssetHandle.js b/src/domain/value-objects/AssetHandle.js new file mode 100644 index 00000000..221b2f90 --- /dev/null +++ b/src/domain/value-objects/AssetHandle.js @@ -0,0 +1,163 @@ +import createCasError from '../errors/createCasError.js'; +import { ErrorCodes } from '../errors/index.js'; +import Oid from './Oid.js'; + +export const ASSET_HANDLE_VERSION = 1; +export const ASSET_HANDLE_KIND = 'asset'; +export const ASSET_HANDLE_FORMAT = 'manifest-tree'; + +const TOKEN_PREFIX = 'git-cas'; +const CODEC_PATTERN = /^[a-z0-9][a-z0-9.-]{0,31}$/u; + +/** + * Immutable, repository-independent locator for a git-cas asset graph. + */ +export default class AssetHandle { + /** + * @param {object} value + * @param {number} [value.version] + * @param {string} [value.kind] + * @param {string} [value.format] + * @param {string} value.codec + * @param {string} [value.hashAlgorithm] + * @param {string} value.oid + */ + constructor(value) { + AssetHandle.#assertObject(value); + AssetHandle.#assertEnvelope(value); + + let oid; + try { + oid = Oid.from(value.oid).toString(); + } catch (error) { + throw AssetHandle.#invalid('Asset handle contains an invalid Git object identifier', { + value, + originalError: error, + }); + } + const hashAlgorithm = oid.length === 40 ? 'sha1' : 'sha256'; + if (value.hashAlgorithm !== undefined && value.hashAlgorithm !== hashAlgorithm) { + throw AssetHandle.#invalid( + 'Asset handle hash algorithm does not match its object identifier', + { + hashAlgorithm: value.hashAlgorithm, + oid, + } + ); + } + + this.version = ASSET_HANDLE_VERSION; + this.kind = ASSET_HANDLE_KIND; + this.format = ASSET_HANDLE_FORMAT; + this.codec = value.codec; + this.hashAlgorithm = hashAlgorithm; + this.oid = oid; + Object.freeze(this); + } + + /** + * @param {AssetHandle|string|object} value + * @returns {AssetHandle} + */ + static from(value) { + if (value instanceof AssetHandle) { + return value; + } + if (typeof value === 'string') { + return AssetHandle.parse(value); + } + return new AssetHandle(value); + } + + /** + * @param {string} token + * @returns {AssetHandle} + */ + static parse(token) { + if (typeof token !== 'string') { + throw AssetHandle.#invalid('Asset handle token must be a string', { token }); + } + const fields = token.split(':'); + if (fields.length !== 7 || fields[0] !== TOKEN_PREFIX) { + throw AssetHandle.#invalid('Asset handle token has an invalid shape', { token }); + } + const [, version, kind, format, codec, hashAlgorithm, oid] = fields; + if (kind !== ASSET_HANDLE_KIND) { + throw createCasError('Handle kind is not asset', ErrorCodes.HANDLE_KIND_MISMATCH, { + expectedKind: ASSET_HANDLE_KIND, + actualKind: kind, + }); + } + const handle = new AssetHandle({ + version: Number(version), + kind, + format, + codec, + hashAlgorithm, + oid, + }); + if (handle.toString() !== token) { + throw AssetHandle.#invalid('Asset handle token is not canonical', { token }); + } + return handle; + } + + /** + * @returns {string} + */ + toString() { + return [ + TOKEN_PREFIX, + this.version, + this.kind, + this.format, + this.codec, + this.hashAlgorithm, + this.oid, + ].join(':'); + } + + /** + * @returns {object} + */ + toJSON() { + return { + version: this.version, + kind: this.kind, + format: this.format, + codec: this.codec, + hashAlgorithm: this.hashAlgorithm, + oid: this.oid, + }; + } + + static #assertObject(value) { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + throw AssetHandle.#invalid('Asset handle must be an object or canonical token', { value }); + } + } + + static #assertEnvelope(value) { + if (value.version !== undefined && value.version !== ASSET_HANDLE_VERSION) { + throw AssetHandle.#invalid(`Unsupported asset handle version: ${value.version}`, { value }); + } + if (value.kind !== undefined && value.kind !== ASSET_HANDLE_KIND) { + throw createCasError('Handle kind is not asset', ErrorCodes.HANDLE_KIND_MISMATCH, { + expectedKind: ASSET_HANDLE_KIND, + actualKind: value.kind, + }); + } + if (value.format !== undefined && value.format !== ASSET_HANDLE_FORMAT) { + throw AssetHandle.#invalid(`Unsupported asset handle format: ${value.format}`, { value }); + } + if (typeof value.codec !== 'string' || !CODEC_PATTERN.test(value.codec)) { + throw AssetHandle.#invalid('Asset handle codec must be a canonical lowercase identifier', { + codec: value.codec, + }); + } + } + + static #invalid(message, meta) { + return createCasError(message, ErrorCodes.HANDLE_INVALID, meta); + } +} diff --git a/src/domain/value-objects/RetentionWitness.js b/src/domain/value-objects/RetentionWitness.js new file mode 100644 index 00000000..3e776c51 --- /dev/null +++ b/src/domain/value-objects/RetentionWitness.js @@ -0,0 +1,101 @@ +import createCasError from '../errors/createCasError.js'; +import { ErrorCodes } from '../errors/index.js'; +import assertCanonicalTimestamp from '../helpers/assertCanonicalTimestamp.js'; +import AssetHandle from './AssetHandle.js'; +import Oid from './Oid.js'; + +const POLICIES = Object.freeze(['pinned', 'evictable']); +const REACHABILITY = Object.freeze(['anchored', 'orphaned', 'volatile']); +const ROOT_KINDS = Object.freeze(['root-set', 'publication', 'cache-set', 'expiring-set']); + +/** + * Immutable evidence that one observed Git generation retained a handle. + */ +export default class RetentionWitness { + /** + * @param {object} value + * @param {AssetHandle|string|object} value.handle + * @param {'pinned'|'evictable'} value.policy + * @param {'anchored'|'orphaned'|'volatile'} value.reachability + * @param {object} value.root + * @param {string} value.observedAt + */ + constructor({ handle, policy, reachability, root, observedAt }) { + if (!POLICIES.includes(policy)) { + throw RetentionWitness.#invalid('Retention witness policy must be pinned or evictable', { + policy, + }); + } + if (!REACHABILITY.includes(reachability)) { + throw RetentionWitness.#invalid('Retention witness has an invalid reachability state', { + reachability, + }); + } + const normalizedRoot = RetentionWitness.#normalizeRoot(root); + assertCanonicalTimestamp(observedAt, { + invalid: RetentionWitness.#invalid, + message: 'Retention witness observation time must be a canonical UTC timestamp', + }); + + this.version = 1; + this.handle = AssetHandle.from(handle); + this.policy = policy; + this.reachability = reachability; + this.root = Object.freeze(normalizedRoot); + this.observedAt = observedAt; + Object.freeze(this); + } + + /** + * @returns {object} + */ + toJSON() { + return { + version: this.version, + handle: this.handle.toString(), + policy: this.policy, + reachability: this.reachability, + root: { ...this.root }, + observedAt: this.observedAt, + }; + } + + static #normalizeRoot(root) { + if (!root || typeof root !== 'object' || Array.isArray(root)) { + throw RetentionWitness.#invalid('Retention witness root must be an object', { root }); + } + if (!ROOT_KINDS.includes(root.kind)) { + throw RetentionWitness.#invalid('Retention witness root kind is invalid', { root }); + } + for (const field of ['namespace', 'ref', 'path']) { + if (typeof root[field] !== 'string' || root[field].length === 0) { + throw RetentionWitness.#invalid(`Retention witness root ${field} is required`, { root }); + } + } + if (!root.ref.startsWith('refs/')) { + throw RetentionWitness.#invalid('Retention witness root ref must be fully qualified', { + root, + }); + } + let generation; + try { + generation = Oid.from(root.generation).toString(); + } catch (error) { + throw RetentionWitness.#invalid('Retention witness root generation is invalid', { + root, + originalError: error, + }); + } + return { + kind: root.kind, + namespace: root.namespace, + ref: root.ref, + generation, + path: root.path, + }; + } + + static #invalid(message, meta) { + return createCasError(message, ErrorCodes.RETENTION_WITNESS_INVALID, meta); + } +} diff --git a/src/domain/value-objects/StagedAsset.js b/src/domain/value-objects/StagedAsset.js new file mode 100644 index 00000000..cbc2d5ba --- /dev/null +++ b/src/domain/value-objects/StagedAsset.js @@ -0,0 +1,63 @@ +import createCasError from '../errors/createCasError.js'; +import { ErrorCodes } from '../errors/index.js'; +import assertCanonicalTimestamp from '../helpers/assertCanonicalTimestamp.js'; +import AssetHandle from './AssetHandle.js'; + +/** + * Immutable result for an asset graph written without a reachability root. + */ +export default class StagedAsset { + /** + * @param {object} value + * @param {AssetHandle|string|object} value.handle + * @param {string} value.slug + * @param {string} value.filename + * @param {number} value.size + * @param {string} value.observedAt + */ + constructor({ handle, slug, filename, size, observedAt }) { + if (typeof slug !== 'string' || slug.length === 0) { + throw StagedAsset.#invalid('Staged asset slug must be a non-empty string', { slug }); + } + if (typeof filename !== 'string' || filename.length === 0) { + throw StagedAsset.#invalid('Staged asset filename must be a non-empty string', { filename }); + } + if (!Number.isSafeInteger(size) || size < 0) { + throw StagedAsset.#invalid('Staged asset size must be a non-negative safe integer', { size }); + } + assertCanonicalTimestamp(observedAt, { + invalid: StagedAsset.#invalid, + message: 'Staged asset observation time must be a canonical UTC timestamp', + }); + + this.version = 1; + this.state = 'staged'; + this.handle = AssetHandle.from(handle); + this.asset = Object.freeze({ slug, filename, size }); + this.retention = Object.freeze({ + policy: null, + reachability: 'unanchored', + protection: 'not-established', + }); + this.observedAt = observedAt; + Object.freeze(this); + } + + /** + * @returns {object} + */ + toJSON() { + return { + version: this.version, + state: this.state, + handle: this.handle.toString(), + asset: { ...this.asset }, + retention: { ...this.retention }, + observedAt: this.observedAt, + }; + } + + static #invalid(message, meta) { + return createCasError(message, ErrorCodes.HANDLE_INVALID, meta); + } +} diff --git a/src/infrastructure/adapters/GitRefAdapter.js b/src/infrastructure/adapters/GitRefAdapter.js index fcbd5a21..1d3b0b30 100644 --- a/src/infrastructure/adapters/GitRefAdapter.js +++ b/src/infrastructure/adapters/GitRefAdapter.js @@ -12,7 +12,6 @@ import { isGitMissingRefError } from '../../domain/helpers/gitRefErrors.js'; * an unref'd timer that allows Node to exit before the next attempt starts. */ const DEFAULT_POLICY = Policy.timeout(30_000); -const GIT_NULL_OID = '0'.repeat(40); /** * {@link GitRefPort} implementation backed by `@git-stunts/plumbing`. @@ -89,13 +88,15 @@ export default class GitRefAdapter extends GitRefPort { * @param {Object} options * @param {string} options.treeOid - Tree OID for the commit. * @param {string|null} [options.parentOid] - Parent commit OID. + * @param {string[]} [options.parentOids] - Ordered parent commit OIDs. * @param {string} options.message - Commit message. * @returns {Promise} The new commit OID. */ - async createCommit({ treeOid, parentOid, message }) { + async createCommit({ treeOid, parentOid, parentOids, message }) { const args = ['commit-tree', treeOid, '-m', message]; - if (parentOid) { - args.push('-p', parentOid); + const parents = parentOids ?? (parentOid ? [parentOid] : []); + for (const parent of parents) { + args.push('-p', parent); } return this.policy.execute(() => this.plumbing.execute({ args }), @@ -113,7 +114,7 @@ export default class GitRefAdapter extends GitRefPort { async updateRef({ ref, newOid, expectedOldOid }) { const args = ['update-ref', ref, newOid]; if (expectedOldOid !== undefined) { - args.push(expectedOldOid ?? GIT_NULL_OID); + args.push(expectedOldOid ?? '0'.repeat(newOid.length)); } await this.policy.execute(() => this.plumbing.execute({ args }), diff --git a/src/ports/GitRefPort.js b/src/ports/GitRefPort.js index cd08fa66..13b35d08 100644 --- a/src/ports/GitRefPort.js +++ b/src/ports/GitRefPort.js @@ -36,6 +36,7 @@ export default class GitRefPort { * @param {Object} _options * @param {string} _options.treeOid - Tree OID for the commit. * @param {string|null} [_options.parentOid] - Parent commit OID (null for root commit). + * @param {string[]} [_options.parentOids] - Ordered parent commit OIDs. * @param {string} _options.message - Commit message. * @returns {Promise} The new commit OID. */ diff --git a/test/helpers/MemoryRefAdapter.js b/test/helpers/MemoryRefAdapter.js index 4c7a15ce..af30af66 100644 --- a/test/helpers/MemoryRefAdapter.js +++ b/test/helpers/MemoryRefAdapter.js @@ -2,16 +2,16 @@ import { createHash } from 'node:crypto'; import GitRefPort from '../../src/ports/GitRefPort.js'; import { CasError, ErrorCodes } from '../../src/domain/errors/index.js'; -function commitOid({ treeOid, parentOid, message }) { - return createHash('sha1') +function commitOid({ treeOid, parentOids, message }) { + const hash = createHash('sha1') .update('commit') .update('\0') .update(treeOid) - .update('\0') - .update(parentOid || '') - .update('\0') - .update(message) - .digest('hex'); + .update('\0'); + for (const parent of parentOids) { + hash.update(parent).update('\0'); + } + return hash.update(message).digest('hex'); } /** @@ -50,12 +50,13 @@ export default class MemoryRefAdapter extends GitRefPort { { commitOid: commitOidToResolve }, ); } - return commit.parentOid ? [commit.parentOid] : []; + return [...commit.parentOids]; } - async createCommit({ treeOid, parentOid, message }) { - const oid = commitOid({ treeOid, parentOid, message }); - this.#commits.set(oid, { treeOid, parentOid, message }); + async createCommit({ treeOid, parentOid, parentOids, message }) { + const parents = parentOids ?? (parentOid ? [parentOid] : []); + const oid = commitOid({ treeOid, parentOids: parents, message }); + this.#commits.set(oid, { treeOid, parentOids: [...parents], message }); return oid; } diff --git a/test/integration/application-storage.test.js b/test/integration/application-storage.test.js new file mode 100644 index 00000000..f4557c81 --- /dev/null +++ b/test/integration/application-storage.test.js @@ -0,0 +1,200 @@ +/** + * Real-Git proof for opaque assets, retention witnesses, and publication. + * + * MUST run inside Docker (GIT_STUNTS_DOCKER=1). Refuses to run on the host. + */ + +import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { spawnSync } from 'node:child_process'; +import os from 'node:os'; +import path from 'node:path'; +import ContentAddressableStore, { AssetHandle } from '../../index.js'; +import { createGitPlumbing } from '../../src/infrastructure/createGitPlumbing.js'; + +if (process.env.GIT_STUNTS_DOCKER !== '1') { + throw new Error( + 'Integration tests MUST run inside Docker (GIT_STUNTS_DOCKER=1). ' + + 'Use: npm run test:integration:node' + ); +} + +vi.setConfig({ testTimeout: 20_000, hookTimeout: 30_000 }); + +const ROOT_SET_REF = 'refs/cas/rootsets/integration/application-assets'; +const OBSERVED_AT = '2026-07-13T10:00:00.000Z'; +const tempDirs = []; +let repoDir; +let cas; + +function tempDir(prefix) { + const dir = mkdtempSync(path.join(os.tmpdir(), prefix)); + tempDirs.push(dir); + return dir; +} + +function gitAt(cwd, args, input) { + const result = spawnSync('git', args, { cwd, encoding: 'utf8', input }); + if (result.error) { + throw result.error; + } + if (result.status !== 0) { + throw new Error(`${result.stderr || result.stdout || 'git failed'}`.trim()); + } + return result.stdout.trim(); +} + +function git(args, input) { + return gitAt(repoDir, args, input); +} + +function prunableOids() { + return new Set( + git(['prune', '-n', '--expire=now']) + .split('\n') + .filter(Boolean) + .map((line) => line.split(' ')[0]) + ); +} + +async function* source(bytes) { + const window = 333; + for (let offset = 0; offset < bytes.length; offset += window) { + yield bytes.subarray(offset, offset + window); + } +} + +async function collect(iterable) { + const chunks = []; + for await (const chunk of iterable) { + chunks.push(Buffer.from(chunk)); + } + return Buffer.concat(chunks); +} + +async function stage(value, slug) { + const payload = Buffer.from(value.repeat(1024)); + const staged = await cas.assets.put({ + source: source(payload), + slug, + filename: `${slug}.txt`, + }); + return { payload, staged }; +} + +beforeAll(async () => { + repoDir = tempDir('cas-application-storage-'); + git(['init', '--bare']); + cas = new ContentAddressableStore({ + plumbing: await createGitPlumbing({ cwd: repoDir }), + chunkSize: 1024, + applicationRefPrefixes: ['refs/warp/'], + clock: { now: () => new Date(OBSERVED_AT) }, + }); +}); + +afterAll(() => { + for (const dir of tempDirs.reverse()) { + rmSync(dir, { recursive: true, force: true }); + } +}); + +describe('application asset retention', () => { + it('moves staged and replaced assets through real reachability states', async () => { + const first = await stage('first', 'retained-first'); + expect(await collect(cas.assets.open({ handle: first.staged.handle }))).toEqual(first.payload); + expect(prunableOids()).toContain(first.staged.handle.oid); + + const retained = await cas.retention.retain({ + handle: first.staged.handle, + root: { ref: ROOT_SET_REF, name: 'current' }, + policy: 'evictable', + }); + expect(prunableOids()).not.toContain(first.staged.handle.oid); + expect(git(['rev-parse', ROOT_SET_REF])).toBe(retained.witness.root.generation); + expect( + git(['ls-tree', retained.witness.root.generation, retained.witness.root.path]) + ).toContain(first.staged.handle.oid); + + const second = await stage('second', 'retained-second'); + await cas.retention.retain({ + handle: second.staged.handle, + root: { ref: ROOT_SET_REF, name: 'current' }, + policy: 'evictable', + }); + expect(prunableOids()).toContain(first.staged.handle.oid); + expect(prunableOids()).not.toContain(second.staged.handle.oid); + }); +}); + +describe('application publication', () => { + it('atomically publishes ordered causal history and reports conflicts', async () => { + const first = await stage('publication-one', 'publication-one'); + const initial = await cas.publications.commit({ + root: first.staged.handle, + commit: { message: 'initial', parents: [] }, + ref: { name: 'refs/warp/events', expected: null }, + }); + expect(git(['rev-parse', 'refs/warp/events'])).toBe(initial.commitId); + expect(git(['rev-parse', `${initial.commitId}^{tree}`])).toBe(first.staged.handle.oid); + + await expect( + cas.publications.commit({ + root: first.staged.handle, + commit: { message: 'invalid tree parent', parents: [first.staged.handle.oid] }, + ref: { name: 'refs/warp/invalid-parent', expected: null }, + }) + ).rejects.toMatchObject({ code: 'PUBLICATION_INVALID' }); + + const second = await stage('publication-two', 'publication-two'); + const next = await cas.publications.commit({ + root: second.staged.handle, + commit: { message: 'next', parents: [initial.commitId] }, + ref: { name: 'refs/warp/events', expected: initial.commitId }, + }); + expect(git(['rev-list', '--parents', '-n', '1', next.commitId]).split(/\s+/u)).toEqual([ + next.commitId, + initial.commitId, + ]); + + await expect( + cas.publications.commit({ + root: first.staged.handle, + commit: { message: 'stale', parents: [initial.commitId] }, + ref: { name: 'refs/warp/events', expected: initial.commitId }, + }) + ).rejects.toMatchObject({ + code: 'PUBLICATION_CONFLICT', + meta: { expected: initial.commitId, observed: next.commitId }, + }); + expect(git(['rev-parse', 'refs/warp/events'])).toBe(next.commitId); + }); +}); + +describe('asset handle transfer', () => { + it('opens the same token after a mirror and fails explicitly without its graph', async () => { + const { payload, staged } = await stage('portable', 'portable'); + await cas.publications.commit({ + root: staged.handle, + commit: { message: 'portable root', parents: [] }, + ref: { name: 'refs/warp/portable', expected: null }, + }); + + const mirrorDir = tempDir('cas-application-mirror-'); + gitAt(path.dirname(mirrorDir), ['clone', '--mirror', repoDir, mirrorDir]); + const mirror = new ContentAddressableStore({ + plumbing: await createGitPlumbing({ cwd: mirrorDir }), + }); + const transferred = AssetHandle.from(staged.handle.toString()); + expect(await collect(mirror.assets.open({ handle: transferred }))).toEqual(payload); + + const emptyDir = tempDir('cas-application-empty-'); + gitAt(emptyDir, ['init', '--bare']); + const empty = new ContentAddressableStore({ + plumbing: await createGitPlumbing({ cwd: emptyDir }), + }); + await expect(collect(empty.assets.open({ handle: transferred }))).rejects.toMatchObject({ + code: 'HANDLE_TARGET_MISSING', + }); + }); +}); diff --git a/test/unit/domain/services/AssetService.test.js b/test/unit/domain/services/AssetService.test.js new file mode 100644 index 00000000..65802d51 --- /dev/null +++ b/test/unit/domain/services/AssetService.test.js @@ -0,0 +1,161 @@ +import { describe, expect, it, vi } from 'vitest'; +import AssetService from '../../../../src/domain/services/AssetService.js'; +import CasService from '../../../../src/domain/services/CasService.js'; +import AssetHandle from '../../../../src/domain/value-objects/AssetHandle.js'; +import StagedAsset from '../../../../src/domain/value-objects/StagedAsset.js'; +import FixedChunker from '../../../../src/infrastructure/chunkers/FixedChunker.js'; +import JsonCodec from '../../../../src/infrastructure/codecs/JsonCodec.js'; +import NodeCompressionAdapter from '../../../../src/infrastructure/adapters/NodeCompressionAdapter.js'; +import SilentObserver from '../../../../src/infrastructure/adapters/SilentObserver.js'; +import MemoryPersistenceAdapter from '../../../helpers/MemoryPersistenceAdapter.js'; +import { getTestCryptoAdapter } from '../../../helpers/crypto-adapter.js'; + +const OBSERVED_AT = '2026-07-13T10:00:00.000Z'; +const testCrypto = await getTestCryptoAdapter(); + +function makeAssets({ concurrency = 1 } = {}) { + const persistence = new MemoryPersistenceAdapter(); + const cas = new CasService({ + persistence, + crypto: testCrypto, + codec: new JsonCodec(), + observability: new SilentObserver(), + chunkSize: 1024, + merkleThreshold: 2, + concurrency, + chunker: new FixedChunker({ chunkSize: 1024 }), + compressionAdapter: new NodeCompressionAdapter(), + }); + const assets = new AssetService({ + cas, + clock: { now: () => new Date(OBSERVED_AT) }, + }); + return { assets, persistence }; +} + +async function* source(bytes, sourceChunkSize = 257) { + for (let offset = 0; offset < bytes.length; offset += sourceChunkSize) { + yield bytes.subarray(offset, offset + sourceChunkSize); + } +} + +async function collect(iterable) { + const chunks = []; + for await (const chunk of iterable) { + chunks.push(chunk); + } + return Buffer.concat(chunks.map((chunk) => Buffer.from(chunk))); +} + +describe('AssetService put/open', () => { + it('streams a payload through a staged handle round trip', async () => { + const { assets } = makeAssets(); + const payload = Buffer.alloc(8 * 1024 + 17, 0x5a); + + const staged = await assets.put({ + source: source(payload), + slug: 'large-state', + filename: 'state.bin', + }); + + expect(staged).toBeInstanceOf(StagedAsset); + expect(staged.handle).toBeInstanceOf(AssetHandle); + expect(staged.handle.codec).toBe('json'); + expect(staged.asset.size).toBe(payload.length); + expect(staged.retention).toEqual({ + policy: null, + reachability: 'unanchored', + protection: 'not-established', + }); + expect(staged.observedAt).toBe(OBSERVED_AT); + expect(staged).not.toHaveProperty('manifest'); + + const opened = assets.open({ handle: staged.handle }); + expect(opened[Symbol.asyncIterator]).toBeTypeOf('function'); + await expect(collect(opened)).resolves.toEqual(payload); + }); + + it('adopts a validated existing manifest tree without repository identity', async () => { + const { assets } = makeAssets(); + const staged = await assets.put({ + source: source(Buffer.from('adopt me')), + slug: 'adopted', + filename: 'adopted.txt', + }); + + const adopted = await assets.adopt({ treeOid: staged.handle.oid }); + + expect(adopted.handle.toString()).toBe(staged.handle.toString()); + expect(adopted.asset).toEqual(staged.asset); + }); +}); + +describe('AssetService chunk graph validation', () => { + it('validates unique chunk objects with bounded concurrency', async () => { + const { assets, persistence } = makeAssets({ concurrency: 3 }); + const payload = Buffer.alloc(8 * 1024); + for (let index = 0; index < 8; index++) { + payload.fill(index, index * 1024, (index + 1) * 1024); + } + const staged = await assets.put({ + source: source(payload), + slug: 'bounded-validation', + filename: 'bounded.bin', + }); + const readObjectType = persistence.readObjectType.bind(persistence); + let active = 0; + let maximum = 0; + vi.spyOn(persistence, 'readObjectType').mockImplementation(async (oid) => { + active += 1; + maximum = Math.max(maximum, active); + await new Promise((resolve) => setTimeout(resolve, 5)); + try { + return await readObjectType(oid); + } finally { + active -= 1; + } + }); + + await assets.resolveRoot(staged.handle); + + expect(maximum).toBe(3); + }); +}); + +describe('AssetService handle failures', () => { + it('reports a missing transferred object graph explicitly', async () => { + const { assets } = makeAssets(); + const handle = new AssetHandle({ codec: 'json', oid: 'f'.repeat(40) }); + + await expect(collect(assets.open({ handle }))).rejects.toMatchObject({ + code: 'HANDLE_TARGET_MISSING', + meta: { handle: handle.toString() }, + }); + }); + + it('rejects an object with the wrong Git kind', async () => { + const { assets, persistence } = makeAssets(); + const blobOid = await persistence.writeBlob(Buffer.from('not a tree')); + const handle = new AssetHandle({ codec: 'json', oid: blobOid }); + + await expect(assets.resolveRoot(handle)).rejects.toMatchObject({ + code: 'HANDLE_TARGET_TYPE_MISMATCH', + meta: { actualType: 'blob', expectedType: 'tree' }, + }); + }); + + it('rejects a handle for a different manifest codec', async () => { + const { assets } = makeAssets(); + const staged = await assets.put({ + source: source(Buffer.from('codec')), + slug: 'codec', + filename: 'codec.txt', + }); + const handle = new AssetHandle({ codec: 'cbor', oid: staged.handle.oid }); + + await expect(assets.resolveRoot(handle)).rejects.toMatchObject({ + code: 'HANDLE_CODEC_MISMATCH', + meta: { expectedCodec: 'json', actualCodec: 'cbor' }, + }); + }); +}); diff --git a/test/unit/domain/services/PublicationService.test.js b/test/unit/domain/services/PublicationService.test.js new file mode 100644 index 00000000..d31efcf7 --- /dev/null +++ b/test/unit/domain/services/PublicationService.test.js @@ -0,0 +1,215 @@ +import { describe, expect, it, vi } from 'vitest'; +import PublicationService from '../../../../src/domain/services/PublicationService.js'; +import AssetHandle from '../../../../src/domain/value-objects/AssetHandle.js'; +import RetentionWitness from '../../../../src/domain/value-objects/RetentionWitness.js'; +import MemoryPersistenceAdapter from '../../../helpers/MemoryPersistenceAdapter.js'; +import MemoryRefAdapter from '../../../helpers/MemoryRefAdapter.js'; + +const OBSERVED_AT = '2026-07-13T10:00:00.000Z'; + +function makeService({ applicationRefPrefixes = ['refs/warp/'] } = {}) { + const persistence = new MemoryPersistenceAdapter(); + const ref = new MemoryRefAdapter(); + const resolveRoot = async (value) => { + const handle = AssetHandle.from(value); + const type = await persistence.readObjectType(handle.oid); + return Object.freeze({ handle, oid: handle.oid, type }); + }; + const publications = new PublicationService({ + ref, + resolveRoot, + applicationRefPrefixes, + clock: { now: () => new Date(OBSERVED_AT) }, + }); + return { persistence, publications, ref }; +} + +async function makeHandle(persistence, value) { + const blobOid = await persistence.writeBlob(Buffer.from(value)); + const treeOid = await persistence.writeTree([`100644 blob ${blobOid}\tpayload`]); + return new AssetHandle({ codec: 'json', oid: treeOid }); +} + +async function publish(publications, root, overrides = {}) { + return await publications.commit({ + root, + commit: { + message: overrides.message ?? 'publish asset', + parents: overrides.parents ?? [], + }, + ref: { + name: overrides.name ?? 'refs/warp/events', + expected: Object.hasOwn(overrides, 'expected') ? overrides.expected : null, + }, + }); +} + +describe('PublicationService publication evidence', () => { + it('publishes a validated root with an immutable witness', async () => { + const { persistence, publications, ref } = makeService(); + const handle = await makeHandle(persistence, 'first'); + + const result = await publish(publications, handle); + + expect(await ref.resolveRef('refs/warp/events')).toBe(result.commitId); + expect(await ref.resolveTree(result.commitId)).toBe(handle.oid); + expect(result.root).toBe(handle); + expect(result.witness).toBeInstanceOf(RetentionWitness); + expect(result.witness).toMatchObject({ + handle, + policy: 'pinned', + reachability: 'anchored', + root: { + kind: 'publication', + namespace: 'refs/warp/', + ref: 'refs/warp/events', + generation: result.commitId, + path: '/', + }, + observedAt: OBSERVED_AT, + }); + expect(Object.isFrozen(result)).toBe(true); + }); + + it('records the most specific configured namespace in the witness', async () => { + const persistence = new MemoryPersistenceAdapter(); + const ref = new MemoryRefAdapter(); + const publications = new PublicationService({ + ref, + resolveRoot: async (value) => { + const handle = AssetHandle.from(value); + return Object.freeze({ + handle, + oid: handle.oid, + type: await persistence.readObjectType(handle.oid), + }); + }, + applicationRefPrefixes: ['refs/warp/', 'refs/warp/cache/'], + clock: { now: () => new Date(OBSERVED_AT) }, + }); + const result = await publish(publications, await makeHandle(persistence, 'specific'), { + name: 'refs/warp/cache/events', + }); + + expect(result.witness.root.namespace).toBe('refs/warp/cache/'); + }); +}); + +describe('PublicationService causal parents', () => { + it('preserves caller-controlled ordered parents', async () => { + const { persistence, publications, ref } = makeService(); + const left = await publish(publications, await makeHandle(persistence, 'left'), { + name: 'refs/warp/left', + }); + const right = await publish(publications, await makeHandle(persistence, 'right'), { + name: 'refs/warp/right', + }); + const joined = await publish(publications, await makeHandle(persistence, 'joined'), { + name: 'refs/warp/main', + parents: [right.commitId, left.commitId], + }); + + expect(await ref.resolveParents(joined.commitId)).toEqual([right.commitId, left.commitId]); + }); +}); + +describe('PublicationService failures', () => { + it('returns expected and observed heads for a compare-and-swap conflict', async () => { + const { persistence, publications } = makeService(); + const first = await publish(publications, await makeHandle(persistence, 'first')); + const replacement = await makeHandle(persistence, 'replacement'); + + await expect(publish(publications, replacement, { expected: null })).rejects.toMatchObject({ + code: 'PUBLICATION_CONFLICT', + meta: { + ref: 'refs/warp/events', + expected: null, + observed: first.commitId, + }, + }); + }); + + it('rejects refs outside explicitly configured application namespaces', async () => { + const { persistence, publications } = makeService(); + const handle = await makeHandle(persistence, 'forbidden'); + + await expect(publish(publications, handle, { name: 'refs/heads/main' })).rejects.toMatchObject({ + code: 'PUBLICATION_REF_FORBIDDEN', + }); + await expect( + publish(publications, handle, { name: 'refs/cas/rootsets/internal' }) + ).rejects.toMatchObject({ code: 'PUBLICATION_REF_FORBIDDEN' }); + }); +}); + +describe('PublicationService reserved ref boundaries', () => { + it.each([ + 'refs/bisect/session', + 'refs/cas/rootsets/internal', + 'refs/heads/main', + 'refs/notes/commits', + 'refs/remotes/origin/main', + `refs/replace/${'a'.repeat(40)}`, + 'refs/rewritten/main', + 'refs/stash', + 'refs/tags/v1.0.0', + 'refs/worktree/main', + ])('hard-blocks reserved Git ref %s under a broad allowlist', async (name) => { + const { persistence, publications } = makeService({ + applicationRefPrefixes: ['refs/'], + }); + const handle = await makeHandle(persistence, 'reserved'); + + await expect(publish(publications, handle, { name })).rejects.toMatchObject({ + code: 'PUBLICATION_REF_FORBIDDEN', + }); + }); + + it.each([ + 'refs/bisect/', + 'refs/cas/', + 'refs/heads/', + 'refs/notes/', + 'refs/remotes/', + 'refs/replace/', + 'refs/rewritten/', + 'refs/stash/', + 'refs/tags/', + 'refs/worktree/', + ])('rejects reserved namespace configuration %s', (prefix) => { + expect(() => makeService({ applicationRefPrefixes: [prefix] })).toThrow( + expect.objectContaining({ code: 'PUBLICATION_INVALID' }) + ); + }); +}); + +describe('PublicationService failures requiring explicit expectations', () => { + it('requires an explicit expected head', async () => { + const { persistence, publications } = makeService(); + const handle = await makeHandle(persistence, 'expected'); + + await expect( + publications.commit({ + root: handle, + commit: { message: 'missing expectation', parents: [] }, + ref: { name: 'refs/warp/events' }, + }) + ).rejects.toMatchObject({ code: 'PUBLICATION_INVALID' }); + }); +}); + +describe('PublicationService ref failure normalization', () => { + it('normalizes a ref failure even when the current head cannot be observed', async () => { + const { persistence, publications, ref } = makeService(); + const handle = await makeHandle(persistence, 'unobservable'); + const writeError = new Error('ref write failed'); + const observationError = new Error('ref read failed'); + vi.spyOn(ref, 'updateRef').mockRejectedValueOnce(writeError); + vi.spyOn(ref, 'resolveRef').mockRejectedValueOnce(observationError); + + await expect(publish(publications, handle)).rejects.toMatchObject({ + code: 'PUBLICATION_REF_UPDATE_FAILED', + meta: { originalError: writeError, observationError }, + }); + }); +}); diff --git a/test/unit/domain/services/RetentionService.test.js b/test/unit/domain/services/RetentionService.test.js new file mode 100644 index 00000000..089566bf --- /dev/null +++ b/test/unit/domain/services/RetentionService.test.js @@ -0,0 +1,151 @@ +import { describe, expect, it, vi } from 'vitest'; +import AssetService from '../../../../src/domain/services/AssetService.js'; +import CasService from '../../../../src/domain/services/CasService.js'; +import RetentionService from '../../../../src/domain/services/RetentionService.js'; +import RootSetRegistry from '../../../../src/domain/services/RootSetRegistry.js'; +import RetentionWitness from '../../../../src/domain/value-objects/RetentionWitness.js'; +import FixedChunker from '../../../../src/infrastructure/chunkers/FixedChunker.js'; +import JsonCodec from '../../../../src/infrastructure/codecs/JsonCodec.js'; +import NodeCompressionAdapter from '../../../../src/infrastructure/adapters/NodeCompressionAdapter.js'; +import SilentObserver from '../../../../src/infrastructure/adapters/SilentObserver.js'; +import MemoryPersistenceAdapter from '../../../helpers/MemoryPersistenceAdapter.js'; +import MemoryRefAdapter from '../../../helpers/MemoryRefAdapter.js'; +import { getTestCryptoAdapter } from '../../../helpers/crypto-adapter.js'; + +const OBSERVED_AT = '2026-07-13T10:00:00.000Z'; +const ROOT_REF = 'refs/cas/rootsets/application-assets'; +const testCrypto = await getTestCryptoAdapter(); + +function makeServices() { + const persistence = new MemoryPersistenceAdapter(); + const ref = new MemoryRefAdapter(); + const cas = new CasService({ + persistence, + crypto: testCrypto, + codec: new JsonCodec(), + observability: new SilentObserver(), + chunkSize: 1024, + merkleThreshold: 1000, + chunker: new FixedChunker({ chunkSize: 1024 }), + compressionAdapter: new NodeCompressionAdapter(), + }); + const assets = new AssetService({ cas }); + const rootSets = new RootSetRegistry({ persistence, ref }); + const resolveRoot = vi.fn((handle) => assets.resolveRoot(handle)); + const retention = new RetentionService({ + rootSets, + resolveRoot, + clock: { now: () => new Date(OBSERVED_AT) }, + }); + return { assets, persistence, ref, resolveRoot, retention, rootSets }; +} + +async function* source(value) { + yield Buffer.from(value); +} + +async function stage(assets, value) { + return await assets.put({ + source: source(value), + slug: `asset-${value}`, + filename: `${value}.txt`, + }); +} + +describe('RetentionService', () => { + it('retains an asset through the exact witnessed generation and path', async () => { + const { assets, persistence, retention, rootSets } = makeServices(); + const staged = await stage(assets, 'one'); + + const result = await retention.retain({ + handle: staged.handle, + root: { ref: ROOT_REF, name: 'current-state' }, + policy: 'pinned', + }); + + expect(result.changed).toBe(true); + expect(result.witness).toBeInstanceOf(RetentionWitness); + expect(result.witness).toMatchObject({ + handle: staged.handle, + policy: 'pinned', + reachability: 'anchored', + root: { + kind: 'root-set', + namespace: 'application-assets', + ref: ROOT_REF, + path: 'root-00000000', + }, + observedAt: OBSERVED_AT, + }); + + const state = await rootSets.open({ ref: ROOT_REF }).read(); + const generationTree = await persistence.readTree(state.treeOid); + const evidenceEdge = generationTree.find((entry) => entry.name === result.witness.root.path); + expect(state.headOid).toBe(result.witness.root.generation); + expect(evidenceEdge).toMatchObject({ + type: 'tree', + oid: staged.handle.oid, + }); + expect(Object.isFrozen(result)).toBe(true); + }); +}); + +describe('RetentionService root validation', () => { + it('rejects a malformed root ref before resolving or mutating storage', async () => { + const { assets, ref, resolveRoot, retention } = makeServices(); + const staged = await stage(assets, 'invalid-root'); + const updateRef = vi.spyOn(ref, 'updateRef'); + + await expect( + retention.retain({ + handle: staged.handle, + root: { ref: 'refs/heads/main', name: 'current-state' }, + }) + ).rejects.toMatchObject({ code: 'ROOT_SET_REF_INVALID' }); + + expect(resolveRoot).not.toHaveBeenCalled(); + expect(updateRef).not.toHaveBeenCalled(); + }); +}); + +describe('RetentionService replacement', () => { + it('atomically replaces the named root and returns a fresh witness', async () => { + const { assets, retention, rootSets } = makeServices(); + const first = await stage(assets, 'first'); + const second = await stage(assets, 'second'); + const retained = await retention.retain({ + handle: first.handle, + root: { ref: ROOT_REF, name: 'current-state' }, + policy: 'evictable', + }); + + const replaced = await retention.retain({ + handle: second.handle, + root: { ref: ROOT_REF, name: 'current-state' }, + policy: 'evictable', + }); + + expect(replaced.changed).toBe(true); + expect(replaced.witness.root.generation).not.toBe(retained.witness.root.generation); + expect((await rootSets.open({ ref: ROOT_REF }).list())[0]).toMatchObject({ + name: 'current-state', + oid: second.handle.oid, + retention: 'evictable', + }); + }); + + it('returns current-generation evidence for an idempotent retain', async () => { + const { assets, retention } = makeServices(); + const staged = await stage(assets, 'same'); + const options = { + handle: staged.handle, + root: { ref: ROOT_REF, name: 'same' }, + policy: 'pinned', + }; + const first = await retention.retain(options); + const second = await retention.retain(options); + + expect(second.changed).toBe(false); + expect(second.witness.root.generation).toBe(first.witness.root.generation); + }); +}); diff --git a/test/unit/domain/value-objects/ApplicationStorageResults.test.js b/test/unit/domain/value-objects/ApplicationStorageResults.test.js new file mode 100644 index 00000000..32bc72f7 --- /dev/null +++ b/test/unit/domain/value-objects/ApplicationStorageResults.test.js @@ -0,0 +1,126 @@ +import { describe, expect, it } from 'vitest'; +import AssetHandle from '../../../../src/domain/value-objects/AssetHandle.js'; +import RetentionWitness from '../../../../src/domain/value-objects/RetentionWitness.js'; +import StagedAsset from '../../../../src/domain/value-objects/StagedAsset.js'; + +const handle = new AssetHandle({ + codec: 'json', + oid: '0123456789abcdef0123456789abcdef01234567', +}); +const observedAt = '2026-07-13T10:00:00.000Z'; + +function makeStaged(overrides = {}) { + return new StagedAsset({ + handle, + slug: 'alice-avatar', + filename: 'alice.png', + size: 42, + observedAt, + ...overrides, + }); +} + +describe('StagedAsset', () => { + it('states that staging created no reachability root', () => { + const staged = makeStaged(); + + expect(staged).toMatchObject({ + version: 1, + state: 'staged', + handle, + asset: { + slug: 'alice-avatar', + filename: 'alice.png', + size: 42, + }, + retention: { + policy: null, + reachability: 'unanchored', + protection: 'not-established', + }, + observedAt, + }); + expect(Object.isFrozen(staged)).toBe(true); + expect(Object.isFrozen(staged.asset)).toBe(true); + expect(Object.isFrozen(staged.retention)).toBe(true); + expect(staged.toJSON().handle).toBe(handle.toString()); + }); + + it.each([ + ['slug', { slug: '' }], + ['filename', { filename: '' }], + ['size', { size: -1 }], + ['observedAt', { observedAt: '2026-07-13T10:00:00Z' }], + ])('rejects an invalid %s', (_field, override) => { + expect(() => makeStaged(override)).toThrow( + expect.objectContaining({ code: 'HANDLE_INVALID' }) + ); + }); +}); + +describe('RetentionWitness', () => { + it('captures immutable generation-scoped root evidence', () => { + const witness = new RetentionWitness({ + handle, + policy: 'pinned', + reachability: 'anchored', + root: { + kind: 'root-set', + namespace: 'warp-assets', + ref: 'refs/cas/rootsets/warp-assets', + generation: 'a'.repeat(40), + path: 'root-00000000', + }, + observedAt, + }); + + expect(witness.version).toBe(1); + expect(witness.root.generation).toBe('a'.repeat(40)); + expect(witness.toJSON()).toEqual({ + version: 1, + handle: handle.toString(), + policy: 'pinned', + reachability: 'anchored', + root: { + kind: 'root-set', + namespace: 'warp-assets', + ref: 'refs/cas/rootsets/warp-assets', + generation: 'a'.repeat(40), + path: 'root-00000000', + }, + observedAt, + }); + expect(Object.isFrozen(witness)).toBe(true); + expect(Object.isFrozen(witness.root)).toBe(true); + }); +}); + +describe('RetentionWitness validation', () => { + it.each([ + [{ policy: 'forever' }, 'RETENTION_WITNESS_INVALID'], + [{ reachability: 'unknown' }, 'RETENTION_WITNESS_INVALID'], + [{ root: { generation: 'bad' } }, 'RETENTION_WITNESS_INVALID'], + [{ observedAt: 'tomorrow' }, 'RETENTION_WITNESS_INVALID'], + ])('rejects invalid evidence %#', (override, code) => { + const { root: rootOverride, ...topLevelOverride } = override; + const root = { + kind: 'root-set', + namespace: 'warp-assets', + ref: 'refs/cas/rootsets/warp-assets', + generation: 'a'.repeat(40), + path: 'root-00000000', + ...rootOverride, + }; + + const options = { + handle, + policy: 'pinned', + reachability: 'anchored', + root, + observedAt, + ...topLevelOverride, + }; + + expect(() => new RetentionWitness(options)).toThrow(expect.objectContaining({ code })); + }); +}); diff --git a/test/unit/domain/value-objects/AssetHandle.test.js b/test/unit/domain/value-objects/AssetHandle.test.js new file mode 100644 index 00000000..83cc0781 --- /dev/null +++ b/test/unit/domain/value-objects/AssetHandle.test.js @@ -0,0 +1,54 @@ +import { describe, expect, it } from 'vitest'; +import AssetHandle from '../../../../src/domain/value-objects/AssetHandle.js'; + +const OID = '0123456789abcdef0123456789abcdef01234567'; +const TOKEN = `git-cas:1:asset:manifest-tree:json:sha1:${OID}`; + +describe('AssetHandle', () => { + it('round-trips one canonical repository-independent token', () => { + const handle = new AssetHandle({ codec: 'json', oid: OID }); + + expect(handle).toMatchObject({ + version: 1, + kind: 'asset', + format: 'manifest-tree', + codec: 'json', + hashAlgorithm: 'sha1', + oid: OID, + }); + expect(handle.toString()).toBe(TOKEN); + expect(AssetHandle.from(TOKEN)).toEqual(handle); + expect(AssetHandle.from(handle.toJSON())).toEqual(handle); + expect(handle.toJSON()).toEqual({ + version: 1, + kind: 'asset', + format: 'manifest-tree', + codec: 'json', + hashAlgorithm: 'sha1', + oid: OID, + }); + expect(handle.toJSON()).not.toHaveProperty('repository'); + expect(Object.isFrozen(handle)).toBe(true); + }); + + it('derives sha256 from a 64-character object identifier', () => { + const oid = 'a'.repeat(64); + + expect(new AssetHandle({ codec: 'cbor', oid })).toMatchObject({ + hashAlgorithm: 'sha256', + oid, + }); + }); + + it.each([ + ['not a token', 'HANDLE_INVALID'], + [`git-cas:2:asset:manifest-tree:json:sha1:${OID}`, 'HANDLE_INVALID'], + [`git-cas:1:bundle:manifest-tree:json:sha1:${OID}`, 'HANDLE_KIND_MISMATCH'], + [`git-cas:1:asset:unknown:json:sha1:${OID}`, 'HANDLE_INVALID'], + [`git-cas:1:asset:manifest-tree:JSON:sha1:${OID}`, 'HANDLE_INVALID'], + [`git-cas:1:asset:manifest-tree:json:sha256:${OID}`, 'HANDLE_INVALID'], + ['git-cas:1:asset:manifest-tree:json:sha1:nope', 'HANDLE_INVALID'], + ])('rejects malformed token %s', (token, code) => { + expect(() => AssetHandle.from(token)).toThrow(expect.objectContaining({ code })); + }); +}); diff --git a/test/unit/facade/ContentAddressableStore.application-storage.test.js b/test/unit/facade/ContentAddressableStore.application-storage.test.js new file mode 100644 index 00000000..6de7c14f --- /dev/null +++ b/test/unit/facade/ContentAddressableStore.application-storage.test.js @@ -0,0 +1,35 @@ +import { describe, expect, it, vi } from 'vitest'; +import ContentAddressableStore, { + AssetHandle, + RetentionWitness, + StagedAsset, +} from '../../../index.js'; + +function mockPlumbing() { + return { + execute: vi.fn(), + executeStream: vi.fn(), + }; +} + +describe('ContentAddressableStore application storage capabilities', () => { + it('exposes frozen high-level capability groups', () => { + const cas = new ContentAddressableStore({ + plumbing: mockPlumbing(), + applicationRefPrefixes: ['refs/warp/'], + }); + + expect(Object.keys(cas.assets)).toEqual(['put', 'adopt', 'open']); + expect(Object.keys(cas.retention)).toEqual(['retain']); + expect(Object.keys(cas.publications)).toEqual(['commit']); + expect(Object.isFrozen(cas.assets)).toBe(true); + expect(Object.isFrozen(cas.retention)).toBe(true); + expect(Object.isFrozen(cas.publications)).toBe(true); + }); + + it('exports immutable result constructors from the package root', () => { + expect(AssetHandle).toBeTypeOf('function'); + expect(StagedAsset).toBeTypeOf('function'); + expect(RetentionWitness).toBeTypeOf('function'); + }); +}); diff --git a/test/unit/infrastructure/adapters/GitRefAdapter.test.js b/test/unit/infrastructure/adapters/GitRefAdapter.test.js index de9efd0a..ab47d45e 100644 --- a/test/unit/infrastructure/adapters/GitRefAdapter.test.js +++ b/test/unit/infrastructure/adapters/GitRefAdapter.test.js @@ -92,6 +92,20 @@ describe('GitRefAdapter.updateRef()', () => { }); }); + it('uses the repository hash width for SHA-256 create-only updates', async () => { + const { adapter, plumbing } = createAdapter(); + + await adapter.updateRef({ + ref: 'refs/cas/vault', + newOid: 'a'.repeat(64), + expectedOldOid: null, + }); + + expect(plumbing.execute).toHaveBeenCalledWith({ + args: ['update-ref', 'refs/cas/vault', 'a'.repeat(64), '0'.repeat(64)], + }); + }); + it('omits the expected old OID only when the caller explicitly leaves it undefined', async () => { const { adapter, plumbing } = createAdapter(); diff --git a/test/unit/types/declaration-accuracy.test.js b/test/unit/types/declaration-accuracy.test.js index 1fd7c8ed..d04a877d 100644 --- a/test/unit/types/declaration-accuracy.test.js +++ b/test/unit/types/declaration-accuracy.test.js @@ -73,3 +73,18 @@ describe('Root-set declaration accuracy', () => { expect(declarations).toContain('resolveParents(commitOid: string): Promise;'); }); }); + +describe('Application-storage declaration accuracy', () => { + it('declares opaque handles, witnesses, and frozen facade capabilities', () => { + const declarations = read('index.d.ts'); + + expect(declarations).toContain('export declare class AssetHandle'); + expect(declarations).toContain('export declare class StagedAsset'); + expect(declarations).toContain('export declare class RetentionWitness'); + expect(declarations).toContain('readonly assets: AssetCapability;'); + expect(declarations).toContain('readonly retention: RetentionCapability;'); + expect(declarations).toContain('readonly publications: PublicationCapability;'); + expect(declarations).toContain('applicationRefPrefixes?: string[];'); + expect(declarations).toContain('parentOids?: string[];'); + }); +});