diff --git a/.changeset/dynamodb-eql-v3.md b/.changeset/dynamodb-eql-v3.md new file mode 100644 index 000000000..c6303c997 --- /dev/null +++ b/.changeset/dynamodb-eql-v3.md @@ -0,0 +1,74 @@ +--- +'@cipherstash/stack': minor +'stash': patch +--- + +`encryptedDynamoDB` now accepts EQL v3 tables. + +Pass a table built with `encryptedTable` + the `types.*` domains from +`@cipherstash/stack/v3` (or `@cipherstash/stack/eql/v3`) to any of +`encryptModel`, `bulkEncryptModels`, `decryptModel`, `bulkDecryptModels`. Both +the typed client from `EncryptionV3` and the nominal client from +`Encryption({ config: { eqlVersion: 3 } })` are accepted. + +EQL v2 tables continue to work unchanged — this is additive, and no existing +caller needs to change. The table decides which wire format is used, so a +DynamoDB table populated under one version must keep being read with that +version. + +This fixes a latent bug that made v3 unusable: the write path detected an +encrypted value by its `k: 'ct'` tag, but EQL v3 scalars carry no `k` +discriminator at all. Every v3 scalar fell through to the nested-object branch +and was written as a raw map instead of being split into `__source` and +`__hmac`. + +Notes on capability: + +- Only equality is usable on DynamoDB. `__hmac` is written for domains + that mint an `hm` term — the `*Eq` family, plus `TextOrd`/`TextOrdOre`/ + `TextSearch`. Ordering and bloom-filter terms have no DynamoDB query surface + and are not stored, so those columns remain decryptable but not queryable. +- Nested attributes are supported in v3. There is no nested-group authoring + form (that is a compile error), so declare the column flat with a dotted + path — `{ 'profile.ssn': types.TextEq('profile.ssn') }`. The model is + matched by dotted path, so `{ profile: { ssn } }` resolves, and the nested + attribute keeps its `__hmac` for key conditions. +- Audit metadata on `decryptModel` / `bulkDecryptModels` requires the nominal + client; the `EncryptionV3` client has no audit surface on decrypt. + +The DynamoDB adapter also gains its first test coverage — across the v2 and v3 +paths, where it previously had none. + +Robustness, from review: + +- Passing a v3 table to a client that never registered it (one built for a + different schema set, so it is not in v3 mode for that table) now throws a + clear, actionable error naming the table, instead of failing opaquely deep in + the FFI. +- A malformed decrypt result from a non-conforming client is surfaced as a + failure rather than resolving as a silent `undefined` success. +- Reading back a `__source` attribute that matches no declared column now + logs a debug diagnostic instead of silently returning the raw ciphertext. +- Caller input that cannot be structurally cloned no longer reaches the FFI by + reference — the "encryption never mutates a caller's object" guarantee holds + on that path too. +- The write path now splits only declared columns, matched on the same property + path the read path rebuilds from. A pre-encrypted payload placed under an + undeclared nested name is stored whole (and round-trips) instead of being + split into a `__source` the read path could never reassemble. +- A degenerate payload with an empty-string ciphertext is split like any other + ciphertext rather than falling through and being written as a raw map, which + had leaked its `v`/`i` envelope metadata into storage. +- Arrays are documented as a deliberate carve-out: the mapping does not descend + into them, so a payload inside a list is stored whole (still decryptable, but + not queryable and not part of the `__source`/`__hmac` layout). + +The v3 overloads are strongly typed. `encryptModel` / `bulkEncryptModels` check +the input model against the table's column domains, and return the DynamoDB +attribute map that is actually written — the new exported `EncryptedAttributes` +type, where a declared column `email` becomes `email__source` (plus +`email__hmac` for the equality domains that mint one) rather than surviving as +`email`. `decryptModel` / `bulkDecryptModels` invert it via `DecryptedAttributes`. +`AnyEncryptedTable`, `DynamoDBEncryptionClient` and `AuditConfig` are now +exported from `@cipherstash/stack/dynamodb` so these signatures can be named. +The EQL v2 overloads are unchanged. diff --git a/AGENTS.md b/AGENTS.md index 62b8299c7..41b11a82f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -86,7 +86,7 @@ If these variables are missing, tests that require live encryption will fail or - `packages/stack-supabase`: Supabase integration (`@cipherstash/stack-supabase`), depends on `@cipherstash/stack` — `encryptedSupabase` (v2) and `encryptedSupabaseV3` (v3). Split out of `@cipherstash/stack`. - `packages/schema`: Schema builder utilities and types (`encryptedTable`, `encryptedColumn`, `encryptedField`) - `packages/nextjs`: Next.js helpers and Clerk integration (`./clerk` export) -- `packages/protect-dynamodb`: DynamoDB helpers (`encryptedDynamoDB`) +- `packages/protect-dynamodb`: DynamoDB helpers (`protectDynamoDB`), built on `@cipherstash/protect`. A fork of the shipping adapter, kept for existing consumers of the standalone package; EQL v2 only. The maintained implementation is `packages/stack/src/dynamodb` (`encryptedDynamoDB`) - `packages/utils`: Shared config (`utils/config`) and logger (`utils/logger`) - `packages/bench`: Performance / index-engagement benchmarks (private, not published) - `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README) @@ -147,7 +147,7 @@ Three rules to remember when editing CI or pnpm config: ## Key Concepts and APIs - **Initialization**: `EncryptionV3({ schemas })` returns the typed EQL v3 client. (`Encryption({ schemas })` is the legacy v2 client.) Provide at least one `encryptedTable`. -- **Schema (EQL v3, the documented approach)**: Define tables/columns with `encryptedTable` and the `types.*` concrete-domain factories from `@cipherstash/stack/eql/v3` (`types.TextSearch`, `types.IntegerOrd`, `types.Json`, …) — each domain's query capabilities are fixed by its type; there are no chainable capability tuners. Build the client with `EncryptionV3` from `@cipherstash/stack/v3`. (Legacy EQL v2 — `Encryption` + `encryptedColumn(...).equality().freeTextSearch().orderAndRange().searchableJson()` from `@cipherstash/stack/schema` — remains for existing deployments and is what DynamoDB (`encryptedField`) still requires; see #657.) +- **Schema (EQL v3, the documented approach)**: Define tables/columns with `encryptedTable` and the `types.*` concrete-domain factories from `@cipherstash/stack/eql/v3` (`types.TextSearch`, `types.IntegerOrd`, `types.Json`, …) — each domain's query capabilities are fixed by its type; there are no chainable capability tuners. Build the client with `EncryptionV3` from `@cipherstash/stack/v3`. (Legacy EQL v2 — `Encryption` + `encryptedColumn(...).equality().freeTextSearch().orderAndRange().searchableJson()` from `@cipherstash/stack/schema` — remains for existing deployments. DynamoDB accepts both v3 and v2 tables; nested fields work in both — v2 via `encryptedField` groups, v3 via a flat dotted column path (`'profile.ssn': types.TextEq(...)`), since `EncryptedV3TableColumn` admits no nested groups.) - **Operations** (all return Result-like objects and support chaining `.withLockContext(lockContext)` and `.audit()` when applicable): - `encrypt(plaintext, { table, column })` - `decrypt(encryptedPayload)` diff --git a/packages/stack/__tests__/dynamodb/client-compat.test-d.ts b/packages/stack/__tests__/dynamodb/client-compat.test-d.ts new file mode 100644 index 000000000..2c322357a --- /dev/null +++ b/packages/stack/__tests__/dynamodb/client-compat.test-d.ts @@ -0,0 +1,373 @@ +/** + * Type-level contract for `encryptedDynamoDB` (#657). + * + * The runtime `*.test.ts` suites are NOT typechecked (see the comment in + * `vitest.config.ts`), which is exactly how an M1-class regression slips + * through: `encryptedDynamoDB` rejecting the `TypedEncryptionClient` that + * `EncryptionV3` returns compiles nowhere and fails nothing, because the live + * suite that would have caught it is only ever *executed*. + * + * So the claims the adapter's own docs make — both client shapes accepted with + * no cast, both wire versions accepted on all four methods, `.audit()` chains, + * awaiting yields a discriminated Result — are locked here, where `tsc` runs. + */ +import { describe, expectTypeOf, it } from 'vitest' +import type { EncryptedAttributes, EncryptedDynamoDBInstance } from '@/dynamodb' +import { encryptedDynamoDB } from '@/dynamodb' +import type { EncryptionClient } from '@/encryption' +import type { EncryptionV3 } from '@/encryption/v3' +import { encryptedTable as encryptedTableV3, types } from '@/eql/v3' +import { encryptedColumn, encryptedTable as encryptedTableV2 } from '@/schema' + +const usersV3 = encryptedTableV3('users_v3', { + email: types.TextEq('email'), // equality → __source + __hmac + name: types.Text('name'), // storage only → __source + age: types.IntegerOrd('age'), // ordering, no HMAC → __source + meta: types.Json('meta'), // ste_vec array → __source +}) + +const usersV2 = encryptedTableV2('users_v2', { + email: encryptedColumn('email').equality(), +}) + +// Exercises the INNERMOST `true` arm of `HasSearchTerm` (types.ts): a text +// domain that is BOTH equality- and order/range-capable — text equality is +// always HMAC-based, so it mints `__hmac`. `usersV3.age` (IntegerOrd) reaches +// the same conditional but over `number`, taking the `false` arm; without a +// text-ordering column here that `true` arm had no type-level witness. +const searchV3 = encryptedTableV3('search_v3', { + title: types.TextOrd('title'), // equality + orderAndRange over string → __source + __hmac + bio: types.TextMatch('bio'), // free-text only, no equality → __source, NO __hmac +}) + +type V3Model = { pk: string; email?: string; age?: number; role?: string } + +// The two client shapes. `EncryptionV3` returns a `TypedEncryptionClient` +// parameterised by its own schema tuple; `Encryption` returns the nominal +// `EncryptionClient`. Both must be accepted by the factory WITHOUT a cast. +declare const typedClient: Awaited< + ReturnType> +> +declare const nominalClient: EncryptionClient + +describe('encryptedDynamoDB accepts both client shapes without a cast', () => { + it('accepts the typed client from EncryptionV3', () => { + expectTypeOf(encryptedDynamoDB).toBeCallableWith({ + encryptionClient: typedClient, + }) + expectTypeOf( + encryptedDynamoDB({ encryptionClient: typedClient }), + ).toEqualTypeOf() + }) + + it('accepts the nominal EncryptionClient', () => { + expectTypeOf(encryptedDynamoDB).toBeCallableWith({ + encryptionClient: nominalClient, + }) + expectTypeOf( + encryptedDynamoDB({ encryptionClient: nominalClient }), + ).toEqualTypeOf() + }) + + it('rejects a bare object that is not a client', () => { + encryptedDynamoDB({ + // @ts-expect-error - not an encryption client + encryptionClient: { nope: true }, + }) + }) +}) + +const dynamo = encryptedDynamoDB({ encryptionClient: typedClient }) + +describe('all four methods accept both a v3 and a v2 table', () => { + it('encryptModel', () => { + expectTypeOf(dynamo.encryptModel).toBeCallableWith( + { pk: 'a', email: 'a@b.com' }, + usersV3, + ) + expectTypeOf(dynamo.encryptModel).toBeCallableWith( + { pk: 'a', email: 'a@b.com' }, + usersV2, + ) + }) + + it('bulkEncryptModels', () => { + expectTypeOf(dynamo.bulkEncryptModels).toBeCallableWith( + [{ pk: 'a', email: 'a@b.com' }], + usersV3, + ) + expectTypeOf(dynamo.bulkEncryptModels).toBeCallableWith( + [{ pk: 'a', email: 'a@b.com' }], + usersV2, + ) + }) + + it('decryptModel', () => { + expectTypeOf(dynamo.decryptModel).toBeCallableWith( + { pk: 'a', email__source: 'ct' }, + usersV3, + ) + expectTypeOf(dynamo.decryptModel).toBeCallableWith( + { pk: 'a', email__source: 'ct' }, + usersV2, + ) + }) + + it('bulkDecryptModels', () => { + expectTypeOf(dynamo.bulkDecryptModels).toBeCallableWith( + [{ pk: 'a', email__source: 'ct' }], + usersV3, + ) + expectTypeOf(dynamo.bulkDecryptModels).toBeCallableWith( + [{ pk: 'a', email__source: 'ct' }], + usersV2, + ) + }) + + it('rejects a bare object as a table', () => { + dynamo.encryptModel( + { pk: 'a' }, + // @ts-expect-error - a plain object is not an encrypted table + { email: 'not-a-table' }, + ) + }) +}) + +describe('the v3 overload types the DynamoDB storage split', () => { + it('replaces a declared column with __source, and mints __hmac only for equality', async () => { + const result = await dynamo.encryptModel( + { pk: 'a', email: 'a@b.com', age: 3, role: 'admin' }, + usersV3, + ) + if (result.failure) throw new Error(result.failure.message) + + // Equality domain: ciphertext + the queryable HMAC term. + expectTypeOf(result.data.email__source).toEqualTypeOf() + expectTypeOf(result.data.email__hmac).toEqualTypeOf() + // Ordering domain: ciphertext only — `op` has no DynamoDB query surface. + expectTypeOf(result.data.age__source).toEqualTypeOf() + expectTypeOf(result.data).not.toHaveProperty('age__hmac') + // The plaintext key is GONE — typing it as present is the defect this fixes. + expectTypeOf(result.data).not.toHaveProperty('email') + // Non-schema keys pass through untouched (pk / sk / GSI attributes). + expectTypeOf(result.data.pk).toEqualTypeOf() + expectTypeOf(result.data.role).toEqualTypeOf() + }) + + it('types a JSON document as its stored ste_vec entries plus KeyHeader', async () => { + const result = await dynamo.encryptModel( + { pk: 'a', meta: { a: 1 } }, + usersV3, + ) + if (result.failure) throw new Error(result.failure.message) + + // A SteVec document is stored as `{ h, sv }` — the entries plus the + // per-document KeyHeader `h` that protect-ffi 0.30 decrypt requires. + expectTypeOf(result.data.meta__source).toEqualTypeOf<{ + h: unknown + sv: unknown[] + }>() + expectTypeOf(result.data).not.toHaveProperty('meta__hmac') + }) + + it('applies the same storage split to the bulk return type', async () => { + const result = await dynamo.bulkEncryptModels( + [{ pk: 'a', email: 'a@b.com', age: 3 }], + usersV3, + ) + if (result.failure) throw new Error(result.failure.message) + + expectTypeOf(result.data[0].email__source).toEqualTypeOf() + expectTypeOf(result.data[0].email__hmac).toEqualTypeOf() + expectTypeOf(result.data[0].age__source).toEqualTypeOf() + expectTypeOf(result.data[0]).not.toHaveProperty('email') + }) + + it('folds bulk stored attributes back to plaintext on decrypt', async () => { + const result = await dynamo.bulkDecryptModels( + [{ pk: 'a', email__source: 'ct', email__hmac: 'hm' }], + usersV3, + ) + if (result.failure) throw new Error(result.failure.message) + + expectTypeOf(result.data[0].email).toEqualTypeOf() + expectTypeOf(result.data[0]).not.toHaveProperty('email__source') + }) + + it('rejects a field whose type does not match its column domain', () => { + // @ts-expect-error - `email` is a text domain, not a number + dynamo.encryptModel({ pk: 'a', email: 42 }, usersV3) + // @ts-expect-error - `age` is an integer domain, not a string + dynamo.encryptModel({ pk: 'a', age: 'not-a-number' }, usersV3) + }) + + it('folds the stored attributes back to plaintext on decrypt', async () => { + const result = await dynamo.decryptModel( + { pk: 'a', email__source: 'ct', email__hmac: 'hm', role: 'admin' }, + usersV3, + ) + if (result.failure) throw new Error(result.failure.message) + + expectTypeOf(result.data.email).toEqualTypeOf() + // The query term is not data — it does not survive the read. + expectTypeOf(result.data).not.toHaveProperty('email__hmac') + expectTypeOf(result.data).not.toHaveProperty('email__source') + expectTypeOf(result.data.pk).toEqualTypeOf() + expectTypeOf(result.data.role).toEqualTypeOf() + }) + + it('round-trips a declared model shape', async () => { + const model: V3Model = { pk: 'a', email: 'a@b.com', role: 'admin' } + const encrypted = await dynamo.encryptModel(model, usersV3) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + expectTypeOf(dynamo.decryptModel).toBeCallableWith(encrypted.data, usersV3) + }) +}) + +describe('a text-ordering domain reaches the HasSearchTerm `true` arm', () => { + it('mints __hmac for a text equality+ordering column, but not for free-text-only', async () => { + const result = await dynamo.encryptModel( + { pk: 'a', title: 'Hello', bio: 'about me' }, + searchV3, + ) + if (result.failure) throw new Error(result.failure.message) + + // TextOrd is equality- AND order/range-capable over `string`: the innermost + // `[PlaintextForColumn] extends [string] ? true : false` resolves `true`, + // so the queryable HMAC term is present. This is the arm no prior column + // instantiated — `usersV3.age` (IntegerOrd) takes the `false` (number) arm. + expectTypeOf(result.data.title__source).toEqualTypeOf() + expectTypeOf(result.data.title__hmac).toEqualTypeOf() + + // A free-text-only domain (`TextMatch`) is NOT equality-capable, so it never + // reaches the ordering/text arms and writes no HMAC term — the distinct, + // adjacent behaviour that makes the `true` arm meaningful. + expectTypeOf(result.data.bio__source).toEqualTypeOf() + expectTypeOf(result.data).not.toHaveProperty('bio__hmac') + }) +}) + +describe('decrypt passes through suffixed keys whose base names no column', () => { + it('folds a declared column but leaves unrelated __hmac / __source keys intact', async () => { + // `email` IS a declared column; `legit` and `img` are NOT. A stored item can + // legitimately carry a customer attribute that merely ends in `__hmac` + // (an app-level signature) or `__source` (a renamed/foreign column) — the + // runtime read path preserves both, and these two `DecryptedAttributes` arms + // type that: a suffixed key whose base names no column is returned untouched. + const result = await dynamo.decryptModel( + { + pk: 'a', + email__source: 'ct', + email__hmac: 'hm', + legit__hmac: 'sig', + img__source: 'raw', + }, + usersV3, + ) + if (result.failure) throw new Error(result.failure.message) + + // Declared column: `email__source` folds to `email`, `email__hmac` (its base + // IS a column) is dropped as a query term. + expectTypeOf(result.data.email).toEqualTypeOf() + expectTypeOf(result.data).not.toHaveProperty('email__source') + expectTypeOf(result.data).not.toHaveProperty('email__hmac') + + // Non-column suffixed keys: neither folded nor dropped — the two "names no + // column" arms. `legit__hmac`'s base is not a column, so unlike `email__hmac` + // it survives; `img__source`'s base is not a column, so unlike `email__source` + // it is NOT folded to `img`. + expectTypeOf(result.data.legit__hmac).toEqualTypeOf() + expectTypeOf(result.data.img__source).toEqualTypeOf() + expectTypeOf(result.data).not.toHaveProperty('img') + + // Ordinary passthrough attribute, for contrast. + expectTypeOf(result.data.pk).toEqualTypeOf() + }) +}) + +describe('a dotted-path v3 column is not modelled in EncryptedAttributes', () => { + it('passes the parent key through untyped and mints no split attribute', () => { + const dotted = encryptedTableV3('dotted_v3', { + 'profile.ssn': types.TextEq('profile.ssn'), + }) + type Model = { pk: string; profile: { ssn: string; name: string } } + type Mapped = EncryptedAttributes + + // LIMITATION pinned (see the `EncryptedAttributes` docblock in types.ts): a + // column declared under a dotted path is split INSIDE the nested `profile` + // map at runtime, but the model key is `profile`, not `profile.ssn`. The + // mapped type does not recognise `profile` as a column, so it passes through + // with its input type unchanged and no `__source`/`__hmac` term is + // minted at the type level. If a future change starts (or stops) modelling + // the nested split, this assertion breaks on purpose. + expectTypeOf().toEqualTypeOf<{ + ssn: string + name: string + }>() + expectTypeOf().not.toHaveProperty('profile.ssn__source') + expectTypeOf().not.toHaveProperty('profile__source') + expectTypeOf().toEqualTypeOf() + }) +}) + +describe('a required-nullable v3 column keeps __source required', () => { + it('preserves the optional modifier only for optional columns', () => { + // `email` is required + nullable; `name` is optional. The `__source` half of + // `EncryptedAttributes` is a homomorphic key-remapped mapped type (types.ts), + // so it preserves the `?` modifier from the model key: a required-nullable + // column stays required, an optional column stays optional. The `| null` of + // the plaintext is dropped — `SourceAttribute` returns a flat `string` — so a + // null value writing no attribute at runtime is a deliberate, pinned mismatch + // with the optimistic type. `__hmac` is always optional regardless. + type Model = { pk: string; email: string | null; name?: string } + type Mapped = EncryptedAttributes + + expectTypeOf().toEqualTypeOf<{ + pk: string + email__source: string + email__hmac?: string + name__source?: string + }>() + }) +}) + +describe('the v2 overload still returns the input model', () => { + it('keeps an existing v2 caller compiling unchanged', async () => { + const result = await dynamo.encryptModel<{ pk: string; email?: string }>( + { pk: 'a', email: 'a@b.com' }, + usersV2, + ) + if (result.failure) throw new Error(result.failure.message) + + expectTypeOf(result.data).toEqualTypeOf<{ pk: string; email?: string }>() + }) +}) + +describe('operations chain and resolve', () => { + it('.audit() returns the operation so it stays chainable', () => { + const op = dynamo.encryptModel({ pk: 'a', email: 'a@b.com' }, usersV3) + expectTypeOf(op.audit({ metadata: { sub: 'u1' } })).toEqualTypeOf< + typeof op + >() + }) + + it('awaiting yields a discriminated Result', async () => { + const result = await dynamo.encryptModel( + { pk: 'a', email: 'a@b.com' }, + usersV3, + ) + + if (result.failure) { + expectTypeOf(result.failure.message).toEqualTypeOf() + // The success arm is not reachable in this branch — `data` is not even a + // property of the failure member, which is the discrimination working. + expectTypeOf(result).not.toHaveProperty('data') + return + } + + // ...and the failure arm is unreachable in this one. + expectTypeOf(result.failure).toEqualTypeOf() + expectTypeOf(result.data.email__source).toEqualTypeOf() + }) +}) diff --git a/packages/stack/__tests__/dynamodb/construct-guard.test.ts b/packages/stack/__tests__/dynamodb/construct-guard.test.ts new file mode 100644 index 000000000..88c95e997 --- /dev/null +++ b/packages/stack/__tests__/dynamodb/construct-guard.test.ts @@ -0,0 +1,126 @@ +/** + * Construction-time client/table EQL-version guard (#725, thread 13). + * + * A v3 table handed to a client that was NOT built in EQL v3 mode for it (a + * v2-mode client, or one initialised for a different schema set) otherwise + * fails only much later, deep inside the FFI, with an opaque deserialization + * error. `encryptedDynamoDB`'s operation methods guard against that mismatch + * the moment a table is supplied — the earliest point at which BOTH the client + * and the table version are known — and throw a clear, actionable Error. + * + * This suite is pure: it drives the guard with stub clients whose + * `getEncryptConfig()` reports which tables the client knows about, so it needs + * neither credentials nor a live FFI round-trip. The stub operation methods + * throw if ever reached, proving the guard fires BEFORE any client call. + */ +import { describe, expect, it } from 'vitest' +import { encryptedDynamoDB } from '@/dynamodb' +import { encryptedTable as encryptedTableV3, types } from '@/eql/v3' +import { encryptedColumn, encryptedTable as encryptedTableV2 } from '@/schema' + +const usersV3 = encryptedTableV3('users_v3', { + email: types.TextEq('email'), +}) + +const usersV2 = encryptedTableV2('users_v2', { + email: encryptedColumn('email').equality(), +}) + +/** + * A stub client that reports the given table names in its encrypt config. The + * operation methods throw if invoked, so any test that reaches them (rather than + * the synchronous guard) fails loudly. + */ +function stubClient(knownTables: string[]) { + const tables = Object.fromEntries(knownTables.map((t) => [t, {}])) + const unreached = () => { + throw new Error('FFI operation reached — guard did not fire first') + } + return { + getEncryptConfig: () => ({ v: 1, tables }), + encryptModel: unreached, + bulkEncryptModels: unreached, + decryptModel: unreached, + bulkDecryptModels: unreached, + // biome-ignore lint/suspicious/noExplicitAny: deliberately permissive test-stub client + } as any +} + +describe('encryptedDynamoDB client/table version guard', () => { + it('throws a clear version-mismatch error for a v3 table on a client that does not know it', () => { + // A client built for other (e.g. v2) schemas: `users_v3` is absent. + const dynamo = encryptedDynamoDB({ encryptionClient: stubClient([]) }) + + expect(() => + dynamo.encryptModel({ pk: 'a', email: 'a@b.com' }, usersV3), + ).toThrowError(/version mismatch/i) + expect(() => + dynamo.encryptModel({ pk: 'a', email: 'a@b.com' }, usersV3), + ).toThrowError(/v3/) + }) + + it('names the offending table and how to fix it', () => { + const dynamo = encryptedDynamoDB({ encryptionClient: stubClient([]) }) + let message = '' + try { + dynamo.encryptModel({ pk: 'a', email: 'a@b.com' }, usersV3) + } catch (e) { + message = (e as Error).message + } + expect(message).toContain('users_v3') + expect(message).toMatch(/EncryptionV3|eqlVersion/) + }) + + it('guards every operation method, not just encryptModel', () => { + const dynamo = encryptedDynamoDB({ encryptionClient: stubClient([]) }) + expect(() => + dynamo.bulkEncryptModels([{ pk: 'a', email: 'a@b.com' }], usersV3), + ).toThrowError(/version mismatch/i) + expect(() => + dynamo.decryptModel({ pk: 'a', email__source: 'ct' }, usersV3), + ).toThrowError(/version mismatch/i) + expect(() => + dynamo.bulkDecryptModels([{ pk: 'a', email__source: 'ct' }], usersV3), + ).toThrowError(/version mismatch/i) + }) + + it('does NOT throw when a v3 table is registered with the client (matching versions)', () => { + const dynamo = encryptedDynamoDB({ + encryptionClient: stubClient(['users_v3']), + }) + // Constructing the operation must not throw; the stub op methods are never + // awaited, so the "unreached" guard inside them is not tripped. + expect(() => + dynamo.encryptModel({ pk: 'a', email: 'a@b.com' }, usersV3), + ).not.toThrow() + }) + + it('does NOT throw for a v2 table on a v2-mode client (matching versions)', () => { + const dynamo = encryptedDynamoDB({ + encryptionClient: stubClient(['users_v2']), + }) + expect(() => + dynamo.encryptModel({ pk: 'a', email: 'a@b.com' }, usersV2), + ).not.toThrow() + }) + + it('does NOT throw a v3 table on a client whose config cannot be read (cannot determine version)', () => { + // A client shape with no getEncryptConfig — the guard cannot prove a + // mismatch, so it must not false-positive. + const unreached = () => { + throw new Error('FFI operation reached — guard did not fire first') + } + const dynamo = encryptedDynamoDB({ + encryptionClient: { + encryptModel: unreached, + bulkEncryptModels: unreached, + decryptModel: unreached, + bulkDecryptModels: unreached, + // biome-ignore lint/suspicious/noExplicitAny: deliberately permissive test-stub client + } as any, + }) + expect(() => + dynamo.encryptModel({ pk: 'a', email: 'a@b.com' }, usersV3), + ).not.toThrow() + }) +}) diff --git a/packages/stack/__tests__/dynamodb/encrypted-dynamodb-v3.test.ts b/packages/stack/__tests__/dynamodb/encrypted-dynamodb-v3.test.ts new file mode 100644 index 000000000..c32b41f10 --- /dev/null +++ b/packages/stack/__tests__/dynamodb/encrypted-dynamodb-v3.test.ts @@ -0,0 +1,712 @@ +/** + * Live end-to-end tests for `encryptedDynamoDB` against EQL v3 tables (#657). + * + * Mirrors the v2 suite in `encrypted-dynamodb.test.ts` and adds the two things + * only v3 raises: + * + * - both client shapes must work. `EncryptionV3` returns a `TypedEncryptionClient` + * whose `decryptModel` is a plain `Promise>` taking the table as a + * second argument; `Encryption({ config: { eqlVersion: 3 } })` returns the + * nominal chainable client. Audit metadata on decrypt only has somewhere to + * go on the latter. + * - per-domain storage: only equality domains mint the `hm` term that backs a + * DynamoDB key condition. Ordering and free-text terms are not stored. + * + * Requires CipherStash credentials (`CS_*`), like the rest of this suite. + */ +import 'dotenv/config' +import { requireIntegrationEnv } from '@cipherstash/test-kit' +import fc from 'fast-check' +import { beforeAll, describe, expect, it } from 'vitest' +import { encryptedDynamoDB } from '@/dynamodb' +import { toItemWithEqlPayloads } from '@/dynamodb/helpers' +import type { EncryptedDynamoDBInstance } from '@/dynamodb/types' +import type { EncryptionClient } from '@/encryption' +import { EncryptionV3 } from '@/encryption/v3' +import type { JsonValue } from '@/eql/v3' +import { encryptedTable, types } from '@/eql/v3' +import { Encryption } from '@/index' + +/** + * Every block below is a real ZeroKMS round trip. Under the default forks pool + * with file parallelism, many live suites hit ZeroKMS at once and a request can + * be transiently throttled — the test then reds even though the code is correct + * (it passes in isolation). Retry the live suites a couple of times so transient + * throttling doesn't flake CI. Scoped here rather than in `vitest.config.ts` so + * the pure/unit tests are never retried (a retry that hides a real pure-test bug + * is worse than the flake it would paper over). + */ +const liveSuiteOptions = { retry: 2 } as const + +// Note: no `skipIf` gate — the suite requires credentials and fails without them +// (see `beforeAll`), matching the package convention. `liveSuiteOptions` only +// carries the throttling retry. + +const users = encryptedTable('users_v3_dynamo', { + email: types.TextEq('email'), + name: types.Text('name'), + age: types.IntegerOrd('age'), + bio: types.TextSearch('bio'), + meta: types.Json('meta'), +}) + +type User = { + pk: string + email?: string + name?: string + age?: number + bio?: string + // `Record` would not compile against a `types.Json` column: + // `unknown` is not a `JsonValue`. Naming the real element type is the point of + // the v3 overload — it catches a model that claims to hold non-JSON. + meta?: Record + role?: string +} + +/** The typed client from `EncryptionV3` — the documented v3 entry point. */ +let typedDynamo: EncryptedDynamoDBInstance +/** The nominal chainable client, forced into v3 mode. */ +let nominalClient: EncryptionClient +let nominalDynamo: EncryptedDynamoDBInstance + +beforeAll(async () => { + // Fail loudly, never skip, when credentials are absent: a skipped live suite + // is a green run that proves nothing and hides regressions. `requireIntegrationEnv` + // throws naming exactly which of the four CS_* vars are missing — consistent + // with every other live suite in this package. + requireIntegrationEnv(['cipherstash']) + + const typedClient = await EncryptionV3({ schemas: [users] }) + typedDynamo = encryptedDynamoDB({ encryptionClient: typedClient }) + + nominalClient = await Encryption({ + schemas: [users] as never, + config: { eqlVersion: 3 }, + }) + nominalDynamo = encryptedDynamoDB({ encryptionClient: nominalClient }) +}) + +describe('encryptModel with a v3 table', liveSuiteOptions, () => { + it('splits an equality domain into __source and __hmac', async () => { + const result = await typedDynamo.encryptModel( + { pk: 'user#1', email: 'alice@example.com', role: 'admin' }, + users, + ) + + if (result.failure) throw new Error(result.failure.message) + + expect(Object.keys(result.data).sort()).toEqual([ + 'email__hmac', + 'email__source', + 'pk', + 'role', + ]) + expect(typeof result.data.email__source).toBe('string') + expect(typeof result.data.email__hmac).toBe('string') + expect(result.data.email__source).not.toContain('alice@example.com') + }) + + it('regression: a v3 scalar is split, not written out as a raw map', async () => { + const result = await typedDynamo.encryptModel( + { pk: 'user#2', name: 'Bob' }, + users, + ) + + if (result.failure) throw new Error(result.failure.message) + + // Before the port, gating on `k === 'ct'` meant untagged v3 scalars fell + // through to the nested-object branch and were stored as `{ name: { v, i, + // c } }` — no __source, and the ciphertext nested one level down. + expect(result.data).toHaveProperty('name__source') + expect(result.data).not.toHaveProperty('name') + expect(typeof result.data.name__source).toBe('string') + }) + + it('gives an ordering domain a __source but no __hmac', async () => { + const result = await typedDynamo.encryptModel( + { pk: 'user#3', age: 42 }, + users, + ) + + if (result.failure) throw new Error(result.failure.message) + + // `IntegerOrd` is equality-capable in Postgres via its ordering term, but + // that term has no DynamoDB query surface, so nothing is stored for it. + expect(result.data).toHaveProperty('age__source') + expect(result.data).not.toHaveProperty('age__hmac') + }) + + it('keeps only the equality term of a TextSearch domain', async () => { + const result = await typedDynamo.encryptModel( + { pk: 'user#4', bio: 'a long biography' }, + users, + ) + + if (result.failure) throw new Error(result.failure.message) + + // TextSearch mints hm + op + bf; only hm is storable as an attribute. + expect(Object.keys(result.data).sort()).toEqual([ + 'bio__hmac', + 'bio__source', + 'pk', + ]) + }) + + it('stores a Json domain as its sv entries plus the KeyHeader', async () => { + const result = await typedDynamo.encryptModel( + { pk: 'user#5', meta: { a: 1, b: { c: 'deep' } } }, + users, + ) + + if (result.failure) throw new Error(result.failure.message) + + // A SteVec document is stored as `{ h, sv }` under __source: the `sv` + // entries plus the per-document KeyHeader `h` that protect-ffi 0.30 decrypt + // requires. `v`/`i`/`k` are reconstructed on read, so they are not stored. + const stored = result.data.meta__source as { h: unknown; sv: unknown[] } + expect(stored.h).toBeDefined() + expect(Array.isArray(stored.sv)).toBe(true) + expect(stored.sv.length).toBeGreaterThan(0) + }) + + it('passes a null column value through without splitting it', async () => { + // v3 reaches the null check through a different code path than v2: + // `isStoredEqlPayload` runs before the `k`/`c` gates, so a null must be + // recognised as a non-payload and passed through verbatim — no + // `email__source`, no `email__hmac` — with siblings intact. + const result = await typedDynamo.encryptModel( + { pk: 'user#null', email: null, role: 'admin' }, + users, + ) + + if (result.failure) throw new Error(result.failure.message) + + expect(result.data.email).toBeNull() + expect(result.data).not.toHaveProperty('email__source') + expect(result.data).not.toHaveProperty('email__hmac') + expect(result.data.pk).toBe('user#null') + expect(result.data.role).toBe('admin') + + const decrypted = await typedDynamo.decryptModel(result.data, users) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual({ + pk: 'user#null', + email: null, + role: 'admin', + }) + }) +}) + +describe('round trips with a v3 table', liveSuiteOptions, () => { + it('round-trips every domain back to the original item', async () => { + const original: User = { + pk: 'user#6', + email: 'erin@example.com', + name: 'Erin', + age: 42, + bio: 'a long biography', + meta: { a: 1, b: { c: 'deep' } }, + role: 'admin', + } + + const encrypted = await typedDynamo.encryptModel(original, users) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const decrypted = await typedDynamo.decryptModel(encrypted.data, users) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(original) + }) + + it('round-trips the same item through the nominal client', async () => { + const original: User = { + pk: 'user#7', + email: 'frank@example.com', + age: 7, + meta: { z: 1 }, + } + + const encrypted = await nominalDynamo.encryptModel(original, users) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const decrypted = await nominalDynamo.decryptModel(encrypted.data, users) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(original) + }) + + it('produces items either client can decrypt — the wire format is the same', async () => { + const original: User = { pk: 'user#8', email: 'grace@example.com' } + + const encrypted = await typedDynamo.encryptModel(original, users) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const decrypted = await nominalDynamo.decryptModel(encrypted.data, users) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(original) + }) + + /** + * The one LIVE property. Its pure siblings (`properties.test.ts`) cover the + * attribute mapping in both directions with a fake ciphertext; this closes the + * loop over the real thing — every run is a ZeroKMS encrypt + decrypt, so it + * is capped hard at 5 runs. It is the only check that the split/rebuild + * survives contact with actual ciphertext across all five domains at once. + */ + it('property: round-trips arbitrary multi-domain items through ZeroKMS', async () => { + const jsonLeaf = fc.oneof(fc.string(), fc.integer(), fc.boolean()) + const userArb = fc.record({ + pk: fc.string({ minLength: 1 }), + email: fc.string(), + name: fc.string(), + age: fc.integer(), + bio: fc.string(), + meta: fc.dictionary( + fc.string({ minLength: 1 }), + fc.oneof( + jsonLeaf, + fc.dictionary(fc.string({ minLength: 1 }), jsonLeaf, { + maxKeys: 2, + }), + ), + { minKeys: 1, maxKeys: 3 }, + ), + role: fc.string(), + }) + + await fc.assert( + fc.asyncProperty(userArb, async (original) => { + const encrypted = await typedDynamo.encryptModel(original, users) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const decrypted = await typedDynamo.decryptModel(encrypted.data, users) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(original) + }), + // Deliberately small: each run is a real ZeroKMS round trip. The pure + // properties in `properties.test.ts` cover the attribute mapping + // exhaustively and for free; this one only needs to establish that the + // mapping composed with real encryption is identity-preserving over + // arbitrary items, which a handful of runs does. + { numRuns: 5 }, + ) + }, 120000) +}) + +describe('bulk operations with a v3 table', liveSuiteOptions, () => { + it('encrypts and decrypts a batch', async () => { + const items: User[] = [ + { pk: 'user#9', email: 'a@example.com', name: 'A' }, + { pk: 'user#10', email: 'b@example.com', age: 3 }, + ] + + const encrypted = await typedDynamo.bulkEncryptModels(items, users) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + expect(encrypted.data).toHaveLength(2) + for (const item of encrypted.data) { + expect(item).toHaveProperty('email__source') + expect(item).toHaveProperty('email__hmac') + } + + const decrypted = await typedDynamo.bulkDecryptModels(encrypted.data, users) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(items) + }) + + it('round-trips a batch through the nominal client', async () => { + const items: User[] = [ + { pk: 'user#11', email: 'c@example.com' }, + { pk: 'user#12', name: 'D' }, + ] + + const encrypted = await nominalDynamo.bulkEncryptModels(items, users) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const decrypted = await nominalDynamo.bulkDecryptModels( + encrypted.data, + users, + ) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(items) + }) + + it('accepts an empty batch', async () => { + const encrypted = await typedDynamo.bulkEncryptModels([], users) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + expect(encrypted.data).toEqual([]) + }) +}) + +describe('nested attributes with a v3 table', liveSuiteOptions, () => { + // EQL v3 has no nested-object *authoring* form — a nested group is a compile + // error. But a dotted column path is a supported column name, and the model + // walk in `encryption/helpers/model-helpers.ts` is shared with v2 and matches + // on dotted paths. So nested DynamoDB attributes work in v3 today, declared + // flat. This matters for DynamoDB specifically, where items are natively + // nested documents. + const nested = encryptedTable('users_v3_nested', { + 'profile.ssn': types.TextEq('profile.ssn'), + 'profile.note': types.Text('profile.note'), + }) + + type NestedUser = { + pk: string + profile?: { ssn?: string; note?: string; city?: string } + } + + let nestedDynamo: EncryptedDynamoDBInstance + let nestedClient: EncryptionClient + + beforeAll(async () => { + nestedClient = await Encryption({ + schemas: [nested] as never, + config: { eqlVersion: 3 }, + }) + nestedDynamo = encryptedDynamoDB({ encryptionClient: nestedClient }) + }) + + it('splits nested attributes in place, leaving plaintext siblings alone', async () => { + const result = await nestedDynamo.encryptModel( + { + pk: 'u#1', + profile: { ssn: '123-45-6789', note: 'hi', city: 'Sydney' }, + }, + nested, + ) + + if (result.failure) throw new Error(result.failure.message) + + const profile = result.data.profile as Record + expect(Object.keys(profile).sort()).toEqual([ + 'city', + 'note__source', + 'ssn__hmac', + 'ssn__source', + ]) + expect(profile.city).toBe('Sydney') + }) + + it('round-trips a nested item exactly', async () => { + const original: NestedUser = { + pk: 'u#2', + profile: { ssn: '123-45-6789', note: 'hi', city: 'Sydney' }, + } + + const encrypted = await nestedDynamo.encryptModel(original, nested) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const decrypted = await nestedDynamo.decryptModel(encrypted.data, nested) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(original) + }) + + it('rebuilds the nested identifier from its registered dotted name', async () => { + const encrypted = await nestedDynamo.encryptModel( + { pk: 'u#3', profile: { ssn: '123-45-6789' } }, + nested, + ) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const rebuilt = toItemWithEqlPayloads(encrypted.data, nested) + const payload = (rebuilt.profile as Record) + .ssn + + expect(payload.i.c).toBe('profile.ssn') + }) + + it('makes a nested equality attribute queryable by __hmac', async () => { + const ssn = '999-88-7777' + + const stored = await nestedDynamo.encryptModel( + { pk: 'u#4', profile: { ssn } }, + nested, + ) + if (stored.failure) throw new Error(stored.failure.message) + + const term = await nestedClient.encryptQuery(ssn, { + table: nested as never, + column: nested['profile.ssn'] as never, + }) + if (term.failure) throw new Error(term.failure.message) + + const profile = stored.data.profile as Record + // The query term matches the stored nested `profile.ssn__hmac` — usable in a + // FilterExpression (a nested attribute can't back a DynamoDB key condition). + expect((term.data as { hm: string }).hm).toBe(profile.ssn__hmac) + }) + + it('bulk round-trips nested items', async () => { + const items: NestedUser[] = [ + { pk: 'u#5', profile: { ssn: 'a', city: 'Sydney' } }, + { pk: 'u#6', profile: { note: 'b' } }, + ] + + const encrypted = await nestedDynamo.bulkEncryptModels(items, nested) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const decrypted = await nestedDynamo.bulkDecryptModels( + encrypted.data, + nested, + ) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(items) + }) +}) + +describe( + 'a v3 column whose property differs from its DB name', + liveSuiteOptions, + () => { + // Regression: `encryptedAttrs` was derived from `build().columns`, which for + // v3 is keyed by DB name, while the encrypted model is keyed by property + // name. They never matched, so the attribute was never split — no __source, + // no __hmac, and the payload's `i` block was mangled to `i__source`. Decrypt + // still round-tripped, so nothing surfaced it; only a key condition that + // silently matched nothing would have. + const renamed = encryptedTable('users_v3_renamed', { + emailAddress: types.TextEq('email_address'), + }) + + let renamedDynamo: EncryptedDynamoDBInstance + let renamedClient: EncryptionClient + + beforeAll(async () => { + renamedClient = await Encryption({ + schemas: [renamed] as never, + config: { eqlVersion: 3 }, + }) + renamedDynamo = encryptedDynamoDB({ encryptionClient: renamedClient }) + }) + + it('splits the attribute under its property name, with an __hmac', async () => { + const result = await renamedDynamo.encryptModel( + { id: '1', emailAddress: 'a@b.com' }, + renamed, + ) + if (result.failure) throw new Error(result.failure.message) + + expect(Object.keys(result.data).sort()).toEqual([ + 'emailAddress__hmac', + 'emailAddress__source', + 'id', + ]) + // The payload's identifier block must never leak into an attribute. + expect(JSON.stringify(result.data)).not.toContain('i__source') + }) + + it('round-trips, and the query term matches the stored __hmac', async () => { + const original = { id: '2', emailAddress: 'c@d.com' } + + const encrypted = await renamedDynamo.encryptModel(original, renamed) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const decrypted = await renamedDynamo.decryptModel( + encrypted.data, + renamed, + ) + if (decrypted.failure) throw new Error(decrypted.failure.message) + expect(decrypted.data).toEqual(original) + + const term = await renamedClient.encryptQuery('c@d.com', { + table: renamed as never, + column: renamed.emailAddress as never, + }) + if (term.failure) throw new Error(term.failure.message) + + expect((term.data as { hm: string }).hm).toBe( + (encrypted.data as Record).emailAddress__hmac, + ) + }) + + it('splits each item under its property name on the bulk path, and round-trips', async () => { + // Guards the SAME regression on `bulkEncryptModels`: its `encryptedAttrs` + // must come from `resolveEncryptColumnMap` (property names), not + // `build().columns` (DB names). With the DB-name form, every item would be + // stored under `email_address__*` — or not split at all — and no key + // condition on `emailAddress__hmac` would ever match. + const items = [ + { id: 'b1', emailAddress: 'e@f.com' }, + { id: 'b2', emailAddress: 'g@h.com' }, + ] + + const encrypted = await renamedDynamo.bulkEncryptModels(items, renamed) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + expect(encrypted.data).toHaveLength(2) + for (const item of encrypted.data) { + const stored = item as Record + expect(Object.keys(stored).sort()).toEqual([ + 'emailAddress__hmac', + 'emailAddress__source', + 'id', + ]) + // The DB-name form would have produced these instead of the property-name + // attributes above. + expect(stored).not.toHaveProperty('email_address__source') + expect(stored).not.toHaveProperty('email_address__hmac') + } + + const decrypted = await renamedDynamo.bulkDecryptModels( + encrypted.data, + renamed, + ) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(items) + }) + }, +) + +describe( + 'the __hmac key-condition path with a v3 table', + liveSuiteOptions, + () => { + it('mints, via encryptQuery, the same HMAC the item is stored under', async () => { + const email = 'heidi@example.com' + + const stored = await typedDynamo.encryptModel( + { pk: 'user#13', email }, + users, + ) + if (stored.failure) throw new Error(stored.failure.message) + + const term = await nominalClient.encryptQuery(email, { + table: users as never, + column: users.email as never, + }) + if (term.failure) throw new Error(term.failure.message) + + // `encryptQuery` on a v3 equality domain mints the bare term — `{v, i, hm}` + // with no ciphertext — so the key condition consumes `hm` directly. + expect(term.data).not.toHaveProperty('c') + expect((term.data as { hm: string }).hm).toBe(stored.data.email__hmac) + }) + + it('is deterministic across separate encryptions of the same value', async () => { + const email = 'ivan@example.com' + + const first = await typedDynamo.encryptModel({ pk: 'a', email }, users) + const second = await typedDynamo.encryptModel({ pk: 'b', email }, users) + if (first.failure) throw new Error(first.failure.message) + if (second.failure) throw new Error(second.failure.message) + + expect(first.data.email__hmac).toBe(second.data.email__hmac) + expect(first.data.email__source).not.toBe(second.data.email__source) + }) + }, +) + +describe('audit metadata with a v3 table', liveSuiteOptions, () => { + const metadata = { sub: 'user-id-123', action: 'v3-port' } + + it('is carried on every operation of the nominal client', async () => { + const item: User = { pk: 'user#14', email: 'judy@example.com' } + + const encrypted = await nominalDynamo + .encryptModel(item, users) + .audit({ metadata }) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const decrypted = await nominalDynamo + .decryptModel(encrypted.data, users) + .audit({ metadata }) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(item) + }) + + it('is accepted on the typed client, though decrypt cannot carry it', async () => { + const item: User = { pk: 'user#15', email: 'ken@example.com' } + + const encrypted = await typedDynamo + .encryptModel(item, users) + .audit({ metadata }) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + // The typed client's `decryptModel` returns a plain promise with no audit + // surface. The chain must still resolve correctly — the metadata is simply + // not forwarded. Use the nominal client if decrypt audit matters. + const decrypted = await typedDynamo + .decryptModel(encrypted.data, users) + .audit({ metadata }) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(item) + }) +}) + +describe('error handling with a v3 table', liveSuiteOptions, () => { + const unknown = encryptedTable('users_v3_dynamo', { + nope: types.TextEq('nonexistent_column'), + }) + + it('surfaces the FFI error code for an unregistered column', async () => { + const result = await typedDynamo.encryptModel({ nope: 'value' }, unknown) + + expect(result.failure).toBeDefined() + expect(result.failure?.code).toBe('UNKNOWN_COLUMN') + }) + + it('surfaces the FFI ciphertext error code for a malformed __source', async () => { + const result = await typedDynamo.decryptModel( + { email__source: 'not-a-ciphertext' }, + users, + ) + + expect(result.failure).toBeDefined() + expect(result.failure?.name).toBe('EncryptedDynamoDBError') + // Behaviour difference from v2, and an improvement: in v2 mode this path + // produced a bare "Unexpected end of input" with no code, so the adapter + // fell back to DYNAMODB_ENCRYPTION_ERROR (see the v2 suite). v3 rejects it + // as "Invalid EQL ciphertext" with a real FFI code, which is propagated. + expect(result.failure?.code).toBe('INVALID_CIPHERTEXT') + expect(result.failure?.details).toEqual({ context: 'decryptModel' }) + }) + + it('surfaces the FFI error code on the bulk encrypt path', async () => { + const result = await typedDynamo.bulkEncryptModels( + [{ nope: 'value' }], + unknown, + ) + + expect(result.failure).toBeDefined() + expect(result.failure?.code).toBe('UNKNOWN_COLUMN') + }) + + it('surfaces the FFI ciphertext error code on the bulk decrypt path', async () => { + // The v3 bulk decrypt failure path (resolveDecryptResult + + // throwPreservingCode against a typed client) is otherwise never exercised + // end-to-end. Like the single-item v3 path, a malformed __source is rejected + // as "Invalid EQL ciphertext" with a real FFI code — not the v2 fallback. + const result = await typedDynamo.bulkDecryptModels( + [{ email__source: 'not-a-ciphertext' }], + users, + ) + + expect(result.failure).toBeDefined() + expect(result.failure?.name).toBe('EncryptedDynamoDBError') + expect(result.failure?.code).toBe('INVALID_CIPHERTEXT') + expect(result.failure?.details).toEqual({ context: 'bulkDecryptModels' }) + }) + + it('routes v3 failures to the configured errorHandler', async () => { + const seen: string[] = [] + const instrumented = encryptedDynamoDB({ + encryptionClient: nominalClient, + options: { errorHandler: (e) => seen.push(e.code) }, + }) + + await instrumented.encryptModel({ nope: 'value' }, unknown) + + expect(seen).toEqual(['UNKNOWN_COLUMN']) + }) +}) diff --git a/packages/stack/__tests__/dynamodb/encrypted-dynamodb.test.ts b/packages/stack/__tests__/dynamodb/encrypted-dynamodb.test.ts new file mode 100644 index 000000000..cecad04bb --- /dev/null +++ b/packages/stack/__tests__/dynamodb/encrypted-dynamodb.test.ts @@ -0,0 +1,420 @@ +/** + * Characterisation tests for `encryptedDynamoDB` against a live client. + * + * These pin the CURRENT (EQL v2) end-to-end behaviour of the shipping DynamoDB + * adapter before the EQL v3 port (#657): what `encryptModel` puts on the wire, + * that `decryptModel` reverses it exactly, that the `__hmac` attribute an item + * is stored under is the same value `encryptQuery` mints for a key condition, + * and how failures surface. + * + * There is no DynamoDB in the loop — the adapter never touches the AWS SDK. It + * maps between EQL payloads and DynamoDB attribute names, so these tests assert + * on the attribute map that a caller would hand to `PutCommand`. + * + * Requires CipherStash credentials (`CS_*`), like the rest of this suite. + */ +import 'dotenv/config' +import { beforeAll, describe, expect, it } from 'vitest' +import { encryptedDynamoDB } from '@/dynamodb' +import type { EncryptedDynamoDBInstance } from '@/dynamodb/types' +import type { EncryptionClient } from '@/encryption' +import { Encryption } from '@/index' +import { encryptedColumn, encryptedField, encryptedTable } from '@/schema' + +const users = encryptedTable('users', { + email: encryptedColumn('email').equality(), + name: encryptedColumn('name'), + example: { + protected: encryptedField('example.protected'), + }, +}) + +type User = { + pk: string + email?: string | null + name?: string | null + role?: string + example?: { protected?: string | null; notProtected?: string } +} + +/** + * The attribute map `encryptModel` actually writes for the v2 `users` table. + * + * The v2 overload still returns the INPUT model type. Unlike v3, a v2 column + * does not carry its index configuration in the type, so the `__source` / + * `__hmac` split cannot be derived the way `EncryptedAttributes` derives it for + * a v3 table — see the note on `EncryptedDynamoDBInstance.encryptModel`. + * Naming the wire shape here, rather than widening `User` with an index + * signature, keeps these assertions honest about what is actually stored. + */ +type StoredUser = { + pk: string + role?: string + email__source?: string + email__hmac?: string + name__source?: string + example?: { protected__source?: string; notProtected?: string } +} + +/** Read a v2 encrypt result as the attribute map it really is. */ +const storedAttrs = (item: User): StoredUser => item as StoredUser + +let client: EncryptionClient +let dynamo: EncryptedDynamoDBInstance + +beforeAll(async () => { + client = await Encryption({ schemas: [users] }) + dynamo = encryptedDynamoDB({ encryptionClient: client }) +}) + +describe('encryptModel', () => { + it('splits equality columns into __source and __hmac, and leaves plaintext alone', async () => { + const result = await dynamo.encryptModel( + { + pk: 'user#1', + email: 'alice@example.com', + name: 'Alice', + role: 'admin', + }, + users, + ) + + if (result.failure) throw new Error(result.failure.message) + + expect(Object.keys(result.data).sort()).toEqual([ + 'email__hmac', + 'email__source', + 'name__source', + 'pk', + 'role', + ]) + expect(result.data.pk).toBe('user#1') + expect(result.data.role).toBe('admin') + expect(typeof storedAttrs(result.data).email__source).toBe('string') + expect(typeof storedAttrs(result.data).email__hmac).toBe('string') + // Neither stored attribute leaks the plaintext. + expect(storedAttrs(result.data).email__source).not.toContain( + 'alice@example.com', + ) + expect(storedAttrs(result.data).email__hmac).not.toContain( + 'alice@example.com', + ) + }) + + it('gives a non-equality column a __source but no __hmac', async () => { + const result = await dynamo.encryptModel( + { pk: 'user#2', name: 'Bob' }, + users, + ) + + if (result.failure) throw new Error(result.failure.message) + + expect(result.data).toHaveProperty('name__source') + expect(result.data).not.toHaveProperty('name__hmac') + }) + + it('encrypts a nested field in place, keeping siblings plaintext', async () => { + const result = await dynamo.encryptModel( + { + pk: 'user#4', + example: { protected: 'secret', notProtected: 'public' }, + }, + users, + ) + + if (result.failure) throw new Error(result.failure.message) + + expect(result.data.example).toEqual({ + protected__source: expect.any(String), + notProtected: 'public', + }) + }) + + it('passes null through without encrypting it', async () => { + const result = await dynamo.encryptModel( + { pk: 'user#5', email: null, name: 'Carol' }, + users, + ) + + if (result.failure) throw new Error(result.failure.message) + + expect(result.data.email).toBeNull() + expect(result.data).not.toHaveProperty('email__source') + }) + + it('does not mutate the caller’s input object', async () => { + const input: User = { pk: 'user#6', email: 'dave@example.com' } + const snapshot = structuredClone(input) + + await dynamo.encryptModel(input, users) + + expect(input).toEqual(snapshot) + }) +}) + +describe('decryptModel', () => { + it('round-trips every column shape back to the original item', async () => { + const original: User = { + pk: 'user#7', + email: 'erin@example.com', + name: 'Erin', + role: 'admin', + example: { protected: 'secret', notProtected: 'public' }, + } + + const encrypted = await dynamo.encryptModel(original, users) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const decrypted = await dynamo.decryptModel(encrypted.data, users) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(original) + }) + + it('tolerates the __hmac attribute being present on the stored item', async () => { + const encrypted = await dynamo.encryptModel( + { pk: 'user#9', email: 'frank@example.com' }, + users, + ) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + expect(encrypted.data).toHaveProperty('email__hmac') + + const decrypted = await dynamo.decryptModel(encrypted.data, users) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data.email).toBe('frank@example.com') + }) +}) + +describe('bulk operations', () => { + it('encrypts and decrypts a batch, preserving order and per-item shape', async () => { + const items: User[] = [ + { pk: 'user#10', email: 'a@example.com', name: 'A' }, + { pk: 'user#11', email: 'b@example.com', name: 'B', role: 'admin' }, + ] + + const encrypted = await dynamo.bulkEncryptModels(items, users) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + expect(encrypted.data).toHaveLength(2) + for (const item of encrypted.data) { + expect(item).toHaveProperty('email__source') + expect(item).toHaveProperty('email__hmac') + } + + const decrypted = await dynamo.bulkDecryptModels( + encrypted.data, + users, + ) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(items) + }) + + it('handles heterogeneous items in one batch', async () => { + const items: User[] = [ + { pk: 'user#12', email: 'c@example.com' }, + { pk: 'user#13', name: 'D', example: { protected: 'secret' } }, + ] + + const encrypted = await dynamo.bulkEncryptModels(items, users) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + const decrypted = await dynamo.bulkDecryptModels( + encrypted.data, + users, + ) + if (decrypted.failure) throw new Error(decrypted.failure.message) + + expect(decrypted.data).toEqual(items) + }) + + it('accepts an empty batch', async () => { + const encrypted = await dynamo.bulkEncryptModels([], users) + if (encrypted.failure) throw new Error(encrypted.failure.message) + + expect(encrypted.data).toEqual([]) + }) +}) + +describe('the __hmac key-condition path', () => { + it('mints, via encryptQuery, the same HMAC the item is stored under', async () => { + const email = 'grace@example.com' + + const stored = await dynamo.encryptModel( + { pk: 'user#14', email }, + users, + ) + if (stored.failure) throw new Error(stored.failure.message) + + const term = await client.encryptQuery(email, { + table: users, + column: users.email, + queryType: 'equality', + }) + if (term.failure) throw new Error(term.failure.message) + + // This equality is the whole DynamoDB query story: a caller puts + // `term.data.hm` into `KeyConditionExpression: "email__hmac = :e"` and it + // matches the attribute written at encrypt time. + expect((term.data as { hm: string }).hm).toBe( + storedAttrs(stored.data).email__hmac, + ) + }) + + it('mints a different HMAC for a different plaintext', async () => { + const term = await client.encryptQuery('someone-else@example.com', { + table: users, + column: users.email, + queryType: 'equality', + }) + if (term.failure) throw new Error(term.failure.message) + + const stored = await dynamo.encryptModel( + { pk: 'user#15', email: 'grace@example.com' }, + users, + ) + if (stored.failure) throw new Error(stored.failure.message) + + expect((term.data as { hm: string }).hm).not.toBe( + storedAttrs(stored.data).email__hmac, + ) + }) + + it('is deterministic across separate encryptions of the same value', async () => { + const email = 'heidi@example.com' + + const first = await dynamo.encryptModel({ pk: 'a', email }, users) + const second = await dynamo.encryptModel({ pk: 'b', email }, users) + if (first.failure) throw new Error(first.failure.message) + if (second.failure) throw new Error(second.failure.message) + + expect(storedAttrs(first.data).email__hmac).toBe( + storedAttrs(second.data).email__hmac, + ) + // ...while the ciphertext itself is not deterministic. + expect(storedAttrs(first.data).email__source).not.toBe( + storedAttrs(second.data).email__source, + ) + }) +}) + +describe('audit metadata', () => { + it('is accepted on every operation without changing the result', async () => { + const metadata = { sub: 'user-id-123', action: 'characterisation' } + const item: User = { + pk: 'user#16', + email: 'ivan@example.com', + name: 'Ivan', + } + + const encrypted = await dynamo + .encryptModel(item, users) + .audit({ metadata }) + if (encrypted.failure) throw new Error(encrypted.failure.message) + expect(encrypted.data).toHaveProperty('email__hmac') + + const decrypted = await dynamo + .decryptModel(encrypted.data, users) + .audit({ metadata }) + if (decrypted.failure) throw new Error(decrypted.failure.message) + expect(decrypted.data).toEqual(item) + + const bulkEncrypted = await dynamo + .bulkEncryptModels([item], users) + .audit({ metadata }) + if (bulkEncrypted.failure) throw new Error(bulkEncrypted.failure.message) + + const bulkDecrypted = await dynamo + .bulkDecryptModels(bulkEncrypted.data, users) + .audit({ metadata }) + if (bulkDecrypted.failure) throw new Error(bulkDecrypted.failure.message) + expect(bulkDecrypted.data).toEqual([item]) + }) + + it('returns the operation itself so .audit() can be chained before awaiting', () => { + const operation = dynamo.encryptModel({ pk: 'x' }, users) + + expect(operation.audit({ metadata: {} })).toBe(operation) + }) +}) + +describe('error handling', () => { + const unknownColumn = encryptedTable('users', { + nope: encryptedColumn('nonexistent_column'), + }) + + it('surfaces the FFI error code when encrypting an unregistered column', async () => { + const result = await dynamo.encryptModel( + { nope: 'value' }, + // Not among the client's schemas — the FFI rejects the column. + unknownColumn, + ) + + expect(result.failure).toBeDefined() + expect(result.failure?.code).toBe('UNKNOWN_COLUMN') + }) + + it('surfaces the FFI error code on the bulk encrypt path', async () => { + const result = await dynamo.bulkEncryptModels( + [{ nope: 'value' }], + unknownColumn, + ) + + expect(result.failure).toBeDefined() + expect(result.failure?.code).toBe('UNKNOWN_COLUMN') + }) + + it('fails with a DynamoDB error code when __source is not a ciphertext', async () => { + const result = await dynamo.decryptModel( + { email__source: 'not-a-ciphertext' }, + users, + ) + + expect(result.failure).toBeDefined() + expect(result.failure?.name).toBe('EncryptedDynamoDBError') + // No FFI code on this path, so the adapter's own fallback code is used. + expect(result.failure?.code).toBe('DYNAMODB_ENCRYPTION_ERROR') + expect(result.failure?.details).toEqual({ context: 'decryptModel' }) + }) + + it('fails on the bulk decrypt path for malformed ciphertext', async () => { + const result = await dynamo.bulkDecryptModels( + [{ email__source: 'not-a-ciphertext' }], + users, + ) + + expect(result.failure).toBeDefined() + expect(result.failure?.code).toBe('DYNAMODB_ENCRYPTION_ERROR') + expect(result.failure?.details).toEqual({ context: 'bulkDecryptModels' }) + }) + + it('fails the whole batch when one item is undecryptable (all-or-nothing)', async () => { + const result = await dynamo.bulkDecryptModels( + // One malformed item alongside a well-formed plaintext-only one. + [{ email__source: 'not-a-ciphertext' }, { pk: 'ok' }], + users, + ) + + // The batch is all-or-nothing: a single bad item fails the whole call, and + // no partial data is returned. + expect(result.failure).toBeDefined() + expect((result as { data?: unknown }).data).toBeUndefined() + }) + + it('routes failures to the configured errorHandler', async () => { + const seen: { code: string; message: string }[] = [] + const instrumented = encryptedDynamoDB({ + encryptionClient: client, + options: { + errorHandler: (e) => seen.push({ code: e.code, message: e.message }), + }, + }) + + await instrumented.encryptModel({ nope: 'value' }, unknownColumn) + + expect(seen).toHaveLength(1) + expect(seen[0]?.code).toBe('UNKNOWN_COLUMN') + }) +}) diff --git a/packages/stack/__tests__/dynamodb/helpers-v3.test.ts b/packages/stack/__tests__/dynamodb/helpers-v3.test.ts new file mode 100644 index 000000000..543b2eb51 --- /dev/null +++ b/packages/stack/__tests__/dynamodb/helpers-v3.test.ts @@ -0,0 +1,529 @@ +/** + * Pure tests for the EQL v3 attribute mapping (#657). + * + * The v2 equivalents live in `helpers.test.ts` and are the characterisation + * baseline; this file asserts only what v3 changes: + * + * - v3 scalars carry NO `k` discriminator, so the write path must key off the + * presence of a ciphertext. Gating on `k === 'ct'` (as the v2 code did) + * dropped every v3 scalar through to the nested-object branch and wrote it + * out as a raw map. + * - the read path synthesizes a v3 envelope (`v: 3`, no `k`) for a v3 table + * and a v2 one (`v: 2`, `k: 'ct'`) for a v2 table. + * - a v3 JSON document keeps `k: 'sv'`, which is mandatory — deserialization + * fails with "missing field `k`" without it. + */ +import { afterEach, describe, expect, it, vi } from 'vitest' +import { + buildReadContext, + deepClone, + isV3Table, + toEncryptedDynamoItem, + toItemWithEqlPayloads, +} from '@/dynamodb/helpers' +import { encryptedTable, types } from '@/eql/v3' +import { + encryptedColumn, + encryptedField, + encryptedTable as encryptedTableV2, +} from '@/schema' +import { logger } from '@/utils/logger' + +const users = encryptedTable('users', { + email: types.TextEq('email'), + name: types.Text('name'), + age: types.IntegerOrd('age'), + meta: types.Json('meta'), +}) + +const encryptedAttrs = Object.keys(users.build().columns) + +// Several tests here `vi.spyOn(logger, 'debug')` — the same shared singleton that +// `resolve-decrypt.test.ts` also spies. Each test restores its own spy in a +// `finally`, but this hook is the safety net: it guarantees no spy on any shared +// object survives a test boundary, so a thrown assertion can never leak a patched +// method into the next test (`clearAllMocks` only clears calls; `restoreAllMocks` +// un-patches). +afterEach(() => { + vi.restoreAllMocks() +}) + +/** A v3 scalar payload as the FFI returns it — note the absent `k`. */ +const scalar = ( + column: string, + c: string, + terms: Record = {}, +) => ({ + v: 3, + i: { t: 'users', c: column }, + c, + ...terms, +}) + +describe('isV3Table', () => { + it('recognises a v3 table by its concrete-domain columns', () => { + expect(isV3Table(users)).toBe(true) + }) + + it('recognises a flat v2 table', () => { + const v2 = encryptedTableV2('users', { + email: encryptedColumn('email').equality(), + }) + + expect(isV3Table(v2)).toBe(false) + }) + + it('recognises a v2 table whose columns are nested under a group', () => { + const v2 = encryptedTableV2('users', { + example: { protected: encryptedField('example.protected') }, + }) + + expect(isV3Table(v2)).toBe(false) + }) +}) + +describe('toEncryptedDynamoItem with v3 payloads', () => { + it('splits an untagged v3 scalar into __source and __hmac', () => { + const result = toEncryptedDynamoItem( + { + pk: 'user#1', + email: scalar('email', 'ciphertext', { hm: 'hmac-value' }), + }, + encryptedAttrs, + ) + + expect(result).toEqual({ + pk: 'user#1', + email__source: 'ciphertext', + email__hmac: 'hmac-value', + }) + }) + + it('stores a storage-only column with no search term', () => { + const result = toEncryptedDynamoItem( + { name: scalar('name', 'ciphertext') }, + encryptedAttrs, + ) + + expect(result).toEqual({ name__source: 'ciphertext' }) + }) + + it('drops the ordering term of an *Ord domain — DynamoDB cannot use it', () => { + const result = toEncryptedDynamoItem( + { age: scalar('age', 'ciphertext', { op: 'ordering-term' }) }, + encryptedAttrs, + ) + + // Decryptable, but not orderable and not usable in a key condition. + expect(result).toEqual({ age__source: 'ciphertext' }) + expect(result).not.toHaveProperty('age__hmac') + }) + + it('keeps the equality term of a TextSearch domain and drops the rest', () => { + const result = toEncryptedDynamoItem( + { + email: scalar('email', 'ciphertext', { + hm: 'hmac-value', + op: 'ordering-term', + bf: [1, 2, 3], + }), + }, + encryptedAttrs, + ) + + expect(result).toEqual({ + email__source: 'ciphertext', + email__hmac: 'hmac-value', + }) + }) + + it('stores a v3 ste_vec document as its entries plus the KeyHeader', () => { + const entries = [{ s: 'sel', c: 'ct', a: false, hm: 'h' }] + // Only the non-reconstructable parts are stored: the `sv` entries and the + // per-document KeyHeader `h` (protect-ffi 0.30 decrypt requires `h`). The + // `v`/`i`/`k` envelope fields are rebuilt on read, so they are not stored. + const result = toEncryptedDynamoItem( + { + meta: { + v: 3, + k: 'sv', + i: { t: 'users', c: 'meta' }, + h: 'key-header', + sv: entries, + }, + }, + encryptedAttrs, + ) + + expect(result).toEqual({ meta__source: { h: 'key-header', sv: entries } }) + }) +}) + +describe('toItemWithEqlPayloads for a v3 table', () => { + it('rebuilds an untagged v3 scalar envelope', () => { + const result = toItemWithEqlPayloads({ email__source: 'ciphertext' }, users) + + expect(result).toEqual({ + email: { i: { c: 'email', t: 'users' }, v: 3, c: 'ciphertext' }, + }) + expect(result.email).not.toHaveProperty('k') + }) + + it('rebuilds a v3 ste_vec envelope, restoring k and the KeyHeader', () => { + // The stored __source is `{ h, sv }`; the read path rebuilds the full + // envelope, restoring the `k` tag and the `h` KeyHeader that 0.30 decrypt + // requires, with `i`/`v` reconstructed from the schema. + const entries = [{ s: 'sel', c: 'ct' }] + const result = toItemWithEqlPayloads( + { meta__source: { h: 'key-header', sv: entries } }, + users, + ) + + expect(result).toEqual({ + meta: { + i: { c: 'meta', t: 'users' }, + v: 3, + k: 'sv', + h: 'key-header', + sv: entries, + }, + }) + }) + + it('still emits a v2 envelope for a v2 table', () => { + const v2 = encryptedTableV2('users', { + email: encryptedColumn('email').equality(), + }) + + expect(toItemWithEqlPayloads({ email__source: 'ct' }, v2)).toEqual({ + email: { i: { c: 'email', t: 'users' }, v: 2, k: 'ct', c: 'ct' }, + }) + }) + + it('discards __hmac and passes plaintext through, as in v2', () => { + const result = toItemWithEqlPayloads( + { pk: 'user#1', email__source: 'ct', email__hmac: 'h', role: 'admin' }, + users, + ) + + expect(result).toEqual({ + pk: 'user#1', + email: { i: { c: 'email', t: 'users' }, v: 3, c: 'ct' }, + role: 'admin', + }) + }) + + it('always emits both `v` and `i` — a payload missing either is returned raw', () => { + const rebuilt = toItemWithEqlPayloads({ email__source: 'ct' }, users) + .email as Record + + // Not a cosmetic assertion: the FFI treats `v`/`i` as the "is this + // encrypted" detection keys. A payload missing either is silently passed + // through undecrypted rather than erroring. + expect(rebuilt).toHaveProperty('v') + expect(rebuilt).toHaveProperty('i') + }) + + it('round-trips a v3 item through both directions', () => { + const stored = toEncryptedDynamoItem( + { + pk: 'user#1', + email: scalar('email', 'email-ct', { hm: 'email-hmac' }), + role: 'admin', + }, + encryptedAttrs, + ) + + expect(toItemWithEqlPayloads(stored, users)).toEqual({ + pk: 'user#1', + email: { i: { c: 'email', t: 'users' }, v: 3, c: 'email-ct' }, + role: 'admin', + }) + }) +}) + +describe('regressions found in review', () => { + it('does not mistake a nested plaintext object with a `c` key for a payload', () => { + // `c` is an ordinary attribute name (country, currency, count). Treating + // it as a ciphertext rewrote the object to `__source` and DISCARDED + // every sibling key — silent data loss on a PutCommand. + const item = { shipping: { address: { c: 'AU', street: '1 Main St' } } } + + expect(toEncryptedDynamoItem(item, encryptedAttrs)).toEqual(item) + }) + + it('does not drop a plaintext attribute that merely ends in __hmac', () => { + const item = { pk: 'u#1', signature__hmac: 'an app-level hmac' } + + expect(toItemWithEqlPayloads(item, users)).toEqual(item) + }) + + it('does not treat an unregistered nested __source attribute as an envelope', () => { + const item = { grp: { unrelated__source: 'plaintext' } } + + expect(toItemWithEqlPayloads(item, users)).toEqual(item) + }) + + it('logs a debug when a *__source attribute names no declared column', () => { + // A schema rename leaves the stored `__source` orphaned; it reads back + // as raw base64 with no error. Make the silent drop observable. + const spy = vi.spyOn(logger, 'debug').mockImplementation(() => {}) + + try { + toItemWithEqlPayloads({ unknown__source: 'CT' }, users) + + expect(spy).toHaveBeenCalledWith( + expect.stringContaining('unknown__source'), + ) + expect(spy).toHaveBeenCalledWith( + expect.stringContaining('no declared column'), + ) + } finally { + spy.mockRestore() + } + }) + + it('does not rebuild a nested __source whose leaf collides with a top-level column', () => { + // `note` is a TOP-LEVEL column; there is no `profile.note`. A v3 table + // registers full dotted paths, so a nested `note__source` must NOT match + // the top-level `note` by bare leaf — doing so rewrote a plaintext sibling + // as an envelope and handed it to the FFI as a decrypt target. + const t = encryptedTable('t', { + email: types.TextEq('email'), + note: types.Text('note'), + }) + const item = { profile: { note__source: 'plaintext, not a ciphertext' } } + + expect(toItemWithEqlPayloads(item, t)).toEqual(item) + }) + + it('still rebuilds a v3 column declared with a dotted path', () => { + const t = encryptedTable('t', { + 'profile.ssn': types.TextEq('profile.ssn'), + }) + + expect( + toItemWithEqlPayloads({ profile: { ssn__source: 'CT' } }, t), + ).toEqual({ + profile: { ssn: { i: { c: 'profile.ssn', t: 't' }, v: 3, c: 'CT' } }, + }) + }) + + it('round-trips a three-level dotted path', () => { + const t = encryptedTable('t', { 'a.b.c': types.TextEq('a.b.c') }) + const stored = toEncryptedDynamoItem( + { + a: { b: { c: { v: 3, i: { t: 't', c: 'a.b.c' }, c: 'CT', hm: 'H' } } }, + }, + Object.keys(t.buildColumnKeyMap()), + ) + + expect(stored).toEqual({ a: { b: { c__source: 'CT', c__hmac: 'H' } } }) + expect(toItemWithEqlPayloads(stored, t)).toEqual({ + a: { b: { c: { i: { c: 'a.b.c', t: 't' }, v: 3, c: 'CT' } } }, + }) + }) + + it('does not split a nested payload whose path names no declared column', () => { + // The write path must only split DECLARED columns. A pre-encrypted payload + // under an undeclared nested name used to be split into `__source` + // by shape alone — but the read path rebuilds only declared columns, so it + // would never reassemble it: silent, undecryptable data. Leave the whole + // envelope in place so write and read stay symmetric and it round-trips. + const item = { profile: { secret: scalar('secret', 'CT', { hm: 'H' }) } } + + const stored = toEncryptedDynamoItem(item, encryptedAttrs, true) + + expect(stored).toEqual(item) + expect(toItemWithEqlPayloads(stored, users)).toEqual(item) + }) + + it('splits a detected payload with an empty-string ciphertext rather than leaking its envelope', () => { + // `{ v, i, c: '' }` is DETECTED as a payload (it has v+i+c), but the scalar + // arm gated on truthiness (`if (payload.c)`), so an empty ciphertext fell + // through and was written out as a RAW MAP — leaking v/i into storage, the + // exact thing the "no envelope metadata leak" invariant forbids. Split it + // as `__source: ''` like any other ciphertext. + const stored = toEncryptedDynamoItem( + { email: { v: 3, i: { t: 'users', c: 'email' }, c: '' } }, + encryptedAttrs, + ) + + expect(stored).toEqual({ email__source: '' }) + + // The other half of the contract: read must rebuild `c: ''` intact — not + // drop it or treat the empty source as a missing attribute — or the value + // is unrecoverable. A truthiness regression on the read side would strip it. + expect(toItemWithEqlPayloads(stored, users)).toEqual({ + email: { i: { c: 'email', t: 'users' }, v: 3, c: '' }, + }) + }) + + it('identifies a column by its DB name when it differs from the property', () => { + // `emailAddress: types.TextEq('email_address')` — matching must happen on + // the property name, identification on the DB name. + const t = encryptedTable('t', { + emailAddress: types.TextEq('email_address'), + }) + + const stored = toEncryptedDynamoItem( + { + emailAddress: { + v: 3, + i: { t: 't', c: 'email_address' }, + c: 'CT', + hm: 'H', + }, + }, + Object.keys(t.buildColumnKeyMap()), + ) + expect(stored).toEqual({ + emailAddress__source: 'CT', + emailAddress__hmac: 'H', + }) + + expect(toItemWithEqlPayloads(stored, t)).toEqual({ + emailAddress: { i: { c: 'email_address', t: 't' }, v: 3, c: 'CT' }, + }) + }) +}) + +describe('arrays are a deliberate carve-out', () => { + // The mapping descends into nested OBJECTS but never ARRAYS. A payload inside + // an array is therefore stored as its whole envelope, symmetrically skipped on + // read, so it still decrypts — it is just never split into a queryable + // `__source`/`__hmac`. These pin that carve-out with REAL payloads (the older + // "leaves arrays untouched" test used a `{ c }` lookalike the detector would + // skip anyway, so it did not characterise the skip). + + it('leaves a real payload inside an array whole rather than splitting it', () => { + const item = { tags: [scalar('tags', 'CT', { hm: 'H' })] } + + // Whole envelope retained (v, i, c, hm all present); no tags__source/__hmac. + expect(toEncryptedDynamoItem(item, encryptedAttrs, true)).toEqual(item) + }) + + it('passes an envelope nested in an array straight through on read', () => { + const item = { tags: [{ v: 3, i: { t: 'users', c: 'email' }, c: 'CT' }] } + + // Not recursed, not rebuilt, not dropped — the array element reaches the + // caller/FFI unchanged, which is what lets it decrypt. + expect(toItemWithEqlPayloads(item, users)).toEqual(item) + }) + + it('round-trips a payload nested in an array', () => { + const item = { tags: [scalar('tags', 'CT', { hm: 'H' })] } + + const stored = toEncryptedDynamoItem(item, encryptedAttrs, true) + expect(stored).toEqual(item) + expect(toItemWithEqlPayloads(stored, users)).toEqual(item) + }) + + it('does not split a declared column whose value is an array', () => { + // The one shape where the split branch's column gate DOES match + // (`matchColumn('email', '')` succeeds) — only `isStoredEqlPayload([...])` + // being false stops a split. That guard interaction is otherwise untested. + const item = { email: [scalar('email', 'CT', { hm: 'H' })] } + + const stored = toEncryptedDynamoItem(item, encryptedAttrs, true) + expect(stored).toEqual(item) + expect(stored).not.toHaveProperty('email__source') + expect(toItemWithEqlPayloads(stored, users)).toEqual(item) + }) + + it('leaves an array of objects each holding a payload whole, and round-trips it', () => { + // The realistic customer shape: a list of records, each with an encrypted + // field. Every element is stored and read whole. + const item = { + history: [ + { at: 'day1', email: scalar('email', 'CT1', { hm: 'H1' }) }, + { at: 'day2', email: scalar('email', 'CT2') }, + ], + } + + const stored = toEncryptedDynamoItem(item, encryptedAttrs, true) + expect(stored).toEqual(item) + expect(toItemWithEqlPayloads(stored, users)).toEqual(item) + }) +}) + +describe('read context is resolved once per batch', () => { + it('does not call table.build() per item when a context is passed', () => { + // `buildReadContext` resolves the row-invariant facts once; the 3-arg form + // of `toItemWithEqlPayloads` reuses them. Dropping the 3rd arg would rebuild + // (and re-`build()`) per item — the invariant this pins. + const buildSpy = vi.spyOn(users, 'build') + + const context = buildReadContext(users) + buildSpy.mockClear() + + const items = [ + { email__source: 'a' }, + { email__source: 'b' }, + { email__source: 'c' }, + ] + for (const item of items) { + toItemWithEqlPayloads(item, users, context) + } + + expect(buildSpy).not.toHaveBeenCalled() + + // ...and the 3-arg (prebuilt context) form produces the same item as the + // 2-arg (default context) form — passing a context is an optimisation, not a + // behaviour change. + expect(toItemWithEqlPayloads(items[0], users, context)).toEqual( + toItemWithEqlPayloads(items[0], users), + ) + + buildSpy.mockRestore() + }) +}) + +describe('deepClone preserves structured values', () => { + it('preserves Date instances — the whole Timestamp domain family depends on it', () => { + const at = new Date('2020-01-02T03:04:05.000Z') + + expect(deepClone(at)).toBeInstanceOf(Date) + expect(deepClone({ at }).at.getTime()).toBe(at.getTime()) + }) + + it('preserves Map, Set and typed arrays', () => { + expect(deepClone(new Map([['a', 1]])).get('a')).toBe(1) + expect(deepClone(new Set([1, 2])).has(2)).toBe(true) + expect(deepClone(new Uint8Array([1, 2]))).toBeInstanceOf(Uint8Array) + }) + + it('does not blow the stack on a circular reference', () => { + const o: Record = { a: 1 } + o.self = o + + expect(() => deepClone(o)).not.toThrow() + }) + + it('never hands back the caller original when structuredClone throws', () => { + // A function-valued property makes `structuredClone` throw. The catch must + // still produce a NEW object — returning the original voids the docblock + // guarantee that encryption never mutates a caller's object. + const input = { pk: 'u#1', onSave: () => {} } + + const cloned = deepClone(input) + + expect(cloned).not.toBe(input) + expect(cloned.pk).toBe('u#1') + }) + + it('clones a class instance into a plain object, dropping the prototype', () => { + class Model { + pk = 'u#1' + // An own function property forces the structuredClone fallback. + onSave = () => {} + } + const input = new Model() + + const cloned = deepClone(input) + + expect(cloned).not.toBe(input) + expect(cloned).not.toBeInstanceOf(Model) + expect(cloned.pk).toBe('u#1') + }) +}) diff --git a/packages/stack/__tests__/dynamodb/helpers.test.ts b/packages/stack/__tests__/dynamodb/helpers.test.ts new file mode 100644 index 000000000..37f4ef1ce --- /dev/null +++ b/packages/stack/__tests__/dynamodb/helpers.test.ts @@ -0,0 +1,349 @@ +/** + * Characterisation tests for the pure DynamoDB attribute-mapping helpers. + * + * These pin the CURRENT (EQL v2) behaviour of `toEncryptedDynamoItem` and + * `toItemWithEqlPayloads` before the EQL v3 port (#657). They need no + * credentials and make no network calls — every assertion is about the shape + * of the attribute map on the way into and out of DynamoDB. + * + * Read them as a specification of the storage format: + * `` (an EQL payload) <-> `__source` (+ `__hmac`) + */ +import { describe, expect, it } from 'vitest' +import { + ciphertextAttrSuffix, + deepClone, + EncryptedDynamoDBErrorImpl, + handleError, + searchTermAttrSuffix, + toEncryptedDynamoItem, + toItemWithEqlPayloads, +} from '@/dynamodb/helpers' +import { encryptedColumn, encryptedField, encryptedTable } from '@/schema' + +const users = encryptedTable('users', { + email: encryptedColumn('email').equality(), + name: encryptedColumn('name'), + blob: encryptedColumn('blob').dataType('json'), + example: { + protected: encryptedField('example.protected'), + }, +}) + +const encryptedAttrs = Object.keys(users.build().columns) + +/** A minimal v2 scalar-ciphertext payload as the FFI returns it. */ +const ct = (c: string, hm?: string) => ({ + k: 'ct' as const, + v: 2, + i: { t: 'users', c: 'email' }, + c, + ...(hm ? { hm } : {}), +}) + +describe('attribute suffixes', () => { + it('are the documented DynamoDB naming convention', () => { + expect(ciphertextAttrSuffix).toBe('__source') + expect(searchTermAttrSuffix).toBe('__hmac') + }) +}) + +describe('toEncryptedDynamoItem (write path)', () => { + it('splits a ciphertext payload with an HMAC into __source and __hmac', () => { + const result = toEncryptedDynamoItem( + { pk: 'user#1', email: ct('ciphertext', 'hmac-value') }, + encryptedAttrs, + ) + + expect(result).toEqual({ + pk: 'user#1', + email__source: 'ciphertext', + email__hmac: 'hmac-value', + }) + // The original attribute name is consumed, not retained alongside. + expect(result).not.toHaveProperty('email') + }) + + it('emits only __source when the payload carries no HMAC', () => { + const result = toEncryptedDynamoItem( + { name: { ...ct('ciphertext'), i: { t: 'users', c: 'name' } } }, + encryptedAttrs, + ) + + expect(result).toEqual({ name__source: 'ciphertext' }) + expect(result).not.toHaveProperty(`name${searchTermAttrSuffix}`) + }) + + it('passes attributes absent from the schema through untouched', () => { + const item = { + pk: 'user#1', + role: 'admin', + count: 42, + flag: true, + tags: ['a', 'b'], + } + + expect(toEncryptedDynamoItem(item, encryptedAttrs)).toEqual(item) + }) + + it('preserves null and undefined without adding suffixed attributes', () => { + const result = toEncryptedDynamoItem( + { email: null, name: undefined }, + encryptedAttrs, + ) + + expect(result).toEqual({ email: null, name: undefined }) + }) + + it('recurses into nested objects and splits payloads found inside', () => { + const result = toEncryptedDynamoItem( + { + example: { + protected: { ...ct('nested-ct'), i: { t: 'users', c: 'protected' } }, + notProtected: 'plaintext', + }, + }, + encryptedAttrs, + ) + + expect(result).toEqual({ + example: { + protected__source: 'nested-ct', + notProtected: 'plaintext', + }, + }) + }) + + it('leaves arrays untouched rather than recursing into them', () => { + const result = toEncryptedDynamoItem( + { items: [{ c: 'looks-like-a-payload' }] }, + encryptedAttrs, + ) + + expect(result).toEqual({ items: [{ c: 'looks-like-a-payload' }] }) + }) + + it('leaves a real v2 payload inside an array whole and round-trips it', () => { + // The array carve-out with a genuine `v+i+c` payload (the case above uses a + // `{ c }` lookalike the detector skips anyway): stored whole, not split, and + // symmetrically passed through on read so it still decrypts. + const item = { tags: [ct('array-ct', 'array-hmac')] } + + const stored = toEncryptedDynamoItem(item, encryptedAttrs) + expect(stored).toEqual(item) + expect(stored).not.toHaveProperty('tags__source') + expect(toItemWithEqlPayloads(stored, users)).toEqual(item) + }) + + it('does not split a nested payload whose leaf names no declared v2 column', () => { + // The v2 branch carries the bare-leaf fallback; its POSITIVE case is tested + // above. The negative — a nested leaf that is NOT a declared column must be + // left whole (the read path rebuilds only declared columns, so a split here + // would be unrecoverable). Only the v3 branch (no fallback) covered this. + const item = { profile: { secret: ct('CT', 'H') } } + + const stored = toEncryptedDynamoItem(item, encryptedAttrs) + expect(stored).toEqual(item) + expect(toItemWithEqlPayloads(stored, users)).toEqual(item) + }) +}) + +describe('toItemWithEqlPayloads (read path)', () => { + it('rebuilds a v2 scalar ciphertext envelope from __source', () => { + const result = toItemWithEqlPayloads( + { pk: 'user#1', email__source: 'ciphertext' }, + users, + ) + + expect(result).toEqual({ + pk: 'user#1', + email: { + i: { c: 'email', t: 'users' }, + v: 2, + k: 'ct', + c: 'ciphertext', + }, + }) + }) + + it('drops the __hmac attribute — it is not part of the envelope', () => { + const result = toItemWithEqlPayloads( + { email__source: 'ciphertext', email__hmac: 'hmac-value' }, + users, + ) + + expect(Object.keys(result)).toEqual(['email']) + expect(result.email).not.toHaveProperty('hm') + }) + + it('rebuilds a scalar envelope for a JSON column without a ste_vec index', () => { + const result = toItemWithEqlPayloads({ blob__source: 'ciphertext' }, users) + + expect(result).toEqual({ + blob: { + i: { c: 'blob', t: 'users' }, + v: 2, + k: 'ct', + c: 'ciphertext', + }, + }) + }) + + it('rebuilds nested payloads, identifying them by their registered dotted name', () => { + const result = toItemWithEqlPayloads( + { example: { protected__source: 'nested-ct', notProtected: 'plain' } }, + users, + ) + + expect(result).toEqual({ + example: { + protected: { + // The column is registered as `example.protected`, so that is what + // the rebuilt identifier carries. This originally emitted the bare + // leaf (`protected`), which only worked because the FFI treats `i` as + // a detection key and never validates it against the ciphertext. + i: { c: 'example.protected', t: 'users' }, + v: 2, + k: 'ct', + c: 'nested-ct', + }, + notProtected: 'plain', + }, + }) + }) + + it('falls back to the leaf name when only the leaf is registered', () => { + // `encryptedField('amount')` under a `details` group — the convention the + // schema docs show — registers `amount`, not `details.amount`. + const orders = encryptedTable('orders', { + details: { amount: encryptedField('amount') }, + }) + + const result = toItemWithEqlPayloads( + { details: { amount__source: 'ct' } }, + orders, + ) + + expect(result).toEqual({ + details: { + amount: { i: { c: 'amount', t: 'orders' }, v: 2, k: 'ct', c: 'ct' }, + }, + }) + }) + + it('splits and drops a nested v2 grouped field __hmac via the bare-leaf fallback', () => { + // `encryptedField('amount')` under a `details` group registers the bare + // leaf `amount`. A nested equality payload must split to `amount__hmac` on + // write and have it dropped on read through the v2 bare-leaf fallback — the + // v2 twin of the v3 dotted-path __hmac coverage, which was otherwise only + // exercised in the live suite. + const orders = encryptedTable('orders', { + details: { amount: encryptedField('amount') }, + }) + const attrs = Object.keys(orders.build().columns) + + const stored = toEncryptedDynamoItem( + { + details: { + amount: { + k: 'ct', + v: 2, + i: { t: 'orders', c: 'amount' }, + c: 'ct', + hm: 'h', + }, + }, + }, + attrs, + ) + expect(stored).toEqual({ + details: { amount__source: 'ct', amount__hmac: 'h' }, + }) + + expect(toItemWithEqlPayloads(stored, orders)).toEqual({ + details: { + amount: { i: { c: 'amount', t: 'orders' }, v: 2, k: 'ct', c: 'ct' }, + }, + }) + }) + + it('passes plaintext attributes and null/undefined through untouched', () => { + const item = { pk: 'user#1', role: 'admin', a: null, b: undefined } + + expect(toItemWithEqlPayloads(item, users)).toEqual(item) + }) + + it('round-trips an item through both directions', () => { + const payloads = { + pk: 'user#1', + email: ct('email-ct', 'email-hmac'), + role: 'admin', + } + + const stored = toEncryptedDynamoItem(payloads, encryptedAttrs) + const rebuilt = toItemWithEqlPayloads(stored, users) + + // The HMAC is not recoverable from the rebuilt envelope: it lives only in + // the `__hmac` attribute, which the read path deliberately discards. + expect(rebuilt).toEqual({ + pk: 'user#1', + email: { i: { c: 'email', t: 'users' }, v: 2, k: 'ct', c: 'email-ct' }, + role: 'admin', + }) + }) +}) + +describe('deepClone', () => { + it('returns primitives unchanged', () => { + expect(deepClone(1)).toBe(1) + expect(deepClone('a')).toBe('a') + expect(deepClone(null)).toBe(null) + expect(deepClone(undefined)).toBe(undefined) + }) + + it('clones nested objects and arrays without sharing references', () => { + const source = { a: { b: [1, { c: 2 }] } } + const clone = deepClone(source) + + expect(clone).toEqual(source) + expect(clone.a).not.toBe(source.a) + expect(clone.a.b).not.toBe(source.a.b) + }) +}) + +describe('handleError', () => { + it('falls back to DYNAMODB_ENCRYPTION_ERROR for a plain Error', () => { + const error = handleError(new Error('boom'), 'encryptModel') + + expect(error).toBeInstanceOf(EncryptedDynamoDBErrorImpl) + expect(error.name).toBe('EncryptedDynamoDBError') + expect(error.message).toBe('boom') + expect(error.code).toBe('DYNAMODB_ENCRYPTION_ERROR') + expect(error.details).toEqual({ context: 'encryptModel' }) + }) + + it('preserves a `code` carried on the thrown object', () => { + const thrown = Object.assign(new Error('nope'), { code: 'UNKNOWN_COLUMN' }) + + expect(handleError(thrown, 'encryptModel').code).toBe('UNKNOWN_COLUMN') + }) + + it('stringifies a non-Error throw', () => { + expect(handleError('just a string', 'decryptModel').message).toBe( + 'just a string', + ) + }) + + it('invokes the errorHandler and logger callbacks when supplied', () => { + const seen: unknown[] = [] + const logged: string[] = [] + + const error = handleError(new Error('boom'), 'decryptModel', { + errorHandler: (e) => seen.push(e), + logger: { error: (message) => logged.push(message) }, + }) + + expect(seen).toEqual([error]) + expect(logged).toEqual(['Error in decryptModel']) + }) +}) diff --git a/packages/stack/__tests__/dynamodb/hmac-domains.test.ts b/packages/stack/__tests__/dynamodb/hmac-domains.test.ts new file mode 100644 index 000000000..d1108e42c --- /dev/null +++ b/packages/stack/__tests__/dynamodb/hmac-domains.test.ts @@ -0,0 +1,77 @@ +/** + * Ground truth for which EQL v3 domains produce a DynamoDB `__hmac` attribute. + * + * On DynamoDB, `encryptedDynamoDB` writes `__hmac` only when the encrypted + * payload carries an `hm` equality term (the gate is `if (encryptPayload.hm)` in + * `helpers.ts`). The FFI emits `hm` exactly when the column's `build()` config + * carries the `unique` index, minted by `indexesForCapabilities` + * (`eql/v3/columns.ts`): a column is `equality`-capable AND either not ordering + * or text-cast. So `unique`-index presence per domain IS the ground truth for + * `__hmac`, and this test pins it for EVERY `types.*` domain. + * + * It exists because the `skills/stash-dynamodb` doc makes a per-domain claim + * about `__hmac` and nothing else checks it — skills ship to customers and drift + * silently. If a domain's capabilities change, this fails and the doc's domain + * table must be re-checked against it. + * + * Note the subtlety this locks: several `*Ord` domains are `equality`-capable + * (they answer equality via an ordering term in Postgres) yet write NO `__hmac`, + * because equality on DynamoDB needs the deterministic `hm`, which only the + * `unique` index provides. "Equality-capable" is therefore NOT the same set as + * "writes `__hmac`". + */ +import { describe, expect, it } from 'vitest' +import { types } from '@/eql/v3' + +/** + * The domains that mint an `hm` equality term, and so write `__hmac`: + * every `*Eq`, plus the text-cast ordering/search domains (text equality is + * always HMAC-based). Everything else — non-text `*Ord`/`*OrdOre`, the bare + * storage-only domains, `TextMatch`, and `Json` — does not. + */ +const WRITES_HMAC = new Set([ + 'TextEq', + 'IntegerEq', + 'SmallintEq', + 'BigintEq', + 'DateEq', + 'TimestampEq', + 'NumericEq', + 'RealEq', + 'DoubleEq', + 'TextOrd', + 'TextOrdOre', + 'TextSearch', +]) + +/** Does this domain's column config carry the `unique` index (→ `hm` → `__hmac`)? */ +function buildsUniqueIndex(factory: (name: string) => unknown): boolean { + const column = factory('c') as { + build(): { indexes: Record } + } + return 'unique' in column.build().indexes +} + +const allDomains = Object.keys(types) as Array + +describe('EQL v3 domain → __hmac ground truth', () => { + it('covers every domain the skill can reference', () => { + // Guard against a new domain being added without a verdict here. + expect(allDomains.length).toBe(40) + }) + + it.each(allDomains)('%s matches the documented __hmac set', (name) => { + const factory = types[name] as (n: string) => unknown + expect(buildsUniqueIndex(factory)).toBe(WRITES_HMAC.has(name as string)) + }) + + it('is exactly the 12 documented domains — no more, no fewer', () => { + const actual = allDomains + .filter((n) => buildsUniqueIndex(types[n] as (x: string) => unknown)) + .map((n) => n as string) + .sort() + + expect(actual).toEqual([...WRITES_HMAC].sort()) + expect(actual).toHaveLength(12) + }) +}) diff --git a/packages/stack/__tests__/dynamodb/properties.test.ts b/packages/stack/__tests__/dynamodb/properties.test.ts new file mode 100644 index 000000000..f72ca706d --- /dev/null +++ b/packages/stack/__tests__/dynamodb/properties.test.ts @@ -0,0 +1,616 @@ +/** + * Property-based coverage (fast-check) for the DynamoDB attribute mapping. + * + * The example-based suites (`helpers.test.ts`, `helpers-v3.test.ts`) pin the + * mapping pointwise. Four data-integrity bugs still slipped through all of + * them, because every one needed a *shape the examples never wrote down*: a + * plaintext key that happens to collide with an EQL envelope key, an attribute + * that happens to end in a storage suffix, a `Date` in a cloned model, a column + * whose JS property name differs from its DB name. These are the cross-cutting + * invariants that make those shapes unrepresentable rather than merely untested: + * + * 1. ROUND TRIP: for any column set and any ciphertext, + * `toItemWithEqlPayloads(toEncryptedDynamoItem(x))` preserves every + * ciphertext and rebuilds the envelope the FFI will accept — scalar + * (untagged) and ste_vec (`k: 'sv'`) alike. + * 2. PASSTHROUGH: an item with NO encrypted payloads is returned unchanged by + * BOTH directions, at any nesting depth, for any attribute name — + * including names that collide with envelope keys (`c`, `hm`, `k`, `sv`, + * `v`, `i`) and with the storage suffixes (`legit__hmac`, `img__source`). + * 3. NO ATTRIBUTE LOSS: neither direction ever reduces the top-level key + * count of an all-plaintext item. + * 4. deepClone is identity-preserving, `Date`-preserving, and shares no + * object reference with its source. + * 5. VERSION FOLLOWS THE TABLE: a v3 table always yields `v: 3` and never + * `k: 'ct'`; a v2 table always yields `v: 2` with `k: 'ct'`. + * 6. NO ENVELOPE METADATA LEAK: a split scalar writes only `__source` + * (plus `__hmac` when `hm` is present) and never leaks `k`/`v`/`i` + * into a stored attribute value. + * 7. PROPERTY vs DB NAME: the stored attribute is keyed by the JS PROPERTY + * name while the rebuilt identifier `i.c` is the DB name. + * + * Every property here is pure — no client, no credentials, no network. The one + * live property (a ZeroKMS `decryptModel(encryptModel(x))` round trip) lives in + * `encrypted-dynamodb-v3.test.ts` with the rest of the live suite. + */ +import fc from 'fast-check' +import { describe, expect, it } from 'vitest' +import { + ciphertextAttrSuffix, + deepClone, + isV3Table, + searchTermAttrSuffix, + toEncryptedDynamoItem, + toItemWithEqlPayloads, +} from '@/dynamodb/helpers' +import { encryptedTable, types } from '@/eql/v3' +import { + encryptedColumn, + encryptedField, + encryptedTable as encryptedTableV2, +} from '@/schema' + +// --------------------------------------------------------------------------- +// Generators +// --------------------------------------------------------------------------- + +/** + * `EncryptedTable.build()` THROWS on duplicate DB names and `encryptedTable()` + * throws on a column name that shadows a table member, so generated schemas are + * kept inside a charset that can produce neither: lowercase, max 8 chars (too + * short to contain a `__source`/`__hmac` suffix), minus the one reserved word + * the charset can reach. + */ +const RESERVED_COLUMN_NAMES = new Set(['build']) + +/** The plaintext attribute the payload properties carry alongside the columns. */ +const PLAINTEXT_KEY = 'zz_plain' + +const safeName = fc + .stringMatching(/^[a-z][a-z0-9_]{0,7}$/) + .filter((n) => !RESERVED_COLUMN_NAMES.has(n) && n !== PLAINTEXT_KEY) + +/** + * Includes the empty string deliberately. The write path detects a scalar by + * the PRESENCE of `c`, not its truthiness, so `c: ''` must split to + * `__source: ''` and round-trip like any other ciphertext rather than + * falling through and leaking its `v`/`i` envelope into a stored raw map. + */ +const ciphertext = fc.string() + +/** ste_vec entries, as the FFI emits them for a JSON document. */ +const steVecEntries = fc.array( + fc.record({ s: fc.string(), c: fc.string({ minLength: 1 }) }), + { minLength: 1, maxLength: 3 }, +) + +/** + * Attribute names chosen to be maximally hostile to the mapping's heuristics: + * six that collide with EQL envelope keys, two with the storage suffixes. + */ +const hostileKey = fc.oneof( + fc.constantFrom('c', 'hm', 'k', 'sv', 'v', 'i', 'legit__hmac', 'img__source'), + safeName, +) + +/** An all-plaintext attribute tree: no value anywhere is an EQL payload. */ +const plaintextValue = fc.letrec<{ node: unknown; leaf: unknown }>((tie) => ({ + leaf: fc.oneof( + fc.string(), + fc.integer(), + fc.boolean(), + fc.constant(null), + fc.double({ noNaN: true }), + ), + node: fc.oneof( + { maxDepth: 3, depthSize: 'small' }, + tie('leaf'), + fc.array(tie('node'), { maxLength: 3 }), + fc.dictionary(hostileKey, tie('node'), { maxKeys: 4 }), + ), +})).node + +const plaintextItem = fc.dictionary(hostileKey, plaintextValue, { maxKeys: 5 }) + +/** + * Would the mapping's payload detector see an envelope anywhere in this tree? + * + * The detector is a heuristic — `v` + `i` + (`c` | `sv`) — so an all-plaintext + * object that happens to carry exactly those three keys is genuinely + * indistinguishable from a payload and is *supposed* to be rewritten. The + * passthrough properties are about everything else, so those cases are + * discarded rather than asserted (documenting the heuristic's exact blind spot). + */ +function looksLikeEnvelope(value: unknown): boolean { + if (value === null || typeof value !== 'object') return false + if (Array.isArray(value)) return value.some(looksLikeEnvelope) + const hasEnvelopeKeys = + 'v' in value && 'i' in value && ('c' in value || 'sv' in value) + return hasEnvelopeKeys || Object.values(value).some(looksLikeEnvelope) +} + +// --------------------------------------------------------------------------- +// 1. ROUND TRIP +// --------------------------------------------------------------------------- + +/** A generated v3 schema plus one payload per column. */ +const roundTripCase = fc + .uniqueArray(safeName, { minLength: 1, maxLength: 5 }) + .chain((names) => + fc.record({ + names: fc.constant(names), + kinds: fc.tuple( + ...names.map(() => fc.constantFrom('scalar' as const, 'json' as const)), + ), + cts: fc.tuple(...names.map(() => ciphertext)), + svs: fc.tuple(...names.map(() => steVecEntries)), + // The per-document SteVec KeyHeader (`h`). Opaque to the mapping, but + // protect-ffi 0.30 decrypt requires it, so it must survive the round trip. + hs: fc.tuple(...names.map(() => fc.string({ minLength: 1 }))), + hms: fc.tuple( + ...names.map(() => + fc.option(fc.string({ minLength: 1 }), { nil: undefined }), + ), + ), + plain: fc.string(), + }), + ) + +describe('property: write → read round trip', () => { + it('preserves every ciphertext and rebuilds the envelope, for any column set', () => { + fc.assert( + fc.property( + roundTripCase, + ({ names, kinds, cts, svs, hs, hms, plain }) => { + const table = encryptedTable( + 'users', + Object.fromEntries( + names.map((n, idx) => [ + n, + kinds[idx] === 'json' ? types.Json(n) : types.TextEq(n), + ]), + ), + ) + const encryptedAttrs = Object.keys(table.buildColumnKeyMap()) + + const item: Record = { [PLAINTEXT_KEY]: plain } + const expectedStored: Record = { + [PLAINTEXT_KEY]: plain, + } + const expectedRebuilt: Record = { + [PLAINTEXT_KEY]: plain, + } + + names.forEach((name, idx) => { + const i = { t: 'users', c: name } + if (kinds[idx] === 'json') { + // A SteVec document stores its `sv` entries plus the KeyHeader `h` + // (0.30 decrypt requires `h`); `v`/`i`/`k` are rebuilt on read. + item[name] = { v: 3, k: 'sv', i, h: hs[idx], sv: svs[idx] } + expectedStored[`${name}${ciphertextAttrSuffix}`] = { + h: hs[idx], + sv: svs[idx], + } + expectedRebuilt[name] = { + i: { c: name, t: 'users' }, + v: 3, + k: 'sv', + h: hs[idx], + sv: svs[idx], + } + return + } + item[name] = { + v: 3, + i, + c: cts[idx], + ...(hms[idx] === undefined ? {} : { hm: hms[idx] }), + } + expectedStored[`${name}${ciphertextAttrSuffix}`] = cts[idx] + if (hms[idx] !== undefined) { + expectedStored[`${name}${searchTermAttrSuffix}`] = hms[idx] + } + // The search term is a write-side index, not part of the envelope: + // the read path drops it rather than round-tripping it. + expectedRebuilt[name] = { + i: { c: name, t: 'users' }, + v: 3, + c: cts[idx], + } + }) + + const stored = toEncryptedDynamoItem(item, encryptedAttrs) + expect(stored).toEqual(expectedStored) + expect(toItemWithEqlPayloads(stored, table)).toEqual(expectedRebuilt) + }, + ), + ) + }) +}) + +// --------------------------------------------------------------------------- +// 2. PASSTHROUGH — pins two of the four review bugs +// --------------------------------------------------------------------------- + +describe('property: plaintext passthrough (both directions)', () => { + const users = encryptedTable('users', { + email: types.TextEq('email'), + note: types.Text('note'), + meta: types.Json('meta'), + }) + const encryptedAttrs = Object.keys(users.buildColumnKeyMap()) + + it('the write path returns an item with no payloads unchanged, at any depth', () => { + // BUG PINNED: a nested plaintext object with a `c` key was mistaken for a + // ciphertext, rewritten to `__source`, and every sibling key + // DISCARDED — silent data loss on a PutCommand. + fc.assert( + fc.property(plaintextItem, (item) => { + fc.pre(!looksLikeEnvelope(item)) + expect(toEncryptedDynamoItem(item, encryptedAttrs)).toEqual(item) + }), + { numRuns: 1000 }, + ) + }) + + it('the read path returns an item with no envelopes unchanged, at any depth', () => { + // BUG PINNED: ANY attribute ending in `__hmac` was dropped on read, so a + // customer's own `signature__hmac` vanished; and any nested `*__source` was + // handed to the FFI as if it were a ciphertext. + fc.assert( + fc.property(plaintextItem, (item) => { + expect(toItemWithEqlPayloads(item, users)).toEqual(item) + }), + { numRuns: 1000 }, + ) + }) + + // --------------------------------------------------------------------------- + // 3. NO ATTRIBUTE LOSS + // --------------------------------------------------------------------------- + + it('neither direction reduces the top-level key count of an all-plaintext item', () => { + fc.assert( + fc.property(plaintextItem, (item) => { + fc.pre(!looksLikeEnvelope(item)) + const keys = Object.keys(item) + const written = toEncryptedDynamoItem(item, encryptedAttrs) + const read = toItemWithEqlPayloads(item, users) + + expect(Object.keys(written).length).toBeGreaterThanOrEqual(keys.length) + expect(Object.keys(read).length).toBeGreaterThanOrEqual(keys.length) + for (const key of keys) { + expect(written).toHaveProperty([key]) + expect(read).toHaveProperty([key]) + } + }), + { numRuns: 1000 }, + ) + }) +}) + +// --------------------------------------------------------------------------- +// 4. deepClone +// --------------------------------------------------------------------------- + +/** Every object/array reachable from `value`, cycle-safe. */ +function objectRefs(value: unknown, seen = new Set()): Set { + if (value === null || typeof value !== 'object') return seen + if (seen.has(value)) return seen + seen.add(value) + for (const child of Object.values(value)) objectRefs(child, seen) + return seen +} + +describe('property: deepClone', () => { + // BUG PINNED: the previous hand-rolled `Object.entries` reduce flattened every + // non-plain object to `{}`, silently destroying `Date` values and making the + // whole `types.Timestamp*` / `types.Date*` domain family unusable here. + it('is identity-preserving over arbitrary JSON documents', () => { + fc.assert( + fc.property(fc.jsonValue(), (value) => { + expect(deepClone(value)).toEqual(value) + }), + ) + }) + + it('preserves a Date as a real Date with the same time', () => { + fc.assert( + fc.property(fc.date({ noInvalidDate: true }), (at) => { + const cloned = deepClone({ at, nested: { at } }) + expect(cloned.at).toBeInstanceOf(Date) + expect(cloned.at.getTime()).toBe(at.getTime()) + expect(cloned.nested.at).toBeInstanceOf(Date) + expect(cloned.nested.at.getTime()).toBe(at.getTime()) + }), + ) + }) + + it('shares no object reference with its source', () => { + fc.assert( + fc.property( + fc.dictionary(fc.string(), fc.jsonValue(), { minKeys: 1 }), + (value) => { + const cloned = deepClone(value) + const sourceRefs = objectRefs(value) + for (const ref of objectRefs(cloned)) { + expect(sourceRefs.has(ref)).toBe(false) + } + }, + ), + ) + }) +}) + +// --------------------------------------------------------------------------- +// 5. VERSION FOLLOWS THE TABLE +// --------------------------------------------------------------------------- + +describe('property: the rebuilt wire version follows the table', () => { + it('a v3 table always yields v: 3 and never a k: "ct" tag', () => { + fc.assert( + fc.property(safeName, ciphertext, (name, ct) => { + const table = encryptedTable('t', { [name]: types.TextEq(name) }) + expect(isV3Table(table)).toBe(true) + + const rebuilt = toItemWithEqlPayloads( + { [`${name}${ciphertextAttrSuffix}`]: ct }, + table, + )[name] as Record + + expect(rebuilt.v).toBe(3) + expect(rebuilt).not.toHaveProperty('k') + expect(rebuilt.c).toBe(ct) + }), + ) + }) + + it('a v3 JSON column keeps v: 3, the mandatory k: "sv", and the KeyHeader', () => { + fc.assert( + fc.property( + safeName, + steVecEntries, + fc.string({ minLength: 1 }), + (name, entries, h) => { + const table = encryptedTable('t', { [name]: types.Json(name) }) + + const rebuilt = toItemWithEqlPayloads( + { [`${name}${ciphertextAttrSuffix}`]: { h, sv: entries } }, + table, + )[name] as Record + + expect(rebuilt.v).toBe(3) + expect(rebuilt.k).toBe('sv') + // The per-document KeyHeader survives — 0.30 decrypt requires it. + expect(rebuilt.h).toBe(h) + expect(rebuilt.sv).toEqual(entries) + }, + ), + ) + }) + + it('a v2 table always yields v: 2 with k: "ct"', () => { + fc.assert( + fc.property(safeName, ciphertext, (name, ct) => { + const table = encryptedTableV2('t', { + [name]: encryptedColumn(name).equality(), + }) + expect(isV3Table(table)).toBe(false) + + const rebuilt = toItemWithEqlPayloads( + { [`${name}${ciphertextAttrSuffix}`]: ct }, + table, + )[name] as Record + + expect(rebuilt.v).toBe(2) + expect(rebuilt.k).toBe('ct') + expect(rebuilt.c).toBe(ct) + }), + ) + }) + + it('a v2 table with a grouped field still yields v: 2 with k: "ct"', () => { + fc.assert( + fc.property(safeName, safeName, ciphertext, (group, leaf, ct) => { + const table = encryptedTableV2('t', { + [group]: { [leaf]: encryptedField(`${group}.${leaf}`) }, + }) + expect(isV3Table(table)).toBe(false) + + const rebuilt = toItemWithEqlPayloads( + { [group]: { [`${leaf}${ciphertextAttrSuffix}`]: ct } }, + table, + )[group] as Record> + + expect(rebuilt[leaf]?.v).toBe(2) + expect(rebuilt[leaf]?.k).toBe('ct') + }), + ) + }) +}) + +// --------------------------------------------------------------------------- +// 6. NO ENVELOPE METADATA LEAK +// --------------------------------------------------------------------------- + +/** Does any value in this tree carry an envelope metadata key? */ +function hasEnvelopeMetadata(value: unknown): boolean { + if (value === null || typeof value !== 'object') return false + if (Array.isArray(value)) return value.some(hasEnvelopeMetadata) + if ('k' in value || 'v' in value || 'i' in value) return true + return Object.values(value).some(hasEnvelopeMetadata) +} + +describe('property: no envelope metadata reaches storage', () => { + it('a split scalar writes only __source (and __hmac when hm is present)', () => { + fc.assert( + fc.property( + safeName, + ciphertext, + fc.option(fc.string({ minLength: 1 }), { nil: undefined }), + // Terms with no DynamoDB query surface: they must be dropped, not stored. + fc.record({ + op: fc.option(fc.string(), { nil: undefined }), + ob: fc.option(fc.array(fc.string()), { nil: undefined }), + bf: fc.option(fc.array(fc.integer()), { nil: undefined }), + }), + (name, ct, hm, noise) => { + const table = encryptedTable('t', { [name]: types.TextEq(name) }) + const stored = toEncryptedDynamoItem( + { + [name]: { + v: 3, + i: { t: 't', c: name }, + c: ct, + ...(hm === undefined ? {} : { hm }), + ...noise, + }, + }, + Object.keys(table.buildColumnKeyMap()), + ) + + const expectedKeys = [`${name}${ciphertextAttrSuffix}`] + if (hm !== undefined) { + expectedKeys.push(`${name}${searchTermAttrSuffix}`) + } + expect(Object.keys(stored).sort()).toEqual(expectedKeys.sort()) + expect(stored[`${name}${ciphertextAttrSuffix}`]).toBe(ct) + for (const value of Object.values(stored)) { + expect(hasEnvelopeMetadata(value)).toBe(false) + } + }, + ), + ) + }) + + it('a ste_vec document is stored as its entries plus KeyHeader, no envelope metadata', () => { + fc.assert( + fc.property( + safeName, + steVecEntries, + fc.string({ minLength: 1 }), + (name, entries, h) => { + const table = encryptedTable('t', { [name]: types.Json(name) }) + const stored = toEncryptedDynamoItem( + { + [name]: { v: 3, k: 'sv', i: { t: 't', c: name }, h, sv: entries }, + }, + Object.keys(table.buildColumnKeyMap()), + ) + + expect(Object.keys(stored)).toEqual([ + `${name}${ciphertextAttrSuffix}`, + ]) + expect(stored[`${name}${ciphertextAttrSuffix}`]).toEqual({ + h, + sv: entries, + }) + // `v`/`i`/`k` are reconstructed on read, never stored. + expect(hasEnvelopeMetadata(stored)).toBe(false) + }, + ), + ) + }) +}) + +// --------------------------------------------------------------------------- +// 7. PROPERTY vs DB NAME — the most severe bug found in review +// --------------------------------------------------------------------------- + +describe('property: attributes are keyed by property name, identified by DB name', () => { + // BUG PINNED: the two names were conflated. A column declared + // `emailAddress: types.TextEq('email_address')` was matched against the DB + // name, so the attribute was never split on write, its `__hmac` was never + // emitted (killing every key-condition query on it), and on read the mapping + // recursed into the payload instead of rebuilding it. + const distinctNames = fc + .tuple(safeName, safeName) + .filter(([property, dbName]) => property !== dbName) + + it('splits under the property name and rebuilds i.c as the DB name', () => { + fc.assert( + fc.property( + distinctNames, + ciphertext, + fc.string({ minLength: 1 }), + ([property, dbName], ct, hm) => { + const table = encryptedTable('t', { + [property]: types.TextEq(dbName), + }) + + const stored = toEncryptedDynamoItem( + { [property]: { v: 3, i: { t: 't', c: dbName }, c: ct, hm } }, + Object.keys(table.buildColumnKeyMap()), + ) + + // Keyed by the PROPERTY name — the DB name never appears in storage. + expect(stored).toEqual({ + [`${property}${ciphertextAttrSuffix}`]: ct, + [`${property}${searchTermAttrSuffix}`]: hm, + }) + + // Identified by the DB name — this is what the FFI looks up. + expect(toItemWithEqlPayloads(stored, table)).toEqual({ + [property]: { i: { c: dbName, t: 't' }, v: 3, c: ct }, + }) + }, + ), + ) + }) +}) + +// --------------------------------------------------------------------------- +// 8. ARRAY CARVE-OUT: the mapping descends into objects, never arrays +// --------------------------------------------------------------------------- + +describe('property: a payload inside an array is stored and read whole', () => { + // The write and read paths both skip arrays (`!Array.isArray`), so a real + // envelope wrapped in a list is stored as its whole self — never split into + // `__source`/`__hmac` — and passes back through unchanged, still + // decryptable. A regression that started recursing arrays would split the + // element and break this for any generated column/array-key combination. + it('never splits an array-nested envelope, in either direction, for any column set', () => { + fc.assert( + fc.property(safeName, safeName, ciphertext, (col, arrKey, ct) => { + const table = encryptedTable('t', { [col]: types.TextEq(col) }) + const attrs = Object.keys(table.buildColumnKeyMap()) + // A genuine payload (v+i+c+hm) wrapped in an array — under an arbitrary + // key, declared or not: arrays are skipped regardless. + const item = { + [arrKey]: [{ v: 3, i: { t: 't', c: col }, c: ct, hm: 'H' }], + } + + const stored = toEncryptedDynamoItem(item, attrs, true) + expect(stored).toEqual(item) + expect(toItemWithEqlPayloads(stored, table)).toEqual(item) + }), + ) + }) +}) + +// --------------------------------------------------------------------------- +// 9. COLUMN-AWARE WRITE GATE: split only declared columns +// --------------------------------------------------------------------------- + +describe('property: the write path never splits a payload naming no declared column', () => { + // The write path is column-aware, matched on the same property path the read + // path rebuilds from. A pre-encrypted payload placed under an UNDECLARED name + // — top-level or nested — must be left whole and round-trip, never split into + // a `__source` the read path (which rebuilds only declared columns) + // could never reassemble. + it('leaves an undeclared payload whole and round-trips it, top-level and nested', () => { + fc.assert( + fc.property(safeName, safeName, ciphertext, (col, other, ct) => { + fc.pre(col !== other) + const table = encryptedTable('t', { [col]: types.TextEq(col) }) + const attrs = Object.keys(table.buildColumnKeyMap()) + const payload = { v: 3, i: { t: 't', c: other }, c: ct, hm: 'H' } + const item = { [other]: payload, grp: { [other]: payload } } + + const stored = toEncryptedDynamoItem(item, attrs, true) + expect(stored).toEqual(item) + expect(toItemWithEqlPayloads(stored, table)).toEqual(item) + }), + ) + }) +}) diff --git a/packages/stack/__tests__/dynamodb/resolve-decrypt.test.ts b/packages/stack/__tests__/dynamodb/resolve-decrypt.test.ts new file mode 100644 index 000000000..f742afb9d --- /dev/null +++ b/packages/stack/__tests__/dynamodb/resolve-decrypt.test.ts @@ -0,0 +1,117 @@ +/** + * Pure unit tests for the two client-shape helpers that bridge the nominal + * `EncryptionClient` (chainable, carries `.audit()`) and the typed client from + * `EncryptionV3` (plain `Promise`). Both branches were previously only + * reachable through a live ZeroKMS decrypt; these move that assurance onto the + * pure CI lane. No credentials, no network. + */ +import { afterEach, describe, expect, it, vi } from 'vitest' +import { resolveDecryptResult, throwPreservingCode } from '@/dynamodb/helpers' +import { logger } from '@/utils/logger' + +// The metadata-drop tests `vi.spyOn(logger, 'debug')` — the same shared singleton +// `helpers-v3.test.ts` also spies. Each test restores its own spy in a `finally`; +// this hook is the safety net so a patched method can never survive a test +// boundary (`restoreAllMocks` un-patches; `clearAllMocks` only clears calls). +afterEach(() => { + vi.restoreAllMocks() +}) + +describe('resolveDecryptResult', () => { + it('awaits a plain promise when the operation has no .audit (typed client)', async () => { + const result = await resolveDecryptResult( + Promise.resolve({ data: { x: 1 } }), + { metadata: { ignored: true } }, + ) + + expect(result).toEqual({ data: { x: 1 } }) + }) + + it('chains .audit and forwards metadata when present (nominal client)', async () => { + let seen: unknown + const operation = { + audit(config: { metadata?: Record }) { + seen = config.metadata + return Promise.resolve({ data: { y: 2 } }) + }, + } + + const result = await resolveDecryptResult(operation, { + metadata: { m: 42 }, + }) + + expect(result).toEqual({ data: { y: 2 } }) + expect(seen).toEqual({ m: 42 }) + }) + + it('propagates a failure result unchanged', async () => { + const failure = { failure: { message: 'boom', code: 'X' } } + + await expect( + resolveDecryptResult(Promise.resolve(failure), { metadata: {} }), + ).resolves.toEqual(failure) + }) + + it('returns a failure — not a silent undefined success — for a malformed result', async () => { + // A non-conforming client that resolves to a bare value (or `{}`) has + // neither `data` nor `failure`. Casting it straight through would surface a + // fake success carrying `undefined`; the shape must be rejected instead. + for (const malformed of [{}, 42, undefined]) { + const result = await resolveDecryptResult(Promise.resolve(malformed), {}) + + expect(result.failure).toBeDefined() + expect(typeof result.failure?.message).toBe('string') + expect(result.data).toBeUndefined() + } + }) + + it('logs when audit metadata is dropped for a non-chainable operation', async () => { + const spy = vi.spyOn(logger, 'debug').mockImplementation(() => {}) + + try { + await resolveDecryptResult(Promise.resolve({ data: { x: 1 } }), { + metadata: { m: 42 }, + }) + + expect(spy).toHaveBeenCalledWith( + expect.stringContaining('audit metadata ignored'), + ) + } finally { + spy.mockRestore() + } + }) + + it('does not log about audit metadata when none is passed', async () => { + const spy = vi.spyOn(logger, 'debug').mockImplementation(() => {}) + + try { + await resolveDecryptResult(Promise.resolve({ data: { x: 1 } }), {}) + + expect(spy).not.toHaveBeenCalled() + } finally { + spy.mockRestore() + } + }) +}) + +describe('throwPreservingCode', () => { + it('rethrows as an Error carrying the FFI code', () => { + try { + throwPreservingCode({ message: 'boom', code: 'UNKNOWN_COLUMN' }) + expect.unreachable('should have thrown') + } catch (error) { + expect(error).toBeInstanceOf(Error) + expect((error as Error).message).toBe('boom') + expect((error as { code?: string }).code).toBe('UNKNOWN_COLUMN') + } + }) + + it('tolerates a missing code', () => { + try { + throwPreservingCode({ message: 'boom' }) + expect.unreachable('should have thrown') + } catch (error) { + expect((error as { code?: string }).code).toBeUndefined() + } + }) +}) diff --git a/packages/stack/src/dynamodb/helpers.ts b/packages/stack/src/dynamodb/helpers.ts index 812692d7d..68966d409 100644 --- a/packages/stack/src/dynamodb/helpers.ts +++ b/packages/stack/src/dynamodb/helpers.ts @@ -1,13 +1,27 @@ import type { ProtectErrorCode } from '@cipherstash/protect-ffi' import { ProtectError as FfiProtectError } from '@cipherstash/protect-ffi' -import type { EncryptedTable, EncryptedTableColumn } from '@/schema' +import { resolveEncryptColumnMap } from '@/encryption/helpers/model-helpers' +import type { AnyV3Table } from '@/eql/v3' import type { EncryptedValue } from '@/types' +import { hasBuildColumnKeyMap } from '@/types' import { logger } from '@/utils/logger' -import type { EncryptedDynamoDBError } from './types' +import type { AnyEncryptedTable, EncryptedDynamoDBError } from './types' export const ciphertextAttrSuffix = '__source' export const searchTermAttrSuffix = '__hmac' +/** + * Which EQL wire version to synthesize when rebuilding an envelope from the + * stored `__source` attribute. + * + * `buildColumnKeyMap` is the canonical v3 marker in this codebase — see + * `resolveEqlVersion` (`encryption/index.ts`) and `types.ts`, which document it + * as *the* signal. Only v3 tables define it. + */ +export function isV3Table(table: AnyEncryptedTable): table is AnyV3Table { + return hasBuildColumnKeyMap(table) +} + export class EncryptedDynamoDBErrorImpl extends Error implements EncryptedDynamoDBError @@ -69,70 +83,276 @@ export function handleError( return dynamoError } +/** + * Resolve a decrypt call against either client shape. + * + * The nominal `EncryptionClient` returns a chainable operation that carries + * `.audit()`; the `TypedEncryptionClient` from `EncryptionV3` returns a plain + * `Promise>`. Chain the audit metadata when the client can carry it, + * otherwise await the promise directly — a typed client has no audit surface + * on decrypt, so the metadata has nowhere to go. + */ +export async function resolveDecryptResult( + operation: unknown, + auditData: { metadata?: Record }, +): Promise< + { data: T; failure?: never } | { data?: never; failure: DecryptFailure } +> { + const chainable = operation as { + audit?: (data: { metadata?: Record }) => unknown + } + + if (typeof chainable?.audit !== 'function' && auditData.metadata) { + // The typed EncryptionV3 client returns a plain promise with no decrypt + // audit surface, so the metadata has nowhere to go. Make the drop + // observable rather than silent — use the nominal client for audited + // decrypts. + logger.debug( + 'DynamoDB: decrypt audit metadata ignored — the typed client has no decrypt audit surface; use Encryption({ config: { eqlVersion: 3 } }) for audited decrypts.', + ) + } + + const resolved = + typeof chainable?.audit === 'function' + ? await chainable.audit(auditData) + : await operation + + // A conforming client resolves to `{ data }` or `{ failure }`. A bare value + // (or wrong shape) has neither, so casting it straight through would surface a + // fake success carrying `undefined`. Reject it as a failure instead. + if ( + resolved === null || + typeof resolved !== 'object' || + (!('data' in resolved) && !('failure' in resolved)) + ) { + return { + failure: { + message: + 'DynamoDB: decrypt returned a malformed result — expected { data } or { failure }.', + }, + } + } + + return resolved as + | { data: T; failure?: never } + | { data?: never; failure: DecryptFailure } +} + +type DecryptFailure = { message: string; code?: string } + +/** + * Rethrow a Result failure as an `Error` that preserves the FFI error code. + * `withResult`'s `ensureError` wraps non-Error objects, which would otherwise + * lose the code before `handleError` can read it. + */ +export function throwPreservingCode(failure: { + message: string + code?: string +}): never { + const error = new Error(failure.message) as Error & { code?: string } + error.code = failure.code + throw error +} + +/** + * Clone caller input before it reaches the FFI, so encryption never mutates a + * caller's object. + * + * `structuredClone` rather than a hand-rolled `Object.entries` reduce: the + * reduce flattened every non-plain object to `{}`, which silently destroyed + * `Date` values — making the whole `types.Timestamp*` / `types.Date*` domain + * family unusable through this adapter — and blew the stack on a circular + * reference. `structuredClone` handles Date, Map, Set, TypedArray and cycles + * natively. Values it cannot structurally clone (a function/symbol/WeakMap-valued + * property) fall back to a per-key shallow copy into a FRESH object rather than + * throwing — the offending value passes through by reference, every other key + * is copied, so the caller's original is never handed to the FFI. + */ export function deepClone(obj: T): T { if (obj === null || typeof obj !== 'object') { return obj } - if (Array.isArray(obj)) { - return obj.map((item) => deepClone(item)) as unknown as T + try { + return structuredClone(obj) + } catch { + // `structuredClone` rejects function/symbol/WeakMap-valued properties. Fall + // back to a per-key shallow copy into a FRESH object so the caller's + // original is never handed to the FFI — otherwise the docblock guarantee + // "encryption never mutates a caller's object" is silently voided. The + // offending value passes through by reference; every other key is copied, + // and a class instance is flattened to a plain object (prototype dropped). + if (Array.isArray(obj)) { + return [...obj] as T + } + return { ...(obj as Record) } as T } +} - return Object.entries(obj as Record).reduce( - (acc, [key, value]) => ({ - // biome-ignore lint/performance/noAccumulatingSpread: TODO later - ...acc, - [key]: deepClone(value), - }), - {} as T, - ) +/** + * The storage payloads this adapter splits into DynamoDB attributes, across + * both wire versions. + * + * `EncryptedValue` from `@/types` cannot describe these: its union is + * `EncryptedScalar | EncryptedSteVec`, both v2-only, so it has no member + * matching an untagged v3 scalar and reading `c`/`hm` off it does not compile + * once the `k: 'ct'` narrowing is gone. The mapping only ever touches four + * keys, so state them structurally. + */ +type StoredEqlPayload = { + /** Wire version. Present on every payload in both versions. */ + v?: unknown + /** Identifier `{ t, c }`. Present on every payload in both versions. */ + i?: unknown + /** Absent on v3 scalars; `'ct'` on v2 scalars; `'sv'` on JSON in both. */ + k?: 'ct' | 'sv' + /** Scalar ciphertext. */ + c?: unknown + /** Deterministic equality term — the only one DynamoDB can query. */ + hm?: unknown + /** ste_vec entries for a JSON document. */ + sv?: unknown + /** + * Per-document SteVec KeyHeader. Present on a v3 JSON (`k: 'sv'`) document; + * protect-ffi 0.30 decrypt requires it, so it is stored alongside `sv`. + */ + h?: unknown +} + +/** + * Is this value an EQL payload, as opposed to a plaintext object that merely + * looks like one? + * + * `v` and `i` are the discriminators, deliberately: they are the same keys the + * FFI itself uses to decide whether a value is encrypted, and every payload in + * both wire versions carries both. Testing for a ciphertext alone is not + * enough — `c` is an ordinary attribute name (country, currency, count), and + * treating `{ c: 'AU', d: 1 }` as a payload silently rewrites it to + * `__source` and DISCARDS every sibling key. + * + * The shape checks (`v` a number, `i` an object) mirror the core + * `isEncryptedPayload` exactly, so the two envelope detectors cannot drift + * apart. The `matchColumn` gate already means this only runs against values at + * a declared encrypted column, but pinning the same predicate closes the + * residual window where a plaintext `{ v, i }` lookalike could be split. + */ +function isStoredEqlPayload(value: unknown): value is StoredEqlPayload { + if (value === null || typeof value !== 'object') return false + const obj = value as Record + if (!('v' in obj) || typeof obj.v !== 'number') return false + if (!('i' in obj) || typeof obj.i !== 'object') return false + return 'c' in obj || 'sv' in obj +} + +/** + * Resolve an attribute (`leaf` under `prefix`) back to the column path it was + * declared under, or `undefined` if it names no declared column. Shared by the + * write and read paths so the two split and rebuild EXACTLY the same set of + * attributes — an asymmetry here writes data one side can never reassemble. + * + * `columnPaths` are the JS property paths a model's fields are matched on. The + * dotted form is tried first; a v2 grouped `encryptedField('amount')` registers + * the bare leaf, so a nested `amount` falls back to it. v3 always registers the + * full dotted path (`'profile.ssn'`), so it never needs the fallback — and must + * NOT use it, or a nested `note` would match a same-named TOP-LEVEL `note` + * column. Scope the fallback to nested v2 attributes only. + */ +export function makeColumnMatcher(isV3: boolean, columnPaths: string[]) { + return function matchColumn( + leaf: string, + prefix: string, + ): string | undefined { + const dotted = prefix ? `${prefix}.${leaf}` : leaf + if (columnPaths.includes(dotted)) return dotted + if (!isV3 && prefix && columnPaths.includes(leaf)) return leaf + return undefined + } } export function toEncryptedDynamoItem( encrypted: Record, encryptedAttrs: string[], + // `false` (v2) by default so existing 2-arg callers keep the v2 bare-leaf + // fallback; the operations pass `isV3Table(table)` so a v3 write splits the + // same columns a v3 read rebuilds. + isV3 = false, ): Record { + const matchColumn = makeColumnMatcher(isV3, encryptedAttrs) + function processValue( attrName: string, attrValue: unknown, - isNested: boolean, + prefix: string, ): Record { if (attrValue === null || attrValue === undefined) { return { [attrName]: attrValue } } - // Handle encrypted payload - if ( - encryptedAttrs.includes(attrName) || - (isNested && - typeof attrValue === 'object' && - 'c' in (attrValue as object)) - ) { - const encryptPayload = attrValue as EncryptedValue - if (encryptPayload?.k === 'ct' && encryptPayload.c) { + // Handle encrypted payload. Split only a value that BOTH is a payload and + // names a declared column — matched on its property path, exactly as the + // read path rebuilds it. Splitting an undeclared nested payload (matched by + // shape alone) would write a `__source` the read path never + // reassembles, i.e. silent undecryptable data. + if (matchColumn(attrName, prefix) && isStoredEqlPayload(attrValue)) { + const encryptPayload = attrValue + + // A JSON document keeps its `k: 'sv'` tag. Its index terms live *inside* + // the `sv` entries, so there is no separate search-term attribute to split + // out. Store the `sv` entries together with the per-document KeyHeader + // `h`: protect-ffi 0.30's SteVec decrypt requires `h` (there is no root + // `c` to reconstruct from) and it is not derivable, whereas `v`/`i`/`k` + // ARE reconstructed on read and so are kept out of the stored attribute. + if (encryptPayload?.k === 'sv' && encryptPayload.sv) { const result: Record = {} - if (encryptPayload.hm) { - result[`${attrName}${searchTermAttrSuffix}`] = encryptPayload.hm + result[`${attrName}${ciphertextAttrSuffix}`] = { + h: encryptPayload.h, + sv: encryptPayload.sv, } - result[`${attrName}${ciphertextAttrSuffix}`] = encryptPayload.c return result } - if (encryptPayload?.k === 'sv' && encryptPayload.sv) { + // Scalars. v2 tags every payload `k: 'ct'`; v3 scalars carry NO `k` + // discriminator at all, so the presence of a ciphertext is the signal — + // gating on `k === 'ct'` would drop every v3 scalar through to the + // nested-object branch below and write it out as a raw map. Test + // PRESENCE, not truthiness: a `{ v, i, c: '' }` payload is still a + // payload, and letting it fall through would leak `v`/`i` into storage. + if ('c' in encryptPayload) { const result: Record = {} - result[`${attrName}${ciphertextAttrSuffix}`] = encryptPayload.sv + // `hm` is the deterministic equality term, and the only one a DynamoDB + // key condition can use. Ordering terms (`op`/`ob`) and the match + // bloom filter (`bf`) have no DynamoDB query surface, so they are not + // stored — the value stays decryptable, it is just not orderable or + // text-searchable inside DynamoDB. + if (encryptPayload.hm) { + result[`${attrName}${searchTermAttrSuffix}`] = encryptPayload.hm + } + result[`${attrName}${ciphertextAttrSuffix}`] = encryptPayload.c return result } } - // Handle nested objects recursively + // Handle nested objects recursively, carrying the path so a nested column + // is matched against its registered dotted name. + // + // Arrays are a deliberate carve-out: they are NOT recursed into, so a + // payload inside an array is stored as its whole envelope rather than split + // into `__source`/`__hmac`. A DynamoDB key condition cannot + // target an array element anyway, so there is nothing to gain from a split; + // the read path skips arrays symmetrically, so such a value still + // round-trips and decrypts. Documented in the DynamoDB skill's limitations. if (typeof attrValue === 'object' && !Array.isArray(attrValue)) { const nestedResult = Object.entries( attrValue as Record, ).reduce( (acc, [key, val]) => { - const processed = processValue(key, val, true) - return Object.assign({}, acc, processed) + const processed = processValue( + key, + val, + prefix ? `${prefix}.${attrName}` : attrName, + ) + Object.assign(acc, processed) + return acc }, {} as Record, ) @@ -145,78 +365,158 @@ export function toEncryptedDynamoItem( return Object.entries(encrypted).reduce( (putItem, [attrName, attrValue]) => { - const processed = processValue(attrName, attrValue, false) - return Object.assign({}, putItem, processed) + const processed = processValue(attrName, attrValue, '') + Object.assign(putItem, processed) + return putItem }, {} as Record, ) } +/** + * The row-invariant table facts the read path needs. `build()` and + * `resolveEncryptColumnMap` are not memoized, so a bulk decrypt of N items must + * build this ONCE and reuse it — see `buildReadContext`. `columnPaths` are the + * keys a model's fields are MATCHED on (JS property names); `toColumnName` maps + * a match to the name the FFI config is keyed by. For v3 those differ whenever + * a column is declared `emailAddress: types.TextEq('email_address')`. + */ +export type ReadContext = { + isV3: boolean + v: 2 | 3 + encryptConfig: { + tableName: string + columns: Record< + string, + { cast_as?: string; indexes: Record } + > + } + columnPaths: string[] + toColumnName: (path: string) => string +} + +/** Resolve the row-invariant read context for a table, once. */ +export function buildReadContext(schema: AnyEncryptedTable): ReadContext { + const isV3 = isV3Table(schema) + const { columnPaths, toColumnName } = resolveEncryptColumnMap(schema) + return { + isV3, + v: isV3 ? 3 : 2, + encryptConfig: schema.build(), + columnPaths, + toColumnName, + } +} + export function toItemWithEqlPayloads( decrypted: Record, - encryptionSchema: EncryptedTable, + encryptionSchema: AnyEncryptedTable, + // Resolved once here for the single-item path; passed in by the bulk path so + // it is not rebuilt per item. + context: ReadContext = buildReadContext(encryptionSchema), ): Record { + const { isV3, v, encryptConfig, columnPaths, toColumnName } = context + + // The same matcher the write path splits with, so the two stay symmetric. + const matchColumn = makeColumnMatcher(isV3, columnPaths) + function processValue( attrName: string, attrValue: unknown, - isNested: boolean, + prefix = '', ): Record { if (attrValue === null || attrValue === undefined) { return { [attrName]: attrValue } } - // Skip HMAC fields + // Drop the search term — but only when it belongs to a column we know + // about. An unrelated customer attribute that merely ends in `__hmac` + // (an app-level signature, say) must survive the read untouched. if (attrName.endsWith(searchTermAttrSuffix)) { - return {} + const term = attrName.slice(0, -searchTermAttrSuffix.length) + if (matchColumn(term, prefix)) return {} + return { [attrName]: attrValue } } - const encryptConfig = encryptionSchema.build() - const encryptedAttrs = Object.keys(encryptConfig.columns) const columnName = attrName.slice(0, -ciphertextAttrSuffix.length) - // Handle encrypted payload - if ( - attrName.endsWith(ciphertextAttrSuffix) && - (encryptedAttrs.includes(columnName) || isNested) - ) { - const i = { c: columnName, t: encryptConfig.tableName } - const v = 2 - - // Nested values are not searchable, so we can just return the standard EQL payload. - // Worth noting, that encryptConfig.columns[columnName] will be undefined if isNested is true. - if ( - !isNested && - encryptConfig.columns[columnName].cast_as === 'json' && - encryptConfig.columns[columnName].indexes.ste_vec - ) { + // Resolve the attribute back to a declared column. `matchColumn` prefers + // the dotted path (`encryptedField('example.protected')`, and v3's dotted + // property form) and falls back to the bare leaf (`encryptedField('amount')` + // under a `details` group), so both authoring conventions resolve. + const matched = matchColumn(columnName, prefix) + + // A stored ciphertext attribute that names no declared column is almost + // always a schema rename: the value reads back as raw base64 with no error. + // Leave the passthrough behaviour intact (an unmatched `*__source` is not an + // envelope), but make the silent drop observable. + if (attrName.endsWith(ciphertextAttrSuffix) && !matched) { + const dotted = prefix ? `${prefix}.${attrName}` : attrName + logger.debug( + `DynamoDB: attribute "${dotted}" ends in ${ciphertextAttrSuffix} but names no declared column — passing it through as raw ciphertext (a column rename can cause this).`, + ) + } + + // Handle encrypted payload. An unmatched attribute is NOT an envelope, even + // when nested — previously `|| isNested` made every nested `*__source` a + // decrypt target, so an unrelated customer attribute was handed to the FFI. + if (attrName.endsWith(ciphertextAttrSuffix) && matched) { + // Match on the property name, but identify by the DB column name — they + // differ whenever a v3 column is declared + // `emailAddress: types.TextEq('email_address')`. + const i = { c: toColumnName(matched), t: encryptConfig.tableName } + + // A JSON document is stored as `{ h, sv }` (see the write path); rebuild + // the full SteVec envelope around it. protect-ffi 0.30 deserialization is + // strict — it requires `k` ("missing field `k`") and the per-document + // KeyHeader `h` ("missing field `h`"), and there is no root `c`. `i`/`v`/`k` + // are reconstructed; `h`/`sv` come from storage. Look the config up by the + // resolved identifier so a nested JSON column (registered under a dotted + // path) is detected too. A v3 column builds the same `{ cast_as, indexes }` + // shape as a v2 one, so this detection needs no version branch. + const columnConfig = encryptConfig.columns[toColumnName(matched)] + if (columnConfig?.cast_as === 'json' && columnConfig.indexes.ste_vec) { + const stored = attrValue as { h?: unknown; sv?: unknown } return { [columnName]: { i, v, k: 'sv', - sv: attrValue, + h: stored.h, + sv: stored.sv, }, } } + // v3 scalars are untagged; only v2 carries `k: 'ct'`. Both versions + // require `v` and `i` — a payload missing either is not recognized as + // encrypted and is returned to the caller verbatim rather than erroring. return { [columnName]: { i, v, - k: 'ct', + ...(v === 2 ? { k: 'ct' } : {}), c: attrValue, }, } } - // Handle nested objects recursively + // Handle nested objects recursively, carrying the path so a nested column + // can be matched against its registered dotted name. Arrays are skipped + // symmetrically with the write path (which stores array-nested payloads + // whole), so such a value passes straight to the FFI and still decrypts. if (typeof attrValue === 'object' && !Array.isArray(attrValue)) { const nestedResult = Object.entries( attrValue as Record, ).reduce( (acc, [key, val]) => { - const processed = processValue(key, val, true) - return Object.assign({}, acc, processed) + const processed = processValue( + key, + val, + prefix ? `${prefix}.${attrName}` : attrName, + ) + Object.assign(acc, processed) + return acc }, {} as Record, ) @@ -229,8 +529,9 @@ export function toItemWithEqlPayloads( return Object.entries(decrypted).reduce( (formattedItem, [attrName, attrValue]) => { - const processed = processValue(attrName, attrValue, false) - return Object.assign({}, formattedItem, processed) + const processed = processValue(attrName, attrValue) + Object.assign(formattedItem, processed) + return formattedItem }, {} as Record, ) diff --git a/packages/stack/src/dynamodb/index.ts b/packages/stack/src/dynamodb/index.ts index 4d54ad53a..bce47df8b 100644 --- a/packages/stack/src/dynamodb/index.ts +++ b/packages/stack/src/dynamodb/index.ts @@ -1,14 +1,66 @@ -import type { EncryptedTable, EncryptedTableColumn } from '@/schema' import type { EncryptedValue } from '@/types' +import { isV3Table } from './helpers' import { BulkDecryptModelsOperation } from './operations/bulk-decrypt-models' import { BulkEncryptModelsOperation } from './operations/bulk-encrypt-models' import { DecryptModelOperation } from './operations/decrypt-model' import { EncryptModelOperation } from './operations/encrypt-model' import type { + AnyEncryptedTable, EncryptedDynamoDBConfig, EncryptedDynamoDBInstance, } from './types' +/** + * Fail fast on an EQL version mismatch between the client and the table. + * + * A v3 table encrypted through a client that is NOT in EQL v3 mode for it — a + * v2-mode client, or one initialised for a different schema set — otherwise + * surfaces only much later as an opaque FFI deserialization error, far from the + * misconfiguration that caused it. + * + * The client does not expose its EQL wire version directly (it is baked into the + * FFI client at `newClient` time and neither the nominal `EncryptionClient` nor + * the typed `EncryptionV3` wrapper re-surfaces it). The reliable, public signal + * both client shapes DO expose is `getEncryptConfig()`: a v3 table can only be + * encrypted by a client that registered it, and a client built for v3 registers + * it under its `tableName`. So the guard fires only when we can PROVE the table + * is absent from a readable config — never on an unreadable one, to avoid a + * false positive. + * + * This runs at the first point where both the client and a concrete table are in + * hand: the operation methods below, when a table is supplied. `encryptedDynamoDB` + * itself receives no table, so it cannot check earlier. + * + * RESIDUAL GAP: a client explicitly forced to `eqlVersion: 2` over a v3 schema + * set (a deliberate migration path) DOES register the table yet emits v2 wire; + * that is not detectable without a wire-version accessor the client does not + * provide, so it is out of scope here. + */ +function assertClientTableVersionMatch( + encryptionClient: EncryptedDynamoDBConfig['encryptionClient'], + table: AnyEncryptedTable, +): void { + // Only v3 tables carry the strict wire-format requirement this guards. + if (!isV3Table(table)) return + + const getEncryptConfig = ( + encryptionClient as { + getEncryptConfig?: () => { tables?: Record } | undefined + } + ).getEncryptConfig + + // Without a readable config we cannot prove a mismatch — stay silent rather + // than false-positive on a client shape that does not expose it. + if (typeof getEncryptConfig !== 'function') return + const config = getEncryptConfig.call(encryptionClient) + if (!config?.tables) return + if (table.tableName in config.tables) return + + throw new Error( + `encryptedDynamoDB: EQL version mismatch — the EQL v3 table "${table.tableName}" is not registered with this encryption client, so the client is not in EQL v3 mode for it. A v3 table requires a v3-mode client: build it with EncryptionV3({ schemas: [] }) (or Encryption({ schemas: [
], config: { eqlVersion: 3 } })) and pass that client to encryptedDynamoDB. Otherwise encrypt/decrypt fails later inside the FFI with an opaque deserialization error.`, + ) +} + /** * Create an encrypted DynamoDB helper bound to an `EncryptionClient`. * @@ -16,11 +68,36 @@ import type { * and `bulkDecryptModels` methods that transparently encrypt/decrypt DynamoDB * items according to the provided table schema. * + * Accepts EQL v3 tables (`types.*` domains) and EQL v2 tables + * (`encryptedColumn`/`encryptedField`) alike — the table decides which wire + * format is synthesized on read. + * + * Only equality is meaningful on DynamoDB: an `hm` term is stored alongside the + * ciphertext as `__hmac` and can back a key condition. Ordering and + * free-text terms have no DynamoDB query surface and are not stored, so values + * in those domains remain decryptable but not searchable within DynamoDB. + * * @param config - Configuration containing the `encryptionClient` and optional * logging / error-handling callbacks. * @returns An {@link EncryptedDynamoDBInstance} with encrypt/decrypt operations. * - * @example + * @example EQL v3 + * ```typescript + * import { EncryptionV3, encryptedTable, types } from "@cipherstash/stack/v3" + * import { encryptedDynamoDB } from "@cipherstash/stack/dynamodb" + * + * const users = encryptedTable("users", { + * email: types.TextEq("email"), // equality → queryable via email__hmac + * name: types.Text("name"), // storage only + * }) + * + * const client = await EncryptionV3({ schemas: [users] }) + * const dynamo = encryptedDynamoDB({ encryptionClient: client }) + * + * const encrypted = await dynamo.encryptModel({ email: "a@b.com" }, users) + * ``` + * + * @example EQL v2 (existing deployments) * ```typescript * import { Encryption } from "@cipherstash/stack" * import { encryptedDynamoDB } from "@cipherstash/stack/dynamodb" @@ -32,8 +109,6 @@ import type { * * const client = await Encryption({ schemas: [users] }) * const dynamo = encryptedDynamoDB({ encryptionClient: client }) - * - * const encrypted = await dynamo.encryptModel({ email: "a@b.com" }, users) * ``` */ export function encryptedDynamoDB( @@ -41,58 +116,83 @@ export function encryptedDynamoDB( ): EncryptedDynamoDBInstance { const { encryptionClient, options } = config - return { - encryptModel>( - item: T, - table: EncryptedTable, - ) { - return new EncryptModelOperation( - encryptionClient, - item, - table, - options, - ) - }, + // The public interface is OVERLOADED per wire version (a v3 overload that + // knows the storage split, a v2 one that does not). A single generic + // implementation signature cannot be inferred as satisfying both arms — the + // v3 overload's return is the derived `EncryptedAttributes` storage split, + // which the erased implementation (built against `AnyEncryptedTable`) cannot + // reproduce. So each method is written once against the erased shape and cast + // to ITS OWN interface member, and the assembled object is annotated with the + // public type. Nothing about the runtime differs between the overloads — the + // table decides everything, and it does so at runtime. + // + // Why per-method casts rather than one object-level `as`: the object literal + // is checked against the annotation, so a method renamed/added/removed on the + // interface (drift between the four impls and the eight-overload public type) + // now fails to compile — a single `as EncryptedDynamoDBInstance` erased that. + // What a per-method cast still cannot check is the v3 RETURN precision + // (`EncryptedAttributes`), for the erasure reason above; that surface is + // locked instead by `__tests__/dynamodb/client-compat.test-d.ts`. + const encryptModel = >( + item: T, + table: AnyEncryptedTable, + ) => { + assertClientTableVersionMatch(encryptionClient, table) + return new EncryptModelOperation(encryptionClient, item, table, options) + } + + const bulkEncryptModels = >( + items: T[], + table: AnyEncryptedTable, + ) => { + assertClientTableVersionMatch(encryptionClient, table) + return new BulkEncryptModelsOperation( + encryptionClient, + items, + table, + options, + ) + } - bulkEncryptModels>( - items: T[], - table: EncryptedTable, - ) { - return new BulkEncryptModelsOperation( - encryptionClient, - items, - table, - options, - ) - }, + const decryptModel = >( + item: Record, + table: AnyEncryptedTable, + ) => { + assertClientTableVersionMatch(encryptionClient, table) + return new DecryptModelOperation(encryptionClient, item, table, options) + } - decryptModel>( - item: Record, - table: EncryptedTable, - ) { - return new DecryptModelOperation( - encryptionClient, - item, - table, - options, - ) - }, + const bulkDecryptModels = >( + items: Record[], + table: AnyEncryptedTable, + ) => { + assertClientTableVersionMatch(encryptionClient, table) + return new BulkDecryptModelsOperation( + encryptionClient, + items, + table, + options, + ) + } - bulkDecryptModels>( - items: Record[], - table: EncryptedTable, - ) { - return new BulkDecryptModelsOperation( - encryptionClient, - items, - table, - options, - ) - }, + const instance: EncryptedDynamoDBInstance = { + encryptModel: encryptModel as EncryptedDynamoDBInstance['encryptModel'], + bulkEncryptModels: + bulkEncryptModels as EncryptedDynamoDBInstance['bulkEncryptModels'], + decryptModel: decryptModel as EncryptedDynamoDBInstance['decryptModel'], + bulkDecryptModels: + bulkDecryptModels as EncryptedDynamoDBInstance['bulkDecryptModels'], } + + return instance } +export type { AuditConfig } from './operations/base-operation' export type { + AnyEncryptedTable, + DecryptedAttributes, + DynamoDBEncryptionClient, + EncryptedAttributes, EncryptedDynamoDBConfig, EncryptedDynamoDBError, EncryptedDynamoDBInstance, diff --git a/packages/stack/src/dynamodb/operations/bulk-decrypt-models.ts b/packages/stack/src/dynamodb/operations/bulk-decrypt-models.ts index 110e058df..d6d8b8b3f 100644 --- a/packages/stack/src/dynamodb/operations/bulk-decrypt-models.ts +++ b/packages/stack/src/dynamodb/operations/bulk-decrypt-models.ts @@ -1,10 +1,19 @@ import { type Result, withResult } from '@byteslice/result' -import type { EncryptionClient } from '@/encryption' -import type { EncryptedTable, EncryptedTableColumn } from '@/schema' import type { Decrypted, EncryptedValue } from '@/types' import { logger } from '@/utils/logger' -import { handleError, toItemWithEqlPayloads } from '../helpers' -import type { EncryptedDynamoDBError } from '../types' +import { + buildReadContext, + handleError, + resolveDecryptResult, + throwPreservingCode, + toItemWithEqlPayloads, +} from '../helpers' +import type { + AnyEncryptedTable, + CallableEncryptionClient, + DynamoDBEncryptionClient, + EncryptedDynamoDBError, +} from '../types' import { DynamoDBOperation, type DynamoDBOperationOptions, @@ -13,14 +22,14 @@ import { export class BulkDecryptModelsOperation< T extends Record, > extends DynamoDBOperation[]> { - private encryptionClient: EncryptionClient + private encryptionClient: DynamoDBEncryptionClient private items: Record[] - private table: EncryptedTable + private table: AnyEncryptedTable constructor( - encryptionClient: EncryptionClient, + encryptionClient: DynamoDBEncryptionClient, items: Record[], - table: EncryptedTable, + table: AnyEncryptedTable, options?: DynamoDBOperationOptions, ) { super(options) @@ -35,22 +44,23 @@ export class BulkDecryptModelsOperation< logger.debug(`DynamoDB: bulk decrypting ${this.items.length} models.`) return await withResult( async () => { + // Resolve the table's read context once, not once per item — `build()` + // and the column map are row-invariant. + const readContext = buildReadContext(this.table) const itemsWithEqlPayloads = this.items.map((item) => - toItemWithEqlPayloads(item, this.table), + toItemWithEqlPayloads(item, this.table, readContext), ) - const decryptResult = await this.encryptionClient - .bulkDecryptModels(itemsWithEqlPayloads as T[]) - .audit(this.getAuditData()) + const client = this.encryptionClient as CallableEncryptionClient + const decryptResult = await resolveDecryptResult[]>( + // The second argument is required by the typed client and ignored by + // the nominal one, which derives the table from the payloads. + client.bulkDecryptModels(itemsWithEqlPayloads, this.table), + this.getAuditData(), + ) if (decryptResult.failure) { - // Create an Error object that preserves the FFI error code - // This is necessary because withResult's ensureError wraps non-Error objects - const error = new Error(decryptResult.failure.message) as Error & { - code?: string - } - error.code = decryptResult.failure.code - throw error + throwPreservingCode(decryptResult.failure) } return decryptResult.data diff --git a/packages/stack/src/dynamodb/operations/bulk-encrypt-models.ts b/packages/stack/src/dynamodb/operations/bulk-encrypt-models.ts index 7f6cd0b75..eafee15aa 100644 --- a/packages/stack/src/dynamodb/operations/bulk-encrypt-models.ts +++ b/packages/stack/src/dynamodb/operations/bulk-encrypt-models.ts @@ -1,9 +1,19 @@ import { type Result, withResult } from '@byteslice/result' -import type { EncryptionClient } from '@/encryption' -import type { EncryptedTable, EncryptedTableColumn } from '@/schema' +import { resolveEncryptColumnMap } from '@/encryption/helpers/model-helpers' import { logger } from '@/utils/logger' -import { deepClone, handleError, toEncryptedDynamoItem } from '../helpers' -import type { EncryptedDynamoDBError } from '../types' +import { + deepClone, + handleError, + isV3Table, + throwPreservingCode, + toEncryptedDynamoItem, +} from '../helpers' +import type { + AnyEncryptedTable, + CallableEncryptionClient, + DynamoDBEncryptionClient, + EncryptedDynamoDBError, +} from '../types' import { DynamoDBOperation, type DynamoDBOperationOptions, @@ -12,14 +22,14 @@ import { export class BulkEncryptModelsOperation< T extends Record, > extends DynamoDBOperation { - private encryptionClient: EncryptionClient + private encryptionClient: DynamoDBEncryptionClient private items: T[] - private table: EncryptedTable + private table: AnyEncryptedTable constructor( - encryptionClient: EncryptionClient, + encryptionClient: DynamoDBEncryptionClient, items: T[], - table: EncryptedTable, + table: AnyEncryptedTable, options?: DynamoDBOperationOptions, ) { super(options) @@ -32,7 +42,8 @@ export class BulkEncryptModelsOperation< logger.debug(`DynamoDB: bulk encrypting ${this.items.length} models.`) return await withResult( async () => { - const encryptResult = await this.encryptionClient + const client = this.encryptionClient as CallableEncryptionClient + const encryptResult = await client .bulkEncryptModels( this.items.map((item) => deepClone(item)), this.table, @@ -40,20 +51,23 @@ export class BulkEncryptModelsOperation< .audit(this.getAuditData()) if (encryptResult.failure) { - // Create an Error object that preserves the FFI error code - // This is necessary because withResult's ensureError wraps non-Error objects - const error = new Error(encryptResult.failure.message) as Error & { - code?: string - } - error.code = encryptResult.failure.code - throw error + throwPreservingCode(encryptResult.failure) } const data = encryptResult.data.map((item) => deepClone(item)) - const encryptedAttrs = Object.keys(this.table.build().columns) + // Property names, NOT `build().columns` keys: for v3 those are the DB + // column names, which differ whenever a column is declared + // `emailAddress: types.TextEq('email_address')`. The encrypted model + // coming back from the client is keyed by property name, so matching on + // config keys silently missed the attribute entirely. + const { columnPaths: encryptedAttrs } = resolveEncryptColumnMap( + this.table, + ) + const isV3 = isV3Table(this.table) return data.map( - (encrypted) => toEncryptedDynamoItem(encrypted, encryptedAttrs) as T, + (encrypted) => + toEncryptedDynamoItem(encrypted, encryptedAttrs, isV3) as T, ) }, (error) => diff --git a/packages/stack/src/dynamodb/operations/decrypt-model.ts b/packages/stack/src/dynamodb/operations/decrypt-model.ts index dd8702f2e..aea746b36 100644 --- a/packages/stack/src/dynamodb/operations/decrypt-model.ts +++ b/packages/stack/src/dynamodb/operations/decrypt-model.ts @@ -1,10 +1,18 @@ import { type Result, withResult } from '@byteslice/result' -import type { EncryptionClient } from '@/encryption' -import type { EncryptedTable, EncryptedTableColumn } from '@/schema' import type { Decrypted, EncryptedValue } from '@/types' import { logger } from '@/utils/logger' -import { handleError, toItemWithEqlPayloads } from '../helpers' -import type { EncryptedDynamoDBError } from '../types' +import { + handleError, + resolveDecryptResult, + throwPreservingCode, + toItemWithEqlPayloads, +} from '../helpers' +import type { + AnyEncryptedTable, + CallableEncryptionClient, + DynamoDBEncryptionClient, + EncryptedDynamoDBError, +} from '../types' import { DynamoDBOperation, type DynamoDBOperationOptions, @@ -13,14 +21,14 @@ import { export class DecryptModelOperation< T extends Record, > extends DynamoDBOperation> { - private encryptionClient: EncryptionClient + private encryptionClient: DynamoDBEncryptionClient private item: Record - private table: EncryptedTable + private table: AnyEncryptedTable constructor( - encryptionClient: EncryptionClient, + encryptionClient: DynamoDBEncryptionClient, item: Record, - table: EncryptedTable, + table: AnyEncryptedTable, options?: DynamoDBOperationOptions, ) { super(options) @@ -37,18 +45,16 @@ export class DecryptModelOperation< async () => { const withEqlPayloads = toItemWithEqlPayloads(this.item, this.table) - const decryptResult = await this.encryptionClient - .decryptModel(withEqlPayloads as T) - .audit(this.getAuditData()) + const client = this.encryptionClient as CallableEncryptionClient + const decryptResult = await resolveDecryptResult>( + // The second argument is required by the typed client and ignored by + // the nominal one, which derives the table from the payloads. + client.decryptModel(withEqlPayloads, this.table), + this.getAuditData(), + ) if (decryptResult.failure) { - // Create an Error object that preserves the FFI error code - // This is necessary because withResult's ensureError wraps non-Error objects - const error = new Error(decryptResult.failure.message) as Error & { - code?: string - } - error.code = decryptResult.failure.code - throw error + throwPreservingCode(decryptResult.failure) } return decryptResult.data diff --git a/packages/stack/src/dynamodb/operations/encrypt-model.ts b/packages/stack/src/dynamodb/operations/encrypt-model.ts index 754420a40..56a56ea47 100644 --- a/packages/stack/src/dynamodb/operations/encrypt-model.ts +++ b/packages/stack/src/dynamodb/operations/encrypt-model.ts @@ -1,9 +1,19 @@ import { type Result, withResult } from '@byteslice/result' -import type { EncryptionClient } from '@/encryption' -import type { EncryptedTable, EncryptedTableColumn } from '@/schema' +import { resolveEncryptColumnMap } from '@/encryption/helpers/model-helpers' import { logger } from '@/utils/logger' -import { deepClone, handleError, toEncryptedDynamoItem } from '../helpers' -import type { EncryptedDynamoDBError } from '../types' +import { + deepClone, + handleError, + isV3Table, + throwPreservingCode, + toEncryptedDynamoItem, +} from '../helpers' +import type { + AnyEncryptedTable, + CallableEncryptionClient, + DynamoDBEncryptionClient, + EncryptedDynamoDBError, +} from '../types' import { DynamoDBOperation, type DynamoDBOperationOptions, @@ -12,14 +22,14 @@ import { export class EncryptModelOperation< T extends Record, > extends DynamoDBOperation { - private encryptionClient: EncryptionClient + private encryptionClient: DynamoDBEncryptionClient private item: T - private table: EncryptedTable + private table: AnyEncryptedTable constructor( - encryptionClient: EncryptionClient, + encryptionClient: DynamoDBEncryptionClient, item: T, - table: EncryptedTable, + table: AnyEncryptedTable, options?: DynamoDBOperationOptions, ) { super(options) @@ -32,24 +42,30 @@ export class EncryptModelOperation< logger.debug('DynamoDB: encrypting model.') return await withResult( async () => { - const encryptResult = await this.encryptionClient + const client = this.encryptionClient as CallableEncryptionClient + const encryptResult = await client .encryptModel(deepClone(this.item), this.table) .audit(this.getAuditData()) if (encryptResult.failure) { - // Create an Error object that preserves the FFI error code - // This is necessary because withResult's ensureError wraps non-Error objects - const error = new Error(encryptResult.failure.message) as Error & { - code?: string - } - error.code = encryptResult.failure.code - throw error + throwPreservingCode(encryptResult.failure) } const data = deepClone(encryptResult.data) - const encryptedAttrs = Object.keys(this.table.build().columns) + // Property names, NOT `build().columns` keys: for v3 those are the DB + // column names, which differ whenever a column is declared + // `emailAddress: types.TextEq('email_address')`. The encrypted model + // coming back from the client is keyed by property name, so matching on + // config keys silently missed the attribute entirely. + const { columnPaths: encryptedAttrs } = resolveEncryptColumnMap( + this.table, + ) - return toEncryptedDynamoItem(data, encryptedAttrs) as T + return toEncryptedDynamoItem( + data, + encryptedAttrs, + isV3Table(this.table), + ) as T }, (error) => handleError(error, 'encryptModel', { diff --git a/packages/stack/src/dynamodb/types.ts b/packages/stack/src/dynamodb/types.ts index a7500d1b8..1bfbca989 100644 --- a/packages/stack/src/dynamodb/types.ts +++ b/packages/stack/src/dynamodb/types.ts @@ -1,14 +1,105 @@ import type { ProtectErrorCode } from '@cipherstash/protect-ffi' import type { EncryptionClient } from '@/encryption' +import type { + AnyV3Table, + EncryptedTable as EncryptedV3Table, + InferPlaintext, + PlaintextForColumn, + QueryTypesForColumn, + V3ModelInput, +} from '@/eql/v3' import type { EncryptedTable, EncryptedTableColumn } from '@/schema' import type { EncryptedValue } from '@/types' +import type { ciphertextAttrSuffix, searchTermAttrSuffix } from './helpers' import type { BulkDecryptModelsOperation } from './operations/bulk-decrypt-models' import type { BulkEncryptModelsOperation } from './operations/bulk-encrypt-models' import type { DecryptModelOperation } from './operations/decrypt-model' import type { EncryptModelOperation } from './operations/encrypt-model' +/** + * A table this adapter accepts: either an EQL v2 table (`encryptedTable` + + * `encryptedColumn`/`encryptedField` from `@cipherstash/stack/schema`) or an + * EQL v3 one (`encryptedTable` + `types.*` from `@cipherstash/stack/eql/v3`). + * + * Both are supported deliberately. DynamoDB shares none of the v2 Postgres + * machinery — there is no EQL extension to install and no migration to run — + * so accepting v3 is purely additive and no existing caller has to change. + */ +export type AnyEncryptedTable = + | EncryptedTable + | AnyV3Table + +/** + * The client capability this adapter consumes, declared structurally so it is + * satisfied by the nominal {@link EncryptionClient} AND by the + * `TypedEncryptionClient` that `EncryptionV3` returns, neither needing a cast. + * Mirrors the approach the Drizzle v3 operators take for the same reason: a + * nominal `TypedEncryptionClient` parameter would reject a client built for + * a narrower schema tuple. + * + * The two clients differ at runtime on the decrypt paths — the nominal client + * returns a chainable operation carrying `.audit()`, the typed wrapper returns + * a plain `Promise>` and takes the table as a second argument. The + * operation classes handle both; see `DecryptModelOperation`. The consequence + * for callers is that **audit metadata on decrypt requires the nominal + * client** — with a client from `EncryptionV3` there is nowhere to put it. + */ +export type DynamoDBEncryptionClient = { + encryptModel(input: never, table: never): unknown + bulkEncryptModels(input: never, table: never): unknown + decryptModel(input: never, table: never): unknown + bulkDecryptModels(input: never, table: never): unknown +} + +type ChainableEncryptOperation = { + audit(data: { + metadata?: Record + }): PromiseLike< + | { data: T; failure?: never } + | { data?: never; failure: { message: string; code?: string } } + > +} + +/** + * @internal Callable view of {@link DynamoDBEncryptionClient}. + * + * The public type declares `never` operands so both client shapes satisfy it + * without a cast; a callable signature cannot be written that both a generic + * `EncryptionClient` method and a generic `TypedEncryptionClient` method + * satisfy. The operation classes therefore cast to this shape at the call site + * — the same split the Drizzle v3 operators use. + * + * `decryptModel` is intentionally untyped in its return: the nominal client + * returns a chainable operation, the typed client a plain promise. See + * `resolveDecryptResult`. + */ +export type CallableEncryptionClient = { + encryptModel( + input: Record, + table: AnyEncryptedTable, + ): ChainableEncryptOperation> + bulkEncryptModels( + input: Record[], + table: AnyEncryptedTable, + ): ChainableEncryptOperation[]> + decryptModel( + input: Record, + table?: AnyEncryptedTable, + ): unknown + bulkDecryptModels( + input: Record[], + table?: AnyEncryptedTable, + ): unknown +} + export interface EncryptedDynamoDBConfig { - encryptionClient: EncryptionClient + /** + * Either the nominal client from `Encryption(...)` / `Encryption({ schemas, + * config: { eqlVersion: 3 } })`, or the typed client from `EncryptionV3(...)`. + * For EQL v3 tables the client must be in v3 mode — `EncryptionV3` forces + * this; with `Encryption` you must pass `config: { eqlVersion: 3 }` yourself. + */ + encryptionClient: EncryptionClient | DynamoDBEncryptionClient options?: { logger?: { error: (message: string, error: Error) => void @@ -22,22 +113,182 @@ export interface EncryptedDynamoDBError extends Error { details?: Record } +// --------------------------------------------------------------------------- +// The DynamoDB storage split, at the type level +// --------------------------------------------------------------------------- + +/** The `__source` / `__hmac` suffixes, read off the runtime constants so the + * types cannot drift from the mapping in `helpers.ts`. */ +type CiphertextSuffix = typeof ciphertextAttrSuffix +type SearchTermSuffix = typeof searchTermAttrSuffix + +/** The column map a v3 table was declared with. */ +type V3Columns
= + Table extends EncryptedV3Table ? C : never + +/** + * What `toEncryptedDynamoItem` writes into `__source` for a column: for a + * JSON document, the SteVec entries plus the per-document KeyHeader `h` + * (`{ h, sv }`) — protect-ffi 0.30 decrypt requires `h`; for every scalar, the + * base64 ciphertext `c`. + */ +type SourceAttribute = + 'searchableJson' extends QueryTypesForColumn + ? { h: unknown; sv: unknown[] } + : string + +/** + * Does this column mint the `hm` term that becomes `__hmac`? + * + * Mirrors `indexesForCapabilities` (eql/v3/columns.ts) exactly: `hm` comes from + * the `unique` index, which is emitted when a domain is equality-capable AND is + * either not an ordering domain or is text (text equality is always HMAC-based, + * numeric/date ordering domains answer equality via their ordering term and + * emit no `unique`). A JSON document keeps its terms inside `sv`, so it has no + * separate search-term attribute. + * + * Derived from the public `QueryTypesForColumn` / `PlaintextForColumn` rather + * than the internal domain literal, so it stays inside the v3 barrel's API. + */ +type HasSearchTerm = + 'searchableJson' extends QueryTypesForColumn + ? false + : 'equality' extends QueryTypesForColumn + ? 'orderAndRange' extends QueryTypesForColumn + ? [PlaintextForColumn] extends [string] + ? true + : false + : true + : false + +/** Flatten an intersection into a single object type for readable errors. */ +type Simplify = { [K in keyof T]: T[K] } + +/** + * The DynamoDB attribute map `encryptModel` actually returns for a v3 table. + * + * A declared column `email` does NOT survive as `email`: the adapter deletes it + * and writes `email__source` (plus `email__hmac` for equality domains). Typing + * the result as the input model — what the v2 overload still does — is a lie + * that type-checks `result.data.email` (always `undefined` at runtime) and + * rejects `result.data.email__source` (the value you actually want). + * + * Keys that name no column pass through untouched — partition/sort keys, GSI + * attributes, anything else on the item. + * + * LIMITATION: a v3 column declared under a dotted path (`'profile.ssn'`) is + * split *inside* the nested `profile` map at runtime. The model key is + * `profile`, not `profile.ssn`, so it passes through here unchanged and the + * nested split is not modelled. + * + * LIMITATION: values inside an ARRAY are not descended into — the write path + * skips arrays, so a payload in a list is stored whole rather than split into + * `__source`/`__hmac`. It still decrypts on read, but this mapped type + * describes it as its plaintext input shape, not a split. Documented in the + * DynamoDB skill's limitations. + */ +export type EncryptedAttributes
= Simplify< + { + [K in keyof T as K extends keyof V3Columns
& string + ? `${K}${CiphertextSuffix}` + : K]: K extends keyof V3Columns
+ ? SourceAttribute[K]> + : T[K] + } & { + // Optional: the term is only written when the encrypted value produced one, + // so a null/absent field leaves the attribute off the item entirely. + [K in keyof T as K extends keyof V3Columns
& string + ? HasSearchTerm[K]> extends true + ? `${K}${SearchTermSuffix}` + : never + : never]?: string + } +> + +/** + * The inverse of {@link EncryptedAttributes}: the plaintext model + * `decryptModel` returns for an item read back out of DynamoDB. + * + * `__source` folds back to `col` with the column's plaintext type, + * `__hmac` is dropped (it is a query term, not data), and every other + * attribute passes through. Declared this way — rather than taking the + * plaintext model as the input parameter — so `T` is inferred from the argument + * a caller actually has: the stored attribute map. + */ +export type DecryptedAttributes
= Simplify<{ + [K in keyof T as K extends `${infer Base}${SearchTermSuffix}` + ? Base extends keyof V3Columns
+ ? never + : K + : K extends `${infer Base}${CiphertextSuffix}` + ? Base extends keyof V3Columns
+ ? Base + : K + : K]: K extends `${infer Base}${CiphertextSuffix}` + ? Base extends keyof InferPlaintext
+ ? InferPlaintext
[Base] + : T[K] + : T[K] +}> + export interface EncryptedDynamoDBInstance { + /** + * EQL v3: the input model is checked against the table's column types, and + * the result is the {@link EncryptedAttributes} storage split. + */ + encryptModel
>( + item: V3ModelInput, + table: Table, + ): EncryptModelOperation> + /** + * EQL v2. Unchanged, so existing callers keep compiling — v2 columns do not + * carry their index configuration in the type, so the storage split cannot be + * derived. The returned `T` is the INPUT model, not what is on the wire; read + * `__source` / `__hmac` through a type of your own. + */ encryptModel>( item: T, table: EncryptedTable, ): EncryptModelOperation + /** EQL v3. See {@link EncryptedDynamoDBInstance.encryptModel}. */ + bulkEncryptModels< + Table extends AnyV3Table, + T extends Record, + >( + items: Array>, + table: Table, + ): BulkEncryptModelsOperation> + /** EQL v2. See {@link EncryptedDynamoDBInstance.encryptModel}. */ bulkEncryptModels>( items: T[], table: EncryptedTable, ): BulkEncryptModelsOperation + /** + * EQL v3: `item` is the stored attribute map (`__source` / + * `__hmac`), and the result is the {@link DecryptedAttributes} plaintext + * model it folds back to. + */ + decryptModel
>( + item: T, + table: Table, + ): DecryptModelOperation> + /** EQL v2. Unchanged. */ decryptModel>( item: Record, table: EncryptedTable, ): DecryptModelOperation + /** EQL v3. See {@link EncryptedDynamoDBInstance.decryptModel}. */ + bulkDecryptModels< + Table extends AnyV3Table, + T extends Record, + >( + items: T[], + table: Table, + ): BulkDecryptModelsOperation> + /** EQL v2. Unchanged. */ bulkDecryptModels>( items: Record[], table: EncryptedTable, diff --git a/packages/stack/src/encryption/index.ts b/packages/stack/src/encryption/index.ts index ae36a7e00..a66817e79 100644 --- a/packages/stack/src/encryption/index.ts +++ b/packages/stack/src/encryption/index.ts @@ -29,6 +29,7 @@ import type { Plaintext, ScalarQueryTerm, } from '@/types' +import { hasBuildColumnKeyMap } from '@/types' import { logger } from '@/utils/logger' import { toFfiKeysetIdentifier } from './helpers' import { isScalarQueryTermArray } from './helpers/type-guards' @@ -88,9 +89,7 @@ export function resolveEqlVersion( schemas: readonly BuildableTable[], explicit?: 2 | 3, ): 2 | 3 | undefined { - const v3Count = schemas.filter( - (schema) => typeof schema.buildColumnKeyMap === 'function', - ).length + const v3Count = schemas.filter(hasBuildColumnKeyMap).length if (v3Count > 0 && v3Count < schemas.length) { throw new Error( diff --git a/packages/stack/src/types.ts b/packages/stack/src/types.ts index fdb0ff4d9..c516d848e 100644 --- a/packages/stack/src/types.ts +++ b/packages/stack/src/types.ts @@ -243,6 +243,24 @@ export interface BuildableTable { buildColumnKeyMap?(): Record } +/** + * The canonical EQL v3 marker: v3 tables expose `buildColumnKeyMap()`, v2 tables + * don't. This is the single predicate the codebase uses to decide which wire + * version a table targets — a second hand-written spelling of the check is how a + * v2 envelope eventually gets built for a v3 table once the marker drifts, so + * every site (`encryption/index.ts`, `wasm-inline.ts`, `dynamodb/helpers.ts`) + * routes through here. + */ +export function hasBuildColumnKeyMap( + table: T, +): table is T & { buildColumnKeyMap: () => Record } { + return ( + 'buildColumnKeyMap' in table && + typeof (table as { buildColumnKeyMap?: unknown }).buildColumnKeyMap === + 'function' + ) +} + export type EncryptionClientConfig = { schemas: AtLeastOneCsTable config?: ClientConfig diff --git a/packages/stack/src/wasm-inline.ts b/packages/stack/src/wasm-inline.ts index a617a4b81..a3409a8fd 100644 --- a/packages/stack/src/wasm-inline.ts +++ b/packages/stack/src/wasm-inline.ts @@ -113,6 +113,7 @@ import type { Plaintext, QueryTypeName, } from '@/types' +import { hasBuildColumnKeyMap } from '@/types' // ----------------------------------------------------------------------- // Schema + type re-exports @@ -982,9 +983,7 @@ export async function Encryption( // `buildColumnKeyMap` marker) with a clear message rather than pinning the // client to v3 wire against a v2 schema and failing opaquely inside the FFI. for (const table of schemas) { - const isV3 = - typeof (table as { buildColumnKeyMap?: unknown }).buildColumnKeyMap === - 'function' + const isV3 = hasBuildColumnKeyMap(table) if (!isV3) { throw new Error( '[encryption]: `@cipherstash/stack/wasm-inline` is EQL v3 only — author schemas with `types` / `encryptedTable` from this entry. (EQL v2 is available on the native `@cipherstash/stack` entry.)', diff --git a/skills/stash-dynamodb/SKILL.md b/skills/stash-dynamodb/SKILL.md index 3c8c26b19..cf3d44d47 100644 --- a/skills/stash-dynamodb/SKILL.md +++ b/skills/stash-dynamodb/SKILL.md @@ -1,6 +1,6 @@ --- name: stash-dynamodb -description: Integrate CipherStash encryption with Amazon DynamoDB using @cipherstash/stack/dynamodb. Covers the encryptedDynamoDB helper for encrypting items before PutItem and decrypting after GetItem, bulk encrypt/decrypt for BatchWrite and BatchGet, querying with encrypted partition and sort keys via HMAC attributes, nested object encryption, audit logging, and the DynamoDB attribute naming conventions (__source/__hmac). Use when adding encryption to a DynamoDB project, encrypting items before writes, decrypting items after reads, or querying encrypted DynamoDB attributes. +description: Integrate CipherStash encryption with Amazon DynamoDB using @cipherstash/stack/dynamodb. Covers the encryptedDynamoDB helper for encrypting items before PutItem and decrypting after GetItem with EQL v3 (types.* domains) or EQL v2 schemas, bulk encrypt/decrypt for BatchWrite and BatchGet, querying with encrypted partition and sort keys via HMAC attributes, nested object encryption, audit logging, and the DynamoDB attribute naming conventions (__source/__hmac). Use when adding encryption to a DynamoDB project, encrypting items before writes, decrypting items after reads, or querying encrypted DynamoDB attributes. --- # CipherStash Stack - DynamoDB Integration @@ -36,68 +36,163 @@ CipherStash encrypts each attribute into two DynamoDB attributes: | Original Attribute | Stored As | Purpose | |---|---|---| | `email` | `email__source` | Encrypted ciphertext | -| `email` | `email__hmac` | HMAC for equality lookups (only if `.equality()` index is set) | +| `email` | `email__hmac` | HMAC — written whenever the domain produces an equality term, usable only for equality lookups (see the domain table below) | Non-encrypted attributes pass through unchanged. On decryption, the `__source` and `__hmac` attributes are recombined back into the original attribute name with the plaintext value. +**Only equality is usable on DynamoDB.** Ordering terms and free-text bloom filters have no DynamoDB query surface, so they are not stored. A column in an ordering or free-text domain still encrypts and decrypts correctly — it just cannot back a key condition. + +## Choosing a Schema Version + +`encryptedDynamoDB` accepts both EQL v3 and EQL v2 tables. **Use EQL v3 for new projects.** + +| | EQL v3 (recommended) | EQL v2 (existing deployments) | +|---|---|---| +| Import | `@cipherstash/stack/v3` | `@cipherstash/stack` + `@cipherstash/stack/schema` | +| Schema | `encryptedTable` + `types.*` | `encryptedTable` + `encryptedColumn` | +| Client | `EncryptionV3({ schemas })` | `Encryption({ schemas })` | +| Nested fields | Flat dotted path (`"profile.ssn"`) | Nested group + `encryptedField` | + +There is no infrastructure migration between them — DynamoDB has no EQL extension to install and no schema to alter — but there is no automatic *data* migration path either. The two write **different wire formats**, so items written under one version cannot be decrypted by the other. Pick one per table and stay on it; to switch an existing table you must re-encrypt every item. + +The client must be built for the table. Build the client with the same v3 table you hand to `encryptedDynamoDB` — `EncryptionV3({ schemas: [users] })` (or `Encryption({ schemas: [users], config: { eqlVersion: 3 } })`). Passing a v3 table to a client that never registered it (a client built for a different schema set) throws a clear error naming the table on the first operation, instead of failing later with an opaque FFI deserialization error. + +**Nested attributes work in both versions**, with different authoring syntax. + +DynamoDB items are natively nested, so this matters here more than on Postgres. Both versions encrypt *selected leaves in place*: the item keeps its shape and unlisted siblings stay plaintext. + +EQL v2 uses a nested group with `encryptedField`: + +```typescript +const users = encryptedTable("users", { + profile: { ssn: encryptedField("profile.ssn") }, +}) +``` + +EQL v3 has no nested-group syntax — a nested object in a v3 column map is a compile error. Declare the column **flat, with a dotted path** instead: + +```typescript +const users = encryptedTable("users", { + "profile.ssn": types.TextEq("profile.ssn"), + "profile.note": types.Text("profile.note"), +}) +``` + +Both produce the same nested DynamoDB attribute *layout*, but the encrypted contents are version-specific: an item written under one version cannot be decrypted under the other (see the compatibility note above). + +```jsonc +{ "pk": "u#1", + "profile": { + "ssn__source": "", + "ssn__hmac": "", // equality term — FilterExpression only, not a key condition + "note__source": "", + "city": "Sydney" // not in schema, stays plaintext + } } +``` + +> A nested equality term like `profile.ssn__hmac` lives *inside* the `profile` +> map, so it can only be matched with a `FilterExpression` — DynamoDB key +> conditions and secondary-index keys must be top-level scalar attributes. If you +> need the HMAC to back a key condition or GSI, declare the field as a top-level +> column (`ssn: types.TextEq("ssn")`) so it is stored as a top-level `ssn__hmac`. + +> The dotted string is the *property key* as well as the column name — the model +> is matched by dotted path, so `{ profile: { ssn } }` resolves correctly. + +To encrypt a whole subtree as one value instead of per-leaf, use `types.Json`, which stores it as a single ste_vec attribute (`profile__source`). + +> **Arrays are not descended into.** The adapter splits encrypted leaves inside +> nested *objects* only. A value inside an array is stored whole — it is not +> split into `__source`/`__hmac`, so its ciphertext still decrypts on read but +> can never back a key condition, and it does not appear in the `__source`/`__hmac` +> attribute layout above. A DynamoDB key condition cannot target an array element +> in any case. To encrypt list data, either promote the searched field to a +> top-level column, or wrap the subtree in a single `types.Json` column. + ## Rolling Encryption Out to Production DynamoDB encryption is **single-deploy**. There is no rollout/cutover split — unlike the Postgres path, DynamoDB has no row-level rename swap and no shared-state proxy. The application owns every write, so adding encryption is an application-side change that ships in one PR: 1. Declare the encrypted schema (see Setup below). -2. Wrap your DynamoDB client with `encryptedDynamoDB` (or call `encryptItem` / `decryptItem` directly at write/read sites). +2. Build an `encryptedDynamoDB` helper and call `encryptModel` / `decryptModel` at your write and read sites. 3. Ship the change. -For tables with **existing populated items**, the `__source` and `__hmac` attributes are added by the next write that touches each item. If you need every existing item encrypted at once (e.g. because a query uses `email__hmac` and would miss legacy items), run a one-shot script that reads every item, calls `encryptItem`, and writes it back. Idempotent: re-running an already-encrypted item is a no-op as long as the schema hasn't changed. +For tables with **existing populated items**, the `__source` and `__hmac` attributes are added by the next write that touches each item. If you need every existing item encrypted at once (e.g. because a query uses `email__hmac` and would miss legacy items), run a one-shot script that reads every item, calls `encryptModel`, and writes it back. Idempotent: re-running an already-encrypted item is a no-op as long as the schema hasn't changed. > **Where am I?** Run `stash status` (or `bunx`/`pnpm dlx`/`yarn dlx` per your runner) for a project-wide view across both Postgres and DynamoDB integrations. DynamoDB columns surface in the quest log as already-complete since there is no staged lifecycle to track. ## Setup -### 1. Define Encrypted Schema +### 1. Define Encrypted Schema (EQL v3) + +Each `types.*` factory is a concrete domain with fixed query capabilities. There are no chainable index methods — the type *is* the capability. ```typescript -import { encryptedTable, encryptedColumn, encryptedField } from "@cipherstash/stack/schema" +import { encryptedTable, types } from "@cipherstash/stack/v3" const users = encryptedTable("users", { - email: encryptedColumn("email").equality(), // searchable via HMAC - name: encryptedColumn("name"), // encrypt-only, no search - phone: encryptedColumn("phone"), // encrypt-only - metadata: encryptedColumn("metadata").dataType("json"), // encrypt-only JSON (use .searchableJson() for queryable JSON) + email: types.TextEq("email"), // equality -> queryable via email__hmac + name: types.Text("name"), // storage only + phone: types.Text("phone"), // storage only + age: types.IntegerOrd("age"), // decryptable, NOT queryable on DynamoDB + metadata: types.Json("metadata"), // JSON document }) ``` -> **Note:** `encryptedColumn` also supports `.orderAndRange()`, `.freeTextSearch()`, and `.searchableJson()` index methods, but only `.equality()` produces HMAC values usable for DynamoDB key condition queries. +Which domains give you a `__hmac` attribute you can query on: -Nested objects are supported with `encryptedField`: +| Domain family | `__hmac` written? | Notes | +|---|---|---| +| `types.TextEq`, `IntegerEq`, `BigintEq`, `DateEq`, `TimestampEq`, `NumericEq`, `RealEq`, `DoubleEq`, `SmallintEq` | Yes | The equality domains — use these for anything you query | +| `types.TextOrd`, `TextOrdOre`, `TextSearch` | Yes | Text equality is HMAC-based even on ordering/search domains | +| `types.IntegerOrd`, `DateOrd`, `TimestampOrd`, and the other non-text `*Ord`/`*OrdOre` | **No** | Equality resolves through an ordering term in Postgres, which DynamoDB cannot use | +| `types.Text`, `Integer`, `Boolean`, and the other bare domains | No | Storage only | +| `types.Json` | No | Index terms live inside the ste_vec array; not splittable into an attribute | -```typescript -const users = encryptedTable("users", { - email: encryptedColumn("email").equality(), - profile: { - ssn: encryptedField("profile.ssn"), - address: { - street: encryptedField("profile.address.street"), - }, - }, -}) -``` +> **Rule of thumb:** if an attribute will appear in a `KeyConditionExpression`, declare it with an `*Eq` domain. ### 2. Initialize Clients ```typescript import { DynamoDBClient } from "@aws-sdk/client-dynamodb" import { DynamoDBDocumentClient } from "@aws-sdk/lib-dynamodb" -import { Encryption } from "@cipherstash/stack" +import { EncryptionV3 } from "@cipherstash/stack/v3" import { encryptedDynamoDB } from "@cipherstash/stack/dynamodb" const dynamoClient = new DynamoDBClient({ region: "us-east-1" }) const docClient = DynamoDBDocumentClient.from(dynamoClient) +const encryptionClient = await EncryptionV3({ schemas: [users] }) +const dynamo = encryptedDynamoDB({ encryptionClient }) +``` + +> **Audit metadata on decrypt:** the client from `EncryptionV3` has no audit +> surface on its decrypt methods, so `.audit()` on `decryptModel` / +> `bulkDecryptModels` resolves normally but the metadata is not recorded. If you +> need audited decrypts, build the client with `Encryption({ schemas: [users], +> config: { eqlVersion: 3 } })` instead — same v3 wire format, chainable decrypt. + +### EQL v2 Schema (existing deployments) + +```typescript +import { encryptedTable, encryptedColumn, encryptedField } from "@cipherstash/stack/schema" +import { Encryption } from "@cipherstash/stack" + +const users = encryptedTable("users", { + email: encryptedColumn("email").equality(), // searchable via HMAC + name: encryptedColumn("name"), // encrypt-only, no search + metadata: encryptedColumn("metadata").dataType("json"), + profile: { // nested objects: v2 only + ssn: encryptedField("profile.ssn"), + }, +}) + const encryptionClient = await Encryption({ schemas: [users] }) const dynamo = encryptedDynamoDB({ encryptionClient }) ``` +> **Note:** `encryptedColumn` also supports `.orderAndRange()`, `.freeTextSearch()`, and `.searchableJson()` index methods, but only `.equality()` produces HMAC values usable for DynamoDB key condition queries. + ### Optional: Logger and Error Handler ```typescript @@ -228,19 +323,19 @@ When an encrypted attribute is the partition key (e.g., `email__hmac`): ```typescript import { QueryCommand } from "@aws-sdk/lib-dynamodb" -// 1. Encrypt the search value to get the HMAC -const queryResult = await encryptionClient.encryptQuery([{ - value: "alice@example.com", - column: users.email, +// 1. Encrypt the search value to get the HMAC. +// On an EQL v3 equality domain this mints the bare term — `{ v, i, hm }` +// with no ciphertext — so `hm` is used directly. +const queryResult = await encryptionClient.encryptQuery("alice@example.com", { table: users, - queryType: "equality", -}]) + column: users.email, +}) if (queryResult.failure) { throw new Error(`Query encryption failed: ${queryResult.failure.message}`) } -const emailHmac = queryResult.data[0]?.hm +const emailHmac = queryResult.data.hm // 2. Use the HMAC in a DynamoDB query const result = await docClient.send(new QueryCommand({ @@ -316,7 +411,7 @@ For each encrypted field with an equality index, two attributes are stored: - `{field}__source` - The encrypted ciphertext (binary/string) - `{field}__hmac` - Deterministic HMAC for equality lookups -Fields without `.equality()` only get `__source` (no HMAC, so they can't be queried). +Fields without equality capability only get `__source` (no HMAC, so they can't be queried) — that means EQL v2 columns without `.equality()`, and EQL v3 columns outside the domains listed in the table above. ### Key Schema Design @@ -361,7 +456,8 @@ if (result.failure) { import { encryptedDynamoDB } from "@cipherstash/stack/dynamodb" const dynamo = encryptedDynamoDB({ - encryptionClient, // EncryptionClient instance + // From EncryptionV3(...) or Encryption(...) + encryptionClient, options: { // optional logger: { error: (message, error) => void }, errorHandler: (error) => void, @@ -371,56 +467,81 @@ const dynamo = encryptedDynamoDB({ ### Instance Methods -| Method | Signature | Returns | +`table` is either an EQL v3 table (`types.*` domains) or an EQL v2 one. + +Each method is overloaded on the table's EQL version. + +**EQL v3** — the item is checked against the table's column domains, and the result is typed as the attribute map that is actually stored: a declared column `email` becomes `email__source` (plus `email__hmac` if its domain mints one), NOT `email`. + +| Method | Signature | Resolves to | +|---|---|---| +| `encryptModel` | `(item, v3Table)` | `EncryptedAttributes` | +| `bulkEncryptModels` | `(items, v3Table)` | `EncryptedAttributes[]` | +| `decryptModel` | `(storedItem, v3Table)` | `DecryptedAttributes` — `__source` folded back to the column, `__hmac` dropped | +| `bulkDecryptModels` | `(storedItems, v3Table)` | `DecryptedAttributes[]` | + +Let `T` be inferred from the argument; do not pass explicit type arguments on the v3 path. + +**EQL v2** — unchanged. `T` is the model you pass in, so the result type does NOT reflect the `__source` / `__hmac` split; declare a type of your own to read those attributes. + +| Method | Signature | Resolves to | |---|---|---| -| `encryptModel` | `(item: T, table: EncryptedTable)` | `EncryptModelOperation` | -| `bulkEncryptModels` | `(items: T[], table: EncryptedTable)` | `BulkEncryptModelsOperation` | -| `decryptModel` | `(item: Record, table: EncryptedTable)` | `DecryptModelOperation` (resolves to `Decrypted`) | -| `bulkDecryptModels` | `(items: Record[], table: EncryptedTable)` | `BulkDecryptModelsOperation` (resolves to `Decrypted[]`) | +| `encryptModel` | `(item: T, v2Table)` | `T` | +| `bulkEncryptModels` | `(items: T[], v2Table)` | `T[]` | +| `decryptModel` | `(item: Record, v2Table)` | `T` | +| `bulkDecryptModels` | `(items: Record[], v2Table)` | `T[]` | + +All operations are thenable (awaitable) and support `.audit({ metadata })` chaining. See the note in Setup about decrypt-side audit metadata and the `EncryptionV3` client. -All operations are thenable (awaitable) and support `.audit({ metadata })` chaining. +Types exported from `@cipherstash/stack/dynamodb`: `EncryptedDynamoDBInstance`, `EncryptedDynamoDBConfig`, `EncryptedDynamoDBError`, `AnyEncryptedTable`, `DynamoDBEncryptionClient`, `EncryptedAttributes`, `DecryptedAttributes`, `AuditConfig`. ### Querying Encrypted Attributes Use the encryption client directly (not the DynamoDB helper): ```typescript -// Single value form (recommended for DynamoDB lookups): +// EQL v3 — the domain fixes the query type, so no `queryType` is needed: const result = await encryptionClient.encryptQuery( "search-value", - { column: schema.fieldName, table: schema, queryType: "equality" } + { table: users, column: users.email } ) -const hmac = result.data?.hm - -// Batch array form: -const batchResult = await encryptionClient.encryptQuery([{ - value: "search-value", - column: schema.fieldName, - table: schema, - queryType: "equality", -}]) -const hmac = batchResult.data[0]?.hm // Use this in DynamoDB key conditions +// encryptQuery returns a Result; check the failure branch before reading `data`. +// `data?.hm` would mask a failure (and a null-plaintext result) as `undefined`, +// producing a malformed key condition rather than a clear error. +if (result.failure) throw new Error(result.failure.message) +const hmac = result.data.hm // Use this in DynamoDB key conditions + +// EQL v2 — pass queryType explicitly: +const v2Result = await encryptionClient.encryptQuery( + "search-value", + { table: users, column: users.email, queryType: "equality" } +) +if (v2Result.failure) throw new Error(v2Result.failure.message) +const v2Hmac = v2Result.data.hm ``` +> On a `types.TextSearch` column `encryptQuery` returns `hm` alongside ordering +> and bloom-filter terms regardless of `queryType`. Only `hm` is meaningful for +> DynamoDB — a free-text query cannot be expressed as a key condition. + ## Complete Example ```typescript import { DynamoDBClient } from "@aws-sdk/client-dynamodb" import { DynamoDBDocumentClient, PutCommand, GetCommand, QueryCommand } from "@aws-sdk/lib-dynamodb" -import { Encryption } from "@cipherstash/stack" +import { EncryptionV3, encryptedTable, types } from "@cipherstash/stack/v3" import { encryptedDynamoDB } from "@cipherstash/stack/dynamodb" -import { encryptedTable, encryptedColumn } from "@cipherstash/stack/schema" // Schema const users = encryptedTable("users", { - email: encryptedColumn("email").equality(), - name: encryptedColumn("name"), + email: types.TextEq("email"), + name: types.Text("name"), }) // Clients const dynamoClient = new DynamoDBClient({ region: "us-east-1" }) const docClient = DynamoDBDocumentClient.from(dynamoClient) -const encryptionClient = await Encryption({ schemas: [users] }) +const encryptionClient = await EncryptionV3({ schemas: [users] }) const dynamo = encryptedDynamoDB({ encryptionClient }) // Write @@ -441,13 +562,12 @@ if (!decResult.failure) { } // Query by encrypted email (via HMAC) -const queryEnc = await encryptionClient.encryptQuery([{ - value: "alice@example.com", - column: users.email, +const queryEnc = await encryptionClient.encryptQuery("alice@example.com", { table: users, - queryType: "equality", -}]) -const hmac = queryEnc.data[0]?.hm + column: users.email, +}) +if (queryEnc.failure) throw new Error(queryEnc.failure.message) +const hmac = queryEnc.data.hm const queryResult = await docClient.send(new QueryCommand({ TableName: "Users",