diff --git a/AGENTS.md b/AGENTS.md index 64d5883..a4fcc3e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,11 +2,13 @@ ## Project overview -DialCache is a TypeScript caching library with explicit request-scoped enablement, local and Redis layers, runtime rollout controls, request coalescing, targeted invalidation, and adapter-based observability. +DialCache is a TypeScript caching library with explicit request-scoped enablement, local and Redis layers, runtime rollout controls, request coalescing, detached Redis shadow validation and bootstrap, targeted invalidation, and adapter-based observability. ## Structure ```text +README.md # Adoption guide, safety model, and reference routing +docs/ # Focused user-facing configuration and operations guides src/ dialcache.ts # Main DialCache API and cached-function wrapper config.ts # Public configuration and rollout types @@ -14,8 +16,10 @@ src/ key.ts # Structured cache keys and Redis hash tags metrics.ts # Backend-neutral metrics adapter contract prometheus.ts # Optional Prometheus adapter + datadog.ts # Optional Datadog adapter redis-client.ts # Client-independent semantic Redis interface node-redis.ts # node-redis adapter and script registration + valkey-glide.ts # Valkey GLIDE adapter and script registration redis-protocol.ts # Public Lua protocol exports serializer.ts # Serializer contract and JSON implementation internal/ # Cache layers, runtime config, and Lua scripts @@ -28,7 +32,14 @@ test/ # Unit and Redis integration tests - Disabled calls are true pass-through and must not build keys, resolve config, or coalesce work. - Active same-key work is coalesced before the first active cache layer, using request scope for request-local caching and process scope for shared layers. +- Shadow validation is opt-in, detached, tracked-key-only work that never + supplies or delays the caller; it has separate sampling, capacity, deadline, + invalidation, and observability contracts. +- Shadow policy is grouped under `DialCacheKeyConfig.shadow`. Mismatch logging + is default-off diagnostic output; its size limits are not redaction. - Cache plumbing fails open; explicit maintenance operations surface mutation failures. +- `invalidateRemote()` requires a configured Redis client and rejects when the + client is absent or the watermark mutation fails. - Tracked Redis values and invalidation watermarks share a Redis Cluster hash tag. - Tracked reads run on primaries so replica lag cannot hide invalidation. - Local entries are process-local and are not synchronously invalidated across instances. @@ -36,8 +47,13 @@ test/ # Unit and Redis integration tests ## Conventions - Preserve strict TypeScript settings and public abstraction boundaries. +- Keep the README focused on evaluation and adoption. Put complete operational + contracts in a focused `docs/` guide and link it from the relevant README + summary. - Keep Redis client-specific behavior in adapters; core code depends on `DialCacheRedisClient`. -- Public exports belong in the root or an explicit integration entry point such as `src/node-redis.ts`, `src/prometheus.ts`, or `src/redis-protocol.ts`. +- Public exports belong in the root or an explicit integration entry point such + as `src/node-redis.ts`, `src/valkey-glide.ts`, `src/prometheus.ts`, + `src/datadog.ts`, or `src/redis-protocol.ts`. - Use `corepack pnpm` for project commands. ## Validation diff --git a/README.md b/README.md index 2b8ec06..c7ac8b4 100644 --- a/README.md +++ b/README.md @@ -4,48 +4,77 @@ [![Codecov](https://codecov.io/gh/lan17/DialCache/branch/main/graph/badge.svg)](https://codecov.io/gh/lan17/DialCache) [![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/lan17/DialCache/badge)](https://scorecard.dev/viewer/?uri=github.com/lan17/DialCache) -Fine-grained TypeScript caching with explicit enabled contexts, request-local memoization, process-local and Redis TTL caching, stable key construction, runtime rollout controls, request coalescing, adapter-based observability, and Redis watermark-based targeted invalidation. +**Read-through caching with the controls production systems need.** + +DialCache is a TypeScript read-through caching library for database and service +reads in production Node.js applications. + +Wrap a reusable function with `cached()` or keep a loader inline with +`getOrLoad()`; when the active cache layers miss, DialCache calls your loader +and publishes the result to whichever request-local, bounded process-local, +and optional Redis or Valkey layers are active. + +Around that core path, DialCache provides patterns that high-scale services +otherwise have to build themselves: request coalescing, per-use-case runtime +policy, deterministic ramp-up and ramp-down, detached shadow validation, +fail-open cache access, targeted invalidation, serialization, deadlines, and +backend-neutral metrics. + +The “dial” is the runtime policy: start a use case at zero, expand local or +remote caching to stable key cohorts, and reverse the rollout without changing +the loader. + +DialCache is a backend application library—not a frontend data cache, cache +server, Redis or Valkey client, or configuration control plane. Your service +owns the loader, clients, dynamic configuration source, cache identity, TTLs, +invalidation windows, admission control, and resource budgets. + +## Safety comes from explicit controls + +- **Off by default.** Outside `dialcache.enable(...)`, calls go straight to the + loader without building a key, resolving policy, accessing a cache, or + coalescing work. +- **Gradual and reversible.** Start process-local and remote ramps at `0`, + expand either to a stable key cohort, and turn every cache layer back off + through runtime policy. +- **Fail-open cache path.** Cache-plumbing failures fall through to the loader + instead of replacing a usable result. Explicit invalidation failures still + surface to the caller. ## Contents - [Install](#install) - [Quick start](#quick-start) -- [How caching works](#how-caching-works) -- [Enabled context](#enabled-context) -- [Defining cached functions](#defining-cached-functions) - - [One-shot inline cache blocks](#one-shot-inline-cache-blocks) -- [Keys, ids, and extra dimensions](#keys-ids-and-extra-dimensions) -- [Runtime config and ramp controls](#runtime-config-and-ramp-controls) -- [Cache layers](#cache-layers) - - [Request-local cache](#request-local-cache) · [Process-local cache](#process-local-cache) · [Redis-backed TTL cache](#redis-backed-ttl-cache) · [Remote read deadlines](#remote-read-deadlines-and-async-liveness) · [Serialization](#serialization) · [Shadow validation](#shadow-validation) -- [Cached-value ownership](#cached-value-ownership) -- [Targeted invalidation and watermarks](#targeted-invalidation-and-watermarks) -- [Request coalescing](#request-coalescing) - - [Fallback deadlines](#fallback-deadlines) · [Coalescing state](#coalescing-state) -- [Metrics](#metrics) -- [Maintainers](#maintainers) +- [Dial caching up or down](#dial-caching-up-or-down) +- [Validate Redis before serving it](#validate-redis-before-serving-it) +- [How the read path works](#how-the-read-path-works) +- [Core concepts](#core-concepts) +- [Production checklist](#production-checklist) +- [Reference guides](#reference-guides) ## Install ```bash -pnpm add dialcache -# Choose a Redis client when using the remote layer: -pnpm add redis@~4.7.1 -# or -pnpm add @valkey/valkey-glide -# Add a metrics client only when using its adapter: -pnpm add prom-client@^15.1.3 -# or -pnpm add hot-shots@^17.0.0 +npm install dialcache ``` DialCache requires Node.js 22.0.0 or newer. Production deployments should use a [currently supported LTS release](https://nodejs.org/en/about/previous-releases). +Redis, Valkey, Prometheus, and Datadog integrations are optional and keep their +clients application-owned: + +- [Redis and Valkey setup](https://github.com/lan17/DialCache/blob/main/docs/redis.md) +- [Prometheus and Datadog setup](https://github.com/lan17/DialCache/blob/main/docs/observability.md) + ## Quick start +Most services create one long-lived `DialCache` instance and reuse it across +the process. It owns one process-local LRU and one process-coalescing scope; +create separate instances only when those resources should be isolated: + ```ts -import { DialCache, DialCacheKeyConfig } from "dialcache"; +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; const dialcache = new DialCache(); @@ -55,422 +84,88 @@ const getUser = dialcache.cached( keyType: "user_id", useCase: "GetUser", cacheKey: (userId) => userId, - defaultConfig: DialCacheKeyConfig.enabled(60), + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 60 }, + }), }, ); -// Caching is OFF outside an enable() scope (see "Enabled context"), so this runs the fn uncached: +// Outside enable(), this is a true pass-through to db.fetchUser: await getUser("123"); -// Inside enable(), reads are cached: -const user = await dialcache.enable(() => getUser("123")); -``` - -## How caching works - -The wrapped function is the **fallback**: it runs whenever no active cache layer returns a value, whether because layers missed, were disabled, or failed open. - -When caching is enabled, reads flow through: - -```text -request-local cache -> process-local cache -> Redis cache -> fallback function +// Inside enable(), the first call loads and the second reuses the cached value: +const user = await dialcache.enable(async () => { + await getUser("123"); // db.fetchUser, then populate process-local cache + return await getUser("123"); // process-local hit +}); ``` -- Request-local hits return the value memoized in the current outermost `enable()` scope. -- Results from the lower chain are memoized request-locally when that layer is enabled. -- Process-local hits return immediately. -- Process-local misses try Redis and populate the process-local cache on a Redis hit. -- Redis misses call the fallback and attempt to populate Redis and, when active, the process-local cache. Tracked invalidation may suppress both publications. -- Selected tracked Redis keys can execute non-serving [shadow work](#shadow-validation) that validates hits and fills clean misses, even before Redis is allowed to serve callers. -- Redis read failures and timeouts are logged, counted in metrics, and fail open without attempting a second Redis operation. Redis write failures also fail open. `invalidateRemote` requires a configured Redis client; missing configuration and Redis failures are logged, counted, and rethrown so callers do not assume invalidation succeeded. -- Cache-key construction and config-provider failures also fail open and run the fallback uncached. -- A missing effective process-local/Redis TTL disables that layer by policy; a configured TTL with no ramp defaults to 100%. Disabled layers record a disabled reason and fall through to the next layer/fallback. - -Caching as a whole is only active inside an enabled context, described next. - -## Enabled context +`cached(fn, options)` preserves the function's parameters and returns a +Promise-based wrapper. The configuration above enables only the process-local +layer with a 60-second TTL; its omitted ramp defaults to `100`. Request-local +memoization and the remote layer remain off. -Caching is **off by default** and only active inside a `dialcache.enable(...)` scope. This is deliberate: it lets you turn caching **off in write paths** so a stale read can't be cached around a write. DialCache uses Node `AsyncLocalStorage` to keep enabled state scoped to the current asynchronous call chain. +For a one-shot calculation that should remain inline, +[`getOrLoad()`](https://github.com/lan17/DialCache/blob/main/docs/configuration.md#one-shot-inline-loaders) +accepts a +zero-argument loader and a direct key through the same cache contract. -**Enable once at your request boundary** (e.g. a middleware that wraps read-request handling) so individual call sites don't each need it; wrap mutation handlers in `disable()`: +Enable caching once at a read-request boundary instead of at every call site. +Keep nested mutation work uncached with `disable()`: ```ts await dialcache.enable(async () => { - await getUser("123"); // cached + const user = await getUser("123"); await dialcache.disable(async () => { - await updateUser("123", patch); // reads here are uncached + await updateUser("123", patch); }); - - await getUser("123"); // cached again }); ``` -- Default is disabled — `cached()` and `getOrLoad()` calls made **outside** any `enable()` scope simply run their loader uncached (no error), so wrap your read paths to actually cache. -- Enabled state is async-scope-local, not process-global. -- Nested `enable` / `disable` scopes restore the previous behavior when the callback completes. Nested `enable()` calls reuse the outer request-local scope rather than creating a new one. +Enabled state follows the current asynchronous call chain through Node +`AsyncLocalStorage`; it is not process-global. Nested scopes restore the +previous state when their callbacks settle. -## Defining cached functions +`disable()` prevents cache access during its callback; it does not evict values +cached before a mutation. Use the appropriate invalidation or TTL policy before +serving later reads of mutable data. -Use `cached(fn, options)` for an extracted, reusable function. The wrapped callable has the same parameters and always returns a `Promise`. For a one-shot calculation that should remain inline, use [`getOrLoad()`](#one-shot-inline-cache-blocks). +### From local trial to production -| Option | Required | Description | -| --- | --- | --- | -| `keyType` | yes | The kind of id the key addresses (e.g. `"user_id"`). Together with the id, the invalidation unit for tracked entries. | -| `useCase` | yes | Identifies the individual cache: part of the stored key and the metrics label. | -| `cacheKey` | yes | Selector over `fn`'s parameters; returns a bare id or `{ id, args }`. | -| `defaultConfig` | no | `DialCacheKeyConfig` baseline policy that runtime config overlays field by field (see [Runtime config](#runtime-config-and-ramp-controls)). | -| `serializer` | when the return type is not statically JSON-compatible | Per-function `Serializer` for Redis values (see [Serialization](#serialization)). | -| `shadowComparator` | no | Synchronous application-level equality for shadow validation; defaults to Node's strict deep equality. | -| `trackForInvalidation` | no (default `false`) | Opts this use case's Redis entries into watermark-based targeted invalidation. | -| `fallbackTimeoutMs` | no (default `60_000`) | Fallback deadline in milliseconds, at most 2,147,483,647; `null` disables it (see [Fallback deadlines](#fallback-deadlines)). | +A typical adoption path treats the quick start as local verification, not as +the initial production rollout policy: -`cached()` validates `useCase` at registration: a duplicate within one `DialCache` instance throws `UseCaseIsAlreadyRegisteredError`. Both APIs reject the internal name `watermark` with `UseCaseNameIsReservedError`. +1. verify the loader, key, and process-local behavior in development; +2. before production, add + [Prometheus or Datadog](https://github.com/lan17/DialCache/blob/main/docs/observability.md) + and an application-owned runtime policy that sets both shared serving ramps + and `shadow.ramp` to `0`; +3. add a remote TTL and + [Redis or Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) + while the remote serving ramp remains `0`; +4. for tracked use cases, optionally + [validate and fill Redis in shadow mode](https://github.com/lan17/DialCache/blob/main/docs/shadow-validation.md) + without serving it; and +5. increase process-local, shadow, and remote cohorts independently while + monitoring their load and outcomes. -### One-shot inline cache blocks +## Dial caching up or down -`getOrLoad(load, options)` runs one zero-argument loader through the same policy, cache layers, coalescing, invalidation, metrics, serialization, deadlines, and fail-open behavior as `cached()`. It is useful when only part of a larger function should be cached and the loader needs to capture local values: +Every cache operation can declare a stable `defaultConfig`. An optional +`cacheConfigProvider` returns a sparse runtime overlay for the current key, so +policy can change independently of the loader. -```ts -// Reuse the caller-owned defaults; getOrLoad() snapshots them per invocation. -const profileCacheDefaults = DialCacheKeyConfig.enabled(60); - -const profile = await dialcache.getOrLoad( - async () => { - const user = await db.getUser(userId); - return renderProfile(user, locale); - }, - { - keyType: "user_id", - useCase: "BuildProfile", - key: { id: userId, args: { locale } }, - defaultConfig: profileCacheDefaults, - }, -); -``` - -The options match `cached()` except that the direct `key` replaces the `cacheKey` selector. `defaultConfig` and `fallbackTimeoutMs` are validated and snapshotted for each invocation. Outside an enabled scope, DialCache invokes `load` directly without constructing a key or resolving runtime policy. - -`getOrLoad()` does not register its `useCase`, so repeated calls should reuse one stable, deployment-defined name such as `"BuildProfile"`. Keep it bounded: never derive `useCase` from a user, request, id, or other high-cardinality input because it is part of both cache identity and metrics labels. Put those values in `key` instead. - -Every captured value that can change the result belongs in the bare id or `{ id, args }` key. Concurrent same-key calls may share one caller's in-flight loader and cached value, so all call sites for that identity must also agree on value meaning and serialization. Prefer `cached()` when a loader is reusable; prefer `getOrLoad()` when the calculation is intentionally local to one call site. - -## Keys, ids, and extra dimensions - -For `cached()`, the key comes from the required `cacheKey` selector whose parameters are inferred from `fn`. `getOrLoad()` accepts the same bare id or `{ id, args }` shape directly through `key`: - -The selected or direct key is the value identity contract. It must include every input dimension that can affect the returned value; otherwise distinct calls can reuse the same cached value or share the same in-flight fallback through request coalescing. - -```ts -const searchPosts = dialcache.cached( - (userId: string, page: number, filter: string) => db.searchPosts(userId, page, filter), - { - keyType: "user_id", - useCase: "SearchPosts", - cacheKey: (userId, page, filter) => ({ id: userId, args: { page, filter } }), - defaultConfig: DialCacheKeyConfig.enabled(60), - }, -); -await dialcache.enable(() => searchPosts("u1", 2, "active")); -``` - -`DialCacheConfig.namespace` is the logical cache namespace and the first component of every key. It defaults to `"urn"`, producing keys such as `urn:user_id:123#GetUser`. Set a stable application-specific value when multiple applications may use the same Redis deployment: - -```ts -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: redisClient }, -}); -``` - -That produces Redis keys beginning with `users-api:...`, or `{users-api:...}` for invalidation-tracked values. `namespace` is DialCache's single cache-identity and key-partitioning setting: it participates in request-local, process-local, Redis, coalescing, deterministic ramp, invalidation, and metrics. It may not contain `{` or `}` because DialCache reserves those characters for Redis Cluster hash tags. Use a namespace to express any required application or environment separation, such as `production-users-api`. - -- **`keyType` + `id` is the invalidation unit for tracked Redis entries.** `dialcache.invalidateRemote("user_id", "123", futureBufferMs)` writes one watermark for that user; any `trackForInvalidation` Redis entry with the same `keyType` and `id` is refreshed across all `args` variants when Redis is read. `invalidateRemote` does not evict existing request-local or process-local entries (see [Targeted invalidation](#targeted-invalidation-and-watermarks)), and untracked Redis entries do not consult the watermark. `useCase` identifies the individual cache (it's the metrics label and part of the stored key). -- **`args` are part of the cache key** — different `args` produce different entries — but invalidation is by `id` only. -- **Scalar key equality is string-based.** Runtime type is not an identity dimension: for matching surrounding dimensions, numeric `1`, string `"1"`, and bigint `1n` identify the same key; argument values `null` and `"null"` also match. `-0` matches `0`, and an `undefined` argument is omitted. If a deployment changes the logical meaning represented by a scalar, change an explicit identity dimension such as `keyType`, `useCase`, or an argument name/value. -- **Non-key inputs** (for example a db handle) are parameters ignored by a `cacheKey` selector or values captured by a `getOrLoad()` loader. They still reach non-coalesced executions, but concurrent same-key cache misses share the leader's execution, so do not omit values like auth context, locale, or cancellation behavior unless sharing one result is correct. -- **Methods:** pass `obj.method.bind(obj)` (or `(...a) => obj.method(...a)`) — a bare `obj.method` reference loses `this`. - -Changing the namespace value intentionally creates a cold-cache boundary across every layer. Old and new keyspaces do not share Redis values or invalidation watermarks. During an overlapping deployment, an invalidation handled by one version is invisible to the other, which can continue serving a stale tracked value until its value TTL expires. If remote invalidation correctness matters, a normal rolling deployment is unsafe: use a coordinated no-overlap cutover, or an operational bridge that prevents both versions from serving remote cache across mutations (for example, temporarily disable and clear remote caching during the transition). After the cutover, provision for fallback/refill load and allow old Redis keys to expire by TTL. - -## Runtime config and ramp controls - -Instance-wide behavior is set through the `DialCache` constructor: - -| `DialCacheConfig` option | Default | Description | -| --- | --- | --- | -| `namespace` | `"urn"` | Logical cache namespace and first key component (see [Keys, ids, and extra dimensions](#keys-ids-and-extra-dimensions)). | -| `redis` | none | `{ client: DialCacheRedisClient, readTimeoutMs?: number }`; enables the Redis layer with a 50 ms default read deadline (see [Redis-backed TTL cache](#redis-backed-ttl-cache)). | -| `localMaxSize` | `10_000` | Global process-local entry cap; `0` disables process-local storage. Nonnegative safe integer. | -| `shadowMaxInFlight` | `1` | Maximum scheduled or active shadow jobs per `DialCache` instance, including uncancellable underlying work. Positive safe integer; excess work is dropped without queuing. | -| `cacheConfigProvider` | none | Resolves runtime config per enabled invocation as a sparse overlay on the function's `defaultConfig`; `null` applies no overrides. | -| `metrics` | disabled | A `DialCacheMetricsAdapter` (see [Metrics](#metrics)). | -| `logger` | `console` | Receives operational cache failures and opted-in confirmed shadow mismatch warnings (`debug`, `warn`, `error`). Synchronous throws and rejections from returned promises or thenables are isolated without being awaited. | - -Per-invocation cache policy is a `DialCacheKeyConfig`: per-layer `ttlSec` and `ramp` maps keyed by `CacheLayer.LOCAL` (process-local) and `CacheLayer.REMOTE` (Redis), a `requestLocal` boolean, an optional `remoteReadTimeoutMs`, and an optional `shadow` group. `ShadowConfig` contains the independent shadow `ramp` percentage plus the default-off `logMismatches` control. - -Every cached definition or `getOrLoad()` invocation can provide an optional per-use-case `defaultConfig`. It is the baseline policy, and the `cacheConfigProvider` result is a sparse field-level overlay on that baseline. For cache enablement fields, precedence is runtime config, then `defaultConfig`, then DialCache's disabled baseline. For the remote-read deadline, precedence is runtime `remoteReadTimeoutMs`, `defaultConfig.remoteReadTimeoutMs`, `redis.readTimeoutMs`, then the 50 ms library default. - -The disabled baseline sets `requestLocal` to false, leaves the process-local and Redis TTLs unset, and sets `shadow.ramp` to 0% with mismatch logging false. A shared layer with no effective TTL is disabled by policy. When a shared layer has an effective TTL but no effective ramp, its ramp defaults to 100%. Shadow work remains disabled unless `shadow.ramp` is explicitly greater than zero. - -`DialCacheKeyConfig` preserves an omitted `requestLocal` as `undefined` so the overlay can distinguish omission from an explicit `false`; the effective value still defaults to false after resolution. - -A provider result of `null` (or defensive `undefined`) applies no overrides. An empty `DialCacheKeyConfig` and omitted runtime fields also inherit the baseline. Top-level fields, cache-layer leaves, and leaves inside `shadow` merge independently; an explicit `false` logging flag overrides an inherited `true`. Use explicit values to override inherited policy: `requestLocal: false` disables request-local caching and a layer ramp of `0` disables that shared layer. `DialCacheKeyConfig.disabled()` is the complete new-cache-invocation kill switch in one call: request-local and shadow work off, shadow logging off, and both shared layers ramped to 0. It does not cancel already-admitted work, and explicit maintenance operations such as `invalidateRemote()` remain available. To stop new cache-invocation Redis reads and fills while preserving other runtime settings, explicitly set both `ramp.remote` and `shadow.ramp` to `0`; the remote ramp alone stops serving but does not override an inherited nonzero shadow ramp. - -DialCache validates `defaultConfig` when `cached()` registers a definition and whenever `getOrLoad()` is invoked: TTLs must be positive safe integers no greater than 31,536,000 seconds (a fixed 365-day duration), remote-read deadlines must be positive safe integers within their documented limit, layer and shadow ramps must be finite percentages from 0 to 100, layer maps and `shadow` must be objects, and `requestLocal` and `shadow.logMismatches` must be booleans when present. Invalid defaults are rejected immediately. - -Each registration or one-shot invocation captures an immutable internal snapshot of `defaultConfig`; mutating the supplied config or its maps later does not change that operation's baseline. Runtime policy changes belong in the provider's returned overlay. - -Runtime TTL and ramp leaves are used as supplied instead of falling back to valid default leaves. A TTL outside the same 1-to-31,536,000-second range disables that layer with `invalid_ttl`; a nonnumeric, non-finite, or out-of-range ramp disables it with `invalid_ramp`. Valid ramps include both `0` and `100`. Other layers can still run, and invalid leaves also record a `config_resolution` error so provider garbage is alertable separately from intentional ramp-downs. A malformed runtime config object, layer-map shape, `requestLocal` value, explicit `remoteReadTimeoutMs`, or removed top-level `shadowRamp` fails config resolution for the invocation, records `config_error`, and executes the fallback uncached without attempting Redis. The public `DialCacheKeyConfig` constructor and static defaults likewise reject `shadowRamp` immediately; migrate it to `shadow.ramp`. - -An invalid runtime `shadow.ramp` does not affect the cache result or disable an otherwise valid Redis policy. If normal traversal reaches an otherwise shadow-eligible Redis path, DialCache skips shadow work and records a `config_resolution` error. An invalid runtime `shadow.logMismatches` likewise preserves the cache result, Redis policy, shadow result, and shadow metric while suppressing the warning. DialCache validates this diagnostic leaf only after the metrics hook, exact-key cohort, and capacity gates admit shadow work, then records one remote `config_resolution` error for that admitted resolution. - -`cacheConfigProvider` is called for every enabled cache invocation before DialCache performs any cache lookup. Keep it cheap, cache any remote/config-store reads inside the provider, and avoid work that would erase the benefit of a cache hit. +The example below focuses on runtime policy. Remote ramp settings take effect +only when a Redis or Valkey client is configured. ```ts import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; -const dialcache = new DialCache({ - cacheConfigProvider: async (key) => { - if (key.useCase === "GetUser") { - return new DialCacheKeyConfig({ - // Sparse override: inherit both TTLs and the local ramp from defaultConfig. - ramp: { [CacheLayer.REMOTE]: 25 }, - // Independently sample tracked Redis keys for detached validation/fill. - shadow: { - ramp: 5, - // Emit one warning with a bounded key and native-JSON value strings. - logMismatches: true, - }, - // Can be changed by the provider at runtime for this use case. - remoteReadTimeoutMs: 35, - }); - } - return null; // apply no overrides; use the cached function's baseline - }, -}); - -const getUser = dialcache.cached((userId: string) => db.fetchUser(userId), { - keyType: "user_id", - useCase: "GetUser", - cacheKey: (userId) => userId, - defaultConfig: new DialCacheKeyConfig({ - // Omitted ramps default to 100% because these layers have TTLs. - ttlSec: { [CacheLayer.LOCAL]: 30, [CacheLayer.REMOTE]: 300 }, - }), -}); -``` - -`ramp` values are percentages from 0 to 100. `0` disables the layer, `100` enables it, and intermediate values are deterministically sampled by cache key and layer, so the same key is consistently sampled in or out of a partial rollout across calls and instances. The assignment algorithm is owned by DialCache and remains stable across releases. Applications that need an externally coordinated cohort can use `cacheConfigProvider` to return a sparse per-key ramp override of `0` or `100`. DialCache fetches and resolves one config snapshot per enabled invocation. Provider errors do not activate defaults: they fail open, record `config_error`, and execute the fallback function uncached. - -`shadow.ramp` uses the same inclusive 0–100 percentage domain but is independent of cache-layer serving ramps. Omission and `0` disable shadow work; `100` selects every eligible tracked Redis key; intermediate values assign each exact cache key to a stable shadow cohort across calls and instances. A valid remote policy can therefore use `ramp.remote: 0` with a nonzero `shadow.ramp` to exercise and populate Redis without serving from it. A nonzero value explicitly authorizes detached tracked writes after clean shadow-only misses. It does not create another `CacheLayer`, activate Redis without a valid remote TTL, or make a request-local/process-local hit continue to Redis. - -Remote serving and shadow sampling use independent deterministic cohorts. Equal partial percentages do not imply the same keys, so a partial shadow cohort does not guarantee that every key admitted by a later partial serving ramp was warmed or validated. Use `shadow: { ramp: 100 }` when every otherwise eligible invocation must exercise the non-serving Redis path before a serving-ramp increase. - -## Cache layers - -### Request-local cache - -Set `requestLocal: true` to memoize resolved values for the lifetime of the outermost `enable()` scope: - -```ts -import { DialCache, DialCacheKeyConfig } from "dialcache"; - -const dialcache = new DialCache(); -const getUser = dialcache.cached( - (userId: string) => db.fetchUser(userId), - { - keyType: "user_id", - useCase: "GetUser", - cacheKey: (userId) => userId, - defaultConfig: new DialCacheKeyConfig({ requestLocal: true }), - }, -); -``` - -`requestLocal` is a runtime boolean rather than a TTL/ramp-controlled `CacheLayer`. The `cacheConfigProvider` can turn it on or off for each invocation. `DialCacheKeyConfig.enabled(ttlSec)` enables only process-local and Redis caching, so request-local caching must be selected explicitly. - -DialCache resolves the runtime config once per enabled invocation and uses it for the entire lookup. When the effective `requestLocal` value is false, the invocation skips request-local lookup and storage without deleting an entry already memoized in the scope. A later invocation that enables request-local caching can reuse that entry. - -The outermost `enable()` call owns the request-local lifetime, and nested `enable()` calls reuse that scope. Request-local state is allocated lazily, only when an invocation enables the layer, so scopes that use only process-local or Redis caching do not allocate it. - -Wrap the complete Node HTTP handler so the request-local scope matches the handler's lifetime: - -```ts -import { createServer } from "node:http"; - -const server = createServer((req, res) => { - void dialcache - .enable(async () => { - const user = await getUser(readUserId(req)); - res.setHeader("content-type", "application/json"); - res.end(JSON.stringify(user)); - }) - .catch((error: unknown) => handleRequestError(error, res)); -}); -``` - -Request-local storage has no capacity limit, eviction, or overflow mode. Entries are retained until the outermost `enable()` callback settles. Use it for short-lived scopes with bounded key cardinality; split long-running streams or large batch jobs into smaller scopes when necessary. - -### Process-local cache - -The process-local layer (`CacheLayer.LOCAL`) uses one LRU per `DialCache` instance. It keeps at most 10,000 entries by default across all use cases while retaining each entry's configured TTL. Set `localMaxSize` to a nonnegative safe integer to change the global entry cap; `0` disables process-local storage: - -```ts -const dialcache = new DialCache({ localMaxSize: 25_000 }); -``` - -The limit counts entries rather than estimating JavaScript object memory. Recently read entries stay resident ahead of less recently used entries when the limit is reached. - -### Redis-backed TTL cache - -The Redis layer supports standalone Redis, Valkey, and Redis Cluster. Register DialCache's native node-redis scripts when creating the client, then pass that client to DialCache: - -```ts -import { createClient } from "redis"; -import { DialCache } from "dialcache"; -import { createNodeRedisDialCacheClient, dialcacheRedisScripts } from "dialcache/node-redis"; - -const redisClient = createClient({ - url: process.env.REDIS_URL, - scripts: dialcacheRedisScripts, - disableOfflineQueue: true, - commandsQueueMaxLength: 1_000, - socket: { connectTimeout: 2_000 }, -}); -await redisClient.connect(); - -const dialcache = new DialCache({ - namespace: "users-api", - redis: { - client: createNodeRedisDialCacheClient(redisClient), - // Optional instance default; omit to use DialCache's 50 ms default. - readTimeoutMs: 100, - }, -}); - -async function shutdown(): Promise { - // Stop new work and await every outstanding request-path call and invalidation first. - // Detached shadow work is best-effort and has no drain handle. - await redisClient.quit(); -} -``` - -`redis.client` is required when Redis is configured and accepts the semantic `DialCacheRedisClient` interface. `redis.readTimeoutMs` is optional and sets the instance default for remote reads; omit it to use 50 ms. Create and connect the underlying client before constructing `DialCache`. Node-redis users should register the supplied scripts and wrap their client with `createNodeRedisDialCacheClient` as shown above. - -Valkey GLIDE users pass an already-created standalone or cluster client and its -module namespace to the GLIDE adapter: - -```ts -import * as valkeyGlide from "@valkey/valkey-glide"; -import { DialCache } from "dialcache"; -import { createValkeyGlideDialCacheClient } from "dialcache/valkey-glide"; - -const glideClient = await valkeyGlide.GlideClient.createClient({ - addresses: [{ host: "127.0.0.1", port: 6379 }], - requestTimeout: 2_000, - advancedConfiguration: { connectionTimeout: 2_000 }, -}); -const redisClient = createValkeyGlideDialCacheClient(glideClient, valkeyGlide); -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: redisClient }, -}); - -function shutdown(): void { - // After draining request-path calls and invalidations, release scripts before closing GLIDE. - // Detached shadow work is best-effort and has no drain handle. - redisClient.dispose(); - glideClient.close(); -} -``` - -Pass the same module namespace that created the client. DialCache uses its -`Script` constructor and `Decoder.Bytes` value without importing a GLIDE runtime -itself, so linked workspaces and applications with another installed GLIDE -version cannot accidentally mix native script handles. - -The application owns the complete Redis lifecycle. It creates and connects the underlying client and passes the semantic adapter to DialCache. During shutdown, stop starting DialCache-backed work and await every promise returned by a cached function, `getOrLoad()`, or `invalidateRemote()`, including calls still running fallbacks that may later write Redis. A read that crossed DialCache's wait deadline may still be active inside the client, so use client-native telemetry and shutdown controls to drain or terminate that work before disposing adapter-owned resources and closing the connection. DialCache only borrows `redis.client`; it has no close or drain method and never disposes or closes caller resources. - -Awaiting those public promises does not drain detached shadow work. Shadow scheduling and deadline timers are unreferenced and completion is not guaranteed during shutdown; Redis operations, source reads, serializers, and asynchronous telemetry already started by shadow work remain caller-owned and may still be active. Stop new work before closing their dependencies and accept that an in-flight shadow fill may have been dispatched even if its final outcome is lost during teardown. DialCache does not add a shutdown hook or keep the process alive to deliver best-effort outcomes. - -The node-redis adapter owns no additional resources, so the application closes the underlying node-redis client after draining work. The GLIDE adapter owns five native `Script` handles but not the wrapped connection. After outstanding operations finish, call its idempotent `dispose()` before closing GLIDE as shown above; disposal while an adapter operation is in flight throws rather than releasing a live script. - -Node-redis computes each script's SHA, uses `EVALSHA`, and retries with `EVAL` after `NOSCRIPT`. Its cluster client routes scripts by their first key and performs that fallback on the selected shard. The GLIDE adapter uses GLIDE's native `Script` lifecycle and byte decoder; GLIDE routes scripts from their declared keys. Tracked reads are deliberately routed to primaries so a lagging replica cannot hide an invalidation watermark. - -#### Remote read deadlines and async liveness - -DialCache bounds every active Redis read. The effective timeout is resolved per use case and per invocation: runtime `remoteReadTimeoutMs`, then `defaultConfig.remoteReadTimeoutMs`, then optional instance `redis.readTimeoutMs`, then 50 ms. Values must be positive safe integers no greater than 2,147,483,647. There is no unbounded escape hatch for remote reads. - -When the deadline expires, DialCache aborts the optional `RedisReadContext.signal`, records one `cache_read_timeout` error, logs a `RedisReadTimeoutError`, and starts the source fallback. Late read fulfillment or rejection is consumed and ignored. A read failure or timeout never triggers a post-fallback Redis write; an untracked active process-local miss may retain the source value, while a tracked key suppresses local publication because the failed read did not establish watermark safety. - -Same-key followers share the leader's remaining remote-read budget. The timer covers only the semantic Redis read, not config resolution, serializer load, fallback work, Redis writes, or invalidation. `fallbackTimeoutMs` starts separately when the source fallback begins. - -The bundled node-redis adapter passes the signal through per-command options, which can remove queued work where supported. Aborting after dispatch does not unsend a command or prove that Redis stopped executing it. GLIDE's current script API has no per-invocation signal, so its invocation may continue after DialCache has fallen back. Keep client-native connection, retry, queue, and response budgets in place; they bound underlying resource lifetime while DialCache's deadline bounds caller wait time. - -Writes, invalidations, async `cacheConfigProvider` calls, and custom serializer methods still need finite application-owned budgets. Do not put mutations behind a bare `Promise.race`: rejecting the outer promise neither removes queued work nor proves whether a dispatched mutation executed. - -#### Serialization - -The core Redis boundary is the client-agnostic `DialCacheRedisClient` interface. It exchanges serialized values as `string | Buffer` and does not expose client commands or wire encodings. Distinct untracked/tracked read and write Lua sources, the invalidation source, and wire constants are available from `dialcache/redis-protocol`. Custom adapters can throw the root-exported `DialCacheRedisPayloadError`, `DialCacheRedisPayloadEncodingError`, and `DialCacheRedisProtocolError` classes to distinguish malformed payloads, unsupported encodings, and Lua reply-domain violations in logs. DialCache records bounded `cache_read`, `cache_write`, or `invalidation` metrics by failure site. - -Redis values use a compact binary frame: - -```text -byte 1 format version -bytes 2-9 Redis-created timestamp in milliseconds (uint64, big-endian) -byte 10 payload encoding (0 = UTF-8, 1 = raw binary) -bytes 11... serialized payload -``` - -Redis's Lua `struct` library packs and unpacks the timestamp. Redis TTL is authoritative, so expiry metadata is not duplicated in the frame. `payload` is produced by the operation's serializer, or by `JsonSerializer` by default. Custom serializers can return either `string` or `Buffer`; strings are stored as UTF-8 and Buffers are stored byte-for-byte without base64 expansion. Adapters restore the same representation before calling `serializer.load`. - -DialCache uses native `JSON.stringify` and `JSON.parse` by default. There is no runtime validation pass, so the default adds no traversal beyond JSON serialization itself. A top-level `undefined` result is supported with an internal sentinel. - -When `serializer.load` rejects a Redis payload, DialCache records a `serialization_load` error, counts the read as a remote cache miss, runs the fallback, and attempts to replace the rejected payload. A validating custom serializer can therefore treat an incompatible cached value as a refreshable miss without adding a schema version to the cache key. - -`JsonSerializer` validates JSON syntax only. It cannot detect that a structurally valid payload came from an incompatible application value schema. Applications that keep the same `useCase` across deployments must keep default-JSON values backward compatible. For an incompatible change, either provide a serializer whose `load` method validates and rejects the old shape, or change `useCase` to isolate the new cache entries. On the caller-serving Redis path, mutually incompatible validating serializers in a mixed deployment can repeatedly reject and replace each other's values; correctness is preserved, but expect additional fallback and Redis-write load until the rollout converges. Shadow work reports a non-null payload that fails `load` as `deserialization_error` and never replaces it. - -When a cached function or inline loader's resolved return type is statically JSON-compatible, `serializer` remains optional. This includes JSON primitives, arrays, plain object/interface shapes, optional object fields, and a top-level `undefined`. Types known not to survive the default round trip require a typed `Serializer`: - -```ts -import { DialCache, type Serializer } from "dialcache"; - -const dialcache = new DialCache(); -const dateSerializer: Serializer = { - dump: (value) => value.toISOString(), - load: (value) => new Date(Buffer.isBuffer(value) ? value.toString("utf8") : value), -}; - -const getUpdatedAt = dialcache.cached( - (userId: string) => db.fetchUpdatedAt(userId), - { - keyType: "user_id", - useCase: "GetUpdatedAt", - cacheKey: (userId) => userId, - serializer: dateSerializer, - }, -); -``` - -The compile-time guard rejects known incompatible shapes such as `Date`, `Map`, `Set`, `bigint`, symbols, functions, Buffers, typed arrays, method-bearing class instances, required nested `undefined`, `unknown`, and `any`. It applies to every `cached()` declaration and `getOrLoad()` invocation because active layers are selected at runtime. A global Redis serializer is not parameterized by each returned type, so it cannot discharge this requirement; non-JSON operations must select a typed serializer. - -This guard is deliberately conservative and is not a proof of runtime data. TypeScript cannot detect non-finite numbers, cyclic/shared references, runtime getter or `toJSON` behavior, or data-only class instances that look like plain objects. Opaque, generic, or deeply recursive types may also require an explicit serializer. Providing `Serializer` (including an explicitly typed `JsonSerializer`) is a trusted caller assertion; DialCache does not serialize-and-deserialize again to validate it. - -#### Shadow validation - -Shadow mode runs a sampled, detached Redis path for tracked keys without allowing that path to serve the caller. A Redis hit is compared with the source of truth (SoT); a clean Redis miss can be filled from the caller-accepted SoT value. Redis serving and shadow execution are independent, and shadowing is opt-in per use case through `shadow.ramp`: - -```ts -import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; +const runtimePolicies = new Map(); const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: redisClient }, - // A bundled or custom adapter with shadowValidation support is required. - metrics, - // At most one scheduled or running shadow job by default; tune deliberately. - shadowMaxInFlight: 4, + cacheConfigProvider: (key) => runtimePolicies.get(key.useCase) ?? null, }); const getUser = dialcache.cached( @@ -479,381 +174,310 @@ const getUser = dialcache.cached( keyType: "user_id", useCase: "GetUser", cacheKey: (userId) => userId, - trackForInvalidation: true, - // Optional: override strict deep equality with use-case semantics. - shadowComparator: (cached, source) => - cached.id === source.id && cached.version === source.version, defaultConfig: new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.REMOTE]: 300 }, - // Exercise and populate Redis without serving it to callers. - ramp: { [CacheLayer.REMOTE]: 0 }, - shadow: { - ramp: 5, - // Default-off warning with a bounded key and native-JSON value strings. - // Enable only after approving the logger and data-handling policy. - logMismatches: true, + ttlSec: { + [CacheLayer.LOCAL]: 60, + [CacheLayer.REMOTE]: 60, + }, + ramp: { + [CacheLayer.LOCAL]: 0, + [CacheLayer.REMOTE]: 0, }, + shadow: { ramp: 0 }, }), }, ); -``` - -Shadow work is eligible only when the operation sets `trackForInvalidation: true`, a valid remote TTL/policy exists, its effective `shadow.ramp` selects the exact cache key, a configured metrics adapter implements `shadowValidation`, and capacity is available. Logging is supplemental to that metric; enabling `logMismatches` does not activate shadow work without the metrics hook. The bundled Prometheus and Datadog adapters implement that hook. There are two paths: - -- When remote serving is enabled and produces a tracked Redis hit, DialCache retains the exact serialized payload that supplied the caller as `C0`. -- When the remote policy is valid but disabled specifically by `ramped_down`, DialCache starts a detached tracked Redis read for `C0`. Its result can be validated or used to decide whether a clean miss may be filled, but can never supply the caller or populate an in-memory layer. - -A missing or invalid remote policy, config-provider failure, absent Redis client, disabled call, untracked key, omitted metrics hook, zero/omitted shadow ramp, cohort exclusion, capacity rejection, or earlier request-local/process-local hit does not launch a shadow-only Redis path. Shadow work begins only if normal traversal reaches the Redis layer. A normally enabled remote miss already follows the caller's ordinary fallback-and-fill path and does not launch a duplicate shadow fill. - -On a served hit, DialCache returns the already-decoded cached value before starting the SoT read or any confirmation work. On a ramped-down path, the caller invokes and awaits its normal configured fallback exactly once and receives only that result; detached work reuses the same accepted `S` instead of calling the loader again. The caller never awaits shadow `C0`, comparison, confirmation `C1`, shadow serialization, or fill. Slow, failed, or timed-out shadow work cannot delay, reject, or change the caller result. - -The detached job uses this bounded algorithm: - -1. Obtain the original tracked Redis payload as `C0`. -2. If `C0` is missing, wait for the caller's successfully accepted `S` after its configured fallback boundary and attempt one normal tracked Redis write using the resolved TTL and watermark machinery. Before the whole-job deadline, emit `filled` when Redis accepts it, `fill_blocked` when the invalidation watermark rejects it, or `fill_error` when serialization or the write fails. -3. If `C0` is non-null, obtain `S`, deserialize an isolated snapshot of `C0`, and run the default or custom semantic comparator. Any non-null `C0` is observation-only: DialCache never repairs or overwrites it, including when deserialization fails. -4. If `C0` and `S` match semantically, emit `match` without another Redis read. -5. Otherwise, reread tracked Redis directly as `C1`, bypassing request-local and process-local cache. -6. If `C1` is missing or differs byte-for-byte from `C0`, emit `superseded`; if it is identical, emit `mismatch`. -7. If the confirmation read fails or reaches its Redis-read deadline, emit `confirmation_error`. - -Here a clean miss means the tracked semantic Redis read returned `null`; it does not include a non-null payload that later fails deserialization. A caller fallback rejection or timeout never becomes accepted `S` and never starts the fill. - -Both detached Redis reads use the existing watermark-aware tracked-read protocol, primary-read behavior, and effective `remoteReadTimeoutMs`. The clean-miss fill uses the same serializer, TTL, Redis-time timestamp, and invalidation watermark as an ordinary tracked fill. Strings compare exactly, Buffers compare by bytes, and string/Buffer pairs compare by their UTF-8 bytes. DialCache does not deserialize `C1`, compare it with `S`, or chase another version. - -The detached scheduler, Redis-read deadline timers, and overall shadow deadline timer are unreferenced, so they do not keep an otherwise idle process alive. Detachment is asynchronous work on the Node event loop, not a worker thread: synchronous source, serializer, or comparator work can still occupy the event loop after the request path has been released. - -Detached execution retains the original `cached()` argument references or `getOrLoad()` loader closure; DialCache cannot generically clone them. Treat object arguments, captured source-selection state, and the returned `S` as immutable, or snapshot them before invoking DialCache. Mutating them after the caller continues can compare or serialize a value that no longer corresponds to the already-built key. - -`shadowMaxInFlight` is a per-instance positive safe integer and defaults to `1`. It counts admitted jobs until shadow-owned Redis/source/serializer/comparator work settles, including detached reads or dispatched writes whose DialCache deadline already elapsed. The optional `C1` and clean-miss fill remain in the original slot. On a ramped-down path, shadow work shares the caller's SoT promise; once the shadow deadline expires, the raw caller-owned loader may continue without retaining the shadow slot, including when `fallbackTimeoutMs` is `null`. DialCache also suppresses another job for the same exact key while shadow-owned work remains active. There is no queue: exact-key duplicates and work above the instance cap are dropped and reported as `dropped`. Separate instances have independent limits, so this is not a fleet-wide source-of-truth or Redis concurrency cap. - -Each job has one monotonic deadline across detached `C0`, the SoT result, serializer work, comparison, optional `C1`, and clean-miss fill. Served-hit timing begins when its detached validation callback starts. On a ramped-down path, timing begins immediately before the caller's SoT invocation so synchronous source work that runs before admission still consumes the same budget. Each Redis read also has its effective read deadline. A finite `fallbackTimeoutMs` is reused as the overall shadow budget. When `fallbackTimeoutMs` is `null`, the normal fallback remains intentionally unbounded, but detached shadow work still uses a 60-second budget. Once timeout delivery marks a job abandoned, DialCache releases retained `C0` references and prevents later phases from starting. - -JavaScript promises and Redis writes do not provide a general cancellation or transaction boundary. Work already dispatched may continue and keeps the shadow slot until it settles. A write rejection, `fill_error`, or shadow `timeout` after dispatch does not prove that Redis was unchanged; the command may have executed before its result became unavailable. Conversely, `filled` means the semantic client returned success before the shadow deadline, not that the value is still present. Give dependencies finite native budgets and treat shadow outcomes as best-effort operational evidence. - -A `match` means application-level value equality: - -- By default, DialCache uses Node's `util.isDeepStrictEqual`. Plain-object property insertion order does not affect the result; values, array order, prototypes, constructors, Buffers, Maps, Sets, and other supported structures remain strictly compared. -- An optional typed `shadowComparator(cachedValue, sourceValue)` on `cached()` or `getOrLoad()` can define narrower domain equality, such as ignoring a volatile timestamp. It must synchronously return a boolean and must be deterministic, side-effect-free, non-mutating, and bounded. -- A comparator throw or non-boolean return is `comparison_error`, never `mismatch`. An accidental promise is not accepted as a comparison result; DialCache consumes its settlement while retaining the shadow slot, subject to the same detached deadline. - -DialCache retains the semantic `string | Buffer` returned by the Redis client but never exposes it to the comparator. After the source read completes, detached work calls the same effective serializer's `load` method to create an independent cached snapshot, then compares that snapshot with the raw value returned by the source loader. It does not reuse the cached object already returned to a served-hit caller, so caller mutation cannot contaminate validation. No payload copy, shadow deserialization, deep comparison, or hash is added to the served-hit request path. - -The effective serializer's `load` method therefore runs a second time for a sampled served hit and once in detached work for a shadow-only hit. It must be repeatable, non-mutating, and return independently usable values. On a clean miss, its `dump` method may run after the caller has received `S`, so `S` must remain immutable through detached serialization. A custom `DialCacheRedisClient` must return an operation-owned payload whose string/Buffer contents remain stable after `read()` settles. Comparing the deserialized cached snapshot with the raw source value intentionally detects lossy serialization; use a custom comparator only when such normalization or ignored fields are valid use-case semantics. - -Shadow metrics use the bounded outcomes `match`, `mismatch`, `superseded`, `filled`, `fill_blocked`, `fill_error`, `redis_error`, `source_error`, `deserialization_error`, `comparison_error`, `confirmation_error`, `timeout`, and `dropped`. `redis_error` applies to the initial shadow-only `C0` read; `confirmation_error` applies to `C1`. A clean `C0` miss is an ordinary `miss{layer="remote_shadow"}` and terminates with a fill, source, or timeout outcome rather than a second shadow outcome for the miss itself. Labels never contain cache ids, values, payloads, Redis keys, or raw exception text. - -Confirmed-mismatch logging is separately opt-in through `shadow.logMismatches`; it never replaces the required `shadowValidation` metric or emits for a mismatch candidate that becomes `superseded`. The single warning contains `cacheNamespace`, `useCase`, `keyType`, `outcome: "mismatch"`, `cacheKey`, `cachedValueJson`, and `sourceValueJson`. `cacheKey` is the logical DialCache URN capped at 2 KiB, not the physical Redis storage key. DialCache independently applies native `JSON.stringify` to the deserialized cached snapshot and raw source value supplied to the comparator, then caps each resulting string at 8 KiB. A byte-clipped field ends in `...[truncated]`, counted inside its cap. If native JSON throws or returns `undefined`, the corresponding JSON field is `null`; the other side is still attempted. DialCache does not compute a textual diff or call the configured serializer again for logging. -Mismatch logging is intentionally default-off. Logical URNs can contain ids and arguments, while cached and source values may contain secrets or personal data; truncation is not redaction. DialCache creates the JSON strings only after terminal mismatch confirmation and never passes the raw compared-value references to the logger. Native JSON semantics apply: getters and `toJSON` methods may run, unsupported values may be omitted or normalized, and cycles or `bigint` can make a field unavailable. Stringification is synchronous, and the 8 KiB caps apply only after `JSON.stringify` returns; they do not bound input traversal, hook execution, event-loop time, or the intermediate JSON string. Enable mismatch logging only for trusted, reasonably bounded values and with an approved logger, redaction, transport, access, and retention policy. - -The byte caps apply before logger framing or escaping, so they do not guarantee a final transport event below a sink-specific limit; the metadata fields are not size-clamped. A detail-construction failure degrades to the metadata-only warning. Logger throws and rejected promises or thenables remain isolated from cache and shadow correctness. - -Detached Redis reads, serializer loads/dumps, payload sizes, and read/write errors use the existing layer label with `layer="remote_shadow"`. This distinguishes non-serving Redis cost from caller-path `layer="remote"` telemetry without adding a metric or label. The established `observeGet{layer="remote"}` boundary includes caller-path deserialization, while `observeGet{layer="remote_shadow"}` ends when the deadline-bounded Redis read result settles; detached serializer work and any later raw-client settlement are outside that timer. The request-path read that supplied a served `C0` keeps `layer="remote"`, and a ramped-down caller keeps `disabled{layer="remote", reason="ramped_down"}`. No `disabled{layer="remote_shadow"}` event is emitted for ineligible or dropped work; `dropped` remains the terminal shadow outcome. Confirmation reads use the same `remote_shadow` value, with `superseded` or `confirmation_error` describing their role. - -The command amplification is bounded: a selected served hit adds one SoT read and adds `C1` only for a semantic mismatch candidate; a selected ramped-down hit adds detached `C0`, reuses the caller's existing SoT read, and likewise adds `C1` only for a candidate; a selected ramped-down miss adds detached `C0` and at most one tracked fill. `superseded` means only that the original observation could not be confirmed. `mismatch` means the exact `C0` payload survived a tracked Redis read after the SoT disagreement; it is not a cross-system atomic snapshot or a guarantee that the mismatch persists. - -The initial `C0` read and later fill are not atomic. The fill is a normal tracked overwrite, not a compare-and-set or write-if-still-missing operation: another writer can populate Redis after the clean miss and be overwritten by the shadow fill. Invalidation watermarks still fence writes using Redis time, so size `futureBufferMs` to cover the complete SoT, serialization, client queue, network, and write interval when stale-publication protection matters. Shadow mode never repairs a non-null `C0`, refreshes its TTL, invalidates, evicts local state, or changes the value returned to the caller. - -A served-hit sample invokes the wrapped function or inline loader as an additional source read, so that loader must be safe to call for observation. A ramped-down sample reuses the caller's ordinary invocation and does not add another SoT call. - -For valid policies, shadow-specific source calls, cache-path Redis traffic, returned values, and metrics are unchanged when `shadow` is omitted or `shadow.ramp` is `0`. Shadow policy is grouped under `DialCacheKeyConfig.shadow`; consumers of the former flat ramp field must migrate to `shadow: { ramp }`. The public constructor, static defaults, and runtime provider results reject the removed field. `DialCacheKeyConfig.disabled()` explicitly disables the shadow ramp and mismatch logging. `shadowComparator` remains a typed `cached()` / `getOrLoad()` option because it defines stable use-case equality, while `shadowMaxInFlight` remains a per-instance concurrency limit. The clean-miss bootstrap adds no additional ramp knob, Redis protocol operation, metric instrument, or label key; enabling shadowing authorizes the tracked Redis write described above. Exported unions include `remote_shadow` in `MetricLayer` and `superseded`, `filled`, `fill_blocked`, `fill_error`, `redis_error`, and `confirmation_error` in `ShadowValidationOutcome`. TypeScript consumers with exhaustive switches or `Record` values must include those cases, and dashboards restricted to `layer="remote"` intentionally exclude detached traffic. - -## Cached-value ownership - -Treat values returned by cached functions or `getOrLoad()` as immutable. DialCache does not clone or freeze values stored in request-local or process-local memory. Mutating a cached object can therefore be observed by later callers in the same request, callers in other requests that hit the process-local cache, or callers that coalesced onto the same in-flight result. - -This contract includes nested objects and arrays, `Map`, `Set`, `Buffer`, typed arrays, and class instances. Redis deserialization can produce a different reference from an in-memory hit, so reference identity is layer-dependent and is not part of the API contract; never rely on a specific layer cloning a value before mutation. - -If a caller needs a mutable value, copy it explicitly before changing it: - -```ts -const sharedUser = await getUser("123"); -const editableUser = structuredClone(sharedUser); -editableUser.displayName = "New name"; -``` - -Use a narrower copy when its semantics are sufficient; the ownership boundary is the caller's responsibility. - -## Targeted invalidation and watermarks - -Mutable Redis-backed use cases can opt into targeted invalidation by setting `trackForInvalidation: true` in the options and calling `dialcache.invalidateRemote(keyType, id, futureBufferMs)` after writes. `invalidateRemote` requires `DialCacheConfig.redis`; local-only caching remains supported, but this explicit remote maintenance operation rejects when Redis is absent. The buffer is an application-owned safety value; DialCache cannot choose a universally safe nonzero value: - -```ts -import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; -import { createNodeRedisDialCacheClient } from "dialcache/node-redis"; - -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: createNodeRedisDialCacheClient(redisClient) }, -}); - -// Chosen from this application's clock-skew bound and measured worst-case source/fallback timings. -const USER_INVALIDATION_BUFFER_MS = 5_000; - -const getUser = dialcache.cached( - (userId: string) => db.fetchUser(userId), - { - keyType: "user_id", - useCase: "GetMutableUser", - cacheKey: (userId) => userId, - trackForInvalidation: true, - // Strongly invalidated mutable data should disable request-local and process-local caching. - defaultConfig: new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.REMOTE]: 300 }, - ramp: { [CacheLayer.REMOTE]: 100 }, - }), - }, +// Start with the local 10% ramp cohort; keep the remote layer off. +runtimePolicies.set( + "GetUser", + new DialCacheKeyConfig({ + ramp: { + [CacheLayer.LOCAL]: 10, + [CacheLayer.REMOTE]: 0, + }, + shadow: { ramp: 0 }, + }), ); -await updateUser("123", patch); -await dialcache.invalidateRemote("user_id", "123", USER_INVALIDATION_BUFFER_MS); -``` - -Invalidation writes a Redis watermark at `{encodedNamespace:encodedKeyType:encodedId}#watermark`. Tracked Redis cache entries use the same Redis Cluster hash tag, for example `{users-api:user_id:123}?locale=en#GetMutableUser:dialcache-frame-v1`, so the value key and watermark key live in the same slot. Key components are percent-encoded before joining so delimiters inside IDs or args cannot collide with delimiters in the key format. Components may not contain `{` or `}` because those characters would corrupt the hash tag. - -The internal `:dialcache-frame-v1` suffix identifies values written with DialCache's binary protocol. Watermarks are stored as decimal timestamps. - -A cached Redis value whose Redis-created timestamp is older than or equal to the watermark is treated as stale and refreshed through fallback. `invalidateRemote(keyType, id, futureBufferMs)` sets the watermark to the greater of its existing value and Redis's current time plus the buffer. While that future window is active, an invocation that reaches the tracked Redis read treats the covered value as a miss. If its fallback then reaches the tracked Redis write, Redis rejects the write and DialCache also suppresses the corresponding process-local population; the fallback value still returns to its caller. Request-local memoization remains unconditional. A ramped-out invocation without shadow work does not consult the watermark; a selected shadow path does consult it for `C0` and any clean-miss fill, although caller-path request-local/process-local publication remains independent. - -The bundled timestamp protocol assumes that system clocks are synchronized across every Redis node eligible for primary promotion. Redis does not guarantee that `TIME` is monotonic across nodes, and DialCache does not detect or compensate for cross-node clock skew. If this deployment assumption is violated, failover can temporarily suppress tracked cache fills or allow a pre-invalidation value to remain readable until it expires or a later invalidation advances the watermark past its timestamp. - -Watermarks are invalidation state, not disposable cache entries. The Redis deployment must preserve them for their derived TTL: use `noeviction` or an equivalent guarantee for deployments that rely on the publication fence, and choose persistence and failover guarantees appropriate to the application's consistency requirements. A missing watermark makes tracked reads miss, but a later tracked write cannot distinguish an empty cache from lost invalidation history; it creates a new baseline watermark and can publish fallback data that the lost future watermark would have rejected. Redis replication is asynchronous by default, and DialCache does not issue `WAIT` or provide strong consistency across failover. - -Tracked writes create a baseline watermark and extend its TTL to at least the value TTL plus one minute. Neither tracked writes nor invalidation shorten a longer or persistent watermark TTL; invalidation extends it to at least the remaining future-buffer window plus one minute. There is no fixed watermark retention floor, and reads do not extend watermark lifetime. - -`futureBufferMs` must be a nonnegative safe integer no greater than 31,536,000,000 (a fixed 365-day duration). The default is zero, but zero provides no stale-publication protection once Redis time advances. Every production invalidation should pass a named, application-owned nonzero value based on that application's measured or conservatively bounded timings; there is no universally safe library value. - -Size the buffer to cover the maximum expected negative clock skew between promotion-eligible Redis nodes plus the complete interval in which stale data could still reach the Redis write: source visibility or replication lag, the full remaining tail of any fallback that may already have observed the pre-mutation value, `serializer.dump`, Redis client queue and network latency, Lua script execution, the write itself, and a safety margin. Invalidate only after the source mutation commits. Underestimating this interval can allow a delayed stale fallback to repopulate Redis after the watermark window ends. Overestimating it lengthens the tracked Redis miss/write-suppression window described above, increasing fallback load without publishing stale values. A larger buffer does not delay or suppress returning fallback values to callers. - -This is a timing contract rather than a cancellation or acquisition fence: the buffer prevents stale fallback results from passing that tracked Redis write only while the configured window remains active, and it does not force a fallback to read from an authoritative source. - -Targeted invalidation is remote-only and enforced by Redis watermarks. `invalidateRemote` does not evict existing request-local or process-local entries. Strongly invalidated mutable data should disable request-local and process-local caching (or use a very short process-local TTL only when stale reads are acceptable). - -## Request coalescing - -DialCache coalesces in-flight work at the lifetime of the first active cache layer: - -- When request-local caching is enabled, same-key callers in one outermost `enable()` scope share request-scoped in-flight work before the request-local lookup. Its resolved value is then memoized for later sequential calls in that scope. -- When process-local or Redis caching is enabled, same-key callers share in-flight work within one `DialCache` instance before the first active shared layer. This is reported as `scope="process"`, still applies when request-local caching is off, and can combine leaders from separate request scopes using the same instance. - -```ts -await dialcache.enable(async () => { - // Same cold key, concurrent calls: one fallback execution, one shared result. - const [a, b] = await Promise.all([getUser("456"), getUser("456")]); -}); -``` - -With Redis configured, an instance-scoped leader that misses the process-local cache runs one bounded Redis read and, on a normal miss, the fallback/cache write; followers share its remaining read budget and await the same result. On a shadow-selected served Redis hit, only that leader can schedule detached validation, so followers do not multiply source reads. Process-local-only misses share the leader's fallback/cache write. This protects Redis and the source of truth from a thundering herd on hot keys. - -Coalescing only applies when at least one cache layer is active. Calls outside `enable()` are true pass-through. Calls where request-local, process-local, and Redis serving are all disabled are uncached and uncoalesced, even if shadowing independently schedules detached Redis work; same-key shadow deduplication drops duplicate jobs but does not combine caller fallbacks. Because these calls were initially enabled, the fallback deadline below still applies. - -Because coalescing is keyed by the selected or direct key, concurrent calls with the same key share the leader's execution. Any function argument or captured value omitted from the key must be safe to share this way; include inputs such as locale, auth context, or cancellation behavior when they can change the returned value or whether the underlying loader should run separately. - -### Fallback deadlines - -Once an initially enabled invocation starts its fallback, DialCache applies a 60-second monotonic deadline by default. Set `fallbackTimeoutMs` once on a cached wrapper or on each `getOrLoad()` invocation to choose a positive integer deadline in milliseconds, up to 2,147,483,647, or set it to `null` to preserve an intentionally unbounded fallback: - -```ts -import { FallbackTimeoutError } from "dialcache"; - -const getUser = dialcache.cached( - (userId: string) => db.fetchUser(userId), - { - keyType: "user_id", - useCase: "GetUserWithDeadline", - cacheKey: (userId) => userId, - defaultConfig: DialCacheKeyConfig.enabled(60), - fallbackTimeoutMs: 2_000, - }, +// Later, ramp the process-local and remote layers to 100%. +runtimePolicies.set( + "GetUser", + new DialCacheKeyConfig({ + ramp: { + [CacheLayer.LOCAL]: 100, + [CacheLayer.REMOTE]: 100, + }, + shadow: { ramp: 0 }, + }), ); -try { - await dialcache.enable(() => getUser("123")); -} catch (error) { - if (error instanceof FallbackTimeoutError) { - logger.warn("source lookup exceeded its DialCache budget", { - useCase: error.useCase, - timeoutMs: error.timeoutMs, - }); - } -} +// Reverse the rollout without changing getUser. +runtimePolicies.set("GetUser", DialCacheKeyConfig.disabled()); ``` -The timer starts only when the fallback begins, including after a remote-read deadline has elapsed. Same-key followers share the process or request-local leader's remaining budget and receive its `FallbackTimeoutError`; pass-through invocations where every layer is disabled have independent timers. Cache hits create no fallback timer. Calls that were initially outside an enabled context remain true pass-through and are not timed out, even when the operation configures `fallbackTimeoutMs`. +In this example, the zero-ramp `defaultConfig` is the safety net: if the +provider has no matching entry, both shared serving layers and shadow work +remain off. -Deadline delivery requires the JavaScript event loop to make progress. It cannot preempt a synchronous fallback prefix or other event-loop blocking, so rejection can arrive later than the configured duration; when control returns, DialCache checks the monotonic deadline before accepting the result. The deadline timer remains referenced until the fallback settles or times out. Consequently, an abandoned enabled fallback can keep an otherwise idle short-lived process alive until that deadline; shutdown code should drain outstanding DialCache work rather than discarding its promises. +In production, the provider can read from an application-owned dynamic config +client instead of an in-memory map. DialCache resolves one policy snapshot per +enabled `cached()` or `getOrLoad()` invocation. -Timing out rejects the DialCache chain and clears its flight normally. A later fallback resolution is ignored, so that timed-out invocation cannot become the accepted `S` for a shadow fill or proceed to ordinary serializer, Redis, or local-cache publication. The underlying function is not canceled and may continue its own I/O or side effects; give the source operation its own native timeout or `AbortSignal` whenever possible. `fallbackTimeoutMs: null` disables this guard and makes finite fallback settlement entirely application-owned. Use the `null` escape hatch only after intentionally accepting that liveness risk. It does not create an unbounded detached shadow operation: [shadow validation](#shadow-validation) still uses a 60-second whole-job budget. +The process-local and remote layers each need an effective TTL. With a TTL but +no ramp, a layer defaults to `100`; a ramp of `0` disables it, `100` selects +every key, and an intermediate value selects a stable key cohort. -Timeout failures retain the bounded metrics classification `error="fallback"` with `in_fallback="true"`; the typed error provides the timeout details without adding high-cardinality labels. +Ramps select key cohorts, not requests or load, so `10` does not guarantee 10% +of calls. Increasing or decreasing a ramp preserves membership for keys that +remain inside the threshold, and local and remote cohorts are layer-specific. +DialCache keeps the assignment stable across releases. -### Coalescing state +Ramping down, including with `DialCacheKeyConfig.disabled()`, bypasses existing +entries rather than deleting them; a later ramp-up can reuse entries that +remain valid. -`getCoalescingState()` returns a detached, point-in-time snapshot of process-scoped flights owned by that `DialCache` instance: +Request-local caching is controlled separately by the `requestLocal` boolean. +`DialCacheKeyConfig.disabled()` sets it to `false`, sets `shadow.ramp` to `0` +and `shadow.logMismatches` to `false`, and ramps both the process-local and +remote layers to `0`. -```ts -const state = dialcache.getCoalescingState(); +A remote serving ramp of `0` alone does not override an inherited nonzero +`shadow.ramp`; set both ramps to `0` to stop new invocation-driven Redis reads +and fills. -state.process.activeLeaders; -state.process.activeFollowers; -state.process.oldestLeaderAgeMs; // null when idle -``` +See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) +for sparse-overlay precedence, provider failure behavior, externally +coordinated cohorts, remote-read deadlines, and layer validation. -A leader is one exact cache key currently tracked by the instance-scoped coalescer. A follower is each later invocation that joined that pending leader; the initiating invocation is not counted as a follower. Followers remain counted until their leader settles because abandoning a JavaScript promise is not observable. Request-local flights are deliberately excluded because their lifecycle is bounded by the outer `enable()` scope. `oldestLeaderAgeMs` uses a monotonic clock and is computed when the snapshot is requested. +## Validate Redis before serving it -There is no library-wide flight cap or age-based replacement. A registry cap would bound only DialCache metadata while overflow or eviction could still create unbounded source work and unsafe duplicate publication. Finite operation deadlines provide eventual cleanup; application admission control and backpressure remain responsible for bounding simultaneous distinct-key work. Monitor leader count and oldest age to verify that those budgets hold in production. +For invalidation-tracked use cases, shadow mode can exercise Redis before Redis +is allowed to serve callers. On a selected tracked Redis hit, DialCache returns +the cached value first, then compares a fresh decoding of the retained payload +with a detached source read. -## Metrics +When the remote serving ramp excludes a selected key, shadow work reuses the +caller's source result to inspect Redis and can fill a clean miss. -Metrics are disabled unless a `DialCacheMetricsAdapter` is passed to the constructor. `new DialCache()` does not import a metrics backend, register collectors, or emit metrics. +Shadow work never supplies, delays, or rejects the caller. It is separately +sampled by `shadow.ramp`, bounded per instance by `shadowMaxInFlight`, and +disabled unless the metrics adapter implements the shadow outcome hook. -### Prometheus +Shadow validation can add source and Redis work, remains best-effort during +shutdown, and requires tracked keys plus a valid remote TTL. -Install `prom-client` separately, create the registry your application owns, and pass the explicit Prometheus adapter to DialCache: +Confirmed mismatch warnings are separately opt-in through +`shadow.logMismatches`. They can include logical cache keys and JSON-serialized +values: the fields are capped, not redacted, so enable them only for trusted, +bounded values under an approved logging and data-handling policy. -```bash -pnpm add prom-client@^15.1.3 -``` +See [Shadow validation and Redis bootstrap](https://github.com/lan17/DialCache/blob/main/docs/shadow-validation.md) +for eligibility, rollout design, comparison semantics, command amplification, +deadlines, metrics, invalidation, and lifecycle requirements. -```ts -import { Registry } from "prom-client"; -import { DialCache } from "dialcache"; -import { createPrometheusDialCacheMetrics } from "dialcache/prometheus"; +## How the read path works -const registry = new Registry(); -const dialcache = new DialCache({ - namespace: "users-api", - metrics: createPrometheusDialCacheMetrics({ - registry, - prefix: "myapp_", // myapp_dialcache_request_counter, etc. - }), -}); +Inside an enabled scope, active layers are checked in order: -app.get("/metrics", async (_req, res) => { - res.type(registry.contentType).send(await registry.metrics()); -}); +```text +request-local -> process-local LRU -> Redis or Valkey -> source loader ``` -The adapter requires a caller-owned `Registry`; it never uses the global default registry and does not clear or otherwise own the registry lifecycle. Multiple adapters with the same registry and prefix reuse existing collectors when their type, help, labels, histogram buckets, and exemplar mode match. Adapter construction fails before registering anything if a same-name collector has an incompatible schema; use a unique prefix or a separate registry to resolve the collision. - -The Prometheus adapter emits: - -| Metric | Type | Labels | Description | -| --- | --- | --- | --- | -| `dialcache_request_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | -| `dialcache_miss_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | -| `dialcache_disabled_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips (`context`, `policy_disabled`, `invalid_ttl`, `invalid_ramp`, `ramped_down`, `config_error`) | -| `dialcache_error_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache/fallback errors classified by a bounded failure site | -| `dialcache_invalidation_counter` | Counter | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | -| `dialcache_coalesced_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests split by `request_local` or `process` scope | -| `dialcache_shadow_validation_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `outcome` | Sampled Redis shadow-job outcomes | -| `dialcache_get_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | -| `dialcache_fallback_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the underlying function settles or timeout rejection is delivered | -| `dialcache_serialization_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency | -| `dialcache_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | - -`policy_disabled` means that a process-local or Redis layer has no effective TTL after runtime overlays are applied. It is an intentional policy outcome, including the default when `defaultConfig` is omitted, rather than a configuration-loading failure. - -Every metric carries `cache_namespace`, including disabled-context, key-construction, coalescing, shadow-validation, and invalidation paths that do not have a constructed key. Its value is `DialCacheConfig.namespace`, defaulting to `urn`. The `layer` label is `request_local`, `local` (process-local), `remote` (caller-serving Redis), or `remote_shadow` (detached, non-serving Redis work); `noop` means no cache layer was reached. Detached reads, serializer work, payload sizes, and Redis read/write errors use `remote_shadow`, while the dedicated bounded shadow `outcome` records the terminal job result. The bounded `scope` label on `dialcache_coalesced_counter` distinguishes request-local from instance-scoped single-flight work. `scope="process"` coordinates calls only within one `DialCache` instance; separate instances in the same process do not share in-flight state. - -### Datadog - -Install `hot-shots` separately, create the DogStatsD client your application owns, and pass it to the Datadog adapter: - -```bash -pnpm add hot-shots@^17.0.0 -``` +The wrapped function or inline loader is the fallback and remains the source of +the returned value when every active layer misses or a cache operation fails +open. + +- A request-local hit returns the value memoized in the current outermost + `enable()` scope. +- A process-local hit returns from the `DialCache` instance's bounded LRU. +- A process-local miss can read Redis and populate the process-local cache. +- A remote miss runs the fallback and attempts to populate the active cache + layers. +- Selected tracked keys can schedule detached shadow work after a Redis serving + hit or when the Redis serving ramp excludes the key. Shadow work never serves + the caller; it can validate a hit or fill a clean miss. +- A caller-path remote read failure or timeout runs the fallback without a + second caller-path Redis operation. Tracked invalidation adds a stricter + [publication rule](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md#read-and-write-behavior). +- Same-key concurrent work is coalesced within the scope of the first active + layer. + +When all serving layers are disabled by policy, an initially enabled call +remains uncached and uncoalesced even if selected shadow work runs +independently; its fallback deadline still applies. A call that started outside +an enabled scope remains a true pass-through and does not get a DialCache +deadline or shadow work. + +## Core concepts + +### Cache operations and keys + +`cached(fn, options)` defines both a callable and the value-identity contract: + +| Option | Required | Purpose | +| --- | --- | --- | +| `keyType` | yes | Names the kind of id and, with `id`, the invalidation unit for tracked Redis entries. | +| `useCase` | yes | Identifies this individual cache in stored keys and metrics. | +| `cacheKey` | yes | Selects the bare id or `{ id, args }` from the function parameters. | +| `defaultConfig` | no | Supplies the baseline policy overlaid by runtime config. | +| `serializer` | for statically non-JSON return types | Defines the Redis representation for this operation's value. | +| `shadowComparator` | no | Defines synchronous application-level equality for shadow validation; strict deep equality is the default. | +| `trackForInvalidation` | no | Opts the remote entries into watermark-based targeted invalidation. | +| `fallbackTimeoutMs` | no | Sets the fallback deadline; defaults to `60_000`, and `null` disables it. | + +Use `getOrLoad(load, options)` when a one-shot calculation should remain inline. +It follows the same cache, policy, coalescing, invalidation, serialization, and +deadline contracts, but takes a direct `key` instead of a `cacheKey` selector. +For repeated inline calls that represent the same operation, reuse one stable, +deployment-defined `useCase`. + +The selected or direct key must include every input dimension that can affect +the returned value. Same-key concurrent calls may share the leader's execution, +so ignored function arguments or captured values such as auth context, locale, +or cancellation behavior must truly be safe to share. + +Set a stable, application-specific `namespace` when applications or +environments share Redis: ```ts -import StatsD from "hot-shots"; -import { DialCache } from "dialcache"; -import { createDatadogDialCacheMetrics } from "dialcache/datadog"; - -const dogStatsD = new StatsD({ - host: process.env.DD_AGENT_HOST, - globalTags: { service: "users-api", env: process.env.DD_ENV ?? "development" }, - errorHandler: (error) => logger.warn("DogStatsD error", { error }), -}); - const dialcache = new DialCache({ - namespace: "users-api", // cache identity and cache_namespace tag - metrics: createDatadogDialCacheMetrics({ - client: dogStatsD, - observationMetricType: "distribution", - namespace: "dialcache", // metric-name prefix: dialcache.request.count, etc. - }), + namespace: "production-users-api", + redis: { client: dialCacheRedisClient }, }); - -// After outstanding cache operations finish during application shutdown: -dogStatsD.close(); ``` -`hot-shots` is the supported and tested client, but the adapter depends only on the exported `DatadogDogStatsDClient` structural interface. DialCache does not import or install `hot-shots`, create a client, flush buffers, close sockets, or otherwise own the client lifecycle. - -`observationMetricType` is required. `"distribution"` is recommended when latency and size percentiles must aggregate across hosts; enable the desired distribution percentiles and aggregations in Datadog. Choose `"histogram"` when host-level histogram aggregation matches your existing Datadog setup. The choice applies uniformly to all four duration/size metrics. Both modes produce Datadog custom metrics. Distribution volume scales with unique tag-value combinations: Datadog counts five baseline aggregations per combination, and enabling percentile aggregations adds five more. Review [Datadog's custom-metrics billing guidance](https://docs.datadoghq.com/account_management/billing/custom_metrics/) before rollout. Do not send both types under the same namespace: when changing types, use a new namespace during migration so one metric identity never mixes histogram and distribution points. - -`DatadogMetricsOptions.namespace` is the metric-name namespace and defaults to `dialcache`. It is separate from `DialCacheConfig.namespace`, the logical cache namespace emitted as the `cache_namespace` tag. The Datadog metric namespace must start with a letter and contain only letters, numbers, underscores, and dot-separated non-empty segments. The adapter rejects invalid metric namespaces and final metric names longer than 200 characters rather than relying on client-side normalization. A `hot-shots` `prefix` is applied after the adapter constructs the name, so include that prefix when checking the final length and avoid combining it with the metric namespace accidentally. Client-level `globalTags` are appended by `hot-shots`; the table below lists the tags added by the adapter. - -The Datadog adapter emits exact increments of `1` for counters and preserves seconds and bytes without unit conversion: - -| Metric | Type | Tags | Description | -| --- | --- | --- | --- | -| `dialcache.request.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | -| `dialcache.miss.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | -| `dialcache.disabled.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | -| `dialcache.error.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache/fallback errors by bounded failure site | -| `dialcache.invalidation.count` | Count | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | -| `dialcache.coalesced.count` | Count | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests by sharing scope | -| `dialcache.shadow.count` | Count | `cache_namespace`, `use_case`, `key_type`, `outcome` | Sampled Redis shadow-job outcomes | -| `dialcache.get.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | -| `dialcache.fallback.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the underlying function settles or timeout rejection is delivered | -| `dialcache.serialization.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | -| `dialcache.serialization.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | +See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) +for key encoding, secondary arguments, namespace changes, serializers, and +value ownership. -Observer throws and rejections from returned promises or thenables are isolated by DialCache's fail-open metrics boundary. Buffered transport failures that are not represented by a returned thenable happen outside that boundary, so configure the DogStatsD client's error handling and shutdown behavior as part of application ownership. +### Cache layers -### Error categories - -The `error` label reports where an operation failed rather than copying the thrown value's class or `Error.name`: - -| `error` | Meaning | -| --- | --- | -| `key_construction` | The cache-key selector or `DialCacheKey` construction failed | -| `config_resolution` | Runtime or layer configuration, or ramp resolution, failed | -| `cache_read` | A local-cache or Redis read failed | -| `cache_read_timeout` | A Redis read exceeded its effective remote-read deadline | -| `cache_write` | A local-cache or Redis write failed | -| `serialization_load` | Deserializing a Redis payload failed | -| `serialization_dump` | Serializing a value for Redis failed | -| `invalidation` | Writing an invalidation watermark failed | -| `fallback` | The wrapped application function failed or exceeded its DialCache deadline | -| `unknown` | Reserved for an otherwise unclassified future failure site | - -These values are defined by the backend-neutral core and are identical for every metrics adapter. Raw thrown values, error names, messages, timeout values, cache IDs, arguments, and Redis keys are never included in metric labels. Operational errors are still passed to the configured logger where the existing failure path logs them. `in_fallback` remains the explicit cache-plumbing-versus-application distinction. +| Layer | Scope | Primary use | +| --- | --- | --- | +| Request-local | Outermost `enable()` scope | Memoize repeated reads during one bounded request or job. | +| Process-local | One `DialCache` instance | Serve hot values from a bounded in-process LRU. | +| Redis or Valkey | Shared remote store | Reuse TTL-cached values across processes and hosts. | -### Custom adapters +Each invocation uses one resolved policy snapshot for all three layers. +Request-local storage has no capacity limit, so use it only for short-lived +scopes with bounded key cardinality. Process-local values count toward one +instance-wide entry cap. Remote values use a serializer selected by the cache +operation or the Redis configuration. -For other telemetry backends, implement `DialCacheMetricsAdapter` and pass the adapter through `new DialCache({ metrics })`. Every backend-neutral label object exposes the logical namespace as camel-case `cacheNamespace`; adapters should map it to their backend's `cache_namespace` label/tag. This field is present even when no key or cache layer was reached. Implement the optional `shadowValidation` method to enable shadow work as well as record its outcomes; omitting it leaves all shadow work disabled even when `shadow.ramp` is nonzero or mismatch logging is enabled. Every metrics callback is fire-and-forget: DialCache isolates synchronous throws and consumes rejections from returned promises or thenables, but never awaits or drains observer work. Omit `metrics` to disable metrics. +Shadow validation uses detached Redis work but is not another serving +`CacheLayer`. Its sampling, capacity, deduplication, deadline, and metrics +contracts are independent of request coalescing and the remote serving ramp. -## Maintainers +Cached in-memory values are shared by reference. Treat every returned value as +immutable, or copy it explicitly before mutation. -### Cache-path benchmark +### Targeted invalidation -From a repository checkout, run the semantic microbenchmark after installing dependencies: +Mutable Redis-backed use cases can opt into watermark-based invalidation with +`trackForInvalidation: true`, then call: -```bash -pnpm benchmark:request-local +```ts +await updateUser("123", patch); +await dialcache.invalidateRemote( + "user_id", + "123", + USER_INVALIDATION_BUFFER_MS, +); ``` -The command builds `dist` before reporting ten scenarios: sequential request-local hits, sequential process-local hits, enabled bounded fallbacks, request-local coalescing fan-out, process coalescing fan-out, remote-read-deadline coalescing fan-out, tracked Redis hits with shadow omitted, tracked Redis hits deterministically outside a partial shadow ramp, a ramped-down warm-hit confirmation, and a ramped-down clean-miss fill. Both shadow scenarios prove that the caller completes before detached Redis work. The benchmark is a maintainer tool and is not included in the published package. It asserts fallback counts, Redis behavior, coalescing state, timer cleanup, returned values, exactly-once SoT reuse, and conditional confirmation/fill without applying a timing threshold. Override its work sizes with `DIALCACHE_BENCH_ITERATIONS` and `DIALCACHE_BENCH_FANOUT`. - -### Releasing - -Publishing starts by manually running the `Release` workflow from current `main`. After the package checks pass, Semantic Release selects the next version from Conventional Commits since the highest stable `vX.Y.Z` tag. Breaking changes bump major, `feat` bumps minor, and every other normal PR-title type (`fix`, `perf`, `docs`, `style`, `refactor`, `test`, `build`, `chore`, `ci`, and `revert`) bumps patch. The highest required bump wins. - -The workflow opens a `release: ` PR whose only change is the matching `package.json` version. `release` is a reserved Conventional Commit type configured not to request another release, so the version-control commit does not cause an extra bump. GitHub marks workflow runs for a PR opened with `GITHUB_TOKEN` as approval-required; approve those runs, review the PR, and squash-merge it normally through the protected branch. - -The merge triggers the publish job. Before any release side effect, it verifies current `main`, the release commit subject, the one-file diff, the package version, the absent tag, and Semantic Release's independently calculated version and commit. It then reruns the package checks and asks Semantic Release to create the matching Git tag, publish the public npm package with provenance, and publish the GitHub release. - -The repository must enable **Allow GitHub Actions to create and approve pull requests** under Actions workflow permissions. This workflow uses that capability only to create the version PR; it never approves or merges one, and no ruleset bypass actor or persistent release credential is required. +`invalidateRemote()` is an explicit remote maintenance operation and requires +this `DialCache` instance to have a Redis or Valkey client. Without one, it +rejects instead of reporting a no-op as successful. + +Invalidation is deliberately remote-only. It does not evict existing +request-local or process-local values, so strongly invalidated mutable data +should disable those layers or tolerate their TTL-bounded staleness. + +The buffer must be a named, application-owned nonzero value no greater than +365 days, sized for clock skew and the full stale-work window. See +[Targeted invalidation](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md) +before enabling it in production. + +### Request coalescing and fallback deadlines + +Concurrent callers with the same cache key share active work within the first +active cache scope: one outer request for request-local caching, or one +`DialCache` instance when process-local or remote caching is active. This +mitigates hot-key stampedes inside that scope; it is not cross-process +coordination. + +Same-key followers share the leader's remaining remote-read budget. The +fallback deadline starts separately only if and when the source loader begins. + +Enabled fallbacks have a 60-second monotonic deadline by default. Timing out +rejects the DialCache chain and prevents the late result from being published, +but it does not cancel the underlying function. Give source operations their +own native timeout or `AbortSignal`. + +See [Coalescing and fallback liveness](https://github.com/lan17/DialCache/blob/main/docs/coalescing.md) +for exact sharing, deadline, cleanup, and admission-control contracts. + +### Observability + +Metrics are disabled unless a `DialCacheMetricsAdapter` is supplied. First-party +adapters support caller-owned Prometheus registries and Datadog DogStatsD +clients. Their fixed schemas report layer requests, misses, disabled reasons, +coalescing scopes, serialization work, shadow outcomes, and cache versus +fallback failures. + +Keep application-owned namespaces, use-case names, and key types stable and +low-cardinality. Optional confirmed-mismatch warnings are value-bearing logs, +not metrics, and require separate data-handling review. + +See [Observability](https://github.com/lan17/DialCache/blob/main/docs/observability.md) +for installation, collector schemas, metric names, and custom adapters. + +## Production checklist + +Before ramping a use case: + +- enable DialCache only around read paths, and keep mutation paths inside + `disable()` or outside the enabled boundary; +- verify that every selected or direct key includes each value and execution + dimension that is unsafe to share; +- begin at `0` or a small deterministic key cohort, monitor source load, cache + errors, hit rate, latency, remote-read and fallback timeouts, and coalescing + state, then increase in controlled steps; +- keep a runtime path to `DialCacheKeyConfig.disabled()`; +- choose an effective DialCache remote-read deadline, and configure + resource-native budgets for the underlying Redis work, config providers, + serializers, and source operation; +- use a conservative `localMaxSize` and bounded request-local scopes; +- treat cached values as immutable; +- verify serializer compatibility across mixed application versions; +- before enabling shadow mode, confirm the loader is safe for an extra + observational read, preserve immutable inputs and results, bound concurrency, + and monitor added load and outcomes; +- before enabling shadow mismatch logging, approve how logical keys and + serialized values are redacted, transported, accessed, and retained; +- plan shutdown around detached shadow work and application-owned dependencies; and +- for tracked invalidation, use synchronized Redis clocks, durable non-evictable + watermarks, and an application-sized nonzero buffer. + +## Reference guides + +- [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) — definitions, keys, + runtime overlays, request-local and process-local behavior, and value + ownership. +- [Redis and Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) — node-redis and GLIDE setup, lifecycle, + liveness, binary protocol, and serialization. +- [Shadow validation and Redis bootstrap](https://github.com/lan17/DialCache/blob/main/docs/shadow-validation.md) — + non-serving rollout, eligibility, comparison, clean-miss filling, capacity, + deadlines, metrics, mismatch diagnostics, and lifecycle. +- [Targeted invalidation](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md) — watermarks, Redis Cluster + placement, clock assumptions, and buffer sizing. +- [Coalescing and fallback liveness](https://github.com/lan17/DialCache/blob/main/docs/coalescing.md) — sharing scopes, + deadlines, state inspection, cleanup, and backpressure. +- [Observability](https://github.com/lan17/DialCache/blob/main/docs/observability.md) — Prometheus, Datadog, metric schemas, + error categories, and custom adapters. +- [Maintainer guide](https://github.com/lan17/DialCache/blob/main/docs/maintainers.md) — benchmarks and the protected release + workflow. + +DialCache is licensed under the +[MIT License](https://github.com/lan17/DialCache/blob/main/LICENSE). diff --git a/docs/coalescing.md b/docs/coalescing.md new file mode 100644 index 0000000..5f6c133 --- /dev/null +++ b/docs/coalescing.md @@ -0,0 +1,271 @@ +# Coalescing and fallback liveness + +[Back to the README](../README.md) + +DialCache shares same-key in-flight work within the lifetime of the first active +cache layer. It applies a finite deadline to each active remote read and a +separate default deadline once an initially enabled invocation begins its +fallback loader. + +These mechanisms reduce duplicate source work. Their deadlines help flights +settle, but eventual cleanup still requires finite application-owned budgets +for every injected operation. They do not replace cross-process coordination, +source-native cancellation, admission control, or backpressure. + +Detached [shadow work](shadow-validation.md) has a separate instance-level +registry and capacity limit. It is not another coalescing scope. + +## Request coalescing + +DialCache has two sharing scopes. + +### Request-local scope + +When request-local caching is active, callers with the same key in one outermost +`enable()` scope share in-flight work before the request-local lookup. + +The resolved value is memoized for later sequential calls in that scope. A +different outer request has a different request-local flight registry. + +### Process scope + +When process-local or remote caching is active, same-key callers share work +within one `DialCache` instance before the first active process-local or remote +layer. + +This is reported as `scope="process"`, but it is instance-scoped: + +- separate requests using the same `DialCache` instance can share; +- separate `DialCache` instances in one process do not share; and +- separate processes or hosts do not share. + +```ts +await dialcache.enable(async () => { + // Same cold key and active process-local or remote layer: + // one fallback execution, one shared result. + const [first, second] = await Promise.all([ + getUser("456"), + getUser("456"), + ]); +}); +``` + +With a remote layer configured, an instance-scoped leader that misses +process-local cache performs one bounded Redis read. Followers share that read +and its remaining deadline. On a remote miss, the leader runs the fallback and +cache write; followers await that result. + +For a process-local-only miss, followers share the leader's fallback and local +write. This mitigates a thundering herd on one hot key within the instance. + +## When calls do not coalesce + +Coalescing applies only when at least one cache layer is active: + +- calls that start outside `enable()` are true pass-through; +- initially enabled calls with every layer disabled are uncached and + uncoalesced; and +- process-scoped work is never shared across `DialCache` instances. + +An initially enabled all-disabled call still receives the fallback deadline +described below. + +Because coalescing is keyed by the full constructed cache key, concurrent calls +with the same identity share the leader's execution. Every function argument +or captured value omitted from the selected or direct key must be safe to share +this way. + +Include locale, auth context, cancellation behavior, or any other input in the +key when it can change: + +- the returned value; +- whether the underlying function should run independently; or +- whether two callers may safely share one result. + +### Shadow work does not enable caller coalescing + +Shadow admission does not make an otherwise all-disabled caller path +coalesced. + +When the remote layer is the only configured serving layer and its ramp +excludes a key, concurrent calls each run their own source fallback. Same-key +shadow jobs are deduplicated by admitting one and reporting the others as +`dropped`; callers do not join or await that job. + +A serving Redis hit reached through a process-scoped leader schedules at most +one shadow job for its coalesced followers. `shadowMaxInFlight` limits scheduled +or running shadow jobs across the instance, independently of request-local and +process-scoped flights. See +[Shadow validation and Redis bootstrap](shadow-validation.md) for the full +admission and lifecycle contract. + +## Fallback deadlines + +Once an initially enabled invocation begins its wrapped fallback, DialCache +applies a 60-second monotonic deadline by default. + +Set `fallbackTimeoutMs` on a cached wrapper or `getOrLoad()` invocation to +choose a positive integer deadline in milliseconds, up to 2,147,483,647. Set +it to `null` only when the application intentionally accepts an unbounded +fallback: + +```ts +import { FallbackTimeoutError } from "dialcache"; + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUserWithDeadline", + cacheKey: (userId) => userId, + defaultConfig: DialCacheKeyConfig.enabled(60), + fallbackTimeoutMs: 2_000, + }, +); + +try { + await dialcache.enable(() => getUser("123")); +} catch (error) { + if (error instanceof FallbackTimeoutError) { + logger.warn("source lookup exceeded its DialCache budget", { + useCase: error.useCase, + timeoutMs: error.timeoutMs, + }); + } + throw error; +} +``` + +### When the timer runs + +The timer starts only when the fallback begins: + +- same-key followers share the request-local or process leader's remaining + budget and receive its `FallbackTimeoutError`; +- a remote read failure or timeout starts the fallback timer only when the + source loader begins; +- enabled pass-through invocations where every layer is disabled have + independent timers; +- cache hits create no fallback timer; and +- calls that began outside an enabled context remain true pass-through and are + not timed out, even when the operation has `fallbackTimeoutMs`. + +The fallback deadline does not cover work that happens before fallback. An +active remote read has its own resolved +[remote-read deadline](redis.md#remote-read-deadlines-and-async-liveness), while +a pending config provider or serializer load does not. Serialization and a +Redis write after fallback also remain outside it. Give every injected +operation its own finite, resource-native budget. + +### Event-loop behavior + +Deadline delivery requires the JavaScript event loop to make progress. It +cannot preempt a synchronous fallback prefix or other event-loop blocking. +Rejection can therefore arrive later than the configured duration. + +When control returns, DialCache checks the monotonic deadline before accepting +the result. The timer remains referenced until the fallback settles or times +out. An abandoned enabled fallback can keep an otherwise idle short-lived +process alive until the deadline. + +Shutdown code should await outstanding caller-path DialCache promises rather +than discard them. This does not drain detached shadow jobs; see +[Redis lifecycle ownership](redis.md#lifecycle-ownership) for dependency +shutdown requirements. + +### Timeout does not cancel the source + +Timing out: + +1. rejects the DialCache chain; +2. clears its tracked flight normally; +3. ignores a later fallback resolution; and +4. prevents that invocation from proceeding to serializer, Redis, or local + publication. + +The underlying loader is not canceled and may continue its own I/O or side +effects. Give the source operation a native timeout or `AbortSignal` whenever +possible. + +`fallbackTimeoutMs: null` disables the guard and makes finite fallback +settlement entirely application-owned. Use that escape hatch only after +intentionally accepting the liveness risk. + +Timeout failures retain the bounded metrics classification +`error="fallback"` with `in_fallback="true"`. The typed error carries timeout +details without adding high-cardinality labels. + +A shared remote-read timeout emits one `cache_read_timeout` error for the +leader, not one per follower. + +### Shadow deadlines are separate + +A finite `fallbackTimeoutMs` also supplies the whole-job deadline for detached +shadow work. Setting it to `null` removes the caller fallback deadline, but +shadow work still uses the 60-second default. + +For a served Redis hit, the shadow clock starts when detached validation +begins. For a remote-ramped-down call, it starts before the caller's source +operation, so synchronous source work consumes the same budget. Shadow work +never delays or rejects the caller. + +The shadow scheduler and deadline timer are unreferenced. A deadline prevents +later serialization or write dispatch, but cannot cancel an already-started +source call, serializer, raw Redis read, or dispatched Redis write. + +Underlying shadow-owned work that has already started can retain a capacity +slot until it settles, even after the bounded outcome is reported. A +caller-owned source promise reused by a ramped-down shadow path is the +exception: by itself, it stops retaining that slot at the shadow deadline. + +## Inspecting process-scoped flights + +`getCoalescingState()` returns a point-in-time copy of caller-path +process-scoped flights owned by one `DialCache` instance: + +```ts +const state = dialcache.getCoalescingState(); + +state.process.activeLeaders; +state.process.activeFollowers; +state.process.oldestLeaderAgeMs; // null when idle +``` + +A leader is one exact cache key currently tracked by the instance-scoped +coalescer. A follower is each later invocation that joined that pending leader; +the initiating invocation is not counted as a follower. + +Followers remain counted until their leader's DialCache promise settles, +including by deadline rejection. The underlying source operation may continue +after that point. + +Request-local flights are deliberately excluded because their lifecycle is +bounded by the outer `enable()` scope. Shadow jobs are also excluded; they use +their own capacity registry and outcome metrics. +`oldestLeaderAgeMs` uses a monotonic clock and is computed when the snapshot is +requested. + +## Admission control remains application-owned + +There is no library-wide cap or age-based replacement for caller-path +request-local or process-scoped flights. `shadowMaxInFlight` bounds only +detached shadow jobs. + +A registry cap would bound only DialCache metadata. Overflow or eviction could +still create unbounded source work and unsafe duplicate publication. +DialCache's remote-read and fallback deadlines cover only those phases; +provider, serializer, and Redis-write settlement remains application-owned. +Admission control and backpressure remain responsible for bounding +simultaneous distinct-key work. + +Monitor: + +- active leader count; +- active follower count; +- oldest leader age; +- remote-read timeout errors; +- fallback deadline errors; and +- source concurrency and saturation. + +Use those signals to verify that application budgets and admission control hold +under production load. diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..97b4eea --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,580 @@ +# Configuration and cache layers + +[Back to the README](../README.md) + +This guide covers reusable cached functions, one-shot inline loaders, cache +identity, runtime policy, request-local and process-local behavior, and +cached-value ownership. For the shared remote layer, see +[Redis and Valkey](redis.md). + +## Defining cache operations + +### Reusable cached functions + +`cached(fn, options)` wraps a function; the wrapped callable has the same +parameters and always returns a `Promise`. + +| Option | Required | Description | +| --- | --- | --- | +| `keyType` | yes | The kind of id the key addresses, such as `"user_id"`. Together with the id, this is the invalidation unit for tracked entries. | +| `useCase` | yes | Identifies the individual cache. It is part of the stored key and a metrics label. | +| `cacheKey` | yes | Selects a bare id or `{ id, args }` from `fn`'s parameters. | +| `defaultConfig` | no | Provides the `DialCacheKeyConfig` baseline that runtime config overlays field by field. | +| `serializer` | when the return type is not statically JSON-compatible | Selects a per-function `Serializer` for Redis values; see [Serialization](redis.md#serialization). | +| `shadowComparator` | no | Defines synchronous application-level equality for [shadow validation](shadow-validation.md); Node strict deep equality is the default. | +| `trackForInvalidation` | no; default `false` | Opts this use case's Redis entries into watermark-based [targeted invalidation](invalidation.md). | +| `fallbackTimeoutMs` | no; default `60_000` | Sets the fallback deadline in milliseconds, up to 2,147,483,647. `null` disables it; see [Fallback deadlines](coalescing.md#fallback-deadlines). | + +`useCase` is validated when the function is registered. A duplicate within one +`DialCache` instance throws `UseCaseIsAlreadyRegisteredError`, and the internal +name `watermark` throws `UseCaseNameIsReservedError`. + +### One-shot inline loaders + +`getOrLoad(load, options)` runs one zero-argument synchronous or asynchronous +loader through the same cache layers, runtime policy, coalescing, invalidation, +metrics, serialization, and deadline behavior as `cached()`. Cache-plumbing +failures fall through to the loader; loader failures still reject and clear +their tracked flight: + +```ts +const profile = await dialcache.enable(() => + dialcache.getOrLoad( + async () => { + const user = await db.getUser(userId); + return renderProfile(user, locale); + }, + { + keyType: "user_id", + useCase: "BuildProfile", + key: { id: userId, args: { locale } }, + defaultConfig: DialCacheKeyConfig.enabled(60), + }, + ), +); +``` + +The options match `cached()` except that the direct `key` replaces the +`cacheKey` selector. `defaultConfig` and `fallbackTimeoutMs` are validated and +snapshotted for each invocation. Outside an enabled scope, DialCache calls +`load` directly without constructing a key or resolving runtime policy. + +`getOrLoad()` does not register its `useCase` or detect duplicates, but it still +rejects the reserved internal name `"watermark"`. + +Repeated calls should reuse one stable, deployment-defined name such as +`"BuildProfile"`. Never derive it from a user, request, id, or other +high-cardinality input because it is part of both cache identity and metrics +labels. Put those values in `key` instead. + +Every captured value that can change the result belongs in the bare id or +`{ id, args }` key. Concurrent same-key calls may share one caller's in-flight +loader and cached value, so all call sites for that identity must also agree on +value meaning and serialization. + +Shadow work can run the loader later, after the caller has continued. Snapshot +mutable arguments or captured state before invoking the operation so that the +detached source read still represents the selected key. See +[Shadow validation and Redis bootstrap](shadow-validation.md). + +Prefer `cached()` for reusable loaders and `getOrLoad()` for calculations +intentionally local to one call site. + +## Enable and disable scopes + +DialCache performs cache work only inside an enabled asynchronous scope. Most +services create one instance and reuse it for the service process. Each +instance owns one process-local LRU, one process-coalescing registry, and one +shadow deduplication and capacity registry. Create separate instances only +when those resources should be isolated: + +| API | Behavior | +| --- | --- | +| `enable(fn)` | Enables caching for `fn` and the asynchronous work it awaits. The outermost call owns any request-local state. | +| `disable(fn)` | Temporarily restores pass-through behavior, commonly around nested mutation work. It does not evict existing values. | +| `isEnabled()` | Reports whether the current asynchronous call chain is inside a live enabled scope. | +| `withEnabled(fn)` | Exact alias for `enable(fn)`. | +| `withDisabled(fn)` | Exact alias for `disable(fn)`. | + +All five methods are instance-scoped. `enable()` and `disable()` always return a +`Promise`, including when their callback returns synchronously. Nested scopes +restore the previous state when their callbacks settle, and a nested +`enable()` inside `disable()` can opt a smaller read region back in. + +Enabled state follows Node's `AsyncLocalStorage`; it is not a process-global +flag. Once the outermost `enable()` callback settles, detached asynchronous work +that inherited the old context becomes pass-through and cannot repopulate its +closed request-local state. + +The root-exported `DialCacheContext` exposes the lower-level +`enable()`, `disable()`, and `isEnabled()` context primitive. It does not attach +itself to a `DialCache` instance or perform cache work. Most applications should +use the methods on `DialCache`. + +Keep mutation work outside the enabled boundary or inside `disable()`. Because +disabling does not evict existing values, mutable data still needs an +appropriate TTL or [targeted invalidation](invalidation.md) policy. + +## Keys, ids, and extra dimensions + +For `cached()`, the required `cacheKey` selector receives the wrapped +function's inferred parameters. `getOrLoad()` accepts the same bare id or +`{ id, args }` shape directly through `key`: + +```ts +const searchPosts = dialcache.cached( + (userId: string, page: number, filter: string) => + db.searchPosts(userId, page, filter), + { + keyType: "user_id", + useCase: "SearchPosts", + cacheKey: (userId, page, filter) => ({ + id: userId, + args: { page, filter }, + }), + defaultConfig: DialCacheKeyConfig.enabled(60), + }, +); + +await dialcache.enable(() => searchPosts("u1", 2, "active")); +``` + +The selected or direct key is the value-identity contract. It must include +every input dimension that can affect the returned value. Otherwise, distinct +calls can reuse the same cached value or share the same in-flight fallback +through request coalescing. + +### Namespace + +`DialCacheConfig.namespace` is the logical cache namespace and the first +component of every key. It defaults to `"urn"`, producing keys such as +`urn:user_id:123#GetUser`. + +Set a stable application-specific value when applications or environments may +share one Redis deployment: + +```ts +const dialcache = new DialCache({ + namespace: "production-users-api", + redis: { client: dialCacheRedisClient }, +}); +``` + +That produces Redis keys beginning with `production-users-api:...`, or +`{production-users-api:...}` for invalidation-tracked values. `namespace` is +DialCache's single cache-identity and key-partitioning setting. It participates +in request-local, process-local, Redis, coalescing, deterministic ramp, +invalidation, and metrics. + +A namespace may not contain `{` or `}` because DialCache reserves those +characters for Redis Cluster hash tags. + +### Identity rules + +- **`keyType` plus `id` is the invalidation unit for tracked Redis entries.** + `dialcache.invalidateRemote("user_id", "123", futureBufferMs)` writes one + watermark for that user. Any tracked Redis entry with the same `keyType` and + `id` is refreshed across all `args` variants when Redis is read. Untracked + entries do not consult the watermark. Invalidation does not evict existing + request-local or process-local entries. +- **`args` are part of the cache key.** Different arguments produce different + entries, but targeted invalidation is by id rather than by argument. +- **Scalar equality is string-based.** For matching surrounding dimensions: + - numeric `1`, string `"1"`, and bigint `1n` identify the same key; and + - argument values `null` and `"null"` match, `-0` matches `0`, and an + `undefined` argument is omitted. + + If a deployment changes the logical meaning represented by a scalar, change + an explicit identity dimension such as `keyType`, `useCase`, or an argument + name or value. +- **Non-key inputs still reach the loader.** A database handle can be a normal + function parameter ignored by `cacheKey` or a value captured by a + `getOrLoad()` loader. Concurrent same-key misses share the leader's + execution. Do not omit values such as `AbortSignal`, auth context, locale, or + other request-scoped inputs unless sharing one result is correct. +- **Methods need a receiver.** Pass `obj.method.bind(obj)` or + `(...args) => obj.method(...args)`; a bare `obj.method` reference loses + `this`. + +### Changing a namespace + +Changing `namespace` intentionally creates a cold-cache boundary across every +layer. Old and new keyspaces do not share Redis values or invalidation +watermarks. + +During an overlapping deployment, an invalidation handled by one version is +invisible to the other. The other version can continue serving a stale tracked +value until its value TTL expires. If remote invalidation correctness matters, +a normal rolling namespace change is unsafe. + +Use a coordinated no-overlap cutover, or an operational bridge that prevents +both versions from serving remote cache across mutations. For example, +temporarily disable and clear remote caching during the transition. After the +cutover, provision for fallback and refill load, and allow old Redis keys to +expire by TTL. + +## Runtime config and ramp controls + +Instance-wide behavior is set through the `DialCache` constructor: + +| `DialCacheConfig` option | Default | Description | +| --- | --- | --- | +| `namespace` | `"urn"` | Logical cache namespace and first key component. | +| `redis` | none | `{ client, readTimeoutMs?, serializer? }`; enables the [remote layer](redis.md). Remote reads default to a 50 ms deadline. | +| `localMaxSize` | `10_000` | Global process-local entry cap. `0` disables process-local storage. Must be a nonnegative safe integer. | +| `cacheConfigProvider` | none | Resolves runtime config per enabled invocation as a sparse overlay on the operation's `defaultConfig`; `null` applies no overrides. | +| `shadowMaxInFlight` | `1` | Maximum scheduled or running shadow jobs per instance. Must be a positive safe integer. There is no queue; excess jobs are dropped and measured. | +| `metrics` | disabled | A `DialCacheMetricsAdapter`; see [Observability](observability.md). | +| `logger` | `console` | Receives operational cache failures and opted-in confirmed shadow mismatch warnings through `debug`, `warn`, and `error`. | + +Per-invocation policy is a `DialCacheKeyConfig`: per-layer `ttlSec` and `ramp` +maps keyed by `CacheLayer.LOCAL` and `CacheLayer.REMOTE`, a `requestLocal` +boolean, an optional `remoteReadTimeoutMs`, and an optional `shadow` group. +The root-exported `ShadowConfig` type defines that group's independent `ramp` +and default-off `logMismatches` leaves. + +### Baseline and overlay precedence + +Every cached definition or `getOrLoad()` invocation can provide an optional +per-use-case `defaultConfig`. That is the baseline policy. The +`cacheConfigProvider` result is a sparse field-level overlay on it. + +Enablement fields use this precedence: + +```text +runtime field -> defaultConfig field -> DialCache disabled baseline +``` + +The disabled baseline sets `requestLocal` to `false`, leaves the process-local +and remote TTLs unset, and leaves `shadow` absent. + +Either serving layer is disabled by policy when it has no effective TTL. With +an effective TTL but no effective ramp, that layer defaults to a 100% ramp. +Shadow work remains off unless `shadow.ramp` is explicitly greater than zero. + +The remote-read deadline has two additional fallbacks: + +```text +runtime remoteReadTimeoutMs + -> defaultConfig.remoteReadTimeoutMs + -> redis.readTimeoutMs + -> 50 ms +``` + +This value bounds how long DialCache waits for an active Redis or Valkey read. +It can be tuned per use case at runtime, but it cannot be disabled. + +`DialCacheKeyConfig` preserves an omitted `requestLocal` as `undefined`, so the +overlay can distinguish omission from an explicit `false`. Its effective value +still defaults to `false` after resolution. + +A provider result of `null`, or a defensive `undefined`, applies no overrides. +An empty `DialCacheKeyConfig` and omitted runtime fields also inherit the +baseline. + +Overlay merging is sparse at each leaf. The local and remote entries inside +`ttlSec` and `ramp` merge independently, as do `shadow.ramp` and +`shadow.logMismatches`. For example, `shadow: { ramp: 0 }` disables inherited +shadow admission while preserving an inherited logging preference; +`shadow: { logMismatches: false }` suppresses warnings without changing the +inherited shadow cohort. + +Use explicit values to replace inherited policy: + +- `requestLocal: false` disables request-local caching; +- a process-local or remote ramp of `0` disables that serving layer; +- `shadow: { ramp: 0 }` disables new shadow work; and +- `DialCacheKeyConfig.disabled()` turns request-local and shadow work off and + ramps both serving layers to `0`. + +The remote serving and shadow cohorts are independent. A remote ramp of `0` +does not override an inherited nonzero `shadow.ramp`; set both to `0` when the +runtime policy must stop new invocation-driven Redis reads and fills. + +`DialCacheKeyConfig.disabled()` returns the complete overlay explicitly: +`requestLocal: false`, both serving ramps at `0`, `shadow.ramp: 0`, and +`shadow.logMismatches: false`. Its `ttlSec` map is empty, so inherited TTLs +remain available for a later ramp-up but inactive under this overlay. The kill +switch does not cancel already-admitted work or disable explicit maintenance +operations such as `invalidateRemote()`. + +### Validation and snapshots + +DialCache validates `defaultConfig` when `cached()` registers a definition and +whenever `getOrLoad()` is invoked: + +- TTLs must be positive safe integers no greater than `31_536_000` seconds + (365 days); +- serving ramps and `shadow.ramp` must be finite percentages in the inclusive + range `0` through `100`; +- layer maps and `shadow` must be objects; +- `requestLocal` and `shadow.logMismatches` must be booleans when present; and +- remote-read deadlines must be positive safe integers no greater than + 2,147,483,647 milliseconds. + +Invalid instance `redis.readTimeoutMs` values throw during `DialCache` +construction, as does an invalid `shadowMaxInFlight`. Invalid defaults are +rejected when `cached()` registers a definition or `getOrLoad()` is invoked. +`null`, zero, fractional, non-finite, string, and larger timeout values are +invalid; remote reads have no unbounded escape hatch. + +Each registration or one-shot invocation captures an immutable internal +snapshot, including the nested `shadow` object. Mutating the supplied config or +its maps later does not change that operation's baseline. Runtime policy +changes belong in the provider's returned overlay. + +Runtime TTL and ramp leaves are used as supplied rather than falling back to +valid default leaves: + +- an invalid TTL disables that layer with `invalid_ttl`; +- a serving ramp that is nonnumeric, non-finite, below `0`, or above `100` + disables that layer with `invalid_ramp`; values are never clamped; and +- other valid layers can continue to run. + +Invalid leaves also record a `config_resolution` error, distinguishing provider +garbage from an intentional ramp-down. A malformed runtime config object, +layer-map or `shadow` shape, `requestLocal`, or `remoteReadTimeoutMs` value fails +config resolution for the whole invocation. DialCache records +`config_resolution`, marks the no-layer path `config_error`, and runs the +fallback without a Redis read or write. + +Runtime shadow leaves are isolated from caller-serving policy: + +- An invalid `shadow.ramp` records remote `config_resolution` and skips shadow + work when an otherwise eligible Redis path evaluates it. DialCache does not + clamp the value or disable valid serving layers. +- An invalid `shadow.logMismatches` preserves the cache result, shadow work, + and terminal shadow metric, but suppresses the warning and records remote + `config_resolution`. This diagnostic leaf is evaluated only after the + metrics hook, cohort, and capacity gates admit the job. + +Static invalid shadow leaves remain definition-time errors for `cached()` and +invocation-time errors for `getOrLoad()`. The former flat `shadowRamp` field is +removed rather than aliased: `DialCacheKeyConfig` and static defaults reject it +with `DialCacheKeyConfig.shadowRamp was replaced by "shadow.ramp"`; a runtime +provider result containing it fails config resolution for the whole invocation +and runs the loader uncached. + +### Provider behavior + +`cacheConfigProvider` is called for every enabled cache invocation before any +cache lookup. Keep it cheap, cache remote or config-store reads inside the +provider, and give asynchronous work a finite application-owned deadline. + +DialCache fetches and resolves one config snapshot per enabled invocation. +Provider errors do not activate defaults: they fail open, record +`config_error`, and execute the fallback uncached. + +```ts +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; + +const dialcache = new DialCache({ + redis: { + client: dialCacheRedisClient, + readTimeoutMs: 75, + }, + cacheConfigProvider: async (key) => { + if (key.useCase === "GetUser") { + return new DialCacheKeyConfig({ + // Sparse override: inherit both TTLs and the local ramp. + ramp: { [CacheLayer.REMOTE]: 25 }, + // Shadow leaves merge independently with defaultConfig.shadow. + shadow: { + // Inherit the baseline logMismatches: false. + ramp: 5, + }, + // Per-use-case override of the instance's 75 ms read deadline. + remoteReadTimeoutMs: 35, + }); + } + return null; + }, +}); + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUser", + cacheKey: (userId) => userId, + trackForInvalidation: true, + defaultConfig: new DialCacheKeyConfig({ + // Omitted ramps default to 100% because these layers have TTLs. + ttlSec: { + [CacheLayer.LOCAL]: 30, + [CacheLayer.REMOTE]: 300, + }, + shadow: { + ramp: 0, + logMismatches: false, + }, + }), + }, +); +``` + +Ramp values are thresholds from 0 to 100. `0` disables the layer, `100` enables +it for every key, and an intermediate value selects keys whose DialCache-owned +deterministic bucket for the full cache key and layer is below that threshold. + +For a fixed cache identity and layer, increasing a ramp only adds keys and +decreasing it only removes keys; it does not reshuffle existing membership. +Local and remote cohorts are layer-specific. + +Ramps select key cohorts, not requests or load, so a ramp of `10` does not +guarantee 10% of calls, especially for a small or skewed key population. +DialCache keeps the assignment stable across releases. + +Applications that need an externally coordinated cohort can use +`cacheConfigProvider` to return a sparse per-key ramp override of `0` or `100`. + +Ramping down bypasses affected entries; it does not evict them, so a later +ramp-up can reuse entries that remain valid. + +`shadow.ramp` uses its own stable exact-key cohort, independent of both serving +ramps. Omitted and `0` disable shadow work; `100` selects every otherwise +eligible key. `shadow.logMismatches` separately opts confirmed mismatches into +byte-capped JSON warning fields; it does not enable shadow work and defaults +to `false`. Review the data-handling contract before turning it on. + +Shadowing requires a valid remote TTL, invalidation tracking, and a metrics +adapter with the shadow outcome hook. It can validate a served Redis hit or +exercise Redis while the remote serving ramp excludes the key. See +[Shadow validation and Redis bootstrap](shadow-validation.md) for eligibility, +clean-miss filling, deadlines, capacity, and rollout guidance. + +`shadowComparator` is stable operation behavior rather than runtime policy. It +defaults to Node strict deep equality and receives borrowed decoded-cache and +source values. A custom comparator must be synchronous, deterministic, +side-effect-free, non-mutating, and bounded. + +### Provider key input + +`cacheConfigProvider` receives the fully constructed, read-only `DialCacheKey` +for the invocation: + +| Field | Meaning | +| --- | --- | +| `namespace` | Logical application or environment namespace. | +| `keyType` and `id` | Primary identity. The selected id has already been converted to a string. | +| `args` | Secondary dimensions as normalized, name-sorted string pairs; entries whose value was `undefined` are omitted. | +| `useCase` | Stable operation name used in cache identity and metrics. | +| `prefix` | Encoded identity prefix, including a Redis Cluster hash tag when invalidation tracking is enabled. | +| `urn` | Complete encoded cache identity, including arguments and `useCase`. | +| `defaultConfig` | The operation's snapshotted baseline policy, or `null`. | +| `serializer` | The operation-specific serializer, or `null`. | +| `trackForInvalidation` | Whether the operation uses remote watermark tracking. | + +Use the identity fields to select policy; do not derive policy names or metric +dimensions from unbounded user input. The provider result remains a sparse +overlay and must not mutate the key. + +Most applications do not construct keys directly. Custom integrations can use +the root exports: + +- `new DialCacheKey(init)` to build the same public key shape; +- `normalizeArgs(record)` to omit `undefined`, stringify scalar values, and + sort argument names; +- `invalidationPrefix(namespace, keyType, id)` to build the encoded tracked + identity; and +- `redisClusterHashTag(value)` to wrap a validated value in a Redis Cluster hash + tag. + +The namespace and hash-tag components reject `{` and `}` as described under +[Identity rules](#identity-rules). + +## Request-local cache + +Set `requestLocal: true` to memoize resolved values for the lifetime of the +outermost `enable()` scope: + +```ts +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUser", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ requestLocal: true }), + }, +); +``` + +`requestLocal` is a runtime boolean rather than a TTL/ramp-controlled +`CacheLayer`. The provider can turn it on or off for each invocation. +`DialCacheKeyConfig.enabled(ttlSec)` enables only process-local and remote +caching, so request-local caching must be selected explicitly. + +The resolved config applies to the whole invocation. When `requestLocal` is +false, the invocation skips request-local lookup and storage without deleting a +value already memoized in the scope. A later invocation that enables it can +reuse that value. + +The outermost `enable()` call owns the request-local lifetime; nested `enable()` +calls reuse the same scope. State is allocated lazily, so scopes that use only +process-local or remote caching do not allocate it. + +Wrap the complete Node HTTP handler so the scope matches the request: + +```ts +import { createServer } from "node:http"; + +const server = createServer((req, res) => { + void dialcache + .enable(async () => { + const user = await getUser(readUserId(req)); + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify(user)); + }) + .catch((error: unknown) => handleRequestError(error, res)); +}); +``` + +Request-local storage has no capacity limit, eviction, or overflow mode. Values +are retained until the outermost callback settles. Use it for short-lived +scopes with bounded key cardinality. Split long-running streams or large batch +jobs into smaller scopes. + +## Process-local cache + +The process-local layer, `CacheLayer.LOCAL`, uses one LRU per `DialCache` +instance. It keeps at most 10,000 entries by default across all use cases while +retaining each entry's configured TTL. + +Set `localMaxSize` to a nonnegative safe integer to change the global entry cap. +`0` disables process-local storage: + +```ts +const dialcache = new DialCache({ localMaxSize: 25_000 }); +``` + +The limit counts entries rather than estimating JavaScript object memory. +Recently read entries stay resident ahead of less recently used entries when +the limit is reached. + +## Cached-value ownership + +Treat values returned by cached functions or `getOrLoad()` as immutable. +DialCache does not clone or freeze values stored in request-local or +process-local memory. +Mutating a cached object can be observed by: + +- later callers in the same request; +- callers in other requests that hit the process-local cache; and +- callers that coalesced onto the same in-flight result. + +This contract includes nested objects and arrays, `Map`, `Set`, `Buffer`, typed +arrays, and class instances. Redis deserialization can produce a different +reference from an in-memory hit, so reference identity is layer-dependent and +is not part of the API contract. + +Copy a value explicitly before changing it: + +```ts +const sharedUser = await getUser("123"); +const editableUser = structuredClone(sharedUser); +editableUser.displayName = "New name"; +``` + +Use a narrower copy when its semantics are sufficient. The ownership boundary +remains the caller's responsibility. diff --git a/docs/invalidation.md b/docs/invalidation.md new file mode 100644 index 0000000..ee80ebc --- /dev/null +++ b/docs/invalidation.md @@ -0,0 +1,259 @@ +# Targeted invalidation + +[Back to the README](../README.md) + +DialCache can invalidate related Redis entries without scanning or enumerating +keys. The mechanism is opt-in, remote-only, and based on per-identity Redis +watermarks. + +Read this complete contract before using targeted invalidation for mutable +production data. Correctness depends on cache-layer policy, Redis clock +synchronization, and an application-owned timing buffer. + +## Configure a tracked use case + +Set `trackForInvalidation: true` on a Redis-backed cached function or +`getOrLoad()` operation. After the source mutation commits, call +`dialcache.invalidateRemote(keyType, id, futureBufferMs)`: + +```ts +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { client: dialCacheRedisClient }, +}); + +// Chosen from this application's clock-skew bound and measured +// worst-case source and fallback timings. +const USER_INVALIDATION_BUFFER_MS = 5_000; + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetMutableUser", + cacheKey: (userId) => userId, + trackForInvalidation: true, + // Strongly invalidated mutable data should not use in-memory layers. + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { + [CacheLayer.REMOTE]: 300, + }, + ramp: { + [CacheLayer.REMOTE]: 100, + }, + }), + }, +); + +await updateUser("123", patch); +await dialcache.invalidateRemote( + "user_id", + "123", + USER_INVALIDATION_BUFFER_MS, +); +``` + +The buffer is an application-owned safety value. DialCache cannot choose a +universally safe nonzero default. It must be a nonnegative safe integer no +greater than `31_536_000_000` milliseconds (365 days). + +`invalidateRemote()` is an explicit remote maintenance operation and requires +`DialCacheConfig.redis`. A local-only `DialCache` remains valid for normal cache +operations, but invalidation does not silently become a no-op: without Redis it +rejects a `TypeError` whose message is +`DialCache invalidateRemote requires a configured Redis client`. + +## Identity and Redis Cluster placement + +Invalidation writes a watermark at: + +```text +{encodedNamespace:encodedKeyType:encodedId}#watermark +``` + +Tracked Redis values use the same Redis Cluster hash tag. For example: + +```text +{users-api:user_id:123}?locale=en#GetMutableUser:dialcache-frame-v1 +``` + +The value and watermark therefore live in the same Redis Cluster slot. Key +components are percent-encoded before joining, so delimiters inside ids or +arguments cannot collide with delimiters in the key format. + +`namespace` may never contain `{` or `}`; tracked `keyType` and `id` values may +not contain them because those three components form the hash tag. `args` and +`useCase` are encoded outside the hash tag and may contain braces. + +The internal `:dialcache-frame-v1` suffix identifies values written with +DialCache's binary protocol. Watermarks are stored as decimal timestamps. + +`keyType` plus `id` is the invalidation unit. One watermark covers every tracked +`useCase` and `args` variant with that identity. Untracked values do not consult +it. + +## Read and write behavior + +A tracked Redis value whose Redis-created timestamp is older than or equal to +the watermark is treated as stale and refreshed through fallback. + +`invalidateRemote(keyType, id, futureBufferMs)` sets the watermark to the +greater of: + +- its existing value; and +- Redis's current time plus the buffer. + +While that future window is active: + +1. A tracked Redis read treats the covered value as a miss. +2. The invocation runs its fallback. +3. If that fallback reaches the tracked Redis write before the window ends, + Redis rejects the write. +4. DialCache also suppresses the corresponding process-local population. +5. The fallback value still returns to its caller. + +Request-local memoization remains unconditional. A ramped-out invocation +without selected shadow work does not consult the watermark and is not fenced +by it. + +This is a timing contract, not a cancellation or acquisition fence. The buffer +blocks stale fallback results from passing the tracked Redis write only while +the configured window remains active. It does not cancel the fallback or force +it to read from an authoritative source. + +If a tracked remote read rejects or exceeds its deadline, DialCache cannot +establish watermark safety. It runs the fallback but skips both the Redis write +and process-local publication. This differs from a normal tracked miss, which +can attempt the fenced Redis write. Untracked fallbacks may still populate +process-local cache, and request-local memoization remains unconditional. + +### Shadow reads and fills + +[Shadow mode](shadow-validation.md) uses the same tracked protocol. A sampled +path can perform a tracked Redis read even when the remote serving ramp excludes +the key. A definitive `null` result can then attempt a tracked fill from the +caller-accepted source value, using the invocation's resolved remote TTL. + +An active future watermark rejects that fill and produces the bounded +`fill_blocked` shadow outcome. It does not reject or replace the value returned +to the caller. Caller-path request-local and process-local publication remains +independent when the remote serving layer is ramped out. + +The shadow read and fill are not atomic. An ordinary tracked cache write can +land between them, and either write can overwrite the other according to +arrival order when the watermark permits it. Shadow mode never repairs or +overwrites a non-null initial Redis payload; it only fills a definitive clean +miss. + +## Redis clock contract + +The bundled timestamp protocol assumes synchronized system clocks across every +Redis node eligible for primary promotion. + +Redis does not guarantee that `TIME` is monotonic across nodes, and DialCache +does not detect or compensate for cross-node clock skew. If the assumption is +violated, failover can: + +- temporarily suppress tracked cache fills; or +- allow a pre-invalidation value to remain readable until it expires or a later + invalidation advances the watermark past its timestamp. + +Monitor and bound the maximum negative clock skew across all promotion-eligible +nodes. Include that bound when sizing `futureBufferMs`. + +## Watermark durability + +Watermarks are invalidation state, not disposable cache entries. Redis must +preserve each marker for its derived TTL with `noeviction` or an equivalent +guarantee. Choose persistence, restore, and failover behavior that matches the +application's consistency requirements. + +Losing a marker through eviction, failover, restore, or external deletion +removes its prior publication fence. A missing marker makes tracked reads miss, +but a later tracked write creates a new baseline and can publish data that a +lost future watermark would have rejected. + +Redis replication is asynchronous. DialCache does not issue `WAIT` and does not +provide strong consistency across failover. + +## Watermark lifetime + +Tracked writes create a missing baseline watermark and ensure its TTL is at +least the value TTL plus one minute. They never shorten a longer or persistent +watermark TTL. Because cache TTLs cap at 365 days, the derived marker TTL can +reach 365 days plus the fixed one-minute margin. + +Invalidation ensures the TTL covers both the requested future buffer and any +still-future existing watermark, plus one minute. It also preserves a longer or +persistent TTL. The one-minute safety margin is fixed; there is no separate +configurable or global retention floor, and reads do not extend watermark +lifetime. + +## Choosing `futureBufferMs` + +`futureBufferMs` must be a nonnegative safe integer no greater than +`31_536_000_000` milliseconds (365 days). The API default is zero, but zero +provides no stale-publication protection once Redis time advances. + +Larger values, negative values, fractions, non-finite values, and wrong-type +values are rejected with a `RangeError` before DialCache records metrics, logs, +checks whether Redis is configured, or calls the client. An invalid buffer +therefore takes precedence over the missing-Redis `TypeError` and has no +invalidation telemetry side effects. + +Every production invalidation should pass a named, application-owned nonzero +value based on measured or conservatively bounded timings. Size it to cover: + +- maximum expected negative clock skew between promotion-eligible Redis nodes; +- source visibility or replication lag; +- the full remaining tail of any fallback that may already have observed the + pre-mutation value; +- `serializer.dump`; +- Redis client queue and network latency; +- Lua script execution; +- the Redis write itself; and +- a safety margin. + +Include the remaining lifetime of any sampled shadow fill based on a source +read that may have observed the pre-mutation state. A shadow deadline can stop +work before write dispatch, but it cannot prove that an already-dispatched +Redis command did not execute. + +Account for the underlying client's queue, dispatch, retry, and settlement +bounds as well as DialCache's shadow deadline. + +Invalidate only after the source mutation commits. + +Underestimating the interval can allow a delayed stale fallback to repopulate +Redis after the watermark window ends. Overestimating it lengthens the tracked +Redis miss and write-suppression window, increasing fallback load without +publishing stale values. + +A larger buffer does not delay or suppress returning fallback values to +callers. + +## Failure behavior and telemetry + +For a valid buffer, DialCache invokes the configured invalidation metric hook +with `layer="remote"` before it checks the Redis prerequisite. + +Missing configuration and Redis write failures then follow the same observable +failure path: DialCache logs `Error writing DialCache invalidation watermark`, +invokes the configured error metric hook with `useCase="watermark"`, +`layer="remote"`, `error="invalidation"`, and `inFallback=false`, then rethrows +the original error. Logger and metrics callback failures are isolated and +cannot replace that rejection. + +## In-memory layers remain local + +Targeted invalidation is remote-only. `invalidateRemote` does not evict existing +request-local or process-local entries. + +Strongly invalidated mutable data should disable request-local and process-local +caching. A short process-local TTL is appropriate only when the application +explicitly accepts that bounded stale-read window. + +If those layers remain enabled, their existing values can be returned without +reaching the remote watermark. diff --git a/docs/maintainers.md b/docs/maintainers.md new file mode 100644 index 0000000..7045011 --- /dev/null +++ b/docs/maintainers.md @@ -0,0 +1,136 @@ +# Maintainer guide + +[Back to the README](../README.md) + +## Validation + +Use the repository's pinned pnpm version through Corepack: + +```bash +corepack pnpm install --frozen-lockfile +corepack pnpm check +corepack pnpm test:integration +``` + +`pnpm check` runs strict typechecking, the unit suite with coverage, +bundles/declarations, and packed ESM/CJS consumer tests. The integration suite +uses Testcontainers and requires a working Docker-compatible container runtime +for Redis, Valkey, and Redis Cluster. + +CI runs development and integration checks on Node.js 24, then switches to the +declared minimum Node.js 22.0.0 to test the packed package. Keep the consumer +floor separate from the development runtime so a new dependency or emitted +syntax cannot silently raise the published requirement. + +Before changing a compatibility-sensitive surface, identify and extend the +corresponding packed, unit, and integration assertions: + +- package root and explicit adapter/protocol entry points in packed ESM and CJS + consumers; +- full cache-key identity, encoding, namespace behavior, and Redis Cluster hash + tags; +- deterministic serving- and shadow-ramp assignment, whose independent cohorts + must not reshuffle across releases, plus nested shadow-policy snapshot, + overlay, validation, and legacy `shadowRamp` rejection; +- the binary Redis frame, Lua arguments and reply domains, tracked + read/write/invalidation semantics, mixed-version serializer behavior, and the + ownership and immutability contract for retained string and `Buffer` + payloads; +- rejection and bounded error telemetry when `invalidateRemote()` is called + without a configured Redis client; +- shadow confirmation, clean-miss fill, capacity, deadline, detached work, and + payload-release behavior, plus default-off confirmed-mismatch logging and its + byte-capped native-JSON detail fields; +- exhaustive public unions and packed exports, including `MetricLayer`, + `ShadowValidationOutcome`, `ShadowComparator`, and `ShadowConfig`; and +- bounded metrics names, labels, reasons, error categories, scopes, outcomes, + units, and observer isolation from synchronous throws and rejected thenables. + +When changing user-facing examples, parse TypeScript fences, validate local +files and anchors, verify that README repository links are absolute for npm +rendering, and inspect the packed README. The package ships `README.md` but not +`docs/`. + +See [Shadow validation](shadow-validation.md) for the contract that the shadow +unit, integration, adapter, package, and benchmark assertions protect. + +## Cache-path benchmark + +From a repository checkout, install dependencies and run: + +```bash +corepack pnpm benchmark:request-local +``` + +The command builds `dist` before reporting ten scenarios: + +- sequential request-local hits; +- sequential process-local hits; +- enabled bounded fallbacks; +- request-local coalescing fan-out; +- process coalescing fan-out; +- Redis read-deadline coalescing; +- tracked Redis hits with shadow omitted; +- tracked Redis hits outside a partial shadow cohort; +- ramped-down Redis shadow-read detachment; and +- ramped-down Redis shadow-fill detachment. + +The benchmark is a maintainer tool and is not included in the published +package. + +It asserts fallback counts, coalescing state, returned values, semantic Redis +calls, cleaned-up deadline timers, stable shadow exclusion, tracked dark reads +and fills, caller detachment, mismatch confirmation, and the absence of shadow +writes or invalidations on warm hits. It deliberately applies no timing +threshold. + +Override its work sizes with: + +- `DIALCACHE_BENCH_ITERATIONS`; and +- `DIALCACHE_BENCH_FANOUT`. + +## Releasing + +Publishing starts by manually running the `Release` workflow from current +`main`. + +After the package checks pass, Semantic Release selects the next version from +Conventional Commits since the highest stable `vX.Y.Z` tag: + +- breaking changes bump major; +- `feat` bumps minor; and +- every other normal PR-title type bumps patch. + +Patch types are `fix`, `perf`, `docs`, `style`, `refactor`, `test`, `build`, +`chore`, `ci`, and `revert`. The highest required bump wins. + +The workflow opens a `release: ` pull request whose only change is the +matching `package.json` version. `release` is a reserved Conventional Commit +type configured not to request another release, so the version-control commit +does not cause an extra bump. + +GitHub marks workflow runs for a pull request opened with `GITHUB_TOKEN` as +approval-required. Approve those runs, review the pull request, and squash-merge +it normally through the protected branch. + +The merge triggers the publish job. Before any release side effect, it verifies: + +- current `main`; +- the release commit subject; +- the one-file diff; +- the package version; +- the absent tag; and +- Semantic Release's independently calculated version and commit. + +It then reruns the package checks and asks Semantic Release to: + +1. create the matching Git tag; +2. publish the public npm package with provenance; and +3. publish the GitHub release. + +The repository must enable **Allow GitHub Actions to create and approve pull +requests** under Actions workflow permissions. + +The workflow uses that capability only to create the version pull request. It +never approves or merges one, and no ruleset bypass actor or persistent release +credential is required. diff --git a/docs/observability.md b/docs/observability.md new file mode 100644 index 0000000..7c378b6 --- /dev/null +++ b/docs/observability.md @@ -0,0 +1,360 @@ +# Observability + +[Back to the README](../README.md) + +Metrics are disabled unless a `DialCacheMetricsAdapter` is passed to the +constructor. `new DialCache()` does not import a metrics backend, register +collectors, or emit metrics. + +DialCache provides first-party adapters for Prometheus and Datadog. Both use +caller-created, caller-owned clients and preserve one backend-neutral set of +bounded labels. + +## Prometheus + +Install `prom-client` separately: + +```bash +npm install prom-client@^15.1.3 +``` + +Create the registry your application owns, then pass an explicit adapter to +DialCache: + +```ts +import { Registry } from "prom-client"; +import { DialCache } from "dialcache"; +import { createPrometheusDialCacheMetrics } from "dialcache/prometheus"; + +const registry = new Registry(); + +const dialcache = new DialCache({ + namespace: "users-api", + metrics: createPrometheusDialCacheMetrics({ + registry, + prefix: "myapp_", + }), +}); + +app.get("/metrics", async (_req, res) => { + res.type(registry.contentType).send(await registry.metrics()); +}); +``` + +The adapter requires a caller-owned `Registry`. It never uses the global +default registry, and it does not clear or otherwise own the registry +lifecycle. + +Multiple adapters with the same registry and prefix reuse existing collectors +when their type, help, labels, histogram buckets, and exemplar mode match. +Adapter construction fails before registering anything if a same-name +collector has an incompatible schema. Use a unique prefix or separate registry +to resolve a collision. + +### Prometheus metrics + +The names below exclude the optional caller-selected prefix: + +| Metric | Type | Labels | Description | +| --- | --- | --- | --- | +| `dialcache_request_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | +| `dialcache_miss_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | +| `dialcache_disabled_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | +| `dialcache_error_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache or fallback errors by bounded failure site | +| `dialcache_invalidation_counter` | Counter | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | +| `dialcache_coalesced_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests split by request-local or process scope | +| `dialcache_shadow_validation_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `outcome` | Terminal outcomes for sampled Redis shadow jobs | +| `dialcache_get_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | +| `dialcache_fallback_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the wrapped fallback settles or timeout rejection is delivered | +| `dialcache_serialization_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | +| `dialcache_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | + +The disabled reasons are: + +- `context`; +- `policy_disabled`; +- `invalid_ttl`; +- `invalid_ramp`; +- `ramped_down`; and +- `config_error`. + +`policy_disabled` means that a process-local or remote layer has no effective +TTL after runtime overlays. This is an intentional policy result, including the +default when `defaultConfig` is omitted, rather than a configuration-loading +failure. + +Every metric includes `cache_namespace`, even disabled-context, +key-construction, coalescing, and invalidation paths that do not have a +constructed key. Its value is `DialCacheConfig.namespace`, which defaults to +`urn`. + +The `layer` label is: + +- `request_local`; +- `local`, meaning process-local; +- `remote`; +- `remote_shadow` for Redis reads, fills, serialization, and payload sizes + performed by detached shadow jobs; or +- `noop` for disabled-context, key-construction, and config-provider failures + where no cache layer was reached. + +The bounded `scope` label on `dialcache_coalesced_counter` distinguishes +`request_local` from `process`. `scope="process"` coordinates calls only within +one `DialCache` instance; separate instances in the same process do not share +in-flight state. + +### Shadow outcomes + +`dialcache_shadow_validation_counter` reports one terminal outcome for each +admitted or explicitly dropped shadow job. Datadog exposes the same bounded +outcomes through `dialcache.shadow.count`: + +| `outcome` | Meaning | +| --- | --- | +| `match` | The cached and source values matched. | +| `mismatch` | They differed, and a confirmation read found the original Redis payload unchanged. | +| `superseded` | They differed, but the Redis payload changed or disappeared before confirmation. | +| `filled` | A clean shadow miss was populated successfully. | +| `fill_blocked` | An invalidation watermark blocked a clean-miss fill. | +| `fill_error` | Serializing or writing a clean-miss fill failed. | +| `redis_error` | The initial detached Redis read failed. | +| `source_error` | The source-of-truth read failed. | +| `deserialization_error` | The retained Redis payload could not be deserialized for comparison. | +| `comparison_error` | The comparator threw or did not return a synchronous boolean. | +| `confirmation_error` | The confirmation Redis read failed. | +| `timeout` | The shadow deadline expired. | +| `dropped` | Per-key deduplication or the instance flight cap rejected the job. | + +The outcome counter deliberately has no `layer` or cache-id label. Operational +Redis metrics produced inside the same job use `layer="remote_shadow"`, which +keeps detached work separate from caller-serving `layer="remote"` telemetry. +See [Shadow validation](shadow-validation.md) for the read, confirmation, fill, +and deadline semantics behind these outcomes. + +### Confirmed mismatch warnings + +Shadow metrics remain bounded and contain no cache ids or values. A use case can +separately set `shadow.logMismatches: true` to emit one warning after a terminal +`mismatch` is confirmed. Logging is default-off, does not replace the outcome +metric, and does not activate shadow work without the `shadowValidation` hook. + +The warning contains stable metadata, the logical cache key capped at 2 KiB, +and independently generated native-JSON strings for the cached and source +comparator inputs capped at 8 KiB each. Those fields are value-bearing, and +truncation is not redaction. + +See +[Confirmed mismatch logging](shadow-validation.md#confirmed-mismatch-logging) +for confirmation semantics, exact fields, JSON behavior, operational limits, +and the required data-handling review. + +## Datadog + +Install `hot-shots` separately: + +```bash +npm install hot-shots@^17.0.0 +``` + +Create the DogStatsD client your application owns, then pass it to the Datadog +adapter: + +```ts +import StatsD from "hot-shots"; +import { DialCache } from "dialcache"; +import { createDatadogDialCacheMetrics } from "dialcache/datadog"; + +const dogStatsD = new StatsD({ + host: process.env.DD_AGENT_HOST, + globalTags: { + service: "users-api", + env: process.env.DD_ENV ?? "development", + }, + errorHandler: (error) => + logger.warn("DogStatsD error", { error }), +}); + +const dialcache = new DialCache({ + namespace: "users-api", + metrics: createDatadogDialCacheMetrics({ + client: dogStatsD, + observationMetricType: "distribution", + namespace: "dialcache", + }), +}); + +function shutdown(): void { + // Drain outstanding cache operations before application shutdown. + dogStatsD.close(); +} +``` + +`hot-shots` is the supported and tested client, but the adapter depends only on +the exported `DatadogDogStatsDClient` structural interface. + +DialCache does not: + +- import or install `hot-shots`; +- create a client; +- flush buffers; +- close sockets; or +- otherwise own the client lifecycle. + +### Distribution or histogram + +`observationMetricType` is required. + +Choose `"distribution"` when latency and size percentiles must aggregate across +hosts. Enable the desired distribution percentiles and aggregations in +Datadog. + +Choose `"histogram"` when host-level histogram aggregation matches the existing +Datadog setup. The choice applies uniformly to all four duration and size +metrics. Both modes produce Datadog custom metrics. + +Distribution volume scales with unique tag-value combinations. Datadog counts +five baseline aggregations per combination; enabling percentile aggregations +adds five more. Review +[Datadog's custom-metrics billing guidance](https://docs.datadoghq.com/account_management/billing/custom_metrics/) +before rollout. + +Do not send both observation types under the same metric namespace. When +changing types, use a new namespace during migration so one metric identity +never mixes histogram and distribution points. + +### Datadog namespaces + +`DatadogMetricsOptions.namespace` is the metric-name namespace and defaults to +`dialcache`. It is separate from `DialCacheConfig.namespace`, the logical cache +namespace emitted as the `cache_namespace` tag. + +The Datadog metric namespace must: + +- start with a letter; +- contain only letters, numbers, underscores, and dot-separated non-empty + segments; and +- produce final metric names no longer than 200 characters. + +The adapter rejects invalid namespaces and overlong final names instead of +relying on client-side normalization. + +A `hot-shots` `prefix` is applied after the adapter constructs the name. Include +that prefix when checking final length, and avoid accidentally combining it +with the adapter namespace. Client-level `globalTags` are appended by +`hot-shots`; the table below lists only tags added by DialCache. + +### Datadog metrics + +The adapter emits exact increments of `1` for counters and preserves seconds +and bytes without unit conversion: + +| Metric | Type | Tags | Description | +| --- | --- | --- | --- | +| `dialcache.request.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | +| `dialcache.miss.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | +| `dialcache.disabled.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | +| `dialcache.error.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache or fallback errors by bounded failure site | +| `dialcache.invalidation.count` | Count | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | +| `dialcache.coalesced.count` | Count | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests by sharing scope | +| `dialcache.shadow.count` | Count | `cache_namespace`, `use_case`, `key_type`, `outcome` | Terminal outcomes for sampled Redis shadow jobs | +| `dialcache.get.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | +| `dialcache.fallback.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the wrapped fallback settles or timeout rejection is delivered | +| `dialcache.serialization.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | +| `dialcache.serialization.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | + +Client throws and rejected returned thenables are isolated by DialCache's +fire-and-forget observer boundary. Buffered transport failures that happen +after the client call returns remain outside that boundary. Configure the +DogStatsD client's error handling and shutdown behavior as part of application +ownership. + +## Error categories + +The `error` label reports the operation that failed instead of copying the +thrown value's class or `Error.name`: + +| `error` | Meaning | +| --- | --- | +| `key_construction` | The cache-key selector or `DialCacheKey` construction failed | +| `config_resolution` | Runtime or layer configuration validation or resolution failed | +| `cache_read` | A process-local read or non-timeout remote read failed | +| `cache_read_timeout` | A remote read exceeded its effective DialCache deadline | +| `cache_write` | A process-local or remote cache write failed | +| `serialization_load` | Deserializing a Redis payload failed | +| `serialization_dump` | Serializing a value for Redis failed | +| `invalidation` | Writing an invalidation watermark failed | +| `fallback` | The source loader failed or exceeded its DialCache deadline | +| `unknown` | Reserved for a future failure site that cannot be classified otherwise | + +These values are defined by the backend-neutral core and are identical for +every adapter. + +A valid `invalidateRemote()` call without a configured Redis client is still an +invalidation attempt: DialCache records `dialcache_invalidation_counter` (or +`dialcache.invalidation.count`), logs the failure, records +`error="invalidation"`, and rejects with the original focused `TypeError`. +Invalid `futureBufferMs` input is rejected before these observers run. + +Remote-read timeouts use `layer="remote"` and `in_fallback="false"`. They are +errors rather than misses, and the remote get-duration observation includes +the wait. Coalesced followers do not multiply the timeout error. Deadline +details remain out of labels and are available on the logged +`RedisReadTimeoutError`. + +Raw thrown values, error names, messages, cache ids, arguments, and Redis keys +are never included in labels. When DialCache logs a cache-plumbing failure, the +raw details remain available through the configured logger; not every metric +error or shadow outcome has a matching log entry. + +The explicitly opted-in confirmed-mismatch warning is a separate value-bearing +log and does not alter the metric schema. + +`in_fallback` remains the explicit distinction between cache plumbing and +application fallback failures. + +## Custom adapters + +Implement `DialCacheMetricsAdapter` and pass it through +`new DialCache({ metrics })` for another telemetry backend. + +| Hook | Required | Value | +| --- | --- | --- | +| `request(labels)` | yes | One active cache-layer lookup. | +| `miss(labels)` | yes | One cache miss. | +| `disabled(labels)` | yes | One skipped layer or no-layer invocation with a bounded `reason`. | +| `error(labels)` | yes | One bounded failure site with `inFallback`. | +| `invalidation(labels)` | yes | One explicit remote invalidation call. | +| `coalesced(labels)` | no | One follower that joined request-local or process-scoped work. | +| `shadowValidation(labels)` | no | One terminal sampled-shadow outcome. This hook must be implemented for shadow jobs to execute. | +| `observeGet(labels, seconds)` | yes | Cache-read duration in seconds. | +| `observeFallback(labels, seconds)` | yes | Fallback duration in seconds. | +| `observeSerialization(labels, seconds)` | yes | Serializer dump/load duration in seconds. | +| `observeSize(labels, bytes)` | yes | Serialized remote payload size in bytes. | + +The root package exports `DialCacheMetricsAdapter` and every associated label, +reason, error-kind, layer, scope, and shadow-outcome type, including +`ShadowValidationMetricLabels` and `ShadowValidationOutcome`. +`shadowValidation` remains optional so existing custom adapters keep +compiling, but DialCache does not admit shadow work when the configured +adapter omits it. The Prometheus and Datadog adapters implement the hook. + +Metrics and logger methods are typed `void` and invoked as fire-and-forget +observers. DialCache also defensively consumes, but never awaits, a thenable +returned at runtime. + +Synchronous throws and asynchronous rejections are isolated so telemetry +cannot change cache correctness, fallback results, or shadow outcomes. + +A custom adapter may buffer or transmit asynchronously, but it owns delivery, +flushing, resources, and shutdown after the call returns. Keep +application-owned namespace, use-case, and key-type labels stable and +low-cardinality, and preserve the seconds and bytes units shown above. + +Every backend-neutral label object exposes the logical namespace as camel-case +`cacheNamespace`. Map it to the backend's `cache_namespace` label or tag. This +field is present even when no key or cache layer was reached. + +Omit `metrics` to disable metrics entirely. Because shadow jobs require an +observable terminal outcome, omitting metrics also disables shadow execution +even when a key policy sets `shadow.ramp` or enables +`shadow.logMismatches`. diff --git a/docs/redis.md b/docs/redis.md new file mode 100644 index 0000000..e74e92a --- /dev/null +++ b/docs/redis.md @@ -0,0 +1,491 @@ +# Redis and Valkey + +[Back to the README](../README.md) + +DialCache's remote TTL layer supports standalone Redis, standalone Valkey, and +Redis Cluster. The application creates, connects, configures, drains, and closes +the underlying client. DialCache borrows a client-independent +`DialCacheRedisClient` adapter and does not own the connection lifecycle. + +Sampled non-serving Redis reads and fills use the same adapter and tracked +protocol. See [Redis shadow validation](shadow-validation.md) for eligibility, +comparison, capacity, metrics, and rollout behavior. + +## Install a client + +Choose one supported integration: + +```bash +# node-redis +npm install redis@~4.7.1 + +# or Valkey GLIDE +npm install @valkey/valkey-glide +``` + +## node-redis + +Register DialCache's native scripts when creating the client, connect it, and +pass the DialCache-compatible adapter to `DialCache`: + +```ts +import { createClient } from "redis"; +import { DialCache } from "dialcache"; +import { + createNodeRedisDialCacheClient, + dialcacheRedisScripts, +} from "dialcache/node-redis"; + +const redisClient = createClient({ + url: process.env.REDIS_URL, + scripts: dialcacheRedisScripts, + disableOfflineQueue: true, + commandsQueueMaxLength: 1_000, + socket: { connectTimeout: 2_000 }, +}); + +await redisClient.connect(); + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { + client: createNodeRedisDialCacheClient(redisClient), + }, +}); + +async function shutdown(): Promise { + // Stop new work and await every cached call and invalidation first. + await redisClient.quit(); +} +``` + +`redis.client` is required when the remote layer is configured. Node-redis +users should register the supplied scripts and wrap the connected client with +`createNodeRedisDialCacheClient` as shown above. Active remote reads have a +50-millisecond DialCache deadline by default. Set `redis.readTimeoutMs` for an +instance-wide value or use `DialCacheKeyConfig.remoteReadTimeoutMs` for +per-use-case static and runtime policy. + +Local-only caching does not require a Redis client, but the explicit remote +maintenance operation `invalidateRemote()` does. It rejects when Redis is not +configured so a caller cannot mistake an absent watermark write for successful +invalidation. See [Targeted invalidation](invalidation.md) for the complete +contract. + +The adapter computes each script's SHA, uses `EVALSHA`, and retries with `EVAL` +after `NOSCRIPT`. Its cluster client routes scripts by their first key and +performs that fallback on the selected shard. Tracked reads are deliberately +routed to primaries so a lagging replica cannot hide an invalidation watermark. + +Deployments using tracked invalidation must also satisfy the +[watermark durability](invalidation.md#watermark-durability) contract. + +## Valkey GLIDE + +Pass an already-created standalone or cluster client and the exact module +namespace that created it: + +```ts +import * as valkeyGlide from "@valkey/valkey-glide"; +import { DialCache } from "dialcache"; +import { createValkeyGlideDialCacheClient } from "dialcache/valkey-glide"; + +const glideClient = await valkeyGlide.GlideClient.createClient({ + addresses: [{ host: "127.0.0.1", port: 6379 }], + requestTimeout: 2_000, + advancedConfiguration: { + connectionTimeout: 2_000, + }, +}); + +const redisClient = createValkeyGlideDialCacheClient( + glideClient, + valkeyGlide, +); + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { client: redisClient }, +}); + +function shutdown(): void { + // Drain cached calls and invalidations before releasing resources. + redisClient.dispose(); + glideClient.close(); +} +``` + +DialCache uses the supplied namespace's `Script` constructor and +`Decoder.Bytes` value without importing a GLIDE runtime itself. Passing the same +module namespace that created the client prevents linked workspaces or +applications with another installed GLIDE version from mixing native script +handles. + +The GLIDE adapter uses GLIDE's native script lifecycle and byte decoder. GLIDE +routes scripts from their declared keys. + +## Lifecycle ownership + +The application owns the complete Redis lifecycle: + +1. Create and connect the underlying client. +2. Construct the DialCache-compatible adapter. +3. Pass that adapter as `redis.client`. +4. During shutdown, stop starting DialCache-backed work. +5. Await every outstanding cached-function, `getOrLoad()`, and + `invalidateRemote()` promise, including fallbacks that may still write + Redis. +6. Drain or terminate client-native Redis work that may have outlived + DialCache's caller-serving or shadow-read wait. +7. Dispose adapter-owned resources. +8. Close the underlying connection. + +DialCache has no `close()` or drain method. It never disposes or closes caller +resources. + +Awaiting public DialCache promises does not drain detached shadow work. Shadow +scheduling and deadline timers are unreferenced, and there is no shadow drain +handle. + +A source read, serializer, Redis command, or metrics delivery started by a +shadow job can remain active during teardown. Stop new work before closing +dependencies and use their native drain or termination controls. An +already-dispatched shadow fill may have executed even if its outcome is lost. + +The node-redis adapter owns no additional resources, so close the underlying +client after draining work. + +The GLIDE adapter owns five native `Script` handles but not the wrapped +connection. Call its idempotent `dispose()` after operations finish and before +closing GLIDE. Disposing while an adapter operation is in flight throws rather +than releasing a live script. A DialCache read timeout does not prove that the +client-side invocation has settled. + +## Remote-read deadlines and async liveness + +Every caller-serving Redis read and each detached shadow read has a finite +monotonic deadline. DialCache uses this precedence for `cached()` and +`getOrLoad()`: + +```text +runtime remoteReadTimeoutMs + -> defaultConfig.remoteReadTimeoutMs + -> redis.readTimeoutMs + -> 50 ms +``` + +Each explicit value must be a positive safe integer no greater than +2,147,483,647 milliseconds. Remote reads have no unbounded escape hatch. + +Outside an enabled scope and on an earlier in-memory hit, DialCache creates no +remote-read timer. A key ramped out of Redis serving creates no caller-serving +timer, but an independently selected shadow job can create unreferenced timers +for its tracked `C0` and optional `C1` reads. + +### Caller-serving timeout and fail-open behavior + +When the deadline expires, DialCache: + +1. aborts the optional adapter signal; +2. logs a root-exported `RedisReadTimeoutError` carrying `useCase` and + `timeoutMs`; +3. records `cache_read_timeout`; +4. consumes and ignores any late read fulfillment or rejection; and +5. runs the source fallback. + +The deadline bounds caller wait and cache publication. It does not guarantee +server-side cancellation, and an event-loop-blocking operation cannot be +preempted. When control returns, DialCache still checks the monotonic deadline +before accepting the result. + +A remote read rejection or timeout does not count as a miss and never triggers +a second Redis operation. After fallback, an untracked key may still populate +an active process-local cache. A tracked key suppresses process-local +publication because watermark safety was not established. Request-local +memoization remains unconditional. + +Same-key callers in one request-local or process coalescing scope share the +leader's read, timer, and remaining budget. A later independent invocation may +start a new remote read even if the prior client operation is still settling. + +The `fallbackTimeoutMs` timer is separate and starts only if and when the source +loader begins. The remote-read timer covers neither config resolution, +serializer loading, the fallback, Redis writes, nor invalidation. + +### Shadow-read deadlines + +Shadow `C0` and `C1` reads use the same effective `remoteReadTimeoutMs` and +cooperative abort signal as caller-serving reads. Their timers are +unreferenced. A `C0` failure or read timeout produces `redis_error`; the same +failure at `C1` produces `confirmation_error`. Both paths also record the +ordinary bounded Redis error under `layer="remote_shadow"`. + +The read deadline bounds DialCache's wait, not the underlying client +operation. Shadow capacity remains occupied while a timed-out raw Redis read +is still settling. The whole shadow job has a separate deadline described in +[Redis shadow validation](shadow-validation.md#capacity-deadlines-and-detachment). + +### Custom-client contract + +Custom adapters implement the complete client-independent read, write, and +invalidate contract: + +```ts +type Awaitable = T | Promise; + +interface RedisReadContext { + readonly timeoutMs: number; + readonly signal: AbortSignal; +} + +interface DialCacheRedisClient { + read( + request: RedisReadRequest, + context?: RedisReadContext, + ): Awaitable; + write(request: RedisWriteRequest): Awaitable; + invalidate(request: RedisInvalidationRequest): Awaitable; +} +``` + +#### `read` + +- Return an operation-owned serialized `string` or `Buffer`, or `null` for a + miss. +- Keep the payload stable after settlement because DialCache can retain it for + shadow work. An adapter that recycles response storage must return a + dedicated `Buffer`. +- For a tracked request, compare the value timestamp and watermark atomically. + A missing watermark or a value at or behind it is a miss. + +#### `write` + +- Accept `cacheTtlMs` as a positive integer no greater than + `31_536_000_000` milliseconds (365 days), apply that TTL, and record server + time atomically. +- For a tracked request, create a missing baseline and retain it for at least + the value TTL plus one minute without shortening a longer or persistent + lifetime. +- Return `true` only when the value was written and `false` when publication is + rejected. + +#### `invalidate` + +- Accept `futureBufferMs` as a nonnegative integer no greater than + `31_536_000_000` milliseconds (365 days). +- Advance `watermarkKey` monotonically to at least server time plus that buffer. +- Retain the watermark long enough to cover the buffer and any still-future + existing watermark, plus one minute, without shortening a longer or + persistent lifetime. +- Reject on failure. + +`write()` returning `false` is a safe publication refusal, not an adapter error. +DialCache still returns the fallback value but skips the corresponding +process-local population. A thrown cache-write error fails open; a thrown +explicit invalidation error is rethrown to the caller. + +The optional `RedisReadContext` keeps existing one-argument readers +structurally compatible. Adapters should use its signal for cooperative +cancellation where their client supports it, but the core deadline remains +authoritative when they do not. + +The bundled node-redis adapter forwards the signal in per-command options. This +can remove queued work where supported, but aborting after dispatch cannot +unsend a command or prove that Redis stopped executing it. + +The GLIDE script API has no per-invocation signal, so a timed-out script +invocation may continue inside the adapter. Its configured +[`requestTimeout`](https://glide.valkey.io/languages/nodejs/api/interfaces/BaseClient.BaseClientConfiguration.html) +and +[`advancedConfiguration.connectionTimeout`](https://glide.valkey.io/languages/nodejs/api/interfaces/BaseClient.AdvancedBaseClientConfiguration.html) +still bound client-native work. + +### Native operation budgets + +DialCache's read deadline bounds its caller wait, not the complete lifetime of +the underlying client work. Configure finite client-native budgets for: + +- connection establishment; +- reconnection and retries; +- offline queueing; +- dispatch; and +- response time. + +For node-redis 4.7, `socket.connectTimeout`, `disableOfflineQueue`, and +`commandsQueueMaxLength` bound connection or queue behavior but do not impose a +strict response deadline after dispatch. Use client-native shutdown or +termination behavior that matches the application's resource and ambiguity +requirements. + +Redis writes and invalidations, asynchronous `cacheConfigProvider` work, and +custom `Serializer` methods still require their own finite budgets. The same +is true for detached source, Redis, serializer, and telemetry work admitted by +shadow validation. + +Do not put writes or invalidations behind a bare `Promise.race`: rejecting the +outer promise neither removes queued work nor proves that a dispatched +mutation did not execute. + +DialCache's [fallback deadline](coalescing.md#fallback-deadlines) covers only the +source loader. Prefer resource-native budgets and cooperative cancellation for +every injected operation. + +## Serialization + +DialCache uses `JsonSerializer` by default. A cache operation can select a +typed serializer, and `redis.serializer` supplies the instance default when an +operation does not select one. Serializers run only for remote reads and +writes; request-local and process-local values remain native references. + +### Default JSON behavior + +DialCache uses native `JSON.stringify` and `JSON.parse` by default. There is no +runtime validation pass, so the default adds no traversal beyond JSON +serialization itself. A top-level `undefined` result is supported with an +internal sentinel. + +When `serializer.load` rejects a Redis payload, DialCache: + +1. records a `serialization_load` error; +2. counts the read as a remote miss; +3. runs the fallback; and +4. attempts to replace the rejected payload. + +A validating custom serializer can therefore treat an incompatible cached +value as a refreshable miss without adding a schema version to the cache key. +This replacement behavior describes the caller-serving path. Shadow work +reports `deserialization_error` for a non-null payload and never repairs it. + +`JsonSerializer` validates JSON syntax only. It cannot detect that a +structurally valid payload came from an incompatible application value schema. +Applications that retain one `useCase` across deployments must keep +default-JSON values backward compatible. + +For an incompatible change, either: + +- provide a serializer whose `load` method validates and rejects the old shape; + or +- change `useCase` to isolate the new cache entries. + +During a mixed deployment, mutually incompatible validating serializers can +repeatedly reject and replace each other's values. Correctness is preserved, +but expect additional fallback and Redis-write load until the rollout +converges. + +### Typed serializer requirement + +When a cached function or inline loader's resolved return type is statically +JSON-compatible, `serializer` is optional. This includes JSON primitives, +arrays, plain object or interface shapes, optional object fields, and a +top-level `undefined`. + +Types known not to survive the default round trip require a typed +`Serializer`: + +```ts +import { DialCache, type Serializer } from "dialcache"; + +const dialcache = new DialCache(); + +const dateSerializer: Serializer = { + dump: (value) => value.toISOString(), + load: (value) => + new Date(Buffer.isBuffer(value) ? value.toString("utf8") : value), +}; + +const getUpdatedAt = dialcache.cached( + (userId: string) => db.fetchUpdatedAt(userId), + { + keyType: "user_id", + useCase: "GetUpdatedAt", + cacheKey: (userId) => userId, + serializer: dateSerializer, + }, +); +``` + +The compile-time guard rejects known incompatible shapes such as: + +- `Date`, `Map`, and `Set`; +- `bigint`, symbols, and functions; +- Buffers and typed arrays; +- method-bearing class instances; +- required nested `undefined`; and +- `unknown` and `any`. + +The guard applies to every `cached()` declaration and `getOrLoad()` invocation +because active layers are selected at runtime. A global Redis serializer is not +parameterized by each returned type, so it cannot discharge this requirement. +Non-JSON operations must select a typed serializer. + +This guard is deliberately conservative rather than a proof of runtime data. +TypeScript cannot detect non-finite numbers, cyclic or shared references, +runtime getters, `toJSON` behavior, or data-only class instances that resemble +plain objects. Opaque, generic, or deeply recursive types may also require an +explicit serializer. + +Providing `Serializer`, including an explicitly typed +`JsonSerializer`, is a trusted caller assertion. DialCache does not perform +an additional serialize-and-deserialize cycle to validate it. + +Opted-in confirmed-mismatch logging is separate from cache serialization. It +uses native `JSON.stringify` on the deserialized cached snapshot and source +value and does not call the configured serializer again. Its byte caps are not +redaction; review the data-handling contract in +[Redis shadow validation](shadow-validation.md) before enabling it. + +Shadow validation can call `load` again for the same served payload and can +call `dump` after a ramped-down caller has received its source result. Custom +serializers must treat payloads and values as borrowed and immutable, return +independent values from repeated loads, and copy a Buffer before mutating it. +See [Data ownership and custom integrations](shadow-validation.md#data-ownership-and-custom-integrations). + +### Advanced wire protocol + +The core Redis boundary is the client-independent `DialCacheRedisClient` +interface. It exchanges serialized values as `string | Buffer` and does not +expose client-specific commands or wire encodings. + +The `dialcache/redis-protocol` entry point exports the exact bundled protocol +building blocks: + +- `READ_CACHE_SCRIPT` and `READ_TRACKED_CACHE_SCRIPT`; +- `WRITE_CACHE_SCRIPT` and `WRITE_TRACKED_CACHE_SCRIPT`; +- `INVALIDATE_CACHE_SCRIPT`; and +- `REDIS_FRAME_VERSION`, `REDIS_ENCODING_UTF8`, and + `REDIS_ENCODING_BINARY`. + +The scripts implement the atomic read, publication, invalidation, server-time, +and derived-watermark-lifetime behavior required above. Custom adapters can +throw these root-exported error classes: + +- `DialCacheRedisPayloadError`; +- `DialCacheRedisPayloadEncodingError`; and +- `DialCacheRedisProtocolError`. + +They distinguish malformed payloads, unsupported encodings, and invalid Lua +reply domains in logs. DialCache records bounded `cache_read`, +`cache_read_timeout`, `cache_write`, or `invalidation` metrics by failure site. + +Shadow validation adds no Redis protocol operation. It composes the same +tracked read and write requests: `C0` and `C1` are tracked reads, and a clean +miss can use one tracked write. + +#### Binary frame + +Redis values use a compact binary frame: + +```text +byte 1 format version +bytes 2-9 Redis-created timestamp in milliseconds (uint64, big-endian) +byte 10 payload encoding (0 = UTF-8, 1 = raw binary) +bytes 11... serialized payload +``` + +Redis's Lua `struct` library packs and unpacks the timestamp. Redis TTL is +authoritative, so expiry metadata is not duplicated in the frame. + +The payload comes from the cache operation's serializer or `JsonSerializer` by +default. Custom serializers can return `string` or `Buffer`. Strings are +stored as UTF-8; Buffers are stored byte-for-byte without base64 expansion. +Adapters restore the same representation before calling `serializer.load`. diff --git a/docs/shadow-validation.md b/docs/shadow-validation.md new file mode 100644 index 0000000..27c13ae --- /dev/null +++ b/docs/shadow-validation.md @@ -0,0 +1,503 @@ +# Redis shadow validation + +[Back to the README](../README.md) + +Shadow validation lets a service exercise and inspect tracked Redis behavior +without letting the shadow path choose the caller's result. It is useful for +validating warm entries against the source of truth and for bootstrapping clean +misses before increasing the Redis serving ramp. + +Shadow mode is an operational rollout tool, not a new serving layer or a +correctness boundary. It adds source and Redis work, provides best-effort +evidence through bounded metrics, and relies on the same invalidation, +serialization, deadline, and client-lifecycle contracts as the remote layer. + +## At a glance + +| Path reached by the caller | Caller receives | Selected shadow work | +| --- | --- | --- | +| Tracked, serving Redis hit | The decoded Redis value | Read the source later, compare it with the retained Redis payload, and confirm a candidate mismatch with one more tracked Redis read. | +| Valid remote policy, but the key is ramped out of Redis serving | The normal source result | Read Redis later without serving it. Compare a hit, or fill a clean miss from the caller-accepted source result. | +| Serving Redis miss | The normal source result | None. The ordinary request path already performs fallback and fill. | +| Request-local or process-local hit | The in-memory value | None. Normal traversal never reached Redis. | + +Redis serving and shadow selection use independent deterministic cohorts. A +key can therefore be served without being shadowed, shadowed without being +served, selected for both, or selected for neither. + +## Configure a shadow cohort + +Shadow validation requires a tracked operation, a valid remote TTL, a metrics +adapter with the optional shadow hook, and a positive `shadow.ramp`: + +```ts +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { client: dialCacheRedisClient }, + metrics, + // Per-instance cap; the default is 1. + shadowMaxInFlight: 4, +}); + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUser", + cacheKey: (userId) => userId, + trackForInvalidation: true, + shadowComparator: (cached, source) => + cached.id === source.id && cached.version === source.version, + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.REMOTE]: 300 }, + // Exercise and fill Redis without allowing it to serve this cohort. + ramp: { [CacheLayer.REMOTE]: 0 }, + shadow: { ramp: 5 }, + }), + }, +); +``` + +`shadow.ramp` accepts percentages from `0` through `100`. An effective value of +`0`, or omission from both baseline and runtime policy, disables shadow work; +`100` selects every otherwise eligible exact cache key. + +A sparse runtime overlay that omits the field inherits the baseline. +Intermediate values select a stable key cohort across calls and instances. The +shadow bucket is independent of the remote serving bucket, so equal ramp +percentages do not select the same keys. + +Use `shadow: { ramp: 100 }` when every eligible invocation should exercise the +non-serving Redis path before a later serving-ramp increase. This still +observes only keys invoked during the shadow period; it is not a full keyspace +scan or warming guarantee. + +The root-exported `ShadowConfig` groups `ramp` with the default-off +`logMismatches` diagnostic policy. Runtime overlays merge those two leaves +independently: an omitted leaf inherits its baseline, while +`logMismatches: false` explicitly disables inherited logging. + +The former flat `shadowRamp` field has been removed; migrate it to +`shadow.ramp`. Public construction and static defaults reject the old field, +and a raw runtime provider result containing it fails config resolution and +runs the fallback uncached rather than inheriting defaults. + +## Eligibility + +DialCache schedules shadow work only when all of these conditions hold: + +- the call began inside an enabled DialCache scope; +- a Redis or Valkey adapter is configured; +- normal traversal reaches the remote layer; +- the operation sets `trackForInvalidation: true`; +- the resolved remote policy has a valid TTL; +- the effective `shadow.ramp` is positive and selects the exact key; +- the configured metrics adapter implements `shadowValidation`; and +- the instance has capacity and no shadow job already owns that exact key. + +A remote serving ramp of `0` is eligible because it preserves a valid remote +policy while excluding the key from serving. + +Missing or invalid remote policy, provider failure, an untracked key, an +omitted metrics hook, an invalid or zero shadow ramp, cohort exclusion, an +earlier in-memory hit, or a disabled call does not start a shadow-only Redis +path. + +An invalid runtime `shadow.ramp` does not disturb an otherwise valid +caller-serving Redis hit. DialCache skips shadow work and records a +`config_resolution` error. + +`DialCacheKeyConfig.disabled()` sets the shadow ramp and both serving ramps to +`0`, and sets `shadow.logMismatches` to `false`, so it is the complete kill +switch for new cache invocations. It does not cancel work already admitted. + +An invalid runtime `shadow.logMismatches` does not change the cache result, +shadow result, or terminal shadow metric. For an admitted job, DialCache records +one remote `config_resolution` error and suppresses the warning. + +DialCache validates this diagnostic leaf only after the metrics-hook, +exact-key-cohort, and capacity gates; ineligible, cohort-excluded, and +explicitly dropped work does not report that configuration error. + +See [Configuration and cache layers](configuration.md) for runtime-overlay +precedence and policy validation. + +## Serving-hit and ramped-down paths + +### Serving Redis hit + +The request path performs its normal tracked Redis read and deserialization. +It returns that cached value without waiting for shadow work and retains the +exact serialized payload as `C0`. + +On a later unreferenced event-loop turn, the shadow job: + +1. invokes the source loader inside a disabled DialCache context; +2. deserializes `C0` again into an independent cached snapshot; +3. compares that snapshot with the source value `S`; and +4. performs a confirmation read only when the values differ. + +The additional source call must be safe to run for observation. Process +coalescing means one serving Redis leader schedules at most one job for its +coalesced followers. + +### Ramped down from Redis serving + +When a valid remote policy excludes the key specifically because its serving +ramp is down, the caller runs and awaits the normal source loader. Shadow work +reuses that same caller-owned promise as `S`; it does not invoke the loader a +second time. + +The detached job reads tracked Redis as `C0`. A hit is compared with `S`. A +clean miss can be filled from `S` using the invocation's resolved remote TTL +snapshot. + +The shadow Redis value never supplies the caller or populates request-local or +process-local memory. If those in-memory layers are active, only the caller's +source result can populate them. + +When request-local and process-local caching are off and Redis serving is +ramped down, caller invocations remain uncached and do not gain process +coalescing merely because shadowing is enabled. Concurrent same-key callers +can each run the source; shadow deduplication independently admits one shadow +job and reports the others as `dropped`. + +## The `C0` / `S` / `C1` algorithm + +`C0` is the original tracked Redis payload: either the payload that served the +caller or the result of the detached ramped-down read. `S` is the successfully +accepted source value. `C1` is an optional tracked confirmation read. + +1. Obtain `C0`. +2. If `C0` is `null`, follow the clean-miss fill path described below. +3. Otherwise, obtain `S`, deserialize a new snapshot from `C0`, and compare the + cached and source values. +4. When they match, emit `match`; no confirmation read is needed. +5. When they differ, read tracked Redis again as `C1`, bypassing the + request-local and process-local caches. +6. If `C1` is missing or its bytes differ from `C0`, emit `superseded`. +7. If `C1` is byte-identical to `C0`, emit `mismatch`. + +String payloads compare by their UTF-8 bytes, Buffer payloads compare by bytes, +and a string/Buffer pair with the same UTF-8 bytes is identical for +confirmation. DialCache does not deserialize `C1`, compare it with `S`, or +chase another version. + +`mismatch` therefore means that the exact observed Redis payload survived one +tracked confirmation read after a semantic disagreement. It is not a +cross-system atomic snapshot or a guarantee that the mismatch still exists. +`superseded` means only that the original observation could not be confirmed. + +No non-null `C0` is repaired, overwritten, invalidated, or given a refreshed +TTL. That rule also applies when detached deserialization fails. + +### Clean-miss fill + +A clean miss means the tracked semantic Redis read returned `null`. It does +not include a non-null payload that the serializer cannot load. + +On the ramped-down path, DialCache can serialize the caller-accepted `S` and +attempt one ordinary tracked Redis write with the resolved TTL: + +- `filled` means the client returned `true` before the shadow deadline; +- `fill_blocked` means the invalidation watermark returned `false`; and +- `fill_error` means serialization or the Redis write failed. + +A source rejection or caller fallback timeout never produces an accepted `S` +and never starts the fill. Once serialization has finished, DialCache checks +the whole-job deadline again before dispatching the write. + +The `C0` read and fill are not atomic. The fill is a normal tracked overwrite, +not a compare-and-set or write-if-still-missing operation. Another writer can +populate Redis after `C0` misses and then be overwritten by the shadow fill. + +## Comparison semantics + +By default, DialCache uses Node's `util.isDeepStrictEqual`. Plain-object +property insertion order does not affect equality; values, array order, +prototypes, constructors, Buffers, Maps, Sets, and other supported structures +remain strictly compared. + +`shadowComparator(cachedValue, sourceValue)` can define application-level +equality, such as ignoring a volatile refresh timestamp. It is stable +use-case behavior, not runtime rollout policy. The comparator must: + +- return a boolean synchronously; +- be deterministic and side-effect-free; +- avoid mutating either borrowed input; and +- complete in bounded time. + +A throw or non-boolean result is `comparison_error`, not `mismatch`. An +accidental promise is not accepted as a result. DialCache consumes its +settlement so a later rejection is not unhandled, while the job remains +subject to its deadline and capacity rules. + +The cached comparator input is newly deserialized from `C0`; it is not the +object already returned to a serving-hit caller. The source input is the raw +loader result. This deliberately exposes lossy serialization unless a custom +comparator declares that normalization acceptable. + +## Confirmed mismatch logging + +The bounded shadow outcome metric is the default diagnostic. A use case can +separately opt in to one warning for each terminal `mismatch`: + +```ts +new DialCacheKeyConfig({ + shadow: { + ramp: 5, + logMismatches: true, + }, +}); +``` + +Logging does not activate shadow work: the operation must still pass every +eligibility gate, including the required `metrics.shadowValidation` hook. A +warning is emitted only after `C1` is byte-identical to `C0` and the terminal +`mismatch` metric is recorded. Matches, candidates that become `superseded`, +timeouts, dropped work, and errors remain metric-only. + +The warning message is `DialCache shadow validation mismatch`. Its details are: + +| Field | Value | +| --- | --- | +| `cacheNamespace`, `useCase`, `keyType`, `outcome` | Stable metadata; `outcome` is always `"mismatch"`. | +| `cacheKey` | The logical DialCache URN, not the physical Redis storage key; capped at 2 KiB of UTF-8. | +| `cachedValueJson` | Native JSON for the newly deserialized `C0` comparator input; capped at 8 KiB of UTF-8. | +| `sourceValueJson` | Native JSON for the raw source comparator input; capped at 8 KiB of UTF-8. | + +DialCache applies `JSON.stringify` independently to the two values. If it throws +or returns `undefined`, that field is `null` and the other side is still +attempted. + +A byte-clipped string ends in `...[truncated]` within its cap without splitting +a UTF-8 sequence. DialCache never passes the raw compared-value references to +the logger, calls the configured serializer again, or computes a textual diff. + +These bounds limit the fields handed to the logger, not the data DialCache must +inspect to build them: + +- truncation is not redaction; the logical URN can contain ids and arguments, + and either value can contain secrets or personal data; +- native JSON invokes getters and `toJSON`, can omit or normalize unsupported + values, and produces `null` here for cycles, `bigint`, or other failures; +- stringification is synchronous, and the 8 KiB cap applies only after it + returns, so it does not bound traversal, hook execution, event-loop time, or + the intermediate string; and +- metadata, logger framing, and transport escaping are outside the field caps, + so the final event can exceed a sink-specific size limit. + +Enable mismatch logging only for trusted, reasonably bounded values and with an +approved logger, redaction, transport, access, and retention policy. An +unexpected detail-construction failure degrades to the metadata-only warning. +Synchronous logger throws and rejected promises or thenables remain isolated +from cache and shadow correctness. + +## Capacity, deadlines, and detachment + +`shadowMaxInFlight` is a positive safe integer on each `DialCache` instance and +defaults to `1`. It counts scheduled and running jobs. The optional `C1` read +and clean-miss fill remain part of the original slot. + +There is no queue. DialCache emits `dropped` when: + +- another shadow job already owns the exact cache key; or +- the instance has reached `shadowMaxInFlight`. + +Separate instances have separate caps. This is not fleet-wide admission +control for Redis or the source of truth. + +Each job has one monotonic deadline across the detached Redis read, source +result, serializer work, comparison, confirmation read, and clean-miss fill: + +- a finite `fallbackTimeoutMs` is also the whole shadow budget; +- when `fallbackTimeoutMs` is `null`, the caller fallback is unbounded but + shadow work still uses 60 seconds; +- serving-hit timing begins when the detached callback starts; and +- ramped-down timing begins immediately before the caller's source invocation, + including its synchronous prefix. + +Each `C0` or `C1` Redis read also uses the effective +`remoteReadTimeoutMs`. That read deadline bounds DialCache's wait, not the raw +client operation. + +The scheduler, Redis-read timers, and overall shadow timer are unreferenced. +They do not keep an otherwise idle process alive. Detachment is still work on +the Node event loop, not a worker thread; synchronous source, serializer, or +comparator code can occupy the event loop after the request continues. + +Deadline expiry records the applicable outcome, releases retained `C0`, and +prevents later phases from starting. It does not cancel JavaScript promises or +prove that a dispatched Redis command stopped. + +Shadow-owned reads, serializer calls, comparators, and writes can therefore +retain the slot after the DialCache deadline until their underlying work +settles. On the ramped-down path, the shared source promise is caller-owned and +does not retain the shadow slot after the job abandons it. + +Give every source, serializer, Redis client, and telemetry transport a finite +native resource budget. A shadow `timeout` or `fill_error` after write dispatch +does not prove Redis was unchanged. Conversely, `filled` means the client +reported success before the deadline, not that the entry is still present. + +## Data ownership and custom integrations + +Detached execution preserves the original `cached()` argument references or +the `getOrLoad()` loader closure. DialCache cannot generically clone captured +state. Snapshot mutable source-selection inputs before invoking DialCache so +the later source read still describes the already-built cache key. + +Treat the source result `S` as immutable after return. A clean-miss serializer +may inspect it after the caller has continued. + +The effective serializer has additional shadow requirements: + +- `load` can run twice for a sampled serving hit; +- repeated loads of the same payload must be independent; +- `load` must not mutate a borrowed Buffer; and +- asynchronous `load` and `dump` methods need finite application-owned + deadlines. + +A custom `DialCacheRedisClient.read()` must return an operation-owned payload +whose contents remain stable after the method settles. DialCache can retain +that exact `string | Buffer` for detached deserialization and confirmation. +An adapter that pools or recycles response storage must return a dedicated +Buffer. + +A custom metrics adapter must implement the optional `shadowValidation` hook +to admit shadow work. The hook remains optional at the type level so existing +adapters continue to compile; omitting it deliberately keeps shadow execution +off. + +See [Redis and Valkey](redis.md) for the complete custom-client, payload, +deadline, and connection-lifecycle contracts. + +## Invalidation and race boundaries + +Shadow mode is limited to invalidation-tracked keys. Both `C0` and `C1` use the +tracked read protocol, which atomically checks the value timestamp against the +watermark. Bundled adapters route tracked reads to the primary. + +A clean-miss fill uses the same serializer, Redis-time timestamp, value TTL, +and watermark-aware tracked write as an ordinary fill. A future watermark can +reject it as `fill_blocked`. Size `futureBufferMs` to cover the complete source, +serialization, client queue, network, script, and write interval if stale +publication protection matters. + +The watermark fences the tracked Redis write; it does not make the earlier +`C0` read and later fill atomic. It also does not synchronously invalidate +request-local or process-local entries. Shadow mode never evicts those layers. + +See [Targeted invalidation](invalidation.md) for the clock, durability, +retention, and future-buffer contracts. + +### Command amplification + +| Selected path | Added source work | Added Redis work | +| --- | --- | --- | +| Serving Redis hit, semantic match | One observational source read | None beyond the serving read | +| Serving Redis hit, mismatch candidate | One observational source read | One tracked confirmation read | +| Ramped-down Redis hit, semantic match | None beyond the caller's source read | One tracked `C0` read | +| Ramped-down Redis hit, mismatch candidate | None beyond the caller's source read | Tracked `C0` and `C1` reads | +| Ramped-down clean Redis miss | None beyond the caller's source read | One tracked `C0` read and at most one tracked write | +| Serving Redis miss | None beyond the ordinary path | None beyond the ordinary read and fill | + +Capacity limits bound concurrent jobs, not total work over time. Measure source +and Redis load while increasing `shadow.ramp`. + +## Metrics and compatibility + +Every admitted job, and every job explicitly rejected by deduplication or the +capacity cap, reports one bounded terminal outcome: + +| `outcome` | Meaning | +| --- | --- | +| `match` | The deserialized `C0` and source value matched semantically. | +| `mismatch` | They differed and byte-identical `C1` confirmed the original `C0`. | +| `superseded` | They differed, but `C1` was missing or had different bytes. | +| `filled` | A clean miss was populated successfully before the deadline. | +| `fill_blocked` | The invalidation watermark rejected the clean-miss fill. | +| `fill_error` | Serialization or the clean-miss Redis write failed. | +| `redis_error` | The initial detached `C0` read failed or reached its read deadline. | +| `source_error` | The source loader rejected without being the caller's own DialCache fallback timeout. | +| `deserialization_error` | The retained non-null `C0` could not be deserialized. | +| `comparison_error` | The comparator threw or did not return a synchronous boolean. | +| `confirmation_error` | The `C1` read failed or reached its read deadline. | +| `timeout` | The whole shadow deadline expired, including a shared caller fallback timeout. | +| `dropped` | Exact-key deduplication or the per-instance cap rejected the job. | + +Ineligible or cohort-excluded invocations do not emit a shadow outcome. +Outcome labels contain the logical cache namespace, `useCase`, and `keyType`; +they never contain ids, values, payloads, Redis keys, or exception text. The +separately opted-in mismatch warning is value-bearing and does not change this +metric-label contract. + +Redis reads, serializer work, payload sizes, and Redis errors inside detached +jobs use the existing metric hooks with `layer="remote_shadow"`. This keeps +their cost separate from caller-serving `layer="remote"` traffic. The read +that supplied a serving-hit `C0` remains caller-path `remote` work, and a +ramped-down caller still reports `disabled{layer="remote", +reason="ramped_down"}`. + +The dedicated shadow outcome has no `layer` label. A clean `C0` miss also +records the ordinary `miss{layer="remote_shadow"}` before the job's terminal +fill, source, or timeout outcome. + +`MetricLayer` includes `remote_shadow`, and `ShadowValidationOutcome` includes +every value in the table above. TypeScript consumers with exhaustive switches +or `Record` values must handle those cases. Dashboards filtered to +`layer="remote"` intentionally exclude detached cost. + +See [Observability](observability.md) for exact Prometheus and Datadog metric +names, units, labels, and the custom-adapter interface. + +## Rollout checklist + +Before increasing `shadow.ramp`: + +- confirm the source loader is safe to invoke observationally on serving hits; +- use a valid remote TTL with the serving ramp at `0`; +- enable tracked invalidation and choose a defensible `futureBufferMs`; +- verify serializer, comparator, Redis client, source, and telemetry budgets; +- start with a small per-instance capacity and measure `dropped`; +- account for the command amplification above; and +- arrange application-owned shutdown for dependencies that can outlive the + DialCache deadline. + +Keep `shadow.logMismatches` off unless the logged key and values have been +classified and the application has approved their synchronous JSON cost, +redaction, transport, access, and retention policy. + +During rollout, monitor outcome ratios together with detached source latency, +`remote_shadow` Redis errors, fill load, and ordinary source health. Treat +`mismatch` as a signal to investigate value meaning, serialization, key +identity, and source consistency—not as an automatic repair instruction. + +Increase the Redis serving ramp only after the observed cohort, source load, +and invalidation behavior meet the application's acceptance criteria. To stop +new Redis serving and shadow activity through runtime policy, set both the +remote serving ramp and `shadow.ramp` to `0`. + +## Shutdown + +DialCache has no shadow drain or close method. Awaiting promises returned by +`cached()`, `getOrLoad()`, and `invalidateRemote()` drains request-path work, +not detached shadow jobs. + +During shutdown: + +1. stop admitting new DialCache-backed requests; +2. set serving and shadow ramps to `0` if runtime policy remains active; +3. await request-path cache calls and invalidations; +4. use source, Redis-client, serializer, and telemetry-native controls to drain + or terminate their work; +5. dispose adapter-owned resources; and +6. close underlying connections. + +Unreferenced shadow scheduling means the process may exit before an outcome is +delivered. Already-started source reads, serializers, Redis commands, or +telemetry can still be active, and an already-dispatched fill may have +executed even when its outcome is lost during teardown. Shadow validation is +therefore best-effort during shutdown by design.