From 4a2dc71094a14d7a13eae004ec360e9d18e1f548 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Thu, 23 Jul 2026 10:38:25 -0700 Subject: [PATCH 1/6] docs: focus README on safe cache rollouts --- AGENTS.md | 5 + README.md | 913 ++++++++++++++---------------------------- docs/coalescing.md | 220 ++++++++++ docs/configuration.md | 434 ++++++++++++++++++++ docs/invalidation.md | 205 ++++++++++ docs/maintainers.md | 76 ++++ docs/observability.md | 261 ++++++++++++ docs/redis.md | 377 +++++++++++++++++ 8 files changed, 1868 insertions(+), 623 deletions(-) create mode 100644 docs/coalescing.md create mode 100644 docs/configuration.md create mode 100644 docs/invalidation.md create mode 100644 docs/maintainers.md create mode 100644 docs/observability.md create mode 100644 docs/redis.md diff --git a/AGENTS.md b/AGENTS.md index 64d5883..2aaeca8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,6 +7,8 @@ DialCache is a TypeScript caching library with explicit request-scoped enablemen ## Structure ```text +README.md # Adoption guide, safety model, and reference routing +docs/ # Focused user-facing configuration and operations guides src/ dialcache.ts # Main DialCache API and cached-function wrapper config.ts # Public configuration and rollout types @@ -36,6 +38,9 @@ test/ # Unit and Redis integration tests ## Conventions - Preserve strict TypeScript settings and public abstraction boundaries. +- Keep the README focused on evaluation and adoption. Put complete operational + contracts in a focused `docs/` guide and link it from the relevant README + summary. - Keep Redis client-specific behavior in adapters; core code depends on `DialCacheRedisClient`. - Public exports belong in the root or an explicit integration entry point such as `src/node-redis.ts`, `src/prometheus.ts`, or `src/redis-protocol.ts`. - Use `corepack pnpm` for project commands. diff --git a/README.md b/README.md index 8c4ce1c..e676737 100644 --- a/README.md +++ b/README.md @@ -4,44 +4,80 @@ [![Codecov](https://codecov.io/gh/lan17/DialCache/branch/main/graph/badge.svg)](https://codecov.io/gh/lan17/DialCache) [![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/lan17/DialCache/badge)](https://scorecard.dev/viewer/?uri=github.com/lan17/DialCache) -Fine-grained TypeScript caching with explicit enabled contexts, request-local memoization, process-local and Redis TTL caching, stable key construction, runtime rollout controls, request coalescing, adapter-based observability, and Redis watermark-based targeted invalidation. +**Roll out backend caching like a feature—not a leap of faith.** + +**DialCache is** a TypeScript library for caching database and service reads +inside Node.js backends. It routes reusable async functions and inline loaders +through one read-through path with request-local memoization, a bounded +in-process LRU, and optional Redis or Valkey caching. + +The “dial” is per-use-case runtime control. Start with caching off, dial the +process-local and remote layers up for stable cohorts of keys, and dial them +back down without changing the loader. + +**DialCache is not** a frontend data cache, cache server, Redis or Valkey +client, or runtime configuration service. It supplies the cache path and +rollout controls; your application still decides what is safe to cache and +owns loader behavior, connections, runtime configuration, keys, TTLs, +invalidation policy, and resource budgets. + +## Safety comes from explicit controls + +- **Off by default.** Outside `dialcache.enable(...)`, both cached wrappers and + inline loaders are true pass-throughs: DialCache does not build a key, + resolve config, access a cache, or coalesce the call. Inside an enabled + scope, a layer still needs an effective policy before it participates. +- **Gradual and reversible rollout.** Configure TTL and ramp independently for + the process-local and remote layers. A ramp of `0` is off, `100` is fully on, + and `DialCacheKeyConfig.disabled()` is the all-layer policy kill switch. +- **Fail-open cache path.** Key, config, cache-read, and serialization-load + failures fall through to the source loader. Cache-write, + serialization-dump, logging, and metrics failures do not replace an otherwise + usable fallback result. Explicit remote invalidation failures are rethrown so + callers never assume a mutation was made safe when it was not. +- **Bounded defaults.** The process-local cache has a 10,000-entry default cap, + active remote reads have a 50-millisecond default deadline, and enabled + fallback executions have a 60-second default deadline. The read deadline + bounds DialCache's wait, not necessarily the underlying Redis command; + applications still need resource-native budgets for client work, config + providers, serializers, and source I/O. + +Use DialCache when you want to: + +- add caching to database or service reads without scattering cache get/set + plumbing across call sites; +- begin with one layer or a small deterministic key cohort, observe it, and + expand or reverse the rollout per use case; +- combine request-local, process-local, and shared caching behind one key and + policy contract; or +- coalesce hot-key misses, invalidate related Redis entries, and emit bounded + cache metrics without rebuilding those mechanisms for every function. ## Contents - [Install](#install) - [Quick start](#quick-start) -- [How caching works](#how-caching-works) -- [Enabled context](#enabled-context) -- [Defining cached functions](#defining-cached-functions) - - [One-shot inline cache blocks](#one-shot-inline-cache-blocks) -- [Keys, ids, and extra dimensions](#keys-ids-and-extra-dimensions) -- [Runtime config and ramp controls](#runtime-config-and-ramp-controls) -- [Cache layers](#cache-layers) - - [Request-local cache](#request-local-cache) · [Process-local cache](#process-local-cache) · [Redis-backed TTL cache](#redis-backed-ttl-cache) · [Remote read deadlines](#remote-read-deadlines-and-async-liveness) · [Serialization](#serialization) -- [Cached-value ownership](#cached-value-ownership) -- [Targeted invalidation and watermarks](#targeted-invalidation-and-watermarks) -- [Request coalescing](#request-coalescing) - - [Fallback deadlines](#fallback-deadlines) · [Coalescing state](#coalescing-state) -- [Metrics](#metrics) -- [Maintainers](#maintainers) +- [Dial caching up or down](#dial-caching-up-or-down) +- [How the read path works](#how-the-read-path-works) +- [Core concepts](#core-concepts) +- [Production checklist](#production-checklist) +- [Reference guides](#reference-guides) ## Install ```bash pnpm add dialcache -# Choose a Redis client when using the remote layer: -pnpm add redis@~4.7.1 -# or -pnpm add @valkey/valkey-glide -# Add a metrics client only when using its adapter: -pnpm add prom-client@^15.1.3 -# or -pnpm add hot-shots@^17.0.0 ``` DialCache requires Node.js 22.0.0 or newer. Production deployments should use a [currently supported LTS release](https://nodejs.org/en/about/previous-releases). +Redis, Valkey, Prometheus, and Datadog integrations are optional and keep their +clients application-owned: + +- [Redis and Valkey setup](https://github.com/lan17/DialCache/blob/main/docs/redis.md) +- [Prometheus and Datadog setup](https://github.com/lan17/DialCache/blob/main/docs/observability.md) + ## Quick start ```ts @@ -59,678 +95,309 @@ const getUser = dialcache.cached( }, ); -// Caching is OFF outside an enable() scope (see "Enabled context"), so this runs the fn uncached: +// Outside enable(), this is a true pass-through to db.fetchUser: await getUser("123"); -// Inside enable(), reads are cached: +// Inside enable(), the active cache layers participate: const user = await dialcache.enable(() => getUser("123")); ``` -## How caching works - -The wrapped function is the **fallback**: it runs whenever no active cache layer returns a value, whether because layers missed, were disabled, or failed open. - -When caching is enabled, reads flow through: - -```text -request-local cache -> process-local cache -> Redis cache -> fallback function -``` - -- Request-local hits return the value memoized in the current outermost `enable()` scope. -- Results from the lower chain are memoized request-locally when that layer is enabled. -- Process-local hits return immediately. -- Process-local misses try Redis and populate the process-local cache on a Redis hit. -- Redis misses call the fallback and attempt to populate Redis and, when active, the process-local cache. Tracked invalidation may suppress both publications. -- Redis read failures and timeouts are logged, counted in metrics, and fail open without attempting a second Redis operation. Redis write failures also fail open. `invalidateRemote` logs/counts Redis failures and rethrows them so callers do not assume invalidation succeeded. -- Cache-key construction and config-provider failures also fail open and run the fallback uncached. -- A missing effective process-local/Redis TTL disables that layer by policy; a configured TTL with no ramp defaults to 100%. Disabled layers record a disabled reason and fall through to the next layer/fallback. +`cached(fn, options)` preserves the function's parameters and returns a +Promise-based wrapper. `DialCacheKeyConfig.enabled(60)` gives the process-local +and remote layers a 60-second baseline TTL; the remote layer participates only +when a Redis or Valkey client is configured. -Caching as a whole is only active inside an enabled context, described next. +For a one-shot calculation that should remain inline, +[`getOrLoad()`](https://github.com/lan17/DialCache/blob/main/docs/configuration.md#one-shot-inline-loaders) +accepts a +zero-argument loader and a direct key through the same cache contract. -## Enabled context - -Caching is **off by default** and only active inside a `dialcache.enable(...)` scope. This is deliberate: it lets you turn caching **off in write paths** so a stale read can't be cached around a write. DialCache uses Node `AsyncLocalStorage` to keep enabled state scoped to the current asynchronous call chain. - -**Enable once at your request boundary** (e.g. a middleware that wraps read-request handling) so individual call sites don't each need it; wrap mutation handlers in `disable()`: +Enable caching once at a read-request boundary instead of at every call site. +Keep nested mutation work uncached with `disable()`: ```ts await dialcache.enable(async () => { - await getUser("123"); // cached + const user = await getUser("123"); await dialcache.disable(async () => { - await updateUser("123", patch); // reads here are uncached + await updateUser("123", patch); }); - - await getUser("123"); // cached again -}); -``` - -- Default is disabled — `cached()` and `getOrLoad()` calls made **outside** any `enable()` scope simply run their loader uncached (no error), so wrap your read paths to actually cache. -- Enabled state is async-scope-local, not process-global. -- Nested `enable` / `disable` scopes restore the previous behavior when the callback completes. Nested `enable()` calls reuse the outer request-local scope rather than creating a new one. - -## Defining cached functions - -Use `cached(fn, options)` for an extracted, reusable function. The wrapped callable has the same parameters and always returns a `Promise`. For a one-shot calculation that should remain inline, use [`getOrLoad()`](#one-shot-inline-cache-blocks). - -| Option | Required | Description | -| --- | --- | --- | -| `keyType` | yes | The kind of id the key addresses (e.g. `"user_id"`). Together with the id, the invalidation unit for tracked entries. | -| `useCase` | yes | Identifies the individual cache: part of the stored key and the metrics label. | -| `cacheKey` | yes | Selector over `fn`'s parameters; returns a bare id or `{ id, args }`. | -| `defaultConfig` | no | `DialCacheKeyConfig` baseline policy that runtime config overlays field by field (see [Runtime config](#runtime-config-and-ramp-controls)). | -| `serializer` | when the return type is not statically JSON-compatible | Per-function `Serializer` for Redis values (see [Serialization](#serialization)). | -| `trackForInvalidation` | no (default `false`) | Opts this use case's Redis entries into watermark-based targeted invalidation. | -| `fallbackTimeoutMs` | no (default `60_000`) | Fallback deadline in milliseconds, at most 2,147,483,647; `null` disables it (see [Fallback deadlines](#fallback-deadlines)). | - -`cached()` validates `useCase` at registration: a duplicate within one `DialCache` instance throws `UseCaseIsAlreadyRegisteredError`. Both APIs reject the internal name `watermark` with `UseCaseNameIsReservedError`. - -### One-shot inline cache blocks - -`getOrLoad(load, options)` runs one zero-argument loader through the same policy, cache layers, coalescing, invalidation, metrics, serialization, deadlines, and fail-open behavior as `cached()`. It is useful when only part of a larger function should be cached and the loader needs to capture local values: - -```ts -// Reuse the caller-owned defaults; getOrLoad() snapshots them per invocation. -const profileCacheDefaults = DialCacheKeyConfig.enabled(60); - -const profile = await dialcache.getOrLoad( - async () => { - const user = await db.getUser(userId); - return renderProfile(user, locale); - }, - { - keyType: "user_id", - useCase: "BuildProfile", - key: { id: userId, args: { locale } }, - defaultConfig: profileCacheDefaults, - }, -); -``` - -The options match `cached()` except that the direct `key` replaces the `cacheKey` selector. `defaultConfig` and `fallbackTimeoutMs` are validated and snapshotted for each invocation. Outside an enabled scope, DialCache invokes `load` directly without constructing a key or resolving runtime policy. - -`getOrLoad()` does not register its `useCase`, so repeated calls should reuse one stable, deployment-defined name such as `"BuildProfile"`. Keep it bounded: never derive `useCase` from a user, request, id, or other high-cardinality input because it is part of both cache identity and metrics labels. Put those values in `key` instead. - -Every captured value that can change the result belongs in the bare id or `{ id, args }` key. Concurrent same-key calls may share one caller's in-flight loader and cached value, so all call sites for that identity must also agree on value meaning and serialization. Prefer `cached()` when a loader is reusable; prefer `getOrLoad()` when the calculation is intentionally local to one call site. - -## Keys, ids, and extra dimensions - -For `cached()`, the key comes from the required `cacheKey` selector whose parameters are inferred from `fn`. `getOrLoad()` accepts the same bare id or `{ id, args }` shape directly through `key`: - -The selected or direct key is the value identity contract. It must include every input dimension that can affect the returned value; otherwise distinct calls can reuse the same cached value or share the same in-flight fallback through request coalescing. - -```ts -const searchPosts = dialcache.cached( - (userId: string, page: number, filter: string) => db.searchPosts(userId, page, filter), - { - keyType: "user_id", - useCase: "SearchPosts", - cacheKey: (userId, page, filter) => ({ id: userId, args: { page, filter } }), - defaultConfig: DialCacheKeyConfig.enabled(60), - }, -); -await dialcache.enable(() => searchPosts("u1", 2, "active")); -``` - -`DialCacheConfig.namespace` is the logical cache namespace and the first component of every key. It defaults to `"urn"`, producing keys such as `urn:user_id:123#GetUser`. Set a stable application-specific value when multiple applications may use the same Redis deployment: - -```ts -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: redisClient }, }); ``` -That produces Redis keys beginning with `users-api:...`, or `{users-api:...}` for invalidation-tracked values. `namespace` is DialCache's single cache-identity and key-partitioning setting: it participates in request-local, process-local, Redis, coalescing, deterministic ramp, invalidation, and metrics. It may not contain `{` or `}` because DialCache reserves those characters for Redis Cluster hash tags. Use a namespace to express any required application or environment separation, such as `production-users-api`. - -- **`keyType` + `id` is the invalidation unit for tracked Redis entries.** `dialcache.invalidateRemote("user_id", "123", futureBufferMs)` writes one watermark for that user; any `trackForInvalidation` Redis entry with the same `keyType` and `id` is refreshed across all `args` variants when Redis is read. `invalidateRemote` does not evict existing request-local or process-local entries (see [Targeted invalidation](#targeted-invalidation-and-watermarks)), and untracked Redis entries do not consult the watermark. `useCase` identifies the individual cache (it's the metrics label and part of the stored key). -- **`args` are part of the cache key** — different `args` produce different entries — but invalidation is by `id` only. -- **Scalar key equality is string-based.** Runtime type is not an identity dimension: for matching surrounding dimensions, numeric `1`, string `"1"`, and bigint `1n` identify the same key; argument values `null` and `"null"` also match. `-0` matches `0`, and an `undefined` argument is omitted. If a deployment changes the logical meaning represented by a scalar, change an explicit identity dimension such as `keyType`, `useCase`, or an argument name/value. -- **Non-key inputs** (for example a db handle) are parameters ignored by a `cacheKey` selector or values captured by a `getOrLoad()` loader. They still reach non-coalesced executions, but concurrent same-key cache misses share the leader's execution, so do not omit values like auth context, locale, or cancellation behavior unless sharing one result is correct. -- **Methods:** pass `obj.method.bind(obj)` (or `(...a) => obj.method(...a)`) — a bare `obj.method` reference loses `this`. - -Changing the namespace value intentionally creates a cold-cache boundary across every layer. Old and new keyspaces do not share Redis values or invalidation watermarks. During an overlapping deployment, an invalidation handled by one version is invisible to the other, which can continue serving a stale tracked value until its value TTL expires. If remote invalidation correctness matters, a normal rolling deployment is unsafe: use a coordinated no-overlap cutover, or an operational bridge that prevents both versions from serving remote cache across mutations (for example, temporarily disable and clear remote caching during the transition). After the cutover, provision for fallback/refill load and allow old Redis keys to expire by TTL. - -## Runtime config and ramp controls - -Instance-wide behavior is set through the `DialCache` constructor: - -| `DialCacheConfig` option | Default | Description | -| --- | --- | --- | -| `namespace` | `"urn"` | Logical cache namespace and first key component (see [Keys, ids, and extra dimensions](#keys-ids-and-extra-dimensions)). | -| `redis` | none | `{ client: DialCacheRedisClient, readTimeoutMs?: number }`; enables the Redis layer with a 50 ms default read deadline (see [Redis-backed TTL cache](#redis-backed-ttl-cache)). | -| `localMaxSize` | `10_000` | Global process-local entry cap; `0` disables process-local storage. Nonnegative safe integer. | -| `cacheConfigProvider` | none | Resolves runtime config per enabled invocation as a sparse overlay on the function's `defaultConfig`; `null` applies no overrides. | -| `metrics` | disabled | A `DialCacheMetricsAdapter` (see [Metrics](#metrics)). | -| `logger` | `console` | Receives operational cache failures (`debug`, `warn`, `error`). | - -Per-invocation cache policy is a `DialCacheKeyConfig`: per-layer `ttlSec` and `ramp` maps keyed by `CacheLayer.LOCAL` (process-local) and `CacheLayer.REMOTE` (Redis), a `requestLocal` boolean, and an optional `remoteReadTimeoutMs`. +Enabled state follows the current asynchronous call chain through Node +`AsyncLocalStorage`; it is not process-global. Nested scopes restore the +previous state when their callbacks settle. -Every cached definition or `getOrLoad()` invocation can provide an optional per-use-case `defaultConfig`. It is the baseline policy, and the `cacheConfigProvider` result is a sparse field-level overlay on that baseline. For cache enablement fields, precedence is runtime config, then `defaultConfig`, then DialCache's disabled baseline. For the remote-read deadline, precedence is runtime `remoteReadTimeoutMs`, `defaultConfig.remoteReadTimeoutMs`, `redis.readTimeoutMs`, then the 50 ms library default. +`disable()` prevents cache access during its callback; it does not evict values +cached before a mutation. Use the appropriate invalidation or TTL policy before +serving later reads of mutable data. -The disabled baseline sets `requestLocal` to false and leaves the process-local and Redis TTLs unset. A shared layer with no effective TTL is disabled by policy. When a shared layer has an effective TTL but no effective ramp, its ramp defaults to 100%. +## Dial caching up or down -`DialCacheKeyConfig` preserves an omitted `requestLocal` as `undefined` so the overlay can distinguish omission from an explicit `false`; the effective value still defaults to false after resolution. - -A provider result of `null` (or defensive `undefined`) applies no overrides. An empty `DialCacheKeyConfig` and omitted runtime fields also inherit the baseline. Use explicit values to override inherited policy: `requestLocal: false` disables request-local caching and a layer ramp of `0` disables that shared layer. `DialCacheKeyConfig.disabled()` is that explicit kill switch in one call: request-local off and both shared layers ramped to 0. - -DialCache validates `defaultConfig` when `cached()` registers a definition and whenever `getOrLoad()` is invoked: TTLs and remote-read deadlines must be positive safe integers, ramps must be finite percentages from 0 to 100, layer maps must be objects, and `requestLocal` must be a boolean when present. Invalid defaults are rejected immediately. - -Each registration or one-shot invocation captures an immutable internal snapshot of `defaultConfig`; mutating the supplied config or its maps later does not change that operation's baseline. Runtime policy changes belong in the provider's returned overlay. - -Runtime TTL and ramp leaves are used as supplied instead of falling back to valid default leaves. An invalid TTL disables that layer with `invalid_ttl`; a non-finite or nonnumeric ramp disables it with `invalid_ramp`; finite runtime ramps retain the defensive clamp to 0–100. Other layers can still run, and invalid leaves also record a `config_resolution` error so provider garbage is alertable separately from intentional ramp-downs. A malformed runtime config object, layer-map shape, `requestLocal` value, or explicit `remoteReadTimeoutMs` fails config resolution for the invocation, records `config_error`, and executes the fallback uncached without attempting Redis. - -`cacheConfigProvider` is called for every enabled cache invocation before DialCache performs any cache lookup. Keep it cheap, cache any remote/config-store reads inside the provider, and avoid work that would erase the benefit of a cache hit. +Every cache operation can declare a stable `defaultConfig`. An optional +`cacheConfigProvider` returns a sparse runtime overlay for the current key, so +policy can change independently of the loader: ```ts import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; -const dialcache = new DialCache({ - cacheConfigProvider: async (key) => { - if (key.useCase === "GetUser") { - return new DialCacheKeyConfig({ - // Sparse override: inherit both TTLs and the local ramp from defaultConfig. - ramp: { [CacheLayer.REMOTE]: 25 }, - // Can be changed by the provider at runtime for this use case. - remoteReadTimeoutMs: 35, - }); - } - return null; // apply no overrides; use the cached function's baseline - }, -}); +const runtimePolicies = new Map(); -const getUser = dialcache.cached((userId: string) => db.fetchUser(userId), { - keyType: "user_id", - useCase: "GetUser", - cacheKey: (userId) => userId, - defaultConfig: new DialCacheKeyConfig({ - // Omitted ramps default to 100% because these layers have TTLs. - ttlSec: { [CacheLayer.LOCAL]: 30, [CacheLayer.REMOTE]: 300 }, - }), +const dialcache = new DialCache({ + cacheConfigProvider: (key) => runtimePolicies.get(key.useCase) ?? null, }); -``` - -`ramp` values are percentages from 0 to 100. `0` disables the layer, `100` enables it, and intermediate values are deterministically sampled by cache key and layer, so the same key is consistently sampled in or out of a partial rollout across calls and instances. The assignment algorithm is owned by DialCache and remains stable across releases. Applications that need an externally coordinated cohort can use `cacheConfigProvider` to return a sparse per-key ramp override of `0` or `100`. DialCache fetches and resolves one config snapshot per enabled invocation. Provider errors do not activate defaults: they fail open, record `config_error`, and execute the fallback function uncached. - -## Cache layers - -### Request-local cache - -Set `requestLocal: true` to memoize resolved values for the lifetime of the outermost `enable()` scope: -```ts -import { DialCache, DialCacheKeyConfig } from "dialcache"; - -const dialcache = new DialCache(); const getUser = dialcache.cached( (userId: string) => db.fetchUser(userId), { keyType: "user_id", useCase: "GetUser", cacheKey: (userId) => userId, - defaultConfig: new DialCacheKeyConfig({ requestLocal: true }), + defaultConfig: DialCacheKeyConfig.enabled(60), }, ); -``` - -`requestLocal` is a runtime boolean rather than a TTL/ramp-controlled `CacheLayer`. The `cacheConfigProvider` can turn it on or off for each invocation. `DialCacheKeyConfig.enabled(ttlSec)` enables only process-local and Redis caching, so request-local caching must be selected explicitly. - -DialCache resolves the runtime config once per enabled invocation and uses it for the entire lookup. When the effective `requestLocal` value is false, the invocation skips request-local lookup and storage without deleting an entry already memoized in the scope. A later invocation that enables request-local caching can reuse that entry. - -The outermost `enable()` call owns the request-local lifetime, and nested `enable()` calls reuse that scope. Request-local state is allocated lazily, only when an invocation enables the layer, so scopes that use only process-local or Redis caching do not allocate it. - -Wrap the complete Node HTTP handler so the request-local scope matches the handler's lifetime: - -```ts -import { createServer } from "node:http"; - -const server = createServer((req, res) => { - void dialcache - .enable(async () => { - const user = await getUser(readUserId(req)); - res.setHeader("content-type", "application/json"); - res.end(JSON.stringify(user)); - }) - .catch((error: unknown) => handleRequestError(error, res)); -}); -``` - -Request-local storage has no capacity limit, eviction, or overflow mode. Entries are retained until the outermost `enable()` callback settles. Use it for short-lived scopes with bounded key cardinality; split long-running streams or large batch jobs into smaller scopes when necessary. - -### Process-local cache - -The process-local layer (`CacheLayer.LOCAL`) uses one LRU per `DialCache` instance. It keeps at most 10,000 entries by default across all use cases while retaining each entry's configured TTL. Set `localMaxSize` to a nonnegative safe integer to change the global entry cap; `0` disables process-local storage: - -```ts -const dialcache = new DialCache({ localMaxSize: 25_000 }); -``` - -The limit counts entries rather than estimating JavaScript object memory. Recently read entries stay resident ahead of less recently used entries when the limit is reached. - -### Redis-backed TTL cache - -The Redis layer supports standalone Redis, Valkey, and Redis Cluster. Register DialCache's native node-redis scripts when creating the client, then pass that client to DialCache: - -```ts -import { createClient } from "redis"; -import { DialCache } from "dialcache"; -import { createNodeRedisDialCacheClient, dialcacheRedisScripts } from "dialcache/node-redis"; - -const redisClient = createClient({ - url: process.env.REDIS_URL, - scripts: dialcacheRedisScripts, - disableOfflineQueue: true, - commandsQueueMaxLength: 1_000, - socket: { connectTimeout: 2_000 }, -}); -await redisClient.connect(); - -const dialcache = new DialCache({ - namespace: "users-api", - redis: { - client: createNodeRedisDialCacheClient(redisClient), - // Optional instance default; omit to use DialCache's 50 ms default. - readTimeoutMs: 100, - }, -}); - -async function shutdown(): Promise { - // Stop new work and await every outstanding cached call and invalidation first. - await redisClient.quit(); -} -``` - -`redis.client` is required when Redis is configured and accepts the semantic `DialCacheRedisClient` interface. `redis.readTimeoutMs` is optional and sets the instance default for remote reads; omit it to use 50 ms. Create and connect the underlying client before constructing `DialCache`. Node-redis users should register the supplied scripts and wrap their client with `createNodeRedisDialCacheClient` as shown above. - -Valkey GLIDE users pass an already-created standalone or cluster client and its -module namespace to the GLIDE adapter: - -```ts -import * as valkeyGlide from "@valkey/valkey-glide"; -import { DialCache } from "dialcache"; -import { createValkeyGlideDialCacheClient } from "dialcache/valkey-glide"; - -const glideClient = await valkeyGlide.GlideClient.createClient({ - addresses: [{ host: "127.0.0.1", port: 6379 }], - requestTimeout: 2_000, - advancedConfiguration: { connectionTimeout: 2_000 }, -}); -const redisClient = createValkeyGlideDialCacheClient(glideClient, valkeyGlide); -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: redisClient }, -}); - -function shutdown(): void { - // After draining cached calls and invalidations, release scripts before closing GLIDE. - redisClient.dispose(); - glideClient.close(); -} -``` - -Pass the same module namespace that created the client. DialCache uses its -`Script` constructor and `Decoder.Bytes` value without importing a GLIDE runtime -itself, so linked workspaces and applications with another installed GLIDE -version cannot accidentally mix native script handles. - -The application owns the complete Redis lifecycle. It creates and connects the underlying client and passes the semantic adapter to DialCache. During shutdown, stop starting DialCache-backed work and await every promise returned by a cached function, `getOrLoad()`, or `invalidateRemote()`, including calls still running fallbacks that may later write Redis. A read that crossed DialCache's wait deadline may still be active inside the client, so use client-native telemetry and shutdown controls to drain or terminate that work before disposing adapter-owned resources and closing the connection. DialCache only borrows `redis.client`; it has no close or drain method and never disposes or closes caller resources. - -The node-redis adapter owns no additional resources, so the application closes the underlying node-redis client after draining work. The GLIDE adapter owns five native `Script` handles but not the wrapped connection. After outstanding operations finish, call its idempotent `dispose()` before closing GLIDE as shown above; disposal while an adapter operation is in flight throws rather than releasing a live script. - -Node-redis computes each script's SHA, uses `EVALSHA`, and retries with `EVAL` after `NOSCRIPT`. Its cluster client routes scripts by their first key and performs that fallback on the selected shard. The GLIDE adapter uses GLIDE's native `Script` lifecycle and byte decoder; GLIDE routes scripts from their declared keys. Tracked reads are deliberately routed to primaries so a lagging replica cannot hide an invalidation watermark. - -#### Remote read deadlines and async liveness - -DialCache bounds every active Redis read. The effective timeout is resolved per use case and per invocation: runtime `remoteReadTimeoutMs`, then `defaultConfig.remoteReadTimeoutMs`, then optional instance `redis.readTimeoutMs`, then 50 ms. Values must be positive safe integers no greater than 2,147,483,647. There is no unbounded escape hatch for remote reads. - -When the deadline expires, DialCache aborts the optional `RedisReadContext.signal`, records one `cache_read_timeout` error, logs a `RedisReadTimeoutError`, and starts the source fallback. Late read fulfillment or rejection is consumed and ignored. A read failure or timeout never triggers a post-fallback Redis write; an untracked active process-local miss may retain the source value, while a tracked key suppresses local publication because the failed read did not establish watermark safety. - -Same-key followers share the leader's remaining remote-read budget. The timer covers only the semantic Redis read, not config resolution, serializer load, fallback work, Redis writes, or invalidation. `fallbackTimeoutMs` starts separately when the source fallback begins. - -The bundled node-redis adapter passes the signal through per-command options, which can remove queued work where supported. Aborting after dispatch does not unsend a command or prove that Redis stopped executing it. GLIDE's current script API has no per-invocation signal, so its invocation may continue after DialCache has fallen back. Keep client-native connection, retry, queue, and response budgets in place; they bound underlying resource lifetime while DialCache's deadline bounds caller wait time. - -Writes, invalidations, async `cacheConfigProvider` calls, and custom serializer methods still need finite application-owned budgets. Do not put mutations behind a bare `Promise.race`: rejecting the outer promise neither removes queued work nor proves whether a dispatched mutation executed. - -#### Serialization - -The core Redis boundary is the client-agnostic `DialCacheRedisClient` interface. It exchanges serialized values as `string | Buffer` and does not expose client commands or wire encodings. Distinct untracked/tracked read and write Lua sources, the invalidation source, and wire constants are available from `dialcache/redis-protocol`. Custom adapters can throw the root-exported `DialCacheRedisPayloadError`, `DialCacheRedisPayloadEncodingError`, and `DialCacheRedisProtocolError` classes to distinguish malformed payloads, unsupported encodings, and Lua reply-domain violations in logs. DialCache records bounded `cache_read`, `cache_write`, or `invalidation` metrics by failure site. -Redis values use a compact binary frame: - -```text -byte 1 format version -bytes 2-9 Redis-created timestamp in milliseconds (uint64, big-endian) -byte 10 payload encoding (0 = UTF-8, 1 = raw binary) -bytes 11... serialized payload -``` - -Redis's Lua `struct` library packs and unpacks the timestamp. Redis TTL is authoritative, so expiry metadata is not duplicated in the frame. `payload` is produced by the operation's serializer, or by `JsonSerializer` by default. Custom serializers can return either `string` or `Buffer`; strings are stored as UTF-8 and Buffers are stored byte-for-byte without base64 expansion. Adapters restore the same representation before calling `serializer.load`. - -DialCache uses native `JSON.stringify` and `JSON.parse` by default. There is no runtime validation pass, so the default adds no traversal beyond JSON serialization itself. A top-level `undefined` result is supported with an internal sentinel. - -When `serializer.load` rejects a Redis payload, DialCache records a `serialization_load` error, counts the read as a remote cache miss, runs the fallback, and attempts to replace the rejected payload. A validating custom serializer can therefore treat an incompatible cached value as a refreshable miss without adding a schema version to the cache key. - -`JsonSerializer` validates JSON syntax only. It cannot detect that a structurally valid payload came from an incompatible application value schema. Applications that keep the same `useCase` across deployments must keep default-JSON values backward compatible. For an incompatible change, either provide a serializer whose `load` method validates and rejects the old shape, or change `useCase` to isolate the new cache entries. During a mixed deployment, mutually incompatible validating serializers can repeatedly reject and replace each other's values; correctness is preserved, but expect additional fallback and Redis-write load until the rollout converges. - -When a cached function or inline loader's resolved return type is statically JSON-compatible, `serializer` remains optional. This includes JSON primitives, arrays, plain object/interface shapes, optional object fields, and a top-level `undefined`. Types known not to survive the default round trip require a typed `Serializer`: - -```ts -import { DialCache, type Serializer } from "dialcache"; - -const dialcache = new DialCache(); -const dateSerializer: Serializer = { - dump: (value) => value.toISOString(), - load: (value) => new Date(Buffer.isBuffer(value) ? value.toString("utf8") : value), -}; - -const getUpdatedAt = dialcache.cached( - (userId: string) => db.fetchUpdatedAt(userId), - { - keyType: "user_id", - useCase: "GetUpdatedAt", - cacheKey: (userId) => userId, - serializer: dateSerializer, - }, +// Start with the local 10% ramp cohort; keep the remote layer off. +runtimePolicies.set( + "GetUser", + new DialCacheKeyConfig({ + ramp: { + [CacheLayer.LOCAL]: 10, + [CacheLayer.REMOTE]: 0, + }, + }), ); -``` - -The compile-time guard rejects known incompatible shapes such as `Date`, `Map`, `Set`, `bigint`, symbols, functions, Buffers, typed arrays, method-bearing class instances, required nested `undefined`, `unknown`, and `any`. It applies to every `cached()` declaration and `getOrLoad()` invocation because active layers are selected at runtime. A global Redis serializer is not parameterized by each returned type, so it cannot discharge this requirement; non-JSON operations must select a typed serializer. - -This guard is deliberately conservative and is not a proof of runtime data. TypeScript cannot detect non-finite numbers, cyclic/shared references, runtime getter or `toJSON` behavior, or data-only class instances that look like plain objects. Opaque, generic, or deeply recursive types may also require an explicit serializer. Providing `Serializer` (including an explicitly typed `JsonSerializer`) is a trusted caller assertion; DialCache does not serialize-and-deserialize again to validate it. - -## Cached-value ownership - -Treat values returned by cached functions or `getOrLoad()` as immutable. DialCache does not clone or freeze values stored in request-local or process-local memory. Mutating a cached object can therefore be observed by later callers in the same request, callers in other requests that hit the process-local cache, or callers that coalesced onto the same in-flight result. - -This contract includes nested objects and arrays, `Map`, `Set`, `Buffer`, typed arrays, and class instances. Redis deserialization can produce a different reference from an in-memory hit, so reference identity is layer-dependent and is not part of the API contract; never rely on a specific layer cloning a value before mutation. - -If a caller needs a mutable value, copy it explicitly before changing it: - -```ts -const sharedUser = await getUser("123"); -const editableUser = structuredClone(sharedUser); -editableUser.displayName = "New name"; -``` - -Use a narrower copy when its semantics are sufficient; the ownership boundary is the caller's responsibility. - -## Targeted invalidation and watermarks - -Mutable Redis-backed use cases can opt into targeted invalidation by setting `trackForInvalidation: true` in the options and calling `dialcache.invalidateRemote(keyType, id, futureBufferMs)` after writes. The buffer is an application-owned safety value; DialCache cannot choose a universally safe nonzero value: -```ts -import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; -import { createNodeRedisDialCacheClient } from "dialcache/node-redis"; - -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: createNodeRedisDialCacheClient(redisClient) }, -}); - -// Chosen from this application's clock-skew bound and measured worst-case source/fallback timings. -const USER_INVALIDATION_BUFFER_MS = 5_000; - -const getUser = dialcache.cached( - (userId: string) => db.fetchUser(userId), - { - keyType: "user_id", - useCase: "GetMutableUser", - cacheKey: (userId) => userId, - trackForInvalidation: true, - // Strongly invalidated mutable data should disable request-local and process-local caching. - defaultConfig: new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.REMOTE]: 300 }, - ramp: { [CacheLayer.REMOTE]: 100 }, - }), - }, +// Later, ramp both shared layers to 100%. +runtimePolicies.set( + "GetUser", + new DialCacheKeyConfig({ + ramp: { + [CacheLayer.LOCAL]: 100, + [CacheLayer.REMOTE]: 100, + }, + }), ); -await updateUser("123", patch); -await dialcache.invalidateRemote("user_id", "123", USER_INVALIDATION_BUFFER_MS); +// Reverse the rollout without changing getUser. +runtimePolicies.set("GetUser", DialCacheKeyConfig.disabled()); ``` -Invalidation writes a Redis watermark at `{encodedNamespace:encodedKeyType:encodedId}#watermark`. Tracked Redis cache entries use the same Redis Cluster hash tag, for example `{users-api:user_id:123}?locale=en#GetMutableUser:dialcache-frame-v1`, so the value key and watermark key live in the same slot. Key components are percent-encoded before joining so delimiters inside IDs or args cannot collide with delimiters in the key format. Components may not contain `{` or `}` because those characters would corrupt the hash tag. - -The internal `:dialcache-frame-v1` suffix identifies values written with DialCache's binary protocol. Watermarks are stored as decimal timestamps. +In production, the provider can read from an application-owned dynamic config +client instead of an in-memory map. DialCache resolves one policy snapshot per +enabled invocation. Keep the provider cheap and give any asynchronous work its +own finite budget. -A cached Redis value whose Redis-created timestamp is older than or equal to the watermark is treated as stale and refreshed through fallback. `invalidateRemote(keyType, id, futureBufferMs)` sets the watermark to the greater of its existing value and Redis's current time plus the buffer. While that future window is active, an invocation that reaches the tracked Redis read treats the covered value as a miss. If its fallback then reaches the tracked Redis write, Redis rejects the write and DialCache also suppresses the corresponding process-local population; the fallback value still returns to its caller. Request-local memoization remains unconditional, and invocations whose remote layer is disabled or ramped out do not consult the watermark and are not fenced by it. +For the process-local and remote layers: -The bundled timestamp protocol assumes that system clocks are synchronized across every Redis node eligible for primary promotion. Redis does not guarantee that `TIME` is monotonic across nodes, and DialCache does not detect or compensate for cross-node clock skew. If this deployment assumption is violated, failover can temporarily suppress tracked cache fills or allow a pre-invalidation value to remain readable until it expires or a later invalidation advances the watermark past its timestamp. +- a missing effective TTL disables that layer by policy; +- a configured TTL with no ramp defaults to `100`; +- `0` disables the layer; +- `100` enables the layer for every key; and +- an intermediate ramp uses DialCache's deterministic key-and-layer + assignment. -Watermarks are invalidation state, not disposable cache entries. The Redis deployment must preserve them for their derived TTL: use `noeviction` or an equivalent guarantee for deployments that rely on the publication fence, and choose persistence and failover guarantees appropriate to the application's consistency requirements. A missing watermark makes tracked reads miss, but a later tracked write cannot distinguish an empty cache from lost invalidation history; it creates a new baseline watermark and can publish fallback data that the lost future watermark would have rejected. Redis replication is asynchronous by default, and DialCache does not issue `WAIT` or provide strong consistency across failover. +Ramps select key cohorts, not requests or load, so `10` does not guarantee 10% +of calls. Increasing or decreasing a ramp preserves membership for keys that +remain inside the threshold, and local and remote cohorts are layer-specific. +DialCache keeps the assignment stable across releases. -Tracked writes create a baseline watermark and extend its TTL to at least the value TTL plus one minute. Neither tracked writes nor invalidation shorten a longer or persistent watermark TTL; invalidation extends it to at least the remaining future-buffer window plus one minute. There is no fixed watermark retention floor, and reads do not extend watermark lifetime. +If an application needs an externally coordinated cohort, its +`cacheConfigProvider` can return a per-key ramp override of `0` or `100`. +Ramping down, including with `DialCacheKeyConfig.disabled()`, bypasses existing +entries rather than deleting them; a later ramp-up can reuse entries that +remain valid. -`futureBufferMs` must be a nonnegative safe integer. The default is zero, but zero provides no stale-publication protection once Redis time advances. Every production invalidation should pass a named, application-owned nonzero value based on that application's measured or conservatively bounded timings; there is no universally safe library value. +Request-local caching is controlled separately by the `requestLocal` boolean. +`DialCacheKeyConfig.disabled()` sets it to `false` and ramps both shared layers +to `0`. Provider errors do not silently activate the baseline: the invocation +records a config error and runs the source loader uncached. -Size the buffer to cover the maximum expected negative clock skew between promotion-eligible Redis nodes plus the complete interval in which stale data could still reach the Redis write: source visibility or replication lag, the full remaining tail of any fallback that may already have observed the pre-mutation value, `serializer.dump`, Redis client queue and network latency, Lua script execution, the write itself, and a safety margin. Invalidate only after the source mutation commits. Underestimating this interval can allow a delayed stale fallback to repopulate Redis after the watermark window ends. Overestimating it lengthens the tracked Redis miss/write-suppression window described above, increasing fallback load without publishing stale values. A larger buffer does not delay or suppress returning fallback values to callers. +Remote-read waiting is runtime-controlled too. An overlay +`remoteReadTimeoutMs` takes precedence over the operation's `defaultConfig`, +then the instance's `redis.readTimeoutMs`, then the 50-millisecond core default. +Remote reads always have a finite positive deadline. -This is a timing contract rather than a cancellation or acquisition fence: the buffer prevents stale fallback results from passing that tracked Redis write only while the configured window remains active, and it does not force a fallback to read from an authoritative source. +See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) +for sparse-overlay precedence, validation, and layer behavior. -Targeted invalidation is remote-only and enforced by Redis watermarks. `invalidateRemote` does not evict existing request-local or process-local entries. Strongly invalidated mutable data should disable request-local and process-local caching (or use a very short process-local TTL only when stale reads are acceptable). +## How the read path works -## Request coalescing +Inside an enabled scope, active layers are checked in order: -DialCache coalesces in-flight work at the lifetime of the first active cache layer: - -- When request-local caching is enabled, same-key callers in one outermost `enable()` scope share request-scoped in-flight work before the request-local lookup. Its resolved value is then memoized for later sequential calls in that scope. -- When process-local or Redis caching is enabled, same-key callers share in-flight work within one `DialCache` instance before the first active shared layer. This is reported as `scope="process"`, still applies when request-local caching is off, and can combine leaders from separate request scopes using the same instance. - -```ts -await dialcache.enable(async () => { - // Same cold key, concurrent calls: one fallback execution, one shared result. - const [a, b] = await Promise.all([getUser("456"), getUser("456")]); -}); -``` - -With Redis configured, an instance-scoped leader that misses the process-local cache runs one bounded Redis read and, on a normal miss, the fallback/cache write; followers share its remaining read budget and await the same result. Process-local-only misses share the leader's fallback/cache write. This protects Redis and the source of truth from a thundering herd on hot keys. - -Coalescing only applies when at least one cache layer is active. Calls outside `enable()` are true pass-through. Calls where request-local, process-local, and Redis are all disabled are uncached and uncoalesced, but because they were initially enabled, the fallback deadline below still applies. - -Because coalescing is keyed by the selected or direct key, concurrent calls with the same key share the leader's execution. Any function argument or captured value omitted from the key must be safe to share this way; include inputs such as locale, auth context, or cancellation behavior when they can change the returned value or whether the underlying loader should run separately. - -### Fallback deadlines - -Once an initially enabled invocation starts its fallback, DialCache applies a 60-second monotonic deadline by default. Set `fallbackTimeoutMs` once on a cached wrapper or on each `getOrLoad()` invocation to choose a positive integer deadline in milliseconds, up to 2,147,483,647, or set it to `null` to preserve an intentionally unbounded fallback: - -```ts -import { FallbackTimeoutError } from "dialcache"; - -const getUser = dialcache.cached( - (userId: string) => db.fetchUser(userId), - { - keyType: "user_id", - useCase: "GetUserWithDeadline", - cacheKey: (userId) => userId, - defaultConfig: DialCacheKeyConfig.enabled(60), - fallbackTimeoutMs: 2_000, - }, -); - -try { - await dialcache.enable(() => getUser("123")); -} catch (error) { - if (error instanceof FallbackTimeoutError) { - logger.warn("source lookup exceeded its DialCache budget", { - useCase: error.useCase, - timeoutMs: error.timeoutMs, - }); - } -} -``` - -The timer starts only when the fallback begins, including after a remote-read deadline has elapsed. Same-key followers share the process or request-local leader's remaining budget and receive its `FallbackTimeoutError`; pass-through invocations where every layer is disabled have independent timers. Cache hits create no fallback timer. Calls that were initially outside an enabled context remain true pass-through and are not timed out, even when the operation configures `fallbackTimeoutMs`. - -Deadline delivery requires the JavaScript event loop to make progress. It cannot preempt a synchronous fallback prefix or other event-loop blocking, so rejection can arrive later than the configured duration; when control returns, DialCache checks the monotonic deadline before accepting the result. The deadline timer remains referenced until the fallback settles or times out. Consequently, an abandoned enabled fallback can keep an otherwise idle short-lived process alive until that deadline; shutdown code should drain outstanding DialCache work rather than discarding its promises. - -Timing out rejects the DialCache chain and clears its flight normally. A later fallback resolution is ignored, so that timed-out invocation cannot proceed to serializer, Redis, or local-cache publication. The underlying function is not canceled and may continue its own I/O or side effects; give the source operation its own native timeout or `AbortSignal` whenever possible. `fallbackTimeoutMs: null` disables this guard and makes finite fallback settlement entirely application-owned. Use the `null` escape hatch only after intentionally accepting that liveness risk. - -Timeout failures retain the bounded metrics classification `error="fallback"` with `in_fallback="true"`; the typed error provides the timeout details without adding high-cardinality labels. - -### Coalescing state - -`getCoalescingState()` returns a detached, point-in-time snapshot of process-scoped flights owned by that `DialCache` instance: - -```ts -const state = dialcache.getCoalescingState(); - -state.process.activeLeaders; -state.process.activeFollowers; -state.process.oldestLeaderAgeMs; // null when idle +```text +request-local -> process-local LRU -> Redis or Valkey -> source loader ``` -A leader is one exact cache key currently tracked by the instance-scoped coalescer. A follower is each later invocation that joined that pending leader; the initiating invocation is not counted as a follower. Followers remain counted until their leader settles because abandoning a JavaScript promise is not observable. Request-local flights are deliberately excluded because their lifecycle is bounded by the outer `enable()` scope. `oldestLeaderAgeMs` uses a monotonic clock and is computed when the snapshot is requested. +The wrapped function or inline loader is the fallback and remains the source of +the returned value when every active layer misses or a cache operation fails +open. -There is no library-wide flight cap or age-based replacement. A registry cap would bound only DialCache metadata while overflow or eviction could still create unbounded source work and unsafe duplicate publication. Finite operation deadlines provide eventual cleanup; application admission control and backpressure remain responsible for bounding simultaneous distinct-key work. Monitor leader count and oldest age to verify that those budgets hold in production. +- A request-local hit returns the value memoized in the current outermost + `enable()` scope. +- A process-local hit returns from the `DialCache` instance's bounded LRU. +- A process-local miss can read Redis and populate the process-local cache. +- A remote miss runs the fallback and attempts to populate active shared + layers. +- A remote read failure or timeout runs the fallback without a second Redis + operation. An untracked result may still populate process-local cache; a + tracked result does not, because watermark safety was not established. +- Same-key concurrent work is coalesced at the lifetime of the first active + layer. -## Metrics +When all layers are disabled by policy, an initially enabled call remains +uncached and uncoalesced, but its fallback deadline still applies. A call that +started outside an enabled scope remains a true pass-through and does not get a +DialCache deadline. -Metrics are disabled unless a `DialCacheMetricsAdapter` is passed to the constructor. `new DialCache()` does not import a metrics backend, register collectors, or emit metrics. +## Core concepts -### Prometheus +### Cache operations and keys -Install `prom-client` separately, create the registry your application owns, and pass the explicit Prometheus adapter to DialCache: +`cached(fn, options)` defines both a callable and the value-identity contract: -```bash -pnpm add prom-client@^15.1.3 -``` +| Option | Required | Purpose | +| --- | --- | --- | +| `keyType` | yes | Names the kind of id and, with `id`, the invalidation unit for tracked Redis entries. | +| `useCase` | yes | Identifies this individual cache in stored keys and metrics. | +| `cacheKey` | yes | Selects the bare id or `{ id, args }` from the function parameters. | +| `defaultConfig` | no | Supplies the baseline policy overlaid by runtime config. | +| `serializer` | for statically non-JSON return types | Defines the Redis representation for this operation's value. | +| `trackForInvalidation` | no | Opts the remote entries into watermark-based targeted invalidation. | +| `fallbackTimeoutMs` | no | Sets the fallback deadline; defaults to `60_000`, and `null` disables it. | + +Use `getOrLoad(load, options)` when a one-shot calculation should remain inline. +It follows the same cache, policy, coalescing, invalidation, serialization, and +deadline contracts, but takes a direct `key` instead of a `cacheKey` selector. +It does not register `useCase`, so repeated calls should reuse one stable, +deployment-defined name. + +The selected or direct key must include every input dimension that can affect +the returned value. Same-key concurrent calls may share the leader's execution, +so ignored function arguments or captured values such as auth context, locale, +or cancellation behavior must truly be safe to share. + +Set a stable, application-specific `namespace` when applications or +environments share Redis: ```ts -import { Registry } from "prom-client"; -import { DialCache } from "dialcache"; -import { createPrometheusDialCacheMetrics } from "dialcache/prometheus"; - -const registry = new Registry(); const dialcache = new DialCache({ - namespace: "users-api", - metrics: createPrometheusDialCacheMetrics({ - registry, - prefix: "myapp_", // myapp_dialcache_request_counter, etc. - }), -}); - -app.get("/metrics", async (_req, res) => { - res.type(registry.contentType).send(await registry.metrics()); + namespace: "production-users-api", + redis: { client: dialCacheRedisClient }, }); ``` -The adapter requires a caller-owned `Registry`; it never uses the global default registry and does not clear or otherwise own the registry lifecycle. Multiple adapters with the same registry and prefix reuse existing collectors when their type, help, labels, histogram buckets, and exemplar mode match. Adapter construction fails before registering anything if a same-name collector has an incompatible schema; use a unique prefix or a separate registry to resolve the collision. +See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) +for key encoding, secondary arguments, namespace changes, serializers, and +value ownership. -The Prometheus adapter emits: +### Cache layers -| Metric | Type | Labels | Description | -| --- | --- | --- | --- | -| `dialcache_request_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | -| `dialcache_miss_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | -| `dialcache_disabled_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips (`context`, `policy_disabled`, `invalid_ttl`, `invalid_ramp`, `ramped_down`, `config_error`) | -| `dialcache_error_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache/fallback errors classified by a bounded failure site | -| `dialcache_invalidation_counter` | Counter | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | -| `dialcache_coalesced_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests split by `request_local` or `process` scope | -| `dialcache_get_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | -| `dialcache_fallback_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the underlying function settles or timeout rejection is delivered | -| `dialcache_serialization_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency | -| `dialcache_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | +| Layer | Scope | Primary use | +| --- | --- | --- | +| Request-local | Outermost `enable()` scope | Memoize repeated reads during one bounded request or job. | +| Process-local | One `DialCache` instance | Serve hot values from a bounded in-process LRU. | +| Redis or Valkey | Shared remote store | Reuse TTL-cached values across processes and hosts. | -`policy_disabled` means that a process-local or Redis layer has no effective TTL after runtime overlays are applied. It is an intentional policy outcome, including the default when `defaultConfig` is omitted, rather than a configuration-loading failure. +Each invocation uses one resolved policy snapshot for all three layers. +Request-local storage has no capacity limit, so use it only for short-lived +scopes with bounded key cardinality. Process-local values count toward one +instance-wide entry cap. Remote values use a serializer selected by the cache +operation or the Redis configuration. -Every metric carries `cache_namespace`, including disabled-context, key-construction, coalescing, and invalidation paths that do not have a constructed key. Its value is `DialCacheConfig.namespace`, defaulting to `urn`. The `layer` label is `request_local`, `local` (process-local), or `remote`. Disabled-context, key-construction, and config-provider failures use `noop` because no cache layer was reached. The bounded `scope` label on `dialcache_coalesced_counter` distinguishes request-local from instance-scoped single-flight work. `scope="process"` coordinates calls only within one `DialCache` instance; separate instances in the same process do not share in-flight state. +Cached in-memory values are shared by reference. Treat every returned value as +immutable, or copy it explicitly before mutation. -### Datadog +### Targeted invalidation -Install `hot-shots` separately, create the DogStatsD client your application owns, and pass it to the Datadog adapter: - -```bash -pnpm add hot-shots@^17.0.0 -``` +Mutable Redis-backed use cases can opt into watermark-based invalidation with +`trackForInvalidation: true`, then call: ```ts -import StatsD from "hot-shots"; -import { DialCache } from "dialcache"; -import { createDatadogDialCacheMetrics } from "dialcache/datadog"; - -const dogStatsD = new StatsD({ - host: process.env.DD_AGENT_HOST, - globalTags: { service: "users-api", env: process.env.DD_ENV ?? "development" }, - errorHandler: (error) => logger.warn("DogStatsD error", { error }), -}); - -const dialcache = new DialCache({ - namespace: "users-api", // cache identity and cache_namespace tag - metrics: createDatadogDialCacheMetrics({ - client: dogStatsD, - observationMetricType: "distribution", - namespace: "dialcache", // metric-name prefix: dialcache.request.count, etc. - }), -}); - -// After outstanding cache operations finish during application shutdown: -dogStatsD.close(); -``` - -`hot-shots` is the supported and tested client, but the adapter depends only on the exported `DatadogDogStatsDClient` structural interface. DialCache does not import or install `hot-shots`, create a client, flush buffers, close sockets, or otherwise own the client lifecycle. - -`observationMetricType` is required. `"distribution"` is recommended when latency and size percentiles must aggregate across hosts; enable the desired distribution percentiles and aggregations in Datadog. Choose `"histogram"` when host-level histogram aggregation matches your existing Datadog setup. The choice applies uniformly to all four duration/size metrics. Both modes produce Datadog custom metrics. Distribution volume scales with unique tag-value combinations: Datadog counts five baseline aggregations per combination, and enabling percentile aggregations adds five more. Review [Datadog's custom-metrics billing guidance](https://docs.datadoghq.com/account_management/billing/custom_metrics/) before rollout. Do not send both types under the same namespace: when changing types, use a new namespace during migration so one metric identity never mixes histogram and distribution points. - -`DatadogMetricsOptions.namespace` is the metric-name namespace and defaults to `dialcache`. It is separate from `DialCacheConfig.namespace`, the logical cache namespace emitted as the `cache_namespace` tag. The Datadog metric namespace must start with a letter and contain only letters, numbers, underscores, and dot-separated non-empty segments. The adapter rejects invalid metric namespaces and final metric names longer than 200 characters rather than relying on client-side normalization. A `hot-shots` `prefix` is applied after the adapter constructs the name, so include that prefix when checking the final length and avoid combining it with the metric namespace accidentally. Client-level `globalTags` are appended by `hot-shots`; the table below lists the tags added by the adapter. - -The Datadog adapter emits exact increments of `1` for counters and preserves seconds and bytes without unit conversion: - -| Metric | Type | Tags | Description | -| --- | --- | --- | --- | -| `dialcache.request.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | -| `dialcache.miss.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | -| `dialcache.disabled.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | -| `dialcache.error.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache/fallback errors by bounded failure site | -| `dialcache.invalidation.count` | Count | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | -| `dialcache.coalesced.count` | Count | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests by sharing scope | -| `dialcache.get.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | -| `dialcache.fallback.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the underlying function settles or timeout rejection is delivered | -| `dialcache.serialization.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | -| `dialcache.serialization.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | - -Synchronous client throws are isolated by DialCache's fail-open metrics boundary. Buffered transport failures happen outside that synchronous call, so configure the DogStatsD client's error handling and shutdown behavior as part of application ownership. - -### Error categories - -The `error` label reports where an operation failed rather than copying the thrown value's class or `Error.name`: - -| `error` | Meaning | -| --- | --- | -| `key_construction` | The cache-key selector or `DialCacheKey` construction failed | -| `config_resolution` | Runtime or layer configuration, or ramp resolution, failed | -| `cache_read` | A local-cache or Redis read failed | -| `cache_read_timeout` | A Redis read exceeded its effective remote-read deadline | -| `cache_write` | A local-cache or Redis write failed | -| `serialization_load` | Deserializing a Redis payload failed | -| `serialization_dump` | Serializing a value for Redis failed | -| `invalidation` | Writing an invalidation watermark failed | -| `fallback` | The wrapped application function failed or exceeded its DialCache deadline | -| `unknown` | Reserved for an otherwise unclassified future failure site | - -These values are defined by the backend-neutral core and are identical for every metrics adapter. Raw thrown values, error names, messages, timeout values, cache IDs, arguments, and Redis keys are never included in metric labels. Operational errors are still passed to the configured logger where the existing failure path logs them. `in_fallback` remains the explicit cache-plumbing-versus-application distinction. - -### Custom adapters - -For other telemetry backends, implement `DialCacheMetricsAdapter` and pass the adapter through `new DialCache({ metrics })`. Every backend-neutral label object exposes the logical namespace as camel-case `cacheNamespace`; adapters should map it to their backend's `cache_namespace` label/tag. This field is present even when no key or cache layer was reached. Synchronous adapter failures are isolated from cache behavior and application fallbacks. Omit `metrics` to disable metrics. - -## Maintainers - -### Cache-path benchmark - -From a repository checkout, run the semantic microbenchmark after installing dependencies: - -```bash -pnpm benchmark:request-local +await updateUser("123", patch); +await dialcache.invalidateRemote( + "user_id", + "123", + USER_INVALIDATION_BUFFER_MS, +); ``` -The command builds `dist` before reporting six scenarios: sequential request-local hits, sequential process-local hits, enabled bounded fallbacks, request-local coalescing fan-out, process coalescing fan-out, and remote-read-deadline coalescing fan-out. The benchmark is a maintainer tool and is not included in the published package. It asserts fallback counts, coalescing state, timer cleanup, and returned values but deliberately applies no timing threshold. Override its work sizes with `DIALCACHE_BENCH_ITERATIONS` and `DIALCACHE_BENCH_FANOUT`. - -### Releasing - -Publishing starts by manually running the `Release` workflow from current `main`. After the package checks pass, Semantic Release selects the next version from Conventional Commits since the highest stable `vX.Y.Z` tag. Breaking changes bump major, `feat` bumps minor, and every other normal PR-title type (`fix`, `perf`, `docs`, `style`, `refactor`, `test`, `build`, `chore`, `ci`, and `revert`) bumps patch. The highest required bump wins. - -The workflow opens a `release: ` PR whose only change is the matching `package.json` version. `release` is a reserved Conventional Commit type configured not to request another release, so the version-control commit does not cause an extra bump. GitHub marks workflow runs for a PR opened with `GITHUB_TOKEN` as approval-required; approve those runs, review the PR, and squash-merge it normally through the protected branch. - -The merge triggers the publish job. Before any release side effect, it verifies current `main`, the release commit subject, the one-file diff, the package version, the absent tag, and Semantic Release's independently calculated version and commit. It then reruns the package checks and asks Semantic Release to create the matching Git tag, publish the public npm package with provenance, and publish the GitHub release. - -The repository must enable **Allow GitHub Actions to create and approve pull requests** under Actions workflow permissions. This workflow uses that capability only to create the version PR; it never approves or merges one, and no ruleset bypass actor or persistent release credential is required. +Invalidation is deliberately remote-only. It does not evict existing +request-local or process-local values, so strongly invalidated mutable data +should disable those layers or tolerate their TTL-bounded staleness. + +The buffer must be a named, application-owned nonzero value sized for clock +skew and the full stale-work window. See +[Targeted invalidation](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md) +before enabling it in production. + +### Request coalescing and fallback deadlines + +Concurrent callers with the same cache key share active work within the first +active cache scope: one outer request for request-local caching, or one +`DialCache` instance for the shared layers. This mitigates hot-key stampedes +inside that scope; it is not cross-process coordination. + +Same-key followers share the leader's remaining remote-read budget. The +fallback deadline starts separately only if and when the source loader begins. + +Enabled fallbacks have a 60-second monotonic deadline by default. Timing out +rejects the DialCache chain and prevents the late result from being published, +but it does not cancel the underlying function. Give source operations their +own native timeout or `AbortSignal`. + +See [Coalescing and fallback liveness](https://github.com/lan17/DialCache/blob/main/docs/coalescing.md) +for exact sharing, deadline, cleanup, and admission-control contracts. + +### Observability + +Metrics are disabled unless a `DialCacheMetricsAdapter` is supplied. First-party +adapters support caller-owned Prometheus registries and Datadog DogStatsD +clients. Bounded labels report layer requests, misses, disabled reasons, +coalescing scopes, serialization work, and cache versus fallback failures. + +See [Observability](https://github.com/lan17/DialCache/blob/main/docs/observability.md) +for installation, collector schemas, metric names, and custom adapters. + +## Production checklist + +Before ramping a use case: + +- enable DialCache only around read paths, and keep mutation paths inside + `disable()` or outside the enabled boundary; +- verify that every selected or direct key includes each value and execution + dimension that is unsafe to share; +- begin at `0` or a small deterministic key cohort, monitor source load, cache + errors, hit rate, latency, remote-read and fallback timeouts, and coalescing + state, then increase in controlled steps; +- keep a runtime path to `DialCacheKeyConfig.disabled()`; +- choose an effective DialCache remote-read deadline, and configure + resource-native budgets for the underlying Redis work, config providers, + serializers, and source operation; +- use a conservative `localMaxSize` and bounded request-local scopes; +- treat cached values as immutable; +- verify serializer compatibility across mixed application versions; and +- for tracked invalidation, synchronize promotion-eligible Redis clocks, + preserve watermark keys for their derived TTL with `noeviction` or an + equivalent guarantee, choose suitable persistence and failover behavior, and + size a nonzero buffer from measured or conservatively bounded timings. + +## Reference guides + +- [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) — definitions, keys, + runtime overlays, request-local and process-local behavior, and value + ownership. +- [Redis and Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) — node-redis and GLIDE setup, lifecycle, + liveness, binary protocol, and serialization. +- [Targeted invalidation](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md) — watermarks, Redis Cluster + placement, clock assumptions, and buffer sizing. +- [Coalescing and fallback liveness](https://github.com/lan17/DialCache/blob/main/docs/coalescing.md) — sharing scopes, + deadlines, state inspection, cleanup, and backpressure. +- [Observability](https://github.com/lan17/DialCache/blob/main/docs/observability.md) — Prometheus, Datadog, metric schemas, + error categories, and custom adapters. +- [Maintainer guide](https://github.com/lan17/DialCache/blob/main/docs/maintainers.md) — benchmarks and the protected release + workflow. + +DialCache is licensed under the +[MIT License](https://github.com/lan17/DialCache/blob/main/LICENSE). diff --git a/docs/coalescing.md b/docs/coalescing.md new file mode 100644 index 0000000..4ce4c34 --- /dev/null +++ b/docs/coalescing.md @@ -0,0 +1,220 @@ +# Coalescing and fallback liveness + +[Back to the README](../README.md) + +DialCache shares same-key in-flight work within the lifetime of the first active +cache layer. It applies a finite deadline to each active remote read and a +separate default deadline once an initially enabled invocation begins its +fallback loader. + +These mechanisms reduce duplicate source work and give active flights eventual +cleanup. They do not replace cross-process coordination, source-native +cancellation, application admission control, or backpressure. + +## Request coalescing + +DialCache has two sharing scopes. + +### Request-local scope + +When request-local caching is active, callers with the same key in one outermost +`enable()` scope share in-flight work before the request-local lookup. + +The resolved value is memoized for later sequential calls in that scope. A +different outer request has a different request-local flight registry. + +### Process scope + +When process-local or remote caching is active, same-key callers share work +within one `DialCache` instance before the first active shared layer. + +This is reported as `scope="process"`, but it is instance-scoped: + +- separate requests using the same `DialCache` instance can share; +- separate `DialCache` instances in one process do not share; and +- separate processes or hosts do not share. + +```ts +await dialcache.enable(async () => { + // Same cold key and active shared layer: + // one fallback execution, one shared result. + const [first, second] = await Promise.all([ + getUser("456"), + getUser("456"), + ]); +}); +``` + +With a remote layer configured, an instance-scoped leader that misses +process-local cache performs one bounded Redis read. Followers share that read +and its remaining deadline. On a remote miss, the leader runs the fallback and +cache write; followers await that result. + +For a process-local-only miss, followers share the leader's fallback and local +write. This mitigates a thundering herd on one hot key within the instance. + +## When calls do not coalesce + +Coalescing applies only when at least one cache layer is active: + +- calls that start outside `enable()` are true pass-through; +- initially enabled calls with every layer disabled are uncached and + uncoalesced; and +- process-scoped work is never shared across `DialCache` instances. + +An initially enabled all-disabled call still receives the fallback deadline +described below. + +Because coalescing is keyed by the full constructed cache key, concurrent calls +with the same identity share the leader's execution. Every function argument +or captured value omitted from the selected or direct key must be safe to share +this way. + +Include locale, auth context, cancellation behavior, or any other input in the +key when it can change: + +- the returned value; +- whether the underlying function should run independently; or +- whether two callers may safely share one result. + +## Fallback deadlines + +Once an initially enabled invocation begins its wrapped fallback, DialCache +applies a 60-second monotonic deadline by default. + +Set `fallbackTimeoutMs` on a cached wrapper or `getOrLoad()` invocation to +choose a positive integer deadline in milliseconds, up to 2,147,483,647. Set +it to `null` only when the application intentionally accepts an unbounded +fallback: + +```ts +import { FallbackTimeoutError } from "dialcache"; + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUserWithDeadline", + cacheKey: (userId) => userId, + defaultConfig: DialCacheKeyConfig.enabled(60), + fallbackTimeoutMs: 2_000, + }, +); + +try { + await dialcache.enable(() => getUser("123")); +} catch (error) { + if (error instanceof FallbackTimeoutError) { + logger.warn("source lookup exceeded its DialCache budget", { + useCase: error.useCase, + timeoutMs: error.timeoutMs, + }); + } + throw error; +} +``` + +### When the timer runs + +The timer starts only when the fallback begins: + +- same-key followers share the request-local or process leader's remaining + budget and receive its `FallbackTimeoutError`; +- a remote read failure or timeout starts the fallback timer only when the + source loader begins; +- enabled pass-through invocations where every layer is disabled have + independent timers; +- cache hits create no fallback timer; and +- calls that began outside an enabled context remain true pass-through and are + not timed out, even when the operation has `fallbackTimeoutMs`. + +The fallback deadline does not cover work that happens before fallback. An +active remote read has its own resolved +[remote-read deadline](redis.md#remote-read-deadlines-and-async-liveness), while +a pending config provider or serializer load does not. Serialization and a +Redis write after fallback also remain outside it. Give every injected +operation its own finite, resource-native budget. + +### Event-loop behavior + +Deadline delivery requires the JavaScript event loop to make progress. It +cannot preempt a synchronous fallback prefix or other event-loop blocking. +Rejection can therefore arrive later than the configured duration. + +When control returns, DialCache checks the monotonic deadline before accepting +the result. The timer remains referenced until the fallback settles or times +out. An abandoned enabled fallback can keep an otherwise idle short-lived +process alive until the deadline. + +Shutdown code should drain outstanding DialCache work rather than discarding +its promises. + +### Timeout does not cancel the source + +Timing out: + +1. rejects the DialCache chain; +2. clears its tracked flight normally; +3. ignores a later fallback resolution; and +4. prevents that invocation from proceeding to serializer, Redis, or local + publication. + +The underlying loader is not canceled and may continue its own I/O or side +effects. Give the source operation a native timeout or `AbortSignal` whenever +possible. + +`fallbackTimeoutMs: null` disables the guard and makes finite fallback +settlement entirely application-owned. Use that escape hatch only after +intentionally accepting the liveness risk. + +Timeout failures retain the bounded metrics classification +`error="fallback"` with `in_fallback="true"`. The typed error carries timeout +details without adding high-cardinality labels. + +A shared remote-read timeout emits one `cache_read_timeout` error for the +leader, not one per follower. + +## Inspecting process-scoped flights + +`getCoalescingState()` returns a detached, point-in-time snapshot of +process-scoped flights owned by one `DialCache` instance: + +```ts +const state = dialcache.getCoalescingState(); + +state.process.activeLeaders; +state.process.activeFollowers; +state.process.oldestLeaderAgeMs; // null when idle +``` + +A leader is one exact cache key currently tracked by the instance-scoped +coalescer. A follower is each later invocation that joined that pending leader; +the initiating invocation is not counted as a follower. + +Followers remain counted until their leader settles because abandoning a +JavaScript promise is not observable. Request-local flights are deliberately +excluded because their lifecycle is bounded by the outer `enable()` scope. +`oldestLeaderAgeMs` uses a monotonic clock and is computed when the snapshot is +requested. + +## Admission control remains application-owned + +There is no library-wide flight cap or age-based replacement. + +A registry cap would bound only DialCache metadata. Overflow or eviction could +still create unbounded source work and unsafe duplicate publication. Finite +operation deadlines provide eventual cleanup; application admission control +and backpressure remain responsible for bounding simultaneous distinct-key +work. + +Monitor: + +- active leader count; +- active follower count; +- oldest leader age; +- remote-read timeout errors; +- fallback deadline errors; and +- source concurrency and saturation. + +Use those signals to verify that application budgets and admission control hold +under production load. diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..fbaf568 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,434 @@ +# Configuration and cache layers + +[Back to the README](../README.md) + +This guide covers reusable cached functions, one-shot inline loaders, cache +identity, runtime policy, request-local and process-local behavior, and +cached-value ownership. For the shared remote layer, see +[Redis and Valkey](redis.md). + +## Defining cache operations + +### Reusable cached functions + +`cached(fn, options)` wraps a function; the wrapped callable has the same +parameters and always returns a `Promise`. + +| Option | Required | Description | +| --- | --- | --- | +| `keyType` | yes | The kind of id the key addresses, such as `"user_id"`. Together with the id, this is the invalidation unit for tracked entries. | +| `useCase` | yes | Identifies the individual cache. It is part of the stored key and a metrics label. | +| `cacheKey` | yes | Selects a bare id or `{ id, args }` from `fn`'s parameters. | +| `defaultConfig` | no | Provides the `DialCacheKeyConfig` baseline that runtime config overlays field by field. | +| `serializer` | when the return type is not statically JSON-compatible | Selects a per-function `Serializer` for Redis values; see [Serialization](redis.md#serialization). | +| `trackForInvalidation` | no; default `false` | Opts this use case's Redis entries into watermark-based [targeted invalidation](invalidation.md). | +| `fallbackTimeoutMs` | no; default `60_000` | Sets the fallback deadline in milliseconds, up to 2,147,483,647. `null` disables it; see [Fallback deadlines](coalescing.md#fallback-deadlines). | + +`useCase` is validated when the function is registered. A duplicate within one +`DialCache` instance throws `UseCaseIsAlreadyRegisteredError`, and the internal +name `watermark` throws `UseCaseNameIsReservedError`. + +### One-shot inline loaders + +`getOrLoad(load, options)` runs one zero-argument synchronous or asynchronous +loader through the same cache layers, runtime policy, coalescing, invalidation, +metrics, serialization, and deadline behavior as `cached()`. Cache-plumbing +failures fall through to the loader; loader failures still reject and clear +their tracked flight: + +```ts +const profile = await dialcache.enable(() => + dialcache.getOrLoad( + async () => { + const user = await db.getUser(userId); + return renderProfile(user, locale); + }, + { + keyType: "user_id", + useCase: "BuildProfile", + key: { id: userId, args: { locale } }, + defaultConfig: DialCacheKeyConfig.enabled(60), + }, + ), +); +``` + +The options match `cached()` except that the direct `key` replaces the +`cacheKey` selector. `defaultConfig` and `fallbackTimeoutMs` are validated and +snapshotted for each invocation. Outside an enabled scope, DialCache calls +`load` directly without constructing a key or resolving runtime policy. + +`getOrLoad()` does not register its `useCase` or detect duplicates, but it still +rejects the reserved internal name `"watermark"`. + +Repeated calls should reuse one stable, deployment-defined name such as +`"BuildProfile"`. Never derive it from a user, request, id, or other +high-cardinality input because it is part of both cache identity and metrics +labels. Put those values in `key` instead. + +Every captured value that can change the result belongs in the bare id or +`{ id, args }` key. Concurrent same-key calls may share one caller's in-flight +loader and cached value, so all call sites for that identity must also agree on +value meaning and serialization. + +Prefer `cached()` for reusable loaders and `getOrLoad()` for calculations +intentionally local to one call site. + +## Keys, ids, and extra dimensions + +For `cached()`, the required `cacheKey` selector receives the wrapped +function's inferred parameters. `getOrLoad()` accepts the same bare id or +`{ id, args }` shape directly through `key`: + +```ts +const searchPosts = dialcache.cached( + (userId: string, page: number, filter: string) => + db.searchPosts(userId, page, filter), + { + keyType: "user_id", + useCase: "SearchPosts", + cacheKey: (userId, page, filter) => ({ + id: userId, + args: { page, filter }, + }), + defaultConfig: DialCacheKeyConfig.enabled(60), + }, +); + +await dialcache.enable(() => searchPosts("u1", 2, "active")); +``` + +The selected or direct key is the value-identity contract. It must include +every input dimension that can affect the returned value. Otherwise, distinct +calls can reuse the same cached value or share the same in-flight fallback +through request coalescing. + +### Namespace + +`DialCacheConfig.namespace` is the logical cache namespace and the first +component of every key. It defaults to `"urn"`, producing keys such as +`urn:user_id:123#GetUser`. + +Set a stable application-specific value when applications or environments may +share one Redis deployment: + +```ts +const dialcache = new DialCache({ + namespace: "production-users-api", + redis: { client: dialCacheRedisClient }, +}); +``` + +That produces Redis keys beginning with `production-users-api:...`, or +`{production-users-api:...}` for invalidation-tracked values. `namespace` is +DialCache's single cache-identity and key-partitioning setting. It participates +in request-local, process-local, Redis, coalescing, deterministic ramp, +invalidation, and metrics. + +A namespace may not contain `{` or `}` because DialCache reserves those +characters for Redis Cluster hash tags. + +### Identity rules + +- **`keyType` plus `id` is the invalidation unit for tracked Redis entries.** + `dialcache.invalidateRemote("user_id", "123", futureBufferMs)` writes one + watermark for that user. Any tracked Redis entry with the same `keyType` and + `id` is refreshed across all `args` variants when Redis is read. Untracked + entries do not consult the watermark. Invalidation does not evict existing + request-local or process-local entries. +- **`args` are part of the cache key.** Different arguments produce different + entries, but targeted invalidation is by id rather than by argument. +- **Scalar equality is string-based.** For matching surrounding dimensions: + - numeric `1`, string `"1"`, and bigint `1n` identify the same key; and + - argument values `null` and `"null"` match, `-0` matches `0`, and an + `undefined` argument is omitted. + + If a deployment changes the logical meaning represented by a scalar, change + an explicit identity dimension such as `keyType`, `useCase`, or an argument + name or value. +- **Non-key inputs still reach the loader.** A database handle can be a normal + function parameter ignored by `cacheKey` or a value captured by a + `getOrLoad()` loader. Concurrent same-key misses share the leader's + execution. Do not omit values such as `AbortSignal`, auth context, locale, or + other request-scoped inputs unless sharing one result is correct. +- **Methods need a receiver.** Pass `obj.method.bind(obj)` or + `(...args) => obj.method(...args)`; a bare `obj.method` reference loses + `this`. + +### Changing a namespace + +Changing `namespace` intentionally creates a cold-cache boundary across every +layer. Old and new keyspaces do not share Redis values or invalidation +watermarks. + +During an overlapping deployment, an invalidation handled by one version is +invisible to the other. The other version can continue serving a stale tracked +value until its value TTL expires. If remote invalidation correctness matters, +a normal rolling namespace change is unsafe. + +Use a coordinated no-overlap cutover, or an operational bridge that prevents +both versions from serving remote cache across mutations. For example, +temporarily disable and clear remote caching during the transition. After the +cutover, provision for fallback and refill load, and allow old Redis keys to +expire by TTL. + +## Runtime config and ramp controls + +Instance-wide behavior is set through the `DialCache` constructor: + +| `DialCacheConfig` option | Default | Description | +| --- | --- | --- | +| `namespace` | `"urn"` | Logical cache namespace and first key component. | +| `redis` | none | `{ client, readTimeoutMs?, serializer? }`; enables the [remote layer](redis.md). Remote reads default to a 50 ms deadline. | +| `localMaxSize` | `10_000` | Global process-local entry cap. `0` disables process-local storage. Must be a nonnegative safe integer. | +| `cacheConfigProvider` | none | Resolves runtime config per enabled invocation as a sparse overlay on the operation's `defaultConfig`; `null` applies no overrides. | +| `metrics` | disabled | A `DialCacheMetricsAdapter`; see [Observability](observability.md). | +| `logger` | `console` | Receives operational cache failures through `debug`, `warn`, and `error`. | + +Per-invocation policy is a `DialCacheKeyConfig`: per-layer `ttlSec` and `ramp` +maps keyed by `CacheLayer.LOCAL` and `CacheLayer.REMOTE`, a `requestLocal` +boolean, and an optional `remoteReadTimeoutMs`. + +### Baseline and overlay precedence + +Every cached definition or `getOrLoad()` invocation can provide an optional +per-use-case `defaultConfig`. That is the baseline policy. The +`cacheConfigProvider` result is a sparse field-level overlay on it. + +Enablement fields use this precedence: + +```text +runtime field -> defaultConfig field -> DialCache disabled baseline +``` + +The disabled baseline sets `requestLocal` to `false` and leaves the +process-local and remote TTLs unset. A shared layer with no effective TTL is +disabled by policy. A shared layer with an effective TTL but no effective ramp +defaults to a 100% ramp. + +The remote-read deadline has two additional fallbacks: + +```text +runtime remoteReadTimeoutMs + -> defaultConfig.remoteReadTimeoutMs + -> redis.readTimeoutMs + -> 50 ms +``` + +This value bounds how long DialCache waits for an active Redis or Valkey read. +It can be tuned per use case at runtime, but it cannot be disabled. + +`DialCacheKeyConfig` preserves an omitted `requestLocal` as `undefined`, so the +overlay can distinguish omission from an explicit `false`. Its effective value +still defaults to `false` after resolution. + +A provider result of `null`, or a defensive `undefined`, applies no overrides. +An empty `DialCacheKeyConfig` and omitted runtime fields also inherit the +baseline. + +Use explicit values to replace inherited policy: + +- `requestLocal: false` disables request-local caching; +- a shared-layer ramp of `0` disables that layer; and +- `DialCacheKeyConfig.disabled()` turns request-local off and ramps both shared + layers to `0`. + +### Validation and snapshots + +DialCache validates `defaultConfig` when `cached()` registers a definition and +whenever `getOrLoad()` is invoked: + +- TTLs must be positive safe integers; +- ramps must be finite percentages from 0 to 100; +- layer maps must be objects; +- `requestLocal` must be a boolean when present; and +- remote-read deadlines must be positive safe integers no greater than + 2,147,483,647 milliseconds. + +Invalid instance `redis.readTimeoutMs` values throw during `DialCache` +construction. Invalid defaults are rejected when `cached()` registers a +definition or `getOrLoad()` is invoked. `null`, zero, fractional, non-finite, +string, and larger timeout values are invalid; remote reads have no unbounded +escape hatch. + +Each registration or one-shot invocation captures an immutable internal +snapshot, so mutating the supplied config or its maps later does not change +that operation's baseline. Runtime policy changes belong in the provider's +returned overlay. + +Runtime TTL and ramp leaves are used as supplied rather than falling back to +valid default leaves: + +- an invalid TTL disables that layer with `invalid_ttl`; +- a nonnumeric or non-finite ramp disables it with `invalid_ramp`; +- a finite runtime ramp retains a defensive clamp to 0 through 100; and +- other valid layers can continue to run. + +Invalid leaves also record a `config_resolution` error, distinguishing provider +garbage from an intentional ramp-down. A malformed runtime config object, +layer-map shape, `requestLocal`, or `remoteReadTimeoutMs` value fails config +resolution for the whole invocation. DialCache records `config_resolution`, +marks the no-layer path `config_error`, and runs the fallback without a Redis +read or write. + +### Provider behavior + +`cacheConfigProvider` is called for every enabled cache invocation before any +cache lookup. Keep it cheap, cache remote or config-store reads inside the +provider, and give asynchronous work a finite application-owned deadline. + +DialCache fetches and resolves one config snapshot per enabled invocation. +Provider errors do not activate defaults: they fail open, record +`config_error`, and execute the fallback uncached. + +```ts +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; + +const dialcache = new DialCache({ + redis: { + client: dialCacheRedisClient, + readTimeoutMs: 75, + }, + cacheConfigProvider: async (key) => { + if (key.useCase === "GetUser") { + return new DialCacheKeyConfig({ + // Sparse override: inherit both TTLs and the local ramp. + ramp: { [CacheLayer.REMOTE]: 25 }, + // Per-use-case override of the instance's 75 ms read deadline. + remoteReadTimeoutMs: 35, + }); + } + return null; + }, +}); + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUser", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ + // Omitted ramps default to 100% because these layers have TTLs. + ttlSec: { + [CacheLayer.LOCAL]: 30, + [CacheLayer.REMOTE]: 300, + }, + }), + }, +); +``` + +Ramp values are thresholds from 0 to 100. `0` disables the layer, `100` enables +it for every key, and an intermediate value selects keys whose DialCache-owned +deterministic bucket for the full cache key and layer is below that threshold. + +For a fixed cache identity and layer, increasing a ramp only adds keys and +decreasing it only removes keys; it does not reshuffle existing membership. +Local and remote cohorts are layer-specific. + +Ramps select key cohorts, not requests or load, so a ramp of `10` does not +guarantee 10% of calls, especially for a small or skewed key population. +DialCache keeps the assignment stable across releases. + +Applications that need an externally coordinated cohort can use +`cacheConfigProvider` to return a sparse per-key ramp override of `0` or `100`. + +Ramping down bypasses affected entries; it does not evict them, so a later +ramp-up can reuse entries that remain valid. + +## Request-local cache + +Set `requestLocal: true` to memoize resolved values for the lifetime of the +outermost `enable()` scope: + +```ts +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUser", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ requestLocal: true }), + }, +); +``` + +`requestLocal` is a runtime boolean rather than a TTL/ramp-controlled +`CacheLayer`. The provider can turn it on or off for each invocation. +`DialCacheKeyConfig.enabled(ttlSec)` enables only process-local and remote +caching, so request-local caching must be selected explicitly. + +The resolved config applies to the whole invocation. When `requestLocal` is +false, the invocation skips request-local lookup and storage without deleting a +value already memoized in the scope. A later invocation that enables it can +reuse that value. + +The outermost `enable()` call owns the request-local lifetime; nested `enable()` +calls reuse the same scope. State is allocated lazily, so scopes that use only +process-local or remote caching do not allocate it. + +Wrap the complete Node HTTP handler so the scope matches the request: + +```ts +import { createServer } from "node:http"; + +const server = createServer((req, res) => { + void dialcache + .enable(async () => { + const user = await getUser(readUserId(req)); + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify(user)); + }) + .catch((error: unknown) => handleRequestError(error, res)); +}); +``` + +Request-local storage has no capacity limit, eviction, or overflow mode. Values +are retained until the outermost callback settles. Use it for short-lived +scopes with bounded key cardinality. Split long-running streams or large batch +jobs into smaller scopes. + +## Process-local cache + +The process-local layer, `CacheLayer.LOCAL`, uses one LRU per `DialCache` +instance. It keeps at most 10,000 entries by default across all use cases while +retaining each entry's configured TTL. + +Set `localMaxSize` to a nonnegative safe integer to change the global entry cap. +`0` disables process-local storage: + +```ts +const dialcache = new DialCache({ localMaxSize: 25_000 }); +``` + +The limit counts entries rather than estimating JavaScript object memory. +Recently read entries stay resident ahead of less recently used entries when +the limit is reached. + +## Cached-value ownership + +Treat values returned by cached functions or `getOrLoad()` as immutable. +DialCache does not clone or freeze values stored in request-local or +process-local memory. +Mutating a cached object can be observed by: + +- later callers in the same request; +- callers in other requests that hit the process-local cache; and +- callers that coalesced onto the same in-flight result. + +This contract includes nested objects and arrays, `Map`, `Set`, `Buffer`, typed +arrays, and class instances. Redis deserialization can produce a different +reference from an in-memory hit, so reference identity is layer-dependent and +is not part of the API contract. + +Copy a value explicitly before changing it: + +```ts +const sharedUser = await getUser("123"); +const editableUser = structuredClone(sharedUser); +editableUser.displayName = "New name"; +``` + +Use a narrower copy when its semantics are sufficient. The ownership boundary +remains the caller's responsibility. diff --git a/docs/invalidation.md b/docs/invalidation.md new file mode 100644 index 0000000..b853ec5 --- /dev/null +++ b/docs/invalidation.md @@ -0,0 +1,205 @@ +# Targeted invalidation + +[Back to the README](../README.md) + +DialCache can invalidate related Redis entries without scanning or enumerating +keys. The mechanism is opt-in, remote-only, and based on per-identity Redis +watermarks. + +Read this complete contract before using targeted invalidation for mutable +production data. Correctness depends on cache-layer policy, Redis clock +synchronization, and an application-owned timing buffer. + +## Configure a tracked use case + +Set `trackForInvalidation: true` on a Redis-backed cached function or +`getOrLoad()` operation. After the source mutation commits, call +`dialcache.invalidateRemote(keyType, id, futureBufferMs)`: + +```ts +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { client: dialCacheRedisClient }, +}); + +// Chosen from this application's clock-skew bound and measured +// worst-case source and fallback timings. +const USER_INVALIDATION_BUFFER_MS = 5_000; + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetMutableUser", + cacheKey: (userId) => userId, + trackForInvalidation: true, + // Strongly invalidated mutable data should not use in-memory layers. + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { + [CacheLayer.REMOTE]: 300, + }, + ramp: { + [CacheLayer.REMOTE]: 100, + }, + }), + }, +); + +await updateUser("123", patch); +await dialcache.invalidateRemote( + "user_id", + "123", + USER_INVALIDATION_BUFFER_MS, +); +``` + +The buffer is an application-owned safety value. DialCache cannot choose a +universally safe nonzero default. + +## Identity and Redis Cluster placement + +Invalidation writes a watermark at: + +```text +{encodedNamespace:encodedKeyType:encodedId}#watermark +``` + +Tracked Redis values use the same Redis Cluster hash tag. For example: + +```text +{users-api:user_id:123}?locale=en#GetMutableUser:dialcache-frame-v1 +``` + +The value and watermark therefore live in the same Redis Cluster slot. Key +components are percent-encoded before joining, so delimiters inside ids or +arguments cannot collide with delimiters in the key format. + +`namespace` may never contain `{` or `}`; tracked `keyType` and `id` values may +not contain them because those three components form the hash tag. `args` and +`useCase` are encoded outside the hash tag and may contain braces. + +The internal `:dialcache-frame-v1` suffix identifies values written with +DialCache's binary protocol. Watermarks are stored as decimal timestamps. + +`keyType` plus `id` is the invalidation unit. One watermark covers every tracked +`useCase` and `args` variant with that identity. Untracked values do not consult +it. + +## Read and write behavior + +A tracked Redis value whose Redis-created timestamp is older than or equal to +the watermark is treated as stale and refreshed through fallback. + +`invalidateRemote(keyType, id, futureBufferMs)` sets the watermark to the +greater of: + +- its existing value; and +- Redis's current time plus the buffer. + +While that future window is active: + +1. A tracked Redis read treats the covered value as a miss. +2. The invocation runs its fallback. +3. If that fallback reaches the tracked Redis write before the window ends, + Redis rejects the write. +4. DialCache also suppresses the corresponding process-local population. +5. The fallback value still returns to its caller. + +Request-local memoization remains unconditional. An invocation whose remote +layer is disabled or ramped out does not consult the watermark and is not +fenced by it. + +This is a timing contract, not a cancellation or acquisition fence. The buffer +blocks stale fallback results from passing the tracked Redis write only while +the configured window remains active. It does not cancel the fallback or force +it to read from an authoritative source. + +If a tracked remote read rejects or exceeds its deadline, DialCache cannot +establish watermark safety. It runs the fallback but skips both the Redis write +and process-local publication. This differs from a normal tracked miss, which +can attempt the fenced Redis write. Untracked fallbacks may still populate +process-local cache, and request-local memoization remains unconditional. + +## Redis clock contract + +The bundled timestamp protocol assumes synchronized system clocks across every +Redis node eligible for primary promotion. + +Redis does not guarantee that `TIME` is monotonic across nodes, and DialCache +does not detect or compensate for cross-node clock skew. If the assumption is +violated, failover can: + +- temporarily suppress tracked cache fills; or +- allow a pre-invalidation value to remain readable until it expires or a later + invalidation advances the watermark past its timestamp. + +Monitor and bound the maximum negative clock skew across all promotion-eligible +nodes. Include that bound when sizing `futureBufferMs`. + +## Watermark durability + +Watermarks are invalidation state, not disposable cache entries. Redis must +preserve each marker for its derived TTL with `noeviction` or an equivalent +guarantee. Choose persistence, restore, and failover behavior that matches the +application's consistency requirements. + +Losing a marker through eviction, failover, restore, or external deletion +removes its prior publication fence. A missing marker makes tracked reads miss, +but a later tracked write creates a new baseline and can publish data that a +lost future watermark would have rejected. + +Redis replication is asynchronous. DialCache does not issue `WAIT` and does not +provide strong consistency across failover. + +## Watermark lifetime + +Tracked writes create a missing baseline watermark and ensure its TTL is at +least the value TTL plus one minute. They never shorten a longer or persistent +watermark TTL. + +Invalidation ensures the TTL covers both the requested future buffer and any +still-future existing watermark, plus one minute. It also preserves a longer or +persistent TTL. There is no fixed or configurable retention floor, and reads do +not extend watermark lifetime. + +## Choosing `futureBufferMs` + +`futureBufferMs` must be a nonnegative safe integer. The API default is zero, +but zero provides no stale-publication protection once Redis time advances. + +Every production invalidation should pass a named, application-owned nonzero +value based on measured or conservatively bounded timings. Size it to cover: + +- maximum expected negative clock skew between promotion-eligible Redis nodes; +- source visibility or replication lag; +- the full remaining tail of any fallback that may already have observed the + pre-mutation value; +- `serializer.dump`; +- Redis client queue and network latency; +- Lua script execution; +- the Redis write itself; and +- a safety margin. + +Invalidate only after the source mutation commits. + +Underestimating the interval can allow a delayed stale fallback to repopulate +Redis after the watermark window ends. Overestimating it lengthens the tracked +Redis miss and write-suppression window, increasing fallback load without +publishing stale values. + +A larger buffer does not delay or suppress returning fallback values to +callers. + +## In-memory layers remain local + +Targeted invalidation is remote-only. `invalidateRemote` does not evict existing +request-local or process-local entries. + +Strongly invalidated mutable data should disable request-local and process-local +caching. A short process-local TTL is appropriate only when the application +explicitly accepts that bounded stale-read window. + +If those layers remain enabled, their existing values can be returned without +reaching the remote watermark. diff --git a/docs/maintainers.md b/docs/maintainers.md new file mode 100644 index 0000000..4da5075 --- /dev/null +++ b/docs/maintainers.md @@ -0,0 +1,76 @@ +# Maintainer guide + +[Back to the README](../README.md) + +## Cache-path benchmark + +From a repository checkout, install dependencies and run: + +```bash +corepack pnpm benchmark:request-local +``` + +The command builds `dist` before reporting six scenarios: + +- sequential request-local hits; +- sequential process-local hits; +- enabled bounded fallbacks; +- request-local coalescing fan-out; +- process coalescing fan-out; and +- Redis read-deadline coalescing. + +The benchmark is a maintainer tool and is not included in the published +package. It asserts fallback counts, coalescing state, returned values, and one +semantic read and one cleaned-up deadline timer for the remote coalescing +scenario. It deliberately applies no timing threshold. + +Override its work sizes with: + +- `DIALCACHE_BENCH_ITERATIONS`; and +- `DIALCACHE_BENCH_FANOUT`. + +## Releasing + +Publishing starts by manually running the `Release` workflow from current +`main`. + +After the package checks pass, Semantic Release selects the next version from +Conventional Commits since the highest stable `vX.Y.Z` tag: + +- breaking changes bump major; +- `feat` bumps minor; and +- every other normal PR-title type bumps patch. + +Patch types are `fix`, `perf`, `docs`, `style`, `refactor`, `test`, `build`, +`chore`, `ci`, and `revert`. The highest required bump wins. + +The workflow opens a `release: ` pull request whose only change is the +matching `package.json` version. `release` is a reserved Conventional Commit +type configured not to request another release, so the version-control commit +does not cause an extra bump. + +GitHub marks workflow runs for a pull request opened with `GITHUB_TOKEN` as +approval-required. Approve those runs, review the pull request, and squash-merge +it normally through the protected branch. + +The merge triggers the publish job. Before any release side effect, it verifies: + +- current `main`; +- the release commit subject; +- the one-file diff; +- the package version; +- the absent tag; and +- Semantic Release's independently calculated version and commit. + +It then reruns the package checks and asks Semantic Release to: + +1. create the matching Git tag; +2. publish the public npm package with provenance; and +3. publish the GitHub release. + +The repository must enable **Allow GitHub Actions to create and approve pull +requests** under Actions workflow permissions. + +The workflow uses that capability only to create the version pull request. It +never approves or merges one, and no ruleset bypass actor or persistent release +credential is required. diff --git a/docs/observability.md b/docs/observability.md new file mode 100644 index 0000000..ac08f1c --- /dev/null +++ b/docs/observability.md @@ -0,0 +1,261 @@ +# Observability + +[Back to the README](../README.md) + +Metrics are disabled unless a `DialCacheMetricsAdapter` is passed to the +constructor. `new DialCache()` does not import a metrics backend, register +collectors, or emit metrics. + +DialCache provides first-party adapters for Prometheus and Datadog. Both use +caller-created, caller-owned clients and preserve one backend-neutral set of +bounded labels. + +## Prometheus + +Install `prom-client` separately: + +```bash +pnpm add prom-client@^15.1.3 +``` + +Create the registry your application owns, then pass an explicit adapter to +DialCache: + +```ts +import { Registry } from "prom-client"; +import { DialCache } from "dialcache"; +import { createPrometheusDialCacheMetrics } from "dialcache/prometheus"; + +const registry = new Registry(); + +const dialcache = new DialCache({ + namespace: "users-api", + metrics: createPrometheusDialCacheMetrics({ + registry, + prefix: "myapp_", + }), +}); + +app.get("/metrics", async (_req, res) => { + res.type(registry.contentType).send(await registry.metrics()); +}); +``` + +The adapter requires a caller-owned `Registry`. It never uses the global +default registry, and it does not clear or otherwise own the registry +lifecycle. + +Multiple adapters with the same registry and prefix reuse existing collectors +when their type, help, labels, histogram buckets, and exemplar mode match. +Adapter construction fails before registering anything if a same-name +collector has an incompatible schema. Use a unique prefix or separate registry +to resolve a collision. + +### Prometheus metrics + +The names below exclude the optional caller-selected prefix: + +| Metric | Type | Labels | Description | +| --- | --- | --- | --- | +| `dialcache_request_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | +| `dialcache_miss_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | +| `dialcache_disabled_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | +| `dialcache_error_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache or fallback errors by bounded failure site | +| `dialcache_invalidation_counter` | Counter | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | +| `dialcache_coalesced_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests split by request-local or process scope | +| `dialcache_get_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | +| `dialcache_fallback_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the wrapped fallback settles or timeout rejection is delivered | +| `dialcache_serialization_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | +| `dialcache_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | + +The disabled reasons are: + +- `context`; +- `policy_disabled`; +- `invalid_ttl`; +- `invalid_ramp`; +- `ramped_down`; and +- `config_error`. + +`policy_disabled` means that a process-local or remote layer has no effective +TTL after runtime overlays. This is an intentional policy result, including the +default when `defaultConfig` is omitted, rather than a configuration-loading +failure. + +Every metric includes `cache_namespace`, even disabled-context, +key-construction, coalescing, and invalidation paths that do not have a +constructed key. Its value is `DialCacheConfig.namespace`, which defaults to +`urn`. + +The `layer` label is: + +- `request_local`; +- `local`, meaning process-local; +- `remote`; or +- `noop` for disabled-context, key-construction, and config-provider failures + where no cache layer was reached. + +The bounded `scope` label on `dialcache_coalesced_counter` distinguishes +`request_local` from `process`. `scope="process"` coordinates calls only within +one `DialCache` instance; separate instances in the same process do not share +in-flight state. + +## Datadog + +Install `hot-shots` separately: + +```bash +pnpm add hot-shots@^17.0.0 +``` + +Create the DogStatsD client your application owns, then pass it to the Datadog +adapter: + +```ts +import StatsD from "hot-shots"; +import { DialCache } from "dialcache"; +import { createDatadogDialCacheMetrics } from "dialcache/datadog"; + +const dogStatsD = new StatsD({ + host: process.env.DD_AGENT_HOST, + globalTags: { + service: "users-api", + env: process.env.DD_ENV ?? "development", + }, + errorHandler: (error) => + logger.warn("DogStatsD error", { error }), +}); + +const dialcache = new DialCache({ + namespace: "users-api", + metrics: createDatadogDialCacheMetrics({ + client: dogStatsD, + observationMetricType: "distribution", + namespace: "dialcache", + }), +}); + +// Drain outstanding cache operations before application shutdown. +dogStatsD.close(); +``` + +`hot-shots` is the supported and tested client, but the adapter depends only on +the exported `DatadogDogStatsDClient` structural interface. + +DialCache does not: + +- import or install `hot-shots`; +- create a client; +- flush buffers; +- close sockets; or +- otherwise own the client lifecycle. + +### Distribution or histogram + +`observationMetricType` is required. + +Choose `"distribution"` when latency and size percentiles must aggregate across +hosts. Enable the desired distribution percentiles and aggregations in +Datadog. + +Choose `"histogram"` when host-level histogram aggregation matches the existing +Datadog setup. The choice applies uniformly to all four duration and size +metrics. Both modes produce Datadog custom metrics. + +Distribution volume scales with unique tag-value combinations. Datadog counts +five baseline aggregations per combination; enabling percentile aggregations +adds five more. Review +[Datadog's custom-metrics billing guidance](https://docs.datadoghq.com/account_management/billing/custom_metrics/) +before rollout. + +Do not send both observation types under the same metric namespace. When +changing types, use a new namespace during migration so one metric identity +never mixes histogram and distribution points. + +### Datadog namespaces + +`DatadogMetricsOptions.namespace` is the metric-name namespace and defaults to +`dialcache`. It is separate from `DialCacheConfig.namespace`, the logical cache +namespace emitted as the `cache_namespace` tag. + +The Datadog metric namespace must: + +- start with a letter; +- contain only letters, numbers, underscores, and dot-separated non-empty + segments; and +- produce final metric names no longer than 200 characters. + +The adapter rejects invalid namespaces and overlong final names instead of +relying on client-side normalization. + +A `hot-shots` `prefix` is applied after the adapter constructs the name. Include +that prefix when checking final length, and avoid accidentally combining it +with the adapter namespace. Client-level `globalTags` are appended by +`hot-shots`; the table below lists only tags added by DialCache. + +### Datadog metrics + +The adapter emits exact increments of `1` for counters and preserves seconds +and bytes without unit conversion: + +| Metric | Type | Tags | Description | +| --- | --- | --- | --- | +| `dialcache.request.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | +| `dialcache.miss.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | +| `dialcache.disabled.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | +| `dialcache.error.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache or fallback errors by bounded failure site | +| `dialcache.invalidation.count` | Count | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | +| `dialcache.coalesced.count` | Count | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests by sharing scope | +| `dialcache.get.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | +| `dialcache.fallback.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the wrapped fallback settles or timeout rejection is delivered | +| `dialcache.serialization.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | +| `dialcache.serialization.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | + +Synchronous client throws are isolated by DialCache's fail-open metrics +boundary. Buffered transport failures happen outside that synchronous call. +Configure the DogStatsD client's error handling and shutdown behavior as part +of application ownership. + +## Error categories + +The `error` label reports the operation that failed instead of copying the +thrown value's class or `Error.name`: + +| `error` | Meaning | +| --- | --- | +| `key_construction` | The cache-key selector or `DialCacheKey` construction failed | +| `config_resolution` | Runtime or layer configuration validation or resolution failed | +| `cache_read` | A process-local read or non-timeout remote read failed | +| `cache_read_timeout` | A remote read exceeded its effective DialCache deadline | +| `cache_write` | A process-local or remote cache write failed | +| `serialization_load` | Deserializing a Redis payload failed | +| `serialization_dump` | Serializing a value for Redis failed | +| `invalidation` | Writing an invalidation watermark failed | +| `fallback` | The source loader failed or exceeded its DialCache deadline | +| `unknown` | Reserved for a future failure site that cannot be classified otherwise | + +These values are defined by the backend-neutral core and are identical for +every adapter. + +Remote-read timeouts use `layer="remote"` and `in_fallback="false"`. They are +errors rather than misses, and the remote get-duration observation includes +the wait. Coalesced followers do not multiply the timeout error. Deadline +details remain out of labels and are available on the logged +`RedisReadTimeoutError`. + +Raw thrown values, error names, messages, cache ids, arguments, and Redis keys +are never included in labels. Operational errors still reach the configured +logger. `in_fallback` remains the explicit distinction between cache plumbing +and application fallback failures. + +## Custom adapters + +Implement `DialCacheMetricsAdapter` and pass it through +`new DialCache({ metrics })` for another telemetry backend. + +Every backend-neutral label object exposes the logical namespace as camel-case +`cacheNamespace`. Map it to the backend's `cache_namespace` label or tag. This +field is present even when no key or cache layer was reached. + +Synchronous adapter failures are isolated from cache behavior and application +fallbacks. Omit `metrics` to disable metrics entirely. diff --git a/docs/redis.md b/docs/redis.md new file mode 100644 index 0000000..bd3f81d --- /dev/null +++ b/docs/redis.md @@ -0,0 +1,377 @@ +# Redis and Valkey + +[Back to the README](../README.md) + +DialCache's remote TTL layer supports standalone Redis, standalone Valkey, and +Redis Cluster. The application creates, connects, configures, drains, and closes +the underlying client. DialCache borrows a semantic `DialCacheRedisClient` and +does not own the connection lifecycle. + +## Install a client + +Choose one supported integration: + +```bash +# node-redis +pnpm add redis@~4.7.1 + +# or Valkey GLIDE +pnpm add @valkey/valkey-glide +``` + +## node-redis + +Register DialCache's native scripts when creating the client, connect it, and +pass the semantic adapter to `DialCache`: + +```ts +import { createClient } from "redis"; +import { DialCache } from "dialcache"; +import { + createNodeRedisDialCacheClient, + dialcacheRedisScripts, +} from "dialcache/node-redis"; + +const redisClient = createClient({ + url: process.env.REDIS_URL, + scripts: dialcacheRedisScripts, + disableOfflineQueue: true, + commandsQueueMaxLength: 1_000, + socket: { connectTimeout: 2_000 }, +}); + +await redisClient.connect(); + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { + client: createNodeRedisDialCacheClient(redisClient), + }, +}); + +async function shutdown(): Promise { + // Stop new work and await every cached call and invalidation first. + await redisClient.quit(); +} +``` + +`redis.client` is required when the remote layer is configured. Node-redis +users should register the supplied scripts and wrap the connected client with +`createNodeRedisDialCacheClient` as shown above. Active remote reads have a +50-millisecond DialCache deadline by default. Set `redis.readTimeoutMs` for an +instance-wide value or use `DialCacheKeyConfig.remoteReadTimeoutMs` for +per-use-case static and runtime policy. + +The adapter computes each script's SHA, uses `EVALSHA`, and retries with `EVAL` +after `NOSCRIPT`. Its cluster client routes scripts by their first key and +performs that fallback on the selected shard. Tracked reads are deliberately +routed to primaries so a lagging replica cannot hide an invalidation watermark. + +Deployments using tracked invalidation must also satisfy the +[watermark durability](invalidation.md#watermark-durability) contract. + +## Valkey GLIDE + +Pass an already-created standalone or cluster client and the exact module +namespace that created it: + +```ts +import * as valkeyGlide from "@valkey/valkey-glide"; +import { DialCache } from "dialcache"; +import { createValkeyGlideDialCacheClient } from "dialcache/valkey-glide"; + +const glideClient = await valkeyGlide.GlideClient.createClient({ + addresses: [{ host: "127.0.0.1", port: 6379 }], + requestTimeout: 2_000, + advancedConfiguration: { + connectionTimeout: 2_000, + }, +}); + +const redisClient = createValkeyGlideDialCacheClient( + glideClient, + valkeyGlide, +); + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { client: redisClient }, +}); + +function shutdown(): void { + // Drain cached calls and invalidations before releasing resources. + redisClient.dispose(); + glideClient.close(); +} +``` + +DialCache uses the supplied namespace's `Script` constructor and +`Decoder.Bytes` value without importing a GLIDE runtime itself. Passing the same +module namespace that created the client prevents linked workspaces or +applications with another installed GLIDE version from mixing native script +handles. + +The GLIDE adapter uses GLIDE's native script lifecycle and byte decoder. GLIDE +routes scripts from their declared keys. + +## Lifecycle ownership + +The application owns the complete Redis lifecycle: + +1. Create and connect the underlying client. +2. Construct the semantic DialCache adapter. +3. Pass that adapter as `redis.client`. +4. During shutdown, stop starting DialCache-backed work. +5. Await every outstanding cached-function, `getOrLoad()`, and + `invalidateRemote()` promise, including fallbacks that may still write + Redis. +6. Drain or terminate client-native Redis work that may have outlived + DialCache's remote-read wait. +7. Dispose adapter-owned resources. +8. Close the underlying connection. + +DialCache has no `close()` or drain method. It never disposes or closes caller +resources. + +The node-redis adapter owns no additional resources, so close the underlying +client after draining work. + +The GLIDE adapter owns five native `Script` handles but not the wrapped +connection. Call its idempotent `dispose()` after operations finish and before +closing GLIDE. Disposing while an adapter operation is in flight throws rather +than releasing a live script. A DialCache read timeout does not prove that the +client-side invocation has settled. + +## Remote-read deadlines and async liveness + +Every active semantic remote-read leader has a finite monotonic deadline. +DialCache uses this precedence for `cached()` and `getOrLoad()`: + +```text +runtime remoteReadTimeoutMs + -> defaultConfig.remoteReadTimeoutMs + -> redis.readTimeoutMs + -> 50 ms +``` + +Each explicit value must be a positive safe integer no greater than +2,147,483,647 milliseconds. Remote reads have no unbounded escape hatch. +Outside an enabled scope, on a local hit, or when remote policy is disabled or +ramped out, DialCache creates no remote-read timer. + +### Timeout and fail-open behavior + +When the deadline expires, DialCache: + +1. aborts the optional adapter signal; +2. logs a root-exported `RedisReadTimeoutError` carrying `useCase` and + `timeoutMs`; +3. records `cache_read_timeout`; +4. consumes and ignores any late read fulfillment or rejection; and +5. runs the source fallback. + +The deadline bounds caller wait and cache publication. It does not guarantee +server-side cancellation, and an event-loop-blocking operation cannot be +preempted. When control returns, DialCache still checks the monotonic deadline +before accepting the result. + +A remote read rejection or timeout does not count as a miss and never triggers +a second Redis operation. After fallback, an untracked key may still populate +an active process-local cache. A tracked key suppresses process-local +publication because watermark safety was not established. Request-local +memoization remains unconditional. + +Same-key callers in one request-local or process coalescing scope share the +leader's read, timer, and remaining budget. A later independent invocation may +start a new remote read even if the prior client operation is still settling. + +The `fallbackTimeoutMs` timer is separate and starts only if and when the source +loader begins. The remote-read timer covers neither config resolution, +serializer loading, the fallback, Redis writes, nor invalidation. + +### Custom-client read context + +The semantic boundary exposes an optional second argument: + +```ts +interface RedisReadContext { + readonly timeoutMs: number; + readonly signal: AbortSignal; +} + +interface DialCacheRedisClient { + read( + request: RedisReadRequest, + context?: RedisReadContext, + ): Awaitable; +} +``` + +The optional argument keeps existing one-argument custom clients structurally +compatible. Adapters should use the signal for cooperative cancellation where +their client supports it, but the core deadline remains authoritative when +they do not. + +The bundled node-redis adapter forwards the signal in per-command options. This +can remove queued work where supported, but aborting after dispatch cannot +unsend a command or prove that Redis stopped executing it. + +The GLIDE script API has no per-invocation signal, so a timed-out script +invocation may continue inside the adapter. Its configured +[`requestTimeout`](https://glide.valkey.io/languages/nodejs/api/interfaces/BaseClient.BaseClientConfiguration.html) +and +[`advancedConfiguration.connectionTimeout`](https://glide.valkey.io/languages/nodejs/api/interfaces/BaseClient.AdvancedBaseClientConfiguration.html) +still bound client-native work. + +### Native operation budgets + +DialCache's read deadline bounds its caller wait, not the complete lifetime of +the underlying client work. Configure finite client-native budgets for: + +- connection establishment; +- reconnection and retries; +- offline queueing; +- dispatch; and +- response time. + +For node-redis 4.7, `socket.connectTimeout`, `disableOfflineQueue`, and +`commandsQueueMaxLength` bound connection or queue behavior but do not impose a +strict response deadline after dispatch. Use client-native shutdown or +termination behavior that matches the application's resource and ambiguity +requirements. + +Redis writes and invalidations, asynchronous `cacheConfigProvider` work, and +custom `Serializer` methods still require their own finite budgets. Do not put +writes or invalidations behind a bare `Promise.race`: rejecting the outer +promise neither removes queued work nor proves that a dispatched mutation did +not execute. + +DialCache's [fallback deadline](coalescing.md#fallback-deadlines) covers only the +source loader. Prefer resource-native budgets and cooperative cancellation for +every injected operation. + +## Serialization + +The core Redis boundary is the client-agnostic `DialCacheRedisClient` interface. +It exchanges serialized values as `string | Buffer` and does not expose +client-specific commands or wire encodings. + +Distinct untracked and tracked read/write Lua sources, the invalidation source, +and wire constants are exported from `dialcache/redis-protocol`. Custom adapters +can throw these root-exported error classes: + +- `DialCacheRedisPayloadError`; +- `DialCacheRedisPayloadEncodingError`; and +- `DialCacheRedisProtocolError`. + +They distinguish malformed payloads, unsupported encodings, and invalid Lua +reply domains in logs. DialCache records bounded `cache_read`, +`cache_read_timeout`, `cache_write`, or `invalidation` metrics by failure site. + +### Binary frame + +Redis values use a compact binary frame: + +```text +byte 1 format version +bytes 2-9 Redis-created timestamp in milliseconds (uint64, big-endian) +byte 10 payload encoding (0 = UTF-8, 1 = raw binary) +bytes 11... serialized payload +``` + +Redis's Lua `struct` library packs and unpacks the timestamp. Redis TTL is +authoritative, so expiry metadata is not duplicated in the frame. + +The payload comes from the cache operation's serializer or `JsonSerializer` by +default. Custom serializers can return `string` or `Buffer`. Strings are +stored as UTF-8; Buffers are stored byte-for-byte without base64 expansion. +Adapters restore the same representation before calling `serializer.load`. + +### Default JSON behavior + +DialCache uses native `JSON.stringify` and `JSON.parse` by default. There is no +runtime validation pass, so the default adds no traversal beyond JSON +serialization itself. A top-level `undefined` result is supported with an +internal sentinel. + +When `serializer.load` rejects a Redis payload, DialCache: + +1. records a `serialization_load` error; +2. counts the read as a remote miss; +3. runs the fallback; and +4. attempts to replace the rejected payload. + +A validating custom serializer can therefore treat an incompatible cached +value as a refreshable miss without adding a schema version to the cache key. + +`JsonSerializer` validates JSON syntax only. It cannot detect that a +structurally valid payload came from an incompatible application value schema. +Applications that retain one `useCase` across deployments must keep +default-JSON values backward compatible. + +For an incompatible change, either: + +- provide a serializer whose `load` method validates and rejects the old shape; + or +- change `useCase` to isolate the new cache entries. + +During a mixed deployment, mutually incompatible validating serializers can +repeatedly reject and replace each other's values. Correctness is preserved, +but expect additional fallback and Redis-write load until the rollout +converges. + +### Typed serializer requirement + +When a cached function or inline loader's resolved return type is statically +JSON-compatible, `serializer` is optional. This includes JSON primitives, +arrays, plain object or interface shapes, optional object fields, and a +top-level `undefined`. + +Types known not to survive the default round trip require a typed +`Serializer`: + +```ts +import { DialCache, type Serializer } from "dialcache"; + +const dialcache = new DialCache(); + +const dateSerializer: Serializer = { + dump: (value) => value.toISOString(), + load: (value) => + new Date(Buffer.isBuffer(value) ? value.toString("utf8") : value), +}; + +const getUpdatedAt = dialcache.cached( + (userId: string) => db.fetchUpdatedAt(userId), + { + keyType: "user_id", + useCase: "GetUpdatedAt", + cacheKey: (userId) => userId, + serializer: dateSerializer, + }, +); +``` + +The compile-time guard rejects known incompatible shapes such as: + +- `Date`, `Map`, and `Set`; +- `bigint`, symbols, and functions; +- Buffers and typed arrays; +- method-bearing class instances; +- required nested `undefined`; and +- `unknown` and `any`. + +The guard applies to every `cached()` declaration and `getOrLoad()` invocation +because active layers are selected at runtime. A global Redis serializer is not +parameterized by each returned type, so it cannot discharge this requirement. +Non-JSON operations must select a typed serializer. + +This guard is deliberately conservative rather than a proof of runtime data. +TypeScript cannot detect non-finite numbers, cyclic or shared references, +runtime getters, `toJSON` behavior, or data-only class instances that resemble +plain objects. Opaque, generic, or deeply recursive types may also require an +explicit serializer. + +Providing `Serializer`, including an explicitly typed +`JsonSerializer`, is a trusted caller assertion. DialCache does not perform +an additional serialize-and-deserialize cycle to validate it. From 183395bca67916a033f59ddc7bc1808ccbaf7416 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Sat, 25 Jul 2026 20:26:48 -0700 Subject: [PATCH 2/6] docs: improve onboarding and reference coverage --- README.md | 98 +++++++++++++++++++------------------------ docs/configuration.md | 68 ++++++++++++++++++++++++++++++ docs/maintainers.md | 37 ++++++++++++++++ docs/observability.md | 22 +++++++++- docs/redis.md | 44 ++++++++++++++----- 5 files changed, 201 insertions(+), 68 deletions(-) diff --git a/README.md b/README.md index e676737..43391e3 100644 --- a/README.md +++ b/README.md @@ -23,35 +23,14 @@ invalidation policy, and resource budgets. ## Safety comes from explicit controls -- **Off by default.** Outside `dialcache.enable(...)`, both cached wrappers and - inline loaders are true pass-throughs: DialCache does not build a key, - resolve config, access a cache, or coalesce the call. Inside an enabled - scope, a layer still needs an effective policy before it participates. -- **Gradual and reversible rollout.** Configure TTL and ramp independently for - the process-local and remote layers. A ramp of `0` is off, `100` is fully on, - and `DialCacheKeyConfig.disabled()` is the all-layer policy kill switch. -- **Fail-open cache path.** Key, config, cache-read, and serialization-load - failures fall through to the source loader. Cache-write, - serialization-dump, logging, and metrics failures do not replace an otherwise - usable fallback result. Explicit remote invalidation failures are rethrown so - callers never assume a mutation was made safe when it was not. -- **Bounded defaults.** The process-local cache has a 10,000-entry default cap, - active remote reads have a 50-millisecond default deadline, and enabled - fallback executions have a 60-second default deadline. The read deadline - bounds DialCache's wait, not necessarily the underlying Redis command; - applications still need resource-native budgets for client work, config - providers, serializers, and source I/O. - -Use DialCache when you want to: - -- add caching to database or service reads without scattering cache get/set - plumbing across call sites; -- begin with one layer or a small deterministic key cohort, observe it, and - expand or reverse the rollout per use case; -- combine request-local, process-local, and shared caching behind one key and - policy contract; or -- coalesce hot-key misses, invalidate related Redis entries, and emit bounded - cache metrics without rebuilding those mechanisms for every function. +- **Off by default.** Outside `dialcache.enable(...)`, calls go straight to the + loader without building a key, resolving policy, accessing a cache, or + coalescing work. +- **Gradual and reversible.** Start either shared layer at `0`, expand it by a + stable subset of keys, and turn every cache layer off through runtime policy. +- **Fail-open cache path.** Cache-plumbing failures fall through to the loader + instead of replacing a usable result. Explicit invalidation failures still + surface to the caller. ## Contents @@ -66,7 +45,7 @@ Use DialCache when you want to: ## Install ```bash -pnpm add dialcache +npm install dialcache ``` DialCache requires Node.js 22.0.0 or newer. Production deployments should use a @@ -80,6 +59,9 @@ clients application-owned: ## Quick start +Create one long-lived `DialCache` instance for each cache and coalescing domain, +typically once per service process: + ```ts import { DialCache, DialCacheKeyConfig } from "dialcache"; @@ -98,14 +80,18 @@ const getUser = dialcache.cached( // Outside enable(), this is a true pass-through to db.fetchUser: await getUser("123"); -// Inside enable(), the active cache layers participate: -const user = await dialcache.enable(() => getUser("123")); +// Inside enable(), the first call loads and the second reuses the cached value: +const user = await dialcache.enable(async () => { + await getUser("123"); // db.fetchUser, then populate process-local cache + return await getUser("123"); // process-local hit +}); ``` `cached(fn, options)` preserves the function's parameters and returns a Promise-based wrapper. `DialCacheKeyConfig.enabled(60)` gives the process-local -and remote layers a 60-second baseline TTL; the remote layer participates only -when a Redis or Valkey client is configured. +and remote layers a 60-second baseline TTL. It does not enable request-local +memoization, and the remote layer participates only when a Redis or Valkey +client is configured. For a one-shot calculation that should remain inline, [`getOrLoad()`](https://github.com/lan17/DialCache/blob/main/docs/configuration.md#one-shot-inline-loaders) @@ -133,11 +119,26 @@ previous state when their callbacks settle. cached before a mutation. Use the appropriate invalidation or TTL policy before serving later reads of mutable data. +### From local trial to production + +A typical adoption path is: + +1. start with the process-local cache shown above; +2. add [Redis or Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) + when values should be shared across processes or hosts; +3. add [Prometheus or Datadog](https://github.com/lan17/DialCache/blob/main/docs/observability.md) + before increasing production exposure; and +4. connect an application-owned runtime configuration source, then ramp a + stable subset of keys as described next. + ## Dial caching up or down Every cache operation can declare a stable `defaultConfig`. An optional `cacheConfigProvider` returns a sparse runtime overlay for the current key, so -policy can change independently of the loader: +policy can change independently of the loader. + +The example below focuses on runtime policy. Remote ramp settings take effect +only when a Redis or Valkey client is configured. ```ts import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; @@ -186,41 +187,28 @@ runtimePolicies.set("GetUser", DialCacheKeyConfig.disabled()); In production, the provider can read from an application-owned dynamic config client instead of an in-memory map. DialCache resolves one policy snapshot per -enabled invocation. Keep the provider cheap and give any asynchronous work its -own finite budget. +enabled invocation. -For the process-local and remote layers: - -- a missing effective TTL disables that layer by policy; -- a configured TTL with no ramp defaults to `100`; -- `0` disables the layer; -- `100` enables the layer for every key; and -- an intermediate ramp uses DialCache's deterministic key-and-layer - assignment. +A shared layer needs an effective TTL. With a TTL but no ramp, it defaults to +`100`; a ramp of `0` disables it, `100` selects every key, and an intermediate +value selects a stable key cohort for that layer. Ramps select key cohorts, not requests or load, so `10` does not guarantee 10% of calls. Increasing or decreasing a ramp preserves membership for keys that remain inside the threshold, and local and remote cohorts are layer-specific. DialCache keeps the assignment stable across releases. -If an application needs an externally coordinated cohort, its -`cacheConfigProvider` can return a per-key ramp override of `0` or `100`. Ramping down, including with `DialCacheKeyConfig.disabled()`, bypasses existing entries rather than deleting them; a later ramp-up can reuse entries that remain valid. Request-local caching is controlled separately by the `requestLocal` boolean. `DialCacheKeyConfig.disabled()` sets it to `false` and ramps both shared layers -to `0`. Provider errors do not silently activate the baseline: the invocation -records a config error and runs the source loader uncached. - -Remote-read waiting is runtime-controlled too. An overlay -`remoteReadTimeoutMs` takes precedence over the operation's `defaultConfig`, -then the instance's `redis.readTimeoutMs`, then the 50-millisecond core default. -Remote reads always have a finite positive deadline. +to `0`. See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) -for sparse-overlay precedence, validation, and layer behavior. +for sparse-overlay precedence, provider failure behavior, externally +coordinated cohorts, remote-read deadlines, and layer validation. ## How the read path works diff --git a/docs/configuration.md b/docs/configuration.md index fbaf568..5b42998 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -74,6 +74,39 @@ value meaning and serialization. Prefer `cached()` for reusable loaders and `getOrLoad()` for calculations intentionally local to one call site. +## Enable and disable scopes + +DialCache performs cache work only inside an enabled asynchronous scope. Create +each `DialCache` instance once and reuse it for the lifetime of its cache and +coalescing domain, typically one service process: + +| API | Behavior | +| --- | --- | +| `enable(fn)` | Enables caching for `fn` and the asynchronous work it awaits. The outermost call owns any request-local state. | +| `disable(fn)` | Temporarily restores pass-through behavior, commonly around nested mutation work. It does not evict existing values. | +| `isEnabled()` | Reports whether the current asynchronous call chain is inside a live enabled scope. | +| `withEnabled(fn)` | Exact alias for `enable(fn)`. | +| `withDisabled(fn)` | Exact alias for `disable(fn)`. | + +All five methods are instance-scoped. `enable()` and `disable()` always return a +`Promise`, including when their callback returns synchronously. Nested scopes +restore the previous state when their callbacks settle, and a nested +`enable()` inside `disable()` can opt a smaller read region back in. + +Enabled state follows Node's `AsyncLocalStorage`; it is not a process-global +flag. Once the outermost `enable()` callback settles, detached asynchronous work +that inherited the old context becomes pass-through and cannot repopulate its +closed request-local state. + +The root-exported `DialCacheContext` exposes the lower-level +`enable()`, `disable()`, and `isEnabled()` context primitive. It does not attach +itself to a `DialCache` instance or perform cache work. Most applications should +use the methods on `DialCache`. + +Keep mutation work outside the enabled boundary or inside `disable()`. Because +disabling does not evict existing values, mutable data still needs an +appropriate TTL or [targeted invalidation](invalidation.md) policy. + ## Keys, ids, and extra dimensions For `cached()`, the required `cacheKey` selector receives the wrapped @@ -337,6 +370,41 @@ Applications that need an externally coordinated cohort can use Ramping down bypasses affected entries; it does not evict them, so a later ramp-up can reuse entries that remain valid. +### Provider key input + +`cacheConfigProvider` receives the fully constructed, read-only `DialCacheKey` +for the invocation: + +| Field | Meaning | +| --- | --- | +| `namespace` | Logical application or environment namespace. | +| `keyType` and `id` | Primary identity. The selected id has already been converted to a string. | +| `args` | Secondary dimensions as normalized, name-sorted string pairs; entries whose value was `undefined` are omitted. | +| `useCase` | Stable operation name used in cache identity and metrics. | +| `prefix` | Encoded identity prefix, including a Redis Cluster hash tag when invalidation tracking is enabled. | +| `urn` | Complete encoded cache identity, including arguments and `useCase`. | +| `defaultConfig` | The operation's snapshotted baseline policy, or `null`. | +| `serializer` | The operation-specific serializer, or `null`. | +| `trackForInvalidation` | Whether the operation uses remote watermark tracking. | + +Use the identity fields to select policy; do not derive policy names or metric +dimensions from unbounded user input. The provider result remains a sparse +overlay and must not mutate the key. + +Most applications do not construct keys directly. Custom integrations can use +the root exports: + +- `new DialCacheKey(init)` to build the same public key shape; +- `normalizeArgs(record)` to omit `undefined`, stringify scalar values, and + sort argument names; +- `invalidationPrefix(namespace, keyType, id)` to build the encoded tracked + identity; and +- `redisClusterHashTag(value)` to wrap a validated value in a Redis Cluster hash + tag. + +The namespace and hash-tag components reject `{` and `}` as described under +[Identity rules](#identity-rules). + ## Request-local cache Set `requestLocal: true` to memoize resolved values for the lifetime of the diff --git a/docs/maintainers.md b/docs/maintainers.md index 4da5075..7db9422 100644 --- a/docs/maintainers.md +++ b/docs/maintainers.md @@ -2,6 +2,43 @@ [Back to the README](../README.md) +## Validation + +Use the repository's pinned pnpm version through Corepack: + +```bash +corepack pnpm install --frozen-lockfile +corepack pnpm check +corepack pnpm test:integration +``` + +`pnpm check` runs strict typechecking, the unit suite with coverage, +bundles/declarations, and packed ESM/CJS consumer tests. The integration suite +uses Testcontainers and requires a working Docker-compatible container runtime +for Redis, Valkey, and Redis Cluster. + +CI runs development and integration checks on Node.js 24, then switches to the +declared minimum Node.js 22.0.0 to test the packed package. Keep the consumer +floor separate from the development runtime so a new dependency or emitted +syntax cannot silently raise the published requirement. + +Before changing a compatibility-sensitive surface, identify and extend the +corresponding packed, unit, and integration assertions: + +- package root and explicit adapter/protocol entry points; +- full cache-key identity, encoding, namespace behavior, and Redis Cluster hash + tags; +- deterministic partial-ramp assignment, which must not reshuffle cohorts + across releases; +- the binary Redis frame, Lua arguments and reply domains, tracked + read/write/invalidation semantics, and mixed-version serializer behavior; and +- bounded metrics names, labels, reasons, error categories, scopes, and units. + +When changing user-facing examples, parse TypeScript fences, validate local +files and anchors, verify that README repository links are absolute for npm +rendering, and inspect the packed README. The package ships `README.md` but not +`docs/`. + ## Cache-path benchmark From a repository checkout, install dependencies and run: diff --git a/docs/observability.md b/docs/observability.md index ac08f1c..599a530 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -15,7 +15,7 @@ bounded labels. Install `prom-client` separately: ```bash -pnpm add prom-client@^15.1.3 +npm install prom-client@^15.1.3 ``` Create the registry your application owns, then pass an explicit adapter to @@ -105,7 +105,7 @@ in-flight state. Install `hot-shots` separately: ```bash -pnpm add hot-shots@^17.0.0 +npm install hot-shots@^17.0.0 ``` Create the DogStatsD client your application owns, then pass it to the Datadog @@ -253,6 +253,24 @@ and application fallback failures. Implement `DialCacheMetricsAdapter` and pass it through `new DialCache({ metrics })` for another telemetry backend. +| Hook | Required | Value | +| --- | --- | --- | +| `request(labels)` | yes | One active cache-layer lookup. | +| `miss(labels)` | yes | One cache miss. | +| `disabled(labels)` | yes | One skipped layer or no-layer invocation with a bounded `reason`. | +| `error(labels)` | yes | One bounded failure site with `inFallback`. | +| `invalidation(labels)` | yes | One explicit remote invalidation call. | +| `coalesced(labels)` | no | One follower that joined request-local or process-scoped work. | +| `observeGet(labels, seconds)` | yes | Cache-read duration in seconds. | +| `observeFallback(labels, seconds)` | yes | Fallback duration in seconds. | +| `observeSerialization(labels, seconds)` | yes | Serializer dump/load duration in seconds. | +| `observeSize(labels, bytes)` | yes | Serialized remote payload size in bytes. | + +The root package exports `DialCacheMetricsAdapter` and every associated label, +reason, error-kind, layer, and scope type. All hooks are synchronous; adapters +that buffer or transmit asynchronously own that later lifecycle. Keep label +values bounded and preserve the seconds and bytes units shown above. + Every backend-neutral label object exposes the logical namespace as camel-case `cacheNamespace`. Map it to the backend's `cache_namespace` label or tag. This field is present even when no key or cache layer was reached. diff --git a/docs/redis.md b/docs/redis.md index bd3f81d..36c01ef 100644 --- a/docs/redis.md +++ b/docs/redis.md @@ -13,10 +13,10 @@ Choose one supported integration: ```bash # node-redis -pnpm add redis@~4.7.1 +npm install redis@~4.7.1 # or Valkey GLIDE -pnpm add @valkey/valkey-glide +npm install @valkey/valkey-glide ``` ## node-redis @@ -189,9 +189,9 @@ The `fallbackTimeoutMs` timer is separate and starts only if and when the source loader begins. The remote-read timer covers neither config resolution, serializer loading, the fallback, Redis writes, nor invalidation. -### Custom-client read context +### Custom-client contract -The semantic boundary exposes an optional second argument: +Custom adapters implement the complete client-agnostic semantic boundary: ```ts interface RedisReadContext { @@ -204,13 +204,26 @@ interface DialCacheRedisClient { request: RedisReadRequest, context?: RedisReadContext, ): Awaitable; + write(request: RedisWriteRequest): Awaitable; + invalidate(request: RedisInvalidationRequest): Awaitable; } ``` -The optional argument keeps existing one-argument custom clients structurally -compatible. Adapters should use the signal for cooperative cancellation where -their client supports it, but the core deadline remains authoritative when -they do not. +| Method | Required semantics | +| --- | --- | +| `read` | Return the serialized `string` or `Buffer`, or `null` for a miss. A tracked request includes `watermarkKey`; compare the value timestamp and watermark atomically. | +| `write` | Apply `cacheTtlMs` and record server time atomically. A tracked request includes `watermarkKey`; return `false` when the watermark rejects publication and `true` when the value was written. | +| `invalidate` | Advance `watermarkKey` monotonically to at least server time plus `futureBufferMs`, while preserving the required derived lifetime. Reject on failure. | + +`write()` returning `false` is a safe publication refusal, not an adapter error. +DialCache still returns the fallback value but skips the corresponding +process-local population. A thrown cache-write error fails open; a thrown +explicit invalidation error is rethrown to the caller. + +The optional `RedisReadContext` keeps existing one-argument readers +structurally compatible. Adapters should use its signal for cooperative +cancellation where their client supports it, but the core deadline remains +authoritative when they do not. The bundled node-redis adapter forwards the signal in per-command options. This can remove queued work where supported, but aborting after dispatch cannot @@ -256,9 +269,18 @@ The core Redis boundary is the client-agnostic `DialCacheRedisClient` interface. It exchanges serialized values as `string | Buffer` and does not expose client-specific commands or wire encodings. -Distinct untracked and tracked read/write Lua sources, the invalidation source, -and wire constants are exported from `dialcache/redis-protocol`. Custom adapters -can throw these root-exported error classes: +The `dialcache/redis-protocol` entry point exports the exact bundled protocol +building blocks: + +- `READ_CACHE_SCRIPT` and `READ_TRACKED_CACHE_SCRIPT`; +- `WRITE_CACHE_SCRIPT` and `WRITE_TRACKED_CACHE_SCRIPT`; +- `INVALIDATE_CACHE_SCRIPT`; and +- `REDIS_FRAME_VERSION`, `REDIS_ENCODING_UTF8`, and + `REDIS_ENCODING_BINARY`. + +The scripts implement the atomic read, publication, invalidation, server-time, +and derived-watermark-lifetime behavior required above. Custom adapters can +throw these root-exported error classes: - `DialCacheRedisPayloadError`; - `DialCacheRedisPayloadEncodingError`; and From 0a833b2241c05d9784ac92ddcaa70832f17ef961 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Sat, 25 Jul 2026 20:36:02 -0700 Subject: [PATCH 3/6] docs: refine rollout safety guidance --- README.md | 51 +++++++++++--------- docs/coalescing.md | 21 ++++---- docs/configuration.md | 4 +- docs/invalidation.md | 5 +- docs/observability.md | 6 ++- docs/redis.md | 110 ++++++++++++++++++++++-------------------- 6 files changed, 107 insertions(+), 90 deletions(-) diff --git a/README.md b/README.md index 43391e3..ce8e5de 100644 --- a/README.md +++ b/README.md @@ -26,8 +26,9 @@ invalidation policy, and resource budgets. - **Off by default.** Outside `dialcache.enable(...)`, calls go straight to the loader without building a key, resolving policy, accessing a cache, or coalescing work. -- **Gradual and reversible.** Start either shared layer at `0`, expand it by a - stable subset of keys, and turn every cache layer off through runtime policy. +- **Gradual and reversible.** Start the process-local or remote layer at `0`, + expand it by a stable subset of keys, and turn every cache layer off through + runtime policy. - **Fail-open cache path.** Cache-plumbing failures fall through to the loader instead of replacing a usable result. Explicit invalidation failures still surface to the caller. @@ -63,7 +64,7 @@ Create one long-lived `DialCache` instance for each cache and coalescing domain, typically once per service process: ```ts -import { DialCache, DialCacheKeyConfig } from "dialcache"; +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; const dialcache = new DialCache(); @@ -73,7 +74,9 @@ const getUser = dialcache.cached( keyType: "user_id", useCase: "GetUser", cacheKey: (userId) => userId, - defaultConfig: DialCacheKeyConfig.enabled(60), + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 60 }, + }), }, ); @@ -88,10 +91,9 @@ const user = await dialcache.enable(async () => { ``` `cached(fn, options)` preserves the function's parameters and returns a -Promise-based wrapper. `DialCacheKeyConfig.enabled(60)` gives the process-local -and remote layers a 60-second baseline TTL. It does not enable request-local -memoization, and the remote layer participates only when a Redis or Valkey -client is configured. +Promise-based wrapper. The configuration above enables only the process-local +layer with a 60-second TTL; its omitted ramp defaults to `100`. Request-local +memoization and the remote layer remain off. For a one-shot calculation that should remain inline, [`getOrLoad()`](https://github.com/lan17/DialCache/blob/main/docs/configuration.md#one-shot-inline-loaders) @@ -124,12 +126,12 @@ serving later reads of mutable data. A typical adoption path is: 1. start with the process-local cache shown above; -2. add [Redis or Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) - when values should be shared across processes or hosts; -3. add [Prometheus or Datadog](https://github.com/lan17/DialCache/blob/main/docs/observability.md) +2. add [Prometheus or Datadog](https://github.com/lan17/DialCache/blob/main/docs/observability.md) before increasing production exposure; and -4. connect an application-owned runtime configuration source, then ramp a - stable subset of keys as described next. +3. connect an application-owned runtime configuration source with the remote + ramp at `0`; then +4. add [Redis or Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) + and ramp a stable subset of keys as described next. ## Dial caching up or down @@ -170,7 +172,7 @@ runtimePolicies.set( }), ); -// Later, ramp both shared layers to 100%. +// Later, ramp the process-local and remote layers to 100%. runtimePolicies.set( "GetUser", new DialCacheKeyConfig({ @@ -189,9 +191,9 @@ In production, the provider can read from an application-owned dynamic config client instead of an in-memory map. DialCache resolves one policy snapshot per enabled invocation. -A shared layer needs an effective TTL. With a TTL but no ramp, it defaults to -`100`; a ramp of `0` disables it, `100` selects every key, and an intermediate -value selects a stable key cohort for that layer. +The process-local and remote layers each need an effective TTL. With a TTL but +no ramp, a layer defaults to `100`; a ramp of `0` disables it, `100` selects +every key, and an intermediate value selects a stable key cohort. Ramps select key cohorts, not requests or load, so `10` does not guarantee 10% of calls. Increasing or decreasing a ramp preserves membership for keys that @@ -203,8 +205,8 @@ entries rather than deleting them; a later ramp-up can reuse entries that remain valid. Request-local caching is controlled separately by the `requestLocal` boolean. -`DialCacheKeyConfig.disabled()` sets it to `false` and ramps both shared layers -to `0`. +`DialCacheKeyConfig.disabled()` sets it to `false` and ramps both the +process-local and remote layers to `0`. See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) for sparse-overlay precedence, provider failure behavior, externally @@ -226,11 +228,11 @@ open. `enable()` scope. - A process-local hit returns from the `DialCache` instance's bounded LRU. - A process-local miss can read Redis and populate the process-local cache. -- A remote miss runs the fallback and attempts to populate active shared +- A remote miss runs the fallback and attempts to populate the active cache layers. - A remote read failure or timeout runs the fallback without a second Redis - operation. An untracked result may still populate process-local cache; a - tracked result does not, because watermark safety was not established. + operation. Tracked invalidation adds a stricter + [publication rule](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md#read-and-write-behavior). - Same-key concurrent work is coalesced at the lifetime of the first active layer. @@ -324,8 +326,9 @@ before enabling it in production. Concurrent callers with the same cache key share active work within the first active cache scope: one outer request for request-local caching, or one -`DialCache` instance for the shared layers. This mitigates hot-key stampedes -inside that scope; it is not cross-process coordination. +`DialCache` instance when process-local or remote caching is active. This +mitigates hot-key stampedes inside that scope; it is not cross-process +coordination. Same-key followers share the leader's remaining remote-read budget. The fallback deadline starts separately only if and when the source loader begins. diff --git a/docs/coalescing.md b/docs/coalescing.md index 4ce4c34..ca7ffaa 100644 --- a/docs/coalescing.md +++ b/docs/coalescing.md @@ -7,9 +7,10 @@ cache layer. It applies a finite deadline to each active remote read and a separate default deadline once an initially enabled invocation begins its fallback loader. -These mechanisms reduce duplicate source work and give active flights eventual -cleanup. They do not replace cross-process coordination, source-native -cancellation, application admission control, or backpressure. +These mechanisms reduce duplicate source work. Their deadlines help flights +settle, but eventual cleanup still requires finite application-owned budgets +for every injected operation. They do not replace cross-process coordination, +source-native cancellation, admission control, or backpressure. ## Request coalescing @@ -26,7 +27,8 @@ different outer request has a different request-local flight registry. ### Process scope When process-local or remote caching is active, same-key callers share work -within one `DialCache` instance before the first active shared layer. +within one `DialCache` instance before the first active process-local or remote +layer. This is reported as `scope="process"`, but it is instance-scoped: @@ -36,7 +38,7 @@ This is reported as `scope="process"`, but it is instance-scoped: ```ts await dialcache.enable(async () => { - // Same cold key and active shared layer: + // Same cold key and active process-local or remote layer: // one fallback execution, one shared result. const [first, second] = await Promise.all([ getUser("456"), @@ -202,10 +204,11 @@ requested. There is no library-wide flight cap or age-based replacement. A registry cap would bound only DialCache metadata. Overflow or eviction could -still create unbounded source work and unsafe duplicate publication. Finite -operation deadlines provide eventual cleanup; application admission control -and backpressure remain responsible for bounding simultaneous distinct-key -work. +still create unbounded source work and unsafe duplicate publication. +DialCache's remote-read and fallback deadlines cover only those phases; +provider, serializer, and Redis-write settlement remains application-owned. +Admission control and backpressure remain responsible for bounding +simultaneous distinct-key work. Monitor: diff --git a/docs/configuration.md b/docs/configuration.md index 5b42998..20b21d1 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -235,8 +235,8 @@ runtime field -> defaultConfig field -> DialCache disabled baseline ``` The disabled baseline sets `requestLocal` to `false` and leaves the -process-local and remote TTLs unset. A shared layer with no effective TTL is -disabled by policy. A shared layer with an effective TTL but no effective ramp +process-local and remote TTLs unset. Either layer is disabled by policy when it +has no effective TTL. With an effective TTL but no effective ramp, that layer defaults to a 100% ramp. The remote-read deadline has two additional fallbacks: diff --git a/docs/invalidation.md b/docs/invalidation.md index b853ec5..1652293 100644 --- a/docs/invalidation.md +++ b/docs/invalidation.md @@ -161,8 +161,9 @@ watermark TTL. Invalidation ensures the TTL covers both the requested future buffer and any still-future existing watermark, plus one minute. It also preserves a longer or -persistent TTL. There is no fixed or configurable retention floor, and reads do -not extend watermark lifetime. +persistent TTL. The one-minute safety margin is fixed; there is no separate +configurable or global retention floor, and reads do not extend watermark +lifetime. ## Choosing `futureBufferMs` diff --git a/docs/observability.md b/docs/observability.md index 599a530..ca361d9 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -135,8 +135,10 @@ const dialcache = new DialCache({ }), }); -// Drain outstanding cache operations before application shutdown. -dogStatsD.close(); +function shutdown(): void { + // Drain outstanding cache operations before application shutdown. + dogStatsD.close(); +} ``` `hot-shots` is the supported and tested client, but the adapter depends only on diff --git a/docs/redis.md b/docs/redis.md index 36c01ef..8f0b52e 100644 --- a/docs/redis.md +++ b/docs/redis.md @@ -4,8 +4,8 @@ DialCache's remote TTL layer supports standalone Redis, standalone Valkey, and Redis Cluster. The application creates, connects, configures, drains, and closes -the underlying client. DialCache borrows a semantic `DialCacheRedisClient` and -does not own the connection lifecycle. +the underlying client. DialCache borrows a client-independent +`DialCacheRedisClient` adapter and does not own the connection lifecycle. ## Install a client @@ -22,7 +22,7 @@ npm install @valkey/valkey-glide ## node-redis Register DialCache's native scripts when creating the client, connect it, and -pass the semantic adapter to `DialCache`: +pass the DialCache-compatible adapter to `DialCache`: ```ts import { createClient } from "redis"; @@ -144,7 +144,7 @@ client-side invocation has settled. ## Remote-read deadlines and async liveness -Every active semantic remote-read leader has a finite monotonic deadline. +Every active remote-read leader has a finite monotonic deadline. DialCache uses this precedence for `cached()` and `getOrLoad()`: ```text @@ -191,7 +191,8 @@ serializer loading, the fallback, Redis writes, nor invalidation. ### Custom-client contract -Custom adapters implement the complete client-agnostic semantic boundary: +Custom adapters implement the complete client-independent read, write, and +invalidate contract: ```ts interface RedisReadContext { @@ -211,9 +212,9 @@ interface DialCacheRedisClient { | Method | Required semantics | | --- | --- | -| `read` | Return the serialized `string` or `Buffer`, or `null` for a miss. A tracked request includes `watermarkKey`; compare the value timestamp and watermark atomically. | -| `write` | Apply `cacheTtlMs` and record server time atomically. A tracked request includes `watermarkKey`; return `false` when the watermark rejects publication and `true` when the value was written. | -| `invalidate` | Advance `watermarkKey` monotonically to at least server time plus `futureBufferMs`, while preserving the required derived lifetime. Reject on failure. | +| `read` | Return the serialized `string` or `Buffer`, or `null` for a miss. For a tracked request, compare the value timestamp and watermark atomically; a missing watermark or a value at or behind it is a miss. | +| `write` | Apply `cacheTtlMs` and record server time atomically. For a tracked request, create a missing baseline, retain it for at least the value TTL plus one minute without shortening a longer or persistent lifetime, and return `false` when it rejects publication. Return `true` only when the value was written. | +| `invalidate` | Advance `watermarkKey` monotonically to at least server time plus `futureBufferMs`. Retain it long enough to cover that buffer and any still-future existing watermark, plus one minute, without shortening a longer or persistent lifetime. Reject on failure. | `write()` returning `false` is a safe publication refusal, not an adapter error. DialCache still returns the fallback value but skips the corresponding @@ -265,49 +266,10 @@ every injected operation. ## Serialization -The core Redis boundary is the client-agnostic `DialCacheRedisClient` interface. -It exchanges serialized values as `string | Buffer` and does not expose -client-specific commands or wire encodings. - -The `dialcache/redis-protocol` entry point exports the exact bundled protocol -building blocks: - -- `READ_CACHE_SCRIPT` and `READ_TRACKED_CACHE_SCRIPT`; -- `WRITE_CACHE_SCRIPT` and `WRITE_TRACKED_CACHE_SCRIPT`; -- `INVALIDATE_CACHE_SCRIPT`; and -- `REDIS_FRAME_VERSION`, `REDIS_ENCODING_UTF8`, and - `REDIS_ENCODING_BINARY`. - -The scripts implement the atomic read, publication, invalidation, server-time, -and derived-watermark-lifetime behavior required above. Custom adapters can -throw these root-exported error classes: - -- `DialCacheRedisPayloadError`; -- `DialCacheRedisPayloadEncodingError`; and -- `DialCacheRedisProtocolError`. - -They distinguish malformed payloads, unsupported encodings, and invalid Lua -reply domains in logs. DialCache records bounded `cache_read`, -`cache_read_timeout`, `cache_write`, or `invalidation` metrics by failure site. - -### Binary frame - -Redis values use a compact binary frame: - -```text -byte 1 format version -bytes 2-9 Redis-created timestamp in milliseconds (uint64, big-endian) -byte 10 payload encoding (0 = UTF-8, 1 = raw binary) -bytes 11... serialized payload -``` - -Redis's Lua `struct` library packs and unpacks the timestamp. Redis TTL is -authoritative, so expiry metadata is not duplicated in the frame. - -The payload comes from the cache operation's serializer or `JsonSerializer` by -default. Custom serializers can return `string` or `Buffer`. Strings are -stored as UTF-8; Buffers are stored byte-for-byte without base64 expansion. -Adapters restore the same representation before calling `serializer.load`. +DialCache uses `JsonSerializer` by default. A cache operation can select a +typed serializer, and `redis.serializer` supplies the instance default when an +operation does not select one. Serializers run only for remote reads and +writes; request-local and process-local values remain native references. ### Default JSON behavior @@ -397,3 +359,49 @@ explicit serializer. Providing `Serializer`, including an explicitly typed `JsonSerializer`, is a trusted caller assertion. DialCache does not perform an additional serialize-and-deserialize cycle to validate it. + +### Advanced wire protocol + +The core Redis boundary is the client-independent `DialCacheRedisClient` +interface. It exchanges serialized values as `string | Buffer` and does not +expose client-specific commands or wire encodings. + +The `dialcache/redis-protocol` entry point exports the exact bundled protocol +building blocks: + +- `READ_CACHE_SCRIPT` and `READ_TRACKED_CACHE_SCRIPT`; +- `WRITE_CACHE_SCRIPT` and `WRITE_TRACKED_CACHE_SCRIPT`; +- `INVALIDATE_CACHE_SCRIPT`; and +- `REDIS_FRAME_VERSION`, `REDIS_ENCODING_UTF8`, and + `REDIS_ENCODING_BINARY`. + +The scripts implement the atomic read, publication, invalidation, server-time, +and derived-watermark-lifetime behavior required above. Custom adapters can +throw these root-exported error classes: + +- `DialCacheRedisPayloadError`; +- `DialCacheRedisPayloadEncodingError`; and +- `DialCacheRedisProtocolError`. + +They distinguish malformed payloads, unsupported encodings, and invalid Lua +reply domains in logs. DialCache records bounded `cache_read`, +`cache_read_timeout`, `cache_write`, or `invalidation` metrics by failure site. + +#### Binary frame + +Redis values use a compact binary frame: + +```text +byte 1 format version +bytes 2-9 Redis-created timestamp in milliseconds (uint64, big-endian) +byte 10 payload encoding (0 = UTF-8, 1 = raw binary) +bytes 11... serialized payload +``` + +Redis's Lua `struct` library packs and unpacks the timestamp. Redis TTL is +authoritative, so expiry metadata is not duplicated in the frame. + +The payload comes from the cache operation's serializer or `JsonSerializer` by +default. Custom serializers can return `string` or `Buffer`. Strings are +stored as UTF-8; Buffers are stored byte-for-byte without base64 expansion. +Adapters restore the same representation before calling `serializer.load`. From 6502c7c82ac3d9c268dbc4a33eaf20212d9e2502 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Sat, 25 Jul 2026 20:43:07 -0700 Subject: [PATCH 4/6] docs: make rollout examples fail safe --- README.md | 23 ++++++++++++++++++----- docs/configuration.md | 7 ++++--- 2 files changed, 22 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index ce8e5de..c83595f 100644 --- a/README.md +++ b/README.md @@ -60,8 +60,9 @@ clients application-owned: ## Quick start -Create one long-lived `DialCache` instance for each cache and coalescing domain, -typically once per service process: +Most services create one long-lived `DialCache` instance and reuse it across +the process. It owns one process-local LRU and one process-coalescing scope; +create separate instances only when those resources should be isolated: ```ts import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; @@ -128,8 +129,8 @@ A typical adoption path is: 1. start with the process-local cache shown above; 2. add [Prometheus or Datadog](https://github.com/lan17/DialCache/blob/main/docs/observability.md) before increasing production exposure; and -3. connect an application-owned runtime configuration source with the remote - ramp at `0`; then +3. extend the policy with a remote TTL and a remote ramp of `0`, using an + application-owned runtime configuration source; then 4. add [Redis or Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) and ramp a stable subset of keys as described next. @@ -157,7 +158,16 @@ const getUser = dialcache.cached( keyType: "user_id", useCase: "GetUser", cacheKey: (userId) => userId, - defaultConfig: DialCacheKeyConfig.enabled(60), + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { + [CacheLayer.LOCAL]: 60, + [CacheLayer.REMOTE]: 60, + }, + ramp: { + [CacheLayer.LOCAL]: 0, + [CacheLayer.REMOTE]: 0, + }, + }), }, ); @@ -187,6 +197,9 @@ runtimePolicies.set( runtimePolicies.set("GetUser", DialCacheKeyConfig.disabled()); ``` +The zero-ramp baseline is the safety net: if the provider has no matching +entry, both layers remain off. + In production, the provider can read from an application-owned dynamic config client instead of an in-memory map. DialCache resolves one policy snapshot per enabled invocation. diff --git a/docs/configuration.md b/docs/configuration.md index 20b21d1..3ac1c44 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -76,9 +76,10 @@ intentionally local to one call site. ## Enable and disable scopes -DialCache performs cache work only inside an enabled asynchronous scope. Create -each `DialCache` instance once and reuse it for the lifetime of its cache and -coalescing domain, typically one service process: +DialCache performs cache work only inside an enabled asynchronous scope. Most +services create one instance and reuse it for the service process. Each +instance owns one process-local LRU and one process-coalescing registry; create +separate instances only to isolate those resources: | API | Behavior | | --- | --- | From e639a05d1857eda46a7e33395ac1d0193f060853 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Sat, 25 Jul 2026 20:54:16 -0700 Subject: [PATCH 5/6] docs: clarify read-through cache positioning --- README.md | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index c83595f..10a109b 100644 --- a/README.md +++ b/README.md @@ -4,31 +4,37 @@ [![Codecov](https://codecov.io/gh/lan17/DialCache/branch/main/graph/badge.svg)](https://codecov.io/gh/lan17/DialCache) [![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/lan17/DialCache/badge)](https://scorecard.dev/viewer/?uri=github.com/lan17/DialCache) -**Roll out backend caching like a feature—not a leap of faith.** +**Read-through caching with the controls production systems need.** -**DialCache is** a TypeScript library for caching database and service reads -inside Node.js backends. It routes reusable async functions and inline loaders -through one read-through path with request-local memoization, a bounded -in-process LRU, and optional Redis or Valkey caching. +DialCache is a TypeScript read-through caching library for asynchronous +database and service reads in Node.js. Wrap a reusable function with +`cached()` or keep a loader inline with `getOrLoad()`; when the active cache +layers miss, DialCache calls your loader and publishes the result to whichever +request-local, bounded process-local, and optional Redis or Valkey layers are +active. -The “dial” is per-use-case runtime control. Start with caching off, dial the -process-local and remote layers up for stable cohorts of keys, and dial them -back down without changing the loader. +Around that core path, DialCache provides patterns that high-scale services +otherwise have to build themselves: request coalescing, per-use-case runtime +policy, deterministic ramp-up and ramp-down, fail-open cache access, targeted +invalidation, serialization, deadlines, and backend-neutral metrics. -**DialCache is not** a frontend data cache, cache server, Redis or Valkey -client, or runtime configuration service. It supplies the cache path and -rollout controls; your application still decides what is safe to cache and -owns loader behavior, connections, runtime configuration, keys, TTLs, -invalidation policy, and resource budgets. +The “dial” is the runtime policy: start a use case at zero, expand local or +remote caching to stable key cohorts, and reverse the rollout without changing +the loader. + +DialCache is a backend application library—not a frontend data cache, cache +server, Redis or Valkey client, or configuration control plane. Your service +owns the loader, clients, dynamic configuration source, cache identity, TTLs, +invalidation windows, admission control, and resource budgets. ## Safety comes from explicit controls - **Off by default.** Outside `dialcache.enable(...)`, calls go straight to the loader without building a key, resolving policy, accessing a cache, or coalescing work. -- **Gradual and reversible.** Start the process-local or remote layer at `0`, - expand it by a stable subset of keys, and turn every cache layer off through - runtime policy. +- **Gradual and reversible.** Start process-local and remote ramps at `0`, + expand either to a stable key cohort, and turn every cache layer back off + through runtime policy. - **Fail-open cache path.** Cache-plumbing failures fall through to the loader instead of replacing a usable result. Explicit invalidation failures still surface to the caller. From b44c9de34fae1344b063a9e5a6d0e3b8fdfadfc1 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Sat, 25 Jul 2026 20:55:30 -0700 Subject: [PATCH 6/6] docs: make README opening easier to scan --- README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 10a109b..1b83fc6 100644 --- a/README.md +++ b/README.md @@ -6,12 +6,13 @@ **Read-through caching with the controls production systems need.** -DialCache is a TypeScript read-through caching library for asynchronous -database and service reads in Node.js. Wrap a reusable function with -`cached()` or keep a loader inline with `getOrLoad()`; when the active cache -layers miss, DialCache calls your loader and publishes the result to whichever -request-local, bounded process-local, and optional Redis or Valkey layers are -active. +DialCache is a TypeScript read-through caching library for async database and +service reads in Node.js. + +Wrap a reusable function with `cached()` or keep a loader inline with +`getOrLoad()`; when the active cache layers miss, DialCache calls your loader +and publishes the result to whichever request-local, bounded process-local, +and optional Redis or Valkey layers are active. Around that core path, DialCache provides patterns that high-scale services otherwise have to build themselves: request coalescing, per-use-case runtime