Add a repeatable performance and scale benchmark suite

[lan17]

Priority

P2 — Medium — shared measurement infrastructure for performance and scale work; not a correctness blocker by itself.

v0.14.0 grooming verification (2026-08-01)

The maintained harness now has 10 semantic scenarios: the original six plus tracked Redis hits with shadow omitted, tracked Redis hits with shadow ramped out, detached dark-shadow validation, and detached dark-shadow fill. It still runs one console-only sample with no warmup/repetitions, JSON artifact, environment capture, real Redis/payload matrix, allocation profile, or CI execution. The deadline timers column also still omits timers allocated by bounded fallbacks. Keep P2 as the shared evidence prerequisite for #42 and #43.

Historical baseline and current gap

DialCache has a checked-in benchmark command:

corepack pnpm benchmark:request-local

At the v0.11.0 baseline, the harness provided six useful semantic scenarios:

• sequential request-local hits;
• sequential process-local hits;
• enabled bounded fallbacks;
• request-local coalescing fan-out;
• process-scoped coalescing fan-out;
• remote-read-deadline coalescing fan-out.

The deadline scenario correctly asserts that 1,000 followers share one semantic Redis read and one leader timer.

Current evidence:

• Command:

  DialCache/package.json

  Lines 78 to 86 in e06d833

  	"scripts": {
  	"benchmark:request-local": "pnpm build && node scripts/benchmark-request-local.mjs",
  	"build": "tsup src/index.ts src/datadog.ts src/node-redis.ts src/prometheus.ts src/redis-protocol.ts src/valkey-glide.ts --format esm,cjs --dts --clean",
  	"check": "pnpm typecheck && pnpm test && pnpm build && pnpm test:package",
  	"typecheck": "tsc --noEmit",
  	"test": "vitest run --coverage",
  	"test:integration": "vitest run --config vitest.integration.config.ts",
  	"test:package": "node scripts/test-package.mjs",
  	"test:watch": "vitest"

• Harness: https://github.com/lan17/DialCache/blob/e06d833ba245706a499d66056fdad15dc1210b68/scripts/benchmark-request-local.mjs
• Current documentation: https://github.com/lan17/DialCache/blob/e06d833ba245706a499d66056fdad15dc1210b68/README.md#cache-path-benchmark

Timings remain informational. Output is a single console table and captures no environment metadata, repeated-sample distribution, real Redis/payload matrix, allocation profile, JSON artifact, comparable baseline, or regression budget.

The current deadline timers column is also narrower than its label suggests: it reports remote-read leader deadline timers, while the enabled bounded-fallback scenario allocates fallback deadline timers but reports zero. The improved harness should name and count these timer classes separately so an optimization cannot hide timer allocation behind an ambiguous zero.

Fresh directional evidence

A clean build of main@e06d833ba245706a499d66056fdad15dc1210b68 was probed under Node.js 22.22.0.

Five-round medians from the maintained benchmark:

Scenario	Median
50,000 sequential request-local hits	75.3 ms / 1.51 µs per call
50,000 sequential process-local hits	114.2 ms / 2.28 µs per call
50,000 enabled bounded fallbacks	179.3 ms / 3.59 µs per call

Focused repeated probes showed why comparable profiling is needed:

Component or path	Median
Full process-local hit	1,358.6 ns/op
Full request-local hit	978.3 ns/op
Direct resolved LRU read	36.7 ns/op
Key construction	224.3 ns/op
Resolved local-config async boundary	208.3 ns/op
Process-flight sequential bookkeeping	127.1 ns/op
Request-flight sequential bookkeeping	100.5 ns/op
performance.now()	16.8 ns/call

These figures are directional only. They were not produced by an environment-normalized committed harness and must not become regression thresholds.

Phase 1 — comparable core harness

• Preserve the existing semantic assertions.
• Add warmup plus repeated measured rounds.
• Emit machine-readable JSON alongside readable output.
• Record commit, Node version, platform, architecture, dependency versions, iteration counts, configuration, and Redis/Valkey versions when applicable.
• Report at least median and tail/distribution information across measured rounds.
• Cover 0-, 1-, and 20-argument request-local and process-local hits.
• Compare metrics disabled with a deterministic no-op adapter.
• Distinguish tracked and untracked key construction.
• Distinguish remote-read deadline timers from fallback deadline timers.
• Cover representative tracked/untracked Redis read and write paths across a small payload matrix.
• Provide documented CPU/allocation profiling commands or an optional profiling mode.
• Add a deterministic CI semantic smoke run without noisy timing gates.

Avoid adding a benchmarking dependency unless the existing Node runtime cannot provide stable repeated sampling and JSON output simply.

Phase 2 — issue-specific evidence

Add focused scenarios only when a concrete issue needs them:

• Characterize and guard watermark cardinality and Redis Cluster hot slots #42 owns watermark cardinality, ID skew, Redis-primary QPS, and cluster hot-slot evidence.
• Establish a local-hit performance budget and remove avoidable hot-path work #43 owns local-hit profiles and before/after budgets.
• Benchmark lower-copy Redis protocol alternatives for large payloads #37 owns lower-copy protocol alternatives.
• Add thresholded client-side compression for Redis payloads #60 owns compression payload/CPU tradeoffs.
• Make zero local capacity consistent and evaluate byte-aware limits #40 owns any resumed byte-aware local-capacity work.

Cluster skew, failure injection, compression, multi-megabyte payloads, and batch matrices should not inflate the universal first harness.

Acceptance criteria for Phase 1

• Reproducible JSON and human-readable output with captured environment and configuration.
• Warmup and repeated-sample summaries.
• Local hit/miss and representative tracked/untracked Redis coverage.
• Metrics-on/off and argument-count comparisons needed by Establish a local-hit performance budget and remove avoidable hot-path work #43.
• Separate, accurate counters for remote-read and fallback deadline timers.
• Deterministic semantic assertions, including coalescing and one-remote-read-timer-per-leader behavior.
• Lightweight CI semantic smoke without unstable performance thresholds.
• Baselines recorded before any budget or regression threshold is selected.