feat(capability-registry): opt-in read cache to cut redundant agent calls + LT guidance gaps - #399
Conversation
The discover→invoke surface has no memory, so nothing stops an agent re-fetching the same stable read several times in one task. Add an opt-in response cache: a capability declares `cache.ttlSec` and only then is its answer stored. Constraints keep it safe for the shared surface: - mode:read only, and only a complete 2xx answer is stored - keyed by the CALLING CREDENTIAL, so a reused registration can never serve one account's read to another (same reason credentialsFor is read per call, not captured) - a successful write to a product drops that product's cached reads - TTL- and size-bounded (256 entries) A cache hit returns the product's body unchanged with `cached:true`. Volatile reads (run status, active runs, reports) declare no cache and stay live. tm is unaffected until its index opts in.
…uidance gaps Mark the reads whose answer is stable within a task as cacheable: getLoadTest and metricsManifest (a test's config / metric catalogue) and the project and load-test listings; leave every run/report/trend/quota read live. Cuts the repeated getLoadTest / list calls seen in agent runs. Guidance the existing anti-refetch prose did not cover: - createLoadTest: testType (plu/blu/hybrid) and framework are required and never inferred — ask the user rather than guess; and to duplicate a test copy the full source config, nothing is inherited. - updateLoadTest: config fields replace wholesale — tags overwrite, they do not append; send the union. - estimate: a protocol run bills at full pod capacity + a 5-min floor, so a small run estimates higher than VUs×duration — expected, not an error.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Central YAML (base), Organization UI (inherited), Workspace UI (inherited) Review profile: ASSERTIVE Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
|
Closing in favor of #400 — that PR keeps the change index-only (guidance edits to |
Why
Agent runs against the Load Testing capability index show the model re-fetching the same endpoint 3–9× in a single task (
getLoadTest×3, project/test listings repeated during id-resolution, etc.). The index already carries anti-refetch prose (see #394) and the model still repeats calls — because the genericdiscover → invokesurface has no memory: every re-derivation re-runssearchCapability→invokeEndpoint, and identical invokes are not deduplicated. Prose alone cannot fix this.This PR adds the missing layer — a small, opt-in response cache in the generic registry — and closes three Load-Testing guidance gaps that prose can fix.
What
1. Opt-in per-capability read cache (generic —
src/tools/capability-registry/)cache: { ttlSec }; only then is its answer cached. Absent = never cached (the safe default).mode:"read"only, and only a complete 2xx answer is stored.credentialsForis read per call, not captured) can never serve one account's read to another.cached:trueenvelope flag.tmis unaffected until its index opts in.2. Load Testing index (
capability/loadtesting.capability-index.json)getLoadTest(300s),metricsManifest(900s),listLoadTestProjects(300s),listLoadTests(300s).testType(plu/blu/hybrid) andframeworkare required and never inferred: ask the user, don't guess; to duplicate a test, copy the full source config (nothing is inherited).configfields replace wholesale;tagsoverwrite (they don't append) — send the union.Tests
tests/tools/capabilityRegistryCache.test.ts(8 cases): cache hit +cachedflag, arg-keying and order-independence, non-cacheable read always hits, write invalidation, credential isolation, non-2xx not cached, TTL expiry.tsc --noEmitclean.Notes for reviewers
user_permissiongate is model-satisfiable by design; and a cache reduces backend load/latency and nudges viacached:true, but the LLM still decides whether to emit a call — descriptions + this signal are the levers, not a hard stop.