Skip to content

feat(capability-registry): opt-in read cache to cut redundant agent calls + LT guidance gaps - #399

Closed
sourabhd-cbu wants to merge 2 commits into
browserstack:feat/capability-registryfrom
sourabhd-cbu:feat/capability-registry-read-cache
Closed

feat(capability-registry): opt-in read cache to cut redundant agent calls + LT guidance gaps#399
sourabhd-cbu wants to merge 2 commits into
browserstack:feat/capability-registryfrom
sourabhd-cbu:feat/capability-registry-read-cache

Conversation

@sourabhd-cbu

Copy link
Copy Markdown
Collaborator

Why

Agent runs against the Load Testing capability index show the model re-fetching the same endpoint 3–9× in a single task (getLoadTest ×3, project/test listings repeated during id-resolution, etc.). The index already carries anti-refetch prose (see #394) and the model still repeats calls — because the generic discover → invoke surface has no memory: every re-derivation re-runs searchCapabilityinvokeEndpoint, and identical invokes are not deduplicated. Prose alone cannot fix this.

This PR adds the missing layer — a small, opt-in response cache in the generic registry — and closes three Load-Testing guidance gaps that prose can fix.

What

1. Opt-in per-capability read cache (generic — src/tools/capability-registry/)

  • A capability declares cache: { ttlSec }; only then is its answer cached. Absent = never cached (the safe default).
  • Safety constraints for the shared surface:
    • mode:"read" only, and only a complete 2xx answer is stored.
    • Keyed by the calling credential — a reused registration (the remote host rebuilds config per session, which is why credentialsFor is read per call, not captured) can never serve one account's read to another.
    • A successful write to a product drops that product's cached reads for the writer.
    • TTL- and size-bounded (256 entries).
  • A cache hit returns the product's body unchanged, with a cached:true envelope flag.
  • tm is unaffected until its index opts in.

2. Load Testing index (capability/loadtesting.capability-index.json)

  • Mark the stable reads cacheable: getLoadTest (300s), metricsManifest (900s), listLoadTestProjects (300s), listLoadTests (300s).
  • Deliberately NOT cached (must stay live): run status, active runs, run report, insights, trends, quota, estimate, compare, project/test run history.
  • Guidance the existing anti-refetch prose didn't cover:
    • createLoadTesttestType (plu/blu/hybrid) and framework are required and never inferred: ask the user, don't guess; to duplicate a test, copy the full source config (nothing is inherited).
    • updateLoadTestconfig fields replace wholesale; tags overwrite (they don't append) — send the union.
    • estimate — a protocol (plu) run bills at full pod capacity + a 5-min floor, so a small run estimates higher than VUs×duration — expected, not an error.

Tests

  • New tests/tools/capabilityRegistryCache.test.ts (8 cases): cache hit + cached flag, arg-keying and order-independence, non-cacheable read always hits, write invalidation, credential isolation, non-2xx not cached, TTL expiry.
  • Full suite green: 497 passed; lint + tsc --noEmit clean.

Notes for reviewers

  • The generic cache and the LT index edits are split into two commits so the shared-infra change can be reviewed (or split into its own PR) independently.
  • Not addressed here (inherent, not fixable in this repo): the write-consent user_permission gate is model-satisfiable by design; and a cache reduces backend load/latency and nudges via cached:true, but the LLM still decides whether to emit a call — descriptions + this signal are the levers, not a hard stop.

The discover→invoke surface has no memory, so nothing stops an agent
re-fetching the same stable read several times in one task. Add an
opt-in response cache: a capability declares `cache.ttlSec` and only then
is its answer stored. Constraints keep it safe for the shared surface:

- mode:read only, and only a complete 2xx answer is stored
- keyed by the CALLING CREDENTIAL, so a reused registration can never
  serve one account's read to another (same reason credentialsFor is
  read per call, not captured)
- a successful write to a product drops that product's cached reads
- TTL- and size-bounded (256 entries)

A cache hit returns the product's body unchanged with `cached:true`.
Volatile reads (run status, active runs, reports) declare no cache and
stay live. tm is unaffected until its index opts in.
…uidance gaps

Mark the reads whose answer is stable within a task as cacheable:
getLoadTest and metricsManifest (a test's config / metric catalogue) and
the project and load-test listings; leave every run/report/trend/quota
read live. Cuts the repeated getLoadTest / list calls seen in agent runs.

Guidance the existing anti-refetch prose did not cover:
- createLoadTest: testType (plu/blu/hybrid) and framework are required and
  never inferred — ask the user rather than guess; and to duplicate a test
  copy the full source config, nothing is inherited.
- updateLoadTest: config fields replace wholesale — tags overwrite, they do
  not append; send the union.
- estimate: a protocol run bills at full pod capacity + a 5-min floor, so a
  small run estimates higher than VUs×duration — expected, not an error.
@coderabbitai

coderabbitai Bot commented Sep 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited), Workspace UI (inherited)

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 2494c430-b46a-42f6-bc99-118a201f7b2b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@sourabhd-cbu

Copy link
Copy Markdown
Collaborator Author

Closing in favor of #400 — that PR keeps the change index-only (guidance edits to loadtesting.capability-index.json). This PR's generic src/tools/capability-registry/ read-cache code is out of scope for an LT-index change; the cache/dedup will be a separate, generic mcp-server PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant