Skip to content

feat(loadtesting): add Load Testing capability index (regression) - #389

Open
sourabhd-cbu wants to merge 5 commits into
feat/capability-registryfrom
feat/loadtesting-capability-index-reg
Open

feat(loadtesting): add Load Testing capability index (regression)#389
sourabhd-cbu wants to merge 5 commits into
feat/capability-registryfrom
feat/loadtesting-capability-index-reg

Conversation

@sourabhd-cbu

@sourabhd-cbu sourabhd-cbu commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

What

The regression variant of the Load Testing capability index (base_url = https://load-testing-rengg-lts.bsstag.com), now updated to:

  1. Resolve the merge conflict against feat/capability-registry. The base advanced since this PR opened (prod index feat(loadtesting): add Load Testing capability index (prod) #388 merged, auth scheme feat(loadtesting): declare the HTTP Basic auth scheme in the index #391 merged). This branch now merges base cleanly and adopts the auth: { type: http, scheme: basic } block from feat(loadtesting): declare the HTTP Basic auth scheme in the index #391 — which resolves the earlier invoke-401 caveat (the registry now declares HTTP Basic).
  2. Add agent guidance that fixes the tool-call efficiency issues QA found on the agent surface (LTAI-270) — identical to the prod companion, differing only in base_url.

Guidance changes (guidance/summary text only; no endpoint or schema changes)

  • report: prefer narrow slices (metrics / errorCategory / slaOnly / sinceIso+untilIso / detail=per-txn) over refetching the full report.
  • getLoadTest / trends / run-history: one call returns the whole result — reuse within a task instead of re-fetching.
  • estimateLoadTestRunCost: always use it for cost/VU-hour math (models ramp-up) instead of computing by hand.
  • compareLoadTestRuns: use it for run deltas instead of diffing two reports.
  • getLoadTestRunStatus: a status question needs status only, not the report.
  • summary: state delete/scheduling are not offered (point to the dashboard) so the agent stops searching to confirm the gap.

Notes

Ref: LTAI-270.

@coderabbitai

coderabbitai Bot commented Sep 2, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited), Workspace UI (inherited)

Review profile: ASSERTIVE

Plan: Team

Run ID: adcc0c03-ecd3-498f-ad7c-5cf052cb90a5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

Adds capability/loadtesting.capability-index.json in the per-product format
(#387): product at top level, version field, and
schemas/responses/paging tables with per-capability response refs. 19
capabilities over load-api /api/v1/agent/* and 9 entities. base_url baked to
the regression host (load-testing-rengg-lts.bsstag.com) for this branch.
@sourabhd-cbu
sourabhd-cbu force-pushed the feat/loadtesting-capability-index-reg branch from 62b493c to fea275d Compare September 2, 2026 14:27
…rsion is an ISO token

BE (LTAI-270) now 400s scriptRef.source url/inline and reports version as the
JobConfiguration updated_at ISO token, not a numeric serialId. Align the
capability index so the LLM isn't mis-instructed.
…ity index

Companion to the existing project trends endpoint — GET /projects/{projectId}/runs
lets the LLM list a whole project's run history (TB: listLoadTestRuns 'for a test
or a whole project').
… trends

BE rejects weekly/monthly aggregation and ignores groupBy on the trends
endpoints, so stop advertising them to the LLM (avoids a guaranteed 400 on
aggregation=monthly and a silent no-op on groupBy).
…eat/loadtesting-capability-index-reg

# Conflicts:
#	capability/loadtesting.capability-index.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant