feat(loadtesting): close create/update/estimate/run guidance gaps - #400
Merged
sourabhd-cbu merged 2 commits intoSep 8, 2026
Conversation
Guidance the existing anti-refetch prose did not cover, all within the Load Testing capability index: - createLoadTest: testType (plu/blu/hybrid) and framework are required and never inferred — ask rather than guess; the load profile (VUs/vuRamp and durationSec) is required too and never defaulted — ask if unstated; and to duplicate a test copy the full source config, nothing is inherited. - startLoadTestRun: a non-dryRun run makes real load and burns VU-hours — dryRun first, show the estimate and the parameters, and start the real run only after the user confirms; the model's own confirmation is not the user's. - updateLoadTest: config fields replace wholesale — tags overwrite, not append; send the union. Tags live on the test, not its runs; to tag every test in a project, page listLoadTests until hasMore is false and update all. - estimate: a protocol (plu) run bills at full pod capacity + a 5-minute floor, so a small run estimates higher than VUs x duration — expected.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Central YAML (base), Organization UI (inherited), Workspace UI (inherited) Review profile: ASSERTIVE Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
…ks the "include-comma" 400) `include` and `errorCategory` are comma-separated multi-value params, but the index declared them as single-value enums. The generic bind/coerce validates the WHOLE value against the enum, so `include=config,thresholds` is rejected client- side with "must be one of ..." before it reaches the backend — the root cause of the QA "include-comma 400" (verified live through the beta MCP). It also forces one call per sub-resource (a driver of the getLoadTest re-fetch thrash). Drop the single-value enum on both; enumerate the options in the description (so the model still sees them) and nudge combining into one call. The backend already validates these values, so client-side validation isn't lost — it moves one hop.
sourabhd-cbu
merged commit Sep 8, 2026
efec81f
into
browserstack:feat/capability-registry
1 check passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Guidance-only edits to the Load Testing capability index (
capability/loadtesting.capability-index.json) — no code, index-only. Closes gaps the existing anti-refetch guidance (#394) did not cover.Changes
testType(plu/blu/hybrid) andframeworkare required and never inferred: ask the user, don't guess; the load profile (VUs/vuRamp+durationSec) is required too and never defaulted; to duplicate a test, copy the full source config (nothing is inherited).configfields replace wholesale:tagsoverwrite (they don't append) — send the union; tags live on the test, not its runs; to tag every test in a project, pagelistLoadTestsuntilhasMoreis false and update all.dryRunrun makes real load and burns VU-hours: dry-run first, show the estimate + parameters, and start only after the user confirms (the model's own confirmation is not the user's).Scope note
This supersedes #399 (closed): that PR also carried a generic opt-in read cache in
src/tools/capability-registry/(shared withtm). Per review it's kept index-only here; the generic cache / dedup is deferred to a separate, generic mcp-server change.Notes for reviewers
These close the "assumed a value / skipped a step" class of agent failures. They do not address the redundant-call (thrash) class — that needs the generic cache/dedup, intentionally not in this PR.