Skip to content

Reject capacity results that underdeliver the declared load step #40

Description

@rmcdaniel

Problem

The capacity matrix treats each load_step as a target workflow admission rate, but the result contract never proves that rate was delivered. Admission pauses whenever the open-work semaphore is full and also runs on the same thread as synchronous infrastructure sampling. The reducer checks completion ratio only among starts that happened; it does not compare accepted or attempted starts with the declared step. A nominal high load step can therefore execute much less traffic and remain eligible when the smaller admitted cohort completes.

Query demand has the same gap: the controller uses one serial query loop, observations contain latency samples but no attempted/completed query counter, and eligibility requires only one query latency sample rather than the declared requests per load unit per second.

This can overstate the maximum sustained operating point even though result rates disclose the lower realized throughput.

Acceptance criteria

  • Give each load step an explicit offered-load contract and bounded delivery tolerance for workflow starts and query operations.
  • Record attempted, accepted, completed, rejected, and throttled demand needed to distinguish generator underdelivery from Server saturation.
  • Keep infrastructure collection and other controller overhead from silently blocking admission, or account for that loss as an ineligible step.
  • Mark a step ineligible when realized workflow or query demand falls below its declared target.
  • Prove with adversarial coverage that open-slot saturation, slow collection, and a serial query bottleneck cannot qualify an underdelivered step.

Delete when

Delete when every publishable load step proves the declared workflow and query demand was actually offered for the complete measurement window, and any shortfall is explicit saturation or a failed run rather than an eligible operating point.

Metadata

Metadata

Assignees

No one assigned

    Labels

    authority:githubGitHub is the authoritative lifecycle record for this workbeta:compatibleMay be completed without changing the frozen beta contractkind:defectA public product behavior is incorrectpriority:P1High-priority product or release riskrepo:serverOwned by the standalone server repositorystatus:readyReady for implementation

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions