Resolve any metric row through the metric endpoint - #1014
Draft
epompeii wants to merge 3 commits into
Draft
Conversation
Contributor
|
Bencher Report| Project | Bencher |
| Branch | u/ep/parameters-api/metric-endpoint |
| Testbed | intel-v1 |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result microseconds (碌s) (Result 螖%) | Upper Boundary microseconds (碌s) (Limit %) |
|---|---|---|---|
| Adapter::Json | 馃搱 view plot 馃毞 view threshold | 5.23 碌s(+10.41%)Baseline: 4.73 碌s | 5.42 碌s (96.48%) |
| Adapter::Magic (JSON) | 馃搱 view plot 馃毞 view threshold | 5.04 碌s(+9.63%)Baseline: 4.59 碌s | 5.21 碌s (96.72%) |
| Adapter::Magic (Rust) | 馃搱 view plot 馃毞 view threshold | 27.36 碌s(+5.24%)Baseline: 25.99 碌s | 27.90 碌s (98.03%) |
| Adapter::Rust | 馃搱 view plot 馃毞 view threshold | 4.66 碌s(+28.11%)Baseline: 3.63 碌s | 5.01 碌s (92.93%) |
| Adapter::RustBench | 馃搱 view plot 馃毞 view threshold | 4.64 碌s(+27.77%)Baseline: 3.63 碌s | 5.00 碌s (92.75%) |
epompeii
force-pushed
the
u/ep/parameters-api/metric-endpoint
branch
from
August 27, 2026 03:46
9faab9d to
c1aa850
Compare
The perf query answered per benchmark and returned one scalar per point. A benchmark now has grid points, and a measure now has named scalars, so it answers per grid point and returns all of them. `parameters` is a new query parameter: a comma separated list of URL encoded parameter sets. A grid point is queried when at least one of them is a subset of its set, which makes the filter an OR across its elements and an AND within one. Leaving it off queries every grid point. The filter resolves to parameter set row identifiers in Rust, over the benchmark's own sets, so what reaches SQL is an indexed lookup and never a JSON predicate. A line is now keyed per grid point. `JsonPerfMetrics` carries the parameter set it plots, between the benchmark and the measure, because that is the order the dimensions run in: branch, testbed, benchmark, parameters, measure, metric. Two grid points of one benchmark are two lines. Each point carries `metrics`, every named scalar the measure ingested, keyed by name. Each entry carries its value and the thresholds that gated it, with the boundary each produced and any alert it raised. That list is a list from birth, because a later layer lets several thresholds gate one named scalar. The deprecated fields keep saying what they always said. `metric` is still the triple, rebuilt from the `value` row and its `lower_value` and `upper_value` siblings, and the singular `threshold`, `boundary`, and `alert` are still the ones that gated the `value` row. A project that only ever reported the triple gets back every field it got back before, unchanged. `metric` is optional, following the report response. A measure that names no `value` still measured something, and it is stored, billed, and queryable, so it is still a point on the line: it carries its named scalars in `metrics` and leaves the triple absent rather than disappearing from the response that serves it. Nothing an older client can produce reaches that case, so the field is present for every one of them. The plot leaves such a point out of the line it draws, because a measurement with no point estimate has nothing to place on the axis. Reads move off the `metric_boundary` view onto the `metric` table. All of a grid point's named scalars for one measure sit together on the index over the report benchmark, the measure, and the name, so one bounded range read returns every one of them, where the view had to seek each conventional name separately and could only ever return those three. The boundary, its threshold, its model, and its alert are chained flat with explicit `ON` clauses: SQLite cannot flatten a compound right operand of an outer join, and the nested form makes it scan the whole boundary table once per request. The migration byte compatibility test drops its perf leg. It captures responses on both sides of the metric migration, which only works for a reader that goes through the view, and the report response already sat out for the same reason. What the perf response owes older clients is pinned where that response lives.
The image endpoint takes `parameters`, the same comma separated list of URL
encoded parameter sets the query endpoint takes, spelled the same way and read by
the same code path. The image query struct mirrors the query struct field for
field, doc comment included, so the image is the plot of the query it mirrors.
A line is a grid point, so the key names the grid point when the benchmark name no
longer says which line is which. A benchmark whose every line in the image plots
the empty parameter set keeps the bare benchmark name it has always had. One non
empty set among a benchmark's lines names them all, the empty set among them
included, which reads `{}`. The set is spelled in its canonical form and follows
the benchmark name on the same row, so the wrapping the key text already does for
a long benchmark name loses the tail of the set spelling first and the name never.
A project that never reported a parameter set is drawn from the inputs it was
always drawn from: every line of every such benchmark is labeled exactly as
before, which the fixtures pin, and nothing else on the draw path moved.
A point whose measure named no `value` has no point estimate to place on the axis,
so the plot leaves it out of the line. That already held; a fixture now pins it.
`GET /v0/projects/{project}/metrics/{metric}` resolved through the
`metric_boundary` view, which drives on `name = 'value'`, so the UUID of a
`lower_value`, an `upper_value`, or any other named scalar was a 404 even though
the report response hands those UUIDs out. The lookup moves onto the `metric`
table, so every metric row UUID resolves.
The response now describes the addressed row. It gains `name` and `value`, the
addressed row's own name and scalar, and `parameter`, the grid point it was
measured under. `metric` becomes optional and is present only when the addressed
row is the `value` row: reconstructing the triple around a bound or a named
scalar would assert numbers the address does not name. `threshold`, `boundary`,
and `alert` describe the boundary attached to the addressed row itself.
A `value` row address is unchanged but for those three additions, which a
fixture pins key by key.
epompeii
force-pushed
the
u/ep/parameters-api/metric-endpoint
branch
from
August 27, 2026 05:11
c1aa850 to
e1af4d2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
GET /v0/projects/{project}/metrics/{metric}resolved through themetric_boundaryview. That view drives onWHERE metric.name = 'value', so it holds one row per point estimate and nothing else. The UUID of alower_value, anupper_value, or any other named scalar was a 404, even though the report response hands those UUIDs out inJsonReportMetric. Post a report with bounds, read back a bound's UUID from the response, ask for it, and the answer was "not found".The lookup moves off the view and onto the
metrictable. Every metric row UUID resolves.The addressed row
The response describes the row the UUID addresses, and nothing else.
nameandvalueare the addressed row's own name and scalar. Every name is equal here:value,lower_value,upper_value, andp99are four rows, each resolving on its own terms.parameteris the grid point the row was measured under, placed betweenbenchmarkandmeasure, the way a perf line already carries it.metric, the triple, becomes optional and is present only when the addressed row is thevaluerow. The triple is a convention over three names, so it means something only when the address names the point estimate it is built around. Reconstructing it around a bound or a named scalar would assert numbers the address does not name.threshold,boundary, andalertdescribe the boundary attached to the addressed row itself. Today only avaluerow carries one, so for any other row they answernull, which is what the join says rather than a special case.Compatibility
A
valuerow address is unchanged but for the three additions. The previous shape answered with thirteen top-level keys; this one answers with those thirteen, each carrying the same value, plusname,value, andparameter.threshold,boundary, andalertkeep serializing asnullrather than vanishing, so nothing a client reads today moves.That claim is a fixture, not a paragraph:
metrics_get_value_row_is_unchanged_but_for_the_additionspins the exact key set and every value, and the expectations were read off the previous shape, which answered with exactly those thirteen keys and 404ed on the two bound rows of the same report.The query
One indexed lookup on
metric.uuid, joined flat toreport_benchmark, its parameter set, and the report context, with the boundary, its threshold, its model, and its alert hung off the addressed row as a flat chain of left joins. The joins stay flat with explicitONclauses for the same reason the perf query's do: SQLite cannot flatten a compound right operand of an outer join, so nesting them makes it scan the whole boundary table once per request. When the addressed row is thevaluerow, one further lookup fetches itslower_valueandupper_valuesiblings on the(report_benchmark_id, measure_id, name)unique index to build the triple. Every step of both queries is an index seek.The
metric_boundaryview keeps its other readers, the alerts endpoint among them, untouched.The migration test
metric_migration.rscompares response bytes across the single-valued metric migration, and its premise is that every reader it captures goes through the view, whose column list the migration holds unchanged. The metric endpoint is no longer such a reader, so it leaves that comparison, exactly as the perf and report responses already had: all three are built from the namedmetricrows, which do not exist before the migration, so none of them has a pre-migration form to compare against.What the metric endpoint owes the migration is asserted directly instead. Every seeded metric has a response afterwards, carrying the triple the legacy row held, and a down and up round trip leaves every metric response unchanged.