Version the report payload with bmf_version

[epompeii]

The key

A report payload now says what format it is written in. bmf_version is a top level key of the report payload, next to results, and it versions the whole payload rather than any one results string. It accepts exactly 0 and 1 today.

An absent key is version 0. That is the whole compatibility story: every payload that ingests today ingests unchanged, byte for byte, because absent and 0 take the same path and bencher run still sends nothing.

Anything else is refused at the door, and the refusal names the accepted versions. 2, 255, 256, -1, "1", 1.5, and true all answer 400 with "The accepted versions are 0 or 1". This is a field a client deliberately sent, so a value it does not recognize is a mistake worth reporting rather than something to guess around. Two messages carry that list, because an integer outside the range is reported by the validation error and a value that is not an unsigned integer at all is reported by the deserializer.

One type, two values

BmfVersion is a validated newtype in bencher_valid, alongside Index and the model types, with the same shape: TryFrom<u8>, a visitor that names what it expects, FromStr, and an is_valid_bmf_version predicate.

It replaces the BmfVersion enum that already lived in bencher_adapter, rather than sitting next to it. That enum named the same scale for the same format: the version a results payload was parsed as. The payload's declared version and the leaf's parsed version are two variables of one type, not two types, and a payload that declares version 1 while holding v0 results is exactly why they have to be able to differ in value while agreeing on meaning. BmfVersion::V0 and BmfVersion::V1 are associated constants, so every existing comparison reads as it did.

The preference

At version 1 the json node tries its json_v1 leaf first and falls back to json_v0. At version 0, which is also what an absent key means, it tries them the other way around, which is today's order.

That is a reordering of the attempts, not a filter on the payload. Every payload that only one leaf claims lands on that leaf either way, so version 1 refuses no v0 shape and version 0 still ingests a v1 shape through the second attempt. The empty payload {} is the only payload both leaves claim, and so it is the only payload whose parsed version the key moves.

The magic node inherits the preference without reading the key, because it reaches the JSON leaves only through the json node.

Fold and the empty payload

The one payload both leaves claim is also the one payload the reordering could have cost something. Fold is refused for BMF v1 and is all or nothing across the results array, so an empty iteration reading as v1 at version 1 would disable fold for every v0 iteration beside it: ["{}", v0, v0] under fold would land as two iterations and two metric rows at version 1 where version 0 writes one folded iteration. A metric row count is not a detail this key is allowed to move.

An empty payload is therefore foldable at any version. The refusal exists because a pooled statistic cannot be recomputed from per iteration values, and a payload that reported nothing has none, so it has nothing to refuse over. A payload that actually reported something as v1 is refused exactly as before, at either declared version, because the refusal keys on what was parsed.

An explicitly named leaf is unaffected. json_v0 and json_v1 are already exact statements about the payload, so bmf_version does not override them: json_v0 with bmf_version: 1 still refuses a v1 payload. Non-JSON adapters never read the key at all; it reaches them as an unused field of the adapter settings.

/v0/run

The run endpoint takes its own payload and converts it into a report payload, so the key reaches ingest because JsonNewRun carries it and From<JsonNewRun> for JsonNewReport forwards it. That conversion destructures every field, so the compiler required the forwarding rather than allowing a silent drop.

A job based run is the one path that does not carry the key. Its results are the runner's own output rather than the submitted payload, and bmf_version versions that payload, so a job parses in the default order, which is the order every run has always parsed in. Persisting a declared version alongside a job is a later question.

What waits

• The v1 thresholds shape and the project gate are later layers.
• The CLI gains no flag here. bencher run and bencher report create send no bmf_version, which is deferred, not forgotten.

Tests

The strong claim is pinned where it is exact. At the adapter level, an absent version and an explicit 0 parse every JSON fixture to equal results, and so do version 1 and version 0, over every fixture including the two that no leaf claims. Both assertions fail if the reorder loses its fallback, and adapter_json_empty_is_v1_at_version_1 fails if the reorder is not there at all.

Fold is pinned at both levels. At the adapter level, ["{}", v0(10), v0(20)] folds to equal results at version 1 and version 0 for all four operations, a pure v0 array does the same, and a genuine v1 payload is still refused at either version. End to end, the same two arrays under each fold produce equal reports at either declared version and land as one folded iteration, while a v1 payload under fold ingests unfolded, one iteration per result, at either version. The assertions are between the two versions rather than about the folded value: the mean divides by the length of the array, so an empty iteration dilutes it, which is existing behavior this layer leaves where it found it.

End to end, bmf_version.rs posts real reports and compares whole responses with only the minted identity normalized away: every uuid becomes the position it was first seen at, which preserves aliasing, and creation times become a placeholder. Everything else, every measured value, every count, every alert, and the echoed adapter, is compared as it came off the wire. A payload with no bmf_version key at all, one sending null, and one sending 0 all produce the same report. A v1 payload at version 1 produces the report a v1 payload at version 0 produces; a v0 payload at version 1 produces the report it produces at version 0; and a v0 payload's report differs from a v1 payload's, which is what keeps the comparison from being vacuous.

[github-actions]

Bencher Report

Project	Bencher
Branch	u/ep/parameters-api/bmf-version
Testbed	intel-v1

Click to view all benchmark results

Benchmark	Latency	Benchmark Result microseconds (µs) (Result Δ%)	Upper Boundary microseconds (µs) (Limit %)
Adapter::Json	📈 view plot 🚷 view threshold	5.20 µs (+9.67%) Baseline: 4.74 µs	5.47 µs (95.11%)
Adapter::Magic (JSON)	📈 view plot 🚷 view threshold	5.02 µs (+9.20%) Baseline: 4.60 µs	5.25 µs (95.67%)
Adapter::Magic (Rust)	📈 view plot 🚷 view threshold	27.87 µs (+7.12%) Baseline: 26.02 µs	28.04 µs (99.40%)
Adapter::Rust	📈 view plot 🚷 view threshold	4.63 µs (+26.71%) Baseline: 3.65 µs	5.12 µs (90.44%)
Adapter::RustBench	📈 view plot 🚷 view threshold	4.61 µs (+26.44%) Baseline: 3.65 µs	5.11 µs (90.35%)

🐰 View full continuous benchmarking report in Bencher