Conversation
|
Bencher Report| Project | Example |
| Branch | u/ep/bare-metal-e2e |
| Testbed | ubuntu-latest |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result nanoseconds (ns) (Result Δ%) | Upper Boundary nanoseconds (ns) (Limit %) |
|---|---|---|---|
| bencher::mock_0 | 📈 view plot 🚷 view threshold | 4.08 ns(-17.10%)Baseline: 4.92 ns | 11.41 ns (35.77%) |
| bencher::mock_1 | 📈 view plot 🚷 view threshold | 12.94 ns(-18.17%)Baseline: 15.81 ns | 22.26 ns (58.10%) |
| bencher::mock_2 | 📈 view plot 🚷 view threshold | 27.99 ns(+15.05%)Baseline: 24.33 ns | 31.14 ns (89.90%) |
| bencher::mock_3 | 📈 view plot 🚷 view threshold | 36.58 ns(+5.39%)Baseline: 34.71 ns | 42.26 ns (86.57%) |
| bencher::mock_4 | 📈 view plot 🚷 view threshold | 48.19 ns(+8.89%)Baseline: 44.26 ns | 50.87 ns (94.74%) |
|
| Project | Example |
| Branch | fork/u/ep/bare-metal-e2e |
| Testbed | ubuntu-latest |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result nanoseconds (ns) (Result Δ%) | Upper Boundary nanoseconds (ns) (Limit %) |
|---|---|---|---|
| bencher::mock_0 | 📈 view plot 🚷 view threshold | 1.00 ns(-79.68%)Baseline: 4.92 ns | 11.41 ns (8.76%) |
|
| Project | Callback E2E |
| Branch | u/ep/bare-metal-e2e |
| Testbed | Updated Spec |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | Latency | microseconds (µs) |
|---|---|---|
| e2e::callback | 📈 view plot | 1.00 µs |
|
| Project | Callback E2E |
| Branch | u/ep/bare-metal-e2e |
| Testbed | Updated Spec |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | Latency | microseconds (µs) |
|---|---|---|
| e2e::callback | 📈 view plot | 1.00 µs |
|
| Project | Callback E2E |
| Branch | u/ep/bare-metal-e2e |
| Testbed | Updated Spec |
🐰 View full continuous benchmarking report in Bencher
⚠️ WARNING: No benchmarks found!
|
| Project | Callback E2E |
| Branch | u/ep/bare-metal-e2e |
| Testbed | Updated Spec |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | Latency | microseconds (µs) |
|---|---|---|
| e2e::callback | 📈 view plot | 1.00 µs |
|
| Project | Callback E2E |
| Branch | u/ep/bare-metal-e2e |
| Testbed | Updated Spec |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | Build Time | seconds (s) |
|---|---|---|
| /usr/local/bin/e2e-bench run | 📈 view plot | 3.01 s |
|
| Project | Callback E2E |
| Branch | u/ep/bare-metal-e2e |
| Testbed | Updated Spec |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | Latency | microseconds (µs) |
|---|---|---|
| e2e::callback | 📈 view plot | 1.00 µs |
|
| Project | Callback E2E |
| Branch | u/ep/bare-metal-e2e |
| Testbed | No Sandbox Spec |
🐰 View full continuous benchmarking report in Bencher
⚠️ WARNING: No benchmarks found!
Runs the job callback scenarios against the dev API from a test pull request, and the push scenario from a push to a dedicated branch. The submit jobs start the detached runs and exit, then a runner on a KVM machine picks up the queued Jobs, and the attach workflow on main reports each one when its callback arrives. Each submit job checks the CLI's raw output for the tokens it was given, so a leak fails the job before the log is masked. This is test tooling only and must never be merged.
Every Job in a run finishes within about a minute of the runner starting, so the 30 minute timeout only held the runner, and the next run waiting on the concurrency group, idle.
The job callbacks branch now sends the dispatch payload as a bencher object and a github object, so the raw callback body no longer carries the old top-level keys.
…lback A repository_dispatch callback is now sealed on every plan, so the Job in the free organization that composes a dispatch expects it delivered like the happy path. A new submit job sends a custom callback from the same organization and expects it skipped, since every other callback still needs a paid plan. Its header carries a throwaway value for the redaction check, and the check for a printed callback URL path now takes that job's path.
32c7270 to
f9701e7
Compare
|
| Project | Callback E2E Free |
| Branch | u/ep/bare-metal-e2e |
| Testbed | Updated Spec |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | Latency | microseconds (µs) |
|---|---|---|
| e2e::callback | 📈 view plot | 1.00 µs |
The raw callback body's ci_id embeds the project name placeholder inside text, so the attach's check name shows the server substituted it.
|
| Project | Callback E2E |
| Branch | u/ep/bare-metal-e2e |
| Testbed | Updated Spec |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | Latency | microseconds (µs) |
|---|---|---|
| e2e::callback | 📈 view plot | 1.00 µs |
|
The bare metal job callbacks shipped, so this end-to-end test is done. The docs workflows replace it in #13. |
1 participant
Do not merge. This pull request exists to run an end-to-end test of bare metal job callbacks against the dev API.
Each run builds the Bencher CLI and runner from the job callbacks branch, makes sure the dev projects, plan, and image exist, and submits several detached bare metal runs with a callback. A runner hosted in this workflow then runs the Jobs, and each Job's callback sends a
bencher_runrepository_dispatchthat the attach workflow onmainhandles. The checks and comments on this pull request are the results.