ci: trust bazel cache except on push, fix checkout/setup-bazel order - #356
Merged
Merged
Conversation
- quality_runners.py: add --trust-cache flag; when set, drop --nocache_test_results so Bazel can reuse cached test/coverage results for modules unaffected by the current change. - test_and_docs.yml: pass --trust-cache for every event except 'push' (the only trigger that saves the disk-cache, updates the published coverage numbers, and runs docs-deploy with authoritative results). PR/merge_group/release/workflow_dispatch runs benefit from Bazel's own cache for unaffected modules and get fast feedback; push to main always re-measures every module fresh, so the numbers that get published/cached are never based on a stale cached test result.
…ache hashes reflect real content
bazel-contrib/setup-bazel computes its disk-cache and repository-cache
keys by hashing MODULE.bazel/WORKSPACE*/BUILD*/BUILD.bazel files found
under the job's current working directory at the time it runs. Since
Setup Bazel previously ran before any checkout step, that directory
was always empty at hash-computation time, making the resulting cache
key constant regardless of actual repo content - verified empirically
by pushing real content changes to MODULE.bazel and a BUILD file on a
fork and observing the restore-key hash stay identical across both.
Practical consequence: once a cache entry existed under that constant
key, every subsequent job's cache-save silently failed ('another job
may be creating this cache', GitHub cache entries are immutable per
key), freezing the disk-cache/repository-cache at whatever state they
were in on their first successful save - never picking up newer
external dependency pins or build graph changes since. This did not
affect build correctness (Bazel's own per-action content hashing,
computed after checkout against the real current sources, still
governs whether any individual action result is reused), only cache
freshness/size and the fraction of the build that has to run cold.
Moving the checkout steps ahead of Setup Bazel lets the hash actually
reflect the repository state at each run, allowing the cache to
evolve normally instead of staying pinned to its first snapshot.
antonkri
requested review from
AlexanderLanin,
FScholPer,
MaximilianSoerenPollak,
PiotrKorkus,
dcalavrezo-qorix,
lurtz,
nradakovic,
opajonk and
pawelrutkaq
as code owners
September 22, 2026 10:54
release builds attach the test-report ZIP and build-tools SBOM as official downloadable assets on the GitHub Release (see the "Create archive of test reports" / "Upload release asset" steps, gated on github.ref_type == 'tag'). Those published artifacts should reflect a real, freshly-executed test run, not a potentially stale cached result - same reasoning as for push to main.
|
The created documentation from the pull request is available at: docu-html |
PiotrKorkus
approved these changes
Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Three CI fixes for
test_and_docs.yml, in separate commits:1.
ci: trust bazel cache for unit tests except on push to mainAdds a
--trust-cacheflag toscripts/quality_runners.py. When set, itdrops
--nocache_test_resultsfrom thebazel coverageinvocation, soBazel can reuse cached test/coverage results for modules unaffected by the
current change instead of always re-executing them.
The workflow passes
--trust-cachefor every trigger event exceptpush(the only one that saves the disk-cache, publishes coverage numbers,and runs
docs-deploy). Concretely:pull_request_target/merge_group/workflow_dispatch:fast, cache-friendly runs for iteration/checks.
pushtomain: always re-measures every module fresh, so publishedcoverage numbers are never based on a stale cached test result.
2.
ci: checkout repository before Setup Bazel so disk-cache/repository-cache hashes reflect real contentMoves the two
Checkout repositorysteps ahead ofSetup Bazel.bazel-contrib/setup-bazelcomputes itsdisk-cache/repository-cachekeys by hashing
MODULE.bazel/WORKSPACE*/BUILD/BUILD.bazelfilesfound under the job's current working directory at the time it runs.
Since
Setup Bazelpreviously ran before any checkout step, thatdirectory was always empty at hash-computation time, making the resulting
cache key constant regardless of actual repo content.
I verified this empirically on a fork: pushing real content changes to
MODULE.bazeland to aBUILDfile (both via a realpushevent, not aPR) left the
Setup Bazelrestore-key hash completely unchanged.Why it matters
Because GitHub Actions cache entries are immutable per key, once a cache
existed under this (constant) key, every subsequent job's
cache-savesilently failed (
Failed to save: Unable to reserve cache with key ..., another job may be creating this cache). That freezes thedisk-cache/repository-cache at whatever state it reached on its first
successful save - it never picks up newer external dependency pins or
build graph changes afterwards.
This does not affect build correctness: Bazel's own per-action content
hashing (computed after checkout, against the real current sources) still
governs whether any individual action result is reused. It only affects
cache freshness/size and how much of the build has to run cold on every
CI run.
3.
ci: also treat release events as authoritative (no --trust-cache)releasebuilds attach the test-report ZIP and build-tools SBOM asofficial downloadable assets on the GitHub Release (see the
"Create archive of test reports" / "Upload release asset" steps, gated
on
github.ref_type == 'tag'). Those published artifacts should reflecta real, freshly-executed test run rather than a potentially stale cached
result - same reasoning as for
pushtomain. The condition is nowgithub.event_name != 'push' && github.event_name != 'release' && '--trust-cache' || '',so only
pull_request_target/merge_group/workflow_dispatchrunsget the fast, cache-friendly path.
Testing
Both changes were tested end-to-end on a personal fork
(
antonkri/reference_integration) via direct pushes and PRs against thefork's own
main, including:--trust-cacheis present/absent in the actual invokedcommand depending on event type (checked via job logs),
before/after real
MODULE.bazelandBUILDfile content changes(reproducing the bug this PR fixes),
integration tests, SBOM, docs build, docs-deploy) still completes
successfully with the new step order.