Skip to content

ci: trust bazel cache except on push, fix checkout/setup-bazel order - #356

Merged
antonkri merged 3 commits into
mainfrom
ci/trust-cache-and-checkout-order-fix
Sep 22, 2026
Merged

antonkri merged 3 commits into
mainfrom
ci/trust-cache-and-checkout-order-fix

Conversation

@antonkri

@antonkri antonkri commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

What

Three CI fixes for test_and_docs.yml, in separate commits:

1. ci: trust bazel cache for unit tests except on push to main

Adds a --trust-cache flag to scripts/quality_runners.py. When set, it
drops --nocache_test_results from the bazel coverage invocation, so
Bazel can reuse cached test/coverage results for modules unaffected by the
current change instead of always re-executing them.

The workflow passes --trust-cache for every trigger event except
push (the only one that saves the disk-cache, publishes coverage numbers,
and runs docs-deploy). Concretely:

  • pull_request_target / merge_group / workflow_dispatch:
    fast, cache-friendly runs for iteration/checks.
  • push to main: always re-measures every module fresh, so published
    coverage numbers are never based on a stale cached test result.

2. ci: checkout repository before Setup Bazel so disk-cache/repository-cache hashes reflect real content

Moves the two Checkout repository steps ahead of Setup Bazel.

bazel-contrib/setup-bazel computes its disk-cache/repository-cache
keys by hashing MODULE.bazel/WORKSPACE*/BUILD/BUILD.bazel files
found under the job's current working directory at the time it runs.
Since Setup Bazel previously ran before any checkout step, that
directory was always empty at hash-computation time, making the resulting
cache key constant regardless of actual repo content.

I verified this empirically on a fork: pushing real content changes to
MODULE.bazel and to a BUILD file (both via a real push event, not a
PR) left the Setup Bazel restore-key hash completely unchanged.

Why it matters

Because GitHub Actions cache entries are immutable per key, once a cache
existed under this (constant) key, every subsequent job's cache-save
silently failed (Failed to save: Unable to reserve cache with key ..., another job may be creating this cache). That freezes the
disk-cache/repository-cache at whatever state it reached on its first
successful save - it never picks up newer external dependency pins or
build graph changes afterwards.

This does not affect build correctness: Bazel's own per-action content
hashing (computed after checkout, against the real current sources) still
governs whether any individual action result is reused. It only affects
cache freshness/size and how much of the build has to run cold on every
CI run.

3. ci: also treat release events as authoritative (no --trust-cache)

release builds attach the test-report ZIP and build-tools SBOM as
official downloadable assets on the GitHub Release (see the
"Create archive of test reports" / "Upload release asset" steps, gated
on github.ref_type == 'tag'). Those published artifacts should reflect
a real, freshly-executed test run rather than a potentially stale cached
result - same reasoning as for push to main. The condition is now
github.event_name != 'push' && github.event_name != 'release' && '--trust-cache' || '',
so only pull_request_target / merge_group / workflow_dispatch runs
get the fast, cache-friendly path.

Testing

Both changes were tested end-to-end on a personal fork
(antonkri/reference_integration) via direct pushes and PRs against the
fork's own main, including:

  • confirming --trust-cache is present/absent in the actual invoked
    command depending on event type (checked via job logs),
  • confirming the disk-cache/repository-cache restore-key hash is identical
    before/after real MODULE.bazel and BUILD file content changes
    (reproducing the bug this PR fixes),
  • confirming the existing behavior (docs preflight, unit tests, feature
    integration tests, SBOM, docs build, docs-deploy) still completes
    successfully with the new step order.

- quality_runners.py: add --trust-cache flag; when set, drop
  --nocache_test_results so Bazel can reuse cached test/coverage
  results for modules unaffected by the current change.
- test_and_docs.yml: pass --trust-cache for every event except
  'push' (the only trigger that saves the disk-cache, updates the
  published coverage numbers, and runs docs-deploy with authoritative
  results). PR/merge_group/release/workflow_dispatch runs benefit
  from Bazel's own cache for unaffected modules and get fast
  feedback; push to main always re-measures every module fresh, so
  the numbers that get published/cached are never based on a stale
  cached test result.
…ache hashes reflect real content

bazel-contrib/setup-bazel computes its disk-cache and repository-cache
keys by hashing MODULE.bazel/WORKSPACE*/BUILD*/BUILD.bazel files found
under the job's current working directory at the time it runs. Since
Setup Bazel previously ran before any checkout step, that directory
was always empty at hash-computation time, making the resulting cache
key constant regardless of actual repo content - verified empirically
by pushing real content changes to MODULE.bazel and a BUILD file on a
fork and observing the restore-key hash stay identical across both.

Practical consequence: once a cache entry existed under that constant
key, every subsequent job's cache-save silently failed ('another job
may be creating this cache', GitHub cache entries are immutable per
key), freezing the disk-cache/repository-cache at whatever state they
were in on their first successful save - never picking up newer
external dependency pins or build graph changes since. This did not
affect build correctness (Bazel's own per-action content hashing,
computed after checkout against the real current sources, still
governs whether any individual action result is reused), only cache
freshness/size and the fraction of the build that has to run cold.

Moving the checkout steps ahead of Setup Bazel lets the hash actually
reflect the repository state at each run, allowing the cache to
evolve normally instead of staying pinned to its first snapshot.
Comment thread .github/workflows/test_and_docs.yml Dismissed
release builds attach the test-report ZIP and build-tools SBOM as
official downloadable assets on the GitHub Release (see the
"Create archive of test reports" / "Upload release asset" steps,
gated on github.ref_type == 'tag'). Those published artifacts should
reflect a real, freshly-executed test run, not a potentially stale
cached result - same reasoning as for push to main.
@github-actions

Copy link
Copy Markdown

The created documentation from the pull request is available at: docu-html

@antonkri
antonkri merged commit 236ecea into main Sep 22, 2026
17 of 18 checks passed
@antonkri
antonkri deleted the ci/trust-cache-and-checkout-order-fix branch September 22, 2026 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants