ci: run mirror-upstream and aggregate weekly instead of 4x daily - #116
Open
dnplkndll wants to merge 19 commits into
Open
ci: run mirror-upstream and aggregate weekly instead of 4x daily#116dnplkndll wants to merge 19 commits into
dnplkndll wants to merge 19 commits into
Conversation
…l-OCA test Squashed fork-only CI customizations on top of OCA/19.0 (was 23 granular commits): - mirror-upstream: daily force-pull OCA/19.0 -> fork 19.0 + drift alert - aggregate: gitaggregate repos.yaml merges -> force-push aggregated - build-image: amd64 image to registry.hz.ledoweb.com (Zot), repository_dispatch chain - test-migration: expanded triggers (19.0-mig-*/19.0-fix-*/aggregated/ledoent), OCA DOWNLOADS pin, full build toolchain for the lean 'oca forks' self-hosted runner - test-migration-enriched: fork-only enriched 18.0 seed (postgres:16, deps baked) - test-migration-real-oca: SMB+OCA realistic seed, clone OCA repos in addons-path - documentation-commit: restrict to OCA upstream pushes
…lates Squashed fork-only docs (was 5 granular commits): internal 18->19 roadmap + 19->20 prep, proof-of-work migration record, upstream-PR feedback templates. Fork-only — not for upstream.
Stop duplicating the aggregate merge list in this workflow's heredoc. The list now lives ONLY in ledoent/openupgrade-lab repos.yaml (./openupgrade block); this workflow checks out the lab repo and extracts it. Keeps the OpenUpgrade fork's working/migration branches clean of aggregation config — they stay based on pristine OCA/19.0 and upstream-ready. Trigger also covers 19.0-mig-* now. See docs/fork-aggregation-model.md in the lab repo.
openupgrade-lab is private; default GITHUB_TOKEN is scoped to the OpenUpgrade repo only -> 'Repository not found' on the cross-repo checkout. Use the existing AGGREGATE_PUSH_TOKEN PAT (already used for push + dispatch) which has cross-repo read.
AGGREGATE_PUSH_TOKEN isn't configured as a fork secret (only ZOT_* are), and the default GITHUB_TOKEN can't read the private ledoent/openupgrade-lab. Switch to a dedicated LAB_READ_TOKEN secret (fine-grained PAT, read-only on the lab repo). Until that secret is added, the aggregate's lab-checkout step fails fast with 'Input required and not supplied: token'.
The LAB_READ_TOKEN PAT is read-only. actions/checkout defaults persist-credentials to true, which configures the token for push and fails with 403 'Write access not granted' even on a read-only fetch. We only read repos.yaml — disable it.
29 ready PR branches each mark their module's status row in the shared coverage tracker. gitaggregate's sequential 'git pull --no-rebase' conflicts whenever two branches touch nearby rows. A union merge driver auto-combines the non-overlapping row additions (same pattern already used for test-requirements.txt). Branches that mark the SAME module are de-duped separately so union doesn't double the row.
…gle_gmail (#103) geoip2 already covered website; cloud_storage_google (and google_gmail) declare google-auth. google-auth's runtime deps are cachetools/pyasn1-modules/rsa — it does NOT pull cryptography, so it won't disturb the runner's pyOpenSSL. (auth_ldap's python-ldap, which needs apt build libs, is added separately if/when it's marked.)
…isabled on fork) (#101)
Each merged migration branch marks its modules in docsource/modules180-190.rst, so gitaggregate replays one row per branch that touches a module — 13 modules ended up with duplicate rows on aggregated (16 surplus rows). Duplicates make the coverage table unreadable and let a blank row mask a marked one. New step collapses to the first occurrence per module (preferring a marked row over a blank) and commits onto the aggregated HEAD that gets force-pushed. Per-branch diffs are untouched; the dedup lives only on the throwaway aggregated tree.
The 'Trigger build-image workflow' step called `gh api`, but the self-hosted runner pool doesn't ship the gh CLI, so the step failed with "gh: command not found" (exit 127) and the aggregate run went red even though gitaggregate and the force-push had both succeeded. Replace the gh call with an equivalent curl POST to the repository_dispatch API (curl is universally available). Same event_type and client_payload; the no-token fallback is preserved.
…onflict (#106) google-auth requires cryptography>=38, which evicts Odoo's pinned cryptography==3.4.8 and desyncs the matched pyopenssl==21.0.0 / urllib3 1.26 stack (X509_V_FLAG_NOTIFY_POLICY and hazmat.backends.openssl.x509 are gone in modern cryptography) → import OpenSSL fails, base won't load, every migration run dies before a script executes. #103 added google-auth on 2026-06-03 and silently broke this. The only thing that needs google-auth is cloud_storage_google, whose 19.0 manifest declares it as an external dependency (18.0 didn't). It's only in the enriched seed. - Per-PR (test-migration.yml): drop google-auth — the OCA seed has no google-auth-dependent module, so this restores Odoo 18's matched crypto stack and greens the per-PR test. - Enriched (test-migration-enriched.yml): drop google-auth AND mark cloud_storage_google uninstalled post-restore so OpenUpgrade skips it (nothing depends on it; its 18->19 change needs live Google config to exercise). Both halves keep cryptography untouched. Restores both migration gates, which had been red since #103 (per-PR) and longer (enriched, on cloud_storage_google's missing dep then the crypto break).
… PG lock limit (#109) - the OCA 18.0 seed also carries cloud_storage_google: button_upgrade refuses on the missing google-auth external dep (same class #106 fixed for the enriched seed). - the enriched migration now runs to completion and dies only in the final _process_end unlink sweep: out of shared memory at the default max_locks_per_transaction=64; raise to 1024 and restart the service.
…r-step-container runners) (#111)
… dep The enriched gate force-updates l10n_es_edi_verifactu (more l10n_es modules in MODULES_NEW than baseline), which inherits certificate.certificate but omits the dep from its manifest -> registry build dies. Mirror the same odoo-checkout patch already in test-migration.yml. Upstream fix: odoo/odoo#271120.
12 upstream commits landed within one daily window (2026-07-13), leaving fork 19.0 behind and polluting rebased PRs' commit lists with upstream commits. Claude-Session: https://claude.ai/code/session_01MpdyzmGjVAYqAEqMYr3EnX
mirror-upstream ticked every 6h and aggregate daily. That tempo was set when this fork was an active contribution target and open PRs drifted from their base faster than a daily tick could keep up. It no longer is one. The cost is not local to this repository. Mirroring force-pushes 19.0, which fires aggregate's push trigger, which rebuilds and force-pushes 'aggregated', which in turn runs tests, pre-commit and the migration suite. Four of those cascades a day saturate the organisation-wide 20-concurrent-job cap on the Free plan — unrelated repositories were observed queueing behind 18 running and 20 queued OpenUpgrade jobs. Both now run Sunday early morning, mirror at 05:00 UTC and aggregate at 06:00. The one-hour gap preserves the existing ordering requirement: aggregate must run after the mirror, or it rebuilds against a stale 19.0. The tradeoff is the detection window for an upstream merge conflicting with a repos.yaml branch, which widens from a day to a week. Acceptable while this is not a contribution target. workflow_dispatch is retained on both for manual catch-up, and is now the expected way to force a refresh. Claude-Session: https://claude.ai/code/session_01JYdwXZhodF13nMyBxPRSe2
dnplkndll
force-pushed
the
ledoent
branch
2 times, most recently
from
August 11, 2026 14:24
80357a6 to
e632ffc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
mirror-upstreamticked every 6 hours andaggregatedaily. That tempowas set on 2026-07-13 because Tecnativa-pace upstream outran a daily tick and
open PRs drifted from their base. This fork is no longer an active contribution
target, so that justification has lapsed.
The cost is not local to this repository. The schedule cascades:
Four of those a day saturate the org-wide 20-concurrent-job cap on the Free
plan. Observed today: 18 running + 20 queued OpenUpgrade jobs, with
unrelated
ledoentrepositories queueing behind them for ~10 minutes.What changed
mirror-upstream.yml0 */6 * * *(4x daily)0 5 * * 0— Sun 05:00 UTCaggregate.yml0 12 * * *(daily)0 6 * * 0— Sun 06:00 UTCThe one-hour gap is deliberate.
aggregatemust run after the mirror or itrebuilds against a stale
19.0— the existing comment documents this orderingand it is preserved.
Nothing else is touched:
generate-analysis-cron(weekly Mon),generate-testdb-cron(monthly) andstale(weekly Sun) are unchanged, as areall
push/pull_requesttriggers.Tradeoff
The detection window for an upstream OCA merge silently conflicting with a
repos.yamlbranch — the l10n_es / OCA#5646 case in the existing comment —widens from a day to a week. That is the real cost, and it is acceptable
while this is not a contribution target.
workflow_dispatchis retained on both and is now the expected way to force arefresh between ticks.
https://claude.ai/code/session_01JYdwXZhodF13nMyBxPRSe2