Move the remaining native Linux CI jobs off EC2 - #22108
Draft
huydhn wants to merge 1 commit into
Draft
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22108
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 81c2f2b with merge base db8b643 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
huydhn
force-pushed
the
osdc/native-linux-jobs
branch
2 times, most recently
from
August 25, 2026 01:16
d2686e7 to
0a8e483
Compare
huydhn
added a commit
that referenced
this pull request
Aug 28, 2026
Part 1 of moving ExecuTorch's Linux CI to [OSDC](https://github.com/pytorch/test-infra/blob/main/docs/osdc_runners.md). The `linux_job_v2` -> `v3` migration follows in #22245 onward; #22108 covers the native jobs. OSDC pods have no docker daemon, so the builds move to the in-cluster BuildKit pool: `build.sh` switches to `docker buildx build --push` under `REMOTE_BUILDKIT`, driven by test-infra's `docker-build-remote-buildkit` action. The tag is computed in the workflow now, but it is the same string `calculate-docker-image` produced, so jobs still on v2 keep resolving. That action only retries failures from before BuildKit starts, so the three retries `calculate-docker-image` wrapped the build in are kept around the SDK downloads. sccache resolved its S3 credentials from EC2 instance metadata, which a BuildKit pod has none of, so every compiler invocation died on a metadata timeout. They now reach the build as a BuildKit secret, and `install_pytorch.sh` caches locally when it is absent. Dropping the S3 backend would have been the smaller change, but the PyTorch build layer takes ~19 minutes only because sccache serves nearly all of it, and uncached it would not fit the job timeout. `pull_request` gives way to `ciflow/docker`, since a fork PR gets no OIDC token and could never push. The tag is registered in `pytorch-probot.yml` and auto-applied on the paths that need a rebuild, so such a PR still rebuilds its images without anyone having to remember — the same rule pytorch has. Last commit adds `_docker-image.yml`, the consumer-side counterpart: it emits `git rev-parse HEAD:.ci/docker` for the v3 jobs to name their image with. Nothing calls it yet. All 16 images build and push green on OSDC in [run 32799247406](https://github.com/pytorch/executorch/actions/runs/32799247406): 16–48 min each against 37–56 on EC2, arm64 included. Authored with Claude Code.
These are the Linux jobs that write their own steps instead of calling linux_job, so the v3 migration did not reach them. The two Cadence Xtensa workflows were native specifically because the OIDC token had to be minted on the host: the ACTIONS_ID_TOKEN_REQUEST_* variables did not cross into the docker exec, so the job assumed the Cadence role outside the container and passed the credentials in through docker run. OSDC has no host docker daemon, and a job there already runs entirely inside its container, so the workaround is no longer needed and the docker run wrapper collapses into plain steps. The reason these stay native rather than becoming linux_job_v3 is now only that they assume the Cadence artifacts role rather than role/arc. Three jobs did no ExecuTorch work at all and move to GitHub-hosted runners instead of OSDC: the stale-PR bot, which only calls the GitHub API, and the two jobs that repackage an artifact into S3, which now assume role/arc through OIDC rather than relying on the EC2 instance role they used to inherit. apple.yml's test-demo-ios keeps its EC2 label, because mobile_job.yml resolves its actions by relative path and that does not work on OSDC. This builds on the linux_job_v3 migration for _docker-image.yml, which resolves the CI image the Xtensa jobs now name in `container:`. Authored with Claude Code.
huydhn
force-pushed
the
osdc/native-linux-jobs
branch
from
August 28, 2026 02:38
0a8e483 to
81c2f2b
Compare
github-actions
Bot
added
ciflow/trunk
module: arm
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves the Linux jobs that write their own steps, so the
linux_job_v3migration in #22245 onward does not reach them. Independent of that stack — it only needs_docker-image.yml, which landed with #22106.The Xtensa workflows were native because
ACTIONS_ID_TOKEN_REQUEST_*did not cross into the docker exec, so they assumed the Cadence role outside the container and passed credentials intodocker run. OSDC has no host docker and the job already runs inside its container, so that collapses into plain steps. They stay native only because they assume the Cadence role rather thanrole/arc.stale.ymland the two artifact-to-S3 jobs do no ExecuTorch work, so they move toubuntu-latestand assumerole/arcthrough OIDC.apple.yml'stest-demo-iosstays on EC2:mobile_job.ymlresolves its actions by relative path, which does not work on OSDC.Authored with Claude Code.
cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani