Skip to content

Migrate all references to the newer avocado.yaml structure (ENG-2182) - #23

Open
nicksinas wants to merge 11 commits into
mainfrom
nsinas-eng-2182
Open

Migrate all references to the newer avocado.yaml structure (ENG-2182)#23
nicksinas wants to merge 11 commits into
mainfrom
nsinas-eng-2182

Conversation

@nicksinas

Copy link
Copy Markdown
Collaborator

What

Migrates all 30 references to the current avocado.yaml structure (ENG-2182).

Per-file changes:

  • Remove the deprecated cli_requirement key
  • Point the SDK image at docker.io/avocadolinux/sdk:{{ config.distro.release }} (drop the -channel suffix; use the config.* namespace)
  • Move the root password out of the config/config-dev confext into a top-level permissions profile
  • Add top-level rootfs and initramfs keys referencing that profile
  • Drop the now-empty config/config-dev extension and its runtime references

Overlays, packages, services, on_merge/on_unmerge, docker_images, SDK args, and all real extensions are untouched. iphone-travel-router keeps its non-empty avocado password (Cockpit/PAM). icam-540 additionally retargets to jetson-orin-nx (its underlying SoM, default_target_board: icam-540) so it resolves in the feed.

Also included:

  • .gitignore: repo-wide rules so per-build state is never committed (avocado.lock, .avocado-desktop.json, .avocado/, .avocado-state) — previously only 1 of 30 references ignored these.
  • scripts/build-check-references.sh: a reference-by-reference build harness (cleanunlockinstallbuild→teardown-to-clean-git-state), with a live markdown report and PASS/FAIL summary.

Build verification

Every reference was run through install + build via the new harness. Each migrated file also validates against the avocado schema (ok:true).

24 / 30 build clean. The remaining failures were investigated and are not caused by this migration — they fail either before the migrated sections are consumed (feed fetch) or inside code compilation (host toolchain), and 24 references using the identical migration pattern build green:

Reference Cause Category
astra-1680-deepx avocado-ext-dev missing from feed for grinn-astra-1680-sbc feed gap
imx8mp-npu-nnstreamer avocado-ext-dev missing from feed for ucm-imx8m-plus feed gap
imx8mp-npu-pose avocado-ext-dev missing from feed for ucm-imx8m-plus feed gap
linux-custom-kernel HOSTCC = target cross-compiler → host tool can't run when cross-building x86 kernel on an arm64 host (builds on x86_64 hosts) cross-arch env
rubicon upstream avocado-ext-cli package build exits non-zero after compiling + installing the CLI upstream package

c-gpio initially failed on a transient clock-skew flake and passes on a clean rebuild. zephyr-imx8mp-evk was still building at PR time (Zephyr pulls dozens of module repos).

Follow-ups for maintainers (not blocking this PR): publish avocado-ext-dev for grinn-astra-1680-sbc and ucm-imx8m-plus to 2024/edge; investigate the avocado-ext-cli package build (CLI 1.0.0-rc.1).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates reference avocado.yaml files to the newer schema structure (removing deprecated keys, updating SDK image templating, and moving root-password configuration into top-level permissions profiles), and adds repo-wide tooling/ignores to support repeatable reference build verification without committing build state.

Changes:

  • Update reference avocado.yaml files to use permissions + rootfs/initramfs and remove deprecated cli_requirement and config/config-dev confext usage.
  • Standardize SDK image to docker.io/avocadolinux/sdk:{{ config.distro.release }}.
  • Add a reference-by-reference build verification harness and expand .gitignore to exclude Avocado-generated state.

Reviewed changes

Copilot reviewed 31 out of 32 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
zephyr-imx8mp-evk/avocado.yaml Removes deprecated keys and introduces top-level permissions + rootfs/initramfs; updates SDK image template.
webkit-ui/avocado.yaml Moves root password config into permissions and updates SDK image template; removes config-dev runtime reference.
shell-heartbeat/avocado.yaml Drops config confext usage, adds permissions + rootfs/initramfs, updates SDK image template.
scripts/build-check-references.sh New harness to clean/unlock/install/build each reference and produce a live markdown report.
rust-vitals/avocado.yaml Removes config-dev confext and adds permissions + rootfs/initramfs; updates SDK image template.
rubicon/avocado.yaml Removes config-dev confext and adds permissions + rootfs/initramfs; updates SDK image template.
ros2-ufactory-lite6/avocado.yaml Removes deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
react-dashboard/avocado.yaml Removes config-dev runtime refs across targets; adds permissions + rootfs/initramfs; updates SDK image template.
qemu-quickstart/avocado.yaml Drops config runtime/confext; adds permissions + rootfs/initramfs; updates SDK image template.
python-yolo/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
python-whisper/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
python-multiversion-uv/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
python-mqtt/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
python-flask/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
pi-metrics-exporter/avocado.yaml Removes config-dev confext; adds permissions + rootfs/initramfs; updates SDK image template.
nvidia-gstreamer-yolo/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
nvidia-deepstream/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
nodejs-dashboard/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
linux-custom-kernel/avocado.yaml Removes config-dev runtime/confext; adds permissions + rootfs/initramfs; updates SDK image template.
java-hello/avocado.yaml Removes config-dev confext; adds permissions + rootfs/initramfs; updates SDK image template.
iphone-travel-router/avocado.yaml Moves root password config into permissions while preserving non-empty password; updates SDK image template.
imx8mp-npu-pose/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
imx8mp-npu-nnstreamer/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
icam-540/avocado.yaml Updates default target/board mapping and migrates to permissions + rootfs/initramfs; updates SDK image template.
elixir-phoenix/avocado.yaml Removes config-dev runtime/confext and a now-empty confext; adds permissions + rootfs/initramfs; updates SDK image template.
docker-save/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
docker-registry/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
dev/avocado.yaml Removes config-dev confext; adds permissions + rootfs/initramfs; updates SDK image template.
cpp-tui-dashboard/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
c-gpio/avocado.yaml Drops deprecated keys/config confext; adds permissions + rootfs/initramfs; updates SDK image template.
astra-1680-deepx/avocado.yaml Migrates to permissions + rootfs/initramfs; updates SDK image template; also removes a local distro repo override.
.gitignore Adds repo-wide ignores for Avocado lock/state files.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread scripts/build-check-references.sh
Comment thread astra-1680-deepx/avocado.yaml

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 37 out of 38 changed files in this pull request and generated 1 comment.

Comment thread scripts/build-check-references.sh Outdated

@jetm jetm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the avocado.yaml schema migration. It's clean and consistent across all references: no dangling config/config-dev extension references, the SDK image tag change (dropping the channel) is applied uniformly, and every reference got the new permissions: block. One minor cross-file consistency nit inline on icam-540.

Comment thread icam-540/avocado.yaml
Comment thread c-gpio/avocado.yaml
Comment thread dev/avocado.yaml
Comment thread icam-540/avocado.yaml
Comment thread scripts/build-check-references.sh Outdated
Update all 30 references to the current avocado.yaml layout:
- Remove the deprecated cli_requirement key
- Point the SDK image at docker.io/avocadolinux/sdk:{{ config.distro.release }}
  (drop the -channel suffix; use the config.* namespace)
- Move the root password out of the config/config-dev confext into a
  top-level permissions profile
- Add top-level rootfs and initramfs keys referencing that profile
- Drop the now-empty config/config-dev extension and its runtime references

icam-540 additionally retargets to jetson-orin-nx (its underlying SoM,
with default_target_board: icam-540), which resolves in the package feed.
Add top-level .gitignore rules so per-build generated files are never
committed from any reference:
- avocado.lock
- .avocado-desktop.json
- .avocado/
- .avocado-state

Previously only 1 of 30 references ignored these locally.
Reference-by-reference build check: for each reference runs
clean/unlock/install/build, then tears down to a clean git state and
strips build artifacts. Writes a live markdown report after each step
(scripts/build-report.md), continues past failures, and prints a
PASS/FAIL summary.
- elixir-phoenix: restore the avocado-ext-connect-config extension that
  `avocado connect clean` stripped from avocado.yaml during a verification
  run (unintended side effect, not part of the migration). Verified the
  reference still builds with it in place.
- build-check-references.sh: snapshot avocado.yaml per reference and restore
  it after teardown so `avocado connect clean` can no longer leak edits into
  tracked config; fix the REPORT_FILE default doc to match the code
  (scripts/build-report.md).
The reference shipped without the spec-required docs. Add a frontmatter
README (language / targets / topics) and a full getting_started guide
(prerequisites → install → build → deploy → verify → customize), sourced
from the reference's avocado.yaml and files/ scripts.
Bring the three remaining references into compliance with the docs spec
(spec_readme.md / spec_getting_started.md):
- imx8mp-npu-nnstreamer, imx8mp-npu-pose: prepend README frontmatter
  (language / targets / topics); README bodies unchanged.
- zephyr-imx8mp-evk: prepend README frontmatter and add the missing
  getting_started.md.

All references now pass the spec (frontmatter README + getting_started.md).
Teardown previously ran `git clean -fdx`, which deletes ALL untracked files
under a reference (including a developer's local work-in-progress). Switch to
`-X` (ignored-only): build state is fully gitignored so it's still removed,
while untracked non-ignored files are left in place and flagged by the
residual-changes check. Addresses PR review feedback.
- getting_started.md (dev, rubicon, webkit-ui, qemu-quickstart): drop the
  `- config` / `- config-dev` line from the "add/remove extensions" examples.
  The migration deleted those extensions, so the copy-paste blocks pointed at
  a non-existent extension.
- build-check-references.sh: the terminal summary piped the loop to `tee`,
  running it in a subshell so the pass/fail counters were lost (always printed
  PASS: 0 FAIL: 0). Redirect to the summary file and `cat` it, keeping the
  counters in the parent shell. The markdown report totals were already correct.

@jetm jetm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed cold at 7cf4106. The restructure is mechanically clean, and I want to say that first because most of what follows is edge cases: all 30 avocado.yaml files parse, all 30 carry rootfs: + initramfs: + a matching permissions.dev profile, every runtime extension name resolves, there are no config/config-dev/cli_requirement leftovers in YAML, no duplicate top-level keys, and the SDK image line is byte-identical across all 30 and matches the CLI's own scaffold (avocado-cli/configs/default.yaml:93). default_target_board is correct for CLI 0.37.0+ and keeps avocado-bsp-{{ avocado.target.board }} resolving to avocado-bsp-icam-540. The summary-counter fix in 7cf4106 is genuinely correct - a redirected for ... done > file does not subshell in bash, which I checked rather than assumed.

Five findings are inline. Separately, I independently landed on the two issues already raised in open threads here, and traced up some extra evidence worth having on each:

On the cli_requirement removal thread. Agreed, and it is the most serious thing in the PR. Two details that sharpen it. cli_requirement is not deprecated - it is enforced on every config load (avocado-cli/src/utils/config.rs:1415, called at :1732 and :3441), and the CLI's own avocado init template still emits cli_requirement: ">=0.41.0" (configs/default.yaml:1, asserted by src/commands/init.rs:1149). So if the PR body describes the key as deprecated, that premise is wrong. And the breakage is actually two distinct windows, not one: on CLI 0.26-0.35 the new blocks are silently dropped (no deny_unknown_fields anywhere in config.rs at any version), the build reports success, and the image ships root:*: in /etc/shadow so nobody can log in - but on 0.36-0.40.2 it is a hard error instead, because permissions is missing from the rootfs singleton whitelist at that version (git show 0.40.2:src/utils/config.rs:148-159), so rootfs: {permissions: dev} takes the named-map branch and dies with invalid type: string ... expected struct ImageConfig. Cryptic, but at least loud. Restoring cli_requirement: ">=0.41.0" across all 30 covers both.

On the icam-540 retarget thread. Agreed on the README/spec mismatch. Two corrections to the rationale, though, if it is going in the commit message: the icam-540 package feed does still resolve (2024/edge/target/icam-540/repodata/repomd.xml returns 200, revision 2026-05-01), it is just about two months staler than jetson-orin-nx at 2026-07-08 - so "stale" is accurate and "does not resolve" would not be. And I could not confirm avocado-bsp-icam-540 is installable against the jetson-orin-nx feed; no avocado-bsp-* packages showed up in any feed I could enumerate, so that one wants an actual avocado install against the reference rather than a reasoned argument.

Three more findings that I could not anchor inline, two of them because the files are not in this diff:

  • docker-registry/getting_started.md:78 and nvidia-deepstream/getting_started.md:99 still say "the default config extension sets an empty root password for development". 7cf4106 swept the YAML-snippet form of this out of the dev, qemu-quickstart, rubicon, and webkit-ui docs but missed the prose form in these two. A reader greps avocado.yaml for config, finds nothing, and cannot work out where the empty root password comes from. astra-1680-deepx/getting_started.md:93 has the wording you want.
  • docker-save/getting_started.md:221 and docker-registry/getting_started.md:367 both still run docker.io/avocadolinux/sdk:2024-edge while their own avocado.yaml now resolves to docker.io/avocadolinux/sdk:2024. The config change is right (it matches configs/default.yaml:93, and docs/.../0.41.0.md:94 documents dropping the channel suffix from the generated tag; distro.channel stays load-bearing because it feeds the dnf $releasever). Both tags currently resolve to the same digest, so nothing breaks - the docs are just stale.
  • Worth one sentence somewhere: moving the root password out of a confext and into the rootfs (avocado-cli/src/commands/rootfs/image.rs:309-322 edits $ROOTFS_WORK/etc in place pre-mkfs) means changing that password is no longer a ~KB confext delta shipped via avocado deploy - it is now part of the full .aos OS bundle. Still OTA-updatable, just a much bigger payload. No reference's getting-started doc mentions this, and the "Customize -> Permissions" bullets imply the old cheap path.

Traced and deliberately dropped, so nobody re-audits them: the iphone-travel-router password: "avocado" divergence (deliberate, documented in-file for Cockpit PAM); the removed distro.repo.url: http://localhost:8080 in astra-1680-deepx (that is a fix); the {{ avocado.distro.* }} to {{ config.distro.release }} switch (both namespaces are valid, and the config.* form fails loudly on a missing key, which is better); the dead-but-defined extensions deepx-app, htop, avocado-ext-connect (all pre-existing); and the redundancy between the new root .gitignore and the 28 per-reference ones that already carry .avocado/ and .avocado-state.

Comment thread .gitignore Outdated
Comment thread .gitignore
Comment thread astra-1680-deepx/README.md
Comment thread scripts/build-check-references.sh Outdated
Comment thread scripts/build-check-references.sh
jetson-trt landed on main after the ENG-2182 migration and still carried
`cli_requirement: ">=0.41.0"`. Remove it so no reference pins a CLI floor
(consistent with the other 30). The reference was otherwise already in the
new structure (top-level permissions/rootfs/initramfs, config.* SDK image).
Review fixes (jetm cold review):
- .gitignore: un-ignore avocado.lock (tracked pin file); keep ignoring
  .avocado/ / .avocado-state / .avocado-desktop.json
- untrack docker-{registry,save}/.avocado-state (machine-specific state);
  fix their .gitignore trailing slash so it's caught going forward
- spec_readme.md: add the real target ids used in frontmatter
  (grinn-astra-1680-sbc, ucm-imx8m-plus, jetson-agx-thor/-orin,
  jetson-orin-nano, rubikpi3, rzv2n-sr-som) + note the table isn't the
  definitive hardware list (docs site is)
- getting_started docs: drop stale `config` extension prose and the
  sdk:2024-edge tag (now sdk:2024)
- ros2-ufactory-lite6: set default_target to imx8mp-evk (was "*")

build-check-references.sh:
- resolve_target parses block-list supported_targets, warns instead of
  silently defaulting to qemuarm64
- residual-changes check tolerates staged avocado.yaml; teardown uses
  git clean -fdX (ignored-only) and unlock -t <target>
- record per-step install/build durations and per-failure reasons

scripts/build-report.md: full 31-reference report (25 pass / 6 fail) with a
Failures section documenting each failure's reason.
- zephyr-compile.sh: clear WEST_TOPDIR before `west init` when it holds a
  stale/partial checkout (a prior zephyrproject/zephyr without a valid .west),
  which otherwise aborts init with "target directory already exists". The fast
  incremental path (valid .west -> skip init -> west update) is unchanged.
- build-check-references.sh: teardown uses `git clean -ffdX` (double force) so
  west's nested git repos (zephyrproject/*, mcuboot, tools/*) are removed; a
  single -f skips nested repos and left a partial tree behind across runs.

@jetm jetm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All five of my earlier findings verified fixed at 066c025, checked against the tree rather than the replies: avocado.lock is un-ignored with a comment explaining why it is tracked, both .avocado-state files are out of the index with the trailing-slash ignore corrected and the script's restore hack removed, the three frontmatter target ids are in spec_readme.md, resolve_target now parses block and flow sequences and warns instead of silently defaulting to qemuarm64 (plus ros2-ufactory-lite6 no longer relies on the fallback), and the residual check tolerates a staged avocado.yaml with build-report.md now tracked. The atomic .tmp + mv on the report write was not something I raised - good addition.

I also accept the default_target / default_target_board close: if the split is intentional and the support matrix is the definitive hardware list, then icam-540 staying a valid id in spec_readme.md makes the README frontmatter validate, and my concern there is answered.

One disagreement, inline: the jetson-trt Orin targets. I probed the feed rather than reason about it, and only jetson-agx-thor resolves. Details in the comment.

jetson-trt Orin targets (jetson-trt/avocado.yaml:6)

Re-raising the target-slug finding with a feed probe, because I think the "Orin variants share the same feed" premise is wrong.** I probed the 2026/edge base repos directly rather than argue from inference:

jetson-agx-thor              200
jetson-agx-orin             404
jetson-orin-nano            404
jetson-agx-orin-devkit      404
jetson-orin-nano-devkit     404

(https://repo.avocadolinux.org/2026/edge/target/<slug>/repodata/repomd.xml. The 200 on jetson-agx-thor is what makes the 404s meaningful - the URL shape is right, those two base repos genuinely are not published.)

So a user who reads supported_targets and runs the documented avocado init --reference jetson-trt --target jetson-agx-orin gets nothing resolvable from avocado install -f. Note the -devkit spelling I mentioned last round does not rescue it either - I checked, and it 404s too, so there is no slug correction that makes those two work today.

If the Orin feeds are planned but not yet published, then advertising them here is a promise the repo cannot keep yet, and jetson-agx-thor should stand alone until they land (the comment above already says Thor is the testing target). If I am probing the wrong host or the CLI resolves these some other way, say so and I will drop it - but the 200/404 split above is the whole basis for my disagreeing with the close.

@nicksinas

Copy link
Copy Markdown
Collaborator Author

One disagreement, inline: the jetson-trt Orin targets. I probed the feed rather than reason about it, and only jetson-agx-thor resolves. Details in the comment.

jetson-trt Orin targets (jetson-trt/avocado.yaml:6)

Re-raising the target-slug finding with a feed probe, because I think the "Orin variants share the same feed" premise is wrong.** I probed the 2026/edge base repos directly rather than argue from inference:

jetson-agx-thor              200
jetson-agx-orin             404
jetson-orin-nano            404
jetson-agx-orin-devkit      404
jetson-orin-nano-devkit     404

I think this is OK for now. Publishing is happening this week. I think the fixes to all of the other references are more important. These targets being live in the 2026 feeds will happen within days. Plus this reference is already merged in main.

@jetm

jetm commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Fair enough on the Orin slugs - your call, and the reasoning holds: the reference is already on main, publishing is this week, and the other reference fixes matter more than a slug that will resolve within days. Dropping it.

The two changes since my last look both check out, and I verified the -ffdX rationale rather than taking it on faith, since "add another -f" is the kind of change that can be right for the wrong reason:

WEST_TOPDIR is ${BUILD_DIR}/zephyrproject and BUILD_DIR is ${HOST_SRC}/zephyr-m7/build, which zephyr-imx8mp-evk/.gitignore ignores as zephyr-m7/build/. So the west workspace does sit inside an ignored directory, which is what makes -X reach it at all - and a single -f refuses to recurse into the nested git repos west clones there, leaving the whole tree behind. -ff removes them. The fix is correct and so is the stated reason.

The stale-workspace guard in zephyr-compile.sh is the right shape too: west init aborts on an existing target dir, and the [ -e ] && [ ! -d .west ] test clears exactly the partial-checkout case without touching a valid workspace.

No further findings from me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants