Add 9p backends: diod over TCP + QEMU virtio-9p via virtme-ng (vagrant/QEMU stack)

.github/matrix.yaml

@@ -34,6 +34,15 @@ src-dir: /opt/eval-under-src
 # did upstream add a test?).
 refs:
   git: v2.55.0
+  # Guest kernel for the 9p-virtio backend's CI rows (vng downloads the
+  # Ubuntu mainline build; tag-shaped, leading "v" required). The v9fs
+  # *client* is the kernel, and its semantics move between releases
+  # (cache-mode rework in 6.4, netfs buffered writes in 6.8) -- an
+  # unpinned guest kernel would make a newly-red 9p cell ambiguous in
+  # exactly the way these pins exist to prevent. Local runs default to
+  # the host kernel instead; this pin is handed to the backend
+  # explicitly by bin/ci/run-under.sh.
+  9p-kernel: v6.8
   # pjd/pjdfstest carries exactly one tag upstream, "0.1" (2016), and it
   # no longer builds: major()/minor()/makedev() moved to
   # <sys/sysmacros.h> in glibc 2.28 and the tree compiles with -Werror,
@@ -90,6 +99,12 @@ targets:
 
 # Row order of the README CI matrix. `version` is the literal "n/a" for
 # backends with nothing to pin (see bin/ci/install-backend.sh).
+#
+# `runs-on` (optional, default ubuntu-22.04) picks the runner image per
+# row: BeeGFS DKMS constrains its rows to 22.04, while the 9p rows want
+# 24.04, where virtme-ng/virtiofsd are packaged. Mixing images costs
+# some cross-row comparability (different host kernel and tool
+# versions) -- use it only when a backend actually needs it.
 backends:
   - backend: beegfs
     version: 7.4.6
@@ -106,3 +121,18 @@ backends:
   - backend: loop
     version: ext4
     label: Loop ext4
+  # v9fs client against diod (9P2000.L over TCP on localhost) -- the
+  # no-VM 9p row, architecturally the NFS backend's sibling.
+  - backend: 9p-tcp
+    version: n/a
+    label: 9p tcp (diod)
+    runs-on: ubuntu-24.04
+  # v9fs client against QEMU's virtfs server over virtio, inside a
+  # virtme-ng guest -- the vagrant-libvirt "9p" synced-folder stack.
+  # "mapped" = security_model=mapped-xattr (vagrant accessmode
+  # "mapped"); vagrant's *default* (passthrough under an unprivileged
+  # QEMU) is a future row, see GOTCHAS.md "Not yet covered".
+  - backend: 9p-virtio
+    version: mapped
+    label: 9p virtio (mapped)
+    runs-on: ubuntu-24.04

.github/workflows/test.yaml

@@ -46,11 +46,12 @@ jobs:
   test:
     needs: matrix
     name: ${{ matrix.name }}
-    # ubuntu-22.04: kernel 5.15/6.5, within BeeGFS 7.4.x and 8.x DKMS
-    # support. ubuntu-24.04 hosted runners ship 6.17-azure which BeeGFS
-    # kernel modules cannot build against. NFS + loop backends don't
-    # need this, but sharing the runner OS keeps the matrix uniform.
-    runs-on: ubuntu-22.04
+    # Per-cell runner image, from the backend row's optional `runs-on`
+    # in .github/matrix.yaml (default ubuntu-22.04: its kernel stays
+    # within BeeGFS DKMS support, while ubuntu-24.04 ships an azure
+    # kernel the BeeGFS modules cannot build against). The 9p rows
+    # override to ubuntu-24.04, where virtme-ng/virtiofsd are packaged.
+    runs-on: ${{ matrix.runs-on }}
     timeout-minutes: 60
     strategy:
       fail-fast: false
@@ -98,12 +99,13 @@ jobs:
           name: logs-${{ matrix.slug }}
           path: |
             /var/log/beegfs-*
+            /var/log/eval-under-9p-virtio.log
             /opt/eval-under-src/git/t/test-results/**
           if-no-files-found: ignore
 
       # One tiny artifact per cell, read back by the `badges` job. A
       # matrix job cannot contribute to a `needs.*.outputs` map, so this
-      # is how a fan-out reports 20 individual verdicts to a fan-in.
+      # is how a fan-out reports its per-cell verdicts to a fan-in.
       - name: Record cell result
         if: always()
         run: |

GOTCHAS.md

@@ -86,6 +86,91 @@ a kernel module built against the runner's kernel.
 | Client conf | `fixtures/beegfs/beegfs-client.conf.template` | Auth disabled, all daemons on `127.0.0.1`, non-default ports (8004-8008) so nothing collides with the runner. |
 | `sysMountSanityCheckMS` | `0` **on v8 only** | BeeGFS v8 dropped the standalone `beegfs-helperd` binary; with no helperd the sanity check cannot complete and the mount would hang. v7 keeps the check. |
 
+### 9p tcp (`bin/eval-under-9p-tcp`)
+
+A fresh directory exported on `127.0.0.1` by diod (LLNL's 9P2000.L
+server, Ubuntu universe) and mounted with the kernel v9fs client --
+architecturally the NFS backend's sibling. Same client code as the
+virtio row below, different server.
+
+| Knob | Value | Why |
+| --- | --- | --- |
+| Server | `diod --foreground --no-auth --listen 127.0.0.1:5640 --export <dir>` | Loopback throwaway export; munge auth has no place here. |
+| Server identity | the *invoking user* (single-user mode), unless `--run-as-root` | diod's documented "simplest and safest" mode; also means the server itself needs no privilege. |
+| Mount (default) | `-t 9p -o trans=tcp,port=5640,version=9p2000.L,aname=<dir>,uname=<user>,access=<uid>` | diod's documented single-user pairing. `access=<uid>` restricts the mount to that uid -- root included, which is why the usability probe runs as the invoker. |
+| Mount (`--run-as-root`) | `...,uname=root,access=client`, diod as root | diod's documented multi-user (NFS-like) pairing; the 9p analog of `--no-root-squash`, applied by `target_needs_root()`. |
+| `msize` | not set -- kernel requests its default (128 KiB since 5.15), **diod caps at 64 KiB** | The `/proc/mounts` line the backend echoes records the effective value; results are relative to it. |
+| `cache` | not set -- kernel default, which is **no caching in every kernel era** | `--cache loose` exists to reproduce the classic stale-read foot-gun deliberately. |
+
+**diod locking is whole-file.** diod implements 9P `Tlock` as BSD
+`flock()` on the host file: byte-range `fcntl` locks collapse to
+whole-file locks, and diod's own docs say distributed record locking
+will deadlock ("test your use case!"). A locking red on this row is a
+diod-server finding first -- compare with the virtio row, whose server
+fakes locks differently (below), before blaming the client.
+
+### 9p virtio (`bin/eval-under-9p-virtio`)
+
+QEMU's virtfs server (`-virtfs local,...`) exporting a fresh host dir
+into a virtme-ng guest booted from the host's own root filesystem; the
+suite runs *inside the guest* on a `mount -t 9p -o trans=virtio` mount.
+This is the vagrant-libvirt `type: "9p"` synced-folder stack.
+
+| Knob | Value | Why |
+| --- | --- | --- |
+| Server | QEMU `-virtfs local,path=<dir>,mount_tag=eval9p,security_model=mapped-xattr` (the `mapped` row) | vagrant-libvirt `accessmode: "mapped"`. Ownership/mode/devices are faked in `user.virtfs.*` xattrs on the host files. |
+| Guest kernel (CI) | pinned in `.github/matrix.yaml` `refs: 9p-kernel` (`vng --run`, Ubuntu mainline build) | The v9fs *client is the kernel*: cache-mode rework landed in 6.4, netfs buffered writes in 6.8. Unpinned, a newly-red cell can't distinguish "filesystem regressed" from "client changed". Local runs default to the host kernel instead. |
+| Guest machine | `vng --disable-microvm` | vng's microvm has no PCI bus stock guest kernels can enumerate; `-virtfs` is virtio-9p-**pci**, so without this the mount tag silently never appears. |
+| Mount | `-t 9p -o trans=virtio,version=9p2000.L` (msize/cache unset = kernel defaults, as above; virtio transport caps msize at 512000) | The backend echoes the guest's `/proc/mounts` line and `uname -r` -- that pair is what a result is relative to. |
+| Command identity | dropped to the invoking user via `runuser` in the guest; `--run-as-root` keeps root (`target_needs_root()` rows) | A vagrant user's synced folder is an unprivileged view; guest scripts otherwise run as root in virtme. |
+| `writeout` | not set (QEMU default) | vagrant-libvirt sets `wrpolicy="immediate"`; `--writeout immediate` reproduces that when wanted. |
+
+**mapped-xattr absorbs privileged operations.** Under this security
+model `chown` "succeeds" into an xattr, and `mknod` creates a host
+*regular file* wearing a device-node xattr -- so pjdfstest's and
+stress-ng's privileged assertions pass or fail against the *mapping
+layer*, not a kernel. That is the measurement (it is exactly the
+"returns success while doing the wrong thing" class stress-ng exists to
+catch), but read those cells with this table in hand. Corollaries: unix
+sockets and FIFOs work unprivileged; host-side the backing files read
+as mode 0600/0700 owned by the QEMU user (`--keep` shows exactly that).
+
+**QEMU virtfs locking is a polite lie.** QEMU's 9p server answers every
+`TLOCK` with success (single-client assumption); the guest kernel takes
+the local VFS lock first, so locking is coherent *within* the guest and
+invisible to the host. Locks therefore mostly "work" on this row where
+diod's whole-file behavior differs -- a divergence between the two 9p
+rows on locking tests is expected, not noise.
+
+**Other v9fs client facts red cells trace back to** (both 9p rows):
+writable `MAP_SHARED` mmap fails `EINVAL` under the default no-cache
+mode (read-only mmap -- git's main use -- works); `O_TMPFILE` is
+unsupported (`EOPNOTSUPP`); there is no *remote*-change notification in
+the protocol at all (inotify for the guest's own operations works
+normally), which is also why `cache=loose` can serve stale data
+indefinitely.
+
+**vng's `--rwdir` shares are not uid-faithful** (measured on the
+runner's exact stack): writes arrive on the host as the sharing
+daemon's identity -- root when the backend runs under sudo -- and a
+non-root guest process gets `EACCES` inside a directory it just
+"successfully" created. This is why the git target's
+`t/test-results/**` travels via the backend's `--copy-out` (through
+the 9p export itself, ownership normalized to the invoker) rather than
+via `--share-rw`, and why the source tree's in-suite writes
+(`chainlinttmp`, `test-results`) are left to the guest's uid-faithful
+tmpfs overlay instead of a share.
+
+**The guest is disposable; the log is not.** Everything the suite and
+the guest console print is teed to `/var/log/eval-under-9p-virtio.log`
+on the host (stage2 appends the guest `dmesg` tail on failure), because
+by the time `dump-failure-logs.sh` runs, the guest -- and the backing
+dir, via teardown -- are gone. vng quirk worth knowing: exit **255**
+is also its "guest died before reporting" sentinel, so a suite exiting
+255 is indistinguishable from a crash; and exit **124** under CI is the
+backend's own `--vm-timeout` catching a boot/mount hang that the
+in-guest per-target timeout cannot see.
+
 ## Known-red cells
 
 A red cell here is a finding, not a bug report against this repo. These
@@ -180,6 +265,61 @@ layers on top, or in the syscalls the pjdfstest column is flagging.
 Pre-dates the matrix; ext4 is the control row, so this one *is* a real
 bug worth chasing rather than a filesystem property.
 
+### `9p * / git-annex test` -- hangs, on both servers identically
+
+The suite's very first group (`testremote type git`) fails `init`
+within seconds ("git-annex: Not initialized"), then the run goes
+silent inside `unavailable remote / removeKey` until the per-target
+timeout kills it (exit 124) ~40 minutes later. Identical on diod and
+on QEMU virtfs, so it is v9fs-client/protocol semantics rather than a
+server quirk -- and diod's own docs warn that distributed record
+locking "will deadlock", which is the pre-registered suspect. A hang
+*is* the finding: git-annex on a real vagrant 9p share stalls the same
+way. Not yet broken down further; the cell burns its full timeout by
+design (a runaway suite reports as `timeout`, not a bare cancellation).
+
+### `9p * / stress-ng` -- the unlinked-open-file class, server-dependent
+
+Both rows run all 20 stressors to a clean tally; the failures are
+`fstat`/`ftruncate` returning **ENOENT on files that are open but
+unlinked** (stress-ng's create-unlink-keep-fd pattern):
+
+- diod: failed `fallocate`, `hdd`, `copy-file` (17 passed);
+- QEMU virtfs: failed `copy-file`, `hdd` -- **`fallocate` passes**, a
+  clean server divergence worth keeping both rows for.
+
+On the QEMU row the guest kernel (pinned v6.8) also logs WARN traces
+in `v9fs_fid_lookup_with_uid -> v9fs_vfs_getattr_dotl` while these
+stressors run -- the client side of the same fid-on-unlinked-file gap,
+and part of why the guest kernel is pinned: this signature is
+kernel-version-specific.
+
+### `9p * / pjdfstest`
+
+Runs to completion on both rows (238 files, ~8800 assertions, ~3 min).
+On diod the divergence concentrates in: every `*/03.t` (the
+long-pathname scripts) across chown/ftruncate/link/mkdir/mkfifo/mknod/
+open/rmdir/symlink/truncate/unlink; `mkdir|mkfifo|mknod|open/00.t`
+assertions 25-27; the `open/06.t` flags matrix (62 of 144);
+`rename/10.t` (6 of 2099); `unlink/14.t` #4 (the same assertion the
+NFS row flags); `utimensat/08.t`. The QEMU-virtfs row's tally is in
+its `logs-*` artifact; per-assertion breakdown not yet done (same
+status as the BeeGFS rows).
+
+### `9p * / git testsuite`
+
+The full `t0*.sh t1*.sh` selection runs under prove on both rows and
+ends with totals; per-script `.out` files are in the artifacts (on the
+virtio row they travel out of the guest via the backend's
+`--copy-out`). Roll-up: diod 12 scripts / 194 failed assertions, QEMU
+virtfs 10 / 177. Shared failures include `t1517-outside-repo` (104)
+and `t0450-txt-doc-vs-help` (51) -- both green on the ext4 control
+row, so 9p-related, not yet run down. **`t1050-large` (15 failures)
+is diod-only** -- consistent with diod's 64 KiB msize cap and the
+unlinked-file gap above; it passes under QEMU virtfs. Several of the
+remaining entries are `# TODO known breakage` noise the dump's
+`not ok` count includes.
+
 ## Red that is not a finding
 
 Distinct from the cells above: these are harness races, and the fix is in
@@ -228,6 +368,24 @@ check the mount actually came up.
   `actimeo=0`), locking (`lock` vs `nolock`, and whether `rpc.statd` is
   even up), `sync` vs `async` on the export, and squashing. Each is a
   plausible row of its own.
+- **The actual vagrant-libvirt 9p default.** vagrant-libvirt's default
+  `accessmode` is `passthrough`, and distro libvirt runs QEMU as an
+  unprivileged user -- so the out-of-the-box vagrant 9p share is
+  *passthrough served by a QEMU that cannot chown*, the very
+  configuration behind the classic "permission surprises". That is a
+  third variant, distinct from both the current `mapped` row and
+  passthrough-as-root. Reproduce it locally today:
+  `bin/eval-under-9p-virtio --security-model passthrough` run
+  *without* sudo. A `9p-virtio / passthrough` CI row (root QEMU) is the
+  cheap second row; the unprivileged flavour needs a small backend
+  tweak to skip sudo on that path.
+- **9p cache/msize variants.** `--cache loose` (the vagrant stale-read
+  foot-gun) and a small-`msize` row are one matrix line each once the
+  base rows have settled.
+- **virtiofs.** The designated successor to virtio-9p in the same
+  vagrant/QEMU role (`type: "virtiofs"`), and the natural control row
+  for "is this 9p, or any VM-shared filesystem?". Nearly free once the
+  vng plumbing exists: same guest, different device and server.
 - **Per-assertion breakdown** of the BeeGFS pjdfstest failures.
 - **A `Loop vfat` mask variant**, if we ever want to separate "vfat is
   not POSIX" from "vfat mounted with defaults is not POSIX".