fix(os/image): enable MMCONFIG so Blackwell GPUs can be probed

[kvinwang]

Why

pci=nommconf confines PCI config space access to the legacy CF8h/CFCh port I/O path, which physically cannot reach past the first 256 bytes. pci_find_ext_capability() then short-circuits on dev->cfg_size <= PCI_CFG_SPACE_SIZE, so the whole PCIe extended config space (offset >= 0x100) is unreadable.

Blackwell GPU drivers walk that space to find NVIDIA's vendor DVSEC. With MMCONFIG off the reads return 0xFFFFFFFF, the driver asserts on every probe, and every later cuInit() fails with CUDA_ERROR_SYSTEM_NOT_READY (802). No other mechanism reaches extended config space, so GPU passthrough cannot work without MMCONFIG. This is a property of the access method, not a driver bug.

What changed

Dropped pci=nommconf from the guest kernel command line, in both places it is stated — the shared mkosi definition (os/image/kernel-cmdline.sh) and the yocto UKI recipe (dstack-uki.bb) — plus a comment recording why it must not come back.

What deliberately did not change

pci=noearly stays. It gates a different mechanism: early type 1 scanning, which runs before ACPI is parsed and dispatches into vendor fixups keyed on a vendor/device/class triple read straight out of host-controlled config space. early_quirks() is the only caller of the early_qrk[] handlers, so disabling it removes that code path rather than deferring it. GPU drivers bind during normal PCI enumeration and do not depend on it, and Intel's confidential-computing guest hardening names early PCI code as something to disable.

If a B300 bring-up turns out to need pci=noearly removed as well, that should be a separate change with its own evidence — this PR is the minimal one that is provably required.

Security note

Restoring MMCONFIG re-exposes ~15 extended-capability parsers (SR-IOV, AER, ACS, DOE, Resizable BAR, ...) to host-controlled data. Scoping that honestly:

• Not a new attack surface class. The first 256 bytes are already fully host-controlled and already parsed (MSI, MSI-X, PM, VPD).
• The extended capability walker is bounded — PCI_FIND_NEXT_EXT_CAP caps at (4096-256)/8 iterations and PCI_EXT_CAP_NEXT() masks offsets to 0xffc, so no loop and no out-of-bounds read.
• MCFG itself is covered by attestation: dstack-mr independently regenerates the QEMU ACPI tables and measures loader/rsdp/tables into RTMR0, so a host lying about the ECAM base is detectable — provided the verifier compares against the computed expectation rather than replaying the guest event log.

Testing

• bash -n and shellcheck on kernel-cmdline.sh.
• Verified the two backends still emit an identical token set (diffed the generated command line against UKI_CMDLINE_BASE).

Impact

This changes the measured guest command line. The UKI PE section digests, metadata.json and the derived RTMRs all shift, so attestation baselines have to be refreshed with the image that ships this.

Follow-up

The command line is stated independently in the two image backends and only the mkosi path is guarded by the "must not be restated" check in os/mkosi/tests/acceptance.sh. This PR had to edit both by hand. A cross-backend consistency check would be worth adding separately.

[Copilot]

🔵 Needs a closer look

It changes security/attestation-sensitive guest kernel cmdline behavior in a way that warrants final human review despite the diff being small.

Pull request overview

This PR updates the guest OS kernel command line to re-enable PCIe MMCONFIG access (by removing pci=nommconf), which is required for NVIDIA Blackwell GPU passthrough because the driver needs PCIe extended config space to probe vendor DVSEC.

Changes:

• Removed pci=nommconf from the mkosi-generated guest kernel cmdline (os/image/kernel-cmdline.sh).
• Removed pci=nommconf from the Yocto UKI cmdline base (os/yocto/.../dstack-uki.bb) and added a sync note pointing to the canonical cmdline definition.

File summaries

File	Description
os/yocto/layers/meta-dstack/recipes-core/images/dstack-uki.bb	Drops pci=nommconf from UKI_CMDLINE_BASE and adds a comment to keep it aligned with the canonical cmdline definition.
os/image/kernel-cmdline.sh	Drops pci=nommconf from dstack_kernel_cmdline() and documents why MMCONFIG must remain enabled (and why pci=noearly stays).

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.