fix(os): drop guest kernel drivers no CVM can reach - #1160
Merged
Merged
Conversation
The guest kernels inherit a bare-metal device set from their baselines -- x86_64_defconfig for mkosi, the linux-yocto machine config for yocto -- and neither fragment says anything about it. So both images ship Broadcom, Intel 1G, Marvell, nForce and Realtek NIC drivers, PCMCIA, AGP, Macintosh platform glue and legacy NVRAM, none of which can appear in a CVM. A host that presents the matching PCI IDs can steer the guest into any of them, and the kernel config is the only device allowlist this project actually has: Intel's authorize_allow_devs filter is out-of-tree and TDX-only, while CONFIG_* is enforced at link time and applies to TDX and SEV-SNP alike. Also dropped are two early-boot debug paths and one egress channel that are harder to justify than the drivers. PROVIDE_OHCI1394_DMA_INIT exists to give a FireWire device access to memory before the kernel is up, which is the primitive an untrusted host would want rather than a feature a tenant needs; EARLY_PRINTK_DBGP is its USB debug port equivalent. NETCONSOLE ships the kernel log to a UDP peer, configurable at runtime by root. Both fragments carry the same list so the two guest kernels keep the same device policy, and both note that passing a ConnectX NIC through later means adding CONFIG_MLX5_CORE and the INFINIBAND stack, not relaxing this. Verified by running each backend's own kconfig flow against linux 6.18.39. mkosi: x86_64_defconfig, merge_config.sh, olddefconfig, then check-kernel-config.sh over the fragment -- exit 0, all sixteen symbols end up unset. yocto: bitbake -c configure virtual/kernel, same sixteen unset in the resulting .config. CONFIG_VIRTIO_NET, CONFIG_GVE, CONFIG_ENA_ETHERNET and CONFIG_DRM are unaffected in both. Note for review: os/yocto's dstack.cfg is not gated by check-kernel-config.sh today (export-artifacts.sh only gates dstack-docker.cfg, because dstack.cfg carries six lines the build does not satisfy), so the yocto half of this rests on the configure run above rather than on CI. Gating it is worth a follow-up.
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Kernel configuration changes affect guest boot/runtime behavior and the PR notes the yocto fragment isn’t CI-gated, so a human should confirm end-to-end image builds/boot networking across supported platforms.
Pull request overview
This PR tightens the guest kernel “device allowlist” by explicitly disabling a set of bare‑metal network/bus/platform drivers (and a few early-debug/egress features) that dstack CVMs should never be able to use, reducing reachable kernel attack surface across both the mkosi and yocto guest images.
Changes:
- Add explicit
CONFIG_*=ndisables for legacy PCI NIC drivers and other bare-metal subsystems that cannot appear in a CVM. - Disable early-boot debug paths and
NETCONSOLEin the mkosi kernel fragment with detailed rationale. - Mirror the same device policy in the yocto kernel fragment to keep both guest kernels aligned.
File summaries
| File | Description |
|---|---|
| os/yocto/layers/meta-dstack/recipes-kernel/linux/files/dstack.cfg | Adds a block disabling unreachable bare-metal drivers/features to align yocto guest policy with mkosi. |
| os/mkosi/components/kernel/kernel.config | Adds the same disables plus more detailed security rationale for early-debug and netconsole-related symbols. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Both guest kernels inherit a bare-metal device set from their baselines —
x86_64_defconfigfor mkosi, the linux-yocto machine config for yocto — and neither fragment says anything about it. So both images currently ship:None of these can appear in a CVM. A dstack guest sees virtio, plus gVNIC on GCP and ENA on AWS — all three already enabled deliberately. Everything else is driver code that a host presenting the matching PCI IDs can steer the guest into, through ordinary enumeration.
This matters because the kernel config is the only device allowlist this project actually has. Intel's
authorize_allow_devsdevice filter lives in the out-of-treeccc-linux-guest-hardeningpatchset and is TDX-only (arch/x86/kernel/tdx-filter.cdoes not exist upstream), while dstack supports TDX and SEV-SNP from one image.CONFIG_*is enforced at link time — the code is not present at all — and applies to both.The fragments already do this well for whole subsystems (
WLAN,BT,USB_SUPPORT,SCSI,ATA,SOUND,INPUT,KEXEC,HIBERNATION, …). This extends the same policy to what the baseline drags in underneath.Beyond the drivers
Three entries are harder to justify than the rest and are worth calling out:
PROVIDE_OHCI1394_DMA_INITexists to give a FireWire device access to memory before the kernel is up. That is the primitive an untrusted host would want, not a feature a tenant needs.EARLY_PRINTK_DBGPis the USB debug port equivalent.NETCONSOLEships the kernel log to a UDP peer, configurable at runtime by root, and nothing in the image uses it.Testing
Each backend's own kconfig flow, run against linux 6.18.39:
mkosi —
make x86_64_defconfig,merge_config.sh,make olddefconfig, then the repo's owncheck-kernel-config.shover the fragment:All sixteen symbols end up unset.
yocto —
bitbake -c configure virtual/kernel, then the resulting.config: same sixteen unset.Unaffected in both:
CONFIG_VIRTIO_NET=y,CONFIG_GVE=y,CONFIG_ENA_ETHERNET=y,CONFIG_DRM=y,CONFIG_NET_VENDOR_GOOGLE=y,CONFIG_NET_VENDOR_AMAZON=y.Verification mattered here rather than being a formality:
check-kernel-config.shfails the build when a fragment line does not surviveolddefconfig, so a symbol something else selects back on would have broken CI rather than silently doing nothing.Future RDMA passthrough
Both fragments carry a note that passing a ConnectX NIC into the guest later is an addition —
CONFIG_MLX5_CORE,CONFIG_MLX5_INFINIBAND,CONFIG_INFINIBAND,CONFIG_INFINIBAND_USER_ACCESS— and not a relaxation of anything here. Those symbols are absent from both baselines today, so the two are orthogonal.Note for review
os/yocto'sdstack.cfgis not gated bycheck-kernel-config.sh:export-artifacts.shonly gatesdstack-docker.cfg, becausedstack.cfgcarries six lines the build does not satisfy (HOTPLUG_CPU/SCSI/INPUTforced back on by machine-level features, andTLS/CRYPTO_GCM/CRYPTO_CHACHA20POLY1305not asserted). So the yocto half of this rests on the configure run above rather than on CI. Gating it — which also means resolving those six — is worth a follow-up.