From 20436dddc6b48856d8db9e7c6ddbd1e8143027c9 Mon Sep 17 00:00:00 2001 From: Kevin Wang Date: Thu, 24 Sep 2026 19:35:07 -0700 Subject: [PATCH] fix(dstack-mr-cli): default --hotplug-off to the VMM's default #1387 made the VMM default qemu_hotplug_off to true and made ACPI generation reject hotplug with GPU root ports, but left the CLI defaulting to false. A measurement of a default VMM CVM therefore came out wrong, and one with GPUs failed with HotplugWithRootPorts, unless --hotplug-off true was passed. Signed-off-by: Kevin Wang --- CHANGELOG.md | 2 +- dstack/dstack-mr/cli/src/main.rs | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c312372d4..d810d9148 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,7 +62,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - gateway: WireGuard config is applied by one background worker instead of under the routing lock. `RegisterCvm` returns before the kernel has the peer, and every registration, removal, reload and startup queues a full apply that the worker coalesces and retries with backoff. A failure to render or write the config no longer stops the gateway from starting and is no longer returned by `Admin.RemoveCvm`; it is retried like a failed `wg syncconf`, so watch the WireGuard reconfigure failure counter instead -- vmm: `qemu_hotplug_off` defaults to `true`, and ACPI table generation (verifier, KMS, `dstack-mr`) rejects PCI hotplug with GPU or NVSwitch root ports. QEMU emits hotplug AML for those root ports that the generator does not model, so such CVMs failed lite verification with an ACPI digest mismatch. **Breaking for hosts that relied on the default:** every CVM gets different ACPI tables, and so different RTMR values, on its next start, so anything pinning those measurements (such as the KMS aggregated-MR allowlist) must be updated. Hosts without GPUs can set `qemu_hotplug_off = false` to keep their current measurements +- vmm: `qemu_hotplug_off` defaults to `true`, and ACPI table generation (verifier, KMS, `dstack-mr`) rejects PCI hotplug with GPU or NVSwitch root ports. QEMU emits hotplug AML for those root ports that the generator does not model, so such CVMs failed lite verification with an ACPI digest mismatch. **Breaking for hosts that relied on the default:** every CVM gets different ACPI tables, and so different RTMR values, on its next start, so anything pinning those measurements (such as the KMS aggregated-MR allowlist) must be updated. Hosts without GPUs can set `qemu_hotplug_off = false` to keep their current measurements. `dstack-mr measure --hotplug-off` defaults to `true` to match, so a measurement of a VMM-default CVM, including one with GPUs, needs no extra flag - certbot: the standalone `certbot` issues every renewal under a new private key, as the gateway already does, instead of reusing `live/key.pem`. Anything pinning the certificate's public key (DANE/TLSA, SPKI pins) must be updated on each renewal. `live/cert.pem` and `live/key.pem` now resolve through a `live/.current` symlink that is swapped atomically; existing layouts migrate on the next publish - dstackup: the default KMS image is digest-pinned (`dstacktee/dstack-kms:0.5.11@sha256:84b793fe…`) instead of a mutable tag, since it is measured into the compose hash. Fresh installs therefore register a different KMS app id; existing deployments are unaffected - kms: client certificates are authenticated by the attestation they carry rather than by their issuer. Rocket configures mutual TLS through rustls' `WebPkiClientVerifier`, which pins a CA — but an RA-TLS certificate is self-issued and carries its identity in a TEE quote, so there is nothing to chain to. `GetTempCaCert` bridged the gap by handing every caller a shared CA private key purely so the minted certificate would chain somewhere; the CA established nothing (its key is public by design, and the endpoint is unauthenticated) and the check that has always carried the meaning is the quote verification that runs afterwards. The KMS now hands rustls a verifier that requires an attestation and ignores the issuer. Nothing changes for callers: guests and KMS-to-KMS onboarding still mint their client certificates from the temp CA, and those are now accepted for the attestation they carry. What changes is that the TLS layer went from admitting any certificate signed by a public key to requiring an attested one, and that a self-issued certificate is now accepted — which is what lets callers be migrated off `GetTempCaCert` in a follow-up. `[rpc.tls.mutual]` is no longer the trust anchor and is dropped from `kms.toml` and the KMS config templates; leaving it in an existing deployment's config is inert. The gateway's `[tls.mutual]` is unaffected — it pins the KMS root CA, which is a real trust anchor diff --git a/dstack/dstack-mr/cli/src/main.rs b/dstack/dstack-mr/cli/src/main.rs index 9c54fdd6e..cbc3c509d 100644 --- a/dstack/dstack-mr/cli/src/main.rs +++ b/dstack/dstack-mr/cli/src/main.rs @@ -81,8 +81,8 @@ struct MachineConfig { #[arg(long, default_value = "false")] swtpm: bool, - /// Disable hotplug - #[arg(long, default_value = "false")] + /// Disable hotplug; matches the VMM default (`qemu_hotplug_off = true`) + #[arg(long, default_value = "true")] hotplug_off: Bool, /// Enable root verity