Reach Cloud Hypervisor parity for the NVX microVM backend - #8983
Conversation
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Credential filtering, write-policy traversal, guest permissions, and copy-back ownership currently have blocking correctness and security defects.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 4
Open (10)
What changed in this PR
Adds NVX workspace staging/copy-back, filesystem write controls, environment passthrough, workdir support, and end-to-end smoke coverage.
Changes:
- Adds workspace/tool-cache exports with post-run synchronization.
- Adds guest environment scripts and write-policy enforcement.
- Documents runtime constraints and introduces Copilot smoke coverage.
| File | Description |
|---|---|
src/types/runtime-options.ts |
Defines NVX mount policies. |
src/nvx/workspace-layer.ts |
Implements staging and copy-back. |
src/nvx/workspace-layer.test.ts |
Tests workspace synchronization. |
src/nvx/workspace-export.ts |
Resolves guest exports. |
src/nvx/workspace-export.test.ts |
Tests export validation. |
src/nvx/runtime-validation.ts |
Validates workdirs and policies. |
src/nvx/runtime-validation.test.ts |
Tests compatibility checks. |
src/nvx/runtime-backend.ts |
Integrates workspace lifecycle. |
src/nvx/runtime-backend.test.ts |
Updates backend tests. |
src/nvx/preflight.ts |
Requires copy-back tools. |
src/nvx/manager.ts |
Coordinates staging and extraction. |
src/nvx/index.ts |
Exports new NVX APIs. |
src/nvx/guest-environment-builder.ts |
Builds guest environments. |
src/nvx/guest-entrypoint.ts |
Generates guest run scripts. |
src/nvx/guest-entrypoint.test.ts |
Tests script generation. |
src/nvx/filesystem-write-policy.ts |
Plans writable guest paths. |
src/nvx/filesystem-write-policy.test.ts |
Tests write-policy planning. |
src/nvx/filesystem-builder.ts |
Preserves layer ownership. |
src/config-mapper.ts |
Maps mount-policy configuration. |
src/config-file.ts |
Types the new config field. |
src/commands/build-config.ts |
Parses NVX mount policies. |
src/cli-options.ts |
Adds the CLI option. |
src/awf-config-schema.json |
Updates runtime schema copy. |
docs/nvx-security-design.md |
Documents security and fallback behavior. |
docs/awf-config.schema.json |
Updates canonical configuration schema. |
docs/awf-config-spec.md |
Documents configuration usage. |
.github/workflows/smoke-nvx-copilot.md |
Adds NVX Copilot smoke coverage. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| const normalized = path.posix.normalize(containerWorkDir).replace(/\/+$/, '') | ||
| || NVX_GUEST_WORKSPACE; | ||
| if ( | ||
| normalized !== NVX_GUEST_WORKSPACE && | ||
| !normalized.startsWith(`${NVX_GUEST_WORKSPACE}/`) | ||
| ) { |
| function isReservedGuestTarget(target: string): boolean { | ||
| const reserved = [path.dirname(NVX_GUEST_RUN_SCRIPT), NVX_GUEST_RUN_SCRIPT, NVX_GUEST_HOME]; | ||
| return reserved.some((entry) => entry === target || isWithin(entry, target)); | ||
| } |
| await copySafeTree(source, destination, source, (relative) => excluded.some( | ||
| (credential) => relative === credential || relative.startsWith(`${credential}/`), | ||
| )); |
| async function describeHostEntry(absolutePath: string, stat: Stats): Promise<string> { | ||
| if (stat.isSymbolicLink()) return `symlink:${await fs.readlink(absolutePath)}`; | ||
| if (stat.isDirectory()) return `directory:${stat.mode & 0o7777}`; | ||
| if (stat.isFile()) return `file:${stat.mode & 0o7777}:${stat.size}:${stat.mtimeMs}`; | ||
| return 'other'; |
| if (!isNvxWritableGuestPath(this.config.writePlan, guestPath)) { | ||
| outcome.rejected.push(guestPath); | ||
| continue; | ||
| } |
| if (stat.isDirectory()) { | ||
| await fs.mkdir(hostPath, { recursive: true, mode: stat.mode & 0o7777 }); | ||
| await this.mergeTree(childUpperPath, exportEntry, outcome); | ||
| continue; | ||
| } |
| await fs.rm(hostPath, { recursive: true, force: true }); | ||
| await fs.copyFile(childUpperPath, hostPath); | ||
| await fs.chmod(hostPath, stat.mode & 0o7777); |
| if (stat.isSymbolicLink()) { | ||
| await options.lchown(absolute, writable ? options.uid : 0, writable ? options.gid : 0); | ||
| return; | ||
| } | ||
| await options.chown(absolute, writable ? options.uid : 0, writable ? options.gid : 0); | ||
| const mode = stat.mode & 0o7777; | ||
| await fs.chmod(absolute, writable ? mode | 0o600 : mode & ~0o222); |
| // Overlay records a deletion as a 0/0 character device in the upper layer; | ||
| // a plain file stands in for it here since mknod requires privileges. | ||
| await fs.writeFile(path.join(guestWorkspace, 'README.md'), ''); |
|
@copilot address review feedback |
|
✅ Copilot review passed with no inline comments. @copilot Add the |
Documentation PreviewDocumentation has been built for this PR. To view locally:
Built from commit ea8f8cd |
|
| Metric | Base | PR | Delta |
|---|---|---|---|
| Lines | 92.82% | 92.74% | 📉 -0.08% |
| Statements | 91.36% | 91.26% | 📉 -0.10% |
| Functions | 89.78% | 89.25% | 📉 -0.53% |
| Branches | 84.64% | 84.43% | 📉 -0.21% |
📁 Per-file Coverage Changes (8 files)
| File | Lines (Before → After) | Statements (Before → After) |
|---|---|---|
src/commands/validators/infrastructure-validator.ts |
100.0% → 90.3% (-9.73%) | 100.0% → 90.4% (-9.59%) |
src/nvx/runtime-backend.ts |
88.2% → 81.6% (-6.52%) | 87.3% → 81.4% (-5.97%) |
src/commands/build-config.ts |
84.4% → 79.4% (-4.96%) | 85.1% → 80.3% (-4.79%) |
src/nvx/manager.ts |
82.9% → 79.7% (-3.21%) | 82.8% → 79.3% (-3.53%) |
src/nvx/runtime-validation.ts |
100.0% → 98.7% (-1.27%) | 100.0% → 98.8% (-1.22%) |
src/nvx/filesystem-builder.ts |
89.1% → 88.2% (-0.95%) | 88.3% → 87.1% (-1.19%) |
src/nvx/one-shot-adapter.ts |
83.5% → 82.9% (-0.60%) | 80.3% → 79.8% (-0.56%) |
src/log-directory-setup.ts |
96.2% → 100.0% (+3.78%) | 96.3% → 100.0% (+3.71%) |
✨ New Files (5 files)
src/nvx/filesystem-write-policy.ts: 97.7% linessrc/nvx/guest-entrypoint.ts: 100.0% linessrc/nvx/guest-environment-builder.ts: 90.0% linessrc/nvx/workspace-export.ts: 98.5% linessrc/nvx/workspace-layer.ts: 92.5% lines
Coverage comparison generated by scripts/ci/compare-coverage.ts
|
✅ Smoke Gemini completed. All facets verified. 💎 Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"
- "play.googleapis.com"See Network Configuration for more information.
|
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 Warning Firewall blocked 12 domainsThe following domains were blocked by the firewall during workflow execution:
[!TIP] tools:
github:
mode: gh-proxySee GitHub Tools for more information on To allow these domains, add them to the network:
allowed:
- defaults
- "ab.chatgpt.com"
- "accounts.google.com"
- "api.github.com"
- "clients2.google.com"
- "collector.github.com"
- "contentautofill.googleapis.com"
- "github.com"
- "github.githubassets.com"
- "msfeed25.pkgs.visualstudio.com"
- "update.googleapis.com"
- "www.google.com"
- "www.gstatic.com"See Network Configuration for more information.
|
|
🔌 Smoke Services — All services reachable! ✅
|
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤
|
|
✅ Smoke Claude passed Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
|
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓
|
|
🚀 Security Guard has started processing this pull request |
Smoke Test: Copilot Engine
Overall: PASS cc
|
Smoke Test: Cloud Hypervisor + Copilot
Result: PASS (4/4) Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"
- "github.com"See Network Configuration for more information.
|
Smoke Test: Claude Engine Validation
Overall result: ✅ PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
|
github-actions
Bot
added
smoke-claude
smoke-copilot-network-isolation
|
EGRESS_RESULT allow=pass deny=pass ✅ Allowed domain (github.com) reachable: Overall: PASS — network isolation egress enforcement working as expected. cc Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
[!TIP] tools:
github:
mode: gh-proxySee GitHub Tools for more information on To allow these domains, add them to the network:
allowed:
- defaults
- "api.github.com"
- "example.com"See Network Configuration for more information.
|
|
Smoke Test: Services Connectivity
Overall: PASS
|
|
Smoke Test: Copilot BYOK (Direct) Mode — PASS ✅
Status: Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com /cc
|
Gemini Smoke Test Results
Overall Status: PASS Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"
- "play.googleapis.com"See Network Configuration for more information.
|
|
Extract dynamic enclave launch test fixture Warning Firewall blocked 12 domainsThe following domains were blocked by the firewall during workflow execution:
[!TIP] tools:
github:
mode: gh-proxySee GitHub Tools for more information on To allow these domains, add them to the network:
allowed:
- defaults
- "ab.chatgpt.com"
- "accounts.google.com"
- "api.github.com"
- "clients2.google.com"
- "collector.github.com"
- "contentautofill.googleapis.com"
- "github.com"
- "github.githubassets.com"
- "msfeed25.pkgs.visualstudio.com"
- "update.googleapis.com"
- "www.google.com"
- "www.gstatic.com"See Network Configuration for more information.
|
Chroot Version Comparison
Overall result: ❌ FAILED — Node.js version mismatch between host and chroot environment. The
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — PASS Note: Java Maven builds initially failed with All 18 projects across all 8 language ecosystems built/installed and passed their tests successfully through the AWF firewall proxy. Warning Firewall blocked 8 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.nuget.org"
- "bun.sh"
- "dc.services.visualstudio.com"
- "deno.land"
- "dl.deno.land"
- "github.com"
- "releaseassets.githubusercontent.com"
- "repo.maven.apache.org"See Network Configuration for more information.
|
OTEL Tracing Smoke Test Results
Overall: Success. All implemented scenarios pass; span export (Scenario 5) is not applicable without a configured OTLP endpoint in this environment, consistent with expected graceful degradation behavior. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "o205451.ingest.us.sentry.io"See Network Configuration for more information.
|
NVX landed as an opt-in preview backend with several deliberate boundaries relative to Cloud Hypervisor: no live host workspace export (files had to be pre-baked into an
--nvx-layer), no per-run env passthrough, and blanket rejection of--container-workdir/--network-subnet. This closes those gaps and resolves the open design question about fallback behaviour.Everything here works within the existing attested NVX guest contract — the initramfs and OpenVMM fork are signed upstream artifacts pinned by
src/nvx/preflight.ts, and the guest interface is kernel-cmdline-only. No new guest token, block device, or virtio-fs share was needed.How the workspace export works
The guest assembles its rootfs as
overlayfs(lowerdir=custom:runtime:distro, upperdir=<scratch>/upper). AWF owns thecustomlayer, so staging the host workspace there makes it read-write in-guest via overlay copy-up, with all writes landing in the scratch ext4 image. After the microVM exits, those writes are pulled back out withdebugfs— no guest cooperation required.Changes
src/nvx/workspace-layer.ts(staging, ownership policy, copy-back, host-conflict detection) andsrc/nvx/workspace-export.ts(guest layout constants, export resolution). Mount policiesworkspace-only(default) andworkspace-and-tool-cachevia--nvx-mount-policy/nvx.mountPolicy.src/nvx/filesystem-builder.tsgainedpreserveOwnershipsomkfs.erofscan omit--all-root— with it, every entry is root-owned and the non-root workload cannot copy-up.src/nvx/filesystem-write-policy.tsplansfilesystem.allowWrite. Read-only subtrees are staged uid/gid 0 with write bits cleared; the workload runs with an empty capability set (noCAP_FOWNER/CAP_DAC_OVERRIDE), so this is enforced by the kernel rather than by convention. Copy-back additionally filters non-writable guest paths.guest-environment-builder.ts(mirrors the Cloud Hypervisor builder) plusguest-entrypoint.ts, which generates a root-owned0555script in the custom layer that exports the environment,cds to the working directory, and execs the command. This also fixes multi-word agent commands, which the whitespace-freenvx_arg=cmdline token could never carry — the previousentrypoint: /bin/sh, args: ['-lc', agentCommand]was already broken for anything beyond a single token.--container-workdir— accepted when it resolves inside the guest workspace export (assertNvxContainerWorkDir); the blanket rejection is gone.--network-subnet— still rejected, now with a recorded rationale:src/microvm/infrastructure.tsasserts the Docker network matches the compile-timeNETWORK_SUBNETand is shared with Cloud Hypervisor, so this is not an NVX-local change.docs/nvx-security-design.md. Rationale: NVX is explicitly opt-in preview, and silently downgrading to Docker would misrepresent the isolation boundary and let preview validation runs pass without ever booting a microVM..github/workflows/smoke-nvx-copilot.mdruns the pinned Copilot CLI inside an NVX microVM and records four checks: microVM run, in-guest assertions (workspace readable, workdir honoured, env present, no credential variables leaked), workspace copy-back, and inference through the API proxy. Because gh-aw'ssandbox.agent.runtimeenum has nonvxentry, the microVM runs from pre-agentsteps:and the gh-aw agent analyses the recorded evidence.preflight.tsnow requiresdebugfsande2fsck.Review notes
debugfs rdumpfaithfully reproduces overlay whiteout char devices, and thatmkfs.erofspreserves uid/gid without--all-root.rdump; documented as a known limitation.