fix: tolerate benign VMM thread churn in confinement re-verification

[lpcox]

Fixes #9012

Problem

verifyNvxConfinement, and the identical pattern in verifyCloudHypervisorConfinement, snapshotted /proc/<pid>/task and then required it to be byte-identical after the async cgroup and namespace checks. OpenVMM creates and retires worker threads while it initialises, so launches failed nondeterministically with process identity or thread-set race: one pass and two failures on identical config (see #9012). This also blocks hardware validation of #9014.

Fix

The security property is unchanged: the process that was verified is the process being confined.

• Still fails closed: a changed process start time, a changed executable (for NVX, both path and dev/inode), a main thread that vanishes, a new thread that violates policy, or a final thread set over the 256-thread bound.
• Now accepted safely:
  • Worker threads that exit between samples, or between readdir and the read (ENOENT/ESRCH), are dropped.
  • New threads and recycled TIDs (start time changed) are fully re-verified before being accepted: Tgid, uid/gid, groups, capabilities, no_new_privs, and seccomp. /proc/<pid>/task only lists members of this thread group, and the Tgid check confirms that.
• The identity re-check now runs after thread re-verification, so it brackets all the evidence.
• Errors now name the broken invariant and show the observed and expected values.
• threadCount / observedThreadCount in the evidence now report the final verified live-thread count.

Validation

• Added churn tests for both verifiers covering new, exited, vanished-mid-read, and recycled threads, a policy-violating new thread, executable replacement, main-thread loss, and the final-sample bound.
• Mutation check: running the new tests against the old verifiers gives 14 failures; with the fix, 31/31 pass.
• jest src/nvx src/cloud-hypervisor: 669 passed. tsc is clean, with no new eslint errors.
• Hardware validation via smoke-nvx-copilot follows in PR comments.

[Copilot]

Copilot review overview

🟡 Changes recommended

Both verifiers can accept a final task listing that omits the main thread while retaining workers.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2

Open (2)

• Reject churn results that omit the process leader · New
• Require the final task listing to contain the main TID · New

What changed in this PR

Updates NVX and Cloud Hypervisor confinement verification to tolerate benign thread churn while preserving identity and policy checks.

Changes:

• Re-verifies new or recycled threads and tolerates vanished workers.
• Improves invariant-specific errors and final thread-count evidence.
• Adds thread-churn and fail-closed tests.

File	Description
src/​nvx/​confinement.ts	Adds churn-aware NVX verification.
src/​nvx/​confinement.test.ts	Tests NVX churn scenarios.
src/​cloud-hypervisor/​confinement-verifier.ts	Adds churn-aware Cloud Hypervisor verification.
src/​cloud-hypervisor/​confinement-verifier.test.ts	Tests Cloud Hypervisor churn scenarios.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric	Base	PR	Delta
Lines	92.65%	92.66%	➡️ +0.01%
Statements	91.13%	91.13%	➡️ +0.00%
Functions	89.11%	89.17%	📈 +0.06%
Branches	84.26%	84.23%	📉 -0.03%

📁 Per-file Coverage Changes (4 files)

File	Lines (Before → After)	Statements (Before → After)
src/nvx/one-shot-adapter.ts	83.5% → 82.9% (-0.60%)	80.3% → 79.8% (-0.56%)
src/nvx/confinement.ts	88.2% → 88.9% (+0.73%)	83.3% → 84.6% (+1.24%)
src/cloud-hypervisor/confinement-verifier.ts	82.9% → 85.1% (+2.24%)	81.6% → 83.3% (+1.76%)
src/log-directory-setup.ts	96.2% → 100.0% (+3.78%)	96.3% → 100.0% (+3.71%)

---

Coverage comparison generated by scripts/ci/compare-coverage.ts

[github-actions]

✅ Copilot review passed with no inline comments.

@lpcox Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

[lpcox]

Hardware validation on real KVM (smoke-nvx-copilot on this branch): 3/3 runs passed all 4 checks (microvm-run, guest-assertions, workspace-copy-back, copilot-inference), with no identity/thread-set race errors. Before this fix, 2 of 3 runs on identical config failed at this verifier.

• https://github.com/github/gh-aw-firewall/actions/runs/36192044758
• https://github.com/github/gh-aw-firewall/actions/runs/36192055071
• https://github.com/github/gh-aw-firewall/actions/runs/36192066014

[github-actions]

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric	Base	PR	Delta
Lines	92.64%	92.65%	📈 +0.01%
Statements	91.12%	91.13%	➡️ +0.01%
Functions	89.11%	89.17%	📈 +0.06%
Branches	84.24%	84.22%	📉 -0.02%

📁 Per-file Coverage Changes (3 files)

File	Lines (Before → After)	Statements (Before → After)
src/nvx/confinement.ts	88.2% → 88.9% (+0.73%)	83.3% → 84.6% (+1.24%)
src/cloud-hypervisor/confinement-verifier.ts	82.9% → 84.7% (+1.83%)	81.6% → 83.0% (+1.39%)
src/log-directory-setup.ts	96.2% → 100.0% (+3.78%)	96.3% → 100.0% (+3.71%)

---

Coverage comparison generated by scripts/ci/compare-coverage.ts