Skip to content

agentHost: stabilize command auto-approver initialization - #330548

Merged
Paul (pwang347) merged 5 commits into
mainfrom
pwang347/fix-command-auto-approver-init-flake
Aug 13, 2026
Merged

agentHost: stabilize command auto-approver initialization#330548
Paul (pwang347) merged 5 commits into
mainfrom
pwang347/fix-command-auto-approver-init-flake

Conversation

@pwang347

@pwang347 Paul (pwang347) commented Aug 12, 2026

Copy link
Copy Markdown
Member

Fixes the flaky Windows node.js unit-test failures reported for main (build 464074, build 464010), where 41 tests across agentSideEffects.test.js, commandAutoApprover.test.js and sessionPermissions.test.js fail together — every assertion returning 'noMatch'/undefined where 'approved' or 'denied' was expected.

Root cause

@vscode/tree-sitter-wasm's initializeBinding has no single-flight guard — it awaits before assigning the cached module:

var Module3 = null;
async function initializeBinding(moduleOptions) {
  if (!Module3) {
    Module3 = await tree_sitter_default(moduleOptions);  // await precedes assignment
  }
  return Module3;
}

Any caller arriving before the first one resolves also passes the !Module3 check and instantiates a competing WASM module. This reproduces against the library alone, with no VS Code code involved — two overlapping Parser.init() calls fail outright:

TypeError: _ts_init is not a function
    at Parser.init (tree-sitter.js:3871:25)

The loser of the race observes a half-constructed module whose wasmExports are not yet bound.

Neither degenerate case fails, which is what makes this intermittent rather than deterministic:

Pattern Result
50 simultaneous inits passes — all await before touching the module handle
Sequential re-init passes — Module3 is already cached
Genuinely overlapping inits fails — competing modules, half-initialized loser

CommandAutoApprover fires _initTreeSitter() un-awaited from its constructor, and one approver is constructed per AgentSideEffectsSessionPermissionManager. In agentSideEffects.test.ts that is one per test via setup() (164 tests), plus ~30 more inside individual tests, plus the commandAutoApprover and sessionPermissions suites — hundreds of overlapping initializations in a single Node process.

When an approver loses the race, the catch in _initTreeSitter swallows the error and leaves _parser / _bashLanguage / _powershellLanguage undefined. _extractSubCommands then returns undefined and evaluate fails closed to noMatch. Because the degradation is per-approver and permanent, every test using that approver fails — hence the all-or-nothing block of 41 failures.

Why it surfaced now

There is no single regression commit. The race has been latent since fbabc5c (tree-sitter adoption, Mar 31). Two gradual pressures pushed it past the threshold:

  • cb0cb56 (Jul 29) added the PowerShell grammar, doubling per-approver WASM work from one Language.load() to two.
  • agentSideEffects.test.ts grew from 148 to 164 tests since late July, each adding another overlapping initialization.

On contended Windows CI agents this widened the overlap window enough to lose the race in roughly 30% of runs (6 of 20 iterations in build 464074).

Note

The automated investigation for build 464010 attributed this to c4d713e (#330424) and filed microsoft/vscode-engineering#3584 against that author. That attribution is incorrect — build 463994, at the earlier commit b741771, already showed the identical 41 failures, and later builds containing c4d713e passed. #3584 should be retargeted or closed against this PR.

Credit to Benjamin Christopher Simmonds (@benibenj), whose independent investigation earlier the same day reached the same conclusion and the same fix shape.

Change

  • Hoist tree-sitter module init and grammar loading into a single process-wide promise, so Parser.init() and each Language.load() happen exactly once and every approver observes a fully initialized parser.
  • Keep the Parser instance per CommandAutoApprover so it remains individually disposable; the shared Language/Query objects are not deleted by instances.
  • Fail-closed behavior is unchanged: a grammar that fails to load leaves its language undefined, so those commands still require confirmation rather than being auto-approved.

Second fix: a racy sibling test exposed by the timing change

CI surfaced a second, independent failure on the Electron jobs (macOS and Linux):

1) AgentService (node dispatcher) disposeSession
     reports failed unregistration and allows deletion to retry durably:
   AssertionError [ERR_ASSERTION]: Missing expected rejection.

That test budgets exactly two registry write failures via db.failRegistryWrites(2) and relies on both being consumed by the unregistration, because _retryRegistryMutation makes exactly two attempts before giving up. The provider backfill's markProviderBackfilled write is fire-and-forget, so when it lands after the injection it steals part of that budget — the retry then succeeds and disposeSession resolves instead of rejecting.

Instrumenting the fake database shows the intended ordering, with the backfill settling before the injection:

write#1 by registerSessionIfNotTombstoned
write#2 by markProviderBackfilled      <- fire-and-forget; must land before the injection
write#3 by registerSession
write#4 by clearSessionTombstone
--- failRegistryWrites(2) ---
write#1..#3 by tombstoneAndUnregisterSession

This is latent rather than newly introduced, but it is coupled to this PR: removing the per-approver WASM work also removes roughly two file reads and two WASM compiles from every AgentService construction, which is enough to reorder when that fire-and-forget write lands on slower CI agents.

The fix mirrors the pattern already used by the sibling test retries a transient registry registration failure before reporting creation success, which awaits listSessions() before injecting failures. listSessions()_ensureRegistryBackfilled() awaits the in-flight provider backfill, so after it no background registry write can be pending and the budget is deterministically consumed by disposeSession alone.

Testing

  • Added a regression test that initializes 20 approvers concurrently and asserts both the bash and PowerShell grammars resolve for every one of them.
  • agentSideEffects.test.ts + commandAutoApprover.test.ts: 207 passing.
  • Stress-ran both suites for 20 fresh-process iterations (4,140 tests) with no failures.
  • Full unit suite locally: 30553 passing, 0 failing.
  • Flaky pipeline validation (definition 700, Windows, 20 iterations): build 464161.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dcd23755-c332-4ebe-aa93-75679170b5be
Copilot AI balanced review requested due to automatic review settings August 12, 2026 21:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Stabilizes tree-sitter initialization for command auto-approval by sharing process-wide WASM resources.

Changes:

  • Loads tree-sitter and shell grammars once through a shared promise.
  • Keeps disposable parser instances scoped per approver.
  • Adds concurrent-initialization regression coverage.
Show a summary per file
File Description
commandAutoApprover.ts Shares tree-sitter initialization and grammar resources.
commandAutoApprover.test.ts Tests concurrent approver initialization.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Balanced

roblourens
roblourens previously approved these changes Aug 12, 2026
@pwang347
Paul (pwang347) enabled auto-merge (squash) August 12, 2026 22:56
…ilures

The disposeSession retry test budgets exactly two registry write failures and relies on both being consumed by the unregistration. The provider backfill's markProviderBackfilled write is fire-and-forget, so when it lands after failRegistryWrites it steals part of that budget, letting _retryRegistryMutation succeed on its retry and producing 'Missing expected rejection'.

Await listSessions() first, matching the sibling registration-retry test, so no background registry write can be pending.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dcd23755-c332-4ebe-aa93-75679170b5be
@pwang347
Paul (pwang347) merged commit 8121fc5 into main Aug 13, 2026
27 checks passed
@pwang347
Paul (pwang347) deleted the pwang347/fix-command-auto-approver-init-flake branch August 13, 2026 00:26
@vs-code-engineering vs-code-engineering Bot added this to the 1.134.0 milestone Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants