Skip to content

Flaky: nested-write.integration.test.ts sandbox hook exceeds the 250ms deadline on CI (Test Core) #3259

Description

@os-zhuang

Summary

packages/runtime/src/sandbox/nested-write.integration.test.ts"#1867 nested cross-object write from a hook (real engine + sandbox)" → "a child afterInsert hook writes the parent rollup without crashing (parent has its own hook → real nested VM)" fails intermittently in the Test Core job with:

SandboxError: hook 'rollup_parent_total' exceeded timeout of 250ms (after N pump iterations)
 ❯ QuickJSScriptRunner.execute src/sandbox/quickjs-runner.ts:260:17
 ❯ boundBodyHandler src/sandbox/body-runner.ts:68:24
 ❯ runWithTimeout ../objectql/src/hook-wrappers.ts:115:7
 ...

Result each time: 564 passed | 1 failed (565) — only this one test.

Evidence (3 occurrences across 2 unrelated PRs)

PR run detail
#3227 29652626394 nested-write timeout after 215 pump iterations; a re-kick then flaked differently (@objectstack/http-conformance vitest worker write EPIPE); a 2nd re-kick went fully green
#3258 29677316696 nested-write timeout after 246 pump iterations

Both PRs are unrelated to the sandbox/hook path (one adds summaryOperations.filter to spec/objectql; the other only edits a fieldForm metadata layout) — and the test is not even in those branches except via the merge-with-main commit. The variable pump count (215 vs 246) before hitting the same wall-clock deadline is the signature of a load/timing flake, not a logic bug.

Likely cause

The QuickJS sandbox enforces a 250ms wall-clock deadline (quickjs-runner.ts:260). This test is the heaviest sandbox scenario — a child afterInsert hook that triggers the parent's hook → a real nested VM. On a loaded CI runner the nested pump loop legitimately needs >250ms, so the deadline trips even though the VM is making progress. It's green on a fast/idle runner, red on a loaded one → flaky.

Suggested directions

  1. Give this nested-VM integration test a larger/looser deadline (it is inherently heavier than a single-hook run), or make the sandbox deadline env-overridable so CI can raise it (OS_SANDBOX_* per the OS_{DOMAIN}_* convention).
  2. Or trim the test's hook bodies so the nested run completes with comfortable headroom under 250ms.
  3. Separately track the @objectstack/http-conformance vitest worker EPIPE (worker-pool crash) seen on feat(objectql): filtered roll-up summary fields (#1868) #3227's first re-kick — likely a distinct runner-resource transient, not the same root cause.

Impact

Non-deterministic red on Test Core for PRs that have nothing to do with the sandbox, requiring manual re-kicks (2 needed on #3227). Wastes CI cycles and erodes trust in the required check.

Filed per AGENTS.md Prime Directive #10 while babysitting #3258; the flake is upstream on main, not introduced by that PR.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions