Skip to content

Parallelize benchmark ingestion#542

Draft
cquil11 wants to merge 1 commit into
feat/agentxfrom
codex/parallel-agentic-ingest
Draft

Parallelize benchmark ingestion#542
cquil11 wants to merge 1 commit into
feat/agentxfrom
codex/parallel-agentic-ingest

Conversation

@cquil11

@cquil11 cquil11 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Speeds up the AgentX ingest path by processing independent benchmark configs concurrently.

  • Uses a bounded worker pool (3 by default, configurable with INGEST_CONFIG_CONCURRENCY, capped at 4).
  • Moves trace compression and derived chart/timeline computation to worker threads.
  • Keeps each config’s DB writes ordered.
  • Adds row locks around server-log and trace linking so concurrent workers remain idempotent.

This is a draft for benchmarking and review before merge.

Validated with the pre-commit lint, formatting, and full typecheck hooks; 33 relevant unit tests; and a multi-worker runtime smoke test.

@vercel

vercel Bot commented Jul 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
inferencemax-app Ready Ready Preview, Comment Jul 9, 2026 6:35pm

Request Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant