GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)#2122
GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)#2122cquil11 wants to merge 3 commits into
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
cquil11
force-pushed
the
chore/gb200-agentx
branch
from
be6f299 to
e83df02
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28976572219 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Full agentic-coding sweep of the DSv4 FP4 GB200 dynamo-vllm disagg topologies on the June 21 AgentX corpus" | ||
| - "3p2d TEP8 prefill / TP8 decode (40 inference GPUs): c4-c80" | ||
| - "2p1d DEP8 prefill / DEP8 decode (24 inference GPUs): c32-c256 including exploratory tail past the c160 normalized-throughput peak" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2122 |
There was a problem hiding this comment.
🔴 The new perf-changelog entry has pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER — a forgotten template token. This isn't just a dead link: utils/validate_perf_changelog.py only accepts either the canonical .../pull/2122 URL or the literal placeholders XXX / .../pull/XXX, so the changelog validator will hard-fail CI until fixed. Replace PLACEHOLDER with 2122.
Extended reasoning...
The bug
The new entry at perf-changelog.yaml:4645 ends with:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDERThe literal string PLACEHOLDER is a forgotten template token that no other entry in the file uses. The three immediately-preceding entries all use their real PR numbers (pull/2113, pull/2114, pull/2115), so the convention is clear — and per PR metadata this is PR #2122, so the value should be https://github.com/SemiAnalysisAI/InferenceX/pull/2122.
Why this fails CI (not just a dead link)
utils/validate_perf_changelog.py defines the set of accepted placeholder tokens for PR-authored runs:
# utils/validate_perf_changelog.py:24-27
PR_LINK_PLACEHOLDERS = {
"XXX",
"https://github.com/SemiAnalysisAI/InferenceX/pull/XXX",
}and validate_added_pr_link (lines 144-160) requires the link to match either the canonical .../pull/<pr_number> URL or one of those two placeholder tokens:
if link not in PR_LINK_PLACEHOLDERS and link != expected:
raise ChangelogValidationError(
f"new PR entry must use {expected!r} or an XXX placeholder; "
f"found {link!r}"
)Step-by-step proof
- On this PR, the validator runs with
pr_number=2122. - It reads the new entry's
pr-link="https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER". - It computes
expected = "https://github.com/SemiAnalysisAI/InferenceX/pull/2122"— does not match. - It checks
link in PR_LINK_PLACEHOLDERS— the set only contains"XXX"and".../pull/XXX", so"PLACEHOLDER"/".../pull/PLACEHOLDER"is not a member. - Both branches fail →
ChangelogValidationErroris raised → the changelog-validation CI job fails.
Impact
- CI blocks the merge on the changelog-validation check until fixed.
- Even if bypassed, the URL 404s and permanently breaks the changelog's provenance/traceability convention.
Fix
One-token change on line 4645:
- pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER
+ pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2122| - config-keys: | ||
| - dsv4-fp4-gb200-dynamo-vllm-agentic-3p2d-tep8-tp8 | ||
| - dsv4-fp4-gb200-dynamo-vllm-agentic-2p1d-dep8-dep8 | ||
| description: | ||
| - "Full agentic-coding sweep of the DSv4 FP4 GB200 dynamo-vllm disagg topologies on the June 21 AgentX corpus" | ||
| - "3p2d TEP8 prefill / TP8 decode (40 inference GPUs): c4-c80" | ||
| - "2p1d DEP8 prefill / DEP8 decode (24 inference GPUs): c32-c256 including exploratory tail past the c160 normalized-throughput peak" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2122 |
There was a problem hiding this comment.
🟡 Nit (pre-existing style rule): per AGENTS.md line 7, PR titles and descriptions must be bilingual — the title needs a / <中文标题> half and the body needs a ## 中文说明 section mirroring the English summary. Recent sibling PRs #2114/#2115 follow this format; please update the PR title and description on GitHub. No code changes needed.
Extended reasoning...
What the rule is. AGENTS.md line 7 states: "PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format: <English title> / <中文标题>. In the PR/issue body, follow the English content with its Chinese translation (e.g. a ## 中文说明 section mirroring the summary). ... This applies to every PR and every issue."
What this PR has. The title is "GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)" — no / <中文标题> half. The description ends after the English bullet list about the two topologies and their concurrency chunks — there is no ## 中文说明 (or equivalent) section mirroring that content.
Sibling PRs confirm the convention. Recently-merged neighbors in perf-changelog.yaml follow the bilingual format, e.g. commit d527fc8 — "Add GLM-5.1 NVFP4 GB300 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 NVFP4 GB300 分离式 dynamo-sglang MTP 配置 (#2114)" — and bed23ea (#2115) has the analogous GB200 Chinese half. #1780 (c57f1e6) does too.
Impact. Purely a policy/metadata issue. No code path is affected, the sweep runs regardless, and the changelog YAML itself has no bilingual requirement. But the rule is documented as mandatory-for-every-PR, so it is worth flagging.
Step-by-step proof.
- Open
AGENTS.mdand read line 7 — the bilingual requirement is stated verbatim with "This applies to every PR and every issue." - Look at this PR's title — "GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)". There is no
/separator followed by Chinese characters. - Look at this PR's description above — it contains only English prose and no
## 中文说明heading. - Compare with
git log --oneline -5output, which shows Add GLM-5.1 NVFP4 GB300 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 NVFP4 GB300 分离式 dynamo-sglang MTP 配置 #2114/Add GLM-5.1 NVFP4 GB200 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 NVFP4 GB200 分离式 dynamo-sglang MTP 配置 #2115/[NV] add glm5-fp4-gb200-dynamo-sglang / 新增 GLM-5 NVFP4 GB200 Dynamo SGLang 分离式基准测试 #1780 all include the/ <中文标题>half.
How to fix. In the GitHub PR UI, edit the title to append / <中文标题> (e.g. / GB200 DSv4 vLLM 智能体编码全量 sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)) and append a ## 中文说明 section to the body mirroring the English bullets. No file changes required.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28977119907 |
sweep-multi-node-agentic rendered the conc array directly into string
inputs ('[${{ matrix.config.conc }}]' and conc), which fails template
expansion for multinode agentic entries where conc is a list. Mirror
e2e-tests.yml: toJson() for conc-list and conc[0] for the first point.
Both first-pass sweep attempts of the 3p2d TEP8/TP8 topology died to prefill CUDA OOM on serving-time transients (sparse-indexer logits alloc during c64, DeepGEMM JIT module load during c4) with <1 GiB free per GPU at gpu-memory-utilization 0.9. MLA KV is compact, so 0.85 trades unneeded KV space for allocator headroom. Decode stays at 0.9; all 2p1d DEP8/DEP8 chunks passed untouched.
cquil11
force-pushed
the
chore/gb200-agentx
branch
from
964b9d8 to
5fc7c84
Compare
cquil11
added
the
agentx
ChatGPT Codex Connector
matrix.config.conc for multi-node agentic entries is a JSON array
(chunked concurrencies per allocation), but sweep-multi-node-agentic
passed it directly into benchmark-multinode-tmpl.yml's `conc` input,
which is declared `type: string` ("First concurrency for
agentic-coding scenarios; CONC_LIST carries the full batch"). GitHub
Actions' reusable-workflow input validator rejects a sequence value
for a string-typed input at evaluation time, so the whole job failed
to even load:
evaluate reusable workflow inputs: .github/workflows/run-sweep.yml
(Line: 554, Col: 19): A sequence was not expected
Since the job never materializes when this happens, sweep-multi-node-
agentic silently disappeared from run summaries entirely instead of
showing as failed. Slice to the first element (matching the intended
"first concurrency" semantics and the same fix already applied
elsewhere, e.g. PR #2122) to restore a scalar value.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29028294481 |
1 participant
Perf-changelog entry to run the full agentic-coding sweep of the DSv4 FP4 GB200 dynamo-vllm disagg topologies on the June 21 AgentX corpus:
dsv4-fp4-gb200-dynamo-vllm-agentic-3p2d-tep8-tp8— 3 TEP8 prefill + 2 TP8 decode workers (40 inference GPUs), concurrencies 4-80 in three chunks.dsv4-fp4-gb200-dynamo-vllm-agentic-2p1d-dep8-dep8— 2 DEP8 prefill + 1 DEP8 decode workers (24 inference GPUs), concurrencies 32-256 in three chunks, including the exploratory tail past the measured c160 normalized-throughput peak.No config or recipe changes; changelog-only to trigger the sweep of the existing search space.