Refresh DeepSeek-V4-Pro GB200 Dynamo vLLM stack / 更新 DeepSeek-V4-Pro GB200 Dynamo vLLM 推理栈#2139
Refresh DeepSeek-V4-Pro GB200 Dynamo vLLM stack / 更新 DeepSeek-V4-Pro GB200 Dynamo vLLM 推理栈#2139Ankur-singh wants to merge 3 commits into
Conversation
Update the vLLM image and Dynamo wheel, and enable multiple frontends across all five GB200 DeepSeek-V4-Pro topologies. 中文：更新 vLLM 镜像和 Dynamo wheel，并为 DeepSeek-V4-Pro 的全部五种 GB200 拓扑启用多个前端。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Backfill the canonical pull request URL after creating the draft PR. 中文：创建草稿 PR 后，将正式 PR 链接回填至性能变更日志。
|
Claude finished @Ankur-singh's task in 4m 57s —— View job Review
One 🟡 WARNING (posted inline): all five modified recipes still carry the old image and Dynamo wheel in the Checks that passed:
One thing I could not verify (sandbox blocks network calls): that the tag I also could not run |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| dynamo: | ||
| install: true | ||
| wheel: "1.2.0.dev20260426" | ||
| wheel: "1.3.0.dev1" |
There was a problem hiding this comment.
🟡 WARNING: Incomplete stack refresh — the identity: block at the bottom of this file (lines 147–155) still pins the old stack, while model.container and dynamo.wheel here were updated.
Why it matters: identity.container.image still says vllm/vllm-openai:v0.20.0-ubuntu2404 and identity.frameworks.dynamo still says 1.2.0.dev20260426, contradicting the stack that actually runs (dsv4-megamoe-mxfp4-arm64-cu130-4ba0a72 / 1.3.0.dev1). Every other dsv4 recipe keeps these fields in sync with the top-level ones (e.g. disagg-gb200-low-latency-mtp2.yaml lines 10/131), so anything consuming this block for provenance will attribute the new benchmark numbers to the old image and wheel.
Fix: Update the identity block to match:
identity:
model:
repo: "deepseek-ai/DeepSeek-V4-Pro"
revision: "0366e4e064385807ea86b088a5c6c878ff23343b"
container:
image: "vllm/vllm-openai:dsv4-megamoe-mxfp4-arm64-cu130-4ba0a72"
frameworks:
dynamo: "1.3.0.dev1"
vllm: "0.20.0" # confirm the vLLM version shipped in the 4ba0a72 imageidentity.frameworks.vllm: "0.20.0" is also likely stale for the new dev image — please confirm what vLLM version 4ba0a72 ships and update accordingly.
This same stale identity: block exists in all five modified recipes (same lines): disagg-gb200-high-tpt-megamoe.yaml, disagg-gb200-low-latency.yaml, disagg-gb200-low-middle-curve.yaml, disagg-gb200-max-tpt-megamoe.yaml, disagg-gb200-mid-curve-megamoe.yaml. The header comment at line 13 of each file ("model.container set to vllm/vllm-openai:v0.20.0-ubuntu2404 to match nvidia-master.yaml") is also stale — worth fixing in the same pass since it documents the master-config-match invariant this PR relies on.
There was a problem hiding this comment.
LGTM — mechanical container/wheel bump plus multi-frontend flip; the flagged inline is a doc-drift nit with no runtime impact, and full-sweep-fail-fast will validate the new stack.
Extended reasoning...
Overview
This PR refreshes the DeepSeek-V4-Pro GB200 Dynamo vLLM stack across 5 sibling recipes:
model.container:v0.20.0-ubuntu2404→dsv4-megamoe-mxfp4-arm64-cu130-4ba0a72dynamo.wheel:1.2.0.dev20260426→1.3.0.dev1frontend.enable_multiple_frontends:false→true- Matching
configs/nvidia-master.yamlimage bump for thedsv4-fp4-gb200-dynamo-vllmkey - Append-only
perf-changelog.yamlentry linking to this PR
All 7 files are YAML/config with no code logic changes.
Security risks
None. No auth, crypto, permissions, secrets, or shell-invoked substitution paths are touched. Container image tags are pulled from the same vllm/vllm-openai namespace as before.
Level of scrutiny
Low. This is a config-only bump following the same pattern as prior GB200 refreshes (e.g. sibling disagg-gb200-high-tpt-megamoe-mtp2.yaml). The nvidia-master.yaml image and model.container are kept in sync — preserving the load-bearing invariant documented in the per-file header comment. No topology, concurrency, parallelism, or vLLM engine flags changed.
Other factors
- The
full-sweep-fail-fastlabel is set, so the full 8K/1K benchmark curve will validate that the new image + wheel + multi-frontend flag actually runs end-to-end. That's much stronger validation than static review can provide. - Only one finding surfaced (an inline nit about stale
identity:blocks andLocal deltas vs upstreamheader comments referencing the old tag). Grep confirms nothing consumes theidentityblock orframeworks.dynamo/vllmfields at runtime — it is provenance metadata only. Worth fixing but not blocking. - PR author's stated validation (
pytest utils/matrix_logic/ -q= 188 passed; target config generation for 5 rows; YAML/staged-diff/append-only checks) matches what this diff would exercise.
| path: "deepseek-v4-pro" | ||
| container: "vllm/vllm-openai:v0.20.0-ubuntu2404" | ||
| container: "vllm/vllm-openai:dsv4-megamoe-mxfp4-arm64-cu130-4ba0a72" | ||
| precision: "fp4" | ||
|
|
||
| dynamo: | ||
| install: true | ||
| wheel: "1.2.0.dev20260426" | ||
| wheel: "1.3.0.dev1" |
There was a problem hiding this comment.
🟡 Documentation drift across all 5 modified GB200 recipes: both the 'Local deltas vs upstream' header comment (lines ~13-16, referencing v0.20.0-ubuntu2404) and the identity block at the bottom (container.image: v0.20.0-ubuntu2404, frameworks.dynamo: 1.2.0.dev20260426, frameworks.vllm: 0.20.0) were not updated to reflect the new model.container (dsv4-megamoe-mxfp4-arm64-cu130-4ba0a72) and dynamo.wheel (1.3.0.dev1) values. No runtime impact — nothing consumes these fields today — but each recipe is now internally inconsistent (compare with sibling disagg-gb200-high-tpt-megamoe-mtp2.yaml where identity mirrors model.container). Please refresh the header comment and identity block in all 5 files (disagg-gb200-{high-tpt-megamoe,low-latency,low-middle-curve,max-tpt-megamoe,mid-curve-megamoe}.yaml).
Extended reasoning...
What is stale
Each of the 5 modified recipes has two documentation surfaces that the PR should have kept in sync with the model.container and dynamo.wheel bumps but didn't:
1. Header comment (Local deltas vs upstream, lines ~13-16)
* model.container set to vllm/vllm-openai:v0.20.0-ubuntu2404 to
match nvidia-master.yaml image (which the launch script registers as
the alias key in srtslurm.yaml). Upstream variants ship either the
non-dynamo floating tag or a sha256 pin.
This comment is the load-bearing documentation of a local-delta invariant (AGENTS.md: model.container must equal the image: entry in nvidia-master.yaml). configs/nvidia-master.yaml line 8590 was bumped in this same PR to dsv4-megamoe-mxfp4-arm64-cu130-4ba0a72, so the invariant still holds — but the tag literal in the comment now contradicts the value on the very next config line.
2. identity block at the bottom of each file
identity:
container:
image: "vllm/vllm-openai:v0.20.0-ubuntu2404"
frameworks:
dynamo: "1.2.0.dev20260426"
vllm: "0.20.0"Sibling recipe disagg-gb200-high-tpt-megamoe-mtp2.yaml demonstrates the intended convention: its identity.container.image mirrors its model.container. This PR breaks that pattern in all 5 modified files.
Step-by-step proof (using disagg-gb200-high-tpt-megamoe.yaml)
- Line 15 of the header comment says
model.container set to vllm/vllm-openai:v0.20.0-ubuntu2404. - Line 21 (
model.container) now readsvllm/vllm-openai:dsv4-megamoe-mxfp4-arm64-cu130-4ba0a72. The comment and the value it describes disagree. - Line ~152 (
identity.container.image) still saysvllm/vllm-openai:v0.20.0-ubuntu2404— same old value the PR removed above. - Line ~154 (
identity.frameworks.dynamo) still says1.2.0.dev20260426, while line 26 (dynamo.wheel) is now1.3.0.dev1. - The same three-way mismatch reproduces in each of the other 4 files (
low-latency,low-middle-curve,max-tpt-megamoe,mid-curve-megamoe).
Impact
No runtime effect — the benchmark reads from model.container and dynamo.wheel, both of which are correct. A grep across the repo finds no Python consumer of the identity: block or of frameworks.dynamo/frameworks.vllm; identity is provenance metadata for humans and downstream tooling. So this is documentation drift only.
The concern is that any reviewer or downstream tool that trusts identity to describe "what actually ran" will now see the pre-bump image and wheel, not the new ones. And the header comment — which explicitly documents why the local pin exists — reads as if the pin is still the old tag.
How to fix
Mechanical text replacement in the 5 affected files:
- Header comment: change
v0.20.0-ubuntu2404->dsv4-megamoe-mxfp4-arm64-cu130-4ba0a72in theLocal deltas vs upstreamblock. identity.container.image: change tovllm/vllm-openai:dsv4-megamoe-mxfp4-arm64-cu130-4ba0a72.identity.frameworks.dynamo: change to1.3.0.dev1.identity.frameworks.vllm: the new image tag doesn't embed a clear upstream vLLM version, so pick whatever the vendored wheel inside the new image reports (or drop the field if unknown — matching sibling conventions).
Severity is nit because nothing breaks at runtime, but the fix is cheap and worth doing before merge since 5 files now carry contradictory self-description.
Set DYN_REQUEST_PLANE_CODEC=msgpack on every GB200 Dynamo frontend, prefill worker, and decode worker. 中文：为所有 GB200 Dynamo 前端、预填充 worker 和解码 worker 设置 DYN_REQUEST_PLANE_CODEC=msgpack。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29049519395 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29050099766 |
1 participant
Summary
vllm/vllm-openai:v0.20.0-ubuntu2404tovllm/vllm-openai:dsv4-megamoe-mxfp4-arm64-cu130-4ba0a72.1.2.0.dev20260426to1.3.0.dev1.frontend.enable_multiple_frontendsacross all five GB200 serving topologies.DYN_REQUEST_PLANE_CODEC=msgpackon every frontend, prefill worker, and decode worker.Validation
python -m pytest utils/matrix_logic/ -q— 188 passed.Benchmark plan
full-sweep-fail-fastlabel.中文说明
vllm/vllm-openai:v0.20.0-ubuntu2404更新为vllm/vllm-openai:dsv4-megamoe-mxfp4-arm64-cu130-4ba0a72。1.2.0.dev20260426升级至1.3.0.dev1。frontend.enable_multiple_frontends。DYN_REQUEST_PLANE_CODEC=msgpack。验证
python -m pytest utils/matrix_logic/ -q— 188 项测试通过。基准测试计划
full-sweep-fail-fast标签，运行完整的 8K/1K 基准测试曲线。