[https://nvbugs/6435097][fix] Remove the single stale nvbugs/6427411 waiver line for… - #16266
[https://nvbugs/6435097][fix] Remove the single stale nvbugs/6427411 waiver line for…#16266trtllm-agent wants to merge 1 commit into
nvbugs/6427411 waiver line for…#16266Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
💤 Files with no reviewable changes (1)
WalkthroughThe integration waiver list removes the Changes
Estimated code review effort: 1 (Trivial) | ~2 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
03137a5 to
261fd97
Compare
261fd97 to
384414e
Compare
mzweilz
left a comment
There was a problem hiding this comment.
LGTM if the unwaived test passes.
fredricz-20070104
left a comment
There was a problem hiding this comment.
Review summary - Approve
Reviewed the full diff; no blocking or major issues found.
Minor, non-blocking notes:
tests/integration/test_lists/waives.txt: Sibling waiver for the same bug id 6427411 left in place
Automated review by NVCortex Lite, run by @fredricz-20070104.
384414e to
5878223
Compare
The TorchSampler IndexError described in nvbug 6435097 was introduced by PR NVIDIA#15920 ("Move greedy stop checks to host") and cleanly reverted by PR NVIDIA#16163 (commit 9a8ec05). PR NVIDIA#16163 is already present at HEAD, so the underlying code bug is fixed. The waiver at nvbugs/6427411 for test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=1-ctx_pp=2] was added by PRs NVIDIA#16103 / NVIDIA#16105 / NVIDIA#16127 before the revert landed and was not cleaned up afterward. Verified locally on this HEAD (e523b43): pytest tests/integration/defs/accuracy/test_disaggregated_serving.py::\ TestLlama3_1_8BInstruct::test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=1-ctx_pp=2] -> PASSED, GSM8K accuracy = 73.465 (threshold 70.997). Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
5878223 to
f0380c6
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
brnguyen2
left a comment
There was a problem hiding this comment.
Approving — the comments below are optional touch-ups, not blockers.
The waiver being removed is one of 29 lines in waives.txt citing nvbugs/6427411, including sibling parametrizations of the same test (gen_tp=2-ctx_pp=2, MMLU-gen_tp=1-ctx_pp=2, test_tp_pp_symmetric[GSM8K-tp1pp2]). If the revert in #16163 fixed the root cause, those are stale too, and the bug can't be closed while they remain; if only this one parametrization was verified, say so in the description so the rest aren't assumed handled.
Since this un-skips a post-merge test, trigger the owning stage explicitly (/bot run --extra-stage "...") rather than relying on the default pipeline.
crazydemo
left a comment
There was a problem hiding this comment.
Review summary - CONCERNS
Verdict: The change itself is correct and low-risk (removes one stale SKIP for a test whose root cause was reverted in #16163), but it cannot merge as-is: mergeable_state is dirty, so the branch needs a rebase, and the un-waived test is not shown to pass.
Issues
- [MINOR]
tests/integration/test_lists/waives.txt:3- sibling waiver for same bug id 6427411 left in place
QA view
- Test coverage: partial - only a SKIP entry is removed; nothing in the diff demonstrates the un-waived
test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=1-ctx_pp=2]now passes against the #16163 revert. - SM coverage: architecture-independent - test-list waiver edit only, no arch-guarded code.
- Test code: waiver hygiene is inconsistent - other 6427411 waivers remain in the same file.
- Test time: small - re-enables one disaggregated-serving accuracy parametrization in the post-merge suite.
- Needs
/qa-verify: yes - unwaives a post-merge test with no in-diff proof it passes; trigger the owning stage explicitly and confirm green.
Does this actually fix nvbugs/6435097 / 6427411?
Partial. The real fix was the revert in #16163, present at HEAD; this PR only drops the SKIP so the test runs again, which is correct for this one parametrization. But bug 6427411 also guards sibling parametrizations still waived here (e.g. line 3 gen_tp=2-ctx_pp=2). If the revert fixed the shared root cause, those are stale too and 6427411 can't be closed while they remain; if only this parametrization was verified, the rest are uncovered. Please state the intended scope in the description.
Possible new issues
- Branch is in a merge-conflict (
dirty) state and must be rebased before it can land. - Re-enabling the test on main will fail CI if the revert did not fully cover the
gen_tp=1-ctx_pp=2path.
What I could not verify
Whether the un-waived test actually passes now, and whether the sibling 6427411 waivers share the same (now-fixed) root cause - neither is determinable from the one-line diff.
Automated review by NVCortex Lite, run by @crazydemo.
| accuracy/test_disaggregated_serving.py::TestDeepSeekV3Lite::test_auto_dtype_with_helix[fifo_v2-cudagraph:with_padding-pp1dp2cp2] SKIP (https://nvbugs/6567057) | ||
| accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=1-ctx_pp=2] SKIP (https://nvbugs/6427411) | ||
| accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=1-ctx_pp=4] SKIP (https://nvbugs/6428069) | ||
| accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=2-ctx_pp=2] SKIP (https://nvbugs/6427411) |
There was a problem hiding this comment.
[MINOR] Sibling waiver for same bug id 6427411 left in place
This line still waives test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=2-ctx_pp=2] under the same nvbugs/6427411 that the removed line cited. If the revert in #16163 fixed the shared root cause (under-populated token buffers on non-last PP ranks), this and the other 6427411 waivers are equally stale and 6427411 cannot be closed while they remain. If instead only the gen_tp=1-ctx_pp=2 parametrization was actually verified, state that in the PR description so the remaining ones are not assumed handled. No change required to merge, but the scope should be made explicit.
crazydemo
left a comment
There was a problem hiding this comment.
Review summary - Approve (non-blocking)
Approving so this is not blocked on me. The points raised in my review comment above are non-blocking — please read them and address what you agree with before merging.
Worth doing before this is relied on: This unwaives a post-merge test without any in-diff proof it passes; QA should trigger the owning stage explicitly and confirm green before this is trusted, and confirm whether the sibling 6427411 waivers can also be removed.
Automated review by NVCortex Lite, run by @crazydemo.
Summary
use_host_stop_criteriafast path that under-populated per-step/seq_slot/beam token buffers on non-last PP ranks, causing IndexError; PR [TRTLLM-14155][fix] Revert host-side greedy stop checks from #15920 #16163 (commit 9a8ec05) reverted PR [None][perf] Move greedy stop checks to host #15920 and is present at HEAD, so the code is already fixed. The waiver line for this test was left behind by cleanup PRs [https://nvbugs/6427411][chore] Waive failed tests in Pre-merge #16103/[https://nvbugs/6427411][chore] Waive a failed test in Pre-merge #16105/[https://nvbugs/6427411][chore] Waive remaining PP sampler regressions #16127.nvbugs/6427411waiver line fortest_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=1-ctx_pp=2]inwaives.txt; do not touch other test entries or other bug IDs.Test plan
Links
Dev Engineer Review
nvbugs/6427411waiver andSKIPmarker forTestLlama3_1_8BInstruct::test_ctx_pp_gen_tp_asymmetric[GSM8K-gen_tp=1-ctx_pp=2].waives.txtentry.QA Engineer Review
test-db/orqa/files were modified.tests/integration/test_lists/waives.txt.