[https://nvbugs/6619882][test] Unwaive unittest/_torch/sampler on DGX_B200 - #17946
[https://nvbugs/6619882][test] Unwaive unittest/_torch/sampler on DGX_B200#17946zhaoyangwang-nvidia wants to merge 1 commit into
Conversation
…_B200 The whole `unittest/_torch/sampler` entry was waived on DGX_B200 by NVIDIA#17782 because of an intermittent failure (about 4 occurrences per month). The reported failure is not an assertion failure: the inner pytest process dies without emitting a summary or traceback, and the outer wrapper only surfaces `AssertionError: failure reported in unittests`. Waiving the whole entry removes coverage for the entire sampler directory on Blackwell, which is much broader than the intermittent failure itself. Unwaive it so the case runs in CI again and the intermittent failure can be characterized on current main. Signed-off-by: ZhaoyangWang <zhaoyangw@nvidia.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
💤 Files with no reviewable changes (1)
Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review. WalkthroughThe integration waiver list removes the skip entry for ChangesSampler waiver update
Estimated code review effort: 1 (Trivial) | ~2 minutes Merge Risk: ⚪ Minimal · up to This change removes a test waiver and restores sampler test coverage on DGX_B200 without changing product behavior; no actionable merge-blocking risk remains beyond normal checks and review. Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
/bot run |
|
/bot run --stage-list "DGX_B200-PyTorch-1" |
|
PR_Github #67299 [ run ] triggered by Bot. Commit: |
|
PR_Github #67300 [ run ] triggered by Bot. Commit: |
|
PR_Github #67299 [ run ] completed with state |
|
PR_Github #67300 [ run ] completed with state |
|
/bot run --stage-list "DGX_B200-PyTorch-1" |
|
PR_Github #67374 [ run ] triggered by Bot. Commit: |
Dev Engineer Review
_torch/sampler -k "not test_speculative_d2h_parity_real_predictor"waiver fromtests/integration/test_lists/waives.txt.QA Engineer Review
tests/integration/test_lists/waives.txt._torch/samplerwaiver entry.Verdict: needs follow-up
Description
unittest/_torch/sampler -k "not test_speculative_d2h_parity_real_predictor"was waived on DGX_B200 by #17782 (a098858cef) because of an intermittent failure that shows up roughly 4 times per month.The reported failure is not an assertion failure. In the CI log the inner pytest process dies without printing a summary or a traceback, right after the KV cache allocation of a
test_beam_search_e2e[multi_process-...]case; the outer wrapper then only surfaces the genericAssertionError: failure reported in unittests.Waiving the whole entry drops coverage for the entire sampler directory on Blackwell, which is far broader than the intermittent failure itself. This PR removes the waive so the case runs in CI again and the intermittent failure can be characterized on current main.
Test Coverage
unittest/_torch/sampler -k "not test_speculative_d2h_parity_real_predictor"(DGX_B200-PyTorch-1) is re-enabled; no other test is affected.Local run on an 8xB200 node at
bd90276f6b, bypassing the waive list:PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.