[TRTLLM-15011][infra] Unwaive TestDeepSeekV4Flash::test_auto_dtype

[lowsfer]

Description

Removes the waive for accuracy/test_llm_api_pytorch.py::TestDeepSeekV4Flash::test_auto_dtype
(nvbugs/6561677, TRTLLM-15011).

The test aborted during LLM teardown with:

RuntimeError: SlotAllocator::finishShrink: cannot finish shrink yet

thrown from SlotAllocator::finishShrink (storage/core.cpp), reached via
PoolGroupBase::destroy() → prepareForShrink(0) → finishShrink(). The shrink to zero
fails because a slot is still occupied, i.e. a page was still alive at shutdown.

nvbugs/6553427
(accuracy/test_llm_api_pytorch.py::TestQwen3_5_35B_A3B::test_fp8[enable_block_reuse=True],
RuntimeError: shrink can't be finished) is the same root cause seen from the Python backend:
the KV cache manager tears down while requests are still open, so KvCache objects outlive the
KVCacheManagerV2 that owns their pages.

Both are fixed by #17163, which drains in-flight requests before clearing the reuse state and
guards clearReusableBlocks() / shutdown() so a call with live KV caches is rejected up front
rather than corrupting the pool.

Note that 6553427 was never waived (it was a pre-merge failure) and is already closed as
Bug - Fixed, so this PR only has one line to remove.

Test Coverage

No new tests — this re-enables an existing one. The waived test ran in the post-merge stage
DGX_B200-4_GPUs-PyTorch-3, so CI must be run with that stage included:

/bot run --extra-stage "DGX_B200-4_GPUs-PyTorch-3"

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

🤖 Generated with Claude Code

Dev Engineer Review

• Removed the waiver for accuracy/test_llm_api_pytorch.py::TestDeepSeekV4Flash::test_auto_dtype from tests/integration/test_lists/waives.txt.
• The waiver removal matches the teardown fix from PR #17163.
• The change has no API, performance, or error-handling impact.
• The waiver file format and scope are correct.
• No duplicate waiver or invalid test path was introduced.
• CI should include the DGX_B200-4_GPUs-PyTorch-3 post-merge stage.

QA Engineer Review

• Modified test-list file: tests/integration/test_lists/waives.txt.
• Removed the waiver for accuracy/test_llm_api_pytorch.py::TestDeepSeekV4Flash::test_auto_dtype.
• No test-db/ or qa/ files were modified.
• No test code was added, modified, or removed.
• Verdict: needs follow-up because CBTS coverage data is unavailable.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: decc5304-9fe2-42d6-b215-69ec126b4a42

📥 Commits

Reviewing files that changed from the base of the PR and between df43752 and 7e74dbb.

📒 Files selected for processing (1)

• tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

• tests/integration/test_lists/waives.txt

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

The pull request removes the skip waiver for TestDeepSeekV4Flash::test_auto_dtype.

Changes

DeepSeek test waiver

Layer / File(s)	Summary
Remove DeepSeek auto-dtype waiver tests/integration/test_lists/waives.txt	Removes the waiver entry for TestDeepSeekV4Flash::test_auto_dtype.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Merge Risk: ⚪ Minimal · up to 7e74d

This localized change re-enables an existing DeepSeekV4Flash test without changing production code or runtime behavior. No actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description explains the issue, solution, affected test, CI coverage, and checklist status.
Title check	✅ Passed	The title clearly identifies the ticket, change type, and removal of the specific test waiver.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[lowsfer]

Superseded by #18095, which carries the actual fix for this bug family plus this waive removal.

This PR's premise was that #17163 fixed the failure. That turned out to be wrong: the same root cause kept reproducing after #17163 merged (see https://nvbugs/6625710, filed 11 days later for the same signature). #18095 fixes it properly — an SSM snapshot evicted from the radix tree was detaching a block a live request still held — and removes this waive in the same change.

[lowsfer]

Reopening — my earlier closing comment was wrong, and I want to correct the record.

I closed this claiming the same root cause kept reproducing after #17163 merged. That conflated two distinct failure signatures:

• SlotAllocator::finishShrink: cannot finish shrink yet (this bug, 6561677; also 6553427) — a page still occupied when the pool group shrinks to zero, i.e. shutdown running before all KvCaches are closed. That is what [None][fix] Drain in-flight requests before clearing the KV cache reuse state #17163 fixed.
• Segfault in Block::tokensPerBlock() (6625710) — a radix-tree block detached while a live request still held it. That one demonstrably survived [None][fix] Drain in-flight requests before clearing the KV cache reuse state #17163 and is fixed separately in [https://nvbugs/6625710][fix] Re-attach radix-tree blocks detached under a live request #18095.

The dates support the original premise here: 6561677 was filed Aug 4 and waived Aug 5, both before #17163 merged on Aug 6. It has produced no signal since, because it has been skipped ever since. 6625710, by contrast, was filed Aug 17 — after the merge — which is the evidence I mistakenly applied to this bug too.

A stale/orphaned block also cannot produce the finishShrink failure: the producing request keeps it alive via SeqBlock::treeBlock, and when the request closes, _clearBlocks() drops that reference, the block is destroyed and its pages are released. Nothing stays occupied into teardown. So #18095's fix is not the explanation for this failure — #17163 remains the best one.

Per the bug's own instructions, CI here still needs the post-merge stage that test runs in:

/bot run --extra-stage "DGX_B200-4_GPUs-PyTorch-3"

[lowsfer]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #68515 [ run ] triggered by Bot. Commit: fa715ce Link to invocation

[tensorrt-cicd]

PR_Github #68515 [ run ] completed with state SUCCESS. Commit: fa715ce
/LLM/main/L0_MergeRequest_PR pipeline #55931 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[lowsfer]

/bot run --disable-fail-fast

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[tensorrt-cicd]

PR_Github #68758 [ run ] triggered by Bot. Commit: 7e74dbb Link to invocation

[tensorrt-cicd]

PR_Github #68758 [ run ] completed with state FAILURE. Commit: 7e74dbb
/LLM/main/L0_MergeRequest_PR pipeline #56156 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation