[TRTLLM-15078][test] Prune non-Llama-3.1-8B Llama tests

[xinhe-nv]

Dev Engineer Review

• Pruned non-Llama-3.1-8B model references, accuracy data, performance configurations, and integration tests.
• Replaced selected TinyLlama usage with Llama-3.1-8B-Instruct.
• Removed Ray, MPI, LoRA, KV-cache, LLM API, and disaggregated-serving coverage.
• Added Blackwell SM120 Mixtral E2E coverage and Qwen3 Eagle3 coverage.
• Validate removed functional coverage, model paths, YAML files, test references, waiver entries, and test_llm.py golden outputs.
• No production runtime changes were identified.

QA Engineer Review

Test code changed across accuracy, disaggregated serving, E2E, KV-cache, LLM API, Ray, MPI, LoRA, sampler, and AutoDeploy areas.

Removed or modified tests include accuracy classes, disaggregated-serving functions, Ray and MPI examples, KV-cache tests, LLM API tests, LoRA helpers, sampler tests, and model-path fixtures. Several retained tests now use Llama-3.1-8B-Instruct.

Affected manifests:

• tests/integration/test_lists/qa/llm_function_core.txt
• tests/integration/test_lists/qa/llm_spark_core.txt
• tests/integration/test_lists/qa/llm_spark_func.yml
• tests/integration/test_lists/qa/llm_spark_perf.yml
• tests/integration/test_lists/test-db/l0_a10.yml
• tests/integration/test_lists/test-db/l0_a100.yml
• tests/integration/test_lists/test-db/l0_b200.yml
• tests/integration/test_lists/test-db/l0_dgx_b200.yml
• tests/integration/test_lists/test-db/l0_dgx_h100.yml
• tests/integration/test_lists/test-db/l0_dgx_h200.yml
• tests/integration/test_lists/test-db/l0_gb200_multi_gpus.yml
• tests/integration/test_lists/test-db/l0_h100.yml
• tests/integration/test_lists/test-db/l0_l40s.yml
• tests/integration/test_lists/test-db/l0_sanity_check.yml
• tests/integration/test_lists/waives.txt

CBTS coverage data is unavailable. Verdict: needs follow-up.

Description

The background is https://docs.google.com/spreadsheets/d/17qLRr5oQESqn1rT9LrIUnpjvhH8wcC59273YEPT9k44/edit?gid=177162358#gid=177162358

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[xinhe-nv]

@coderabbitai review the change, we need to retire llama tests, if the test is must have, please show me in a table.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

The PR removes obsolete model accuracy, performance, disaggregated-serving, Ray, LoRA, and API test coverage. It updates integration manifests and retargets selected unit tests from TinyLlama to Llama 3.1 8B Instruct.

Changes

Test matrix cleanup

Layer / File(s)	Summary
Model references and accuracy coverage tests/integration/defs/accuracy/*, tests/integration/defs/conftest.py, tests/integration/defs/perf/*, tests/test_common/llm_data.py	Removed obsolete accuracy records, model mappings, fixtures, and performance configurations.
Disaggregated and Ray coverage tests/integration/defs/disaggregated/*, tests/integration/defs/examples/test_ray.py	Removed TinyLlama disaggregated tests, Ray flows, model helpers, and configurations.
Integration definitions and manifests tests/integration/defs/test_e2e.py, tests/integration/test_lists/**, tests/scripts/perf-sanity/**	Removed obsolete cases and retained or added current Qwen, DeepSeek, GPT-OSS, and decoder-encoding coverage.
Unit-test model migration tests/unittest/_torch/**, tests/unittest/llmapi/test_async_llm.py, tests/unittest/llmapi/test_executor.py, tests/unittest/llmapi/test_grpc.py	Changed selected tests to use Llama 3.1 8B Instruct.
LLM API and LoRA cleanup tests/unittest/llmapi/apps/*, tests/unittest/llmapi/lora_test_utils.py, tests/unittest/llmapi/test_llm_pytorch.py	Removed obsolete API and LoRA harnesses. Updated the remaining GQA LoRA test dimensions and output checks.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#14660: Removes deprecated model coverage and shared model-path mappings.
• NVIDIA/TensorRT-LLM#17444: Removes obsolete entries from the same waiver file.
• NVIDIA/TensorRT-LLM#17487: Reorganizes Ray-related modules and imports used by these tests.

Suggested reviewers: qijune, pcastonguay, joyang-nv

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description provides only a background link and does not explain the changes, rationale, or relevant test coverage.	Add a concise description of the changes and rationale, and list the relevant tests or validation performed.
Docstring Coverage	⚠️ Warning	Docstring coverage is 32.61% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: pruning tests for models other than Llama-3.1-8B.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

⚔️ Resolve merge conflicts 💡

• Resolve merge conflict in branch prune-llama

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[xinhe-nv]

/bot run

[tensorrt-cicd]

PR_Github #66723 [ run ] triggered by Bot. Commit: 8990ba0 Link to invocation

[tensorrt-cicd]

PR_Github #66723 [ run ] completed with state FAILURE. Commit: 8990ba0
/LLM/main/L0_MergeRequest_PR pipeline #54334 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation