uvm: coalesce TLB batch ranges and pick least-busy channels

[aljojoby9]

Two small nvidia-uvm.ko hot-path changes. No HAL, lock-order, ISR, or membar policy changes. tlb_batch.max_ranges is left at 8.

TLB batch coalescing

uvm_tlb_batch_invalidate() currently stores every interval in a 4-slot batch, then falls back to PDB-wide invalidate-all. Sequential 4K PTE writes (block_gpu_pte_write_4k) and ATS region flushes burn those slots immediately.

Merge the new half-open interval into the last queued range when they overlap or touch and neither start + size wraps. On merge, OR page_sizes, keep uvm_membar_max, and do not increment count or total_ranges. Wrap and out-of-bounds ranges[] access are real if checks (not UVM_ASSERT, which is compiled out of release).

Flush still uses min page size of the OR'd mask for the HAL page_size argument and max page size for depth, matching tlb_batch_flush_invalidate_per_va().

Tests are in UVM_TEST_PAGE_TREE (test_tlb_batch_coalesce): adjacent, overlap, reverse-adjacent, gap, mixed page sizes, membar, 4-slot cap, count==4 touch, already-all + touch, and wrap of both the new and last range. Existing gapped spacing (base + j * 2 * size) is unchanged.

Least-busy channel reserve (fast path)

channel_reserve_in_pool() first-fits channel 0 (Bug 1764953). Under one pool lock, pick the channel with the most free GPFIFO entries and rotate next_hint on ties.

The spin-wait path stays first-fit from index 0. Saturated pools still prefer channel 0 once no slot is free. The same pick is used on the Confidential Computing first scan (skip_locked_for_push is CC-only). uvm_down(push_sem) still happens before that scan. The pool lock is not held across uvm_channel_update_progress().

UVM_TEST_CHANNEL_SANITY now checks that the first N concurrent begins on an idle pool with num_channels >= 2 land on N distinct channels. NULL SEC2/WLC/LCIC defaults are skipped when CC is off. The pool is waited idle first so leftover GPFIFO from earlier sanity cases does not hide the pick.

Out of scope

• Tracker-aware pick (uvm_push.c)
• Wait-path least-busy
• Bug 1767241 retune of max_ranges
• Bug 3770539 SYS membar on semaphore release
• src/ OS-agnostic code

[CLAassistant]

All committers have signed the CLA.

[Copilot]

Copilot review overview

🟢 Approval recommended

The changes are localized, preserve existing fallback behavior, and add targeted tests covering the new coalescing and selection logic.

Review effort: Lite
Findings: None

What changed in this PR

This PR improves two nvidia-uvm.ko hot paths by (1) reducing avoidable TLB batch range consumption through range coalescing and (2) reducing channel-0 contention by selecting the least-busy channel (by available GPFIFO entries) on the fast reserve path, with accompanying unit tests for both behaviors.

Changes:

• Coalesce overlapping/adjacent TLB invalidate ranges into the last queued batch range to avoid premature fallback to invalidate-all.
• Select the least-busy channel under the pool lock for the fast-path reserve (and CC first scan), using a rotating hint to break ties.
• Add/extend test coverage for TLB coalescing and least-busy channel selection behavior.

File	Description
kernel-open/​nvidia-uvm/​uvm_tlb_batch.c	Adds “try coalesce with last range” logic to reduce TLB batch range pressure.
kernel-open/​nvidia-uvm/​uvm_page_tree_test.c	Adds test_tlb_batch_coalesce coverage for adjacency/overlap/page-size/membar/cap/wrap cases.
kernel-open/​nvidia-uvm/​uvm_channel.h	Adds next_hint to channel pool for tie-breaking/rotation in least-busy selection.
kernel-open/​nvidia-uvm/​uvm_channel.c	Implements least-busy channel picking under pool lock and updates reserve paths to use it.
kernel-open/​nvidia-uvm/​uvm_channel_test.c	Adds sanity test ensuring N outstanding begins on an idle multi-channel pool select distinct channels.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.