Skip to content

drm/amdkfd: account for non-KFD DRM VRAM residency - #224

Open
liminfei-amd wants to merge 1 commit into
ROCm:masterfrom
liminfei-amd:amd-rocm/223-kfd-drm-vram-accounting
Open

drm/amdkfd: account for non-KFD DRM VRAM residency#224
liminfei-amd wants to merge 1 commit into
ROCm:masterfrom
liminfei-amd:amd-rocm/223-kfd-drm-vram-accounting

Conversation

@liminfei-amd

Copy link
Copy Markdown

Fixes #223.

Summary

KFD available-memory accounting does not include unpinned VRAM resident through
other DRM clients, so hipMemGetInfo can over-report free VRAM on Linux dGPUs.

Track non-KFD user BO residency with the VRAM resource lifetime and subtract it
from the KFD query and allocation limit. Mark BOs already represented by KFD
accounting to avoid double counting, and leave APU and memory-partition paths
unchanged.

Validation

  • RX 7900 GRE / gfx1100, ROCm 7.2.1, Ubuntu kernels 6.14 and 7.0.
  • Baseline: a 256 MiB libdrm VRAM BO reduced DRM free by 256 MiB and KFD free by
    0 B.
  • Patched module: the same allocation reduced both values by 256 MiB in 3/3
    runs; KFD allocation/free behavior remained unchanged.
  • Current master built a SVM-enabled amdgpu.o; strict checkpatch.pl
    reported 0 errors, 0 warnings, and 0 checks.

APU and XCP/memory-partition runtime coverage is not included yet.


This is an experimental, AI-generated code change, reviewed by AMD engineers before submission.

AMDKFD_IOC_AVAILABLE_MEMORY only subtracts KFD allocations and pinned
VRAM. Unpinned VRAM buffer objects allocated through DRM therefore remain
invisible to HSA_AMD_AGENT_INFO_MEMORY_AVAIL and can make compute
applications overcommit VRAM.

Track non-KFD user BO residency in the VRAM resource manager and subtract
it from KFD availability. Mark KFD allocations already represented by
vram_used_aligned to avoid double counting, and keep an orthogonal counter
for pinned VRAM not represented by either source. Preserve the existing
paths on APUs and partitioned devices until their memory-domain attribution
is validated.

Closes: ROCm#223
Signed-off-by: liminfei-amd <91481003+liminfei-amd@users.noreply.github.com>
@liminfei-amd
liminfei-amd marked this pull request as ready for review August 5, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KFD available-memory ioctl ignores non-KFD DRM VRAM residency

1 participant