* [PATCH v3 0/5] drm/amdgpu: SVM VRAM migration via drm_pagemap
@ 2026-04-27 10:05 Junhua Shen
2026-04-27 10:05 ` [PATCH v3 1/5] drm/amdgpu: add VRAM migration infrastructure for drm_pagemap Junhua Shen
` (5 more replies)
0 siblings, 6 replies; 13+ messages in thread
From: Junhua Shen @ 2026-04-27 10:05 UTC (permalink / raw)
To: Alexander.Deucher, Felix.Kuehling, Christian.Koenig, Oak.Zeng,
Jenny-Jing.Liu, Philip.Yang, Xiaogang.Chen, Ray.Huang,
honglei1.huang, Lingshan.Zhu
Cc: amd-gfx, dri-devel, Junhua Shen
This series adds VRAM migration support to the amdgpu SVM (Shared
Virtual Memory) subsystem, built on top of the drm_pagemap
infrastructure [1].
It enables transparent page migration between system RAM and device
VRAM using SDMA, driven by userspace SVM attribute hints (prefetch
location, access attributes) through the existing AMDGPU SVM ioctl
interface.
Limitations:
- Single GPU only; multi-GPU migration is not addressed
- No XNACK-on GPU fault-driven migration (XNACK-off ioctl
path only)
- No VRAM-to-VRAM (peer GPU) migration
- No eviction fence / VRAM overcommit handling yet
Design highlights:
- ZONE_DEVICE pages managed via devm_memremap_pages / drm_pagemap
- SDMA-based migration with proper DMA fence synchronization
- Migration decision layer that evaluates SVM attributes to
determine when and where to migrate
- Zero modifications to the KFD subsystem
Patch breakdown:
1. Core VRAM migration infrastructure (ZONE_DEVICE, drm_pagemap_ops)
2. SDMA migration callbacks (copy_to_ram / copy_to_dev)
3. Migration decision layer (policy evaluation)
4. SVM attribute extensions (prefetch force-trigger)
5. Integration into SVM range map path + ZONE_DEVICE registration
Built on top of the drm_pagemap SVM series [1].
Changes since v2:
- Moved amdgpu_pagemap entirely to amdgpu side, eliminating all KFD
modifications
- Split commits for better reviewability: separated infrastructure
from SDMA callbacks, decision layer from integration
- Merged ZONE_DEVICE registration hook into the integration patch
Changes since v1:
- Dropped the eviction fence patch per Christian König's review
(violates dma_fence contract)
[1] https://lore.kernel.org/all/20260317-drm-svm-v2-0-4bceef04e41e@amd.com/
v1: https://lore.kernel.org/all/20260410113146.146212-1-Junhua.Shen@amd.com/
v2: https://lore.kernel.org/all/20260413103031.181953-1-Junhua.Shen@amd.com/
Junhua Shen (5):
drm/amdgpu: add VRAM migration infrastructure for drm_pagemap
drm/amdgpu: implement drm_pagemap SDMA migration callbacks
drm/amdgpu: introduce SVM range migration decision layer
drm/amdgpu: add SVM attr prefetch/force-trigger functionality
drm/amdgpu: integrate VRAM migration into SVM range map path
drivers/gpu/drm/amd/amdgpu/Makefile | 6 +-
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 8 +
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +
drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c | 789 ++++++++++++++++++
drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h | 98 +++
drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c | 4 +
drivers/gpu/drm/amd/amdgpu/amdgpu_svm.c | 4 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c | 34 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c | 136 +--
drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.h | 5 +-
.../drm/amd/amdgpu/amdgpu_svm_range_migrate.c | 140 ++++
.../drm/amd/amdgpu/amdgpu_svm_range_migrate.h | 60 ++
12 files changed, 1212 insertions(+), 76 deletions(-)
create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h
create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c
create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v3 1/5] drm/amdgpu: add VRAM migration infrastructure for drm_pagemap
2026-04-27 10:05 [PATCH v3 0/5] drm/amdgpu: SVM VRAM migration via drm_pagemap Junhua Shen
@ 2026-04-27 10:05 ` Junhua Shen
2026-04-28 4:43 ` Claude review: " Claude Code Review Bot
2026-04-27 10:05 ` [PATCH v3 2/5] drm/amdgpu: implement drm_pagemap SDMA migration callbacks Junhua Shen
` (4 subsequent siblings)
5 siblings, 1 reply; 13+ messages in thread
From: Junhua Shen @ 2026-04-27 10:05 UTC (permalink / raw)
To: Alexander.Deucher, Felix.Kuehling, Christian.Koenig, Oak.Zeng,
Jenny-Jing.Liu, Philip.Yang, Xiaogang.Chen, Ray.Huang,
honglei1.huang, Lingshan.Zhu
Cc: amd-gfx, dri-devel, Junhua Shen
Add the drm_pagemap-based VRAM migration infrastructure:
- Define struct amdgpu_pagemap wrapping dev_pagemap + drm_pagemap
- Define AMDGPU_SVM_PGMAP_OWNER() and AMDGPU_INTERCONNECT_VRAM macros
- Implement amdgpu_svm_migration_init() to register ZONE_DEVICE via
devm_memremap_pages() and initialize the drm_pagemap
- Add amdgpu_pagemap pointer (apagemap) to struct amdgpu_device
Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
---
drivers/gpu/drm/amd/amdgpu/Makefile | 6 +-
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 8 +
drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c | 179 ++++++++++++++++++++
drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h | 98 +++++++++++
4 files changed, 289 insertions(+), 2 deletions(-)
create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile b/drivers/gpu/drm/amd/amdgpu/Makefile
index 7700f81a246e..e64abb5c8ab8 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -323,12 +323,14 @@ amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_hmm.o
# svm support
amdgpu-$(CONFIG_DRM_AMDGPU_SVM) += amdgpu_svm.o amdgpu_svm_attr.o \
- amdgpu_svm_range.o
+ amdgpu_svm_range.o amdgpu_migrate.o
.PHONY: clean-svm
clean-svm:
rm -f $(obj)/amdgpu_svm.o $(obj)/amdgpu_svm_attr.o $(obj)/amdgpu_svm_range.o \
- $(obj)/.amdgpu_svm.o.cmd $(obj)/.amdgpu_svm_attr.o.cmd $(obj)/.amdgpu_svm_range.o.cmd
+ $(obj)/amdgpu_migrate.o \
+ $(obj)/.amdgpu_svm.o.cmd $(obj)/.amdgpu_svm_attr.o.cmd $(obj)/.amdgpu_svm_range.o.cmd \
+ $(obj)/.amdgpu_migrate.o.cmd
include $(FULL_AMD_PATH)/pm/Makefile
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 49e7881750fa..fe6ba9911d9f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -325,6 +325,7 @@ struct amdgpu_fpriv;
struct amdgpu_bo_va_mapping;
struct kfd_vm_fault_info;
struct amdgpu_hive_info;
+struct amdgpu_pagemap;
struct amdgpu_reset_context;
struct amdgpu_reset_control;
struct amdgpu_coredump_info;
@@ -1200,6 +1201,13 @@ struct amdgpu_device {
struct amdgpu_uma_carveout_info uma_info;
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
+ /* SVM VRAM migration via drm_pagemap (drm_gpusvm path).
+ * Allocated in amdgpu_svm_migration_init(), NULL if SVM disabled.
+ */
+ struct amdgpu_pagemap *apagemap;
+#endif
+
/* KFD
* Must be last --ends in a flexible-array member.
*/
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
new file mode 100644
index 000000000000..170e2eadc106
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright 2026 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+/**
+ * DOC: AMDGPU SVM Migration
+ *
+ * This file implements the drm_pagemap-based migration infrastructure for
+ * AMDGPU SVM. It provides the callbacks that the DRM GPUSVM / drm_pagemap
+ * framework needs to:
+ *
+ * 1. Map ZONE_DEVICE pages to GPU-visible VRAM MC addresses (device_map)
+ * 2. Allocate VRAM and migrate pages from system memory (populate_mm)
+ * 3. Copy data between RAM and VRAM using SDMA (copy_to_devmem / copy_to_ram)
+ * 4. Release VRAM backing when pages migrate back to system memory (devmem_release)
+ *
+ * Architecture overview::
+ *
+ * adev->apagemap->dpagemap (struct drm_pagemap)
+ * .ops = &amdgpu_svm_drm_pagemap_ops
+ * |
+ * +---+-------------------+
+ * | |
+ * .populate_mm .device_map
+ * (alloc BO + migrate) (page -> VRAM MC addr)
+ * |
+ * v
+ * drm_pagemap_devmem_ops (per-BO migration mechanics)
+ * .populate_devmem_pfn -> BO buddy blocks -> PFN array
+ * .copy_to_devmem -> SDMA copy RAM -> VRAM
+ * .copy_to_ram -> SDMA copy VRAM -> RAM
+ * .devmem_release -> release BO reference
+ *
+ * The three address spaces involved:
+ *
+ * VRAM offset [0, real_vram_size) - buddy allocator managed
+ * + hpa_base
+ * HPA / PFN [hpa_base, hpa_base+..) - ZONE_DEVICE struct page management
+ * + vm_manager.vram_base_offset
+ * PTE address [vram_base_offset, ..] - GPU page table entries (from MMHUB FB_OFFSET)
+ */
+
+#include <drm/drm_pagemap.h>
+#include <linux/memremap.h>
+#include <linux/migrate.h>
+
+#include "amdgpu_amdkfd.h"
+#include "amdgpu_migrate.h"
+#include "amdgpu.h"
+
+static inline struct amdgpu_pagemap *
+dpagemap_to_apagemap(struct drm_pagemap *dpagemap)
+{
+ return container_of(dpagemap, struct amdgpu_pagemap, dpagemap);
+}
+
+static inline struct amdgpu_device *
+dpagemap_to_adev(struct drm_pagemap *dpagemap)
+{
+ return drm_to_adev(dpagemap->drm);
+}
+
+/**
+ * amdgpu_svm_page_to_apagemap - Get amdgpu_pagemap from a ZONE_DEVICE page
+ * @page: A ZONE_DEVICE page backed by VRAM
+ *
+ * Follows: page -> pgmap -> container_of(apagemap)
+ */
+static inline struct amdgpu_pagemap *
+amdgpu_svm_page_to_apagemap(struct page *page)
+{
+ struct dev_pagemap *pgmap = page_pgmap(page);
+
+ return container_of(pgmap, struct amdgpu_pagemap, pgmap);
+}
+
+
+const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops = { };
+
+/**
+ * amdgpu_svm_migration_init - Register ZONE_DEVICE and initialize drm_pagemap
+ * @adev: AMDGPU device to set up VRAM migration for
+ *
+ * Allocates a ZONE_DEVICE region covering the GPU's VRAM, registers it
+ * via devm_memremap_pages() with drm_pagemap's generic dev_pagemap_ops,
+ * and then initializes the drm_pagemap (dpagemap) that provides the
+ * device_map / populate_mm callbacks for the DRM GPUSVM migration path.
+ *
+ * For XGMI-connected CPUs, uses the device's aperture directly
+ * (MEMORY_DEVICE_COHERENT). For discrete GPUs, requests a free
+ * iomem region for MEMORY_DEVICE_PRIVATE pages.
+ *
+ * Return: 0 on success, -EINVAL if GPU IP too old, negative error on failure
+ */
+int amdgpu_svm_migration_init(struct amdgpu_device *adev)
+{
+ struct amdgpu_pagemap *svm_dm;
+ struct dev_pagemap *pgmap;
+ struct resource *res = NULL;
+ unsigned long size;
+ void *r;
+
+ if (amdgpu_ip_version(adev, GC_HWIP, 0) < IP_VERSION(9, 0, 1))
+ return -EINVAL;
+
+ if (adev->apu_prefer_gtt)
+ return 0;
+
+ if (adev->apagemap && adev->apagemap->initialized)
+ return 0;
+
+ svm_dm = devm_kzalloc(adev->dev, sizeof(*svm_dm), GFP_KERNEL);
+ if (!svm_dm)
+ return -ENOMEM;
+
+ pgmap = &svm_dm->pgmap;
+
+ size = ALIGN(adev->gmc.real_vram_size, 2ULL << 20);
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ pgmap->range.start = adev->gmc.aper_base;
+ pgmap->range.end = adev->gmc.aper_base + adev->gmc.aper_size - 1;
+ pgmap->type = MEMORY_DEVICE_COHERENT;
+ } else {
+ res = devm_request_free_mem_region(adev->dev, &iomem_resource, size);
+ if (IS_ERR(res))
+ return PTR_ERR(res);
+ pgmap->range.start = res->start;
+ pgmap->range.end = res->end;
+ pgmap->type = MEMORY_DEVICE_PRIVATE;
+ }
+
+ pgmap->nr_range = 1;
+ pgmap->flags = 0;
+ pgmap->ops = drm_pagemap_pagemap_ops_get();
+ pgmap->owner = AMDGPU_SVM_PGMAP_OWNER(adev);
+
+ r = devm_memremap_pages(adev->dev, pgmap);
+ if (IS_ERR(r)) {
+ dev_err(adev->dev, "SVM: failed to register HMM device memory\n");
+ if (pgmap->type == MEMORY_DEVICE_PRIVATE && res)
+ devm_release_mem_region(adev->dev, res->start, resource_size(res));
+ pgmap->type = 0;
+ return PTR_ERR(r);
+ }
+
+ if (drm_pagemap_init(&svm_dm->dpagemap, pgmap, adev_to_drm(adev),
+ &amdgpu_svm_drm_pagemap_ops)) {
+ dev_err(adev->dev, "SVM: failed to init drm_pagemap\n");
+ return -EINVAL;
+ }
+ svm_dm->adev = adev;
+ svm_dm->hpa_base = pgmap->range.start;
+ svm_dm->initialized = true;
+ adev->apagemap = svm_dm;
+
+ dev_info(adev->dev, "SVM: registered %ldMB device memory, hpa_base=0x%llx\n",
+ size >> 20, svm_dm->hpa_base);
+ return 0;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h
new file mode 100644
index 000000000000..e20698fb1597
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h
@@ -0,0 +1,98 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright 2026 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __AMDGPU_MIGRATE_H__
+#define __AMDGPU_MIGRATE_H__
+
+#include <drm/drm_pagemap.h>
+#include <linux/memremap.h>
+
+struct amdgpu_device;
+
+/*
+ * AMDGPU_INTERCONNECT_VRAM - Protocol identifier for local VRAM access.
+ *
+ * Used in drm_pagemap_addr to distinguish device-local VRAM addresses from
+ * DMA-mapped system memory addresses. drm_gpusvm_get_pages() uses this to
+ * identify pages that are already in device memory and need no DMA mapping.
+ *
+ * Value must be non-zero (0 == DRM_INTERCONNECT_SYSTEM).
+ */
+#define AMDGPU_INTERCONNECT_VRAM 1
+
+/*
+ * AMDGPU_SVM_PGMAP_OWNER - Unique owner token for dev_pagemap registration.
+ *
+ * migrate_vma_setup() uses pgmap->owner to distinguish "own" device pages
+ * from "foreign" device pages (e.g., another GPU in an XGMI hive).
+ * Pages whose page->pgmap->owner matches the migration source are skipped
+ * (they're already in the right place).
+ *
+ * For XGMI hive: all GPUs in the hive share the same owner (the hive pointer)
+ * so intra-hive pages are treated as local.
+ * For standalone GPU: use the adev pointer itself as a unique per-device token.
+ */
+#define AMDGPU_SVM_PGMAP_OWNER(adev) \
+ ((adev)->hive ? (void *)(adev)->hive : (void *)(adev))
+
+/**
+ * struct amdgpu_pagemap - VRAM migration infrastructure for drm_pagemap
+ * @dpagemap: DRM pagemap wrapper providing device_map/populate_mm callbacks
+ * @adev: back-pointer to the owning amdgpu_device
+ * @hpa_base: host physical address base of the ZONE_DEVICE region
+ * @initialized: set to true after successful registration
+ * @pgmap: the dev_pagemap registered with devm_memremap_pages();
+ * must be last — contains a flexible-array member (ranges[])
+ *
+ * Allocated with devm_kzalloc() in amdgpu_svm_migration_init() and stored
+ * as adev->apagemap. Lifetime is tied to the device via devres.
+ */
+struct amdgpu_pagemap {
+ struct drm_pagemap dpagemap;
+ struct amdgpu_device *adev;
+ resource_size_t hpa_base;
+ bool initialized;
+ struct dev_pagemap pgmap; /* must be last — flex-array */
+};
+
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
+int amdgpu_svm_migration_init(struct amdgpu_device *adev);
+#else
+static inline
+int amdgpu_svm_migration_init(struct amdgpu_device *adev)
+{
+ return 0;
+}
+#endif
+
+/**
+ * amdgpu_svm_drm_pagemap_ops - drm_pagemap_ops for AMDGPU VRAM migration
+ *
+ * Provides:
+ * .device_map - Convert ZONE_DEVICE page to VRAM address
+ * .populate_mm - Allocate VRAM BO and migrate pages from system memory
+ */
+extern const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops;
+
+#endif /* __AMDGPU_MIGRATE_H__ */
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3 2/5] drm/amdgpu: implement drm_pagemap SDMA migration callbacks
2026-04-27 10:05 [PATCH v3 0/5] drm/amdgpu: SVM VRAM migration via drm_pagemap Junhua Shen
2026-04-27 10:05 ` [PATCH v3 1/5] drm/amdgpu: add VRAM migration infrastructure for drm_pagemap Junhua Shen
@ 2026-04-27 10:05 ` Junhua Shen
2026-04-27 22:20 ` Felix Kuehling
2026-04-28 4:43 ` Claude review: " Claude Code Review Bot
2026-04-27 10:05 ` [PATCH v3 3/5] drm/amdgpu: introduce SVM range migration decision layer Junhua Shen
` (3 subsequent siblings)
5 siblings, 2 replies; 13+ messages in thread
From: Junhua Shen @ 2026-04-27 10:05 UTC (permalink / raw)
To: Alexander.Deucher, Felix.Kuehling, Christian.Koenig, Oak.Zeng,
Jenny-Jing.Liu, Philip.Yang, Xiaogang.Chen, Ray.Huang,
honglei1.huang, Lingshan.Zhu
Cc: amd-gfx, dri-devel, Junhua Shen
Implement the drm_pagemap_devmem_ops and drm_pagemap_ops callbacks
that the DRM GPUSVM migration framework requires:
drm_pagemap_ops (top-level entry points):
- device_map: convert ZONE_DEVICE page to GPU PTE address
- populate_mm: allocate VRAM BO and trigger migration
drm_pagemap_devmem_ops (per-BO migration mechanics):
- populate_devmem_pfn: walk BO buddy blocks to build PFN array
- copy_to_devmem: SDMA copy system RAM -> VRAM via GART window
- copy_to_ram: SDMA copy VRAM -> system RAM via GART window
- devmem_release: free BO when all pages migrate back
Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c | 616 +++++++++++++++++++-
1 file changed, 613 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
index 170e2eadc106..42092651b4d5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
@@ -64,12 +64,20 @@
#include <linux/memremap.h>
#include <linux/migrate.h>
-#include "amdgpu_amdkfd.h"
#include "amdgpu_migrate.h"
#include "amdgpu.h"
+#include "amdgpu_ttm.h"
+#include "amdgpu_res_cursor.h"
+
+#define AMDGPU_MIGRATE_TRACE(fmt, ...) \
+ pr_debug("%s: " fmt, __func__, ##__VA_ARGS__)
+
+/* SDMA copy direction */
+#define FROM_RAM_TO_VRAM 0
+#define FROM_VRAM_TO_RAM 1
static inline struct amdgpu_pagemap *
-dpagemap_to_apagemap(struct drm_pagemap *dpagemap)
+to_amdgpu_pagemap(struct drm_pagemap *dpagemap)
{
return container_of(dpagemap, struct amdgpu_pagemap, dpagemap);
}
@@ -94,8 +102,610 @@ amdgpu_svm_page_to_apagemap(struct page *page)
return container_of(pgmap, struct amdgpu_pagemap, pgmap);
}
+/* drm_pagemap_devmem_ops — per-BO migration mechanics */
+
+/**
+ * struct amdgpu_svm_bo - Wrapper linking drm_pagemap_devmem to amdgpu_bo
+ *
+ * @devmem: drm_pagemap device memory allocation (passed to framework)
+ * @bo: The backing VRAM amdgpu_bo
+ *
+ * It is allocated per-migration in populate_mm() and freed by
+ * devmem_release() when all device-private pages have migrated
+ * back to system memory.
+ *
+ * Lifecycle is managed by the drm_pagemap framework's internal zdd refcount:
+ * - zdd->devmem_allocation points to &svm_bo->devmem
+ * - When zdd refcount drops to zero, framework calls devmem_release()
+ * - devmem_release() frees both the BO reference and the svm_bo itself
+ */
+struct amdgpu_svm_bo {
+ struct amdgpu_bo *bo;
+ struct drm_pagemap_devmem devmem;
+};
+
+static inline struct amdgpu_svm_bo *
+to_amdgpu_svm_bo(struct drm_pagemap_devmem *devmem_allocation)
+{
+ return container_of(devmem_allocation, struct amdgpu_svm_bo, devmem);
+}
+
+/**
+ * amdgpu_svm_devmem_release - Release BO when all device pages migrate back
+ *
+ * Called by the drm_pagemap framework (via drm_pagemap_zdd_destroy) when the
+ * last device-private page backed by this allocation has been migrated back
+ * to system memory (or the owning process exits).
+ *
+ * Frees both the amdgpu_bo reference and the wrapper amdgpu_svm_bo itself.
+ */
+static void
+amdgpu_svm_devmem_release(struct drm_pagemap_devmem *devmem_allocation)
+{
+ struct amdgpu_svm_bo *svm_bo = to_amdgpu_svm_bo(devmem_allocation);
+
+ AMDGPU_MIGRATE_TRACE("Release svm_bo=%px bo=%px\n", svm_bo, svm_bo->bo);
+ amdgpu_bo_unref(&svm_bo->bo);
+ kfree(svm_bo);
+}
+
+/**
+ * amdgpu_svm_populate_devmem_pfn - Convert BO VRAM allocation to PFN array
+ * @devmem_allocation: The devmem allocation in the amdgpu_svm_bo wrapper
+ * @npages: Number of PFN entries to fill
+ * @pfn: Output PFN array
+ *
+ * Iterates over the BO's TTM vram_mgr buddy blocks and converts each
+ * block's VRAM offset to ZONE_DEVICE PFNs:
+ *
+ * PFN = PHYS_PFN(block_offset + apagemap.hpa_base) + page_index
+ *
+ * This is called by drm_pagemap_migrate_to_devmem() to build the
+ * destination PFN array for migrate_vma_pages().
+ *
+ * Return: 0 on success
+ */
+static int
+amdgpu_svm_populate_devmem_pfn(struct drm_pagemap_devmem *devmem_allocation,
+ unsigned long npages, unsigned long *pfn)
+{
+ struct amdgpu_pagemap *svm_dm = to_amdgpu_pagemap(devmem_allocation->dpagemap);
+ struct amdgpu_svm_bo *svm_bo = to_amdgpu_svm_bo(devmem_allocation);
+ struct amdgpu_bo *bo = svm_bo->bo;
+ struct amdgpu_res_cursor cursor;
+ unsigned long i = 0;
+ int ret;
+
+ ret = amdgpu_bo_reserve(bo, false);
+ if (ret)
+ return ret;
+
+ amdgpu_res_first(bo->tbo.resource, 0, npages << PAGE_SHIFT, &cursor);
+ while (cursor.remaining && i < npages) {
+ u64 pfn_base = PHYS_PFN(cursor.start + svm_dm->hpa_base);
+ u64 pages = cursor.size >> PAGE_SHIFT;
+ unsigned long j;
+
+ for (j = 0; j < pages && i < npages; j++, i++)
+ pfn[i] = pfn_base + j;
+
+ amdgpu_res_next(&cursor, cursor.size);
+ }
+
+ amdgpu_bo_unreserve(bo);
+
+ AMDGPU_MIGRATE_TRACE("populate_devmem_pfn: npages=%lu first_pfn=0x%lx\n",
+ npages, npages > 0 ? pfn[0] : 0);
+
+ return 0;
+}
+
+/* SDMA copy helpers — GART window based data transfer */
+
+/**
+ * amdgpu_svm_direct_mapping_addr - Convert VRAM offset to MC address
+ * @adev: AMDGPU device
+ * @vram_offset: Byte offset within VRAM
+ *
+ * Return: MC address suitable for SDMA src/dst
+ */
+static u64
+amdgpu_svm_direct_mapping_addr(struct amdgpu_device *adev, u64 vram_offset)
+{
+ return vram_offset + amdgpu_ttm_domain_start(adev, TTM_PL_VRAM);
+}
+
+/**
+ * amdgpu_svm_gart_map - Map system DMA addresses into GART window
+ * @ring: SDMA ring for the GART update job
+ * @npages: Number of pages to map
+ * @addr: Array of system memory DMA addresses
+ * @gart_addr: Output — GART base address to use in SDMA copy
+ * @flags: PTE flags (e.g. writeable for RAM-to-VRAM src)
+ *
+ * Builds GART PTEs pointing at the given DMA addresses, submits an
+ * SDMA job to update the GART entries, and returns the GART address
+ * that can be used as src or dst in a subsequent amdgpu_copy_buffer().
+ *
+ * Uses GART window 0, protected by gtt_window_lock.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+amdgpu_svm_gart_map(struct amdgpu_ring *ring,
+ struct amdgpu_ttm_buffer_entity *entity,
+ u64 npages,
+ dma_addr_t *addr, u64 *gart_addr, u64 flags)
+{
+ struct amdgpu_device *adev = ring->adev;
+ struct amdgpu_job *job;
+ unsigned int num_dw, num_bytes;
+ struct dma_fence *fence;
+ u64 src_addr, dst_addr;
+ u64 pte_flags;
+ void *cpu_addr;
+ int r;
+
+ /* Use entity's GART window 0 */
+ *gart_addr = amdgpu_compute_gart_address(&adev->gmc, entity, 0);
+
+ num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8);
+ num_bytes = npages * 8 * AMDGPU_GPU_PAGES_IN_CPU_PAGE;
+
+ r = amdgpu_job_alloc_with_ib(adev, &entity->base,
+ AMDGPU_FENCE_OWNER_UNDEFINED,
+ num_dw * 4 + num_bytes,
+ AMDGPU_IB_POOL_DELAYED,
+ &job,
+ AMDGPU_KERNEL_JOB_ID_KFD_GART_MAP);
+ if (r)
+ return r;
+
+ src_addr = num_dw * 4;
+ src_addr += job->ibs[0].gpu_addr;
+
+ dst_addr = amdgpu_bo_gpu_offset(adev->gart.bo);
+ dst_addr += (entity->gart_window_offs[0] >> AMDGPU_GPU_PAGE_SHIFT) * 8;
+ amdgpu_emit_copy_buffer(adev, &job->ibs[0], src_addr,
+ dst_addr, num_bytes, 0);
+
+ amdgpu_ring_pad_ib(ring, &job->ibs[0]);
+ WARN_ON(job->ibs[0].length_dw > num_dw);
+
+ pte_flags = AMDGPU_PTE_VALID | AMDGPU_PTE_READABLE;
+ pte_flags |= AMDGPU_PTE_SYSTEM | AMDGPU_PTE_SNOOPED;
+ if (flags & AMDGPU_PTE_WRITEABLE)
+ pte_flags |= AMDGPU_PTE_WRITEABLE;
+ pte_flags |= adev->gart.gart_pte_flags;
+
+ cpu_addr = &job->ibs[0].ptr[num_dw];
+
+ amdgpu_gart_map(adev, 0, npages, addr, pte_flags, cpu_addr);
+ fence = amdgpu_job_submit(job);
+ dma_fence_put(fence);
+
+ return 0;
+}
+
+/**
+ * amdgpu_svm_copy_memory_gart - SDMA copy between system RAM and VRAM
+ * @adev: AMDGPU device
+ * @sys: Array of DMA addresses for system memory pages
+ * @vram: Array of VRAM byte offsets (relative to start of VRAM)
+ * @npages: Number of pages to copy
+ * @direction: FROM_RAM_TO_VRAM or FROM_VRAM_TO_RAM
+ * @mfence: In/out — carries the last SDMA fence for serialization
+ *
+ * Maps system memory pages into the GART window and uses SDMA to copy
+ * data to/from VRAM. Handles splitting into AMDGPU_GTT_MAX_TRANSFER_SIZE
+ * chunks. Acquires entity->lock internally to protect the GART window,
+ * matching the KFD svm_migrate_copy_memory_gart() pattern.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+amdgpu_svm_copy_memory_gart(struct amdgpu_device *adev, dma_addr_t *sys,
+ u64 *vram, u64 npages, int direction,
+ struct dma_fence **mfence)
+{
+ const u64 max_pages = AMDGPU_GTT_MAX_TRANSFER_SIZE;
+ struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;
+ struct amdgpu_ttm_buffer_entity *entity = &adev->mman.move_entity;
+ u64 gart_s, gart_d;
+ struct dma_fence *next;
+ u64 size;
+ int r;
+
+ mutex_lock(&entity->lock);
+
+ while (npages) {
+ size = min(max_pages, npages);
+
+ if (direction == FROM_VRAM_TO_RAM) {
+ gart_s = amdgpu_svm_direct_mapping_addr(adev, *vram);
+ r = amdgpu_svm_gart_map(ring, entity, size, sys,
+ &gart_d, AMDGPU_PTE_WRITEABLE);
+ } else {
+ r = amdgpu_svm_gart_map(ring, entity, size, sys,
+ &gart_s, 0);
+ gart_d = amdgpu_svm_direct_mapping_addr(adev, *vram);
+ }
+ if (r) {
+ dev_err(adev->dev, "failed %d to map GART for SDMA\n", r);
+ goto out_unlock;
+ }
+
+ AMDGPU_MIGRATE_TRACE("SDMA_COPY: %s npages=%llu vram_off=0x%llx\n",
+ direction == FROM_RAM_TO_VRAM ? "RAM->VRAM" : "VRAM->RAM",
+ size, (u64)*vram);
+
+ r = amdgpu_copy_buffer(adev, entity, gart_s, gart_d,
+ size * PAGE_SIZE,
+ NULL, &next, true, 0);
+ if (r) {
+ dev_err(adev->dev, "failed %d to copy buffer\n", r);
+ goto out_unlock;
+ }
+
+ dma_fence_put(*mfence);
+ *mfence = next;
+ npages -= size;
+ if (npages) {
+ sys += size;
+ vram += size;
+ }
+ }
+
+out_unlock:
+ mutex_unlock(&entity->lock);
+
+ return r;
+}
+
+/**
+ * amdgpu_svm_copy_to_devmem - SDMA copy system memory -> VRAM
+ * @pages: Array of destination ZONE_DEVICE pages (VRAM-backed)
+ * @pagemap_addr: Array of source DMA addresses (system memory, already mapped)
+ * @npages: Number of pages to copy
+ *
+ * Builds parallel sys[] and vram[] arrays from the framework-provided
+ * pagemap_addr and device pages, then submits batched SDMA copies via
+ * the GART window.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+amdgpu_svm_copy_to_devmem(struct page **pages,
+ struct drm_pagemap_addr *pagemap_addr,
+ unsigned long npages,
+ struct dma_fence *pre_migrate_fence)
+{
+ struct amdgpu_device *adev;
+ struct amdgpu_pagemap *svm_dm;
+ struct dma_fence *mfence = NULL;
+ dma_addr_t *sys;
+ u64 *vram;
+ unsigned long i, j;
+ int ret = 0;
+
+ if (!npages)
+ return 0;
+
+ /*
+ * Find the first non-NULL page to derive the device.
+ * The pages array may contain NULL entries for positions where
+ * no valid device page exists.
+ */
+ for (i = 0; i < npages; i++) {
+ if (pages[i])
+ break;
+ }
+ if (i == npages)
+ return 0;
+
+ svm_dm = amdgpu_svm_page_to_apagemap(pages[i]);
+ adev = svm_dm->adev;
+
+ sys = kvcalloc(npages, sizeof(*sys), GFP_KERNEL);
+ vram = kvcalloc(npages, sizeof(*vram), GFP_KERNEL);
+ if (!sys || !vram) {
+ ret = -ENOMEM;
+ goto out_free;
+ }
+
+ for (i = 0, j = 0; i < npages; i++) {
+ if (!pagemap_addr[i].addr)
+ goto flush;
+
+ sys[j] = pagemap_addr[i].addr;
+ vram[j] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
+ svm_dm->hpa_base;
+
+ /* Check if next vram page is contiguous with current */
+ if (j > 0 && vram[j] != vram[j - 1] + PAGE_SIZE)
+ goto flush;
+
+ j++;
+ continue;
+flush:
+ if (j) {
+ ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
+ FROM_RAM_TO_VRAM,
+ &mfence);
+ if (ret)
+ goto out_fence;
+ j = 0;
+ }
+ /* Re-process current page if it was valid but broke contiguity */
+ if (pagemap_addr[i].addr) {
+ sys[0] = pagemap_addr[i].addr;
+ vram[0] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
+ svm_dm->hpa_base;
+ j = 1;
+ }
+ }
+
+ /* Flush remaining batch */
+ if (j)
+ ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
+ FROM_RAM_TO_VRAM, &mfence);
+
+out_fence:
+ if (mfence) {
+ dma_fence_wait(mfence, false);
+ dma_fence_put(mfence);
+ }
+
+ AMDGPU_MIGRATE_TRACE("copy_to_devmem done: npages=%ld ret=%d\n",
+ npages, ret);
+
+out_free:
+ kvfree(vram);
+ kvfree(sys);
+ return ret;
+}
+
+/**
+ * amdgpu_svm_copy_to_ram - SDMA copy VRAM -> system memory
+ * @pages: Array of source ZONE_DEVICE pages (VRAM-backed)
+ * @pagemap_addr: Array of destination DMA addresses (system memory, already mapped)
+ * @npages: Number of pages to copy
+ *
+ * Mirror of copy_to_devmem with src/dst swapped.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+amdgpu_svm_copy_to_ram(struct page **pages,
+ struct drm_pagemap_addr *pagemap_addr,
+ unsigned long npages,
+ struct dma_fence *pre_migrate_fence)
+{
+ struct amdgpu_device *adev;
+ struct amdgpu_pagemap *svm_dm;
+ struct dma_fence *mfence = NULL;
+ dma_addr_t *sys;
+ u64 *vram;
+ unsigned long i, j;
+ int ret = 0;
+
+ if (!npages)
+ return 0;
+
+ for (i = 0; i < npages; i++) {
+ if (pages[i])
+ break;
+ }
+ if (i == npages)
+ return 0;
+
+ svm_dm = amdgpu_svm_page_to_apagemap(pages[i]);
+ adev = svm_dm->adev;
+
+ sys = kvcalloc(npages, sizeof(*sys), GFP_KERNEL);
+ vram = kvcalloc(npages, sizeof(*vram), GFP_KERNEL);
+ if (!sys || !vram) {
+ ret = -ENOMEM;
+ goto out_free;
+ }
+
+ for (i = 0, j = 0; i < npages; i++) {
+ if (!pagemap_addr[i].addr || !pages[i])
+ goto flush;
+
+ vram[j] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
+ svm_dm->hpa_base;
+ sys[j] = pagemap_addr[i].addr;
+
+ /* Check if next vram page is contiguous with current */
+ if (j > 0 && vram[j] != vram[j - 1] + PAGE_SIZE)
+ goto flush;
+
+ j++;
+ continue;
+flush:
+ if (j) {
+ ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
+ FROM_VRAM_TO_RAM,
+ &mfence);
+ if (ret)
+ goto out_fence;
+ j = 0;
+ }
+ /* Re-process current page if it was valid but broke contiguity */
+ if (pagemap_addr[i].addr && pages[i]) {
+ vram[0] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
+ svm_dm->hpa_base;
+ sys[0] = pagemap_addr[i].addr;
+ j = 1;
+ }
+ }
+
+ /* Flush remaining batch */
+ if (j)
+ ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
+ FROM_VRAM_TO_RAM, &mfence);
+
+out_fence:
+ if (mfence) {
+ dma_fence_wait(mfence, false);
+ dma_fence_put(mfence);
+ }
+
+ AMDGPU_MIGRATE_TRACE("copy_to_ram done: npages=%ld ret=%d\n", npages, ret);
+
+out_free:
+ kvfree(vram);
+ kvfree(sys);
+ return ret;
+}
+
+static const struct drm_pagemap_devmem_ops amdgpu_pagemap_ops = {
+ .devmem_release = amdgpu_svm_devmem_release,
+ .populate_devmem_pfn = amdgpu_svm_populate_devmem_pfn,
+ .copy_to_devmem = amdgpu_svm_copy_to_devmem,
+ .copy_to_ram = amdgpu_svm_copy_to_ram,
+};
+
+/* drm_pagemap_ops — top-level migration entry points */
+
+/**
+ * amdgpu_svm_device_map - Convert ZONE_DEVICE page to GPU PTE address
+ * @dpagemap: The drm_pagemap for this device
+ * @dev: Requesting device (for P2P check)
+ * @page: ZONE_DEVICE page backed by VRAM
+ * @order: Page order (0 = 4K, 9 = 2M, etc.)
+ * @dir: DMA direction (unused for local VRAM)
+ *
+ * Address conversion chain:
+ * page -> PFN -> HPA -> VRAM offset -> PTE address
+ *
+ * HPA = page_to_pfn(page) << PAGE_SHIFT
+ * VRAM offset = HPA - apagemap.hpa_base
+ * PTE address = VRAM offset + adev->vm_manager.vram_base_offset
+ *
+ * Return: drm_pagemap_addr with PTE address and AMDGPU_INTERCONNECT_VRAM protocol
+ */
+static struct drm_pagemap_addr
+amdgpu_svm_device_map(struct drm_pagemap *dpagemap,
+ struct device *dev,
+ struct page *page,
+ unsigned int order,
+ enum dma_data_direction dir)
+{
+ struct amdgpu_pagemap *svm_dm = to_amdgpu_pagemap(dpagemap);
+ struct amdgpu_device *adev = dpagemap_to_adev(dpagemap);
+ dma_addr_t addr;
+
+ if (dpagemap->drm->dev == dev) {
+ /* Same device: return VRAM PTE address */
+ u64 hpa = (u64)page_to_pfn(page) << PAGE_SHIFT;
+ u64 vram_offset = hpa - svm_dm->hpa_base;
+
+ addr = vram_offset + adev->vm_manager.vram_base_offset;
+ } else {
+ /* Cross-device P2P: not yet supported */
+ addr = DMA_MAPPING_ERROR;
+ }
+
+ return drm_pagemap_addr_encode(addr,
+ AMDGPU_INTERCONNECT_VRAM, order, dir);
+}
+
+/**
+ * amdgpu_svm_bo_alloc - Allocate an amdgpu_svm_bo wrapper with VRAM backing
+ * @adev: AMDGPU device
+ * @dpagemap: The drm_pagemap for this device
+ * @mm: mm_struct of the owning process
+ * @size: Allocation size in bytes
+ *
+ * Return: Pointer to allocated amdgpu_svm_bo on success, ERR_PTR on failure
+ */
+static struct amdgpu_svm_bo *
+amdgpu_svm_bo_alloc(struct amdgpu_device *adev,
+ struct drm_pagemap *dpagemap,
+ struct mm_struct *mm, unsigned long size)
+{
+ struct amdgpu_svm_bo *svm_bo;
+ struct amdgpu_bo_param bp = {};
+ struct amdgpu_bo *bo;
+ int ret;
+
+ svm_bo = kzalloc(sizeof(*svm_bo), GFP_KERNEL);
+ if (!svm_bo)
+ return ERR_PTR(-ENOMEM);
+
+ bp.size = size;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
+ bp.domain = AMDGPU_GEM_DOMAIN_VRAM;
+ bp.type = ttm_bo_type_device;
+ bp.flags = AMDGPU_GEM_CREATE_NO_CPU_ACCESS |
+ AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS |
+ AMDGPU_GEM_CREATE_VRAM_CLEARED;
+
+ ret = amdgpu_bo_create(adev, &bp, &bo);
+ if (ret) {
+ AMDGPU_MIGRATE_TRACE("Failed to create SVM BO\n");
+ kfree(svm_bo);
+ return ERR_PTR(ret);
+ }
+
+ amdgpu_bo_unreserve(bo);
+ svm_bo->bo = bo;
+
+ drm_pagemap_devmem_init(&svm_bo->devmem,
+ adev->dev, mm,
+ &amdgpu_pagemap_ops,
+ dpagemap, size, NULL);
+
+ return svm_bo;
+}
+
+/**
+ * amdgpu_svm_populate_mm - Allocate VRAM BO and migrate pages
+ * @dpagemap: The drm_pagemap for this device
+ * @start: Start virtual address of the range to migrate
+ * @end: End virtual address (exclusive)
+ * @mm: mm_struct of the owning process
+ * @timeslice_ms: Maximum time to spend migrating (for fairness)
+ *
+ * Core migration entry point called by drm_pagemap_populate_mm().
+ * Allocates an amdgpu_svm_bo via amdgpu_svm_bo_alloc(), then calls
+ * drm_pagemap_migrate_to_devmem() to execute the actual migration.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+amdgpu_svm_populate_mm(struct drm_pagemap *dpagemap,
+ unsigned long start, unsigned long end,
+ struct mm_struct *mm,
+ unsigned long timeslice_ms)
+{
+ struct amdgpu_device *adev = dpagemap_to_adev(dpagemap);
+ struct drm_pagemap_migrate_details mdetails = {
+ .timeslice_ms = timeslice_ms,
+ };
+ struct amdgpu_svm_bo *svm_bo;
+ int ret;
+
+ svm_bo = amdgpu_svm_bo_alloc(adev, dpagemap, mm, end - start);
+ if (IS_ERR(svm_bo))
+ return PTR_ERR(svm_bo);
+
+ AMDGPU_MIGRATE_TRACE("populate_mm: [0x%lx-0x%lx] size=%lu\n",
+ start, end, end - start);
+
+ ret = drm_pagemap_migrate_to_devmem(&svm_bo->devmem,
+ mm, start, end,
+ &mdetails);
+
+ return ret;
+}
-const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops = { };
+const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops = {
+ .device_map = amdgpu_svm_device_map,
+ .populate_mm = amdgpu_svm_populate_mm,
+};
/**
* amdgpu_svm_migration_init - Register ZONE_DEVICE and initialize drm_pagemap
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3 3/5] drm/amdgpu: introduce SVM range migration decision layer
2026-04-27 10:05 [PATCH v3 0/5] drm/amdgpu: SVM VRAM migration via drm_pagemap Junhua Shen
2026-04-27 10:05 ` [PATCH v3 1/5] drm/amdgpu: add VRAM migration infrastructure for drm_pagemap Junhua Shen
2026-04-27 10:05 ` [PATCH v3 2/5] drm/amdgpu: implement drm_pagemap SDMA migration callbacks Junhua Shen
@ 2026-04-27 10:05 ` Junhua Shen
2026-04-28 4:43 ` Claude review: " Claude Code Review Bot
2026-04-27 10:05 ` [PATCH v3 4/5] drm/amdgpu: add SVM attr prefetch/force-trigger functionality Junhua Shen
` (2 subsequent siblings)
5 siblings, 1 reply; 13+ messages in thread
From: Junhua Shen @ 2026-04-27 10:05 UTC (permalink / raw)
To: Alexander.Deucher, Felix.Kuehling, Christian.Koenig, Oak.Zeng,
Jenny-Jing.Liu, Philip.Yang, Xiaogang.Chen, Ray.Huang,
honglei1.huang, Lingshan.Zhu
Cc: amd-gfx, dri-devel, Junhua Shen
Add the migration decision infrastructure for drm_pagemap-based SVM
VRAM migration:
- Define enum amdgpu_svm_migrate_mode (NONE, TO_VRAM, TO_SYSMEM,
PREFERRED) to carry migration intent
- Implement amdgpu_svm_range_migrate_range() which handles both
directions: TO_VRAM via drm_pagemap_populate_mm(), TO_SYSMEM via
drm_gpusvm_range_evict()
Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
---
drivers/gpu/drm/amd/amdgpu/Makefile | 6 +-
.../drm/amd/amdgpu/amdgpu_svm_range_migrate.c | 140 ++++++++++++++++++
.../drm/amd/amdgpu/amdgpu_svm_range_migrate.h | 60 ++++++++
3 files changed, 203 insertions(+), 3 deletions(-)
create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c
create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile b/drivers/gpu/drm/amd/amdgpu/Makefile
index e64abb5c8ab8..cf4f453b7e68 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -323,14 +323,14 @@ amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_hmm.o
# svm support
amdgpu-$(CONFIG_DRM_AMDGPU_SVM) += amdgpu_svm.o amdgpu_svm_attr.o \
- amdgpu_svm_range.o amdgpu_migrate.o
+ amdgpu_svm_range.o amdgpu_svm_range_migrate.o amdgpu_migrate.o
.PHONY: clean-svm
clean-svm:
rm -f $(obj)/amdgpu_svm.o $(obj)/amdgpu_svm_attr.o $(obj)/amdgpu_svm_range.o \
- $(obj)/amdgpu_migrate.o \
+ $(obj)/amdgpu_svm_range_migrate.o $(obj)/amdgpu_migrate.o \
$(obj)/.amdgpu_svm.o.cmd $(obj)/.amdgpu_svm_attr.o.cmd $(obj)/.amdgpu_svm_range.o.cmd \
- $(obj)/.amdgpu_migrate.o.cmd
+ $(obj)/.amdgpu_svm_range_migrate.o.cmd $(obj)/.amdgpu_migrate.o.cmd
include $(FULL_AMD_PATH)/pm/Makefile
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c
new file mode 100644
index 000000000000..f29d4cdb5c64
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c
@@ -0,0 +1,140 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright 2026 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "amdgpu.h"
+#include "amdgpu_svm.h"
+#include "amdgpu_svm_range.h"
+#include "amdgpu_migrate.h"
+#include "amdgpu_svm_range_migrate.h"
+
+static bool
+range_in_vram(struct drm_gpusvm_range *range)
+{
+ struct drm_gpusvm_pages_flags flags = {
+ /* Pairs with WRITE_ONCE in drm_gpusvm_get_pages() */
+ .__flags = READ_ONCE(range->pages.flags.__flags),
+ };
+
+ return flags.has_devmem_pages;
+}
+
+static struct drm_pagemap *
+amdgpu_svm_get_dpagemap(struct amdgpu_svm *svm)
+{
+ struct amdgpu_pagemap *apagemap = svm->adev->apagemap;
+
+ if (!apagemap->initialized)
+ return NULL;
+
+ return &apagemap->dpagemap;
+}
+
+static bool
+range_needs_migrate_to_vram(struct drm_gpusvm_range *range,
+ enum amdgpu_svm_migrate_mode mode)
+{
+ if (!range->pages.flags.migrate_devmem)
+ return false;
+
+ if (mode == AMDGPU_SVM_MIGRATE_TO_SYSMEM)
+ return false;
+
+ if (range_in_vram(range))
+ return false;
+
+ return true;
+}
+
+/**
+ * amdgpu_svm_range_migrate_to_vram - Migrate a single range's pages to VRAM
+ * @svm: Pointer to the AMDGPU SVM structure
+ * @range: Pointer to the GPU SVM range to migrate
+ *
+ * Return: 0 on success, negative errno on failure
+ */
+static int
+amdgpu_svm_range_migrate_to_vram(struct amdgpu_svm *svm,
+ struct drm_gpusvm_range *range)
+{
+ struct drm_pagemap *dpagemap = amdgpu_svm_get_dpagemap(svm);
+ unsigned long start = drm_gpusvm_range_start(range);
+ unsigned long end = drm_gpusvm_range_end(range);
+ int ret;
+
+ if (!dpagemap)
+ return -ENODEV;
+
+ ret = drm_pagemap_populate_mm(dpagemap, start, end,
+ svm->gpusvm.mm, 0);
+ if (ret) {
+ AMDGPU_SVM_TRACE("migrate_to_vram failed: ret=%d [0x%lx-0x%lx]\n",
+ ret, start, end);
+ return ret;
+ }
+
+ return 0;
+}
+
+bool
+amdgpu_pagemap_capable(struct amdgpu_svm *svm)
+{
+ if (svm->adev->gmc.is_app_apu)
+ return false;
+
+ if (!amdgpu_svm_get_dpagemap(svm))
+ return false;
+
+ return true;
+}
+
+/**
+ * amdgpu_svm_range_migrate_range - Per-range migration
+ * @svm: Pointer to the AMDGPU SVM structure
+ * @range: The GPU SVM range to migrate
+ * @migrate_mode: Migration intent
+ *
+ * NOTE: Only AMDGPU_SVM_MIGRATE_TO_VRAM and
+ * AMDGPU_SVM_MIGRATE_TO_SYSMEM are valid inputs. Passing any other
+ * migrate mode will trigger a WARN_ON() and the function will return
+ * -EINVAL. The caller should resolve PREFERRED/OTHER higher-level
+ * intents into one of the two concrete directions before calling this
+ * helper.
+ */
+int
+amdgpu_svm_range_migrate_range(struct amdgpu_svm *svm,
+ struct drm_gpusvm_range *range,
+ enum amdgpu_svm_migrate_mode migrate_mode)
+{
+ if (WARN_ON(migrate_mode != AMDGPU_SVM_MIGRATE_TO_VRAM &&
+ migrate_mode != AMDGPU_SVM_MIGRATE_TO_SYSMEM))
+ return -EINVAL;
+
+ if (range_needs_migrate_to_vram(range, migrate_mode))
+ return amdgpu_svm_range_migrate_to_vram(svm, range);
+ else if (migrate_mode == AMDGPU_SVM_MIGRATE_TO_SYSMEM &&
+ range_in_vram(range))
+ return drm_gpusvm_range_evict(&svm->gpusvm, range);
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h
new file mode 100644
index 000000000000..eba0b5f90e6e
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright 2026 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __AMDGPU_SVM_RANGE_MIGRATE_H__
+#define __AMDGPU_SVM_RANGE_MIGRATE_H__
+
+struct amdgpu_svm;
+struct drm_gpusvm_range;
+
+/**
+ * enum amdgpu_svm_migrate_mode - Migration mode for SVM range migration
+ *
+ * @AMDGPU_SVM_MIGRATE_NONE: No migration requested. Callers use it to
+ * skip the migration call and allow early-out for already-mapped ranges.
+ *
+ * @AMDGPU_SVM_MIGRATE_TO_VRAM: Migrate pages to VRAM before mapping.
+ *
+ * @AMDGPU_SVM_MIGRATE_TO_SYSMEM: Evict VRAM pages back to system memory.
+ *
+ * @AMDGPU_SVM_MIGRATE_PREFERRED: Follow preferred_loc to decide migration.
+ * Callers resolve this to TO_VRAM or TO_SYSMEM before calling
+ * amdgpu_svm_range_migrate_range().
+ *
+ * Only TO_VRAM and TO_SYSMEM are valid inputs to
+ * amdgpu_svm_range_migrate_range().
+ */
+enum amdgpu_svm_migrate_mode {
+ AMDGPU_SVM_MIGRATE_NONE,
+ AMDGPU_SVM_MIGRATE_TO_VRAM,
+ AMDGPU_SVM_MIGRATE_TO_SYSMEM,
+ AMDGPU_SVM_MIGRATE_PREFERRED
+};
+
+bool amdgpu_pagemap_capable(struct amdgpu_svm *svm);
+int amdgpu_svm_range_migrate_range(struct amdgpu_svm *svm,
+ struct drm_gpusvm_range *range,
+ enum amdgpu_svm_migrate_mode migrate_mode);
+
+#endif /* __AMDGPU_SVM_RANGE_MIGRATE_H__ */
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3 4/5] drm/amdgpu: add SVM attr prefetch/force-trigger functionality
2026-04-27 10:05 [PATCH v3 0/5] drm/amdgpu: SVM VRAM migration via drm_pagemap Junhua Shen
` (2 preceding siblings ...)
2026-04-27 10:05 ` [PATCH v3 3/5] drm/amdgpu: introduce SVM range migration decision layer Junhua Shen
@ 2026-04-27 10:05 ` Junhua Shen
2026-04-28 4:43 ` Claude review: " Claude Code Review Bot
2026-04-27 10:05 ` [PATCH v3 5/5] drm/amdgpu: integrate VRAM migration into SVM range map path Junhua Shen
2026-04-28 4:43 ` Claude review: drm/amdgpu: SVM VRAM migration via drm_pagemap Claude Code Review Bot
5 siblings, 1 reply; 13+ messages in thread
From: Junhua Shen @ 2026-04-27 10:05 UTC (permalink / raw)
To: Alexander.Deucher, Felix.Kuehling, Christian.Koenig, Oak.Zeng,
Jenny-Jing.Liu, Philip.Yang, Xiaogang.Chen, Ray.Huang,
honglei1.huang, Lingshan.Zhu
Cc: amd-gfx, dri-devel, Junhua Shen
Add attr_has_prefetch_loc() helper to detect prefetch-loc attributes
and use it to force-trigger migration even when the stored value has
not changed (prefetch is a one-shot command).
Refine attr_change_ctx_trigger() to only fire LOCATION_CHANGE on
prefetch_loc changes, not preferred_loc changes.
Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c | 34 +++++++++++++++-----
1 file changed, 26 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c
index cd972026f39b..52e402c18b0e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c
@@ -159,8 +159,7 @@ attr_change_ctx_trigger(const struct amdgpu_svm_attrs *prev_attrs,
trigger |= AMDGPU_SVM_ATTR_TRIGGER_PTE_FLAG_CHANGE;
if (changed_flags & AMDGPU_SVM_MAPPING_FLAG_MASK)
trigger |= AMDGPU_SVM_ATTR_TRIGGER_MAPPING_FLAG_CHANGE;
- if (prev_attrs->preferred_loc != new_attrs->preferred_loc ||
- prev_attrs->prefetch_loc != new_attrs->prefetch_loc)
+ if (prev_attrs->prefetch_loc != new_attrs->prefetch_loc)
trigger |= AMDGPU_SVM_ATTR_TRIGGER_LOCATION_CHANGE;
if (prev_attrs->granularity != new_attrs->granularity)
trigger |= AMDGPU_SVM_ATTR_TRIGGER_GRANULARITY_CHANGE;
@@ -187,9 +186,22 @@ static bool attr_has_access(uint32_t nattr,
return false;
}
+static bool attr_has_prefetch_loc(uint32_t nattr,
+ const struct drm_amdgpu_svm_attribute *attrs)
+{
+ uint32_t i;
+
+ for (i = 0; i < nattr; i++) {
+ if (attrs[i].type == AMDGPU_SVM_ATTR_PREFETCH_LOC)
+ return true;
+ }
+
+ return false;
+}
+
static struct amdgpu_svm_attr_range *
attr_alloc_range(unsigned long start,
- unsigned long last,
+ unsigned long last,
const struct amdgpu_svm_attrs *attrs)
{
struct amdgpu_svm_attr_range *range;
@@ -477,23 +489,29 @@ amdgpu_svm_attr_set_existing(struct amdgpu_svm_attr_tree *attr_tree,
struct amdgpu_svm_attrs old_attrs;
struct amdgpu_svm_attrs new_attrs;
uint32_t trigger;
- bool force_trigger;
+ uint32_t force_trigger = 0;
lockdep_assert_held(&attr_tree->lock);
old_attrs = range->attrs;
- /* The attr layer doesn't store the gpu mapped state, and for align with KFD,
- * need force trigger range layer to check if gpu mapped.
+ /*
+ * Force the range layer to act even when the attr value is
+ * unchanged: ACCESS needs re-check on xnack-off because the
+ * attr layer has no gpu_mapped state; PREFETCH_LOC is one-shot
+ * so repeated prefetch must always trigger migration.
*/
- force_trigger = !attr_tree->svm->xnack_enabled && attr_has_access(nattr, attrs);
+ if (!attr_tree->svm->xnack_enabled && attr_has_access(nattr, attrs))
+ force_trigger |= AMDGPU_SVM_ATTR_TRIGGER_ACCESS_CHANGE;
+ if (attr_has_prefetch_loc(nattr, attrs))
+ force_trigger |= AMDGPU_SVM_ATTR_TRIGGER_LOCATION_CHANGE;
if (attr_same_attrs(range, nattr, attrs)) {
if (!force_trigger)
return 0;
amdgpu_svm_attr_change_ctx_set(change, start, last,
- AMDGPU_SVM_ATTR_TRIGGER_ACCESS_CHANGE,
+ force_trigger,
&old_attrs, &old_attrs);
return 0;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3 5/5] drm/amdgpu: integrate VRAM migration into SVM range map path
2026-04-27 10:05 [PATCH v3 0/5] drm/amdgpu: SVM VRAM migration via drm_pagemap Junhua Shen
` (3 preceding siblings ...)
2026-04-27 10:05 ` [PATCH v3 4/5] drm/amdgpu: add SVM attr prefetch/force-trigger functionality Junhua Shen
@ 2026-04-27 10:05 ` Junhua Shen
2026-04-28 4:43 ` Claude review: " Claude Code Review Bot
2026-04-28 4:43 ` Claude review: drm/amdgpu: SVM VRAM migration via drm_pagemap Claude Code Review Bot
5 siblings, 1 reply; 13+ messages in thread
From: Junhua Shen @ 2026-04-27 10:05 UTC (permalink / raw)
To: Alexander.Deucher, Felix.Kuehling, Christian.Koenig, Oak.Zeng,
Jenny-Jing.Liu, Philip.Yang, Xiaogang.Chen, Ray.Huang,
honglei1.huang, Lingshan.Zhu
Cc: amd-gfx, dri-devel, Junhua Shen
hook up ZONE_DEVICE registration in device init and reset.
Wire the migration decision layer into the SVM range map call chain:
- Add migrate_mode parameter to amdgpu_svm_range_map_attr_ranges()
and propagate it through map_interval -> per-range map loop
- Call amdgpu_svm_range_migrate_range() on each drm_gpusvm_range
before GPU mapping to perform VRAM migration or eviction
- Pass appropriate migrate_mode from SVM attr prefetch triggers
and the restore worker
Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +
drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c | 4 +
drivers/gpu/drm/amd/amdgpu/amdgpu_svm.c | 4 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c | 136 +++++++++---------
drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.h | 5 +-
5 files changed, 87 insertions(+), 66 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index fbe553c38583..3be51a2c0106 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -78,6 +78,7 @@
#include "amdgpu_reset.h"
#include "amdgpu_virt.h"
#include "amdgpu_dev_coredump.h"
+#include "amdgpu_migrate.h"
#include <linux/suspend.h>
#include <drm/task_barrier.h>
@@ -4076,6 +4077,9 @@ int amdgpu_device_init(struct amdgpu_device *adev,
/* Don't init kfd if whole hive need to be reset during init */
if (adev->init_lvl->level != AMDGPU_INIT_LEVEL_MINIMAL_XGMI) {
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
+ amdgpu_svm_migration_init(adev);
+#endif
kgd2kfd_init_zone_device(adev);
kfd_update_svm_support_properties(adev);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
index 28c4ad62f50e..c94d43f3ab42 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
@@ -25,6 +25,7 @@
#include "aldebaran.h"
#include "sienna_cichlid.h"
#include "smu_v13_0_10.h"
+#include "amdgpu_migrate.h"
static int amdgpu_reset_xgmi_reset_on_init_suspend(struct amdgpu_device *adev)
{
@@ -87,6 +88,9 @@ static int amdgpu_reset_xgmi_reset_on_init_restore_hwctxt(
return r;
list_for_each_entry(tmp_adev, reset_device_list, reset_list) {
if (!tmp_adev->kfd.init_complete) {
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
+ amdgpu_svm_migration_init(tmp_adev);
+#endif
kgd2kfd_init_zone_device(tmp_adev);
amdgpu_amdkfd_device_init(tmp_adev);
amdgpu_amdkfd_drm_client_create(tmp_adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm.c
index 57103a140164..cd8dbe56a9e9 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm.c
@@ -32,6 +32,7 @@
#include "amdgpu_svm.h"
#include "amdgpu_svm_attr.h"
#include "amdgpu_svm_range.h"
+#include "amdgpu_svm_range_migrate.h"
#include "amdgpu_vm.h"
#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
@@ -341,7 +342,8 @@ int amdgpu_svm_handle_fault(struct amdgpu_device *adev, uint32_t pasid,
AMDGPU_SVM_TRACE("handle_fault: map_attr page=0x%lx\n", fault_page);
down_write(&svm->svm_lock);
- ret = amdgpu_svm_range_map_attr_ranges(svm, fault_page, fault_page);
+ ret = amdgpu_svm_range_map_attr_ranges(svm, fault_page, fault_page,
+ AMDGPU_SVM_MIGRATE_PREFERRED);
up_write(&svm->svm_lock);
if (ret)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c
index 472a641fb836..8880e2ba79cc 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c
@@ -25,6 +25,7 @@
#include "amdgpu_svm.h"
#include "amdgpu_svm_attr.h"
#include "amdgpu_svm_range.h"
+#include "amdgpu_migrate.h"
#include "amdgpu.h"
#include "amdgpu_amdkfd.h"
#include "amdgpu_vm.h"
@@ -114,7 +115,6 @@ range_pages_valid(struct amdgpu_svm *svm,
return drm_gpusvm_range_pages_valid(&svm->gpusvm, range);
}
-
static int
amdgpu_svm_range_gpu_unmap_in_notifier(struct amdgpu_svm *svm,
struct drm_gpusvm_range *range,
@@ -247,11 +247,11 @@ amdgpu_svm_range_attr_pte_flags(struct amdgpu_svm *svm,
return pte_flags;
}
- /*
- * POC/WA: reuse kfd apis for queue quiesce/resume
- * But kfd apis are for process level, not for GPU VM level
- * need consider potential issues
- */
+/*
+ * POC/WA: reuse kfd apis for queue quiesce/resume
+ * But kfd apis are for process level, not for GPU VM level
+ * need consider potential issues
+ */
void amdgpu_svm_range_restore_begin_compute(struct amdgpu_svm *svm)
{
int ret;
@@ -317,39 +317,6 @@ static int amdgpu_svm_range_lock_vm_pd(struct amdgpu_svm *svm, struct drm_exec *
return 0;
}
-static int
-amdgpu_svm_range_update_gpu(struct amdgpu_svm *svm, unsigned long start_page,
- unsigned long last_page, uint64_t pte_flags,
- dma_addr_t *pages_addr, bool flush_tlb,
- bool update_pdes, bool wait_fence)
-{
- struct drm_exec exec;
- struct dma_fence *fence = NULL;
- int ret;
-
- ret = amdgpu_svm_range_lock_vm_pd(svm, &exec);
- if (ret)
- return ret;
-
- ret = amdgpu_vm_update_range(svm->adev, svm->vm, false, false,
- flush_tlb, true,
- NULL, start_page, last_page, pte_flags, 0, 0,
- NULL, pages_addr, wait_fence ? &fence : NULL);
- if (!ret && wait_fence && fence) {
- ret = dma_fence_wait(fence, false);
- if (ret < 0)
- AMDGPU_SVM_TRACE("wait unmap fence failed: ret=%d [0x%lx-0x%lx]-0x%lx\n",
- ret, start_page, last_page,
- last_page - start_page + 1);
- }
- if (!ret && update_pdes)
- ret = amdgpu_vm_update_pdes(svm->adev, svm->vm, false);
-
- dma_fence_put(fence);
- drm_exec_fini(&exec);
- return ret;
-}
-
static int
amdgpu_svm_range_update_gpu_range(struct amdgpu_svm *svm,
struct drm_gpusvm_range *range,
@@ -376,9 +343,11 @@ amdgpu_svm_range_update_gpu_range(struct amdgpu_svm *svm,
npages - mapped_pages);
dma_addr_t seg_addr = entry->addr;
unsigned long start_page, last_page;
+ uint64_t seg_pte_flags = pte_flags;
bool is_last_seg;
- if (entry->proto != DRM_INTERCONNECT_SYSTEM)
+ if (entry->proto != DRM_INTERCONNECT_SYSTEM &&
+ entry->proto != AMDGPU_INTERCONNECT_VRAM)
return -EOPNOTSUPP;
while (mapped_pages + seg_pages < npages) {
@@ -399,9 +368,13 @@ amdgpu_svm_range_update_gpu_range(struct amdgpu_svm *svm,
last_page = start_page + seg_pages - 1;
is_last_seg = mapped_pages + seg_pages == npages;
+ /* For VRAM pages, Clear the SYSTEM and SNOOPED bits */
+ if (entry->proto == AMDGPU_INTERCONNECT_VRAM)
+ seg_pte_flags &= ~(AMDGPU_PTE_SYSTEM | AMDGPU_PTE_SNOOPED);
+
ret = amdgpu_vm_update_range(svm->adev, svm->vm, false, false,
flush_tlb && is_last_seg, true, NULL,
- start_page, last_page, pte_flags,
+ start_page, last_page, seg_pte_flags,
0, seg_addr, NULL, NULL,
wait_fence && is_last_seg ? fence : NULL);
if (ret)
@@ -413,17 +386,35 @@ amdgpu_svm_range_update_gpu_range(struct amdgpu_svm *svm,
return 0;
}
+static bool
+preferred_loc_is_vram(const struct amdgpu_svm_attrs *attrs)
+{
+ return attrs->preferred_loc != AMDGPU_SVM_LOCATION_SYSMEM &&
+ attrs->preferred_loc != AMDGPU_SVM_LOCATION_UNDEFINED;
+}
+
static int
amdgpu_svm_range_map(struct amdgpu_svm *svm,
unsigned long start,
unsigned long end,
const struct amdgpu_svm_attrs *attrs,
const struct drm_gpusvm_ctx *gpusvm_ctx,
- uint64_t pte_flags)
+ uint64_t pte_flags,
+ enum amdgpu_svm_migrate_mode migrate_mode)
{
unsigned long addr = start;
int ret;
+ /*
+ * For preferred migration mode, determine the
+ * actual migration direction based on the preferred
+ * location attribute.
+ */
+ if (migrate_mode == AMDGPU_SVM_MIGRATE_PREFERRED)
+ migrate_mode = preferred_loc_is_vram(attrs) ?
+ AMDGPU_SVM_MIGRATE_TO_VRAM :
+ AMDGPU_SVM_MIGRATE_TO_SYSMEM;
+
while (addr < end) {
struct drm_exec exec;
struct drm_gpusvm_ctx map_ctx;
@@ -462,6 +453,10 @@ amdgpu_svm_range_map(struct amdgpu_svm *svm,
if (next_addr <= addr)
return -EINVAL;
+ /* Per-range migration */
+ if (migrate_mode != AMDGPU_SVM_MIGRATE_NONE)
+ amdgpu_svm_range_migrate_range(svm, range, migrate_mode);
+
range_pte_flags = map_ctx.read_only ?
(pte_flags & ~AMDGPU_PTE_WRITEABLE) : pte_flags;
@@ -529,10 +524,16 @@ amdgpu_svm_range_map(struct amdgpu_svm *svm,
static int
amdgpu_svm_range_map_interval(struct amdgpu_svm *svm, unsigned long start_page,
unsigned long last_page,
- const struct amdgpu_svm_attrs *attrs)
+ const struct amdgpu_svm_attrs *attrs,
+ enum amdgpu_svm_migrate_mode migrate_mode)
{
+ bool hw_devmem = amdgpu_pagemap_capable(svm);
+
struct drm_gpusvm_ctx gpusvm_ctx = {
.read_only = !!(attrs->flags & AMDGPU_SVM_FLAG_GPU_RO),
+ .devmem_possible = hw_devmem,
+ .device_private_page_owner = hw_devmem ?
+ AMDGPU_SVM_PGMAP_OWNER(svm->adev) : NULL,
};
unsigned long start = start_page << PAGE_SHIFT;
unsigned long end = (last_page + 1) << PAGE_SHIFT;
@@ -542,7 +543,7 @@ amdgpu_svm_range_map_interval(struct amdgpu_svm *svm, unsigned long start_page,
pte_flags = amdgpu_svm_range_attr_pte_flags(svm, attrs);
ret = amdgpu_svm_range_map(svm, start, end, attrs, &gpusvm_ctx,
- pte_flags);
+ pte_flags, migrate_mode);
if (ret)
AMDGPU_SVM_TRACE("map_interval failed: ret=%d [0x%lx-0x%lx)-0x%lx\n",
ret, start, end, end - start);
@@ -553,7 +554,8 @@ amdgpu_svm_range_map_interval(struct amdgpu_svm *svm, unsigned long start_page,
int
amdgpu_svm_range_map_attr_ranges(struct amdgpu_svm *svm,
unsigned long start_page,
- unsigned long last_page)
+ unsigned long last_page,
+ enum amdgpu_svm_migrate_mode migrate_mode)
{
lockdep_assert_held_write(&svm->svm_lock);
@@ -573,9 +575,10 @@ amdgpu_svm_range_map_attr_ranges(struct amdgpu_svm *svm,
seg_last = min(seg_last, last_page);
if (range_has_access(attrs.access)) {
- /* map may fail here cause no vma or access deny */
- ret = amdgpu_svm_range_map_interval(svm, cursor, seg_last,
- &attrs);
+ ret = amdgpu_svm_range_map_interval(svm, cursor,
+ seg_last,
+ &attrs,
+ migrate_mode);
if (ret)
return ret;
}
@@ -657,7 +660,6 @@ static int amdgpu_svm_range_rebuild_locked(struct amdgpu_svm *svm,
unsigned long rebuild_start = start_page;
unsigned long rebuild_last = last_page;
bool removed;
- int ret;
lockdep_assert_held_write(&svm->svm_lock);
@@ -673,14 +675,10 @@ static int amdgpu_svm_range_rebuild_locked(struct amdgpu_svm *svm,
/* scan rebuild start end to build the extra removed ranges */
if (rebuild)
return amdgpu_svm_range_map_attr_ranges(svm, rebuild_start,
- rebuild_last);
+ rebuild_last,
+ AMDGPU_SVM_MIGRATE_PREFERRED);
- ret = amdgpu_svm_range_update_gpu(svm, rebuild_start, rebuild_last,
- 0, NULL, true, true, true);
- if (!ret)
- svm->flush_tlb(svm);
-
- return ret;
+ return 0;
}
static void
@@ -706,10 +704,11 @@ amdgpu_svm_range_process_notifier_ranges(struct amdgpu_svm *svm,
if (clear_pte) {
amdgpu_svm_range_gpu_unmap_in_notifier(svm, range,
mmu_range);
- range_invalidate_gpu_mapping(range);
}
drm_gpusvm_range_unmap_pages(&svm->gpusvm, range, &ctx);
+ range_invalidate_gpu_mapping(range);
+
if (is_unmap)
drm_gpusvm_range_set_unmapped(range, mmu_range);
@@ -758,6 +757,7 @@ int amdgpu_svm_range_apply_attr_change(struct amdgpu_svm *svm,
{
lockdep_assert_held_write(&svm->svm_lock);
+ enum amdgpu_svm_migrate_mode migrate_mode = AMDGPU_SVM_MIGRATE_NONE;
bool old_access, new_access;
bool update_mapping = false;
@@ -788,16 +788,24 @@ int amdgpu_svm_range_apply_attr_change(struct amdgpu_svm *svm,
new_access)
update_mapping = true;
- if (trigger & AMDGPU_SVM_ATTR_TRIGGER_LOCATION_CHANGE) {
- /* TODO: add migration */
+ if ((trigger & AMDGPU_SVM_ATTR_TRIGGER_LOCATION_CHANGE) &&
+ new_access) {
+ int32_t loc = new_attrs->prefetch_loc;
+
+ if (loc == AMDGPU_SVM_LOCATION_SYSMEM) {
+ migrate_mode = AMDGPU_SVM_MIGRATE_TO_SYSMEM;
+ update_mapping = true;
+ } else if (loc != AMDGPU_SVM_LOCATION_UNDEFINED) {
+ migrate_mode = AMDGPU_SVM_MIGRATE_TO_VRAM;
+ update_mapping = true;
+ }
}
if (!update_mapping)
return 0;
- AMDGPU_SVM_TRACE("mapping update: remap interval [0x%lx-0x%lx]-0x%lx\n",
- start, last, last - start + 1);
- return amdgpu_svm_range_map_interval(svm, start, last, new_attrs);
+ return amdgpu_svm_range_map_interval(svm, start, last, new_attrs,
+ migrate_mode);
}
static bool
@@ -1004,7 +1012,8 @@ static void amdgpu_svm_range_restore_worker(struct work_struct *w)
down_write(&svm->svm_lock);
ret = amdgpu_svm_range_map_attr_ranges(svm, op_ctx.start,
- op_ctx.last);
+ op_ctx.last,
+ AMDGPU_SVM_MIGRATE_NONE);
up_write(&svm->svm_lock);
if (ret) {
@@ -1042,7 +1051,6 @@ static void amdgpu_svm_range_restore_worker(struct work_struct *w)
drm_gpusvm_notifier_unlock(&svm->gpusvm);
svm->end_restore(svm);
return;
-
}
drm_gpusvm_notifier_unlock(&svm->gpusvm);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.h
index 18bf3dad13fd..0065ae50c700 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.h
@@ -30,6 +30,8 @@
#include <linux/list.h>
#include <linux/types.h>
+#include "amdgpu_svm_range_migrate.h"
+
struct amdgpu_svm;
struct amdgpu_svm_attrs;
struct drm_gpusvm_notifier;
@@ -62,7 +64,8 @@ void amdgpu_svm_range_flush(struct amdgpu_svm *svm);
void amdgpu_svm_range_sync_work(struct amdgpu_svm *svm);
int amdgpu_svm_range_map_attr_ranges(struct amdgpu_svm *svm,
unsigned long start_page,
- unsigned long last_page);
+ unsigned long last_page,
+ enum amdgpu_svm_migrate_mode migrate_mode);
int amdgpu_svm_range_apply_attr_change(
struct amdgpu_svm *svm, unsigned long start, unsigned long last,
uint32_t trigger, const struct amdgpu_svm_attrs *prev_attrs,
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH v3 2/5] drm/amdgpu: implement drm_pagemap SDMA migration callbacks
2026-04-27 10:05 ` [PATCH v3 2/5] drm/amdgpu: implement drm_pagemap SDMA migration callbacks Junhua Shen
@ 2026-04-27 22:20 ` Felix Kuehling
2026-04-28 4:43 ` Claude review: " Claude Code Review Bot
1 sibling, 0 replies; 13+ messages in thread
From: Felix Kuehling @ 2026-04-27 22:20 UTC (permalink / raw)
To: Junhua Shen, Alexander.Deucher, Christian.Koenig, Oak.Zeng,
Jenny-Jing.Liu, Philip.Yang, Xiaogang.Chen, Ray.Huang,
honglei1.huang, Lingshan.Zhu
Cc: amd-gfx, dri-devel
On 2026-04-27 06:05, Junhua Shen wrote:
> Implement the drm_pagemap_devmem_ops and drm_pagemap_ops callbacks
> that the DRM GPUSVM migration framework requires:
>
> drm_pagemap_ops (top-level entry points):
> - device_map: convert ZONE_DEVICE page to GPU PTE address
> - populate_mm: allocate VRAM BO and trigger migration
>
> drm_pagemap_devmem_ops (per-BO migration mechanics):
> - populate_devmem_pfn: walk BO buddy blocks to build PFN array
> - copy_to_devmem: SDMA copy system RAM -> VRAM via GART window
> - copy_to_ram: SDMA copy VRAM -> system RAM via GART window
> - devmem_release: free BO when all pages migrate back
>
> Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c | 616 +++++++++++++++++++-
> 1 file changed, 613 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
> index 170e2eadc106..42092651b4d5 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
> @@ -64,12 +64,20 @@
> #include <linux/memremap.h>
> #include <linux/migrate.h>
>
> -#include "amdgpu_amdkfd.h"
> #include "amdgpu_migrate.h"
> #include "amdgpu.h"
> +#include "amdgpu_ttm.h"
> +#include "amdgpu_res_cursor.h"
> +
> +#define AMDGPU_MIGRATE_TRACE(fmt, ...) \
> + pr_debug("%s: " fmt, __func__, ##__VA_ARGS__)
> +
> +/* SDMA copy direction */
> +#define FROM_RAM_TO_VRAM 0
> +#define FROM_VRAM_TO_RAM 1
>
> static inline struct amdgpu_pagemap *
> -dpagemap_to_apagemap(struct drm_pagemap *dpagemap)
> +to_amdgpu_pagemap(struct drm_pagemap *dpagemap)
> {
> return container_of(dpagemap, struct amdgpu_pagemap, dpagemap);
> }
> @@ -94,8 +102,610 @@ amdgpu_svm_page_to_apagemap(struct page *page)
> return container_of(pgmap, struct amdgpu_pagemap, pgmap);
> }
>
> +/* drm_pagemap_devmem_ops — per-BO migration mechanics */
> +
> +/**
> + * struct amdgpu_svm_bo - Wrapper linking drm_pagemap_devmem to amdgpu_bo
> + *
> + * @devmem: drm_pagemap device memory allocation (passed to framework)
> + * @bo: The backing VRAM amdgpu_bo
> + *
> + * It is allocated per-migration in populate_mm() and freed by
> + * devmem_release() when all device-private pages have migrated
> + * back to system memory.
> + *
> + * Lifecycle is managed by the drm_pagemap framework's internal zdd refcount:
> + * - zdd->devmem_allocation points to &svm_bo->devmem
> + * - When zdd refcount drops to zero, framework calls devmem_release()
> + * - devmem_release() frees both the BO reference and the svm_bo itself
> + */
> +struct amdgpu_svm_bo {
> + struct amdgpu_bo *bo;
> + struct drm_pagemap_devmem devmem;
> +};
> +
> +static inline struct amdgpu_svm_bo *
> +to_amdgpu_svm_bo(struct drm_pagemap_devmem *devmem_allocation)
> +{
> + return container_of(devmem_allocation, struct amdgpu_svm_bo, devmem);
> +}
> +
> +/**
> + * amdgpu_svm_devmem_release - Release BO when all device pages migrate back
> + *
> + * Called by the drm_pagemap framework (via drm_pagemap_zdd_destroy) when the
> + * last device-private page backed by this allocation has been migrated back
> + * to system memory (or the owning process exits).
> + *
> + * Frees both the amdgpu_bo reference and the wrapper amdgpu_svm_bo itself.
> + */
> +static void
> +amdgpu_svm_devmem_release(struct drm_pagemap_devmem *devmem_allocation)
> +{
> + struct amdgpu_svm_bo *svm_bo = to_amdgpu_svm_bo(devmem_allocation);
> +
> + AMDGPU_MIGRATE_TRACE("Release svm_bo=%px bo=%px\n", svm_bo, svm_bo->bo);
> + amdgpu_bo_unref(&svm_bo->bo);
> + kfree(svm_bo);
> +}
> +
> +/**
> + * amdgpu_svm_populate_devmem_pfn - Convert BO VRAM allocation to PFN array
> + * @devmem_allocation: The devmem allocation in the amdgpu_svm_bo wrapper
> + * @npages: Number of PFN entries to fill
> + * @pfn: Output PFN array
> + *
> + * Iterates over the BO's TTM vram_mgr buddy blocks and converts each
> + * block's VRAM offset to ZONE_DEVICE PFNs:
> + *
> + * PFN = PHYS_PFN(block_offset + apagemap.hpa_base) + page_index
> + *
> + * This is called by drm_pagemap_migrate_to_devmem() to build the
> + * destination PFN array for migrate_vma_pages().
> + *
> + * Return: 0 on success
> + */
> +static int
> +amdgpu_svm_populate_devmem_pfn(struct drm_pagemap_devmem *devmem_allocation,
> + unsigned long npages, unsigned long *pfn)
> +{
> + struct amdgpu_pagemap *svm_dm = to_amdgpu_pagemap(devmem_allocation->dpagemap);
> + struct amdgpu_svm_bo *svm_bo = to_amdgpu_svm_bo(devmem_allocation);
> + struct amdgpu_bo *bo = svm_bo->bo;
> + struct amdgpu_res_cursor cursor;
> + unsigned long i = 0;
> + int ret;
> +
> + ret = amdgpu_bo_reserve(bo, false);
> + if (ret)
> + return ret;
> +
> + amdgpu_res_first(bo->tbo.resource, 0, npages << PAGE_SHIFT, &cursor);
How do you ensure that the BO is valid and actually in VRAM at the time?
And how do you ensure that it stays there as long as the zone_device
pages are in use? As far as I can tell, the BO is not pinned and there
is no fence that prevents it from being evicted to GTT by TTM without
warning.
Regards,
Felix
> + while (cursor.remaining && i < npages) {
> + u64 pfn_base = PHYS_PFN(cursor.start + svm_dm->hpa_base);
> + u64 pages = cursor.size >> PAGE_SHIFT;
> + unsigned long j;
> +
> + for (j = 0; j < pages && i < npages; j++, i++)
> + pfn[i] = pfn_base + j;
> +
> + amdgpu_res_next(&cursor, cursor.size);
> + }
> +
> + amdgpu_bo_unreserve(bo);
> +
> + AMDGPU_MIGRATE_TRACE("populate_devmem_pfn: npages=%lu first_pfn=0x%lx\n",
> + npages, npages > 0 ? pfn[0] : 0);
> +
> + return 0;
> +}
> +
> +/* SDMA copy helpers — GART window based data transfer */
> +
> +/**
> + * amdgpu_svm_direct_mapping_addr - Convert VRAM offset to MC address
> + * @adev: AMDGPU device
> + * @vram_offset: Byte offset within VRAM
> + *
> + * Return: MC address suitable for SDMA src/dst
> + */
> +static u64
> +amdgpu_svm_direct_mapping_addr(struct amdgpu_device *adev, u64 vram_offset)
> +{
> + return vram_offset + amdgpu_ttm_domain_start(adev, TTM_PL_VRAM);
> +}
> +
> +/**
> + * amdgpu_svm_gart_map - Map system DMA addresses into GART window
> + * @ring: SDMA ring for the GART update job
> + * @npages: Number of pages to map
> + * @addr: Array of system memory DMA addresses
> + * @gart_addr: Output — GART base address to use in SDMA copy
> + * @flags: PTE flags (e.g. writeable for RAM-to-VRAM src)
> + *
> + * Builds GART PTEs pointing at the given DMA addresses, submits an
> + * SDMA job to update the GART entries, and returns the GART address
> + * that can be used as src or dst in a subsequent amdgpu_copy_buffer().
> + *
> + * Uses GART window 0, protected by gtt_window_lock.
> + *
> + * Return: 0 on success, negative error code on failure
> + */
> +static int
> +amdgpu_svm_gart_map(struct amdgpu_ring *ring,
> + struct amdgpu_ttm_buffer_entity *entity,
> + u64 npages,
> + dma_addr_t *addr, u64 *gart_addr, u64 flags)
> +{
> + struct amdgpu_device *adev = ring->adev;
> + struct amdgpu_job *job;
> + unsigned int num_dw, num_bytes;
> + struct dma_fence *fence;
> + u64 src_addr, dst_addr;
> + u64 pte_flags;
> + void *cpu_addr;
> + int r;
> +
> + /* Use entity's GART window 0 */
> + *gart_addr = amdgpu_compute_gart_address(&adev->gmc, entity, 0);
> +
> + num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8);
> + num_bytes = npages * 8 * AMDGPU_GPU_PAGES_IN_CPU_PAGE;
> +
> + r = amdgpu_job_alloc_with_ib(adev, &entity->base,
> + AMDGPU_FENCE_OWNER_UNDEFINED,
> + num_dw * 4 + num_bytes,
> + AMDGPU_IB_POOL_DELAYED,
> + &job,
> + AMDGPU_KERNEL_JOB_ID_KFD_GART_MAP);
> + if (r)
> + return r;
> +
> + src_addr = num_dw * 4;
> + src_addr += job->ibs[0].gpu_addr;
> +
> + dst_addr = amdgpu_bo_gpu_offset(adev->gart.bo);
> + dst_addr += (entity->gart_window_offs[0] >> AMDGPU_GPU_PAGE_SHIFT) * 8;
> + amdgpu_emit_copy_buffer(adev, &job->ibs[0], src_addr,
> + dst_addr, num_bytes, 0);
> +
> + amdgpu_ring_pad_ib(ring, &job->ibs[0]);
> + WARN_ON(job->ibs[0].length_dw > num_dw);
> +
> + pte_flags = AMDGPU_PTE_VALID | AMDGPU_PTE_READABLE;
> + pte_flags |= AMDGPU_PTE_SYSTEM | AMDGPU_PTE_SNOOPED;
> + if (flags & AMDGPU_PTE_WRITEABLE)
> + pte_flags |= AMDGPU_PTE_WRITEABLE;
> + pte_flags |= adev->gart.gart_pte_flags;
> +
> + cpu_addr = &job->ibs[0].ptr[num_dw];
> +
> + amdgpu_gart_map(adev, 0, npages, addr, pte_flags, cpu_addr);
> + fence = amdgpu_job_submit(job);
> + dma_fence_put(fence);
> +
> + return 0;
> +}
> +
> +/**
> + * amdgpu_svm_copy_memory_gart - SDMA copy between system RAM and VRAM
> + * @adev: AMDGPU device
> + * @sys: Array of DMA addresses for system memory pages
> + * @vram: Array of VRAM byte offsets (relative to start of VRAM)
> + * @npages: Number of pages to copy
> + * @direction: FROM_RAM_TO_VRAM or FROM_VRAM_TO_RAM
> + * @mfence: In/out — carries the last SDMA fence for serialization
> + *
> + * Maps system memory pages into the GART window and uses SDMA to copy
> + * data to/from VRAM. Handles splitting into AMDGPU_GTT_MAX_TRANSFER_SIZE
> + * chunks. Acquires entity->lock internally to protect the GART window,
> + * matching the KFD svm_migrate_copy_memory_gart() pattern.
> + *
> + * Return: 0 on success, negative error code on failure
> + */
> +static int
> +amdgpu_svm_copy_memory_gart(struct amdgpu_device *adev, dma_addr_t *sys,
> + u64 *vram, u64 npages, int direction,
> + struct dma_fence **mfence)
> +{
> + const u64 max_pages = AMDGPU_GTT_MAX_TRANSFER_SIZE;
> + struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;
> + struct amdgpu_ttm_buffer_entity *entity = &adev->mman.move_entity;
> + u64 gart_s, gart_d;
> + struct dma_fence *next;
> + u64 size;
> + int r;
> +
> + mutex_lock(&entity->lock);
> +
> + while (npages) {
> + size = min(max_pages, npages);
> +
> + if (direction == FROM_VRAM_TO_RAM) {
> + gart_s = amdgpu_svm_direct_mapping_addr(adev, *vram);
> + r = amdgpu_svm_gart_map(ring, entity, size, sys,
> + &gart_d, AMDGPU_PTE_WRITEABLE);
> + } else {
> + r = amdgpu_svm_gart_map(ring, entity, size, sys,
> + &gart_s, 0);
> + gart_d = amdgpu_svm_direct_mapping_addr(adev, *vram);
> + }
> + if (r) {
> + dev_err(adev->dev, "failed %d to map GART for SDMA\n", r);
> + goto out_unlock;
> + }
> +
> + AMDGPU_MIGRATE_TRACE("SDMA_COPY: %s npages=%llu vram_off=0x%llx\n",
> + direction == FROM_RAM_TO_VRAM ? "RAM->VRAM" : "VRAM->RAM",
> + size, (u64)*vram);
> +
> + r = amdgpu_copy_buffer(adev, entity, gart_s, gart_d,
> + size * PAGE_SIZE,
> + NULL, &next, true, 0);
> + if (r) {
> + dev_err(adev->dev, "failed %d to copy buffer\n", r);
> + goto out_unlock;
> + }
> +
> + dma_fence_put(*mfence);
> + *mfence = next;
> + npages -= size;
> + if (npages) {
> + sys += size;
> + vram += size;
> + }
> + }
> +
> +out_unlock:
> + mutex_unlock(&entity->lock);
> +
> + return r;
> +}
> +
> +/**
> + * amdgpu_svm_copy_to_devmem - SDMA copy system memory -> VRAM
> + * @pages: Array of destination ZONE_DEVICE pages (VRAM-backed)
> + * @pagemap_addr: Array of source DMA addresses (system memory, already mapped)
> + * @npages: Number of pages to copy
> + *
> + * Builds parallel sys[] and vram[] arrays from the framework-provided
> + * pagemap_addr and device pages, then submits batched SDMA copies via
> + * the GART window.
> + *
> + * Return: 0 on success, negative error code on failure
> + */
> +static int
> +amdgpu_svm_copy_to_devmem(struct page **pages,
> + struct drm_pagemap_addr *pagemap_addr,
> + unsigned long npages,
> + struct dma_fence *pre_migrate_fence)
> +{
> + struct amdgpu_device *adev;
> + struct amdgpu_pagemap *svm_dm;
> + struct dma_fence *mfence = NULL;
> + dma_addr_t *sys;
> + u64 *vram;
> + unsigned long i, j;
> + int ret = 0;
> +
> + if (!npages)
> + return 0;
> +
> + /*
> + * Find the first non-NULL page to derive the device.
> + * The pages array may contain NULL entries for positions where
> + * no valid device page exists.
> + */
> + for (i = 0; i < npages; i++) {
> + if (pages[i])
> + break;
> + }
> + if (i == npages)
> + return 0;
> +
> + svm_dm = amdgpu_svm_page_to_apagemap(pages[i]);
> + adev = svm_dm->adev;
> +
> + sys = kvcalloc(npages, sizeof(*sys), GFP_KERNEL);
> + vram = kvcalloc(npages, sizeof(*vram), GFP_KERNEL);
> + if (!sys || !vram) {
> + ret = -ENOMEM;
> + goto out_free;
> + }
> +
> + for (i = 0, j = 0; i < npages; i++) {
> + if (!pagemap_addr[i].addr)
> + goto flush;
> +
> + sys[j] = pagemap_addr[i].addr;
> + vram[j] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
> + svm_dm->hpa_base;
> +
> + /* Check if next vram page is contiguous with current */
> + if (j > 0 && vram[j] != vram[j - 1] + PAGE_SIZE)
> + goto flush;
> +
> + j++;
> + continue;
> +flush:
> + if (j) {
> + ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
> + FROM_RAM_TO_VRAM,
> + &mfence);
> + if (ret)
> + goto out_fence;
> + j = 0;
> + }
> + /* Re-process current page if it was valid but broke contiguity */
> + if (pagemap_addr[i].addr) {
> + sys[0] = pagemap_addr[i].addr;
> + vram[0] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
> + svm_dm->hpa_base;
> + j = 1;
> + }
> + }
> +
> + /* Flush remaining batch */
> + if (j)
> + ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
> + FROM_RAM_TO_VRAM, &mfence);
> +
> +out_fence:
> + if (mfence) {
> + dma_fence_wait(mfence, false);
> + dma_fence_put(mfence);
> + }
> +
> + AMDGPU_MIGRATE_TRACE("copy_to_devmem done: npages=%ld ret=%d\n",
> + npages, ret);
> +
> +out_free:
> + kvfree(vram);
> + kvfree(sys);
> + return ret;
> +}
> +
> +/**
> + * amdgpu_svm_copy_to_ram - SDMA copy VRAM -> system memory
> + * @pages: Array of source ZONE_DEVICE pages (VRAM-backed)
> + * @pagemap_addr: Array of destination DMA addresses (system memory, already mapped)
> + * @npages: Number of pages to copy
> + *
> + * Mirror of copy_to_devmem with src/dst swapped.
> + *
> + * Return: 0 on success, negative error code on failure
> + */
> +static int
> +amdgpu_svm_copy_to_ram(struct page **pages,
> + struct drm_pagemap_addr *pagemap_addr,
> + unsigned long npages,
> + struct dma_fence *pre_migrate_fence)
> +{
> + struct amdgpu_device *adev;
> + struct amdgpu_pagemap *svm_dm;
> + struct dma_fence *mfence = NULL;
> + dma_addr_t *sys;
> + u64 *vram;
> + unsigned long i, j;
> + int ret = 0;
> +
> + if (!npages)
> + return 0;
> +
> + for (i = 0; i < npages; i++) {
> + if (pages[i])
> + break;
> + }
> + if (i == npages)
> + return 0;
> +
> + svm_dm = amdgpu_svm_page_to_apagemap(pages[i]);
> + adev = svm_dm->adev;
> +
> + sys = kvcalloc(npages, sizeof(*sys), GFP_KERNEL);
> + vram = kvcalloc(npages, sizeof(*vram), GFP_KERNEL);
> + if (!sys || !vram) {
> + ret = -ENOMEM;
> + goto out_free;
> + }
> +
> + for (i = 0, j = 0; i < npages; i++) {
> + if (!pagemap_addr[i].addr || !pages[i])
> + goto flush;
> +
> + vram[j] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
> + svm_dm->hpa_base;
> + sys[j] = pagemap_addr[i].addr;
> +
> + /* Check if next vram page is contiguous with current */
> + if (j > 0 && vram[j] != vram[j - 1] + PAGE_SIZE)
> + goto flush;
> +
> + j++;
> + continue;
> +flush:
> + if (j) {
> + ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
> + FROM_VRAM_TO_RAM,
> + &mfence);
> + if (ret)
> + goto out_fence;
> + j = 0;
> + }
> + /* Re-process current page if it was valid but broke contiguity */
> + if (pagemap_addr[i].addr && pages[i]) {
> + vram[0] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
> + svm_dm->hpa_base;
> + sys[0] = pagemap_addr[i].addr;
> + j = 1;
> + }
> + }
> +
> + /* Flush remaining batch */
> + if (j)
> + ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
> + FROM_VRAM_TO_RAM, &mfence);
> +
> +out_fence:
> + if (mfence) {
> + dma_fence_wait(mfence, false);
> + dma_fence_put(mfence);
> + }
> +
> + AMDGPU_MIGRATE_TRACE("copy_to_ram done: npages=%ld ret=%d\n", npages, ret);
> +
> +out_free:
> + kvfree(vram);
> + kvfree(sys);
> + return ret;
> +}
> +
> +static const struct drm_pagemap_devmem_ops amdgpu_pagemap_ops = {
> + .devmem_release = amdgpu_svm_devmem_release,
> + .populate_devmem_pfn = amdgpu_svm_populate_devmem_pfn,
> + .copy_to_devmem = amdgpu_svm_copy_to_devmem,
> + .copy_to_ram = amdgpu_svm_copy_to_ram,
> +};
> +
> +/* drm_pagemap_ops — top-level migration entry points */
> +
> +/**
> + * amdgpu_svm_device_map - Convert ZONE_DEVICE page to GPU PTE address
> + * @dpagemap: The drm_pagemap for this device
> + * @dev: Requesting device (for P2P check)
> + * @page: ZONE_DEVICE page backed by VRAM
> + * @order: Page order (0 = 4K, 9 = 2M, etc.)
> + * @dir: DMA direction (unused for local VRAM)
> + *
> + * Address conversion chain:
> + * page -> PFN -> HPA -> VRAM offset -> PTE address
> + *
> + * HPA = page_to_pfn(page) << PAGE_SHIFT
> + * VRAM offset = HPA - apagemap.hpa_base
> + * PTE address = VRAM offset + adev->vm_manager.vram_base_offset
> + *
> + * Return: drm_pagemap_addr with PTE address and AMDGPU_INTERCONNECT_VRAM protocol
> + */
> +static struct drm_pagemap_addr
> +amdgpu_svm_device_map(struct drm_pagemap *dpagemap,
> + struct device *dev,
> + struct page *page,
> + unsigned int order,
> + enum dma_data_direction dir)
> +{
> + struct amdgpu_pagemap *svm_dm = to_amdgpu_pagemap(dpagemap);
> + struct amdgpu_device *adev = dpagemap_to_adev(dpagemap);
> + dma_addr_t addr;
> +
> + if (dpagemap->drm->dev == dev) {
> + /* Same device: return VRAM PTE address */
> + u64 hpa = (u64)page_to_pfn(page) << PAGE_SHIFT;
> + u64 vram_offset = hpa - svm_dm->hpa_base;
> +
> + addr = vram_offset + adev->vm_manager.vram_base_offset;
> + } else {
> + /* Cross-device P2P: not yet supported */
> + addr = DMA_MAPPING_ERROR;
> + }
> +
> + return drm_pagemap_addr_encode(addr,
> + AMDGPU_INTERCONNECT_VRAM, order, dir);
> +}
> +
> +/**
> + * amdgpu_svm_bo_alloc - Allocate an amdgpu_svm_bo wrapper with VRAM backing
> + * @adev: AMDGPU device
> + * @dpagemap: The drm_pagemap for this device
> + * @mm: mm_struct of the owning process
> + * @size: Allocation size in bytes
> + *
> + * Return: Pointer to allocated amdgpu_svm_bo on success, ERR_PTR on failure
> + */
> +static struct amdgpu_svm_bo *
> +amdgpu_svm_bo_alloc(struct amdgpu_device *adev,
> + struct drm_pagemap *dpagemap,
> + struct mm_struct *mm, unsigned long size)
> +{
> + struct amdgpu_svm_bo *svm_bo;
> + struct amdgpu_bo_param bp = {};
> + struct amdgpu_bo *bo;
> + int ret;
> +
> + svm_bo = kzalloc(sizeof(*svm_bo), GFP_KERNEL);
> + if (!svm_bo)
> + return ERR_PTR(-ENOMEM);
> +
> + bp.size = size;
> + bp.bo_ptr_size = sizeof(struct amdgpu_bo);
> + bp.domain = AMDGPU_GEM_DOMAIN_VRAM;
> + bp.type = ttm_bo_type_device;
> + bp.flags = AMDGPU_GEM_CREATE_NO_CPU_ACCESS |
> + AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS |
> + AMDGPU_GEM_CREATE_VRAM_CLEARED;
> +
> + ret = amdgpu_bo_create(adev, &bp, &bo);
> + if (ret) {
> + AMDGPU_MIGRATE_TRACE("Failed to create SVM BO\n");
> + kfree(svm_bo);
> + return ERR_PTR(ret);
> + }
> +
> + amdgpu_bo_unreserve(bo);
> + svm_bo->bo = bo;
> +
> + drm_pagemap_devmem_init(&svm_bo->devmem,
> + adev->dev, mm,
> + &amdgpu_pagemap_ops,
> + dpagemap, size, NULL);
> +
> + return svm_bo;
> +}
> +
> +/**
> + * amdgpu_svm_populate_mm - Allocate VRAM BO and migrate pages
> + * @dpagemap: The drm_pagemap for this device
> + * @start: Start virtual address of the range to migrate
> + * @end: End virtual address (exclusive)
> + * @mm: mm_struct of the owning process
> + * @timeslice_ms: Maximum time to spend migrating (for fairness)
> + *
> + * Core migration entry point called by drm_pagemap_populate_mm().
> + * Allocates an amdgpu_svm_bo via amdgpu_svm_bo_alloc(), then calls
> + * drm_pagemap_migrate_to_devmem() to execute the actual migration.
> + *
> + * Return: 0 on success, negative error code on failure
> + */
> +static int
> +amdgpu_svm_populate_mm(struct drm_pagemap *dpagemap,
> + unsigned long start, unsigned long end,
> + struct mm_struct *mm,
> + unsigned long timeslice_ms)
> +{
> + struct amdgpu_device *adev = dpagemap_to_adev(dpagemap);
> + struct drm_pagemap_migrate_details mdetails = {
> + .timeslice_ms = timeslice_ms,
> + };
> + struct amdgpu_svm_bo *svm_bo;
> + int ret;
> +
> + svm_bo = amdgpu_svm_bo_alloc(adev, dpagemap, mm, end - start);
> + if (IS_ERR(svm_bo))
> + return PTR_ERR(svm_bo);
> +
> + AMDGPU_MIGRATE_TRACE("populate_mm: [0x%lx-0x%lx] size=%lu\n",
> + start, end, end - start);
> +
> + ret = drm_pagemap_migrate_to_devmem(&svm_bo->devmem,
> + mm, start, end,
> + &mdetails);
> +
> + return ret;
> +}
>
> -const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops = { };
> +const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops = {
> + .device_map = amdgpu_svm_device_map,
> + .populate_mm = amdgpu_svm_populate_mm,
> +};
>
> /**
> * amdgpu_svm_migration_init - Register ZONE_DEVICE and initialize drm_pagemap
^ permalink raw reply [flat|nested] 13+ messages in thread
* Claude review: drm/amdgpu: SVM VRAM migration via drm_pagemap
2026-04-27 10:05 [PATCH v3 0/5] drm/amdgpu: SVM VRAM migration via drm_pagemap Junhua Shen
` (4 preceding siblings ...)
2026-04-27 10:05 ` [PATCH v3 5/5] drm/amdgpu: integrate VRAM migration into SVM range map path Junhua Shen
@ 2026-04-28 4:43 ` Claude Code Review Bot
5 siblings, 0 replies; 13+ messages in thread
From: Claude Code Review Bot @ 2026-04-28 4:43 UTC (permalink / raw)
To: dri-devel-reviews
Overall Series Review
Subject: drm/amdgpu: SVM VRAM migration via drm_pagemap
Author: Junhua Shen <Junhua.Shen@amd.com>
Patches: 7
Reviewed: 2026-04-28T14:43:03.699323
---
This is a v3 of a 5-patch series adding VRAM migration support to the amdgpu SVM subsystem via `drm_pagemap`. The architecture is sound — it layers cleanly on top of the existing drm_pagemap/drm_gpusvm infrastructure with a clear separation of concerns: infrastructure (patch 1), SDMA callbacks (patch 2), decision layer (patch 3), attribute extensions (patch 4), and integration (patch 5).
The series builds on an unmerged dependency (the drm_pagemap SVM series), which explains why patches don't apply to drm-next. Within its own scope, the code is reasonably well-structured.
**Key concerns:**
1. **The `pre_migrate_fence` parameter is completely ignored** in both `copy_to_devmem` and `copy_to_ram`. The drm_pagemap API documentation explicitly states this fence "must be waited for before migration start." This is a correctness bug that can cause data races.
2. **Migration failure in the map path is silently ignored** (patch 5) — `amdgpu_svm_range_migrate_range()` return value is discarded.
3. **The `amdgpu_svm_range_rebuild_locked()` behavior change** in patch 5 drops the GPU unmap + TLB flush fallback path with no replacement, which looks like a regression for the `!rebuild` case.
4. **Variable declaration after statement** in patch 5 (`enum amdgpu_svm_migrate_mode` declared after `lockdep_assert_held_write()`), which violates C89 style expected by the kernel.
5. **`AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS`** forces contiguous VRAM for every migration, which will cause allocation failures under fragmentation. This should be reconsidered.
---
---
Generated by Claude Code Patch Reviewer
^ permalink raw reply [flat|nested] 13+ messages in thread
* Claude review: drm/amdgpu: add VRAM migration infrastructure for drm_pagemap
2026-04-27 10:05 ` [PATCH v3 1/5] drm/amdgpu: add VRAM migration infrastructure for drm_pagemap Junhua Shen
@ 2026-04-28 4:43 ` Claude Code Review Bot
0 siblings, 0 replies; 13+ messages in thread
From: Claude Code Review Bot @ 2026-04-28 4:43 UTC (permalink / raw)
To: dri-devel-reviews
Patch Review
**Overall:** Good foundational patch. Sets up ZONE_DEVICE registration and drm_pagemap initialization.
**Issue: SPDX vs. license boilerplate mismatch.** The file uses `/* SPDX-License-Identifier: GPL-2.0 OR MIT */` but then includes the full MIT permissive license text in the header block. The SPDX tag should be sufficient; the verbose boilerplate is unnecessary and adds noise. This applies to all new files in this series.
**Issue: `pgmap->flags = 0` is redundant** after `devm_kzalloc`:
```c
+ pgmap->nr_range = 1;
+ pgmap->flags = 0;
```
The struct was zero-initialized already.
**Issue: Error path leak in `drm_pagemap_init` failure.** If `drm_pagemap_init()` fails, the `devm_memremap_pages()` region is not cleaned up:
```c
+ if (drm_pagemap_init(&svm_dm->dpagemap, pgmap, adev_to_drm(adev),
+ &amdgpu_svm_drm_pagemap_ops)) {
+ dev_err(adev->dev, "SVM: failed to init drm_pagemap\n");
+ return -EINVAL;
+ }
```
After a successful `devm_memremap_pages()`, failing to init the pagemap should unwind the memremap. Although devres will eventually clean up, returning `-EINVAL` here leaves `adev->apagemap` as NULL with a phantom ZONE_DEVICE region registered.
**Issue: Alignment of `drm_pagemap_init` arguments:**
```c
+ if (drm_pagemap_init(&svm_dm->dpagemap, pgmap, adev_to_drm(adev),
+ &amdgpu_svm_drm_pagemap_ops)) {
```
The continuation is over-indented — should align after the opening paren per kernel style.
**Nit: `struct dev_pagemap` flex-array comment.** The comment says `pgmap` "must be last — flex-array" but `struct dev_pagemap` in current kernels uses `struct range ranges[]` which is indeed a flex-array. This constraint should be documented in the `amdgpu_pagemap` definition with `static_assert` or a build-time check, not just a comment.
**Question: `AMDGPU_SVM_PGMAP_OWNER` and `adev->hive`.** The macro:
```c
+#define AMDGPU_SVM_PGMAP_OWNER(adev) \
+ ((adev)->hive ? (void *)(adev)->hive : (void *)(adev))
```
The `hive` field on `struct amdgpu_device` is `struct amdgpu_hive_info *hive`. This is valid, but note that `hive` can be set/cleared during runtime (XGMI topology changes). If the owner changes between registration and migration, `migrate_vma_setup()` will misidentify pages. Is this a concern for hot-plug scenarios?
**Nit:** `dpagemap_to_adev()` and `amdgpu_svm_page_to_apagemap()` are defined here but not used until patch 2. Unused code in patch 1 could trigger compiler warnings depending on config.
---
Generated by Claude Code Patch Reviewer
^ permalink raw reply [flat|nested] 13+ messages in thread
* Claude review: drm/amdgpu: implement drm_pagemap SDMA migration callbacks
2026-04-27 10:05 ` [PATCH v3 2/5] drm/amdgpu: implement drm_pagemap SDMA migration callbacks Junhua Shen
2026-04-27 22:20 ` Felix Kuehling
@ 2026-04-28 4:43 ` Claude Code Review Bot
1 sibling, 0 replies; 13+ messages in thread
From: Claude Code Review Bot @ 2026-04-28 4:43 UTC (permalink / raw)
To: dri-devel-reviews
Patch Review
**Overall:** This is the largest patch (613 lines added) and implements the core migration mechanics. The SDMA copy logic follows the established KFD `svm_migrate_copy_memory_gart()` pattern, which is good.
**Bug: `pre_migrate_fence` is completely ignored.** Both callbacks accept the fence but never wait on it:
```c
+static int
+amdgpu_svm_copy_to_devmem(struct page **pages,
+ struct drm_pagemap_addr *pagemap_addr,
+ unsigned long npages,
+ struct dma_fence *pre_migrate_fence)
+{
```
The `drm_pagemap_devmem_ops` documentation says: "dma-fence to wait for before migration start. May be NULL." If this fence represents pending operations on the source pages (e.g., from another GPU or DMA engine), ignoring it can cause data corruption. At minimum, add:
```c
if (pre_migrate_fence) {
ret = dma_fence_wait(pre_migrate_fence, true);
if (ret)
return ret;
}
```
Or better, chain it with the SDMA fence.
**Issue: `AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS` is unnecessarily restrictive:**
```c
+ bp.flags = AMDGPU_GEM_CREATE_NO_CPU_ACCESS |
+ AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS |
+ AMDGPU_GEM_CREATE_VRAM_CLEARED;
```
The `populate_devmem_pfn` callback already handles non-contiguous buddy blocks via `amdgpu_res_cursor`. Requiring contiguous VRAM for every migration will fail under fragmentation. The `VRAM_CLEARED` flag also adds unnecessary SDMA overhead since the pages will be immediately overwritten by `copy_to_devmem`. Consider dropping both flags.
**Issue: Batch break logic has a subtle correctness concern.** In `amdgpu_svm_copy_to_devmem`:
```c
+ /* Check if next vram page is contiguous with current */
+ if (j > 0 && vram[j] != vram[j - 1] + PAGE_SIZE)
+ goto flush;
```
This only checks VRAM contiguity but not system DMA address contiguity. The GART window maps each system page individually so this is probably fine, but the comment should clarify this is intentional.
**Issue: `copy_to_ram` and `copy_to_devmem` have significant code duplication.** These two functions are nearly identical with src/dst swapped. Consider factoring the common batch-and-flush loop into a shared helper.
**Nit: Trace uses `%px` for pointer printing:**
```c
+ AMDGPU_MIGRATE_TRACE("Release svm_bo=%px bo=%px\n", svm_bo, svm_bo->bo);
```
Using `%px` exposes raw kernel pointers, which is a security concern (KASLR bypass). Use `%p` (hashed) unless this is for debug builds only. Since this is `pr_debug`, it's gated, but best practice is still `%p`.
**Issue: `amdgpu_svm_device_map` returns `DMA_MAPPING_ERROR` for P2P but encodes it successfully:**
```c
+ } else {
+ /* Cross-device P2P: not yet supported */
+ addr = DMA_MAPPING_ERROR;
+ }
+
+ return drm_pagemap_addr_encode(addr,
+ AMDGPU_INTERCONNECT_VRAM, order, dir);
```
This encodes the error address with `AMDGPU_INTERCONNECT_VRAM` protocol. The caller may not check for `DMA_MAPPING_ERROR` if the protocol looks valid. Should return a proper error indication — check how the framework expects failures to be signaled from `device_map`.
**Issue: `amdgpu_svm_gart_map` submits a job but never waits for it.** The GART PTE update job is submitted and the fence is immediately dropped:
```c
+ fence = amdgpu_job_submit(job);
+ dma_fence_put(fence);
```
The subsequent `amdgpu_copy_buffer()` call needs the GART entries to be committed. This relies on GPU command ordering within the same ring — if the GART update and the copy are on the same SDMA ring, the HW will serialize them. But this assumption should be documented, and if entity scheduling ever routes these to different rings, it would break.
---
Generated by Claude Code Patch Reviewer
^ permalink raw reply [flat|nested] 13+ messages in thread
* Claude review: drm/amdgpu: introduce SVM range migration decision layer
2026-04-27 10:05 ` [PATCH v3 3/5] drm/amdgpu: introduce SVM range migration decision layer Junhua Shen
@ 2026-04-28 4:43 ` Claude Code Review Bot
0 siblings, 0 replies; 13+ messages in thread
From: Claude Code Review Bot @ 2026-04-28 4:43 UTC (permalink / raw)
To: dri-devel-reviews
Patch Review
**Overall:** Clean, minimal patch. The decision layer is well-factored.
**Issue: `range_needs_migrate_to_vram` doesn't check for `AMDGPU_SVM_MIGRATE_TO_VRAM` explicitly:**
```c
+static bool
+range_needs_migrate_to_vram(struct drm_gpusvm_range *range,
+ enum amdgpu_svm_migrate_mode mode)
+{
+ if (!range->pages.flags.migrate_devmem)
+ return false;
+
+ if (mode == AMDGPU_SVM_MIGRATE_TO_SYSMEM)
+ return false;
```
This means if `mode == AMDGPU_SVM_MIGRATE_NONE` (which is excluded by the WARN_ON caller, but still), it would return true. The function is only called from `amdgpu_svm_range_migrate_range()` which validates mode, so this is safe in practice but the logic could be clearer.
**Nit:** `amdgpu_pagemap_capable()` checks `adev->gmc.is_app_apu` — the name `is_app_apu` seems like an unusual field. Confirm this is the right check for "this device can do VRAM migration."
**Question:** The `AMDGPU_SVM_MIGRATE_PREFERRED` mode is defined in the enum but never handled in `amdgpu_svm_range_migrate_range()` — it triggers the WARN_ON. The resolution happens in the caller (`amdgpu_svm_range_map` in patch 5). This layering is correct but the docstring on the enum should make it even more explicit that PREFERRED must be resolved before calling migrate_range.
---
Generated by Claude Code Patch Reviewer
^ permalink raw reply [flat|nested] 13+ messages in thread
* Claude review: drm/amdgpu: add SVM attr prefetch/force-trigger functionality
2026-04-27 10:05 ` [PATCH v3 4/5] drm/amdgpu: add SVM attr prefetch/force-trigger functionality Junhua Shen
@ 2026-04-28 4:43 ` Claude Code Review Bot
0 siblings, 0 replies; 13+ messages in thread
From: Claude Code Review Bot @ 2026-04-28 4:43 UTC (permalink / raw)
To: dri-devel-reviews
Patch Review
**Overall:** Small and focused. The one-shot prefetch semantics are well-motivated.
**Issue: `preferred_loc` changes no longer trigger `LOCATION_CHANGE`:**
```c
- if (prev_attrs->preferred_loc != new_attrs->preferred_loc ||
- prev_attrs->prefetch_loc != new_attrs->prefetch_loc)
+ if (prev_attrs->prefetch_loc != new_attrs->prefetch_loc)
trigger |= AMDGPU_SVM_ATTR_TRIGGER_LOCATION_CHANGE;
```
This means changing `preferred_loc` alone will never trigger migration. In patch 5, `amdgpu_svm_range_apply_attr_change()` only acts on `AMDGPU_SVM_ATTR_TRIGGER_LOCATION_CHANGE` for migration. So if userspace sets a new `preferred_loc` without also issuing a prefetch, the new location preference won't take effect until the next fault or restore event. Is this intentional? The commit message doesn't explain this tradeoff.
**Good:** The `force_trigger` change from `bool` to `uint32_t` bitmask is the right approach — it allows composing multiple trigger reasons.
---
Generated by Claude Code Patch Reviewer
^ permalink raw reply [flat|nested] 13+ messages in thread
* Claude review: drm/amdgpu: integrate VRAM migration into SVM range map path
2026-04-27 10:05 ` [PATCH v3 5/5] drm/amdgpu: integrate VRAM migration into SVM range map path Junhua Shen
@ 2026-04-28 4:43 ` Claude Code Review Bot
0 siblings, 0 replies; 13+ messages in thread
From: Claude Code Review Bot @ 2026-04-28 4:43 UTC (permalink / raw)
To: dri-devel-reviews
Patch Review
**Overall:** This is the integration patch tying everything together. It has the most issues.
**Bug: Migration failure is silently ignored:**
```c
+ /* Per-range migration */
+ if (migrate_mode != AMDGPU_SVM_MIGRATE_NONE)
+ amdgpu_svm_range_migrate_range(svm, range, migrate_mode);
```
The return value of `amdgpu_svm_range_migrate_range()` is discarded. If migration fails (e.g., VRAM full, SDMA error), the code proceeds to map the range as if nothing happened. This could result in stale system memory mappings when VRAM was expected, or worse, mapping garbage. At minimum, log the failure. Depending on the desired semantics, either fail the map or fall back gracefully with a warning.
**Bug: Variable declaration after statement (C89 violation):**
```c
lockdep_assert_held_write(&svm->svm_lock);
+ enum amdgpu_svm_migrate_mode migrate_mode = AMDGPU_SVM_MIGRATE_NONE;
bool old_access, new_access;
```
The `lockdep_assert_held_write()` is a statement, and variable declarations must come before any statements in kernel C (C89/C99-mixed style). Move the declaration before the `lockdep_assert_held_write()` call, or move the assert after all declarations.
**Concern: `amdgpu_svm_range_rebuild_locked()` behavior change.** The `!rebuild` path was:
```c
- ret = amdgpu_svm_range_update_gpu(svm, rebuild_start, rebuild_last,
- 0, NULL, true, true, true);
- if (!ret)
- svm->flush_tlb(svm);
- return ret;
+ return 0;
```
This changes from "unmap GPU mappings and flush TLB" to "do nothing." If the `!rebuild` case represents an invalidation (pages gone, need to unmap GPU PTEs), then returning 0 silently leaves stale GPU mappings. What guarantees that the GPU unmap already happened by this point? This needs justification in the commit message.
**Issue: `#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)` guards in `amdgpu_device.c` and `amdgpu_reset.c`:**
```c
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
+ amdgpu_svm_migration_init(adev);
+#endif
```
The header already provides a static inline stub returning 0 when `CONFIG_DRM_AMDGPU_SVM` is disabled. These `#if` guards are unnecessary and add noise — just call the function directly and rely on the stub.
**Issue: `range_invalidate_gpu_mapping()` reordering in notifier.** Moving this call:
```c
- if (clear_pte) {
- amdgpu_svm_range_gpu_unmap_in_notifier(svm, range,
- mmu_range);
- range_invalidate_gpu_mapping(range);
- }
+ if (clear_pte) {
+ amdgpu_svm_range_gpu_unmap_in_notifier(svm, range,
+ mmu_range);
+ }
drm_gpusvm_range_unmap_pages(&svm->gpusvm, range, &ctx);
+ range_invalidate_gpu_mapping(range);
```
Now `range_invalidate_gpu_mapping()` is called unconditionally (outside the `if (clear_pte)` block), and after `drm_gpusvm_range_unmap_pages()`. This is a semantic change — previously it only happened when PTE clearing was needed. Is this intentional? The commit message should explain why invalidation now always happens and why the ordering change is correct.
**Minor: Deleted function `amdgpu_svm_range_update_gpu` still has callers?** The function is removed in this patch, but the `!rebuild` path in `amdgpu_svm_range_rebuild_locked()` was the only caller. Since that path now returns 0, the deletion is consistent. Good.
**Nit: Trailing whitespace removal:**
```c
-
}
```
Line 1565 — this is fine cleanup but should ideally be in a separate trivial patch to keep the diff focused.
**Good:** The `devmem_possible` and `device_private_page_owner` additions to `gpusvm_ctx` are cleanly integrated:
```c
+ struct drm_gpusvm_ctx gpusvm_ctx = {
+ .read_only = !!(attrs->flags & AMDGPU_SVM_FLAG_GPU_RO),
+ .devmem_possible = hw_devmem,
+ .device_private_page_owner = hw_devmem ?
+ AMDGPU_SVM_PGMAP_OWNER(svm->adev) : NULL,
+ };
```
**Good:** The VRAM PTE flag handling is correct — clearing SYSTEM and SNOOPED bits for VRAM pages:
```c
+ if (entry->proto == AMDGPU_INTERCONNECT_VRAM)
+ seg_pte_flags &= ~(AMDGPU_PTE_SYSTEM | AMDGPU_PTE_SNOOPED);
```
---
Generated by Claude Code Patch Reviewer
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2026-04-28 4:43 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-04-27 10:05 [PATCH v3 0/5] drm/amdgpu: SVM VRAM migration via drm_pagemap Junhua Shen
2026-04-27 10:05 ` [PATCH v3 1/5] drm/amdgpu: add VRAM migration infrastructure for drm_pagemap Junhua Shen
2026-04-28 4:43 ` Claude review: " Claude Code Review Bot
2026-04-27 10:05 ` [PATCH v3 2/5] drm/amdgpu: implement drm_pagemap SDMA migration callbacks Junhua Shen
2026-04-27 22:20 ` Felix Kuehling
2026-04-28 4:43 ` Claude review: " Claude Code Review Bot
2026-04-27 10:05 ` [PATCH v3 3/5] drm/amdgpu: introduce SVM range migration decision layer Junhua Shen
2026-04-28 4:43 ` Claude review: " Claude Code Review Bot
2026-04-27 10:05 ` [PATCH v3 4/5] drm/amdgpu: add SVM attr prefetch/force-trigger functionality Junhua Shen
2026-04-28 4:43 ` Claude review: " Claude Code Review Bot
2026-04-27 10:05 ` [PATCH v3 5/5] drm/amdgpu: integrate VRAM migration into SVM range map path Junhua Shen
2026-04-28 4:43 ` Claude review: " Claude Code Review Bot
2026-04-28 4:43 ` Claude review: drm/amdgpu: SVM VRAM migration via drm_pagemap Claude Code Review Bot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox