Claude review: drm/amdgpu: add VRAM migration infrastructure for drm_pagemap

Claude review: drm/amdgpu: add VRAM migration infrastructure for drm_pagemap

[Claude Code Review Bot]

Patch Review

**Overall:** Good foundational patch. Sets up ZONE_DEVICE registration and drm_pagemap initialization.

**Issue: SPDX vs. license boilerplate mismatch.** The file uses `/* SPDX-License-Identifier: GPL-2.0 OR MIT */` but then includes the full MIT permissive license text in the header block. The SPDX tag should be sufficient; the verbose boilerplate is unnecessary and adds noise. This applies to all new files in this series.

**Issue: `pgmap->flags = 0` is redundant** after `devm_kzalloc`:
```c
+	pgmap->nr_range = 1;
+	pgmap->flags = 0;
```
The struct was zero-initialized already.

**Issue: Error path leak in `drm_pagemap_init` failure.** If `drm_pagemap_init()` fails, the `devm_memremap_pages()` region is not cleaned up:
```c
+	if (drm_pagemap_init(&svm_dm->dpagemap, pgmap, adev_to_drm(adev),
+							&amdgpu_svm_drm_pagemap_ops)) {
+		dev_err(adev->dev, "SVM: failed to init drm_pagemap\n");
+		return -EINVAL;
+	}
```
After a successful `devm_memremap_pages()`, failing to init the pagemap should unwind the memremap. Although devres will eventually clean up, returning `-EINVAL` here leaves `adev->apagemap` as NULL with a phantom ZONE_DEVICE region registered.

**Issue: Alignment of `drm_pagemap_init` arguments:**
```c
+	if (drm_pagemap_init(&svm_dm->dpagemap, pgmap, adev_to_drm(adev),
+							&amdgpu_svm_drm_pagemap_ops)) {
```
The continuation is over-indented — should align after the opening paren per kernel style.

**Nit: `struct dev_pagemap` flex-array comment.** The comment says `pgmap` "must be last — flex-array" but `struct dev_pagemap` in current kernels uses `struct range ranges[]` which is indeed a flex-array. This constraint should be documented in the `amdgpu_pagemap` definition with `static_assert` or a build-time check, not just a comment.

**Question: `AMDGPU_SVM_PGMAP_OWNER` and `adev->hive`.** The macro:
```c
+#define AMDGPU_SVM_PGMAP_OWNER(adev) \
+	((adev)->hive ? (void *)(adev)->hive : (void *)(adev))
```
The `hive` field on `struct amdgpu_device` is `struct amdgpu_hive_info *hive`. This is valid, but note that `hive` can be set/cleared during runtime (XGMI topology changes). If the owner changes between registration and migration, `migrate_vma_setup()` will misidentify pages. Is this a concern for hot-plug scenarios?

**Nit:** `dpagemap_to_adev()` and `amdgpu_svm_page_to_apagemap()` are defined here but not used until patch 2. Unused code in patch 1 could trigger compiler warnings depending on config.

---
Generated by Claude Code Patch Reviewer

[PATCH v4 0/6] drm/amdgpu: SVM VRAM migration via drm_pagemap (XNACK-on)

[Junhua Shen]

This series adds VRAM migration support to amdgpu's SVM (Shared Virtual
Memory) implementation, using the drm_pagemap framework for ZONE_DEVICE
page management and SDMA for data migration.

This is the XNACK-on (GPU fault-driven) version of the migration
series, built on top of the drm_gpusvm-based amdgpu SVM core [1].
Previous v1/v2/v3 were XNACK-off (ioctl-driven) based on an earlier
SVM core; this v4 is a rewrite targeting the XNACK-on path.

The implementation follows the Xe driver's approach for TTM eviction,
using synchronous bo_move to migrate device-private pages back to system
RAM when TTM needs to evict SVM BOs.

Key design points:
- GPU VRAM registered as ZONE_DEVICE via devm_memremap_pages(),
  wrapped in struct amdgpu_pagemap with drm_pagemap state
- SDMA-based data transfer through GART aperture window for both
  copy_to_devmem and copy_to_ram callbacks
- amdgpu_bo_svm: lightweight BO subtype with drm_pagemap_devmem for
  ZONE_DEVICE page ownership tracking
- Synchronous TTM eviction via drm_pagemap_evict_to_ram() in bo_move,
  following the Xe pattern (no eviction fences needed)
- Migration policy driven by SVM range attributes (preferred location,
  prefetch hints) and GPU fault path

Limitations:
- Single GPU only; multi-GPU migration is not addressed
- No VRAM-to-VRAM (peer GPU) migration

Open issue:
- Unnecessary TTM system memory allocation during eviction: when TTM
  evicts an SVM BO, it allocates a destination system memory resource
  (TTM_PL_SYSTEM) before calling bo_move, then frees it afterwards.
  This allocation is unnecessary because the actual data migration is
  done via drm_pagemap_evict_to_ram() → migrate_device_* which
  migrates device-private pages directly to regular system pages,
  bypassing the TTM-allocated resource entirely. The current TTM
  framework does not support num_placement=0 to skip this redundant
  allocation; this needs further discussion.

Dependencies:

This series applies on top of the amdgpu drm_gpusvm SVM core [1].

[1] https://lore.kernel.org/amd-gfx/20260508075129.1161157-1-honglei1.huang@amd.com/

Changes since v3:
- Rebased on drm_gpusvm-based amdgpu SVM core [1], switching from
  XNACK-off ioctl-driven to XNACK-on GPU fault-driven migration
- Introduced amdgpu_bo_svm subtype with drm_pagemap_devmem embedding
  and two-layer reference counting (GEM refcount + TTM kref)
- Added synchronous TTM eviction via drm_pagemap_evict_to_ram() in
  amdgpu_bo_move(), following the Xe driver pattern
- Added amdgpu_bo_is_amdgpu_bo() check for SVM BOs in TTM path
- Cleaned up container_of macros to follow amdgpu conventions
  (to_amdgpu_bo_svm as #define, devmem_to_amdgpu_bo_svm as inline)

Changes since v2:
- Moved amdgpu_pagemap entirely to amdgpu side, eliminating all KFD
  modifications
- Split commits for better reviewability: separated infrastructure
  from SDMA callbacks, decision layer from integration
- Merged ZONE_DEVICE registration hook into the integration patch

Changes since v1:
- Dropped the eviction fence patch (was 4/6) after Christian König
  pointed out it violates the dma_fence contract
- Refactored migration integration: extracted migration logic into
  new files amdgpu_svm_range_migrate.{c,h}
- Introduced enum amdgpu_svm_migrate_mode (PREFERRED, TO_VRAM,
  TO_SYSMEM, NONE) to make migration intent explicit, replacing
  the _ex functions used in v1

Previous versions:
v1 (XNACK-off): https://lore.kernel.org/amd-gfx/20260410113146.146212-1-Junhua.Shen@amd.com/
v2 (XNACK-off): https://lore.kernel.org/amd-gfx/20260413103031.181953-1-Junhua.Shen@amd.com/
v3 (XNACK-off): https://lore.kernel.org/amd-gfx/20260427100522.7014-1-Junhua.Shen@amd.com/

Test results:
  Tested on gfx943 (MI300X) and gfx906 (MI60) with XNACK on:
  - KFD test: 95%+ passed.
  - ROCR test: all passed.

Patch overview:
  1/6  Core VRAM migration infrastructure (ZONE_DEVICE registration,
       amdgpu_pagemap, amdgpu_bo_svm subtype, drm_pagemap_ops)
  2/6  SDMA migration callbacks (copy_to_devmem, copy_to_ram,
       populate_devmem_pfn via GART aperture window)
  3/6  Synchronous TTM eviction for SVM BOs (amdgpu_svm_bo_evict
       in bo_move path, amdgpu_bo_is_amdgpu_bo check)
  4/6  SVM range migration helpers (range-level migrate_to_vram /
       migrate_to_sysmem decision layer)
  5/6  Hook up ZONE_DEVICE registration in device init and GPU reset
  6/6  Wire up VRAM migration into SVM range map and GPU fault paths

Junhua Shen (6):
  drm/amdgpu: add VRAM migration infrastructure for drm_pagemap
  drm/amdgpu: implement drm_pagemap SDMA migration callbacks
  drm/amdgpu: implement synchronous TTM eviction for SVM BOs
  drm/amdgpu: add SVM range migration helpers for drm_pagemap
  drm/amdgpu: hook up ZONE_DEVICE registration in device init and reset
  drm/amdgpu: integrate VRAM migration into SVM range map and fault
    paths

 drivers/gpu/drm/amd/amdgpu/Makefile           |   6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu.h           |   8 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    |   4 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c   | 831 ++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h   | 110 +++
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c    |   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c     |   4 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c  |   4 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.c |   9 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c |  21 +-
 .../drm/amd/amdgpu/amdgpu_svm_range_migrate.c | 122 +++
 .../drm/amd/amdgpu/amdgpu_svm_range_migrate.h |  47 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  20 +
 13 files changed, 1181 insertions(+), 9 deletions(-)
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h

-- 
2.34.1

[PATCH v4 1/6] drm/amdgpu: add VRAM migration infrastructure for drm_pagemap

[Junhua Shen]

Add the drm_pagemap-based VRAM migration infrastructure:

- Define struct amdgpu_pagemap wrapping dev_pagemap + drm_pagemap
- Define AMDGPU_PGMAP_OWNER() and AMDGPU_INTERCONNECT_VRAM macros
- Implement amdgpu_svm_migration_init() to register ZONE_DEVICE via
  devm_memremap_pages() and initialize the drm_pagemap
- Add amdgpu_pagemap pointer (apagemap) to struct amdgpu_device

Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/Makefile         |   6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu.h         |   8 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c | 179 ++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h |  98 +++++++++++
 4 files changed, 289 insertions(+), 2 deletions(-)
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h

diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile b/drivers/gpu/drm/amd/amdgpu/Makefile
index a40a42995b9a..3905a97795aa 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -323,12 +323,14 @@ amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_hmm.o
 
 # svm support
 amdgpu-$(CONFIG_DRM_AMDGPU_SVM) += amdgpu_svm.o amdgpu_svm_attr.o \
-	amdgpu_svm_fault.o amdgpu_svm_range.o
+	amdgpu_svm_fault.o amdgpu_svm_range.o amdgpu_migrate.o
 
 .PHONY: clean-svm
 clean-svm:
 	rm -f $(obj)/amdgpu_svm.o $(obj)/amdgpu_svm_attr.o $(obj)/amdgpu_svm_fault.o $(obj)/amdgpu_svm_range.o \
-	      $(obj)/.amdgpu_svm.o.cmd $(obj)/.amdgpu_svm_attr.o.cmd $(obj)/.amdgpu_svm_fault.o.cmd $(obj)/.amdgpu_svm_range.o.cmd
+	      $(obj)/amdgpu_migrate.o \
+	      $(obj)/.amdgpu_svm.o.cmd $(obj)/.amdgpu_svm_attr.o.cmd $(obj)/.amdgpu_svm_fault.o.cmd $(obj)/.amdgpu_svm_range.o.cmd \
+	      $(obj)/.amdgpu_migrate.o.cmd
 
 include $(FULL_AMD_PATH)/pm/Makefile
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 49e7881750fa..fe6ba9911d9f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -325,6 +325,7 @@ struct amdgpu_fpriv;
 struct amdgpu_bo_va_mapping;
 struct kfd_vm_fault_info;
 struct amdgpu_hive_info;
+struct amdgpu_pagemap;
 struct amdgpu_reset_context;
 struct amdgpu_reset_control;
 struct amdgpu_coredump_info;
@@ -1200,6 +1201,13 @@ struct amdgpu_device {
 
 	struct amdgpu_uma_carveout_info uma_info;
 
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
+	/* SVM VRAM migration via drm_pagemap (drm_gpusvm path).
+	 * Allocated in amdgpu_svm_migration_init(), NULL if SVM disabled.
+	 */
+	struct amdgpu_pagemap		*apagemap;
+#endif
+
 	/* KFD
 	 * Must be last --ends in a flexible-array member.
 	 */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
new file mode 100644
index 000000000000..a8b067831b99
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright 2026 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+/**
+ * DOC: AMDGPU SVM Migration
+ *
+ * This file implements the drm_pagemap-based migration infrastructure for
+ * AMDGPU SVM. It provides the callbacks that the DRM GPUSVM / drm_pagemap
+ * framework needs to:
+ *
+ * 1. Map ZONE_DEVICE pages to GPU-visible VRAM MC addresses (device_map)
+ * 2. Allocate VRAM and migrate pages from system memory (populate_mm)
+ * 3. Copy data between RAM and VRAM using SDMA (copy_to_devmem / copy_to_ram)
+ * 4. Release VRAM backing when pages migrate back to system memory (devmem_release)
+ *
+ * Architecture overview::
+ *
+ *   adev->apagemap->dpagemap   (struct drm_pagemap)
+ *       .ops = &amdgpu_svm_drm_pagemap_ops
+ *           |
+ *       +---+-------------------+
+ *       |                       |
+ *  .populate_mm            .device_map
+ *  (alloc BO + migrate)    (page -> VRAM MC addr)
+ *       |
+ *       v
+ *   drm_pagemap_devmem_ops  (per-BO migration mechanics)
+ *       .populate_devmem_pfn  -> BO buddy blocks -> PFN array
+ *       .copy_to_devmem       -> SDMA copy RAM -> VRAM
+ *       .copy_to_ram          -> SDMA copy VRAM -> RAM
+ *       .devmem_release       -> release BO reference
+ *
+ * The three address spaces involved:
+ *
+ *   VRAM offset  [0, real_vram_size)     - buddy allocator managed
+ *       + hpa_base
+ *   HPA / PFN    [hpa_base, hpa_base+..) - ZONE_DEVICE struct page management
+ *       + vm_manager.vram_base_offset
+ *   PTE address  [vram_base_offset, ..]  - GPU page table entries (from MMHUB FB_OFFSET)
+ */
+
+#include <drm/drm_pagemap.h>
+#include <linux/memremap.h>
+#include <linux/migrate.h>
+
+#include "amdgpu_amdkfd.h"
+#include "amdgpu_migrate.h"
+#include "amdgpu.h"
+
+static inline struct amdgpu_pagemap *
+dpagemap_to_apagemap(struct drm_pagemap *dpagemap)
+{
+	return container_of(dpagemap, struct amdgpu_pagemap, dpagemap);
+}
+
+static inline struct amdgpu_device *
+dpagemap_to_adev(struct drm_pagemap *dpagemap)
+{
+	return drm_to_adev(dpagemap->drm);
+}
+
+/**
+ * amdgpu_svm_page_to_apagemap - Get amdgpu_pagemap from a ZONE_DEVICE page
+ * @page: A ZONE_DEVICE page backed by VRAM
+ *
+ * Follows: page -> pgmap -> container_of(apagemap)
+ */
+static inline struct amdgpu_pagemap *
+amdgpu_svm_page_to_apagemap(struct page *page)
+{
+	struct dev_pagemap *pgmap = page_pgmap(page);
+
+	return container_of(pgmap, struct amdgpu_pagemap, pgmap);
+}
+
+
+const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops = { };
+
+/**
+ * amdgpu_svm_migration_init - Register ZONE_DEVICE and initialize drm_pagemap
+ * @adev: AMDGPU device to set up VRAM migration for
+ *
+ * Allocates a ZONE_DEVICE region covering the GPU's VRAM, registers it
+ * via devm_memremap_pages() with drm_pagemap's generic dev_pagemap_ops,
+ * and then initializes the drm_pagemap (dpagemap) that provides the
+ * device_map / populate_mm callbacks for the DRM GPUSVM migration path.
+ *
+ * For XGMI-connected CPUs, uses the device's aperture directly
+ * (MEMORY_DEVICE_COHERENT).  For discrete GPUs, requests a free
+ * iomem region for MEMORY_DEVICE_PRIVATE pages.
+ *
+ * Return: 0 on success, -EINVAL if GPU IP too old, negative error on failure
+ */
+int amdgpu_svm_migration_init(struct amdgpu_device *adev)
+{
+	struct amdgpu_pagemap *svm_dm;
+	struct dev_pagemap *pgmap;
+	struct resource *res = NULL;
+	unsigned long size;
+	void *r;
+
+	if (amdgpu_ip_version(adev, GC_HWIP, 0) < IP_VERSION(9, 0, 1))
+		return -EINVAL;
+
+	if (adev->apu_prefer_gtt)
+		return 0;
+
+	if (adev->apagemap && adev->apagemap->initialized)
+		return 0;
+
+	svm_dm = devm_kzalloc(adev->dev, sizeof(*svm_dm), GFP_KERNEL);
+	if (!svm_dm)
+		return -ENOMEM;
+
+	pgmap = &svm_dm->pgmap;
+
+	size = ALIGN(adev->gmc.real_vram_size, 2ULL << 20);
+	if (adev->gmc.xgmi.connected_to_cpu) {
+		pgmap->range.start = adev->gmc.aper_base;
+		pgmap->range.end = adev->gmc.aper_base + adev->gmc.aper_size - 1;
+		pgmap->type = MEMORY_DEVICE_COHERENT;
+	} else {
+		res = devm_request_free_mem_region(adev->dev, &iomem_resource, size);
+		if (IS_ERR(res))
+			return PTR_ERR(res);
+		pgmap->range.start = res->start;
+		pgmap->range.end = res->end;
+		pgmap->type = MEMORY_DEVICE_PRIVATE;
+	}
+
+	pgmap->nr_range = 1;
+	pgmap->flags = 0;
+	pgmap->ops = drm_pagemap_pagemap_ops_get();
+	pgmap->owner = AMDGPU_PGMAP_OWNER(adev);
+
+	r = devm_memremap_pages(adev->dev, pgmap);
+	if (IS_ERR(r)) {
+		dev_err(adev->dev, "SVM: failed to register HMM device memory\n");
+		if (pgmap->type == MEMORY_DEVICE_PRIVATE && res)
+			devm_release_mem_region(adev->dev, res->start, resource_size(res));
+		pgmap->type = 0;
+		return PTR_ERR(r);
+	}
+
+	if (drm_pagemap_init(&svm_dm->dpagemap, pgmap, adev_to_drm(adev),
+							&amdgpu_svm_drm_pagemap_ops)) {
+		dev_err(adev->dev, "SVM: failed to init drm_pagemap\n");
+		return -EINVAL;
+	}
+	svm_dm->adev = adev;
+	svm_dm->hpa_base = pgmap->range.start;
+	svm_dm->initialized = true;
+	adev->apagemap = svm_dm;
+
+	dev_info(adev->dev, "SVM: registered %ldMB device memory, hpa_base=0x%llx\n",
+			size >> 20, svm_dm->hpa_base);
+	return 0;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h
new file mode 100644
index 000000000000..75796983f6a5
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h
@@ -0,0 +1,98 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright 2026 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __AMDGPU_MIGRATE_H__
+#define __AMDGPU_MIGRATE_H__
+
+#include <drm/drm_pagemap.h>
+#include <linux/memremap.h>
+
+struct amdgpu_device;
+
+/*
+ * AMDGPU_INTERCONNECT_VRAM - Protocol identifier for local VRAM access.
+ *
+ * Used in drm_pagemap_addr to distinguish device-local VRAM addresses from
+ * DMA-mapped system memory addresses.  drm_gpusvm_get_pages() uses this to
+ * identify pages that are already in device memory and need no DMA mapping.
+ *
+ * Value must be non-zero (0 == DRM_INTERCONNECT_SYSTEM).
+ */
+#define AMDGPU_INTERCONNECT_VRAM	1
+
+/*
+ * AMDGPU_PGMAP_OWNER - Unique owner token for dev_pagemap registration.
+ *
+ * migrate_vma_setup() uses pgmap->owner to distinguish "own" device pages
+ * from "foreign" device pages (e.g., another GPU in an XGMI hive).
+ * Pages whose page->pgmap->owner matches the migration source are skipped
+ * (they're already in the right place).
+ *
+ * For XGMI hive: all GPUs in the hive share the same owner (the hive pointer)
+ * so intra-hive pages are treated as local.
+ * For standalone GPU: use the adev pointer itself as a unique per-device token.
+ */
+#define AMDGPU_PGMAP_OWNER(adev) \
+	((adev)->hive ? (void *)(adev)->hive : (void *)(adev))
+
+/**
+ * struct amdgpu_pagemap - VRAM migration infrastructure for drm_pagemap
+ * @dpagemap: DRM pagemap wrapper providing device_map/populate_mm callbacks
+ * @adev: back-pointer to the owning amdgpu_device
+ * @hpa_base: host physical address base of the ZONE_DEVICE region
+ * @initialized: set to true after successful registration
+ * @pgmap: the dev_pagemap registered with devm_memremap_pages();
+ *         must be last — contains a flexible-array member (ranges[])
+ *
+ * Allocated with devm_kzalloc() in amdgpu_svm_migration_init() and stored
+ * as adev->apagemap.  Lifetime is tied to the device via devres.
+ */
+struct amdgpu_pagemap {
+	struct drm_pagemap	dpagemap;
+	struct amdgpu_device	*adev;
+	resource_size_t		hpa_base;
+	bool			initialized;
+	struct dev_pagemap	pgmap;	/* must be last — flex-array */
+};
+
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
+int amdgpu_svm_migration_init(struct amdgpu_device *adev);
+#else
+static inline
+int amdgpu_svm_migration_init(struct amdgpu_device *adev)
+{
+	return 0;
+}
+#endif
+
+/**
+ * amdgpu_svm_drm_pagemap_ops - drm_pagemap_ops for AMDGPU VRAM migration
+ *
+ * Provides:
+ *   .device_map  - Convert ZONE_DEVICE page to VRAM address
+ *   .populate_mm - Allocate VRAM BO and migrate pages from system memory
+ */
+extern const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops;
+
+#endif /* __AMDGPU_MIGRATE_H__ */
-- 
2.34.1

[PATCH v4 2/6] drm/amdgpu: implement drm_pagemap SDMA migration callbacks

[Junhua Shen]

Implement the drm_pagemap_devmem_ops and drm_pagemap_ops callbacks
that the DRM GPUSVM migration framework requires.

Introduce struct amdgpu_bo_svm as a BO subtype (following the
amdgpu_bo_user/amdgpu_bo_vm pattern) that embeds struct amdgpu_bo
and carries a drm_pagemap_devmem allocation.

drm_pagemap_ops (top-level entry points):
  - device_map:   convert ZONE_DEVICE page to GPU PTE address
  - populate_mm:  allocate amdgpu_bo_svm and trigger migration

drm_pagemap_devmem_ops (per-BO migration mechanics):
  - populate_devmem_pfn: walk BO buddy blocks to build PFN array
  - copy_to_devmem:      SDMA copy system RAM -> VRAM via GART window
  - copy_to_ram:         SDMA copy VRAM -> system RAM via GART window
  - devmem_release:      drop BO ref (triggers destroy to free amdgpu_bo_svm)

Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c | 628 +++++++++++++++++++-
 1 file changed, 626 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
index a8b067831b99..54253d4dcc8d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
@@ -64,9 +64,19 @@
 #include <linux/memremap.h>
 #include <linux/migrate.h>
 
-#include "amdgpu_amdkfd.h"
 #include "amdgpu_migrate.h"
 #include "amdgpu.h"
+#include "amdgpu_ttm.h"
+#include "amdgpu_res_cursor.h"
+
+#define AMDGPU_MIGRATE_TRACE(fmt, ...) \
+	pr_debug("%s: " fmt, __func__, ##__VA_ARGS__)
+
+/* SDMA copy direction */
+#define FROM_RAM_TO_VRAM	0
+#define FROM_VRAM_TO_RAM	1
+
+static const struct drm_pagemap_devmem_ops amdgpu_pagemap_ops;
 
 static inline struct amdgpu_pagemap *
 dpagemap_to_apagemap(struct drm_pagemap *dpagemap)
@@ -94,8 +104,622 @@ amdgpu_svm_page_to_apagemap(struct page *page)
 	return container_of(pgmap, struct amdgpu_pagemap, pgmap);
 }
 
+/* drm_pagemap_devmem_ops — per-BO migration mechanics */
+
+/**
+ * struct amdgpu_bo_svm - SVM BO subtype with drm_pagemap devmem allocation
+ *
+ * @bo: Embedded base amdgpu_bo
+ * @devmem: drm_pagemap device memory allocation (passed to framework)
+ *
+ * Lifecycle is managed by the drm_pagemap framework's internal zdd refcount:
+ *   - zdd->devmem_allocation points to &svm_bo->devmem
+ *   - When zdd refcount drops to zero, framework calls devmem_release()
+ *   - devmem_release() drops the BO reference (triggering destroy callback)
+ */
+struct amdgpu_bo_svm {
+	struct amdgpu_bo bo;
+	struct drm_pagemap_devmem devmem;
+};
+
+#define to_amdgpu_bo_svm(abo) container_of((abo), struct amdgpu_bo_svm, bo)
+
+static inline struct amdgpu_bo_svm *
+devmem_to_amdgpu_bo_svm(struct drm_pagemap_devmem *devmem_allocation)
+{
+	return container_of(devmem_allocation, struct amdgpu_bo_svm, devmem);
+}
+
+/**
+ * amdgpu_bo_svm_destroy - TTM destroy callback for SVM BO
+ *
+ * Called when the last reference to the BO is dropped.
+ * Follows amdgpu_bo_user_destroy pattern: delegates to amdgpu_bo_base_destroy.
+ */
+static void amdgpu_bo_svm_destroy(struct ttm_buffer_object *tbo)
+{
+	struct amdgpu_bo *bo = ttm_to_amdgpu_bo(tbo);
+	struct amdgpu_bo_svm *svm_bo = to_amdgpu_bo_svm(bo);
+
+	amdgpu_bo_kunmap(bo);
+	drm_gem_object_release(&bo->tbo.base);
+	amdgpu_bo_unref(&bo->parent);
+	kvfree(svm_bo);
+}
+
+/**
+ * amdgpu_bo_svm_alloc - Allocate an amdgpu_bo_svm with VRAM backing
+ * @adev: AMDGPU device
+ * @dpagemap: The drm_pagemap for this device
+ * @mm: mm_struct of the owning process
+ * @size: Allocation size in bytes
+ *
+ * Uses bp->bo_ptr_size = sizeof(struct amdgpu_bo_svm) so that
+ * amdgpu_bo_create() allocates the full amdgpu_bo_svm structure,
+ * with the embedded amdgpu_bo as the base.
+ *
+ * Return: Pointer to allocated amdgpu_bo_svm on success, ERR_PTR on failure
+ */
+static struct amdgpu_bo_svm *
+amdgpu_bo_svm_alloc(struct amdgpu_device *adev,
+		     struct drm_pagemap *dpagemap,
+		     struct mm_struct *mm, unsigned long size)
+{
+	struct amdgpu_bo_param bp = {};
+	struct amdgpu_bo_svm *svm_bo;
+	struct amdgpu_bo *bo;
+	int ret;
+
+	bp.size = size;
+	bp.bo_ptr_size = sizeof(struct amdgpu_bo_svm);
+	bp.destroy = &amdgpu_bo_svm_destroy;
+	bp.domain = AMDGPU_GEM_DOMAIN_VRAM;
+	bp.type = ttm_bo_type_device;
+	bp.flags = AMDGPU_GEM_CREATE_NO_CPU_ACCESS |
+		   AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS |
+		   AMDGPU_GEM_CREATE_VRAM_CLEARED;
+
+	ret = amdgpu_bo_create(adev, &bp, &bo);
+	if (ret) {
+		AMDGPU_MIGRATE_TRACE("Failed to create SVM BO\n");
+		return ERR_PTR(ret);
+	}
+
+	svm_bo = to_amdgpu_bo_svm(bo);
+
+	drm_pagemap_devmem_init(&svm_bo->devmem,
+				adev->dev, mm,
+				&amdgpu_pagemap_ops,
+				dpagemap, size, NULL);
+
+	return svm_bo;
+}
+
+/**
+ * amdgpu_svm_devmem_release - Release BO when all device pages migrate back
+ *
+ * Called by the drm_pagemap framework (via drm_pagemap_zdd_destroy) when the
+ * last device-private page backed by this allocation has been migrated back
+ * to system memory (or the owning process exits).
+ *
+ * Drops the BO reference, which triggers the destroy callback to free
+ * the amdgpu_bo_svm structure.
+ */
+static void
+amdgpu_svm_devmem_release(struct drm_pagemap_devmem *devmem_allocation)
+{
+	struct amdgpu_bo_svm *svm_bo = devmem_to_amdgpu_bo_svm(devmem_allocation);
+	struct amdgpu_bo *bo = &svm_bo->bo;
+
+	AMDGPU_MIGRATE_TRACE("Release svm_bo=%px bo=%px\n", svm_bo, bo);
+	amdgpu_bo_unref(&bo);
+}
+
+/**
+ * amdgpu_svm_populate_devmem_pfn - Convert BO VRAM allocation to PFN array
+ * @devmem_allocation: The devmem allocation in the amdgpu_bo_svm wrapper
+ * @npages: Number of PFN entries to fill
+ * @pfn: Output PFN array
+ *
+ * Iterates over the BO's TTM vram_mgr buddy blocks and converts each
+ * block's VRAM offset to ZONE_DEVICE PFNs:
+ *
+ *   PFN = PHYS_PFN(block_offset + apagemap.hpa_base) + page_index
+ *
+ * This is called by drm_pagemap_migrate_to_devmem() to build the
+ * destination PFN array for migrate_vma_pages().
+ *
+ * Return: 0 on success
+ */
+static int
+amdgpu_svm_populate_devmem_pfn(struct drm_pagemap_devmem *devmem_allocation,
+				unsigned long npages, unsigned long *pfn)
+{
+	struct amdgpu_pagemap *svm_dm = dpagemap_to_apagemap(devmem_allocation->dpagemap);
+	struct amdgpu_bo_svm *svm_bo = devmem_to_amdgpu_bo_svm(devmem_allocation);
+	struct amdgpu_bo *bo = &svm_bo->bo;
+	struct amdgpu_res_cursor cursor;
+	unsigned long i = 0;
+
+	dma_resv_assert_held(bo->tbo.base.resv);
+
+	amdgpu_res_first(bo->tbo.resource, 0, npages << PAGE_SHIFT, &cursor);
+	while (cursor.remaining && i < npages) {
+		u64 pfn_base = PHYS_PFN(cursor.start + svm_dm->hpa_base);
+		u64 pages = cursor.size >> PAGE_SHIFT;
+		unsigned long j;
+
+		for (j = 0; j < pages && i < npages; j++, i++)
+			pfn[i] = pfn_base + j;
+
+		amdgpu_res_next(&cursor, cursor.size);
+	}
+
+	AMDGPU_MIGRATE_TRACE("populate_devmem_pfn: npages=%lu first_pfn=0x%lx\n",
+			  npages, npages > 0 ? pfn[0] : 0);
+
+	return 0;
+}
+
+/* SDMA copy helpers — GART window based data transfer */
+
+/**
+ * amdgpu_svm_direct_mapping_addr - Convert VRAM offset to MC address
+ * @adev: AMDGPU device
+ * @vram_offset: Byte offset within VRAM
+ *
+ * Return: MC address suitable for SDMA src/dst
+ */
+static u64
+amdgpu_svm_direct_mapping_addr(struct amdgpu_device *adev, u64 vram_offset)
+{
+	return vram_offset + amdgpu_ttm_domain_start(adev, TTM_PL_VRAM);
+}
+
+/**
+ * amdgpu_svm_gart_map - Map system DMA addresses into GART window
+ * @ring: SDMA ring for the GART update job
+ * @npages: Number of pages to map
+ * @addr: Array of system memory DMA addresses
+ * @gart_addr: Output — GART base address to use in SDMA copy
+ * @flags: PTE flags (e.g. writeable for RAM-to-VRAM src)
+ *
+ * Builds GART PTEs pointing at the given DMA addresses, submits an
+ * SDMA job to update the GART entries, and returns the GART address
+ * that can be used as src or dst in a subsequent amdgpu_copy_buffer().
+ *
+ * Uses GART window 0, protected by gtt_window_lock.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+amdgpu_svm_gart_map(struct amdgpu_ring *ring,
+		     struct amdgpu_ttm_buffer_entity *entity,
+		     u64 npages,
+		     dma_addr_t *addr, u64 *gart_addr, u64 flags)
+{
+	struct amdgpu_device *adev = ring->adev;
+	struct amdgpu_job *job;
+	unsigned int num_dw, num_bytes;
+	struct dma_fence *fence;
+	u64 src_addr, dst_addr;
+	u64 pte_flags;
+	void *cpu_addr;
+	int r;
+
+	/* Use entity's GART window 0 */
+	*gart_addr = amdgpu_compute_gart_address(&adev->gmc, entity, 0);
+
+	num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8);
+	num_bytes = npages * 8 * AMDGPU_GPU_PAGES_IN_CPU_PAGE;
+
+	r = amdgpu_job_alloc_with_ib(adev, &entity->base,
+				     AMDGPU_FENCE_OWNER_UNDEFINED,
+				     num_dw * 4 + num_bytes,
+				     AMDGPU_IB_POOL_DELAYED,
+				     &job,
+				     AMDGPU_KERNEL_JOB_ID_KFD_GART_MAP);
+	if (r)
+		return r;
+
+	src_addr = num_dw * 4;
+	src_addr += job->ibs[0].gpu_addr;
+
+	dst_addr = amdgpu_bo_gpu_offset(adev->gart.bo);
+	dst_addr += (entity->gart_window_offs[0] >> AMDGPU_GPU_PAGE_SHIFT) * 8;
+	amdgpu_emit_copy_buffer(adev, &job->ibs[0], src_addr,
+				dst_addr, num_bytes, 0);
+
+	amdgpu_ring_pad_ib(ring, &job->ibs[0]);
+	WARN_ON(job->ibs[0].length_dw > num_dw);
+
+	pte_flags = AMDGPU_PTE_VALID | AMDGPU_PTE_READABLE;
+	pte_flags |= AMDGPU_PTE_SYSTEM | AMDGPU_PTE_SNOOPED;
+	if (flags & AMDGPU_PTE_WRITEABLE)
+		pte_flags |= AMDGPU_PTE_WRITEABLE;
+	pte_flags |= adev->gart.gart_pte_flags;
+
+	cpu_addr = &job->ibs[0].ptr[num_dw];
+
+	amdgpu_gart_map(adev, 0, npages, addr, pte_flags, cpu_addr);
+	fence = amdgpu_job_submit(job);
+	dma_fence_put(fence);
+
+	return 0;
+}
+
+/**
+ * amdgpu_svm_copy_memory_gart - SDMA copy between system RAM and VRAM
+ * @adev: AMDGPU device
+ * @sys: Array of DMA addresses for system memory pages
+ * @vram: Array of VRAM byte offsets (relative to start of VRAM)
+ * @npages: Number of pages to copy
+ * @direction: FROM_RAM_TO_VRAM or FROM_VRAM_TO_RAM
+ * @mfence: In/out — carries the last SDMA fence for serialization
+ *
+ * Maps system memory pages into the GART window and uses SDMA to copy
+ * data to/from VRAM. Handles splitting into AMDGPU_GTT_MAX_TRANSFER_SIZE
+ * chunks. Acquires entity->lock internally to protect the GART window,
+ * matching the KFD svm_migrate_copy_memory_gart() pattern.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+amdgpu_svm_copy_memory_gart(struct amdgpu_device *adev, dma_addr_t *sys,
+			    u64 *vram, u64 npages, int direction,
+			    struct dma_fence **mfence)
+{
+	const u64 max_pages = AMDGPU_GTT_MAX_TRANSFER_SIZE;
+	struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;
+	struct amdgpu_ttm_buffer_entity *entity = &adev->mman.move_entity;
+	u64 gart_s, gart_d;
+	struct dma_fence *next;
+	u64 size;
+	int r;
+
+	mutex_lock(&entity->lock);
+
+	while (npages) {
+		size = min(max_pages, npages);
+
+		if (direction == FROM_VRAM_TO_RAM) {
+			gart_s = amdgpu_svm_direct_mapping_addr(adev, *vram);
+			r = amdgpu_svm_gart_map(ring, entity, size, sys,
+						&gart_d, AMDGPU_PTE_WRITEABLE);
+		} else {
+			r = amdgpu_svm_gart_map(ring, entity, size, sys,
+						&gart_s, 0);
+			gart_d = amdgpu_svm_direct_mapping_addr(adev, *vram);
+		}
+		if (r) {
+			dev_err(adev->dev, "failed %d to map GART for SDMA\n", r);
+			goto out_unlock;
+		}
+
+		AMDGPU_MIGRATE_TRACE("SDMA_COPY: %s npages=%llu vram_off=0x%llx\n",
+				  direction == FROM_RAM_TO_VRAM ? "RAM->VRAM" : "VRAM->RAM",
+				  size, (u64)*vram);
+
+		r = amdgpu_copy_buffer(adev, entity, gart_s, gart_d,
+				       size * PAGE_SIZE,
+				       NULL, &next, true, 0);
+		if (r) {
+			dev_err(adev->dev, "failed %d to copy buffer\n", r);
+			goto out_unlock;
+		}
+
+		dma_fence_put(*mfence);
+		*mfence = next;
+		npages -= size;
+		if (npages) {
+			sys += size;
+			vram += size;
+		}
+	}
+
+out_unlock:
+	mutex_unlock(&entity->lock);
+
+	return r;
+}
+
+/**
+ * amdgpu_svm_copy_to_devmem - SDMA copy system memory -> VRAM
+ * @pages: Array of destination ZONE_DEVICE pages (VRAM-backed)
+ * @pagemap_addr: Array of source DMA addresses (system memory, already mapped)
+ * @npages: Number of pages to copy
+ *
+ * Builds parallel sys[] and vram[] arrays from the framework-provided
+ * pagemap_addr and device pages, then submits batched SDMA copies via
+ * the GART window.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+amdgpu_svm_copy_to_devmem(struct page **pages,
+			   struct drm_pagemap_addr *pagemap_addr,
+			   unsigned long npages,
+			   struct dma_fence *pre_migrate_fence)
+{
+	struct amdgpu_device *adev;
+	struct amdgpu_pagemap *svm_dm;
+	struct dma_fence *mfence = NULL;
+	dma_addr_t *sys;
+	u64 *vram;
+	unsigned long i, j;
+	int ret = 0;
+
+	if (!npages)
+		return 0;
+
+	/*
+	 * Find the first non-NULL page to derive the device.
+	 * The pages array may contain NULL entries for positions where
+	 * no valid device page exists.
+	 */
+	for (i = 0; i < npages; i++) {
+		if (pages[i])
+			break;
+	}
+	if (i == npages)
+		return 0;
+
+	svm_dm = amdgpu_svm_page_to_apagemap(pages[i]);
+	adev = svm_dm->adev;
+
+	sys = kvcalloc(npages, sizeof(*sys), GFP_KERNEL);
+	vram = kvcalloc(npages, sizeof(*vram), GFP_KERNEL);
+	if (!sys || !vram) {
+		ret = -ENOMEM;
+		goto out_free;
+	}
+
+	for (i = 0, j = 0; i < npages; i++) {
+		if (!pagemap_addr[i].addr)
+			goto flush;
+
+		sys[j] = pagemap_addr[i].addr;
+		vram[j] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
+			  svm_dm->hpa_base;
+
+		/* Check if next vram page is contiguous with current */
+		if (j > 0 && vram[j] != vram[j - 1] + PAGE_SIZE)
+			goto flush;
+
+		j++;
+		continue;
+flush:
+		if (j) {
+			ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
+							  FROM_RAM_TO_VRAM,
+							  &mfence);
+			if (ret)
+				goto out_fence;
+			j = 0;
+		}
+		/* Re-process current page if it was valid but broke contiguity */
+		if (pagemap_addr[i].addr) {
+			sys[0] = pagemap_addr[i].addr;
+			vram[0] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
+				  svm_dm->hpa_base;
+			j = 1;
+		}
+	}
+
+	/* Flush remaining batch */
+	if (j)
+		ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
+						  FROM_RAM_TO_VRAM, &mfence);
+
+out_fence:
+	if (mfence) {
+		dma_fence_wait(mfence, false);
+		dma_fence_put(mfence);
+	}
+
+	AMDGPU_MIGRATE_TRACE("copy_to_devmem done: npages=%ld ret=%d\n",
+			  npages, ret);
+
+out_free:
+	kvfree(vram);
+	kvfree(sys);
+	return ret;
+}
+
+/**
+ * amdgpu_svm_copy_to_ram - SDMA copy VRAM -> system memory
+ * @pages: Array of source ZONE_DEVICE pages (VRAM-backed)
+ * @pagemap_addr: Array of destination DMA addresses (system memory, already mapped)
+ * @npages: Number of pages to copy
+ *
+ * Mirror of copy_to_devmem with src/dst swapped.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+amdgpu_svm_copy_to_ram(struct page **pages,
+			struct drm_pagemap_addr *pagemap_addr,
+			unsigned long npages,
+			struct dma_fence *pre_migrate_fence)
+{
+	struct amdgpu_device *adev;
+	struct amdgpu_pagemap *svm_dm;
+	struct dma_fence *mfence = NULL;
+	dma_addr_t *sys;
+	u64 *vram;
+	unsigned long i, j;
+	int ret = 0;
+
+	if (!npages)
+		return 0;
+
+	for (i = 0; i < npages; i++) {
+		if (pages[i])
+			break;
+	}
+	if (i == npages)
+		return 0;
+
+	svm_dm = amdgpu_svm_page_to_apagemap(pages[i]);
+	adev = svm_dm->adev;
+
+	sys = kvcalloc(npages, sizeof(*sys), GFP_KERNEL);
+	vram = kvcalloc(npages, sizeof(*vram), GFP_KERNEL);
+	if (!sys || !vram) {
+		ret = -ENOMEM;
+		goto out_free;
+	}
+
+	for (i = 0, j = 0; i < npages; i++) {
+		if (!pagemap_addr[i].addr || !pages[i])
+			goto flush;
+
+		vram[j] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
+			  svm_dm->hpa_base;
+		sys[j] = pagemap_addr[i].addr;
+
+		/* Check if next vram page is contiguous with current */
+		if (j > 0 && vram[j] != vram[j - 1] + PAGE_SIZE)
+			goto flush;
+
+		j++;
+		continue;
+flush:
+		if (j) {
+			ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
+							  FROM_VRAM_TO_RAM,
+							  &mfence);
+			if (ret)
+				goto out_fence;
+			j = 0;
+		}
+		/* Re-process current page if it was valid but broke contiguity */
+		if (pagemap_addr[i].addr && pages[i]) {
+			vram[0] = ((u64)page_to_pfn(pages[i]) << PAGE_SHIFT) -
+				  svm_dm->hpa_base;
+			sys[0] = pagemap_addr[i].addr;
+			j = 1;
+		}
+	}
+
+	/* Flush remaining batch */
+	if (j)
+		ret = amdgpu_svm_copy_memory_gart(adev, sys, vram, j,
+						  FROM_VRAM_TO_RAM, &mfence);
+
+out_fence:
+	if (mfence) {
+		dma_fence_wait(mfence, false);
+		dma_fence_put(mfence);
+	}
+
+	AMDGPU_MIGRATE_TRACE("copy_to_ram done: npages=%ld ret=%d\n", npages, ret);
+
+out_free:
+	kvfree(vram);
+	kvfree(sys);
+	return ret;
+}
+
+static const struct drm_pagemap_devmem_ops amdgpu_pagemap_ops = {
+	.devmem_release      = amdgpu_svm_devmem_release,
+	.populate_devmem_pfn = amdgpu_svm_populate_devmem_pfn,
+	.copy_to_devmem      = amdgpu_svm_copy_to_devmem,
+	.copy_to_ram         = amdgpu_svm_copy_to_ram,
+};
+
+/* drm_pagemap_ops — top-level migration entry points */
+
+/**
+ * amdgpu_svm_device_map - Convert ZONE_DEVICE page to GPU PTE address
+ * @dpagemap: The drm_pagemap for this device
+ * @dev: Requesting device (for P2P check)
+ * @page: ZONE_DEVICE page backed by VRAM
+ * @order: Page order (0 = 4K, 9 = 2M, etc.)
+ * @dir: DMA direction (unused for local VRAM)
+ *
+ * Address conversion chain:
+ *   page -> PFN -> HPA -> VRAM offset -> PTE address
+ *
+ *   HPA = page_to_pfn(page) << PAGE_SHIFT
+ *   VRAM offset = HPA - apagemap.hpa_base
+ *   PTE address = VRAM offset + adev->vm_manager.vram_base_offset
+ *
+ * Return: drm_pagemap_addr with PTE address and AMDGPU_INTERCONNECT_VRAM protocol
+ */
+static struct drm_pagemap_addr
+amdgpu_svm_device_map(struct drm_pagemap *dpagemap,
+		       struct device *dev,
+		       struct page *page,
+		       unsigned int order,
+		       enum dma_data_direction dir)
+{
+	struct amdgpu_pagemap *svm_dm = dpagemap_to_apagemap(dpagemap);
+	struct amdgpu_device *adev = dpagemap_to_adev(dpagemap);
+	dma_addr_t addr;
+
+	if (dpagemap->drm->dev == dev) {
+		/* Same device: return VRAM PTE address */
+		u64 hpa = (u64)page_to_pfn(page) << PAGE_SHIFT;
+		u64 vram_offset = hpa - svm_dm->hpa_base;
+
+		addr = vram_offset + adev->vm_manager.vram_base_offset;
+	} else {
+		/* Cross-device P2P: not yet supported */
+		addr = DMA_MAPPING_ERROR;
+	}
+
+	return drm_pagemap_addr_encode(addr,
+				AMDGPU_INTERCONNECT_VRAM, order, dir);
+}
+
+/**
+ * amdgpu_svm_populate_mm - Allocate VRAM BO and migrate pages
+ * @dpagemap: The drm_pagemap for this device
+ * @start: Start virtual address of the range to migrate
+ * @end: End virtual address (exclusive)
+ * @mm: mm_struct of the owning process
+ * @timeslice_ms: Maximum time to spend migrating (for fairness)
+ *
+ * Core migration entry point called by drm_pagemap_populate_mm().
+ * Allocates an amdgpu_bo_svm via amdgpu_bo_svm_alloc(), then calls
+ * drm_pagemap_migrate_to_devmem() to execute the actual migration.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+amdgpu_svm_populate_mm(struct drm_pagemap *dpagemap,
+			unsigned long start, unsigned long end,
+			struct mm_struct *mm,
+			unsigned long timeslice_ms)
+{
+	struct amdgpu_device *adev = dpagemap_to_adev(dpagemap);
+	struct drm_pagemap_migrate_details mdetails = {
+		.timeslice_ms = timeslice_ms,
+	};
+	struct amdgpu_bo_svm *svm_bo;
+	int ret;
+
+	svm_bo = amdgpu_bo_svm_alloc(adev, dpagemap, mm, end - start);
+	if (IS_ERR(svm_bo))
+		return PTR_ERR(svm_bo);
+
+	AMDGPU_MIGRATE_TRACE("populate_mm: [0x%lx-0x%lx] size=%lu\n",
+			  start, end, end - start);
+
+	ret = drm_pagemap_migrate_to_devmem(&svm_bo->devmem,
+					     mm, start, end,
+					     &mdetails);
+
+	amdgpu_bo_unreserve(&svm_bo->bo);
+
+	return ret;
+}
 
-const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops = { };
+const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops = {
+	.device_map = amdgpu_svm_device_map,
+	.populate_mm = amdgpu_svm_populate_mm,
+};
 
 /**
  * amdgpu_svm_migration_init - Register ZONE_DEVICE and initialize drm_pagemap
-- 
2.34.1

[PATCH v4 3/6] drm/amdgpu: implement synchronous TTM eviction for SVM BOs

[Junhua Shen]

Implement the TTM eviction path for SVM (Shared Virtual Memory) BOs,
enabling VRAM overcommit scenarios where device-private pages must be
migrated back to system memory when VRAM is full.

- amdgpu_ttm.c: Add SVM BO detection in amdgpu_evict_flags() to evict
  directly to SYSTEM domain. Add SVM BO handling in amdgpu_bo_move()
  to call amdgpu_svm_bo_evict() for VRAM->SYSTEM transitions, which
  synchronously migrates device-private pages back to RAM via
  SDMA.

- amdgpu_migrate.c: Add amdgpu_svm_bo_evict() and
  amdgpu_svm_bo_is_svm() helpers.

- amdgpu_object.c: Register SVM BO destroy callback in
  amdgpu_bo_is_amdgpu_bo() so TTM eviction can identify and handle
  SVM BOs correctly.

Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c | 28 +++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h | 12 +++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c  |  4 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c     | 20 +++++++++++++++
 4 files changed, 63 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
index 54253d4dcc8d..ff45489c458e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c
@@ -628,6 +628,34 @@ static const struct drm_pagemap_devmem_ops amdgpu_pagemap_ops = {
 	.copy_to_ram         = amdgpu_svm_copy_to_ram,
 };
 
+/**
+ * amdgpu_svm_bo_evict - Evict SVM BO by migrating device pages back to RAM
+ * @bo: The amdgpu_bo to evict (must be an amdgpu_bo_svm)
+ *
+ * Called from amdgpu_bo_move() when TTM needs to evict a SVM BO from VRAM.
+ * Uses drm_pagemap_evict_to_ram() to synchronously migrate all device-private
+ * pages back to system memory via SDMA.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+int amdgpu_svm_bo_evict(struct amdgpu_bo *bo)
+{
+	struct amdgpu_bo_svm *sbo = to_amdgpu_bo_svm(bo);
+
+	return drm_pagemap_evict_to_ram(&sbo->devmem);
+}
+
+/**
+ * amdgpu_svm_bo_is_svm - Check if a TTM BO is an SVM BO
+ * @tbo: TTM buffer object
+ *
+ * Return: true if this is an amdgpu_bo_svm (identified by destroy callback)
+ */
+bool amdgpu_svm_bo_is_svm(struct ttm_buffer_object *tbo)
+{
+	return tbo->destroy == &amdgpu_bo_svm_destroy;
+}
+
 /* drm_pagemap_ops — top-level migration entry points */
 
 /**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h
index 75796983f6a5..63da96c17e27 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h
@@ -29,6 +29,8 @@
 #include <linux/memremap.h>
 
 struct amdgpu_device;
+struct amdgpu_bo;
+struct ttm_buffer_object;
 
 /*
  * AMDGPU_INTERCONNECT_VRAM - Protocol identifier for local VRAM access.
@@ -78,12 +80,22 @@ struct amdgpu_pagemap {
 
 #if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
 int amdgpu_svm_migration_init(struct amdgpu_device *adev);
+int amdgpu_svm_bo_evict(struct amdgpu_bo *bo);
+bool amdgpu_svm_bo_is_svm(struct ttm_buffer_object *tbo);
 #else
 static inline
 int amdgpu_svm_migration_init(struct amdgpu_device *adev)
 {
 	return 0;
 }
+static inline int amdgpu_svm_bo_evict(struct amdgpu_bo *bo)
+{
+	return -ENODEV;
+}
+static inline bool amdgpu_svm_bo_is_svm(struct ttm_buffer_object *tbo)
+{
+	return false;
+}
 #endif
 
 /**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
index 1fb956400696..78a677ff32bf 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
@@ -43,6 +43,7 @@
 #include "amdgpu_vram_mgr.h"
 #include "amdgpu_vm.h"
 #include "amdgpu_dma_buf.h"
+#include "amdgpu_migrate.h"
 
 /**
  * DOC: amdgpu_object
@@ -93,7 +94,8 @@ static void amdgpu_bo_user_destroy(struct ttm_buffer_object *tbo)
 bool amdgpu_bo_is_amdgpu_bo(struct ttm_buffer_object *bo)
 {
 	if (bo->destroy == &amdgpu_bo_destroy ||
-	    bo->destroy == &amdgpu_bo_user_destroy)
+	    bo->destroy == &amdgpu_bo_user_destroy ||
+	    amdgpu_svm_bo_is_svm(bo))
 		return true;
 
 	return false;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index 714fd8d12ca5..5a45a29ef051 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -59,6 +59,7 @@
 #include "amdgpu_hmm.h"
 #include "amdgpu_atomfirmware.h"
 #include "amdgpu_res_cursor.h"
+#include "amdgpu_migrate.h"
 #include "bif/bif_4_1_d.h"
 
 MODULE_IMPORT_NS("DMA_BUF");
@@ -118,6 +119,15 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo,
 		return;
 	}
 
+	/* SVM BOs must evict directly to system memory for drm_pagemap
+	 * migration back to RAM in amdgpu_bo_move().
+	 */
+	if (amdgpu_svm_bo_is_svm(bo)) {
+		amdgpu_bo_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_CPU);
+		*placement = abo->placement;
+		return;
+	}
+
 	switch (bo->resource->mem_type) {
 	case AMDGPU_PL_GDS:
 	case AMDGPU_PL_GWS:
@@ -568,6 +578,16 @@ static int amdgpu_bo_move(struct ttm_buffer_object *bo, bool evict,
 		abo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
 	}
 
+	/* SVM BO eviction: migrate device-private pages back to RAM */
+	if (amdgpu_svm_bo_is_svm(bo) &&
+	    old_mem->mem_type == TTM_PL_VRAM &&
+	    new_mem->mem_type == TTM_PL_SYSTEM) {
+		r = amdgpu_svm_bo_evict(abo);
+		if (!r)
+			ttm_bo_move_null(bo, new_mem);
+		return r;
+	}
+
 	if (adev->mman.buffer_funcs_enabled &&
 	    ((old_mem->mem_type == TTM_PL_SYSTEM &&
 	      new_mem->mem_type == TTM_PL_VRAM) ||
-- 
2.34.1

[PATCH v4 4/6] drm/amdgpu: add SVM range migration helpers for drm_pagemap

[Junhua Shen]

Add amdgpu_svm_range_migrate.c/.h implementing the per-range migration
helpers for SVM VRAM migration via drm_pagemap:

- Define amdgpu_svm_migrate_mode enum (TO_VRAM, TO_SYSMEM) to
  express migration intent from callers.
- Implement range_needs_migrate_to_vram() checking migrate_devmem
  capability and current backing location.
- Add amdgpu_svm_range_migrate_to_vram() wrapping
  drm_pagemap_populate_mm() for RAM-to-VRAM migration.
- Add amdgpu_svm_range_migrate_range() entry point dispatching to
  migrate_to_vram or drm_gpusvm_range_evict() based on mode.
- Add amdgpu_pagemap_capable() to check device memory support.

Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/Makefile           |   6 +-
 .../drm/amd/amdgpu/amdgpu_svm_range_migrate.c | 122 ++++++++++++++++++
 .../drm/amd/amdgpu/amdgpu_svm_range_migrate.h |  47 +++++++
 3 files changed, 172 insertions(+), 3 deletions(-)
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h

diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile b/drivers/gpu/drm/amd/amdgpu/Makefile
index 3905a97795aa..f34d93389da3 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -323,14 +323,14 @@ amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_hmm.o
 
 # svm support
 amdgpu-$(CONFIG_DRM_AMDGPU_SVM) += amdgpu_svm.o amdgpu_svm_attr.o \
-	amdgpu_svm_fault.o amdgpu_svm_range.o amdgpu_migrate.o
+	amdgpu_svm_fault.o amdgpu_svm_range.o amdgpu_svm_range_migrate.o amdgpu_migrate.o
 
 .PHONY: clean-svm
 clean-svm:
 	rm -f $(obj)/amdgpu_svm.o $(obj)/amdgpu_svm_attr.o $(obj)/amdgpu_svm_fault.o $(obj)/amdgpu_svm_range.o \
-	      $(obj)/amdgpu_migrate.o \
+	      $(obj)/amdgpu_svm_range_migrate.o $(obj)/amdgpu_migrate.o \
 	      $(obj)/.amdgpu_svm.o.cmd $(obj)/.amdgpu_svm_attr.o.cmd $(obj)/.amdgpu_svm_fault.o.cmd $(obj)/.amdgpu_svm_range.o.cmd \
-	      $(obj)/.amdgpu_migrate.o.cmd
+	      $(obj)/.amdgpu_svm_range_migrate.o.cmd $(obj)/.amdgpu_migrate.o.cmd
 
 include $(FULL_AMD_PATH)/pm/Makefile
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c
new file mode 100644
index 000000000000..bc6f242b680b
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.c
@@ -0,0 +1,122 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright 2026 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "amdgpu.h"
+#include "amdgpu_svm.h"
+#include "amdgpu_svm_range.h"
+#include "amdgpu_migrate.h"
+#include "amdgpu_svm_range_migrate.h"
+
+static bool
+range_in_vram(struct drm_gpusvm_range *range)
+{
+	struct drm_gpusvm_pages_flags flags = {
+		/* Pairs with WRITE_ONCE in drm_gpusvm_get_pages() */
+		.__flags = READ_ONCE(range->pages.flags.__flags),
+	};
+
+	return flags.has_devmem_pages;
+}
+
+static struct drm_pagemap *
+amdgpu_svm_get_dpagemap(struct amdgpu_svm *svm)
+{
+	struct amdgpu_pagemap *apagemap = svm->adev->apagemap;
+
+	if (!apagemap->initialized)
+		return NULL;
+
+	return &apagemap->dpagemap;
+}
+
+static bool
+range_needs_migrate_to_vram(struct drm_gpusvm_range *range,
+			    enum amdgpu_svm_migrate_mode mode)
+{
+	if (!range->pages.flags.migrate_devmem)
+		return false;
+	if (range_in_vram(range))
+		return false;
+
+	if (mode == AMDGPU_SVM_MIGRATE_TO_VRAM)
+		return true;
+
+	return false;
+}
+
+static int
+amdgpu_svm_range_migrate_to_vram(struct amdgpu_svm *svm,
+				  struct drm_gpusvm_range *range)
+{
+	struct drm_pagemap *dpagemap = amdgpu_svm_get_dpagemap(svm);
+	unsigned long start = drm_gpusvm_range_start(range);
+	unsigned long end = drm_gpusvm_range_end(range);
+	int ret;
+
+	if (!dpagemap)
+		return -ENODEV;
+
+	ret = drm_pagemap_populate_mm(dpagemap, start, end,
+				      svm->gpusvm.mm, 0);
+
+	if (ret) {
+		AMDGPU_SVM_TRACE("migrate_to_vram failed: ret=%d [0x%lx-0x%lx]\n",
+				 ret, start, end);
+		return ret;
+	}
+
+	return 0;
+}
+
+bool
+amdgpu_pagemap_capable(struct amdgpu_svm *svm)
+{
+	if (svm->adev->gmc.is_app_apu)
+		return false;
+
+	if (!amdgpu_svm_get_dpagemap(svm))
+		return false;
+
+	return true;
+}
+
+/**
+ * amdgpu_svm_range_migrate_range - Per-range migration
+ * @svm: Pointer to the AMDGPU SVM structure
+ * @range: The GPU SVM range to migrate
+ * @migrate_mode: Migration intent
+ */
+int
+amdgpu_svm_range_migrate_range(struct amdgpu_svm *svm,
+			       struct drm_gpusvm_range *range,
+			       enum amdgpu_svm_migrate_mode migrate_mode)
+{
+	if (range_needs_migrate_to_vram(range, migrate_mode))
+		return amdgpu_svm_range_migrate_to_vram(svm, range);
+	else if (migrate_mode == AMDGPU_SVM_MIGRATE_TO_SYSMEM &&
+		 range_in_vram(range))
+		return drm_gpusvm_range_evict(&svm->gpusvm, range);
+
+	return 0;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h
new file mode 100644
index 000000000000..9cc80b8350c7
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range_migrate.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/*
+ * Copyright 2026 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __AMDGPU_SVM_RANGE_MIGRATE_H__
+#define __AMDGPU_SVM_RANGE_MIGRATE_H__
+
+struct amdgpu_svm;
+struct drm_gpusvm_range;
+
+/**
+ * enum amdgpu_svm_migrate_mode - Migration mode for SVM range migration
+ *
+ * @AMDGPU_SVM_MIGRATE_TO_VRAM: Migrate pages to VRAM before mapping.
+ * @AMDGPU_SVM_MIGRATE_TO_SYSMEM: Evict VRAM pages back to system memory
+ */
+enum amdgpu_svm_migrate_mode {
+	AMDGPU_SVM_MIGRATE_TO_VRAM,
+	AMDGPU_SVM_MIGRATE_TO_SYSMEM,
+};
+
+bool amdgpu_pagemap_capable(struct amdgpu_svm *svm);
+int amdgpu_svm_range_migrate_range(struct amdgpu_svm *svm,
+				    struct drm_gpusvm_range *range,
+				    enum amdgpu_svm_migrate_mode migrate_mode);
+
+#endif /* __AMDGPU_SVM_RANGE_MIGRATE_H__ */
-- 
2.34.1

[PATCH v4 5/6] drm/amdgpu: hook up ZONE_DEVICE registration in device init and reset

[Junhua Shen]

Call amdgpu_svm_migration_init() in the device initialization and
XGMI reset-restore paths to register the GPU's VRAM as a ZONE_DEVICE
region before KFD initialization.

This activates the drm_pagemap migration infrastructure.

Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 ++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c  | 4 ++++
 2 files changed, 8 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index fbe553c38583..3be51a2c0106 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -78,6 +78,7 @@
 #include "amdgpu_reset.h"
 #include "amdgpu_virt.h"
 #include "amdgpu_dev_coredump.h"
+#include "amdgpu_migrate.h"
 
 #include <linux/suspend.h>
 #include <drm/task_barrier.h>
@@ -4076,6 +4077,9 @@ int amdgpu_device_init(struct amdgpu_device *adev,
 
 	/* Don't init kfd if whole hive need to be reset during init */
 	if (adev->init_lvl->level != AMDGPU_INIT_LEVEL_MINIMAL_XGMI) {
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
+		amdgpu_svm_migration_init(adev);
+#endif
 		kgd2kfd_init_zone_device(adev);
 		kfd_update_svm_support_properties(adev);
 	}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
index 28c4ad62f50e..c94d43f3ab42 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
@@ -25,6 +25,7 @@
 #include "aldebaran.h"
 #include "sienna_cichlid.h"
 #include "smu_v13_0_10.h"
+#include "amdgpu_migrate.h"
 
 static int amdgpu_reset_xgmi_reset_on_init_suspend(struct amdgpu_device *adev)
 {
@@ -87,6 +88,9 @@ static int amdgpu_reset_xgmi_reset_on_init_restore_hwctxt(
 		return r;
 	list_for_each_entry(tmp_adev, reset_device_list, reset_list) {
 		if (!tmp_adev->kfd.init_complete) {
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
+			amdgpu_svm_migration_init(tmp_adev);
+#endif
 			kgd2kfd_init_zone_device(tmp_adev);
 			amdgpu_amdkfd_device_init(tmp_adev);
 			amdgpu_amdkfd_drm_client_create(tmp_adev);
-- 
2.34.1

[PATCH v4 6/6] drm/amdgpu: integrate VRAM migration into SVM range map and fault paths

[Junhua Shen]

Wire the migration layer into the SVM range map and GPU fault
call chains:

- Add amdgpu_pagemap_capable() guard in amdgpu_svm_attr_devmem_possible()
  to disable devmem when pagemap is not supported
- Add device_private_page_owner to drm_gpusvm_ctx in both prefetch
  and fault map contexts
- Call amdgpu_svm_range_migrate_range() before GPU mapping in both
  fault and prefetch paths to perform VRAM migration when preferred
- Support AMDGPU_INTERCONNECT_VRAM in amdgpu_svm_range_update_gpu_range()
  by clearing AMDGPU_PTE_SYSTEM and AMDGPU_PTE_SNOOPED flags for
  VRAM pages

Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c  |  4 ++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.c |  9 ++++++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c | 21 +++++++++++++++----
 3 files changed, 28 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c
index e50b67540c99..115bda12e625 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_attr.c
@@ -25,6 +25,7 @@
 #include "amdgpu_svm.h"
 #include "amdgpu_svm_attr.h"
 #include "amdgpu.h"
+#include "amdgpu_svm_range_migrate.h"
 
 #include <linux/err.h>
 #include <linux/errno.h>
@@ -55,6 +56,9 @@ struct attr_get_ctx {
 bool amdgpu_svm_attr_devmem_possible(struct amdgpu_svm *svm,
 				     const struct amdgpu_svm_attrs *attrs)
 {
+	if (!amdgpu_pagemap_capable(svm))
+		return false;
+
 	if (svm->adev->apu_prefer_gtt)
 		return false;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.c
index 7763eb029eaa..b5d21a66a228 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.c
@@ -26,6 +26,8 @@
 #include "amdgpu_svm_attr.h"
 #include "amdgpu_svm_fault.h"
 #include "amdgpu_svm_range.h"
+#include "amdgpu_svm_range_migrate.h"
+#include "amdgpu_migrate.h"
 #include "amdgpu.h"
 #include "amdgpu_vm.h"
 #include "amdgpu_gmc.h"
@@ -153,13 +155,14 @@ static int amdgpu_svm_range_map_fault(struct amdgpu_svm *svm,
 	const struct amdgpu_svm_attrs *attrs = &attr_range->attrs;
 	bool devmem_possible = amdgpu_svm_attr_devmem_possible(svm, attrs);
 	bool need_vram_migration = amdgpu_svm_attr_prefer_vram(svm, attrs);
-	devmem_possible = false; /* TODO: add migration */
 	struct drm_gpusvm_ctx map_ctx = {
 		.read_only = !!(attrs->flags & AMDGPU_SVM_ATTR_BIT_GPU_RO),
 		.devmem_possible = devmem_possible,
 		.check_pages_threshold = devmem_possible ? SZ_64K : 0,
 		.devmem_only = need_vram_migration && devmem_possible,
 		.timeslice_ms = need_vram_migration && devmem_possible ? 5 : 0,
+		.device_private_page_owner = devmem_possible ?
+			AMDGPU_PGMAP_OWNER(svm->adev) : NULL,
 	};
 	struct amdgpu_svm_range *range;
 	ktime_t timestamp = ktime_get_boottime();
@@ -228,7 +231,9 @@ static int amdgpu_svm_range_map_fault(struct amdgpu_svm *svm,
 	}
 
 	AMDGPU_SVM_RANGE_DEBUG(range, "PAGE FAULT");
-	/* TODO: add migration*/
+	if (need_vram_migration)
+		amdgpu_svm_range_migrate_range(svm, &range->base,
+					      AMDGPU_SVM_MIGRATE_TO_VRAM);
 
 	AMDGPU_SVM_RANGE_DEBUG(range, "GET PAGES");
 	ret = amdgpu_svm_range_get_pages(svm, &range->base, &map_ctx);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c
index fe543a16b399..b77f3a52f3ae 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c
@@ -26,6 +26,8 @@
 #include "amdgpu_svm_attr.h"
 #include "amdgpu_svm_range.h"
 #include "amdgpu_svm_fault.h"
+#include "amdgpu_svm_range_migrate.h"
+#include "amdgpu_migrate.h"
 #include "amdgpu.h"
 #include "amdgpu_vm.h"
 
@@ -244,9 +246,11 @@ amdgpu_svm_range_update_gpu_range(struct amdgpu_svm *svm,
 		unsigned long seg_pages = min_t(unsigned long, 1UL << entry->order,
 						npages - mapped_pages);
 		unsigned long start_page, last_page;
+		uint64_t seg_pte_flags = pte_flags;
 		bool is_last_seg;
 
-		if (entry->proto != DRM_INTERCONNECT_SYSTEM)
+		if (entry->proto != DRM_INTERCONNECT_SYSTEM &&
+		    entry->proto != AMDGPU_INTERCONNECT_VRAM)
 			return -EOPNOTSUPP;
 
 		start_page = range_start_page + mapped_pages;
@@ -254,9 +258,13 @@ amdgpu_svm_range_update_gpu_range(struct amdgpu_svm *svm,
 		mapped_pages += seg_pages;
 		is_last_seg = mapped_pages == npages;
 
+		/* For VRAM pages, clear the SYSTEM and SNOOPED bits */
+		if (entry->proto == AMDGPU_INTERCONNECT_VRAM)
+			seg_pte_flags &= ~(AMDGPU_PTE_SYSTEM | AMDGPU_PTE_SNOOPED);
+
 		ret = amdgpu_vm_update_range(svm->adev, svm->vm, false, false,
 					     flush_tlb && is_last_seg, true, NULL,
-					     start_page, last_page, pte_flags,
+					     start_page, last_page, seg_pte_flags,
 					     0, entry->addr, NULL, NULL,
 					     wait_fence && is_last_seg ? fence : NULL);
 		if (ret)
@@ -365,12 +373,13 @@ amdgpu_svm_range_map_attrs(struct amdgpu_svm *svm,
 	int ret;
 	bool devmem_possible = amdgpu_svm_attr_devmem_possible(svm, attrs);
 	bool need_vram_migration = amdgpu_svm_attr_prefer_vram(svm, attrs);
-	devmem_possible = false; /* TODO: add migration */
 	struct drm_gpusvm_ctx map_ctx = {
 		.read_only = !!(attrs->flags & AMDGPU_SVM_ATTR_BIT_GPU_RO),
 		.devmem_possible = devmem_possible,
 		.devmem_only = need_vram_migration && devmem_possible,
 		.check_pages_threshold = devmem_possible ? SZ_64K : 0,
+		.device_private_page_owner = devmem_possible ?
+			AMDGPU_PGMAP_OWNER(svm->adev) : NULL,
 	};
 
 	while (addr < end) {
@@ -399,7 +408,11 @@ amdgpu_svm_range_map_attrs(struct amdgpu_svm *svm,
 			continue;
 		}
 
-		/* TODO: add migration */
+		if (need_vram_migration) {
+			AMDGPU_SVM_RANGE_DEBUG(range, "PREFETCH - MIGRATION PAGES");
+			amdgpu_svm_range_migrate_range(svm, &range->base,
+						      AMDGPU_SVM_MIGRATE_TO_VRAM);
+		}
 
 		AMDGPU_SVM_RANGE_DEBUG(range, "PREFETCH - GET PAGES");
 
-- 
2.34.1

Re: [PATCH v4 5/6] drm/amdgpu: hook up ZONE_DEVICE registration in device init and reset

[Christian König]

On 5/13/26 11:57, Junhua Shen wrote:
> Call amdgpu_svm_migration_init() in the device initialization and
> XGMI reset-restore paths to register the GPU's VRAM as a ZONE_DEVICE
> region before KFD initialization.
> 
> This activates the drm_pagemap migration infrastructure.
> 
> Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 ++++
>  drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c  | 4 ++++
>  2 files changed, 8 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index fbe553c38583..3be51a2c0106 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -78,6 +78,7 @@
>  #include "amdgpu_reset.h"
>  #include "amdgpu_virt.h"
>  #include "amdgpu_dev_coredump.h"
> +#include "amdgpu_migrate.h"
>  
>  #include <linux/suspend.h>
>  #include <drm/task_barrier.h>
> @@ -4076,6 +4077,9 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>  
>  	/* Don't init kfd if whole hive need to be reset during init */
>  	if (adev->init_lvl->level != AMDGPU_INIT_LEVEL_MINIMAL_XGMI) {
> +#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
> +		amdgpu_svm_migration_init(adev);
> +#endif

Looks good in general, but just a style advise:

Instead of spread those #if IS_ENABLED() It's good practice to do something like this in the header:

#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)

void amdgpu_svm_migration_init(struct amdgpu_device *adev);

#else

void amdgpu_svm_migration_init(struct amdgpu_device *adev) {}

#endif

Especially when the function is used multiple times.

Regards,
Christian.

>  		kgd2kfd_init_zone_device(adev);
>  		kfd_update_svm_support_properties(adev);
>  	}
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
> index 28c4ad62f50e..c94d43f3ab42 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
> @@ -25,6 +25,7 @@
>  #include "aldebaran.h"
>  #include "sienna_cichlid.h"
>  #include "smu_v13_0_10.h"
> +#include "amdgpu_migrate.h"
>  
>  static int amdgpu_reset_xgmi_reset_on_init_suspend(struct amdgpu_device *adev)
>  {
> @@ -87,6 +88,9 @@ static int amdgpu_reset_xgmi_reset_on_init_restore_hwctxt(
>  		return r;
>  	list_for_each_entry(tmp_adev, reset_device_list, reset_list) {
>  		if (!tmp_adev->kfd.init_complete) {
> +#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
> +			amdgpu_svm_migration_init(tmp_adev);
> +#endif
>  			kgd2kfd_init_zone_device(tmp_adev);
>  			amdgpu_amdkfd_device_init(tmp_adev);
>  			amdgpu_amdkfd_drm_client_create(tmp_adev);

Re: [PATCH v4 5/6] drm/amdgpu: hook up ZONE_DEVICE registration in device init and reset

[Junhua Shen]

On Wed, May 13, 2026 at 03:47:33PM +0200, Christian König wrote:
> 
> 
> On 5/13/26 11:57, Junhua Shen wrote:
> > Call amdgpu_svm_migration_init() in the device initialization and
> > XGMI reset-restore paths to register the GPU's VRAM as a ZONE_DEVICE
> > region before KFD initialization.
> > 
> > This activates the drm_pagemap migration infrastructure.
> > 
> > Signed-off-by: Junhua Shen <Junhua.Shen@amd.com>
> > ---
> >  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 ++++
> >  drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c  | 4 ++++
> >  2 files changed, 8 insertions(+)
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> > index fbe553c38583..3be51a2c0106 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> > @@ -78,6 +78,7 @@
> >  #include "amdgpu_reset.h"
> >  #include "amdgpu_virt.h"
> >  #include "amdgpu_dev_coredump.h"
> > +#include "amdgpu_migrate.h"
> >  
> >  #include <linux/suspend.h>
> >  #include <drm/task_barrier.h>
> > @@ -4076,6 +4077,9 @@ int amdgpu_device_init(struct amdgpu_device *adev,
> >  
> >  	/* Don't init kfd if whole hive need to be reset during init */
> >  	if (adev->init_lvl->level != AMDGPU_INIT_LEVEL_MINIMAL_XGMI) {
> > +#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
> > +		amdgpu_svm_migration_init(adev);
> > +#endif
> 
> Looks good in general, but just a style advise:
> 
> Instead of spread those #if IS_ENABLED() It's good practice to do something like this in the header:
> 
> #if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
> 
> void amdgpu_svm_migration_init(struct amdgpu_device *adev);
> 
> #else
> 
> void amdgpu_svm_migration_init(struct amdgpu_device *adev) {}
> 
> #endif
> 
> Especially when the function is used multiple times.
>
Thank you for the style advice.

The header (amdgpu_migrate.h) already provides the static inline stub
for the CONFIG_DRM_AMDGPU_SVM case.

I'll drop the #if IS_ENABLED() guards at the call sites in both
amdgpu_device.c and amdgpu_reset.c, and just call
amdgpu_svm_migration_init() directly.

Regards,
Junhua

> Regards,
> Christian.
> 
> >  		kgd2kfd_init_zone_device(adev);
> >  		kfd_update_svm_support_properties(adev);
> >  	}
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
> > index 28c4ad62f50e..c94d43f3ab42 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
> > @@ -25,6 +25,7 @@
> >  #include "aldebaran.h"
> >  #include "sienna_cichlid.h"
> >  #include "smu_v13_0_10.h"
> > +#include "amdgpu_migrate.h"
> >  
> >  static int amdgpu_reset_xgmi_reset_on_init_suspend(struct amdgpu_device *adev)
> >  {
> > @@ -87,6 +88,9 @@ static int amdgpu_reset_xgmi_reset_on_init_restore_hwctxt(
> >  		return r;
> >  	list_for_each_entry(tmp_adev, reset_device_list, reset_list) {
> >  		if (!tmp_adev->kfd.init_complete) {
> > +#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
> > +			amdgpu_svm_migration_init(tmp_adev);
> > +#endif
> >  			kgd2kfd_init_zone_device(tmp_adev);
> >  			amdgpu_amdkfd_device_init(tmp_adev);
> >  			amdgpu_amdkfd_drm_client_create(tmp_adev);
> 

Claude review: drm/amdgpu: SVM VRAM migration via drm_pagemap (XNACK-on)

[Claude Code Review Bot]

Overall Series Review

Subject: drm/amdgpu: SVM VRAM migration via drm_pagemap (XNACK-on)
Author: Junhua Shen <Junhua.Shen@amd.com>
Patches: 9
Reviewed: 2026-05-16T12:15:37.387075

---

This v4 series adds VRAM migration support to amdgpu's new drm_gpusvm-based SVM implementation, enabling GPU fault-driven (XNACK-on) migration of pages between system RAM and VRAM using SDMA. The series is well-structured, with each patch building logically on the previous one, and the architecture — wrapping `dev_pagemap` + `drm_pagemap` in `amdgpu_pagemap`, introducing `amdgpu_bo_svm` as a lightweight BO subtype — is sound and follows the Xe driver precedent.

**Strengths:**
- Good documentation: the address-space diagram in patch 1 and per-function kerneldoc are helpful
- Clean layering: infrastructure → SDMA callbacks → TTM eviction → range helpers → hookup
- Proper use of existing APIs (devm_memremap_pages, drm_pagemap framework, GART windowed SDMA)

**Concerns:**
- The `drm_pagemap_init()` error path in patch 1 leaks the `devm_memremap_pages` registration
- The SDMA copy functions have significant code duplication between `copy_to_devmem` and `copy_to_ram`
- `pre_migrate_fence` parameter is unused in both copy callbacks
- The `#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)` guards in patch 5 are unnecessary since `amdgpu_svm_migration_init()` already has a static inline stub
- Migration failure in the fault path is silently ignored, which may lead to infinite fault loops

---

---
Generated by Claude Code Patch Reviewer

Claude review: drm/amdgpu: add VRAM migration infrastructure for drm_pagemap

[Claude Code Review Bot]

Patch Review

**SPDX comment style:** The `.c` file uses `/* SPDX-License-Identifier: GPL-2.0 OR MIT */` which is the comment style for header files. C source files should use `// SPDX-License-Identifier: GPL-2.0 OR MIT`.

**`drm_pagemap_init` error path leaks resources:**
```c
if (drm_pagemap_init(&svm_dm->dpagemap, pgmap, adev_to_drm(adev),
						&amdgpu_svm_drm_pagemap_ops)) {
	dev_err(adev->dev, "SVM: failed to init drm_pagemap\n");
	return -EINVAL;
}
```
If `drm_pagemap_init()` fails, the `devm_memremap_pages()` registration is not unwound. While devres will eventually clean it up at device teardown, the function has already registered ZONE_DEVICE pages that will have no functional `drm_pagemap` backing — this is a window for use-after-free if the SVM paths try to use them. The error path should call `devm_memunmap_pages()` or equivalent, similar to the `IS_ERR(r)` path above it.

**Alignment in `drm_pagemap_init` call:**
```c
if (drm_pagemap_init(&svm_dm->dpagemap, pgmap, adev_to_drm(adev),
						&amdgpu_svm_drm_pagemap_ops)) {
```
The continuation line has excessive indentation (tabs to column ~40). Align to the opening parenthesis or use a single extra tab.

**`amdgpu_amdkfd.h` include removed but no explanation:** The diff removes `#include "amdgpu_amdkfd.h"` from the includes. Patch 2 replaces the include list entirely, so this is fine functionally, but it's an artifact of the patch split — patch 1 adds the file, patch 2 rewrites the includes. The original file in patch 1 never needed this include in the first place (nothing from amdkfd is used), so this is just slightly messy.

**`amdgpu_svm_page_to_apagemap` is defined but unused in this patch.** It's used later in patch 2. This is minor but could trigger a compiler warning depending on config.

**Double blank line** at line 291 (between `amdgpu_svm_page_to_apagemap` and `amdgpu_svm_drm_pagemap_ops`).

---
Generated by Claude Code Patch Reviewer

Claude review: drm/amdgpu: implement drm_pagemap SDMA migration callbacks

[Claude Code Review Bot]

Patch Review

This is the largest and most complex patch. Overall well-structured.

**`pre_migrate_fence` unused in both callbacks:**
```c
static int
amdgpu_svm_copy_to_devmem(struct page **pages,
			   struct drm_pagemap_addr *pagemap_addr,
			   unsigned long npages,
			   struct dma_fence *pre_migrate_fence)
```
The `pre_migrate_fence` is passed by the framework but completely ignored. If the framework provides a fence to wait on before migrating (e.g., for serialization), ignoring it could cause data corruption. Either wait on it before starting SDMA copies, or add a comment explaining why it can safely be ignored in this context.

**`copy_to_devmem` and `copy_to_ram` are nearly identical** — ~60 lines of the same batch/flush logic. The only differences are: (1) direction constant, (2) `copy_to_ram` also checks `!pages[i]` in the loop. A shared helper parameterized on direction would reduce duplication and maintenance burden.

**GART window usage — only window 0 is used:**
```c
*gart_addr = amdgpu_compute_gart_address(&adev->gmc, entity, 0);
```
The entity has two GART windows (`gart_window_offs[2]`). Using only window 0 means that in `amdgpu_svm_copy_memory_gart`, for `FROM_RAM_TO_VRAM` the source is GART-mapped and the destination is a direct VRAM MC address, which is correct. But it means only one GART mapping per SDMA copy submission — the existing KFD migrate code uses both windows (0 for src, 1 for dst) for VRAM↔VRAM copies. For RAM↔VRAM this is fine since one side is always direct-mapped, but worth a comment.

**`amdgpu_svm_gart_map` submits a fire-and-forget fence:**
```c
fence = amdgpu_job_submit(job);
dma_fence_put(fence);
```
The GART update fence is dropped immediately. The subsequent `amdgpu_copy_buffer` call will naturally serialize on the same ring, so this is correct as long as both jobs go through the same entity. Since both use `entity->base`, this holds.

**`amdgpu_svm_copy_memory_gart` uses `move_entity` (singular):**
```c
struct amdgpu_ttm_buffer_entity *entity = &adev->mman.move_entity;
```
The current drm-next tree has `move_entities[]` (plural array). This must depend on the base SVM series changing the struct. This is fine if the dependency is correct, but will cause a build failure if someone tries to apply without the base.

**`amdgpu_bo_svm_destroy` doesn't handle imported BOs:**
```c
static void amdgpu_bo_svm_destroy(struct ttm_buffer_object *tbo)
{
	struct amdgpu_bo *bo = ttm_to_amdgpu_bo(tbo);
	struct amdgpu_bo_svm *svm_bo = to_amdgpu_bo_svm(bo);

	amdgpu_bo_kunmap(bo);
	drm_gem_object_release(&bo->tbo.base);
	amdgpu_bo_unref(&bo->parent);
	kvfree(svm_bo);
}
```
Compare to `amdgpu_bo_destroy` which has:
```c
if (drm_gem_is_imported(&bo->tbo.base))
	drm_prime_gem_destroy(&bo->tbo.base, bo->tbo.sg);
```
SVM BOs should never be imported, so this is likely fine, but a `WARN_ON(drm_gem_is_imported(...))` would make the assumption explicit and catch bugs.

**BO created with VRAM_CLEARED but migration will overwrite immediately:**
```c
bp.flags = AMDGPU_GEM_CREATE_NO_CPU_ACCESS |
	   AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS |
	   AMDGPU_GEM_CREATE_VRAM_CLEARED;
```
Clearing VRAM before immediately overwriting it with SDMA copy is wasteful. Consider dropping `AMDGPU_GEM_CREATE_VRAM_CLEARED` — the data will be written by `copy_to_devmem` before any read.

**`populate_mm` unreserves after `drm_pagemap_migrate_to_devmem`:**
```c
ret = drm_pagemap_migrate_to_devmem(&svm_bo->devmem,
				     mm, start, end,
				     &mdetails);

amdgpu_bo_unreserve(&svm_bo->bo);
```
`amdgpu_bo_create()` returns with the BO reserved. The unreserve happens after migration. But if `drm_pagemap_migrate_to_devmem` fails, does the framework still hold any reference to the BO? If `devmem_release` can be called asynchronously after a partial migration, the unreserve here might race. The lifecycle needs careful thought — who drops the last ref on the BO if migration fails completely?

---
Generated by Claude Code Patch Reviewer

Claude review: drm/amdgpu: implement synchronous TTM eviction for SVM BOs

[Claude Code Review Bot]

Patch Review

**`amdgpu_svm_bo_is_svm` relies on function pointer comparison:**
```c
bool amdgpu_svm_bo_is_svm(struct ttm_buffer_object *tbo)
{
	return tbo->destroy == &amdgpu_bo_svm_destroy;
}
```
This requires `amdgpu_bo_svm_destroy` to be visible across translation units, but it's declared `static` in `amdgpu_migrate.c`. This means `amdgpu_svm_bo_is_svm` must also be in `amdgpu_migrate.c`, which it is (added in the same file's diff). However, this coupling is fragile — if someone moves the function or adds LTO optimizations that merge identical functions, the pointer comparison breaks. Consider using a flag in the BO instead.

**`amdgpu_bo_is_amdgpu_bo` now calls `amdgpu_svm_bo_is_svm` unconditionally:**
```c
if (bo->destroy == &amdgpu_bo_destroy ||
    bo->destroy == &amdgpu_bo_user_destroy ||
    amdgpu_svm_bo_is_svm(bo))
	return true;
```
This adds a function call overhead on the hot path for every BO check, even when SVM is disabled. The `!IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)` stub returns `false`, so this is just a minor performance nit — the compiler should inline and eliminate it.

**Eviction in `amdgpu_bo_move` — no error message on failure:**
```c
r = amdgpu_svm_bo_evict(abo);
if (!r)
	ttm_bo_move_null(bo, new_mem);
return r;
```
If `amdgpu_svm_bo_evict` fails, `r` is returned directly with no logging. TTM will retry eviction, potentially in an infinite loop. Consider at least a `dev_warn`.

---
Generated by Claude Code Patch Reviewer

Claude review: drm/amdgpu: add SVM range migration helpers for drm_pagemap

[Claude Code Review Bot]

Patch Review

**`amdgpu_svm_get_dpagemap` doesn't NULL-check `apagemap`:**
```c
static struct drm_pagemap *
amdgpu_svm_get_dpagemap(struct amdgpu_svm *svm)
{
	struct amdgpu_pagemap *apagemap = svm->adev->apagemap;

	if (!apagemap->initialized)
		return NULL;

	return &apagemap->dpagemap;
}
```
If `apagemap` is NULL (SVM migration not initialized), this will dereference NULL. Needs:
```c
if (!apagemap || !apagemap->initialized)
    return NULL;
```
This is called from `amdgpu_pagemap_capable()` which is used in the fault path, so it's reachable when migration_init wasn't called (e.g., old GPU, APU, or init failure).

**`range_needs_migrate_to_vram` accesses `range->pages.flags.migrate_devmem` without READ_ONCE:**
```c
static bool
range_needs_migrate_to_vram(struct drm_gpusvm_range *range,
			    enum amdgpu_svm_migrate_mode mode)
{
	if (!range->pages.flags.migrate_devmem)
		return false;
```
But `range_in_vram()` carefully uses `READ_ONCE` on the flags:
```c
struct drm_gpusvm_pages_flags flags = {
	.__flags = READ_ONCE(range->pages.flags.__flags),
};
```
Inconsistent synchronization — if `flags.__flags` can change concurrently (as the comment "Pairs with WRITE_ONCE" suggests), then `range_needs_migrate_to_vram` should also use `READ_ONCE`.

**`amdgpu_svm_range_migrate_range` ignores migration errors:**
```c
if (range_needs_migrate_to_vram(range, migrate_mode))
	return amdgpu_svm_range_migrate_to_vram(svm, range);
```
The return value propagates here, but callers in patch 6 (fault path) ignore it — see patch 6 review.

**SPDX comment style:** Same issue as patch 1 — `.c` file uses `/* SPDX */` instead of `// SPDX`.

---
Generated by Claude Code Patch Reviewer

Claude review: drm/amdgpu: hook up ZONE_DEVICE registration in device init and reset

[Claude Code Review Bot]

Patch Review

**Unnecessary `#if IS_ENABLED` guards:**
```c
#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
	amdgpu_svm_migration_init(adev);
#endif
```
The header already provides a static inline stub returning 0 when `CONFIG_DRM_AMDGPU_SVM` is disabled. The `#if` guard is redundant and adds visual noise. Just call the function directly.

**Return value of `amdgpu_svm_migration_init` is ignored** in both callsites. If initialization fails (e.g., `devm_memremap_pages` fails), the device will continue without VRAM migration support. The cover letter says `adev->apagemap` is NULL when disabled, and the helper functions check for this, so this is likely intentional — but a `dev_warn` on failure would help debugging.

**In the reset path:**
```c
if (!tmp_adev->kfd.init_complete) {
#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM)
	amdgpu_svm_migration_init(tmp_adev);
#endif
	kgd2kfd_init_zone_device(tmp_adev);
```
`amdgpu_svm_migration_init` has an early-return check `if (adev->apagemap && adev->apagemap->initialized) return 0;` — so calling it again after reset is safe. Good.

---
Generated by Claude Code Patch Reviewer

Claude review: drm/amdgpu: integrate VRAM migration into SVM range map and fault paths

[Claude Code Review Bot]

Patch Review

**Migration failure silently ignored in fault path:**
```c
if (need_vram_migration)
	amdgpu_svm_range_migrate_range(svm, &range->base,
				      AMDGPU_SVM_MIGRATE_TO_VRAM);
```
The return value is discarded. If migration fails, execution continues and `amdgpu_svm_range_get_pages()` will map system RAM pages instead. This is reasonable as a fallback, but:
1. If `devmem_only` is true in `map_ctx`, the subsequent `get_pages` may fail because it expects VRAM pages
2. No logging means silent performance degradation is invisible
3. If migration fails due to VRAM pressure, the GPU will keep faulting on the same pages — potential infinite retry loop

At minimum, log the failure. Consider also clearing `map_ctx.devmem_only` after migration failure.

**Same issue in the prefetch path:**
```c
if (need_vram_migration) {
	AMDGPU_SVM_RANGE_DEBUG(range, "PREFETCH - MIGRATION PAGES");
	amdgpu_svm_range_migrate_range(svm, &range->base,
				      AMDGPU_SVM_MIGRATE_TO_VRAM);
}
```
Return value ignored again.

**`devmem_possible = false` TODO removal is the right cleanup**, but the declaration-after-statement pattern (`bool devmem_possible = ...; bool need_vram_migration = ...; struct drm_gpusvm_ctx map_ctx = { ... };`) mixing declarations and assigned variables should work fine with modern C standards but worth noting that some kernel subsystems still prefer C89-style declarations. The amdgpu driver generally allows mixed declarations, so this is fine.

**VRAM PTE flags handling looks correct:**
```c
if (entry->proto == AMDGPU_INTERCONNECT_VRAM)
	seg_pte_flags &= ~(AMDGPU_PTE_SYSTEM | AMDGPU_PTE_SNOOPED);
```
Clearing SYSTEM and SNOOPED for local VRAM pages is the right thing — these are direct VRAM accesses through MMHUB, not system memory through IOMMU.

---
Generated by Claude Code Patch Reviewer