Hello,
This patchset introduces memory shrinker for the VirtIO-GPU DRM driver.
During OOM, the shrinker will release BOs that are marked as "not needed"
by userspace using the new madvise IOCTL, it will also evict idling BOs
to SWAP. The userspace in this case is the Mesa VirGL driver, it will mark
the cached BOs as "not needed", allowing kernel driver to release memory
of the cached shmem BOs on lowmem situations, preventing OOM kills.
This patchset adds memory purging and eviction support to VirtIO-GPU driver.
The Panfrost driver is switched to use generic memory shrinker. Eviction
support will come later on, after resolving the blocker bug in Panfrost.
This patchset also includes couple improvements and fixes for various
minor things that I found while was working on the shrinker.
The Mesa and IGT patches will be kept on hold until this kernel series
will be approved and merged.
This patchset was tested using Qemu and crosvm, including both cases of
IOMMU off/on.
Mesa: https://gitlab.freedesktop.org/digetx/mesa/-/commits/virgl-madvise
IGT: https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/virtio-madvise
https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/panfrost-madvise
Changelog:
v3: - Hardened shrinker's count() with usage of READ_ONCE() since we don't
use atomic type for counting and technically compiler is free to
re-fetch counter's variable.
- "Correct drm_gem_shmem_get_sg_table() error handling" now uses
PTR_ERR_OR_ZERO(), fixing typo that was made in v2.
- Removed obsoleted shrinker from the Panfrost driver, which I missed to
do in v2 by accident and Alyssa Rosenzweig managed to notice it.
- CCed stable kernels in all patches that make fixes, even the minor ones,
like was suggested by Emil Velikov and added his r-b to the patches.
- Added t-b from Steven Price to the Panfrost's shrinker patch.
- Corrected doc-comment of drm_gem_shmem_object.madv, like was suggested
by Steven Price. Comment now says that madv=1 means "object is purged"
instead of saying that value is unused.
- Added more doc-comments to the new shmem shrinker API.
- The "Improve DMA API usage for shmem BOs" patch got more improvements
by removing the obsoleted drm_dev_set_unique() quirk and its comment.
- Added patch that makes Virtio-GPU driver to use common dev_is_pci()
helper, which was suggested by Robin Murphy.
- Added new "drm/shmem-helper: Take GEM reservation lock instead of
drm_gem_shmem locks" patch, which was suggested by Daniel Vetter.
- Added new "drm/virtio: Simplify error handling of
virtio_gpu_object_create()" patch.
- Improved "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
like was suggested by Daniel Vetter, by saying that function returns
ERR_PTR() and not errno.
- virtio_gpu_purge_object() is fenced properly now, turned out
virtio_gpu_notify() doesn't do fencing as I was supposing before.
Stress testing of memory eviction revealed that.
- Added new patch that corrects virtio_gpu_plane_cleanup_fb() to use
appropriate atomic plane state.
- SHMEM shrinker got eviction support.
- VirtIO-GPU driver now supports memory eviction. It's enabled for a
non-blob GEMs only, i.e. for VirGL. The blobs don't support dynamic
attaching/detaching of guest's memory, so it's not trivial to enable
them.
- Added patch that removes obsoleted drm_gem_shmem_purge()
- Added patch that makes drm_gem_shmem_get_pages() private.
- Added patch that fixes lockup on dma_resv_reserve_fences() error.
v2: - Improved shrinker by using a more fine-grained locking to reduce
contention during scan of objects and dropped locking from the
'counting' callback by tracking count of shrinkable pages. This
was suggested by Rob Clark in the comment to v1.
- Factored out common shrinker code into drm_gem_shmem_helper.c
and switched Panfrost driver to use the new common memory shrinker.
This was proposed by Thomas Zimmermann in his prototype series that
he shared with us in the comment to v1. Note that I only compile-tested
the Panfrost driver.
- Shrinker now takes object_name_lock during scan to prevent racing
with dma-buf exporting.
- Shrinker now takes vmap_lock during scan to prevent racing with shmem
vmap/unmap code.
- Added "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
which I sent out previously as a standalone change, since the
drm_gem_shmem_helper.c is now touched by this patchset anyways and
it doesn't hurt to group all the patches together.
Dmitry Osipenko (15):
drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling
drm/virtio: Check whether transferred 2D BO is shmem
drm/virtio: Unlock GEM reservations on virtio_gpu_object_shmem_init()
error
drm/virtio: Unlock reservations on dma_resv_reserve_fences() error
drm/virtio: Use appropriate atomic state in
virtio_gpu_plane_cleanup_fb()
drm/virtio: Simplify error handling of virtio_gpu_object_create()
drm/virtio: Improve DMA API usage for shmem BOs
drm/virtio: Use dev_is_pci()
drm/shmem-helper: Correct doc-comment of drm_gem_shmem_get_sg_table()
drm/shmem-helper: Take reservation lock instead of drm_gem_shmem locks
drm/shmem-helper: Add generic memory shrinker
drm/virtio: Support memory shrinking
drm/panfrost: Switch to generic memory shrinker
drm/shmem-helper: Make drm_gem_shmem_get_pages() private
drm/shmem-helper: Remove drm_gem_shmem_purge()
drivers/gpu/drm/drm_gem_shmem_helper.c | 815 ++++++++++++++++--
drivers/gpu/drm/lima/lima_gem.c | 8 +-
drivers/gpu/drm/panfrost/Makefile | 1 -
drivers/gpu/drm/panfrost/panfrost_device.h | 4 -
drivers/gpu/drm/panfrost/panfrost_drv.c | 19 +-
drivers/gpu/drm/panfrost/panfrost_gem.c | 30 +-
drivers/gpu/drm/panfrost/panfrost_gem.h | 9 -
.../gpu/drm/panfrost/panfrost_gem_shrinker.c | 122 ---
drivers/gpu/drm/panfrost/panfrost_job.c | 18 +-
drivers/gpu/drm/panfrost/panfrost_mmu.c | 15 +-
drivers/gpu/drm/virtio/virtgpu_drv.c | 53 +-
drivers/gpu/drm/virtio/virtgpu_drv.h | 20 +-
drivers/gpu/drm/virtio/virtgpu_gem.c | 50 +-
drivers/gpu/drm/virtio/virtgpu_ioctl.c | 37 +
drivers/gpu/drm/virtio/virtgpu_kms.c | 16 +-
drivers/gpu/drm/virtio/virtgpu_object.c | 204 +++--
drivers/gpu/drm/virtio/virtgpu_plane.c | 23 +-
drivers/gpu/drm/virtio/virtgpu_vq.c | 55 +-
include/drm/drm_device.h | 4 +
include/drm/drm_gem.h | 35 +
include/drm/drm_gem_shmem_helper.h | 114 ++-
include/uapi/drm/virtgpu_drm.h | 14 +
22 files changed, 1278 insertions(+), 388 deletions(-)
delete mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
--
2.35.1
Use common dev_is_pci() helper to replace the strcmp("pci") used by driver.
Suggested-by: Robin Murphy <[email protected]>
Signed-off-by: Dmitry Osipenko <[email protected]>
---
drivers/gpu/drm/virtio/virtgpu_drv.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c b/drivers/gpu/drm/virtio/virtgpu_drv.c
index 0141b7df97ec..0035affc3e59 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.c
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.c
@@ -87,7 +87,7 @@ static int virtio_gpu_probe(struct virtio_device *vdev)
return PTR_ERR(dev);
vdev->priv = dev;
- if (!strcmp(vdev->dev.parent->bus->name, "pci")) {
+ if (dev_is_pci(vdev->dev.parent)) {
ret = virtio_gpu_pci_quirk(dev);
if (ret)
goto err_free;
--
2.35.1
drm_gem_shmem_get_sg_table() never returns NULL on error, but a ERR_PTR.
Correct the doc comment which says that it returns NULL on error.
Signed-off-by: Dmitry Osipenko <[email protected]>
---
drivers/gpu/drm/drm_gem_shmem_helper.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 8ad0e02991ca..30ee46348a99 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -662,7 +662,7 @@ EXPORT_SYMBOL(drm_gem_shmem_print_info);
* drm_gem_shmem_get_pages_sgt() instead.
*
* Returns:
- * A pointer to the scatter/gather table of pinned pages or NULL on failure.
+ * A pointer to the scatter/gather table of pinned pages or errno on failure.
*/
struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem)
{
@@ -688,7 +688,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_get_sg_table);
* drm_gem_shmem_get_sg_table() should not be directly called by drivers.
*
* Returns:
- * A pointer to the scatter/gather table of pinned pages or errno on failure.
+ * A pointer to the scatter/gather table of pinned pages ERR_PTR()-encoded
+ * error code on failure.
*/
struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
{
--
2.35.1