Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752079AbcKGGPv (ORCPT ); Mon, 7 Nov 2016 01:15:51 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:36149 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751918AbcKGGPu (ORCPT ); Mon, 7 Nov 2016 01:15:50 -0500 Subject: Re: [PATCH v11 00/22] Add Mediated device support To: Kirti Wankhede , alex.williamson@redhat.com, pbonzini@redhat.com, kraxel@redhat.com, cjia@nvidia.com References: <1478293856-8191-1-git-send-email-kwankhede@nvidia.com> <40bfdbe9-1070-b5e6-fba2-6b97999b558b@nvidia.com> Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, kevin.tian@intel.com, jike.song@intel.com, bjsdjshi@linux.vnet.ibm.com, linux-kernel@vger.kernel.org From: Alexey Kardashevskiy Message-ID: Date: Mon, 7 Nov 2016 17:15:38 +1100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <40bfdbe9-1070-b5e6-fba2-6b97999b558b@nvidia.com> Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6714 Lines: 144 On 07/11/16 14:59, Kirti Wankhede wrote: > > > On 11/7/2016 9:00 AM, Alexey Kardashevskiy wrote: >> On 05/11/16 08:10, Kirti Wankhede wrote: >>> This series adds Mediated device support to Linux host kernel. Purpose >>> of this series is to provide a common interface for mediated device >>> management that can be used by different devices. This series introduces >>> Mdev core module that creates and manages mediated devices, VFIO based >>> driver for mediated devices that are created by mdev core module and >>> update VFIO type1 IOMMU module to support pinning & unpinning for mediated >>> devices. >>> >>> What changed in v11? >>> mdev core: >>> Register mdev_bus class when first device is registed to avoid panic if >>> any vendor driver and mdev driver are selected as built-in but vendor >>> driver loads first and then mdev module. >>> vfio_mdev: >>> Added notifier callback function to mdev parent's ops so that notifer >>> is registered from vfio_mdev module during device open and unregistered >>> it from device close call. This is a optional callback. Some drivers >>> using mdev framework might not pin or unpin pages, for example the >>> sample mtty driver that simulates serial port. Vendor driver who need to >>> pin/unpin pages should provide this callback. Otherwise pin request >>> would fail. >>> vfio_iommu_type1: >>> Updated to keep track of who (task and address space) mapped iova range. >>> During DMA_UNMAP, same task who mapped it or other task who shares same >>> address space is allowed to unmap, otherwise unmap fails. >>> QEMU maps few iova ranges initially, then fork threads and from the child >>> thread calls DMA_UNMAP on previously mapped iova. Since child shares same >>> address space, DMA_UNMAP is successful. >>> Address space keeps track of pages pinned (pfn_list) by external user / >>> mdev devices. This pfn_list is used to verify pfn during unpin_request, >>> re-accounting of pages when direct device assigned in hot-unplugged and >>> mdev device is present in same container. >>> When the container is released, all mapped iova from all tasks are >>> unmapped and removed. >>> >>> Tested by assigning below combinations of devices to a single VM: >>> - GPU pass through only >>> - vGPU device only >>> - One GPU pass through and one vGPU device >>> - Linux VM hot plug and unplug vGPU device while GPU pass through device >>> exist >>> - Linux VM hot plug and unplug GPU pass through device while vGPU device >>> exist >>> >>> Patch series tested with linux-next upto commit 14970f204b19 @Fri Oct 28 >>> Resolved against conflicting change: >> >> Patch 11/22 does not apply if the patchset is applied on top of 14970f204b19. >> >> > > Pull commit 05692d7005a3 (below) commit. Nah, it was my fault, thunderbird's plugin saved mails in wrong order, false alarm, sorry. Speaking of order, 11/22 is better be squashed into 02/22 (as it does not seem to make sense alone; it also does not depend on something between 02/22 and 11/22); and both 01/22 and 02/22 better be moved where "docs" patches start as they do not makes sense earlier anyway, or do they? > >> >>> 05692d7005a3 vfio/pci: Fix integer overflows, bitmask check >>> > > Thanks, > Kirti > >>> >>> Kirti Wankhede (22): >>> vfio: Mediated device Core driver >>> vfio: VFIO based driver for Mediated devices >>> vfio: Rearrange functions to get vfio_group from dev >>> vfio: Common function to increment container_users >>> vfio iommu: Added pin and unpin callback functions to >>> vfio_iommu_driver_ops >>> vfio iommu type1: Update arguments of vfio_lock_acct >>> vfio iommu type1: Update argument of vaddr_get_pfn() >>> vfio iommu type1: Add find_iommu_group() function >>> vfio iommu type1: Add task structure to vfio_dma >>> vfio iommu type1: Add support for mediated devices >>> vfio iommu: Add blocking notifier to notify DMA_UNMAP >>> vfio: Add notifier callback to parent's ops structure of mdev >>> vfio: Introduce common function to add capabilities >>> vfio_pci: Update vfio_pci to use vfio_info_add_capability() >>> vfio: Introduce vfio_set_irqs_validate_and_prepare() >>> vfio_pci: Updated to use vfio_set_irqs_validate_and_prepare() >>> vfio_platform: Updated to use vfio_set_irqs_validate_and_prepare() >>> vfio: Define device_api strings >>> docs: Add Documentation for Mediated devices >>> docs: Sysfs ABI for mediated device framework >>> docs: Sample driver to demonstrate how to use Mediated device >>> framework. >>> MAINTAINERS: Add entry VFIO based Mediated device drivers >>> >>> Documentation/ABI/testing/sysfs-bus-vfio-mdev | 111 ++ >>> Documentation/vfio-mediated-device.txt | 399 +++++++ >>> MAINTAINERS | 9 + >>> drivers/vfio/Kconfig | 1 + >>> drivers/vfio/Makefile | 1 + >>> drivers/vfio/mdev/Kconfig | 17 + >>> drivers/vfio/mdev/Makefile | 5 + >>> drivers/vfio/mdev/mdev_core.c | 388 +++++++ >>> drivers/vfio/mdev/mdev_driver.c | 122 ++ >>> drivers/vfio/mdev/mdev_private.h | 41 + >>> drivers/vfio/mdev/mdev_sysfs.c | 286 +++++ >>> drivers/vfio/mdev/vfio_mdev.c | 167 +++ >>> drivers/vfio/pci/vfio_pci.c | 83 +- >>> drivers/vfio/platform/vfio_platform_common.c | 31 +- >>> drivers/vfio/vfio.c | 334 +++++- >>> drivers/vfio/vfio_iommu_type1.c | 831 ++++++++++++-- >>> include/linux/mdev.h | 176 +++ >>> include/linux/vfio.h | 32 +- >>> include/uapi/linux/vfio.h | 10 + >>> samples/vfio-mdev/Makefile | 13 + >>> samples/vfio-mdev/mtty.c | 1503 +++++++++++++++++++++++++ >>> 21 files changed, 4342 insertions(+), 218 deletions(-) >>> create mode 100644 Documentation/ABI/testing/sysfs-bus-vfio-mdev >>> create mode 100644 Documentation/vfio-mediated-device.txt >>> create mode 100644 drivers/vfio/mdev/Kconfig >>> create mode 100644 drivers/vfio/mdev/Makefile >>> create mode 100644 drivers/vfio/mdev/mdev_core.c >>> create mode 100644 drivers/vfio/mdev/mdev_driver.c >>> create mode 100644 drivers/vfio/mdev/mdev_private.h >>> create mode 100644 drivers/vfio/mdev/mdev_sysfs.c >>> create mode 100644 drivers/vfio/mdev/vfio_mdev.c >>> create mode 100644 include/linux/mdev.h >>> create mode 100644 samples/vfio-mdev/Makefile >>> create mode 100644 samples/vfio-mdev/mtty.c >>> >> >> -- Alexey