Received: by 2002:a25:683:0:0:0:0:0 with SMTP id 125csp1310823ybg; Thu, 11 Jun 2020 06:41:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzIKRoGR7HCKfizcGU/qhb4UYS8MFkgbrpczkE45t8xh8iw0Yhuxfv0EZwDcRK0Se2LDRty X-Received: by 2002:a17:906:6b8e:: with SMTP id l14mr8654075ejr.32.1591882901722; Thu, 11 Jun 2020 06:41:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591882901; cv=none; d=google.com; s=arc-20160816; b=KTDDrTyblTfMQ4gPYOZiIdrjV3Ql8Rpp7cNQcIsa14xhy5Y8F6OvgeqViISdW03jv4 A050ejnneUg5DrrJpaXQ3AgSyKijdye93Gw1BvvulheT5ZKxoN5GJeVUOR8wUcrrEByJ bVvak57nE1B4XL/nVxNGvk/ZXwa+B21YACJEzrlGaif2ZdWPIIyyeIVIlVLT4h6Sjr5/ E6awi0+IRgtUzqWVuHfGwg2vSbkufLCx8Sw94RederTRHp/QfNUHJRltsKvGqL35qodk zn+SIUqX/zii7T+9glqTnsrxMJmqDWMejs6XoAo0BAzcRSr/v9OvHginTXDMmyxaEFy/ lADw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :ironport-sdr:ironport-sdr; bh=x5LaPMq4hY5XuO7FvT6OGZ5Awve/ya8VDeYgXJOhBzQ=; b=Yu5IgwyPsL+1LLsFruI7L3DVY5uPrk/4RcPcz1xJqQBkAiO2uqFA/1Rdu/d2hByfde LzASpWKdUY2J8mT1MATuyID0hxgwZjbvLurOPmTuY+t4G4/vhW7ashnDMYV2OEWHrNEr relXEFlAkovh4rHs5ZX4Ml8pSN8NT2kEgMJFVNyiNuhNAUsv3bzQ9DffqYnUvKbBTLdS dqt3s+Tz83eqPgJ9TiensMzDQAejgjo2MFh92Ap8GjXvSKYxeUBWy96VSOJ6vqiehL6i WS3F8u7e5hXPbu0VIAUzFE9rmEJ2Bs9jgA4dIkJkedV6GS/yT7IP6C/gmofd+muSZlY3 E0RQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id x10si2013960ejc.477.2020.06.11.06.41.18; Thu, 11 Jun 2020 06:41:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728015AbgFKMJK (ORCPT + 99 others); Thu, 11 Jun 2020 08:09:10 -0400 Received: from mga17.intel.com ([192.55.52.151]:41824 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727849AbgFKMJK (ORCPT ); Thu, 11 Jun 2020 08:09:10 -0400 IronPort-SDR: Qzh73Zpvtv4Q+8OeusGrmhzSWPyzLaF8qoEk+crYkCIic8dYaup/zGynwFoibennyF5QTcifRo AIiQDSj/nzIA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jun 2020 05:09:09 -0700 IronPort-SDR: slupFsEoGCu3+aZxDJFA/0zL5b17ZqrOsg0F2camZjWpwi38u1nWMEY+JyM7+b0EFAF+80FeiR uSzRmt7F1iIw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,499,1583222400"; d="scan'208";a="419082464" Received: from jacob-builder.jf.intel.com ([10.7.199.155]) by orsmga004.jf.intel.com with ESMTP; 11 Jun 2020 05:09:09 -0700 From: Liu Yi L To: alex.williamson@redhat.com, eric.auger@redhat.com, baolu.lu@linux.intel.com, joro@8bytes.org Cc: kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, ashok.raj@intel.com, yi.l.liu@intel.com, jun.j.tian@intel.com, yi.y.sun@intel.com, jean-philippe@linaro.org, peterx@redhat.com, hao.wu@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 00/15] vfio: expose virtual Shared Virtual Addressing to VMs Date: Thu, 11 Jun 2020 05:15:19 -0700 Message-Id: <1591877734-66527-1-git-send-email-yi.l.liu@intel.com> X-Mailer: git-send-email 2.7.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Shared Virtual Addressing (SVA), a.k.a, Shared Virtual Memory (SVM) on Intel platforms allows address space sharing between device DMA and applications. SVA can reduce programming complexity and enhance security. This VFIO series is intended to expose SVA usage to VMs. i.e. Sharing guest application address space with passthru devices. This is called vSVA in this series. The whole vSVA enabling requires QEMU/VFIO/IOMMU changes. For IOMMU and QEMU changes, they are in separate series (listed in the "Related series"). The high-level architecture for SVA virtualization is as below, the key design of vSVA support is to utilize the dual-stage IOMMU translation ( also known as IOMMU nesting translation) capability in host IOMMU. .-------------. .---------------------------. | vIOMMU | | Guest process CR3, FL only| | | '---------------------------' .----------------/ | PASID Entry |--- PASID cache flush - '-------------' | | | V | | CR3 in GPA '-------------' Guest ------| Shadow |--------------------------|-------- v v v Host .-------------. .----------------------. | pIOMMU | | Bind FL for GVA-GPA | | | '----------------------' .----------------/ | | PASID Entry | V (Nested xlate) '----------------\.------------------------------. | | |SL for GPA-HPA, default domain| | | '------------------------------' '-------------' Where: - FL = First level/stage one page tables - SL = Second level/stage two page tables Patch Overview: 1. a refactor to vfio_iommu_type1 ioctl (patch 0001) 2. reports IOMMU nesting info to userspace ( patch 0002, 0003 and 0015) 3. vfio support for PASID allocation and free for VMs (patch 0004, 0005, 0006) 4. vfio support for binding guest page table to host (patch 0007, 0008, 0009, 0010) 5. vfio support for IOMMU cache invalidation from VMs (patch 0011) 6. vfio support for vSVA usage on IOMMU-backed mdevs (patch 0012) 7. expose PASID capability to VM (patch 0013) 8. add doc for VFIO dual stage control (patch 0014) The complete vSVA kernel upstream patches are divided into three phases: 1. Common APIs and PCI device direct assignment 2. IOMMU-backed Mediated Device assignment 3. Page Request Services (PRS) support This patchset is aiming for the phase 1 and phase 2, and based on Jacob's below series. [PATCH v13 0/8] Nested Shared Virtual Address (SVA) VT-d support - merged https://lkml.org/lkml/2020/5/13/1582 [PATCH v2 0/3] IOMMU user API enhancement - wip https://lkml.org/lkml/2020/6/11/5 [PATCH 00/10] IOASID extensions for guest SVA - wip https://lkml.org/lkml/2020/3/25/874 The latest IOASID code added below new interface for itertate all PASIDs of an ioasid_set. The implementation is not sent out yet as Jacob needs some cleanup, it can be found in branch vsva-linux-5.7-rc4-v2. int ioasid_set_for_each_ioasid(int sid, void (*fn)(ioasid_t id, void *data), void *data); Complete set for current vSVA can be found in below branch. This branch also includes some extra modifications to IOASID core code and vt-d iommu driver cleanup patches. https://github.com/luxis1999/linux-vsva.git:vsva-linux-5.7-rc4-v2 The corresponding QEMU patch series is included in below branch: https://github.com/luxis1999/qemu.git:vsva_5.7_rc4_qemu_rfcv6 Regards, Yi Liu Changelog: - Patch v1 -> Patch v2: a) Refactor vfio_iommu_type1_ioctl() per suggestion from Christoph Hellwig. b) Re-sequence the patch series for better bisect support. c) Report IOMMU nesting cap info in detail instead of a format in v1. d) Enforce one group per nesting type container for vfio iommu type1 driver. e) Build the vfio_mm related code from vfio.c to be a separate vfio_pasid.ko. f) Add PASID ownership check in IOMMU driver. g) Adopted to latest IOMMU UAPI design. Removed IOMMU UAPI version check. Added iommu_gpasid_unbind_data for unbind requests from userspace. h) Define a single ioctl:VFIO_IOMMU_NESTING_OP for bind/unbind_gtbl and cahce_invld. i) Document dual stage control in vfio.rst. Patch v1: https://lore.kernel.org/linux-iommu/1584880325-10561-1-git-send-email-yi.l.liu@intel.com/ - RFC v3 -> Patch v1: a) Address comments to the PASID request(alloc/free) path b) Report PASID alloc/free availabitiy to user-space c) Add a vfio_iommu_type1 parameter to support pasid quota tuning d) Adjusted to latest ioasid code implementation. e.g. remove the code for tracking the allocated PASIDs as latest ioasid code will track it, VFIO could use ioasid_free_set() to free all PASIDs. RFC v3: https://lore.kernel.org/linux-iommu/1580299912-86084-1-git-send-email-yi.l.liu@intel.com/ - RFC v2 -> v3: a) Refine the whole patchset to fit the roughly parts in this series b) Adds complete vfio PASID management framework. e.g. pasid alloc, free, reclaim in VM crash/down and per-VM PASID quota to prevent PASID abuse. c) Adds IOMMU uAPI version check and page table format check to ensure version compatibility and hardware compatibility. d) Adds vSVA vfio support for IOMMU-backed mdevs. RFC v2: https://lore.kernel.org/linux-iommu/1571919983-3231-1-git-send-email-yi.l.liu@intel.com/ - RFC v1 -> v2: Dropped vfio: VFIO_IOMMU_ATTACH/DETACH_PASID_TABLE. RFC v1: https://lore.kernel.org/linux-iommu/1562324772-3084-1-git-send-email-yi.l.liu@intel.com/ Eric Auger (1): vfio: Document dual stage control Liu Yi L (13): vfio/type1: Refactor vfio_iommu_type1_ioctl() iommu: Report domain nesting info vfio/type1: Report iommu nesting info to userspace vfio: Add PASID allocation/free support iommu/vt-d: Support setting ioasid set to domain vfio/type1: Add VFIO_IOMMU_PASID_REQUEST (alloc/free) iommu/uapi: Add iommu_gpasid_unbind_data iommu/vt-d: Check ownership for PASIDs from user-space vfio/type1: Support binding guest page tables to PASID vfio/type1: Allow invalidating first-level/stage IOMMU cache vfio/type1: Add vSVA support for IOMMU-backed mdevs vfio/pci: Expose PCIe PASID capability to guest iommu/vt-d: Support reporting nesting capability info Yi Sun (1): iommu: Pass domain and unbind_data to sva_unbind_gpasid() Documentation/driver-api/vfio.rst | 64 ++++ drivers/iommu/intel-iommu.c | 107 ++++++- drivers/iommu/intel-svm.c | 20 +- drivers/iommu/iommu.c | 4 +- drivers/vfio/Kconfig | 6 + drivers/vfio/Makefile | 1 + drivers/vfio/pci/vfio_pci_config.c | 2 +- drivers/vfio/vfio_iommu_type1.c | 614 ++++++++++++++++++++++++++++++++----- drivers/vfio/vfio_pasid.c | 191 ++++++++++++ include/linux/intel-iommu.h | 23 +- include/linux/iommu.h | 10 +- include/linux/vfio.h | 54 ++++ include/uapi/linux/iommu.h | 47 +++ include/uapi/linux/vfio.h | 78 +++++ 14 files changed, 1134 insertions(+), 87 deletions(-) create mode 100644 drivers/vfio/vfio_pasid.c -- 2.7.4