From:   Jacob Pan <jacob.jun.pan@linux.intel.com>
To:     iommu@lists.linux-foundation.org,
        LKML <linux-kernel@vger.kernel.org>,
        Joerg Roedel <joro@8bytes.org>,
        Jason Gunthorpe <jgg@nvidia.com>,
        "Christoph Hellwig" <hch@infradead.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Jean-Philippe Brucker <jean-philippe@linaro.com>
Cc:     Jacob Pan <jacob.jun.pan@intel.com>,
        "Lu Baolu" <baolu.lu@linux.intel.com>,
        Raj Ashok <ashok.raj@intel.com>,
        "Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
        Dave Jiang <dave.jiang@intel.com>,
        Tony Luck <tony.luck@intel.com>, Yi Liu <yi.l.liu@intel.com>,
        "Tian, Kevin" <kevin.tian@intel.com>,
        Barry Song <21cnbao@gmail.com>,
        "Zanussi, Tom" <tom.zanussi@intel.com>,
        Dan Williams <dan.j.williams@intel.com>
Subject: [PATCH 0/4] Enable PASID for DMA API users
Date:   Tue,  7 Dec 2021 05:47:10 -0800
Message-Id: <1638884834-83028-1-git-send-email-jacob.jun.pan@linux.intel.com>
Precedence: bulk

Modern accelerators such as Intel's Data Streaming Accelerator (DSA) can
perform DMA requests with PASID, which is a finer granularity than the
device's requester ID(RID). In fact, work submissions on DSA shared work
queues require PASID.

DMA mapping API is the de facto standard for in-kernel DMA. However, it
operates on a per device/RID basis which is not PASID-aware.

This patch introduces the following driver facing API that enables DMA API
PASID usage: ioasid_t iommu_enable_pasid_dma(struct device *dev);

A PASID field is added to struct device for the purposes of storing kernel
DMA PASID and flushing device IOTLBs. A separate use case in interrupt
message store (IMS) also hinted adding a PASID field to struct device.
https://lore.kernel.org/all/87pmx73tfw.ffs@nanos.tec.linutronix.de/
IMS virtualization and DMA API does not overlap.

Once enabled, device drivers can continue to use DMA APIs as-is. There is
no difference in terms of mapping in dma_handle between without PASID and
with PASID.  The DMA mapping performed by IOMMU will be identical for both
requests with and without PASID (legacy), let it be IOVA or PA in case of
pass-through.

In addition, this set converts the current support for in-kernel PASID DMA
from SVA lib to DMA API. There have been security and functional issues
with the kernel SVA approach:
(https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/)
The highlights are as the following:
 - The lack of IOTLB synchronization upon kernel page table updates.
   (vmalloc, module/BPF loading, CONFIG_DEBUG_PAGEALLOC etc.)
 - Other than slight more protection, using kernel virtual address (KVA)
 has little advantage over physical address.
There are also no use cases yet where DMA engines need kernel virtual
addresses for in-kernel DMA.

Once this set is accepted, more cleanup patches will follow. The plan is to
remove the usage of sva_bind_device() for in-kernel usages. Removing page
requests and other special cases around kernel SVA in VT-d driver.


Jacob Pan (4):
  ioasid: Reserve a global PASID for in-kernel DMA
  iommu: Add PASID support for DMA mapping API users
  iommu/vt-d: Support PASID DMA for in-kernel usage
  dmaengine: idxd: Use DMA API for in-kernel DMA with PASID

 .../admin-guide/kernel-parameters.txt         |   6 -
 drivers/dma/Kconfig                           |  10 --
 drivers/dma/idxd/idxd.h                       |   1 -
 drivers/dma/idxd/init.c                       |  59 +++-------
 drivers/dma/idxd/sysfs.c                      |   7 --
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |   2 +-
 drivers/iommu/dma-iommu.c                     |  71 ++++++++++++
 drivers/iommu/intel/iommu.c                   | 109 +++++++++++++++++-
 drivers/iommu/intel/pasid.c                   |   7 ++
 drivers/iommu/intel/pasid.h                   |   3 +-
 drivers/iommu/intel/svm.c                     |   2 +-
 drivers/iommu/ioasid.c                        |   2 +
 include/linux/device.h                        |   1 +
 include/linux/dma-iommu.h                     |   7 ++
 include/linux/intel-iommu.h                   |   3 +-
 include/linux/ioasid.h                        |   4 +
 include/linux/iommu.h                         |   4 +
 17 files changed, 226 insertions(+), 72 deletions(-)

-- 
2.25.1