Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Tue, 15 Mar 2022 11:35:35 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc:     iommu@lists.linux-foundation.org,
        LKML <linux-kernel@vger.kernel.org>,
        Joerg Roedel <joro@8bytes.org>,
        Christoph Hellwig <hch@infradead.org>,
        Lu Baolu <baolu.lu@linux.intel.com>,
        Jean-Philippe Brucker <jean-philippe@linaro.com>,
        Jacob Pan <jacob.jun.pan@intel.com>,
        Raj Ashok <ashok.raj@intel.com>,
        "Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
        Dave Jiang <dave.jiang@intel.com>,
        Tony Luck <tony.luck@intel.com>,
        "Zanussi, Tom" <tom.zanussi@intel.com>,
        Dan Williams <dan.j.williams@intel.com>,
        "Tian, Kevin" <kevin.tian@intel.com>, Yi Liu <yi.l.liu@intel.com>
Subject: Re: [PATCH v2 5/8] iommu: Add PASID support for DMA mapping API users
Message-ID: <20220315143535.GX11336@nvidia.com>
References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com>
 <20220315050713.2000518-6-jacob.jun.pan@linux.intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220315050713.2000518-6-jacob.jun.pan@linux.intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?fMR79ECQybKv1yK0PZLUHuFOpn2Hddm6+RWvLSUMUCSVk443gwJbQPs03XbO?=
 =?us-ascii?Q?NNAoyPP8+ZcJBh/VPxKcwkvHw+jqan2Zh5c8IgrCYs8gudfAVGq9KtutOSw5?=
 =?us-ascii?Q?x91ZJIwocf4M+cma94T324aDOZo0P0P0ca1tDzOdfe+cuZNCFf+V8NAfTgaO?=
 =?us-ascii?Q?PpBmX9b43xv3cGor8aYUmRd7o5xEJkp0cU//7VHXsGcO5vhR2vTEqHgE+BIC?=
 =?us-ascii?Q?+2FyRFXMipYkw/PzLcQWpNEfLKWGNO24ZVbK6pqWaJ5A9rDVF2/vagQcuXlZ?=
 =?us-ascii?Q?Xd7fLUN44C93g4yMO3LnO+XtHPrws8kucMaKcnV6Mt6KGwngTCO+VpSQi77Y?=
 =?us-ascii?Q?rv2scccl9wqVNBZ+EXpetbTRyHDMuiQyXUMUssFXERxu/H8QH6fLjJP9ruhk?=
 =?us-ascii?Q?peO1OV7MD3JuaBtmdTYHnSaumHw2OmRLmqeewuoPKW4bXtL+Svueu8IE1YZ3?=
 =?us-ascii?Q?Awhkj72VyTs8Ah9F+Lqxpd7dEAzWP2eDBHkavpy36VeuLuI5qhBL71WFftRc?=
 =?us-ascii?Q?utk+plE+xGG9TbbsRzZjKh1g6+bXlJJNHBZJuwTYMHhxQJYfYCkAme0nv79A?=
 =?us-ascii?Q?w/FXpnWeMxj6Zw7iw1+2vawOaZwnR7Bh7nm4eo30rMWp3AQTHnq35XBpxm32?=
 =?us-ascii?Q?ExXg48EFwL0C5TUIg/sSweef6fOcYWlNpYfi/xA/Q2mLe6GL6EAIv5t4mBPG?=
 =?us-ascii?Q?CsW5htu8iM+56H+JvkHNeRh7YNpTA/7GfBddt3YDQDEhgnM4ZTRSt6D2hF+K?=
 =?us-ascii?Q?nb1ZLfC/n9PjY1p7D+qg+Xmjn7Be9NJtaCbEX6QmZc7u7FW+3vEXWNtzqfwp?=
 =?us-ascii?Q?7H7ZmhKdn575nVDQwBJjRQ9QsH6BzkN9WTHzl5eBnSU6+ibYO+ylsWEZocnX?=
 =?us-ascii?Q?X+yHvwKlH7/qskSvYa45beyN54acunDtKisqEJGQ0vzfO2dCYsTppFSYATKH?=
 =?us-ascii?Q?Yn+osUgVrkC7C7YPAoAwHGaAB8ZhP2/vNXq/9ycB4PT4wQKxlin4smwBEC1P?=
 =?us-ascii?Q?I2KPlYsnUnjh6dsxJqUUGQ1RTQNaZWO9vkB4z8SiHcZB4Syu1fRgxr9tnuGi?=
 =?us-ascii?Q?H6uxf/JkRaw9GpFr4D2kHgtrxzHrLzyC02RwkwY/k4/o6H42NbOLIJdAo3IR?=
 =?us-ascii?Q?F8UODB0w+6uWSmF2I3u7QW2il1qFEYLn32fmWf9wECDrIzJtKicG73ZA6n8b?=
 =?us-ascii?Q?qmKOzVdSrTFlbdhW1s72PTd7xEFsPycDSATYvhoj/8kXhyhFYdEZuVzWkQV5?=
 =?us-ascii?Q?a9NLOSbUMdXiNt+zeXNOqSjz1+Rkj2Hx/7tqQicMZ25Mh7AcF8kUiPLw99pz?=
 =?us-ascii?Q?v6E/dJIdxFXB6pk8v85cEiC3myZsqWz+q0ofaQi54NNq3wY2cPdijIYwRV+a?=
 =?us-ascii?Q?Iww9xtU18OMHC3qSm/sVUj1wL7IHDAkpp/A69OkEoFSjuZS5Q84qP/x7KpZb?=
 =?us-ascii?Q?kQuaQ7QEK25FWC7DqnwFoqhuvP9J0S21?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ca7bb9b-b20e-420c-3a0e-08da069111ce
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Mar 2022 14:35:36.6973
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Zgx8Uind7fqolXKtE+6rN5Kd3gheNck3Perbb9YggTA2R8xUYeHctLl9222hukF6
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4824
X-Spam-Status: No, score=-3.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, Mar 14, 2022 at 10:07:09PM -0700, Jacob Pan wrote:
> DMA mapping API is the de facto standard for in-kernel DMA. It operates
> on a per device/RID basis which is not PASID-aware.
> 
> Some modern devices such as Intel Data Streaming Accelerator, PASID is
> required for certain work submissions. To allow such devices use DMA
> mapping API, we need the following functionalities:
> 1. Provide device a way to retrieve a PASID for work submission within
> the kernel
> 2. Enable the kernel PASID on the IOMMU for the device
> 3. Attach the kernel PASID to the device's default DMA domain, let it
> be IOVA or physical address in case of pass-through.
> 
> This patch introduces a driver facing API that enables DMA API
> PASID usage. Once enabled, device drivers can continue to use DMA APIs as
> is. There is no difference in dma_handle between without PASID and with
> PASID.
> 
> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
>  drivers/iommu/dma-iommu.c | 65 +++++++++++++++++++++++++++++++++++++++
>  include/linux/dma-iommu.h |  7 +++++
>  include/linux/iommu.h     |  9 ++++++
>  3 files changed, 81 insertions(+)
> 
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index b22034975301..d0ff1a34b1b6 100644
> +++ b/drivers/iommu/dma-iommu.c
> @@ -39,6 +39,8 @@ enum iommu_dma_cookie_type {
>  	IOMMU_DMA_MSI_COOKIE,
>  };
>  
> +static DECLARE_IOASID_SET(iommu_dma_pasid);
> +
>  struct iommu_dma_cookie {
>  	enum iommu_dma_cookie_type	type;
>  	union {
> @@ -370,6 +372,69 @@ void iommu_put_dma_cookie(struct iommu_domain *domain)
>  	domain->iova_cookie = NULL;
>  }
>  
> +/**
> + * iommu_enable_pasid_dma --Enable in-kernel DMA request with PASID
> + * @dev:	Device to be enabled
> + *
> + * DMA request with PASID will be mapped the same way as the legacy DMA.
> + * If the device is in pass-through, PASID will also pass-through. If the
> + * device is in IOVA map, the supervisor PASID will point to the same IOVA
> + * page table.
> + *
> + * @return the kernel PASID to be used for DMA or INVALID_IOASID on failure
> + */
> +int iommu_enable_pasid_dma(struct device *dev, ioasid_t *pasid)
> +{
> +	struct iommu_domain *dom;
> +	ioasid_t id, max;
> +	int ret;
> +
> +	dom = iommu_get_domain_for_dev(dev);
> +	if (!dom || !dom->ops || !dom->ops->attach_dev_pasid)
> +		return -ENODEV;

Given the purpose of this API I think it should assert that the device
has the DMA API in-use using the machinery from the other series.

ie this should not be used to clone non-DMA API iommu_domains..

> diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
> index 24607dc3c2ac..e6cb9b52a420 100644
> +++ b/include/linux/dma-iommu.h
> @@ -18,6 +18,13 @@ int iommu_get_dma_cookie(struct iommu_domain *domain);
>  int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base);
>  void iommu_put_dma_cookie(struct iommu_domain *domain);
>  
> +/*
> + * For devices that can do DMA request with PASID, setup a system PASID.
> + * Address modes (IOVA, PA) are selected by the platform code.
> + */
> +int iommu_enable_pasid_dma(struct device *dev, ioasid_t *pasid);
> +void iommu_disable_pasid_dma(struct device *dev, ioasid_t pasid);

The functions already have a kdoc, don't need two..

Jason