Message-ID: <43e9711f-042e-eaf9-e091-07db78cad943@intel.com>
Date:   Thu, 2 Mar 2023 21:52:42 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Firefox/102.0 Thunderbird/102.7.1
Subject: Re: [PATCH v4 3/6] iommu/sva: Stop using ioasid_set for SVA
To:     Jacob Pan <jacob.jun.pan@linux.intel.com>,
        LKML <linux-kernel@vger.kernel.org>, <iommu@lists.linux.dev>,
        Jason Gunthorpe <jgg@nvidia.com>,
        Lu Baolu <baolu.lu@linux.intel.com>,
        Joerg Roedel <joro@8bytes.org>,
        Jean-Philippe Brucker <jean-philippe@linaro.com>,
        "Dave Hansen" <dave.hansen@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        "X86 Kernel" <x86@kernel.org>, <bp@alien8.de>,
        "H. Peter Anvin" <hpa@zytor.com>,
        Peter Zijlstra <peterz@infradead.org>, <corbet@lwn.net>,
        <vkoul@kernel.org>, <dmaengine@vger.kernel.org>,
        <linux-doc@vger.kernel.org>
CC:     Robin Murphy <robin.murphy@arm.com>, Will Deacon <will@kernel.org>,
        "David Woodhouse" <dwmw2@infradead.org>,
        Raj Ashok <ashok.raj@intel.com>,
        "Tian, Kevin" <kevin.tian@intel.com>, Yi Liu <yi.l.liu@intel.com>,
        "Yu, Fenghua" <fenghua.yu@intel.com>,
        Dave Jiang <dave.jiang@intel.com>,
        Kirill Shutemov <kirill.shutemov@linux.intel.com>
References: <20230301235646.2692846-1-jacob.jun.pan@linux.intel.com>
 <20230301235646.2692846-4-jacob.jun.pan@linux.intel.com>
Content-Language: en-US
From:   Tina Zhang <tina.zhang@intel.com>
In-Reply-To: <20230301235646.2692846-4-jacob.jun.pan@linux.intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dHh0ZmxjV0wxaHJqNGFFMFVWNjZUdEVQMEV0NHo1QWVYR0lEODM5bmR0T1Jz?=
 =?utf-8?B?T1RlNFZUSmFIVVFyUU1iTjdUMjdoWCtjK3BHa05rdDNYSzBWdlBQQVF3MUM4?=
 =?utf-8?B?bmVySXM0eWp6dEFSRERwYkUyMENJdUhOUDFkTXJMSEtINWhTK3RJSVJpRm9v?=
 =?utf-8?B?Y2o0a3N0eW16MTF2aVg3VUh5UGlOSXhzVVR1MWNzZzVUc0Z2Qzc5OVFWTnBB?=
 =?utf-8?B?bEhXMFBTeFphZG9BbGtSZmNTM0E1bE4vQVFQZGVCckpBZFBVR1NFSkkzRlJ4?=
 =?utf-8?B?UHU0MWtTdm9iUEJIWjg2b3YzZThBMXJ6cVdENXRjeldPS0tBeFN6ampETEZk?=
 =?utf-8?B?dm9pcFBqOVUzRFZMVGltbGN1VWNBUElhakJQRWZwRWtjNUFOaHRMMTlwMzdT?=
 =?utf-8?B?U1RLbG1pN24zTFp3WlpyUE5Fb3M4UER4OTUxSjdIWFZXRGYxRVd2S3F6K0Vw?=
 =?utf-8?B?U3pVS1ZGTTBVd2RnT0dFU3Zpb2NrTTJ2MzVNdk9xTmdCNmw4K3Fxc2c1VTYx?=
 =?utf-8?B?R01EeUpOL045VVdyODFuTUg4NEx3NHdqZjQwdjRnaFB3WEticFZQUXhiY0dj?=
 =?utf-8?B?S0gwaGZWMGhTR0dwdFhFUm1vQWdxdjVnZ0E0Z0ZaaU52c3Q3WmVZUllibEgz?=
 =?utf-8?B?VU5Xbm8zWEJ1ak9zL1JRTGtLb2RyanlmYStMRWxReHhMYk5tZ2dJK1dnUkUx?=
 =?utf-8?B?aDhDZXcrY1JndE5hQk9EVkZBM3N0VklEdXAyMnlSSmE3b2VJRThQekNoWTdV?=
 =?utf-8?B?Q3NKeHY2UDJtd01naVEzSkIzdXlJdDNNVTcrc00yRHlEejVaL3huMzdJWlNt?=
 =?utf-8?B?a2ZrUC9hSmNDTHpwSUdBOHkzRkxCd1ZuWXNjVWx3SzJ0aGx0cVVRUDYyUzRp?=
 =?utf-8?B?SE01WlZPNWFwRXRSQlQrenlOc0U2VFg5S3pvd2lhS1Bvb2lYZU81REJESVlH?=
 =?utf-8?B?RWhldXQ3YVBUdjRQdHFiREY2MEdXeFlEM1lUQnJnWWdhY2dGVVpCazZVaHJz?=
 =?utf-8?B?eFZ0ZGkyWDMwcmJXRnBxcVdwZVRRVmlvRjhDTURGWVVnRitBQmxLbVQrZld3?=
 =?utf-8?B?UmI5YVRPM3FrR1hpay9lMUZQaGVxMHljUmMwb2VrWFRLUVJGQTVmQklBeFk5?=
 =?utf-8?B?TElDbWdFYjFFbGhzZmFjZ2RJSkpmZDdmUmp1RTVPS1RDYklWazhiVTErN1Fz?=
 =?utf-8?B?OGJnaFFMS1IrTUZzWmI3b01FYzZKQ2w1ZERuT2s5bEN6ZnFhYjlVS2tndHVl?=
 =?utf-8?B?SDkydWl0ekxFaXUyN1QxVWo5VDEvYUszQm16OUNsSktkb0YzbUVNUEdaWHVm?=
 =?utf-8?B?bkVHUWNGeHMrU05sem1Kb2x5UjZqOE9iUlNlNDM2bEFYclVaWlA3STlpbEpn?=
 =?utf-8?B?SGdNenhWd1Rhci9pL2dGNk10WUlKa2ZVR0dyUDRmVUpPQ3FNQUxqWStoN3Nx?=
 =?utf-8?B?ZUFHVUdHSG4rYWRNUVhnN29XOCtwMGdERW5UbEFENGNGeFNkeXQrbGxQQWp1?=
 =?utf-8?B?bTBweTJlbmFNZ0VZbVg5ZlRKSVZjM2dLKzQzeWxpcHlBanhMUjZOWkMzV3ZV?=
 =?utf-8?B?RUY2d2JteFRHdTlzQU1VdG1sMVh0TWxMeVhrM2g4djMrcURPNmNXcC9NN1Ra?=
 =?utf-8?B?dXY3OXhQdGNOVEJZM0JYWnlRSmFaZGlrRWNqMmNCNlhyRldqczh2S1RaNksw?=
 =?utf-8?B?TWlkSHoyNHRaN0l5YVdRQnR0NFFCSUkzanJ3NG5hVSs1c2xtV1ptRXVtQXg3?=
 =?utf-8?B?aDFMbi9iVjFvNTduU2M5YlhRWnRNc2xzcVZ3bUNuOVlTamRVK0NNQklVd0cv?=
 =?utf-8?B?TVZBQ01kb0tVa1Bod1BxTWlzaXE3M2dscjJtZVhEaDJqNklZcTd1bjFOSXU3?=
 =?utf-8?B?UllXcnZ3N090b2JTeHdXT0dEbzA3b0NkRXdjZzhRMk40YVdTU1ZBOVNXVGl1?=
 =?utf-8?B?VUw0WHlJdkJVRmdhZXFUTnA2cXRORDB6aVQvSENlZkdzblBNZG9USmJQMEJS?=
 =?utf-8?B?bXBjcjZNdmovV3A4NWZydXhxdXMwWmp1NVBoam1Tc3hQZU5BSm05ckd5SzRX?=
 =?utf-8?B?NUc2SzJqUTZGQW1yUlh2c2hyT2U0SWVzN0YvOGlDTi9CNFNEUDFJQnMzNVoz?=
 =?utf-8?B?N0xpYWxOcllrVEN2b3R5MHVBek5nNC93enI2U3puRjJQQy8vUSt6b2gxVDlW?=
 =?utf-8?B?V1E9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 95c5681a-7dc4-49b9-6351-08db1b256d5e
X-MS-Exchange-CrossTenant-AuthSource: MW5PR11MB5881.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Mar 2023 13:52:56.9930
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RH/5AQY9dFBUE+Z/k42exoDq7gmVMVX9WrfPf4CBFF2CJFUCkel1TGXr3e7J3fAU4yDTD8bzxwpcarl8lInK+w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB5023
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi,

On 3/2/23 07:56, Jacob Pan wrote:
> From: Jason Gunthorpe <jgg@nvidia.com>
> 
> Instead SVA drivers can use a simple global IDA to allocate PASIDs for
> each mm_struct.
> 
> Future work would be to allow drivers using the SVA APIs to reserve global
> PASIDs from this IDA for their internal use, eg with the DMA API PASID
> support.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
> ---
> v4:
> 	- Keep GFP_ATOMIC flag for PASID allocation, will changed to
> 	GFP_KERNEL in a separate patch.
> ---
>   drivers/iommu/iommu-sva.c | 62 ++++++++++-----------------------------
>   drivers/iommu/iommu-sva.h |  3 --
>   2 files changed, 15 insertions(+), 50 deletions(-)
> 
> diff --git a/drivers/iommu/iommu-sva.c b/drivers/iommu/iommu-sva.c
> index 376b2a9e2543..297852ae5e7c 100644
> --- a/drivers/iommu/iommu-sva.c
> +++ b/drivers/iommu/iommu-sva.c
> @@ -9,26 +9,13 @@
>   #include "iommu-sva.h"
>   
>   static DEFINE_MUTEX(iommu_sva_lock);
> -static DECLARE_IOASID_SET(iommu_sva_pasid);
> +static DEFINE_IDA(iommu_global_pasid_ida);
>   
> -/**
> - * iommu_sva_alloc_pasid - Allocate a PASID for the mm
> - * @mm: the mm
> - * @min: minimum PASID value (inclusive)
> - * @max: maximum PASID value (inclusive)
> - *
> - * Try to allocate a PASID for this mm, or take a reference to the existing one
> - * provided it fits within the [@min, @max] range. On success the PASID is
> - * available in mm->pasid and will be available for the lifetime of the mm.
> - *
> - * Returns 0 on success and < 0 on error.
> - */
> -int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max)
> +static int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max)
>   {
> -	int ret = 0;
> -	ioasid_t pasid;
> +	int ret;
>   
> -	if (min == INVALID_IOASID || max == INVALID_IOASID ||
> +	if (min == IOMMU_PASID_INVALID || max == IOMMU_PASID_INVALID ||
>   	    min == 0 || max < min)
>   		return -EINVAL;
>   
> @@ -37,39 +24,20 @@ int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max)
>   	if (pasid_valid(mm->pasid)) {
>   		if (mm->pasid < min || mm->pasid >= max)
Here seems not right, since the valid range is defined [min, max]. 
Shouldn't the invalid range be:
		if (mm->pasid < min || mm->pasid > max)

Regards,
-Tina

>   			ret = -EOVERFLOW;
> +		else
> +			ret = 0;
>   		goto out;
>   	}
>   
> -	pasid = ioasid_alloc(&iommu_sva_pasid, min, max, mm);
> -	if (!pasid_valid(pasid))
> -		ret = -ENOMEM;
> -	else
> -		mm->pasid = ret;
> +	ret = ida_alloc_range(&iommu_global_pasid_ida, min, max, GFP_ATOMIC);
> +	if (ret < min)
> +		goto out;
> +	mm->pasid = ret;
> +	ret = 0;
>   out:
>   	mutex_unlock(&iommu_sva_lock);
>   	return ret;
>   }
> -EXPORT_SYMBOL_GPL(iommu_sva_alloc_pasid);
> -
> -/* ioasid_find getter() requires a void * argument */
> -static bool __mmget_not_zero(void *mm)
> -{
> -	return mmget_not_zero(mm);
> -}
> -
> -/**
> - * iommu_sva_find() - Find mm associated to the given PASID
> - * @pasid: Process Address Space ID assigned to the mm
> - *
> - * On success a reference to the mm is taken, and must be released with mmput().
> - *
> - * Returns the mm corresponding to this PASID, or an error if not found.
> - */
> -struct mm_struct *iommu_sva_find(ioasid_t pasid)
> -{
> -	return ioasid_find(&iommu_sva_pasid, pasid, __mmget_not_zero);
> -}
> -EXPORT_SYMBOL_GPL(iommu_sva_find);
>   
>   /**
>    * iommu_sva_bind_device() - Bind a process address space to a device
> @@ -241,8 +209,8 @@ iommu_sva_handle_iopf(struct iommu_fault *fault, void *data)
>   
>   void mm_pasid_drop(struct mm_struct *mm)
>   {
> -	if (pasid_valid(mm->pasid)) {
> -		ioasid_free(mm->pasid);
> -		mm->pasid = INVALID_IOASID;
> -	}
> +	if (likely(!pasid_valid(mm->pasid)))
> +		return;
> +
> +	ida_free(&iommu_global_pasid_ida, mm->pasid);
>   }
> diff --git a/drivers/iommu/iommu-sva.h b/drivers/iommu/iommu-sva.h
> index 7215a761b962..c22d0174ad61 100644
> --- a/drivers/iommu/iommu-sva.h
> +++ b/drivers/iommu/iommu-sva.h
> @@ -8,9 +8,6 @@
>   #include <linux/ioasid.h>
>   #include <linux/mm_types.h>
>   
> -int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max);
> -struct mm_struct *iommu_sva_find(ioasid_t pasid);
> -
>   /* I/O Page fault */
>   struct device;
>   struct iommu_fault;