Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
Message-ID: <a5bc36fd-dcbe-4b09-b9e3-90578871f776@intel.com>
Date:   Tue, 12 Dec 2023 11:02:13 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/3] vfio: Add VFIO_DEVICE_PASID_[AT|DE]TACH_IOMMUFD_PT
Content-Language: en-US
To:     Alex Williamson <alex.williamson@redhat.com>
CC:     <joro@8bytes.org>, <jgg@nvidia.com>, <kevin.tian@intel.com>,
        <robin.murphy@arm.com>, <baolu.lu@linux.intel.com>,
        <cohuck@redhat.com>, <eric.auger@redhat.com>,
        <nicolinc@nvidia.com>, <kvm@vger.kernel.org>,
        <mjrosato@linux.ibm.com>, <chao.p.peng@linux.intel.com>,
        <yi.y.sun@linux.intel.com>, <peterx@redhat.com>,
        <jasowang@redhat.com>, <shameerali.kolothum.thodi@huawei.com>,
        <lulu@redhat.com>, <suravee.suthikulpanit@amd.com>,
        <iommu@lists.linux.dev>, <linux-kernel@vger.kernel.org>,
        <linux-kselftest@vger.kernel.org>, <zhenzhong.duan@intel.com>,
        <joao.m.martins@oracle.com>, <xin.zeng@intel.com>,
        <yan.y.zhao@intel.com>
References: <20231127063909.129153-1-yi.l.liu@intel.com>
 <20231127063909.129153-3-yi.l.liu@intel.com>
 <20231211100501.1c4032ce.alex.williamson@redhat.com>
From:   Yi Liu <yi.l.liu@intel.com>
In-Reply-To: <20231211100501.1c4032ce.alex.williamson@redhat.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?elloeGFNR2tMeUZKUVJzL0NMNGQ5MkthYTQ1aW5Xckh2aHVUWEdHbkhVRmhQ?=
 =?utf-8?B?enh3SzZwQTBJdUJ5SzdoYnMrb0tGVjdVT0g3dDg4MU1tdzNyR0sxbnBFQU9K?=
 =?utf-8?B?VG9kVmJqK1RmOHB4aXp0RXFDVEhQYlpBUEFJcit3YXdINEgyYUhFV2d3ZFFL?=
 =?utf-8?B?c2hPSzF0OXMreE9xTURpWE9PczV1bzYzM25PTCtCQXF1eDcxbVVuWkhSTWUv?=
 =?utf-8?B?Z29NVzFyaDVlVE0zMTBzSW1YWThTUjZLMFNsZkd6R1hWYnk5WWdJbUJScjBh?=
 =?utf-8?B?ZnNLcjZTV1ZWdmdjNllPR3h4bnhscEl5MUhTYm14V0FndllLN01nbWg1Sm1Z?=
 =?utf-8?B?dzdseTdDU25RUEtPYkhEMDVlMmRQK00zZWlrSE9DMEVqWFdnS2ErSVZRQ2c4?=
 =?utf-8?B?TWFMYWFOWVpZenNnVW15ZUtIWnJPVldScGNSNW45TmpoaHlMTkRBWm9uMDNq?=
 =?utf-8?B?bGt5dkFCVTBCdWw4aXZPZkE3dzhKTWRMWFFwRzB2MUJVOWR1OTVpYTJIZ3Nz?=
 =?utf-8?B?L2FDK2VlTUt2c0ZIaEliY3FaaW1sbTVIQk5yUWJ4emgrUlRvVXo1ZW0xdzdY?=
 =?utf-8?B?S1lKS3ltcHlOSk9EOWpUM3A0ZjZUZzdaUER6MEJkSzRHaWdvMGVvWUVEM1oz?=
 =?utf-8?B?bW5VQU1ZbmlIaWk1djkybVBCUG55YWxUUEUvTXREK1VVYU5RU0w2K0ZkVnE2?=
 =?utf-8?B?OHVlUHRUN0pOYmRrcjlqME9lZ1dlYVlOK1pFb01xNkh1NmRUNWI0SUtlTlFw?=
 =?utf-8?B?Vy9idlN5MmJaQkFDM0x2WEZjVE01Z2xVSmphK3NDREVuWk1TV0oyREpVcU5S?=
 =?utf-8?B?SllXVlo1SzhVVU40bzE2eTlBd2N4clNLbW9JNHU2KzZkZllaY0licFo0YTlk?=
 =?utf-8?B?V0FKSGNVb3MrNndEMEdVTXFpK3Z5SmF1bmduZ3dkZmpWRDlBa3p1VTZPSzlP?=
 =?utf-8?B?a0Z0eDFIT1p2L3pCd0Y4L0hoNlF4c2JnTXBnc05FUGtwM2l2eURMWHVXNlFl?=
 =?utf-8?B?ZFdnR21jR3pPUjRzdjRDVGpTQkNWMVlyRUlIODRhd1g0R1lhbTRDQUQ0Y3kx?=
 =?utf-8?B?T1M2a281S3VzMHhmZnMyK2k2cmp0ZkNKZE1MRkZnWXFOYmt6RlJ0aUFDWk5p?=
 =?utf-8?B?aEY2UnFUWnVWdzZDZjlHVGtZWG9UT1ZTR3ZJRzd3VCtKMCtXNU1zWXBab0Zv?=
 =?utf-8?B?T2QvQXo3N2dkZ0ZFam1Qd2JOb1l2Z1BJVkZhR3lzZDFLSUFVWGxvaDRHWUFE?=
 =?utf-8?B?akVUWURjdjl4a2JPV21zQ2RHa1RleE5KcExoSFYyNE5QSk91TGNDSWRvL2N4?=
 =?utf-8?B?SHIyeDU2SWVibDBLRk9EWnJPM000cmo3eTBJdVlBOTdGa3UvWnptMHExbVhh?=
 =?utf-8?B?RG5MK1BoNUVEdmpTbE1IeklGcTRLMGZwNCthUG1xb0tCRlgvL0l6VTNmemVv?=
 =?utf-8?B?Mlg4Y0d0bDdMRzdqWXVqYjFwamlrNWRjelNjSkw0Y0c1bk4vREdFR090RkhJ?=
 =?utf-8?B?RkY5dUlSaXVRdHgwVnVjWnczWWtlRFI2MFdPbEVEc1YxVHN3d0twZjdUMHZ4?=
 =?utf-8?B?cER0VXVlbDRjUEdRa0N4NkdCZU1jUkcvdWM1Y0YwVXF4dDFxTmRFNFNrN20x?=
 =?utf-8?B?aWI5eElMNGtMSEdpVVJaSlEyRFJ3NkFOaldoT0QyTk5xSkJLZ0F4UW1ZMmVE?=
 =?utf-8?B?SVEyMWlueVoxV0hnbjJYYWJjSG9keEZDdjc4MmRyVlB1MXZIN3FFRVFwYjM0?=
 =?utf-8?B?a01TR0xwNDliVTZWdTlWRXpYbThsbXdxRzV5MWQvVjlyZExPb25GVDlMaHM2?=
 =?utf-8?B?b3lwU0FjakR4S054Z3ZSaE1NQUZNd3BsaDYvS0ExZWl0RW15TkFPY015YjZV?=
 =?utf-8?B?ZHMrZnhwK21MRkViNUNKVldzLzl6Z2JHckVENVJZRFpEVEFXSnhiN0VTUjFU?=
 =?utf-8?B?bTE3c1hTYzgyNDN6OEpvNXZsb1NvN0MvcVNsejhJUnAvQkNvd3kyVFBmRHln?=
 =?utf-8?B?ZFAvenZHUnkxaEFsdXJsVGVTeVEyVmF5V3FDdXlVR09qUFN0WUdMcXZCQWsv?=
 =?utf-8?B?Q2gzM1piM2RKeElDb1dJNDQyMGNaRVpMRGsxMzVVQTczWDlKQTFuaWtscEVn?=
 =?utf-8?Q?y/Y5FkHUE9rF301+RZMrGU8RN?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 79492446-7adc-4d44-5915-08dbfabe61dc
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7529.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2023 02:59:39.8990
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qaWaHOykDXM0NonHYNXvORpgDn4TNYTjeG6viL+DarBAFBs547/LGhu0vMfJ6aJiwabSXUq9XM6g/2jcGCLonw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR11MB7353
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 11 Dec 2023 19:00:09 -0800 (PST)

On 2023/12/12 01:05, Alex Williamson wrote:
> On Sun, 26 Nov 2023 22:39:08 -0800
> Yi Liu <yi.l.liu@intel.com> wrote:
> 
>> This adds ioctls for the userspace to attach a given pasid of a vfio
>> device to/from an IOAS/HWPT.
>>
>> Signed-off-by: Yi Liu <yi.l.liu@intel.com>
>> ---
>>   drivers/vfio/device_cdev.c | 45 +++++++++++++++++++++++++++++++
>>   drivers/vfio/vfio.h        |  4 +++
>>   drivers/vfio/vfio_main.c   |  8 ++++++
>>   include/uapi/linux/vfio.h  | 55 ++++++++++++++++++++++++++++++++++++++
>>   4 files changed, 112 insertions(+)
>>
>> diff --git a/drivers/vfio/device_cdev.c b/drivers/vfio/device_cdev.c
>> index e75da0a70d1f..c2ac7ed44537 100644
>> --- a/drivers/vfio/device_cdev.c
>> +++ b/drivers/vfio/device_cdev.c
>> @@ -210,6 +210,51 @@ int vfio_df_ioctl_detach_pt(struct vfio_device_file *df,
>>   	return 0;
>>   }
>>   
>> +int vfio_df_ioctl_pasid_attach_pt(struct vfio_device_file *df,
>> +				  struct vfio_device_pasid_attach_iommufd_pt __user *arg)
>> +{
>> +	struct vfio_device *device = df->device;
>> +	struct vfio_device_pasid_attach_iommufd_pt attach;
>> +	unsigned long minsz;
>> +	int ret;
>> +
>> +	minsz = offsetofend(struct vfio_device_pasid_attach_iommufd_pt, pt_id);
>> +
>> +	if (copy_from_user(&attach, arg, minsz))
>> +		return -EFAULT;
>> +
>> +	if (attach.argsz < minsz || attach.flags)
>> +		return -EINVAL;
>> +
>> +	mutex_lock(&device->dev_set->lock);
>> +	ret = device->ops->pasid_attach_ioas(device, attach.pasid, &attach.pt_id);
> 
> These callbacks were only implemented for vfio-pci in the previous
> patch but they're called unconditionally.  Thanks,

yes, will correct it and below. thanks for catching it.

> 
> Alex
> 
>> +	mutex_unlock(&device->dev_set->lock);
>> +
>> +	return ret;
>> +}
>> +
>> +int vfio_df_ioctl_pasid_detach_pt(struct vfio_device_file *df,
>> +				  struct vfio_device_pasid_detach_iommufd_pt __user *arg)
>> +{
>> +	struct vfio_device *device = df->device;
>> +	struct vfio_device_pasid_detach_iommufd_pt detach;
>> +	unsigned long minsz;
>> +
>> +	minsz = offsetofend(struct vfio_device_pasid_detach_iommufd_pt, flags);
>> +
>> +	if (copy_from_user(&detach, arg, minsz))
>> +		return -EFAULT;
>> +
>> +	if (detach.argsz < minsz || detach.flags)
>> +		return -EINVAL;
>> +
>> +	mutex_lock(&device->dev_set->lock);
>> +	device->ops->pasid_detach_ioas(device, detach.pasid);
>> +	mutex_unlock(&device->dev_set->lock);
>> +
>> +	return 0;
>> +}
>> +
>>   static char *vfio_device_devnode(const struct device *dev, umode_t *mode)
>>   {
>>   	return kasprintf(GFP_KERNEL, "vfio/devices/%s", dev_name(dev));
>> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
>> index 307e3f29b527..d228cdb6b345 100644
>> --- a/drivers/vfio/vfio.h
>> +++ b/drivers/vfio/vfio.h
>> @@ -353,6 +353,10 @@ int vfio_df_ioctl_attach_pt(struct vfio_device_file *df,
>>   			    struct vfio_device_attach_iommufd_pt __user *arg);
>>   int vfio_df_ioctl_detach_pt(struct vfio_device_file *df,
>>   			    struct vfio_device_detach_iommufd_pt __user *arg);
>> +int vfio_df_ioctl_pasid_attach_pt(struct vfio_device_file *df,
>> +				  struct vfio_device_pasid_attach_iommufd_pt __user *arg);
>> +int vfio_df_ioctl_pasid_detach_pt(struct vfio_device_file *df,
>> +				  struct vfio_device_pasid_detach_iommufd_pt __user *arg);
>>   
>>   #if IS_ENABLED(CONFIG_VFIO_DEVICE_CDEV)
>>   void vfio_init_device_cdev(struct vfio_device *device);
>> diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
>> index 8d4995ada74a..ff50c239873d 100644
>> --- a/drivers/vfio/vfio_main.c
>> +++ b/drivers/vfio/vfio_main.c
>> @@ -1240,6 +1240,14 @@ static long vfio_device_fops_unl_ioctl(struct file *filep,
>>   		case VFIO_DEVICE_DETACH_IOMMUFD_PT:
>>   			ret = vfio_df_ioctl_detach_pt(df, uptr);
>>   			goto out;
>> +
>> +		case VFIO_DEVICE_PASID_ATTACH_IOMMUFD_PT:
>> +			ret = vfio_df_ioctl_pasid_attach_pt(df, uptr);
>> +			goto out;
>> +
>> +		case VFIO_DEVICE_PASID_DETACH_IOMMUFD_PT:
>> +			ret = vfio_df_ioctl_pasid_detach_pt(df, uptr);
>> +			goto out;
>>   		}
>>   	}
>>   
>> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
>> index 94b3badefde3..495193629029 100644
>> --- a/include/uapi/linux/vfio.h
>> +++ b/include/uapi/linux/vfio.h
>> @@ -977,6 +977,61 @@ struct vfio_device_detach_iommufd_pt {
>>   
>>   #define VFIO_DEVICE_DETACH_IOMMUFD_PT		_IO(VFIO_TYPE, VFIO_BASE + 20)
>>   
>> +/*
>> + * VFIO_DEVICE_PASID_ATTACH_IOMMUFD_PT - _IOW(VFIO_TYPE, VFIO_BASE + 21,
>> + *					      struct vfio_device_pasid_attach_iommufd_pt)
>> + * @argsz:	User filled size of this data.
>> + * @flags:	Must be 0.
>> + * @pasid:	The pasid to be attached.
>> + * @pt_id:	Input the target id which can represent an ioas or a hwpt
>> + *		allocated via iommufd subsystem.
>> + *		Output the input ioas id or the attached hwpt id which could
>> + *		be the specified hwpt itself or a hwpt automatically created
>> + *		for the specified ioas by kernel during the attachment.
>> + *
>> + * Associate a pasid (of a cdev device) with an address space within the
>> + * bound iommufd. Undo by VFIO_DEVICE_PASID_DETACH_IOMMUFD_PT or device fd
>> + * close. This is only allowed on cdev fds.
>> + *
>> + * If a pasid is currently attached to a valid hw_pagetable (hwpt), without
>> + * doing a VFIO_DEVICE_PASID_DETACH_IOMMUFD_PT, a second
>> + * VFIO_DEVICE_PASID_ATTACH_IOMMUFD_PT ioctl passing in another hwpt id is
>> + * allowed. This action, also known as a hwpt replacement, will replace the
>> + * pasid's currently attached hwpt with a new hwpt corresponding to the given
>> + * @pt_id.
>> + *
>> + * Return: 0 on success, -errno on failure.
>> + */
>> +struct vfio_device_pasid_attach_iommufd_pt {
>> +	__u32	argsz;
>> +	__u32	flags;
>> +	__u32	pasid;
>> +	__u32	pt_id;
>> +};
>> +
>> +#define VFIO_DEVICE_PASID_ATTACH_IOMMUFD_PT	_IO(VFIO_TYPE, VFIO_BASE + 21)
>> +
>> +/*
>> + * VFIO_DEVICE_PASID_DETACH_IOMMUFD_PT - _IOW(VFIO_TYPE, VFIO_BASE + 22,
>> + *					      struct vfio_device_pasid_detach_iommufd_pt)
>> + * @argsz:	User filled size of this data.
>> + * @flags:	Must be 0.
>> + * @pasid:	The pasid to be detached.
>> + *
>> + * Remove the association of a pasid (of a cdev device) and its current
>> + * associated address space.  After it, the pasid of the device should be in
>> + * a blocking DMA state.  This is only allowed on cdev fds.
>> + *
>> + * Return: 0 on success, -errno on failure.
>> + */
>> +struct vfio_device_pasid_detach_iommufd_pt {
>> +	__u32	argsz;
>> +	__u32	flags;
>> +	__u32	pasid;
>> +};
>> +
>> +#define VFIO_DEVICE_PASID_DETACH_IOMMUFD_PT	_IO(VFIO_TYPE, VFIO_BASE + 22)
>> +
>>   /*
>>    * Provide support for setting a PCI VF Token, which is used as a shared
>>    * secret between PF and VF drivers.  This feature may only be set on a
> 

-- 
Regards,
Yi Liu