Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
Message-ID: <12ebf6fc-9228-47a1-88dc-a177f7f7d5db@intel.com>
Date:   Mon, 6 Nov 2023 16:54:53 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC v1 6/8] vdpa: change the map/unmap process to support
 iommufd
Content-Language: en-US
To:     Cindy Lu <lulu@redhat.com>, <jasowang@redhat.com>,
        <mst@redhat.com>, <jgg@nvidia.com>, <linux-kernel@vger.kernel.org>,
        <virtualization@lists.linux-foundation.org>,
        <netdev@vger.kernel.org>
References: <20231103171641.1703146-1-lulu@redhat.com>
 <20231103171641.1703146-7-lulu@redhat.com>
From:   Yi Liu <yi.l.liu@intel.com>
In-Reply-To: <20231103171641.1703146-7-lulu@redhat.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?M0RhT2V1SEY5c0kxdFJhaXNtNVdqNEtLQklPYW1TK2J3SVFGaUpHY3d6ZmY2?=
 =?utf-8?B?ZUQyY2gva2RqZ0VWOW5BejlTemhIcDc1Q2hlVmRlRnJXaWlFQnRkSUNFLzRl?=
 =?utf-8?B?SnpsQVdTbjhCYnZ3MUhUdEw5d1FkREVlQXFxSUZnZHZUREhid2ZHL2Y5dlJO?=
 =?utf-8?B?VU9Ec3VyckhKUnBjbGJqVmVXdENVa0piTTRWeXg2MUhpYkpTY093dEJPaG1F?=
 =?utf-8?B?RGFITFdpUE8wbzZ3dzRuK3V5RU1weTdmYk5ybHlzSW0rOE8vMHB4SkJ6UFlN?=
 =?utf-8?B?QWlja0JwcU4zdGZaNm4rYjJkUUtwQ3Q5R3R6Vldob3lJMDhsUi8rVDlWT3Np?=
 =?utf-8?B?a0x6NWRRaFMvNEZUYndmU0w5TXplMW16ZWVRYVhvMnJRblBncUtoN0JkTUlR?=
 =?utf-8?B?N3UzV3hvei9nTWwyaXlVMXplWFMrRHplWTlTWFc3N0ZhTC95RXB6TkVoNDlR?=
 =?utf-8?B?YXF4dE5Rc2pVLzhrbGZ4eXRaVjBjYldtOVA4Q3ljdE03OVBxSWw5WGR5a2ZU?=
 =?utf-8?B?SlJqQ0hJYUdnaWd1MzY5dVhCMjdKRVRsbjZyNnZzbnhpVWpZN24ybU9nQnEw?=
 =?utf-8?B?bEx2b3hVanVuM1BFaTZRZkdVMmZGRHNjeThHMzNRYUUzNmhPaUl2bnNNamp3?=
 =?utf-8?B?c1NaNlhMTmloQmZpSFd3eGdvUHRQTVR3aUFmVWFsOE9oV2JCZytKS3dmWnBQ?=
 =?utf-8?B?VEZXWXFwRFBpaC9DdGZWN2RhcWhiZ0lIeXlYMDgzNStZRUZNa09UNmNCclU4?=
 =?utf-8?B?TWx1R2VIT3BEQ0V0d0VtT0RQa2drc3g5TW0yM3dTR3pGcFJucFkxY2RRTS9Y?=
 =?utf-8?B?TmV6THE4L1A1RUtUaTNVNnFVejFwb3pEOE9DdVBEYXVaRHJZZDNtd3NPQlhl?=
 =?utf-8?B?ZXhRZm1RMWphSHptZG0ybTB6bWV0QnlBTDFFcG9BekgyaTdTVU9ablQwM2Iz?=
 =?utf-8?B?Q0FVeTg0RjBPS2krc1czbkppVmhFOGhFMTFpSjFCenI5anRkWm1nYXRNWVZx?=
 =?utf-8?B?WVVFQWdSTE41d2hFVWkrNjFJYUR1c3RvaHlKT2oya2I3RDB3bjJ5OGluM1FC?=
 =?utf-8?B?cHFab0NjOERyNkFaTnBKODdPTlhscTc1RVRHUURpa1dwSVpTOWh3VHdXcmpZ?=
 =?utf-8?B?b09NekZSQTBYVi84UjR5RmlXV1N2bHE1T1I2VWQyZFdRdUhqQzA5TVpBZUJE?=
 =?utf-8?B?bVR3bERBTFdvTmRiWFd1R3d1MTJabHE5TGlCRXdFU1I2a2U5Yy90VkhLbDlO?=
 =?utf-8?B?eTZRZnJBMlNWYzNhQm85bW5Va0kxYldtQk9sZlh1TWllTkVHWGswSEx1QVJk?=
 =?utf-8?B?L2hqT0EraTlnYkRlNXowK2JyMTErZFFWZVBUZzhMQlZtbE5XTUNFejRPZHZE?=
 =?utf-8?B?UHVrOFQrMzU4QXNPeVNXeklOL25LbG1UT3ZKazllU0NZaDNUeGJieUJjU2h5?=
 =?utf-8?B?SkVNSDVmb2UvTkFReWI0TklTbmpMNFIrWFZRZUZSa3piaUoxdndubUx2aG9Z?=
 =?utf-8?B?bXkzWlRscEFpem1CdWtRK29vamZZbTNkYTBGNCtRRGJSZVZNb1JuWDVVZVFI?=
 =?utf-8?B?SHlYamoySmNLSWp5TGNFYUtzRTl4QnZTY0FwS0FPd3RXSS96dFowcjJBaC9F?=
 =?utf-8?B?QjFEbFkvZmdiSnpCV0NtQlkvcFF0MFRWZnMwTXZ4MEhGaHpidFM0Uy8rT0xB?=
 =?utf-8?B?ZGVOSEZHRnBSc29RdU5FZ0RHbFpZMm9vZjBYMjIwYWFHVnQxSGVhcWxFUTVZ?=
 =?utf-8?B?YnlrUnhzODIxK3hBemM1amlNS1lDOEVMV0xWb3Q1dDZjWjAxL0k1M293dkZQ?=
 =?utf-8?B?SVh3TEdScWxsZVUyTmV5azFEclNaTVNzaTJJekZVdHR3b2d6OVViK2VHaUNt?=
 =?utf-8?B?alQvRFhzY29jdVN1R2Zjd1kwNVVZaWVxaHozRkVySXRabUR4MEhGck94Zm13?=
 =?utf-8?B?aWc2dWVCbXc3eURNVEUzRWtQWitVZyt3bXlJT2I1am5FWjVET0dOTFVWc2dN?=
 =?utf-8?B?ZmxxbDlPSC9uZWZNSzAxWDlHNTFZZG96a3lNU1IrL0gwTUkyZVJLa2xZOFps?=
 =?utf-8?B?U0RlamcwNzZESjg2LzB2Q0p5TTFNRWFWTUN6aWc4WDJoN1Z2dDFhNCtwN2FM?=
 =?utf-8?Q?ZqLJu930KX+2w3IA1gh6PMOiA?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5fe212fb-02f0-4c08-ec09-08dbdea5ad4e
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7529.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Nov 2023 08:52:16.4535
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Tco0jmtAQ4Ic2qE06sXrnQbrKkqN1Q+r5pjCwM9G6D/FsK68fjDhg4jb4j11BBoqMOzNbtfwceoTbMF6C0w2yQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4552
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-1.4 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 06 Nov 2023 00:52:42 -0800 (PST)

On 2023/11/4 01:16, Cindy Lu wrote:
> Add the check for iommufd_ictx,If vdpa don't have the iommufd_ictx
> then will use the Legacy iommu domain pathway
> 
> Signed-off-by: Cindy Lu <lulu@redhat.com>
> ---
>   drivers/vhost/vdpa.c | 43 ++++++++++++++++++++++++++++++++++++++-----
>   1 file changed, 38 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
> index dfaddd833364..0e2dba59e1ce 100644
> --- a/drivers/vhost/vdpa.c
> +++ b/drivers/vhost/vdpa.c
> @@ -1067,9 +1067,6 @@ static int vhost_vdpa_map(struct vhost_vdpa *v, struct vhost_iotlb *iotlb,
>   		/* Legacy iommu domain pathway without IOMMUFD */
>   		r = iommu_map(v->domain, iova, pa, size,
>   			      perm_to_iommu_flags(perm), GFP_KERNEL);
> -	} else {
> -		r = iommu_map(v->domain, iova, pa, size,
> -			      perm_to_iommu_flags(perm), GFP_KERNEL);
>   	}
>   	if (r) {
>   		vhost_iotlb_del_range(iotlb, iova, iova + size - 1);
> @@ -1095,8 +1092,10 @@ static void vhost_vdpa_unmap(struct vhost_vdpa *v,
>   	if (ops->set_map) {
>   		if (!v->in_batch)
>   			ops->set_map(vdpa, asid, iotlb);
> +	} else if (!vdpa->iommufd_ictx) {
> +		/* Legacy iommu domain pathway without IOMMUFD */
> +		iommu_unmap(v->domain, iova, size);
>   	}
> -
>   }
>   
>   static int vhost_vdpa_va_map(struct vhost_vdpa *v,
> @@ -1149,7 +1148,36 @@ static int vhost_vdpa_va_map(struct vhost_vdpa *v,
>   
>   	return ret;
>   }
> +#if 0
> +int vhost_pin_pages(struct vdpa_device *device, dma_addr_t iova, int npage,
> +		    int prot, struct page **pages)
> +{
> +	if (!pages || !npage)
> +		return -EINVAL;
> +	//if (!device->config->dma_unmap)
> +	//return -EINVAL;
> +
> +	if (0) { //device->iommufd_access) {
> +		int ret;
> +
> +		if (iova > ULONG_MAX)
> +			return -EINVAL;
>   
> +		ret = iommufd_access_pin_pages(
> +			device->iommufd_access, iova, npage * PAGE_SIZE, pages,
> +			(prot & IOMMU_WRITE) ? IOMMUFD_ACCESS_RW_WRITE : 0);
> +		if (ret) {
> +
> +			return ret;
> +		}
> +
> +		return npage;
> +	} else {
> +		return pin_user_pages(iova, npage, prot, pages);
> +	}
> +	return -EINVAL;
> +}
> +#endif

Is above code needed or not?

>   static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
>   			     struct vhost_iotlb *iotlb,
>   			     u64 iova, u64 size, u64 uaddr, u32 perm)
> @@ -1418,9 +1446,13 @@ static void vhost_vdpa_free_domain(struct vhost_vdpa *v)
>   	struct device *dma_dev = vdpa_get_dma_dev(vdpa);
>   
>   	if (v->domain) {
> -		iommu_detach_device(v->domain, dma_dev);
> +		if (!vdpa->iommufd_ictx) {
> +			iommu_detach_device(v->domain, dma_dev);
> +		}
>   		iommu_domain_free(v->domain);
>   	}
> +	if (vdpa->iommufd_ictx)
> +		vdpa_iommufd_unbind(vdpa);
>   
>   	v->domain = NULL;
>   }
> @@ -1645,6 +1677,7 @@ static int vhost_vdpa_probe(struct vdpa_device *vdpa)
>   	}
>   
>   	atomic_set(&v->opened, 0);
> +	atomic_set(&vdpa->iommufd_users, 0);
>   	v->minor = minor;
>   	v->vdpa = vdpa;
>   	v->nvqs = vdpa->nvqs;

-- 
Regards,
Yi Liu