Received-SPF: pass (google.com: domain of linux-kernel+bounces-177146-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 216.228.117.160 as permitted sender) receiver=protection.outlook.com;
 client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C
Date: Sun, 12 May 2024 21:43:42 -0700
From: Nicolin Chen <nicolinc@nvidia.com>
To: Jason Gunthorpe <jgg@nvidia.com>
CC: <will@kernel.org>, <robin.murphy@arm.com>, <kevin.tian@intel.com>,
	<suravee.suthikulpanit@amd.com>, <joro@8bytes.org>,
	<linux-kernel@vger.kernel.org>, <iommu@lists.linux.dev>,
	<linux-arm-kernel@lists.infradead.org>, <linux-tegra@vger.kernel.org>,
	<yi.l.liu@intel.com>, <eric.auger@redhat.com>, <vasant.hegde@amd.com>,
	<jon.grimm@amd.com>, <santosh.shukla@amd.com>, <Dhaval.Giani@amd.com>,
	<shameerali.kolothum.thodi@huawei.com>
Subject: Re: [PATCH RFCv1 13/14] iommufd: Add mmap infrastructure
Message-ID: <ZkGafm+qAe77bvIW@nvidia.com>
References: <cover.1712978212.git.nicolinc@nvidia.com>
 <6fef9ff9944381d51dd18f83ec03785a26754dcf.1712978213.git.nicolinc@nvidia.com>
 <ZkDd8A2qAYnuxGJQ@nvidia.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ZkDd8A2qAYnuxGJQ@nvidia.com>
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 May 2024 04:44:04.8752
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c4aeb330-ebae-4741-861e-08dc73075190
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044F3.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8290

On Sun, May 12, 2024 at 12:19:12PM -0300, Jason Gunthorpe wrote:
> On Fri, Apr 12, 2024 at 08:47:10PM -0700, Nicolin Chen wrote:
> > Add for sharing the kernel page with user space. This allows to pass
> > through HW resource (VCMDQ MMIO pages for example) to user space VMM
> > and guest OS. Use vma->vm_pgoff as the carrier of a viommu_id.
> > 
> > Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> > ---
> >  drivers/iommu/iommufd/main.c | 40 ++++++++++++++++++++++++++++++++++++
> >  include/linux/iommufd.h      |  4 ++++
> >  2 files changed, 44 insertions(+)
> > 
> > diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c
> > index 96ef81530809..5b401c80cca8 100644
> > --- a/drivers/iommu/iommufd/main.c
> > +++ b/drivers/iommu/iommufd/main.c
> > @@ -16,6 +16,7 @@
> >  #include <linux/mutex.h>
> >  #include <linux/bug.h>
> >  #include <uapi/linux/iommufd.h>
> > +#include <linux/iommu.h>
> >  #include <linux/iommufd.h>
> >  
> >  #include "io_pagetable.h"
> > @@ -427,11 +428,50 @@ static long iommufd_fops_ioctl(struct file *filp, unsigned int cmd,
> >  	return ret;
> >  }
> >  
> > +static int iommufd_fops_mmap(struct file *filp, struct vm_area_struct *vma)
> > +{
> > +	struct iommufd_ctx *ictx = filp->private_data;
> > +	size_t size = vma->vm_end - vma->vm_start;
> > +	u32 viommu_id = (u32)vma->vm_pgoff;
> 
> Don't do mmaps this way, it doesn't scale well for future things.
> 
> The pgoff/length should *always* come from the kernel as some
> 'mmap_offset' output. I usually call this the mmap cookie.
> 
> In this case have the mmap cookie for the tegra doorbell return in the
> viommu's driver data struct, then userspace just passes the opaque
> cookie to mmap to get the correct tegra doorbell.
> 
> The core code has some simple xarray/maple tree to allocate cookies
> and dispatch them to the correct mmap callback. Usually I'd say to
> provide a mmap callback pointer when allocating the cookie.
> 
> Also look at the RDMA Code around mmap there is a bunch of VMA
> validations needed. Ie we must insist on VM_SHARED and check
> permissions, etc.

Yea, the vm_pgoff as a carrier is a bit of hack, as mentioned
in the cover-letter. Let me revisit the whole thing and study
from RDMA code also.

Thanks
Nicolin