Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 19 Apr 2023 14:05:54 -0700
From:   Jacob Pan <jacob.jun.pan@linux.intel.com>
To:     Baolu Lu <baolu.lu@linux.intel.com>
Cc:     "Tian, Kevin" <kevin.tian@intel.com>,
        LKML <linux-kernel@vger.kernel.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        Robin Murphy <robin.murphy@arm.com>,
        Jason Gunthorpe <jgg@nvidia.com>,
        Joerg Roedel <joro@8bytes.org>,
        "dmaengine@vger.kernel.org" <dmaengine@vger.kernel.org>,
        "vkoul@kernel.org" <vkoul@kernel.org>,
        Will Deacon <will@kernel.org>,
        David Woodhouse <dwmw2@infradead.org>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "Liu, Yi L" <yi.l.liu@intel.com>,
        "Yu, Fenghua" <fenghua.yu@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "Luck, Tony" <tony.luck@intel.com>,
        "Zanussi, Tom" <tom.zanussi@intel.com>,
        jacob.jun.pan@linux.intel.com
Subject: Re: [PATCH v4 3/7] iommu: Support allocation of global PASIDs
 outside SVA
Message-ID: <20230419140554.66578c1e@jacob-builder>
In-Reply-To: <7e182125-f5ab-3201-e69d-b6c96eeede01@linux.intel.com>
References: <20230407180554.2784285-1-jacob.jun.pan@linux.intel.com>
        <20230407180554.2784285-4-jacob.jun.pan@linux.intel.com>
        <BN9PR11MB5276D09F18BA65AD074777948C9A9@BN9PR11MB5276.namprd11.prod.outlook.com>
        <5882ee52-9657-250d-0474-13edffa7b6b9@linux.intel.com>
        <20230417094629.59fcfde6@jacob-builder>
        <a1a82bc0-9a7a-5363-cda8-a0226eff0073@linux.intel.com>
        <20230418160450.4ea7fb7d@jacob-builder>
        <7e182125-f5ab-3201-e69d-b6c96eeede01@linux.intel.com>
Organization: OTC
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi Baolu,

On Wed, 19 Apr 2023 10:40:46 +0800, Baolu Lu <baolu.lu@linux.intel.com>
wrote:

> On 4/19/23 7:04 AM, Jacob Pan wrote:
> > On Tue, 18 Apr 2023 10:06:12 +0800, Baolu Lu<baolu.lu@linux.intel.com>
> > wrote:
> >   
> >> On 4/18/23 12:46 AM, Jacob Pan wrote:  
> >>> On Wed, 12 Apr 2023 09:37:48 +0800, Baolu Lu<baolu.lu@linux.intel.com>
> >>> wrote:
> >>>      
> >>>> On 4/11/23 4:02 PM, Tian, Kevin wrote:  
> >>>>>> From: Jacob Pan<jacob.jun.pan@linux.intel.com>
> >>>>>> Sent: Saturday, April 8, 2023 2:06 AM
> >>>>>> @@ -28,8 +26,8 @@ static int iommu_sva_alloc_pasid(struct mm_struct
> >>>>>> *mm, ioasid_t min, ioasid_t ma
> >>>>>>     		goto out;
> >>>>>>     	}
> >>>>>>
> >>>>>> -	ret = ida_alloc_range(&iommu_global_pasid_ida, min, max,
> >>>>>> GFP_KERNEL);
> >>>>>> -	if (ret < min)
> >>>>>> +	ret = iommu_alloc_global_pasid(min, max);  
> >>>>> I wonder whether this can take a device pointer so
> >>>>> dev->iommu->max_pasids is enforced inside the alloc function.  
> >>>> Agreed. Instead of using the open code, it looks better to have a
> >>>> helper like dev_iommu_max_pasids().  
> >>> yes, probably export dev_iommu_get_max_pasids(dev)?
> >>>
> >>> But if I understood Kevin correctly, he's also suggesting that the
> >>> interface should be changed to iommu_alloc_global_pasid(dev), my
> >>> concern is that how do we use this function to reserve RID_PASID which
> >>> is not specific to a device?  
> >> Probably we can introduce a counterpart dev->iommu->min_pasids, so that
> >> there's no need to reserve the RID_PASID. At present, we can set it to
> >> 1 in the core as ARM/AMD/Intel all treat PASID 0 as a special pasid.
> >>
> >> In the future, if VT-d supports using arbitrary number as RID_PASID for
> >> any specific device, we can call iommu_alloc_global_pasid() for that
> >> device.
> >>
> >> The device drivers don't know and don't need to know the range of
> >> viable PASIDs, so the @min, @max parameters seem to be unreasonable.  
> > Sure, that is reasonable. Another question is whether global PASID
> > allocation is always for a single device, if not I prefer to keep the
> > current iommu_alloc_global_pasid() and add a wrapper
> > iommu_alloc_global_pasid_dev(dev) to extract the @min, @max. OK?  
> 
> No problem from the code perspective. But we only need one API.
> 
> We can now add the kAPI that we really need. In this series, the idxd
> driver wants to allocate a global PASID for its kernel dma with pasid
> purpose. So, iommu_alloc_global_pasid_dev() seems to be sufficient.
> 
> If, in the future, we will have a need to provide global pasid
> allocation other than device drivers, we can easily add the variants.
> 
sounds good, I will only add iommu_alloc_global_pasid_dev(dev). let the
core code set @min, @max for devices.

Thanks,

Jacob