Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 11 May 2022 09:00:50 +0100
From:   Jean-Philippe Brucker <jean-philippe@linaro.org>
To:     Baolu Lu <baolu.lu@linux.intel.com>
Cc:     Jason Gunthorpe <jgg@nvidia.com>, Joerg Roedel <joro@8bytes.org>,
        Christoph Hellwig <hch@infradead.org>,
        Kevin Tian <kevin.tian@intel.com>,
        Ashok Raj <ashok.raj@intel.com>, Will Deacon <will@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>,
        Jean-Philippe Brucker <jean-philippe@linaro.com>,
        Dave Jiang <dave.jiang@intel.com>,
        Vinod Koul <vkoul@kernel.org>,
        Eric Auger <eric.auger@redhat.com>,
        Liu Yi L <yi.l.liu@intel.com>,
        Jacob jun Pan <jacob.jun.pan@intel.com>,
        iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v6 02/12] iommu: Add pasid_bits field in struct dev_iommu
Message-ID: <YnttMpX2geTKRzVV@myrica>
References: <20220510061738.2761430-1-baolu.lu@linux.intel.com>
 <20220510061738.2761430-3-baolu.lu@linux.intel.com>
 <20220510143405.GE49344@nvidia.com>
 <3fe05f18-6726-276a-8c42-79e0b134dfdc@linux.intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <3fe05f18-6726-276a-8c42-79e0b134dfdc@linux.intel.com>
Precedence: bulk

On Wed, May 11, 2022 at 10:25:48AM +0800, Baolu Lu wrote:
> On 2022/5/10 22:34, Jason Gunthorpe wrote:
> > On Tue, May 10, 2022 at 02:17:28PM +0800, Lu Baolu wrote:
> > 
> > >   int iommu_device_register(struct iommu_device *iommu,
> > > diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> > > index 627a3ed5ee8f..afc63fce6107 100644
> > > +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> > > @@ -2681,6 +2681,8 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev)
> > >   	    smmu->features & ARM_SMMU_FEAT_STALL_FORCE)
> > >   		master->stall_enabled = true;
> > > +	dev->iommu->pasid_bits = master->ssid_bits;
> > >   	return &smmu->iommu;
> > >   err_free_master:
> > > diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> > > index 2990f80c5e08..99643f897f26 100644
> > > +++ b/drivers/iommu/intel/iommu.c
> > > @@ -4624,8 +4624,11 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
> > >   			if (pasid_supported(iommu)) {
> > >   				int features = pci_pasid_features(pdev);
> > > -				if (features >= 0)
> > > +				if (features >= 0) {
> > >   					info->pasid_supported = features | 1;
> > > +					dev->iommu->pasid_bits =
> > > +						fls(pci_max_pasids(pdev)) - 1;
> > > +				}
> > 
> > It is not very nice that both the iommu drivers have to duplicate the
> > code to read the pasid capability out of the PCI device.
> > 
> > IMHO it would make more sense for the iommu layer to report the
> > capability of its own HW block only, and for the core code to figure
> > out the master's limitation using a bus-specific approach.
> 
> Fair enough. The iommu hardware capability could be reported in
> 
> /**
>  * struct iommu_device - IOMMU core representation of one IOMMU hardware
>  *                       instance
>  * @list: Used by the iommu-core to keep a list of registered iommus
>  * @ops: iommu-ops for talking to this iommu
>  * @dev: struct device for sysfs handling
>  */
> struct iommu_device {
>         struct list_head list;
>         const struct iommu_ops *ops;
>         struct fwnode_handle *fwnode;
>         struct device *dev;
> };
> 
> I haven't checked ARM code yet, but it works for x86 as far as I can
> see.

Arm also supports non-PCI PASID by reading a firmware property:

        device_property_read_u32(dev, "pasid-num-bits", &master->ssid_bits);

should be the only difference

Thanks,
Jean

> 
> > 
> > It is also unfortunate that the enable/disable pasid is inside the
> > iommu driver as well - ideally the PCI driver itself would do this
> > when it knows it wants to use PASIDs.
> > 
> > The ordering interaction with ATS makes this look quite annoying
> > though. :(
> > 
> > I'm also not convinced individual IOMMU drivers should be forcing ATS
> > on, there are performance and functional implications here. Using ATS
> > or not is possibly best left as an administrator policy controlled by
> > the core code. Again we seem to have some mess.
> 
> Agreed with you. This has already been in my task list. I will start to
> solve it after the iommufd tasks.
> 
> Best regards,
> baolu