Date:   Mon, 6 Mar 2023 10:43:46 -0800
From:   Jacob Pan <jacob.jun.pan@linux.intel.com>
To:     "Tian, Kevin" <kevin.tian@intel.com>
Cc:     Baolu Lu <baolu.lu@linux.intel.com>,
        LKML <linux-kernel@vger.kernel.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        Jason Gunthorpe <jgg@nvidia.com>,
        Joerg Roedel <joro@8bytes.org>,
        "dmaengine@vger.kernel.org" <dmaengine@vger.kernel.org>,
        "vkoul@kernel.org" <vkoul@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>,
        Will Deacon <will@kernel.org>,
        David Woodhouse <dwmw2@infradead.org>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "Liu, Yi L" <yi.l.liu@intel.com>,
        "Yu, Fenghua" <fenghua.yu@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "Luck, Tony" <tony.luck@intel.com>,
        "Zanussi, Tom" <tom.zanussi@intel.com>,
        jacob.jun.pan@linux.intel.com
Subject: Re: [PATCH 1/4] iommu/vt-d: Implement set device pasid op for
 default domain
Message-ID: <20230306104346.022849ee@jacob-builder>
In-Reply-To: <BN9PR11MB527649B07E006466882F3D278CB69@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20230302005959.2695267-1-jacob.jun.pan@linux.intel.com>
        <20230302005959.2695267-2-jacob.jun.pan@linux.intel.com>
        <fad7f28f-b4e8-c1c3-4ca4-a48c5c6d7f4a@linux.intel.com>
        <BN9PR11MB527627C597F6478536477A8F8CB39@BN9PR11MB5276.namprd11.prod.outlook.com>
        <20230303083512.663ce758@jacob-builder>
        <c54aebf7-4282-b8a6-f03b-03af2deea59c@linux.intel.com>
        <BN9PR11MB527649B07E006466882F3D278CB69@BN9PR11MB5276.namprd11.prod.outlook.com>
Organization: OTC
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi Kevin,

On Mon, 6 Mar 2023 08:18:37 +0000, "Tian, Kevin" <kevin.tian@intel.com>
wrote:

> > From: Baolu Lu <baolu.lu@linux.intel.com>
> > Sent: Sunday, March 5, 2023 11:06 AM
> > 
> > On 3/4/23 12:35 AM, Jacob Pan wrote:  
> > >>> From: Baolu Lu<baolu.lu@linux.intel.com>
> > >>> Sent: Thursday, March 2, 2023 10:07 PM
> > >>>  
> > >>>> +
> > >>>> +	if (hw_pass_through && domain_type_is_si(dmar_domain))
> > >>>> +		ret = intel_pasid_setup_pass_through(iommu,
> > >>>> dmar_domain,
> > >>>> +						     dev, pasid);
> > >>>> +	else if (dmar_domain->use_first_level)
> > >>>> +		ret = domain_setup_first_level(iommu, dmar_domain,
> > >>>> +					       dev, pasid);
> > >>>> +	else
> > >>>> +		ret = intel_pasid_setup_second_level(iommu,
> > >>>> dmar_domain,
> > >>>> +						     dev, pasid);
> > >>>> +
> > >>>> +	return ret;
> > >>>> +}  
> > >>> Do you need to consider pasid cache invalidation?
> > >>>  
> > >> To avoid confusion this is not about invalidation of pasid cache
> > >> itself which should be covered by above setup functions already.
> > >>
> > >> Here actually means per-PASID invalidation in iotlb and devtlb. Today
> > >> only RID is tracked per domain for invalidation. it needs extension
> > >> to walk attached pasid too.  
> > > Yes, will add.
> > >
> > > For the set up path, there is no need to flush IOTLBs,  because we're
> > > going from non present to present.
> > >
> > > On the remove path, IOTLB flush should be covered when device driver
> > > calls iommu_detach_device_pasid(). Covered with this patch.  
> > 
> > It's not only for the PASID teardown path, but also for unmap(). As the
> > device has issued DMA requests with PASID, the IOMMU probably will cache
> > the DMA translation with PASID tagged. Hence, we need to invalidate the
> > PASID-specific IOTLB and device TLB in the unmap() path.
> > 
> > I once had a patch for this:
> > 
> > https://lore.kernel.org/linux-iommu/20220614034411.1634238-1-
> > baolu.lu@linux.intel.com/
> > 
> > Probably you can use it as a starting point.
> >   
> 
> just that we should not have a sub-device term there. Just name
> the tracking information per pasid.
Sounds good, I should be enable to use Baolu's patch for the most part.

Thanks,

Jacob