Date:   Thu, 9 Dec 2021 08:57:15 -0800
From:   "Raj, Ashok" <ashok.raj@intel.com>
To:     Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc:     Lu Baolu <baolu.lu@linux.intel.com>,
        "Tian, Kevin" <kevin.tian@intel.com>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Joerg Roedel <joro@8bytes.org>,
        Jason Gunthorpe <jgg@nvidia.com>,
        Christoph Hellwig <hch@infradead.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Jean-Philippe Brucker <jean-philippe@linaro.com>,
        "Pan, Jacob jun" <jacob.jun.pan@intel.com>,
        "Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "Luck, Tony" <tony.luck@intel.com>,
        "Liu, Yi L" <yi.l.liu@intel.com>, Barry Song <21cnbao@gmail.com>,
        "Zanussi, Tom" <tom.zanussi@intel.com>,
        "Williams, Dan J" <dan.j.williams@intel.com>,
        Ashok Raj <ashok.raj@intel.com>
Subject: Re: [PATCH 2/4] iommu: Add PASID support for DMA mapping API users
Message-ID: <20211209165715.GA566788@otc-nc-03>
References: <1638884834-83028-1-git-send-email-jacob.jun.pan@linux.intel.com>
 <1638884834-83028-3-git-send-email-jacob.jun.pan@linux.intel.com>
 <16408193-c8bc-3046-b32f-9274bf0b415c@linux.intel.com>
 <20211208104939.732fa5b9@jacob-builder>
 <BN9PR11MB5276676474FA6A35016B6BB88C709@BN9PR11MB5276.namprd11.prod.outlook.com>
 <1b3ee13d-0148-1156-52ad-b96bca51cb6f@linux.intel.com>
 <20211209083249.7d775512@jacob-builder>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20211209083249.7d775512@jacob-builder>
Precedence: bulk

On Thu, Dec 09, 2021 at 08:32:49AM -0800, Jacob Pan wrote:
> Hi Lu,
> 
> On Thu, 9 Dec 2021 10:21:38 +0800, Lu Baolu <baolu.lu@linux.intel.com>
> wrote:
> 
> > On 12/9/21 9:56 AM, Tian, Kevin wrote:
> > >> From: Jacob Pan <jacob.jun.pan@linux.intel.com>
> > >> Sent: Thursday, December 9, 2021 2:50 AM
> > >>  
> > >>> Can a device issue DMA requests with PASID even there's no system  
> > >> IOMMU  
> > >>> or the system IOMMU is disabled?
> > >>>  
> > >> Good point.
> > >> If IOMMU is not enabled, device cannot issue DMA requests with PASID.
> > >> This API will not be available. Forgot to add dummy functions to the
> > >> header. 
> > > 
> > > PASID is a PCI thing, not defined by IOMMU.

True, but RP is just a forwarding agent on these EETLP prefixes. I'm not
sure how RP's will behave if they receive a TLP they don't understand
and I suspect they might pull the system down via some UR type response
when IOMMU isn't enabled.

> > > 
> > > I think the key is physically if IOMMU is disabled, how will root
> > > complex handle a PCI memory request including a PASID TLP prefix? Does
> > > it block such request due to no IOMMU to consume PASID or simply ignore
> > > PASID and continue routing the request to the memory controller?
> > > 
> > > If block, then having an iommu interface makes sense.
> > > 
> > > If ignore, possibly a DMA API call makes more sense instead, implying
> > > that this extension can be used even when iommu is disabled.
> > > 
> > > I think that is what Baolu wants to point out.  
> > 
> Thanks for clarifying, very good point.
> Looking at the PCIe spec. I don't see specific rules for RC to ignore or
> block PASID TLP if not enabled.
> "- A Root Complex that supports PASID TLP Prefixes must have a device
> specific mechanism for enabling them. By default usage of PASID TLP
> Prefixes is disabled
>  - Root Complexes may optionally support TLPs with PASID TLP Prefixes. The
> mechanism used to detect whether a Root Complex supports the PASID TLP
> Prefix is implementation specific

Isn't implementation specific mechanism is IOMMU?

> "
> For all practical purposes, why would someone sets up PASID for DMA just to
> be ignored? An IOMMU interface makes sense to me.
> 
> > Yes, exactly. Imagining in the VM guest environment, do we require a
> > vIOMMU for this functionality? vIOMMU is not performance friendly if we
> > put aside the security considerations.
> > 
> The primary use case for accelerators to use in-kernel DMA will be in
> pass-through mode. vIOMMU should be able to do PT with good performance,
> right? no nesting, IO page faults.

But from an enabling perspective when PASID is in use we have to mandate
either the presence of an IOMMU, or some hypercall that will do the
required plumbing for PASID isn't it?