Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
Date:   Thu, 30 Nov 2023 08:15:44 -0400
From:   Jason Gunthorpe <jgg@ziepe.ca>
To:     Baolu Lu <baolu.lu@linux.intel.com>
Cc:     Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>,
        Kevin Tian <kevin.tian@intel.com>, iommu@lists.linux.dev,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/1] iommu/vt-d: Omit devTLB invalidation requests when
 TES=0
Message-ID: <20231130121544.GC1394392@ziepe.ca>
References: <20231114011036.70142-1-baolu.lu@linux.intel.com>
 <20231114011036.70142-2-baolu.lu@linux.intel.com>
 <20231129201020.GK1312390@ziepe.ca>
 <2f2582df-fb56-4b46-8ce3-364879b34734@linux.intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2f2582df-fb56-4b46-8ce3-364879b34734@linux.intel.com>
Precedence: bulk

On Thu, Nov 30, 2023 at 12:06:59PM +0800, Baolu Lu wrote:
> On 2023/11/30 4:10, Jason Gunthorpe wrote:
> > On Tue, Nov 14, 2023 at 09:10:34AM +0800, Lu Baolu wrote:
> > > The latest VT-d spec indicates that when remapping hardware is disabled
> > > (TES=0 in Global Status Register), upstream ATS Invalidation Completion
> > > requests are treated as UR (Unsupported Request).
> > > 
> > > Consequently, the spec recommends in section 4.3 Handling of Device-TLB
> > > Invalidations that software refrain from submitting any Device-TLB
> > > invalidation requests when address remapping hardware is disabled.
> > > 
> > > Verify address remapping hardware is enabled prior to submitting Device-
> > > TLB invalidation requests.
> > > 
> > > Fixes: 792fb43ce2c9 ("iommu/vt-d: Enable Intel IOMMU scalable mode by default")
> > > Signed-off-by: Lu Baolu<baolu.lu@linux.intel.com>
> > > ---
> > >   drivers/iommu/intel/dmar.c | 18 ++++++++++++++++++
> > >   1 file changed, 18 insertions(+)
> > How did you get to the point where flush_dev_iotlb could even be
> > called if the iommu has somehow been globally disabled?
> > 
> > Shouldn't the attach of the domain compeltely fail if the HW is
> > disabled?
> > 
> > If the domain is not attached to anything why would flushing happen?
> 
> The VT-d hardware can be in a state where the hardware is on but DMA
> translation is deactivated. In this state, the device probe process
> during boot proceeds as follows:
> 
> 1) Initialize the IOMMU contexts: This sets up the data structures that
>    the IOMMU uses to manage address translation for DMA operations.
> 
> 2) Register the IOMMU devices: This registers the IOMMU devices to the
>    core. The core then probes devices on buses like PCI.
> 
> 3) Enable DMA translation: This step activates DMA translation.
> 
> With regard to step 2), the call to iommu_flush_iotlb_all() in
> iommu_create_device_direct_mappings() can potentially cause device TBL
> invalidation when the VT-d DMA translation is deactivated.

You are trying to create an atomic change at boot from non-translating
to DMA translating for HW that doesn't support the identity mode?

This should probably get a comment in this patch..

Jason