On Tue, Nov 14, 2023 at 09:10:34AM +0800, Lu Baolu wrote:
> The latest VT-d spec indicates that when remapping hardware is disabled
> (TES=0 in Global Status Register), upstream ATS Invalidation Completion
> requests are treated as UR (Unsupported Request).
>
> Consequently, the spec recommends in section 4.3 Handling of Device-TLB
> Invalidations that software refrain from submitting any Device-TLB
> invalidation requests when address remapping hardware is disabled.
>
> Verify address remapping hardware is enabled prior to submitting Device-
> TLB invalidation requests.
>
> Fixes: 792fb43ce2c9 ("iommu/vt-d: Enable Intel IOMMU scalable mode by default")
> Signed-off-by: Lu Baolu <[email protected]>
> ---
> drivers/iommu/intel/dmar.c | 18 ++++++++++++++++++
> 1 file changed, 18 insertions(+)
How did you get to the point where flush_dev_iotlb could even be
called if the iommu has somehow been globally disabled?
Shouldn't the attach of the domain compeltely fail if the HW is
disabled?
If the domain is not attached to anything why would flushing happen?
Jason
On 2023/11/30 4:10, Jason Gunthorpe wrote:
> On Tue, Nov 14, 2023 at 09:10:34AM +0800, Lu Baolu wrote:
>> The latest VT-d spec indicates that when remapping hardware is disabled
>> (TES=0 in Global Status Register), upstream ATS Invalidation Completion
>> requests are treated as UR (Unsupported Request).
>>
>> Consequently, the spec recommends in section 4.3 Handling of Device-TLB
>> Invalidations that software refrain from submitting any Device-TLB
>> invalidation requests when address remapping hardware is disabled.
>>
>> Verify address remapping hardware is enabled prior to submitting Device-
>> TLB invalidation requests.
>>
>> Fixes: 792fb43ce2c9 ("iommu/vt-d: Enable Intel IOMMU scalable mode by default")
>> Signed-off-by: Lu Baolu<[email protected]>
>> ---
>> drivers/iommu/intel/dmar.c | 18 ++++++++++++++++++
>> 1 file changed, 18 insertions(+)
> How did you get to the point where flush_dev_iotlb could even be
> called if the iommu has somehow been globally disabled?
>
> Shouldn't the attach of the domain compeltely fail if the HW is
> disabled?
>
> If the domain is not attached to anything why would flushing happen?
The VT-d hardware can be in a state where the hardware is on but DMA
translation is deactivated. In this state, the device probe process
during boot proceeds as follows:
1) Initialize the IOMMU contexts: This sets up the data structures that
the IOMMU uses to manage address translation for DMA operations.
2) Register the IOMMU devices: This registers the IOMMU devices to the
core. The core then probes devices on buses like PCI.
3) Enable DMA translation: This step activates DMA translation.
With regard to step 2), the call to iommu_flush_iotlb_all() in
iommu_create_device_direct_mappings() can potentially cause device TBL
invalidation when the VT-d DMA translation is deactivated.
Best regards,
baolu
On Thu, Nov 30, 2023 at 12:06:59PM +0800, Baolu Lu wrote:
> On 2023/11/30 4:10, Jason Gunthorpe wrote:
> > On Tue, Nov 14, 2023 at 09:10:34AM +0800, Lu Baolu wrote:
> > > The latest VT-d spec indicates that when remapping hardware is disabled
> > > (TES=0 in Global Status Register), upstream ATS Invalidation Completion
> > > requests are treated as UR (Unsupported Request).
> > >
> > > Consequently, the spec recommends in section 4.3 Handling of Device-TLB
> > > Invalidations that software refrain from submitting any Device-TLB
> > > invalidation requests when address remapping hardware is disabled.
> > >
> > > Verify address remapping hardware is enabled prior to submitting Device-
> > > TLB invalidation requests.
> > >
> > > Fixes: 792fb43ce2c9 ("iommu/vt-d: Enable Intel IOMMU scalable mode by default")
> > > Signed-off-by: Lu Baolu<[email protected]>
> > > ---
> > > drivers/iommu/intel/dmar.c | 18 ++++++++++++++++++
> > > 1 file changed, 18 insertions(+)
> > How did you get to the point where flush_dev_iotlb could even be
> > called if the iommu has somehow been globally disabled?
> >
> > Shouldn't the attach of the domain compeltely fail if the HW is
> > disabled?
> >
> > If the domain is not attached to anything why would flushing happen?
>
> The VT-d hardware can be in a state where the hardware is on but DMA
> translation is deactivated. In this state, the device probe process
> during boot proceeds as follows:
>
> 1) Initialize the IOMMU contexts: This sets up the data structures that
> the IOMMU uses to manage address translation for DMA operations.
>
> 2) Register the IOMMU devices: This registers the IOMMU devices to the
> core. The core then probes devices on buses like PCI.
>
> 3) Enable DMA translation: This step activates DMA translation.
>
> With regard to step 2), the call to iommu_flush_iotlb_all() in
> iommu_create_device_direct_mappings() can potentially cause device TBL
> invalidation when the VT-d DMA translation is deactivated.
You are trying to create an atomic change at boot from non-translating
to DMA translating for HW that doesn't support the identity mode?
This should probably get a comment in this patch..
Jason