Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
Message-ID: <2f2582df-fb56-4b46-8ce3-364879b34734@linux.intel.com>
Date:   Thu, 30 Nov 2023 12:06:59 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Cc:     baolu.lu@linux.intel.com, Joerg Roedel <joro@8bytes.org>,
        Will Deacon <will@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>,
        Kevin Tian <kevin.tian@intel.com>, iommu@lists.linux.dev,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/1] iommu/vt-d: Omit devTLB invalidation requests when
 TES=0
Content-Language: en-US
To:     Jason Gunthorpe <jgg@ziepe.ca>
References: <20231114011036.70142-1-baolu.lu@linux.intel.com>
 <20231114011036.70142-2-baolu.lu@linux.intel.com>
 <20231129201020.GK1312390@ziepe.ca>
From:   Baolu Lu <baolu.lu@linux.intel.com>
In-Reply-To: <20231129201020.GK1312390@ziepe.ca>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2023/11/30 4:10, Jason Gunthorpe wrote:
> On Tue, Nov 14, 2023 at 09:10:34AM +0800, Lu Baolu wrote:
>> The latest VT-d spec indicates that when remapping hardware is disabled
>> (TES=0 in Global Status Register), upstream ATS Invalidation Completion
>> requests are treated as UR (Unsupported Request).
>>
>> Consequently, the spec recommends in section 4.3 Handling of Device-TLB
>> Invalidations that software refrain from submitting any Device-TLB
>> invalidation requests when address remapping hardware is disabled.
>>
>> Verify address remapping hardware is enabled prior to submitting Device-
>> TLB invalidation requests.
>>
>> Fixes: 792fb43ce2c9 ("iommu/vt-d: Enable Intel IOMMU scalable mode by default")
>> Signed-off-by: Lu Baolu<baolu.lu@linux.intel.com>
>> ---
>>   drivers/iommu/intel/dmar.c | 18 ++++++++++++++++++
>>   1 file changed, 18 insertions(+)
> How did you get to the point where flush_dev_iotlb could even be
> called if the iommu has somehow been globally disabled?
> 
> Shouldn't the attach of the domain compeltely fail if the HW is
> disabled?
> 
> If the domain is not attached to anything why would flushing happen?

The VT-d hardware can be in a state where the hardware is on but DMA
translation is deactivated. In this state, the device probe process
during boot proceeds as follows:

1) Initialize the IOMMU contexts: This sets up the data structures that
    the IOMMU uses to manage address translation for DMA operations.

2) Register the IOMMU devices: This registers the IOMMU devices to the
    core. The core then probes devices on buses like PCI.

3) Enable DMA translation: This step activates DMA translation.

With regard to step 2), the call to iommu_flush_iotlb_all() in
iommu_create_device_direct_mappings() can potentially cause device TBL
invalidation when the VT-d DMA translation is deactivated.

Best regards,
baolu