Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <bbc6da4d-c2d9-c5ee-aced-3f08831672bc@linux.intel.com>
Date:   Wed, 21 Sep 2022 14:49:18 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.2.2
Cc:     baolu.lu@linux.intel.com, Joerg Roedel <joro@8bytes.org>,
        Will Deacon <will@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>,
        Jacob Pan <jacob.jun.pan@linux.intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache
 invalidation
Content-Language: en-US
To:     "Tian, Kevin" <kevin.tian@intel.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>
References: <20220919062523.3438951-1-baolu.lu@linux.intel.com>
 <20220919062523.3438951-2-baolu.lu@linux.intel.com>
 <BN9PR11MB52760E6985B16235A30B57978C4C9@BN9PR11MB5276.namprd11.prod.outlook.com>
From:   Baolu Lu <baolu.lu@linux.intel.com>
In-Reply-To: <BN9PR11MB52760E6985B16235A30B57978C4C9@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2022/9/20 15:51, Tian, Kevin wrote:
>> From: Lu Baolu <baolu.lu@linux.intel.com>
>> Sent: Monday, September 19, 2022 2:25 PM
>>
>> Some VT-d hardware implementations invalidate all interrupt remapping
>> hardware translation caches as part of SIRTP flow. The VT-d spec adds
>> a ESIRTPS (Enhanced Set Interrupt Remap Table Pointer Support, section
>> 11.4.2 in VT-d spec) capability bit to indicate this. With this bit set,
>> software has no need to issue the global invalidation request.
>>
>> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>> ---
>>   drivers/iommu/intel/iommu.h         | 1 +
>>   drivers/iommu/intel/irq_remapping.c | 3 ++-
>>   2 files changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
>> index eef1a166b855..5407d82df4d1 100644
>> --- a/drivers/iommu/intel/iommu.h
>> +++ b/drivers/iommu/intel/iommu.h
>> @@ -146,6 +146,7 @@
>>   /*
>>    * Decoding Capability Register
>>    */
>> +#define cap_esirtps(c)		(((c) >> 62) & 1)
>>   #define cap_fl5lp_support(c)	(((c) >> 60) & 1)
>>   #define cap_pi_support(c)	(((c) >> 59) & 1)
>>   #define cap_fl1gp_support(c)	(((c) >> 56) & 1)
>> diff --git a/drivers/iommu/intel/irq_remapping.c
>> b/drivers/iommu/intel/irq_remapping.c
>> index 2e9683e970f8..b4a91fbd1c2f 100644
>> --- a/drivers/iommu/intel/irq_remapping.c
>> +++ b/drivers/iommu/intel/irq_remapping.c
>> @@ -494,7 +494,8 @@ static void iommu_set_irq_remapping(struct
>> intel_iommu *iommu, int mode)
>>   	 * Global invalidation of interrupt entry cache to make sure the
>>   	 * hardware uses the new irq remapping table.
>>   	 */
>> -	qi_global_iec(iommu);
>> +	if (!cap_esirtps(iommu->cap))
>> +		qi_global_iec(iommu);
>>   }
>>
> 
> Same check is also required in iommu_disable_irq_remapping().
> 
> The spec says that:
> 
> --
> For implementations reporting the Enhanced Set Interrupt Remap Table
> Pointer Support (ESIRTPS) field as Set, hardware performs global
> invalidation on all Interrupt remapping caches as part of Interrupt
> Remapping Disable operation.
> --

Yes. I will add the same check in iommu_disable_irq_remapping(). Thanks
for the reminding.

> While looking at that code I think it's problematic to invalidate interrupt
> cache before disabling interrupt remapping. Presumably invalidations
> should be done only after a configuration is changed, no matter it's 'enable'
> or 'disable'.

I understand your concern. The spec doesn't state the sequence in the
interrupt remapping disabling flow. Need to check it with the spec
developers before any change here.

Best regards,
baolu