Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Lu Baolu <baolu.lu@linux.intel.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>
CC:     Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
        "Robin Murphy" <robin.murphy@arm.com>,
        Jacob Pan <jacob.jun.pan@linux.intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache
 invalidation
Thread-Topic: [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache
 invalidation
Thread-Index: AQHYy/F3/EWalWXJdE2oiCmU8rdmj63n7vnA
Date:   Tue, 20 Sep 2022 07:51:08 +0000
Message-ID: <BN9PR11MB52760E6985B16235A30B57978C4C9@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20220919062523.3438951-1-baolu.lu@linux.intel.com>
 <20220919062523.3438951-2-baolu.lu@linux.intel.com>
In-Reply-To: <20220919062523.3438951-2-baolu.lu@linux.intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?gIasX3DW76gSqhASLhFAusus1+Fy+6+RzM9b/D7BeH68UONYN/uZ9wrAdXHa?=
 =?us-ascii?Q?2KtfcxVlUPQ4DgHTJvpN8MvQnDB+9dSDvEiOjwyoBi3p5FYt3xYja8vhlDzM?=
 =?us-ascii?Q?GnXfmajLlH/4tJbgjwvkteolSAHaWnO+LIiuudTEl5W81tOl5DVoBnyw714A?=
 =?us-ascii?Q?DPFLBeEqr2+tGfWrRdyXqzLiWoK2m68wPrxcOpCMMMdHiLbKT83V9TTOLFkA?=
 =?us-ascii?Q?zSDQLAHy7ywrP/E8eSpMVLwdjswqSGeXAF37SNUkcfY6baaeag/XVSh2nY76?=
 =?us-ascii?Q?pyY355SPs5fckqUknO9DS7vFpm4k9JFf7XTsWhlpxAmxfBhaYRe8N7/I+XNf?=
 =?us-ascii?Q?gmHhz8wMMwNmAY41Y43w5UIKi4wpYNXklLg0RInQlD9ydhTwYPhU+OlNHvZZ?=
 =?us-ascii?Q?5IGcG/kS0+yEsgt3jM/fVFaWcfh0yIa2Uhg+o2UfEgCtZH1VQZSHzn+nZLTV?=
 =?us-ascii?Q?E1Somv96KBtpc3zAfkuEdWQuBkNqyC+ve5EmJgnOnknWDZCPkvR3wW60q1R/?=
 =?us-ascii?Q?a5N9PRn3OLeaGhdxl9QD2y3GOPojkgP3TgpWb0AynjZ2Ms/d+0ETslu5hyRl?=
 =?us-ascii?Q?vc/RlqCkd6LoS6Id2fhHDqzNMF3AwLDVkWbtZeMaLEfCCJELot2VYaD/ehF4?=
 =?us-ascii?Q?lVPps8iJJWv5sn1vxPk6L1AdtF62tTdFUMFvPls5QQVx3vvdNpGHu/Fyua65?=
 =?us-ascii?Q?Uy+Lta4LF16Gyu5+M0/SpKxTxa760tXGDRQjwApPmvmZIKRXPMjMcUAqUNQA?=
 =?us-ascii?Q?JgjicMvNfADLetNZppyck6y5i512jZDeI9sVusYe9oap8QIk4+jpuchINLyO?=
 =?us-ascii?Q?jSKqmISrkSs3PzcvE+EBp2m4Pk32zMrUe5v3omRt1HSzsa/d2zUb3OGwDtG7?=
 =?us-ascii?Q?ZO+FQNL/XnIpJGvKtOsFLnojkaQo5ju1BonKahciSkvf3c9XdKR/edoyfBjb?=
 =?us-ascii?Q?bFsHyOUEmFgYWpsLLafDeX9rf/y3G87BeHj6sAS8qZC+XK/KlhhdZuhoGBp3?=
 =?us-ascii?Q?o+TagbTVtADXxm8bH82Wo3l2zU09Ywu3hcBCZHnnChST0VMF6Cm9zFHfJYzQ?=
 =?us-ascii?Q?gJoqf2L9B7kz0gtlLuolC9Qlr0ODvkE7urcrgz9IUC+fAs5HrLCAIgoBJkHG?=
 =?us-ascii?Q?oX2a22NDV4e0VLoYgtamjMsj5zUPK8ZT8gQ9XkE9Woyz03R2VROI3XAeJWyb?=
 =?us-ascii?Q?fmcOhH1pZBlGOIBcbAXImGOZQAJ/FjGxg/41rs3GW0GP0f2mmPII9lfG8UWW?=
 =?us-ascii?Q?Cn3ZIhwepYzyDZrVah3WaPluOSS98BPJ9j1arD68jCxGNyWy3h2OaVY8JlQC?=
 =?us-ascii?Q?EpBnIBFEknmT0FfD7CY+i8/nZEOipqQsOSCZJBA9sArYJ1xQ8Os7lZ0hccQV?=
 =?us-ascii?Q?u5Vb26xaDYSvlqIj2Js1MyMi13hYLdz8ZGSMUFxSDzQrVAFKOHV/LosaFVX7?=
 =?us-ascii?Q?0+KWDlDr6dEMzi2rqAtauwtsSN+0bmG9hLPulWYgumoC/6t6R1OWL9pFMJOp?=
 =?us-ascii?Q?R0JVHB40m+7vrySOnFGON25JWrZXJ/h5vytlPKsOYvlph3vV/q8jJgLf8acd?=
 =?us-ascii?Q?1EtkQaX2tJXiF5w43MJlYLW5X+Vqig0lDf8wAe6w?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 48487a9a-3a5c-421c-778d-08da9adce0ec
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Sep 2022 07:51:08.2797
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 6Ih14btTbKu4LcnyLKz/0PHY/HicI7G6o5XHBAqxDBMsfwMW2COv1O9pBMikQPOrbYAB0YeP4DpqNke4RPCEiw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR11MB5705
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Lu Baolu <baolu.lu@linux.intel.com>
> Sent: Monday, September 19, 2022 2:25 PM
>=20
> Some VT-d hardware implementations invalidate all interrupt remapping
> hardware translation caches as part of SIRTP flow. The VT-d spec adds
> a ESIRTPS (Enhanced Set Interrupt Remap Table Pointer Support, section
> 11.4.2 in VT-d spec) capability bit to indicate this. With this bit set,
> software has no need to issue the global invalidation request.
>=20
> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> ---
>  drivers/iommu/intel/iommu.h         | 1 +
>  drivers/iommu/intel/irq_remapping.c | 3 ++-
>  2 files changed, 3 insertions(+), 1 deletion(-)
>=20
> diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
> index eef1a166b855..5407d82df4d1 100644
> --- a/drivers/iommu/intel/iommu.h
> +++ b/drivers/iommu/intel/iommu.h
> @@ -146,6 +146,7 @@
>  /*
>   * Decoding Capability Register
>   */
> +#define cap_esirtps(c)		(((c) >> 62) & 1)
>  #define cap_fl5lp_support(c)	(((c) >> 60) & 1)
>  #define cap_pi_support(c)	(((c) >> 59) & 1)
>  #define cap_fl1gp_support(c)	(((c) >> 56) & 1)
> diff --git a/drivers/iommu/intel/irq_remapping.c
> b/drivers/iommu/intel/irq_remapping.c
> index 2e9683e970f8..b4a91fbd1c2f 100644
> --- a/drivers/iommu/intel/irq_remapping.c
> +++ b/drivers/iommu/intel/irq_remapping.c
> @@ -494,7 +494,8 @@ static void iommu_set_irq_remapping(struct
> intel_iommu *iommu, int mode)
>  	 * Global invalidation of interrupt entry cache to make sure the
>  	 * hardware uses the new irq remapping table.
>  	 */
> -	qi_global_iec(iommu);
> +	if (!cap_esirtps(iommu->cap))
> +		qi_global_iec(iommu);
>  }
>=20

Same check is also required in iommu_disable_irq_remapping().

The spec says that:

--
For implementations reporting the Enhanced Set Interrupt Remap Table
Pointer Support (ESIRTPS) field as Set, hardware performs global
invalidation on all Interrupt remapping caches as part of Interrupt
Remapping Disable operation.
--

While looking at that code I think it's problematic to invalidate interrupt
cache before disabling interrupt remapping. Presumably invalidations
should be done only after a configuration is changed, no matter it's 'enabl=
e'
or 'disable'.

Thanks
Kevin