Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Jacob Pan <jacob.jun.pan@linux.intel.com>,
        LKML <linux-kernel@vger.kernel.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        Lu Baolu <baolu.lu@linux.intel.com>,
        Joerg Roedel <joro@8bytes.org>
CC:     Robin Murphy <robin.murphy@arm.com>, Will Deacon <will@kernel.org>,
        "David Woodhouse" <dwmw2@infradead.org>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "Liu, Yi L" <yi.l.liu@intel.com>,
        "stable@vger.kernel.org" <stable@vger.kernel.org>,
        "Ghorai, Sukumar" <sukumar.ghorai@intel.com>
Subject: RE: [PATCH v3] iommu/vt-d: Fix PASID directory pointer coherency
Thread-Topic: [PATCH v3] iommu/vt-d: Fix PASID directory pointer coherency
Thread-Index: AQHZZ9UC392MOCylEECV+/pkunR29a8z8Ghg
Date:   Thu, 20 Apr 2023 08:16:05 +0000
Message-ID: <BN9PR11MB52762EABB8EF8E685291604A8C639@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20230405154447.2436308-1-jacob.jun.pan@linux.intel.com>
In-Reply-To: <20230405154447.2436308-1-jacob.jun.pan@linux.intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?OAgjVgJfFq/dwmTPaiwH3UksvtfLhaaGCq3s2KsB1GP0VmZ66qi3rGidJe6O?=
 =?us-ascii?Q?wVNiABd4cgsnodf9NvmdcXty7KOXIGLKscxk/Vpm6XzfxJR6VYHPyfHCrne0?=
 =?us-ascii?Q?8i0Adyd97JfV0R4JvhIQjzyiDJTN1xGd+autQ2EgZ9MPiUgdMguPAnAQJnb8?=
 =?us-ascii?Q?Atiu9dXAVAkwB9YmKzZIpobPLhSw9cWU+HyoP18AoSv11h9d63OiX8+5lve1?=
 =?us-ascii?Q?b9JwYs3Sq/uF6cF02HS0aKQJflCJ3Gc9mjgMrPja2KKX7ZBxPoXTUl4/ax9f?=
 =?us-ascii?Q?cX0RaYcMBE9SVonFTVT4wauM/+73Y7R7kGlw/fo6GDW8AjqhgJqjSVzXBrVa?=
 =?us-ascii?Q?ZO+ex86QVOc0qDNyOiVilOvG4VCS02/wRTuPcH17q/PE/+Wt/tdFfkBWTKMK?=
 =?us-ascii?Q?wLme5n1WK0KPSP47zfggeR9SpKjIe9e31QHGtyT3d796mKA9tLzoJNceRcQF?=
 =?us-ascii?Q?2TXO1AtV6RvEXBpLIruiW9tGglLNuO6Wz0ROA6SXDAqzv84acOdDh3cyptVa?=
 =?us-ascii?Q?ujxHKsw2x6MOM0RdxlUdNEztX7bGWy1LT3C5/YDjFoPMTG3VFanLMigtiQsD?=
 =?us-ascii?Q?3pAw+iWH7i6Btqjpo/NzOcqsiWHNPXLPscx9lAhj8w9ZcgqF4qOb/YOWZ4NT?=
 =?us-ascii?Q?PfYcpUUGYm0QKLa1DSazOP0qCUaPn6KWYg62nj2Hz8acMoH/OgpyqHV8N6XE?=
 =?us-ascii?Q?HbFe4xug8AnU8cm2bqBHc55buRzD01zaG59+102VKvVIgfJY9AQaX+FtPXEK?=
 =?us-ascii?Q?EzZwd3LiqMs2r/iIGnnQzLcEDRw9vh47FKnSVY04AEk4zSaQOC8Z/2Z4cDmU?=
 =?us-ascii?Q?FYyE0YTYe7ih5gxU0pItF++gDzdcK35c5ST7BziKRwS3liAaW++DmMhF3nJd?=
 =?us-ascii?Q?WlHFAjPno6ashEAlyRJyKqzglAEO4FzCx9sfsM1VnGJLQcJv2CeakqMrf6Eq?=
 =?us-ascii?Q?SHttOAwHAOj8UslbsDZRZbhJ+N6Ip0mC/kksF6xHTpxg4HlyxESZx9B5SaX2?=
 =?us-ascii?Q?vyPMO3sOaUzQwuiFzSVDEXHr9W8R4/Mkm/LG+CKfkFqgrit9RXy5eOoHKHHS?=
 =?us-ascii?Q?6ClWZJ0JTduDwM23ozwkiUAzzjLI7qDM8LbCgbGBt4H3bIDqlPA/boGSfINS?=
 =?us-ascii?Q?nvoOSfhrFTiU6QEe8TjM3hV5ZIPp2/zZM3tiPIolzqUeqmpo8RDc+Gbzkn2Y?=
 =?us-ascii?Q?10sQKYyspMra82jUhw9L77PeAv2Oy0LixVLl2Q3UU/Zam+bcD0rU3X56QmuH?=
 =?us-ascii?Q?Dt9nGecIZeyZKhZ9cBcaDWJ2fL0238e6UJ3ttG4p41/Fi5FluTffoF94z+/R?=
 =?us-ascii?Q?RcU8+HiCOlZgeTRbXnVccOFdrzEZg4B+j19EvJvXmX07MzxS370qem/fZuLF?=
 =?us-ascii?Q?8VrqjKWLwC5/1ZlDniy6nUEzTDf5nfWSv7EFPEvqk4iIHGmHBidhRrwoBOng?=
 =?us-ascii?Q?FdZDEDxVlXhS6OSZ60qcqB9BFAdkA7/3IE+uLIIxSGVK8ATAUM+zNvSvfayG?=
 =?us-ascii?Q?EGc4rxur2wd3hH82NjNqqtJY1ypomGxD9fVOALkzAsTsqATDgZq4ZT2jkDi8?=
 =?us-ascii?Q?8r0y/SvrCCgr7W4shgjFgzqnlRUaZnZthOuoXMfK?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 600106ac-5185-4f75-4ab8-08db41777cd0
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Apr 2023 08:16:05.3401
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: SChZS9oeYd8Hv6v0LwpCJngZ5zZe+Vppcn1GEAt6bK/ZfPTQLd+LXK9URENiqnup6W/IioD8OqHC5KM4evwedg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB8130
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Jacob Pan <jacob.jun.pan@linux.intel.com>
> Sent: Wednesday, April 5, 2023 11:45 PM
>=20
> On platforms that do not support IOMMU Extended capability bit 0
> Page-walk Coherency, CPU caches are not snooped when IOMMU is
> accessing
> any translation structures. IOMMU access goes only directly to
> memory. Intel IOMMU code was missing a flush for the PASID table
> directory that resulted in the unrecoverable fault as shown below.
>=20
> This patch adds clflush calls whenever allocating and updating
> a PASID table directory to ensure cache coherency.
>=20
> On the reverse direction, there's no need to clflush the PASID directory
> pointer when we deactivate a context entry in that IOMMU hardware will
> not see the old PASID directory pointer after we clear the context entry.
> PASID directory entries are also never freed once allocated.
>=20
> [    0.555386] DMAR: DRHD: handling fault status reg 3
> [    0.555805] DMAR: [DMA Read NO_PASID] Request device [00:0d.2] fault
> addr 0x1026a4000 [fault reason 0x51] SM: Present bit in Directory Entry i=
s
> clear
> [    0.556348] DMAR: Dump dmar1 table entries for IOVA 0x1026a4000
> [    0.556348] DMAR: scalable mode root entry: hi 0x0000000102448001, low
> 0x0000000101b3e001
> [    0.556348] DMAR: context entry: hi 0x0000000000000000, low
> 0x0000000101b4d401
> [    0.556348] DMAR: pasid dir entry: 0x0000000101b4e001
> [    0.556348] DMAR: pasid table entry[0]: 0x0000000000000109
> [    0.556348] DMAR: pasid table entry[1]: 0x0000000000000001
> [    0.556348] DMAR: pasid table entry[2]: 0x0000000000000000
> [    0.556348] DMAR: pasid table entry[3]: 0x0000000000000000
> [    0.556348] DMAR: pasid table entry[4]: 0x0000000000000000
> [    0.556348] DMAR: pasid table entry[5]: 0x0000000000000000
> [    0.556348] DMAR: pasid table entry[6]: 0x0000000000000000
> [    0.556348] DMAR: pasid table entry[7]: 0x0000000000000000
> [    0.556348] DMAR: PTE not present at level 4
>=20
> Cc: <stable@vger.kernel.org>
> Fixes: 0bbeb01a4faf ("iommu/vt-d: Manage scalalble mode PASID tables")
> Reported-by: Sukumar Ghorai <sukumar.ghorai@intel.com>
> Signed-off-by: Ashok Raj <ashok.raj@intel.com>
> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
> ---
> v3: Add clflush after PASID directory allocation to prevent malicious
> device attack with unauthorized PASIDs. Also flush all the PASID entries
> after directory updates. (Baolu)
> v2: Add clflush to PASID directory update case (Baolu, Kevin review)
> ---
>  drivers/iommu/intel/iommu.c | 2 ++
>  drivers/iommu/intel/pasid.c | 7 +++++++
>  2 files changed, 9 insertions(+)
>=20
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 59df7e42fd53..161342e7149d 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -1976,6 +1976,8 @@ static int domain_context_mapping_one(struct
> dmar_domain *domain,
>  		pds =3D context_get_sm_pds(table);
>  		context->lo =3D (u64)virt_to_phys(table->table) |
>  				context_pdts(pds);
> +		if (!ecap_coherent(iommu->ecap))
> +			clflush_cache_range(table->table, sizeof(u64));

v2 of this patch was already merged w/o this change.

can you elaborate the purpose of v3? Here no flush is required as long
as it's done in other two places below.

>=20
>  		/* Setup the RID_PASID field: */
>  		context_set_sm_rid2pasid(context, PASID_RID2PASID);
> diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c
> index fb3c7020028d..979f796175b1 100644
> --- a/drivers/iommu/intel/pasid.c
> +++ b/drivers/iommu/intel/pasid.c
> @@ -128,6 +128,9 @@ int intel_pasid_alloc_table(struct device *dev)
>  	pasid_table->max_pasid =3D 1 << (order + PAGE_SHIFT + 3);
>  	info->pasid_table =3D pasid_table;
>=20
> +	if (!ecap_coherent(info->iommu->ecap))
> +		clflush_cache_range(pasid_table->table, size);
> +
>  	return 0;
>  }
>=20
> @@ -215,6 +218,10 @@ static struct pasid_entry
> *intel_pasid_get_entry(struct device *dev, u32 pasid)
>  			free_pgtable_page(entries);
>  			goto retry;
>  		}
> +		if (!ecap_coherent(info->iommu->ecap)) {
> +			clflush_cache_range(entries, VTD_PAGE_SIZE);
> +			clflush_cache_range(&dir[dir_index].val, sizeof(*dir));
> +		}
>  	}
>=20
>  	return &entries[index];
> --
> 2.25.1