From:   Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com>
To:     linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
        iommu@lists.linux.dev, robin.murphy@arm.com, will@kernel.org,
        joro@8bytes.org
Cc:     jean-philippe@linaro.org, darren@os.amperecomputing.com,
        scott@os.amperecomputing.com, gankulkarni@os.amperecomputing.com
Subject: [PATCH] iommu/arm-smmu-v3: Enable PCI ATS in passthrough mode as well
Date:   Thu,  2 Feb 2023 04:40:53 -0800
Message-Id: <20230202124053.848792-1-gankulkarni@os.amperecomputing.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?bSFKmPSomKCnMupTSDp2wAXqfilZG9xno+8lnvb4QaZhEeWvb1kYDwZrvXyc?=
 =?us-ascii?Q?PZxpL3FXyHreI5Gir7pK7CnH1J6RXptRly6DIYgRV7ZnKWX0Ny23hl3fgcnO?=
 =?us-ascii?Q?lMNat55FMFhk8dbH1JpXrl7PXqoQ7M/aRUWf0Gpk7j/MRyYdv+Dm0Xz7bFhe?=
 =?us-ascii?Q?E86THXWagyptD1HYTr5qSebbTQSdPdAphz+mHFdkibtFxc7hL/zF/tH9zH4B?=
 =?us-ascii?Q?+tJy64q6dCuIW9a0y6hU4N40V4OIJsCXbMdB6SnyTn0SKnv7t9xA+oqMU5iq?=
 =?us-ascii?Q?Gd9X6oC+P8aCxnwVT726VKpg4EdnhhLB77prehzgSt5iRj9PsIVsDpASnM9F?=
 =?us-ascii?Q?crWn/y1SGDBR28XWWGOEqpxHTFXJ7m9BBNJJH3MXR42K4kRZ25gcvHbZwoyp?=
 =?us-ascii?Q?wDk+rJIKilNWcb6gvtLDBuLKwHo9qySyf1Da9SHiWRvGz3S/HR84JsuVdnrN?=
 =?us-ascii?Q?aCiBUBUrkxSnYXf4HH4wuBfxYHkmlAYOeZBccuvGAa7kNSd3HyzADlFXw7Lj?=
 =?us-ascii?Q?amtSBSxvJ6TA9QVCYGEr/a7lunHf+tbupApBJtZKcO6IV6kbJoe06mNgh1xH?=
 =?us-ascii?Q?cGCTu5FT3JG8+OpnlPgJxq4k6WBIq0SNyPCjJ1lSRfpKKfkVtKqdcRdxmCQJ?=
 =?us-ascii?Q?4+MONS6/7vx5AZhmHTf3YKlKxgdUTgCNHwk4Jzk6yMo2zGprD/zgvOnvRFWN?=
 =?us-ascii?Q?lPeeqv0of76EVmguhGUA4PO20CWViRLiFQQ10ryLpgXy0orCQyRMeQWGuf9G?=
 =?us-ascii?Q?BhoE3dMCaMb1X5Vd9+H8ruZSvGwN1TbvCUafVs3K6MW2H4/hnKWPNPoSJ12Q?=
 =?us-ascii?Q?yi7INKbwqPiLRB8vEQcOJXJwnZug/63OKLIO32R8jGr819ernxU22v8cWxif?=
 =?us-ascii?Q?Nyc6LhE9BMan6M1DcuhbEkFyiC7pcb6hGA2xEHeOJJn8vclX8iTuNTTrOYTX?=
 =?us-ascii?Q?ZYp34kfQ4YO8ijFa8nrpC54IXPuvdimKJuh9ayfI//PxcDKKCPxsGAzDBXZl?=
 =?us-ascii?Q?RyIrf/G6TEtK30TMpo4L0kioZ/3SvcCYi+kj9IKgrpiyO1rcH0I753Z4bVNQ?=
 =?us-ascii?Q?4Pr6a1ZKoiZYUIdUyBk7BFjZcYRjL24nHBrzN+HUuhaUh8pyNuIbgfwO4LOt?=
 =?us-ascii?Q?WbeFZTBCVK8uvbswZ81r2gB4HemKFzAi2yNMm5ZvHxULKvDJFyBf0HuNUm3I?=
 =?us-ascii?Q?fdX6df7aEPtdpkDdinJfasyzNhT8sxf9DUVrBu4LWjra9Gpfw842Q4+QNrX5?=
 =?us-ascii?Q?rs7wFVssX8xcIE3n+5IqUVb8j1dcqHJxOT8A/Q1yLm9N3f05IioMCqbdG0FU?=
 =?us-ascii?Q?3mxXJXINv8REgUn9lb0s7bhSJfBuKq1gTjwmHuDyZyeXr+YvM0ZJ4hw7jKqf?=
 =?us-ascii?Q?TbYVDYajRtZVVVt5sCBs4zdMrOKmK7r7T2dWh+DuM9DCaUl3QewLhQyoF+/J?=
 =?us-ascii?Q?wQWZvP9+mtx5oAVKstfUjCAyb7kbq9o9KTlENVNVte+R2JIGJCuxeR1TpvKx?=
 =?us-ascii?Q?4jNmkVaFJen140Ln0fVnezHx4iMIW5jqm8U91JhRJ4VarHqumzynWvGSUys5?=
 =?us-ascii?Q?3hGbrizIs55aNf1zT2iNyRlt6v3rPwaGuZ22Km/Oxem+/f/NU8zVKqXVhoOd?=
 =?us-ascii?Q?1r/RPGQgwphajVWaBgKl2xA=3D?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 25fbf559-9139-4a5f-0d91-08db051ac573
X-MS-Exchange-CrossTenant-AuthSource: DM8PR01MB6824.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2023 12:41:14.4870
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9obSNp/BVMPZW1zjQzkm+nJ4bb002nPbinRlOVkkYN6YBCah8O2h4fhbld23jL8yQznuD60orHDoGn0tjIAy30TiAU6SGidjv2TV+sGhiuhWG1pEKmLdDe/fF8dspAR2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY3PR01MB6689
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

The current smmu-v3 driver does not enable PCI ATS for physical functions
of ATS capable End Points when booted in smmu bypass mode
(iommu.passthrough=1). This will not allow virtual functions to enable
ATS(even though EP supports it) while they are attached to a VM using
VFIO driver.

This patch adds changes to enable ATS support for physical functions
in passthrough/bypass mode as well.

Also, adding check to avoid disabling of ATS if it is not enabled,
to avoid unnecessary call-traces.

Signed-off-by: Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 6d5df91c5c46..5a605cb5ccef 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2313,11 +2313,16 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master)
 static void arm_smmu_disable_ats(struct arm_smmu_master *master)
 {
 	struct arm_smmu_domain *smmu_domain = master->domain;
+	struct pci_dev *pdev;
 
 	if (!master->ats_enabled)
 		return;
 
-	pci_disable_ats(to_pci_dev(master->dev));
+	pdev = to_pci_dev(master->dev);
+
+	if (pdev->ats_enabled)
+		pci_disable_ats(pdev);
+
 	/*
 	 * Ensure ATS is disabled at the endpoint before we issue the
 	 * ATC invalidation via the SMMU.
@@ -2453,8 +2458,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev)
 
 	master->domain = smmu_domain;
 
-	if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS)
-		master->ats_enabled = arm_smmu_ats_supported(master);
+	master->ats_enabled = arm_smmu_ats_supported(master);
 
 	arm_smmu_install_ste_for_dev(master);
 
-- 
2.37.3