Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
To:     linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org
Cc:     joro@8bytes.org, will@kernel.org,
        Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Subject: [PATCH] iommu/amd: Fix performance counter initialization
Date:   Mon,  8 Feb 2021 06:27:12 -0600
Message-Id: <20210208122712.5048-1-suravee.suthikulpanit@amd.com>
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from ethanolx5673host.amd.com (165.204.78.2) by SN4PR0501CA0134.namprd05.prod.outlook.com (2603:10b6:803:2c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.11 via Frontend Transport; Mon, 8 Feb 2021 12:20:42 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-HT: Tenant
X-MS-Office365-Filtering-Correlation-Id: add04125-9116-405f-f2f0-08d8cc2bf495
X-MS-TrafficTypeDiagnostic: BYAPR12MB2789:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR12MB278915558AB11D2B2762F2F9F38F9@BYAPR12MB2789.namprd12.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7691;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: zvuERM6cjiZN6oJOGuzbKkBfe4WaMdBDNZMaRDv/XJyz7+Gw1Nj5M3eWoPentCYnpOpOWd2DkaKars/ykhhk9jA5dXqt1V1TweBIOEWzp28OHl74n3+MPHWx9zlFT6D6lMOfiqgKO6XL5A7PZsSVmOONHa25QROHOdZlnyBw0E6UlOrQhPuzvp49PGpijQ2LlDCVJiii2j4hOPgEI8uDuG1SPx+C1XcM+kizusB+k92tSDq2A53n3oCtBC4JHonlOJwBQPpmBSMpRAnDVQLDQM/46zCyVoP4iRPeDTwFzTpRGY0MTpNbDNbvD3w+25cqs6jdlHZJb8lueR17XlyNtE9TCtsehfT5r345CPRCDu+9Im2pz9AgOT7gLNWzlC72dJqatViiyXN+8rXTD+CNUT23fA+4g06m2r260GAmy/W0iXCNRBCv8d0xjGhWlcGVxua8uHrjaFSBzmTyjpKMSVqB8Ujfu74tmgZb+s2M7hzMkWOA53gyly5EU+bWpqjiKmDq4w+PqXoyG3P1w2BWInu6r6ReNSkUQ27ux9tzMtpIWroIZkv+VAKZEqqDiCpcb9/b+ZEkLIIIvPWPBZ6dt5dckyxKEr5ltpbPm9pySiA=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB4597.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(396003)(136003)(346002)(366004)(376002)(956004)(478600001)(86362001)(5660300002)(36756003)(8936002)(8676002)(2906002)(1076003)(6666004)(66476007)(316002)(83380400001)(186003)(6486002)(966005)(16526019)(26005)(66556008)(52116002)(4326008)(44832011)(2616005)(66946007)(7696005);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: =?us-ascii?Q?vsRxLPTyQJjuSDumVVMXq/kdAfjvb1r5nmhxLjX3CKP0L6QlkKuZWDl/DC6U?=
 =?us-ascii?Q?9+FNl3TukIO8VD9f+nU1c5IYV/d/FSIgmPH+HNxSA93kb/6y+KWh4ht/8cha?=
 =?us-ascii?Q?tepvcKBuEFBNWqkh1xlaxO0aFq41mbUHPtI5W1OOKjWnKbpNmzqjSf/W9CaB?=
 =?us-ascii?Q?8Oi/khXgg34Ak5NzvBzwo5Uj0c9HKdwCVr1Fh2iCVWZudZkACvAuYp8fTFLy?=
 =?us-ascii?Q?eVO7yqpyBPZBlmaPjLkIj0YM20MwvFJa1oO0hYSZOxM0Hwa5hcdkV1t0ryop?=
 =?us-ascii?Q?KvYAAdqL5DKcElPCdhLYJR6wt/6qHVkTg/LJlcBXghNKjxKaqz8ZPDBLCXWx?=
 =?us-ascii?Q?b7Vpwb4hZ7Y2T/tHpSZjAQdrufsyswZQzY3lV9CufhjFpEdqSvRSdF/p+EZl?=
 =?us-ascii?Q?J6o6SnsYgY+P8ttfkaL5bCpkp/RUytumw3Ew2oXepLZBsv8Leg6TuOW/IJeQ?=
 =?us-ascii?Q?gNeqoHBPNJavAYYxyJ/g9DTgKc7YvXqXnW/rRUJqLpZILnuIlYf3ZgVCr/ok?=
 =?us-ascii?Q?vWJzt++L1BoBgUg1g8K00+XiGRxnqAH5Jh27zKl1WP0oDGTVzCe+6LbqCx58?=
 =?us-ascii?Q?PYWod0c+iud1HMkf/u4+iFi9/Pm/DS2eiXC1fHBuuyNQ2pi272iNyI+UcTjr?=
 =?us-ascii?Q?7RFZjxLjLmCsKhajNcNHE30Baf4aTRVTrXYxnd54vchmUbs8YdCswd3q5iuI?=
 =?us-ascii?Q?QshU6kmSu8KwqEVFRlF1EMEBYgZPthARYn8qZv6+EsZtvkCzppnA15+rQLu2?=
 =?us-ascii?Q?Aa3klH9bbT881pPEuyuk9xK5gRowFQaLk/+4c4rhHaX6/gHu2peGeY4fh/IV?=
 =?us-ascii?Q?9LGA2eKbsqoYm/Qh5ZGHWDwAa6eLkfK4seFLbPFcd89/5SuM2kiai1U1ntJr?=
 =?us-ascii?Q?ZpBMr4tQ7RqNkdZGeLkj7k1USpBlmyUI8dMcbkxlItuNhWeRIjax5NTAxqt7?=
 =?us-ascii?Q?CJFMiLGQHeqASUx+457lt1Kdc13ndOqS9fXg8VWsHScelvCswuIrTJyPAHIY?=
 =?us-ascii?Q?YBp/XSe7qS9es5jgRG/Uw1zo2hLX5LrdwyZPeBOc0kO2BvUkBEkP7IB+U9xq?=
 =?us-ascii?Q?p5RTGAvhQR9FKQiT9h2f6N2TRy6uAcBtcstBsphIVqk0I6HWdIVgCixWsWqK?=
 =?us-ascii?Q?MXsypy5OgVpG6FXnH2xs3jWjCuCh+2amB41qybRD3Qvdx797zT8JNFFpXT6w?=
 =?us-ascii?Q?Ri8ZhvVy6CNREsfwgD21DSS9hsL+hLS7hl246bKEbDYWKW4ziDsk6zchJ6sU?=
 =?us-ascii?Q?YN46z6ytWmbrHUr2DTzRc/XIQZSwCteNJq/rNmLZaq+gSqZRBdY0G/QwhBXO?=
 =?us-ascii?Q?woiJ0wnJnlnGN4SVeneYVxKQ?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: add04125-9116-405f-f2f0-08d8cc2bf495
X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB4597.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2021 12:20:43.6240
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: hGwWk0etmxleosEFIg573aIQpI9OANPCcIYjTTYThi46hnoNjr0oZdUIrmt2zH8AOZHElflzKRYfBW3WVN7Vxg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2789
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Certain AMD platforms enable power gating feature for IOMMU PMC,
which prevents the IOMMU driver from updating the counter while
trying to validate the PMC functionality in the init_iommu_perf_ctr().
This results in disabling PMC support and the following error message:

    "AMD-Vi: Unable to read/write to IOMMU perf counter"

To workaround this issue, disable power gating temporarily by programming
the counter source to non-zero value while validating the counter,
and restore the prior state afterward.

Tested-by: Tj (Elloe Linux) <ml.linux@elloe.vision>
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=201753
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
---
 drivers/iommu/amd/init.c | 45 ++++++++++++++++++++++++++++++----------
 1 file changed, 34 insertions(+), 11 deletions(-)

diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 83d8ab2aed9f..01da76dc1caa 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -12,6 +12,7 @@
 #include <linux/acpi.h>
 #include <linux/list.h>
 #include <linux/bitmap.h>
+#include <linux/delay.h>
 #include <linux/slab.h>
 #include <linux/syscore_ops.h>
 #include <linux/interrupt.h>
@@ -254,6 +255,8 @@ static enum iommu_init_state init_state = IOMMU_START_STATE;
 static int amd_iommu_enable_interrupts(void);
 static int __init iommu_go_to_state(enum iommu_init_state state);
 static void init_device_table_dma(void);
+static int iommu_pc_get_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr,
+				u8 fxn, u64 *value, bool is_write);
 
 static bool amd_iommu_pre_enabled = true;
 
@@ -1712,13 +1715,11 @@ static int __init init_iommu_all(struct acpi_table_header *table)
 	return 0;
 }
 
-static int iommu_pc_get_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr,
-				u8 fxn, u64 *value, bool is_write);
-
-static void init_iommu_perf_ctr(struct amd_iommu *iommu)
+static void __init init_iommu_perf_ctr(struct amd_iommu *iommu)
 {
+	int retry;
 	struct pci_dev *pdev = iommu->dev;
-	u64 val = 0xabcd, val2 = 0, save_reg = 0;
+	u64 val = 0xabcd, val2 = 0, save_reg, save_src;
 
 	if (!iommu_feature(iommu, FEATURE_PC))
 		return;
@@ -1726,17 +1727,39 @@ static void init_iommu_perf_ctr(struct amd_iommu *iommu)
 	amd_iommu_pc_present = true;
 
 	/* save the value to restore, if writable */
-	if (iommu_pc_get_set_reg(iommu, 0, 0, 0, &save_reg, false))
+	if (iommu_pc_get_set_reg(iommu, 0, 0, 0, &save_reg, false) ||
+	    iommu_pc_get_set_reg(iommu, 0, 0, 8, &save_src, false))
 		goto pc_false;
 
-	/* Check if the performance counters can be written to */
-	if ((iommu_pc_get_set_reg(iommu, 0, 0, 0, &val, true)) ||
-	    (iommu_pc_get_set_reg(iommu, 0, 0, 0, &val2, false)) ||
-	    (val != val2))
+	/*
+	 * Disable power gating by programing the performance counter
+	 * source to 20 (i.e. counts the reads and writes from/to IOMMU
+	 * Reserved Register [MMIO Offset 1FF8h] that are ignored.),
+	 * which never get incremented during this init phase.
+	 * (Note: The event is also deprecated.)
+	 */
+	val = 20;
+	if (iommu_pc_get_set_reg(iommu, 0, 0, 8, &val, true))
 		goto pc_false;
 
+	/* Check if the performance counters can be written to */
+	val = 0xabcd;
+	for (retry = 5; retry; retry--) {
+		if (iommu_pc_get_set_reg(iommu, 0, 0, 0, &val, true) ||
+		    iommu_pc_get_set_reg(iommu, 0, 0, 0, &val2, false) ||
+		    val2)
+			break;
+
+		/* Wait about 20 msec for power gating to disable and retry. */
+		msleep(20);
+	}
+
 	/* restore */
-	if (iommu_pc_get_set_reg(iommu, 0, 0, 0, &save_reg, true))
+	if (iommu_pc_get_set_reg(iommu, 0, 0, 0, &save_reg, true) ||
+	    iommu_pc_get_set_reg(iommu, 0, 0, 8, &save_src, true))
+		goto pc_false;
+
+	if (val != val2)
 		goto pc_false;
 
 	pci_info(pdev, "IOMMU performance counters supported\n");
-- 
2.17.1