Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
From:   Darren Hart <darren@os.amperecomputing.com>
To:     LKML <linux-kernel@vger.kernel.org>,
        Linux ACPI <linux-acpi@vger.kernel.org>
Cc:     "Rafael J. Wysocki" <rafael@kernel.org>,
        Len Brown <lenb@kernel.org>, James Morse <james.morse@arm.com>,
        Tony Luck <tony.luck@intel.com>,
        Borislav Petkov <bp@alien8.de>,
        Doug Rady <dcrady@os.amperecomputing.com>
Subject: [PATCH] ACPI/APEI: Limit printable size of BERT table data
Date:   Tue,  8 Mar 2022 10:50:48 -0800
Message-Id: <43dfaba0646d498fe94c1a8479b812346133f438.1646765290.git.darren@os.amperecomputing.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?XcFRXf45ct6m+hx8X8b2f14ZEHxGOetFZXTxXGPJ7lYQMDCft4zLy7jVCAbL?=
 =?us-ascii?Q?07wu3D8U7ejUGZnLwba0CRqrP3uwDs8UPhfVPvF60uUMJPDFfa+XC7B5pmNl?=
 =?us-ascii?Q?lfQMgTTTvqOpvK3SQk4eM1L0J2YRa9da//a1SKD/6pYEHcLvXXi0297cTY7P?=
 =?us-ascii?Q?Om+8TYH7qYCqIwSCsHaWJCAbRQmmjdiC9oh8lrk+Feyypg5UB6So5NmD7mRn?=
 =?us-ascii?Q?B3jIzRFEcJQjb0dE/6rfH1l3kJGLS+t7nncEFVW4sc9mu2aZSbGuOOMGQZCy?=
 =?us-ascii?Q?LubBghKUAEj39VneV+2yviKyzxjrZo5aapgE3LfEbxfDSGaZaBjCeG7hhTwQ?=
 =?us-ascii?Q?FdD4CoIatdC098Xl1iSy3pinZ+hkjX5sGxVETAqneQ2uEuCAdWF2NClMUkUB?=
 =?us-ascii?Q?zdQmeKqw4u9fQ9hT8/TzMWzfzvZUMyYJLTBmvp2UuwZEyfrJhlAwR6KDo+d/?=
 =?us-ascii?Q?+v3nMr75bqImqO2PRabX1vo8Uz60k5Xkvc8r38LbWfonl2pesGx0CQ3jOgCP?=
 =?us-ascii?Q?U1RyjIackR3Uw/7TlGpCDEjjd8OCU5Wwith9UVxD/C666HXtw4jJdiBZAJsc?=
 =?us-ascii?Q?zsHi6qV0M2PyyFQcMZmySUVP2wVYWbz/opEjZISPmBrS3iK8n7qD+g8ubcov?=
 =?us-ascii?Q?4l+XL6GqA4yAiQgx+7GxubmvP6aU/yPdhwzt3Iv0lN+MH+bp1AiAJM+2OfYY?=
 =?us-ascii?Q?FR+t7S/DZuWOgfUNnnFhErTZ+a2WUkn2j0krBo3F/eAuTKUqbmpaBSb2+psf?=
 =?us-ascii?Q?mXGCaYMmqpXuzqLLGAZ9RN9VBhlTcpj7MWtWOGui7zIjfk9t7fmJ3SJq7AKQ?=
 =?us-ascii?Q?8vMWwx30ez2QnJoC441SpHN5M+iwZ7ZR2fVclmJ4OUlR+fZpQkOTw5DZEnON?=
 =?us-ascii?Q?dqhLDiEkDdXwdUhtepdwuNh5OwU2uaj1edM+Op3HAX6CMBRPLbkcAoqeHKPE?=
 =?us-ascii?Q?bojLUqonOPhfTtzWOthuy+3q/0X6Kg60ZB4DxONrPX8bIJBHgZXXqps8Mn1y?=
 =?us-ascii?Q?1FT+r741T3OZgqeuhhXtY96ssB0ZUeIxb+mBOp5Ob/QeRhvVwM9APzKEWTkj?=
 =?us-ascii?Q?Mwr6lvBx4e6Bg6MHuJM52p7RZ/ZsfkZyLir+0UTpZNxomUxjhwvxiogDUS/w?=
 =?us-ascii?Q?0lfHygIHDbJVPfHlmPzoc0UUSMUV0E4ZsToNqdRgY4KtCXaCaQ4pz9m2xT6q?=
 =?us-ascii?Q?8Ueui/C7k7fcjjx+vlDpcAbi0LAZ/+hpsdxd5CgyfSDjpwqni3tuooJCJ1kW?=
 =?us-ascii?Q?x9/4CT6Z5nw9edB+opN99FfF5ynTFkgKeEDzDhjaHf9jJLIe0b0DkqR/tV0D?=
 =?us-ascii?Q?znNeG2dID4NwuRpGRwauVtI/8cgUMl45RvnubC/tpb4J2IW6Kw6/0zClagh9?=
 =?us-ascii?Q?YMZDZ4O+u7FWzuHQgU5fFI0V2SByT1OxN0GPIpbLCRZTHeR+4OSwGzXP41kR?=
 =?us-ascii?Q?ySKPKOCuqCP0ktOPDnJoV6iHJB9BqJ6bV/y/yVPGnZnRyCVL7fXKtTYMsCkK?=
 =?us-ascii?Q?+k65WTvTrg/yANm8vdcg3Y56BgUZN+Br0lOJSTGm0Kv3P0Td0YAS79wR71/o?=
 =?us-ascii?Q?Y1T0XSRMMk8Eh/1ft2/UbVjLZDPPvNU8DAMU7M1Q4KGFBzj8TNPHPZZy22s8?=
 =?us-ascii?Q?4aE+zLB+rf3R4iYzjufnzhk3/czOwZb339lRVWG5pMbtXGMe0RYRp++yaMEy?=
 =?us-ascii?Q?5nNwbua80qgZQU2XqvVlzVTmY/4SIxQw943DRO/yXomDQo5R4bCx+S7+NIFv?=
 =?us-ascii?Q?PgcppgXauA=3D=3D?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ea944381-f9e7-4891-2d3c-08da01349cc5
X-MS-Exchange-CrossTenant-AuthSource: SA0PR01MB6329.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Mar 2022 18:51:10.8159
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: syM71At/aqhjjeXb+xW2YG+ka6/M4VwwG74GDQroj9KAIgz22CSBCMBiTza26q1ridmDGvmPLSbkn8aLfw8WUkYc6q9jFOuFjf0wC4lFQ+oR7lDctho7sQg9EzhxQ+2w
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR01MB6854
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Platforms with large BERT table data can trigger soft lockup errors
while attempting to print the entire BERT table data to the console at
boot:

  watchdog: BUG: soft lockup - CPU#160 stuck for 23s! [swapper/0:1]

Observed on Ampere Altra systems with a single BERT record of ~250KB.

The original bert driver appears to have assumed relatively small table
data. Since it is impractical to reassemble large table data from
interwoven console messages, and the table data is available in

  /sys/firmware/acpi/tables/data/BERT

limit the size for tables printed to the console to 1024 (for no reason
other than it seemed like a good place to kick off the discussion, would
appreciate feedback from existing users in terms of what size would
maintain their current usage model).

Alternatively, we could make printing a CONFIG option, use the
bert_disable boot arg (or something similar), or use a debug log level.
However, all those solutions require extra steps or change the existing
behavior for small table data. Limiting the size preserves existing
behavior on existing platforms with small table data, and eliminates the
soft lockups for platforms with large table data, while still making it
available.

Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Len Brown <lenb@kernel.org>
Cc: James Morse <james.morse@arm.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Doug Rady <dcrady@os.amperecomputing.com>
Signed-off-by: Darren Hart <darren@os.amperecomputing.com>
---
 drivers/acpi/apei/bert.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/acpi/apei/bert.c b/drivers/acpi/apei/bert.c
index 19e50fcbf4d6..ad8ab3f12cf3 100644
--- a/drivers/acpi/apei/bert.c
+++ b/drivers/acpi/apei/bert.c
@@ -29,6 +29,7 @@
 
 #undef pr_fmt
 #define pr_fmt(fmt) "BERT: " fmt
+#define ACPI_BERT_PRINT_MAX_LEN 1024
 
 static int bert_disable;
 
@@ -58,8 +59,11 @@ static void __init bert_print_all(struct acpi_bert_region *region,
 		}
 
 		pr_info_once("Error records from previous boot:\n");
-
-		cper_estatus_print(KERN_INFO HW_ERR, estatus);
+		if (region_len < ACPI_BERT_PRINT_MAX_LEN)
+			cper_estatus_print(KERN_INFO HW_ERR, estatus);
+		else
+			pr_info_once("Max print length exceeded, table data is available at:\n"
+				     "/sys/firmware/acpi/tables/data/BERT");
 
 		/*
 		 * Because the boot error source is "one-time polled" type,
-- 
2.31.1