Received: by 2002:a05:6358:3188:b0:123:57c1:9b43 with SMTP id q8csp10335293rwd; Wed, 21 Jun 2023 21:22:21 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5hTidV7yT6p5EIiTk2WZXIqf8hcWpG3DxjYRF9gaWYNST5DapR4SzMZhE4Y+PIikf8DWG+ X-Received: by 2002:a17:902:ea01:b0:1b2:1ee9:682 with SMTP id s1-20020a170902ea0100b001b21ee90682mr18713216plg.23.1687407741146; Wed, 21 Jun 2023 21:22:21 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1687407741; cv=pass; d=google.com; s=arc-20160816; b=bKKt8j6BoEuFvcAa8bIA0jiuoJqU/s/+mt7rid2wME4gDGXzfTyZ10aS378hX9l2ok xsXfqIeKa+Ftlk97FTV4wsDtyLkS11oEyV+7+d2TpnlkgzQVCIW7gcOsL/qv5rlCEhtS mpvr2DU/Oi6ujNnY+TCJjWL7qBawZP/0ksTa2GlIJ0IKI85hzn3ZeX60Ssp7kqss/csM k0tzwe4i1gg+ISdHNzf0DKFq3rDVxwdb0KvIpIq+wdlpk0qI8W5XwE5u477B2KaPP21r j4l0G6mi3mdKGhxxXxits1crhTaQmQEW7aKUTCy2Tp1nME6/dgGjVoqqQIKf8uPHbo1Y X2tA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=N2/XgMTWuARAg6h/D9gYdx1ekNbDBST9o6bpKIvL1nc=; b=IQCmYKeneS/OOztbOdzUTWs2CgDW8Xg8TMs+nrRo1COl+Si3pu2wFJkZmS8+PBzdBr Z+9Ert7p/qFHMT9BmOTpgjK63QJaVk+pYoCzXDISwlh2JEILlXgmC+vtRxLbE1U0bjJP 21PkZj3gfaNXDXEk9F7LKMU9tDF46wBCU9+3iP+8nYpleNG0HntWkffIlru3dUEsxCDd Dew8BEyY49pCaV06hc+lGxOAhA1zxw09XKmXa6SZVUZZSn/JXiXEgpZzAmRCUBzynZ2A LI/Tm8vy/JOs1apTtBkE3UTZ6Ec/QHj/dHf788n8ggVyMn0FSgAqjd97jZRkG5s2SRyp mZjQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=BlShjtIU; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l12-20020a170902f68c00b001b51ff840fcsi6221492plg.456.2023.06.21.21.22.09; Wed, 21 Jun 2023 21:22:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=BlShjtIU; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229680AbjFVD7a (ORCPT + 99 others); Wed, 21 Jun 2023 23:59:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229958AbjFVD6u (ORCPT ); Wed, 21 Jun 2023 23:58:50 -0400 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2043.outbound.protection.outlook.com [40.107.243.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93C972961; Wed, 21 Jun 2023 20:57:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZM+2w9pO8INAwi9/r9Tqa0pWO3/frepQyNQ/CH0Vgeu84A74PanjODhaXrH3qbfe3i31IU+/U6328+21cunOSLLSU3sKUdt2rd6QPUe022IAC7AVHIrC9VzObvNhdRRmPo4yo0xZKFaYkpXaiAUN5FsrjPeVqpyQsT00eR+iRSUQw9OMdpoNujXM5nLoBtSeGJNByiPc+3IXm3hhLeRjzbKGE4LmpCuZmuNsXNfmO0Aq0pDjN4USJ7y34Os6tMlAl+C9McmyPueHj5pLH59tIwMPl2alop5TWLUMzqnQHA9U+saUHOgpN1rPsdesnDjmz2vNree2QY32DGICPWnjdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=N2/XgMTWuARAg6h/D9gYdx1ekNbDBST9o6bpKIvL1nc=; b=biBTBhaaxJriraAJJWLJ3byHtIm4nEPBqs2YbJH9swUls3z0G8hRizl+f+YCNsiOrm0tj316Xrym+gErsbS+4m1CjyIMJ7Jh3wsmqeVboaoK3gaim7BzLSjNwz+D8gwAkxRqOFJiFfn8/ZhtHGceJK9+zVFkYndmF0vOD4OfKD0qtEM25xdZk1Kjhg9F0Qi/8AnBKMTwiH8c2KlxMkbvOHV6ai1lfClBlCRp0J0Ybh6tkRSA5XA5uGslC9j7tUWKmpSWr+soGUYdcl47fyNjx+QgQoQc8rPWXaX8JwAEZoyXK36PZa7XN0CC5osaPv75cstOuuSD+XQ5g4NEUFmy0g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N2/XgMTWuARAg6h/D9gYdx1ekNbDBST9o6bpKIvL1nc=; b=BlShjtIU7E6OVzWVyvrbyrl9q/89vKb+9W9n5FEieGZizrRqobIMnhobRMGT96bTeiLYLlcu3NGfnd3sue0/vvhYFOJ/uo8MpvrMn+cLpH3VlNGqB0tm2oktZlQWGb0jbZc7PIV4ZPK/yMX4YRuU4EWmKBRVLVEG1NBo1XLG+os= Received: from BN9PR03CA0597.namprd03.prod.outlook.com (2603:10b6:408:10d::32) by CH3PR12MB9100.namprd12.prod.outlook.com (2603:10b6:610:1a8::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.23; Thu, 22 Jun 2023 03:57:07 +0000 Received: from BN8NAM11FT098.eop-nam11.prod.protection.outlook.com (2603:10b6:408:10d:cafe::ab) by BN9PR03CA0597.outlook.office365.com (2603:10b6:408:10d::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.24 via Frontend Transport; Thu, 22 Jun 2023 03:57:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT098.mail.protection.outlook.com (10.13.177.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6500.39 via Frontend Transport; Thu, 22 Jun 2023 03:57:07 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Wed, 21 Jun 2023 22:57:05 -0500 From: Terry Bowman To: , , , , , , , CC: , , , Subject: [PATCH v6 24/27] cxl/pci: Add RCH downstream port error logging Date: Wed, 21 Jun 2023 22:51:23 -0500 Message-ID: <20230622035126.4130151-25-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230622035126.4130151-1-terry.bowman@amd.com> References: <20230622035126.4130151-1-terry.bowman@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT098:EE_|CH3PR12MB9100:EE_ X-MS-Office365-Filtering-Correlation-Id: c905c40d-9c02-4251-b855-08db72d4bf48 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TaM9kTDH/dP8BInWBswDGC5AWtsULgL0h0yQzWYpSOOTONnAUX0/592+DQqK2Rji/MnTUegLBcs4+eTnE6bryWF7fOvQWQm8Oqvl6GAKHY+wJXJEDO3Qj79YsYexcjpY16rd5QzROIpMKaSIB03yFvWidRHBl6WeYww0VF0L7ccLpZNh3tx1kj++NzjuQRkQ0rsGmfaEHZCM9wsk5ipBT2mNKE6bAWotIzruPS1TOFh54PF9Zds0s/lnnaUCp4Oj4bl/qFn8eGjTpwMOmDw2HAr57Hg1/lCRQYaoIcTI5V+kKxHn5+h1CLQlFZpdCkH7FHoz1HHHUY5l8WG8pleWO1a6uHSvLByQ1wIUU51kNfbmltcCqox3zwDxtghMtsi6/SRomPk2IY4meBDxrCHWHwUewaJzgcUmb+J81aXOC+lL/gdm9V5YxKZwHBVYwtv4TpRGdWlljLRxDGRhGoCI+0Zb5kOVLhUWpyAzlqZVLBhRmC1RTfMWN449jGrR4k6uHSafq+7yiYFM0oOvWgVgCb1gdEYmPuZYK4iAofcnD+M7ylIbpFb/WxYbOzKJUZ6GABAv1HbzUoyDSYt2zJGe3tpj6vmyYY1DV2WoQuLvQAJIlj10E/6kZVtT0ynX+akSNNNHkcPKvHDSSRvGsEEsO9hPundaK73ezt23EkhSFK2YK8sDx+rvFfEaJgLATgt8qyfVJ4yptyt/ECCOM1m2mDyj353ykYxAix58PrD9/o6TuqpM9CMgJnowtvj76neaj8aYZy0tAj44BdjI9MWE4Q== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(376002)(39860400002)(346002)(136003)(396003)(451199021)(36840700001)(40470700004)(46966006)(356005)(81166007)(82740400003)(2906002)(40480700001)(44832011)(7416002)(36860700001)(47076005)(426003)(83380400001)(36756003)(86362001)(1076003)(2616005)(186003)(16526019)(26005)(7696005)(40460700003)(336012)(82310400005)(6666004)(54906003)(110136005)(478600001)(70586007)(70206006)(41300700001)(8676002)(316002)(5660300002)(4326008)(8936002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2023 03:57:07.0435 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c905c40d-9c02-4251-b855-08db72d4bf48 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT098.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9100 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org RCH downstream port error logging is missing in the current CXL driver. The missing AER and RAS error logging is needed for communicating driver error details to userspace. Update the driver to include PCIe AER and CXL RAS error logging. Add RCH downstream port error handling into the existing RCiEP handler. The downstream port error handler is added to the RCiEP error handler because the downstream port is implemented in a RCRB, is not PCI enumerable, and as a result is not directly accessible to the PCI AER root port driver. The AER root port driver calls the RCiEP handler for handling RCD errors and RCH downstream port protocol errors. Update existing RCiEP correctable and uncorrectable handlers to also call the RCH handler. The RCH handler will read the RCH AER registers, check for error severity, and if an error exists will log using an existing kernel AER trace routine. The RCH handler will also log downstream port RAS errors if they exist. Co-developed-by: Robert Richter Signed-off-by: Robert Richter Signed-off-by: Terry Bowman Reviewed-by: Jonathan Cameron --- drivers/cxl/core/pci.c | 101 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 9cb39835e154..9e0eba5ccfc4 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -747,10 +748,107 @@ static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) return __cxl_handle_ras(cxlds, cxlds->regs.ras); } +#ifdef CONFIG_PCIEAER_CXL + +static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, + struct cxl_dport *dport) +{ + return __cxl_handle_cor_ras(cxlds, dport->regs.ras); +} + +static bool cxl_handle_rdport_ras(struct cxl_dev_state *cxlds, + struct cxl_dport *dport) +{ + return __cxl_handle_ras(cxlds, dport->regs.ras); +} + +/* + * Copy the AER capability registers using 32 bit read accesses. + * This is necessary because RCRB AER capability is MMIO mapped. Clear the + * status after copying. + * + * @aer_base: base address of AER capability block in RCRB + * @aer_regs: destination for copying AER capability + */ +static bool cxl_rch_get_aer_info(void __iomem *aer_base, + struct aer_capability_regs *aer_regs) +{ + int read_cnt = sizeof(struct aer_capability_regs) / sizeof(u32); + u32 *aer_regs_buf = (u32 *)aer_regs; + int n; + + if (!aer_base) + return false; + + /* Use readl() to guarantee 32-bit accesses */ + for (n = 0; n < read_cnt; n++) + aer_regs_buf[n] = readl(aer_base + n * sizeof(u32)); + + writel(aer_regs->uncor_status, aer_base + PCI_ERR_UNCOR_STATUS); + writel(aer_regs->cor_status, aer_base + PCI_ERR_COR_STATUS); + + return true; +} + +/* Get AER severity. Return false if there is no error. */ +static bool cxl_rch_get_aer_severity(struct aer_capability_regs *aer_regs, + int *severity) +{ + if (aer_regs->uncor_status & ~aer_regs->uncor_mask) { + if (aer_regs->uncor_status & PCI_ERR_ROOT_FATAL_RCV) + *severity = AER_FATAL; + else + *severity = AER_NONFATAL; + return true; + } + + if (aer_regs->cor_status & ~aer_regs->cor_mask) { + *severity = AER_CORRECTABLE; + return true; + } + + return false; +} + +static void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds) +{ + struct pci_dev *pdev = to_pci_dev(cxlds->dev); + struct aer_capability_regs aer_regs; + struct cxl_dport *dport; + struct cxl_port *port; + int severity; + + port = cxl_pci_find_port(pdev, &dport); + if (!port) + return; + + put_device(&port->dev); + + if (!cxl_rch_get_aer_info(dport->regs.dport_aer, &aer_regs)) + return; + + if (!cxl_rch_get_aer_severity(&aer_regs, &severity)) + return; + + pci_print_aer(pdev, severity, &aer_regs); + + if (severity == AER_CORRECTABLE) + cxl_handle_rdport_cor_ras(cxlds, dport); + else + cxl_handle_rdport_ras(cxlds, dport); +} + +#else +static void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds) { } +#endif + void cxl_cor_error_detected(struct pci_dev *pdev) { struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); + if (cxlds->rcd) + cxl_handle_rdport_errors(cxlds); + cxl_handle_endpoint_cor_ras(cxlds); } EXPORT_SYMBOL_NS_GPL(cxl_cor_error_detected, CXL); @@ -763,6 +861,9 @@ pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, struct device *dev = &cxlmd->dev; bool ue; + if (cxlds->rcd) + cxl_handle_rdport_errors(cxlds); + /* * A frozen channel indicates an impending reset which is fatal to * CXL.mem operation, and will likely crash the system. On the off -- 2.34.1