Date:   Fri, 3 Mar 2023 11:38:40 -0800
From:   Darren Hart <darren@os.amperecomputing.com>
To:     Marc Zyngier <maz@kernel.org>
Cc:     Aristeu Rozanski <aris@redhat.com>, linux-kernel@vger.kernel.org,
        "D. Scott Phillips" <scott@os.amperecomputing.com>
Subject: Re: Error reports at boot time in Ampere Altra machines since
 c733ebb7c
Message-ID: <ZAJMwLMNizPMPzS3@fedora>
References: <20230302201732.pwnhg46mum6st2bv@redhat.com>
 <865ybizqfi.wl-maz@kernel.org>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <865ybizqfi.wl-maz@kernel.org>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?DO2NcKRBHv4g16uPwHBdBfFzyLjaLQP/LI//pWcoDkFmJCOMnnEonrYRWOp4?=
 =?us-ascii?Q?N6cva5M2TQ7raTSiF5xXnS0y03oC9IsancYRFcIs6BenVqKRNVOcScLdG3yM?=
 =?us-ascii?Q?Lg0x6H+c1sJzdi+5wpqZoZWHe82BlDyYqnmrydqItTa2hxkGveR8hR79JCy9?=
 =?us-ascii?Q?e7hkE3WvL/E9jC1BYMNTD8lXkMtXOKLi3HYARXgsBHjQOpv06OquH2XvWE+x?=
 =?us-ascii?Q?WYIL/p0629PgrAtJaLCoocfTDc5/UuDs101W9TVTK0oG25A3wJ/MBlghSXk9?=
 =?us-ascii?Q?ivuQXTiEZgWRRrD0tBP2JiOlmtjwhpShv2oMAMhVAzkElmDyb6+NNeIwxMTo?=
 =?us-ascii?Q?pA6owTIje9AsG//HI0/GZP+mL2Co8rFQ7Oc8+7xgKqmWtyJ+vGxFRrLWQDmA?=
 =?us-ascii?Q?ItQGR6GaEP9ugLDRIbvk/AqJUwwos3fwi/U2e8AuJWXzl2C+JlA1iUzUh3Bd?=
 =?us-ascii?Q?jqpO4vOUCbwYO7pckgCHUoJN1/x4dVp55yk4XADKYH3l5nK/gNM/SrHSTWoS?=
 =?us-ascii?Q?aCcJgl1lvSMGhp2ot4ws2dBALXLur7gDsMoAbh3LKe+vPWWimQ2cxZqyxmTx?=
 =?us-ascii?Q?8P+cayGVYRsUvapWOWqpTshtbHC4DIqK+x7LDfiXX1sgsFjS0/KobGoIP6nN?=
 =?us-ascii?Q?hqvSL+PpAcQjxsyxeiyeKh62nIzufeyts87EkBsHP4+LlvRaPfW3NWWLnT9A?=
 =?us-ascii?Q?d8uejl+sMjNZ6Km4CNDp6Y5RS9XeDD6ozNar/jYZb1pRlK7rH54Xgkde/WNZ?=
 =?us-ascii?Q?/TGw2gLMjknvsO644Bz9jgmh+5SqhPbh8iZNVtJkcYOdyVYaTr7onh7fxCzO?=
 =?us-ascii?Q?nWaJRGf8Z3yqZ2xOO2y7pEd9AfCyw0qKUHrG2wdDFwkVIUJx2+zgFpPFQXHH?=
 =?us-ascii?Q?Cay58XrSkSeO4zH1smYT5DotDEUt17FpF84Nu8aMGV7Po884n/Is84wVjsvR?=
 =?us-ascii?Q?cCXEYCdPdME127j/N9WB3dxuM9hj4YDpxzS6cAVaGcw2eTW0QeOfGQIfXqpi?=
 =?us-ascii?Q?suVfAdSE+F/h/57mN9B5ZvIjHnPtVQdLD/+Qju0ljqlQNmEwwlKW7Qr30tOF?=
 =?us-ascii?Q?CsIEa0e+nEP1VxRVNLpQfOzp+Uwx6ip1/kp7iBabTBITKb4XGvVdB8NURMhl?=
 =?us-ascii?Q?WqGFF3otfia0azC09Z/roaJQm7oYcyV3wKe2WOr5XzPuciTvJ3lewAKvm7U6?=
 =?us-ascii?Q?/pg+vXZYuqRWnOZH0WC+7ymuxukagSAZuARZuiUFSLhF3wxcvb3mihg1I3cx?=
 =?us-ascii?Q?S35BS9IoG7AD9I2Be/3T0KJuo0brYMw6Ms97JLrDUrofySLa6/YQen+JklRM?=
 =?us-ascii?Q?Xs5D+uSTM5tkfJVabLjD4aX1CSziJLPuBfqFdxfDxKrmWiuiPuRfKfwar9XB?=
 =?us-ascii?Q?ic1+jUFfdId/DGlBsUvbFyeCtefie6mndtj230iSvO9h4E/7lLpvJg96yNzF?=
 =?us-ascii?Q?EIENfJnevwr7jTijoOyGbfAohi77r4133NFqcZf0DrGyJc6lT78wb1lT0kbN?=
 =?us-ascii?Q?qxm1C4zILwh4Zmav+obwEryfKl4xuDDdGDiP5RNOjYD/5beqpKYYx0g5rCtn?=
 =?us-ascii?Q?4iqUHv2vsTMoO4mEsRtoKDfE5qYbbrsu/f7RjiJ0K1hwlzxSLM+akY64FASv?=
 =?us-ascii?Q?6NmcoP/qZsz1S0M99V5jpug=3D?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e83576fe-bc4f-46fc-4355-08db1c1ee4e3
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR01MB7432.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Mar 2023 19:38:42.0096
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: d5oW5MBpzNWVRQkXko6ApZjlk5r5m5mKib5/d5lDducMuqqVBG4qlWkbTU5f0SpPGB1wTwRWFqpD01L0bYLljgV6d9D2A9SgHrTP8upOTxpjcbhCIZWKiweUIXx7TTZ1
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR0102MB3539
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Mar 02, 2023 at 11:25:37PM +0000, Marc Zyngier wrote:
> On Thu, 02 Mar 2023 20:17:32 +0000,
> Aristeu Rozanski <aris@redhat.com> wrote:
> > 
> > Hi Marc,
> > 
> > Since c733ebb7cb67d ("irqchip/gic-v3-its: Reset each ITS's BASERn
> > register before probe"), Ampere Altra machines are reporting corrected
> > errors during boot:
> > 
> > 	[    0.294334] HEST: Table parsing has been initialized.
> > 	[    0.294397] sdei: SDEIv1.0 (0x0) detected in firmware.
> > 	[    0.299622] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 0
> > 	[    0.299626] {1}[Hardware Error]: event severity: recoverable
> > 	[    0.299629] {1}[Hardware Error]:  Error 0, type: recoverable
> > 	[    0.299633] {1}[Hardware Error]:   section type: unknown, e8ed898d-df16-43cc-8ecc-54f060ef157f
> > 	[    0.299638] {1}[Hardware Error]:   section length: 0x30
> > 	[    0.299645] {1}[Hardware Error]:   00000000: 00000005 ec30000e 00080110 80001001  ......0.........
> > 	[    0.299648] {1}[Hardware Error]:   00000010: 00000300 00000000 00000000 00000000  ................
> > 	[    0.299650] {1}[Hardware Error]:   00000020: 00000000 00000000 00000000 00000000  ................
> > 	[    0.299714] {2}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 3
> > 	[    0.299716] {2}[Hardware Error]: event severity: recoverable
> > 	[    0.299717] {2}[Hardware Error]:  Error 0, type: recoverable
> > 	[    0.299718] {2}[Hardware Error]:   section type: unknown, e8ed898d-df16-43cc-8ecc-54f060ef157f
> > 	[    0.299720] {2}[Hardware Error]:   section length: 0x30
> > 	[    0.299722] {2}[Hardware Error]:   00000000: 40000005 ec30000e 00080110 80005001  ...@..0......P..
> > 	[    0.299724] {2}[Hardware Error]:   00000010: 00000300 00000000 00000000 00000000  ................
> > 	[    0.299726] {2}[Hardware Error]:   00000020: 00000000 00000000 00000000 00000000  ................
> > 	[    0.299912] GHES: APEI firmware first mode is enabled by APEI bit.
> > 
> > Because the errors are being reported later in boot, it's hard to
> > pinpoint exactly what's causing it without decoding the error information,
> > which I currently don't know how to do it.
> 
> + Darren
> 
> Hopefully someone at Ampere can decode this and tell us what is happening.

Hi Marc,

+ D Scott

Thanks for the connection.

This is reporting that something attempted to access GITS2_BASER2, the base
register for the gicv4 vcpu table. Altra doesn't support gicv4. Is c733ebb7c
assuming GITS_BASER2 should be accessible on gicv3?


> 
> > There're no problems other than of course triggering tests because of
> > the warnings.
> 
> It says "Hardware Error". In my book, that's pretty bad. Do you see
> this on more than a single machine?
> 
> > Do you know what's going on here?
> 
> No idea. I haven't seen this on the Altra I have access to so far,
> 
> It could be related to firmware and/or things like power management,
> but again, someone needs to help us with the error report above.
> 
> Thanks,
> 
> 	M.
> 
> -- 
> Without deviation from the norm, progress is not possible.

-- 
Darren Hart
Ampere Computing / OS and Kernel