Received: by 2002:a05:6358:3188:b0:123:57c1:9b43 with SMTP id q8csp36437881rwd; Tue, 11 Jul 2023 00:36:03 -0700 (PDT) X-Google-Smtp-Source: APBJJlG7ax3XNvKSvMjZlpgLfFJn9hOuaxnoum03MZcVsfS8RzORP2Ygim0cCk5WT+2eaH7RAgHj X-Received: by 2002:a05:6358:9486:b0:134:cbff:c5a4 with SMTP id i6-20020a056358948600b00134cbffc5a4mr16809457rwb.18.1689060962904; Tue, 11 Jul 2023 00:36:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689060962; cv=none; d=google.com; s=arc-20160816; b=h2PsflX5a0aNqWGPKPZJPBDxmJTGjjBaXsvXcDwNiIDnv5+ee0hYBhjxJQJHrsTY6t yjJ7YvQ6DHgjetoQMAhkgREWU0eSAS0RiBYRFVL4D50EY5qcvXtJoTfvD09GYAfBV0Ij b8VAUPiwFfdwaEniKr//I2yfDXh7EyAigEMiqEqKnoCdWldyh65iExrEvGUAUTCsmSHy c0E2uk80st1SjQlJfX+ZgLmiD0BZmf2iFpnlhe2ItHDjZBhQt+GA5RGKiJ51PLF4xQ0r d22GZh2d5Jia9vAASiWIsK9PrKNTk7jNa+oH9ZbZnTwQX/HdWDY2WJfx19lQFBwX6ork nPsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:in-reply-to :subject:cc:to:from:message-id:date:dkim-signature; bh=YIfQWD0wppY/96MJq5E42CbuJh31LEyMDPCBuiWf18Q=; fh=pZaznGCCyrtFVmjyMJMJN6nkeNcYp4yVW1POo88DbvA=; b=FRImpuajqH7t8w/VAD0uJxDXYk0l2+p3qL2fCOMbo8SbzEoeeR8XhscZvsVzDRgpP7 F/39sfUeig3LJGZeiB0NGe5WslVMdmRNKFGrkpIEt9XiKyYr8TzGOXGPz0R3Y+RpVECW cO3JTt2MxL58ZtWQkLL8BDvUUAUgGwBySf0oaruxEVucqKwh8Dpd+W7CCnt6XjG248cY QhUOLZ8aqbBEKGOhvnEtdyLf/m8RE5T4GhSyBJizNdovhRbBLiAWFTmsjoByKQWDm/qr 6Flyg6M/SMMVEMWLBuZskMiaclrAXyFg0Yiu8RVCsvrCKyVKX7SbwPXvr3gE0CIPydjl eK0Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=tDUnRYYQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l129-20020a633e87000000b00543e355adb5si970293pga.294.2023.07.11.00.35.50; Tue, 11 Jul 2023 00:36:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=tDUnRYYQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230314AbjGKG53 (ORCPT + 99 others); Tue, 11 Jul 2023 02:57:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230148AbjGKG52 (ORCPT ); Tue, 11 Jul 2023 02:57:28 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D10B1A8; Mon, 10 Jul 2023 23:57:27 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id AF0E76132C; Tue, 11 Jul 2023 06:57:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1DA20C433C8; Tue, 11 Jul 2023 06:57:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689058646; bh=XbueLfbh+xpb2riKkcfsNl0zXNJkp7eCcVQSRNk/hHc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=tDUnRYYQgv3S4zJf/g5HmzTmuA9OY72vHb9Zq968WcO8YETf+o/zoT6TUAuWVSEXy xWSCsYTcvxoMRjz3mRujltcJX2Bz/ioEnU+zugbAYHW1/eTBWStfEQZR5f4FUuz/Sg ImADdSIECbyQZV9dXQFCcvxGSpoMXvbg2KUGfU5ozV6SZcvG394mJLSliBQS3399mg 5tEM4FnUKA9PDfp3es8crk82z158wd2vqaWbYOpqgRdBnygnSoBjgs+UCcnz24JMlx jUr7vHakZVLkRNN3QIRlN/DdLAfd5IcoBdivycNGOGZtKNhMZWR6MHRvbNErVTStt1 w4n8j79CuvjzA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1qJ7JL-00C29S-Cl; Tue, 11 Jul 2023 07:57:23 +0100 Date: Tue, 11 Jul 2023 07:57:23 +0100 Message-ID: <86lefnvsto.wl-maz@kernel.org> From: Marc Zyngier To: "Aiqun(Maria) Yu" Cc: , , , , , , , , , , Subject: Re: [PATCH] arm64: Add the arm64.nolse_atomics command line option In-Reply-To: <6e07ad52-2629-346e-6217-ec07777ebc5b@quicinc.com> References: <20230710055955.36551-1-quic_aiquny@quicinc.com> <875y6s8bwb.wl-maz@kernel.org> <32f442e3-3d5c-4cec-9791-0da039f88287@quicinc.com> <874jmc8654.wl-maz@kernel.org> <6e07ad52-2629-346e-6217-ec07777ebc5b@quicinc.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: quic_aiquny@quicinc.com, will@kernel.org, corbet@lwn.net, catalin.marinas@arm.com, quic_pkondeti@quicinc.com, quic_kaushalk@quicinc.com, quic_satyap@quicinc.com, quic_shashim@quicinc.com, quic_songxue@quicinc.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 11 Jul 2023 04:30:44 +0100, "Aiqun(Maria) Yu" wrote: > > On 7/10/2023 5:31 PM, Marc Zyngier wrote: > > On Mon, 10 Jul 2023 09:19:54 +0100, > > "Aiqun(Maria) Yu" wrote: > >> > >> On 7/10/2023 3:27 PM, Marc Zyngier wrote: > >>> On Mon, 10 Jul 2023 06:59:55 +0100, > >>> Maria Yu wrote: > >>>> > >>>> In order to be able to disable lse_atomic even if cpu > >>>> support it, most likely because of memory controller > >>>> cannot deal with the lse atomic instructions, use a > >>>> new idreg override to deal with it. > >>> > >>> In general, the idreg overrides are *not* there to paper over HW bugs. > >>> They are there to force the kernel to use or disable a feature for > >>> performance reason or to guide the *enabling* of a feature, but not > >>> because the HW is broken. > >>> > >>> The broken status of a HW platform must also be documented so that we > >>> know what to expect when we look at, for example, a bad case of memory > >>> corruption (something I'd expect to see on a system that only > >>> partially implements atomic memory operations). > >>> > >> > >> good idea. A noc error would be happened if the lse atomic instruction > >> happened during a memory controller doesn't support lse atomic > >> instructions. > >> I can put the information in next patchset comment message. Pls feel > >> free to let know if there is other place to have this kind of > >> information with. > > > > For a start, Documentation/arch/arm64/silicon-errata.rst should > > contain an entry for the actual erratum, and a description of the > > symptoms of the issue (you're mentioning a "noc error": how is that > > reported to the CPU?). > > This is not a cpu's errata as my understanding. It is the DDR > subsystem which don't have the LSE atomic feature supported. CPU or not doesn't matter. We also track system errata. > > > > The workaround should also be detected at runtime -- we cannot rely on > > the user to provide a command-line argument to disable an essential > > feature that anyone has taken for granted for most of a decade... > > We are also seeking help from DDR Subsystem POC to see whether it is > possible to detect the LSE atomic feature support or not at runtime. Keying it off a DT compatible (or something similar) would work. > In my opinion, LSE atomic is a system level feature instead of a cpu > only feature. So currently solution we is that even if cpu support lse > atomic, but it still needed to be disabled if the cpu working with a > lse atomic not support by current system's DDR subsystem. In the absence of a detection mechanism for anything past the CPU, this is a moot point. At this stage, this is a bit like saying "writing to memory is a system thing, not only a CPU feature". And this also breaks KVM if these CPUs don't have FWB, as a guest can always map a piece of memory as non-cacheable, and trigger the issue you describe in your reply to Will, even if you hide the atomics on the host. M. -- Without deviation from the norm, progress is not possible.