Received: by 2002:a05:6a10:f347:0:0:0:0 with SMTP id d7csp10226606pxu; Tue, 29 Dec 2020 18:35:49 -0800 (PST) X-Google-Smtp-Source: ABdhPJxlwRAUhuuNE9sHL1iiqILzK+nTi4dqnhAVvcqAESsC7WLBpJJYMItOns8naTSy3k49Usck X-Received: by 2002:a17:906:9605:: with SMTP id s5mr48858015ejx.179.1609295749625; Tue, 29 Dec 2020 18:35:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1609295749; cv=none; d=google.com; s=arc-20160816; b=Fknemt5LRC3Gxntt0lNR6yBQY/HEehkN4U+y6d666QFKUPOo0/fE9MksL4LZEpdR2l ruNpiOjW3VrmJ6CTY/sf4MUkzsLQdp3i35HQeOy/9c/RnCqA5zxzWuJY5NI6XV7RLzMt 2Ob3SxUAqFMBEVJ+RAEen7GB/zYk4xuf+wclWniL6wVjT/ok6A0ghRbeUwjb9v7aGBGL yxeWogl/UoyvHSlg3B5ikxxF2QOaNWjgqF6sylhE/CjfZ6+qlNYR2RYce5TGUFCAKkHC 6z2EjOfsdpMQPSqoVg3jR7qCdZxaBhuen+/kq7Oj+tEJEwHv45LDaMBZYFjYNSiOi2G0 iBYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:message-id :mime-version:in-reply-to:references:cc:to:subject:from:date :dkim-signature; bh=PfIl0EjmWMxnSF+EOk80ZAbjAPzwaEf3qyLecMl/XRc=; b=utAAFYI13UoIjsC51qEPvz76ThRK/22tWGLwG/4dyJN/9pkdeCp8yuob0438j1QUs9 5gbY6S44dwKJzClRdX1Z1VsCICeLLtepYOh4tAXYX7KmN7kacJ2jRikBaNWuH/7NaTmq BTo+JhUkxsZZ3BH9ioj7cgqkTGj5G9JUQN25jkYl4KPcL27/revol+UXLvz0StzaUj1f 1aE0eIlEc9PE58zMD69wvUCSZhmzDFcIinqjQQuDOoPEH1P3mbMuyf8qzYY3OoVR3EKM qKeQW/3vXuoweAzGNYYWAzhX0zwpxx7XHboR6THUSZggqQV/m795Sjc/fyvNnjxO3Ifz yPlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=nDaFL5lx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id j12si21285161ejb.431.2020.12.29.18.35.27; Tue, 29 Dec 2020 18:35:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=nDaFL5lx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726350AbgL3Cdv (ORCPT + 99 others); Tue, 29 Dec 2020 21:33:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56500 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726325AbgL3Cdu (ORCPT ); Tue, 29 Dec 2020 21:33:50 -0500 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46D53C06179C; Tue, 29 Dec 2020 18:33:10 -0800 (PST) Received: by mail-pf1-x42c.google.com with SMTP id c12so8966254pfo.10; Tue, 29 Dec 2020 18:33:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:subject:to:cc:references:in-reply-to:mime-version :message-id:content-transfer-encoding; bh=PfIl0EjmWMxnSF+EOk80ZAbjAPzwaEf3qyLecMl/XRc=; b=nDaFL5lxYCUZxEPCVotfMjioI5BLNIhicG92GYRl+bW9o7TrWxGxPe0PqDXDUdvxGL ovvYBF0bISGn/Zpzf1pyHYOdLFAxrz3UfAR1lXz51N/K03KuWE71VFegfMD1DDIP/drp KUhqJ9Fn3WcSfrmIuDfPZ/QOnNSC22L/Z/ORAmWhZN1D2xASIe52OucM/qXVfSayaGAT 57fbAo1T7kYgyhJGZ9Evd5nqFiAIc1edFYHuzkkIxctQBFyOJMPSHfXVdoKBjZA9clIV kEZ8gdlTRJigmHZJrUOZzCZPE80ijU9zMl4P4nZYuEkJlMxEzvQaP8bXxzoxR5aFvcz5 +Drg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:subject:to:cc:references:in-reply-to :mime-version:message-id:content-transfer-encoding; bh=PfIl0EjmWMxnSF+EOk80ZAbjAPzwaEf3qyLecMl/XRc=; b=R6NiuCYK4Ut9dRe3OD4qgs8HrMmZEUHke/1bbzKnnsGMJ20knc4qNRR4hPm5K/Wsad 2zc2F16BJ533Q0I4JJr/xLNLAAnjlrmWYOqA3kZcwQ4wdKLKY99WVUilz8XOZAgAWt8I P04aQ6HaTkn/i686C/eHr977CKP2EqUXveXjXa6MNkzgdfrLaCQxKJ8yY3X7s8r5zx+i m5Oirx2CQD7Rr48DOCH7705mJs0C/PsY1loSHUyjTWJBEL6yv1XKqNVuo+FBMfR6wshM Q5C4vsHM4bzRG5pEamm7dPwEBqWoPlWKRly71qY5qzkAsTiJi1HV+jz/tmIJKZb3jcxB wWbQ== X-Gm-Message-State: AOAM532bt8w44ndQjX0cW6qCfB8jp3FwC+PXlkTaHlfhlYuF8Jp0cj9I a6zdbSEuYn8rN4buCSpDLbs= X-Received: by 2002:a63:c04b:: with SMTP id z11mr50606521pgi.74.1609295589813; Tue, 29 Dec 2020 18:33:09 -0800 (PST) Received: from localhost (193-116-97-30.tpgi.com.au. [193.116.97.30]) by smtp.gmail.com with ESMTPSA id 6sm39988438pfj.216.2020.12.29.18.33.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Dec 2020 18:33:08 -0800 (PST) Date: Wed, 30 Dec 2020 12:33:02 +1000 From: Nicholas Piggin Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() To: Russell King - ARM Linux admin Cc: Arnd Bergmann , Benjamin Herrenschmidt , Catalin Marinas , Jann Horn , linux-arm-kernel , linux-kernel , linuxppc-dev , Andy Lutomirski , Mathieu Desnoyers , Michael Ellerman , paulmck , Paul Mackerras , Peter Zijlstra , stable , Will Deacon , x86 References: <20201228102537.GG1551@shell.armlinux.org.uk> <20201228190852.GI1551@shell.armlinux.org.uk> <1086654515.3607.1609187556216.JavaMail.zimbra@efficios.com> <1609200902.me5niwm2t6.astroid@bobo.none> <1609210162.4d8dqilke6.astroid@bobo.none> <20201229104456.GK1551@shell.armlinux.org.uk> In-Reply-To: <20201229104456.GK1551@shell.armlinux.org.uk> MIME-Version: 1.0 Message-Id: <1609290821.wrfh89v23a.astroid@bobo.none> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Excerpts from Russell King - ARM Linux admin's message of December 29, 2020= 8:44 pm: > On Tue, Dec 29, 2020 at 01:09:12PM +1000, Nicholas Piggin wrote: >> I think it should certainly be documented in terms of what guarantees >> it provides to application, _not_ the kinds of instructions it may or >> may not induce the core to execute. And if existing API can't be >> re-documented sanely, then deprecatd and new ones added that DTRT. >> Possibly under a new system call, if arch's like ARM want a range >> flush and we don't want to expand the multiplexing behaviour of >> membarrier even more (sigh). >=20 > The 32-bit ARM sys_cacheflush() is there only to support self-modifying > code, and takes whatever actions are necessary to support that. > Exactly what actions it takes are cache implementation specific, and > should be of no concern to the caller, but the underlying thing is... > it's to support self-modifying code. Caveat cacheflush() should not be used in programs intended to be portab= le. On Linux, this call first appeared on the MIPS architecture, but no= wa=E2=80=90 days, Linux provides a cacheflush() system call on some other archit= ec=E2=80=90 tures, but with different arguments. What a disaster. Another badly designed interface, although it didn't=20 originate in Linux it sounds like we weren't to be outdone so we messed it up even worse. flushing caches is neither necessary nor sufficient for code modification on many processors. Maybe some old MIPS specific private thing was fine, but certainly before it grew to other architectures, somebody should=20 have thought for more than 2 minutes about it. Sigh. >=20 > Sadly, because it's existed for 20+ years, and it has historically been > sufficient for other purposes too, it has seen quite a bit of abuse > despite its design purpose not changing - it's been used by graphics > drivers for example. They quickly learnt the error of their ways with > ARMv6+, since it does not do sufficient for their purposes given the > cache architectures found there. >=20 > Let's not go around redesigning this after twenty odd years, requiring > a hell of a lot of pain to users. This interface is called by code > generated by GCC, so to change it you're looking at patching GCC as > well as the kernel, and you basically will make new programs > incompatible with older kernels - very bad news for users. For something to be redesigned it had to have been designed in the first=20 place, so there is no danger of that don't worry... But no I never=20 suggested making incompatible changes to any existing system call, I=20 said "re-documented". And yes I said deprecated but in Linux that really=20 means kept indefinitely. If ARM, MIPS, 68k etc programs and toolchains keep using what they are=20 using it'll keep working no problem. The point is we're growing new interfaces, and making the same mistakes.=20 It's not portable (ARCH_HAS_MEMBARRIER_SYNC_CORE), it's also specified=20 in terms of low level processor operations rather than higher level=20 intent, and also is not sufficient for self-modifying code (without=20 additional cache flush on some processors). The application wants a call that says something like "memory modified=20 before the call will be visible as instructions (including illegal=20 instructions) by all threads in the program after the system call=20 returns, and no threads will be subject to any effects of executing the=20 previous contents of that memory. So I think the basics are simple (although should confirm with some JIT=20 and debugger etc developers, and not just Android mind you). There are=20 some complications in details, address ranges, virtual/physical, thread=20 local vs process vs different process or system-wide, memory ordering=20 and propagation of i and d sides, etc. But that can be worked through,=20 erring on the side of sanity rather than pointless micro-optmisations. Thanks, Nick