Received: by 2002:a05:6a10:f347:0:0:0:0 with SMTP id d7csp9458843pxu; Mon, 28 Dec 2020 17:17:44 -0800 (PST) X-Google-Smtp-Source: ABdhPJxMrzKJ50r45FwH0JDmIPQurUjvL7hI803OCUJbyXbbPKEDeyp7oDfe46fC2mYbPGTO2xuP X-Received: by 2002:a17:906:1741:: with SMTP id d1mr34112997eje.182.1609204664700; Mon, 28 Dec 2020 17:17:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1609204664; cv=none; d=google.com; s=arc-20160816; b=oP+89r3kOuyiNzGZe+7Swan3TGq7OjsdVYBBXFV27fVMwT939B180OH1fkNqQiJ6r1 H4VT5CBffxt5r6GtVORj3GoAtJYr1KzTsvHoGAmyquzHyIrmKMZS2mv+Ouu8mJ/GWfAF 2OobIvj+CnnB5gqzSZRnO1uvvZa9OR9vjEaFfh/vt1j+H1gWnyOcA+Dm+hYzUjW3FgBO lj4Jo9yNtq9R3wT3tKA1fEkGAJolZdXe4vEhdyK0UYqEwst2u1cpwkt4jpQ5pCJAJUlk dn0wuXDghxYNsQcHdZbqMN5KdZGlceAOkQKTveziJPIpcd5J4LPdpmpfwzm/l9L+BpHy ZY2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:dkim-signature; bh=WcE43Fgn3spFCZPvKJxx0LNE1rDojIXPuwxkq9VZjMs=; b=F0M1y7fz4WN6Amy6d+M+sndSyMuNUZr4Z5VTJhype+hA1TcYwOH/XuJaWIbkGqc4eC AxGK51RBt1aH3Ed74BePJtIwZDFeRx0NBc9hsQqx0zvDoGa5rM3njJpqasOFIcAlDjje NhaV+t48lTlWz/AQsqH9BJk6HvRrS2Ovw9Hhq84jWD4DJFgO0ykksWz7PDCyrioP+H4j dP2Poj5f81kYgK34fs94nLqGavubNhf3EDC+RsoOGNi1ezL5G3f4MQ/H7Gf9o6pnTbbr 10JZywwMud4GHiHP+kuQVB9/wUREWtAZiig8nstxGnEymkbaxoVdTeZQJbW1/mjKQKDe nrDg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail (test mode) header.i=@armlinux.org.uk header.s=pandora-2019 header.b=wIixOSh8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=armlinux.org.uk Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id t4si21516819edq.607.2020.12.28.17.17.22; Mon, 28 Dec 2020 17:17:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=fail (test mode) header.i=@armlinux.org.uk header.s=pandora-2019 header.b=wIixOSh8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=armlinux.org.uk Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728286AbgL1RXw (ORCPT + 99 others); Mon, 28 Dec 2020 12:23:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728228AbgL1RXw (ORCPT ); Mon, 28 Dec 2020 12:23:52 -0500 Received: from pandora.armlinux.org.uk (pandora.armlinux.org.uk [IPv6:2001:4d48:ad52:32c8:5054:ff:fe00:142]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 092C9C0613D6; Mon, 28 Dec 2020 09:23:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2019; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=WcE43Fgn3spFCZPvKJxx0LNE1rDojIXPuwxkq9VZjMs=; b=wIixOSh8cQ511bYlN+QRGms/S nlAdDjkMveRiuMhD8mhh0CeW0NehL/0Dn8mod9w5LwgJBucUnFdHoghh7ceGJoQy7c4Z20Mroq+fS IGvKjERhnM1xpUPmR2Cxb7eLcKb48q05lZjPSGJ+x0/rDDVZ08MH8CCYXgRelrxMkV5zLd0v7/fQI iJcLMDF0e6ykoNh+LFQdipAHqMu/H4YidcAg+cJ/nIcjd6vn+z5IAFK4kAFzGZTAsObU5HobbUiqK WubJHS8RCrt33xk81kIhCbo4WlQyAxWnn7VbU1Too9U9QmcVOb4Ro6PZng6ArUiliRak7h23kn2ra YEDMUb5Mw==; Received: from shell.armlinux.org.uk ([fd8f:7570:feb6:1:5054:ff:fe00:4ec]:44606) by pandora.armlinux.org.uk with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ktwEa-0004aV-K5; Mon, 28 Dec 2020 17:23:04 +0000 Received: from linux by shell.armlinux.org.uk with local (Exim 4.92) (envelope-from ) id 1ktwEX-0000RT-6Y; Mon, 28 Dec 2020 17:23:01 +0000 Date: Mon, 28 Dec 2020 17:23:01 +0000 From: Russell King - ARM Linux admin To: Andy Lutomirski Cc: Mathieu Desnoyers , x86 , linux-kernel , Nicholas Piggin , Arnd Bergmann , Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , linuxppc-dev , Catalin Marinas , Will Deacon , linux-arm-kernel , stable Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() Message-ID: <20201228172301.GH1551@shell.armlinux.org.uk> References: <1836294649.3345.1609100294833.JavaMail.zimbra@efficios.com> <20201228102537.GG1551@shell.armlinux.org.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: Russell King - ARM Linux admin Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Dec 28, 2020 at 09:14:23AM -0800, Andy Lutomirski wrote: > On Mon, Dec 28, 2020 at 2:25 AM Russell King - ARM Linux admin > wrote: > > > > On Sun, Dec 27, 2020 at 01:36:13PM -0800, Andy Lutomirski wrote: > > > On Sun, Dec 27, 2020 at 12:18 PM Mathieu Desnoyers > > > wrote: > > > > > > > > ----- On Dec 27, 2020, at 1:28 PM, Andy Lutomirski luto@kernel.org wrote: > > > > > > > > > > > > > > > > > I admit that I'm rather surprised that the code worked at all on arm64, > > > > > and I'm suspicious that it has never been very well tested. My apologies > > > > > for not reviewing this more carefully in the first place. > > > > > > > > Please refer to Documentation/features/sched/membarrier-sync-core/arch-support.txt > > > > > > > > It clearly states that only arm, arm64, powerpc and x86 support the membarrier > > > > sync core feature as of now: > > > > > > Sigh, I missed arm (32). Russell or ARM folks, what's the right > > > incantation to make the CPU notice instruction changes initiated by > > > other cores on 32-bit ARM? > > > > You need to call flush_icache_range(), since the changes need to be > > flushed from the data cache to the point of unification (of the Harvard > > I and D), and the instruction cache needs to be invalidated so it can > > then see those updated instructions. This will also take care of the > > necessary barriers that the CPU requires for you. > > With what parameters? From looking at the header, this is for the > case in which the kernel writes some memory and then intends to > execute it. That's not what membarrier() does at all. membarrier() > works like this: You didn't specify that you weren't looking at kernel memory. If you're talking about userspace, then the interface you require is flush_icache_user_range(), which does the same as flush_icache_range() but takes userspace addresses. Note that this requires that the memory is currently mapped at those userspace addresses. If that doesn't fit your needs, there isn't an interface to do what you require, and it basically means creating something brand new on every architecture. What you are asking for is not "just a matter of a few instructions". I have stated the required steps to achieve what you require above; that is the minimum when you have non-snooping harvard caches, which the majority of 32-bit ARMs have. > User thread 1: > > write to RWX memory *or* write to an RW alias of an X region. > membarrier(...); > somehow tell thread 2 that we're ready (with a store release, perhaps, > or even just a relaxed store.) > > User thread 2: > > wait for the indication from thread 1. > barrier(); > jump to the code. > > membarrier() is, for better or for worse, not given a range of addresses. Then, I'm sorry, it can't work on 32-bit ARM. -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!