Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 16 Jun 2021 13:10:58 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     "Russell King (Oracle)" <linux@armlinux.org.uk>
Cc:     Andy Lutomirski <luto@kernel.org>, x86@kernel.org,
        Dave Hansen <dave.hansen@intel.com>,
        LKML <linux-kernel@vger.kernel.org>, linux-mm@kvack.org,
        Andrew Morton <akpm@linux-foundation.org>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Nicholas Piggin <npiggin@gmail.com>,
        linux-arm-kernel@lists.infradead.org, Will Deacon <will@kernel.org>
Subject: Re: [PATCH 7/8] membarrier: Remove arm (32) support for SYNC_CORE
Message-ID: <YMncQv1uT5QyQ84w@hirez.programming.kicks-ass.net>
References: <cover.1623813516.git.luto@kernel.org>
 <2142129092ff9aa00e600c42a26c4015b7f5ceec.1623813516.git.luto@kernel.org>
 <YMnPezLs6vb418Wz@hirez.programming.kicks-ass.net>
 <YMnQVoKvM5G34Yan@hirez.programming.kicks-ass.net>
 <20210616103446.GC22278@shell.armlinux.org.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210616103446.GC22278@shell.armlinux.org.uk>
Precedence: bulk

On Wed, Jun 16, 2021 at 11:34:46AM +0100, Russell King (Oracle) wrote:
> On Wed, Jun 16, 2021 at 12:20:06PM +0200, Peter Zijlstra wrote:
> > On Wed, Jun 16, 2021 at 12:16:27PM +0200, Peter Zijlstra wrote:
> > > On Tue, Jun 15, 2021 at 08:21:12PM -0700, Andy Lutomirski wrote:
> > > > On arm32, the only way to safely flush icache from usermode is to call
> > > > cacheflush(2).  This also handles any required pipeline flushes, so
> > > > membarrier's SYNC_CORE feature is useless on arm.  Remove it.
> > > 
> > > So SYNC_CORE is there to help an architecture that needs to do something
> > > per CPU. If I$ invalidation is broadcast and I$ invalidation also
> > > triggers the flush of any uarch caches derived from it (if there are
> > > any).
> > 
> > Incomplete sentence there: + then we don't need SYNC_CORE.
> > 
> > > Now arm_syscall() NR(cacheflush) seems to do flush_icache_user_range(),
> > > which, if I read things right, end up in arch/arm/mm/*.S, but that
> > > doesn't consider cache_ops_need_broadcast().
> > > 
> > > Will suggests that perhaps ARM 11MPCore might need this due to their I$
> > > flush maybe not being broadcast
> 
> If it leaves other cores with incoherent I cache, then that's already
> a problem for SMP cores, since there could be no guarantee that the
> modifications made by one core will be visible to some other core that
> ends up running that code - and there is little option for userspace to
> work around that except by pinning the thread making the modifications
> and subsequently executing the code to a core.

That's where SYNC_CORE can help. Or you make sys_cacheflush() do a
system wide IPI.

> The same is also true of flush_icache_range() - which is used when
> loading a kernel module. In the case Will is referring to, these alias
> to the same code.

Yes, cache_ops_need_broadcast() seems to be missing in more places.