Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 2 Aug 2022 11:57:32 -0400 (EDT)
From:   Mikulas Patocka <mpatocka@redhat.com>
To:     Will Deacon <will@kernel.org>
cc:     Linus Torvalds <torvalds@linux-foundation.org>,
        "Paul E. McKenney" <paulmck@kernel.org>,
        Ard Biesheuvel <ardb@kernel.org>,
        Alexander Viro <viro@zeniv.linux.org.uk>,
        Alan Stern <stern@rowland.harvard.edu>,
        Andrea Parri <parri.andrea@gmail.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Boqun Feng <boqun.feng@gmail.com>,
        Nicholas Piggin <npiggin@gmail.com>,
        David Howells <dhowells@redhat.com>,
        Jade Alglave <j.alglave@ucl.ac.uk>,
        Luc Maranget <luc.maranget@inria.fr>,
        Akira Yokosawa <akiyks@gmail.com>,
        Daniel Lustig <dlustig@nvidia.com>,
        Joel Fernandes <joel@joelfernandes.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        linux-arch <linux-arch@vger.kernel.org>,
        linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH v4 1/2] introduce test_bit_acquire and use it in
 wait_on_bit
In-Reply-To: <20220802133652.GA27253@willie-the-truck>
Message-ID: <alpine.LRH.2.02.2208021137430.12343@file01.intranet.prod.int.rdu2.redhat.com>
References: <alpine.LRH.2.02.2207311104020.16444@file01.intranet.prod.int.rdu2.redhat.com> <CAHk-=wiC_oidYZeMD7p0E-=TAuLgrNQ86-sB99=hRqFM8fVLDQ@mail.gmail.com> <alpine.LRH.2.02.2207311542280.21273@file01.intranet.prod.int.rdu2.redhat.com>
 <alpine.LRH.2.02.2207311639360.21350@file01.intranet.prod.int.rdu2.redhat.com> <CAHk-=wjA8HBrVqAqAetUvwNr=hcvhfnO7oMrOAd4V8bbSqokNA@mail.gmail.com> <alpine.LRH.2.02.2208010640260.22006@file01.intranet.prod.int.rdu2.redhat.com> <20220801155421.GB26280@willie-the-truck>
 <alpine.LRH.2.02.2208011206430.31960@file01.intranet.prod.int.rdu2.redhat.com> <20220802084015.GB26962@willie-the-truck> <alpine.LRH.2.02.2208020726220.6971@file01.intranet.prod.int.rdu2.redhat.com> <20220802133652.GA27253@willie-the-truck>
User-Agent: Alpine 2.02 (LRH 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk


On Tue, 2 Aug 2022, Will Deacon wrote:

> On Tue, Aug 02, 2022 at 07:38:17AM -0400, Mikulas Patocka wrote:
> > 
> > 
> > On Tue, 2 Aug 2022, Will Deacon wrote:
> > 
> > > On Mon, Aug 01, 2022 at 12:12:47PM -0400, Mikulas Patocka wrote:
> > > > On Mon, 1 Aug 2022, Will Deacon wrote:
> > > > > On Mon, Aug 01, 2022 at 06:42:15AM -0400, Mikulas Patocka wrote:
> > > > > 
> > > > > > Index: linux-2.6/arch/x86/include/asm/bitops.h
> > > > > > ===================================================================
> > > > > > --- linux-2.6.orig/arch/x86/include/asm/bitops.h	2022-08-01 12:27:43.000000000 +0200
> > > > > > +++ linux-2.6/arch/x86/include/asm/bitops.h	2022-08-01 12:27:43.000000000 +0200
> > > > > > @@ -203,8 +203,10 @@ arch_test_and_change_bit(long nr, volati
> > > > > >  
> > > > > >  static __always_inline bool constant_test_bit(long nr, const volatile unsigned long *addr)
> > > > > >  {
> > > > > > -	return ((1UL << (nr & (BITS_PER_LONG-1))) &
> > > > > > +	bool r = ((1UL << (nr & (BITS_PER_LONG-1))) &
> > > > > >  		(addr[nr >> _BITOPS_LONG_SHIFT])) != 0;
> > > > > > +	barrier();
> > > > > > +	return r;
> > > > > 
> > > > > Hmm, I find it a bit weird to have a barrier() here given that 'addr' is
> > > > > volatile and we don't need a barrier() like this in the definition of
> > > > > READ_ONCE(), for example.
> > > > 
> > > > gcc doesn't reorder two volatile accesses, but it can reorder non-volatile
> > > > accesses around volatile accesses.
> > > > 
> > > > The purpose of the compiler barrier is to make sure that the non-volatile 
> > > > accesses that follow test_bit are not reordered by the compiler before the 
> > > > volatile access to addr.
> > > 
> > > If we need these accesses to be ordered reliably, then we need a CPU barrier
> > > and that will additionally prevent the compiler reordering. So I still don't
> > > think we need the barrier() here.
> > 
> > This is x86-specific code. x86 has strong memory ordering, so we only care 
> > about compiler reordering.
> 
> Indeed, but what I'm trying to say is that the _caller_ would have a memory
> barrier in this case, and so there's no need for one in here. test_bit() does
> not have ordering semantics.
> 
> Will

But constant_test_bit() is also used for test_bit_acquire(), and for 
test_bit_acquire(), the barrier is needed. Without the barrier, it doesn't 
have acquire semantics, because the compiler (not CPU) can move the 
following non-volatile accesses before the volatile access to "addr[nr >> 
_BITOPS_LONG_SHIFT]".

See this piece of code in arch/x86/include/asm/bitops.h in the patch:
+#define arch_test_bit_acquire(nr, addr)                \
+	arch_test_bit(nr, addr)

We could split constant_test_bit() to two functions: constant_test_bit() 
and constant_test_bit_acquire() and put the barrier only in 
constant_test_bit_acquire(). But I chose not to do it because code 
duplication is bad and because the overhead of the compiler barrier is 
none.

Mikulas