Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 29 Nov 2017 14:44:37 -0500 (EST)
From:   Alan Stern <stern@rowland.harvard.edu>
To:     Daniel Lustig <dlustig@nvidia.com>
cc:     "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
        Andrea Parri <parri.andrea@gmail.com>,
        Luc Maranget <luc.maranget@inria.fr>,
        Jade Alglave <j.alglave@ucl.ac.uk>,
        Boqun Feng <boqun.feng@gmail.com>,
        Nicholas Piggin <npiggin@gmail.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Will Deacon <will.deacon@arm.com>,
        David Howells <dhowells@redhat.com>,
        Palmer Dabbelt <palmer@dabbelt.com>,
        Kernel development list <linux-kernel@vger.kernel.org>
Subject: Re: Unlock-lock questions and the Linux Kernel Memory Model
In-Reply-To: <17506ed0-1ce8-791d-7cf1-c40426015a99@nvidia.com>
Message-ID: <Pine.LNX.4.44L0.1711291409440.1369-100000@iolanthe.rowland.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, 29 Nov 2017, Daniel Lustig wrote:

> While we're here, let me ask about another test which isn't directly
> about unlock/lock but which is still somewhat related to this
> discussion:
> 
> "MP+wmb+xchg-acq" (or some such)
> 
> {}
> 
> P0(int *x, int *y)
> {
>         WRITE_ONCE(*x, 1);
>         smp_wmb();
>         WRITE_ONCE(*y, 1);
> }
> 
> P1(int *x, int *y)
> {
>         r1 = atomic_xchg_relaxed(y, 2);
>         r2 = smp_load_acquire(y);
>         r3 = READ_ONCE(*x);
> }
> 
> exists (1:r1=1 /\ 1:r2=2 /\ 1:r3=0)
> 
> C/C++ would call the atomic_xchg_relaxed part of a release sequence
> and hence would forbid this outcome.
> 
> x86 and Power would forbid this.  ARM forbids this via a special-case
> rule in the memory model, ordering atomics with later load-acquires.
> 
> RISC-V, however, wouldn't forbid this by default using RCpc or RCsc
> atomics for smp_load_acquire().  It's an "fri; rfi" type of pattern,
> because xchg doesn't have an inherent internal data dependency.
> 
> If the Linux memory model is going to forbid this outcome, then
> RISC-V would either need to use fences instead, or maybe we'd need to
> add a special rule to our memory model similarly.  This is one detail
> where RISC-V is still actively deciding what to do.
> 
> Have you all thought about this test before?  Any idea which way you
> are leaning regarding the outcome above?

Good questions.  Currently the LKMM allows this, and I think it should
because xchg doesn't have a dependency from its read to its write.

On the other hand, herd isn't careful enough in the way it implements 
internal dependencies for RMW operations.  If we change 
atomic_xchg_relaxed(y, 2) to atomic_inc(y) and remove r1 from the test:

C MP+wmb+inc-acq

{}

P0(int *x, int *y)
{
        WRITE_ONCE(*x, 1);
        smp_wmb();
        WRITE_ONCE(*y, 1);
}

P1(int *x, int *y)
{
        atomic_inc(y);
        r2 = smp_load_acquire(y);
        r3 = READ_ONCE(*x);
}

exists (1:r2=2 /\ 1:r3=0)

then the test _should_ be forbidden, but it isn't -- herd doesn't
realize that all atomic RMW operations other than xchg must have a
dependency (either data or control) between their internal read and
write.

(Although the smp_load_acquire is allowed to execute before the write 
part of the atomic_inc, it cannot execute before the read part.  I 
think a similar argument applies even on ARM.)

Luc, consider this a bug report.  :-)

Alan


From 1585430191349248614@xxx Wed Nov 29 19:34:13 +0000 2017
X-GM-THRID: 1585255508732072548
X-Gmail-Labels: Inbox,Category Forums,HistoricalUnread