Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <BCDEA397-AA7A-4FDE-8046-C68625CDE166@joelfernandes.org>
 <20230728124412.GA21303@willie-the-truck> <CAEXW_YRtUd4jUP68jzMgDgWxAy8tdJQortK07TZgCxVLNAgaNA@mail.gmail.com>
 <9fd99405-a3ff-4ab7-b6b7-e74849f1d334@rowland.harvard.edu>
In-Reply-To: <9fd99405-a3ff-4ab7-b6b7-e74849f1d334@rowland.harvard.edu>
From:   Joel Fernandes <joel@joelfernandes.org>
Date:   Fri, 28 Jul 2023 14:03:09 -0400
Message-ID: <CAEXW_YS-axyXvX4-aMc9a2EWY59KAyHvirMewVuoNGOGSh35Vw@mail.gmail.com>
Subject: Re: [PATCH 0/2] fix vma->anon_vma check for per-VMA locking; fix
 anon_vma memory ordering
To:     Alan Stern <stern@rowland.harvard.edu>
Cc:     Will Deacon <will@kernel.org>, Jann Horn <jannh@google.com>,
        paulmck@kernel.org, Andrew Morton <akpm@linux-foundation.org>,
        Linus Torvalds <torvalds@linuxfoundation.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Suren Baghdasaryan <surenb@google.com>,
        Matthew Wilcox <willy@infradead.org>,
        linux-kernel@vger.kernel.org, linux-mm@kvack.org,
        Andrea Parri <parri.andrea@gmail.com>,
        Boqun Feng <boqun.feng@gmail.com>,
        Nicholas Piggin <npiggin@gmail.com>,
        David Howells <dhowells@redhat.com>,
        Jade Alglave <j.alglave@ucl.ac.uk>,
        Luc Maranget <luc.maranget@inria.fr>,
        Akira Yokosawa <akiyks@gmail.com>,
        Daniel Lustig <dlustig@nvidia.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Fri, Jul 28, 2023 at 1:51=E2=80=AFPM Alan Stern <stern@rowland.harvard.e=
du> wrote:
>
> On Fri, Jul 28, 2023 at 01:35:43PM -0400, Joel Fernandes wrote:
> > On Fri, Jul 28, 2023 at 8:44=E2=80=AFAM Will Deacon <will@kernel.org> w=
rote:
> > >
> > > On Thu, Jul 27, 2023 at 12:34:44PM -0400, Joel Fernandes wrote:
> > > > > On Jul 27, 2023, at 10:57 AM, Will Deacon <will@kernel.org> wrote=
:
> > > > > =EF=BB=BFOn Thu, Jul 27, 2023 at 04:39:34PM +0200, Jann Horn wrot=
e:
> > > > >> if (READ_ONCE(vma->anon_vma) !=3D NULL) {
> > > > >>  // we now know that vma->anon_vma cannot change anymore
> > > > >>
> > > > >>  // access the same memory location again with a plain load
> > > > >>  struct anon_vma *a =3D vma->anon_vma;
> > > > >>
> > > > >>  // this needs to be address-dependency-ordered against one of
> > > > >>  // the loads from vma->anon_vma
> > > > >>  struct anon_vma *root =3D a->root;
> > > > >> }
> > > > >>
> > > > >>
> > > > >> Is this fine? If it is not fine just because the compiler might
> > > > >> reorder the plain load of vma->anon_vma before the READ_ONCE() l=
oad,
> > > > >> would it be fine after adding a barrier() directly after the
> > > > >> READ_ONCE()?
> > > > >
> > > > > I'm _very_ wary of mixing READ_ONCE() and plain loads to the same=
 variable,
> > > > > as I've run into cases where you have sequences such as:
> > > > >
> > > > >    // Assume *ptr is initially 0 and somebody else writes it to 1
> > > > >    // concurrently
> > > > >
> > > > >    foo =3D *ptr;
> > > > >    bar =3D READ_ONCE(*ptr);
> > > > >    baz =3D *ptr;
> > > > >
> > > > > and you can get foo =3D=3D baz =3D=3D 0 but bar =3D=3D 1 because =
the compiler only
> > > > > ends up reading from memory twice.
> > > > >
> > > > > That was the root cause behind f069faba6887 ("arm64: mm: Use READ=
_ONCE
> > > > > when dereferencing pointer to pte table"), which was very unpleas=
ant to
> > > > > debug.
> > > >
> > > > Will, Unless I am missing something fundamental, this case is diffe=
rent though.
> > > > This case does not care about fewer reads. As long as the first rea=
d is volatile, the subsequent loads (even plain)
> > > > should work fine, no?
> > > > I am not seeing how the compiler can screw that up, so please do en=
lighten :).
> > >
> > > I guess the thing I'm worried about is if there is some previous read=
 of
> > > 'vma->anon_vma' which didn't use READ_ONCE() and the compiler kept th=
e
> > > result around in a register. In that case, 'a' could be NULL, even if
> > > the READ_ONCE(vma->anon_vma) returned non-NULL.
> >
> > If I can be a bit brave enough to say -- that appears to be a compiler
> > bug to me. It seems that the compiler in such an instance violates the
> > "Sequential Consistency Per Variable" rule? I mean if it can't even
> > keep SCPV true for a same memory-location load (plain or not) for a
> > sequence of code, how can it expect the hardware to.
>
> It's not a compiler bug.  In this example, some other thread performs a
> write that changes vma->anon_vma from NULL to non-NULL.  This write
> races with the plain reads, and compilers are not required to obey the
> "Sequential Consistency Per Variable" rule (or indeed, any rule) when
> there is a data race.

So you're saying the following code behavior is OK?

/* Say anon_vma can only ever transition from NULL to non-NULL values */
a =3D vma->anon_vma;  // Reads NULL
b =3D READ_ONCE(vma->anon_vma); // Reads non-NULL
c =3D vma->anon_vma;  // Reads NULL!!!
if (b) {
  c->some_attribute++; // Oopsie
}

thanks,

 - Joel
(On the road now, so my replies will be slightly delayed for few hours)