Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <alpine.LRH.2.02.2207310703170.14394@file01.intranet.prod.int.rdu2.redhat.com>
 <CAMj1kXFYRNrP2k8yppgfdKg+CxWeYfHTbzLBuyBqJ9UVAR_vaQ@mail.gmail.com>
 <alpine.LRH.2.02.2207310920390.6506@file01.intranet.prod.int.rdu2.redhat.com>
 <alpine.LRH.2.02.2207311104020.16444@file01.intranet.prod.int.rdu2.redhat.com>
 <CAHk-=wiC_oidYZeMD7p0E-=TAuLgrNQ86-sB99=hRqFM8fVLDQ@mail.gmail.com>
 <alpine.LRH.2.02.2207311542280.21273@file01.intranet.prod.int.rdu2.redhat.com>
 <alpine.LRH.2.02.2207311641060.21350@file01.intranet.prod.int.rdu2.redhat.com>
 <Yub+vPb53zt6dDpn@casper.infradead.org>
In-Reply-To: <Yub+vPb53zt6dDpn@casper.infradead.org>
From:   Ard Biesheuvel <ardb@kernel.org>
Date:   Mon, 1 Aug 2022 00:31:40 +0200
Message-ID: <CAMj1kXE-TpUGgFxBOaZbsF7k3rdHdjBoqoxZ1bvDz5AoTGADxQ@mail.gmail.com>
Subject: Re: [PATCH v3 2/2] make buffer_locked provide an acquire semantics
To:     Matthew Wilcox <willy@infradead.org>
Cc:     Mikulas Patocka <mpatocka@redhat.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Will Deacon <will@kernel.org>,
        "Paul E. McKenney" <paulmck@kernel.org>,
        Alexander Viro <viro@zeniv.linux.org.uk>,
        Alan Stern <stern@rowland.harvard.edu>,
        Andrea Parri <parri.andrea@gmail.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Boqun Feng <boqun.feng@gmail.com>,
        Nicholas Piggin <npiggin@gmail.com>,
        David Howells <dhowells@redhat.com>,
        Jade Alglave <j.alglave@ucl.ac.uk>,
        Luc Maranget <luc.maranget@inria.fr>,
        Akira Yokosawa <akiyks@gmail.com>,
        Daniel Lustig <dlustig@nvidia.com>,
        Joel Fernandes <joel@joelfernandes.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        linux-arch <linux-arch@vger.kernel.org>,
        linux-fsdevel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, 1 Aug 2022 at 00:14, Matthew Wilcox <willy@infradead.org> wrote:
>
> On Sun, Jul 31, 2022 at 04:43:08PM -0400, Mikulas Patocka wrote:
> > Let's have a look at this piece of code in __bread_slow:
> >       get_bh(bh);
> >       bh->b_end_io = end_buffer_read_sync;
> >       submit_bh(REQ_OP_READ, 0, bh);
> >       wait_on_buffer(bh);
> >       if (buffer_uptodate(bh))
> >               return bh;
> > Neither wait_on_buffer nor buffer_uptodate contain a memory barrier.
> > Consequently, if someone calls sb_bread and then reads the buffer data,
> > the read of buffer data may be executed before wait_on_buffer(bh) on
> > architectures with weak memory ordering and it may return invalid data.
>
> I think we should be consistent between PageUptodate() and
> buffer_uptodate().  Here's how it's done for pages currently:
>
> static inline bool folio_test_uptodate(struct folio *folio)
>         bool ret = test_bit(PG_uptodate, folio_flags(folio, 0));
>         /*
>          * Must ensure that the data we read out of the folio is loaded
>          * _after_ we've loaded folio->flags to check the uptodate bit.
>          * We can skip the barrier if the folio is not uptodate, because
>          * we wouldn't be reading anything from it.
>          *
>          * See folio_mark_uptodate() for the other side of the story.
>          */
>         if (ret)
>                 smp_rmb();
>
>         return ret;
>
> ...
>
> static __always_inline void folio_mark_uptodate(struct folio *folio)
>         /*
>          * Memory barrier must be issued before setting the PG_uptodate bit,
>          * so that all previous stores issued in order to bring the folio
>          * uptodate are actually visible before folio_test_uptodate becomes true.
>          */
>         smp_wmb();
>         set_bit(PG_uptodate, folio_flags(folio, 0));
>
> I'm happy for these to also be changed to use acquire/release; no
> attachment to the current code.  But bufferheads & pages should have the
> same semantics, or we'll be awfully confused.

I suspect that adding acquire/release annotations at the bitops level
is not going to get us anywhere, given that for the uptodate flag, it
is the set operation that has release semantics, whereas for a lock
flag, it will be the clear operation. Reverting to the legacy barrier
instructions to try and avoid this ambiguity will likely only make
things worse.

I was cc'ed only on patch #1 of your v3, so I'm not sure where this is
headed, but I strongly +1 Matthew's point above that this should be
done at the level that defines how the bit fields should be
interpreted wrt to the contents of the data structure that they
describe/guard.