Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
From:   Andy Lutomirski <luto@amacapital.net>
Mime-Version: 1.0 (1.0)
Subject: Re: [PATCH 2/8] x86: use exit_lazy_tlb rather than membarrier_mm_sync_core_before_usermode
Date:   Sat, 5 Dec 2020 08:11:54 -0800
Message-Id: <116A6B40-C77B-4B6A-897B-18342CD62CEC@amacapital.net>
References: <1607152918.fkgmomgfw9.astroid@bobo.none>
Cc:     Andy Lutomirski <luto@kernel.org>,
        Anton Blanchard <anton@ozlabs.org>,
        Arnd Bergmann <arnd@arndb.de>,
        linux-arch <linux-arch@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Linux-MM <linux-mm@kvack.org>,
        linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Peter Zijlstra <peterz@infradead.org>, X86 ML <x86@kernel.org>
In-Reply-To: <1607152918.fkgmomgfw9.astroid@bobo.none>
To:     Nicholas Piggin <npiggin@gmail.com>
Precedence: bulk


> On Dec 5, 2020, at 12:00 AM, Nicholas Piggin <npiggin@gmail.com> wrote:
>=20
>=20
> I disagree. Until now nobody following it noticed that the mm gets
> un-lazied in other cases, because that was not too clear from the
> code (only indirectly using non-standard terminology in the arch
> support document).

> In other words, membarrier needs a special sync to deal with the case=20
> when a kthread takes the mm.

I don=E2=80=99t think this is actually true. Somehow the x86 oddities about C=
R3 writes leaked too much into the membarrier core code and comments. (I dou=
bt this is x86 specific.  The actual x86 specific part seems to be that we c=
an return to user mode without syncing the instruction stream.)

As far as I can tell, membarrier doesn=E2=80=99t care at all about laziness.=
 Membarrier cares about rq->curr->mm.  The fact that a cpu can switch its ac=
tual loaded mm without scheduling at all (on x86 at least) is entirely besid=
e the point except insofar as it has an effect on whether a subsequent switc=
h_mm() call serializes.  If we notify membarrier about x86=E2=80=99s asynchr=
onous CR3 writes, then membarrier needs to understand what to do with them, w=
hich results in an unmaintainable mess in membarrier *and* in the x86 code.

I=E2=80=99m currently trying to document how membarrier actually works, and h=
opefully this will result in untangling membarrier from mmdrop() and such.

A silly part of this is that x86 already has a high quality implementation o=
f most of membarrier(): flush_tlb_mm().  If you flush an mm=E2=80=99s TLB, w=
e carefully propagate the flush to all threads, with attention to memory ord=
ering.  We can=E2=80=99t use this directly as an arch-specific implementatio=
n of membarrier because it has the annoying side affect of flushing the TLB a=
nd because upcoming hardware might be able to flush without guaranteeing a c=
ore sync.  (Upcoming means Zen 3, but the Zen 3 implementation is sadly not u=
sable by Linux.)