Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain;
        charset=utf-8
Mime-Version: 1.0 (1.0)
Subject: Re: [PATCH 4/7] x86,tlb: make lazy TLB mode lazier
From:   Andy Lutomirski <luto@amacapital.net>
In-Reply-To: <A3FE5372-2418-4D83-A023-36ECE6E6F423@surriel.com>
Date:   Tue, 17 Jul 2018 12:27:53 -1000
Cc:     Andy Lutomirski <luto@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>, X86 ML <x86@kernel.org>,
        Mike Galbraith <efault@gmx.de>,
        kernel-team <kernel-team@fb.com>, Ingo Molnar <mingo@kernel.org>,
        Dave Hansen <dave.hansen@intel.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <67F32577-24D8-4E9F-ADB1-927B3AC18B5A@amacapital.net>
References: <20180716190337.26133-1-riel@surriel.com> <20180716190337.26133-5-riel@surriel.com> <CALCETrUMntRe_LYX5wj9YD4xZ+84QExK+ZNb3yxEBDEKa7nePQ@mail.gmail.com> <FF977B78-140F-4787-AA57-0EA934017D85@surriel.com> <CALCETrXtpGPm2Hgm3R=p1ezvNQi==jhQWw5ee9K+H2avNNGsUg@mail.gmail.com> <A3FE5372-2418-4D83-A023-36ECE6E6F423@surriel.com>
To:     Rik van Riel <riel@surriel.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


> On Jul 17, 2018, at 12:05 PM, Rik van Riel <riel@surriel.com> wrote:
>=20
>=20
>=20
>> On Jul 17, 2018, at 5:29 PM, Andy Lutomirski <luto@kernel.org> wrote:
>>=20
>> On Tue, Jul 17, 2018 at 1:16 PM, Rik van Riel <riel@surriel.com> wrote:
>>> Can I skip both the cr4 and let switches when the TLB contents
>>> are no longer valid and got reloaded?
>>>=20
>>> If the TLB contents are still valid, either because we never went
>>> into lazy TLB mode, or because no invalidates happened while
>>> we were lazy, we immediately return.
>>>=20
>>> The cr4 and ldt reloads only happen if the TLB was invalidated
>>> while we were in lazy TLB mode.
>>=20
>> Yes, since the only events that would change the LDT or the required
>> CR4 value will unconditionally broadcast to every CPU in mm_cpumask
>> regardless of whether they're lazy.  The interesting case is that you
>> go lazy, you miss an invalidation IPI because you were lazy, then you
>> go unlazy, notice the tlb_gen change, and flush.  If this happens, you
>> know that you only missed a page table update and not an LDT update or
>> a CR4 update, because the latter would have sent the IPI even though
>> you were lazy.  So you should skip the CR4 and LDT updates.
>>=20
>> I suppose a different approach would be to fix the issue below and to
>> try to track when the LDT actually needs reloading.  But that latter
>> part seems a bit complicated for minimal gain.
>>=20
>> (Do you believe me?  If not, please argue back!)
>>=20
> I believe you :)
>=20
>>>> Hmm.  load_mm_cr4() should bypass itself when mm =3D=3D &init_mm.  Want=
 to
>>>> fix that part or should I?
>>>=20
>>> I would be happy to send in a patch for this, and one for
>>> the above optimization you pointed out.
>>>=20
>>=20
>> Yes please!
>>=20
> There is a third optimization left to do. Currently every time
> we switch into lazy tlb mode, we take a refcount on the mm,
> even when switching from one kernel thread to another, or
> when repeatedly switching between the same mm and kernel
> threads.
>=20
> We could keep that refcount (on a per cpu basis) from the time
> we first switch to that mm in lazy tlb mode, to when we switch
> the CPU to a different mm.
>=20
> That would allow us to not bounce the cache line with the
> mm_struct reference count on every lazy TLB context switch.
>=20
> Does that seem like a reasonable optimization?

Are you referring to the core sched code that deals with mm_count and active=
_mm?  If so, last time I looked at it, I convinced myself that it was totall=
y useless, at least on x86. I think the my reasoning was that, when mm_users=
 went to zero, we already waited for RCU before tearing down page tables.

Things may have changed, but I strongly suspect that it should be possibly f=
or at least x86 to opt out of mm_count and maybe even active_mm entirely.  I=
f nothing else, you=E2=80=99re shooting the mm out of CR3 on all CPUs whenev=
er the pagetables get freed, and more or less the same logic should be suffi=
cient so that, whenever mm_users hits zero, we can synchronously or via RCU c=
allback kill the mm entirely.

Want to take a look at that?

>=20
> Am I overlooking anything?
>=20
> I'll try to get all three optimizations working, and will run them
> through some testing here before posting upstream.
>=20