Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752463AbdFUJBq (ORCPT ); Wed, 21 Jun 2017 05:01:46 -0400 Received: from Galois.linutronix.de ([146.0.238.70]:48344 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752150AbdFUJBo (ORCPT ); Wed, 21 Jun 2017 05:01:44 -0400 Date: Wed, 21 Jun 2017 11:01:35 +0200 (CEST) From: Thomas Gleixner To: Andy Lutomirski cc: x86@kernel.org, linux-kernel@vger.kernel.org, Borislav Petkov , Linus Torvalds , Andrew Morton , Mel Gorman , "linux-mm@kvack.org" , Nadav Amit , Rik van Riel , Dave Hansen , Arjan van de Ven , Peter Zijlstra , Andrew Banman , Mike Travis , Dimitri Sivanich , Juergen Gross , Boris Ostrovsky Subject: Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking In-Reply-To: <70f3a61658aa7c1c89f4db6a4f81d8df9e396ade.1498022414.git.luto@kernel.org> Message-ID: References: <70f3a61658aa7c1c89f4db6a4f81d8df9e396ade.1498022414.git.luto@kernel.org> User-Agent: Alpine 2.20 (DEB 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2558 Lines: 57 On Tue, 20 Jun 2017, Andy Lutomirski wrote: > -/* > - * The flush IPI assumes that a thread switch happens in this order: > - * [cpu0: the cpu that switches] > - * 1) switch_mm() either 1a) or 1b) > - * 1a) thread switch to a different mm > - * 1a1) set cpu_tlbstate to TLBSTATE_OK > - * Now the tlb flush NMI handler flush_tlb_func won't call leave_mm > - * if cpu0 was in lazy tlb mode. > - * 1a2) update cpu active_mm > - * Now cpu0 accepts tlb flushes for the new mm. > - * 1a3) cpu_set(cpu, new_mm->cpu_vm_mask); > - * Now the other cpus will send tlb flush ipis. > - * 1a4) change cr3. > - * 1a5) cpu_clear(cpu, old_mm->cpu_vm_mask); > - * Stop ipi delivery for the old mm. This is not synchronized with > - * the other cpus, but flush_tlb_func ignore flush ipis for the wrong > - * mm, and in the worst case we perform a superfluous tlb flush. > - * 1b) thread switch without mm change > - * cpu active_mm is correct, cpu0 already handles flush ipis. > - * 1b1) set cpu_tlbstate to TLBSTATE_OK > - * 1b2) test_and_set the cpu bit in cpu_vm_mask. > - * Atomically set the bit [other cpus will start sending flush ipis], > - * and test the bit. > - * 1b3) if the bit was 0: leave_mm was called, flush the tlb. > - * 2) switch %%esp, ie current > - * > - * The interrupt must handle 2 special cases: > - * - cr3 is changed before %%esp, ie. it cannot use current->{active_,}mm. > - * - the cpu performs speculative tlb reads, i.e. even if the cpu only > - * runs in kernel space, the cpu could load tlb entries for user space > - * pages. > - * > - * The good news is that cpu_tlbstate is local to each cpu, no > - * write/read ordering problems. While the new code is really well commented, it would be a good thing to have a single place where all of this including the ordering constraints are documented. > @@ -215,12 +200,13 @@ static void flush_tlb_func_common(const struct flush_tlb_info *f, > VM_WARN_ON(this_cpu_read(cpu_tlbstate.ctxs[0].ctx_id) != > loaded_mm->context.ctx_id); > > - if (this_cpu_read(cpu_tlbstate.state) != TLBSTATE_OK) { > + if (!cpumask_test_cpu(smp_processor_id(), mm_cpumask(loaded_mm))) { > /* > - * leave_mm() is adequate to handle any type of flush, and > - * we would prefer not to receive further IPIs. > + * We're in lazy mode -- don't flush. We can get here on > + * remote flushes due to races and on local flushes if a > + * kernel thread coincidentally flushes the mm it's lazily > + * still using. Ok. That's more informative. Reviewed-by: Thomas Gleixner