Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750928AbdGYFt1 (ORCPT ); Tue, 25 Jul 2017 01:49:27 -0400 Received: from mail.kernel.org ([198.145.29.99]:60196 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750778AbdGYFt0 (ORCPT ); Tue, 25 Jul 2017 01:49:26 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9562322BDF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=luto@kernel.org MIME-Version: 1.0 In-Reply-To: <231630A0-21DB-4347-B126-F49AFD32B851@gmail.com> References: <695299daa67239284e8db5a60d4d7eb88c914e0a.1500957502.git.luto@kernel.org> <231630A0-21DB-4347-B126-F49AFD32B851@gmail.com> From: Andy Lutomirski Date: Mon, 24 Jul 2017 22:43:28 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v5 2/2] x86/mm: Improve TLB flush documentation To: Nadav Amit Cc: Andy Lutomirski , X86 ML , LKML , Borislav Petkov , Linus Torvalds , Andrew Morton , Mel Gorman , "linux-mm@kvack.org" , Rik van Riel , Dave Hansen , Arjan van de Ven , Peter Zijlstra Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2781 Lines: 64 On Mon, Jul 24, 2017 at 9:47 PM, Nadav Amit wrote: > Andy Lutomirski wrote: > >> Improve comments as requested by PeterZ and also add some >> documentation at the top of the file. >> >> Signed-off-by: Andy Lutomirski >> --- >> arch/x86/mm/tlb.c | 43 +++++++++++++++++++++++++++++++++---------- >> 1 file changed, 33 insertions(+), 10 deletions(-) >> >> diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c >> index ce104b962a17..d4ee781ca656 100644 >> --- a/arch/x86/mm/tlb.c >> +++ b/arch/x86/mm/tlb.c >> @@ -15,17 +15,24 @@ >> #include >> >> /* >> - * TLB flushing, formerly SMP-only >> - * c/o Linus Torvalds. >> + * The code in this file handles mm switches and TLB flushes. >> * >> - * These mean you can really definitely utterly forget about >> - * writing to user space from interrupts. (Its not allowed anyway). >> + * An mm's TLB state is logically represented by a totally ordered sequence >> + * of TLB flushes. Each flush increments the mm's tlb_gen. >> * >> - * Optimizations Manfred Spraul >> + * Each CPU that might have an mm in its TLB (and that might ever use >> + * those TLB entries) will have an entry for it in its cpu_tlbstate.ctxs >> + * array. The kernel maintains the following invariant: for each CPU and >> + * for each mm in its cpu_tlbstate.ctxs array, the CPU has performed all >> + * flushes in that mms history up to the tlb_gen in cpu_tlbstate.ctxs >> + * or the CPU has performed an equivalent set of flushes. >> * >> - * More scalable flush, from Andi Kleen >> - * >> - * Implement flush IPI by CALL_FUNCTION_VECTOR, Alex Shi >> + * For this purpose, an equivalent set is a set that is at least as strong. >> + * So, for example, if the flush history is a full flush at time 1, >> + * a full flush after time 1 is sufficient, but a full flush before time 1 >> + * is not. Similarly, any number of flushes can be replaced by a single >> + * full flush so long as that replacement flush is after all the flushes >> + * that it's replacing. >> */ >> >> atomic64_t last_mm_ctx_id = ATOMIC64_INIT(1); >> @@ -138,7 +145,16 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, >> return; >> } >> >> - /* Resume remote flushes and then read tlb_gen. */ >> + /* >> + * Resume remote flushes and then read tlb_gen. The >> + * implied barrier in atomic64_read() synchronizes >> + * with inc_mm_tlb_gen() like this: > > You mean the implied memory barrier in cpumask_set_cpu(), no? > Ugh, yes. And I misread PeterZ's email and incorrectly removed the smp_mb__after_atomic(). I'll respin this patch.