Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752698AbdGEMSW (ORCPT ); Wed, 5 Jul 2017 08:18:22 -0400 Received: from bombadil.infradead.org ([65.50.211.133]:54462 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752546AbdGEMSV (ORCPT ); Wed, 5 Jul 2017 08:18:21 -0400 Date: Wed, 5 Jul 2017 14:18:07 +0200 From: Peter Zijlstra To: Andy Lutomirski Cc: x86@kernel.org, linux-kernel@vger.kernel.org, Borislav Petkov , Linus Torvalds , Andrew Morton , Mel Gorman , "linux-mm@kvack.org" , Nadav Amit , Rik van Riel , Dave Hansen , Arjan van de Ven Subject: Re: [PATCH v4 10/10] x86/mm: Try to preserve old TLB entries using PCID Message-ID: <20170705121807.GF4941@worktop> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.22.1 (2013-10-16) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1949 Lines: 56 On Thu, Jun 29, 2017 at 08:53:22AM -0700, Andy Lutomirski wrote: > @@ -104,18 +140,20 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, > > /* Resume remote flushes and then read tlb_gen. */ > cpumask_set_cpu(cpu, mm_cpumask(next)); Barriers should have a comment... what is being ordered here against what? > + smp_mb__after_atomic(); > next_tlb_gen = atomic64_read(&next->context.tlb_gen); > > + if (this_cpu_read(cpu_tlbstate.ctxs[prev_asid].tlb_gen) < > + next_tlb_gen) { > /* > * Ideally, we'd have a flush_tlb() variant that > * takes the known CR3 value as input. This would > * be faster on Xen PV and on hypothetical CPUs > * on which INVPCID is fast. > */ > + this_cpu_write(cpu_tlbstate.ctxs[prev_asid].tlb_gen, > next_tlb_gen); > + write_cr3(__pa(next->pgd) | prev_asid); > trace_tlb_flush(TLB_FLUSH_ON_TASK_SWITCH, > TLB_FLUSH_ALL); > } > @@ -152,14 +190,25 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, > * Start remote flushes and then read tlb_gen. > */ > cpumask_set_cpu(cpu, mm_cpumask(next)); > + smp_mb__after_atomic(); idem > next_tlb_gen = atomic64_read(&next->context.tlb_gen); > > + choose_new_asid(next, next_tlb_gen, &new_asid, &need_flush); > > + if (need_flush) { > + this_cpu_write(cpu_tlbstate.ctxs[new_asid].ctx_id, next->context.ctx_id); > + this_cpu_write(cpu_tlbstate.ctxs[new_asid].tlb_gen, next_tlb_gen); > + write_cr3(__pa(next->pgd) | new_asid); > + trace_tlb_flush(TLB_FLUSH_ON_TASK_SWITCH, > + TLB_FLUSH_ALL); > + } else { > + /* The new ASID is already up to date. */ > + write_cr3(__pa(next->pgd) | new_asid | CR3_NOFLUSH); > + trace_tlb_flush(TLB_FLUSH_ON_TASK_SWITCH, 0); > + } > + > + this_cpu_write(cpu_tlbstate.loaded_mm, next); > + this_cpu_write(cpu_tlbstate.loaded_mm_asid, new_asid); > } > > load_mm_cr4(next);