Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752151AbdIKBrK (ORCPT ); Sun, 10 Sep 2017 21:47:10 -0400 Received: from mail.kernel.org ([198.145.29.99]:43384 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751942AbdIKBrJ (ORCPT ); Sun, 10 Sep 2017 21:47:09 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7D08921EAC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=luto@kernel.org X-Google-Smtp-Source: AOwi7QCYhpAnprgDY19PAGM6QB17leLtFvRVWRJ+f/57HQjFCZGNrB1J+4DSMQ91IiluMOGGB0Zm+M8GDOfyTAsyKq0= MIME-Version: 1.0 In-Reply-To: <1505092341.21121.34.camel@redhat.com> References: <20170909143335.ja2iwjsbeyfxz4ez@pd.tnic> <20170909144350.GA290@x4> <20170909163225.GA290@x4> <20170909170537.6xmxtzwripplhhwi@pd.tnic> <20170909172352.GA290@x4> <20170909173633.4ttfk7maooxkcwum@pd.tnic> <20170909181445.GA281@x4> <20170909182952.itqad4ryngjwrgqf@pd.tnic> <20170909190948.xydyega7i2rjnlqt@pd.tnic> <1505092341.21121.34.camel@redhat.com> From: Andy Lutomirski Date: Sun, 10 Sep 2017 18:46:47 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Current mainline git (24e700e291d52bd2) hangs when building e.g. perf To: Rik van Riel Cc: Andy Lutomirski , Borislav Petkov , Linus Torvalds , Markus Trippelsdorf , Ingo Molnar , Thomas Gleixner , Peter Zijlstra , LKML , Ingo Molnar , Tom Lendacky Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2186 Lines: 46 On Sun, Sep 10, 2017 at 6:12 PM, Rik van Riel wrote: > On Sat, 2017-09-09 at 12:28 -0700, Andy Lutomirski wrote: >> - >> I propose the following fix. If PCID is on, then, in >> enter_lazy_tlb(), we switch to init_mm with the no-flush flag set. >> (And we give init_mm its own dedicated ASID to keep it simple and >> fast >> -- no need to use the LRU ASID mapping to assign one >> dynamically.) We >> clear the bit in mm_cpumask. That is, we more or less just skip the >> whole lazy TLB optimization and rely on PCID CPUs having reasonably >> fast CR3 writes. No extra IPIs. > > Avoiding the IPIs is probably what matters the most, especially > on systems with deep C states, and virtual machines where the > host may be running something else, causing the IPI service time > to go through the roof for idle VCPUs. > >> Also, sorry Rik, this means your old increased laziness optimization >> is dead in the water. It will have exactly the same speculative load >> problem. > > Doesn't a memory barrier solve that speculative load > problem? > > The memory barrier could be added only to the path > that potentially skips reloading the TLB, under the > assumption that a memory barrier is cheaper than a > TLB reload (even with ASID). No, nothing stops the problematic speculative load. Here's the issue. One CPU removes a reference to a page table from a higher-level page table, flushes, and then frees the page table. Then it re-allocates it and writes something unrelated there. Another CPU that has CR3 pointing to the page hierarchy in question could have a reference to the freed table in its paging structure cache. Even if it's guaranteed to not try to access the addresses in question (because they're user addresses and the other CPU is in kernel mode, etc), but there is never a guarantee that the CPU doesn't randomly try to fill its TLB for the affected addresses. This results in invalid PTEs in the TLB, possible accesses using bogus memory types, and maybe even reads from IO space. It looks like we actually need to propagate flushes everywhere that could have references to the flushed range, even if the software won't access that range.