Received: by 2002:ac0:a5a6:0:0:0:0:0 with SMTP id m35-v6csp2336532imm; Tue, 4 Sep 2018 02:41:02 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbN6lpZkSSv06uPch5Yuf0/H0qxOYmJhEBtJACJvpaHP2DqMV7uLMogj7Bgqd7OlAhE2st0 X-Received: by 2002:a17:902:7b83:: with SMTP id w3-v6mr32950223pll.192.1536054062465; Tue, 04 Sep 2018 02:41:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536054062; cv=none; d=google.com; s=arc-20160816; b=SslYj+srm/TBuWnPbJ+tAXOcJf1mw+pAF5DIlhOjXo75cX2rL00vEwQkvcjuZ4xEIO RSQhsmc3kyLrrib8kmlPaRgXwv4GRiCVvUqWVxj8DXkAsVendt4EOMDAqBQyYiUsVuhO 0GoxDA/K//+UKeg+pae2GJzhjzLA5Gy81J2uX0honm7quHl+LepBwzFsDtex5BSwtGwf 27pkX03OXMWNNWhDpMQ0nCdmWvgyGTs+UC5vplheAy2iITZXv8u4ywqgIv/Hx74SZvbP Kv7XYSlZUoYvzBtBTsk10hEXK4aysFLPMhb4xCv1zxytPLYsxN0d+udbuo4iihkS36jm O6/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=C2Hf41+IEZMk5xpKFP+uDreWWT+A7kXsuHfwxjzf0Cg=; b=MG2JgBb96CoRtACStSXHi/oq+nLw3myOvsPZjxogWXCEA/SG1m+nvmR9Jnw2H418T9 WwafVNNbba4nJ+SiCnu09vuDcvH165CrmtfhJB2WwkGkg6HH7UC9iemcwstfeIESSPPA VQ0HBxp+TYNvDn7AbPqcSBgnG9PaeDt7Zsf4ny3p4bt4H957KdpCxaTSgQkq4HC89mjZ Ul1FQMFeZCMBJ0mCoPZDxpbq6iRSUPyv/QC1HTGZ+0j6EBMOeFyQTn/T1cHKRfWT50Fq 5eo5E1s4chKG1Sd3GfrYx590b31jFDzfzJrrm8+5DRgFfStWLLmjrTaL6kLog2ZHo3Om fP9A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@amazon.de header.s=amazon201209 header.b=lh2GfvMi; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.de Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b23-v6si19750580pls.349.2018.09.04.02.40.46; Tue, 04 Sep 2018 02:41:02 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@amazon.de header.s=amazon201209 header.b=lh2GfvMi; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727366AbeIDODO (ORCPT + 99 others); Tue, 4 Sep 2018 10:03:14 -0400 Received: from smtp-fw-6002.amazon.com ([52.95.49.90]:46796 "EHLO smtp-fw-6002.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726108AbeIDODN (ORCPT ); Tue, 4 Sep 2018 10:03:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209; t=1536053934; x=1567589934; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=C2Hf41+IEZMk5xpKFP+uDreWWT+A7kXsuHfwxjzf0Cg=; b=lh2GfvMidmkFRXgSAqNoNzvm8YDoROxuqFjDd9ol+J2XxQTbRPaIsvMi ikfTGzcPOdwOGNjy7JjsQiQmQPfcBQTRTdtDvprxtKqmU7qMttREXP0zw 15WKVBuk56gCGiYBvhvdiDH+gji7YznhgBRhi3PQd9odi+paWhNCR7DW2 4=; X-IronPort-AV: E=Sophos;i="5.53,328,1531785600"; d="scan'208";a="360497512" Received: from iad6-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1d-74cf8b49.us-east-1.amazon.com) ([10.124.125.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 04 Sep 2018 09:38:50 +0000 Received: from u54ee758033e858cfa736.ant.amazon.com (iad7-ws-svc-lb50-vlan2.amazon.com [10.0.93.210]) by email-inbound-relay-1d-74cf8b49.us-east-1.amazon.com (8.14.7/8.14.7) with ESMTP id w849bVrO039982 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 4 Sep 2018 09:37:34 GMT Received: from u54ee758033e858cfa736.ant.amazon.com (localhost [127.0.0.1]) by u54ee758033e858cfa736.ant.amazon.com (8.15.2/8.15.2/Debian-3) with ESMTP id w849bSVP025410; Tue, 4 Sep 2018 11:37:29 +0200 Received: (from jsteckli@localhost) by u54ee758033e858cfa736.ant.amazon.com (8.15.2/8.15.2/Submit) id w849bPfQ025409; Tue, 4 Sep 2018 11:37:25 +0200 X-Authentication-Warning: u54ee758033e858cfa736.ant.amazon.com: jsteckli set sender to jsteckli@amazon.de using -f From: Julian Stecklina To: Andi Kleen Cc: Linus Torvalds , David Woodhouse , "Konrad Rzeszutek Wilk" , , Jim Mattson , "Andrew Cooper" , Linux Kernel Mailing List , Boris Ostrovsky , linux-mm , Thomas Gleixner , , , Khalid Aziz , , Liran Alon , Kees Cook , Kernel Hardening , , Tyler Hicks , John Haxby , Jon Masters Subject: Re: Redoing eXclusive Page Frame Ownership (XPFO) with isolated CPUs in mind (for KVM to isolate its guests per CPU) References: <20180903152616.GE27886@tassilo.jf.intel.com> Date: Tue, 04 Sep 2018 11:37:25 +0200 In-Reply-To: <20180903152616.GE27886@tassilo.jf.intel.com> (Andi Kleen's message of "Mon, 3 Sep 2018 08:26:16 -0700") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Andi Kleen writes: > On Sat, Sep 01, 2018 at 02:38:43PM -0700, Linus Torvalds wrote: >> On Fri, Aug 31, 2018 at 12:45 AM Julian Stecklina wrote: >> > >> > I've been spending some cycles on the XPFO patch set this week. For the >> > patch set as it was posted for v4.13, the performance overhead of >> > compiling a Linux kernel is ~40% on x86_64[1]. The overhead comes almost >> > completely from TLB flushing. If we can live with stale TLB entries >> > allowing temporary access (which I think is reasonable), we can remove >> > all TLB flushing (on x86). This reduces the overhead to 2-3% for >> > kernel compile. >> >> I have to say, even 2-3% for a kernel compile sounds absolutely horrendous. >> >> Kernel bullds are 90% user space at least for me, so a 2-3% slowdown >> from a kernel is not some small unnoticeable thing. > > Also the problem is that depending on the workload everything may fit > into the TLBs, so the temporary stale TLB entries may be around > for a long time. Modern CPUs have very large TLBs, and good > LRU policies. For the kernel entries with global bit set and > which are used for something there may be no reason ever to evict. > > Julian, I think you would need at least some quantitative perfmon data about > TLB replacement rates in the kernel to show that it's "reasonable" > instead of hand waving. That's a fair point. It definitely depends on the workload. My idle laptop gnome GUI session still causes ~40k dtlb-load-misses per second per core. My idle server (some shells, IRC client) still has ~8k dTLB load misses per second per core. Compiling something pushes this to millions of misses per second. For comparison according to https://www.7-cpu.com/cpu/Skylake_X.html SKX can fit 1536 entries into its L2 dTLB. > Most likely I suspect you would need a low frequency regular TLB > flush for the global entries at least, which will increase > the overhead again. Given the tiny experiment above, I don't think this is necessary except for highly special usecases. If stale TLB entries are a concern, the better intermediate step is to do INVLPG on the core that modified the page table. And even with these shortcomings, XPFO severely limits the data an attacker can leak from the kernel. Julian Amazon Development Center Germany GmbH Berlin - Dresden - Aachen main office: Krausenstr. 38, 10117 Berlin Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger Ust-ID: DE289237879 Eingetragen am Amtsgericht Charlottenburg HRB 149173 B