Received: by 2002:ac0:aa62:0:0:0:0:0 with SMTP id w31-v6csp706378ima; Wed, 24 Oct 2018 08:01:31 -0700 (PDT) X-Google-Smtp-Source: AJdET5dbxsMpPZ4MkS4umzqKBWqJyN7byCFOWPHy2b1ygg8ua/gTc28UHZ13dA9XGlHFcG30SwIS X-Received: by 2002:a62:4dc4:: with SMTP id a187-v6mr3011428pfb.92.1540393291753; Wed, 24 Oct 2018 08:01:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540393291; cv=none; d=google.com; s=arc-20160816; b=YjdCdlW1EOw0bUt3nHqZz565NcWW/qhR7X/85ufpv1M58nLWd06zfcdhwz417QFZTQ g/7mAZahxnSyc3wLn5SwwgmYr0xvutmYM640+LmLisfKIvwlK4KPfjJrp7bca9NHbM6Y vi046FrnUDM/0vDhC7QjJVvhurOnGPAIcNDpZYFxnekPvc7xI8EhI4v6R84OctPVkfRg rzXnd80IkGTGwsxiD1N7A1GKongCHg0eTnpB1upSPhBp6DIgcLrGDD+0nPqwsxfF5ry3 z7nHSE5WLADD/xnGCPgO0ZE6nApdHGw8gTyDQ/xUfpzJ8CtpNV5L+s0q8Bqo425NpNgN fwGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:dkim-signature; bh=9jStwo6WLQ1qKFH+n6UnzhTtzYHHqiefUGPudeKO1OU=; b=pTA2lAPUFhrYL/ZlfQmicq5Yw/5Akr4HYzwpXYJPA4Dh9PJru5V9nm5A3wq5GoYP2q vX6NByA6q5DNC6ctqX3ZM6kag3GqDGb4/dN6kS/dDR2oSiE4Y+4s8x9eWN7MEUMndaIn 9uv1pIViNALkDynUYx1u1a6dfBywFHYI+bbW47EC8plC0Bti61XUVVzQDss4phOm4WXb d9jASZ/0BXRVeXhB8E5jlQtHP8TtiYYNpvNLdrecgmp0NGu+qMO7na6SBs/oAwa8R8VN qTejLmSB+1P9G6KD9MkF0ICQFeG//C0Fh7daoB2M8XE35DWW9TJoRGBrAvIIez0VSgPm pPDQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@tycho-ws.20150623.gappssmtp.com header.s=20150623 header.b=txnWwhbz; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r22-v6si6178062pfr.18.2018.10.24.08.01.13; Wed, 24 Oct 2018 08:01:31 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@tycho-ws.20150623.gappssmtp.com header.s=20150623 header.b=txnWwhbz; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726942AbeJXX3N (ORCPT + 99 others); Wed, 24 Oct 2018 19:29:13 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:35596 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726748AbeJXX3N (ORCPT ); Wed, 24 Oct 2018 19:29:13 -0400 Received: by mail-qk1-f195.google.com with SMTP id v68-v6so3496180qka.2 for ; Wed, 24 Oct 2018 08:00:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tycho-ws.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=9jStwo6WLQ1qKFH+n6UnzhTtzYHHqiefUGPudeKO1OU=; b=txnWwhbzDOHa2eaUxgFLUfeSezrS5VXz2IvGkztsUDxFskTWKMLgQSeGC9emnivYCo PwOqHQ2UXaDEDqELuYZ35uB82oCtp2y3V9CWv6h6uycfgtLloULPjHB/CERJzwxw+Yii AL52tU/HYXrgfEmHrjrscxCLaXFEP2gGMkN3yGnHElh+rEkK0mmbbrYUJ9R2iEGvHFro 2tSvcFPLXmRx3gy8CoYcJsNMvTXvcw/kLjr4GY4U9PqcF4V2ZzlnqtsYKeX08UFWyWKl mCaycH5NhTY1MFBtt1RNB8r4HUgHgvAA8PS3pZIRrO7zy1l/SPs/EbnskCnWEOhtdPEk lq/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=9jStwo6WLQ1qKFH+n6UnzhTtzYHHqiefUGPudeKO1OU=; b=jDK3dDQgFjv3qGsVaoqGxdZvWLu/CW7ELSPm0WKqsNhuZHhmTGmzqJ7YErLCfCXdd6 +gPHVnoCJSCgzRjKWR977qNrVcTIPP+kRBqc8h0zgfvX7olHB6imh3bMECXM+rcW2gEY /QfsLSoXnnRaeEJWSkpaXYpaXarUY4x358PQBXPAAB2aK5Q1nr5skKgLqYlyEInVpeR/ q/OPK8/d3offrE0h7hX0wlX4eVVSvbN0V8mFEku45cUX+pcm9pr5ZgOKLwQ4GhFE7nzb kwyvWs0bwbmsLuBo3Mq8ioijEcZxYLq+DLGeYolNp9gtCw/XQzF71t7x09VGGMku5F4/ CPrw== X-Gm-Message-State: AGRZ1gLtps8RBxyEevYoT0JIgmqUK+VWKVhe0NHcHQSyQqOAfSMnVNrX fmRE77ujL7KqWGwl1hChEVb9DA== X-Received: by 2002:ac8:dcd:: with SMTP id t13-v6mr1775207qti.291.1540393245148; Wed, 24 Oct 2018 08:00:45 -0700 (PDT) Received: from cisco ([192.241.255.151]) by smtp.gmail.com with ESMTPSA id l186-v6sm2659945qkf.60.2018.10.24.08.00.32 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Oct 2018 08:00:38 -0700 (PDT) Date: Wed, 24 Oct 2018 16:00:29 +0100 From: Tycho Andersen To: Khalid Aziz Cc: "Stecklina, Julian" , "juerg.haefliger@hpe.com" , "deepa.srinivasan@oracle.com" , "jmattson@google.com" , "andrew.cooper3@citrix.com" , "Woodhouse, David" , "torvalds@linux-foundation.org" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "boris.ostrovsky@oracle.com" , "pradeep.vincent@oracle.com" , "konrad.wilk@oracle.com" , "tglx@linutronix.de" , "kanth.ghatraju@oracle.com" , "joao.m.martins@oracle.com" , "liran.alon@oracle.com" , "ak@linux.intel.com" , "keescook@google.com" , "kernel-hardening@lists.openwall.com" , "chris.hyser@oracle.com" , "tyhicks@canonical.com" , "john.haxby@oracle.com" , "jcm@redhat.com" Subject: Re: Redoing eXclusive Page Frame Ownership (XPFO) with isolated CPUs in mind (for KVM to isolate its guests per CPU) Message-ID: <20181024150029.GB9019@cisco> References: <5efc291c-b0ed-577e-02d1-285d080c293d@oracle.com> <7221975d-6b67-effa-2747-06c22c041e78@oracle.com> <1537800341.9745.20.camel@amazon.de> <063f5efc-afb2-471f-eb4b-79bf90db22dd@oracle.com> <6cc985bb-6aed-4fb7-0ef2-43aad2717095@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6cc985bb-6aed-4fb7-0ef2-43aad2717095@oracle.com> User-Agent: Mutt/1.9.4 (2018-02-28) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Oct 24, 2018 at 04:30:42PM +0530, Khalid Aziz wrote: > On 10/15/2018 01:37 PM, Khalid Aziz wrote: > > On 09/24/2018 08:45 AM, Stecklina, Julian wrote: > > > I didn't test the version with TLB flushes, because it's clear that the > > > overhead is so bad that no one wants to use this. > > > > I don't think we can ignore the vulnerability caused by not flushing > > stale TLB entries. On a mostly idle system, TLB entries hang around long > > enough to make it fairly easy to exploit this. I was able to use the > > additional test in lkdtm module added by this patch series to > > successfully read pages unmapped from physmap by just waiting for system > > to become idle. A rogue program can simply monitor system load and mount > > its attack using ret2dir exploit when system is mostly idle. This brings > > us back to the prohibitive cost of TLB flushes. If we are unmapping a > > page from physmap every time the page is allocated to userspace, we are > > forced to incur the cost of TLB flushes in some way. Work Tycho was > > doing to implement Dave's suggestion can help here. Once Tycho has > > something working, I can measure overhead on my test machine. Tycho, I > > can help with your implementation if you need. > > I looked at Tycho's last patch with batch update from > . I ported it on top of Julian's > patches and got it working well enough to gather performance numbers. Here > is what I see for system times on a machine with dual Xeon E5-2630 and 256GB > of memory when running "make -j30 all" on 4.18.6 kernel (percentages are > relative to base 4.19-rc8 kernel without xpfo): > > > Base 4.19-rc8 913.84s > 4.19-rc8 + xpfo, no TLB flush 1027.985s (+12.5%) > 4.19-rc8 + batch update, no TLB flush 970.39s (+6.2%) > 4.19-rc8 + xpfo, TLB flush 8458.449s (+825.6%) > 4.19-rc8 + batch update, TLB flush 4665.659s (+410.6%) > > Batch update is significant improvement but we are starting so far behind > baseline, it is still a huge slow down. There's some other stuff that Dave suggested that I didn't do; in particular coalesce xpfo bits instead of setting things once per page when mappings are shared, etc. Perhaps that will help more? I'm still stuck working on something else for now, but I hope to be able to participate more on this Soon (TM). Thanks for the testing! Tycho