Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753135AbeADOUa (ORCPT + 1 other); Thu, 4 Jan 2018 09:20:30 -0500 Received: from mx1.redhat.com ([209.132.183.28]:41218 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752845AbeADOU3 (ORCPT ); Thu, 4 Jan 2018 09:20:29 -0500 Subject: Re: Avoid speculative indirect calls in kernel To: "Woodhouse, David" , "pavel@ucw.cz" Cc: "tim.c.chen@linux.intel.com" , "linux-kernel@vger.kernel.org" , "torvalds@linux-foundation.org" , "tglx@linutronix.de" , "andi@firstfloor.org" , "gnomes@lxorguk.ukuu.org.uk" , "dave.hansen@intel.com" , "gregkh@linux-foundation.org" , Andrea Arcangeli References: <20180103230934.15788-1-andi@firstfloor.org> <20180104114231.GB1702@amd> <1515066469.12987.112.camel@amazon.co.uk> From: Paolo Bonzini Message-ID: <94b12025-b27c-04d2-8726-c07a3af6b265@redhat.com> Date: Thu, 4 Jan 2018 15:20:24 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <1515066469.12987.112.camel@amazon.co.uk> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 04 Jan 2018 14:20:29 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Return-Path: On 04/01/2018 12:47, Woodhouse, David wrote: > On Thu, 2018-01-04 at 12:42 +0100, Pavel Machek wrote: >> >>> No, really. The full mitigation with the microcode update and IBRS >>> support is *slow*. Horribly slow. >> >> What is IBRS? Invalidate BRanch prediction bufferS? > > That isn't the precise acronym, but yes. My stab at backronyming it was "indirect branch restricted speculation", and there's also IBPB which is "indirect branch prediction barrier". > The branch predictor flush that, without retpoline, we have to do on > every entry to the kernel. Requires new microcode, and the patches that > I believe Intel are *about* to post... Based on our experiments it was actually pretty good on Skylake, and bad to horrible on earlier machines. Still, microbenchmarks were in the same ballpark as PTI. BTW, the choices we have implemented in RHEL are basically: * do nothing * turn off indirect branch prediction in kernel mode (IBRS=1 in ring 0) * completely turn off indirect branch prediction (IBRS=1 always, which would also let future CPUs do their thing), which can be good for paranoia but also for performance in some worklods * never turn off indirect branch prediction, but use a branch prediction barrier on every mode switch (needed for current AMD microcode) and though we're obviously not wed to the specific debugfs paths and names, I'd really like all of them to be available upstream too. The effect on performance (or lack of performance...) *is* there, so people should be able to understand the effect on their workloads and customize accordingly. I hope that, after the first few weeks of panic, people will learn to decide on a case-by-case basis whether to enable both PTI and IBRS/IBPB, or none. Distros will provide tuning guide and automatic tuning profiles, etc., and the world will go on. Paolo ps: BTW^2 (and this is of course not about you, David) I'm disappointed that for "Spectre" there was no discussion between upstream developers, or between Linux vendors, or in fact hardly any discussion beyond "these are the patches". I understand that (unlike PTI) there was no back story to cover up the actual vulnerability, but... grow up, folks. Seriously, "these are the patches" won't fly with either upstream or distros. > The first variant (all they can do on current CPUs with a microcode > update) is really slow, and thus retpoline is *very* much the preferred > option to protect the kernel on current CPUs. > Later CPUs will apparently have a better version of IBRS which is > preferred, so we'll ALTERNATIVE out the retpoline if we discover we're > running on one of those. > > Public docs will, presumably, be forthcoming Real Soon Now™. > > > > Amazon Web Services UK Limited. Registered in England and Wales with > registration number 08650665 and which has its registered office at 60 > Holborn Viaduct, London EC1A 2FD, United Kingdom.