Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753388AbeADPQn (ORCPT + 1 other); Thu, 4 Jan 2018 10:16:43 -0500 Received: from smtp.eu.citrix.com ([185.25.65.24]:2371 "EHLO SMTP.EU.CITRIX.COM" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753199AbeADPQl (ORCPT ); Thu, 4 Jan 2018 10:16:41 -0500 X-IronPort-AV: E=Sophos;i="5.45,507,1508803200"; d="scan'208";a="65591725" X-DWL: YES Subject: Re: Avoid speculative indirect calls in kernel To: Paolo Bonzini , "Woodhouse, David" , "pavel@ucw.cz" CC: "tim.c.chen@linux.intel.com" , "linux-kernel@vger.kernel.org" , "torvalds@linux-foundation.org" , "tglx@linutronix.de" , "andi@firstfloor.org" , "gnomes@lxorguk.ukuu.org.uk" , "dave.hansen@intel.com" , "gregkh@linux-foundation.org" , Andrea Arcangeli References: <20180103230934.15788-1-andi@firstfloor.org> <20180104114231.GB1702@amd> <1515066469.12987.112.camel@amazon.co.uk> <94b12025-b27c-04d2-8726-c07a3af6b265@redhat.com> From: Andrew Cooper Message-ID: <7a3584c6-0c00-d807-5130-13d1f4b34102@citrix.com> Date: Thu, 4 Jan 2018 14:51:13 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <94b12025-b27c-04d2-8726-c07a3af6b265@redhat.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Content-Language: en-GB X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Return-Path: On 04/01/18 14:20, Paolo Bonzini wrote: > On 04/01/2018 12:47, Woodhouse, David wrote: >> On Thu, 2018-01-04 at 12:42 +0100, Pavel Machek wrote: >>>> No, really. The full mitigation with the microcode update and IBRS >>>> support is *slow*. Horribly slow. >>> What is IBRS? Invalidate BRanch prediction bufferS? >> That isn't the precise acronym, but yes. > My stab at backronyming it was "indirect branch restricted speculation", > and there's also IBPB which is "indirect branch prediction barrier". > >> The branch predictor flush that, without retpoline, we have to do on >> every entry to the kernel. Requires new microcode, and the patches that >> I believe Intel are *about* to post... > Based on our experiments it was actually pretty good on Skylake, and bad > to horrible on earlier machines. Still, microbenchmarks were in the > same ballpark as PTI. > > BTW, the choices we have implemented in RHEL are basically: > > * do nothing > > * turn off indirect branch prediction in kernel mode (IBRS=1 in ring 0) > > * completely turn off indirect branch prediction (IBRS=1 always, which > would also let future CPUs do their thing), which can be good for > paranoia but also for performance in some worklods > > * never turn off indirect branch prediction, but use a branch prediction > barrier on every mode switch (needed for current AMD microcode) Where have you got this idea from?  Using IBPB on every mode switch would be an insane overhead to take, and isn't necessary. Also, remember that PTI and these mitigations are for orthogonal issues. Perhaps it is easiest to refer directly to the Xen SP2 mitigations and my commentary of what is going on: http://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen.git;a=blob;f=xen/arch/x86/spec_ctrl.c;h=79aedf774a390293dfd564ce978500085344e305;hb=refs/heads/sp2-mitigations-v6.5#l192 With the GCC -mindirect-branch=thunk-external support, and microcode, Xen will make a boot-time choice between using Retpoline, Lfence (which is the better AMD option, and more performant than retpoline), or IBRS on Skylake and newer processors where it is strictly necessary, as well as using IBPB whenever available. It also supports virtualising IBRS for guest usage when the kernel has chosen not to use it; a configuration I haven't seen in any of the Linux patch series thusfar. ~Andrew