Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Tue, 23 Jan 2018 08:29:30 +0100
From:   Ingo Molnar <mingo@kernel.org>
To:     David Woodhouse <dwmw2@infradead.org>
Cc:     Linus Torvalds <torvalds@linux-foundation.org>,
        KarimAllah Ahmed <karahmed@amazon.de>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Andi Kleen <ak@linux.intel.com>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Andy Lutomirski <luto@kernel.org>,
        Arjan van de Ven <arjan@linux.intel.com>,
        Ashok Raj <ashok.raj@intel.com>,
        Asit Mallick <asit.k.mallick@intel.com>,
        Borislav Petkov <bp@suse.de>,
        Dan Williams <dan.j.williams@intel.com>,
        Dave Hansen <dave.hansen@intel.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
        Janakarajan Natarajan <Janakarajan.Natarajan@amd.com>,
        Joerg Roedel <joro@8bytes.org>,
        Jun Nakajima <jun.nakajima@intel.com>,
        Laura Abbott <labbott@redhat.com>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Radim =?utf-8?B?S3LEjW3DocWZ?= <rkrcmar@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Tim Chen <tim.c.chen@linux.intel.com>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        KVM list <kvm@vger.kernel.org>,
        the arch/x86 maintainers <x86@kernel.org>,
        Arjan Van De Ven <arjan.van.de.ven@intel.com>
Subject: Re: [RFC 09/10] x86/enter: Create macros to restrict/unrestrict
 Indirect Branch Speculation
Message-ID: <20180123072930.soz25cyky3u4hpgv@gmail.com>
References: <1516476182-5153-1-git-send-email-karahmed@amazon.de>
 <1516476182-5153-10-git-send-email-karahmed@amazon.de>
 <CA+55aFxUEPHc7JRQG3zmxEsfmfJOiJa1QqFBKCBNb5Tt5vfiSg@mail.gmail.com>
 <1516566497.9814.78.camel@infradead.org>
 <CA+55aFwOkH8RH12Dzs=hT3e7eS3Ckz9Ed6x_-9a0zs1cw9zbKA@mail.gmail.com>
 <1516572013.9814.109.camel@infradead.org>
 <CA+55aFwoxXmY0u84iPT-RP8DoG+Qb8t-mh2r8RLhKVAH3jm6qg@mail.gmail.com>
 <1516638426.9521.20.camel@infradead.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1516638426.9521.20.camel@infradead.org>
User-Agent: NeoMutt/20170609 (1.8.3)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

* David Woodhouse <dwmw2@infradead.org> wrote:

> But wait, why did I say "mostly"? Well, not everyone has a retpoline
> compiler yet... but OK, screw them; they need to update.
> 
> Then there's Skylake, and that generation of CPU cores. For complicated
> reasons they actually end up being vulnerable not just on indirect
> branches, but also on a 'ret' in some circumstances (such as 16+ CALLs
> in a deep chain).
> 
> The IBRS solution, ugly though it is, did address that. Retpoline
> doesn't. There are patches being floated to detect and prevent deep
> stacks, and deal with some of the other special cases that bite on SKL,
> but those are icky too. And in fact IBRS performance isn't anywhere
> near as bad on this generation of CPUs as it is on earlier CPUs
> *anyway*, which makes it not quite so insane to *contemplate* using it
> as Intel proposed.

There's another possible method to avoid deep stacks on Skylake, without compiler 
support:

  - Use the existing mcount based function tracing live patching machinery
    (CONFIG_FUNCTION_TRACER=y) to install a _very_ fast and simple stack depth 
    tracking tracer which would issue a retpoline when stack depth crosses 
    boundaries of ~16 entries.

The overhead of that would _still_ very likely be much cheaper than a hundreds 
(thousands) of cycle expensive MSR write at every kernel entry (syscall entry, IRQ 
entry, etc.).

Note the huge number of advantages:

 - All distro kernels already enable the mcount based patching options, so there's
   literally zero overhead to anything except SkyLake.

 - It is fully kernel patching based and can be activated on Skylake only

 - It doesn't require any microcode updates, so it will work on all existing CPUs
   with no firmware or microcode modificatons

 - It doesn't require any compiler updates

 - SkyLake performance is very likely to be much less fragile than relying on a 
   hastily deployed microcode hack

 - The "SkyLake stack depth tracer" can be tested on other CPUs as well in debug 
   builds, broadening the testing base

 - The tracer is very obviously simple and reviewable, and we can forget about it
   in the far future.

 - It's much more backportable to older kernels: should there be a new class of
   exploits then this machinery could be updated to cover that too - while 
   upgrades to newer kernels would give the higher performant solution.

Yes, there are some practical complications like always enabling 
CONFIG_FUNCTION_TRACER=y on x86, plus the ftrace interaction has to be sorted out, 
but in practice it's enabled on all major distros anyway, due to ftrace.

Is there any reason why this wouldn't work?

Thanks,

	Ingo