MIME-Version: 1.0
In-Reply-To: <dfbc88a7-8417-af05-4500-5c78b0f28b6a@redhat.com>
References: <1515058213.12987.89.camel@amazon.co.uk> <20180104143710.8961-1-dwmw@amazon.co.uk>
 <20180104181744.komdplek7nfdvlsw@ast-mbp> <CA+55aFxRwAEe1VH-E_HMPf+jpxAUWaUkwLbJLXG2U44qHga+Dg@mail.gmail.com>
 <20180104183559.wlqoxmp7rf4d44ku@ast-mbp> <1515094078.29312.17.camel@infradead.org>
 <20180105102824.GA247671@google.com> <dfbc88a7-8417-af05-4500-5c78b0f28b6a@redhat.com>
From: Paul Turner <pjt@google.com>
Date: Fri, 5 Jan 2018 04:20:01 -0800
Message-ID: <CAPM31RJ_zbCkQQCk1XEY+uKgeDO750E+8F+zM+X9Rq=KP+wDmg@mail.gmail.com>
Subject: Re: [PATCH v3 01/13] x86/retpoline: Add initial retpoline support
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: David Woodhouse <dwmw2@infradead.org>,
        Alexei Starovoitov <alexei.starovoitov@gmail.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Andi Kleen <ak@linux.intel.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Greg Kroah-Hartman <gregkh@linux-foundation.org>,
        Tim Chen <tim.c.chen@linux.intel.com>,
        Dave Hansen <dave.hansen@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Kees Cook <keescook@google.com>,
        Rik van Riel <riel@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Andy Lutomirski <luto@amacapital.net>,
        Jiri Kosina <jikos@kernel.org>,
        One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org

On Fri, Jan 5, 2018 at 3:26 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 05/01/2018 11:28, Paul Turner wrote:
>>
>> The "pause; jmp" sequence proved minutely faster than "lfence;jmp" which is why
>> it was chosen.
>>
>>   "pause; jmp" 33.231 cycles/call 9.517 ns/call
>>   "lfence; jmp" 33.354 cycles/call 9.552 ns/call
>
> Do you have timings for a non-retpolined indirect branch with the
> predictor suppressed via IBRS=1?  So at least we can compute the break
> even point.

The data I collected here previously had the run-time cost as a wash.
On Skylake, an IBRS=1 and a retpolined indirect branch had cost within
a few cycles.

The costs to consider when making a choice here are:

- The transition overheads.  This is how frequently will you be
switching in and out of protected code (as IBRS needs to be enabled
and disabled at these boundaries).
- The frequency at which you will be executing protected code on one
sibling, and unprotected code on another (enabling IBRS may affect
sibling execution, depending on SKU)
- The implementation cost (retpoline requires auditing/rebuilding your
target, while IBRS can be used out of the box).


>
> Paolo