Date: Tue, 9 Jan 2018 15:47:15 -0500
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Paolo Bonzini <pbonzini@redhat.com>, t@char.us.oracle.com
Cc: Arjan van de Ven <arjan@linux.intel.com>,
        Liran Alon <liran.alon@oracle.com>, jmattson@google.com,
        dwmw@amazon.co.uk, bp@alien8.de, aliguori@amazon.com,
        thomas.lendacky@amd.com, linux-kernel@vger.kernel.org,
        kvm@vger.kernel.org
Subject: Re: [PATCH 6/7] x86/svm: Set IBPB when running a different VCPU
Message-ID: <20180109204715.GL19756@char.us.oracle.com>
References: <a8eef7f8-98f8-4ff1-a05f-6b5258dd3a4d@default>
 <b0f1aadc-2d07-d9a5-851c-081914a564bc@linux.intel.com>
 <74e86dd8-804e-c9f2-098f-773283ac7065@redhat.com>
 <ed3fce02-8928-a45c-7360-1d42ed52d730@linux.intel.com>
 <1255f660-55c5-86f0-07d0-b5846af35c4a@redhat.com>
 <20180109203909.GG19756@char.us.oracle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
In-Reply-To: <20180109203909.GG19756@char.us.oracle.com>
User-Agent: Mutt/1.8.3 (2017-05-23)
Content-Transfer-Encoding: 8BIT
Sender: linux-kernel-owner@vger.kernel.org

On Tue, Jan 09, 2018 at 03:39:09PM -0500, Konrad Rzeszutek Wilk wrote:
> On Tue, Jan 09, 2018 at 05:49:08PM +0100, Paolo Bonzini wrote:
> > On 09/01/2018 17:23, Arjan van de Ven wrote:
> > > On 1/9/2018 8:17 AM, Paolo Bonzini wrote:
> > >> On 09/01/2018 16:19, Arjan van de Ven wrote:
> > >>> On 1/9/2018 7:00 AM, Liran Alon wrote:
> > >>>>
> > >>>> ----- arjan@linux.intel.com wrote:
> > >>>>
> > >>>>> On 1/9/2018 3:41 AM, Paolo Bonzini wrote:
> > >>>>>> The above ("IBRS simply disables the indirect branch predictor")
> > >>>>>> was my
> > >>>>>> take-away message from private discussion with Intel.? My guess is
> > >>>>>> that
> > >>>>>> the vendors are just handwaving a spec that doesn't match what
> > >>>>>> they have
> > >>>>>> implemented, because honestly a microcode update is unlikely to do
> > >>>>>> much
> > >>>>>> more than an old-fashioned chicken bit.? Maybe on Skylake it does
> > >>>>>> though, since the performance characteristics of IBRS are so
> > >>>>>> different
> > >>>>>> from previous processors.? Let's ask Arjan who might have more
> > >>>>>> information about it, and hope he actually can disclose it...
> > >>>>>
> > >>>>> IBRS will ensure that, when set after the ring transition, no earlier
> > >>>>> branch prediction data is used for indirect branches while IBRS is
> > >>>>> set
> > >>
> > >> Let me ask you my questions, which are independent of L0/L1/L2
> > >> terminology.
> > >>
> > >> 1) Is vmentry/vmexit considered a ring transition, even if the guest is
> > >> running in ring 0?? If IBRS=1 in the guest and the host is using IBRS,
> > >> the host will not do a wrmsr on exit.? Is this safe for the host kernel?
> > > 
> > > I think the CPU folks would want us to write the msr again.
> > 
> > Want us, or need us---and if we don't do that, what happens?  And if we
> > have to do it, how is IBRS=1 different from an IBPB?...
> 
> Arjan says 'ring transition' but I am pretty sure it is more of 'prediction
> mode change'. And from what I have gathered so far moving from lower (guest)
> to higher (hypervisor) has no bearing on the branch predicator. Meaning
> the guest ring0 can attack us if we don't touch this MSR.
> 
> We have to WRMSR 0x48 to 1 to flush out lower prediction. Aka this is a
> 'reset' button and at every 'prediction mode' you have to hit this.

I suppose means that when we VMENTER the original fix (where we
compare the host to guest) can stay - as we entering an lower prediction
mode. I wonder then what does writting 0 do to it? A nop?

> 
> 
> <sigh> Can we have a discussion on making an kvm-security mailing list
> where we can figure all this out during embargo and not have these
> misunderstandings.
> 
> > 
> > Since I am at it, what happens on *current generation* CPUs if you
> > always leave IBRS=1?  Slow and safe, or fast and unsafe?
> > 
> > >> 2) How will the future processors work where IBRS should always be =1?
> > > 
> > > IBRS=1 should be "fire and forget this ever happened".
> > > This is the only time anyone should use IBRS in practice
> > 
> > And IBPB too I hope?  But besides that, I need to know exactly how that
> > is implemented to ensure that it's doing the right thing.
> > 
> > > (and then the host turns it on and makes sure to not expose it to the
> > > guests I hope)
> > 
> > That's not that easy, because guests might have support for SPEC_CTRL
> > but not for IA32_ARCH_CAPABILITIES.
> > 
> > You could disable the SPEC_CTRL bit, but then the guest might think it
> > is not secure.  It might also actually *be* insecure, if you migrated to
> > an older CPU where IBRS is not fire-and-forget.
> > 
> > Paolo