Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [RFC,05/10] x86/speculation: Add basic IBRS support
 infrastructure
To:     Linus Torvalds <torvalds@linux-foundation.org>,
        David Woodhouse <dwmw2@infradead.org>
Cc:     Arjan van de Ven <arjan@linux.intel.com>,
        Eduardo Habkost <ehabkost@redhat.com>,
        KarimAllah Ahmed <karahmed@amazon.de>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Andi Kleen <ak@linux.intel.com>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Andy Lutomirski <luto@kernel.org>,
        Ashok Raj <ashok.raj@intel.com>,
        Asit Mallick <asit.k.mallick@intel.com>,
        Borislav Petkov <bp@suse.de>,
        Dan Williams <dan.j.williams@intel.com>,
        Dave Hansen <dave.hansen@intel.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
        Janakarajan Natarajan <Janakarajan.Natarajan@amd.com>,
        Joerg Roedel <joro@8bytes.org>,
        Jun Nakajima <jun.nakajima@intel.com>,
        Laura Abbott <labbott@redhat.com>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= <rkrcmar@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Tim Chen <tim.c.chen@linux.intel.com>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        KVM list <kvm@vger.kernel.org>,
        the arch/x86 maintainers <x86@kernel.org>,
        "Dr. David Alan Gilbert" <dgilbert@redhat.com>
References: <1516476182-5153-6-git-send-email-karahmed@amazon.de>
 <20180129201404.GA1588@localhost.localdomain>
 <1517257022.18619.30.camel@infradead.org>
 <20180129204256.GV25150@localhost.localdomain>
 <31415b7f-9c76-c102-86cd-6bf4e23e3aee@linux.intel.com>
 <1517259759.18619.38.camel@infradead.org>
 <CA+55aFxBh3LvsQq9wy313NQCn3iu+yuAmsi0zNVxWmGDUUds-A@mail.gmail.com>
From:   Christian Borntraeger <borntraeger@de.ibm.com>
Date:   Tue, 30 Jan 2018 13:11:25 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.5.2
MIME-Version: 1.0
In-Reply-To: <CA+55aFxBh3LvsQq9wy313NQCn3iu+yuAmsi0zNVxWmGDUUds-A@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Message-Id: <56a33b36-5568-5d6e-a858-3b22ea335bcb@de.ibm.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


On 01/30/2018 01:23 AM, Linus Torvalds wrote:
[...]
> 
> So I actually have a _different_ question to the virtualization
> people. This includes the vmware people, but it also obviously
> incldues the Amazon AWS kind of usage.
> 
> When you're a hypervisor (whether vmware or Amazon), why do you even
> end up caring about these things so much? You're protected from
> meltdown thanks to the virtual environment already having separate
> page tables.  And the "big hammer" approach to spectre would seem to
> be to just make sure the BTB and RSB are flushed at vmexit time - and
> even then you might decide that you really want to just move it to
> vmenter time, and only do it if the VM has changed since last time
> (per CPU).
> 
> Why do you even _care_ about the guest, and how it acts wrt Skylake?
> What you should care about is not so much the guests (which do their
> own thing) but protect guests from each other, no?
> 
> So I'm a bit mystified by some of this discussion within the context
> of virtual machines. I think that is separate from any measures that
> the guest machine may then decide to partake in.
> 
> If you are ever going to migrate to Skylake, I think you should just
> always tell the guests that you're running on Skylake. That way the
> guests will always assume the worst case situation wrt Specte.
> 
> Maybe that mystification comes from me missing something.

I can only speak for KVM, but I think the hypervisor issues come from
the fact that for migration purposes the hypervisor "lies" to the guest
in regard to what kind of CPU is running.  (it has to lie, see below).

This is to avoid random guest crashes by not announcing features. For
example if you want to migrate forth and back between a system that
has AVX512 and another one that has not you must tell the guest that
AVX512 is not available - even if it runs on the capable system.

To protect against new features the hypervisor only announces features
that it understands.
So you essentially start a VM in QEMU of a given CPU type that is
constructed of a base cpu type plus extra features. Before migration, 
it is checked if  he target system can run a guest of given type - 
otherwise migration is rejected. 

The management stack also knows things like baselining - basically
creating the best possible guest CPU given a set of hosts.

The problem now is: If you have lets say Broadwell and Skylakes.
What kind of CPU type are you telling your guest? If you claim
broadwell but run on skylake then you prevent that the guest can 
protect itself, because the guest does not know that it should do 
something special. If you say skylake the guest might start using
features that broadwell does not understand.

So I think what we have here is that the current (guest) cpu model
for hypervisors was always designed for architectural features.
Presenting a microarchitectural knowledge for workarounds does
not seem to be well integrated into hypervisors.


PS: For a list of potential cpus/features look at
https://libvirt.org/git/?p=libvirt.git;a=blob;f=src/cpu/cpu_map.xml