Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain;
        charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 11.1 \(3445.4.7\))
Subject: Re: [RFC KVM 00/27] KVM Address Space Isolation
From:   Liran Alon <liran.alon@oracle.com>
In-Reply-To: <CALCETrXK8+tUxNA=iVDse31nFRZyiQYvcrQxV1JaidhnL4GC0w@mail.gmail.com>
Date:   Tue, 14 May 2019 11:05:44 +0300
Cc:     Alexandre Chartre <alexandre.chartre@oracle.com>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Radim Krcmar <rkrcmar@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        "H. Peter Anvin" <hpa@zytor.com>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Peter Zijlstra <peterz@infradead.org>,
        kvm list <kvm@vger.kernel.org>, X86 ML <x86@kernel.org>,
        Linux-MM <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
        jan.setjeeilers@oracle.com, Jonathan Adams <jwadams@google.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <1BFC571D-6C85-409C-8FD3-1E34559A277D@oracle.com>
References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com>
 <CALCETrVhRt0vPgcun19VBqAU_sWUkRg1RDVYk4osY6vK0SKzgg@mail.gmail.com>
 <C2A30CC6-1459-4182-B71A-D8FF121A19F2@oracle.com>
 <CALCETrXK8+tUxNA=iVDse31nFRZyiQYvcrQxV1JaidhnL4GC0w@mail.gmail.com>
To:     Andy Lutomirski <luto@kernel.org>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


> On 14 May 2019, at 5:07, Andy Lutomirski <luto@kernel.org> wrote:
>=20
> On Mon, May 13, 2019 at 2:09 PM Liran Alon <liran.alon@oracle.com> =
wrote:
>>=20
>>=20
>>=20
>>> On 13 May 2019, at 21:17, Andy Lutomirski <luto@kernel.org> wrote:
>>>=20
>>>> I expect that the KVM address space can eventually be expanded to =
include
>>>> the ioctl syscall entries. By doing so, and also adding the KVM =
page table
>>>> to the process userland page table (which should be safe to do =
because the
>>>> KVM address space doesn't have any secret), we could potentially =
handle the
>>>> KVM ioctl without having to switch to the kernel pagetable (thus =
effectively
>>>> eliminating KPTI for KVM). Then the only overhead would be if a =
VM-Exit has
>>>> to be handled using the full kernel address space.
>>>>=20
>>>=20
>>> In the hopefully common case where a VM exits and then gets =
re-entered
>>> without needing to load full page tables, what code actually runs?
>>> I'm trying to understand when the optimization of not switching is
>>> actually useful.
>>>=20
>>> Allowing ioctl() without switching to kernel tables sounds...
>>> extremely complicated.  It also makes the dubious assumption that =
user
>>> memory contains no secrets.
>>=20
>> Let me attempt to clarify what we were thinking when creating this =
patch series:
>>=20
>> 1) It is never safe to execute one hyperthread inside guest while =
it=E2=80=99s sibling hyperthread runs in a virtual address space which =
contains secrets of host or other guests.
>> This is because we assume that using some speculative gadget (such as =
half-Spectrev2 gadget), it will be possible to populate *some* CPU core =
resource which could then be *somehow* leaked by the hyperthread running =
inside guest. In case of L1TF, this would be data populated to the L1D =
cache.
>>=20
>> 2) Because of (1), every time a hyperthread runs inside host kernel, =
we must make sure it=E2=80=99s sibling is not running inside guest. i.e. =
We must kick the sibling hyperthread outside of guest using IPI.
>>=20
>> 3) =46rom (2), we should have theoretically deduced that for every =
#VMExit, there is a need to kick the sibling hyperthread also outside of =
guest until the #VMExit is completed. Such a patch series was =
implemented at some point but it had (obviously) significant performance =
hit.
>>=20
>>=20
> 4) The main goal of this patch series is to preserve (2), but to avoid
> the overhead specified in (3).
>>=20
>> The way this patch series achieves (4) is by observing that during =
the run of a VM, most #VMExits can be handled rather quickly and locally =
inside KVM and doesn=E2=80=99t need to reference any data that is not =
relevant to this VM or KVM code. Therefore, if we will run these =
#VMExits in an isolated virtual address space (i.e. KVM isolated address =
space), there is no need to kick the sibling hyperthread from guest =
while these #VMExits handlers run.
>=20
> Thanks!  This clarifies a lot of things.
>=20
>> The hope is that the very vast majority of #VMExit handlers will be =
able to completely run without requiring to switch to full address =
space. Therefore, avoiding the performance hit of (2).
>> However, for the very few #VMExits that does require to run in full =
kernel address space, we must first kick the sibling hyperthread outside =
of guest and only then switch to full kernel address space and only once =
all hyperthreads return to KVM address space, then allow then to enter =
into guest.
>=20
> What exactly does "kick" mean in this context?  It sounds like you're
> going to need to be able to kick sibling VMs from extremely atomic
> contexts like NMI and MCE.

Yes that=E2=80=99s true.
=E2=80=9Ckick=E2=80=9D in this context will probably mean sending an IPI =
to all sibling hyperthreads.
This IPI will cause these sibling hyperthreads to exit from guest to =
host on EXTERNAL_INTERRUPT
and wait for a condition that again allows to enter back into guest.
This condition will be once all hyperthreads of CPU core is again =
running only within KVM isolated address space of this VM.

-Liran