Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
In-Reply-To: <cover.1506559196.git.yi.z.zhang@linux.intel.com>
References: <cover.1506559196.git.yi.z.zhang@linux.intel.com>
From:   Jim Mattson <jmattson@google.com>
Date:   Fri, 13 Oct 2017 09:57:45 -0700
Message-ID: <CALMp9eRroi5m_t8oVfiJWBLUWchF2pUt8Lenh_JEsAdiR5fWcA@mail.gmail.com>
Subject: Re: [PATCH RFC 00/10] Intel EPT-Based Sub-page Write Protection Support.
To:     Zhang Yi <yi.z.zhang@linux.intel.com>
Cc:     kvm list <kvm@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= <rkrcmar@redhat.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

I'll ask before Paolo does: Can you please add kvm-unit-tests to
exercise all of this new code?

BTW, what generation of hardware do we need to exercise this code ourselves=
?

On Fri, Oct 13, 2017 at 4:11 PM, Zhang Yi <yi.z.zhang@linux.intel.com> wrot=
e:
> From: Zhang Yi Z <yi.z.zhang@linux.intel.com>
>
> Hi All,
>
> Here is a patch-series which adding EPT-Based Sub-page Write Protection S=
upport. You can get It's software developer manuals from:
>
> https://software.intel.com/sites/default/files/managed/c5/15/architecture=
-instruction-set-extensions-programming-reference.pdf
>
> In Chapter 4 EPT-BASED SUB-PAGE PERMISSIONS.
>
> Introduction:
>
> EPT-Based Sub-page Write Protection referred to as SPP, it is a capabilit=
y which allow Virtual Machine Monitors(VMM) to specify write-permission for=
 guest physical memory at a sub-page(128 byte) granularity.  When this capa=
bility is utilized, the CPU enforces write-access permissions for sub-page =
regions of 4K pages as specified by the VMM. EPT-based sub-page permissions=
 is intended to enable fine-grained memory write enforcement by a VMM for s=
ecurity(guest OS monitoring) and usages such as device virtualization and m=
emory check-point.
>
> How SPP Works:
>
> SPP is active when the "sub-page write protection" VM-execution control i=
s 1. A new 4-level paging structure named SPP page table(SPPT) is introduce=
d, SPPT will look up the guest physical addresses to derive a 64 bit "sub-p=
age permission" value containing sub-page write permissions. The lookup fro=
m guest-physical addresses to the sub-page region permissions is determined=
 by a set of this SPPT paging structures.
>
> The SPPT is used to lookup write permission bits for the 128 byte sub-pag=
e regions containing in the 4KB guest physical page. EPT specifies the 4KB =
page level privileges that software is allowed when accessing the guest phy=
sical address, whereas SPPT defines the write permissions for software at t=
he 128 byte granularity regions within a 4KB page. Write accesses prevented=
 due to sub-page permissions looked up via SPPT are reported as EPT violati=
on VM exits. Similar to EPT, a logical processor uses SPPT to lookup sub-pa=
ge region write permissions for guest-physical addresses only when those ad=
dresses are used to access memory.
>
> Guest write access --> GPA --> Walk EPT --> EPT leaf entry -=E2=94=90
> =E2=94=8C-----------------------------------------------------------=E2=
=94=98
> =E2=94=94-> if VMexec_control.spp && ept_leaf_entry.spp_bit (bit 61)
>      |
>      =E2=94=94-> <false> --> EPT legacy behavior
>      |
>      |
>      =E2=94=94-> <true>  --> if ept_leaf_entry.writable
>                       |
>                       =E2=94=94-> <true>  --> Ignore SPP
>                       |
>                       =E2=94=94-> <false> --> GPA --> Walk SPP 4-level ta=
ble--=E2=94=90
>                                                                       |
> =E2=94=8C------------<----------get-the-SPPT-point-from-VMCS-filed-----<-=
-----=E2=94=98
> |
> Walk SPP L4E table
> |
> =E2=94=94=E2=94=90--> entry misconfiguration ------------>----------=E2=
=94=90<----------------=E2=94=90
>  |                                                  |                 |
> else                                                |                 |
>  |                                                  |                 |
>  |   =E2=94=8C------------------SPP VMexit<-----------------=E2=94=98    =
             |
>  |   |                                                                |
>  |   =E2=94=94-> exit_qualification & sppt_misconfig --> sppt misconfig  =
     |
>  |   |                                                                |
>  |   =E2=94=94-> exit_qualification & sppt_miss --> sppt miss            =
     |
>  =E2=94=94--=E2=94=90                                                    =
             |
>     |                                                                 |
> walk SPPT L3E--=E2=94=90--> if-entry-misconfiguration------------>-------=
-----=E2=94=98
>                |                                                      |
>               else                                                    |
>                |                                                      |
>                |                                                      |
>         walk SPPT L2E --=E2=94=90--> if-entry-misconfiguration-------->--=
-----=E2=94=98
>                         |                                             |
>                        else                                           |
>                         |                                             |
>                         |                                             |
>                  walk SPPT L1E --=E2=94=90-> if-entry-misconfiguration---=
>----=E2=94=98
>                                  |
>                                 else
>                                  |
>                                  =E2=94=94-> if sub-page writable
>                                       =E2=94=94-> <true>  allow, write ac=
cess
>                                       =E2=94=94-> <false> disallow, EPT v=
iolation
>
> Patch-sets Description:
>
> Patch 1: Documentation.
>
> Patch 2: This patch adds reporting SPP capability from VMX Procbased MSR,=
 according to the definition of hardware spec, bit 23 is the control of the=
 SPP capability.
>
> Patch 3: Add new secondary processor-based VM-execution control bit which=
 defined as "sub-page write permission", same as VMX Procbased MSR, bit 23 =
is the enable bit of SPP.
> Also we introduced a kernel parameter "enable_ept_spp", now SPP is active=
 when the "Sub-page Write Protection" in Secondary  VM-Execution Control is=
 set and enable the kernel parameter by "enable_ept_spp=3D1".
>
> Patch 4: Introduced the spptp and spp page table.
> The sub-page permission table is referenced via a 64-bit control field ca=
lled Sub-Page Permission Table Pointer (SPPTP) which contains a 4K-aligned =
physical address. The index and encoding for this VMCS field if defined 0x2=
030 at this time The format of SPPTP is shown in below figure 2:
> this patch introduced the Spp paging structures, which root page will cre=
ated at kvm mmu page initialization.
> Also we added a mmu page role type spp to distinguish it is a spp page or=
 a EPT page.
>
> Patch 5: Introduced the SPP-Induced VM exit and it's handle.
> Accesses using guest-physical addresses may cause SPP-induced VM exits du=
e to an SPPT misconfiguration or an SPPT miss. The basic VM exit reason cod=
e reporte for SPP-induced VM exits is 66.
>
> Also introduced the new exit qualification for SPPT-induced vmexits.
>
> | Bit   | Contents                                                       =
   |
> | :---- | :--------------------------------------------------------------=
-- |
> | 10:0  | Reserved (0).                                                  =
   |
> | 11    | SPPT VM exit type. Set for SPPT Miss, cleared for SPPT Misconfi=
g. |
> | 12    | NMI unblocking due to IRET                                     =
   |
> | 63:13 | Reserved (0)                                                   =
   |
>
> Patch 6: Added a handle of EPT subpage write protection fault.
> A control bit in EPT leaf paging-structure entries is defined as =E2=80=
=9CSub-Page Permission=E2=80=9D (SPP bit). The bit position is 61; it is ch=
osen from among the bits that are currently ignored by the processor and av=
ailable to software.
> While hardware walking the SPP page table, If the sub-page region write p=
ermission bit is set, the write is allowed, else the write is disallowed an=
d results in an EPT violation.
> We need peek this case in EPT violation handler, and trigger a user-space=
 exit, return the write protected address(GVA) to user(qemu).
>
> Patch 7: Introduce ioctls to set/get Sub-Page Write Protection.
> We introduced 2 ioctls to let user application to set/get subpage write p=
rotection bitmap per gfn, each gfn corresponds to a bitmap.
> The user application, qemu, or some other security control daemon. will s=
et the protection bitmap via this ioctl.
> the API defined as:
>         struct kvm_subpage {
>                 __u64 base_gfn;
>                 __u64 npages;
>                 /* sub-page write-access bitmap array */
>                 __u32 access_map[SUBPAGE_MAX_BITMAP];
>                 }sp;
>         kvm_vm_ioctl(s, KVM_SUBPAGES_SET_ACCESS, &sp)
>         kvm_vm_ioctl(s, KVM_SUBPAGES_GET_ACCESS, &sp)
>
> Patch 8 ~ Patch 9: Setup spp page table and update the EPT leaf entry ind=
icated with the SPP enable bit.
> If the sub-page write permission VM-execution control is set, treatment o=
f write accesses to guest-physical accesses depends on the state of the acc=
umulated write-access bit (position 1) and sub-page permission bit (positio=
n 61) in the EPT leaf paging-structure.
> Software will update the EPT leaf entry sub-page permission bit while kvm=
_set_subpage(patch 7). If the EPT write-access bit set to 0 and the SPP bit=
 set to 1 in the leaf EPT paging-structure entry that maps a 4KB page, then=
 the hardware will look up a VMM-managed Sub-Page Permission Table (SPPT), =
which will be prepared by setup kvm_set_subpage(patch 8).
> The hardware uses the guest-physical address and bits 11:7 of the address=
 accessed to lookup the SPPT to fetch a write permission bit for the 128 by=
te wide sub-page region being accessed within the 4K guest-physical page. I=
f the sub-page region write permission bit is set, the write is allowed, ot=
herwise the write is disallowed and results in an EPT violation.
> Guest-physical pages mapped via leaf EPT-paging-structures for which the =
accumulated write-access bit and the SPP bits are both clear (0) generate E=
PT violations on memory writes accesses. Guest-physical pages mapped via EP=
T-paging-structure for which the accumulated write-access bit is set (1) al=
low writes, effectively ignoring the SPP bit on the leaf EPT-paging structu=
re.
> Software will setup the spp page table level4,3,2 as well as EPT page str=
ucture, and fill the level 1 page via the 32 bit bitmaps per a single 4K pa=
ge. Now it could be divided to 32 x 128 sub-pages.
>
> The SPP L4E L3E L2E is defined as below figure.
>
> | Bit    | Contents                                                      =
         |
> | :----- | :-------------------------------------------------------------=
-------- |
> | 0      | Valid entry when set; indicates whether the entry is present  =
         |
> | 11:1   | Reserved (0)                                                  =
         |
> | N-1:12 | Physical address of 4K aligned SPPT LX-1 Table referenced by t=
he entry |
> | 51:N   | Reserved (0)                                                  =
         |
> | 63:52  | Reserved (0)                                                  =
         |
> Note: N is the physical address width supported by the processor, X is th=
e page level
>
> The SPP L1E format is defined as below figure.
> | Bit   | Contents                                                       =
   |
> | :---- | :--------------------------------------------------------------=
-- |
> | 0+2i  | Write permission for i-th 128 byte sub-page region.            =
   |
> | 1+2i  | Reserved (0).                                                  =
   |
> Note: `0<=3Di<=3D31`
>
>
> Zhang Yi Z (10):
>   KVM: VMX: Added EPT Subpage Protection Documentation.
>   x86/cpufeature: Add intel Sub-Page Protection to CPU features
>   KVM: VMX: Added VMX SPP feature flags and VM-Execution Controls.
>   KVM: VMX: Introduce the SPPTP and SPP page table.
>   KVM: VMX: Introduce SPP-Induced vm exit and it's handle.
>   KVM: VMX: Added handle of SPP write protection fault.
>   KVM: VMX: Introduce ioctls to set/get Sub-Page Write Protection.
>   KVM: VMX: Update the EPT leaf entry indicated with the SPP enable bit.
>   KVM: VMX: Added setup spp page structure.
>   KVM: VMX: implement setup SPP page structure in spp miss.
>
>  Documentation/virtual/kvm/spp_design_kvm.txt | 272 +++++++++++++++++++++
>  arch/x86/include/asm/cpufeatures.h           |   1 +
>  arch/x86/include/asm/kvm_host.h              |  18 +-
>  arch/x86/include/asm/vmx.h                   |  10 +
>  arch/x86/include/uapi/asm/vmx.h              |   2 +
>  arch/x86/kernel/cpu/intel.c                  |   4 +
>  arch/x86/kvm/mmu.c                           | 340 +++++++++++++++++++++=
+++++-
>  arch/x86/kvm/mmu.h                           |   1 +
>  arch/x86/kvm/vmx.c                           | 104 ++++++++
>  arch/x86/kvm/x86.c                           |  99 +++++++-
>  include/linux/kvm_host.h                     |   5 +
>  include/uapi/linux/kvm.h                     |  16 ++
>  virt/kvm/kvm_main.c                          |  26 ++
>  13 files changed, 893 insertions(+), 5 deletions(-)
>  create mode 100644 Documentation/virtual/kvm/spp_design_kvm.txt
>
> --
> 2.7.4
>

From 1581152810958355510@xxx Fri Oct 13 14:27:06 +0000 2017
X-GM-THRID: 1581152810958355510
X-Gmail-Labels: Inbox,Category Forums