Intel Trusted Domain Extensions (TDX) protects guest VMs from malicious
host and certain physical attacks. TDX specs are available in [1].
This series provides the initial support to enable TDX in the host kernel
with minimal code. More patch series will follow up as next steps.
Specifically, below is our plan for the TDX host kernel support:
1) This initial version to enable TDX with minimal code, allowing KVM
to use TDX to create and run TDX guests. It doesn't support all
functionalities (i.e. exposing TDX module via /sysfs), and doesn't
aim to resolve all things perfectly (i.e. some optimizations are
not done). Especially, memory hotplug is not handled (please see
"Design Considerations" section below).
2) Additional patch series to handle memory hotplug or per-node TDX
capability flag.
3) More patch series to add additional functions (/sysfs, etc) and
optimizations (i.e. initializing the TDMRs).
(For memory hotplug, sorry for broadcasting widely but I cc'ed the
[email protected] following Kirill's suggestion so MM experts can also
help to provide comments.)
KVM support for TDX is being developed separately[2]. A new "userspace
inaccessible memfd" approach to support TDX private memory is also being
developed[3]. The KVM will only support the new "userspace inaccessible
memfd" as TDX guest memory backend.
I highly appreciate if anyone can help to review this series.
Hi Dave, Dan (and Intel reviewers),
Please kindly help to review, and I would appreciate reviewed-by or
acked-by tags if the patches look good to you.
This series has been reviewed by Isaku who is developing KVM TDX patches.
The first 4 patches have been reviewed by Kirill as well.
----- Changelog history: ------
- v5 -> v6:
- Removed ACPI CPU/memory hotplug patches. (Intel internal discussion)
- Removed patch to disable driver-managed memory hotplug (Intel
internal discussion).
- Added one patch to introduce enum type for TDX supported page size
level to replace the hard-coded values in TDX guest code (Dave).
- Added one patch to make TDX depends on X2APIC being enabled (Dave).
- Added one patch to build all boot-time present memory regions as TDX
memory during kernel boot.
- Added Reviewed-by from others to some patches.
- For all others please see individual patch changelog history.
- v4 -> v5:
This is essentially a resent of v4. Sorry I forgot to consult
get_maintainer.pl when sending out v4, so I forgot to add linux-acpi
and linux-mm mailing list and the relevant people for 4 new patches.
There are also very minor code and commit message update from v4:
- Rebased to latest tip/x86/tdx.
- Fixed a checkpatch issue that I missed in v4.
- Removed an obsoleted comment that I missed in patch 6.
- Very minor update to the commit message of patch 12.
For other changes to individual patches since v3, please refer to the
changelog histroy of individual patches (I just used v3 -> v5 since
there's basically no code change to v4).
- v3 -> v4 (addressed Dave's comments, and other comments from others):
- Simplified SEAMRR and TDX keyID detection.
- Added patches to handle ACPI CPU hotplug.
- Added patches to handle ACPI memory hotplug and driver managed memory
hotplug.
- Removed tdx_detect() but only use single tdx_init().
- Removed detecting TDX module via P-SEAMLDR.
- Changed from using e820 to using memblock to convert system RAM to TDX
memory.
- Excluded legacy PMEM from TDX memory.
- Removed the boot-time command line to disable TDX patch.
- Addressed comments for other individual patches (please see individual
patches).
- Improved the documentation patch based on the new implementation.
- V2 -> v3:
- Addressed comments from Isaku.
- Fixed memory leak and unnecessary function argument in the patch to
configure the key for the global keyid (patch 17).
- Enhanced a little bit to the patch to get TDX module and CMR
information (patch 09).
- Fixed an unintended change in the patch to allocate PAMT (patch 13).
- Addressed comments from Kevin:
- Slightly improvement on commit message to patch 03.
- Removed WARN_ON_ONCE() in the check of cpus_booted_once_mask in
seamrr_enabled() (patch 04).
- Changed documentation patch to add TDX host kernel support materials
to Documentation/x86/tdx.rst together with TDX guest staff, instead
of a standalone file (patch 21)
- Very minor improvement in commit messages.
- RFC (v1) -> v2:
- Rebased to Kirill's latest TDX guest code.
- Fixed two issues that are related to finding all RAM memory regions
based on e820.
- Minor improvement on comments and commit messages.
v5:
https://lore.kernel.org/lkml/[email protected]/T/
v3:
https://lore.kernel.org/lkml/[email protected]/T/
V2:
https://lore.kernel.org/lkml/[email protected]/T/
RFC (v1):
https://lore.kernel.org/all/e0ff030a49b252d91c789a89c303bb4206f85e3d.1646007267.git.kai.huang@intel.com/T/
== Background ==
TDX introduces a new CPU mode called Secure Arbitration Mode (SEAM)
and a new isolated range pointed by the SEAM Ranger Register (SEAMRR).
A CPU-attested software module called 'the TDX module' runs in the new
isolated region as a trusted hypervisor to create/run protected VMs.
TDX also leverages Intel Multi-Key Total Memory Encryption (MKTME) to
provide crypto-protection to the VMs. TDX reserves part of MKTME KeyIDs
as TDX private KeyIDs, which are only accessible within the SEAM mode.
TDX is different from AMD SEV/SEV-ES/SEV-SNP, which uses a dedicated
secure processor to provide crypto-protection. The firmware runs on the
secure processor acts a similar role as the TDX module.
The host kernel communicates with SEAM software via a new SEAMCALL
instruction. This is conceptually similar to a guest->host hypercall,
except it is made from the host to SEAM software instead.
Before being able to manage TD guests, the TDX module must be loaded
and properly initialized. This series assumes the TDX module is loaded
by BIOS before the kernel boots.
How to initialize the TDX module is described at TDX module 1.0
specification, chapter "13.Intel TDX Module Lifecycle: Enumeration,
Initialization and Shutdown".
== Design Considerations ==
1. Initialize the TDX module at runtime
There are basically two ways the TDX module could be initialized: either
in early boot, or at runtime before the first TDX guest is run. This
series implements the runtime initialization.
This series adds a function tdx_enable() to allow the caller to initialize
TDX at runtime:
if (tdx_enable())
goto no_tdx;
// TDX is ready to create TD guests.
This approach has below pros:
1) Initializing the TDX module requires to reserve ~1/256th system RAM as
metadata. Enabling TDX on demand allows only to consume this memory when
TDX is truly needed (i.e. when KVM wants to create TD guests).
2) SEAMCALL requires CPU being already in VMX operation (VMXON has been
done). So far, KVM is the only user of TDX, and it already handles VMXON.
Letting KVM to initialize TDX avoids handling VMXON in the core kernel.
3) It is more flexible to support "TDX module runtime update" (not in
this series). After updating to the new module at runtime, kernel needs
to go through the initialization process again.
2. CPU hotplug
TDX doesn't support physical (ACPI) CPU hotplug. A non-buggy BIOS should
never support hotpluggable CPU devicee and/or deliver ACPI CPU hotplug
event to the kernel. This series doesn't handle physical (ACPI) CPU
hotplug at all but depends on the BIOS to behave correctly.
Note TDX works with CPU logical online/offline, thus this series still
allows to do logical CPU online/offline.
3. Kernel policy on TDX memory
The TDX architecture allows the VMM to designate specific memory as
usable for TDX private memory. This series chooses to designate _all_
boot-time system RAM as TDX to avoid having to modify the page allocator
to distinguish TDX and non-TDX-capable memory.
4. Memory Hotplug
The TDX module reports a list of "Convertible Memory Region" (CMR) to
indicate which memory regions are TDX-capable. TDX convertible memory
must be physically present during machine boot. TDX also assumes
convertible memory won't be hot-removed. A non-buggy BIOS should never
support physical hot-removal of any TDX convertible memory. This series
doesn't handle physical hot-removal of convertible memory but depends on
the BIOS to behave correctly.
It's possible that one machine can have both TDX and non-TDX memory.
Specifically, runtime hot-added physical memory is not TDX convertible
memory. Also, for now NVDIMM and CXL memory are not TDX convertible
memory, no matter whether they are physically present during boot or not.
Plugging non-TDX memory to the page allocator could result in failing to
create a TDX guest, or killing a running TDX guest.
To keep things simple, this series doesn't handle memory hotplug at all,
but depends on the machine owner to not do any memory hotplug operation.
For exmaple, the machine owner should not plug any NVDIMM and CXL memory
into the machine, or use kmem driver to plug NVDIMM or CXL memory to the
core-mm.
This will be enhanced in the future after first submission. We are also
looking into options on how to handle:
- One solution is to enforce the kernel to always guarantee all pages in
the page allocator are TDX memory (i.e. via rejecting non-TDX memory in
memory hotplug).
- Another solution is to manage TDX and non-TDX memory in different NUMA
nodes, and use per-node TDX memory capability flag to show which nodes
are TDX-capable. Userspace needs to explicitly bind TDX guests to those
TDX-capable NUMA nodes.
The second option is similar to the per-node memory encryption flag
support in below sereies:
https://lore.kernel.org/linux-mm/[email protected]/t/
5. Kexec()
Just like SME, TDX hosts require special cache flushing before kexec().
Similar to SME handling, the kernel uses wbinvd() to flush cache in
stop_this_cpu() when TDX is enabled.
===== Reference ======
[1]: TDX specs:
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2]: KVM TDX basic feature support
https://lore.kernel.org/lkml/CAAhR5DFrwP+5K8MOxz5YK7jYShhaK4A+2h1Pi31U_9+Z+cz-0A@mail.gmail.com/T/
[3]: KVM: mm: fd-based approach for supporting KVM
https://lore.kernel.org/lkml/[email protected]/T/
Kai Huang (21):
x86/tdx: Use enum to define page level of TDX supported page sizes
x86/virt/tdx: Detect TDX during kernel boot
x86/virt/tdx: Disable TDX if X2APIC is not enabled
x86/virt/tdx: Use all boot-time system memory as TDX memory
x86/virt/tdx: Add skeleton to initialize TDX on demand
x86/virt/tdx: Implement functions to make SEAMCALL
x86/virt/tdx: Shut down TDX module in case of error
x86/virt/tdx: Do TDX module global initialization
x86/virt/tdx: Do logical-cpu scope TDX module initialization
x86/virt/tdx: Get information about TDX module and TDX-capable memory
x86/virt/tdx: Sanity check all TDX memory ranges are convertible
memory
x86/virt/tdx: Add placeholder to construct TDMRs to cover all TDX
memory regions
x86/virt/tdx: Create TDMRs to cover all TDX memory regions
x86/virt/tdx: Allocate and set up PAMTs for TDMRs
x86/virt/tdx: Set up reserved areas for all TDMRs
x86/virt/tdx: Reserve TDX module global KeyID
x86/virt/tdx: Configure TDX module with TDMRs and global KeyID
x86/virt/tdx: Configure global KeyID on all packages
x86/virt/tdx: Initialize all TDMRs
x86/virt/tdx: Flush cache in kexec() when TDX is enabled
Documentation/x86: Add documentation for TDX host support
Documentation/x86/tdx.rst | 209 ++++-
arch/x86/Kconfig | 14 +
arch/x86/Makefile | 2 +
arch/x86/coco/tdx/tdx.c | 20 +-
arch/x86/include/asm/tdx.h | 51 ++
arch/x86/kernel/process.c | 9 +-
arch/x86/virt/Makefile | 2 +
arch/x86/virt/vmx/Makefile | 2 +
arch/x86/virt/vmx/tdx/Makefile | 2 +
arch/x86/virt/vmx/tdx/seamcall.S | 52 ++
arch/x86/virt/vmx/tdx/tdx.c | 1441 ++++++++++++++++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 118 +++
arch/x86/virt/vmx/tdx/tdxcall.S | 19 +-
13 files changed, 1911 insertions(+), 30 deletions(-)
create mode 100644 arch/x86/virt/Makefile
create mode 100644 arch/x86/virt/vmx/Makefile
create mode 100644 arch/x86/virt/vmx/tdx/Makefile
create mode 100644 arch/x86/virt/vmx/tdx/seamcall.S
create mode 100644 arch/x86/virt/vmx/tdx/tdx.c
create mode 100644 arch/x86/virt/vmx/tdx/tdx.h
base-commit: 5eb443db589a4526b2bef750a998ce7f0dc9c87b
--
2.37.3
TDX supports 4K, 2M and 1G page sizes. When TDX guest accepts one page
via try_accept_one(), it passes the page size level to the TDX module.
Currently try_accept_one() uses hard-coded magic number for that.
Introduce a new enum type to represent the page level of TDX supported
page sizes to replace the hard-coded values. Both initializing the TDX
module and KVM TDX support will need to use that too.
Also, currently try_accept_one() uses an open-coded switch statement to
get the TDX page level from the kernel page level. As KVM will also
need to do the same thing, introduce a common helper to convert the
kernel page level to the TDX page level.
Reviewed-by: Kirill A. Shutemov <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
arch/x86/coco/tdx/tdx.c | 20 ++++----------------
arch/x86/include/asm/tdx.h | 33 +++++++++++++++++++++++++++++++++
2 files changed, 37 insertions(+), 16 deletions(-)
diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c
index 928dcf7a20d9..c5ff9647213d 100644
--- a/arch/x86/coco/tdx/tdx.c
+++ b/arch/x86/coco/tdx/tdx.c
@@ -655,7 +655,6 @@ static bool try_accept_one(phys_addr_t *start, unsigned long len,
{
unsigned long accept_size = page_level_size(pg_level);
u64 tdcall_rcx;
- u8 page_size;
if (!IS_ALIGNED(*start, accept_size))
return false;
@@ -663,27 +662,16 @@ static bool try_accept_one(phys_addr_t *start, unsigned long len,
if (len < accept_size)
return false;
+ /* TDX only supports 4K/2M/1G page sizes */
+ if (pg_level < PG_LEVEL_4K || pg_level > PG_LEVEL_1G)
+ return false;
/*
* Pass the page physical address to the TDX module to accept the
* pending, private page.
*
* Bits 2:0 of RCX encode page size: 0 - 4K, 1 - 2M, 2 - 1G.
*/
- switch (pg_level) {
- case PG_LEVEL_4K:
- page_size = 0;
- break;
- case PG_LEVEL_2M:
- page_size = 1;
- break;
- case PG_LEVEL_1G:
- page_size = 2;
- break;
- default:
- return false;
- }
-
- tdcall_rcx = *start | page_size;
+ tdcall_rcx = *start | to_tdx_pg_level(pg_level);
if (__tdx_module_call(TDX_ACCEPT_PAGE, tdcall_rcx, 0, 0, 0, NULL))
return false;
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 020c81a7c729..1c166fb9c22f 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -20,6 +20,39 @@
#ifndef __ASSEMBLY__
+#include <asm/pgtable_types.h>
+
+/*
+ * The page levels of TDX supported page sizes (4K/2M/1G).
+ *
+ * Those values are part of the TDX module ABI. Do not change them.
+ */
+enum tdx_pg_level {
+ TDX_PG_LEVEL_4K,
+ TDX_PG_LEVEL_2M,
+ TDX_PG_LEVEL_1G,
+ TDX_PG_LEVEL_NUM
+};
+
+/*
+ * Get the TDX page level based on the kernel page level. The caller
+ * to make sure only pass 4K/2M/1G kernel page level.
+ */
+static inline enum tdx_pg_level to_tdx_pg_level(enum pg_level pglvl)
+{
+ switch (pglvl) {
+ case PG_LEVEL_4K:
+ return TDX_PG_LEVEL_4K;
+ case PG_LEVEL_2M:
+ return TDX_PG_LEVEL_2M;
+ case PG_LEVEL_1G:
+ return TDX_PG_LEVEL_1G;
+ default:
+ WARN_ON_ONCE(1);
+ }
+ return TDX_PG_LEVEL_NUM;
+}
+
/*
* Used to gather the output registers values of the TDCALL and SEAMCALL
* instructions when requesting services from the TDX module.
--
2.37.3
TDX provides increased levels of memory confidentiality and integrity.
This requires special hardware support for features like memory
encryption and storage of memory integrity checksums. Not all memory
satisfies these requirements.
As a result, the TDX introduced the concept of a "Convertible Memory
Region" (CMR). During boot, the BIOS builds a list of all of the memory
regions which can provide the TDX security guarantees. The list of CMRs
is available to the kernel by querying the TDX module or its loader.
However those TDX-capable memory regions are not automatically usable to
the TDX module. The kernel needs to choose the memory regions that will
be used as TDX memory and pass those memory regions to the TDX module
when initializing it. Once passed to the TDX module, those TDX-usable
memory regions are fixed during the module's lifetime. The TDX module
doesn't support adding new TDX-usable memory after it gets initialized.
To keep things simple, this initial implementation chooses to use all
boot-time present memory managed by the page allocator as TDX memory.
This requires all boot-time present memory to be TDX convertible memory,
which is true in practice. If there's any boot-time memory which isn't
TDX convertible memory (which is allowed from TDX architecture's point
of view), it will be caught later during TDX module initialization and
the initialization will fail.
However one machine may support both TDX and non-TDX memory both at
machine boot time and runtime. For example, any memory hot-added at
runtime cannot be TDX memory. Also, for now NVDIMM and CXL memory are
not TDX memory, no matter whether they are present at machine boot time
or not.
This raises a problem that, if any non-TDX memory is hot-added to the
system-wide memory allocation pool, a non-TDX page may be allocated to a
TDX guest, which will result in failing to create the TDX guest, or
killing it at runtime.
The current implementation doesn't explicitly prevent adding any non-TDX
memory to system-wide memory pool, but depends on the machine owner to
make sure such operation won't happen. For example, the machine owner
should never plug any NVDIMM or CXL memory into the machine, or use kmem
driver to hot-add any to the core-mm.
This will be enhanced in the future. One solution is the kernel can be
enforced to always guarantee all pages in the page allocator are TDX
memory (i.e. by rejecting non-TDX memory in memory hotplug). Another
option is the kernel may support different memory capabilities on basis
of NUMA node. For example, the kernel can have both TDX-compatible NUMA
node and non-TDX-compatible memory NUMA node, and the userspace needs to
explicitly bind TDX guests to those TDX-compatible memory NUMA nodes.
Use memblock to get all memory regions as it is the source from where
the memory pages are freed to the page allocator. This saves the
metadata that is needed to be allocated and given to the TDX module when
initializing it in case like 'memmap' kernel command is used.
Also, don't use memblock directly when initializing the TDX module, but
make a snapshot of all memory regions in the memblock during kernel
boot. This is because:
1) Memblock is discarded after kernel boots if CONFIG_ARCH_KEEP_MEMBLOCK
is not enabled.
2) Memblock can be changed by memory hotplug (i.e. memory removal of
boot-time present memory) at runtime if it is kept.
3) The first 1MB may not be enumerated as CMR on some platforms, so the
first 1MB cannot be included as TDX memory. Looping over all TDX
memory blocks will be done couple of times when initializing the TDX
module. Instead of using memblock directly and manually excluding
the first 1MB for each loop, just do a snapshot to exclude the first
1MB once.
Reviewed-by: Kirill A. Shutemov <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
arch/x86/virt/vmx/tdx/tdx.c | 141 ++++++++++++++++++++++++++++++++++++
1 file changed, 141 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 8d943bdc8335..c5d260e27cad 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -10,15 +10,27 @@
#include <linux/types.h>
#include <linux/init.h>
#include <linux/printk.h>
+#include <linux/list.h>
+#include <linux/memblock.h>
#include <asm/msr-index.h>
#include <asm/msr.h>
#include <asm/apic.h>
#include <asm/tdx.h>
#include "tdx.h"
+struct tdx_memblock {
+ struct list_head list;
+ unsigned long start_pfn;
+ unsigned long end_pfn;
+ int nid;
+};
+
static u32 tdx_keyid_start __ro_after_init;
static u32 tdx_keyid_num __ro_after_init;
+/* All TDX-usable memory regions */
+static LIST_HEAD(tdx_memlist);
+
/*
* Detect TDX private KeyIDs to see whether TDX has been enabled by the
* BIOS. Both initializing the TDX module and running TDX guest require
@@ -66,6 +78,125 @@ static void __init clear_tdx(void)
tdx_keyid_start = tdx_keyid_num = 0;
}
+static void __init tdx_memory_destroy(void)
+{
+ while (!list_empty(&tdx_memlist)) {
+ struct tdx_memblock *tmb = list_first_entry(&tdx_memlist,
+ struct tdx_memblock, list);
+
+ list_del(&tmb->list);
+ kfree(tmb);
+ }
+}
+
+/* Add one TDX memory block after all existing TDX memory blocks */
+static int __init tdx_memory_add_block(unsigned long start_pfn,
+ unsigned long end_pfn,
+ int nid)
+{
+ struct tdx_memblock *tmb;
+
+ tmb = kmalloc(sizeof(*tmb), GFP_KERNEL);
+ if (!tmb)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&tmb->list);
+ tmb->start_pfn = start_pfn;
+ tmb->end_pfn = end_pfn;
+ tmb->nid = nid;
+
+ list_add_tail(&tmb->list, &tdx_memlist);
+
+ return 0;
+}
+
+/*
+ * TDX reports a list of "Convertible Memory Regions" (CMR) to indicate
+ * all memory regions that _can_ be used by TDX, but the kernel needs to
+ * choose the _actual_ regions that TDX can use and pass those regions
+ * to the TDX module when initializing it. After the TDX module gets
+ * initialized, no more TDX-usable memory can be hot-added to the TDX
+ * module.
+ *
+ * TDX convertible memory must be physically present during machine boot.
+ * To keep things simple, the current implementation simply chooses to
+ * use all boot-time present memory regions as TDX memory so that all
+ * pages allocated via the page allocator are TDX memory.
+ *
+ * Build all boot-time memory regions managed by memblock as TDX-usable
+ * memory regions by making a snapshot of memblock memory regions during
+ * kernel boot. Memblock is discarded when CONFIG_ARCH_KEEP_MEMBLOCK is
+ * not enabled after kernel boots. Also, memblock can be changed due to
+ * memory hotplug (i.e. memory removal from core-mm) even if it is kept.
+ *
+ * Those regions will be verified when CMRs become available when the TDX
+ * module gets initialized. At this stage, it's not possible to get CMRs
+ * during kernel boot as the core-kernel doesn't support VMXON.
+ *
+ * Note: this means the current implementation _requires_ all boot-time
+ * present memory regions are TDX convertible memory to enable TDX. This
+ * is true in practice. Also, this can be enhanced in the future when
+ * the core-kernel gets VMXON support.
+ *
+ * Important note:
+ *
+ * TDX doesn't work with physical memory hotplug, as all hot-added memory
+ * are not convertible memory.
+ *
+ * Also to keep things simple, the current implementation doesn't handle
+ * memory hotplug at all for TDX. To use TDX, it is the machine owner's
+ * responsibility to not do any operation that will hot-add any non-TDX
+ * memory to the page allocator. For example, the machine owner should
+ * not plug any non-CMR memory (such as NVDIMM and CXL memory) to the
+ * machine, or should not use kmem driver to plug any NVDIMM or CXL
+ * memory to the core-mm.
+ *
+ * This will be enhanced in the future.
+ *
+ * Note: tdx_init() is called before acpi_init(), which will scan the
+ * entire ACPI namespace and hot-add all ACPI memory devices if there
+ * are any. This belongs to the memory hotplug category as mentioned
+ * above.
+ */
+static int __init build_tdx_memory(void)
+{
+ unsigned long start_pfn, end_pfn;
+ int i, nid, ret;
+
+ /*
+ * Cannot use for_each_free_mem_range() here as some reserved
+ * memory (i.e. initrd image) will be freed to the page allocator
+ * at the late phase of kernel boot.
+ */
+ for_each_mem_pfn_range(i, MAX_NUMNODES, &start_pfn, &end_pfn, &nid) {
+ /*
+ * The first 1MB is not reported as TDX convertible
+ * memory on some platforms. Manually exclude them as
+ * TDX memory. This is fine as the first 1MB is already
+ * reserved in reserve_real_mode() and won't end up to
+ * ZONE_DMA as free page anyway.
+ */
+ if (start_pfn < (SZ_1M >> PAGE_SHIFT))
+ start_pfn = (SZ_1M >> PAGE_SHIFT);
+ if (start_pfn >= end_pfn)
+ continue;
+
+ /*
+ * All TDX memory blocks must be in address ascending
+ * order when initializing the TDX module. Memblock
+ * already guarantees that.
+ */
+ ret = tdx_memory_add_block(start_pfn, end_pfn, nid);
+ if (ret)
+ goto err;
+ }
+
+ return 0;
+err:
+ tdx_memory_destroy();
+ return ret;
+}
+
static int __init tdx_init(void)
{
if (detect_tdx())
@@ -92,6 +223,16 @@ static int __init tdx_init(void)
goto no_tdx;
}
+ /*
+ * Build all boot-time system memory managed in memblock as
+ * TDX-usable memory. As part of initializing the TDX module,
+ * those regions will be passed to the TDX module.
+ */
+ if (build_tdx_memory()) {
+ pr_err("Build TDX-usable memory regions failed. Disable TDX.\n");
+ goto no_tdx;
+ }
+
return 0;
no_tdx:
clear_tdx();
--
2.37.3
Before the TDX module can be used to create and run TDX guests, it must
be loaded into the isolated region pointed by the SEAMRR and properly
initialized. The TDX module is expected to be loaded by the BIOS before
booting to the kernel, and the kernel is expected to detect and
initialize it.
The host kernel communicates with the TDX module via a new SEAMCALL
instruction. The TDX module implements a set of SEAMCALL leaf functions
to allow the host kernel to initialize it.
The TDX module can be initialized only once in its lifetime. Instead
of always initializing it at boot time, this implementation chooses an
"on demand" approach to initialize TDX until there is a real need (e.g
when requested by KVM). This approach has below pros:
1) It avoids consuming the memory that must be allocated by kernel and
given to the TDX module as metadata (~1/256th of the TDX-usable memory),
and also saves the CPU cycles of initializing the TDX module (and the
metadata) when TDX is not used at all.
2) It is more flexible to support TDX module runtime updating in the
future (after updating the TDX module, it needs to be initialized
again).
3) It avoids having to do a "temporary" solution to handle VMXON in the
core (non-KVM) kernel for now. This is because SEAMCALL requires CPU
being in VMX operation (VMXON is done), but currently only KVM handles
VMXON. Adding VMXON support to the core kernel isn't trivial. More
importantly, from long-term a reference-based approach is likely needed
in the core kernel as more kernel components are likely needed to
support TDX as well. Allow KVM to initialize the TDX module avoids
having to handle VMXON during kernel boot for now.
Add a placeholder tdx_enable() to detect and initialize the TDX module
on demand, with a state machine protected by mutex to support concurrent
calls from multiple callers.
The TDX module will be initialized in multi-steps defined by the TDX
module:
1) Global initialization;
2) Logical-CPU scope initialization;
3) Enumerate the TDX module capabilities and platform configuration;
4) Configure the TDX module about TDX usable memory ranges and global
KeyID information;
5) Package-scope configuration for the global KeyID;
6) Initialize usable memory ranges based on 4).
The TDX module can also be shut down at any time during its lifetime.
In case of any error during the initialization process, shut down the
module. It's pointless to leave the module in any intermediate state
during the initialization.
Also, as mentioned above, tdx_enable() relies on the caller to guarantee
CPU is already in VMX operation before calling SEAMCALL, but doesn't
explicitly handle VMXON.
Reviewed-by: Chao Gao <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
v5 -> v6:
- Added code to set status to TDX_MODULE_NONE if TDX module is not
loaded (Chao)
- Added Chao's Reviewed-by.
- Improved comments around cpus_read_lock().
- v3->v5 (no feedback on v4):
- Removed the check that SEAMRR and TDX KeyID have been detected on
all present cpus.
- Removed tdx_detect().
- Added num_online_cpus() to MADT-enabled CPUs check within the CPU
hotplug lock and return early with error message.
- Improved dmesg printing for TDX module detection and initialization.
---
arch/x86/include/asm/tdx.h | 2 +
arch/x86/virt/vmx/tdx/tdx.c | 152 ++++++++++++++++++++++++++++++++++++
2 files changed, 154 insertions(+)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 9b63f33e9c91..80c76b426adf 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -123,8 +123,10 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,
#ifdef CONFIG_INTEL_TDX_HOST
bool platform_tdx_enabled(void);
+int tdx_enable(void);
#else /* !CONFIG_INTEL_TDX_HOST */
static inline bool platform_tdx_enabled(void) { return false; }
+static inline int tdx_enable(void) { return -ENODEV; }
#endif /* CONFIG_INTEL_TDX_HOST */
#endif /* !__ASSEMBLY__ */
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index c5d260e27cad..a137350d5d0e 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -12,6 +12,9 @@
#include <linux/printk.h>
#include <linux/list.h>
#include <linux/memblock.h>
+#include <linux/mutex.h>
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
#include <asm/msr-index.h>
#include <asm/msr.h>
#include <asm/apic.h>
@@ -25,12 +28,30 @@ struct tdx_memblock {
int nid;
};
+/*
+ * TDX module status during initialization
+ */
+enum tdx_module_status_t {
+ /* TDX module hasn't been detected and initialized */
+ TDX_MODULE_UNKNOWN,
+ /* TDX module is not loaded */
+ TDX_MODULE_NONE,
+ /* TDX module is initialized */
+ TDX_MODULE_INITIALIZED,
+ /* TDX module is shut down due to initialization error */
+ TDX_MODULE_SHUTDOWN,
+};
+
static u32 tdx_keyid_start __ro_after_init;
static u32 tdx_keyid_num __ro_after_init;
/* All TDX-usable memory regions */
static LIST_HEAD(tdx_memlist);
+static enum tdx_module_status_t tdx_module_status;
+/* Prevent concurrent attempts on TDX detection and initialization */
+static DEFINE_MUTEX(tdx_module_lock);
+
/*
* Detect TDX private KeyIDs to see whether TDX has been enabled by the
* BIOS. Both initializing the TDX module and running TDX guest require
@@ -245,3 +266,134 @@ bool platform_tdx_enabled(void)
{
return !!tdx_keyid_num;
}
+
+/*
+ * Detect and initialize the TDX module.
+ *
+ * Return -ENODEV when the TDX module is not loaded, 0 when it
+ * is successfully initialized, or other error when it fails to
+ * initialize.
+ */
+static int init_tdx_module(void)
+{
+ /* The TDX module hasn't been detected */
+ return -ENODEV;
+}
+
+static void shutdown_tdx_module(void)
+{
+ /* TODO: Shut down the TDX module */
+}
+
+static int __tdx_enable(void)
+{
+ int ret;
+
+ /*
+ * Initializing the TDX module requires doing SEAMCALL on all
+ * boot-time present CPUs. For simplicity temporarily disable
+ * CPU hotplug to prevent any CPU from going offline during
+ * the initialization.
+ */
+ cpus_read_lock();
+
+ /*
+ * Check whether all boot-time present CPUs are online and
+ * return early with a message so the user can be aware.
+ *
+ * Note a non-buggy BIOS should never support physical (ACPI)
+ * CPU hotplug when TDX is enabled, and all boot-time present
+ * CPU should be enabled in MADT, so there should be no
+ * disabled_cpus and num_processors won't change at runtime
+ * either.
+ */
+ if (disabled_cpus || num_online_cpus() != num_processors) {
+ pr_err("Unable to initialize the TDX module when there's offline CPU(s).\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ret = init_tdx_module();
+ if (ret == -ENODEV) {
+ pr_info("TDX module is not loaded.\n");
+ tdx_module_status = TDX_MODULE_NONE;
+ goto out;
+ }
+
+ /*
+ * Shut down the TDX module in case of any error during the
+ * initialization process. It's meaningless to leave the TDX
+ * module in any middle state of the initialization process.
+ *
+ * Shutting down the module also requires doing SEAMCALL on all
+ * MADT-enabled CPUs. Do it while CPU hotplug is disabled.
+ *
+ * Return all errors during the initialization as -EFAULT as the
+ * module is always shut down.
+ */
+ if (ret) {
+ pr_info("Failed to initialize TDX module. Shut it down.\n");
+ shutdown_tdx_module();
+ tdx_module_status = TDX_MODULE_SHUTDOWN;
+ ret = -EFAULT;
+ goto out;
+ }
+
+ pr_info("TDX module initialized.\n");
+ tdx_module_status = TDX_MODULE_INITIALIZED;
+out:
+ cpus_read_unlock();
+
+ return ret;
+}
+
+/**
+ * tdx_enable - Enable TDX by initializing the TDX module
+ *
+ * Caller to make sure all CPUs are online and in VMX operation before
+ * calling this function. CPU hotplug is temporarily disabled internally
+ * to prevent any cpu from going offline.
+ *
+ * This function can be called in parallel by multiple callers.
+ *
+ * Return:
+ *
+ * * 0: The TDX module has been successfully initialized.
+ * * -ENODEV: The TDX module is not loaded, or TDX is not supported.
+ * * -EINVAL: The TDX module cannot be initialized due to certain
+ * conditions are not met (i.e. when not all MADT-enabled
+ * CPUs are not online).
+ * * -EFAULT: Other internal fatal errors, or the TDX module is in
+ * shutdown mode due to it failed to initialize in previous
+ * attempts.
+ */
+int tdx_enable(void)
+{
+ int ret;
+
+ if (!platform_tdx_enabled())
+ return -ENODEV;
+
+ mutex_lock(&tdx_module_lock);
+
+ switch (tdx_module_status) {
+ case TDX_MODULE_UNKNOWN:
+ ret = __tdx_enable();
+ break;
+ case TDX_MODULE_NONE:
+ ret = -ENODEV;
+ break;
+ case TDX_MODULE_INITIALIZED:
+ ret = 0;
+ break;
+ default:
+ WARN_ON_ONCE(tdx_module_status != TDX_MODULE_SHUTDOWN);
+ ret = -EFAULT;
+ break;
+ }
+
+ mutex_unlock(&tdx_module_lock);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(tdx_enable);
--
2.37.3
TDX supports shutting down the TDX module at any time during its
lifetime. After the module is shut down, no further TDX module SEAMCALL
leaf functions can be made to the module on any logical cpu.
Shut down the TDX module in case of any error during the initialization
process. It's pointless to leave the TDX module in some middle state.
Shutting down the TDX module requires calling TDH.SYS.LP.SHUTDOWN on all
BIOS-enabled CPUs, and the SEMACALL can run concurrently on different
CPUs. Implement a mechanism to run SEAMCALL concurrently on all online
CPUs and use it to shut down the module. Later logical-cpu scope module
initialization will use it too.
Reviewed-by: Isaku Yamahata <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
v5 -> v6:
- Removed the seamcall() wrapper to previous patch (Dave).
- v3 -> v5 (no feedback on v4):
- Added a wrapper of __seamcall() to print error code if SEAMCALL fails.
- Made the seamcall_on_each_cpu() void.
- Removed 'seamcall_ret' and 'tdx_module_out' from
'struct seamcall_ctx', as they must be local variable.
- Added the comments to tdx_init() and one paragraph to changelog to
explain the caller should handle VMXON.
- Called out after shut down, no "TDX module" SEAMCALL can be made.
---
arch/x86/virt/vmx/tdx/tdx.c | 43 +++++++++++++++++++++++++++++++++----
arch/x86/virt/vmx/tdx/tdx.h | 5 +++++
2 files changed, 44 insertions(+), 4 deletions(-)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index f1154ef15549..5246335abe07 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -15,6 +15,8 @@
#include <linux/mutex.h>
#include <linux/cpu.h>
#include <linux/cpumask.h>
+#include <linux/smp.h>
+#include <linux/atomic.h>
#include <asm/msr-index.h>
#include <asm/msr.h>
#include <asm/apic.h>
@@ -267,15 +269,27 @@ bool platform_tdx_enabled(void)
return !!tdx_keyid_num;
}
+/*
+ * Data structure to make SEAMCALL on multiple CPUs concurrently.
+ * @err is set to -EFAULT when SEAMCALL fails on any cpu.
+ */
+struct seamcall_ctx {
+ u64 fn;
+ u64 rcx;
+ u64 rdx;
+ u64 r8;
+ u64 r9;
+ atomic_t err;
+};
+
/*
* Wrapper of __seamcall() to convert SEAMCALL leaf function error code
* to kernel error code. @seamcall_ret and @out contain the SEAMCALL
* leaf function return code and the additional output respectively if
* not NULL.
*/
-static int __always_unused seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
- u64 *seamcall_ret,
- struct tdx_module_output *out)
+static int seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
+ u64 *seamcall_ret, struct tdx_module_output *out)
{
u64 sret;
@@ -309,6 +323,25 @@ static int __always_unused seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
}
}
+static void seamcall_smp_call_function(void *data)
+{
+ struct seamcall_ctx *sc = data;
+ int ret;
+
+ ret = seamcall(sc->fn, sc->rcx, sc->rdx, sc->r8, sc->r9, NULL, NULL);
+ if (ret)
+ atomic_set(&sc->err, -EFAULT);
+}
+
+/*
+ * Call the SEAMCALL on all online CPUs concurrently. Caller to check
+ * @sc->err to determine whether any SEAMCALL failed on any cpu.
+ */
+static void seamcall_on_each_cpu(struct seamcall_ctx *sc)
+{
+ on_each_cpu(seamcall_smp_call_function, sc, true);
+}
+
/*
* Detect and initialize the TDX module.
*
@@ -324,7 +357,9 @@ static int init_tdx_module(void)
static void shutdown_tdx_module(void)
{
- /* TODO: Shut down the TDX module */
+ struct seamcall_ctx sc = { .fn = TDH_SYS_LP_SHUTDOWN };
+
+ seamcall_on_each_cpu(&sc);
}
static int __tdx_enable(void)
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index 92a8de957dc7..215cc1065d78 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -12,6 +12,11 @@
/* MSR to report KeyID partitioning between MKTME and TDX */
#define MSR_IA32_MKTME_KEYID_PARTITIONING 0x00000087
+/*
+ * TDX module SEAMCALL leaf functions
+ */
+#define TDH_SYS_LP_SHUTDOWN 44
+
/*
* Do not put any hardware-defined TDX structure representations below
* this comment!
--
2.37.3
So far the TDX module hasn't been detected yet. __seamcall() returns
TDX_SEAMCALL_VMFAILINVALID when the target SEAM software is not loaded.
loaded. Just use __seamcall() to detect the TDX module.
The first step of initializing the module is to call TDH.SYS.INIT once
on any logical cpu to do module global initialization. Do the module
global initialization and detect the TDX module.
Signed-off-by: Kai Huang <[email protected]>
---
arch/x86/virt/vmx/tdx/tdx.c | 19 +++++++++++++++++--
arch/x86/virt/vmx/tdx/tdx.h | 1 +
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 5246335abe07..68fb9bc201d6 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -351,8 +351,23 @@ static void seamcall_on_each_cpu(struct seamcall_ctx *sc)
*/
static int init_tdx_module(void)
{
- /* The TDX module hasn't been detected */
- return -ENODEV;
+ int ret;
+
+ /*
+ * Call TDH.SYS.INIT to do the global initialization of
+ * the TDX module. It also detects the module.
+ */
+ ret = seamcall(TDH_SYS_INIT, 0, 0, 0, 0, NULL, NULL);
+ if (ret)
+ goto out;
+
+ /*
+ * Return -EINVAL until all steps of TDX module initialization
+ * process are done.
+ */
+ ret = -EINVAL;
+out:
+ return ret;
}
static void shutdown_tdx_module(void)
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index 215cc1065d78..0b415805c921 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -15,6 +15,7 @@
/*
* TDX module SEAMCALL leaf functions
*/
+#define TDH_SYS_INIT 33
#define TDH_SYS_LP_SHUTDOWN 44
/*
--
2.37.3
After the global module initialization, the next step is logical-cpu
scope module initialization. Logical-cpu initialization requires
calling TDH.SYS.LP.INIT on all BIOS-enabled CPUs. This SEAMCALL can run
concurrently on all CPUs.
Use the helper introduced for shutting down the module to do logical-cpu
scope initialization.
Signed-off-by: Kai Huang <[email protected]>
---
arch/x86/virt/vmx/tdx/tdx.c | 15 +++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 1 +
2 files changed, 16 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 68fb9bc201d6..8a1c98d961f3 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -342,6 +342,15 @@ static void seamcall_on_each_cpu(struct seamcall_ctx *sc)
on_each_cpu(seamcall_smp_call_function, sc, true);
}
+static int tdx_module_init_cpus(void)
+{
+ struct seamcall_ctx sc = { .fn = TDH_SYS_LP_INIT };
+
+ seamcall_on_each_cpu(&sc);
+
+ return atomic_read(&sc.err);
+}
+
/*
* Detect and initialize the TDX module.
*
@@ -361,6 +370,12 @@ static int init_tdx_module(void)
if (ret)
goto out;
+ /* Logical-cpu scope initialization */
+ ret = tdx_module_init_cpus();
+ if (ret)
+ goto out;
+
+
/*
* Return -EINVAL until all steps of TDX module initialization
* process are done.
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index 0b415805c921..9ba11808bd45 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -16,6 +16,7 @@
* TDX module SEAMCALL leaf functions
*/
#define TDH_SYS_INIT 33
+#define TDH_SYS_LP_INIT 35
#define TDH_SYS_LP_SHUTDOWN 44
/*
--
2.37.3
TDX introduces a new CPU mode: Secure Arbitration Mode (SEAM). This
mode runs only the TDX module itself or other code to load the TDX
module.
The host kernel communicates with SEAM software via a new SEAMCALL
instruction. This is conceptually similar to a guest->host hypercall,
except it is made from the host to SEAM software instead.
The TDX module defines a set of SEAMCALL leaf functions to allow the
host to initialize it, and to create and run protected VMs. SEAMCALL
leaf functions use an ABI different from the x86-64 system-v ABI.
Instead, they share the same ABI with the TDCALL leaf functions.
Implement a function __seamcall() to allow the host to make SEAMCALL
to SEAM software using the TDX_MODULE_CALL macro which is the common
assembly for both SEAMCALL and TDCALL.
SEAMCALL instruction causes #GP when SEAMRR isn't enabled, and #UD when
CPU is not in VMX operation. The current TDX_MODULE_CALL macro doesn't
handle any of them. There's no way to check whether the CPU is in VMX
operation or not.
Initializing the TDX module is done at runtime on demand, and it depends
on the caller to ensure CPU is in VMX operation before making SEAMCALL.
To avoid getting Oops when the caller mistakenly tries to initialize the
TDX module when CPU is not in VMX operation, extend the TDX_MODULE_CALL
macro to handle #UD (and also #GP, which can theoretically still happen
when TDX isn't actually enabled by the BIOS, i.e. due to BIOS bug).
Introduce two new TDX error codes for #UD and #GP respectively so the
caller can distinguish. Also, Opportunistically put the new TDX error
codes and the existing TDX_SEAMCALL_VMFAILINVALID into INTEL_TDX_HOST
Kconfig option as they are only used when it is on.
As __seamcall() can potentially return multiple error codes, besides the
actual SEAMCALL leaf function return code, also introduce a wrapper
function seamcall() to convert the __seamcall() error code to the kernel
error code, so the caller doesn't need to duplicate the code to check
return value of __seamcall() and return kernel error code accordingly.
Signed-off-by: Kai Huang <[email protected]>
---
v5 -> v6:
- Added code to handle #UD and #GP (Dave).
- Moved the seamcall() wrapper function to this patch, and used a
temporary __always_unused to avoid compile warning (Dave).
- v3 -> v5 (no feedback on v4):
- Explicitly tell TDX_SEAMCALL_VMFAILINVALID is returned if the
SEAMCALL itself fails.
- Improve the changelog.
---
arch/x86/include/asm/tdx.h | 9 ++++++
arch/x86/virt/vmx/tdx/Makefile | 2 +-
arch/x86/virt/vmx/tdx/seamcall.S | 52 ++++++++++++++++++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.c | 42 ++++++++++++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 8 +++++
arch/x86/virt/vmx/tdx/tdxcall.S | 19 ++++++++++--
6 files changed, 129 insertions(+), 3 deletions(-)
create mode 100644 arch/x86/virt/vmx/tdx/seamcall.S
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 80c76b426adf..d568f17da742 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -8,6 +8,10 @@
#include <asm/ptrace.h>
#include <asm/shared/tdx.h>
+#ifdef CONFIG_INTEL_TDX_HOST
+
+#include <asm/trapnr.h>
+
/*
* SW-defined error codes.
*
@@ -18,6 +22,11 @@
#define TDX_SW_ERROR (TDX_ERROR | GENMASK_ULL(47, 40))
#define TDX_SEAMCALL_VMFAILINVALID (TDX_SW_ERROR | _UL(0xFFFF0000))
+#define TDX_SEAMCALL_GP (TDX_SW_ERROR | X86_TRAP_GP)
+#define TDX_SEAMCALL_UD (TDX_SW_ERROR | X86_TRAP_UD)
+
+#endif
+
#ifndef __ASSEMBLY__
#include <asm/pgtable_types.h>
diff --git a/arch/x86/virt/vmx/tdx/Makefile b/arch/x86/virt/vmx/tdx/Makefile
index 93ca8b73e1f1..38d534f2c113 100644
--- a/arch/x86/virt/vmx/tdx/Makefile
+++ b/arch/x86/virt/vmx/tdx/Makefile
@@ -1,2 +1,2 @@
# SPDX-License-Identifier: GPL-2.0-only
-obj-y += tdx.o
+obj-y += tdx.o seamcall.o
diff --git a/arch/x86/virt/vmx/tdx/seamcall.S b/arch/x86/virt/vmx/tdx/seamcall.S
new file mode 100644
index 000000000000..f81be6b9c133
--- /dev/null
+++ b/arch/x86/virt/vmx/tdx/seamcall.S
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/linkage.h>
+#include <asm/frame.h>
+
+#include "tdxcall.S"
+
+/*
+ * __seamcall() - Host-side interface functions to SEAM software module
+ * (the P-SEAMLDR or the TDX module).
+ *
+ * Transform function call register arguments into the SEAMCALL register
+ * ABI. Return TDX_SEAMCALL_VMFAILINVALID if the SEAMCALL itself fails,
+ * or the completion status of the SEAMCALL leaf function. Additional
+ * output operands are saved in @out (if it is provided by the caller).
+ *
+ *-------------------------------------------------------------------------
+ * SEAMCALL ABI:
+ *-------------------------------------------------------------------------
+ * Input Registers:
+ *
+ * RAX - SEAMCALL Leaf number.
+ * RCX,RDX,R8-R9 - SEAMCALL Leaf specific input registers.
+ *
+ * Output Registers:
+ *
+ * RAX - SEAMCALL completion status code.
+ * RCX,RDX,R8-R11 - SEAMCALL Leaf specific output registers.
+ *
+ *-------------------------------------------------------------------------
+ *
+ * __seamcall() function ABI:
+ *
+ * @fn (RDI) - SEAMCALL Leaf number, moved to RAX
+ * @rcx (RSI) - Input parameter 1, moved to RCX
+ * @rdx (RDX) - Input parameter 2, moved to RDX
+ * @r8 (RCX) - Input parameter 3, moved to R8
+ * @r9 (R8) - Input parameter 4, moved to R9
+ *
+ * @out (R9) - struct tdx_module_output pointer
+ * stored temporarily in R12 (not
+ * used by the P-SEAMLDR or the TDX
+ * module). It can be NULL.
+ *
+ * Return (via RAX) the completion status of the SEAMCALL, or
+ * TDX_SEAMCALL_VMFAILINVALID.
+ */
+SYM_FUNC_START(__seamcall)
+ FRAME_BEGIN
+ TDX_MODULE_CALL host=1
+ FRAME_END
+ RET
+SYM_FUNC_END(__seamcall)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index a137350d5d0e..f1154ef15549 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -267,6 +267,48 @@ bool platform_tdx_enabled(void)
return !!tdx_keyid_num;
}
+/*
+ * Wrapper of __seamcall() to convert SEAMCALL leaf function error code
+ * to kernel error code. @seamcall_ret and @out contain the SEAMCALL
+ * leaf function return code and the additional output respectively if
+ * not NULL.
+ */
+static int __always_unused seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
+ u64 *seamcall_ret,
+ struct tdx_module_output *out)
+{
+ u64 sret;
+
+ sret = __seamcall(fn, rcx, rdx, r8, r9, out);
+
+ /* Save SEAMCALL return code if caller wants it */
+ if (seamcall_ret)
+ *seamcall_ret = sret;
+
+ /* SEAMCALL was successful */
+ if (!sret)
+ return 0;
+
+ switch (sret) {
+ case TDX_SEAMCALL_GP:
+ /*
+ * platform_tdx_enabled() is checked to be true
+ * before making any SEAMCALL.
+ */
+ WARN_ON_ONCE(1);
+ fallthrough;
+ case TDX_SEAMCALL_VMFAILINVALID:
+ /* Return -ENODEV if the TDX module is not loaded. */
+ return -ENODEV;
+ case TDX_SEAMCALL_UD:
+ /* Return -EINVAL if CPU isn't in VMX operation. */
+ return -EINVAL;
+ default:
+ /* Return -EIO if the actual SEAMCALL leaf failed. */
+ return -EIO;
+ }
+}
+
/*
* Detect and initialize the TDX module.
*
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index d00074abcb20..92a8de957dc7 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -12,4 +12,12 @@
/* MSR to report KeyID partitioning between MKTME and TDX */
#define MSR_IA32_MKTME_KEYID_PARTITIONING 0x00000087
+/*
+ * Do not put any hardware-defined TDX structure representations below
+ * this comment!
+ */
+
+struct tdx_module_output;
+u64 __seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
+ struct tdx_module_output *out);
#endif
diff --git a/arch/x86/virt/vmx/tdx/tdxcall.S b/arch/x86/virt/vmx/tdx/tdxcall.S
index 49a54356ae99..757b0c34be10 100644
--- a/arch/x86/virt/vmx/tdx/tdxcall.S
+++ b/arch/x86/virt/vmx/tdx/tdxcall.S
@@ -1,6 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0 */
#include <asm/asm-offsets.h>
#include <asm/tdx.h>
+#include <asm/asm.h>
/*
* TDCALL and SEAMCALL are supported in Binutils >= 2.36.
@@ -45,6 +46,7 @@
/* Leave input param 2 in RDX */
.if \host
+1:
seamcall
/*
* SEAMCALL instruction is essentially a VMExit from VMX root
@@ -57,10 +59,23 @@
* This value will never be used as actual SEAMCALL error code as
* it is from the Reserved status code class.
*/
- jnc .Lno_vmfailinvalid
+ jnc .Lseamcall_out
mov $TDX_SEAMCALL_VMFAILINVALID, %rax
-.Lno_vmfailinvalid:
+ jmp .Lseamcall_out
+2:
+ /*
+ * SEAMCALL caused #GP or #UD. By reaching here %eax contains
+ * the trap number. Convert the trap number to the TDX error
+ * code by setting TDX_SW_ERROR to the high 32-bits of %rax.
+ *
+ * Note cannot OR TDX_SW_ERROR directly to %rax as OR instruction
+ * only accepts 32-bit immediate at most.
+ */
+ mov $TDX_SW_ERROR, %r12
+ orq %r12, %rax
+ _ASM_EXTABLE_FAULT(1b, 2b)
+.Lseamcall_out:
.else
tdcall
.endif
--
2.37.3
After the array of TDMRs and the global KeyID are configured to the TDX
module, use TDH.SYS.KEY.CONFIG to configure the key of the global KeyID
on all packages.
TDH.SYS.KEY.CONFIG must be done on one (any) cpu for each package. And
it cannot run concurrently on different CPUs. Implement a helper to
run SEAMCALL on one cpu for each package one by one, and use it to
configure the global KeyID on all packages.
Intel hardware doesn't guarantee cache coherency across different
KeyIDs. The kernel needs to flush PAMT's dirty cachelines (associated
with KeyID 0) before the TDX module uses the global KeyID to access the
PAMT. Following the TDX module specification, flush cache before
configuring the global KeyID on all packages.
Given the PAMT size can be large (~1/256th of system RAM), just use
WBINVD on all CPUs to flush.
Reviewed-by: Isaku Yamahata <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
arch/x86/virt/vmx/tdx/tdx.c | 83 ++++++++++++++++++++++++++++++++++++-
arch/x86/virt/vmx/tdx/tdx.h | 1 +
2 files changed, 82 insertions(+), 2 deletions(-)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index fdfce715dda6..9cfb01e7666a 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -354,6 +354,46 @@ static void seamcall_on_each_cpu(struct seamcall_ctx *sc)
on_each_cpu(seamcall_smp_call_function, sc, true);
}
+/*
+ * Call one SEAMCALL on one (any) cpu for each physical package in
+ * serialized way. Return immediately in case of any error if
+ * SEAMCALL fails on any cpu.
+ *
+ * Note for serialized calls 'struct seamcall_ctx::err' doesn't have
+ * to be atomic, but for simplicity just reuse it instead of adding
+ * a new one.
+ */
+static int seamcall_on_each_package_serialized(struct seamcall_ctx *sc)
+{
+ cpumask_var_t packages;
+ int cpu, ret = 0;
+
+ if (!zalloc_cpumask_var(&packages, GFP_KERNEL))
+ return -ENOMEM;
+
+ for_each_online_cpu(cpu) {
+ if (cpumask_test_and_set_cpu(topology_physical_package_id(cpu),
+ packages))
+ continue;
+
+ ret = smp_call_function_single(cpu, seamcall_smp_call_function,
+ sc, true);
+ if (ret)
+ break;
+
+ /*
+ * Doesn't have to use atomic_read(), but it doesn't
+ * hurt either.
+ */
+ ret = atomic_read(&sc->err);
+ if (ret)
+ break;
+ }
+
+ free_cpumask_var(packages);
+ return ret;
+}
+
static int tdx_module_init_cpus(void)
{
struct seamcall_ctx sc = { .fn = TDH_SYS_LP_INIT };
@@ -1096,6 +1136,21 @@ static int config_tdx_module(struct tdmr_info *tdmr_array, int tdmr_num,
return ret;
}
+static int config_global_keyid(void)
+{
+ struct seamcall_ctx sc = { .fn = TDH_SYS_KEY_CONFIG };
+
+ /*
+ * Configure the key of the global KeyID on all packages by
+ * calling TDH.SYS.KEY.CONFIG on all packages.
+ *
+ * TDH.SYS.KEY.CONFIG may fail with entropy error (which is
+ * a recoverable error). Assume this is exceedingly rare and
+ * just return error if encountered instead of retrying.
+ */
+ return seamcall_on_each_package_serialized(&sc);
+}
+
/*
* Detect and initialize the TDX module.
*
@@ -1159,15 +1214,39 @@ static int init_tdx_module(void)
if (ret)
goto out_free_pamts;
+ /*
+ * Hardware doesn't guarantee cache coherency across different
+ * KeyIDs. The kernel needs to flush PAMT's dirty cachelines
+ * (associated with KeyID 0) before the TDX module can use the
+ * global KeyID to access the PAMT. Given PAMTs are potentially
+ * large (~1/256th of system RAM), just use WBINVD on all cpus
+ * to flush the cache.
+ *
+ * Follow the TDX spec to flush cache before configuring the
+ * global KeyID on all packages.
+ */
+ wbinvd_on_all_cpus();
+
+ /* Config the key of global KeyID on all packages */
+ ret = config_global_keyid();
+ if (ret)
+ goto out_free_pamts;
+
/*
* Return -EINVAL until all steps of TDX module initialization
* process are done.
*/
ret = -EINVAL;
out_free_pamts:
- if (ret)
+ if (ret) {
+ /*
+ * Part of PAMT may already have been initialized by
+ * TDX module. Flush cache before returning PAMT back
+ * to the kernel.
+ */
+ wbinvd_on_all_cpus();
tdmrs_free_pamt_all(tdmr_array, tdmr_num);
- else
+ } else
pr_info("%lu pages allocated for PAMT.\n",
tdmrs_count_pamt_pages(tdmr_array, tdmr_num));
out_free_tdmrs:
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index c26bab2555ca..768d097412ab 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -15,6 +15,7 @@
/*
* TDX module SEAMCALL leaf functions
*/
+#define TDH_SYS_KEY_CONFIG 31
#define TDH_SYS_INFO 32
#define TDH_SYS_INIT 33
#define TDH_SYS_LP_INIT 35
--
2.37.3
TDX provides increased levels of memory confidentiality and integrity.
This requires special hardware support for features like memory
encryption and storage of memory integrity checksums. Not all memory
satisfies these requirements.
As a result, TDX introduced the concept of a "Convertible Memory Region"
(CMR). During boot, the firmware builds a list of all of the memory
ranges which can provide the TDX security guarantees. The list of these
ranges, along with TDX module information, is available to the kernel by
querying the TDX module via TDH.SYS.INFO SEAMCALL.
The host kernel can choose whether or not to use all convertible memory
regions as TDX-usable memory. Before the TDX module is ready to create
any TDX guests, the kernel needs to configure the TDX-usable memory
regions by passing an array of "TD Memory Regions" (TDMRs) to the TDX
module. Constructing the TDMR array requires information of both the
TDX module (TDSYSINFO_STRUCT) and the Convertible Memory Regions. Call
TDH.SYS.INFO to get this information as preparation.
Use static variables for both TDSYSINFO_STRUCT and CMR array to avoid
having to pass them as function arguments when constructing the TDMR
array. And they are too big to be put to the stack anyway. Also, KVM
needs to use the TDSYSINFO_STRUCT to create TDX guests.
Reviewed-by: Isaku Yamahata <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
v5 -> v6:
- Added to also print TDX module's attribute (Isaku).
- Removed all arguments in tdx_gete_sysinfo() to use static variables
of 'tdx_sysinfo' and 'tdx_cmr_array' directly as they are all used
directly in other functions in later patches.
- Added Isaku's Reviewed-by.
- v3 -> v5 (no feedback on v4):
- Renamed sanitize_cmrs() to check_cmrs().
- Removed unnecessary sanity check against tdx_sysinfo and tdx_cmr_array
actual size returned by TDH.SYS.INFO.
- Changed -EFAULT to -EINVAL in couple places.
- Added comments around tdx_sysinfo and tdx_cmr_array saying they are
used by TDH.SYS.INFO ABI.
- Changed to pass 'tdx_sysinfo' and 'tdx_cmr_array' as function
arguments in tdx_get_sysinfo().
- Changed to only print BIOS-CMR when check_cmrs() fails.
---
arch/x86/virt/vmx/tdx/tdx.c | 135 ++++++++++++++++++++++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 61 ++++++++++++++++
2 files changed, 196 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 8a1c98d961f3..7d7205615873 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -54,6 +54,11 @@ static enum tdx_module_status_t tdx_module_status;
/* Prevent concurrent attempts on TDX detection and initialization */
static DEFINE_MUTEX(tdx_module_lock);
+/* Below two are used in TDH.SYS.INFO SEAMCALL ABI */
+static struct tdsysinfo_struct tdx_sysinfo;
+static struct cmr_info tdx_cmr_array[MAX_CMRS] __aligned(CMR_INFO_ARRAY_ALIGNMENT);
+static int tdx_cmr_num;
+
/*
* Detect TDX private KeyIDs to see whether TDX has been enabled by the
* BIOS. Both initializing the TDX module and running TDX guest require
@@ -351,6 +356,133 @@ static int tdx_module_init_cpus(void)
return atomic_read(&sc.err);
}
+static inline bool cmr_valid(struct cmr_info *cmr)
+{
+ return !!cmr->size;
+}
+
+static void print_cmrs(struct cmr_info *cmr_array, int cmr_num,
+ const char *name)
+{
+ int i;
+
+ for (i = 0; i < cmr_num; i++) {
+ struct cmr_info *cmr = &cmr_array[i];
+
+ pr_info("%s : [0x%llx, 0x%llx)\n", name,
+ cmr->base, cmr->base + cmr->size);
+ }
+}
+
+/*
+ * Check the CMRs reported by TDH.SYS.INFO and update the actual number
+ * of CMRs. The CMRs returned by the TDH.SYS.INFO may contain invalid
+ * CMRs after the last valid CMR, but there should be no invalid CMRs
+ * between two valid CMRs. Check and update the actual number of CMRs
+ * number by dropping all tail empty CMRs.
+ */
+static int check_cmrs(struct cmr_info *cmr_array, int *actual_cmr_num)
+{
+ int cmr_num = *actual_cmr_num;
+ int i, j;
+
+ /*
+ * Intel TDX module spec, 20.7.3 CMR_INFO:
+ *
+ * TDH.SYS.INFO leaf function returns a MAX_CMRS (32) entry
+ * array of CMR_INFO entries. The CMRs are sorted from the
+ * lowest base address to the highest base address, and they
+ * are non-overlapping.
+ *
+ * This implies that BIOS may generate invalid empty entries
+ * if total CMRs are less than 32. Skip them manually.
+ */
+ for (i = 0; i < cmr_num; i++) {
+ struct cmr_info *cmr = &cmr_array[i];
+ struct cmr_info *prev_cmr = NULL;
+
+ /* Skip further invalid CMRs */
+ if (!cmr_valid(cmr))
+ break;
+
+ if (i > 0)
+ prev_cmr = &cmr_array[i - 1];
+
+ /*
+ * It is a TDX firmware bug if CMRs are not
+ * in address ascending order.
+ */
+ if (prev_cmr && ((prev_cmr->base + prev_cmr->size) >
+ cmr->base)) {
+ print_cmrs(cmr_array, cmr_num, "BIOS-CMR");
+ pr_err("Firmware bug: CMRs not in address ascending order.\n");
+ return -EINVAL;
+ }
+ }
+
+ /*
+ * Also a sane BIOS should never generate invalid CMR(s) between
+ * two valid CMRs. Sanity check this and simply return error in
+ * this case.
+ *
+ * By reaching here @i is the index of the first invalid CMR (or
+ * cmr_num). Starting with the next entry of @i since it has
+ * already been checked.
+ */
+ for (j = i + 1; j < cmr_num; j++) {
+ if (cmr_valid(&cmr_array[j])) {
+ print_cmrs(cmr_array, cmr_num, "BIOS-CMR");
+ pr_err("Firmware bug: invalid CMR(s) before valid CMRs.\n");
+ return -EINVAL;
+ }
+ }
+
+ /*
+ * Trim all tail invalid empty CMRs. BIOS should generate at
+ * least one valid CMR, otherwise it's a TDX firmware bug.
+ */
+ if (i == 0) {
+ print_cmrs(cmr_array, cmr_num, "BIOS-CMR");
+ pr_err("Firmware bug: No valid CMR.\n");
+ return -EINVAL;
+ }
+
+ /* Update the actual number of CMRs */
+ *actual_cmr_num = i;
+
+ /* Print kernel checked CMRs */
+ print_cmrs(cmr_array, *actual_cmr_num, "Kernel-checked-CMR");
+
+ return 0;
+}
+
+static int tdx_get_sysinfo(void)
+{
+ struct tdx_module_output out;
+ int ret;
+
+ BUILD_BUG_ON(sizeof(struct tdsysinfo_struct) != TDSYSINFO_STRUCT_SIZE);
+
+ ret = seamcall(TDH_SYS_INFO, __pa(&tdx_sysinfo), TDSYSINFO_STRUCT_SIZE,
+ __pa(tdx_cmr_array), MAX_CMRS, NULL, &out);
+ if (ret)
+ return ret;
+
+ /* R9 contains the actual entries written the CMR array. */
+ tdx_cmr_num = out.r9;
+
+ pr_info("TDX module: atributes 0x%x, vendor_id 0x%x, major_version %u, minor_version %u, build_date %u, build_num %u",
+ tdx_sysinfo.attributes, tdx_sysinfo.vendor_id,
+ tdx_sysinfo.major_version, tdx_sysinfo.minor_version,
+ tdx_sysinfo.build_date, tdx_sysinfo.build_num);
+
+ /*
+ * check_cmrs() updates the actual number of CMRs by dropping all
+ * tail invalid CMRs.
+ */
+ return check_cmrs(tdx_cmr_array, &tdx_cmr_num);
+}
+
/*
* Detect and initialize the TDX module.
*
@@ -375,6 +507,9 @@ static int init_tdx_module(void)
if (ret)
goto out;
+ ret = tdx_get_sysinfo();
+ if (ret)
+ goto out;
/*
* Return -EINVAL until all steps of TDX module initialization
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index 9ba11808bd45..8e273756098c 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -15,10 +15,71 @@
/*
* TDX module SEAMCALL leaf functions
*/
+#define TDH_SYS_INFO 32
#define TDH_SYS_INIT 33
#define TDH_SYS_LP_INIT 35
#define TDH_SYS_LP_SHUTDOWN 44
+struct cmr_info {
+ u64 base;
+ u64 size;
+} __packed;
+
+#define MAX_CMRS 32
+#define CMR_INFO_ARRAY_ALIGNMENT 512
+
+struct cpuid_config {
+ u32 leaf;
+ u32 sub_leaf;
+ u32 eax;
+ u32 ebx;
+ u32 ecx;
+ u32 edx;
+} __packed;
+
+#define TDSYSINFO_STRUCT_SIZE 1024
+#define TDSYSINFO_STRUCT_ALIGNMENT 1024
+
+struct tdsysinfo_struct {
+ /* TDX-SEAM Module Info */
+ u32 attributes;
+ u32 vendor_id;
+ u32 build_date;
+ u16 build_num;
+ u16 minor_version;
+ u16 major_version;
+ u8 reserved0[14];
+ /* Memory Info */
+ u16 max_tdmrs;
+ u16 max_reserved_per_tdmr;
+ u16 pamt_entry_size;
+ u8 reserved1[10];
+ /* Control Struct Info */
+ u16 tdcs_base_size;
+ u8 reserved2[2];
+ u16 tdvps_base_size;
+ u8 tdvps_xfam_dependent_size;
+ u8 reserved3[9];
+ /* TD Capabilities */
+ u64 attributes_fixed0;
+ u64 attributes_fixed1;
+ u64 xfam_fixed0;
+ u64 xfam_fixed1;
+ u8 reserved4[32];
+ u32 num_cpuid_config;
+ /*
+ * The actual number of CPUID_CONFIG depends on above
+ * 'num_cpuid_config'. The size of 'struct tdsysinfo_struct'
+ * is 1024B defined by TDX architecture. Use a union with
+ * specific padding to make 'sizeof(struct tdsysinfo_struct)'
+ * equal to 1024.
+ */
+ union {
+ struct cpuid_config cpuid_configs[0];
+ u8 reserved5[892];
+ };
+} __packed __aligned(TDSYSINFO_STRUCT_ALIGNMENT);
+
/*
* Do not put any hardware-defined TDX structure representations below
* this comment!
--
2.37.3
Intel Trust Domain Extensions (TDX) protects guest VMs from malicious
host and certain physical attacks. A CPU-attested software module
called 'the TDX module' runs inside a new isolated memory range as a
trusted hypervisor to manage and run protected VMs.
Pre-TDX Intel hardware has support for a memory encryption architecture
called MKTME. The memory encryption hardware underpinning MKTME is also
used for Intel TDX. TDX ends up "stealing" some of the physical address
space from the MKTME architecture for crypto-protection to VMs. The
BIOS is responsible for partitioning the "KeyID" space between legacy
MKTME and TDX. The KeyIDs reserved for TDX are called 'TDX private
KeyIDs' or 'TDX KeyIDs' for short.
TDX doesn't trust the BIOS. During machine boot, TDX verifies the TDX
private KeyIDs are consistently and correctly programmed by the BIOS
across all CPU packages before it enables TDX on any CPU core. A valid
TDX private KeyID range on BSP indicates TDX has been enabled by the
BIOS, otherwise the BIOS is buggy.
The TDX module is expected to be loaded by the BIOS when it enables TDX,
but the kernel needs to properly initialize it before it can be used to
create and run any TDX guests. The TDX module will be initialized at
runtime by the user (i.e. KVM) on demand.
Add a new early_initcall(tdx_init) to do TDX early boot initialization.
Only detect TDX private KeyIDs for now. Some other early checks will
follow up. Also add a new function to report whether TDX has been
enabled by BIOS (TDX private KeyID range is valid). Kexec() will also
need it to determine whether need to flush dirty cachelines that are
associated with any TDX private KeyIDs before booting to the new kernel.
To start to support TDX, create a new arch/x86/virt/vmx/tdx/tdx.c for
TDX host kernel support. Add a new Kconfig option CONFIG_INTEL_TDX_HOST
to opt-in TDX host kernel support (to distinguish with TDX guest kernel
support). So far only KVM is the only user of TDX. Make the new config
option depend on KVM_INTEL.
Reviewed-by: Kirill A. Shutemov <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
v5 -> v6:
- Removed SEAMRR detection to make code simpler.
- Removed the 'default N' in the KVM_TDX_HOST Kconfig (Kirill).
- Changed to use 'obj-y' in arch/x86/virt/vmx/tdx/Makefile (Kirill).
---
arch/x86/Kconfig | 12 +++++
arch/x86/Makefile | 2 +
arch/x86/include/asm/tdx.h | 7 +++
arch/x86/virt/Makefile | 2 +
arch/x86/virt/vmx/Makefile | 2 +
arch/x86/virt/vmx/tdx/Makefile | 2 +
arch/x86/virt/vmx/tdx/tdx.c | 95 ++++++++++++++++++++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 15 ++++++
8 files changed, 137 insertions(+)
create mode 100644 arch/x86/virt/Makefile
create mode 100644 arch/x86/virt/vmx/Makefile
create mode 100644 arch/x86/virt/vmx/tdx/Makefile
create mode 100644 arch/x86/virt/vmx/tdx/tdx.c
create mode 100644 arch/x86/virt/vmx/tdx/tdx.h
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index b52ad13f0f44..b9bd5d994ba7 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1955,6 +1955,18 @@ config X86_SGX
If unsure, say N.
+config INTEL_TDX_HOST
+ bool "Intel Trust Domain Extensions (TDX) host support"
+ depends on CPU_SUP_INTEL
+ depends on X86_64
+ depends on KVM_INTEL
+ help
+ Intel Trust Domain Extensions (TDX) protects guest VMs from malicious
+ host and certain physical attacks. This option enables necessary TDX
+ support in host kernel to run protected VMs.
+
+ If unsure, say N.
+
config EFI
bool "EFI runtime service support"
depends on ACPI
diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 1640e005092b..bba792a26f0e 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -252,6 +252,8 @@ archheaders:
libs-y += arch/x86/lib/
+core-y += arch/x86/virt/
+
# drivers-y are linked after core-y
drivers-$(CONFIG_MATH_EMULATION) += arch/x86/math-emu/
drivers-$(CONFIG_PCI) += arch/x86/pci/
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 1c166fb9c22f..9b63f33e9c91 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -120,5 +120,12 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,
return -ENODEV;
}
#endif /* CONFIG_INTEL_TDX_GUEST && CONFIG_KVM_GUEST */
+
+#ifdef CONFIG_INTEL_TDX_HOST
+bool platform_tdx_enabled(void);
+#else /* !CONFIG_INTEL_TDX_HOST */
+static inline bool platform_tdx_enabled(void) { return false; }
+#endif /* CONFIG_INTEL_TDX_HOST */
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASM_X86_TDX_H */
diff --git a/arch/x86/virt/Makefile b/arch/x86/virt/Makefile
new file mode 100644
index 000000000000..1e36502cd738
--- /dev/null
+++ b/arch/x86/virt/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-y += vmx/
diff --git a/arch/x86/virt/vmx/Makefile b/arch/x86/virt/vmx/Makefile
new file mode 100644
index 000000000000..feebda21d793
--- /dev/null
+++ b/arch/x86/virt/vmx/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_INTEL_TDX_HOST) += tdx/
diff --git a/arch/x86/virt/vmx/tdx/Makefile b/arch/x86/virt/vmx/tdx/Makefile
new file mode 100644
index 000000000000..93ca8b73e1f1
--- /dev/null
+++ b/arch/x86/virt/vmx/tdx/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-y += tdx.o
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
new file mode 100644
index 000000000000..982d9c453b6b
--- /dev/null
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright(c) 2022 Intel Corporation.
+ *
+ * Intel Trusted Domain Extensions (TDX) support
+ */
+
+#define pr_fmt(fmt) "tdx: " fmt
+
+#include <linux/types.h>
+#include <linux/init.h>
+#include <linux/printk.h>
+#include <asm/msr-index.h>
+#include <asm/msr.h>
+#include <asm/tdx.h>
+#include "tdx.h"
+
+static u32 tdx_keyid_start __ro_after_init;
+static u32 tdx_keyid_num __ro_after_init;
+
+/*
+ * Detect TDX private KeyIDs to see whether TDX has been enabled by the
+ * BIOS. Both initializing the TDX module and running TDX guest require
+ * TDX private KeyID.
+ *
+ * TDX doesn't trust BIOS. TDX verifies all configurations from BIOS
+ * are correct before enabling TDX on any core. TDX requires the BIOS
+ * to correctly and consistently program TDX private KeyIDs on all CPU
+ * packages. Unless there is a BIOS bug, detecting a valid TDX private
+ * KeyID range on BSP indicates TDX has been enabled by the BIOS. If
+ * there's such BIOS bug, it will be caught later when initializing the
+ * TDX module.
+ */
+static int __init detect_tdx(void)
+{
+ int ret;
+
+ /*
+ * IA32_MKTME_KEYID_PARTIONING:
+ * Bit [31:0]: Number of MKTME KeyIDs.
+ * Bit [63:32]: Number of TDX private KeyIDs.
+ */
+ ret = rdmsr_safe(MSR_IA32_MKTME_KEYID_PARTITIONING, &tdx_keyid_start,
+ &tdx_keyid_num);
+ if (ret)
+ return -ENODEV;
+
+ if (!tdx_keyid_num)
+ return -ENODEV;
+
+ /*
+ * KeyID 0 is for TME. MKTME KeyIDs start from 1. TDX private
+ * KeyIDs start after the last MKTME KeyID.
+ */
+ tdx_keyid_start++;
+
+ pr_info("TDX enabled by BIOS. TDX private KeyID range: [%u, %u)\n",
+ tdx_keyid_start, tdx_keyid_start + tdx_keyid_num);
+
+ return 0;
+}
+
+static void __init clear_tdx(void)
+{
+ tdx_keyid_start = tdx_keyid_num = 0;
+}
+
+static int __init tdx_init(void)
+{
+ if (detect_tdx())
+ return -ENODEV;
+
+ /*
+ * Initializing the TDX module requires one TDX private KeyID.
+ * If there's only one TDX KeyID then after module initialization
+ * KVM won't be able to run any TDX guest, which makes the whole
+ * thing worthless. Just disable TDX in this case.
+ */
+ if (tdx_keyid_num < 2) {
+ pr_info("Disable TDX as there's only one TDX private KeyID available.\n");
+ goto no_tdx;
+ }
+
+ return 0;
+no_tdx:
+ clear_tdx();
+ return -ENODEV;
+}
+early_initcall(tdx_init);
+
+/* Return whether the BIOS has enabled TDX */
+bool platform_tdx_enabled(void)
+{
+ return !!tdx_keyid_num;
+}
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
new file mode 100644
index 000000000000..d00074abcb20
--- /dev/null
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _X86_VIRT_TDX_H
+#define _X86_VIRT_TDX_H
+
+/*
+ * This file contains both macros and data structures defined by the TDX
+ * architecture and Linux defined software data structures and functions.
+ * The two should not be mixed together for better readability. The
+ * architectural definitions come first.
+ */
+
+/* MSR to report KeyID partitioning between MKTME and TDX */
+#define MSR_IA32_MKTME_KEYID_PARTITIONING 0x00000087
+
+#endif
--
2.37.3
TDX provides increased levels of memory confidentiality and integrity.
This requires special hardware support for features like memory
encryption and storage of memory integrity checksums. Not all memory
satisfies these requirements.
As a result, the TDX introduced the concept of a "Convertible Memory
Region" (CMR). During boot, the firmware builds a list of all of the
memory ranges which can provide the TDX security guarantees. The list
of these ranges is available to the kernel by querying the TDX module.
The TDX architecture needs additional metadata to record things like
which TD guest "owns" a given page of memory. This metadata essentially
serves as the 'struct page' for the TDX module. The space for this
metadata is not reserved by the hardware up front and must be allocated
by the kernel and given to the TDX module.
Since this metadata consumes space, the VMM can choose whether or not to
allocate it for a given area of convertible memory. If it chooses not
to, the memory cannot receive TDX protections and can not be used by TDX
guests as private memory.
For every memory region that the VMM wants to use as TDX memory, it sets
up a "TD Memory Region" (TDMR). Each TDMR represents a physically
contiguous convertible range and must also have its own physically
contiguous metadata table, referred to as a Physical Address Metadata
Table (PAMT), to track status for each page in the TDMR range.
Unlike a CMR, each TDMR requires 1G granularity and alignment. To
support physical RAM areas that don't meet those strict requirements,
each TDMR permits a number of internal "reserved areas" which can be
placed over memory holes. If PAMT metadata is placed within a TDMR it
must be covered by one of these reserved areas.
Let's summarize the concepts:
CMR - Firmware-enumerated physical ranges that support TDX. CMRs are
4K aligned.
TDMR - Physical address range which is chosen by the kernel to support
TDX. 1G granularity and alignment required. Each TDMR has
reserved areas where TDX memory holes and overlapping PAMTs can
be put into.
PAMT - Physically contiguous TDX metadata. One table for each page size
per TDMR. Roughly 1/256th of TDMR in size. 256G TDMR = ~1G
PAMT.
As one step of initializing the TDX module, the kernel configures
TDX-usable memory regions by passing an array of TDMRs to the TDX module.
Constructing the array of TDMRs consists below steps:
1) Create TDMRs to cover all memory regions that the TDX module can use;
2) Allocate and set up PAMT for each TDMR;
3) Set up reserved areas for each TDMR.
Add a placeholder to construct TDMRs to do the above steps after all
TDX memory regions are verified to be truly convertible. Always free
TDMRs at the end of the initialization (no matter successful or not)
as TDMRs are only used during the initialization.
Reviewed-by: Isaku Yamahata <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
v5 -> v6:
- construct_tdmrs_memblock() -> construct_tdmrs() as 'tdx_memblock' is
used instead of memblock.
- Added Isaku's Reviewed-by.
- v3 -> v5 (no feedback on v4):
- Moved calculating TDMR size to this patch.
- Changed to use alloc_pages_exact() to allocate buffer for all TDMRs
once, instead of allocating each TDMR individually.
- Removed "crypto protection" in the changelog.
- -EFAULT -> -EINVAL in couple of places.
---
arch/x86/virt/vmx/tdx/tdx.c | 72 +++++++++++++++++++++++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 23 ++++++++++++
2 files changed, 95 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index ff3ef7ed4509..ba577d357aef 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -16,6 +16,8 @@
#include <linux/cpu.h>
#include <linux/cpumask.h>
#include <linux/smp.h>
+#include <linux/gfp.h>
+#include <linux/align.h>
#include <linux/atomic.h>
#include <asm/msr-index.h>
#include <asm/msr.h>
@@ -536,6 +538,53 @@ static int sanity_check_tdx_memory(void)
return 0;
}
+/* Calculate the actual TDMR_INFO size */
+static inline int cal_tdmr_size(void)
+{
+ int tdmr_sz;
+
+ /*
+ * The actual size of TDMR_INFO depends on the maximum number
+ * of reserved areas.
+ */
+ tdmr_sz = sizeof(struct tdmr_info);
+ tdmr_sz += sizeof(struct tdmr_reserved_area) *
+ tdx_sysinfo.max_reserved_per_tdmr;
+
+ /*
+ * TDX requires each TDMR_INFO to be 512-byte aligned. Always
+ * round up TDMR_INFO size to the 512-byte boundary.
+ */
+ return ALIGN(tdmr_sz, TDMR_INFO_ALIGNMENT);
+}
+
+static struct tdmr_info *alloc_tdmr_array(int *array_sz)
+{
+ /*
+ * TDX requires each TDMR_INFO to be 512-byte aligned.
+ * Use alloc_pages_exact() to allocate all TDMRs at once.
+ * Each TDMR_INFO will still be 512-byte aligned since
+ * cal_tdmr_size() always return 512-byte aligned size.
+ */
+ *array_sz = cal_tdmr_size() * tdx_sysinfo.max_tdmrs;
+
+ /*
+ * Zero the buffer so 'struct tdmr_info::size' can be
+ * used to determine whether a TDMR is valid.
+ */
+ return alloc_pages_exact(*array_sz, GFP_KERNEL | __GFP_ZERO);
+}
+
+/*
+ * Construct an array of TDMRs to cover all TDX memory ranges.
+ * The actual number of TDMRs is kept to @tdmr_num.
+ */
+static int construct_tdmrs(struct tdmr_info *tdmr_array, int *tdmr_num)
+{
+ /* Return -EINVAL until constructing TDMRs is done */
+ return -EINVAL;
+}
+
/*
* Detect and initialize the TDX module.
*
@@ -545,6 +594,9 @@ static int sanity_check_tdx_memory(void)
*/
static int init_tdx_module(void)
{
+ struct tdmr_info *tdmr_array;
+ int tdmr_array_sz;
+ int tdmr_num;
int ret;
/*
@@ -572,11 +624,31 @@ static int init_tdx_module(void)
ret = sanity_check_tdx_memory();
if (ret)
goto out;
+
+ /* Prepare enough space to construct TDMRs */
+ tdmr_array = alloc_tdmr_array(&tdmr_array_sz);
+ if (!tdmr_array) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ /* Construct TDMRs to cover all TDX memory ranges */
+ ret = construct_tdmrs(tdmr_array, &tdmr_num);
+ if (ret)
+ goto out_free_tdmrs;
+
/*
* Return -EINVAL until all steps of TDX module initialization
* process are done.
*/
ret = -EINVAL;
+out_free_tdmrs:
+ /*
+ * The array of TDMRs is freed no matter the initialization is
+ * successful or not. They are not needed anymore after the
+ * module initialization.
+ */
+ free_pages_exact(tdmr_array, tdmr_array_sz);
out:
return ret;
}
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index 8e273756098c..a737f2b51474 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -80,6 +80,29 @@ struct tdsysinfo_struct {
};
} __packed __aligned(TDSYSINFO_STRUCT_ALIGNMENT);
+struct tdmr_reserved_area {
+ u64 offset;
+ u64 size;
+} __packed;
+
+#define TDMR_INFO_ALIGNMENT 512
+
+struct tdmr_info {
+ u64 base;
+ u64 size;
+ u64 pamt_1g_base;
+ u64 pamt_1g_size;
+ u64 pamt_2m_base;
+ u64 pamt_2m_size;
+ u64 pamt_4k_base;
+ u64 pamt_4k_size;
+ /*
+ * Actual number of reserved areas depends on
+ * 'struct tdsysinfo_struct'::max_reserved_per_tdmr.
+ */
+ struct tdmr_reserved_area reserved_areas[0];
+} __packed __aligned(TDMR_INFO_ALIGNMENT);
+
/*
* Do not put any hardware-defined TDX structure representations below
* this comment!
--
2.37.3
To support kexec(), if the TDX module is initialized, the kernel needs
to flush all dirty cachelines associated with any TDX private KeyID,
otherwise they may silently corrupt the new kernel.
Following SME support, use wbinvd() to flush cache in stop_this_cpu().
Theoretically, cache flush is only needed when the TDX module has been
initialized. However initializing the TDX module is done on demand at
runtime, and it takes a mutex to read the module status. Just check
whether TDX is enabled by BIOS instead to flush cache.
The current TDX module architecture doesn't play nicely with kexec().
The TDX module can only be initialized once during its lifetime, and
there is no SEAMCALL to reset the module to give a new clean slate to
the new kernel. Therefore, ideally, if the module is ever initialized,
it's better to shut down the module. The new kernel won't be able to
use TDX anyway (as it needs to go through the TDX module initialization
process which will fail immediately at the first step).
However, there's no guarantee CPU is in VMX operation during kexec().
This means it's impractical to shut down the module. Just do nothing
but leave the module open.
Reviewed-by: Isaku Yamahata <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
arch/x86/kernel/process.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index c21b7347a26d..a8f482c6e600 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -765,8 +765,15 @@ void __noreturn stop_this_cpu(void *dummy)
*
* Test the CPUID bit directly because the machine might've cleared
* X86_FEATURE_SME due to cmdline options.
+ *
+ * Similar to SME, if the TDX module is ever initialized, the
+ * cachelines associated with any TDX private KeyID must be
+ * flushed before transiting to the new kernel. The TDX module
+ * is initialized on demand, and it takes the mutex to read it's
+ * status. Just check whether TDX is enabled by BIOS instead to
+ * flush cache.
*/
- if (cpuid_eax(0x8000001f) & BIT(0))
+ if (cpuid_eax(0x8000001f) & BIT(0) || platform_tdx_enabled())
native_wbinvd();
for (;;) {
/*
--
2.37.3
The MMIO/xAPIC interface has some problems, most notably the APIC LEAK
[1]. This bug allows an attacker to use the APIC MMIO interface to
extract data from the SGX enclave.
TDX is not immune from this either. Early check X2APIC and disable TDX
if X2APIC is not enabled, and make INTEL_TDX_HOST depend on X86_X2APIC.
More info:
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
[1]: https://aepicleak.com/aepicleak.pdf
Signed-off-by: Kai Huang <[email protected]>
---
arch/x86/Kconfig | 1 +
arch/x86/virt/vmx/tdx/tdx.c | 11 +++++++++++
2 files changed, 12 insertions(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index b9bd5d994ba7..f6f5e4f7a760 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1960,6 +1960,7 @@ config INTEL_TDX_HOST
depends on CPU_SUP_INTEL
depends on X86_64
depends on KVM_INTEL
+ depends on X86_X2APIC
help
Intel Trust Domain Extensions (TDX) protects guest VMs from malicious
host and certain physical attacks. This option enables necessary TDX
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 982d9c453b6b..8d943bdc8335 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -12,6 +12,7 @@
#include <linux/printk.h>
#include <asm/msr-index.h>
#include <asm/msr.h>
+#include <asm/apic.h>
#include <asm/tdx.h>
#include "tdx.h"
@@ -81,6 +82,16 @@ static int __init tdx_init(void)
goto no_tdx;
}
+ /*
+ * TDX requires X2APIC being enabled to prevent potential data
+ * leak via APIC MMIO registers. Just disable TDX if not using
+ * X2APIC.
+ */
+ if (!x2apic_enabled()) {
+ pr_info("Disable TDX as X2APIC is not enabled.\n");
+ goto no_tdx;
+ }
+
return 0;
no_tdx:
clear_tdx();
--
2.37.3
The TDX module uses additional metadata to record things like which
guest "owns" a given page of memory. This metadata, referred as
Physical Address Metadata Table (PAMT), essentially serves as the
'struct page' for the TDX module. PAMTs are not reserved by hardware
up front. They must be allocated by the kernel and then given to the
TDX module.
TDX supports 3 page sizes: 4K, 2M, and 1G. Each "TD Memory Region"
(TDMR) has 3 PAMTs to track the 3 supported page sizes. Each PAMT must
be a physically contiguous area from a Convertible Memory Region (CMR).
However, the PAMTs which track pages in one TDMR do not need to reside
within that TDMR but can be anywhere in CMRs. If one PAMT overlaps with
any TDMR, the overlapping part must be reported as a reserved area in
that particular TDMR.
Use alloc_contig_pages() since PAMT must be a physically contiguous area
and it may be potentially large (~1/256th of the size of the given TDMR).
The downside is alloc_contig_pages() may fail at runtime. One (bad)
mitigation is to launch a TD guest early during system boot to get those
PAMTs allocated at early time, but the only way to fix is to add a boot
option to allocate or reserve PAMTs during kernel boot.
TDX only supports a limited number of reserved areas per TDMR to cover
both PAMTs and memory holes within the given TDMR. If many PAMTs are
allocated within a single TDMR, the reserved areas may not be sufficient
to cover all of them.
Adopt the following policies when allocating PAMTs for a given TDMR:
- Allocate three PAMTs of the TDMR in one contiguous chunk to minimize
the total number of reserved areas consumed for PAMTs.
- Try to first allocate PAMT from the local node of the TDMR for better
NUMA locality.
Also dump out how many pages are allocated for PAMTs when the TDX module
is initialized successfully.
Reviewed-by: Isaku Yamahata <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
v5 -> v6:
- Rebase due to using 'tdx_memblock' instead of memblock.
- 'int pamt_entry_nr' -> 'unsigned long nr_pamt_entries' (Dave/Sagis).
- Improved comment around tdmr_get_nid() (Dave).
- Improved comment in tdmr_set_up_pamt() around breaking the PAMT
into PAMTs for 4K/2M/1G (Dave).
- tdmrs_get_pamt_pages() -> tdmrs_count_pamt_pages() (Dave).
- v3 -> v5 (no feedback on v4):
- Used memblock to get the NUMA node for given TDMR.
- Removed tdmr_get_pamt_sz() helper but use open-code instead.
- Changed to use 'switch .. case..' for each TDX supported page size in
tdmr_get_pamt_sz() (the original __tdmr_get_pamt_sz()).
- Added printing out memory used for PAMT allocation when TDX module is
initialized successfully.
- Explained downside of alloc_contig_pages() in changelog.
- Addressed other minor comments.
---
arch/x86/Kconfig | 1 +
arch/x86/virt/vmx/tdx/tdx.c | 193 ++++++++++++++++++++++++++++++++++++
2 files changed, 194 insertions(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index f6f5e4f7a760..bb291b2de830 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1961,6 +1961,7 @@ config INTEL_TDX_HOST
depends on X86_64
depends on KVM_INTEL
depends on X86_X2APIC
+ depends on CONTIG_ALLOC
help
Intel Trust Domain Extensions (TDX) protects guest VMs from malicious
host and certain physical attacks. This option enables necessary TDX
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index f6dde82d94cc..f7142f45bb0c 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -669,6 +669,189 @@ static int create_tdmrs(struct tdmr_info *tdmr_array, int *tdmr_num)
return 0;
}
+/*
+ * Calculate PAMT size given a TDMR and a page size. The returned
+ * PAMT size is always aligned up to 4K page boundary.
+ */
+static unsigned long tdmr_get_pamt_sz(struct tdmr_info *tdmr,
+ enum tdx_pg_level pgsz)
+{
+ unsigned long pamt_sz, nr_pamt_entries;
+
+ switch (pgsz) {
+ case TDX_PG_LEVEL_4K:
+ nr_pamt_entries = tdmr->size >> PAGE_SHIFT;
+ break;
+ case TDX_PG_LEVEL_2M:
+ nr_pamt_entries = tdmr->size >> PMD_SHIFT;
+ break;
+ case TDX_PG_LEVEL_1G:
+ nr_pamt_entries = tdmr->size >> PUD_SHIFT;
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ return 0;
+ }
+
+ pamt_sz = nr_pamt_entries * tdx_sysinfo.pamt_entry_size;
+ /* TDX requires PAMT size must be 4K aligned */
+ pamt_sz = ALIGN(pamt_sz, PAGE_SIZE);
+
+ return pamt_sz;
+}
+
+/*
+ * Pick a NUMA node on which to allocate this TDMR's metadata.
+ *
+ * This is imprecise since TDMRs are 1G aligned and NUMA nodes might
+ * not be. If the TDMR covers more than one node, just use the _first_
+ * one. This can lead to small areas of off-node metadata for some
+ * memory.
+ */
+static int tdmr_get_nid(struct tdmr_info *tdmr)
+{
+ struct tdx_memblock *tmb;
+
+ /* Find the first memory region covered by the TDMR */
+ list_for_each_entry(tmb, &tdx_memlist, list) {
+ if (tmb->end_pfn > (tdmr_start(tdmr) >> PAGE_SHIFT))
+ return tmb->nid;
+ }
+
+ /*
+ * Fall back to allocating the TDMR's metadata from node 0 when
+ * no TDX memory block can be found. This should never happen
+ * since TDMRs originate from TDX memory blocks.
+ */
+ WARN_ON_ONCE(1);
+ return 0;
+}
+
+static int tdmr_set_up_pamt(struct tdmr_info *tdmr)
+{
+ unsigned long pamt_base[TDX_PG_LEVEL_NUM];
+ unsigned long pamt_size[TDX_PG_LEVEL_NUM];
+ unsigned long tdmr_pamt_base;
+ unsigned long tdmr_pamt_size;
+ enum tdx_pg_level pgsz;
+ struct page *pamt;
+ int nid;
+
+ nid = tdmr_get_nid(tdmr);
+
+ /*
+ * Calculate the PAMT size for each TDX supported page size
+ * and the total PAMT size.
+ */
+ tdmr_pamt_size = 0;
+ for (pgsz = TDX_PG_LEVEL_4K; pgsz < TDX_PG_LEVEL_NUM; pgsz++) {
+ pamt_size[pgsz] = tdmr_get_pamt_sz(tdmr, pgsz);
+ tdmr_pamt_size += pamt_size[pgsz];
+ }
+
+ /*
+ * Allocate one chunk of physically contiguous memory for all
+ * PAMTs. This helps minimize the PAMT's use of reserved areas
+ * in overlapped TDMRs.
+ */
+ pamt = alloc_contig_pages(tdmr_pamt_size >> PAGE_SHIFT, GFP_KERNEL,
+ nid, &node_online_map);
+ if (!pamt)
+ return -ENOMEM;
+
+ /*
+ * Break the contiguous allocation back up into the
+ * individual PAMTs for each page size.
+ */
+ tdmr_pamt_base = page_to_pfn(pamt) << PAGE_SHIFT;
+ for (pgsz = TDX_PG_LEVEL_4K; pgsz < TDX_PG_LEVEL_NUM; pgsz++) {
+ pamt_base[pgsz] = tdmr_pamt_base;
+ tdmr_pamt_base += pamt_size[pgsz];
+ }
+
+ tdmr->pamt_4k_base = pamt_base[TDX_PG_LEVEL_4K];
+ tdmr->pamt_4k_size = pamt_size[TDX_PG_LEVEL_4K];
+ tdmr->pamt_2m_base = pamt_base[TDX_PG_LEVEL_2M];
+ tdmr->pamt_2m_size = pamt_size[TDX_PG_LEVEL_2M];
+ tdmr->pamt_1g_base = pamt_base[TDX_PG_LEVEL_1G];
+ tdmr->pamt_1g_size = pamt_size[TDX_PG_LEVEL_1G];
+
+ return 0;
+}
+
+static void tdmr_get_pamt(struct tdmr_info *tdmr, unsigned long *pamt_pfn,
+ unsigned long *pamt_npages)
+{
+ unsigned long pamt_base, pamt_sz;
+
+ /*
+ * The PAMT was allocated in one contiguous unit. The 4K PAMT
+ * should always point to the beginning of that allocation.
+ */
+ pamt_base = tdmr->pamt_4k_base;
+ pamt_sz = tdmr->pamt_4k_size + tdmr->pamt_2m_size + tdmr->pamt_1g_size;
+
+ *pamt_pfn = pamt_base >> PAGE_SHIFT;
+ *pamt_npages = pamt_sz >> PAGE_SHIFT;
+}
+
+static void tdmr_free_pamt(struct tdmr_info *tdmr)
+{
+ unsigned long pamt_pfn, pamt_npages;
+
+ tdmr_get_pamt(tdmr, &pamt_pfn, &pamt_npages);
+
+ /* Do nothing if PAMT hasn't been allocated for this TDMR */
+ if (!pamt_npages)
+ return;
+
+ if (WARN_ON_ONCE(!pamt_pfn))
+ return;
+
+ free_contig_range(pamt_pfn, pamt_npages);
+}
+
+static void tdmrs_free_pamt_all(struct tdmr_info *tdmr_array, int tdmr_num)
+{
+ int i;
+
+ for (i = 0; i < tdmr_num; i++)
+ tdmr_free_pamt(tdmr_array_entry(tdmr_array, i));
+}
+
+/* Allocate and set up PAMTs for all TDMRs */
+static int tdmrs_set_up_pamt_all(struct tdmr_info *tdmr_array, int tdmr_num)
+{
+ int i, ret = 0;
+
+ for (i = 0; i < tdmr_num; i++) {
+ ret = tdmr_set_up_pamt(tdmr_array_entry(tdmr_array, i));
+ if (ret)
+ goto err;
+ }
+
+ return 0;
+err:
+ tdmrs_free_pamt_all(tdmr_array, tdmr_num);
+ return ret;
+}
+
+static unsigned long tdmrs_count_pamt_pages(struct tdmr_info *tdmr_array,
+ int tdmr_num)
+{
+ unsigned long pamt_npages = 0;
+ int i;
+
+ for (i = 0; i < tdmr_num; i++) {
+ unsigned long pfn, npages;
+
+ tdmr_get_pamt(tdmr_array_entry(tdmr_array, i), &pfn, &npages);
+ pamt_npages += npages;
+ }
+
+ return pamt_npages;
+}
+
/*
* Construct an array of TDMRs to cover all TDX memory ranges.
* The actual number of TDMRs is kept to @tdmr_num.
@@ -681,8 +864,13 @@ static int construct_tdmrs(struct tdmr_info *tdmr_array, int *tdmr_num)
if (ret)
goto err;
+ ret = tdmrs_set_up_pamt_all(tdmr_array, *tdmr_num);
+ if (ret)
+ goto err;
+
/* Return -EINVAL until constructing TDMRs is done */
ret = -EINVAL;
+ tdmrs_free_pamt_all(tdmr_array, *tdmr_num);
err:
return ret;
}
@@ -744,6 +932,11 @@ static int init_tdx_module(void)
* process are done.
*/
ret = -EINVAL;
+ if (ret)
+ tdmrs_free_pamt_all(tdmr_array, tdmr_num);
+ else
+ pr_info("%lu pages allocated for PAMT.\n",
+ tdmrs_count_pamt_pages(tdmr_array, tdmr_num));
out_free_tdmrs:
/*
* The array of TDMRs is freed no matter the initialization is
--
2.37.3
All TDX-usable memory ranges were built during early kernel boot, and
they were not verified that they are truly convertible memory since CMRs
were not available until now.
Explicitly check all TDX memory ranges to make sure they are convertible
memory before passing those ranges to the TDX module.
Signed-off-by: Kai Huang <[email protected]>
---
v5 -> v6:
- Added a comment to explain two contiguous CMRs case (Isaku).
- Rebase due to using 'tdx_memblock' to represent TDX memory, thus
removed using memblock directly, and the handling of excluding
first 1MB as TDX memory.
v3 -> v4 (no feedback on v4):
- Changed to use memblock from e820.
- Simplified changelog a lot.
---
arch/x86/virt/vmx/tdx/tdx.c | 61 +++++++++++++++++++++++++++++++++++++
1 file changed, 61 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 7d7205615873..ff3ef7ed4509 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -483,6 +483,59 @@ static int tdx_get_sysinfo(void)
return check_cmrs(tdx_cmr_array, &tdx_cmr_num);
}
+/* Check whether the first range is the subrange of the second */
+static bool is_subrange(u64 r1_start, u64 r1_end, u64 r2_start, u64 r2_end)
+{
+ return r1_start >= r2_start && r1_end <= r2_end;
+}
+
+/* Check whether the address range is covered by any CMR or not. */
+static bool range_covered_by_cmr(struct cmr_info *cmr_array, int cmr_num,
+ u64 start, u64 end)
+{
+ int i;
+
+ for (i = 0; i < cmr_num; i++) {
+ struct cmr_info *cmr = &cmr_array[i];
+
+ if (is_subrange(start, end, cmr->base, cmr->base + cmr->size))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Check whether all memory regions in memblock are TDX convertible
+ * memory. Return 0 if all memory regions are convertible, or error.
+ */
+static int sanity_check_tdx_memory(void)
+{
+ struct tdx_memblock *tmb;
+
+ list_for_each_entry(tmb, &tdx_memlist, list) {
+ u64 start = tmb->start_pfn << PAGE_SHIFT;
+ u64 end = tmb->end_pfn << PAGE_SHIFT;
+
+ /*
+ * Note: The spec doesn't say two CMRs cannot be
+ * contiguous. Theoretically a memory region crossing
+ * two contiguous CMRs (but still falls into the two
+ * CMRs) should be treated as covered by CMR. But this
+ * is purely theoretically thing that doesn't occur in
+ * practice.
+ */
+ if (!range_covered_by_cmr(tdx_cmr_array, tdx_cmr_num, start,
+ end)) {
+ pr_err("[0x%llx, 0x%llx) is not fully convertible memory\n",
+ start, end);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
/*
* Detect and initialize the TDX module.
*
@@ -511,6 +564,14 @@ static int init_tdx_module(void)
if (ret)
goto out;
+ /*
+ * TDX memory ranges were built during kernel boot. Need to
+ * make sure all those ranges are truly convertible memory
+ * before passing them to the TDX module.
+ */
+ ret = sanity_check_tdx_memory();
+ if (ret)
+ goto out;
/*
* Return -EINVAL until all steps of TDX module initialization
* process are done.
--
2.37.3
The kernel configures TDX-usable memory regions by passing an array of
"TD Memory Regions" (TDMRs) to the TDX module. Each TDMR contains the
information of the base/size of a memory region, the base/size of the
associated Physical Address Metadata Table (PAMT) and a list of reserved
areas in the region.
Create a number of TDMRs to cover all TDX memory regions. To keep it
simple, always try to create one TDMR for each memory region. As the
first step only set up the base/size for each TDMR.
Each TDMR must be 1G aligned and the size must be in 1G granularity.
This implies that one TDMR could cover multiple memory regions. If a
memory region spans the 1GB boundary and the former part is already
covered by the previous TDMR, just create a new TDMR for the remaining
part.
TDX only supports a limited number of TDMRs. Disable TDX if all TDMRs
are consumed but there is more memory region to cover.
Signed-off-by: Kai Huang <[email protected]>
---
v5 -> v6:
- Rebase due to using 'tdx_memblock' instead of memblock.
- v3 -> v5 (no feedback on v4):
- Removed allocating TDMR individually.
- Improved changelog by using Dave's words.
- Made TDMR_START() and TDMR_END() as static inline function.
---
arch/x86/virt/vmx/tdx/tdx.c | 104 +++++++++++++++++++++++++++++++++++-
1 file changed, 103 insertions(+), 1 deletion(-)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index ba577d357aef..f6dde82d94cc 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -538,6 +538,24 @@ static int sanity_check_tdx_memory(void)
return 0;
}
+/* TDMR must be 1gb aligned */
+#define TDMR_ALIGNMENT BIT_ULL(30)
+#define TDMR_PFN_ALIGNMENT (TDMR_ALIGNMENT >> PAGE_SHIFT)
+
+/* Align up and down the address to TDMR boundary */
+#define TDMR_ALIGN_DOWN(_addr) ALIGN_DOWN((_addr), TDMR_ALIGNMENT)
+#define TDMR_ALIGN_UP(_addr) ALIGN((_addr), TDMR_ALIGNMENT)
+
+static inline u64 tdmr_start(struct tdmr_info *tdmr)
+{
+ return tdmr->base;
+}
+
+static inline u64 tdmr_end(struct tdmr_info *tdmr)
+{
+ return tdmr->base + tdmr->size;
+}
+
/* Calculate the actual TDMR_INFO size */
static inline int cal_tdmr_size(void)
{
@@ -575,14 +593,98 @@ static struct tdmr_info *alloc_tdmr_array(int *array_sz)
return alloc_pages_exact(*array_sz, GFP_KERNEL | __GFP_ZERO);
}
+static struct tdmr_info *tdmr_array_entry(struct tdmr_info *tdmr_array,
+ int idx)
+{
+ return (struct tdmr_info *)((unsigned long)tdmr_array +
+ cal_tdmr_size() * idx);
+}
+
+/*
+ * Create TDMRs to cover all TDX memory regions. The actual number
+ * of TDMRs is set to @tdmr_num.
+ */
+static int create_tdmrs(struct tdmr_info *tdmr_array, int *tdmr_num)
+{
+ struct tdx_memblock *tmb;
+ int tdmr_idx = 0;
+
+ /*
+ * Loop over TDX memory regions and create TDMRs to cover them.
+ * To keep it simple, always try to use one TDMR to cover
+ * one memory region.
+ */
+ list_for_each_entry(tmb, &tdx_memlist, list) {
+ struct tdmr_info *tdmr;
+ u64 start, end;
+
+ tdmr = tdmr_array_entry(tdmr_array, tdmr_idx);
+ start = TDMR_ALIGN_DOWN(tmb->start_pfn << PAGE_SHIFT);
+ end = TDMR_ALIGN_UP(tmb->end_pfn << PAGE_SHIFT);
+
+ /*
+ * If the current TDMR's size hasn't been initialized,
+ * it is a new TDMR to cover the new memory region.
+ * Otherwise, the current TDMR has already covered the
+ * previous memory region. In the latter case, check
+ * whether the current memory region has been fully or
+ * partially covered by the current TDMR, since TDMR is
+ * 1G aligned.
+ */
+ if (tdmr->size) {
+ /*
+ * Loop to the next memory region if the current
+ * block has already been fully covered by the
+ * current TDMR.
+ */
+ if (end <= tdmr_end(tdmr))
+ continue;
+
+ /*
+ * If part of the current memory region has
+ * already been covered by the current TDMR,
+ * skip the already covered part.
+ */
+ if (start < tdmr_end(tdmr))
+ start = tdmr_end(tdmr);
+
+ /*
+ * Create a new TDMR to cover the current memory
+ * region, or the remaining part of it.
+ */
+ tdmr_idx++;
+ if (tdmr_idx >= tdx_sysinfo.max_tdmrs)
+ return -E2BIG;
+
+ tdmr = tdmr_array_entry(tdmr_array, tdmr_idx);
+ }
+
+ tdmr->base = start;
+ tdmr->size = end - start;
+ }
+
+ /* @tdmr_idx is always the index of last valid TDMR. */
+ *tdmr_num = tdmr_idx + 1;
+
+ return 0;
+}
+
/*
* Construct an array of TDMRs to cover all TDX memory ranges.
* The actual number of TDMRs is kept to @tdmr_num.
*/
static int construct_tdmrs(struct tdmr_info *tdmr_array, int *tdmr_num)
{
+ int ret;
+
+ ret = create_tdmrs(tdmr_array, tdmr_num);
+ if (ret)
+ goto err;
+
/* Return -EINVAL until constructing TDMRs is done */
- return -EINVAL;
+ ret = -EINVAL;
+err:
+ return ret;
}
/*
--
2.37.3
Add documentation for TDX host kernel support. There is already one
file Documentation/x86/tdx.rst containing documentation for TDX guest
internals. Also reuse it for TDX host kernel support.
Introduce a new level menu "TDX Guest Support" and move existing
materials under it, and add a new menu for TDX host kernel support.
Signed-off-by: Kai Huang <[email protected]>
---
Documentation/x86/tdx.rst | 209 ++++++++++++++++++++++++++++++++++++--
1 file changed, 198 insertions(+), 11 deletions(-)
diff --git a/Documentation/x86/tdx.rst b/Documentation/x86/tdx.rst
index b8fa4329e1a5..59481dbe64b2 100644
--- a/Documentation/x86/tdx.rst
+++ b/Documentation/x86/tdx.rst
@@ -10,6 +10,193 @@ encrypting the guest memory. In TDX, a special module running in a special
mode sits between the host and the guest and manages the guest/host
separation.
+TDX Host Kernel Support
+=======================
+
+TDX introduces a new CPU mode called Secure Arbitration Mode (SEAM) and
+a new isolated range pointed by the SEAM Ranger Register (SEAMRR). A
+CPU-attested software module called 'the TDX module' runs inside the new
+isolated range to provide the functionalities to manage and run protected
+VMs.
+
+TDX also leverages Intel Multi-Key Total Memory Encryption (MKTME) to
+provide crypto-protection to the VMs. TDX reserves part of MKTME KeyIDs
+as TDX private KeyIDs, which are only accessible within the SEAM mode.
+BIOS is responsible for partitioning legacy MKTME KeyIDs and TDX KeyIDs.
+
+Before the TDX module can be used to create and run protected VMs, it
+must be loaded into the isolated range and properly initialized. The TDX
+architecture doesn't require the BIOS to load the TDX module, but the
+kernel assumes it is loaded by the BIOS.
+
+TDX boot-time detection
+-----------------------
+
+The kernel detects TDX by detecting TDX private KeyIDs during kernel
+boot. Below dmesg shows when TDX is enabled by BIOS::
+
+ [..] tdx: TDX enabled by BIOS. TDX private KeyID range: [16, 64).
+
+TDX module detection and initialization
+---------------------------------------
+
+There is no CPUID or MSR to detect the TDX module. The kernel detects it
+by initializing it.
+
+The kernel talks to the TDX module via the new SEAMCALL instruction. The
+TDX module implements SEAMCALL leaf functions to allow the kernel to
+initialize it.
+
+Initializing the TDX module consumes roughly ~1/256th system RAM size to
+use it as 'metadata' for the TDX memory. It also takes additional CPU
+time to initialize those metadata along with the TDX module itself. Both
+are not trivial. The kernel initializes the TDX module at runtime on
+demand. The caller to call tdx_enable() to initialize the TDX module::
+
+ ret = tdx_enable();
+ if (ret)
+ goto no_tdx;
+ // TDX is ready to use
+
+Initializing the TDX module requires all logical CPUs being online.
+tdx_enable() internally temporarily disables CPU hotplug to prevent any
+CPU from going offline, but the caller still needs to guarantee all
+present CPUs are online before calling tdx_enable().
+
+Also, tdx_enable() requires all CPUs are already in VMX operation
+(requirement of making SEAMCALL). Currently, tdx_enable() doesn't handle
+VMXON internally, but depends on the caller to guarantee that. So far
+KVM is the only user of TDX and KVM already handles VMXON.
+
+User can consult dmesg to see the presence of the TDX module, and whether
+it has been initialized.
+
+If the TDX module is not loaded, dmesg shows below::
+
+ [..] tdx: TDX module is not loaded.
+
+If the TDX module is initialized successfully, dmesg shows something
+like below::
+
+ [..] tdx: TDX module: attributes 0x0, vendor_id 0x8086, major_version 1, minor_version 0, build_date 20211209, build_num 160
+ [..] tdx: 65667 pages allocated for PAMT.
+ [..] tdx: TDX module initialized.
+
+If the TDX module failed to initialize, dmesg shows below::
+
+ [..] tdx: Failed to initialize TDX module. Shut it down.
+
+TDX Interaction to Other Kernel Components
+------------------------------------------
+
+TDX Memory Policy
+~~~~~~~~~~~~~~~~~
+
+The TDX module reports a list of "Convertible Memory Region" (CMR) to
+indicate which memory regions are TDX-capable. Those regions are
+generated by BIOS and verified by the MCHECK so that they are truly
+present during platform boot and can meet security guarantees.
+
+However those TDX convertible memory regions are not automatically usable
+to the TDX module. The kernel needs to choose all TDX-usable memory
+regions and pass those regions to the TDX module when initializing it.
+After TDX module is initialized, no more TDX-usable memory can be added
+to the TDX module.
+
+To keep things simple, this initial implementation chooses to use all
+boot-time present memory managed by the page allocator as TDX memory.
+This _requires_ all boot-time present memory is TDX convertible memory,
+which is true in practice. If there's any boot-time memory isn't TDX
+convertible memory (which is allowed from TDX architecture's point of
+view), it will be caught later during TDX module initialization and the
+initialization will fail.
+
+However one machine may support both TDX and non-TDX memory both at
+machine boot time and runtime. For example, any memory hot-added at
+runtime cannot be TDX memory. Also, for now NVDIMM and CXL memory are
+not TDX memory, no matter whether they are present at machine boot time
+or not.
+
+This raises a problem that, if any non-TDX memory is hot-added to the
+system-wide memory allocation pool, a non-TDX page may be allocated to a
+TDX guest, which will result in failing to create the TDX guest, or
+killing it at runtime.
+
+The current implementation doesn't explicitly prevent adding any non-TDX
+memory to system-wide memory pool, but depends on the machine owner to
+make sure such operation won't happen. For example, the machine owner
+should never plug any NVDIMM or CXL memory to the machine, or use kmem
+driver to hot-add any to the core-mm.
+
+This will be enhanced in the future. One solution is the kernel can be
+enforced to always guarantee all pages in the page allocator are TDX
+memory (i.e. by rejecting non-TDX memory in memory hotplug). Another
+option is the kernel may support different memory capabilities on basis
+of NUMA node. For example, the kernel can have both TDX-compatible NUMA
+node and non-TDX-compatible memory NUMA node, and the userspace needs to
+explicitly bind TDX guests to those TDX-compatible memory NUMA nodes.
+
+CPU Hotplug
+~~~~~~~~~~~
+
+TDX doesn't support physical (ACPI) CPU hotplug. During machine boot,
+TDX verifies all boot-time present logical CPUs are TDX compatible before
+enabling TDX. A non-buggy BIOS should never support hot-add/removal of
+physical CPU. Currently the kernel doesn't handle physical CPU hotplug,
+but depends on the BIOS to behave correctly.
+
+Note TDX works with CPU logical online/offline, thus the kernel still
+allows to offline logical CPU and online it again.
+
+Memory Hotplug
+~~~~~~~~~~~~~~
+
+TDX doesn't support ACPI memory hotplug of convertible memory. The list
+of "Convertible Memory Regions" (CMR) is static during machine's runtime.
+TDX also assumes convertible memory won't be hot-removed. A non-buggy
+BIOS should never support physical hot-removal of convertible memory.
+Currently the kernel doesn't handle hot-removal of convertible memory but
+depends on the BIOS to behave correctly.
+
+It's possible that one machine can have both TDX and non-TDX memory.
+Specifically, all hot-added physical memory are not TDX convertible
+memory. Also, for now NVDIMM and CXL memory are not TDX convertible
+memory, no matter whether they are physically present during boot or not.
+
+Plug non-TDX memory to the page allocator could result in failing to
+create a TDX guest, or killing a running TDX guest.
+
+To keep things simple, this series doesn't handle memory hotplug at all,
+but depends on the machine owner to not do any memory hotplug operation.
+For example, the machine owner should not plug any NVDIMM or CXL memory
+into the machine, or use kmem driver to plug NVDIMM or CXL memory to the
+core-mm.
+
+Kexec()
+~~~~~~~
+
+TDX (and MKTME) doesn't guarantee cache coherency among different KeyIDs.
+If the TDX module is ever initialized, the kernel needs to flush dirty
+cachelines associated with any TDX private KeyID, otherwise they may
+slightly corrupt the new kernel.
+
+Similar to SME support, the kernel uses wbinvd() to flush cache in
+stop_this_cpu().
+
+The current TDX module architecture doesn't play nicely with kexec().
+The TDX module can only be initialized once during its lifetime, and
+there is no SEAMCALL to reset the module to give a new clean slate to
+the new kernel. Therefore, ideally, if the module is ever initialized,
+it's better to shut down the module. The new kernel won't be able to
+use TDX anyway (as it needs to go through the TDX module initialization
+process which will fail immediately at the first step).
+
+However, there's no guarantee CPU is in VMX operation during kexec(), so
+it's impractical to shut down the module. Currently, the kernel just
+leaves the module in open state.
+
+TDX Guest Support
+=================
Since the host cannot directly access guest registers or memory, much
normal functionality of a hypervisor must be moved into the guest. This is
implemented using a Virtualization Exception (#VE) that is handled by the
@@ -20,7 +207,7 @@ TDX includes new hypercall-like mechanisms for communicating from the
guest to the hypervisor or the TDX module.
New TDX Exceptions
-==================
+------------------
TDX guests behave differently from bare-metal and traditional VMX guests.
In TDX guests, otherwise normal instructions or memory accesses can cause
@@ -30,7 +217,7 @@ Instructions marked with an '*' conditionally cause exceptions. The
details for these instructions are discussed below.
Instruction-based #VE
----------------------
+~~~~~~~~~~~~~~~~~~~~~
- Port I/O (INS, OUTS, IN, OUT)
- HLT
@@ -41,7 +228,7 @@ Instruction-based #VE
- CPUID*
Instruction-based #GP
----------------------
+~~~~~~~~~~~~~~~~~~~~~
- All VMX instructions: INVEPT, INVVPID, VMCLEAR, VMFUNC, VMLAUNCH,
VMPTRLD, VMPTRST, VMREAD, VMRESUME, VMWRITE, VMXOFF, VMXON
@@ -52,7 +239,7 @@ Instruction-based #GP
- RDMSR*,WRMSR*
RDMSR/WRMSR Behavior
---------------------
+~~~~~~~~~~~~~~~~~~~~
MSR access behavior falls into three categories:
@@ -73,7 +260,7 @@ trapping and handling in the TDX module. Other than possibly being slow,
these MSRs appear to function just as they would on bare metal.
CPUID Behavior
---------------
+~~~~~~~~~~~~~~
For some CPUID leaves and sub-leaves, the virtualized bit fields of CPUID
return values (in guest EAX/EBX/ECX/EDX) are configurable by the
@@ -93,7 +280,7 @@ not know how to handle. The guest kernel may ask the hypervisor for the
value with a hypercall.
#VE on Memory Accesses
-======================
+----------------------
There are essentially two classes of TDX memory: private and shared.
Private memory receives full TDX protections. Its content is protected
@@ -107,7 +294,7 @@ entries. This helps ensure that a guest does not place sensitive
information in shared memory, exposing it to the untrusted hypervisor.
#VE on Shared Memory
---------------------
+~~~~~~~~~~~~~~~~~~~~
Access to shared mappings can cause a #VE. The hypervisor ultimately
controls whether a shared memory access causes a #VE, so the guest must be
@@ -127,7 +314,7 @@ be careful not to access device MMIO regions unless it is also prepared to
handle a #VE.
#VE on Private Pages
---------------------
+~~~~~~~~~~~~~~~~~~~~
An access to private mappings can also cause a #VE. Since all kernel
memory is also private memory, the kernel might theoretically need to
@@ -145,7 +332,7 @@ The hypervisor is permitted to unilaterally move accepted pages to a
to handle the exception.
Linux #VE handler
-=================
+-----------------
Just like page faults or #GP's, #VE exceptions can be either handled or be
fatal. Typically, an unhandled userspace #VE results in a SIGSEGV.
@@ -167,7 +354,7 @@ While the block is in place, any #VE is elevated to a double fault (#DF)
which is not recoverable.
MMIO handling
-=============
+-------------
In non-TDX VMs, MMIO is usually implemented by giving a guest access to a
mapping which will cause a VMEXIT on access, and then the hypervisor
@@ -189,7 +376,7 @@ MMIO access via other means (like structure overlays) may result in an
oops.
Shared Memory Conversions
-=========================
+-------------------------
All TDX guest memory starts out as private at boot. This memory can not
be accessed by the hypervisor. However, some kernel users like device
--
2.37.3
After the TDX-usable memory regions are constructed in an array of TDMRs
and the global KeyID is reserved, configure them to the TDX module using
TDH.SYS.CONFIG SEAMCALL. TDH.SYS.CONFIG can only be called once and can
be done on any logical cpu.
Reviewed-by: Isaku Yamahata <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
arch/x86/virt/vmx/tdx/tdx.c | 38 +++++++++++++++++++++++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 2 ++
2 files changed, 40 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 0820ba781f97..fdfce715dda6 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -17,6 +17,7 @@
#include <linux/cpumask.h>
#include <linux/smp.h>
#include <linux/gfp.h>
+#include <linux/slab.h>
#include <linux/align.h>
#include <linux/atomic.h>
#include <linux/sort.h>
@@ -1064,6 +1065,37 @@ static int construct_tdmrs(struct tdmr_info *tdmr_array, int *tdmr_num)
return ret;
}
+static int config_tdx_module(struct tdmr_info *tdmr_array, int tdmr_num,
+ u64 global_keyid)
+{
+ u64 *tdmr_pa_array;
+ int i, array_sz;
+ u64 ret;
+
+ /*
+ * TDMR_INFO entries are configured to the TDX module via an
+ * array of the physical address of each TDMR_INFO. TDX module
+ * requires the array itself to be 512-byte aligned. Round up
+ * the array size to 512-byte aligned so the buffer allocated
+ * by kzalloc() will meet the alignment requirement.
+ */
+ array_sz = ALIGN(tdmr_num * sizeof(u64), TDMR_INFO_PA_ARRAY_ALIGNMENT);
+ tdmr_pa_array = kzalloc(array_sz, GFP_KERNEL);
+ if (!tdmr_pa_array)
+ return -ENOMEM;
+
+ for (i = 0; i < tdmr_num; i++)
+ tdmr_pa_array[i] = __pa(tdmr_array_entry(tdmr_array, i));
+
+ ret = seamcall(TDH_SYS_CONFIG, __pa(tdmr_pa_array), tdmr_num,
+ global_keyid, 0, NULL, NULL);
+
+ /* Free the array as it is not required anymore. */
+ kfree(tdmr_pa_array);
+
+ return ret;
+}
+
/*
* Detect and initialize the TDX module.
*
@@ -1122,11 +1154,17 @@ static int init_tdx_module(void)
*/
tdx_global_keyid = tdx_keyid_start;
+ /* Pass the TDMRs and the global KeyID to the TDX module */
+ ret = config_tdx_module(tdmr_array, tdmr_num, tdx_global_keyid);
+ if (ret)
+ goto out_free_pamts;
+
/*
* Return -EINVAL until all steps of TDX module initialization
* process are done.
*/
ret = -EINVAL;
+out_free_pamts:
if (ret)
tdmrs_free_pamt_all(tdmr_array, tdmr_num);
else
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index a737f2b51474..c26bab2555ca 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -19,6 +19,7 @@
#define TDH_SYS_INIT 33
#define TDH_SYS_LP_INIT 35
#define TDH_SYS_LP_SHUTDOWN 44
+#define TDH_SYS_CONFIG 45
struct cmr_info {
u64 base;
@@ -86,6 +87,7 @@ struct tdmr_reserved_area {
} __packed;
#define TDMR_INFO_ALIGNMENT 512
+#define TDMR_INFO_PA_ARRAY_ALIGNMENT 512
struct tdmr_info {
u64 base;
--
2.37.3
TDX module initialization requires to use one TDX private KeyID as the
global KeyID to protect the TDX module metadata. The global KeyID is
configured to the TDX module along with TDMRs.
Just reserve the first TDX private KeyID as the global KeyID. Keep the
global KeyID as a static variable as KVM will need to use it too.
Reviewed-by: Isaku Yamahata <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
arch/x86/virt/vmx/tdx/tdx.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 5d74ada072ca..0820ba781f97 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -62,6 +62,9 @@ static struct tdsysinfo_struct tdx_sysinfo;
static struct cmr_info tdx_cmr_array[MAX_CMRS] __aligned(CMR_INFO_ARRAY_ALIGNMENT);
static int tdx_cmr_num;
+/* TDX module global KeyID. Used in TDH.SYS.CONFIG ABI. */
+static u32 tdx_global_keyid;
+
/*
* Detect TDX private KeyIDs to see whether TDX has been enabled by the
* BIOS. Both initializing the TDX module and running TDX guest require
@@ -1113,6 +1116,12 @@ static int init_tdx_module(void)
if (ret)
goto out_free_tdmrs;
+ /*
+ * Reserve the first TDX KeyID as global KeyID to protect
+ * TDX module metadata.
+ */
+ tdx_global_keyid = tdx_keyid_start;
+
/*
* Return -EINVAL until all steps of TDX module initialization
* process are done.
--
2.37.3
As the last step of constructing TDMRs, set up reserved areas for all
TDMRs. For each TDMR, put all memory holes within this TDMR to the
reserved areas. And for all PAMTs which overlap with this TDMR, put
all the overlapping parts to reserved areas too.
Reviewed-by: Isaku Yamahata <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
v5 -> v6:
- Rebase due to using 'tdx_memblock' instead of memblock.
- Split tdmr_set_up_rsvd_areas() into two functions to handle memory
hole and PAMT respectively.
- Added Isaku's Reviewed-by.
---
arch/x86/virt/vmx/tdx/tdx.c | 190 +++++++++++++++++++++++++++++++++++-
1 file changed, 188 insertions(+), 2 deletions(-)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index f7142f45bb0c..5d74ada072ca 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -19,6 +19,7 @@
#include <linux/gfp.h>
#include <linux/align.h>
#include <linux/atomic.h>
+#include <linux/sort.h>
#include <asm/msr-index.h>
#include <asm/msr.h>
#include <asm/apic.h>
@@ -852,6 +853,187 @@ static unsigned long tdmrs_count_pamt_pages(struct tdmr_info *tdmr_array,
return pamt_npages;
}
+static int tdmr_add_rsvd_area(struct tdmr_info *tdmr, int *p_idx,
+ u64 addr, u64 size)
+{
+ struct tdmr_reserved_area *rsvd_areas = tdmr->reserved_areas;
+ int idx = *p_idx;
+
+ /* Reserved area must be 4K aligned in offset and size */
+ if (WARN_ON(addr & ~PAGE_MASK || size & ~PAGE_MASK))
+ return -EINVAL;
+
+ /* Cannot exceed maximum reserved areas supported by TDX */
+ if (idx >= tdx_sysinfo.max_reserved_per_tdmr)
+ return -E2BIG;
+
+ rsvd_areas[idx].offset = addr - tdmr->base;
+ rsvd_areas[idx].size = size;
+
+ *p_idx = idx + 1;
+
+ return 0;
+}
+
+static int tdmr_set_up_memory_hole_rsvd_areas(struct tdmr_info *tdmr,
+ int *rsvd_idx)
+{
+ struct tdx_memblock *tmb;
+ u64 prev_end;
+ int ret;
+
+ /* Mark holes between memory regions as reserved */
+ prev_end = tdmr_start(tdmr);
+ list_for_each_entry(tmb, &tdx_memlist, list) {
+ u64 start, end;
+
+ start = tmb->start_pfn << PAGE_SHIFT;
+ end = tmb->end_pfn << PAGE_SHIFT;
+
+ /* Break if this region is after the TDMR */
+ if (start >= tdmr_end(tdmr))
+ break;
+
+ /* Exclude regions before this TDMR */
+ if (end < tdmr_start(tdmr))
+ continue;
+
+ /*
+ * Skip if no hole exists before this region. "<=" is
+ * used because one memory region might span two TDMRs
+ * (when the previous TDMR covers part of this region).
+ * In this case the start address of this region is
+ * smaller than the start address of the second TDMR.
+ *
+ * Update the prev_end to the end of this region where
+ * the possible memory hole starts.
+ */
+ if (start <= prev_end) {
+ prev_end = end;
+ continue;
+ }
+
+ /* Add the hole before this region */
+ ret = tdmr_add_rsvd_area(tdmr, rsvd_idx, prev_end,
+ start - prev_end);
+ if (ret)
+ return ret;
+
+ prev_end = end;
+ }
+
+ /* Add the hole after the last region if it exists. */
+ if (prev_end < tdmr_end(tdmr)) {
+ ret = tdmr_add_rsvd_area(tdmr, rsvd_idx, prev_end,
+ tdmr_end(tdmr) - prev_end);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int tdmr_set_up_pamt_rsvd_areas(struct tdmr_info *tdmr, int *rsvd_idx,
+ struct tdmr_info *tdmr_array,
+ int tdmr_num)
+{
+ int i, ret;
+
+ /*
+ * If any PAMT overlaps with this TDMR, the overlapping part
+ * must also be put to the reserved area too. Walk over all
+ * TDMRs to find out those overlapping PAMTs and put them to
+ * reserved areas.
+ */
+ for (i = 0; i < tdmr_num; i++) {
+ struct tdmr_info *tmp = tdmr_array_entry(tdmr_array, i);
+ unsigned long pamt_start_pfn, pamt_npages;
+ u64 pamt_start, pamt_end;
+
+ tdmr_get_pamt(tmp, &pamt_start_pfn, &pamt_npages);
+ /* Each TDMR must already have PAMT allocated */
+ WARN_ON_ONCE(!pamt_npages || !pamt_start_pfn);
+
+ pamt_start = pamt_start_pfn << PAGE_SHIFT;
+ pamt_end = pamt_start + (pamt_npages << PAGE_SHIFT);
+
+ /* Skip PAMTs outside of the given TDMR */
+ if ((pamt_end <= tdmr_start(tdmr)) ||
+ (pamt_start >= tdmr_end(tdmr)))
+ continue;
+
+ /* Only mark the part within the TDMR as reserved */
+ if (pamt_start < tdmr_start(tdmr))
+ pamt_start = tdmr_start(tdmr);
+ if (pamt_end > tdmr_end(tdmr))
+ pamt_end = tdmr_end(tdmr);
+
+ ret = tdmr_add_rsvd_area(tdmr, rsvd_idx, pamt_start,
+ pamt_end - pamt_start);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+/* Compare function called by sort() for TDMR reserved areas */
+static int rsvd_area_cmp_func(const void *a, const void *b)
+{
+ struct tdmr_reserved_area *r1 = (struct tdmr_reserved_area *)a;
+ struct tdmr_reserved_area *r2 = (struct tdmr_reserved_area *)b;
+
+ if (r1->offset + r1->size <= r2->offset)
+ return -1;
+ if (r1->offset >= r2->offset + r2->size)
+ return 1;
+
+ /* Reserved areas cannot overlap. The caller should guarantee. */
+ WARN_ON_ONCE(1);
+ return -1;
+}
+
+/* Set up reserved areas for a TDMR, including memory holes and PAMTs */
+static int tdmr_set_up_rsvd_areas(struct tdmr_info *tdmr,
+ struct tdmr_info *tdmr_array,
+ int tdmr_num)
+{
+ int ret, rsvd_idx = 0;
+
+ /* Put all memory holes within the TDMR into reserved areas */
+ ret = tdmr_set_up_memory_hole_rsvd_areas(tdmr, &rsvd_idx);
+ if (ret)
+ return ret;
+
+ /* Put all (overlapping) PAMTs within the TDMR into reserved areas */
+ ret = tdmr_set_up_pamt_rsvd_areas(tdmr, &rsvd_idx, tdmr_array, tdmr_num);
+ if (ret)
+ return ret;
+
+ /* TDX requires reserved areas listed in address ascending order */
+ sort(tdmr->reserved_areas, rsvd_idx, sizeof(struct tdmr_reserved_area),
+ rsvd_area_cmp_func, NULL);
+
+ return 0;
+}
+
+static int tdmrs_set_up_rsvd_areas_all(struct tdmr_info *tdmr_array,
+ int tdmr_num)
+{
+ int i;
+
+ for (i = 0; i < tdmr_num; i++) {
+ int ret;
+
+ ret = tdmr_set_up_rsvd_areas(tdmr_array_entry(tdmr_array, i),
+ tdmr_array, tdmr_num);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
/*
* Construct an array of TDMRs to cover all TDX memory ranges.
* The actual number of TDMRs is kept to @tdmr_num.
@@ -868,8 +1050,12 @@ static int construct_tdmrs(struct tdmr_info *tdmr_array, int *tdmr_num)
if (ret)
goto err;
- /* Return -EINVAL until constructing TDMRs is done */
- ret = -EINVAL;
+ ret = tdmrs_set_up_rsvd_areas_all(tdmr_array, *tdmr_num);
+ if (ret)
+ goto err_free_pamts;
+
+ return 0;
+err_free_pamts:
tdmrs_free_pamt_all(tdmr_array, *tdmr_num);
err:
return ret;
--
2.37.3
On Wed, 2022-10-26 at 16:26 -0700, Dave Hansen wrote:
> On 10/26/22 16:15, Kai Huang wrote:
> > To keep things simple, this series doesn't handle memory hotplug at all,
> > but depends on the machine owner to not do any memory hotplug operation.
> > For exmaple, the machine owner should not plug any NVDIMM and CXL memory
> > into the machine, or use kmem driver to plug NVDIMM or CXL memory to the
> > core-mm.
> >
> > This will be enhanced in the future after first submission. We are also
> > looking into options on how to handle:
>
> This is also known as the "hopes and prayers" approach to software
> enabling. "Let's just hope and pray that nobody does these things which
> we know are broken."
>
> In the spirit of moving this submission forward, I'm willing to continue
> to _review_ this series.
>
Thank you Dave!
> But, I don't think it can go upstream until it
> contains at least _some_ way to handle memory hotplug.
>
>
Yes I agree.
One intention of sending out this series is actually to hear feedbacks on how to
handle. As mentioned in the cover letter, AFAICT we have two options:
1) to enforce the kernel to always guarantee all pages in the page allocator are
TDX memory (i.e. via rejecting non-TDX memory in memory hotplug). Non-TDX
memory can be used via devdax.
2) to manage TDX and non-TDX memory in different NUMA nodes, and use per-node
TDX memory capability flag to show which nodes are TDX-capable. Userspace needs
to explicitly bind TDX guests to those TDX-capable NUMA nodes.
I think the important thing is we need to get consensus on which direction to go
as this is kinda related to userspace ABI AFAICT.
Kirill has some thoughts on the second option, such as we may need some
additional work to split NUMA node which contains both TDX and non-TDX memory.
I am not entirely clear how hard this work will be, but my thinking is, the
above two are not necessarily conflicting. For example, from userspace ABI's
perspective we can go option 2, but at the meantime, we still reject hotplug of
non-TDX memory. This effectively equals to reporting all nodes as TDX-capable.
Splitting NUMA nodes which contains both TDX and non-TDX memory can be enhanced
in the future as it doesn't break userspace ABI -- userspace needs to
explicitly bind TDX guests to TDX-capable nodes anyway.
Initialize TDMRs via TDH.SYS.TDMR.INIT as the last step to complete the
TDX initialization.
All TDMRs need to be initialized using TDH.SYS.TDMR.INIT SEAMCALL before
the memory pages can be used by the TDX module. The time to initialize
TDMR is proportional to the size of the TDMR because TDH.SYS.TDMR.INIT
internally initializes the PAMT entries using the global KeyID.
To avoid long latency caused in one SEAMCALL, TDH.SYS.TDMR.INIT only
initializes an (implementation-specific) subset of PAMT entries of one
TDMR in one invocation. The caller needs to call TDH.SYS.TDMR.INIT
iteratively until all PAMT entries of the given TDMR are initialized.
TDH.SYS.TDMR.INITs can run concurrently on multiple CPUs as long as they
are initializing different TDMRs. To keep it simple, just initialize
all TDMRs one by one. On a 2-socket machine with 2.2G CPUs and 64GB
memory, each TDH.SYS.TDMR.INIT roughly takes couple of microseconds on
average, and it takes roughly dozens of milliseconds to complete the
initialization of all TDMRs while system is idle.
Reviewed-by: Isaku Yamahata <[email protected]>
Signed-off-by: Kai Huang <[email protected]>
---
arch/x86/virt/vmx/tdx/tdx.c | 70 ++++++++++++++++++++++++++++++++++---
arch/x86/virt/vmx/tdx/tdx.h | 1 +
2 files changed, 66 insertions(+), 5 deletions(-)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 9cfb01e7666a..68ec1ebecb49 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1151,6 +1151,66 @@ static int config_global_keyid(void)
return seamcall_on_each_package_serialized(&sc);
}
+/* Initialize one TDMR */
+static int init_tdmr(struct tdmr_info *tdmr)
+{
+ u64 next;
+
+ /*
+ * Initializing PAMT entries might be time-consuming (in
+ * proportion to the size of the requested TDMR). To avoid long
+ * latency in one SEAMCALL, TDH.SYS.TDMR.INIT only initializes
+ * an (implementation-defined) subset of PAMT entries in one
+ * invocation.
+ *
+ * Call TDH.SYS.TDMR.INIT iteratively until all PAMT entries
+ * of the requested TDMR are initialized (if next-to-initialize
+ * address matches the end address of the TDMR).
+ */
+ do {
+ struct tdx_module_output out;
+ int ret;
+
+ ret = seamcall(TDH_SYS_TDMR_INIT, tdmr->base, 0, 0, 0, NULL,
+ &out);
+ if (ret)
+ return ret;
+ /*
+ * RDX contains 'next-to-initialize' address if
+ * TDH.SYS.TDMR.INT succeeded.
+ */
+ next = out.rdx;
+ /* Allow scheduling when needed */
+ if (need_resched())
+ cond_resched();
+ } while (next < tdmr->base + tdmr->size);
+
+ return 0;
+}
+
+/* Initialize all TDMRs */
+static int init_tdmrs(struct tdmr_info *tdmr_array, int tdmr_num)
+{
+ int i;
+
+ /*
+ * Initialize TDMRs one-by-one for simplicity, though the TDX
+ * architecture does allow different TDMRs to be initialized in
+ * parallel on multiple CPUs. Parallel initialization could
+ * be added later when the time spent in the serialized scheme
+ * becomes a real concern.
+ */
+ for (i = 0; i < tdmr_num; i++) {
+ int ret;
+
+ ret = init_tdmr(tdmr_array_entry(tdmr_array, i));
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
/*
* Detect and initialize the TDX module.
*
@@ -1232,11 +1292,11 @@ static int init_tdx_module(void)
if (ret)
goto out_free_pamts;
- /*
- * Return -EINVAL until all steps of TDX module initialization
- * process are done.
- */
- ret = -EINVAL;
+ /* Initialize TDMRs to complete the TDX module initialization */
+ ret = init_tdmrs(tdmr_array, tdmr_num);
+ if (ret)
+ goto out_free_pamts;
+
out_free_pamts:
if (ret) {
/*
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index 768d097412ab..891691b1ea50 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -19,6 +19,7 @@
#define TDH_SYS_INFO 32
#define TDH_SYS_INIT 33
#define TDH_SYS_LP_INIT 35
+#define TDH_SYS_TDMR_INIT 36
#define TDH_SYS_LP_SHUTDOWN 44
#define TDH_SYS_CONFIG 45
--
2.37.3
On 10/26/22 16:15, Kai Huang wrote:
> To keep things simple, this series doesn't handle memory hotplug at all,
> but depends on the machine owner to not do any memory hotplug operation.
> For exmaple, the machine owner should not plug any NVDIMM and CXL memory
> into the machine, or use kmem driver to plug NVDIMM or CXL memory to the
> core-mm.
>
> This will be enhanced in the future after first submission. We are also
> looking into options on how to handle:
This is also known as the "hopes and prayers" approach to software
enabling. "Let's just hope and pray that nobody does these things which
we know are broken."
In the spirit of moving this submission forward, I'm willing to continue
to _review_ this series. But, I don't think it can go upstream until it
contains at least _some_ way to handle memory hotplug.
On 10/27/2022 7:16 AM, Kai Huang wrote:
> TDX supports 4K, 2M and 1G page sizes. When TDX guest accepts one page
> via try_accept_one(), it passes the page size level to the TDX module.
> Currently try_accept_one() uses hard-coded magic number for that.
>
> Introduce a new enum type to represent the page level of TDX supported
> page sizes to replace the hard-coded values. Both initializing the TDX
> module and KVM TDX support will need to use that too.
>
> Also, currently try_accept_one() uses an open-coded switch statement to
> get the TDX page level from the kernel page level. As KVM will also
> need to do the same thing, introduce a common helper to convert the
> kernel page level to the TDX page level.
>
> Reviewed-by: Kirill A. Shutemov <[email protected]>
> Signed-off-by: Kai Huang <[email protected]>
> ---
> arch/x86/coco/tdx/tdx.c | 20 ++++----------------
> arch/x86/include/asm/tdx.h | 33 +++++++++++++++++++++++++++++++++
> 2 files changed, 37 insertions(+), 16 deletions(-)
>
> diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c
> index 928dcf7a20d9..c5ff9647213d 100644
> --- a/arch/x86/coco/tdx/tdx.c
> +++ b/arch/x86/coco/tdx/tdx.c
> @@ -655,7 +655,6 @@ static bool try_accept_one(phys_addr_t *start, unsigned long len,
> {
> unsigned long accept_size = page_level_size(pg_level);
> u64 tdcall_rcx;
> - u8 page_size;
>
> if (!IS_ALIGNED(*start, accept_size))
> return false;
> @@ -663,27 +662,16 @@ static bool try_accept_one(phys_addr_t *start, unsigned long len,
> if (len < accept_size)
> return false;
>
> + /* TDX only supports 4K/2M/1G page sizes */
yes, a page can be mapped as 1G size to TD via secure/shared EPT. But
for this particular TDX_ACCEPT_PAGE case, it only supports 4K and 2M
currently, which is defined in TDX module spec.
This also implies one thing can be improved in current kernel that
trying accepting a page from 1G in tdx_enc_status_changed() can be
optimized to from 2M. It can be changed to start from 1G when TDX
supports accepting 1G page directly.
> + if (pg_level < PG_LEVEL_4K || pg_level > PG_LEVEL_1G)
> + return false;
> /*
> * Pass the page physical address to the TDX module to accept the
> * pending, private page.
> *
> * Bits 2:0 of RCX encode page size: 0 - 4K, 1 - 2M, 2 - 1G.
Maybe the “page size” can be adjusted to “TDX page level” accordingly.
> */
> - switch (pg_level) {
> - case PG_LEVEL_4K:
> - page_size = 0;
> - break;
> - case PG_LEVEL_2M:
> - page_size = 1;
> - break;
> - case PG_LEVEL_1G:
> - page_size = 2;
> - break;
> - default:
> - return false;
> - }
> -
> - tdcall_rcx = *start | page_size;
> + tdcall_rcx = *start | to_tdx_pg_level(pg_level);
> if (__tdx_module_call(TDX_ACCEPT_PAGE, tdcall_rcx, 0, 0, 0, NULL))
> return false;
>
> diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
> index 020c81a7c729..1c166fb9c22f 100644
> --- a/arch/x86/include/asm/tdx.h
> +++ b/arch/x86/include/asm/tdx.h
> @@ -20,6 +20,39 @@
>
> #ifndef __ASSEMBLY__
>
> +#include <asm/pgtable_types.h>
> +
> +/*
> + * The page levels of TDX supported page sizes (4K/2M/1G).
> + *
> + * Those values are part of the TDX module ABI. Do not change them.
> + */
> +enum tdx_pg_level {
> + TDX_PG_LEVEL_4K,
> + TDX_PG_LEVEL_2M,
> + TDX_PG_LEVEL_1G,
> + TDX_PG_LEVEL_NUM
> +};
> +
> +/*
> + * Get the TDX page level based on the kernel page level. The caller
> + * to make sure only pass 4K/2M/1G kernel page level.
> + */
> +static inline enum tdx_pg_level to_tdx_pg_level(enum pg_level pglvl)
> +{
> + switch (pglvl) {
> + case PG_LEVEL_4K:
> + return TDX_PG_LEVEL_4K;
> + case PG_LEVEL_2M:
> + return TDX_PG_LEVEL_2M;
> + case PG_LEVEL_1G:
> + return TDX_PG_LEVEL_1G;
> + default:
> + WARN_ON_ONCE(1);
> + }
> + return TDX_PG_LEVEL_NUM;
> +}
> +
> /*
> * Used to gather the output registers values of the TDCALL and SEAMCALL
> * instructions when requesting services from the TDX module.
On Thu, 2022-10-27 at 15:08 +0800, Li, Xiaoyao wrote:
> > @@ -663,27 +662,16 @@ static bool try_accept_one(phys_addr_t *start,
> > unsigned long len,
> > if (len < accept_size)
> > return false;
> >
> > + /* TDX only supports 4K/2M/1G page sizes */
>
> yes, a page can be mapped as 1G size to TD via secure/shared EPT. But
> for this particular TDX_ACCEPT_PAGE case, it only supports 4K and 2M
> currently, which is defined in TDX module spec.
I checked the TDX module public spec, and it appears you are right. But I am
not sure whether it will be changed in the future?
Anyway this patch doesn't intend to bring any functional change (I should have
stated this in the changelog), so I think fixing to this, if ever needed, should
be another patch.
Hi Isaku,
You suggested to introduce a helper, but this reminds me how KVM is going to use
this helper? KVM secure EPT can accept more levels than try_accept_one().
Perhaps I can just get rid of this helper? TDX host series only needs some
definitions to represent 4K/2M/1G page to get rid of using magic numbers.
>
> This also implies one thing can be improved in current kernel that
> trying accepting a page from 1G in tdx_enc_status_changed() can be
> optimized to from 2M. It can be changed to start from 1G when TDX
> supports accepting 1G page directly.
Ditto.
>
> > + if (pg_level < PG_LEVEL_4K || pg_level > PG_LEVEL_1G)
> > + return false;
> > /*
> > * Pass the page physical address to the TDX module to accept the
> > * pending, private page.
> > *
> > * Bits 2:0 of RCX encode page size: 0 - 4K, 1 - 2M, 2 - 1G.
>
> Maybe the “page size” can be adjusted to “TDX page level” accordingly.
Perhaps, but I think "page size" is also not that wrong, so I am reluctant to
change the existing comment. And let's see Isaku's response to my above
question first.
> + next = out.rdx;
> + /* Allow scheduling when needed */
> + if (need_resched())
> + cond_resched();
cond_resched already includes the need_resched check. That is why it is
called cond_
On 10/26/2022 4:16 PM, Kai Huang wrote:
> TDX module initialization requires to use one TDX private KeyID as the
> global KeyID to protect the TDX module metadata. The global KeyID is
> configured to the TDX module along with TDMRs.
>
> Just reserve the first TDX private KeyID as the global KeyID. Keep the
> global KeyID as a static variable as KVM will need to use it too.
>
> Reviewed-by: Isaku Yamahata <[email protected]>
> Signed-off-by: Kai Huang <[email protected]>
> ---
> arch/x86/virt/vmx/tdx/tdx.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> index 5d74ada072ca..0820ba781f97 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -62,6 +62,9 @@ static struct tdsysinfo_struct tdx_sysinfo;
> static struct cmr_info tdx_cmr_array[MAX_CMRS] __aligned(CMR_INFO_ARRAY_ALIGNMENT);
> static int tdx_cmr_num;
>
> +/* TDX module global KeyID. Used in TDH.SYS.CONFIG ABI. */
> +static u32 tdx_global_keyid;
Comment how this is serialized (or doesn't need it)
On 10/26/2022 4:16 PM, Kai Huang wrote:
> Add documentation for TDX host kernel support. There is already one
> file Documentation/x86/tdx.rst containing documentation for TDX guest
> internals. Also reuse it for TDX host kernel support.
>
> Introduce a new level menu "TDX Guest Support" and move existing
> materials under it, and add a new menu for TDX host kernel support.
>
> Signed-off-by: Kai Huang <[email protected]>
> ---
> Documentation/x86/tdx.rst | 209 ++++++++++++++++++++++++++++++++++++--
> 1 file changed, 198 insertions(+), 11 deletions(-)
>
> diff --git a/Documentation/x86/tdx.rst b/Documentation/x86/tdx.rst
> index b8fa4329e1a5..59481dbe64b2 100644
> --- a/Documentation/x86/tdx.rst
> +++ b/Documentation/x86/tdx.rst
> @@ -10,6 +10,193 @@ encrypting the guest memory. In TDX, a special module running in a special
> mode sits between the host and the guest and manages the guest/host
> separation.
>
> +TDX Host Kernel Support
> +=======================
> +
> +TDX introduces a new CPU mode called Secure Arbitration Mode (SEAM) and
> +a new isolated range pointed by the SEAM Ranger Register (SEAMRR). A
> +CPU-attested software module called 'the TDX module' runs inside the new
> +isolated range to provide the functionalities to manage and run protected
> +VMs.
> +
> +TDX also leverages Intel Multi-Key Total Memory Encryption (MKTME) to
> +provide crypto-protection to the VMs. TDX reserves part of MKTME KeyIDs
> +as TDX private KeyIDs, which are only accessible within the SEAM mode.
> +BIOS is responsible for partitioning legacy MKTME KeyIDs and TDX KeyIDs.
> +
> +Before the TDX module can be used to create and run protected VMs, it
> +must be loaded into the isolated range and properly initialized. The TDX
> +architecture doesn't require the BIOS to load the TDX module, but the
> +kernel assumes it is loaded by the BIOS.
> +
> +TDX boot-time detection
> +-----------------------
> +
> +The kernel detects TDX by detecting TDX private KeyIDs during kernel
> +boot. Below dmesg shows when TDX is enabled by BIOS::
> +
> + [..] tdx: TDX enabled by BIOS. TDX private KeyID range: [16, 64).
> +
> +TDX module detection and initialization
> +---------------------------------------
> +
> +There is no CPUID or MSR to detect the TDX module. The kernel detects it
> +by initializing it.
> +
> +The kernel talks to the TDX module via the new SEAMCALL instruction. The
> +TDX module implements SEAMCALL leaf functions to allow the kernel to
> +initialize it.
> +
> +Initializing the TDX module consumes roughly ~1/256th system RAM size to
> +use it as 'metadata' for the TDX memory. It also takes additional CPU
> +time to initialize those metadata along with the TDX module itself. Both
> +are not trivial. The kernel initializes the TDX module at runtime on
> +demand. The caller to call tdx_enable() to initialize the TDX module::
> +
> + ret = tdx_enable();
> + if (ret)
> + goto no_tdx;
> + // TDX is ready to use
> +
> +Initializing the TDX module requires all logical CPUs being online.
> +tdx_enable() internally temporarily disables CPU hotplug to prevent any
> +CPU from going offline, but the caller still needs to guarantee all
> +present CPUs are online before calling tdx_enable().
> +
> +Also, tdx_enable() requires all CPUs are already in VMX operation
> +(requirement of making SEAMCALL). Currently, tdx_enable() doesn't handle
> +VMXON internally, but depends on the caller to guarantee that. So far
> +KVM is the only user of TDX and KVM already handles VMXON.
> +
> +User can consult dmesg to see the presence of the TDX module, and whether
> +it has been initialized.
> +
> +If the TDX module is not loaded, dmesg shows below::
> +
> + [..] tdx: TDX module is not loaded.
> +
> +If the TDX module is initialized successfully, dmesg shows something
> +like below::
> +
> + [..] tdx: TDX module: attributes 0x0, vendor_id 0x8086, major_version 1, minor_version 0, build_date 20211209, build_num 160
> + [..] tdx: 65667 pages allocated for PAMT.
> + [..] tdx: TDX module initialized.
> +
> +If the TDX module failed to initialize, dmesg shows below::
> +
> + [..] tdx: Failed to initialize TDX module. Shut it down.
> +
> +TDX Interaction to Other Kernel Components
> +------------------------------------------
> +
> +TDX Memory Policy
> +~~~~~~~~~~~~~~~~~
> +
> +The TDX module reports a list of "Convertible Memory Region" (CMR) to
> +indicate which memory regions are TDX-capable. Those regions are
> +generated by BIOS and verified by the MCHECK so that they are truly
> +present during platform boot and can meet security guarantees.
> +
> +However those TDX convertible memory regions are not automatically usable
> +to the TDX module. The kernel needs to choose all TDX-usable memory
> +regions and pass those regions to the TDX module when initializing it.
> +After TDX module is initialized, no more TDX-usable memory can be added
> +to the TDX module.
> +
> +To keep things simple, this initial implementation chooses to use all
> +boot-time present memory managed by the page allocator as TDX memory.
> +This _requires_ all boot-time present memory is TDX convertible memory,
> +which is true in practice. If there's any boot-time memory isn't TDX
> +convertible memory (which is allowed from TDX architecture's point of
> +view), it will be caught later during TDX module initialization and the
> +initialization will fail.
> +
> +However one machine may support both TDX and non-TDX memory both at
> +machine boot time and runtime. For example, any memory hot-added at
> +runtime cannot be TDX memory. Also, for now NVDIMM and CXL memory are
> +not TDX memory, no matter whether they are present at machine boot time
> +or not.
> +
> +This raises a problem that, if any non-TDX memory is hot-added to the
> +system-wide memory allocation pool, a non-TDX page may be allocated to a
> +TDX guest, which will result in failing to create the TDX guest, or
> +killing it at runtime.
> +
> +The current implementation doesn't explicitly prevent adding any non-TDX
> +memory to system-wide memory pool, but depends on the machine owner to
> +make sure such operation won't happen. For example, the machine owner
> +should never plug any NVDIMM or CXL memory to the machine, or use kmem
> +driver to hot-add any to the core-mm.
I assume that will be fixed in some form, so doesn't need to be in the
documentation.
> +
> +To keep things simple, this series doesn't handle memory hotplug at all,
> +but depends on the machine owner to not do any memory hotplug operation.
> +For example, the machine owner should not plug any NVDIMM or CXL memory
> +into the machine, or use kmem driver to plug NVDIMM or CXL memory to the
> +core-mm.
Dito. Documentation/* shouldn't contain temporary things like a commit log.
On 10/26/2022 4:16 PM, Kai Huang wrote:
> After the array of TDMRs and the global KeyID are configured to the TDX
> module, use TDH.SYS.KEY.CONFIG to configure the key of the global KeyID
> on all packages.
>
> TDH.SYS.KEY.CONFIG must be done on one (any) cpu for each package. And
> it cannot run concurrently on different CPUs. Implement a helper to
> run SEAMCALL on one cpu for each package one by one, and use it to
> configure the global KeyID on all packages.
>
> Intel hardware doesn't guarantee cache coherency across different
> KeyIDs. The kernel needs to flush PAMT's dirty cachelines (associated
> with KeyID 0) before the TDX module uses the global KeyID to access the
> PAMT. Following the TDX module specification, flush cache before
> configuring the global KeyID on all packages.
>
> Given the PAMT size can be large (~1/256th of system RAM), just use
> WBINVD on all CPUs to flush.
>
> Reviewed-by: Isaku Yamahata <[email protected]>
> Signed-off-by: Kai Huang <[email protected]>
> ---
> arch/x86/virt/vmx/tdx/tdx.c | 83 ++++++++++++++++++++++++++++++++++++-
> arch/x86/virt/vmx/tdx/tdx.h | 1 +
> 2 files changed, 82 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> index fdfce715dda6..9cfb01e7666a 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -354,6 +354,46 @@ static void seamcall_on_each_cpu(struct seamcall_ctx *sc)
> on_each_cpu(seamcall_smp_call_function, sc, true);
> }
>
> +/*
> + * Call one SEAMCALL on one (any) cpu for each physical package in
> + * serialized way. Return immediately in case of any error if
> + * SEAMCALL fails on any cpu.
It's not clear what are you serializing against (against itself or other
calls of this functions)
I assume its because the TDX module errors out for parallel calls
instead of waiting.
The code seems to only do itself, so where is the check against others?
I assume in the callers but that would need to be explained. Also could
it need serialization against other kinds of seam calls?
Perhaps it might be more efficient to just broad cast and handle a retry
with some synchronization in the low level code.
That likely would cause less review thrash than just reimplementing a
common function like this here.
-Andi
On Thu, Oct 27, 2022 at 08:42:16AM +0000, Huang, Kai wrote:
> On Thu, 2022-10-27 at 15:08 +0800, Li, Xiaoyao wrote:
> > > @@ -663,27 +662,16 @@ static bool try_accept_one(phys_addr_t *start,
> > > unsigned long len,
> > > if (len < accept_size)
> > > return false;
> > >
> > > + /* TDX only supports 4K/2M/1G page sizes */
> >
> > yes, a page can be mapped as 1G size to TD via secure/shared EPT. But
> > for this particular TDX_ACCEPT_PAGE case, it only supports 4K and 2M
> > currently, which is defined in TDX module spec.
>
> I checked the TDX module public spec, and it appears you are right. But I am
> not sure whether it will be changed in the future?
The spec says:
Level of the Secure EPT entry that maps the private page to be
accepted: either 0 (4KB) or 1 (2MB) – see 20.5.1
Yes, it is about 4k and 2M, but it also refers 20.5.1, which lists sizes
up to 16PB.
Ultimately, it does not matter: if TDX module doesn't support the size or
cannot accept the memory for other reason guest kernel will fallback to
smaller size.
--
Kiryl Shutsemau / Kirill A. Shutemov
On 10/26/22 16:16, Kai Huang wrote:
> +/*
> + * Get the TDX page level based on the kernel page level. The caller
> + * to make sure only pass 4K/2M/1G kernel page level.
> + */
> +static inline enum tdx_pg_level to_tdx_pg_level(enum pg_level pglvl)
> +{
> + switch (pglvl) {
> + case PG_LEVEL_4K:
> + return TDX_PG_LEVEL_4K;
> + case PG_LEVEL_2M:
> + return TDX_PG_LEVEL_2M;
> + case PG_LEVEL_1G:
> + return TDX_PG_LEVEL_1G;
> + default:
> + WARN_ON_ONCE(1);
> + }
> + return TDX_PG_LEVEL_NUM;
> +}
Is TDX_PG_LEVEL_NUM part of the ABI? Or, is this going to accidentally
pass a whacky value to the SEAM module?
This needs something like this at the call-site:
page_size = to_tdx_pg_level(pg_level);
if (page_size >= TDX_PG_LEVEL_NUM)
return false;
> +/* Calculate the actual TDMR_INFO size */
> +static inline int cal_tdmr_size(void)
> +{
> + int tdmr_sz;
> +
> + /*
> + * The actual size of TDMR_INFO depends on the maximum number
> + * of reserved areas.
> + */
> + tdmr_sz = sizeof(struct tdmr_info);
> + tdmr_sz += sizeof(struct tdmr_reserved_area) *
> + tdx_sysinfo.max_reserved_per_tdmr;
would seem safer to have a overflow check here.
On Thu, Oct 27, 2022 at 08:42:16AM +0000,
"Huang, Kai" <[email protected]> wrote:
> On Thu, 2022-10-27 at 15:08 +0800, Li, Xiaoyao wrote:
> > > @@ -663,27 +662,16 @@ static bool try_accept_one(phys_addr_t *start,
> > > unsigned long len,
> > > if (len < accept_size)
> > > return false;
> > >
> > > + /* TDX only supports 4K/2M/1G page sizes */
> >
> > yes, a page can be mapped as 1G size to TD via secure/shared EPT. But
> > for this particular TDX_ACCEPT_PAGE case, it only supports 4K and 2M
> > currently, which is defined in TDX module spec.
>
> I checked the TDX module public spec, and it appears you are right. But I am
> not sure whether it will be changed in the future?
>
> Anyway this patch doesn't intend to bring any functional change (I should have
> stated this in the changelog), so I think fixing to this, if ever needed, should
> be another patch.
>
> Hi Isaku,
>
> You suggested to introduce a helper, but this reminds me how KVM is going to use
> this helper? KVM secure EPT can accept more levels than try_accept_one().
>
> Perhaps I can just get rid of this helper? TDX host series only needs some
> definitions to represent 4K/2M/1G page to get rid of using magic numbers.
Ok, let remove the helper function. The usage seems different from KVM case.
In KVM side, it can introduce its own helper.
--
Isaku Yamahata <[email protected]>
On Thu, 2022-10-27 at 08:27 -0700, Dave Hansen wrote:
> On 10/26/22 16:16, Kai Huang wrote:
> > +/*
> > + * Get the TDX page level based on the kernel page level. The caller
> > + * to make sure only pass 4K/2M/1G kernel page level.
> > + */
> > +static inline enum tdx_pg_level to_tdx_pg_level(enum pg_level pglvl)
> > +{
> > + switch (pglvl) {
> > + case PG_LEVEL_4K:
> > + return TDX_PG_LEVEL_4K;
> > + case PG_LEVEL_2M:
> > + return TDX_PG_LEVEL_2M;
> > + case PG_LEVEL_1G:
> > + return TDX_PG_LEVEL_1G;
> > + default:
> > + WARN_ON_ONCE(1);
> > + }
> > + return TDX_PG_LEVEL_NUM;
> > +}
>
> Is TDX_PG_LEVEL_NUM part of the ABI? Or, is this going to accidentally
> pass a whacky value to the SEAM module?
The intention is TDX_PG_LEVEL_NUM is not part of ABI, but looks I was wrong.
KVM secure EPT can accept larger page level of 1G as page table.
> This needs something like this at the call-site:
>
> page_size = to_tdx_pg_level(pg_level);
> if (page_size >= TDX_PG_LEVEL_NUM)
> return false;
Yes. Thanks for the time to review. It's bad, and should go away.
This reminds me I have mixed two tings together: 1) leaf page sizes (4K/2M/1G);
2) page table levels, which can have larger level than 1G.
In fact, the TDX module spec has a separate definition for the leaf page sizes:
Table 20.10: Page Size Definition
PS_1G 1G 2
PS_2M 2M 1
PS_4K 4K 0
While TDX guest and TDX host code only needs leaf page sizes, KVM needs all the
page table levels, so it's not necessarily to provide a common helper to get TDX
page level from kernel page level.
As Isaku also replied, I'll remove the helper.
Hi Kirill,
You expressed perhaps we can use macro definitions instead of the enum type.
Does below look good to you?
--- a/arch/x86/coco/tdx/tdx.c
+++ b/arch/x86/coco/tdx/tdx.c
@@ -671,13 +671,13 @@ static bool try_accept_one(phys_addr_t *start, unsigned
long len,
*/
switch (pg_level) {
case PG_LEVEL_4K:
- page_size = 0;
+ page_size = TDX_PS_4K;
break;
case PG_LEVEL_2M:
- page_size = 1;
+ page_size = TDX_PS_2M;
break;
case PG_LEVEL_1G:
- page_size = 2;
+ page_size = TDX_PS_1G;
break;
default:
return false;
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 020c81a7c729..74845d014d1c 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -20,6 +20,18 @@
#ifndef __ASSEMBLY__
+/*
+ * TDX supported page sizes (4K/2M/1G).
+ *
+ * Please refer to the TDX module 1.0 spec 20.4.1 Physical Page Size.
+ *
+ * Those values are part of the TDX module ABI (except TDX_PS_NUM).
+ */
+#define TDX_PS_4K 0
+#define TDX_PS_2M 1
+#define TDX_PS_1G 2
+#define TDX_PS_NUM 3
+
Btw, TDX host patch will use them in below way (please refer to patch 14:
x86/virt/tdx: Allocate and set up PAMTs for TDMRs):
unsigned long pamt_size[TDX_PS_NUM];
/*
* Calculate the PAMT size for each TDX supported page size
* and the total PAMT size. TDX_PS_* are contiguous from 0 to 3.
*/
for (pgsz = TDX_PS_4K; pgsz < TDX_PS_NUM; pgsz++) {
pamt_size[pgsz] = tdmr_get_pamt_sz(tdmr, pgsz);
...
}
On Thu, 2022-10-27 at 16:51 +0300, Kirill A. Shutemov wrote:
> On Thu, Oct 27, 2022 at 08:42:16AM +0000, Huang, Kai wrote:
> > On Thu, 2022-10-27 at 15:08 +0800, Li, Xiaoyao wrote:
> > > > @@ -663,27 +662,16 @@ static bool try_accept_one(phys_addr_t *start,
> > > > unsigned long len,
> > > > if (len < accept_size)
> > > > return false;
> > > >
> > > > + /* TDX only supports 4K/2M/1G page sizes */
> > >
> > > yes, a page can be mapped as 1G size to TD via secure/shared EPT. But
> > > for this particular TDX_ACCEPT_PAGE case, it only supports 4K and 2M
> > > currently, which is defined in TDX module spec.
> >
> > I checked the TDX module public spec, and it appears you are right. But I am
> > not sure whether it will be changed in the future?
>
> The spec says:
>
> Level of the Secure EPT entry that maps the private page to be
> accepted: either 0 (4KB) or 1 (2MB) – see 20.5.1
>
> Yes, it is about 4k and 2M, but it also refers 20.5.1, which lists sizes
> up to 16PB.
Also, I think we are mixing two things: 1) leaf page sizes (4K/2M/1G); 2) page
table levels. The latter has more levels than the former. For try_accept_one()
(and TDX host code), we actually care only about the former.
KVM needs the latter (or both), so it's better for KVM to handle on its own.
>
> Ultimately, it does not matter: if TDX module doesn't support the size or
> cannot accept the memory for other reason guest kernel will fallback to
> smaller size.
>
Yes.
On Thu, 2022-10-27 at 05:30 -0700, Andi Kleen wrote:
> > + next = out.rdx;
> > + /* Allow scheduling when needed */
> > + if (need_resched())
> > + cond_resched();
>
> cond_resched already includes the need_resched check. That is why it is
> called cond_
>
Will just use cond_resched(). Thanks!
Thanks for the review!
> >
> > +/*
> > + * Call one SEAMCALL on one (any) cpu for each physical package in
> > + * serialized way. Return immediately in case of any error if
> > + * SEAMCALL fails on any cpu.
>
>
> It's not clear what are you serializing against (against itself or other
> calls of this functions)
>
> I assume its because the TDX module errors out for parallel calls
> instead of waiting.
Yes.
>
> The code seems to only do itself, so where is the check against others?
> I assume in the callers but that would need to be explained.
>
Yes in the callers. In short there's no need, or it doesn't make sense to check
against others as SEAMCALLs involved during the module initialization are not
supposed can be made in random.
> Also could
> it need serialization against other kinds of seam calls?
The TDX module initialization is essentially a state machine -- it involves a
couple of steps to finish the process and each step moves the TDX module's
current state to a later state.
Each step involves a different SEAMCALL, but this SEAMCALL may be only called
one (any) cpu, or needs to be called for all cpus, or one (any) cpu for each
package.
The TDX module initialization code do the whole process step by step, so the
caller guarantees no random SEAMCALLs will be made when it is doing one step of
the initialization.
>
> Perhaps it might be more efficient to just broad cast and handle a retry
> with some synchronization in the low level code.
>
> That likely would cause less review thrash than just reimplementing a
> common function like this here.
As mentioned above the whole initialization process is just a machine state, so
not all SEAMCALLs are subject to the logic of retry. To me the retry only makes
sense for one particular SEAMCALL which must be done on multiple cpus but
requires some serialization.
Does all above make sense?
On Thu, 2022-10-27 at 05:40 -0700, Andi Kleen wrote:
> On 10/26/2022 4:16 PM, Kai Huang wrote:
> > TDX module initialization requires to use one TDX private KeyID as the
> > global KeyID to protect the TDX module metadata. The global KeyID is
> > configured to the TDX module along with TDMRs.
> >
> > Just reserve the first TDX private KeyID as the global KeyID. Keep the
> > global KeyID as a static variable as KVM will need to use it too.
> >
> > Reviewed-by: Isaku Yamahata <[email protected]>
> > Signed-off-by: Kai Huang <[email protected]>
> > ---
> > arch/x86/virt/vmx/tdx/tdx.c | 9 +++++++++
> > 1 file changed, 9 insertions(+)
> >
> > diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> > index 5d74ada072ca..0820ba781f97 100644
> > --- a/arch/x86/virt/vmx/tdx/tdx.c
> > +++ b/arch/x86/virt/vmx/tdx/tdx.c
> > @@ -62,6 +62,9 @@ static struct tdsysinfo_struct tdx_sysinfo;
> > static struct cmr_info tdx_cmr_array[MAX_CMRS] __aligned(CMR_INFO_ARRAY_ALIGNMENT);
> > static int tdx_cmr_num;
> >
> > +/* TDX module global KeyID. Used in TDH.SYS.CONFIG ABI. */
> > +static u32 tdx_global_keyid;
>
>
> Comment how this is serialized (or doesn't need it)
>
>
TDH.SYS.CONFIG, which takes 'tdx_global_keyid' as input, only needs to be called
once on any cpu, so no serialization is needed.
TDH.SYS.KEY.CONFIG, which doesn't take 'tdx_global_keyid' as input but
internally programs it, does require some serialization as this SEAMCALL must be
called on one cpu for each package, and it cannot run concurrently on different
cpus. How about adding the comment in the patch which does TDH.SYS.KEY.CONFIG?
How about below (taken from patch 18 "x86/virt/tdx: Configure global KeyID on
all packages", but added "in serialized way as it cannot run concurrently on
different cpus" at the end of the first sentence in the comment)?
static int config_global_keyid(void)
{
struct seamcall_ctx sc = { .fn = TDH_SYS_KEY_CONFIG };
/*
* Configure the key of the global KeyID on all packages by
* calling TDH.SYS.KEY.CONFIG on all packages in serialized
* way as it cannot run concurrently on different cpus.
*
* ......
*/
return seamcall_on_each_package_serialized(&sc);
}
On Thu, 2022-10-27 at 05:38 -0700, Andi Kleen wrote:
> > +The current implementation doesn't explicitly prevent adding any non-TDX
> > +memory to system-wide memory pool, but depends on the machine owner to
> > +make sure such operation won't happen. For example, the machine owner
> > +should never plug any NVDIMM or CXL memory to the machine, or use kmem
> > +driver to hot-add any to the core-mm.
>
>
> I assume that will be fixed in some form, so doesn't need to be in the
> documentation.
Based on we need to handle this in some way so yes this will go away.
>
>
> > +
> > +To keep things simple, this series doesn't handle memory hotplug at all,
> > +but depends on the machine owner to not do any memory hotplug operation.
> > +For example, the machine owner should not plug any NVDIMM or CXL memory
> > +into the machine, or use kmem driver to plug NVDIMM or CXL memory to the
> > +core-mm.
>
>
> Dito. Documentation/* shouldn't contain temporary things like a commit log.
Oops. I just copied from some commit log. Thanks for catching!
On Thu, 2022-10-27 at 08:31 -0700, Andi Kleen wrote:
> > +/* Calculate the actual TDMR_INFO size */
> > +static inline int cal_tdmr_size(void)
> > +{
> > + int tdmr_sz;
> > +
> > + /*
> > + * The actual size of TDMR_INFO depends on the maximum number
> > + * of reserved areas.
> > + */
> > + tdmr_sz = sizeof(struct tdmr_info);
> > + tdmr_sz += sizeof(struct tdmr_reserved_area) *
> > + tdx_sysinfo.max_reserved_per_tdmr;
>
>
> would seem safer to have a overflow check here.
>
>
How about below?
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -614,6 +614,14 @@ static inline int cal_tdmr_size(void)
tdmr_sz += sizeof(struct tdmr_reserved_area) *
tdx_sysinfo.max_reserved_per_tdmr;
+ /*
+ * Do simple check against overflow, and return 0 (invalid)
+ * TDMR_INFO size if it happened. Also WARN() as it should
+ * should never happen in reality.
+ */
+ if (WARN_ON_ONCE(tdmr_sz < 0))
+ return 0;
+
/*
* TDX requires each TDMR_INFO to be 512-byte aligned. Always
* round up TDMR_INFO size to the 512-byte boundary.
@@ -623,19 +631,27 @@ static inline int cal_tdmr_size(void)
static struct tdmr_info *alloc_tdmr_array(int *array_sz)
{
+ int sz;
+
/*
* TDX requires each TDMR_INFO to be 512-byte aligned.
* Use alloc_pages_exact() to allocate all TDMRs at once.
* Each TDMR_INFO will still be 512-byte aligned since
* cal_tdmr_size() always return 512-byte aligned size.
*/
- *array_sz = cal_tdmr_size() * tdx_sysinfo.max_tdmrs;
+ sz = cal_tdmr_size() * tdx_sysinfo.max_tdmrs;
+
+ /* Overflow */
+ if (!sz || WARN_ON_ONCE(sz < 0))
+ return NULL;
+
+ *array_sz = sz;
/*
* Zero the buffer so 'struct tdmr_info::size' can be
* used to determine whether a TDMR is valid.
*/
- return alloc_pages_exact(*array_sz, GFP_KERNEL | __GFP_ZERO);
+ return alloc_pages_exact(sz, GFP_KERNEL | __GFP_ZERO);
}
Btw, should I use alloc_contig_pages() instead of alloc_pages_exact() as IIUC
the latter should fail if the size is larger than 4MB? In reality, the entire
array only takes dozens of KBs, though.
On Thu, Oct 27, 2022 at 12:16:20PM +1300, Kai Huang wrote:
> diff --git a/Documentation/x86/tdx.rst b/Documentation/x86/tdx.rst
> index b8fa4329e1a5..59481dbe64b2 100644
> --- a/Documentation/x86/tdx.rst
> +++ b/Documentation/x86/tdx.rst
> @@ -10,6 +10,193 @@ encrypting the guest memory. In TDX, a special module running in a special
> mode sits between the host and the guest and manages the guest/host
> separation.
>
> +TDX Host Kernel Support
> +=======================
> +
> +TDX introduces a new CPU mode called Secure Arbitration Mode (SEAM) and
> +a new isolated range pointed by the SEAM Ranger Register (SEAMRR). A
> +CPU-attested software module called 'the TDX module' runs inside the new
> +isolated range to provide the functionalities to manage and run protected
> +VMs.
> +
> +TDX also leverages Intel Multi-Key Total Memory Encryption (MKTME) to
> +provide crypto-protection to the VMs. TDX reserves part of MKTME KeyIDs
> +as TDX private KeyIDs, which are only accessible within the SEAM mode.
> +BIOS is responsible for partitioning legacy MKTME KeyIDs and TDX KeyIDs.
> +
> +Before the TDX module can be used to create and run protected VMs, it
> +must be loaded into the isolated range and properly initialized. The TDX
> +architecture doesn't require the BIOS to load the TDX module, but the
> +kernel assumes it is loaded by the BIOS.
> +
> +TDX boot-time detection
> +-----------------------
> +
> +The kernel detects TDX by detecting TDX private KeyIDs during kernel
> +boot. Below dmesg shows when TDX is enabled by BIOS::
> +
> + [..] tdx: TDX enabled by BIOS. TDX private KeyID range: [16, 64).
> +
> +TDX module detection and initialization
> +---------------------------------------
> +
> +There is no CPUID or MSR to detect the TDX module. The kernel detects it
> +by initializing it.
> +
> +The kernel talks to the TDX module via the new SEAMCALL instruction. The
> +TDX module implements SEAMCALL leaf functions to allow the kernel to
> +initialize it.
> +
> +Initializing the TDX module consumes roughly ~1/256th system RAM size to
> +use it as 'metadata' for the TDX memory. It also takes additional CPU
> +time to initialize those metadata along with the TDX module itself. Both
> +are not trivial. The kernel initializes the TDX module at runtime on
> +demand. The caller to call tdx_enable() to initialize the TDX module::
> +
> + ret = tdx_enable();
> + if (ret)
> + goto no_tdx;
> + // TDX is ready to use
> +
> +Initializing the TDX module requires all logical CPUs being online.
> +tdx_enable() internally temporarily disables CPU hotplug to prevent any
> +CPU from going offline, but the caller still needs to guarantee all
> +present CPUs are online before calling tdx_enable().
> +
> +Also, tdx_enable() requires all CPUs are already in VMX operation
> +(requirement of making SEAMCALL). Currently, tdx_enable() doesn't handle
> +VMXON internally, but depends on the caller to guarantee that. So far
> +KVM is the only user of TDX and KVM already handles VMXON.
> +
> +User can consult dmesg to see the presence of the TDX module, and whether
> +it has been initialized.
> +
> +If the TDX module is not loaded, dmesg shows below::
> +
> + [..] tdx: TDX module is not loaded.
> +
> +If the TDX module is initialized successfully, dmesg shows something
> +like below::
> +
> + [..] tdx: TDX module: attributes 0x0, vendor_id 0x8086, major_version 1, minor_version 0, build_date 20211209, build_num 160
> + [..] tdx: 65667 pages allocated for PAMT.
> + [..] tdx: TDX module initialized.
> +
> +If the TDX module failed to initialize, dmesg shows below::
> +
> + [..] tdx: Failed to initialize TDX module. Shut it down.
> +
> +TDX Interaction to Other Kernel Components
> +------------------------------------------
> +
> +TDX Memory Policy
> +~~~~~~~~~~~~~~~~~
> +
> +The TDX module reports a list of "Convertible Memory Region" (CMR) to
> +indicate which memory regions are TDX-capable. Those regions are
> +generated by BIOS and verified by the MCHECK so that they are truly
> +present during platform boot and can meet security guarantees.
> +
> +However those TDX convertible memory regions are not automatically usable
> +to the TDX module. The kernel needs to choose all TDX-usable memory
> +regions and pass those regions to the TDX module when initializing it.
> +After TDX module is initialized, no more TDX-usable memory can be added
> +to the TDX module.
> +
> +To keep things simple, this initial implementation chooses to use all
> +boot-time present memory managed by the page allocator as TDX memory.
> +This _requires_ all boot-time present memory is TDX convertible memory,
> +which is true in practice. If there's any boot-time memory isn't TDX
> +convertible memory (which is allowed from TDX architecture's point of
> +view), it will be caught later during TDX module initialization and the
> +initialization will fail.
> +
> +However one machine may support both TDX and non-TDX memory both at
> +machine boot time and runtime. For example, any memory hot-added at
> +runtime cannot be TDX memory. Also, for now NVDIMM and CXL memory are
> +not TDX memory, no matter whether they are present at machine boot time
> +or not.
> +
> +This raises a problem that, if any non-TDX memory is hot-added to the
> +system-wide memory allocation pool, a non-TDX page may be allocated to a
> +TDX guest, which will result in failing to create the TDX guest, or
> +killing it at runtime.
> +
> +The current implementation doesn't explicitly prevent adding any non-TDX
> +memory to system-wide memory pool, but depends on the machine owner to
> +make sure such operation won't happen. For example, the machine owner
> +should never plug any NVDIMM or CXL memory to the machine, or use kmem
> +driver to hot-add any to the core-mm.
> +
> +This will be enhanced in the future. One solution is the kernel can be
> +enforced to always guarantee all pages in the page allocator are TDX
> +memory (i.e. by rejecting non-TDX memory in memory hotplug). Another
> +option is the kernel may support different memory capabilities on basis
> +of NUMA node. For example, the kernel can have both TDX-compatible NUMA
> +node and non-TDX-compatible memory NUMA node, and the userspace needs to
> +explicitly bind TDX guests to those TDX-compatible memory NUMA nodes.
> +
> +CPU Hotplug
> +~~~~~~~~~~~
> +
> +TDX doesn't support physical (ACPI) CPU hotplug. During machine boot,
> +TDX verifies all boot-time present logical CPUs are TDX compatible before
> +enabling TDX. A non-buggy BIOS should never support hot-add/removal of
> +physical CPU. Currently the kernel doesn't handle physical CPU hotplug,
> +but depends on the BIOS to behave correctly.
> +
> +Note TDX works with CPU logical online/offline, thus the kernel still
> +allows to offline logical CPU and online it again.
> +
> +Memory Hotplug
> +~~~~~~~~~~~~~~
> +
> +TDX doesn't support ACPI memory hotplug of convertible memory. The list
> +of "Convertible Memory Regions" (CMR) is static during machine's runtime.
> +TDX also assumes convertible memory won't be hot-removed. A non-buggy
> +BIOS should never support physical hot-removal of convertible memory.
> +Currently the kernel doesn't handle hot-removal of convertible memory but
> +depends on the BIOS to behave correctly.
> +
> +It's possible that one machine can have both TDX and non-TDX memory.
> +Specifically, all hot-added physical memory are not TDX convertible
> +memory. Also, for now NVDIMM and CXL memory are not TDX convertible
> +memory, no matter whether they are physically present during boot or not.
> +
> +Plug non-TDX memory to the page allocator could result in failing to
> +create a TDX guest, or killing a running TDX guest.
> +
> +To keep things simple, this series doesn't handle memory hotplug at all,
> +but depends on the machine owner to not do any memory hotplug operation.
> +For example, the machine owner should not plug any NVDIMM or CXL memory
> +into the machine, or use kmem driver to plug NVDIMM or CXL memory to the
> +core-mm.
> +
> +Kexec()
> +~~~~~~~
> +
> +TDX (and MKTME) doesn't guarantee cache coherency among different KeyIDs.
> +If the TDX module is ever initialized, the kernel needs to flush dirty
> +cachelines associated with any TDX private KeyID, otherwise they may
> +slightly corrupt the new kernel.
> +
> +Similar to SME support, the kernel uses wbinvd() to flush cache in
> +stop_this_cpu().
> +
> +The current TDX module architecture doesn't play nicely with kexec().
> +The TDX module can only be initialized once during its lifetime, and
> +there is no SEAMCALL to reset the module to give a new clean slate to
> +the new kernel. Therefore, ideally, if the module is ever initialized,
> +it's better to shut down the module. The new kernel won't be able to
> +use TDX anyway (as it needs to go through the TDX module initialization
> +process which will fail immediately at the first step).
> +
> +However, there's no guarantee CPU is in VMX operation during kexec(), so
> +it's impractical to shut down the module. Currently, the kernel just
> +leaves the module in open state.
> +
> +TDX Guest Support
> +=================
> Since the host cannot directly access guest registers or memory, much
> normal functionality of a hypervisor must be moved into the guest. This is
> implemented using a Virtualization Exception (#VE) that is handled by the
> @@ -20,7 +207,7 @@ TDX includes new hypercall-like mechanisms for communicating from the
> guest to the hypervisor or the TDX module.
>
> New TDX Exceptions
> -==================
> +------------------
>
> TDX guests behave differently from bare-metal and traditional VMX guests.
> In TDX guests, otherwise normal instructions or memory accesses can cause
> @@ -30,7 +217,7 @@ Instructions marked with an '*' conditionally cause exceptions. The
> details for these instructions are discussed below.
>
> Instruction-based #VE
> ----------------------
> +~~~~~~~~~~~~~~~~~~~~~
>
> - Port I/O (INS, OUTS, IN, OUT)
> - HLT
> @@ -41,7 +228,7 @@ Instruction-based #VE
> - CPUID*
>
> Instruction-based #GP
> ----------------------
> +~~~~~~~~~~~~~~~~~~~~~
>
> - All VMX instructions: INVEPT, INVVPID, VMCLEAR, VMFUNC, VMLAUNCH,
> VMPTRLD, VMPTRST, VMREAD, VMRESUME, VMWRITE, VMXOFF, VMXON
> @@ -52,7 +239,7 @@ Instruction-based #GP
> - RDMSR*,WRMSR*
>
> RDMSR/WRMSR Behavior
> ---------------------
> +~~~~~~~~~~~~~~~~~~~~
>
> MSR access behavior falls into three categories:
>
> @@ -73,7 +260,7 @@ trapping and handling in the TDX module. Other than possibly being slow,
> these MSRs appear to function just as they would on bare metal.
>
> CPUID Behavior
> ---------------
> +~~~~~~~~~~~~~~
>
> For some CPUID leaves and sub-leaves, the virtualized bit fields of CPUID
> return values (in guest EAX/EBX/ECX/EDX) are configurable by the
> @@ -93,7 +280,7 @@ not know how to handle. The guest kernel may ask the hypervisor for the
> value with a hypercall.
>
> #VE on Memory Accesses
> -======================
> +----------------------
>
> There are essentially two classes of TDX memory: private and shared.
> Private memory receives full TDX protections. Its content is protected
> @@ -107,7 +294,7 @@ entries. This helps ensure that a guest does not place sensitive
> information in shared memory, exposing it to the untrusted hypervisor.
>
> #VE on Shared Memory
> ---------------------
> +~~~~~~~~~~~~~~~~~~~~
>
> Access to shared mappings can cause a #VE. The hypervisor ultimately
> controls whether a shared memory access causes a #VE, so the guest must be
> @@ -127,7 +314,7 @@ be careful not to access device MMIO regions unless it is also prepared to
> handle a #VE.
>
> #VE on Private Pages
> ---------------------
> +~~~~~~~~~~~~~~~~~~~~
>
> An access to private mappings can also cause a #VE. Since all kernel
> memory is also private memory, the kernel might theoretically need to
> @@ -145,7 +332,7 @@ The hypervisor is permitted to unilaterally move accepted pages to a
> to handle the exception.
>
> Linux #VE handler
> -=================
> +-----------------
>
> Just like page faults or #GP's, #VE exceptions can be either handled or be
> fatal. Typically, an unhandled userspace #VE results in a SIGSEGV.
> @@ -167,7 +354,7 @@ While the block is in place, any #VE is elevated to a double fault (#DF)
> which is not recoverable.
>
> MMIO handling
> -=============
> +-------------
>
> In non-TDX VMs, MMIO is usually implemented by giving a guest access to a
> mapping which will cause a VMEXIT on access, and then the hypervisor
> @@ -189,7 +376,7 @@ MMIO access via other means (like structure overlays) may result in an
> oops.
>
> Shared Memory Conversions
> -=========================
> +-------------------------
>
> All TDX guest memory starts out as private at boot. This memory can not
> be accessed by the hypervisor. However, some kernel users like device
No new htmldocs warnings but I think the wording can be better:
---- >8 ----
diff --git a/Documentation/x86/tdx.rst b/Documentation/x86/tdx.rst
index 59481dbe64b22a..5da707e2f83baa 100644
--- a/Documentation/x86/tdx.rst
+++ b/Documentation/x86/tdx.rst
@@ -26,8 +26,8 @@ BIOS is responsible for partitioning legacy MKTME KeyIDs and TDX KeyIDs.
Before the TDX module can be used to create and run protected VMs, it
must be loaded into the isolated range and properly initialized. The TDX
-architecture doesn't require the BIOS to load the TDX module, but the
-kernel assumes it is loaded by the BIOS.
+architecture doesn't require the BIOS to load the TDX module, however the
+kernel assumes that it is loaded by the BIOS.
TDX boot-time detection
-----------------------
@@ -40,49 +40,47 @@ boot. Below dmesg shows when TDX is enabled by BIOS::
TDX module detection and initialization
---------------------------------------
-There is no CPUID or MSR to detect the TDX module. The kernel detects it
-by initializing it.
+There is no CPUID or MSR to detect the TDX module. Instead, the kernel detects
+the module by initializing it.
-The kernel talks to the TDX module via the new SEAMCALL instruction. The
-TDX module implements SEAMCALL leaf functions to allow the kernel to
-initialize it.
+The kernel talks to the module via the new SEAMCALL instruction. The module
+implements SEAMCALL leaf functions to allow the kernel to initialize it.
-Initializing the TDX module consumes roughly ~1/256th system RAM size to
+Initializing the module consumes roughly ~1/256th system RAM size to
use it as 'metadata' for the TDX memory. It also takes additional CPU
time to initialize those metadata along with the TDX module itself. Both
-are not trivial. The kernel initializes the TDX module at runtime on
-demand. The caller to call tdx_enable() to initialize the TDX module::
+resource usages are not trivial. The kernel initializes the TDX module at
+runtime on demand. The caller calls tdx_enable() to initialize the module::
ret = tdx_enable();
if (ret)
goto no_tdx;
// TDX is ready to use
-Initializing the TDX module requires all logical CPUs being online.
-tdx_enable() internally temporarily disables CPU hotplug to prevent any
-CPU from going offline, but the caller still needs to guarantee all
-present CPUs are online before calling tdx_enable().
+Initializing the module requires all logical CPUs being online. tdx_enable()
+internally temporarily disables CPU hotplug to prevent any CPU from going
+offline, but the caller still needs to guarantee all present CPUs are online
+before calling tdx_enable().
Also, tdx_enable() requires all CPUs are already in VMX operation
(requirement of making SEAMCALL). Currently, tdx_enable() doesn't handle
VMXON internally, but depends on the caller to guarantee that. So far
KVM is the only user of TDX and KVM already handles VMXON.
-User can consult dmesg to see the presence of the TDX module, and whether
-it has been initialized.
+User can consult dmesg to see the module presence and initialization status.
-If the TDX module is not loaded, dmesg shows below::
+If the module is not loaded, dmesg shows below::
[..] tdx: TDX module is not loaded.
-If the TDX module is initialized successfully, dmesg shows something
+If the module is initialized successfully, dmesg shows something
like below::
[..] tdx: TDX module: attributes 0x0, vendor_id 0x8086, major_version 1, minor_version 0, build_date 20211209, build_num 160
[..] tdx: 65667 pages allocated for PAMT.
[..] tdx: TDX module initialized.
-If the TDX module failed to initialize, dmesg shows below::
+If the module fails to initialize, dmesg shows below::
[..] tdx: Failed to initialize TDX module. Shut it down.
@@ -98,17 +96,16 @@ generated by BIOS and verified by the MCHECK so that they are truly
present during platform boot and can meet security guarantees.
However those TDX convertible memory regions are not automatically usable
-to the TDX module. The kernel needs to choose all TDX-usable memory
-regions and pass those regions to the TDX module when initializing it.
-After TDX module is initialized, no more TDX-usable memory can be added
-to the TDX module.
+to the module. The kernel needs to choose all TDX-usable memory regions and
+pass these to the TDX module when initializing it. After the module is
+initialized, no more TDX-usable memory can be added to the module.
To keep things simple, this initial implementation chooses to use all
boot-time present memory managed by the page allocator as TDX memory.
This _requires_ all boot-time present memory is TDX convertible memory,
-which is true in practice. If there's any boot-time memory isn't TDX
+which is true in practice. If there's any boot-time memory that isn't TDX
convertible memory (which is allowed from TDX architecture's point of
-view), it will be caught later during TDX module initialization and the
+view), it will be caught later during the module initialization and the
initialization will fail.
However one machine may support both TDX and non-TDX memory both at
@@ -119,58 +116,57 @@ or not.
This raises a problem that, if any non-TDX memory is hot-added to the
system-wide memory allocation pool, a non-TDX page may be allocated to a
-TDX guest, which will result in failing to create the TDX guest, or
-killing it at runtime.
+TDX guest, which will result in failing to create the guest, or killing it
+at runtime.
The current implementation doesn't explicitly prevent adding any non-TDX
-memory to system-wide memory pool, but depends on the machine owner to
-make sure such operation won't happen. For example, the machine owner
-should never plug any NVDIMM or CXL memory to the machine, or use kmem
-driver to hot-add any to the core-mm.
+memory to system-wide memory pool, but depends on administrators to
+make sure such operation are prevented. For example, they should never
+plug any NVDIMM or CXL memory to the machine, or use kmem driver to
+hot-add these to the core-mm.
This will be enhanced in the future. One solution is the kernel can be
enforced to always guarantee all pages in the page allocator are TDX
memory (i.e. by rejecting non-TDX memory in memory hotplug). Another
-option is the kernel may support different memory capabilities on basis
-of NUMA node. For example, the kernel can have both TDX-compatible NUMA
-node and non-TDX-compatible memory NUMA node, and the userspace needs to
-explicitly bind TDX guests to those TDX-compatible memory NUMA nodes.
+option is the kernel may support different memory capabilities per NUMA node.
+For example, the kernel can have both TDX-compatible NUMA node and
+non-TDX-compatible counterpart, in which the userspace needs to explicitly
+bind TDX guests to the former.
CPU Hotplug
~~~~~~~~~~~
TDX doesn't support physical (ACPI) CPU hotplug. During machine boot,
-TDX verifies all boot-time present logical CPUs are TDX compatible before
-enabling TDX. A non-buggy BIOS should never support hot-add/removal of
+it verifies all boot-time present logical CPUs are TDX compatible before
+being enabled. A non-buggy BIOS should never support hot-add/removal of
physical CPU. Currently the kernel doesn't handle physical CPU hotplug,
-but depends on the BIOS to behave correctly.
+but assumes that BIOS behaves correctly.
-Note TDX works with CPU logical online/offline, thus the kernel still
-allows to offline logical CPU and online it again.
+Note that TDX works with logical CPU onlining/offlining, thus the kernel
+still allows to switch the logical CPU online or offline.
Memory Hotplug
~~~~~~~~~~~~~~
TDX doesn't support ACPI memory hotplug of convertible memory. The list
of "Convertible Memory Regions" (CMR) is static during machine's runtime.
-TDX also assumes convertible memory won't be hot-removed. A non-buggy
+It also assumes that convertible memory won't be hot-removed. A non-buggy
BIOS should never support physical hot-removal of convertible memory.
-Currently the kernel doesn't handle hot-removal of convertible memory but
-depends on the BIOS to behave correctly.
+Currently the kernel that hot-removal but assumes that BIOS behaves
+correctly.
It's possible that one machine can have both TDX and non-TDX memory.
Specifically, all hot-added physical memory are not TDX convertible
memory. Also, for now NVDIMM and CXL memory are not TDX convertible
memory, no matter whether they are physically present during boot or not.
-Plug non-TDX memory to the page allocator could result in failing to
-create a TDX guest, or killing a running TDX guest.
+Plugging in non-TDX memory to the page allocator could result in failing
+to create a TDX guest, or killing a running one.
To keep things simple, this series doesn't handle memory hotplug at all,
-but depends on the machine owner to not do any memory hotplug operation.
-For example, the machine owner should not plug any NVDIMM or CXL memory
-into the machine, or use kmem driver to plug NVDIMM or CXL memory to the
-core-mm.
+but depends on administrators to not doing that. For example, they should
+not plug any NVDIMM or CXL memory into the machine, or use kmem driver to
+plug it to the core-mm.
Kexec()
~~~~~~~
@@ -183,17 +179,17 @@ slightly corrupt the new kernel.
Similar to SME support, the kernel uses wbinvd() to flush cache in
stop_this_cpu().
-The current TDX module architecture doesn't play nicely with kexec().
-The TDX module can only be initialized once during its lifetime, and
-there is no SEAMCALL to reset the module to give a new clean slate to
+The current module architecture doesn't play nicely with kexec(). The
+module can only be initialized once during its lifetime, and there is no
+SEAMCALL to reset the module to give a new clean slate to
the new kernel. Therefore, ideally, if the module is ever initialized,
-it's better to shut down the module. The new kernel won't be able to
-use TDX anyway (as it needs to go through the TDX module initialization
-process which will fail immediately at the first step).
+it's better to remove the module. The new kernel won't be able to
+use TDX anyway (as it needs to go through the module initialization
+process again).
However, there's no guarantee CPU is in VMX operation during kexec(), so
-it's impractical to shut down the module. Currently, the kernel just
-leaves the module in open state.
+it's impractical to remove the module (reboot is required). Currently,
+the kernel just leaves the module in open state.
TDX Guest Support
=================
@@ -201,7 +197,7 @@ Since the host cannot directly access guest registers or memory, much
normal functionality of a hypervisor must be moved into the guest. This is
implemented using a Virtualization Exception (#VE) that is handled by the
guest kernel. A #VE is handled entirely inside the guest kernel, but some
-require the hypervisor to be consulted.
+require hypervisor assist.
TDX includes new hypercall-like mechanisms for communicating from the
guest to the hypervisor or the TDX module.
@@ -252,7 +248,7 @@ indicates a bug in the guest. The guest may try to handle the #GP with a
hypercall but it is unlikely to succeed.
The #VE MSRs are typically able to be handled by the hypervisor. Guests
-can make a hypercall to the hypervisor to handle the #VE.
+can make a hypercall to the hypervisor to handle them.
The "just works" MSRs do not need any special guest handling. They might
be implemented by directly passing through the MSR to the hardware or by
@@ -273,7 +269,7 @@ virtualization types:
- Bit fields for which the hypervisor configures the value such that the
guest TD either sees their native value or a value of 0. For these bit
fields, the hypervisor can mask off the native values, but it can not
- turn *on* values.
+ turn *on* them.
A #VE is generated for CPUID leaves and sub-leaves that the TDX module does
not know how to handle. The guest kernel may ask the hypervisor for the
@@ -309,9 +305,9 @@ stacks. A good rule of thumb is that hypervisor-shared memory should be
treated the same as memory mapped to userspace. Both the hypervisor and
userspace are completely untrusted.
-MMIO for virtual devices is implemented as shared memory. The guest must
-be careful not to access device MMIO regions unless it is also prepared to
-handle a #VE.
+MMIO for virtual devices is implemented as shared memory. As such, the
+guest must be careful not to access device MMIO regions unless it is also
+prepared to handle a #VE.
#VE on Private Pages
~~~~~~~~~~~~~~~~~~~~
@@ -334,8 +330,8 @@ to handle the exception.
Linux #VE handler
-----------------
-Just like page faults or #GP's, #VE exceptions can be either handled or be
-fatal. Typically, an unhandled userspace #VE results in a SIGSEGV.
+Just like page faults or #GP's, #VE exceptions can be either handled or
+turned into fatal. Typically, an unhandled userspace #VE results in SIGSEGV.
An unhandled kernel #VE results in an oops.
Handling nested exceptions on x86 is typically nasty business. A #VE
@@ -345,12 +341,12 @@ feature to make it slightly less nasty.
During #VE handling, the TDX module ensures that all interrupts (including
NMIs) are blocked. The block remains in place until the guest makes a
-TDG.VP.VEINFO.GET TDCALL. This allows the guest to control when interrupts
+TDG.VP.VEINFO.GET TDCALL. This allows the guest to control when to interrupt
or a new #VE can be delivered.
However, the guest kernel must still be careful to avoid potential
#VE-triggering actions (discussed above) while this block is in place.
-While the block is in place, any #VE is elevated to a double fault (#DF)
+While this is the case, any #VE is raised to a double fault (#DF)
which is not recoverable.
MMIO handling
@@ -359,7 +355,7 @@ MMIO handling
In non-TDX VMs, MMIO is usually implemented by giving a guest access to a
mapping which will cause a VMEXIT on access, and then the hypervisor
emulates the access. That is not possible in TDX guests because VMEXIT
-will expose the register state to the host. TDX guests don't trust the host
+will expose the register state to the host. They don't trust the host
and can't have their state exposed to the host.
In TDX, MMIO regions typically trigger a #VE exception in the guest. The
@@ -382,13 +378,13 @@ All TDX guest memory starts out as private at boot. This memory can not
be accessed by the hypervisor. However, some kernel users like device
drivers might have a need to share data with the hypervisor. To do this,
memory must be converted between shared and private. This can be
-accomplished using some existing memory encryption helpers:
+accomplished using following existing memory encryption helpers:
* set_memory_decrypted() converts a range of pages to shared.
* set_memory_encrypted() converts memory back to private.
-Device drivers are the primary user of shared memory, but there's no need
-to touch every driver. DMA buffers and ioremap() do the conversions
+Device drivers are the primary user of shared memory, however there's no
+need to touch every driver. DMA buffers and ioremap() do the conversions
automatically.
TDX uses SWIOTLB for most DMA allocations. The SWIOTLB buffer is
Thanks.
--
An old man doll... just what I always wanted! - Clara
On 10/28/22 05:52, Bagas Sanjaya wrote:
> -architecture doesn't require the BIOS to load the TDX module, but the
> -kernel assumes it is loaded by the BIOS.
> +architecture doesn't require the BIOS to load the TDX module, however the
> +kernel assumes that it is loaded by the BIOS.
Hi Bagas,
I just read the first hunk of your suggestions. What Kai had was fine.
There's no reason to change "but" to "however". Both are, to my eye,
perfectly fine.
I appreciate that these suggestions are trying to improve things. But,
I don't think they're an appreciable improvement.
OK, I lied. I went and read one more random hunk:
> -Currently the kernel doesn't handle hot-removal of convertible memory but
> -depends on the BIOS to behave correctly.
> +Currently the kernel that hot-removal but assumes that BIOS behaves
> +correctly.
This turns a perfectly good sentence into gibberish. It makes Kai's
documentation demonstrably worse. To make matters worse, it's mixed in
with those arbitrary changes like but->however to make it harder to find.
Please stop sending these patches. They're not helping. In fact, they
are consuming reviewer and contributor time, so they're actually making
the situation _worse_.
On Fri, 2022-10-28 at 02:21 +0000, Huang, Kai wrote:
> On Thu, 2022-10-27 at 08:31 -0700, Andi Kleen wrote:
> > > +/* Calculate the actual TDMR_INFO size */
> > > +static inline int cal_tdmr_size(void)
> > > +{
> > > + int tdmr_sz;
> > > +
> > > + /*
> > > + * The actual size of TDMR_INFO depends on the maximum number
> > > + * of reserved areas.
> > > + */
> > > + tdmr_sz = sizeof(struct tdmr_info);
> > > + tdmr_sz += sizeof(struct tdmr_reserved_area) *
> > > + tdx_sysinfo.max_reserved_per_tdmr;
> >
> >
> > would seem safer to have a overflow check here.
> >
> >
>
> How about below?
>
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -614,6 +614,14 @@ static inline int cal_tdmr_size(void)
> tdmr_sz += sizeof(struct tdmr_reserved_area) *
> tdx_sysinfo.max_reserved_per_tdmr;
>
> + /*
> + * Do simple check against overflow, and return 0 (invalid)
> + * TDMR_INFO size if it happened. Also WARN() as it should
> + * should never happen in reality.
> + */
> + if (WARN_ON_ONCE(tdmr_sz < 0))
> + return 0;
> +
> /*
> * TDX requires each TDMR_INFO to be 512-byte aligned. Always
> * round up TDMR_INFO size to the 512-byte boundary.
> @@ -623,19 +631,27 @@ static inline int cal_tdmr_size(void)
>
> static struct tdmr_info *alloc_tdmr_array(int *array_sz)
> {
> + int sz;
> +
> /*
> * TDX requires each TDMR_INFO to be 512-byte aligned.
> * Use alloc_pages_exact() to allocate all TDMRs at once.
> * Each TDMR_INFO will still be 512-byte aligned since
> * cal_tdmr_size() always return 512-byte aligned size.
> */
> - *array_sz = cal_tdmr_size() * tdx_sysinfo.max_tdmrs;
> + sz = cal_tdmr_size() * tdx_sysinfo.max_tdmrs;
> +
> + /* Overflow */
> + if (!sz || WARN_ON_ONCE(sz < 0))
> + return NULL;
> +
> + *array_sz = sz;
>
> /*
> * Zero the buffer so 'struct tdmr_info::size' can be
> * used to determine whether a TDMR is valid.
> */
> - return alloc_pages_exact(*array_sz, GFP_KERNEL | __GFP_ZERO);
> + return alloc_pages_exact(sz, GFP_KERNEL | __GFP_ZERO);
> }
>
>
> Btw, should I use alloc_contig_pages() instead of alloc_pages_exact() as IIUC
> the latter should fail if the size is larger than 4MB? In reality, the entire
> array only takes dozens of KBs, though.
Hi Andi,
Could you take a look whether this is OK?
Also could you take a look my replies to your other comments?
Thanks!
On 10/27/22 08:31, Andi Kleen wrote:
>
>> +/* Calculate the actual TDMR_INFO size */
>> +static inline int cal_tdmr_size(void)
>> +{
>> + int tdmr_sz;
>> +
>> + /*
>> + * The actual size of TDMR_INFO depends on the maximum number
>> + * of reserved areas.
>> + */
>> + tdmr_sz = sizeof(struct tdmr_info);
>> + tdmr_sz += sizeof(struct tdmr_reserved_area) *
>> + tdx_sysinfo.max_reserved_per_tdmr;
>
> would seem safer to have a overflow check here.
tdmr_reserved_area is 16 bytes. To overflow a signed int, tdmr_sz would
need to be for an allocation >2GB. alloc_pages_exact() tops out at
supplying 4MB allocations.
So, sure, this breaks at max_reserved_per_tdmr>2^27, but it actually
breaks *EARLIER* at max_reserved_per_tdmr>2^18 because the page
allocator is borked.
Plus, max_reserved_per_tdmr is barely in double digits today. It's a
*LOOOOOOOOONG* way from either of those limits. If you want to add a
warning here, then go for it and enforce a sane value on
max_reserved_per_tdmr.
But, the overflow is *LITERALLY* an order of magnitude more obscure than
overwhelming the page allocator. Let's not clutter up the code with
silly checks like that.
On Thu, 2022-11-03 at 08:05 -0700, Hansen, Dave wrote:
> Plus, max_reserved_per_tdmr is barely in double digits today. It's a
> *LOOOOOOOOONG* way from either of those limits. If you want to add a
> warning here, then go for it and enforce a sane value on
> max_reserved_per_tdmr.
Hi Dave,
Thanks. By "enforce a sane value on max_reserved_per_tdmr' could you be more
specific? Did you mean if we find its value is insanely big, we can change it to
a reasonable smaller value?
But I don't think we can as the TDMR_INFO is used by the TDX module, so reducing
max_reserved_per_tdmr by the kernel doesn't actually work?
Perhaps for now we can make the kernel to assume TDMR_INFO won't exceed a
reasonable value (i.e. 4K/8K/16K?) and max_tdmrs (which is 64 currently) won't
exceed a reasonable value either (i.e. 1K/512/256?), so that we can just use
alloc_pages_exact() to allocate the entire TDMR array? If kernel found either
is too big, then kernel could just fail to initialize the TDX module.