2024-02-09 19:06:00

by Paolo Bonzini

[permalink] [raw]
Subject: [PATCH 00/10] KVM: SEV: allow customizing VMSA features

The idea that no parameter would ever be necessary when enabling SEV or
SEV-ES for a VM was decidedly optimistic. The first source of variability
that was encountered is the desired set of VMSA features, as that affects
the measurement of the VM's initial state and cannot be changed
arbitrarily by the hypervisor.

This series adds all the APIs that are needed to customize the features,
with room for future enhancements:

- a new /dev/kvm device attribute to retrieve the set of supported
features (right now, only debug swap)

- a new sub-operation for KVM_MEM_ENCRYPT_OP that can take a struct,
replacing the existing KVM_SEV_INIT and KVM_SEV_ES_INIT

It then puts the new op to work by including the VMSA features as a field
of the The existing KVM_SEV_INIT and KVM_SEV_ES_INIT use the full set of
supported VMSA features for backwards compatibility; but I am considering
also making them use zero as the feature mask, and will gladly adjust the
patches if so requested.

In order to avoid creating *two* new KVM_MEM_ENCRYPT_OPs, I decided that
I could as well make SEV and SEV-ES use VM types. And then, why not make
a SEV-ES VM, when created with the new VM type instead of KVM_SEV_ES_INIT,
reject KVM_GET_REGS/KVM_SET_REGS and friends on the vCPU file descriptor
once the VMSA has been encrypted... Which is how the API should have
always behaved.

The series is defined as follows:

- patches 1 and 2 are unrelated fixes and improvements for the SEV API

- patches 3 to 5 introduce the new device attribute to retrieve supported
VMSA features

- patches 6 to 7 introduce new infrastructure for VM types, partly lifted
out of the TDX patches

- patches 8 and 9 introduce respectively the new VM types for SEV and
SEV-ES, and KVM_SEV_INIT2 as a new sub-operation for KVM_MEM_ENCRYPT_OP.

- patches 10 and 11 are tests. The last patch is not intended to be applied
in order to keep some coverage of KVM_SEV_INIT and KVM_SEV_ES_INIT in
self tests; but it is there as "proof" that migration can be made to
work with the new API as well.


The idea is that SEV SNP will only ever support KVM_SEV_INIT2. I have
patches in progress for QEMU to support this new API.

Thanks,

Paolo

Isaku Yamahata (1):
KVM: x86: Add is_vm_type_supported callback

Paolo Bonzini (10):
KVM: x86: define standard behavior for bits 0/1 of VM type
KVM: SEV: fix compat ABI for KVM_MEMORY_ENCRYPT_OP
KVM: introduce new vendor op for KVM_GET_DEVICE_ATTR
Documentation: kvm/sev: separate description of firmware
KVM: SEV: publish supported VMSA features
KVM: SEV: store VMSA features in kvm_sev_info
KVM: SEV: define VM types for SEV and SEV-ES
KVM: SEV: introduce KVM_SEV_INIT2 operation
selftests: kvm: add tests for KVM_SEV_INIT2
selftests: kvm: switch sev_migrate_tests to KVM_SEV_INIT2

Documentation/virt/kvm/api.rst | 2 +
.../virt/kvm/x86/amd-memory-encryption.rst | 81 +++++++--
arch/x86/include/asm/kvm-x86-ops.h | 2 +
arch/x86/include/asm/kvm_host.h | 4 +-
arch/x86/include/uapi/asm/kvm.h | 42 ++++-
arch/x86/kvm/svm/sev.c | 104 +++++++++++-
arch/x86/kvm/svm/svm.c | 14 +-
arch/x86/kvm/svm/svm.h | 6 +-
arch/x86/kvm/x86.c | 158 ++++++++++++++----
arch/x86/kvm/x86.h | 2 +
tools/testing/selftests/kvm/Makefile | 1 +
.../selftests/kvm/include/kvm_util_base.h | 6 +-
.../selftests/kvm/set_memory_region_test.c | 8 +-
.../selftests/kvm/x86_64/sev_init2_tests.c | 147 ++++++++++++++++
.../selftests/kvm/x86_64/sev_migrate_tests.c | 45 ++---
15 files changed, 530 insertions(+), 92 deletions(-)
create mode 100644 tools/testing/selftests/kvm/x86_64/sev_init2_tests.c

--
2.39.0




2024-02-09 19:07:43

by Paolo Bonzini

[permalink] [raw]
Subject: [PATCH 03/10] Documentation: kvm/sev: separate description of firmware

The description of firmware is included part under the "SEV Key Management"
header, part under the KVM_SEV_INIT ioctl. Put these two bits together and
and rename "SEV Key Management" to what it actually is, namely a description
of the KVM_MEMORY_ENCRYPT_OP API.

Signed-off-by: Paolo Bonzini <[email protected]>
---
.../virt/kvm/x86/amd-memory-encryption.rst | 29 +++++++++++--------
1 file changed, 17 insertions(+), 12 deletions(-)

diff --git a/Documentation/virt/kvm/x86/amd-memory-encryption.rst b/Documentation/virt/kvm/x86/amd-memory-encryption.rst
index 995780088eb2..37c5c37f4f6e 100644
--- a/Documentation/virt/kvm/x86/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/x86/amd-memory-encryption.rst
@@ -46,14 +46,8 @@ SEV hardware uses ASIDs to associate a memory encryption key with a VM.
Hence, the ASID for the SEV-enabled guests must be from 1 to a maximum value
defined in the CPUID 0x8000001f[ecx] field.

-SEV Key Management
-==================
-
-The SEV guest key management is handled by a separate processor called the AMD
-Secure Processor (AMD-SP). Firmware running inside the AMD-SP provides a secure
-key management interface to perform common hypervisor activities such as
-encrypting bootstrap code, snapshot, migrating and debugging the guest. For more
-information, see the SEV Key Management spec [api-spec]_
+``KVM_MEMORY_ENCRYPT_OP`` API
+=============================

The main ioctl to access SEV is KVM_MEMORY_ENCRYPT_OP. If the argument
to KVM_MEMORY_ENCRYPT_OP is NULL, the ioctl returns 0 if SEV is enabled
@@ -87,10 +81,6 @@ guests, such as launching, running, snapshotting, migrating and decommissioning.
The KVM_SEV_INIT command is used by the hypervisor to initialize the SEV platform
context. In a typical workflow, this command should be the first command issued.

-The firmware can be initialized either by using its own non-volatile storage or
-the OS can manage the NV storage for the firmware using the module parameter
-``init_ex_path``. If the file specified by ``init_ex_path`` does not exist or
-is invalid, the OS will create or override the file with output from PSP.

Returns: 0 on success, -negative on error

@@ -434,6 +424,21 @@ issued by the hypervisor to make the guest ready for execution.

Returns: 0 on success, -negative on error

+Firmware Management
+===================
+
+The SEV guest key management is handled by a separate processor called the AMD
+Secure Processor (AMD-SP). Firmware running inside the AMD-SP provides a secure
+key management interface to perform common hypervisor activities such as
+encrypting bootstrap code, snapshot, migrating and debugging the guest. For more
+information, see the SEV Key Management spec [api-spec]_
+
+The AMD-SP firmware can be initialized either by using its own non-volatile
+storage or the OS can manage the NV storage for the firmware using
+parameter ``init_ex_path`` of the ``ccp`` module. If the file specified
+by ``init_ex_path`` does not exist or is invalid, the OS will create or
+override the file with PSP non-volatile storage.
+
References
==========

--
2.39.0



2024-02-09 19:10:49

by Paolo Bonzini

[permalink] [raw]
Subject: [PATCH 09/10] KVM: SEV: introduce KVM_SEV_INIT2 operation

The idea that no parameter would ever be necessary when enabling SEV or
SEV-ES for a VM was decidedly optimistic. In fact, in some sense it's
already a parameter whether SEV or SEV-ES is desired. Another possible
source of variability is the desired set of VMSA features, as that affects
the measurement of the VM's initial state and cannot be changed
arbitrarily by the hypervisor.

Create a new sub-operation for KVM_MEM_ENCRYPT_OP that can take a struct,
and put the new op to work by including the VMSA features as a field of the
struct. The existing KVM_SEV_INIT and KVM_SEV_ES_INIT use the full set of
supported VMSA features for backwards compatibility.

The struct also includes the usual bells and whistles for future
extensibility: a flags field that must be zero for now, and some padding
at the end.

Signed-off-by: Paolo Bonzini <[email protected]>
---
.../virt/kvm/x86/amd-memory-encryption.rst | 41 ++++++++++++++--
arch/x86/include/uapi/asm/kvm.h | 10 ++++
arch/x86/kvm/svm/sev.c | 48 +++++++++++++++++--
3 files changed, 92 insertions(+), 7 deletions(-)

diff --git a/Documentation/virt/kvm/x86/amd-memory-encryption.rst b/Documentation/virt/kvm/x86/amd-memory-encryption.rst
index 5ed11bc16b96..a4291e7bd8ed 100644
--- a/Documentation/virt/kvm/x86/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/x86/amd-memory-encryption.rst
@@ -75,15 +75,50 @@ are defined in ``<linux/psp-dev.h>``.
KVM implements the following commands to support common lifecycle events of SEV
guests, such as launching, running, snapshotting, migrating and decommissioning.

-1. KVM_SEV_INIT
----------------
+1. KVM_SEV_INIT2
+----------------

-The KVM_SEV_INIT command is used by the hypervisor to initialize the SEV platform
+The KVM_SEV_INIT2 command is used by the hypervisor to initialize the SEV platform
context. In a typical workflow, this command should be the first command issued.

+For this command to be accepted, either KVM_X86_SEV_VM or KVM_X86_SEV_ES_VM
+must have been passed to the KVM_CREATE_VM ioctl. A virtual machine created
+with those machine types in turn cannot be run until KVM_SEV_INIT2 is invoked.
+
+Parameters: struct kvm_sev_init (in)

Returns: 0 on success, -negative on error

+::
+
+ struct struct kvm_sev_init {
+ __u32 flags; /* must be 0 */
+ __u64 vmsa_features; /* initial value of features field in VMSA */
+ __u32 pad[8];
+ };
+
+It is an error if the hypervisor does not support any of the bits that
+are set in ``flags`` or ``vmsa_features``.
+
+This command replaces the deprecated KVM_SEV_INIT and KVM_SEV_ES_INIT commands.
+The commands did not have any parameters (the ```data``` field was unused) and
+only work for the KVM_X86_DEFAULT_VM machine type (0).
+
+They behave as if:
+
+* the VM type is KVM_X86_SEV_VM for KVM_SEV_INIT, or KVM_X86_SEV_ES_VM for
+ KVM_SEV_ES_INIT
+
+* the ``flags`` field of ``struct kvm_sev_init`` is set to zero
+
+* the ``vmsa_features`` field of ``struct kvm_sev_init`` is set to all features
+ supported by the hypervisor (as returned by ``KVM_GET_DEVICE_ATTR`` when
+ passed group 0 and attribute id ``KVM_X86_SEV_VMSA_FEATURES``).
+
+If the ``KVM_X86_SEV_VMSA_FEATURES`` attribute does not exist, the hypervisor only
+supports KVM_SEV_INIT and KVM_SEV_ES_INIT. In that case the set of VMSA features is
+undefined.
+
2. KVM_SEV_LAUNCH_START
-----------------------

diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index 7c46e96cfe62..6baf18335c7b 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -683,6 +683,9 @@ enum sev_cmd_id {
/* Guest Migration Extension */
KVM_SEV_SEND_CANCEL,

+ /* Second time is the charm; improved versions of the above ioctls. */
+ KVM_SEV_INIT2,
+
KVM_SEV_NR_MAX,
};

@@ -694,6 +697,13 @@ struct kvm_sev_cmd {
__u32 sev_fd;
};

+struct kvm_sev_init {
+ __u32 vm_type;
+ __u32 flags;
+ __u64 vmsa_features;
+ __u32 pad[8];
+};
+
struct kvm_sev_launch_start {
__u32 handle;
__u32 policy;
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index acf5c45ef14e..78c52764453f 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -252,7 +252,9 @@ static void sev_unbind_asid(struct kvm *kvm, unsigned int handle)
sev_decommission(handle);
}

-static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
+static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp,
+ struct kvm_sev_init *data,
+ unsigned long vm_type)
{
struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
int asid, ret;
@@ -260,7 +262,10 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
if (kvm->created_vcpus)
return -EINVAL;

- if (kvm->arch.vm_type != KVM_X86_DEFAULT_VM)
+ if (data->flags)
+ return -EINVAL;
+
+ if (data->vmsa_features & ~sev_supported_vmsa_features)
return -EINVAL;

ret = -EBUSY;
@@ -268,8 +273,8 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
return ret;

sev->active = true;
- sev->es_active = argp->id == KVM_SEV_ES_INIT;
- sev->vmsa_features = sev_supported_vmsa_features;
+ sev->es_active = (vm_type & __KVM_X86_PROTECTED_STATE_TYPE) != 0;
+ sev->vmsa_features = data->vmsa_features;

asid = sev_asid_new(sev);
if (asid < 0)
@@ -298,6 +303,38 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
return ret;
}

+static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+ struct kvm_sev_init data = {
+ .vmsa_features = sev_supported_vmsa_features,
+ };
+ unsigned long vm_type;
+
+ if (kvm->arch.vm_type != KVM_X86_DEFAULT_VM)
+ return -EINVAL;
+
+ vm_type = (argp->id == KVM_SEV_INIT ? KVM_X86_SEV_VM : KVM_X86_SEV_ES_VM);
+ return __sev_guest_init(kvm, argp, &data, vm_type);
+}
+
+static int sev_guest_init2(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+ struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+ struct kvm_sev_init data;
+
+ if (!sev->need_init)
+ return -EINVAL;
+
+ if (kvm->arch.vm_type != KVM_X86_SEV_VM &&
+ kvm->arch.vm_type != KVM_X86_SEV_ES_VM)
+ return -EINVAL;
+
+ if (copy_from_user(&data, (void __user *)(uintptr_t)argp->data, sizeof(data)))
+ return -EFAULT;
+
+ return __sev_guest_init(kvm, argp, &data, kvm->arch.vm_type);
+}
+
static int sev_bind_asid(struct kvm *kvm, unsigned int handle, int *error)
{
struct sev_data_activate activate;
@@ -1915,6 +1952,9 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
case KVM_SEV_INIT:
r = sev_guest_init(kvm, &sev_cmd);
break;
+ case KVM_SEV_INIT2:
+ r = sev_guest_init2(kvm, &sev_cmd);
+ break;
case KVM_SEV_LAUNCH_START:
r = sev_launch_start(kvm, &sev_cmd);
break;
--
2.39.0



2024-02-09 19:11:13

by Paolo Bonzini

[permalink] [raw]
Subject: [PATCH 11/10] selftests: kvm: switch sev_migrate_tests to KVM_SEV_INIT2

Signed-off-by: Paolo Bonzini <[email protected]>
---
.../selftests/kvm/x86_64/sev_migrate_tests.c | 45 ++++++++++---------
1 file changed, 23 insertions(+), 22 deletions(-)

diff --git a/tools/testing/selftests/kvm/x86_64/sev_migrate_tests.c b/tools/testing/selftests/kvm/x86_64/sev_migrate_tests.c
index c7ef97561038..301f7083cad0 100644
--- a/tools/testing/selftests/kvm/x86_64/sev_migrate_tests.c
+++ b/tools/testing/selftests/kvm/x86_64/sev_migrate_tests.c
@@ -50,11 +50,12 @@ static void sev_ioctl(int vm_fd, int cmd_id, void *data)
static struct kvm_vm *sev_vm_create(bool es)
{
struct kvm_vm *vm;
+ struct kvm_sev_init init = { 0 };
struct kvm_sev_launch_start start = { 0 };
int i;

- vm = vm_create_barebones();
- sev_ioctl(vm->fd, es ? KVM_SEV_ES_INIT : KVM_SEV_INIT, NULL);
+ vm = vm_create_barebones_type(es ? KVM_X86_SEV_ES_VM : KVM_X86_SEV_VM);
+ sev_ioctl(vm->fd, KVM_SEV_INIT2, &init);
for (i = 0; i < NR_MIGRATE_TEST_VCPUS; ++i)
__vm_vcpu_add(vm, i);
if (es)
@@ -65,12 +66,12 @@ static struct kvm_vm *sev_vm_create(bool es)
return vm;
}

-static struct kvm_vm *aux_vm_create(bool with_vcpus)
+static struct kvm_vm *aux_vm_create(bool es, bool with_vcpus)
{
struct kvm_vm *vm;
int i;

- vm = vm_create_barebones();
+ vm = vm_create_barebones_type(es ? KVM_X86_SEV_ES_VM : KVM_X86_SEV_VM);
if (!with_vcpus)
return vm;

@@ -102,7 +103,7 @@ static void test_sev_migrate_from(bool es)

src_vm = sev_vm_create(es);
for (i = 0; i < NR_MIGRATE_TEST_VMS; ++i)
- dst_vms[i] = aux_vm_create(true);
+ dst_vms[i] = aux_vm_create(es, true);

/* Initial migration from the src to the first dst. */
sev_migrate_from(dst_vms[0], src_vm);
@@ -164,16 +165,17 @@ static void test_sev_migrate_locking(void)

static void test_sev_migrate_parameters(void)
{
- struct kvm_vm *sev_vm, *sev_es_vm, *vm_no_vcpu, *vm_no_sev,
+ struct kvm_vm *sev_vm, *sev_es_vm, *vm_no_vcpu,
*sev_es_vm_no_vmsa;
int ret;

vm_no_vcpu = vm_create_barebones();
- vm_no_sev = aux_vm_create(true);
- ret = __sev_migrate_from(vm_no_vcpu, vm_no_sev);
+ sev_vm = aux_vm_create(false, true);
+ ret = __sev_migrate_from(vm_no_vcpu, sev_vm);
TEST_ASSERT(ret == -1 && errno == EINVAL,
"Migrations require SEV enabled. ret %d, errno: %d\n", ret,
errno);
+ kvm_vm_free(sev_vm);

if (!have_sev_es)
goto out;
@@ -213,7 +215,6 @@ static void test_sev_migrate_parameters(void)
kvm_vm_free(sev_es_vm_no_vmsa);
out:
kvm_vm_free(vm_no_vcpu);
- kvm_vm_free(vm_no_sev);
}

static int __sev_mirror_create(struct kvm_vm *dst, struct kvm_vm *src)
@@ -272,7 +273,7 @@ static void test_sev_mirror(bool es)
int i;

src_vm = sev_vm_create(es);
- dst_vm = aux_vm_create(false);
+ dst_vm = aux_vm_create(es, false);

sev_mirror_create(dst_vm, src_vm);

@@ -295,8 +296,8 @@ static void test_sev_mirror_parameters(void)
int ret;

sev_vm = sev_vm_create(/* es= */ false);
- vm_with_vcpu = aux_vm_create(true);
- vm_no_vcpu = aux_vm_create(false);
+ vm_with_vcpu = aux_vm_create(false, true);
+ vm_no_vcpu = aux_vm_create(false, false);

ret = __sev_mirror_create(sev_vm, sev_vm);
TEST_ASSERT(
@@ -345,13 +346,13 @@ static void test_sev_move_copy(void)
*dst_mirror_vm, *dst2_mirror_vm, *dst3_mirror_vm;

sev_vm = sev_vm_create(/* es= */ false);
- dst_vm = aux_vm_create(true);
- dst2_vm = aux_vm_create(true);
- dst3_vm = aux_vm_create(true);
- mirror_vm = aux_vm_create(false);
- dst_mirror_vm = aux_vm_create(false);
- dst2_mirror_vm = aux_vm_create(false);
- dst3_mirror_vm = aux_vm_create(false);
+ dst_vm = aux_vm_create(false, true);
+ dst2_vm = aux_vm_create(false, true);
+ dst3_vm = aux_vm_create(false, true);
+ mirror_vm = aux_vm_create(false, false);
+ dst_mirror_vm = aux_vm_create(false, false);
+ dst2_mirror_vm = aux_vm_create(false, false);
+ dst3_mirror_vm = aux_vm_create(false, false);

sev_mirror_create(mirror_vm, sev_vm);

@@ -378,9 +379,9 @@ static void test_sev_move_copy(void)
* destruction is done safely.
*/
sev_vm = sev_vm_create(/* es= */ false);
- dst_vm = aux_vm_create(true);
- mirror_vm = aux_vm_create(false);
- dst_mirror_vm = aux_vm_create(false);
+ dst_vm = aux_vm_create(false, true);
+ mirror_vm = aux_vm_create(false, false);
+ dst_mirror_vm = aux_vm_create(false, false);

sev_mirror_create(mirror_vm, sev_vm);

--
2.39.0


2024-02-09 19:13:13

by Paolo Bonzini

[permalink] [raw]
Subject: [PATCH 02/10] KVM: introduce new vendor op for KVM_GET_DEVICE_ATTR

Allow vendor modules to provide their own attributes on /dev/kvm.
To avoid proliferation of vendor ops, implement KVM_HAS_DEVICE_ATTR
and KVM_GET_DEVICE_ATTR in terms of the same function. You're not
supposed to use KVM_GET_DEVICE_ATTR to do complicated computations,
especially on /dev/kvm.

Signed-off-by: Paolo Bonzini <[email protected]>
---
arch/x86/include/asm/kvm-x86-ops.h | 1 +
arch/x86/include/asm/kvm_host.h | 1 +
arch/x86/kvm/x86.c | 52 +++++++++++++++++++-----------
3 files changed, 36 insertions(+), 18 deletions(-)

diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h
index 378ed944b849..ac8b7614e79d 100644
--- a/arch/x86/include/asm/kvm-x86-ops.h
+++ b/arch/x86/include/asm/kvm-x86-ops.h
@@ -122,6 +122,7 @@ KVM_X86_OP(enter_smm)
KVM_X86_OP(leave_smm)
KVM_X86_OP(enable_smi_window)
#endif
+KVM_X86_OP_OPTIONAL(dev_get_attr)
KVM_X86_OP_OPTIONAL(mem_enc_ioctl)
KVM_X86_OP_OPTIONAL(mem_enc_register_region)
KVM_X86_OP_OPTIONAL(mem_enc_unregister_region)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index d271ba20a0b2..0bcd9ae16097 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1769,6 +1769,7 @@ struct kvm_x86_ops {
void (*enable_smi_window)(struct kvm_vcpu *vcpu);
#endif

+ int (*dev_get_attr)(u64 attr, u64 *val);
int (*mem_enc_ioctl)(struct kvm *kvm, void __user *argp);
int (*mem_enc_register_region)(struct kvm *kvm, struct kvm_enc_region *argp);
int (*mem_enc_unregister_region)(struct kvm *kvm, struct kvm_enc_region *argp);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index bf10a9073a09..8746530930d5 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -4804,37 +4804,53 @@ static inline void __user *kvm_get_attr_addr(struct kvm_device_attr *attr)
return uaddr;
}

-static int kvm_x86_dev_get_attr(struct kvm_device_attr *attr)
+static int __kvm_x86_dev_get_attr(struct kvm_device_attr *attr, u64 *val)
{
- u64 __user *uaddr = kvm_get_attr_addr(attr);
+ int r;

if (attr->group)
return -ENXIO;

+ switch (attr->attr) {
+ case KVM_X86_XCOMP_GUEST_SUPP:
+ r = 0;
+ *val = kvm_caps.supported_xcr0;
+ break;
+ default:
+ r = -ENXIO;
+ if (kvm_x86_ops.dev_get_attr)
+ r = kvm_x86_ops.dev_get_attr(attr->attr, val);
+ break;
+ }
+
+ return r;
+}
+
+static int kvm_x86_dev_get_attr(struct kvm_device_attr *attr)
+{
+ u64 __user *uaddr;
+ int r;
+ u64 val;
+
+ r = __kvm_x86_dev_get_attr(attr, &val);
+ if (r < 0)
+ return r;
+
+ uaddr = kvm_get_attr_addr(attr);
if (IS_ERR(uaddr))
return PTR_ERR(uaddr);

- switch (attr->attr) {
- case KVM_X86_XCOMP_GUEST_SUPP:
- if (put_user(kvm_caps.supported_xcr0, uaddr))
- return -EFAULT;
- return 0;
- default:
- return -ENXIO;
- }
+ if (put_user(val, uaddr))
+ return -EFAULT;
+
+ return 0;
}

static int kvm_x86_dev_has_attr(struct kvm_device_attr *attr)
{
- if (attr->group)
- return -ENXIO;
+ u64 val;

- switch (attr->attr) {
- case KVM_X86_XCOMP_GUEST_SUPP:
- return 0;
- default:
- return -ENXIO;
- }
+ return __kvm_x86_dev_get_attr(attr, &val);
}

long kvm_arch_dev_ioctl(struct file *filp,
--
2.39.0



2024-02-09 19:13:28

by Paolo Bonzini

[permalink] [raw]
Subject: [PATCH 01/10] KVM: SEV: fix compat ABI for KVM_MEMORY_ENCRYPT_OP

The data structs for KVM_MEMORY_ENCRYPT_OP have different sizes for 32- and 64-bit
kernels, but they do not make any attempt to convert from one ABI to the other.
Fix this by adding the appropriate padding.

No functional change intended for 64-bit userspace.

Signed-off-by: Paolo Bonzini <[email protected]>
---
arch/x86/include/uapi/asm/kvm.h | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index 0ad6bda1fc39..b305daff056e 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -687,6 +687,7 @@ enum sev_cmd_id {

struct kvm_sev_cmd {
__u32 id;
+ __u32 pad0;
__u64 data;
__u32 error;
__u32 sev_fd;
@@ -697,28 +698,35 @@ struct kvm_sev_launch_start {
__u32 policy;
__u64 dh_uaddr;
__u32 dh_len;
+ __u32 pad0;
__u64 session_uaddr;
__u32 session_len;
+ __u32 pad1;
};

struct kvm_sev_launch_update_data {
__u64 uaddr;
__u32 len;
+ __u32 pad0;
};


struct kvm_sev_launch_secret {
__u64 hdr_uaddr;
__u32 hdr_len;
+ __u32 pad0;
__u64 guest_uaddr;
__u32 guest_len;
+ __u32 pad1;
__u64 trans_uaddr;
__u32 trans_len;
+ __u32 pad2;
};

struct kvm_sev_launch_measure {
__u64 uaddr;
__u32 len;
+ __u32 pad0;
};

struct kvm_sev_guest_status {
@@ -731,33 +739,43 @@ struct kvm_sev_dbg {
__u64 src_uaddr;
__u64 dst_uaddr;
__u32 len;
+ __u32 pad0;
};

struct kvm_sev_attestation_report {
__u8 mnonce[16];
__u64 uaddr;
__u32 len;
+ __u32 pad0;
};

struct kvm_sev_send_start {
__u32 policy;
+ __u32 pad0;
__u64 pdh_cert_uaddr;
__u32 pdh_cert_len;
+ __u32 pad1;
__u64 plat_certs_uaddr;
__u32 plat_certs_len;
+ __u32 pad2;
__u64 amd_certs_uaddr;
__u32 amd_certs_len;
+ __u32 pad3;
__u64 session_uaddr;
__u32 session_len;
+ __u32 pad4;
};

struct kvm_sev_send_update_data {
__u64 hdr_uaddr;
__u32 hdr_len;
+ __u32 pad0;
__u64 guest_uaddr;
__u32 guest_len;
+ __u32 pad1;
__u64 trans_uaddr;
__u32 trans_len;
+ __u32 pad2;
};

struct kvm_sev_receive_start {
@@ -765,17 +783,22 @@ struct kvm_sev_receive_start {
__u32 policy;
__u64 pdh_uaddr;
__u32 pdh_len;
+ __u32 pad0;
__u64 session_uaddr;
__u32 session_len;
+ __u32 pad1;
};

struct kvm_sev_receive_update_data {
__u64 hdr_uaddr;
__u32 hdr_len;
+ __u32 pad0;
__u64 guest_uaddr;
__u32 guest_len;
+ __u32 pad1;
__u64 trans_uaddr;
__u32 trans_len;
+ __u32 pad2;
};

#define KVM_X2APIC_API_USE_32BIT_IDS (1ULL << 0)
--
2.39.0



2024-02-09 19:40:25

by Sean Christopherson

[permalink] [raw]
Subject: Re: [PATCH 00/10] KVM: SEV: allow customizing VMSA features

On Fri, Feb 09, 2024, Paolo Bonzini wrote:
> The idea that no parameter would ever be necessary when enabling SEV or
> SEV-ES for a VM was decidedly optimistic.

That implies there was a conscious decision regarding the uAPI. AFAICT, all of
the SEV uAPIs are direct reflections of the PSP invocations. Which is why I'm
being so draconian about the SNP uAPIs; this time around, we need to actually
design something.

> The first source of variability that was encountered is the desired set of
> VMSA features, as that affects the measurement of the VM's initial state and
> cannot be changed arbitrarily by the hypervisor.
>
> This series adds all the APIs that are needed to customize the features,
> with room for future enhancements:
>
> - a new /dev/kvm device attribute to retrieve the set of supported
> features (right now, only debug swap)
>
> - a new sub-operation for KVM_MEM_ENCRYPT_OP that can take a struct,
> replacing the existing KVM_SEV_INIT and KVM_SEV_ES_INIT
>
> It then puts the new op to work by including the VMSA features as a field
> of the The existing KVM_SEV_INIT and KVM_SEV_ES_INIT use the full set of
> supported VMSA features for backwards compatibility; but I am considering
> also making them use zero as the feature mask, and will gladly adjust the
> patches if so requested.

Rather than add a new KVM_MEMORY_ENCRYPT_OP, I think we should go for broke and
start building the generic set of "protected VM" APIs. E.g. TDX wants to add
KVM_TDX_INIT_VM, and I'm guessing ARM needs similar functionality. And AFAIK,
every technology follows an INIT => ADD (MEASURE) * N => FINALIZE type sequence.

If need be, I would rather have a massive union, a la kvm_run, to hold the vendor
specific bits than end up with sub-sub-ioctls and every vendor implementation
reinventing the wheel.

If it's sane and feasible for userspace, maybe even KVM_CREATE_VM2?

> In order to avoid creating *two* new KVM_MEM_ENCRYPT_OPs, I decided that
> I could as well make SEV and SEV-ES use VM types. And then, why not make
> a SEV-ES VM, when created with the new VM type instead of KVM_SEV_ES_INIT,
> reject KVM_GET_REGS/KVM_SET_REGS and friends on the vCPU file descriptor
> once the VMSA has been encrypted... Which is how the API should have
> always behaved.

+1000

2024-02-09 22:40:58

by Paolo Bonzini

[permalink] [raw]
Subject: Re: [PATCH 00/10] KVM: SEV: allow customizing VMSA features

On Fri, Feb 9, 2024 at 8:40 PM Sean Christopherson <[email protected]> wrote:
> On Fri, Feb 09, 2024, Paolo Bonzini wrote:
> > The idea that no parameter would ever be necessary when enabling SEV or
> > SEV-ES for a VM was decidedly optimistic.
>
> That implies there was a conscious decision regarding the uAPI. AFAICT, all of
> the SEV uAPIs are direct reflections of the PSP invocations. Which is why I'm
> being so draconian about the SNP uAPIs; this time around, we need to actually
> design something.

You liked that word, heh? :) The part that I am less sure about, is
that it's actually _possible_ to design something.

If you end up with a KVM_CREATE_VM2 whose arguments are

uint32_t flags;
uint32_t vm_type;
uint64_t arch_mishmash_0; /* Intel only */
uint64_t arch_mishmash_1; /* AMD only */
uint64_t arch_mishmash_2; /* Intel only */
uint64_t arch_mishmash_3; /* AMD only */

and half of the flags are Intel only, the other half are AMD only...
do you actually gain anything over a vendor-specific ioctl?

Case in point being that the SEV VMSA features would be one of the
fields above, and they would obviously not be available for TDX.

kvm_run is a different story because it's the result of mmap, and not
a ioctl. But in this case we have:

- pretty generic APIs like UPDATE_DATA and MEASURE that should just be
renamed to remove SEV references. Even DBG_DECRYPT and DBG_ENCRYPT
fall in this category

- APIs that seem okay but may depend on specific initialization flows,
for example LAUNCH_UPDATE_VMSA. One example of the problems with
initialization flows is LAUNCH_FINISH, which seems pretty tame but is
different between SEV{,-ES} and SNP. Another example could be CPUID
which is slightly different between vendors.

- APIs that are currently vendor-specific, but where a second version
has a chance of being cross-vendor, for example LAUNCH_SECRET or
GET_ATTESTATION_REPORT. Or maybe not.

- others that have no hope, because they include so many pieces of
vendor-specific data that there's hardly anything to share. INIT is
one of them. I guess you could fit the Intel CPUID square hole into
AMD's CPUID round peg or vice versa, but there's really little in
common between the two.

I think we should try to go for the first three, but realistically
will have to stop at the first one in most cases. Honestly, this
unified API effort should have started years ago if we wanted to go
there. I see where you're coming from, but the benefits are marginal
(like the amount of userspace code that could be reused) and the
effort huge in comparison. And especially, it's much worse to get an
attempt at a unified API wrong, than to have N different APIs.

This is not a free-for-all, there are definitely some
KVM_MEMORY_ENCRYPT_OP that can be shared between SEV and TDX. The
series also adds VM type support for SEV which fixes a poor past
choice. I don't think there's much to gain from sharing the whole INIT
phase though.

Paolo


2024-02-13 14:46:13

by Paolo Bonzini

[permalink] [raw]
Subject: Re: [PATCH 00/10] KVM: SEV: allow customizing VMSA features

On Tue, Feb 13, 2024 at 3:46 AM Sean Christopherson <[email protected]> wrote:
> __u32 flags;
> __u32 vm_type;
> union {
> struct tdx;
> struct sev;
> struct sev_es;
> struct sev_snp;
> __u8 pad[<big size>]
> };
>
> Rinse and repeat for APIs that have a common purpose, but different payloads.
>
> Similar to KVM_{SET,GET}_NESTED_STATE, where the data is wildly different, and
> there's very little overlap between {svm,vmx}_set_nested_state(), I find it quite
> valuable to have a single set of APIs. E.g. I don't have to translate between
> VMX and SVM terminology when thinking about the APIs, when discussing them, etc.
>
> That's especially true for all this CoCo goo, where the names are ridiculously
> divergent, and often not exactly intuitive. E.g. LAUNCH_MEASURE reads like
> "measure the launch", but surprise, it's "get the measurement".

I agree, but then you'd have to do things like "CPUID data is passed
via UPDATE_DATA for SEV and INIT_VM for TDX (and probably not at all
for pKVM)". And in one case the firmware may prefer to encrypt in
place, in the other you cannot do that at all.

There was a reason why SVM support was not added from the beginning.
Before adding nested get/set support for SVM, the whole nested
virtualization was made as similar as possible in design and
functionality to VMX. Of course it cannot be entirely the same, but
for example they share the overall idea that pending events and L2
state are taken from vCPU state; kvm_nested_state only stores global
processor state (VMXON/VMCS pointers on VMX, and GIF on SVM) and,
while in guest mode, L1 state and control bits. This ensures that the
same userspace flow can work for both VMX and SVM. However, in this
case we can't really control what is done in firmware.

> The effort doesn't seem huge, so long as we don't try to make the parameters
> common across vendor code. The list of APIs doesn't seem insurmountable (note,
> I'm not entirely sure these are correct mappings):

While the effort isn't huge, the benefit is also pretty small, which
comes to a second big difference with GET/SET_NESTED_STATE: because
there is a GET ioctl, we have the possibility of retrieving the "black
box" and passing it back. With CoCo it's anyway userspace's task to
fill in the parameter structs. I just don't see the possibility of
sharing any code except the final ioctl, which to be honest is not
much to show. And the higher price might be in re-reviewing code that
has already been reviewed, both in KVM and in userspace.

Paolo

> create
> init VM (LAUNCH_START / TDH.MNG.INIT)
> update (LAUNCH_UPDATE_DATA / TDH.MEM.PAGE.ADD+TDH.MR.EXTEND)
> init vCPU (LAUNCH_UPDATE_VMSA / TDH.VP.INIT)
> finalize (LAUNCH_FINISH / TDH.MR.FINALIZE)


2024-02-15 01:20:30

by Michael Roth

[permalink] [raw]
Subject: Re: [PATCH 01/10] KVM: SEV: fix compat ABI for KVM_MEMORY_ENCRYPT_OP

On Fri, Feb 09, 2024 at 01:37:33PM -0500, Paolo Bonzini wrote:
> The data structs for KVM_MEMORY_ENCRYPT_OP have different sizes for 32- and 64-bit
> kernels, but they do not make any attempt to convert from one ABI to the other.
> Fix this by adding the appropriate padding.
>
> No functional change intended for 64-bit userspace.
>
> Signed-off-by: Paolo Bonzini <[email protected]>

Reviewed-by: Michael Roth <[email protected]>

> ---
> arch/x86/include/uapi/asm/kvm.h | 23 +++++++++++++++++++++++
> 1 file changed, 23 insertions(+)
>
> diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
> index 0ad6bda1fc39..b305daff056e 100644
> --- a/arch/x86/include/uapi/asm/kvm.h
> +++ b/arch/x86/include/uapi/asm/kvm.h
> @@ -687,6 +687,7 @@ enum sev_cmd_id {
>
> struct kvm_sev_cmd {
> __u32 id;
> + __u32 pad0;
> __u64 data;
> __u32 error;
> __u32 sev_fd;
> @@ -697,28 +698,35 @@ struct kvm_sev_launch_start {
> __u32 policy;
> __u64 dh_uaddr;
> __u32 dh_len;
> + __u32 pad0;
> __u64 session_uaddr;
> __u32 session_len;
> + __u32 pad1;
> };
>
> struct kvm_sev_launch_update_data {
> __u64 uaddr;
> __u32 len;
> + __u32 pad0;
> };
>
>
> struct kvm_sev_launch_secret {
> __u64 hdr_uaddr;
> __u32 hdr_len;
> + __u32 pad0;
> __u64 guest_uaddr;
> __u32 guest_len;
> + __u32 pad1;
> __u64 trans_uaddr;
> __u32 trans_len;
> + __u32 pad2;
> };
>
> struct kvm_sev_launch_measure {
> __u64 uaddr;
> __u32 len;
> + __u32 pad0;
> };
>
> struct kvm_sev_guest_status {
> @@ -731,33 +739,43 @@ struct kvm_sev_dbg {
> __u64 src_uaddr;
> __u64 dst_uaddr;
> __u32 len;
> + __u32 pad0;
> };
>
> struct kvm_sev_attestation_report {
> __u8 mnonce[16];
> __u64 uaddr;
> __u32 len;
> + __u32 pad0;
> };
>
> struct kvm_sev_send_start {
> __u32 policy;
> + __u32 pad0;
> __u64 pdh_cert_uaddr;
> __u32 pdh_cert_len;
> + __u32 pad1;
> __u64 plat_certs_uaddr;
> __u32 plat_certs_len;
> + __u32 pad2;
> __u64 amd_certs_uaddr;
> __u32 amd_certs_len;
> + __u32 pad3;
> __u64 session_uaddr;
> __u32 session_len;
> + __u32 pad4;
> };
>
> struct kvm_sev_send_update_data {
> __u64 hdr_uaddr;
> __u32 hdr_len;
> + __u32 pad0;
> __u64 guest_uaddr;
> __u32 guest_len;
> + __u32 pad1;
> __u64 trans_uaddr;
> __u32 trans_len;
> + __u32 pad2;
> };
>
> struct kvm_sev_receive_start {
> @@ -765,17 +783,22 @@ struct kvm_sev_receive_start {
> __u32 policy;
> __u64 pdh_uaddr;
> __u32 pdh_len;
> + __u32 pad0;
> __u64 session_uaddr;
> __u32 session_len;
> + __u32 pad1;
> };
>
> struct kvm_sev_receive_update_data {
> __u64 hdr_uaddr;
> __u32 hdr_len;
> + __u32 pad0;
> __u64 guest_uaddr;
> __u32 guest_len;
> + __u32 pad1;
> __u64 trans_uaddr;
> __u32 trans_len;
> + __u32 pad2;
> };
>
> #define KVM_X2APIC_API_USE_32BIT_IDS (1ULL << 0)
> --
> 2.39.0
>
>

2024-02-15 01:21:58

by Michael Roth

[permalink] [raw]
Subject: Re: [PATCH 03/10] Documentation: kvm/sev: separate description of firmware

On Fri, Feb 09, 2024 at 01:37:35PM -0500, Paolo Bonzini wrote:
> The description of firmware is included part under the "SEV Key Management"
> header, part under the KVM_SEV_INIT ioctl. Put these two bits together and
> and rename "SEV Key Management" to what it actually is, namely a description
> of the KVM_MEMORY_ENCRYPT_OP API.
>
> Signed-off-by: Paolo Bonzini <[email protected]>

Reviewed-by: Michael Roth <[email protected]>

> ---
> .../virt/kvm/x86/amd-memory-encryption.rst | 29 +++++++++++--------
> 1 file changed, 17 insertions(+), 12 deletions(-)
>
> diff --git a/Documentation/virt/kvm/x86/amd-memory-encryption.rst b/Documentation/virt/kvm/x86/amd-memory-encryption.rst
> index 995780088eb2..37c5c37f4f6e 100644
> --- a/Documentation/virt/kvm/x86/amd-memory-encryption.rst
> +++ b/Documentation/virt/kvm/x86/amd-memory-encryption.rst
> @@ -46,14 +46,8 @@ SEV hardware uses ASIDs to associate a memory encryption key with a VM.
> Hence, the ASID for the SEV-enabled guests must be from 1 to a maximum value
> defined in the CPUID 0x8000001f[ecx] field.
>
> -SEV Key Management
> -==================
> -
> -The SEV guest key management is handled by a separate processor called the AMD
> -Secure Processor (AMD-SP). Firmware running inside the AMD-SP provides a secure
> -key management interface to perform common hypervisor activities such as
> -encrypting bootstrap code, snapshot, migrating and debugging the guest. For more
> -information, see the SEV Key Management spec [api-spec]_
> +``KVM_MEMORY_ENCRYPT_OP`` API
> +=============================
>
> The main ioctl to access SEV is KVM_MEMORY_ENCRYPT_OP. If the argument
> to KVM_MEMORY_ENCRYPT_OP is NULL, the ioctl returns 0 if SEV is enabled
> @@ -87,10 +81,6 @@ guests, such as launching, running, snapshotting, migrating and decommissioning.
> The KVM_SEV_INIT command is used by the hypervisor to initialize the SEV platform
> context. In a typical workflow, this command should be the first command issued.
>
> -The firmware can be initialized either by using its own non-volatile storage or
> -the OS can manage the NV storage for the firmware using the module parameter
> -``init_ex_path``. If the file specified by ``init_ex_path`` does not exist or
> -is invalid, the OS will create or override the file with output from PSP.
>
> Returns: 0 on success, -negative on error
>
> @@ -434,6 +424,21 @@ issued by the hypervisor to make the guest ready for execution.
>
> Returns: 0 on success, -negative on error
>
> +Firmware Management
> +===================
> +
> +The SEV guest key management is handled by a separate processor called the AMD
> +Secure Processor (AMD-SP). Firmware running inside the AMD-SP provides a secure
> +key management interface to perform common hypervisor activities such as
> +encrypting bootstrap code, snapshot, migrating and debugging the guest. For more
> +information, see the SEV Key Management spec [api-spec]_
> +
> +The AMD-SP firmware can be initialized either by using its own non-volatile
> +storage or the OS can manage the NV storage for the firmware using
> +parameter ``init_ex_path`` of the ``ccp`` module. If the file specified
> +by ``init_ex_path`` does not exist or is invalid, the OS will create or
> +override the file with PSP non-volatile storage.
> +
> References
> ==========
>
> --
> 2.39.0
>
>
>

2024-02-15 01:34:56

by Michael Roth

[permalink] [raw]
Subject: Re: [PATCH 09/10] KVM: SEV: introduce KVM_SEV_INIT2 operation

On Fri, Feb 09, 2024 at 01:37:41PM -0500, Paolo Bonzini wrote:
> The idea that no parameter would ever be necessary when enabling SEV or
> SEV-ES for a VM was decidedly optimistic. In fact, in some sense it's
> already a parameter whether SEV or SEV-ES is desired. Another possible
> source of variability is the desired set of VMSA features, as that affects
> the measurement of the VM's initial state and cannot be changed
> arbitrarily by the hypervisor.
>
> Create a new sub-operation for KVM_MEM_ENCRYPT_OP that can take a struct,
> and put the new op to work by including the VMSA features as a field of the
> struct. The existing KVM_SEV_INIT and KVM_SEV_ES_INIT use the full set of
> supported VMSA features for backwards compatibility.
>
> The struct also includes the usual bells and whistles for future
> extensibility: a flags field that must be zero for now, and some padding
> at the end.
>
> Signed-off-by: Paolo Bonzini <[email protected]>
> ---
> .../virt/kvm/x86/amd-memory-encryption.rst | 41 ++++++++++++++--
> arch/x86/include/uapi/asm/kvm.h | 10 ++++
> arch/x86/kvm/svm/sev.c | 48 +++++++++++++++++--
> 3 files changed, 92 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/virt/kvm/x86/amd-memory-encryption.rst b/Documentation/virt/kvm/x86/amd-memory-encryption.rst
> index 5ed11bc16b96..a4291e7bd8ed 100644
> --- a/Documentation/virt/kvm/x86/amd-memory-encryption.rst
> +++ b/Documentation/virt/kvm/x86/amd-memory-encryption.rst
> @@ -75,15 +75,50 @@ are defined in ``<linux/psp-dev.h>``.
> KVM implements the following commands to support common lifecycle events of SEV
> guests, such as launching, running, snapshotting, migrating and decommissioning.
>
> -1. KVM_SEV_INIT
> ----------------
> +1. KVM_SEV_INIT2
> +----------------
>
> -The KVM_SEV_INIT command is used by the hypervisor to initialize the SEV platform
> +The KVM_SEV_INIT2 command is used by the hypervisor to initialize the SEV platform
> context. In a typical workflow, this command should be the first command issued.
>
> +For this command to be accepted, either KVM_X86_SEV_VM or KVM_X86_SEV_ES_VM
> +must have been passed to the KVM_CREATE_VM ioctl. A virtual machine created
> +with those machine types in turn cannot be run until KVM_SEV_INIT2 is invoked.
> +
> +Parameters: struct kvm_sev_init (in)
>
> Returns: 0 on success, -negative on error
>
> +::
> +
> + struct struct kvm_sev_init {

Missing the vm_type param here.

> + __u32 flags; /* must be 0 */
> + __u64 vmsa_features; /* initial value of features field in VMSA */
> + __u32 pad[8];
> + };
> +
> +It is an error if the hypervisor does not support any of the bits that
> +are set in ``flags`` or ``vmsa_features``.
> +
> +This command replaces the deprecated KVM_SEV_INIT and KVM_SEV_ES_INIT commands.
> +The commands did not have any parameters (the ```data``` field was unused) and
> +only work for the KVM_X86_DEFAULT_VM machine type (0).
> +
> +They behave as if:
> +
> +* the VM type is KVM_X86_SEV_VM for KVM_SEV_INIT, or KVM_X86_SEV_ES_VM for
> + KVM_SEV_ES_INIT
> +
> +* the ``flags`` field of ``struct kvm_sev_init`` is set to zero
> +
> +* the ``vmsa_features`` field of ``struct kvm_sev_init`` is set to all features
> + supported by the hypervisor (as returned by ``KVM_GET_DEVICE_ATTR`` when
> + passed group 0 and attribute id ``KVM_X86_SEV_VMSA_FEATURES``).
> +
> +If the ``KVM_X86_SEV_VMSA_FEATURES`` attribute does not exist, the hypervisor only
> +supports KVM_SEV_INIT and KVM_SEV_ES_INIT. In that case the set of VMSA features is
> +undefined.

It's hard to imagine userspace implementation support for querying
KVM_X86_SEV_VMSA_FEATURES but still insisting on KVM_SEV_INIT. Maybe it
would be better to just lock in that VMSA_FEATURES at what is currently
supported: DEBUG_SWAP=on/off depending on the kvm_amd module param, and
then for all other features require opt-in via KVM_SEV_INIT2, and then
bake that into the documentation. That way way they could still reference
this documentation to properly calculate measurements for older/existing
VMM implementations.

-Mike

> +
> 2. KVM_SEV_LAUNCH_START
> -----------------------
>
> diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
> index 7c46e96cfe62..6baf18335c7b 100644
> --- a/arch/x86/include/uapi/asm/kvm.h
> +++ b/arch/x86/include/uapi/asm/kvm.h
> @@ -683,6 +683,9 @@ enum sev_cmd_id {
> /* Guest Migration Extension */
> KVM_SEV_SEND_CANCEL,
>
> + /* Second time is the charm; improved versions of the above ioctls. */
> + KVM_SEV_INIT2,
> +
> KVM_SEV_NR_MAX,
> };
>
> @@ -694,6 +697,13 @@ struct kvm_sev_cmd {
> __u32 sev_fd;
> };
>
> +struct kvm_sev_init {
> + __u32 vm_type;
> + __u32 flags;
> + __u64 vmsa_features;
> + __u32 pad[8];
> +};
> +
> struct kvm_sev_launch_start {
> __u32 handle;
> __u32 policy;
> diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> index acf5c45ef14e..78c52764453f 100644
> --- a/arch/x86/kvm/svm/sev.c
> +++ b/arch/x86/kvm/svm/sev.c
> @@ -252,7 +252,9 @@ static void sev_unbind_asid(struct kvm *kvm, unsigned int handle)
> sev_decommission(handle);
> }
>
> -static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
> +static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp,
> + struct kvm_sev_init *data,
> + unsigned long vm_type)
> {
> struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
> int asid, ret;
> @@ -260,7 +262,10 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
> if (kvm->created_vcpus)
> return -EINVAL;
>
> - if (kvm->arch.vm_type != KVM_X86_DEFAULT_VM)
> + if (data->flags)
> + return -EINVAL;
> +
> + if (data->vmsa_features & ~sev_supported_vmsa_features)
> return -EINVAL;
>
> ret = -EBUSY;
> @@ -268,8 +273,8 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
> return ret;
>
> sev->active = true;
> - sev->es_active = argp->id == KVM_SEV_ES_INIT;
> - sev->vmsa_features = sev_supported_vmsa_features;
> + sev->es_active = (vm_type & __KVM_X86_PROTECTED_STATE_TYPE) != 0;
> + sev->vmsa_features = data->vmsa_features;
>
> asid = sev_asid_new(sev);
> if (asid < 0)
> @@ -298,6 +303,38 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
> return ret;
> }
>
> +static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
> +{
> + struct kvm_sev_init data = {
> + .vmsa_features = sev_supported_vmsa_features,
> + };
> + unsigned long vm_type;
> +
> + if (kvm->arch.vm_type != KVM_X86_DEFAULT_VM)
> + return -EINVAL;
> +
> + vm_type = (argp->id == KVM_SEV_INIT ? KVM_X86_SEV_VM : KVM_X86_SEV_ES_VM);
> + return __sev_guest_init(kvm, argp, &data, vm_type);
> +}
> +
> +static int sev_guest_init2(struct kvm *kvm, struct kvm_sev_cmd *argp)
> +{
> + struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
> + struct kvm_sev_init data;
> +
> + if (!sev->need_init)
> + return -EINVAL;
> +
> + if (kvm->arch.vm_type != KVM_X86_SEV_VM &&
> + kvm->arch.vm_type != KVM_X86_SEV_ES_VM)
> + return -EINVAL;
> +
> + if (copy_from_user(&data, (void __user *)(uintptr_t)argp->data, sizeof(data)))
> + return -EFAULT;
> +
> + return __sev_guest_init(kvm, argp, &data, kvm->arch.vm_type);
> +}
> +
> static int sev_bind_asid(struct kvm *kvm, unsigned int handle, int *error)
> {
> struct sev_data_activate activate;
> @@ -1915,6 +1952,9 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
> case KVM_SEV_INIT:
> r = sev_guest_init(kvm, &sev_cmd);
> break;
> + case KVM_SEV_INIT2:
> + r = sev_guest_init2(kvm, &sev_cmd);
> + break;
> case KVM_SEV_LAUNCH_START:
> r = sev_launch_start(kvm, &sev_cmd);
> break;
> --
> 2.39.0
>
>

2024-02-15 01:35:49

by Michael Roth

[permalink] [raw]
Subject: Re: [PATCH 02/10] KVM: introduce new vendor op for KVM_GET_DEVICE_ATTR

On Fri, Feb 09, 2024 at 01:37:34PM -0500, Paolo Bonzini wrote:
> Allow vendor modules to provide their own attributes on /dev/kvm.
> To avoid proliferation of vendor ops, implement KVM_HAS_DEVICE_ATTR
> and KVM_GET_DEVICE_ATTR in terms of the same function. You're not
> supposed to use KVM_GET_DEVICE_ATTR to do complicated computations,
> especially on /dev/kvm.
>
> Signed-off-by: Paolo Bonzini <[email protected]>

Reviewed-by: Michael Roth <[email protected]>

> ---
> arch/x86/include/asm/kvm-x86-ops.h | 1 +
> arch/x86/include/asm/kvm_host.h | 1 +
> arch/x86/kvm/x86.c | 52 +++++++++++++++++++-----------
> 3 files changed, 36 insertions(+), 18 deletions(-)
>
> diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h
> index 378ed944b849..ac8b7614e79d 100644
> --- a/arch/x86/include/asm/kvm-x86-ops.h
> +++ b/arch/x86/include/asm/kvm-x86-ops.h
> @@ -122,6 +122,7 @@ KVM_X86_OP(enter_smm)
> KVM_X86_OP(leave_smm)
> KVM_X86_OP(enable_smi_window)
> #endif
> +KVM_X86_OP_OPTIONAL(dev_get_attr)
> KVM_X86_OP_OPTIONAL(mem_enc_ioctl)
> KVM_X86_OP_OPTIONAL(mem_enc_register_region)
> KVM_X86_OP_OPTIONAL(mem_enc_unregister_region)
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index d271ba20a0b2..0bcd9ae16097 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1769,6 +1769,7 @@ struct kvm_x86_ops {
> void (*enable_smi_window)(struct kvm_vcpu *vcpu);
> #endif
>
> + int (*dev_get_attr)(u64 attr, u64 *val);
> int (*mem_enc_ioctl)(struct kvm *kvm, void __user *argp);
> int (*mem_enc_register_region)(struct kvm *kvm, struct kvm_enc_region *argp);
> int (*mem_enc_unregister_region)(struct kvm *kvm, struct kvm_enc_region *argp);
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index bf10a9073a09..8746530930d5 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -4804,37 +4804,53 @@ static inline void __user *kvm_get_attr_addr(struct kvm_device_attr *attr)
> return uaddr;
> }
>
> -static int kvm_x86_dev_get_attr(struct kvm_device_attr *attr)
> +static int __kvm_x86_dev_get_attr(struct kvm_device_attr *attr, u64 *val)
> {
> - u64 __user *uaddr = kvm_get_attr_addr(attr);
> + int r;
>
> if (attr->group)
> return -ENXIO;
>
> + switch (attr->attr) {
> + case KVM_X86_XCOMP_GUEST_SUPP:
> + r = 0;
> + *val = kvm_caps.supported_xcr0;
> + break;
> + default:
> + r = -ENXIO;
> + if (kvm_x86_ops.dev_get_attr)
> + r = kvm_x86_ops.dev_get_attr(attr->attr, val);
> + break;
> + }
> +
> + return r;
> +}
> +
> +static int kvm_x86_dev_get_attr(struct kvm_device_attr *attr)
> +{
> + u64 __user *uaddr;
> + int r;
> + u64 val;
> +
> + r = __kvm_x86_dev_get_attr(attr, &val);
> + if (r < 0)
> + return r;
> +
> + uaddr = kvm_get_attr_addr(attr);
> if (IS_ERR(uaddr))
> return PTR_ERR(uaddr);
>
> - switch (attr->attr) {
> - case KVM_X86_XCOMP_GUEST_SUPP:
> - if (put_user(kvm_caps.supported_xcr0, uaddr))
> - return -EFAULT;
> - return 0;
> - default:
> - return -ENXIO;
> - }
> + if (put_user(val, uaddr))
> + return -EFAULT;
> +
> + return 0;
> }
>
> static int kvm_x86_dev_has_attr(struct kvm_device_attr *attr)
> {
> - if (attr->group)
> - return -ENXIO;
> + u64 val;
>
> - switch (attr->attr) {
> - case KVM_X86_XCOMP_GUEST_SUPP:
> - return 0;
> - default:
> - return -ENXIO;
> - }
> + return __kvm_x86_dev_get_attr(attr, &val);
> }
>
> long kvm_arch_dev_ioctl(struct file *filp,
> --
> 2.39.0
>
>

2024-02-15 11:09:47

by Alexey Kardashevskiy

[permalink] [raw]
Subject: Re: [PATCH 09/10] KVM: SEV: introduce KVM_SEV_INIT2 operation



On 10/2/24 05:37, Paolo Bonzini wrote:
> The idea that no parameter would ever be necessary when enabling SEV or
> SEV-ES for a VM was decidedly optimistic. In fact, in some sense it's
> already a parameter whether SEV or SEV-ES is desired. Another possible
> source of variability is the desired set of VMSA features, as that affects
> the measurement of the VM's initial state and cannot be changed
> arbitrarily by the hypervisor.
>
> Create a new sub-operation for KVM_MEM_ENCRYPT_OP that can take a struct,

a typo here: KVM_MEMORY_ENCRYPT_OP.


--
Alexey


2024-02-15 13:45:06

by Paolo Bonzini

[permalink] [raw]
Subject: Re: [PATCH 09/10] KVM: SEV: introduce KVM_SEV_INIT2 operation

On 2/15/24 02:34, Michael Roth wrote:
>> + struct struct kvm_sev_init {
> Missing the vm_type param here.

It can go away in the struct actually. Also, "struct struct".

>> +If the ``KVM_X86_SEV_VMSA_FEATURES`` attribute does not exist, the hypervisor only
>> +supports KVM_SEV_INIT and KVM_SEV_ES_INIT. In that case the set of VMSA features is
>> +undefined.
>
> It's hard to imagine userspace implementation support for querying
> KVM_X86_SEV_VMSA_FEATURES but still insisting on KVM_SEV_INIT.

.. except for backwards compatibility with old kernels. For example,
the VMM could first invoke HAS_DEVICE_ATTR, and then fall back to
KVM_SEV_INIT after checking that the user did not explicitly request or
forbid one or more VMSA features.

> Maybe it
> would be better to just lock in that VMSA_FEATURES at what is currently
> supported: DEBUG_SWAP=on/off depending on the kvm_amd module param, and
> then for all other features require opt-in via KVM_SEV_INIT2, and then
> bake that into the documentation. That way way they could still reference
> this documentation to properly calculate measurements for older/existing
> VMM implementations.

Thinking more about it, I think all features including debug_swap should
be disabled with the old SEV_INIT/SEV_ES_INIT. Because the features
need to match between the VMM and the measurement calculation, they need
to be added explicitly on e.g. the QEMU command line.

Paolo


2024-02-15 14:45:55

by Michael Roth

[permalink] [raw]
Subject: Re: [PATCH 09/10] KVM: SEV: introduce KVM_SEV_INIT2 operation

On Thu, Feb 15, 2024 at 02:44:47PM +0100, Paolo Bonzini wrote:
> On 2/15/24 02:34, Michael Roth wrote:
> > > + struct struct kvm_sev_init {
> > Missing the vm_type param here.
>
> It can go away in the struct actually. Also, "struct struct".
>
> > > +If the ``KVM_X86_SEV_VMSA_FEATURES`` attribute does not exist, the hypervisor only
> > > +supports KVM_SEV_INIT and KVM_SEV_ES_INIT. In that case the set of VMSA features is
> > > +undefined.
> >
> > It's hard to imagine userspace implementation support for querying
> > KVM_X86_SEV_VMSA_FEATURES but still insisting on KVM_SEV_INIT.
>
> ... except for backwards compatibility with old kernels. For example, the
> VMM could first invoke HAS_DEVICE_ATTR, and then fall back to KVM_SEV_INIT
> after checking that the user did not explicitly request or forbid one or
> more VMSA features.

What I mean is that if userspace is modified for these checks, it's
reasonable to also inform them that only VMSA features present in
those older kernels (i.e. debug-swap) will be available via KVM_SEV_INIT,
and for anything else they will need to use KVM_SEV_INIT.

That way we can provide clear documentation on what to expect regarding
VMSA features for KVM_SEV_INIT and not have to have the "undefined"
wording: it'll never use anything other than debug-swap depending on the
module param setting.

>
> > Maybe it
> > would be better to just lock in that VMSA_FEATURES at what is currently
> > supported: DEBUG_SWAP=on/off depending on the kvm_amd module param, and
> > then for all other features require opt-in via KVM_SEV_INIT2, and then
> > bake that into the documentation. That way way they could still reference
> > this documentation to properly calculate measurements for older/existing
> > VMM implementations.
>
> Thinking more about it, I think all features including debug_swap should be
> disabled with the old SEV_INIT/SEV_ES_INIT. Because the features need to
> match between the VMM and the measurement calculation, they need to be added
> explicitly on e.g. the QEMU command line.

That seems reasonable, but the main thing I was hoping to avoid was
another round of VMSA features changing out from underneath the covers
again. The module param setting is something we've needed to convey
internally/externally a good bit due to the fallout and making this
change would lead to another repeat. Not the end of the world but would
be nice to avoid if possible.

-Mike

>
> Paolo
>

2024-02-15 17:28:49

by Paolo Bonzini

[permalink] [raw]
Subject: Re: [PATCH 09/10] KVM: SEV: introduce KVM_SEV_INIT2 operation

On Thu, Feb 15, 2024 at 3:44 PM Michael Roth <[email protected]> wrote:
> What I mean is that if userspace is modified for these checks, it's
> reasonable to also inform them that only VMSA features present in
> those older kernels (i.e. debug-swap) will be available via KVM_SEV_INIT,
> and for anything else they will need to use KVM_SEV_INIT.
>
> That way we can provide clear documentation on what to expect regarding
> VMSA features for KVM_SEV_INIT and not have to have the "undefined"
> wording: it'll never use anything other than debug-swap depending on the
> module param setting.

Ah, I agree.

> That seems reasonable, but the main thing I was hoping to avoid was
> another round of VMSA features changing out from underneath the covers
> again. The module param setting is something we've needed to convey
> internally/externally a good bit due to the fallout and making this
> change would lead to another repeat. Not the end of the world but would
> be nice to avoid if possible.

The fallout was caused by old kernels not supporting debug-swap and
now by failing measurements. As far as I know there is no downside of
leaving it disabled by default, and it will fix booting old guest
kernels.

Paolo


2024-02-15 17:55:55

by Michael Roth

[permalink] [raw]
Subject: Re: [PATCH 09/10] KVM: SEV: introduce KVM_SEV_INIT2 operation

On Thu, Feb 15, 2024 at 06:28:18PM +0100, Paolo Bonzini wrote:
> On Thu, Feb 15, 2024 at 3:44 PM Michael Roth <[email protected]> wrote:
> > What I mean is that if userspace is modified for these checks, it's
> > reasonable to also inform them that only VMSA features present in
> > those older kernels (i.e. debug-swap) will be available via KVM_SEV_INIT,
> > and for anything else they will need to use KVM_SEV_INIT.
> >
> > That way we can provide clear documentation on what to expect regarding
> > VMSA features for KVM_SEV_INIT and not have to have the "undefined"
> > wording: it'll never use anything other than debug-swap depending on the
> > module param setting.
>
> Ah, I agree.
>
> > That seems reasonable, but the main thing I was hoping to avoid was
> > another round of VMSA features changing out from underneath the covers
> > again. The module param setting is something we've needed to convey
> > internally/externally a good bit due to the fallout and making this
> > change would lead to another repeat. Not the end of the world but would
> > be nice to avoid if possible.
>
> The fallout was caused by old kernels not supporting debug-swap and
> now by failing measurements. As far as I know there is no downside of
> leaving it disabled by default, and it will fix booting old guest
> kernels.

Yah, agreed on older guest kernels, but it's the measurement side of things
where we'd expect some additional fallout. The guidance was essentially that
if you run a newer host kernel with debug-swap support, you need either need
to:

a) update your measurements to account for the additional VMSA feature
b) disable debug-swap param to maintain previous behavior/measurement

So those who'd taken approach a) would see another unexpected measurement
change when they eventually update to a newer kernel.

-Mike

>
> Paolo
>

2024-02-15 18:15:42

by Paolo Bonzini

[permalink] [raw]
Subject: Re: [PATCH 09/10] KVM: SEV: introduce KVM_SEV_INIT2 operation

On Thu, Feb 15, 2024 at 6:55 PM Michael Roth <[email protected]> wrote:
> > The fallout was caused by old kernels not supporting debug-swap and
> > now by failing measurements. As far as I know there is no downside of
> > leaving it disabled by default, and it will fix booting old guest
> > kernels.
>
> Yah, agreed on older guest kernels, but it's the measurement side of things
> where we'd expect some additional fallout. The guidance was essentially that
> if you run a newer host kernel with debug-swap support, you need either need
> to:
>
> a) update your measurements to account for the additional VMSA feature
> b) disable debug-swap param to maintain previous behavior/measurement

Out of curiosity, where was this documented? While debug-swap was a
pretty obvious culprit of the failed measurement, I didn't see any
mention to it anywhere (and also didn't see any mention that old
kernels would fail to boot in the KVM patches---which would have been
a pretty clear indication that something like these patches was
needed).

> So those who'd taken approach a) would see another unexpected measurement
> change when they eventually update to a newer kernel.

But they'd see it anyway if userspace starts disabling it by default.
In general, enabling _anything_ by default is a mistake in either KVM
or userspace if you care about guest ABI (which you obviously do in
the case of confidential computing).

Paolo


2024-02-15 20:44:38

by Michael Roth

[permalink] [raw]
Subject: Re: [PATCH 09/10] KVM: SEV: introduce KVM_SEV_INIT2 operation

On Thu, Feb 15, 2024 at 07:08:06PM +0100, Paolo Bonzini wrote:
> On Thu, Feb 15, 2024 at 6:55 PM Michael Roth <[email protected]> wrote:
> > > The fallout was caused by old kernels not supporting debug-swap and
> > > now by failing measurements. As far as I know there is no downside of
> > > leaving it disabled by default, and it will fix booting old guest
> > > kernels.
> >
> > Yah, agreed on older guest kernels, but it's the measurement side of things
> > where we'd expect some additional fallout. The guidance was essentially that
> > if you run a newer host kernel with debug-swap support, you need either need
> > to:
> >
> > a) update your measurements to account for the additional VMSA feature
> > b) disable debug-swap param to maintain previous behavior/measurement
>
> Out of curiosity, where was this documented? While debug-swap was a
> pretty obvious culprit of the failed measurement, I didn't see any
> mention to it anywhere (and also didn't see any mention that old
> kernels would fail to boot in the KVM patches---which would have been
> a pretty clear indication that something like these patches was
> needed).

Yes, this was reactive rather than proactive guidance unfortunately,
resulting from various internal/external bug reports where we needed to
suggest the above-mentioned options.

In retrospect, I think we would've handled things differently as well.
Which is why I'm hoping it's possible to ease the pain of another
potential measurement change for those who've since incorporated
debug-swap into their measurement workflow. But maybe it's not
realistic...

>
> > So those who'd taken approach a) would see another unexpected measurement
> > change when they eventually update to a newer kernel.
>
> But they'd see it anyway if userspace starts disabling it by default.

My thinking was that this wording would be specific to KVM_SEV_INIT, as
opposed to KVM_SEV_INIT2 where disabling all features by default should
absolutely be the way to go.

But realistically, it's not easy for a user to tell whether their VMM is
using KVM_SEV_INIT vs KVM_SEV_INIT2, and it does seem possible that
having the defaults be different between the 2 would also cause some
pain down the road since even in looking at the documentation it
wouldn't be immediately clear whether or not debug-swap would be enabled.

So maybe your approach of always disabling by default and requiring
userspace to opt-in would be better in the long-run since this behavior
is fairly new from a distro perspective and it's likely only
developers/early adopters that we'd be sticking the genie back in the
bottle for.

-Mike

> In general, enabling _anything_ by default is a mistake in either KVM
> or userspace if you care about guest ABI (which you obviously do in
> the case of confidential computing).
>
> Paolo
>

2024-02-15 21:14:43

by Tom Lendacky

[permalink] [raw]
Subject: Re: [PATCH 09/10] KVM: SEV: introduce KVM_SEV_INIT2 operation

On 2/9/24 12:37, Paolo Bonzini wrote:
> The idea that no parameter would ever be necessary when enabling SEV or
> SEV-ES for a VM was decidedly optimistic. In fact, in some sense it's
> already a parameter whether SEV or SEV-ES is desired. Another possible
> source of variability is the desired set of VMSA features, as that affects
> the measurement of the VM's initial state and cannot be changed
> arbitrarily by the hypervisor.
>
> Create a new sub-operation for KVM_MEM_ENCRYPT_OP that can take a struct,
> and put the new op to work by including the VMSA features as a field of the
> struct. The existing KVM_SEV_INIT and KVM_SEV_ES_INIT use the full set of
> supported VMSA features for backwards compatibility.
>
> The struct also includes the usual bells and whistles for future
> extensibility: a flags field that must be zero for now, and some padding
> at the end.
>
> Signed-off-by: Paolo Bonzini <[email protected]>
> ---
> .../virt/kvm/x86/amd-memory-encryption.rst | 41 ++++++++++++++--
> arch/x86/include/uapi/asm/kvm.h | 10 ++++
> arch/x86/kvm/svm/sev.c | 48 +++++++++++++++++--
> 3 files changed, 92 insertions(+), 7 deletions(-)
>

..

> diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> index acf5c45ef14e..78c52764453f 100644
> --- a/arch/x86/kvm/svm/sev.c
> +++ b/arch/x86/kvm/svm/sev.c
> @@ -252,7 +252,9 @@ static void sev_unbind_asid(struct kvm *kvm, unsigned int handle)
> sev_decommission(handle);
> }
>
> -static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
> +static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp,
> + struct kvm_sev_init *data,
> + unsigned long vm_type)
> {
> struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
> int asid, ret;
> @@ -260,7 +262,10 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
> if (kvm->created_vcpus)
> return -EINVAL;
>
> - if (kvm->arch.vm_type != KVM_X86_DEFAULT_VM)
> + if (data->flags)
> + return -EINVAL;
> +
> + if (data->vmsa_features & ~sev_supported_vmsa_features)

An SEV guest doesn't have protected state and so it doesn't have a VMSA to
which you can apply features. So this should be:

if (vm_type != KVM_X86_SEV_VM &&
(data->vmsa_features & ~sev_supported_vmsa_features))

Thanks,
Tom

> return -EINVAL;
>
> ret = -EBUSY;
> @@ -268,8 +273,8 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
> return ret;
>
> sev->active = true;
> - sev->es_active = argp->id == KVM_SEV_ES_INIT;
> - sev->vmsa_features = sev_supported_vmsa_features;
> + sev->es_active = (vm_type & __KVM_X86_PROTECTED_STATE_TYPE) != 0;
> + sev->vmsa_features = data->vmsa_features;
>
> asid = sev_asid_new(sev);
> if (asid < 0)
> @@ -298,6 +303,38 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
> return ret;
> }
>
> +static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
> +{
> + struct kvm_sev_init data = {
> + .vmsa_features = sev_supported_vmsa_features,
> + };
> + unsigned long vm_type;
> +
> + if (kvm->arch.vm_type != KVM_X86_DEFAULT_VM)
> + return -EINVAL;
> +
> + vm_type = (argp->id == KVM_SEV_INIT ? KVM_X86_SEV_VM : KVM_X86_SEV_ES_VM);
> + return __sev_guest_init(kvm, argp, &data, vm_type);
> +}
> +
> +static int sev_guest_init2(struct kvm *kvm, struct kvm_sev_cmd *argp)
> +{
> + struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
> + struct kvm_sev_init data;
> +
> + if (!sev->need_init)
> + return -EINVAL;
> +
> + if (kvm->arch.vm_type != KVM_X86_SEV_VM &&
> + kvm->arch.vm_type != KVM_X86_SEV_ES_VM)
> + return -EINVAL;
> +
> + if (copy_from_user(&data, (void __user *)(uintptr_t)argp->data, sizeof(data)))
> + return -EFAULT;
> +
> + return __sev_guest_init(kvm, argp, &data, kvm->arch.vm_type);
> +}
> +
> static int sev_bind_asid(struct kvm *kvm, unsigned int handle, int *error)
> {
> struct sev_data_activate activate;
> @@ -1915,6 +1952,9 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
> case KVM_SEV_INIT:
> r = sev_guest_init(kvm, &sev_cmd);
> break;
> + case KVM_SEV_INIT2:
> + r = sev_guest_init2(kvm, &sev_cmd);
> + break;
> case KVM_SEV_LAUNCH_START:
> r = sev_launch_start(kvm, &sev_cmd);
> break;

2024-02-17 01:41:07

by Sean Christopherson

[permalink] [raw]
Subject: Re: [PATCH 00/10] KVM: SEV: allow customizing VMSA features

On Tue, Feb 13, 2024, Paolo Bonzini wrote:
> On Tue, Feb 13, 2024 at 3:46 AM Sean Christopherson <[email protected]> wrote:
> > __u32 flags;
> > __u32 vm_type;
> > union {
> > struct tdx;
> > struct sev;
> > struct sev_es;
> > struct sev_snp;
> > __u8 pad[<big size>]
> > };
> >
> > Rinse and repeat for APIs that have a common purpose, but different payloads.
> >
> > Similar to KVM_{SET,GET}_NESTED_STATE, where the data is wildly different, and
> > there's very little overlap between {svm,vmx}_set_nested_state(), I find it quite
> > valuable to have a single set of APIs. E.g. I don't have to translate between
> > VMX and SVM terminology when thinking about the APIs, when discussing them, etc.
> >
> > That's especially true for all this CoCo goo, where the names are ridiculously
> > divergent, and often not exactly intuitive. E.g. LAUNCH_MEASURE reads like
> > "measure the launch", but surprise, it's "get the measurement".
>
> I agree, but then you'd have to do things like "CPUID data is passed
> via UPDATE_DATA for SEV and INIT_VM for TDX (and probably not at all
> for pKVM)". And in one case the firmware may prefer to encrypt in
> place, in the other you cannot do that at all.
>
> There was a reason why SVM support was not added from the beginning.
> Before adding nested get/set support for SVM, the whole nested
> virtualization was made as similar as possible in design and
> functionality to VMX. Of course it cannot be entirely the same, but
> for example they share the overall idea that pending events and L2
> state are taken from vCPU state; kvm_nested_state only stores global
> processor state (VMXON/VMCS pointers on VMX, and GIF on SVM) and,
> while in guest mode, L1 state and control bits. This ensures that the
> same userspace flow can work for both VMX and SVM. However, in this
> case we can't really control what is done in firmware.
>
> > The effort doesn't seem huge, so long as we don't try to make the parameters
> > common across vendor code. The list of APIs doesn't seem insurmountable (note,
> > I'm not entirely sure these are correct mappings):
>
> While the effort isn't huge, the benefit is also pretty small, which
> comes to a second big difference with GET/SET_NESTED_STATE: because
> there is a GET ioctl, we have the possibility of retrieving the "black
> box" and passing it back. With CoCo it's anyway userspace's task to
> fill in the parameter structs. I just don't see the possibility of
> sharing any code except the final ioctl, which to be honest is not
> much to show. And the higher price might be in re-reviewing code that
> has already been reviewed, both in KVM and in userspace.

Yeah, I realize I'm probably grasping at straws. *sigh*