LinuxLists.cc - [RFC] Support for Arm CCA VMs on Linux

[permalink] [raw]

Subject: [RFC kvmtool 00/31] arm64: Support for Arm Confidential Compute Architecture

This series is an initial version of the support for running VMs under the
Arm Confidential Compute Architecture. The purpose of the series is to gather
feedback on the proposed UABI changes for running Confidential VMs with KVM.
More information on the Arm CCA and instructions for how to get, build and run
the entire software stack is available here [0].

A new option, `--realm` is added to the the `run` command to mark the VM as a
confidential compute VM. This version doesn't use the Guest private memory [1]
support yet, instead uses normal anonymous/hugetlbfs backed memory. Our aim is
to switch to the guest private memory for the Realm.

The host including the kernel and kvmtool, must not access any memory allocated
to the protected IPA of the Realm.

The series adds the support for managing the lifecycle of the Realm, which includes:
* Configuration
* Creation of Realm (RD)
* Load initial memory images
* Creation of Realm Execution Contexts (RECs aka VCPUs)a
* Activation of the Realm.

Patches are split as follows :

Patches 1 and 2 are fixes to existing code.
Patch 3 adds a new option --nocompat to disable compat warnings
Patches 4 - 6 are some preparations for Realm specific changes.

The remaining patches adds Realm support and using the --realm option is
enabled in patch 30.

The v1.0 of the Realm Management Monitor (RMM) specification doesn't support
paging protected memory of a Realm. Thus all of the memory backing the RAM
is locked by the VMM.

Since the IPA space of a Realm is split into Protected and Unprotected, with
one alias of the other, the VMM doubles the IPA Size for a Realm VM.

The KVM support for Arm CCA is advertised with a new cap KVM_CAP_ARM_RME.
A new "VM type" field is defined in the vm_type for CREATE_VM ioctl to indicate
that a VM is "Realm". Once the VM is created, the life cycle of the Realm is
managed via KVM_ENABLE_CAP of KVM_CAP_ARM_RME.

Command line options are also added to configure the Realm parameters.
These include :
- Hash algorithm for measurements
- Realm personalisation value
- SVE vector Length (Optional feature in v1.0 RMM spec. Not yet supported
by the TF-RMM. coming soon).

Support for PMU and self-hosted debug (number of watchpoint/breakpoit registers)
are not supported yet in the KVM/RMM implementation. This will be added soon.

The UABI doesn't support discovering the "supported" configuration values. In
real world, the Realm configuration 'affects' the initial measurement of the
Realms and which may be verified by a remote entity. Thus, the VMM is not at
liberty to make choices for configurations based on the "host" capabilities.
Instead, VMM should launch a Realm with the user requested parameters. If this
cannot be satisfied, there is no point in running the Realm. We are happy to
change this if there is interest.

Special actions are required to load the initial memory images (e.g, kernel,
firmware, DTB, initrd) in to the Realm memory.

For VCPUs, we add a new feature KVM_ARM_VCPU_REC, which will be used to control
the creation of the REC object (via KVM_ARM_VCPU_FINALIZE). This must be done
after the initial register state of the VCPUs are set.
RMM imposes an order in which the RECs are created. i.e., they must be created
in the ascending order of the MPIDR. This is for now a responsibility of the
VMM.

Once the Realm images are loaded, VCPUs created, Realm is activated before
the first vCPU is run.

virtio for the Realms enforces VIRTIO_F_ACCESS_PLATFORM flag.

Also, added support for injecting SEA into the VM for unhandled MMIO.

A tree with the patches are also available here :

https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1

Running the Realm
------------------

A realm VM can be launched using :

$ lkvm run \
--realm \
--disable-sve \
[ --measurement-algo="sha256","sha512" ] \
[ --realm-pv="<realm-pv>" ] \
<normal-VM options>

[0] https://lkml.kernel.org/r/[email protected]
[1] https://lkml.kernel.org/r/[email protected]

To: [email protected]
To: [email protected]
Cc: Alexandru Elisei <[email protected]>
Cc: Andrew Jones <[email protected]>
Cc: Christoffer Dall <[email protected]>
Cc: Fuad Tabba <[email protected]>
Cc: Jean-Philippe Brucker <[email protected]>
Cc: Joey Gouly <[email protected]>
Cc: Marc Zyngier <[email protected]>
Cc: Mark Rutland <[email protected]>
Cc: Oliver Upton <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: Quentin Perret <[email protected]>
Cc: Steven Price <[email protected]>
Cc: Thomas Huth <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: Zenghui Yu <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]

Alexandru Elisei (11):
Add --nocompat option to disable compat warnings
arm64: Add --realm command line option
arm64: Lock realm RAM in memory
arm64: Create Realm Descriptor
arm: Add kernel size to VM context
arm64: Populate initial realm contents
arm64: Finalize realm VCPU after reset
init: Add last_{init, exit} list macros
arm64: Activate realm before the first VCPU is run
arm64: Don't try to debug a realm
arm64: Allow the user to create a realm

Christoffer Dall (4):
arm64: Create a realm virtual machine
arm64: Add --measurement-algo command line option for a realm
arm64: Don't try to set PSTATE for VCPUs belonging to a realm
arm64: Specify SMC as the PSCI conduits for realms

Joey Gouly (2):
mmio: add arch hook for an unhandled MMIO access
arm64: realm: inject an abort on an unhandled MMIO access

Suzuki K Poulose (14):
arm64: Disable MTE when CFI flash is emulated
script: update_headers: Ignore missing architectures
hw: cfi flash: Handle errors in memory transitions
arm64: Check pvtime support against the KVM instance
arm64: Check SVE capability on the VM instance
arm64: Add option to disable SVE
linux: Update kernel headers for RME support
arm64: Add configuration step for Realms
arm64: Add support for Realm Personalisation Value
arm64: Add support for specifying the SVE vector length for Realm
arm64: realm: Double the IPA space
virtio: Add a wrapper for get_host_features
virtio: Add arch specific hook for virtio host flags
arm64: realm: Enforce virtio F_ACCESS_PLATFORM flag

Makefile | 1 +
arm/aarch32/include/asm/realm.h | 13 ++
arm/aarch32/kvm.c | 5 +
arm/aarch64/include/asm/kvm.h | 64 ++++++
arm/aarch64/include/asm/realm.h | 13 ++
arm/aarch64/include/kvm/kvm-config-arch.h | 16 +-
arm/aarch64/kvm-cpu.c | 41 +++-
arm/aarch64/kvm.c | 95 ++++++++-
arm/aarch64/pvtime.c | 4 +-
arm/aarch64/realm.c | 229 ++++++++++++++++++++++
arm/fdt.c | 15 +-
arm/include/arm-common/kvm-arch.h | 4 +
arm/include/arm-common/kvm-config-arch.h | 5 +
arm/kvm-cpu.c | 13 ++
arm/kvm.c | 75 ++++++-
builtin-run.c | 5 +-
guest_compat.c | 1 +
hw/cfi_flash.c | 4 +
include/kvm/kvm-config.h | 1 +
include/kvm/kvm-cpu.h | 2 +
include/kvm/kvm.h | 2 +
include/kvm/util-init.h | 6 +-
include/kvm/virtio.h | 2 +
include/linux/kernel.h | 1 +
include/linux/kvm.h | 22 ++-
include/linux/virtio_blk.h | 19 --
include/linux/virtio_net.h | 14 +-
include/linux/virtio_ring.h | 16 +-
mips/kvm-cpu.c | 4 +
mips/kvm.c | 5 +
mmio.c | 3 +
powerpc/kvm-cpu.c | 4 +
powerpc/kvm.c | 5 +
riscv/kvm-cpu.c | 4 +
riscv/kvm.c | 5 +
util/update_headers.sh | 1 +
virtio/core.c | 8 +
virtio/mmio-legacy.c | 2 +-
virtio/mmio-modern.c | 2 +-
virtio/pci-legacy.c | 2 +-
virtio/pci-modern.c | 2 +-
x86/kvm-cpu.c | 4 +
x86/kvm.c | 5 +
43 files changed, 667 insertions(+), 77 deletions(-)
create mode 100644 arm/aarch32/include/asm/realm.h
create mode 100644 arm/aarch64/include/asm/realm.h
create mode 100644 arm/aarch64/realm.c

--
2.34.1

2023-01-27 11:40:57

[permalink] [raw]

Subject: [RFC kvmtool 03/31] hw: cfi flash: Handle errors in memory transitions

Handle failures in creating the memory maps and back in
transitioning the CFI flash. e.g., with MTE enabled, CFI
flash emulation breaks with the map operation, silently.
And we later hit unhandled aborts in the guest.

To avoid such issues, let us make sure we catch the error
and handle it right at source.

Signed-off-by: Suzuki K Poulose <[email protected]>
---
hw/cfi_flash.c | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/hw/cfi_flash.c b/hw/cfi_flash.c
index 7faecdfb..bce546bc 100644
--- a/hw/cfi_flash.c
+++ b/hw/cfi_flash.c
@@ -455,6 +455,8 @@ static int map_flash_memory(struct kvm *kvm, struct cfi_flash_device *sfdev)
KVM_MEM_TYPE_RAM | KVM_MEM_TYPE_READONLY);
if (!ret)
sfdev->is_mapped = true;
+ else
+ die("CFI Flash: ERROR: Unable to map memory: %d\n", ret);

return ret;
}
@@ -472,6 +474,8 @@ static int unmap_flash_memory(struct kvm *kvm, struct cfi_flash_device *sfdev)

if (!ret)
sfdev->is_mapped = false;
+ else
+ die("CFI Flash: Failed to unmap Flash %d", ret);

return ret;
}
--
2.34.1

2023-01-27 11:41:20

[permalink] [raw]

Subject: [RFC kvmtool 05/31] arm64: Check pvtime support against the KVM instance

KVM_CAP_STEAL_TIME can be checked against a VM instance.
To allow controlling the feature depending on the VM type,
use the cap against the VM.

Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/pvtime.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arm/aarch64/pvtime.c b/arm/aarch64/pvtime.c
index 2933ac7c..839aa8a7 100644
--- a/arm/aarch64/pvtime.c
+++ b/arm/aarch64/pvtime.c
@@ -58,8 +58,8 @@ int kvm_cpu__setup_pvtime(struct kvm_cpu *vcpu)
if (kvm_cfg->no_pvtime)
return 0;

- has_stolen_time = kvm__supports_extension(vcpu->kvm,
- KVM_CAP_STEAL_TIME);
+ has_stolen_time = kvm__supports_vm_extension(vcpu->kvm,
+ KVM_CAP_STEAL_TIME);
if (!has_stolen_time) {
kvm_cfg->no_pvtime = true;
return 0;
--
2.34.1

2023-01-27 11:41:23

[permalink] [raw]

Subject: [RFC kvmtool 06/31] arm64: Check SVE capability on the VM instance

Similar to PVtime, check the SVE capability on the VM instance
to account for the different VM types and the corresponding support.

Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/kvm-cpu.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/arm/aarch64/kvm-cpu.c b/arm/aarch64/kvm-cpu.c
index c8be10b3..da809806 100644
--- a/arm/aarch64/kvm-cpu.c
+++ b/arm/aarch64/kvm-cpu.c
@@ -150,13 +150,15 @@ void kvm_cpu__select_features(struct kvm *kvm, struct kvm_vcpu_init *init)
}

/* Enable SVE if available */
- if (kvm__supports_extension(kvm, KVM_CAP_ARM_SVE))
+ if (kvm__supports_vm_extension(kvm, KVM_CAP_ARM_SVE))
init->features[0] |= 1UL << KVM_ARM_VCPU_SVE;
}

int kvm_cpu__configure_features(struct kvm_cpu *vcpu)
{
- if (kvm__supports_extension(vcpu->kvm, KVM_CAP_ARM_SVE)) {
+ struct kvm *kvm = vcpu->kvm;
+
+ if (kvm__supports_vm_extension(kvm, KVM_CAP_ARM_SVE)) {
int feature = KVM_ARM_VCPU_SVE;

if (ioctl(vcpu->vcpu_fd, KVM_ARM_VCPU_FINALIZE, &feature)) {
--
2.34.1

2023-01-27 11:41:26

[permalink] [raw]

Subject: [RFC kvmtool 07/31] arm64: Add option to disable SVE

kvmtool enables SVE whenever it is supported by the KVM.
However, Realm VMs may want controlled features, which gets
measured during the creation. Thus, provide an option to disable
the SVE, to preserve the current behavior of SVE on by default.

Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/include/kvm/kvm-config-arch.h | 4 +++-
arm/aarch64/kvm-cpu.c | 8 +++++---
arm/include/arm-common/kvm-config-arch.h | 1 +
3 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/arm/aarch64/include/kvm/kvm-config-arch.h b/arm/aarch64/include/kvm/kvm-config-arch.h
index eae8080d..b055fef4 100644
--- a/arm/aarch64/include/kvm/kvm-config-arch.h
+++ b/arm/aarch64/include/kvm/kvm-config-arch.h
@@ -19,7 +19,9 @@ int vcpu_affinity_parser(const struct option *opt, const char *arg, int unset);
"Specify random seed for Kernel Address Space " \
"Layout Randomization (KASLR)"), \
OPT_BOOLEAN('\0', "no-pvtime", &(cfg)->no_pvtime, "Disable" \
- " stolen time"),
+ " stolen time"), \
+ OPT_BOOLEAN('\0', "disable-sve", &(cfg)->disable_sve, \
+ "Disable SVE"),
#include "arm-common/kvm-config-arch.h"

#endif /* KVM__KVM_CONFIG_ARCH_H */
diff --git a/arm/aarch64/kvm-cpu.c b/arm/aarch64/kvm-cpu.c
index da809806..e7649239 100644
--- a/arm/aarch64/kvm-cpu.c
+++ b/arm/aarch64/kvm-cpu.c
@@ -149,8 +149,9 @@ void kvm_cpu__select_features(struct kvm *kvm, struct kvm_vcpu_init *init)
init->features[0] |= 1UL << KVM_ARM_VCPU_PTRAUTH_GENERIC;
}

- /* Enable SVE if available */
- if (kvm__supports_vm_extension(kvm, KVM_CAP_ARM_SVE))
+ /* If SVE is not disabled explicitly, enable if available */
+ if (!kvm->cfg.arch.disable_sve &&
+ kvm__supports_vm_extension(kvm, KVM_CAP_ARM_SVE))
init->features[0] |= 1UL << KVM_ARM_VCPU_SVE;
}

@@ -158,7 +159,8 @@ int kvm_cpu__configure_features(struct kvm_cpu *vcpu)
{
struct kvm *kvm = vcpu->kvm;

- if (kvm__supports_vm_extension(kvm, KVM_CAP_ARM_SVE)) {
+ if (!kvm->cfg.arch.disable_sve &&
+ kvm__supports_vm_extension(kvm, KVM_CAP_ARM_SVE)) {
int feature = KVM_ARM_VCPU_SVE;

if (ioctl(vcpu->vcpu_fd, KVM_ARM_VCPU_FINALIZE, &feature)) {
diff --git a/arm/include/arm-common/kvm-config-arch.h b/arm/include/arm-common/kvm-config-arch.h
index 9949bfe4..6599305b 100644
--- a/arm/include/arm-common/kvm-config-arch.h
+++ b/arm/include/arm-common/kvm-config-arch.h
@@ -15,6 +15,7 @@ struct kvm_config_arch {
enum irqchip_type irqchip;
u64 fw_addr;
bool no_pvtime;
+ bool disable_sve;
};

int irqchip_parser(const struct option *opt, const char *arg, int unset);
--
2.34.1

2023-01-27 11:41:29

[permalink] [raw]

Subject: [RFC kvmtool 08/31] linux: Update kernel headers for RME support

Update the RME specific ABI bits from the kernel headers.

Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/include/asm/kvm.h | 64 +++++++++++++++++++++++++++++++++++
include/linux/kvm.h | 22 +++++++++---
include/linux/virtio_blk.h | 19 -----------
include/linux/virtio_net.h | 14 ++++----
include/linux/virtio_ring.h | 16 +++------
5 files changed, 93 insertions(+), 42 deletions(-)

diff --git a/arm/aarch64/include/asm/kvm.h b/arm/aarch64/include/asm/kvm.h
index 316917b9..653a08fb 100644
--- a/arm/aarch64/include/asm/kvm.h
+++ b/arm/aarch64/include/asm/kvm.h
@@ -108,6 +108,7 @@ struct kvm_regs {
#define KVM_ARM_VCPU_SVE 4 /* enable SVE for this CPU */
#define KVM_ARM_VCPU_PTRAUTH_ADDRESS 5 /* VCPU uses address authentication */
#define KVM_ARM_VCPU_PTRAUTH_GENERIC 6 /* VCPU uses generic authentication */
+#define KVM_ARM_VCPU_REC 7 /* VCPU REC state as part of Realm */

struct kvm_vcpu_init {
__u32 target;
@@ -400,6 +401,69 @@ enum {
#define KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES 3
#define KVM_DEV_ARM_ITS_CTRL_RESET 4

+/* KVM_CAP_ARM_RME kvm_enable_cap->args[0] points to this */
+#define KVM_CAP_ARM_RME_CONFIG_REALM 0
+#define KVM_CAP_ARM_RME_CREATE_RD 1
+#define KVM_CAP_ARM_RME_INIT_IPA_REALM 2
+#define KVM_CAP_ARM_RME_POPULATE_REALM 3
+#define KVM_CAP_ARM_RME_ACTIVATE_REALM 4
+
+#define KVM_CAP_ARM_RME_MEASUREMENT_ALGO_ZERO (0x01ULL << 7)
+#define KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA256 0
+#define KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA512 1
+
+#define KVM_CAP_ARM_RME_RPV_SIZE 64
+
+/* List of configuration items accepted for KVM_CAP_ARM_RME_CONFIG_REALM */
+#define KVM_CAP_ARM_RME_CFG_RPV 0
+#define KVM_CAP_ARM_RME_CFG_HASH_ALGO 1
+#define KVM_CAP_ARM_RME_CFG_SVE 2
+#define KVM_CAP_ARM_RME_CFG_DBG 3
+#define KVM_CAP_ARM_RME_CFG_PMU 4
+
+struct kvm_cap_arm_rme_config_item {
+ __u32 cfg;
+ union {
+ /* cfg == KVM_CAP_ARM_RME_CFG_RPV */
+ struct {
+ __u8 rpv[KVM_CAP_ARM_RME_RPV_SIZE];
+ };
+
+ /* cfg == KVM_CAP_ARM_RME_CFG_HASH_ALGO */
+ struct {
+ __u32 hash_algo;
+ };
+
+ /* cfg == KVM_CAP_ARM_RME_CFG_SVE */
+ struct {
+ __u32 sve_vq;
+ };
+
+ /* cfg == KVM_CAP_ARM_RME_CFG_DBG */
+ struct {
+ __u32 num_brps;
+ __u32 num_wrps;
+ };
+
+ /* cfg == KVM_CAP_ARM_RME_CFG_PMU */
+ struct {
+ __u32 num_pmu_cntrs;
+ };
+ /* Fix the size of the union */
+ __u8 reserved[256];
+ };
+};
+
+struct kvm_cap_arm_rme_populate_realm_args {
+ __u64 populate_ipa_base;
+ __u64 populate_ipa_size;
+};
+
+struct kvm_cap_arm_rme_init_ipa_args {
+ __u64 init_ipa_base;
+ __u64 init_ipa_size;
+};
+
/* Device Control API on vcpu fd */
#define KVM_ARM_VCPU_PMU_V3_CTRL 0
#define KVM_ARM_VCPU_PMU_V3_IRQ 0
diff --git a/include/linux/kvm.h b/include/linux/kvm.h
index 0d5d4419..789c7f89 100644
--- a/include/linux/kvm.h
+++ b/include/linux/kvm.h
@@ -903,14 +903,25 @@ struct kvm_ppc_resize_hpt {
#define KVM_S390_SIE_PAGE_OFFSET 1

/*
- * On arm64, machine type can be used to request the physical
- * address size for the VM. Bits[7-0] are reserved for the guest
- * PA size shift (i.e, log2(PA_Size)). For backward compatibility,
- * value 0 implies the default IPA size, 40bits.
+ * On arm64, machine type can be used to request both the machine type and
+ * the physical address size for the VM.
+ *
+ * Bits[11-8] are reserved for the ARM specific machine type.
+ *
+ * Bits[7-0] are reserved for the guest PA size shift (i.e, log2(PA_Size)).
+ * For backward compatibility, value 0 implies the default IPA size, 40bits.
*/
+#define KVM_VM_TYPE_ARM_SHIFT 8
+#define KVM_VM_TYPE_ARM_MASK (0xfULL << KVM_VM_TYPE_ARM_SHIFT)
+#define KVM_VM_TYPE_ARM(_type) \
+ (((_type) << KVM_VM_TYPE_ARM_SHIFT) & KVM_VM_TYPE_ARM_MASK)
+#define KVM_VM_TYPE_ARM_NORMAL KVM_VM_TYPE_ARM(0)
+#define KVM_VM_TYPE_ARM_REALM KVM_VM_TYPE_ARM(1)
+
#define KVM_VM_TYPE_ARM_IPA_SIZE_MASK 0xffULL
#define KVM_VM_TYPE_ARM_IPA_SIZE(x) \
((x) & KVM_VM_TYPE_ARM_IPA_SIZE_MASK)
+
/*
* ioctls for /dev/kvm fds:
*/
@@ -1177,7 +1188,8 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_VM_DISABLE_NX_HUGE_PAGES 220
#define KVM_CAP_S390_ZPCI_OP 221
#define KVM_CAP_S390_CPU_TOPOLOGY 222
-#define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223
+
+#define KVM_CAP_ARM_RME 300 // FIXME: Large number to prevent conflicts

#ifdef KVM_CAP_IRQ_ROUTING

diff --git a/include/linux/virtio_blk.h b/include/linux/virtio_blk.h
index 58e70b24..d888f013 100644
--- a/include/linux/virtio_blk.h
+++ b/include/linux/virtio_blk.h
@@ -40,7 +40,6 @@
#define VIRTIO_BLK_F_MQ 12 /* support more than one vq */
#define VIRTIO_BLK_F_DISCARD 13 /* DISCARD is supported */
#define VIRTIO_BLK_F_WRITE_ZEROES 14 /* WRITE ZEROES is supported */
-#define VIRTIO_BLK_F_SECURE_ERASE 16 /* Secure Erase is supported */

/* Legacy feature bits */
#ifndef VIRTIO_BLK_NO_LEGACY
@@ -122,21 +121,6 @@ struct virtio_blk_config {
__u8 write_zeroes_may_unmap;

__u8 unused1[3];
-
- /* the next 3 entries are guarded by VIRTIO_BLK_F_SECURE_ERASE */
- /*
- * The maximum secure erase sectors (in 512-byte sectors) for
- * one segment.
- */
- __virtio32 max_secure_erase_sectors;
- /*
- * The maximum number of secure erase segments in a
- * secure erase command.
- */
- __virtio32 max_secure_erase_seg;
- /* Secure erase commands must be aligned to this number of sectors. */
- __virtio32 secure_erase_sector_alignment;
-
} __attribute__((packed));

/*
@@ -171,9 +155,6 @@ struct virtio_blk_config {
/* Write zeroes command */
#define VIRTIO_BLK_T_WRITE_ZEROES 13

-/* Secure erase command */
-#define VIRTIO_BLK_T_SECURE_ERASE 14
-
#ifndef VIRTIO_BLK_NO_LEGACY
/* Barrier before this op. */
#define VIRTIO_BLK_T_BARRIER 0x80000000
diff --git a/include/linux/virtio_net.h b/include/linux/virtio_net.h
index 6cb842ea..29ced555 100644
--- a/include/linux/virtio_net.h
+++ b/include/linux/virtio_net.h
@@ -56,7 +56,7 @@
#define VIRTIO_NET_F_MQ 22 /* Device supports Receive Flow
* Steering */
#define VIRTIO_NET_F_CTRL_MAC_ADDR 23 /* Set MAC address */
-#define VIRTIO_NET_F_NOTF_COAL 53 /* Device supports notifications coalescing */
+#define VIRTIO_NET_F_NOTF_COAL 53 /* Guest can handle notifications coalescing */
#define VIRTIO_NET_F_HASH_REPORT 57 /* Supports hash report */
#define VIRTIO_NET_F_RSS 60 /* Supports RSS RX steering */
#define VIRTIO_NET_F_RSC_EXT 61 /* extended coalescing info */
@@ -364,24 +364,24 @@ struct virtio_net_hash_config {
*/
#define VIRTIO_NET_CTRL_NOTF_COAL 6
/*
- * Set the tx-usecs/tx-max-packets parameters.
+ * Set the tx-usecs/tx-max-packets patameters.
+ * tx-usecs - Maximum number of usecs to delay a TX notification.
+ * tx-max-packets - Maximum number of packets to send before a TX notification.
*/
struct virtio_net_ctrl_coal_tx {
- /* Maximum number of packets to send before a TX notification */
__le32 tx_max_packets;
- /* Maximum number of usecs to delay a TX notification */
__le32 tx_usecs;
};

#define VIRTIO_NET_CTRL_NOTF_COAL_TX_SET 0

/*
- * Set the rx-usecs/rx-max-packets parameters.
+ * Set the rx-usecs/rx-max-packets patameters.
+ * rx-usecs - Maximum number of usecs to delay a RX notification.
+ * rx-max-frames - Maximum number of packets to receive before a RX notification.
*/
struct virtio_net_ctrl_coal_rx {
- /* Maximum number of packets to receive before a RX notification */
__le32 rx_max_packets;
- /* Maximum number of usecs to delay a RX notification */
__le32 rx_usecs;
};

diff --git a/include/linux/virtio_ring.h b/include/linux/virtio_ring.h
index f8c20d3d..476d3e5c 100644
--- a/include/linux/virtio_ring.h
+++ b/include/linux/virtio_ring.h
@@ -93,21 +93,15 @@
#define VRING_USED_ALIGN_SIZE 4
#define VRING_DESC_ALIGN_SIZE 16

-/**
- * struct vring_desc - Virtio ring descriptors,
- * 16 bytes long. These can chain together via @next.
- *
- * @addr: buffer address (guest-physical)
- * @len: buffer length
- * @flags: descriptor flags
- * @next: index of the next descriptor in the chain,
- * if the VRING_DESC_F_NEXT flag is set. We chain unused
- * descriptors via this, too.
- */
+/* Virtio ring descriptors: 16 bytes. These can chain together via "next". */
struct vring_desc {
+ /* Address (guest-physical). */
__virtio64 addr;
+ /* Length. */
__virtio32 len;
+ /* The flags as indicated above. */
__virtio16 flags;
+ /* We chain unused descriptors via this, too */
__virtio16 next;
};

--
2.34.1

2023-01-27 11:41:34

[permalink] [raw]

Subject: [RFC kvmtool 09/31] arm64: Add --realm command line option

From: Alexandru Elisei <[email protected]>

Add the --realm command line option which causes kvmtool to exit with an
error if specified, but which will be enabled once realms are fully
supported by kvmtool.

Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/include/kvm/kvm-config-arch.h | 5 ++++-
arm/aarch64/kvm.c | 20 ++++++++++++++++++--
arm/include/arm-common/kvm-config-arch.h | 1 +
3 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/arm/aarch64/include/kvm/kvm-config-arch.h b/arm/aarch64/include/kvm/kvm-config-arch.h
index b055fef4..d2df850a 100644
--- a/arm/aarch64/include/kvm/kvm-config-arch.h
+++ b/arm/aarch64/include/kvm/kvm-config-arch.h
@@ -21,7 +21,10 @@ int vcpu_affinity_parser(const struct option *opt, const char *arg, int unset);
OPT_BOOLEAN('\0', "no-pvtime", &(cfg)->no_pvtime, "Disable" \
" stolen time"), \
OPT_BOOLEAN('\0', "disable-sve", &(cfg)->disable_sve, \
- "Disable SVE"),
+ "Disable SVE"), \
+ OPT_BOOLEAN('\0', "realm", &(cfg)->is_realm, \
+ "Create VM running in a realm using Arm RME"),
+
#include "arm-common/kvm-config-arch.h"

#endif /* KVM__KVM_CONFIG_ARCH_H */
diff --git a/arm/aarch64/kvm.c b/arm/aarch64/kvm.c
index 5a53badb..25be2f2d 100644
--- a/arm/aarch64/kvm.c
+++ b/arm/aarch64/kvm.c
@@ -38,9 +38,8 @@ int vcpu_affinity_parser(const struct option *opt, const char *arg, int unset)
return 0;
}

-void kvm__arch_validate_cfg(struct kvm *kvm)
+static void validate_mem_cfg(struct kvm *kvm)
{
-
if (kvm->cfg.ram_addr < ARM_MEMORY_AREA) {
die("RAM address is below the I/O region ending at %luGB",
ARM_MEMORY_AREA >> 30);
@@ -52,6 +51,23 @@ void kvm__arch_validate_cfg(struct kvm *kvm)
}
}

+static void validate_realm_cfg(struct kvm *kvm)
+{
+ if (!kvm->cfg.arch.is_realm)
+ return;
+
+ if (kvm->cfg.arch.aarch32_guest)
+ die("Realms supported only for 64bit guests");
+
+ die("Realms not supported");
+}
+
+void kvm__arch_validate_cfg(struct kvm *kvm)
+{
+ validate_mem_cfg(kvm);
+ validate_realm_cfg(kvm);
+}
+
u64 kvm__arch_default_ram_address(void)
{
return ARM_MEMORY_AREA;
diff --git a/arm/include/arm-common/kvm-config-arch.h b/arm/include/arm-common/kvm-config-arch.h
index 6599305b..5eb791da 100644
--- a/arm/include/arm-common/kvm-config-arch.h
+++ b/arm/include/arm-common/kvm-config-arch.h
@@ -11,6 +11,7 @@ struct kvm_config_arch {
bool aarch32_guest;
bool has_pmuv3;
bool mte_disabled;
+ bool is_realm;
u64 kaslr_seed;
enum irqchip_type irqchip;
u64 fw_addr;
--
2.34.1

2023-01-27 11:41:37

[permalink] [raw]

Subject: [RFC kvmtool 10/31] arm64: Create a realm virtual machine

From: Christoffer Dall <[email protected]>

Set the machine type to realm when creating a VM via the KVM_CREATE_VM
ioctl.

Signed-off-by: Christoffer Dall <[email protected]>
[ Alex E: Reworked patch, split the command line option into a different
patch ]
Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/kvm.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/arm/aarch64/kvm.c b/arm/aarch64/kvm.c
index 25be2f2d..5db4c572 100644
--- a/arm/aarch64/kvm.c
+++ b/arm/aarch64/kvm.c
@@ -131,12 +131,15 @@ int kvm__arch_get_ipa_limit(struct kvm *kvm)
int kvm__get_vm_type(struct kvm *kvm)
{
unsigned int ipa_bits, max_ipa_bits;
- unsigned long max_ipa;
+ unsigned long max_ipa, vm_type;

- /* If we're running on an old kernel, use 0 as the VM type */
+ vm_type = kvm->cfg.arch.is_realm ? \
+ KVM_VM_TYPE_ARM_REALM : KVM_VM_TYPE_ARM_NORMAL;
+
+ /* If we're running on an old kernel, use 0 as the IPA bits */
max_ipa_bits = kvm__arch_get_ipa_limit(kvm);
if (!max_ipa_bits)
- return 0;
+ return vm_type;

/* Otherwise, compute the minimal required IPA size */
max_ipa = kvm->cfg.ram_addr + kvm->cfg.ram_size - 1;
@@ -147,7 +150,8 @@ int kvm__get_vm_type(struct kvm *kvm)
if (ipa_bits > max_ipa_bits)
die("Memory too large for this system (needs %d bits, %d available)", ipa_bits, max_ipa_bits);

- return KVM_VM_TYPE_ARM_IPA_SIZE(ipa_bits);
+ vm_type |= KVM_VM_TYPE_ARM_IPA_SIZE(ipa_bits);
+ return vm_type;
}

void kvm__arch_enable_mte(struct kvm *kvm)
--
2.34.1

2023-01-27 11:41:40

[permalink] [raw]

Subject: [RFC kvmtool 11/31] arm64: Lock realm RAM in memory

From: Alexandru Elisei <[email protected]>

RMM doesn't yet support paging protected memory pages. Thus the VMM
must pin the entire VM memory.

Use mlock2 to keep the realm pages pinned in memory once they are faulted
in. Use the MLOCK_ONFAULT flag to prevent pre-mapping the pages and
maintain some semblance of on demand-paging for a realm VM.

Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/kvm.c | 44 ++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 42 insertions(+), 2 deletions(-)

diff --git a/arm/kvm.c b/arm/kvm.c
index d51cc15d..0e40b753 100644
--- a/arm/kvm.c
+++ b/arm/kvm.c
@@ -7,6 +7,8 @@

#include "arm-common/gic.h"

+#include <sys/resource.h>
+
#include <linux/kernel.h>
#include <linux/kvm.h>
#include <linux/sizes.h>
@@ -24,6 +26,25 @@ bool kvm__arch_cpu_supports_vm(void)
return true;
}

+static void try_increase_mlock_limit(struct kvm *kvm)
+{
+ u64 size = kvm->arch.ram_alloc_size;
+ struct rlimit mlock_limit, new_limit;
+
+ if (getrlimit(RLIMIT_MEMLOCK, &mlock_limit)) {
+ perror("getrlimit(RLIMIT_MEMLOCK)");
+ return;
+ }
+
+ if (mlock_limit.rlim_cur > size)
+ return;
+
+ new_limit.rlim_cur = size;
+ new_limit.rlim_max = max((rlim_t)size, mlock_limit.rlim_max);
+ /* Requires CAP_SYS_RESOURCE capability. */
+ setrlimit(RLIMIT_MEMLOCK, &new_limit);
+}
+
void kvm__init_ram(struct kvm *kvm)
{
u64 phys_start, phys_size;
@@ -49,8 +70,27 @@ void kvm__init_ram(struct kvm *kvm)
kvm->ram_start = (void *)ALIGN((unsigned long)kvm->arch.ram_alloc_start,
SZ_2M);

- madvise(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size,
- MADV_MERGEABLE);
+ /*
+ * Do not merge pages if this is a Realm.
+ * a) We cannot replace a page in realm stage2 without export/import
+ *
+ * Pin the realm memory until we have export/import, due to the same
+ * reason as above.
+ *
+ * Use mlock2(,,MLOCK_ONFAULT) to allow faulting in pages and thus
+ * allowing to lazily populate the PAR.
+ */
+ if (kvm->cfg.arch.is_realm) {
+ int ret;
+
+ try_increase_mlock_limit(kvm);
+ ret = mlock2(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size,
+ MLOCK_ONFAULT);
+ if (ret)
+ die_perror("mlock2");
+ } else {
+ madvise(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size, MADV_MERGEABLE);
+ }

madvise(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size,
MADV_HUGEPAGE);
--
2.34.1

2023-01-27 11:41:44

[permalink] [raw]

Subject: [RFC kvmtool 12/31] arm64: Create Realm Descriptor

From: Alexandru Elisei <[email protected]>

Create the Realm Descriptor using the measurement algorithm set
with --measurement-algo.

Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
Makefile | 1 +
arm/aarch32/include/asm/realm.h | 10 ++++++++++
arm/aarch64/include/asm/realm.h | 10 ++++++++++
arm/aarch64/realm.c | 14 ++++++++++++++
arm/kvm.c | 3 +++
5 files changed, 38 insertions(+)
create mode 100644 arm/aarch32/include/asm/realm.h
create mode 100644 arm/aarch64/include/asm/realm.h
create mode 100644 arm/aarch64/realm.c

diff --git a/Makefile b/Makefile
index ed2414bd..88cdf6d2 100644
--- a/Makefile
+++ b/Makefile
@@ -192,6 +192,7 @@ ifeq ($(ARCH), arm64)
OBJS += arm/aarch64/kvm.o
OBJS += arm/aarch64/pvtime.o
OBJS += arm/aarch64/pmu.o
+ OBJS += arm/aarch64/realm.o
ARCH_INCLUDE := $(HDRS_ARM_COMMON)
ARCH_INCLUDE += -Iarm/aarch64/include

diff --git a/arm/aarch32/include/asm/realm.h b/arm/aarch32/include/asm/realm.h
new file mode 100644
index 00000000..5aca6cca
--- /dev/null
+++ b/arm/aarch32/include/asm/realm.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __ASM_REALM_H
+#define __ASM_REALM_H
+
+#include "kvm/kvm.h"
+
+static inline void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm) {}
+
+#endif /* ! __ASM_REALM_H */
diff --git a/arm/aarch64/include/asm/realm.h b/arm/aarch64/include/asm/realm.h
new file mode 100644
index 00000000..e176f15f
--- /dev/null
+++ b/arm/aarch64/include/asm/realm.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __ASM_REALM_H
+#define __ASM_REALM_H
+
+#include "kvm/kvm.h"
+
+void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm);
+
+#endif /* ! __ASM_REALM_H */
diff --git a/arm/aarch64/realm.c b/arm/aarch64/realm.c
new file mode 100644
index 00000000..3a4adb66
--- /dev/null
+++ b/arm/aarch64/realm.c
@@ -0,0 +1,14 @@
+#include "kvm/kvm.h"
+
+#include <asm/realm.h>
+
+void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm)
+{
+ struct kvm_enable_cap rme_create_rd = {
+ .cap = KVM_CAP_ARM_RME,
+ .args[0] = KVM_CAP_ARM_RME_CREATE_RD,
+ };
+
+ if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &rme_create_rd) < 0)
+ die_perror("KVM_CAP_RME(KVM_CAP_ARM_RME_CREATE_RD)");
+}
diff --git a/arm/kvm.c b/arm/kvm.c
index 0e40b753..2510a322 100644
--- a/arm/kvm.c
+++ b/arm/kvm.c
@@ -127,6 +127,9 @@ void kvm__arch_set_cmdline(char *cmdline, bool video)

void kvm__arch_init(struct kvm *kvm)
{
+ if (kvm->cfg.arch.is_realm)
+ kvm_arm_realm_create_realm_descriptor(kvm);
+
/* Create the virtual GIC. */
if (gic__create(kvm, kvm->cfg.arch.irqchip))
die("Failed to create virtual GIC");
--
2.34.1

2023-01-27 11:41:47

[permalink] [raw]

Subject: [RFC kvmtool 13/31] arm64: Add --measurement-algo command line option for a realm

From: Christoffer Dall <[email protected]>

Add the command line option to specify the algorithm that will be used
to create the cryptographic measurement of the realm. Valid options are
"sha256" and "sha512". The final measurement will be a hash using the
selected algorithm

Signed-off-by: Christoffer Dall <[email protected]>
Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/include/kvm/kvm-config-arch.h | 5 ++++-
arm/aarch64/kvm.c | 17 ++++++++++++++++-
arm/include/arm-common/kvm-arch.h | 1 +
arm/include/arm-common/kvm-config-arch.h | 1 +
4 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/arm/aarch64/include/kvm/kvm-config-arch.h b/arm/aarch64/include/kvm/kvm-config-arch.h
index d2df850a..b93999b6 100644
--- a/arm/aarch64/include/kvm/kvm-config-arch.h
+++ b/arm/aarch64/include/kvm/kvm-config-arch.h
@@ -23,7 +23,10 @@ int vcpu_affinity_parser(const struct option *opt, const char *arg, int unset);
OPT_BOOLEAN('\0', "disable-sve", &(cfg)->disable_sve, \
"Disable SVE"), \
OPT_BOOLEAN('\0', "realm", &(cfg)->is_realm, \
- "Create VM running in a realm using Arm RME"),
+ "Create VM running in a realm using Arm RME"), \
+ OPT_STRING('\0', "measurement-algo", &(cfg)->measurement_algo, \
+ "sha256, sha512", \
+ "Realm Measurement algorithm, default: sha256"),

#include "arm-common/kvm-config-arch.h"

diff --git a/arm/aarch64/kvm.c b/arm/aarch64/kvm.c
index 5db4c572..a5a98b2e 100644
--- a/arm/aarch64/kvm.c
+++ b/arm/aarch64/kvm.c
@@ -53,12 +53,27 @@ static void validate_mem_cfg(struct kvm *kvm)

static void validate_realm_cfg(struct kvm *kvm)
{
- if (!kvm->cfg.arch.is_realm)
+ if (!kvm->cfg.arch.is_realm) {
+ if (kvm->cfg.arch.measurement_algo)
+ die("--measurement-algo valid only with --realm");
return;
+ }

if (kvm->cfg.arch.aarch32_guest)
die("Realms supported only for 64bit guests");

+ if (kvm->cfg.arch.measurement_algo) {
+ if (strcmp(kvm->cfg.arch.measurement_algo, "sha256") == 0)
+ kvm->arch.measurement_algo = KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA256;
+ else if (strcmp(kvm->cfg.arch.measurement_algo, "sha512") == 0)
+ kvm->arch.measurement_algo = KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA512;
+ else
+ die("unknown realm measurement algorithm");
+ } else {
+ pr_debug("Realm Hash algorithm: Using default SHA256\n");
+ kvm->arch.measurement_algo = KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA256;
+ }
+
die("Realms not supported");
}

diff --git a/arm/include/arm-common/kvm-arch.h b/arm/include/arm-common/kvm-arch.h
index b2ae373c..68224b1c 100644
--- a/arm/include/arm-common/kvm-arch.h
+++ b/arm/include/arm-common/kvm-arch.h
@@ -113,6 +113,7 @@ struct kvm_arch {
u64 dtb_guest_start;

cpu_set_t *vcpu_affinity_cpuset;
+ u64 measurement_algo;
};

#endif /* ARM_COMMON__KVM_ARCH_H */
diff --git a/arm/include/arm-common/kvm-config-arch.h b/arm/include/arm-common/kvm-config-arch.h
index 5eb791da..a2faa3af 100644
--- a/arm/include/arm-common/kvm-config-arch.h
+++ b/arm/include/arm-common/kvm-config-arch.h
@@ -6,6 +6,7 @@
struct kvm_config_arch {
const char *dump_dtb_filename;
const char *vcpu_affinity;
+ const char *measurement_algo;
unsigned int force_cntfrq;
bool virtio_trans_pci;
bool aarch32_guest;
--
2.34.1

2023-01-27 11:41:49

[permalink] [raw]

Subject: [RFC kvmtool 14/31] arm64: Add configuration step for Realms

Realm must be configured before it is created. Add the
step to specify the parameters for the Realm.

Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/realm.c | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

diff --git a/arm/aarch64/realm.c b/arm/aarch64/realm.c
index 3a4adb66..31543e55 100644
--- a/arm/aarch64/realm.c
+++ b/arm/aarch64/realm.c
@@ -2,6 +2,29 @@

#include <asm/realm.h>

+
+static void realm_configure_hash_algo(struct kvm *kvm)
+{
+ struct kvm_cap_arm_rme_config_item hash_algo_cfg = {
+ .cfg = KVM_CAP_ARM_RME_CFG_HASH_ALGO,
+ .hash_algo = kvm->arch.measurement_algo,
+ };
+
+ struct kvm_enable_cap rme_config = {
+ .cap = KVM_CAP_ARM_RME,
+ .args[0] = KVM_CAP_ARM_RME_CONFIG_REALM,
+ .args[1] = (u64)&hash_algo_cfg,
+ };
+
+ if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &rme_config) < 0)
+ die_perror("KVM_CAP_RME(KVM_CAP_ARM_RME_CONFIG_REALM) hash_algo");
+}
+
+static void realm_configure_parameters(struct kvm *kvm)
+{
+ realm_configure_hash_algo(kvm);
+}
+
void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm)
{
struct kvm_enable_cap rme_create_rd = {
@@ -9,6 +32,7 @@ void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm)
.args[0] = KVM_CAP_ARM_RME_CREATE_RD,
};

+ realm_configure_parameters(kvm);
if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &rme_create_rd) < 0)
die_perror("KVM_CAP_RME(KVM_CAP_ARM_RME_CREATE_RD)");
}
--
2.34.1

2023-01-27 11:42:03

[permalink] [raw]

Subject: [RFC kvmtool 16/31] arm64: Add support for specifying the SVE vector length for Realm

Add option to specify SVE vector length for realms.

Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/include/kvm/kvm-config-arch.h | 6 ++++--
arm/aarch64/kvm.c | 23 +++++++++++++++++++++++
arm/aarch64/realm.c | 21 +++++++++++++++++++++
arm/include/arm-common/kvm-arch.h | 1 +
arm/include/arm-common/kvm-config-arch.h | 1 +
5 files changed, 50 insertions(+), 2 deletions(-)

diff --git a/arm/aarch64/include/kvm/kvm-config-arch.h b/arm/aarch64/include/kvm/kvm-config-arch.h
index f2e659ad..0f42c2c2 100644
--- a/arm/aarch64/include/kvm/kvm-config-arch.h
+++ b/arm/aarch64/include/kvm/kvm-config-arch.h
@@ -29,8 +29,10 @@ int vcpu_affinity_parser(const struct option *opt, const char *arg, int unset);
"Realm Measurement algorithm, default: sha256"),\
OPT_STRING('\0', "realm-pv", &(cfg)->realm_pv, \
"personalisation value", \
- "Personalisation Value (only) for Realm VMs"),
-
+ "Personalisation Value (only) for Realm VMs"), \
+ OPT_U64('\0', "sve-vl", &(cfg)->sve_vl, \
+ "SVE Vector Length the VM" \
+ "(only supported for Realms)"),

#include "arm-common/kvm-config-arch.h"

diff --git a/arm/aarch64/kvm.c b/arm/aarch64/kvm.c
index 4798e359..fca1410b 100644
--- a/arm/aarch64/kvm.c
+++ b/arm/aarch64/kvm.c
@@ -51,13 +51,19 @@ static void validate_mem_cfg(struct kvm *kvm)
}
}

+#define SVE_VL_ALIGN 128
+
static void validate_realm_cfg(struct kvm *kvm)
{
+ u32 sve_vl;
+
if (!kvm->cfg.arch.is_realm) {
if (kvm->cfg.arch.measurement_algo)
die("--measurement-algo valid only with --realm");
if (kvm->cfg.arch.realm_pv)
die("--realm-pv valid only with --realm");
+ if (kvm->cfg.arch.sve_vl)
+ die("--sve-vl valid only with --realm");
return;
}

@@ -76,6 +82,23 @@ static void validate_realm_cfg(struct kvm *kvm)
kvm->arch.measurement_algo = KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA256;
}

+ sve_vl = kvm->cfg.arch.sve_vl;
+ if (sve_vl) {
+ if (kvm->cfg.arch.disable_sve)
+ die("SVE VL requested when SVE is disabled");
+ if (!IS_ALIGNED(sve_vl, SVE_VL_ALIGN))
+ die("SVE VL is not aligned to %dbit\n", SVE_VL_ALIGN);
+ kvm->arch.sve_vq = (sve_vl / SVE_VL_ALIGN) - 1;
+ } else {
+ /*
+ * Disable SVE for Realms, if a VL is not requested.
+ * The SVE VL will be measured as part of the parameter
+ * and we do not want to add an unknown entity to the
+ * measurement.
+ */
+ kvm->cfg.arch.disable_sve = true;
+ }
+
if (kvm->cfg.arch.realm_pv) {
if (strlen(kvm->cfg.arch.realm_pv) > KVM_CAP_ARM_RME_RPV_SIZE)
die("Invalid size for Realm Personalization Value\n");
diff --git a/arm/aarch64/realm.c b/arm/aarch64/realm.c
index 2e0be982..fc7f8d6a 100644
--- a/arm/aarch64/realm.c
+++ b/arm/aarch64/realm.c
@@ -42,10 +42,31 @@ static void realm_configure_rpv(struct kvm *kvm)
die_perror("KVM_CAP_RME(KVM_CAP_ARM_RME_CONFIG_REALM) RPV");
}

+static void realm_configure_sve(struct kvm *kvm)
+{
+ struct kvm_cap_arm_rme_config_item sve_cfg = {
+ .cfg = KVM_CAP_ARM_RME_CFG_SVE,
+ .sve_vq = kvm->arch.sve_vq,
+ };
+
+ struct kvm_enable_cap rme_config = {
+ .cap = KVM_CAP_ARM_RME,
+ .args[0] = KVM_CAP_ARM_RME_CONFIG_REALM,
+ .args[1] = (u64)&sve_cfg,
+ };
+
+ if (kvm->cfg.arch.disable_sve)
+ return;
+
+ if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &rme_config) < 0)
+ die_perror("KVM_CAP_RME(KVM_CAP_ARM_RME_CONFIG_REALM) SVE");
+}
+
static void realm_configure_parameters(struct kvm *kvm)
{
realm_configure_hash_algo(kvm);
realm_configure_rpv(kvm);
+ realm_configure_sve(kvm);
}

void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm)
diff --git a/arm/include/arm-common/kvm-arch.h b/arm/include/arm-common/kvm-arch.h
index 68224b1c..41b31f11 100644
--- a/arm/include/arm-common/kvm-arch.h
+++ b/arm/include/arm-common/kvm-arch.h
@@ -114,6 +114,7 @@ struct kvm_arch {

cpu_set_t *vcpu_affinity_cpuset;
u64 measurement_algo;
+ u64 sve_vq;
};

#endif /* ARM_COMMON__KVM_ARCH_H */
diff --git a/arm/include/arm-common/kvm-config-arch.h b/arm/include/arm-common/kvm-config-arch.h
index 80a3b18e..d923fd9e 100644
--- a/arm/include/arm-common/kvm-config-arch.h
+++ b/arm/include/arm-common/kvm-config-arch.h
@@ -19,6 +19,7 @@ struct kvm_config_arch {
u64 fw_addr;
bool no_pvtime;
bool disable_sve;
+ u64 sve_vl;
};

int irqchip_parser(const struct option *opt, const char *arg, int unset);
--
2.34.1

2023-01-27 11:42:07

[permalink] [raw]

Subject: [RFC kvmtool 17/31] arm: Add kernel size to VM context

From: Alexandru Elisei <[email protected]>

Add the kernel image size to the VM context, as we are going to use it
later. This matches what we already do with the initrd.

Signed-off-by: Alexandru Elisei <[email protected]>
[Fix kernel size printed in debug messages]
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/include/arm-common/kvm-arch.h | 1 +
arm/kvm.c | 8 +++++---
2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/arm/include/arm-common/kvm-arch.h b/arm/include/arm-common/kvm-arch.h
index 41b31f11..b5a4b851 100644
--- a/arm/include/arm-common/kvm-arch.h
+++ b/arm/include/arm-common/kvm-arch.h
@@ -108,6 +108,7 @@ struct kvm_arch {
*/
u64 memory_guest_start;
u64 kern_guest_start;
+ u64 kern_size;
u64 initrd_guest_start;
u64 initrd_size;
u64 dtb_guest_start;
diff --git a/arm/kvm.c b/arm/kvm.c
index 2510a322..acb627b2 100644
--- a/arm/kvm.c
+++ b/arm/kvm.c
@@ -153,7 +153,6 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm, int fd_kernel, int fd_initrd,
limit = kvm->ram_start + min(kvm->ram_size, (u64)SZ_256M) - 1;

pos = kvm->ram_start + kvm__arch_get_kern_offset(kvm, fd_kernel);
- kvm->arch.kern_guest_start = host_to_guest_flat(kvm, pos);
file_size = read_file(fd_kernel, pos, limit - pos);
if (file_size < 0) {
if (errno == ENOMEM)
@@ -161,9 +160,12 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm, int fd_kernel, int fd_initrd,

die_perror("kernel read");
}
+
+ kvm->arch.kern_guest_start = host_to_guest_flat(kvm, pos);
+ kvm->arch.kern_size = file_size;
kernel_end = pos + file_size;
- pr_debug("Loaded kernel to 0x%llx (%zd bytes)",
- kvm->arch.kern_guest_start, file_size);
+ pr_debug("Loaded kernel to 0x%llx (%llu bytes)",
+ kvm->arch.kern_guest_start, kvm->arch.kern_size);

/*
* Now load backwards from the end of memory so the kernel
--
2.34.1

2023-01-27 11:42:12

[permalink] [raw]

Subject: [RFC kvmtool 19/31] arm64: Don't try to set PSTATE for VCPUs belonging to a realm

From: Christoffer Dall <[email protected]>

RME doesn't allow setting the PSTATE but resets it to an architectural
value, and KVM also does not allow setting this register from user
space, so stop trying to do that.

Signed-off-by: Christoffer Dall <[email protected]>
Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/kvm-cpu.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/arm/aarch64/kvm-cpu.c b/arm/aarch64/kvm-cpu.c
index e7649239..37f9aa9d 100644
--- a/arm/aarch64/kvm-cpu.c
+++ b/arm/aarch64/kvm-cpu.c
@@ -92,11 +92,13 @@ static void reset_vcpu_aarch64(struct kvm_cpu *vcpu)

reg.addr = (u64)&data;

- /* pstate = all interrupts masked */
- data = PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT | PSR_MODE_EL1h;
- reg.id = ARM64_CORE_REG(regs.pstate);
- if (ioctl(vcpu->vcpu_fd, KVM_SET_ONE_REG, &reg) < 0)
- die_perror("KVM_SET_ONE_REG failed (spsr[EL1])");
+ if (!kvm->cfg.arch.is_realm) {
+ /* pstate = all interrupts masked */
+ data = PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT | PSR_MODE_EL1h;
+ reg.id = ARM64_CORE_REG(regs.pstate);
+ if (ioctl(vcpu->vcpu_fd, KVM_SET_ONE_REG, &reg) < 0)
+ die_perror("KVM_SET_ONE_REG failed (PSTATE)");
+ }

/* x1...x3 = 0 */
data = 0;
--
2.34.1

2023-01-27 11:42:31

[permalink] [raw]

Subject: [RFC kvmtool 23/31] arm64: Specify SMC as the PSCI conduits for realms

From: Christoffer Dall <[email protected]>

This lets the VM use the RMM implementation for PSCI.

Signed-off-by: Christoffer Dall <[email protected]>
Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/fdt.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/arm/fdt.c b/arm/fdt.c
index 762a604d..c46ff410 100644
--- a/arm/fdt.c
+++ b/arm/fdt.c
@@ -208,7 +208,14 @@ static int setup_fdt(struct kvm *kvm)
_FDT(fdt_property_string(fdt, "compatible", "arm,psci"));
fns = &psci_0_1_fns;
}
- _FDT(fdt_property_string(fdt, "method", "hvc"));
+
+
+ if (kvm->cfg.arch.is_realm) {
+ _FDT(fdt_property_string(fdt, "method", "smc"));
+ } else {
+ _FDT(fdt_property_string(fdt, "method", "hvc"));
+ }
+
_FDT(fdt_property_cell(fdt, "cpu_suspend", fns->cpu_suspend));
_FDT(fdt_property_cell(fdt, "cpu_off", fns->cpu_off));
_FDT(fdt_property_cell(fdt, "cpu_on", fns->cpu_on));
--
2.34.1

2023-01-27 11:42:31

[permalink] [raw]

Subject: [RFC kvmtool 24/31] arm64: Don't try to debug a realm

From: Alexandru Elisei <[email protected]>

Don't read the register values for a running realm, because they don't
reflect the actual hardware state of a realm. And don't try to read realm
memory, because that will promptly lead to kvmtool being killed.

Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/kvm-cpu.c | 8 ++++++++
1 file changed, 8 insertions(+)

diff --git a/arm/aarch64/kvm-cpu.c b/arm/aarch64/kvm-cpu.c
index 32fa7609..a29a3413 100644
--- a/arm/aarch64/kvm-cpu.c
+++ b/arm/aarch64/kvm-cpu.c
@@ -250,6 +250,9 @@ void kvm_cpu__show_code(struct kvm_cpu *vcpu)

reg.addr = (u64)&data;

+ if (vcpu->kvm->cfg.arch.is_realm)
+ return;
+
dprintf(debug_fd, "\n*pc:\n");
reg.id = ARM64_CORE_REG(regs.pc);
if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
@@ -274,6 +277,11 @@ void kvm_cpu__show_registers(struct kvm_cpu *vcpu)
reg.addr = (u64)&data;
dprintf(debug_fd, "\n Registers:\n");

+ if (vcpu->kvm->cfg.arch.is_realm) {
+ dprintf(debug_fd, " UNACCESSIBLE\n");
+ return;
+ }
+
reg.id = ARM64_CORE_REG(regs.pc);
if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
die("KVM_GET_ONE_REG failed (pc)");
--
2.34.1

2023-01-27 11:42:48

[permalink] [raw]

Subject: [RFC kvmtool 26/31] virtio: Add a wrapper for get_host_features

Add a wrapper to the vdev->ops->get_host_features() to allow
setting platform specific flags outside the device

Signed-off-by: Suzuki K Poulose <[email protected]>
---
include/kvm/virtio.h | 2 ++
virtio/core.c | 5 +++++
virtio/mmio-legacy.c | 2 +-
virtio/mmio-modern.c | 2 +-
virtio/pci-legacy.c | 2 +-
virtio/pci-modern.c | 2 +-
6 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/include/kvm/virtio.h b/include/kvm/virtio.h
index 94bddefe..e95cfad5 100644
--- a/include/kvm/virtio.h
+++ b/include/kvm/virtio.h
@@ -248,4 +248,6 @@ void virtio_set_guest_features(struct kvm *kvm, struct virtio_device *vdev,
void virtio_notify_status(struct kvm *kvm, struct virtio_device *vdev,
void *dev, u8 status);

+u64 virtio_dev_get_host_features(struct virtio_device *vdev, struct kvm *kvm, void *dev);
+
#endif /* KVM__VIRTIO_H */
diff --git a/virtio/core.c b/virtio/core.c
index ea0e5b65..50e7f86d 100644
--- a/virtio/core.c
+++ b/virtio/core.c
@@ -283,6 +283,11 @@ void virtio_notify_status(struct kvm *kvm, struct virtio_device *vdev,
vdev->ops->notify_status(kvm, dev, ext_status);
}

+u64 virtio_dev_get_host_features(struct virtio_device *vdev, struct kvm *kvm, void *dev)
+{
+ return vdev->ops->get_host_features(kvm, dev);
+}
+
bool virtio_access_config(struct kvm *kvm, struct virtio_device *vdev,
void *dev, unsigned long offset, void *data,
size_t size, bool is_write)
diff --git a/virtio/mmio-legacy.c b/virtio/mmio-legacy.c
index 7ca7e69f..42673236 100644
--- a/virtio/mmio-legacy.c
+++ b/virtio/mmio-legacy.c
@@ -26,7 +26,7 @@ static void virtio_mmio_config_in(struct kvm_cpu *vcpu,
break;
case VIRTIO_MMIO_DEVICE_FEATURES:
if (vmmio->hdr.host_features_sel == 0)
- val = vdev->ops->get_host_features(vmmio->kvm,
+ val = virtio_dev_get_host_features(vdev, vmmio->kvm,
vmmio->dev);
ioport__write32(data, val);
break;
diff --git a/virtio/mmio-modern.c b/virtio/mmio-modern.c
index 6c0bb382..a09fa8e9 100644
--- a/virtio/mmio-modern.c
+++ b/virtio/mmio-modern.c
@@ -26,7 +26,7 @@ static void virtio_mmio_config_in(struct kvm_cpu *vcpu,
case VIRTIO_MMIO_DEVICE_FEATURES:
if (vmmio->hdr.host_features_sel > 1)
break;
- features |= vdev->ops->get_host_features(vmmio->kvm, vmmio->dev);
+ features |= virtio_dev_get_host_features(vdev, vmmio->kvm, vmmio->dev);
val = features >> (32 * vmmio->hdr.host_features_sel);
break;
case VIRTIO_MMIO_QUEUE_NUM_MAX:
diff --git a/virtio/pci-legacy.c b/virtio/pci-legacy.c
index 58047967..d5f5dee7 100644
--- a/virtio/pci-legacy.c
+++ b/virtio/pci-legacy.c
@@ -44,7 +44,7 @@ static bool virtio_pci__data_in(struct kvm_cpu *vcpu, struct virtio_device *vdev

switch (offset) {
case VIRTIO_PCI_HOST_FEATURES:
- val = vdev->ops->get_host_features(kvm, vpci->dev);
+ val = virtio_dev_get_host_features(vdev, kvm, vpci->dev);
ioport__write32(data, val);
break;
case VIRTIO_PCI_QUEUE_PFN:
diff --git a/virtio/pci-modern.c b/virtio/pci-modern.c
index c5b4bc50..2c5bf3f8 100644
--- a/virtio/pci-modern.c
+++ b/virtio/pci-modern.c
@@ -158,7 +158,7 @@ static bool virtio_pci__common_read(struct virtio_device *vdev,
case VIRTIO_PCI_COMMON_DF:
if (vpci->device_features_sel > 1)
break;
- features |= vdev->ops->get_host_features(vpci->kvm, vpci->dev);
+ features |= virtio_dev_get_host_features(vdev, vpci->kvm, vpci->dev);
val = features >> (32 * vpci->device_features_sel);
ioport__write32(data, val);
break;
--
2.34.1

2023-01-27 11:43:03

[permalink] [raw]

Subject: [RFC kvmtool 27/31] virtio: Add arch specific hook for virtio host flags

Add callbacks for archs to provide virtio host flags.

Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch32/kvm.c | 5 +++++
arm/aarch64/kvm.c | 5 +++++
include/kvm/kvm.h | 2 ++
mips/kvm.c | 5 +++++
powerpc/kvm.c | 5 +++++
riscv/kvm.c | 5 +++++
virtio/core.c | 5 ++++-
x86/kvm.c | 5 +++++
8 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/arm/aarch32/kvm.c b/arm/aarch32/kvm.c
index 768a56bb..849c55d3 100644
--- a/arm/aarch32/kvm.c
+++ b/arm/aarch32/kvm.c
@@ -12,3 +12,8 @@ u64 kvm__arch_default_ram_address(void)
{
return ARM_MEMORY_AREA;
}
+
+u64 kvm__arch_get_virtio_host_features(struct kvm *kvm)
+{
+ return 0;
+}
diff --git a/arm/aarch64/kvm.c b/arm/aarch64/kvm.c
index 344c568b..a4664237 100644
--- a/arm/aarch64/kvm.c
+++ b/arm/aarch64/kvm.c
@@ -234,3 +234,8 @@ void kvm__arch_enable_mte(struct kvm *kvm)

pr_debug("MTE capability enabled");
}
+
+u64 kvm__arch_get_virtio_host_features(struct kvm *kvm)
+{
+ return 0;
+}
diff --git a/include/kvm/kvm.h b/include/kvm/kvm.h
index 3872dc65..a3624de4 100644
--- a/include/kvm/kvm.h
+++ b/include/kvm/kvm.h
@@ -203,6 +203,8 @@ int kvm__arch_free_firmware(struct kvm *kvm);
bool kvm__arch_cpu_supports_vm(void);
void kvm__arch_read_term(struct kvm *kvm);

+u64 kvm__arch_get_virtio_host_features(struct kvm *kvm);
+
#ifdef ARCH_HAS_CFG_RAM_ADDRESS
static inline bool kvm__arch_has_cfg_ram_address(void)
{
diff --git a/mips/kvm.c b/mips/kvm.c
index 0faa03a9..e23d5cf9 100644
--- a/mips/kvm.c
+++ b/mips/kvm.c
@@ -374,3 +374,8 @@ void ioport__map_irq(u8 *irq)
void serial8250__inject_sysrq(struct kvm *kvm, char sysrq)
{
}
+
+u64 kvm__arch_get_virtio_host_features(struct kvm *kvm)
+{
+ return 0;
+}
diff --git a/powerpc/kvm.c b/powerpc/kvm.c
index 7b0d0669..6b3ab93f 100644
--- a/powerpc/kvm.c
+++ b/powerpc/kvm.c
@@ -529,3 +529,8 @@ int kvm__arch_free_firmware(struct kvm *kvm)
{
return 0;
}
+
+u64 kvm__arch_get_virtio_host_features(struct kvm *kvm)
+{
+ return 0;
+}
diff --git a/riscv/kvm.c b/riscv/kvm.c
index 4d6f5cb5..884321ca 100644
--- a/riscv/kvm.c
+++ b/riscv/kvm.c
@@ -182,3 +182,8 @@ int kvm__arch_setup_firmware(struct kvm *kvm)
{
return 0;
}
+
+u64 kvm__arch_get_virtio_host_features(struct kvm *kvm)
+{
+ return 0;
+}
diff --git a/virtio/core.c b/virtio/core.c
index 50e7f86d..674f6fae 100644
--- a/virtio/core.c
+++ b/virtio/core.c
@@ -285,7 +285,10 @@ void virtio_notify_status(struct kvm *kvm, struct virtio_device *vdev,

u64 virtio_dev_get_host_features(struct virtio_device *vdev, struct kvm *kvm, void *dev)
{
- return vdev->ops->get_host_features(kvm, dev);
+ u64 features = kvm__arch_get_virtio_host_features(kvm);
+
+ features |= vdev->ops->get_host_features(kvm, dev);
+ return features;
}

bool virtio_access_config(struct kvm *kvm, struct virtio_device *vdev,
diff --git a/x86/kvm.c b/x86/kvm.c
index 328fa750..961b5d3f 100644
--- a/x86/kvm.c
+++ b/x86/kvm.c
@@ -387,3 +387,8 @@ void kvm__arch_read_term(struct kvm *kvm)
serial8250__update_consoles(kvm);
virtio_console__inject_interrupt(kvm);
}
+
+u64 kvm__arch_get_virtio_host_features(struct kvm *kvm)
+{
+ return 0;
+}
--
2.34.1

2023-01-27 11:43:03

[permalink] [raw]

Subject: [RFC kvmtool 28/31] arm64: realm: Enforce virtio F_ACCESS_PLATFORM flag

For realms, force the F_ACCESS_PLATFORM flag to ensure DMA-APIs
are triggered for virtio in Linux

Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/kvm.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/arm/aarch64/kvm.c b/arm/aarch64/kvm.c
index a4664237..1f3a0def 100644
--- a/arm/aarch64/kvm.c
+++ b/arm/aarch64/kvm.c
@@ -5,6 +5,7 @@
#include <linux/byteorder.h>
#include <linux/cpumask.h>
#include <linux/sizes.h>
+#include <linux/virtio_config.h>

#include <kvm/util.h>

@@ -237,5 +238,10 @@ void kvm__arch_enable_mte(struct kvm *kvm)

u64 kvm__arch_get_virtio_host_features(struct kvm *kvm)
{
- return 0;
+ u64 features = 0;
+
+ /* Enforce F_ACCESS_PLATFORM for Realms */
+ if (kvm->cfg.arch.is_realm)
+ features |= (1ULL << VIRTIO_F_ACCESS_PLATFORM);
+ return features;
}
--
2.34.1

2023-01-27 11:43:03

[permalink] [raw]

Subject: [RFC kvmtool 31/31] arm64: Allow the user to create a realm

From: Alexandru Elisei <[email protected]>

We have everything in place to create a realm, allow the user to do so.

Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/kvm.c | 2 --
1 file changed, 2 deletions(-)

diff --git a/arm/aarch64/kvm.c b/arm/aarch64/kvm.c
index 1f3a0def..422dbec2 100644
--- a/arm/aarch64/kvm.c
+++ b/arm/aarch64/kvm.c
@@ -104,8 +104,6 @@ static void validate_realm_cfg(struct kvm *kvm)
if (strlen(kvm->cfg.arch.realm_pv) > KVM_CAP_ARM_RME_RPV_SIZE)
die("Invalid size for Realm Personalization Value\n");
}
-
- die("Realms not supported");
}

void kvm__arch_validate_cfg(struct kvm *kvm)
--
2.34.1

2023-01-27 11:43:03

[permalink] [raw]

Subject: [RFC kvmtool 29/31] mmio: add arch hook for an unhandled MMIO access

From: Joey Gouly <[email protected]>

Add a hook that allows an architecture to run some code on an
unhandled MMIO access.

Signed-off-by: Joey Gouly <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/kvm-cpu.c | 4 ++++
include/kvm/kvm-cpu.h | 2 ++
mips/kvm-cpu.c | 4 ++++
mmio.c | 3 +++
powerpc/kvm-cpu.c | 4 ++++
riscv/kvm-cpu.c | 4 ++++
x86/kvm-cpu.c | 4 ++++
7 files changed, 25 insertions(+)

diff --git a/arm/kvm-cpu.c b/arm/kvm-cpu.c
index 98bc5fdf..90a15ae9 100644
--- a/arm/kvm-cpu.c
+++ b/arm/kvm-cpu.c
@@ -152,3 +152,7 @@ bool kvm_cpu__handle_exit(struct kvm_cpu *vcpu)
void kvm_cpu__show_page_tables(struct kvm_cpu *vcpu)
{
}
+
+void kvm_cpu__arch_unhandled_mmio(struct kvm_cpu *vcpu)
+{
+}
diff --git a/include/kvm/kvm-cpu.h b/include/kvm/kvm-cpu.h
index 0f16f8d6..d0c40598 100644
--- a/include/kvm/kvm-cpu.h
+++ b/include/kvm/kvm-cpu.h
@@ -29,4 +29,6 @@ void kvm_cpu__show_page_tables(struct kvm_cpu *vcpu);
void kvm_cpu__arch_nmi(struct kvm_cpu *cpu);
void kvm_cpu__run_on_all_cpus(struct kvm *kvm, struct kvm_cpu_task *task);

+void kvm_cpu__arch_unhandled_mmio(struct kvm_cpu *cpu);
+
#endif /* KVM__KVM_CPU_H */
diff --git a/mips/kvm-cpu.c b/mips/kvm-cpu.c
index 30a3de18..0ce88ac3 100644
--- a/mips/kvm-cpu.c
+++ b/mips/kvm-cpu.c
@@ -217,3 +217,7 @@ void kvm_cpu__show_code(struct kvm_cpu *vcpu)
void kvm_cpu__show_page_tables(struct kvm_cpu *vcpu)
{
}
+
+void kvm_cpu__arch_unhandled_mmio(struct kvm_cpu *cpu)
+{
+}
diff --git a/mmio.c b/mmio.c
index 5a114e99..7e31079b 100644
--- a/mmio.c
+++ b/mmio.c
@@ -206,6 +206,9 @@ bool kvm__emulate_mmio(struct kvm_cpu *vcpu, u64 phys_addr, u8 *data,
fprintf(stderr, "Warning: Ignoring MMIO %s at %016llx (length %u)\n",
to_direction(is_write),
(unsigned long long)phys_addr, len);
+
+ kvm_cpu__arch_unhandled_mmio(vcpu);
+
goto out;
}

diff --git a/powerpc/kvm-cpu.c b/powerpc/kvm-cpu.c
index 461e0a90..e0c20f9d 100644
--- a/powerpc/kvm-cpu.c
+++ b/powerpc/kvm-cpu.c
@@ -288,3 +288,7 @@ void kvm_cpu__show_page_tables(struct kvm_cpu *vcpu)
{
/* Does nothing yet */
}
+
+void kvm_cpu__arch_unhandled_mmio(struct kvm_cpu *cpu)
+{
+}
diff --git a/riscv/kvm-cpu.c b/riscv/kvm-cpu.c
index f98bd7ae..8417e361 100644
--- a/riscv/kvm-cpu.c
+++ b/riscv/kvm-cpu.c
@@ -461,3 +461,7 @@ void kvm_cpu__show_registers(struct kvm_cpu *vcpu)

kvm_cpu__show_csrs(vcpu);
}
+
+void kvm_cpu__arch_unhandled_mmio(struct kvm_cpu *cpu)
+{
+}
diff --git a/x86/kvm-cpu.c b/x86/kvm-cpu.c
index b02ff65e..ac075ee4 100644
--- a/x86/kvm-cpu.c
+++ b/x86/kvm-cpu.c
@@ -444,3 +444,7 @@ void kvm_cpu__arch_nmi(struct kvm_cpu *cpu)

ioctl(cpu->vcpu_fd, KVM_NMI);
}
+
+void kvm_cpu__arch_unhandled_mmio(struct kvm_cpu *cpu)
+{
+}
--
2.34.1

2023-01-27 11:43:06

[permalink] [raw]

Subject: [RFC kvmtool 30/31] arm64: realm: inject an abort on an unhandled MMIO access

From: Joey Gouly <[email protected]>

For Realms, inject a synchronous external abort, instead of ignoring unknown
MMIO accesses.

Signed-off-by: Joey Gouly <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/kvm-cpu.c | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/arm/kvm-cpu.c b/arm/kvm-cpu.c
index 90a15ae9..c96d75eb 100644
--- a/arm/kvm-cpu.c
+++ b/arm/kvm-cpu.c
@@ -155,4 +155,13 @@ void kvm_cpu__show_page_tables(struct kvm_cpu *vcpu)

void kvm_cpu__arch_unhandled_mmio(struct kvm_cpu *vcpu)
{
+ struct kvm_vcpu_events events = { };
+
+ if (!vcpu->kvm->cfg.arch.is_realm)
+ return;
+
+ events.exception.ext_dabt_pending = 1;
+
+ if (ioctl(vcpu->vcpu_fd, KVM_SET_VCPU_EVENTS, &events) < 0)
+ die_perror("KVM_SET_VCPU_EVENTS failed");
}
--
2.34.1

2023-01-27 11:43:53

[permalink] [raw]

Subject: [RFC kvmtool 25/31] arm64: realm: Double the IPA space

The Realm's IPA space is divided into 2 halves. Protected
(lower half) and Unprotected (upper half). KVM implements
aliasing of the IPA, where the unprotected IPA is alias of
the corresponding protected ipa. Thus we must double the
IPA space required for a given VM.

Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/kvm.c | 3 +++
1 file changed, 3 insertions(+)

diff --git a/arm/aarch64/kvm.c b/arm/aarch64/kvm.c
index fca1410b..344c568b 100644
--- a/arm/aarch64/kvm.c
+++ b/arm/aarch64/kvm.c
@@ -189,6 +189,9 @@ int kvm__get_vm_type(struct kvm *kvm)
/* Otherwise, compute the minimal required IPA size */
max_ipa = kvm->cfg.ram_addr + kvm->cfg.ram_size - 1;
ipa_bits = max(32, fls_long(max_ipa));
+ /* Realm needs double the IPA space */
+ if (kvm->cfg.arch.is_realm)
+ ipa_bits++;
pr_debug("max_ipa %lx ipa_bits %d max_ipa_bits %d",
max_ipa, ipa_bits, max_ipa_bits);

--
2.34.1

2023-01-27 11:44:18

[permalink] [raw]

Subject: [RFC kvm-unit-tests 00/27] Support for Arm Confidential Compute Architecture

This series adds support for running the kvm-unit-tests in the Arm CCA reference
software architecture. See more details on Arm CCA and how to build/run the
entire stack here [0].

This involves enlightening the boot/setup code with the Realm Service Interface
(RSI). The series also includes new test cases that exercise the RSI calls.

Currently we only support "kvmtool" as the VMM for running Realms. There was
an attempt to add support for running the test scripts using with kvmtool here [1],
which hasn't progressed. It would be good to have that resolved, so that we can
run all the tests without manually specifying the commandlines for each run.
For the purposes of running the Realm specific tests, we have added a "temporary"
script "run-realm-tests" until the kvmtool support is added. We do not expect
this to be merged.

Base Realm Support
-------------------

Realm IPA Space
---------------
When running on in Realm world, the (Guest) Physical Address - aka Intermediate
Physical Address (IPA) in Arm terminology - space of the VM is split into two halves,
protected (lower half) and un-protected (upper half). A protected IPA will
always map pages in the "realm world" and the contents are not accessible to
the host. An unprotected IPA on the other hand can be mapped to page in the
"normal world" and thus shared with the host. All host emulated MMIO ranges must
be in unprotected IPA space.

Realm can query the Realm Management Monitor for the configuration via RSI call
(RSI_REALM_CONFIG) and identify the "boundary" of the "IPA" split.

As far as the hyp/VMM is concerned, there is only one "IPA space" (the lower
half) of memory map. The "upper half" is "unprotected alias" of the memory map.

In the guest, this is achieved by "treating the MSB (1 << (IPA_WIDTH - 1))" as
a protection attribute (PTE_NS_SHARED), where the Realm applies this to any
address, it thinks is acccessed/managed by host (e.g., MMIO, shared pages).
Given that this is runtime variable (but fixed for a given Realm), uses a
variable to track the value.

All I/O regions are marked as "shared". Care is taken to ensure I/O access (uart)
with MMU off uses the "Unprotected Physical address".

Realm IPA State
---------------
Additionally, each page (4K) in the protected IPA space has a state associated
(Realm IPA State - RIPAS) with it. It is either of :
RIPAS_EMPTY
RIPAS_RAM

Any IPA backed by RAM, must be marked as RIPAS_RAM before an access is made to
it. The hypervisor/VMM does this for the initial image loaded into the Realm
memory before the Realm starts execution. Given the kvm-unit-test flat files do
not contain a metadata header (e.g., like the arm64 Linux kernel Image),
indicating the "actual image size in memory", the VMM cannot transition the
area towards the end of the image (e.g., bss, stack) which are accessed very
early during boot. Thus the early boot assembly code will mark the area upto
the stack as RAM.

Once we land in the C code, we mark target relocation area for FDT and
initrd as RIPAS_RAM. At this point, we can scan the FDT and mark all RAM memory
blocks as RIPAS_RAM.

TODO: It would be good to add an image header to the flat files indicating the
size, which can take the burden off doing the early assembly boot code RSI calls.

Shared Memory support
---------------------
Given the "default" memory of a VM is not accessible to host, we add new page
alloc/free routines for "memory shared" with the host. e.g., GICv3-ITS must use
shared pages for ITS emulation.

RSI Test suites
--------------
There are new testcases added to exercise the RSI interfaces and the RMM flows.

Attestation and measurement services related RSI tests require parsing tokens
and claims returned by the RMM. This is achieved with the help of QCBOR library
[2], which is added as a submodule to the project. We have also added a wrapper
library - libtokenverifier - around the QCBOR to parse the tokens according to
the RMM specifications.

The patches are also available here:

https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1

[0] https://lore.kernel.org/all/[email protected]/
[1] https://lkml.kernel.org/r/[email protected]
[2] https://github.com/laurencelundblade/QCBOR

Thanks,
Joey

Cc: Alexandru Elisei <[email protected]>
Cc: Andrew Jones <[email protected]>
Cc: Christoffer Dall <[email protected]>
Cc: Fuad Tabba <[email protected]>
Cc: Jean-Philippe Brucker <[email protected]>
Cc: Joey Gouly <[email protected]>
Cc: Marc Zyngier <[email protected]>
Cc: Mark Rutland <[email protected]>
Cc: Oliver Upton <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: Quentin Perret <[email protected]>
Cc: Steven Price <[email protected]>
Cc: Suzuki K Poulose <[email protected]>
Cc: Thomas Huth <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: Zenghui Yu <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]

Alexandru Elisei (3):
arm: Expand SMCCC arguments and return values
arm: selftest: realm: skip pabt test when running in a realm
NOT-FOR-MERGING: add run-realm-tests

Djordje Kovacevic (1):
arm: realm: Add tests for in realm SEA

Gareth Stockwell (1):
arm: realm: add hvc and RSI_HOST_CALL tests

Jean-Philippe Brucker (1):
arm: Move io_init after vm initialization

Joey Gouly (9):
lib/string: include stddef.h for size_t
arm: realm: Add RSI interface header
arm: Make physical address mask dynamic
arm: Introduce NS_SHARED PTE attribute
arm: realm: Make uart available before MMU is enabled
arm: realm: Realm initialisation
arm: realm: Add support for changing the state of memory
arm: realm: Add RSI version test
lib/alloc_page: Add shared page allocation support

Mate Toth-Pal (2):
arm: Add a library to verify tokens using the QCBOR library
arm: realm: Add Realm attestation tests

Subhasish Ghosh (1):
arm: realm: Add test for FPU/SIMD context save/restore

Suzuki K Poulose (9):
arm: realm: Set RIPAS state for RAM
arm: realm: Early memory setup
arm: gic-v3-its: Use shared pages wherever needed
arm: realm: Enable memory encryption
qcbor: Add QCBOR as a submodule
arm: Add build steps for QCBOR library
arm: realm: add RSI interface for attestation measurements
arm: realm: Add helpers to decode RSI return codes
arm: realm: Add a test for shared memory

.gitmodules | 3 +
arm/Makefile.arm64 | 17 +-
arm/Makefile.common | 1 +
arm/cstart.S | 49 +-
arm/cstart64.S | 123 ++-
arm/realm-attest.c | 1125 +++++++++++++++++++++++++++
arm/realm-fpu.c | 242 ++++++
arm/realm-ns-memory.c | 86 ++
arm/realm-rsi.c | 157 ++++
arm/realm-sea.c | 143 ++++
arm/run-realm-tests | 56 ++
arm/selftest.c | 9 +-
arm/unittests.cfg | 94 +++
lib/alloc_page.c | 34 +-
lib/alloc_page.h | 24 +
lib/arm/asm/arm-smccc.h | 44 ++
lib/arm/asm/psci.h | 13 +-
lib/arm/asm/rsi.h | 16 +
lib/arm/gic-v3.c | 6 +-
lib/arm/io.c | 24 +-
lib/arm/mmu.c | 73 +-
lib/arm/psci.c | 19 +-
lib/arm/setup.c | 17 +-
lib/arm64/asm/arm-smccc.h | 6 +
lib/arm64/asm/io.h | 6 +
lib/arm64/asm/pgtable-hwdef.h | 6 -
lib/arm64/asm/pgtable.h | 20 +
lib/arm64/asm/processor.h | 8 +
lib/arm64/asm/rsi.h | 84 ++
lib/arm64/asm/smc-rsi.h | 139 ++++
lib/arm64/gic-v3-its.c | 6 +-
lib/arm64/rsi.c | 143 ++++
lib/libcflat.h | 1 +
lib/qcbor | 1 +
lib/string.h | 2 +
lib/token_verifier/attest_defines.h | 50 ++
lib/token_verifier/token_dumper.c | 158 ++++
lib/token_verifier/token_dumper.h | 15 +
lib/token_verifier/token_verifier.c | 591 ++++++++++++++
lib/token_verifier/token_verifier.h | 77 ++
40 files changed, 3640 insertions(+), 48 deletions(-)
create mode 100644 .gitmodules
create mode 100644 arm/realm-attest.c
create mode 100644 arm/realm-fpu.c
create mode 100644 arm/realm-ns-memory.c
create mode 100644 arm/realm-rsi.c
create mode 100644 arm/realm-sea.c
create mode 100755 arm/run-realm-tests
create mode 100644 lib/arm/asm/arm-smccc.h
create mode 100644 lib/arm/asm/rsi.h
create mode 100644 lib/arm64/asm/arm-smccc.h
create mode 100644 lib/arm64/asm/rsi.h
create mode 100644 lib/arm64/asm/smc-rsi.h
create mode 100644 lib/arm64/rsi.c
create mode 160000 lib/qcbor
create mode 100644 lib/token_verifier/attest_defines.h
create mode 100644 lib/token_verifier/token_dumper.c
create mode 100644 lib/token_verifier/token_dumper.h
create mode 100644 lib/token_verifier/token_verifier.c
create mode 100644 lib/token_verifier/token_verifier.h

--
2.17.1

2023-01-27 11:44:59

[permalink] [raw]

Subject: [RFC kvm-unit-tests 05/27] arm: Introduce NS_SHARED PTE attribute

Introduce a new attribute to indicate the mapping is "Shared" with the
host. This will be used by the Realms to share pages with the Host.
For normal VMs, this is always 0.

For realms, this is dynamic, depending on the IPA width. The top bit of the
IPA is "treated" as the "NS_SHARED" attribute, making the VM access the
unprotected alias of the IPA.

By default, apply the NS_SHARED attribute for all I/O.

Signed-off-by: Joey Gouly <[email protected]>
---
lib/arm/mmu.c | 5 ++++-
lib/arm64/asm/pgtable.h | 6 ++++++
2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
index acaf5614..6f1f42f5 100644
--- a/lib/arm/mmu.c
+++ b/lib/arm/mmu.c
@@ -22,6 +22,8 @@

pgd_t *mmu_idmap;

+/* Used by Realms, depends on IPA size */
+unsigned long prot_ns_shared = 0;
unsigned long phys_mask_shift = 48;

/* CPU 0 starts with disabled MMU */
@@ -194,7 +196,8 @@ void __iomem *__ioremap(phys_addr_t phys_addr, size_t size)
{
phys_addr_t paddr_aligned = phys_addr & PAGE_MASK;
phys_addr_t paddr_end = PAGE_ALIGN(phys_addr + size);
- pgprot_t prot = __pgprot(PTE_UNCACHED | PTE_USER | PTE_UXN | PTE_PXN);
+ pgprot_t prot = __pgprot(PTE_UNCACHED | PTE_USER | PTE_UXN |
+ PTE_PXN | PTE_NS_SHARED);
pgd_t *pgtable;

assert(sizeof(long) == 8 || !(phys_addr >> 32));
diff --git a/lib/arm64/asm/pgtable.h b/lib/arm64/asm/pgtable.h
index 22ce64f0..5b9f40b0 100644
--- a/lib/arm64/asm/pgtable.h
+++ b/lib/arm64/asm/pgtable.h
@@ -22,6 +22,12 @@
#include <linux/compiler.h>

extern unsigned long prot_ns_shared;
+/*
+ * The Non-secure shared bit for Realms is actually part of the output
+ * address, however it is modeled as a PTE attribute.
+*/
+#define PTE_NS_SHARED (prot_ns_shared)
+
/*
* Highest possible physical address supported.
*/
--
2.17.1

2023-01-27 11:45:12

[permalink] [raw]

Subject: [RFC kvm-unit-tests 07/27] arm: realm: Make uart available before MMU is enabled

A Realm must access any emulated I/O mappings with the PTE_NS_SHARED bit set.
This is modelled as a PTE attribute, but is actually part of the address.

So, when MMU is disabled, the "physical address" must reflect this bit set. We
access the UART early before the MMU is enabled. So, make sure the UART is
accessed always with the bit set.

Signed-off-by: Joey Gouly <[email protected]>
---
lib/arm/io.c | 24 +++++++++++++++++++++++-
lib/arm64/asm/pgtable.h | 5 +++++
2 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/lib/arm/io.c b/lib/arm/io.c
index 343e1082..f7c6c771 100644
--- a/lib/arm/io.c
+++ b/lib/arm/io.c
@@ -15,6 +15,8 @@
#include <asm/psci.h>
#include <asm/spinlock.h>
#include <asm/io.h>
+#include <asm/mmu-api.h>
+#include <asm/pgtable.h>

#include "io.h"

@@ -29,6 +31,24 @@ static struct spinlock uart_lock;
#define UART_EARLY_BASE (u8 *)(unsigned long)CONFIG_UART_EARLY_BASE
static volatile u8 *uart0_base = UART_EARLY_BASE;

+static inline volatile u8 *get_uart_base(void)
+{
+ /*
+ * The address of the UART base may be different
+ * based on whether we are running with/without
+ * MMU enabled.
+ *
+ * For realms, we must force to use the shared physical
+ * alias with MMU disabled, to make sure the I/O can
+ * be emulated.
+ * When the MMU is turned ON, the mappings are created
+ * appropriately.
+ */
+ if (mmu_enabled())
+ return uart0_base;
+ return (u8 *)arm_shared_phys_alias((void *)uart0_base);
+}
+
static void uart0_init(void)
{
/*
@@ -81,9 +101,11 @@ void io_init(void)

void puts(const char *s)
{
+ volatile u8 *uart_base = get_uart_base();
+
spin_lock(&uart_lock);
while (*s)
- writeb(*s++, uart0_base);
+ writeb(*s++, uart_base);
spin_unlock(&uart_lock);
}

diff --git a/lib/arm64/asm/pgtable.h b/lib/arm64/asm/pgtable.h
index 5b9f40b0..871c03e9 100644
--- a/lib/arm64/asm/pgtable.h
+++ b/lib/arm64/asm/pgtable.h
@@ -28,6 +28,11 @@ extern unsigned long prot_ns_shared;
*/
#define PTE_NS_SHARED (prot_ns_shared)

+static inline unsigned long arm_shared_phys_alias(void *addr)
+{
+ return ((unsigned long)addr | PTE_NS_SHARED);
+}
+
/*
* Highest possible physical address supported.
*/
--
2.17.1

2023-01-27 11:45:23

[permalink] [raw]

Subject: [RFC kvm-unit-tests 13/27] arm: selftest: realm: skip pabt test when running in a realm

From: Alexandru Elisei <[email protected]>

The realm manager treats instruction aborts as fatal errors, skip this
test.

Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
arm/selftest.c | 7 +++++++
1 file changed, 7 insertions(+)

diff --git a/arm/selftest.c b/arm/selftest.c
index 6f825add..174f2ebc 100644
--- a/arm/selftest.c
+++ b/arm/selftest.c
@@ -18,6 +18,7 @@
#include <asm/smp.h>
#include <asm/mmu.h>
#include <asm/barrier.h>
+#include <asm/rsi.h>

static cpumask_t ready, valid;

@@ -392,11 +393,17 @@ static void check_vectors(void *arg __unused)
user_psci_system_off);
#endif
} else {
+ if (is_realm()) {
+ report_skip("pabt test not supported in a realm");
+ goto out;
+ }
+
if (!check_pabt_init())
report_skip("Couldn't guess an invalid physical address");
else
report(check_pabt(), "pabt");
}
+out:
exit(report_summary());
}

--
2.17.1

2023-01-27 11:46:04

[permalink] [raw]

Subject: [RFC kvm-unit-tests 14/27] arm: realm: add hvc and RSI_HOST_CALL tests

From: Gareth Stockwell <[email protected]>

Test that a HVC instruction in a Realm is turned into an undefined exception.

Test that RSI_HOST_CALL passes through to the Hypervisor.

Signed-off-by: Gareth Stockwell <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
arm/realm-rsi.c | 110 +++++++++++++++++++++++++++++++++++++++++++++-
arm/unittests.cfg | 15 +++++++
2 files changed, 124 insertions(+), 1 deletion(-)

diff --git a/arm/realm-rsi.c b/arm/realm-rsi.c
index d793f305..8a7e9622 100644
--- a/arm/realm-rsi.c
+++ b/arm/realm-rsi.c
@@ -14,6 +14,96 @@
#include <asm/pgtable.h>
#include <asm/processor.h>

+#define FID_SMCCC_VERSION 0x80000000
+#define FID_INVALID 0xc5000041
+
+#define SMCCC_VERSION_1_1 0x10001
+#define SMCCC_SUCCESS 0
+#define SMCCC_NOT_SUPPORTED -1
+
+static bool unknown_taken;
+
+static void unknown_handler(struct pt_regs *regs, unsigned int esr)
+{
+ report_info("unknown_handler: esr=0x%x", esr);
+ unknown_taken = true;
+}
+
+static void hvc_call(unsigned int fid)
+{
+ struct smccc_result res;
+
+ unknown_taken = false;
+ arm_smccc_hvc(fid, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &res);
+
+ if (unknown_taken) {
+ report(true, "FID=0x%x caused Unknown exception", fid);
+ } else {
+ report(false, "FID=0x%x did not cause Unknown exception", fid);
+ report_info("x0: 0x%lx", res.r0);
+ report_info("x1: 0x%lx", res.r1);
+ report_info("x2: 0x%lx", res.r2);
+ report_info("x3: 0x%lx", res.r3);
+ report_info("x4: 0x%lx", res.r4);
+ report_info("x5: 0x%lx", res.r5);
+ report_info("x6: 0x%lx", res.r6);
+ report_info("x7: 0x%lx", res.r7);
+ }
+}
+
+static void rsi_test_hvc(void)
+{
+ report_prefix_push("hvc");
+
+ /* Test that HVC causes Undefined exception, regardless of FID */
+ install_exception_handler(EL1H_SYNC, ESR_EL1_EC_UNKNOWN, unknown_handler);
+ hvc_call(FID_SMCCC_VERSION);
+ hvc_call(FID_INVALID);
+ install_exception_handler(EL1H_SYNC, ESR_EL1_EC_UNKNOWN, NULL);
+
+ report_prefix_pop();
+}
+
+static void host_call(unsigned int fid, unsigned long expected_x0)
+{
+ struct smccc_result res;
+ struct rsi_host_call __attribute__((aligned(256))) host_call_data = { 0 };
+
+ host_call_data.gprs[0] = fid;
+
+ arm_smccc_smc(SMC_RSI_HOST_CALL, virt_to_phys(&host_call_data),
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &res);
+
+ if (res.r0) {
+ report(false, "RSI_HOST_CALL returned 0x%lx", res.r0);
+ } else {
+ if (host_call_data.gprs[0] == expected_x0) {
+ report(true, "FID=0x%x x0=0x%lx",
+ fid, host_call_data.gprs[0]);
+ } else {
+ report(false, "FID=0x%x x0=0x%lx expected=0x%lx",
+ fid, host_call_data.gprs[0], expected_x0);
+ report_info("x1: 0x%lx", host_call_data.gprs[1]);
+ report_info("x2: 0x%lx", host_call_data.gprs[2]);
+ report_info("x3: 0x%lx", host_call_data.gprs[3]);
+ report_info("x4: 0x%lx", host_call_data.gprs[4]);
+ report_info("x5: 0x%lx", host_call_data.gprs[5]);
+ report_info("x6: 0x%lx", host_call_data.gprs[6]);
+ }
+ }
+}
+
+static void rsi_test_host_call(void)
+{
+ report_prefix_push("host_call");
+
+ /* Test that host calls return expected values */
+ host_call(FID_SMCCC_VERSION, SMCCC_VERSION_1_1);
+ host_call(FID_INVALID, SMCCC_NOT_SUPPORTED);
+
+ report_prefix_pop();
+}
+
static void rsi_test_version(void)
{
int version;
@@ -36,6 +126,8 @@ static void rsi_test_version(void)

int main(int argc, char **argv)
{
+ int i;
+
report_prefix_push("rsi");

if (!is_realm()) {
@@ -43,7 +135,23 @@ int main(int argc, char **argv)
goto exit;
}

- rsi_test_version();
+ if (argc < 2) {
+ rsi_test_version();
+ rsi_test_host_call();
+ rsi_test_hvc();
+ } else {
+ for (i = 1; i < argc; i++) {
+ if (strcmp(argv[i], "version") == 0) {
+ rsi_test_version();
+ } else if (strcmp(argv[i], "hvc") == 0) {
+ rsi_test_hvc();
+ } else if (strcmp(argv[i], "host_call") == 0) {
+ rsi_test_host_call();
+ } else {
+ report_abort("Unknown subtest '%s'", argv[1]);
+ }
+ }
+ }
exit:
return report_summary();
}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index ce1b5ad9..3cdb1a98 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -280,5 +280,20 @@ groups = debug migration
[realm-rsi]
file = realm-rsi.flat
groups = nodefault realms
+extra_params = -append 'version'
+accel = kvm
+arch = arm64
+
+[realm-host-call]
+file = realm-rsi.flat
+groups = nodefault realms
+extra_params = -append 'host_call'
+accel = kvm
+arch = arm64
+
+[realm-hvc]
+file = realm-rsi.flat
+groups = nodefault realms
+extra_params = -append 'hvc'
accel = kvm
arch = arm64
--
2.17.1

2023-01-27 11:46:24

[permalink] [raw]

Subject: [RFC kvm-unit-tests 19/27] arm: realm: Enable memory encryption

From: Suzuki K Poulose <[email protected]>

Enable memory encryption support for Realms.

When a page is "decrypted", we set the RIPAS to EMPTY, hinting to the hypervisor
that it could reclaim the page backing the IPA. Also the pagetable is updated
with the PTE_NS_SHARED attrbiute, whic in effect turns the "ipa" to the
unprotected alias.

Similarly for "encryption" we mark the IPA back to RIPAS_RAM and clear the
PTE_NS_SHARED attribute.

Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
lib/arm/mmu.c | 65 ++++++++++++++++++++++++++++++++++++++++++++--
lib/arm64/asm/io.h | 6 +++++
2 files changed, 69 insertions(+), 2 deletions(-)

diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
index 2b5a7141..d4fbe56a 100644
--- a/lib/arm/mmu.c
+++ b/lib/arm/mmu.c
@@ -22,6 +22,7 @@
#include <linux/compiler.h>

pgd_t *mmu_idmap;
+unsigned long idmap_end;

/* Used by Realms, depends on IPA size */
unsigned long prot_ns_shared = 0;
@@ -30,6 +31,11 @@ unsigned long phys_mask_shift = 48;
/* CPU 0 starts with disabled MMU */
static cpumask_t mmu_enabled_cpumask;

+static bool is_idmap_address(phys_addr_t pa)
+{
+ return pa < idmap_end;
+}
+
bool mmu_enabled(void)
{
/*
@@ -92,12 +98,17 @@ static pteval_t *get_pte(pgd_t *pgtable, uintptr_t vaddr)
return &pte_val(*pte);
}

+static void set_pte(uintptr_t vaddr, pteval_t *p_pte, pteval_t pte)
+{
+ WRITE_ONCE(*p_pte, pte);
+ flush_tlb_page(vaddr);
+}
+
static pteval_t *install_pte(pgd_t *pgtable, uintptr_t vaddr, pteval_t pte)
{
pteval_t *p_pte = get_pte(pgtable, vaddr);

- WRITE_ONCE(*p_pte, pte);
- flush_tlb_page(vaddr);
+ set_pte(vaddr, p_pte, pte);
return p_pte;
}

@@ -122,6 +133,39 @@ phys_addr_t virt_to_pte_phys(pgd_t *pgtable, void *mem)
+ ((ulong)mem & (PAGE_SIZE - 1));
}

+/*
+ * __idmap_set_range_prot - Apply permissions to the given idmap range.
+ */
+static void __idmap_set_range_prot(unsigned long virt_offset, size_t size, pgprot_t prot)
+{
+ pteval_t *ptep;
+ pteval_t default_prot = PTE_TYPE_PAGE | PTE_AF | PTE_SHARED;
+
+ while (size > 0) {
+ pteval_t pte = virt_offset | default_prot | pgprot_val(prot);
+
+ if (!is_idmap_address(virt_offset))
+ break;
+ /* Break before make : Clear the PTE entry first */
+ ptep = install_pte(mmu_idmap, (uintptr_t)virt_offset, 0);
+ /* Now apply the changes */
+ set_pte((uintptr_t)virt_offset, ptep, pte);
+
+ size -= PAGE_SIZE;
+ virt_offset += PAGE_SIZE;
+ }
+}
+
+static void idmap_set_range_shared(unsigned long virt_offset, size_t size)
+{
+ return __idmap_set_range_prot(virt_offset, size, __pgprot(PTE_WBWA | PTE_USER | PTE_NS_SHARED));
+}
+
+static void idmap_set_range_protected(unsigned long virt_offset, size_t size)
+{
+ __idmap_set_range_prot(virt_offset, size, __pgprot(PTE_WBWA | PTE_USER));
+}
+
void mmu_set_range_ptes(pgd_t *pgtable, uintptr_t virt_offset,
phys_addr_t phys_start, phys_addr_t phys_end,
pgprot_t prot)
@@ -190,6 +234,7 @@ void *setup_mmu(phys_addr_t phys_end, void *unused)
}

mmu_enable(mmu_idmap);
+ idmap_end = phys_end;
return mmu_idmap;
}

@@ -278,3 +323,19 @@ void mmu_clear_user(pgd_t *pgtable, unsigned long vaddr)
flush_tlb_page(vaddr);
}
}
+
+void set_memory_encrypted(unsigned long va, size_t size)
+{
+ if (is_realm()) {
+ arm_set_memory_protected(__virt_to_phys(va), size);
+ idmap_set_range_protected(va, size);
+ }
+}
+
+void set_memory_decrypted(unsigned long va, size_t size)
+{
+ if (is_realm()) {
+ arm_set_memory_shared(__virt_to_phys(va), size);
+ idmap_set_range_shared(va, size);
+ }
+}
diff --git a/lib/arm64/asm/io.h b/lib/arm64/asm/io.h
index be19f471..3f71254d 100644
--- a/lib/arm64/asm/io.h
+++ b/lib/arm64/asm/io.h
@@ -89,6 +89,12 @@ static inline void *phys_to_virt(phys_addr_t x)
return (void *)__phys_to_virt(x);
}

+extern void set_memory_decrypted(unsigned long va, size_t size);
+#define set_memory_decrypted set_memory_decrypted
+
+extern void set_memory_encrypted(unsigned long va, size_t size);
+#define set_memory_encrypted set_memory_encrypted
+
#include <asm-generic/io.h>

#endif /* _ASMARM64_IO_H_ */
--
2.17.1

2023-01-27 11:46:40

[permalink] [raw]

Subject: [RFC kvm-unit-tests 17/27] lib/alloc_page: Add shared page allocation support

Add support for allocating "pages" that can be shared with the host.
Or in other words, decrypted pages. This is achieved by adding hooks for
setting a memory region as "encrypted" or "decrypted", which can be overridden
by the architecture specific backends.

Also add a new flag - FLAG_SHARED - for allocating shared pages.

The page allocation/free routines get a "_shared_" variant too.
These will be later used for Realm support and tests.

Signed-off-by: Joey Gouly <[email protected]>
---
lib/alloc_page.c | 34 +++++++++++++++++++++++++++++++---
lib/alloc_page.h | 24 ++++++++++++++++++++++++
2 files changed, 55 insertions(+), 3 deletions(-)

diff --git a/lib/alloc_page.c b/lib/alloc_page.c
index 84f01e11..8b811b15 100644
--- a/lib/alloc_page.c
+++ b/lib/alloc_page.c
@@ -53,6 +53,20 @@ static struct mem_area areas[MAX_AREAS];
/* Mask of initialized areas */
static unsigned int areas_mask;
/* Protects areas and areas mask */
+
+#ifndef set_memory_encrypted
+static inline void set_memory_encrypted(unsigned long mem, unsigned long size)
+{
+}
+#endif
+
+#ifndef set_memory_decrypted
+static inline void set_memory_decrypted(unsigned long mem, unsigned long size)
+{
+}
+#endif
+
+
static struct spinlock lock;

bool page_alloc_initialized(void)
@@ -263,7 +277,7 @@ static bool coalesce(struct mem_area *a, u8 order, pfn_t pfn, pfn_t pfn2)
* - no pages in the memory block were already free
* - no pages in the memory block are special
*/
-static void _free_pages(void *mem)
+static void _free_pages(void *mem, u32 flags)
{
pfn_t pfn2, pfn = virt_to_pfn(mem);
struct mem_area *a = NULL;
@@ -281,6 +295,9 @@ static void _free_pages(void *mem)
p = pfn - a->base;
order = a->page_states[p] & ORDER_MASK;

+ if (flags & FLAG_SHARED)
+ set_memory_encrypted((unsigned long)mem, BIT(order) * PAGE_SIZE);
+
/* ensure that the first page is allocated and not special */
assert(IS_ALLOCATED(a->page_states[p]));
/* ensure that the order has a sane value */
@@ -320,7 +337,14 @@ static void _free_pages(void *mem)
void free_pages(void *mem)
{
spin_lock(&lock);
- _free_pages(mem);
+ _free_pages(mem, 0);
+ spin_unlock(&lock);
+}
+
+void free_pages_shared(void *mem)
+{
+ spin_lock(&lock);
+ _free_pages(mem, FLAG_SHARED);
spin_unlock(&lock);
}

@@ -353,7 +377,7 @@ static void _unreserve_one_page(pfn_t pfn)
i = pfn - a->base;
assert(a->page_states[i] == STATUS_SPECIAL);
a->page_states[i] = STATUS_ALLOCATED;
- _free_pages(pfn_to_virt(pfn));
+ _free_pages(pfn_to_virt(pfn), 0);
}

int reserve_pages(phys_addr_t addr, size_t n)
@@ -401,6 +425,10 @@ static void *page_memalign_order_flags(u8 al, u8 ord, u32 flags)
if (area & BIT(i))
res = page_memalign_order(areas + i, al, ord, fresh);
spin_unlock(&lock);
+
+ if (res && (flags & FLAG_SHARED))
+ set_memory_decrypted((unsigned long)res, BIT(ord) * PAGE_SIZE);
+
if (res && !(flags & FLAG_DONTZERO))
memset(res, 0, BIT(ord) * PAGE_SIZE);
return res;
diff --git a/lib/alloc_page.h b/lib/alloc_page.h
index 060e0418..847a7fda 100644
--- a/lib/alloc_page.h
+++ b/lib/alloc_page.h
@@ -21,6 +21,7 @@

#define FLAG_DONTZERO 0x10000
#define FLAG_FRESH 0x20000
+#define FLAG_SHARED 0x40000

/* Returns true if the page allocator has been initialized */
bool page_alloc_initialized(void);
@@ -121,4 +122,27 @@ int reserve_pages(phys_addr_t addr, size_t npages);
*/
void unreserve_pages(phys_addr_t addr, size_t npages);

+/* Shared page operations */
+static inline void *alloc_pages_shared(unsigned long order)
+{
+ return alloc_pages_flags(order, FLAG_SHARED);
+}
+
+static inline void *alloc_page_shared(void)
+{
+ return alloc_pages_shared(0);
+}
+
+void free_pages_shared(void *mem);
+
+static inline void free_page_shared(void *page)
+{
+ free_pages_shared(page);
+}
+
+static inline void free_pages_shared_by_order(void *mem, unsigned long order)
+{
+ free_pages_shared(mem);
+}
+
#endif
--
2.17.1

2023-01-27 11:47:05

[permalink] [raw]

Subject: [RFC kvm-unit-tests 25/27] arm: realm: Add Realm attestation tests

From: Mate Toth-Pal <[email protected]>

Add tests for Attestation and measurement related RSI calls.

Signed-off-by: Mate Toth-Pal <[email protected]>
Co-developed-by: Suzuki K Poulose <[email protected]>
[ Rewrote the test cases, keeping the core testing data/logic ]
Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
arm/Makefile.arm64 | 1 +
arm/realm-attest.c | 1125 ++++++++++++++++++++++++++++++++++++++++++++
arm/unittests.cfg | 50 ++
lib/libcflat.h | 1 +
4 files changed, 1177 insertions(+)
create mode 100644 arm/realm-attest.c

diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index f57d0a95..0a0c4f2c 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -41,6 +41,7 @@ tests += $(TEST_DIR)/micro-bench.flat
tests += $(TEST_DIR)/cache.flat
tests += $(TEST_DIR)/debug.flat
tests += $(TEST_DIR)/realm-rsi.flat
+tests += $(TEST_DIR)/realm-attest.flat
tests += $(TEST_DIR)/realm-fpu.flat
tests += $(TEST_DIR)/realm-sea.flat

diff --git a/arm/realm-attest.c b/arm/realm-attest.c
new file mode 100644
index 00000000..6c357fb5
--- /dev/null
+++ b/arm/realm-attest.c
@@ -0,0 +1,1125 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+#include <libcflat.h>
+
+#include <attest_defines.h>
+#include <alloc.h>
+#include <stdlib.h>
+#include <token_dumper.h>
+#include <token_verifier.h>
+
+#include <asm/io.h>
+#include <asm/page.h>
+#include <asm/rsi.h>
+#include <asm/setup.h>
+#include <asm/smp.h>
+
+
+#define SHA256_SIZE 32
+
+struct challenge {
+ unsigned long words[8];
+};
+
+struct measurement {
+ unsigned long words[8];
+};
+
+static char __attribute__((aligned(SZ_2M))) __attribute__((section(".data")))
+ block_buf_data[SZ_2M * 2];
+
+static char __attribute__((aligned(SZ_2M))) __attribute__((section(".bss")))
+ block_buf_bss[SZ_2M];
+
+static char __attribute__((aligned(SZ_4K))) __attribute__((section(".data")))
+ page_buf_data[SZ_4K];
+
+static char __attribute__((aligned(SZ_4K))) __attribute__((section(".bss")))
+ page_buf_bss[SZ_4K];
+
+/* Page aligned offset within the block mapped buffer */
+#define BLOCK_BUF_OFFSET (SZ_8K)
+
+static inline void debug_print_raw_token(void *buf, size_t size)
+{
+#ifdef PRINT_RAW_TOKEN
+ print_raw_token(buf, size);
+#endif
+}
+
+static inline void debug_print_token(struct attestation_claims *claim)
+{
+#ifdef PRINT_TOKEN
+ print_token(claim);
+#endif
+}
+
+static bool claims_verify_token(char *token, size_t token_size,
+ struct attestation_claims *claims,
+ bool report_success)
+{
+ int verify_rc = verify_token(token, token_size, claims);
+ int cpu = smp_processor_id();
+
+ if (verify_rc == TOKEN_VERIFICATION_ERR_SUCCESS) {
+ if (report_success)
+ report(true, "CPU%d: Verfication of token passed", cpu);
+ return true;
+ }
+
+ report(false,
+ "CPU%d: Verification of token failed with error code %d",
+ cpu, verify_rc);
+
+ return false;
+}
+
+static inline void attest_token_init(phys_addr_t addr,
+ struct challenge *ch,
+ struct smccc_result *res)
+{
+ rsi_attest_token_init(addr, &ch->words[0], res);
+}
+
+
+static inline void attest_token_continue(phys_addr_t addr,
+ struct smccc_result *res)
+{
+ rsi_attest_token_continue(addr, res);
+}
+
+static inline void attest_token_complete(phys_addr_t addr,
+ struct smccc_result *res)
+{
+ do {
+ attest_token_continue(addr, res);
+ } while (res->r0 == RSI_INCOMPLETE);
+}
+
+static void get_attest_token(phys_addr_t ipa,
+ struct challenge *ch,
+ struct smccc_result *res)
+{
+ attest_token_init(ipa, ch, res);
+
+ if (res->r0)
+ return;
+ attest_token_complete(ipa, res);
+}
+
+/*
+ * __get_attest_token_claims: Get attestation token and verify the claims.
+ * If @claims is not NULL, token is parsed and the @claims is populated.
+ * All failures reported. Success is only reported if the @report_success is
+ * true.
+ * Returns whether the calls and verification succeeds
+ */
+static bool __get_attest_token_claims(void *buf, struct challenge *ch,
+ struct attestation_claims *claims,
+ size_t *token_size, bool report_success)
+{
+ struct smccc_result result;
+ struct attestation_claims local_claims;
+ struct attestation_claims *claimsp;
+ bool rc = false;
+
+ /* Use the local_claims if claims is not supplied */
+ claimsp = claims ? : &local_claims;
+
+ get_attest_token(virt_to_phys(buf), ch, &result);
+ if (result.r0) {
+ report(false, "Get attestation token with : %ld", result.r0);
+ return rc;
+ }
+
+ if (report_success)
+ report(true, "Get attestation token");
+
+ /* Update token_size if necessary */
+ if (token_size)
+ *token_size = result.r1;
+
+ return claims_verify_token(buf, result.r1, claimsp, report_success);
+}
+
+static bool get_attest_token_claims(void *buf, struct challenge *ch,
+ struct attestation_claims *claims,
+ size_t *token_size)
+{
+ return __get_attest_token_claims(buf, ch, claims, token_size, false);
+}
+
+static void get_verify_attest_token(void *buf, struct challenge *ch,
+ const char *desc)
+{
+ report_prefix_push(desc);
+ __get_attest_token_claims(buf, ch, NULL, NULL, true);
+ report_prefix_pop();
+}
+
+static void get_verify_attest_token_verbose(void *buf,
+ struct challenge *ch,
+ const char *desc)
+{
+ size_t token_size;
+ struct attestation_claims claims;
+
+ report_prefix_push(desc);
+ if (__get_attest_token_claims(buf, ch, &claims, &token_size, true)) {
+ debug_print_raw_token(buf, token_size);
+ debug_print_token(&claims);
+ }
+ report_prefix_pop();
+}
+
+static void test_get_attest_token(void)
+{
+ char stack_buf[SZ_4K]__attribute__((aligned(SZ_4K)));
+ char *heap_buf;
+ struct challenge ch;
+
+ memset(&ch, 0xAB, sizeof(ch));
+
+ /* Heap buffer */
+ heap_buf = memalign(SZ_4K, SZ_4K);
+ if (heap_buf) {
+ get_verify_attest_token(heap_buf, &ch, "heap buffer");
+ free(heap_buf);
+ } else {
+ report_skip("heap buffer: Failed to allocate");
+ }
+
+ /* Stack buffer */
+ get_verify_attest_token(stack_buf, &ch, "stack buffer");
+ /* Page aligned buffer .data segment */
+ get_verify_attest_token(page_buf_data, &ch, ".data segment buffer");
+ /* Page aligned buffer .bss segment */
+ get_verify_attest_token(page_buf_bss, &ch, ".bss segment buffer");
+ /* Block mapped buffer in .data segment */
+ get_verify_attest_token(&block_buf_data[BLOCK_BUF_OFFSET], &ch,
+ "block mapped .data segment buffer");
+ /* Block mapped buffer in .bss segment */
+ get_verify_attest_token_verbose(&block_buf_bss[BLOCK_BUF_OFFSET],
+ &ch, "block mapped .bss segment buffer");
+}
+
+static void get_attest_token_check_fail(phys_addr_t ipa,
+ struct challenge *ch,
+ return_code_t exp,
+ const char *buf_desc)
+{
+ struct smccc_result result;
+ return_code_t rc;
+
+ report_prefix_push(buf_desc);
+ get_attest_token(ipa, ch, &result);
+ rc = unpack_return_code(result.r0);
+ if (rc.status != exp.status) {
+ report(false, "Get attestation token "
+ "got (%d) expected (%d)",
+ rc.status, exp.status);
+ } else {
+ report(true, "Get attestation token fails as expected");
+ }
+ report_prefix_pop();
+}
+
+static void test_get_attest_token_bad_input(void)
+{
+ struct challenge ch;
+ return_code_t exp;
+
+ memset(page_buf_data, 0, sizeof(page_buf_data));
+ memset(&ch, 0xAB, sizeof(ch));
+ exp = make_return_code(RSI_ERROR_INPUT, 0);
+ get_attest_token_check_fail(virt_to_phys(page_buf_data + 0x100),
+ &ch, exp, "unaligned buffer");
+ get_attest_token_check_fail(__phys_end + SZ_512M, &ch, exp,
+ "buffer outside PAR");
+}
+
+static void test_get_attest_token_abi_misuse(void)
+{
+ struct smccc_result result;
+ struct challenge ch;
+ phys_addr_t ipa = virt_to_phys(page_buf_data);
+ return_code_t rc;
+
+ memset(&ch, 0xAB, sizeof(ch));
+
+ /*
+ * Testcase 1 - Miss call to RSI_ATTEST_TOKEN_INIT
+ *
+ * step1. Execute a successful test to reset the state machine.
+ */
+ report_prefix_push("miss token init");
+ get_attest_token(ipa, &ch, &result);
+ if (result.r0) {
+ report(false, "Get attestation failed %ld", result.r0);
+ report_prefix_pop(); /* miss token init */
+ return;
+ }
+ /*
+ * step2. Execute RSI_ATTEST_TOKEN_CONTINUE without an RSI_ATTEST_TOKEN_INIT.
+ * Expect an error == RSI_ERROR_STATE
+ */
+ attest_token_continue(ipa, &result);
+ rc = unpack_return_code(result.r0);
+ if (rc.status != RSI_ERROR_STATE) {
+ report(false, "Unexpected result (%d, %d) vs (%d) expected",
+ rc.status, rc.index, RSI_ERROR_STATE);
+ report_prefix_pop(); /* miss token init */
+ return;
+ }
+
+ report(true, "Fails as expected");
+ report_prefix_pop(); /* miss token init */
+
+ /*
+ * Test case 2 - Calling with inconsistend input.
+ * step1. Issue RSI_ATTEST_TOKEN_INIT
+ * step2. Modify the challenge and issue RSI_ATTEST_TOKEN_CONTINUE.
+ * Test : Expect error == (RSI_ERROR_INPUT, 0)
+ */
+ report_prefix_push("inconsistent input");
+ attest_token_init(ipa, &ch, &result);
+ rc = unpack_return_code(result.r0);
+ if (result.r0) {
+ report(false, "RSI_ATTEST_TOKEN_INIT failed unexpectedly (%d, %d)",
+ rc.status, rc.index);
+ report_prefix_pop(); /* inconsistent input */
+ return;
+ }
+
+ /*
+ * Corrupt the IPA address input to ATTEST_TOKEN_CONTINUE,
+ */
+ attest_token_continue(ipa ^ 0x1UL, &result);
+ rc = unpack_return_code(result.r0);
+ if (rc.status != RSI_ERROR_INPUT) {
+ report(false, "Attest token continue unexpected results"
+ " (%d) vs expected (%d)",
+ rc.status, RSI_ERROR_INPUT);
+ }
+
+ report_prefix_pop(); /* inconsistent input */
+
+ /*
+ * Test case 3
+ * step1. Complete the token attestation from with proper values.
+ * Failures in the Test case 2 should not affect the completion.
+ */
+ report_prefix_push("valid input after inconsistent input");
+ attest_token_complete(ipa, &result);
+ rc = unpack_return_code(result.r0);
+ if (result.r0) {
+ report(false, "Attest token continue failed with (%d, %d)",
+ rc.status, rc.index);
+ return;
+ } else {
+ report(true, "Attest token continue complete");
+ }
+ report_prefix_pop(); /* Valid input after inconsistent input */
+}
+
+static void test_get_attest_token_abi_abort_req(void)
+{
+ int i;
+ char *p;
+ size_t size;
+ struct attestation_claims claims;
+ struct smccc_result result;
+ struct challenge ch;
+ char stack_buf[SZ_4K] __attribute__((aligned(SZ_4K))) = { 0 };
+ phys_addr_t addr = virt_to_phys(stack_buf);
+
+ /* Set the initial challenge, which will be aborted */
+ memset(&ch, 0xAB, sizeof(ch));
+ attest_token_init(addr, &ch, &result);
+ if (result.r0) {
+ report(false, "Attest token init failed %ld", result.r0);
+ return;
+ }
+
+ /* Execute a few cycles, but not let it complete */
+ for (i = 0; i < 3; i++) {
+ attest_token_continue(addr, &result);
+ if (result.r0 != RSI_INCOMPLETE) {
+ if (result.r0)
+ report(false, "Attest token continue : unexpected "
+ "failure %ld", result.r0);
+ else
+ report_skip("Attest token finished at iteration %d",
+ i + 1);
+ return;
+ }
+ }
+
+ /* Issue a fresh Attest Token request with updated challenge */
+ memset(&ch, 0xEE, sizeof(ch));
+ get_attest_token(addr, &ch, &result);
+ if (result.r0) {
+ report(false, "Attest Token failed %ld", result.r0);
+ return;
+ }
+ claims_verify_token(stack_buf, result.r1, &claims, false);
+
+ /*
+ * TODO: Index of claim in the array depends on the init sequence
+ * in token_verifier.c: init_claim()
+ */
+ p = (char*)claims.realm_token_claims[0].buffer_data.ptr;
+ size = claims.realm_token_claims[0].buffer_data.len;
+
+ /* Verify that token contains the updated challenge. */
+ if (size != sizeof(ch)) {
+ report(false, "Attestation token: abort request: "
+ "claim size mismatch : %ld", result.r0);
+ return;
+ }
+ if (memcmp(p, &ch, size)) {
+ report(false, "Attestation token: abort request: "
+ "claim value mismatch: %ld", result.r0);
+ return;
+ }
+ report(true, "Aborting ongoing request");
+}
+
+static void run_rsi_attest_tests(void)
+{
+ report_prefix_push("attest");
+
+ test_get_attest_token();
+
+ report_prefix_push("bad input");
+ test_get_attest_token_bad_input();
+ report_prefix_pop();
+
+ report_prefix_push("ABI misuse");
+ test_get_attest_token_abi_misuse();
+ report_prefix_pop();
+
+ report_prefix_push("ABI Abort");
+ test_get_attest_token_abi_abort_req();
+ report_prefix_pop();
+
+ report_prefix_pop(); /* attest */
+}
+
+static void run_get_token_times(void *data)
+{
+ char buf[SZ_4K] __attribute__((aligned(SZ_4K)));
+ struct challenge ch;
+ struct attestation_claims claims;
+ unsigned long runs = ((size_t)data);
+ int i, j;
+ int cpu = smp_processor_id();
+
+ report_info("CPU%d: Running get token test %ld times", cpu, runs);
+ for (i = 0; i < runs; i++) {
+ uint8_t pattern = (cpu << 4) | (i & 0xf);
+ size_t token_size;
+ struct claim_t *claim;
+
+ memset(buf, 0, sizeof(buf));
+ memset(&ch, pattern, sizeof(ch));
+
+ if (!get_attest_token_claims(buf, &ch, &claims, &token_size))
+ return;
+ claim = claims.realm_token_claims;
+ if (claim->key != CCA_REALM_CHALLENGE ||
+ claim->buffer_data.len != sizeof(ch)) {
+ report(false, "Invalid challenge size in parsed token:"
+ " %zu (expected %zu)",
+ claim->buffer_data.len, sizeof(ch));
+ return;
+ }
+
+ for (j = 0; j < sizeof(ch); j++) {
+ uint8_t byte = ((uint8_t *)claim->buffer_data.ptr)[j];
+ if (byte != pattern) {
+ report(false, "Invalid byte in challenge[%d]: "
+ " %02x (expected %02x)",
+ j, byte, pattern);
+ return;
+ }
+ }
+ }
+ report(true, "CPU%d: Completed runs", cpu);
+}
+
+static void run_rsi_attest_smp_test(void)
+{
+ unsigned long runs = 100;
+
+ report_prefix_push("attest_smp");
+ on_cpus(run_get_token_times, (void *)runs);
+ report_prefix_pop();
+}
+
+/*
+ * There are 7 slots for measurements. The first is reserved for initial
+ * content measurement. The rest are meant to store runtime measurements.
+ * Runtime measurements are extended (concatenated and hashed). Reading
+ * them back separately is unsupported. They can be queried in an
+ * attestation token.
+ *
+ * Measurement size is 64bytes maximum to accommodate a SHA512 hash.
+ */
+
+static void measurement_extend(int idx, struct measurement *m, size_t size,
+ struct smccc_result *res)
+{
+ rsi_extend_measurement(idx, size, &m->words[0], res);
+}
+
+static void test_extend_measurement(void)
+{
+ struct smccc_result result;
+ struct measurement m;
+ return_code_t rc;
+ int idx;
+
+ memset(&m, 0xEE, sizeof(m));
+ /*
+ * Store Runtime measurements for all possible slots.
+ */
+ for (idx = 1; idx <= REM_COUNT; idx ++) {
+ measurement_extend(idx, &m, sizeof(m.words), &result);
+ rc = unpack_return_code(result.r0);
+ report(!rc.status, "Extend measurement idx: %d (%d, %d)",
+ idx, rc.status, rc.index);
+ }
+}
+
+static void test_extend_measurement_bad_index(struct measurement *m)
+{
+ struct smccc_result result;
+ return_code_t rc;
+ int indices[] = { 0, REM_COUNT + 1 };
+ const char *idx_descs[] = { "reserved", "out-of-bounds" };
+ int i;
+
+ report_prefix_push("index");
+ for (i = 0; i < ARRAY_SIZE(indices); i++) {
+ report_prefix_push(idx_descs[i]);
+ measurement_extend(indices[i], m, sizeof(m->words), &result);
+ rc = unpack_return_code(result.r0);
+
+ if (rc.status != RSI_ERROR_INPUT)
+ report(false, "Extend measurement index: "
+ "actual (%d) vs expected (%d)",
+ rc.status, RSI_ERROR_INPUT);
+ else
+ report(true, "Extend measurement index fails as expected");
+ report_prefix_pop(); /* idx_descs[i] */
+ }
+ report_prefix_pop(); /* index */
+}
+
+static void test_extend_measurement_bad_size(struct measurement *m)
+{
+ struct smccc_result result;
+ return_code_t rc;
+
+ report_prefix_push("size");
+ rsi_extend_measurement(1, 65, &m->words[0], &result);
+ rc = unpack_return_code(result.r0);
+ if (rc.status != RSI_ERROR_INPUT)
+ report(false, "Measurement extend "
+ "actual (%d) vs expected (%d)",
+ rc.status, RSI_ERROR_INPUT);
+ else
+ report(true, "Extend measurement fails as expected");
+ report_prefix_pop(); /* size */
+}
+
+static void test_extend_measurement_bad_input(void)
+{
+ struct measurement m;
+
+ report_prefix_push("bad input");
+ memset(&m, 0xEE, sizeof(m));
+ test_extend_measurement_bad_index(&m);
+ test_extend_measurement_bad_size(&m);
+ report_prefix_pop(); /* bad input */
+}
+
+static void run_rsi_extend_tests(void)
+{
+ report_prefix_push("extend");
+ test_extend_measurement();
+ test_extend_measurement_bad_input();
+ report_prefix_pop(); /* extend */
+}
+
+/*
+ * cpu_extend_run - Parameters for the extend measurement SMP run.
+ * @idx - Pointer to the index
+ * @size - Size of the measurement data
+ * @m - Measurement data.
+ */
+struct cpu_extend_run {
+ int *idx;
+ struct measurement *m;
+ size_t size;
+ unsigned long rc;
+};
+
+/*
+ * We get an array of the parameters for the extend measurement.
+ * The cpu number is the index to the array. At the moment we
+ * only support 2 cpus.
+ */
+static void cpu_run_extend_measurement(void *data)
+{
+ struct smccc_result result;
+ struct cpu_extend_run *run;
+ int me = smp_processor_id();
+
+ assert(me >= 0);
+
+ /* Tests for only 2 CPUs */
+ if (me > 1)
+ return;
+ run = (struct cpu_extend_run *)data + me;
+ rsi_extend_measurement(*run->idx, run->size, &run->m->words[0], &result);
+ run->rc = result.r0;
+ if (result.r0 != 0)
+ report(false, "CPU%d: Extend measurement failed for slot %d",
+ me, *run->idx);
+}
+
+static bool claims_uses_sha256_algo(struct attestation_claims *claims)
+{
+ struct claim_t *claim = claims->realm_token_claims + 2; /* CCA_REALM_HASH_ALGO_ID */
+
+ /* claim->buffer_data.ptr: Not NULL terminated, so using memcmp */
+ return !memcmp(claim->buffer_data.ptr, "sha-256", strlen("sha-256"));
+}
+
+static void test_rsi_extend_smp(void)
+{
+ int slot, m_idx;
+ struct measurement m[2];
+ struct challenge ch;
+ struct attestation_claims claims;
+ size_t token_size;
+
+ /*
+ * Measurements to extend with
+ *
+ * Run CPU0 data CPU1 data
+ * 1: [31 - 0] [55 - 24]
+ * 2: [39 - 8] [63 - 32]
+ * 3: [47 - 16] [71 - 40]
+ */
+ char measure_bytes[] = {
+ 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11,
+ 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11,
+ 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11,
+ 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11,
+ 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+ 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F,
+ 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
+ 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F,
+ 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27
+ };
+
+ /*
+ * The expected measurement values. Each element in the array contains
+ * a possible extended measurement value. (Multiple values are possible
+ * as the extend function might be called in any order by the cores.)
+ * The array contains results for all the possible orders. The number of
+ * possibilities can be calculated as here:
+ * https://math.stackexchange.com/q/1065374
+ */
+ struct extend_smp_expected {
+ const char *sequence;
+ char measurement[SHA256_SIZE];
+ } expected[] = {
+ {
+ "[ cpu0#0 cpu0#1 cpu0#2 cpu1#0 cpu1#1 cpu1#2 ]",
+ {
+ 0xB1, 0xBE, 0x04, 0x25, 0xBB, 0xBC, 0x04, 0x9F,
+ 0x98, 0x4F, 0xFB, 0xDE, 0xAA, 0x00, 0xC9, 0xBC,
+ 0x41, 0x43, 0xDB, 0x16, 0xBB, 0x2A, 0x5F, 0x4B,
+ 0x8B, 0x36, 0xAE, 0x3F, 0xFE, 0x24, 0x23, 0xA4
+ },
+ },
+ {
+ "[ cpu0#0 cpu0#1 cpu1#0 cpu0#2 cpu1#1 cpu1#2 ]",
+ {
+ 0x99, 0x00, 0x5E, 0xB7, 0xF8, 0x84, 0xA3, 0x99,
+ 0x7E, 0x12, 0xDE, 0xD1, 0x5B, 0xA7, 0x07, 0xF4,
+ 0x24, 0x3E, 0x77, 0xED, 0x60, 0xC0, 0xBD, 0x43,
+ 0x3B, 0x60, 0x7E, 0x38, 0xDD, 0x58, 0xC7, 0x46
+ },
+ },
+ {
+ "[ cpu0#0 cpu0#1 cpu1#0 cpu1#1 cpu0#2 cpu1#2 ]",
+ {
+ 0x0B, 0x5E, 0x31, 0x69, 0xAC, 0xAF, 0xA0, 0x8B,
+ 0x4F, 0x90, 0xD1, 0x86, 0xCC, 0x8E, 0x11, 0x42,
+ 0x0B, 0x74, 0x49, 0x6C, 0xA1, 0x27, 0x1B, 0x7C,
+ 0x52, 0x77, 0x7F, 0x2F, 0x53, 0x2F, 0x9A, 0xC1
+ },
+ },
+ {
+ "[ cpu0#0 cpu0#1 cpu1#0 cpu1#1 cpu1#2 cpu0#2 ]",
+ {
+ 0x99, 0xDE, 0xF8, 0x02, 0x27, 0xE9, 0x6F, 0x6F,
+ 0xA6, 0x55, 0xFC, 0x56, 0xCC, 0x7A, 0xFC, 0xEF,
+ 0x2F, 0x0C, 0x45, 0x3E, 0x01, 0xE0, 0x4B, 0xA1,
+ 0x60, 0x96, 0xEE, 0xB1, 0x4A, 0x25, 0x86, 0x89},
+ },
+ {
+ "[ cpu0#0 cpu1#0 cpu0#1 cpu0#2 cpu1#1 cpu1#2 ]",
+ { 0x88, 0x40, 0x05, 0xF5, 0xA6, 0x95, 0xC1, 0xC7,
+ 0xD3, 0x69, 0x16, 0x82, 0x0D, 0x79, 0xC1, 0x5B,
+ 0x4A, 0x48, 0xCA, 0x7F, 0xA5, 0xF3, 0x77, 0x37,
+ 0xBE, 0x0D, 0xAC, 0x2E, 0x42, 0x3E, 0x03, 0x37
+ },
+ },
+ {
+ "[ cpu0#0 cpu1#0 cpu0#1 cpu1#1 cpu0#2 cpu1#2 ]",
+ {
+ 0x68, 0x32, 0xC6, 0xAF, 0x8C, 0x86, 0x77, 0x09,
+ 0x4A, 0xB9, 0xA1, 0x9E, 0xBB, 0x2B, 0x42, 0x35,
+ 0xF8, 0xDE, 0x9A, 0x98, 0x37, 0x7B, 0x3E, 0x82,
+ 0x59, 0x0B, 0xC4, 0xAD, 0x1D, 0x01, 0x28, 0xCA
+ },
+ },
+ {
+ "[ cpu0#0 cpu1#0 cpu0#1 cpu1#1 cpu1#2 cpu0#2 ]",
+ {
+ 0xF5, 0x96, 0x77, 0x68, 0xD9, 0x6A, 0xA2, 0xFC,
+ 0x08, 0x8C, 0xF5, 0xA9, 0x6B, 0xE7, 0x1E, 0x20,
+ 0x35, 0xC1, 0x92, 0xCE, 0xBC, 0x3A, 0x75, 0xEA,
+ 0xB4, 0xEB, 0x17, 0xE5, 0x77, 0x50, 0x85, 0x40
+ },
+
+ },
+ {
+ "[ cpu0#0 cpu1#0 cpu1#1 cpu0#1 cpu0#2 cpu1#2 ]",
+ {
+ 0x4E, 0xA2, 0xD2, 0x79, 0x55, 0x75, 0xCB, 0x86,
+ 0x87, 0x34, 0x35, 0xE7, 0x75, 0xDF, 0xD5, 0x59,
+ 0x58, 0xDE, 0x74, 0x35, 0x68, 0x2B, 0xDC, 0xC8,
+ 0x85, 0x72, 0x97, 0xBE, 0x58, 0xB1, 0x1E, 0xA7
+ },
+
+ },
+ {
+ "[ cpu0#0 cpu1#0 cpu1#1 cpu0#1 cpu1#2 cpu0#2 ]",
+ {
+ 0xD1, 0xC2, 0xC8, 0x08, 0x00, 0x64, 0xB8, 0x1F,
+ 0xA0, 0xA5, 0x32, 0x20, 0xAA, 0x08, 0xC0, 0x48,
+ 0xDB, 0xB1, 0xED, 0xE7, 0xAF, 0x18, 0x2F, 0x7F,
+ 0x3C, 0xB8, 0x58, 0x83, 0xEC, 0xF9, 0x38, 0xFD
+ },
+
+ },
+ {
+ "[ cpu0#0 cpu1#0 cpu1#1 cpu1#2 cpu0#1 cpu0#2 ]",
+ {
+ 0xD1, 0xB8, 0x31, 0x98, 0x8E, 0xF2, 0xE7, 0xF5,
+ 0xBB, 0xD1, 0xE1, 0xC7, 0x3E, 0xB7, 0xA9, 0x18,
+ 0x3B, 0xCC, 0x58, 0x98, 0xED, 0x22, 0x1E, 0xE2,
+ 0x04, 0x76, 0xA1, 0xB9, 0x92, 0x54, 0xB5, 0x5B
+ },
+
+ },
+ {
+ "[ cpu1#0 cpu0#0 cpu0#1 cpu0#2 cpu1#1 cpu1#2 ]",
+ {
+ 0xAB, 0x50, 0x2A, 0x68, 0x28, 0x35, 0x16, 0xA9,
+ 0xDE, 0x26, 0x77, 0xAA, 0x99, 0x29, 0x0E, 0x9C,
+ 0x67, 0x41, 0x64, 0x28, 0x6E, 0xFF, 0x54, 0x33,
+ 0xE5, 0x29, 0xC4, 0xA5, 0x98, 0x40, 0x7E, 0xC9
+ },
+
+ },
+ {
+ "[ cpu1#0 cpu0#0 cpu0#1 cpu1#1 cpu0#2 cpu1#2 ]",
+ {
+ 0xA3, 0x4D, 0xB0, 0x28, 0xAB, 0x01, 0x56, 0xBB,
+ 0x7D, 0xE5, 0x0E, 0x86, 0x26, 0xBB, 0xBB, 0xDE,
+ 0x58, 0x91, 0x88, 0xBB, 0x9F, 0x6A, 0x58, 0x78,
+ 0x30, 0x2C, 0x22, 0x2E, 0x85, 0x7F, 0x87, 0xF6
+ },
+
+ },
+ {
+ "[ cpu1#0 cpu0#0 cpu0#1 cpu1#1 cpu1#2 cpu0#2 ]",
+ {
+ 0x1A, 0x2E, 0xD2, 0xC2, 0x0C, 0xBD, 0x30, 0xDA,
+ 0x4F, 0x37, 0x6B, 0x90, 0xE3, 0x67, 0xFE, 0x61,
+ 0x4F, 0x30, 0xBB, 0x29, 0xBC, 0xAA, 0x6E, 0xC5,
+ 0x60, 0x6E, 0x13, 0x6B, 0x33, 0x3D, 0xC0, 0x11
+ },
+
+ },
+ {
+ "[ cpu1#0 cpu0#0 cpu1#1 cpu0#1 cpu0#2 cpu1#2 ]",
+ {
+ 0x8F, 0xEA, 0xD1, 0x80, 0xE0, 0xBE, 0x27, 0xF7,
+ 0x8D, 0x19, 0xBF, 0x65, 0xBE, 0x92, 0x83, 0x7C,
+ 0x61, 0x8F, 0xC5, 0x8D, 0x0F, 0xAD, 0x89, 0x1E,
+ 0xAE, 0x0A, 0x75, 0xAC, 0x3E, 0x5F, 0xD5, 0x31
+ },
+
+ },
+ {
+ "[ cpu1#0 cpu0#0 cpu1#1 cpu0#1 cpu1#2 cpu0#2 ]",
+ {
+ 0x0F, 0x7B, 0xEE, 0xA5, 0x9A, 0xCD, 0xED, 0x8D,
+ 0x5A, 0x52, 0xFF, 0xD6, 0x30, 0xF4, 0xD9, 0xE9,
+ 0xF4, 0xC1, 0x1A, 0x0C, 0x86, 0x2B, 0x96, 0x2C,
+ 0x0E, 0x2D, 0x1A, 0x2A, 0xFE, 0xE6, 0x7C, 0xAD
+ },
+
+ },
+ {
+ "[ cpu1#0 cpu0#0 cpu1#1 cpu1#2 cpu0#1 cpu0#2 ]",
+ {
+ 0x4A, 0xBA, 0xFF, 0x0B, 0x0B, 0x06, 0xD1, 0xCE,
+ 0x95, 0x91, 0x70, 0x68, 0x20, 0xD6, 0xF2, 0x23,
+ 0xC5, 0x6A, 0x63, 0x2B, 0xCA, 0xDF, 0x37, 0xB5,
+ 0x0B, 0xDC, 0x64, 0x6A, 0xA3, 0xC9, 0x8F, 0x1E
+ },
+
+ },
+ {
+ "[ cpu1#0 cpu1#1 cpu0#0 cpu0#1 cpu0#2 cpu1#2 ]",
+ {
+ 0x3D, 0xB1, 0xE1, 0xBD, 0x85, 0x2C, 0xA0, 0x04,
+ 0xE6, 0x43, 0xE8, 0x82, 0xC3, 0x77, 0xF3, 0xCE,
+ 0x4D, 0x62, 0x2C, 0xF4, 0x65, 0xF6, 0x29, 0x5F,
+ 0x17, 0xDA, 0xD5, 0x79, 0x55, 0xE2, 0x3D, 0x0C
+ },
+
+ },
+ {
+ "[ cpu1#0 cpu1#1 cpu0#0 cpu0#1 cpu1#2 cpu0#2 ]",
+ {
+ 0x5B, 0xFE, 0x29, 0xA4, 0xDA, 0x9F, 0xE7, 0x13,
+ 0x5F, 0xA2, 0xCE, 0x53, 0x40, 0xC0, 0x38, 0xBC,
+ 0x10, 0x7A, 0xF0, 0x29, 0x3C, 0xD6, 0xAF, 0x8A,
+ 0x03, 0x40, 0xED, 0xE1, 0xFD, 0x46, 0xB7, 0x06
+ },
+
+ },
+ {
+ "[ cpu1#0 cpu1#1 cpu0#0 cpu1#2 cpu0#1 cpu0#2 ]",
+ {
+ 0x66, 0x20, 0xA7, 0xBE, 0xED, 0x90, 0x0A, 0x14,
+ 0x95, 0x7A, 0x93, 0x47, 0x1E, 0xA8, 0xDD, 0x6E,
+ 0x25, 0xCB, 0x73, 0x18, 0x77, 0x77, 0x91, 0xE9,
+ 0xCA, 0x17, 0x26, 0x16, 0xAA, 0xC9, 0x34, 0x7A
+ },
+
+ },
+ {
+ "[ cpu1#0 cpu1#1 cpu1#2 cpu0#0 cpu0#1 cpu0#2 ]",
+ {
+ 0x4D, 0xF6, 0xC7, 0x74, 0x37, 0x66, 0x4C, 0x6A,
+ 0x40, 0x32, 0x94, 0x01, 0x17, 0xA2, 0xE6, 0x3D,
+ 0xA8, 0x00, 0x3E, 0xB7, 0x89, 0x24, 0xF4, 0x04,
+ 0x14, 0xA8, 0xA1, 0xD1, 0xCD, 0x5B, 0xC3, 0x60
+ },
+
+ },
+ };
+
+ struct cpu_extend_run cpus[2] = {
+ /* CPU0 */
+ { .idx = &slot, .m = &m[0], .size = SHA256_SIZE },
+ /* CPU1 */
+ { .idx = &slot, .m = &m[1], .size = SHA256_SIZE },
+ };
+
+ for (slot = 1; slot <= REM_COUNT; slot++) {
+ for (m_idx = 0; m_idx < 3; m_idx++) {
+ memcpy(m[0].words, &measure_bytes[m_idx * 8], SHA256_SIZE);
+ memcpy(m[1].words, &measure_bytes[24 + m_idx * 8], SHA256_SIZE);
+ on_cpus(cpu_run_extend_measurement, (void *)&cpus[0]);
+ if (cpus[0].rc || cpus[1].rc)
+ return;
+ }
+ }
+
+ /* Get the token and parse the claims */
+ memset(page_buf_data, 0, sizeof(page_buf_data));
+ memset(&ch, 0xAB, sizeof(ch));
+ if (!get_attest_token_claims(page_buf_data, &ch, &claims, &token_size))
+ return;
+
+ /*
+ * Hard-coded test data expects sha-256 algorithm, skip the measurement
+ * value comparison if realm hash algo is different.
+ */
+ if (!claims_uses_sha256_algo(&claims)) {
+ report_skip("Hash algo is different than sha-256,"
+ " skip measurement value comparison");
+ return;
+ }
+
+ for (slot = 0; slot < REM_COUNT; slot++) {
+ struct claim_t *claim = &claims.realm_measurement_claims[slot];
+ const char *data = claim->buffer_data.ptr;
+ const size_t len = claim->buffer_data.len;
+
+ if (len != SHA256_SIZE) {
+ report(false, "Realm measurement size mismatch "
+ "%zu vs %d (expected)", len, SHA256_SIZE);
+ continue;
+ }
+
+ for (m_idx = 0; m_idx < ARRAY_SIZE(expected); m_idx++) {
+ struct extend_smp_expected *em = &expected[m_idx];
+
+ if (memcmp(data, em->measurement, SHA256_SIZE) == 0) {
+ report(true, "Hash found for slot %d: %s",
+ slot, em->sequence);
+ break;
+ }
+ }
+
+ if (m_idx == ARRAY_SIZE(expected))
+ report(false, "Measurement doesn't match any expected "
+ "sequence for slot %d", slot);
+ }
+}
+
+static void run_rsi_extend_smp_tests(void)
+{
+ report_prefix_push("extend_smp");
+ test_rsi_extend_smp();
+ report_prefix_pop();
+}
+
+static void test_rsi_extend_and_attest(void)
+{
+ struct challenge ch;
+ struct measurement m;
+ struct attestation_claims claims;
+ size_t token_size;
+ int i, j;
+
+ char measure_bytes[] = {
+ 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, /*slot 1*/
+ 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, /*slot 2*/
+ 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, /*slot 3*/
+ 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, /*slot 4*/
+ 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, /*slot 5*/
+ 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, /*slot 6*/
+ 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
+ 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F,
+ 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27
+ };
+
+ /* The following expectations assume extending with SHA256 */
+ char expected_measurements[][SHA256_SIZE] = {
+ {
+ 0x88, 0x78, 0xb1, 0x5a, 0x7d, 0x6a, 0x3a, 0x4f,
+ 0x46, 0x4e, 0x8f, 0x9f, 0x42, 0x59, 0x1d, 0xbc,
+ 0x0c, 0xf4, 0xbe, 0xde, 0xa0, 0xec, 0x30, 0x90,
+ 0x03, 0xd2, 0xb2, 0xee, 0x53, 0x65, 0x5e, 0xf8
+ },
+ {
+ 0x58, 0x32, 0x3b, 0xdf, 0x7a, 0x91, 0xf6, 0x8e,
+ 0x80, 0xc7, 0xc8, 0x7f, 0xda, 0x1e, 0x22, 0x6c,
+ 0x8b, 0xe7, 0xee, 0xa9, 0xef, 0x64, 0xa5, 0x21,
+ 0xdb, 0x2c, 0x09, 0xa7, 0xd7, 0x01, 0x92, 0x05
+ },
+ {
+ 0x66, 0xe3, 0x3b, 0x99, 0x49, 0x4d, 0xf4, 0xdd,
+ 0xbc, 0x7a, 0x61, 0x7a, 0xa1, 0x56, 0x7b, 0xf8,
+ 0x96, 0x3f, 0x0a, 0xf3, 0x1e, 0xab, 0xdd, 0x16,
+ 0x37, 0xb0, 0xfb, 0xe0, 0x71, 0x82, 0x66, 0xce
+ },
+ {
+ 0x97, 0x5e, 0x9f, 0x64, 0x79, 0x90, 0xa1, 0x51,
+ 0xd2, 0x5b, 0x73, 0x75, 0x50, 0x94, 0xeb, 0x54,
+ 0x90, 0xbb, 0x1e, 0xf8, 0x3b, 0x2c, 0xb8, 0x3b,
+ 0x6f, 0x24, 0xf3, 0x86, 0x07, 0xe0, 0x58, 0x13
+ },
+ {
+ 0x68, 0x99, 0x86, 0x64, 0x9b, 0xeb, 0xa2, 0xe4,
+ 0x4d, 0x07, 0xbb, 0xb3, 0xa1, 0xd9, 0x2d, 0x07,
+ 0x76, 0x7f, 0x86, 0x19, 0xb8, 0x5f, 0x14, 0x48,
+ 0x1f, 0x38, 0x4b, 0x87, 0x51, 0xdc, 0x10, 0x31
+ },
+ {
+ 0xee, 0x8f, 0xb3, 0xe9, 0xc8, 0xa5, 0xbe, 0x4f,
+ 0x12, 0x90, 0x4a, 0x52, 0xb9, 0xc8, 0x62, 0xd1,
+ 0x8a, 0x44, 0x31, 0xf7, 0x56, 0x7d, 0x96, 0xda,
+ 0x97, 0x7a, 0x9e, 0x96, 0xae, 0x6a, 0x78, 0x43
+ },
+ };
+ int times_to_extend[] = {1, 2, 3, 4, 5, 6};
+
+ memset(page_buf_data, 0, sizeof(page_buf_data));
+ memset(&ch, 0xAB, sizeof(ch));
+ if (!__get_attest_token_claims(page_buf_data, &ch,
+ &claims, &token_size, true))
+ return;
+
+ for (i = 0; i < REM_COUNT; i++) {
+ struct claim_t c = claims.realm_measurement_claims[i];
+ for (j = 0; j < c.buffer_data.len; j++) {
+ if (((char *)c.buffer_data.ptr)[j])
+ break;
+ }
+ }
+
+ report((i == REM_COUNT), "Initial measurements must be 0");
+
+ /* Extend the possible measurements (i.e., 1 to REM_COUNT) */
+ for (i = 1; i <= REM_COUNT; i++) {
+ memcpy(&m.words[0], &measure_bytes[(i - 1) * 8], SHA256_SIZE);
+ for (j = 0; j < times_to_extend[i - 1]; j++) {
+ struct smccc_result r;
+
+ measurement_extend(i, &m, SHA256_SIZE, &r);
+ if (r.r0) {
+ report(false, "Extend measurment slot %d, iteration %d "
+ "failed with %ld", i, j, r.r0);
+ return;
+ }
+ }
+ }
+ report(true, "Extend measurement for all slots completed");
+
+ /* Get the attestation token again */
+ if (!__get_attest_token_claims(page_buf_data, &ch,
+ &claims, &token_size, true))
+ return;
+
+ /*
+ * Hard-coded test data expects sha-256 algorithm, skip the measurement
+ * value comparison if realm hash algo is different.
+ */
+ if (!claims_uses_sha256_algo(&claims))
+ return;
+
+ /* Verify the extended measurements */
+ for (i = 0; i < REM_COUNT; i++) {
+ const char *exp = expected_measurements[i];
+ const char *actual = claims.realm_measurement_claims[i].buffer_data.ptr;
+ const size_t len = claims.realm_measurement_claims[i].buffer_data.len;
+
+ if (len != SHA256_SIZE) {
+ report(false, "Realm measurement: slot: %d, unexpected size "
+ "actual %ld vs %d expected", i, len,
+ SHA256_SIZE);
+ return;
+ }
+ if (memcmp(exp, actual, len)) {
+ report(false, "Measurement doesn't match for slot %d", i);
+ printf("Expected:\n");
+ for (j = 0; j < len; j++)
+ printf("0x%2x ", exp[j]);
+ printf("\nActual:\n");
+ for (j = 0; j < len; j++)
+ printf("0x%2x ", actual[j]);
+ printf("\n");
+ } else {
+ report(true, "Extended measurement match expected for "
+ "slot %d", i);
+
+ }
+ }
+}
+
+static void run_rsi_extend_and_attest_tests(void)
+{
+ report_prefix_push("extend_and_attest");
+ test_rsi_extend_and_attest();
+ report_prefix_pop();
+}
+
+#define MEASUREMENT_MAX_SIZE_LONGS 8
+
+static void test_read_measurement(void)
+{
+ struct smccc_result result;
+ return_code_t rc;
+ unsigned long *m;
+ int i, j;
+
+ /*
+ * We must be able to read all measurements
+ * 0 (Initial read-only measurement and the
+ * realm extendable ones, 1 to REM_COUNT.
+ */
+ for (i = 0; i <= REM_COUNT; i++) {
+ rsi_read_measurement(i, &result);
+ rc = unpack_return_code(result.r0);
+ if (rc.status) {
+ report(false, "Read measurement failed for slot %d with "
+ "(%d, %d)", i, rc.status, rc.index);
+ return;
+ }
+ m = &result.r1;
+ printf("Read measurement slot:%d, Hash = ", i);
+ for (j = 0; j < MEASUREMENT_MAX_SIZE_LONGS; j++)
+ printf("%lx", __builtin_bswap64(*m++));
+ printf("\n");
+ report(true, "Read Measurement Slot: %d", i);
+ }
+}
+
+static void test_read_measurement_bad_input(void)
+{
+ struct smccc_result result;
+ return_code_t rc;
+
+ report_prefix_push("out-of-range index");
+ rsi_read_measurement(REM_COUNT + 1, &result);
+ rc = unpack_return_code(result.r0);
+ if (rc.status != RSI_ERROR_INPUT) {
+ report(false, "Read measurement fails, "
+ "expected (%d), got (%d)",
+ RSI_ERROR_INPUT, rc.status);
+ } else {
+ report(true, "Read measurement fails as expected");
+ }
+ report_prefix_pop(); /* out-of-range index */
+}
+
+static void run_rsi_read_measurement_tests(void)
+{
+ report_prefix_push("measurement");
+ test_read_measurement();
+ test_read_measurement_bad_input();
+ report_prefix_pop();
+}
+
+int main(int argc, char **argv)
+{
+ int i;
+ report_prefix_push("attestation");
+
+ for (i = 1; i < argc; i++) {
+ if (strcmp(argv[i], "attest") == 0)
+ run_rsi_attest_tests();
+ else if (strcmp(argv[i], "attest_smp") == 0)
+ run_rsi_attest_smp_test();
+ else if (strcmp(argv[i], "extend") == 0)
+ run_rsi_extend_tests();
+ else if (strcmp(argv[i], "extend_smp") == 0)
+ run_rsi_extend_smp_tests();
+ else if (strcmp(argv[i], "extend_and_attest") == 0)
+ run_rsi_extend_and_attest_tests();
+ else if (strcmp(argv[i], "measurement") == 0)
+ run_rsi_read_measurement_tests();
+ else
+ report_info("Unknown subtest '%s'", argv[i]);
+ }
+ return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index bc2354c7..5e9e1cbd 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -311,3 +311,53 @@ file = realm-sea.flat
groups = nodefault realms
accel = kvm
arch = arm64
+
+# Realm Attestation realted tests
+[realm-attest]
+file = realm-attest.flat
+groups = nodefault realms
+smp = 1
+extra_params = -m 32 -append 'attest'
+accel = kvm
+arch = arm64
+
+[realm-attest-smp]
+file = realm-attest.flat
+groups = nodefault realms
+smp = 2
+extra_params = -m 32 -append 'attest_smp'
+accel = kvm
+arch = arm64
+
+[realm-extend]
+file = realm-attest.flat
+groups = nodefault realms
+smp = 1
+extra_params = -m 32 -append 'extend'
+accel = kvm
+arch = arm64
+
+[realm-extend-smp]
+file = realm-attest.flat
+groups = nodefault realms
+smp = 2
+extra_params = -m 32 -append 'extend_smp'
+accel = kvm
+arch = arm64
+
+[realm-extend-and-attest]
+file = realm-attest.flat
+groups = nodefault realms
+smp = 1
+extra_params = -m 32 -append 'extend_and_attest'
+accel = kvm
+arch = arm64
+
+
+[realm-measurement]
+file = realm-attest.flat
+groups = nodefault realms
+smp = 1
+extra_params = -m 32 -append 'measurement'
+accel = kvm
+arch = arm64
diff --git a/lib/libcflat.h b/lib/libcflat.h
index c1fd31ff..893fee6f 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -163,6 +163,7 @@ extern void setup_vm(void);
#define SZ_64K (1 << 16)
#define SZ_1M (1 << 20)
#define SZ_2M (1 << 21)
+#define SZ_512M (1 << 29)
#define SZ_1G (1 << 30)
#define SZ_2G (1ul << 31)

--
2.17.1

2023-01-27 11:47:50

[permalink] [raw]

Subject: [RFC kvm-unit-tests 15/27] arm: realm: Add test for FPU/SIMD context save/restore

From: Subhasish Ghosh <[email protected]>

Test that the FPU/SIMD registers are saved and restored correctly when
context switching VCPUs.

In order to test fpu/simd functionality, we need to make sure that
kvm-unit-tests doesn't generate code that uses the fpu registers, as that
might interfere with the test results. Thus make sure we compile the tests
with -mgeneral-regs-only.

Signed-off-by: Subhasish Ghosh <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
arm/Makefile.arm64 | 1 +
arm/Makefile.common | 1 +
arm/realm-fpu.c | 242 ++++++++++++++++++++++++++++++++++++++++++++
arm/unittests.cfg | 8 ++
4 files changed, 252 insertions(+)
create mode 100644 arm/realm-fpu.c

diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index eed77d3a..90ec6815 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -34,6 +34,7 @@ tests += $(TEST_DIR)/micro-bench.flat
tests += $(TEST_DIR)/cache.flat
tests += $(TEST_DIR)/debug.flat
tests += $(TEST_DIR)/realm-rsi.flat
+tests += $(TEST_DIR)/realm-fpu.flat

include $(SRCDIR)/$(TEST_DIR)/Makefile.common

diff --git a/arm/Makefile.common b/arm/Makefile.common
index 1bbec64f..b339b62d 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -25,6 +25,7 @@ CFLAGS += -std=gnu99
CFLAGS += -ffreestanding
CFLAGS += -O2
CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
+CFLAGS += -mgeneral-regs-only

# We want to keep intermediate files
.PRECIOUS: %.elf %.o
diff --git a/arm/realm-fpu.c b/arm/realm-fpu.c
new file mode 100644
index 00000000..35cfdf09
--- /dev/null
+++ b/arm/realm-fpu.c
@@ -0,0 +1,242 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+
+#include <libcflat.h>
+#include <asm/smp.h>
+#include <stdlib.h>
+
+#include <asm/rsi.h>
+
+#define CPU0_ID 0
+#define CPU1_ID (CPU0_ID + 1)
+#define CPUS_MAX (CPU1_ID + 1)
+#define RMM_FPU_QREG_MAX 32
+#define RMM_FPU_RESULT_PASS (-1U)
+
+#define fpu_reg_read(val) \
+({ \
+ uint64_t *__val = (val); \
+ asm volatile("stp q0, q1, [%0], #32\n\t" \
+ "stp q2, q3, [%0], #32\n\t" \
+ "stp q4, q5, [%0], #32\n\t" \
+ "stp q6, q7, [%0], #32\n\t" \
+ "stp q8, q9, [%0], #32\n\t" \
+ "stp q10, q11, [%0], #32\n\t" \
+ "stp q12, q13, [%0], #32\n\t" \
+ "stp q14, q15, [%0], #32\n\t" \
+ "stp q16, q17, [%0], #32\n\t" \
+ "stp q18, q19, [%0], #32\n\t" \
+ "stp q20, q21, [%0], #32\n\t" \
+ "stp q22, q23, [%0], #32\n\t" \
+ "stp q24, q25, [%0], #32\n\t" \
+ "stp q26, q27, [%0], #32\n\t" \
+ "stp q28, q29, [%0], #32\n\t" \
+ "stp q30, q31, [%0], #32\n\t" \
+ : "=r" (__val) \
+ : \
+ : "q0", "q1", "q2", "q3", \
+ "q4", "q5", "q6", "q7", \
+ "q8", "q9", "q10", "q11", \
+ "q12", "q13", "q14", \
+ "q15", "q16", "q17", \
+ "q18", "q19", "q20", \
+ "q21", "q22", "q23", \
+ "q24", "q25", "q26", \
+ "q27", "q28", "q29", \
+ "q30", "q31", "memory"); \
+})
+
+#define fpu_reg_write(val) \
+do { \
+ uint64_t *__val = (val); \
+ asm volatile("ldp q0, q1, [%0]\n\t" \
+ "ldp q2, q3, [%0]\n\t" \
+ "ldp q4, q5, [%0]\n\t" \
+ "ldp q6, q7, [%0]\n\t" \
+ "ldp q8, q9, [%0]\n\t" \
+ "ldp q10, q11, [%0]\n\t" \
+ "ldp q12, q13, [%0]\n\t" \
+ "ldp q14, q15, [%0]\n\t" \
+ "ldp q16, q17, [%0]\n\t" \
+ "ldp q18, q19, [%0]\n\t" \
+ "ldp q20, q21, [%0]\n\t" \
+ "ldp q22, q23, [%0]\n\t" \
+ "ldp q24, q25, [%0]\n\t" \
+ "ldp q26, q27, [%0]\n\t" \
+ "ldp q28, q29, [%0]\n\t" \
+ "ldp q30, q31, [%0]\n\t" \
+ : \
+ : "r" (__val) \
+ : "q0", "q1", "q2", "q3", \
+ "q4", "q5", "q6", "q7", \
+ "q8", "q9", "q10", "q11",\
+ "q12", "q13", "q14", \
+ "q15", "q16", "q17", \
+ "q18", "q19", "q20", \
+ "q21", "q22", "q23", \
+ "q24", "q25", "q26", \
+ "q27", "q28", "q29", \
+ "q30", "q31", "memory");\
+} while (0)
+
+static void nr_cpu_check(int nr)
+{
+ if (nr_cpus < nr)
+ report_abort("At least %d cpus required", nr);
+}
+/**
+ * @brief check if the FPU/SIMD register contents are the same as
+ * the input data provided.
+ */
+static uint32_t __realm_fpuregs_testall(uint64_t *indata)
+{
+ /* 128b aligned array to read data into */
+ uint64_t outdata[RMM_FPU_QREG_MAX * 2]
+ __attribute__((aligned(sizeof(__uint128_t)))) = {
+ [0 ... ((RMM_FPU_QREG_MAX * 2) - 1)] = 0 };
+ uint8_t regcnt = 0;
+ uint32_t result = 0;
+
+ if (indata == NULL)
+ report_abort("invalid data pointer received");
+
+ /* read data from FPU registers */
+ fpu_reg_read(outdata);
+
+ /* check is the data is the same */
+ for (regcnt = 0; regcnt < (RMM_FPU_QREG_MAX * 2); regcnt += 2) {
+ if ((outdata[regcnt] != indata[regcnt % 4]) ||
+ (outdata[regcnt+1] != indata[(regcnt+1) % 4])) {
+ report_info(
+ "fpu/simd save/restore failed for reg: q%d expected: %lx_%lx received: %lx_%lx\n",
+ regcnt / 2, indata[(regcnt+1) % 4],
+ indata[regcnt % 4], outdata[regcnt+1],
+ outdata[regcnt]);
+ } else {
+ /* populate a bitmask indicating which
+ * registers passed/failed
+ */
+ result |= (1 << (regcnt / 2));
+ }
+ }
+
+ return result;
+}
+
+/**
+ * @brief writes randomly sampled data into the FPU/SIMD registers.
+ */
+static void __realm_fpuregs_writeall_random(uint64_t **indata)
+{
+
+ /* allocate 128b aligned memory */
+ *indata = memalign(sizeof(__uint128_t), sizeof(uint64_t) * 4);
+
+ /* populate the memory with sampled data from a counter */
+ (*indata)[0] = get_cntvct();
+ (*indata)[1] = get_cntvct();
+ (*indata)[2] = get_cntvct();
+ (*indata)[3] = get_cntvct();
+
+ /* write data into FPU registers */
+ fpu_reg_write(*indata);
+}
+
+static void realm_fpuregs_writeall_run(void *data)
+{
+
+ uint64_t **indata = (uint64_t **)data;
+
+ __realm_fpuregs_writeall_random(indata);
+}
+
+static void realm_fpuregs_testall_run(void *data)
+{
+
+ uint64_t *indata = (uint64_t *)data;
+ uint32_t result = 0;
+
+ result = __realm_fpuregs_testall(indata);
+ report((result == RMM_FPU_RESULT_PASS),
+ "fpu/simd register save/restore mask: 0x%x", result);
+}
+
+/**
+ * @brief This test uses two VCPU to test FPU/SIMD save/restore
+ * @details REC1 (vcpu1) writes random data into FPU/SIMD
+ * registers, REC0 (vcpu0) corrupts/overwrites the data and finally
+ * REC1 checks if the data remains unchanged in its context.
+ */
+static void realm_fpuregs_context_switch_cpu1(void)
+{
+ int target = CPU1_ID;
+ uint64_t *indata_remote = NULL;
+ uint64_t *indata_local = NULL;
+
+ /* write data from REC1/VCPU1 */
+ on_cpu(target, realm_fpuregs_writeall_run, &indata_remote);
+
+ /* Overwrite from REC0/VCPU0 */
+ __realm_fpuregs_writeall_random(&indata_local);
+
+ /* check data consistency */
+ on_cpu(target, realm_fpuregs_testall_run, indata_remote);
+
+ free(indata_remote);
+ free(indata_local);
+}
+
+/**
+ * @brief This test uses two VCPU to test FPU/SIMD save/restore
+ * @details REC0 (vcpu0) writes random data into FPU/SIMD
+ * registers, REC1 (vcpu1) corrupts/overwrites the data and finally
+ * REC0 checks if the data remains unchanged in its context.
+ */
+static void realm_fpuregs_context_switch_cpu0(void)
+{
+
+ int target = CPU1_ID;
+ uint64_t *indata_local = NULL;
+ uint64_t *indata_remote = NULL;
+ uint32_t result = 0;
+
+ /* write data from REC0/VCPU0 */
+ __realm_fpuregs_writeall_random(&indata_local);
+
+ /* Overwrite from REC1/VCPU1 */
+ on_cpu(target, realm_fpuregs_writeall_run, &indata_remote);
+
+ /* check data consistency */
+ result = __realm_fpuregs_testall(indata_local);
+ report((result == RMM_FPU_RESULT_PASS),
+ "fpu/simd register save/restore mask: 0x%x", result);
+
+ free(indata_remote);
+ free(indata_local);
+}
+/**
+ * checks if during realm context switch, FPU/SIMD registers
+ * are saved/restored.
+ */
+static void realm_fpuregs_context_switch(void)
+{
+
+ realm_fpuregs_context_switch_cpu0();
+ realm_fpuregs_context_switch_cpu1();
+}
+
+int main(int argc, char **argv)
+{
+ report_prefix_pushf("realm-fpu");
+
+ if (!is_realm())
+ report_skip("Not running in Realm world, skipping");
+
+ nr_cpu_check(CPUS_MAX);
+ realm_fpuregs_context_switch();
+
+ return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 3cdb1a98..a60dc6a9 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -297,3 +297,11 @@ groups = nodefault realms
extra_params = -append 'hvc'
accel = kvm
arch = arm64
+
+# Realm FPU/SIMD test
+[realm-fpu-context]
+file = realm-fpu.flat
+smp = 2
+groups = nodefault realms
+accel = kvm
+arch = arm64
--
2.17.1

2023-01-27 11:47:55

[permalink] [raw]

Subject: [RFC kvm-unit-tests 11/27] arm: realm: Early memory setup

From: Suzuki K Poulose <[email protected]>

A Realm must mark areas of memory as RIPAS_RAM before an access is made.

The binary image is loaded by the VMM and thus the area is converted.
However, the file image may not cover tail portion of the "memory" image (e.g,
BSS, stack etc.). Convert the area touched by the early boot code to RAM
before the access is made in early assembly code.

Once, we land in the C code, we take care of converting the entire RAM region
to RIPAS_RAM.

Please note that this operation doesn't require the host to commit memory to
the Realm.

Signed-off-by: Suzuki K Poulose <[email protected]>
Co-developed-by: Jean-Philippe Brucker <[email protected]>
Signed-off-by: Jean-Philippe Brucker <[email protected]>
Co-developed-by: Joey Gouly <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
arm/cstart64.S | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 72 insertions(+)

diff --git a/arm/cstart64.S b/arm/cstart64.S
index b689b132..b0861594 100644
--- a/arm/cstart64.S
+++ b/arm/cstart64.S
@@ -14,6 +14,7 @@
#include <asm/pgtable-hwdef.h>
#include <asm/thread_info.h>
#include <asm/sysreg.h>
+#include <asm/smc-rsi.h>

.macro zero_range, tmp1, tmp2
9998: cmp \tmp1, \tmp2
@@ -61,6 +62,7 @@ start:
b 1b

1:
+ bl __early_mem_setup
/* zero BSS */
adrp x4, bss
add x4, x4, :lo12:bss
@@ -170,6 +172,76 @@ arm_smccc_hvc:
arm_smccc_smc:
do_smccc_call smc

+__early_mem_setup:
+ /* Preserve x0 - x3 */
+ mov x5, x0
+ mov x6, x1
+ mov x7, x2
+ mov x8, x3
+
+ /*
+ * Check for EL3, otherwise an SMC instruction
+ * will cause an UNDEFINED exception.
+ */
+ mrs x9, ID_AA64PFR0_EL1
+ lsr x9, x9, #12
+ and x9, x9, 0b11
+ cbnz x9, 1f
+ ret
+
+1:
+ /*
+ * Are we a realm? Request the RSI ABI version.
+ * If KVM is catching SMCs, it returns an error in x0 (~0UL)
+ */
+ ldr x0, =SMC_RSI_ABI_VERSION
+ smc #0
+
+ ldr x1, =RSI_ABI_VERSION
+ cmp x0, x1
+ bne 3f
+
+ /*
+ * For realms, we must mark area from bss
+ * to the end of stack as memory before it is
+ * accessed, as they are not populated as part
+ * of the initial image. As such we can run
+ * this unconditionally irrespective of whether
+ * we are a normal VM or Realm.
+ *
+ * x1 = bss_start.
+ */
+ adrp x1, bss
+
+ /* x9 = (end of stack - bss_start) */
+ adrp x9, (stacktop + PAGE_SIZE)
+2:
+ /* calculate the size as (end - start) */
+ sub x2, x9, x1
+
+ /* x3 = RIPAS_RAM */
+ mov x3, #1
+
+ /* x0 = SMC_RSI_IPA_STATE_SET */
+ movz x0, :abs_g2_s:SMC_RSI_IPA_STATE_SET
+ movk x0, :abs_g1_nc:SMC_RSI_IPA_STATE_SET
+ movk x0, :abs_g0_nc:SMC_RSI_IPA_STATE_SET
+
+ /* Run the RSI request */
+ smc #0
+
+ /* halt if there is an error */
+ cbnz x0, halt
+
+ cmp x1, x9
+ bne 2b
+3:
+ mov x3, x8
+ mov x2, x7
+ mov x1, x6
+ mov x0, x5
+ ret
+
get_mmu_off:
adrp x0, auxinfo
ldr x0, [x0, :lo12:auxinfo + 8]
--
2.17.1

2023-01-27 11:48:17

[permalink] [raw]

Subject: [RFC kvm-unit-tests 26/27] arm: realm: Add a test for shared memory

From: Suzuki K Poulose <[email protected]>

Do some basic tests that trigger marking a memory region as
RIPAS_EMPTY and accessing the shared memory. Also, convert it back
to RAM and make sure the contents are scrubbed.

Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
arm/Makefile.arm64 | 1 +
arm/realm-ns-memory.c | 86 +++++++++++++++++++++++++++++++++++++++++++
arm/unittests.cfg | 8 ++++
3 files changed, 95 insertions(+)
create mode 100644 arm/realm-ns-memory.c

diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 0a0c4f2c..9b41e841 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -44,6 +44,7 @@ tests += $(TEST_DIR)/realm-rsi.flat
tests += $(TEST_DIR)/realm-attest.flat
tests += $(TEST_DIR)/realm-fpu.flat
tests += $(TEST_DIR)/realm-sea.flat
+tests += $(TEST_DIR)/realm-ns-memory.flat

include $(SRCDIR)/$(TEST_DIR)/Makefile.common

diff --git a/arm/realm-ns-memory.c b/arm/realm-ns-memory.c
new file mode 100644
index 00000000..8360c371
--- /dev/null
+++ b/arm/realm-ns-memory.c
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+
+#include <asm/io.h>
+#include <alloc_page.h>
+#include <bitops.h>
+
+#define GRANULE_SIZE 0x1000
+#define BUF_SIZE (PAGE_SIZE * 2)
+#define BUF_PAGES (BUF_SIZE / PAGE_SIZE)
+#define BUF_GRANULES (BUF_SIZE / GRANULE_SIZE)
+
+static char __attribute__((aligned(PAGE_SIZE))) buffer[BUF_SIZE];
+
+static void static_shared_buffer_test(void)
+{
+ int i;
+
+ set_memory_decrypted((unsigned long)buffer, sizeof(buffer));
+ for (i = 0; i < sizeof(buffer); i += GRANULE_SIZE)
+ buffer[i] = (char)i;
+
+ /*
+ * Verify the content of the NS buffer
+ */
+ for (i = 0; i < sizeof(buffer); i += GRANULE_SIZE) {
+ if (buffer[i] != (char)i) {
+ report(false, "Failed to set Non Secure memory");
+ return;
+ }
+ }
+
+ /* Make the buffer back to protected... */
+ set_memory_encrypted((unsigned long)buffer, sizeof(buffer));
+ /* .. and check if the contents were destroyed */
+ for (i = 0; i < sizeof(buffer); i += GRANULE_SIZE) {
+ if (buffer[i] != 0) {
+ report(false, "Failed to scrub protected memory");
+ return;
+ }
+ }
+
+ report(true, "Conversion of protected memory to shared and back");
+}
+
+static void dynamic_shared_buffer_test(void)
+{
+ char *ns_buffer;
+ int i;
+ int order = get_order(BUF_PAGES);
+
+ ns_buffer = alloc_pages_shared(order);
+ assert(ns_buffer);
+ for (i = 0; i < sizeof(buffer); i += GRANULE_SIZE)
+ ns_buffer[i] = (char)i;
+
+ /*
+ * Verify the content of the NS buffer
+ */
+ for (i = 0; i < sizeof(buffer); i += GRANULE_SIZE) {
+ if (ns_buffer[i] != (char)i) {
+ report(false, "Failed to set Non Secure memory");
+ return;
+ }
+ }
+ free_pages_shared(ns_buffer);
+ report(true, "Dynamic allocation and free of shared memory\n");
+}
+
+static void ns_test(void)
+{
+ static_shared_buffer_test();
+ dynamic_shared_buffer_test();
+}
+
+int main(int argc, char **argv)
+{
+ report_prefix_pushf("ns-memory");
+ ns_test();
+ report_prefix_pop();
+
+ return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 5e9e1cbd..8173ccfe 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -361,3 +361,11 @@ smp = 1
extra_params = -m 32 -append 'measurement'
accel = kvm
arch = arm64
+
+[realm-ns-memory]
+file=realm-ns-memory.flat
+groups = nodefault realms
+smp = 1
+extra_params = -m 32
+accel = kvm
+arch = arm64
--
2.17.1

2023-01-27 11:48:20

[permalink] [raw]

Subject: [RFC kvm-unit-tests 24/27] arm: realm: Add helpers to decode RSI return codes

From: Suzuki K Poulose <[email protected]>

RMM encodes error code and index in the result of an operation.
Add helpers to decode this information for use with the attestation
tests.

Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
lib/arm64/asm/rsi.h | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)

diff --git a/lib/arm64/asm/rsi.h b/lib/arm64/asm/rsi.h
index 50bab993..1d01a929 100644
--- a/lib/arm64/asm/rsi.h
+++ b/lib/arm64/asm/rsi.h
@@ -16,6 +16,39 @@

extern bool rsi_present;

+/*
+ * Logical representation of return code returned by RMM commands.
+ * Each failure mode of a given command should return a unique return code, so
+ * that the caller can use it to unambiguously identify the failure mode. To
+ * avoid having a very large list of enumerated values, the return code is
+ * composed of a status which identifies the category of the error (for example,
+ * an address was misaligned), and an index which disambiguates between multiple
+ * similar failure modes (for example, a command may take multiple addresses as
+ * its input; the index identifies _which_ of them was misaligned.)
+ */
+typedef unsigned int status_t;
+typedef struct {
+ status_t status;
+ unsigned int index;
+} return_code_t;
+
+/*
+ * Convenience function for creating a return_code_t.
+ */
+static inline return_code_t make_return_code(unsigned int status,
+ unsigned int index)
+{
+ return (return_code_t) {status, index};
+}
+
+/*
+ * Unpacks a return code.
+ */
+static inline return_code_t unpack_return_code(unsigned long error_code)
+{
+ return make_return_code(error_code & 0xff, error_code >> 8);
+}
+
void arm_rsi_init(void);

int rsi_invoke(unsigned int function_id, unsigned long arg0,
--
2.17.1

2023-01-27 11:48:29

[permalink] [raw]

Subject: [RFC kvm-unit-tests 18/27] arm: gic-v3-its: Use shared pages wherever needed

From: Suzuki K Poulose <[email protected]>

GICv3-ITS is emulated by the host and thus we should allocate shared pages for
access by the host. Make sure the allocations are shared.

Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
lib/arm/gic-v3.c | 6 ++++--
lib/arm64/gic-v3-its.c | 6 +++---
2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
index 2f7870ab..813cd5a6 100644
--- a/lib/arm/gic-v3.c
+++ b/lib/arm/gic-v3.c
@@ -171,7 +171,9 @@ void gicv3_lpi_alloc_tables(void)
u64 prop_val;
int cpu;

- gicv3_data.lpi_prop = alloc_pages(order);
+ assert(gicv3_redist_base());
+
+ gicv3_data.lpi_prop = alloc_pages_shared(order);

/* ID bits = 13, ie. up to 14b LPI INTID */
prop_val = (u64)(virt_to_phys(gicv3_data.lpi_prop)) | 13;
@@ -186,7 +188,7 @@ void gicv3_lpi_alloc_tables(void)

writeq(prop_val, ptr + GICR_PROPBASER);

- gicv3_data.lpi_pend[cpu] = alloc_pages(order);
+ gicv3_data.lpi_pend[cpu] = alloc_pages_shared(order);
pend_val = (u64)(virt_to_phys(gicv3_data.lpi_pend[cpu]));
writeq(pend_val, ptr + GICR_PENDBASER);
}
diff --git a/lib/arm64/gic-v3-its.c b/lib/arm64/gic-v3-its.c
index 2c69cfda..07dbeb81 100644
--- a/lib/arm64/gic-v3-its.c
+++ b/lib/arm64/gic-v3-its.c
@@ -54,7 +54,7 @@ static void its_baser_alloc_table(struct its_baser *baser, size_t size)
void *reg_addr = gicv3_its_base() + GITS_BASER + baser->index * 8;
u64 val = readq(reg_addr);

- baser->table_addr = alloc_pages(order);
+ baser->table_addr = alloc_pages_shared(order);

val |= virt_to_phys(baser->table_addr) | GITS_BASER_VALID;

@@ -70,7 +70,7 @@ static void its_cmd_queue_init(void)
unsigned long order = get_order(SZ_64K >> PAGE_SHIFT);
u64 cbaser;

- its_data.cmd_base = alloc_pages(order);
+ its_data.cmd_base = alloc_pages_shared(order);

cbaser = virt_to_phys(its_data.cmd_base) | (SZ_64K / SZ_4K - 1) | GITS_CBASER_VALID;

@@ -123,7 +123,7 @@ struct its_device *its_create_device(u32 device_id, int nr_ites)
new->nr_ites = nr_ites;

n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
- new->itt = alloc_pages(get_order(n));
+ new->itt = alloc_pages_shared(get_order(n));

its_data.nr_devices++;
return new;
--
2.17.1

2023-01-27 11:49:04

[permalink] [raw]

Subject: [RFC kvm-unit-tests 04/27] arm: Make physical address mask dynamic

We are about to add Realm support where the physical address width may be known
via RSI. Make the Physical Address mask dynamic, so that it can be adjusted
to the limit for the realm. This will be required for making pages shared, as
we introduce the "sharing" attribute as the top bit of the IPA.

Signed-off-by: Joey Gouly <[email protected]>
---
lib/arm/mmu.c | 2 ++
lib/arm/setup.c | 1 +
lib/arm64/asm/pgtable-hwdef.h | 6 ------
lib/arm64/asm/pgtable.h | 9 +++++++++
4 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
index e1a72fe4..acaf5614 100644
--- a/lib/arm/mmu.c
+++ b/lib/arm/mmu.c
@@ -22,6 +22,8 @@

pgd_t *mmu_idmap;

+unsigned long phys_mask_shift = 48;
+
/* CPU 0 starts with disabled MMU */
static cpumask_t mmu_enabled_cpumask;

diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index bcdf0d78..81052a3d 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -22,6 +22,7 @@
#include <asm/thread_info.h>
#include <asm/setup.h>
#include <asm/page.h>
+#include <asm/pgtable.h>
#include <asm/processor.h>
#include <asm/smp.h>
#include <asm/timer.h>
diff --git a/lib/arm64/asm/pgtable-hwdef.h b/lib/arm64/asm/pgtable-hwdef.h
index 8c41fe12..ac95550b 100644
--- a/lib/arm64/asm/pgtable-hwdef.h
+++ b/lib/arm64/asm/pgtable-hwdef.h
@@ -115,12 +115,6 @@
#define PTE_ATTRINDX(t) (_AT(pteval_t, (t)) << 2)
#define PTE_ATTRINDX_MASK (_AT(pteval_t, 7) << 2)

-/*
- * Highest possible physical address supported.
- */
-#define PHYS_MASK_SHIFT (48)
-#define PHYS_MASK ((UL(1) << PHYS_MASK_SHIFT) - 1)
-
/*
* TCR flags.
*/
diff --git a/lib/arm64/asm/pgtable.h b/lib/arm64/asm/pgtable.h
index bfb8a993..22ce64f0 100644
--- a/lib/arm64/asm/pgtable.h
+++ b/lib/arm64/asm/pgtable.h
@@ -21,6 +21,15 @@

#include <linux/compiler.h>

+extern unsigned long prot_ns_shared;
+/*
+ * Highest possible physical address supported.
+ */
+extern unsigned long phys_mask_shift;
+#define PHYS_MASK_SHIFT (phys_mask_shift)
+#define PHYS_MASK ((UL(1) << PHYS_MASK_SHIFT) - 1)
+
+
/*
* We can convert va <=> pa page table addresses with simple casts
* because we always allocate their pages with alloc_page(), and
--
2.17.1

2023-01-27 11:49:24

[permalink] [raw]

Subject: [RFC kvm-unit-tests 23/27] arm: realm: add RSI interface for attestation measurements

From: Suzuki K Poulose <[email protected]>

Add wrappers for the Attestation and measurement related RSI calls.
These will be later used in the test cases

Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
lib/arm64/asm/rsi.h | 7 +++++++
lib/arm64/rsi.c | 32 ++++++++++++++++++++++++++++++++
2 files changed, 39 insertions(+)

diff --git a/lib/arm64/asm/rsi.h b/lib/arm64/asm/rsi.h
index c8179341..50bab993 100644
--- a/lib/arm64/asm/rsi.h
+++ b/lib/arm64/asm/rsi.h
@@ -27,6 +27,13 @@ int rsi_invoke(unsigned int function_id, unsigned long arg0,
struct smccc_result *result);

int rsi_get_version(void);
+void rsi_attest_token_init(phys_addr_t addr, unsigned long *challenge,
+ struct smccc_result *res);
+void rsi_attest_token_continue(phys_addr_t addr, struct smccc_result *res);
+void rsi_extend_measurement(unsigned int index, unsigned long size,
+ unsigned long *measurement,
+ struct smccc_result *res);
+void rsi_read_measurement(unsigned int index, struct smccc_result *res);

static inline bool is_realm(void)
{
diff --git a/lib/arm64/rsi.c b/lib/arm64/rsi.c
index 08c77889..63d0620a 100644
--- a/lib/arm64/rsi.c
+++ b/lib/arm64/rsi.c
@@ -66,6 +66,38 @@ void arm_rsi_init(void)
prot_ns_shared = (1UL << phys_mask_shift);
}

+void rsi_attest_token_init(phys_addr_t addr, unsigned long *challenge,
+ struct smccc_result *res)
+{
+ rsi_invoke(SMC_RSI_ATTEST_TOKEN_INIT, addr,
+ challenge[0], challenge[1], challenge[2],
+ challenge[3], challenge[4], challenge[5],
+ challenge[6], challenge[7], 0, 0, res);
+}
+
+void rsi_attest_token_continue(phys_addr_t addr, struct smccc_result *res)
+{
+ rsi_invoke(SMC_RSI_ATTEST_TOKEN_CONTINUE, addr,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, res);
+}
+
+void rsi_extend_measurement(unsigned int index, unsigned long size,
+ unsigned long *measurement, struct smccc_result *res)
+{
+ rsi_invoke(SMC_RSI_MEASUREMENT_EXTEND, index, size,
+ measurement[0], measurement[1],
+ measurement[2], measurement[3],
+ measurement[4], measurement[5],
+ measurement[6], measurement[7],
+ 0, res);
+}
+
+void rsi_read_measurement(unsigned int index, struct smccc_result *res)
+{
+ rsi_invoke(SMC_RSI_MEASUREMENT_READ, index, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, res);
+}
+
static unsigned rsi_set_addr_range_state(unsigned long start, unsigned long size,
enum ripas_t state, unsigned long *top)
{
--
2.17.1

2023-01-27 11:51:15

[permalink] [raw]

Subject: [RFC kvm-unit-tests 02/27] arm: Expand SMCCC arguments and return values

From: Alexandru Elisei <[email protected]>

PSCI uses the SMC Calling Convention (SMCCC) to communicate with the higher
level software. PSCI uses at most 4 arguments and expend only one return
value. However, SMCCC has provisions for more arguments (upto 17 depending
on the SMCCC version) and upto 10 distinct return values.

We are going to be adding tests that make use of it, so add support for the
extended number of arguments and return values.

Also rename the SMCCC functions to generic, non-PSCI names, so they
can be used for Realm services.

Signed-off-by: Alexandru Elisei <[email protected]>
[ Expand the number of args to 11 /results 10]
Signed-off-by: Joey Gouly <[email protected]>
---
arm/cstart.S | 49 ++++++++++++++++++++++++++++------
arm/cstart64.S | 55 +++++++++++++++++++++++++++++++++------
arm/selftest.c | 2 +-
lib/arm/asm/arm-smccc.h | 44 +++++++++++++++++++++++++++++++
lib/arm/asm/psci.h | 13 +++++----
lib/arm/psci.c | 19 +++++++++++---
lib/arm64/asm/arm-smccc.h | 6 +++++
7 files changed, 160 insertions(+), 28 deletions(-)
create mode 100644 lib/arm/asm/arm-smccc.h
create mode 100644 lib/arm64/asm/arm-smccc.h

diff --git a/arm/cstart.S b/arm/cstart.S
index 7036e67f..db377668 100644
--- a/arm/cstart.S
+++ b/arm/cstart.S
@@ -96,26 +96,59 @@ start:
.text

/*
- * psci_invoke_hvc / psci_invoke_smc
+ * arm_smccc_hvc / arm_smccc_smc
*
* Inputs:
* r0 -- function_id
* r1 -- arg0
* r2 -- arg1
* r3 -- arg2
+ * [sp] - arg3
+ * [sp + #4] - arg4
+ * [sp + #8] - arg5
+ * [sp + #12] - arg6
+ * [sp + #16] - arg7
+ * [sp + #20] - arg8
+ * [sp + #24] - arg9
+ * [sp + #28] - arg10
+ * [sp + #32] - result (as a pointer to a struct smccc_result)
*
* Outputs:
* r0 -- return code
+ *
+ * If result pointer is not NULL:
+ * result.r0 -- return code
+ * result.r1 -- r1
+ * result.r2 -- r2
+ * result.r3 -- r3
+ * result.r4 -- r4
+ * result.r5 -- r5
+ * result.r6 -- r6
+ * result.r7 -- r7
+ * result.r8 -- r8
+ * result.r9 -- r9
*/
-.globl psci_invoke_hvc
-psci_invoke_hvc:
- hvc #0
+.macro do_smccc_call instr
+ mov r12, sp
+ push {r4-r11}
+ ldm r12, {r4-r11}
+ \instr #0
+ ldr r10, [sp, #64]
+ cmp r10, #0
+ beq 1f
+ stm r10, {r0-r9}
+1:
+ pop {r4-r11}
mov pc, lr
+.endm

-.globl psci_invoke_smc
-psci_invoke_smc:
- smc #0
- mov pc, lr
+.globl arm_smccc_hvc
+arm_smccc_hvc:
+ do_smccc_call hvc
+
+.globl arm_smccc_smc
+arm_smccc_smc:
+ do_smccc_call smc

enable_vfp:
/* Enable full access to CP10 and CP11: */
diff --git a/arm/cstart64.S b/arm/cstart64.S
index e4ab7d06..b689b132 100644
--- a/arm/cstart64.S
+++ b/arm/cstart64.S
@@ -110,26 +110,65 @@ start:
.text

/*
- * psci_invoke_hvc / psci_invoke_smc
+ * arm_smccc_hvc / arm_smccc_smc
*
* Inputs:
* w0 -- function_id
* x1 -- arg0
* x2 -- arg1
* x3 -- arg2
+ * x4 -- arg3
+ * x5 -- arg4
+ * x6 -- arg5
+ * x7 -- arg6
+ * sp -- { arg7, arg8, arg9, arg10, result }
*
* Outputs:
* x0 -- return code
+ *
+ * If result pointer is not NULL:
+ * result.r0 -- return code
+ * result.r1 -- x1
+ * result.r2 -- x2
+ * result.r3 -- x3
+ * result.r4 -- x4
+ * result.r5 -- x5
+ * result.r6 -- x6
+ * result.r7 -- x7
+ * result.r8 -- x8
+ * result.r9 -- x9
*/
-.globl psci_invoke_hvc
-psci_invoke_hvc:
- hvc #0
+.macro do_smccc_call instr
+ /* Save x8-x11 on stack */
+ stp x9, x8, [sp, #-16]!
+ stp x11, x10, [sp, #-16]!
+ /* Load arg7 - arg10 from the stack */
+ ldp x8, x9, [sp, #32]
+ ldp x10, x11, [sp, #48]
+ \instr #0
+ /* Get the result address */
+ ldr x10, [sp, #64]
+ cmp x10, xzr
+ b.eq 1f
+ stp x0, x1, [x10, #0]
+ stp x2, x3, [x10, #16]
+ stp x4, x5, [x10, #32]
+ stp x6, x7, [x10, #48]
+ stp x8, x9, [x10, #64]
+1:
+ /* Restore x8-x11 from stack */
+ ldp x11, x10, [sp], #16
+ ldp x9, x8, [sp], #16
ret
+.endm

-.globl psci_invoke_smc
-psci_invoke_smc:
- smc #0
- ret
+.globl arm_smccc_hvc
+arm_smccc_hvc:
+ do_smccc_call hvc
+
+.globl arm_smccc_smc
+arm_smccc_smc:
+ do_smccc_call smc

get_mmu_off:
adrp x0, auxinfo
diff --git a/arm/selftest.c b/arm/selftest.c
index 9f459ed3..6f825add 100644
--- a/arm/selftest.c
+++ b/arm/selftest.c
@@ -405,7 +405,7 @@ static void psci_print(void)
int ver = psci_invoke(PSCI_0_2_FN_PSCI_VERSION, 0, 0, 0);
report_info("PSCI version: %d.%d", PSCI_VERSION_MAJOR(ver),
PSCI_VERSION_MINOR(ver));
- report_info("PSCI method: %s", psci_invoke == psci_invoke_hvc ?
+ report_info("PSCI method: %s", psci_invoke_fn == arm_smccc_hvc ?
"hvc" : "smc");
}

diff --git a/lib/arm/asm/arm-smccc.h b/lib/arm/asm/arm-smccc.h
new file mode 100644
index 00000000..5d85b01a
--- /dev/null
+++ b/lib/arm/asm/arm-smccc.h
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+#ifndef _ASMARM_ARM_SMCCC_H_
+#define _ASMARM_ARM_SMCCC_H_
+
+struct smccc_result {
+ unsigned long r0;
+ unsigned long r1;
+ unsigned long r2;
+ unsigned long r3;
+ unsigned long r4;
+ unsigned long r5;
+ unsigned long r6;
+ unsigned long r7;
+ unsigned long r8;
+ unsigned long r9;
+};
+
+typedef int (*smccc_invoke_fn)(unsigned int function_id, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2,
+ unsigned long arg3, unsigned long arg4,
+ unsigned long arg5, unsigned long arg6,
+ unsigned long arg7, unsigned long arg8,
+ unsigned long arg9, unsigned long arg10,
+ struct smccc_result *result);
+extern int arm_smccc_hvc(unsigned int function_id, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2,
+ unsigned long arg3, unsigned long arg4,
+ unsigned long arg5, unsigned long arg6,
+ unsigned long arg7, unsigned long arg8,
+ unsigned long arg9, unsigned long arg10,
+ struct smccc_result *result);
+extern int arm_smccc_smc(unsigned int function_id, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2,
+ unsigned long arg3, unsigned long arg4,
+ unsigned long arg5, unsigned long arg6,
+ unsigned long arg7, unsigned long arg8,
+ unsigned long arg9, unsigned long arg10,
+ struct smccc_result *result);
+
+#endif /* _ASMARM_ARM_SMCCC_H_ */
diff --git a/lib/arm/asm/psci.h b/lib/arm/asm/psci.h
index cf03449b..6a399621 100644
--- a/lib/arm/asm/psci.h
+++ b/lib/arm/asm/psci.h
@@ -3,13 +3,12 @@
#include <libcflat.h>
#include <linux/psci.h>

-typedef int (*psci_invoke_fn)(unsigned int function_id, unsigned long arg0,
- unsigned long arg1, unsigned long arg2);
-extern psci_invoke_fn psci_invoke;
-extern int psci_invoke_hvc(unsigned int function_id, unsigned long arg0,
- unsigned long arg1, unsigned long arg2);
-extern int psci_invoke_smc(unsigned int function_id, unsigned long arg0,
- unsigned long arg1, unsigned long arg2);
+#include <asm/arm-smccc.h>
+
+extern smccc_invoke_fn psci_invoke_fn;
+
+extern int psci_invoke(unsigned int function_id, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2);
extern void psci_set_conduit(void);
extern int psci_cpu_on(unsigned long cpuid, unsigned long entry_point);
extern void psci_system_reset(void);
diff --git a/lib/arm/psci.c b/lib/arm/psci.c
index 9c031a12..0a1d0e82 100644
--- a/lib/arm/psci.c
+++ b/lib/arm/psci.c
@@ -13,13 +13,24 @@
#include <asm/smp.h>

static int psci_invoke_none(unsigned int function_id, unsigned long arg0,
- unsigned long arg1, unsigned long arg2)
+ unsigned long arg1, unsigned long arg2,
+ unsigned long arg3, unsigned long arg4,
+ unsigned long arg5, unsigned long arg6,
+ unsigned long arg7, unsigned long arg8,
+ unsigned long arg9, unsigned long arg10,
+ struct smccc_result *result)
{
printf("No PSCI method configured! Can't invoke...\n");
return PSCI_RET_NOT_PRESENT;
}

-psci_invoke_fn psci_invoke = psci_invoke_none;
+smccc_invoke_fn psci_invoke_fn = psci_invoke_none;
+
+int psci_invoke(unsigned int function_id, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2)
+{
+ return psci_invoke_fn(function_id, arg0, arg1, arg2, 0, 0, 0, 0, 0, 0, 0, 0, NULL);
+}

int psci_cpu_on(unsigned long cpuid, unsigned long entry_point)
{
@@ -69,9 +80,9 @@ void psci_set_conduit(void)
assert(method != NULL && len == 4);

if (strcmp(method->data, "hvc") == 0)
- psci_invoke = psci_invoke_hvc;
+ psci_invoke_fn = arm_smccc_hvc;
else if (strcmp(method->data, "smc") == 0)
- psci_invoke = psci_invoke_smc;
+ psci_invoke_fn = arm_smccc_smc;
else
assert_msg(false, "Unknown PSCI conduit: %s", method->data);
}
diff --git a/lib/arm64/asm/arm-smccc.h b/lib/arm64/asm/arm-smccc.h
new file mode 100644
index 00000000..ab649489
--- /dev/null
+++ b/lib/arm64/asm/arm-smccc.h
@@ -0,0 +1,6 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+#include "../../arm/asm/arm-smccc.h"
--
2.17.1

2023-01-27 11:52:37

[permalink] [raw]

Subject: [RFC kvm-unit-tests 06/27] arm: Move io_init after vm initialization

From: Jean-Philippe Brucker <[email protected]>

To create shared pages, the NS_SHARED bit must be written into the
idmap. Before VM initializations, idmap hasn't necessarily been created.
To write shared pages, access must be done on a IPA with the NS_SHARED
bit. When the stage-1 MMU is enabled, that bit is set in the PTE. But
when the stage-1 MMU is disabled, then the realm must write to the IPA
with NS_SHARED directly.

To avoid changing the whole virtio infrastructure to support pre-MMU in
a realm, move the IO initialization after MMU enablement.

Signed-off-by: Jean-Philippe Brucker <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
lib/arm/setup.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 81052a3d..65d98e97 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -274,9 +274,6 @@ void setup(const void *fdt, phys_addr_t freemem_start)
/* cpu_init must be called before thread_info_init */
thread_info_init(current_thread_info(), 0);

- /* mem_init must be called before io_init */
- io_init();
-
timer_save_state();

ret = dt_get_bootargs(&bootargs);
@@ -292,4 +289,7 @@ void setup(const void *fdt, phys_addr_t freemem_start)

if (!(auxinfo.flags & AUXINFO_MMU_OFF))
setup_vm();
+
+ /* mem_init and setup_vm must be called before io_init */
+ io_init();
}
--
2.17.1

2023-01-27 11:52:39

[permalink] [raw]

Subject: [RFC kvm-unit-tests 01/27] lib/string: include stddef.h for size_t

Don't implicitly rely on this header being included.

Signed-off-by: Joey Gouly <[email protected]>
---
lib/string.h | 2 ++
1 file changed, 2 insertions(+)

diff --git a/lib/string.h b/lib/string.h
index b07763ea..758dca8a 100644
--- a/lib/string.h
+++ b/lib/string.h
@@ -7,6 +7,8 @@
#ifndef _STRING_H_
#define _STRING_H_

+#include <stddef.h> /* For size_t */
+
extern size_t strlen(const char *buf);
extern size_t strnlen(const char *buf, size_t maxlen);
extern char *strcat(char *dest, const char *src);
--
2.17.1

2023-01-27 11:55:33

[permalink] [raw]

Subject: [RFC kvm-unit-tests 12/27] arm: realm: Add RSI version test

Add basic test for checking the RSI version command.

Signed-off-by: Joey Gouly <[email protected]>
---
arm/Makefile.arm64 | 1 +
arm/realm-rsi.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++
arm/unittests.cfg | 7 +++++++
3 files changed, 57 insertions(+)
create mode 100644 arm/realm-rsi.c

diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index ab557f84..eed77d3a 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -33,6 +33,7 @@ tests = $(TEST_DIR)/timer.flat
tests += $(TEST_DIR)/micro-bench.flat
tests += $(TEST_DIR)/cache.flat
tests += $(TEST_DIR)/debug.flat
+tests += $(TEST_DIR)/realm-rsi.flat

include $(SRCDIR)/$(TEST_DIR)/Makefile.common

diff --git a/arm/realm-rsi.c b/arm/realm-rsi.c
new file mode 100644
index 00000000..d793f305
--- /dev/null
+++ b/arm/realm-rsi.c
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+
+#include <libcflat.h>
+#include <asm/io.h>
+#include <asm/page.h>
+#include <asm/processor.h>
+#include <asm/psci.h>
+#include <alloc_page.h>
+#include <asm/rsi.h>
+#include <asm/pgtable.h>
+#include <asm/processor.h>
+
+static void rsi_test_version(void)
+{
+ int version;
+
+ report_prefix_push("version");
+
+ version = rsi_get_version();
+ if (version < 0) {
+ report(false, "SMC_RSI_ABI_VERSION failed (%d)", version);
+ return;
+ }
+
+ report(version == RSI_ABI_VERSION, "RSI ABI version %u.%u (expected: %u.%u)",
+ RSI_ABI_VERSION_GET_MAJOR(version),
+ RSI_ABI_VERSION_GET_MINOR(version),
+ RSI_ABI_VERSION_GET_MAJOR(RSI_ABI_VERSION),
+ RSI_ABI_VERSION_GET_MINOR(RSI_ABI_VERSION));
+ report_prefix_pop();
+}
+
+int main(int argc, char **argv)
+{
+ report_prefix_push("rsi");
+
+ if (!is_realm()) {
+ report_skip("Not a realm, skipping tests");
+ goto exit;
+ }
+
+ rsi_test_version();
+exit:
+ return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 5e67b558..ce1b5ad9 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -275,3 +275,10 @@ file = debug.flat
arch = arm64
extra_params = -append 'ss-migration'
groups = debug migration
+
+# Realm RSI ABI test
+[realm-rsi]
+file = realm-rsi.flat
+groups = nodefault realms
+accel = kvm
+arch = arm64
--
2.17.1

2023-01-27 11:55:37

[permalink] [raw]

Subject: [RFC kvm-unit-tests 27/27] NOT-FOR-MERGING: add run-realm-tests

From: Alexandru Elisei <[email protected]>

Until we add support for KVMTOOL to run the tests using the
scripts, provide a temporary script to run all the Realm tests.

Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
arm/run-realm-tests | 56 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 56 insertions(+)
create mode 100755 arm/run-realm-tests

diff --git a/arm/run-realm-tests b/arm/run-realm-tests
new file mode 100755
index 00000000..39f431d5
--- /dev/null
+++ b/arm/run-realm-tests
@@ -0,0 +1,56 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (C) 2023, Arm Ltd
+# All rights reserved
+#
+
+TASKSET=${TASKSET:-taskset}
+LKVM=${LKVM:-lkvm}
+ARGS="--realm --irqchip=gicv3 --console=serial --network mode=none --nodefaults"
+
+TESTDIR="."
+while getopts "d:" option; do
+ case "${option}" in
+ d) TESTDIR=${OPTARG};;
+ ?)
+ exit 1
+ ;;
+ esac
+done
+if [[ ! -d ${TESTDIR} ]]; then
+ echo "Invalid directory: ${TESTDIR}"
+ exit 1
+fi
+
+function run_tests {
+ DIR="$1"
+
+ $LKVM run $ARGS -c 2 -m 16 -k $DIR/selftest.flat -p "setup smp=2 mem=16"
+ $LKVM run $ARGS -c 1 -m 16 -k $DIR/selftest.flat -p "vectors-kernel"
+ $LKVM run $ARGS -c 1 -m 16 -k $DIR/selftest.flat -p "vectors-user"
+ $LKVM run $ARGS -c 4 -m 32 -k $DIR/selftest.flat -p "smp"
+
+ $LKVM run $ARGS -c 1 -m 32 -k $DIR/realm-ns-memory.flat
+
+ $LKVM run $ARGS -c 4 -m 32 -k $DIR/psci.flat
+
+ $LKVM run $ARGS -c 4 -m 32 -k $DIR/gic.flat -p "ipi"
+ $LKVM run $ARGS -c 4 -m 32 -k $DIR/gic.flat -p "active"
+
+ $LKVM run $ARGS -c 1 -m 16 -k $DIR/timer.flat
+
+ $LKVM run $ARGS -c 1 -m 16 -k $DIR/realm-rsi.flat -p "version"
+ $LKVM run $ARGS -c 1 -m 16 -k $DIR/realm-rsi.flat -p "host_call hvc"
+ $LKVM run $ARGS -c 1 -m 16 -k $DIR/realm-sea.flat
+
+ $LKVM run $ARGS -c 1 -m 24 -k $DIR/realm-attest.flat -p "attest"
+ $LKVM run $ARGS -c 2 -m 24 -k $DIR/realm-attest.flat -p "attest_smp"
+ $LKVM run $ARGS -c 1 -m 24 -k $DIR/realm-attest.flat -p "extend"
+ $LKVM run $ARGS -c 2 -m 24 -k $DIR/realm-attest.flat -p "extend_smp"
+ $LKVM run $ARGS -c 1 -m 24 -k $DIR/realm-attest.flat -p "extend_and_attest"
+ $LKVM run $ARGS -c 1 -m 24 -k $DIR/realm-attest.flat -p "measurement"
+
+ $TASKSET -c 0 $LKVM run $ARGS -c 4 -m 32 -k $DIR/realm-fpu.flat
+}
+
+run_tests "${TESTDIR}"
--
2.17.1

2023-01-27 11:57:27

[permalink] [raw]

Subject: [RFC kvm-unit-tests 09/27] arm: realm: Add support for changing the state of memory

For a Realm, the guest physical address (in reality the IPA/GPA of the VM)
has an associated state (Realm IPA State, RIPAS) which is either of :
RIPAS_RAM
RIPAS_EMPTY

The state of the physical address decides certain behaviors. e.g., any access
to a RIPAS_EMPTY PA will generate a Synchronous External Abort back to the Realm,
from the RMM.

All "PA" that represents RAM for the Realm, must be set to RIPAS_RAM before
an access is made. When the initial image (e.g., test, DTB) of a Realm is
loaded, the hypervisor/VMM can transition the state of the loaded "area" to
RIPAS_RAM. The rest of the "RAM" must be transitioned by the test payload
before any access is made.

Similarly, a Realm could set an "IPA" to RIPAS_EMPTY, when it is about to use
the "unprotected" alias of the IPA. This is a hint for the host to reclaim the
page from the protected "IPA.

This patchs adds supporting helpers for setting the IPA state from Realm. These
will be used later for the Realm.

Co-developed-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
lib/arm/mmu.c | 1 +
lib/arm64/asm/rsi.h | 8 ++++++++
lib/arm64/rsi.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 53 insertions(+)

diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
index 6f1f42f5..2b5a7141 100644
--- a/lib/arm/mmu.c
+++ b/lib/arm/mmu.c
@@ -12,6 +12,7 @@
#include <asm/setup.h>
#include <asm/page.h>
#include <asm/io.h>
+#include <asm/rsi.h>

#include "alloc_page.h"
#include "vmalloc.h"
diff --git a/lib/arm64/asm/rsi.h b/lib/arm64/asm/rsi.h
index 8b9b91b2..c8179341 100644
--- a/lib/arm64/asm/rsi.h
+++ b/lib/arm64/asm/rsi.h
@@ -33,4 +33,12 @@ static inline bool is_realm(void)
return rsi_present;
}

+enum ripas_t {
+ RIPAS_EMPTY,
+ RIPAS_RAM,
+};
+
+void arm_set_memory_protected(unsigned long va, size_t size);
+void arm_set_memory_shared(unsigned long va, size_t size);
+
#endif /* __ASMARM64_RSI_H_ */
diff --git a/lib/arm64/rsi.c b/lib/arm64/rsi.c
index 23a4e963..08c77889 100644
--- a/lib/arm64/rsi.c
+++ b/lib/arm64/rsi.c
@@ -65,3 +65,47 @@ void arm_rsi_init(void)
/* Set the upper bit of the IPA as the NS_SHARED pte attribute */
prot_ns_shared = (1UL << phys_mask_shift);
}
+
+static unsigned rsi_set_addr_range_state(unsigned long start, unsigned long size,
+ enum ripas_t state, unsigned long *top)
+{
+ struct smccc_result res;
+
+ rsi_invoke(SMC_RSI_IPA_STATE_SET, start, size, state, 0, 0, 0, 0, 0, 0, 0, 0, &res);
+ *top = res.r1;
+ return res.r0;
+}
+
+static void arm_set_memory_state(unsigned long start,
+ unsigned long size,
+ unsigned int ripas)
+{
+ int ret;
+ unsigned long end, top;
+ unsigned long old_start = start;
+
+ if (!is_realm())
+ return;
+
+ start = ALIGN_DOWN(start, RSI_GRANULE_SIZE);
+ if (start != old_start)
+ size += old_start - start;
+ end = ALIGN(start + size, RSI_GRANULE_SIZE);
+ while (start != end) {
+ ret = rsi_set_addr_range_state(start, (end - start),
+ ripas, &top);
+ assert(!ret);
+ assert(top <= end);
+ start = top;
+ }
+}
+
+void arm_set_memory_protected(unsigned long start, unsigned long size)
+{
+ arm_set_memory_state(start, size, RIPAS_RAM);
+}
+
+void arm_set_memory_shared(unsigned long start, unsigned long size)
+{
+ arm_set_memory_state(start, size, RIPAS_EMPTY);
+}
--
2.17.1

2023-01-27 11:58:40

[permalink] [raw]

Subject: [RFC kvm-unit-tests 21/27] arm: Add build steps for QCBOR library

From: Suzuki K Poulose <[email protected]>

The QCBOR library will be used for Realm attestation.

Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
arm/Makefile.arm64 | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 8448af36..8d450de9 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -9,6 +9,8 @@ ldarch = elf64-littleaarch64
arch_LDFLAGS = -pie -n
arch_LDFLAGS += -z notext
CFLAGS += -mstrict-align
+CFLAGS += -I $(SRCDIR)/lib/qcbor/inc
+CFLAGS += -DQCBOR_DISABLE_FLOAT_HW_USE -DQCBOR_DISABLE_PREFERRED_FLOAT -DUSEFULBUF_DISABLE_ALL_FLOAT

mno_outline_atomics := $(call cc-option, -mno-outline-atomics, "")
CFLAGS += $(mno_outline_atomics)
@@ -25,6 +27,7 @@ cflatobjs += lib/arm64/processor.o
cflatobjs += lib/arm64/spinlock.o
cflatobjs += lib/arm64/gic-v3-its.o lib/arm64/gic-v3-its-cmd.o
cflatobjs += lib/arm64/rsi.o
+cflatobjs += lib/qcbor/src/qcbor_decode.o lib/qcbor/src/UsefulBuf.o

OBJDIRS += lib/arm64

@@ -40,4 +43,5 @@ tests += $(TEST_DIR)/realm-sea.flat
include $(SRCDIR)/$(TEST_DIR)/Makefile.common

arch_clean: arm_clean
- $(RM) lib/arm64/.*.d
+ $(RM) lib/arm64/.*.d \
+ lib/qcbor/src/.*.d
--
2.17.1

2023-01-27 11:59:01

[permalink] [raw]

Subject: [RFC kvm-unit-tests 20/27] qcbor: Add QCBOR as a submodule

From: Suzuki K Poulose <[email protected]>

Adds the library QCBOR as submodule. This will be later used
for arm64 realm attestation token parsing. The repository is
available at:

https://github.com/laurencelundblade/QCBOR tag v1.0

Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
.gitmodules | 3 +++
lib/qcbor | 1 +
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 lib/qcbor

diff --git a/.gitmodules b/.gitmodules
new file mode 100644
index 00000000..29fdbc5d
--- /dev/null
+++ b/.gitmodules
@@ -0,0 +1,3 @@
+[submodule "lib/qcbor"]
+ path = lib/qcbor
+ url = https://github.com/laurencelundblade/QCBOR.git
diff --git a/lib/qcbor b/lib/qcbor
new file mode 160000
index 00000000..56b17bf9
--- /dev/null
+++ b/lib/qcbor
@@ -0,0 +1 @@
+Subproject commit 56b17bf9f74096774944bcac0829adcd887d391e
--
2.17.1

2023-01-27 11:59:16

[permalink] [raw]

Subject: [RFC kvm-unit-tests 03/27] arm: realm: Add RSI interface header

Add the defintions for the Realm Service Interface (RSI). RSI calls are a way
for the Realm to communicate with the RMM and request information/services.

Signed-off-by: Joey Gouly <[email protected]>
---
lib/arm64/asm/smc-rsi.h | 139 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 139 insertions(+)
create mode 100644 lib/arm64/asm/smc-rsi.h

diff --git a/lib/arm64/asm/smc-rsi.h b/lib/arm64/asm/smc-rsi.h
new file mode 100644
index 00000000..cd05e9c6
--- /dev/null
+++ b/lib/arm64/asm/smc-rsi.h
@@ -0,0 +1,139 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+#ifndef __SMC_RSI_H_
+#define __SMC_RSI_H_
+
+/*
+ * This file describes the Realm Services Interface (RSI) Application Binary
+ * Interface (ABI) for SMC calls made from within the Realm to the RMM and
+ * serviced by the RMM.
+ */
+
+#define SMC_RSI_CALL_BASE 0xC4000000
+
+/*
+ * The major version number of the RSI implementation. Increase this whenever
+ * the binary format or semantics of the SMC calls change.
+ */
+#define RSI_ABI_VERSION_MAJOR 12
+
+/*
+ * The minor version number of the RSI implementation. Increase this when
+ * a bug is fixed, or a feature is added without breaking binary compatibility.
+ */
+#define RSI_ABI_VERSION_MINOR 0
+
+#define RSI_ABI_VERSION ((RSI_ABI_VERSION_MAJOR << 16) | \
+ RSI_ABI_VERSION_MINOR)
+
+#define RSI_ABI_VERSION_GET_MAJOR(_version) ((_version) >> 16)
+#define RSI_ABI_VERSION_GET_MINOR(_version) ((_version) & 0xFFFF)
+
+#define RSI_SUCCESS 0
+#define RSI_ERROR_INPUT 1
+#define RSI_ERROR_STATE 2
+#define RSI_INCOMPLETE 3
+#define RSI_ERROR_MEMORY 4
+
+#define SMC_RSI_FID(_x) (SMC_RSI_CALL_BASE + (_x))
+
+#define SMC_RSI_ABI_VERSION SMC_RSI_FID(0x190)
+
+/*
+ * arg1 == The IPA of token buffer
+ * arg2 == Challenge value, bytes: 0 - 7
+ * arg3 == Challenge value, bytes: 7 - 15
+ * arg4 == Challenge value, bytes: 16 - 23
+ * arg5 == Challenge value, bytes: 24 - 31
+ * arg6 == Challenge value, bytes: 32 - 39
+ * arg7 == Challenge value, bytes: 40 - 47
+ * arg8 == Challenge value, bytes: 48 - 55
+ * arg9 == Challenge value, bytes: 56 - 63
+ * ret0 == Status / error
+ */
+#define SMC_RSI_ATTEST_TOKEN_INIT SMC_RSI_FID(0x194)
+
+/*
+ * arg1 == The IPA of token buffer
+ * ret0 == Status / error
+ * ret1 == Size of completed token in bytes
+ */
+#define SMC_RSI_ATTEST_TOKEN_CONTINUE SMC_RSI_FID(0x195)
+
+/*
+ * arg1 == Index (1..4), which measurement (REM) to extend
+ * arg2 == Size of realm measurement in bytes, max 64 bytes
+ * arg3 == Measurement value, bytes: 0 - 7
+ * arg4 == Measurement value, bytes: 7 - 15
+ * arg5 == Measurement value, bytes: 16 - 23
+ * arg6 == Measurement value, bytes: 24 - 31
+ * arg7 == Measurement value, bytes: 32 - 39
+ * arg8 == Measurement value, bytes: 40 - 47
+ * arg9 == Measurement value, bytes: 48 - 55
+ * arg10 == Measurement value, bytes: 56 - 63
+ * ret0 == Status / error
+ */
+#define SMC_RSI_MEASUREMENT_EXTEND SMC_RSI_FID(0x193)
+
+/*
+ * arg1 == Index (0..4), which measurement (RIM or REM) to read
+ * ret0 == Status / error
+ * ret1 == Measurement value, bytes: 0 - 7
+ * ret2 == Measurement value, bytes: 7 - 15
+ * ret3 == Measurement value, bytes: 16 - 23
+ * ret4 == Measurement value, bytes: 24 - 31
+ * ret5 == Measurement value, bytes: 32 - 39
+ * ret6 == Measurement value, bytes: 40 - 47
+ * ret7 == Measurement value, bytes: 48 - 55
+ * ret8 == Measurement value, bytes: 56 - 63
+ */
+#define SMC_RSI_MEASUREMENT_READ SMC_RSI_FID(0x192)
+
+#ifndef __ASSEMBLY__
+
+struct rsi_realm_config {
+ union {
+ struct {
+ unsigned long ipa_width; /* Width of IPA in bits */
+ };
+ unsigned char __reserved0[0x1000];
+ };
+ /* Offset 0x1000 */
+};
+
+#endif /* __ASSEMBLY__ */
+
+/*
+ * arg0 == struct rsi_realm_config addr
+ */
+#define SMC_RSI_REALM_CONFIG SMC_RSI_FID(0x196)
+
+/*
+ * arg0 == IPA address of target region
+ * arg1 == size of target region in bytes
+ * arg2 == RIPAS value
+ * ret0 == Status / error
+ * ret1 == Top of modified IPA range
+ */
+#define SMC_RSI_IPA_STATE_SET SMC_RSI_FID(0x197)
+
+#define RSI_HOST_CALL_NR_GPRS 31
+
+#ifndef __ASSEMBLY__
+
+struct rsi_host_call {
+ unsigned int imm;
+ unsigned long gprs[RSI_HOST_CALL_NR_GPRS];
+};
+
+#endif /* __ASSEMBLY__ */
+
+/*
+ * arg0 == struct rsi_host_call addr
+ */
+#define SMC_RSI_HOST_CALL SMC_RSI_FID(0x199)
+
+#endif /* __SMC_RSI_H_ */
--
2.17.1

2023-01-27 12:06:04

[permalink] [raw]

Subject: [RFC kvm-unit-tests 22/27] arm: Add a library to verify tokens using the QCBOR library

From: Mate Toth-Pal <[email protected]>

Add a library wrapper around the QCBOR for parsing the Arm CCA attestation
tokens.

Signed-off-by: Mate Toth-Pal <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
arm/Makefile.arm64 | 7 +-
lib/token_verifier/attest_defines.h | 50 +++
lib/token_verifier/token_dumper.c | 158 ++++++++
lib/token_verifier/token_dumper.h | 15 +
lib/token_verifier/token_verifier.c | 591 ++++++++++++++++++++++++++++
lib/token_verifier/token_verifier.h | 77 ++++
6 files changed, 897 insertions(+), 1 deletion(-)
create mode 100644 lib/token_verifier/attest_defines.h
create mode 100644 lib/token_verifier/token_dumper.c
create mode 100644 lib/token_verifier/token_dumper.h
create mode 100644 lib/token_verifier/token_verifier.c
create mode 100644 lib/token_verifier/token_verifier.h

diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 8d450de9..f57d0a95 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -11,6 +11,7 @@ arch_LDFLAGS += -z notext
CFLAGS += -mstrict-align
CFLAGS += -I $(SRCDIR)/lib/qcbor/inc
CFLAGS += -DQCBOR_DISABLE_FLOAT_HW_USE -DQCBOR_DISABLE_PREFERRED_FLOAT -DUSEFULBUF_DISABLE_ALL_FLOAT
+CFLAGS += -I $(SRCDIR)/lib/token_verifier

mno_outline_atomics := $(call cc-option, -mno-outline-atomics, "")
CFLAGS += $(mno_outline_atomics)
@@ -28,6 +29,9 @@ cflatobjs += lib/arm64/spinlock.o
cflatobjs += lib/arm64/gic-v3-its.o lib/arm64/gic-v3-its-cmd.o
cflatobjs += lib/arm64/rsi.o
cflatobjs += lib/qcbor/src/qcbor_decode.o lib/qcbor/src/UsefulBuf.o
+cflatobjs += lib/token_verifier/token_verifier.o
+cflatobjs += lib/token_verifier/token_dumper.o
+

OBJDIRS += lib/arm64

@@ -44,4 +48,5 @@ include $(SRCDIR)/$(TEST_DIR)/Makefile.common

arch_clean: arm_clean
$(RM) lib/arm64/.*.d \
- lib/qcbor/src/.*.d
+ lib/qcbor/src/.*.d \
+ lib/token_verifier/.*.d
diff --git a/lib/token_verifier/attest_defines.h b/lib/token_verifier/attest_defines.h
new file mode 100644
index 00000000..daf51c5f
--- /dev/null
+++ b/lib/token_verifier/attest_defines.h
@@ -0,0 +1,50 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+
+#ifndef __ATTEST_DEFINES_H__
+#define __ATTEST_DEFINES_H__
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define TAG_COSE_SIGN1 (18)
+#define TAG_CCA_TOKEN (399)
+
+#define CCA_PLAT_TOKEN (44234) /* 0xACCA */
+#define CCA_REALM_DELEGATED_TOKEN (44241)
+
+/* CCA Platform Attestation Token */
+#define CCA_PLAT_CHALLENGE (10) /* EAT nonce */
+#define CCA_PLAT_INSTANCE_ID (256) /* EAT ueid */
+#define CCA_PLAT_PROFILE (265) /* EAT profile */
+#define CCA_PLAT_SECURITY_LIFECYCLE (2395)
+#define CCA_PLAT_IMPLEMENTATION_ID (2396)
+#define CCA_PLAT_SW_COMPONENTS (2399)
+#define CCA_PLAT_VERIFICATION_SERVICE (2400)
+#define CCA_PLAT_CONFIGURATION (2401)
+#define CCA_PLAT_HASH_ALGO_ID (2402)
+
+/* CCA Realm Delegated Attestation Token */
+#define CCA_REALM_CHALLENGE (10) /* EAT nonce */
+#define CCA_REALM_PERSONALIZATION_VALUE (44235)
+#define CCA_REALM_HASH_ALGO_ID (44236)
+#define CCA_REALM_PUB_KEY (44237)
+#define CCA_REALM_INITIAL_MEASUREMENT (44238)
+#define CCA_REALM_EXTENSIBLE_MEASUREMENTS (44239)
+#define CCA_REALM_PUB_KEY_HASH_ALGO_ID (44240)
+
+/* Software components */
+#define CCA_SW_COMP_MEASUREMENT_VALUE (2)
+#define CCA_SW_COMP_VERSION (4)
+#define CCA_SW_COMP_SIGNER_ID (5)
+#define CCA_SW_COMP_HASH_ALGORITHM (6)
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* __ATTEST_DEFINES_H__ */
diff --git a/lib/token_verifier/token_dumper.c b/lib/token_verifier/token_dumper.c
new file mode 100644
index 00000000..15f17956
--- /dev/null
+++ b/lib/token_verifier/token_dumper.c
@@ -0,0 +1,158 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+
+#include <stdio.h>
+#include <inttypes.h>
+#include "attest_defines.h"
+#include "token_dumper.h"
+
+#define COLUMN_WIDTH "20"
+
+void print_raw_token(const char *token, size_t size)
+{
+ int i;
+ char byte;
+
+ printf("\r\nCopy paste token to http://www.cbor.me\r\n");
+ for (i = 0; i < size; ++i) {
+ byte = token[i];
+ if (byte == 0)
+ printf("0x%#02x ", byte);
+ else
+ printf("0x%02x ", byte);
+ if (((i + 1) % 8) == 0)
+ printf("\r\n");
+ }
+ printf("\r\n");
+}
+
+static void print_indent(int indent_level)
+{
+ int i;
+
+ for (i = 0; i < indent_level; ++i) {
+ printf(" ");
+ }
+}
+
+static void print_byte_string(const char *name, int index,
+ struct q_useful_buf_c buf)
+{
+ int i;
+
+ printf("%-"COLUMN_WIDTH"s (#%d) = [", name, index);
+ for (i = 0; i < buf.len; ++i) {
+ printf("%02x", ((uint8_t *)buf.ptr)[i]);
+ }
+ printf("]\r\n");
+}
+
+static void print_text(const char *name, int index, struct q_useful_buf_c buf)
+{
+ int i;
+
+ printf("%-"COLUMN_WIDTH"s (#%d) = \"", name, index);
+ for (i = 0; i < buf.len; ++i) {
+ printf("%c", ((uint8_t *)buf.ptr)[i]);
+ }
+ printf("\"\r\n");
+}
+
+static void print_claim(struct claim_t *claim, int indent_level)
+{
+ print_indent(indent_level);
+ if (claim->present) {
+ switch (claim->type) {
+ case CLAIM_INT64:
+ printf("%-"COLUMN_WIDTH"s (#%" PRId64 ") = %" PRId64
+ "\r\n", claim->title,
+ claim->key, claim->int_data);
+ break;
+ case CLAIM_BOOL:
+ printf("%-"COLUMN_WIDTH"s (#%" PRId64 ") = %s\r\n",
+ claim->title, claim->key,
+ claim->bool_data?"true":"false");
+ break;
+ case CLAIM_BSTR:
+ print_byte_string(claim->title, claim->key,
+ claim->buffer_data);
+ break;
+ case CLAIM_TEXT:
+ print_text(claim->title, claim->key,
+ claim->buffer_data);
+ break;
+ default:
+ printf("* Internal error at %s:%d.\r\n", __FILE__,
+ (int)__LINE__);
+ break;
+ }
+ } else {
+ printf("* Missing%s claim with key: %" PRId64 " (%s)\r\n",
+ claim->mandatory?" mandatory":"",
+ claim->key, claim->title);
+ }
+}
+
+static void print_cose_sign1_wrapper(const char *token_type,
+ struct claim_t *cose_sign1_wrapper)
+{
+ printf("\r\n== %s Token cose header:\r\n", token_type);
+ print_claim(cose_sign1_wrapper + 0, 0);
+ /* Don't print wrapped token bytestring */
+ print_claim(cose_sign1_wrapper + 2, 0);
+ printf("== End of %s Token cose header\r\n\r\n", token_type);
+}
+
+void print_token(struct attestation_claims *claims)
+{
+ int i;
+
+ print_cose_sign1_wrapper("Realm", claims->realm_cose_sign1_wrapper);
+
+ printf("\r\n== Realm Token:\r\n");
+ /* print the claims except the last one. That is printed in detail
+ * below.
+ */
+ for (i = 0; i < CLAIM_COUNT_REALM_TOKEN; ++i) {
+ struct claim_t *claim = claims->realm_token_claims + i;
+
+ print_claim(claim, 0);
+ }
+
+ printf("%-"COLUMN_WIDTH"s (#%d)\r\n", "Realm measurements",
+ CCA_REALM_EXTENSIBLE_MEASUREMENTS);
+ for (i = 0; i < CLAIM_COUNT_REALM_EXTENSIBLE_MEASUREMENTS; ++i) {
+ struct claim_t *claim = claims->realm_measurement_claims + i;
+
+ print_claim(claim, 1);
+ }
+ printf("== End of Realm Token.\r\n");
+
+ print_cose_sign1_wrapper("Platform", claims->plat_cose_sign1_wrapper);
+
+ printf("\r\n== Platform Token:\r\n");
+ for (i = 0; i < CLAIM_COUNT_PLATFORM_TOKEN; ++i) {
+ struct claim_t *claim = claims->plat_token_claims + i;
+
+ print_claim(claim, 0);
+ }
+ printf("== End of Platform Token\r\n\r\n");
+
+ printf("\r\n== Platform Token SW components:\r\n");
+
+ for (i = 0; i < MAX_SW_COMPONENT_COUNT; ++i) {
+ struct sw_component_t *component =
+ claims->sw_component_claims + i;
+
+ if (component->present) {
+ printf(" SW component #%d:\r\n", i);
+ for (int j = 0; j < CLAIM_COUNT_SW_COMPONENT; ++j) {
+ print_claim(component->claims + j, 2);
+ }
+ }
+ }
+ printf("== End of Platform Token SW components\r\n\r\n");
+}
diff --git a/lib/token_verifier/token_dumper.h b/lib/token_verifier/token_dumper.h
new file mode 100644
index 00000000..96cc0744
--- /dev/null
+++ b/lib/token_verifier/token_dumper.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+
+#ifndef __TOKEN_DUMPER_H__
+#define __TOKEN_DUMPER_H__
+
+#include "token_verifier.h"
+
+void print_raw_token(const char *token, size_t size);
+void print_token(struct attestation_claims *claims);
+
+#endif /* __TOKEN_DUMPER_H__ */
diff --git a/lib/token_verifier/token_verifier.c b/lib/token_verifier/token_verifier.c
new file mode 100644
index 00000000..ba2a89f6
--- /dev/null
+++ b/lib/token_verifier/token_verifier.c
@@ -0,0 +1,591 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+
+#include <libcflat.h>
+#include <inttypes.h>
+#include <qcbor/qcbor_decode.h>
+#include <qcbor/qcbor_spiffy_decode.h>
+#include "attest_defines.h"
+#include "token_verifier.h"
+#include "token_dumper.h"
+
+#define SHA256_SIZE 32
+#define SHA512_SIZE 64
+
+#define RETURN_ON_DECODE_ERROR(p_context) \
+ do { \
+ QCBORError ret; \
+ ret = QCBORDecode_GetError(p_context); \
+ if (ret != QCBOR_SUCCESS) { \
+ printf("QCBOR decode failed with error at %s:%d." \
+ " err = %d\r\n", \
+ __FILE__, (int)__LINE__, (int)ret); \
+ return TOKEN_VERIFICATION_ERR_QCBOR(ret); \
+ } \
+ } while (0)
+
+static void init_claim(struct claim_t *claim,
+ bool mandatory, enum claim_data_type type,
+ int64_t key, const char *title, bool present)
+{
+ claim->mandatory = mandatory;
+ claim->type = type;
+ claim->key = key;
+ claim->title = title;
+ claim->present = present;
+}
+
+static int init_cose_wrapper_claim(struct claim_t *cose_sign1_wrapper)
+{
+ struct claim_t *c;
+
+ /* The cose wrapper looks like the following:
+ * - Protected header (bytestring).
+ * - Unprotected header: might contain 0 items. This is a map. Due to
+ * the way this thing is implemented, it is not in the below list,
+ * but is handled in the verify_token_cose_sign1_wrapping
+ * function.
+ * - Payload: Platform token (bytestring). The content is passed for
+ * verify_platform_token.
+ * - Signature.
+ */
+ c = cose_sign1_wrapper;
+ /* This structure is in an array, so the key is not used */
+ init_claim(c++, true, CLAIM_BSTR, 0, "Protected header", false);
+ init_claim(c++, true, CLAIM_BSTR, 0, "Platform token payload", false);
+ init_claim(c++, true, CLAIM_BSTR, 0, "Signature", false);
+ if (c > cose_sign1_wrapper + CLAIM_COUNT_COSE_SIGN1_WRAPPER) {
+ return TOKEN_VERIFICATION_ERR_INIT_ERROR;
+ }
+ return 0;
+}
+
+static int init_claims(struct attestation_claims *attest_claims)
+{
+ int i;
+ int ret;
+ struct claim_t *c;
+ /* TODO: All the buffer overwrite checks are happening too late.
+ * Either remove, or find a better way.
+ */
+ c = attest_claims->realm_token_claims;
+ init_claim(c++, true, CLAIM_BSTR, CCA_REALM_CHALLENGE, "Realm challenge", false);
+ init_claim(c++, true, CLAIM_BSTR, CCA_REALM_PERSONALIZATION_VALUE, "Realm personalization value", false);
+ init_claim(c++, true, CLAIM_TEXT, CCA_REALM_HASH_ALGO_ID, "Realm hash algo id", false);
+ init_claim(c++, true, CLAIM_TEXT, CCA_REALM_PUB_KEY_HASH_ALGO_ID, "Realm public key hash algo id", false);
+ init_claim(c++, true, CLAIM_BSTR, CCA_REALM_PUB_KEY, "Realm signing public key", false);
+ init_claim(c++, true, CLAIM_BSTR, CCA_REALM_INITIAL_MEASUREMENT, "Realm initial measurement", false);
+ /* Realm extensible measurements are not present here as they are
+ * encoded as a CBOR array, and it is handled specially in
+ * verify_realm_token().
+ */
+ if (c > attest_claims->realm_token_claims + CLAIM_COUNT_REALM_TOKEN) {
+ return TOKEN_VERIFICATION_ERR_INIT_ERROR;
+ }
+
+ ret = init_cose_wrapper_claim(attest_claims->realm_cose_sign1_wrapper);
+ if (ret != 0) {
+ return ret;
+ }
+ ret = init_cose_wrapper_claim(attest_claims->plat_cose_sign1_wrapper);
+ if (ret != 0) {
+ return ret;
+ }
+
+ c = attest_claims->plat_token_claims;
+ init_claim(c++, true, CLAIM_BSTR, CCA_PLAT_CHALLENGE, "Challenge", false);
+ init_claim(c++, false, CLAIM_TEXT, CCA_PLAT_VERIFICATION_SERVICE, "Verification service", false);
+ init_claim(c++, true, CLAIM_TEXT, CCA_PLAT_PROFILE, "Profile", false);
+ init_claim(c++, true, CLAIM_BSTR, CCA_PLAT_INSTANCE_ID, "Instance ID", false);
+ init_claim(c++, true, CLAIM_BSTR, CCA_PLAT_IMPLEMENTATION_ID, "Implementation ID", false);
+ init_claim(c++, true, CLAIM_INT64, CCA_PLAT_SECURITY_LIFECYCLE, "Lifecycle", false);
+ init_claim(c++, true, CLAIM_BSTR, CCA_PLAT_CONFIGURATION, "Configuration", false);
+ init_claim(c++, true, CLAIM_TEXT, CCA_PLAT_HASH_ALGO_ID, "Platform hash algo", false);
+ if (c > attest_claims->plat_token_claims +
+ CLAIM_COUNT_PLATFORM_TOKEN) {
+ return TOKEN_VERIFICATION_ERR_INIT_ERROR;
+ }
+
+ for (i = 0; i < CLAIM_COUNT_REALM_EXTENSIBLE_MEASUREMENTS; ++i) {
+ c = attest_claims->realm_measurement_claims + i;
+ init_claim(c, true, CLAIM_BSTR, i,
+ "Realm extensible measurements", false);
+ }
+
+ for (i = 0; i < MAX_SW_COMPONENT_COUNT; ++i) {
+ struct sw_component_t *component =
+ attest_claims->sw_component_claims + i;
+
+ component->present = false;
+ c = component->claims;
+ init_claim(c++, false, CLAIM_TEXT, CCA_SW_COMP_HASH_ALGORITHM, "Hash algo.", false);
+ init_claim(c++, true, CLAIM_BSTR, CCA_SW_COMP_MEASUREMENT_VALUE, "Meas. val.", false);
+ init_claim(c++, false, CLAIM_TEXT, CCA_SW_COMP_VERSION, "Version", false);
+ init_claim(c++, true, CLAIM_BSTR, CCA_SW_COMP_SIGNER_ID, "Signer ID", false);
+ if (c > component->claims + CLAIM_COUNT_SW_COMPONENT) {
+ return TOKEN_VERIFICATION_ERR_INIT_ERROR;
+ }
+ }
+ return TOKEN_VERIFICATION_ERR_SUCCESS;
+}
+
+static int handle_claim_decode_error(const struct claim_t *claim,
+ QCBORError err)
+{
+ if (err == QCBOR_ERR_LABEL_NOT_FOUND) {
+ if (claim->mandatory) {
+ printf("Mandatory claim with key %" PRId64 " (%s) is "
+ "missing from token.\r\n", claim->key,
+ claim->title);
+ return TOKEN_VERIFICATION_ERR_MISSING_MANDATORY_CLAIM;
+ }
+ } else {
+ printf("Decode failed with error at %s:%d. err = %d key = %"
+ PRId64 " (%s).\r\n", __FILE__, (int)__LINE__, err,
+ claim->key, claim->title);
+ return TOKEN_VERIFICATION_ERR_QCBOR(err);
+ }
+ return TOKEN_VERIFICATION_ERR_SUCCESS;
+}
+
+/* Consume claims from a map.
+ *
+ * This function iterates on the array 'claims', and looks up items with the
+ * specified keys. If a claim flagged as mandatory is not found, an error is
+ * returned. The function doesn't checks for extra items. So if the map contains
+ * items with keys that are not in the claims array, no error is reported.
+ *
+ * The map needs to be 'entered' before calling this function, and be 'exited'
+ * after it returns.
+ */
+static int get_claims_from_map(QCBORDecodeContext *p_context,
+ struct claim_t *claims,
+ size_t num_of_claims)
+{
+ QCBORError err;
+ int token_verification_error;
+ int i;
+
+ for (i = 0; i < num_of_claims; ++i) {
+ struct claim_t *claim = claims + i;
+
+ switch (claim->type) {
+ case CLAIM_INT64:
+ QCBORDecode_GetInt64InMapN(p_context, claim->key,
+ &(claim->int_data));
+ break;
+ case CLAIM_BOOL:
+ QCBORDecode_GetBoolInMapN(p_context, claim->key,
+ &(claim->bool_data));
+ break;
+ case CLAIM_BSTR:
+ QCBORDecode_GetByteStringInMapN(p_context, claim->key,
+ &(claim->buffer_data));
+ break;
+ case CLAIM_TEXT:
+ QCBORDecode_GetTextStringInMapN(p_context, claim->key,
+ &(claim->buffer_data));
+ break;
+ default:
+ printf("Internal error at %s:%d.\r\n",
+ __FILE__, (int)__LINE__);
+ return TOKEN_VERIFICATION_ERR_INTERNAL_ERROR;
+ }
+ err = QCBORDecode_GetAndResetError(p_context);
+ if (err == QCBOR_SUCCESS) {
+ claim->present = true;
+ } else {
+ token_verification_error =
+ handle_claim_decode_error(claim, err);
+ if (token_verification_error !=
+ TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return token_verification_error;
+ }
+ }
+ }
+ return TOKEN_VERIFICATION_ERR_SUCCESS;
+}
+
+/* Consume a single claim from an array and from the top level.
+ *
+ * The claim's 'key' and 'mandatory' attribute is not used in this function.
+ * The claim is considered mandatory.
+ */
+static int get_claim(QCBORDecodeContext *p_context, struct claim_t *claim)
+{
+ QCBORError err;
+
+ switch (claim->type) {
+ case CLAIM_INT64:
+ QCBORDecode_GetInt64(p_context, &(claim->int_data));
+ break;
+ case CLAIM_BOOL:
+ QCBORDecode_GetBool(p_context, &(claim->bool_data));
+ break;
+ case CLAIM_BSTR:
+ QCBORDecode_GetByteString(p_context, &(claim->buffer_data));
+ break;
+ case CLAIM_TEXT:
+ QCBORDecode_GetTextString(p_context, &(claim->buffer_data));
+ break;
+ default:
+ printf("Internal error at %s:%d.\r\n",
+ __FILE__, (int)__LINE__);
+ break;
+ }
+ err = QCBORDecode_GetAndResetError(p_context);
+ if (err == QCBOR_SUCCESS) {
+ claim->present = true;
+ return TOKEN_VERIFICATION_ERR_SUCCESS;
+ }
+ printf("Decode failed with error at %s:%d. err = %d claim: \"%s\".\r\n",
+ __FILE__, (int)__LINE__, err, claim->title);
+ return TOKEN_VERIFICATION_ERR_QCBOR(err);
+}
+
+/* Consume claims from an array and from the top level.
+ *
+ * This function iterates on the array 'claims', and gets an item for each
+ * element. If the array or the cbor runs out of elements before reaching the
+ * end of the 'claims' array, then error is returned.
+ *
+ * The claim's 'key' and 'mandatory' attribute is not used in this function.
+ * All the elements considered mandatory.
+ */
+static int get_claims(QCBORDecodeContext *p_context, struct claim_t *claims,
+ size_t num_of_claims)
+{
+ QCBORError err;
+ int i;
+
+ for (i = 0; i < num_of_claims; ++i) {
+ struct claim_t *claim = claims + i;
+
+ err = get_claim(p_context, claim);
+ if (err != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return err;
+ }
+ }
+ return TOKEN_VERIFICATION_ERR_SUCCESS;
+}
+
+static int verify_platform_token(struct q_useful_buf_c buf,
+ struct attestation_claims *attest_claims)
+{
+ QCBORDecodeContext context;
+ int err;
+ int label, index;
+
+ QCBORDecode_Init(&context, buf, QCBOR_DECODE_MODE_NORMAL);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ QCBORDecode_EnterMap(&context, NULL);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ err = get_claims_from_map(&context,
+ attest_claims->plat_token_claims,
+ CLAIM_COUNT_PLATFORM_TOKEN);
+ if (err != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return err;
+ }
+
+ label = CCA_PLAT_SW_COMPONENTS;
+ QCBORDecode_EnterArrayFromMapN(&context, label);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ index = 0;
+ while (1) {
+ QCBORDecode_EnterMap(&context, NULL);
+ if (QCBORDecode_GetError(&context) == QCBOR_ERR_NO_MORE_ITEMS) {
+ /* This is OK. We just reached the end of the array.
+ * Break from the loop.
+ */
+ break;
+ }
+
+ if (index >= MAX_SW_COMPONENT_COUNT) {
+ printf("Not enough slots in sw_component_claims.\r\n");
+ printf("Increase MAX_SW_COMPONENT_COUNT in %s.\r\n",
+ __FILE__);
+ return TOKEN_VERIFICATION_ERR_INTERNAL_ERROR;
+ }
+
+ err = get_claims_from_map(&context,
+ attest_claims->sw_component_claims[index].claims,
+ CLAIM_COUNT_SW_COMPONENT);
+ if (err != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return err;
+ }
+ attest_claims->sw_component_claims[index].present = true;
+
+ QCBORDecode_ExitMap(&context);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ ++index;
+ }
+ /* We only get here if the decode error code was a
+ * QCBOR_ERR_NO_MORE_ITEMS which is expected when the end of an array is
+ * reached. In this case the processing must be continued, so clear the
+ * error.
+ */
+ QCBORDecode_GetAndResetError(&context);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ QCBORDecode_ExitArray(&context);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ QCBORDecode_ExitMap(&context);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ QCBORDecode_Finish(&context);
+
+ return TOKEN_VERIFICATION_ERR_SUCCESS;
+}
+
+static bool verify_length_of_measurement(size_t len)
+{
+ size_t allowed_lengths[] = {SHA256_SIZE, SHA512_SIZE};
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(allowed_lengths); ++i) {
+ if (len == allowed_lengths[i])
+ return true;
+ }
+
+ return false;
+}
+
+static int verify_realm_token(struct q_useful_buf_c buf,
+ struct attestation_claims *attest_claims)
+{
+ QCBORDecodeContext context;
+ int err;
+ int i;
+
+ QCBORDecode_Init(&context, buf, QCBOR_DECODE_MODE_NORMAL);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ QCBORDecode_EnterMap(&context, NULL);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ err = get_claims_from_map(&context, attest_claims->realm_token_claims,
+ CLAIM_COUNT_REALM_TOKEN);
+ if (err != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return err;
+ }
+
+ /* Now get the realm extensible measurements */
+ QCBORDecode_EnterArrayFromMapN(&context,
+ CCA_REALM_EXTENSIBLE_MEASUREMENTS);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ err = get_claims(&context,
+ attest_claims->realm_measurement_claims,
+ CLAIM_COUNT_REALM_EXTENSIBLE_MEASUREMENTS);
+ if (err != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return err;
+ }
+
+ for (i = 0; i < CLAIM_COUNT_REALM_EXTENSIBLE_MEASUREMENTS; ++i) {
+ struct claim_t *claims =
+ attest_claims->realm_measurement_claims;
+ struct q_useful_buf_c buf = claims[i].buffer_data;
+
+ if (!verify_length_of_measurement(buf.len)) {
+ return TOKEN_VERIFICATION_ERR_INVALID_CLAIM_LEN;
+ }
+ }
+
+ QCBORDecode_ExitArray(&context);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ QCBORDecode_ExitMap(&context);
+ QCBORDecode_Finish(&context);
+
+ return TOKEN_VERIFICATION_ERR_SUCCESS;
+}
+
+/* Returns a pointer to the wrapped token in: 'token_payload'.
+ * Returns the claims in the wrapper in cose_sign1_wrapper.
+ */
+static int verify_token_cose_sign1_wrapping(
+ struct q_useful_buf_c token,
+ struct q_useful_buf_c *token_payload,
+ struct claim_t *cose_sign1_wrapper)
+{
+ QCBORDecodeContext context;
+ QCBORItem item;
+ int err;
+
+ QCBORDecode_Init(&context, token, QCBOR_DECODE_MODE_NORMAL);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ /* Check COSE tag. */
+ QCBORDecode_PeekNext(&context, &item);
+ if (!QCBORDecode_IsTagged(&context, &item,
+ TAG_COSE_SIGN1)) {
+ return TOKEN_VERIFICATION_ERR_INVALID_COSE_TAG;
+ }
+
+ QCBORDecode_EnterArray(&context, NULL);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ /* Protected header */
+ err = get_claim(&context, cose_sign1_wrapper);
+ if (err != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return err;
+ }
+
+ /* Unprotected header. The map is always present, but may contain 0
+ * items.
+ */
+ QCBORDecode_EnterMap(&context, NULL);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ /* Skip the content for now. */
+
+ QCBORDecode_ExitMap(&context);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ /* Payload */
+ err = get_claim(&context, cose_sign1_wrapper + 1);
+ if (err != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return err;
+ }
+
+ /* Signature */
+ err = get_claim(&context, cose_sign1_wrapper + 2);
+ if (err != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return err;
+ }
+
+ QCBORDecode_ExitArray(&context);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ *token_payload = cose_sign1_wrapper[1].buffer_data;
+
+ return TOKEN_VERIFICATION_ERR_SUCCESS;
+}
+
+static int verify_cca_token(struct q_useful_buf_c token,
+ struct q_useful_buf_c *platform_token,
+ struct q_useful_buf_c *realm_token)
+{
+ QCBORDecodeContext context;
+ QCBORItem item;
+ QCBORError err;
+
+ QCBORDecode_Init(&context, token, QCBOR_DECODE_MODE_NORMAL);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ /* ================== Check CCA_TOKEN tag =========================== */
+ QCBORDecode_PeekNext(&context, &item);
+ if (!QCBORDecode_IsTagged(&context, &item, TAG_CCA_TOKEN)) {
+ return TOKEN_VERIFICATION_ERR_INVALID_COSE_TAG;
+ }
+
+ /* ================== Get the the platform token ==================== */
+ QCBORDecode_EnterMap(&context, NULL);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ /*
+ * First element is the CCA platfrom token which is a
+ * COSE_Sign1_Tagged object. It has byte stream wrapper.
+ */
+ QCBORDecode_GetByteStringInMapN(&context, CCA_PLAT_TOKEN,
+ platform_token);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ /* ================== Get the the realm token ======================= */
+ /*
+ * Second element is the delegated realm token which is a
+ * COSE_Sign1_Tagged object. It has byte stream wrapper.
+ */
+ QCBORDecode_GetByteStringInMapN(&context, CCA_REALM_DELEGATED_TOKEN,
+ realm_token);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ QCBORDecode_ExitMap(&context);
+ RETURN_ON_DECODE_ERROR(&context);
+
+ /* Finishing up the decoding of the top-level wrapper */
+ err = QCBORDecode_Finish(&context);
+ if (err != QCBOR_SUCCESS) {
+ printf("QCBOR decode failed with error at %s:%d. err = %d\r\n",
+ __FILE__, (int)__LINE__, (int)err);
+ return TOKEN_VERIFICATION_ERR_QCBOR(err);
+ }
+
+ return TOKEN_VERIFICATION_ERR_SUCCESS;
+}
+
+/*
+ * This function expect two COSE_Sing1_Tagged object wrapped with a tagged map:
+ *
+ * cca-token = #6.44234(cca-token-map) ; 44234 = 0xACCA
+ *
+ * cca-platform-token = COSE_Sign1_Tagged
+ * cca-realm-delegated-token = COSE_Sign1_Tagged
+ *
+ * cca-token-map = {
+ * 0 => cca-platform-token
+ * 1 => cca-realm-delegated-token
+ * }
+ *
+ * COSE_Sign1_Tagged = #6.18(COSE_Sign1)
+ */
+int verify_token(const char *token, size_t size,
+ struct attestation_claims *attest_claims)
+{
+ /* TODO: do signature check */
+ /* TODO: Add tag check on tokens */
+ struct q_useful_buf_c buf = {token, size};
+ int ret;
+ struct q_useful_buf_c realm_token;
+ struct q_useful_buf_c realm_token_payload;
+ struct q_useful_buf_c platform_token;
+ struct q_useful_buf_c platform_token_payload;
+
+ ret = init_claims(attest_claims);
+ if (ret != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return ret;
+ }
+
+ /* Verify top-level token map and extract the two sub-tokens */
+ ret = verify_cca_token(buf, &platform_token, &realm_token);
+ if (ret != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return ret;
+ }
+
+ /* Verify the COSE_Sign1 wrapper of the realm token */
+ ret = verify_token_cose_sign1_wrapping(realm_token,
+ &realm_token_payload,
+ attest_claims->realm_cose_sign1_wrapper);
+ if (ret != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return ret;
+ }
+ /* Verify the payload of the realm token */
+ ret = verify_realm_token(realm_token_payload, attest_claims);
+ if (ret != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return ret;
+ }
+
+ /* Verify the COSE_Sign1 wrapper of the platform token */
+ ret = verify_token_cose_sign1_wrapping(platform_token,
+ &platform_token_payload,
+ attest_claims->plat_cose_sign1_wrapper);
+ if (ret != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return ret;
+ }
+ /* Verify the payload of the platform token */
+ ret = verify_platform_token(platform_token_payload, attest_claims);
+ if (ret != TOKEN_VERIFICATION_ERR_SUCCESS) {
+ return ret;
+ }
+
+ return TOKEN_VERIFICATION_ERR_SUCCESS;
+}
+
diff --git a/lib/token_verifier/token_verifier.h b/lib/token_verifier/token_verifier.h
new file mode 100644
index 00000000..ec3ab9c9
--- /dev/null
+++ b/lib/token_verifier/token_verifier.h
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+
+#ifndef __TOKEN_VERIFIER_H__
+#define __TOKEN_VERIFIER_H__
+
+#include <qcbor/qcbor_decode.h>
+
+#define TOKEN_VERIFICATION_ERR_SUCCESS 0
+#define TOKEN_VERIFICATION_ERR_INIT_ERROR 1
+#define TOKEN_VERIFICATION_ERR_MISSING_MANDATORY_CLAIM 2
+#define TOKEN_VERIFICATION_ERR_INVALID_COSE_TAG 3
+#define TOKEN_VERIFICATION_ERR_INVALID_CLAIM_LEN 4
+#define TOKEN_VERIFICATION_ERR_INTERNAL_ERROR 5
+#define TOKEN_VERIFICATION_ERR_QCBOR(qcbor_err) (1000 + qcbor_err)
+
+/* Number of realm extensible measurements (REM) */
+#define REM_COUNT 4
+
+#define MAX_SW_COMPONENT_COUNT 16
+
+#define CLAIM_COUNT_REALM_TOKEN 6
+#define CLAIM_COUNT_COSE_SIGN1_WRAPPER 3
+#define CLAIM_COUNT_PLATFORM_TOKEN 8
+#define CLAIM_COUNT_REALM_EXTENSIBLE_MEASUREMENTS REM_COUNT
+#define CLAIM_COUNT_SW_COMPONENT 4
+
+/* This tells how the data should be interpreted in the claim_t struct, and not
+ * necessarily is the same as the item's major type in the token.
+ */
+enum claim_data_type {
+ CLAIM_INT64,
+ CLAIM_BOOL,
+ CLAIM_BSTR,
+ CLAIM_TEXT,
+};
+
+struct claim_t {
+ /* 'static' */
+ bool mandatory;
+ enum claim_data_type type;
+ int64_t key;
+ const char *title;
+
+ /* filled during verification */
+ bool present;
+ union {
+ int64_t int_data;
+ bool bool_data;
+ /* Used for text and bytestream as well */
+ /* TODO: Add expected length check as well? */
+ struct q_useful_buf_c buffer_data;
+ };
+};
+
+struct sw_component_t {
+ bool present;
+ struct claim_t claims[CLAIM_COUNT_SW_COMPONENT];
+};
+
+struct attestation_claims {
+ struct claim_t realm_cose_sign1_wrapper[CLAIM_COUNT_COSE_SIGN1_WRAPPER];
+ struct claim_t realm_token_claims[CLAIM_COUNT_REALM_TOKEN];
+ struct claim_t realm_measurement_claims[CLAIM_COUNT_REALM_EXTENSIBLE_MEASUREMENTS];
+ struct claim_t plat_cose_sign1_wrapper[CLAIM_COUNT_COSE_SIGN1_WRAPPER];
+ struct claim_t plat_token_claims[CLAIM_COUNT_PLATFORM_TOKEN];
+ struct sw_component_t sw_component_claims[MAX_SW_COMPONENT_COUNT];
+};
+
+/* Returns TOKEN_VERIFICATION_ERR* */
+int verify_token(const char *token, size_t size,
+ struct attestation_claims *attest_claims);
+
+#endif /* __TOKEN_VERIFIER_H__ */
--
2.17.1

2023-01-27 12:10:12

[permalink] [raw]

Subject: [RFC kvm-unit-tests 08/27] arm: realm: Realm initialisation

During the boot, run a check for the presence of RMM. If we are Realm,
detect the Realm configuration using RSI and initialise the key parameters.

Also expose a helper to indicate if this is running inside a Realm

Co-developed-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
arm/Makefile.arm64 | 1 +
lib/arm/asm/rsi.h | 16 ++++++++++
lib/arm/setup.c | 3 ++
lib/arm64/asm/processor.h | 8 +++++
lib/arm64/asm/rsi.h | 36 +++++++++++++++++++++
lib/arm64/rsi.c | 67 +++++++++++++++++++++++++++++++++++++++
6 files changed, 131 insertions(+)
create mode 100644 lib/arm/asm/rsi.h
create mode 100644 lib/arm64/asm/rsi.h
create mode 100644 lib/arm64/rsi.c

diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 42e18e77..ab557f84 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -24,6 +24,7 @@ cstart.o = $(TEST_DIR)/cstart64.o
cflatobjs += lib/arm64/processor.o
cflatobjs += lib/arm64/spinlock.o
cflatobjs += lib/arm64/gic-v3-its.o lib/arm64/gic-v3-its-cmd.o
+cflatobjs += lib/arm64/rsi.o

OBJDIRS += lib/arm64

diff --git a/lib/arm/asm/rsi.h b/lib/arm/asm/rsi.h
new file mode 100644
index 00000000..d1f72c25
--- /dev/null
+++ b/lib/arm/asm/rsi.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+#ifndef __ASMARM_RSI_H_
+#define __ASMARM_RSI_H_
+
+#include <stdbool.h>
+
+static inline bool is_realm(void)
+{
+ return false;
+}
+
+#endif /* __ASMARM_RSI_H_ */
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 65d98e97..36d4d826 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -24,6 +24,7 @@
#include <asm/page.h>
#include <asm/pgtable.h>
#include <asm/processor.h>
+#include <asm/rsi.h>
#include <asm/smp.h>
#include <asm/timer.h>
#include <asm/psci.h>
@@ -244,6 +245,8 @@ void setup(const void *fdt, phys_addr_t freemem_start)
u32 fdt_size;
int ret;

+ arm_rsi_init();
+
assert(sizeof(long) == 8 || freemem_start < (3ul << 30));
freemem = (void *)(unsigned long)freemem_start;

diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 1c73ba32..320ebaef 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -114,6 +114,14 @@ static inline unsigned long get_id_aa64mmfr0_el1(void)
#define ID_AA64MMFR0_TGRAN64_SUPPORTED 0x0
#define ID_AA64MMFR0_TGRAN16_SUPPORTED 0x1

+static inline unsigned long get_id_aa64pfr0_el1(void)
+{
+ return read_sysreg(id_aa64pfr0_el1);
+}
+
+#define ID_AA64PFR0_EL1_EL3 (0xf << 12)
+#define ID_AA64PFR0_EL1_EL3_NI (0x0 << 12)
+
static inline bool system_supports_granule(size_t granule)
{
u32 shift;
diff --git a/lib/arm64/asm/rsi.h b/lib/arm64/asm/rsi.h
new file mode 100644
index 00000000..8b9b91b2
--- /dev/null
+++ b/lib/arm64/asm/rsi.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+#ifndef __ASMARM64_RSI_H_
+#define __ASMARM64_RSI_H_
+
+#include <stdbool.h>
+
+#include <asm/arm-smccc.h>
+#include <asm/io.h>
+#include <asm/smc-rsi.h>
+
+#define RSI_GRANULE_SIZE SZ_4K
+
+extern bool rsi_present;
+
+void arm_rsi_init(void);
+
+int rsi_invoke(unsigned int function_id, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2,
+ unsigned long arg3, unsigned long arg4,
+ unsigned long arg5, unsigned long arg6,
+ unsigned long arg7, unsigned long arg8,
+ unsigned long arg9, unsigned long arg10,
+ struct smccc_result *result);
+
+int rsi_get_version(void);
+
+static inline bool is_realm(void)
+{
+ return rsi_present;
+}
+
+#endif /* __ASMARM64_RSI_H_ */
diff --git a/lib/arm64/rsi.c b/lib/arm64/rsi.c
new file mode 100644
index 00000000..23a4e963
--- /dev/null
+++ b/lib/arm64/rsi.c
@@ -0,0 +1,67 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+#include <libcflat.h>
+
+#include <asm/pgtable.h>
+#include <asm/processor.h>
+#include <asm/rsi.h>
+
+bool rsi_present;
+
+int rsi_invoke(unsigned int function_id, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2,
+ unsigned long arg3, unsigned long arg4,
+ unsigned long arg5, unsigned long arg6,
+ unsigned long arg7, unsigned long arg8,
+ unsigned long arg9, unsigned long arg10,
+ struct smccc_result *result)
+{
+ return arm_smccc_smc(function_id, arg0, arg1, arg2, arg3, arg4, arg5,
+ arg6, arg7, arg8, arg9, arg10, result);
+}
+
+struct rsi_realm_config __attribute__((aligned(RSI_GRANULE_SIZE))) config;
+
+static unsigned long rsi_get_realm_config(struct rsi_realm_config *cfg)
+{
+ struct smccc_result res;
+
+ rsi_invoke(SMC_RSI_REALM_CONFIG, __virt_to_phys((unsigned long)cfg),
+ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &res);
+
+ return res.r0;
+}
+
+int rsi_get_version(void)
+{
+ struct smccc_result res = {};
+ int ret;
+
+ if ((get_id_aa64pfr0_el1() & ID_AA64PFR0_EL1_EL3) == ID_AA64PFR0_EL1_EL3_NI)
+ return -1;
+
+ ret = rsi_invoke(SMC_RSI_ABI_VERSION, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ &res);
+ if (ret)
+ return ret;
+
+ return res.r0;
+}
+
+void arm_rsi_init(void)
+{
+ if (rsi_get_version() != RSI_ABI_VERSION)
+ return;
+
+ if (rsi_get_realm_config(&config))
+ return;
+
+ rsi_present = true;
+
+ phys_mask_shift = (config.ipa_width - 1);
+ /* Set the upper bit of the IPA as the NS_SHARED pte attribute */
+ prot_ns_shared = (1UL << phys_mask_shift);
+}
--
2.17.1

2023-01-27 12:11:16

[permalink] [raw]

Subject: [RFC kvm-unit-tests 10/27] arm: realm: Set RIPAS state for RAM

From: Suzuki K Poulose <[email protected]>

A Realm must ensure that the "RAM" region is set to RIPAS_RAM, before any
access is made. This patch makes sure that all memory blocks are marked as
RIPAS_RAM. Also, before we relocate the "FDT" and "initrd", make sure the
target location is marked too. This happens before we parse the memory blocks.

It is OK to do this operation on a given IPA multiple times. So, we don't
exclude the inital image areas from the "target" list.

Also, this operation doesn't require the host to commit physical memory to back
the IPAs yet. It can be done on demand via fault handling.

Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
lib/arm/setup.c | 7 +++++++
1 file changed, 7 insertions(+)

diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 36d4d826..7b3dc0b8 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -181,6 +181,7 @@ static void mem_init(phys_addr_t freemem_start)
while (r && r->end != mem.end)
r = mem_region_find(r->end);
assert(r);
+ arm_set_memory_protected(r->start, r->end - r->start);

/* Ensure our selected freemem range is somewhere in our full range */
assert(freemem_start >= mem.start && freemem->end <= mem.end);
@@ -252,6 +253,11 @@ void setup(const void *fdt, phys_addr_t freemem_start)

/* Move the FDT to the base of free memory */
fdt_size = fdt_totalsize(fdt);
+ /*
+ * Before we touch the memory @freemem, make sure it
+ * is set to protected for Realms.
+ */
+ arm_set_memory_protected((unsigned long)freemem, fdt_size);
ret = fdt_move(fdt, freemem, fdt_size);
assert(ret == 0);
ret = dt_init(freemem);
@@ -263,6 +269,7 @@ void setup(const void *fdt, phys_addr_t freemem_start)
assert(ret == 0 || ret == -FDT_ERR_NOTFOUND);
if (ret == 0) {
initrd = freemem;
+ arm_set_memory_protected((unsigned long)initrd, initrd_size);
memmove(initrd, tmp, initrd_size);
freemem += initrd_size;
}
--
2.17.1

2023-01-27 12:12:29

[permalink] [raw]

Subject: [RFC kvm-unit-tests 16/27] arm: realm: Add tests for in realm SEA

From: Djordje Kovacevic <[email protected]>

The RMM/Host could inject Synchronous External Aborts in to the Realm
for various reasons.

RMM injects the SEA for :
* Instruction/Data fetch from an IPA that is in RIPAS_EMPTY state
* Instruction fetch from an Unprotected IPA.

Trigger these conditions from within the Realm and verify that the
SEAs are received.

Signed-off-by: Djordje Kovacevic <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Joey Gouly <[email protected]>
---
arm/Makefile.arm64 | 1 +
arm/realm-sea.c | 143 +++++++++++++++++++++++++++++++++++++++++++++
arm/unittests.cfg | 6 ++
3 files changed, 150 insertions(+)
create mode 100644 arm/realm-sea.c

diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 90ec6815..8448af36 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -35,6 +35,7 @@ tests += $(TEST_DIR)/cache.flat
tests += $(TEST_DIR)/debug.flat
tests += $(TEST_DIR)/realm-rsi.flat
tests += $(TEST_DIR)/realm-fpu.flat
+tests += $(TEST_DIR)/realm-sea.flat

include $(SRCDIR)/$(TEST_DIR)/Makefile.common

diff --git a/arm/realm-sea.c b/arm/realm-sea.c
new file mode 100644
index 00000000..5ef3e2a4
--- /dev/null
+++ b/arm/realm-sea.c
@@ -0,0 +1,143 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Arm Limited.
+ * All rights reserved.
+ */
+#include <libcflat.h>
+#include <vmalloc.h>
+#include <asm/ptrace.h>
+#include <asm/thread_info.h>
+#include <asm/mmu.h>
+#include <asm/rsi.h>
+#include <linux/compiler.h>
+#include <alloc_page.h>
+#include <asm/pgtable.h>
+
+typedef void (*empty_fn)(void);
+
+static bool test_passed;
+
+/*
+ * The virtual address of the page that the test has made the access to
+ * in order to cause the I/DAbort with I/DFSC = Synchronous External Abort.
+ */
+static void* target_page_va;
+
+/*
+ * Ensure that the @va is the executable location from EL1:
+ * - SCTLR_EL1.WXN must be off.
+ * - Disable the access from EL0 (controlled by AP[1] in PTE).
+ */
+static void enable_instruction_fetch(void* va)
+{
+ unsigned long sctlr = read_sysreg(sctlr_el1);
+ if (sctlr & SCTLR_EL1_WXN) {
+ sctlr &= ~SCTLR_EL1_WXN;
+ write_sysreg(sctlr, sctlr_el1);
+ isb();
+ flush_tlb_all();
+ }
+
+ mmu_clear_user(current_thread_info()->pgtable, (u64)va);
+}
+
+static void data_abort_handler(struct pt_regs *regs, unsigned int esr)
+{
+ if ((esr & ESR_EL1_FSC_MASK) == ESR_EL1_FSC_EXTABT)
+ test_passed = true;
+
+ report_info("esr = %x", esr);
+ /*
+ * Advance the PC to complete the test.
+ */
+ regs->pc += 4;
+}
+
+static void data_access_to_empty(void)
+{
+ test_passed = false;
+ target_page_va = alloc_page();
+ phys_addr_t empty_ipa = virt_to_phys(target_page_va);
+
+ arm_set_memory_shared(empty_ipa, SZ_4K);
+
+ install_exception_handler(EL1H_SYNC, ESR_EL1_EC_DABT_EL1, data_abort_handler);
+ READ_ONCE(((char*)target_page_va)[0x55]);
+ install_exception_handler(EL1H_SYNC, ESR_EL1_EC_DABT_EL1, NULL);
+
+ report(test_passed, " ");
+}
+
+static void instruction_abort_handler(struct pt_regs *regs, unsigned int esr)
+{
+ if (((esr & ESR_EL1_FSC_MASK) == ESR_EL1_FSC_EXTABT) &&
+ (regs->pc == (u64)target_page_va))
+ test_passed = true;
+
+ report_info("esr = %x", esr);
+ /*
+ * Simulate the RET instruction to complete the test.
+ */
+ regs->pc = regs->regs[30];
+}
+
+static void instr_fetch_from_empty(void)
+{
+ phys_addr_t empty_ipa;
+
+ test_passed = false;
+ target_page_va = alloc_page();
+ enable_instruction_fetch(target_page_va);
+
+ empty_ipa = virt_to_phys((void*)target_page_va);
+
+ arm_set_memory_shared(empty_ipa, SZ_4K);
+
+ install_exception_handler(EL1H_SYNC, ESR_EL1_EC_IABT_EL1, instruction_abort_handler);
+ /*
+ * This should cause the IAbort with IFSC = SEA
+ */
+ ((empty_fn)target_page_va)();
+ install_exception_handler(EL1H_SYNC, ESR_EL1_EC_IABT_EL1, NULL);
+
+ report(test_passed, " ");
+}
+
+static void instr_fetch_from_unprotected(void)
+{
+ test_passed = false;
+ /*
+ * The test will attempt to execute an instruction from the start of
+ * the unprotected IPA space.
+ */
+ target_page_va = vmap(PTE_NS_SHARED, SZ_4K);
+ enable_instruction_fetch(target_page_va);
+
+ install_exception_handler(EL1H_SYNC, ESR_EL1_EC_IABT_EL1, instruction_abort_handler);
+ /*
+ * This should cause the IAbort with IFSC = SEA
+ */
+ ((empty_fn)target_page_va)();
+ install_exception_handler(EL1H_SYNC, ESR_EL1_EC_IABT_EL1, NULL);
+
+ report(test_passed, " ");
+}
+
+int main(int argc, char **argv)
+{
+ report_prefix_push("in_realm_sea");
+
+ report_prefix_push("data_access_to_empty");
+ data_access_to_empty();
+ report_prefix_pop();
+
+ report_prefix_push("instr_fetch_from_empty");
+ instr_fetch_from_empty();
+ report_prefix_pop();
+
+ report_prefix_push("instr_fetch_from_unprotected");
+ instr_fetch_from_unprotected();
+ report_prefix_pop();
+
+ return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index a60dc6a9..bc2354c7 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -305,3 +305,9 @@ smp = 2
groups = nodefault realms
accel = kvm
arch = arm64
+
+[realm-sea]
+file = realm-sea.flat
+groups = nodefault realms
+accel = kvm
+arch = arm64
--
2.17.1

2023-01-27 12:16:10

[permalink] [raw]

Subject: [RFC kvmtool 21/31] init: Add last_{init, exit} list macros

From: Alexandru Elisei <[email protected]>

Add a last_init macro for constructor functions that will be executed last
in the initialization process. Add a symmetrical macro, last_exit, for
destructor functions that will be the last to be executed when kvmtool
exits.

The list priority for the late_{init, exit} macros has been bumped down a
spot, but their relative priority remains unchanged, to keep the same size
for the init_lists and exit_lists.

Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
include/kvm/util-init.h | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/include/kvm/util-init.h b/include/kvm/util-init.h
index 13d4f04d..e6a0e169 100644
--- a/include/kvm/util-init.h
+++ b/include/kvm/util-init.h
@@ -39,7 +39,8 @@ static void __attribute__ ((constructor)) __init__##cb(void) \
#define dev_init(cb) __init_list_add(cb, 5)
#define virtio_dev_init(cb) __init_list_add(cb, 6)
#define firmware_init(cb) __init_list_add(cb, 7)
-#define late_init(cb) __init_list_add(cb, 9)
+#define late_init(cb) __init_list_add(cb, 8)
+#define last_init(cb) __init_list_add(cb, 9)

#define core_exit(cb) __exit_list_add(cb, 0)
#define base_exit(cb) __exit_list_add(cb, 2)
@@ -47,5 +48,6 @@ static void __attribute__ ((constructor)) __init__##cb(void) \
#define dev_exit(cb) __exit_list_add(cb, 5)
#define virtio_dev_exit(cb) __exit_list_add(cb, 6)
#define firmware_exit(cb) __exit_list_add(cb, 7)
-#define late_exit(cb) __exit_list_add(cb, 9)
+#define late_exit(cb) __exit_list_add(cb, 8)
+#define last_exit(cb) __exit_list_add(cb, 9)
#endif
--
2.34.1

2023-01-27 12:16:40

[permalink] [raw]

Subject: [RFC kvmtool 22/31] arm64: Activate realm before the first VCPU is run

From: Alexandru Elisei <[email protected]>

Before KVM can run a VCPU belong to a realm, the realm be activated.
Activating a realm prevents the adding of new object and seals the
cryptographic measurement of that realm. The VCPU state is part of the
measurement, which means that realm activation must be performed after
all VCPUs have been reset.

Current RMM implementation can only create RECs in the order of their
MPIDRs. VCPUs get assigned MPIDRs by KVM based on their VCPU id. Reset the
VCPUs in the order they were created from the main thread instead of doing
it from their own thread, which doesn't guarantee any ordering.

Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/kvm-cpu.c | 4 ++++
arm/aarch64/realm.c | 35 +++++++++++++++++++++++++++++++
arm/include/arm-common/kvm-arch.h | 1 +
3 files changed, 40 insertions(+)

diff --git a/arm/aarch64/kvm-cpu.c b/arm/aarch64/kvm-cpu.c
index 24e570c4..32fa7609 100644
--- a/arm/aarch64/kvm-cpu.c
+++ b/arm/aarch64/kvm-cpu.c
@@ -187,6 +187,10 @@ void kvm_cpu__reset_vcpu(struct kvm_cpu *vcpu)
cpu_set_t *affinity;
int ret;

+ /* VCPU reset is done before activating the realm. */
+ if (kvm->arch.realm_is_active)
+ return;
+
affinity = kvm->arch.vcpu_affinity_cpuset;
if (affinity) {
ret = sched_setaffinity(0, sizeof(cpu_set_t), affinity);
diff --git a/arm/aarch64/realm.c b/arm/aarch64/realm.c
index eddccece..808d39c5 100644
--- a/arm/aarch64/realm.c
+++ b/arm/aarch64/realm.c
@@ -1,4 +1,5 @@
#include "kvm/kvm.h"
+#include "kvm/kvm-cpu.h"

#include <linux/byteorder.h>
#include <asm/image.h>
@@ -192,3 +193,37 @@ void kvm_arm_realm_populate_dtb(struct kvm *kvm)
if (end > start)
realm_populate(kvm, start, end - start);
}
+
+static void kvm_arm_realm_activate_realm(struct kvm *kvm)
+{
+ struct kvm_enable_cap activate_realm = {
+ .cap = KVM_CAP_ARM_RME,
+ .args[0] = KVM_CAP_ARM_RME_ACTIVATE_REALM,
+ };
+
+ if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &activate_realm) < 0)
+ die_perror("KVM_CAP_ARM_RME(KVM_CAP_ARM_RME_ACTIVATE_REALM)");
+
+ kvm->arch.realm_is_active = true;
+}
+
+static int kvm_arm_realm_finalize(struct kvm *kvm)
+{
+ int i;
+
+ if (!kvm->cfg.arch.is_realm)
+ return 0;
+
+ /*
+ * VCPU reset must happen before the realm is activated, because their
+ * state is part of the cryptographic measurement for the realm.
+ */
+ for (i = 0; i < kvm->nrcpus; i++)
+ kvm_cpu__reset_vcpu(kvm->cpus[i]);
+
+ /* Activate and seal the measurement for the realm. */
+ kvm_arm_realm_activate_realm(kvm);
+
+ return 0;
+}
+last_init(kvm_arm_realm_finalize)
diff --git a/arm/include/arm-common/kvm-arch.h b/arm/include/arm-common/kvm-arch.h
index b5a4b851..6d48e13c 100644
--- a/arm/include/arm-common/kvm-arch.h
+++ b/arm/include/arm-common/kvm-arch.h
@@ -116,6 +116,7 @@ struct kvm_arch {
cpu_set_t *vcpu_affinity_cpuset;
u64 measurement_algo;
u64 sve_vq;
+ bool realm_is_active;
};

#endif /* ARM_COMMON__KVM_ARCH_H */
--
2.34.1

2023-01-27 12:20:30

[permalink] [raw]

Subject: [RFC kvmtool 18/31] arm64: Populate initial realm contents

From: Alexandru Elisei <[email protected]>

Populate the realm memory with the initial contents, which include
the device tree blob, the kernel image, and initrd, if specified,
or the firmware image.

Populating an image in the realm involves two steps:
a) Mark the IPA area as RAM - INIT_IPA_REALM
b) Load the contents into the IPA - POPULATE_REALM

Wherever we know the actual size of an image in memory, we make
sure the "memory area" is initialised to RAM.
e.g., Linux kernel image size from the header which includes the bss etc.
The "file size" on disk for the Linux image is much smaller.
We mark the region of size Image.header.size as RAM (a), from the kernel
load address. And load the Image file into the memory (b) above.
At the moment we only detect the Arm64 Linux Image header format.

Since we're already touching the code that copies the
initrd in guest memory, let's do a bit of cleaning and remove a
useless local variable.

Signed-off-by: Alexandru Elisei <[email protected]>
[ Make sure the Linux kernel image area is marked as RAM ]
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch32/include/asm/realm.h | 3 +
arm/aarch64/include/asm/realm.h | 3 +
arm/aarch64/realm.c | 112 ++++++++++++++++++++++++++++++++
arm/fdt.c | 6 ++
arm/kvm.c | 20 ++++--
include/linux/kernel.h | 1 +
6 files changed, 140 insertions(+), 5 deletions(-)

diff --git a/arm/aarch32/include/asm/realm.h b/arm/aarch32/include/asm/realm.h
index 5aca6cca..fcff0e55 100644
--- a/arm/aarch32/include/asm/realm.h
+++ b/arm/aarch32/include/asm/realm.h
@@ -6,5 +6,8 @@
#include "kvm/kvm.h"

static inline void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm) {}
+static inline void kvm_arm_realm_populate_kernel(struct kvm *kvm) {}
+static inline void kvm_arm_realm_populate_initrd(struct kvm *kvm) {}
+static inline void kvm_arm_realm_populate_dtb(struct kvm *kvm) {}

#endif /* ! __ASM_REALM_H */
diff --git a/arm/aarch64/include/asm/realm.h b/arm/aarch64/include/asm/realm.h
index e176f15f..6e760ac9 100644
--- a/arm/aarch64/include/asm/realm.h
+++ b/arm/aarch64/include/asm/realm.h
@@ -6,5 +6,8 @@
#include "kvm/kvm.h"

void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm);
+void kvm_arm_realm_populate_kernel(struct kvm *kvm);
+void kvm_arm_realm_populate_initrd(struct kvm *kvm);
+void kvm_arm_realm_populate_dtb(struct kvm *kvm);

#endif /* ! __ASM_REALM_H */
diff --git a/arm/aarch64/realm.c b/arm/aarch64/realm.c
index fc7f8d6a..eddccece 100644
--- a/arm/aarch64/realm.c
+++ b/arm/aarch64/realm.c
@@ -1,5 +1,7 @@
#include "kvm/kvm.h"

+#include <linux/byteorder.h>
+#include <asm/image.h>
#include <asm/realm.h>

@@ -80,3 +82,113 @@ void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm)
if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &rme_create_rd) < 0)
die_perror("KVM_CAP_RME(KVM_CAP_ARM_RME_CREATE_RD)");
}
+
+static void realm_init_ipa_range(struct kvm *kvm, u64 start, u64 size)
+{
+ struct kvm_cap_arm_rme_init_ipa_args init_ipa_args = {
+ .init_ipa_base = start,
+ .init_ipa_size = size
+ };
+ struct kvm_enable_cap rme_init_ipa_realm = {
+ .cap = KVM_CAP_ARM_RME,
+ .args[0] = KVM_CAP_ARM_RME_INIT_IPA_REALM,
+ .args[1] = (u64)&init_ipa_args
+ };
+
+ if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &rme_init_ipa_realm) < 0)
+ die("unable to intialise IPA range for Realm %llx - %llx (size %llu)",
+ start, start + size, size);
+
+}
+
+static void __realm_populate(struct kvm *kvm, u64 start, u64 size)
+{
+ struct kvm_cap_arm_rme_populate_realm_args populate_args = {
+ .populate_ipa_base = start,
+ .populate_ipa_size = size
+ };
+ struct kvm_enable_cap rme_populate_realm = {
+ .cap = KVM_CAP_ARM_RME,
+ .args[0] = KVM_CAP_ARM_RME_POPULATE_REALM,
+ .args[1] = (u64)&populate_args
+ };
+
+ if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &rme_populate_realm) < 0)
+ die("unable to populate Realm memory %llx - %llx (size %llu)",
+ start, start + size, size);
+}
+
+static void realm_populate(struct kvm *kvm, u64 start, u64 size)
+{
+ realm_init_ipa_range(kvm, start, size);
+ __realm_populate(kvm, start, size);
+}
+
+static bool is_arm64_linux_kernel_image(void *header)
+{
+ struct arm64_image_header *hdr = header;
+
+ return memcmp(&hdr->magic, ARM64_IMAGE_MAGIC, sizeof(hdr->magic)) == 0;
+}
+
+static ssize_t arm64_linux_kernel_image_size(void *header)
+{
+ struct arm64_image_header *hdr = header;
+
+ if (is_arm64_linux_kernel_image(header))
+ return le64_to_cpu(hdr->image_size);
+ die("Not arm64 Linux kernel Image");
+}
+
+void kvm_arm_realm_populate_kernel(struct kvm *kvm)
+{
+ u64 start, end, mem_size;
+ void *header = guest_flat_to_host(kvm, kvm->arch.kern_guest_start);
+
+ start = ALIGN_DOWN(kvm->arch.kern_guest_start, SZ_4K);
+ end = ALIGN(kvm->arch.kern_guest_start + kvm->arch.kern_size, SZ_4K);
+
+ if (is_arm64_linux_kernel_image(header))
+ mem_size = arm64_linux_kernel_image_size(header);
+ else
+ mem_size = end - start;
+
+ realm_init_ipa_range(kvm, start, mem_size);
+ __realm_populate(kvm, start, end - start);
+}
+
+void kvm_arm_realm_populate_initrd(struct kvm *kvm)
+{
+ u64 kernel_end, start, end;
+
+ kernel_end = ALIGN(kvm->arch.kern_guest_start + kvm->arch.kern_size, SZ_4K);
+ start = ALIGN_DOWN(kvm->arch.initrd_guest_start, SZ_4K);
+ /*
+ * Because we align the initrd to 4 bytes, it is theoretically possible
+ * for the start of the initrd to overlap with the same page where the
+ * kernel ends.
+ */
+ if (start < kernel_end)
+ start = kernel_end;
+ end = ALIGN(kvm->arch.initrd_guest_start + kvm->arch.initrd_size, SZ_4K);
+ if (end > start)
+ realm_populate(kvm, start, end - start);
+}
+
+void kvm_arm_realm_populate_dtb(struct kvm *kvm)
+{
+ u64 initrd_end, start, end;
+
+ initrd_end = ALIGN(kvm->arch.initrd_guest_start + kvm->arch.initrd_size, SZ_4K);
+ start = ALIGN_DOWN(kvm->arch.dtb_guest_start, SZ_4K);
+ /*
+ * Same situation as with the initrd, but now it is the DTB which is
+ * overlapping with the last page of the initrd, because the initrd is
+ * populated first.
+ */
+ if (start < initrd_end)
+ start = initrd_end;
+ end = ALIGN(kvm->arch.dtb_guest_start + FDT_MAX_SIZE, SZ_4K);
+ if (end > start)
+ realm_populate(kvm, start, end - start);
+}
diff --git a/arm/fdt.c b/arm/fdt.c
index 286ccadf..762a604d 100644
--- a/arm/fdt.c
+++ b/arm/fdt.c
@@ -7,6 +7,8 @@
#include "arm-common/gic.h"
#include "arm-common/pci.h"

+#include <asm/realm.h>
+
#include <stdbool.h>

#include <linux/byteorder.h>
@@ -231,6 +233,10 @@ static int setup_fdt(struct kvm *kvm)

if (kvm->cfg.arch.dump_dtb_filename)
dump_fdt(kvm->cfg.arch.dump_dtb_filename, fdt_dest);
+
+ if (kvm->cfg.arch.is_realm)
+ kvm_arm_realm_populate_dtb(kvm);
+
return 0;
}
late_init(setup_fdt);
diff --git a/arm/kvm.c b/arm/kvm.c
index acb627b2..57c5b5f7 100644
--- a/arm/kvm.c
+++ b/arm/kvm.c
@@ -6,6 +6,7 @@
#include "kvm/fdt.h"

#include "arm-common/gic.h"
+#include <asm/realm.h>

#include <sys/resource.h>

@@ -167,6 +168,9 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm, int fd_kernel, int fd_initrd,
pr_debug("Loaded kernel to 0x%llx (%llu bytes)",
kvm->arch.kern_guest_start, kvm->arch.kern_size);

+ if (kvm->cfg.arch.is_realm)
+ kvm_arm_realm_populate_kernel(kvm);
+
/*
* Now load backwards from the end of memory so the kernel
* decompressor has plenty of space to work with. First up is
@@ -188,7 +192,6 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm, int fd_kernel, int fd_initrd,
/* ... and finally the initrd, if we have one. */
if (fd_initrd != -1) {
struct stat sb;
- unsigned long initrd_start;

if (fstat(fd_initrd, &sb))
die_perror("fstat");
@@ -199,7 +202,6 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm, int fd_kernel, int fd_initrd,
if (pos < kernel_end)
die("initrd overlaps with kernel image.");

- initrd_start = guest_addr;
file_size = read_file(fd_initrd, pos, limit - pos);
if (file_size == -1) {
if (errno == ENOMEM)
@@ -208,11 +210,13 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm, int fd_kernel, int fd_initrd,
die_perror("initrd read");
}

- kvm->arch.initrd_guest_start = initrd_start;
+ kvm->arch.initrd_guest_start = guest_addr;
kvm->arch.initrd_size = file_size;
pr_debug("Loaded initrd to 0x%llx (%llu bytes)",
- kvm->arch.initrd_guest_start,
- kvm->arch.initrd_size);
+ kvm->arch.initrd_guest_start, kvm->arch.initrd_size);
+
+ if (kvm->cfg.arch.is_realm)
+ kvm_arm_realm_populate_initrd(kvm);
} else {
kvm->arch.initrd_size = 0;
}
@@ -269,6 +273,8 @@ bool kvm__load_firmware(struct kvm *kvm, const char *firmware_filename)

/* Kernel isn't loaded by kvm, point start address to firmware */
kvm->arch.kern_guest_start = fw_addr;
+ kvm->arch.kern_size = fw_sz;
+
pr_debug("Loaded firmware to 0x%llx (%zd bytes)",
kvm->arch.kern_guest_start, fw_sz);

@@ -283,6 +289,10 @@ bool kvm__load_firmware(struct kvm *kvm, const char *firmware_filename)
kvm->arch.dtb_guest_start,
kvm->arch.dtb_guest_start + FDT_MAX_SIZE);

+ if (kvm->cfg.arch.is_realm)
+ /* We hijack the kernel fields to describe the firmware. */
+ kvm_arm_realm_populate_kernel(kvm);
+
return true;
}

diff --git a/include/linux/kernel.h b/include/linux/kernel.h
index 6c22f1c0..25f19c20 100644
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -9,6 +9,7 @@

#define DIV_ROUND_UP(n,d) (((n) + (d) - 1) / (d))

+#define ALIGN_DOWN(x,a) __ALIGN_MASK(x - (typeof(x))((a) - 1),(typeof(x))(a)-1)
#define ALIGN(x,a) __ALIGN_MASK(x,(typeof(x))(a)-1)
#define __ALIGN_MASK(x,mask) (((x)+(mask))&~(mask))
#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
--
2.34.1

2023-01-27 12:20:30

[permalink] [raw]

Subject: [RFC kvmtool 20/31] arm64: Finalize realm VCPU after reset

From: Alexandru Elisei <[email protected]>

In order to run a VCPU belonging to a realm, that VCPU must be in the
finalized state. Finalize the CPU after reset, since kvmtool won't be
touching the VCPU state afterwards.

Signed-off-by: Alexandru Elisei <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/kvm-cpu.c | 7 +++++++
1 file changed, 7 insertions(+)

diff --git a/arm/aarch64/kvm-cpu.c b/arm/aarch64/kvm-cpu.c
index 37f9aa9d..24e570c4 100644
--- a/arm/aarch64/kvm-cpu.c
+++ b/arm/aarch64/kvm-cpu.c
@@ -128,6 +128,13 @@ static void reset_vcpu_aarch64(struct kvm_cpu *vcpu)
if (ioctl(vcpu->vcpu_fd, KVM_SET_ONE_REG, &reg) < 0)
die_perror("KVM_SET_ONE_REG failed (pc)");
}
+
+ if (kvm->cfg.arch.is_realm) {
+ int feature = KVM_ARM_VCPU_REC;
+
+ if (ioctl(vcpu->vcpu_fd, KVM_ARM_VCPU_FINALIZE, &feature) < 0)
+ die_perror("KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_REC)");
+ }
}

void kvm_cpu__select_features(struct kvm *kvm, struct kvm_vcpu_init *init)
--
2.34.1

2023-01-27 12:20:30

[permalink] [raw]

Subject: [RFC kvmtool 15/31] arm64: Add support for Realm Personalisation Value

Add option to specify Realm personalisation value

Signed-off-by: Suzuki K Poulose <[email protected]>
---
arm/aarch64/include/kvm/kvm-config-arch.h | 6 +++++-
arm/aarch64/kvm.c | 7 +++++++
arm/aarch64/realm.c | 23 +++++++++++++++++++++++
arm/include/arm-common/kvm-config-arch.h | 1 +
4 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/arm/aarch64/include/kvm/kvm-config-arch.h b/arm/aarch64/include/kvm/kvm-config-arch.h
index b93999b6..f2e659ad 100644
--- a/arm/aarch64/include/kvm/kvm-config-arch.h
+++ b/arm/aarch64/include/kvm/kvm-config-arch.h
@@ -26,7 +26,11 @@ int vcpu_affinity_parser(const struct option *opt, const char *arg, int unset);
"Create VM running in a realm using Arm RME"), \
OPT_STRING('\0', "measurement-algo", &(cfg)->measurement_algo, \
"sha256, sha512", \
- "Realm Measurement algorithm, default: sha256"),
+ "Realm Measurement algorithm, default: sha256"),\
+ OPT_STRING('\0', "realm-pv", &(cfg)->realm_pv, \
+ "personalisation value", \
+ "Personalisation Value (only) for Realm VMs"),
+

#include "arm-common/kvm-config-arch.h"

diff --git a/arm/aarch64/kvm.c b/arm/aarch64/kvm.c
index a5a98b2e..4798e359 100644
--- a/arm/aarch64/kvm.c
+++ b/arm/aarch64/kvm.c
@@ -56,6 +56,8 @@ static void validate_realm_cfg(struct kvm *kvm)
if (!kvm->cfg.arch.is_realm) {
if (kvm->cfg.arch.measurement_algo)
die("--measurement-algo valid only with --realm");
+ if (kvm->cfg.arch.realm_pv)
+ die("--realm-pv valid only with --realm");
return;
}

@@ -74,6 +76,11 @@ static void validate_realm_cfg(struct kvm *kvm)
kvm->arch.measurement_algo = KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA256;
}

+ if (kvm->cfg.arch.realm_pv) {
+ if (strlen(kvm->cfg.arch.realm_pv) > KVM_CAP_ARM_RME_RPV_SIZE)
+ die("Invalid size for Realm Personalization Value\n");
+ }
+
die("Realms not supported");
}

diff --git a/arm/aarch64/realm.c b/arm/aarch64/realm.c
index 31543e55..2e0be982 100644
--- a/arm/aarch64/realm.c
+++ b/arm/aarch64/realm.c
@@ -20,9 +20,32 @@ static void realm_configure_hash_algo(struct kvm *kvm)
die_perror("KVM_CAP_RME(KVM_CAP_ARM_RME_CONFIG_REALM) hash_algo");
}

+static void realm_configure_rpv(struct kvm *kvm)
+{
+ struct kvm_cap_arm_rme_config_item rpv_cfg = {
+ .cfg = KVM_CAP_ARM_RME_CFG_RPV,
+ };
+
+ struct kvm_enable_cap rme_config = {
+ .cap = KVM_CAP_ARM_RME,
+ .args[0] = KVM_CAP_ARM_RME_CONFIG_REALM,
+ .args[1] = (u64)&rpv_cfg,
+ };
+
+ if (!kvm->cfg.arch.realm_pv)
+ return;
+
+ memset(&rpv_cfg.rpv, 0, sizeof(rpv_cfg.rpv));
+ memcpy(&rpv_cfg.rpv, kvm->cfg.arch.realm_pv, strlen(kvm->cfg.arch.realm_pv));
+
+ if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &rme_config) < 0)
+ die_perror("KVM_CAP_RME(KVM_CAP_ARM_RME_CONFIG_REALM) RPV");
+}
+
static void realm_configure_parameters(struct kvm *kvm)
{
realm_configure_hash_algo(kvm);
+ realm_configure_rpv(kvm);
}

void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm)
diff --git a/arm/include/arm-common/kvm-config-arch.h b/arm/include/arm-common/kvm-config-arch.h
index a2faa3af..80a3b18e 100644
--- a/arm/include/arm-common/kvm-config-arch.h
+++ b/arm/include/arm-common/kvm-config-arch.h
@@ -7,6 +7,7 @@ struct kvm_config_arch {
const char *dump_dtb_filename;
const char *vcpu_affinity;
const char *measurement_algo;
+ const char *realm_pv;
unsigned int force_cntfrq;
bool virtio_trans_pci;
bool aarch32_guest;
--
2.34.1

2023-01-27 15:26:43

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Fri, Jan 27, 2023 at 11:22:48AM +0000, Suzuki K Poulose wrote:
> We are happy to announce the early RFC version of the Arm
> Confidential Compute Architecture (CCA) support for the Linux
> stack. The intention is to seek early feedback in the following areas:
> * KVM integration of the Arm CCA
> * KVM UABI for managing the Realms, seeking to generalise the operations
> wherever possible with other Confidential Compute solutions.

A prototype for launching Realm VMs with QEMU is available at:
https://lore.kernel.org/qemu-devel/[email protected]/

Thanks,
Jean

2023-01-31 14:44:23

by Thomas Huth

[permalink] [raw]

Subject: Re: [RFC kvm-unit-tests 01/27] lib/string: include stddef.h for size_t

On 27/01/2023 12.40, Joey Gouly wrote:
> Don't implicitly rely on this header being included.
>
> Signed-off-by: Joey Gouly <[email protected]>
> ---
> lib/string.h | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/lib/string.h b/lib/string.h
> index b07763ea..758dca8a 100644
> --- a/lib/string.h
> +++ b/lib/string.h
> @@ -7,6 +7,8 @@
> #ifndef _STRING_H_
> #define _STRING_H_
>
> +#include <stddef.h> /* For size_t */
> +
> extern size_t strlen(const char *buf);
> extern size_t strnlen(const char *buf, size_t maxlen);
> extern char *strcat(char *dest, const char *src);

Reviewed-by: Thomas Huth <[email protected]>

2023-02-10 16:51:34

by Ryan Roberts

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On 27/01/2023 11:22, Suzuki K Poulose wrote:
> [...]

> Running the stack
> ====================
>
> To run/test the stack, you would need the following components :
>
> 1) FVP Base AEM RevC model with FEAT_RME support [4]
> 2) TF-A firmware for EL3 [5]
> 3) TF-A RMM for R-EL2 [3]
> 4) Linux Kernel [6]
> 5) kvmtool [7]
> 6) kvm-unit-tests [8]
>
> Instructions for building the firmware components and running the model are
> available here [9]. Once, the host kernel is booted, a Realm can be launched by
> invoking the `lkvm` commad as follows:
>
> $ lkvm run --realm \
> --measurement-algo=["sha256", "sha512"] \
> --disable-sve \
> <normal-vm-options>
>
> Where:
> * --measurement-algo (Optional) specifies the algorithm selected for creating the
> initial measurements by the RMM for this Realm (defaults to sha256).
> * GICv3 is mandatory for the Realms.
> * SVE is not yet supported in the TF-RMM, and thus must be disabled using
> --disable-sve
>
> You may also run the kvm-unit-tests inside the Realm world, using the similar
> options as above.

Building all of these components and configuring the FVP correctly can be quite
tricky, so I thought I would plug a tool we have called Shrinkwrap, which can
simplify all of this.

The tool accepts a yaml input configuration that describes how a set of
components should be built and packaged, and how the FVP should be configured
and booted. And by default, it uses a Docker container on its backend, which
contains all the required tools, including the FVP. You can optionally use
Podman or have it run on your native system if you prefer. It supports both
x86_64 and aarch64. And you can even run it in --dry-run mode to see the set of
shell commands that would have been executed.

It comes with two CCA configs out-of-the-box; cca-3world.yaml builds TF-A, RMM,
Linux (for both host and guest), kvmtool and kvm-unit-tests. cca-4world.yaml
adds Hafnium and some demo SPs for the secure world (although since Hafnium
requires x86_64 to build, cca-4world.yaml doesn't currently work on an aarch64
build host).

See the documentation [1] and repository [2] for more info.

Brief instructions to get you up and running:

# Install shrinkwrap. (I assume you have Docker installed):
sudo pip3 install pyyaml termcolor tuxmake
git clone https://git.gitlab.arm.com/tooling/shrinkwrap.git
export PATH=$PWD/shrinkwrap/shrinkwrap:$PATH

# If running Python < 3.9:
sudo pip3 install graphlib-backport

# Build all the CCA components:
shrinkwrap build cca-3world.yaml [--dry-run]

# Run the stack in the FVP:
shrinkwrap run cca-3world.yaml -r ROOTFS=<my_rootfs.ext4> [--dry-run]

By default, building is done at ~/.shrinkwrap/build/cca-3world and the package
is created at ~/.shrinkwrap/package/cca-3world (this can be changed with
envvars).

The 'run' command will boot TF-A, RMM and host Linux kernel in the FVP, and
mount the provided rootfs. You will likely want to have copied the userspace
pieces into the rootfs before running, so you can create realms:

- ~/.shrinkwrap/package/cca-3world/Image (kernel with RMI and RSI support)
- ~/.shrinkwrap/package/cca-3world/lkvm (kvmtool able to launch realms)
- ~/.shrinkwrap/package/cca-3world/kvm-unit-tests.tgz (built kvm-unit-tests)

Once the FVP is booted to a shell, you can do something like this to launch a
Linux guest in a realm:

lkvm run --realm --disable-sve -c 1 -m 256 -k Image

[1] https://shrinkwrap.docs.arm.com
[2] https://gitlab.arm.com/tooling/shrinkwrap

2023-02-10 22:53:25

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Sat, Feb 11, 2023 at 1:56 AM Ryan Roberts <[email protected]> wrote:
>
> On 27/01/2023 11:22, Suzuki K Poulose wrote:
> > [...]
>
> > Running the stack
> > ====================
> >
> > To run/test the stack, you would need the following components :
> >
> > 1) FVP Base AEM RevC model with FEAT_RME support [4]
> > 2) TF-A firmware for EL3 [5]
> > 3) TF-A RMM for R-EL2 [3]
> > 4) Linux Kernel [6]
> > 5) kvmtool [7]
> > 6) kvm-unit-tests [8]
> >
> > Instructions for building the firmware components and running the model are
> > available here [9]. Once, the host kernel is booted, a Realm can be launched by
> > invoking the `lkvm` commad as follows:
> >
> > $ lkvm run --realm \
> > --measurement-algo=["sha256", "sha512"] \
> > --disable-sve \
> > <normal-vm-options>
> >
> > Where:
> > * --measurement-algo (Optional) specifies the algorithm selected for creating the
> > initial measurements by the RMM for this Realm (defaults to sha256).
> > * GICv3 is mandatory for the Realms.
> > * SVE is not yet supported in the TF-RMM, and thus must be disabled using
> > --disable-sve
> >
> > You may also run the kvm-unit-tests inside the Realm world, using the similar
> > options as above.
>
> Building all of these components and configuring the FVP correctly can be quite
> tricky, so I thought I would plug a tool we have called Shrinkwrap, which can
> simplify all of this.
>
> The tool accepts a yaml input configuration that describes how a set of
> components should be built and packaged, and how the FVP should be configured
> and booted. And by default, it uses a Docker container on its backend, which
> contains all the required tools, including the FVP. You can optionally use
> Podman or have it run on your native system if you prefer. It supports both
> x86_64 and aarch64. And you can even run it in --dry-run mode to see the set of
> shell commands that would have been executed.
>
> It comes with two CCA configs out-of-the-box; cca-3world.yaml builds TF-A, RMM,
> Linux (for both host and guest), kvmtool and kvm-unit-tests. cca-4world.yaml
> adds Hafnium and some demo SPs for the secure world (although since Hafnium
> requires x86_64 to build, cca-4world.yaml doesn't currently work on an aarch64
> build host).
>
> See the documentation [1] and repository [2] for more info.
>
> Brief instructions to get you up and running:
>
> # Install shrinkwrap. (I assume you have Docker installed):
> sudo pip3 install pyyaml termcolor tuxmake
> git clone https://git.gitlab.arm.com/tooling/shrinkwrap.git
> export PATH=$PWD/shrinkwrap/shrinkwrap:$PATH
>
> # If running Python < 3.9:
> sudo pip3 install graphlib-backport
>
> # Build all the CCA components:
> shrinkwrap build cca-3world.yaml [--dry-run]

This has been working on my Multipass instance on M1, thanks for the tool.

Thanks,
Itaru.

>
> # Run the stack in the FVP:
> shrinkwrap run cca-3world.yaml -r ROOTFS=<my_rootfs.ext4> [--dry-run]
>
> By default, building is done at ~/.shrinkwrap/build/cca-3world and the package
> is created at ~/.shrinkwrap/package/cca-3world (this can be changed with
> envvars).
>
> The 'run' command will boot TF-A, RMM and host Linux kernel in the FVP, and
> mount the provided rootfs. You will likely want to have copied the userspace
> pieces into the rootfs before running, so you can create realms:
>
> - ~/.shrinkwrap/package/cca-3world/Image (kernel with RMI and RSI support)
> - ~/.shrinkwrap/package/cca-3world/lkvm (kvmtool able to launch realms)
> - ~/.shrinkwrap/package/cca-3world/kvm-unit-tests.tgz (built kvm-unit-tests)
>
> Once the FVP is booted to a shell, you can do something like this to launch a
> Linux guest in a realm:
>
> lkvm run --realm --disable-sve -c 1 -m 256 -k Image
>
> [1] https://shrinkwrap.docs.arm.com
> [2] https://gitlab.arm.com/tooling/shrinkwrap
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

2023-02-14 17:14:35

by Dr. David Alan Gilbert

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

* Suzuki K Poulose ([email protected]) wrote:
> We are happy to announce the early RFC version of the Arm
> Confidential Compute Architecture (CCA) support for the Linux
> stack. The intention is to seek early feedback in the following areas:
> * KVM integration of the Arm CCA
> * KVM UABI for managing the Realms, seeking to generalise the operations
> wherever possible with other Confidential Compute solutions.
> Note: This version doesn't support Guest Private memory, which will be added
> later (see below).
> * Linux Guest support for Realms
>
> Arm CCA Introduction
> =====================
>
> The Arm CCA is a reference software architecture and implementation that builds
> on the Realm Management Extension (RME), enabling the execution of Virtual
> machines, while preventing access by more privileged software, such as hypervisor.
> The Arm CCA allows the hypervisor to control the VM, but removes the right for
> access to the code, register state or data that is used by VM.
> More information on the architecture is available here[0].
>
> Arm CCA Reference Software Architecture
>
> Realm World || Normal World || Secure World ||
> || | || ||
> EL0 x-------x || x----x | x------x || ||
> | Realm | || | | | | | || ||
> | | || | VM | | | | || ||
> ----| VM* |---------||-| |---| |-||----------------||
> | | || | | | | H | || ||
> EL1 x-------x || x----x | | | || ||
> ^ || | | o | || ||
> | || | | | || ||
> ------- R*------------------------| s -|---------------------
> S || | | || ||
> I || | t | || ||
> | || | | || ||
> v || x------x || ||
> EL2 RMM* || ^ || ||
> ^ || | || ||
> ========|=============================|========================
> | | SMC
> x--------- *RMI* -------------x
>
> EL3 Root World
> EL3 Firmware
> ===============================================================
> Where :
> RMM - Realm Management Monitor
> RMI - Realm Management Interface
> RSI - Realm Service Interface
> SMC - Secure Monitor Call

Hi,
It's nice to see this full stack posted - thanks!

Are there any pointers to information on attestation and similar
measurement things? In particular, are there any plans for a vTPM
for Realms - if there were, it would make life easy for us, since we
can share some user space stuff with other CoCo systems.

Dave

> RME introduces a new security state "Realm world", in addition to the
> traditional Secure and Non-Secure states. The Arm CCA defines a new component,
> Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of
> firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at
> system boot.
>
> The RMM provides standard interfaces - Realm Management Interface (RMI) - to the
> Normal world hypervisor to manage the VMs running in the Realm world (also called
> Realms in short). These are exposed via SMC and are routed through the EL3
> firmwre.
> The RMI interface includes:
> - Move a physical page from the Normal world to the Realm world
> - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD)
> - Creating VCPUs aka Realm Execution Context (REC), with initial register state.
> - Create stage2 translation table at any level.
> - Load initial images into Realm Memory from normal world memory
> - Schedule RECs (vCPUs) and handle exits
> - Inject virtual interrupts into the Realm
> - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM).
> - Create "shared" mappings that can be accessed by VMM/Hyp.
> - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables)
>
> However v1.0 of RMM specifications doesn't support:
> - Paging protected memory of a Realm VM. Thus the pages backing the protected
> memory region must be pinned.
> - Live migration of Realms.
> - Trusted Device assignment.
> - Physical interrupt backed Virtual interrupts for Realms
>
> RMM also provides certain services to the Realms via SMC, called Realm Service
> Interface (RSI). These include:
> - Realm Guest Configuration.
> - Attestation & Measurement services
> - Managing the state of an Intermediate Physical Address (IPA aka GPA) page.
> - Host Call service (Communication with the Normal world Hypervisor)
>
> The specifications for the RMM software is currently at *v1.0-Beta2* and the
> latest version is available here [1].
>
> The Trusted Firmware foundation has an implementation of the RMM - TF-RMM -
> available here [3].
>
> Implementation
> =================
>
> This version of the stack is based on the RMM specification v1.0-Beta0[2], with
> following exceptions :
> - TF-RMM/KVM currently doesn't support the optional features of PMU,
> SVE and Self-hosted debug (coming soon).
> - The RSI_HOST_CALL structure alignment requirement is reduced to match
> RMM v1.0 Beta1
> - RMI/RSI version numbers do not match the RMM spec. This will be
> resolved once the spec/implementation is complete, across TF-RMM+Linux stack.
>
> We plan to update the stack to support the latest version of the RMMv1.0 spec
> in the coming revisions.
>
> This release includes the following components :
>
> a) Linux Kernel
> i) Host / KVM support - Support for driving the Realms via RMI. This is
> dependent on running in the Kernel at EL2 (aka VHE mode). Also provides
> UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page
> size, matching the Stage2 granule supported by RMM. The VMM is responsible
> for making sure the guest memory is locked.
>
> TODO: Guest Private memory[10] integration - We have been following the
> series and support will be added once it is merged upstream.
>
> ii) Guest support - Support for a Linux Kernel to run in the Realm VM at
> Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0
> only). All I/O are treated as non-secure/shared.
>
> c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory
> as mentioned above.
> d) kvm-unit-tests - Support for running in Realms along with additional tests
> for RSI ABI.
>
> Running the stack
> ====================
>
> To run/test the stack, you would need the following components :
>
> 1) FVP Base AEM RevC model with FEAT_RME support [4]
> 2) TF-A firmware for EL3 [5]
> 3) TF-A RMM for R-EL2 [3]
> 4) Linux Kernel [6]
> 5) kvmtool [7]
> 6) kvm-unit-tests [8]
>
> Instructions for building the firmware components and running the model are
> available here [9]. Once, the host kernel is booted, a Realm can be launched by
> invoking the `lkvm` commad as follows:
>
> $ lkvm run --realm \
> --measurement-algo=["sha256", "sha512"] \
> --disable-sve \
> <normal-vm-options>
>
> Where:
> * --measurement-algo (Optional) specifies the algorithm selected for creating the
> initial measurements by the RMM for this Realm (defaults to sha256).
> * GICv3 is mandatory for the Realms.
> * SVE is not yet supported in the TF-RMM, and thus must be disabled using
> --disable-sve
>
> You may also run the kvm-unit-tests inside the Realm world, using the similar
> options as above.
>
>
> Links
> ============
>
> [0] Arm CCA Landing page (See Key Resources section for various documentations)
> https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture
>
> [1] RMM Specification Latest
> https://developer.arm.com/documentation/den0137/latest
>
> [2] RMM v1.0-Beta0 specification
> https://developer.arm.com/documentation/den0137/1-0bet0/
>
> [3] Trusted Firmware RMM - TF-RMM
> https://www.trustedfirmware.org/projects/tf-rmm/
> GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
>
> [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
> https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
>
> [5] Trusted Firmware for A class
> https://www.trustedfirmware.org/projects/tf-a/
>
> [6] Linux kernel support for Arm-CCA
> https://gitlab.arm.com/linux-arm/linux-cca
> Host Support branch: cca-host/rfc-v1
> Guest Support branch: cca-guest/rfc-v1
>
> [7] kvmtool support for Arm CCA
> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1
>
> [8] kvm-unit-tests support for Arm CCA
> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1
>
> [9] Instructions for Building Firmware components and running the model, see
> section 4.19.2 "Building and running TF-A with RME"
> https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
>
> [10] fd based Guest Private memory for KVM
> https://lkml.kernel.org/r/[email protected]
>
> Cc: Alexandru Elisei <[email protected]>
> Cc: Andrew Jones <[email protected]>
> Cc: Catalin Marinas <[email protected]>
> Cc: Chao Peng <[email protected]>
> Cc: Christoffer Dall <[email protected]>
> Cc: Fuad Tabba <[email protected]>
> Cc: James Morse <[email protected]>
> Cc: Jean-Philippe Brucker <[email protected]>
> Cc: Joey Gouly <[email protected]>
> Cc: Marc Zyngier <[email protected]>
> Cc: Mark Rutland <[email protected]>
> Cc: Oliver Upton <[email protected]>
> Cc: Paolo Bonzini <[email protected]>
> Cc: Quentin Perret <[email protected]>
> Cc: Sean Christopherson <[email protected]>
> Cc: Steven Price <[email protected]>
> Cc: Thomas Huth <[email protected]>
> Cc: Will Deacon <[email protected]>
> Cc: Zenghui Yu <[email protected]>
> To: [email protected]
> To: [email protected]
> Cc: [email protected]
> Cc: [email protected]
> To: [email protected]
> To: [email protected]
>
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

2023-02-17 08:03:08

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Sat, Feb 11, 2023 at 7:53 AM Itaru Kitayama <[email protected]> wrote:
>
> On Sat, Feb 11, 2023 at 1:56 AM Ryan Roberts <[email protected]> wrote:
> >
> > On 27/01/2023 11:22, Suzuki K Poulose wrote:
> > > [...]
> >
> > > Running the stack
> > > ====================
> > >
> > > To run/test the stack, you would need the following components :
> > >
> > > 1) FVP Base AEM RevC model with FEAT_RME support [4]
> > > 2) TF-A firmware for EL3 [5]
> > > 3) TF-A RMM for R-EL2 [3]
> > > 4) Linux Kernel [6]
> > > 5) kvmtool [7]
> > > 6) kvm-unit-tests [8]
> > >
> > > Instructions for building the firmware components and running the model are
> > > available here [9]. Once, the host kernel is booted, a Realm can be launched by
> > > invoking the `lkvm` commad as follows:
> > >
> > > $ lkvm run --realm \
> > > --measurement-algo=["sha256", "sha512"] \
> > > --disable-sve \
> > > <normal-vm-options>
> > >
> > > Where:
> > > * --measurement-algo (Optional) specifies the algorithm selected for creating the
> > > initial measurements by the RMM for this Realm (defaults to sha256).
> > > * GICv3 is mandatory for the Realms.
> > > * SVE is not yet supported in the TF-RMM, and thus must be disabled using
> > > --disable-sve
> > >
> > > You may also run the kvm-unit-tests inside the Realm world, using the similar
> > > options as above.
> >
> > Building all of these components and configuring the FVP correctly can be quite
> > tricky, so I thought I would plug a tool we have called Shrinkwrap, which can
> > simplify all of this.
> >
> > The tool accepts a yaml input configuration that describes how a set of
> > components should be built and packaged, and how the FVP should be configured
> > and booted. And by default, it uses a Docker container on its backend, which
> > contains all the required tools, including the FVP. You can optionally use
> > Podman or have it run on your native system if you prefer. It supports both
> > x86_64 and aarch64. And you can even run it in --dry-run mode to see the set of
> > shell commands that would have been executed.
> >
> > It comes with two CCA configs out-of-the-box; cca-3world.yaml builds TF-A, RMM,
> > Linux (for both host and guest), kvmtool and kvm-unit-tests. cca-4world.yaml
> > adds Hafnium and some demo SPs for the secure world (although since Hafnium
> > requires x86_64 to build, cca-4world.yaml doesn't currently work on an aarch64
> > build host).
> >
> > See the documentation [1] and repository [2] for more info.
> >
> > Brief instructions to get you up and running:
> >
> > # Install shrinkwrap. (I assume you have Docker installed):
> > sudo pip3 install pyyaml termcolor tuxmake
> > git clone https://git.gitlab.arm.com/tooling/shrinkwrap.git
> > export PATH=$PWD/shrinkwrap/shrinkwrap:$PATH
> >
> > # If running Python < 3.9:
> > sudo pip3 install graphlib-backport
> >
> > # Build all the CCA components:
> > shrinkwrap build cca-3world.yaml [--dry-run]
>
> This has been working on my Multipass instance on M1, thanks for the tool.
>
> Thanks,
> Itaru.

It took a while though I've just booted an Ubuntu 22.10 disk image
with the cca-3world.yaml config on M1.

Thanks,
Itaru.

>
> >
> > # Run the stack in the FVP:
> > shrinkwrap run cca-3world.yaml -r ROOTFS=<my_rootfs.ext4> [--dry-run]
> >
> > By default, building is done at ~/.shrinkwrap/build/cca-3world and the package
> > is created at ~/.shrinkwrap/package/cca-3world (this can be changed with
> > envvars).
> >
> > The 'run' command will boot TF-A, RMM and host Linux kernel in the FVP, and
> > mount the provided rootfs. You will likely want to have copied the userspace
> > pieces into the rootfs before running, so you can create realms:
> >
> > - ~/.shrinkwrap/package/cca-3world/Image (kernel with RMI and RSI support)
> > - ~/.shrinkwrap/package/cca-3world/lkvm (kvmtool able to launch realms)
> > - ~/.shrinkwrap/package/cca-3world/kvm-unit-tests.tgz (built kvm-unit-tests)
> >
> > Once the FVP is booted to a shell, you can do something like this to launch a
> > Linux guest in a realm:
> >
> > lkvm run --realm --disable-sve -c 1 -m 256 -k Image
> >
> > [1] https://shrinkwrap.docs.arm.com
> > [2] https://gitlab.arm.com/tooling/shrinkwrap
> >
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > [email protected]
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

2023-02-20 10:52:08

by Ryan Roberts

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On 17/02/2023 08:02, Itaru Kitayama wrote:
> On Sat, Feb 11, 2023 at 7:53 AM Itaru Kitayama <[email protected]> wrote:
>>
>> On Sat, Feb 11, 2023 at 1:56 AM Ryan Roberts <[email protected]> wrote:
>>>
>>> On 27/01/2023 11:22, Suzuki K Poulose wrote:
>>>> [...]
>>>
>>>> Running the stack
>>>> ====================
>>>>
>>>> To run/test the stack, you would need the following components :
>>>>
>>>> 1) FVP Base AEM RevC model with FEAT_RME support [4]
>>>> 2) TF-A firmware for EL3 [5]
>>>> 3) TF-A RMM for R-EL2 [3]
>>>> 4) Linux Kernel [6]
>>>> 5) kvmtool [7]
>>>> 6) kvm-unit-tests [8]
>>>>
>>>> Instructions for building the firmware components and running the model are
>>>> available here [9]. Once, the host kernel is booted, a Realm can be launched by
>>>> invoking the `lkvm` commad as follows:
>>>>
>>>> $ lkvm run --realm \
>>>> --measurement-algo=["sha256", "sha512"] \
>>>> --disable-sve \
>>>> <normal-vm-options>
>>>>
>>>> Where:
>>>> * --measurement-algo (Optional) specifies the algorithm selected for creating the
>>>> initial measurements by the RMM for this Realm (defaults to sha256).
>>>> * GICv3 is mandatory for the Realms.
>>>> * SVE is not yet supported in the TF-RMM, and thus must be disabled using
>>>> --disable-sve
>>>>
>>>> You may also run the kvm-unit-tests inside the Realm world, using the similar
>>>> options as above.
>>>
>>> Building all of these components and configuring the FVP correctly can be quite
>>> tricky, so I thought I would plug a tool we have called Shrinkwrap, which can
>>> simplify all of this.
>>>
>>> The tool accepts a yaml input configuration that describes how a set of
>>> components should be built and packaged, and how the FVP should be configured
>>> and booted. And by default, it uses a Docker container on its backend, which
>>> contains all the required tools, including the FVP. You can optionally use
>>> Podman or have it run on your native system if you prefer. It supports both
>>> x86_64 and aarch64. And you can even run it in --dry-run mode to see the set of
>>> shell commands that would have been executed.
>>>
>>> It comes with two CCA configs out-of-the-box; cca-3world.yaml builds TF-A, RMM,
>>> Linux (for both host and guest), kvmtool and kvm-unit-tests. cca-4world.yaml
>>> adds Hafnium and some demo SPs for the secure world (although since Hafnium
>>> requires x86_64 to build, cca-4world.yaml doesn't currently work on an aarch64
>>> build host).
>>>
>>> See the documentation [1] and repository [2] for more info.
>>>
>>> Brief instructions to get you up and running:
>>>
>>> # Install shrinkwrap. (I assume you have Docker installed):
>>> sudo pip3 install pyyaml termcolor tuxmake
>>> git clone https://git.gitlab.arm.com/tooling/shrinkwrap.git
>>> export PATH=$PWD/shrinkwrap/shrinkwrap:$PATH
>>>
>>> # If running Python < 3.9:
>>> sudo pip3 install graphlib-backport
>>>
>>> # Build all the CCA components:
>>> shrinkwrap build cca-3world.yaml [--dry-run]
>>
>> This has been working on my Multipass instance on M1, thanks for the tool.
>>
>> Thanks,
>> Itaru.
>
> It took a while though I've just booted an Ubuntu 22.10 disk image
> with the cca-3world.yaml config on M1.

That's good to hear - If you have any feedback (or patches ;-)) for Shrinkwrap
that would improve the experience, do let me know!

>
> Thanks,
> Itaru.
>
>>
>>>
>>> # Run the stack in the FVP:
>>> shrinkwrap run cca-3world.yaml -r ROOTFS=<my_rootfs.ext4> [--dry-run]
>>>
>>> By default, building is done at ~/.shrinkwrap/build/cca-3world and the package
>>> is created at ~/.shrinkwrap/package/cca-3world (this can be changed with
>>> envvars).
>>>
>>> The 'run' command will boot TF-A, RMM and host Linux kernel in the FVP, and
>>> mount the provided rootfs. You will likely want to have copied the userspace
>>> pieces into the rootfs before running, so you can create realms:
>>>
>>> - ~/.shrinkwrap/package/cca-3world/Image (kernel with RMI and RSI support)
>>> - ~/.shrinkwrap/package/cca-3world/lkvm (kvmtool able to launch realms)
>>> - ~/.shrinkwrap/package/cca-3world/kvm-unit-tests.tgz (built kvm-unit-tests)
>>>
>>> Once the FVP is booted to a shell, you can do something like this to launch a
>>> Linux guest in a realm:
>>>
>>> lkvm run --realm --disable-sve -c 1 -m 256 -k Image
>>>
>>> [1] https://shrinkwrap.docs.arm.com
>>> [2] https://gitlab.arm.com/tooling/shrinkwrap
>>>
>>>
>>> _______________________________________________
>>> linux-arm-kernel mailing list
>>> [email protected]
>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

2023-02-28 23:35:21

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Sat, Jan 28, 2023 at 12:30 AM Jean-Philippe Brucker
<[email protected]> wrote:
>
> On Fri, Jan 27, 2023 at 11:22:48AM +0000, Suzuki K Poulose wrote:
> > We are happy to announce the early RFC version of the Arm
> > Confidential Compute Architecture (CCA) support for the Linux
> > stack. The intention is to seek early feedback in the following areas:
> > * KVM integration of the Arm CCA
> > * KVM UABI for managing the Realms, seeking to generalise the operations
> > wherever possible with other Confidential Compute solutions.
>
> A prototype for launching Realm VMs with QEMU is available at:
> https://lore.kernel.org/qemu-devel/[email protected]/
>
> Thanks,
> Jean

Hi Jean,
I've tried your series in Real on CCA Host, but the KVM arch init
emits an Invalid argument error and terminates.
I configure it with the aarch64-softmmu target only and built, any
other steps I should worry?

Itaru.

>
>
> _______________________________________________
> linux-arm-kernel mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

2023-03-01 09:20:36

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

Hi Itaru,

On Wed, Mar 01, 2023 at 08:35:05AM +0900, Itaru Kitayama wrote:
> Hi Jean,
> I've tried your series in Real on CCA Host, but the KVM arch init
> emits an Invalid argument error and terminates.

Do you know which call returns this error? Normally the RMEGuest support
should print more detailed errors. Are you able to launch normal guests
(without the rme-guest object and confidential-guest-support machine
parameter)? Could you give the complete QEMU command-line?

> I configure it with the aarch64-softmmu target only and built, any
> other steps I should worry?

No, that should be enough

Thanks,
Jean

2023-03-01 09:59:37

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

Hi Dave

Thanks for your response, and apologies for the delay. Response, in line.

On 14/02/2023 17:13, Dr. David Alan Gilbert wrote:
> * Suzuki K Poulose ([email protected]) wrote:
>> We are happy to announce the early RFC version of the Arm
>> Confidential Compute Architecture (CCA) support for the Linux
>> stack. The intention is to seek early feedback in the following areas:
>> * KVM integration of the Arm CCA
>> * KVM UABI for managing the Realms, seeking to generalise the operations
>> wherever possible with other Confidential Compute solutions.
>> Note: This version doesn't support Guest Private memory, which will be added
>> later (see below).
>> * Linux Guest support for Realms
>>
>> Arm CCA Introduction
>> =====================
>>
>> The Arm CCA is a reference software architecture and implementation that builds
>> on the Realm Management Extension (RME), enabling the execution of Virtual
>> machines, while preventing access by more privileged software, such as hypervisor.
>> The Arm CCA allows the hypervisor to control the VM, but removes the right for
>> access to the code, register state or data that is used by VM.
>> More information on the architecture is available here[0].
>>
>> Arm CCA Reference Software Architecture
>>
>> Realm World || Normal World || Secure World ||
>> || | || ||
>> EL0 x-------x || x----x | x------x || ||
>> | Realm | || | | | | | || ||
>> | | || | VM | | | | || ||
>> ----| VM* |---------||-| |---| |-||----------------||
>> | | || | | | | H | || ||
>> EL1 x-------x || x----x | | | || ||
>> ^ || | | o | || ||
>> | || | | | || ||
>> ------- R*------------------------| s -|---------------------
>> S || | | || ||
>> I || | t | || ||
>> | || | | || ||
>> v || x------x || ||
>> EL2 RMM* || ^ || ||
>> ^ || | || ||
>> ========|=============================|========================
>> | | SMC
>> x--------- *RMI* -------------x
>>
>> EL3 Root World
>> EL3 Firmware
>> ===============================================================
>> Where :
>> RMM - Realm Management Monitor
>> RMI - Realm Management Interface
>> RSI - Realm Service Interface
>> SMC - Secure Monitor Call
>
> Hi,
> It's nice to see this full stack posted - thanks!
>
> Are there any pointers to information on attestation and similar
> measurement things? In particular, are there any plans for a vTPM

The RMM v1.0 provides attestation and measurement services to the Realm,
via Realm Service Interface (RSI) calls. However, there is no support
for partitioning the Realm VM with v1.0. This is currently under
development and should be available in the near future.

With that in place, a vTPM could reside in a partition of the Realm VM
along side the OS in another. Does that answer your question ?

Kind regards
Suzuki

> for Realms - if there were, it would make life easy for us, since we
> can share some user space stuff with other CoCo systems.
>
> Dave
>
>> RME introduces a new security state "Realm world", in addition to the
>> traditional Secure and Non-Secure states. The Arm CCA defines a new component,
>> Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of
>> firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at
>> system boot.
>>
>> The RMM provides standard interfaces - Realm Management Interface (RMI) - to the
>> Normal world hypervisor to manage the VMs running in the Realm world (also called
>> Realms in short). These are exposed via SMC and are routed through the EL3
>> firmwre.
>> The RMI interface includes:
>> - Move a physical page from the Normal world to the Realm world
>> - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD)
>> - Creating VCPUs aka Realm Execution Context (REC), with initial register state.
>> - Create stage2 translation table at any level.
>> - Load initial images into Realm Memory from normal world memory
>> - Schedule RECs (vCPUs) and handle exits
>> - Inject virtual interrupts into the Realm
>> - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM).
>> - Create "shared" mappings that can be accessed by VMM/Hyp.
>> - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables)
>>
>> However v1.0 of RMM specifications doesn't support:
>> - Paging protected memory of a Realm VM. Thus the pages backing the protected
>> memory region must be pinned.
>> - Live migration of Realms.
>> - Trusted Device assignment.
>> - Physical interrupt backed Virtual interrupts for Realms
>>
>> RMM also provides certain services to the Realms via SMC, called Realm Service
>> Interface (RSI). These include:
>> - Realm Guest Configuration.
>> - Attestation & Measurement services
>> - Managing the state of an Intermediate Physical Address (IPA aka GPA) page.
>> - Host Call service (Communication with the Normal world Hypervisor)
>>
>> The specifications for the RMM software is currently at *v1.0-Beta2* and the
>> latest version is available here [1].
>>
>> The Trusted Firmware foundation has an implementation of the RMM - TF-RMM -
>> available here [3].
>>
>> Implementation
>> =================
>>
>> This version of the stack is based on the RMM specification v1.0-Beta0[2], with
>> following exceptions :
>> - TF-RMM/KVM currently doesn't support the optional features of PMU,
>> SVE and Self-hosted debug (coming soon).
>> - The RSI_HOST_CALL structure alignment requirement is reduced to match
>> RMM v1.0 Beta1
>> - RMI/RSI version numbers do not match the RMM spec. This will be
>> resolved once the spec/implementation is complete, across TF-RMM+Linux stack.
>>
>> We plan to update the stack to support the latest version of the RMMv1.0 spec
>> in the coming revisions.
>>
>> This release includes the following components :
>>
>> a) Linux Kernel
>> i) Host / KVM support - Support for driving the Realms via RMI. This is
>> dependent on running in the Kernel at EL2 (aka VHE mode). Also provides
>> UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page
>> size, matching the Stage2 granule supported by RMM. The VMM is responsible
>> for making sure the guest memory is locked.
>>
>> TODO: Guest Private memory[10] integration - We have been following the
>> series and support will be added once it is merged upstream.
>>
>> ii) Guest support - Support for a Linux Kernel to run in the Realm VM at
>> Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0
>> only). All I/O are treated as non-secure/shared.
>>
>> c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory
>> as mentioned above.
>> d) kvm-unit-tests - Support for running in Realms along with additional tests
>> for RSI ABI.
>>
>> Running the stack
>> ====================
>>
>> To run/test the stack, you would need the following components :
>>
>> 1) FVP Base AEM RevC model with FEAT_RME support [4]
>> 2) TF-A firmware for EL3 [5]
>> 3) TF-A RMM for R-EL2 [3]
>> 4) Linux Kernel [6]
>> 5) kvmtool [7]
>> 6) kvm-unit-tests [8]
>>
>> Instructions for building the firmware components and running the model are
>> available here [9]. Once, the host kernel is booted, a Realm can be launched by
>> invoking the `lkvm` commad as follows:
>>
>> $ lkvm run --realm \
>> --measurement-algo=["sha256", "sha512"] \
>> --disable-sve \
>> <normal-vm-options>
>>
>> Where:
>> * --measurement-algo (Optional) specifies the algorithm selected for creating the
>> initial measurements by the RMM for this Realm (defaults to sha256).
>> * GICv3 is mandatory for the Realms.
>> * SVE is not yet supported in the TF-RMM, and thus must be disabled using
>> --disable-sve
>>
>> You may also run the kvm-unit-tests inside the Realm world, using the similar
>> options as above.
>>
>>
>> Links
>> ============
>>
>> [0] Arm CCA Landing page (See Key Resources section for various documentations)
>> https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture
>>
>> [1] RMM Specification Latest
>> https://developer.arm.com/documentation/den0137/latest
>>
>> [2] RMM v1.0-Beta0 specification
>> https://developer.arm.com/documentation/den0137/1-0bet0/
>>
>> [3] Trusted Firmware RMM - TF-RMM
>> https://www.trustedfirmware.org/projects/tf-rmm/
>> GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
>>
>> [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
>> https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
>>
>> [5] Trusted Firmware for A class
>> https://www.trustedfirmware.org/projects/tf-a/
>>
>> [6] Linux kernel support for Arm-CCA
>> https://gitlab.arm.com/linux-arm/linux-cca
>> Host Support branch: cca-host/rfc-v1
>> Guest Support branch: cca-guest/rfc-v1
>>
>> [7] kvmtool support for Arm CCA
>> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1
>>
>> [8] kvm-unit-tests support for Arm CCA
>> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1
>>
>> [9] Instructions for Building Firmware components and running the model, see
>> section 4.19.2 "Building and running TF-A with RME"
>> https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
>>
>> [10] fd based Guest Private memory for KVM
>> https://lkml.kernel.org/r/[email protected]
>>
>> Cc: Alexandru Elisei <[email protected]>
>> Cc: Andrew Jones <[email protected]>
>> Cc: Catalin Marinas <[email protected]>
>> Cc: Chao Peng <[email protected]>
>> Cc: Christoffer Dall <[email protected]>
>> Cc: Fuad Tabba <[email protected]>
>> Cc: James Morse <[email protected]>
>> Cc: Jean-Philippe Brucker <[email protected]>
>> Cc: Joey Gouly <[email protected]>
>> Cc: Marc Zyngier <[email protected]>
>> Cc: Mark Rutland <[email protected]>
>> Cc: Oliver Upton <[email protected]>
>> Cc: Paolo Bonzini <[email protected]>
>> Cc: Quentin Perret <[email protected]>
>> Cc: Sean Christopherson <[email protected]>
>> Cc: Steven Price <[email protected]>
>> Cc: Thomas Huth <[email protected]>
>> Cc: Will Deacon <[email protected]>
>> Cc: Zenghui Yu <[email protected]>
>> To: [email protected]
>> To: [email protected]
>> Cc: [email protected]
>> Cc: [email protected]
>> To: [email protected]
>> To: [email protected]
>>

2023-03-01 22:12:41

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Wed, Mar 1, 2023 at 6:20 PM Jean-Philippe Brucker
<[email protected]> wrote:
>
> Hi Itaru,
>
> On Wed, Mar 01, 2023 at 08:35:05AM +0900, Itaru Kitayama wrote:
> > Hi Jean,
> > I've tried your series in Real on CCA Host, but the KVM arch init
> > emits an Invalid argument error and terminates.
>
> Do you know which call returns this error? Normally the RMEGuest support
> should print more detailed errors. Are you able to launch normal guests
> (without the rme-guest object and confidential-guest-support machine
> parameter)? Could you give the complete QEMU command-line?

No, I cant say which. Yes, the CCA-capable QEMU boots if I don't set
RME-related options.

Here's mine (based upon your command-line):
qemu-system-aarch64 -cpu host -accel kvm -machine
virt,gic-version=3,confidential-guest-support=rme0 -smp 2 -m 256M
-nographic -object rme-guest,id=rme0,measurement-algo=sha512 -kernel
Image -initrd rootfs.ext2 -append 'console=ttyAMA0 earlycon'
-overcommit mem-lock=on

Itaru.
>
> > I configure it with the aarch64-softmmu target only and built, any
> > other steps I should worry?
>
> No, that should be enough
>
> Thanks,
> Jean

2023-03-02 09:18:30

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Thu, Mar 02, 2023 at 07:12:24AM +0900, Itaru Kitayama wrote:
> On Wed, Mar 1, 2023 at 6:20 PM Jean-Philippe Brucker
> <[email protected]> wrote:
> >
> > Hi Itaru,
> >
> > On Wed, Mar 01, 2023 at 08:35:05AM +0900, Itaru Kitayama wrote:
> > > Hi Jean,
> > > I've tried your series in Real on CCA Host, but the KVM arch init
> > > emits an Invalid argument error and terminates.
> >
> > Do you know which call returns this error? Normally the RMEGuest support
> > should print more detailed errors. Are you able to launch normal guests
> > (without the rme-guest object and confidential-guest-support machine
> > parameter)? Could you give the complete QEMU command-line?
>
> No, I cant say which. Yes, the CCA-capable QEMU boots if I don't set
> RME-related options.
>
> Here's mine (based upon your command-line):
> qemu-system-aarch64 -cpu host -accel kvm -machine
> virt,gic-version=3,confidential-guest-support=rme0 -smp 2 -m 256M
> -nographic -object rme-guest,id=rme0,measurement-algo=sha512 -kernel
> Image -initrd rootfs.ext2 -append 'console=ttyAMA0 earlycon'
> -overcommit mem-lock=on

Thank you, this works on my setup so I'm not sure what's wrong. Check that
KVM initialized successfully, with this in the host kernel log:
"[ 0.267019] kvm [1]: Using prototype RMM support (version 56.0)"

Next step would be to find out where the EINVAL comes from, with printfs
or GDB. This seems rather specific so I'll email you directly to avoid
filling up everyone's inbox.

Thanks,
Jean

2023-03-02 14:03:31

by Piotr Sawicki

[permalink] [raw]

Subject: Re: [RFC kvmtool 18/31] arm64: Populate initial realm contents

Hi,

> From: Alexandru Elisei <[email protected]>
>
> Populate the realm memory with the initial contents, which include
> the device tree blob, the kernel image, and initrd, if specified,
> or the firmware image.
>
> Populating an image in the realm involves two steps:
> a) Mark the IPA area as RAM - INIT_IPA_REALM
> b) Load the contents into the IPA - POPULATE_REALM
>
> Wherever we know the actual size of an image in memory, we make
> sure the "memory area" is initialised to RAM.
> e.g., Linux kernel image size from the header which includes the bss etc.
> The "file size" on disk for the Linux image is much smaller.
> We mark the region of size Image.header.size as RAM (a), from the kernel
> load address. And load the Image file into the memory (b) above.
> At the moment we only detect the Arm64 Linux Image header format.
>
> Since we're already touching the code that copies the
> initrd in guest memory, let's do a bit of cleaning and remove a
> useless local variable.
>
> Signed-off-by: Alexandru Elisei <[email protected]>
> [ Make sure the Linux kernel image area is marked as RAM ]
> Signed-off-by: Suzuki K Poulose <[email protected]>
> ---
> arm/aarch32/include/asm/realm.h | 3 +
> arm/aarch64/include/asm/realm.h | 3 +
> arm/aarch64/realm.c | 112 ++++++++++++++++++++++++++++++++
> arm/fdt.c | 6 ++
> arm/kvm.c | 20 ++++--
> include/linux/kernel.h | 1 +
> 6 files changed, 140 insertions(+), 5 deletions(-)
>
> diff --git a/arm/aarch32/include/asm/realm.h b/arm/aarch32/include/asm/realm.h
> index 5aca6cca..fcff0e55 100644
> --- a/arm/aarch32/include/asm/realm.h
> +++ b/arm/aarch32/include/asm/realm.h
> @@ -6,5 +6,8 @@
> #include "kvm/kvm.h"
>
> static inline void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm) {}
> +static inline void kvm_arm_realm_populate_kernel(struct kvm *kvm) {}
> +static inline void kvm_arm_realm_populate_initrd(struct kvm *kvm) {}
> +static inline void kvm_arm_realm_populate_dtb(struct kvm *kvm) {}
>
> #endif /* ! __ASM_REALM_H */
> diff --git a/arm/aarch64/include/asm/realm.h b/arm/aarch64/include/asm/realm.h
> index e176f15f..6e760ac9 100644
> --- a/arm/aarch64/include/asm/realm.h
> +++ b/arm/aarch64/include/asm/realm.h
> @@ -6,5 +6,8 @@
> #include "kvm/kvm.h"
>
> void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm);
> +void kvm_arm_realm_populate_kernel(struct kvm *kvm);
> +void kvm_arm_realm_populate_initrd(struct kvm *kvm);
> +void kvm_arm_realm_populate_dtb(struct kvm *kvm);
>
> #endif /* ! __ASM_REALM_H */
> diff --git a/arm/aarch64/realm.c b/arm/aarch64/realm.c
> index fc7f8d6a..eddccece 100644
> --- a/arm/aarch64/realm.c
> +++ b/arm/aarch64/realm.c
> @@ -1,5 +1,7 @@
> #include "kvm/kvm.h"
>
> +#include <linux/byteorder.h>
> +#include <asm/image.h>
> #include <asm/realm.h>
>
>
> @@ -80,3 +82,113 @@ void kvm_arm_realm_create_realm_descriptor(struct kvm *kvm)
> if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &rme_create_rd) < 0)
> die_perror("KVM_CAP_RME(KVM_CAP_ARM_RME_CREATE_RD)");
> }
> +
> +static void realm_init_ipa_range(struct kvm *kvm, u64 start, u64 size)
> +{
> + struct kvm_cap_arm_rme_init_ipa_args init_ipa_args = {
> + .init_ipa_base = start,
> + .init_ipa_size = size
> + };
> + struct kvm_enable_cap rme_init_ipa_realm = {
> + .cap = KVM_CAP_ARM_RME,
> + .args[0] = KVM_CAP_ARM_RME_INIT_IPA_REALM,
> + .args[1] = (u64)&init_ipa_args
> + };
> +
> + if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &rme_init_ipa_realm) < 0)
> + die("unable to intialise IPA range for Realm %llx - %llx (size %llu)",
> + start, start + size, size);
> +
> +}
> +
> +static void __realm_populate(struct kvm *kvm, u64 start, u64 size)
> +{
> + struct kvm_cap_arm_rme_populate_realm_args populate_args = {
> + .populate_ipa_base = start,
> + .populate_ipa_size = size
> + };
> + struct kvm_enable_cap rme_populate_realm = {
> + .cap = KVM_CAP_ARM_RME,
> + .args[0] = KVM_CAP_ARM_RME_POPULATE_REALM,
> + .args[1] = (u64)&populate_args
> + };
> +
> + if (ioctl(kvm->vm_fd, KVM_ENABLE_CAP, &rme_populate_realm) < 0)
> + die("unable to populate Realm memory %llx - %llx (size %llu)",
> + start, start + size, size);
> +}
> +
> +static void realm_populate(struct kvm *kvm, u64 start, u64 size)
> +{
> + realm_init_ipa_range(kvm, start, size);
> + __realm_populate(kvm, start, size);
> +}
> +
> +static bool is_arm64_linux_kernel_image(void *header)
> +{
> + struct arm64_image_header *hdr = header;
> +
> + return memcmp(&hdr->magic, ARM64_IMAGE_MAGIC, sizeof(hdr->magic)) == 0;
> +}
> +
> +static ssize_t arm64_linux_kernel_image_size(void *header)
> +{
> + struct arm64_image_header *hdr = header;
> +
> + if (is_arm64_linux_kernel_image(header))
> + return le64_to_cpu(hdr->image_size);
> + die("Not arm64 Linux kernel Image");
> +}
> +
> +void kvm_arm_realm_populate_kernel(struct kvm *kvm)
> +{
> + u64 start, end, mem_size;
> + void *header = guest_flat_to_host(kvm, kvm->arch.kern_guest_start);
> +
> + start = ALIGN_DOWN(kvm->arch.kern_guest_start, SZ_4K);
> + end = ALIGN(kvm->arch.kern_guest_start + kvm->arch.kern_size, SZ_4K);
> +
> + if (is_arm64_linux_kernel_image(header))
> + mem_size = arm64_linux_kernel_image_size(header);
> + else
> + mem_size = end - start;
> +
> + realm_init_ipa_range(kvm, start, mem_size);
> + __realm_populate(kvm, start, end - start);
> +}
> +
> +void kvm_arm_realm_populate_initrd(struct kvm *kvm)
> +{
> + u64 kernel_end, start, end;
> +
> + kernel_end = ALIGN(kvm->arch.kern_guest_start + kvm->arch.kern_size, SZ_4K);
> + start = ALIGN_DOWN(kvm->arch.initrd_guest_start, SZ_4K);
> + /*
> + * Because we align the initrd to 4 bytes, it is theoretically possible
> + * for the start of the initrd to overlap with the same page where the
> + * kernel ends.
> + */
> + if (start < kernel_end)
> + start = kernel_end;
> + end = ALIGN(kvm->arch.initrd_guest_start + kvm->arch.initrd_size, SZ_4K);
> + if (end > start)
> + realm_populate(kvm, start, end - start);
> +}
> +
> +void kvm_arm_realm_populate_dtb(struct kvm *kvm)
> +{
> + u64 initrd_end, start, end;
> +
> + initrd_end = ALIGN(kvm->arch.initrd_guest_start + kvm->arch.initrd_size, SZ_4K);
> + start = ALIGN_DOWN(kvm->arch.dtb_guest_start, SZ_4K);
> + /*
> + * Same situation as with the initrd, but now it is the DTB which is
> + * overlapping with the last page of the initrd, because the initrd is
> + * populated first.
> + */
> + if (start < initrd_end)
> + start = initrd_end;
> + end = ALIGN(kvm->arch.dtb_guest_start + FDT_MAX_SIZE, SZ_4K);
> + if (end > start)
> + realm_populate(kvm, start, end - start);
> +}
> diff --git a/arm/fdt.c b/arm/fdt.c
> index 286ccadf..762a604d 100644
> --- a/arm/fdt.c
> +++ b/arm/fdt.c
> @@ -7,6 +7,8 @@
> #include "arm-common/gic.h"
> #include "arm-common/pci.h"
>
> +#include <asm/realm.h>
> +
> #include <stdbool.h>
>
> #include <linux/byteorder.h>
> @@ -231,6 +233,10 @@ static int setup_fdt(struct kvm *kvm)
>
> if (kvm->cfg.arch.dump_dtb_filename)
> dump_fdt(kvm->cfg.arch.dump_dtb_filename, fdt_dest);
> +
> + if (kvm->cfg.arch.is_realm)
> + kvm_arm_realm_populate_dtb(kvm);
> +
> return 0;
> }
> late_init(setup_fdt);
> diff --git a/arm/kvm.c b/arm/kvm.c
> index acb627b2..57c5b5f7 100644
> --- a/arm/kvm.c
> +++ b/arm/kvm.c
> @@ -6,6 +6,7 @@
> #include "kvm/fdt.h"
>
> #include "arm-common/gic.h"
> +#include <asm/realm.h>
>
> #include <sys/resource.h>
>
> @@ -167,6 +168,9 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm, int fd_kernel, int fd_initrd,
> pr_debug("Loaded kernel to 0x%llx (%llu bytes)",
> kvm->arch.kern_guest_start, kvm->arch.kern_size);

I've noticed that multiple calling of the measurement test from the
kvm-unit-tests suite results in different Realm Initial Measurements,
although the kernel image is always the same.

After short investigation, I've found that the RIM starts being
different while populating the last 4kB chunk of the kernel image.
The issue occurs when the image size is not aligned to the page size (4kB).

After zeroing the unused area of the last chunk, the measurements become
repeatable.

> + if (kvm->cfg.arch.is_realm)
> + kvm_arm_realm_populate_kernel(kvm);
> +
> /*
> * Now load backwards from the end of memory so the kernel
> * decompressor has plenty of space to work with. First up is
> @@ -188,7 +192,6 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm, int fd_kernel, int fd_initrd,
> /* ... and finally the initrd, if we have one. */
> if (fd_initrd != -1) {
> struct stat sb;
> - unsigned long initrd_start;
>
> if (fstat(fd_initrd, &sb))
> die_perror("fstat");
> @@ -199,7 +202,6 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm, int fd_kernel, int fd_initrd,
> if (pos < kernel_end)
> die("initrd overlaps with kernel image.");
>
> - initrd_start = guest_addr;
> file_size = read_file(fd_initrd, pos, limit - pos);
> if (file_size == -1) {
> if (errno == ENOMEM)
> @@ -208,11 +210,13 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm, int fd_kernel, int fd_initrd,
> die_perror("initrd read");
> }
>
> - kvm->arch.initrd_guest_start = initrd_start;
> + kvm->arch.initrd_guest_start = guest_addr;
> kvm->arch.initrd_size = file_size;
> pr_debug("Loaded initrd to 0x%llx (%llu bytes)",
> - kvm->arch.initrd_guest_start,
> - kvm->arch.initrd_size);
> + kvm->arch.initrd_guest_start, kvm->arch.initrd_size);
> +
> + if (kvm->cfg.arch.is_realm)
> + kvm_arm_realm_populate_initrd(kvm);
> } else {
> kvm->arch.initrd_size = 0;
> }
> @@ -269,6 +273,8 @@ bool kvm__load_firmware(struct kvm *kvm, const char *firmware_filename)
>
> /* Kernel isn't loaded by kvm, point start address to firmware */
> kvm->arch.kern_guest_start = fw_addr;
> + kvm->arch.kern_size = fw_sz;
> +
> pr_debug("Loaded firmware to 0x%llx (%zd bytes)",
> kvm->arch.kern_guest_start, fw_sz);
>
> @@ -283,6 +289,10 @@ bool kvm__load_firmware(struct kvm *kvm, const char *firmware_filename)
> kvm->arch.dtb_guest_start,
> kvm->arch.dtb_guest_start + FDT_MAX_SIZE);
>
> + if (kvm->cfg.arch.is_realm)
> + /* We hijack the kernel fields to describe the firmware. */
> + kvm_arm_realm_populate_kernel(kvm);
> +
> return true;
> }
>
> diff --git a/include/linux/kernel.h b/include/linux/kernel.h
> index 6c22f1c0..25f19c20 100644
> --- a/include/linux/kernel.h
> +++ b/include/linux/kernel.h
> @@ -9,6 +9,7 @@
>
> #define DIV_ROUND_UP(n,d) (((n) + (d) - 1) / (d))
>
> +#define ALIGN_DOWN(x,a) __ALIGN_MASK(x - (typeof(x))((a) - 1),(typeof(x))(a)-1)
> #define ALIGN(x,a) __ALIGN_MASK(x,(typeof(x))(a)-1)
> #define __ALIGN_MASK(x,mask) (((x)+(mask))&~(mask))
> #define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)

Kind regards,
Piotr Sawicki

2023-03-02 14:06:45

[permalink] [raw]

Subject: Re: [RFC kvmtool 18/31] arm64: Populate initial realm contents

Hi Piotr

On 02/03/2023 14:03, Piotr Sawicki wrote:
> Hi,
>
>> From: Alexandru Elisei <[email protected]>
>>
>> Populate the realm memory with the initial contents, which include
>> the device tree blob, the kernel image, and initrd, if specified,
>> or the firmware image.
>>
>> Populating an image in the realm involves two steps:
>> a) Mark the IPA area as RAM - INIT_IPA_REALM
>> b) Load the contents into the IPA - POPULATE_REALM
>>
>> Wherever we know the actual size of an image in memory, we make
>> sure the "memory area" is initialised to RAM.
>> e.g., Linux kernel image size from the header which includes the bss etc.
>> The "file size" on disk for the Linux image is much smaller.
>> We mark the region of size Image.header.size as RAM (a), from the kernel
>> load address. And load the Image file into the memory (b) above.
>> At the moment we only detect the Arm64 Linux Image header format.
>>
>> Since we're already touching the code that copies the
>> initrd in guest memory, let's do a bit of cleaning and remove a
>> useless local variable.
>>
>> Signed-off-by: Alexandru Elisei <[email protected]>
>> [ Make sure the Linux kernel image area is marked as RAM ]
>> Signed-off-by: Suzuki K Poulose <[email protected]>

>> diff --git a/arm/kvm.c b/arm/kvm.c
>> index acb627b2..57c5b5f7 100644
>> --- a/arm/kvm.c
>> +++ b/arm/kvm.c
>> @@ -6,6 +6,7 @@
>> #include "kvm/fdt.h"
>> #include "arm-common/gic.h"
>> +#include <asm/realm.h>
>> #include <sys/resource.h>
>> @@ -167,6 +168,9 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm,
>> int fd_kernel, int fd_initrd,
>> pr_debug("Loaded kernel to 0x%llx (%llu bytes)",
>> kvm->arch.kern_guest_start, kvm->arch.kern_size);
>
>
> I've noticed that multiple calling of the measurement test from the
> kvm-unit-tests suite results in different Realm Initial Measurements,
> although the kernel image is always the same.
>
> After short investigation, I've found that the RIM starts being
> different while populating the last 4kB chunk of the kernel image.
> The issue occurs when the image size is not aligned to the page size (4kB).
>
> After zeroing the unused area of the last chunk, the measurements become
> repeatable.
>

That is a good point. We could memset() the remaining bits of the 4K
page to 0. I will make this change.

Suzuki

2023-03-02 16:47:31

by Dr. David Alan Gilbert

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

* Suzuki K Poulose ([email protected]) wrote:
> Hi Dave
>
> Thanks for your response, and apologies for the delay. Response, in line.
>
> On 14/02/2023 17:13, Dr. David Alan Gilbert wrote:
> > * Suzuki K Poulose ([email protected]) wrote:
> > > We are happy to announce the early RFC version of the Arm
> > > Confidential Compute Architecture (CCA) support for the Linux
> > > stack. The intention is to seek early feedback in the following areas:
> > > * KVM integration of the Arm CCA
> > > * KVM UABI for managing the Realms, seeking to generalise the operations
> > > wherever possible with other Confidential Compute solutions.
> > > Note: This version doesn't support Guest Private memory, which will be added
> > > later (see below).
> > > * Linux Guest support for Realms
> > >
> > > Arm CCA Introduction
> > > =====================
> > >
> > > The Arm CCA is a reference software architecture and implementation that builds
> > > on the Realm Management Extension (RME), enabling the execution of Virtual
> > > machines, while preventing access by more privileged software, such as hypervisor.
> > > The Arm CCA allows the hypervisor to control the VM, but removes the right for
> > > access to the code, register state or data that is used by VM.
> > > More information on the architecture is available here[0].
> > >
> > > Arm CCA Reference Software Architecture
> > >
> > > Realm World || Normal World || Secure World ||
> > > || | || ||
> > > EL0 x-------x || x----x | x------x || ||
> > > | Realm | || | | | | | || ||
> > > | | || | VM | | | | || ||
> > > ----| VM* |---------||-| |---| |-||----------------||
> > > | | || | | | | H | || ||
> > > EL1 x-------x || x----x | | | || ||
> > > ^ || | | o | || ||
> > > | || | | | || ||
> > > ------- R*------------------------| s -|---------------------
> > > S || | | || ||
> > > I || | t | || ||
> > > | || | | || ||
> > > v || x------x || ||
> > > EL2 RMM* || ^ || ||
> > > ^ || | || ||
> > > ========|=============================|========================
> > > | | SMC
> > > x--------- *RMI* -------------x
> > >
> > > EL3 Root World
> > > EL3 Firmware
> > > ===============================================================
> > > Where :
> > > RMM - Realm Management Monitor
> > > RMI - Realm Management Interface
> > > RSI - Realm Service Interface
> > > SMC - Secure Monitor Call
> >
> > Hi,
> > It's nice to see this full stack posted - thanks!
> >
> > Are there any pointers to information on attestation and similar
> > measurement things? In particular, are there any plans for a vTPM
>
> The RMM v1.0 provides attestation and measurement services to the Realm,
> via Realm Service Interface (RSI) calls.

Can you point me at some docs for that?

> However, there is no support
> for partitioning the Realm VM with v1.0. This is currently under
> development and should be available in the near future.
>
> With that in place, a vTPM could reside in a partition of the Realm VM along
> side the OS in another. Does that answer your question ?

Possibly; it would be great to be able to use a standard vTPM interface
here rather than have to do anything special. People already have this
working on AMD SEV-SNP.

Dave

> Kind regards
> Suzuki
>
>
> > for Realms - if there were, it would make life easy for us, since we
> > can share some user space stuff with other CoCo systems.
> >
> > Dave
> >
> > > RME introduces a new security state "Realm world", in addition to the
> > > traditional Secure and Non-Secure states. The Arm CCA defines a new component,
> > > Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of
> > > firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at
> > > system boot.
> > >
> > > The RMM provides standard interfaces - Realm Management Interface (RMI) - to the
> > > Normal world hypervisor to manage the VMs running in the Realm world (also called
> > > Realms in short). These are exposed via SMC and are routed through the EL3
> > > firmwre.
> > > The RMI interface includes:
> > > - Move a physical page from the Normal world to the Realm world
> > > - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD)
> > > - Creating VCPUs aka Realm Execution Context (REC), with initial register state.
> > > - Create stage2 translation table at any level.
> > > - Load initial images into Realm Memory from normal world memory
> > > - Schedule RECs (vCPUs) and handle exits
> > > - Inject virtual interrupts into the Realm
> > > - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM).
> > > - Create "shared" mappings that can be accessed by VMM/Hyp.
> > > - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables)
> > >
> > > However v1.0 of RMM specifications doesn't support:
> > > - Paging protected memory of a Realm VM. Thus the pages backing the protected
> > > memory region must be pinned.
> > > - Live migration of Realms.
> > > - Trusted Device assignment.
> > > - Physical interrupt backed Virtual interrupts for Realms
> > >
> > > RMM also provides certain services to the Realms via SMC, called Realm Service
> > > Interface (RSI). These include:
> > > - Realm Guest Configuration.
> > > - Attestation & Measurement services
> > > - Managing the state of an Intermediate Physical Address (IPA aka GPA) page.
> > > - Host Call service (Communication with the Normal world Hypervisor)
> > >
> > > The specifications for the RMM software is currently at *v1.0-Beta2* and the
> > > latest version is available here [1].
> > >
> > > The Trusted Firmware foundation has an implementation of the RMM - TF-RMM -
> > > available here [3].
> > >
> > > Implementation
> > > =================
> > >
> > > This version of the stack is based on the RMM specification v1.0-Beta0[2], with
> > > following exceptions :
> > > - TF-RMM/KVM currently doesn't support the optional features of PMU,
> > > SVE and Self-hosted debug (coming soon).
> > > - The RSI_HOST_CALL structure alignment requirement is reduced to match
> > > RMM v1.0 Beta1
> > > - RMI/RSI version numbers do not match the RMM spec. This will be
> > > resolved once the spec/implementation is complete, across TF-RMM+Linux stack.
> > >
> > > We plan to update the stack to support the latest version of the RMMv1.0 spec
> > > in the coming revisions.
> > >
> > > This release includes the following components :
> > >
> > > a) Linux Kernel
> > > i) Host / KVM support - Support for driving the Realms via RMI. This is
> > > dependent on running in the Kernel at EL2 (aka VHE mode). Also provides
> > > UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page
> > > size, matching the Stage2 granule supported by RMM. The VMM is responsible
> > > for making sure the guest memory is locked.
> > >
> > > TODO: Guest Private memory[10] integration - We have been following the
> > > series and support will be added once it is merged upstream.
> > > ii) Guest support - Support for a Linux Kernel to run in the Realm VM at
> > > Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0
> > > only). All I/O are treated as non-secure/shared.
> > > c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory
> > > as mentioned above.
> > > d) kvm-unit-tests - Support for running in Realms along with additional tests
> > > for RSI ABI.
> > >
> > > Running the stack
> > > ====================
> > >
> > > To run/test the stack, you would need the following components :
> > >
> > > 1) FVP Base AEM RevC model with FEAT_RME support [4]
> > > 2) TF-A firmware for EL3 [5]
> > > 3) TF-A RMM for R-EL2 [3]
> > > 4) Linux Kernel [6]
> > > 5) kvmtool [7]
> > > 6) kvm-unit-tests [8]
> > >
> > > Instructions for building the firmware components and running the model are
> > > available here [9]. Once, the host kernel is booted, a Realm can be launched by
> > > invoking the `lkvm` commad as follows:
> > >
> > > $ lkvm run --realm \
> > > --measurement-algo=["sha256", "sha512"] \
> > > --disable-sve \
> > > <normal-vm-options>
> > >
> > > Where:
> > > * --measurement-algo (Optional) specifies the algorithm selected for creating the
> > > initial measurements by the RMM for this Realm (defaults to sha256).
> > > * GICv3 is mandatory for the Realms.
> > > * SVE is not yet supported in the TF-RMM, and thus must be disabled using
> > > --disable-sve
> > >
> > > You may also run the kvm-unit-tests inside the Realm world, using the similar
> > > options as above.
> > >
> > >
> > > Links
> > > ============
> > >
> > > [0] Arm CCA Landing page (See Key Resources section for various documentations)
> > > https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture
> > >
> > > [1] RMM Specification Latest
> > > https://developer.arm.com/documentation/den0137/latest
> > >
> > > [2] RMM v1.0-Beta0 specification
> > > https://developer.arm.com/documentation/den0137/1-0bet0/
> > >
> > > [3] Trusted Firmware RMM - TF-RMM
> > > https://www.trustedfirmware.org/projects/tf-rmm/
> > > GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
> > >
> > > [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
> > > https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
> > >
> > > [5] Trusted Firmware for A class
> > > https://www.trustedfirmware.org/projects/tf-a/
> > >
> > > [6] Linux kernel support for Arm-CCA
> > > https://gitlab.arm.com/linux-arm/linux-cca
> > > Host Support branch: cca-host/rfc-v1
> > > Guest Support branch: cca-guest/rfc-v1
> > >
> > > [7] kvmtool support for Arm CCA
> > > https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1
> > >
> > > [8] kvm-unit-tests support for Arm CCA
> > > https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1
> > >
> > > [9] Instructions for Building Firmware components and running the model, see
> > > section 4.19.2 "Building and running TF-A with RME"
> > > https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
> > >
> > > [10] fd based Guest Private memory for KVM
> > > https://lkml.kernel.org/r/[email protected]
> > >
> > > Cc: Alexandru Elisei <[email protected]>
> > > Cc: Andrew Jones <[email protected]>
> > > Cc: Catalin Marinas <[email protected]>
> > > Cc: Chao Peng <[email protected]>
> > > Cc: Christoffer Dall <[email protected]>
> > > Cc: Fuad Tabba <[email protected]>
> > > Cc: James Morse <[email protected]>
> > > Cc: Jean-Philippe Brucker <[email protected]>
> > > Cc: Joey Gouly <[email protected]>
> > > Cc: Marc Zyngier <[email protected]>
> > > Cc: Mark Rutland <[email protected]>
> > > Cc: Oliver Upton <[email protected]>
> > > Cc: Paolo Bonzini <[email protected]>
> > > Cc: Quentin Perret <[email protected]>
> > > Cc: Sean Christopherson <[email protected]>
> > > Cc: Steven Price <[email protected]>
> > > Cc: Thomas Huth <[email protected]>
> > > Cc: Will Deacon <[email protected]>
> > > Cc: Zenghui Yu <[email protected]>
> > > To: [email protected]
> > > To: [email protected]
> > > Cc: [email protected]
> > > Cc: [email protected]
> > > To: [email protected]
> > > To: [email protected]
> > >
>
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

2023-03-02 19:02:51

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On 02/03/2023 16:46, Dr. David Alan Gilbert wrote:
> * Suzuki K Poulose ([email protected]) wrote:
>> Hi Dave
>>
>> Thanks for your response, and apologies for the delay. Response, in line.
>>
>> On 14/02/2023 17:13, Dr. David Alan Gilbert wrote:
>>> * Suzuki K Poulose ([email protected]) wrote:
>>>> We are happy to announce the early RFC version of the Arm
>>>> Confidential Compute Architecture (CCA) support for the Linux
>>>> stack. The intention is to seek early feedback in the following areas:
>>>> * KVM integration of the Arm CCA
>>>> * KVM UABI for managing the Realms, seeking to generalise the operations
>>>> wherever possible with other Confidential Compute solutions.
>>>> Note: This version doesn't support Guest Private memory, which will be added
>>>> later (see below).
>>>> * Linux Guest support for Realms
>>>>
>>>> Arm CCA Introduction
>>>> =====================
>>>>
>>>> The Arm CCA is a reference software architecture and implementation that builds
>>>> on the Realm Management Extension (RME), enabling the execution of Virtual
>>>> machines, while preventing access by more privileged software, such as hypervisor.
>>>> The Arm CCA allows the hypervisor to control the VM, but removes the right for
>>>> access to the code, register state or data that is used by VM.
>>>> More information on the architecture is available here[0].
>>>>
>>>> Arm CCA Reference Software Architecture
>>>>
>>>> Realm World || Normal World || Secure World ||
>>>> || | || ||
>>>> EL0 x-------x || x----x | x------x || ||
>>>> | Realm | || | | | | | || ||
>>>> | | || | VM | | | | || ||
>>>> ----| VM* |---------||-| |---| |-||----------------||
>>>> | | || | | | | H | || ||
>>>> EL1 x-------x || x----x | | | || ||
>>>> ^ || | | o | || ||
>>>> | || | | | || ||
>>>> ------- R*------------------------| s -|---------------------
>>>> S || | | || ||
>>>> I || | t | || ||
>>>> | || | | || ||
>>>> v || x------x || ||
>>>> EL2 RMM* || ^ || ||
>>>> ^ || | || ||
>>>> ========|=============================|========================
>>>> | | SMC
>>>> x--------- *RMI* -------------x
>>>>
>>>> EL3 Root World
>>>> EL3 Firmware
>>>> ===============================================================
>>>> Where :
>>>> RMM - Realm Management Monitor
>>>> RMI - Realm Management Interface
>>>> RSI - Realm Service Interface
>>>> SMC - Secure Monitor Call
>>>
>>> Hi,
>>> It's nice to see this full stack posted - thanks!
>>>
>>> Are there any pointers to information on attestation and similar
>>> measurement things? In particular, are there any plans for a vTPM
>>
>> The RMM v1.0 provides attestation and measurement services to the Realm,
>> via Realm Service Interface (RSI) calls.
>
> Can you point me at some docs for that?
>

It is part of the RMM specification [1], linked below.
Please see "Chapter A7. Realm Measurement and Attestation"

[1] https://developer.arm.com/documentation/den0137/latest

>> However, there is no support
>> for partitioning the Realm VM with v1.0. This is currently under
>> development and should be available in the near future.
>>
>> With that in place, a vTPM could reside in a partition of the Realm VM along
>> side the OS in another. Does that answer your question ?
>
> Possibly; it would be great to be able to use a standard vTPM interface
> here rather than have to do anything special. People already have this
> working on AMD SEV-SNP.

Ok.

>
> Dave

...

>>>>
>>>> [1] RMM Specification Latest
>>>> https://developer.arm.com/documentation/den0137/latest

Suzuki

>>>>
>>>> [2] RMM v1.0-Beta0 specification
>>>> https://developer.arm.com/documentation/den0137/1-0bet0/
>>>>
>>>> [3] Trusted Firmware RMM - TF-RMM
>>>> https://www.trustedfirmware.org/projects/tf-rmm/
>>>> GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
>>>>
>>>> [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
>>>> https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
>>>>
>>>> [5] Trusted Firmware for A class
>>>> https://www.trustedfirmware.org/projects/tf-a/
>>>>
>>>> [6] Linux kernel support for Arm-CCA
>>>> https://gitlab.arm.com/linux-arm/linux-cca
>>>> Host Support branch: cca-host/rfc-v1
>>>> Guest Support branch: cca-guest/rfc-v1
>>>>
>>>> [7] kvmtool support for Arm CCA
>>>> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1
>>>>
>>>> [8] kvm-unit-tests support for Arm CCA
>>>> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1
>>>>
>>>> [9] Instructions for Building Firmware components and running the model, see
>>>> section 4.19.2 "Building and running TF-A with RME"
>>>> https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
>>>>
>>>> [10] fd based Guest Private memory for KVM
>>>> https://lkml.kernel.org/r/[email protected]
>>>>
>>>> Cc: Alexandru Elisei <[email protected]>
>>>> Cc: Andrew Jones <[email protected]>
>>>> Cc: Catalin Marinas <[email protected]>
>>>> Cc: Chao Peng <[email protected]>
>>>> Cc: Christoffer Dall <[email protected]>
>>>> Cc: Fuad Tabba <[email protected]>
>>>> Cc: James Morse <[email protected]>
>>>> Cc: Jean-Philippe Brucker <[email protected]>
>>>> Cc: Joey Gouly <[email protected]>
>>>> Cc: Marc Zyngier <[email protected]>
>>>> Cc: Mark Rutland <[email protected]>
>>>> Cc: Oliver Upton <[email protected]>
>>>> Cc: Paolo Bonzini <[email protected]>
>>>> Cc: Quentin Perret <[email protected]>
>>>> Cc: Sean Christopherson <[email protected]>
>>>> Cc: Steven Price <[email protected]>
>>>> Cc: Thomas Huth <[email protected]>
>>>> Cc: Will Deacon <[email protected]>
>>>> Cc: Zenghui Yu <[email protected]>
>>>> To: [email protected]
>>>> To: [email protected]
>>>> Cc: [email protected]
>>>> Cc: [email protected]
>>>> To: [email protected]
>>>> To: [email protected]
>>>>
>>

2023-03-03 09:46:23

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Thu, Mar 02, 2023 at 07:12:24AM +0900, Itaru Kitayama wrote:
> > > I've tried your series in Real on CCA Host, but the KVM arch init
> > > emits an Invalid argument error and terminates.

This was the KVM_SET_ONE_REG for the SVE vector size. During my tests I
didn't enable SVE in the host but shrinkwrap enables more options.

Until we figure out support for SVE, disable it on the QEMU command-line
(similarly to '--disable-sve' needed for kvmtool boot):

-cpu host,sve=off

Thanks,
Jean

2023-03-03 09:55:02

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On 03/03/2023 09:46, Jean-Philippe Brucker wrote:
> On Thu, Mar 02, 2023 at 07:12:24AM +0900, Itaru Kitayama wrote:
>>>> I've tried your series in Real on CCA Host, but the KVM arch init
>>>> emits an Invalid argument error and terminates.
>
> This was the KVM_SET_ONE_REG for the SVE vector size. During my tests I
> didn't enable SVE in the host but shrinkwrap enables more options.

Does the Qemu check for SVE capability on /dev/kvm ? For kvmtool, we
changed to using the VM instance and that would prevent using SVE,
until the RMM supports it.

Suzuki

>
> Until we figure out support for SVE, disable it on the QEMU command-line
> (similarly to '--disable-sve' needed for kvmtool boot):
>
> -cpu host,sve=off
>
> Thanks,
> Jean

2023-03-03 11:39:15

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Fri, Mar 03, 2023 at 09:54:47AM +0000, Suzuki K Poulose wrote:
> On 03/03/2023 09:46, Jean-Philippe Brucker wrote:
> > On Thu, Mar 02, 2023 at 07:12:24AM +0900, Itaru Kitayama wrote:
> > > > > I've tried your series in Real on CCA Host, but the KVM arch init
> > > > > emits an Invalid argument error and terminates.
> >
> > This was the KVM_SET_ONE_REG for the SVE vector size. During my tests I
> > didn't enable SVE in the host but shrinkwrap enables more options.
>
> Does the Qemu check for SVE capability on /dev/kvm ? For kvmtool, we
> changed to using the VM instance and that would prevent using SVE,
> until the RMM supports it.

Yes, QEMU does check the SVE cap on /dev/kvm. I can propose changing it or
complementing it with a VM check in my next version, it seems to work
(though I need to double-check the VM fd lifetime). Same goes for
KVM_CAP_STEAL_TIME, which I need to disable explicitly at the moment.

Thanks,
Jean

>
> Suzuki
>
> >
> > Until we figure out support for SVE, disable it on the QEMU command-line
> > (similarly to '--disable-sve' needed for kvmtool boot):
> >
> > -cpu host,sve=off
> >
> > Thanks,
> > Jean
>
>

2023-03-03 12:08:12

by Andrew Jones

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Fri, Mar 03, 2023 at 11:39:05AM +0000, Jean-Philippe Brucker wrote:
> On Fri, Mar 03, 2023 at 09:54:47AM +0000, Suzuki K Poulose wrote:
> > On 03/03/2023 09:46, Jean-Philippe Brucker wrote:
> > > On Thu, Mar 02, 2023 at 07:12:24AM +0900, Itaru Kitayama wrote:
> > > > > > I've tried your series in Real on CCA Host, but the KVM arch init
> > > > > > emits an Invalid argument error and terminates.
> > >
> > > This was the KVM_SET_ONE_REG for the SVE vector size. During my tests I
> > > didn't enable SVE in the host but shrinkwrap enables more options.
> >
> > Does the Qemu check for SVE capability on /dev/kvm ? For kvmtool, we
> > changed to using the VM instance and that would prevent using SVE,
> > until the RMM supports it.
>
> Yes, QEMU does check the SVE cap on /dev/kvm. I can propose changing it or
> complementing it with a VM check in my next version, it seems to work
> (though I need to double-check the VM fd lifetime). Same goes for
> KVM_CAP_STEAL_TIME, which I need to disable explicitly at the moment.

I'm probably missing something since I haven't looked at this, but I'm
wondering what the "VM instance" check is and why it should be necessary.
Shouldn't KVM only expose capabilities which it can provide? I.e. the
"VM instance" check should be done by KVM and, when it fails, the SVE and
steal-time capabilities should return 0.

Thanks,
drew

2023-03-03 12:19:56

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On 03/03/2023 12:08, Andrew Jones wrote:
> On Fri, Mar 03, 2023 at 11:39:05AM +0000, Jean-Philippe Brucker wrote:
>> On Fri, Mar 03, 2023 at 09:54:47AM +0000, Suzuki K Poulose wrote:
>>> On 03/03/2023 09:46, Jean-Philippe Brucker wrote:
>>>> On Thu, Mar 02, 2023 at 07:12:24AM +0900, Itaru Kitayama wrote:
>>>>>>> I've tried your series in Real on CCA Host, but the KVM arch init
>>>>>>> emits an Invalid argument error and terminates.
>>>>
>>>> This was the KVM_SET_ONE_REG for the SVE vector size. During my tests I
>>>> didn't enable SVE in the host but shrinkwrap enables more options.
>>>
>>> Does the Qemu check for SVE capability on /dev/kvm ? For kvmtool, we
>>> changed to using the VM instance and that would prevent using SVE,
>>> until the RMM supports it.
>>
>> Yes, QEMU does check the SVE cap on /dev/kvm. I can propose changing it or
>> complementing it with a VM check in my next version, it seems to work
>> (though I need to double-check the VM fd lifetime). Same goes for
>> KVM_CAP_STEAL_TIME, which I need to disable explicitly at the moment.
>
> I'm probably missing something since I haven't looked at this, but I'm
> wondering what the "VM instance" check is and why it should be necessary.

Userspace can check for a KVM_CAP_ on KVM fd (/dev/kvm) or a VM fd
(returned via KVM_CREATE_VM).

> Shouldn't KVM only expose capabilities which it can provide? I.e. the

Correct, given now that we have different "types" of VMs possible on
Arm64, (Normal vs Realm vs pVM), the capabilities of each of these
could be different and thus we should use the KVM_CAP_ on the VM fd (
referred to VM instance above) and not the generic KVM fd.

> "VM instance" check should be done by KVM and, when it fails, the SVE and
> steal-time capabilities should return 0.
>

Correct.

Suzuki

> Thanks,
> drew

2023-03-03 13:07:04

by Cornelia Huck

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Fri, Mar 03 2023, Suzuki K Poulose <[email protected]> wrote:

> On 03/03/2023 12:08, Andrew Jones wrote:
>> On Fri, Mar 03, 2023 at 11:39:05AM +0000, Jean-Philippe Brucker wrote:
>>> On Fri, Mar 03, 2023 at 09:54:47AM +0000, Suzuki K Poulose wrote:
>>>> On 03/03/2023 09:46, Jean-Philippe Brucker wrote:
>>>>> On Thu, Mar 02, 2023 at 07:12:24AM +0900, Itaru Kitayama wrote:
>>>>>>>> I've tried your series in Real on CCA Host, but the KVM arch init
>>>>>>>> emits an Invalid argument error and terminates.
>>>>>
>>>>> This was the KVM_SET_ONE_REG for the SVE vector size. During my tests I
>>>>> didn't enable SVE in the host but shrinkwrap enables more options.
>>>>
>>>> Does the Qemu check for SVE capability on /dev/kvm ? For kvmtool, we
>>>> changed to using the VM instance and that would prevent using SVE,
>>>> until the RMM supports it.
>>>
>>> Yes, QEMU does check the SVE cap on /dev/kvm. I can propose changing it or
>>> complementing it with a VM check in my next version, it seems to work
>>> (though I need to double-check the VM fd lifetime). Same goes for
>>> KVM_CAP_STEAL_TIME, which I need to disable explicitly at the moment.
>>
>> I'm probably missing something since I haven't looked at this, but I'm
>> wondering what the "VM instance" check is and why it should be necessary.
>
> Userspace can check for a KVM_CAP_ on KVM fd (/dev/kvm) or a VM fd
> (returned via KVM_CREATE_VM).
>
>> Shouldn't KVM only expose capabilities which it can provide? I.e. the
>
> Correct, given now that we have different "types" of VMs possible on
> Arm64, (Normal vs Realm vs pVM), the capabilities of each of these
> could be different and thus we should use the KVM_CAP_ on the VM fd (
> referred to VM instance above) and not the generic KVM fd.

Using the vm ioctl is even encouraged in the doc for
KVM_CHECK_EXTENSION:

"Based on their initialization different VMs may have different capabilities.
It is thus encouraged to use the vm ioctl to query for capabilities"

It would probably make sense to convert QEMU to use the vm ioctl
everywhere (the wrapper falls back to the global version on failure
anyway.)

>
>> "VM instance" check should be done by KVM and, when it fails, the SVE and
>> steal-time capabilities should return 0.
>>
>
> Correct.
>
> Suzuki
>
>> Thanks,
>> drew

2023-03-03 13:57:34

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Fri, Mar 03, 2023 at 02:06:07PM +0100, Cornelia Huck wrote:
> On Fri, Mar 03 2023, Suzuki K Poulose <[email protected]> wrote:
>
> > On 03/03/2023 12:08, Andrew Jones wrote:
> >> On Fri, Mar 03, 2023 at 11:39:05AM +0000, Jean-Philippe Brucker wrote:
> >>> On Fri, Mar 03, 2023 at 09:54:47AM +0000, Suzuki K Poulose wrote:
> >>>> On 03/03/2023 09:46, Jean-Philippe Brucker wrote:
> >>>>> On Thu, Mar 02, 2023 at 07:12:24AM +0900, Itaru Kitayama wrote:
> >>>>>>>> I've tried your series in Real on CCA Host, but the KVM arch init
> >>>>>>>> emits an Invalid argument error and terminates.
> >>>>>
> >>>>> This was the KVM_SET_ONE_REG for the SVE vector size. During my tests I
> >>>>> didn't enable SVE in the host but shrinkwrap enables more options.
> >>>>
> >>>> Does the Qemu check for SVE capability on /dev/kvm ? For kvmtool, we
> >>>> changed to using the VM instance and that would prevent using SVE,
> >>>> until the RMM supports it.
> >>>
> >>> Yes, QEMU does check the SVE cap on /dev/kvm. I can propose changing it or
> >>> complementing it with a VM check in my next version, it seems to work
> >>> (though I need to double-check the VM fd lifetime). Same goes for
> >>> KVM_CAP_STEAL_TIME, which I need to disable explicitly at the moment.
> >>
> >> I'm probably missing something since I haven't looked at this, but I'm
> >> wondering what the "VM instance" check is and why it should be necessary.
> >
> > Userspace can check for a KVM_CAP_ on KVM fd (/dev/kvm) or a VM fd
> > (returned via KVM_CREATE_VM).
> >
> >> Shouldn't KVM only expose capabilities which it can provide? I.e. the
> >
> > Correct, given now that we have different "types" of VMs possible on
> > Arm64, (Normal vs Realm vs pVM), the capabilities of each of these
> > could be different and thus we should use the KVM_CAP_ on the VM fd (
> > referred to VM instance above) and not the generic KVM fd.
>
> Using the vm ioctl is even encouraged in the doc for
> KVM_CHECK_EXTENSION:
>
> "Based on their initialization different VMs may have different capabilities.
> It is thus encouraged to use the vm ioctl to query for capabilities"
>
> It would probably make sense to convert QEMU to use the vm ioctl
> everywhere (the wrapper falls back to the global version on failure
> anyway.)

Indeed, I'll see if I can come up with something generic, thanks for the
pointer

Thanks,
Jean

2023-07-14 13:55:08

by Jonathan Cameron

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Fri, 27 Jan 2023 11:22:48 +0000
Suzuki K Poulose <[email protected]> wrote:

Hi Suzuki,

Looking at this has been on the backlog for a while from our side and we are finally
getting to it. So before we dive in and given it's been 6 months, I wanted to check
if you expect to post a new version shortly or if there is a rebased tree available?

Jonathan

> We are happy to announce the early RFC version of the Arm
> Confidential Compute Architecture (CCA) support for the Linux
> stack. The intention is to seek early feedback in the following areas:
> * KVM integration of the Arm CCA
> * KVM UABI for managing the Realms, seeking to generalise the operations
> wherever possible with other Confidential Compute solutions.
> Note: This version doesn't support Guest Private memory, which will be added
> later (see below).
> * Linux Guest support for Realms
>
> Arm CCA Introduction
> =====================
>
> The Arm CCA is a reference software architecture and implementation that builds
> on the Realm Management Extension (RME), enabling the execution of Virtual
> machines, while preventing access by more privileged software, such as hypervisor.
> The Arm CCA allows the hypervisor to control the VM, but removes the right for
> access to the code, register state or data that is used by VM.
> More information on the architecture is available here[0].
>
> Arm CCA Reference Software Architecture
>
> Realm World || Normal World || Secure World ||
> || | || ||
> EL0 x-------x || x----x | x------x || ||
> | Realm | || | | | | | || ||
> | | || | VM | | | | || ||
> ----| VM* |---------||-| |---| |-||----------------||
> | | || | | | | H | || ||
> EL1 x-------x || x----x | | | || ||
> ^ || | | o | || ||
> | || | | | || ||
> ------- R*------------------------| s -|---------------------
> S || | | || ||
> I || | t | || ||
> | || | | || ||
> v || x------x || ||
> EL2 RMM* || ^ || ||
> ^ || | || ||
> ========|=============================|========================
> | | SMC
> x--------- *RMI* -------------x
>
> EL3 Root World
> EL3 Firmware
> ===============================================================
> Where :
> RMM - Realm Management Monitor
> RMI - Realm Management Interface
> RSI - Realm Service Interface
> SMC - Secure Monitor Call
>
> RME introduces a new security state "Realm world", in addition to the
> traditional Secure and Non-Secure states. The Arm CCA defines a new component,
> Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of
> firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at
> system boot.
>
> The RMM provides standard interfaces - Realm Management Interface (RMI) - to the
> Normal world hypervisor to manage the VMs running in the Realm world (also called
> Realms in short). These are exposed via SMC and are routed through the EL3
> firmwre.
> The RMI interface includes:
> - Move a physical page from the Normal world to the Realm world
> - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD)
> - Creating VCPUs aka Realm Execution Context (REC), with initial register state.
> - Create stage2 translation table at any level.
> - Load initial images into Realm Memory from normal world memory
> - Schedule RECs (vCPUs) and handle exits
> - Inject virtual interrupts into the Realm
> - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM).
> - Create "shared" mappings that can be accessed by VMM/Hyp.
> - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables)
>
> However v1.0 of RMM specifications doesn't support:
> - Paging protected memory of a Realm VM. Thus the pages backing the protected
> memory region must be pinned.
> - Live migration of Realms.
> - Trusted Device assignment.
> - Physical interrupt backed Virtual interrupts for Realms
>
> RMM also provides certain services to the Realms via SMC, called Realm Service
> Interface (RSI). These include:
> - Realm Guest Configuration.
> - Attestation & Measurement services
> - Managing the state of an Intermediate Physical Address (IPA aka GPA) page.
> - Host Call service (Communication with the Normal world Hypervisor)
>
> The specifications for the RMM software is currently at *v1.0-Beta2* and the
> latest version is available here [1].
>
> The Trusted Firmware foundation has an implementation of the RMM - TF-RMM -
> available here [3].
>
> Implementation
> =================
>
> This version of the stack is based on the RMM specification v1.0-Beta0[2], with
> following exceptions :
> - TF-RMM/KVM currently doesn't support the optional features of PMU,
> SVE and Self-hosted debug (coming soon).
> - The RSI_HOST_CALL structure alignment requirement is reduced to match
> RMM v1.0 Beta1
> - RMI/RSI version numbers do not match the RMM spec. This will be
> resolved once the spec/implementation is complete, across TF-RMM+Linux stack.
>
> We plan to update the stack to support the latest version of the RMMv1.0 spec
> in the coming revisions.
>
> This release includes the following components :
>
> a) Linux Kernel
> i) Host / KVM support - Support for driving the Realms via RMI. This is
> dependent on running in the Kernel at EL2 (aka VHE mode). Also provides
> UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page
> size, matching the Stage2 granule supported by RMM. The VMM is responsible
> for making sure the guest memory is locked.
>
> TODO: Guest Private memory[10] integration - We have been following the
> series and support will be added once it is merged upstream.
>
> ii) Guest support - Support for a Linux Kernel to run in the Realm VM at
> Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0
> only). All I/O are treated as non-secure/shared.
>
> c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory
> as mentioned above.
> d) kvm-unit-tests - Support for running in Realms along with additional tests
> for RSI ABI.
>
> Running the stack
> ====================
>
> To run/test the stack, you would need the following components :
>
> 1) FVP Base AEM RevC model with FEAT_RME support [4]
> 2) TF-A firmware for EL3 [5]
> 3) TF-A RMM for R-EL2 [3]
> 4) Linux Kernel [6]
> 5) kvmtool [7]
> 6) kvm-unit-tests [8]
>
> Instructions for building the firmware components and running the model are
> available here [9]. Once, the host kernel is booted, a Realm can be launched by
> invoking the `lkvm` commad as follows:
>
> $ lkvm run --realm \
> --measurement-algo=["sha256", "sha512"] \
> --disable-sve \
> <normal-vm-options>
>
> Where:
> * --measurement-algo (Optional) specifies the algorithm selected for creating the
> initial measurements by the RMM for this Realm (defaults to sha256).
> * GICv3 is mandatory for the Realms.
> * SVE is not yet supported in the TF-RMM, and thus must be disabled using
> --disable-sve
>
> You may also run the kvm-unit-tests inside the Realm world, using the similar
> options as above.
>
>
> Links
> ============
>
> [0] Arm CCA Landing page (See Key Resources section for various documentations)
> https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture
>
> [1] RMM Specification Latest
> https://developer.arm.com/documentation/den0137/latest
>
> [2] RMM v1.0-Beta0 specification
> https://developer.arm.com/documentation/den0137/1-0bet0/
>
> [3] Trusted Firmware RMM - TF-RMM
> https://www.trustedfirmware.org/projects/tf-rmm/
> GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
>
> [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
> https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
>
> [5] Trusted Firmware for A class
> https://www.trustedfirmware.org/projects/tf-a/
>
> [6] Linux kernel support for Arm-CCA
> https://gitlab.arm.com/linux-arm/linux-cca
> Host Support branch: cca-host/rfc-v1
> Guest Support branch: cca-guest/rfc-v1
>
> [7] kvmtool support for Arm CCA
> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1
>
> [8] kvm-unit-tests support for Arm CCA
> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1
>
> [9] Instructions for Building Firmware components and running the model, see
> section 4.19.2 "Building and running TF-A with RME"
> https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
>
> [10] fd based Guest Private memory for KVM
> https://lkml.kernel.org/r/[email protected]
>
> Cc: Alexandru Elisei <[email protected]>
> Cc: Andrew Jones <[email protected]>
> Cc: Catalin Marinas <[email protected]>
> Cc: Chao Peng <[email protected]>
> Cc: Christoffer Dall <[email protected]>
> Cc: Fuad Tabba <[email protected]>
> Cc: James Morse <[email protected]>
> Cc: Jean-Philippe Brucker <[email protected]>
> Cc: Joey Gouly <[email protected]>
> Cc: Marc Zyngier <[email protected]>
> Cc: Mark Rutland <[email protected]>
> Cc: Oliver Upton <[email protected]>
> Cc: Paolo Bonzini <[email protected]>
> Cc: Quentin Perret <[email protected]>
> Cc: Sean Christopherson <[email protected]>
> Cc: Steven Price <[email protected]>
> Cc: Thomas Huth <[email protected]>
> Cc: Will Deacon <[email protected]>
> Cc: Zenghui Yu <[email protected]>
> To: [email protected]
> To: [email protected]
> Cc: [email protected]
> Cc: [email protected]
> To: [email protected]
> To: [email protected]
>
> _______________________________________________
> linux-arm-kernel mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

2023-07-14 15:20:43

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

Hi Jonathan

On 14/07/2023 14:46, Jonathan Cameron wrote:
> On Fri, 27 Jan 2023 11:22:48 +0000
> Suzuki K Poulose <[email protected]> wrote:
>
>
> Hi Suzuki,
>
> Looking at this has been on the backlog for a while from our side and we are finally
> getting to it. So before we dive in and given it's been 6 months, I wanted to check
> if you expect to post a new version shortly or if there is a rebased tree available?

Thanks for your interest. We have been updating our trees to the latest
RMM specification (v1.0-eac2 now) and also rebasing Linux/KVM on top of
v6.5-rc1. We will post this as soon as we have all the components ready
(and the TF-RMM). At the earliest, this would be around early September.

That said, the revised version will have the following changes :
- Changes to the Stage2 management
- Changes to RMM memory management for Realm
- PMU/SVE support

Otherwise, most of the changes remain the same (e.g., UABI). Happy to
hear feedback on those areas.

Kind regards
Suzuki

>
> Jonathan
>
>> We are happy to announce the early RFC version of the Arm
>> Confidential Compute Architecture (CCA) support for the Linux
>> stack. The intention is to seek early feedback in the following areas:
>> * KVM integration of the Arm CCA
>> * KVM UABI for managing the Realms, seeking to generalise the operations
>> wherever possible with other Confidential Compute solutions.
>> Note: This version doesn't support Guest Private memory, which will be added
>> later (see below).
>> * Linux Guest support for Realms
>>
>> Arm CCA Introduction
>> =====================
>>
>> The Arm CCA is a reference software architecture and implementation that builds
>> on the Realm Management Extension (RME), enabling the execution of Virtual
>> machines, while preventing access by more privileged software, such as hypervisor.
>> The Arm CCA allows the hypervisor to control the VM, but removes the right for
>> access to the code, register state or data that is used by VM.
>> More information on the architecture is available here[0].
>>
>> Arm CCA Reference Software Architecture
>>
>> Realm World || Normal World || Secure World ||
>> || | || ||
>> EL0 x-------x || x----x | x------x || ||
>> | Realm | || | | | | | || ||
>> | | || | VM | | | | || ||
>> ----| VM* |---------||-| |---| |-||----------------||
>> | | || | | | | H | || ||
>> EL1 x-------x || x----x | | | || ||
>> ^ || | | o | || ||
>> | || | | | || ||
>> ------- R*------------------------| s -|---------------------
>> S || | | || ||
>> I || | t | || ||
>> | || | | || ||
>> v || x------x || ||
>> EL2 RMM* || ^ || ||
>> ^ || | || ||
>> ========|=============================|========================
>> | | SMC
>> x--------- *RMI* -------------x
>>
>> EL3 Root World
>> EL3 Firmware
>> ===============================================================
>> Where :
>> RMM - Realm Management Monitor
>> RMI - Realm Management Interface
>> RSI - Realm Service Interface
>> SMC - Secure Monitor Call
>>
>> RME introduces a new security state "Realm world", in addition to the
>> traditional Secure and Non-Secure states. The Arm CCA defines a new component,
>> Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of
>> firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at
>> system boot.
>>
>> The RMM provides standard interfaces - Realm Management Interface (RMI) - to the
>> Normal world hypervisor to manage the VMs running in the Realm world (also called
>> Realms in short). These are exposed via SMC and are routed through the EL3
>> firmwre.
>> The RMI interface includes:
>> - Move a physical page from the Normal world to the Realm world
>> - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD)
>> - Creating VCPUs aka Realm Execution Context (REC), with initial register state.
>> - Create stage2 translation table at any level.
>> - Load initial images into Realm Memory from normal world memory
>> - Schedule RECs (vCPUs) and handle exits
>> - Inject virtual interrupts into the Realm
>> - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM).
>> - Create "shared" mappings that can be accessed by VMM/Hyp.
>> - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables)
>>
>> However v1.0 of RMM specifications doesn't support:
>> - Paging protected memory of a Realm VM. Thus the pages backing the protected
>> memory region must be pinned.
>> - Live migration of Realms.
>> - Trusted Device assignment.
>> - Physical interrupt backed Virtual interrupts for Realms
>>
>> RMM also provides certain services to the Realms via SMC, called Realm Service
>> Interface (RSI). These include:
>> - Realm Guest Configuration.
>> - Attestation & Measurement services
>> - Managing the state of an Intermediate Physical Address (IPA aka GPA) page.
>> - Host Call service (Communication with the Normal world Hypervisor)
>>
>> The specifications for the RMM software is currently at *v1.0-Beta2* and the
>> latest version is available here [1].
>>
>> The Trusted Firmware foundation has an implementation of the RMM - TF-RMM -
>> available here [3].
>>
>> Implementation
>> =================
>>
>> This version of the stack is based on the RMM specification v1.0-Beta0[2], with
>> following exceptions :
>> - TF-RMM/KVM currently doesn't support the optional features of PMU,
>> SVE and Self-hosted debug (coming soon).
>> - The RSI_HOST_CALL structure alignment requirement is reduced to match
>> RMM v1.0 Beta1
>> - RMI/RSI version numbers do not match the RMM spec. This will be
>> resolved once the spec/implementation is complete, across TF-RMM+Linux stack.
>>
>> We plan to update the stack to support the latest version of the RMMv1.0 spec
>> in the coming revisions.
>>
>> This release includes the following components :
>>
>> a) Linux Kernel
>> i) Host / KVM support - Support for driving the Realms via RMI. This is
>> dependent on running in the Kernel at EL2 (aka VHE mode). Also provides
>> UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page
>> size, matching the Stage2 granule supported by RMM. The VMM is responsible
>> for making sure the guest memory is locked.
>>
>> TODO: Guest Private memory[10] integration - We have been following the
>> series and support will be added once it is merged upstream.
>>
>> ii) Guest support - Support for a Linux Kernel to run in the Realm VM at
>> Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0
>> only). All I/O are treated as non-secure/shared.
>>
>> c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory
>> as mentioned above.
>> d) kvm-unit-tests - Support for running in Realms along with additional tests
>> for RSI ABI.
>>
>> Running the stack
>> ====================
>>
>> To run/test the stack, you would need the following components :
>>
>> 1) FVP Base AEM RevC model with FEAT_RME support [4]
>> 2) TF-A firmware for EL3 [5]
>> 3) TF-A RMM for R-EL2 [3]
>> 4) Linux Kernel [6]
>> 5) kvmtool [7]
>> 6) kvm-unit-tests [8]
>>
>> Instructions for building the firmware components and running the model are
>> available here [9]. Once, the host kernel is booted, a Realm can be launched by
>> invoking the `lkvm` commad as follows:
>>
>> $ lkvm run --realm \
>> --measurement-algo=["sha256", "sha512"] \
>> --disable-sve \
>> <normal-vm-options>
>>
>> Where:
>> * --measurement-algo (Optional) specifies the algorithm selected for creating the
>> initial measurements by the RMM for this Realm (defaults to sha256).
>> * GICv3 is mandatory for the Realms.
>> * SVE is not yet supported in the TF-RMM, and thus must be disabled using
>> --disable-sve
>>
>> You may also run the kvm-unit-tests inside the Realm world, using the similar
>> options as above.
>>
>>
>> Links
>> ============
>>
>> [0] Arm CCA Landing page (See Key Resources section for various documentations)
>> https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture
>>
>> [1] RMM Specification Latest
>> https://developer.arm.com/documentation/den0137/latest
>>
>> [2] RMM v1.0-Beta0 specification
>> https://developer.arm.com/documentation/den0137/1-0bet0/
>>
>> [3] Trusted Firmware RMM - TF-RMM
>> https://www.trustedfirmware.org/projects/tf-rmm/
>> GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
>>
>> [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
>> https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
>>
>> [5] Trusted Firmware for A class
>> https://www.trustedfirmware.org/projects/tf-a/
>>
>> [6] Linux kernel support for Arm-CCA
>> https://gitlab.arm.com/linux-arm/linux-cca
>> Host Support branch: cca-host/rfc-v1
>> Guest Support branch: cca-guest/rfc-v1
>>
>> [7] kvmtool support for Arm CCA
>> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1
>>
>> [8] kvm-unit-tests support for Arm CCA
>> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1
>>
>> [9] Instructions for Building Firmware components and running the model, see
>> section 4.19.2 "Building and running TF-A with RME"
>> https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
>>
>> [10] fd based Guest Private memory for KVM
>> https://lkml.kernel.org/r/[email protected]
>>
>> Cc: Alexandru Elisei <[email protected]>
>> Cc: Andrew Jones <[email protected]>
>> Cc: Catalin Marinas <[email protected]>
>> Cc: Chao Peng <[email protected]>
>> Cc: Christoffer Dall <[email protected]>
>> Cc: Fuad Tabba <[email protected]>
>> Cc: James Morse <[email protected]>
>> Cc: Jean-Philippe Brucker <[email protected]>
>> Cc: Joey Gouly <[email protected]>
>> Cc: Marc Zyngier <[email protected]>
>> Cc: Mark Rutland <[email protected]>
>> Cc: Oliver Upton <[email protected]>
>> Cc: Paolo Bonzini <[email protected]>
>> Cc: Quentin Perret <[email protected]>
>> Cc: Sean Christopherson <[email protected]>
>> Cc: Steven Price <[email protected]>
>> Cc: Thomas Huth <[email protected]>
>> Cc: Will Deacon <[email protected]>
>> Cc: Zenghui Yu <[email protected]>
>> To: [email protected]
>> To: [email protected]
>> Cc: [email protected]
>> Cc: [email protected]
>> To: [email protected]
>> To: [email protected]
>>
>> _______________________________________________
>> linux-arm-kernel mailing list
>> [email protected]
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>

2023-07-14 16:57:03

by Jonathan Cameron

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Fri, 14 Jul 2023 16:03:37 +0100
Suzuki K Poulose <[email protected]> wrote:

> Hi Jonathan
>
> On 14/07/2023 14:46, Jonathan Cameron wrote:
> > On Fri, 27 Jan 2023 11:22:48 +0000
> > Suzuki K Poulose <[email protected]> wrote:
> >
> >
> > Hi Suzuki,
> >
> > Looking at this has been on the backlog for a while from our side and we are finally
> > getting to it. So before we dive in and given it's been 6 months, I wanted to check
> > if you expect to post a new version shortly or if there is a rebased tree available?
>
> Thanks for your interest. We have been updating our trees to the latest
> RMM specification (v1.0-eac2 now) and also rebasing Linux/KVM on top of
> v6.5-rc1. We will post this as soon as we have all the components ready
> (and the TF-RMM). At the earliest, this would be around early September.
>
> That said, the revised version will have the following changes :
> - Changes to the Stage2 management
> - Changes to RMM memory management for Realm
> - PMU/SVE support
>
> Otherwise, most of the changes remain the same (e.g., UABI). Happy to
> hear feedback on those areas.

Hi Suzuki,

Thanks for the update. If there is any chance of visibility of changes
via a git tree etc that would be great in the meantime. If not, such is life
and I'll try to wait patiently :) + we'll review the existing code.

Jonathan

>
>
> Kind regards
> Suzuki
>
> >
> > Jonathan
> >
> >> We are happy to announce the early RFC version of the Arm
> >> Confidential Compute Architecture (CCA) support for the Linux
> >> stack. The intention is to seek early feedback in the following areas:
> >> * KVM integration of the Arm CCA
> >> * KVM UABI for managing the Realms, seeking to generalise the operations
> >> wherever possible with other Confidential Compute solutions.
> >> Note: This version doesn't support Guest Private memory, which will be added
> >> later (see below).
> >> * Linux Guest support for Realms
> >>
> >> Arm CCA Introduction
> >> =====================
> >>
> >> The Arm CCA is a reference software architecture and implementation that builds
> >> on the Realm Management Extension (RME), enabling the execution of Virtual
> >> machines, while preventing access by more privileged software, such as hypervisor.
> >> The Arm CCA allows the hypervisor to control the VM, but removes the right for
> >> access to the code, register state or data that is used by VM.
> >> More information on the architecture is available here[0].
> >>
> >> Arm CCA Reference Software Architecture
> >>
> >> Realm World || Normal World || Secure World ||
> >> || | || ||
> >> EL0 x-------x || x----x | x------x || ||
> >> | Realm | || | | | | | || ||
> >> | | || | VM | | | | || ||
> >> ----| VM* |---------||-| |---| |-||----------------||
> >> | | || | | | | H | || ||
> >> EL1 x-------x || x----x | | | || ||
> >> ^ || | | o | || ||
> >> | || | | | || ||
> >> ------- R*------------------------| s -|---------------------
> >> S || | | || ||
> >> I || | t | || ||
> >> | || | | || ||
> >> v || x------x || ||
> >> EL2 RMM* || ^ || ||
> >> ^ || | || ||
> >> ========|=============================|========================
> >> | | SMC
> >> x--------- *RMI* -------------x
> >>
> >> EL3 Root World
> >> EL3 Firmware
> >> ===============================================================
> >> Where :
> >> RMM - Realm Management Monitor
> >> RMI - Realm Management Interface
> >> RSI - Realm Service Interface
> >> SMC - Secure Monitor Call
> >>
> >> RME introduces a new security state "Realm world", in addition to the
> >> traditional Secure and Non-Secure states. The Arm CCA defines a new component,
> >> Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of
> >> firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at
> >> system boot.
> >>
> >> The RMM provides standard interfaces - Realm Management Interface (RMI) - to the
> >> Normal world hypervisor to manage the VMs running in the Realm world (also called
> >> Realms in short). These are exposed via SMC and are routed through the EL3
> >> firmwre.
> >> The RMI interface includes:
> >> - Move a physical page from the Normal world to the Realm world
> >> - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD)
> >> - Creating VCPUs aka Realm Execution Context (REC), with initial register state.
> >> - Create stage2 translation table at any level.
> >> - Load initial images into Realm Memory from normal world memory
> >> - Schedule RECs (vCPUs) and handle exits
> >> - Inject virtual interrupts into the Realm
> >> - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM).
> >> - Create "shared" mappings that can be accessed by VMM/Hyp.
> >> - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables)
> >>
> >> However v1.0 of RMM specifications doesn't support:
> >> - Paging protected memory of a Realm VM. Thus the pages backing the protected
> >> memory region must be pinned.
> >> - Live migration of Realms.
> >> - Trusted Device assignment.
> >> - Physical interrupt backed Virtual interrupts for Realms
> >>
> >> RMM also provides certain services to the Realms via SMC, called Realm Service
> >> Interface (RSI). These include:
> >> - Realm Guest Configuration.
> >> - Attestation & Measurement services
> >> - Managing the state of an Intermediate Physical Address (IPA aka GPA) page.
> >> - Host Call service (Communication with the Normal world Hypervisor)
> >>
> >> The specifications for the RMM software is currently at *v1.0-Beta2* and the
> >> latest version is available here [1].
> >>
> >> The Trusted Firmware foundation has an implementation of the RMM - TF-RMM -
> >> available here [3].
> >>
> >> Implementation
> >> =================
> >>
> >> This version of the stack is based on the RMM specification v1.0-Beta0[2], with
> >> following exceptions :
> >> - TF-RMM/KVM currently doesn't support the optional features of PMU,
> >> SVE and Self-hosted debug (coming soon).
> >> - The RSI_HOST_CALL structure alignment requirement is reduced to match
> >> RMM v1.0 Beta1
> >> - RMI/RSI version numbers do not match the RMM spec. This will be
> >> resolved once the spec/implementation is complete, across TF-RMM+Linux stack.
> >>
> >> We plan to update the stack to support the latest version of the RMMv1.0 spec
> >> in the coming revisions.
> >>
> >> This release includes the following components :
> >>
> >> a) Linux Kernel
> >> i) Host / KVM support - Support for driving the Realms via RMI. This is
> >> dependent on running in the Kernel at EL2 (aka VHE mode). Also provides
> >> UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page
> >> size, matching the Stage2 granule supported by RMM. The VMM is responsible
> >> for making sure the guest memory is locked.
> >>
> >> TODO: Guest Private memory[10] integration - We have been following the
> >> series and support will be added once it is merged upstream.
> >>
> >> ii) Guest support - Support for a Linux Kernel to run in the Realm VM at
> >> Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0
> >> only). All I/O are treated as non-secure/shared.
> >>
> >> c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory
> >> as mentioned above.
> >> d) kvm-unit-tests - Support for running in Realms along with additional tests
> >> for RSI ABI.
> >>
> >> Running the stack
> >> ====================
> >>
> >> To run/test the stack, you would need the following components :
> >>
> >> 1) FVP Base AEM RevC model with FEAT_RME support [4]
> >> 2) TF-A firmware for EL3 [5]
> >> 3) TF-A RMM for R-EL2 [3]
> >> 4) Linux Kernel [6]
> >> 5) kvmtool [7]
> >> 6) kvm-unit-tests [8]
> >>
> >> Instructions for building the firmware components and running the model are
> >> available here [9]. Once, the host kernel is booted, a Realm can be launched by
> >> invoking the `lkvm` commad as follows:
> >>
> >> $ lkvm run --realm \
> >> --measurement-algo=["sha256", "sha512"] \
> >> --disable-sve \
> >> <normal-vm-options>
> >>
> >> Where:
> >> * --measurement-algo (Optional) specifies the algorithm selected for creating the
> >> initial measurements by the RMM for this Realm (defaults to sha256).
> >> * GICv3 is mandatory for the Realms.
> >> * SVE is not yet supported in the TF-RMM, and thus must be disabled using
> >> --disable-sve
> >>
> >> You may also run the kvm-unit-tests inside the Realm world, using the similar
> >> options as above.
> >>
> >>
> >> Links
> >> ============
> >>
> >> [0] Arm CCA Landing page (See Key Resources section for various documentations)
> >> https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture
> >>
> >> [1] RMM Specification Latest
> >> https://developer.arm.com/documentation/den0137/latest
> >>
> >> [2] RMM v1.0-Beta0 specification
> >> https://developer.arm.com/documentation/den0137/1-0bet0/
> >>
> >> [3] Trusted Firmware RMM - TF-RMM
> >> https://www.trustedfirmware.org/projects/tf-rmm/
> >> GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
> >>
> >> [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
> >> https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
> >>
> >> [5] Trusted Firmware for A class
> >> https://www.trustedfirmware.org/projects/tf-a/
> >>
> >> [6] Linux kernel support for Arm-CCA
> >> https://gitlab.arm.com/linux-arm/linux-cca
> >> Host Support branch: cca-host/rfc-v1
> >> Guest Support branch: cca-guest/rfc-v1
> >>
> >> [7] kvmtool support for Arm CCA
> >> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1
> >>
> >> [8] kvm-unit-tests support for Arm CCA
> >> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1
> >>
> >> [9] Instructions for Building Firmware components and running the model, see
> >> section 4.19.2 "Building and running TF-A with RME"
> >> https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
> >>
> >> [10] fd based Guest Private memory for KVM
> >> https://lkml.kernel.org/r/[email protected]
> >>
> >> Cc: Alexandru Elisei <[email protected]>
> >> Cc: Andrew Jones <[email protected]>
> >> Cc: Catalin Marinas <[email protected]>
> >> Cc: Chao Peng <[email protected]>
> >> Cc: Christoffer Dall <[email protected]>
> >> Cc: Fuad Tabba <[email protected]>
> >> Cc: James Morse <[email protected]>
> >> Cc: Jean-Philippe Brucker <[email protected]>
> >> Cc: Joey Gouly <[email protected]>
> >> Cc: Marc Zyngier <[email protected]>
> >> Cc: Mark Rutland <[email protected]>
> >> Cc: Oliver Upton <[email protected]>
> >> Cc: Paolo Bonzini <[email protected]>
> >> Cc: Quentin Perret <[email protected]>
> >> Cc: Sean Christopherson <[email protected]>
> >> Cc: Steven Price <[email protected]>
> >> Cc: Thomas Huth <[email protected]>
> >> Cc: Will Deacon <[email protected]>
> >> Cc: Zenghui Yu <[email protected]>
> >> To: [email protected]
> >> To: [email protected]
> >> Cc: [email protected]
> >> Cc: [email protected]
> >> To: [email protected]
> >> To: [email protected]
> >>
> >> _______________________________________________
> >> linux-arm-kernel mailing list
> >> [email protected]
> >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> >
>
>

2023-07-17 10:01:18

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On 14/07/2023 17:28, Jonathan Cameron wrote:
> On Fri, 14 Jul 2023 16:03:37 +0100
> Suzuki K Poulose <[email protected]> wrote:
>
>> Hi Jonathan
>>
>> On 14/07/2023 14:46, Jonathan Cameron wrote:
>>> On Fri, 27 Jan 2023 11:22:48 +0000
>>> Suzuki K Poulose <[email protected]> wrote:
>>>
>>>
>>> Hi Suzuki,
>>>
>>> Looking at this has been on the backlog for a while from our side and we are finally
>>> getting to it. So before we dive in and given it's been 6 months, I wanted to check
>>> if you expect to post a new version shortly or if there is a rebased tree available?
>>
>> Thanks for your interest. We have been updating our trees to the latest
>> RMM specification (v1.0-eac2 now) and also rebasing Linux/KVM on top of
>> v6.5-rc1. We will post this as soon as we have all the components ready
>> (and the TF-RMM). At the earliest, this would be around early September.
>>
>> That said, the revised version will have the following changes :
>> - Changes to the Stage2 management
>> - Changes to RMM memory management for Realm
>> - PMU/SVE support
>>
>> Otherwise, most of the changes remain the same (e.g., UABI). Happy to
>> hear feedback on those areas.
>
> Hi Suzuki,
>
> Thanks for the update. If there is any chance of visibility of changes
> via a git tree etc that would be great in the meantime. If not, such is life
> and I'll try to wait patiently :) + we'll review the existing code.

I am afraid not yet. Thanks for reviewing the changes :-)

Suzuki

2023-10-02 14:06:34

by Piotr Sawicki

[permalink] [raw]

Subject: Re: [RFC kvmtool 00/31] arm64: Support for Arm Confidential Compute Architecture

Hi Suzuki

> This series is an initial version of the support for running VMs under the
> Arm Confidential Compute Architecture. The purpose of the series is to gather
> feedback on the proposed UABI changes for running Confidential VMs with KVM.
> More information on the Arm CCA and instructions for how to get, build and run
> the entire software stack is available here [0].
>
> A new option, `--realm` is added to the the `run` command to mark the VM as a
> confidential compute VM. This version doesn't use the Guest private memory [1]
> support yet, instead uses normal anonymous/hugetlbfs backed memory. Our aim is
> to switch to the guest private memory for the Realm.
>
> The host including the kernel and kvmtool, must not access any memory allocated
> to the protected IPA of the Realm.
>
> The series adds the support for managing the lifecycle of the Realm, which includes:
> * Configuration
> * Creation of Realm (RD)
> * Load initial memory images
> * Creation of Realm Execution Contexts (RECs aka VCPUs)a
> * Activation of the Realm.
>
> Patches are split as follows :
>
> Patches 1 and 2 are fixes to existing code.
> Patch 3 adds a new option --nocompat to disable compat warnings
> Patches 4 - 6 are some preparations for Realm specific changes.
>
> The remaining patches adds Realm support and using the --realm option is
> enabled in patch 30.
>
> The v1.0 of the Realm Management Monitor (RMM) specification doesn't support
> paging protected memory of a Realm. Thus all of the memory backing the RAM
> is locked by the VMM.
>
> Since the IPA space of a Realm is split into Protected and Unprotected, with
> one alias of the other, the VMM doubles the IPA Size for a Realm VM.
>
> The KVM support for Arm CCA is advertised with a new cap KVM_CAP_ARM_RME.
> A new "VM type" field is defined in the vm_type for CREATE_VM ioctl to indicate
> that a VM is "Realm". Once the VM is created, the life cycle of the Realm is
> managed via KVM_ENABLE_CAP of KVM_CAP_ARM_RME.
>
> Command line options are also added to configure the Realm parameters.
> These include :
> - Hash algorithm for measurements
> - Realm personalisation value
> - SVE vector Length (Optional feature in v1.0 RMM spec. Not yet supported
> by the TF-RMM. coming soon).
>
> Support for PMU and self-hosted debug (number of watchpoint/breakpoit registers)
> are not supported yet in the KVM/RMM implementation. This will be added soon.
>
> The UABI doesn't support discovering the "supported" configuration values. In
> real world, the Realm configuration 'affects' the initial measurement of the
> Realms and which may be verified by a remote entity. Thus, the VMM is not at
> liberty to make choices for configurations based on the "host" capabilities.
> Instead, VMM should launch a Realm with the user requested parameters. If this
> cannot be satisfied, there is no point in running the Realm. We are happy to
> change this if there is interest.
>
> Special actions are required to load the initial memory images (e.g, kernel,
> firmware, DTB, initrd) in to the Realm memory.
>
> For VCPUs, we add a new feature KVM_ARM_VCPU_REC, which will be used to control
> the creation of the REC object (via KVM_ARM_VCPU_FINALIZE). This must be done
> after the initial register state of the VCPUs are set.
> RMM imposes an order in which the RECs are created. i.e., they must be created
> in the ascending order of the MPIDR. This is for now a responsibility of the
> VMM.
>
> Once the Realm images are loaded, VCPUs created, Realm is activated before
> the first vCPU is run.
>
> virtio for the Realms enforces VIRTIO_F_ACCESS_PLATFORM flag.
>
> Also, added support for injecting SEA into the VM for unhandled MMIO.
>

I wonder if there is a plan to develop a dedicated (stand-alone) tool
that allows a realm developer to calculate Realm Initial Measurements
for realms. I mean a tool that can be compiled and run on a Linux PC
machine.

As you know, the remote attestation mechanism requires a verifier to be
provisioned with reference values. In this case, a realm verifier should
have access to the initial reference measurement (RIM) of a realm that
is intended to be run on a remote Arm CCA platform.

The algorithm that measures the initial state of realms (RIM) is highly
sensitive to the content of a realm memory and the order of RMI
operations. This means that not only the content of populated realm
memory matters but also the implementation of the host components (e.g.
kvm, kvmtool/qemu).In the of kvmtool-cca, the layout of memory and the
content of DTB highly depend on the provided options (DTB is generated
in run-time). Unfortunately, the content of DTB also depends on the
linking order of object files (the order of DTB generation is imposed by
__attribute__((constructor)) that is used to register devices). This
complicates development of a separate tool for calculating RIM, as the
tool would have to emulate all quirks of the kvmtool.

One of the solution of retrieving Realm Initial Measurements seems to be
running the whole firmware/software (e.g. kvmtool/Linux host/TF-RMM)
stack on the FVP emulator and gathering the RIM directly from the
TF-RMM. This would require a realm developer to have access to the whole
firmware/software stack and the emulator of the CCA platform.

The other solution would require the implementation of a dedicated tool.
For instance, a sensible approach could be to extend the functionality
of kvmtool.

Is Arm going to develop a dedicated, stand-alone tool for calculating RIMs?

What is the recommended way of retrieving/calculating RIMs for realms?

Kind regards,
Piotr Sawicki

2023-10-02 15:29:38

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

Hi,

> We are happy to announce the early RFC version of the Arm
> Confidential Compute Architecture (CCA) support for the Linux
> stack. The intention is to seek early feedback in the following areas:
> * KVM integration of the Arm CCA
> * KVM UABI for managing the Realms, seeking to generalise the operations
> wherever possible with other Confidential Compute solutions.
> Note: This version doesn't support Guest Private memory, which will be added
> later (see below).
> * Linux Guest support for Realms
>

We have updated the stack for Arm CCA Linux support to RMM-v1.0-EAC2 (See links)
We are not posting the patches for review yet, as we plan to update our
stack to support the latest RMM-v1.0 specification, which includes some
functional changes to support PSCI monitoring by the VMM along with other
minor changes. All relevant components are updated on a new branch "rmm-v1.0-eac2"
Guest-mem support is not included, but is in progress.

Change log :
- KVM RMI support updated to v1.0-eac2, with optimisations to stage2 tear down
- Guest (Linux and kvm-unit-test) support for RSI compliant to v1.0-eac2
- SVE, PMU support for Realms

kvmtool :
- Dropped no-compat and switched to --loglevel (merged upstream)
- Support for SVE, --sve-vl for vector length

> Arm CCA Introduction
> =====================
>
> The Arm CCA is a reference software architecture and implementation that builds
> on the Realm Management Extension (RME), enabling the execution of Virtual
> machines, while preventing access by more privileged software, such as hypervisor.
> The Arm CCA allows the hypervisor to control the VM, but removes the right for
> access to the code, register state or data that is used by VM.
> More information on the architecture is available here[0].
>
> Arm CCA Reference Software Architecture
>
> Realm World || Normal World || Secure World ||
> || | || ||
> EL0 x-------x || x----x | x------x || ||
> | Realm | || | | | | | || ||
> | | || | VM | | | | || ||
> ----| VM* |---------||-| |---| |-||----------------||
> | | || | | | | H | || ||
> EL1 x-------x || x----x | | | || ||
> ^ || | | o | || ||
> | || | | | || ||
> ------- R*------------------------| s -|---------------------
> S || | | || ||
> I || | t | || ||
> | || | | || ||
> v || x------x || ||
> EL2 RMM* || ^ || ||
> ^ || | || ||
> ========|=============================|========================
> | | SMC
> x--------- *RMI* -------------x
>
> EL3 Root World
> EL3 Firmware
> ===============================================================
> Where :
> RMM - Realm Management Monitor
> RMI - Realm Management Interface
> RSI - Realm Service Interface
> SMC - Secure Monitor Call
>
> RME introduces a new security state "Realm world", in addition to the
> traditional Secure and Non-Secure states. The Arm CCA defines a new component,
> Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of
> firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at
> system boot.
>
> The RMM provides standard interfaces - Realm Management Interface (RMI) - to the
> Normal world hypervisor to manage the VMs running in the Realm world (also called
> Realms in short). These are exposed via SMC and are routed through the EL3
> firmwre.
> The RMI interface includes:
> - Move a physical page from the Normal world to the Realm world
> - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD)
> - Creating VCPUs aka Realm Execution Context (REC), with initial register state.
> - Create stage2 translation table at any level.
> - Load initial images into Realm Memory from normal world memory
> - Schedule RECs (vCPUs) and handle exits
> - Inject virtual interrupts into the Realm
> - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM).
> - Create "shared" mappings that can be accessed by VMM/Hyp.
> - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables)
>
> However v1.0 of RMM specifications doesn't support:
> - Paging protected memory of a Realm VM. Thus the pages backing the protected
> memory region must be pinned.
> - Live migration of Realms.
> - Trusted Device assignment.
> - Physical interrupt backed Virtual interrupts for Realms
>
> RMM also provides certain services to the Realms via SMC, called Realm Service
> Interface (RSI). These include:
> - Realm Guest Configuration.
> - Attestation & Measurement services
> - Managing the state of an Intermediate Physical Address (IPA aka GPA) page.
> - Host Call service (Communication with the Normal world Hypervisor)
>
> The specifications for the RMM software is currently at *v1.0-Beta2* and the
> latest version is available here [1].
>
> The Trusted Firmware foundation has an implementation of the RMM - TF-RMM -
> available here [3].
>
> Implementation
> =================
>
> This version of the stack is based on the RMM specification v1.0-Beta0[2], with
> following exceptions :
> - TF-RMM/KVM currently doesn't support the optional features of PMU,
> SVE and Self-hosted debug (coming soon).
> - The RSI_HOST_CALL structure alignment requirement is reduced to match
> RMM v1.0 Beta1
> - RMI/RSI version numbers do not match the RMM spec. This will be
> resolved once the spec/implementation is complete, across TF-RMM+Linux stack.
>
> We plan to update the stack to support the latest version of the RMMv1.0 spec
> in the coming revisions.
>
> This release includes the following components :
>
> a) Linux Kernel
> i) Host / KVM support - Support for driving the Realms via RMI. This is
> dependent on running in the Kernel at EL2 (aka VHE mode). Also provides
> UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page
> size, matching the Stage2 granule supported by RMM. The VMM is responsible
> for making sure the guest memory is locked.
>
> TODO: Guest Private memory[10] integration - We have been following the
> series and support will be added once it is merged upstream.
>
> ii) Guest support - Support for a Linux Kernel to run in the Realm VM at
> Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0
> only). All I/O are treated as non-secure/shared.
>
> c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory
> as mentioned above.
> d) kvm-unit-tests - Support for running in Realms along with additional tests
> for RSI ABI.
>
> Running the stack
> ====================
>
> To run/test the stack, you would need the following components :
>
> 1) FVP Base AEM RevC model with FEAT_RME support [4]
> 2) TF-A firmware for EL3 [5]
> 3) TF-A RMM for R-EL2 [3]
> 4) Linux Kernel [6]
> 5) kvmtool [7]
> 6) kvm-unit-tests [8]
>
> Instructions for building the firmware components and running the model are
> available here [9]. Once, the host kernel is booted, a Realm can be launched by
> invoking the `lkvm` commad as follows:
>
> $ lkvm run --realm \
> --measurement-algo=["sha256", "sha512"] \
> --disable-sve \

As noted above, this is no longer required.

> <normal-vm-options>
>
> Where:
> * --measurement-algo (Optional) specifies the algorithm selected for creating the
> initial measurements by the RMM for this Realm (defaults to sha256).
> * GICv3 is mandatory for the Realms.
> * SVE is not yet supported in the TF-RMM, and thus must be disabled using
> --disable-sve
>
> You may also run the kvm-unit-tests inside the Realm world, using the similar
> options as above.
>
>
> Links
> ============
>
> [0] Arm CCA Landing page (See Key Resources section for various documentations)
> https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture
>
> [1] RMM Specification Latest
> https://developer.arm.com/documentation/den0137/latest
>
> [2] RMM v1.0-Beta0 specification
> https://developer.arm.com/documentation/den0137/1-0bet0/

EAC2 spec: https://developer.arm.com/documentation/den0137/1-0eac2/
>
> [3] Trusted Firmware RMM - TF-RMM
> https://www.trustedfirmware.org/projects/tf-rmm/
> GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
>
> [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
> https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
>
> [5] Trusted Firmware for A class
> https://www.trustedfirmware.org/projects/tf-a/
>
> [6] Linux kernel support for Arm-CCA
> https://gitlab.arm.com/linux-arm/linux-cca
> Host Support branch: cca-host/rfc-v1

Update branch : cca-host/rmm-v1.0-eac2

> Guest Support branch: cca-guest/rfc-v1

Update branch : cca-guest/rmm-v1.0-eac2

Combined tree for host and guest is also available at: "cca-full/rmm-v1.0-eac2"

>
> [7] kvmtool support for Arm CCA
> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1

Update branch : cca/rmm-v1.0-eac2

>
> [8] kvm-unit-tests support for Arm CCA
> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1
>

Update branch : cca/rmm-v1.0-eac2

Suzuki

> [9] Instructions for Building Firmware components and running the model, see
> section 4.19.2 "Building and running TF-A with RME"
> https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
>
> [10] fd based Guest Private memory for KVM
> https://lkml.kernel.org/r/[email protected]

Cc: Alexandru Elisei <[email protected]>
Cc: Andrew Jones <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Chao Peng <[email protected]>
Cc: Christoffer Dall <[email protected]>
Cc: Fuad Tabba <[email protected]>
Cc: Jonathan Cameron <[email protected]>
Cc: James Morse <[email protected]>
Cc: Jean-Philippe Brucker <[email protected]>
Cc: Joey Gouly <[email protected]>
Cc: Marc Zyngier <[email protected]>
Cc: Mark Rutland <[email protected]>
Cc: Oliver Upton <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: Quentin Perret <[email protected]>
Cc: Sean Christopherson <[email protected]>
Cc: Steven Price <[email protected]>
Cc: Thomas Huth <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: Zenghui Yu <[email protected]>
To: [email protected]
To: [email protected]
Cc: [email protected]
To: [email protected]
To: [email protected]

2023-10-02 18:12:15

by Piotr Sawicki

[permalink] [raw]

Subject: Re: [RFC kvmtool 18/31] arm64: Populate initial realm contents

Hi Suzuki

> Hi Piotr
>
> On 02/03/2023 14:03, Piotr Sawicki wrote:
>> Hi,
>>
>>> From: Alexandru Elisei <[email protected]>
>>>
>>> Populate the realm memory with the initial contents, which include
>>> the device tree blob, the kernel image, and initrd, if specified,
>>> or the firmware image.
>>>
>>> Populating an image in the realm involves two steps:
>>> a) Mark the IPA area as RAM - INIT_IPA_REALM
>>> b) Load the contents into the IPA - POPULATE_REALM
>>>
>>> Wherever we know the actual size of an image in memory, we make
>>> sure the "memory area" is initialised to RAM.
>>> e.g., Linux kernel image size from the header which includes the bss
>>> etc.
>>> The "file size" on disk for the Linux image is much smaller.
>>> We mark the region of size Image.header.size as RAM (a), from the kernel
>>> load address. And load the Image file into the memory (b) above.
>>> At the moment we only detect the Arm64 Linux Image header format.
>>>
>>> Since we're already touching the code that copies the
>>> initrd in guest memory, let's do a bit of cleaning and remove a
>>> useless local variable.
>>>
>>> Signed-off-by: Alexandru Elisei <[email protected]>
>>> [ Make sure the Linux kernel image area is marked as RAM ]
>>> Signed-off-by: Suzuki K Poulose <[email protected]>
>
>
>>> diff --git a/arm/kvm.c b/arm/kvm.c
>>> index acb627b2..57c5b5f7 100644
>>> --- a/arm/kvm.c
>>> +++ b/arm/kvm.c
>>> @@ -6,6 +6,7 @@
>>> #include "kvm/fdt.h"
>>> #include "arm-common/gic.h"
>>> +#include <asm/realm.h>
>>> #include <sys/resource.h>
>>> @@ -167,6 +168,9 @@ bool kvm__arch_load_kernel_image(struct kvm *kvm,
>>> int fd_kernel, int fd_initrd,
>>> pr_debug("Loaded kernel to 0x%llx (%llu bytes)",
>>> kvm->arch.kern_guest_start, kvm->arch.kern_size);
>>
>>
>> I've noticed that multiple calling of the measurement test from the
>> kvm-unit-tests suite results in different Realm Initial Measurements,
>> although the kernel image is always the same.
>>
>> After short investigation, I've found that the RIM starts being
>> different while populating the last 4kB chunk of the kernel image.
>> The issue occurs when the image size is not aligned to the page size
>> (4kB).
>>
>> After zeroing the unused area of the last chunk, the measurements
>> become repeatable.
>>
>
> That is a good point. We could memset() the remaining bits of the 4K
> page to 0. I will make this change.

It looks that this is somewhat related to the implementation of the 9p
filesystem (Linux host and/or the FVP emulator).

I'm getting this issue only when the initrd and the guest kernel images
are located in the shared folder that uses the 9p filesystem. Moving
those files to the ramdisk (e.g. to the /root folder) and running lkvm
tool on them resolves the issue.

Kind regards,
Piotr Sawicki

2024-01-10 05:40:58

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Mon, Oct 02, 2023 at 01:43:11PM +0100, Suzuki K Poulose wrote:
> Hi,
>
>
> > We are happy to announce the early RFC version of the Arm
> > Confidential Compute Architecture (CCA) support for the Linux
> > stack. The intention is to seek early feedback in the following areas:
> > * KVM integration of the Arm CCA
> > * KVM UABI for managing the Realms, seeking to generalise the operations
> > wherever possible with other Confidential Compute solutions.
> > Note: This version doesn't support Guest Private memory, which will be added
> > later (see below).
> > * Linux Guest support for Realms
> >
>
> We have updated the stack for Arm CCA Linux support to RMM-v1.0-EAC2 (See links)
> We are not posting the patches for review yet, as we plan to update our
> stack to support the latest RMM-v1.0 specification, which includes some
> functional changes to support PSCI monitoring by the VMM along with other
> minor changes. All relevant components are updated on a new branch "rmm-v1.0-eac2"
> Guest-mem support is not included, but is in progress.
>
> Change log :
> - KVM RMI support updated to v1.0-eac2, with optimisations to stage2 tear down
> - Guest (Linux and kvm-unit-test) support for RSI compliant to v1.0-eac2
> - SVE, PMU support for Realms
>
> kvmtool :
> - Dropped no-compat and switched to --loglevel (merged upstream)
> - Support for SVE, --sve-vl for vector length
>
> > Arm CCA Introduction
> > =====================
> >
> > The Arm CCA is a reference software architecture and implementation that builds
> > on the Realm Management Extension (RME), enabling the execution of Virtual
> > machines, while preventing access by more privileged software, such as hypervisor.
> > The Arm CCA allows the hypervisor to control the VM, but removes the right for
> > access to the code, register state or data that is used by VM.
> > More information on the architecture is available here[0].
> >
> > Arm CCA Reference Software Architecture
> >
> > Realm World || Normal World || Secure World ||
> > || | || ||
> > EL0 x-------x || x----x | x------x || ||
> > | Realm | || | | | | | || ||
> > | | || | VM | | | | || ||
> > ----| VM* |---------||-| |---| |-||----------------||
> > | | || | | | | H | || ||
> > EL1 x-------x || x----x | | | || ||
> > ^ || | | o | || ||
> > | || | | | || ||
> > ------- R*------------------------| s -|---------------------
> > S || | | || ||
> > I || | t | || ||
> > | || | | || ||
> > v || x------x || ||
> > EL2 RMM* || ^ || ||
> > ^ || | || ||
> > ========|=============================|========================
> > | | SMC
> > x--------- *RMI* -------------x
> >
> > EL3 Root World
> > EL3 Firmware
> > ===============================================================
> > Where :
> > RMM - Realm Management Monitor
> > RMI - Realm Management Interface
> > RSI - Realm Service Interface
> > SMC - Secure Monitor Call
> >
> > RME introduces a new security state "Realm world", in addition to the
> > traditional Secure and Non-Secure states. The Arm CCA defines a new component,
> > Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of
> > firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at
> > system boot.
> >
> > The RMM provides standard interfaces - Realm Management Interface (RMI) - to the
> > Normal world hypervisor to manage the VMs running in the Realm world (also called
> > Realms in short). These are exposed via SMC and are routed through the EL3
> > firmwre.
> > The RMI interface includes:
> > - Move a physical page from the Normal world to the Realm world
> > - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD)
> > - Creating VCPUs aka Realm Execution Context (REC), with initial register state.
> > - Create stage2 translation table at any level.
> > - Load initial images into Realm Memory from normal world memory
> > - Schedule RECs (vCPUs) and handle exits
> > - Inject virtual interrupts into the Realm
> > - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM).
> > - Create "shared" mappings that can be accessed by VMM/Hyp.
> > - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables)
> >
> > However v1.0 of RMM specifications doesn't support:
> > - Paging protected memory of a Realm VM. Thus the pages backing the protected
> > memory region must be pinned.
> > - Live migration of Realms.
> > - Trusted Device assignment.
> > - Physical interrupt backed Virtual interrupts for Realms
> >
> > RMM also provides certain services to the Realms via SMC, called Realm Service
> > Interface (RSI). These include:
> > - Realm Guest Configuration.
> > - Attestation & Measurement services
> > - Managing the state of an Intermediate Physical Address (IPA aka GPA) page.
> > - Host Call service (Communication with the Normal world Hypervisor)
> >
> > The specifications for the RMM software is currently at *v1.0-Beta2* and the
> > latest version is available here [1].
> >
> > The Trusted Firmware foundation has an implementation of the RMM - TF-RMM -
> > available here [3].
> >
> > Implementation
> > =================
> >
> > This version of the stack is based on the RMM specification v1.0-Beta0[2], with
> > following exceptions :
> > - TF-RMM/KVM currently doesn't support the optional features of PMU,
> > SVE and Self-hosted debug (coming soon).
> > - The RSI_HOST_CALL structure alignment requirement is reduced to match
> > RMM v1.0 Beta1
> > - RMI/RSI version numbers do not match the RMM spec. This will be
> > resolved once the spec/implementation is complete, across TF-RMM+Linux stack.
> >
> > We plan to update the stack to support the latest version of the RMMv1.0 spec
> > in the coming revisions.
> >
> > This release includes the following components :
> >
> > a) Linux Kernel
> > i) Host / KVM support - Support for driving the Realms via RMI. This is
> > dependent on running in the Kernel at EL2 (aka VHE mode). Also provides
> > UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page
> > size, matching the Stage2 granule supported by RMM. The VMM is responsible
> > for making sure the guest memory is locked.
> >
> > TODO: Guest Private memory[10] integration - We have been following the
> > series and support will be added once it is merged upstream.
> >
> > ii) Guest support - Support for a Linux Kernel to run in the Realm VM at
> > Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0
> > only). All I/O are treated as non-secure/shared.
> >
> > c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory
> > as mentioned above.
> > d) kvm-unit-tests - Support for running in Realms along with additional tests
> > for RSI ABI.
> >
> > Running the stack
> > ====================
> >
> > To run/test the stack, you would need the following components :
> >
> > 1) FVP Base AEM RevC model with FEAT_RME support [4]
> > 2) TF-A firmware for EL3 [5]
> > 3) TF-A RMM for R-EL2 [3]
> > 4) Linux Kernel [6]
> > 5) kvmtool [7]
> > 6) kvm-unit-tests [8]
> >
> > Instructions for building the firmware components and running the model are
> > available here [9]. Once, the host kernel is booted, a Realm can be launched by
> > invoking the `lkvm` commad as follows:
> >
> > $ lkvm run --realm \
> > --measurement-algo=["sha256", "sha512"] \
> > --disable-sve \
>
> As noted above, this is no longer required.
>
> > <normal-vm-options>
> >
> > Where:
> > * --measurement-algo (Optional) specifies the algorithm selected for creating the
> > initial measurements by the RMM for this Realm (defaults to sha256).
> > * GICv3 is mandatory for the Realms.
> > * SVE is not yet supported in the TF-RMM, and thus must be disabled using
> > --disable-sve
> >
> > You may also run the kvm-unit-tests inside the Realm world, using the similar
> > options as above.
> >
> >
> > Links
> > ============
> >
> > [0] Arm CCA Landing page (See Key Resources section for various documentations)
> > https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture
> >
> > [1] RMM Specification Latest
> > https://developer.arm.com/documentation/den0137/latest
> >
> > [2] RMM v1.0-Beta0 specification
> > https://developer.arm.com/documentation/den0137/1-0bet0/
>
> EAC2 spec: https://developer.arm.com/documentation/den0137/1-0eac2/
> >
> > [3] Trusted Firmware RMM - TF-RMM
> > https://www.trustedfirmware.org/projects/tf-rmm/
> > GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
> >
> > [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
> > https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
> >
> > [5] Trusted Firmware for A class
> > https://www.trustedfirmware.org/projects/tf-a/
> >
> > [6] Linux kernel support for Arm-CCA
> > https://gitlab.arm.com/linux-arm/linux-cca
> > Host Support branch: cca-host/rfc-v1
>
> Update branch : cca-host/rmm-v1.0-eac2
>
> > Guest Support branch: cca-guest/rfc-v1
>
> Update branch : cca-guest/rmm-v1.0-eac2
>
> Combined tree for host and guest is also available at: "cca-full/rmm-v1.0-eac2"
>
> >
> > [7] kvmtool support for Arm CCA
> > https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1
>
> Update branch : cca/rmm-v1.0-eac2
>
> >
> > [8] kvm-unit-tests support for Arm CCA
> > https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1
> >
>
> Update branch : cca/rmm-v1.0-eac2
>
>
> Suzuki
>
> > [9] Instructions for Building Firmware components and running the model, see
> > section 4.19.2 "Building and running TF-A with RME"
> > https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
> >
> > [10] fd based Guest Private memory for KVM
> > https://lkml.kernel.org/r/[email protected]
>
>
>
>
>
> Cc: Alexandru Elisei <[email protected]>
> Cc: Andrew Jones <[email protected]>
> Cc: Catalin Marinas <[email protected]>
> Cc: Chao Peng <[email protected]>
> Cc: Christoffer Dall <[email protected]>
> Cc: Fuad Tabba <[email protected]>
> Cc: Jonathan Cameron <[email protected]>
> Cc: James Morse <[email protected]>
> Cc: Jean-Philippe Brucker <[email protected]>
> Cc: Joey Gouly <[email protected]>
> Cc: Marc Zyngier <[email protected]>
> Cc: Mark Rutland <[email protected]>
> Cc: Oliver Upton <[email protected]>
> Cc: Paolo Bonzini <[email protected]>
> Cc: Quentin Perret <[email protected]>
> Cc: Sean Christopherson <[email protected]>
> Cc: Steven Price <[email protected]>
> Cc: Thomas Huth <[email protected]>
> Cc: Will Deacon <[email protected]>
> Cc: Zenghui Yu <[email protected]>
> To: [email protected]
> To: [email protected]
> Cc: [email protected]
> To: [email protected]
> To: [email protected]

Suzuki,
Any update to the Arm CCA series (v3?) since last October?

Thanks,
Itaru.

2024-01-10 11:41:57

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

Hi Itaru,

On 10/01/2024 05:40, Itaru Kitayama wrote:
> On Mon, Oct 02, 2023 at 01:43:11PM +0100, Suzuki K Poulose wrote:
>> Hi,
>>
>>
>>> We are happy to announce the early RFC version of the Arm
>>> Confidential Compute Architecture (CCA) support for the Linux
>>> stack. The intention is to seek early feedback in the following areas:
>>> * KVM integration of the Arm CCA
>>> * KVM UABI for managing the Realms, seeking to generalise the operations
>>> wherever possible with other Confidential Compute solutions.
>>> Note: This version doesn't support Guest Private memory, which will be added
>>> later (see below).
>>> * Linux Guest support for Realms
>>>
>>
>> We have updated the stack for Arm CCA Linux support to RMM-v1.0-EAC2 (See links)
>> We are not posting the patches for review yet, as we plan to update our
>> stack to support the latest RMM-v1.0 specification, which includes some
>> functional changes to support PSCI monitoring by the VMM along with other
>> minor changes. All relevant components are updated on a new branch "rmm-v1.0-eac2"
>> Guest-mem support is not included, but is in progress.
>>
>> Change log :
>> - KVM RMI support updated to v1.0-eac2, with optimisations to stage2 tear down
>> - Guest (Linux and kvm-unit-test) support for RSI compliant to v1.0-eac2
>> - SVE, PMU support for Realms
>>
>> kvmtool :
>> - Dropped no-compat and switched to --loglevel (merged upstream)
>> - Support for SVE, --sve-vl for vector length
>>
>>> Arm CCA Introduction
>>> =====================
>>>
>>> The Arm CCA is a reference software architecture and implementation that builds
>>> on the Realm Management Extension (RME), enabling the execution of Virtual
>>> machines, while preventing access by more privileged software, such as hypervisor.
>>> The Arm CCA allows the hypervisor to control the VM, but removes the right for
>>> access to the code, register state or data that is used by VM.
>>> More information on the architecture is available here[0].
>>>
>>> Arm CCA Reference Software Architecture
>>>
>>> Realm World || Normal World || Secure World ||
>>> || | || ||
>>> EL0 x-------x || x----x | x------x || ||
>>> | Realm | || | | | | | || ||
>>> | | || | VM | | | | || ||
>>> ----| VM* |---------||-| |---| |-||----------------||
>>> | | || | | | | H | || ||
>>> EL1 x-------x || x----x | | | || ||
>>> ^ || | | o | || ||
>>> | || | | | || ||
>>> ------- R*------------------------| s -|---------------------
>>> S || | | || ||
>>> I || | t | || ||
>>> | || | | || ||
>>> v || x------x || ||
>>> EL2 RMM* || ^ || ||
>>> ^ || | || ||
>>> ========|=============================|========================
>>> | | SMC
>>> x--------- *RMI* -------------x
>>>
>>> EL3 Root World
>>> EL3 Firmware
>>> ===============================================================
>>> Where :
>>> RMM - Realm Management Monitor
>>> RMI - Realm Management Interface
>>> RSI - Realm Service Interface
>>> SMC - Secure Monitor Call
>>>
>>> RME introduces a new security state "Realm world", in addition to the
>>> traditional Secure and Non-Secure states. The Arm CCA defines a new component,
>>> Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of
>>> firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at
>>> system boot.
>>>
>>> The RMM provides standard interfaces - Realm Management Interface (RMI) - to the
>>> Normal world hypervisor to manage the VMs running in the Realm world (also called
>>> Realms in short). These are exposed via SMC and are routed through the EL3
>>> firmwre.
>>> The RMI interface includes:
>>> - Move a physical page from the Normal world to the Realm world
>>> - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD)
>>> - Creating VCPUs aka Realm Execution Context (REC), with initial register state.
>>> - Create stage2 translation table at any level.
>>> - Load initial images into Realm Memory from normal world memory
>>> - Schedule RECs (vCPUs) and handle exits
>>> - Inject virtual interrupts into the Realm
>>> - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM).
>>> - Create "shared" mappings that can be accessed by VMM/Hyp.
>>> - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables)
>>>
>>> However v1.0 of RMM specifications doesn't support:
>>> - Paging protected memory of a Realm VM. Thus the pages backing the protected
>>> memory region must be pinned.
>>> - Live migration of Realms.
>>> - Trusted Device assignment.
>>> - Physical interrupt backed Virtual interrupts for Realms
>>>
>>> RMM also provides certain services to the Realms via SMC, called Realm Service
>>> Interface (RSI). These include:
>>> - Realm Guest Configuration.
>>> - Attestation & Measurement services
>>> - Managing the state of an Intermediate Physical Address (IPA aka GPA) page.
>>> - Host Call service (Communication with the Normal world Hypervisor)
>>>
>>> The specifications for the RMM software is currently at *v1.0-Beta2* and the
>>> latest version is available here [1].
>>>
>>> The Trusted Firmware foundation has an implementation of the RMM - TF-RMM -
>>> available here [3].
>>>
>>> Implementation
>>> =================
>>>
>>> This version of the stack is based on the RMM specification v1.0-Beta0[2], with
>>> following exceptions :
>>> - TF-RMM/KVM currently doesn't support the optional features of PMU,
>>> SVE and Self-hosted debug (coming soon).
>>> - The RSI_HOST_CALL structure alignment requirement is reduced to match
>>> RMM v1.0 Beta1
>>> - RMI/RSI version numbers do not match the RMM spec. This will be
>>> resolved once the spec/implementation is complete, across TF-RMM+Linux stack.
>>>
>>> We plan to update the stack to support the latest version of the RMMv1.0 spec
>>> in the coming revisions.
>>>
>>> This release includes the following components :
>>>
>>> a) Linux Kernel
>>> i) Host / KVM support - Support for driving the Realms via RMI. This is
>>> dependent on running in the Kernel at EL2 (aka VHE mode). Also provides
>>> UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page
>>> size, matching the Stage2 granule supported by RMM. The VMM is responsible
>>> for making sure the guest memory is locked.
>>>
>>> TODO: Guest Private memory[10] integration - We have been following the
>>> series and support will be added once it is merged upstream.
>>>
>>> ii) Guest support - Support for a Linux Kernel to run in the Realm VM at
>>> Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0
>>> only). All I/O are treated as non-secure/shared.
>>>
>>> c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory
>>> as mentioned above.
>>> d) kvm-unit-tests - Support for running in Realms along with additional tests
>>> for RSI ABI.
>>>
>>> Running the stack
>>> ====================
>>>
>>> To run/test the stack, you would need the following components :
>>>
>>> 1) FVP Base AEM RevC model with FEAT_RME support [4]
>>> 2) TF-A firmware for EL3 [5]
>>> 3) TF-A RMM for R-EL2 [3]
>>> 4) Linux Kernel [6]
>>> 5) kvmtool [7]
>>> 6) kvm-unit-tests [8]
>>>
>>> Instructions for building the firmware components and running the model are
>>> available here [9]. Once, the host kernel is booted, a Realm can be launched by
>>> invoking the `lkvm` commad as follows:
>>>
>>> $ lkvm run --realm \
>>> --measurement-algo=["sha256", "sha512"] \
>>> --disable-sve \
>>
>> As noted above, this is no longer required.
>>
>>> <normal-vm-options>
>>>
>>> Where:
>>> * --measurement-algo (Optional) specifies the algorithm selected for creating the
>>> initial measurements by the RMM for this Realm (defaults to sha256).
>>> * GICv3 is mandatory for the Realms.
>>> * SVE is not yet supported in the TF-RMM, and thus must be disabled using
>>> --disable-sve
>>>
>>> You may also run the kvm-unit-tests inside the Realm world, using the similar
>>> options as above.
>>>
>>>
>>> Links
>>> ============
>>>
>>> [0] Arm CCA Landing page (See Key Resources section for various documentations)
>>> https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture
>>>
>>> [1] RMM Specification Latest
>>> https://developer.arm.com/documentation/den0137/latest
>>>
>>> [2] RMM v1.0-Beta0 specification
>>> https://developer.arm.com/documentation/den0137/1-0bet0/
>>
>> EAC2 spec: https://developer.arm.com/documentation/den0137/1-0eac2/
>>>
>>> [3] Trusted Firmware RMM - TF-RMM
>>> https://www.trustedfirmware.org/projects/tf-rmm/
>>> GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
>>>
>>> [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
>>> https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
>>>
>>> [5] Trusted Firmware for A class
>>> https://www.trustedfirmware.org/projects/tf-a/ >>>
>>> [6] Linux kernel support for Arm-CCA
>>> https://gitlab.arm.com/linux-arm/linux-cca
>>> Host Support branch: cca-host/rfc-v1
>>
>> Update branch : cca-host/rmm-v1.0-eac2
>>
>>> Guest Support branch: cca-guest/rfc-v1
>>
>> Update branch : cca-guest/rmm-v1.0-eac2
>>
>> Combined tree for host and guest is also available at: "cca-full/rmm-v1.0-eac2"
>>
>>>
>>> [7] kvmtool support for Arm CCA
>>> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1
>>
>> Update branch : cca/rmm-v1.0-eac2
>>
>>>
>>> [8] kvm-unit-tests support for Arm CCA
>>> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1
>>>
>>
>> Update branch : cca/rmm-v1.0-eac2
>>
>>
>> Suzuki
>>
>>> [9] Instructions for Building Firmware components and running the model, see
>>> section 4.19.2 "Building and running TF-A with RME"
>>> https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
>>>
>>> [10] fd based Guest Private memory for KVM
>>> https://lkml.kernel.org/r/[email protected]
>>
>>
>>
>>
>>
>> Cc: Alexandru Elisei <[email protected]>
>> Cc: Andrew Jones <[email protected]>
>> Cc: Catalin Marinas <[email protected]>
>> Cc: Chao Peng <[email protected]>
>> Cc: Christoffer Dall <[email protected]>
>> Cc: Fuad Tabba <[email protected]>
>> Cc: Jonathan Cameron <[email protected]>
>> Cc: James Morse <[email protected]>
>> Cc: Jean-Philippe Brucker <[email protected]>
>> Cc: Joey Gouly <[email protected]>
>> Cc: Marc Zyngier <[email protected]>
>> Cc: Mark Rutland <[email protected]>
>> Cc: Oliver Upton <[email protected]>
>> Cc: Paolo Bonzini <[email protected]>
>> Cc: Quentin Perret <[email protected]>
>> Cc: Sean Christopherson <[email protected]>
>> Cc: Steven Price <[email protected]>
>> Cc: Thomas Huth <[email protected]>
>> Cc: Will Deacon <[email protected]>
>> Cc: Zenghui Yu <[email protected]>
>> To: [email protected]
>> To: [email protected]
>> Cc: [email protected]
>> To: [email protected]
>> To: [email protected]
>
> Suzuki,
> Any update to the Arm CCA series (v3?) since last October?

Yes, we now have a version that supports the final RMM-v1.0
specification (RMM-v1.0-EAC5). We also have the UEFI EDK2 firmware
support for Guests in Realm world.

We are planning to post the changes for review in the v6.8-rc cycle. We
are trying to integrate the guest_mem support (available in v6.8-rc1) as
well as reusing some of the arm64 kvm generic interface for configuring
the Realm parameters (e.g., PMU, SVE_VL etc).

Here is a version that is missing the items mentioned above, based
on v6.7-rc4, if anyone would like to try.

Also, the easiest way to get the components built and model kick started
is using the shrinkwrap [6] tool, using the cca-3world configuration.
The tool pulls all the required software components, builds (including
the buildroot for rootfs) and can run a model using these built
components.

[0] Linux Repo:
Where: [email protected]:linux-arm/linux-cca.git
KVM Support branch: cca-host/rmm-v1.0-eac5
Linux Guest branch: cca-guest/rmm-v1.0-eac5
Full stack branch: cca-full/rmm-v1.0-eac5

[1] kvmtool Repo:
Where: [email protected]:linux-arm/kvmtool-cca.git
Branch: cca/rmm-v1.0-eac5

[2] kvm-unit-tests Repo:
Where: [email protected]:linux-arm/kvm-unit-tests-cca.git
Branch: cca/rmm-v1.0-eac5

[3] UEFI Guest firmware:
edk2: https://git.gitlab.arm.com/linux-arm/edk2-cca.git
revision: 2802_arm_cca_rmm-v1.0-eac5

edk2-platforms:
https://git.gitlab.arm.com/linux-arm/edk2-platforms-cca.git
revision: 2802_arm_cca_rmm-v1.0-eac5

[4] RMM Repo:
Where: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
tag : tf-rmm-v0.4.0

[5] TF-A repo:
Where: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git
Tag: v2.10

[6] https://shrinkwrap.docs.arm.com/en/latest/
config: cca-3world.yaml

Kind regards
Suzuki

2024-01-10 13:45:07

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On 10/01/2024 11:41, Suzuki K Poulose wrote:
> Hi Itaru,
>
> On 10/01/2024 05:40, Itaru Kitayama wrote:
>> On Mon, Oct 02, 2023 at 01:43:11PM +0100, Suzuki K Poulose wrote:
>>> Hi,
>>>
>>>

..

>>
>> Suzuki,
>> Any update to the Arm CCA series (v3?) since last October?
>
> Yes, we now have a version that supports the final RMM-v1.0
> specification (RMM-v1.0-EAC5). We also have the UEFI EDK2 firmware
> support for Guests in Realm world.
>
> We are planning to post the changes for review in the v6.8-rc cycle. We
> are trying to integrate the guest_mem support (available in v6.8-rc1) as
> well as reusing some of the arm64 kvm generic interface for configuring
> the Realm parameters (e.g., PMU, SVE_VL etc).
>
> Here is a version that is missing the items mentioned above, based
> on v6.7-rc4, if anyone would like to try.
>
> Also, the easiest way to get the components built and model kick started
> is using the shrinkwrap [6] tool, using the cca-3world configuration.
> The tool pulls all the required software components, builds (including
> the buildroot for rootfs) and can run a model using these built
> components.

Also, please see 'arm/run-realm-tests.sh' in the kvm-unit-tests-cca
repository for sample command lines to invoke kvmtool to create Realm
VMs.

>
>
>
> [0] Linux Repo:
>       Where: [email protected]:linux-arm/linux-cca.git
>       KVM Support branch: cca-host/rmm-v1.0-eac5
>       Linux Guest branch: cca-guest/rmm-v1.0-eac5
>       Full stack branch: cca-full/rmm-v1.0-eac5
>
> [1] kvmtool Repo:
>       Where: [email protected]:linux-arm/kvmtool-cca.git
>       Branch: cca/rmm-v1.0-eac5
>
> [2] kvm-unit-tests Repo:
>       Where: [email protected]:linux-arm/kvm-unit-tests-cca.git
>       Branch: cca/rmm-v1.0-eac5
>
> [3] UEFI Guest firmware:
>       edk2:     https://git.gitlab.arm.com/linux-arm/edk2-cca.git
>       revision: 2802_arm_cca_rmm-v1.0-eac5
>
>       edk2-platforms:
> https://git.gitlab.arm.com/linux-arm/edk2-platforms-cca.git
>       revision:       2802_arm_cca_rmm-v1.0-eac5
>
>
> [4] RMM Repo:
>       Where: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
>       tag : tf-rmm-v0.4.0
>
> [5] TF-A repo:
>       Where: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git
>       Tag: v2.10
>
>
> [6] https://shrinkwrap.docs.arm.com/en/latest/
>     config: cca-3world.yaml
>

Suzuki

2024-01-12 05:02:26

[permalink] [raw]

Subject: Re: [RFC] Support for Arm CCA VMs on Linux

On Wed, Jan 10, 2024 at 11:41:09AM +0000, Suzuki K Poulose wrote:
> Hi Itaru,
>
> On 10/01/2024 05:40, Itaru Kitayama wrote:
> > On Mon, Oct 02, 2023 at 01:43:11PM +0100, Suzuki K Poulose wrote:
> > > Hi,
> > >
> > >
> > > > We are happy to announce the early RFC version of the Arm
> > > > Confidential Compute Architecture (CCA) support for the Linux
> > > > stack. The intention is to seek early feedback in the following areas:
> > > > * KVM integration of the Arm CCA
> > > > * KVM UABI for managing the Realms, seeking to generalise the operations
> > > > wherever possible with other Confidential Compute solutions.
> > > > Note: This version doesn't support Guest Private memory, which will be added
> > > > later (see below).
> > > > * Linux Guest support for Realms
> > > >
> > >
> > > We have updated the stack for Arm CCA Linux support to RMM-v1.0-EAC2 (See links)
> > > We are not posting the patches for review yet, as we plan to update our
> > > stack to support the latest RMM-v1.0 specification, which includes some
> > > functional changes to support PSCI monitoring by the VMM along with other
> > > minor changes. All relevant components are updated on a new branch "rmm-v1.0-eac2"
> > > Guest-mem support is not included, but is in progress.
> > >
> > > Change log :
> > > - KVM RMI support updated to v1.0-eac2, with optimisations to stage2 tear down
> > > - Guest (Linux and kvm-unit-test) support for RSI compliant to v1.0-eac2
> > > - SVE, PMU support for Realms
> > >
> > > kvmtool :
> > > - Dropped no-compat and switched to --loglevel (merged upstream)
> > > - Support for SVE, --sve-vl for vector length
> > >
> > > > Arm CCA Introduction
> > > > =====================
> > > >
> > > > The Arm CCA is a reference software architecture and implementation that builds
> > > > on the Realm Management Extension (RME), enabling the execution of Virtual
> > > > machines, while preventing access by more privileged software, such as hypervisor.
> > > > The Arm CCA allows the hypervisor to control the VM, but removes the right for
> > > > access to the code, register state or data that is used by VM.
> > > > More information on the architecture is available here[0].
> > > >
> > > > Arm CCA Reference Software Architecture
> > > >
> > > > Realm World || Normal World || Secure World ||
> > > > || | || ||
> > > > EL0 x-------x || x----x | x------x || ||
> > > > | Realm | || | | | | | || ||
> > > > | | || | VM | | | | || ||
> > > > ----| VM* |---------||-| |---| |-||----------------||
> > > > | | || | | | | H | || ||
> > > > EL1 x-------x || x----x | | | || ||
> > > > ^ || | | o | || ||
> > > > | || | | | || ||
> > > > ------- R*------------------------| s -|---------------------
> > > > S || | | || ||
> > > > I || | t | || ||
> > > > | || | | || ||
> > > > v || x------x || ||
> > > > EL2 RMM* || ^ || ||
> > > > ^ || | || ||
> > > > ========|=============================|========================
> > > > | | SMC
> > > > x--------- *RMI* -------------x
> > > >
> > > > EL3 Root World
> > > > EL3 Firmware
> > > > ===============================================================
> > > > Where :
> > > > RMM - Realm Management Monitor
> > > > RMI - Realm Management Interface
> > > > RSI - Realm Service Interface
> > > > SMC - Secure Monitor Call
> > > >
> > > > RME introduces a new security state "Realm world", in addition to the
> > > > traditional Secure and Non-Secure states. The Arm CCA defines a new component,
> > > > Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of
> > > > firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at
> > > > system boot.
> > > >
> > > > The RMM provides standard interfaces - Realm Management Interface (RMI) - to the
> > > > Normal world hypervisor to manage the VMs running in the Realm world (also called
> > > > Realms in short). These are exposed via SMC and are routed through the EL3
> > > > firmwre.
> > > > The RMI interface includes:
> > > > - Move a physical page from the Normal world to the Realm world
> > > > - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD)
> > > > - Creating VCPUs aka Realm Execution Context (REC), with initial register state.
> > > > - Create stage2 translation table at any level.
> > > > - Load initial images into Realm Memory from normal world memory
> > > > - Schedule RECs (vCPUs) and handle exits
> > > > - Inject virtual interrupts into the Realm
> > > > - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM).
> > > > - Create "shared" mappings that can be accessed by VMM/Hyp.
> > > > - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables)
> > > >
> > > > However v1.0 of RMM specifications doesn't support:
> > > > - Paging protected memory of a Realm VM. Thus the pages backing the protected
> > > > memory region must be pinned.
> > > > - Live migration of Realms.
> > > > - Trusted Device assignment.
> > > > - Physical interrupt backed Virtual interrupts for Realms
> > > >
> > > > RMM also provides certain services to the Realms via SMC, called Realm Service
> > > > Interface (RSI). These include:
> > > > - Realm Guest Configuration.
> > > > - Attestation & Measurement services
> > > > - Managing the state of an Intermediate Physical Address (IPA aka GPA) page.
> > > > - Host Call service (Communication with the Normal world Hypervisor)
> > > >
> > > > The specifications for the RMM software is currently at *v1.0-Beta2* and the
> > > > latest version is available here [1].
> > > >
> > > > The Trusted Firmware foundation has an implementation of the RMM - TF-RMM -
> > > > available here [3].
> > > >
> > > > Implementation
> > > > =================
> > > >
> > > > This version of the stack is based on the RMM specification v1.0-Beta0[2], with
> > > > following exceptions :
> > > > - TF-RMM/KVM currently doesn't support the optional features of PMU,
> > > > SVE and Self-hosted debug (coming soon).
> > > > - The RSI_HOST_CALL structure alignment requirement is reduced to match
> > > > RMM v1.0 Beta1
> > > > - RMI/RSI version numbers do not match the RMM spec. This will be
> > > > resolved once the spec/implementation is complete, across TF-RMM+Linux stack.
> > > >
> > > > We plan to update the stack to support the latest version of the RMMv1.0 spec
> > > > in the coming revisions.
> > > >
> > > > This release includes the following components :
> > > >
> > > > a) Linux Kernel
> > > > i) Host / KVM support - Support for driving the Realms via RMI. This is
> > > > dependent on running in the Kernel at EL2 (aka VHE mode). Also provides
> > > > UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page
> > > > size, matching the Stage2 granule supported by RMM. The VMM is responsible
> > > > for making sure the guest memory is locked.
> > > >
> > > > TODO: Guest Private memory[10] integration - We have been following the
> > > > series and support will be added once it is merged upstream.
> > > > ii) Guest support - Support for a Linux Kernel to run in the Realm VM at
> > > > Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0
> > > > only). All I/O are treated as non-secure/shared.
> > > > c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory
> > > > as mentioned above.
> > > > d) kvm-unit-tests - Support for running in Realms along with additional tests
> > > > for RSI ABI.
> > > >
> > > > Running the stack
> > > > ====================
> > > >
> > > > To run/test the stack, you would need the following components :
> > > >
> > > > 1) FVP Base AEM RevC model with FEAT_RME support [4]
> > > > 2) TF-A firmware for EL3 [5]
> > > > 3) TF-A RMM for R-EL2 [3]
> > > > 4) Linux Kernel [6]
> > > > 5) kvmtool [7]
> > > > 6) kvm-unit-tests [8]
> > > >
> > > > Instructions for building the firmware components and running the model are
> > > > available here [9]. Once, the host kernel is booted, a Realm can be launched by
> > > > invoking the `lkvm` commad as follows:
> > > >
> > > > $ lkvm run --realm \
> > > > --measurement-algo=["sha256", "sha512"] \
> > > > --disable-sve \
> > >
> > > As noted above, this is no longer required.
> > >
> > > > <normal-vm-options>
> > > >
> > > > Where:
> > > > * --measurement-algo (Optional) specifies the algorithm selected for creating the
> > > > initial measurements by the RMM for this Realm (defaults to sha256).
> > > > * GICv3 is mandatory for the Realms.
> > > > * SVE is not yet supported in the TF-RMM, and thus must be disabled using
> > > > --disable-sve
> > > >
> > > > You may also run the kvm-unit-tests inside the Realm world, using the similar
> > > > options as above.
> > > >
> > > >
> > > > Links
> > > > ============
> > > >
> > > > [0] Arm CCA Landing page (See Key Resources section for various documentations)
> > > > https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture
> > > >
> > > > [1] RMM Specification Latest
> > > > https://developer.arm.com/documentation/den0137/latest
> > > >
> > > > [2] RMM v1.0-Beta0 specification
> > > > https://developer.arm.com/documentation/den0137/1-0bet0/
> > >
> > > EAC2 spec: https://developer.arm.com/documentation/den0137/1-0eac2/
> > > >
> > > > [3] Trusted Firmware RMM - TF-RMM
> > > > https://www.trustedfirmware.org/projects/tf-rmm/
> > > > GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
> > > >
> > > > [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
> > > > https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
> > > >
> > > > [5] Trusted Firmware for A class
> > > > https://www.trustedfirmware.org/projects/tf-a/ >>>
> > > > [6] Linux kernel support for Arm-CCA
> > > > https://gitlab.arm.com/linux-arm/linux-cca
> > > > Host Support branch: cca-host/rfc-v1
> > >
> > > Update branch : cca-host/rmm-v1.0-eac2
> > >
> > > > Guest Support branch: cca-guest/rfc-v1
> > >
> > > Update branch : cca-guest/rmm-v1.0-eac2
> > >
> > > Combined tree for host and guest is also available at: "cca-full/rmm-v1.0-eac2"
> > >
> > > >
> > > > [7] kvmtool support for Arm CCA
> > > > https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1
> > >
> > > Update branch : cca/rmm-v1.0-eac2
> > >
> > > >
> > > > [8] kvm-unit-tests support for Arm CCA
> > > > https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1
> > > >
> > >
> > > Update branch : cca/rmm-v1.0-eac2
> > >
> > >
> > > Suzuki
> > >
> > > > [9] Instructions for Building Firmware components and running the model, see
> > > > section 4.19.2 "Building and running TF-A with RME"
> > > > https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
> > > >
> > > > [10] fd based Guest Private memory for KVM
> > > > https://lkml.kernel.org/r/[email protected]
> > >
> > >
> > >
> > >
> > >
> > > Cc: Alexandru Elisei <[email protected]>
> > > Cc: Andrew Jones <[email protected]>
> > > Cc: Catalin Marinas <[email protected]>
> > > Cc: Chao Peng <[email protected]>
> > > Cc: Christoffer Dall <[email protected]>
> > > Cc: Fuad Tabba <[email protected]>
> > > Cc: Jonathan Cameron <[email protected]>
> > > Cc: James Morse <[email protected]>
> > > Cc: Jean-Philippe Brucker <[email protected]>
> > > Cc: Joey Gouly <[email protected]>
> > > Cc: Marc Zyngier <[email protected]>
> > > Cc: Mark Rutland <[email protected]>
> > > Cc: Oliver Upton <[email protected]>
> > > Cc: Paolo Bonzini <[email protected]>
> > > Cc: Quentin Perret <[email protected]>
> > > Cc: Sean Christopherson <[email protected]>
> > > Cc: Steven Price <[email protected]>
> > > Cc: Thomas Huth <[email protected]>
> > > Cc: Will Deacon <[email protected]>
> > > Cc: Zenghui Yu <[email protected]>
> > > To: [email protected]
> > > To: [email protected]
> > > Cc: [email protected]
> > > To: [email protected]
> > > To: [email protected]
> >
> > Suzuki,
> > Any update to the Arm CCA series (v3?) since last October?
>
> Yes, we now have a version that supports the final RMM-v1.0
> specification (RMM-v1.0-EAC5). We also have the UEFI EDK2 firmware
> support for Guests in Realm world.
>
> We are planning to post the changes for review in the v6.8-rc cycle. We
> are trying to integrate the guest_mem support (available in v6.8-rc1) as
> well as reusing some of the arm64 kvm generic interface for configuring
> the Realm parameters (e.g., PMU, SVE_VL etc).
>
> Here is a version that is missing the items mentioned above, based
> on v6.7-rc4, if anyone would like to try.
>
> Also, the easiest way to get the components built and model kick started
> is using the shrinkwrap [6] tool, using the cca-3world configuration.
> The tool pulls all the required software components, builds (including
> the buildroot for rootfs) and can run a model using these built
> components.

Hi Suzuki,

This is great news! I've just booted you guys WIP Linux kernel through
shrinkwrap (cca-3world.yaml) without an issue.
Many thanks to Ryan who delivered an extremely handy tool to us.

Thanks,
Itaru.

>
>
>
> [0] Linux Repo:
> Where: [email protected]:linux-arm/linux-cca.git
> KVM Support branch: cca-host/rmm-v1.0-eac5
> Linux Guest branch: cca-guest/rmm-v1.0-eac5
> Full stack branch: cca-full/rmm-v1.0-eac5
>
> [1] kvmtool Repo:
> Where: [email protected]:linux-arm/kvmtool-cca.git
> Branch: cca/rmm-v1.0-eac5
>
> [2] kvm-unit-tests Repo:
> Where: [email protected]:linux-arm/kvm-unit-tests-cca.git
> Branch: cca/rmm-v1.0-eac5
>
> [3] UEFI Guest firmware:
> edk2: https://git.gitlab.arm.com/linux-arm/edk2-cca.git
> revision: 2802_arm_cca_rmm-v1.0-eac5
>
> edk2-platforms:
> https://git.gitlab.arm.com/linux-arm/edk2-platforms-cca.git
> revision: 2802_arm_cca_rmm-v1.0-eac5
>
>
> [4] RMM Repo:
> Where: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
> tag : tf-rmm-v0.4.0
>
> [5] TF-A repo:
> Where: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git
> Tag: v2.10
>
>
> [6] https://shrinkwrap.docs.arm.com/en/latest/
> config: cca-3world.yaml
>
> Kind regards
> Suzuki
>
>

2024-01-19 01:27:28