From: Shuo Liu <[email protected]>
ACRN is a Type 1 reference hypervisor stack, running directly on the bare-metal
hardware, and is suitable for a variety of IoT and embedded device solutions.
ACRN implements a hybrid VMM architecture, using a privileged Service VM. The
Service VM manages the system resources (CPU, memory, etc.) and I/O devices of
User VMs. Multiple User VMs are supported, with each of them running Linux,
Android OS or Windows. Both Service VM and User VMs are guest VM.
Below figure shows the architecture.
Service VM User VM
+----------------------------+ | +------------------+
| +--------------+ | | | |
| |ACRN userspace| | | | |
| +--------------+ | | | |
|-----------------ioctl------| | | | ...
|kernel space +----------+ | | | |
| | HSM | | | | Drivers |
| +----------+ | | | |
+--------------------|-------+ | +------------------+
+---------------------hypercall----------------------------------------+
| ACRN Hypervisor |
+----------------------------------------------------------------------+
| Hardware |
+----------------------------------------------------------------------+
There is only one Service VM which could run Linux as OS.
In a typical case, the Service VM will be auto started when ACRN Hypervisor is
booted. Then the ACRN userspace (an application running in Service VM) could be
used to start/stop User VMs by communicating with ACRN Hypervisor Service
Module (HSM).
ACRN Hypervisor Service Module (HSM) is a middle layer that allows the ACRN
userspace and Service VM OS kernel to communicate with ACRN Hypervisor
and manage different User VMs. This middle layer provides the following
functionalities,
- Issues hypercalls to the hypervisor to manage User VMs:
* VM/vCPU management
* Memory management
* Device passthrough
* Interrupts injection
- I/O requests handling from User VMs.
- Exports ioctl through HSM char device.
- Exports function calls for other kernel modules
ACRN is focused on embedded system. So it doesn't support some features.
E.g.,
- ACRN doesn't support VM migration.
- ACRN doesn't support vCPU migration.
This patch set adds the HSM to the Linux kernel.
The basic ARCN support was merged to upstream already.
https://lore.kernel.org/lkml/[email protected]/
ChangeLog:
v2:
- Removed API version related code. (Dave)
- Replaced pr_*() by dev_*(). (Greg)
- Used -ENOTTY as the error code of unsupported ioctl. (Greg)
Shuo Liu (16):
docs: acrn: Introduce ACRN
x86/acrn: Introduce acrn_{setup, remove}_intr_handler()
x86/acrn: Introduce hypercall interfaces
virt: acrn: Introduce ACRN HSM basic driver
virt: acrn: Introduce VM management interfaces
virt: acrn: Introduce an ioctl to set vCPU registers state
virt: acrn: Introduce EPT mapping management
virt: acrn: Introduce I/O request management
virt: acrn: Introduce PCI configuration space PIO accesses combiner
virt: acrn: Introduce interfaces for PCI device passthrough
virt: acrn: Introduce interrupt injection interfaces
virt: acrn: Introduce interfaces to query C-states and P-states
allowed by hypervisor
virt: acrn: Introduce I/O ranges operation interfaces
virt: acrn: Introduce ioeventfd
virt: acrn: Introduce irqfd
virt: acrn: Introduce an interface for Service VM to control vCPU
Yin Fengwei (1):
x86/acrn: Introduce an API to check if a VM is privileged
.../userspace-api/ioctl/ioctl-number.rst | 1 +
Documentation/virt/acrn/index.rst | 11 +
Documentation/virt/acrn/introduction.rst | 40 ++
Documentation/virt/acrn/io-request.rst | 97 +++
Documentation/virt/index.rst | 1 +
MAINTAINERS | 9 +
arch/x86/include/asm/acrn.h | 74 ++
arch/x86/kernel/cpu/acrn.c | 38 +-
drivers/virt/Kconfig | 2 +
drivers/virt/Makefile | 1 +
drivers/virt/acrn/Kconfig | 15 +
drivers/virt/acrn/Makefile | 3 +
drivers/virt/acrn/acrn_drv.h | 228 +++++++
drivers/virt/acrn/hsm.c | 427 ++++++++++++
drivers/virt/acrn/hypercall.h | 254 +++++++
drivers/virt/acrn/ioeventfd.c | 276 ++++++++
drivers/virt/acrn/ioreq.c | 645 ++++++++++++++++++
drivers/virt/acrn/irqfd.c | 236 +++++++
drivers/virt/acrn/mm.c | 303 ++++++++
drivers/virt/acrn/vm.c | 121 ++++
include/uapi/linux/acrn.h | 486 +++++++++++++
21 files changed, 3267 insertions(+), 1 deletion(-)
create mode 100644 Documentation/virt/acrn/index.rst
create mode 100644 Documentation/virt/acrn/introduction.rst
create mode 100644 Documentation/virt/acrn/io-request.rst
create mode 100644 arch/x86/include/asm/acrn.h
create mode 100644 drivers/virt/acrn/Kconfig
create mode 100644 drivers/virt/acrn/Makefile
create mode 100644 drivers/virt/acrn/acrn_drv.h
create mode 100644 drivers/virt/acrn/hsm.c
create mode 100644 drivers/virt/acrn/hypercall.h
create mode 100644 drivers/virt/acrn/ioeventfd.c
create mode 100644 drivers/virt/acrn/ioreq.c
create mode 100644 drivers/virt/acrn/irqfd.c
create mode 100644 drivers/virt/acrn/mm.c
create mode 100644 drivers/virt/acrn/vm.c
create mode 100644 include/uapi/linux/acrn.h
base-commit: 18445bf405cb331117bc98427b1ba6f12418ad17
--
2.28.0
From: Shuo Liu <[email protected]>
The ACRN Hypervisor builds an I/O request when a trapped I/O access
happens in User VM. Then, ACRN Hypervisor issues an upcall by sending
a notification interrupt to the Service VM. HSM in the Service VM needs
to hook the notification interrupt to handle I/O requests.
Notification interrupts from ACRN Hypervisor are already supported and
a, currently uninitialized, callback called.
Export two APIs for HSM to setup/remove its callback.
Originally-by: Yakui Zhao <[email protected]>
Signed-off-by: Shuo Liu <[email protected]>
Reviewed-by: Zhi Wang <[email protected]>
Reviewed-by: Reinette Chatre <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: Sean Christopherson <[email protected]>
Cc: Dan Williams <[email protected]>
Cc: Fengwei Yin <[email protected]>
Cc: Zhi Wang <[email protected]>
Cc: Zhenyu Wang <[email protected]>
Cc: Yu Wang <[email protected]>
Cc: Reinette Chatre <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
---
arch/x86/include/asm/acrn.h | 8 ++++++++
arch/x86/kernel/cpu/acrn.c | 19 +++++++++++++++++++
2 files changed, 27 insertions(+)
create mode 100644 arch/x86/include/asm/acrn.h
diff --git a/arch/x86/include/asm/acrn.h b/arch/x86/include/asm/acrn.h
new file mode 100644
index 000000000000..ff259b69cde7
--- /dev/null
+++ b/arch/x86/include/asm/acrn.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_X86_ACRN_H
+#define _ASM_X86_ACRN_H
+
+void acrn_setup_intr_handler(void (*handler)(void));
+void acrn_remove_intr_handler(void);
+
+#endif /* _ASM_X86_ACRN_H */
diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c
index 0b2c03943ac6..bd1d7e759a0f 100644
--- a/arch/x86/kernel/cpu/acrn.c
+++ b/arch/x86/kernel/cpu/acrn.c
@@ -9,7 +9,11 @@
*
*/
+#define pr_fmt(fmt) "acrn: " fmt
+
#include <linux/interrupt.h>
+
+#include <asm/acrn.h>
#include <asm/apic.h>
#include <asm/cpufeatures.h>
#include <asm/desc.h>
@@ -55,6 +59,21 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_acrn_hv_callback)
set_irq_regs(old_regs);
}
+void acrn_setup_intr_handler(void (*handler)(void))
+{
+ if (acrn_intr_handler)
+ pr_warn("Overwrite the acrn_intr_handler.\n");
+
+ acrn_intr_handler = handler;
+}
+EXPORT_SYMBOL_GPL(acrn_setup_intr_handler);
+
+void acrn_remove_intr_handler(void)
+{
+ acrn_intr_handler = NULL;
+}
+EXPORT_SYMBOL_GPL(acrn_remove_intr_handler);
+
const __initconst struct hypervisor_x86 x86_hyper_acrn = {
.name = "ACRN",
.detect = acrn_detect,
--
2.28.0
From: Shuo Liu <[email protected]>
ACRN supports partition mode to achieve real-time requirements. In
partition mode, a CPU core can be dedicated to a vCPU of User VM. The
local APIC of the dedicated CPU core can be passthrough to the User VM.
The Service VM controls the assignment of the CPU cores.
Introduce an interface for the Service VM to remove the control of CPU
core from hypervisor perspective so that the CPU core can be a dedicated
CPU core of User VM.
Signed-off-by: Shuo Liu <[email protected]>
Reviewed-by: Zhi Wang <[email protected]>
Reviewed-by: Reinette Chatre <[email protected]>
Cc: Zhi Wang <[email protected]>
Cc: Zhenyu Wang <[email protected]>
Cc: Yu Wang <[email protected]>
Cc: Reinette Chatre <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
---
drivers/virt/acrn/hsm.c | 49 +++++++++++++++++++++++++++++++++++
drivers/virt/acrn/hypercall.h | 14 ++++++++++
2 files changed, 63 insertions(+)
diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c
index f5bb7ae34001..dbca38323c15 100644
--- a/drivers/virt/acrn/hsm.c
+++ b/drivers/virt/acrn/hsm.c
@@ -12,6 +12,7 @@
#define pr_fmt(fmt) "acrn: " fmt
#define dev_fmt(fmt) "acrn: " fmt
+#include <linux/cpu.h>
#include <linux/io.h>
#include <linux/miscdevice.h>
#include <linux/mm.h>
@@ -344,6 +345,47 @@ static struct miscdevice acrn_dev = {
.fops = &acrn_fops,
};
+static ssize_t remove_cpu_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ u64 cpu, lapicid;
+ int ret;
+
+ if (kstrtoull(buf, 0, &cpu) < 0)
+ return -EINVAL;
+
+ if (cpu >= num_possible_cpus() || cpu == 0 || !cpu_is_hotpluggable(cpu))
+ return -EINVAL;
+
+ if (cpu_online(cpu))
+ remove_cpu(cpu);
+
+ lapicid = cpu_data(cpu).apicid;
+ dev_dbg(dev, "Try to remove cpu %lld with lapicid %lld\n", cpu, lapicid);
+ ret = hcall_sos_remove_cpu(lapicid);
+ if (ret < 0) {
+ dev_err(dev, "Failed to remove cpu %lld!\n", cpu);
+ goto fail_remove;
+ }
+
+ return count;
+
+fail_remove:
+ add_cpu(cpu);
+ return ret;
+}
+static DEVICE_ATTR_WO(remove_cpu);
+
+static struct attribute *acrn_attrs[] = {
+ &dev_attr_remove_cpu.attr,
+ NULL
+};
+
+static struct attribute_group acrn_attr_group = {
+ .attrs = acrn_attrs,
+};
+
static int __init hsm_init(void)
{
int ret;
@@ -361,13 +403,20 @@ static int __init hsm_init(void)
}
dev = acrn_dev.this_device;
+ if (sysfs_create_group(&dev->kobj, &acrn_attr_group)) {
+ dev_warn(dev, "sysfs create failed\n");
+ misc_deregister(&acrn_dev);
+ return -EINVAL;
+ }
acrn_ioreq_intr_setup();
+
return 0;
}
static void __exit hsm_exit(void)
{
acrn_ioreq_intr_remove();
+ sysfs_remove_group(&acrn_dev.this_device->kobj, &acrn_attr_group);
misc_deregister(&acrn_dev);
}
module_init(hsm_init);
diff --git a/drivers/virt/acrn/hypercall.h b/drivers/virt/acrn/hypercall.h
index e640632366f0..0cfad05bd1a9 100644
--- a/drivers/virt/acrn/hypercall.h
+++ b/drivers/virt/acrn/hypercall.h
@@ -13,6 +13,9 @@
#define HC_ID 0x80UL
+#define HC_ID_GEN_BASE 0x0UL
+#define HC_SOS_REMOVE_CPU _HC_ID(HC_ID, HC_ID_GEN_BASE + 0x01)
+
#define HC_ID_VM_BASE 0x10UL
#define HC_CREATE_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x00)
#define HC_DESTROY_VM _HC_ID(HC_ID, HC_ID_VM_BASE + 0x01)
@@ -42,6 +45,17 @@
#define HC_ID_PM_BASE 0x80UL
#define HC_PM_GET_CPU_STATE _HC_ID(HC_ID, HC_ID_PM_BASE + 0x00)
+/**
+ * hcall_sos_remove_cpu() - Remove a vCPU of Service VM
+ * @cpu: The vCPU to be removed
+ *
+ * Return: 0 on success, <0 on failure
+ */
+static inline long hcall_sos_remove_cpu(u64 cpu)
+{
+ return acrn_hypercall1(HC_SOS_REMOVE_CPU, cpu);
+}
+
/**
* hcall_create_vm() - Create a User VM
* @vminfo: Service VM GPA of info of User VM creation
--
2.28.0