This series introduces support for creating and running guest machines
while running on the Microsoft Hypervisor. [0]
This is done via an IOCTL interface accessed through /dev/mshv, similar to
/dev/kvm. Another series introducing this support was previously posted.
[1]
These interfaces support VMMs running in:
1. The root patition - provided in the mshv_root module, and
2. VTL 2 - provided in the mshv_vtl module [2]
Patches breakdown
-----------------
The first 7 patches are refactoring and adding some helper functions.
They provide some benefit on their own and could be applied independently
as cleanup patches.
Patches 8-12 just set things up for the driver code to come. These are very
small. They come first so that the remaining patches are more self-contained.
The final 3 patches are the meat of the series:
- Patch 13 contains new header files used by the driver.
These are designed to mirror the ABI headers exported by Hyper-V. This is
done to avoid polluting hyperv-tlfs.h and help track changes to the ABIs
that are still unstable. (See FAQ below).
- Patch 14 conditionally includes these new header files into mshyperv.h
and linux/hyperv.h, in order to be able to use these files in the new
drivers while remaining independent from hyperv-tlfs.h.
- Patch 15 contains the new driver code located in drivers/hv. This is a
large amount of code and new files, but it is mostly self-contained and
all within drivers/hv - apart from the IOCTL interface itself in uapi.
Patch 15 is rather big and has bounced back from some mailing lists. If you
did not get a copy in your inbox, you can view it here instead:
https://github.com/NunoDasNeves/linux/commit/4dd436f13d95a6a0daecd75da012a2caa6c57c8c
FAQ on include/uapi/hyperv/*.h
------------------------------
Q:
Why not just add these definitions to hyperv-tlfs.h?
A:
The intention of hyperv-tlfs.h is to contain stable definitions documented
in the public TLFS document. These new definitions don't fit that criteria,
so they should be separate.
Q:
Why are these files named hvgdk.h, hvgdk_mini.h, hvhdk.h and hvhdk_mini.h?
A:
The precise meaning of the names reflects conventions used internally at
Microsoft.
Naming them this way makes it easy to find where particular Hyper-V
definitions come from, and check their correctness.
It also facilitates the future work of automatically generating these files.
Q:
Why are they in uapi?
A:
In short, to keep things simple. There are many definitions needed in both
the kernel and the VMM in userspace. Separating them doesn't serve much
purpose, and makes it more laborious to import definitions from Hyper-V
code.
Q:
The new headers redefine many things that are already in hyperv-tlfs.h - why?
A:
Some definitions are extended compared to what is documented in the TLFS.
In order to avoid adding undocumented or unstable definitions to hyperv-tlfs.h,
the new headers must compile independently.
Therefore, the new headers must redefine many things in hyperv-tlfs.h in order
to compile.
--------------------------
[0] "Hyper-V" is more well-known, but it really refers to the whole stack
including the hypervisor and other components that run in Windows
kernel and userspace.
[1] Previous /dev/mshv patch series and discussion:
https://lore.kernel.org/linux-hyperv/1632853875-20261-1-git-send-email-nunodasneves@linux.microsoft.com/
[2] Virtual Secure Mode (VSM) and Virtual Trust Levels (VTL):
https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/vsm
--------------------------
Changes since v1:
* Clean up formatting, capitalization in commit messages
* Add detail to commit message for patch 15
* Remove errant lines in Makefile and Kconfig in patch 15
* Move a reference to CONFIG_MSHV_VTL from patch 9 to 15
Nuno Das Neves (15):
hyperv-tlfs: Change shared HV_REGISTER_* defines to HV_MSR_*
mshyperv: Introduce hv_get_hypervisor_version function
mshyperv: Introduce numa_node_to_proximity_domain_info
asm-generic/mshyperv: Introduce hv_recommend_using_aeoi()
hyperv: Move hv_connection_id to hyperv-tlfs
hyperv-tlfs: Introduce hv_status_to_string and hv_status_to_errno
Drivers: hv: Move hv_call_deposit_pages and hv_call_create_vp to
common code
Drivers: hv: Introduce per-cpu event ring tail
Drivers: hv: Introduce hv_output_arg_exists in hv_common.c
x86: hyperv: Add mshv_handler irq handler and setup function
Drivers: hv: export vmbus_isr, hv_context and hv_post_message
Documentation: Reserve ioctl number for mshv driver
uapi: hyperv: Add mshv driver headers hvhdk.h, hvhdk_mini.h, hvgdk.h,
hvgdk_mini.h
asm-generic: hyperv: Use mshv headers conditionally. Add
asm-generic/hyperv-defs.h
Drivers: hv: Add modules to expose /dev/mshv to VMMs running on
Hyper-V
.../userspace-api/ioctl/ioctl-number.rst | 2 +
arch/arm64/hyperv/mshyperv.c | 23 +-
arch/arm64/include/asm/hyperv-tlfs.h | 25 +
arch/arm64/include/asm/mshyperv.h | 2 +-
arch/x86/hyperv/hv_init.c | 2 +-
arch/x86/hyperv/hv_proc.c | 166 +-
arch/x86/include/asm/hyperv-tlfs.h | 137 +-
arch/x86/include/asm/mshyperv.h | 13 +-
arch/x86/kernel/cpu/mshyperv.c | 71 +-
drivers/acpi/numa/srat.c | 1 +
drivers/clocksource/hyperv_timer.c | 24 +-
drivers/hv/Kconfig | 50 +
drivers/hv/Makefile | 20 +
drivers/hv/hv.c | 46 +-
drivers/hv/hv_call.c | 119 +
drivers/hv/hv_common.c | 222 +-
drivers/hv/hyperv_vmbus.h | 2 +-
drivers/hv/mshv.h | 156 ++
drivers/hv/mshv_eventfd.c | 758 +++++++
drivers/hv/mshv_eventfd.h | 80 +
drivers/hv/mshv_main.c | 208 ++
drivers/hv/mshv_msi.c | 129 ++
drivers/hv/mshv_portid_table.c | 84 +
drivers/hv/mshv_root.h | 194 ++
drivers/hv/mshv_root_hv_call.c | 1064 +++++++++
drivers/hv/mshv_root_main.c | 1964 +++++++++++++++++
drivers/hv/mshv_synic.c | 689 ++++++
drivers/hv/mshv_vtl.h | 52 +
drivers/hv/mshv_vtl_main.c | 1542 +++++++++++++
drivers/hv/vmbus_drv.c | 3 +-
drivers/hv/xfer_to_guest.c | 28 +
include/asm-generic/hyperv-defs.h | 26 +
include/asm-generic/hyperv-tlfs.h | 77 +-
include/asm-generic/mshyperv.h | 76 +-
include/linux/hyperv.h | 11 +-
include/uapi/hyperv/hvgdk.h | 41 +
include/uapi/hyperv/hvgdk_mini.h | 1077 +++++++++
include/uapi/hyperv/hvhdk.h | 1352 ++++++++++++
include/uapi/hyperv/hvhdk_mini.h | 164 ++
include/uapi/linux/mshv.h | 298 +++
40 files changed, 10645 insertions(+), 353 deletions(-)
create mode 100644 drivers/hv/hv_call.c
create mode 100644 drivers/hv/mshv.h
create mode 100644 drivers/hv/mshv_eventfd.c
create mode 100644 drivers/hv/mshv_eventfd.h
create mode 100644 drivers/hv/mshv_main.c
create mode 100644 drivers/hv/mshv_msi.c
create mode 100644 drivers/hv/mshv_portid_table.c
create mode 100644 drivers/hv/mshv_root.h
create mode 100644 drivers/hv/mshv_root_hv_call.c
create mode 100644 drivers/hv/mshv_root_main.c
create mode 100644 drivers/hv/mshv_synic.c
create mode 100644 drivers/hv/mshv_vtl.h
create mode 100644 drivers/hv/mshv_vtl_main.c
create mode 100644 drivers/hv/xfer_to_guest.c
create mode 100644 include/asm-generic/hyperv-defs.h
create mode 100644 include/uapi/hyperv/hvgdk.h
create mode 100644 include/uapi/hyperv/hvgdk_mini.h
create mode 100644 include/uapi/hyperv/hvhdk.h
create mode 100644 include/uapi/hyperv/hvhdk_mini.h
create mode 100644 include/uapi/linux/mshv.h
--
2.25.1
This will handle SYNIC interrupts such as intercepts, doorbells, and
scheduling messages intended for the mshv driver.
Signed-off-by: Nuno Das Neves <[email protected]>
Reviewed-by: Wei Liu <[email protected]>
---
arch/x86/kernel/cpu/mshyperv.c | 9 +++++++++
drivers/hv/hv_common.c | 5 +++++
include/asm-generic/mshyperv.h | 2 ++
3 files changed, 16 insertions(+)
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index e80e503c8cf5..6568d73e7d60 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -105,6 +105,7 @@ void hv_set_register(unsigned int reg, u64 value)
}
EXPORT_SYMBOL_GPL(hv_set_register);
+static void (*mshv_handler)(void);
static void (*vmbus_handler)(void);
static void (*hv_stimer0_handler)(void);
static void (*hv_kexec_handler)(void);
@@ -115,6 +116,9 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_hyperv_callback)
struct pt_regs *old_regs = set_irq_regs(regs);
inc_irq_stat(irq_hv_callback_count);
+ if (mshv_handler)
+ mshv_handler();
+
if (vmbus_handler)
vmbus_handler();
@@ -124,6 +128,11 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_hyperv_callback)
set_irq_regs(old_regs);
}
+void hv_setup_mshv_irq(void (*handler)(void))
+{
+ mshv_handler = handler;
+}
+
void hv_setup_vmbus_handler(void (*handler)(void))
{
vmbus_handler = handler;
diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
index 79efedc8bf39..13f972e72375 100644
--- a/drivers/hv/hv_common.c
+++ b/drivers/hv/hv_common.c
@@ -548,6 +548,11 @@ void __weak hv_remove_vmbus_handler(void)
}
EXPORT_SYMBOL_GPL(hv_remove_vmbus_handler);
+void __weak hv_setup_mshv_irq(void (*handler)(void))
+{
+}
+EXPORT_SYMBOL_GPL(hv_setup_mshv_irq);
+
void __weak hv_setup_kexec_handler(void (*handler)(void))
{
}
diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
index 9118d678b27a..2b20994d809e 100644
--- a/include/asm-generic/mshyperv.h
+++ b/include/asm-generic/mshyperv.h
@@ -193,6 +193,8 @@ void hv_remove_vmbus_handler(void);
void hv_setup_stimer0_handler(void (*handler)(void));
void hv_remove_stimer0_handler(void);
+void hv_setup_mshv_irq(void (*handler)(void));
+
void hv_setup_kexec_handler(void (*handler)(void));
void hv_remove_kexec_handler(void);
void hv_setup_crash_handler(void (*handler)(struct pt_regs *regs));
--
2.25.1
x86_64 and arm64 implementations to get the hypervisor version
information. Store these in hv_hypervisor_version_info structure to
simplify parsing the fields.
Replace the existing parsing when printing the version numbers at boot.
Signed-off-by: Nuno Das Neves <[email protected]>
---
arch/arm64/hyperv/mshyperv.c | 23 +++++++++++-------
arch/x86/kernel/cpu/mshyperv.c | 40 ++++++++++++++++++++-----------
include/asm-generic/hyperv-tlfs.h | 23 ++++++++++++++++++
include/asm-generic/mshyperv.h | 2 ++
4 files changed, 66 insertions(+), 22 deletions(-)
diff --git a/arch/arm64/hyperv/mshyperv.c b/arch/arm64/hyperv/mshyperv.c
index f1b8a04ee9f2..53bdd3bc81ac 100644
--- a/arch/arm64/hyperv/mshyperv.c
+++ b/arch/arm64/hyperv/mshyperv.c
@@ -19,10 +19,19 @@
static bool hyperv_initialized;
+int hv_get_hypervisor_version(union hv_hypervisor_version_info *info)
+{
+ hv_get_vpreg_128(HV_REGISTER_HYPERVISOR_VERSION,
+ (struct hv_get_vp_registers_output *)info);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(hv_get_hypervisor_version);
+
static int __init hyperv_init(void)
{
- struct hv_get_vp_registers_output result;
- u32 a, b, c, d;
+ struct hv_get_vp_registers_output result;
+ union hv_hypervisor_version_info version;
u64 guest_id;
int ret;
@@ -55,13 +64,11 @@ static int __init hyperv_init(void)
ms_hyperv.misc_features);
/* Get information about the Hyper-V host version */
- hv_get_vpreg_128(HV_REGISTER_HYPERVISOR_VERSION, &result);
- a = result.as32.a;
- b = result.as32.b;
- c = result.as32.c;
- d = result.as32.d;
+ hv_get_hypervisor_version(&version);
pr_info("Hyper-V: Host Build %d.%d.%d.%d-%d-%d\n",
- b >> 16, b & 0xFFFF, a, d & 0xFFFFFF, c, d >> 24);
+ version.major_version, version.minor_version,
+ version.build_number, version.service_number,
+ version.service_pack, version.service_branch);
ret = hv_common_init();
if (ret)
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 4d38cafe6f2c..e80e503c8cf5 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -313,13 +313,30 @@ static void __init hv_smp_prepare_cpus(unsigned int max_cpus)
}
#endif
+int hv_get_hypervisor_version(union hv_hypervisor_version_info *info)
+{
+ unsigned int hv_max_functions;
+
+ hv_max_functions = cpuid_eax(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS);
+ if (hv_max_functions < HYPERV_CPUID_VERSION) {
+ pr_err("%s: Could not detect Hyper-V version\n",
+ __func__);
+ return -ENODEV;
+ }
+
+ info->eax = cpuid_eax(HYPERV_CPUID_VERSION);
+ info->ebx = cpuid_ebx(HYPERV_CPUID_VERSION);
+ info->ecx = cpuid_ecx(HYPERV_CPUID_VERSION);
+ info->edx = cpuid_edx(HYPERV_CPUID_VERSION);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(hv_get_hypervisor_version);
+
static void __init ms_hyperv_init_platform(void)
{
int hv_max_functions_eax;
- int hv_host_info_eax;
- int hv_host_info_ebx;
- int hv_host_info_ecx;
- int hv_host_info_edx;
+ union hv_hypervisor_version_info version;
#ifdef CONFIG_PARAVIRT
pv_info.name = "Hyper-V";
@@ -373,16 +390,11 @@ static void __init ms_hyperv_init_platform(void)
/*
* Extract host information.
*/
- if (hv_max_functions_eax >= HYPERV_CPUID_VERSION) {
- hv_host_info_eax = cpuid_eax(HYPERV_CPUID_VERSION);
- hv_host_info_ebx = cpuid_ebx(HYPERV_CPUID_VERSION);
- hv_host_info_ecx = cpuid_ecx(HYPERV_CPUID_VERSION);
- hv_host_info_edx = cpuid_edx(HYPERV_CPUID_VERSION);
-
- pr_info("Hyper-V: Host Build %d.%d.%d.%d-%d-%d\n",
- hv_host_info_ebx >> 16, hv_host_info_ebx & 0xFFFF,
- hv_host_info_eax, hv_host_info_edx & 0xFFFFFF,
- hv_host_info_ecx, hv_host_info_edx >> 24);
+ if (hv_get_hypervisor_version(&version) == 0) {
+ pr_info("Hyper-V Host Build:%d-%d.%d-%d-%d.%d\n",
+ version.build_number, version.major_version,
+ version.minor_version, version.service_pack,
+ version.service_branch, version.service_number);
}
if (ms_hyperv.features & HV_ACCESS_FREQUENCY_MSRS &&
diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
index f4e4cc4f965f..373f26efa18a 100644
--- a/include/asm-generic/hyperv-tlfs.h
+++ b/include/asm-generic/hyperv-tlfs.h
@@ -786,6 +786,29 @@ struct hv_input_unmap_device_interrupt {
#define HV_SOURCE_SHADOW_NONE 0x0
#define HV_SOURCE_SHADOW_BRIDGE_BUS_RANGE 0x1
+/*
+ * Version info reported by hypervisor
+ */
+union hv_hypervisor_version_info {
+ struct {
+ u32 build_number;
+
+ u32 minor_version : 16;
+ u32 major_version : 16;
+
+ u32 service_pack;
+
+ u32 service_number : 24;
+ u32 service_branch : 8;
+ };
+ struct {
+ u32 eax;
+ u32 ebx;
+ u32 ecx;
+ u32 edx;
+ };
+};
+
/*
* The whole argument should fit in a page to be able to pass to the hypervisor
* in one hypercall.
diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
index 094c57320ed1..233c976344e5 100644
--- a/include/asm-generic/mshyperv.h
+++ b/include/asm-generic/mshyperv.h
@@ -153,6 +153,8 @@ static inline void vmbus_signal_eom(struct hv_message *msg, u32 old_msg_type)
}
}
+int hv_get_hypervisor_version(union hv_hypervisor_version_info *info);
+
void hv_setup_vmbus_handler(void (*handler)(void));
void hv_remove_vmbus_handler(void);
void hv_setup_stimer0_handler(void (*handler)(void));
--
2.25.1
Factor out logic for determining if we should set the auto eoi flag in SINT
register.
Signed-off-by: Nuno Das Neves <[email protected]>
Reviewed-by: Wei Liu <[email protected]>
---
drivers/hv/hv.c | 12 +-----------
include/asm-generic/mshyperv.h | 13 +++++++++++++
2 files changed, 14 insertions(+), 11 deletions(-)
diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
index 3ddacdebe886..7929554d32f7 100644
--- a/drivers/hv/hv.c
+++ b/drivers/hv/hv.c
@@ -212,17 +212,7 @@ void hv_synic_enable_regs(unsigned int cpu)
shared_sint.vector = vmbus_interrupt;
shared_sint.masked = false;
-
- /*
- * On architectures where Hyper-V doesn't support AEOI (e.g., ARM64),
- * it doesn't provide a recommendation flag and AEOI must be disabled.
- */
-#ifdef HV_DEPRECATING_AEOI_RECOMMENDED
- shared_sint.auto_eoi =
- !(ms_hyperv.hints & HV_DEPRECATING_AEOI_RECOMMENDED);
-#else
- shared_sint.auto_eoi = 0;
-#endif
+ shared_sint.auto_eoi = hv_recommend_using_aeoi();
hv_set_register(HV_MSR_SINT0 + VMBUS_MESSAGE_SINT,
shared_sint.as_uint64);
diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
index 447e7ebe67ee..90fcbb95f1ee 100644
--- a/include/asm-generic/mshyperv.h
+++ b/include/asm-generic/mshyperv.h
@@ -77,6 +77,19 @@ extern u64 hv_do_hypercall(u64 control, void *inputaddr, void *outputaddr);
extern u64 hv_do_fast_hypercall8(u16 control, u64 input8);
extern bool hv_isolation_type_snp(void);
+/*
+ * On architectures where Hyper-V doesn't support AEOI (e.g., ARM64),
+ * it doesn't provide a recommendation flag and AEOI must be disabled.
+ */
+static inline bool hv_recommend_using_aeoi(void)
+{
+#ifdef HV_DEPRECATING_AEOI_RECOMMENDED
+ return !(ms_hyperv.hints & HV_DEPRECATING_AEOI_RECOMMENDED);
+#else
+ return false;
+#endif
+}
+
/* Helper functions that provide a consistent pattern for checking Hyper-V hypercall status. */
static inline int hv_result(u64 status)
{
--
2.25.1