2021-08-02 19:29:29

by Oliver Upton

[permalink] [raw]
Subject: [PATCH v3 0/3] KVM: arm64: Use generic guest entry infrastructure

The arm64 kernel doesn't yet support the full generic entry
infrastructure. That being said, KVM/arm64 doesn't properly handle
TIF_NOTIFY_RESUME and could pick this up by switching to the generic
guest entry infrasturture.

Patch 1 adds a missing vCPU stat to ARM64 to record the number of signal
exits to userspace.

Patch 2 unhitches entry-kvm from entry-generic, as ARM64 doesn't
currently support the generic infrastructure.

Patch 3 replaces the open-coded entry handling with the generic xfer
function.

This series was tested on an Ampere Mt. Jade reference system. The
series cleanly applies to kvm/queue (note that this is deliberate as the
generic kvm stats patches have not yet propagated to kvm-arm/queue) at
the following commit:

8ad5e63649ff ("KVM: Don't take mmu_lock for range invalidation unless necessary")

v1 -> v2:
- Address Jing's comment
- Carry Jing's r-b tag

v2 -> v3:
- Roll all exit conditions into kvm_vcpu_exit_request() (Marc)
- Avoid needlessly checking for work twice (Marc)

v1: http://lore.kernel.org/r/[email protected]
v2: http://lore.kernel.org/r/[email protected]

Oliver Upton (3):
KVM: arm64: Record number of signal exits as a vCPU stat
entry: KVM: Allow use of generic KVM entry w/o full generic support
KVM: arm64: Use generic KVM xfer to guest work function

arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/kvm/Kconfig | 1 +
arch/arm64/kvm/arm.c | 71 +++++++++++++++++++------------
arch/arm64/kvm/guest.c | 1 +
include/linux/entry-kvm.h | 6 ++-
5 files changed, 52 insertions(+), 28 deletions(-)

--
2.32.0.554.ge1b32706d8-goog



2021-08-02 19:29:36

by Oliver Upton

[permalink] [raw]
Subject: [PATCH v3 1/3] KVM: arm64: Record number of signal exits as a vCPU stat

Most other architectures that implement KVM record a statistic
indicating the number of times a vCPU has exited due to a pending
signal. Add support for that stat to arm64.

Reviewed-by: Jing Zhang <[email protected]>
Signed-off-by: Oliver Upton <[email protected]>
---
arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/kvm/arm.c | 1 +
arch/arm64/kvm/guest.c | 1 +
3 files changed, 3 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 41911585ae0c..70e129f2b574 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -576,6 +576,7 @@ struct kvm_vcpu_stat {
u64 wfi_exit_stat;
u64 mmio_exit_user;
u64 mmio_exit_kernel;
+ u64 signal_exits;
u64 exits;
};

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index e9a2b8f27792..60d0a546d7fd 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -783,6 +783,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
if (signal_pending(current)) {
ret = -EINTR;
run->exit_reason = KVM_EXIT_INTR;
+ ++vcpu->stat.signal_exits;
}

/*
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index 1dfb83578277..853d1e8d2e73 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -50,6 +50,7 @@ const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = {
STATS_DESC_COUNTER(VCPU, wfi_exit_stat),
STATS_DESC_COUNTER(VCPU, mmio_exit_user),
STATS_DESC_COUNTER(VCPU, mmio_exit_kernel),
+ STATS_DESC_COUNTER(VCPU, signal_exits),
STATS_DESC_COUNTER(VCPU, exits)
};
static_assert(ARRAY_SIZE(kvm_vcpu_stats_desc) ==
--
2.32.0.554.ge1b32706d8-goog


2021-08-02 19:29:44

by Oliver Upton

[permalink] [raw]
Subject: [PATCH v3 2/3] entry: KVM: Allow use of generic KVM entry w/o full generic support

Some architectures (e.g. arm64) have yet to adopt the generic entry
infrastructure. Despite that, it would be nice to use some common
plumbing for guest entry/exit handling. For example, KVM/arm64 currently
does not handle TIF_NOTIFY_PENDING correctly.

Allow use of only the generic KVM entry code by tightening up the
include list. No functional change intended.

Signed-off-by: Oliver Upton <[email protected]>
---
include/linux/entry-kvm.h | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/include/linux/entry-kvm.h b/include/linux/entry-kvm.h
index 136b8d97d8c0..0d7865a0731c 100644
--- a/include/linux/entry-kvm.h
+++ b/include/linux/entry-kvm.h
@@ -2,7 +2,11 @@
#ifndef __LINUX_ENTRYKVM_H
#define __LINUX_ENTRYKVM_H

-#include <linux/entry-common.h>
+#include <linux/static_call_types.h>
+#include <linux/tracehook.h>
+#include <linux/syscalls.h>
+#include <linux/seccomp.h>
+#include <linux/sched.h>
#include <linux/tick.h>

/* Transfer to guest mode work */
--
2.32.0.554.ge1b32706d8-goog


2021-08-02 19:30:47

by Oliver Upton

[permalink] [raw]
Subject: [PATCH v3 3/3] KVM: arm64: Use generic KVM xfer to guest work function

Clean up handling of checks for pending work by switching to the generic
infrastructure to do so.

We pick up handling for TIF_NOTIFY_RESUME from this switch, meaning that
task work will be correctly handled.

Signed-off-by: Oliver Upton <[email protected]>
---
arch/arm64/kvm/Kconfig | 1 +
arch/arm64/kvm/arm.c | 72 ++++++++++++++++++++++++++----------------
2 files changed, 45 insertions(+), 28 deletions(-)

diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index a4eba0908bfa..8bc1fac5fa26 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -26,6 +26,7 @@ menuconfig KVM
select HAVE_KVM_ARCH_TLB_FLUSH_ALL
select KVM_MMIO
select KVM_GENERIC_DIRTYLOG_READ_PROTECT
+ select KVM_XFER_TO_GUEST_WORK
select SRCU
select KVM_VFIO
select HAVE_KVM_EVENTFD
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 60d0a546d7fd..8245efc6e88f 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -6,6 +6,7 @@

#include <linux/bug.h>
#include <linux/cpu_pm.h>
+#include <linux/entry-kvm.h>
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/kvm_host.h>
@@ -714,6 +715,45 @@ static bool vcpu_mode_is_bad_32bit(struct kvm_vcpu *vcpu)
static_branch_unlikely(&arm64_mismatched_32bit_el0);
}

+/**
+ * kvm_vcpu_exit_request - returns true if the VCPU should *not* enter the guest
+ * @vcpu: The VCPU pointer
+ * @ret: Pointer to write optional return code
+ *
+ * Returns: true if the VCPU needs to return to a preemptible + interruptible
+ * and skip guest entry.
+ *
+ * This function disambiguates between two different types of exits: exits to a
+ * preemptible + interruptible kernel context and exits to userspace. For an
+ * exit to userspace, this function will write the return code to ret and return
+ * true. For an exit to preemptible + interruptible kernel context (i.e. check
+ * for pending work and re-enter), return true without writing to ret.
+ */
+static bool kvm_vcpu_exit_request(struct kvm_vcpu *vcpu, int *ret)
+{
+ struct kvm_run *run = vcpu->run;
+
+ /*
+ * If we're using a userspace irqchip, then check if we need
+ * to tell a userspace irqchip about timer or PMU level
+ * changes and if so, exit to userspace (the actual level
+ * state gets updated in kvm_timer_update_run and
+ * kvm_pmu_update_run below).
+ */
+ if (static_branch_unlikely(&userspace_irqchip_in_use)) {
+ if (kvm_timer_should_notify_user(vcpu) ||
+ kvm_pmu_should_notify_user(vcpu)) {
+ *ret = -EINTR;
+ run->exit_reason = KVM_EXIT_INTR;
+ return true;
+ }
+ }
+
+ return kvm_request_pending(vcpu) ||
+ need_new_vmid_gen(&vcpu->arch.hw_mmu->vmid) ||
+ xfer_to_guest_mode_work_pending();
+}
+
/**
* kvm_arch_vcpu_ioctl_run - the main VCPU run function to execute guest code
* @vcpu: The VCPU pointer
@@ -757,7 +797,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
/*
* Check conditions before entering the guest
*/
- cond_resched();
+ ret = xfer_to_guest_mode_handle_work(vcpu);
+ if (!ret)
+ ret = 1;

update_vmid(&vcpu->arch.hw_mmu->vmid);

@@ -776,31 +818,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)

kvm_vgic_flush_hwstate(vcpu);

- /*
- * Exit if we have a signal pending so that we can deliver the
- * signal to user space.
- */
- if (signal_pending(current)) {
- ret = -EINTR;
- run->exit_reason = KVM_EXIT_INTR;
- ++vcpu->stat.signal_exits;
- }
-
- /*
- * If we're using a userspace irqchip, then check if we need
- * to tell a userspace irqchip about timer or PMU level
- * changes and if so, exit to userspace (the actual level
- * state gets updated in kvm_timer_update_run and
- * kvm_pmu_update_run below).
- */
- if (static_branch_unlikely(&userspace_irqchip_in_use)) {
- if (kvm_timer_should_notify_user(vcpu) ||
- kvm_pmu_should_notify_user(vcpu)) {
- ret = -EINTR;
- run->exit_reason = KVM_EXIT_INTR;
- }
- }
-
/*
* Ensure we set mode to IN_GUEST_MODE after we disable
* interrupts and before the final VCPU requests check.
@@ -809,8 +826,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
*/
smp_store_mb(vcpu->mode, IN_GUEST_MODE);

- if (ret <= 0 || need_new_vmid_gen(&vcpu->arch.hw_mmu->vmid) ||
- kvm_request_pending(vcpu)) {
+ if (ret <= 0 || kvm_vcpu_exit_request(vcpu, &ret)) {
vcpu->mode = OUTSIDE_GUEST_MODE;
isb(); /* Ensure work in x_flush_hwstate is committed */
kvm_pmu_sync_hwstate(vcpu);
--
2.32.0.554.ge1b32706d8-goog


2021-08-15 07:52:12

by Oliver Upton

[permalink] [raw]
Subject: Re: [PATCH v3 0/3] KVM: arm64: Use generic guest entry infrastructure

Friendly ping :)

--
Thanks,
Oliver

On Mon, Aug 2, 2021 at 12:28 PM Oliver Upton <[email protected]> wrote:
>
> The arm64 kernel doesn't yet support the full generic entry
> infrastructure. That being said, KVM/arm64 doesn't properly handle
> TIF_NOTIFY_RESUME and could pick this up by switching to the generic
> guest entry infrasturture.
>
> Patch 1 adds a missing vCPU stat to ARM64 to record the number of signal
> exits to userspace.
>
> Patch 2 unhitches entry-kvm from entry-generic, as ARM64 doesn't
> currently support the generic infrastructure.
>
> Patch 3 replaces the open-coded entry handling with the generic xfer
> function.
>
> This series was tested on an Ampere Mt. Jade reference system. The
> series cleanly applies to kvm/queue (note that this is deliberate as the
> generic kvm stats patches have not yet propagated to kvm-arm/queue) at
> the following commit:
>
> 8ad5e63649ff ("KVM: Don't take mmu_lock for range invalidation unless necessary")
>
> v1 -> v2:
> - Address Jing's comment
> - Carry Jing's r-b tag
>
> v2 -> v3:
> - Roll all exit conditions into kvm_vcpu_exit_request() (Marc)
> - Avoid needlessly checking for work twice (Marc)
>
> v1: http://lore.kernel.org/r/[email protected]
> v2: http://lore.kernel.org/r/[email protected]
>
> Oliver Upton (3):
> KVM: arm64: Record number of signal exits as a vCPU stat
> entry: KVM: Allow use of generic KVM entry w/o full generic support
> KVM: arm64: Use generic KVM xfer to guest work function
>
> arch/arm64/include/asm/kvm_host.h | 1 +
> arch/arm64/kvm/Kconfig | 1 +
> arch/arm64/kvm/arm.c | 71 +++++++++++++++++++------------
> arch/arm64/kvm/guest.c | 1 +
> include/linux/entry-kvm.h | 6 ++-
> 5 files changed, 52 insertions(+), 28 deletions(-)
>
> --
> 2.32.0.554.ge1b32706d8-goog
>

2021-08-15 09:22:24

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH v3 0/3] KVM: arm64: Use generic guest entry infrastructure

On Sun, 15 Aug 2021 08:50:09 +0100,
Oliver Upton <[email protected]> wrote:
>
> Friendly ping :)

Ah, thanks for reminding me of this. I'll have a look again early next
week.

M.

--
Without deviation from the norm, progress is not possible.

2021-08-19 10:30:32

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH v3 0/3] KVM: arm64: Use generic guest entry infrastructure

On Mon, 2 Aug 2021 19:28:06 +0000, Oliver Upton wrote:
> The arm64 kernel doesn't yet support the full generic entry
> infrastructure. That being said, KVM/arm64 doesn't properly handle
> TIF_NOTIFY_RESUME and could pick this up by switching to the generic
> guest entry infrasturture.
>
> Patch 1 adds a missing vCPU stat to ARM64 to record the number of signal
> exits to userspace.
>
> [...]

Applied to next, thanks!

[1/3] KVM: arm64: Record number of signal exits as a vCPU stat
commit: fe5161d2c39b8c2801f0e786631460c6e8a1cae4
[2/3] entry: KVM: Allow use of generic KVM entry w/o full generic support
commit: e1c6b9e1669e44fb7f9688e34e460b759e3b9187
[3/3] KVM: arm64: Use generic KVM xfer to guest work function
commit: 6caa5812e2d126a0aa8a17816c1ba6f0a0c2b309

Cheers,

M.
--
Without deviation from the norm, progress is not possible.