This series forms the kvm host part of paravirtual spinlock
based against kvm tree.
Please refer https://lkml.org/lkml/2013/8/6/178 for kvm guest part
of the series.
Please note that:
kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi is a common patch
for both guest and host.
Srivatsa Vaddagiri (1):
kvm hypervisor : Add a hypercall to KVM hypervisor to support pv-ticketlocks
Raghavendra K T (4):
kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi
kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration
kvm hypervisor: Simplify kvm_for_each_vcpu with kvm_irq_delivery_to_apic
Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock
---
Documentation/virtual/kvm/cpuid.txt | 4 ++++
Documentation/virtual/kvm/hypercalls.txt | 14 ++++++++++++++
arch/x86/include/asm/kvm_host.h | 5 +++++
arch/x86/include/uapi/asm/kvm_para.h | 1 +
arch/x86/kvm/cpuid.c | 3 ++-
arch/x86/kvm/lapic.c | 5 ++++-
arch/x86/kvm/x86.c | 31 ++++++++++++++++++++++++++++++-
include/uapi/linux/kvm_para.h | 1 +
8 files changed, 61 insertions(+), 3 deletions(-)
kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi
From: Raghavendra K T <[email protected]>
this is needed by both guest and host.
Originally-from: Srivatsa Vaddagiri <[email protected]>
Signed-off-by: Raghavendra K T <[email protected]>
Acked-by: Gleb Natapov <[email protected]>
Acked-by: Ingo Molnar <[email protected]>
---
arch/x86/include/uapi/asm/kvm_para.h | 1 +
include/uapi/linux/kvm_para.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h
index 06fdbd9..94dc8ca 100644
--- a/arch/x86/include/uapi/asm/kvm_para.h
+++ b/arch/x86/include/uapi/asm/kvm_para.h
@@ -23,6 +23,7 @@
#define KVM_FEATURE_ASYNC_PF 4
#define KVM_FEATURE_STEAL_TIME 5
#define KVM_FEATURE_PV_EOI 6
+#define KVM_FEATURE_PV_UNHALT 7
/* The last 8 bits are used to indicate how to interpret the flags field
* in pvclock structure. If no bits are set, all flags are ignored.
diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h
index cea2c5c..2841f86 100644
--- a/include/uapi/linux/kvm_para.h
+++ b/include/uapi/linux/kvm_para.h
@@ -19,6 +19,7 @@
#define KVM_HC_MMU_OP 2
#define KVM_HC_FEATURES 3
#define KVM_HC_PPC_MAP_MAGIC_PAGE 4
+#define KVM_HC_KICK_CPU 5
/*
* hypercalls use architecture specific
kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration
From: Raghavendra K T <[email protected]>
During migration, any vcpu that got kicked but did not become runnable
(still in halted state) should be runnable after migration.
Signed-off-by: Raghavendra K T <[email protected]>
Acked-by: Gleb Natapov <[email protected]>
Acked-by: Ingo Molnar <[email protected]>
---
arch/x86/kvm/x86.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index dae4575..1e73dab 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6284,7 +6284,12 @@ int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu,
struct kvm_mp_state *mp_state)
{
kvm_apic_accept_events(vcpu);
- mp_state->mp_state = vcpu->arch.mp_state;
+ if (vcpu->arch.mp_state == KVM_MP_STATE_HALTED &&
+ vcpu->arch.pv.pv_unhalted)
+ mp_state->mp_state = KVM_MP_STATE_RUNNABLE;
+ else
+ mp_state->mp_state = vcpu->arch.mp_state;
+
return 0;
}
kvm hypervisor: Simplify kvm_for_each_vcpu with kvm_irq_delivery_to_apic
From: Raghavendra K T <[email protected]>
Note that we are using APIC_DM_REMRD which has reserved usage.
In future if APIC_DM_REMRD usage is standardized, then we should
find some other way or go back to old method.
Suggested-by: Gleb Natapov <[email protected]>
Signed-off-by: Raghavendra K T <[email protected]>
Acked-by: Gleb Natapov <[email protected]>
Acked-by: Ingo Molnar <[email protected]>
---
arch/x86/kvm/lapic.c | 5 ++++-
arch/x86/kvm/x86.c | 25 ++++++-------------------
2 files changed, 10 insertions(+), 20 deletions(-)
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index afc1124..48c13c9 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -706,7 +706,10 @@ out:
break;
case APIC_DM_REMRD:
- apic_debug("Ignoring delivery mode 3\n");
+ result = 1;
+ vcpu->arch.pv.pv_unhalted = 1;
+ kvm_make_request(KVM_REQ_EVENT, vcpu);
+ kvm_vcpu_kick(vcpu);
break;
case APIC_DM_SMI:
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 1e73dab..640d112 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5502,27 +5502,14 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
*/
static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid)
{
- struct kvm_vcpu *vcpu = NULL;
- int i;
+ struct kvm_lapic_irq lapic_irq;
- kvm_for_each_vcpu(i, vcpu, kvm) {
- if (!kvm_apic_present(vcpu))
- continue;
+ lapic_irq.shorthand = 0;
+ lapic_irq.dest_mode = 0;
+ lapic_irq.dest_id = apicid;
- if (kvm_apic_match_dest(vcpu, 0, 0, apicid, 0))
- break;
- }
- if (vcpu) {
- /*
- * Setting unhalt flag here can result in spurious runnable
- * state when unhalt reset does not happen in vcpu_block.
- * But that is harmless since that should soon result in halt.
- */
- vcpu->arch.pv.pv_unhalted = true;
- /* We need everybody see unhalt before vcpu unblocks */
- smp_wmb();
- kvm_vcpu_kick(vcpu);
- }
+ lapic_irq.delivery_mode = APIC_DM_REMRD;
+ kvm_irq_delivery_to_apic(kvm, 0, &lapic_irq, NULL);
}
int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
kvm hypervisor : Add a hypercall to KVM hypervisor to support pv-ticketlocks
From: Srivatsa Vaddagiri <[email protected]>
kvm_hc_kick_cpu allows the calling vcpu to kick another vcpu out of halt state.
the presence of these hypercalls is indicated to guest via
kvm_feature_pv_unhalt.
Signed-off-by: Srivatsa Vaddagiri <[email protected]>
Signed-off-by: Suzuki Poulose <[email protected]>
[Raghu: Apic related changes, folding pvunhalted into vcpu_runnable
Added flags for future use (suggested by Gleb)]
Signed-off-by: Raghavendra K T <[email protected]>
Acked-by: Gleb Natapov <[email protected]>
Acked-by: Ingo Molnar <[email protected]>
---
arch/x86/include/asm/kvm_host.h | 5 +++++
arch/x86/kvm/cpuid.c | 3 ++-
arch/x86/kvm/x86.c | 37 +++++++++++++++++++++++++++++++++++++
3 files changed, 44 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index f87f7fc..1d1f711 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -511,6 +511,11 @@ struct kvm_vcpu_arch {
* instruction.
*/
bool write_fault_to_shadow_pgtable;
+
+ /* pv related host specific info */
+ struct {
+ bool pv_unhalted;
+ } pv;
};
struct kvm_lpage_info {
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index a20ecb5..b110fe6 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -413,7 +413,8 @@ static int do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
(1 << KVM_FEATURE_CLOCKSOURCE2) |
(1 << KVM_FEATURE_ASYNC_PF) |
(1 << KVM_FEATURE_PV_EOI) |
- (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT);
+ (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT) |
+ (1 << KVM_FEATURE_PV_UNHALT);
if (sched_info_on())
entry->eax |= (1 << KVM_FEATURE_STEAL_TIME);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index d21bce5..dae4575 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5495,6 +5495,36 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
return 1;
}
+/*
+ * kvm_pv_kick_cpu_op: Kick a vcpu.
+ *
+ * @apicid - apicid of vcpu to be kicked.
+ */
+static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid)
+{
+ struct kvm_vcpu *vcpu = NULL;
+ int i;
+
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (!kvm_apic_present(vcpu))
+ continue;
+
+ if (kvm_apic_match_dest(vcpu, 0, 0, apicid, 0))
+ break;
+ }
+ if (vcpu) {
+ /*
+ * Setting unhalt flag here can result in spurious runnable
+ * state when unhalt reset does not happen in vcpu_block.
+ * But that is harmless since that should soon result in halt.
+ */
+ vcpu->arch.pv.pv_unhalted = true;
+ /* We need everybody see unhalt before vcpu unblocks */
+ smp_wmb();
+ kvm_vcpu_kick(vcpu);
+ }
+}
+
int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
{
unsigned long nr, a0, a1, a2, a3, ret;
@@ -5528,6 +5558,10 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
case KVM_HC_VAPIC_POLL_IRQ:
ret = 0;
break;
+ case KVM_HC_KICK_CPU:
+ kvm_pv_kick_cpu_op(vcpu->kvm, a0, a1);
+ ret = 0;
+ break;
default:
ret = -KVM_ENOSYS;
break;
@@ -5950,6 +5984,7 @@ static int __vcpu_run(struct kvm_vcpu *vcpu)
kvm_apic_accept_events(vcpu);
switch(vcpu->arch.mp_state) {
case KVM_MP_STATE_HALTED:
+ vcpu->arch.pv.pv_unhalted = false;
vcpu->arch.mp_state =
KVM_MP_STATE_RUNNABLE;
case KVM_MP_STATE_RUNNABLE:
@@ -6770,6 +6805,7 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
BUG_ON(vcpu->kvm == NULL);
kvm = vcpu->kvm;
+ vcpu->arch.pv.pv_unhalted = false;
vcpu->arch.emulate_ctxt.ops = &emulate_ops;
if (!irqchip_in_kernel(kvm) || kvm_vcpu_is_bsp(vcpu))
vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE;
@@ -7103,6 +7139,7 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
!vcpu->arch.apf.halted)
|| !list_empty_careful(&vcpu->async_pf.done)
|| kvm_apic_has_events(vcpu)
+ || vcpu->arch.pv.pv_unhalted
|| atomic_read(&vcpu->arch.nmi_queued) ||
(kvm_arch_interrupt_allowed(vcpu) &&
kvm_cpu_has_interrupt(vcpu));
Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock
From: Raghavendra K T <[email protected]>
KVM_HC_KICK_CPU hypercall added to wakeup halted vcpu in paravirtual spinlock
enabled guest.
KVM_FEATURE_PV_UNHALT enables guest to check whether pv spinlock can be enabled
in guest.
Thanks Vatsa for rewriting KVM_HC_KICK_CPU
Signed-off-by: Srivatsa Vaddagiri <[email protected]>
Signed-off-by: Raghavendra K T <[email protected]>
Acked-by: Gleb Natapov <[email protected]>
Acked-by: Ingo Molnar <[email protected]>
---
Documentation/virtual/kvm/cpuid.txt | 4 ++++
Documentation/virtual/kvm/hypercalls.txt | 14 ++++++++++++++
2 files changed, 18 insertions(+)
diff --git a/Documentation/virtual/kvm/cpuid.txt b/Documentation/virtual/kvm/cpuid.txt
index 83afe65..22ff659 100644
--- a/Documentation/virtual/kvm/cpuid.txt
+++ b/Documentation/virtual/kvm/cpuid.txt
@@ -43,6 +43,10 @@ KVM_FEATURE_CLOCKSOURCE2 || 3 || kvmclock available at msrs
KVM_FEATURE_ASYNC_PF || 4 || async pf can be enabled by
|| || writing to msr 0x4b564d02
------------------------------------------------------------------------------
+KVM_FEATURE_PV_UNHALT || 7 || guest checks this feature bit
+ || || before enabling paravirtualized
+ || || spinlock support.
+------------------------------------------------------------------------------
KVM_FEATURE_CLOCKSOURCE_STABLE_BIT || 24 || host will warn if no guest-side
|| || per-cpu warps are expected in
|| || kvmclock.
diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virtual/kvm/hypercalls.txt
index ea113b5..022198e 100644
--- a/Documentation/virtual/kvm/hypercalls.txt
+++ b/Documentation/virtual/kvm/hypercalls.txt
@@ -64,3 +64,17 @@ Purpose: To enable communication between the hypervisor and guest there is a
shared page that contains parts of supervisor visible register state.
The guest can map this shared page to access its supervisor register through
memory using this hypercall.
+
+5. KVM_HC_KICK_CPU
+------------------------
+Architecture: x86
+Status: active
+Purpose: Hypercall used to wakeup a vcpu from HLT state
+Usage example : A vcpu of a paravirtualized guest that is busywaiting in guest
+kernel mode for an event to occur (ex: a spinlock to become available) can
+execute HLT instruction once it has busy-waited for more than a threshold
+time-interval. Execution of HLT instruction would cause the hypervisor to put
+the vcpu to sleep until occurence of an appropriate event. Another vcpu of the
+same guest can wakeup the sleeping vcpu by issuing KVM_HC_KICK_CPU hypercall,
+specifying APIC ID (a1) of the vcpu to be woken up. An additional argument (a0)
+is used in the hypercall for future use.
On Tue, Aug 6, 2013 at 11:23 AM, Raghavendra K T
<[email protected]> wrote:
> kvm : Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration
>
> From: Raghavendra K T <[email protected]>
>
> During migration, any vcpu that got kicked but did not become runnable
> (still in halted state) should be runnable after migration.
If this is about migration correctness, could it get folded into the
previous patch 2/5, so that there's not a broken commit which could
hurt bisection?
>
> Signed-off-by: Raghavendra K T <[email protected]>
> Acked-by: Gleb Natapov <[email protected]>
> Acked-by: Ingo Molnar <[email protected]>
> ---
> arch/x86/kvm/x86.c | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index dae4575..1e73dab 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -6284,7 +6284,12 @@ int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu,
> struct kvm_mp_state *mp_state)
> {
> kvm_apic_accept_events(vcpu);
> - mp_state->mp_state = vcpu->arch.mp_state;
> + if (vcpu->arch.mp_state == KVM_MP_STATE_HALTED &&
> + vcpu->arch.pv.pv_unhalted)
> + mp_state->mp_state = KVM_MP_STATE_RUNNABLE;
> + else
> + mp_state->mp_state = vcpu->arch.mp_state;
> +
> return 0;
> }
>
>
> _______________________________________________
> Virtualization mailing list
> [email protected]
> https://lists.linuxfoundation.org/mailman/listinfo/virtualization
On 08/07/2013 12:02 AM, Eric Northup wrote:
>
> If this is about migration correctness, could it get folded into the
> previous patch 2/5, so that there's not a broken commit which could
> hurt bisection?
Yes. It could be. Only reason I maintained like that was,
original author in the previous patch is different (Srivatsa) and I did
not want to merge this hunk when the patch series got evolved to mix
the sign-offs.
Gleb, Paolo please let me know.
On Wed, Aug 07, 2013 at 12:40:36AM +0530, Raghavendra K T wrote:
> On 08/07/2013 12:02 AM, Eric Northup wrote:
> >
> >If this is about migration correctness, could it get folded into the
> >previous patch 2/5, so that there's not a broken commit which could
> >hurt bisection?
>
> Yes. It could be. Only reason I maintained like that was,
> original author in the previous patch is different (Srivatsa) and I did
> not want to merge this hunk when the patch series got evolved to mix
> the sign-offs.
>
> Gleb, Paolo please let me know.
>
Yes please, do so.
--
Gleb.