This series forms the kvm host part of paravirtual spinlock
based against kvm tree.
Please refer to https://lkml.org/lkml/2013/8/9/265 for
kvm guest and Xen, x86 part merged to -tip spinlocks.
Please note that:
kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi is a common patch
for both guest and host.
Changes since V12:
fold the patch 3 into patch 2 for bisection. (Eric Northup)
Raghavendra K T (3):
kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi
kvm hypervisor: Simplify kvm_for_each_vcpu with
kvm_irq_delivery_to_apic
Documentation/kvm : Add documentation on Hypercalls and features used
for PV spinlock
Srivatsa Vaddagiri (1):
kvm hypervisor : Add a hypercall to KVM hypervisor to support
pv-ticketlocks
Documentation/virtual/kvm/cpuid.txt | 4 ++++
Documentation/virtual/kvm/hypercalls.txt | 14 ++++++++++++++
arch/x86/include/asm/kvm_host.h | 5 +++++
arch/x86/include/uapi/asm/kvm_para.h | 1 +
arch/x86/kvm/cpuid.c | 3 ++-
arch/x86/kvm/lapic.c | 5 ++++-
arch/x86/kvm/x86.c | 31 ++++++++++++++++++++++++++++++-
include/uapi/linux/kvm_para.h | 1 +
8 files changed, 61 insertions(+), 3 deletions(-)
--
1.7.11.7
this is needed by both guest and host.
Originally-from: Srivatsa Vaddagiri <[email protected]>
Signed-off-by: Raghavendra K T <[email protected]>
Acked-by: Gleb Natapov <[email protected]>
Acked-by: Ingo Molnar <[email protected]>
---
arch/x86/include/uapi/asm/kvm_para.h | 1 +
include/uapi/linux/kvm_para.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h
index 06fdbd9..94dc8ca 100644
--- a/arch/x86/include/uapi/asm/kvm_para.h
+++ b/arch/x86/include/uapi/asm/kvm_para.h
@@ -23,6 +23,7 @@
#define KVM_FEATURE_ASYNC_PF 4
#define KVM_FEATURE_STEAL_TIME 5
#define KVM_FEATURE_PV_EOI 6
+#define KVM_FEATURE_PV_UNHALT 7
/* The last 8 bits are used to indicate how to interpret the flags field
* in pvclock structure. If no bits are set, all flags are ignored.
diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h
index cea2c5c..2841f86 100644
--- a/include/uapi/linux/kvm_para.h
+++ b/include/uapi/linux/kvm_para.h
@@ -19,6 +19,7 @@
#define KVM_HC_MMU_OP 2
#define KVM_HC_FEATURES 3
#define KVM_HC_PPC_MAP_MAGIC_PAGE 4
+#define KVM_HC_KICK_CPU 5
/*
* hypercalls use architecture specific
--
1.7.11.7
Note that we are using APIC_DM_REMRD which has reserved usage.
In future if APIC_DM_REMRD usage is standardized, then we should
find some other way or go back to old method.
Suggested-by: Gleb Natapov <[email protected]>
Signed-off-by: Raghavendra K T <[email protected]>
Acked-by: Gleb Natapov <[email protected]>
Acked-by: Ingo Molnar <[email protected]>
---
arch/x86/kvm/lapic.c | 5 ++++-
arch/x86/kvm/x86.c | 25 ++++++-------------------
2 files changed, 10 insertions(+), 20 deletions(-)
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index afc1124..48c13c9 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -706,7 +706,10 @@ out:
break;
case APIC_DM_REMRD:
- apic_debug("Ignoring delivery mode 3\n");
+ result = 1;
+ vcpu->arch.pv.pv_unhalted = 1;
+ kvm_make_request(KVM_REQ_EVENT, vcpu);
+ kvm_vcpu_kick(vcpu);
break;
case APIC_DM_SMI:
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 1e73dab..640d112 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5502,27 +5502,14 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
*/
static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid)
{
- struct kvm_vcpu *vcpu = NULL;
- int i;
+ struct kvm_lapic_irq lapic_irq;
- kvm_for_each_vcpu(i, vcpu, kvm) {
- if (!kvm_apic_present(vcpu))
- continue;
+ lapic_irq.shorthand = 0;
+ lapic_irq.dest_mode = 0;
+ lapic_irq.dest_id = apicid;
- if (kvm_apic_match_dest(vcpu, 0, 0, apicid, 0))
- break;
- }
- if (vcpu) {
- /*
- * Setting unhalt flag here can result in spurious runnable
- * state when unhalt reset does not happen in vcpu_block.
- * But that is harmless since that should soon result in halt.
- */
- vcpu->arch.pv.pv_unhalted = true;
- /* We need everybody see unhalt before vcpu unblocks */
- smp_wmb();
- kvm_vcpu_kick(vcpu);
- }
+ lapic_irq.delivery_mode = APIC_DM_REMRD;
+ kvm_irq_delivery_to_apic(kvm, 0, &lapic_irq, NULL);
}
int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
--
1.7.11.7
KVM_HC_KICK_CPU hypercall added to wakeup halted vcpu in paravirtual spinlock
enabled guest.
KVM_FEATURE_PV_UNHALT enables guest to check whether pv spinlock can be enabled
in guest.
Thanks Vatsa for rewriting KVM_HC_KICK_CPU
Cc: Rob Landley <[email protected]>
Signed-off-by: Srivatsa Vaddagiri <[email protected]>
Signed-off-by: Raghavendra K T <[email protected]>
Acked-by: Gleb Natapov <[email protected]>
Acked-by: Ingo Molnar <[email protected]>
---
Documentation/virtual/kvm/cpuid.txt | 4 ++++
Documentation/virtual/kvm/hypercalls.txt | 14 ++++++++++++++
2 files changed, 18 insertions(+)
diff --git a/Documentation/virtual/kvm/cpuid.txt b/Documentation/virtual/kvm/cpuid.txt
index 83afe65..22ff659 100644
--- a/Documentation/virtual/kvm/cpuid.txt
+++ b/Documentation/virtual/kvm/cpuid.txt
@@ -43,6 +43,10 @@ KVM_FEATURE_CLOCKSOURCE2 || 3 || kvmclock available at msrs
KVM_FEATURE_ASYNC_PF || 4 || async pf can be enabled by
|| || writing to msr 0x4b564d02
------------------------------------------------------------------------------
+KVM_FEATURE_PV_UNHALT || 7 || guest checks this feature bit
+ || || before enabling paravirtualized
+ || || spinlock support.
+------------------------------------------------------------------------------
KVM_FEATURE_CLOCKSOURCE_STABLE_BIT || 24 || host will warn if no guest-side
|| || per-cpu warps are expected in
|| || kvmclock.
diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virtual/kvm/hypercalls.txt
index ea113b5..022198e 100644
--- a/Documentation/virtual/kvm/hypercalls.txt
+++ b/Documentation/virtual/kvm/hypercalls.txt
@@ -64,3 +64,17 @@ Purpose: To enable communication between the hypervisor and guest there is a
shared page that contains parts of supervisor visible register state.
The guest can map this shared page to access its supervisor register through
memory using this hypercall.
+
+5. KVM_HC_KICK_CPU
+------------------------
+Architecture: x86
+Status: active
+Purpose: Hypercall used to wakeup a vcpu from HLT state
+Usage example : A vcpu of a paravirtualized guest that is busywaiting in guest
+kernel mode for an event to occur (ex: a spinlock to become available) can
+execute HLT instruction once it has busy-waited for more than a threshold
+time-interval. Execution of HLT instruction would cause the hypervisor to put
+the vcpu to sleep until occurence of an appropriate event. Another vcpu of the
+same guest can wakeup the sleeping vcpu by issuing KVM_HC_KICK_CPU hypercall,
+specifying APIC ID (a1) of the vcpu to be woken up. An additional argument (a0)
+is used in the hypercall for future use.
--
1.7.11.7
From: Srivatsa Vaddagiri <[email protected]>
kvm_hc_kick_cpu allows the calling vcpu to kick another vcpu out of halt state.
the presence of these hypercalls is indicated to guest via
kvm_feature_pv_unhalt.
Fold pv_unhalt flag into GET_MP_STATE ioctl to aid migration
During migration, any vcpu that got kicked but did not become runnable
(still in halted state) should be runnable after migration.
Signed-off-by: Srivatsa Vaddagiri <[email protected]>
Signed-off-by: Suzuki Poulose <[email protected]>
[Raghu: Apic related changes, folding pvunhalted into vcpu_runnable
Added flags for future use (suggested by Gleb)]
[ Raghu: fold pv_unhalt flag as suggested by Eric Northup]
Signed-off-by: Raghavendra K T <[email protected]>
Acked-by: Gleb Natapov <[email protected]>
Acked-by: Ingo Molnar <[email protected]>
---
arch/x86/include/asm/kvm_host.h | 5 +++++
arch/x86/kvm/cpuid.c | 3 ++-
arch/x86/kvm/x86.c | 44 ++++++++++++++++++++++++++++++++++++++++-
3 files changed, 50 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index f87f7fc..1d1f711 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -511,6 +511,11 @@ struct kvm_vcpu_arch {
* instruction.
*/
bool write_fault_to_shadow_pgtable;
+
+ /* pv related host specific info */
+ struct {
+ bool pv_unhalted;
+ } pv;
};
struct kvm_lpage_info {
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index a20ecb5..b110fe6 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -413,7 +413,8 @@ static int do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
(1 << KVM_FEATURE_CLOCKSOURCE2) |
(1 << KVM_FEATURE_ASYNC_PF) |
(1 << KVM_FEATURE_PV_EOI) |
- (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT);
+ (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT) |
+ (1 << KVM_FEATURE_PV_UNHALT);
if (sched_info_on())
entry->eax |= (1 << KVM_FEATURE_STEAL_TIME);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index d21bce5..1e73dab 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5495,6 +5495,36 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
return 1;
}
+/*
+ * kvm_pv_kick_cpu_op: Kick a vcpu.
+ *
+ * @apicid - apicid of vcpu to be kicked.
+ */
+static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid)
+{
+ struct kvm_vcpu *vcpu = NULL;
+ int i;
+
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (!kvm_apic_present(vcpu))
+ continue;
+
+ if (kvm_apic_match_dest(vcpu, 0, 0, apicid, 0))
+ break;
+ }
+ if (vcpu) {
+ /*
+ * Setting unhalt flag here can result in spurious runnable
+ * state when unhalt reset does not happen in vcpu_block.
+ * But that is harmless since that should soon result in halt.
+ */
+ vcpu->arch.pv.pv_unhalted = true;
+ /* We need everybody see unhalt before vcpu unblocks */
+ smp_wmb();
+ kvm_vcpu_kick(vcpu);
+ }
+}
+
int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
{
unsigned long nr, a0, a1, a2, a3, ret;
@@ -5528,6 +5558,10 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
case KVM_HC_VAPIC_POLL_IRQ:
ret = 0;
break;
+ case KVM_HC_KICK_CPU:
+ kvm_pv_kick_cpu_op(vcpu->kvm, a0, a1);
+ ret = 0;
+ break;
default:
ret = -KVM_ENOSYS;
break;
@@ -5950,6 +5984,7 @@ static int __vcpu_run(struct kvm_vcpu *vcpu)
kvm_apic_accept_events(vcpu);
switch(vcpu->arch.mp_state) {
case KVM_MP_STATE_HALTED:
+ vcpu->arch.pv.pv_unhalted = false;
vcpu->arch.mp_state =
KVM_MP_STATE_RUNNABLE;
case KVM_MP_STATE_RUNNABLE:
@@ -6249,7 +6284,12 @@ int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu,
struct kvm_mp_state *mp_state)
{
kvm_apic_accept_events(vcpu);
- mp_state->mp_state = vcpu->arch.mp_state;
+ if (vcpu->arch.mp_state == KVM_MP_STATE_HALTED &&
+ vcpu->arch.pv.pv_unhalted)
+ mp_state->mp_state = KVM_MP_STATE_RUNNABLE;
+ else
+ mp_state->mp_state = vcpu->arch.mp_state;
+
return 0;
}
@@ -6770,6 +6810,7 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
BUG_ON(vcpu->kvm == NULL);
kvm = vcpu->kvm;
+ vcpu->arch.pv.pv_unhalted = false;
vcpu->arch.emulate_ctxt.ops = &emulate_ops;
if (!irqchip_in_kernel(kvm) || kvm_vcpu_is_bsp(vcpu))
vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE;
@@ -7103,6 +7144,7 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
!vcpu->arch.apf.halted)
|| !list_empty_careful(&vcpu->async_pf.done)
|| kvm_apic_has_events(vcpu)
+ || vcpu->arch.pv.pv_unhalted
|| atomic_read(&vcpu->arch.nmi_queued) ||
(kvm_arch_interrupt_allowed(vcpu) &&
kvm_cpu_has_interrupt(vcpu));
--
1.7.11.7
On Mon, Aug 26, 2013 at 02:18:32PM +0530, Raghavendra K T wrote:
>
> This series forms the kvm host part of paravirtual spinlock
> based against kvm tree.
>
> Please refer to https://lkml.org/lkml/2013/8/9/265 for
> kvm guest and Xen, x86 part merged to -tip spinlocks.
>
> Please note that:
> kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi is a common patch
> for both guest and host.
>
Thanks, applied. The patchset is not against kvm.git queue though, so I
had to fix one minor conflict manually.
> Changes since V12:
> fold the patch 3 into patch 2 for bisection. (Eric Northup)
>
> Raghavendra K T (3):
> kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi
> kvm hypervisor: Simplify kvm_for_each_vcpu with
> kvm_irq_delivery_to_apic
> Documentation/kvm : Add documentation on Hypercalls and features used
> for PV spinlock
>
> Srivatsa Vaddagiri (1):
> kvm hypervisor : Add a hypercall to KVM hypervisor to support
> pv-ticketlocks
>
> Documentation/virtual/kvm/cpuid.txt | 4 ++++
> Documentation/virtual/kvm/hypercalls.txt | 14 ++++++++++++++
> arch/x86/include/asm/kvm_host.h | 5 +++++
> arch/x86/include/uapi/asm/kvm_para.h | 1 +
> arch/x86/kvm/cpuid.c | 3 ++-
> arch/x86/kvm/lapic.c | 5 ++++-
> arch/x86/kvm/x86.c | 31 ++++++++++++++++++++++++++++++-
> include/uapi/linux/kvm_para.h | 1 +
> 8 files changed, 61 insertions(+), 3 deletions(-)
>
> --
> 1.7.11.7
--
Gleb.
On 08/26/2013 03:34 PM, Gleb Natapov wrote:
> On Mon, Aug 26, 2013 at 02:18:32PM +0530, Raghavendra K T wrote:
>>
>> This series forms the kvm host part of paravirtual spinlock
>> based against kvm tree.
>>
>> Please refer to https://lkml.org/lkml/2013/8/9/265 for
>> kvm guest and Xen, x86 part merged to -tip spinlocks.
>>
>> Please note that:
>> kvm uapi: Add KICK_CPU and PV_UNHALT definition to uapi is a common patch
>> for both guest and host.
>>
> Thanks, applied. The patchset is not against kvm.git queue though, so I
> had to fix one minor conflict manually.
Thank you Gleb.
On 08/26/2013 03:48:36 AM, Raghavendra K T wrote:
> KVM_HC_KICK_CPU hypercall added to wakeup halted vcpu in paravirtual
> spinlock
> enabled guest.
>
> KVM_FEATURE_PV_UNHALT enables guest to check whether pv spinlock can
> be enabled
> in guest.
>
> Thanks Vatsa for rewriting KVM_HC_KICK_CPU
> Cc: Rob Landley <[email protected]>
Looks like documentation to me:
Acked-by: Rob Landley <[email protected]>
Rob-