Guests on new hypersiors might set KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT
bit when enabling async_PF, but this bit is reserved on old hypervisors,
which results in a failure upon migration.
Guests at least expect that KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT might not
be present when booting, so we allow userspace to handle migration
compatibility by adding a KVM CPUID flag that determines the presence of
KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT.
Fixes: 52a5c155cf79 ("KVM: async_pf: Let guest support delivery of async_pf from guest mode")
Cc: <[email protected]>
Signed-off-by: Radim Krčmář <[email protected]>
---
arch/x86/include/uapi/asm/kvm_para.h | 1 +
arch/x86/kernel/kvm.c | 8 ++++----
arch/x86/kvm/cpuid.c | 3 ++-
arch/x86/kvm/cpuid.h | 11 +++++++++++
arch/x86/kvm/x86.c | 6 ++++--
5 files changed, 22 insertions(+), 7 deletions(-)
diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h
index 7a2ade4aa235..6cfa9c8cb7d6 100644
--- a/arch/x86/include/uapi/asm/kvm_para.h
+++ b/arch/x86/include/uapi/asm/kvm_para.h
@@ -26,6 +26,7 @@
#define KVM_FEATURE_PV_EOI 6
#define KVM_FEATURE_PV_UNHALT 7
#define KVM_FEATURE_PV_TLB_FLUSH 9
+#define KVM_FEATURE_ASYNC_PF_VMEXIT 10
/* The last 8 bits are used to indicate how to interpret the flags field
* in pvclock structure. If no bits are set, all flags are ignored.
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index 4e37d1a851a6..971babe964d2 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -341,10 +341,10 @@ static void kvm_guest_cpu_init(void)
#endif
pa |= KVM_ASYNC_PF_ENABLED;
- /* Async page fault support for L1 hypervisor is optional */
- if (wrmsr_safe(MSR_KVM_ASYNC_PF_EN,
- (pa | KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT) & 0xffffffff, pa >> 32) < 0)
- wrmsrl(MSR_KVM_ASYNC_PF_EN, pa);
+ if (kvm_para_has_feature(KVM_FEATURE_ASYNC_PF_VMEXIT))
+ pa |= KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT;
+
+ wrmsrl(MSR_KVM_ASYNC_PF_EN, pa);
__this_cpu_write(apf_reason.enabled, 1);
printk(KERN_INFO"KVM setup async PF for cpu %d\n",
smp_processor_id());
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 20e491b94f44..7fc04a176c57 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -604,7 +604,8 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
(1 << KVM_FEATURE_PV_EOI) |
(1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT) |
(1 << KVM_FEATURE_PV_UNHALT) |
- (1 << KVM_FEATURE_PV_TLB_FLUSH);
+ (1 << KVM_FEATURE_PV_TLB_FLUSH) |
+ (1 << KVM_FEATURE_ASYNC_PF_VMEXIT);
if (sched_info_on())
entry->eax |= (1 << KVM_FEATURE_STEAL_TIME);
diff --git a/arch/x86/kvm/cpuid.h b/arch/x86/kvm/cpuid.h
index c2cea6651279..f20731dfe28e 100644
--- a/arch/x86/kvm/cpuid.h
+++ b/arch/x86/kvm/cpuid.h
@@ -105,6 +105,17 @@ static __always_inline bool guest_cpuid_has(struct kvm_vcpu *vcpu, unsigned x86_
return *reg & bit(x86_feature);
}
+static inline bool guest_kvm_cpuid_has(struct kvm_vcpu *vcpu, unsigned kvm_feature)
+{
+ struct kvm_cpuid_entry2 *entry;
+
+ entry = kvm_find_cpuid_entry(vcpu, KVM_CPUID_FEATURES, 0);
+ if (!entry)
+ return false;
+
+ return entry->eax & bit(kvm_feature);
+}
+
static __always_inline void guest_cpuid_clear(struct kvm_vcpu *vcpu, unsigned x86_feature)
{
int *reg;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 4c3103f449a3..c16740a06f0c 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2139,8 +2139,10 @@ static int kvm_pv_enable_async_pf(struct kvm_vcpu *vcpu, u64 data)
{
gpa_t gpa = data & ~0x3f;
- /* Bits 3:5 are reserved, Should be zero */
- if (data & 0x38)
+ /* Bits 3:5 are reserved, Should be zero. */
+ if (data & 0x38 ||
+ (data & KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT &&
+ !guest_kvm_cpuid_has(vcpu, KVM_FEATURE_ASYNC_PF_VMEXIT)))
return 1;
vcpu->arch.apf.msr_val = data;
--
2.15.1
On 01/02/2018 12:50, Radim Krčmář wrote:
> Guests on new hypersiors might set KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT
> bit when enabling async_PF, but this bit is reserved on old hypervisors,
> which results in a failure upon migration.
>
> Guests at least expect that KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT might not
> be present when booting, so we allow userspace to handle migration
> compatibility by adding a KVM CPUID flag that determines the presence of
> KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT.
>
> Fixes: 52a5c155cf79 ("KVM: async_pf: Let guest support delivery of async_pf from guest mode")
> Cc: <[email protected]>
> Signed-off-by: Radim Krčmář <[email protected]>
This has to be documented in Documentation/virtual/kvm/cpuid.txt.
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 4c3103f449a3..c16740a06f0c 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -2139,8 +2139,10 @@ static int kvm_pv_enable_async_pf(struct kvm_vcpu *vcpu, u64 data)
> {
> gpa_t gpa = data & ~0x3f;
>
> - /* Bits 3:5 are reserved, Should be zero */
> - if (data & 0x38)
> + /* Bits 3:5 are reserved, Should be zero. */
> + if (data & 0x38 ||
> + (data & KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT &&
> + !guest_kvm_cpuid_has(vcpu, KVM_FEATURE_ASYNC_PF_VMEXIT)))
> return 1;
>
> vcpu->arch.apf.msr_val = data;
>
This check will break migration if the source guest and host both have
the recent kernels which support KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT, so
I am not sure about it. Otherwise, the patch is okay!
Thanks,
Paolo
2018-02-01 13:09-0500, Paolo Bonzini:
> On 01/02/2018 12:50, Radim Krčmář wrote:
> > Guests on new hypersiors might set KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT
> > bit when enabling async_PF, but this bit is reserved on old hypervisors,
> > which results in a failure upon migration.
> >
> > Guests at least expect that KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT might not
> > be present when booting, so we allow userspace to handle migration
> > compatibility by adding a KVM CPUID flag that determines the presence of
> > KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT.
> >
> > Fixes: 52a5c155cf79 ("KVM: async_pf: Let guest support delivery of async_pf from guest mode")
> > Cc: <[email protected]>
> > Signed-off-by: Radim Krčmář <[email protected]>
>
> This has to be documented in Documentation/virtual/kvm/cpuid.txt.
Will add, also to the MSR if we agree on v2.
> > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> > index 4c3103f449a3..c16740a06f0c 100644
> > --- a/arch/x86/kvm/x86.c
> > +++ b/arch/x86/kvm/x86.c
> > @@ -2139,8 +2139,10 @@ static int kvm_pv_enable_async_pf(struct kvm_vcpu *vcpu, u64 data)
> > {
> > gpa_t gpa = data & ~0x3f;
> >
> > - /* Bits 3:5 are reserved, Should be zero */
> > - if (data & 0x38)
> > + /* Bits 3:5 are reserved, Should be zero. */
> > + if (data & 0x38 ||
> > + (data & KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT &&
> > + !guest_kvm_cpuid_has(vcpu, KVM_FEATURE_ASYNC_PF_VMEXIT)))
> > return 1;
> >
> > vcpu->arch.apf.msr_val = data;
> >
>
> This check will break migration if the source guest and host both have
> the recent kernels which support KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT, so
> I am not sure about it. Otherwise, the patch is okay!
Good point, breaking forward migration is worse than doing nothing.
A compromise solution would be to drop the feature check from the
hypervisor. Newer guests would work everywhere and there would be no
change to old systems, so v4.13-v4.15 guests could at least upgrade.
Slightly better than doing nothing, IMO,
thanks.
On 01/02/2018 14:10, Radim Krčmář wrote:
>>>
>> This check will break migration if the source guest and host both have
>> the recent kernels which support KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT, so
>> I am not sure about it. Otherwise, the patch is okay!
> Good point, breaking forward migration is worse than doing nothing.
>
> A compromise solution would be to drop the feature check from the
> hypervisor. Newer guests would work everywhere and there would be no
> change to old systems, so v4.13-v4.15 guests could at least upgrade.
>
> Slightly better than doing nothing, IMO,
Yes, I agree. Just removing the guest_kvm_cpuid_has should be fine.
Paolo