2017-12-01 08:15:18

by Wanpeng Li

Subject: [PATCH] KVM: X86: Reduce the overhead when lapic_timer_advance is disabled

When I run ebizzy in a 32 vCPUs guest on a 32 pCPUs Xeon box, I can observe
~8000 kvm_wait_lapic_expire CurAvg/s through kvm_stat tool even if the advance
tscdeadline hrtimer expiration is disabled. Each call to wait_lapic_expire()
will consume ~70 cycles when a timer fires since apic_timer_expire() will
set expired_tscdeadline and then wait_lapic_expire() will do some caculation
before bailing out. So total ~175us per second is lost on this 3.2Ghz machine.
This patch reduces the overhead by skipping the function wait_lapic_expire()
when lapic_timer_advance is disabled.

Cc: Paolo Bonzini <[email protected]>
Cc: Radim Krčmář <[email protected]>
Signed-off-by: Wanpeng Li <[email protected]>
---
arch/x86/kvm/x86.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 50c8361..d1abf13 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6994,7 +6994,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
}

trace_kvm_entry(vcpu->vcpu_id);
- wait_lapic_expire(vcpu);
+ if (lapic_timer_advance_ns)
+ wait_lapic_expire(vcpu);
guest_enter_irqoff();

if (unlikely(vcpu->arch.switch_db_regs)) {
--
2.7.4

2017-12-01 08:36:02

by Paolo Bonzini

Subject: Re: [PATCH] KVM: X86: Reduce the overhead when lapic_timer_advance is disabled

On 01/12/2017 09:15, Wanpeng Li wrote:
> When I run ebizzy in a 32 vCPUs guest on a 32 pCPUs Xeon box, I can observe
> ~8000 kvm_wait_lapic_expire CurAvg/s through kvm_stat tool even if the advance
> tscdeadline hrtimer expiration is disabled. Each call to wait_lapic_expire()
> will consume ~70 cycles when a timer fires since apic_timer_expire() will
> set expired_tscdeadline and then wait_lapic_expire() will do some caculation
> before bailing out. So total ~175us per second is lost on this 3.2Ghz machine.
> This patch reduces the overhead by skipping the function wait_lapic_expire()
> when lapic_timer_advance is disabled.
>
> Cc: Paolo Bonzini <[email protected]>
> Cc: Radim Krčmář <[email protected]>
> Signed-off-by: Wanpeng Li <[email protected]>
> ---
> arch/x86/kvm/x86.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)

Reviewed-by: Paolo Bonzini <[email protected]>

Paolo

> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 50c8361..d1abf13 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -6994,7 +6994,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
> }
>
> trace_kvm_entry(vcpu->vcpu_id);
> - wait_lapic_expire(vcpu);
> + if (lapic_timer_advance_ns)
> + wait_lapic_expire(vcpu);
> guest_enter_irqoff();
>
> if (unlikely(vcpu->arch.switch_db_regs)) {
>

2017-12-12 14:00:49

by Paolo Bonzini

Subject: Re: [PATCH] KVM: X86: Reduce the overhead when lapic_timer_advance is disabled

On 01/12/2017 09:15, Wanpeng Li wrote:
> When I run ebizzy in a 32 vCPUs guest on a 32 pCPUs Xeon box, I can observe
> ~8000 kvm_wait_lapic_expire CurAvg/s through kvm_stat tool even if the advance
> tscdeadline hrtimer expiration is disabled. Each call to wait_lapic_expire()
> will consume ~70 cycles when a timer fires since apic_timer_expire() will
> set expired_tscdeadline and then wait_lapic_expire() will do some caculation
> before bailing out. So total ~175us per second is lost on this 3.2Ghz machine.
> This patch reduces the overhead by skipping the function wait_lapic_expire()
> when lapic_timer_advance is disabled.
>
> Cc: Paolo Bonzini <[email protected]>
> Cc: Radim Krčmář <[email protected]>
> Signed-off-by: Wanpeng Li <[email protected]>
> ---
> arch/x86/kvm/x86.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 50c8361..d1abf13 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -6994,7 +6994,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
> }
>
> trace_kvm_entry(vcpu->vcpu_id);
> - wait_lapic_expire(vcpu);
> + if (lapic_timer_advance_ns)
> + wait_lapic_expire(vcpu);
> guest_enter_irqoff();
>
> if (unlikely(vcpu->arch.switch_db_regs)) {
>

Queued, thanks.

Paolo