Date: Fri, 3 Jul 2015 15:17:13 +0200
From: Peter Zijlstra <peterz@infradead.org>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: Pontus Fuchs <pontus.fuchs@gmail.com>,
        Linus Torvalds <torvalds@linux-foundation.org>, mingo@redhat.com,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        gleb@kernel.org
Subject: Re: [PATCH] sched,kvm: Fix KVM preempt_notifier usage
Message-ID: <20150703131712.GJ19282@twins.programming.kicks-ass.net>
References: <558BED42.1030000@gmail.com>
 <20150625120949.GZ3644@twins.programming.kicks-ass.net>
 <558BF0E4.50602@gmail.com>
 <20150625125514.GA3644@twins.programming.kicks-ass.net>
 <55966E0B.7060100@redhat.com>
 <20150703121907.GH19282@twins.programming.kicks-ass.net>
 <5596809D.4000905@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <5596809D.4000905@redhat.com>
User-Agent: Mutt/1.5.21 (2012-12-30)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2148
Lines: 58

On Fri, Jul 03, 2015 at 02:31:25PM +0200, Paolo Bonzini wrote:
> On 03/07/2015 14:19, Peter Zijlstra wrote:
> > On Fri, Jul 03, 2015 at 01:12:11PM +0200, Paolo Bonzini wrote:
> >> In fact you shouldn't have just tested the patch on a case _without_
> >> preemption notifiers, you should have also benchmarked the impact that
> >> static keys have _with_ preemption notifiers.  In a
> >> not-really-artificial case (one single-processor guest running on the
> >> host), the static key patch adds a static_key_slow_inc on a relatively
> >> hot path for KVM, which is not acceptable.
> > 
> > Spawning the first vcpu is a hot path?
> 
> This is not *spawning* the first VCPU.  Basically any critical section
> for vcpu->mutex includes a preempt_notifier_register/unregister pair:
> 
> /*
>  * Switches to specified vcpu, until a matching vcpu_put()
>  */
> int vcpu_load(struct kvm_vcpu *vcpu)
> {
>         int cpu;
> 
>         if (mutex_lock_killable(&vcpu->mutex))
>                 return -EINTR;
>         cpu = get_cpu();
>         preempt_notifier_register(&vcpu->preempt_notifier);
>         kvm_arch_vcpu_load(vcpu, cpu);
>         put_cpu();
>         return 0;
> }
> 
> void vcpu_put(struct kvm_vcpu *vcpu)
> {
>         preempt_disable();
>         kvm_arch_vcpu_put(vcpu);
>         preempt_notifier_unregister(&vcpu->preempt_notifier);
>         preempt_enable();
>         mutex_unlock(&vcpu->mutex);
> }
> 
> So basically you're adding at least one static_key_slow_inc/dec pair to
> every userspace exit.

Ugh, ok that is not what I was expecting to happen. I'll ask Ingo to
queue a revert until we can fix this better.

I thought these were vcpu create/destroy functions.

That said, the slow_inc/dec are really only slow on the 0<->!0
transitions.

But, could we rework the code so that you register the preempt notifier
when creating the vcpu thread and leave it installed forevermore?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/