On 04/15/2010 07:58 AM, Srivatsa Vaddagiri wrote:
> On Sun, Apr 11, 2010 at 11:40 PM, Avi Kivity <[email protected]
> <mailto:[email protected]>> wrote:
>
> The current handing of PLE is very suboptimal. With proper
> directed yield we should be much better there.
>
>
>
> Hi Avi,
> By directed yield, do you mean transfer the timeslice of
> one thread (which is contending for a lock) to another thread (which
> is holding a lock)?
It's a priority transfer (in CFS terms, vruntime) (we don't know who
holds the lock, so we pick a co-vcpu at random).
> If at that point in time, the lock-holder thread/VCPU is actually not
> running currently, ie it is at the back of the runqueue, would it help
> much? In such case, it will take time for the lock holder to run again
> and the default timeslice it would have got could have been sufficient
> to release the lock?
The idea is to increase the chances to the target vcpu to run, and to
decrease the changes of the spinner to run (hopefully they change places).
>
> I am also working on a prototype for some other technique here - to
> avoid preempting guest threads/VCPUs in the middle of their
> (spin-lock) critical section. This requires guest to hint host when
> there are in such a section. [1] has shown 33% improvement to an
> apache benchmark based on this idea.
>
Certainly that has even greater potential for Linux guests. Note that
we spin on mutexes now, so we need to prevent preemption while the lock
owner is running.
--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
On Thu, 2010-04-15 at 11:18 +0300, Avi Kivity wrote:
>
> Certainly that has even greater potential for Linux guests. Note that
> we spin on mutexes now, so we need to prevent preemption while the lock
> owner is running.
either that, or disable spinning on (para) virt kernels. Para virt
kernels could possibly extend the thing by also checking to see if the
owner's vcpu is running.
On Thu, Apr 15, 2010 at 03:33:18PM +0200, Peter Zijlstra wrote:
> On Thu, 2010-04-15 at 11:18 +0300, Avi Kivity wrote:
> >
> > Certainly that has even greater potential for Linux guests. Note that
> > we spin on mutexes now, so we need to prevent preemption while the lock
> > owner is running.
>
> either that, or disable spinning on (para) virt kernels. Para virt
> kernels could possibly extend the thing by also checking to see if the
> owner's vcpu is running.
I suspect we will need a combination of both approaches, given that we will not
be able to avoid preempting guests in their critical section always (too long
critical sections or real-time tasks wanting to preempt). Other idea is to
gang-schedule VCPUs of the same guest as much as possible?
- vatsa
On Thu, 2010-04-15 at 09:43 -0700, Srivatsa Vaddagiri wrote:
> On Thu, Apr 15, 2010 at 03:33:18PM +0200, Peter Zijlstra wrote:
> > On Thu, 2010-04-15 at 11:18 +0300, Avi Kivity wrote:
> > >
> > > Certainly that has even greater potential for Linux guests. Note that
> > > we spin on mutexes now, so we need to prevent preemption while the lock
> > > owner is running.
> >
> > either that, or disable spinning on (para) virt kernels. Para virt
> > kernels could possibly extend the thing by also checking to see if the
> > owner's vcpu is running.
>
> I suspect we will need a combination of both approaches, given that we will not
> be able to avoid preempting guests in their critical section always (too long
> critical sections or real-time tasks wanting to preempt). Other idea is to
> gang-schedule VCPUs of the same guest as much as possible?
Except gang scheduling is a scalability nightmare waiting to happen. I
much prefer this hint thing.
On 04/15/2010 04:33 PM, Peter Zijlstra wrote:
> On Thu, 2010-04-15 at 11:18 +0300, Avi Kivity wrote:
>
>> Certainly that has even greater potential for Linux guests. Note that
>> we spin on mutexes now, so we need to prevent preemption while the lock
>> owner is running.
>>
> either that, or disable spinning on (para) virt kernels.
What would you do instead?
Note we can't disable spinning on Windows or pre 2.6.36 kernels.
> Para virt
> kernels could possibly extend the thing by also checking to see if the
> owner's vcpu is running.
>
Certainly that's worth doing.
--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.