by Jeremy Fitzhardinge

[permalink] [raw]

Subject: Re: [Xen-devel] Re: [PATCH 09/14] xen/pvticketlock: Xen implementation for PV ticket locks

On 11/17/2010 02:34 AM, Jan Beulich wrote:
>> Actually, on second thoughts, maybe it doesn't matter so much. The main
>> issue is making sure that the interrupt will make the VCPU drop out of
>> xen_poll_irq() - if it happens before xen_poll_irq(), it should leave
>> the event pending, which will cause the poll to return immediately. I
>> hope. Certainly disabling interrupts for some of the function will make
>> it easier to analyze with respect to interrupt nesting.
> That's not my main concern. Instead, what if you get interrupted
> anywhere here, the interrupt handler tries to acquire another
> spinlock and also has to go into the slow path? It'll overwrite part
> or all of the outer context's state.

That doesn't matter if the outer context doesn't end up blocking. If it
has already blocked then it will unblock as a result of the interrupt;
if it hasn't yet blocked, then the inner context will leave the event
pending and cause it to not block. Either way, it no longer uses or
needs that per-cpu state: it will return to the spin loop and (maybe)
get re-entered, setting it all up again.

I think there is a problem with the code as posted because it sets up
the percpu data before clearing the pending event, so it can end up
blocking with bad percpu data.

>> Another issue may be making sure the writes and reads of "w->want" and
>> "w->lock" are ordered properly to make sure that xen_unlock_kick() never
>> sees an inconsistent view of the (lock,want) tuple. The risk being that
>> xen_unlock_kick() sees a random, spurious (lock,want) pairing and sends
>> the kick event to the wrong VCPU, leaving the deserving one hung.
> Yes, proper operation sequence (and barriers) is certainly
> required here. If you allowed nesting, this may even become
> simpler (as you'd have a single write making visible the new
> "head" pointer, after having written all relevant fields of the
> new "head" structure).

Yes, simple nesting should be quite straightforward (ie allowing an
interrupt handler to take some other lock than the one the outer context
is waiting on).

J

2011-02-18 19:02:45

by Srivatsa Vaddagiri

[permalink] [raw]

Subject: Re: [PATCH 13/14] x86/ticketlock: add slowpath logic

> On Mon, Jan 24, 2011 at 01:56:53PM -0800, Jeremy Fitzhardinge wrote:

For some reason, I seem to be missing emails from your id/domain and hence had
missed this completely!

> > * bits. However, we need to be careful about this because someone
> > * may just be entering as we leave, and enter the slowpath.
> > */
> > -void __ticket_unlock_release_slowpath(struct arch_spinlock *lock)
> > +void __ticket_unlock_slowpath(struct arch_spinlock *lock)
> > {
> > struct arch_spinlock old, new;
> >
> > BUILD_BUG_ON(((__ticket_t)NR_CPUS) != NR_CPUS);
> >
> > old = ACCESS_ONCE(*lock);
> > -
> > new = old;
> > - new.tickets.head += TICKET_LOCK_INC;
> >
> > /* Clear the slowpath flag */
> > new.tickets.tail &= ~TICKET_SLOWPATH_FLAG;
> > + if (new.tickets.head == new.tickets.tail)
> > + cmpxchg(&lock->head_tail, old.head_tail, new.head_tail);
> >
> > - /*
> > - * If there's currently people waiting or someone snuck in
> > - * since we read the lock above, then do a normal unlock and
> > - * kick. If we managed to unlock with no queued waiters, then
> > - * we can clear the slowpath flag.
> > - */
> > - if (new.tickets.head != new.tickets.tail ||
> > - cmpxchg(&lock->head_tail,
> > - old.head_tail, new.head_tail) != old.head_tail) {
> > - /* still people waiting */
> > - __ticket_unlock_release(lock);
> > - }
> > -
> > + /* Wake up an appropriate waiter */
> > __ticket_unlock_kick(lock, new.tickets.head);
>
> Does the __ticket_unlock_kick need to be unconditional?

I recall having tried optimizing it to be conditional, something along these
lines:

if (new.ticket.head == new.tickets.tail) {
cmpxchg();
} else {
__ticket_unlock_kick(lock, new.tickets.head);
}

but it didn't work for some reason. I left the call unconditional as was the
case previously based on that experiment.

- vatsa