2006-09-07 08:12:19

by Atsushi Nemoto

[permalink] [raw]
Subject: Re: NPTL mutex and the scheduling priority

On Tue, 13 Jun 2006 01:06:28 +0900 (JST), Atsushi Nemoto <[email protected]> wrote:
> > > Really FUTEX_WAKE/FUTEX_REQUEUE can't use a FIFO. I think there was a patch
> > > floating around to use a plist there instead, which is one possibility,
> > > another one is to keep the queue sorted by priority (and adjust whenever
> > > priority changes - one thread can be waiting on at most one futex at a
> > > time).
> > >
> >
> > The patch you refer to is at
> > http://marc.theaimsgroup.com/?l=linux-kernel&m=114725326712391&w=2
>
> Thank you all. I'll look into PI futexes which seems the right
> direction, but I still welcome short term (limited) solutions,
> hopefully work with existing glibc. I'll look at the plist patch.

Three months after, I have tried kernel 2.6.18 with recent glibc. I
got desired results for pthread_mutex_unlock and
pthread_cond_broadcast, with PI-mutex.

But pthread_cond_signal and sem_post still wakeup a thread in FIFO
order, as you can guess.

With the plist patch (applied by hand), I can get desired behavior.
Thank you. But It seems the patch lacks reordering on priority
changes.

Are there any patch or future plan to address remaining wakeup-order
issues?


<off_topic>
BTW, If I tried to create a PI mutex on a kernel without PI futex
support, pthread_mutexattr_setprotocol(PTHREAD_PRIO_INHERIT) returned
0 and pthread_mutex_init() returned ENOTSUP. This is not a right
behavior according to the manual ...
</off_topic>

---
Atsushi Nemoto


2006-09-07 08:33:11

by Jakub Jelinek

[permalink] [raw]
Subject: Re: NPTL mutex and the scheduling priority

On Thu, Sep 07, 2006 at 05:11:58PM +0900, Atsushi Nemoto wrote:
> Three months after, I have tried kernel 2.6.18 with recent glibc. I
> got desired results for pthread_mutex_unlock and
> pthread_cond_broadcast, with PI-mutex.
>
> But pthread_cond_signal and sem_post still wakeup a thread in FIFO
> order, as you can guess.
>
> With the plist patch (applied by hand), I can get desired behavior.
> Thank you. But It seems the patch lacks reordering on priority
> changes.

Yes, either something like the plist patch for FUTEX_WAKE etc. or, if that
proves to be too slow for the usual case (non-RT threads), FIFO wakeup
initially and conversion to plist wakeup whenever first waiter with realtime
priority is added, is still needed. That will cure e.g. non-PI
pthread_mutex_unlock and sem_post. For pthread_cond_{signal,broadcast} we
need further kernel changes, so that the condvar's internal lock can be
always a PI lock.

> <off_topic>
> BTW, If I tried to create a PI mutex on a kernel without PI futex
> support, pthread_mutexattr_setprotocol(PTHREAD_PRIO_INHERIT) returned
> 0 and pthread_mutex_init() returned ENOTSUP. This is not a right
> behavior according to the manual ...
> </off_topic>

Why?
POSIX doesn't forbid ENOTSUP in pthread_mutex_init to my knowledge.

Jakub

2006-09-07 09:30:31

by Atsushi Nemoto

[permalink] [raw]
Subject: Re: NPTL mutex and the scheduling priority

On Thu, 7 Sep 2006 04:32:44 -0400, Jakub Jelinek <[email protected]> wrote:
> > But pthread_cond_signal and sem_post still wakeup a thread in FIFO
> > order, as you can guess.
> >
> > With the plist patch (applied by hand), I can get desired behavior.
> > Thank you. But It seems the patch lacks reordering on priority
> > changes.
>
> Yes, either something like the plist patch for FUTEX_WAKE etc. or, if that
> proves to be too slow for the usual case (non-RT threads), FIFO wakeup
> initially and conversion to plist wakeup whenever first waiter with realtime
> priority is added, is still needed. That will cure e.g. non-PI
> pthread_mutex_unlock and sem_post. For pthread_cond_{signal,broadcast} we
> need further kernel changes, so that the condvar's internal lock can be
> always a PI lock.

Thank you, I'll stay tuned.

> > <off_topic>
> > BTW, If I tried to create a PI mutex on a kernel without PI futex
> > support, pthread_mutexattr_setprotocol(PTHREAD_PRIO_INHERIT) returned
> > 0 and pthread_mutex_init() returned ENOTSUP. This is not a right
> > behavior according to the manual ...
> > </off_topic>
>
> Why?
> POSIX doesn't forbid ENOTSUP in pthread_mutex_init to my knowledge.

http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setprotocol.html
http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_init.html

>From ERRORS section of pthread_mutexattr_setprotocol:

The pthread_mutexattr_setprotocol() function shall fail if:
[ENOTSUP]
The value specified by protocol is an unsupported value.

And ENOTSUP is not enumerated in ERRORS section of pthread_mutex_init.

---
Atsushi Nemoto

2006-09-07 09:38:16

by Andreas Schwab

[permalink] [raw]
Subject: Re: NPTL mutex and the scheduling priority

Atsushi Nemoto <[email protected]> writes:

> And ENOTSUP is not enumerated in ERRORS section of pthread_mutex_init.

POSIX does not forbid additional error conditions, as long as the
described conditions are properly reported with the documented error
numbers.

Andreas.

--
Andreas Schwab, SuSE Labs, [email protected]
SuSE Linux Products GmbH, Maxfeldstra?e 5, 90409 N?rnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."

2006-09-07 09:42:40

by Atsushi Nemoto

[permalink] [raw]
Subject: Re: NPTL mutex and the scheduling priority

On Thu, 07 Sep 2006 11:37:54 +0200, Andreas Schwab <[email protected]> wrote:
> > And ENOTSUP is not enumerated in ERRORS section of pthread_mutex_init.
>
> POSIX does not forbid additional error conditions, as long as the
> described conditions are properly reported with the documented error
> numbers.

Oh, I see the point. Thank you.

---
Atsushi Nemoto