2008-01-18 07:24:51

by Mark Hansen

[permalink] [raw]
Subject: priority based thread wakeup

Hello,

Firstly, may I apologise as I am not a member of the LKML, and ask that
I be CC'd in any responses that may be forthcoming.

My question concerns the following patch which was incorporated into the
2.6.22 kernel (quoted from that change log):

>Today, all threads waiting for a given futex are woken in FIFO
>order (first waiter woken first) instead of priority order.
>
>This patch makes use of plist (pirotity ordered lists) instead
>of simple list in futex_hash_bucket.
>
>All non-RT threads are stored with priority MAX_RT_PRIO, causing
>them to be woken last, in FIFO order (RT-threads are woken first,
>in priority order).
>
>Signed-off-by: Sebastien Dugue <[email protected]>
>Signed-off-by: Pierre Peiffer <[email protected]>

After updating to this version of the kernel, I was able to observe the
above fix, where multiple RT threads invoking pthread_cond_wait(), and
the highest priority thread will acquire the mutex first, after the
thread holding the mutex calls pthread_cond_signal();
pthread_mutex_unlock()

However, since kernel 2.6.23, it seems that the functionality relating
to this "priority based wakeup" has disappeared.

I understand there have been significant changes in this kernel
concerning the "Completely Fair Scheduler" replacing the "mainline"
scheduler; however my understanding is that the RT functionality would
be preserved. This does not appear to be the case based on repeating the
experiment described above.

I was wondering if this functionality is considered no longer
desirable/necessary?

If not, is it anticipated that this functionality could/would be
included in a later kernel?

Regards,
Mark Hansen


2008-01-18 13:34:18

by Gregory Haskins

[permalink] [raw]
Subject: Re: priority based thread wakeup

Mark Hansen wrote:
> Hello,
>
> Firstly, may I apologise as I am not a member of the LKML, and ask that
> I be CC'd in any responses that may be forthcoming.
>
> My question concerns the following patch which was incorporated into the
> 2.6.22 kernel (quoted from that change log):
>
>> Today, all threads waiting for a given futex are woken in FIFO
>> order (first waiter woken first) instead of priority order.
>>
>> This patch makes use of plist (pirotity ordered lists) instead
>> of simple list in futex_hash_bucket.
>>
>> All non-RT threads are stored with priority MAX_RT_PRIO, causing
>> them to be woken last, in FIFO order (RT-threads are woken first,
>> in priority order).
>>
>> Signed-off-by: Sebastien Dugue <[email protected]>
>> Signed-off-by: Pierre Peiffer <[email protected]>
>
> After updating to this version of the kernel, I was able to observe the
> above fix, where multiple RT threads invoking pthread_cond_wait(), and
> the highest priority thread will acquire the mutex first, after the
> thread holding the mutex calls pthread_cond_signal();
> pthread_mutex_unlock()
>
> However, since kernel 2.6.23, it seems that the functionality relating
> to this "priority based wakeup" has disappeared.
>
> I understand there have been significant changes in this kernel
> concerning the "Completely Fair Scheduler" replacing the "mainline"
> scheduler; however my understanding is that the RT functionality would
> be preserved. This does not appear to be the case based on repeating the
> experiment described above.
>
> I was wondering if this functionality is considered no longer
> desirable/necessary?
>
> If not, is it anticipated that this functionality could/would be
> included in a later kernel?
>

Hi Mark,
In a coffee deprived stupor ;), I responded to this mail in the wrong
thread, here:

http://lkml.org/lkml/2008/1/18/207

Sorry for the confusion
HTH

-Greg


> Regards,
> Mark Hansen
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

2008-01-21 04:28:30

by Mark Hansen

[permalink] [raw]
Subject: RE: priority based thread wakeup

>>My question concerns the following patch which was incorporated into
the >>2.6.22 kernel (quoted from that change log):
... "futex priority based wakeup"
>>After updating to this version of the kernel, I was able to observe
the >>above fix, where multiple RT threads invoking pthread_cond_wait(),
and the >>highest priority thread will acquire the mutex first, after
the thread >>holding the mutex calls pthread_cond_signal();
pthread_mutex_unlock()

>>However, since kernel 2.6.23, it seems that the functionality relating
to >>this "priority based wakeup" has disappeared.

>>I understand there have been significant changes in this kernel
concerning >>the "Completely Fair Scheduler" replacing the "mainline"
scheduler; >>however my understanding is that the RT functionality would
be preserved. >>This does not appear to be the case based on repeating
the experiment >>described above.

>Hi,
> I am not sure if this is whats happening to you (*), but there is a
>known problem in the mainline tree related to the system allowing lower
>priority threads to stay on the CPU under certain circumstances.
>
>You can see the problem happening with the following test:
>
>http://rt.wiki.kernel.org/index.php/Preemption_Test
>
>and I know the problem goes back to at least 2.6.16.
>
>Steven Rostedt and I submitted some patches to address the issue, and
>they have been accepted into Ingo's sched-devel tree. I would suggest
>trying your code against Ingo's tree and see if the problem persists.
>You can get it from:
>
>git://git.kernel.org/pub/scm/linux/kernel/git/mingo/linux-2.6-sched->de
vel.git

Thanks for the suggestion, however I've just tried this kernel, but the
pthread_cond_wait() behaviour still remains as per 2.6.23 described
above (i.e. highest prio thread doesnt get woken first).

>>(*) I have no information on whether the futex-plist implemetation was

>>pulled from the tree to cause your regression.

I've just had a look at the code from the 2.6.24 kernel and it seems as
though the futex priority list patch code is still included.

>>It is possible that the changes between 22 and 23 are just tickling
>>your environment enough to bring out this RT-preempt issue.

Or that the changes between 22 and 23 are bringing out another issue
whereby the desired behaviour of futex-plist is somehow being
circumvented.

If so, my questions still stand to anyone who might know what has caused
the change in pthread_cond_wait() behaviour for RT threads between the
22 and 23 kernels.

Cheers,
Mark