2005-05-09 07:37:29

by Ingo Molnar

[permalink] [raw]
Subject: [patch] Real-Time Preemption, -RT-2.6.12-rc4-V0.7.47-00


i have released the -V0.7.47-00 Real-Time Preemption patch, which can be
downloaded from the usual place:

http://redhat.com/~mingo/realtime-preempt/

this patch reintroduces the plist.h code from Daniel Walker and Inaky
Perez-Gonzalez. It's also a merge to 2.6.12-rc4.

to build a -V0.7.47-00 tree, the following patches have to be applied:

http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.11.tar.bz2
http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.12-rc4.bz2
http://redhat.com/~mingo/realtime-preempt/realtime-preempt-2.6.12-rc4-V0.7.47-00

Ingo


2005-05-09 09:10:42

by kus Kusche Klaus

[permalink] [raw]
Subject: RE: [patch] Real-Time Preemption, -RT-2.6.12-rc4-V0.7.47-00

> i have released the -V0.7.47-00 Real-Time Preemption patch,
> which can be
> downloaded from the usual place:
>
> http://redhat.com/~mingo/realtime-preempt/
>
> this patch reintroduces the plist.h code from Daniel Walker and Inaky
> Perez-Gonzalez. It's also a merge to 2.6.12-rc4.
>
> to build a -V0.7.47-00 tree, the following patches have to be applied:
>
> http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.11.tar.bz2
>
> http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.12-rc4.bz2
>
>
http://redhat.com/~mingo/realtime-preempt/realtime-preempt-2.6.12-rc4-V0
.7.47-00

It lacks "plist.h", but two "#include" refer to it?

--
Klaus Kusche (Software Development - Control Systems)
KEBA AG Gewerbepark Urfahr, A-4041 Linz, Austria (Europe)
Tel: +43 / 732 / 7090-3120 Fax: +43 / 732 / 7090-6301
E-Mail: [email protected] WWW: http://www.keba.com

2005-05-09 09:18:35

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch] Real-Time Preemption, -RT-2.6.12-rc4-V0.7.47-00

* kus Kusche Klaus <[email protected]> wrote:

> > i have released the -V0.7.47-00 Real-Time Preemption patch,
> > which can be
> > downloaded from the usual place:
> >
> > http://redhat.com/~mingo/realtime-preempt/
> >
> > this patch reintroduces the plist.h code from Daniel Walker and Inaky
> > Perez-Gonzalez. It's also a merge to 2.6.12-rc4.
> >
> > to build a -V0.7.47-00 tree, the following patches have to be applied:
> >
> > http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.11.tar.bz2
> >
> > http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.12-rc4.bz2
> >
> >
> http://redhat.com/~mingo/realtime-preempt/realtime-preempt-2.6.12-rc4-V0
> .7.47-00
>
> It lacks "plist.h", but two "#include" refer to it?

yeah - patch messup. I've uploaded -01 which adds the missing file.

Ingo

2005-05-09 19:54:51

by George Anzinger

[permalink] [raw]
Subject: Re: [patch] Real-Time Preemption, -RT-2.6.12-rc4-V0.7.47-00

Source: MontaVista Software, Inc. George Anzinger<[email protected]>
MR: 11506
Type: Defect Fix
Disposition: needs submitting to RT community patch
Keywords:
Signed-off-by: George Anzinger<[email protected]>
Description:

This change adds code to protect timers while they are being accesed by
the timer call back code. This is now needed because the call back code
is preemptable when ever softirqs are preemptable. In the past this was
needed only for SMP systems and it is that code which is expanded and
enhanced to account for the possiblility that the timer call back may be
preempted for some period of time. (In the SMP case this time was
rather short so a spin was accetable.)

posix-timers.c | 82 ++++++++++++++++++++++++++++++++++++++++-----------------
1 files changed, 58 insertions(+), 24 deletions(-)

Index: linux-2.6.10/kernel/posix-timers.c
===================================================================
--- linux-2.6.10.orig/kernel/posix-timers.c
+++ linux-2.6.10/kernel/posix-timers.c
@@ -94,7 +94,7 @@ static DEFINE_SPINLOCK(idr_lock);
#define TIMER_INACTIVE 1
#define TIMER_RETRY 1

-#ifdef CONFIG_SMP
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
# define timer_active(tmr) \
((tmr)->it_timer.entry.prev != (void *)TIMER_INACTIVE)
# define set_timer_inactive(tmr) \
@@ -102,10 +102,28 @@ static DEFINE_SPINLOCK(idr_lock);
(tmr)->it_timer.entry.prev = (void *)TIMER_INACTIVE; \
} while (0)
#else
-# define timer_active(tmr) BARFY // error to use outside of SMP
+# define timer_active(tmr) BARFY /* error to use outside of SMP | RT */
# define set_timer_inactive(tmr) do { } while (0)
#endif
/*
+ * For RT the timer call backs are preemptable. This means that folks
+ * trying to delete timers may run into timers that are "active" for
+ * long times. To help out with this we provide a wake up function to
+ * wake up a caller who wants waking when a timer clears the call back.
+ * This is the same sort of thing that the del_timer_sync does, but we
+ * need (in the HRT case) to cover two lists and not just the one.
+ */
+#ifdef CONFIG_PREEMPT_SOFTIRQS
+#include <linux/wait.h>
+static DECLARE_WAIT_QUEUE_HEAD(timer_wake_queue);
+#define wake_timer_waiters() wake_up(&timer_wake_queue)
+#define wait_for_timer(timer) wait_event(timer_wake_queue, !timer_active(timer))
+
+#else
+#define wake_timer_waiters()
+#define wait_for_timer(timer)
+#endif
+/*
* we assume that the new SIGEV_THREAD_ID shares no bits with the other
* SIGEV values. Here we put out an error if this assumption fails.
*/
@@ -461,6 +479,7 @@ static void posix_timer_fn(unsigned long
schedule_next_timer(timr);
}
unlock_timer(timr, flags); /* hold thru abs lock to keep irq off */
+ wake_timer_waiters();
}


@@ -922,18 +941,20 @@ do_timer_settime(struct k_itimer *timr,
* careful here. If smp we could be in the "fire" routine which will
* be spinning as we hold the lock. But this is ONLY an SMP issue.
*/
-#ifdef CONFIG_SMP
- if (timer_active(timr) && !del_timer(&timr->it_timer))
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
+ if (timer_active(timr) && !del_timer(&timr->it_timer)) {
/*
- * It can only be active if on an other cpu. Since
- * we have cleared the interval stuff above, it should
- * clear once we release the spin lock. Of course once
- * we do that anything could happen, including the
- * complete melt down of the timer. So return with
- * a "retry" exit status.
+ * It can only be active if on an other cpu (unless RT).
+ * Since we have cleared the interval stuff above, it
+ * should clear once we release the spin lock. Of
+ * course once we do that anything could happen,
+ * including the complete melt down of the timer. So
+ * return with a "retry" exit status. If RT we do a
+ * formal wait as the function code is fully
+ * preemptable...
*/
return TIMER_RETRY;
-
+ }
set_timer_inactive(timr);
#else
del_timer(&timr->it_timer);
@@ -1011,7 +1032,8 @@ retry:
&new_spec, rtn);
unlock_timer(timr, flag);
if (error == TIMER_RETRY) {
- rtn = NULL; // We already got the old time...
+ wait_for_timer(timr);
+ rtn = NULL; /* We already got the old time... */
goto retry;
}

@@ -1025,17 +1047,19 @@ retry:
static inline int do_timer_delete(struct k_itimer *timer)
{
timer->it_incr = 0;
-#ifdef CONFIG_SMP
- if (timer_active(timer) && !del_timer(&timer->it_timer))
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
+ if (timer_active(timer) && !del_timer(&timer->it_timer)) {
/*
- * It can only be active if on an other cpu. Since
- * we have cleared the interval stuff above, it should
- * clear once we release the spin lock. Of course once
- * we do that anything could happen, including the
- * complete melt down of the timer. So return with
- * a "retry" exit status.
+ * It can only be active if on an other cpu (unless RT).
+ * Since we have cleared the interval stuff above, it
+ * should clear once we release the spin lock. Of
+ * course once we do that anything could happen,
+ * including the complete melt down of the timer. So
+ * return with a "retry" exit status. For RT we do a
+ * formal wait as it could take a while.
*/
return TIMER_RETRY;
+ }
#else
del_timer(&timer->it_timer);
#endif
@@ -1051,7 +1075,7 @@ sys_timer_delete(timer_t timer_id)
struct k_itimer *timer;
long flags;

-#ifdef CONFIG_SMP
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
int error;
retry_delete:
#endif
@@ -1059,11 +1083,12 @@ retry_delete:
if (!timer)
return -EINVAL;

-#ifdef CONFIG_SMP
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
error = p_timer_del(&posix_clocks[timer->it_clock], timer);

if (error == TIMER_RETRY) {
unlock_timer(timer, flags);
+ wait_for_timer(timer);
goto retry_delete;
}
#else
@@ -1092,17 +1117,18 @@ static inline void itimer_delete(struct
{
unsigned long flags;

-#ifdef CONFIG_SMP
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
int error;
retry_delete:
#endif
spin_lock_irqsave(&timer->it_lock, flags);

-#ifdef CONFIG_SMP
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
error = p_timer_del(&posix_clocks[timer->it_clock], timer);

if (error == TIMER_RETRY) {
unlock_timer(timer, flags);
+ wait_for_timer(timer);
goto retry_delete;
}
#else
@@ -1369,6 +1395,14 @@ void clock_was_set(void)
list_del_init(&timr->abs_timer_entry);
if (add_clockset_delta(timr, &new_wall_to) &&
del_timer(&timr->it_timer)) /* timer run yet? */
+ /*
+ * Note that we only do this if the timer is/was
+ * in the list. If it happens to be active an
+ * not in the timer list, it must be in the call
+ * back function, we leave it to that code to do
+ * the right thing. I.e we do NOT need
+ * del_timer_sync()
+ */
add_timer(&timr->it_timer);
list_add(&timr->abs_timer_entry, &abs_list.list);
spin_unlock_irq(&abs_list.lock);


Attachments:
rt-timers-fix.patch (6.88 kB)

2005-05-16 00:02:48

by Lee Revell

[permalink] [raw]
Subject: Re: [patch] Real-Time Preemption, -RT-2.6.12-rc4-V0.7.47-00

On Mon, 2005-05-09 at 11:17 +0200, Ingo Molnar wrote:
> * kus Kusche Klaus <[email protected]> wrote:
>
> > > i have released the -V0.7.47-00 Real-Time Preemption patch,
> > > which can be
> > > downloaded from the usual place:
> > >
> > > http://redhat.com/~mingo/realtime-preempt/
> > >
> > > this patch reintroduces the plist.h code from Daniel Walker and Inaky
> > > Perez-Gonzalez. It's also a merge to 2.6.12-rc4.
> > >
> > > to build a -V0.7.47-00 tree, the following patches have to be applied:
> > >
> > > http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.11.tar.bz2
> > >
> > > http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.12-rc4.bz2
> > >
> > >
> > http://redhat.com/~mingo/realtime-preempt/realtime-preempt-2.6.12-rc4-V0
> > .7.47-00
> >
> > It lacks "plist.h", but two "#include" refer to it?
>
> yeah - patch messup. I've uploaded -01 which adds the missing file.
>

Ingo,

Can you add Mingming's ext3 patch to the next version? For my workload
at least, this seems to be the last important latency breaker that we
need to go upstream.

Lee

2005-05-23 07:55:12

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch] Real-Time Preemption, -RT-2.6.12-rc4-V0.7.47-00


* Lee Revell <[email protected]> wrote:

> Ingo,
>
> Can you add Mingming's ext3 patch to the next version? For my
> workload at least, this seems to be the last important latency breaker
> that we need to go upstream.

yeah, agreed - i've applied it to my tree and it's looking good in my
ext3 tests.

Ingo

2005-05-24 07:59:52

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch] Real-Time Preemption, -RT-2.6.12-rc4-V0.7.47-00


* George Anzinger <[email protected]> wrote:

> Also, I think that del_timer_sync and friends need to be turned on if
> soft_irq is preemptable.

you mean the #ifdef CONFIG_SMP should be:

#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)

?

Ingo

2005-05-24 14:28:02

by George Anzinger

[permalink] [raw]
Subject: Re: [patch] Real-Time Preemption, -RT-2.6.12-rc4-V0.7.47-00

Ingo Molnar wrote:
> * George Anzinger <[email protected]> wrote:
>
>
>>Also, I think that del_timer_sync and friends need to be turned on if
>>soft_irq is preemptable.
>
>
> you mean the #ifdef CONFIG_SMP should be:
>
> #if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
>
> ?
Yes, exactly.

>

--
George Anzinger [email protected]
High-res-timers: http://sourceforge.net/projects/high-res-timers/

2005-05-27 03:00:56

by George Anzinger

[permalink] [raw]
Subject: Re: [patch] Real-Time Preemption, -RT-2.6.12-rc4-V0.7.47-00

Source: MontaVista Software, Inc. George Anzinger<[email protected]>
MR: 11506
Type: Defect Fix
Disposition: needs submitting to RT community patch
Keywords:
Signed-off-by: George Anzinger<[email protected]>
Description:

This change adds code to protect timers while they are being accesed by
the timer call back code. This is now needed because the call back code
is preemptable when ever softirqs are preemptable. In the past this was
needed only for SMP systems and it is that code which is expanded and
enhanced to account for the possiblility that the timer call back may be
preempted for some period of time. (In the SMP case this time was
rather short so a spin was accetable.)

posix-timers.c | 93 ++++++++++++++++++++++++++++++++++++++-------------------
1 files changed, 63 insertions(+), 30 deletions(-)

Index: linux-2.6.12-rc/kernel/posix-timers.c
===================================================================
--- linux-2.6.12-rc.orig/kernel/posix-timers.c
+++ linux-2.6.12-rc/kernel/posix-timers.c
@@ -94,7 +94,7 @@ static DEFINE_SPINLOCK(idr_lock);
*/
#define TIMER_INACTIVE 1

-#ifdef CONFIG_SMP
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
# define timer_active(tmr) \
((tmr)->it.real.timer.entry.prev != (void *)TIMER_INACTIVE)
# define set_timer_inactive(tmr) \
@@ -102,10 +102,28 @@ static DEFINE_SPINLOCK(idr_lock);
(tmr)->it.real.timer.entry.prev = (void *)TIMER_INACTIVE; \
} while (0)
#else
-# define timer_active(tmr) BARFY // error to use outside of SMP
+# define timer_active(tmr) BARFY /* error to use outside of SMP | RT */
# define set_timer_inactive(tmr) do { } while (0)
#endif
/*
+ * For RT the timer call backs are preemptable. This means that folks
+ * trying to delete timers may run into timers that are "active" for
+ * long times. To help out with this we provide a wake up function to
+ * wake up a caller who wants waking when a timer clears the call back.
+ * This is the same sort of thing that the del_timer_sync does, but we
+ * need (in the HRT case) to cover two lists and not just the one.
+ */
+#ifdef CONFIG_PREEMPT_SOFTIRQS
+#include <linux/wait.h>
+static DECLARE_WAIT_QUEUE_HEAD(timer_wake_queue);
+#define wake_timer_waiters() wake_up(&timer_wake_queue)
+#define wait_for_timer(timer) wait_event(timer_wake_queue, !timer_active(timer))
+
+#else
+#define wake_timer_waiters()
+#define wait_for_timer(timer)
+#endif
+/*
* we assume that the new SIGEV_THREAD_ID shares no bits with the other
* SIGEV values. Here we put out an error if this assumption fails.
*/
@@ -527,6 +545,7 @@ static void posix_timer_fn(unsigned long
schedule_next_timer(timr);
}
unlock_timer(timr, flags); /* hold thru abs lock to keep irq off */
+ wake_timer_waiters();
}


@@ -983,18 +1002,20 @@ common_timer_set(struct k_itimer *timr,
* careful here. If smp we could be in the "fire" routine which will
* be spinning as we hold the lock. But this is ONLY an SMP issue.
*/
-#ifdef CONFIG_SMP
- if (timer_active(timr) && !del_timer(&timr->it.real.timer))
- /*
- * It can only be active if on an other cpu. Since
- * we have cleared the interval stuff above, it should
- * clear once we release the spin lock. Of course once
- * we do that anything could happen, including the
- * complete melt down of the timer. So return with
- * a "retry" exit status.
- */
- return TIMER_RETRY;
-
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
+ if (timer_active(timr) && !del_timer(&timr->it.real.timer)) {
+ /*
+ * It can only be active if on an other cpu (unless RT).
+ * Since we have cleared the interval stuff above, it
+ * should clear once we release the spin lock. Of
+ * course once we do that anything could happen,
+ * including the complete melt down of the timer. So
+ * return with a "retry" exit status. If RT we do a
+ * formal wait as the function code is fully
+ * preemptable...
+ */
+ return TIMER_RETRY;
+ }
set_timer_inactive(timr);
#else
del_timer(&timr->it.real.timer);
@@ -1069,7 +1090,8 @@ retry:

unlock_timer(timr, flag);
if (error == TIMER_RETRY) {
- rtn = NULL; // We already got the old time...
+ wait_for_timer(timr);
+ rtn = NULL; /* We already got the old time... */
goto retry;
}

@@ -1083,17 +1105,19 @@ retry:
static inline int common_timer_del(struct k_itimer *timer)
{
timer->it.real.incr = 0;
-#ifdef CONFIG_SMP
- if (timer_active(timer) && !del_timer(&timer->it.real.timer))
- /*
- * It can only be active if on an other cpu. Since
- * we have cleared the interval stuff above, it should
- * clear once we release the spin lock. Of course once
- * we do that anything could happen, including the
- * complete melt down of the timer. So return with
- * a "retry" exit status.
- */
- return TIMER_RETRY;
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
+ if (timer_active(timer) && !del_timer(&timer->it.real.timer)) {
+ /*
+ * It can only be active if on an other cpu (unless RT).
+ * Since we have cleared the interval stuff above, it
+ * should clear once we release the spin lock. Of
+ * course once we do that anything could happen,
+ * including the complete melt down of the timer. So
+ * return with a "retry" exit status. For RT we do a
+ * formal wait as it could take a while.
+ */
+ return TIMER_RETRY;
+ }
#else
del_timer(&timer->it.real.timer);
#endif
@@ -1114,7 +1138,7 @@ sys_timer_delete(timer_t timer_id)
struct k_itimer *timer;
long flags;

-#ifdef CONFIG_SMP
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
int error;
retry_delete:
#endif
@@ -1122,7 +1146,7 @@ retry_delete:
if (!timer)
return -EINVAL;

-#ifdef CONFIG_SMP
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
error = timer_delete_hook(timer);

if (error == TIMER_RETRY) {
@@ -1155,17 +1179,18 @@ static inline void itimer_delete(struct
{
unsigned long flags;

-#ifdef CONFIG_SMP
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
int error;
retry_delete:
#endif
spin_lock_irqsave(&timer->it_lock, flags);

-#ifdef CONFIG_SMP
+#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
error = timer_delete_hook(timer);

if (error == TIMER_RETRY) {
unlock_timer(timer, flags);
+ wait_for_timer(timer);
goto retry_delete;
}
#else
@@ -1424,6 +1449,14 @@ void clock_was_set(void)
list_del_init(&timr->it.real.abs_timer_entry);
if (add_clockset_delta(timr, &new_wall_to) &&
del_timer(&timr->it.real.timer)) /* timer run yet? */
+ /*
+ * Note that we only do this if the timer is/was
+ * in the list. If it happens to be active an
+ * not in the timer list, it must be in the call
+ * back function, we leave it to that code to do
+ * the right thing. I.e we do NOT need
+ * del_timer_sync()
+ */
add_timer(&timr->it.real.timer);
list_add(&timr->it.real.abs_timer_entry, &abs_list.list);
spin_unlock_irq(&abs_list.lock);


Attachments:
rt-timers-fix.patch (6.85 kB)

2005-05-27 07:32:57

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch] Real-Time Preemption, -RT-2.6.12-rc4-V0.7.47-00


* George Anzinger <[email protected]> wrote:

> Ingo Molnar wrote:
> >* George Anzinger <[email protected]> wrote:
> >
> >
> >>Also, I think that del_timer_sync and friends need to be turned on if
> >>soft_irq is preemptable.
> >
> >
> >you mean the #ifdef CONFIG_SMP should be:
> >
> > #if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_SOFTIRQS)
> >
> >?
> Yes, exactly.

i have done this in -47-09 and it seems to work fine - is it sufficient?

Ingo