2007-01-24 19:03:49

by john stultz

[permalink] [raw]
Subject: Re: One-shot high-resolution POSIX timer periodically late

On Wed, 2007-01-24 at 10:41 +0100, John wrote:
> I'm using the POSIX timers API. My platform is x86 running Linux
> 2.6.18.6 patched with the high-resolution timer subsystem.
>
> http://www.tglx.de/hrtimers.html
>
> I've written a small "de-jittering engine" that receives packets in
> small bursts due to network jitter (typical average rate of 1000 packets
> per second), and re-sends them at a "smooth" rate.
>
> Just before I re-send a packet, I arm a one-shot timer in order to
> receive a signal when it is time to send the next packet.
>
> I've noticed a strange phenomenon that I cannot explain.
>
> Sometimes (rarely) the one-shot timer will expire more than 50 µs later
> than expected. This would seem normal, except that it happens periodically.
>
> For example, my app had been running normally for 2 minutes when it
> started printing diagnostics (see below).
>
> The first T_NEXT_POP is the date the timer was supposed to expire,
>
> NOW is the date the timer was handled after returning from sigwaitinfo
> (I am aware that blocking signals, and handling them at a specific point
> in the code will add some latency)
>
> The second T_NEXT_POP is the date the next timer is supposed to expire.
>
> DIFF is the difference between real and expected dates.
>
> (All dates are CLOCK_MONOTONIC by the way.)
>
> As you can see, the first diagnostic came at 472.410501014... Then
> another diagnostic almost exactly two seconds apart 9 times in a row!
>
> My process is the only SCHED_FIFO process on the system. There are no
> user-space processes with a higher priority. AFAICT, only a kernel
> thread could keep the CPU away from my app.
>
> Is there a periodic kernel thread that runs every 2 seconds, cannot be
> preempted, and runs for over 50 µs??

This sounds like a BIOS SMI issue. Can you reproduce this behavior on
different hardware?

thanks
-john


2007-01-24 20:11:08

by Ingo Molnar

[permalink] [raw]
Subject: Re: One-shot high-resolution POSIX timer periodically late


* john stultz <[email protected]> wrote:

> On Wed, 2007-01-24 at 10:41 +0100, John wrote:
> > I'm using the POSIX timers API. My platform is x86 running Linux
> > 2.6.18.6 patched with the high-resolution timer subsystem.
> >
> > http://www.tglx.de/hrtimers.html

> > My process is the only SCHED_FIFO process on the system. There are
> > no user-space processes with a higher priority. AFAICT, only a
> > kernel thread could keep the CPU away from my app.
> >
> > Is there a periodic kernel thread that runs every 2 seconds, cannot
> > be preempted, and runs for over 50 ?s??
>
> This sounds like a BIOS SMI issue. Can you reproduce this behavior on
> different hardware?

note that only the -hrt patchset is used - not the full -rt patchset -
so 50 usecs delays (and more) are quite possible and common.

My question would be: does the same problem occur with the full -rt
patchset and PREEMPT_RT? (see http://rt.wiki.kernel.org for details)

Ingo