Hi
I was trying to get minimum possible in-kernel latencies on a freescale
8360 CPU based board (powerpc arch). There is a project where <30 us worst
case latency is required, and question was - is it possible to use
linux+preempt-rt (which is preferred for a number of reason) or not.
I used 2.6.29.5-rt21 kernel, which, for my best knowledge, was the
last "release" of preempt-rt at the moment when I started.
I used 8360's Generic Timer Module to both to generate interrupts and to
measure lanencies. GTM timers continue to count after firing an interrupt,
so latency may be measured my reading current timer value and comparing
that with the value at which interrupt was generated. I believe this
method is both simple and accurate.
The result is:
- average latency up to IRQF_NODELAY interrupt header is 2-3 us, even on a
system with high i/o and cpu load. To generate load, I used a flood ping
and several 'while true; do true; done' cpu-eaters.
- but worst-case latency is BAD. I occasionally got >50 us even on idle
system.
I tried hard to identify the latency source, and at some moment discovered
tick_sched_timer() from kernel/time/tick-sched.c. This routine is called
from timer interrupt with hardware interrupts disabled, and may execute
for 50 us and more.
Could someone please comment on this?
Is it possible to move (part of) tick_sched_timer() call tree out of
hardware-interrupts-disabled context without breaking things?
Nikita
Please CC: reply to my e-mail address. Thanks.
On Tue, 23 Jun 2009, Nikita V. Youshchenko wrote:
> I tried hard to identify the latency source, and at some moment discovered
> tick_sched_timer() from kernel/time/tick-sched.c. This routine is called
> from timer interrupt with hardware interrupts disabled, and may execute
> for 50 us and more.
>
> Could someone please comment on this?
Did you have CONFIG_NOHZ enabled ? If yes, please disable.
> Is it possible to move (part of) tick_sched_timer() call tree out of
> hardware-interrupts-disabled context without breaking things?
Yes, we can move things out and split the irq disabled regions. One
particular thing which can be split out is the time update, which we
already had in the timer softirq at some point, but we moved it back
as it showed problems with NTP stability. It's one of the things which
can be revisited. The whole call chain needs some investigation, but
that's not in my main focus now.
Thanks,
tglx
> > I tried hard to identify the latency source, and at some moment
> > discovered tick_sched_timer() from kernel/time/tick-sched.c. This
> > routine is called from timer interrupt with hardware interrupts
> > disabled, and may execute for 50 us and more.
> >
> > Could someone please comment on this?
>
> Did you have CONFIG_NOHZ enabled ? If yes, please disable.
I have disabled CONFIG_NOHZ long ago, since it really influences latency.
> > Is it possible to move (part of) tick_sched_timer() call tree out of
> > hardware-interrupts-disabled context without breaking things?
>
> Yes, we can move things out and split the irq disabled regions. One
> particular thing which can be split out is the time update, which we
> already had in the timer softirq at some point, but we moved it back
> as it showed problems with NTP stability. It's one of the things which
> can be revisited. The whole call chain needs some investigation, but
> that's not in my main focus now.
Is that code with time update split out available anywhere?
Nikita