2002-07-03 18:10:14

by Xinwen Fu

[permalink] [raw]
Subject: kernel timers vs network card interrupt

Hi, all,
I'm curious that if a network card interrupt happens at the same
time as the kernel timer expires, what will happen?

It's said the kernel timer is guaranteed accurate. But if
interrupts are not masked off, the network interrupt also should get
response when a kernel timer expires. So I don't know who will preempt
who.

Thanks for information!

Xinwen Fu



2002-07-03 18:32:38

by Richard B. Johnson

[permalink] [raw]
Subject: Re: kernel timers vs network card interrupt

On Wed, 3 Jul 2002, Xinwen - Fu wrote:

> Hi, all,
> I'm curious that if a network card interrupt happens at the same
> time as the kernel timer expires, what will happen?
>
> It's said the kernel timer is guaranteed accurate. But if
> interrupts are not masked off, the network interrupt also should get
> response when a kernel timer expires. So I don't know who will preempt
> who.
>
> Thanks for information!
>
> Xinwen Fu

The highest priority interrupt will get serviced first. It's the timer.
Interrupts are serviced in priority-order. Hardware "remembers" which
ones are pending so none are lost if some driver doesn't do something
stupid.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).

Windows-2000/Professional isn't.

2002-07-03 23:52:19

by Xinwen Fu

[permalink] [raw]
Subject: timer queue is still influenced by network load

Richard,
I did a few experiments using the example (jiq, I changed jiffies
to do_gettimeofday() ) from Linux Device
Driver, 2nd version (p196).

I have two machines m1 and m2. On m1, I run a timer queue (jiq)
module. Then I download a big file from m1 to m2. The timings are
different between before ftp and during ftp.

----------------------------------------
before ftp
----------------------------------------
time pid cpu command
420590 1 0 0 swapper
430580 1 0 0 swapper
440579 1 0 0 swapper
450579 1 0 0 swapper
460579 1 0 0 swapper
470579 1 0 0 swapper
480579 1 0 0 swapper
490579 1 0 0 swapper
500579 1 0 0 swapper

----------------------------------------
during ftp
----------------------------------------
time pid cpu command
370605 1524 0 in.ftpd
380645 0 0 swapper
390583 0 0 swapper
400667 0 0 swapper
410703 1524 0 in.ftpd
420679 0 0 swapper
430634 0 0 swapper
440624 0 0 swapper
450648 0 0 swapper



It shows that
timer queue is still not accurate. So
the conclusion of " you're guaranteed that the queue will run at the next
clock tick, thus eliminating latency caused by system load" is WRONG!!!

What is your opinion?

Xinwen Fu


On Wed, 3 Jul 2002, Richard B. Johnson wrote:

> On Wed, 3 Jul 2002, Xinwen - Fu wrote:
>
> > Hi, all,
> > I'm curious that if a network card interrupt happens at the same
> > time as the kernel timer expires, what will happen?
> >
> > It's said the kernel timer is guaranteed accurate. But if
> > interrupts are not masked off, the network interrupt also should get
> > response when a kernel timer expires. So I don't know who will preempt
> > who.
> >
> > Thanks for information!
> >
> > Xinwen Fu
>
> The highest priority interrupt will get serviced first. It's the timer.
> Interrupts are serviced in priority-order. Hardware "remembers" which
> ones are pending so none are lost if some driver doesn't do something
> stupid.
>
> Cheers,
> Dick Johnson
>
> Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
>
> Windows-2000/Professional isn't.
>
>

2002-07-04 07:44:31

by George Anzinger

[permalink] [raw]
Subject: Re: kernel timers vs network card interrupt

"Richard B. Johnson" wrote:
>
> On Wed, 3 Jul 2002, Xinwen - Fu wrote:
>
> > Hi, all,
> > I'm curious that if a network card interrupt happens at the same
> > time as the kernel timer expires, what will happen?
> >
> > It's said the kernel timer is guaranteed accurate. But if
> > interrupts are not masked off, the network interrupt also should get
> > response when a kernel timer expires. So I don't know who will preempt
> > who.
> >
> > Thanks for information!
> >
> > Xinwen Fu
>
> The highest priority interrupt will get serviced first. It's the timer.
> Interrupts are serviced in priority-order. Hardware "remembers" which
> ones are pending so none are lost if some driver doesn't do something
> stupid.

That is true as far as it goes, HOWEVER, timers are serviced
by bottom half code which is run at the end of the
interrupt, WITH THE INTERRUPT SYSTEM ON. Therefore, timer
servicing can be interrupted by an interrupt and thus be
delayed.

--
George Anzinger [email protected]
High-res-timers:
http://sourceforge.net/projects/high-res-timers/
Real time sched: http://sourceforge.net/projects/rtsched/
Preemption patch:
http://www.kernel.org/pub/linux/kernel/people/rml

2002-07-04 16:07:46

by Xinwen Fu

[permalink] [raw]
Subject: Re: kernel timers vs network card interrupt

In fact I want a timer (either in user level or kernel level).
This timer (hope it is a periodic timer) must expire at the interval that
I specify. For example, if I
want that the timer expires at 10ms, it should never be fired at
10.0000000001ms or
9.9999999999ms. That is the key part that I want!

Have an idea?

Thanks!

Xinwen Fu


On Thu, 4 Jul 2002, george anzinger wrote:

> "Richard B. Johnson" wrote:
> >
> > On Wed, 3 Jul 2002, Xinwen - Fu wrote:
> >
> > > Hi, all,
> > > I'm curious that if a network card interrupt happens at the same
> > > time as the kernel timer expires, what will happen?
> > >
> > > It's said the kernel timer is guaranteed accurate. But if
> > > interrupts are not masked off, the network interrupt also should get
> > > response when a kernel timer expires. So I don't know who will preempt
> > > who.
> > >
> > > Thanks for information!
> > >
> > > Xinwen Fu
> >
> > The highest priority interrupt will get serviced first. It's the timer.
> > Interrupts are serviced in priority-order. Hardware "remembers" which
> > ones are pending so none are lost if some driver doesn't do something
> > stupid.
>
> That is true as far as it goes, HOWEVER, timers are serviced
> by bottom half code which is run at the end of the
> interrupt, WITH THE INTERRUPT SYSTEM ON. Therefore, timer
> servicing can be interrupted by an interrupt and thus be
> delayed.
>
> --
> George Anzinger [email protected]
> High-res-timers:
> http://sourceforge.net/projects/high-res-timers/
> Real time sched: http://sourceforge.net/projects/rtsched/
> Preemption patch:
> http://www.kernel.org/pub/linux/kernel/people/rml
>

2002-07-04 18:21:52

by Mark Hahn

[permalink] [raw]
Subject: Re: kernel timers vs network card interrupt

> In fact I want a timer (either in user level or kernel level).
> This timer (hope it is a periodic timer) must expire at the interval that
> I specify. For example, if I

this sounds *perfect* for /dev/rtc, assuming you can deal with
its fixed set of periodic rates.

2002-07-05 06:09:45

by George Anzinger

[permalink] [raw]
Subject: Re: kernel timers vs network card interrupt

Xinwen - Fu wrote:
>
> In fact I want a timer (either in user level or kernel level).
> This timer (hope it is a periodic timer) must expire at the interval that
> I specify. For example, if I
> want that the timer expires at 10ms, it should never be fired at
> 10.0000000001ms or
> 9.9999999999ms. That is the key part that I want!

10 nines! Lots of luck. You need to spend a LOT more money
than I have. Cesium clocks may be able to do this, but not
computers...

But first, please define "fire". If you mean that the
interrupt is generated at this rate, well we can do maybe 4
or 5 nines. If, on the other hand you mean "your timer
function gets cpu cycles", I don't think you will find a
machine that can do much better than one or 2 nines. Even
if the timer is the only interrupt, you still have interrupt
off times and cache indeterminism to contend with.

If the idea is to to "tickle" some hardware with this
signal, you will do better to not involve a computer in the
link.

The utime project had some software that would schedule a
timer tick early and then loop reading the TSC until the
"exact" time. This still has the problems of interrupts and
cache misses, but it is probably the only way to approach
what you want. Nothing magic, you just figure the worst
case latency and set your timer to expire early enough to be
ahead of the appointed time. Then you loop on the TSC
waiting for your exact time.

-g
>
> Have an idea?
>
> Thanks!
>
> Xinwen Fu
>
> On Thu, 4 Jul 2002, george anzinger wrote:
>
> > "Richard B. Johnson" wrote:
> > >
> > > On Wed, 3 Jul 2002, Xinwen - Fu wrote:
> > >
> > > > Hi, all,
> > > > I'm curious that if a network card interrupt happens at the same
> > > > time as the kernel timer expires, what will happen?
> > > >
> > > > It's said the kernel timer is guaranteed accurate. But if
> > > > interrupts are not masked off, the network interrupt also should get
> > > > response when a kernel timer expires. So I don't know who will preempt
> > > > who.
> > > >
> > > > Thanks for information!
> > > >
> > > > Xinwen Fu
> > >
> > > The highest priority interrupt will get serviced first. It's the timer.
> > > Interrupts are serviced in priority-order. Hardware "remembers" which
> > > ones are pending so none are lost if some driver doesn't do something
> > > stupid.
> >
> > That is true as far as it goes, HOWEVER, timers are serviced
> > by bottom half code which is run at the end of the
> > interrupt, WITH THE INTERRUPT SYSTEM ON. Therefore, timer
> > servicing can be interrupted by an interrupt and thus be
> > delayed.
> >
> > --
> > George Anzinger [email protected]
> > High-res-timers:
> > http://sourceforge.net/projects/high-res-timers/
> > Real time sched: http://sourceforge.net/projects/rtsched/
> > Preemption patch:
> > http://www.kernel.org/pub/linux/kernel/people/rml
> >

--
George Anzinger [email protected]
High-res-timers:
http://sourceforge.net/projects/high-res-timers/
Real time sched: http://sourceforge.net/projects/rtsched/
Preemption patch:
http://www.kernel.org/pub/linux/kernel/people/rml

2002-07-05 18:47:58

by Xinwen Fu

[permalink] [raw]
Subject: Re: kernel timers vs network card interrupt

Hi,
Do you know if there are existing tools to test the network driver
latency (between time to start the interrupt handler (irq.c) and time to
start tasklet (bh))?

Thanks!
Xinwen Fu


On Thu, 4 Jul 2002, george anzinger wrote:

> Xinwen - Fu wrote:
> >
> > In fact I want a timer (either in user level or kernel level).
> > This timer (hope it is a periodic timer) must expire at the interval that
> > I specify. For example, if I
> > want that the timer expires at 10ms, it should never be fired at
> > 10.0000000001ms or
> > 9.9999999999ms. That is the key part that I want!
>
> 10 nines! Lots of luck. You need to spend a LOT more money
> than I have. Cesium clocks may be able to do this, but not
> computers...
>
> But first, please define "fire". If you mean that the
> interrupt is generated at this rate, well we can do maybe 4
> or 5 nines. If, on the other hand you mean "your timer
> function gets cpu cycles", I don't think you will find a
> machine that can do much better than one or 2 nines. Even
> if the timer is the only interrupt, you still have interrupt
> off times and cache indeterminism to contend with.
>
> If the idea is to to "tickle" some hardware with this
> signal, you will do better to not involve a computer in the
> link.
>
> The utime project had some software that would schedule a
> timer tick early and then loop reading the TSC until the
> "exact" time. This still has the problems of interrupts and
> cache misses, but it is probably the only way to approach
> what you want. Nothing magic, you just figure the worst
> case latency and set your timer to expire early enough to be
> ahead of the appointed time. Then you loop on the TSC
> waiting for your exact time.
>
> -g
> >
> > Have an idea?
> >
> > Thanks!
> >
> > Xinwen Fu
> >
> > On Thu, 4 Jul 2002, george anzinger wrote:
> >
> > > "Richard B. Johnson" wrote:
> > > >
> > > > On Wed, 3 Jul 2002, Xinwen - Fu wrote:
> > > >
> > > > > Hi, all,
> > > > > I'm curious that if a network card interrupt happens at the same
> > > > > time as the kernel timer expires, what will happen?
> > > > >
> > > > > It's said the kernel timer is guaranteed accurate. But if
> > > > > interrupts are not masked off, the network interrupt also should get
> > > > > response when a kernel timer expires. So I don't know who will preempt
> > > > > who.
> > > > >
> > > > > Thanks for information!
> > > > >
> > > > > Xinwen Fu
> > > >
> > > > The highest priority interrupt will get serviced first. It's the timer.
> > > > Interrupts are serviced in priority-order. Hardware "remembers" which
> > > > ones are pending so none are lost if some driver doesn't do something
> > > > stupid.
> > >
> > > That is true as far as it goes, HOWEVER, timers are serviced
> > > by bottom half code which is run at the end of the
> > > interrupt, WITH THE INTERRUPT SYSTEM ON. Therefore, timer
> > > servicing can be interrupted by an interrupt and thus be
> > > delayed.
> > >
> > > --
> > > George Anzinger [email protected]
> > > High-res-timers:
> > > http://sourceforge.net/projects/high-res-timers/
> > > Real time sched: http://sourceforge.net/projects/rtsched/
> > > Preemption patch:
> > > http://www.kernel.org/pub/linux/kernel/people/rml
> > >
>
> --
> George Anzinger [email protected]
> High-res-timers:
> http://sourceforge.net/projects/high-res-timers/
> Real time sched: http://sourceforge.net/projects/rtsched/
> Preemption patch:
> http://www.kernel.org/pub/linux/kernel/people/rml
>

2002-07-06 04:24:12

by Bill Davidsen

[permalink] [raw]
Subject: Re: kernel timers vs network card interrupt

On Thu, 4 Jul 2002, Xinwen - Fu wrote:

> In fact I want a timer (either in user level or kernel level).
> This timer (hope it is a periodic timer) must expire at the interval that
> I specify. For example, if I
> want that the timer expires at 10ms, it should never be fired at
> 10.0000000001ms or
> 9.9999999999ms. That is the key part that I want!

If you find a way to do picosecond latency on a PC let us know. I (a)
don't believe you need any such thing, and (b) think this is a troll. Or
is the femtosec?

--
bill davidsen <[email protected]>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

2002-07-08 12:23:43

by Richard B. Johnson

[permalink] [raw]
Subject: Re: timer queue is still influenced by network load

On Wed, 3 Jul 2002, Xinwen - Fu wrote:

> Richard,
> I did a few experiments using the example (jiq, I changed jiffies
> to do_gettimeofday() ) from Linux Device
> Driver, 2nd version (p196).
>
> I have two machines m1 and m2. On m1, I run a timer queue (jiq)
> module. Then I download a big file from m1 to m2. The timings are
> different between before ftp and during ftp.
>

>
>
> It shows that
> timer queue is still not accurate. So
> the conclusion of " you're guaranteed that the queue will run at the next
> clock tick, thus eliminating latency caused by system load" is WRONG!!!
>
> What is your opinion?

Well you are guaranteed that it will run. You just don't know how fast
it will run. The bottom-half code run off the timer-queue is run with
the interrupts enabled. It can get interrupted and it may be interrupted
by network driver code that loops in ISRs, taking a large percentage
of the CPU cycles.

So, I don't think you are measuring what you think you are measuring.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).

Windows-2000/Professional isn't.