Date: Wed, 30 Mar 2005 09:41:40 -0500 (EST)
From: Mark Hahn <hahn@physics.mcmaster.ca>
To: kus Kusche Klaus <kus@keba.com>
Cc: linux-kernel@vger.kernel.org, <linux-ide@vger.kernel.org>
Subject: Re: 2.6.11, IDE: Strange scheduling behaviour: high-pri RT process
 not scheduled?
In-Reply-To: <AAD6DA242BC63C488511C611BD51F3673231C2@MAILIT.keba.co.at>
Message-ID: <Pine.LNX.4.44.0503300931350.7542-100000@coffee.psychology.mcmaster.ca>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2510
Lines: 62

> I've written a small test program which enables periodic RTC interrupts 
> at 8192 Hz and then goes into a loop reading /dev/rtc and collecting 
> timing statistics (using the rdtscl macro).

straightforward test, used for many years in the linux community
(I claim to have been the first to publish it on lkml ;)

> The test system runs a 2.6.11 kernel (no SMP) on a Pentium3 500 MHz 
> embedded hardware.

which probably has memory bandwidth of at most a couple hundred MB/s,
which is really horrible by modern standards.

> However, things break seriously when exercising the CF card in parallel 
> (e.g. with a dd if=/dev/hda of=/dev/null):
> 
> * The rtc *interrupt handler* is delayed for up to 250 *micro*seconds. 
> This is very bad for my purpose, but easy to explain: It is roughly the 
> time needed to transfer 512 Bytes from a CF card which can transfer 2 
> Mbyte/sec, and obviously, the CPU blocks all interrupts while making pio
> 
> transfers. (Why? Is this really necessary?)

even with -u1, isn't there still a softirq queue that will delay the wakeup
of your user-space tester?

> * The *test program* is regularly blocked for 63 *milli*seconds, 
> sometimes for up to 300 *milli*seconds, which is absolutely
> unacceptable.

guessing that's VM housekeeping.

> Now the big question:
> *** Why doesn't my rt test program get *any* CPU for 63 jiffies? ***
> (the system ticks at 1000 HZ)

because it's user-space.  the 'rt' is a bit of a misnomer - it's merely
a higher priority, less preemptable job.

> * The dd program obviously gets some CPU regularly (because it copies 2 
> MB/s, and because no other program could cause the 1 % user CPU load). 
> The dd runs at normal shell scheduling priority, so it should be 
> preempted immediately by my test program!

out of curiosity, does running it with "nice -n 19" change anything?

> 2.) Using a realtime-preempt-2.6.12-rc1-V0.7.41-11 kernel with
> PREEMPT_RT:
>     If my test program runs at rtpri 99, the problem is gone: It is 
>     scheduled within 30 microseconds after the rtc interrupt.
>     If my test program runs at rtpri 1, it still suffers from delays 
>     in the 30 to 300 millisecond range.

so your problem is solved, no?

also, did you try a (plain) preemptable kernel?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/