2006-11-20 22:04:31

by Ingo Molnar

[permalink] [raw]
Subject: 2.6.19-rc6-rt5

i've released the 2.6.19-rc6-rt5 tree, which can be downloaded from the
usual place:

http://redhat.com/~mingo/realtime-preempt/

the -rt YUM repository for Fedora Core 6 can be activated via:

cd /etc/yum.repos.d
wget http://people.redhat.com/~mingo/realtime-preempt/rt.repo
yum install kernel-rt

on x86_64, do:

yum install kernel-rt.x86_64

lots of fixes and improvements were done to -rt5. In particular
SMP/dual-core systems should get quite a bit faster. Changes:

- implemented proper per-cpu page allocation (PCP-list) in
page_alloc.c, for PREEMPT_RT too. (previously this code was #ifdef-ed
out and we allocated straight from the zones - but this caused the
zone lock to act as a global lock)

- speedup of PREEMPT_RT's implementation of atomic_dec_and_lock().
(this was a major bottleneck for workloads like kernel compiles.)

- more tracer features: symbolic stack backtraces embedded in
/proc/latency_trace for certain types of events, switchable syscall
tracing.

- irq-threading cleanups, based on the comments from Sergei Shtylyov,
Daniel Walker and Benjamin Herrenschmidt.

- vsyscall & tracing fixes: 'notsc' should not be required on the YUM
rpms anymore.

to build a 2.6.19-rc6-rt5 tree, the following patches should be applied:

http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.18.tar.bz2
http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.19-rc6.bz2
http://redhat.com/~mingo/realtime-preempt/patch-2.6.19-rc6-rt5

as usual, bugreports, fixes and suggestions are welcome,

Ingo


2006-11-20 22:39:29

by Alistair John Strachan

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

Hi Ingo,

On Monday 20 November 2006 22:02, Ingo Molnar wrote:
> i've released the 2.6.19-rc6-rt5 tree, which can be downloaded from the
> usual place:
>
> http://redhat.com/~mingo/realtime-preempt/

Is there a git tree for -rt?

--
Cheers,
Alistair.

Final year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.

2006-11-20 23:18:12

by Alistair John Strachan

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

On Monday 20 November 2006 22:02, Ingo Molnar wrote:
> i've released the 2.6.19-rc6-rt5 tree, which can be downloaded from the
> usual place:

Couple of problems:

[ 320.142298] ehci_hcd: Unknown symbol usb_hcd_pci_suspend
[ 320.142837] ehci_hcd: Unknown symbol usb_free_urb
[ 320.142901] ehci_hcd: Unknown symbol usb_hub_tt_clear_buffer
[ 320.142954] ehci_hcd: Unknown symbol usb_hcd_resume_root_hub
[ 320.143006] ehci_hcd: Unknown symbol usb_hcd_pci_probe
[ 320.143144] ehci_hcd: Unknown symbol usb_calc_bus_time
[ 320.143201] ehci_hcd: Unknown symbol usb_hcd_pci_shutdown
[ 320.143255] ehci_hcd: Unknown symbol usb_hcd_pci_resume
[ 320.143365] ehci_hcd: Unknown symbol usb_get_urb
[ 320.143417] ehci_hcd: Unknown symbol usb_hcd_giveback_urb
[ 320.143478] ehci_hcd: Unknown symbol usb_hcd_pci_remove
[ 320.143544] ehci_hcd: Unknown symbol usb_root_hub_lost_power

Not sure why. Don't notice it on 2.6.19-rc6. Means my distro's udev script
takes eons to complete, but I can get to init 3.

Got this when I mounted my /home partition:

[ 375.927366] XFS mounting filesystem md2
[ 376.724666] Ending clean XFS mount for filesystem: md2
[ 441.121031] stopped custom tracer.
[ 441.121067]
[ 441.121068] =============================================
[ 441.121131] [ INFO: possible recursive locking detected ]
[ 441.121165] 2.6.19-rc6-rt5 #3
[ 441.121197] ---------------------------------------------
[ 441.121231] as/1018 is trying to acquire lock:
[ 441.121264] ((struct compat_rw_semaphore *)(&(&ip->i_lock)->mr_lock)){----}, at: [<ffffffff80332b27>] xfs_ilock+0x67/0xa0
[ 441.121397]
[ 441.121398] but task is already holding lock:
[ 441.121459] ((struct compat_rw_semaphore *)(&(&ip->i_lock)->mr_lock)){----}, at: [<ffffffff80332b27>] xfs_ilock+0x67/0xa0
[ 441.121589]
[ 441.121590] other info that might help us debug this:
[ 441.121652] 2 locks held by as/1018:
[ 441.121684] #0: (&inode->i_mutex){--..}, at: [<ffffffff8021c380>] open_namei+0x100/0x6c0
[ 441.121833] #1: ((struct compat_rw_semaphore *)(&(&ip->i_lock)->mr_lock)){----}, at: [<ffffffff80332b27>] xfs_ilock+0x67
/0xa0
[ 441.121984]
[ 441.121985] stack backtrace:
[ 441.122044]
[ 441.122045] Call Trace:
[ 441.122108] [<ffffffff80271eb3>] dump_trace+0xc3/0x4a0
[ 441.122145] [<ffffffff802722d3>] show_trace+0x43/0x70
[ 441.122180] [<ffffffff80272315>] dump_stack+0x15/0x20
[ 441.122217] [<ffffffff802adaa1>] __lock_acquire+0x991/0xce0
[ 441.122254] [<ffffffff802ade78>] lock_acquire+0x88/0xc0
[ 441.122289] [<ffffffff802a8989>] compat_down_write+0x39/0x50
[ 441.122325] [<ffffffff80332b27>] xfs_ilock+0x67/0xa0
[ 441.122361] [<ffffffff80333794>] xfs_iget+0x384/0x880
[ 441.122397] [<ffffffff8034b4c4>] xfs_trans_iget+0xc4/0x150
[ 441.122433] [<ffffffff80337d2e>] xfs_ialloc+0x9e/0x4d0
[ 441.122469] [<ffffffff8034c09f>] xfs_dir_ialloc+0x7f/0x2d0
[ 441.122505] [<ffffffff80352a94>] xfs_create+0x354/0x6c0
[ 441.122541] [<ffffffff8035c73a>] xfs_vn_mknod+0x15a/0x2f0
[ 441.122577] [<ffffffff8035c8eb>] xfs_vn_create+0xb/0x10
[ 441.122613] [<ffffffff8023d000>] vfs_create+0x90/0xf0
[ 441.122649] [<ffffffff8021c455>] open_namei+0x1d5/0x6c0
[ 441.122685] [<ffffffff80229818>] do_filp_open+0x28/0x50
[ 441.122721] [<ffffffff8021af0a>] do_sys_open+0x5a/0xf0
[ 441.122757] [<ffffffff80233d3b>] sys_open+0x1b/0x20
[ 441.122793] [<ffffffff80265d5e>] system_call+0x7e/0x83
[ 441.122831] [<000000337b1ac630>]
[ 441.122863]
[ 441.122893] 2 locks held by as/1018:
[ 441.122925] #0: (&inode->i_mutex){--..}, at: [<ffffffff8021c380>] open_namei+0x100/0x6c0
[ 441.123073] #1: ((struct compat_rw_semaphore *)(&(&ip->i_lock)->mr_lock)){----}, at: [<ffffffff80332b27>] xfs_ilock+0x67/0xa0
[ 441.123224] ---------------------------
[ 441.123256] | preempt count: 00000000 ]
[ 441.123288] | 0-level deep critical section nesting:
[ 441.123322] ----------------------------------------

Also, I assume latency tracing still isn't working on x86_64? I don't
get a /proc/sys/kernel/preempt_max_latency file, but I have
the right option enabled.

Find my config and the entire dmesg log here:

http://devzero.co.uk/~alistair/2.6.19-rc6-rt5/

--
Cheers,
Alistair.

Final year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.

2006-11-21 01:35:10

by Sergio Monteiro Basto

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

On Mon, 2006-11-20 at 23:02 +0100, Ingo Molnar wrote:
> - vsyscall & tracing fixes: 'notsc' should not be required on the YUM
> rpms anymore.

Well I still need it else no boot.

Sorry for insist, but so difficult after build kernel, copy kernel-devel
too, into yum directory ?

Thanks,
--
S?rgio M.B.


Attachments:
smime.p7s (2.12 kB)

2006-11-21 03:08:34

by Sergio Monteiro Basto

[permalink] [raw]
Subject: compile problems Re: 2.6.19-rc6-rt5

On Mon, 2006-11-20 at 23:02 +0100, Ingo Molnar wrote:
> http://redhat.com/~mingo/realtime-preempt/patch-2.6.19-rc6-rt5

if I don't put in .config
CONFIG_HOTPLUG_CPU=y

I got
UPD include/linux/compile.h
arch/x86_64/kernel/vsyscall.c: In function 'vsyscall_init':
arch/x86_64/kernel/vsyscall.c:334: error: 'cpu_vsyscall_notifier' undeclared (first use in this function)
arch/x86_64/kernel/vsyscall.c:334: error: (Each undeclared identifier is reported only once
arch/x86_64/kernel/vsyscall.c:334: error: for each function it appears in.)
make[1]: *** [arch/x86_64/kernel/vsyscall.o] Error 1

Now with CONFIG_HOTPLUG_CPU=y I got:
UPD include/linux/compile.h
kernel/fork.c:72: error: section of 'tasklist_lock' conflicts with previous declaration
make[1]: *** [kernel/fork.o] Error 1
make: *** [kernel] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.64648 (%build)

Other curious thing now I have a way to reproduce an hang on my system
is build kernel with -j165 for build parallelism

Just in case send my .config in attach:

Thanks,
--
S?rgio M.B.


Attachments:
kernel-2.6.18-x86_64.config (68.20 kB)
smime.p7s (2.12 kB)
Download all attachments

2006-11-21 07:19:09

by Ingo Molnar

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5


* Alistair John Strachan <[email protected]> wrote:

> On Monday 20 November 2006 22:02, Ingo Molnar wrote:
> > i've released the 2.6.19-rc6-rt5 tree, which can be downloaded from the
> > usual place:
> >
> > http://redhat.com/~mingo/realtime-preempt/
>
> Is there a git tree for -rt?

nope - and it's not really something that fits the git model that well.
(it's frequently rebased)

Ingo

2006-11-21 07:21:54

by Ingo Molnar

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5


* Alistair John Strachan <[email protected]> wrote:

> Got this when I mounted my /home partition:
>
> [ 375.927366] XFS mounting filesystem md2
> [ 376.724666] Ending clean XFS mount for filesystem: md2
> [ 441.121031] stopped custom tracer.
> [ 441.121067]
> [ 441.121068] =============================================
> [ 441.121131] [ INFO: possible recursive locking detected ]

if you use XFS you should turn off CONFIG_PROVE_LOCKING.

> Also, I assume latency tracing still isn't working on x86_64? I don't
> get a /proc/sys/kernel/preempt_max_latency file, but I have the right
> option enabled.

i'm using it every day on both x86_64 and i386. I'll try your config.

Ingo

2006-11-21 07:44:44

by Ingo Molnar

[permalink] [raw]
Subject: Re: compile problems Re: 2.6.19-rc6-rt5


* Sergio Monteiro Basto <[email protected]> wrote:

> Now with CONFIG_HOTPLUG_CPU=y I got:
> UPD include/linux/compile.h
> kernel/fork.c:72: error: section of 'tasklist_lock' conflicts with previous declaration
> make[1]: *** [kernel/fork.o] Error 1

this one should be fixed by the patch below.

Ingo

Index: linux/kernel/fork.c
===================================================================
--- linux.orig/kernel/fork.c
+++ linux/kernel/fork.c
@@ -69,7 +69,7 @@ int max_threads; /* tunable limit on nr

DEFINE_PER_CPU(unsigned long, process_counts) = 0;

-__cacheline_aligned DEFINE_RWLOCK(tasklist_lock); /* outer */
+DEFINE_RWLOCK(tasklist_lock); /* outer */

/*
* Delayed mmdrop. In the PREEMPT_RT case we

2006-11-21 07:55:28

by Ingo Molnar

[permalink] [raw]
Subject: Re: compile problems Re: 2.6.19-rc6-rt5


* Sergio Monteiro Basto <[email protected]> wrote:

> On Mon, 2006-11-20 at 23:02 +0100, Ingo Molnar wrote:
> > http://redhat.com/~mingo/realtime-preempt/patch-2.6.19-rc6-rt5
>
> if I don't put in .config
> CONFIG_HOTPLUG_CPU=y
>
> I got
> UPD include/linux/compile.h
> arch/x86_64/kernel/vsyscall.c: In function 'vsyscall_init':
> arch/x86_64/kernel/vsyscall.c:334: error: 'cpu_vsyscall_notifier' undeclared (first use in this function)
> arch/x86_64/kernel/vsyscall.c:334: error: (Each undeclared identifier is reported only once

this one should be fixed by the patch below.

Ingo

Index: linux/arch/x86_64/kernel/vsyscall.c
===================================================================
--- linux.orig/arch/x86_64/kernel/vsyscall.c
+++ linux/arch/x86_64/kernel/vsyscall.c
@@ -300,7 +300,6 @@ static void __cpuinit cpu_vsyscall_init(
vsyscall_set_cpu(raw_smp_processor_id());
}

-#ifdef CONFIG_HOTPLUG_CPU
static int __cpuinit
cpu_vsyscall_notifier(struct notifier_block *n, unsigned long action, void *arg)
{
@@ -309,7 +308,6 @@ cpu_vsyscall_notifier(struct notifier_bl
smp_call_function_single(cpu, cpu_vsyscall_init, NULL, 0, 1);
return NOTIFY_DONE;
}
-#endif

static void __init map_vsyscall(void)
{
Index: linux/mm/page_alloc.c
===================================================================
--- linux.orig/mm/page_alloc.c
+++ linux/mm/page_alloc.c
@@ -2768,7 +2768,6 @@ void __init free_area_init(unsigned long
__pa(PAGE_OFFSET) >> PAGE_SHIFT, NULL);
}

-#ifdef CONFIG_HOTPLUG_CPU
static int page_alloc_cpu_notify(struct notifier_block *self,
unsigned long action, void *hcpu)
{
@@ -2786,7 +2785,6 @@ static int page_alloc_cpu_notify(struct
}
return NOTIFY_OK;
}
-#endif /* CONFIG_HOTPLUG_CPU */

void __init page_alloc_init(void)
{

2006-11-21 08:06:09

by Ingo Molnar

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5


* Sergio Monteiro Basto <[email protected]> wrote:

> On Mon, 2006-11-20 at 23:02 +0100, Ingo Molnar wrote:
> > - vsyscall & tracing fixes: 'notsc' should not be required on the YUM
> > rpms anymore.
>
> Well I still need it else no boot.

hmm ... still no log capture of that boot failure?

> Sorry for insist, but so difficult after build kernel, copy
> kernel-devel too, into yum directory ?

sure, i've uploaded it for the latest kernel, and it will probably be
part of the repository in the future too. Could you check that it works
for you?

Ingo

2006-11-21 12:01:12

by Marcus Hartig

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

HI!

rt4 runs fine. But while compiling 2.6.19-rc6-rt5 I get this error:

CC mm/page_alloc.o
mm/page_alloc.c: In function 'page_alloc_init':
mm/page_alloc.c:2793: error: 'page_alloc_cpu_notify' undeclared (first use
in this function)
mm/page_alloc.c:2793: error: (Each undeclared identifier is reported only once
mm/page_alloc.c:2793: error: for each function it appears in.)
make[1]: *** [mm/page_alloc.o] Error 1
make: *** [mm] Error 2

Regards,
Marcus

--
http://www.marcush.de

2006-11-21 12:45:09

by Karsten Wiese

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

Am Dienstag, 21. November 2006 13:01 schrieb Marcus Hartig:
> HI!
>
> rt4 runs fine. But while compiling 2.6.19-rc6-rt5 I get this error:
>
> CC mm/page_alloc.o
> mm/page_alloc.c: In function 'page_alloc_init':
> mm/page_alloc.c:2793: error: 'page_alloc_cpu_notify' undeclared (first use
> in this function)
> mm/page_alloc.c:2793: error: (Each undeclared identifier is reported only once
> mm/page_alloc.c:2793: error: for each function it appears in.)
> make[1]: *** [mm/page_alloc.o] Error 1
> make: *** [mm] Error 2

Ingo posted a fix in another thread.
The (whitespace damaged, handapply!) excerpt of that fix below made
i386 compile here.

Karsten

Index: linux/mm/page_alloc.c
===================================================================
--- linux.orig/mm/page_alloc.c
+++ linux/mm/page_alloc.c
@@ -2768,7 +2768,6 @@ void __init free_area_init(unsigned long
__pa(PAGE_OFFSET) >> PAGE_SHIFT, NULL);
}

-#ifdef CONFIG_HOTPLUG_CPU
static int page_alloc_cpu_notify(struct notifier_block *self,
unsigned long action, void *hcpu)
{
@@ -2786,7 +2785,6 @@ static int page_alloc_cpu_notify(struct
}
return NOTIFY_OK;
}
-#endif /* CONFIG_HOTPLUG_CPU */

void __init page_alloc_init(void)
{
-

2006-11-22 11:38:00

by Robert Schwebel

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

On Mon, Nov 20, 2006 at 11:02:30PM +0100, Ingo Molnar wrote:
> i've released the 2.6.19-rc6-rt5 tree, which can be downloaded from the
> usual place:

[...]

> as usual, bugreports, fixes and suggestions are welcome,

I have problems runinng cyclictest on the more recent -rt kernels. A
similar problem I reported recently on a Pentium M / ICH6 Dell box is
solved by using the acpi_pm timer.

The problematic box is a 700 MHz Celeron (Coppermine) which doesn't have
ACPI.

The last kernel that run with sane cyclictest results was 2.6.18-rt6:

- root@krachkiste:~/cyclictest-v0.11 ./cyclictest -n -t 4 -p 90 -l 10000
0.13 0.10 0.13 1/52 2721

T: 0 ( 2718) P:90 I: 1000 C: 10000 Min: 14 Act: 47 Avg: 52 Max: 170
T: 1 ( 2719) P:89 I: 1500 C: 6667 Min: 16 Act: 22 Avg: 45 Max: 138
T: 2 ( 2720) P:88 I: 2000 C: 5001 Min: 16 Act: 35 Avg: 44 Max: 101
T: 3 ( 2721) P:87 I: 2500 C: 4001 Min: 14 Act: 28 Avg: 38 Max: 114

The following numbers have been taken on 2.6.19-rc6-rt5, the effect is there
since 2.6.18-rt7.

- "cyclictest -n -p 90 -t 4" is just "running upwards".

root@krachkiste:~/cyclictest-v0.11 ./cyclictest -n -p 90 -t 4 -l 1000
0.00 0.00 0.00 1/54 2843

T: 0 ( 2840) P:90 I: 1000 C: 1000 Min: 7294 Act: 3008486 Avg: 1508852 Max: 3008486
T: 1 ( 2841) P:89 I: 1500 C: 1000 Min: 7195 Act: 2508885 Avg: 1258994 Max: 2508885
T: 2 ( 2842) P:88 I: 2000 C: 1000 Min: 7188 Act: 2009375 Avg: 1009235 Max: 2009375
T: 3 ( 2843) P:87 I: 2500 C: 1000 Min: 7184 Act: 1509868 Avg: 759479 Max: 1509868

- Using a relative timer (-r), I get huge latencies:

root@krachkiste:~/cyclictest-v0.11 ./cyclictest -n -p 90 -t 4 -r -l 1000
0.00 0.00 0.00 1/56 2838

T: 0 ( 2835) P:90 I: 1000 C: 1000 Min: 742 Act: 6937 Avg: 6952 Max: 9214
T: 1 ( 2836) P:89 I: 1500 C: 1000 Min: 244 Act: 6424 Avg: 6454 Max: 8718
T: 2 ( 2837) P:88 I: 2000 C: 999 Min: 1288 Act: 5910 Avg: 5962 Max: 8216
T: 3 ( 2838) P:87 I: 2500 C: 999 Min: 778 Act: 5409 Avg: 5462 Max: 7713

This is with "lapic lapictimer" on the kernel commandline.

The system has chosen "pit" as it's clocksource, switching to tsc or jiffies
doesn't change anything.

With 2.6.18-rt6 I've seen these lines in the dmesg output:

Nov 22 12:05:24 krachkiste kernel: Time: tsc clocksource has been installed.
Nov 22 12:05:24 krachkiste kernel: Event source pit disabled
Nov 22 12:05:24 krachkiste kernel: Event source lapic configured with caps set: 08
Nov 22 12:05:24 krachkiste kernel: hrtimers: Switched to high resolution mode CPU 0

Especially the "Switched to high resolution mode" isn't there with the later
kernels (high resolution timers is on in the config, dyntick is off).

Robert
--
Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
Pengutronix - Linux Solutions for Science and Industry
Handelsregister: Amtsgericht Hildesheim, HRA 2686
Hannoversche Str. 2, 31134 Hildesheim, Germany
Phone: +49-5121-206917-0 | Fax: +49-5121-206917-9

2006-11-22 14:06:46

by Mark Knecht

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

On 11/20/06, Ingo Molnar <[email protected]> wrote:
> i've released the 2.6.19-rc6-rt5 tree, which can be downloaded from the
> usual place:
>
> http://redhat.com/~mingo/realtime-preempt/
>
<SNIP>
>
> as usual, bugreports, fixes and suggestions are welcome,
>
> Ingo

Ingo,
I started building the new kernels a few days ago with your
2.6.19-rc6-rt0 announcement. The kernels have built fine but so far I
am unable to build the realtime-lsm package against them so no reason
to reboot.

I know there were some comments awhile back about being required to
switch to PAM. Has that occurred?

If not then there is a regression issue for realtime-lsm.

Thanks,
Mark

2006-11-23 20:43:10

by Eduardo Valentin

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

Hello Robert and Ingo,


I'm beginning to play with -rt patches. I started by using the
2.6.19-rc6-rt5, after reading the rt-wiki
(http://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO). I applied the
last two compile corrections patches. However, I got a similar problem
as reported by Robert:

2.6.19-rc6:
T: 0 ( 4861) P:80 I: 10000 C: 10000 Min: 2592 Act: 4878 Avg:
6137 Max: 10652
2.6.19-rc6-rt5:
T: 0 ( 3661) P:80 I: 10000 C: 10000 Min: 828 Act: 1698 Avg:
3291 Max: 7171
These results are quite different from what is reported at the wiki.

I'm also attaching my .config and here is a description of my cpu:
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Xeon(TM) CPU 3.40GHz
stepping : 1
cpu MHz : 3391.745
cache size : 1024 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 3
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx
lm constant_tsc up pni monitor ds_cpl est cid cx16 xtpr
bogomips : 6788.16



BR,


Eduardo Valentin

On 11/22/06, Robert Schwebel <[email protected]> wrote:
> On Mon, Nov 20, 2006 at 11:02:30PM +0100, Ingo Molnar wrote:
> > i've released the 2.6.19-rc6-rt5 tree, which can be downloaded from the
> > usual place:
>
> [...]
>
> > as usual, bugreports, fixes and suggestions are welcome,
>
> I have problems runinng cyclictest on the more recent -rt kernels. A
> similar problem I reported recently on a Pentium M / ICH6 Dell box is
> solved by using the acpi_pm timer.
>
> The problematic box is a 700 MHz Celeron (Coppermine) which doesn't have
> ACPI.
>
> The last kernel that run with sane cyclictest results was 2.6.18-rt6:
>
> - root@krachkiste:~/cyclictest-v0.11 ./cyclictest -n -t 4 -p 90 -l 10000
> 0.13 0.10 0.13 1/52 2721
>
> T: 0 ( 2718) P:90 I: 1000 C: 10000 Min: 14 Act: 47 Avg: 52 Max: 170
> T: 1 ( 2719) P:89 I: 1500 C: 6667 Min: 16 Act: 22 Avg: 45 Max: 138
> T: 2 ( 2720) P:88 I: 2000 C: 5001 Min: 16 Act: 35 Avg: 44 Max: 101
> T: 3 ( 2721) P:87 I: 2500 C: 4001 Min: 14 Act: 28 Avg: 38 Max: 114
>
> The following numbers have been taken on 2.6.19-rc6-rt5, the effect is there
> since 2.6.18-rt7.
>
> - "cyclictest -n -p 90 -t 4" is just "running upwards".
>
> root@krachkiste:~/cyclictest-v0.11 ./cyclictest -n -p 90 -t 4 -l 1000
> 0.00 0.00 0.00 1/54 2843
>
> T: 0 ( 2840) P:90 I: 1000 C: 1000 Min: 7294 Act: 3008486 Avg: 1508852 Max: 3008486
> T: 1 ( 2841) P:89 I: 1500 C: 1000 Min: 7195 Act: 2508885 Avg: 1258994 Max: 2508885
> T: 2 ( 2842) P:88 I: 2000 C: 1000 Min: 7188 Act: 2009375 Avg: 1009235 Max: 2009375
> T: 3 ( 2843) P:87 I: 2500 C: 1000 Min: 7184 Act: 1509868 Avg: 759479 Max: 1509868
>
> - Using a relative timer (-r), I get huge latencies:
>
> root@krachkiste:~/cyclictest-v0.11 ./cyclictest -n -p 90 -t 4 -r -l 1000
> 0.00 0.00 0.00 1/56 2838
>
> T: 0 ( 2835) P:90 I: 1000 C: 1000 Min: 742 Act: 6937 Avg: 6952 Max: 9214
> T: 1 ( 2836) P:89 I: 1500 C: 1000 Min: 244 Act: 6424 Avg: 6454 Max: 8718
> T: 2 ( 2837) P:88 I: 2000 C: 999 Min: 1288 Act: 5910 Avg: 5962 Max: 8216
> T: 3 ( 2838) P:87 I: 2500 C: 999 Min: 778 Act: 5409 Avg: 5462 Max: 7713
>
> This is with "lapic lapictimer" on the kernel commandline.
>
> The system has chosen "pit" as it's clocksource, switching to tsc or jiffies
> doesn't change anything.
>
> With 2.6.18-rt6 I've seen these lines in the dmesg output:
>
> Nov 22 12:05:24 krachkiste kernel: Time: tsc clocksource has been installed.
> Nov 22 12:05:24 krachkiste kernel: Event source pit disabled
> Nov 22 12:05:24 krachkiste kernel: Event source lapic configured with caps set: 08
> Nov 22 12:05:24 krachkiste kernel: hrtimers: Switched to high resolution mode CPU 0
>
> Especially the "Switched to high resolution mode" isn't there with the later
> kernels (high resolution timers is on in the config, dyntick is off).
>
> Robert
> --
> Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
> Pengutronix - Linux Solutions for Science and Industry
> Handelsregister: Amtsgericht Hildesheim, HRA 2686
> Hannoversche Str. 2, 31134 Hildesheim, Germany
> Phone: +49-5121-206917-0 | Fax: +49-5121-206917-9
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>


--
Eduardo Bezerra Valentin


Attachments:
(No filename) (4.88 kB)
config (53.55 kB)
Download all attachments

2006-11-23 21:11:48

by Robert Schwebel

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

Eduardo,

On Thu, Nov 23, 2006 at 04:43:01PM -0400, Eduardo Valentin wrote:
> However, I got a similar problem as reported by Robert:
>
> 2.6.19-rc6:
> T: 0 ( 4861) P:80 I: 10000 C: 10000 Min: 2592 Act: 4878 Avg: 6137 Max: 10652
> 2.6.19-rc6-rt5:
> T: 0 ( 3661) P:80 I: 10000 C: 10000 Min: 828 Act: 1698 Avg: 3291 Max: 7171
>
> These results are quite different from what is reported at the wiki.

Is this with -r or without? I didn't get fixed maxima at all on the
celeron box. It also looks like you've changed the interval time to
10ms; nevertheless: a delay of > 10 ms @ a cylce time of 10 ms means
missing the deadlines.

Do you have the possibility to switch off the SMI for your chipset? As
tglx noted on irc recently this may be dangerous because thermal
throtteling can be implemented via SMI, but to get an idea if the SMI is
the reason for the delays, switching it off for a short time should do
the job.

Robert
--
Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
Pengutronix - Linux Solutions for Science and Industry
Handelsregister: Amtsgericht Hildesheim, HRA 2686
Hannoversche Str. 2, 31134 Hildesheim, Germany
Phone: +49-5121-206917-0 | Fax: +49-5121-206917-9

2006-11-25 22:05:24

by Thomas Kuther

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

Something is really wrong with page alloc on this one. Compiled 2.6.19-rc6-rt5
with the one patch to page_alloc.c as posted on the list here.

Kernel uses around 50% mem and 30% swap without doing anything.
I get a lot of these:

X invoked oom-killer: gfp_mask=0xd0, order=0, oomkilladj=0
[<c0148426>] out_of_memory+0x176/0x1d0
[<c0149dc6>] __alloc_pages+0x286/0x2f0
[<c0149e76>] __get_free_pages+0x46/0x60
[<c01749e0>] __pollwait+0xb0/0x100
[<c04e92f6>] unix_poll+0xc6/0xd0
[<c0477c03>] sock_poll+0x23/0x30
[<c0174038>] do_select+0x288/0x4c0
[<c0174930>] __pollwait+0x0/0x100
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0174493>] core_sys_select+0x223/0x360
[<c04eed09>] __schedule+0x2e9/0x6b0
[<c0108732>] convert_fxsr_from_user+0x22/0xf0
[<c0174b2f>] sys_select+0xff/0x1e0
[<c0121e1b>] sys_gettimeofday+0x3b/0x90
[<c01030e5>] sysenter_past_esp+0x56/0x79
=======================
Mem-info:
DMA per-cpu:
CPU 0: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
Normal per-cpu:
CPU 0: Hot: hi: 186, btch: 31 usd: 31 Cold: hi: 62, btch: 15 usd: 58
HighMem per-cpu:
CPU 0: Hot: hi: 186, btch: 31 usd: 66 Cold: hi: 62, btch: 15 usd: 14
Active:111463 inactive:36109 dirty:0 writeback:0 unstable:0 free:4018
slab:163934 mapped:26114 pagetables:874
DMA free:3560kB min:68kB low:84kB high:100kB active:396kB inactive:356kB
present:16256kB pages_scanned:1370 all_unreclaimable? yes
lowmem_reserve[]: 0 873 1254
Normal free:3720kB min:3744kB low:4680kB high:5616kB active:111304kB
inactive:108296kB present:894080kB pages_scanned:339028 all_unreclaimable? yes
lowmem_reserve[]: 0 0 3047
HighMem free:8792kB min:380kB low:788kB high:1196kB active:334152kB
inactive:35784kB present:390084kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB
0*4096kB = 3560kB
Normal: 0*4kB 5*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB
1*2048kB 0*4096kB = 3720kB
HighMem: 924*4kB 517*8kB 40*16kB 2*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB
0*2048kB 0*4096kB = 8792kB
Swap cache: add 107141, delete 56933, find 4493/5856, race 0+0
Free swap = 113440kB
Total swap = 488336kB
Free swap: 113440kB
327664 pages of RAM
98288 pages of HIGHMEM
4383 reserved pages
94253 pages shared
50208 pages swap cached
0 pages dirty
0 pages writeback
26114 pages mapped
163934 pages slab
874 pages pagetables
327664 pages of RAM
98288 pages of HIGHMEM
4383 reserved pages
94253 pages shared
50208 pages swap cached
0 pages dirty
0 pages writeback
26114 pages mapped
163934 pages slab
874 pages pagetables
audacious invoked oom-killer: gfp_mask=0xd0, order=0, oomkilladj=0
[<c0148426>] out_of_memory+0x176/0x1d0
[<c0149dc6>] __alloc_pages+0x286/0x2f0
[<c016205e>] cache_alloc_refill+0x30e/0x5d0
[<c0161d47>] kmem_cache_alloc+0x57/0x60
[<c0477d29>] sock_alloc_inode+0x19/0x60
[<c017b219>] alloc_inode+0x19/0x190
[<c0166d45>] fget_light+0x85/0xa0
[<c017c066>] new_inode+0x16/0x90
[<c0478bf4>] sock_alloc+0x14/0x70
[<c047a446>] sys_accept+0x56/0x270
[<c0102a12>] do_notify_resume+0x402/0x750
[<c0108732>] convert_fxsr_from_user+0x22/0xf0
[<c047a731>] sys_socketcall+0xd1/0x280
[<c01030e5>] sysenter_past_esp+0x56/0x79
=======================
Mem-info:
DMA per-cpu:
CPU 0: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
Normal per-cpu:
CPU 0: Hot: hi: 186, btch: 31 usd: 31 Cold: hi: 62, btch: 15 usd: 58
HighMem per-cpu:
CPU 0: Hot: hi: 186, btch: 31 usd: 66 Cold: hi: 62, btch: 15 usd: 14
Active:111494 inactive:36078 dirty:0 writeback:0 unstable:0 free:4018
slab:163934 mapped:26114 pagetables:874
DMA free:3560kB min:68kB low:84kB high:100kB active:396kB inactive:356kB
present:16256kB pages_scanned:1370 all_unreclaimable? yes
lowmem_reserve[]: 0 873 1254
Normal free:3720kB min:3744kB low:4680kB high:5616kB active:111420kB
inactive:108180kB present:894080kB pages_scanned:339127 all_unreclaimable? yes
lowmem_reserve[]: 0 0 3047
HighMem free:8792kB min:380kB low:788kB high:1196kB active:334160kB
inactive:35776kB present:390084kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 0*4kB 1*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB
0*4096kB = 3560kB
Normal: 0*4kB 5*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB
1*2048kB 0*4096kB = 3720kB
HighMem: 924*4kB 517*8kB 40*16kB 2*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB
0*2048kB 0*4096kB = 8792kB
Swap cache: add 107141, delete 56933, find 4493/5856, race 0+0
Free swap = 113440kB
Total swap = 488336kB
Free swap: 113440kB
X invoked oom-killer: gfp_mask=0xd0, order=0, oomkilladj=0
[<c0148426>] out_of_memory+0x176/0x1d0
[<c0149dc6>] __alloc_pages+0x286/0x2f0
[<c0149e76>] __get_free_pages+0x46/0x60
[<c01749e0>] __pollwait+0xb0/0x100
[<c04e92f6>] unix_poll+0xc6/0xd0
[<c0477c03>] sock_poll+0x23/0x30
[<c0174038>] do_select+0x288/0x4c0
[<c0174930>] __pollwait+0x0/0x100
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0118db0>] default_wake_function+0x0/0x20
[<c0174493>] core_sys_select+0x223/0x360
[<c04eed09>] __schedule+0x2e9/0x6b0
[<c0108732>] convert_fxsr_from_user+0x22/0xf0
[<c0174b2f>] sys_select+0xff/0x1e0
[<c0121e1b>] sys_gettimeofday+0x3b/0x90
[<c01030e5>] sysenter_past_esp+0x56/0x79
=======================
...
...
Out of Memory: Kill process 13677 (kdeinit) score 320232 and children.
Out of memory: Killed process 13691 (kio_file).
Out of Memory: Kill process 13677 (kdeinit) score 317057 and children.
Out of memory: Killed process 13727 (gxine).
Out of Memory: Kill process 11641 (mozilla-launche) score 184632 and children.
Out of memory: Killed process 11650 (firefox-bin).
Out of Memory: Kill process 9293 (apache2) score 102567 and children.
Out of memory: Killed process 9294 (apache2).
Out of Memory: Kill process 9293 (apache2) score 102420 and children.
Out of memory: Killed process 9389 (apache2).




2006-11-26 14:39:40

by Karsten Wiese

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

> i've released the 2.6.19-rc6-rt5 tree, which can be downloaded from the

Hi

this fixes issues like rmmod hanging and inodes leaking.

Karsten

--- fs/dcache.c~ 2006-11-21 11:25:11.000000000 +0100
+++ fs/dcache.c 2006-11-26 15:20:31.000000000 +0100
@@ -150,7 +150,7 @@ void dput(struct dentry *dentry)
repeat:
if (atomic_read(&dentry->d_count) == 1)
might_sleep();
- if (atomic_dec_and_test(&dentry->d_count))
+ if (!atomic_dec_and_test(&dentry->d_count))
return;

spin_lock(&dentry->d_lock);

2006-11-28 08:42:14

by Ingo Molnar

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5


* Karsten Wiese <[email protected]> wrote:

> this fixes issues like rmmod hanging and inodes leaking.

thanks ... i have reverted the other dcache.c changes as well.

Ingo

2006-11-28 17:33:05

by Lee Revell

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

On Wed, 2006-11-22 at 06:06 -0800, Mark Knecht wrote:
> Ingo,
> I started building the new kernels a few days ago with your
> 2.6.19-rc6-rt0 announcement. The kernels have built fine but so far I
> am unable to build the realtime-lsm package against them so no reason
> to reboot.
>
> I know there were some comments awhile back about being required to
> switch to PAM. Has that occurred?
>
> If not then there is a regression issue for realtime-lsm.

As Realtime LSM is an out of tree module and there's no stable kernel
module API it's impossible to prevent regressions.

That being said, the realtime LSM patch is so simple that it should work
- how exactly does it fail?

Lee

2006-11-28 19:53:12

by Mark Knecht

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

On 11/28/06, Lee Revell <[email protected]> wrote:
> On Wed, 2006-11-22 at 06:06 -0800, Mark Knecht wrote:
> > Ingo,
> > I started building the new kernels a few days ago with your
> > 2.6.19-rc6-rt0 announcement. The kernels have built fine but so far I
> > am unable to build the realtime-lsm package against them so no reason
> > to reboot.
> >
> > I know there were some comments awhile back about being required to
> > switch to PAM. Has that occurred?
> >
> > If not then there is a regression issue for realtime-lsm.
>
> As Realtime LSM is an out of tree module and there's no stable kernel
> module API it's impossible to prevent regressions.
>
> That being said, the realtime LSM patch is so simple that it should work
> - how exactly does it fail?
>
> Lee

Hi Lee,
The failure is a Gentoo sandbax failure. On the surface of it I
didn't really think it was a kernel problem but I know you've pushed
me to move to PAM telling me realtime-lsm wasn't going to work in the
future. I really just wanted to know that PAM was now a requirement
instead of only best practice.

Thanks,
Mark

2006-11-28 20:15:37

by Lee Revell

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

On Tue, 2006-11-28 at 11:53 -0800, Mark Knecht wrote:
> I know you've pushed
> me to move to PAM telling me realtime-lsm wasn't going to work in the
> future. I really just wanted to know that PAM was now a requirement
> instead of only best practice.

I said it was not guaranteed to work. It should work as long as someone
maintains it.

I don't think anyone expected it to take so long for distros to update
their PAM packages so as to make patching the kernel unnecessary.

Lee

2006-11-28 20:17:19

by Ingo Molnar

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5


* Lee Revell <[email protected]> wrote:

> > I know there were some comments awhile back about being required
> > to switch to PAM. Has that occurred?
> >
> > If not then there is a regression issue for realtime-lsm.
>
> As Realtime LSM is an out of tree module and there's no stable kernel
> module API it's impossible to prevent regressions.
>
> That being said, the realtime LSM patch is so simple that it should
> work - how exactly does it fail?

i can include it in -rt if it's simple enough - if someone's interested
then please send me a patch.

Ingo

2006-11-28 22:52:17

by Mark Knecht

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5

Forwarding it off list.

Thanks Ingo. I'm very interested if it works for you to do this.

Cheers,
Mark

On 11/28/06, Ingo Molnar <[email protected]> wrote:
>
> * Lee Revell <[email protected]> wrote:
>
> > > I know there were some comments awhile back about being required
> > > to switch to PAM. Has that occurred?
> > >
> > > If not then there is a regression issue for realtime-lsm.
> >
> > As Realtime LSM is an out of tree module and there's no stable kernel
> > module API it's impossible to prevent regressions.
> >
> > That being said, the realtime LSM patch is so simple that it should
> > work - how exactly does it fail?
>
> i can include it in -rt if it's simple enough - if someone's interested
> then please send me a patch.
>
> Ingo
>

2006-11-29 06:57:33

by Ingo Molnar

[permalink] [raw]
Subject: Re: 2.6.19-rc6-rt5


* Mark Knecht <[email protected]> wrote:

> Forwarding it off list.
>
> Thanks Ingo. I'm very interested if it works for you to do this.

i've integrated it into -rt (see the patch below), but i marked it
obsolete and i might not be able to carry it for long - we'll see. The
preferred solution is to use newer PAM and its rt-limits features. But
to ease migration i'll keep the realtime-lsm for a while.

Ingo


Attachments:
(No filename) (420.00 B)
realtime-lsm.patch (5.98 kB)
Download all attachments