We are pleased to announce the 2.6.24.2-rt2 tree, which can be
downloaded from the location:
http://rt.et.redhat.com/download/
Information on the RT patch can be found at:
http://rt.wiki.kernel.org/index.php/Main_Page
Changes since 2.6.24-rt1
- ported to 2.6.24.2
- *** New ftrace utility ***
The old latency_tracer has now been replaced with the cleaned up
version that is being prepared for mainline.
- compiler warning fix (Shi Weihua)
to build a 2.6.24.2-rt2 tree, the following patches should be applied:
http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.24.tar.bz2
http://kernel.org/pub/linux/kernel/v2.6/patch-2.6.24.2.bz2
http://rt.et.redhat.com/download/patch-2.6.24.2-rt2.bz2
And like always, my RT version of Matt Mackall's ketchup will get this
for you nicely:
http://people.redhat.com/srostedt/rt/tools/ketchup-0.9.8-rt3
The broken out patches are also available.
-- Steve
Steven Rostedt wrote:
> We are pleased to announce the 2.6.24.2-rt2 tree, which can be
> downloaded from the location:
>
> http://rt.et.redhat.com/download/
>
> Information on the RT patch can be found at:
>
> http://rt.wiki.kernel.org/index.php/Main_Page
>
> Changes since 2.6.24-rt1
>
> - ported to 2.6.24.2
>
> - *** New ftrace utility ***
> The old latency_tracer has now been replaced with the cleaned up
> version that is being prepared for mainline.
>
> - compiler warning fix (Shi Weihua)
This important fix is still missing:
http://lkml.org/lkml/2008/2/5/295
At this chance: We still see the same unbalanced sched-other load on our
NUMA box as Gernot once reported [1]:
top - 11:19:20 up 4 min, 1 user, load average: 29.52, 9.54, 3.37
Tasks: 502 total, 41 running, 461 sleeping, 0 stopped, 0 zombie
Cpu0 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu4 : 0.0%us, 0.3%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu5 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu6 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu7 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu9 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu10 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu11 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu12 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu13 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu14 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu15 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 65513284k total, 1032032k used, 64481252k free, 6444k buffers
Swap: 3204896k total, 0k used, 3204896k free, 37312k cached
PR PID NI VIRT SHR S %CPU %MEM TIME+ COMMAND
20 5603 0 705m 464 R 100 0.3 1:18.19 load-balance-te
20 5604 0 705m 464 R 100 0.3 1:18.16 load-balance-te
20 5605 0 705m 464 R 100 0.3 1:18.18 load-balance-te
20 5608 0 705m 464 R 100 0.3 1:18.18 load-balance-te
20 5611 0 705m 464 R 25 0.3 0:19.58 load-balance-te
20 5620 0 705m 464 R 25 0.3 0:19.54 load-balance-te
20 5606 0 705m 464 R 25 0.3 0:19.56 load-balance-te
20 5616 0 705m 464 R 25 0.3 0:19.54 load-balance-te
20 5607 0 705m 464 R 20 0.3 0:15.64 load-balance-te
20 5609 0 705m 464 R 20 0.3 0:15.66 load-balance-te
20 5614 0 705m 464 R 20 0.3 0:15.68 load-balance-te
20 5617 0 705m 464 R 20 0.3 0:15.64 load-balance-te
20 5619 0 705m 464 R 20 0.3 0:15.64 load-balance-te
20 5610 0 705m 464 R 17 0.3 0:13.10 load-balance-te
20 5618 0 705m 464 R 17 0.3 0:13.04 load-balance-te
20 5621 0 705m 464 R 17 0.3 0:13.02 load-balance-te
20 5622 0 705m 464 R 17 0.3 0:13.02 load-balance-te
20 5623 0 705m 464 R 17 0.3 0:13.06 load-balance-te
20 5624 0 705m 464 R 17 0.3 0:13.02 load-balance-te
20 5615 0 705m 464 R 5 0.3 0:03.76 load-balance-te
20 5633 0 705m 464 R 5 0.3 0:03.72 load-balance-te
20 5634 0 705m 464 R 5 0.3 0:03.82 load-balance-te
20 5635 0 705m 464 R 5 0.3 0:03.74 load-balance-te
20 5636 0 705m 464 R 5 0.3 0:03.72 load-balance-te
20 5638 0 705m 464 R 5 0.3 0:03.72 load-balance-te
20 5640 0 705m 464 R 5 0.3 0:03.72 load-balance-te
20 5612 0 705m 464 R 5 0.3 0:03.78 load-balance-te
20 5613 0 705m 464 R 5 0.3 0:03.78 load-balance-te
20 5625 0 705m 464 R 5 0.3 0:03.74 load-balance-te
20 5626 0 705m 464 R 5 0.3 0:03.74 load-balance-te
20 5627 0 705m 464 R 5 0.3 0:03.82 load-balance-te
20 5628 0 705m 464 R 5 0.3 0:03.72 load-balance-te
20 5629 0 705m 464 R 5 0.3 0:03.72 load-balance-te
20 5630 0 705m 464 R 5 0.3 0:03.72 load-balance-te
20 5631 0 705m 464 R 5 0.3 0:03.72 load-balance-te
20 5632 0 705m 464 R 5 0.3 0:03.72 load-balance-te
20 5637 0 705m 464 R 5 0.3 0:03.72 load-balance-te
20 5639 0 705m 464 R 5 0.3 0:03.74 load-balance-te
20 5641 0 705m 464 R 5 0.3 0:03.70 load-balance-te
20 5642 0 705m 464 R 5 0.3 0:03.80 load-balance-te
I just accidentally compiled the kernel with NR_CPUS=8 first, and then
the load was balanced. Weird.
Some time ago I tried to dig into this but didn't get far due to the
limited bisectability of -rt (was at least true for .23-rtX) and limited
time. But based on this search, I suspect now that the issue is
introduced via some patch somewhere at the end of the series. If anyone
has a good pointer what to check, I would try to look into this again.
Jan
[1] http://thread.gmane.org/gmane.linux.rt.user/1640/focus=1746
--
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
Jan Kiszka wrote:
> At this chance: We still see the same unbalanced sched-other load on our
> NUMA box as Gernot once reported [1]:
>
> top - 11:19:20 up 4 min, 1 user, load average: 29.52, 9.54, 3.37
> Tasks: 502 total, 41 running, 461 sleeping, 0 stopped, 0 zombie
> Cpu0 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu1 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu4 : 0.0%us, 0.3%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu5 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu6 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu7 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu9 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu10 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu11 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu12 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu13 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu14 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu15 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Mem: 65513284k total, 1032032k used, 64481252k free, 6444k buffers
> Swap: 3204896k total, 0k used, 3204896k free, 37312k cached
>
ETOOMANYKERNELS, this was from 2.6.23.12-rt14. 2.6.24.2-rt2 shows a
different patter under identical load:
top - 12:55:27 up 2 min, 1 user, load average: 9.97, 2.42, 0.81
Tasks: 491 total, 42 running, 449 sleeping, 0 stopped, 0 zombie
Cpu0 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 99.7%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st
Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu4 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu5 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu6 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu7 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu9 : 0.3%us, 0.3%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu10 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu11 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu12 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu13 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu14 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu15 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 65512480k total, 580704k used, 64931776k free, 8964k buffers
Swap: 3204896k total, 0k used, 3204896k free, 129720k cached
Jan
--
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
On Tue, 26 Feb 2008, Jan Kiszka wrote:
> Jan Kiszka wrote:
> > At this chance: We still see the same unbalanced sched-other load on our
> > NUMA box as Gernot once reported [1]:
> >
> > top - 11:19:20 up 4 min, 1 user, load average: 29.52, 9.54, 3.37
> > Tasks: 502 total, 41 running, 461 sleeping, 0 stopped, 0 zombie
> > Cpu0 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu1 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu4 : 0.0%us, 0.3%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu5 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu6 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu7 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu9 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu10 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu11 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu12 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu13 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu14 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Cpu15 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> > Mem: 65513284k total, 1032032k used, 64481252k free, 6444k buffers
> > Swap: 3204896k total, 0k used, 3204896k free, 37312k cached
> >
>
> ETOOMANYKERNELS, this was from 2.6.23.12-rt14. 2.6.24.2-rt2 shows a
> different patter under identical load:
There has been CFS updates, which may account for the differences. Seems
better though.
>
> top - 12:55:27 up 2 min, 1 user, load average: 9.97, 2.42, 0.81
> Tasks: 491 total, 42 running, 449 sleeping, 0 stopped, 0 zombie
> Cpu0 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu1 : 99.7%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st
> Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu4 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu5 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu6 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu7 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu9 : 0.3%us, 0.3%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu10 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu11 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu12 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu13 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu14 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu15 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Mem: 65512480k total, 580704k used, 64931776k free, 8964k buffers
> Swap: 3204896k total, 0k used, 3204896k free, 129720k cached
>
What's the NUMA topology? What tasks are running, and at what priorities?
Those three idle CPUS, should they have tasks running on them?
-- Steve
Steven Rostedt wrote:
>
> On Tue, 26 Feb 2008, Jan Kiszka wrote:
>
>> Jan Kiszka wrote:
>>> At this chance: We still see the same unbalanced sched-other load on our
>>> NUMA box as Gernot once reported [1]:
>>>
>>> top - 11:19:20 up 4 min, 1 user, load average: 29.52, 9.54, 3.37
>>> Tasks: 502 total, 41 running, 461 sleeping, 0 stopped, 0 zombie
>>> Cpu0 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu1 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu4 : 0.0%us, 0.3%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu5 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu6 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu7 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu9 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu10 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu11 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu12 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu13 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu14 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu15 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Mem: 65513284k total, 1032032k used, 64481252k free, 6444k buffers
>>> Swap: 3204896k total, 0k used, 3204896k free, 37312k cached
>>>
>> ETOOMANYKERNELS, this was from 2.6.23.12-rt14. 2.6.24.2-rt2 shows a
>> different patter under identical load:
>
> There has been CFS updates, which may account for the differences. Seems
> better though.
>
>> top - 12:55:27 up 2 min, 1 user, load average: 9.97, 2.42, 0.81
>> Tasks: 491 total, 42 running, 449 sleeping, 0 stopped, 0 zombie
>> Cpu0 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu1 : 99.7%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st
>> Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu4 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu5 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu6 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu7 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu9 : 0.3%us, 0.3%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu10 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu11 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu12 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu13 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu14 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu15 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>> Mem: 65512480k total, 580704k used, 64931776k free, 8964k buffers
>> Swap: 3204896k total, 0k used, 3204896k free, 129720k cached
>>
>
> What's the NUMA topology?
4 nodes. I'm not sure if it is really NUMA related, but the same kernel
runs that test as expected on a non-NUMA 2x2 box.
> What tasks are running, and at what priorities?
40 pthreads, created with default parameters from a main thread which
runs with default parameters as well. The threads simply run endless loops.
>
> Those three idle CPUS, should they have tasks running on them?
For sure, given the overload situation of the system (40x full load vs.
16 cores). Neither did we fiddle with any parameter of the system
(knowingly, its a standard openSUSE 10.3 underneath) nor did we set
thread affinities.
Jan
--
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
On Tue, 26 Feb 2008, Jan Kiszka wrote:
> >>
> >
> > What's the NUMA topology?
>
> 4 nodes. I'm not sure if it is really NUMA related, but the same kernel
> runs that test as expected on a non-NUMA 2x2 box.
>
> > What tasks are running, and at what priorities?
>
> 40 pthreads, created with default parameters from a main thread which
> runs with default parameters as well. The threads simply run endless loops.
>
> >
> > Those three idle CPUS, should they have tasks running on them?
>
> For sure, given the overload situation of the system (40x full load vs.
> 16 cores). Neither did we fiddle with any parameter of the system
> (knowingly, its a standard openSUSE 10.3 underneath) nor did we set
> thread affinities.
>
Do you get different behaviour with 2.6.24.2?
-- Steve
Steven Rostedt wrote:
>
> On Tue, 26 Feb 2008, Jan Kiszka wrote:
>>> What's the NUMA topology?
>> 4 nodes. I'm not sure if it is really NUMA related, but the same kernel
>> runs that test as expected on a non-NUMA 2x2 box.
>>
>>> What tasks are running, and at what priorities?
>> 40 pthreads, created with default parameters from a main thread which
>> runs with default parameters as well. The threads simply run endless loops.
>>
>>> Those three idle CPUS, should they have tasks running on them?
>> For sure, given the overload situation of the system (40x full load vs.
>> 16 cores). Neither did we fiddle with any parameter of the system
>> (knowingly, its a standard openSUSE 10.3 underneath) nor did we set
>> thread affinities.
>>
>
> Do you get different behaviour with 2.6.24.2?
Last time I checked mainline (I think 2.6.24), it was fine. It was
definitely fine for 2.6.23. But I'm going to revalidate this once the
machine is free again (tomorrow).
Jan
--
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux