Con, Ingo, List,
I'm overjoyed with decent ht-aware scheduling in 2.6.7-rc2 and it does
mostly the right thing. However, the 'nice' work by Con shows some slight
problems.
Please find attached program 'eat-time.cc'. Make sure not to compile it with
-O which might confuse things as this program basically does nothing.
Run it without arguments to determine the speed of 1 cpu, it outputs a
number (megaloops/second). Then start it with that number as a parameter:
Sample:
$ ./eat-time
592
$ ./eat-time 592
99%
99%
100%
etc
Now starting four of these at the same time gives the desired result:
$ ./eat-time 592 & ./eat-time 592 & ./eat-time 592 & ./eat-time 592
50%
50%
50%
50%
etc
This however:
$ ./eat-time 592 & ./eat-time 592 &
100%
99%
In another xterm:
$ nice -n +19 ./eat-time 592 & nice -n +19 ./eat-time 592
5%
5%
5%
Fails sometimes, with all processes getting 50%. The above 'screenshot' is
from the working and expected situation, which happens most of the time.
When it goes wrong, top shows me that Cpu0 and Cpu1 are 100% user, while
Cpu2 and Cpu3 are both 100% nice. The niced processes show up in top as
PRiority 39, the unniced ones (NI = 0) as PR 25.
I've also seen it that Cpu2 and Cpu3 are 100% busy, and 0 and 1 are 100%
nice.
I'd say this situation happens once every 5 or 8 invocations, and perhaps
somewhat more when first starting the niced processes.
Perhaps related, when running the above 'nice -n +19' line on its own, I see
all CPUs getting load over time, the two processes are wandering. After a
while they settle down, only to go on wandering again some time later, also
touching configurations where a physical cpu suddenly hosts two processes.
Without nice, two processes get firmly pegged to different physical CPUs.
Anything I can do to help resolve this, just yell.
Thanks!
--
http://www.PowerDNS.com Open source, database driven DNS Software
http://lartc.org Linux Advanced Routing & Traffic Control HOWTO
On Mon, 7 Jun 2004 18:56, bert hubert wrote:
> Con, Ingo, List,
>
> I'm overjoyed with decent ht-aware scheduling in 2.6.7-rc2 and it does
> mostly the right thing. However, the 'nice' work by Con shows some slight
> problems.
>
> Please find attached program 'eat-time.cc'. Make sure not to compile it
> with -O which might confuse things as this program basically does nothing.
>
> Run it without arguments to determine the speed of 1 cpu, it outputs a
> number (megaloops/second). Then start it with that number as a parameter:
>
> Sample:
>
> $ ./eat-time
> 592
> $ ./eat-time 592
> 99%
> 99%
> 100%
> etc
>
> Now starting four of these at the same time gives the desired result:
>
> $ ./eat-time 592 & ./eat-time 592 & ./eat-time 592 & ./eat-time 592
> 50%
> 50%
> 50%
> 50%
> etc
>
> This however:
>
> $ ./eat-time 592 & ./eat-time 592 &
> 100%
> 99%
> In another xterm:
> $ nice -n +19 ./eat-time 592 & nice -n +19 ./eat-time 592
> 5%
> 5%
> 5%
>
> Fails sometimes, with all processes getting 50%. The above 'screenshot' is
> from the working and expected situation, which happens most of the time.
>
> When it goes wrong, top shows me that Cpu0 and Cpu1 are 100% user, while
> Cpu2 and Cpu3 are both 100% nice. The niced processes show up in top as
> PRiority 39, the unniced ones (NI = 0) as PR 25.
This is just because the scheduler balancing is not aware of nice and when two
same niceness tasks are on the same physical core they get equal shares. The
ht-aware nice only works at keeping different nice values on the same
physical core fair. There is no more that can be done using the current ht
aware mechanism; a far more complicated balancing algorithm that takes nice
into account would be required.
Con
bert hubert wrote:
> Fails sometimes, with all processes getting 50%. The above 'screenshot' is
> from the working and expected situation, which happens most of the time.
>
> When it goes wrong, top shows me that Cpu0 and Cpu1 are 100% user, while
> Cpu2 and Cpu3 are both 100% nice. The niced processes show up in top as
> PRiority 39, the unniced ones (NI = 0) as PR 25.
>
> I've also seen it that Cpu2 and Cpu3 are 100% busy, and 0 and 1 are 100%
> nice.
>
Ahh, that is because we don't do nice-aware SMP balancing. It
is orthogonal to the HT-nice work.
Fixing it properly requires someone to sit down and do some careful
design of nice-aware balancing that should be general enough to handle
the HT problems as a matter of course.