I'm wondering linux kernel's HT support is correct or not, or whether
it's a feature of P4 HT.
I'm running boinc/seti in the background with nice=19 on my P4 2.8G HT
enabled linux box, kernel 2.6.11.9, where SMT/HT is enabled.
I often watch system monitor applet on gnome desktop or top command in
a termianl window and see when no other applications than boinc is
running, boinc takes full power of both virtual cpus. It is designed
to run to "fill" the idle power of the cpu(s). However any
application is running, there is always some "idle" part appears on
virtual cpus, hence it looks like it wastes up to half of cpu power as
"idle."
For ex, see this "top" result while a vmware is running. (HT is
enabled) setiathome-4.7(blah--) are the background boinc applications
with nice=19.
-----------------------------------------------------------------
top - 03:21:30 up 1:58, 15 users, load average: 4.53, 4.56, 4.78
Tasks: 183 total, 5 running, 178 sleeping, 0 stopped, 0 zombie
Cpu0 : 0.3% us, 94.0% sy, 4.7% ni, 0.7% id, 0.0% wa, 0.3% hi, 0.0% si
Cpu1 : 2.7% us, 1.0% sy, 11.0% ni, 82.4% id, 3.0% wa, 0.0% hi, 0.0% si
Mem: 905068k total, 893584k used, 11484k free, 3840k buffers
Swap: 1662688k total, 0k used, 1662688k free, 526900k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8490 tetsuji 25 0 205m 164m 154m R 93.7 18.7 75:10.29 vmware-vmx
7092 boinc 39 19 22828 18m 2012 R 5.6 2.1 27:22.04 setiathome-4.7.
8605 boinc 39 19 24992 20m 3212 R 5.6 2.3 20:58.46 setiathome-4.7.
7091 boinc 39 19 22696 18m 2012 R 4.7 2.1 27:01.92 setiathome-4.7.
7220 root 16 0 180m 32m 9396 S 1.3 3.6 3:23.59 X
7295 tetsuji 15 0 40936 28m 8356 S 1.3 3.2 0:38.64 gnome-terminal
8483 tetsuji 6 -10 205m 164m 154m S 0.7 18.7 1:04.59 vmware-vmx
7370 tetsuji 17 0 18348 9356 7176 S 0.3 1.0 0:09.53 clock-applet
-----------------------------------------------------------------
When HT is disabled, top looks like this: note idle is 0. nice fills
the vacant cpu power.
-----------------------------------------------------------------
top - 03:37:25 up 10 min, 13 users, load average: 5.31, 3.49, 1.66
Tasks: 137 total, 3 running, 134 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.7% us, 43.9% sy, 51.2% ni, 0.0% id, 0.0% wa, 3.3% hi, 0.0% si
Mem: 905068k total, 895784k used, 9284k free, 1328k buffers
Swap: 1662688k total, 0k used, 1662688k free, 672892k cached
Change delay from 3.0 to:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7350 boinc 39 19 22620 18m 2012 R 54.2 2.1 5:46.97 setiathome-4.7.
8126 tetsuji 15 0 93104 52m 47m S 35.2 6.0 1:14.76 vmware-vmx
8118 root 18 0 2200 876 724 R 5.7 0.1 0:13.71 tar
97 root 10 -5 0 0 0 S 1.0 0.0 0:02.98 kblockd/0
157 root 15 0 0 0 0 S 1.0 0.0 0:04.26 kswapd0
7169 root 15 0 171m 23m 7840 S 0.7 2.6 0:06.90 X
8166 root 15 0 0 0 0 S 0.7 0.0 0:00.06 pdflush
5182 root 15 0 0 0 0 S 0.3 0.0 0:02.84 kjournald
7242 tetsuji 15 0 29556 17m 8184 S 0.3 2.0 0:02.94 gnome-terminal
-----------------------------------------------------------------
Overall power is higher when HT is enabled, so I want to use it with HT enabled.
regards,
--
Luckiest in the world / Weapon of Mass Distraction
http://maverick6664.bravehost.com/
Aviation Jokes: http://www.geocities.com/tetsuji_rai/
Background: http://maverick.ns1.name/
On Thu, 19 May 2005 04:42, Tetsuji "Maverick" Rai wrote:
> I'm wondering linux kernel's HT support is correct or not, or whether
> it's a feature of P4 HT.
>
> I'm running boinc/seti in the background with nice=19 on my P4 2.8G HT
> enabled linux box, kernel 2.6.11.9, where SMT/HT is enabled.
>
> I often watch system monitor applet on gnome desktop or top command in
> a termianl window and see when no other applications than boinc is
> running, boinc takes full power of both virtual cpus. It is designed
> to run to "fill" the idle power of the cpu(s). However any
> application is running, there is always some "idle" part appears on
> virtual cpus, hence it looks like it wastes up to half of cpu power as
> "idle."
>
> For ex, see this "top" result while a vmware is running. (HT is
> enabled) setiathome-4.7(blah--) are the background boinc applications
> with nice=19.
Hyperthread sibling cpus share cpu power. If you let a nice 19 task run full
power on the sibling cpu of a nice 0 task it will drain performance from the
nice 0 task and make it run approximately 40% slower. The only way around
this is to temporarily make the sibling run idle so that a nice 0 task gets
the appropriate proportion of cpu resources compared to a nice 19 task. It is
intentional and quite unique to the linux cpu scheduler as far as I can tell.
On any other scheduler or OS a nice 19 "background" task will make your
machine run much slower.
Cheers,
Con
On 5/19/05, Con Kolivas <[email protected]> wrote:
> ------------snip---------------
> Hyperthread sibling cpus share cpu power. If you let a nice 19 task run full
> power on the sibling cpu of a nice 0 task it will drain performance from the
> nice 0 task and make it run approximately 40% slower. The only way around
> this is to temporarily make the sibling run idle so that a nice 0 task gets
> the appropriate proportion of cpu resources compared to a nice 19 task. It is
> intentional and quite unique to the linux cpu scheduler as far as I can tell.
> On any other scheduler or OS a nice 19 "background" task will make your
> machine run much slower.
>
> Cheers,
> Con
>
Thanks. I understood it's a feature of linux kernel and am satisfied
with it. Actually on Windows xp my application sometimes slows down
maybe due to inpropoer scheduler.
--
Luckiest in the world / Weapon of Mass Distraction
http://maverick6664.bravehost.com/
Aviation Jokes: http://www.geocities.com/tetsuji_rai/
Background: http://maverick.ns1.name/
On Thu, 19 May 2005 04:58 pm, Tetsuji "Maverick" Rai wrote:
> On 5/19/05, Con Kolivas <[email protected]> wrote:
> > ------------snip---------------
> > Hyperthread sibling cpus share cpu power. If you let a nice 19 task run
> > full power on the sibling cpu of a nice 0 task it will drain performance
> > from the nice 0 task and make it run approximately 40% slower. The only
> > way around this is to temporarily make the sibling run idle so that a
> > nice 0 task gets the appropriate proportion of cpu resources compared to
> > a nice 19 task. It is intentional and quite unique to the linux cpu
> > scheduler as far as I can tell. On any other scheduler or OS a nice 19
> > "background" task will make your machine run much slower.
> >
> Thanks. I understood it's a feature of linux kernel and am satisfied
> with it. Actually on Windows xp my application sometimes slows down
> maybe due to inpropoer scheduler.
Well I invented it so it's very unlikely that Windows* will have it (?yet) :D
Cheers,
Con
But I would like a kernel to let boinc (the "nice=19" processes) fill
the idle time as much as possible. The current kernel scheduler seems
very sensitive to low-nice (higher priority) processes.
How can I change this sensitivity? I'm looking at kernel/sched.c,
but it's more complicated than a few years ago when I hacked this
before :) and that I'm using HT (SMP).. Will you tell me any hint
where to modify, and/or what to take care of? or any pointer to
proper resources on the Internet?
When I use gnome desktop with a system monitor applet, I see there's
always some idle part and top also shows the idle time on gnome even
if I don't run any specific applications besides system daemons and
gnome background processes. However without gnome/X window, top shows
no or very small idle power (as a matter of course.) So I want the
kernel to be less sensitive. Maybe it will utilize more cpu power.
best regards,
On 5/19/05, Con Kolivas <[email protected]> wrote:
> On Thu, 19 May 2005 04:58 pm, Tetsuji "Maverick" Rai wrote:
> > On 5/19/05, Con Kolivas <[email protected]> wrote:
> > > ------------snip---------------
> > > Hyperthread sibling cpus share cpu power. If you let a nice 19 task run
> > > full power on the sibling cpu of a nice 0 task it will drain performance
> > > from the nice 0 task and make it run approximately 40% slower. The only
> > > way around this is to temporarily make the sibling run idle so that a
> > > nice 0 task gets the appropriate proportion of cpu resources compared to
> > > a nice 19 task. It is intentional and quite unique to the linux cpu
> > > scheduler as far as I can tell. On any other scheduler or OS a nice 19
> > > "background" task will make your machine run much slower.
> > >
> > Thanks. I understood it's a feature of linux kernel and am satisfied
> > with it. Actually on Windows xp my application sometimes slows down
> > maybe due to inpropoer scheduler.
>
> Well I invented it so it's very unlikely that Windows* will have it (?yet) :D
>
> Cheers,
> Con
>
--
Luckiest in the world / Weapon of Mass Distraction
http://maverick6664.bravehost.com/
Aviation Jokes: http://www.geocities.com/tetsuji_rai/
Background: http://maverick.ns1.name/
I've done a temporary minor hacking, which tells kernel only the half
value of nice in all processes. Actually idle percentage was lowered,
but the response of the main application became slower (as a matter of
course.)
I'm not sure which is better..if possible I want to take advantages of
each one :) Am I expecting too much?
regards,
---------------------my minor hacking--------------
maverick:/usr/src/linux/kernel% diff sched.c sched.c.orig
57d56
< /*
61,68d59
< */
< /*
< * hacked by Tetsuji Maverick Rai
< */
< #define NICE_TO_PRIO(nice) (MAX_RT_PRIO + (nice/2) + 20)
< #define PRIO_TO_NICE(prio) (((prio) - MAX_RT_PRIO - 20)*2)
< #define TASK_NICE(p) PRIO_TO_NICE((p)->static_prio)
<
--------------------
On 5/19/05, Tetsuji Maverick Rai <[email protected]> wrote:
> But I would like a kernel to let boinc (the "nice=19" processes) fill
> the idle time as much as possible. The current kernel scheduler seems
> very sensitive to low-nice (higher priority) processes.
>
> How can I change this sensitivity? I'm looking at kernel/sched.c,
> but it's more complicated than a few years ago when I hacked this
> before :) and that I'm using HT (SMP).. Will you tell me any hint
> where to modify, and/or what to take care of? or any pointer to
> proper resources on the Internet?
>
> When I use gnome desktop with a system monitor applet, I see there's
> always some idle part and top also shows the idle time on gnome even
> if I don't run any specific applications besides system daemons and
> gnome background processes. However without gnome/X window, top shows
> no or very small idle power (as a matter of course.) So I want the
> kernel to be less sensitive. Maybe it will utilize more cpu power.
>
> best regards,
>
> On 5/19/05, Con Kolivas <[email protected]> wrote:
> > On Thu, 19 May 2005 04:58 pm, Tetsuji "Maverick" Rai wrote:
> > > On 5/19/05, Con Kolivas <[email protected]> wrote:
> > > > ------------snip---------------
> > > > Hyperthread sibling cpus share cpu power. If you let a nice 19 task run
> > > > full power on the sibling cpu of a nice 0 task it will drain performance
> > > > from the nice 0 task and make it run approximately 40% slower. The only
> > > > way around this is to temporarily make the sibling run idle so that a
> > > > nice 0 task gets the appropriate proportion of cpu resources compared to
> > > > a nice 19 task. It is intentional and quite unique to the linux cpu
> > > > scheduler as far as I can tell. On any other scheduler or OS a nice 19
> > > > "background" task will make your machine run much slower.
> > > >
> > > Thanks. I understood it's a feature of linux kernel and am satisfied
> > > with it. Actually on Windows xp my application sometimes slows down
> > > maybe due to inpropoer scheduler.
> >
> > Well I invented it so it's very unlikely that Windows* will have it (?yet) :D
> >
> > Cheers,
> > Con
> >
>
--
Luckiest in the world / Weapon of Mass Distraction
http://maverick6664.bravehost.com/
Aviation Jokes: http://www.geocities.com/tetsuji_rai/
Background: http://maverick.ns1.name/
On Thu, 19 May 2005 08:46 pm, Tetsuji "Maverick" Rai wrote:
> I've done a temporary minor hacking, which tells kernel only the half
> value of nice in all processes. Actually idle percentage was lowered,
> but the response of the main application became slower (as a matter of
> course.)
>
> I'm not sure which is better..if possible I want to take advantages of
> each one :) Am I expecting too much?
Yes you are. Hyperthreading (currently depending on workload) only gives you
on average 15-25% more cpu with multiple threads. You can't get something for
nothing. Either the nice 0 task runs slower because a low priority task is
bound to the sibling, or it runs at the same speed and the low priority task
runs for less. If you want the nice 19 task to use more cpu run it at nice 0
- because this is effectively what you are trying to do. If you want more cpu
you need extra true physical cpus, not logical cores.
Your code does not do what you think it is doing either. If you want to change
the bias between nice levels across logical cores search the code for where
the value of sd->per_cpu_gain is set. It is currently set to 25% and you want
to increase it (although as I said you will derive no real world benefit as
your nice 0 task will just slow down).
Cheers,
Con
okay, I'm tired of my temporary hacking (yes, it's just temporary;
only even "nice" values are allowed) and I've just returned to the
original 2.6.11.10 kernel and read your message :)
I understand it is the "destiny" of HT.
Thanks!
On 5/19/05, Con Kolivas <[email protected]> wrote:
> On Thu, 19 May 2005 08:46 pm, Tetsuji "Maverick" Rai wrote:
> > I've done a temporary minor hacking, which tells kernel only the half
> > value of nice in all processes. Actually idle percentage was lowered,
> > but the response of the main application became slower (as a matter of
> > course.)
> >
> > I'm not sure which is better..if possible I want to take advantages of
> > each one :) Am I expecting too much?
>
> Yes you are. Hyperthreading (currently depending on workload) only gives you
> on average 15-25% more cpu with multiple threads. You can't get something for
> nothing. Either the nice 0 task runs slower because a low priority task is
> bound to the sibling, or it runs at the same speed and the low priority task
> runs for less. If you want the nice 19 task to use more cpu run it at nice 0
> - because this is effectively what you are trying to do. If you want more cpu
> you need extra true physical cpus, not logical cores.
>
> Your code does not do what you think it is doing either. If you want to change
> the bias between nice levels across logical cores search the code for where
> the value of sd->per_cpu_gain is set. It is currently set to 25% and you want
> to increase it (although as I said you will derive no real world benefit as
> your nice 0 task will just slow down).
>
> Cheers,
> Con
>
--
Luckiest in the world / Weapon of Mass Distraction
http://maverick6664.bravehost.com/
Aviation Jokes: http://www.geocities.com/tetsuji_rai/
Background: http://maverick.ns1.name/
Thanks for nice hints!! I'm trying with several values, and found
actually as you said, it affects the nice=0 process.
On 5/19/05, Con Kolivas <[email protected]> wrote:
> Your code does not do what you think it is doing either. If you want to change
> the bias between nice levels across logical cores search the code for where
> the value of sd->per_cpu_gain is set. It is currently set to 25% and you want
> to increase it (although as I said you will derive no real world benefit as
> your nice 0 task will just slow down).
>
> Cheers,
> Con
>
--
Luckiest in the world / Weapon of Mass Distraction
http://maverick6664.bravehost.com/
Aviation Jokes: http://www.geocities.com/tetsuji_rai/
Background: http://maverick.ns1.name/
After several tests, I found the default value is the best also to me.
Thanks Con for nice advises and hints on scheduler. It's fun to
see/modify kernel source files. But I think kernel isn't my higher
priority.
regards,
On 5/19/05, Con Kolivas <[email protected]> wrote:
> On Thu, 19 May 2005 08:46 pm, Tetsuji "Maverick" Rai wrote:
> > I've done a temporary minor hacking, which tells kernel only the half
> > value of nice in all processes. Actually idle percentage was lowered,
> > but the response of the main application became slower (as a matter of
> > course.)
> >
> > I'm not sure which is better..if possible I want to take advantages of
> > each one :) Am I expecting too much?
>
> Yes you are. Hyperthreading (currently depending on workload) only gives you
> on average 15-25% more cpu with multiple threads. You can't get something for
> nothing. Either the nice 0 task runs slower because a low priority task is
> bound to the sibling, or it runs at the same speed and the low priority task
> runs for less. If you want the nice 19 task to use more cpu run it at nice 0
> - because this is effectively what you are trying to do. If you want more cpu
> you need extra true physical cpus, not logical cores.
>
> Your code does not do what you think it is doing either. If you want to change
> the bias between nice levels across logical cores search the code for where
> the value of sd->per_cpu_gain is set. It is currently set to 25% and you want
> to increase it (although as I said you will derive no real world benefit as
> your nice 0 task will just slow down).
>
> Cheers,
> Con
>
--
Luckiest in the world / Weapon of Mass Distraction
http://maverick6664.bravehost.com/
Aviation Jokes: http://www.geocities.com/tetsuji_rai/
Background: http://maverick.ns1.name/