I've been running seti@home niced to 19 in the background since 1999
without any problems. No noticeable effect even when playing a movie
or a game. But since 2.6.38 the new fix-all-problems automatic
grouping has been messing a bit with me. These are some timed compiles
on my 8 cores.
time make -j12 # with seti@home running
real 4m16.753s
user 10m33.770s
sys 1m39.710s
time make -j12 # without seti@home running
real 2m12.480s
user 10m11.580s
sys 1m39.980s
echo 0 > /proc/sys/kernel/sched_autogroup_enabled
time make -j12 # no autogroup, seti@home running again
real 2m33.276s
user 10m37.540s
sys 1m43.190s
All compiles already had all files cached in RAM.
Now I can take the 10% performance hit, but not the 100% hit of
running stuff super niced in the background. Processes niced to 19
should only use spare cycles and not take up half of the cores even
with autogroup. I would really like to run autogroup since it is a
neat idea, but it can't mess up running niced processes in the
background which have been working fine for 12 years.
/Carl-Johan Kjellander
Cc'ing more people.
On Fri, May 13, 2011 at 3:39 PM, Carl-Johan Kjellander
<[email protected]> wrote:
> I've been running seti@home niced to 19 in the background since 1999
> without any problems. No noticeable effect even when playing a movie
> or a game. But since 2.6.38 the new fix-all-problems automatic
> grouping has been messing a bit with me. These are some timed compiles
> on my 8 cores.
>
> time make -j12 # with seti@home running
> real 4m16.753s
> user 10m33.770s
> sys 1m39.710s
>
> time make -j12 # without seti@home running
> real 2m12.480s
> user 10m11.580s
> sys 1m39.980s
>
> echo 0 > /proc/sys/kernel/sched_autogroup_enabled
> time make -j12 # no autogroup, seti@home running again
> real 2m33.276s
> user 10m37.540s
> sys 1m43.190s
>
> All compiles already had all files cached in RAM.
>
> Now I can take the 10% performance hit, but not the 100% hit of
> running stuff super niced in the background. Processes niced to 19
> should only use spare cycles and not take up half of the cores even
> with autogroup. I would really like to run autogroup since it is a
> neat idea, but it can't mess up running niced processes in the
> background which have been working fine for 12 years.
Then how about change the nice value of seti@home->autogroup?
echo 19 > /proc/'pid of seti@home'/autogroup
Thanks,
Yong
>
> /Carl-Johan Kjellander
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
--
Only stand for myself
On Fri, 2011-05-13 at 15:53 +0800, Yong Zhang wrote:
> Cc'ing more people.
>
> On Fri, May 13, 2011 at 3:39 PM, Carl-Johan Kjellander
> <[email protected]> wrote:
> > I've been running seti@home niced to 19 in the background since 1999
> > without any problems. No noticeable effect even when playing a movie
> > or a game. But since 2.6.38 the new fix-all-problems automatic
> > grouping has been messing a bit with me. These are some timed compiles
> > on my 8 cores.
Heh, it's not a fix-all-problems thingy, and was never intended to be.
It's also not enabled by default.
> > time make -j12 # with seti@home running
> > real 4m16.753s
> > user 10m33.770s
> > sys 1m39.710s
> >
> > time make -j12 # without seti@home running
> > real 2m12.480s
> > user 10m11.580s
> > sys 1m39.980s
> >
> > echo 0 > /proc/sys/kernel/sched_autogroup_enabled
> > time make -j12 # no autogroup, seti@home running again
> > real 2m33.276s
> > user 10m37.540s
> > sys 1m43.190s
> >
> > All compiles already had all files cached in RAM.
> >
> > Now I can take the 10% performance hit, but not the 100% hit of
> > running stuff super niced in the background. Processes niced to 19
> > should only use spare cycles and not take up half of the cores even
> > with autogroup. I would really like to run autogroup since it is a
> > neat idea, but it can't mess up running niced processes in the
> > background which have been working fine for 12 years.
>
> Then how about change the nice value of seti@home->autogroup?
> echo 19 > /proc/'pid of seti@home'/autogroup
Yup. Overhead and whatnot is the dark side of group scheduling. The
thing to do is to turn group scheduling off if you don't like what it
does for/to you.
-Mike
* Mike Galbraith <[email protected]> wrote:
> > > time make -j12 # with seti@home running
> > > real 4m16.753s
> > > user 10m33.770s
> > > sys 1m39.710s
> > >
> > > time make -j12 # without seti@home running
> > > real 2m12.480s
> > > user 10m11.580s
> > > sys 1m39.980s
I think the practical question here is to make seti@home run more idle.
Are there some magic cgroup commands you could recommend for that?
Thanks,
Ingo
On Fri, 2011-05-13 at 10:22 +0200, Ingo Molnar wrote:
> * Mike Galbraith <[email protected]> wrote:
>
> > > > time make -j12 # with seti@home running
> > > > real 4m16.753s
> > > > user 10m33.770s
> > > > sys 1m39.710s
> > > >
> > > > time make -j12 # without seti@home running
> > > > real 2m12.480s
> > > > user 10m11.580s
> > > > sys 1m39.980s
>
> I think the practical question here is to make seti@home run more idle.
>
> Are there some magic cgroup commands you could recommend for that?
Yong already did.
* Peter Zijlstra <[email protected]> wrote:
> On Fri, 2011-05-13 at 10:22 +0200, Ingo Molnar wrote:
> > * Mike Galbraith <[email protected]> wrote:
> >
> > > > > time make -j12 # with seti@home running
> > > > > real 4m16.753s
> > > > > user 10m33.770s
> > > > > sys 1m39.710s
> > > > >
> > > > > time make -j12 # without seti@home running
> > > > > real 2m12.480s
> > > > > user 10m11.580s
> > > > > sys 1m39.980s
> >
> > I think the practical question here is to make seti@home run more idle.
> >
> > Are there some magic cgroup commands you could recommend for that?
>
> Yong already did.
Oh, indeed, stupid me. This teaches me to not stop at the first paragraph of
interesting looking emails ;-)
Could we somehow automate this:
> echo 19 > /proc/'pid of seti@home'/autogroup
and split off nice 19 tasks into separate groups and lower the group's
priority?
That would fit into the general principle of auto-sched as well.
Another thing we could do is to lower the priority of a cgroup if it *only*
runs reniced tasks. I.e. track the 'maximum priority' of cgroups and propagate
that to their weight.
This way renicing within cgroups will be more powerful and people do not have
to muck with cgroup details.
Thanks,
Ingo
On Fri, 2011-05-13 at 11:05 +0200, Ingo Molnar wrote:
> Could we somehow automate this:
>
> > echo 19 > /proc/'pid of seti@home'/autogroup
>
> and split off nice 19 tasks into separate groups and lower the group's
> priority?
Well I guess you can stack on all kinds of heuristics, do we want to?
I'd argue for not, keep is simple.
On Fri, May 13, 2011 at 11:07 AM, Peter Zijlstra <[email protected]> wrote:
> On Fri, 2011-05-13 at 11:05 +0200, Ingo Molnar wrote:
>> Could we somehow automate this:
>>
>> > echo 19 > /proc/'pid of seti@home'/autogroup
Tried this.
echo 19 >/proc/23760/autogroup
echo 1 > /proc/sys/kernel/sched_autogroup_enabled
time make -j12 #seti@home, autogroup, group reniced.
real 3m9.274s
user 11m3.020s
sys 1m45.550s
So 50% increase in compilation time.
Will these autogroups remember the 19, the boinc manager starts 8
processes and it will spawn more as time goes by and it finishes
tasks.
>> and split off nice 19 tasks into separate groups and lower the group's
>> priority?
>
> Well I guess you can stack on all kinds of heuristics, do we want to?
> I'd argue for not, keep is simple.
>
I'd again argue, I use nice 19 for stuff that I want to run on the
spare cycles, I don't want them stealing time from my important work
or surfing or movie viewing.
/cjk
* Peter Zijlstra <[email protected]> wrote:
> On Fri, 2011-05-13 at 11:05 +0200, Ingo Molnar wrote:
> > Could we somehow automate this:
> >
> > > echo 19 > /proc/'pid of seti@home'/autogroup
> >
> > and split off nice 19 tasks into separate groups and lower the group's
> > priority?
>
> Well I guess you can stack on all kinds of heuristics, do we want to?
Well have you seen my non-heuristic suggestion:
| Another thing we could do is to lower the priority of a cgroup if it *only*
| runs reniced tasks. I.e. track the 'maximum priority' of cgroups and
| propagate that to their weight.
|
| This way renicing within cgroups will be more powerful and people do not have
| to muck with cgroup details.
A cgroup assuming the highest priority of all tasks it contains is a pretty
natural definition and extension of priorities and also solves this usecase.
Thanks,
Ingo
On Fri, 2011-05-13 at 11:29 +0200, Ingo Molnar wrote:
> * Peter Zijlstra <[email protected]> wrote:
>
> > On Fri, 2011-05-13 at 11:05 +0200, Ingo Molnar wrote:
> > > Could we somehow automate this:
> > >
> > > > echo 19 > /proc/'pid of seti@home'/autogroup
> > >
> > > and split off nice 19 tasks into separate groups and lower the group's
> > > priority?
> >
> > Well I guess you can stack on all kinds of heuristics, do we want to?
>
> Well have you seen my non-heuristic suggestion:
>
> | Another thing we could do is to lower the priority of a cgroup if it *only*
> | runs reniced tasks. I.e. track the 'maximum priority' of cgroups and
> | propagate that to their weight.
> |
> | This way renicing within cgroups will be more powerful and people do not have
> | to muck with cgroup details.
>
> A cgroup assuming the highest priority of all tasks it contains is a pretty
> natural definition and extension of priorities and also solves this usecase.
Well, that a heuristic in my book, and it totally destroys the
independence of groups from tasks (resulting in O(n) task nice
behaviour).
I really don't see why we should do this, if people don't want what it
does, don't use it. If you want something else, you can do all these
things from userspace to suit your exact needs.
We have enough knobs to set things up as you want them, no need to make
things more complicated.
* Peter Zijlstra <[email protected]> wrote:
> On Fri, 2011-05-13 at 11:29 +0200, Ingo Molnar wrote:
> > * Peter Zijlstra <[email protected]> wrote:
> >
> > > On Fri, 2011-05-13 at 11:05 +0200, Ingo Molnar wrote:
> > > > Could we somehow automate this:
> > > >
> > > > > echo 19 > /proc/'pid of seti@home'/autogroup
> > > >
> > > > and split off nice 19 tasks into separate groups and lower the group's
> > > > priority?
> > >
> > > Well I guess you can stack on all kinds of heuristics, do we want to?
> >
> > Well have you seen my non-heuristic suggestion:
> >
> > | Another thing we could do is to lower the priority of a cgroup if it *only*
> > | runs reniced tasks. I.e. track the 'maximum priority' of cgroups and
> > | propagate that to their weight.
> > |
> > | This way renicing within cgroups will be more powerful and people do not have
> > | to muck with cgroup details.
> >
> > A cgroup assuming the highest priority of all tasks it contains is a pretty
> > natural definition and extension of priorities and also solves this usecase.
>
> Well, that a heuristic in my book, and it totally destroys the independence
> of groups from tasks (resulting in O(n) task nice behaviour).
>
> I really don't see why we should do this, if people don't want what it does,
> don't use it. If you want something else, you can do all these things from
> userspace to suit your exact needs.
>
> We have enough knobs to set things up as you want them, no need to make
> things more complicated.
Ok, i guess you are right, propagating priorities does break the clean
hieararchy we have currently.
Still, the other important problem is that we still seem to have a bug, even
with the cgroup set to low prio seti@home is sucking up CPU resources ...
Thanks,
Ingo
On Fri, 2011-05-13 at 12:04 +0200, Ingo Molnar wrote:
> Still, the other important problem is that we still seem to have a bug, even
> with the cgroup set to low prio seti@home is sucking up CPU resources ...
I don't see how. Other than the expected nice 19 overrun when nice 0
group blocks, it works fine on my little Q6600 box.
time make -j4 vmlinux (cache hot)
real 2m22.996s
user 7m6.887s
sys 0m48.999s
echo 0 > sched_autogroup_enabled
time make -j4 vmlinux
real 2m17.052s (darn, no free lunch)
user 7m5.483s
sys 0m49.415s
echo 1 > sched_autogroup_enabled
simultaneous massive_intr 8 9999 in nice 19 autogroup and
time make -j4 vmlinux in a nice 0 autogroup
real 2m30.863s
user 7m5.363s
sys 0m47.359s
142.996/150.863 = .947 (a tad low)
repeat with 2 kbuild tasks/core to cut nice 0 group's idle time
time make -j8 vmlinux
real 2m24.925s
user 7m16.327s
sys 0m50.807s
142.996/144.925 = .986 (all better)
-Mike
On Fri, 2011-05-13 at 15:13 +0200, Mike Galbraith wrote:
>
> > Still, the other important problem is that we still seem to have a bug, even
> > with the cgroup set to low prio seti@home is sucking up CPU resources ...
>
> I don't see how.
Agreed, with two groups, a spinner each, and then setting the group
weight low yields things like:
1927 root 20 0 105m 672 192 R 98.1 0.0 0:20.50 bash
1933 root 20 0 105m 656 180 R 2.0 0.0 0:07.29 bash
So all just seems to work as advertised.
On Fri, May 13, 2011 at 3:13 PM, Mike Galbraith <[email protected]> wrote:
> On Fri, 2011-05-13 at 12:04 +0200, Ingo Molnar wrote:
>
>> Still, the other important problem is that we still seem to have a bug, even
>> with the cgroup set to low prio seti@home is sucking up CPU resources ...
>
> I don't see how. Other than the expected nice 19 overrun when nice 0
> group blocks, it works fine on my little Q6600 box.
Dunno if I've done it correct, but I've set 19 to the boinc manager
autogroup and some of
the seti@home clients, but the clients of course keep changing.
boinc 1172 0.1 0.0 81896 13408 ? SN May09 7:00
/usr/bin/boinc --check_all_logins --redirectio --dir /var/lib/boi
boinc 18983 82.6 0.3 98172 65224 ? RNl 08:10 364:28 \_
../../projects/setiathome.berkeley.edu/setiathome_enhanced
boinc 19162 83.0 0.4 98836 65948 ? RNl 08:16 360:32 \_
../../projects/setiathome.berkeley.edu/setiathome_enhanced
boinc 20295 84.6 0.3 98356 65468 ? RNl 08:57 332:42 \_
../../projects/setiathome.berkeley.edu/setiathome_enhanced
boinc 22980 82.5 0.3 97880 64992 ? RNl 09:32 295:55 \_
../../projects/setiathome.berkeley.edu/setiathome_enhanced
boinc 23760 81.6 0.3 98064 65168 ? RNl 09:59 270:25 \_
../../projects/setiathome.berkeley.edu/setiathome_enhanced
boinc 634 83.1 0.3 98224 65276 ? RNl 11:02 223:24 \_
../../projects/setiathome.berkeley.edu/setiathome_enhanced
boinc 31758 83.5 0.4 99116 65736 ? RNl 11:33 198:48 \_
../../projects/setiathome.berkeley.edu/setiathome_enhanced
boinc 5931 81.4 0.3 98456 65464 ? RNl 14:06 68:55 \_
../../projects/setiathome.berkeley.edu/setiathome_enhan
But when I build on Intel(R) Core(TM) i7 CPU 870 @ 2.93GHz,
it's still a lot slower.
time make -j12
real 2m58.437s
user 10m58.010s
sys 1m45.610s
I can try the same thing at home on my Q6600 machine if I upgrade it,
cause of course the Core i7 doesn't actually have 8 cores, they are
just hyperthreaded. It might be a factor.
Or am doing something horribly wrong when I try to set the autogroup to 19?
/cjk
On Fri, 2011-05-13 at 15:36 +0200, Carl-Johan Kjellander wrote:
> I can try the same thing at home on my Q6600 machine if I upgrade it,
> cause of course the Core i7 doesn't actually have 8 cores, they are
> just hyperthreaded. It might be a factor.
>
> Or am doing something horribly wrong when I try to set the autogroup to 19?
You seemingly haven't niced the parent's group, else new clients would
behave themselves.
cat /proc/pid/NN/autogroup will show nice level for group of pid NN.
(Watch out you don't nice down something like kdeinit;)
-Mike