I have a number of dual CPU and dual CPU/dual core Opteron systems that
are used as compute servers. In an effort to reduce power consumption
and reduce heat output, I would like to make use of the PowerNow!
capabilities to clock back the CPUs when the machines are idle.
These machines are running a 2.6.9-42 RHEL4 kernel with the powernow-k8
module loaded - which I believe have backported cpufreq support from
more recent mainline kernels.
In trying to achieve what I want, I've become rather confused as to how
cpufreq in a multi-CPU environment works:
There is a directory under /sys/devices/system/cpu/cpu*/cpufreq for each
CPU, which seems to imply that each CPU speed can be controlled
separately - can this really be the case? Can separate CPU cores run at
different speeds?
e.g. I can echo 4 different governor names to the scaling_governor file
in each /sys/devices/system/cpu/cpu[0-3]/cpufreq directory on a 4 core
machine - and the resulting scaling_cur_freq file can contain a
different value.
However, the "cpu MHz" fields in /proc/cpuinfo are all the same for each
each CPU - I assume the values in /proc/cpuinfo are the 'correct' values ??
Also, if I set all the governors to userspace, and then set each CPU's
speed via scaling_setspeed to a different (allowed) value, then it
appears quite random as to which value is then reflected in
/proc/cpuinfo i.e. sometimes it will take the value given to CPU 0,
other times it will be CPU 1 etc.
If I set all the governors to ondemand, the CPUs will from time to time,
clock back their speed in situations where one or more CPUs are being
heavily used. i.e it appears that each CPU is treated separately, and if
one CPU is deemed to be idle enough by its given metrics, then it can
reduce the speed of all CPUs, regardless of other CPUs being 'busy' ...
I've also tried a couple of userspace daemons (cpuspeed and powernowd) -
again, these treat each CPU separately and will also reduce the speed of
an 'idle' CPU - and hence reduce the speed of all the CPUs, again,
regardless of other CPUs being 'busy'.
Essentially what I want to achieve is something like: if _any_ CPU is
'busy' (usage over some threshold over some sampling period), then run
at full speed and if _all_ CPUs are 'idle' (all below some threshold
over some sampling period) then clock back the CPUs.
Is there something/some setting(s) that can do this in a multi-CPU machine?
Thanks
James Pearson
> These machines are running a 2.6.9-42 RHEL4 kernel with the
> powernow-k8 module loaded - which I believe have backported
> cpufreq support from more recent mainline kernels.
To an extent. The problem is not the powernow-k8 driver,
but the cpufreq infrastructure. See below.
> In trying to achieve what I want, I've become rather confused
> as to how cpufreq in a multi-CPU environment works:
>
> There is a directory under
> /sys/devices/system/cpu/cpu*/cpufreq for each
> CPU, which seems to imply that each CPU speed can be controlled
> separately - can this really be the case? Can separate CPU
> cores run at different speeds?
No, but prior to 2.6.10, cpufreq discovers each core
independently, and creates a directory independently.
The powernow-k8 knows which cores are on the same processors,
and handles them together.
After 2.6.10, cpufreq understands multicore devices and
creates 1 directory for the entire processor and then
creates symbolic links for all the cores that share the
same pstates.
> e.g. I can echo 4 different governor names to the
> scaling_governor file in each
> /sys/devices/system/cpu/cpu[0-3]/cpufreq directory on
> a 4 core machine - and the resulting scaling_cur_freq
> file can contain a different value.
>
> However, the "cpu MHz" fields in /proc/cpuinfo are all the
> same for each each CPU - I assume the values in /proc/cpuinfo
> are the 'correct' values ??
>
> Also, if I set all the governors to userspace, and then set
> each CPU's speed via scaling_setspeed to a different
> (allowed) value, then it appears quite random as to which
> value is then reflected in /proc/cpuinfo i.e. sometimes
> it will take the value given to CPU 0,
> other times it will be CPU 1 etc.
It's always the highest requested value of all cores on
a processor. powernow-k8 keeps track of the requested
frequencies, though, so you can have some weird situations.
ie, from boot up:
core0 800 MHz core1 800 MHz CPU 800 MHz
core0 to 2000 MHz core1 800 MHz CPU 2000 MHz
core0 2000 MHz core1 to 1800 MHz CPU 2000 MHz
core0 to 800 MHz core1 1800 MHz CPU 1800 MHz
> If I set all the governors to ondemand, the CPUs will from
> time to time, clock back their speed in situations where
> one or more CPUs are being heavily used. i.e it appears
> that each CPU is treated separately, and if
> one CPU is deemed to be idle enough by its given metrics,
> then it can reduce the speed of all CPUs, regardless of
> other CPUs being 'busy' ...
That shouldn't happen, unless ondemand requests a low
frequency to all cores.
> Essentially what I want to achieve is something like:
> if _any_ CPU is 'busy' (usage over some threshold over
> some sampling period), then run at full speed and
> if _all_ CPUs are 'idle' (all below some threshold
> over some sampling period) then clock back the CPUs.
This should be happening automatically in RHEL4.
Note that the 2.6.10 kernel changed cpufreq behavior
dramatically, and now the most recent frequency request
wins.
-Mark Langsdorf
powernow-k8 maintainer
AMD, Inc.