Thermal governors can respond to an overheat event of a cpu by
capping the cpu's maximum possible frequency. This in turn
means that the maximum available compute capacity of the
cpu is restricted. But today in the kernel, task scheduler is
not notified of capping of maximum frequency of a cpu.
In other words, scheduler is unaware of maximum capacity
restrictions placed on a cpu due to thermal activity.
This patch series attempts to address this issue.
The benefits identified are better task placement among available
cpus in event of overheating which in turn leads to better
performance numbers.
The reduction in the maximum possible capacity of a cpu due to a
thermal event can be considered as thermal pressure. Instantaneous
thermal pressure is hard to record and can sometime be erroneous
as there can be mismatch between the actual capping of capacity
and scheduler recording it. Thus solution is to have a weighted
average per cpu value for thermal pressure over time.
The weight reflects the amount of time the cpu has spent at a
capped maximum frequency. Since thermal pressure is recorded as
an average, it must be decayed periodically. Exisiting algorithm
in the kernel scheduler pelt framework is re-used to calculate
the weighted average. This patch series also defines a sysctl
inerface to allow for a configurable decay period.
Regarding testing, basic build, boot and sanity testing have been
performed on db845c platform with debian file system.
Further, dhrystone and hackbench tests have been
run with the thermal pressure algorithm. During testing, due to
constraints of step wise governor in dealing with big little systems,
trip point 0 temperature was made assymetric between cpus in little
cluster and big cluster; the idea being that
big core will heat up and cpu cooling device will throttle the
frequency of the big cores faster, there by limiting the maximum available
capacity and the scheduler will spread out tasks to little cores as well.
Test Results
Hackbench: 1 group , 30000 loops, 10 runs
Result SD
(Secs) (% of mean)
No Thermal Pressure 14.03 2.69%
Thermal Pressure PELT Algo. Decay : 32 ms 13.29 0.56%
Thermal Pressure PELT Algo. Decay : 64 ms 12.57 1.56%
Thermal Pressure PELT Algo. Decay : 128 ms 12.71 1.04%
Thermal Pressure PELT Algo. Decay : 256 ms 12.29 1.42%
Thermal Pressure PELT Algo. Decay : 512 ms 12.42 1.15%
Dhrystone Run Time : 20 threads, 3000 MLOOPS
Result SD
(Secs) (% of mean)
No Thermal Pressure 9.452 4.49%
Thermal Pressure PELT Algo. Decay : 32 ms 8.793 5.30%
Thermal Pressure PELT Algo. Decay : 64 ms 8.981 5.29%
Thermal Pressure PELT Algo. Decay : 128 ms 8.647 6.62%
Thermal Pressure PELT Algo. Decay : 256 ms 8.774 6.45%
Thermal Pressure PELT Algo. Decay : 512 ms 8.603 5.41%
A Brief History
The first version of this patch-series was posted with resuing
PELT algorithm to decay thermal pressure signal. The discussions
that followed were around whether intanteneous thermal pressure
solution is better and whether a stand-alone algortihm to accumulate
and decay thermal pressure is more appropriate than re-using the
PELT framework.
Tests on Hikey960 showed the stand-alone algorithm performing slightly
better than resuing PELT algorithm and V2 was posted with the stand
alone algorithm. Test results were shared as part of this series.
Discussions were around re-using PELT algorithm and running
further tests with more granular decay period.
For some time after this development was impeded due to hardware
unavailability, some other unforseen and possibly unfortunate events.
For this version, h/w was switched from hikey960 to db845c.
Also Instantaneous thermal pressure was never tested as part of this
cycle as it is clear that weighted average is a better implementation.
The non-PELT algorithm never gave any conclusive results to prove that it
is better than reusing PELT algorithm, in this round of testing.
Also reusing PELT algorithm means thermal pressure tracks the
other utilization signals in the scheduler.
v3->v4:
- "Patch 3/7:sched: Initialize per cpu thermal pressure structure"
is dropped as it is no longer needed following changes in other
other patches.
- rest of the change log mentioned in specific patches.
v5->v6:
- "Added arch_ interface APIs to access and update thermal pressure.
Moved declaration of per cpu thermal_pressure valriable and
infrastructure to update the variable to topology files.
Thara Gopinath (6):
sched/pelt.c: Add support to track thermal pressure
sched/fair: Add infrastructure to store and update instantaneous
thermal pressure
sched/fair: Enable periodic update of thermal pressure
sched/fair: update cpu_capcity to reflect thermal pressure
thermal/cpu-cooling: Update thermal pressure in case of a maximum
frequency capping
sched/fair: Enable tuning of decay period
Thara Gopinath (7):
sched/pelt.c: Add support to track thermal pressure
sched: Add hook to read per cpu thermal pressure.
Add infrastructure to store and update instantaneous thermal pressure
sched/fair: Enable periodic update of average thermal pressure
sched/fair: update cpu_capacity to reflect thermal pressure
thermal/cpu-cooling: Update thermal pressure in case of a maximum
frequency capping
sched/fair: Enable tuning of decay period
Documentation/admin-guide/kernel-parameters.txt | 5 +++
arch/arm/include/asm/topology.h | 3 ++
arch/arm64/include/asm/topology.h | 3 ++
drivers/base/arch_topology.c | 13 +++++++
drivers/thermal/cpu_cooling.c | 19 +++++++++--
include/linux/arch_topology.h | 11 ++++++
include/linux/sched/topology.h | 8 +++++
include/trace/events/sched.h | 4 +++
kernel/sched/fair.c | 45 +++++++++++++++++++++++++
kernel/sched/pelt.c | 22 ++++++++++++
kernel/sched/pelt.h | 7 ++++
kernel/sched/sched.h | 1 +
12 files changed, 139 insertions(+), 2 deletions(-)
--
2.1.4
On 12/16/2019 09:56 AM, Peter Zijlstra wrote:
> On Wed, Dec 11, 2019 at 11:11:41PM -0500, Thara Gopinath wrote:
>> Test Results
>>
>> Hackbench: 1 group , 30000 loops, 10 runs
>> Result SD
>> (Secs) (% of mean)
>> No Thermal Pressure 14.03 2.69%
>> Thermal Pressure PELT Algo. Decay : 32 ms 13.29 0.56%
>> Thermal Pressure PELT Algo. Decay : 64 ms 12.57 1.56%
>> Thermal Pressure PELT Algo. Decay : 128 ms 12.71 1.04%
>> Thermal Pressure PELT Algo. Decay : 256 ms 12.29 1.42%
>> Thermal Pressure PELT Algo. Decay : 512 ms 12.42 1.15%
>>
>> Dhrystone Run Time : 20 threads, 3000 MLOOPS
>> Result SD
>> (Secs) (% of mean)
>> No Thermal Pressure 9.452 4.49%
>> Thermal Pressure PELT Algo. Decay : 32 ms 8.793 5.30%
>> Thermal Pressure PELT Algo. Decay : 64 ms 8.981 5.29%
>> Thermal Pressure PELT Algo. Decay : 128 ms 8.647 6.62%
>> Thermal Pressure PELT Algo. Decay : 256 ms 8.774 6.45%
>> Thermal Pressure PELT Algo. Decay : 512 ms 8.603 5.41%
>
> What is the conclusion, if any from these results? Clearly thermal
> pressuse seems to help, but what window? ISTR we default to 32ms, which
> is a wash for drystone, but sub-optimal for hackbench.
Hi Peter,
Thanks for the reviews. IMHO, the conclusion is that thermal pressure is
beneficial but the decay period to be used depends on the architecture
and/or use-cases. Sticking to 32ms should give some improvement but it
can be tuned depending on the system.
>
>
> Anyway, the patches look more or less acceptible, just a bunch of nits,
> the biggest being the fact that even if an architecture does not support
> this there is still the code and runtime overhead.
I am fixing this and sending out a v7.
>
--
Warm Regards
Thara