LinuxLists.cc - [PATCH 0/2] Rework interface between scheduler and schedutil governor

2023-10-13 15:16:42

Subject: [PATCH 0/2] Rework interface between scheduler and schedutil governor

Following the discussion with Qais [1] about how to handle uclamp
requirements and after syncing with him, we agreed that I should move
forward on the patchset to rework the interface between scheduler and
schedutil governor to provide more information to the latter. Scheduler
(and EAS in particular) doesn't need anymore to guess estimate which
headroom the governor wants to apply and will directly ask for the target
freq. Then the governor directly gets the actual utilization and new
minimum and maximum boundaries to select this target frequency and
doesn't have to deal anymore with scheduler internals like uclamp when
including iowait boost.

[1] https://lore.kernel.org/lkml/CAKfTPtA5JqNCauG-rP3wGfq+p8EEVx9Tvwj6ksM3SYCwRmfCTg@mail.gmail.com/

Vincent Guittot (2):
sched/schedutil: rework performance estimation
sched/schedutil: rework iowait boost

include/linux/energy_model.h | 1 -
kernel/sched/core.c | 85 ++++++++++++--------------------
kernel/sched/cpufreq_schedutil.c | 72 +++++++++++++++++----------
kernel/sched/fair.c | 22 +++++++--
kernel/sched/sched.h | 84 +++----------------------------
5 files changed, 105 insertions(+), 159 deletions(-)

--
2.34.1

2023-10-26 10:21:40

by Wyes Karny

[permalink] [raw]

Subject: Re: [PATCH 0/2] Rework interface between scheduler and schedutil governor

Hi Vincent,

On 13 Oct 17:14, > Following the > requirements > forward on the > schedutil governor > (and EAS in > headroom the > freq. Then the > minimum and > doesn't have > including iowait boost.

I ran a duty_cycle
I used below bpftrace bpftrace -e 'kretprobe:eff
Below are the results
-------------------------- -------------------------- busy: 0%, idle: 100% : @eff_util[1]: count @eff_util_hist[1]: [1] 6 | [2, 4) 10 | [4, 8) [8, 16) [16, 32) 52 | [32, 64) 44 | [64, 128) 40 | [128, 256) 38 | [256, 512) 43 | [512, 1K) 40 | [1K, 2K) 6 | -------------------------- busy: 100%, idle: 0% : @eff_util[1]: count @eff_util_hist[1]: [0] 9 | [1] 0 | [2, 4) 0 | [4, 8) 0 | [8, 16) 0 | [16, 32) 0 | [32, 64) 0 | [64, 128) 0 | [128, 256) 1 | [256, 512) 0 | [512, 1K) [1K, 2K) 2 | -------------------------- busy: 10%, idle: 90% : @eff_util[1]: count @eff_util_hist[1]: [1] 10 | [2, 4) 6 | [4, 8) 0 | [8, 16) 0 | [16, 32) 0 | [32, 64) 0 | [64, 128) -------------------------- busy: 20%, idle: 80% : @eff_util[1]: count @eff_util_hist[1]: [1] 13 | [2, 4) 6 | [4, 8) 0 | [8, 16) 1 | [16, 32) 0 | [32, 64) 0 | [64, 128) 0 | [128, 256) -------------------------- busy: 30%, idle: 70% : @eff_util[1]: count @eff_util_hist[1]: [1] 7 | [2, 4) 8 | [4, 8) 0 | [8, 16) 1 | [16, 32) 0 | [32, 64) 0 | [64, 128) 0 | [128, 256) 0 | [256, 512) -------------------------- busy: 40%, idle: 60% : @eff_util[1]: count @eff_util_hist[1]: [0] 2 | [1] 9 | [2, 4) 2 | [4, 8) 0 | [8, 16) 0 | [16, 32) 0 | [32, 64) 0 | [64, 128) 0 | [128, 256) 0 | [256, 512) -------------------------- busy: 50%, idle: 50% : @eff_util[1]: count @eff_util_hist[1]: [0] 10 | [1] 0 | [2, 4) 0 | [4, 8) 0 | [8, 16) 0 | [16, 32) 0 | [32, 64) 0 | [64, 128) 0 | [128, 256) 0 | [256, 512) [512, 1K) 28 | -------------------------- busy: 60%, idle: 40% : @eff_util[1]: count @eff_util_hist[1]: [1] 2 | [2, 4) 8 | [4, 8) 4 | [8, 16) 0 | [16, 32) 0 | [32, 64) 0 | [64, 128) 0 | [128, 256) 0 | [256, 512) 0 | [512, 1K) -------------------------- busy: 70%, idle: 30% : @eff_util[1]: count @eff_util_hist[1]: [0] 9 | [1] 0 | [2, 4) 0 | [4, 8) 0 | [8, 16) 0 | [16, 32) 0 | [32, 64) 0 | [64, 128) 1 | [128, 256) 0 | [256, 512) 0 | [512, 1K) -------------------------- busy: 80%, idle: 20% : @eff_util[1]: count @eff_util_hist[1]: [1] 8 | [2, 4) 6 | [4, 8) 0 | [8, 16) 0 | [16, 32) 0 | [32, 64) 0 | [64, 128) 0 | [128, 256) 0 | [256, 512) 0 | [512, 1K) -------------------------- busy: 90%, idle: 10% : @eff_util[1]: count @eff_util_hist[1]: [0] 9 | [1] 0 | [2, 4) 0 | [4, 8) 0 | [8, 16) 0 | [16, 32) 0 | [32, 64) 0 | [64, 128) 1 | [128, 256) 0 | [256, 512) 0 | [512, 1K) [1K, 2K) 2 | --------------------------
Thanks,
Wyes
>
> [1] https://lore.kernel.org/lkml/CAKfTPtA5JqNCauG-rP3wGfq+p8EEVx9Tvwj6ksM3SYCwRmfCTg@mail.gmail.com/
rework performance estimation
rework iowait boost
| 1 -
| 85 ++++++++++++--------------------
| 72 +++++++++++++++++----------
| 22 +++++++--
| 84 +++----------------------------
105 insertions(+), 159 deletions(-)

2023-10-26 15:12:01

by Vincent Guittot

[permalink] [raw]

Subject: Re: [PATCH 0/2] Rework interface between scheduler and schedutil governor

Hi Wyes,

On Thu, 26 Oct 2023 at 12:19, Wyes Karny <[email protected]> wrote:
>
> Hi Vincent,
>
> On 13 Oct 17:14, Vincent Guittot wrote:
> > Following the discussion with Qais [1] about how to handle uclamp
> > requirements and after syncing with him, we agreed that I should move
> > forward on the patchset to rework the interface between scheduler and
> > schedutil governor to provide more information to the latter. Scheduler
> > (and EAS in particular) doesn't need anymore to guess estimate which
> > headroom the governor wants to apply and will directly ask for the target
> > freq. Then the governor directly gets the actual utilization and new
> > minimum and maximum boundaries to select this target frequency and
> > doesn't have to deal anymore with scheduler internals like uclamp when
> > including iowait boost.
>
> I ran a duty_cycle (one cpu 1) test which does timed busy and idle repeatedly based on user input.
>

Thanks for the tests. IIUC your result below, you don't see any
significant difference in the figures with and without the patch which
is exactly the goal in your case. The difference happens when you use
either uclamp_min/max or deadline tasks or EAS or
cpufreq_driver_adjust_perf()

> I used below bpftrace program to trace effective utilization:
> bpftrace -e 'kretprobe:effective_cpu_util / cpu == 1/ { @eff_util[cpu] = stats(retval); @eff_util_hist[cpu] = hist(retval);}'

Minor point but this should not make any difference in your case
The new effective_cpu_util() replaces the legacy
effective_cpu_util(cpu, util_cfs, type == ENERGY_UTIL, p)
And the new sugov_effective_cpu_perf() replaces the legacy
effective_cpu_util(cpu, util_cfs, type == FREQUENCY_UTIL, p) + dvfs
headroom

Thanks,
Vincent

>
> Below are the results on AMD server system:
>
> --------------------------------------------------------------------------------+ -------------------------------------------------------------------------------+
> Without patches on 6.6-rc6 | With patches on 6.6-rc6 |
> ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------|
> busy: 0%, idle: 100% : | busy: 0%, idle: 100% : |
> | |
> @eff_util[1]: count 4923, average 22, total 110935 | @eff_util[1]: count 5556, average 10, total 58857 |
> | |
> @eff_util_hist[1]: | @eff_util_hist[1]: |
> [1] 6 | | | [1] 14 | ||
> [2, 4) 10 | | | [2, 4) 16 | ||
> [4, 8) 862 |@@@@@@@@@@@ | | [4, 8) 1628 |@@@@@@@@@@@@@@@@@@@@@ ||
> [8, 16) 3782 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| | [8, 16) 3896 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@||
> [16, 32) 52 | | | [16, 32) 2 | ||
> [32, 64) 44 | | | |
> [64, 128) 40 | | | |
> [128, 256) 38 | | | |
> [256, 512) 43 | | | |
> [512, 1K) 40 | | | |
> [1K, 2K) 6 | | | |
> ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------|
> busy: 100%, idle: 0% : | busy: 100%, idle: 0% : |
> | |
> @eff_util[1]: count 5544, average 974, total 5400203 | @eff_util[1]: count 5588, average 972, total 5435602 |
> | |
> @eff_util_hist[1]: | @eff_util_hist[1]: |
> [0] 9 | | | [0] 17 | ||
> [1] 0 | | | [1] 0 | ||
> [2, 4) 0 | | | [2, 4) 0 | ||
> [4, 8) 0 | | | [4, 8) 0 | ||
> [8, 16) 0 | | | [8, 16) 0 | ||
> [16, 32) 0 | | | [16, 32) 0 | ||
> [32, 64) 0 | | | [32, 64) 0 | ||
> [64, 128) 0 | | | [64, 128) 0 | ||
> [128, 256) 1 | | | [128, 256) 0 | ||
> [256, 512) 0 | | | [256, 512) 0 | ||
> [512, 1K) 5532 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| | [512, 1K) 5571 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@||
> [1K, 2K) 2 | | | |
> ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------|
> busy: 10%, idle: 90% : | busy: 10%, idle: 90% : |
> | |
> @eff_util[1]: count 5073, average 102, total 519454 | @eff_util[1]: count 5555, average 101, total 566563 |
> | |
> @eff_util_hist[1]: | @eff_util_hist[1]: |
> [1] 10 | | | [1] 21 | ||
> [2, 4) 6 | | | [2, 4) 10 | ||
> [4, 8) 0 | | | [4, 8) 0 | ||
> [8, 16) 0 | | | [8, 16) 0 | ||
> [16, 32) 0 | | | [16, 32) 0 | ||
> [32, 64) 0 | | | [32, 64) 0 | ||
> [64, 128) 5057 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| | [64, 128) 5524 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@||
> ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------|
> busy: 20%, idle: 80% : | busy: 20%, idle: 80% : |
> | |
> @eff_util[1]: count 5112, average 198, total 1017056 | @eff_util[1]: count 5553, average 201, total 1118650 |
> | |
> @eff_util_hist[1]: | @eff_util_hist[1]: |
> [1] 13 | | | [2, 4) 22 | ||
> [2, 4) 6 | | | [4, 8) 10 | ||
> [4, 8) 0 | | | [8, 16) 0 | ||
> [8, 16) 1 | | | [16, 32) 0 | ||
> [16, 32) 0 | | | [32, 64) 0 | ||
> [32, 64) 0 | | | [64, 128) 0 | ||
> [64, 128) 0 | | | [128, 256) 5521 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@||
> [128, 256) 5092 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| | |
> ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------|
> busy: 30%, idle: 70% : | busy: 30%, idle: 70% : |
> | |
> @eff_util[1]: count 5136, average 297, total 1528840 | @eff_util[1]: count 5548, average 297, total 1650683 |
> | |
> @eff_util_hist[1]: | @eff_util_hist[1]: |
> [1] 7 | | | [0] 17 | ||
> [2, 4) 8 | | | [1] 0 | ||
> [4, 8) 0 | | | [2, 4) 0 | ||
> [8, 16) 1 | | | [4, 8) 0 | ||
> [16, 32) 0 | | | [8, 16) 0 | ||
> [32, 64) 0 | | | [16, 32) 0 | ||
> [64, 128) 0 | | | [32, 64) 0 | ||
> [128, 256) 0 | | | [64, 128) 0 | ||
> [256, 512) 5120 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| | [128, 256) 0 | ||
> | [256, 512) 5531 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@||
> ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------|
> busy: 40%, idle: 60% : | busy: 40%, idle: 60% : |
> | |
> @eff_util[1]: count 5161, average 394, total 2036421 | @eff_util[1]: count 5552, average 394, total 2189976 |
> | |
> @eff_util_hist[1]: | @eff_util_hist[1]: |
> [0] 2 | | | [0] 16 | ||
> [1] 9 | | | [1] 0 | ||
> [2, 4) 2 | | | [2, 4) 0 | ||
> [4, 8) 0 | | | [4, 8) 0 | ||
> [8, 16) 0 | | | [8, 16) 0 | ||
> [16, 32) 0 | | | [16, 32) 0 | ||
> [32, 64) 0 | | | [32, 64) 0 | ||
> [64, 128) 0 | | | [64, 128) 0 | ||
> [128, 256) 0 | | | [128, 256) 0 | ||
> [256, 512) 5148 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| | [256, 512) 5536 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@||
> ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------|
> busy: 50%, idle: 50% : | busy: 50%, idle: 50% : |
> | |
> @eff_util[1]: count 5226, average 491, total 2567889 | @eff_util[1]: count 5559, average 489, total 2722999 |
> | |
> @eff_util_hist[1]: | @eff_util_hist[1]: |
> [0] 10 | | | [0] 2 | ||
> [1] 0 | | | [1] 20 | ||
> [2, 4) 0 | | | [2, 4) 6 | ||
> [4, 8) 0 | | | [4, 8) 0 | ||
> [8, 16) 0 | | | [8, 16) 0 | ||
> [16, 32) 0 | | | [16, 32) 0 | ||
> [32, 64) 0 | | | [32, 64) 0 | ||
> [64, 128) 0 | | | [64, 128) 0 | ||
> [128, 256) 0 | | | [128, 256) 0 | ||
> [256, 512) 5188 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| | [256, 512) 5526 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@||
> [512, 1K) 28 | | | [512, 1K) 5 | ||
> ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------|
> busy: 60%, idle: 40% : | busy: 60%, idle: 40% : |
> | |
> @eff_util[1]: count 5303, average 587, total 3115494 | @eff_util[1]: count 5549, average 588, total 3264071 |
> | |
> @eff_util_hist[1]: | @eff_util_hist[1]: |
> [1] 2 | | | [0] 17 | ||
> [2, 4) 8 | | | [1] 0 | ||
> [4, 8) 4 | | | [2, 4) 0 | ||
> [8, 16) 0 | | | [4, 8) 0 | ||
> [16, 32) 0 | | | [8, 16) 0 | ||
> [32, 64) 0 | | | [16, 32) 0 | ||
> [64, 128) 0 | | | [32, 64) 0 | ||
> [128, 256) 0 | | | [64, 128) 0 | ||
> [256, 512) 0 | | | [128, 256) 0 | ||
> [512, 1K) 5289 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| | [256, 512) 0 | ||
> | [512, 1K) 5532 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@||
> ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------|
> busy: 70%, idle: 30% : | busy: 70%, idle: 30% : |
> | |
> @eff_util[1]: count 5325, average 685, total 3648392 | @eff_util[1]: count 5542, average 685, total 3796277 |
> | |
> @eff_util_hist[1]: | @eff_util_hist[1]: |
> [0] 9 | | | [0] 15 | ||
> [1] 0 | | | [1] 2 | ||
> [2, 4) 0 | | | [2, 4) 0 | ||
> [4, 8) 0 | | | [4, 8) 0 | ||
> [8, 16) 0 | | | [8, 16) 0 | ||
> [16, 32) 0 | | | [16, 32) 0 | ||
> [32, 64) 0 | | | [32, 64) 0 | ||
> [64, 128) 1 | | | [64, 128) 0 | ||
> [128, 256) 0 | | | [128, 256) 0 | ||
> [256, 512) 0 | | | [256, 512) 0 | ||
> [512, 1K) 5315 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| | [512, 1K) 5525 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@||
> ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------|
> busy: 80%, idle: 20% : | busy: 80%, idle: 20% : |
> | |
> @eff_util[1]: count 5327, average 780, total 4160266 | @eff_util[1]: count 5541, average 780, total 4326164 |
> | |
> @eff_util_hist[1]: | @eff_util_hist[1]: |
> [1] 8 | | | [0] 17 | ||
> [2, 4) 6 | | | [1] 0 | ||
> [4, 8) 0 | | | [2, 4) 0 | ||
> [8, 16) 0 | | | [4, 8) 0 | ||
> [16, 32) 0 | | | [8, 16) 0 | ||
> [32, 64) 0 | | | [16, 32) 0 | ||
> [64, 128) 0 | | | [32, 64) 0 | ||
> [128, 256) 0 | | | [64, 128) 0 | ||
> [256, 512) 0 | | | [128, 256) 0 | ||
> [512, 1K) 5313 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| | [256, 512) 0 | ||
> | [512, 1K) 5524 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@||
> ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------|
> busy: 90%, idle: 10% : | busy: 90%, idle: 10% : |
> | |
> @eff_util[1]: count 5424, average 877, total 4762032 | @eff_util[1]: count 5548, average 877, total 4869975 |
> | |
> @eff_util_hist[1]: | @eff_util_hist[1]: |
> [0] 9 | | | [0] 17 | ||
> [1] 0 | | | [1] 0 | ||
> [2, 4) 0 | | | [2, 4) 0 | ||
> [4, 8) 0 | | | [4, 8) 0 | ||
> [8, 16) 0 | | | [8, 16) 0 | ||
> [16, 32) 0 | | | [16, 32) 0 | ||
> [32, 64) 0 | | | [32, 64) 0 | ||
> [64, 128) 1 | | | [64, 128) 0 | ||
> [128, 256) 0 | | | [128, 256) 0 | ||
> [256, 512) 0 | | | [256, 512) 0 | ||
> [512, 1K) 5412 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| | [512, 1K) 5531 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@||
> [1K, 2K) 2 | | | |
> --------------------------------------------------------------------------------+ -------------------------------------------------------------------------------|
>
> Thanks,
> Wyes
> >
> > [1] https://lore.kernel.org/lkml/CAKfTPtA5JqNCauG-rP3wGfq+p8EEVx9Tvwj6ksM3SYCwRmfCTg@mail.gmail.com/
> >
> > Vincent Guittot (2):
> > sched/schedutil: rework performance estimation
> > sched/schedutil: rework iowait boost
> >
> > include/linux/energy_model.h | 1 -
> > kernel/sched/core.c | 85 ++++++++++++--------------------
> > kernel/sched/cpufreq_schedutil.c | 72 +++++++++++++++++----------
> > kernel/sched/fair.c | 22 +++++++--
> > kernel/sched/sched.h | 84 +++----------------------------
> > 5 files changed, 105 insertions(+), 159 deletions(-)
> >
> > --
> > 2.34.1
> >