Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4;
Date:   Wed, 25 Oct 2023 15:34:02 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     Dietmar Eggemann <dietmar.eggemann@arm.com>
CC:     Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Peter Zijlstra <peterz@infradead.org>,
        <linux-kernel@vger.kernel.org>, Ingo Molnar <mingo@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        "Vincent Guittot" <vincent.guittot@linaro.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Swapnil Sapkal <Swapnil.Sapkal@amd.com>,
        Aaron Lu <aaron.lu@intel.com>,
        "Tim Chen" <tim.c.chen@intel.com>,
        K Prateek Nayak <kprateek.nayak@amd.com>,
        "Gautham R . Shenoy" <gautham.shenoy@amd.com>, <x86@kernel.org>
Subject: Re: [RFC PATCH v2 1/2] sched/fair: Introduce UTIL_FITS_CAPACITY
 feature (v2)
Message-ID: <ZTjE6ofbjhsMcLVP@chenyu5-mobl2.ccr.corp.intel.com>
References: <20231019160523.1582101-1-mathieu.desnoyers@efficios.com>
 <20231019160523.1582101-2-mathieu.desnoyers@efficios.com>
 <f40522de-b71d-4848-8aa3-5b87d38bb847@arm.com>
 <c79ac631-61c7-4953-a657-74047a264029@efficios.com>
 <ZTdf0529DL51pj8m@chenyu5-mobl2.ccr.corp.intel.com>
 <14ab201e-0170-4dd7-a1ec-7587fe27385a@arm.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <14ab201e-0170-4dd7-a1ec-7587fe27385a@arm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?THT6llqn88doZfWY8fW2MNlvI3yIVUYmKWpAc/GXyuNoODTQznvUT7Cw2PRm?=
 =?us-ascii?Q?27EF4X5dlpLfLjBssSGMplQiTKS1blS0nTjtQ2IW61DxyXfqWFc3JEsji9pW?=
 =?us-ascii?Q?rJgLynsMWq2bSsFZ5B8z50ZG9/br1n6zXv87ihz6RC2+ylMWV+wG6/qg7d1r?=
 =?us-ascii?Q?wO8RwwVoKosipBbqBG0IngP15MUVwUivR+223s9XQ3R/g5KuQslakOFZdFn7?=
 =?us-ascii?Q?B979/RYw/A7jMKg3iZglbEtzE5uroN+wGuSEaUcSLkVXzyMAiDseAwrGcQtH?=
 =?us-ascii?Q?lHWn424yCJsoTTAhe9vGmeJXG4QpacrzviT7RmJuzA5N2vNrgA2cvqrqN2ZX?=
 =?us-ascii?Q?1nH2oIXUoA2e60JqYCeU5sugqHVPpy24JyKBxnbZRLf6cgNQnMgYqJhCH6kE?=
 =?us-ascii?Q?71nmUzWDZceeFgCiI02V+BwkwhOVivStf9mq7wHecU5HvIgFU3ySe9i1jViJ?=
 =?us-ascii?Q?jCBcC6ZwKK1zgVtuUenFizZmsxiAMlxSDagaoceJ1C790PvvvyCc8Z/F/wRn?=
 =?us-ascii?Q?l7OOjtyPvIf0LQV6dDBTE9DXPfOaXgTn3bsxls9DUWGuI0TUy5uZxNpzPLYA?=
 =?us-ascii?Q?bXyvYceEMeJ80lsSW13dveTXGHnB7jJMGMcXevadOL+Hh18Eao4uj2/nAW/U?=
 =?us-ascii?Q?B7en75gfzjn1ZYX+v2y55P3apl7vPS2JFZozop/fhgRHCDJjEppOiKmJmpOk?=
 =?us-ascii?Q?SFAmFUIihs1V/4areDiyJJWXPgq0HJwTmF7/8AuZoLDD+JskXlOeuyNv201p?=
 =?us-ascii?Q?Gzy4KVYJnB0EmEe5PbykYndTOKsjbd8bsmHuKOSXl+G378F6hbeLuGMQbha0?=
 =?us-ascii?Q?qQRj+M6EKdJeF7ubTADVHk5+OhiiN2NVvDu6OIGESS7E/mrb2T9+LPMT5E1Y?=
 =?us-ascii?Q?PpOKlKbFdBXrXIUfZdvbbYRVeehnTyWvmLqFmn00ISXGA7/rv0dFtUSN79Xe?=
 =?us-ascii?Q?h+ljnQjwObi5cAG9wpndMGYy+NY/JwonXHCWc5q+JTuJmTIIls73QBbYpoSp?=
 =?us-ascii?Q?OvmHWsxpS/DU5pyO7MD2PrzLZCZjfXGzQ31Q+n6RgVg6umfBeUjpHPrAdxqo?=
 =?us-ascii?Q?Wc++YbOF9pM69i5mlfFThyniTFN1SL4BR6uig7y3oGj5lQla3TyS776TR7CY?=
 =?us-ascii?Q?vLdcAnkpcVeIsBHLHP8NOOFONDKVLs0HT4YMauw42H93ZVWyegIHVdp54+Hk?=
 =?us-ascii?Q?FuHarDe+HY61z9whxikc3Iv4h8AnNfvmUY98ST/ARxdGo8nwr97L2FxmC4Lg?=
 =?us-ascii?Q?41lUT/+8/iYy9+qTb8lfFN87ATD/Vik+RJeMrDvkv65UzuCEIKjUK0F9Dtxu?=
 =?us-ascii?Q?OEAo6SJv9e7B2e7hnt+2W1FVCYz6pnTYpKoFYwqJzKQn8jUMtdSGvsxSdDK1?=
 =?us-ascii?Q?kn6jVv6HqgeZWbLfI1I5/kR0iyy09OnsmFCkCw+nD6BdwMCqiJVIIcZMqwop?=
 =?us-ascii?Q?a0T+Lso/dyIZ+XakV8WHa/jEOR9TljVf0wYDvW7cd8ugqAITwJVfPm2DE6Fr?=
 =?us-ascii?Q?mFSp3HNdDsdzpDBQ4JkCqd2GTSt8Q5nOaMNlAq+4nour4lp5Avr+bRmqF62g?=
 =?us-ascii?Q?ddhzxnEYhpzuIg0SOPtl5+Za16uDgVsHzgKvt3SQ?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d3d5cd51-957f-4b43-6283-08dbd52ccc15
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2023 07:34:18.5546
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LXG5FXib+bNv7pmvd/KIefKvuTBG8SbBEwA6C5xliALVqQHTApkpjRIt9OucHEs5QXN2Nl9KgsVA5Z2te2xoBA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR11MB8608
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Wed, 25 Oct 2023 00:46:33 -0700 (PDT)

On 2023-10-24 at 17:03:25 +0200, Dietmar Eggemann wrote:
> On 24/10/2023 08:10, Chen Yu wrote:
> > On 2023-10-23 at 11:04:49 -0400, Mathieu Desnoyers wrote:
> >> On 2023-10-23 10:11, Dietmar Eggemann wrote:
> >>> On 19/10/2023 18:05, Mathieu Desnoyers wrote:
> 
> [...]
> 
> >>> Or like find_energy_efficient_cpu() (feec(), used in
> >>> Energy-Aware-Scheduling (EAS)) which uses cpu_util(cpu, p, cpu, 0) to get:
> >>>
> >>>    max(util_avg(CPU + p), util_est(CPU + p))
> >>
> >> I've tried using cpu_util(), but unfortunately anything that considers
> >> blocked/sleeping tasks in its utilization total does not work for my
> >> use-case.
> >>
> >> From cpu_util():
> >>
> >>  * CPU utilization is the sum of running time of runnable tasks plus the
> >>  * recent utilization of currently non-runnable tasks on that CPU.
> >>
> > 
> > I thought cpu_util() indicates the utilization decay sum of task that was once
> > "running" on this CPU, but will not sum up the "util/load" of the blocked/sleeping
> > task?
> 
> cpu_util() here refers to:
> 
>     cpu_util(int cpu, struct task_struct *p, int dst_cpu, int boost)
> 
> which when called with (cpu, p, cpu, 0) and task_cpu(p) != cpu returns:
> 
>     max(util_avg(CPU + p), util_est(CPU + p))
> 
> The term `CPU utilization` in cpu_util()'s header stands for
> cfs_rq->avg.util_avg.
> 
> It does not sum up the utilization of blocked tasks but it can contain
> it. They have to be a blocked tasks and not tasks which were running in
> cfs_rq since we subtract utilization of tasks which are migrating away
> from the cfs_rq (cfs_rq->removed.util_avg in remove_entity_load_avg()
> and update_cfs_rq_load_avg()).

Thanks for this description in detail, Dietmar. Yes, I just realized that,
if the blocked tasks once ran on this cfs_rq and not being migrated away,
the cfs_rq's util_avg will contain those utils.

thanks,
Chenyu

> > accumulate_sum()
> >     /* only the running task's util will be sum up */
> >     if (running)
> >        sa->util_sum += contrib << SCHED_CAPACITY_SHIFT;
> > 
> > WRITE_ONCE(sa->util_avg, sa->util_sum / divider);
> 
> __update_load_avg_cfs_rq()
> 
>   ___update_load_sum(..., cfs_rq->curr != NULL
>                           ^^^^^^^^^^^^^^^^^^^^
>                           running
>     accumulate_sum()
> 
>       if (periods)
>         /* decay _sum */
>         sa->util_sum = decay_load(sa->util_sum, ...)
> 
>         if (load)
>           /* decay and accrue _sum */
>           contrib = __accumulate_pelt_segments(...)
> 
> When crossing periods we decay the old _sum and when additionally load
> != 0 we decay and accrue the new _sum as well.