Received-SPF: pass (google.com: domain of linux-kernel+bounces-209812-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <4d127882-7552-4454-a151-b051fd0abf92@oppo.com>
Date: Tue, 11 Jun 2024 19:55:42 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH_V2] sched/fair: updates weight of cfs_rq before
 update_cfs_group() in enqueue_entity()
To: Vincent Guittot <vincent.guittot@linaro.org>
Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com,
 dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com,
 mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com,
 linux-kernel@vger.kernel.org
References: <20240603091805.858-1-pugaowei@oppo.com>
 <CAKfTPtDhXVc4HVeRJzgf9o1n9nOrXM=SgKSru3cynLngFPdxSg@mail.gmail.com>
From: Gaowei Pu <pugaowei@oppo.com>
In-Reply-To: <CAKfTPtDhXVc4HVeRJzgf9o1n9nOrXM=SgKSru3cynLngFPdxSg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NzRsNkwxTitxeG8vTGlOT1l4Wm1pSWtkMTl2Tk9vd1psU3E5UFNlbUVMWkk1?=
 =?utf-8?B?QVdlSzRsTzJuSlZPYVpMWFIrWVdSdWJoZnJXNytiMEgwL09nZzJ6OEc3RmhL?=
 =?utf-8?B?RzJrc0VyYVB3N05vVFZWWVVYTjVYUENvVmVQTXpvNnA1WVF1dURYUnFMci85?=
 =?utf-8?B?RW1SVjhmUmlUVys0eEtETkxWaEJSbjdyeW9pWm9ERDZTZkZaUG9NMkFaQXpy?=
 =?utf-8?B?anBSSDRaSFljeVVVajBkWDlWUGk4UG81cm50Q0lCQVNIZ0szYWFlcHFQZGN3?=
 =?utf-8?B?Vmpxdy9PK0xia0QwOUZsakJZRXB1MkhMTWczMzNBZnVJOFlpTXB3SmNYVW5q?=
 =?utf-8?B?ZDhUZmdPb3dESzRTOXllZytkT1Jna3A3VHYyR2JWVk9wU2xIbXAvN1YwU3Mr?=
 =?utf-8?B?V2pwYWxGUkpSbHlxeVZnd1hwN1FTa2lCNW9iS1hLWWt2OS9FME1mTXFYV1dO?=
 =?utf-8?B?ME10aEdPN05HVUYwNDkyK1FwODZJRjR6a3lvS3QzTG5vOWtMR2k3dHBuK0la?=
 =?utf-8?B?dTFMaS9oc2tLOG5PUGxMNDNYQkJHSkxNb3VFR0JhdkVJNkFyaURWLzF3bFZ1?=
 =?utf-8?B?Tzl3bngvL0ltQVAvL3FBYjl4N3IzVzBNNXJTbHFDVE5FWnpCT2ZONUVPVTJr?=
 =?utf-8?B?WGNhdlNTRUdsUEpieUdrNGpFc05tZjR3N0FtYUZ1THNETXlOZXpzdUpRS3pE?=
 =?utf-8?B?ZFNPTVUrSW5qQUZnT0UyZWVvVjBScHdqL3F1T2Uva0k3VDROcUpnaEpoRUp6?=
 =?utf-8?B?YXliVUNaT1pxTzRvTk9vdW5XQWV1NmN0aFkrWDl0S2VUNXdLL1pScGZDTmdt?=
 =?utf-8?B?SW5DYkV3SGhWNkduOXMvY3l1Y1Z3MFJsZkZJTis4Yi9raHhvdis4WExPRzE2?=
 =?utf-8?B?aGliaDNUR3RkMXk4YzVYVlc4eTFlc2F2Sk9FajhFYjdsV0RZZ0tVNDVZODVT?=
 =?utf-8?B?TnFKTS8rVEF0dlFIUU1OK053N3M1bWFsZjRMN1A1RHRnUXlhUDg2cFNIbmVW?=
 =?utf-8?B?WUhpVGZPQ0N1U3dJWXhUNnpLN2NqQTJqeGNVTFpzNitHQzdGZW40UTVEQWdJ?=
 =?utf-8?B?eFJjY3EvQ3l6SVQwOUxscmI2NXJtajNBZlF3R3JKTEJxL0trYkIrOFczR2Vq?=
 =?utf-8?B?eTlUSVVSQ3pqTm9CNERjNnhqb1FwYk0rVFRpaDVDL2tyOFFvOVlPNnptYUxF?=
 =?utf-8?B?VjdyUUZscm05aHgvQzRDL0F6eTFvNkNLVzdRRGo0TzRybXU3U3VqOUlscE80?=
 =?utf-8?B?cVZMVmc3RkhmVjhhMzVOVFlsQVBrMzI2NWNGVGdsdGY4czZOa283NitsTFU0?=
 =?utf-8?B?L0NZL0NWYlQ1RmlHcUNMd3NETlBSR0FNaS9DL2hjVTlkZEZhOC80ZHBaTTUx?=
 =?utf-8?B?YURmaE1Od3JWdjZlUllscWcxRDBYZkVlNldCL1pvQ2k0WU9meC9DK2JGU0xw?=
 =?utf-8?B?bnpzYUZNbUlXWnllcFBLNWdsZ2NTdk5kbkJCNmdiVjMzay9oSmVTU2krRjcw?=
 =?utf-8?B?S24vVWlUcTVUQUl2L3RVYU1vV2NvRlZ3bzMvMXQ1MVVHdmN3VjRJSmJObUVG?=
 =?utf-8?B?OGRiWXdUYUttOEpQa2dCRCt1dnVtcStkMjBRc0ZnQmxwdmFyemYzWnlRSmsy?=
 =?utf-8?B?VkxzT2VpTDhHOCs4aDMyN1JuYkUva3BkWDF1VU1YNUdLa1Q3REZFZUN3ckEw?=
 =?utf-8?B?aEcrUmVvYWU0VU1hem5zSEZ0VFZqWDFTeHZWQ21Sc2FZYVVBUDZEYzlmZGZV?=
 =?utf-8?B?Sk1KMm1zdGxJNFNDREIwTkNuT1VGc0hJcEtRMWxsZWZEZkRlK1N1WVcwQlBk?=
 =?utf-8?B?QWZmbE5XUHBEazFmMVdSOEFiWDJZQTkxWXkrWWRaa01FSWlzQk5KNFB5SG9u?=
 =?utf-8?B?dkZwdGtRUWI5R3hZOTlLcmt1SEVjRkdRSXJqR05xZlpRS3U3YWN3SUU3aWNi?=
 =?utf-8?B?VmZZYmdxa2pXV2tnYnZVM1VDVlBHVHJRT2V1UkF4QXZXcktEVDRCNnV1amtL?=
 =?utf-8?B?TTc1WDFpWjRvR0tPY3BBd0R6VjY2ckxKM1FYTnVnbGFpS3E5YzZvRGQzZjRt?=
 =?utf-8?B?NmlZZUhUem1qSTdqSm1UYlVwRUNjUE1UdFhOWDhGMFdPL2U4bTlpZEZjTXRm?=
 =?utf-8?Q?SmS85M7J/1uviD2Oa621dPp8e?=
X-OriginatorOrg: oppo.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 19d6550c-cea7-42c5-0739-08dc8a0d6f2a
X-MS-Exchange-CrossTenant-AuthSource: SEYPR02MB8152.apcprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jun 2024 11:55:48.5744
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f1905eb1-c353-41c5-9516-62b4a54b5ee6
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jZIQ+GFTiRE27hzoY6bGKgzws1dPIBypdLoAJTFS85vH2dZ+Qb2HjU4JUpFjhlJtRehNXmhoZM/Yvc04nyoiCw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR02MB7099

Hi vincent,

On 2024/6/11 18:59, Vincent Guittot wrote:
> On Mon, 3 Jun 2024 at 11:18, Gaowei.Pu <pugaowei@oppo.com> wrote:
>>
>> From: pugaowei <pugaowei@oppo.com>
>>
>> we should update the weight of cfs_rq before update_cfs_group().
> 
> update_cfs_group() updates the weight of se that is about to be
> enqueued on cfs_rq so the current order looks good for me:
> 
> update_cfs_group()
>   reweight_entity(cfs_rq, se, shares); //se->on_rq == 0
>     update_load_set(&se->load, weight);
>       se->load.weight = weight
> 
> account_entity_enqueue(cfs_rq, se);
>   update_load_add(&cfs_rq->load, se->load.weight);
>     cfs_rq->load.weight += se->load.weight
> 
> Have you faced some problems in particular ?
sorry, made a mistask.
account_entity_enqueue(cfs_rq, se) updates weight of cfs_rq on which entity queued.
update_cfs_group(se) updates weight of the cfs_rq owned by the entity. so there is no conflict
or running order problem.
thanks for your reply~
> 
>> Ensure that we can get accurate shares of the cfs_rq when its
>> weights changes. we can find this work was done correctly in
>> dequeue_entity(). so fix it.
>>
>> patch_V1 :
>> https://lore.kernel.org/lkml/20240531030833.3375-1-pugaowei@oppo.com/T/#u
>> trigger a warnning below because of the changing order of
>> account_entity_enqueue().
>>
>> [ 0.400603][ T0] ? __warn (kernel/panic.c:693)
>>
>> [ 0.400603][ T0] ? place_entity (kernel/sched/fair.c:5256 (discriminator 1))
>>
>> [ 0.400603][ T0] ? report_bug (lib/bug.c:180 lib/bug.c:219)
>>
>> [ 0.400603][ T0] ? handle_bug (arch/x86/kernel/traps.c:239)
>>
>> [ 0.400603][ T0] ? exc_invalid_op (arch/x86/kernel/traps.c:260 (discriminator 1))
>>
>> [ 0.400603][ T0] ? asm_exc_invalid_op (arch/x86/include/asm/idtentry.h:621)
>>
>> [ 0.400603][ T0] ? place_entity (kernel/sched/fair.c:5256 (discriminator 1))
>>
>> [ 0.400603][ T0] ? place_entity (kernel/sched/fair.c:5182)
>>
>> [ 0.400603][ T0] enqueue_entity (kernel/sched/fair.c:5328)
>>
>> [ 0.400603][ T0] enqueue_task_fair (kernel/sched/fair.c:6785)
>>
>> V2 fix the warnning and keep the lag without inflating it when it is
>> the first sched_entity queued on the cfs_rq.
>>
>> Signed-off-by: pugaowei <pugaowei@oppo.com>
>> ---
>>  kernel/sched/fair.c | 24 ++++++++++++++++--------
>>  1 file changed, 16 insertions(+), 8 deletions(-)
>>
>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>> index 8a5b1ae0aa55..2fb1fbcfdda3 100644
>> --- a/kernel/sched/fair.c
>> +++ b/kernel/sched/fair.c
>> @@ -5190,12 +5190,12 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
>>          *
>>          * EEVDF: placement strategy #1 / #2
>>          */
>> -       if (sched_feat(PLACE_LAG) && cfs_rq->nr_running) {
>> +       if (sched_feat(PLACE_LAG)) {
>>                 struct sched_entity *curr = cfs_rq->curr;
>> -               unsigned long load;
>> +               unsigned long load, se_load;
>>
>>                 lag = se->vlag;
>> -
>> +               se_load = scale_load_down(se->load.weight);
>>                 /*
>>                  * If we want to place a task and preserve lag, we have to
>>                  * consider the effect of the new entity on the weighted
>> @@ -5252,9 +5252,13 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
>>                 if (curr && curr->on_rq)
>>                         load += scale_load_down(curr->load.weight);
>>
>> -               lag *= load + scale_load_down(se->load.weight);
>> -               if (WARN_ON_ONCE(!load))
>> -                       load = 1;
>> +               lag *= load + se_load;
>> +               /*
>> +                * we just need to keep the lag whithout inflating it when the se is
>> +                * the first sched_entity queued on cfs_rq.
>> +                */
>> +               if (!load)
>> +                       load = se_load;
>>                 lag = div_s64(lag, load);
>>         }
>>
>> @@ -5304,6 +5308,12 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
>>          */
>>         update_load_avg(cfs_rq, se, UPDATE_TG | DO_ATTACH);
>>         se_update_runnable(se);
>> +
>> +       /*
>> +        * we should update the weight of cfs_rq before update_cfs_group.
>> +        * Ensure we can get accurate shares of the cfs_rq when its weights changes.
>> +        */
>> +       account_entity_enqueue(cfs_rq, se);
>>         /*
>>          * XXX update_load_avg() above will have attached us to the pelt sum;
>>          * but update_cfs_group() here will re-adjust the weight and have to
>> @@ -5318,8 +5328,6 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
>>         if (!curr)
>>                 place_entity(cfs_rq, se, flags);
>>
>> -       account_entity_enqueue(cfs_rq, se);
>> -
>>         /* Entity has migrated, no longer consider this task hot */
>>         if (flags & ENQUEUE_MIGRATED)
>>                 se->exec_start = 0;
>> --
>> 2.17.1
>>