Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2;
Date:   Sat, 30 Sep 2023 15:11:39 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC:     Peter Zijlstra <peterz@infradead.org>,
        <linux-kernel@vger.kernel.org>, Ingo Molnar <mingo@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>,
        "Mel Gorman" <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Swapnil Sapkal <Swapnil.Sapkal@amd.com>,
        Aaron Lu <aaron.lu@intel.com>, Tim Chen <tim.c.chen@intel.com>,
        K Prateek Nayak <kprateek.nayak@amd.com>,
        "Gautham R . Shenoy" <gautham.shenoy@amd.com>, <x86@kernel.org>
Subject: Re: [RFC PATCH] sched/fair: Bias runqueue selection towards almost
 idle prev CPU
Message-ID: <ZRfKKxBzfu+kf0tM@chenyu5-mobl2.ccr.corp.intel.com>
References: <20230929183350.239721-1-mathieu.desnoyers@efficios.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230929183350.239721-1-mathieu.desnoyers@efficios.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?A9U+AEC0XEjhaeLdFOuASM3nmW/EbmHC24fZPZFj6NdlX/LtkmcBnarSExHX?=
 =?us-ascii?Q?mIYKHeL+C0YV9zMg09ceihDeHl1/P7n+Ghpc3Z7HIbDQbdYc2m1IthdQf4PH?=
 =?us-ascii?Q?gI+smkN/Ki3g3MmgBUzkRkMPaMKvz2X8vVxCxlK8G8r9tuIE+LMueKEny5KN?=
 =?us-ascii?Q?1HY+WiyEoUKvWZ8efFbj+dPN92PaATcYiSAvUHVThBanVNMZHYBPzGBzauQQ?=
 =?us-ascii?Q?/CNLup41MyodKRfKCcNDRXYtYw5g4xVJBDJxOw1Kw502y8LZu8BCW0bFoTtx?=
 =?us-ascii?Q?LJvADvgY3r+XSx4V+tQd+/Zq6ZTp+u7yEXx5J8r7heMZ02HwJfsFIwwRZIYh?=
 =?us-ascii?Q?UK8gKjfc9s92+zx3EnaCJuHP7SqVQuuPgOAaOq+OaMFQqhQdlTqg5O0UwdwK?=
 =?us-ascii?Q?ncSxCjxIjbzYX4TAP/YIRIyep6PwaJvYHQR1fhZPF5jIhGKVBZQoviQwkl5c?=
 =?us-ascii?Q?Mq8MRQtWhle5AG/Gh3zSQl2ybC4dIMXHs8YoDr+hevky2Keu1FYqW/nrdfGI?=
 =?us-ascii?Q?x/l7D970ifcmDI6GjtrLdkWOTODHbe5YL644xOCuZuoQgRm1J5lBJsTYedCw?=
 =?us-ascii?Q?fnAY96zejQNR7B34o/uAmbDRoOgtGIKCgYbOX/GmFEfJnRs7xoHkjstvjeQU?=
 =?us-ascii?Q?eZamKFM+/CBC3tTsZsoyJT6FoWRFLbX/xMfjs7Sb1JUtdtfXHfM6crLjbOIB?=
 =?us-ascii?Q?/i6lIVpxZ7akT5GuvcOWAAKFtgJXIdoJs+Mw91EeHWOddYjF8rwnTV7txVhb?=
 =?us-ascii?Q?E2SE2EpyPTOzVzSs23QgXvRo60/lRubMONV/D8l6lMAQmMWfgAYjwzsRoE7o?=
 =?us-ascii?Q?J+1YT16TyPy5hcNOnTLn9veNixh2j9fLPA8hWUVcU7wwYa4V5WUDqhbpu5va?=
 =?us-ascii?Q?oaC+YeOeq3PlMqi8FkM+jPzWC848YXEHJ9Ba5AwJ4GwZmUR5dXRZpQc0S619?=
 =?us-ascii?Q?8z3bQc2VH6IUi3Gl84uGQ0MSVkBg9OvqEJbp1XJjJjP0ZWvGxgFvQgLD71My?=
 =?us-ascii?Q?MtrTWKA0owWyudgZ/pIgRZ2xNC5HSj4vFqt/E36UkQhjEq63+3P4ceAkQwZE?=
 =?us-ascii?Q?iKvS2NtWuvJBbkiBgb4AW8hTPoGM/6A233NKZWcT+mWVKfGHzJg7+35Jj/wp?=
 =?us-ascii?Q?SQGSv41baDqcklDnwfZ1GMA8lHO9PHj72tI+RL6+n0pON3oF8+rrpmiN/2SP?=
 =?us-ascii?Q?8qStQEXeNYYlUWeNTV5Sv1Yls0N70RsYQTfys5HAlrL6VleDQ4ROV8K6oBFf?=
 =?us-ascii?Q?UG8r/UPHQ7BtsCM1eNf3o9oGZvB/HcMTgjBa0agx4aRoA3w+0S/dolQ7a1T8?=
 =?us-ascii?Q?xQb1oVkJcDntoI5g3gtvVYsWsyImFBNtw4y0n1PsFIZAcQJ5HUoHlEumeork?=
 =?us-ascii?Q?1H6+Kqlu2C9czjp0OU5/LdJjf6aZOvPkoRH8s9rCw4ABjL29pbhl1/DBUBWn?=
 =?us-ascii?Q?I/R11qSeoKbHqpjEAO/mTaO3zTRVBOKOkNmHY7eqDrD6wjtLj3KV+I/3AH9r?=
 =?us-ascii?Q?eG657z/pEY0v9tkHNgarHd6EwN8ir8qHjTv++4WVnt8iCFCdMLVvMurP5h5+?=
 =?us-ascii?Q?KLWVRLHSVD8egu7fGhcuE69hd/5c5BR6CYKvvgEl?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f0dedc22-33a3-401f-7e74-08dbc18487d0
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2023 07:11:57.1557
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NQg+UKSfaygwoxCy74lLa5+aa4aeXq8AX3CMZIIxPC1Y9uQBqsolRw5RKV6bSfWlfwhl4ABa5Bu2IXb4vm4CRw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR11MB5224
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Sat, 30 Sep 2023 00:12:31 -0700 (PDT)

Hi Mathieu,

On 2023-09-29 at 14:33:50 -0400, Mathieu Desnoyers wrote:
> Introduce the WAKEUP_BIAS_PREV_IDLE scheduler feature. It biases
> select_task_rq towards the previous CPU if it was almost idle
> (avg_load <= 0.1%).

Yes, this is a promising direction IMO. One question is that,
can cfs_rq->avg.load_avg be used for percentage comparison?
If I understand correctly, load_avg reflects that more than
1 tasks could have been running this runqueue, and the
load_avg is the direct proportion to the load_weight of that
cfs_rq. Besides, LOAD_AVG_MAX seems to not be the max value
that load_avg can reach, it is the sum of
1024 * (y + y^1 + y^2 ... )

For example,
taskset -c 1 nice -n -20 stress -c 1
cat /sys/kernel/debug/sched/debug | grep 'cfs_rq\[1\]' -A 12 | grep "\.load_avg"
  .load_avg                      : 88763
  .load_avg                      : 1024

88763 is higher than LOAD_AVG_MAX=47742
Maybe the util_avg can be used for precentage comparison I suppose?

> It eliminates frequent task migrations from almost
> idle CPU to completely idle CPUs. This is achieved by using the CPU
> load of the previously used CPU as "almost idle" criterion in
> wake_affine_idle() and select_idle_sibling().
> 
> The following benchmarks are performed on a v6.5.5 kernel with
> mitigations=off.
> 
> This speeds up the following hackbench workload on a 192 cores AMD EPYC
> 9654 96-Core Processor (over 2 sockets):
> 
> hackbench -g 32 -f 20 --threads --pipe -l 480000 -s 100
> 
> from 49s to 32s. (34% speedup)
> 
> We can observe that the number of migrations is reduced significantly
> (-94%) with this patch, which may explain the speedup:
> 
> Baseline:      118M cpu-migrations  (9.286 K/sec)
> With patch:      7M cpu-migrations  (0.709 K/sec)
> 
> As a consequence, the stalled-cycles-backend are reduced:
> 
> Baseline:     8.16% backend cycles idle
> With patch:   6.70% backend cycles idle
> 
> Interestingly, the rate of context switch increases with the patch, but
> it does not appear to be an issue performance-wise:
> 
> Baseline:     454M context-switches (35.677 K/sec)
> With patch:   654M context-switches (62.290 K/sec)
> 
> This was developed as part of the investigation into a weird regression
> reported by AMD where adding a raw spinlock in the scheduler context
> switch accelerated hackbench. It turned out that changing this raw
> spinlock for a loop of 10000x cpu_relax within do_idle() had similar
> benefits.
> 
> This patch achieves a similar effect without the busy-waiting by
> allowing select_task_rq to favor almost idle previously used CPUs based
> on the CPU load of that CPU. The threshold of 0.1% avg_load for almost
> idle CPU load has been identified empirically using the hackbench
> workload.
> 
> Feedback is welcome. I am especially interested to learn whether this
> patch has positive or detrimental effects on performance of other
> workloads.
> 
> Link: https://lore.kernel.org/r/09e0f469-a3f7-62ef-75a1-e64cec2dcfc5@amd.com
> Link: https://lore.kernel.org/lkml/20230725193048.124796-1-mathieu.desnoyers@efficios.com/
> Link: https://lore.kernel.org/lkml/20230810140635.75296-1-mathieu.desnoyers@efficios.com/
> Link: https://lore.kernel.org/lkml/20230810140635.75296-1-mathieu.desnoyers@efficios.com/
> Link: https://lore.kernel.org/lkml/f6dc1652-bc39-0b12-4b6b-29a2f9cd8484@amd.com/
> Link: https://lore.kernel.org/lkml/20230822113133.643238-1-mathieu.desnoyers@efficios.com/
> Link: https://lore.kernel.org/lkml/20230823060832.454842-1-aaron.lu@intel.com/
> Link: https://lore.kernel.org/lkml/20230905171105.1005672-1-mathieu.desnoyers@efficios.com/
> Link: https://lore.kernel.org/lkml/cover.1695704179.git.yu.c.chen@intel.com/
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Valentin Schneider <vschneid@redhat.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Ben Segall <bsegall@google.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Juri Lelli <juri.lelli@redhat.com>
> Cc: Swapnil Sapkal <Swapnil.Sapkal@amd.com>
> Cc: Aaron Lu <aaron.lu@intel.com>
> Cc: Chen Yu <yu.c.chen@intel.com>
> Cc: Tim Chen <tim.c.chen@intel.com>
> Cc: K Prateek Nayak <kprateek.nayak@amd.com>
> Cc: Gautham R . Shenoy <gautham.shenoy@amd.com>
> Cc: x86@kernel.org
> ---
>  kernel/sched/fair.c     | 18 +++++++++++++-----
>  kernel/sched/features.h |  6 ++++++
>  2 files changed, 19 insertions(+), 5 deletions(-)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 1d9c2482c5a3..65a7d923ea61 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -6599,6 +6599,14 @@ static int wake_wide(struct task_struct *p)
>  	return 1;
>  }
>  
> +static bool
> +almost_idle_cpu(int cpu, struct task_struct *p)
> +{
> +	if (!sched_feat(WAKEUP_BIAS_PREV_IDLE))
> +		return false;
> +	return cpu_load_without(cpu_rq(cpu), p) <= LOAD_AVG_MAX / 1000;

Or
return cpu_util_without(cpu_rq(cpu), p) * 1000 <= capacity_orig_of(cpu) ?

thanks,
Chenyu