Received-SPF: pass (google.com: domain of linux-kernel+bounces-71474-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Mon, 19 Feb 2024 22:24:39 +0800
From: Chen Yu <yu.c.chen@intel.com>
To: Hillf Danton <hdanton@sina.com>
CC: Peter Zijlstra <peterz@infradead.org>, Mathieu Desnoyers
	<mathieu.desnoyers@efficios.com>, Vincent Guittot
	<vincent.guittot@linaro.org>, Mel Gorman <mgorman@suse.de>, K Prateek Nayak
	<kprateek.nayak@amd.com>, Chen Yu <yu.chen.surf@gmail.com>,
	<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2 3/3] sched/fair: do not scribble cache-hot CPU in
 select_idle_cpu()
Message-ID: <ZdNkp5DP8aKnsYGt@chenyu5-mobl2>
References: <cover.1700548379.git.yu.c.chen@intel.com>
 <20240219115014.1333-1-hdanton@sina.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240219115014.1333-1-hdanton@sina.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?TpqwfekDaOScb3oyU/V/uTpzWGFMywJvS1ayqc7bInoEHrknhIIvocwkyP1q?=
 =?us-ascii?Q?4D5297m8ug6mcKVLduRAnSufrXG5HFCwGYBtCKt44aImZ/ChLMsUH144R/Rc?=
 =?us-ascii?Q?9Q/38gkVAHXHpzU7piMhij8y2/tkc7/akFKlz7IfNXd+mU9iI3NCG4J9bEZN?=
 =?us-ascii?Q?66XzrwxhbmzQM6f/mNBIZMu4oP0MEzVYOQmi7sVAASDMEykRUXoOsoQttmlh?=
 =?us-ascii?Q?iAo8Gs2u7Lcot/QiBUKAZBFamDVmnp28DjpzS9V7L3JUo53+BYSBcFBEZ3iY?=
 =?us-ascii?Q?w2c6OyQfE3jhSnPUC/v0JtXcJvyUb6N616daQKJyciJT5kwWemnhaL9tYTlY?=
 =?us-ascii?Q?SYtH3TfUijG6a3P+hvYUT9dkS/4K5TBPs4j0klITrz+tgzCtq/cGtcxIfZp5?=
 =?us-ascii?Q?exPAMfFcUJl07bUu033hGKsSmAu3kzx+1vCydmaYd7IRNJlMg2AIhmdMESJ+?=
 =?us-ascii?Q?Zd9AzH1r7io1cimhm71I2vYx30bAgb+d9BTnGKGDNFU5iFz+YlL3gK2soTJ1?=
 =?us-ascii?Q?u8g/2rWh7+pQOL+uyHEkusuu0IkVjAZcJF0wHGCE2AF80a+Kw3Ct4sNZpvop?=
 =?us-ascii?Q?73IkQ3Dcg0uPaYsFgCDkKhgfCNAy5vLWXeecIjvu+Y/IIUq58l2mWaoTAeqI?=
 =?us-ascii?Q?2jEReBK1h4etEXuHOx7tNQ/SD8R20bUGx+oEiOT78qyi7aXqs2/fcuuAtqGz?=
 =?us-ascii?Q?/LaK+b7T2Hj0dsU4bYOhGDu0wZGzZsDMKS3KnaJgEREpubB/NR3tJ5z20IZX?=
 =?us-ascii?Q?rmZ6a7K2+E0WfOKAo9JVAdHCrlaKNK5rmP6bGtGdSG3iDsbGf1m/5+Vxi4es?=
 =?us-ascii?Q?b5l3b8ATLz7DRpYBXUtbwtia8uNLwjlWlE/7I3uZ5LQv4v8L6bEbGsWtvu/g?=
 =?us-ascii?Q?azOChsIcNWO9PYXPtG247aSoyYvbzuMrBAD3xlU9RT5XX6GSJSknnD1xf7Is?=
 =?us-ascii?Q?N5Vdf/vRoR5w+TEGA8PMBFVCGb8zIBFh0aXIN63yD/u8kpNdpnDT6qcOj2Sb?=
 =?us-ascii?Q?THywKBH2DNInd9Li9nnEWDaTm+cuUgD13IWad833qjLScH0bz2PmODDGaJRa?=
 =?us-ascii?Q?s8b9sE7T1fy5n0gNykmDdkOPe0Ka79CJ8EedEdVUIDf30C3HsQL3JPC7xNku?=
 =?us-ascii?Q?RMtTb61tY+yxtuWs5SSGjeyl3G7evrKiZYD1ZhbFtlZUjLxTgQjFJFveXlmz?=
 =?us-ascii?Q?9Lz0QVC9WEkaHjYwrc4ov+e134ctbvxWtUT4njqm6ipErx7F4qbhoyDhBbp9?=
 =?us-ascii?Q?eZ9LPq7N9N9olz75xC9nLRLGkTjb0OLY8HaWlXq5sLz1sGqYiQEbyAIFYZJd?=
 =?us-ascii?Q?ixWvUB2VSoio4NEatY9YtcCW0HCbTA1HcfGllf9KxOF78+Bi4opgxLJV7w8S?=
 =?us-ascii?Q?SGLkjFt6dh/vDHDdIGbrNdb0eKnD32hlK7JbaMz1TKeU0H6vOq5no6Q/7+6V?=
 =?us-ascii?Q?Ax5gWduXsS6EYPGlZUuwxUbWFyUyYmI8hWcgSnIlno2N2g6Q/9juGZURNkLF?=
 =?us-ascii?Q?Hgvb2N6fbMus0XjhEP7jvHNcL79AAG6Rd+3gTHRzE6S7X3BpdsaC3bsO0CPS?=
 =?us-ascii?Q?jhfAbFpsEbpJUQT1dLWydTrOHTo31U93CaNOlL1s?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 6c6c124f-b96c-44a3-c9c3-08dc3156884b
X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6020.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Feb 2024 14:24:50.6472
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kq2UmlRv2I9HyhZRgmkTjkNAFFRum5LBVqC0pYc1zJC27BkiiKknQ+1mpG2jcLFiPio99Vo835D69y38cybdaQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR11MB7169
X-OriginatorOrg: intel.com

Hi Hillf,

On 2024-02-19 at 19:50:14 +0800, Hillf Danton wrote:
> On Tue, 21 Nov 2023 15:40:14 +0800 Chen Yu <yu.c.chen@intel.com>
> > Problem statement:
> > When task p is woken up, the scheduler leverages select_idle_sibling()
> > to find an idle CPU for it. p's previous CPU is usually a preference
> > because it can improve cache locality. However in many cases, the
> > previous CPU has already been taken by other wakees, thus p has to
> > find another idle CPU.
> > 
> > Proposal:
> > Introduce the SIS_CACHE. It considers the sleep time of the task for
> > better task placement. Based on the task's short sleeping history,
> > tag p's previous CPU as cache-hot. Later when p is woken up, it can
> > choose its previous CPU in select_idle_sibling(). When other task is
> > woken up, skip this cache-hot idle CPU when possible.
> > 
> > SIS_CACHE still prefers to choose an idle CPU during task wakeup,
> > the idea is to optimize the idle CPU scan sequence.
> 
> Could you specify why the currently selected cpu fails to work in the
> scenario described above?
>

Thank you for your review.

I assume that "currently select cpu" means "target". First, the "target"
could be chosen by wake_affine() -> wake_affine_weight(), which can
return non-idle candidate CPU. That is to say, when we reached below code,
the target and the prev could both be non-idle. Second, when target and
prev are both non-idle, select_idle_sibling() has to traverse the other CPUs
to find an idle one. What we do in SIS_CACHE is to increase the possibility
that the prev is idle and return directly in below code path.

Say, task p1's previous CPU is CPU1, and p1 is sleeping. With SIS_CACHE,
when another task p2 is woken up in select_idle_sibling()->select_idle_cpu(),
p2 tries to find CPU2 or CPU3 or CPU4... but not CPU1. This makes it easier
for p1 to find that CPU1 is idle when p1 is woken up, and choose CPU1 when
possible.

thanks,
Chenyu
> 	/*
> 	 * If the previous CPU is cache affine and idle, don't be stupid:
> 	 */
> 	if (prev != target && cpus_share_cache(prev, target) &&
> 	    (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
> 	    asym_fits_cpu(task_util, util_min, util_max, prev))
> 		return prev;