Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
Date:   Tue, 31 Oct 2023 14:18:23 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     Keisuke Nishimura <keisuke.nishimura@inria.fr>
CC:     Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        <linux-kernel@vger.kernel.org>,
        Shrikanth Hegde <sshegde@linux.vnet.ibm.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Mel Gorman <mgorman@suse.de>,
        Valentin Schneider <vschneid@redhat.com>,
        Julia Lawall <julia.lawall@inria.fr>
Subject: Re: [PATCH v2] sched/fair: Fix the decision for load balance
Message-ID: <ZUCcL28IrCCiLPx2@chenyu5-mobl2.ccr.corp.intel.com>
References: <20231030172945.1505532-1-keisuke.nishimura@inria.fr>
 <ZUCXwmzaoNbRbNpR@chenyu5-mobl2.ccr.corp.intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ZUCXwmzaoNbRbNpR@chenyu5-mobl2.ccr.corp.intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?r2mqWEWFihKEvavcZdQUdCdR/aUY6ZKOA5vxpMRMTu8kwNxPf/nb90tK0yH7?=
 =?us-ascii?Q?EX5/cfa/sobbXLJRhlQxeEpBymy3ex1kg9Rh4VNiw0nbgIZhJOk3ObBlAbv0?=
 =?us-ascii?Q?gQ2GjabEbDdB6ZPVjerctmuoaSVFpfQK0bIhbLQpmQ3gdiYSmTh8N/MzlJCc?=
 =?us-ascii?Q?bIdDT4C0h/8B/RECZLOI7BwMz7Fmnt+Np0Rg+OMiSTHETPLaZUv0NR/5Bday?=
 =?us-ascii?Q?s/fk8QNxK1Y2C4CX5HsN4raODpH4mYgyGALA4WJluDz/4C9reN6rzJta5Ncf?=
 =?us-ascii?Q?XW/VRUi2uy0ocIet6kQG09IQxrlb9b0BIHqVPFu04OfvB8A5H3UP9VXWKmXw?=
 =?us-ascii?Q?uhy1tKiM1oMPqOcryLWSpJOGuN542JN58uqBUyCEVWrOMcQ6+UPAGxb+L/3t?=
 =?us-ascii?Q?UD9RYioFjnPFza+2lUYFXcxfpcTJfjuEqJ58B/tbqqdaIP8b6vsyw38c2ZYa?=
 =?us-ascii?Q?cHi4/vRo8Z6kjoFyeykZsZVQegKVQpME+tXiT8gg4iOnNDDOJ8u3oLt0luXv?=
 =?us-ascii?Q?mhGy0i3IFB+puOBa5mX+R54O5M14gZ5gdK2sbrmqur9M7qT5gRU3ZkY2yc5D?=
 =?us-ascii?Q?vWHzI2GaTJHY2BjlRklRf7dsuqrTnd/CSti7gAiKL4YcJOqe74PQS0GW5T19?=
 =?us-ascii?Q?RXz+nGZocTeSdlip83NiSFLX+FmaalzlMirjgr6yAIJ7yOXIzoBnm8kKAsYu?=
 =?us-ascii?Q?COQl+AvG/r+sAKSve/xxe75LcN9xfaLSsaAUKxPnQLYLAfWvN/1xrE8qqmXf?=
 =?us-ascii?Q?eUn9pTyA2CJI2SN8RK4ZM34ef5AbeZXcugG7HdPD7NgR4SXH9eZl39aOUl0R?=
 =?us-ascii?Q?bQNeWljqi0kJ2yP8TlCA+h5uW6inI0QTtDzLLgig8L5PTYUHeq9o8GCiWd6x?=
 =?us-ascii?Q?BkiBsijtmX1g5ejJyZGny6iaYSb1PdyinbR8PMLgYxRR+PB/4OSbGxNWQOgC?=
 =?us-ascii?Q?YJZUGTPpcBST+DmRBYrcEKX+lwKD0QicNUX4Pb+WhO2qTql4DlXlHC485/q3?=
 =?us-ascii?Q?Pemea6ckeOK0+2l8P06z4bQJj8Hp42nWEuMbqkMwCSwanzDkb4cySq27l1Gh?=
 =?us-ascii?Q?QsxSeFz3pcbaWEjF+qxgb/hZ+2zVAJJ53Eji15VJn/1wzkewP7K9F8dlZuux?=
 =?us-ascii?Q?W1rcEheMpUO4hrTOBEZphQtkhuSl0DHp7ZtkiBRkuhAOdGDsBT1njSCupJWT?=
 =?us-ascii?Q?Tw7DY6lG+bTbAW5eXeavUbH8DwKHyG4wTwQkGd7qi6KrvAMXOo2rd3fcDSic?=
 =?us-ascii?Q?Lz+/jZgFUwD4stY88swc8f01BSfeRavfPq6lHpCc3ZTYiaug51+cJPHYqos8?=
 =?us-ascii?Q?wVBhrmRmPCUHkKDFT1fdLLiGOgDd3Bj97gqd5yTkGSQHFw+oykA/YyltFXNy?=
 =?us-ascii?Q?2jvyngaMfSrg1GCTucRKq7ZtHviyxnIVr/ASXIX8+0QTXjpTdfhHt7ZHkSFx?=
 =?us-ascii?Q?oQCgONrn02mz6Gda+15hNe/K2iy2oa0GFH/rvBv7HyzlH8AYdSFH8CoFAJ8J?=
 =?us-ascii?Q?2fqz08nUo12jZcAZGT47uhrLhnOZBhwFhEm+WS/hnC6WzTn0FqwMRkSqh0Fz?=
 =?us-ascii?Q?MBamOL5UVvN33I2K51nwpK1ZJgeWMqF8plMfe6RX?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 181d1857-efdf-46b2-3bce-08dbd9d93662
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2023 06:18:34.7874
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ADVgxPVSrO+i16phMx+txucl7IaITlgAeJYGqXYlYrUg69ESjyZfq7/pb7R9jj8Bd1t1zBFTZkDM9MyfFJ/wMw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB8573
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-1.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 30 Oct 2023 23:19:01 -0700 (PDT)

On 2023-10-31 at 13:59:30 +0800, Chen Yu wrote:
> On 2023-10-30 at 18:29:46 +0100, Keisuke Nishimura wrote:
> > should_we_balance is called for the decision to do load-balancing.
> > When sched ticks invoke this function, only one CPU should return
> > true. However, in the current code, two CPUs can return true. The
> > following situation, where b means busy and i means idle, is an
> > example, because CPU 0 and CPU 2 return true.
> > 
> >         [0, 1] [2, 3]
> >          b  b   i  b
> > 
> > This fix checks if there exists an idle CPU with busy sibling(s)
> > after looking for a CPU on an idle core. If some idle CPUs with busy
> > siblings are found, just the first one should do load-balancing.
> > 
> > Fixes: b1bfeab9b002 ("sched/fair: Consider the idle state of the whole core for load balance")
> > Signed-off-by: Keisuke Nishimura <keisuke.nishimura@inria.fr>
> > ---
> >  kernel/sched/fair.c | 10 +++++++---
> >  1 file changed, 7 insertions(+), 3 deletions(-)
> > 
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 2048138ce54b..69d63fae34f4 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -11079,12 +11079,16 @@ static int should_we_balance(struct lb_env *env)
> >  			continue;
> >  		}
> >  
> > -		/* Are we the first idle CPU? */
> > +		/*
> > +		 * Are we the first idle core in a MC or higher domain
> 
> It is possible that the Cluster domain is lower than a MC.
> cluser domain: CPUs share the same L2
> MC domain: CPUs share the same LLC
> 
>  grep . domain*/{name,flags}
> domain0/name:CLS
> domain1/name:MC
> domain2/name:NUMA
> domain0/flags:SD_BALANCE_NEWIDLE SD_BALANCE_EXEC SD_BALANCE_FORK SD_WAKE_AFFINE SD_SHARE_PKG_RESOURCES SD_PREFER_SIBLING 
> domain1/flags:SD_BALANCE_NEWIDLE SD_BALANCE_EXEC SD_BALANCE_FORK SD_WAKE_AFFINE SD_SHARE_PKG_RESOURCES SD_PREFER_SIBLING 
> domain2/flags:SD_BALANCE_NEWIDLE SD_BALANCE_EXEC SD_BALANCE_FORK SD_WAKE_AFFINE SD_SERIALIZE SD_OVERLAP SD_NUMA 
> 
> So, maybe:
> Are we the first idle core in a non-SMT domain or higher,
>

I suppose you can also carry the Reviewed-by tags in V1(Shrikanth and Vincent's)
as there is no code change.

Reviewed-by: Chen Yu <yu.c.chen@intel.com>

thanks,
Chenyu