Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Sat, 22 Apr 2023 14:04:50 +0800
From:   Aaron Lu <aaron.lu@intel.com>
To:     Chen Yu <yu.c.chen@intel.com>
CC:     Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>,
        "Mel Gorman" <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Tim Chen <tim.c.chen@intel.com>,
        Nitin Tekchandani <nitin.tekchandani@intel.com>,
        Waiman Long <longman@redhat.com>,
        <linux-kernel@vger.kernel.org>
Subject: Re: [RFC PATCH] sched/fair: Make tg->load_avg per node
Message-ID: <20230422060450.GA264770@ziqianlu-desk2>
References: <20230327053955.GA570404@ziqianlu-desk2>
 <ZENcN3suKMCaHm1w@chenyu5-mobl1>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ZENcN3suKMCaHm1w@chenyu5-mobl1>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?CL47207t4YpIlZN7jVzfJqHX+SF94GAJOMb6z0RVaPNwyHOLxEZE3bpg6lj/?=
 =?us-ascii?Q?wSuVKFSG1F5cJ1dV2hha70drKayk9f9/Axw4YHOvqFmoXlr1NlCQh3cp0ylD?=
 =?us-ascii?Q?51W48Hsfiy0r1Ll58La0dmOWOBDJOy11mxSxxE0+9/gQKyNbtHi5ntxTDbSr?=
 =?us-ascii?Q?B7P2IJb3J3euxBQmIHDFNXo2OkIv5EtalDcYVSvBYZXEdYoHVYsmSDwQEZ6R?=
 =?us-ascii?Q?tG4lo8Q3JJzoUlpAPT3gB+TfHBFhpMhifZRtN1W1WM3xxGsR7KXecYYN3xg6?=
 =?us-ascii?Q?clE3nYCEtpWEq5KXlnKW1s5wJfWPX38f8cpV88+kOSq+ctBGUfALmsQotZFA?=
 =?us-ascii?Q?RaAH6N6TECucoQ/tq49elOa7lYcBPcM5No68GeW18HswxLS+lh6HzN9gmCGW?=
 =?us-ascii?Q?+1+4+BBy6KofS3O8e2enZ0BOlDj/YACrjLVFmfYcpIYu0XZERqpa1i8WpJMI?=
 =?us-ascii?Q?Fu1++crpgDegWa9Ad2PenUup2kIGsPx5Urf/YPS/kLQacgskYGi1iwWfw+56?=
 =?us-ascii?Q?Or38NiXcTore5DRUk/otc9mzlYyc2bdA60LIZQfnK1f4P6X/yPntQxXITnFn?=
 =?us-ascii?Q?Zw5VG/9ndCsOSBnd7BlTd8sLZAnjCRJzMvJgXTPP5T+ieL56NKXLBV4OGd5K?=
 =?us-ascii?Q?GZYZGyNjtTCYdkZD0hJ0WBlrbDXdhIMt4Wa1qtN8OB5QHL4bewgVZ4NoZH5u?=
 =?us-ascii?Q?8Bag0HtuPgYGLC4ldxZviPkW63GMnOyVMhO6lLgO8Dsbm1zCKaKX8JX/QGcA?=
 =?us-ascii?Q?0MXeki/ss4AkDh7M07Asfeq/04g7ulK/FhuDJVnim81V/BpsclItT1cd2gGE?=
 =?us-ascii?Q?Hxfu2jC7BwNeX37w086l9AceT9dAy+aaR9JBtU0WSYsp2tGCOPTKDkc9A6RO?=
 =?us-ascii?Q?6PSksryNkEHibEc1I/R7/gXYJ7DLdLuRcmlVxyPvjCPO0d17G/7LTn8p5Fnr?=
 =?us-ascii?Q?bqqz33lp93WrwrvtzEQBYzufv/YhJIHfde4o22PMVlWhdT7a5HaYFh5F5HZu?=
 =?us-ascii?Q?wlKHB7vJ3uR7kehJMTixD3k2dYe9SVTe8PdPuzo5HYroXt02r9b4O4Pbdr+n?=
 =?us-ascii?Q?W1qV1ef/9xYlTA0UxnorFZ4/Sk9YKV3OjCVYa16Yh3NYZqST8TWNFodOLq3h?=
 =?us-ascii?Q?Dez8hAO4J/IJXhjVGo2LQZR5L0/fFzqF5znNIE5rhEJax2MLbVZDLQPXfhFN?=
 =?us-ascii?Q?n6Q3odOsR3B1TsmZB1EB+U54PR+Ms5hRSbFQuXvJeCv6vJ742j+8Sf6kHbbs?=
 =?us-ascii?Q?qs7ezkzIPNBRvYpqdD+AaDjYg+cfcWTsLToGA+Q8sk0p0UXE30rtsL/iIFD2?=
 =?us-ascii?Q?9HKDyHsP/icstguAXzQnWzq7tdYhOEkWli/Psi6DrsFJ4wJVKC7OLvY7GL2k?=
 =?us-ascii?Q?5oJGnf1XXJH56r9BxwuPz4NooZ+iItsZ5B9tI+fwVCkkoRITxNlEmmd9Mq4o?=
 =?us-ascii?Q?qGRmwol5ivPjoKWz0ZGCK25dgmPk4/9rQ56jddc/1ywv99kkPnC07Ermk5y/?=
 =?us-ascii?Q?9t83TZyY5ORUVroA83zjW/ZHmv+7+25y4BxCWp3Kw2qi7h0Q/WLMavH1gmbK?=
 =?us-ascii?Q?T9UMlpvYK57FzlHqBF/LuN95ze97WTKbR6wcxAuh?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 66952aa3-2d45-4184-c5f5-08db42f78683
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3062.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2023 06:05:08.8379
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gKSDm04OLCzH3bYqTTf8Y7tXTj57G8gEOV2SziDFHxANUpI9sBbKfJ0WF9gKtTC+zDHP0NpIAgE2nkr94g6G7A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4743
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sat, Apr 22, 2023 at 12:01:59PM +0800, Chen Yu wrote:
> On 2023-03-27 at 13:39:55 +0800, Aaron Lu wrote:
> > 
> > The reason why only cpus of one node has bigger overhead is: task_group
> > is allocated on demand from a slab and whichever cpu happens to do the
> > allocation, the allocated tg will be located on that node and accessing
> > to tg->load_avg will have a lower cost for cpus on the same node and
> > a higer cost for cpus of the remote node.
> [...]
> >  static inline void update_tg_load_avg(struct cfs_rq *cfs_rq)
> >  {
> >  	long delta = cfs_rq->avg.load_avg - cfs_rq->tg_load_avg_contrib;
> > +	int node = cpu_to_node(cfs_rq->rq->cpu);
> >  
> >  	/*
> >  	 * No need to update load_avg for root_task_group as it is not used.
> > @@ -3616,7 +3617,7 @@ static inline void update_tg_load_avg(struct cfs_rq *cfs_rq)
> >  		return;
> >  
> >  	if (abs(delta) > cfs_rq->tg_load_avg_contrib / 64) {
> > -		atomic_long_add(delta, &cfs_rq->tg->load_avg);
> > +		atomic_long_add(delta, &cfs_rq->tg->node_info[node]->load_avg);
> When entered enqueue_entity(cfs_rq, se) -> update_tg_load_avg(cfs_rq)
> the cfs_rq->rq->cpu is not necessary the current cpu, so the node returned by
> cpu_to_node(cfs_rq->rq->cpu) is not necessary the current node as the current
> CPU, would atomic_add still introduce cross-node overhead due to remote access
> to cfs_rq->tg->node_info[node]->load_avg in this case, or do I miss something?

That's good point.

The chance cpu being different is high, but the node being different is
pretty low though. The wakeup path will not do cross LLC activation with
TTWU_QUEUE, but the load balance path does make it possible to dequeue
a task that is on a remote node and I think that's the only path where
the two cpus(current vs cfs_rq->rq->cpu) can be on different nodes.

A quick test shows during a 5s window of running hackbench on a VM, the
number of times when the two nodes derived from current and cfs_rq->rq->cpu
being different are less than 100 while being equal are 992548.

I'll switch to using smp_processor_id() in next posting since it's the
right thing to do, thanks for spotting this!