Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 2 Aug 2023 13:28:36 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Aaron Lu <aaron.lu@intel.com>
Cc:     Ingo Molnar <mingo@redhat.com>, Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Daniel Jordan <daniel.m.jordan@oracle.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Tim Chen <tim.c.chen@intel.com>,
        Nitin Tekchandani <nitin.tekchandani@intel.com>,
        Yu Chen <yu.c.chen@intel.com>,
        Waiman Long <longman@redhat.com>, linux-kernel@vger.kernel.org,
        yury.norov@gmail.com, andriy.shevchenko@linux.intel.com,
        linux@rasmusvillemoes.dk, rppt@kernel.org
Subject: Re: [RFC PATCH 2/4] sched/fair: Make tg->load_avg per node
Message-ID: <20230802112836.GA212435@hirez.programming.kicks-ass.net>
References: <20230718134120.81199-1-aaron.lu@intel.com>
 <20230718134120.81199-3-aaron.lu@intel.com>
 <20230719115358.GB3529734@hirez.programming.kicks-ass.net>
 <20230719134500.GB91858@ziqianlu-dell>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230719134500.GB91858@ziqianlu-dell>
Precedence: bulk

On Wed, Jul 19, 2023 at 09:45:00PM +0800, Aaron Lu wrote:
> On Wed, Jul 19, 2023 at 01:53:58PM +0200, Peter Zijlstra wrote:
> > On Tue, Jul 18, 2023 at 09:41:18PM +0800, Aaron Lu wrote:
> > > +#if defined(CONFIG_FAIR_GROUP_SCHED) && defined(CONFIG_SMP)
> > > +static inline long tg_load_avg(struct task_group *tg)
> > > +{
> > > +	long load_avg = 0;
> > > +	int i;
> > > +
> > > +	/*
> > > +	 * The only path that can give us a root_task_group
> > > +	 * here is from print_cfs_rq() thus unlikely.
> > > +	 */
> > > +	if (unlikely(tg == &root_task_group))
> > > +		return 0;
> > > +
> > > +	for_each_node(i)
> > > +		load_avg += atomic_long_read(&tg->node_info[i]->load_avg);
> > > +
> > > +	return load_avg;
> > > +}
> > > +#endif
> > 
> > So I was working on something else numa and noticed that for_each_node()
> > (and most of the nodemask stuff) is quite moronic, afaict we should do
> > something like the below.
> > 
> > I now see Mike added the nr_node_ids thing fairly recent, but given
> > distros have NODES_SHIFT=10 and actual machines typically only have <=4
> > nodes, this would save a factor of 256 scanning.

More complete nodemask patch here:

  https://lkml.kernel.org/r/20230802112458.230221601%40infradead.org