Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Fri, 28 Jan 2022 16:13:54 +0100
From:   Peter Zijlstra <peterz@infradead.org>
To:     "Huang, Ying" <ying.huang@intel.com>
Cc:     Srikar Dronamraju <srikar@linux.vnet.ibm.com>,
        Mel Gorman <mgorman@suse.de>, linux-kernel@vger.kernel.org,
        Ingo Molnar <mingo@redhat.com>, Rik van Riel <riel@surriel.com>
Subject: Re: [RFC PATCH 1/2] NUMA balancing: fix NUMA topology type for
 memory tiering system
Message-ID: <YfQIMmbY7nHusQRK@hirez.programming.kicks-ass.net>
References: <20220128023842.1946583-1-ying.huang@intel.com>
 <20220128052345.GA618915@linux.vnet.ibm.com>
 <87czkctiz9.fsf@yhuang6-desk2.ccr.corp.intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87czkctiz9.fsf@yhuang6-desk2.ccr.corp.intel.com>
Precedence: bulk

On Fri, Jan 28, 2022 at 03:30:50PM +0800, Huang, Ying wrote:
> Srikar Dronamraju <srikar@linux.vnet.ibm.com> writes:
> 
> > * Huang Ying <ying.huang@intel.com> [2022-01-28 10:38:41]:
> >
> >> 
> >> One possible fix is to ignore CPU-less nodes when detecting NUMA
> >> topology type in init_numa_topology_type().  That works well for the
> >> example system.  Is it good in general for any system with CPU-less
> >> nodes?
> >> 
> >
> > A CPUless node at the time online doesn't necessarily mean a CPUless node
> > for the entire boot. For example: On PowerVM Lpars, aka powerpc systems,
> > some of the nodes may start as CPUless nodes and then CPUS may get
> > populated/hotplugged on them.
> 
> Got it!
> 
> > Hence I am not sure if adding a check for CPUless nodes at node online may
> > work for such systems.
> 
> How about something as below?

I'm thinking that might not be enough in that scenario; if we're going
to consistently skip CPU-less nodes (as I really think we should) then
__sched_domains_numa_masks_set() is not sufficient for the hotplug case
since sched_domains_numa_levels and sched_max_numa_distance can also
change.

This means we need to re-do more of sched_init_numa() and possibly
re-alloc some of those arrays etc..

Same for offline ofc.