Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Fri, 28 Jan 2022 11:03:41 +0530
From:   Srikar Dronamraju <srikar@linux.vnet.ibm.com>
To:     Huang Ying <ying.huang@intel.com>
Cc:     Peter Zijlstra <peterz@infradead.org>,
        Mel Gorman <mgorman@suse.de>, linux-kernel@vger.kernel.org,
        Ingo Molnar <mingo@redhat.com>, Rik van Riel <riel@surriel.com>
Subject: Re: [RFC PATCH 2/2] NUMA balancing: avoid to migrate task to
 CPU-less node
Message-ID: <20220128053341.GB618915@linux.vnet.ibm.com>
Reply-To: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
References: <20220128023842.1946583-1-ying.huang@intel.com>
 <20220128023842.1946583-2-ying.huang@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
In-Reply-To: <20220128023842.1946583-2-ying.huang@intel.com>
Precedence: bulk

* Huang Ying <ying.huang@intel.com> [2022-01-28 10:38:42]:

> In a typical memory tiering system, there's no CPU in slow (PMEM) NUMA
> nodes.  But if the number of the hint page faults on a PMEM node is
> the max for a task, The current NUMA balancing policy may try to place
> the task on the PMEM node instead of DRAM node.  This is unreasonable,
> because there's no CPU in PMEM NUMA nodes.  To fix this, CPU-less
> nodes are ignored when searching the migration target node for a task
> in this patch.
> 
> To test the patch, we run a workload that accesses more memory in PMEM
> node than memory in DRAM node.  Without the patch, the PMEM node will
> be chosen as preferred node in task_numa_placement().  While the DRAM
> node will be chosen instead with the patch.
> 
> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Rik van Riel <riel@surriel.com>
> Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
> ---
>  kernel/sched/fair.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 54e1aad1c5d7..e462ac5c1e48 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -2393,6 +2393,10 @@ static void task_numa_placement(struct task_struct *p)
>  			}
>  		}
> 
> +		/* Cannot migrate task to CPU-less node */
> +		if (!node_state(nid, N_CPU))
> +			continue;
> +

Lets take the example that you quoted 2 socket machine with 1 DRAM node and
1 PMEM node per socket.  Now lets say the task is placed on a CPU in node 1
but most of its memory faults are coming from node 2, which is the PMEM node
attached to node 0. Now without the hunk, there is a chance that the task
got moved to node 0. However with the change, are we inhibiting such a move?

>  		if (!ng) {
>  			if (faults > max_faults) {
>  				max_faults = faults;
> -- 
> 2.30.2
> 

-- 
Thanks and Regards
Srikar Dronamraju