Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
Message-ID: <e9f8466e-d138-3cc6-11c6-2d62f4c9dc4a@intel.com>
Date:   Mon, 25 Sep 2023 15:46:25 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Subject: Re: [PATCH 3/4] Revert "sched/topology: Introduce
 sched_numa_hop_mask()"
Content-Language: en-US
To:     Yury Norov <yury.norov@gmail.com>, <linux-kernel@vger.kernel.org>,
        <netdev@vger.kernel.org>, <linux-rdma@vger.kernel.org>
CC:     Tariq Toukan <ttoukan.linux@gmail.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Maher Sanalla <msanalla@nvidia.com>,
        Ingo Molnar <mingo@kernel.org>, Mel Gorman <mgorman@suse.de>,
        Saeed Mahameed <saeedm@nvidia.com>,
        Leon Romanovsky <leon@kernel.org>,
        "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Pawel Chmielewski <pawel.chmielewski@intel.com>,
        Yury Norov <ynorov@nvidia.com>
References: <20230925020528.777578-1-yury.norov@gmail.com>
 <20230925020528.777578-4-yury.norov@gmail.com>
From:   Jacob Keller <jacob.e.keller@intel.com>
In-Reply-To: <20230925020528.777578-4-yury.norov@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bGdOcWJNUUE1WFRDT0xRWnpaWFZQRXpTRzhUd2tGamNNeFZpVXBZRTdzQzhJ?=
 =?utf-8?B?R0hGNXpkMnpzcldjUkt6L29JcElsUW9PUmdzWmRDakNtb093R0hsdGFTWXoz?=
 =?utf-8?B?RUV5bkM5OG8xL3JReXpNT2k2eWdpYUYvS3NWcXhmTnE1TEprcTlXTEpnL28y?=
 =?utf-8?B?bWhoR2ZyOGRoSkd3QndUQVlyenhIaW9Fa1RId3B1aUNKdUVWQzhrSElxbWZo?=
 =?utf-8?B?eUp6WGdueEZtKzFqY2F1Q2VlYm9PZWdSd0l4M2NoTGw0c05vTzg3eXo4dWRT?=
 =?utf-8?B?aHdaOW9CcDk4Ym1hd1F3cElRTU5BbWhLUzV1aS9wY0pFQ2dWSWk5UjRoTkcz?=
 =?utf-8?B?RlBGTU51UEsrTFVuNktRYVF3K0tNVks1K2hFYnhmMmdyQmdsQS9mYkU0RzZG?=
 =?utf-8?B?VWhubk8xTU5iRDBUc2tzV1M4bFBnNXVab09hWFNocWRVQ1hwK0E3N3cwb3hC?=
 =?utf-8?B?RnpFT3NIak9ZS2w0QXltRytUQ0lsSHppSjRrNFNaQU02S21QdTYxWTRHU1JQ?=
 =?utf-8?B?SE8zTTh4ZkhJS0MrUWJVMzhETjc5VHVFRTMySzBRR2VDUDVvd0xPcDR6Yk42?=
 =?utf-8?B?d040ZkxacGZVMXZ5Y2pJWHlvRytWMnNsdHMyVlU0M2t5aDVOOXFLYloxNTli?=
 =?utf-8?B?dTdTdzNaTDJGakZSYXpyVHArVWVTd1A4cGJkNFN5dkxmL1dXMWpON1Z4Rzlr?=
 =?utf-8?B?MzgvaVBrbnhWSmsybklZb1lVVWRORWtJdEhKL0Q5TzVXaHE2bFhvQVFmUERs?=
 =?utf-8?B?dElkcXdJT0pBNlhkeHN4bUx3Yzh2cUJCTmZWQ2gxeUZxVXBhdzdvTWdSNElq?=
 =?utf-8?B?RWlLSVA2QTlSQ1ZhblZpV2lzMy9uZC9laXExT09xOEpQTmJUTlNMeUVmaWlX?=
 =?utf-8?B?RTNoWFFWd2tZM2t6SGdPYmdMQUtaZkF3SFJ0anZlMmhnOXN5WCtUcFNnaUt4?=
 =?utf-8?B?a0Ntb3BENUFueHhpK3Z1a2p4SEljMWtJenNUcWRzRmpZbnY4a2xLWm1zYjFB?=
 =?utf-8?B?cWVJVjJlU1JXS0FpNEFLUFY3ODBLREtmcWFFQnZsZmkySlFVaGwvTSt6MmJu?=
 =?utf-8?B?QTR3TjRUNFFLOTlkMm0vMGpHNU50ZUNrays5dldYSlZvQlhaKzBEK2hYSjhw?=
 =?utf-8?B?ajBUVHBEb3IzaDdVM09wSVFnWE82aXdkQmFEN1BxbHZmMFg2WDJSUm11WUg3?=
 =?utf-8?B?Ry9DajhOeW9PVCtLaHZRbGQxbFYvb2w0S2trNHpMck91dnJINnN5anQ5SnNC?=
 =?utf-8?B?aWtBam5jVWNyOWJJOVhqOStPekdlUUNFU21FNmJRbWlYV2QwK1ZqNllTa21l?=
 =?utf-8?B?Y3ptcDQyaVJXTlRYNXhjSU85NUlmVEgzYkluL2Q1K3dMTDdpZXAzZFRFN3hT?=
 =?utf-8?B?djdaWWhwVXdIWTJ3SUlaZHQydVpSTVRiMFhjSkNWM2QvMHk4eXJZMi9WWmE4?=
 =?utf-8?B?ZTRXVEFQM3lVbnUweXBqQTY5N1JRMDE1NGJHYjNLT0pxSk9vQkl3STJOODcw?=
 =?utf-8?B?eEFYWUdHOEtFRnpDVjhuWkJ6UEI0ZGJKWHkyVnpnWVFwTnpteWxXVmNEWFU2?=
 =?utf-8?B?NVVtYlJyY1BoOVFXblYyTGxQbGpZRlV1aktRMEdpQ3FOL2pDTEt6bjhrSzMz?=
 =?utf-8?B?V1EwUndSbTQxNXA5UHdOOW5zaHRyUFNYdXJhaEVVSVNNV3FwWHZ3RTloZkV1?=
 =?utf-8?B?NnViRXgxQ1hBekpITlg2bnFOOVdmVlVkZE9ZM1FyWVBObkFrTkFTUFNGV0pt?=
 =?utf-8?B?SDJsM2VRMGZYVDlZQWJjSVphd0hsdzZiUHJVVE5WWUpySXVFZm1PLytvbmRC?=
 =?utf-8?B?R1VIYWo2MmUrOTE1TGlPcWZYaHZhQWtCdVZqT2xSakdhSHgvNWRyMFIvYXRx?=
 =?utf-8?B?Nmc0WmZoZXE4dWdmRFdzazRpVHEvTU1iZTcwSm42TEEza2xJM1ZrTXoyd2FE?=
 =?utf-8?B?ZHhNWm9ieURteUFuaVZaKzlDa2NGdTRlTTVwTWZteEFwR29ZMkJHb3F3Mm81?=
 =?utf-8?B?RGhwbmFjTXFncGxXOGNqanduWlJ0WEdXSCtzeExaNCs2V2R4UFJEWVJLRUV6?=
 =?utf-8?B?Rjc4dnFuZ0gycEd4bFJyMUZOOG1YNHYyZWR3Z1haeGt3enpQVnAyT1luUWRR?=
 =?utf-8?B?UGZ6V2gxK2MvaDA1VlNPU21Xd2tWTjg1NHdYSmdlK25PVldpVkpSVWZ1UTkv?=
 =?utf-8?B?VEE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 18eb968f-b105-4135-d87c-08dbbe19413f
X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB5089.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Sep 2023 22:46:28.2323
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: X9NBLLZtdJI0zNiCKvsoHcatpMrYGVRzEXRZMsGwU4FLMNRHW1c/87HP5Av47SGoVtj9czQHkRHcird3Wm/6Rkc3LCj5C9q9Zoh1CpzAN0Y=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB5829
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-3.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 25 Sep 2023 15:46:38 -0700 (PDT)


On 9/24/2023 7:05 PM, Yury Norov wrote:
> This reverts commit 9feae65845f7b16376716fe70b7d4b9bf8721848.
> 
> Now that for_each_numa_hop_mask() is reverted, revert underlying
> machinery.
> 
> Signed-off-by: Yury Norov <yury.norov@gmail.com>
> Signed-off-by: Yury Norov <ynorov@nvidia.com>
> ---
>  include/linux/topology.h |  7 -------
>  kernel/sched/topology.c  | 33 ---------------------------------
>  2 files changed, 40 deletions(-)
> 


Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>

> diff --git a/include/linux/topology.h b/include/linux/topology.h
> index 344c2362755a..72f264575698 100644
> --- a/include/linux/topology.h
> +++ b/include/linux/topology.h
> @@ -247,18 +247,11 @@ static inline const struct cpumask *cpu_cpu_mask(int cpu)
>  
>  #ifdef CONFIG_NUMA
>  int sched_numa_find_nth_cpu(const struct cpumask *cpus, int cpu, int node);
> -extern const struct cpumask *sched_numa_hop_mask(unsigned int node, unsigned int hops);
>  #else
>  static __always_inline int sched_numa_find_nth_cpu(const struct cpumask *cpus, int cpu, int node)
>  {
>  	return cpumask_nth(cpu, cpus);
>  }
> -
> -static inline const struct cpumask *
> -sched_numa_hop_mask(unsigned int node, unsigned int hops)
> -{
> -	return ERR_PTR(-EOPNOTSUPP);
> -}
>  #endif	/* CONFIG_NUMA */
>  
>  #endif /* _LINUX_TOPOLOGY_H */
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index 05a5bc678c08..3f1c09a9ef6d 100644
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -2143,39 +2143,6 @@ int sched_numa_find_nth_cpu(const struct cpumask *cpus, int cpu, int node)
>  	return ret;
>  }
>  EXPORT_SYMBOL_GPL(sched_numa_find_nth_cpu);
> -
> -/**
> - * sched_numa_hop_mask() - Get the cpumask of CPUs at most @hops hops away from
> - *                         @node
> - * @node: The node to count hops from.
> - * @hops: Include CPUs up to that many hops away. 0 means local node.
> - *
> - * Return: On success, a pointer to a cpumask of CPUs at most @hops away from
> - * @node, an error value otherwise.
> - *
> - * Requires rcu_lock to be held. Returned cpumask is only valid within that
> - * read-side section, copy it if required beyond that.
> - *
> - * Note that not all hops are equal in distance; see sched_init_numa() for how
> - * distances and masks are handled.
> - * Also note that this is a reflection of sched_domains_numa_masks, which may change
> - * during the lifetime of the system (offline nodes are taken out of the masks).
> - */
> -const struct cpumask *sched_numa_hop_mask(unsigned int node, unsigned int hops)
> -{
> -	struct cpumask ***masks;
> -
> -	if (node >= nr_node_ids || hops >= sched_domains_numa_levels)
> -		return ERR_PTR(-EINVAL);
> -
> -	masks = rcu_dereference(sched_domains_numa_masks);
> -	if (!masks)
> -		return ERR_PTR(-EBUSY);
> -
> -	return masks[hops][node];
> -}
> -EXPORT_SYMBOL_GPL(sched_numa_hop_mask);
> -
>  #endif /* CONFIG_NUMA */
>  
>  static int __sdt_alloc(const struct cpumask *cpu_map)