Received-SPF: pass (google.com: domain of linux-kernel+bounces-211995-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Wed, 12 Jun 2024 10:23:00 -0700 (PDT)
From: "Christoph Lameter (Ampere)" <cl@linux.com>
To: Peter Zijlstra <peterz@infradead.org>
cc: tglx@linutronix.de, axboe@kernel.dk, linux-kernel@vger.kernel.org, 
    mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, 
    andrealmeid@igalia.com, Andrew Morton <akpm@linux-foundation.org>, 
    urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, 
    Arnd Bergmann <arnd@arndb.de>, linux-api@vger.kernel.org, 
    linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de
Subject: Re: [PATCH v1 11/14] futex: Implement FUTEX2_NUMA
In-Reply-To: <20230721105744.434742902@infradead.org>
Message-ID: <9dc04e4c-2adc-5084-4ea1-b200d82be29f@linux.com>
References: <20230721102237.268073801@infradead.org> <20230721105744.434742902@infradead.org>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII; format=flowed

On Fri, 21 Jul 2023, Peter Zijlstra wrote:

> Extend the futex2 interface to be numa aware.

Sorry to be chiming in that late but it seems that this is useful to 
mitigate NUMA issues also for our platform.

> When FUTEX2_NUMA is not set, the node is simply an extention of the
> hash, such that traditional futexes are still interleaved over the
> nodes.


Could we follow NUMA policies like with other metadata allocations during 
systen call processing? If there is no NUMA task policy then the futex
should be placed on the local NUMA node.

That way the placement of the futex can be controlled by the tasks memory 
policy. We could skip the FUTEX2_NUMA option.

> @@ -114,10 +137,29 @@ late_initcall(fail_futex_debugfs);
>  */
> struct futex_hash_bucket *futex_hash(union futex_key *key)
> {
> -	u32 hash = jhash2((u32 *)key, offsetof(typeof(*key), both.offset) / 4,
> +	u32 hash = jhash2((u32 *)key,
> +			  offsetof(typeof(*key), both.offset) / sizeof(u32),
> 			  key->both.offset);
> +	int node = key->both.node;
>
> -	return &futex_queues[hash & (futex_hashsize - 1)];
> +	if (node == -1) {

> +		/*
> +		 * In case of !FLAGS_NUMA, use some unused hash bits to pick a
> +		 * node -- this ensures regular futexes are interleaved across
> +		 * the nodes and avoids having to allocate multiple
> +		 * hash-tables.
> +		 *
> +		 * NOTE: this isn't perfectly uniform, but it is fast and
> +		 * handles sparse node masks.
> +		 */
> +		node = (hash >> futex_hashshift) % nr_node_ids;
> +		if (!node_possible(node)) {
> +			node = find_next_bit_wrap(node_possible_map.bits,
> +						  nr_node_ids, node);
> +		}

Use memory allocation policies here instead?