Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32;
Message-Id: <20230922200120.011184118@infradead.org>
User-Agent: quilt/0.65
Date:   Fri, 22 Sep 2023 22:01:20 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     tglx@linutronix.de, axboe@kernel.dk
Cc:     linux-kernel@vger.kernel.org, peterz@infradead.org,
        mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net,
        andrealmeid@igalia.com, Andrew Morton <akpm@linux-foundation.org>,
        urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com,
        Arnd Bergmann <arnd@arndb.de>, linux-api@vger.kernel.org,
        linux-mm@kvack.org, linux-arch@vger.kernel.org,
        malteskarupke@web.de, steve.shaw@intel.com,
        marko.makela@mariadb.com, andrei.artemev@intel.com
In-Reply-To: <20230921104505.717750284@noisy.programming.kicks-ass.net>
References: <20230921104505.717750284@noisy.programming.kicks-ass.net>
 <20230921104505.717750284@noisy.programming.kicks-ass.net>
Subject: futex2 numa stuff
Precedence: bulk

Hi!

Updated version of patch 15/15 and a few extra patches for testing the
FUTEX2_NUMA bits. The last patch (17/15) should never be applied for anything
you care about and exists purely because I'm too lazy to generate actual
hash-bucket contention.

On my 2 node IVB-EP:

 $ echo FUTEX_SQUASH > /debug/sched/features

Effectively reducing each node to 1 bucket.

 $ numactl -m0 -N0 ./futex_numa -c10 -t2 -n0 -N0 &
   numactl -m1 -N1 ./futex_numa -c10 -t2 -n0 -N0

 ...
 contenders: 16154935
 contenders: 16202472

 $ numactl -m0 -N0 ./futex_numa -c10 -t2 -n0 -N0 &
   numactl -m1 -N1 ./futex_numa -c10 -t2 -n0 -N1

 contenders: 48584991
 contenders: 48680560

(loop counts, higher is better)

Clearly showing how separating the hashes works. 

The first one runs 10 contenders on each node but forces the (numa) futex to
hash to node 0 for both. This ensures all 20 contenders hash to the same
bucket and *ouch*.

The second one does the same, except now fully separates the nodes. Performance
is much improved.

Proving the per-node hashing actually works as advertised.

Further:

 $ ./futex_numa -t2 -n50000 -s1 -N
 ...
 node: -1
 node: -1
 node: 0
 node: 0
 node: -1
 node: -1
 node: 1
 node: 1
 ...
 total: 8980

Shows how a FUTEX2_NUMA lock can bounce around the nodes. The test has some
trivial asserts trying to show critical section integrity, but otherwise does
lock+unlock cycles with a nanosleep.

This both illustrates how to build a (trivial) lock using FUTEX2_NUMA and
proves the functionality works.