Received-SPF: pass (google.com: domain of linux-kernel+bounces-9714-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
From: David Laight <David.Laight@ACULAB.COM>
To: 'Zeng Heng' <zengheng4@huawei.com>, "mingo@redhat.com" <mingo@redhat.com>,
	"will@kernel.org" <will@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "longman@redhat.com" <longman@redhat.com>,
	"boqun.feng@gmail.com" <boqun.feng@gmail.com>
CC: "xiexiuqi@huawei.com" <xiexiuqi@huawei.com>, "liwei391@huawei.com"
	<liwei391@huawei.com>, "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>
Subject: RE: [PATCH v2] locking/osq_lock: Avoid false sharing in
 optimistic_spin_node
Thread-Topic: [PATCH v2] locking/osq_lock: Avoid false sharing in
 optimistic_spin_node
Thread-Index: AQHaNM91saK4G5Mya06aToQ2I11sa7C1Om0A
Date: Fri, 22 Dec 2023 12:40:16 +0000
Message-ID: <9c48658677a6475eb4fe13baed798e97@AcuMS.aculab.com>
References: <20231222121040.2635879-1-zengheng4@huawei.com>
In-Reply-To: <20231222121040.2635879-1-zengheng4@huawei.com>
Accept-Language: en-GB, en-US
Precedence: bulk
MIME-Version: 1.0
Content-Language: en-US
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

From: Zeng Heng
> Sent: 22 December 2023 12:11
>=20
> Using the UnixBench test suite, we clearly find that osq_lock() cause
> extremely high overheads with perf tool in the File Copy items:
>=20
> Overhead  Shared Object            Symbol
>   94.25%  [kernel]                 [k] osq_lock
>    0.74%  [kernel]                 [k] rwsem_spin_on_owner
>    0.32%  [kernel]                 [k] filemap_get_read_batch
>=20
> In response to this, we conducted an analysis and made some gains:
>=20
> In the prologue of osq_lock(), it set `cpu` member of percpu struct
> optimistic_spin_node with the local cpu id, after that the value of the
> percpu struct would never change in fact. Based on that, we can regard
> the `cpu` member as a constant variable.
>=20
...
> @@ -9,7 +11,13 @@
>  struct optimistic_spin_node {
>  =09struct optimistic_spin_node *next, *prev;
>  =09int locked; /* 1 if lock acquired */
> -=09int cpu; /* encoded CPU # + 1 value */
> +
> +=09CACHELINE_PADDING(_pad1_);
> +=09/*
> +=09 * Stores an encoded CPU # + 1 value.
> +=09 * Only read by other cpus, so split into different cache lines.
> +=09 */
> +=09int cpu;
>  };

Isn't this structure embedded in every mutex and rwsem (etc)?
So that is a significant bloat especially on systems with
large cache lines.

Did you try just moving the initialisation of the per-cpu 'node'
below the first fast-path (uncontended) test in osq_lock()?

OTOH if you really have multiple cpu spinning on the same rwsem
perhaps the test and/or filemap code are really at fault!

=09David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1=
PT, UK
Registration No: 1397386 (Wales)