Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Liam Howlett <liam.howlett@oracle.com>
To:     "maple-tree@lists.infradead.org" <maple-tree@lists.infradead.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        "damon @ lists . linux . dev" <damon@lists.linux.dev>,
        SeongJae Park <sj@kernel.org>,
        David Hildenbrand <david@redhat.com>
Subject: [PATCH v10 05/69] radix tree test suite: add support for slab bulk
 APIs
Thread-Topic: [PATCH v10 05/69] radix tree test suite: add support for slab
 bulk APIs
Thread-Index: AQHYhbAHVW8soav6GUSOJoDTfQbtPQ==
Date:   Tue, 21 Jun 2022 20:46:49 +0000
Message-ID: <20220621204632.3370049-6-Liam.Howlett@oracle.com>
References: <20220621204632.3370049-1-Liam.Howlett@oracle.com>
In-Reply-To: <20220621204632.3370049-1-Liam.Howlett@oracle.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?cQUWFpO5F1YeuctS9wvBMtrokdQ2DgKclugrmsgkPKOElckkq9hgWtz4yr?=
 =?iso-8859-1?Q?i7OWI8XVCJqgMp5Y0/TmUQGyC3A9UNmEW07WKYC6+LLjKgLn0g7I6x4oX7?=
 =?iso-8859-1?Q?6iVTsCetRsn0OhjoJrMuVDcf31qdXlz0Roha6PcDe5BAQuwrjZZAdZwac5?=
 =?iso-8859-1?Q?8LCwyuADa/o3arnamlr8pM5AUnDQOtBjbsx1WR85noqw/C8oLOdUXjrhdf?=
 =?iso-8859-1?Q?FhHJGKoGgXqoDy3u1TXEezWuxjdOTdSxRTCA780GOjRNVIIpgzoDEywp/B?=
 =?iso-8859-1?Q?9fF0ziwZACIHbf2F/AnqhsTqgVwCfe1ThtGGvFjcdMdNiPZ8zor/Z5zmLG?=
 =?iso-8859-1?Q?lMEQXw5l66k1Ml6tTc60JxSh3y/OmkUZ0sgsnRr+mdj3wM607oUgWPfIxi?=
 =?iso-8859-1?Q?cu7eK1qS0PLXAODIuy64NolllJ7HTr+EczHu91Fkw07TTnoOAAKnEZhISn?=
 =?iso-8859-1?Q?guYaO/F/a8xATbNO4NCHg3hi/piycecaaXEDv+3cut8Sj3fgVR2LaHwvBm?=
 =?iso-8859-1?Q?lVw5DqXpgSMUNE093JmGvjve2DPiDT6aJ+xuOU4Q8MWIl9pB2e3b9NiJJO?=
 =?iso-8859-1?Q?qNh0j4VZGQdqQhBtGqT7zNvojHrxbQGNUe+o/kk6iM8KsqJZeJZH+FtgJ1?=
 =?iso-8859-1?Q?GjNlKT0miWhH3tOBUq5e282S60KezqOGAPlmZLVBCFm1NKrM4fbLNOv/cI?=
 =?iso-8859-1?Q?nmMdqocT1lblHmReDPYjjF4A0us40etId1kpSCFgyH4Tgphy8wJlzVB41T?=
 =?iso-8859-1?Q?mIiYaMSzuT+06nVaVjQOaxyVmB0/JBv248PC+UuAq34uYDMdCy8qJ98ikE?=
 =?iso-8859-1?Q?QtsLt0uLi/80/4WpJaNYpqSXqZ6cT/0ojrmLlM4qaiax5pyvpJ26hcKncN?=
 =?iso-8859-1?Q?2bfvtYOXzcdQvxjQdkQhq8n3ikXQvjRYKT+0ZLzqSEKmufuFNQWzob48pP?=
 =?iso-8859-1?Q?MKnRsQVut4TIaqleBKY1tIXalX6IprogrAM4D6UmposZ9MUa89Jsfnc2yA?=
 =?iso-8859-1?Q?7/a6NB4C+6CgNVe2nrTGOS67B9nSpOK00pRDpI3tWQO+GCFcJddibCLUdH?=
 =?iso-8859-1?Q?7+AmH6LjrdDoDbiBnqS6ts/GE6c0w5eUKZ/fOc7zvaYyGeUeghZdy6hNLj?=
 =?iso-8859-1?Q?JYaz3PBwWkxcK+CDRUmE9UnYiTVT5pHz9SHxmXHQCXhRxDklpqkJvpnqZm?=
 =?iso-8859-1?Q?W7+T4hEnDoirFCP0Lbua5QzxCBdbloeadS47bCdi+Y3WPzEIBbK46c7g26?=
 =?iso-8859-1?Q?QPYUZGSmKBre7PmmBFD+VhOygOvNtGw/H1dSUSyC8uL8spq5qyBWNtYGGs?=
 =?iso-8859-1?Q?LYncA70amoAJ6oiVKxjB+4w8hIsL+VgOMjZA86P1L81gMq0OsdxrS8JgHt?=
 =?iso-8859-1?Q?tR5jxTGWykwA63fFkkbN5xsVfLX2t7cIQiVQFBGioZfO3J3k4RIok5csWd?=
 =?iso-8859-1?Q?WVAzUdXNJrSdJyWhDWBxN5XxKE7vwltbGt7PR3DstwnhhIipzNfYyJJBbU?=
 =?iso-8859-1?Q?Jrlzjgvl1UoiBJv4zzo+4BbHLs1Z4no3qUNdoBL97VbAKRK+IsBvo8blTr?=
 =?iso-8859-1?Q?LsKCB26w58PgM9qw8drm4XP4bzQssvS1CH1rd2Ez7mS4psvU4k4Vtd2BnN?=
 =?iso-8859-1?Q?FjIsGRb2cIfUMG3Y2TXatuy5eFo7Zwa/MD+mT5VlS4PSpXV9Fl61Z5jAkr?=
 =?iso-8859-1?Q?/TeymtWBCVMI8smSta7aPeL5RxgUuvaF1PIadrOQatEPk246jgww6WJtSt?=
 =?iso-8859-1?Q?QvpQ6fz6le6nVwxiMTJJz3z/zJQy3b92Uju1++YDKx9DpM+JrCOBF5Sfdv?=
 =?iso-8859-1?Q?c/DSirPsFAUXXmXtVhmBNQf1CBxodyo=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 087a3a73-a87f-4761-257d-08da53c72a8a
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jun 2022 20:46:49.0672
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: CXnjo3S0Za14AVYFfTuhN192GzC3ivounG4IySDDOSLKmtH6b1bpo9n1s5Ld8vuDANfspJSfuW8dw+/l/LtwWw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR10MB3074
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.883
 definitions=2022-06-21_09:2022-06-21,2022-06-21 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999
 malwarescore=0 adultscore=0 phishscore=0 bulkscore=0 mlxscore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2204290000 definitions=main-2206210087
X-Proofpoint-GUID: MJg_Ya8G64Wax4i6iAw06v0P17CB3Lpz
X-Proofpoint-ORIG-GUID: MJg_Ya8G64Wax4i6iAw06v0P17CB3Lpz
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: "Liam R. Howlett" <Liam.Howlett@Oracle.com>

Add support for kmem_cache_free_bulk() and kmem_cache_alloc_bulk() to the
radix tree test suite.

Link: https://lkml.kernel.org/r/20220504010716.661115-7-Liam.Howlett@oracle=
.com
Signed-off-by: Liam R. Howlett <Liam.Howlett@Oracle.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: David Howells <dhowells@redhat.com>
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Cc: SeongJae Park <sj@kernel.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
 tools/include/linux/slab.h       |   4 ++
 tools/testing/radix-tree/linux.c | 118 ++++++++++++++++++++++++++++++-
 2 files changed, 120 insertions(+), 2 deletions(-)

diff --git a/tools/include/linux/slab.h b/tools/include/linux/slab.h
index 0616409513eb..311759ea25e9 100644
--- a/tools/include/linux/slab.h
+++ b/tools/include/linux/slab.h
@@ -41,4 +41,8 @@ struct kmem_cache *kmem_cache_create(const char *name, un=
signed int size,
 			unsigned int align, unsigned int flags,
 			void (*ctor)(void *));
=20
+void kmem_cache_free_bulk(struct kmem_cache *cachep, size_t size, void **l=
ist);
+int kmem_cache_alloc_bulk(struct kmem_cache *cachep, gfp_t gfp, size_t siz=
e,
+			  void **list);
+
 #endif		/* _TOOLS_SLAB_H */
diff --git a/tools/testing/radix-tree/linux.c b/tools/testing/radix-tree/li=
nux.c
index f20529ae4dbe..2048d12c31df 100644
--- a/tools/testing/radix-tree/linux.c
+++ b/tools/testing/radix-tree/linux.c
@@ -93,14 +93,13 @@ void *kmem_cache_alloc_lru(struct kmem_cache *cachep, s=
truct list_lru *lru,
 	return p;
 }
=20
-void kmem_cache_free(struct kmem_cache *cachep, void *objp)
+void kmem_cache_free_locked(struct kmem_cache *cachep, void *objp)
 {
 	assert(objp);
 	uatomic_dec(&nr_allocated);
 	uatomic_dec(&cachep->nr_allocated);
 	if (kmalloc_verbose)
 		printf("Freeing %p to slab\n", objp);
-	pthread_mutex_lock(&cachep->lock);
 	if (cachep->nr_objs > 10 || cachep->align) {
 		memset(objp, POISON_FREE, cachep->size);
 		free(objp);
@@ -110,9 +109,80 @@ void kmem_cache_free(struct kmem_cache *cachep, void *=
objp)
 		node->parent =3D cachep->objs;
 		cachep->objs =3D node;
 	}
+}
+
+void kmem_cache_free(struct kmem_cache *cachep, void *objp)
+{
+	pthread_mutex_lock(&cachep->lock);
+	kmem_cache_free_locked(cachep, objp);
 	pthread_mutex_unlock(&cachep->lock);
 }
=20
+void kmem_cache_free_bulk(struct kmem_cache *cachep, size_t size, void **l=
ist)
+{
+	if (kmalloc_verbose)
+		pr_debug("Bulk free %p[0-%lu]\n", list, size - 1);
+
+	pthread_mutex_lock(&cachep->lock);
+	for (int i =3D 0; i < size; i++)
+		kmem_cache_free_locked(cachep, list[i]);
+	pthread_mutex_unlock(&cachep->lock);
+}
+
+int kmem_cache_alloc_bulk(struct kmem_cache *cachep, gfp_t gfp, size_t siz=
e,
+			  void **p)
+{
+	size_t i;
+
+	if (kmalloc_verbose)
+		pr_debug("Bulk alloc %lu\n", size);
+
+	if (!(gfp & __GFP_DIRECT_RECLAIM)) {
+		if (cachep->non_kernel < size)
+			return 0;
+
+		cachep->non_kernel -=3D size;
+	}
+
+	pthread_mutex_lock(&cachep->lock);
+	if (cachep->nr_objs >=3D size) {
+		struct radix_tree_node *node;
+
+		for (i =3D 0; i < size; i++) {
+			node =3D cachep->objs;
+			cachep->nr_objs--;
+			cachep->objs =3D node->parent;
+			p[i] =3D node;
+			node->parent =3D NULL;
+		}
+		pthread_mutex_unlock(&cachep->lock);
+	} else {
+		pthread_mutex_unlock(&cachep->lock);
+		for (i =3D 0; i < size; i++) {
+			if (cachep->align) {
+				posix_memalign(&p[i], cachep->align,
+					       cachep->size * size);
+			} else {
+				p[i] =3D malloc(cachep->size * size);
+			}
+			if (cachep->ctor)
+				cachep->ctor(p[i]);
+			else if (gfp & __GFP_ZERO)
+				memset(p[i], 0, cachep->size);
+		}
+	}
+
+	for (i =3D 0; i < size; i++) {
+		uatomic_inc(&nr_allocated);
+		uatomic_inc(&cachep->nr_allocated);
+		uatomic_inc(&cachep->nr_tallocated);
+		if (kmalloc_verbose)
+			printf("Allocating %p from slab\n", p[i]);
+	}
+
+	return size;
+}
+
 struct kmem_cache *
 kmem_cache_create(const char *name, unsigned int size, unsigned int align,
 		unsigned int flags, void (*ctor)(void *))
@@ -130,3 +200,47 @@ kmem_cache_create(const char *name, unsigned int size,=
 unsigned int align,
 	ret->non_kernel =3D 0;
 	return ret;
 }
+
+/*
+ * Test the test infrastructure for kem_cache_alloc/free and bulk counterp=
arts.
+ */
+void test_kmem_cache_bulk(void)
+{
+	int i;
+	void *list[12];
+	static struct kmem_cache *test_cache, *test_cache2;
+
+	/*
+	 * Testing the bulk allocators without aligned kmem_cache to force the
+	 * bulk alloc/free to reuse
+	 */
+	test_cache =3D kmem_cache_create("test_cache", 256, 0, SLAB_PANIC, NULL);
+
+	for (i =3D 0; i < 5; i++)
+		list[i] =3D kmem_cache_alloc(test_cache, __GFP_DIRECT_RECLAIM);
+
+	for (i =3D 0; i < 5; i++)
+		kmem_cache_free(test_cache, list[i]);
+	assert(test_cache->nr_objs =3D=3D 5);
+
+	kmem_cache_alloc_bulk(test_cache, __GFP_DIRECT_RECLAIM, 5, list);
+	kmem_cache_free_bulk(test_cache, 5, list);
+
+	for (i =3D 0; i < 12 ; i++)
+		list[i] =3D kmem_cache_alloc(test_cache, __GFP_DIRECT_RECLAIM);
+
+	for (i =3D 0; i < 12; i++)
+		kmem_cache_free(test_cache, list[i]);
+
+	/* The last free will not be kept around */
+	assert(test_cache->nr_objs =3D=3D 11);
+
+	/* Aligned caches will immediately free */
+	test_cache2 =3D kmem_cache_create("test_cache2", 128, 128, SLAB_PANIC, NU=
LL);
+
+	kmem_cache_alloc_bulk(test_cache2, __GFP_DIRECT_RECLAIM, 10, list);
+	kmem_cache_free_bulk(test_cache2, 10, list);
+	assert(!test_cache2->nr_objs);
+
+
+}
--=20
2.35.1