Received-SPF: pass (google.com: domain of linux-kernel+bounces-127331-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
From: Ming Yang <yangming73@huawei.com>
To: <cl@linux.com>, <penberg@kernel.org>, <rientjes@google.com>,
	<iamjoonsoo.kim@lge.com>, <akpm@linux-foundation.org>, <vbabka@suse.cz>,
	<roman.gushchin@linux.dev>, <42.hyeyoo@gmail.com>, <linux-mm@kvack.org>,
	<linux-kernel@vger.kernel.org>
CC: <yangming73@huawei.com>, <zhangliang5@huawei.com>,
	<wangzhigang17@huawei.com>, <liushixin2@huawei.com>, <alex.chen@huawei.com>,
	<pengyi.pengyi@huawei.com>, <xiqi2@huawei.com>
Subject: [PATCH] slub: fix slub segmentation
Date: Tue, 2 Apr 2024 11:10:25 +0800
Message-ID: <20240402031025.1097-1-yangming73@huawei.com>
Precedence: bulk
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

When one of numa nodes runs out of memory and lots of processes still
booting, slabinfo shows much slub segmentation exits. The following
shows some of them:

tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs>
<num_slabs> <sharedavail>
kmalloc-512        84309 380800   1024   32    8 :
tunables    0    0    0 : slabdata  11900  11900      0
kmalloc-256        65869 365408    512   32    4 :
tunables    0    0    0 : slabdata  11419  11419      0

365408 "kmalloc-256" objects are alloced but only 65869 of them are
used; While 380800 "kmalloc-512" objects are alloced but only 84309
of them are used.

This problem exits in the following senario:
1. Multiple numa nodes, e.g. four nodes.
2. Lack of memory in any one node.
3. Functions which alloc many slub memory in certain numa nodes,
like alloc_fair_sched_group.

The slub segmentation generated because of the following reason:
In function "___slab_alloc" a new slab is attempted to be gotten via
function "get_partial". If the argument 'node' is assigned but there
are neither partial memory nor buddy memory in that assigned node, no
slab could be gotten. And then the program attempt to alloc new slub
from buddy system, as mentationed before: no buddy memory in that
assigned node left, a new slub might be alloced from the buddy system
of other node directly, no matter whether there is free partil memory
left on other node. As a result slub segmentation generated.

The key point of above allocation flow is: the slab should be alloced
from the partial of other node first, instead of the buddy system of
other node directly.

In this commit a new slub allocation flow is proposed:
1. Attempt to get a slab via function get_partial (first step in
new_objects lable).
2. If no slab is gotten and 'node' is assigned, try to alloc a new
slab just from the assigned node instead of all node.
3. If no slab could be alloced from the assigned node, try to alloc
slub from partial of other node.
4. If the alloctation in step 3 fails, alloc a new slub from buddy
system of all node.

Signed-off-by: Ming Yang <yangming73@huawei.com>
Signed-off-by: Liang Zhang <zhangliang5@huawei.com>
Signed-off-by: Zhigang Wang <wangzhigang17@huawei.com>
Reviewed-by: Shixin Liu <liushixin2@huawei.com>
---
This patch can be tested and verified by following steps:
1. First, try to run out memory on node0. echo 1000(depending on your memory) > 
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages.
2. Second, boot 10000(depending on your memory) processes which use setsid 
systemcall, as the setsid systemcall may likely call function 
alloc_fair_sched_group.
3. Last, check slabinfo, cat /proc/slabinfo.

Hardware info:
Memory : 8GiB
CPU (total #): 120
numa node: 4

Test clang code example:
int main() {
    void *p = malloc(1024);
    setsid();
    while(1);
}

 mm/slub.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/mm/slub.c b/mm/slub.c
index 1bb2a93cf7..3eb2e7d386 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -3522,7 +3522,18 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
 	}
 
 	slub_put_cpu_ptr(s->cpu_slab);
+	if (node != NUMA_NO_NODE) {
+		slab = new_slab(s, gfpflags | __GFP_THISNODE, node);
+		if (slab)
+			goto slab_alloced;
+
+		slab = get_any_partial(s, &pc);
+		if (slab)
+			goto slab_alloced;
+	}
 	slab = new_slab(s, gfpflags, node);
+
+slab_alloced:
 	c = slub_get_cpu_ptr(s->cpu_slab);
 
 	if (unlikely(!slab)) {
-- 
2.33.0