When testing cpu online and offline, warning happened like this:
[ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
[ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
[ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
[ 146.751213] Workqueue: events cpuset_hotplug_workfn
[ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 146.752048] pc : build_sched_domains+0x81c/0x11b0
[ 146.752461] lr : build_sched_domains+0x414/0x11b0
[ 146.752860] sp : ffff800040a83a80
[ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
[ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
[ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
[ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
[ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
[ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
[ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
[ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
[ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
[ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
[ 146.757070] Call trace:
[ 146.757421] build_sched_domains+0x81c/0x11b0
[ 146.757771] partition_sched_domains_locked+0x57c/0x978
[ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
[ 146.758460] rebuild_sched_domains+0x2c/0x48
[ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
[ 146.759114] process_one_work+0x1f4/0x480
[ 146.759429] worker_thread+0x48/0x460
[ 146.759734] kthread+0x158/0x168
[ 146.760030] ret_from_fork+0x10/0x20
[ 146.760318] ---[ end trace 82c44aad6900e81a ]---
For some architectures like risc-v and arm64 which use common code
clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
is set, cluster_sibling in cpu_topology of each sibling adjacent
to CPUx is missed clearing, this may cause checking failed in
topology_span_sane() which don't allow Non-NUMA levels's mask partially
overlaped, finally results in rebuilding topology failure when CPUx
online again.
This warning can only be reproduced when we disable a part of CPUs in
one cluster, then enable one of the disabled CPUs.
Three steps to reproduce (CPU 92, 93, 94, 95 are in one cluster):
echo 0 > cpu92/online
echo 0 > cpu93/online
echo 1 > cpu92/online # WARN_ON !topology_span_sane
Fixes: c5e22feffdd7 ("topology: Represent clusters of CPUs within a die")
Signed-off-by: Wang ShaoBo <[email protected]>
Reviewed-by: Dietmar Eggemann <[email protected]>
Reviewed-by: Sudeep Holla <[email protected]>
Acked-by: Barry Song <[email protected]>
Tested-by: Dietmar Eggemann <[email protected]>
---
drivers/base/arch_topology.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index 981e72a3dafb..ff16a36a908b 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -677,6 +677,8 @@ void remove_cpu_topology(unsigned int cpu)
cpumask_clear_cpu(cpu, topology_core_cpumask(sibling));
for_each_cpu(sibling, topology_sibling_cpumask(cpu))
cpumask_clear_cpu(cpu, topology_sibling_cpumask(sibling));
+ for_each_cpu(sibling, topology_cluster_cpumask(cpu))
+ cpumask_clear_cpu(cpu, topology_cluster_cpumask(sibling));
for_each_cpu(sibling, topology_llc_cpumask(cpu))
cpumask_clear_cpu(cpu, topology_llc_cpumask(sibling));
--
2.25.1