Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Zhouyi Zhou <zhouzhouyi@gmail.com>
To:     mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu,
        atrajeev@linux.vnet.ibm.com, linuxppc-dev@lists.ozlabs.org,
        linux-kernel@vger.kernel.org, lance@osuosl.org, paulmck@kernel.org,
        rcu@vger.kernel.org
Cc:     Zhouyi Zhou <zhouzhouyi@gmail.com>
Subject: [PATCH linux-next][RFC]powerpc: move pseries interrupt cleanup code to under __cpu_disable
Date:   Wed,  9 Nov 2022 09:47:06 +0800
Message-Id: <20221109014706.10484-1-zhouzhouyi@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

According to
Link: https://lore.kernel.org/lkml/20220914021528.15946-1-zhouzhouyi@gmail.com/T/
Link: https://lore.kernel.org/lkml/CN6WCMKCWHOG.LT2QV3910UJ2@bobo/

During the cpu offlining, the sub functions of xive_teardown_cpu will
call __lock_acquire when CONFIG_LOCKDEP=y. The latter function will
travel RCU protected list, so "WARNING: suspicious RCU usage" will be
triggered.

According to Documentation/core-api/cpu_hotplug.rst:
__cpu_disable should shut down the interrupt handler.

This patch move pseries interrupt cleanup code to under __cpu_disable.

RCU torture tested in ppc VM of Open Source Lab of Oregon State University,
by comparing the test results, our fix don't introduce new bugs and fixed
previous "WARNING: suspicious RCU usage" bugs.

Suggested-by: Nicholas Piggin <npiggin@gmail.com> 
Signed-off-by: Zhouyi Zhou <zhouzhouyi@gmail.com>
---
Dear PPC and RCU developers

I tries do move pseries interrupt cleanup code to under __cpu_disable.
During this process, I learn the XIVE interrupt controller architecture
by debugging the qemu virtual machine, and reading the following documents:
Link: https://www.qemu.org/docs/master/specs/ppc-spapr-xive.html
Link: https://www.qemu.org/docs/master/specs/ppc-xive.html
It is a fruitful journey, thank you for you guidance ;-)

I also tested the patch in ppc VM of Open Source Lab of Oregon
State University by invoking
tools/testing/selftests/rcutorture/bin/torture.sh

Thanks again
Zhouyi
--
 arch/powerpc/platforms/pseries/hotplug-cpu.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/hotplug-cpu.c b/arch/powerpc/platforms/pseries/hotplug-cpu.c
index e0a7ac5db15d..c7c86ea0a74d 100644
--- a/arch/powerpc/platforms/pseries/hotplug-cpu.c
+++ b/arch/powerpc/platforms/pseries/hotplug-cpu.c
@@ -62,12 +62,7 @@ static void pseries_cpu_offline_self(void)
 {
 	unsigned int hwcpu = hard_smp_processor_id();
 
-	local_irq_disable();
 	idle_task_exit();
-	if (xive_enabled())
-		xive_teardown_cpu();
-	else
-		xics_teardown_cpu();
 
 	unregister_slb_shadow(hwcpu);
 	rtas_stop_self();
@@ -96,3 +91,10 @@ static int pseries_cpu_disable(void)
 
 	cleanup_cpu_mmu_context();

+	local_irq_disable();
+
+	if (xive_enabled())
+		xive_teardown_cpu();
+	else
+		xics_teardown_cpu();
+
	return 0;
 }
 
-- 
2.25.1