Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Zqiang <qiang1.zhang@intel.com>
To:     paulmck@kernel.org, frederic@kernel.org, joel@joelfernandes.org
Cc:     rcu@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH v2] srcu: Fix flush srcu structure's->sup work warning in cleanup_srcu_struct()
Date:   Thu, 23 Mar 2023 21:46:21 +0800
Message-Id: <20230323134621.336832-1-qiang1.zhang@intel.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

When unloading rcutorture kmod will trigger the following callstack:

insmod rcutorture.ko
rmmod rcutorture.ko

[  209.437327] WARNING: CPU: 0 PID: 508 at kernel/workqueue.c:3167 __flush_work+0x50a/0x540
[  209.437346] Modules linked in: rcutorture(-) torture [last unloaded: rcutorture]
[  209.437382] CPU: 0 PID: 508 Comm: rmmod Tainted: G  W  6.3.0-rc1-yocto-standard+
[  209.437406] RIP: 0010:__flush_work+0x50a/0x540
.....
[  209.437758]  flush_delayed_work+0x36/0x90
[  209.437776]  cleanup_srcu_struct+0x68/0x2e0
[  209.437817]  srcu_module_notify+0x71/0x140
[  209.437854]  blocking_notifier_call_chain+0x9d/0xd0
[  209.437880]  __x64_sys_delete_module+0x223/0x2e0
[  209.438046]  do_syscall_64+0x43/0x90
[  209.438062]  entry_SYSCALL_64_after_hwframe+0x72/0xdc

flush_delayed_work()
->__flush_work()
   ->if (WARN_ON(!work->func))
        return false;

For srcu objects defined with DEFINE_SRCU() or DEFINE_STATIC_SRCU(),
when compiling and loading as modules, the srcu_module_coming() is
invoked, allocate memory for srcu structure's->sda and initialize
sda structure, due to not fully initialize srcu structure's->sup,
so at this time the sup structure's->work.work.func is null, if not
invoke init_srcu_struct_fields() before unloading modules, the
__flush_work() be invoked in srcu_module_going() and find work->func
is empty, will raise the warning above.

This commit add the check of srcu_sup structure's->srcu_gp_seq_needed
to determine whether the check_init_srcu_struct() has been invoked to
initialize srcu objects in srcu_module_going(), if not initialize, there
are no pending or running works, so there is no need to flush, only invoke
free_percpu() to release srcu structure's->sda.

Co-developed-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Zqiang <qiang1.zhang@intel.com>
---
 kernel/rcu/srcutree.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
index 1fb078abbdc9..edf894e3b96e 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -1921,7 +1921,6 @@ static int srcu_module_coming(struct module *mod)
 		ssp->sda = alloc_percpu(struct srcu_data);
 		if (WARN_ON_ONCE(!ssp->sda))
 			return -ENOMEM;
-		init_srcu_struct_data(ssp);
 	}
 	return 0;
 }
@@ -1931,9 +1930,14 @@ static void srcu_module_going(struct module *mod)
 {
 	int i;
 	struct srcu_struct **sspp = mod->srcu_struct_ptrs;
+	struct srcu_struct *ssp;
 
-	for (i = 0; i < mod->num_srcu_structs; i++)
-		cleanup_srcu_struct(*(sspp++));
+	for (i = 0; i < mod->num_srcu_structs; i++) {
+		ssp = (*sspp++);
+		if (!rcu_seq_state(smp_load_acquire(&ssp->srcu_sup->srcu_gp_seq_needed)))
+			cleanup_srcu_struct(ssp);
+		free_percpu(ssp->sda);
+	}
 }
 
 /* Handle one module, either coming or going. */
-- 
2.25.1