2022-03-22 14:09:08

by Zqiang

Subject: [PATCH v4 0/2] rcu: Deoffload rdp if rcuop/rcuog kthreads spawn failed

When CONFIG_RCU_NOCB_CPU is enabled and 'rcu_nocbs' is set, the rcuop
and rcuog kthreads is created. however the rcuop or rcuog kthreads
creation may fail, if failed, deoffload the offloaded rdp.

---
v1->v2:
Invert the locking dependency order between rcu_state.barrier_mutex
and hotplug lock.

Holding nocb_gp_kthread_mutex, ensure that the nocb_gp_kthread exists.

v2->v3:
Adjust the position of the nocb_gp_kthread_mutex, in rdp_gp->nocb_gp_kthread
check upper lock nocb_gp_kthread_mutex.

Do the locking order change in a separate patch.

v3->v4:
Protect SEGCBLIST_KTHREAD_CB flags clear with rcu_nocb lock.

Zqiang(2):
rcu: Call rcu_nocb_rdp_deoffload() directly after rcuog/op kthreads spawn failed
rcu: Invert the locking dependency order between rcu_state.barrier_mutex and hotplug lock

kernel/rcu/tree_nocb.h | 85 ++++++++++++++++++++++++++++++++----------
1 file changed, 65 insertions(+), 20 deletions(-)

--
2.25.1

2022-03-22 14:51:19

by Zqiang

Subject: [PATCH v4 2/2] rcu: Invert the locking dependency order between rcu_state.barrier_mutex and hotplug lock

When call rcutree_prepare_cpu(), the cpus write lock has been held,
just lock the barrier_mutex before calling rcu_nocb_rdp_deoffload()
from failure path. therefore, invert the locking dependency order.

Signed-off-by: Zqiang <[email protected]>
---
kernel/rcu/tree_nocb.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h
index 154934f3daa9..e3d1bd26d6eb 100644
--- a/kernel/rcu/tree_nocb.h
+++ b/kernel/rcu/tree_nocb.h
@@ -1074,8 +1074,8 @@ int rcu_nocb_cpu_deoffload(int cpu)
struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
int ret = 0;

- mutex_lock(&rcu_state.barrier_mutex);
cpus_read_lock();
+ mutex_lock(&rcu_state.barrier_mutex);
if (rcu_rdp_is_offloaded(rdp)) {
if (cpu_online(cpu)) {
ret = work_on_cpu(cpu, rcu_nocb_rdp_deoffload, rdp);
@@ -1086,8 +1086,8 @@ int rcu_nocb_cpu_deoffload(int cpu)
ret = -EINVAL;
}
}
- cpus_read_unlock();
mutex_unlock(&rcu_state.barrier_mutex);
+ cpus_read_unlock();

return ret;
}
@@ -1169,8 +1169,8 @@ int rcu_nocb_cpu_offload(int cpu)
struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
int ret = 0;

- mutex_lock(&rcu_state.barrier_mutex);
cpus_read_lock();
+ mutex_lock(&rcu_state.barrier_mutex);
if (!rcu_rdp_is_offloaded(rdp)) {
if (cpu_online(cpu)) {
ret = work_on_cpu(cpu, rcu_nocb_rdp_offload, rdp);
@@ -1181,8 +1181,8 @@ int rcu_nocb_cpu_offload(int cpu)
ret = -EINVAL;
}
}
- cpus_read_unlock();
mutex_unlock(&rcu_state.barrier_mutex);
+ cpus_read_unlock();

return ret;
}
--
2.25.1

2022-03-22 22:03:28

by Frederic Weisbecker

Subject: Re: [PATCH v4 0/2] rcu: Deoffload rdp if rcuop/rcuog kthreads spawn failed

On Tue, Mar 22, 2022 at 09:17:51PM +0800, Zqiang wrote:
> When CONFIG_RCU_NOCB_CPU is enabled and 'rcu_nocbs' is set, the rcuop
> and rcuog kthreads is created. however the rcuop or rcuog kthreads
> creation may fail, if failed, deoffload the offloaded rdp.

Thanks! I'm queuing these. I might split things and reword a bit.
Also I'll put the second patch first, otherwise bisection will be
broken.