Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
From:   "Joel Fernandes (Google)" <joel@joelfernandes.org>
To:     linux-kernel@vger.kernel.org
Cc:     Joel Fernandes <joel@joelfernandes.org>,
        Josh Triplett <josh@joshtriplett.org>,
        Lai Jiangshan <jiangshanlai@gmail.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        "Paul E. McKenney" <paulmck@kernel.org>, rcu@vger.kernel.org,
        Steven Rostedt <rostedt@goodmis.org>
Subject: [PATCH v2] rcu/nocb: Add an option to offload all CPUs on boot
Date:   Thu, 14 Apr 2022 20:06:42 +0000
Message-Id: <20220414200642.1778806-1-joel@joelfernandes.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

From: Joel Fernandes <joel@joelfernandes.org>

On systems with CONFIG_RCU_NOCB_CPU=y, there is no default mask provided
which ends up not offloading any CPU. This patch removes a dependency
from the bootloader having to know about RCU, about how many CPUs the
system has, and about how to provide the mask.

With the new option enabled, all CPUs will be offloaded on boot.

Signed-off-by: Joel Fernandes <joel@joelfernandes.org>
---
 Documentation/admin-guide/kernel-parameters.txt |  4 ++++
 kernel/rcu/Kconfig                              | 12 ++++++++++++
 kernel/rcu/tree_nocb.h                          | 11 +++++++++++
 3 files changed, 27 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index f5a27f067db9..fd8165237954 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -4398,6 +4398,10 @@
 			no-callback mode from boot but the mode may be
 			toggled at runtime via cpusets.
 
+			Note that this argument has no effect if the kernel
+			is built with CONFIG_RCU_NOCB_CPU_ALL=y, in which case
+			all CPUs will be offloaded on boot.
+
 	rcu_nocb_poll	[KNL]
 			Rather than requiring that offloaded CPUs
 			(specified by rcu_nocbs= above) explicitly
diff --git a/kernel/rcu/Kconfig b/kernel/rcu/Kconfig
index bf8e341e75b4..00a040f17b1d 100644
--- a/kernel/rcu/Kconfig
+++ b/kernel/rcu/Kconfig
@@ -223,6 +223,18 @@ config RCU_NOCB_CPU
 	  Say Y here if you need reduced OS jitter, despite added overhead.
 	  Say N here if you are unsure.
 
+config RCU_NOCB_CPU_ALL
+	bool "Offload RCU callback processing from all CPUs"
+	depends on RCU_NOCB_CPU
+	default n
+	help
+	  Use this option to offload callback processing from all CPUs.
+	  This also avoids the need to use any boot arguments to achieve
+	  the same effect, when it is desired to offload all CPUs.
+
+	  Say Y here if you are sure you always want all CPUs offloaded.
+	  Say N here if you are unsure.
+
 config TASKS_TRACE_RCU_READ_MB
 	bool "Tasks Trace RCU readers use memory barriers in user and idle"
 	depends on RCU_EXPERT
diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h
index eeafb546a7a0..df2e4c0a6178 100644
--- a/kernel/rcu/tree_nocb.h
+++ b/kernel/rcu/tree_nocb.h
@@ -1168,6 +1168,13 @@ void __init rcu_init_nohz(void)
 	bool need_rcu_nocb_mask = false;
 	struct rcu_data *rdp;
 
+	if (IS_ENABLED(CONFIG_RCU_NOCB_CPU_ALL)) {
+		if (rcu_nocb_is_setup) {
+			pr_info("Kernel is configured to offload all CPUs, rcu_nocbs= passed but has no effect.\n");
+		}
+		need_rcu_nocb_mask = true;
+	}
+
 #if defined(CONFIG_NO_HZ_FULL)
 	if (tick_nohz_full_running && cpumask_weight(tick_nohz_full_mask))
 		need_rcu_nocb_mask = true;
@@ -1191,6 +1198,10 @@ void __init rcu_init_nohz(void)
 		cpumask_or(rcu_nocb_mask, rcu_nocb_mask, tick_nohz_full_mask);
 #endif /* #if defined(CONFIG_NO_HZ_FULL) */
 
+	/* Set all CPUs as offloaded if RCU_NOCB_CPU_ALL=y. */
+	if (IS_ENABLED(CONFIG_RCU_NOCB_CPU_ALL))
+		cpumask_setall(rcu_nocb_mask);
+
 	if (!cpumask_subset(rcu_nocb_mask, cpu_possible_mask)) {
 		pr_info("\tNote: kernel parameter 'rcu_nocbs=', 'nohz_full', or 'isolcpus=' contains nonexistent CPUs.\n");
 		cpumask_and(rcu_nocb_mask, cpu_possible_mask,
-- 
2.36.0.rc0.470.gd361397f0d-goog