Received-SPF: pass (google.com: domain of linux-kernel+bounces-62974-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
From: Ankur Arora <ankur.a.arora@oracle.com>
To: linux-kernel@vger.kernel.org
Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org,
        paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org,
        bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com,
        mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org,
        willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org,
        mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com,
        bristot@kernel.org, mathieu.desnoyers@efficios.com,
        geert@linux-m68k.org, glaubitz@physik.fu-berlin.de,
        anton.ivanov@cambridgegreys.com, mattst88@gmail.com,
        krypton@ulrich-teichert.org, rostedt@goodmis.org,
        David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com,
        jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com,
        boris.ostrovsky@oracle.com, konrad.wilk@oracle.com,
        Ankur Arora <ankur.a.arora@oracle.com>
Subject: [PATCH 15/30] rcu: handle quiescent states for PREEMPT_RCU=n, PREEMPT_COUNT=y
Date: Mon, 12 Feb 2024 21:55:39 -0800
Message-Id: <20240213055554.1802415-16-ankur.a.arora@oracle.com>
In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com>
References: <20240213055554.1802415-1-ankur.a.arora@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?n6eOdKp+2mVk7mkEIT4aESeKMKe10KEBEqeE1Y7flc5EbKc7FhmaKo+Kp6X0?=
 =?us-ascii?Q?pnqjVcVq7V/I9qPvM+GGNyvcG3ixA1kTks5fD4BkvaI3T3wlgz3why9aOrae?=
 =?us-ascii?Q?Ogf7mBhf2NTq9hyNoS84IM4SoBra1kEM+DSsCYfQbOO9tmwnXl0yKOsNj3un?=
 =?us-ascii?Q?hn3FrhEbTjDT4FGlMTudrca06rjtAoQuEJkOeyGzMtfPvPVLWtRWrm2XN6si?=
 =?us-ascii?Q?FMni7p7HbWRQPM5f5UjkdxmWBWOroPy+RQ8htxSEv0g37uoCKU9fyQtO1nv/?=
 =?us-ascii?Q?kOasNqCEXf94BRd17kXp17lRpzba/tCjk9XvTToaK5tbJDy1RfE1t1Fj6ZX4?=
 =?us-ascii?Q?xdc3a5A/yhgfCEzAdthPQHgy5JPRzELqiIiKPCcx79ottdhjZqTwjdKGzBXe?=
 =?us-ascii?Q?vyN1YpU2ZJ5Owv5E2MFn/gY6xUPmDI+/gANxeisk6ejS2BHhEoKQsdOUAjR/?=
 =?us-ascii?Q?ltRjFoynuYysY0ww0ZC3QSPWYuCHMOLosHExxAilEH/9LYoqDaR+abIrPLME?=
 =?us-ascii?Q?gvEg/WfdEFGlfAdHh4+A3EWtarIcePnEM4MmMS+/squbVOHjiZbpNDndz1Cc?=
 =?us-ascii?Q?Tbshf2D/4/9knQVz3XM60CXMTSh7fZijGSUig3OP4ZqVLxEE7udcI+ps9/Wo?=
 =?us-ascii?Q?QnX6v6NMq/oOzwW8THGmxGG8WxgfIzZUxaFpBGsYlR7M44Fqmz72CfCETYDV?=
 =?us-ascii?Q?EwJqEcJ6n2O0O585pUEa+LqDixzoPuO2s6axOCqCfslHzvfsex22WfLm9mNr?=
 =?us-ascii?Q?uIfGZ4NqaqxDJUEDKrNMYqHMU5635mQ9biYTtxREH0mXXU6dBZ454me07b64?=
 =?us-ascii?Q?KN7bTB1w2kZVfkurqVHoXBRns59J3LXySWukaZN5GdyQj3yKHhJFxJBjGmrT?=
 =?us-ascii?Q?uiQwNNzzo+fzOQ3COtIphsJ8bRAa5BjOEyhjQ8z3LwHc+1csgGBYiDl/dcur?=
 =?us-ascii?Q?nzCrpGud/lnfwATNYlK8vN1nHG8GCY9Z8kYYEPlh5Jzcn0a85Z0/zvmHEYi/?=
 =?us-ascii?Q?BRV+Pvhd9YWCvG0qV93xxz+SLsyOxugxXI036QbpR6Lro/7NTd/MkygLrokm?=
 =?us-ascii?Q?WXJTaGNnq3dJWD7ssdXBBzZfFtgjr04SrA433U0IYXmhyIVKH+P1GscU28Km?=
 =?us-ascii?Q?b1uozbLZ6baxXwJTUA3D2epMb1baGH0PqZ+UR3B96cabHY5bB/3GIqwgac8F?=
 =?us-ascii?Q?MaTIutCRA/ITYUpoRWdtg3Rw80Io6R8quSuQyYh1Sfl9C+0Sm9Q1+MmYnC5X?=
 =?us-ascii?Q?14qNrxkumP/TuZS37wq7J6XiXvQNK7tsJgt/4gNhc9maYXRxGCSWM0K2cCRs?=
 =?us-ascii?Q?BZzd7aI2g/+lFC99DKTtzgAXUzwMo5oYk8IBPk/tJV2XIbkupqpt86EHn7EY?=
 =?us-ascii?Q?lZFdScXuKXExvE7HGu5Clvo+XwExo6FLR8C+DSmPHO3g2kiOKn6aEk5h0Z7M?=
 =?us-ascii?Q?ppLY5ctllNqNFrUDGVTnKqkJcFLz9ypb/lDc7M9u9qcKqAfX/H+7bYjFADmO?=
 =?us-ascii?Q?hjI4OUFCiGMBGuilU8dFgwhsRHDuxfq5MQKH6OmgyZu3qrtzVBZZfdkrv7Td?=
 =?us-ascii?Q?mOZ5gMuR+sAeUcL9enOSH9j3YvdEPyfoVJQHD99/TQUboyqJ9yAqs8eQFvRH?=
 =?us-ascii?Q?cQ=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	f9pMC49EDpKuUz6+tlePZPIO1frqqTkUdfywS9wU1lWrmBON3JDEEAoqnXYLF2mYpxEd3kFoNRGv4Y5qUTXQzbl/Om+PEEk8jRRJUZpEBWf+5lADMtRNQQ3vwELvi4pWuk2o5hHWPp+YusTfwZI0Fzf20RT2SpDXU3gGefOCKKHB+OmabUrF/POIm7Tw5BdvEZcrJouD3gNGe2CZGneydkIJbMLs5E/eWFGM4GW5rJUBRA1dHUkdXHNsmez9QXFfUu4qBRcMFfrHJi8Kt+G1SgNp0cLIow5w9Ja4REQgiQz5sJv8sWSM54lQAGHlEr7b42kIdW+SvfX2CcDGNjkZ8h/I+WPOXyL4zIVWqpkH1ATKfVGc1RwkmMapBBgH0I+4yIxmZOWL8QuFwWxjhGfsywYQLWP2nAINHsOA/vk5ycWWlaHiR/XWnpQauvBKJxQlbbs/oPc/+awOGn2dsWc14UBJZPm9I2oOKkvY5LtqBrV43oP5Ls+tktmM086FMjat+Q75sNViJXemHkEOtXTqHeqds9eFUDTnTGhIldBUHO79EyKZ6dMmQNIt8+RIsd+iLoIkJOslBUPP6qMDjNEES8l08iJS9LRQnK5lAlhi5nQ=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4e167990-1449-4efd-9094-08dc2c588034
X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:20.0803
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: liU8PpXs/k9jGfapHxe5XQngIFJlxxM2N2ZP0o1Q7StfitM3p0QfG0GEH9roFzQgkEd142DF7Mpfkrf9hoZ++BIXTEc7cb8vkcNiBQEquCA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0
 mlxlogscore=969 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311290000 definitions=main-2402130043
X-Proofpoint-GUID: -jvbt2DS2DVl02mf2QdxwccquCrtnsUh
X-Proofpoint-ORIG-GUID: -jvbt2DS2DVl02mf2QdxwccquCrtnsUh

With PREEMPT_RCU=n, cond_resched() provides urgently needed quiescent
states for read-side critical sections via rcu_all_qs().
One reason why this was necessary: lacking preempt-count, the tick
handler has no way of knowing whether it is executing in a read-side
critical section or not.

With PREEMPT_AUTO=y, there can be configurations with (PREEMPT_COUNT=y,
PREEMPT_RCU=n). This means that cond_resched() is a stub which does
not provide for quiescent states via rcu_all_qs().

So, use the availability of preempt_count() to report quiescent states
in rcu_flavor_sched_clock_irq().

Suggested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
---
 kernel/rcu/tree_plugin.h | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 26c79246873a..9b72e9d2b6fe 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -963,13 +963,16 @@ static void rcu_preempt_check_blocked_tasks(struct rcu_node *rnp)
  */
 static void rcu_flavor_sched_clock_irq(int user)
 {
-	if (user || rcu_is_cpu_rrupt_from_idle()) {
+	if (user || rcu_is_cpu_rrupt_from_idle() ||
+	    (IS_ENABLED(CONFIG_PREEMPT_COUNT) &&
+	     !(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)))) {
 
 		/*
 		 * Get here if this CPU took its interrupt from user
-		 * mode or from the idle loop, and if this is not a
-		 * nested interrupt.  In this case, the CPU is in
-		 * a quiescent state, so note it.
+		 * mode, from the idle loop without this being a nested
+		 * interrupt, or while not holding a preempt count.
+		 * In this case, the CPU is in a quiescent state, so note
+		 * it.
 		 *
 		 * No memory barrier is required here because rcu_qs()
 		 * references only CPU-local variables that other CPUs
-- 
2.31.1