Received-SPF: pass (google.com: domain of linux-kernel+bounces-191561-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
From: Ankur Arora <ankur.a.arora@oracle.com>
To: linux-kernel@vger.kernel.org
Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org,
        paulmck@kernel.org, rostedt@goodmis.org, mark.rutland@arm.com,
        juri.lelli@redhat.com, joel@joelfernandes.org, raghavendra.kt@amd.com,
        sshegde@linux.ibm.com, boris.ostrovsky@oracle.com,
        konrad.wilk@oracle.com, Ankur Arora <ankur.a.arora@oracle.com>
Subject: [PATCH v2 17/35] rcu: handle quiescent states for PREEMPT_RCU=n, PREEMPT_COUNT=y
Date: Mon, 27 May 2024 17:35:03 -0700
Message-Id: <20240528003521.979836-18-ankur.a.arora@oracle.com>
In-Reply-To: <20240528003521.979836-1-ankur.a.arora@oracle.com>
References: <20240528003521.979836-1-ankur.a.arora@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?6/OOxDYF1k50x0etymIZ9WChbWZSTiOXjx4+Ttya0BwldsmzoqAI5/XO6CfD?=
 =?us-ascii?Q?Yp93ZrTS7Xyx2Nbov72cFHB2wAnx6DoO/VUZrJl9BA9g//DUBWDO/KGMmcwz?=
 =?us-ascii?Q?6OfjegbHppO+gHYWuYPJXhfaE+cPI7rm4bEEOozC5yhbPJL0r5U7e4R1JJc/?=
 =?us-ascii?Q?Lxz1qJoOH34PMe/7NG3fPOkqkl+52fZfKFZk2ZRKo8troK6peWvkPo6WPK3j?=
 =?us-ascii?Q?r9lIkDpgSlVufnqp/yp5/aYyEUUukF1uprWWCBEeOUfUg21YQMGWe2F2Za8H?=
 =?us-ascii?Q?qb3AAx4zEBGMu+ah8AbRJFN4b8QfwmNIdxEhD5VrlZCfw1DaQwYqt+cxCobR?=
 =?us-ascii?Q?AHY1wopgyLIi8C36L5vubi/v4C4k7xJIak1324PS2IjZbGFlmxWDDALbeO9m?=
 =?us-ascii?Q?nUz5Ld9Wb+cGY57Vga2kpE7Oqs7FnCmditIwzSGLIJ3ZoNPqyUjHDO9zO2uA?=
 =?us-ascii?Q?6uDN88GErdaI3zbC52hI3mj2sf6p6G9I265dP7vLxox/l8o6osBirudRSRJZ?=
 =?us-ascii?Q?Hhghs4QSYxu3uOKOz2qvQoZ3WTCIZaNBG0ec3bdF+oXVK5BHSDmZxfg/7QIg?=
 =?us-ascii?Q?QYsJ28Y/iQ1cM2qtbXKu6lgIQlP26F2fkoluYqcWUlJ5Zu9Az4ma9Tx8UrHT?=
 =?us-ascii?Q?R5sxXc3XFVrApNNE+/KjFZ7TNGQd13gCXhd4HicqCZTenS11StyJAOla0g+k?=
 =?us-ascii?Q?UKB2vQA1xqTw2r5CVd+Ie0jWz5vyBYkZSUHPUz6KKsgjNYOzuVvQnaO295a2?=
 =?us-ascii?Q?EkN1jKBA+qu/2iD87BoZnXPdpCIruOaTqFhlq9Ji7IIFfG8VESKqqYsLu6kL?=
 =?us-ascii?Q?54oc7v2p2pVnDMTikyfZuWGnYY4fam4/OXG5wXDQzvrcgs9vnyFPhhamuJxb?=
 =?us-ascii?Q?ADDHrctZVh1rUbiaMDU2PZ53MHLyyLOIt0OWny+f9FTr3kLjDeJYuTSHuKeV?=
 =?us-ascii?Q?y+5qGVnrs/JV2LTYdiHP5OwzhaNrQo7E+/HAY72zbJ/BDRA7UxX/a9Dd2Y06?=
 =?us-ascii?Q?l3drNiI9yve6h0W8j36DgsSKxJB5EbtggyLrdSfy51ssOQxF0ieAXrwy6t7d?=
 =?us-ascii?Q?v2LFQluHeOI7z4gJmg0ke8FCvLwitj9VHPrE/bWm4Z38oPXhZAT7RR3e1zST?=
 =?us-ascii?Q?C9mFo1mE+tugQbwWyfpjaEzmZhJ0HjGkq0/k8RdIivr0Wy5ZYKDLpG+zMn1x?=
 =?us-ascii?Q?uO7ElY7Pbv94qSzSdXkPIpzWYvNpH9zIWjqO5txyIiz9JKWWCDiZ2BAshvzm?=
 =?us-ascii?Q?3pxKlD8GbEuAm6R3K8eeRsAAggw7LLDBgwPVUJib2wvRk1fwOxIn2wLLH0gi?=
 =?us-ascii?Q?I/R+/VpEQWztaZH2fk2mNimydvIJQctCx2BlmOWrjt6tp7sGQ+lBOXrL/FO8?=
 =?us-ascii?Q?R7wy9QRHnWS8OcdwrJI1s7RGEvxX4k7leQ/4SKfZpHckR/kmcHInYVkHocme?=
 =?us-ascii?Q?3B56Haa0CHK9QvjYBvDWx+7/ceA5RyQcEqA9zzZXuflqee2zQSNt1jCK3F6Y?=
 =?us-ascii?Q?91sOCnVsaLc30sjmtidiBKH07rxSCivNIhHIUModhL1mvbv3nQR0CKcYqYo+?=
 =?us-ascii?Q?LLkWzdvXs3R2E7AkUIqiFk4ch59vpb0rTK+CSxoeD7VcLGpLKh1qZXixOirF?=
 =?us-ascii?Q?QA=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	yYFYGt4EBbxa4MqhB7beSW0whX7t9ONS73B6zs2wOrnK3i/p12DPwZUFT/NiT+/8o/jK1NKkJ31HJuc8XKLeLbYqsKJG9cSBGDMwTaBRNPzvSVZYL47Nw6mCUSNUtlSkTswyu2LYOakL7UQJV2VinqqqRFnfRI3FNVYVnbsJ4J+HHMeXBKtuUU6QlZo2e+SCZS4PDjcbruoXSkEsL8JwNV116kVFOyk8UXkfAktbHMu4tWg4CtelzbZUldyUaqWUtrqpqwEehf0mn3tDPYesAPYo+FOPffvw7hPBlqRTeCcdiolIcd2nf+CYRbAh7OXFgtHwXg7dqOaFtlPSg9mDk/sNvfHQ+vUOEi7nwWMTPzK/eg9yM1XH6PxXn3lZyHmYhUCg7+4IaV96LyHQgyPM+JyiPbDCMhr5OlwpN/UiOJOZOzpyMdpbieBDcwVUg4CVOcreGlZILb/DCzx0Gp+EkCv7rO3eB0epH/TDkbo6AD7T58BhNax8FauNNdCeIQrLz0bCblmV75vL4u/4EvoTAhJCgpD+LIQ/eso5wg0L0AA+jkoVHfzteE4lczmPvykSdNeID9B6nw3FmGwi1SAf6I4KmoFmRqnspviQBLzK2J0=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f04ad2ae-02df-4fd9-1775-08dc7eae2fd0
X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2024 00:36:17.2564
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jgaOCnW9z3TM8q1q46BoYKvChmr364jL3VMJFqW9m+oZsutLA+OhLQNbpGt+XZVK+pNQv0RWGloIIPsKFvrlSAh8zKnB0Eum3bgdylCPDp8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB7177
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-27_06,2024-05-27_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0
 spamscore=0 adultscore=0 mlxscore=0 phishscore=0 suspectscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2405010000 definitions=main-2405280003
X-Proofpoint-ORIG-GUID: Kk7od-FzMrHqDLI0kBykHnBNKVZRO618
X-Proofpoint-GUID: Kk7od-FzMrHqDLI0kBykHnBNKVZRO618

With PREEMPT_RCU=n, cond_resched() provides urgently needed quiescent
states for read-side critical sections via rcu_all_qs().
One reason why this was necessary: lacking preempt-count, the tick
handler has no way of knowing whether it is executing in a read-side
critical section or not.

With PREEMPT_AUTO=y, there can be configurations with (PREEMPT_COUNT=y,
PREEMPT_RCU=n). This means that cond_resched() is a stub which does
not provide for quiescent states via rcu_all_qs().

So, use the availability of preempt_count() to report quiescent states
in rcu_flavor_sched_clock_irq().

Suggested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
---
 kernel/rcu/tree_plugin.h | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 36a8b5dbf5b5..741476c841a1 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -963,13 +963,16 @@ static void rcu_preempt_check_blocked_tasks(struct rcu_node *rnp)
  */
 static void rcu_flavor_sched_clock_irq(int user)
 {
-	if (user || rcu_is_cpu_rrupt_from_idle()) {
+	if (user || rcu_is_cpu_rrupt_from_idle() ||
+	    (IS_ENABLED(CONFIG_PREEMPT_COUNT) &&
+	     !(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)))) {
 
 		/*
 		 * Get here if this CPU took its interrupt from user
-		 * mode or from the idle loop, and if this is not a
-		 * nested interrupt.  In this case, the CPU is in
-		 * a quiescent state, so note it.
+		 * mode, from the idle loop without this being a nested
+		 * interrupt, or while not holding a preempt count.
+		 * In this case, the CPU is in a quiescent state, so note
+		 * it.
 		 *
 		 * No memory barrier is required here because rcu_qs()
 		 * references only CPU-local variables that other CPUs
-- 
2.31.1