Received-SPF: pass (google.com: domain of linux-kernel+bounces-191550-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
From: Ankur Arora <ankur.a.arora@oracle.com>
To: linux-kernel@vger.kernel.org
Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org,
        paulmck@kernel.org, rostedt@goodmis.org, mark.rutland@arm.com,
        juri.lelli@redhat.com, joel@joelfernandes.org, raghavendra.kt@amd.com,
        sshegde@linux.ibm.com, boris.ostrovsky@oracle.com,
        konrad.wilk@oracle.com, Ankur Arora <ankur.a.arora@oracle.com>,
        Arnd Bergmann <arnd@arndb.de>, Ingo Molnar <mingo@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>
Subject: [PATCH v2 06/35] thread_info: define __tif_need_resched(resched_t)
Date: Mon, 27 May 2024 17:34:52 -0700
Message-Id: <20240528003521.979836-7-ankur.a.arora@oracle.com>
In-Reply-To: <20240528003521.979836-1-ankur.a.arora@oracle.com>
References: <20240528003521.979836-1-ankur.a.arora@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?4+pcMN9sKtlJ7T1CHR7aTq1Sn+YSVsizV0R21FL7GCVb8t6k6/NXiW5rQEAR?=
 =?us-ascii?Q?olYRSmeOREGN3iZ1CywKMRy7CEtRoqKLbFXgJaNN1ZlKEBXB/DAaHw1wCp/L?=
 =?us-ascii?Q?4L2fEZgS+560nzJ+qVSgoUK+mf523j3Tba5TNcku1ciqgP9UgFE0ocaME/6x?=
 =?us-ascii?Q?6vYZ+S5pnZbZWNh0yB9kaS1FhOHQjSsYstaMeC4d2X3d3ffhzvtBWRlLpRpi?=
 =?us-ascii?Q?23k2pWiNu1jCkZbtAOnhTf9GwRr+O5XDeaT1ACCI6mhywyKUzq+Y180NshrC?=
 =?us-ascii?Q?VCl3iPS6mOwhx3spVMtKR9FP8WW1QJfByKHKUAIoJvP0JiKKqrNpxrj7iKdY?=
 =?us-ascii?Q?MPvtDTFdqlHQLLMbQPE2rbuwFKLaDyefOHVXPEgYyho5Bfna/xxrG1LC0j8N?=
 =?us-ascii?Q?YxLUUY4mASOH3e8bKF0fPCGnvTw8N+1pHdJAGVeClJy6cQB0EGuLuiRTi7jG?=
 =?us-ascii?Q?P0G0oH+4IGCQjd7QP3O2fwwER/iSJxOmDPpghk0obR0iAa7TqqkDFXTJ++8e?=
 =?us-ascii?Q?fYOsv+3e6Oa2faL+nHG30Gvujtr2nGyHyRXi7qzUO1iP+c5ULc9J41Fy7Wu8?=
 =?us-ascii?Q?ccbZAbZzm6SfVzfU8wFOilyL5xa4qZkb8v6DK+hwH/y1W3fhzu2rjBMbQkDd?=
 =?us-ascii?Q?nUZfM5i8ZFhxtqrJmrWcX443rJr65ojzkydK/p8X76YDgqoWwGEEnkpfRhBL?=
 =?us-ascii?Q?f2grEAJsymUOFq0tdxnTFRUByPnKzu565fE+wYqQvgNh3hEHBPUxTw7YQWz/?=
 =?us-ascii?Q?pp3dpVHzlqvIdygSXyfqSLTOakMNlVkw9jWj5g/lfxO0y0VFCXoh4eGrkxIA?=
 =?us-ascii?Q?buTJVQXLh/GMewbl1qIyO9GH65FZcLEYlsUT0ChlH2Oaj2IU1hUDZ9lyXcvv?=
 =?us-ascii?Q?RloHzsufFjg+N8YP0JahPvZAtDzvPGngfOxPHRkJrR3r8gf2ZZFF1j616/Dd?=
 =?us-ascii?Q?ACAMcOHNO1wjZmp4GK6HqTsk0ztPnU65fod2gw3wVt+MPXc7hQJrB9VPLcEs?=
 =?us-ascii?Q?J1ItAr3gcInf3Rbu2gt/1Vu/eA2J7DpqLwsCQO877Esx9cQvv5BhcemMtWo5?=
 =?us-ascii?Q?9CcXwgFG36XlEhqDqmKvfiYx55tDokAdqfCGqL3KFvjlZCjjpQD0A0Ze6ttg?=
 =?us-ascii?Q?wDdcTGYfJ7RImZk9fnaqcvjeqjpEVH8BCYNGSQF35TLt6SRp4RHQZ9GMrzvO?=
 =?us-ascii?Q?9SNmrMs5CNiqFDDW9TPN3pIAr/vY3MFMQA0zJ9H4Wl0IwekxUWkpbf9Xc2mg?=
 =?us-ascii?Q?AZ+AoiWA6Ub4gfTU14pYY3molMr0FGDpn7JxioKxMuWh3l+gMGiI6LbH0DzD?=
 =?us-ascii?Q?okcv+e2oV+Ao0MZ9WbWOUtiO8fFTWTIQfZZYp66fd8KuedapB7lSSNvC6RKs?=
 =?us-ascii?Q?GD1Bottk4TVC9NwxXbQ9xu19dr+WPZ7Ii7bNQe+avIBJWZqPpEL1hvXGqmdd?=
 =?us-ascii?Q?XVsjSJ3OhtgxXqnK4PjrMCtGZGPFN1bPExeAhjwK3g17Xiy+GHHn0t/d8S3l?=
 =?us-ascii?Q?cLawm8BlYKpCnZ44TW8rfL8YYZ35XARFnEbp3YayD9PZey9RwPqGCkdfrFbL?=
 =?us-ascii?Q?CKN61E7Z7FT8BF29KXvY/W7QAlLjm80LOVNuFKxZCpzTD0tLijNNefHB6pVm?=
 =?us-ascii?Q?EQ=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	VUDFVU19w2bU72undWPWzmVF43gqeB5S0JMn2uNhQ6aFETBln5/HVvShGlYCt43/qv4Sm9++rDDgRLPQQCAihCeiqPoJ3DeRR5tt84KDbSUzPvHVu3+blwJVYNk0GRaegQmatdtDN/P4wEJm2xcDosucBqDPC1bPjlMHDBypFFqnW7dXccFeTzxQnfdo16niB2S+dT2EVNXLRUutoyDjADqbgpMPj33QbfBNabd8UmGZr8VvqcIZtv0W5xMb7ubD61tWkuW5AbK9KZQKlnredznKV3PlUVVENUCxTrD++gJTU7bmEFxGpfZGAroHefzL7idzO3UZfvuPhJPNOtY/okifEMP5QUYngtDrk58THxhFB52BW5iQvwkiIsZ5wGniYcjaxO8u7a7Cfq9ENSVjJdCSUOFndUfInyElyO2qKoZODhcJGmf2/AVKyY6t/XGoHJ5/sNnyoqkJKwbIcuhMl7kIFEcC1gM360d9leRiiat/rKocYSJfWDOUR7I83df4vH1Yr1brw98SXxmScm0qY2PbicQt48Xd/p2keApKohF1BiBT4c33qC+RzR9DfUak9ytEznmgxcBC0UVTRaiUw4pm12R+GK2DU1RAQNDqrH4=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6801088a-202b-4789-033b-08dc7eae1f1a
X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2024 00:35:49.2100
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xkUtwYL8ol1cNMKrq7P/WTRjEl8LpRm5Siw+zb0jYP6rqkwln3icCeSMbZ9pCVj0435EJd8KW7Wnpurp2/twQpbC72wlg1UGwimF1cNDfcU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR10MB6510
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-27_06,2024-05-27_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 malwarescore=0 mlxscore=0
 mlxlogscore=999 phishscore=0 spamscore=0 adultscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000
 definitions=main-2405280003
X-Proofpoint-GUID: DeB9pXs16TbrUZq8K_lkNXgjV-E5n00t
X-Proofpoint-ORIG-GUID: DeB9pXs16TbrUZq8K_lkNXgjV-E5n00t

Define __tif_need_resched() which takes a resched_t parameter to
decide the immediacy of the need-resched.

Update need_resched() and should_resched() so they both check for
__tif_need_resched(RESCHED_NOW), which keeps the current semantics.

Non scheduling code -- which only cares about any immediately required
preemption -- can continue unchanged since the commonly used interfaces
(need_resched(), should_resched(), tif_need_resched()) stay the same.

This also allows lazy preemption to just be a scheduler detail.

Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Rafael J. Wysocki" <rafael@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Originally-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
---
 include/linux/preempt.h     |  2 +-
 include/linux/sched.h       |  7 ++++++-
 include/linux/thread_info.h | 34 ++++++++++++++++++++++++++++------
 kernel/trace/trace.c        |  2 +-
 4 files changed, 36 insertions(+), 9 deletions(-)

diff --git a/include/linux/preempt.h b/include/linux/preempt.h
index ce76f1a45722..d453f5e34390 100644
--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -312,7 +312,7 @@ do { \
 } while (0)
 #define preempt_fold_need_resched() \
 do { \
-	if (tif_need_resched()) \
+	if (__tif_need_resched(RESCHED_NOW)) \
 		set_preempt_need_resched(); \
 } while (0)
 
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 4808e5dd4f69..37a51115b691 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -2062,7 +2062,12 @@ extern int __cond_resched_rwlock_write(rwlock_t *lock);
 
 static __always_inline bool need_resched(void)
 {
-	return unlikely(tif_need_resched());
+	return unlikely(__tif_need_resched(RESCHED_NOW));
+}
+
+static __always_inline bool need_resched_lazy(void)
+{
+	return unlikely(__tif_need_resched(RESCHED_LAZY));
 }
 
 /*
diff --git a/include/linux/thread_info.h b/include/linux/thread_info.h
index 65e5beedc915..e246b01553a5 100644
--- a/include/linux/thread_info.h
+++ b/include/linux/thread_info.h
@@ -216,22 +216,44 @@ static __always_inline unsigned long read_ti_thread_flags(struct thread_info *ti
 
 #ifdef _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H
 
-static __always_inline bool tif_need_resched(void)
+static __always_inline bool __tif_need_resched_bitop(int nr_flag)
 {
-	return arch_test_bit(TIF_NEED_RESCHED,
-			     (unsigned long *)(&current_thread_info()->flags));
+	return arch_test_bit(nr_flag,
+		     (unsigned long *)(&current_thread_info()->flags));
 }
 
 #else
 
-static __always_inline bool tif_need_resched(void)
+static __always_inline bool __tif_need_resched_bitop(int nr_flag)
 {
-	return test_bit(TIF_NEED_RESCHED,
-			(unsigned long *)(&current_thread_info()->flags));
+	return test_bit(nr_flag,
+		(unsigned long *)(&current_thread_info()->flags));
 }
 
 #endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H */
 
+static __always_inline bool __tif_need_resched(resched_t rs)
+{
+	/*
+	 * With !PREEMPT_AUTO, this check is only meaningful if we
+	 * are checking if tif_resched(RESCHED_NOW) is set.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_AUTO) || rs == RESCHED_NOW)
+		return __tif_need_resched_bitop(tif_resched(rs));
+	else
+		return false;
+}
+
+static __always_inline bool tif_need_resched(void)
+{
+	return __tif_need_resched(RESCHED_NOW);
+}
+
+static __always_inline bool tif_need_resched_lazy(void)
+{
+	return __tif_need_resched(RESCHED_LAZY);
+}
+
 #ifndef CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES
 static inline int arch_within_stack_frames(const void * const stack,
 					   const void * const stackend,
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 233d1af39fff..ed229527be05 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -2511,7 +2511,7 @@ unsigned int tracing_gen_ctx_irq_test(unsigned int irqs_status)
 	if (softirq_count() >> (SOFTIRQ_SHIFT + 1))
 		trace_flags |= TRACE_FLAG_BH_OFF;
 
-	if (tif_need_resched())
+	if (__tif_need_resched(RESCHED_NOW))
 		trace_flags |= TRACE_FLAG_NEED_RESCHED;
 	if (test_preempt_need_resched())
 		trace_flags |= TRACE_FLAG_PREEMPT_RESCHED;
-- 
2.31.1