Received-SPF: pass (google.com: domain of linux-kernel+bounces-191556-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
From: Ankur Arora <ankur.a.arora@oracle.com>
To: linux-kernel@vger.kernel.org
Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org,
        paulmck@kernel.org, rostedt@goodmis.org, mark.rutland@arm.com,
        juri.lelli@redhat.com, joel@joelfernandes.org, raghavendra.kt@amd.com,
        sshegde@linux.ibm.com, boris.ostrovsky@oracle.com,
        konrad.wilk@oracle.com, Ankur Arora <ankur.a.arora@oracle.com>,
        Ingo Molnar <mingo@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>
Subject: [PATCH v2 11/35] sched: __schedule_loop() doesn't need to check for need_resched_lazy()
Date: Mon, 27 May 2024 17:34:57 -0700
Message-Id: <20240528003521.979836-12-ankur.a.arora@oracle.com>
In-Reply-To: <20240528003521.979836-1-ankur.a.arora@oracle.com>
References: <20240528003521.979836-1-ankur.a.arora@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?sZ7fQCiXxofVeHOfvWqrD1f5eRcFXDTULkJ3iU3kICISD6RcuZtxDp3wk4st?=
 =?us-ascii?Q?mB5t8cbKT2YjR8owhS4Dts66qWKqpSv4oX5W3Op936hkhoZFpbPZObKj/46w?=
 =?us-ascii?Q?4Buxi6bEMTGl05DvodZGYfM4f4ZzyiDfRJFib4TUnfCHVAaHomH/llk6Jv2q?=
 =?us-ascii?Q?YNtg/nTOjMjiXr7fIe/Visr7mRyLplCXfadEmOkEzeCMeP3/LgWMcjjcNSDP?=
 =?us-ascii?Q?03cftltud920Cv1JwtOBGcfiWLTKQ4Ho4GKGOEvu1bemdzmInLgOw4bU6Wp8?=
 =?us-ascii?Q?gdLpFHxPUaMCZBTvp1/oh2WKnN6T88e3dr0uhnTrfjFzWu6pm2mTP7sgjtPN?=
 =?us-ascii?Q?pJB2kKYFTA6B72TUiiPg3tBCGwRIprS/oECu8Gf69XeQhMTxbluKr8nso6EB?=
 =?us-ascii?Q?IMf0guJFxME3CN9g3reovIIrS8/8So/NyysdhEq/hhK6vcekqdF3KRF/fYO0?=
 =?us-ascii?Q?VWQBfQS5RcHD/9ByzoC+snhUK0AaoWhhUw2uN0v5M36vkE7HZ7qiUWPrlc8Y?=
 =?us-ascii?Q?Ks51lyV40j6H4z8IjIv0QVUS3OQi/E2lNcnGnwquzAcIZPzIf9msZoNpFYeq?=
 =?us-ascii?Q?ENWUaGjxN4g6152uVCuw048MeWoWqKgu2oYAoJC8gVGJ0oQqDrqWW7GKCvGx?=
 =?us-ascii?Q?uP9GjklSbbGdnQAu80jTk0VbfaOMV0CuQa6W0c2Kz8rcNT8K5Z1hNyl9gd2u?=
 =?us-ascii?Q?+RxErZh5rkw983jLA+DUfMue/Y9ew+aU+S8E8EuoPoxdyMAyGxtfjZIIUfZi?=
 =?us-ascii?Q?sBIn1wCdXfHesO9ZT9b2vPJCnybU23ED5xMJOkiz2ikwtUca3NEBaA7akUos?=
 =?us-ascii?Q?xjTnrg+dr8eEIJ/SRrxIn2GwqzOvwE7ahqmbSEbHNTAAGbZDD0SOBJFsHTX6?=
 =?us-ascii?Q?qt9CSYRxJtuYcHlVx57n9MWsfcRX1Cv8g3AmaS4SrkcX3Rmj2gQyXWxGF3XS?=
 =?us-ascii?Q?obMYLVBwIolRl65V3lfbgAogPpqCbNK+GKH0vkgvMTHosDP9pb0D60HEqptf?=
 =?us-ascii?Q?9zGvktUF3p/8qKn6tpUJh6CtNZtkriBSLNEX9R+Nb0kIV897RJSwfHEYZS7T?=
 =?us-ascii?Q?KYTK1aT5Tc1hhJPNCSrCdJzXA/swI1ntg6W4FweR5dGpHceMmDHhXq4oSogy?=
 =?us-ascii?Q?g05I2JHcUidVHM4zYvF+PHpX8d55Nen+HfYmgcEuj+bxMqGYwJO8b3gP5bga?=
 =?us-ascii?Q?JQE7RkG8kOsfDWyw0b5/Kd666yMp3Z0zwUWtofxDY1E3Yyiy8hKVIorxJH9a?=
 =?us-ascii?Q?ul7B3zefoDBmYYXgBjRhhQbkR3O0WpzzsSFwQQTJaXBdqJ7vDLbkcnyhVMUs?=
 =?us-ascii?Q?FMJ0dvZmPERcdavZGe3Ha4WWJHLAGTICMTEOmWdyBjET/YkkoCuVd8ABdIVk?=
 =?us-ascii?Q?jWgxVPf31ut8W9qAFxNmKuzJepiGMQ6zkttfpyqjUm8xYyBQ6WONIvWiqIdL?=
 =?us-ascii?Q?zxiMxTn7TSO2B9rYYQ05Ib94cA4zzIVSTtKgKBSxpCJs5++giErzWdc3ujj5?=
 =?us-ascii?Q?Rh+MNTLVL1WyEztKBCqqSBTraz2Is8Jn4GFh7eaowpUm2CXtL8gkj5QIeLBu?=
 =?us-ascii?Q?peqesSdh7S+IGJoIr3j1M9v63R7siGyKhNNMhzvsj8tus0lUGkhr6qsGNBrH?=
 =?us-ascii?Q?3w=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	uK7c5mXdgnHx29WeblvBArUVjU+ZYGNelTnvz+Lu2/bTJOn1Cw5GAeKE+JYFLw/RcsQiW5F+oXrIggQoGz8B0KkZxNkPHuXjt7ngzmj27xHe3nVQRB+p3eq/niB2RtrjbgIGgESNUtkRLnzNICGlFaZUHwYGl+4vVyfuDP0gX6efVu9BfGcfneqeac662SOZ2DLXGE0hDJOEgke7Y+nBpLpB0y0EVWFt16w2SGn94bgkl2m9N4VNmkiEVT7uFUivVbaK4qAUwyLcp2xDmfakNreR/Sf1CvXaQop6CUF55kYKRYSPQ/KQtWLJxuiLYIUYXK5wQgIJaKqgCTpllouzvuR4KdqOo7C/w3FLjn03nyFp8/UMGSYs0MtKNu9s19q50nbCKi9ouZ55MzKdg7ZSBiT7rjZAMGmk9hNAoVObBNMRaKbzGUyenDWOpJSkSuLksJhPqO3mBU+348M11FSPhHwbVEZcn52hdJ0AW3Oje24h2Lm2a+DMmwk0mBPU4Cdy+jWW5rDDtpjWLN6kUkCw05riWSBUfnYuu0VjOw356dNlPE2gWBNOdrtHrflshM+xDpGqLq3omKS+gX09A4OcquAvW9dl9rEsJ40XDZ2Lme8=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 67e8db62-4a52-43de-699a-08dc7eae2625
X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2024 00:36:01.0456
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: FWhMFlOQTmWSdg1YrOO9NQtRUlH/ekPhaTutWho1oAnkqcGDjldbbhY3vXqc2XNJliJSYBAmCRIRJSaJgnjDy9Hk4VQaWW5cqilB8wduVwE=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR10MB6510
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-27_06,2024-05-27_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 suspectscore=0
 adultscore=0 phishscore=0 malwarescore=0 mlxlogscore=690 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000
 definitions=main-2405280003
X-Proofpoint-GUID: x4FTu6GFzsKXML3cIjG-gtRpTD9OXSQG
X-Proofpoint-ORIG-GUID: x4FTu6GFzsKXML3cIjG-gtRpTD9OXSQG

Various scheduling loops recheck need_resched() to avoid a missed
scheduling opportunity.

Explicitly note that we don't need to check for need_resched_lazy()
since that only needs to be handled at exit-to-user.

Also update the comment above __schedule() to describe
TIF_NEED_RESCHED_LAZY semantics.

Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Originally-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
---
 kernel/sched/core.c | 28 ++++++++++++++++++----------
 1 file changed, 18 insertions(+), 10 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index d00d7b45303e..0c26b60c1101 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6582,20 +6582,23 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
  *
  *   1. Explicit blocking: mutex, semaphore, waitqueue, etc.
  *
- *   2. TIF_NEED_RESCHED flag is checked on interrupt and userspace return
- *      paths. For example, see arch/x86/entry_64.S.
+ *   2. TIF_NEED_RESCHED flag is checked on interrupt and TIF_NEED_RESCHED[_LAZY]
+ *      flags on userspace return paths. For example, see kernel/entry/common.c
  *
- *      To drive preemption between tasks, the scheduler sets the flag in timer
- *      interrupt handler scheduler_tick().
+ *      To drive preemption between tasks, the scheduler sets one of the need-
+ *      resched flags in the timer interrupt handler scheduler_tick():
+ *        - !CONFIG_PREEMPT_AUTO: TIF_NEED_RESCHED.
+ *        -  CONFIG_PREEMPT_AUTO: TIF_NEED_RESCHED or TIF_NEED_RESCHED_LAZY
+ *           depending on the preemption model.
  *
  *   3. Wakeups don't really cause entry into schedule(). They add a
  *      task to the run-queue and that's it.
  *
  *      Now, if the new task added to the run-queue preempts the current
- *      task, then the wakeup sets TIF_NEED_RESCHED and schedule() gets
- *      called on the nearest possible occasion:
+ *      task, then the wakeup sets TIF_NEED_RESCHED[_LAZY] and schedule()
+ *      gets called on the nearest possible occasion:
  *
- *       - If the kernel is preemptible (CONFIG_PREEMPTION=y):
+ *       - If the kernel is running under preempt_model_preemptible():
  *
  *         - in syscall or exception context, at the next outmost
  *           preempt_enable(). (this might be as soon as the wake_up()'s
@@ -6604,8 +6607,8 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
  *         - in IRQ context, return from interrupt-handler to
  *           preemptible context
  *
- *       - If the kernel is not preemptible (CONFIG_PREEMPTION is not set)
- *         then at the next:
+ *       - If the kernel is running under preempt_model_none(), or
+ *         preempt_model_voluntary(), then at the next:
  *
  *          - cond_resched() call
  *          - explicit schedule() call
@@ -6823,6 +6826,11 @@ static __always_inline void __schedule_loop(unsigned int sched_mode)
 		preempt_disable();
 		__schedule(sched_mode);
 		sched_preempt_enable_no_resched();
+
+		/*
+		 * We don't check for need_resched_lazy() here, since it is
+		 * always handled at exit-to-user.
+		 */
 	} while (need_resched());
 }
 
@@ -6928,7 +6936,7 @@ static void __sched notrace preempt_schedule_common(void)
 		preempt_enable_no_resched_notrace();
 
 		/*
-		 * Check again in case we missed a preemption opportunity
+		 * Check again in case we missed an eager preemption opportunity
 		 * between schedule and now.
 		 */
 	} while (need_resched());
-- 
2.31.1