Received-SPF: pass (google.com: domain of linux-kernel+bounces-191553-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
From: Ankur Arora <ankur.a.arora@oracle.com>
To: linux-kernel@vger.kernel.org
Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org,
        paulmck@kernel.org, rostedt@goodmis.org, mark.rutland@arm.com,
        juri.lelli@redhat.com, joel@joelfernandes.org, raghavendra.kt@amd.com,
        sshegde@linux.ibm.com, boris.ostrovsky@oracle.com,
        konrad.wilk@oracle.com, Ankur Arora <ankur.a.arora@oracle.com>,
        Andy Lutomirski <luto@kernel.org>
Subject: [PATCH v2 08/35] entry: handle lazy rescheduling at user-exit
Date: Mon, 27 May 2024 17:34:54 -0700
Message-Id: <20240528003521.979836-9-ankur.a.arora@oracle.com>
In-Reply-To: <20240528003521.979836-1-ankur.a.arora@oracle.com>
References: <20240528003521.979836-1-ankur.a.arora@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?Qqeu9Zwah5sFO2yJHHQijEhol+C+ruuVZ/R3XqkicTPhqv5nGN7YrpIm+qZs?=
 =?us-ascii?Q?z1yoCPrXyPT+A4WzyDVj2KtGfIWP9k0ZzXohVSInYj9Y70TvYegmI6P8ykqt?=
 =?us-ascii?Q?Iv5yTf0v0df3HZu/7hHGhOFp7GVUisDXAYkqLxW+QPkx01/422FeTq5LHPsx?=
 =?us-ascii?Q?iQG3dvPRjY/7EP68qVlWN/o7HWKLcpDqaFkDeAK2ohwkEi+8lZYC+dGfAc0v?=
 =?us-ascii?Q?LK+sm7dR8wEQMIQkuGozirdRWWWeX9vcpBzD0DljzN8iJ/0pXCEQvmvOfyJY?=
 =?us-ascii?Q?LWK9BMzdneN0tWxiN59Y71iTLZfV6koD4vT+uiV0Cbzg4Q2xkeZWDp0vJ4lw?=
 =?us-ascii?Q?elhFcMrahiY0IhoRNZooLdpBf+lb71E0nUyG8/0WTdhfIHS82aX/ecoZiZ28?=
 =?us-ascii?Q?7LHa+TBOD9zq4YZD19LKvHuKpTqmJuMuNhURhK1DapnkoP7dk/UBMQ4L/3mn?=
 =?us-ascii?Q?+1KDjDOym6araGXvdL9n0QGky+HIv3bfC4BEC3IsSBSxlTGbDc2b3e59CKRa?=
 =?us-ascii?Q?8CMHRp6INWj4UtUOZGnTAI2XjNh8sOGX92XGiP33L2ln0uQuAnYPvLrkB9Ih?=
 =?us-ascii?Q?8ixknWbp0ijTH8yogIJDWZh2NOW0dDssoVbSEGjZvLnBKbaK4gHcU+7TCnRX?=
 =?us-ascii?Q?V8vBxol9Hk1Np+GknNG/UaeP9Ory1V7fUouyv1ljvbVw9gidssi+KuUgI2Kb?=
 =?us-ascii?Q?XJ1/+9AVPalSGmcHp9jspfDhVw6hkCyIEMrUrEwIw0HrdNy+qINFoz1AtyuJ?=
 =?us-ascii?Q?JC6fBj+gz9q9eatUcXR8/VILr0+fIXlR/W5UDFY23Oby7zg8qkkDM8D7syjD?=
 =?us-ascii?Q?Dg66tOq7iosgclAZ6bCzD1Zn/2ojbTm90B4SQRMLy1pHDBOlfj+6CCoP8gEn?=
 =?us-ascii?Q?86qsCJW5Iy1Vwbe+n1CnthsrNiUTLtd+KBVj6IreC+VBAlm5RtFCYbQ297kV?=
 =?us-ascii?Q?WaZgCyd+paxW9LdGmaFbEltoEJdxQ8opelIxktiK5e6gQ7iNCezlVKfz6rPk?=
 =?us-ascii?Q?4a4KDvzZs5T9rUPG9bLrtBkuhq1sDhFAIoz+QRrR+rqHMMVDPd83hXE3YAk2?=
 =?us-ascii?Q?CSJV3N0ADZxhcdNwJZD1Sq5R4SnVAXwicBOAEg3S921qR/UJTpevaK6QH8a2?=
 =?us-ascii?Q?5iUepjBsoduYlg1sKlIcSPHGghc0uRSljuhUyslyfSaXwUQKciZi2iAQ2OGg?=
 =?us-ascii?Q?1T/IPLOdurgzIOYZcle0qHKzjEvHFcdeNSNveXP0/dWSOD2xG0yy/3IoCbh3?=
 =?us-ascii?Q?WGhJhrtIod1R5OcGpWbXKbAV1UY8BEMrWnxs1sZog13K14GYyNpcLUBpTIW+?=
 =?us-ascii?Q?sj7fkEd5PJEb36GiBjq1uLfLosJesySZMSX/V3H0Lwen1TxXmvu9Lmrdp9+S?=
 =?us-ascii?Q?Ksgu2l4pe1Qmcp7gYaxfdsXKHyHXCHWitNAQjhFP9Wg+qYkp1h+tzEqM7wOJ?=
 =?us-ascii?Q?9HtsptiYQOtgRNUl3ZZcNswq21zCnoE4X1c8GR4o+X4RImtgWTRodR+ZEF9A?=
 =?us-ascii?Q?BMHN/iiYVBtJCwluFNzIV2E2q/f/cov+YO85SL7fPXk0fKXeZ149V8Qkuej5?=
 =?us-ascii?Q?7IbKWzLYrrI5wmeKK8Jl5mG008gPZNPbM0I5UMu2uuW33qcngrWQT52vu9gd?=
 =?us-ascii?Q?RQ=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	wcANLd2djhZmuPgZytLKN7mI4bMsmrGXV02O4V+xZ3CAKewTX1fqw2e+0S8ZQ55yYE35mxHGhChLiVbe2AW72mqzQk+KkdaQoBlAiqal2DHr03tmwoTgfumDvRCXui5XNYjTUDbLe6zy19C4itsucOH73c8PlkSaf+YiYD4+o9SoxA18htKtQTdKX9yj4SBm8BM/HDa/6DLWVMwtspi9kXEd9q6OF/4I3ZYzrM4BtJAeB6dG2XF6SLMuMd9dO5WfP7KTSnuChJRmqWlW2a/bdgTp/BTyFZbYVJlHKIs7fP77TcBSVVFQ2ZEEFh7nLq+J6rMGc1yif/OMUiIMW90+mrX9ZIoEAxdI7lzXbFM2MzPNDNXOTeBVuibqxoNkrsXFF8hXBCIXqHOOzU+HuGkD2aJODHSq9b/6PUFRFhPYilZ3vJTgxjvhuA8Ym/QBPNzqflaQYS1YPvwvPd2PBWCbcVLBKxXa0bn+NAG6htZJHwkSMmgsVA3My3cyk9rjn4PiTZpEOezS1RuPXnrIqO2qxiYmgFzce+YB5LA2LR/zLLjkN/s6C09lqioWeD0gdnMgOByBj5pPYsCvTF/1H1g6SvaaH/LIIoUJEhlURP2MWRg=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2882bb66-1f03-456d-e94c-08dc7eae211b
X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2024 00:35:52.5419
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: x8vCoL+2h5XF5+eHswu+KFTB31ZgQyaU7xQDM943slH8OvRDV28ChlR1+PuBs9U/SrkPqa5OFDMqjc2ab7xHIQ7OzmI8Rw9ImXSg3/1suwc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR10MB6510
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16
 definitions=2024-05-27_06,2024-05-27_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 suspectscore=0
 adultscore=0 phishscore=0 malwarescore=0 mlxlogscore=812 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000
 definitions=main-2405280003
X-Proofpoint-ORIG-GUID: 4rDj__0LxYdJV7D2HWdPcQU-M3BpC4v6
X-Proofpoint-GUID: 4rDj__0LxYdJV7D2HWdPcQU-M3BpC4v6

The scheduling policy for TIF_NEED_RESCHED_LAZY is to allow the
running task to voluntarily schedule out, running it to completion.

For archs with GENERIC_ENTRY, do this by adding a check in
exit_to_user_mode_loop().

Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andy Lutomirski <luto@kernel.org>
Originally-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
---
 include/linux/entry-common.h | 2 +-
 kernel/entry/common.c        | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/linux/entry-common.h b/include/linux/entry-common.h
index b0fb775a600d..f5bb19369973 100644
--- a/include/linux/entry-common.h
+++ b/include/linux/entry-common.h
@@ -65,7 +65,7 @@
 #define EXIT_TO_USER_MODE_WORK						\
 	(_TIF_SIGPENDING | _TIF_NOTIFY_RESUME | _TIF_UPROBE |		\
 	 _TIF_NEED_RESCHED | _TIF_PATCH_PENDING | _TIF_NOTIFY_SIGNAL |	\
-	 ARCH_EXIT_TO_USER_MODE_WORK)
+	 _TIF_NEED_RESCHED_LAZY | ARCH_EXIT_TO_USER_MODE_WORK)
 
 /**
  * arch_enter_from_user_mode - Architecture specific sanity check for user mode regs
diff --git a/kernel/entry/common.c b/kernel/entry/common.c
index 90843cc38588..bcb23c866425 100644
--- a/kernel/entry/common.c
+++ b/kernel/entry/common.c
@@ -98,7 +98,7 @@ __always_inline unsigned long exit_to_user_mode_loop(struct pt_regs *regs,
 
 		local_irq_enable_exit_to_user(ti_work);
 
-		if (ti_work & _TIF_NEED_RESCHED)
+		if (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY))
 			schedule();
 
 		if (ti_work & _TIF_UPROBE)
-- 
2.31.1