Received-SPF: pass (google.com: domain of linux-kernel+bounces-62980-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
From: Ankur Arora <ankur.a.arora@oracle.com>
To: linux-kernel@vger.kernel.org
Cc: tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org,
        paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org,
        bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com,
        mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org,
        willy@infradead.org, mgorman@suse.de, jpoimboe@kernel.org,
        mark.rutland@arm.com, jgross@suse.com, andrew.cooper3@citrix.com,
        bristot@kernel.org, mathieu.desnoyers@efficios.com,
        geert@linux-m68k.org, glaubitz@physik.fu-berlin.de,
        anton.ivanov@cambridgegreys.com, mattst88@gmail.com,
        krypton@ulrich-teichert.org, rostedt@goodmis.org,
        David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com,
        jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com,
        boris.ostrovsky@oracle.com, konrad.wilk@oracle.com,
        Ankur Arora <ankur.a.arora@oracle.com>
Subject: [PATCH 24/30] sched: support preempt=none under PREEMPT_AUTO
Date: Mon, 12 Feb 2024 21:55:48 -0800
Message-Id: <20240213055554.1802415-25-ankur.a.arora@oracle.com>
In-Reply-To: <20240213055554.1802415-1-ankur.a.arora@oracle.com>
References: <20240213055554.1802415-1-ankur.a.arora@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?HSf5bciRkztbCdhUqBpGpkrDAeJ7cNlnZT2LhwXgTxS63BuMeMpo469AYDde?=
 =?us-ascii?Q?egPIk/0MwMVJQLg78cr2dny+P01Ftxy8W+hriHl93cZG1kfqHT3gViiIBbXk?=
 =?us-ascii?Q?DZeBiJHMgwzDVCMkj6aXkDZh+ysovrdT0BIZn0VWKN55qes8tWKnvFEhIvqS?=
 =?us-ascii?Q?rW/Y8W3qI3FSC93PauF0zjG3lXYFYQzyvLtdNYl2e5mxV/9+BmD4KzV3TSiG?=
 =?us-ascii?Q?J9FReeaR830SCI/JI6mo8XZzTOof+iijZff5+M5VYjY0mdxs3zkOOSAWU6S/?=
 =?us-ascii?Q?CkozqCVGMiEy6qaYxQq2j5p9KARFBxFLM5C9nL6GUFC0u+Q8r+m+rxd9EK1u?=
 =?us-ascii?Q?AwGEg3WxNAv5F6TZnPLHlEGzyvNMJj6O5t+xPiyqEBvNqE3YMxc5UM57DRkx?=
 =?us-ascii?Q?qB2o0E/SI7wYVBhsNhoSPeYLcoDgXRovIb3ecsUlTjTKY9sCf3SVw8CN41NX?=
 =?us-ascii?Q?HqrkfmwE22os1AOhtDo19Qny4708RA6GMtDcVJQqcnkjyRt9KJds/96hP4aR?=
 =?us-ascii?Q?bWC4c6jPCDDzkD89HN/wG7wPeN8K9Fv7vawIiUo2r6Ec/hKXImqODLm6HA7s?=
 =?us-ascii?Q?/eXurP55HymUfsWwcVG5AHRBpHgBN1/UAZhgkwgrlTZKTaRjzvgy3BMhUSCj?=
 =?us-ascii?Q?8NEptylOykTakk29PAzW9dGWs2Mxvn1IKo4sJCZY1paaVudl5inwwWgjZLoQ?=
 =?us-ascii?Q?KiHMQ1Pg10kcSlududwxqqLCrRxwEcyIu6q9mrGKaXk8dJ0ZdF0MvZh+h/hk?=
 =?us-ascii?Q?TXts0uU+qrlSbpjCb7BPBZKKcln2MJEAAW+PXcRGSe2B5oMAcGgoV5p4SavI?=
 =?us-ascii?Q?nUi3kMcMeCRKIVLY9v/5TO3Tsckl3hJIwmsePDOpIha1oX/m9RYLy8KDW9us?=
 =?us-ascii?Q?ymDTxi5yyimfwjanIzqbp7IJth+bUrCUduGAYlLujiSh0pskdIxOip7RsbRN?=
 =?us-ascii?Q?ncfpjLn3ddeNQrd6xLhhQrCWCAAbisagoJZFl1Pc1p22XHcKY0ry1GyitsAh?=
 =?us-ascii?Q?yL/+ek5+orWwZ2RO/V0se4kPeml7EVjHJb01xBeRzH19MVgdy/jJHA6M6ALa?=
 =?us-ascii?Q?UZiN0/YYauBlPKLVduar/N6N7h5CnSQo15h56JZ6okOjTcltIWR6jMc1UsbR?=
 =?us-ascii?Q?XlSK3l5v3C+lmHtJ7XtKbZ0LEnUwlHk7/1MpbomWjel5WSpNh7NsJdxoOa8E?=
 =?us-ascii?Q?a7Y1dE0a3tQ+La0X5evLGpuGlAZI7y38/A56rgrq36JAGB6g0Zm/SSGcYOHi?=
 =?us-ascii?Q?Y62o6Rm2Aeuv0cB/bMDbxxFQHadIcaRlQfFs2RAb4faqTKZljH8iERMluf24?=
 =?us-ascii?Q?gdXybiNkjhqUWYVnE2iXWrBsv8XWz6nsmODJWqDvzbB8nI98cOzfxqCbCxQf?=
 =?us-ascii?Q?EEJNOtw6zz0kxx5+3EqetZil2dkLsToBZUR+j8l1X8Nv1lCtg5V5faNx1JHD?=
 =?us-ascii?Q?Z817JpJzOu0HlHVDSgitxFPRBktAiNwKi0AMwC2adqwRn/AsN5Z7ALREq5Cj?=
 =?us-ascii?Q?oL9gCjO7cJ6MPKMrnsXafGuwbT+5YhmJvUC2T8GTIpjs8mqjrR0240ti/NYg?=
 =?us-ascii?Q?+bEZff8S+0H0yv/vDfXV1JiN6pToEWIo2o8N2DGyq+1s4BEhnTMFrpuHEt8W?=
 =?us-ascii?Q?ig=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	lRHOTcHJvHDdogDE5NOKvzw7e/eRfdL79u/2wYZ0sfNkJssOuqHpsbwj78UR5wa2vKtUbvVyCRUW24PLAoWsk3XWhVjDBkgJpzN3VXIlwB/SqaA426znzHLULAe7SGoRrCpYevUBQfhGDFsNKl1O073cOvdOy0q5YFU+kUJWRBXHUBtUhzmIT58K7PjodK1tmd4PCZddEaRjPJNeRIXFIJvE64L/JPzKfrGDTGzvOOiUMh1AEU29f7bGOhRik5CclddgzZ/O4mfMEnq0rnz5FhHw1A5iKCmimnC9DRBozCEX5W4pi/M/Il0kS1OArwP/cxLO/jLzY01z5sCBHUAORMK24PDN+HQTLCcddMGqf9Wi/iw7ddtqdLJtv6PkzQtXmd36b06YTe3BzDiKRFeoOl4BpBHZ0YSZwjUmW36mqFM3GxBAx824bM5NlEsLUpuiuvWMSHGX0lq6FfacBdv/oq6AJ4qz8qXkLDcpIRzAicfDrVM2A/0E57ezUFvsMPe6D5ySLWasOK81IQR3+nXfwQuo4QNzKtgYgeLoV9kLtB8g9U8slinwbbc0RocxQ/8v8Tz2htAG+p7JxjQlLjCdsmKr+ORrR0uZsGjbiK5K79M=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: acc2334a-a718-4af6-f99a-08dc2c589178
X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 05:56:49.3059
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zGkd4Me5cL0Fz8It+vEz+xsF1PBLHIjM97PW7q6ikGdhUeBv/P4i72+zVR8TA5/g54OytM+lizScj39TtfC64j26fP0OMO4KWdo6CjHADms=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6906
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-13_02,2024-02-12_03,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0
 mlxlogscore=999 malwarescore=0 mlxscore=0 spamscore=0 suspectscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311290000 definitions=main-2402130043
X-Proofpoint-ORIG-GUID: 3r-F3ECbcNT-BSRGFXVfrcseIKsoCuz8
X-Proofpoint-GUID: 3r-F3ECbcNT-BSRGFXVfrcseIKsoCuz8

The default preemption policy for the no forced preemption model under
PREEMPT_AUTO is to always schedule lazily for well-behaved, non-idle
tasks, preempting at exit-to-user.

We already have that, so enable it.

Comparing a scheduling/IPC workload:

 # perf stat -a -e cs --repeat 10 --  perf bench sched messaging -g 20 -t -l 5000

 PREEMPT_AUTO, preempt=none

	 3,074,466            context-switches      ( +-  0.34% )
	   3.66437 +- 0.00494 seconds time elapsed  ( +-  0.13% )

 PREEMPT_DYNAMIC, preempt=none

	 2,954,976            context-switches      ( +-  0.70% )
	   3.62855 +- 0.00708 seconds time elapsed  ( +-  0.20% )

Both perform similarly, but we incur a slightly higher number of
context-switches with PREEMPT_AUTO.

Drilling down we see that both voluntary and involuntary
context-switches are higher for this test:

 PREEMPT_AUTO, preempt=none

	  2115660.30 +- 20442.34 voluntary context-switches   ( +- 0.960% )
	   784690.40 +- 19629.42 involuntary context-switches ( +- 2.500% )

 PREEMPT_DYNAMIC, preempt=none

          2049027.10 +- 35237.10 voluntary context-switches   ( +- 1.710% )
	   740676.90 +- 20346.45 involuntary context-switches ( +- 2.740% )

Assuming voluntary context-switches due to explicit blocking are
similar, we expect that PREEMPT_AUTO will incur larger context
switches at exit-to-user (counted as voluntary) since that is its
default rescheduling point.

Involuntary context-switches, under PREEMPT_AUTO are seen when a
task has exceeded its time quanta by a tick. Under PREEMPT_DYNAMIC,
these are incurred when a task needs to be rescheduled and then
encounters a cond_resched().
So, these two numbers aren't directly comparable.

Comparing a kernbench workload:

  # Half load (-j 32)

                         PREEMPT_AUTO                           PREEMPT_DYNAMIC

  wall            74.41 +-     0.45 ( +-  0.60% )         74.20 +-    0.33 sec ( +- 0.45% )
  utime         1419.78 +-     2.04 ( +-  0.14% )       1416.40 +-    6.07 sec ( +- 0.42% )
  stime          247.70 +-     0.88 ( +-  0.35% )        246.23 +-    1.20 sec ( +- 0.49% )
  %cpu          2240.20 +-    16.03 ( +-  0.71% )       2240.20 +-   19.34     ( +- 0.86% )
  inv-csw      13056.00 +-   427.58 ( +-  3.27% )      18750.60 +-  771.21     ( +- 4.11% )
  vol-csw     191000.00 +-  1623.25 ( +-  0.84% )     182857.00 +- 2373.12     ( +- 1.29% )

The runtimes are basically identical for both of these. Voluntary
context switches, as above (and in the optimal, maximal runs below),
are higher. Which as mentioned above, does add up.

However, unlike the sched-messaging workload, the involuntary
context-switches are generally lower (also true for the optimal,
maximal runs below.) One reason for that might be that kbuild spends
~20% time executing in the kernel, while sched-messaging spends ~95%
time in the kernel. Which means a greater likelihood of being
preempted due to exceeding its time quanta.

 # Optimal load (-j 256)

                         PREEMPT_AUTO                           PREEMPT_DYNAMIC

  wall           65.15 +-      0.08 ( +-  0.12% )           65.10 +-      0.19 ( +-  0.29% )
  utime        1876.56 +-    477.03 ( +- 25.42% )         1873.63 +-    481.98 ( +- 25.72% )
  stime         295.77 +-     49.17 ( +- 16.62% )          294.41 +-     50.79 ( +- 17.25% )
  %cpu         3179.30 +-    970.30 ( +- 30.51% )         3172.90 +-    983.26 ( +- 30.98% )
  inv-csw    369670.00 +- 375980.00 ( +- 101.70% )      390848.00 +- 392231.00 ( +- 100.35% )
  vol-csw    216544.00 +-  28604.60 ( +- 13.20% )       205117.00 +-  23949.50 ( +- 11.67% )

 # Maximal load (-j 0)

                         PREEMPT_AUTO                           PREEMPT_DYNAMIC

  wall           66.02 +-      0.53 ( +-  0.80% )           65.67 +-      0.55 ( +-  0.83% )
  utime        2024.79 +-    439.74 ( +- 21.71% )         2026.12 +-    446.28 ( +- 22.02% )
  stime         312.13 +-     46.14 ( +- 14.78% )          311.53 +-     47.84 ( +- 15.35% )
  %cpu         3465.40 +-    883.75 ( +- 25.50% )         3473.80 +-    903.27 ( +- 26.00% )
  inv-csw    471639.00 +- 336424.00 ( +- 71.33% )       500981.00 +- 353471.00 ( +- 70.55% )
  vol-csw    190138.00 +-  44947.20 ( +- 23.63% )       177813.00 +-  44345.50 ( +- 24.93% )

Cc: Ingo Molnar <mingo@redhat.com>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Peter Ziljstra <peterz@infradead.org>
Originally-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
---
 kernel/sched/core.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 5df59a4548dc..2d33f3ff51a3 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -8968,7 +8968,9 @@ static void __sched_dynamic_update(int mode)
 {
 	switch (mode) {
 	case preempt_dynamic_none:
-		preempt_dynamic_mode = preempt_dynamic_undefined;
+		if (mode != preempt_dynamic_mode)
+			pr_info("%s: none\n", PREEMPT_MODE);
+		preempt_dynamic_mode = mode;
 		break;
 
 	case preempt_dynamic_voluntary:
-- 
2.31.1