Received-SPF: pass (google.com: domain of linux-kernel+bounces-86189-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
References: <20240213055554.1802415-1-ankur.a.arora@oracle.com>
 <20240213055554.1802415-24-ankur.a.arora@oracle.com>
 <Zd85iRyY6-zlo2wl@localhost.localdomain>
User-agent: mu4e 1.4.10; emacs 27.2
From: Ankur Arora <ankur.a.arora@oracle.com>
To: Juri Lelli <juri.lelli@redhat.com>
Cc: Ankur Arora <ankur.a.arora@oracle.com>, linux-kernel@vger.kernel.org,
        tglx@linutronix.de, peterz@infradead.org,
        torvalds@linux-foundation.org, paulmck@kernel.org,
        akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de,
        dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com,
        vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de,
        jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com,
        andrew.cooper3@citrix.com, bristot@kernel.org,
        mathieu.desnoyers@efficios.com, geert@linux-m68k.org,
        glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com,
        mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org,
        David.Laight@aculab.com, richard@nod.at, mjguzik@gmail.com,
        jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com,
        boris.ostrovsky@oracle.com, konrad.wilk@oracle.com
Subject: Re: [PATCH 23/30] sched/fair: handle tick expiry under lazy preemption
In-reply-to: <Zd85iRyY6-zlo2wl@localhost.localdomain>
Date: Wed, 28 Feb 2024 22:43:34 -0800
Message-ID: <871q8v7otl.fsf@oracle.com>
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?A6vQXNjLdR3ElM3KYAKjhl92HzHbHcnUou01FZ0tgzVG6mroe8ZHZzfcvW1v?=
 =?us-ascii?Q?qkS5roxYiGSE2JcJIHpBUKeK8UG4fRC1Yx1P9IUfJXj1nyVgOyRyhjql1vmk?=
 =?us-ascii?Q?kY+v1PTqU+SV6cwoUb/JbgXtTVx8z214lt7T8BwMf9vT5z7jOB6CyczubhAz?=
 =?us-ascii?Q?kk0t4db3xH6J/gD2cJl5hE5yxr2Hr2ffJGwALBbRfeiIYYRHaD169ShpYO/p?=
 =?us-ascii?Q?e9atMvPA/d96EjWCJj8j/WYTFP/mp5rBzRg4VzdK6qe/FKqiwxKydREA09wt?=
 =?us-ascii?Q?aQPjzp8LnEnucRms+jR7z6UKjxY9Z4cVHWfanU8gDidLNxpmmoKTIQkPzmJs?=
 =?us-ascii?Q?W1RHe+EBIAePfaow8rnvGx557bblhSIGOvSJbX0BQhEcoEzpCOsBd/Ra93zy?=
 =?us-ascii?Q?FOPt4pFNASLllaV0LShmb3m/1LBvZCFPNAhGkEG7d77nOKmW73aosgRAflaC?=
 =?us-ascii?Q?RqUJ7g4zEwt5oCN+PkMd90oe+VXDbFjhW7xwFz6GIaN3ce0JY2a5Jpi3m2zz?=
 =?us-ascii?Q?xX/RGquBRwP30wiBg+9JEt1cVUWiIFuadgFU2UL+7mDFEi045vaGd1P/Q04u?=
 =?us-ascii?Q?AQUjhgyQmcJukTOi1xWVl14NgiTY2H2DCKASB3K5Qa0gATmnBwqFBrGc5Ixl?=
 =?us-ascii?Q?AMshAPN7FuU8MIsADRL49DPKdZRrS3SU7Oi1I/dFisOV4O8oQBBuMAkWzY+y?=
 =?us-ascii?Q?PHtaMtEFiw0MmPMYfXrfcvW1vqc+XtPMr9KQxLQsQ8MQ6r1/ooAC/Uq1/9C/?=
 =?us-ascii?Q?353otDOCZ9WTifYvDa2pzo09mvZkghActueBxuEIcQIyri+78ap3naPCBU6b?=
 =?us-ascii?Q?LJKAJuIKr0XhzqqpGddJjNmVOK07l5KnE8/GXcFyDWnR6QY9Mp00mZFxPrpO?=
 =?us-ascii?Q?KUrJ23jsTVnuiXUqD6nb0pYxy4oUjKChHn1zzCbDG7iCfZWglNz+HedFLL10?=
 =?us-ascii?Q?e5tv9bt2ueoQH7FDQqoG0wi0raA22X7vxOoWwwidGvlI6MYgTn0ehPOjhjb2?=
 =?us-ascii?Q?0xiKt63OisA1UX07NnCrcVSrhjDFn7eZKhQ+1MUWOcWF07Hoak0XlBcDnGuP?=
 =?us-ascii?Q?hIEPufnXkqvqlTXgD8cXWXAIUa4kJ3UdUVK8YaApMo/Tdrusk9L67pnZY+gs?=
 =?us-ascii?Q?1c4XfZu6jlmGbuWS641k59+MJQoTF8zbrBxbogP4r7gaScgemcprfYAi4+6y?=
 =?us-ascii?Q?2t6Hs3MPc8006sXlABs1kCOmhjvK7ppK8j7AxK4vaSN1qDuF9TwFtlVg80Ih?=
 =?us-ascii?Q?wuU6n5/N6t/38/F9KwFAT8WrYwAmSmByLSYGAeqhLDoDMKrAoC+Fgm8EEM71?=
 =?us-ascii?Q?FbPWmgYPYFYofnNPmnrcLx4L7Q3ZiBy8VXjAb9Qa5xQz5bKkrcK+EJSakAe6?=
 =?us-ascii?Q?HQpjfNkTO2A72lnLFea5XSBo4u7OBr71zUtjR6MuEdXLmsXOPuJ5OyOY9ljd?=
 =?us-ascii?Q?wvLm6aVP5nN/t4qhSIeXiRQ2F9Euw1ldi3Q20JI5omB6kP+94ap1vZv2lEIo?=
 =?us-ascii?Q?UGFNAMcWOOuNyOkx5kHmso+9ZdHhkRDwxAgbz4qpTZyrWCfjfUbcT96pEwoJ?=
 =?us-ascii?Q?58yTaCutnugD4wmekibZDF1u9bliMX44KN7YbrzAkELMYuka5bnR6/F0oXzS?=
 =?us-ascii?Q?4Q=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	dZ1sgqPWoWmGzvRkSm+ZjFaTqcBBeZjbOImUOKQP9dmzkkXDj8HQGl/gQQ9pNbxuHpTFtAXXbk071u0a7XTwGv/WfgshgEFDvPo1IzdlJ8vrn69mJ/9+ttwLwzNtQG227AE9yetvDH8Jf4Ss6z6UrEeg0jtJPDMGFdjFHRXDpE7Na+hoizYlpsdF+Izj558HSvPBLxWpA8lyCoHC5aBlJa9ZZdGDAsWggMekgOnOABqYRHW0h818AbXr2ZyOQHvGxNrxI3Ot3YhFVxVWfBuKREdUj9A40qGMCFEeRPgZ6anAQAu6IqDoLfQ790KwaEnjLKrYumU/0596LQD6ARPX1uUULmAtExOxlemmNgJqQ5G3+92nXYlGR13KbPlEyfADL8UDPB+y6rjeZb1ERepxMYV06VjEivC1V529w9uUM3qfZ9OqRYZyhO2bVJZ345oFG3DbVozOH+DccDsXeI4EEDV7AwTkYbn8+JNXnkBbglA7RJU6mWfa634JIoHB/s5tDPRJw/gyLBzyiV0ffPkRmJrte6e4vp+ELcG2behGs/UrfCTmPCJdu9QBjF7kElBLIyPDPikNZ870Z8U9ZpNyaPYMCJ0CVgyiykK8Z6CkIXk=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ccea6f37-1319-40a4-0e47-08dc38f1b246
X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Feb 2024 06:43:11.2697
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ezCBStJ0pz3Ac2YnAUgB71Eah1e4MtNhMv41dhPdeKGouUSrQ5bdTIK7zC863+a+GW8CvrDQ76fv1VGwo322oVsUy8HKGrgSMjvkjZnvoR4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN4PR10MB5542
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-28_08,2024-02-27_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 malwarescore=0 adultscore=0
 mlxscore=0 mlxlogscore=855 phishscore=0 suspectscore=0 spamscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000
 definitions=main-2402290051
X-Proofpoint-ORIG-GUID: 9ZIp6Q5_p94He3dyngU9oUgCVLT89bqh
X-Proofpoint-GUID: 9ZIp6Q5_p94He3dyngU9oUgCVLT89bqh


Juri Lelli <juri.lelli@redhat.com> writes:

> Hi Ankur,
>
> On 12/02/24 21:55, Ankur Arora wrote:
>> The default policy for lazy scheduling is to schedule in exit-to-user,
>> assuming that would happen within the remaining time quanta of the
>> task.
>>
>> However, that runs into the 'hog' problem -- the target task might
>> be running in the kernel and might not relinquish CPU on its own.
>>
>> Handle that by upgrading the ignored tif_resched(NR_lazy) bit to
>> tif_resched(NR_now) at the next tick.
>>
>> Cc: Ingo Molnar <mingo@redhat.com>
>> Cc: Peter Zijlstra <peterz@infradead.org>
>> Cc: Juri Lelli <juri.lelli@redhat.com>
>> Cc: Vincent Guittot <vincent.guittot@linaro.org>
>> Originally-by: Thomas Gleixner <tglx@linutronix.de>
>> Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/
>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>>
>> ---
>> Note:
>>   Instead of special casing the tick, it might be simpler to always
>>   do the upgrade on the second resched_curr().
>>
>>   The theoretical problem with doing that is that the current
>>   approach deterministically provides a well-defined extra unit of
>>   time. Going with a second resched_curr() might mean that the
>>   amount of extra time the task gets depends on the vagaries of
>>   the incoming resched_curr() (which is fine if it's mostly from
>>   the tick; not fine if we could get it due to other reasons.)
>>
>>   Practically, both performed equally well in my tests.
>>
>>   Thoughts?
>
> I'm still digesting the series, so I could simply be confused, but I
> have the impression that the extra unit of time might be a problem for
> deadline (and maybe rt as well?).
>
> For deadline we call resched_curr_tick() from the throttle part of
> update_curr_dl_se() if the dl_se happens to not be the leftmost anymore,
> so in this case I believe we really want to reschedule straight away and
> not wait for the second time around (otherwise we might be breaking the
> new leftmost tasks guarantees)?

Yes, agreed, this looks like it breaks the deadline invariant for both
preempt=none and preempt=voluntary.

For RT, update_curr_rt() seems to have a similar problem if the task
doesn't have RUNTIME_INF.

Relatedly, do you think there's a similar problem when switching to
a task with a higher scheduling class?
(Related to code is in patch 25, 26.)

For preempt=voluntary, wakeup_preempt() will do the right thing, but
for preempt=none, we only reschedule lazily so the target might
continue to run until the end of the tick.

Thanks for the review, btw.

--
ankur