Received-SPF: pass (google.com: domain of linux-kernel+bounces-89882-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <198072f4-6d5f-435b-aab3-1351e3265f48@joelfernandes.org>
Date: Sun, 3 Mar 2024 14:32:23 -0500
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 29/30] Documentation: tracing: add TIF_NEED_RESCHED_LAZY
Content-Language: en-US
To: Ankur Arora <ankur.a.arora@oracle.com>
Cc: Steven Rostedt <rostedt@goodmis.org>, linux-kernel@vger.kernel.org,
 tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org,
 paulmck@kernel.org, akpm@linux-foundation.org, luto@kernel.org,
 bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com,
 juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org,
 mgorman@suse.de, jpoimboe@kernel.org, mark.rutland@arm.com, jgross@suse.com,
 andrew.cooper3@citrix.com, bristot@kernel.org,
 mathieu.desnoyers@efficios.com, geert@linux-m68k.org,
 glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com,
 mattst88@gmail.com, krypton@ulrich-teichert.org, David.Laight@aculab.com,
 richard@nod.at, mjguzik@gmail.com, jon.grimm@amd.com, bharata@amd.com,
 raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com,
 Masami Hiramatsu <mhiramat@kernel.org>, Jonathan Corbet <corbet@lwn.net>
References: <20240213055554.1802415-1-ankur.a.arora@oracle.com>
 <20240213055554.1802415-30-ankur.a.arora@oracle.com>
 <20240221164334.6f8c69e8@gandalf.local.home>
 <65e265e3.050a0220.7044b.19ad@mx.google.com> <87v8652uud.fsf@oracle.com>
From: Joel Fernandes <joel@joelfernandes.org>
In-Reply-To: <87v8652uud.fsf@oracle.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit


On 3/1/2024 10:09 PM, Ankur Arora wrote:
> 
> Joel Fernandes <joel@joelfernandes.org> writes:
> 
>> On Wed, Feb 21, 2024 at 04:43:34PM -0500, Steven Rostedt wrote:
>>> On Mon, 12 Feb 2024 21:55:53 -0800
>>> Ankur Arora <ankur.a.arora@oracle.com> wrote:
>>>
>>>> Document various combinations of resched flags.
>>>>
>>>> Cc: Steven Rostedt <rostedt@goodmis.org>
>>>> Cc: Masami Hiramatsu <mhiramat@kernel.org>
>>>> Cc: Jonathan Corbet <corbet@lwn.net>
>>>> Originally-by: Thomas Gleixner <tglx@linutronix.de>
>>>> Link: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/
>>>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>>>> ---
>>>>  Documentation/trace/ftrace.rst | 6 +++++-
>>>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst
>>>> index 7e7b8ec17934..7f20c0bae009 100644
>>>> --- a/Documentation/trace/ftrace.rst
>>>> +++ b/Documentation/trace/ftrace.rst
>>>> @@ -1036,8 +1036,12 @@ explains which is which.
>>>>  		be printed here.
>>>>
>>>>    need-resched:
>>>> -	- 'N' both TIF_NEED_RESCHED and PREEMPT_NEED_RESCHED is set,
>>>> +	- 'B' all three, TIF_NEED_RESCHED, TIF_NEED_RESCHED_LAZY and PREEMPT_NEED_RESCHED are set,
>>>> +	- 'N' both TIF_NEED_RESCHED and PREEMPT_NEED_RESCHED are set,
>>>> +	- 'L' both TIF_NEED_RESCHED_LAZY and PREEMPT_NEED_RESCHED are set,
>>>> +	- 'b' both TIF_NEED_RESCHED and TIF_NEED_RESCHED_LAZY are set,
>>>>  	- 'n' only TIF_NEED_RESCHED is set,
>>>> +	- 'l' only TIF_NEED_RESCHED_LAZY is set,
>>>>  	- 'p' only PREEMPT_NEED_RESCHED is set,
>>
>> One thing I was curious about in current code, under which situation will
>> "only PREEMPT_NEED_RESCHED is set" case happen? All the code paths I see set
>> the PREEMPT_NEED_RESCHED when then TIF has already been set. That kind of
>> makes sense, why enter the scheduler on preempt_enable() unless there was a
>> reason to, and TIF should have recorded that reason.
> 
> True. And, the only place where we clear PREEMPT_NEED_RESCHED is in
> __schedule() after clearing the TIF_NEED_RESCHED* flags.

Thanks for taking a look.

>> So in other words, if 'p' above cannot happen, then it could be removed as a
>> potential need-resched stat. If it can happen, I'd love to learn in which
>> case?
> 
> Yeah, AFAICT the only case we might see it alone is in case of a bug.

Thanks for confirming as well. Steve, are you Ok with a patch to remove 'p'? Or
you probably want to leave it alone in case something changes in the future? I'd
rather it be removed.

>> Also considering all users of set_tsk_need_resched() also call the
>> set_preempt_need_resched() shortly after, should we add a helper to squash
>> the 2 calls and simplify callers?
>>
> Just to explicitly lay out the reason for these being separate interfaces:
> set_tsk_need_resched() can be set from a remote CPU, while
> set_preempt_need_resched() (or its folding version) is only meant to be
> used on the local CPU.

Yes, agreed.

>> There are some "special cases" where only TIF flag need be set (such as setting
>> rescheduling from an interrupt or when rescheduling a remote CPU). For those,
>> they can directly call the set_tsk_need_resched() like they do now.
> 
> The remote case, as you say is always handled in the scheduler. So, maybe
> it is worth having a set_need_resched_local_cpu() which takes care of
> both of these things but more importantly makes clear the limits of this.
> 
> That said, these interfaces have very few users (just RCU, and the s390
> page fault handler) and both of these are fairly sophisticated users, so
> not sure yet another interface in this area is worth adding.
> 

Sounds good, though there are quite a few RCU users and I do remember in the
past, that during a code review I pointed out that both needed to be set. Not
just one. :) I wonder what Paul McKenney thinks about this as well ;-)

thanks,

 - Joel