Received-SPF: pass (google.com: domain of linux-kernel+bounces-112500-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Sender: Tejun Heo <htejun@gmail.com>
Date: Sat, 23 Mar 2024 12:12:31 -1000
From: Tejun Heo <tj@kernel.org>
To: Joel Fernandes <joel@joelfernandes.org>
Cc: torvalds@linux-foundation.org, mingo@redhat.com, peterz@infradead.org,
	juri.lelli@redhat.com, vincent.guittot@linaro.org,
	dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com,
	mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com,
	ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org,
	martin.lau@kernel.org, joshdon@google.com, brho@google.com,
	pjt@google.com, derkling@google.com, haoluo@google.com,
	dvernet@meta.com, dschatzberg@meta.com, dskarlat@cs.cmu.edu,
	riel@surriel.com, changwoo@igalia.com, himadrics@inria.fr,
	memxor@gmail.com, linux-kernel@vger.kernel.org, bpf@vger.kernel.org,
	kernel-team@meta.com, Andrea Righi <andrea.righi@canonical.com>
Subject: Re: [PATCH 12/36] sched_ext: Implement BPF extensible scheduler class
Message-ID: <Zf9Tz2wHT6KYtqEG@slm.duckdns.org>
References: <20231111024835.2164816-1-tj@kernel.org>
 <20231111024835.2164816-13-tj@kernel.org>
 <20240323023732.GA162856@joelbox2>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240323023732.GA162856@joelbox2>

Hello, Joel.

On Fri, Mar 22, 2024 at 10:37:32PM -0400, Joel Fernandes wrote:
..
> I was wondering about the comment above related to 'wakeup_preempt', could
> you clarify why it is not useful (NOOP) in the sched-ext class?
> 
> wakeup_preempt() may be called via:
> sched_ttwu_pending() ->
> 	ttwu_do_activate() ->
> 		wakeup_preempt()
> 			
> 
> at which point the enqueue of the task could have already happened via:
> 
> sched_ttwu_pending() ->
> 	ttwu_do_activate() ->
> 		activate_task() ->
> 			enqueue_task()
> 
> But the comment above says "task isn't tied to the CPU" ?

In sched_ext, a scheduling queue isn't tied to a particular CPU. For
example, it's trivial to share a single global scheduling queue across the
whole system or any subset of CPUs. To support this behavior, tasks can be
hot-migrated in the dispatch path just before it starts executing:

 https://github.com/sched-ext/sched_ext/blob/sched_ext/kernel/sched/ext.c#L1335  

So, the CPU picked by ops.select_cpu() in the enqueue path often doesn't
determine the CPU the task is going to execute on. If the picked CPU matches
the CPU the task is eventually going to run on, there's a small performance
advantage as the later hot migration can be avoided.

As the task isn't actually tied to the CPU being picked, it's a bit awkward
to ask "does this task preempt this CPU?" Instead, preemption is implemented
by calling scx_bpf_kick_cpu() w/ SCX_KICK_PREEMPT or using the
SCX_ENQ_PREEMPT flag from the enqueue path which allows preempting any CPU.

> Apologies in advance if I missed something as I just recently starting
> looking into the sched-ext patches.

No worries. Happy to answer any questions.

Thanks.

-- 
tejun