Received-SPF: pass (google.com: domain of linux-kernel+bounces-160430-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <81c1590d-076b-4cc1-9fe0-f6ba9304c281@google.com>
Date: Fri, 26 Apr 2024 12:57:26 -0400
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 12/36] sched_ext: Implement BPF extensible scheduler class
To: Joel Fernandes <joel@joelfernandes.org>
Cc: Tejun Heo <tj@kernel.org>, torvalds@linux-foundation.org,
 mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com,
 vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org,
 bsegall@google.com, mgorman@suse.de, bristot@redhat.com,
 vschneid@redhat.com, ast@kernel.org, daniel@iogearbox.net,
 andrii@kernel.org, martin.lau@kernel.org, joshdon@google.com,
 pjt@google.com, derkling@google.com, haoluo@google.com, dvernet@meta.com,
 dschatzberg@meta.com, dskarlat@cs.cmu.edu, riel@surriel.com,
 changwoo@igalia.com, himadrics@inria.fr, memxor@gmail.com,
 linux-kernel@vger.kernel.org, bpf@vger.kernel.org, kernel-team@meta.com,
 Andrea Righi <andrea.righi@canonical.com>
References: <20231111024835.2164816-1-tj@kernel.org>
 <20231111024835.2164816-13-tj@kernel.org> <20240323023732.GA162856@joelbox2>
 <Zf9Tz2wHT6KYtqEG@slm.duckdns.org>
 <CAEXW_YR02g=DetfwM98ZoveWEbGbGGfb1KAikcBeC=Pkvqf4OA@mail.gmail.com>
From: Barret Rhoden <brho@google.com>
Content-Language: en-US
In-Reply-To: <CAEXW_YR02g=DetfwM98ZoveWEbGbGGfb1KAikcBeC=Pkvqf4OA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 4/25/24 17:28, Joel Fernandes wrote:
> But if I understand, sched_ext is below CFS at the moment, so that
> should not be an issue.
> 
> [1] By the way, now that I brought up the higher priority class thing,
> I might as well discuss it here :-D :
> 
> One of my use cases is about scheduling high priority latency sensitive threads:
> I think if sched_ext could have 2 classes, one lower than CFS and one
> above CFS, that would be beneficial to those who want a gradual
> transition to use scx, instead of switching all tasks to scx at once.

The way we initially went with Ghost (which is the Google project 
similar to sched_ext) was to run Ghost below CFS.  That was a "safety 
first" approach, where we didn't have a lot of faith in the schedulers 
we were writing and wanted to convince people (including ourselves) that 
we wouldn't completely hose a machine.

For the same reason you pointed out, we eventually wanted to run our 
schedulers above CFS.  Currently, Ghost has the option to run above or 
below CFS, and we're pretty close to being able to run all of our 
schedulers above CFS.  Once we do that, I imagine we'll drop the "above 
or below" aspect, since it's a bit more complicated.

It's actually even more complicated than that - ghost also has a 
separate "ghost agent" sched class above CFS, even if ghost itself is 
below CFS.  This lets us run a single userspace "agent task" on a cpu to 
make a scheduling decision.

Anyway, when we port Ghost to run on sched_ext instead of our custom 
ghost sched class(es), we'll be running sched_ext above CFS, and I doubt 
we'll want to run below CFS at all.  Though whether that's a local patch 
we carry, or the default upstream probably depends on what everyone else 
wants too.

Thanks,
Barret