Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4;
Date:   Sun, 15 Oct 2023 12:44:28 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Youssef Esmat <youssefesmat@chromium.org>
Cc:     LKML <linux-kernel@vger.kernel.org>, bsegall@google.com,
        mingo@kernel.org, vincent.guittot@linaro.org,
        juri.lelli@redhat.com, dietmar.eggemann@arm.com,
        rostedt@goodmis.org, mgorman@suse.de, bristot@redhat.com,
        corbet@lwn.net, qyousef@layalina.io, chris.hyser@oracle.com,
        patrick.bellasi@matbug.net, pjt@google.com, pavel@ucw.cz,
        qperret@google.com, tim.c.chen@linux.intel.com, joshdon@google.com,
        timj@gnu.org, kprateek.nayak@amd.com, yu.c.chen@intel.com,
        joel@joelfernandes.org, efault@gmx.de, tglx@linutronix.de,
        wuyun.abel@bytedance.com
Subject: Re: [PATCH] sched/eevdf: Toggle eligibility through sched_feat
Message-ID: <20231015104428.GA11840@noisy.programming.kicks-ass.net>
References: <20231013030213.2472697-1-youssefesmat@chromium.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20231013030213.2472697-1-youssefesmat@chromium.org>
Precedence: bulk

On Thu, Oct 12, 2023 at 10:02:13PM -0500, Youssef Esmat wrote:
> Interactive workloads see performance gains by disabling eligibility
> checks (EEVDF->EVDF). Disabling the checks reduces the number of
> context switches and delays less important work (higher deadlines/nice
> values) in favor of more important work (lower deadlines/nice values).
> 
> That said, that can add large latencies for some work loads and as the
> default is eligibility on, but allowing it to be turned off when
> beneficial.
> 
> Signed-off-by: Youssef Esmat <youssefesmat@chromium.org>
> Link: https://lore.kernel.org/lkml/CA+q576MS0-MV1Oy-eecvmYpvNT3tqxD8syzrpxQ-Zk310hvRbw@mail.gmail.com/
> ---
>  kernel/sched/fair.c     | 3 +++
>  kernel/sched/features.h | 1 +
>  2 files changed, 4 insertions(+)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index a751e552f253..16106da5a354 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -728,6 +728,9 @@ int entity_eligible(struct cfs_rq *cfs_rq, struct sched_entity *se)
>  	s64 avg = cfs_rq->avg_vruntime;
>  	long load = cfs_rq->avg_load;
>  
> +	if (!sched_feat(ENFORCE_ELIGIBILITY))
> +		return 1;
> +
>  	if (curr && curr->on_rq) {
>  		unsigned long weight = scale_load_down(curr->load.weight);
>  

Right.. could you pretty please try:

  git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git sched/eevdf

as of yesterday or so.

It defaults to (EEVDF relevant features):

SCHED_FEAT(PLACE_LAG, true)
SCHED_FEAT(PLACE_DEADLINE_INITIAL, true)
SCHED_FEAT(PREEMPT_SHORT, true)
SCHED_FEAT(PLACE_SLEEPER, false)
SCHED_FEAT(GENTLE_SLEEPER, true)
SCHED_FEAT(EVDF, false)
SCHED_FEAT(DELAY_DEQUEUE, true)
SCHED_FEAT(GENTLE_DELAY, true)

If that doesn't do well enough, could you please try, in order of
preference:

2) NO_GENTLE_DELAY
3) NO_DELAY_DEQUEUE, PLACE_SLEEPER
4) NO_DELAY_DEQUEUE, PLACE_SLEEPER, NO_GENTLE_SLEEPER

I really don't like the EVDF option, and I think you'll end up
regretting using it sooner rather than later, just to make this one
benchmark you have happy.

I'm hoping the default is enough, but otherwise any of the above should
be a *much* better scheduler.

Also, bonus points if you can create us a stand alone benchmark that
captures your metric (al-la facebook's schbench) without the whole
chrome nonsense, that'd be epic.