Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 4 Nov 2022 11:47:03 +0000
From:   Qais Yousef <qyousef@layalina.io>
To:     Joel Fernandes <joel@joelfernandes.org>
Cc:     Vincent Guittot <vincent.guittot@linaro.org>, mingo@redhat.com,
        peterz@infradead.org, juri.lelli@redhat.com,
        dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com,
        mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com,
        linux-kernel@vger.kernel.org, parth@linux.ibm.com,
        qais.yousef@arm.com, chris.hyser@oracle.com,
        patrick.bellasi@matbug.net, David.Laight@aculab.com,
        pjt@google.com, pavel@ucw.cz, tj@kernel.org, qperret@google.com,
        tim.c.chen@linux.intel.com, joshdon@google.com, timj@gnu.org,
        kprateek.nayak@amd.com, yu.c.chen@intel.com,
        youssefesmat@chromium.org
Subject: Re: [PATCH v7 6/9] sched/fair: Add sched group latency support
Message-ID: <20221104114703.cdfzuz3w53o77va3@airbuntu>
References: <20221028093403.6673-1-vincent.guittot@linaro.org>
 <20221028093403.6673-7-vincent.guittot@linaro.org>
 <20221101192848.pjns3um7dnrwrk5p@airbuntu>
 <CAKfTPtA=vJQA_=6Un1CqwzMUaOxMeFGTKUZnj8qyqvqmSdYBAg@mail.gmail.com>
 <20221103142732.m5ibwkuymvhcdxkn@airbuntu>
 <CAKfTPtDu=c-psGnHkoWSPRWoh1Z0VBBfsN++g+krv4B1SJmFjg@mail.gmail.com>
 <CAEXW_YQd2y9=RHw3Sge7ghnhBz0AyR+B-U4zG10LXuSbg5bGSA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAEXW_YQd2y9=RHw3Sge7ghnhBz0AyR+B-U4zG10LXuSbg5bGSA@mail.gmail.com>
Precedence: bulk

On 11/04/22 10:14, Joel Fernandes wrote:

> I think the interface solves a different problem which is latency of
> task or cgroup wrt other group. Vincent, you are setting this for a
> “top app” group in android in your tests, and seeing improvement
> correct? AFAICS, this improvement comes because of lower latency
> during *same CPU* competition between different groups by juggling
> around the wakeup-preemption window -- which maybe is good for
> Android.
> 
> OTOH, the “prefer idle” flag in android that Qais is referring to,
> will need a completely different method as I cannot see how a nice
> value can communicate that (that can complement Vincent's changes
> here). And it will need to have a per-task interface as well. We have
> something in ChromeOS as well, which is a proc knob and also
> out-of-tree patch for that [1]. Without [1] we fail Android CTS
> testing on a recent ARM64 ChromeOS device.
> [1] https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/3884575
> The changelog in [1] also has a detailed description of the ChromeOS usecase.
> 
> Qais, any other reason you can see why Vincent's change will not be a
> good thing for Android? Since you 1 CGroup for the whole user-facing
> app (top app), you can just set that to a low "latency_nice" and get
> better wake-up latency for that.

There are two things to consider here:

	1. The interface and its extensibility.
	2. Current use case of improving wake up latency by manipulating
	   vruntime.

(2) looks promising approach, but it's very hard to know if it can be
considered a replacement. So (1) must be spot on to not block adding other
consumers in EAS or anywhere else in the scheduler for what matters.

> 
> (Side rant about latency and CFS -- IMHO a better long term solution
> for lower latency is to use RT but don't throttle -- rather demote. Or
> break CFS into multiple tiers, and apply demotion. This is in a way
> what Vincent is doing, as the task becomes more CPU bound'ish, he's
> taking away the latency boost. Vincent/Qais, somebody was working on
> the RT demotion vs throttling a while back, any idea on the latest on
> that?).

I can see an appetite in the future for userspace to provide more hints about
the characteristics of the tasks to improve performance and power.

I'm starting to lean towards having a framework to encapsulate such description
where latency_nice and potentially new notion of priority or something else can
be part of.

One aspect of prefer_idle in android for instance is that it disables the
packing behavior of EAS. I've seen Prateek showing similar examples in [1]
(FORK_SPREAD).

I don't have a clear idea in my head, but I'm slowly starting to lean towards
the need for a proper QoS/hint framework to describe the characteristic of the
task or a collection of tasks working together. My pony wish for now I guess
:-)

[1] https://lore.kernel.org/lkml/20220910105326.1797-1-kprateek.nayak@amd.com/


Thanks

--
Qais Yousef