Received-SPF: pass (google.com: domain of linux-kernel+bounces-136872-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Precedence: bulk
MIME-Version: 1.0
References: <20240403005930.1587032-1-qyousef@layalina.io> <CAKfTPtDB8D6bouxJN9q8gXqG+BQYcsrJYEodDWtOB2kQwPH53A@mail.gmail.com>
 <20240404220500.dmfl2krll37znbi5@airbuntu> <CAKfTPtDP7if0gozSrnj+E_hH5xR-vpGAM2TwN4qWXcg5BtrEtw@mail.gmail.com>
 <20240405171653.boxbylrdak5fopjv@airbuntu> <20240407122700.ns7gknqwqkpjjyd4@airbuntu>
 <CAKfTPtBZao-Ry=sdAV=rtTwbxbEJmwb-_gNceSjV6u-6EXTY-w@mail.gmail.com>
 <CANDhNCq5HZvecSe9_9f7j5koY2VNdyjM_b3csL6=U1A_8J2ksw@mail.gmail.com> <20240409061909.tb3vxc27h2eawiwg@airbuntu>
In-Reply-To: <20240409061909.tb3vxc27h2eawiwg@airbuntu>
From: Vincent Guittot <vincent.guittot@linaro.org>
Date: Tue, 9 Apr 2024 14:35:27 +0200
Message-ID: <CAKfTPtC4hdbBhn+-hkK9i4vkjO5fBGfsxjESkBrvyOwN6oHCdA@mail.gmail.com>
Subject: Re: [PATCH] sched/pi: Reweight fair_policy() tasks when inheriting prio
To: Qais Yousef <qyousef@layalina.io>
Cc: John Stultz <jstultz@google.com>, Ingo Molnar <mingo@kernel.org>, 
	Peter Zijlstra <peterz@infradead.org>, Juri Lelli <juri.lelli@redhat.com>, 
	Steven Rostedt <rostedt@goodmis.org>, Daniel Bristot de Oliveira <bristot@redhat.com>, 
	Thomas Gleixner <tglx@linutronix.de>, "Paul E. McKenney" <paulmck@kernel.org>, 
	Joel Fernandes <joel@joelfernandes.org>, Dietmar Eggemann <dietmar.eggemann@arm.com>, 
	linux-kernel@vger.kernel.org, Yabin Cui <yabinc@google.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 9 Apr 2024 at 08:19, Qais Yousef <qyousef@layalina.io> wrote:
>
> On 04/08/24 12:51, John Stultz wrote:
> > On Mon, Apr 8, 2024 at 12:17=E2=80=AFAM Vincent Guittot
> > <vincent.guittot@linaro.org> wrote:
> > >
> > > On Sun, 7 Apr 2024 at 14:27, Qais Yousef <qyousef@layalina.io> wrote:
> > > >
> > > > On 04/05/24 18:16, Qais Yousef wrote:
> > > >
> > > > > >
> > > > > > All that to say that I think the weight is not applied on purpo=
se.
> > > > > > This might work for your particular case but there are more cha=
nges to
> > > > > > be done if you want to apply prio inheritance between cfs tasks=
.
> > > > > >
> > > > > > As an example, what about the impact of cgroup on the actual we=
ight
> > > > > > and the inherited priority of a task ? If the owner and the wai=
ter
> > > > > > don't belong to the same cgroup their own prio is meaningless..=
 task
> > > > > > nice -20 in a group with a weight equal to nice 19 vs a task ni=
ce 19
> > > > > > in a group with a weight equals to nice -20
> > > > >
> > > > > That is on my mind actually. But I thought it's a separate proble=
m. That has to
> > > > > do with how we calculate the effective priority of the pi_task. A=
nd probably
> > > > > the sorting order to if we agree we need to revert the above. If =
that is done
> > > >
> > > > Thinking more about it the revert is not the right thing to do. We =
want fair
> > > > tasks to stay ordered in FIFO for better fairness and avoid potenti=
al
> > > > starvation issues. It's just the logic for searching the top_waiter=
 need to be
> > > > different. If the top_waiter is fair, then we need to traverse the =
tree to find
> > > > the highest nice value. We probably can keep track of this while ad=
ding items
> > > > to the tree to avoid the search.
> > > >
> > > > For cgroup; is it reasonable (loosely speaking) to keep track of pi=
_cfs_rq and
> > > > detach_attach_task_cfs_rq() before the reweight? This seems the mos=
t
> > > > straightforward solution and will contain the complexity to keeping=
 track of
> > > > cfs_rq. But it'll have similar issue to proxy execution where a tas=
k that
> > > > doesn't belong to the cgroup will consume its share..
> > >
> > > That's a good point, Would proxy execution be the simplest way to fix=
 all this ?
>
> Is it? Over 4.5 years ago Unity reported to me about performance inversio=
n
> problem and that's when proxy execution work was revived as simplest way =
to fix
> all of this. But still no end in sight from what I see. I was and still t=
hink
> an interim solution in rt_mutex could help a lot of use cases already wit=
hout
> being too complex. Not as elegant and comprehensive like proxy execution,=
 but
> given the impact on both userspace and out of tree kernel hacks are growi=
ng
> waiting for this to be ready, the cost of waiting is high IMHO.
>
> FWIW, I already heard several feedbacks that PTHREAD_PRIO_INHERIT does no=
thing.
> I think this reweight issue is more serious problem and likely why I hear=
d this
> feedback. I could be underestimating the complexity of the fix though. So=
 I'll

Without cgroup, the solution could be straightforward but android uses
extensively cgroup AFAICT and update_cfs_group() makes impossible to
track the top cfs waiter and its "prio"

> trust your judgement and ditch further effort if you think it's more effo=
rt
> than helping proxy execution patchset - for the list at least. I'm likely=
 still
> to pursue something out of tree to get into as many android LTS kernels. =
And
> will be happy to share this work if there's desire to try to pick somethi=
ng up
> for mainline to make the problem less severe at least.
>
> Note that binder has already performance, latency (out of tree) and prior=
ity
> inheritance and without those performance is impacted in many corner case=
s and
> considered indispensable part.
>
> >
> > So, at the moment, in part. It ought to resolve the issue for
> > in-kernel mutexes (blocked tasks stay on rq, if blocked tasks are
> > selected to run we will instead run the runnable lock owner - thus it
> > works across scheduling classes), but it isn't tied into userland
> > futexes the way rt_mutexes are at this point.
> >
> > Review and feedback on the series would be greatly appreciated!
> > (Nudge! Nudge! :)
>
> I am guilty of this, sorry.

me too