Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
From:   Joel Fernandes <joel@joelfernandes.org>
Mime-Version: 1.0 (1.0)
Subject: Re: [RFC PATCH 00/11] Reviving the Proxy Execution Series
Date:   Sun, 16 Oct 2022 23:56:38 -0400
Message-Id: <802E39EA-7444-4B33-AB95-594A600FE404@joelfernandes.org>
References: <d5d72856-417a-0fe7-4b1d-3e27c64c1a85@linux.dev>
Cc:     Connor O'Brien <connoro@google.com>, linux-kernel@vger.kernel.org,
        kernel-team@android.com, John Stultz <jstultz@google.com>,
        Joel Fernandes <joelaf@google.com>,
        Qais Yousef <qais.yousef@arm.com>,
        Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Will Deacon <will@kernel.org>,
        Waiman Long <longman@redhat.com>,
        Boqun Feng <boqun.feng@gmail.com>,
        "Paul E . McKenney" <paulmck@kernel.org>
In-Reply-To: <d5d72856-417a-0fe7-4b1d-3e27c64c1a85@linux.dev>
To:     Chengming Zhou <zhouchengming@bytedance.com>
Precedence: bulk


> On Oct 16, 2022, at 11:25 PM, Chengming Zhou <zhouchengming@bytedance.com>=
 wrote:
>=20
> =EF=BB=BFHello,
>=20
>> On 2022/10/4 05:44, Connor O'Brien wrote:
>> Proxy execution is an approach to implementing priority inheritance
>> based on distinguishing between a task's scheduler context (information
>> required in order to make scheduling decisions about when the task gets
>> to run, such as its scheduler class and priority) and its execution
>> context (information required to actually run the task, such as CPU
>> affinity). With proxy execution enabled, a task p1 that blocks on a
>> mutex remains on the runqueue, but its "blocked" status and the mutex on
>> which it blocks are recorded. If p1 is selected to run while still
>> blocked, the lock owner p2 can run "on its behalf", inheriting p1's
>> scheduler context. Execution context is not inherited, meaning that
>> e.g. the CPUs where p2 can run are still determined by its own affinity
>> and not p1's.
>=20
> This is cool. We have a problem (others should have encountered it too) th=
at
> priority inversion happened when the rwsem writer is waiting for many read=
ers
> which held lock but are throttled by CFS bandwidth control. (In our use ca=
se,
> the rwsem is the mm_struct->mmap_sem)
>=20
> So I'm curious if this work can also solve this problem? If we don't deque=
ue
> the rwsem writer when it blocked on the rwsem, then CFS scheduler pick it t=
o
> run, we can use blocked chain to find the readers to run?

That seems a lot harder and unsupported by current patch set AFAICS (my expo=
sure to this work is about a week so take it with a grain of salt). You coul=
d have multiple readers so how would you choose which reader to proxy for (r=
ound robin?).  Also, you no longer have a chain but a tree of chains, with t=
he leaves being each reader - so you have to track that somehow, then keep m=
igrating the blocked tasks in the chain to each readers CPU. Possibly migrat=
ing a lot more than in the case of a single chain. Also it=E2=80=99s not cle=
ar if it will be beneficial as proxying for one reader does not mean you=E2=80=
=99re improving the situation if it is another reader that is in need of the=
 boost.

Thanks.

>=20
> Thanks!
>=20