Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
From:   Joel Fernandes <joel@joelfernandes.org>
Mime-Version: 1.0 (1.0)
Subject: Re: [RFC PATCH 07/11] sched: Add proxy execution
Date:   Fri, 28 Oct 2022 23:31:13 -0400
Message-Id: <34B2D8B9-A0C1-4280-944D-17224FB24339@joelfernandes.org>
References: <dab347c1-3724-8ac6-c051-9d2caea20101@arm.com>
Cc:     Connor O'Brien <connoro@google.com>, linux-kernel@vger.kernel.org,
        kernel-team@android.com, John Stultz <jstultz@google.com>,
        Joel Fernandes <joelaf@google.com>,
        Qais Yousef <qais.yousef@arm.com>,
        Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Will Deacon <will@kernel.org>,
        Waiman Long <longman@redhat.com>,
        Boqun Feng <boqun.feng@gmail.com>,
        "Paul E . McKenney" <paulmck@kernel.org>
In-Reply-To: <dab347c1-3724-8ac6-c051-9d2caea20101@arm.com>
To:     Dietmar Eggemann <dietmar.eggemann@arm.com>
Precedence: bulk

Hello Dietmar,

> On Oct 24, 2022, at 6:13 AM, Dietmar Eggemann <dietmar.eggemann@arm.com> w=
rote:
>=20
> =EF=BB=BFOn 03/10/2022 23:44, Connor O'Brien wrote:
>> From: Peter Zijlstra <peterz@infradead.org>
>=20
> [...]
>=20
>> + * Returns the task that is going to be used as execution context (the o=
ne
>> + * that is actually going to be put to run on cpu_of(rq)).
>> + */
>> +static struct task_struct *
>> +proxy(struct rq *rq, struct task_struct *next, struct rq_flags *rf)
>> +{
>=20
> [...]
>=20
>> +migrate_task:
>=20
> [...]
>=20
>> +    /*
>> +     * Since we're going to drop @rq, we have to put(@next) first,
>> +     * otherwise we have a reference that no longer belongs to us.  Use
>> +     * @fake_task to fill the void and make the next pick_next_task()
>           ^^^^^^^^^^
>=20
> There was a `static struct task_struct fake_task` in
> https://lkml.kernel.org/r/20181009092434.26221-6-juri.lelli@redhat.com
> but now IMHO we use `rq->idle` <-- (1)

Ok.

>> +     * invocation happy.
>> +     *
>> +     * XXX double, triple think about this.
>> +     * XXX put doesn't work with ON_RQ_MIGRATE
>> +     *
>> +     * CPU0                CPU1
>> +     *
>> +     *                B mutex_lock(X)
>> +     *
>> +     * A mutex_lock(X) <- B
>> +     * A __schedule()
>> +     * A pick->A
>> +     * A proxy->B
>> +     * A migrate A to CPU1
>> +     *                B mutex_unlock(X) -> A
>> +     *                B __schedule()
>> +     *                B pick->A
>> +     *                B switch_to (A)
>> +     *                A ... does stuff
>> +     * A ... is still running here
>> +     *
>> +     *        * BOOM *
>> +     */
>> +    put_prev_task(rq, next);
>> +    if (curr_in_chain) {
>> +        rq->proxy =3D rq->idle;
>> +        set_tsk_need_resched(rq->idle);
>> +        /*
>> +         * XXX [juril] don't we still need to migrate @next to
>> +         * @owner's CPU?
>> +         */
>> +        return rq->idle;
>> +    }
>=20
> --> (1)

Sorry but what has this got to do with your comment below?

>> +    rq->proxy =3D rq->idle;
>> +
>> +    for (; p; p =3D p->blocked_proxy) {
>> +        int wake_cpu =3D p->wake_cpu;
>> +
>> +        WARN_ON(p =3D=3D rq->curr);
>> +
>> +        deactivate_task(rq, p, 0);
>> +        set_task_cpu(p, that_cpu);
>> +        /*
>> +         * We can abuse blocked_entry to migrate the thing, because @p i=
s
>> +         * still on the rq.
>> +         */
>> +        list_add(&p->blocked_entry, &migrate_list);
>> +
>> +        /*
>> +         * Preserve p->wake_cpu, such that we can tell where it
>> +         * used to run later.
>> +         */
>> +        p->wake_cpu =3D wake_cpu;
>> +    }
>> +
>> +    rq_unpin_lock(rq, rf);
>> +    raw_spin_rq_unlock(rq);
>=20
> Don't we run into rq_pin_lock()'s:
>=20
> SCHED_WARN_ON(rq->balance_callback && rq->balance_callback !=3D
> &balance_push_callback)
>=20
> by releasing rq lock between queue_balance_callback(, push_rt/dl_tasks)
> and __balance_callbacks()?

Apologies, I=E2=80=99m a bit lost here. The code you are responding to inlin=
e does not call rq_pin_lock, it calls rq_unpin_lock.  So what scenario does t=
he warning trigger according to you?

Thanks,

- Joel=20


>=20
> [...]