MIME-Version: 1.0
References: <20230317160810.107988-1-vincent.guittot@linaro.org>
 <f0f7bce7-4d98-c5d0-5e75-55f27a9a0d69@huawei.com> <df2cccda-1550-b06b-aa74-e0f054e9fb9d@arm.com>
In-Reply-To: <df2cccda-1550-b06b-aa74-e0f054e9fb9d@arm.com>
From:   Vincent Guittot <vincent.guittot@linaro.org>
Date:   Mon, 20 Mar 2023 14:26:35 +0100
Message-ID: <CAKfTPtADr6ds6kmfE73J-33fT6mkziGZ4=x45rpwmGhv-u1f2A@mail.gmail.com>
Subject: Re: [PATCH v2] sched/fair: sanitize vruntime of entity being migrated
To:     Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc:     Zhang Qiao <zhangqiao22@huawei.com>, linux-kernel@vger.kernel.org,
        mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com,
        rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
        bristot@redhat.com, vschneid@redhat.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Mon, 20 Mar 2023 at 13:29, Dietmar Eggemann <dietmar.eggemann@arm.com> w=
rote:
>
> On 18/03/2023 08:45, Zhang Qiao wrote:
> >
> >
> > =E5=9C=A8 2023/3/18 0:08, Vincent Guittot =E5=86=99=E9=81=93:
> >> Commit 829c1651e9c4 ("sched/fair: sanitize vruntime of entity being pl=
aced")
> >> fixes an overflowing bug, but ignore a case that se->exec_start is res=
et
> >> after a migration.
> >>
> >> For fixing this case, we delay the reset of se->exec_start after
> >> placing the entity which se->exec_start to detect long sleeping task.
> >>
> >> In order to take into account a possible divergence between the clock_=
task
> >> of 2 rqs, we increase the threshold to around 104 days.
> >>
> >>
> >> Fixes: 829c1651e9c4 ("sched/fair: sanitize vruntime of entity being pl=
aced")
> >> Signed-off-by: Zhang Qiao <zhangqiao22@huawei.com>
> >> Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
> >> ---
> >>
> >> My last proposal was not yet correct as the exec_start was not always
> >> reset after migrating a task. I finally found this solution which keep=
s
> >> the long sleep detection to one place as well as the reset of se->exec=
_start.
> >>
> >
> > Tested-by: Zhang Qiao <zhangqiao22@huawei.com>
> >
> > I have retested it with this version, and the result is fine.
> >
> > -------
> >
> >  Performance counter stats for 'hackbench -g 44 -f 20 --process --pipe =
-l 60000 -s 100' (10 runs):
> >
> >              80.10 +- 1.22 seconds time elapsed  ( +-  1.53% )
>
> [...]
>
> >> @@ -8701,7 +8734,7 @@ static void attach_task(struct rq *rq, struct ta=
sk_struct *p)
> >>      lockdep_assert_rq_held(rq);
> >>
> >>      WARN_ON_ONCE(task_rq(p) !=3D rq);
> >> -    activate_task(rq, p, ENQUEUE_NOCLOCK);
> >> +    activate_task(rq, p, ENQUEUE_NOCLOCK | ENQUEUE_MIGRATED);
> >>      check_preempt_curr(rq, p, 0);
>
> Why not:
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index b9bc1ab67aaa..96dd3a62e683 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -7674,7 +7674,10 @@ static void migrate_task_rq_fair(struct task_struc=
t *p, int new_cpu)
>                 se->vruntime -=3D u64_u32_load(cfs_rq->min_vruntime);
>         }
>
> -       if (!task_on_rq_migrating(p)) {
> +       if (task_on_rq_migrating(p)) {
> +               /* We have migrated, no longer consider this task hot */
> +               se->exec_start =3D 0;

mainly to keep the clear of se->exec_start =3D 0 in one place to ease
the maintenance


> +       } else {
>                 remove_entity_load_avg(se);
>
>                 /*
> @@ -8726,7 +8729,7 @@ static void attach_task(struct rq *rq, struct task_=
struct *p)
>         lockdep_assert_rq_held(rq);
>
>         WARN_ON_ONCE(task_rq(p) !=3D rq);
> -       activate_task(rq, p, ENQUEUE_NOCLOCK | ENQUEUE_MIGRATED);
> +       activate_task(rq, p, ENQUEUE_NOCLOCK);
>         check_preempt_curr(rq, p, 0);
>  }
>
>
> entity_is_long_sleeper() will bail early for these rq-migrating tasks
> for which a long-sleep test would make little sense anyway.
>
> Plus move_queued_task() (e.g. from sched_exex()) would be covered as well=
.