Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20200312165429.990-1-vincent.guittot@linaro.org>
 <jhjr1xwjz96.mognet@arm.com> <CAKfTPtCQZMOz9HzdiWg5g9O+W=hC5E-fiG8YVHWCcODjFRfefQ@mail.gmail.com>
 <jhjpndgjxxk.mognet@arm.com> <jhj4kuspgse.mognet@arm.com> <CAKfTPtD67EKA46i12FHpJQT4gTzaH=ASAyb2dhv4=owPHBRSdQ@mail.gmail.com>
 <CAKfTPtBZgvTBYR+kYjj9dHq8_25mG19CZmYzY5s33ijSHdLGyQ@mail.gmail.com>
 <jhj36acp88q.mognet@arm.com> <CAKfTPtAMmYONX+qxp1Awj+XpqkWU3ootcyv7iar7e6z5nSczpw@mail.gmail.com>
 <jhj1rpwp4z1.mognet@arm.com>
In-Reply-To: <jhj1rpwp4z1.mognet@arm.com>
From:   Vincent Guittot <vincent.guittot@linaro.org>
Date:   Fri, 13 Mar 2020 18:12:28 +0100
Message-ID: <CAKfTPtCopLDoUsC+Mt6k99Hdn52pcKkrNYQsYNRW5LdgyMg4Nw@mail.gmail.com>
Subject: Re: [PATCH] sched/fair: improve spreading of utilization
To:     Valentin Schneider <valentin.schneider@arm.com>
Cc:     Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        linux-kernel <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, 13 Mar 2020 at 17:57, Valentin Schneider
<valentin.schneider@arm.com> wrote:
>
>
> On Fri, Mar 13 2020, Vincent Guittot wrote:
>
> >> Good point on the capacity reduction vs group_is_overloaded.
> >>
> >> That said, can't we also reach this with migrate_task? Say the local
> >
> > The test has only been added for migrate_util so migrate_task is not impacted
> >
> >> group is entirely idle, and the busiest group has a few non-idle CPUs
> >> but they all have at most 1 running task. AFAICT we would still go to
> >> calculate_imbalance(), and try to balance out the number of idle CPUs.
> >
> > such case is handled by migrate_task when we try to even the number of
> > tasks between groups
> >
> >>
> >> If the migration_type is migrate_util, that can't happen because of this
> >> change. Since we have this progressive balancing strategy (tasks -> util
> >> -> load), it's a bit odd to have this "gap" in the middle where we get
> >> one less possibility to trigger active balance, don't you think? That
> >> is, providing I didn't say nonsense again :)
> >
> > Right now, I can't think of a use case that could trigger such
> > situation because we use migrate_util when source is overloaded which
> > means that there is at least one waiting task and we favor this task
> > in priority
> >
>
> Right, what I was trying to say is that AIUI migration_type ==
> migrate_task with <= 1 running task per CPU in the busiest group can
> *currently* lead to a balance attempt, and thus a potential active
> balance.
>
> Consider a local group of 4 idle CPUs, and a busiest group of 3 busy 1
> idle CPUs, each busy having only 1 running task. That busiest group
> would be group_has_spare, so we would compute an imbalance of
> (4-1) / 2 == 1 task to move. We'll proceed with the load balance, but
> we'll only move things if we go through an active_balance.

yes because we want to even as much as possible the number of tasks per group

>
> My point is that if we prevent this for migrate_util, it would make
> sense to prevent it for migrate_task, but it's not straightforward since

hmm but we don't want to prevent this active balance for migrate_task
because of cases like the one you mentioned above.

we might consider to finally select a CPU with only 1 running task
with migrate_util if there is no other CPU with more than 1 task. But
this would complexify the code and I don't think it's possible because
migrate_util is used to pull some utilizations from an overloaded
group which must have a CPU with a waiting task to be overloaded.

> we have things like ASYM_PACKING.
>
> >>
> >> It's not a super big deal, but I think it's nice if we can maintain a
> >> consistent / gradual migration policy.
> >>
> >> >>
> >> >> > might be hard to notice in benchmarks.