Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Valentin Schneider <valentin.schneider@arm.com>
To:     Vincent Guittot <vincent.guittot@linaro.org>
Cc:     linux-kernel <linux-kernel@vger.kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@kernel.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>
Subject: Re: [PATCH] sched/fair: Update nohz.next_balance for newly NOHZ-idle CPUs
In-Reply-To: <CAKfTPtCdKK97_6m5C9Jp1yDLraJqm7PF-XL+0h64Bmab-4Px+g@mail.gmail.com>
References: <20210714113928.2795632-1-valentin.schneider@arm.com> <CAKfTPtCKJEgm=hXkKz0pHDE2d3Bz+6wp1kkNB+P+OpzWcR1+wg@mail.gmail.com> <87lf67n63p.mognet@arm.com> <CAKfTPtCdKK97_6m5C9Jp1yDLraJqm7PF-XL+0h64Bmab-4Px+g@mail.gmail.com>
Date:   Thu, 15 Jul 2021 15:51:19 +0100
Message-ID: <87fswfmxzs.mognet@arm.com>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

On 15/07/21 15:01, Vincent Guittot wrote:
> On Thu, 15 Jul 2021 at 13:56, Valentin Schneider <valentin.schneider@arm.com> wrote:
>> On 15/07/21 09:42, Vincent Guittot wrote:
>> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>> > index 44e44c235f1f..91c314f58982 100644
>> > --- a/kernel/sched/fair.c
>> > +++ b/kernel/sched/fair.c
>> > @@ -10657,6 +10657,9 @@ static void nohz_newidle_balance(struct rq *this_rq)
>> >         if (this_rq->avg_idle < sysctl_sched_migration_cost)
>> >                 return;
>> >
>> > +       if (time_before(this_rq->next_balance, READ_ONCE(nohz.next_balance))
>> > +               WRITE_ONCE(nohz.need_update, 1);
>> > +
>>
>> I think we have to do this unconditionally, as we can observe the old
>> nohz.next_balance while a NOHZ balance is ongoing (which will update
>> nohz.next_balance without taking into account this newly idle CPU).
>
> so maybe add this in nohz_balance_enter_idle() after the
> smp_mb__after_atomic(). Ilb will see the cpu in the idle_cpus_mask so
> even if nohz.next_balance is updated, it will take into account this
> newly idle cpu
>
> My goal was to use mechanism similar to what is used of nohz.has_blocked
>

OK, and then clearing it above the smp_mb() in _nohz_idle_balance() should
give us similar guarantees to nohz.has_blocked (i.e. if we don't observe
the cpumask write, then we'll observe the needs_update write).

Thanks for the suggestion, I'll go test this out.