Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210209082125.22176-1-song.bao.hua@hisilicon.com>
 <CAKfTPtCuJoS=Z2nOmeH6y_KTSjw9=hwrZ2N78VgC-ZQwz1d8FQ@mail.gmail.com> <jhjzh0dtqf9.mognet@arm.com>
In-Reply-To: <jhjzh0dtqf9.mognet@arm.com>
From:   Vincent Guittot <vincent.guittot@linaro.org>
Date:   Tue, 9 Feb 2021 14:10:49 +0100
Message-ID: <CAKfTPtC=_KHRszX=ALiVh84YD0tUoCbXEr9Ru1-CyY50-JMtVQ@mail.gmail.com>
Subject: Re: [PATCH v3] sched/topology: fix the issue groups don't span
 domain->span for NUMA diameter > 2
To:     Valentin Schneider <valentin.schneider@arm.com>
Cc:     Barry Song <song.bao.hua@hisilicon.com>,
        Mel Gorman <mgorman@suse.de>, Ingo Molnar <mingo@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Morten Rasmussen <morten.rasmussen@arm.com>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        linuxarm@openeuler.org, "xuwei (O)" <xuwei5@huawei.com>,
        "Liguozhu (Kenneth)" <liguozhu@hisilicon.com>,
        tiantao6@hisilicon.com, wanghuiqiang@huawei.com,
        "Zengtao (B)" <prime.zeng@hisilicon.com>,
        Jonathan Cameron <jonathan.cameron@huawei.com>,
        Guodong Xu <guodong.xu@linaro.org>,
        Meelis Roos <mroos@linux.ee>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Tue, 9 Feb 2021 at 12:46, Valentin Schneider
<valentin.schneider@arm.com> wrote:
>
> On 09/02/21 10:46, Vincent Guittot wrote:
> > On Tue, 9 Feb 2021 at 09:27, Barry Song <song.bao.hua@hisilicon.com> wrote:
> >> Real servers which suffer from this problem include Kunpeng920 and 8-node
> >> Sun Fire X4600-M2, at least.
> >>
> >> Here we move to use the *child* domain of the *child* domain of node2's
> >> domain2 as the new added sched_group. At the same, we re-use the lower
> >> level sgc directly.
> >
> > Have you evaluated the impact on the imbalance and next_update fields ?
> >
>
> sgc->next_update is safe since it's only touched by CPUs that have the
> group span as local group (which is never the case for CPUs where we do
> this "grandchildren" trick).

It would be good to explain this in the commit message

>
> I'm a bit less clear about sgc->imbalance. I think it can be set by remote
> CPUs, but it should only be cleared when running load_balance() by CPUs
> that have that group span as local group, as per:
>
>   int *group_imbalance = &sd_parent->groups->sgc->imbalance;

We are also safe because sd_parent remains the same as the beg of
load_balance and LB only tries other CPUs from the local group