Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
MIME-Version: 1.0
References: <20220505122331.42696-1-wuyun.abel@bytedance.com>
 <CADjb_WTRYjSD8z0BCE=u322iJmQVuEq5cUMTZO6ij9V-=ViY-w@mail.gmail.com>
 <53fcde27-7dd2-5150-633b-4e2acc37bb6f@bytedance.com> <CADjb_WRmXh0tj7nZZR3QybhLxtoxZBy6OMKRNygtKOx-wUPxZA@mail.gmail.com>
In-Reply-To: <CADjb_WRmXh0tj7nZZR3QybhLxtoxZBy6OMKRNygtKOx-wUPxZA@mail.gmail.com>
From:   Chen Yu <yu.chen.surf@gmail.com>
Date:   Mon, 9 May 2022 23:31:21 +0800
Message-ID: <CADjb_WTfXDrKzAs09+Gf-xqrFuFBoi=9=Egy0_M5rU9a21+eRg@mail.gmail.com>
Subject: Re: [PATCH v3] sched/fair: filter out overloaded cpus in SIS
To:     Abel Wu <wuyun.abel@bytedance.com>
Cc:     Peter Zijlstra <peterz@infradead.org>,
        Mel Gorman <mgorman@suse.de>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Josh Don <joshdon@google.com>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, May 9, 2022 at 11:21 PM Chen Yu <yu.chen.surf@gmail.com> wrote:
>
> On Sun, May 8, 2022 at 1:50 AM Abel Wu <wuyun.abel@bytedance.com> wrote:
> >
> > Hi Chen,
> >
> > On 5/8/22 12:09 AM, Chen Yu Wrote:
> [cut]
> > >> @@ -81,8 +81,20 @@ struct sched_domain_shared {
> > >>          atomic_t        ref;
> > >>          atomic_t        nr_busy_cpus;
> > >>          int             has_idle_cores;
> > >> +
> > >> +       /*
> > >> +        * Tracking of the overloaded cpus can be heavy, so start
> > >> +        * a new cacheline to avoid false sharing.
> > >> +        */
> > > Although we put the following items into different cache line compared to
> > > above ones, is it possible that there is still cache false sharing if
> > > CPU1 is reading nr_overloaded_cpus while
> > > CPU2 is updating overloaded_cpus?
> >
> > I think it's not false sharing, it's just cache contention. But yes,
> > it is still possible if the two items mixed with others (by compiler)
> > in one cacheline, which seems out of our control..
> >
> My understanding is that, since nr_overloaded_cpus starts with a new
> cache line,  overloaded_cpus is very likely to be in the same cache line.
> Only If the write to nr_overloaded_cpus mask is not frequent(maybe tick based
> update is not frequent), the read of nr_overloaded_cpus can survive from cache
> false sharing, which is mainly read by SIS.  I have a stupid thought
> that if nr_overloaded_cpus
> mask and nr_overloaded_cpus could be put to 2 cache lines.
Not exactly, as overloaded_cpus and nr_overloaded_cpus are updated at the same
time, it is not a false sharing case.

-- 
Thanks,
Chenyu