Received-SPF: pass (google.com: domain of linux-kernel+bounces-50915-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Precedence: bulk
MIME-Version: 1.0
References: <20240203003414.1067730-1-yosryahmed@google.com> <CALvZod6pKLhLm6v7da1sm_axvSR07f_buOc9czRfLb5mpzOanw@mail.gmail.com>
In-Reply-To: <CALvZod6pKLhLm6v7da1sm_axvSR07f_buOc9czRfLb5mpzOanw@mail.gmail.com>
From: Yosry Ahmed <yosryahmed@google.com>
Date: Fri, 2 Feb 2024 20:22:39 -0800
Message-ID: <CAJD7tkaLs8JTdLEm1UcpO9amYHwDie=TW12f+7q1y_ipxC15cQ@mail.gmail.com>
Subject: Re: [PATCH mm-hotfixes-unstable] mm: memcg: fix struct
 memcg_vmstats_percpu size and alignment
To: Shakeel Butt <shakeelb@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>, Johannes Weiner <hannes@cmpxchg.org>, 
	Michal Hocko <mhocko@kernel.org>, Roman Gushchin <roman.gushchin@linux.dev>, 
	Muchun Song <muchun.song@linux.dev>, cgroups@vger.kernel.org, linux-mm@kvack.org, 
	linux-kernel@vger.kernel.org, Greg Thelen <gthelen@google.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Feb 2, 2024 at 8:13=E2=80=AFPM Shakeel Butt <shakeelb@google.com> w=
rote:
>
> On Fri, Feb 2, 2024 at 4:34=E2=80=AFPM Yosry Ahmed <yosryahmed@google.com=
> wrote:
> >
> > Commit da10d7e140196 ("mm: memcg: optimize parent iteration in
> > memcg_rstat_updated()") added two additional pointers to the end of
> > struct memcg_vmstats_percpu with CACHELINE_PADDING to put them in a
> > separate cacheline. This caused the struct size to increase from 1200 t=
o
> > 1280 on my config (80 extra bytes instead of 16).
> >
> > Upon revisiting, the relevant struct members do not need to be on a
> > separate cacheline, they just need to fit in a single one. This is a
> > percpu struct, so there shouldn't be any contention on that cacheline
> > anyway. Move the members to the beginning of the struct and cachealign
> > the first member. Add a comment about the members that need to fit
> > together in a cacheline.
> >
> > The struct size is now 1216 on my config with this change.
> >
> > Fixes: da10d7e140196 ("mm: memcg: optimize parent iteration in memcg_rs=
tat_updated()")
> > Reported-by: Greg Thelen <gthelen@google.com>
> > Signed-off-by: Yosry Ahmed <yosryahmed@google.com>
> > ---
> >  mm/memcontrol.c | 19 +++++++++----------
> >  1 file changed, 9 insertions(+), 10 deletions(-)
> >
> > diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> > index d9ca0fdbe4ab0..09f09f37e397e 100644
> > --- a/mm/memcontrol.c
> > +++ b/mm/memcontrol.c
> > @@ -621,6 +621,15 @@ static inline int memcg_events_index(enum vm_event=
_item idx)
> >  }
> >
> >  struct memcg_vmstats_percpu {
> > +       /* Stats updates since the last flush */
> > +       unsigned int                    stats_updates ____cacheline_ali=
gned;
>
> Why do you need ____cacheline_aligned here? From what I understand for
> the previous patch you want stats_updates, parent and vmstats on the
> same cacheline, right?

Yes. I am trying to ensure that stats_updates sits at the beginning of
a cacheline to ensure they all fit in one cacheline. Is this
implicitly guaranteed somehow?

>
> I would say just remove the CACHELINE_PADDING() from the previous
> patch and we are good.

IIUC, without CACHELINE_PADDING(), they may end up on different cache
lines, depending on the size of the arrays before them in the struct
(which depends on several configs). Am I misunderstanding?

>
> In the followup I plan to add usage of __cacheline_group_begin() and
> __cacheline_group_end() usage in memcg code. If you want, take a stab
> at it.

For now, I am just looking for something simple to fix the struct size
proliferation for v6.8, but this would be interesting to see. I wonder
how __cacheline_group_end() works since the end is decided already by
__cacheline_group_begin() and the cacheline size.