Received-SPF: pass (google.com: domain of linux-kernel+bounces-89002-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240228032142.396719-1-jhubbard@nvidia.com> <Zd76zrhA4LAwA_WF@krava>
 <856564cf-fba4-4473-bfa9-e9b03115abd1@oracle.com> <983b98db-79c0-4178-b88f-61f39d147cf7@nvidia.com>
 <34157878-c480-44bb-91d6-9024da329998@oracle.com> <f248cf92-038c-480f-b077-f7d56ebc55bc@nvidia.com>
 <ZeHi4qz8HqDSCC4H@krava> <a0ea8c23-96e8-4ad2-8523-6749dc59b462@nvidia.com>
In-Reply-To: <a0ea8c23-96e8-4ad2-8523-6749dc59b462@nvidia.com>
From: Suren Baghdasaryan <surenb@google.com>
Date: Fri, 1 Mar 2024 19:15:36 +0000
Message-ID: <CAJuCfpGMoOTPBJoT=R8GayLg6rfFRppKameeW1w3_V1=YgL6kA@mail.gmail.com>
Subject: Re: [PATCH] fix linux kernel BTF builds: increase max percpu
 variables by 10x
To: John Hubbard <jhubbard@nvidia.com>
Cc: Jiri Olsa <olsajiri@gmail.com>, Alan Maguire <alan.maguire@oracle.com>, 
	Arnaldo Carvalho de Melo <acme@redhat.com>, Alexei Starovoitov <ast@kernel.org>, bpf@vger.kernel.org, 
	LKML <linux-kernel@vger.kernel.org>, dwarves@vger.kernel.org, 
	Kent Overstreet <kent.overstreet@linux.dev>, Johannes Weiner <hannes@cmpxchg.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Mar 1, 2024 at 6:33=E2=80=AFPM John Hubbard <jhubbard@nvidia.com> w=
rote:
>
> On 3/1/24 06:14, Jiri Olsa wrote:
> > On Thu, Feb 29, 2024 at 10:15:23AM -0800, John Hubbard wrote:
> >>> ...
> >>> Running
> >>>
> >>> bpftool btf dump file vmlinux |grep "] VAR"
> >>>
> >>
> >> $ bpftool btf dump file vmlinux |grep "] VAR" | wc -l
> >> 4852
> >>
> >> $ bpftool btf dump file vmlinux |grep "] VAR" | tail -5
> >> [136994] VAR '_alloc_tag_cntr.9' type_id=3D703, linkage=3Dstatic
> >> [137003] VAR '_alloc_tag_cntr.5' type_id=3D703, linkage=3Dstatic
> >> [137004] VAR '_alloc_tag_cntr.7' type_id=3D703, linkage=3Dstatic
> >> [137005] VAR '_alloc_tag_cntr.17' type_id=3D703, linkage=3Dstatic
> >> [137018] VAR '_alloc_tag_cntr.14' type_id=3D703, linkage=3Dstatic
> >>
> >>> ...should give us a sense of what's going on. I only see 375 per-cpu
> >>> variables when I do this so maybe there's something
> >>> kernel-config-specific that might explain why you have so many more?
> >>
> >> Yes, as mentioned earlier, this is specifically due to the .config.
> >> The .config is a huge distro configuration that has a lot of modules
> >> enabled.
> >
> > could you share your .config? I tried with fedora .config and got 396
> > per cpu variables, I wonder where this is coming from
> >
>
> Attaching it. And based on your results, I think that Suren's Memory
> allocation profiling patchset v4 [1] may also be required, as that is wha=
t
> I was building.

Yes, that will definitely increase the number of required per-cpu
variables since it adds a per-cpu variable for each kernel allocation
to track it. I vaguely remember now that Johannes also mentioned
hitting this limit when he was using our patchset. Allocating
encoder->percpu.vars dynamically seems to be a great way to fix this
limitation.

>
> Cc: Suren and Kent. btw, I the whole reason I went down this path was tha=
t
> I recommended your patchset in order to zero in on a memory leak that a
> colleague is debugging. This patchset provides a new view of allocations
> and leaks and we have high hopes for it. :)

Thanks for trying it out John and CC'ing us on this thread!
Suren.

>
>
> [1] https://lore.kernel.org/20240221194052.927623-1-surenb@google.com
>
>
> thanks,
> --
> John Hubbard
> NVIDIA