> ...
> Running
>
> bpftool btf dump file vmlinux |grep "] VAR"
>
$ bpftool btf dump file vmlinux |grep "] VAR" | wc -l
4852
$ bpftool btf dump file vmlinux |grep "] VAR" | tail -5
[136994] VAR '_alloc_tag_cntr.9' type_id=703, linkage=static
[137003] VAR '_alloc_tag_cntr.5' type_id=703, linkage=static
[137004] VAR '_alloc_tag_cntr.7' type_id=703, linkage=static
[137005] VAR '_alloc_tag_cntr.17' type_id=703, linkage=static
[137018] VAR '_alloc_tag_cntr.14' type_id=703, linkage=static
> ...should give us a sense of what's going on. I only see 375 per-cpu
> variables when I do this so maybe there's something
> kernel-config-specific that might explain why you have so many more?
Yes, as mentioned earlier, this is specifically due to the .config.
The .config is a huge distro configuration that has a lot of modules
enabled.
thanks,
--
John Hubbard
NVIDIA
On Thu, Feb 29, 2024 at 10:15:23AM -0800, John Hubbard wrote:
> > ...
> > Running
> >
> > bpftool btf dump file vmlinux |grep "] VAR"
> >
>
> $ bpftool btf dump file vmlinux |grep "] VAR" | wc -l
> 4852
>
> $ bpftool btf dump file vmlinux |grep "] VAR" | tail -5
> [136994] VAR '_alloc_tag_cntr.9' type_id=703, linkage=static
> [137003] VAR '_alloc_tag_cntr.5' type_id=703, linkage=static
> [137004] VAR '_alloc_tag_cntr.7' type_id=703, linkage=static
> [137005] VAR '_alloc_tag_cntr.17' type_id=703, linkage=static
> [137018] VAR '_alloc_tag_cntr.14' type_id=703, linkage=static
>
> > ...should give us a sense of what's going on. I only see 375 per-cpu
> > variables when I do this so maybe there's something
> > kernel-config-specific that might explain why you have so many more?
>
> Yes, as mentioned earlier, this is specifically due to the .config.
> The .config is a huge distro configuration that has a lot of modules
> enabled.
could you share your .config? I tried with fedora .config and got 396
per cpu variables, I wonder where this is coming from
thanks,
jirka
On 3/1/24 06:14, Jiri Olsa wrote:
> On Thu, Feb 29, 2024 at 10:15:23AM -0800, John Hubbard wrote:
>>> ...
>>> Running
>>>
>>> bpftool btf dump file vmlinux |grep "] VAR"
>>>
>>
>> $ bpftool btf dump file vmlinux |grep "] VAR" | wc -l
>> 4852
>>
>> $ bpftool btf dump file vmlinux |grep "] VAR" | tail -5
>> [136994] VAR '_alloc_tag_cntr.9' type_id=703, linkage=static
>> [137003] VAR '_alloc_tag_cntr.5' type_id=703, linkage=static
>> [137004] VAR '_alloc_tag_cntr.7' type_id=703, linkage=static
>> [137005] VAR '_alloc_tag_cntr.17' type_id=703, linkage=static
>> [137018] VAR '_alloc_tag_cntr.14' type_id=703, linkage=static
>>
>>> ...should give us a sense of what's going on. I only see 375 per-cpu
>>> variables when I do this so maybe there's something
>>> kernel-config-specific that might explain why you have so many more?
>>
>> Yes, as mentioned earlier, this is specifically due to the .config.
>> The .config is a huge distro configuration that has a lot of modules
>> enabled.
>
> could you share your .config? I tried with fedora .config and got 396
> per cpu variables, I wonder where this is coming from
>
Attaching it. And based on your results, I think that Suren's Memory
allocation profiling patchset v4 [1] may also be required, as that is what
I was building.
Cc: Suren and Kent. btw, I the whole reason I went down this path was that
I recommended your patchset in order to zero in on a memory leak that a
colleague is debugging. This patchset provides a new view of allocations
and leaks and we have high hopes for it. :)
[1] https://lore.kernel.org/[email protected]
thanks,
--
John Hubbard
NVIDIA
On Fri, Mar 1, 2024 at 6:33 PM John Hubbard <[email protected]> wrote:
>
> On 3/1/24 06:14, Jiri Olsa wrote:
> > On Thu, Feb 29, 2024 at 10:15:23AM -0800, John Hubbard wrote:
> >>> ...
> >>> Running
> >>>
> >>> bpftool btf dump file vmlinux |grep "] VAR"
> >>>
> >>
> >> $ bpftool btf dump file vmlinux |grep "] VAR" | wc -l
> >> 4852
> >>
> >> $ bpftool btf dump file vmlinux |grep "] VAR" | tail -5
> >> [136994] VAR '_alloc_tag_cntr.9' type_id=703, linkage=static
> >> [137003] VAR '_alloc_tag_cntr.5' type_id=703, linkage=static
> >> [137004] VAR '_alloc_tag_cntr.7' type_id=703, linkage=static
> >> [137005] VAR '_alloc_tag_cntr.17' type_id=703, linkage=static
> >> [137018] VAR '_alloc_tag_cntr.14' type_id=703, linkage=static
> >>
> >>> ...should give us a sense of what's going on. I only see 375 per-cpu
> >>> variables when I do this so maybe there's something
> >>> kernel-config-specific that might explain why you have so many more?
> >>
> >> Yes, as mentioned earlier, this is specifically due to the .config.
> >> The .config is a huge distro configuration that has a lot of modules
> >> enabled.
> >
> > could you share your .config? I tried with fedora .config and got 396
> > per cpu variables, I wonder where this is coming from
> >
>
> Attaching it. And based on your results, I think that Suren's Memory
> allocation profiling patchset v4 [1] may also be required, as that is what
> I was building.
Yes, that will definitely increase the number of required per-cpu
variables since it adds a per-cpu variable for each kernel allocation
to track it. I vaguely remember now that Johannes also mentioned
hitting this limit when he was using our patchset. Allocating
encoder->percpu.vars dynamically seems to be a great way to fix this
limitation.
>
> Cc: Suren and Kent. btw, I the whole reason I went down this path was that
> I recommended your patchset in order to zero in on a memory leak that a
> colleague is debugging. This patchset provides a new view of allocations
> and leaks and we have high hopes for it. :)
Thanks for trying it out John and CC'ing us on this thread!
Suren.
>
>
> [1] https://lore.kernel.org/[email protected]
>
>
> thanks,
> --
> John Hubbard
> NVIDIA