Hi,
The v1.27 release of pahole and its friends is out, supporting
parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
tools such as bpftrace to enumerate the available kfuncs and obtain its
function signatures and return types.
Main git repo:
https://git.kernel.org/pub/scm/devel/pahole/pahole.git
Mirror git repo:
https://github.com/acmel/dwarves.git
tarball + gpg signature:
https://fedorapeople.org/~acme/dwarves/dwarves-1.27.tar.xz
https://fedorapeople.org/~acme/dwarves/dwarves-1.27.tar.bz2
https://fedorapeople.org/~acme/dwarves/dwarves-1.27.tar.sign
Thanks a lot to all the contributors and distro packagers, you're on the
CC list, I appreciate a lot the work you put into these tools,
Best Regards,
- Arnaldo
BTF encoder:
- Inject kfunc decl tags into BTF from the BTF IDs ELF section in the Linux
kernel vmlinux file.
This allows tools such as bpftools and pfunct to enumerate the available kfuncs
and to gets its function signature, the type of its return and of its
arguments. See the example in the BTF loader changes description, below.
- Support parallel reproducible builds, where it doesn't matter how many
threads are used, the end BTF encoding result is the same.
- Sanitize unsupported DWARF int type with greater-than-16 byte, as BTF doesn't
support it.
BTF loader:
- Initial support for BTF_KIND_DECL_TAG:
$ pfunct --prototypes -F btf vmlinux.btf.decl_tag,decl_tag_kfuncs | grep ^bpf_kfunc | head
bpf_kfunc void cubictcp_init(struct sock * sk);
bpf_kfunc void cubictcp_cwnd_event(struct sock * sk, enum tcp_ca_event event);
bpf_kfunc void cubictcp_cong_avoid(struct sock * sk, u32 ack, u32 acked);
bpf_kfunc u32 cubictcp_recalc_ssthresh(struct sock * sk);
bpf_kfunc void cubictcp_state(struct sock * sk, u8 new_state);
bpf_kfunc void cubictcp_acked(struct sock * sk, const struct ack_sample * sample);
bpf_kfunc int bpf_iter_css_new(struct bpf_iter_css * it, struct cgroup_subsys_state * start, unsigned int flags);
bpf_kfunc struct cgroup_subsys_state * bpf_iter_css_next(struct bpf_iter_css * it);
bpf_kfunc void bpf_iter_css_destroy(struct bpf_iter_css * it);
bpf_kfunc s64 bpf_map_sum_elem_count(const struct bpf_map * map);
$ pfunct --prototypes -F btf vmlinux.btf.decl_tag,decl_tag_kfuncs | grep ^bpf_kfunc | wc -l
116
$
pretty printing:
- Fix hole discovery with inheritance in C++.
Hi,
Arnaldo Carvalho de Melo wrote on Tue, Jun 11, 2024 at 06:26:53PM -0300:
> The v1.27 release of pahole and its friends is out, supporting
> parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
> tools such as bpftrace to enumerate the available kfuncs and obtain its
> function signatures and return types.
>
> Main git repo:
>
> https://git.kernel.org/pub/scm/devel/pahole/pahole.git
It looks like the v1.27 tag has not been pushed to the git repos (either
this or github), we're using git snapshots for nixpkgs, so it'd be great
if a tag could be pushed out.
(I think some release monitoring tools left and right also use tags,
even if that's less important if you Cc other distro maintainers... I
just happened to see the mail on bpf@vger.)
Thanks,
--
Dominique Martinet | Asmadeus
On Wed, Jun 12, 2024 at 06:38:02AM +0900, Dominique Martinet wrote:
> It looks like the v1.27 tag has not been pushed to the git repos (either
> this or github), we're using git snapshots for nixpkgs, so it'd be great
> if a tag could be pushed out.
Done.
https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tag/?h=v1.27
https://github.com/acmel/dwarves/releases/tag/v1.27
> (I think some release monitoring tools left and right also use tags,
> even if that's less important if you Cc other distro maintainers... I
> just happened to see the mail on bpf@vger.)
May I add your e-mail here:
acme@x1:~/git/pahole$ cat PKG-MAINTAINERS
# Please let me know if I should remove/update/add more distro package maintainers here
# I'm keeping this so that I CC them when releasing new versions, thanks!
Jan Alexander Steffens <[email protected]>
Domenico Andreoli <[email protected]>
Matthias Schwarzott <[email protected]>
Dominique Leuenberger <[email protected]>
acme@x1:~/git/pahole$
So that on the next release I CC you?
Thanks for reporting!
- Arnaldo
Arnaldo Carvalho de Melo wrote on Tue, Jun 11, 2024 at 07:52:12PM -0300:
> On Wed, Jun 12, 2024 at 06:38:02AM +0900, Dominique Martinet wrote:
> > It looks like the v1.27 tag has not been pushed to the git repos (either
> > this or github), we're using git snapshots for nixpkgs, so it'd be great
> > if a tag could be pushed out.
>
> Done.
>
> https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tag/?h=v1.27
> https://github.com/acmel/dwarves/releases/tag/v1.27
Thank you!
> > (I think some release monitoring tools left and right also use tags,
> > even if that's less important if you Cc other distro maintainers... I
> > just happened to see the mail on bpf@vger.)
>
> May I add your e-mail here:
>
> acme@x1:~/git/pahole$ cat PKG-MAINTAINERS
[..]
>
> So that on the next release I CC you?
Sure; in practice this shouldn't change much as package update is
semi-automated within a few days, but getting a heads up cannot hurt.
--
Dominique Martinet | Asmadeus
Am 11.06.24 um 23:26 schrieb Arnaldo Carvalho de Melo:
> Hi,
>
> The v1.27 release of pahole and its friends is out, supporting
> parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
> tools such as bpftrace to enumerate the available kfuncs and obtain its
> function signatures and return types.
>
Regarding packaging of pahole:
What is the state of the contained ostra-cg?
I have no clue what it is and how to use it. Is there still a use-case
for it?
Starting it without arguments only shows the usage string.
Running it with two dummy arguments:
$ ostra-cg x y
Traceback (most recent call last):
File "/usr/bin/ostra-cg", line 404, in <module>
class_def = ostra.class_definition(class_def_file = "%s.fields" %
traced_class,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/dwarves/runtime/python/ostra.py", line 154, in __init__
f = file(class_def_file)
^^^^
NameError: name 'file' is not defined. Did you mean: 'field'?
According to
https://stackoverflow.com/questions/32131230/python-file-function the
function file() does not exist in python3.
This part could be fixed by replacing it with open() but I wonder if
this is worth it.
As nobody has complained about it being broken:
Should ostra just be removed?
Regards
Matthias
Hi Arnaldo,
On Tue, Jun 11, 2024 at 06:26:53PM -0300, Arnaldo Carvalho de Melo wrote:
> The v1.27 release of pahole and its friends is out, supporting
> parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
> tools such as bpftrace to enumerate the available kfuncs and obtain its
> function signatures and return types.
After commit f632e75 ("dwarf_loader: Add the cu to the cus list early,
remove on LSK_DELETE"), I (and others[1]) notice a crash when running
pahole on modules built with Clang when CONFIG_LTO_CLANG is enabled:
$ curl -LSso .config https://gitlab.archlinux.org/archlinux/packaging/packages/linux/-/raw/main/config
$ scripts/config -d LTO_NONE -e LTO_CLANG_THIN
$ make -skj"$(nproc)" ARCH=x86_64 LLVM=1 olddefconfig vmlinux crypto/cast_common.ko
make[3]: *** [scripts/Makefile.modfinal:59: crypto/cast_common.ko] Error 139
I've isolated this to the following commands using the files available
at [2] (these were built with LLVM 18 but I could reproduce it with LLVM
17 and LLVM 19, so it appears to impact a number of versions):
$ tar -tf clang-lto-pahole-1.27-crash.tar.zst
clang-lto-pahole-1.27-crash/
clang-lto-pahole-1.27-crash/cast_common.mod.o
clang-lto-pahole-1.27-crash/module.lds
clang-lto-pahole-1.27-crash/cast_common.o
clang-lto-pahole-1.27-crash/cast_common.ko.bak
clang-lto-pahole-1.27-crash/vmlinux
clang-lto-pahole-1.27-crash/cast_common.ko
$ tar -axf clang-lto-pahole-1.27-crash.tar.zst
$ cd clang-lto-pahole-1.27-crash
$ LLVM_OBJCOPY="llvm-objcopy" pahole-1.26 -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust --btf_base vmlinux cast_common.ko
$ cp cast_common.ko{.bak,}
$ LLVM_OBJCOPY="llvm-objcopy" pahole-1.27 -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust --btf_base vmlinux cast_common.ko
fish: Job 1, '...' terminated by signal SIGSEGV (Address boundary error)
If there is any more information I can provide or patches I can test, I
am more than happy to do so.
[1]: https://gitlab.archlinux.org/archlinux/packaging/packages/pahole/-/issues/1
[2]: https://1drv.ms/u/s!AsQNYeB-IEbqqC2F28JuLy__Q7Vd?e=KsraMU
Cheers,
Nathan
On Wed, Jun 12, 2024 at 12:07:09PM +0200, Matthias Schwarzott wrote:
> Am 11.06.24 um 23:26 schrieb Arnaldo Carvalho de Melo:
> > Hi,
> > The v1.27 release of pahole and its friends is out, supporting
> > parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
> > tools such as bpftrace to enumerate the available kfuncs and obtain its
> > function signatures and return types.
> >
>
> Regarding packaging of pahole:
> What is the state of the contained ostra-cg?
I need to make a decision on that, it is used to produce things like:
http://vger.kernel.org/~acme/ostra/callgraphs/sock/0xf61bf500/
As documented in:
https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/README.ctracer
But yes, it needs to get retested after all these years to see how
difficult it would be to try and get it back working.
- Arnaldo
> I have no clue what it is and how to use it. Is there still a use-case for
> it?
>
> Starting it without arguments only shows the usage string.
> Running it with two dummy arguments:
> $ ostra-cg x y
> Traceback (most recent call last):
> File "/usr/bin/ostra-cg", line 404, in <module>
> class_def = ostra.class_definition(class_def_file = "%s.fields" %
> traced_class,
>
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/share/dwarves/runtime/python/ostra.py", line 154, in __init__
> f = file(class_def_file)
> ^^^^
> NameError: name 'file' is not defined. Did you mean: 'field'?
>
> According to
> https://stackoverflow.com/questions/32131230/python-file-function the
> function file() does not exist in python3.
>
> This part could be fixed by replacing it with open() but I wonder if this is
> worth it.
>
> As nobody has complained about it being broken:
> Should ostra just be removed?
>
> Regards
> Matthias