2021-02-04 22:11:01

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

Hi,

The v1.20 release of pahole and its friends is out, mostly
addressing problems related to gcc 11 defaulting to DWARF5 for -g,
available at the usual places:

Main git repo:

git://git.kernel.org/pub/scm/devel/pahole/pahole.git

Mirror git repo:

https://github.com/acmel/dwarves.git

tarball + gpg signature:

https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.xz
https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.bz2
https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.sign

Best Regards,

- Arnaldo

v1.20:

BTF encoder:

- Improve ELF error reporting using elf_errmsg(elf_errno()).

- Improve objcopy error handling.

- Fix handling of 'restrict' qualifier, that was being treated as a 'const'.

- Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with
more than 65534 sections, for instance, which happens with kernels built
with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may
include when using FG-ASLR, LTO.

- Cope with functions without a name, as seen sometimes when building kernel
images with some versions of clang, when a SEGFAULT was taking place.

- Fix BTF variable generation for kernel modules, not skipping variables at
offset zero.

- Fix address size to match what is in the ELF file being processed, to fix using
a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image.

- Use kernel module ftrace addresses when finding which functions to encode,
which increases the number of functions encoded.

libbpf:

- Allow use of packaged version, for distros wanting to dynamically link with
the system's libbpf package instead of using the libbpf git submodule shipped
in pahole's source code.

DWARF loader:

- Support DW_AT_data_bit_offset

This appeared in DWARF4 but is supported only in gcc's -gdwarf-5,
support it in a way that makes the output be the same for both cases.

$ gcc -gdwarf-5 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
long int a:1; /* 0: 0 8 */
long int b:1; /* 0: 1 8 */
long int c:1; /* 0: 2 8 */

/* XXX 29 bits hole, try to pack */
/* Bitfield combined with next fields */

int after_bitfield; /* 4 4 */

/* size: 8, cachelines: 1, members: 4 */
/* sum members: 4 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* last cacheline: 8 bytes */
};

- DW_FORM_implicit_const in attr_numeric() and attr_offset()

- Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported
DW_TAG_GNU_call_site.

build:

- Fix compilation on 32-bit architectures.

Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>


2021-02-04 22:14:21

by Sedat Dilek

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

On Thu, Feb 4, 2021 at 11:07 PM Arnaldo Carvalho de Melo
<[email protected]> wrote:
>
> Hi,
>
> The v1.20 release of pahole and its friends is out, mostly
> addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> available at the usual places:
>

Congrats and thanks for v1.20 and to all involved folks.

- Sedat -

> Main git repo:
>
> git://git.kernel.org/pub/scm/devel/pahole/pahole.git
>
> Mirror git repo:
>
> https://github.com/acmel/dwarves.git
>
> tarball + gpg signature:
>
> https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.xz
> https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.bz2
> https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.sign
>
> Best Regards,
>
> - Arnaldo
>
> v1.20:
>
> BTF encoder:
>
> - Improve ELF error reporting using elf_errmsg(elf_errno()).
>
> - Improve objcopy error handling.
>
> - Fix handling of 'restrict' qualifier, that was being treated as a 'const'.
>
> - Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with
> more than 65534 sections, for instance, which happens with kernels built
> with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may
> include when using FG-ASLR, LTO.
>
> - Cope with functions without a name, as seen sometimes when building kernel
> images with some versions of clang, when a SEGFAULT was taking place.
>
> - Fix BTF variable generation for kernel modules, not skipping variables at
> offset zero.
>
> - Fix address size to match what is in the ELF file being processed, to fix using
> a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image.
>
> - Use kernel module ftrace addresses when finding which functions to encode,
> which increases the number of functions encoded.
>
> libbpf:
>
> - Allow use of packaged version, for distros wanting to dynamically link with
> the system's libbpf package instead of using the libbpf git submodule shipped
> in pahole's source code.
>
> DWARF loader:
>
> - Support DW_AT_data_bit_offset
>
> This appeared in DWARF4 but is supported only in gcc's -gdwarf-5,
> support it in a way that makes the output be the same for both cases.
>
> $ gcc -gdwarf-5 -c examples/dwarf5/bf.c
> $ pahole bf.o
> struct pea {
> long int a:1; /* 0: 0 8 */
> long int b:1; /* 0: 1 8 */
> long int c:1; /* 0: 2 8 */
>
> /* XXX 29 bits hole, try to pack */
> /* Bitfield combined with next fields */
>
> int after_bitfield; /* 4 4 */
>
> /* size: 8, cachelines: 1, members: 4 */
> /* sum members: 4 */
> /* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
> /* last cacheline: 8 bytes */
> };
>
> - DW_FORM_implicit_const in attr_numeric() and attr_offset()
>
> - Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported
> DW_TAG_GNU_call_site.
>
> build:
>
> - Fix compilation on 32-bit architectures.
>
> Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>

2021-02-05 01:57:36

by Andrii Nakryiko

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

On Thu, Feb 4, 2021 at 2:09 PM Arnaldo Carvalho de Melo <[email protected]> wrote:
>
> Hi,
>
> The v1.20 release of pahole and its friends is out, mostly
> addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> available at the usual places:

Great, thanks, Arnaldo! Do you plan to build RPMs soon as well?

>
> Main git repo:
>
> git://git.kernel.org/pub/scm/devel/pahole/pahole.git
>
> Mirror git repo:
>
> https://github.com/acmel/dwarves.git
>
> tarball + gpg signature:
>
> https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.xz
> https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.bz2
> https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.sign
>
> Best Regards,
>
> - Arnaldo
>

[...]

2021-02-05 04:36:50

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)



On February 4, 2021 9:01:51 PM GMT-03:00, Andrii Nakryiko <[email protected]> wrote:
>On Thu, Feb 4, 2021 at 2:09 PM Arnaldo Carvalho de Melo><[email protected]> wrote:
>> The v1.20 release of pahole and its friends is out, mostly
>> addressing problems related to gcc 11 defaulting to DWARF5 for -g,
>> available at the usual places:
>
>Great, thanks, Arnaldo! Do you plan to build RPMs soon as well?

It's in rawhide already, I'll do it for f33, f32 later,

- Arnaldo

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

2021-02-05 07:44:07

by Andrii Nakryiko

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

On Thu, Feb 4, 2021 at 8:34 PM Arnaldo Carvalho de Melo
<[email protected]> wrote:
>
>
>
> On February 4, 2021 9:01:51 PM GMT-03:00, Andrii Nakryiko <[email protected]> wrote:
> >On Thu, Feb 4, 2021 at 2:09 PM Arnaldo Carvalho de Melo><[email protected]> wrote:
> >> The v1.20 release of pahole and its friends is out, mostly
> >> addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> >> available at the usual places:
> >
> >Great, thanks, Arnaldo! Do you plan to build RPMs soon as well?
>
> It's in rawhide already, I'll do it for f33, f32 later,
>

Do you have a link? I tried to find it, but only see 1.19 so far.

> - Arnaldo
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.

2021-02-05 09:42:25

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)



On February 5, 2021 4:39:47 AM GMT-03:00, Andrii Nakryiko <[email protected]> wrote:
>On Thu, Feb 4, 2021 at 8:34 PM Arnaldo Carvalho de Melo
><[email protected]> wrote:
>>
>>
>>
>> On February 4, 2021 9:01:51 PM GMT-03:00, Andrii Nakryiko
><[email protected]> wrote:
>> >On Thu, Feb 4, 2021 at 2:09 PM Arnaldo Carvalho de
>Melo><[email protected]> wrote:
>> >> The v1.20 release of pahole and its friends is out, mostly
>> >> addressing problems related to gcc 11 defaulting to DWARF5 for -g,
>> >> available at the usual places:
>> >
>> >Great, thanks, Arnaldo! Do you plan to build RPMs soon as well?
>>
>> It's in rawhide already, I'll do it for f33, f32 later,
>>
>
>Do you have a link? I tried to find it, but only see 1.19 so far.


https://koji.fedoraproject.org/koji/buildinfo?buildID=1703678

- Arnaldo

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

2021-02-05 20:38:40

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

Em Fri, Feb 05, 2021 at 06:33:43AM -0300, Arnaldo Carvalho de Melo escreveu:
> On February 5, 2021 4:39:47 AM GMT-03:00, Andrii Nakryiko <[email protected]> wrote:
> >On Thu, Feb 4, 2021 at 8:34 PM Arnaldo Carvalho de Melo ><[email protected]> wrote:
> >> On February 4, 2021 9:01:51 PM GMT-03:00, Andrii Nakryiko
> ><[email protected]> wrote:
> >> >On Thu, Feb 4, 2021 at 2:09 PM Arnaldo Carvalho de
> >Melo><[email protected]> wrote:
> >> >> The v1.20 release of pahole and its friends is out, mostly
> >> >> addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> >> >> available at the usual places:

> >> >Great, thanks, Arnaldo! Do you plan to build RPMs soon as well?

> >> It's in rawhide already, I'll do it for f33, f32 later,

> >Do you have a link? I tried to find it, but only see 1.19 so far.

> https://koji.fedoraproject.org/koji/buildinfo?buildID=1703678

And now for Fedora 33, waiting for karma bumps at:

https://bodhi.fedoraproject.org/updates/FEDORA-2021-804e7a572c

fedpkg buidling for f32 now.

- Arnaldo

2021-02-05 20:43:17

by Sedat Dilek

[permalink] [raw]
Subject: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

Hi,

when building with pahole v1.20 and binutils v2.35.2 plus Clang
v12.0.0-rc1 and DWARF-v5 I see:
...
+ info BTF .btf.vmlinux.bin.o
+ [ != silent_ ]
+ printf %-7s %s\n BTF .btf.vmlinux.bin.o
BTF .btf.vmlinux.bin.o
+ LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
.tmp_vmlinux.btf
[115] INT DW_ATE_unsigned_1 Error emitting BTF type
Encountered error while encoding BTF.
+ /opt/binutils/bin/objcopy --only-section=.BTF --set-section-flags
.BTF=alloc,readonly --strip-all .tmp_vmlinux.btf .btf.vmlinux.bin.o
+ printf \1
+ dd of=.btf.vmlinux.bin.o conv=notrunc bs=1 seek=16 status=none
+ kallsymso=
...
+ info BTFIDS vmlinux
+ [ != silent_ ]
+ printf %-7s %s\n BTFIDS vmlinux
BTFIDS vmlinux
+ ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
FAILED: load BTF from vmlinux: No such file or directory
+ on_exit
+ [ 255 -ne 0 ]
+ cleanup
+ rm -f .btf.vmlinux.bin.o
+ rm -f .tmp_System.map
+ rm -f .tmp_vmlinux.btf .tmp_vmlinux.kallsyms1
.tmp_vmlinux.kallsyms1.S .tmp_vmlinux.kallsyms1.o
.tmp_vmlinux.kallsyms2 .tmp_vmlinux.kallsyms2.S .tmp_vmlinux.kallsyms
2.o
+ rm -f System.map
+ rm -f vmlinux
+ rm -f vmlinux.o
make[3]: *** [Makefile:1166: vmlinux] Error 255

Looks like it does not matter to use
LLVM_OBJCOPY=/opt/llvm-toolchain/bin/llvm-objcopy:

+ info BTF .btf.vmlinux.bin.o
+ [ != silent_ ]
+ printf %-7s %s\n BTF .btf.vmlinux.bin.o
BTF .btf.vmlinux.bin.o
+ LLVM_OBJCOPY=/opt/llvm-toolchain/bin/llvm-objcopy
/opt/pahole/bin/pahole -J .tmp_vmlinux.btf
[115] INT DW_ATE_unsigned_1 Error emitting BTF type
Encountered error while encoding BTF.

My make-line looks like this:

make V=1 -j4 HOSTCC=clang HOSTCXX=clang++
HOSTLD=/opt/binutils/bin/ld.bfd HOSTAR=/opt/binutils/bin/ar CC=clang
LD=/opt/binutils/bin/ld.
bfd AR=/opt/binutils/bin/ar NM=/opt/binutils/bin/nm
STRIP=/opt/binutils/bin/strip OBJCOPY=/opt/binutils/bin/objcopy
OBJDUMP=/opt/binutils/bin/objdump READELF=/opt/binu
tils/bin/readelf PAHOLE=/opt/pahole/bin/pahole
LOCALVERSION=-11-amd64-clang12-bfd KBUILD_VERBOSE=1
KBUILD_BUILD_HOST=iniza [email protected]
KBUILD_BUILD_TIMESTAMP=2021-02-05 bindeb-pkg
KDEB_PKGVERSION=5.11.0~rc6-11~bullseye+dileks1

My kernel-config is attached.

If you need further informations, please let me know.

Thanks.

Regards,
- Sedat -


Attachments:
config-5.11.0-rc6-11-amd64-clang12-bfd (232.62 kB)

2021-02-05 21:14:59

by Nick Desaulniers

[permalink] [raw]
Subject: Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

> Hi,
>
> when building with pahole v1.20 and binutils v2.35.2 plus Clang
> v12.0.0-rc1 and DWARF-v5 I see:
> ...
> + info BTF .btf.vmlinux.bin.o
> + [ != silent_ ]
> + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> BTF .btf.vmlinux.bin.o
> + LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> .tmp_vmlinux.btf
> [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> Encountered error while encoding BTF.

Yes, I observe this error, too.

https://gist.github.com/nickdesaulniers/ae8c9efbe4da69b1cf0dce138c1d2781

via v9 of my DWARF v5 series, which should help make this easier to
reproduce:
https://lore.kernel.org/lkml/CA+icZUW3sg_PkbmKSFMs6EqwQV7=hvKuAgZSsbg=Qr6gTs7RbQ@mail.gmail.com/T/#m45ec7e6df4c4b5e9da034b95d7dfc8e2a0c81dac

2021-02-05 21:20:10

by Sedat Dilek

[permalink] [raw]
Subject: Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

On Fri, Feb 5, 2021 at 10:10 PM Nick Desaulniers
<[email protected]> wrote:
>
> > Hi,
> >
> > when building with pahole v1.20 and binutils v2.35.2 plus Clang
> > v12.0.0-rc1 and DWARF-v5 I see:
> > ...
> > + info BTF .btf.vmlinux.bin.o
> > + [ != silent_ ]
> > + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> > BTF .btf.vmlinux.bin.o
> > + LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> > .tmp_vmlinux.btf
> > [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> > Encountered error while encoding BTF.
>
> Yes, I observe this error, too.
>
> https://gist.github.com/nickdesaulniers/ae8c9efbe4da69b1cf0dce138c1d2781
>
> via v9 of my DWARF v5 series, which should help make this easier to
> reproduce:
> https://lore.kernel.org/lkml/CA+icZUW3sg_PkbmKSFMs6EqwQV7=hvKuAgZSsbg=Qr6gTs7RbQ@mail.gmail.com/T/#m45ec7e6df4c4b5e9da034b95d7dfc8e2a0c81dac

Thanks Nick for confirming the error.

Ah, I see you passed:

make LLVM=1 LLVM_IAS=1 ...

Can you by chance try with KCFLAGS="-fbinutils-version=2.35"?

- Sedat -

2021-02-05 21:28:44

by Sedat Dilek

[permalink] [raw]
Subject: Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

On Fri, Feb 5, 2021 at 3:37 PM Sedat Dilek <[email protected]> wrote:
>
> Hi,
>
> when building with pahole v1.20 and binutils v2.35.2 plus Clang
> v12.0.0-rc1 and DWARF-v5 I see:
> ...
> + info BTF .btf.vmlinux.bin.o
> + [ != silent_ ]
> + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> BTF .btf.vmlinux.bin.o
> + LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> .tmp_vmlinux.btf
> [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> Encountered error while encoding BTF.

Grepping the pahole sources:

$ git grep DW_ATE
dwarf_loader.c: bt->is_bool = encoding == DW_ATE_boolean;
dwarf_loader.c: bt->is_signed = encoding == DW_ATE_signed;

Missing DW_ATE_unsigned encoding?

- Sedat -

2021-02-05 21:43:44

by Nick Desaulniers

[permalink] [raw]
Subject: Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

On Fri, Feb 5, 2021 at 1:17 PM Sedat Dilek <[email protected]> wrote:
>
> On Fri, Feb 5, 2021 at 10:10 PM Nick Desaulniers
> <[email protected]> wrote:
> >
> > > Hi,
> > >
> > > when building with pahole v1.20 and binutils v2.35.2 plus Clang
> > > v12.0.0-rc1 and DWARF-v5 I see:
> > > ...
> > > + info BTF .btf.vmlinux.bin.o
> > > + [ != silent_ ]
> > > + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> > > BTF .btf.vmlinux.bin.o
> > > + LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> > > .tmp_vmlinux.btf
> > > [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> > > Encountered error while encoding BTF.
> >
> > Yes, I observe this error, too.
> >
> > https://gist.github.com/nickdesaulniers/ae8c9efbe4da69b1cf0dce138c1d2781
> >
> > via v9 of my DWARF v5 series, which should help make this easier to
> > reproduce:
> > https://lore.kernel.org/lkml/CA+icZUW3sg_PkbmKSFMs6EqwQV7=hvKuAgZSsbg=Qr6gTs7RbQ@mail.gmail.com/T/#m45ec7e6df4c4b5e9da034b95d7dfc8e2a0c81dac
>
> Thanks Nick for confirming the error.
>
> Ah, I see you passed:
>
> make LLVM=1 LLVM_IAS=1 ...
>
> Can you by chance try with KCFLAGS="-fbinutils-version=2.35"?

$ PATH=/path/to/tot/pahole/build:$PATH make LLVM=1 LLVM_IAS=1 -j72
KCFLAGS="-fbinutils-version=2.35"
...
BTF .btf.vmlinux.bin.o
[12919] INT DW_ATE_unsigned_1 Error emitting BTF type
Encountered error while encoding BTF.
...
LD vmlinux
BTFIDS vmlinux
FAILED: load BTF from vmlinux: Invalid argument
--
Thanks,
~Nick Desaulniers

2021-02-05 21:48:45

by Sedat Dilek

[permalink] [raw]
Subject: Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

On Fri, Feb 5, 2021 at 10:40 PM Nick Desaulniers
<[email protected]> wrote:
>
> On Fri, Feb 5, 2021 at 1:17 PM Sedat Dilek <[email protected]> wrote:
> >
> > On Fri, Feb 5, 2021 at 10:10 PM Nick Desaulniers
> > <[email protected]> wrote:
> > >
> > > > Hi,
> > > >
> > > > when building with pahole v1.20 and binutils v2.35.2 plus Clang
> > > > v12.0.0-rc1 and DWARF-v5 I see:
> > > > ...
> > > > + info BTF .btf.vmlinux.bin.o
> > > > + [ != silent_ ]
> > > > + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> > > > BTF .btf.vmlinux.bin.o
> > > > + LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> > > > .tmp_vmlinux.btf
> > > > [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> > > > Encountered error while encoding BTF.
> > >
> > > Yes, I observe this error, too.
> > >
> > > https://gist.github.com/nickdesaulniers/ae8c9efbe4da69b1cf0dce138c1d2781
> > >
> > > via v9 of my DWARF v5 series, which should help make this easier to
> > > reproduce:
> > > https://lore.kernel.org/lkml/CA+icZUW3sg_PkbmKSFMs6EqwQV7=hvKuAgZSsbg=Qr6gTs7RbQ@mail.gmail.com/T/#m45ec7e6df4c4b5e9da034b95d7dfc8e2a0c81dac
> >
> > Thanks Nick for confirming the error.
> >
> > Ah, I see you passed:
> >
> > make LLVM=1 LLVM_IAS=1 ...
> >
> > Can you by chance try with KCFLAGS="-fbinutils-version=2.35"?
>
> $ PATH=/path/to/tot/pahole/build:$PATH make LLVM=1 LLVM_IAS=1 -j72
> KCFLAGS="-fbinutils-version=2.35"
> ...
> BTF .btf.vmlinux.bin.o
> [12919] INT DW_ATE_unsigned_1 Error emitting BTF type
> Encountered error while encoding BTF.
> ...
> LD vmlinux
> BTFIDS vmlinux
> FAILED: load BTF from vmlinux: Invalid argument
>

OK, thanks.

I stopped my build:

/usr/bin/perf_5.10 stat make V=1 -j4 LLVM=1 LLVM_IAS=1
PAHOLE=/opt/pahole/bin/pahole KCFLAGS=-fbinutils-version=2.3
5 LOCALVERSION=-13-amd64-clang12-llvm KBUILD_VERBOSE=1
KBUILD_BUILD_HOST=iniza [email protected]
KBUILD_BUILD_TIMESTAMP=2021-02-05 bindeb-pkg KDE
B_PKGVERSION=5.11.0~rc6-13~bullseye+dileks1

Enjoy weekend.

- Sedat -

2021-02-05 23:04:07

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

Em Fri, Feb 05, 2021 at 04:23:59PM +0100, Sedat Dilek escreveu:
> On Fri, Feb 5, 2021 at 3:41 PM Sedat Dilek <[email protected]> wrote:
> >
> > On Fri, Feb 5, 2021 at 3:37 PM Sedat Dilek <[email protected]> wrote:
> > >
> > > Hi,
> > >
> > > when building with pahole v1.20 and binutils v2.35.2 plus Clang
> > > v12.0.0-rc1 and DWARF-v5 I see:
> > > ...
> > > + info BTF .btf.vmlinux.bin.o
> > > + [ != silent_ ]
> > > + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> > > BTF .btf.vmlinux.bin.o
> > > + LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> > > .tmp_vmlinux.btf
> > > [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> > > Encountered error while encoding BTF.
> >
> > Grepping the pahole sources:
> >
> > $ git grep DW_ATE
> > dwarf_loader.c: bt->is_bool = encoding == DW_ATE_boolean;
> > dwarf_loader.c: bt->is_signed = encoding == DW_ATE_signed;
> >
> > Missing DW_ATE_unsigned encoding?
> >
>
> Checked the LLVM sources:
>
> clang/lib/CodeGen/CGDebugInfo.cpp: Encoding =
> llvm::dwarf::DW_ATE_unsigned_char;
> clang/lib/CodeGen/CGDebugInfo.cpp: Encoding = llvm::dwarf::DW_ATE_unsigned;
> clang/lib/CodeGen/CGDebugInfo.cpp: Encoding =
> llvm::dwarf::DW_ATE_unsigned_fixed;
> clang/lib/CodeGen/CGDebugInfo.cpp:
> ? llvm::dwarf::DW_ATE_unsigned
> ...
> lld/test/wasm/debuginfo.test:CHECK-NEXT: DW_AT_encoding
> (DW_ATE_unsigned)
>
> So, I will switch from GNU ld.bfd v2.35.2 to LLD-12.

Thanks for the research, probably your conclusion is correct, can you go
the next step and add that part and check if the end result is the
expected one?

- Arnaldo

2021-02-05 23:37:17

by Sedat Dilek

[permalink] [raw]
Subject: Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

On Fri, Feb 5, 2021 at 4:28 PM Arnaldo Carvalho de Melo
<[email protected]> wrote:
>
> Em Fri, Feb 05, 2021 at 04:23:59PM +0100, Sedat Dilek escreveu:
> > On Fri, Feb 5, 2021 at 3:41 PM Sedat Dilek <[email protected]> wrote:
> > >
> > > On Fri, Feb 5, 2021 at 3:37 PM Sedat Dilek <[email protected]> wrote:
> > > >
> > > > Hi,
> > > >
> > > > when building with pahole v1.20 and binutils v2.35.2 plus Clang
> > > > v12.0.0-rc1 and DWARF-v5 I see:
> > > > ...
> > > > + info BTF .btf.vmlinux.bin.o
> > > > + [ != silent_ ]
> > > > + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> > > > BTF .btf.vmlinux.bin.o
> > > > + LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> > > > .tmp_vmlinux.btf
> > > > [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> > > > Encountered error while encoding BTF.
> > >
> > > Grepping the pahole sources:
> > >
> > > $ git grep DW_ATE
> > > dwarf_loader.c: bt->is_bool = encoding == DW_ATE_boolean;
> > > dwarf_loader.c: bt->is_signed = encoding == DW_ATE_signed;
> > >
> > > Missing DW_ATE_unsigned encoding?
> > >
> >
> > Checked the LLVM sources:
> >
> > clang/lib/CodeGen/CGDebugInfo.cpp: Encoding =
> > llvm::dwarf::DW_ATE_unsigned_char;
> > clang/lib/CodeGen/CGDebugInfo.cpp: Encoding = llvm::dwarf::DW_ATE_unsigned;
> > clang/lib/CodeGen/CGDebugInfo.cpp: Encoding =
> > llvm::dwarf::DW_ATE_unsigned_fixed;
> > clang/lib/CodeGen/CGDebugInfo.cpp:
> > ? llvm::dwarf::DW_ATE_unsigned
> > ...
> > lld/test/wasm/debuginfo.test:CHECK-NEXT: DW_AT_encoding
> > (DW_ATE_unsigned)
> >
> > So, I will switch from GNU ld.bfd v2.35.2 to LLD-12.
>
> Thanks for the research, probably your conclusion is correct, can you go
> the next step and add that part and check if the end result is the
> expected one?
>

Started a new build:

$ ps -ef | grep p[e]rf
dileks 529807 529775 0 16:32 pts/1 00:00:00 /usr/bin/perf_5.10
stat make V=1 -j4 LLVM=1 PAHOLE=/opt/pahole/bin/pahole
LOCALVERSION=-12-amd64-clang12-llvm KBUIL
D_VERBOSE=1 KBUILD_BUILD_HOST=iniza
[email protected]
KBUILD_BUILD_TIMESTAMP=2021-02-05 bindeb-pkg
KDEB_PKGVERSION=5.11.0~rc6-12~bullseye+dileks1

$ scripts/diffconfig /boot/config-5.11.0-rc6-10-amd64-clang12-bfd .config
BUILD_SALT "5.11.0-rc6-10-amd64-clang12-bfd" ->
"5.11.0-rc6-11-amd64-clang12-llvm"
DEBUG_INFO_DWARF4 y -> n
LD_VERSION 235020000 -> 0
LLD_VERSION 0 -> 120000
+DEBUG_INFO_DWARF5 y
+LD_IS_LLD y
+TOOLS_SUPPORT_RELR y

Will take approx. 03:15 [hh:mm] and report later.

- Sedat -

2021-02-05 23:53:56

by Sedat Dilek

[permalink] [raw]
Subject: Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

On Fri, Feb 5, 2021 at 3:41 PM Sedat Dilek <[email protected]> wrote:
>
> On Fri, Feb 5, 2021 at 3:37 PM Sedat Dilek <[email protected]> wrote:
> >
> > Hi,
> >
> > when building with pahole v1.20 and binutils v2.35.2 plus Clang
> > v12.0.0-rc1 and DWARF-v5 I see:
> > ...
> > + info BTF .btf.vmlinux.bin.o
> > + [ != silent_ ]
> > + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> > BTF .btf.vmlinux.bin.o
> > + LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> > .tmp_vmlinux.btf
> > [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> > Encountered error while encoding BTF.
>
> Grepping the pahole sources:
>
> $ git grep DW_ATE
> dwarf_loader.c: bt->is_bool = encoding == DW_ATE_boolean;
> dwarf_loader.c: bt->is_signed = encoding == DW_ATE_signed;
>
> Missing DW_ATE_unsigned encoding?
>

Checked the LLVM sources:

clang/lib/CodeGen/CGDebugInfo.cpp: Encoding =
llvm::dwarf::DW_ATE_unsigned_char;
clang/lib/CodeGen/CGDebugInfo.cpp: Encoding = llvm::dwarf::DW_ATE_unsigned;
clang/lib/CodeGen/CGDebugInfo.cpp: Encoding =
llvm::dwarf::DW_ATE_unsigned_fixed;
clang/lib/CodeGen/CGDebugInfo.cpp:
? llvm::dwarf::DW_ATE_unsigned
...
lld/test/wasm/debuginfo.test:CHECK-NEXT: DW_AT_encoding
(DW_ATE_unsigned)

So, I will switch from GNU ld.bfd v2.35.2 to LLD-12.

- Sedat -

2021-02-06 03:57:55

by Andrii Nakryiko

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

On Fri, Feb 5, 2021 at 8:25 AM Arnaldo Carvalho de Melo
<[email protected]> wrote:
>
> Em Fri, Feb 05, 2021 at 06:33:43AM -0300, Arnaldo Carvalho de Melo escreveu:
> > On February 5, 2021 4:39:47 AM GMT-03:00, Andrii Nakryiko <[email protected]> wrote:
> > >On Thu, Feb 4, 2021 at 8:34 PM Arnaldo Carvalho de Melo ><[email protected]> wrote:
> > >> On February 4, 2021 9:01:51 PM GMT-03:00, Andrii Nakryiko
> > ><[email protected]> wrote:
> > >> >On Thu, Feb 4, 2021 at 2:09 PM Arnaldo Carvalho de
> > >Melo><[email protected]> wrote:
> > >> >> The v1.20 release of pahole and its friends is out, mostly
> > >> >> addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> > >> >> available at the usual places:
>
> > >> >Great, thanks, Arnaldo! Do you plan to build RPMs soon as well?
>
> > >> It's in rawhide already, I'll do it for f33, f32 later,
>
> > >Do you have a link? I tried to find it, but only see 1.19 so far.
>
> > https://koji.fedoraproject.org/koji/buildinfo?buildID=1703678
>
> And now for Fedora 33, waiting for karma bumps at:
>
> https://bodhi.fedoraproject.org/updates/FEDORA-2021-804e7a572c
>
> fedpkg buidling for f32 now.
>
> - Arnaldo

Ok, imported dwarves-1.20. Had to fix two dates in changelog (in
spec), day of week didn't match the date, tooling complained about
that. Also had to undo cmake_build and cmake_install fanciness,
because apparently we don't have them or the support for it is not
great. But otherwise everything else looks to be ok.

2021-02-06 04:04:39

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

Em Fri, Feb 05, 2021 at 02:11:44PM -0800, Andrii Nakryiko escreveu:
> On Fri, Feb 5, 2021 at 8:25 AM Arnaldo Carvalho de Melo
> <[email protected]> wrote:
> >
> > Em Fri, Feb 05, 2021 at 06:33:43AM -0300, Arnaldo Carvalho de Melo escreveu:
> > > On February 5, 2021 4:39:47 AM GMT-03:00, Andrii Nakryiko <[email protected]> wrote:
> > > >On Thu, Feb 4, 2021 at 8:34 PM Arnaldo Carvalho de Melo ><[email protected]> wrote:
> > > >> On February 4, 2021 9:01:51 PM GMT-03:00, Andrii Nakryiko
> > > ><[email protected]> wrote:
> > > >> >On Thu, Feb 4, 2021 at 2:09 PM Arnaldo Carvalho de
> > > >Melo><[email protected]> wrote:
> > > >> >> The v1.20 release of pahole and its friends is out, mostly
> > > >> >> addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> > > >> >> available at the usual places:
> >
> > > >> >Great, thanks, Arnaldo! Do you plan to build RPMs soon as well?
> >
> > > >> It's in rawhide already, I'll do it for f33, f32 later,
> >
> > > >Do you have a link? I tried to find it, but only see 1.19 so far.
> >
> > > https://koji.fedoraproject.org/koji/buildinfo?buildID=1703678
> >
> > And now for Fedora 33, waiting for karma bumps at:
> >
> > https://bodhi.fedoraproject.org/updates/FEDORA-2021-804e7a572c
> >
> > fedpkg buidling for f32 now.

> Ok, imported dwarves-1.20. Had to fix two dates in changelog (in
> spec), day of week didn't match the date, tooling complained about
> that. Also had to undo cmake_build and cmake_install fanciness,
> because apparently we don't have them or the support for it is not
> great. But otherwise everything else looks to be ok.

Send patch please, I wasn't expecting this, if you could do some more
and send me tooling bits to help me in the release process, if that is
possible, I'd love to get it, otherwise I'll write it, don't want to go
thru this one more time, sigh :-(


- Arnaldo

2021-02-07 06:42:46

by Andrii Nakryiko

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

On Fri, Feb 5, 2021 at 3:55 PM Arnaldo Carvalho de Melo
<[email protected]> wrote:
>
> Em Fri, Feb 05, 2021 at 02:11:44PM -0800, Andrii Nakryiko escreveu:
> > On Fri, Feb 5, 2021 at 8:25 AM Arnaldo Carvalho de Melo
> > <[email protected]> wrote:
> > >
> > > Em Fri, Feb 05, 2021 at 06:33:43AM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > On February 5, 2021 4:39:47 AM GMT-03:00, Andrii Nakryiko <[email protected]> wrote:
> > > > >On Thu, Feb 4, 2021 at 8:34 PM Arnaldo Carvalho de Melo ><[email protected]> wrote:
> > > > >> On February 4, 2021 9:01:51 PM GMT-03:00, Andrii Nakryiko
> > > > ><[email protected]> wrote:
> > > > >> >On Thu, Feb 4, 2021 at 2:09 PM Arnaldo Carvalho de
> > > > >Melo><[email protected]> wrote:
> > > > >> >> The v1.20 release of pahole and its friends is out, mostly
> > > > >> >> addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> > > > >> >> available at the usual places:
> > >
> > > > >> >Great, thanks, Arnaldo! Do you plan to build RPMs soon as well?
> > >
> > > > >> It's in rawhide already, I'll do it for f33, f32 later,
> > >
> > > > >Do you have a link? I tried to find it, but only see 1.19 so far.
> > >
> > > > https://koji.fedoraproject.org/koji/buildinfo?buildID=1703678
> > >
> > > And now for Fedora 33, waiting for karma bumps at:
> > >
> > > https://bodhi.fedoraproject.org/updates/FEDORA-2021-804e7a572c
> > >
> > > fedpkg buidling for f32 now.
>
> > Ok, imported dwarves-1.20. Had to fix two dates in changelog (in
> > spec), day of week didn't match the date, tooling complained about
> > that. Also had to undo cmake_build and cmake_install fanciness,
> > because apparently we don't have them or the support for it is not
> > great. But otherwise everything else looks to be ok.
>
> Send patch please, I wasn't expecting this, if you could do some more
> and send me tooling bits to help me in the release process, if that is
> possible, I'd love to get it, otherwise I'll write it, don't want to go
> thru this one more time, sigh :-(
>

I just "reverted" some bits of spec file to what it used to be
pre-1.19, I think. There is also a work-around for lack of
ldconfig_scriptlets support in our platform, so I have to work around
that. Here's full diff:

diff -u sources/git/rpm/SPECS/dwarves.spec specfiles/dwarves.spec
--- sources/git/rpm/SPECS/dwarves.spec 2021-02-05 11:19:54.364938716 -0800
+++ specfiles/dwarves.spec 2021-02-05 13:26:52.502859609 -0800
@@ -3,12 +3,14 @@

Name: dwarves
Version: 1.20
-Release: 1%{?dist}
+Release: 1fb1%{?dist}
License: GPLv2
Summary: Debugging Information Manipulation Tools (pahole & friends)
URL: http://acmel.wordpress.com
Source: http://fedorapeople.org/~acme/dwarves/%{name}-%{version}.tar.xz
Requires: %{libname}%{libver} = %{version}-%{release}
+Requires(post): /sbin/ldconfig
+Requires(postun): /sbin/ldconfig
BuildRequires: gcc
BuildRequires: cmake >= 2.8.12
BuildRequires: zlib-devel
@@ -68,13 +70,14 @@

%build
%cmake -DCMAKE_BUILD_TYPE=Release .
-%cmake_build
+make VERBOSE=1 %{?_smp_mflags}

%install
rm -Rf %{buildroot}
-%cmake_install
+make install DESTDIR=%{buildroot}

-%ldconfig_scriptlets -n %{libname}%{libver}
+%post -p /sbin/ldconfig
+%postun -p /sbin/ldconfig

%files
%doc README.ctracer
@@ -295,7 +298,7 @@
* Sat Nov 20 2010 Arnaldo Carvalho de Melo <[email protected]> - 1.9-1
- New release

-* Tue Feb 08 2010 Fedora Release Engineering
<[email protected]> - 1.8-2
+* Mon Feb 08 2010 Fedora Release Engineering
<[email protected]> - 1.8-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_15_Mass_Rebuild

* Fri Dec 4 2009 Arnaldo Carvalho de Melo <[email protected]> - 1.8-1
@@ -446,7 +449,7 @@
- Fix emission of arrays of structs, unions, etc
- use sysconf for the default cacheline size

-* Wed Jan 18 2007 Arnaldo Carvalho de Melo <[email protected]>
+* Thu Jan 18 2007 Arnaldo Carvalho de Melo <[email protected]>
- fab0db03ea9046893ca110bb2b7d71b764f61033
- pdwtags added

>
> - Arnaldo

2021-02-08 02:47:54

by Sedat Dilek

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

On Thu, Feb 4, 2021 at 11:07 PM Arnaldo Carvalho de Melo
<[email protected]> wrote:
>
> Hi,
>
> The v1.20 release of pahole and its friends is out, mostly
> addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> available at the usual places:
>
> Main git repo:
>
> git://git.kernel.org/pub/scm/devel/pahole/pahole.git
>
> Mirror git repo:
>
> https://github.com/acmel/dwarves.git
>
> tarball + gpg signature:
>
> https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.xz
> https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.bz2
> https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.sign
>

FYI:
Debian now ships dwarves package version 1.20-1 in unstable.

Just a small nit to this release and its tagging:

You did:
commit 0d415f68c468b77c5bf8e71965cd08c6efd25fc4 ("pahole: Prep 1.20")

Is this new?

The release before:
commit dd15aa4b0a6421295cbb7c3913429142fef8abe0 ("dwarves: Prep v1.19")

- Sedat -

> Best Regards,
>
> - Arnaldo
>
> v1.20:
>
> BTF encoder:
>
> - Improve ELF error reporting using elf_errmsg(elf_errno()).
>
> - Improve objcopy error handling.
>
> - Fix handling of 'restrict' qualifier, that was being treated as a 'const'.
>
> - Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with
> more than 65534 sections, for instance, which happens with kernels built
> with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may
> include when using FG-ASLR, LTO.
>
> - Cope with functions without a name, as seen sometimes when building kernel
> images with some versions of clang, when a SEGFAULT was taking place.
>
> - Fix BTF variable generation for kernel modules, not skipping variables at
> offset zero.
>
> - Fix address size to match what is in the ELF file being processed, to fix using
> a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image.
>
> - Use kernel module ftrace addresses when finding which functions to encode,
> which increases the number of functions encoded.
>
> libbpf:
>
> - Allow use of packaged version, for distros wanting to dynamically link with
> the system's libbpf package instead of using the libbpf git submodule shipped
> in pahole's source code.
>
> DWARF loader:
>
> - Support DW_AT_data_bit_offset
>
> This appeared in DWARF4 but is supported only in gcc's -gdwarf-5,
> support it in a way that makes the output be the same for both cases.
>
> $ gcc -gdwarf-5 -c examples/dwarf5/bf.c
> $ pahole bf.o
> struct pea {
> long int a:1; /* 0: 0 8 */
> long int b:1; /* 0: 1 8 */
> long int c:1; /* 0: 2 8 */
>
> /* XXX 29 bits hole, try to pack */
> /* Bitfield combined with next fields */
>
> int after_bitfield; /* 4 4 */
>
> /* size: 8, cachelines: 1, members: 4 */
> /* sum members: 4 */
> /* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
> /* last cacheline: 8 bytes */
> };
>
> - DW_FORM_implicit_const in attr_numeric() and attr_offset()
>
> - Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported
> DW_TAG_GNU_call_site.
>
> build:
>
> - Fix compilation on 32-bit architectures.
>
> Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>

2021-02-08 12:40:21

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

Em Mon, Feb 08, 2021 at 03:44:54AM +0100, Sedat Dilek escreveu:
> On Thu, Feb 4, 2021 at 11:07 PM Arnaldo Carvalho de Melo
> <[email protected]> wrote:
> >
> > Hi,
> >
> > The v1.20 release of pahole and its friends is out, mostly
> > addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> > available at the usual places:
> >
> > Main git repo:
> >
> > git://git.kernel.org/pub/scm/devel/pahole/pahole.git
> >
> > Mirror git repo:
> >
> > https://github.com/acmel/dwarves.git
> >
> > tarball + gpg signature:
> >
> > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.xz
> > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.bz2
> > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.sign
> >
>
> FYI:
> Debian now ships dwarves package version 1.20-1 in unstable.
>
> Just a small nit to this release and its tagging:
>
> You did:
> commit 0d415f68c468b77c5bf8e71965cd08c6efd25fc4 ("pahole: Prep 1.20")
>
> Is this new?
>
> The release before:
> commit dd15aa4b0a6421295cbb7c3913429142fef8abe0 ("dwarves: Prep v1.19")

Its minor but intentional, pahole is by far the most well known tool in
dwarves, so using that name more frequently (the git repo is pahole.git
, for instance) may help more quickly associate with the tool needed for
BTF encoding, data analysis, etc. And since its not about only DWARF,
perhaps transitioning to using 'pahole' more widely is interesting.

- Arnaldo

> - Sedat -
>
> > Best Regards,
> >
> > - Arnaldo
> >
> > v1.20:
> >
> > BTF encoder:
> >
> > - Improve ELF error reporting using elf_errmsg(elf_errno()).
> >
> > - Improve objcopy error handling.
> >
> > - Fix handling of 'restrict' qualifier, that was being treated as a 'const'.
> >
> > - Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with
> > more than 65534 sections, for instance, which happens with kernels built
> > with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may
> > include when using FG-ASLR, LTO.
> >
> > - Cope with functions without a name, as seen sometimes when building kernel
> > images with some versions of clang, when a SEGFAULT was taking place.
> >
> > - Fix BTF variable generation for kernel modules, not skipping variables at
> > offset zero.
> >
> > - Fix address size to match what is in the ELF file being processed, to fix using
> > a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image.
> >
> > - Use kernel module ftrace addresses when finding which functions to encode,
> > which increases the number of functions encoded.
> >
> > libbpf:
> >
> > - Allow use of packaged version, for distros wanting to dynamically link with
> > the system's libbpf package instead of using the libbpf git submodule shipped
> > in pahole's source code.
> >
> > DWARF loader:
> >
> > - Support DW_AT_data_bit_offset
> >
> > This appeared in DWARF4 but is supported only in gcc's -gdwarf-5,
> > support it in a way that makes the output be the same for both cases.
> >
> > $ gcc -gdwarf-5 -c examples/dwarf5/bf.c
> > $ pahole bf.o
> > struct pea {
> > long int a:1; /* 0: 0 8 */
> > long int b:1; /* 0: 1 8 */
> > long int c:1; /* 0: 2 8 */
> >
> > /* XXX 29 bits hole, try to pack */
> > /* Bitfield combined with next fields */
> >
> > int after_bitfield; /* 4 4 */
> >
> > /* size: 8, cachelines: 1, members: 4 */
> > /* sum members: 4 */
> > /* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
> > /* last cacheline: 8 bytes */
> > };
> >
> > - DW_FORM_implicit_const in attr_numeric() and attr_offset()
> >
> > - Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported
> > DW_TAG_GNU_call_site.
> >
> > build:
> >
> > - Fix compilation on 32-bit architectures.
> >
> > Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>

--

- Arnaldo

2021-02-08 12:41:25

by Sedat Dilek

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

On Mon, Feb 8, 2021 at 1:32 PM Arnaldo Carvalho de Melo
<[email protected]> wrote:
>
> Em Mon, Feb 08, 2021 at 03:44:54AM +0100, Sedat Dilek escreveu:
> > On Thu, Feb 4, 2021 at 11:07 PM Arnaldo Carvalho de Melo
> > <[email protected]> wrote:
> > >
> > > Hi,
> > >
> > > The v1.20 release of pahole and its friends is out, mostly
> > > addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> > > available at the usual places:
> > >
> > > Main git repo:
> > >
> > > git://git.kernel.org/pub/scm/devel/pahole/pahole.git
> > >
> > > Mirror git repo:
> > >
> > > https://github.com/acmel/dwarves.git
> > >
> > > tarball + gpg signature:
> > >
> > > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.xz
> > > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.bz2
> > > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.sign
> > >
> >
> > FYI:
> > Debian now ships dwarves package version 1.20-1 in unstable.
> >
> > Just a small nit to this release and its tagging:
> >
> > You did:
> > commit 0d415f68c468b77c5bf8e71965cd08c6efd25fc4 ("pahole: Prep 1.20")
> >
> > Is this new?
> >
> > The release before:
> > commit dd15aa4b0a6421295cbb7c3913429142fef8abe0 ("dwarves: Prep v1.19")
>
> Its minor but intentional, pahole is by far the most well known tool in
> dwarves, so using that name more frequently (the git repo is pahole.git
> , for instance) may help more quickly associate with the tool needed for
> BTF encoding, data analysis, etc. And since its not about only DWARF,
> perhaps transitioning to using 'pahole' more widely is interesting.
>

I am fine with that, Arnaldo.
The Git tree is called "pahole" Git, so that makes sense to me.

- Sedat -

> - Arnaldo
>
> > - Sedat -
> >
> > > Best Regards,
> > >
> > > - Arnaldo
> > >
> > > v1.20:
> > >
> > > BTF encoder:
> > >
> > > - Improve ELF error reporting using elf_errmsg(elf_errno()).
> > >
> > > - Improve objcopy error handling.
> > >
> > > - Fix handling of 'restrict' qualifier, that was being treated as a 'const'.
> > >
> > > - Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with
> > > more than 65534 sections, for instance, which happens with kernels built
> > > with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may
> > > include when using FG-ASLR, LTO.
> > >
> > > - Cope with functions without a name, as seen sometimes when building kernel
> > > images with some versions of clang, when a SEGFAULT was taking place.
> > >
> > > - Fix BTF variable generation for kernel modules, not skipping variables at
> > > offset zero.
> > >
> > > - Fix address size to match what is in the ELF file being processed, to fix using
> > > a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image.
> > >
> > > - Use kernel module ftrace addresses when finding which functions to encode,
> > > which increases the number of functions encoded.
> > >
> > > libbpf:
> > >
> > > - Allow use of packaged version, for distros wanting to dynamically link with
> > > the system's libbpf package instead of using the libbpf git submodule shipped
> > > in pahole's source code.
> > >
> > > DWARF loader:
> > >
> > > - Support DW_AT_data_bit_offset
> > >
> > > This appeared in DWARF4 but is supported only in gcc's -gdwarf-5,
> > > support it in a way that makes the output be the same for both cases.
> > >
> > > $ gcc -gdwarf-5 -c examples/dwarf5/bf.c
> > > $ pahole bf.o
> > > struct pea {
> > > long int a:1; /* 0: 0 8 */
> > > long int b:1; /* 0: 1 8 */
> > > long int c:1; /* 0: 2 8 */
> > >
> > > /* XXX 29 bits hole, try to pack */
> > > /* Bitfield combined with next fields */
> > >
> > > int after_bitfield; /* 4 4 */
> > >
> > > /* size: 8, cachelines: 1, members: 4 */
> > > /* sum members: 4 */
> > > /* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
> > > /* last cacheline: 8 bytes */
> > > };
> > >
> > > - DW_FORM_implicit_const in attr_numeric() and attr_offset()
> > >
> > > - Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported
> > > DW_TAG_GNU_call_site.
> > >
> > > build:
> > >
> > > - Fix compilation on 32-bit architectures.
> > >
> > > Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
>
> --
>
> - Arnaldo

2021-02-17 12:12:16

by Domenico Andreoli

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

On Mon, Feb 08, 2021 at 09:32:53AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Mon, Feb 08, 2021 at 03:44:54AM +0100, Sedat Dilek escreveu:
> > On Thu, Feb 4, 2021 at 11:07 PM Arnaldo Carvalho de Melo
> > <[email protected]> wrote:
> > >
> > > Hi,
> > >
> > > The v1.20 release of pahole and its friends is out, mostly
> > > addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> > > available at the usual places:
> > >
> > > Main git repo:
> > >
> > > git://git.kernel.org/pub/scm/devel/pahole/pahole.git
> > >
> > > Mirror git repo:
> > >
> > > https://github.com/acmel/dwarves.git
> > >
> > > tarball + gpg signature:
> > >
> > > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.xz
> > > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.bz2
> > > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.sign
> > >
> >
> > FYI:
> > Debian now ships dwarves package version 1.20-1 in unstable.
> >
> > Just a small nit to this release and its tagging:
> >
> > You did:
> > commit 0d415f68c468b77c5bf8e71965cd08c6efd25fc4 ("pahole: Prep 1.20")
> >
> > Is this new?
> >
> > The release before:
> > commit dd15aa4b0a6421295cbb7c3913429142fef8abe0 ("dwarves: Prep v1.19")
>
> Its minor but intentional, pahole is by far the most well known tool in
> dwarves, so using that name more frequently (the git repo is pahole.git
> , for instance) may help more quickly associate with the tool needed for
> BTF encoding, data analysis, etc. And since its not about only DWARF,
> perhaps transitioning to using 'pahole' more widely is interesting.

Any plan to switch also the release tarball name?

We are planning to rename the Debian package once the Bullseye is
released, currently it's dwarves-dfsg for legacy/unclear reasons.

Would it be a good idea to switch directly to pahole then?

Dom

>
> - Arnaldo
>
> > - Sedat -
> >
> > > Best Regards,
> > >
> > > - Arnaldo
> > >
> > > v1.20:
> > >
> > > BTF encoder:
> > >
> > > - Improve ELF error reporting using elf_errmsg(elf_errno()).
> > >
> > > - Improve objcopy error handling.
> > >
> > > - Fix handling of 'restrict' qualifier, that was being treated as a 'const'.
> > >
> > > - Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with
> > > more than 65534 sections, for instance, which happens with kernels built
> > > with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may
> > > include when using FG-ASLR, LTO.
> > >
> > > - Cope with functions without a name, as seen sometimes when building kernel
> > > images with some versions of clang, when a SEGFAULT was taking place.
> > >
> > > - Fix BTF variable generation for kernel modules, not skipping variables at
> > > offset zero.
> > >
> > > - Fix address size to match what is in the ELF file being processed, to fix using
> > > a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image.
> > >
> > > - Use kernel module ftrace addresses when finding which functions to encode,
> > > which increases the number of functions encoded.
> > >
> > > libbpf:
> > >
> > > - Allow use of packaged version, for distros wanting to dynamically link with
> > > the system's libbpf package instead of using the libbpf git submodule shipped
> > > in pahole's source code.
> > >
> > > DWARF loader:
> > >
> > > - Support DW_AT_data_bit_offset
> > >
> > > This appeared in DWARF4 but is supported only in gcc's -gdwarf-5,
> > > support it in a way that makes the output be the same for both cases.
> > >
> > > $ gcc -gdwarf-5 -c examples/dwarf5/bf.c
> > > $ pahole bf.o
> > > struct pea {
> > > long int a:1; /* 0: 0 8 */
> > > long int b:1; /* 0: 1 8 */
> > > long int c:1; /* 0: 2 8 */
> > >
> > > /* XXX 29 bits hole, try to pack */
> > > /* Bitfield combined with next fields */
> > >
> > > int after_bitfield; /* 4 4 */
> > >
> > > /* size: 8, cachelines: 1, members: 4 */
> > > /* sum members: 4 */
> > > /* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
> > > /* last cacheline: 8 bytes */
> > > };
> > >
> > > - DW_FORM_implicit_const in attr_numeric() and attr_offset()
> > >
> > > - Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported
> > > DW_TAG_GNU_call_site.
> > >
> > > build:
> > >
> > > - Fix compilation on 32-bit architectures.
> > >
> > > Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
>
> --
>
> - Arnaldo

--
rsa4096: 3B10 0CA1 8674 ACBA B4FE FCD2 CE5B CF17 9960 DE13
ed25519: FFB4 0CC3 7F2E 091D F7DA 356E CC79 2832 ED38 CB05


Attachments:
(No filename) (4.92 kB)
signature.asc (849.00 B)
Download all attachments

2021-02-17 12:51:54

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF)

Em Wed, Feb 17, 2021 at 01:08:23PM +0100, Domenico Andreoli escreveu:
> On Mon, Feb 08, 2021 at 09:32:53AM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Feb 08, 2021 at 03:44:54AM +0100, Sedat Dilek escreveu:
> > > On Thu, Feb 4, 2021 at 11:07 PM Arnaldo Carvalho de Melo
> > > <[email protected]> wrote:
> > > >
> > > > Hi,
> > > >
> > > > The v1.20 release of pahole and its friends is out, mostly
> > > > addressing problems related to gcc 11 defaulting to DWARF5 for -g,
> > > > available at the usual places:
> > > >
> > > > Main git repo:
> > > >
> > > > git://git.kernel.org/pub/scm/devel/pahole/pahole.git
> > > >
> > > > Mirror git repo:
> > > >
> > > > https://github.com/acmel/dwarves.git
> > > >
> > > > tarball + gpg signature:
> > > >
> > > > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.xz
> > > > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.bz2
> > > > https://fedorapeople.org/~acme/dwarves/dwarves-1.20.tar.sign
> > > >
> > >
> > > FYI:
> > > Debian now ships dwarves package version 1.20-1 in unstable.
> > >
> > > Just a small nit to this release and its tagging:
> > >
> > > You did:
> > > commit 0d415f68c468b77c5bf8e71965cd08c6efd25fc4 ("pahole: Prep 1.20")
> > >
> > > Is this new?
> > >
> > > The release before:
> > > commit dd15aa4b0a6421295cbb7c3913429142fef8abe0 ("dwarves: Prep v1.19")
> >
> > Its minor but intentional, pahole is by far the most well known tool in
> > dwarves, so using that name more frequently (the git repo is pahole.git
> > , for instance) may help more quickly associate with the tool needed for
> > BTF encoding, data analysis, etc. And since its not about only DWARF,
> > perhaps transitioning to using 'pahole' more widely is interesting.

> Any plan to switch also the release tarball name?

> We are planning to rename the Debian package once the Bullseye is
> released, currently it's dwarves-dfsg for legacy/unclear reasons.

> Would it be a good idea to switch directly to pahole then?

Yeah, I think it is, I'll check what are the bureaucratic steps to do
that rename on Fedora and RHEL, but then, no need for all distros to do
it at once if that is something that requires long term planning or
whatever.

- Arnaldo

> Dom
>
> >
> > - Arnaldo
> >
> > > - Sedat -
> > >
> > > > Best Regards,
> > > >
> > > > - Arnaldo
> > > >
> > > > v1.20:
> > > >
> > > > BTF encoder:
> > > >
> > > > - Improve ELF error reporting using elf_errmsg(elf_errno()).
> > > >
> > > > - Improve objcopy error handling.
> > > >
> > > > - Fix handling of 'restrict' qualifier, that was being treated as a 'const'.
> > > >
> > > > - Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with
> > > > more than 65534 sections, for instance, which happens with kernels built
> > > > with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may
> > > > include when using FG-ASLR, LTO.
> > > >
> > > > - Cope with functions without a name, as seen sometimes when building kernel
> > > > images with some versions of clang, when a SEGFAULT was taking place.
> > > >
> > > > - Fix BTF variable generation for kernel modules, not skipping variables at
> > > > offset zero.
> > > >
> > > > - Fix address size to match what is in the ELF file being processed, to fix using
> > > > a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image.
> > > >
> > > > - Use kernel module ftrace addresses when finding which functions to encode,
> > > > which increases the number of functions encoded.
> > > >
> > > > libbpf:
> > > >
> > > > - Allow use of packaged version, for distros wanting to dynamically link with
> > > > the system's libbpf package instead of using the libbpf git submodule shipped
> > > > in pahole's source code.
> > > >
> > > > DWARF loader:
> > > >
> > > > - Support DW_AT_data_bit_offset
> > > >
> > > > This appeared in DWARF4 but is supported only in gcc's -gdwarf-5,
> > > > support it in a way that makes the output be the same for both cases.
> > > >
> > > > $ gcc -gdwarf-5 -c examples/dwarf5/bf.c
> > > > $ pahole bf.o
> > > > struct pea {
> > > > long int a:1; /* 0: 0 8 */
> > > > long int b:1; /* 0: 1 8 */
> > > > long int c:1; /* 0: 2 8 */
> > > >
> > > > /* XXX 29 bits hole, try to pack */
> > > > /* Bitfield combined with next fields */
> > > >
> > > > int after_bitfield; /* 4 4 */
> > > >
> > > > /* size: 8, cachelines: 1, members: 4 */
> > > > /* sum members: 4 */
> > > > /* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
> > > > /* last cacheline: 8 bytes */
> > > > };
> > > >
> > > > - DW_FORM_implicit_const in attr_numeric() and attr_offset()
> > > >
> > > > - Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported
> > > > DW_TAG_GNU_call_site.
> > > >
> > > > build:
> > > >
> > > > - Fix compilation on 32-bit architectures.
> > > >
> > > > Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
> >
> > --
> >
> > - Arnaldo
>
> --
> rsa4096: 3B10 0CA1 8674 ACBA B4FE FCD2 CE5B CF17 9960 DE13
> ed25519: FFB4 0CC3 7F2E 091D F7DA 356E CC79 2832 ED38 CB05



--

- Arnaldo