by Sedat Dilek

[permalink] [raw]

Subject: Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

On Fri, Feb 5, 2021 at 8:30 PM Fāng-ruì Sòng <[email protected]> wrote:
>
> On Fri, Feb 5, 2021 at 11:21 AM Sedat Dilek <[email protected]> wrote:
> >
> > On Fri, Feb 5, 2021 at 8:15 PM Sedat Dilek <[email protected]> wrote:
> > >
> > > On Fri, Feb 5, 2021 at 8:10 PM Yonghong Song <[email protected]> wrote:
> > > >
> > > >
> > > >
> > > > On 2/5/21 11:06 AM, Sedat Dilek wrote:
> > > > > On Fri, Feb 5, 2021 at 7:53 PM Sedat Dilek <[email protected]> wrote:
> > > > >>
> > > > >> On Fri, Feb 5, 2021 at 6:48 PM Sedat Dilek <[email protected]> wrote:
> > > > >>>
> > > > >>> On Fri, Feb 5, 2021 at 4:28 PM Arnaldo Carvalho de Melo
> > > > >>> <[email protected]> wrote:
> > > > >>>>
> > > > >>>> Em Fri, Feb 05, 2021 at 04:23:59PM +0100, Sedat Dilek escreveu:
> > > > >>>>> On Fri, Feb 5, 2021 at 3:41 PM Sedat Dilek <[email protected]> wrote:
> > > > >>>>>>
> > > > >>>>>> On Fri, Feb 5, 2021 at 3:37 PM Sedat Dilek <[email protected]> wrote:
> > > > >>>>>>>
> > > > >>>>>>> Hi,
> > > > >>>>>>>
> > > > >>>>>>> when building with pahole v1.20 and binutils v2.35.2 plus Clang
> > > > >>>>>>> v12.0.0-rc1 and DWARF-v5 I see:
> > > > >>>>>>> ...
> > > > >>>>>>> + info BTF .btf.vmlinux.bin.o
> > > > >>>>>>> + [ != silent_ ]
> > > > >>>>>>> + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> > > > >>>>>>> BTF .btf.vmlinux.bin.o
> > > > >>>>>>> + LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> > > > >>>>>>> .tmp_vmlinux.btf
> > > > >>>>>>> [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> > > > >>>>>>> Encountered error while encoding BTF.
> > > > >>>>>>
> > > > >>>>>> Grepping the pahole sources:
> > > > >>>>>>
> > > > >>>>>> $ git grep DW_ATE
> > > > >>>>>> dwarf_loader.c: bt->is_bool = encoding == DW_ATE_boolean;
> > > > >>>>>> dwarf_loader.c: bt->is_signed = encoding == DW_ATE_signed;
> > > > >>>>>>
> > > > >>>>>> Missing DW_ATE_unsigned encoding?
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>> Checked the LLVM sources:
> > > > >>>>>
> > > > >>>>> clang/lib/CodeGen/CGDebugInfo.cpp: Encoding =
> > > > >>>>> llvm::dwarf::DW_ATE_unsigned_char;
> > > > >>>>> clang/lib/CodeGen/CGDebugInfo.cpp: Encoding = llvm::dwarf::DW_ATE_unsigned;
> > > > >>>>> clang/lib/CodeGen/CGDebugInfo.cpp: Encoding =
> > > > >>>>> llvm::dwarf::DW_ATE_unsigned_fixed;
> > > > >>>>> clang/lib/CodeGen/CGDebugInfo.cpp:
> > > > >>>>> ? llvm::dwarf::DW_ATE_unsigned
> > > > >>>>> ...
> > > > >>>>> lld/test/wasm/debuginfo.test:CHECK-NEXT: DW_AT_encoding
> > > > >>>>> (DW_ATE_unsigned)
> > > > >>>>>
> > > > >>>>> So, I will switch from GNU ld.bfd v2.35.2 to LLD-12.
> > > > >>>>
> > > > >>>> Thanks for the research, probably your conclusion is correct, can you go
> > > > >>>> the next step and add that part and check if the end result is the
> > > > >>>> expected one?
> > > > >>>>
> > > > >>>
> > > > >>> Still building...
> > > > >>>
> > > > >>> Can you give me a hand on what has to be changed in dwarves/pahole?
> > > > >>>
> > > > >>> I guess switching from ld.bfd to ld.lld will show the same ERROR.
> > > > >>>
> > > > >>
> > > > >> This builds successfully - untested:
> > > > >>
> > > > >> $ git diff
> > > > >> diff --git a/btf_loader.c b/btf_loader.c
> > > > >> index ec286f413f36..a39edd3362db 100644
> > > > >> --- a/btf_loader.c
> > > > >> +++ b/btf_loader.c
> > > > >> @@ -107,6 +107,7 @@ static struct base_type *base_type__new(strings_t
> > > > >> name, uint32_t attrs,
> > > > >> bt->bit_size = size;
> > > > >> bt->is_signed = attrs & BTF_INT_SIGNED;
> > > > >> bt->is_bool = attrs & BTF_INT_BOOL;
> > > > >> + bt->is_unsigned = attrs & BTF_INT_UNSIGNED;
> > > > >> bt->name_has_encoding = false;
> > > > >> bt->float_type = float_type;
> > > > >> }
> > > > >> diff --git a/ctf.h b/ctf.h
> > > > >> index 25b79892bde3..9e47c3c74677 100644
> > > > >> --- a/ctf.h
> > > > >> +++ b/ctf.h
> > > > >> @@ -100,6 +100,7 @@ struct ctf_full_type {
> > > > >> #define CTF_TYPE_INT_CHAR 0x2
> > > > >> #define CTF_TYPE_INT_BOOL 0x4
> > > > >> #define CTF_TYPE_INT_VARARGS 0x8
> > > > >> +#define CTF_TYPE_INT_UNSIGNED 0x16
> > > > >>
> > > > >> #define CTF_TYPE_FP_ATTRS(VAL) ((VAL) >> 24)
> > > > >> #define CTF_TYPE_FP_OFFSET(VAL) (((VAL) >> 16) & 0xff)
> > > > >> diff --git a/dwarf_loader.c b/dwarf_loader.c
> > > > >> index b73d7867e1e6..79d40f183c24 100644
> > > > >> --- a/dwarf_loader.c
> > > > >> +++ b/dwarf_loader.c
> > > > >> @@ -473,6 +473,7 @@ static struct base_type *base_type__new(Dwarf_Die
> > > > >> *die, struct cu *cu)
> > > > >> bt->is_bool = encoding == DW_ATE_boolean;
> > > > >> bt->is_signed = encoding == DW_ATE_signed;
> > > > >> bt->is_varargs = false;
> > > > >> + bt->is_unsigned = encoding == DW_ATE_unsigned;
> > > > >> bt->name_has_encoding = true;
> > > > >> }
> > > > >>
> > > > >> diff --git a/dwarves.h b/dwarves.h
> > > > >> index 98caf1abc54d..edf32d2e6f80 100644
> > > > >> --- a/dwarves.h
> > > > >> +++ b/dwarves.h
> > > > >> @@ -1261,6 +1261,7 @@ struct base_type {
> > > > >> uint8_t is_signed:1;
> > > > >> uint8_t is_bool:1;
> > > > >> uint8_t is_varargs:1;
> > > > >> + uint8_t is_unsigned:1;
> > > > >> uint8_t float_type:4;
> > > > >> };
> > > > >>
> > > > >> diff --git a/lib/bpf b/lib/bpf
> > > > >> --- a/lib/bpf
> > > > >> +++ b/lib/bpf
> > > > >> @@ -1 +1 @@
> > > > >> -Subproject commit 5af3d86b5a2c5fecdc3ab83822d083edd32b4396
> > > > >> +Subproject commit 5af3d86b5a2c5fecdc3ab83822d083edd32b4396-dirty
> > > > >> diff --git a/libbtf.c b/libbtf.c
> > > > >> index 9f7628304495..a0661a7bbed9 100644
> > > > >> --- a/libbtf.c
> > > > >> +++ b/libbtf.c
> > > > >> @@ -247,6 +247,8 @@ static const char *
> > > > >> btf_elf__int_encoding_str(uint8_t encoding)
> > > > >> return "CHAR";
> > > > >> else if (encoding == BTF_INT_BOOL)
> > > > >> return "BOOL";
> > > > >> + else if (encoding == BTF_INT_UNSIGNED)
> > > > >> + return "UNSIGNED";
> > > > >> else
> > > > >> return "UNKN";
> > > > >> }
> > > > >> @@ -379,6 +381,8 @@ int32_t btf_elf__add_base_type(struct btf_elf
> > > > >> *btfe, const struct base_type *bt,
> > > > >> encoding = BTF_INT_SIGNED;
> > > > >> } else if (bt->is_bool) {
> > > > >> encoding = BTF_INT_BOOL;
> > > > >> + } else if (bt->is_unsigned) {
> > > > >> + encoding = BTF_INT_UNSIGNED;
> > > > >> } else if (bt->float_type) {
> > > > >> fprintf(stderr, "float_type is not supported\n");
> > > > >> return -1;
> > > > >>
> > > > >> Additionally - I cannot see it with `git diff`:
> > > > >>
> > > > >> [ lib/bpf/include/uapi/linux/btf.h ]
> > > > >>
> > > > >> /* Attributes stored in the BTF_INT_ENCODING */
> > > > >> #define BTF_INT_SIGNED (1 << 0)
> > > > >> #define BTF_INT_CHAR (1 << 1)
> > > > >> #define BTF_INT_BOOL (1 << 2)
> > > > >> #define BTF_INT_UNSIGNED (1 << 3)
> > > > >>
> > > > >> Comments?
> > > > >>
> > > > >
> > > > > Hmmm...
> > > > >
> > > > > + info BTF .btf.vmlinux.bin.o
> > > > > + [ != silent_ ]
> > > > > + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> > > > > BTF .btf.vmlinux.bin.o
> > > > > + LLVM_OBJCOPY=llvm-objcopy /opt/pahole/bin/pahole -J .tmp_vmlinux.btf
> > > > > [2] INT long unsigned int Error emitting BTF type
> > > > > Encountered error while encoding BTF.
> > > > > + llvm-objcopy --only-section=.BTF --set-section-flags
> > > > > .BTF=alloc,readonly --strip-all .tmp_vmlinux.btf .btf.vmlinux.bin.o
> > > > > ...
> > > > > + info BTFIDS vmlinux
> > > > > + [ != silent_ ]
> > > > > + printf %-7s %s\n BTFIDS vmlinux
> > > > > BTFIDS vmlinux
> > > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > > FAILED: load BTF from vmlinux: Invalid argument
> > > > > + on_exit
> > > > > + [ 255 -ne 0 ]
> > > > > + cleanup
> > > > > + rm -f .btf.vmlinux.bin.o
> > > > > + rm -f .tmp_System.map
> > > > > + rm -f .tmp_vmlinux.btf .tmp_vmlinux.kallsyms1
> > > > > .tmp_vmlinux.kallsyms1.S .tmp_vmlinux.kallsyms1.o
> > > > > .tmp_vmlinux.kallsyms2 .tmp_vmlinux.kallsyms2.S .tmp_vmlinux.kallsyms
> > > > > 2.o
> > > > > + rm -f System.map
> > > > > + rm -f vmlinux
> > > > > + rm -f vmlinux.o
> > > > > make[3]: *** [Makefile:1166: vmlinux] Error 255
> > > > >
> > > > > Grepping through linux.git/tools I guess some BTF tools/libs need to
> > > > > know what BTF_INT_UNSIGNED is?
> > > >
> > > > BTF_INT_UNSIGNED needs kernel support. Maybe to teach pahole to
> > > > ignore this for now until kernel infrastructure is ready.
> > > > Not sure whether this information will be useful or not
> > > > for BTF. This needs to be discussed separately.
> > > >
> > >
> > > [ CC Fangrui ]
> > >
> > > How can I teach pahole to ignore BTF_INT_UNSIGNED?
> > >
> > > Another tryout might be to use "-fbinutils-version=..." which is
> > > available for LLVM-12 according to Fangrui?
> > > Fangrui, which binutils versions can I pass and how?
> > >
> >
> > OK, I checked LLVM-12 sources:
> >
> > clang/docs/ReleaseNotes.rst:101:- New option ``-fbinutils-version=``
> > specifies the targeted binutils version.
> > clang/docs/ReleaseNotes.rst:102: For example,
> > ``-fbinutils-version=2.35`` means compatibility with GNU as/ld
> > clang/docs/ReleaseNotes.rst-103- before 2.35 is not needed: new
> > features can be used and there is no need to
> > clang/docs/ReleaseNotes.rst-104- work around old GNU as/ld bugs.
> >
> > Can I pass (also patchlevel) like 2.35.2?
> > Here I have Debian's v2.35.1 and a selfmade v2.35.2?
> >
> > - Sedat -
> >
> > - Sedat -
>
> Answering specifically this question:
>
> clang -help displays:
> ...
> -fbinutils-version=<major.minor>
> Produced object files can use all ELF
> features supported by this binutils version and newer. If
> -fno-integrated-as is specified, the generated assembly will consider
> GNU as support. 'none' means that all ELF features can be used,
> regardless of binutils support. Defaults to 2.26.
>
> The option was introduced in https://reviews.llvm.org/D85474
> major.minor.patch is not supported. In reality, very few features are
> gated by this option, currently just SHF_MERGE and a pending
> SHF_LINK_ORDER PGO patch.
> I think we will be conservative. If a 2.37 fix is back ported to
> 2.35.2 and 2.36.1, we will ignore that and will use the feature only
> if -fbinutils-version=2.37 or above is specified.
>

Binutils v2.35.2 has DWARF-5 fixes which are not in vanilla v2.36 - I
might be wrong, please correct me.

So what shall I pass now to -fbinutils-version= ?

- Sedat -

2021-02-05 19:50:15

On Sat, Feb 6, 2021 at 9:27 AM Yonghong Song <[email protected]> wrote:
>
>
>
> On 2/5/21 10:52 PM, Sedat Dilek wrote:
> > On Sat, Feb 6, 2021 at 7:26 AM Sedat Dilek <[email protected]> wrote:
> >>
> >> On Sat, Feb 6, 2021 at 6:53 AM Sedat Dilek <[email protected]> wrote:
> >>>
> >>> On Sat, Feb 6, 2021 at 6:44 AM Sedat Dilek <[email protected]> wrote:
> >>>>
> >>>> On Sat, Feb 6, 2021 at 4:34 AM Sedat Dilek <[email protected]> wrote:
> >>>>>
> >>>>> On Fri, Feb 5, 2021 at 10:54 PM Yonghong Song <[email protected]> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On 2/5/21 12:31 PM, Sedat Dilek wrote:
> >>>>>>> On Fri, Feb 5, 2021 at 9:03 PM Yonghong Song <[email protected]> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 2/5/21 11:24 AM, Arnaldo Carvalho de Melo wrote:
> >>>>>>>>> Em Fri, Feb 05, 2021 at 11:10:08AM -0800, Yonghong Song escreveu:
> >>>>>>>>>> On 2/5/21 11:06 AM, Sedat Dilek wrote:
> >>>>>>>>>>> On Fri, Feb 5, 2021 at 7:53 PM Sedat Dilek <[email protected]> wrote:
> >>>>>>>>>>> Grepping through linux.git/tools I guess some BTF tools/libs need to
> >>>>>>>>>>> know what BTF_INT_UNSIGNED is?
> >>>>>>>>>
> >>>>>>>>>> BTF_INT_UNSIGNED needs kernel support. Maybe to teach pahole to
> >>>>>>>>>> ignore this for now until kernel infrastructure is ready.
> >>>>>>>>>
> >>>>>>>>> Yeah, I thought about doing that.
> >>>>>>>>>
> >>>>>>>>>> Not sure whether this information will be useful or not
> >>>>>>>>>> for BTF. This needs to be discussed separately.
> >>>>>>>>>
> >>>>>>>>> Maybe search for the rationale for its introduction in DWARF.
> >>>>>>>>
> >>>>>>>> In LLVM, we have:
> >>>>>>>> uint8_t BTFEncoding;
> >>>>>>>> switch (Encoding) {
> >>>>>>>> case dwarf::DW_ATE_boolean:
> >>>>>>>> BTFEncoding = BTF::INT_BOOL;
> >>>>>>>> break;
> >>>>>>>> case dwarf::DW_ATE_signed:
> >>>>>>>> case dwarf::DW_ATE_signed_char:
> >>>>>>>> BTFEncoding = BTF::INT_SIGNED;
> >>>>>>>> break;
> >>>>>>>> case dwarf::DW_ATE_unsigned:
> >>>>>>>> case dwarf::DW_ATE_unsigned_char:
> >>>>>>>> BTFEncoding = 0;
> >>>>>>>> break;
> >>>>>>>>
> >>>>>>>> I think DW_ATE_unsigned can be ignored in pahole since
> >>>>>>>> the default encoding = 0. A simple comment is enough.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Yonghong Son, do you have a patch/diff for me?
> >>>>>>
> >>>>>> Looking at error message from log:
> >>>>>>
> >>>>>> LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> >>>>>> .tmp_vmlinux.btf
> >>>>>> [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> >>>>>> Encountered error while encoding BTF.
> >>>>>>
> >>>>>> Not exactly what is the root cause. Maybe bt->bit_size is not
> >>>>>> encoded correctly. Could you put vmlinux (in the above it is
> >>>>>> .tmp_vmlinux.btf) somewhere, I or somebody else can investigate
> >>>>>> and provide a proper fix.
> >>>>>>
> >>>>>
> >>>>> [ TO: Masahiro ]
> >>>>>
> >>>>> Thanks for taking care Yonghong - hope this is your first name, if not
> >>>>> I am sorry.
> >>>>> In case of mixing my first and last name you will make me female -
> >>>>> Dilek is a Turkish female first name :-).
> >>>>> So, in some cultures you need to be careful.
> >>>>>
> >>>>> Anyway... back to business and facts.
> >>>>>
> >>>>> Out of frustration I killed my last build via `make distclean`.
> >>>>> The whole day I tested diverse combination of GCC-10 and LLVM-12
> >>>>> together with BTF Kconfigs, selfmade pahole, etc.
> >>>>>
> >>>>> I will do ne run with some little changes:
> >>>>>
> >>>>> #1: Pass LLVM_IAS=1 to make (means use Clang's Integrated ASsembler -
> >>>>> as per Nick this leads to the same error - should be unrelated)
> >>>>> #2: I did: DEBUG_INFO_COMPRESSED y -> n
> >>>>>
> >>>>> #2 I did in case you need vmlinux and I have to upload - I will
> >>>>> compress the resulting vmlinux with ZSTD.
> >>>>> You need vmlinux or .tmp_vmlinux.btf file?
> >>>>> Nick was not allowed from his company to download from a Dropbox link.
> >>>>> So, as an alternative I can offer GoogleDrive...
> >>>>> ...or bomb into your INBOX :-).
> >>>>>
> >>>>> Now, why I CCed Masahiro:
> >>>>>
> >>>>> In case of ERRORs when running `scripts/link-vmlinux.sh` above files
> >>>>> will be removed.
> >>>>>
> >>>>> Last, I found a hack to bypass this - means to keep these files (I
> >>>>> need to check old emails).
> >>>>>
> >>>>> Masahiro, you see a possibility to have a way to keep these files in
> >>>>> case of ERRORs without doing hackery?
> >>>>>
> >>>>> From a previous post in this thread:
> >>>>>
> >>>>> + info BTF .btf.vmlinux.bin.o
> >>>>> + [ != silent_ ]
> >>>>> + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> >>>>> BTF .btf.vmlinux.bin.o
> >>>>> + LLVM_OBJCOPY=llvm-objcopy /opt/pahole/bin/pahole -J .tmp_vmlinux.btf
> >>>>> [2] INT long unsigned int Error emitting BTF type
> >>>>> Encountered error while encoding BTF.
> >>>>> + llvm-objcopy --only-section=.BTF --set-section-flags
> >>>>> .BTF=alloc,readonly --strip-all .tmp_vmlinux.btf .btf.vmlinux.bin.o
> >>>>> ...
> >>>>> + info BTFIDS vmlinux
> >>>>> + [ != silent_ ]
> >>>>> + printf %-7s %s\n BTFIDS vmlinux
> >>>>> BTFIDS vmlinux
> >>>>> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> >>>>> FAILED: load BTF from vmlinux: Invalid argument
> >>>>> + on_exit
> >>>>> + [ 255 -ne 0 ]
> >>>>> + cleanup
> >>>>> + rm -f .btf.vmlinux.bin.o
> >>>>> + rm -f .tmp_System.map
> >>>>> + rm -f .tmp_vmlinux.btf .tmp_vmlinux.kallsyms1
> >>>>> .tmp_vmlinux.kallsyms1.S .tmp_vmlinux.kallsyms1.o
> >>>>> .tmp_vmlinux.kallsyms2 .tmp_vmlinux.kallsyms2.S .tmp_vmlinux.kallsyms
> >>>>> 2.o
> >>>>> + rm -f System.map
> >>>>> + rm -f vmlinux
> >>>>> + rm -f vmlinux.o
> >>>>> make[3]: *** [Makefile:1166: vmlinux] Error 255
> >>>>>
> >>>>> ^^^ Look here.
> >>>>>
> >>>>
> >>>> With this diff:
> >>>>
> >>>> $ git diff scripts/link-vmlinux.sh
> >>>> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> >>>> index eef40fa9485d..40f1b6aae553 100755
> >>>> --- a/scripts/link-vmlinux.sh
> >>>> +++ b/scripts/link-vmlinux.sh
> >>>> @@ -330,7 +330,7 @@ vmlinux_link vmlinux "${kallsymso}" ${btf_vmlinux_bin_o}
> >>>> # fill in BTF IDs
> >>>> if [ -n "${CONFIG_DEBUG_INFO_BTF}" -a -n "${CONFIG_BPF}" ]; then
> >>>> info BTFIDS vmlinux
> >>>> - ${RESOLVE_BTFIDS} vmlinux
> >>>> + ##${RESOLVE_BTFIDS} vmlinux
> >>>> fi
> >>>>
> >>>> if [ -n "${CONFIG_BUILDTIME_TABLE_SORT}" ]; then
> >>>>
> >>>> This files are kept - not removed:
> >>>>
> >>>> $ LC_ALL=C ll .*btf* vmlinux vmlinux.o
> >>>> -rwxr-xr-x 1 dileks dileks 31M Feb 6 06:37 .btf.vmlinux.bin.o
> >>>> -rwxr-xr-x 1 dileks dileks 348M Feb 6 06:37 .tmp_vmlinux.btf
> >>>> -rwxr-xr-x 1 dileks dileks 348M Feb 6 06:37 vmlinux
> >>>> -rw-r--r-- 1 dileks dileks 344M Feb 6 06:37 vmlinux.o
> >>>>
> >>>> Pleas let me know where to upload - Dropbox or GoogleDrive or
> >>>> elsewhere and give me a link.
> >>>>
> >>>
> >>>
> >>> WOW, ZSTD is great :-).
> >>>
> >>> $ zstd -19 -T0 -v vmlinux
> >>> *** zstd command line interface 64-bits v1.4.8, by Yann Collet ***
> >>> Note: 2 physical core(s) detected
> >>> vmlinux : 22.71% (364466016 => 82784801 bytes, vmlinux.zst)
> >>>
> >>> $ du -m vmlinux*
> >>> 348 vmlinux
> >>> 79 vmlinux.zst
> >>>
> >>
> >> Dropbox link:
> >> https://www.dropbox.com/sh/kvyh8ps7na0r1h5/AABfyNfDZ2bESse_bo4h05fFa?dl=0
> >>
> >> I hope this is public available.
> >>
> >
> > Inspecting vmlinux with llvm-dwarf:
> >
> > $ /opt/llvm-toolchain/bin/llvm-dwarfdump vmlinux | grep DW_AT_name |
> > grep DW_ATE_ | sort | uniq
> > DW_AT_name ("DW_ATE_signed_1")
> > DW_AT_name ("DW_ATE_signed_16")
> > DW_AT_name ("DW_ATE_signed_32")
> > DW_AT_name ("DW_ATE_signed_64")
> > DW_AT_name ("DW_ATE_signed_8")
> > DW_AT_name ("DW_ATE_unsigned_1")
> > DW_AT_name ("DW_ATE_unsigned_128")
> > DW_AT_name ("DW_ATE_unsigned_16")
> > DW_AT_name ("DW_ATE_unsigned_24")
> > DW_AT_name ("DW_ATE_unsigned_32")
> > DW_AT_name ("DW_ATE_unsigned_40")
> > DW_AT_name ("DW_ATE_unsigned_64")
> > DW_AT_name ("DW_ATE_unsigned_8")
> >
> > - Sedat -
>
> Thanks for the above dropbot link, I am able to reproduce the issue.
>
> I tried to use latest llvm + Nick's patch + latest pahole + dwarf5
> config option to compile kernel with LLVM=1 LLVM_IAS=1, somehow
> I did not hit the issue. It complained like
>
> MODPOST vmlinux.symvers
> WARNING: modpost: vmlinux.o(.text+0x6ce73): Section mismatch in
> reference from the function __nodes
> _weight() to the variable .init.data:numa_nodes_parsed
> The function __nodes_weight() references
> the variable __initdata numa_nodes_parsed.
> This is often because __nodes_weight lacks a __initdata
> annotation or the annotation of numa_nodes_parsed is wrong.
>
> but otherwise compilation is fine.
>
> With the above vmlinux, the issue appears to be handling
> DW_ATE_signed_1, DW_ATE_unsigned_{1,24,40}.
>
> The following patch should fix the issue:
>
> -bash-4.4$ git diff
>
> diff --git a/dwarf_loader.c b/dwarf_loader.c
>
> index b73d786..0341b5e 100644
>
> --- a/dwarf_loader.c
>
> +++ b/dwarf_loader.c
>
> @@ -467,8 +467,16 @@ static struct base_type *base_type__new(Dwarf_Die
> *die, struct cu *cu)
>
>
> if (bt != NULL) {
>
> tag__init(&bt->tag, cu, die);
>
> - bt->name = strings__add(strings, attr_string(die,
> DW_AT_name));
> - bt->bit_size = attr_numeric(die, DW_AT_byte_size) * 8;
>
> + const char *name = attr_string(die, DW_AT_name);
>
> + bt->name = strings__add(strings, name);
>
> + /* DW_ATE_unsigned_1 has DW_AT_byte_size == 0.
>
> + * specially process it.
>
> + */
>
> + if (strcmp(name, "DW_ATE_unsigned_1") == 0)
>
> + bt->bit_size = 1;
>
> + else
>
> + bt->bit_size = attr_numeric(die,
> DW_AT_byte_size) * 8;
> +
>
> uint64_t encoding = attr_numeric(die, DW_AT_encoding);
> bt->is_bool = encoding == DW_ATE_boolean;
> bt->is_signed = encoding == DW_ATE_signed;
> diff --git a/libbtf.c b/libbtf.c
> index 9f76283..b5aa077 100644
> --- a/libbtf.c
> +++ b/libbtf.c
> @@ -367,13 +367,32 @@ static void btf_log_func_param(const struct
> btf_elf *btfe,
> }
> }
>
> +/* btf requires power-of-2 bytes, yet dwarf may have something like
> + * DW_ATE_unsigned_24 which encodes as 24 bits (3 bytes).
> + */
> +static int bits_to_int_bytes(uint16_t bit_size)
> +{
> + if (bit_size <= 8)
> + return 1;
> + if (bit_size <= 16)
> + return 2;
> + if (bit_size <= 32)
> + return 4;
> + if (bit_size <= 64)
>
> + return 8;
>
> + if (bit_size <= 128)
> + return 16;
> + /* BTF supports upto 16byte int (__int128). */
> + return -1;
> +}
> +
> int32_t btf_elf__add_base_type(struct btf_elf *btfe, const struct
> base_type *bt,
> const char *name)
> {
> struct btf *btf = btfe->btf;
> const struct btf_type *t;
> uint8_t encoding = 0;
> - int32_t id;
> + int32_t id, nbytes;
>
> if (bt->is_signed) {
> encoding = BTF_INT_SIGNED;
> @@ -384,7 +403,13 @@ int32_t btf_elf__add_base_type(struct btf_elf
> *btfe, const struct base_type *b
> t,
> return -1;
> }
> - id = btf__add_int(btf, name, BITS_ROUNDUP_BYTES(bt->bit_size),
> encoding);
> + nbytes = bits_to_int_bytes(bt->bit_size);
> + if (nbytes < 0) {
> + fprintf(stderr, "not supported bit_size %hu\n",
> bt->bit_size);
> + return -1;
> + }
> +
> + id = btf__add_int(btf, name, nbytes, encoding);
> if (id < 0) {
> btf_elf__log_err(btfe, BTF_KIND_INT, name, true, "Error
> emitting BTF type");
> } else {
> -bash-4.4$
>
> Please help do a test. I can submit a formal patch tomorrow.

Thanks for the patch.

Can you attach the diff as Gmail has totally truncated/malformed it?

For the Linux breakage - you will need some additional Clang specific patches.
Is this Linux 5.11-rcX?
The "Blocking bugs" are listed in the first post of "Linux 5.11 release cycle".
Hope this helps.

This is cool co-working :-).

- Sedat -

[1] https://github.com/ClangBuiltLinux/linux/issues/1228

2021-02-06 09:34:01

[ SNIP ]

> > > > > Please help do a test. I can submit a formal patch tomorrow.
> > > >
> > > > Thanks for the patch.
> > > >
> > > > Can you attach the diff as Gmail has totally truncated/malformed it?
> > > >
> > > > For the Linux breakage - you will need some additional Clang specific patches.
> > > > Is this Linux 5.11-rcX?
> > > > The "Blocking bugs" are listed in the first post of "Linux 5.11 release cycle".
> > > > Hope this helps.
> > > >
> > > > This is cool co-working :-).
> > > >
> > > > - Sedat -
> > > >
> > > > [1] https://github.com/ClangBuiltLinux/linux/issues/1228
> > >
> > > With the attached diff and new selfmade pahole looks good here.
> > >
> > > Passed (see line-numbers):
> > >
> > > 11090:+ info LD .tmp_vmlinux.btf
> > > 11099:+ info BTF .btf.vmlinux.bin.o
> > > 11103:+ LLVM_OBJCOPY=llvm-objcopy /opt/pahole/bin/pahole -J .tmp_vmlinux.btf
> > > 11121:+ info LD .tmp_vmlinux.kallsyms1
> > > 11139:+ info KSYMS .tmp_vmlinux.kallsyms1.S
> > > 11145:+ info AS .tmp_vmlinux.kallsyms1.S
> > > 11160:+ info LD .tmp_vmlinux.kallsyms2
> > > 11178:+ info KSYMS .tmp_vmlinux.kallsyms2.S
> > > 11184:+ info AS .tmp_vmlinux.kallsyms2.S
> > > 11200:+ info LD vmlinux
> > > 11210:+ info BTFIDS vmlinux
> > > 11216:+ info SORTTAB vmlinux
> > >
> > > Still building linux-kernel...
> > >
> > > Will report later if I was able to boot on bare metal.
> > >
> >
> > When running scripts/Makefile.modfinal:
> >
> > ...
> > not supported bit_size 160
> > Encountered error while encoding BTF.
> > ...
> > make[5]: *** [scripts/Makefile.modfinal:59:
> > drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko] Error 1
> > make[5]: *** Deleting file
> > 'drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko'
> >
>
> I added "bit_size 160" to libbtf.c:
>
> static int bits_to_int_bytes(uint16_t bit_size)
> {
> if (bit_size <= 8)
> return 1;
> if (bit_size <= 16)
> return 2;
> if (bit_size <= 32)
> return 4;
> if (bit_size <= 64)
> return 8;
> if (bit_size <= 128)
> return 16;
> if (bit_size <= 160)
> return 20;
> /* BTF supports upto 16byte int (__int128). */
> return -1;
> }
>
> It still breaks with:
>
> [521367] INT DW_ATE_unsigned_160 Error emitting BTF type
> Encountered error while encoding BTF.
> make[5]: *** [scripts/Makefile.modfinal:58:
> drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko] Error 1
> make[5]: *** Deleting file
> 'drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko'
>
> Comments?
>

Indeed there is DW_ATE_unsigned_160:

$ /opt/llvm-toolchain/bin/llvm-dwarfdump
drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.o | grep DW_AT_name
| grep DW_ATE_ | sort -u
DW_AT_name ("DW_ATE_signed_32")
DW_AT_name ("DW_ATE_signed_64")
DW_AT_name ("DW_ATE_unsigned_1")
DW_AT_name ("DW_ATE_unsigned_128")
DW_AT_name ("DW_ATE_unsigned_16")
DW_AT_name ("DW_ATE_unsigned_160")
DW_AT_name ("DW_ATE_unsigned_32")
DW_AT_name ("DW_ATE_unsigned_64")
DW_AT_name ("DW_ATE_unsigned_8")

Attached is diff v2 with the "bit_size 160" fix.

- Sedat -

Attachments:

DW_ATE_unsigned_1-pahole_1_20-dileks-v2.diff (2.38 kB)

2021-02-06 11:34:48

[permalink] [raw]

Subject: Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

On Sat, Feb 6, 2021 at 9:13 PM Yonghong Song <[email protected]> wrote:
>
>
>
> On 2/6/21 11:44 AM, Sedat Dilek wrote:
> > On Sat, Feb 6, 2021 at 8:33 PM Yonghong Song <[email protected]> wrote:
> >>
> >>
> >>
> >> On 2/6/21 11:28 AM, Sedat Dilek wrote:
> >>> On Sat, Feb 6, 2021 at 8:22 PM Sedat Dilek <[email protected]> wrote:
> >>>>
> >>>> On Sat, Feb 6, 2021 at 8:17 PM Yonghong Song <[email protected]> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 2/6/21 10:10 AM, Sedat Dilek wrote:
> >>>>>> On Sat, Feb 6, 2021 at 6:53 PM Yonghong Song <[email protected]> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On 2/6/21 8:24 AM, Mark Wieelard wrote:
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> On Sat, Feb 06, 2021 at 12:26:44AM -0800, Yonghong Song wrote:
> >>>>>>>>> With the above vmlinux, the issue appears to be handling
> >>>>>>>>> DW_ATE_signed_1, DW_ATE_unsigned_{1,24,40}.
> >>>>>>>>>
> >>>>>>>>> The following patch should fix the issue:
> >>>>>>>>
> >>>>>>>> That doesn't really make sense to me. Why is the compiler emitting a
> >>>>>>>> DW_TAG_base_type that needs to be interpreted according to the
> >>>>>>>> DW_AT_name attribute?
> >>>>>>>>
> >>>>>>>> If the issue is that the size of the base type cannot be expressed in
> >>>>>>>> bytes then the DWARF spec provides the following option:
> >>>>>>>>
> >>>>>>>> If the value of an object of the given type does not fully occupy
> >>>>>>>> the storage described by a byte size attribute, the base type
> >>>>>>>> entry may also have a DW_AT_bit_size and a DW_AT_data_bit_offset
> >>>>>>>> attribute, both of whose values are integer constant values (see
> >>>>>>>> Section 2.19 on page 55). The bit size attribute describes the
> >>>>>>>> actual size in bits used to represent values of the given
> >>>>>>>> type. The data bit offset attribute is the offset in bits from the
> >>>>>>>> beginning of the containing storage to the beginning of the
> >>>>>>>> value. Bits that are part of the offset are padding. If this
> >>>>>>>> attribute is omitted a default data bit offset of zero is assumed.
> >>>>>>>>
> >>>>>>>> Would it be possible to use that encoding of those special types? If
> >>>>>>>
> >>>>>>> I agree with you. I do not like comparing me as well. Unfortunately,
> >>>>>>> there is no enough information in dwarf to find out actual information.
> >>>>>>> The following is the dwarf dump with vmlinux (Sedat provided) for
> >>>>>>> DW_ATE_unsigned_1.
> >>>>>>>
> >>>>>>> 0x000e97e9: DW_TAG_base_type
> >>>>>>> DW_AT_name ("DW_ATE_unsigned_1")
> >>>>>>> DW_AT_encoding (DW_ATE_unsigned)
> >>>>>>> DW_AT_byte_size (0x00)
> >>>>>>>
> >>>>>>> There is no DW_AT_bit_size and DW_AT_bit_offset for base type.
> >>>>>>> AFAIK, these two attributes typically appear in struct/union members
> >>>>>>> together with DW_AT_byte_size.
> >>>>>>>
> >>>>>>> Maybe compilers (clang in this case) can emit DW_AT_bit_size = 1
> >>>>>>> and DW_AT_bit_offset = 0/7 (depending on big/little endian) and
> >>>>>>> this case, we just test and get DW_AT_bit_size and it should work.
> >>>>>>>
> >>>>>>> But I think BTF does not need this (DW_ATE_unsigned_1) for now.
> >>>>>>> I checked dwarf dump and it is mostly used for some arith operation
> >>>>>>> encoded in dump (in this case, e.g., shift by 1 bit)
> >>>>>>>
> >>>>>>> 0x000015cf: DW_TAG_base_type
> >>>>>>> DW_AT_name ("DW_ATE_unsigned_1")
> >>>>>>> DW_AT_encoding (DW_ATE_unsigned)
> >>>>>>> DW_AT_byte_size (0x00)
> >>>>>>>
> >>>>>>> 0x00010ed9: DW_TAG_formal_parameter
> >>>>>>> DW_AT_location (DW_OP_lit0, DW_OP_not,
> >>>>>>> DW_OP_convert (0x000015cf) "DW_ATE_unsigned_1", DW_OP_convert
> >>>>>>> (0x000015d4) "DW_ATE_unsigned_8", DW_OP_stack_value)
> >>>>>>> DW_AT_abstract_origin (0x00013984 "branch")
> >>>>>>>
> >>>>>>> Look at clang frontend, only the following types are encoded with
> >>>>>>> unsigned dwarf type.
> >>>>>>>
> >>>>>>> case BuiltinType::UShort:
> >>>>>>> case BuiltinType::UInt:
> >>>>>>> case BuiltinType::UInt128:
> >>>>>>> case BuiltinType::ULong:
> >>>>>>> case BuiltinType::WChar_U:
> >>>>>>> case BuiltinType::ULongLong:
> >>>>>>> Encoding = llvm::dwarf::DW_ATE_unsigned;
> >>>>>>> break;
> >>>>>>>
> >>>>>>>
> >>>>>>>> not, can we try to come up with some extension that doesn't require
> >>>>>>>> consumers to match magic names?
> >>>>>>>>
> >>>>>>
> >>>>>> You want me to upload mlx5_core.ko?
> >>>>>
> >>>>> I just sent out a patch. You are cc'ed. I also attached in this email.
> >>>>> Yes, it would be great if you can upload mlx5_core.ko so I can
> >>>>> double check with this DW_ATE_unsigned_160 which is really usual.
> >>>>>
> >>>>
> >>>> Yupp, just built a new pahole :-).
> >>>> Re-building linux-kernel...
> >>>>
> >>>> Will upload mlx5_core.ko - need zstd-ed it before.
> >>>>
> >>>
> >>> Hmm, I guess you want a mlx5_core.ko with your patch applied-to-pahole-1.20 :-)?
> >>
> >> this should work too. I want to check dwarf data. My patch won't impact
> >> dwarf generation.
> >>
> >
> > Usual Dropbox-Link:
> >
> > https://www.dropbox.com/sh/kvyh8ps7na0r1h5/AABfyNfDZ2bESse_bo4h05fFa?dl=0
> >
> > See "for-yhs" directory:
> >
> > 1. mlx5-module_yhs-v1 ("[PATCH dwarves] btf_encoder: sanitize
> > non-regular int base type")
> > 2. mlx5-module_yhs-dileks-v4 (with the last diff-v4 I tried successfully)
>
> Thanks, with llvm-dwarfdump, I can see
>
> 0x00d65616: DW_TAG_base_type
> DW_AT_name ("DW_ATE_unsigned_160")
> DW_AT_encoding (DW_ATE_unsigned)
> DW_AT_byte_size (0x14)
>
> 0x00d88e81: DW_TAG_variable
> DW_AT_location (indexed (0xad) loclist =
> 0x0005df42:
> [0x0000000000088c8e, 0x0000000000088c97):
> DW_OP_breg9 R9+0, DW_OP_convert (0x00d65616) "DW_ATE_unsigned_160",
> DW_OP_convert (0x00d65607) "DW_ATE_unsigned_32", DW_OP_stack_value,
> DW_OP_piece 0x4)
> DW_AT_abstract_origin (0x00d88d37 "_v")
>
>
> 0x00d88d37: DW_TAG_variable
> DW_AT_name ("_v")
> DW_AT_decl_file
> ("/home/dileks/src/linux-kernel/git/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c")
> DW_AT_decl_line (1198)
> DW_AT_type (0x00d68835 "u32")
>
> The source code at line 1198.
> 1198 DR_STE_SET_MASK_V(eth_l3_ipv4_5_tuple, bit_mask,
> 1199 source_port, mask, udp_sport);
>
> This is for struct mlx5dr_match_spec.
>
> struct mlx5dr_match_spec {
> u32 smac_47_16; /* Source MAC address of incoming packet */
> /* Incoming packet Ethertype - this is the Ethertype
> * following the last VLAN tag of the packet
> */
> u32 ethertype:16;
> u32 smac_15_0:16;
> ...
> u32 tcp_dport:16;
> /* TCP source port.;tcp and udp sport/dport are mutually
> exclusive */
> u32 tcp_sport:16;
> u32 ttl_hoplimit:8;
> u32 reserved:24;
> /* UDP destination port.;tcp and udp sport/dport are mutually
> exclusive */
> u32 udp_dport:16;
> /* UDP source port.;tcp and udp sport/dport are mutually
> exclusive */
> u32 udp_sport:16;
> /* IPv6 source address of incoming packets
> * For IPv4 address use bits 31:0 (rest of the bits are reserved)
> * This field should be qualified by an appropriate ethertype
> */
> u32 src_ip_127_96;
> ...
> }
>
> which includes a bunch of bit fields and non-bit fields.
>
> I have no idea why clang will generate
> DW_OP_convert (0x00d65616) "DW_ATE_unsigned_160"
> and possibly try to capture more semantic information?
> But BTF should be able to safely ignore this as described
> in my patch.
>
> Thanks.
>

[ CC Fangrui - the only guy I know who might comment on this ]

Fangrui, feel free to comment?

Get the patch "[PATCH dwarves] btf_encoder: sanitize non-regular int
base type" from Yonghong Son:

link="https://lore.kernel.org/r/[email protected]"
b4 -d am $link

I commented the success in the other thread.
Sorry for cross-posting.

Big Thank-You Yonghong!

- Sedat -

- Sedat -

> >
> > - Sedat -
> >
> >>>
> >>>> - Sedat -
> >>>>
> >>>>>>
> >>>>>> When looking with llvm-dwarf for DW_ATE_unsigned_160:
> >>>>>>
> >>>>>> 0x00d65616: DW_TAG_base_type
> >>>>>> DW_AT_name ("DW_ATE_unsigned_160")
> >>>>>> DW_AT_encoding (DW_ATE_unsigned)
> >>>>>> DW_AT_byte_size (0x14)
> >>>>>>
> >>>>>> If you need further information, please let me know.
> >>>>>>
> >>>>>> Thanks.
> >>>>>>
> >>>>>> - Sedat -
> >>>>>>