2021-01-30 01:24:03

by Nick Desaulniers

[permalink] [raw]
Subject: [PATCH v7 0/2] Kbuild: DWARF v5 support

DWARF v5 is the latest standard of the DWARF debug info format.

DWARF5 wins significantly in terms of size and especially so when mixed
with compression (CONFIG_DEBUG_INFO_COMPRESSED).

Link: http://www.dwarfstd.org/doc/DWARF5.pdf

Patch 1 is a cleanup that lays the ground work and isn't DWARF
v5 specific.
Patch 2 implements Kconfig and Kbuild support for DWARFv5.

Changes from v6:
* Reorder sections from linker script to match order from BFD's internal
linker script.
* Add .debug_names section, as per Fangrui.
* Drop CONFIG_DEBUG_INFO_DWARF2. Patch 0001 becomes a menu with 1
choice. GCC's implicit default version of DWARF has been DWARF v4
since ~4.8.
* Modify the test script to check for the presence of
https://sourceware.org/bugzilla/show_bug.cgi?id=27195.
* Drop the clang without integrated assembler block in
0002. Bumps the version requirement for GAS to 2.35.2, which isn't
released yet (but should be released soon). Folks looking to test
with clang but without the integrated assembler should fetch
binutils-gdb, build it from source, add a symlink to
binutils-gdb/gas/as-new to binutils-gdb/gas/as, then prefix
binutils-gdb/gas/as to their $PATH when building the kernel.

Changes from v5:
* Drop previous patch 1, it has been accepted into kbuild:
https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=kbuild&id=3f4d8ce271c7082be75bacbcbd2048aa78ce2b44
* Trying to set -Wa,-gdwarf-4 in the earlier patch was the source of
additional complexity. Drop it that part of the patch. We can revisit
clang without the integrated assembler setting -Wa,-gdwarf-4 later.
That is a separate problem from generally supporting DWARF v5.
* Rework the final patch for clang without the integrated assembler.
-Wa,-gdwarf-5 is required for DWARF5 in that case otherwise GAS will
not accept the assembler directives clang produces from C code when
generating asm.

Changes from v4:
* drop set -e from script as per Nathan.
* add dependency on !CONFIG_DEBUG_INFO_BTF for DWARF v5 as per Sedat.
* Move LLVM_IAS=1 complexity from patch 2 to patch 3 as per Arvind and
Masahiro. Sorry it took me a few tries to understand the point (I
might still not), but it looks much cleaner this way. Sorry Nathan, I
did not carry forward your previous reviews as a result, but I would
appreciate if you could look again.
* Add Nathan's reviewed by tag to patch 1.
* Reword commit message for patch 3 to mention LLVM_IAS=1 and -gdwarf-5
binutils addition later, and BTF issue.
* I still happen to see a pahole related error spew for the combination
of:
* LLVM=1
* LLVM_IAS=1
* CONFIG_DEBUG_INFO_DWARF4
* CONFIG_DEBUG_INFO_BTF
Though they're non-fatal to the build. I'm not sure yet why removing
any one of the above prevents the warning spew. Maybe we'll need a v6.

Changes from v3:

Changes as per Arvind:
* only add -Wa,-gdwarf-5 for (LLVM=1|CC=clang)+LLVM_IAS=0 builds.
* add -gdwarf-5 to Kconfig shell script.
* only run Kconfig shell script for Clang.

Apologies to Sedat and Nathan; I appreciate previous testing/review, but
I did no carry forward your Tested-by and Reviewed-by tags, as the
patches have changed too much IMO.

Changes from v2:
* Drop two of the earlier patches that have been accepted already.
* Add measurements with GCC 10.2 to commit message.
* Update help text as per Arvind with help from Caroline.
* Improve case/wording between DWARF Versions as per Masahiro.

Changes from the RFC:
* split patch in 3 patch series, include Fangrui's patch, too.
* prefer `DWARF vX` format, as per Fangrui.
* use spaces between assignment in Makefile as per Masahiro.
* simplify setting dwarf-version-y as per Masahiro.
* indent `prompt` in Kconfig change as per Masahiro.
* remove explicit default in Kconfig as per Masahiro.
* add comments to test_dwarf5_support.sh.
* change echo in test_dwarf5_support.sh as per Masahiro.
* remove -u from test_dwarf5_support.sh as per Masahiro.
* add a -gdwarf-5 cc-option check to Kconfig as per Jakub.

Nick Desaulniers (2):
Kbuild: make DWARF version a choice
Kbuild: implement support for DWARF v5

Makefile | 6 +++---
include/asm-generic/vmlinux.lds.h | 7 +++++-
lib/Kconfig.debug | 36 +++++++++++++++++++++++++------
scripts/test_dwarf5_support.sh | 8 +++++++
4 files changed, 47 insertions(+), 10 deletions(-)
create mode 100755 scripts/test_dwarf5_support.sh

--
2.30.0.365.g02bc693789-goog


2021-01-30 10:18:42

by Nick Desaulniers

[permalink] [raw]
Subject: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

DWARF v5 is the latest standard of the DWARF debug info format.

Feature detection of DWARF5 is onerous, especially given that we've
removed $(AS), so we must query $(CC) for DWARF5 assembler directive
support.

The DWARF version of a binary can be validated with:
$ llvm-dwarfdump vmlinux | head -n 4 | grep version
or
$ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version

DWARF5 wins significantly in terms of size when mixed with compression
(CONFIG_DEBUG_INFO_COMPRESSED).

363M vmlinux.clang12.dwarf5.compressed
434M vmlinux.clang12.dwarf4.compressed
439M vmlinux.clang12.dwarf2.compressed
457M vmlinux.clang12.dwarf5
536M vmlinux.clang12.dwarf4
548M vmlinux.clang12.dwarf2

515M vmlinux.gcc10.2.dwarf5.compressed
599M vmlinux.gcc10.2.dwarf4.compressed
624M vmlinux.gcc10.2.dwarf2.compressed
630M vmlinux.gcc10.2.dwarf5
765M vmlinux.gcc10.2.dwarf4
809M vmlinux.gcc10.2.dwarf2

Though the quality of debug info is harder to quantify; size is not a
proxy for quality.

Jakub notes:
All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
option that enabled some small DWARF subset (initially only a few
DW_LANG_* codes newly added to DWARF5 drafts). Only GCC 7 (released
after DWARF 5 has been finalized) started emitting DWARF5 section
headers and got most of the DWARF5 changes in...

Version check GCC so that we don't need to worry about the difference in
command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
validate the DWARF Version in the assembler feature detection script.

GNU `as` only recently gained support for specifying -gdwarf-5, so when
compiling with Clang but without Clang's integrated assembler
(LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.

Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
recognize the new additions to the DWARF debug info. Thanks to Sedat for
the report.

Link: http://www.dwarfstd.org/doc/DWARF5.pdf
Reported-by: Sedat Dilek <[email protected]>
Suggested-by: Arvind Sankar <[email protected]>
Suggested-by: Caroline Tice <[email protected]>
Suggested-by: Fangrui Song <[email protected]>
Suggested-by: Jakub Jelinek <[email protected]>
Suggested-by: Masahiro Yamada <[email protected]>
Suggested-by: Nathan Chancellor <[email protected]>
Signed-off-by: Nick Desaulniers <[email protected]>
---
Makefile | 1 +
include/asm-generic/vmlinux.lds.h | 7 ++++++-
lib/Kconfig.debug | 18 ++++++++++++++++++
scripts/test_dwarf5_support.sh | 8 ++++++++
4 files changed, 33 insertions(+), 1 deletion(-)
create mode 100755 scripts/test_dwarf5_support.sh

diff --git a/Makefile b/Makefile
index d2b4980807e0..5387a6f2f62d 100644
--- a/Makefile
+++ b/Makefile
@@ -831,6 +831,7 @@ KBUILD_AFLAGS += -Wa,-gdwarf-2
endif

dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
+dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
DEBUG_CFLAGS += -gdwarf-$(dwarf-version-y)

ifdef CONFIG_DEBUG_INFO_REDUCED
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 34b7e0d2346c..1e7cde4bd3f9 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -842,8 +842,13 @@
/* DWARF 4 */ \
.debug_types 0 : { *(.debug_types) } \
/* DWARF 5 */ \
+ .debug_addr 0 : { *(.debug_addr) } \
+ .debug_line_str 0 : { *(.debug_line_str) } \
+ .debug_loclists 0 : { *(.debug_loclists) } \
.debug_macro 0 : { *(.debug_macro) } \
- .debug_addr 0 : { *(.debug_addr) }
+ .debug_names 0 : { *(.debug_names) } \
+ .debug_rnglists 0 : { *(.debug_rnglists) } \
+ .debug_str_offsets 0 : { *(.debug_str_offsets) }

/* Stabs debugging sections. */
#define STABS_DEBUG \
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 94c1a7ed6306..ad6f78989d4f 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
It makes the debug information larger, but it significantly
improves the success of resolving variables in gdb on optimized code.

+config DEBUG_INFO_DWARF5
+ bool "Generate DWARF Version 5 debuginfo"
+ depends on GCC_VERSION >= 50000 || CC_IS_CLANG
+ depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
+ depends on !DEBUG_INFO_BTF
+ help
+ Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
+ 5.0+ accepts the -gdwarf-5 flag but only had partial support for some
+ draft features until 7.0), and gdb 8.0+.
+
+ Changes to the structure of debug info in Version 5 allow for around
+ 15-18% savings in resulting image and debug info section sizes as
+ compared to DWARF Version 4. DWARF Version 5 standardizes previous
+ extensions such as accelerators for symbol indexing and the format
+ for fission (.dwo/.dwp) files. Users may not want to select this
+ config if they rely on tooling that has not yet been updated to
+ support DWARF Version 5.
+
endchoice # "DWARF version"

config DEBUG_INFO_BTF
diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
new file mode 100755
index 000000000000..c46e2456b47a
--- /dev/null
+++ b/scripts/test_dwarf5_support.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+
+# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
+# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
+# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
+echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
+ $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
--
2.30.0.365.g02bc693789-goog

2021-01-30 23:12:47

by Sedat Dilek

[permalink] [raw]
Subject: Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

On Sat, Jan 30, 2021 at 1:44 AM Nick Desaulniers
<[email protected]> wrote:
>
> DWARF v5 is the latest standard of the DWARF debug info format.
>
> Feature detection of DWARF5 is onerous, especially given that we've
> removed $(AS), so we must query $(CC) for DWARF5 assembler directive
> support.
>
> The DWARF version of a binary can be validated with:
> $ llvm-dwarfdump vmlinux | head -n 4 | grep version
> or
> $ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version
>
> DWARF5 wins significantly in terms of size when mixed with compression
> (CONFIG_DEBUG_INFO_COMPRESSED).
>
> 363M vmlinux.clang12.dwarf5.compressed
> 434M vmlinux.clang12.dwarf4.compressed
> 439M vmlinux.clang12.dwarf2.compressed
> 457M vmlinux.clang12.dwarf5
> 536M vmlinux.clang12.dwarf4
> 548M vmlinux.clang12.dwarf2
>
> 515M vmlinux.gcc10.2.dwarf5.compressed
> 599M vmlinux.gcc10.2.dwarf4.compressed
> 624M vmlinux.gcc10.2.dwarf2.compressed
> 630M vmlinux.gcc10.2.dwarf5
> 765M vmlinux.gcc10.2.dwarf4
> 809M vmlinux.gcc10.2.dwarf2
>
> Though the quality of debug info is harder to quantify; size is not a
> proxy for quality.
>
> Jakub notes:
> All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
> option that enabled some small DWARF subset (initially only a few
> DW_LANG_* codes newly added to DWARF5 drafts). Only GCC 7 (released
> after DWARF 5 has been finalized) started emitting DWARF5 section
> headers and got most of the DWARF5 changes in...
>
> Version check GCC so that we don't need to worry about the difference in
> command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
> validate the DWARF Version in the assembler feature detection script.
>
> GNU `as` only recently gained support for specifying -gdwarf-5, so when
> compiling with Clang but without Clang's integrated assembler
> (LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.
>
> Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
> recognize the new additions to the DWARF debug info. Thanks to Sedat for
> the report.
>
> Link: http://www.dwarfstd.org/doc/DWARF5.pdf
> Reported-by: Sedat Dilek <[email protected]>
> Suggested-by: Arvind Sankar <[email protected]>
> Suggested-by: Caroline Tice <[email protected]>
> Suggested-by: Fangrui Song <[email protected]>
> Suggested-by: Jakub Jelinek <[email protected]>
> Suggested-by: Masahiro Yamada <[email protected]>
> Suggested-by: Nathan Chancellor <[email protected]>
> Signed-off-by: Nick Desaulniers <[email protected]>
> ---
> Makefile | 1 +
> include/asm-generic/vmlinux.lds.h | 7 ++++++-
> lib/Kconfig.debug | 18 ++++++++++++++++++
> scripts/test_dwarf5_support.sh | 8 ++++++++
> 4 files changed, 33 insertions(+), 1 deletion(-)
> create mode 100755 scripts/test_dwarf5_support.sh
>
> diff --git a/Makefile b/Makefile
> index d2b4980807e0..5387a6f2f62d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -831,6 +831,7 @@ KBUILD_AFLAGS += -Wa,-gdwarf-2
> endif
>
> dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
> DEBUG_CFLAGS += -gdwarf-$(dwarf-version-y)
>
> ifdef CONFIG_DEBUG_INFO_REDUCED
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index 34b7e0d2346c..1e7cde4bd3f9 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -842,8 +842,13 @@
> /* DWARF 4 */ \
> .debug_types 0 : { *(.debug_types) } \
> /* DWARF 5 */ \
> + .debug_addr 0 : { *(.debug_addr) } \
> + .debug_line_str 0 : { *(.debug_line_str) } \
> + .debug_loclists 0 : { *(.debug_loclists) } \
> .debug_macro 0 : { *(.debug_macro) } \
> - .debug_addr 0 : { *(.debug_addr) }
> + .debug_names 0 : { *(.debug_names) } \
> + .debug_rnglists 0 : { *(.debug_rnglists) } \
> + .debug_str_offsets 0 : { *(.debug_str_offsets) }
>

I just looked at binutils 2.36 in the Debian/experimental repositories.

[1] says:

+ PR ld/27230
+ * scripttempl/DWARF.sc: Add DWARF-5 .debug_* sections.

...

- /* DWARF Extension. */
- .debug_macro 0 : { *(.debug_macro) }
+ /* DWARF 5. */
.debug_addr 0 : { *(.debug_addr) }
+ .debug_line_str 0 : { *(.debug_line_str) }
+ .debug_loclists 0 : { *(.debug_loclists) }
+ .debug_macro 0 : { *(.debug_macro) }
+ .debug_names 0 : { *(.debug_names) }
+ .debug_rnglists 0 : { *(.debug_rnglists) }
+ .debug_str_offsets 0 : { *(.debug_str_offsets) }
+ .debug_sup 0 : { *(.debug_sup) }

The list of DWARF-5 .debug_* sections is alphabetically sorted.
AFAICS .debug_sup section is missing?

- Sedat -

[1] https://salsa.debian.org/toolchain-team/binutils/-/commit/f58f33081035672b01a04326a9c8daadbd09a430

> /* Stabs debugging sections. */
> #define STABS_DEBUG \
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 94c1a7ed6306..ad6f78989d4f 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> It makes the debug information larger, but it significantly
> improves the success of resolving variables in gdb on optimized code.
>
> +config DEBUG_INFO_DWARF5
> + bool "Generate DWARF Version 5 debuginfo"
> + depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> + depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
> + depends on !DEBUG_INFO_BTF
> + help
> + Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
> + 5.0+ accepts the -gdwarf-5 flag but only had partial support for some
> + draft features until 7.0), and gdb 8.0+.
> +
> + Changes to the structure of debug info in Version 5 allow for around
> + 15-18% savings in resulting image and debug info section sizes as
> + compared to DWARF Version 4. DWARF Version 5 standardizes previous
> + extensions such as accelerators for symbol indexing and the format
> + for fission (.dwo/.dwp) files. Users may not want to select this
> + config if they rely on tooling that has not yet been updated to
> + support DWARF Version 5.
> +
> endchoice # "DWARF version"
>
> config DEBUG_INFO_BTF
> diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
> new file mode 100755
> index 000000000000..c46e2456b47a
> --- /dev/null
> +++ b/scripts/test_dwarf5_support.sh
> @@ -0,0 +1,8 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0
> +
> +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
> +echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
> + $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
> --
> 2.30.0.365.g02bc693789-goog
>

2021-01-30 23:42:24

by Sedat Dilek

[permalink] [raw]
Subject: Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

On Sun, Jan 31, 2021 at 12:10 AM Sedat Dilek <[email protected]> wrote:
>
> On Sat, Jan 30, 2021 at 1:44 AM Nick Desaulniers
> <[email protected]> wrote:
> >
> > DWARF v5 is the latest standard of the DWARF debug info format.
> >
> > Feature detection of DWARF5 is onerous, especially given that we've
> > removed $(AS), so we must query $(CC) for DWARF5 assembler directive
> > support.
> >
> > The DWARF version of a binary can be validated with:
> > $ llvm-dwarfdump vmlinux | head -n 4 | grep version
> > or
> > $ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version
> >
> > DWARF5 wins significantly in terms of size when mixed with compression
> > (CONFIG_DEBUG_INFO_COMPRESSED).
> >
> > 363M vmlinux.clang12.dwarf5.compressed
> > 434M vmlinux.clang12.dwarf4.compressed
> > 439M vmlinux.clang12.dwarf2.compressed
> > 457M vmlinux.clang12.dwarf5
> > 536M vmlinux.clang12.dwarf4
> > 548M vmlinux.clang12.dwarf2
> >
> > 515M vmlinux.gcc10.2.dwarf5.compressed
> > 599M vmlinux.gcc10.2.dwarf4.compressed
> > 624M vmlinux.gcc10.2.dwarf2.compressed
> > 630M vmlinux.gcc10.2.dwarf5
> > 765M vmlinux.gcc10.2.dwarf4
> > 809M vmlinux.gcc10.2.dwarf2
> >
> > Though the quality of debug info is harder to quantify; size is not a
> > proxy for quality.
> >
> > Jakub notes:
> > All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
> > option that enabled some small DWARF subset (initially only a few
> > DW_LANG_* codes newly added to DWARF5 drafts). Only GCC 7 (released
> > after DWARF 5 has been finalized) started emitting DWARF5 section
> > headers and got most of the DWARF5 changes in...
> >
> > Version check GCC so that we don't need to worry about the difference in
> > command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
> > validate the DWARF Version in the assembler feature detection script.
> >
> > GNU `as` only recently gained support for specifying -gdwarf-5, so when
> > compiling with Clang but without Clang's integrated assembler
> > (LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.
> >
> > Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
> > recognize the new additions to the DWARF debug info. Thanks to Sedat for
> > the report.
> >
> > Link: http://www.dwarfstd.org/doc/DWARF5.pdf
> > Reported-by: Sedat Dilek <[email protected]>
> > Suggested-by: Arvind Sankar <[email protected]>
> > Suggested-by: Caroline Tice <[email protected]>
> > Suggested-by: Fangrui Song <[email protected]>
> > Suggested-by: Jakub Jelinek <[email protected]>
> > Suggested-by: Masahiro Yamada <[email protected]>
> > Suggested-by: Nathan Chancellor <[email protected]>
> > Signed-off-by: Nick Desaulniers <[email protected]>
> > ---
> > Makefile | 1 +
> > include/asm-generic/vmlinux.lds.h | 7 ++++++-
> > lib/Kconfig.debug | 18 ++++++++++++++++++
> > scripts/test_dwarf5_support.sh | 8 ++++++++
> > 4 files changed, 33 insertions(+), 1 deletion(-)
> > create mode 100755 scripts/test_dwarf5_support.sh
> >
> > diff --git a/Makefile b/Makefile
> > index d2b4980807e0..5387a6f2f62d 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -831,6 +831,7 @@ KBUILD_AFLAGS += -Wa,-gdwarf-2
> > endif
> >
> > dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> > +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
> > DEBUG_CFLAGS += -gdwarf-$(dwarf-version-y)
> >
> > ifdef CONFIG_DEBUG_INFO_REDUCED
> > diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> > index 34b7e0d2346c..1e7cde4bd3f9 100644
> > --- a/include/asm-generic/vmlinux.lds.h
> > +++ b/include/asm-generic/vmlinux.lds.h
> > @@ -842,8 +842,13 @@
> > /* DWARF 4 */ \
> > .debug_types 0 : { *(.debug_types) } \
> > /* DWARF 5 */ \
> > + .debug_addr 0 : { *(.debug_addr) } \
> > + .debug_line_str 0 : { *(.debug_line_str) } \
> > + .debug_loclists 0 : { *(.debug_loclists) } \
> > .debug_macro 0 : { *(.debug_macro) } \
> > - .debug_addr 0 : { *(.debug_addr) }
> > + .debug_names 0 : { *(.debug_names) } \
> > + .debug_rnglists 0 : { *(.debug_rnglists) } \
> > + .debug_str_offsets 0 : { *(.debug_str_offsets) }
> >
>
> I just looked at binutils 2.36 in the Debian/experimental repositories.
>
> [1] says:
>
> + PR ld/27230
> + * scripttempl/DWARF.sc: Add DWARF-5 .debug_* sections.
>
> ...
>
> - /* DWARF Extension. */
> - .debug_macro 0 : { *(.debug_macro) }
> + /* DWARF 5. */
> .debug_addr 0 : { *(.debug_addr) }
> + .debug_line_str 0 : { *(.debug_line_str) }
> + .debug_loclists 0 : { *(.debug_loclists) }
> + .debug_macro 0 : { *(.debug_macro) }
> + .debug_names 0 : { *(.debug_names) }
> + .debug_rnglists 0 : { *(.debug_rnglists) }
> + .debug_str_offsets 0 : { *(.debug_str_offsets) }
> + .debug_sup 0 : { *(.debug_sup) }
>
> The list of DWARF-5 .debug_* sections is alphabetically sorted.
> AFAICS .debug_sup section is missing?
>

Here the link to PR ld/27230 ("[PATCH] Add DWARF v5 sections and v4
.debug_types").

- Sedat -

[1] https://sourceware.org/pipermail/binutils/2021-January/115064.html

> - Sedat -
>
> [1] https://salsa.debian.org/toolchain-team/binutils/-/commit/f58f33081035672b01a04326a9c8daadbd09a430
>
> > /* Stabs debugging sections. */
> > #define STABS_DEBUG \
> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > index 94c1a7ed6306..ad6f78989d4f 100644
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> > It makes the debug information larger, but it significantly
> > improves the success of resolving variables in gdb on optimized code.
> >
> > +config DEBUG_INFO_DWARF5
> > + bool "Generate DWARF Version 5 debuginfo"
> > + depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> > + depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
> > + depends on !DEBUG_INFO_BTF
> > + help
> > + Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
> > + 5.0+ accepts the -gdwarf-5 flag but only had partial support for some
> > + draft features until 7.0), and gdb 8.0+.
> > +
> > + Changes to the structure of debug info in Version 5 allow for around
> > + 15-18% savings in resulting image and debug info section sizes as
> > + compared to DWARF Version 4. DWARF Version 5 standardizes previous
> > + extensions such as accelerators for symbol indexing and the format
> > + for fission (.dwo/.dwp) files. Users may not want to select this
> > + config if they rely on tooling that has not yet been updated to
> > + support DWARF Version 5.
> > +
> > endchoice # "DWARF version"
> >
> > config DEBUG_INFO_BTF
> > diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
> > new file mode 100755
> > index 000000000000..c46e2456b47a
> > --- /dev/null
> > +++ b/scripts/test_dwarf5_support.sh
> > @@ -0,0 +1,8 @@
> > +#!/bin/sh
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> > +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> > +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
> > +echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
> > + $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
> > --
> > 2.30.0.365.g02bc693789-goog
> >

2021-01-31 00:01:31

by Sedat Dilek

[permalink] [raw]
Subject: Re: [PATCH v7 0/2] Kbuild: DWARF v5 support

On Sat, Jan 30, 2021 at 1:44 AM Nick Desaulniers
<[email protected]> wrote:
>
> DWARF v5 is the latest standard of the DWARF debug info format.
>
> DWARF5 wins significantly in terms of size and especially so when mixed
> with compression (CONFIG_DEBUG_INFO_COMPRESSED).
>
> Link: http://www.dwarfstd.org/doc/DWARF5.pdf
>
> Patch 1 is a cleanup that lays the ground work and isn't DWARF
> v5 specific.
> Patch 2 implements Kconfig and Kbuild support for DWARFv5.
>
> Changes from v6:
> * Reorder sections from linker script to match order from BFD's internal
> linker script.
> * Add .debug_names section, as per Fangrui.
> * Drop CONFIG_DEBUG_INFO_DWARF2. Patch 0001 becomes a menu with 1
> choice. GCC's implicit default version of DWARF has been DWARF v4
> since ~4.8.
> * Modify the test script to check for the presence of
> https://sourceware.org/bugzilla/show_bug.cgi?id=27195.

Just as a note: GNU Binutils 2.35.2 Release is now available

- Sedat -

[1] https://sourceware.org/pipermail/binutils/2021-January/115150.html

> * Drop the clang without integrated assembler block in
> 0002. Bumps the version requirement for GAS to 2.35.2, which isn't
> released yet (but should be released soon). Folks looking to test
> with clang but without the integrated assembler should fetch
> binutils-gdb, build it from source, add a symlink to
> binutils-gdb/gas/as-new to binutils-gdb/gas/as, then prefix
> binutils-gdb/gas/as to their $PATH when building the kernel.
>
> Changes from v5:
> * Drop previous patch 1, it has been accepted into kbuild:
> https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=kbuild&id=3f4d8ce271c7082be75bacbcbd2048aa78ce2b44
> * Trying to set -Wa,-gdwarf-4 in the earlier patch was the source of
> additional complexity. Drop it that part of the patch. We can revisit
> clang without the integrated assembler setting -Wa,-gdwarf-4 later.
> That is a separate problem from generally supporting DWARF v5.
> * Rework the final patch for clang without the integrated assembler.
> -Wa,-gdwarf-5 is required for DWARF5 in that case otherwise GAS will
> not accept the assembler directives clang produces from C code when
> generating asm.
>
> Changes from v4:
> * drop set -e from script as per Nathan.
> * add dependency on !CONFIG_DEBUG_INFO_BTF for DWARF v5 as per Sedat.
> * Move LLVM_IAS=1 complexity from patch 2 to patch 3 as per Arvind and
> Masahiro. Sorry it took me a few tries to understand the point (I
> might still not), but it looks much cleaner this way. Sorry Nathan, I
> did not carry forward your previous reviews as a result, but I would
> appreciate if you could look again.
> * Add Nathan's reviewed by tag to patch 1.
> * Reword commit message for patch 3 to mention LLVM_IAS=1 and -gdwarf-5
> binutils addition later, and BTF issue.
> * I still happen to see a pahole related error spew for the combination
> of:
> * LLVM=1
> * LLVM_IAS=1
> * CONFIG_DEBUG_INFO_DWARF4
> * CONFIG_DEBUG_INFO_BTF
> Though they're non-fatal to the build. I'm not sure yet why removing
> any one of the above prevents the warning spew. Maybe we'll need a v6.
>
> Changes from v3:
>
> Changes as per Arvind:
> * only add -Wa,-gdwarf-5 for (LLVM=1|CC=clang)+LLVM_IAS=0 builds.
> * add -gdwarf-5 to Kconfig shell script.
> * only run Kconfig shell script for Clang.
>
> Apologies to Sedat and Nathan; I appreciate previous testing/review, but
> I did no carry forward your Tested-by and Reviewed-by tags, as the
> patches have changed too much IMO.
>
> Changes from v2:
> * Drop two of the earlier patches that have been accepted already.
> * Add measurements with GCC 10.2 to commit message.
> * Update help text as per Arvind with help from Caroline.
> * Improve case/wording between DWARF Versions as per Masahiro.
>
> Changes from the RFC:
> * split patch in 3 patch series, include Fangrui's patch, too.
> * prefer `DWARF vX` format, as per Fangrui.
> * use spaces between assignment in Makefile as per Masahiro.
> * simplify setting dwarf-version-y as per Masahiro.
> * indent `prompt` in Kconfig change as per Masahiro.
> * remove explicit default in Kconfig as per Masahiro.
> * add comments to test_dwarf5_support.sh.
> * change echo in test_dwarf5_support.sh as per Masahiro.
> * remove -u from test_dwarf5_support.sh as per Masahiro.
> * add a -gdwarf-5 cc-option check to Kconfig as per Jakub.
>
> Nick Desaulniers (2):
> Kbuild: make DWARF version a choice
> Kbuild: implement support for DWARF v5
>
> Makefile | 6 +++---
> include/asm-generic/vmlinux.lds.h | 7 +++++-
> lib/Kconfig.debug | 36 +++++++++++++++++++++++++------
> scripts/test_dwarf5_support.sh | 8 +++++++
> 4 files changed, 47 insertions(+), 10 deletions(-)
> create mode 100755 scripts/test_dwarf5_support.sh
>
> --
> 2.30.0.365.g02bc693789-goog
>

2021-01-31 00:40:42

by Fangrui Song

[permalink] [raw]
Subject: Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

On Sat, Jan 30, 2021 at 3:10 PM Sedat Dilek <[email protected]> wrote:
>
> On Sat, Jan 30, 2021 at 1:44 AM Nick Desaulniers
> <[email protected]> wrote:
> >
> > DWARF v5 is the latest standard of the DWARF debug info format.
> >
> > Feature detection of DWARF5 is onerous, especially given that we've
> > removed $(AS), so we must query $(CC) for DWARF5 assembler directive
> > support.
> >
> > The DWARF version of a binary can be validated with:
> > $ llvm-dwarfdump vmlinux | head -n 4 | grep version
> > or
> > $ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version
> >
> > DWARF5 wins significantly in terms of size when mixed with compression
> > (CONFIG_DEBUG_INFO_COMPRESSED).
> >
> > 363M vmlinux.clang12.dwarf5.compressed
> > 434M vmlinux.clang12.dwarf4.compressed
> > 439M vmlinux.clang12.dwarf2.compressed
> > 457M vmlinux.clang12.dwarf5
> > 536M vmlinux.clang12.dwarf4
> > 548M vmlinux.clang12.dwarf2
> >
> > 515M vmlinux.gcc10.2.dwarf5.compressed
> > 599M vmlinux.gcc10.2.dwarf4.compressed
> > 624M vmlinux.gcc10.2.dwarf2.compressed
> > 630M vmlinux.gcc10.2.dwarf5
> > 765M vmlinux.gcc10.2.dwarf4
> > 809M vmlinux.gcc10.2.dwarf2
> >
> > Though the quality of debug info is harder to quantify; size is not a
> > proxy for quality.
> >
> > Jakub notes:
> > All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
> > option that enabled some small DWARF subset (initially only a few
> > DW_LANG_* codes newly added to DWARF5 drafts). Only GCC 7 (released
> > after DWARF 5 has been finalized) started emitting DWARF5 section
> > headers and got most of the DWARF5 changes in...
> >
> > Version check GCC so that we don't need to worry about the difference in
> > command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
> > validate the DWARF Version in the assembler feature detection script.
> >
> > GNU `as` only recently gained support for specifying -gdwarf-5, so when
> > compiling with Clang but without Clang's integrated assembler
> > (LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.
> >
> > Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
> > recognize the new additions to the DWARF debug info. Thanks to Sedat for
> > the report.
> >
> > Link: http://www.dwarfstd.org/doc/DWARF5.pdf
> > Reported-by: Sedat Dilek <[email protected]>
> > Suggested-by: Arvind Sankar <[email protected]>
> > Suggested-by: Caroline Tice <[email protected]>
> > Suggested-by: Fangrui Song <[email protected]>
> > Suggested-by: Jakub Jelinek <[email protected]>
> > Suggested-by: Masahiro Yamada <[email protected]>
> > Suggested-by: Nathan Chancellor <[email protected]>
> > Signed-off-by: Nick Desaulniers <[email protected]>
> > ---
> > Makefile | 1 +
> > include/asm-generic/vmlinux.lds.h | 7 ++++++-
> > lib/Kconfig.debug | 18 ++++++++++++++++++
> > scripts/test_dwarf5_support.sh | 8 ++++++++
> > 4 files changed, 33 insertions(+), 1 deletion(-)
> > create mode 100755 scripts/test_dwarf5_support.sh
> >
> > diff --git a/Makefile b/Makefile
> > index d2b4980807e0..5387a6f2f62d 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -831,6 +831,7 @@ KBUILD_AFLAGS += -Wa,-gdwarf-2
> > endif
> >
> > dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> > +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
> > DEBUG_CFLAGS += -gdwarf-$(dwarf-version-y)
> >
> > ifdef CONFIG_DEBUG_INFO_REDUCED
> > diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> > index 34b7e0d2346c..1e7cde4bd3f9 100644
> > --- a/include/asm-generic/vmlinux.lds.h
> > +++ b/include/asm-generic/vmlinux.lds.h
> > @@ -842,8 +842,13 @@
> > /* DWARF 4 */ \
> > .debug_types 0 : { *(.debug_types) } \
> > /* DWARF 5 */ \
> > + .debug_addr 0 : { *(.debug_addr) } \
> > + .debug_line_str 0 : { *(.debug_line_str) } \
> > + .debug_loclists 0 : { *(.debug_loclists) } \
> > .debug_macro 0 : { *(.debug_macro) } \
> > - .debug_addr 0 : { *(.debug_addr) }
> > + .debug_names 0 : { *(.debug_names) } \
> > + .debug_rnglists 0 : { *(.debug_rnglists) } \
> > + .debug_str_offsets 0 : { *(.debug_str_offsets) }
> >
>
> I just looked at binutils 2.36 in the Debian/experimental repositories.
>
> [1] says:
>
> + PR ld/27230
> + * scripttempl/DWARF.sc: Add DWARF-5 .debug_* sections.
>
> ...
>
> - /* DWARF Extension. */
> - .debug_macro 0 : { *(.debug_macro) }
> + /* DWARF 5. */
> .debug_addr 0 : { *(.debug_addr) }
> + .debug_line_str 0 : { *(.debug_line_str) }
> + .debug_loclists 0 : { *(.debug_loclists) }
> + .debug_macro 0 : { *(.debug_macro) }
> + .debug_names 0 : { *(.debug_names) }
> + .debug_rnglists 0 : { *(.debug_rnglists) }
> + .debug_str_offsets 0 : { *(.debug_str_offsets) }
> + .debug_sup 0 : { *(.debug_sup) }
>
> The list of DWARF-5 .debug_* sections is alphabetically sorted.
> AFAICS .debug_sup section is missing?
>
> - Sedat -

No compiler produces .debug_sup section. It could be from some
post-processing tool which is unrelated to the linker.
Omitting it is fine.

> [1] https://salsa.debian.org/toolchain-team/binutils/-/commit/f58f33081035672b01a04326a9c8daadbd09a430
>
> > /* Stabs debugging sections. */
> > #define STABS_DEBUG \
> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > index 94c1a7ed6306..ad6f78989d4f 100644
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> > It makes the debug information larger, but it significantly
> > improves the success of resolving variables in gdb on optimized code.
> >
> > +config DEBUG_INFO_DWARF5
> > + bool "Generate DWARF Version 5 debuginfo"
> > + depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> > + depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
> > + depends on !DEBUG_INFO_BTF
> > + help
> > + Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
> > + 5.0+ accepts the -gdwarf-5 flag but only had partial support for some
> > + draft features until 7.0), and gdb 8.0+.
> > +
> > + Changes to the structure of debug info in Version 5 allow for around
> > + 15-18% savings in resulting image and debug info section sizes as
> > + compared to DWARF Version 4. DWARF Version 5 standardizes previous
> > + extensions such as accelerators for symbol indexing and the format
> > + for fission (.dwo/.dwp) files. Users may not want to select this
> > + config if they rely on tooling that has not yet been updated to
> > + support DWARF Version 5.
> > +
> > endchoice # "DWARF version"
> >
> > config DEBUG_INFO_BTF
> > diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
> > new file mode 100755
> > index 000000000000..c46e2456b47a
> > --- /dev/null
> > +++ b/scripts/test_dwarf5_support.sh
> > @@ -0,0 +1,8 @@
> > +#!/bin/sh
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> > +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> > +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
> > +echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
> > + $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
> > --
> > 2.30.0.365.g02bc693789-goog
> >

2021-01-31 00:44:48

by Sedat Dilek

[permalink] [raw]
Subject: Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

On Sun, Jan 31, 2021 at 1:37 AM Fāng-ruì Sòng <[email protected]> wrote:
>
> On Sat, Jan 30, 2021 at 3:10 PM Sedat Dilek <[email protected]> wrote:
> >
> > On Sat, Jan 30, 2021 at 1:44 AM Nick Desaulniers
> > <[email protected]> wrote:
> > >
> > > DWARF v5 is the latest standard of the DWARF debug info format.
> > >
> > > Feature detection of DWARF5 is onerous, especially given that we've
> > > removed $(AS), so we must query $(CC) for DWARF5 assembler directive
> > > support.
> > >
> > > The DWARF version of a binary can be validated with:
> > > $ llvm-dwarfdump vmlinux | head -n 4 | grep version
> > > or
> > > $ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version
> > >
> > > DWARF5 wins significantly in terms of size when mixed with compression
> > > (CONFIG_DEBUG_INFO_COMPRESSED).
> > >
> > > 363M vmlinux.clang12.dwarf5.compressed
> > > 434M vmlinux.clang12.dwarf4.compressed
> > > 439M vmlinux.clang12.dwarf2.compressed
> > > 457M vmlinux.clang12.dwarf5
> > > 536M vmlinux.clang12.dwarf4
> > > 548M vmlinux.clang12.dwarf2
> > >
> > > 515M vmlinux.gcc10.2.dwarf5.compressed
> > > 599M vmlinux.gcc10.2.dwarf4.compressed
> > > 624M vmlinux.gcc10.2.dwarf2.compressed
> > > 630M vmlinux.gcc10.2.dwarf5
> > > 765M vmlinux.gcc10.2.dwarf4
> > > 809M vmlinux.gcc10.2.dwarf2
> > >
> > > Though the quality of debug info is harder to quantify; size is not a
> > > proxy for quality.
> > >
> > > Jakub notes:
> > > All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
> > > option that enabled some small DWARF subset (initially only a few
> > > DW_LANG_* codes newly added to DWARF5 drafts). Only GCC 7 (released
> > > after DWARF 5 has been finalized) started emitting DWARF5 section
> > > headers and got most of the DWARF5 changes in...
> > >
> > > Version check GCC so that we don't need to worry about the difference in
> > > command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
> > > validate the DWARF Version in the assembler feature detection script.
> > >
> > > GNU `as` only recently gained support for specifying -gdwarf-5, so when
> > > compiling with Clang but without Clang's integrated assembler
> > > (LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.
> > >
> > > Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
> > > recognize the new additions to the DWARF debug info. Thanks to Sedat for
> > > the report.
> > >
> > > Link: http://www.dwarfstd.org/doc/DWARF5.pdf
> > > Reported-by: Sedat Dilek <[email protected]>
> > > Suggested-by: Arvind Sankar <[email protected]>
> > > Suggested-by: Caroline Tice <[email protected]>
> > > Suggested-by: Fangrui Song <[email protected]>
> > > Suggested-by: Jakub Jelinek <[email protected]>
> > > Suggested-by: Masahiro Yamada <[email protected]>
> > > Suggested-by: Nathan Chancellor <[email protected]>
> > > Signed-off-by: Nick Desaulniers <[email protected]>
> > > ---
> > > Makefile | 1 +
> > > include/asm-generic/vmlinux.lds.h | 7 ++++++-
> > > lib/Kconfig.debug | 18 ++++++++++++++++++
> > > scripts/test_dwarf5_support.sh | 8 ++++++++
> > > 4 files changed, 33 insertions(+), 1 deletion(-)
> > > create mode 100755 scripts/test_dwarf5_support.sh
> > >
> > > diff --git a/Makefile b/Makefile
> > > index d2b4980807e0..5387a6f2f62d 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -831,6 +831,7 @@ KBUILD_AFLAGS += -Wa,-gdwarf-2
> > > endif
> > >
> > > dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> > > +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
> > > DEBUG_CFLAGS += -gdwarf-$(dwarf-version-y)
> > >
> > > ifdef CONFIG_DEBUG_INFO_REDUCED
> > > diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> > > index 34b7e0d2346c..1e7cde4bd3f9 100644
> > > --- a/include/asm-generic/vmlinux.lds.h
> > > +++ b/include/asm-generic/vmlinux.lds.h
> > > @@ -842,8 +842,13 @@
> > > /* DWARF 4 */ \
> > > .debug_types 0 : { *(.debug_types) } \
> > > /* DWARF 5 */ \
> > > + .debug_addr 0 : { *(.debug_addr) } \
> > > + .debug_line_str 0 : { *(.debug_line_str) } \
> > > + .debug_loclists 0 : { *(.debug_loclists) } \
> > > .debug_macro 0 : { *(.debug_macro) } \
> > > - .debug_addr 0 : { *(.debug_addr) }
> > > + .debug_names 0 : { *(.debug_names) } \
> > > + .debug_rnglists 0 : { *(.debug_rnglists) } \
> > > + .debug_str_offsets 0 : { *(.debug_str_offsets) }
> > >
> >
> > I just looked at binutils 2.36 in the Debian/experimental repositories.
> >
> > [1] says:
> >
> > + PR ld/27230
> > + * scripttempl/DWARF.sc: Add DWARF-5 .debug_* sections.
> >
> > ...
> >
> > - /* DWARF Extension. */
> > - .debug_macro 0 : { *(.debug_macro) }
> > + /* DWARF 5. */
> > .debug_addr 0 : { *(.debug_addr) }
> > + .debug_line_str 0 : { *(.debug_line_str) }
> > + .debug_loclists 0 : { *(.debug_loclists) }
> > + .debug_macro 0 : { *(.debug_macro) }
> > + .debug_names 0 : { *(.debug_names) }
> > + .debug_rnglists 0 : { *(.debug_rnglists) }
> > + .debug_str_offsets 0 : { *(.debug_str_offsets) }
> > + .debug_sup 0 : { *(.debug_sup) }
> >
> > The list of DWARF-5 .debug_* sections is alphabetically sorted.
> > AFAICS .debug_sup section is missing?
> >
> > - Sedat -
>
> No compiler produces .debug_sup section. It could be from some
> post-processing tool which is unrelated to the linker.
> Omitting it is fine.
>

Thanks for the clarification, Fangrui.

- Sedat -

> > [1] https://salsa.debian.org/toolchain-team/binutils/-/commit/f58f33081035672b01a04326a9c8daadbd09a430
> >
> > > /* Stabs debugging sections. */
> > > #define STABS_DEBUG \
> > > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > > index 94c1a7ed6306..ad6f78989d4f 100644
> > > --- a/lib/Kconfig.debug
> > > +++ b/lib/Kconfig.debug
> > > @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> > > It makes the debug information larger, but it significantly
> > > improves the success of resolving variables in gdb on optimized code.
> > >
> > > +config DEBUG_INFO_DWARF5
> > > + bool "Generate DWARF Version 5 debuginfo"
> > > + depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> > > + depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
> > > + depends on !DEBUG_INFO_BTF
> > > + help
> > > + Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
> > > + 5.0+ accepts the -gdwarf-5 flag but only had partial support for some
> > > + draft features until 7.0), and gdb 8.0+.
> > > +
> > > + Changes to the structure of debug info in Version 5 allow for around
> > > + 15-18% savings in resulting image and debug info section sizes as
> > > + compared to DWARF Version 4. DWARF Version 5 standardizes previous
> > > + extensions such as accelerators for symbol indexing and the format
> > > + for fission (.dwo/.dwp) files. Users may not want to select this
> > > + config if they rely on tooling that has not yet been updated to
> > > + support DWARF Version 5.
> > > +
> > > endchoice # "DWARF version"
> > >
> > > config DEBUG_INFO_BTF
> > > diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
> > > new file mode 100755
> > > index 000000000000..c46e2456b47a
> > > --- /dev/null
> > > +++ b/scripts/test_dwarf5_support.sh
> > > @@ -0,0 +1,8 @@
> > > +#!/bin/sh
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +
> > > +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> > > +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> > > +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
> > > +echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
> > > + $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
> > > --
> > > 2.30.0.365.g02bc693789-goog
> > >

2021-02-03 23:13:13

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

On Sat, Jan 30, 2021 at 9:44 AM 'Nick Desaulniers' via Clang Built
Linux <[email protected]> wrote:
>
> DWARF v5 is the latest standard of the DWARF debug info format.
>
> Feature detection of DWARF5 is onerous, especially given that we've
> removed $(AS), so we must query $(CC) for DWARF5 assembler directive
> support.
>
> The DWARF version of a binary can be validated with:
> $ llvm-dwarfdump vmlinux | head -n 4 | grep version
> or
> $ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version
>
> DWARF5 wins significantly in terms of size when mixed with compression
> (CONFIG_DEBUG_INFO_COMPRESSED).
>
> 363M vmlinux.clang12.dwarf5.compressed
> 434M vmlinux.clang12.dwarf4.compressed
> 439M vmlinux.clang12.dwarf2.compressed
> 457M vmlinux.clang12.dwarf5
> 536M vmlinux.clang12.dwarf4
> 548M vmlinux.clang12.dwarf2
>
> 515M vmlinux.gcc10.2.dwarf5.compressed
> 599M vmlinux.gcc10.2.dwarf4.compressed
> 624M vmlinux.gcc10.2.dwarf2.compressed
> 630M vmlinux.gcc10.2.dwarf5
> 765M vmlinux.gcc10.2.dwarf4
> 809M vmlinux.gcc10.2.dwarf2
>
> Though the quality of debug info is harder to quantify; size is not a
> proxy for quality.
>
> Jakub notes:
> All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
> option that enabled some small DWARF subset (initially only a few
> DW_LANG_* codes newly added to DWARF5 drafts). Only GCC 7 (released
> after DWARF 5 has been finalized) started emitting DWARF5 section
> headers and got most of the DWARF5 changes in...
>
> Version check GCC so that we don't need to worry about the difference in
> command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
> validate the DWARF Version in the assembler feature detection script.
>
> GNU `as` only recently gained support for specifying -gdwarf-5, so when
> compiling with Clang but without Clang's integrated assembler
> (LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.
>
> Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
> recognize the new additions to the DWARF debug info. Thanks to Sedat for
> the report.
>
> Link: http://www.dwarfstd.org/doc/DWARF5.pdf
> Reported-by: Sedat Dilek <[email protected]>
> Suggested-by: Arvind Sankar <[email protected]>
> Suggested-by: Caroline Tice <[email protected]>
> Suggested-by: Fangrui Song <[email protected]>
> Suggested-by: Jakub Jelinek <[email protected]>
> Suggested-by: Masahiro Yamada <[email protected]>
> Suggested-by: Nathan Chancellor <[email protected]>
> Signed-off-by: Nick Desaulniers <[email protected]>
> ---
> Makefile | 1 +
> include/asm-generic/vmlinux.lds.h | 7 ++++++-
> lib/Kconfig.debug | 18 ++++++++++++++++++
> scripts/test_dwarf5_support.sh | 8 ++++++++
> 4 files changed, 33 insertions(+), 1 deletion(-)
> create mode 100755 scripts/test_dwarf5_support.sh




Nick, the patch set is getting simpler and simpler,
and almost good enough to be merged.


Please let me ask two questions below.

There has been a lot of discussion, and
I might have missed the context.





> diff --git a/Makefile b/Makefile
> index d2b4980807e0..5387a6f2f62d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -831,6 +831,7 @@ KBUILD_AFLAGS += -Wa,-gdwarf-2
> endif
>
> dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
> DEBUG_CFLAGS += -gdwarf-$(dwarf-version-y)
>
> ifdef CONFIG_DEBUG_INFO_REDUCED
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index 34b7e0d2346c..1e7cde4bd3f9 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -842,8 +842,13 @@
> /* DWARF 4 */ \
> .debug_types 0 : { *(.debug_types) } \
> /* DWARF 5 */ \
> + .debug_addr 0 : { *(.debug_addr) } \
> + .debug_line_str 0 : { *(.debug_line_str) } \
> + .debug_loclists 0 : { *(.debug_loclists) } \
> .debug_macro 0 : { *(.debug_macro) } \
> - .debug_addr 0 : { *(.debug_addr) }
> + .debug_names 0 : { *(.debug_names) } \
> + .debug_rnglists 0 : { *(.debug_rnglists) } \
> + .debug_str_offsets 0 : { *(.debug_str_offsets) }
>
> /* Stabs debugging sections. */
> #define STABS_DEBUG \
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 94c1a7ed6306..ad6f78989d4f 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> It makes the debug information larger, but it significantly
> improves the success of resolving variables in gdb on optimized code.
>
> +config DEBUG_INFO_DWARF5
> + bool "Generate DWARF Version 5 debuginfo"
> + depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> + depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))

Q1.

This "CC_IS_GCC ||" was introduced by v4.

GCC never outputs '.file 0', which is why
this test is only needed for Clang, correct?






> + depends on !DEBUG_INFO_BTF
> + help
> + Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
> + 5.0+ accepts the -gdwarf-5 flag but only had partial support for some
> + draft features until 7.0), and gdb 8.0+.
> +
> + Changes to the structure of debug info in Version 5 allow for around
> + 15-18% savings in resulting image and debug info section sizes as
> + compared to DWARF Version 4. DWARF Version 5 standardizes previous
> + extensions such as accelerators for symbol indexing and the format
> + for fission (.dwo/.dwp) files. Users may not want to select this
> + config if they rely on tooling that has not yet been updated to
> + support DWARF Version 5.
> +
> endchoice # "DWARF version"
>
> config DEBUG_INFO_BTF
> diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
> new file mode 100755
> index 000000000000..c46e2456b47a
> --- /dev/null
> +++ b/scripts/test_dwarf5_support.sh
> @@ -0,0 +1,8 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0
> +
> +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611


I saw the following links in v6.

https://sourceware.org/bugzilla/show_bug.cgi?id=25612
https://sourceware.org/bugzilla/show_bug.cgi?id=25614

They were dropped in v7. Why?

I just thought they were good to know...



> +echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
> + $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
> --
> 2.30.0.365.g02bc693789-goog
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20210130004401.2528717-3-ndesaulniers%40google.com.



--
Best Regards
Masahiro Yamada

2021-02-03 23:31:31

by Nick Desaulniers

[permalink] [raw]
Subject: Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

On Wed, Feb 3, 2021 at 3:07 PM Masahiro Yamada <[email protected]> wrote:
>
> Nick, the patch set is getting simpler and simpler,
> and almost good enough to be merged.

I agree. I think Sedat pointed out a binutils 2.35.2 release; thanks
to Nick Clifton for that.

>
>
> Please let me ask two questions below.
>
> There has been a lot of discussion, and
> I might have missed the context.
>
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> > It makes the debug information larger, but it significantly
> > improves the success of resolving variables in gdb on optimized code.
> >
> > +config DEBUG_INFO_DWARF5
> > + bool "Generate DWARF Version 5 debuginfo"
> > + depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> > + depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
>
> Q1.
>
> This "CC_IS_GCC ||" was introduced by v4.
>
> GCC never outputs '.file 0', which is why
> this test is only needed for Clang, correct?

This test script is only needed when compiling with clang but without
its integrated assembler. It checks that when clang is used as the
driver, but GAS is used as the assembler, that GAS will be able to
decode the DWARF v5 assembler additions Clang will produce without
needing an explicit -Wa,-gdwarf-5 flag passed.

Technically, it is unnecessary for `LLVM=1 LLVM_IAS=1` or `CC=clang
LLVM_IAS=1` (ie. clang+clang's integrated assembler). But there is no
way to express AS_IS_IAS today in KConfig (similar to
CC_IS_{GCC|CLANG} or LD_IS_LLD). I don't think that's necessary;
whether or not clang's integrated assembler is used, when using clang,
run the simple check.

> > --- /dev/null
> > +++ b/scripts/test_dwarf5_support.sh
> > @@ -0,0 +1,8 @@
> > +#!/bin/sh
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> > +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> > +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
>
>
> I saw the following links in v6.
>
> https://sourceware.org/bugzilla/show_bug.cgi?id=25612
> https://sourceware.org/bugzilla/show_bug.cgi?id=25614
>
> They were dropped in v7. Why?
>
> I just thought they were good to know...

While having fixes for those bugs is required, technically
https://sourceware.org/bugzilla/show_bug.cgi?id=25611 is the latest
bug which was fixed. Testing for a fix of
https://sourceware.org/bugzilla/show_bug.cgi?id=25611 implies that
fixes for 25612 and 25614 exist due to the order they were fixed in
GAS. Technically, you could argue that this script is quite GAS
centric; given an arbitrary "assembler" the test should check a few
things. Realistically, I think that's overkill based on what
assemblers are in use today; we can always grow the script should we
identify other tests additional assemblers may need to pass, but until
then, I suspect YAGNI. Maybe there's a more precise name for the
script to reflect that, but that gets close to "what color shall we
paint the bikeshed?" Given the number of folks on the thread, plz no.
--
Thanks,
~Nick Desaulniers

2021-02-03 23:41:57

by Jakub Jelinek

[permalink] [raw]
Subject: Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

On Thu, Feb 04, 2021 at 08:06:12AM +0900, Masahiro Yamada wrote:
> GCC never outputs '.file 0', which is why
> this test is only needed for Clang, correct?

No, GCC outputs .file 0 if it during configure time detected assembler that
supports it and doesn't have any of the known bugs related to it.
But that means kernel doesn't need to care because GCC already took care of
that.

Jakub

2021-02-04 00:37:49

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

On Thu, Feb 4, 2021 at 8:36 AM Jakub Jelinek <[email protected]> wrote:
>
> On Thu, Feb 04, 2021 at 08:06:12AM +0900, Masahiro Yamada wrote:
> > GCC never outputs '.file 0', which is why
> > this test is only needed for Clang, correct?
>
> No, GCC outputs .file 0 if it during configure time detected assembler that
> supports it and doesn't have any of the known bugs related to it.
> But that means kernel doesn't need to care because GCC already took care of
> that.


OK, then I am fine with the current code.

Thanks for the clarification.







--
Best Regards
Masahiro Yamada

2021-02-04 00:38:53

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

On Thu, Feb 4, 2021 at 8:27 AM Nick Desaulniers <[email protected]> wrote:
>
> On Wed, Feb 3, 2021 at 3:07 PM Masahiro Yamada <[email protected]> wrote:
> >
> > Nick, the patch set is getting simpler and simpler,
> > and almost good enough to be merged.
>
> I agree. I think Sedat pointed out a binutils 2.35.2 release; thanks
> to Nick Clifton for that.
>
> >
> >
> > Please let me ask two questions below.
> >
> > There has been a lot of discussion, and
> > I might have missed the context.
> >
> > > --- a/lib/Kconfig.debug
> > > +++ b/lib/Kconfig.debug
> > > @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> > > It makes the debug information larger, but it significantly
> > > improves the success of resolving variables in gdb on optimized code.
> > >
> > > +config DEBUG_INFO_DWARF5
> > > + bool "Generate DWARF Version 5 debuginfo"
> > > + depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> > > + depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
> >
> > Q1.
> >
> > This "CC_IS_GCC ||" was introduced by v4.
> >
> > GCC never outputs '.file 0', which is why
> > this test is only needed for Clang, correct?
>
> This test script is only needed when compiling with clang but without
> its integrated assembler. It checks that when clang is used as the
> driver, but GAS is used as the assembler, that GAS will be able to
> decode the DWARF v5 assembler additions Clang will produce without
> needing an explicit -Wa,-gdwarf-5 flag passed.
>
> Technically, it is unnecessary for `LLVM=1 LLVM_IAS=1` or `CC=clang
> LLVM_IAS=1` (ie. clang+clang's integrated assembler). But there is no
> way to express AS_IS_IAS today in KConfig (similar to
> CC_IS_{GCC|CLANG} or LD_IS_LLD). I don't think that's necessary;
> whether or not clang's integrated assembler is used, when using clang,
> run the simple check.
>
> > > --- /dev/null
> > > +++ b/scripts/test_dwarf5_support.sh
> > > @@ -0,0 +1,8 @@
> > > +#!/bin/sh
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +
> > > +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> > > +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> > > +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
> >
> >
> > I saw the following links in v6.
> >
> > https://sourceware.org/bugzilla/show_bug.cgi?id=25612
> > https://sourceware.org/bugzilla/show_bug.cgi?id=25614
> >
> > They were dropped in v7. Why?
> >
> > I just thought they were good to know...
>
> While having fixes for those bugs is required, technically
> https://sourceware.org/bugzilla/show_bug.cgi?id=25611 is the latest
> bug which was fixed. Testing for a fix of
> https://sourceware.org/bugzilla/show_bug.cgi?id=25611 implies that
> fixes for 25612 and 25614 exist due to the order they were fixed in
> GAS.


It is difficult to know the patch order in the binutils project.

Personally, I prefer having all the three references here.
Otherwise, it is difficult to understand why
this script is doing such complex checks.





> Technically, you could argue that this script is quite GAS
> centric; given an arbitrary "assembler" the test should check a few
> things. Realistically, I think that's overkill based on what
> assemblers are in use today; we can always grow the script should we
> identify other tests additional assemblers may need to pass, but until
> then, I suspect YAGNI. Maybe there's a more precise name for the
> script to reflect that, but that gets close to "what color shall we
> paint the bikeshed?" Given the number of folks on the thread, plz no.


No argument with this regard. I agree with you.




> --
> Thanks,
> ~Nick Desaulniers



--
Best Regards
Masahiro Yamada