Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the
default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD
uses zstd.
Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:
clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
488M vmlinux
27.6% 136Mi 0.0% 0 .debug_info
6.1% 30.2Mi 0.0% 0 .debug_str_offsets
3.5% 17.2Mi 0.0% 0 .debug_line
3.3% 16.3Mi 0.0% 0 .debug_loclists
0.9% 4.62Mi 0.0% 0 .debug_str
clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
385M vmlinux
21.8% 85.4Mi 0.0% 0 .debug_info
2.1% 8.26Mi 0.0% 0 .debug_str_offsets
2.1% 8.24Mi 0.0% 0 .debug_loclists
1.9% 7.48Mi 0.0% 0 .debug_line
0.5% 1.94Mi 0.0% 0 .debug_str
clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
373M vmlinux
21.4% 81.4Mi 0.0% 0 .debug_info
2.3% 8.85Mi 0.0% 0 .debug_loclists
1.5% 5.71Mi 0.0% 0 .debug_line
0.5% 1.95Mi 0.0% 0 .debug_str_offsets
0.4% 1.62Mi 0.0% 0 .debug_str
That's only a 3.11% overall binary size savings over zlib, but at no
performance regression.
Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
Suggested-by: Sedat Dilek (DHL Supply Chain) <[email protected]>
Signed-off-by: Nick Desaulniers <[email protected]>
---
lib/Kconfig.debug | 26 +++++++++++++++++++++++++-
scripts/Makefile.debug | 4 ++++
2 files changed, 29 insertions(+), 1 deletion(-)
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 3fc7abffc7aa..4085ac77dc12 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -312,8 +312,22 @@ config DEBUG_INFO_REDUCED
DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.
+choice
+ prompt "Compressed Debug information"
+ depends on DEBUG_KERNEL
+ help
+ Compress the resulting debug info. Results in smaller debug info sections,
+ but requires that consumers are able to decompress the results.
+
+ If unsure, choose DEBUG_INFO_UNCOMPRESSED.
+
+config DEBUG_INFO_UNCOMPRESSED
+ bool "Don't compress debug information"
+ help
+ Don't compress debug info sections.
+
config DEBUG_INFO_COMPRESSED
- bool "Compressed debugging information"
+ bool "Compress debugging information with zlib"
depends on $(cc-option,-gz=zlib)
depends on $(ld-option,--compress-debug-sections=zlib)
help
@@ -327,6 +341,16 @@ config DEBUG_INFO_COMPRESSED
preferable to setting $KDEB_COMPRESS to "none" which would be even
larger.
+config DEBUG_INFO_COMPRESSED_ZSTD
+ bool "Compress debugging information with zstd"
+ depends on $(cc-option,-gz=zstd)
+ depends on $(ld-option,--compress-debug-sections=zstd)
+ help
+ Compress the debug information using zstd. Requires GCC 13.0+ or Clang
+ 16.0+, binutils 2.40+, and zstd.
+
+endchoice # "Compressed Debug information"
+
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on $(cc-option,-gsplit-dwarf)
diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
index 332c486f705f..8ac3379d2255 100644
--- a/scripts/Makefile.debug
+++ b/scripts/Makefile.debug
@@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
DEBUG_CFLAGS += -gz=zlib
KBUILD_AFLAGS += -gz=zlib
KBUILD_LDFLAGS += --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS += -gz=zstd
+KBUILD_AFLAGS += -gz=zstd
+KBUILD_LDFLAGS += --compress-debug-sections=zstd
endif
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
--
2.38.0.135.g90850a2211-goog
On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the
> default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD
> uses zstd.
>
> Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
> 488M vmlinux
> 27.6% 136Mi 0.0% 0 .debug_info
> 6.1% 30.2Mi 0.0% 0 .debug_str_offsets
> 3.5% 17.2Mi 0.0% 0 .debug_line
> 3.3% 16.3Mi 0.0% 0 .debug_loclists
> 0.9% 4.62Mi 0.0% 0 .debug_str
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
> 385M vmlinux
> 21.8% 85.4Mi 0.0% 0 .debug_info
> 2.1% 8.26Mi 0.0% 0 .debug_str_offsets
> 2.1% 8.24Mi 0.0% 0 .debug_loclists
> 1.9% 7.48Mi 0.0% 0 .debug_line
> 0.5% 1.94Mi 0.0% 0 .debug_str
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
> 373M vmlinux
> 21.4% 81.4Mi 0.0% 0 .debug_info
> 2.3% 8.85Mi 0.0% 0 .debug_loclists
> 1.5% 5.71Mi 0.0% 0 .debug_line
> 0.5% 1.95Mi 0.0% 0 .debug_str_offsets
> 0.4% 1.62Mi 0.0% 0 .debug_str
>
> That's only a 3.11% overall binary size savings over zlib, but at no
> performance regression.
>
> Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
> Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
> Suggested-by: Sedat Dilek (DHL Supply Chain) <[email protected]>
> Signed-off-by: Nick Desaulniers <[email protected]>
Reviewed-by: Nathan Chancellor <[email protected]>
One small comment below.
> ---
> lib/Kconfig.debug | 26 +++++++++++++++++++++++++-
> scripts/Makefile.debug | 4 ++++
> 2 files changed, 29 insertions(+), 1 deletion(-)
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 3fc7abffc7aa..4085ac77dc12 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -312,8 +312,22 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> +choice
> + prompt "Compressed Debug information"
> + depends on DEBUG_KERNEL
I think you can drop this depends. The entire block is in an
'if DEBUG_INFO', which can only be true if CONFIG_DEBUG_KERNEL is set
because of the dependencies of the "Debug information" prompt above this
file, which is how CONFIG_DEBUG_INFO is selected.
> + help
> + Compress the resulting debug info. Results in smaller debug info sections,
> + but requires that consumers are able to decompress the results.
> +
> + If unsure, choose DEBUG_INFO_UNCOMPRESSED.
> +
> +config DEBUG_INFO_UNCOMPRESSED
> + bool "Don't compress debug information"
> + help
> + Don't compress debug info sections.
> +
> config DEBUG_INFO_COMPRESSED
> - bool "Compressed debugging information"
> + bool "Compress debugging information with zlib"
> depends on $(cc-option,-gz=zlib)
> depends on $(ld-option,--compress-debug-sections=zlib)
> help
> @@ -327,6 +341,16 @@ config DEBUG_INFO_COMPRESSED
> preferable to setting $KDEB_COMPRESS to "none" which would be even
> larger.
>
> +config DEBUG_INFO_COMPRESSED_ZSTD
> + bool "Compress debugging information with zstd"
> + depends on $(cc-option,-gz=zstd)
> + depends on $(ld-option,--compress-debug-sections=zstd)
> + help
> + Compress the debug information using zstd. Requires GCC 13.0+ or Clang
> + 16.0+, binutils 2.40+, and zstd.
> +
> +endchoice # "Compressed Debug information"
> +
> config DEBUG_INFO_SPLIT
> bool "Produce split debuginfo in .dwo files"
> depends on $(cc-option,-gsplit-dwarf)
> diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
> index 332c486f705f..8ac3379d2255 100644
> --- a/scripts/Makefile.debug
> +++ b/scripts/Makefile.debug
> @@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
> DEBUG_CFLAGS += -gz=zlib
> KBUILD_AFLAGS += -gz=zlib
> KBUILD_LDFLAGS += --compress-debug-sections=zlib
> +else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
> +DEBUG_CFLAGS += -gz=zstd
> +KBUILD_AFLAGS += -gz=zstd
> +KBUILD_LDFLAGS += --compress-debug-sections=zstd
> endif
>
> KBUILD_CFLAGS += $(DEBUG_CFLAGS)
> --
> 2.38.0.135.g90850a2211-goog
>
On Thu, Oct 20, 2022 at 8:43 PM Nathan Chancellor <[email protected]> wrote:
>
> On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> > Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the
> > default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD
> > uses zstd.
> >
> > Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:
> >
> > clang-16, x86_64 defconfig plus
> > CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
> > Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
> > 488M vmlinux
> > 27.6% 136Mi 0.0% 0 .debug_info
> > 6.1% 30.2Mi 0.0% 0 .debug_str_offsets
> > 3.5% 17.2Mi 0.0% 0 .debug_line
> > 3.3% 16.3Mi 0.0% 0 .debug_loclists
> > 0.9% 4.62Mi 0.0% 0 .debug_str
> >
> > clang-16, x86_64 defconfig plus
> > CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
> > Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
> > 385M vmlinux
> > 21.8% 85.4Mi 0.0% 0 .debug_info
> > 2.1% 8.26Mi 0.0% 0 .debug_str_offsets
> > 2.1% 8.24Mi 0.0% 0 .debug_loclists
> > 1.9% 7.48Mi 0.0% 0 .debug_line
> > 0.5% 1.94Mi 0.0% 0 .debug_str
> >
> > clang-16, x86_64 defconfig plus
> > CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
> > Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
> > 373M vmlinux
> > 21.4% 81.4Mi 0.0% 0 .debug_info
> > 2.3% 8.85Mi 0.0% 0 .debug_loclists
> > 1.5% 5.71Mi 0.0% 0 .debug_line
> > 0.5% 1.95Mi 0.0% 0 .debug_str_offsets
> > 0.4% 1.62Mi 0.0% 0 .debug_str
> >
> > That's only a 3.11% overall binary size savings over zlib, but at no
> > performance regression.
> >
> > Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
> > Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
> > Suggested-by: Sedat Dilek (DHL Supply Chain) <[email protected]>
> > Signed-off-by: Nick Desaulniers <[email protected]>
>
> Reviewed-by: Nathan Chancellor <[email protected]>
>
> One small comment below.
>
> > ---
> > lib/Kconfig.debug | 26 +++++++++++++++++++++++++-
> > scripts/Makefile.debug | 4 ++++
> > 2 files changed, 29 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > index 3fc7abffc7aa..4085ac77dc12 100644
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -312,8 +312,22 @@ config DEBUG_INFO_REDUCED
> > DEBUG_INFO build and compile times are reduced too.
> > Only works with newer gcc versions.
> >
> > +choice
> > + prompt "Compressed Debug information"
> > + depends on DEBUG_KERNEL
>
> I think you can drop this depends. The entire block is in an
> 'if DEBUG_INFO', which can only be true if CONFIG_DEBUG_KERNEL is set
> because of the dependencies of the "Debug information" prompt above this
> file, which is how CONFIG_DEBUG_INFO is selected.
>
Yes, dropping this is good.
> > + help
> > + Compress the resulting debug info. Results in smaller debug info sections,
> > + but requires that consumers are able to decompress the results.
> > +
> > + If unsure, choose DEBUG_INFO_UNCOMPRESSED.
> > +
> > +config DEBUG_INFO_UNCOMPRESSED
> > + bool "Don't compress debug information"
> > + help
> > + Don't compress debug info sections.
> > +
> > config DEBUG_INFO_COMPRESSED
Also, I prefer...
DEBUG_INFO_COMPRESSED_NONE
DEBUG_INFO_COMPRESSED_ZLIB
DEBUG_INFO_COMPRESSED_ZSTD
Following...
$ ld.lld-16 -v
Debian LLD 16.0.0 (compatible with GNU linkers)
$ ld.lld-16 --help | grep compress-debug-sections
--compress-debug-sections=[none,zlib,zstd]
> > - bool "Compressed debugging information"
> > + bool "Compress debugging information with zlib"
> > depends on $(cc-option,-gz=zlib)
> > depends on $(ld-option,--compress-debug-sections=zlib)
> > help
> > @@ -327,6 +341,16 @@ config DEBUG_INFO_COMPRESSED
> > preferable to setting $KDEB_COMPRESS to "none" which would be even
> > larger.
> >
Anyone, checked $KDEB_COMPRESS - does this need changes?
-Sedat-
> > +config DEBUG_INFO_COMPRESSED_ZSTD
> > + bool "Compress debugging information with zstd"
> > + depends on $(cc-option,-gz=zstd)
> > + depends on $(ld-option,--compress-debug-sections=zstd)
> > + help
> > + Compress the debug information using zstd. Requires GCC 13.0+ or Clang
> > + 16.0+, binutils 2.40+, and zstd.
> > +
> > +endchoice # "Compressed Debug information"
> > +
> > config DEBUG_INFO_SPLIT
> > bool "Produce split debuginfo in .dwo files"
> > depends on $(cc-option,-gsplit-dwarf)
> > diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
> > index 332c486f705f..8ac3379d2255 100644
> > --- a/scripts/Makefile.debug
> > +++ b/scripts/Makefile.debug
> > @@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
> > DEBUG_CFLAGS += -gz=zlib
> > KBUILD_AFLAGS += -gz=zlib
> > KBUILD_LDFLAGS += --compress-debug-sections=zlib
> > +else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
> > +DEBUG_CFLAGS += -gz=zstd
> > +KBUILD_AFLAGS += -gz=zstd
> > +KBUILD_LDFLAGS += --compress-debug-sections=zstd
> > endif
> >
> > KBUILD_CFLAGS += $(DEBUG_CFLAGS)
> > --
> > 2.38.0.135.g90850a2211-goog
> >
On Sun, Oct 23, 2022 at 8:46 AM Sedat Dilek <[email protected]> wrote:
>
> On Thu, Oct 20, 2022 at 8:43 PM Nathan Chancellor <[email protected]> wrote:
> >
> > On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> > > +config DEBUG_INFO_UNCOMPRESSED
> > > + bool "Don't compress debug information"
> > > + help
> > > + Don't compress debug info sections.
> > > +
> > > config DEBUG_INFO_COMPRESSED
>
> Also, I prefer...
>
> DEBUG_INFO_COMPRESSED_NONE
> DEBUG_INFO_COMPRESSED_ZLIB
> DEBUG_INFO_COMPRESSED_ZSTD
>
> Following...
>
> $ ld.lld-16 -v
> Debian LLD 16.0.0 (compatible with GNU linkers)
>
> $ ld.lld-16 --help | grep compress-debug-sections
> --compress-debug-sections=[none,zlib,zstd]
It's a good suggestion, though I deliberately did not modify the
identifier used for the existing DEBUG_INFO_COMPRESSED. No defconfigs
set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with
changing the identifier of existing kconfigs is that upon upgrading to
newer kernel sources, if you run `make oldmodconfig` if you previously
had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has
been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently
disabled, and image sizes would increase due to uncompressed debug
info. That seems developer hostile.
Masahiro, what are your thoughts on how to proceed? Should I change
the pre-existing Kconfig identifier, or "let sleeping dogs lie?"
--
Thanks,
~Nick Desaulniers
-----Ursprüngliche Nachricht-----
Von: Nick Desaulniers <[email protected]>
Gesendet: Montag, 24. Oktober 2022 19:44
An: [email protected]; Masahiro Yamada <[email protected]>
Cc: Nathan Chancellor <[email protected]>; Fangrui Song <[email protected]>; Sedat Dilek (DHL Supply Chain) <[email protected]>; Michal Marek <[email protected]>; Nick Terrell <[email protected]>; Tom Rix <[email protected]>; Andrew Morton <[email protected]>; Peter Zijlstra (Intel) <[email protected]>; David Gow <[email protected]>; Kees Cook <[email protected]>; Josh Poimboeuf <[email protected]>; Dan Williams <[email protected]>; Miguel Ojeda <[email protected]>; Isabella Basso <[email protected]>; Vlastimil Babka <[email protected]>; Rasmus Villemoes <[email protected]>; [email protected]; [email protected]; [email protected]
Betreff: Re: [PATCH] Makefile.debug: support for -gz=zstd
On Sun, Oct 23, 2022 at 8:46 AM Sedat Dilek <[email protected]> wrote:
>
> On Thu, Oct 20, 2022 at 8:43 PM Nathan Chancellor <[email protected]> wrote:
> >
> > On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> > > +config DEBUG_INFO_UNCOMPRESSED
> > > + bool "Don't compress debug information"
> > > + help
> > > + Don't compress debug info sections.
> > > +
> > > config DEBUG_INFO_COMPRESSED
>
> Also, I prefer...
>
> DEBUG_INFO_COMPRESSED_NONE
> DEBUG_INFO_COMPRESSED_ZLIB
> DEBUG_INFO_COMPRESSED_ZSTD
>
> Following...
>
> $ ld.lld-16 -v
> Debian LLD 16.0.0 (compatible with GNU linkers)
>
> $ ld.lld-16 --help | grep compress-debug-sections
> --compress-debug-sections=[none,zlib,zstd]
It's a good suggestion, though I deliberately did not modify the identifier used for the existing DEBUG_INFO_COMPRESSED. No defconfigs set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with changing the identifier of existing kconfigs is that upon upgrading to newer kernel sources, if you run `make oldmodconfig` if you previously had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently disabled, and image sizes would increase due to uncompressed debug info. That seems developer hostile.
Masahiro, what are your thoughts on how to proceed? Should I change the pre-existing Kconfig identifier, or "let sleeping dogs lie?"
***
Cannot say what happens when CONFIG_XXX changes - I have not played with your patch.
Perhaps, we can use CONFIG_ DEBUG_INFO_COMPRESSED="none|zlib|zstd" with default: "none"?
-Sedat-
On Mon, Oct 24, 2022 at 10:44 AM Nick Desaulniers
<[email protected]> wrote:
>
> On Sun, Oct 23, 2022 at 8:46 AM Sedat Dilek <[email protected]> wrote:
> >
> > On Thu, Oct 20, 2022 at 8:43 PM Nathan Chancellor <[email protected]> wrote:
> > >
> > > On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> > > > +config DEBUG_INFO_UNCOMPRESSED
> > > > + bool "Don't compress debug information"
> > > > + help
> > > > + Don't compress debug info sections.
> > > > +
> > > > config DEBUG_INFO_COMPRESSED
> >
> > Also, I prefer...
> >
> > DEBUG_INFO_COMPRESSED_NONE
> > DEBUG_INFO_COMPRESSED_ZLIB
> > DEBUG_INFO_COMPRESSED_ZSTD
> >
> > Following...
> >
> > $ ld.lld-16 -v
> > Debian LLD 16.0.0 (compatible with GNU linkers)
> >
> > $ ld.lld-16 --help | grep compress-debug-sections
> > --compress-debug-sections=[none,zlib,zstd]
>
> It's a good suggestion, though I deliberately did not modify the
> identifier used for the existing DEBUG_INFO_COMPRESSED. No defconfigs
> set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with
> changing the identifier of existing kconfigs is that upon upgrading to
> newer kernel sources, if you run `make oldmodconfig` if you previously
> had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has
> been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently
> disabled, and image sizes would increase due to uncompressed debug
> info. That seems developer hostile.
>
> Masahiro, what are your thoughts on how to proceed? Should I change
> the pre-existing Kconfig identifier, or "let sleeping dogs lie?"
Hi Masahiro,
What are your thoughts on the above?
> --
> Thanks,
> ~Nick Desaulniers
--
Thanks,
~Nick Desaulniers
On Tue, Nov 1, 2022 at 3:49 AM Nick Desaulniers <[email protected]> wrote:
>
> On Mon, Oct 24, 2022 at 10:44 AM Nick Desaulniers
> <[email protected]> wrote:
> >
> > On Sun, Oct 23, 2022 at 8:46 AM Sedat Dilek <[email protected]> wrote:
> > >
> > > On Thu, Oct 20, 2022 at 8:43 PM Nathan Chancellor <[email protected]> wrote:
> > > >
> > > > On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> > > > > +config DEBUG_INFO_UNCOMPRESSED
> > > > > + bool "Don't compress debug information"
> > > > > + help
> > > > > + Don't compress debug info sections.
> > > > > +
> > > > > config DEBUG_INFO_COMPRESSED
> > >
> > > Also, I prefer...
> > >
> > > DEBUG_INFO_COMPRESSED_NONE
> > > DEBUG_INFO_COMPRESSED_ZLIB
> > > DEBUG_INFO_COMPRESSED_ZSTD
> > >
> > > Following...
> > >
> > > $ ld.lld-16 -v
> > > Debian LLD 16.0.0 (compatible with GNU linkers)
> > >
> > > $ ld.lld-16 --help | grep compress-debug-sections
> > > --compress-debug-sections=[none,zlib,zstd]
> >
> > It's a good suggestion, though I deliberately did not modify the
> > identifier used for the existing DEBUG_INFO_COMPRESSED. No defconfigs
> > set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with
> > changing the identifier of existing kconfigs is that upon upgrading to
> > newer kernel sources, if you run `make oldmodconfig` if you previously
> > had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has
> > been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently
> > disabled, and image sizes would increase due to uncompressed debug
> > info. That seems developer hostile.
> >
> > Masahiro, what are your thoughts on how to proceed? Should I change
> > the pre-existing Kconfig identifier, or "let sleeping dogs lie?"
>
> Hi Masahiro,
> What are your thoughts on the above?
We renamed CONFIG options from time to time.
We should strive to update in-tree defconfigs,
but we did not care about external users very much.
(we rely on "make oldconfig" showing new options
and asking users for their choice.)
So, I like Sedat's DEBUG_INFO_COMPRESSED_{ZLIB,ZSTD,NONE}.
(and it is consist with CONFIG_INITRAMFS_COMPRESSION_*)
--
Best Regards
Masahiro Yamada
Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the
default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD
uses zstd.
Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:
clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
488M vmlinux
27.6% 136Mi 0.0% 0 .debug_info
6.1% 30.2Mi 0.0% 0 .debug_str_offsets
3.5% 17.2Mi 0.0% 0 .debug_line
3.3% 16.3Mi 0.0% 0 .debug_loclists
0.9% 4.62Mi 0.0% 0 .debug_str
clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
385M vmlinux
21.8% 85.4Mi 0.0% 0 .debug_info
2.1% 8.26Mi 0.0% 0 .debug_str_offsets
2.1% 8.24Mi 0.0% 0 .debug_loclists
1.9% 7.48Mi 0.0% 0 .debug_line
0.5% 1.94Mi 0.0% 0 .debug_str
clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
373M vmlinux
21.4% 81.4Mi 0.0% 0 .debug_info
2.3% 8.85Mi 0.0% 0 .debug_loclists
1.5% 5.71Mi 0.0% 0 .debug_line
0.5% 1.95Mi 0.0% 0 .debug_str_offsets
0.4% 1.62Mi 0.0% 0 .debug_str
That's only a 3.11% overall binary size savings over zlib, but at no
performance regression.
This renames the existing KConfig options:
* DEBUG_INFO_UNCOMPRESSED -> DEBUG_INFO_COMPRESSED_NONE
* DEBUG_INFO_COMPRESSED -> DEBUG_INFO_COMPRESSED_ZLIB
So users upgrading may need to reset the new Kconfigs.
Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
Suggested-by: Sedat Dilek (DHL Supply Chain) <[email protected]>
Reviewed-by: Nathan Chancellor <[email protected]>
Signed-off-by: Nick Desaulniers <[email protected]>
---
Changes v1 -> v2:
* Remove `depends on DEBUG_KERNEL` as per Nathan.
* Rename Kconfigs as per Sedat and Masahiro.
* Add note about renamed Kconfigs to commit message.
* Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.
lib/Kconfig.debug | 29 +++++++++++++++++++++++++++--
scripts/Makefile.debug | 4 ++++
2 files changed, 31 insertions(+), 2 deletions(-)
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 29280072dc0e..7c28a8fba02e 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -312,8 +312,21 @@ config DEBUG_INFO_REDUCED
DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.
-config DEBUG_INFO_COMPRESSED
- bool "Compressed debugging information"
+choice
+ prompt "Compressed Debug information"
+ help
+ Compress the resulting debug info. Results in smaller debug info sections,
+ but requires that consumers are able to decompress the results.
+
+ If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
+
+config DEBUG_INFO_COMPRESSED_NONE
+ bool "Don't compress debug information"
+ help
+ Don't compress debug info sections.
+
+config DEBUG_INFO_COMPRESSED_ZLIB
+ bool "Compress debugging information with zlib"
depends on $(cc-option,-gz=zlib)
depends on $(ld-option,--compress-debug-sections=zlib)
help
@@ -327,6 +340,18 @@ config DEBUG_INFO_COMPRESSED
preferable to setting $KDEB_COMPRESS to "none" which would be even
larger.
+config DEBUG_INFO_COMPRESSED_ZSTD
+ bool "Compress debugging information with zstd"
+ depends on $(cc-option,-gz=zstd)
+ depends on $(ld-option,--compress-debug-sections=zstd)
+ help
+ Compress the debug information using zstd. This may provide better
+ compression than zlib, for about the same time costs, but requires newer
+ toolchain support. Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
+ zstd.
+
+endchoice # "Compressed Debug information"
+
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on $(cc-option,-gsplit-dwarf)
diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
index 332c486f705f..8ac3379d2255 100644
--- a/scripts/Makefile.debug
+++ b/scripts/Makefile.debug
@@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
DEBUG_CFLAGS += -gz=zlib
KBUILD_AFLAGS += -gz=zlib
KBUILD_LDFLAGS += --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS += -gz=zstd
+KBUILD_AFLAGS += -gz=zstd
+KBUILD_LDFLAGS += --compress-debug-sections=zstd
endif
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
base-commit: f0c4d9fc9cc9462659728d168387191387e903cc
--
2.38.1.431.g37b22c650d-goog
On Mon, Nov 07, 2022 at 10:01:33AM -0800, Nick Desaulniers wrote:
> Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the
> default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD
> uses zstd.
>
> Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
> 488M vmlinux
> 27.6% 136Mi 0.0% 0 .debug_info
> 6.1% 30.2Mi 0.0% 0 .debug_str_offsets
> 3.5% 17.2Mi 0.0% 0 .debug_line
> 3.3% 16.3Mi 0.0% 0 .debug_loclists
> 0.9% 4.62Mi 0.0% 0 .debug_str
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
> 385M vmlinux
> 21.8% 85.4Mi 0.0% 0 .debug_info
> 2.1% 8.26Mi 0.0% 0 .debug_str_offsets
> 2.1% 8.24Mi 0.0% 0 .debug_loclists
> 1.9% 7.48Mi 0.0% 0 .debug_line
> 0.5% 1.94Mi 0.0% 0 .debug_str
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
> 373M vmlinux
> 21.4% 81.4Mi 0.0% 0 .debug_info
> 2.3% 8.85Mi 0.0% 0 .debug_loclists
> 1.5% 5.71Mi 0.0% 0 .debug_line
> 0.5% 1.95Mi 0.0% 0 .debug_str_offsets
> 0.4% 1.62Mi 0.0% 0 .debug_str
>
> That's only a 3.11% overall binary size savings over zlib, but at no
> performance regression.
>
> This renames the existing KConfig options:
> * DEBUG_INFO_UNCOMPRESSED -> DEBUG_INFO_COMPRESSED_NONE
> * DEBUG_INFO_COMPRESSED -> DEBUG_INFO_COMPRESSED_ZLIB
Might you want to update the DEBUG_INFO_(UN|)COMPRESSED above to the new
names?
Kind regards,
Nicolas
> So users upgrading may need to reset the new Kconfigs.
>
> Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
> Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
> Suggested-by: Sedat Dilek (DHL Supply Chain) <[email protected]>
> Reviewed-by: Nathan Chancellor <[email protected]>
> Signed-off-by: Nick Desaulniers <[email protected]>
> ---
> Changes v1 -> v2:
> * Remove `depends on DEBUG_KERNEL` as per Nathan.
> * Rename Kconfigs as per Sedat and Masahiro.
> * Add note about renamed Kconfigs to commit message.
> * Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.
>
> lib/Kconfig.debug | 29 +++++++++++++++++++++++++++--
> scripts/Makefile.debug | 4 ++++
> 2 files changed, 31 insertions(+), 2 deletions(-)
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 29280072dc0e..7c28a8fba02e 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -312,8 +312,21 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> -config DEBUG_INFO_COMPRESSED
> - bool "Compressed debugging information"
> +choice
> + prompt "Compressed Debug information"
> + help
> + Compress the resulting debug info. Results in smaller debug info sections,
> + but requires that consumers are able to decompress the results.
> +
> + If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
> +
> +config DEBUG_INFO_COMPRESSED_NONE
> + bool "Don't compress debug information"
> + help
> + Don't compress debug info sections.
> +
> +config DEBUG_INFO_COMPRESSED_ZLIB
> + bool "Compress debugging information with zlib"
> depends on $(cc-option,-gz=zlib)
> depends on $(ld-option,--compress-debug-sections=zlib)
> help
> @@ -327,6 +340,18 @@ config DEBUG_INFO_COMPRESSED
> preferable to setting $KDEB_COMPRESS to "none" which would be even
> larger.
>
> +config DEBUG_INFO_COMPRESSED_ZSTD
> + bool "Compress debugging information with zstd"
> + depends on $(cc-option,-gz=zstd)
> + depends on $(ld-option,--compress-debug-sections=zstd)
> + help
> + Compress the debug information using zstd. This may provide better
> + compression than zlib, for about the same time costs, but requires newer
> + toolchain support. Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
> + zstd.
> +
> +endchoice # "Compressed Debug information"
> +
> config DEBUG_INFO_SPLIT
> bool "Produce split debuginfo in .dwo files"
> depends on $(cc-option,-gsplit-dwarf)
> diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
> index 332c486f705f..8ac3379d2255 100644
> --- a/scripts/Makefile.debug
> +++ b/scripts/Makefile.debug
> @@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
> DEBUG_CFLAGS += -gz=zlib
> KBUILD_AFLAGS += -gz=zlib
> KBUILD_LDFLAGS += --compress-debug-sections=zlib
> +else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
> +DEBUG_CFLAGS += -gz=zstd
> +KBUILD_AFLAGS += -gz=zstd
> +KBUILD_LDFLAGS += --compress-debug-sections=zstd
> endif
>
> KBUILD_CFLAGS += $(DEBUG_CFLAGS)
>
> base-commit: f0c4d9fc9cc9462659728d168387191387e903cc
> --
> 2.38.1.431.g37b22c650d-goog
>
On Mon, Nov 7, 2022 at 7:08 PM Nick Desaulniers <[email protected]> wrote:
>
> Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the
> default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD
> uses zstd.
>
> Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
> 488M vmlinux
> 27.6% 136Mi 0.0% 0 .debug_info
> 6.1% 30.2Mi 0.0% 0 .debug_str_offsets
> 3.5% 17.2Mi 0.0% 0 .debug_line
> 3.3% 16.3Mi 0.0% 0 .debug_loclists
> 0.9% 4.62Mi 0.0% 0 .debug_str
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
> 385M vmlinux
> 21.8% 85.4Mi 0.0% 0 .debug_info
> 2.1% 8.26Mi 0.0% 0 .debug_str_offsets
> 2.1% 8.24Mi 0.0% 0 .debug_loclists
> 1.9% 7.48Mi 0.0% 0 .debug_line
> 0.5% 1.94Mi 0.0% 0 .debug_str
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
> 373M vmlinux
> 21.4% 81.4Mi 0.0% 0 .debug_info
> 2.3% 8.85Mi 0.0% 0 .debug_loclists
> 1.5% 5.71Mi 0.0% 0 .debug_line
> 0.5% 1.95Mi 0.0% 0 .debug_str_offsets
> 0.4% 1.62Mi 0.0% 0 .debug_str
>
> That's only a 3.11% overall binary size savings over zlib, but at no
> performance regression.
>
> This renames the existing KConfig options:
> * DEBUG_INFO_UNCOMPRESSED -> DEBUG_INFO_COMPRESSED_NONE
> * DEBUG_INFO_COMPRESSED -> DEBUG_INFO_COMPRESSED_ZLIB
> So users upgrading may need to reset the new Kconfigs.
>
> Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
> Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
> Suggested-by: Sedat Dilek (DHL Supply Chain) <[email protected]>
> Reviewed-by: Nathan Chancellor <[email protected]>
> Signed-off-by: Nick Desaulniers <[email protected]>
> ---
> Changes v1 -> v2:
> * Remove `depends on DEBUG_KERNEL` as per Nathan.
> * Rename Kconfigs as per Sedat and Masahiro.
> * Add note about renamed Kconfigs to commit message.
> * Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.
>
> lib/Kconfig.debug | 29 +++++++++++++++++++++++++++--
> scripts/Makefile.debug | 4 ++++
> 2 files changed, 31 insertions(+), 2 deletions(-)
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 29280072dc0e..7c28a8fba02e 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -312,8 +312,21 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> -config DEBUG_INFO_COMPRESSED
> - bool "Compressed debugging information"
> +choice
> + prompt "Compressed Debug information"
> + help
> + Compress the resulting debug info. Results in smaller debug info sections,
> + but requires that consumers are able to decompress the results.
> +
> + If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
> +
> +config DEBUG_INFO_COMPRESSED_NONE
> + bool "Don't compress debug information"
> + help
> + Don't compress debug info sections.
> +
> +config DEBUG_INFO_COMPRESSED_ZLIB
> + bool "Compress debugging information with zlib"
> depends on $(cc-option,-gz=zlib)
> depends on $(ld-option,--compress-debug-sections=zlib)
> help
> @@ -327,6 +340,18 @@ config DEBUG_INFO_COMPRESSED
> preferable to setting $KDEB_COMPRESS to "none" which would be even
> larger.
>
> +config DEBUG_INFO_COMPRESSED_ZSTD
> + bool "Compress debugging information with zstd"
> + depends on $(cc-option,-gz=zstd)
> + depends on $(ld-option,--compress-debug-sections=zstd)
> + help
> + Compress the debug information using zstd. This may provide better
> + compression than zlib, for about the same time costs, but requires newer
> + toolchain support. Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
> + zstd.
> +
> +endchoice # "Compressed Debug information"
> +
> config DEBUG_INFO_SPLIT
> bool "Produce split debuginfo in .dwo files"
> depends on $(cc-option,-gsplit-dwarf)
> diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
> index 332c486f705f..8ac3379d2255 100644
> --- a/scripts/Makefile.debug
> +++ b/scripts/Makefile.debug
> @@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
^^ ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZLIB
...from a quick (re)view.
-Sedat-
> DEBUG_CFLAGS += -gz=zlib
> KBUILD_AFLAGS += -gz=zlib
> KBUILD_LDFLAGS += --compress-debug-sections=zlib
> +else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
> +DEBUG_CFLAGS += -gz=zstd
> +KBUILD_AFLAGS += -gz=zstd
> +KBUILD_LDFLAGS += --compress-debug-sections=zstd
> endif
>
> KBUILD_CFLAGS += $(DEBUG_CFLAGS)
>
> base-commit: f0c4d9fc9cc9462659728d168387191387e903cc
> --
> 2.38.1.431.g37b22c650d-goog
>
Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_COMPRESSED_NONE is the
default, DEBUG_INFO_COMPRESSED_ZLIB uses zlib,
DEBUG_INFO_COMPRESSED_ZSTD uses zstd.
This renames the existing KConfig option DEBUG_INFO_COMPRESSED to
DEBUG_INFO_COMPRESSED_ZLIB so users upgrading may need to reset the new
Kconfigs.
Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:
clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_NONE=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
488M vmlinux
27.6% 136Mi 0.0% 0 .debug_info
6.1% 30.2Mi 0.0% 0 .debug_str_offsets
3.5% 17.2Mi 0.0% 0 .debug_line
3.3% 16.3Mi 0.0% 0 .debug_loclists
0.9% 4.62Mi 0.0% 0 .debug_str
clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZLIB=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
385M vmlinux
21.8% 85.4Mi 0.0% 0 .debug_info
2.1% 8.26Mi 0.0% 0 .debug_str_offsets
2.1% 8.24Mi 0.0% 0 .debug_loclists
1.9% 7.48Mi 0.0% 0 .debug_line
0.5% 1.94Mi 0.0% 0 .debug_str
clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
373M vmlinux
21.4% 81.4Mi 0.0% 0 .debug_info
2.3% 8.85Mi 0.0% 0 .debug_loclists
1.5% 5.71Mi 0.0% 0 .debug_line
0.5% 1.95Mi 0.0% 0 .debug_str_offsets
0.4% 1.62Mi 0.0% 0 .debug_str
That's only a 3.11% overall binary size savings over zlib, but at no
performance regression.
Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
Suggested-by: Sedat Dilek (DHL Supply Chain) <[email protected]>
Reviewed-by: Nathan Chancellor <[email protected]>
Signed-off-by: Nick Desaulniers <[email protected]>
---
Changes v2 -> v3:
* Fix scripts/Makefile.debug as per Sedat.
* Update commit message as per Nicolas.
Changes v1 -> v2:
* Remove `depends on DEBUG_KERNEL` as per Nathan.
* Rename Kconfigs as per Sedat and Masahiro.
* Add note about renamed Kconfigs to commit message.
* Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.
lib/Kconfig.debug | 29 +++++++++++++++++++++++++++--
scripts/Makefile.debug | 6 +++++-
2 files changed, 32 insertions(+), 3 deletions(-)
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 29280072dc0e..7c28a8fba02e 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -312,8 +312,21 @@ config DEBUG_INFO_REDUCED
DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.
-config DEBUG_INFO_COMPRESSED
- bool "Compressed debugging information"
+choice
+ prompt "Compressed Debug information"
+ help
+ Compress the resulting debug info. Results in smaller debug info sections,
+ but requires that consumers are able to decompress the results.
+
+ If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
+
+config DEBUG_INFO_COMPRESSED_NONE
+ bool "Don't compress debug information"
+ help
+ Don't compress debug info sections.
+
+config DEBUG_INFO_COMPRESSED_ZLIB
+ bool "Compress debugging information with zlib"
depends on $(cc-option,-gz=zlib)
depends on $(ld-option,--compress-debug-sections=zlib)
help
@@ -327,6 +340,18 @@ config DEBUG_INFO_COMPRESSED
preferable to setting $KDEB_COMPRESS to "none" which would be even
larger.
+config DEBUG_INFO_COMPRESSED_ZSTD
+ bool "Compress debugging information with zstd"
+ depends on $(cc-option,-gz=zstd)
+ depends on $(ld-option,--compress-debug-sections=zstd)
+ help
+ Compress the debug information using zstd. This may provide better
+ compression than zlib, for about the same time costs, but requires newer
+ toolchain support. Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
+ zstd.
+
+endchoice # "Compressed Debug information"
+
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on $(cc-option,-gsplit-dwarf)
diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
index 332c486f705f..059ff38fe0cb 100644
--- a/scripts/Makefile.debug
+++ b/scripts/Makefile.debug
@@ -27,10 +27,14 @@ else
DEBUG_RUSTFLAGS += -Cdebuginfo=2
endif
-ifdef CONFIG_DEBUG_INFO_COMPRESSED
+ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZLIB
DEBUG_CFLAGS += -gz=zlib
KBUILD_AFLAGS += -gz=zlib
KBUILD_LDFLAGS += --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS += -gz=zstd
+KBUILD_AFLAGS += -gz=zstd
+KBUILD_LDFLAGS += --compress-debug-sections=zstd
endif
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
base-commit: 1767a722a708f1fa3b9af39eb091d79101f8c086
--
2.38.1.431.g37b22c650d-goog
On Fri, Nov 11, 2022 at 5:00 AM Nick Desaulniers
<[email protected]> wrote:
>
> Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_COMPRESSED_NONE is the
> default, DEBUG_INFO_COMPRESSED_ZLIB uses zlib,
> DEBUG_INFO_COMPRESSED_ZSTD uses zstd.
>
> This renames the existing KConfig option DEBUG_INFO_COMPRESSED to
> DEBUG_INFO_COMPRESSED_ZLIB so users upgrading may need to reset the new
> Kconfigs.
>
> Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_NONE=y:
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
> 488M vmlinux
> 27.6% 136Mi 0.0% 0 .debug_info
> 6.1% 30.2Mi 0.0% 0 .debug_str_offsets
> 3.5% 17.2Mi 0.0% 0 .debug_line
> 3.3% 16.3Mi 0.0% 0 .debug_loclists
> 0.9% 4.62Mi 0.0% 0 .debug_str
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZLIB=y:
> Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
> 385M vmlinux
> 21.8% 85.4Mi 0.0% 0 .debug_info
> 2.1% 8.26Mi 0.0% 0 .debug_str_offsets
> 2.1% 8.24Mi 0.0% 0 .debug_loclists
> 1.9% 7.48Mi 0.0% 0 .debug_line
> 0.5% 1.94Mi 0.0% 0 .debug_str
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y:
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
> 373M vmlinux
> 21.4% 81.4Mi 0.0% 0 .debug_info
> 2.3% 8.85Mi 0.0% 0 .debug_loclists
> 1.5% 5.71Mi 0.0% 0 .debug_line
> 0.5% 1.95Mi 0.0% 0 .debug_str_offsets
> 0.4% 1.62Mi 0.0% 0 .debug_str
>
> That's only a 3.11% overall binary size savings over zlib, but at no
> performance regression.
>
> Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
> Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
> Suggested-by: Sedat Dilek (DHL Supply Chain) <[email protected]>
> Reviewed-by: Nathan Chancellor <[email protected]>
> Signed-off-by: Nick Desaulniers <[email protected]>
> ---
Applied to linux-kbuild.
Thanks.
> Changes v2 -> v3:
> * Fix scripts/Makefile.debug as per Sedat.
> * Update commit message as per Nicolas.
>
> Changes v1 -> v2:
> * Remove `depends on DEBUG_KERNEL` as per Nathan.
> * Rename Kconfigs as per Sedat and Masahiro.
> * Add note about renamed Kconfigs to commit message.
> * Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.
>
>
> lib/Kconfig.debug | 29 +++++++++++++++++++++++++++--
> scripts/Makefile.debug | 6 +++++-
> 2 files changed, 32 insertions(+), 3 deletions(-)
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 29280072dc0e..7c28a8fba02e 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -312,8 +312,21 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> -config DEBUG_INFO_COMPRESSED
> - bool "Compressed debugging information"
> +choice
> + prompt "Compressed Debug information"
> + help
> + Compress the resulting debug info. Results in smaller debug info sections,
> + but requires that consumers are able to decompress the results.
> +
> + If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
> +
> +config DEBUG_INFO_COMPRESSED_NONE
> + bool "Don't compress debug information"
> + help
> + Don't compress debug info sections.
> +
> +config DEBUG_INFO_COMPRESSED_ZLIB
> + bool "Compress debugging information with zlib"
> depends on $(cc-option,-gz=zlib)
> depends on $(ld-option,--compress-debug-sections=zlib)
> help
> @@ -327,6 +340,18 @@ config DEBUG_INFO_COMPRESSED
> preferable to setting $KDEB_COMPRESS to "none" which would be even
> larger.
>
> +config DEBUG_INFO_COMPRESSED_ZSTD
> + bool "Compress debugging information with zstd"
> + depends on $(cc-option,-gz=zstd)
> + depends on $(ld-option,--compress-debug-sections=zstd)
> + help
> + Compress the debug information using zstd. This may provide better
> + compression than zlib, for about the same time costs, but requires newer
> + toolchain support. Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
> + zstd.
> +
> +endchoice # "Compressed Debug information"
> +
> config DEBUG_INFO_SPLIT
> bool "Produce split debuginfo in .dwo files"
> depends on $(cc-option,-gsplit-dwarf)
> diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
> index 332c486f705f..059ff38fe0cb 100644
> --- a/scripts/Makefile.debug
> +++ b/scripts/Makefile.debug
> @@ -27,10 +27,14 @@ else
> DEBUG_RUSTFLAGS += -Cdebuginfo=2
> endif
>
> -ifdef CONFIG_DEBUG_INFO_COMPRESSED
> +ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZLIB
> DEBUG_CFLAGS += -gz=zlib
> KBUILD_AFLAGS += -gz=zlib
> KBUILD_LDFLAGS += --compress-debug-sections=zlib
> +else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
> +DEBUG_CFLAGS += -gz=zstd
> +KBUILD_AFLAGS += -gz=zstd
> +KBUILD_LDFLAGS += --compress-debug-sections=zstd
> endif
>
> KBUILD_CFLAGS += $(DEBUG_CFLAGS)
>
> base-commit: 1767a722a708f1fa3b9af39eb091d79101f8c086
> --
> 2.38.1.431.g37b22c650d-goog
>
--
Best Regards
Masahiro Yamada
On Sun, Nov 13, 2022 at 12:01 PM Masahiro Yamada <[email protected]> wrote:
>
> On Fri, Nov 11, 2022 at 5:00 AM Nick Desaulniers
> <[email protected]> wrote:
...
>
> Applied to linux-kbuild.
> Thanks.
...
> > Changes v2 -> v3:
> > * Fix scripts/Makefile.debug as per Sedat.
> > * Update commit message as per Nicolas.
> >
> > Changes v1 -> v2:
> > * Remove `depends on DEBUG_KERNEL` as per Nathan.
> > * Rename Kconfigs as per Sedat and Masahiro.
> > * Add note about renamed Kconfigs to commit message.
> > * Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.
...
> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > index 29280072dc0e..7c28a8fba02e 100644
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
...
> > +config DEBUG_INFO_COMPRESSED_ZSTD
> > + bool "Compress debugging information with zstd"
> > + depends on $(cc-option,-gz=zstd)
> > + depends on $(ld-option,--compress-debug-sections=zstd)
> > + help
> > + Compress the debug information using zstd. This may provide better
> > + compression than zlib, for about the same time costs, but requires newer
> > + toolchain support. Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
> > + zstd.
> > +
> > +endchoice # "Compressed Debug information"
Just saw:
GNU Binutils 2.40 Branched With Zstd Support, Zen 4, New Intel Instructions
Debian GNU/Linux ships version 2.39.90 in unstable release.
Maybe someone can point me to the release schedule of LLVM-16?
Thanks.
Best regards,
-Sedat-
[1] https://www.phoronix.com/news/GNU-Binutils-2.40
[2] https://metadata.ftp-master.debian.org/changelogs//main/b/binutils/binutils_2.39.90.20221231-1_changelog