From: Andi Kleen <[email protected]>
This is an alternative approach to lower the overhead of debug info
(as we discussed a few days ago)
gcc 4.7+ and newer binutils have a new "split debug info" debug info
model where the debug info is only written once into central ".dwo" files.
This avoids having to copy it around multiple times, from the object
files to the final executable. It also lowers the disk space
requirements. In addition it defaults to compressed debug data.
More details here: http://gcc.gnu.org/wiki/DebugFission
This patch adds a new option to enable it. It has to be an option,
because it'll undoubtedly break everyone's debuginfo packaging scheme.
gdb/objdump/etc. all still work, if you have new enough versions.
I don't see big compile wins (maybe a second or two faster or so), but the
object dirs with debuginfo get significantly smaller. My standard kernel
config (slightly bigger than defconfig) shrinks from 2.9G disk space
to 1.1G objdir (with non reduced debuginfo). I presume if you are IO limited
the compile time difference will be larger.
Only problem I've seen so far is that it doesn't play well with older
versions of ccache (apparently fixed, see
https://bugzilla.samba.org/show_bug.cgi?id=10005)
v2: various fixes from Dirk Gouders. Improve commit message slightly.
v3: Fix clean rules and improve Kconfig slightly
Cc: Dirk Gouders <[email protected]>
Cc: [email protected]
Signed-off-by: Andi Kleen <[email protected]>
---
.gitignore | 1 +
Makefile | 5 +++++
lib/Kconfig.debug | 13 +++++++++++++
3 files changed, 19 insertions(+)
diff --git a/.gitignore b/.gitignore
index f4c0b09..e213b27 100644
--- a/.gitignore
+++ b/.gitignore
@@ -34,6 +34,7 @@
*.gcno
modules.builtin
Module.symvers
+*.dwo
#
# Top-level generic files
diff --git a/Makefile b/Makefile
index 1317563..953659e 100644
--- a/Makefile
+++ b/Makefile
@@ -683,9 +683,13 @@ endif
endif
ifdef CONFIG_DEBUG_INFO
+ifdef CONFIG_DEBUG_INFO_SPLIT
+KBUILD_CFLAGS += $(call cc-option, -gsplit-dwarf, -g)
+else
KBUILD_CFLAGS += -g
KBUILD_AFLAGS += -Wa,-gdwarf-2
endif
+endif
ifdef CONFIG_DEBUG_INFO_REDUCED
KBUILD_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
@@ -1371,6 +1375,7 @@ clean: $(clean-dirs)
@find $(if $(KBUILD_EXTMOD), $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \
\( -name '*.[oas]' -o -name '*.ko' -o -name '.*.cmd' \
-o -name '*.ko.*' \
+ -o -name '*.dwo' \
-o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
-o -name '*.symtypes' -o -name 'modules.order' \
-o -name modules.builtin -o -name '.tmp_*.o.*' \
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 7a638aa..fb07c33 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -143,6 +143,19 @@ config DEBUG_INFO_REDUCED
DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.
+config DEBUG_INFO_SPLIT
+ bool "Produce split debuginfo in .dwo files"
+ depends on DEBUG_INFO
+ help
+ Generate debug info into separate .dwo files. This can be
+ faster for building than including the debug information directly
+ in the object files and the vmlinux, as it only needs to
+ be stored once to disk, not multiple times in object files.
+ Requires recent gcc (4.7+) and recent gdb/binutils.
+ Any tool that packages or reads debug information would need
+ to know about the .dwo files and include them.
+ Incompatible with older versions of ccache.
+
config ENABLE_WARN_DEPRECATED
bool "Enable __deprecated logic"
default y
--
1.8.5.2
From: Andi Kleen <[email protected]>
I found that a lot of unresolvable variables when using gdb on the kernel
become resolvable when dwarf4 is enabled. So add a Kconfig flag to enable it.
It definitely increases the debug information size, but on the other
hand this isn't so bad when debug fusion is used.
Signed-off-by: Andi Kleen <[email protected]>
---
Makefile | 3 +++
lib/Kconfig.debug | 9 +++++++++
2 files changed, 12 insertions(+)
diff --git a/Makefile b/Makefile
index 953659e..9c4080b 100644
--- a/Makefile
+++ b/Makefile
@@ -690,6 +690,9 @@ KBUILD_CFLAGS += -g
KBUILD_AFLAGS += -Wa,-gdwarf-2
endif
endif
+ifdef CONFIG_DEBUG_INFO_DWARF4
+KBUILD_CFLAGS += -gdwarf-4
+endif
ifdef CONFIG_DEBUG_INFO_REDUCED
KBUILD_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index fb07c33..1af4d96 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -156,6 +156,15 @@ config DEBUG_INFO_SPLIT
to know about the .dwo files and include them.
Incompatible with older versions of ccache.
+config DEBUG_INFO_DWARF4
+ bool "Generate dwarf4 debuginfo"
+ depends on DEBUG_INFO
+ help
+ Generate dwarf4 debug info. This requires recent versions
+ of gcc and gdb. It makes the debug information larger.
+ But it significantly improves the success of resolving
+ variables in gdb on optimized code.
+
config ENABLE_WARN_DEPRECATED
bool "Enable __deprecated logic"
default y
--
1.8.5.2
Hi Andi.
On Wed, Jul 16, 2014 at 01:16:11AM +0200, Andi Kleen wrote:
> From: Andi Kleen <[email protected]>
>
> This is an alternative approach to lower the overhead of debug info
> (as we discussed a few days ago)
>
> gcc 4.7+ and newer binutils have a new "split debug info" debug info
> model where the debug info is only written once into central ".dwo" files.
>
> This avoids having to copy it around multiple times, from the object
> files to the final executable. It also lowers the disk space
> requirements. In addition it defaults to compressed debug data.
>
> More details here: http://gcc.gnu.org/wiki/DebugFission
>
> This patch adds a new option to enable it. It has to be an option,
> because it'll undoubtedly break everyone's debuginfo packaging scheme.
> gdb/objdump/etc. all still work, if you have new enough versions.
>
> I don't see big compile wins (maybe a second or two faster or so), but the
> object dirs with debuginfo get significantly smaller. My standard kernel
> config (slightly bigger than defconfig) shrinks from 2.9G disk space
> to 1.1G objdir (with non reduced debuginfo). I presume if you are IO limited
> the compile time difference will be larger.
>
> Only problem I've seen so far is that it doesn't play well with older
> versions of ccache (apparently fixed, see
> https://bugzilla.samba.org/show_bug.cgi?id=10005)
>
> v2: various fixes from Dirk Gouders. Improve commit message slightly.
> v3: Fix clean rules and improve Kconfig slightly
> Cc: Dirk Gouders <[email protected]>
> Cc: [email protected]
> Signed-off-by: Andi Kleen <[email protected]>
> ---
> .gitignore | 1 +
> Makefile | 5 +++++
> lib/Kconfig.debug | 13 +++++++++++++
> 3 files changed, 19 insertions(+)
>
> diff --git a/.gitignore b/.gitignore
> index f4c0b09..e213b27 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -34,6 +34,7 @@
> *.gcno
> modules.builtin
> Module.symvers
> +*.dwo
>
> #
> # Top-level generic files
> diff --git a/Makefile b/Makefile
> index 1317563..953659e 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -683,9 +683,13 @@ endif
> endif
>
> ifdef CONFIG_DEBUG_INFO
> +ifdef CONFIG_DEBUG_INFO_SPLIT
> +KBUILD_CFLAGS += $(call cc-option, -gsplit-dwarf, -g)
> +else
> KBUILD_CFLAGS += -g
> KBUILD_AFLAGS += -Wa,-gdwarf-2
> endif
> +endif
Why are as only called with -Wa,-gdwarf-2 in the non-split case?
Is this a bug or intentional?
If it is intentional then it is not explained.
>
> ifdef CONFIG_DEBUG_INFO_REDUCED
> KBUILD_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
> @@ -1371,6 +1375,7 @@ clean: $(clean-dirs)
> @find $(if $(KBUILD_EXTMOD), $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \
> \( -name '*.[oas]' -o -name '*.ko' -o -name '.*.cmd' \
> -o -name '*.ko.*' \
> + -o -name '*.dwo' \
> -o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
> -o -name '*.symtypes' -o -name 'modules.order' \
> -o -name modules.builtin -o -name '.tmp_*.o.*' \
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 7a638aa..fb07c33 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -143,6 +143,19 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> +config DEBUG_INFO_SPLIT
> + bool "Produce split debuginfo in .dwo files"
Jut nitpicking here - but please be consistent with debuginfo.
If it is one or two words.
> + depends on DEBUG_INFO
> + help
> + Generate debug info into separate .dwo files. This can be
> + faster for building than including the debug information directly
Here the "faster" part is promoted.
But in your cover letter you actually highlight the "smaller" case
as more significant.
Maybe include info on both positive items.
> + in the object files and the vmlinux, as it only needs to
> + be stored once to disk, not multiple times in object files.
> + Requires recent gcc (4.7+) and recent gdb/binutils.
> + Any tool that packages or reads debug information would need
> + to know about the .dwo files and include them.
> + Incompatible with older versions of ccache.
Sam
On Wed, Jul 16, 2014 at 01:16:12AM +0200, Andi Kleen wrote:
> From: Andi Kleen <[email protected]>
>
> I found that a lot of unresolvable variables when using gdb on the kernel
> become resolvable when dwarf4 is enabled. So add a Kconfig flag to enable it.
>
> It definitely increases the debug information size, but on the other
> hand this isn't so bad when debug fusion is used.
Would it not make sense to try to enable degug info like this:
First try -gdwarf-4
If this fails try -gdwarf-3
If this fails try -gdwarf-2
And if this also fails fallback to -g
We could then us a Kconfig option to enable the highest debug level - where 4 for now is the highest.
On the other hand if most people have revent utils then they all have -gdwarf-4 supported
and then it is no big deal.
Sam
>
> Signed-off-by: Andi Kleen <[email protected]>
> ---
> Makefile | 3 +++
> lib/Kconfig.debug | 9 +++++++++
> 2 files changed, 12 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index 953659e..9c4080b 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -690,6 +690,9 @@ KBUILD_CFLAGS += -g
> KBUILD_AFLAGS += -Wa,-gdwarf-2
> endif
> endif
> +ifdef CONFIG_DEBUG_INFO_DWARF4
> +KBUILD_CFLAGS += -gdwarf-4
> +endif
Should we sue $(call cc-option ...) here as not all gcc's supports this today?
Sam
On Sun, Jul 20, 2014 at 09:23:15PM +0200, Sam Ravnborg wrote:
> On Wed, Jul 16, 2014 at 01:16:12AM +0200, Andi Kleen wrote:
> > From: Andi Kleen <[email protected]>
> >
> > I found that a lot of unresolvable variables when using gdb on the kernel
> > become resolvable when dwarf4 is enabled. So add a Kconfig flag to enable it.
> >
> > It definitely increases the debug information size, but on the other
> > hand this isn't so bad when debug fusion is used.
>
> Would it not make sense to try to enable degug info like this:
> First try -gdwarf-4
> If this fails try -gdwarf-3
> If this fails try -gdwarf-2
> And if this also fails fallback to -g
>
> We could then us a Kconfig option to enable the highest debug level - where 4 for now is the highest.
> On the other hand if most people have revent utils then they all have -gdwarf-4 supported
> and then it is no big deal.
I made it a separate option, because dwarf4 is somewhat bigger than dwarf3 and it's
probably only useful for people actively using gdb/crash
(and I don't think others users like systemtap/perf probe do dwarf4 yet)
If debug fusion was default it would be fine, but right now it would
need converting a lot of debuginfo packaing scripts first.
-Andi
> Why are as only called with -Wa,-gdwarf-2 in the non-split case?
> Is this a bug or intentional?
> If it is intentional then it is not explained.
Probably was a merging mistake. Will fix.
> > + help
> > + Generate debug info into separate .dwo files. This can be
> > + faster for building than including the debug information directly
> Here the "faster" part is promoted.
> But in your cover letter you actually highlight the "smaller" case
> as more significant.
> Maybe include info on both positive items.
I currently don't see it significantly faster in my setup, will
mention size. It may be in others though. Feel free to test.
-Andi
On Sun, Jul 20, 2014 at 02:19:01PM -0700, Andi Kleen wrote:
> > Why are as only called with -Wa,-gdwarf-2 in the non-split case?
> > Is this a bug or intentional?
> > If it is intentional then it is not explained.
>
> Probably was a merging mistake. Will fix.
>
> > > + help
> > > + Generate debug info into separate .dwo files. This can be
> > > + faster for building than including the debug information directly
> > Here the "faster" part is promoted.
> > But in your cover letter you actually highlight the "smaller" case
> > as more significant.
> > Maybe include info on both positive items.
>
> I currently don't see it significantly faster in my setup, will
> mention size. It may be in others though. Feel free to test.
I tried to test this on my setup - it is a Intel Atom with 2 cores.
16 GB RAM, a 7200 RPM stat disk.
In other words a low-spec setup.
Base config was "make allnoconfig"
make KCFLAGS=-g
real 3m35.967s
user 12m19.270s
sys 1m15.082s
make KCFLAGS=-gsplit-dwarf
real 3m33.366s
user 12m11.156s
sys 1m17.772s
I did like this:
echo time make -j4 -s KCFLAGS=-gsplit-dwarf
time make -j4 -s KCFLAGS=-gsplit-dwarf
echo time make -j4 -s KCFLAGS=-g
time make -j4 -s KCFLAGS=-g
echo time make -j4 -s KCFLAGS=-gsplit-dwarf
time make -j4 -s KCFLAGS=-gsplit-dwarf
echo time make -j4 -s KCFLAGS=-g
time make -j4 -s KCFLAGS=-g
echo time make -j4 -s KCFLAGS=-gsplit-dwarf
time make -j4 -s KCFLAGS=-gsplit-dwarf
echo time make -j4 -s KCFLAGS=-g
time make -j4 -s KCFLAGS=-g
The figures are from last run - but the other runs had compareable times.
So in short I see a ~3 seconds speedup - out of 210 seconds.
Thats less than I anticipated.
But a speed-up never the less - which was consistent over all three runs.
Sam
> echo time make -j4 -s KCFLAGS=-gsplit-dwarf
> time make -j4 -s KCFLAGS=-gsplit-dwarf
> echo time make -j4 -s KCFLAGS=-g
> time make -j4 -s KCFLAGS=-g
> echo time make -j4 -s KCFLAGS=-gsplit-dwarf
> time make -j4 -s KCFLAGS=-gsplit-dwarf
> echo time make -j4 -s KCFLAGS=-g
> time make -j4 -s KCFLAGS=-g
> echo time make -j4 -s KCFLAGS=-gsplit-dwarf
> time make -j4 -s KCFLAGS=-gsplit-dwarf
> echo time make -j4 -s KCFLAGS=-g
> time make -j4 -s KCFLAGS=-g
>
> The figures are from last run - but the other runs had compareable times.
> So in short I see a ~3 seconds speedup - out of 210 seconds.
> Thats less than I anticipated.
Thanks. How much smaller is it?
Just saving disk space is a good thing.
-Andi
--
[email protected] -- Speaking for myself only.
Andi Kleen <[email protected]> writes:
>> echo time make -j4 -s KCFLAGS=-gsplit-dwarf
>> time make -j4 -s KCFLAGS=-gsplit-dwarf
>> echo time make -j4 -s KCFLAGS=-g
>> time make -j4 -s KCFLAGS=-g
>> echo time make -j4 -s KCFLAGS=-gsplit-dwarf
>> time make -j4 -s KCFLAGS=-gsplit-dwarf
>> echo time make -j4 -s KCFLAGS=-g
>> time make -j4 -s KCFLAGS=-g
>> echo time make -j4 -s KCFLAGS=-gsplit-dwarf
>> time make -j4 -s KCFLAGS=-gsplit-dwarf
>> echo time make -j4 -s KCFLAGS=-g
>> time make -j4 -s KCFLAGS=-g
>>
>> The figures are from last run - but the other runs had compareable times.
>> So in short I see a ~3 seconds speedup - out of 210 seconds.
>> Thats less than I anticipated.
>
> Thanks. How much smaller is it?
>
> Just saving disk space is a good thing.
Hi Andi,
I applied v3 to rc6+ and did the rudimentary tests that I did with
v1, i.e. make menuconfig, make, make clean, git status and have nothing
to report.
For times an sizes I did tests with my system's .config -- prior to each
`make' I did a `make clean':
split-debug:
# time make -j5
...
real 6m21.819s
user 22m48.828s
sys 1m1.804s
# du -sh .
2.6G .
no split-debug:
# time make -j5
...
real 6m40.311s
user 23m0.648s
sys 1m4.592s
# du -sh .
4.3G .
Dirk