Hi Ingo,
I'm starting with you because the first file I used to
experiment with this idea was arch/x86/ia32/sys_ia32.c.
I found it strange that the struct sk_buff definition was found
inside the DWARF debugging sections in the generated object, so I verified
and found that there is no need for the files that bring struct sk_buff
definition into this file and verified also that sk_buff is not brought
in indirectly too, thru other headers.
I went on and removed many other unneeded includes and the end
result is:
[acme@doppio net-2.6]$ l /tmp/sys_ia32.o.before /tmp/sys_ia32.o.after
-rw-rw-r-- 1 acme acme 185240 2008-02-06 19:19 /tmp/sys_ia32.o.after
-rw-rw-r-- 1 acme acme 248328 2008-02-06 19:00 /tmp/sys_ia32.o.before
Almost 64KB only on this object file!
There were no other side effects from this change:
[acme@doppio net-2.6]$ objcopy -j "text" /tmp/sys_ia32.o.before /tmp/text.before
[acme@doppio net-2.6]$ objcopy -j "text" /tmp/sys_ia32.o.after /tmp/text.after
[acme@doppio net-2.6]$ md5sum /tmp/text.before /tmp/text.after
b7ac9b17942add68494e698e4f965d36 /tmp/text.before
b7ac9b17942add68494e698e4f965d36 /tmp/text.after
One of the complaints about using tools such as systemtap is
that one has to install the huge kernel-debuginfo package:
[acme@doppio net-2.6]$ rpm -q --qf "%{size}\n" kernel-rt-debuginfo
471737710
543867594
[acme@doppio net-2.6]$
So I'll try to improve automation to find more of these cases.
Here goes the first patch, regards.
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
diff --git a/arch/x86/ia32/sys_ia32.c b/arch/x86/ia32/sys_ia32.c
index abf71d2..38d4ae3 100644
--- a/arch/x86/ia32/sys_ia32.c
+++ b/arch/x86/ia32/sys_ia32.c
@@ -26,50 +26,25 @@
#include <linux/file.h>
#include <linux/signal.h>
#include <linux/syscalls.h>
-#include <linux/resource.h>
#include <linux/times.h>
#include <linux/utsname.h>
-#include <linux/smp.h>
#include <linux/smp_lock.h>
-#include <linux/sem.h>
-#include <linux/msg.h>
#include <linux/mm.h>
-#include <linux/shm.h>
-#include <linux/slab.h>
#include <linux/uio.h>
-#include <linux/nfs_fs.h>
-#include <linux/quota.h>
-#include <linux/module.h>
-#include <linux/sunrpc/svc.h>
-#include <linux/nfsd/nfsd.h>
-#include <linux/nfsd/cache.h>
-#include <linux/nfsd/xdr.h>
-#include <linux/nfsd/syscall.h>
#include <linux/poll.h>
#include <linux/personality.h>
#include <linux/stat.h>
-#include <linux/ipc.h>
#include <linux/rwsem.h>
-#include <linux/binfmts.h>
-#include <linux/init.h>
-#include <linux/aio_abi.h>
-#include <linux/aio.h>
#include <linux/compat.h>
#include <linux/vfs.h>
#include <linux/ptrace.h>
#include <linux/highuid.h>
-#include <linux/vmalloc.h>
-#include <linux/fsnotify.h>
#include <linux/sysctl.h>
#include <asm/mman.h>
#include <asm/types.h>
#include <asm/uaccess.h>
#include <asm/semaphore.h>
#include <asm/atomic.h>
-#include <asm/ldt.h>
-
-#include <net/scm.h>
-#include <net/sock.h>
#include <asm/ia32.h>
#define AA(__x) ((unsigned long)(__x))
----- End forwarded message -----
On Thu, 7 Feb 2008, Arnaldo Carvalho de Melo wrote:
> Hi Ingo,
>
> I'm starting with you because the first file I used to
> experiment with this idea was arch/x86/ia32/sys_ia32.c.
>
> I found it strange that the struct sk_buff definition was found
> inside the DWARF debugging sections in the generated object, so I verified
> and found that there is no need for the files that bring struct sk_buff
> definition into this file and verified also that sk_buff is not brought
> in indirectly too, thru other headers.
>
> I went on and removed many other unneeded includes and the end
> result is:
>
> [acme@doppio net-2.6]$ l /tmp/sys_ia32.o.before /tmp/sys_ia32.o.after
> -rw-rw-r-- 1 acme acme 185240 2008-02-06 19:19 /tmp/sys_ia32.o.after
> -rw-rw-r-- 1 acme acme 248328 2008-02-06 19:00 /tmp/sys_ia32.o.before
>
> Almost 64KB only on this object file!
>
> There were no other side effects from this change:
>
> [acme@doppio net-2.6]$ objcopy -j "text" /tmp/sys_ia32.o.before /tmp/text.before
> [acme@doppio net-2.6]$ objcopy -j "text" /tmp/sys_ia32.o.after /tmp/text.after
> [acme@doppio net-2.6]$ md5sum /tmp/text.before /tmp/text.after
> b7ac9b17942add68494e698e4f965d36 /tmp/text.before
> b7ac9b17942add68494e698e4f965d36 /tmp/text.after
>
> One of the complaints about using tools such as systemtap is
> that one has to install the huge kernel-debuginfo package:
>
> [acme@doppio net-2.6]$ rpm -q --qf "%{size}\n" kernel-rt-debuginfo
> 471737710
> 543867594
> [acme@doppio net-2.6]$
>
> So I'll try to improve automation to find more of these cases.
That'd be grat.
> Here goes the first patch, regards.
Applied. Thanks,
tglx
* Arnaldo Carvalho de Melo <[email protected]> wrote:
> So I'll try to improve automation to find more of these cases.
>
> Here goes the first patch, regards.
randconfig testing found the attached config that fails to build:
arch/x86/ia32/sys_ia32.c: In function 'sys32_gettimeofday':
arch/x86/ia32/sys_ia32.c:447: error: 'sys_tz' undeclared (first use in this function)
arch/x86/ia32/sys_ia32.c:447: error: (Each undeclared identifier is reported only once
arch/x86/ia32/sys_ia32.c:447: error: for each function it appears in.)
adding vgtod.h solved this problem. Updated patch below.
Ingo
------------->
Subject: x86: reducing debuginfo size by removing unneeded includes
From: Arnaldo Carvalho de Melo <[email protected]>
Date: Thu, 7 Feb 2008 21:03:04 -0200
I'm starting with you because the first file I used to
experiment with this idea was arch/x86/ia32/sys_ia32.c.
I found it strange that the struct sk_buff definition was found
inside the DWARF debugging sections in the generated object, so I verified
and found that there is no need for the files that bring struct sk_buff
definition into this file and verified also that sk_buff is not brought
in indirectly too, thru other headers.
I went on and removed many other unneeded includes and the end
result is:
[acme@doppio net-2.6]$ l /tmp/sys_ia32.o.before /tmp/sys_ia32.o.after
-rw-rw-r-- 1 acme acme 185240 2008-02-06 19:19 /tmp/sys_ia32.o.after
-rw-rw-r-- 1 acme acme 248328 2008-02-06 19:00 /tmp/sys_ia32.o.before
Almost 64KB only on this object file!
There were no other side effects from this change:
[acme@doppio net-2.6]$ objcopy -j "text" /tmp/sys_ia32.o.before /tmp/text.before
[acme@doppio net-2.6]$ objcopy -j "text" /tmp/sys_ia32.o.after /tmp/text.after
[acme@doppio net-2.6]$ md5sum /tmp/text.before /tmp/text.after
b7ac9b17942add68494e698e4f965d36 /tmp/text.before
b7ac9b17942add68494e698e4f965d36 /tmp/text.after
One of the complaints about using tools such as systemtap is
that one has to install the huge kernel-debuginfo package:
[acme@doppio net-2.6]$ rpm -q --qf "%{size}\n" kernel-rt-debuginfo
471737710
543867594
[acme@doppio net-2.6]$
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>
---
arch/x86/ia32/sys_ia32.c | 26 +-------------------------
1 file changed, 1 insertion(+), 25 deletions(-)
Index: linux-x86.q/arch/x86/ia32/sys_ia32.c
===================================================================
--- linux-x86.q.orig/arch/x86/ia32/sys_ia32.c
+++ linux-x86.q/arch/x86/ia32/sys_ia32.c
@@ -26,51 +26,27 @@
#include <linux/file.h>
#include <linux/signal.h>
#include <linux/syscalls.h>
-#include <linux/resource.h>
#include <linux/times.h>
#include <linux/utsname.h>
-#include <linux/smp.h>
#include <linux/smp_lock.h>
-#include <linux/sem.h>
-#include <linux/msg.h>
#include <linux/mm.h>
-#include <linux/shm.h>
-#include <linux/slab.h>
#include <linux/uio.h>
-#include <linux/nfs_fs.h>
-#include <linux/quota.h>
-#include <linux/module.h>
-#include <linux/sunrpc/svc.h>
-#include <linux/nfsd/nfsd.h>
-#include <linux/nfsd/cache.h>
-#include <linux/nfsd/xdr.h>
-#include <linux/nfsd/syscall.h>
#include <linux/poll.h>
#include <linux/personality.h>
#include <linux/stat.h>
-#include <linux/ipc.h>
#include <linux/rwsem.h>
-#include <linux/binfmts.h>
-#include <linux/init.h>
-#include <linux/aio_abi.h>
-#include <linux/aio.h>
#include <linux/compat.h>
#include <linux/vfs.h>
#include <linux/ptrace.h>
#include <linux/highuid.h>
-#include <linux/vmalloc.h>
-#include <linux/fsnotify.h>
#include <linux/sysctl.h>
#include <asm/mman.h>
#include <asm/types.h>
#include <asm/uaccess.h>
#include <asm/semaphore.h>
#include <asm/atomic.h>
-#include <asm/ldt.h>
-
-#include <net/scm.h>
-#include <net/sock.h>
#include <asm/ia32.h>
+#include <asm/vgtod.h>
#define AA(__x) ((unsigned long)(__x))
Arnaldo Carvalho de Melo <[email protected]> writes:
>
> [acme@doppio net-2.6]$ l /tmp/sys_ia32.o.before /tmp/sys_ia32.o.after
> -rw-rw-r-- 1 acme acme 185240 2008-02-06 19:19 /tmp/sys_ia32.o.after
> -rw-rw-r-- 1 acme acme 248328 2008-02-06 19:00 /tmp/sys_ia32.o.before
>
> Almost 64KB only on this object file!
Just FYI, newer gcc does this in theory automatically when you specify
-feliminate-unused-debug-types -feliminate-unused-debug-symbols
But in my tests on gcc 4.1 / gcc 4.2 it doesn't seem to make any difference
currently :/ Not sure what is wrong.
There is also -feliminate-dwarf2-dups, but it seems to even increase
obj dir size. Also -feliminate-dwarf2-dups seems to generate a lot of
WARNING: vmlinux.o (.gnu.linkonce.wi.mmzone.h.97561702): unexpected section name.
The (.[number]+) following section name are ld generated and not expected.
Did you forget to use "ax"/"aw" in a .S file?
Note that for example <linux/init.h> contains
section definitions for use in .S files.
etc. warnings But the kernel builds fine even with those warnings.
Still if you just want to shrink objdir size then figuring out
what's wrong with these options and then specifying them would be
probably the best strategy than to try to do it all manually.
That said removing unused includes is of course a valuable
clean up by itself, I'm just not sure it's the best way to get
smaller object dirs.
-Andi
Em Mon, Feb 18, 2008 at 02:12:01PM +0100, Andi Kleen escreveu:
> Arnaldo Carvalho de Melo <[email protected]> writes:
> >
> > [acme@doppio net-2.6]$ l /tmp/sys_ia32.o.before /tmp/sys_ia32.o.after
> > -rw-rw-r-- 1 acme acme 185240 2008-02-06 19:19 /tmp/sys_ia32.o.after
> > -rw-rw-r-- 1 acme acme 248328 2008-02-06 19:00 /tmp/sys_ia32.o.before
> >
> > Almost 64KB only on this object file!
>
> Just FYI, newer gcc does this in theory automatically when you specify
>
> -feliminate-unused-debug-types -feliminate-unused-debug-symbols
>
> But in my tests on gcc 4.1 / gcc 4.2 it doesn't seem to make any difference
> currently :/ Not sure what is wrong.
>
> There is also -feliminate-dwarf2-dups, but it seems to even increase
> obj dir size. Also -feliminate-dwarf2-dups seems to generate a lot of
>
> WARNING: vmlinux.o (.gnu.linkonce.wi.mmzone.h.97561702): unexpected section name.
> The (.[number]+) following section name are ld generated and not expected.
> Did you forget to use "ax"/"aw" in a .S file?
> Note that for example <linux/init.h> contains
> section definitions for use in .S files.
>
> etc. warnings But the kernel builds fine even with those warnings.
>
> Still if you just want to shrink objdir size then figuring out
> what's wrong with these options and then specifying them would be
> probably the best strategy than to try to do it all manually.
>
> That said removing unused includes is of course a valuable
> clean up by itself, I'm just not sure it's the best way to get
> smaller object dirs.
I found just it to be another reason for cleaning up the includes hell,
but I agree completely that the correct solution is for gcc to do the
right thing and eliminate unused debug types.
Thanks for showing that there is support for that in latest gccs, I'll
try to use it when I get back to looking at this issue.
- Arnaldo
On Mon, Feb 18, 2008 at 02:12:01PM +0100, Andi Kleen wrote:
> Arnaldo Carvalho de Melo <[email protected]> writes:
> >
> > [acme@doppio net-2.6]$ l /tmp/sys_ia32.o.before /tmp/sys_ia32.o.after
> > -rw-rw-r-- 1 acme acme 185240 2008-02-06 19:19 /tmp/sys_ia32.o.after
> > -rw-rw-r-- 1 acme acme 248328 2008-02-06 19:00 /tmp/sys_ia32.o.before
> >
> > Almost 64KB only on this object file!
>
> Just FYI, newer gcc does this in theory automatically when you specify
>
> -feliminate-unused-debug-types -feliminate-unused-debug-symbols
>
> But in my tests on gcc 4.1 / gcc 4.2 it doesn't seem to make any difference
> currently :/ Not sure what is wrong.
>
> There is also -feliminate-dwarf2-dups, but it seems to even increase
> obj dir size. Also -feliminate-dwarf2-dups seems to generate a lot of
>
> WARNING: vmlinux.o (.gnu.linkonce.wi.mmzone.h.97561702): unexpected section name.
> The (.[number]+) following section name are ld generated and not expected.
> Did you forget to use "ax"/"aw" in a .S file?
> Note that for example <linux/init.h> contains
> section definitions for use in .S files.
Newly introduced consistency check. It immediately catched a few bugs
here and there not othing seriously. We could either restrict it to
known section names or just drop it again. At least it should not
be the reason to avoid using some good gcc feature.
Sam
> I found just it to be another reason for cleaning up the includes hell,
> but I agree completely that the correct solution is for gcc to do the
> right thing and eliminate unused debug types.
Yes, but I'm a little disappointed that it doesn't work either.
I remember long ago trying an original patch which did something
similar to -feliminate-dwarf2-* on a 3.3-hammer based gcc and
it actually shrunk the objdir size considerably.
-Andi
Em Mon, Feb 18, 2008 at 03:50:06PM +0100, Andi Kleen escreveu:
> > I found just it to be another reason for cleaning up the includes hell,
> > but I agree completely that the correct solution is for gcc to do the
> > right thing and eliminate unused debug types.
>
> Yes, but I'm a little disappointed that it doesn't work either.
> I remember long ago trying an original patch which did something
> similar to -feliminate-dwarf2-* on a 3.3-hammer based gcc and
> it actually shrunk the objdir size considerably.
>From my limited set of tests I expect it to shrunk the objdir by a lot
:-)
- Arnaldo
Hi Andi.
On Mon, Feb 18, 2008 at 02:12:01PM +0100, Andi Kleen wrote:
> Arnaldo Carvalho de Melo <[email protected]> writes:
> >
> > [acme@doppio net-2.6]$ l /tmp/sys_ia32.o.before /tmp/sys_ia32.o.after
> > -rw-rw-r-- 1 acme acme 185240 2008-02-06 19:19 /tmp/sys_ia32.o.after
> > -rw-rw-r-- 1 acme acme 248328 2008-02-06 19:00 /tmp/sys_ia32.o.before
> >
> > Almost 64KB only on this object file!
>
> Just FYI, newer gcc does this in theory automatically when you specify
>
> -feliminate-unused-debug-types -feliminate-unused-debug-symbols
google did not turn up anything useful.
Do you have a poiter to where these are described?
> There is also -feliminate-dwarf2-dups, but it seems to even increase
> obj dir size. Also -feliminate-dwarf2-dups seems to generate a lot of
Google only turned up a patch from Mark Mitchell removing this
in ~2001 timeframe.
So again a pointer would be nice.
I use FC8:
gcc --version
gcc (GCC) 4.1.2 20070925 (Red Hat 4.1.2-33)
So this is far behind trunk that you use - but I have not had any
real incentive to upgrade.
I would like to have a remote understanding of the options before
enabling them.
Sam
Andy> -feliminate-unused-debug-types -feliminate-unused-debug-symbols
Andy> -feliminate-dwarf2-dups
Sam> google did not turn up anything useful. Do you have a poiter to
Sam> where these are described?
>From GCC (svn) man page:
-feliminate-unused-debug-symbols
Produce debugging information in stabs format (if that is sup‐
ported), for only symbols that are actually used.
-feliminate-unused-debug-types
Normally, when producing DWARF2 output, GCC will emit debugging
information for all types declared in a compilation unit, regard‐
less of whether or not they are actually used in that compilation
unit. Sometimes this is useful, such as if, in the debugger, you
want to cast a value to a type that is not actually used in your
program (but is declared). More often, however, this results in a
significant amount of wasted space. With this option, GCC will
avoid producing debug symbol output for types that are nowhere used
in the source file being compiled.
-feliminate-dwarf2-dups
Compress DWARF2 debugging information by eliminating duplicated
information about each symbol. This option only makes sense when
generating DWARF2 debugging information with -gdwarf-2.
Sam
--
Samuel Tardieu -- [email protected] -- http://www.rfc1149.net/
On Sun, Feb 24, 2008 at 09:25:19AM +0100, Samuel Tardieu wrote:
> Andy> -feliminate-unused-debug-types -feliminate-unused-debug-symbols
> Andy> -feliminate-dwarf2-dups
>
> Sam> google did not turn up anything useful. Do you have a poiter to
> Sam> where these are described?
>
> >From GCC (svn) man page:
>
> -feliminate-unused-debug-symbols
> Produce debugging information in stabs format (if that is sup‐
> ported), for only symbols that are actually used.
>
> -feliminate-unused-debug-types
> Normally, when producing DWARF2 output, GCC will emit debugging
> information for all types declared in a compilation unit, regard‐
> less of whether or not they are actually used in that compilation
> unit. Sometimes this is useful, such as if, in the debugger, you
> want to cast a value to a type that is not actually used in your
> program (but is declared). More often, however, this results in a
> significant amount of wasted space. With this option, GCC will
> avoid producing debug symbol output for types that are nowhere used
> in the source file being compiled.
>
> -feliminate-dwarf2-dups
> Compress DWARF2 debugging information by eliminating duplicated
> information about each symbol. This option only makes sense when
> generating DWARF2 debugging information with -gdwarf-2.
>
Hi Sam.
I did not include the '-f' part in my google search. When I did so I got more hits.
Thanks,
Sam - who is not talking to himself ;-)
> > Just FYI, newer gcc does this in theory automatically when you specify
> >
> > -feliminate-unused-debug-types -feliminate-unused-debug-symbols
> google did not turn up anything useful.
> Do you have a poiter to where these are described?
info gcc
>
>
> > There is also -feliminate-dwarf2-dups, but it seems to even increase
> > obj dir size. Also -feliminate-dwarf2-dups seems to generate a lot of
>
> Google only turned up a patch from Mark Mitchell removing this
> in ~2001 timeframe.
> So again a pointer would be nice.
>
> I use FC8:
> gcc --version
> gcc (GCC) 4.1.2 20070925 (Red Hat 4.1.2-33)
>
> So this is far behind trunk that you use - but I have not had any
> real incentive to upgrade.
It's in my info pages for SUSE gcc 4.1. I didn't think it was
a SUSE extension though.
I also checked and all options are in gcc head and in gcc 4.3
>
> I would like to have a remote understanding of the options before
> enabling them.
Right now they are not useful to enable because they don't save
file size and in some cases make it even worse. I checked with some gcc
developers and they explained why that is so. gcc would need to be fixed first.
-Andi