Hi folks
I recompiled my kernel with GCC 3.0.1 (was 2.95.x)
and guess what - it got bigger...
Somehow, I hoped in linux world software gets better
with time, not worse...
Maybe that's my fault (misconfigured GCC etc) ?
What do you see?
Being curious, I looked into vmlinux (uncompressed kernel).
I saw swatches of zero bytes in places, large repeateable
patterns etc. You may look there too in your spare time.
Especially informative are two pages (my console:100x40)
filled with "GCC: (GNU) 3.0.1". Does this gets into
unswappable memory when kernel loads?
--
Best regards, VDA
mailto:[email protected]
I might not be right, but I thing gcc 3.x aligns some data differently
then previous versions, which sometimes causes executables to be
bigger...
[solid]
Registered Linux user number 212159
On Tue, 9 Oct 2001, VDA wrote:
> Hi folks
>
> I recompiled my kernel with GCC 3.0.1 (was 2.95.x)
> and guess what - it got bigger...
> Somehow, I hoped in linux world software gets better
> with time, not worse...
>
> Maybe that's my fault (misconfigured GCC etc) ?
> What do you see?
>
> Being curious, I looked into vmlinux (uncompressed kernel).
> I saw swatches of zero bytes in places, large repeateable
> patterns etc. You may look there too in your spare time.
>
> Especially informative are two pages (my console:100x40)
> filled with "GCC: (GNU) 3.0.1". Does this gets into
> unswappable memory when kernel loads?
> --
> Best regards, VDA
> mailto:[email protected]
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
VDA writes:
> Hi folks
>
> I recompiled my kernel with GCC 3.0.1 (was 2.95.x)
> and guess what - it got bigger...
> Somehow, I hoped in linux world software gets better
> with time, not worse...
>
> Maybe that's my fault (misconfigured GCC etc) ?
> What do you see?
>
> Being curious, I looked into vmlinux (uncompressed kernel).
> I saw swatches of zero bytes in places, large repeateable
> patterns etc. You may look there too in your spare time.
>
> Especially informative are two pages (my console:100x40)
> filled with "GCC: (GNU) 3.0.1". Does this gets into
> unswappable memory when kernel loads?
strings /proc/kcore | egrep GCC
Haha, I got several pieces of your mail message while doing this.
(/proc/kcore is unique file, because grep of *any* string on it would
succeeded).
> --
> Best regards, VDA
> mailto:[email protected]
>
>
Nikita.
On Tue, 9 Oct 2001, VDA wrote:
> Hi folks
>
> I recompiled my kernel with GCC 3.0.1 (was 2.95.x)
> and guess what - it got bigger...
> Somehow, I hoped in linux world software gets better
> with time, not worse...
>
> Maybe that's my fault (misconfigured GCC etc) ?
> What do you see?
>
> Being curious, I looked into vmlinux (uncompressed kernel).
It's much worse than you can imagine!
`strings /proc/kcore | grep GNU' >qqq.qqq`
Causes a file this big to be generated:
-rw-r--r-- 1 root root 1069748 Oct 9 10:01 qqq.qqq
That's how much space is being wasted by GNU advertising.
A single program:
int foo;
Compiled, produces this:
.file "xxx.c"
.version "01.01"
gcc2_compiled.:
.comm foo,4,4
.ident "GCC: (GNU) egcs-2.91.66 19990314 (egcs-1.1.2 release)"
It __might__ be possible to link, without linking in ".ident", which
currently shares space with .rodata. My gcc man pages are not any
better than the usual Red Hat so I can't find out if there is any way
to turn OFF these spurious strings.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).
I was going to compile a list of innovations that could be
attributed to Microsoft. Once I realized that Ctrl-Alt-Del
was handled in the BIOS, I found that there aren't any.
On Tue, Oct 09, 2001 at 10:16:48AM -0400, Richard B. Johnson wrote:
> Compiled, produces this:
>
> .file "xxx.c"
> .version "01.01"
> gcc2_compiled.:
> .comm foo,4,4
> .ident "GCC: (GNU) egcs-2.91.66 19990314 (egcs-1.1.2 release)"
>
> It __might__ be possible to link, without linking in ".ident", which
> currently shares space with .rodata. My gcc man pages are not any
> better than the usual Red Hat so I can't find out if there is any way
> to turn OFF these spurious strings.
strip -R .ident -R .comment -R .note
is your friend.
Or if we would like to solve it more elegant:
--- linux-2.4.10/arch/i386/vmlinux.lds Tue Oct 9 16:36:06 2001
+++ linux/arch/i386/vmlinux.lds Tue Oct 9 16:36:28 2001
@@ -70,6 +70,9 @@
*(.text.exit)
*(.data.exit)
*(.exitcall.exit)
+ *(.ident)
+ *(.comment)
+ *(.note)
}
/* Stabs debugging sections. */
which puts it into the list of sections to be discarde on i386.
Regards
Ingo Oeser
On Tue, 9 Oct 2001, Ingo Oeser wrote:
> On Tue, Oct 09, 2001 at 10:16:48AM -0400, Richard B. Johnson wrote:
> > Compiled, produces this:
> >
> > .file "xxx.c"
> > .version "01.01"
> > gcc2_compiled.:
> > .comm foo,4,4
> > .ident "GCC: (GNU) egcs-2.91.66 19990314 (egcs-1.1.2 release)"
> >
> > It __might__ be possible to link, without linking in ".ident", which
> > currently shares space with .rodata. My gcc man pages are not any
> > better than the usual Red Hat so I can't find out if there is any way
> > to turn OFF these spurious strings.
>
> strip -R .ident -R .comment -R .note
>
> is your friend.
>
> Or if we would like to solve it more elegant:
>
> --- linux-2.4.10/arch/i386/vmlinux.lds Tue Oct 9 16:36:06 2001
> +++ linux/arch/i386/vmlinux.lds Tue Oct 9 16:36:28 2001
> @@ -70,6 +70,9 @@
> *(.text.exit)
> *(.data.exit)
> *(.exitcall.exit)
> + *(.ident)
> + *(.comment)
> + *(.note)
> }
>
Yes! Wonderful...
-rwxr-xr-x 1 root root 1571516 Oct 9 10:50 vmlinux
-rwxr-xr-x 1 root root 1590692 Oct 1 13:26 vmlinux.OLD
That got rid of some cruft.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).
I was going to compile a list of innovations that could be
attributed to Microsoft. Once I realized that Ctrl-Alt-Del
was handled in the BIOS, I found that there aren't any.
"Richard B. Johnson" <[email protected]> said:
> On Tue, 9 Oct 2001, Ingo Oeser wrote:
[...]
> > strip -R .ident -R .comment -R .note
> >
> > is your friend.
[...]
> Yes! Wonderful...
> -rwxr-xr-x 1 root root 1571516 Oct 9 10:50 vmlinux
> -rwxr-xr-x 1 root root 1590692 Oct 1 13:26 vmlinux.OLD
>
> That got rid of some cruft.
Yep. A WHOOPing 1.2% of the total. BTW, is this stuff ever being loaded
into RAM with the executable kernel, discarded on boot, or what?
IMHO, it would be more productive to go after savings via .init*, and
perhaps bug the GCC/binutils people to merge strings...
--
Dr. Horst H. von Brand Usuario #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
On Tue, 9 Oct 2001, Horst von Brand wrote:
> "Richard B. Johnson" <[email protected]> said:
> > On Tue, 9 Oct 2001, Ingo Oeser wrote:
>
> [...]
>
> > > strip -R .ident -R .comment -R .note
> > >
> > > is your friend.
>
> [...]
>
> > Yes! Wonderful...
> > -rwxr-xr-x 1 root root 1571516 Oct 9 10:50 vmlinux
> > -rwxr-xr-x 1 root root 1590692 Oct 1 13:26 vmlinux.OLD
> >
> > That got rid of some cruft.
>
> Yep. A WHOOPing 1.2% of the total. BTW, is this stuff ever being loaded
> into RAM with the executable kernel, discarded on boot, or what?
>
Yes. It shows in /proc/kcore. Just wasted. It does mean something
on an embedded system.
It just __might__ mean that I can use a later kernel than 2.4.1
(they grow, you know). I'm mucking with things now.
> IMHO, it would be more productive to go after savings via .init*, and
> perhaps bug the GCC/binutils people to merge strings...
> --
It would be nice. I don't mind one advertisement in the kernel, but
presently there is one for every file that was linked.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).
I was going to compile a list of innovations that could be
attributed to Microsoft. Once I realized that Ctrl-Alt-Del
was handled in the BIOS, I found that there aren't any.
On Tue, Oct 09, 2001 at 11:43:14AM -0400, Horst von Brand wrote:
> "Richard B. Johnson" <[email protected]> said:
> > On Tue, 9 Oct 2001, Ingo Oeser wrote:
>
> [...]
>
> > > strip -R .ident -R .comment -R .note
> > >
> > > is your friend.
>
> [...]
>
> > Yes! Wonderful...
> > -rwxr-xr-x 1 root root 1571516 Oct 9 10:50 vmlinux
> > -rwxr-xr-x 1 root root 1590692 Oct 1 13:26 vmlinux.OLD
> >
> > That got rid of some cruft.
>
> Yep. A WHOOPing 1.2% of the total. BTW, is this stuff ever being loaded
> into RAM with the executable kernel, discarded on boot, or what?
It is not SHF_ALLOC, so it will not make it into vmlinuz either.
> IMHO, it would be more productive to go after savings via .init*, and
> perhaps bug the GCC/binutils people to merge strings...
?
gcc-2.96-RH (2.96-91 and above) and recent gcc 3.1 CVS together with
recent binutils support merging strings already. Unlike killing
.comment/.note sections, this saves real kernel memory.
This reminds me, I should finally hack up binutils so that it uses SHF_MERGE
for .ident directives automatically.
Jakub
On Tue, 9 Oct 2001 16:43:48 +0200,
Ingo Oeser <[email protected]> wrote:
>strip -R .ident -R .comment -R .note
>
>is your friend.
>
>Or if we would like to solve it more elegant:
>
>--- linux-2.4.10/arch/i386/vmlinux.lds Tue Oct 9 16:36:06 2001
>+++ linux/arch/i386/vmlinux.lds Tue Oct 9 16:36:28 2001
>@@ -70,6 +70,9 @@
> *(.text.exit)
> *(.data.exit)
> *(.exitcall.exit)
>+ *(.ident)
>+ *(.comment)
>+ *(.note)
> }
>
> /* Stabs debugging sections. */
>
>
>which puts it into the list of sections to be discarde on i386.
Already done. When vmlinux is converted to [b]Zimage kbuild runs
objcopy -O binary -R .note -R .comment -S
The comments are stripped out at that stage. Before you start fiddling
with vmlinux.lds, try this:
objcopy -O binary -R .note -R .comment -S vmlinux vmlinux.stripped
Look at vmlinux.stripped, it is what actually gets loaded. If there
are any offending strings from gcc then we can look at them but I
really doubt that there are any.
On Tue, 9 Oct 2001 11:53:48 -0400 (EDT),
"Richard B. Johnson" <[email protected]> wrote:
>Yes. It shows in /proc/kcore. Just wasted. It does mean something
>on an embedded system.
kcore shows all the kernel buffers, including user space stuff being
processed by the loader. My tests show that all the strings that you
are complaining about have been stripped from the kernel before it is
loaded.
"Richard B. Johnson" wrote:
> On Tue, 9 Oct 2001, Horst von Brand wrote:
>
> > "Richard B. Johnson" <[email protected]> said:
> > > On Tue, 9 Oct 2001, Ingo Oeser wrote:
> >
> > [...]
> >
> > > > strip -R .ident -R .comment -R .note
> > > >
> > > > is your friend.
> >
> > [...]
> >
> > > Yes! Wonderful...
> > > -rwxr-xr-x 1 root root 1571516 Oct 9 10:50 vmlinux
> > > -rwxr-xr-x 1 root root 1590692 Oct 1 13:26 vmlinux.OLD
> > >
> > > That got rid of some cruft.
> >
> > Yep. A WHOOPing 1.2% of the total. BTW, is this stuff ever being loaded
> > into RAM with the executable kernel, discarded on boot, or what?
> >
>
> Yes. It shows in /proc/kcore. Just wasted. It does mean something
> on an embedded system.
>
> It just __might__ mean that I can use a later kernel than 2.4.1
> (they grow, you know). I'm mucking with things now.
>
You do know that kcore shows all memory not just what was compiled into the
kernel. even usless old file cache so it's not so easy to se how much space
the strings use of real memory.
Test with creating a file with a string your not likly to have anywhere else.
num=1 ;while ((1)) ; do echo "xenomorph $num"; num=$(($num+1)) ; done >1.txt
strings /proc/kcore | grep xenomorph
Nikita Danilov wrote:
> Haha, I got several pieces of your mail message while doing this.
> (/proc/kcore is unique file, because grep of *any* string on it would
> succeeded).
I tried
strings /proc/kcore | grep any_weird_string_tsst_testtesttest
I expected it to hit a few times - the command line buffer,
the parameter to grep - but I got 7 screenfulls.
Then I understood - this hits the xterm scrollback buffer
too, which makes for some nice recursion.
Doing the same with output to a file hits the cache and
got some repetitions out of that. And then there's
internal buffers of strings and grep.
This reminded me of the commodore 64 game "fort apocalypse",
where your fly a helicopter around in caves. I patched the
machine code once so I could fly through walls. The game simply
loads memory into screen memory depending on coordinates.
So I got an interesting look at how code and state variables
look when interpreted as "terrain". I could identify
the variables holding x and y coordinates by looking at how
they changed when I moved in the two directions.
Then I came upon a very weird area, where everything
moved around in big jumps, changing in weird ways. After a while,
I figured out that I was looking at screen memory, having it
reloaded into itself with a different mapping. Crazy.
Finally - being able to press the fire button and fire
upon pieces of code and variables is the ultimate in
madman debugging and single-click crashes. :-)
Helge Hafting
On Wed, 10 Oct 2001, Keith Owens wrote:
> On Tue, 9 Oct 2001 11:53:48 -0400 (EDT),
> "Richard B. Johnson" <[email protected]> wrote:
> >Yes. It shows in /proc/kcore. Just wasted. It does mean something
> >on an embedded system.
>
> kcore shows all the kernel buffers, including user space stuff being
> processed by the loader. My tests show that all the strings that you
> are complaining about have been stripped from the kernel before it is
> loaded.
>
Yes... but. The final test was using bzImage which became about 1k
shorter after making the changes to vmlinux.lds.
total 748
drwxr-xr-x 4 root 101 4096 Oct 9 13:49 .
drwxr-xr-x 7 root 101 4096 Jan 11 2001 ..
-rw-r--r-- 1 root 101 2799 Dec 20 1999 Makefile
-rwxr-xr-x 1 root root 512 Oct 9 13:32 bbootsect
-rw-r--r-- 1 root root 2352 Oct 9 13:32 bbootsect.o
-rw-r--r-- 1 root root 7981 Oct 9 13:32 bbootsect.s
-rw-r--r-- 1 root 101 9644 Jan 29 2001 bootsect.S
-rwxr-xr-x 1 root root 4501 Oct 9 13:49 bsetup
-rw-r--r-- 1 root root 11704 Oct 9 13:49 bsetup.o
-rw-r--r-- 1 root root 44212 Oct 9 13:49 bsetup.s
-rw-r--r-- 1 root root 575432 Oct 9 13:49 bzImage
-rw-r--r-- 1 root root 581384 Oct 1 13:27 bzImage.OLD
drwxr-xr-x 2 root 101 4096 Oct 9 13:49 compressed
-rw-r--r-- 1 root 101 904 Jan 3 1995 install.sh
-rw-r--r-- 1 root 101 23458 Jan 27 2001 setup.S
drwxr-xr-x 2 root 101 4096 Oct 9 13:32 tools
-rw-r--r-- 1 root 101 39023 Nov 21 1999 video.S
The size change of the raw image is even more evident:
-rwxr-xr-x 1 root root 1548916 Oct 9 13:49 vmlinux
-rwxr-xr-x 1 root root 1590692 Oct 1 13:26 vmlinux.OLD
I did not look at any other scripts that are involved in making
the image.
Also I was not complaining about anything. One respondent noted
that compiling the kernel with a new 'C' compiler resulted in
a large increase in kernel size. Some of us then attempted to
find out simple ways to get the kernel size back down. I showed
that some 'C' compiler versions have enormous ID strings that
are wasting space. I also mentioned that some versions align
everything on 16-byte boundaries and there doesn't seem to
be any way to turn off this 'feature'.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).
I was going to compile a list of innovations that could be
attributed to Microsoft. Once I realized that Ctrl-Alt-Del
was handled in the BIOS, I found that there aren't any.
Hello Richard,
Wednesday, October 10, 2001, 3:00:32 PM, you wrote:
RBJ> Yes... but. The final test was using bzImage which became about 1k
RBJ> shorter after making the changes to vmlinux.lds.
...
RBJ> -rw-r--r-- 1 root root 575432 Oct 9 13:49 bzImage
RBJ> -rw-r--r-- 1 root root 581384 Oct 1 13:27 bzImage.OLD
...
RBJ> The size change of the raw image is even more evident:
...
RBJ> -rwxr-xr-x 1 root root 1548916 Oct 9 13:49 vmlinux
RBJ> -rwxr-xr-x 1 root root 1590692 Oct 1 13:26 vmlinux.OLD
...
RBJ> Also I was not complaining about anything. One respondent noted
RBJ> that compiling the kernel with a new 'C' compiler resulted in
RBJ> a large increase in kernel size. Some of us then attempted to
RBJ> find out simple ways to get the kernel size back down. I showed
RBJ> that some 'C' compiler versions have enormous ID strings that
RBJ> are wasting space. I also mentioned that some versions align
RBJ> everything on 16-byte boundaries and there doesn't seem to
RBJ> be any way to turn off this 'feature'.
Ok. Learning from makefiles: bzImage's copressed part is made
by gzipping toplevel 'vmlinux' processed via objcopy.
If anybody feel so inclined, I suggest doing
#objdump -O binary -R .note -R .comment -S vmlinux vmlinux.stripped
(that's what make bzImage does, feeding result to gzip)
and inspecting 'vmlinux.stripped' for large zeroed areas
and big/strange regular patterns. That might be oversized tables.
I'd like to be enlightened how I can translate offsets in
'vmlinux.stripped' to corresponding kernel func/var...
--
Best regards, vda
mailto:[email protected]
This is what is going into bzImage:
#objdump -O binary -R .note -R .comment -S vmlinux vmlinux.binary
To see lengths of the symbols:
#objdump -R .note -R .comment vmlinux vmlinux.stripped
#nm vmlinux.stripped |
grep -v '\(compiled\)\|\(\.o$$\)\|\( [aUw] \)\|\(\.\.ng$$\)\|\(LASH[RL]DI\)' |
sort >System.stripped.map
(well, I see exactly this grep in the Makefile.
If mail-mangled, look there to restore)
# ./gensize <System.stripped.map >System.stripped.size.map
# sort -r <System.stripped.size.map >System.stripped.sorted.map
gensize script (very silly) is at bottow of my email.
Top lines from my System.stripped.sorted.map follow.
I wonder: are all these have to be that big?
Anybody spots something unusual?
(size) (ofs) t (name)
00010000 c034d4a0 B mmu_gathers
00009380 c037c1e0 B blk_dev
000085c0 c035d4c0 B kstat
00007d83 c0292a00 T stext_lock
00007a7d c029a783 A _etext
00006b40 c02a9aa0 r devfsd_buf_size
00005e40 c02d27e0 R timer_bug_msg
000059b8 c02c89a8 R ahc_num_pci_devs
0000567f c02bdc01 r vendor
00005472 c02a3244 r Unused_offset
00004c00 c033a980 d pci_vendor_list
00004800 c0398960 B net_statistics
00004400 c038c9e0 B fb_display
00004000 c0367320 b log_buf
0000378a c02b640d r ide_hwif_to_major
00002f00 c02f14a0 d rs_table
00002da0 c02b2f40 r days_in_mo
0000247b c02d0365 r prio2band
00002400 c03715a0 b raw_devices
00002000 c03966c0 B icmp_statistics
00002000 c0343800 D init_tss
00002000 c0302000 D init_task_union
00001fe0 c038a060 B ide_hwifs
00001fe0 c0386980 b ro_bits
00001ed0 c01bf930 T vt_ioctl
00001d80 c0290c80 T rwsem_wake
00001d2c c02c5fe8 R num_critical_sections
00001ce5 c02ce680 R fb_con
00001c00 c0345800 D irq_desc
00001b14 c02c42c0 r RCSid
00001900 c0340200 D linux_logo
00001800 c0392fa0 B ip_statistics
000017e0 c02523e0 T tcp_sendmsg
000016c0 c02ba3e0 R sense_key_texts
000015e0 c0202350 T ahc_handle_seqint
000015b0 c02e28f0 A __start___ex_table
00001500 c034bf60 b irq_2_pin
00001400 c0347400 D cpu_data
00001376 c02a872a r start_ind_dev.7
00001320 c0209740 T ahc_init
00001040 c02c3280 r RCSid
00001000 c039d700 b swap_buffer
00001000 c0394900 B tcp_statistics
00001000 c0379d20 B con_buf
00001000 c0377ca0 b ptm_state
00001000 c0375320 b pty_state
00001000 c0365aa0 B pidhash
00001000 c02fc600 d fontdata_8x16
00001000 c02a2200 r tvecs
00001000 c0104000 T empty_zero_page
00001000 c0103000 T pg1
00001000 c0102000 T pg0
00001000 c0101000 T swapper_pg_dir
00000fd0 c0221380 t cdrom_ioctl
00000fb0 c01c5d50 t do_con_trol
00000f40 c01a8320 t ntfs_alloc_mft_record
00000f00 c0203930 T ahc_handle_scsiint
00000ee0 c024eb60 T ip_getsockopt
00000ea0 c02bbaa0 R sense_data_texts
00000ea0 c01e27e0 t ide_ioctl
00000e80 c0155d80 T path_walk
00000e60 c01ff940 t ahc_linux_queue_recovery_cmd
00000e60 c0176e60 t fat_readdirx
00000e40 c02b19c0 r twist_table.0
00000e20 c02f7fa0 d additional
00000dc0 c0321440 d script0
00000db0 c0254240 T tcp_recvmsg
00000da0 c02f99e0 d seqprog
00000d84 c010027c T gdt
00000d60 c0222500 t mmc_ioctl
00000d40 c02bca20 R scsi_device_types
00000d30 c01da1e0 t fd_ioctl
00000d20 c0255300 T tcp_close
gensize (don't laugh. I don't know sed/awk/perl/...)
-------
#!/bin/sh
prevline=""
prevofs=0xc0100000
read prevline
prevofs=0x`echo $prevline | cut -d " " -f 1 -`
read line
while test "$line"; do
ofs=0x`echo $line | cut -d " " -f 1 -`
diff=$(($ofs - $prevofs))
printf "%08x $prevline\n" $diff
prevofs=$ofs
prevline=$line
read line
done
printf "======== $prevline\n"
--
Best regards, vda
mailto:[email protected]