On 31 Jan 02 at 13:47, Jeff Garzik wrote:
> On Thu, Jan 31, 2002 at 02:24:46PM +0100, Petr Vandrovec wrote:
> > I've got strange idea and tried to build diskless machine around
> > 2.5.3... Besides problem with segfaulting crc32 (it is initialized after
> > net/ipv4/ipconfig.c due to lib/lib.a being a library... I had to hardcode
> > lib/crc32.o before --start-group in main Makefile, but it is another
> > story)
>
> Would you be willing to cook up a patch for this problem?
>
> I ran into this too. It was solved by setting CONFIG_CRC32=n and
> letting the Makefile rules pull it in... but lib/lib.a needs to be
> lib/lib.o really.
Unfortunately during conversion I found that there is lib/bust_spinlocks.c,
which is always included in lib.a, is always compiled, even if architecture
provides its own bust_spinlocks function.
As no other module in lib/ uses module_init() initalization, it looks
to me like that we should move crc32.c from lib/ to kernel/, instead of
turning lib.a into lib.o.
But of course if there is consensus that I should convert lib/lib.a
into lib/lib.o, I can either create Config.in symbol
CONFIG_NEED_GENERIC_BUST_SPINLOCK, or add HAVE_ARCH_BUST_SPINLOCK #define
into some of i386, ia64, mips64, s390 and s390x architecture dependent
headers.
Thanks,
Petr Vandrovec
[email protected]
On Thu, Jan 31, 2002 at 09:27:35PM +0100, Petr Vandrovec wrote:
> On 31 Jan 02 at 13:47, Jeff Garzik wrote:
> > On Thu, Jan 31, 2002 at 02:24:46PM +0100, Petr Vandrovec wrote:
> > > I've got strange idea and tried to build diskless machine around
> > > 2.5.3... Besides problem with segfaulting crc32 (it is initialized after
> > > net/ipv4/ipconfig.c due to lib/lib.a being a library... I had to hardcode
> > > lib/crc32.o before --start-group in main Makefile, but it is another
> > > story)
> >
> > Would you be willing to cook up a patch for this problem?
> >
> > I ran into this too. It was solved by setting CONFIG_CRC32=n and
> > letting the Makefile rules pull it in... but lib/lib.a needs to be
> > lib/lib.o really.
>
> Unfortunately during conversion I found that there is lib/bust_spinlocks.c,
> which is always included in lib.a, is always compiled, even if architecture
> provides its own bust_spinlocks function.
Yep
> As no other module in lib/ uses module_init() initalization, it looks
> to me like that we should move crc32.c from lib/ to kernel/, instead of
> turning lib.a into lib.o.
Having lib.a is really a special case, and we can easily fix that by
fixing bust_spinlocks, not by moving what is truly a library routine to
somewhere other than lib/
> But of course if there is consensus that I should convert lib/lib.a
> into lib/lib.o, I can either create Config.in symbol
> CONFIG_NEED_GENERIC_BUST_SPINLOCK, or add HAVE_ARCH_BUST_SPINLOCK #define
> into some of i386, ia64, mips64, s390 and s390x architecture dependent
> headers.
Implementing HAVE_ARCH_BUST_SPINLOCK would follow kernel convention
quite nicely...
Jeff
On Thu, Jan 31, 2002 at 03:31:15PM -0500, Jeff Garzik wrote:
> > Unfortunately during conversion I found that there is lib/bust_spinlocks.c,
> > which is always included in lib.a, is always compiled, even if architecture
> > provides its own bust_spinlocks function.
>
> Yep
> > But of course if there is consensus that I should convert lib/lib.a
> > into lib/lib.o, I can either create Config.in symbol
> > CONFIG_NEED_GENERIC_BUST_SPINLOCK, or add HAVE_ARCH_BUST_SPINLOCK #define
> > into some of i386, ia64, mips64, s390 and s390x architecture dependent
> > headers.
>
> Implementing HAVE_ARCH_BUST_SPINLOCK would follow kernel convention
> quite nicely...
Hi Linus,
included patch (for 2.5.3) fixes problems with late initialization
of lib/crc32.o - as it was part of .a library file, it was picked by
link process AFTER at least one .o file required it - for example
when ipv4 autoconfiguration file needed it. So crc32's initalization
function was invoked after ipconfig's one - and it crashed inside
of ipconfig due to this problem.
Jeff Garzik advised me that I should convert lib.a to lib.o instead
of moving crc32 somewhere else, so I did it. I decided to put
HAVE_ARCH_BUST_SPINLOCKS macro to asm/system.h headers - if you disagree
with this location, tell me.
Couple of architectures adds $(TOPDIR)/lib/lib.o into its LIBS
instead of reusing already set $(LIBS) variable. I was not sure
what PPC people meant with this, so I left it as is.
I'd like to ask architecture maintainers (especially ia64, PPC, Sparc
and Sparc64 maintainers) for verification that my changes did not break
anything... I tested only i386, but changes were obvious...
Best regards,
Petr Vandrovec
[email protected]
diff -urdN linux/Makefile linux/Makefile
--- linux/Makefile Wed Jan 30 02:24:01 2002
+++ linux/Makefile Thu Jan 31 19:36:31 2002
@@ -121,7 +121,7 @@
CORE_FILES =kernel/kernel.o mm/mm.o fs/fs.o ipc/ipc.o
NETWORKS =net/network.o
-LIBS =$(TOPDIR)/lib/lib.a
+LIBS =$(TOPDIR)/lib/lib.o
SUBDIRS =kernel lib drivers mm fs net ipc
DRIVERS-n :=
diff -urdN linux/arch/ia64/boot/Makefile linux/arch/ia64/boot/Makefile
--- linux/arch/ia64/boot/Makefile Thu Jan 4 20:50:17 2001
+++ linux/arch/ia64/boot/Makefile Thu Jan 31 19:31:04 2002
@@ -23,7 +23,7 @@
all: $(targets-y)
bootloader: $(OBJECTS)
- $(LD) $(LINKFLAGS) $(OBJECTS) $(TOPDIR)/lib/lib.a $(TOPDIR)/arch/$(ARCH)/lib/lib.a \
+ $(LD) $(LINKFLAGS) $(OBJECTS) $(TOPDIR)/lib/lib.o $(TOPDIR)/arch/$(ARCH)/lib/lib.a \
-o bootloader
clean:
diff -urdN linux/arch/ppc/boot/chrp/Makefile linux/arch/ppc/boot/chrp/Makefile
--- linux/arch/ppc/boot/chrp/Makefile Tue Aug 28 13:58:33 2001
+++ linux/arch/ppc/boot/chrp/Makefile Thu Jan 31 19:29:34 2002
@@ -23,7 +23,7 @@
OBJS = ../common/crt0.o start.o main.o misc.o ../common/string.o image.o \
../common/ofcommon.o
-LIBS = $(TOPDIR)/lib/lib.a ../lib/zlib.a
+LIBS = $(TOPDIR)/lib/lib.o ../lib/zlib.a
ADDNOTE = ../utils/addnote
PIGGYBACK = ../utils/piggyback
diff -urdN linux/arch/ppc/boot/pmac/Makefile linux/arch/ppc/boot/pmac/Makefile
--- linux/arch/ppc/boot/pmac/Makefile Tue Aug 28 13:58:33 2001
+++ linux/arch/ppc/boot/pmac/Makefile Thu Jan 31 19:29:14 2002
@@ -15,7 +15,7 @@
COMMONOBJS = start.o misc.o ../common/string.o image.o ../common/ofcommon.o
COFFOBJS = ../common/coffcrt0.o $(COMMONOBJS) coffmain.o
CHRPOBJS = ../common/crt0.o $(COMMONOBJS) chrpmain.o
-LIBS = $(TOPDIR)/lib/lib.a ../lib/zlib.a
+LIBS = $(TOPDIR)/lib/lib.o ../lib/zlib.a
MKNOTE := ../utils/mknote
SIZE := ../utils/size
diff -urdN linux/arch/sparc/Makefile linux/arch/sparc/Makefile
--- linux/arch/sparc/Makefile Sat Jul 28 19:12:37 2001
+++ linux/arch/sparc/Makefile Thu Jan 31 19:26:07 2002
@@ -41,7 +41,7 @@
CORE_FILES := arch/sparc/kernel/kernel.o arch/sparc/mm/mm.o $(CORE_FILES) \
arch/sparc/math-emu/math-emu.o
-LIBS := $(TOPDIR)/lib/lib.a $(LIBS) $(TOPDIR)/arch/sparc/prom/promlib.a \
+LIBS := $(LIBS) $(TOPDIR)/arch/sparc/prom/promlib.a \
$(TOPDIR)/arch/sparc/lib/lib.a
# This one has to come last
diff -urdN linux/arch/sparc64/Makefile linux/arch/sparc64/Makefile
--- linux/arch/sparc64/Makefile Mon Jan 14 18:10:44 2002
+++ linux/arch/sparc64/Makefile Thu Jan 31 19:30:09 2002
@@ -69,7 +69,7 @@
CORE_FILES += arch/sparc64/math-emu/math-emu.o
-LIBS := $(TOPDIR)/lib/lib.a $(LIBS) $(TOPDIR)/arch/sparc64/prom/promlib.a \
+LIBS := $(LIBS) $(TOPDIR)/arch/sparc64/prom/promlib.a \
$(TOPDIR)/arch/sparc64/lib/lib.a
vmlinux.aout: vmlinux
diff -urdN linux/include/asm-i386/system.h linux/include/asm-i386/system.h
--- linux/include/asm-i386/system.h Wed Jan 30 05:41:09 2002
+++ linux/include/asm-i386/system.h Thu Jan 31 20:38:16 2002
@@ -123,6 +123,8 @@
__asm__("movl %0,%%cr4": :"r" (x));
#define stts() write_cr0(8 | read_cr0())
+#define HAVE_ARCH_BUST_SPINLOCKS
+
#endif /* __KERNEL__ */
#define wbinvd() \
diff -urdN linux/include/asm-ia64/system.h linux/include/asm-ia64/system.h
--- linux/include/asm-ia64/system.h Fri Nov 9 22:26:17 2001
+++ linux/include/asm-ia64/system.h Thu Jan 31 20:38:55 2002
@@ -412,6 +412,8 @@
} while (0)
#endif
+#define HAVE_ARCH_BUST_SPINLOCKS
+
#endif /* __KERNEL__ */
#endif /* __ASSEMBLY__ */
diff -urdN linux/include/asm-mips64/system.h linux/include/asm-mips64/system.h
--- linux/include/asm-mips64/system.h Wed Jul 4 18:50:39 2001
+++ linux/include/asm-mips64/system.h Thu Jan 31 20:39:49 2002
@@ -256,4 +256,6 @@
extern void set_except_vector(int n, void *addr);
+#define HAVE_ARCH_BUST_SPINLOCKS
+
#endif /* _ASM_SYSTEM_H */
diff -urdN linux/include/asm-s390/system.h linux/include/asm-s390/system.h
--- linux/include/asm-s390/system.h Wed Jul 25 21:12:02 2001
+++ linux/include/asm-s390/system.h Thu Jan 31 20:40:24 2002
@@ -250,6 +250,9 @@
extern void save_fp_regs(s390_fp_regs *fpregs);
extern int restore_fp_regs1(s390_fp_regs *fpregs);
extern void restore_fp_regs(s390_fp_regs *fpregs);
+
+#define HAVE_ARCH_BUST_SPINLOCKS
+
#endif
#endif
diff -urdN linux/include/asm-s390x/system.h linux/include/asm-s390x/system.h
--- linux/include/asm-s390x/system.h Wed Jul 25 21:12:03 2001
+++ linux/include/asm-s390x/system.h Thu Jan 31 20:40:40 2002
@@ -260,6 +260,9 @@
extern void save_fp_regs(s390_fp_regs *fpregs);
extern int restore_fp_regs1(s390_fp_regs *fpregs);
extern void restore_fp_regs(s390_fp_regs *fpregs);
+
+#define HAVE_ARCH_BUST_SPINLOCKS
+
#endif
#endif
diff -urdN linux/lib/Makefile linux/lib/Makefile
--- linux/lib/Makefile Wed Jan 30 05:30:41 2002
+++ linux/lib/Makefile Thu Jan 31 20:28:16 2002
@@ -6,7 +6,7 @@
# unless it's something special (ie not a .c file).
#
-L_TARGET := lib.a
+O_TARGET := lib.o
export-objs := cmdline.o dec_and_lock.o rwsem-spinlock.o rwsem.o crc32.o
diff -urdN linux/lib/bust_spinlocks.c linux/lib/bust_spinlocks.c
--- linux/lib/bust_spinlocks.c Mon Sep 17 04:22:40 2001
+++ linux/lib/bust_spinlocks.c Thu Jan 31 20:43:10 2002
@@ -14,6 +14,8 @@
#include <linux/wait.h>
#include <linux/vt_kern.h>
+#ifndef HAVE_ARCH_BUST_SPINLOCKS
+
extern spinlock_t timerlist_lock;
void bust_spinlocks(int yes)
@@ -38,4 +40,4 @@
}
}
-
+#endif /* HAVE_ARCH_BUST_SPINLOCKS */
From: Petr Vandrovec <[email protected]>
Date: Thu, 31 Jan 2002 23:53:06 +0100
I'd like to ask architecture maintainers (especially ia64, PPC, Sparc
and Sparc64 maintainers) for verification that my changes did not break
anything... I tested only i386, but changes were obvious...
I'll test this out up after Linus releases a pre-patch with this
included.
As a side note, this thing is so tiny (less than 4K on sparc64!) so
why don't we just include it unconditionally instead of having all
of this "turn it on for these drivers" stuff?
On Thu, Jan 31, 2002 at 02:59:04PM -0800, David S. Miller wrote:
> As a side note, this thing is so tiny (less than 4K on sparc64!) so
> why don't we just include it unconditionally instead of having all
> of this "turn it on for these drivers" stuff?
Does that 4K include the BE and LE crc tables?
<shrug> I don't mind much either way, except that I am general
resistant to "turn this on unconditionally" for bloat reasons.
[ie. its a reflex :)]
Jeff
> As a side note, this thing is so tiny (less than 4K on sparc64!) so
> why don't we just include it unconditionally instead of having all
> of this "turn it on for these drivers" stuff?
Because 100 4K drivers suddenly becomes 0.5Mb. There are those of us trying
to stuff Linux into embedded devices who if anything want more configuration
options not people taking stuff out.
What I'd much rather see if this is an issue is:
bool 'Do you want to customise for a very small system'
which auto enables all the random small stuff if you say no, and goes
much deeper into options if you say yes.
Alan
On Thu, Jan 31, 2002 at 11:24:10PM +0000, Alan Cox wrote:
> What I'd much rather see if this is an issue is:
>
> bool 'Do you want to customise for a very small system'
>
> which auto enables all the random small stuff if you say no, and goes
> much deeper into options if you say yes.
CONFIG_SMALL or similar would indeed be nice...
Jeff
From: Alan Cox <[email protected]>
Date: Thu, 31 Jan 2002 23:24:10 +0000 (GMT)
> As a side note, this thing is so tiny (less than 4K on sparc64!) so
> why don't we just include it unconditionally instead of having all
> of this "turn it on for these drivers" stuff?
Because 100 4K drivers suddenly becomes 0.5Mb.
However this isn't a driver, the crc library stuff is more akin to
"strlen()". Are you suggesting to provide a CONFIG_STRINGOPS=n
too? I wish you luck building that kernel :-)
> Because 100 4K drivers suddenly becomes 0.5Mb.
>
> However this isn't a driver, the crc library stuff is more akin to
> "strlen()". Are you suggesting to provide a CONFIG_STRINGOPS=n
> too? I wish you luck building that kernel :-)
For a large number of systems you don't need the CRC library. There are no
systems where you don't need memcpy, so your comparison is stupid to say
the least.
"Gee making this link is hard" is not a good reason to simplify the
config file, its a good reason to simplify the make file system
Em Thu, Jan 31, 2002 at 11:24:10PM +0000, Alan Cox escreveu:
> > As a side note, this thing is so tiny (less than 4K on sparc64!) so
> > why don't we just include it unconditionally instead of having all
> > of this "turn it on for these drivers" stuff?
>
> Because 100 4K drivers suddenly becomes 0.5Mb. There are those of us trying
> to stuff Linux into embedded devices who if anything want more configuration
> options not people taking stuff out.
>
> What I'd much rather see if this is an issue is:
>
> bool 'Do you want to customise for a very small system'
>
> which auto enables all the random small stuff if you say no, and goes
> much deeper into options if you say yes.
heh, after I've read that you managed to boot 2.4 + rmap in a machine with
just 4 MB after tweaking some table sizes I thought about devoting some
time to identify those tables and making them options in make *config, with
even a nice CONFIG_TINY, like you said 8)
I'll eventually do this, and I'd appreciate if people send me suggestions
of tables/data structures that can be trimmed/reduced. Yeah, I'll take a
look at the .config files used in the embedded distros.
- Arnaldo
[email protected] said:
> heh, after I've read that you managed to boot 2.4 + rmap in a machine
> with just 4 MB after tweaking some table sizes I thought about
> devoting some time to identify those tables and making them options in
> make *config, with even a nice CONFIG_TINY, like you said 8)
> I'll eventually do this, and I'd appreciate if people send me
> suggestions of tables/data structures that can be trimmed/reduced.
> Yeah, I'll take a look at the .config files used in the embedded
> distros.
CONFIG_BLK_DEV=n should allow you to trim a lot of cruft that most of my
embedded boxen never use.
--
dwmw2
On Thu, Jan 31, 2002 at 11:24:10PM +0000, Alan Cox wrote:
Because 100 4K drivers suddenly becomes 0.5Mb. There are those of
us trying to stuff Linux into embedded devices who if anything
want more configuration options not people taking stuff out.
Well, I'm more or less in agreement here, especially when working with
small embedded devices which have a few (say 16 or 32) MB of RAM for
EVERYTHING, kernel, userspace AND filesystems.
However, I wonder if we can't have the linker remove unnecessary and
unreferences objects, functions and variables?
What I'd much rather see if this is an issue is:
bool 'Do you want to customise for a very small system'
_IF_ the linker can remove things, it would simplify this too --- we
could if a few important places produce code slightly differently to
favour speed over size and not reference various things. Also, the
above option would turn-off inlining as that seems to makie quite a
difference at times (BTW, I'm not sure about this, but it seems gcc
and C99 don't agree with static/extern inline semantics?)
--cw
Chris Wedgwood wrote:
>
> On Thu, Jan 31, 2002 at 11:24:10PM +0000, Alan Cox wrote:
>
> Because 100 4K drivers suddenly becomes 0.5Mb. There are those of
> us trying to stuff Linux into embedded devices who if anything
> want more configuration options not people taking stuff out.
>
> Well, I'm more or less in agreement here, especially when working with
> small embedded devices which have a few (say 16 or 32) MB of RAM for
> EVERYTHING, kernel, userspace AND filesystems.
>
> However, I wonder if we can't have the linker remove unnecessary and
> unreferences objects, functions and variables?
We can. Graham Stoney had all this going against 2.2. See
http://www.google.com/search?q=stoney+ffunction-sections&hl=en&start=10&sa=N
http://www.cs.helsinki.fi/linux/linux-kernel/Year-2000/2000-29/0415.html
> What I'd much rather see if this is an issue is:
>
> bool 'Do you want to customise for a very small system'
>
> _IF_ the linker can remove things, it would simplify this too --- we
> could if a few important places produce code slightly differently to
> favour speed over size and not reference various things. Also, the
> above option would turn-off inlining as that seems to makie quite a
> difference at times (BTW, I'm not sure about this, but it seems gcc
> and C99 don't agree with static/extern inline semantics?)
The kernel doesn't link when you compile with -fno-inline because of all
the `extern inline' qualifiers. These need to be converted to `static
inline'. Jim Houston has a script which does this. See
http://www.uwsg.iu.edu/hypermail/linux/kernel/0201.3/0888.html
It would be rather good if we could get that script run across the
tree - no-inline has its uses at times.
-
On Fri, Feb 01, 2002 at 07:01:56PM -0800, Andrew Morton wrote:
We can. Graham Stoney had all this going against 2.2. See
http://www.google.com/search?q=stoney+ffunction-sections&hl=en&start=10&sa=N
http://www.cs.helsinki.fi/linux/linux-kernel/Year-2000/2000-29/0415.html
This puts every function it's own section. That seems not only
cumbersome to me, but also a little complex. That said, it may be a
good way if run every now and then to detect when cruft starts to
accumulate for any given .config and allow people to tweak things for
smaller kernels.
Is there no way to write something like:
--snip-- foo.c --snip--
void
blem()
{
}
void
bar()
{
blem();
return 0;
}
int foo()
{
return 1;
}
int main(...)
{
return foo();
}
--snip-- foo.c --snip--
compile and link it, have the linker know main or some part of crt0 is
special, build a graph of the final ELF object, see that bar and blem
are not connected to 'main' and discard them?
I'm pretty sure (but not 100% certain) that oher compilers can do
this, maybe someone can test on other platforms?
A really smart linker (if given enough compiler help) could build a
directional graph and still remove this code even if blem called foo.
Perhaps I'm making something that's extremely complex sound simple,
but it doesn't seem to me that this should be that complex... maybe
someone more farmiliar with binutils and/or gcc can comment and tell
me why I'm being a fool :)
The kernel doesn't link when you compile with -fno-inline because
of all the `extern inline' qualifiers. These need to be converted
to `static inline'. Jim Houston has a script which does this.
Semantically, in gcc land, someone explain the difference between:
inline
extern inline
static inline
please?
My tests seem to indicte:
inline creates an non-inline functions. Simple tests failed to have
this function inlined at all
static inline and extern inline functions can and will be inlined depending
on optimization level
extern inline doesn't product a non-inlined function (even if it is
referenced) and hence barfs if you don't compile with optimisations
for my simple test
Now, I wonder
why 'inline' for me, never inlines?
is there a way to force inlining of a function?
are non-inlined functions ever called when optimizations are
enabled?
Thanks,
--cw
On Fri, Feb 01, 2002 at 11:30:20PM -0800, Chris Wedgwood wrote:
> Is there no way to write something like:
>
> --snip-- foo.c --snip--
> void
> blem()
> {
> }
>
> void
> bar()
> {
> blem();
> return 0;
> }
>
> int foo()
> {
> return 1;
> }
>
> int main(...)
> {
> return foo();
> }
> --snip-- foo.c --snip--
>
> compile and link it, have the linker know main or some part of crt0 is
> special, build a graph of the final ELF object, see that bar and blem
> are not connected to 'main' and discard them?
>
> I'm pretty sure (but not 100% certain) that oher compilers can do
> this, maybe someone can test on other platforms?
>
> A really smart linker (if given enough compiler help) could build a
> directional graph and still remove this code even if blem called foo.
One piece of the necessary compiler help would be -ffunction-sections.
If they are in the same section in the same object file, they simply
can not be removed safely. Relocation information for calls to local
functions is not reliably available at link time.
--
Daniel Jacobowitz Carnegie Mellon University
MontaVista Software Debian GNU/Linux Developer
On Fri, Feb 01, 2002 at 11:30:20PM -0800, Chris Wedgwood wrote:
> A really smart linker (if given enough compiler help) could build a
> directional graph and still remove this code even if blem called foo.
Agreed and this has been my objection to the function-sections patch.
There is no need for it if you make the toolchain smarter.
For example I would love to see a !CONFIG_MODULES build rip out all the
code that was not actively referenced, such as EXPORT_SYMBOL functions
which are never called.
Jeff
#!/usr/bin/perl -w
#
# namespace.pl. Mon Jan 27 22:25:47 EST 1997
#
# Perform a name space analysis on the linux kernel.
#
# Copyright Keith Owens <[email protected]>. GPL.
#
# Invoke by changing directory to the top of the kernel source
# tree then namespace.pl, no parameters.
#
# Tuned for 2.1.x kernels with the new module handling, it will
# work with 2.0 kernels as well. Last change 2.1.81.
#
# The source must be compiled/assembled first, the object files
# are the primary input to this script. Incomplete or missing
# objects will result in a flawed analysis.
#
# Even with complete objects, treat the result of the analysis
# with caution. Some external references are only used by
# certain architectures, others with certain combinations of
# configuration parameters. Ideally the source should include
# something like
#
# #ifndef CONFIG_...
# static
# #endif
# symbol_definition;
#
# so the symbols are defined as static unless a particular
# CONFIG_... requires it to be external.
#
require 5; # at least perl 5
use strict;
use File::Find;
my $nm = "/usr/bin/nm -p"; # in case somebody moves nm
if ($#ARGV != -1) {
print STDERR "usage: $0 takes no parameters\n";
die("giving up\n");
}
my %nmdata = (); # nm data for each object
my %def = (); # all definitions for each name
my %ksymtab = (); # names that appear in __ksymtab_
my %ref = (); # $ref{$name} exists if there is a true external reference to $name
my %export = (); # $export{$name} exists if there is an EXPORT_... of $name
&find(\&linux_objects, '.'); # find the objects and do_nm on them
list_multiply_defined();
resolve_external_references();
list_extra_externals();
exit(0);
sub linux_objects
{
# Select objects, ignoring objects which are only created by
# merging other objects. Also ignore all of modules, scripts
# and compressed.
my $basename = $_;
$_ = $File::Find::name;
s:^\./::;
if (/.*\.o$/ && ! (
m:/fs.o$: || m:/isofs.o$: || m:/nfs.o$:
|| m:/xiafs.o$: || m:/umsdos.o$: || m:/hpfs.o$:
|| m:/smbfs.o$: || m:/ncpfs.o$: || m:/ufs.o$:
|| m:/affs.o$: || m:/romfs.o$: || m:/kernel.o$:
|| m:/mm.o$: || m:/ipc.o$: || m:/ext.o$:
|| m:/msdos.o$: || m:/proc/proc.o$: || m:/minix.o$:
|| m:/ext2.o$: || m:/sysv.o$: || m:/fat.o$:
|| m:/vfat.o$: || m:/unix.o$: || m:/802.o$:
|| m:/appletalk.o$: || m:/ax25.o$: || m:/core.o$:
|| m:/ethernet.o$: || m:/ipv4.o$: || m:/ipx.o$:
|| m:/netrom.o$: || m:/ipv6.o$: || m:/x25.o$:
|| m:/rose.o$: || m:/bridge.o$: || m:/lapb.o$:
|| m:/sock_n_syms.o$: || m:/teles.o$: || m:/pcbit.o$:
|| m:/isdn.o$: || m:/ftape.o$: || m:/scsi_mod.o$:
|| m:/sd_mod.o$: || m:/sr_mod.o$: || m:/lowlevel.o$:
|| m:/sound.o$: || m:/piggy.o$: || m:/bootsect.o$:
|| m:/boot/setup.o$: || m:^modules/: || m:^scripts/:
|| m:/compressed/:
|| m:/autofs.o$: || m:/lockd/lockd.o$: || m:/nfsd.o$:
|| m:/sunrpc.o$: || m:/scsi_n_syms.o$:
|| m:/boot/bbootsect.o$: || m:/boot/bsetup.o$:
|| m:/misc/parport.o$: || m:/nls/nls.o$:
|| m:/debug/debug.o$: || m:/netlink/netlink.o$:
|| m:/sched/sched.o$: || m:/sound/sb.o$:
|| m:/sound/soundcore.o$: || m:/pci/pci_syms.o$:
|| m:/devpts/devpts.o$: || m:/video/fbdev.o$:
)
) {
do_nm($basename, $_);
}
$_ = $basename; # File::Find expects $_ untouched (undocumented)
}
sub do_nm
{
my ($basename, $fullname) = @_;
my ($source, $type, $name);
if (! -e $basename) {
printf STDERR "$basename does not exist\n";
return;
}
if ($fullname !~ /\.o$/) {
printf STDERR "$fullname is not an object file\n";
return;
}
$source = $basename;
$source =~ s/\.o$//;
if (! -e "$source.c" && ! -e "$source.S") {
printf STDERR "No source file found for $fullname\n";
return;
}
if (! open(NMDATA, "$nm $basename|")) {
printf STDERR "$nm $fullname failed $!\n";
return;
}
my @nmdata;
while (<NMDATA>) {
chop;
($type, $name) = (split(/ +/, $_, 3))[1..2];
# Expected types
# B weak external reference to data that has been resolved
# C global variable, uninitialised
# D global variable, initialised
# R global array, initialised
# T global label/procedure
# U external reference
# W weak external reference to text that has been resolved
# a assembler equate
# b static variable, uninitialised
# d static variable, initialised
# r static array, initialised
# t static label/procedures
# ? undefined type, used a lot by modules
if ($type !~ /^[BCDRTUWabdrt?]$/) {
printf STDERR "nm output for $fullname contains unknown type '$_'\n";
}
elsif ($name =~ /\./) {
# name with '.' is local static
}
else {
$name =~ s/_R[a-f0-9]{8}$//; # module versions adds this
if ($type =~ /[BCDRTW]/ &&
$name ne 'init_module' &&
$name ne 'cleanup_module' &&
$name ne 'Using_Versions' &&
$name !~ /^Version_[0-9]+$/ &&
$name ne 'kernel_version') {
if (!exists($def{$name})) {
$def{$name} = [];
}
push(@{$def{$name}}, $fullname);
}
push(@nmdata, "$type $name");
if ($name =~ /^__ksymtab_/) {
$name = substr($name, 10);
if (!exists($ksymtab{$name})) {
$ksymtab{$name} = [];
}
push(@{$ksymtab{$name}}, $fullname);
}
}
}
close(NMDATA);
if ($#nmdata < 0) {
printf "No nm data for $fullname\n";
return;
}
$nmdata{$fullname} = \@nmdata;
}
sub list_multiply_defined
{
my ($name, $module);
foreach $name (keys(%def)) {
if ($#{$def{$name}} > 0) {
printf "$name is multiply defined in :-\n";
foreach $module (@{$def{$name}}) {
printf "\t$module\n";
}
}
}
}
sub resolve_external_references
{
my ($object, $type, $name, $i, $j, $kstrtab, $ksymtab, $export);
printf "\n";
foreach $object (keys(%nmdata)) {
my $nmdata = $nmdata{$object};
for ($i = 0; $i <= $#{$nmdata}; ++$i) {
($type, $name) = split(' ', $nmdata->[$i], 2);
if ($type eq "U") {
if (exists($def{$name}) || exists($ksymtab{$name})) {
# add the owning object to the nmdata
$nmdata->[$i] = "$type $name $object";
# only count as a reference if it is not EXPORT_...
$kstrtab = "? __kstrtab_$name";
$ksymtab = "? __ksymtab_$name";
$export = 0;
for ($j = 0; $j <= $#{$nmdata}; ++$j) {
if ($nmdata->[$j] eq $kstrtab ||
$nmdata->[$j] eq $ksymtab) {
$export = 1;
last;
}
}
if ($export) {
$export{$name} = "";
}
else {
$ref{$name} = ""
}
}
elsif ($name ne "mod_use_count_" &&
$name ne "__this_module" &&
$name ne "_etext" &&
$name ne "_edata" &&
$name ne "_end" &&
$name ne "__start___ksymtab" &&
$name ne "__start___ex_table" &&
$name ne "__stop___ksymtab" &&
$name ne "__stop___ex_table" &&
$name ne "__stop___ex_table" &&
$name ne "__bss_start" &&
$name ne "_text" &&
$name ne "__init_begin" &&
$name ne "__init_end") {
printf "Cannot resolve reference to $name from $object\n";
}
}
}
}
}
sub list_extra_externals
{
my %noref = ();
my ($name, @module, $module, $export);
foreach $name (keys(%def)) {
if (! exists($ref{$name})) {
@module = @{$def{$name}};
foreach $module (@module) {
if (! exists($noref{$module})) {
$noref{$module} = [];
}
push(@{$noref{$module}}, $name);
}
}
}
if (%noref) {
printf "\nExternally defined symbols with no external references\n";
foreach $module (sort(keys(%noref))) {
printf " $module\n";
foreach (sort(@{$noref{$module}})) {
if (exists($export{$_})) {
$export = " (export only)";
}
else {
$export = "";
}
printf " $_$export\n";
}
}
}
}
On Sat, Feb 02, 2002 at 02:42:36AM -0500, Daniel Jacobowitz wrote:
> One piece of the necessary compiler help would be -ffunction-sections.
> If they are in the same section in the same object file, they simply
> can not be removed safely.
Such as the patch that was mentioned earlier in this thread :)
> Relocation information for calls to local
> functions is not reliably available at link time.
With a smarter toolchain it could be.
One will need a smarter toolchain in order to re-order functions
anyway, which is an area where benchmarks on other compilers are
showing benefits. (ie. moving "cold" functions to the end of the
module, given profiling feedback)
Jeff
[email protected] said:
> There is also a problem with exported symbols. To ld, EXPORT_SYMBOL
> looks like a reference to the symbol,
Er, surely that's not a problem at all? This is desired behaviour?
> but the export entry is irrelevant, what really matters is if any module
> refers to those symbols.
Absolutely not. If we mark a symbol EXPORT_SYMBOL, we want it exported. No
questions asked.
The export entry _is_ relevant; furthermore it's the _only_ thing that's
relevant, and there is no way of knowing if there's a module that isn't in
the tree, or maybe even a module that _is_ in the tree but not compiled
today, that needs the symbol in question.
I sincerely hope it's just too early in the morning here and I'm
misunderstanding you :)
--
dwmw2
On Sat, 02 Feb 2002 08:40:11 +0000,
David Woodhouse <[email protected]> wrote:
>[email protected] said:
>> There is also a problem with exported symbols. To ld, EXPORT_SYMBOL
>> looks like a reference to the symbol,
>
>Er, surely that's not a problem at all? This is desired behaviour?
>
>> but the export entry is irrelevant, what really matters is if any module
>> refers to those symbols.
>
>Absolutely not. If we mark a symbol EXPORT_SYMBOL, we want it exported. No
>questions asked.
Does anything still use this symbol or is it dead? Quite valid question.
>The export entry _is_ relevant; furthermore it's the _only_ thing that's
>relevant, and there is no way of knowing if there's a module that isn't in
>the tree, or maybe even a module that _is_ in the tree but not compiled
>today, that needs the symbol in question.
I said that you get false positives because the code that uses the
symbol might not be compiled in your kernel. The script flags symbols
that are extern or exported and are not used in the current kernel.
Somebody then has to go through the report and decide if those symbols
are required under other config settings or by third party modules,
assuming that they care about third party modules.
The script is NOT an automatic list of what can be cleaned up. I doubt
that such a list can be generated in the face of the kernel config
system and third party modules. It is only a starting point for
symbols that need reviewing.
[email protected] said:
> Does anything still use this symbol or is it dead? Quite valid
> question.
Indeed.
> The script is NOT an automatic list of what can be cleaned up. I
> doubt that such a list can be generated in the face of the kernel
> config system and third party modules. It is only a starting point
> for symbols that need reviewing.
OK, cool. As it came out of the discussion about -ffunction-sections and
automatically garbage-collecting unused functions during the build, I
thought you had other, less sane intentions. Sorry for the misunderstanding.
--
dwmw2
On 31 January 2002 21:21, Arnaldo Carvalho de Melo wrote:
> Em Thu, Jan 31, 2002 at 11:24:10PM +0000, Alan Cox escreveu:
> > What I'd much rather see if this is an issue is:
> >
> > bool 'Do you want to customise for a very small system'
> >
> > which auto enables all the random small stuff if you say no, and goes
> > much deeper into options if you say yes.
>
> heh, after I've read that you managed to boot 2.4 + rmap in a machine with
> just 4 MB after tweaking some table sizes I thought about devoting some
> time to identify those tables and making them options in make *config, with
> even a nice CONFIG_TINY, like you said 8)
>
> I'll eventually do this, and I'd appreciate if people send me suggestions
> of tables/data structures that can be trimmed/reduced. Yeah, I'll take a
> look at the .config files used in the embedded distros.
One of inhabitants of my "Don't delete!" mail folder:
Re: Limited RAM - how to save it?
From: Lars Brinkhoff <[email protected]>
To: Jan-Benedict Glaw <[email protected]>
Cc: [email protected]
Jan-Benedict Glaw <[email protected]> writes:
> I'm working on a 4MB linux system (for a customer) which has quite
> limited resources [...] If you've got further ideas on getting the
> kernel a bit smaller, would be nice to get a mail dropped...
I started a port of Linux 2.3.99 to a MIPS device which usually has 2M
flash and 4M RAM.
To reduce the size of the text and data sections, I sorted the output
of the "size" command and investigated the files with the largest
sections.
These are the memory-saving changes I arrived at. They are not really
tested, so some of them may break some functionality. Also, I don't
know whether they apply to the current kernels.
fs/dcache.c
Changed HASH_BITS from 14 to 8. This reduces the size of the
cache from 128K to 2K.
fs/inode.c
Changed HASH_BITS from 14 to 8. This reduces the size of the
cache from 128K to 2K.
include/linux/blk.h
Changed NR_REQUEST from 256 to 16. This reduces the number of
requests that can be queued. The size of the queue is reduced
from 16K to 1K.
include/linux/major.h
Changed MAX_BLKDEV and MAX_CHRDEV from 256 to 10 and 20,
respectively. This reduces the number of block and character
devices and saves about 40K.
kernel/printk.c
Changed LOG_BUF_LEN from 16384 bytes to 2048 bytes.
include/linux/tty.h
Changed NR_PTYS and NR_LDISCS from 256 and 16, respectively,
to 16 and 4, respectively. Saved about 12K.
Warning: this change may break the pty driver, in which case
further modifications will have to be done to
drivers/char/pty.c.
kernel/panic.c
Changed a buffer from 1024 bytes to 200 bytes.
include/linux/sched.h
Changed PIDHASH_SZ from 1024 to 16, which saves 1008 bytes.
include/linux/mmzone.h
NR_GPFINDEX from 0x100 to 0x10. Saves 4800 bytes, but I'm not
sure it doesn't break anything.
net/Makefile, net/socket.c, net/nosocket.c
Replacing socket.c with nosocket.c, a file containing dummy
replacement functions for those in socket.c, saves about 24K.
Warning: this disables the socket API entirely, but it is
currently not used in the product.
mm/Makefile, mm/swapfile.c, mm/swap_state.c, mm/noswapfile.c,
mm/noswap_state.c
Replacing swapfile.c with noswapfile.c, and swap_state.c with
noswap_state.c saves about 12K. The no*.c files contains
empty replacement functions.
Warning: this disables swapping of anonymous memory, which
isn't used in the product. But note that demand paging of
executables still works.
mm/Makefile, mm/mmap.c
The functions in mmap.c could probably also be replaced by
empty functions. Estimated saving: 9K (not included in the
grand total below).
*, CONFIG_MESSAGES
Applying the CONFIG_MESSAGES patch and disabling all kernel
messages saves about 80K.
The CONFIG_MESSAGES patch was written by Graham Stoney
<[email protected]>.
With all of the above, and only this enabled in .config:
CONFIG_EXPERIMENTAL=y
CONFIG_CPU_R3000=y
CONFIG_CPU_LITTLE_ENDIAN=y
CONFIG_ELF_KERNEL=y
CONFIG_BINFMT_ELF=y
CONFIG_MODULES=y
CONFIG_MODVERSIONS=y
CONFIG_KMOD=y
CONFIG_CROSSCOMPILE=y
, the kernel is down to about 550K.
Here is the output of "size vmlinux". I think this is without the
CONFIG_MESSAGES patch (it was long since I worked with this).
text data bss dec hex filename
572128 41964 15860 629952 99cc0 vmlinux
--
Lars Brinkhoff http://lars.nocrew.org/ Linux, GCC, PDP-10
Brinkhoff Consulting http://www.brinkhoff.se/ programming
> include/linux/blk.h
>
> Changed NR_REQUEST from 256 to 16. This reduces the number of
> requests that can be queued. The size of the queue is reduced
> from 16K to 1K.
You can do even more than this -- I dunno what I/O interface these
embedded system generally uses (the mtd stuff?). Provided you provide a
direct make_request_fn entry into these instead of using the queue, you
can basically cut all of ll_rw_blk.c and elevator.c. ll_rw_block,
submit_bh, and generic_make_request would be all that is left.
That should reclaim quite a lot of space. If there's any interest in
this (has it already been done?), I can help out getting it done.
--
Jens Axboe
In article <[email protected]> you wrote:
>> include/linux/blk.h
>>
>> Changed NR_REQUEST from 256 to 16. This reduces the number of
>> requests that can be queued. The size of the queue is reduced
>> from 16K to 1K.
> You can do even more than this -- I dunno what I/O interface these
> embedded system generally uses (the mtd stuff?)
the MTD stuff wants CONFIG_BLOCKLAYER ;)
flash + jffs2 doesn't need any blocklayer at all ;)
On Sat, Feb 02 2002, [email protected] wrote:
> In article <[email protected]> you wrote:
> >> include/linux/blk.h
> >>
> >> Changed NR_REQUEST from 256 to 16. This reduces the number of
> >> requests that can be queued. The size of the queue is reduced
> >> from 16K to 1K.
>
> > You can do even more than this -- I dunno what I/O interface these
> > embedded system generally uses (the mtd stuff?)
>
> the MTD stuff wants CONFIG_BLOCKLAYER ;)
> flash + jffs2 doesn't need any blocklayer at all ;)
Right, that was exactly my point...
--
Jens Axboe
On Sat, Feb 02, 2002 at 03:08:47AM -0500, Jeff Garzik wrote:
> On Sat, Feb 02, 2002 at 02:42:36AM -0500, Daniel Jacobowitz wrote:
> > One piece of the necessary compiler help would be -ffunction-sections.
> > If they are in the same section in the same object file, they simply
> > can not be removed safely.
>
> Such as the patch that was mentioned earlier in this thread :)
Yes, the patch that he was objecting to :)
> > Relocation information for calls to local
> > functions is not reliably available at link time.
>
> With a smarter toolchain it could be.
>
> One will need a smarter toolchain in order to re-order functions
> anyway, which is an area where benchmarks on other compilers are
> showing benefits. (ie. moving "cold" functions to the end of the
> module, given profiling feedback)
No, the linker simply can not assume that this information is present.
That's a limitation of ELF. The right way to do this sort of
optimization is to compile every function into its own section and then
handle the (very small) performance cost of doing so by a relaxation
pass in the linker. You can't safely remove any part of an input
section, no matter what relocation information you think you have.
--
Daniel Jacobowitz Carnegie Mellon University
MontaVista Software Debian GNU/Linux Developer
Alan Cox <[email protected]> writes:
> > As a side note, this thing is so tiny (less than 4K on sparc64!) so
> > why don't we just include it unconditionally instead of having all
> > of this "turn it on for these drivers" stuff?
>
> Because 100 4K drivers suddenly becomes 0.5Mb. There are those of us trying
> to stuff Linux into embedded devices who if anything want more configuration
> options not people taking stuff out.
>
> What I'd much rather see if this is an issue is:
>
> bool 'Do you want to customise for a very small system'
>
> which auto enables all the random small stuff if you say no, and goes
> much deeper into options if you say yes.
I mostly agree. Except when I have looked at trying to get the kernel
(compiled size down) the biggest bloat was in the core. Things like
having both a page and a block cache.
Getting code reuse in the core higher would cut down on kernel size a
lot. But that isn't quick fix territory.
Eric
On Sat, Feb 02, 2002 at 09:14:33PM -0700, Eric W. Biederman wrote:
> > > As a side note, this thing is so tiny (less than 4K on sparc64!) so
> > > why don't we just include it unconditionally instead of having all
> > > of this "turn it on for these drivers" stuff?
> >
> > Because 100 4K drivers suddenly becomes 0.5Mb. There are those of us trying
> > to stuff Linux into embedded devices who if anything want more configuration
> > options not people taking stuff out.
> >
> > What I'd much rather see if this is an issue is:
> >
> > bool 'Do you want to customise for a very small system'
> >
> > which auto enables all the random small stuff if you say no, and goes
> > much deeper into options if you say yes.
>
> I mostly agree. Except when I have looked at trying to get the kernel
> (compiled size down) the biggest bloat was in the core. Things like
> having both a page and a block cache.
>
> Getting code reuse in the core higher would cut down on kernel size a
> lot. But that isn't quick fix territory.
Is it really worth the effort? During the past year the average size of
embedded systems that people want to use for seems to have increased
dramatically. In case of the MIPS port the core activity is about to
move away from the 32-bit to 64-bit kernel.
Ralf
On Sun, Feb 03, 2002 at 08:01:35AM +0100, Ralf Baechle wrote:
Is it really worth the effort? During the past year the average
size of embedded systems that people want to use for seems to have
increased dramatically. In case of the MIPS port the core
activity is about to move away from the 32-bit to 64-bit kernel.
For some hand-held devices (eg. iPAQ), we want as much memory free as
possible as the only filesystem available is ramfs... moving to 64-bit
would be unthinkable! :)
--cw
[email protected] said:
> You can do even more than this -- I dunno what I/O interface these
> embedded system generally uses (the mtd stuff?). Provided you provide
> a direct make_request_fn entry into these instead of using the queue,
> you can basically cut all of ll_rw_blk.c and elevator.c.
We don't even do that - look at jffs2_read_super(). The only reason it even
_looks_ like we're using a block device is because it was the easiest way
to arrange mounting. We only allow you to mount the 'mtdblock' device,
which isn't actually used - all we do is use the minor number to look up
the real underlying MTD device. The dependency on _any_ of the block code
isn't real - I can fix it as soon as there's an incentive to do so (like
CONFIG_BLK_DEV).
> ll_rw_block, submit_bh, and generic_make_request would be all that is left.
Those can go too. Likewise page->buffers.
> That should reclaim quite a lot of space. If there's any interest in
> this (has it already been done?), I can help out getting it done.
I had a go once. Long enough ago that it's probably not worth trying to find
it again. I'd suggest a boolean like CONFIG_BUFFER_CACHE which does the
really intrusive stuff like removing page->buffers, and CONFIG_BLK_DEV which
can be modular (if CONFIG_BUFFER_CACHE is on), and which controls
compilation of everything in drivers/block/ and fs/block_dev.c
--
dwmw2
[email protected] said:
> For some hand-held devices (eg. iPAQ), we want as much memory free as
> possible as the only filesystem available is ramfs...
You took the flash out of your iPAQ? How does it boot?
--
dwmw2
On Sun, Feb 03, 2002 at 12:16:00PM +0000, David Woodhouse wrote:
[email protected] said:
You took the flash out of your iPAQ? How does it boot?
Obviously it still has flash... but much of the filesystem is ramfs
and thus ram is rather precious.
For now anyhow, I think I'll upgrade it to 128M shortly.
--cw
[email protected] said:
> Obviously it still has flash... but much of the filesystem is ramfs
> and thus ram is rather precious.
Ah, I see. What are you doing with it? Mine hardly touches the ramfs.
sh-2.03# df
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/mtdblock/2 32256 26908 5348 84% /
tmpfs 15428 260 15168 2% /mnt/ramfs
--
dwmw2
> Is it really worth the effort? During the past year the average size of
> embedded systems that people want to use for seems to have increased
> dramatically. In case of the MIPS port the core activity is about to
> move away from the 32-bit to 64-bit kernel.
Embedded system and fancy handheld toys are not really the same thing. One
subset of the market gets cheaper the other gets flashier and more high
powered.
Mips is now dead in the handheld market if windows ce drops it.
remember that CML2 claims to be able to detect that it needs to be on and
turn it on when needed
David Lang
On Thu, 31 Jan 2002, Jeff Garzik wrote:
> Date: Thu, 31 Jan 2002 18:08:42 -0500
> From: Jeff Garzik <[email protected]>
> To: David S. Miller <[email protected]>
> Cc: [email protected], [email protected], [email protected],
> [email protected], [email protected], [email protected]
> Subject: Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3
> does not
>
> On Thu, Jan 31, 2002 at 02:59:04PM -0800, David S. Miller wrote:
> > As a side note, this thing is so tiny (less than 4K on sparc64!) so
> > why don't we just include it unconditionally instead of having all
> > of this "turn it on for these drivers" stuff?
>
> Does that 4K include the BE and LE crc tables?
>
> <shrug> I don't mind much either way, except that I am general
> resistant to "turn this on unconditionally" for bloat reasons.
> [ie. its a reflex :)]
>
> Jeff
>
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>