I'm working on an automated scheme to measure the growth of the kernel
from release to release. The current methodology is to do comparisons
from x to y, with the default config for x in both cases. This is
automated with bloat-o-meter from linux-tiny and my kpatchup script
(http://selenic.com/kpatchup) plus a couple glue scripts. I intend to
post these reports regularly with new releases so comments on
improving the content are appreciated.
Here's a current sample report [with some explanatory comments
interspersed].
2.6.3 -> 2.6.4
text data bss dec hex filename
3313135 660247 162472 4135854 3f1bae vmlinux-2.6.3-c2.6.3
3342019 664154 162344 4168517 3f9b45 vmlinux-2.6.4-c2.6.3
[ Results of size <a> <b>. -c2.6.3 means both kernel images were built
with the 2.6.3 defconfig. The dec column tells us that the 2.6.4
kernel image has grown by almost 1% in one release. Output from
bloat-o-meter follows.]
add/remove: 241/88 grow/shrink: 380/265 up/down: 44772/20838
[ A quick summary of modifications. 241 new symbols in 2.6.4, 88
disappeared, 380 functions grew, 265 shrank, and total growth and
shrinkage of listed symbols]
[ Next we list the growth and shrinkage of given functions, sorted by
delta. Symbols that aren't present in a given version are reported as
- and treated as size zero. I'm currently only listing the top 15 gainers
and decliners in the report to keep things manageable, though
bloat-o-meter will generate a complete list. ]
function old new delta
n_tty_receive_buf 1991 4450 +2459
_csr1212_read_keyval - 1253 +1253
hpsb_allocate_and_register_addrspace - 1139 +1139
iso_stream_schedule - 820 +820
do_shmat - 767 +767
csr1212_append_new_cache - 519 +519
csr1212_generate_csr_image - 510 +510
iso_stream_init - 505 +505
csr1212_new_icon_descriptor_leaf - 409 +409
early_serial_init - 398 +398
inet_confirm_addr - 374 +374
csr1212_parse_dir_entry - 350 +350
setup_early_printk - 349 +349
do_unblank_screen - 348 +348
csr1212_generate_tree_subdir - 342 +342
...
iso_stream_find 495 199 -296
parport_parse_params 321 - -321
parport_setup 749 419 -330
itd_stream_schedule 342 - -342
nodemgr_read_text_leaf 348 - -348
rpc_proc_read 350 - -350
svc_proc_read 361 - -361
read_businfo_block 436 - -436
nodemgr_scan_root_directory 470 - -470
acpi_boot_init 684 157 -527
nodemgr_scan_unit_directory 541 - -541
ide_end_taskfile 589 - -589
sys_shmat 767 - -767
ohci_init_config_rom 955 - -955
n_tty_receive_char 2037 - -2037
[ Note that renames like sys_shmat to do_shmat show up in the top and
the bottom. ]
--
Matt Mackall : http://www.selenic.com : Linux development and consulting
Matt Mackall <[email protected]> wrote:
>
> 2.6.3 -> 2.6.4
>
> text data bss dec hex filename
> 3313135 660247 162472 4135854 3f1bae vmlinux-2.6.3-c2.6.3
> 3342019 664154 162344 4168517 3f9b45 vmlinux-2.6.4-c2.6.3
>
> [ Results of size <a> <b>. -c2.6.3 means both kernel images were built
> with the 2.6.3 defconfig.
But defconfig was changed between 2.6.3 and 2.6.4.
On Fri, Mar 12, 2004 at 03:22:06PM -0800, Andrew Morton wrote:
> Matt Mackall <[email protected]> wrote:
> >
> > 2.6.3 -> 2.6.4
> >
> > text data bss dec hex filename
> > 3313135 660247 162472 4135854 3f1bae vmlinux-2.6.3-c2.6.3
> > 3342019 664154 162344 4168517 3f9b45 vmlinux-2.6.4-c2.6.3
> >
> > [ Results of size <a> <b>. -c2.6.3 means both kernel images were built
> > with the 2.6.3 defconfig.
>
> But defconfig was changed between 2.6.3 and 2.6.4.
Yes, and I'm attempting to compensate for that because defconfig
changes tend to overwhelm other stuff in the results.
My strategy here doesn't work as well as I'd hoped. I'm taking the
defconfig from the previous kernel and then running yes "" | make
oldconfig, which sets any new symbols to their defaults. So this deals
with case where existing symbols change defaults, but doesn't address
new symbols at all.
And what's happening with some of the new symbols is that they're off
in defconfig but on in Kconfig. So I need to come up with a way to
take the old defconfig and merge in new symbols from the new
defconfig. Then throw it at make oldconfig to drop out any obsolete
symbols.
--
Matt Mackall : http://www.selenic.com : Linux development and consulting
On Fri, Mar 12, 2004 at 05:53:49PM -0600, Matt Mackall wrote:
> On Fri, Mar 12, 2004 at 03:22:06PM -0800, Andrew Morton wrote:
> > Matt Mackall <[email protected]> wrote:
> > >
> > > 2.6.3 -> 2.6.4
> > >
> > > text data bss dec hex filename
> > > 3313135 660247 162472 4135854 3f1bae vmlinux-2.6.3-c2.6.3
> > > 3342019 664154 162344 4168517 3f9b45 vmlinux-2.6.4-c2.6.3
> > >
> > > [ Results of size <a> <b>. -c2.6.3 means both kernel images were built
> > > with the 2.6.3 defconfig.
> >
> > But defconfig was changed between 2.6.3 and 2.6.4.
>
> Yes, and I'm attempting to compensate for that because defconfig
> changes tend to overwhelm other stuff in the results.
>
> My strategy here doesn't work as well as I'd hoped. I'm taking the
> defconfig from the previous kernel and then running yes "" | make
> oldconfig, which sets any new symbols to their defaults. So this deals
> with case where existing symbols change defaults, but doesn't address
> new symbols at all.
>
> And what's happening with some of the new symbols is that they're off
> in defconfig but on in Kconfig. So I need to come up with a way to
> take the old defconfig and merge in new symbols from the new
> defconfig. Then throw it at make oldconfig to drop out any obsolete
> symbols.
You have to realize that your "automated scheme to measure the growth of
the kernel" doesn't work as you might have expected.
As an example, consider the following option gets added:
config ACPI_NEW_OPTION
bool "new option"
depends on ACPI
default y
help
This option adds support for $new_feature.
It enlarges your kernel by about 100k.
It's save to say Y.
In your size metric, this option would be pure "bloat".
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
On Sat, Mar 13, 2004 at 06:08:40PM +0100, Adrian Bunk wrote:
> On Fri, Mar 12, 2004 at 05:53:49PM -0600, Matt Mackall wrote:
> > On Fri, Mar 12, 2004 at 03:22:06PM -0800, Andrew Morton wrote:
> > > Matt Mackall <[email protected]> wrote:
> > > >
> > > > 2.6.3 -> 2.6.4
> > > >
> > > > text data bss dec hex filename
> > > > 3313135 660247 162472 4135854 3f1bae vmlinux-2.6.3-c2.6.3
> > > > 3342019 664154 162344 4168517 3f9b45 vmlinux-2.6.4-c2.6.3
> > > >
> > > > [ Results of size <a> <b>. -c2.6.3 means both kernel images were built
> > > > with the 2.6.3 defconfig.
> > >
> > > But defconfig was changed between 2.6.3 and 2.6.4.
> >
> > Yes, and I'm attempting to compensate for that because defconfig
> > changes tend to overwhelm other stuff in the results.
> >
> > My strategy here doesn't work as well as I'd hoped. I'm taking the
> > defconfig from the previous kernel and then running yes "" | make
> > oldconfig, which sets any new symbols to their defaults. So this deals
> > with case where existing symbols change defaults, but doesn't address
> > new symbols at all.
> >
> > And what's happening with some of the new symbols is that they're off
> > in defconfig but on in Kconfig. So I need to come up with a way to
> > take the old defconfig and merge in new symbols from the new
> > defconfig. Then throw it at make oldconfig to drop out any obsolete
> > symbols.
>
> You have to realize that your "automated scheme to measure the growth of
> the kernel" doesn't work as you might have expected.
>
> As an example, consider the following option gets added:
>
> config ACPI_NEW_OPTION
> bool "new option"
> depends on ACPI
> default y
> help
> This option adds support for $new_feature.
> It enlarges your kernel by about 100k.
> It's save to say Y.
>
>
> In your size metric, this option would be pure "bloat".
I actually did explicitly note this problem in the part you clipped.
But I think it's fair to say that new features that are on by default
are in fact bloat in some sense.
--
Matt Mackall : http://www.selenic.com : Linux development and consulting
On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
>
> I actually did explicitly note this problem in the part you clipped.
I clipped nothing, I quoted your _complete_ mail.
> But I think it's fair to say that new features that are on by default
> are in fact bloat in some sense.
Perhaps in some sense, but not in any interesting sense.
For the average computer you can buy at your supermarket today it isn't
very interesting whether the kernel is bigger by 1 MB or not.
People who need to care about the size of the kernel [1] use hand-tuned
.config's that are far away from defconfig - and those people wouldn't
enable unneeded features that are on by default.
You use a metric "size increase of a defconfig kernel [2]", and I simply
claim that this metric doesn't measure anything useful for practical
purposes.
cu
Adrian
[1] e.g. for small embedded systems, very old computers or
boot _floppies_
[2] modulo some compensation for changed defconfig's
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
> But I think it's fair to say that new features that are on by default
> are in fact bloat in some sense.
You cannot do any metric based on defconfig, since per. definition defconfig is what
suits Linus's current i386 PC best.
If you really want to do a proper metric do something like the follwoing:
make allnoconfig
Use some sed/awk magic to enable the options you are interested in
- Networking
- procfs?
- ext2
- One net driver
- A bit more which make sense
make oldconfig
This should result in a reliable config for code metrics.
You just have to inspect that noone of your manually added options
are no longer recognized (grep for warnings in output).
If you get this working it would be nice to see the evolution starting from 2.6.1 and
onwards in line with John Cherrys compile stats.
Sam
Matt Mackall <[email protected]> said:
> On Fri, Mar 12, 2004 at 03:22:06PM -0800, Andrew Morton wrote:
> > Matt Mackall <[email protected]> wrote:
> > >
> > > 2.6.3 -> 2.6.4
> > >
> > > text data bss dec hex filename
> > > 3313135 660247 162472 4135854 3f1bae vmlinux-2.6.3-c2.6.3
> > > 3342019 664154 162344 4168517 3f9b45 vmlinux-2.6.4-c2.6.3
> > >
> > > [ Results of size <a> <b>. -c2.6.3 means both kernel images were built
> > > with the 2.6.3 defconfig.
> >
> > But defconfig was changed between 2.6.3 and 2.6.4.
>
> Yes, and I'm attempting to compensate for that because defconfig
> changes tend to overwhelm other stuff in the results.
Use oldconfig?
Or just compare defconfig kernels (with all changes). Or all modules, ...
I.e., what are you trying to measure here?
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
On Sat, Mar 13, 2004 at 06:57:13PM +0100, Adrian Bunk wrote:
> On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
> >
> > I actually did explicitly note this problem in the part you clipped.
>
> I clipped nothing, I quoted your _complete_ mail.
Oh, sorry, indeed you did. And it was this bit:
> And what's happening with some of the new symbols is that they're
> off in defconfig but on in Kconfig. So I need to come up with a way
> to take the old defconfig and merge in new symbols from the new
> defconfig. Then throw it at make oldconfig to drop out any obsolete
> symbols.
> > But I think it's fair to say that new features that are on by default
> > are in fact bloat in some sense.
>
> Perhaps in some sense, but not in any interesting sense.
>
> For the average computer you can buy at your supermarket today it isn't
> very interesting whether the kernel is bigger by 1 MB or not.
>
> People who need to care about the size of the kernel [1] use hand-tuned
> .config's that are far away from defconfig - and those people wouldn't
> enable unneeded features that are on by default.
And my coverage of creep in other _commonly used_ parts of the kernel
would then be nil. Given that allyesconfig can't be expected to build
a kernel on any given day, defconfig is the least arbitrary and most
useful of arbitrary choices.
> You use a metric "size increase of a defconfig kernel [2]", and I simply
> claim that this metric doesn't measure anything useful for practical
> purposes.
defconfig is not an unreasonable approximation of features people use.
If something is added to defconfig, odds are that people will start
using it. Not perfect, obviously, but I've yet to see anyone suggest
anything else that actually provides some coverage.
--
Matt Mackall : http://www.selenic.com : Linux development and consulting
On Sat, Mar 13, 2004 at 11:17:41PM +0100, Sam Ravnborg wrote:
> On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
> > But I think it's fair to say that new features that are on by default
> > are in fact bloat in some sense.
>
> You cannot do any metric based on defconfig, since per. definition defconfig is what
> suits Linus's current i386 PC best.
> If you really want to do a proper metric do something like the follwoing:
>
> make allnoconfig
> Use some sed/awk magic to enable the options you are interested in
> - Networking
> - procfs?
> - ext2
> - One net driver
> - A bit more which make sense
> make oldconfig
And make oldconfig will go and turn on all the new stuff which is off
in defconfig and on in Kconfig.
But starting with a minimal config is not all that useful because it
doesn't show bloat creep in stuff people commonly use.
The point is not to distill everything down to one number, the point
is to get enough into the report that we can notice growth in all the
important areas. Obviously as things get renamed, moved around, etc.,
some closer inspection will be required to figure out what the impact
is. Changes in config workings are in the same bucket.
--
Matt Mackall : http://www.selenic.com : Linux development and consulting
On Sat, Mar 13, 2004 at 05:59:40PM -0600, Matt Mackall wrote:
> On Sat, Mar 13, 2004 at 06:57:13PM +0100, Adrian Bunk wrote:
>...
> > > But I think it's fair to say that new features that are on by default
> > > are in fact bloat in some sense.
> >
> > Perhaps in some sense, but not in any interesting sense.
> >
> > For the average computer you can buy at your supermarket today it isn't
> > very interesting whether the kernel is bigger by 1 MB or not.
> >
> > People who need to care about the size of the kernel [1] use hand-tuned
> > .config's that are far away from defconfig - and those people wouldn't
> > enable unneeded features that are on by default.
>
> And my coverage of creep in other _commonly used_ parts of the kernel
> would then be nil. Given that allyesconfig can't be expected to build
> a kernel on any given day, defconfig is the least arbitrary and most
> useful of arbitrary choices.
>
> > You use a metric "size increase of a defconfig kernel [2]", and I simply
> > claim that this metric doesn't measure anything useful for practical
> > purposes.
>
> defconfig is not an unreasonable approximation of features people use.
What exactly is your goal?
As already said:
*** For the average user, the size of the kernel doesn't matter *** [1]
*** People that care about size don't use defconfig ***
> If something is added to defconfig, odds are that people will start
> using it. Not perfect, obviously, but I've yet to see anyone suggest
> anything else that actually provides some coverage.
Did you ever consider that your approach of an "automated scheme" might
be an approach of very limited value?
cu
Adrian
[1] OK, 10 MB more would matter, but we are more in the ranges of
perhaps a few hundreds kB
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
On Sun, Mar 14, 2004 at 01:32:20AM +0100, Adrian Bunk wrote:
> On Sat, Mar 13, 2004 at 05:59:40PM -0600, Matt Mackall wrote:
> > On Sat, Mar 13, 2004 at 06:57:13PM +0100, Adrian Bunk wrote:
> >...
> > > > But I think it's fair to say that new features that are on by default
> > > > are in fact bloat in some sense.
> > >
> > > Perhaps in some sense, but not in any interesting sense.
> > >
> > > For the average computer you can buy at your supermarket today it isn't
> > > very interesting whether the kernel is bigger by 1 MB or not.
> > >
> > > People who need to care about the size of the kernel [1] use hand-tuned
> > > .config's that are far away from defconfig - and those people wouldn't
> > > enable unneeded features that are on by default.
> >
> > And my coverage of creep in other _commonly used_ parts of the kernel
> > would then be nil. Given that allyesconfig can't be expected to build
> > a kernel on any given day, defconfig is the least arbitrary and most
> > useful of arbitrary choices.
> >
> > > You use a metric "size increase of a defconfig kernel [2]", and I simply
> > > claim that this metric doesn't measure anything useful for practical
> > > purposes.
> >
> > defconfig is not an unreasonable approximation of features people use.
>
> What exactly is your goal?
>
> As already said:
> *** For the average user, the size of the kernel doesn't matter *** [1]
> *** People that care about size don't use defconfig ***
Neither of these things matter. The important thing is that defconfig
encompasses a range of common options that are likely to be used, thus
people care about growth in those areas regardless of what subset or
superset they actually use. It is not possible to see growth in the
code for option FOO if option FOO is not enabled. As I pointed out in
the last message, allyesconfig would be ideal for my purposes and
fails both of the above quite dramatically.
--
Matt Mackall : http://www.selenic.com : Linux development and consulting
On Sat, 2004-03-13 at 14:17, Sam Ravnborg wrote:
> On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
> > But I think it's fair to say that new features that are on by default
> > are in fact bloat in some sense.
>
> You cannot do any metric based on defconfig, since per. definition defconfig is what
> suits Linus's current i386 PC best.
> If you really want to do a proper metric do something like the follwoing:
>
> make allnoconfig
> Use some sed/awk magic to enable the options you are interested in
> - Networking
> - procfs?
> - ext2
> - One net driver
> - A bit more which make sense
> make oldconfig
>
> This should result in a reliable config for code metrics.
> You just have to inspect that noone of your manually added options
> are no longer recognized (grep for warnings in output).
>
> If you get this working it would be nice to see the evolution starting from 2.6.1 and
> onwards in line with John Cherrys compile stats.
I would be glad to add bloat stats to the existing compile statistics.
However, as has been mentioned, the default config options (defconfig,
allyesconfig, and allmodconfig) change over time. The value in the
bloat statistics would be up to the developer who could interpret these
stats relative to the current default configs.
>
> Sam
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
Matt Mackall wrote:
> On Sun, Mar 14, 2004 at 01:32:20AM +0100, Adrian Bunk wrote:
>
>>On Sat, Mar 13, 2004 at 05:59:40PM -0600, Matt Mackall wrote:
>>
>>>On Sat, Mar 13, 2004 at 06:57:13PM +0100, Adrian Bunk wrote:
>>>...
>>>
>>>>>But I think it's fair to say that new features that are on by default
>>>>>are in fact bloat in some sense.
>>>>
>>>>Perhaps in some sense, but not in any interesting sense.
>>>>
>>>>For the average computer you can buy at your supermarket today it isn't
>>>>very interesting whether the kernel is bigger by 1 MB or not.
>>>>
>>>>People who need to care about the size of the kernel [1] use hand-tuned
>>>>.config's that are far away from defconfig - and those people wouldn't
>>>>enable unneeded features that are on by default.
>>>
>>>And my coverage of creep in other _commonly used_ parts of the kernel
>>>would then be nil. Given that allyesconfig can't be expected to build
>>>a kernel on any given day, defconfig is the least arbitrary and most
>>>useful of arbitrary choices.
>>>
>>>
>>>>You use a metric "size increase of a defconfig kernel [2]", and I simply
>>>>claim that this metric doesn't measure anything useful for practical
>>>>purposes.
>>>
>>>defconfig is not an unreasonable approximation of features people use.
>>
>>What exactly is your goal?
>>
>>As already said:
>> *** For the average user, the size of the kernel doesn't matter *** [1]
>> *** People that care about size don't use defconfig ***
>
>
> Neither of these things matter. The important thing is that defconfig
> encompasses a range of common options that are likely to be used, thus
> people care about growth in those areas regardless of what subset or
> superset they actually use. It is not possible to see growth in the
> code for option FOO if option FOO is not enabled. As I pointed out in
> the last message, allyesconfig would be ideal for my purposes and
> fails both of the above quite dramatically.
With CONFIG_BROKEN, in the kernel for a while, why doesn't allyesconfig
work on a stock kernel? Maybe there are some logic errors in kconfig...
Adrian Bunk wrote:
> On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
>>But I think it's fair to say that new features that are on by default
>>are in fact bloat in some sense.
>
> Perhaps in some sense, but not in any interesting sense.
>
> For the average computer you can buy at your supermarket today it isn't
> very interesting whether the kernel is bigger by 1 MB or not.
>
> People who need to care about the size of the kernel [1] use hand-tuned
> .config's that are far away from defconfig - and those people wouldn't
> enable unneeded features that are on by default.
>
> You use a metric "size increase of a defconfig kernel [2]", and I simply
> claim that this metric doesn't measure anything useful for practical
> purposes.
Well maybe the bloat meter is helpful for identifying bloated
features that the kernel developers added to the default configs,
so embedded guys can know to avoid them, or, if they're
interesting, try to unbloat.
Even hand tuners can use some help. The kernel is vast and
progress is fast. (Didn't mean to make a ryhme... ;)
=============================
Tim Bird
Architecture Group Co-Chair
CE Linux Forum
Senior Staff Engineer
Sony Electronics
E-mail: [email protected]
=============================