2003-05-02 00:00:17

by Randy.Dunlap

[permalink] [raw]
Subject: Re: Kernel source tree splitting

On Wed, 30 Apr 2003 17:21:02 -0700 "Randy.Dunlap" <[email protected]> wrote:

| Hi,
|
| I'm probably misreading this...but,
|
| Have you tried this yet? Does it modify/customize all Kconfig
| and Makefiles for the selected tree splits?
|
| A few days ago, in one tree, I rm-ed arch/{all that I don't need}
| and drivers/{all that I don't need}.
| After that I couldn't run "make *config" because it wants all of
| those files, even if I don't want them.
|
| So there are many edits that needed to be done in lots of
| Kconfig and Makefiles if one selectively pulls or omits certain
| sub-directories.

and on 2003-04-30 rddunlap wrote:
| I seem to try for simple solutions when possible and feasible.
|
| In this case, if I were doing it, I would try changing (e.g.) in
| arch/i386/kernel/Kconfig, this line:
| source "drivers/eisa/Kconfig"
| to
| optsource "drivers/eisa/Kconfig"
| where optsource means that the file is optional -- if not found,
| ignore it. And then see what happens, how far it can go,
| what the next problem is....
|
| If this could be made to work, then entire subdirectories/subsystems
| could be optional.

So I did a proof-of-concept version of this, without modifying any
source code. I rm-ed arch/<many>, drivers/<many>, and fs/<many>
and then wrote a shell script that looks for missing dirs, Kconfigs,
and Makefile.lib files and puts empty ones back in their places.
The shell script only works for what I rm-ed, but it could be made
smarter if anyone wants to pursue this. (attached)

After doing that I was able to build and boot that kernel, so it
(concept) did work.

For a kernel source tree that hadn't been built/compiled in, the size
was reduced from roughly 200 MB down to roughly 133 MB.

~Randy


| On Wed, 30 Apr 2003 19:46:13 -0400 rmoser <[email protected]> wrote:
|
| | Eh, Linus won't be happy making a bunch of tarballs.
| | I've made it less work if you read the message here...
| |
| | The message mirrored at:
| |
| | http://marc.theaimsgroup.com/?l=linux-kernel&m=105173077417526&w=2
| |
| | Shows my pre-thought on this subject. I thought a bit more,
| | and began to come up with a simple sketch to lead the
| | way in case anyone becomes interested.
| |
| | First off, the kernel tarballs would be built by a script
| | that splits the source tree apart appropriately and tar's it
| | up. How this is done is explained.
| |
| | Second off, there's always a script to download that runs
| | wget and gets the source tree from which it was downloaded.
| | The whole thing. As in, every tarball is downloaded and
| | untar'd for the user, assembling the full kernel source
| | tree (as it would be if you untar'd it now).
| |
| | Now, I explained LOD's in that message above in small
| | detail. But, for clarity, LOD's are files which explain
| | which pieces of source in the kernel tree belong to the
| | LOD; what gets added to the config; where their makefiles
| | are; what config options depend on other linux options;
| | and what groups these LOD's are in.
| |
| | A command such as `make disttree` should read the LOD's,
| | split apart each linux option, tar 'em together, and
| | then compress the tar's. Then Linus could just scp the
| | new directory of tar's and a script up.
| |
| | As for download, the script that goes up can be
| | downloaded (duh), and then run (... why do I bother?).
| | Now this script would run in "dumb mode" (unless the
| | user tells it not to maybe?) and rip down the whole
| | tree, untar it, and rebuild the original source tree.
| | I think. I'm not sure, I really haven't tried yet.
| | I'll tell you how it works after it's implimented, if
| | ever that happens. This would likely require wget.
| |
| | Of course there's always the ftp method. Go download
| | the pieces you want, untar 'em, copy 'em to the same
| | directory, and the build system adjusts. but newbies
| | and developers, for completely opposite reasons, will
| | want to use the script in dumb mode.
| |
| | For experienced users, this will make configuration
| | somewhat easier, as the user can avoid being prompted
| | for irrelavent drivers. This is just a concept idea,
| | not a fully thought-out idea. What do you think?
| |
| | --Bluefox Icy


Attachments:
builder (1.00 kB)

2003-05-02 00:31:47

by rmoser

[permalink] [raw]
Subject: Re: Kernel source tree splitting



*********** REPLY SEPARATOR ***********

On 5/1/2003 at 5:09 PM Randy.Dunlap wrote:

>On Wed, 30 Apr 2003 17:21:02 -0700 "Randy.Dunlap" <[email protected]>
>wrote:
>
>| Hi,
>|
>| I'm probably misreading this...but,
>|
>| Have you tried this yet? Does it modify/customize all Kconfig
>| and Makefiles for the selected tree splits?
>|
>| A few days ago, in one tree, I rm-ed arch/{all that I don't need}
>| and drivers/{all that I don't need}.
>| After that I couldn't run "make *config" because it wants all of
>| those files, even if I don't want them.
>|
>| So there are many edits that needed to be done in lots of
>| Kconfig and Makefiles if one selectively pulls or omits certain
>| sub-directories.
>
>and on 2003-04-30 rddunlap wrote:
>| I seem to try for simple solutions when possible and feasible.
>|
>| In this case, if I were doing it, I would try changing (e.g.) in
>| arch/i386/kernel/Kconfig, this line:
>| source "drivers/eisa/Kconfig"
>| to
>| optsource "drivers/eisa/Kconfig"
>| where optsource means that the file is optional -- if not found,
>| ignore it. And then see what happens, how far it can go,
>| what the next problem is....
>|
>| If this could be made to work, then entire subdirectories/subsystems
>| could be optional.
>
>So I did a proof-of-concept version of this, without modifying any
>source code. I rm-ed arch/<many>, drivers/<many>, and fs/<many>
>and then wrote a shell script that looks for missing dirs, Kconfigs,
>and Makefile.lib files and puts empty ones back in their places.
>The shell script only works for what I rm-ed, but it could be made
>smarter if anyone wants to pursue this. (attached)
>

Yes. Well the build system (kernel configuration) would be modified
to instead of having a list of Kconfigs and dir's and Makefile.libs and
such, be able to scan a directory of files which tell it about those
things. At least, in my design it would.

>After doing that I was able to build and boot that kernel, so it
>(concept) did work.
>

Well that's good.

>For a kernel source tree that hadn't been built/compiled in, the size
>was reduced from roughly 200 MB down to roughly 133 MB.
>

............... You know. Maybe it's not enough to split into categories.
Maybe it should be categories and categorical breakdowns. I can see
3.0 or something (maybe even 2.8) reaching 1 gig.

--Bluefox Icy
>~Randy
>
>