Okay, as of this point, I think klibc is in quite good shape; my
testing so far is showing that it can be used as a drop-in replacement
for the kernel root-mounting code.
That being said, there is guaranteed to be breakage, for two reasons:
a. There are several architectures which don't have klibc ports yet.
Since I don't have access to them, I can't really do them, either.
It's usually a matter of an afternoon or less to port klibc to a
new architecture, though, if you have a working development
environment for it.
b. There are a gajillion options in the way Linux handles its root
filesystem. I have tried to implement them all, but I haven't been
able to test some of the more exotic conditions, mostly due to the
fact that every boot environment I set up takes a *lot* of time and
maintenance. This has in fact been the by far the bulk of the time
I've spent on klibc, not writing code.
Thus, it's not clear to me what particular approach makes most sense for
pushing upstream.
1. Mechanism: the easiest for me would of course be git pull, but I'm
certainly willing to break it up into patches or any other useful way.
A diff of by git tree from 2.6.16 gives:
766 files changed, 84024 insertions(+), 3059 deletions(-)
2. State: the current git tree cleans up the arch-specific code for i386
and x86-64; other architectures are going to need some minor cleanup in
setup.c or the equvalent.
The current git tree includes a number of utilities, like dash (sh),
which aren't used by the default kinit configuration. Additionally,
right now kinit is built monolitically, in other words there isn't a
CONFIG_ option to turn off nfsmount, for example.
Again, I'm more than willing to put the tree in any particular state
that makes sense.
3. Path: it probably would make sense to push this into -mm first?
It's taken this long, I'd like to make this actually happen...
-hpa
On Mon, 2006-03-20 11:54:01 -0800, H. Peter Anvin <[email protected]> wrote:
> a. There are several architectures which don't have klibc ports yet.
> Since I don't have access to them, I can't really do them, either.
> It's usually a matter of an afternoon or less to port klibc to a
> new architecture, though, if you have a working development
> environment for it.
I haven't yet looked at your code, but what actually needs to be done?
Defining syscall macros?
I'd probably give it a try for VAX.
MfG, JBG
--
Jan-Benedict Glaw [email protected] . +49-172-7608481 _ O _
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O
für einen Freien Staat voll Freier Bürger" | im Internet! | im Irak! O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
On Mon, Mar 20, 2006 at 11:54:01AM -0800, H. Peter Anvin wrote:
> Okay, as of this point, I think klibc is in quite good shape; my
> testing so far is showing that it can be used as a drop-in replacement
> for the kernel root-mounting code.
>
> That being said, there is guaranteed to be breakage, for two reasons:
>
> a. There are several architectures which don't have klibc ports yet.
> Since I don't have access to them, I can't really do them, either.
> It's usually a matter of an afternoon or less to port klibc to a
> new architecture, though, if you have a working development
> environment for it.
Which ones?
Hi,
first thanks for your good work on that!
On Monday, 20. March 2006 20:54, H. Peter Anvin wrote:
> The current git tree includes a number of utilities, like dash (sh),
> which aren't used by the default kinit configuration.
But its quite useful to be dropped into a shell, if anything goes wrong.
e.g. ubuntu assumes an modular kernel, which I don't like to have
and I got dropped into the shell quite often until all scripts worked as
expected.
> Additionally,
> right now kinit is built monolitically, in other words there isn't a
> CONFIG_ option to turn off nfsmount, for example.
Yes, since after development of your init setup, you want to shrink it
as much as possible :-)
But all of this stuff is janitorial and can be done after merging the basic
ideas and completing the setup code move to user space.
Another idea is to still allow the current archive setups for initramfs,
because a static /dev might be best for embedded stuff where
you basically pre-compose a system image and download that to flash.
> 3. Path: it probably would make sense to push this into -mm first?
The usr/ part should go in ASAP, but please rip out the setup stuff
only after a grace period announced in Documentation/feature-removal.txt
or sth. like that.
Regards
Ingo Oeser
Al Viro wrote:
> On Mon, Mar 20, 2006 at 11:54:01AM -0800, H. Peter Anvin wrote:
>
>>Okay, as of this point, I think klibc is in quite good shape; my
>>testing so far is showing that it can be used as a drop-in replacement
>>for the kernel root-mounting code.
>>
>>That being said, there is guaranteed to be breakage, for two reasons:
>>
>>a. There are several architectures which don't have klibc ports yet.
>> Since I don't have access to them, I can't really do them, either.
>> It's usually a matter of an afternoon or less to port klibc to a
>> new architecture, though, if you have a working development
>> environment for it.
>
> Which ones?
>
This is the status of architectures in klibc, at least as far as I know.
Note that 64-bit architectures which have 32-bit fallback modes (e.g.
MIPS) can use the 32-bit klibc if applicable.
alpha: Working
arm-thumb: Untested
arm: Working
arm26: Not yet ported
cris: Working
h8300: Not yet ported
i386: Working
ia64: Working
m32r: Untested
m68k: Not yet ported
mips: Working
mips64: Not yet ported
parisc: Working
parisc64: Not yet ported
ppc: Working
ppc64: Working
s390: Working static, shared untested
s390x: Working
sh: Untested
sh64: Not yet ported
sparc: Working
sparc64: Untested
v850: Not yet ported
x86-64: Working
xtensa: Not yet ported
-hpa
Jan-Benedict Glaw wrote:
>
> I haven't yet looked at your code, but what actually needs to be done?
> Defining syscall macros?
>
> I'd probably give it a try for VAX.
>
Each architecture needs an assembly stub generator for most syscalls, a
crt0.S, plus adjustments for which syscalls are "special" (as in special
ed) on that architecture. There are also a couple of header files which
need per-architecture configuration information. All in all it is
usually a few hours worth of work, no more.
-hpa
Hi,
On Mon, 20 Mar 2006, H. Peter Anvin wrote:
You forgot to provide any information (at least a summary) about what this
is and how this will work. Please don't assume everyone is familiar with
it.
There is one major question: how will this interface to distributions?
How can distributions add their own initializations and configurations or
are they going to put an initrd on top of the kernel initrd? If this will
have a kernel and a distribution part, it poses the question whether klibc
has to be distributed with the kernel at all (a libc has a standard API
after all) and the kernel just provides the kernel specific parts to
whatever the distribution provides.
bye, Roman
H. Peter Anvin wrote:
> Okay, as of this point, I think klibc is in quite good shape; my
> testing so far is showing that it can be used as a drop-in replacement
> for the kernel root-mounting code.
[]
> Thus, it's not clear to me what particular approach makes most sense for
> pushing upstream.
Why this needs to be "pushed" upstream in the first place? Isn't it
simpler/easier/whatever to just require klibc to be present on the
build system for kernel? If klibc is "sufficiently" independent of
the kernel (is it? I see no reason it shouldn't), why it should go
with kernel? Just point your CONFIG_INITRAMFS_SOURCE to some klibc
directory tree and be done with it, no need to distribute/build
klibc with kernel..
Thanks.
/mjt
Followup to: <[email protected]>
By author: Roman Zippel <[email protected]>
In newsgroup: linux.dev.kernel
>
> You forgot to provide any information (at least a summary) about what this
> is and how this will work. Please don't assume everyone is familiar with
> it.
>
> There is one major question: how will this interface to distributions?
>
> How can distributions add their own initializations and configurations or
> are they going to put an initrd on top of the kernel initrd? If this will
> have a kernel and a distribution part, it poses the question whether klibc
> has to be distributed with the kernel at all (a libc has a standard API
> after all) and the kernel just provides the kernel specific parts to
> whatever the distribution provides.
>
Okay... quick summary (again)...
klibc is a small libc, small enough that it provides negible (or even
negative) overhead to bundle it inside the kernel binary.
The kernel tree part is there so that we can rip out in-kernel code
without breaking compatibility, or requiring a distribution-provided
initramfs. In the future, we could consider retaining certain
binaries in the rootfs and have "on-demand userspace" by the kernel,
e.g. to do partition enumeration in userspace in a
backwards-compatible manner.
The default build provides a single binary called kinit, which is
(modulo any bugs) equivalent to the in-kernel root-mounting code, with
all its variants (initrd, nfsroot, load ramdisk from floppy, yadda
yadda.) The existence of kinit allows the in-kernel code to be
removed without actually removing a feature. Hence, the reason to put
this in the kernel tree is to make sure there is zero impact on
distributions.
If the distribution uses initramfs directly, kinit goes unused. The
klibc code is also available as a standalone distribution, which at
least Ubuntu is currently using to build a custom initramfs. Because
the kinit code is still userspace, it can share considerable amounts
of code with the standalone klibc utilities collection; in fact most
of the kinit pieces are available as standalone binaries which can be
weaved together by scripts or other C code.
The advantages of moving this code to userspace, thus is:
- Simpler programming model (harder to screw up)
- Easier to share code with distribution-customized setups
- Code can be tested as standalone userspace binaries at runtime
A lot of the benefit is lost if, like now, there is a piece of code
which has to be written for kernel-mode programming, separate from
anything else and not testable except through a tedious kernel boot
cycle.
-hpa
On 3/22/06, H. Peter Anvin <[email protected]> wrote:
> Followup to: <[email protected]>
> By author: Roman Zippel <[email protected]>
> In newsgroup: linux.dev.kernel
> >
> > You forgot to provide any information (at least a summary) about what this
> > is and how this will work. Please don't assume everyone is familiar with
> > it.
> >
> > There is one major question: how will this interface to distributions?
> >
> > How can distributions add their own initializations and configurations or
> > are they going to put an initrd on top of the kernel initrd? If this will
> > have a kernel and a distribution part, it poses the question whether klibc
> > has to be distributed with the kernel at all (a libc has a standard API
> > after all) and the kernel just provides the kernel specific parts to
> > whatever the distribution provides.
> >
>
> Okay... quick summary (again)...
>
> klibc is a small libc, small enough that it provides negible (or even
> negative) overhead to bundle it inside the kernel binary.
>
> The kernel tree part is there so that we can rip out in-kernel code
> without breaking compatibility, or requiring a distribution-provided
> initramfs. In the future, we could consider retaining certain
> binaries in the rootfs and have "on-demand userspace" by the kernel,
> e.g. to do partition enumeration in userspace in a
> backwards-compatible manner.
>
> The default build provides a single binary called kinit, which is
> (modulo any bugs) equivalent to the in-kernel root-mounting code, with
> all its variants (initrd, nfsroot, load ramdisk from floppy, yadda
> yadda.) The existence of kinit allows the in-kernel code to be
> removed without actually removing a feature. Hence, the reason to put
> this in the kernel tree is to make sure there is zero impact on
> distributions.
>
> If the distribution uses initramfs directly, kinit goes unused. The
> klibc code is also available as a standalone distribution, which at
> least Ubuntu is currently using to build a custom initramfs. Because
> the kinit code is still userspace, it can share considerable amounts
> of code with the standalone klibc utilities collection; in fact most
> of the kinit pieces are available as standalone binaries which can be
> weaved together by scripts or other C code.
>
> The advantages of moving this code to userspace, thus is:
>
> - Simpler programming model (harder to screw up)
> - Easier to share code with distribution-customized setups
> - Code can be tested as standalone userspace binaries at runtime
>
> A lot of the benefit is lost if, like now, there is a piece of code
> which has to be written for kernel-mode programming, separate from
> anything else and not testable except through a tedious kernel boot
> cycle.
>
> -hpa
ISTM that we are (finally? ;) moving piece by piece to a mixed
monolothic+microkenel, or rather that many of the things that were
first implemented in-kernel (on linux or other unixes) are being
slowly jettisoned to a kernel-provided userspace. All in all this is a
good step forward :)
Regarding helping test/develop this, is there any small distro already
using klibc for such purposes? Maybe you, hpa, could share you klibc
testing rig? This looks ripe for a qemu-based testing at the moment,
not being performance critical like many other current developments...
--
Greetz, Antonio Vargas aka winden of network
http://wind.codepixel.com/
[email protected]
[email protected]
Every day, every year
you have to work
you have to study
you have to scene.
Antonio Vargas wrote:
>
> Regarding helping test/develop this, is there any small distro already
> using klibc for such purposes? Maybe you, hpa, could share you klibc
> testing rig? This looks ripe for a qemu-based testing at the moment,
> not being performance critical like many other current developments...
>
The klibc-integrated git tree is at:
git://git.kernel.org/pub/scm/linux/kernel/git/hpa/linux-2.6-klibc.git
At this point, the big thing is to drop this into as many
weirdly-booting Linux setups as possible, and watch them hopefully have
no effect whatsoever :)
For porting klibc, I have a set of tests which are also included.
Unfortunately, they're not currently self-validating, which would be a
significant enhancement.
-hpa