Folks, IMO initramfs (aka. late boot in userland) is suitable
for testing.
Patches are on ftp.math.psu.edu/pub/viro, namespaces-a-S8-pre2 and
initramfs-a-S8-pre2 (the latter is against 2.4.8-pre2 + namespaces).
It is supposed to be a drop-in replacement - any boot setup that works
with vanilla 2.4 should work with it, initrd or not, with or without devfs,
with loading from floppies, etc.
In other words, if you boot normally with 2.4 and something breaks with
initramfs - I want to know about that.
Stuff that went in userland: choosing and mounting root, unpacking/loading
initrd, running /linuxrc, handling nfsroot, finding and starting final
init - basically, everything after do_basic_setup().
The thing unpacks cpio archive (currently - linked into the kernel image)
on root ramfs and execs /init. After that we are in userland code. Said
code (source in init/init.c and init/nfsroot.c) emulates the vanilla
2.4 behaviour. You can replace it with your own - that's just the default
that gives (OK, is supposed to give) a backwards-compatible behaviour.
Thing that had not gone into the userland, but should be there: ipconfig.c.
If somebody feels like writing a userland equivalent - I'd be very
grateful to see it. Currently it's still in the kernel.
Another thing that is definitely needed is less crude RPC (for nfsroot).
Currently it's _very_ quick-and-dirty.
At that stage I'm mostly interested in bug reports regarding the cases
when behaviour differs from the vanilla tree. I _know_ that we need more
elegant way to add initial archive to the kernel image. That's a
separate issue and I'd rather deal with that once userland implementation
of late-boot is decently debugged.
Right now it's x86-only, but that's the matter of adding minimal replacement
of crt1.o for other platforms (i.e. code that picks argc, argv and envp
and calls main() - 7 lines of assembler for x86 and probably about the
same on other platforms). Equivalents of arch/i386/kernel/start.S (see
the patch) are welcome.
It should be pretty safe to debug, for a change - it either gets to
starting /sbin/init (in which case we are done and safe) or it breaks
before any local fs is mounted r/w.
Linus, I'm not putting these patches in the posting - each of them is
above 100Kb and that's way beyond any sane l-k limits. If you want
to get them in email - tell and I'll send them. And yes, I'm going
to split this stuff in small chunks when it will come to submitting
it.
Al
I wonder... May the initramfs be used also for loading modules ???
Hmm... it will require a pico-insmod that can run in the limited initramfs
environment, but I believe that's all !
Reminder-to-self: try this at home...
This may bring the long-awaited revolution in kernel building (everything
is a module!)
-az
----- Original Message -----
From: "Alexander Viro" <[email protected]>
To: <[email protected]>
Cc: "Linus Torvalds" <[email protected]>;
<[email protected]>
Sent: Monday, July 30, 2001 8:05
Subject: [CFT] initramfs patch
> Folks, IMO initramfs (aka. late boot in userland) is suitable
> for testing.
>
> Patches are on ftp.math.psu.edu/pub/viro, namespaces-a-S8-pre2 and
> initramfs-a-S8-pre2 (the latter is against 2.4.8-pre2 + namespaces).
>
> It is supposed to be a drop-in replacement - any boot setup that works
> with vanilla 2.4 should work with it, initrd or not, with or without
devfs,
> with loading from floppies, etc.
>
> In other words, if you boot normally with 2.4 and something breaks with
> initramfs - I want to know about that.
>
> Stuff that went in userland: choosing and mounting root, unpacking/loading
> initrd, running /linuxrc, handling nfsroot, finding and starting final
> init - basically, everything after do_basic_setup().
>
> The thing unpacks cpio archive (currently - linked into the kernel image)
> on root ramfs and execs /init. After that we are in userland code. Said
> code (source in init/init.c and init/nfsroot.c) emulates the vanilla
> 2.4 behaviour. You can replace it with your own - that's just the default
> that gives (OK, is supposed to give) a backwards-compatible behaviour.
>
> Thing that had not gone into the userland, but should be there:
ipconfig.c.
> If somebody feels like writing a userland equivalent - I'd be very
> grateful to see it. Currently it's still in the kernel.
>
> Another thing that is definitely needed is less crude RPC (for nfsroot).
> Currently it's _very_ quick-and-dirty.
>
> At that stage I'm mostly interested in bug reports regarding the cases
> when behaviour differs from the vanilla tree. I _know_ that we need more
> elegant way to add initial archive to the kernel image. That's a
> separate issue and I'd rather deal with that once userland implementation
> of late-boot is decently debugged.
>
> Right now it's x86-only, but that's the matter of adding minimal
replacement
> of crt1.o for other platforms (i.e. code that picks argc, argv and envp
> and calls main() - 7 lines of assembler for x86 and probably about the
> same on other platforms). Equivalents of arch/i386/kernel/start.S (see
> the patch) are welcome.
>
> It should be pretty safe to debug, for a change - it either gets to
> starting /sbin/init (in which case we are done and safe) or it breaks
> before any local fs is mounted r/w.
>
> Linus, I'm not putting these patches in the posting - each of them is
> above 100Kb and that's way beyond any sane l-k limits. If you want
> to get them in email - tell and I'll send them. And yes, I'm going
> to split this stuff in small chunks when it will come to submitting
> it.
> Al
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
On Mon, 30 Jul 2001 18:25:27 +0200,
"Alon Ziv" <[email protected]> wrote:
>I wonder... May the initramfs be used also for loading modules ???
>Hmm... it will require a pico-insmod that can run in the limited initramfs
>environment, but I believe that's all !
Busybox includes a stripped down insmod, along with lots of other tools.
http://www.lineo.com/products/technical_bulletins/busy_box046.html
Alon Ziv <[email protected]> wrote:
>I wonder... May the initramfs be used also for loading modules ???
>Hmm... it will require a pico-insmod that can run in the limited initramfs
>environment, but I believe that's all !
I've built modutils against Felix von Leitner's dietlibc; that might
fit the bill. See these two pages:
http://www.fefe.de/dietlibc/
http://www.leftmind.net/projects/misc/
--
Anthony de Boer, curator, Anthony's Home for Aged Computing Machinery
<[email protected]>
On Mon, 30 Jul 2001, Alon Ziv wrote:
> I wonder... May the initramfs be used also for loading modules ???
> Hmm... it will require a pico-insmod that can run in the limited initramfs
> environment, but I believe that's all !
> Reminder-to-self: try this at home...
> This may bring the long-awaited revolution in kernel building (everything
> is a module!)
Why not? Kernel unpacks cpio archive on root (which lives on ramfs) and
starts /init. That's it - after that you are in userland and free to
do everything you would normally do. If there's a variant of insmod that
would not bring glibc for a ride...
On Mon, Jul 30, 2001 at 02:05:55AM -0400, Alexander Viro wrote:
>
> The thing unpacks cpio archive (currently - linked into the kernel image)
> on root ramfs and execs /init. After that we are in userland code. Said
> code (source in init/init.c and init/nfsroot.c) emulates the vanilla
> 2.4 behaviour. You can replace it with your own - that's just the default
> that gives (OK, is supposed to give) a backwards-compatible behaviour.
One thing that would make embedded systems developers very happy
is the ability to map a romfs or cramfs filesystem directly from
the kernel image, avoiding the extra copy necessitated by the cpio
archive. Are there problems with this approach?
miket
On Mon, 30 Jul 2001, Mike Touloumtzis wrote:
> On Mon, Jul 30, 2001 at 02:05:55AM -0400, Alexander Viro wrote:
> >
> > The thing unpacks cpio archive (currently - linked into the kernel image)
> > on root ramfs and execs /init. After that we are in userland code. Said
> > code (source in init/init.c and init/nfsroot.c) emulates the vanilla
> > 2.4 behaviour. You can replace it with your own - that's just the default
> > that gives (OK, is supposed to give) a backwards-compatible behaviour.
>
> One thing that would make embedded systems developers very happy
> is the ability to map a romfs or cramfs filesystem directly from
> the kernel image, avoiding the extra copy necessitated by the cpio
> archive. Are there problems with this approach?
a) IIRC, both are read-only.
b) what stops you from doing initramfs + romfs-on-initrd? It works.
On Mon, 30 Jul 2001, Mike Touloumtzis wrote:
> On Mon, Jul 30, 2001 at 02:05:55AM -0400, Alexander Viro wrote:
> > The thing unpacks cpio archive (currently - linked into the kernel image)
> > on root ramfs and execs /init. After that we are in userland code. Said
> > code (source in init/init.c and init/nfsroot.c) emulates the vanilla
> > 2.4 behaviour. You can replace it with your own - that's just the default
> > that gives (OK, is supposed to give) a backwards-compatible behaviour.
>
> One thing that would make embedded systems developers very happy
> is the ability to map a romfs or cramfs filesystem directly from
> the kernel image, avoiding the extra copy necessitated by the cpio
> archive. Are there problems with this approach?
Yes -- you need to at that point store initialized structures. Store
the dcache in its unpacked state on the ROM image, etc. That's the only
way to "map" a romfs directly. Otherwise there is ALWAYS an unpacking
or translation step between filesystem image and in-memory image.
Mapping an in-memory image directly may seem like a good idea, but it is
really not. ESPECIALLY for embedded folks.
Use a programmatic solution to unpack your filesystem... like say cpio
format in initramfs. :)
Jeff
On Mon, Jul 30, 2001 at 03:50:33PM -0500, Jeff Garzik wrote:
> On Mon, 30 Jul 2001, Mike Touloumtzis wrote:
> >
> > One thing that would make embedded systems developers very happy
> > is the ability to map a romfs or cramfs filesystem directly from
> > the kernel image, avoiding the extra copy necessitated by the cpio
> > archive. Are there problems with this approach?
>
> Yes -- you need to at that point store initialized structures. Store
> the dcache in its unpacked state on the ROM image, etc. That's the only
> way to "map" a romfs directly. Otherwise there is ALWAYS an unpacking
> or translation step between filesystem image and in-memory image.
>
> Mapping an in-memory image directly may seem like a good idea, but it is
> really not. ESPECIALLY for embedded folks.
I think you're misunderstanding what I propose. I'm talking about
having a device in /dev that would allow access to a filesystem
image (cramfs or romfs) that would be embedded in the in-memory
kernel image.
So yes, there would be an unpacking/translation step to get at the
file data, but it would be the normal memory map/page fault process
combined with the filesystem functionality already in cramfs/romfs,
and (more importantly) it would allow text pages to be dropped and
later reloaded from the kernel image, instead of duplicating data
from the kernel image into a nonpageable ramfs. There would still
be a RAM hit but it would just be the dcache, icache, and other
such in-core metadata, not the entire contents of the files.
The reasons to integrate this into an infrastructure like the new
initramfs (instead of, say, catting the fs image onto the end of
the kernel) are:
a) The filesystem will have alignment requirements, which are
easily specified in a linker script, and
b) We would want a block device to perform the process I describe
above (it essentially just be a readonly ramdisk which knows
where in the kernel image the filesystem resides, probably
based on symbols inserted by the linker).
miket
On Mon, Jul 30, 2001 at 04:49:15PM -0400, Alexander Viro wrote:
>
> On Mon, 30 Jul 2001, Mike Touloumtzis wrote:
>
> > One thing that would make embedded systems developers very happy
> > is the ability to map a romfs or cramfs filesystem directly from
> > the kernel image, avoiding the extra copy necessitated by the cpio
> > archive. Are there problems with this approach?
>
> a) IIRC, both are read-only.
Hmmm, maybe we need ramfs-backed-by-romfs :-). But a lot of people
in the embedded/consumer electronics space could get by just fine
with a read-only / and a ramfs or ramdisk on /tmp.
> b) what stops you from doing initramfs + romfs-on-initrd? It works.
-- Having the FS image compiled into the kernel allows a linker script
to specify the alignment requirements for the FS.
-- initrd is a yucky special case. What I'm advocating is just a
standard way of addressing compiled-in filesystems using the normal
/dev namespace.
miket
>>>>> "Mike" == Mike Touloumtzis <[email protected]> writes:
[snip]
Mike> Hmmm, maybe we need ramfs-backed-by-romfs :-). But a lot of
Mike> people in the embedded/consumer electronics space could get by
Mike> just fine with a read-only / and a ramfs or ramdisk on /tmp.
I am not so sure about this. Typical flash access times are rather
long compared to SDRAM. StrataFlash and other bursting flash are
rather new and require specific CPUs or custom logic to access the
flash in a sequential mode.
In my personal experience, code is usually compressed in flash (or
ROM) and expanded into SDRAM. You can always use an MMU to achieve
the RO effects of flash/ROM. The big win for flash execution is that
the power numbers are typically lower... but since it takes longer to
execute, it washes out to the same. But some paranoid hardware people
don't like `peak drains' on a battery so they might prefer Flash
execution. The flexibility is nice whatever the case; it might become
more of an issue if bursting/sequential flash devices become more
common. I don't know if there is a big push for this though. The cost
of burst flash is still greater than SDRAM afiak.
fwiw,
Bill Pringlemeir.
On Mon, Jul 30, 2001 at 06:16:40PM -0400, Bill Pringlemeir wrote:
> >>>>> "Mike" == Mike Touloumtzis <[email protected]> writes:
> [snip]
> Mike> Hmmm, maybe we need ramfs-backed-by-romfs :-). But a lot of
> Mike> people in the embedded/consumer electronics space could get by
> Mike> just fine with a read-only / and a ramfs or ramdisk on /tmp.
>
> I am not so sure about this. Typical flash access times are rather
> long compared to SDRAM. StrataFlash and other bursting flash are
> rather new and require specific CPUs or custom logic to access the
> flash in a sequential mode.
I was actually thinking about kernels running from RAM (e.g. TFTP
boot or such). The solution I'm talking about would also work
from flash, but as you say, most people are moving away from that.
There are still reasons to run XIP from flash, though: boot speed
is one.
miket
On Mon, 30 Jul 2001, Alexander Viro wrote:
> Stuff that went in userland: choosing and mounting root, unpacking/loading
> initrd, running /linuxrc, handling nfsroot, finding and starting final
> init - basically, everything after do_basic_setup().
Eventually I would like to see firmware uploading in initramfs,
instead of compiling firmware images into the kernel...
Jeff
On Mon, 30 Jul 2001 17:56:25 -0500 (CDT),
Jeff Garzik <[email protected]> wrote:
>Eventually I would like to see firmware uploading in initramfs,
>instead of compiling firmware images into the kernel...
<AOL>Me too</AOL>. Building the firmware images as part of kbuild is
wrong, they should be in separate user space packages. One of my
biggest problems with kbuild 2.5 was supporting the firmware build for
sound and SCSI cards in the kernel, it is all ugly special case code.
On Tue, 31 Jul 2001, Keith Owens wrote:
> On Mon, 30 Jul 2001 17:56:25 -0500 (CDT),
> Jeff Garzik <[email protected]> wrote:
> >Eventually I would like to see firmware uploading in initramfs,
> >instead of compiling firmware images into the kernel...
>
> <AOL>Me too</AOL>. Building the firmware images as part of kbuild is
> wrong, they should be in separate user space packages. One of my
> biggest problems with kbuild 2.5 was supporting the firmware build for
> sound and SCSI cards in the kernel, it is all ugly special case code.
<shrug> Once the late-boot stuff is stable - why not? It's clearly
separate patch and I suspect that the trickiest part will be
to come up with a reasonable way of telling make process what to
put into the archive.
Mike Touloumtzis <[email protected]> writes:
> On Mon, Jul 30, 2001 at 03:50:33PM -0500, Jeff Garzik wrote:
> > On Mon, 30 Jul 2001, Mike Touloumtzis wrote:
> > >
> > > One thing that would make embedded systems developers very happy
> > > is the ability to map a romfs or cramfs filesystem directly from
> > > the kernel image, avoiding the extra copy necessitated by the cpio
> > > archive. Are there problems with this approach?
> >
> > Yes -- you need to at that point store initialized structures. Store
> > the dcache in its unpacked state on the ROM image, etc. That's the only
> > way to "map" a romfs directly. Otherwise there is ALWAYS an unpacking
> > or translation step between filesystem image and in-memory image.
> >
> > Mapping an in-memory image directly may seem like a good idea, but it is
> > really not. ESPECIALLY for embedded folks.
>
> I think you're misunderstanding what I propose. I'm talking about
> having a device in /dev that would allow access to a filesystem
> image (cramfs or romfs) that would be embedded in the in-memory
> kernel image.
The current mtd drivers allow exactly this. Having a filesystem on
your flash or rom device. I don't think any filesystem that runs on
top of them currently supports XIP but the basic infrastructure is
there.
Eric