LinuxLists.cc - [PATCH] MODULE_FIRMWARE for binary firmware(s)

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Mon, Aug 28, 2006 at 05:11:42PM -0500, James Bottomley wrote:
> This is a reference implementation with the debian mkinitrd-tools
> package. It shows how to identify the firmware files necessary for
> drivers in the initrd and also includes a primitive system for loading
> them.
>
> I've tested this with the aic94xx driver using the new MODULE_FIRMWARE()
> tag. Initramfs should be much easier because it already includes most
> of the boot time loading; all it has to do is the piece identifying the
> firmware for the selected modules.

Notice that mkinitrd-tools is dead, and will probably be removed from etch.

mkinitramfs-tools and yaird are the two currently used tools.

Friendly,

Sven Luther

2006-08-28 23:35:38

by Oleg Verych

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

Sven Luther wrote:
> On Mon, Aug 28, 2006 at 05:11:42PM -0500, James Bottomley wrote:
>
>>I've tested this with the aic94xx driver using the new MODULE_FIRMWARE()
>>tag. Initramfs should be much easier because it already includes most
>>of the boot time loading; all it has to do is the piece identifying the
>>firmware for the selected modules.
...
> Notice that mkinitrd-tools is dead, and will probably be removed from etch.
>
request_firmware() is dead also.
YMMV, but three years, and there are still big chunks of binary in kernel.
And please don't add new useless info _in_ it.

Nobody cares.
While this implementation exists, it wasn't well designed and hard to use.
As with in-kernel bootsplash and i18n, everything maybe done in userspace, only
with little help from the kernel:
<http://permalink.gmane.org/gmane.linux.kernel/435955>.

Thanks.

--
-o--=O`C /. .\ (+)
#oo'L O o |
<___=E M ^-- | (you're barking up the wrong tree)

2006-08-28 23:51:04

by James Bottomley

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Tue, 2006-08-29 at 01:04 +0200, Sven Luther wrote:
> Notice that mkinitrd-tools is dead, and will probably be removed from etch.
>
> mkinitramfs-tools and yaird are the two currently used tools.

Yes ... I'm aware of that. That's why this is a reference
implementation. initramfs should be easier ... I just don't have any
initramfs systems at the moment, so I did what I had and could verify.

James

2006-08-28 23:56:18

by James Bottomley

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Tue, 2006-08-29 at 02:35 +0200, Oleg Verych wrote:
> request_firmware() is dead also.
> YMMV, but three years, and there are still big chunks of binary in kernel.
> And please don't add new useless info _in_ it.

I er don't think so.

We (as in the Kernel) are forcing drivers on to this path. You should
have noticed it already with the qla2xxx. The aic94xx is the first
driver that's likely found as the boot driver for a system, that's why I
get to propose the reference implementation.

James

2006-08-29 01:16:00

by Oleg Verych

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

James Bottomley wrote:
> On Tue, 2006-08-29 at 02:35 +0200, Oleg Verych wrote:
>
>>request_firmware() is dead also.
>>YMMV, but three years, and there are still big chunks of binary in kernel.
>>And please don't add new useless info _in_ it.
>
>
> I er don't think so.
>
Hell, what can be as easy as this:
,-
|modprobe $drv
|(dd </lib/firmware/$drv.bin>/dev/blobs && echo OK) || echo Error
`-
where /dev/blobs is similar to /dev/port or even /dev/null char device.
if synchronization is needed, add `echo $drv >/dev/blobs`, remove modprobe,

>
> James
>
?

--
--
-o--=O`C /. .\ (+)
#oo'L O o |
<___=E M ^-- | (you're barking up the wrong tree)

2006-08-29 02:53:12

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Tue, Aug 29, 2006 at 05:14:30AM +0200, Oleg Verych wrote:
> On Mon, Aug 28, 2006 at 06:51:03PM -0700, Greg KH wrote:
> > On Tue, Aug 29, 2006 at 04:15:49AM +0200, Oleg Verych wrote:
> > > >>request_firmware() is dead also.
> > > >>YMMV, but three years, and there are still big chunks of binary in kernel.
> > > >>And please don't add new useless info _in_ it.
> > > Hell, what can be as easy as this:
> > > ,-
> > > |modprobe $drv
> > > |(dd </lib/firmware/$drv.bin>/dev/blobs && echo OK) || echo Error
> > > `-
> > > where /dev/blobs is similar to /dev/port or even /dev/null char device.
> > > if synchronization is needed, add `echo $drv >/dev/blobs`, remove modprobe,
> >
> > I don't see such code in the kernel at this time. So it's not a
> > solution, sorry.
> >
> I know. return -ENOPATCH

Yes, and that's the only way to make changes in the kernel, sorry.

> I'm nether a CS nor software engineer, just wondering why simple thing isn't
> simple _in_ the Kernel. I'm reading list "just for fun (C)" and any good word
> about this (IMHO) unix-way design *may* lead professional programmers to do
> tiny worthy things (think about kevent discussion).
> If it's (i'm) stupid, please, say so (in the way Nicholas Miell did ;).

I don't think it's workable, and goes against the current way the kernel
does things. But please, feel free to prove me wrong with a patch
otherwise. I don't want to debate it otherwise.

I think the current way we handle firmware works quite well, especially
given the wide range of different devices that it works for (everything
from BIOS upgrades to different wireless driver stages).

thanks,

greg k-h

2006-08-29 03:19:39

by Oleg Verych

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

Greg KH wrote:
> On Tue, Aug 29, 2006 at 05:14:30AM +0200, Oleg Verych wrote:
>
>>On Mon, Aug 28, 2006 at 06:51:03PM -0700, Greg KH wrote:
>>
>>>On Tue, Aug 29, 2006 at 04:15:49AM +0200, Oleg Verych wrote:
>>>
>>>>>>request_firmware() is dead also.
>>>>>>YMMV, but three years, and there are still big chunks of binary in kernel.
>>>>>>And please don't add new useless info _in_ it.
>>>>
>>>>Hell, what can be as easy as this:
>>>>,-
>>>>|modprobe $drv
>>>>|(dd </lib/firmware/$drv.bin>/dev/blobs && echo OK) || echo Error
>>>>`-
>>>>where /dev/blobs is similar to /dev/port or even /dev/null char device.
>>>>if synchronization is needed, add `echo $drv >/dev/blobs`, remove modprobe,
>>>
...
>
>>I'm nether a CS nor software engineer, just wondering why simple thing isn't
>>simple _in_ the Kernel. I'm reading list "just for fun (C)" and any good word
>>about this (IMHO) unix-way design *may* lead professional programmers to do
>>tiny worthy things (think about kevent discussion).
>>If it's (i'm) stupid, please, say so (in the way Nicholas Miell did ;).
>
> I don't think it's workable, and goes against the current way the kernel
> does things. But please, feel free to prove me wrong with a patch
> otherwise. I don't want to debate it otherwise.
>
Thanks, and OK, this is my last reply on this.

> I think the current way we handle firmware works quite well, especially
> given the wide range of different devices that it works for (everything
> from BIOS upgrades to different wireless driver stages).
>
Oh, come on, even skilled developers (not particular kernel's) having
difficulties with current hotplug-sysfs-modprobe thing;
in this case one couldn't easily figure out problems and way to solve them
<http://permalink.gmane.org/gmane.comp.hardware.texas-instruments.msp430.gcc.user/5444>

Goodbye.

--
-o--=O`C /. .\ (+)
#oo'L O o |
<___=E M ^-- | (you're barking up the wrong tree)

2006-08-29 11:15:52

by Marcel Holtmann

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

Hi James,

> From: Jon Masters <[email protected]>
>
> Right now, various kernel modules are being migrated over to use
> request_firmware in order to pull in binary firmware blobs from userland
> when the module is loaded. This makes sense.
>
> However, there is right now little mechanism in place to automatically
> determine which binary firmware blobs must be included with a kernel in
> order to satisfy the prerequisites of these drivers. This affects
> vendors, but also regular users to a certain extent too.
>
> The attached patch introduces MODULE_FIRMWARE as a mechanism for
> advertising that a particular firmware file is to be loaded - it will
> then show up via modinfo and could be used e.g. when packaging a kernel.

this patch was debated, but we never came to a final conclusion. I am
all for it. It is simple and can improve the current situation.

> Signed-off-by: Jon Masters <[email protected]>

Signed-off-by: Marcel Holtmann <[email protected]>

Regards

Marcel

2006-08-29 15:52:25

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Mon, 28 Aug 2006, Greg KH wrote:

> I think the current way we handle firmware works quite well, especially
> given the wide range of different devices that it works for (everything
> from BIOS upgrades to different wireless driver stages).

the current system works for many people yes, but not everyone.

I'm still waiting to find a way to get the iw2200 working without having to use
modules.

there was a post a month or two ago from someone who had indicated they found a
way to re-initialize it after the system came up, but after someone asked for
details of how to do it there was no response.

David Lang

2006-08-29 16:31:04

by Michael Büsch

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Tuesday 29 August 2006 04:15, Oleg Verych wrote:
> James Bottomley wrote:
> > On Tue, 2006-08-29 at 02:35 +0200, Oleg Verych wrote:
> >
> >>request_firmware() is dead also.
> >>YMMV, but three years, and there are still big chunks of binary in kernel.
> >>And please don't add new useless info _in_ it.
> >
> >
> > I er don't think so.
> >
> Hell, what can be as easy as this:
> ,-
> |modprobe $drv
> |(dd </lib/firmware/$drv.bin>/dev/blobs && echo OK) || echo Error
> `-
> where /dev/blobs is similar to /dev/port or even /dev/null char device.
> if synchronization is needed, add `echo $drv >/dev/blobs`, remove modprobe,

Please tell me how this is going to work, if we don't
know which firmware (version) is needed before be actually
initialize the device?

--
Greetings Michael.

2006-08-29 18:33:25

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Tue, Aug 29, 2006 at 08:46:45AM -0700, David Lang wrote:
> On Mon, 28 Aug 2006, Greg KH wrote:
>
> >I think the current way we handle firmware works quite well, especially
> >given the wide range of different devices that it works for (everything
> >from BIOS upgrades to different wireless driver stages).
>
> the current system works for many people yes, but not everyone.
>
> I'm still waiting to find a way to get the iw2200 working without having to
> use modules.

Sounds like a bug you need to pester the iw2200 developers about then.
I don't think it has much to do with the firmware subsystem though :)

thanks,

greg k-h

2006-08-29 19:05:54

by Michael Büsch

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Tuesday 29 August 2006 20:32, Greg KH wrote:
> On Tue, Aug 29, 2006 at 08:46:45AM -0700, David Lang wrote:
> > On Mon, 28 Aug 2006, Greg KH wrote:
> >
> > >I think the current way we handle firmware works quite well, especially
> > >given the wide range of different devices that it works for (everything
> > >from BIOS upgrades to different wireless driver stages).
> >
> > the current system works for many people yes, but not everyone.
> >
> > I'm still waiting to find a way to get the iw2200 working without having to
> > use modules.
>
> Sounds like a bug you need to pester the iw2200 developers about then.
> I don't think it has much to do with the firmware subsystem though :)

Well, yes and no.
The ipw needs the firmware on insmod time (in contrast to bcm43xx
for example, which needs it on ifconfig up time).
So ipw needs to call request_firmware at insmod time. In case of
built-in, that is when the initcall happens. No userland is available
and request_firmware can not call the userspace helpers to upload
the firmware to sysfs.
Well, not really easy to find a sane solution for this. :)

--
Greetings Michael.

2006-08-29 20:14:31

by Olaf Hering

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Tue, Aug 29, Michael Buesch wrote:

> On Tuesday 29 August 2006 20:32, Greg KH wrote:
> > On Tue, Aug 29, 2006 at 08:46:45AM -0700, David Lang wrote:
> > > On Mon, 28 Aug 2006, Greg KH wrote:
> > >
> > > >I think the current way we handle firmware works quite well, especially
> > > >given the wide range of different devices that it works for (everything
> > > >from BIOS upgrades to different wireless driver stages).
> > >
> > > the current system works for many people yes, but not everyone.
> > >
> > > I'm still waiting to find a way to get the iw2200 working without having to
> > > use modules.
> >
> > Sounds like a bug you need to pester the iw2200 developers about then.
> > I don't think it has much to do with the firmware subsystem though :)
>
> Well, yes and no.
> The ipw needs the firmware on insmod time (in contrast to bcm43xx
> for example, which needs it on ifconfig up time).
> So ipw needs to call request_firmware at insmod time. In case of
> built-in, that is when the initcall happens. No userland is available
> and request_firmware can not call the userspace helpers to upload
> the firmware to sysfs.

I dont use nor do I have access ipw hardware, but:
If it is an initcall, the initramfs should be usable at that time.
A creative CONFIG_INITRAMFS_SOURCE= will help, add the firmware and a
small helper that does the cat(1).

2006-08-29 20:17:39

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Tue, Aug 29, 2006 at 03:13:34PM +0200, Marcel Holtmann wrote:
> Hi James,
>
> > From: Jon Masters <[email protected]>
> >
> > Right now, various kernel modules are being migrated over to use
> > request_firmware in order to pull in binary firmware blobs from userland
> > when the module is loaded. This makes sense.
> >
> > However, there is right now little mechanism in place to automatically
> > determine which binary firmware blobs must be included with a kernel in
> > order to satisfy the prerequisites of these drivers. This affects
> > vendors, but also regular users to a certain extent too.
> >
> > The attached patch introduces MODULE_FIRMWARE as a mechanism for
> > advertising that a particular firmware file is to be loaded - it will
> > then show up via modinfo and could be used e.g. when packaging a kernel.
>
> this patch was debated, but we never came to a final conclusion. I am
> all for it. It is simple and can improve the current situation.
>
> > Signed-off-by: Jon Masters <[email protected]>
>
> Signed-off-by: Marcel Holtmann <[email protected]>

Since your tree will depend on this change, you can add it there and
add:

Signed-off-by: Greg Kroah-Hartman <[email protected]>

from me to the patch.

thanks,

greg k-h

2006-08-29 20:49:07

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Tue, 29 Aug 2006, Olaf Hering wrote:

> On Tue, Aug 29, Michael Buesch wrote:
>
>> On Tuesday 29 August 2006 20:32, Greg KH wrote:
>>> On Tue, Aug 29, 2006 at 08:46:45AM -0700, David Lang wrote:
>>>> On Mon, 28 Aug 2006, Greg KH wrote:
>>>>
>>>>> I think the current way we handle firmware works quite well, especially
>>>>> given the wide range of different devices that it works for (everything
>>>>> from BIOS upgrades to different wireless driver stages).
>>>>
>>>> the current system works for many people yes, but not everyone.
>>>>
>>>> I'm still waiting to find a way to get the iw2200 working without having to
>>>> use modules.
>>>
>>> Sounds like a bug you need to pester the iw2200 developers about then.
>>> I don't think it has much to do with the firmware subsystem though :)
>>
>> Well, yes and no.
>> The ipw needs the firmware on insmod time (in contrast to bcm43xx
>> for example, which needs it on ifconfig up time).
>> So ipw needs to call request_firmware at insmod time. In case of
>> built-in, that is when the initcall happens. No userland is available
>> and request_firmware can not call the userspace helpers to upload
>> the firmware to sysfs.
>
> I dont use nor do I have access ipw hardware, but:
> If it is an initcall, the initramfs should be usable at that time.
> A creative CONFIG_INITRAMFS_SOURCE= will help, add the firmware and a
> small helper that does the cat(1).
>

you are assuming that

1. modules are enabled and ipw2200 is compiled as a module

2. initrd or initramfs are in use

besides, several kernel versions ago this used to work. the current requirement
is a regression as far as the user is concerned.

David Lang

2006-08-29 20:54:21

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Tue, Aug 29, 2006 at 01:42:56PM -0700, David Lang wrote:
>
> besides, several kernel versions ago this used to work. the current
> requirement is a regression as far as the user is concerned.

Then go bug the driver authors please :)

thanks,

greg k-h

2006-08-29 22:17:20

by Bodo Eggert

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

Michael Buesch <[email protected]> wrote:

> The ipw needs the firmware on insmod time (in contrast to bcm43xx
> for example, which needs it on ifconfig up time).
> So ipw needs to call request_firmware at insmod time. In case of
> built-in, that is when the initcall happens. No userland is available
> and request_firmware can not call the userspace helpers to upload
> the firmware to sysfs.
> Well, not really easy to find a sane solution for this. :)

Can you trigger loading the firmware from userspace?
echo 1 > /sys/module/iw2200/parameters/load_firmware_now
--
Ich danke GMX daf?r, die Verwendung meiner Adressen mittels per SPF
verbreiteten L?gen zu sabotieren.

http://david.woodhou.se/why-not-spf.html

2006-08-30 05:45:23

by Olaf Hering

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Tue, Aug 29, David Lang wrote:

> you are assuming that
>
> 1. modules are enabled and ipw2200 is compiled as a module

No, why?

> 2. initrd or initramfs are in use

initramfs is always in use.

2006-08-30 11:53:01

by Marcel Holtmann

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

Hi Greg,

> > > Right now, various kernel modules are being migrated over to use
> > > request_firmware in order to pull in binary firmware blobs from userland
> > > when the module is loaded. This makes sense.
> > >
> > > However, there is right now little mechanism in place to automatically
> > > determine which binary firmware blobs must be included with a kernel in
> > > order to satisfy the prerequisites of these drivers. This affects
> > > vendors, but also regular users to a certain extent too.
> > >
> > > The attached patch introduces MODULE_FIRMWARE as a mechanism for
> > > advertising that a particular firmware file is to be loaded - it will
> > > then show up via modinfo and could be used e.g. when packaging a kernel.
> >
> > this patch was debated, but we never came to a final conclusion. I am
> > all for it. It is simple and can improve the current situation.
> >
> > > Signed-off-by: Jon Masters <[email protected]>
> >
> > Signed-off-by: Marcel Holtmann <[email protected]>
>
> Since your tree will depend on this change, you can add it there and
> add:
>
> Signed-off-by: Greg Kroah-Hartman <[email protected]>
>
> from me to the patch.

do we consider this 2.6.18 or 2.6.19 material? For the Bluetooth
subsystem I can easily write a patch that makes use of it.

Regards

Marcel

2006-08-30 17:58:39

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Wed, 30 Aug 2006, Olaf Hering wrote:

>> you are assuming that
>>
>> 1. modules are enabled and ipw2200 is compiled as a module
>
> No, why?

becouse if the ipw isn't compiled as a module then it's initialized (without
firmware) before the initramfs or initrd is run. if it could be initialized
later without being a module then it could be initialized as part of the normal
system

>> 2. initrd or initramfs are in use
>
> initramfs is always in use.

not on my machines.

what does it use for the initramfs?

I don't enable the options in the kernel for initrd, and I don't give it any
source for an initramfs.

David Lang

P.S. there was a suggestion yesterday in this thread that I haven't tested yet.
I plan to test that tonight. if it works then the card can be reinitialized
after the system boots, still no initrd or initramfs needed.

2006-08-30 18:15:19

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Wed, Aug 30, 2006 at 10:52:02AM -0700, David Lang wrote:
> On Wed, 30 Aug 2006, Olaf Hering wrote:
>
> >>you are assuming that
> >>
> >>1. modules are enabled and ipw2200 is compiled as a module
> >
> >No, why?
>
> becouse if the ipw isn't compiled as a module then it's initialized
> (without firmware) before the initramfs or initrd is run. if it could be
> initialized later without being a module then it could be initialized as
> part of the normal system

Euh, ...

I wonder why you need to initialize the ipw2200 module so early ? It was my
understanding that the initramfs thingy was run very early in the process, and
i had even thought of moving fbdev modules into it.

Do you really need to bring up ipw2200 so early ? It is some kind of wireless
device, right ?

As for initramfs, you can just cat it behind the kernel, and it should work
just fine, or at least this is how it was supposed to work.

Friendly,

Sven Luther

2006-08-30 18:27:00

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Wed, 30 Aug 2006, Sven Luther wrote:

> On Wed, Aug 30, 2006 at 10:52:02AM -0700, David Lang wrote:
>> On Wed, 30 Aug 2006, Olaf Hering wrote:
>>
>>>> you are assuming that
>>>>
>>>> 1. modules are enabled and ipw2200 is compiled as a module
>>>
>>> No, why?
>>
>> becouse if the ipw isn't compiled as a module then it's initialized
>> (without firmware) before the initramfs or initrd is run. if it could be
>> initialized later without being a module then it could be initialized as
>> part of the normal system
>
> Euh, ...
>
> I wonder why you need to initialize the ipw2200 module so early ? It was my
> understanding that the initramfs thingy was run very early in the process, and
> i had even thought of moving fbdev modules into it.
>
> Do you really need to bring up ipw2200 so early ? It is some kind of wireless
> device, right ?

if modules are not in use the device is initialized when the kernel starts up.
this is before any userspace starts.

> As for initramfs, you can just cat it behind the kernel, and it should work
> just fine, or at least this is how it was supposed to work.

yes, if I want to set one up.

other then this new requirement to have the ipw2200 driver as a module I have no
reason to use one. normal userspace is good enough for me.

David Lang

> Friendly,
>
> Sven Luther
>

2006-08-30 19:17:53

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Wed, Aug 30, 2006 at 11:20:53AM -0700, David Lang wrote:
> On Wed, 30 Aug 2006, Sven Luther wrote:
>
> >On Wed, Aug 30, 2006 at 10:52:02AM -0700, David Lang wrote:
> >>On Wed, 30 Aug 2006, Olaf Hering wrote:
> >>
> >>>>you are assuming that
> >>>>
> >>>>1. modules are enabled and ipw2200 is compiled as a module
> >>>
> >>>No, why?
> >>
> >>becouse if the ipw isn't compiled as a module then it's initialized
> >>(without firmware) before the initramfs or initrd is run. if it could be
> >>initialized later without being a module then it could be initialized as
> >>part of the normal system
> >
> >Euh, ...
> >
> >I wonder why you need to initialize the ipw2200 module so early ? It was my
> >understanding that the initramfs thingy was run very early in the process,
> >and
> >i had even thought of moving fbdev modules into it.
> >
> >Do you really need to bring up ipw2200 so early ? It is some kind of
> >wireless
> >device, right ?
>
> if modules are not in use the device is initialized when the kernel starts
> up. this is before any userspace starts.

Well. but you could do the initialization at open time too, like the other
case that was mentioned here, no ?

> >As for initramfs, you can just cat it behind the kernel, and it should work
> >just fine, or at least this is how it was supposed to work.
>
> yes, if I want to set one up.
>
> other then this new requirement to have the ipw2200 driver as a module I
> have no reason to use one. normal userspace is good enough for me.

Well, ok.

The real question seems to be if we want to keep the firmware inside the
driver or not.

If we want to remove it, then we put, not the module, but the firmware itself
with some basic userspace to load it on demand in the initramfs, and this is
reason enough to create an initramfs. The fact that the builtin device is
initialized before the initramfs is loaded seems like a bug to me, since the
idea of the initramfs (well, one of them at least), was to initialize it early
enough for this kind of things.

If on the other side, it is more important to not have an initramfs (because
of security issues, or bootloader constraints or what not), then sure, there
is not much choice than putting the firmware in the driver or in the kernel
directly.

But again, the initramfs is just a memory space available at the end of the
kernel, and there is no hardware-related constraint to access it which are in
any way different from having the firmware linked in together with the kernel,
so it is only a matter of organisation of code, as well as taking a decision
on the above, and to act accordyingly.

Does using an initramfs really have some negative side, security related ?
Would some kind of signed or encrypted initramfs be preferable there ?

Friendly,

Sven Luther

2006-08-30 19:40:09

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Wed, 30 Aug 2006, Sven Luther wrote:

>>>
>>> Do you really need to bring up ipw2200 so early ? It is some kind of
>>> wireless
>>> device, right ?
>>
>> if modules are not in use the device is initialized when the kernel starts
>> up. this is before any userspace starts.
>
> Well. but you could do the initialization at open time too, like the other
> case that was mentioned here, no ?

no, at least not in the current kernel. as was mentioned earlier in this thread
the ipw2200 needs the firmware at initialization, but some others don't need it
until open. I don't know if it's even possible to re-write the driver to do
this.

>>> As for initramfs, you can just cat it behind the kernel, and it should work
>>> just fine, or at least this is how it was supposed to work.
>>
>> yes, if I want to set one up.
>>
>> other then this new requirement to have the ipw2200 driver as a module I
>> have no reason to use one. normal userspace is good enough for me.
>
> Well, ok.
>
> The real question seems to be if we want to keep the firmware inside the
> driver or not.
>
> If we want to remove it, then we put, not the module, but the firmware itself
> with some basic userspace to load it on demand in the initramfs, and this is
> reason enough to create an initramfs. The fact that the builtin device is
> initialized before the initramfs is loaded seems like a bug to me, since the
> idea of the initramfs (well, one of them at least), was to initialize it early
> enough for this kind of things.

this isn't my understanding.

my understanding is that the kernel fully initializes all built-in drivers, then
loads userspace and starts running it.

that userspace can be on a device that it knows how to read, or it can be
userspace on initramfs so that you can load additional modules to give you
access to the hardware that you want to run on.

this is needed if your root drive is a SCSI drive and you have it's driver
compiled as a module for example.

this is needed if your root drive uses dm and you need to initialize the array
(one advantage of md, from the user standpoint, is that it doesn't require this
additional layer before use)

however this is not soon enough to supply the firmware for devices like this.

> If on the other side, it is more important to not have an initramfs (because
> of security issues, or bootloader constraints or what not), then sure, there
> is not much choice than putting the firmware in the driver or in the kernel
> directly.
>
> But again, the initramfs is just a memory space available at the end of the
> kernel, and there is no hardware-related constraint to access it which are in
> any way different from having the firmware linked in together with the kernel,
> so it is only a matter of organisation of code, as well as taking a decision
> on the above, and to act accordyingly.

if the firmware needed for any drivers compiled in was appended to the kernel
the same way that initramfs is, without requireing the other things needed to
make initrmfs useable I think that would be reasonable (bundling them togeather
as opposed to embedding the firmware in the kernel). it may even be possible to
have the firmware as files in a initramfs that contains nothing else, and the
kernel knows how to read the data directly (without the hotplug firmware request
userspace stuff)

> Does using an initramfs really have some negative side, security related ?
> Would some kind of signed or encrypted initramfs be preferable there ?

adding an initramfs to a system that doesn't need it otherwise adds
complications to the configure and boot process.

requireing modules when they weren't required before adds complication, and
if/when the patch that's floating around to eliminate access to /dev/kmem is
ever accepted, there are security advantages of running a kernel that doesn't
have any support for run-time modifications (i.e. module loading).

I realize that many people want to make initramfs mandatory (with things like
partition detection moved to userspace), but unless there is a standard
initramfs that is shipped and maintained with the kernel to implement things
like this (see the klibc discussion a few weeks ago) you are adding
complications without much of a benifit to the user.

David Lang

2006-08-30 20:59:48

[permalink] [raw]

Subject: Re: [PATCH] MODULE_FIRMWARE for binary firmware(s)

On Wed, Aug 30, 2006 at 12:34:16PM -0700, David Lang wrote:
> On Wed, 30 Aug 2006, Sven Luther wrote:
>
> >>>
> >>>Do you really need to bring up ipw2200 so early ? It is some kind of
> >>>wireless
> >>>device, right ?
> >>
> >>if modules are not in use the device is initialized when the kernel starts
> >>up. this is before any userspace starts.
> >
> >Well. but you could do the initialization at open time too, like the other
> >case that was mentioned here, no ?
>
> no, at least not in the current kernel. as was mentioned earlier in this
> thread the ipw2200 needs the firmware at initialization, but some others
> don't need it until open. I don't know if it's even possible to re-write
> the driver to do this.

Oh well, this doesn't explain why it is so, but i suppose you know what you
speak about.

> >>>As for initramfs, you can just cat it behind the kernel, and it should
> >>>work
> >>>just fine, or at least this is how it was supposed to work.
> >>
> >>yes, if I want to set one up.
> >>
> >>other then this new requirement to have the ipw2200 driver as a module I
> >>have no reason to use one. normal userspace is good enough for me.
> >
> >Well, ok.
> >
> >The real question seems to be if we want to keep the firmware inside the
> >driver or not.
> >
> >If we want to remove it, then we put, not the module, but the firmware
> >itself
> >with some basic userspace to load it on demand in the initramfs, and this
> >is
> >reason enough to create an initramfs. The fact that the builtin device is
> >initialized before the initramfs is loaded seems like a bug to me, since
> >the
> >idea of the initramfs (well, one of them at least), was to initialize it
> >early
> >enough for this kind of things.
>
> this isn't my understanding.

Indeed, in the initrd era, the ramdisk was initialized too late for this kind
of stuff, but it was one of the features of the initramfs ramdisks to
initialize it earlier, which made firmware loading possible.

> my understanding is that the kernel fully initializes all built-in drivers,
> then loads userspace and starts running it.

Well, there is userspace and userspace.

> that userspace can be on a device that it knows how to read, or it can be
> userspace on initramfs so that you can load additional modules to give you
> access to the hardware that you want to run on.

Yep, but initramfs is initialized ways earlier than normal userspace.

> this is needed if your root drive is a SCSI drive and you have it's driver
> compiled as a module for example.

Ok.

> this is needed if your root drive uses dm and you need to initialize the
> array (one advantage of md, from the user standpoint, is that it doesn't
> require this additional layer before use)

Ok.

> however this is not soon enough to supply the firmware for devices like
> this.

Are you sure of this ? This is somewhat contrary to what i have heard, and it
sure would make sense to be able to access the initramfs ramdisk much earlier.

> >If on the other side, it is more important to not have an initramfs
> >(because
> >of security issues, or bootloader constraints or what not), then sure,
> >there
> >is not much choice than putting the firmware in the driver or in the kernel
> >directly.
> >
> >But again, the initramfs is just a memory space available at the end of the
> >kernel, and there is no hardware-related constraint to access it which are
> >in
> >any way different from having the firmware linked in together with the
> >kernel,
> >so it is only a matter of organisation of code, as well as taking a
> >decision
> >on the above, and to act accordyingly.
>
> if the firmware needed for any drivers compiled in was appended to the
> kernel the same way that initramfs is, without requireing the other things
> needed to make initrmfs useable I think that would be reasonable (bundling
> them togeather as opposed to embedding the firmware in the kernel). it may
> even be possible to have the firmware as files in a initramfs that contains
> nothing else, and the kernel knows how to read the data directly (without
> the hotplug firmware request userspace stuff)

Indeed, and it seems to me that exactly this kind of use was indeed considered
when the initramfs infrastructure was designed. Not sure about the latest bit
concerning hotplug though.

> >Does using an initramfs really have some negative side, security related ?
> >Would some kind of signed or encrypted initramfs be preferable there ?
>
> adding an initramfs to a system that doesn't need it otherwise adds
> complications to the configure and boot process.

Yeah, well, if it is just cated at the end of the kernel, it would be rather
minimal.

> requireing modules when they weren't required before adds complication, and
> if/when the patch that's floating around to eliminate access to /dev/kmem
> is ever accepted, there are security advantages of running a kernel that
> doesn't have any support for run-time modifications (i.e. module loading).

Indeed.

> I realize that many people want to make initramfs mandatory (with things
> like partition detection moved to userspace), but unless there is a
> standard initramfs that is shipped and maintained with the kernel to
> implement things like this (see the klibc discussion a few weeks ago) you
> are adding complications without much of a benifit to the user.

Well, since debian moved to 2.6 kernels on powerpc, i have been using ramdisks,
and let me assure you that from a distribution point of view, it is way more
elegant and nice than the non-ramdisk solution.

Friendly,

Sven Luther

2006-08-30 21:18:24