2001-07-03 05:14:11

by Jeff Garzik

[permalink] [raw]
Subject: RFC: modules and 2.5

A couple things that would be nice for 2.5 is
- let MOD_INC_USE_COUNT work even when module is built into kernel, and
- let THIS_MODULE exist and be valid even when module is built into
kernel

This introduces bloat into the static kernel for modules which do not
take advantage of this, so perhaps we can make this new behavior
conditional on CONFIG_xxx option. Individual drivers which make use of
the behavior can do something like

dep_tristate 'my driver' CONFIG_MYDRIVER $CONFIG_PCI
if [ "$CONFIG_MYDRIVER" != "n" -a \
"$CONFIG_STATIC_MODULES" != "y" ]; then
define_bool CONFIG_STATIC_MODULES y
fi



The reasoning behind this is that module use counts are useful sometimes
even when the driver is built into the kernel. Other facilities like
inter_xxx are [obviously] useful when built into the kernel, so it makes
sense to at least optionally support homogenous module treatment across
static or modular builds.

--
Jeff Garzik | "I respect faith, but doubt is
Building 1024 | what gives you an education."
MandrakeSoft | -- Wilson Mizner


2001-07-03 06:57:21

by Sean Hunter

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

Does this defeat my favourite module-related gothcha, that the machine panics
if I have (say) a scsi driver builtin to the kernel and the same driver tries
to load itself as a module?

This normally happens when switching to a custom kernel after a fresh distro
install. RedHat (and others, I think) use an initial ramdisk to make sure that
all the modules needed to mount the root fs get loaded at boot time.

If you build the drivers in, but forget to comment out the initrd line in
/etc/lilo.conf, the machine panics because it tries to load the module for
something that is already a builtin.

Make sense?

Sean

On Tue, Jul 03, 2001 at 01:13:45AM -0400, Jeff Garzik wrote:
> A couple things that would be nice for 2.5 is
> - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> - let THIS_MODULE exist and be valid even when module is built into
> kernel
>
> This introduces bloat into the static kernel for modules which do not
> take advantage of this, so perhaps we can make this new behavior
> conditional on CONFIG_xxx option. Individual drivers which make use of
> the behavior can do something like
>
> dep_tristate 'my driver' CONFIG_MYDRIVER $CONFIG_PCI
> if [ "$CONFIG_MYDRIVER" != "n" -a \
> "$CONFIG_STATIC_MODULES" != "y" ]; then
> define_bool CONFIG_STATIC_MODULES y
> fi
>
>
>
> The reasoning behind this is that module use counts are useful sometimes
> even when the driver is built into the kernel. Other facilities like
> inter_xxx are [obviously] useful when built into the kernel, so it makes
> sense to at least optionally support homogenous module treatment across
> static or modular builds.
>
> --
> Jeff Garzik | "I respect faith, but doubt is
> Building 1024 | what gives you an education."
> MandrakeSoft | -- Wilson Mizner
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

2001-07-03 07:05:44

by Jeff Garzik

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

Sean Hunter wrote:
>
> Does this defeat my favourite module-related gothcha, that the machine panics
> if I have (say) a scsi driver builtin to the kernel and the same driver tries
> to load itself as a module?

Other, existing mechanisms should prevent conflicts here. If a builtin
SCSI driver loads successfully, then its calls to request_region,
request_mem_region, or pci_request_regions should have succeeded.

If so, any attempt to grab those I/O regions by another driver,
including another instance of the same driver, should fail.

--
Jeff Garzik | "I respect faith, but doubt is
Building 1024 | what gives you an education."
MandrakeSoft | -- Wilson Mizner

2001-07-03 07:17:24

by Keith Owens

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

On Tue, 3 Jul 2001 07:50:50 +0100,
Sean Hunter <[email protected]> wrote:
>Does this defeat my favourite module-related gothcha, that the machine panics
>if I have (say) a scsi driver builtin to the kernel and the same driver tries
>to load itself as a module?

No, but other wip for 2.5 will. My 2.5 makefile rewrite already
defines for each object, the module it would be linked into if it were
a module. That gives me a list of all the "modules" already in the
kernel which is what is required to prevent duplicate loads.

2001-07-03 07:18:54

by Keith Owens

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

On Tue, 03 Jul 2001 01:13:45 -0400,
Jeff Garzik <[email protected]> wrote:
>A couple things that would be nice for 2.5 is
>- let MOD_INC_USE_COUNT work even when module is built into kernel, and
>- let THIS_MODULE exist and be valid even when module is built into
>kernel

There is already a list of module related changes that would be nice
for 2.5. I have added this one to the list, but don't expect it until
September at least, after I get 2.5 module versions working.

2001-07-03 07:24:54

by Jeff Garzik

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

Keith Owens wrote:
>
> On Tue, 3 Jul 2001 07:50:50 +0100,
> Sean Hunter <[email protected]> wrote:
> >Does this defeat my favourite module-related gothcha, that the machine panics
> >if I have (say) a scsi driver builtin to the kernel and the same driver tries
> >to load itself as a module?
>
> No, but other wip for 2.5 will.

If this occurs in 2.4 it is a bug and should be fixed in 2.4.

--
Jeff Garzik | "I respect faith, but doubt is
Building 1024 | what gives you an education."
MandrakeSoft | -- Wilson Mizner

2001-07-03 07:39:28

by Keith Owens

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

On Tue, 03 Jul 2001 03:24:26 -0400,
Jeff Garzik <[email protected]> wrote:
>> On Tue, 3 Jul 2001 07:50:50 +0100,
>> Sean Hunter <[email protected]> wrote:
>> >Does this defeat my favourite module-related gothcha, that the machine panics
>> >if I have (say) a scsi driver builtin to the kernel and the same driver tries
>> >to load itself as a module?
>
>If this occurs in 2.4 it is a bug and should be fixed in 2.4.

Human error. Create a new kernel with something built in which used to
be a module and forget to make modules_install, so the code is in the
kernel and in /lib/modules. Then do an explicit insmod, if probing
does not fail the module load then oops is all she wrote. One of the
reasons I changed modules_install to erase the old directory first was
to minimize this problem, but humans can still stuff it up. AFAICT it
cannot be fixed in 2.4 because there is no identifier for which
"modules" are included in the kernel.

2001-07-03 07:45:08

by Jeff Garzik

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

Keith Owens wrote:
> Human error. Create a new kernel with something built in which used to
> be a module and forget to make modules_install, so the code is in the
> kernel and in /lib/modules. Then do an explicit insmod, if probing
> does not fail the module load then oops is all she wrote.

It still sounds like a driver bug. It is the driver's responsibility to
ensure its resources are locked down for its own use.

Probing should not succeed if there is an existing driver in the
kernel. request_??? and register_??? functions return failure to
guarantee this sort of thing.

Someone please provide a list of drivers which behave in this manner, so
that they can be fixed up.

Regards,

Jeff


--
Jeff Garzik | "I respect faith, but doubt is
Building 1024 | what gives you an education."
MandrakeSoft | -- Wilson Mizner

2001-07-03 09:09:55

by Fang Han

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

> If you build the drivers in, but forget to comment out the initrd line in
> /etc/lilo.conf, the machine panics because it tries to load the module for
> something that is already a builtin.
>
The only way to solve it smothly need to modify the bootloader, When the
bootloader like lilo or grub ( it is more powerful ) can read the module from the root partition directly. Your problem will be sloved.

BTW: Is there any system or tools can patch kernel in binary level, It means
that user doesn't need download the whole kernel RPM or TGZ, It just need
an patch to patch the current kernel's binary. I think it is useful for
novice & end user.

Regards

dfbb

2001-07-03 12:11:18

by James Lewis Nance

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

On Tue, Jul 03, 2001 at 01:13:45AM -0400, Jeff Garzik wrote:
> A couple things that would be nice for 2.5 is
> - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> - let THIS_MODULE exist and be valid even when module is built into
> kernel

I have something similar that I have wanted for a long time, and it would
accomplish what you want too. I would like for the .o files for modules and
compiled in drivers to be identical. It seems like this would be better for
testing because it should eleminate module vs non-module bugs. We might
even want them to show up in /proc/modules, perhaps with some mechinism to
keep the reference count from going to 0.

I dont think I would want to unleash it on an end user, but if you omit the
part about letting the reference count go to zero, it should even be possible
to unload a compiled in driver and replace it with a new module. If you
did not load each module into its own section, you would have to leak its
text and data memory, but this still might be useful for development.

Anyway, just some ideas I have been wanting to share for about 5 years.
Thanks for giving me an excuse.

Jim

2001-07-03 17:36:33

by Andrzej Krzysztofowicz

[permalink] [raw]
Subject: Re: RFC: modules and 2.5


>
> A couple things that would be nice for 2.5 is
> - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> - let THIS_MODULE exist and be valid even when module is built into
> kernel
>
> This introduces bloat into the static kernel for modules which do not
> take advantage of this, so perhaps we can make this new behavior
> conditional on CONFIG_xxx option. Individual drivers which make use of
> the behavior can do something like
>
> dep_tristate 'my driver' CONFIG_MYDRIVER $CONFIG_PCI
> if [ "$CONFIG_MYDRIVER" != "n" -a \
^^^^^^^^^^^^^^^^^^^^^^^
> "$CONFIG_STATIC_MODULES" != "y" ]; then
> define_bool CONFIG_STATIC_MODULES y
> fi

Hmmm, shouldn't it be written in CML2 if it is for 2.5 ?

For 2.4 the marked condition ( != n on a variable defined by dep_*)
probably would break xconfig. Don't suggest such solutions...

Andrzej
--
=======================================================================
Andrzej M. Krzysztofowicz [email protected]
tel. (0-58) 347 14 61
Wydz.Fizyki Technicznej i Matematyki Stosowanej Politechniki Gdanskiej

2001-07-03 17:40:03

by Jeff Garzik

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

Andrzej Krzysztofowicz wrote:
>
> >
> > A couple things that would be nice for 2.5 is
> > - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> > - let THIS_MODULE exist and be valid even when module is built into
> > kernel
> >
> > This introduces bloat into the static kernel for modules which do not
> > take advantage of this, so perhaps we can make this new behavior
> > conditional on CONFIG_xxx option. Individual drivers which make use of
> > the behavior can do something like
> >
> > dep_tristate 'my driver' CONFIG_MYDRIVER $CONFIG_PCI
> > if [ "$CONFIG_MYDRIVER" != "n" -a \
> ^^^^^^^^^^^^^^^^^^^^^^^
> > "$CONFIG_STATIC_MODULES" != "y" ]; then
> > define_bool CONFIG_STATIC_MODULES y
> > fi
>
> Hmmm, shouldn't it be written in CML2 if it is for 2.5 ?

no comment


> For 2.4 the marked condition ( != n on a variable defined by dep_*)
> probably would break xconfig. Don't suggest such solutions...

why is != n on a variable defined by dep_xx bad?
That doesn't make sense.

Jeff


--
Jeff Garzik | "I respect faith, but doubt is
Building 1024 | what gives you an education."
MandrakeSoft | -- Wilson Mizner

2001-07-06 23:10:52

by Rusty Russell

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

In message <[email protected]> you write:
> A couple things that would be nice for 2.5 is
> - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> - let THIS_MODULE exist and be valid even when module is built into
> kernel

Hi Jeff,

What use are module use counts, if not used to prevent unloading?

> The reasoning behind this is that module use counts are useful sometimes
> even when the driver is built into the kernel. Other facilities like
> inter_xxx are [obviously] useful when built into the kernel, so it makes

Let's be clear: inter_module_xxx is Broken as Designed. It's a
terrible interface that has the added merit of being badly
implemented.

If you have a module B which has a soft dependency ("must use if
there") on module A, inter_xxx doesn't help without opening a can of
worms (if module B inserted after module A, oops).

The best ways out of this are:
1) Create two versions of module B: an A+B one, and a B-alone one.
2) Place infrastructure in the core kernel.
(This is what I did for ipt_REJECT needing to know about NAT).

Also, I far prefer the simplicity of get_symbol and put_symbol.

Cheers,
Rusty.
--
Premature optmztion is rt of all evl. --DK

2001-07-07 14:12:52

by Jeff Garzik

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

Rusty Russell wrote:
>
> In message <[email protected]> you write:
> > A couple things that would be nice for 2.5 is
> > - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> > - let THIS_MODULE exist and be valid even when module is built into
> > kernel
>
> Hi Jeff,
>
> What use are module use counts, if not used to prevent unloading?

IMHO you should be free to bump the module reference count up and down
as you wish, and be able to read the module reference count.

If you make that assumption, then it becomes possible to use the module
ref count as an internal reference counter, for device opens or
something like that.

In i810_rng.c it eliminated the need for an additional reference
count... until I attempted to compile it into the kernel instead of as a
module, for the first time :)

Jeff


--
Jeff Garzik | A recent study has shown that too much soup
Building 1024 | can cause malaise in laboratory mice.
MandrakeSoft |

2001-07-09 13:05:04

by Rusty Russell

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

In message <[email protected]> you write:
> IMHO you should be free to bump the module reference count up and down
> as you wish, and be able to read the module reference count.
>
> If you make that assumption, then it becomes possible to use the module
> ref count as an internal reference counter, for device opens or
> something like that.

Surely the exception rather than the rule?

Sorry, complicating the code and making everyone pay the penalty so
you can take a confusing short cut in your code is not something we're
going to agree on.

Modules are slower than built in; let's not "fix" this by making
builtin code slower. 8)

Rusty.
--
Premature optmztion is rt of all evl. --DK

2001-07-12 21:04:04

by Pavel Machek

[permalink] [raw]
Subject: Re: RFC: modules and 2.5

Hi!

> > IMHO you should be free to bump the module reference count up and down
> > as you wish, and be able to read the module reference count.
> >
> > If you make that assumption, then it becomes possible to use the module
> > ref count as an internal reference counter, for device opens or
> > something like that.
>
> Surely the exception rather than the rule?
>
> Sorry, complicating the code and making everyone pay the penalty so
> you can take a confusing short cut in your code is not something we're
> going to agree on.

Actually, having uniform interface between kernel and modules is very
nice... And one int per module does not surely hurt, does it?

Perhaps #define NEED_USE_COUNT to do it on per-module basis?
Pavel
--
I'm [email protected]. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at [email protected]