2007-09-26 07:52:25

by Jan Beulich

[permalink] [raw]
Subject: [PATCH] Module use count must be updated as bridges are created/destroyed

Otherwise 'modprobe -r' on a module having a dependency on bridge will
implicitly unload bridge, bringing down all connectivity that was using
bridges.

Signed-off-by: Jan Beulich <[email protected]>

net/bridge/br_if.c | 9 +++++++++
1 file changed, 9 insertions(+)

--- linux-2.6.23-rc8/net/bridge/br_if.c 2007-09-26 09:23:54.000000000 +0200
+++ 2.6.23-rc8-bridge-module-get-put/net/bridge/br_if.c 2007-09-25 14:31:01.000000000 +0200
@@ -276,6 +276,11 @@ int br_add_bridge(const char *name)
if (!dev)
return -ENOMEM;

+ if (!try_module_get(THIS_MODULE)) {
+ free_netdev(dev);
+ return -ENOENT;
+ }
+
rtnl_lock();
if (strchr(dev->name, '%')) {
ret = dev_alloc_name(dev, dev->name);
@@ -294,6 +299,8 @@ int br_add_bridge(const char *name)
unregister_netdevice(dev);
out:
rtnl_unlock();
+ if (ret)
+ module_put(THIS_MODULE);
return ret;
}

@@ -321,6 +328,8 @@ int br_del_bridge(const char *name)
del_br(netdev_priv(dev));

rtnl_unlock();
+ if (ret == 0)
+ module_put(THIS_MODULE);
return ret;
}





2007-09-26 15:37:52

by Stephen Hemminger

[permalink] [raw]
Subject: Re: [PATCH] Module use count must be updated as bridges are created/destroyed

On Wed, 26 Sep 2007 08:53:27 +0100
"Jan Beulich" <[email protected]> wrote:

> Otherwise 'modprobe -r' on a module having a dependency on bridge will
> implicitly unload bridge, bringing down all connectivity that was
> using bridges.
>
> Signed-off-by: Jan Beulich <[email protected]>
>

No, network devices don't do reference counting.
What is the dependency? Where is the source of the module interacting
with the bridge?

2007-09-26 20:51:20

by Oleg Verych

[permalink] [raw]
Subject: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

* Wed, 26 Sep 2007 08:37:05 -0700
* Organization: Linux Foundation
>
> On Wed, 26 Sep 2007 08:53:27 +0100
> "Jan Beulich" <[email protected]> wrote:
>
>> Otherwise 'modprobe -r' on a module having a dependency on bridge will
>> implicitly unload bridge, bringing down all connectivity that was
>> using bridges.
>>
>> Signed-off-by: Jan Beulich <[email protected]>
>>
>
> No, network devices don't do reference counting.

Could you explain why, please?

After `udevd` on boot loads lots of unused crap, i surrendered, and use
$(rmmod `lsmod | just first column`). Networing bravely wipes away. OK,
there are lots of configs: udev, hotplug, modprobe, that somebody might
like to fix. But it came to the end with me. I just don't care. So,
please answer :)
____

2007-09-26 21:06:53

by Stephen Hemminger

[permalink] [raw]
Subject: Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

On Wed, 26 Sep 2007 23:06:53 +0200
Oleg Verych <[email protected]> wrote:

> * Wed, 26 Sep 2007 08:37:05 -0700
> * Organization: Linux Foundation
> >
> > On Wed, 26 Sep 2007 08:53:27 +0100
> > "Jan Beulich" <[email protected]> wrote:
> >
> >> Otherwise 'modprobe -r' on a module having a dependency on bridge will
> >> implicitly unload bridge, bringing down all connectivity that was
> >> using bridges.
> >>
> >> Signed-off-by: Jan Beulich <[email protected]>
> >>
> >
> > No, network devices don't do reference counting.
>
> Could you explain why, please?
>
> After `udevd` on boot loads lots of unused crap, i surrendered, and use
> $(rmmod `lsmod | just first column`). Networing bravely wipes away. OK,
> there are lots of configs: udev, hotplug, modprobe, that somebody might
> like to fix. But it came to the end with me. I just don't care. So,
> please answer :)
> ____

For hotplug and other reasons, the network developers decided that being
able to remove a network module at any time was a good thing. It works.

--
Stephen Hemminger <[email protected]>

2007-09-26 22:19:09

by Jan Engelhardt

[permalink] [raw]
Subject: Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)


On Sep 26 2007 14:06, Stephen Hemminger wrote:
>> >
>> > No, network devices don't do reference counting.
>>
>> Could you explain why, please?
>>
>> After `udevd` on boot loads lots of unused crap, i surrendered, and use
>> $(rmmod `lsmod | just first column`). Networing bravely wipes away. OK,
>> there are lots of configs: udev, hotplug, modprobe, that somebody might
>> like to fix. But it came to the end with me. I just don't care. So,
>> please answer :)
>
>For hotplug and other reasons, the network developers decided that being
>able to remove a network module at any time was a good thing. It works.

Except that for ipv6.ko, it's all opposite. After modprobe,
it already got a refcount like 8 and you're wondering how
to get rid of that.

2007-09-26 22:34:15

by Stephen Hemminger

[permalink] [raw]
Subject: Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

On Thu, 27 Sep 2007 00:18:55 +0200 (CEST)
Jan Engelhardt <[email protected]> wrote:

>
> On Sep 26 2007 14:06, Stephen Hemminger wrote:
> >> >
> >> > No, network devices don't do reference counting.
> >>
> >> Could you explain why, please?
> >>
> >> After `udevd` on boot loads lots of unused crap, i surrendered, and use
> >> $(rmmod `lsmod | just first column`). Networing bravely wipes away. OK,
> >> there are lots of configs: udev, hotplug, modprobe, that somebody might
> >> like to fix. But it came to the end with me. I just don't care. So,
> >> please answer :)
> >
> >For hotplug and other reasons, the network developers decided that being
> >able to remove a network module at any time was a good thing. It works.
>
> Except that for ipv6.ko, it's all opposite. After modprobe,
> it already got a refcount like 8 and you're wondering how
> to get rid of that.

ipv6 is not a network driver, it is a protocol. You might be able to remove it if you zap
all the routes and applications, ...

--
Stephen Hemminger <[email protected]>

2007-09-26 22:57:38

by David Miller

[permalink] [raw]
Subject: Re: why network devices don't do reference counting?

From: Stephen Hemminger <[email protected]>
Date: Wed, 26 Sep 2007 15:33:30 -0700

> ipv6 is not a network driver, it is a protocol. You might be able to
> remove it if you zap all the routes and applications, ...

It is purposefully set to have a permanent elevated reference
count because it is not designed to be unloaded safely.

It has been unloadable forever.

2007-09-27 07:39:17

by Jan Beulich

[permalink] [raw]
Subject: Re: [PATCH] Module use count must be updated as bridges are created/destroyed

>>> Stephen Hemminger <[email protected]> 26.09.07 19:12 >>>
>On Wed, 26 Sep 2007 17:08:19 +0100
>"Jan Beulich" <[email protected]> wrote:
>
>> >>> Stephen Hemminger <[email protected]> 26.09.07 17:37 >>>
>> >On Wed, 26 Sep 2007 08:53:27 +0100
>> >"Jan Beulich" <[email protected]> wrote:
>> >
>> >> Otherwise 'modprobe -r' on a module having a dependency on bridge will
>> >> implicitly unload bridge, bringing down all connectivity that was
>> >> using bridges.
>> >>
>> >> Signed-off-by: Jan Beulich <[email protected]>
>> >>
>> >
>> >No, network devices don't do reference counting.
>> >What is the dependency? Where is the source of the module interacting
>> >with the bridge?
>>
>> On a Xen system, I loaded and then unloaded ebtable_broute. The unload
>> implicitly unloaded bridge, destroying the network. The only way I could see
>> to avoid the implicit unload was to bump the reference count on bridge
>> creation. Otherwise I would have to ask why bridge has a zero reference
>> count despite a bridge being configured.
>>
>> Jan
>
>Sounds like a module utilities problem since unloading one module doesn't
>normally unload others.

I have to disagree here - 'modprobe -r' is specifically unloading all modules the
specified one references as long as they have a use count of zero. The
difference to other networking modules is that the latter normally don't export
symbols, and hence don't have dependent modules (and thus cannot be
subject of implicit unloading). Bridge does have dependents, and hence must
avoid implicit unloading by managing its use count.

Jan

2007-09-27 11:54:48

by Helge Hafting

[permalink] [raw]
Subject: Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

Stephen Hemminger wrote:
> On Thu, 27 Sep 2007 00:18:55 +0200 (CEST)
> Jan Engelhardt <[email protected]> wrote:
>
>
>> On Sep 26 2007 14:06, Stephen Hemminger wrote:
>>
>>>>> No, network devices don't do reference counting.
>>>>>
>>>> Could you explain why, please?
>>>>
>>>> After `udevd` on boot loads lots of unused crap, i surrendered, and use
>>>> $(rmmod `lsmod | just first column`). Networing bravely wipes away. OK,
>>>> there are lots of configs: udev, hotplug, modprobe, that somebody might
>>>> like to fix. But it came to the end with me. I just don't care. So,
>>>> please answer :)
>>>>
>>> For hotplug and other reasons, the network developers decided that being
>>> able to remove a network module at any time was a good thing. It works.
>>>
>> Except that for ipv6.ko, it's all opposite. After modprobe,
>> it already got a refcount like 8 and you're wondering how
>> to get rid of that.
>>
>
> ipv6 is not a network driver, it is a protocol. You might be able to remove it if you zap
> all the routes and applications, ...
>
Wouldn't it be enough to down all the interfaces and close all the sockets?
No need to bring down every app.

Helge Hafting

2007-09-27 14:49:05

by Stephen Hemminger

[permalink] [raw]
Subject: Re: [PATCH] Module use count must be updated as bridges are created/destroyed

On Thu, 27 Sep 2007 08:40:10 +0100
"Jan Beulich" <[email protected]> wrote:

> >>> Stephen Hemminger <[email protected]> 26.09.07 19:12 >>>
> >On Wed, 26 Sep 2007 17:08:19 +0100
> >"Jan Beulich" <[email protected]> wrote:
> >
> >> >>> Stephen Hemminger <[email protected]> 26.09.07 17:37 >>>
> >> >On Wed, 26 Sep 2007 08:53:27 +0100
> >> >"Jan Beulich" <[email protected]> wrote:
> >> >
> >> >> Otherwise 'modprobe -r' on a module having a dependency on bridge will
> >> >> implicitly unload bridge, bringing down all connectivity that was
> >> >> using bridges.
> >> >>
> >> >> Signed-off-by: Jan Beulich <[email protected]>
> >> >>
> >> >
> >> >No, network devices don't do reference counting.
> >> >What is the dependency? Where is the source of the module interacting
> >> >with the bridge?
> >>
> >> On a Xen system, I loaded and then unloaded ebtable_broute. The unload
> >> implicitly unloaded bridge, destroying the network. The only way I could see
> >> to avoid the implicit unload was to bump the reference count on bridge
> >> creation. Otherwise I would have to ask why bridge has a zero reference
> >> count despite a bridge being configured.
> >>
> >> Jan
> >
> >Sounds like a module utilities problem since unloading one module doesn't
> >normally unload others.
>
> I have to disagree here - 'modprobe -r' is specifically unloading all modules the
> specified one references as long as they have a use count of zero. The
> difference to other networking modules is that the latter normally don't export
> symbols, and hence don't have dependent modules (and thus cannot be
> subject of implicit unloading). Bridge does have dependents, and hence must
> avoid implicit unloading by managing its use count.

I want keep the behavior that:
modprobe -r bridge
removes all bridges. It is too useful and may already be in some user scripts.


--
Stephen Hemminger <[email protected]>

2007-09-27 14:51:35

by Stephen Hemminger

[permalink] [raw]
Subject: Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

On Thu, 27 Sep 2007 13:54:23 +0200
Helge Hafting <[email protected]> wrote:

> Stephen Hemminger wrote:
> > On Thu, 27 Sep 2007 00:18:55 +0200 (CEST)
> > Jan Engelhardt <[email protected]> wrote:
> >
> >
> >> On Sep 26 2007 14:06, Stephen Hemminger wrote:
> >>
> >>>>> No, network devices don't do reference counting.
> >>>>>
> >>>> Could you explain why, please?
> >>>>
> >>>> After `udevd` on boot loads lots of unused crap, i surrendered, and use
> >>>> $(rmmod `lsmod | just first column`). Networing bravely wipes away. OK,
> >>>> there are lots of configs: udev, hotplug, modprobe, that somebody might
> >>>> like to fix. But it came to the end with me. I just don't care. So,
> >>>> please answer :)
> >>>>
> >>> For hotplug and other reasons, the network developers decided that being
> >>> able to remove a network module at any time was a good thing. It works.
> >>>
> >> Except that for ipv6.ko, it's all opposite. After modprobe,
> >> it already got a refcount like 8 and you're wondering how
> >> to get rid of that.
> >>
> >
> > ipv6 is not a network driver, it is a protocol. You might be able to remove it if you zap
> > all the routes and applications, ...
> >
> Wouldn't it be enough to down all the interfaces and close all the sockets?
> No need to bring down every app.

You need every socket to close and all routes to go away including the routes through
loopback device, and still there probably are control sockets buried inside ipv6
that hold ref count.

IMHO the kernel should just admit that IPV6 can't be removed.

--- a/net/ipv6/af_inet6.c 2007-09-26 16:28:01.000000000 -0700
+++ b/net/ipv6/af_inet6.c 2007-09-26 17:38:23.000000000 -0700
@@ -914,6 +914,9 @@ out_unregister_tcp_proto:
}
module_init(inet6_init);

+/* Disabled at present because it is impossible to remove all references */
+#ifdef IPV6_UNLOAD
+
static void __exit inet6_exit(void)
{
/* First of all disallow new sockets creation. */
@@ -952,5 +955,6 @@ static void __exit inet6_exit(void)
proto_unregister(&tcpv6_prot);
}
module_exit(inet6_exit);
+#endif

MODULE_ALIAS_NETPROTO(PF_INET6);
--- a/net/ipv6/addrconf.c 2007-09-26 15:07:35.000000000 -0700
+++ b/net/ipv6/addrconf.c 2007-09-26 17:36:52.000000000 -0700
@@ -4255,6 +4255,7 @@ errout:
return err;
}

+#ifdef IPV6_UNLOAD
void __exit addrconf_cleanup(void)
{
struct net_device *dev;
@@ -4308,3 +4309,4 @@ void __exit addrconf_cleanup(void)
proc_net_remove(&init_net, "if_inet6");
#endif
}
+#endif
--- a/net/ipv6/ipv6_sockglue.c 2007-09-26 15:07:35.000000000 -0700
+++ b/net/ipv6/ipv6_sockglue.c 2007-09-26 17:36:17.000000000 -0700
@@ -1132,7 +1132,9 @@ void __init ipv6_packet_init(void)
dev_add_pack(&ipv6_packet_type);
}

-void ipv6_packet_cleanup(void)
+#ifdef IPV6_UNLOAD
+void __exit ipv6_packet_cleanup(void)
{
dev_remove_pack(&ipv6_packet_type);
}
+#endif

2007-09-27 14:55:46

by Jan Engelhardt

[permalink] [raw]
Subject: Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)


On Sep 27 2007 07:51, Stephen Hemminger wrote:
>
>You need every socket to close and all routes to go away including the
>routes through loopback device, and still there probably are control
>sockets buried inside ipv6 that hold ref count.
>
>IMHO the kernel should just admit that IPV6 can't be removed.

I cannot accept that. If ipv6.ko has a way to tack ipv6 structs onto all
sockets, interfaces and addresses, it should also be able to untack it
again.

2007-09-27 15:04:33

by Jan Beulich

[permalink] [raw]
Subject: Re: [PATCH] Module use count must be updated as bridges are created/destroyed

>> >Sounds like a module utilities problem since unloading one module doesn't
>> >normally unload others.
>>
>> I have to disagree here - 'modprobe -r' is specifically unloading all modules the
>> specified one references as long as they have a use count of zero. The
>> difference to other networking modules is that the latter normally don't export
>> symbols, and hence don't have dependent modules (and thus cannot be
>> subject of implicit unloading). Bridge does have dependents, and hence must
>> avoid implicit unloading by managing its use count.
>
>I want keep the behavior that:
> modprobe -r bridge
>removes all bridges. It is too useful and may already be in some user scripts.

So we have an unsolvable problem here then, unless infrastructure gets added
that allows a module to declare itself as not-implicit-unload-safe, forcing
modprobe -r to keep its hands off it. Ugly.

Jan

2007-09-27 19:01:22

by David Miller

[permalink] [raw]
Subject: Re: why network devices don't do reference counting?

From: Helge Hafting <[email protected]>
Date: Thu, 27 Sep 2007 13:54:23 +0200

> Wouldn't it be enough to down all the interfaces and close all the sockets?
> No need to bring down every app.

And there are routes, and neighbour cache entries, and all sorts
of external references to the stack. For example, if a packet
gets stuck in a device because the link just went down, that
can hold references to the ipv6 module from several angles.

But you have to add code to actually keep track of all of these
references and there is no such code in the ipv6 module at all
and it's a nontrivial time consuming job to implement it.

2007-09-27 19:15:31

by David Miller

[permalink] [raw]
Subject: Re: why network devices don't do reference counting?

From: Jan Engelhardt <[email protected]>
Date: Thu, 27 Sep 2007 16:55:36 +0200 (CEST)

>
> On Sep 27 2007 07:51, Stephen Hemminger wrote:
> >
> >You need every socket to close and all routes to go away including the
> >routes through loopback device, and still there probably are control
> >sockets buried inside ipv6 that hold ref count.
> >
> >IMHO the kernel should just admit that IPV6 can't be removed.
>
> I cannot accept that. If ipv6.ko has a way to tack ipv6 structs onto all
> sockets, interfaces and addresses, it should also be able to untack it
> again.

Then you do the work and see how incredibly difficult the
implementation is, others have tried. It's not trivial
and at best it's a very time consuming piece of work to
embark on.

Until then it's un-removable, plain as that :-)

I don't know why we're wasting our fingers discussing this
at all to be honest with you.

2007-09-28 05:38:10

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] Module use count must be updated as bridges are created/destroyed

Jan Beulich <[email protected]> wrote:
>
> So we have an unsolvable problem here then, unless infrastructure gets added
> that allows a module to declare itself as not-implicit-unload-safe, forcing
> modprobe -r to keep its hands off it. Ugly.

Yes I've always wanted to have a separate count that indicates
a module is in use but does not prevent its immediate removal.

Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2007-09-28 18:12:00

by Stephen Hemminger

[permalink] [raw]
Subject: Re: [PATCH] Module use count must be updated as bridges are created/destroyed

On Fri, 28 Sep 2007 13:37:39 +0800
Herbert Xu <[email protected]> wrote:

> Jan Beulich <[email protected]> wrote:
> >
> > So we have an unsolvable problem here then, unless infrastructure gets added
> > that allows a module to declare itself as not-implicit-unload-safe, forcing
> > modprobe -r to keep its hands off it. Ugly.
>
> Yes I've always wanted to have a separate count that indicates
> a module is in use but does not prevent its immediate removal.
>
> Cheers,

It is only the nested removal problem.

--
Stephen Hemminger <[email protected]>

2007-09-29 00:55:34

by Herbert Xu

[permalink] [raw]
Subject: Re: [PATCH] Module use count must be updated as bridges are created/destroyed

On Fri, Sep 28, 2007 at 11:11:07AM -0700, Stephen Hemminger wrote:
>
> It is only the nested removal problem.

Well such a count could apply to direct removals too. In other
words, you could ask modprobe to remove all modules for which
this count is zero.

Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt