Hello,
This may have been discussed, but all search engines couldn't give me a good answer...
I notice that when an interface is up/running, a local route is in the routing table:
$ ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:13:20:0e:2f:ed
inet addr:192.168.1.125 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::213:20ff:fe0e:2fed/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:35984995 errors:0 dropped:0 overruns:0 frame:0
TX packets:7409151 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3252413825 (3.2 GB) TX bytes:1340077250 (1.3 GB)
$ ip route
192.168.20.0/24 dev eth0 proto kernel scope link src 192.168.20.120
192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.125
default via 192.168.20.254 dev eth1 metric 100
After I unplug the cable from eth1, the RUNNING flag disappears, but the route is still there:
$ ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:13:20:0e:2f:ed
inet addr:192.168.1.125 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::213:20ff:fe0e:2fed/64 Scope:Link
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:35985023 errors:0 dropped:0 overruns:0 frame:0
TX packets:7409151 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3252415633 (3.2 GB) TX bytes:1340077250 (1.3 GB)
$ ip route
192.168.20.0/24 dev eth0 proto kernel scope link src 192.168.20.120
192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.125
default via 192.168.20.254 dev eth1 metric 100
And that *prevents* from using the default route to reach 192.168.1/24 subnet after eth1 is out.
I looked at the code, it seems the IFF_RUNNING flag change is ignored in dev_change_flags():
void __dev_notify_flags(struct net_device *dev, unsigned int old_flags)
{
.....
if (dev->flags & IFF_UP &&
(changes & ~(IFF_UP | IFF_PROMISC | IFF_ALLMULTI | IFF_VOLATILE)))
call_netdevice_notifiers(NETDEV_CHANGE, dev);
}
I searched in the Internet, and saw some people suggest using an application listener (eg, netplug) to remove the route.
My question is why cannot the kernel remove the route automatically when the link becomes down? Why should this complexity be pushed to the user to find a program to do that?
Thanks,
Joe
On Wed, 24 Nov 2010 11:48:03 -0800 (PST)
Mike Caoco <[email protected]> wrote:
> Hello,
>
> This may have been discussed, but all search engines couldn't give me a good answer...
>
> I notice that when an interface is up/running, a local route is in the routing table:
>
> $ ifconfig eth1
> eth1 Link encap:Ethernet HWaddr 00:13:20:0e:2f:ed
> inet addr:192.168.1.125 Bcast:192.168.1.255 Mask:255.255.255.0
> inet6 addr: fe80::213:20ff:fe0e:2fed/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:35984995 errors:0 dropped:0 overruns:0 frame:0
> TX packets:7409151 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:3252413825 (3.2 GB) TX bytes:1340077250 (1.3 GB)
>
> $ ip route
> 192.168.20.0/24 dev eth0 proto kernel scope link src 192.168.20.120
> 192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.125
> default via 192.168.20.254 dev eth1 metric 100
>
> After I unplug the cable from eth1, the RUNNING flag disappears, but the route is still there:
>
> $ ifconfig eth1
> eth1 Link encap:Ethernet HWaddr 00:13:20:0e:2f:ed
> inet addr:192.168.1.125 Bcast:192.168.1.255 Mask:255.255.255.0
> inet6 addr: fe80::213:20ff:fe0e:2fed/64 Scope:Link
> UP BROADCAST MULTICAST MTU:1500 Metric:1
> RX packets:35985023 errors:0 dropped:0 overruns:0 frame:0
> TX packets:7409151 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:3252415633 (3.2 GB) TX bytes:1340077250 (1.3 GB)
>
> $ ip route
> 192.168.20.0/24 dev eth0 proto kernel scope link src 192.168.20.120
> 192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.125
> default via 192.168.20.254 dev eth1 metric 100
>
> And that *prevents* from using the default route to reach 192.168.1/24 subnet after eth1 is out.
>
> I looked at the code, it seems the IFF_RUNNING flag change is ignored in dev_change_flags():
>
> void __dev_notify_flags(struct net_device *dev, unsigned int old_flags)
> {
> .....
> if (dev->flags & IFF_UP &&
> (changes & ~(IFF_UP | IFF_PROMISC | IFF_ALLMULTI | IFF_VOLATILE)))
> call_netdevice_notifiers(NETDEV_CHANGE, dev);
> }
>
> I searched in the Internet, and saw some people suggest using an application listener (eg, netplug) to remove the route.
>
> My question is why cannot the kernel remove the route automatically when the link becomes down? Why should this complexity be pushed to the user to find a program to do that?
>
Because there is no reason for the kernel to not expect the link to come back.
It is up to user space to do routing policy. For desktop/laptop users this is
done typically with NetworkManager or Connman; for routers this is done with
Quagga; and for servers use other tools.
If the kernel automatically removed the route, it would cause routing daemons
to recompute the route table (and propagate the change) every time a cable
got pulled or NIC needed to be reset.
--
--- On Wed, 11/24/10, Stephen Hemminger <[email protected]> wrote:
> From: Stephen Hemminger <[email protected]>
> Subject: Re: Unplug ethernet cable, the route persists. Why?
> To: "Mike Caoco" <[email protected]>
> Cc: "Netdev" <[email protected]>, "LKML" <[email protected]>
> Date: Wednesday, November 24, 2010, 12:18 PM
> On Wed, 24 Nov 2010 11:48:03 -0800
> (PST)
> Mike Caoco <[email protected]>
> wrote:
>
> > Hello,
> >
> > This may have been discussed, but all search engines
> couldn't give me a good answer...
> >
> > I notice that when an interface is up/running, a local
> route is in the routing table:
> >
> > $ ifconfig eth1
> > eth1? ? ? Link encap:Ethernet?
> HWaddr 00:13:20:0e:2f:ed?
> >? ? ? ? ???inet
> addr:192.168.1.125? Bcast:192.168.1.255?
> Mask:255.255.255.0
> >? ? ? ? ???inet6
> addr: fe80::213:20ff:fe0e:2fed/64 Scope:Link
> >? ? ? ? ???UP
> BROADCAST RUNNING MULTICAST? MTU:1500? Metric:1
> >? ? ? ? ???RX
> packets:35984995 errors:0 dropped:0 overruns:0 frame:0
> >? ? ? ? ???TX
> packets:7409151 errors:0 dropped:0 overruns:0 carrier:0
> >? ? ? ?
> ???collisions:0 txqueuelen:1000
> >? ? ? ? ???RX
> bytes:3252413825 (3.2 GB)? TX bytes:1340077250 (1.3
> GB)
> >
> > $ ip route
> > 192.168.20.0/24 dev eth0? proto kernel?
> scope link? src 192.168.20.120
> > 192.168.1.0/24 dev eth1? proto kernel? scope
> link? src 192.168.1.125
> > default via 192.168.20.254 dev eth1? metric 100
> >
> > After I unplug the cable from eth1, the RUNNING flag
> disappears, but the route is still there:
> >
> > $ ifconfig eth1
> > eth1? ? ? Link encap:Ethernet?
> HWaddr 00:13:20:0e:2f:ed?
> >? ? ? ? ???inet
> addr:192.168.1.125? Bcast:192.168.1.255?
> Mask:255.255.255.0
> >? ? ? ? ???inet6
> addr: fe80::213:20ff:fe0e:2fed/64 Scope:Link
> >? ? ? ? ???UP
> BROADCAST MULTICAST? MTU:1500? Metric:1
> >? ? ? ? ???RX
> packets:35985023 errors:0 dropped:0 overruns:0 frame:0
> >? ? ? ? ???TX
> packets:7409151 errors:0 dropped:0 overruns:0 carrier:0
> >? ? ? ?
> ???collisions:0 txqueuelen:1000
> >? ? ? ? ???RX
> bytes:3252415633 (3.2 GB)? TX bytes:1340077250 (1.3
> GB)
> >
> > $ ip route
> > 192.168.20.0/24 dev eth0? proto kernel?
> scope link? src 192.168.20.120
> > 192.168.1.0/24 dev eth1? proto kernel? scope
> link? src 192.168.1.125
> > default via 192.168.20.254 dev eth1? metric 100
> >
> > And that *prevents* from using the default route to
> reach 192.168.1/24 subnet after eth1 is out.
> >
> > I looked at the code, it seems the IFF_RUNNING flag
> change is ignored in dev_change_flags():
> >
> > void __dev_notify_flags(struct net_device *dev,
> unsigned int old_flags)
> > {
> >? ? ? ???.....
> >? ? ? ???if
> (dev->flags & IFF_UP &&
> >? ? ? ? ?
> ???(changes & ~(IFF_UP | IFF_PROMISC |
> IFF_ALLMULTI | IFF_VOLATILE)))
> >? ? ? ? ? ? ?
> ???call_netdevice_notifiers(NETDEV_CHANGE,
> dev);
> > }
> >
> > I searched in the Internet, and saw some people
> suggest using an application listener (eg, netplug) to
> remove the route.
> >
> > My question is why cannot the kernel remove the route
> automatically when the link becomes down?? Why should
> this complexity be pushed to the user to find a program to
> do that?
> >
>
> Because there is no reason for the kernel to not expect the
> link to come back.
> It is up to user space to do routing policy. For
> desktop/laptop users this is
> done typically with NetworkManager or Connman; for routers
> this is done with
> Quagga; and for servers use other tools.
>
> If the kernel automatically removed the route, it would
> cause routing daemons
> to recompute the route table (and propagate the change)
> every time a cable
> got pulled or NIC needed to be reset.
>
So if you rely on NetworkManager or Connman or Quagga to remove the route, the routing daemons will recompute the route table anyway. So why cannot this be done in the kernel?
Even when no NetworkManager/Quagga is present, I think it is a legitimate reason to recompute the route when a cable is unplugged, which should not be a frequent event unless when under error conditions.
Thanks,
From: Mike Caoco <[email protected]>
Date: Wed, 24 Nov 2010 12:29:43 -0800 (PST)
> Even when no NetworkManager/Quagga is present, I think it is a
> legitimate reason to recompute the route when a cable is unplugged,
> which should not be a frequent event unless when under error
> conditions.
Cards periodically reset themselves, faulty switchs flap occaisionally,
this is life and it shouldn't cause route table recomputations across
your entire region.
Also Stephen listed places where such policy should be employed in
userspace, he absolutely did not say they should act that way by
default.
On 11/24/2010 08:48 PM, Mike Caoco wrote:
> Hello,
>
> This may have been discussed, but all search engines couldn't give me a good answer...
>
> I notice that when an interface is up/running, a local route is in the routing table:
>
> $ ifconfig eth1
> eth1 Link encap:Ethernet HWaddr 00:13:20:0e:2f:ed
> inet addr:192.168.1.125 Bcast:192.168.1.255 Mask:255.255.255.0
> inet6 addr: fe80::213:20ff:fe0e:2fed/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:35984995 errors:0 dropped:0 overruns:0 frame:0
> TX packets:7409151 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:3252413825 (3.2 GB) TX bytes:1340077250 (1.3 GB)
>
> $ ip route
> 192.168.20.0/24 dev eth0 proto kernel scope link src 192.168.20.120
> 192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.125
> default via 192.168.20.254 dev eth1 metric 100
>
> After I unplug the cable from eth1, the RUNNING flag disappears, but the route is still there:
>
> $ ifconfig eth1
> eth1 Link encap:Ethernet HWaddr 00:13:20:0e:2f:ed
> inet addr:192.168.1.125 Bcast:192.168.1.255 Mask:255.255.255.0
> inet6 addr: fe80::213:20ff:fe0e:2fed/64 Scope:Link
> UP BROADCAST MULTICAST MTU:1500 Metric:1
> RX packets:35985023 errors:0 dropped:0 overruns:0 frame:0
> TX packets:7409151 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:3252415633 (3.2 GB) TX bytes:1340077250 (1.3 GB)
>
> $ ip route
> 192.168.20.0/24 dev eth0 proto kernel scope link src 192.168.20.120
> 192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.125
> default via 192.168.20.254 dev eth1 metric 100
>
> And that *prevents* from using the default route to reach 192.168.1/24 subnet after eth1 is out.
Well suppose the default route is used, and the source address is of the
packets stay 192.168.1.125, then there is no way for the peers to
respond to the packets. I do not know wat sets the source address in
linux, but on windows it will allway's be the one you do not want.
--
Hans
On Wed, 24 Nov 2010 12:29:43 -0800 (PST), Mike Caoco <[email protected]> wrote:
> So if you rely on NetworkManager or Connman or Quagga to remove the
> route, the routing daemons will recompute the route table anyway. So
> why cannot this be done in the kernel?
This is policy. In the Linux world we generally strive to separate
policy from mechanism, leaving the former to userspace. This allows
(potentially complex) policy decisions to be made in user-space. The
reason for this is two-fold: First, every line of kernel code introduces
the potentially for a bug and error handling in the kernel is generally
more complex than it is in user-space. Secondly, allowing user-space to
handle policy allows users to do things with the kernel that kernel
developers did not envision. This flexibility is one reason why the
kernel is so suited for running on anything from your cell-phone to 4000
processor big iron.
> Even when no NetworkManager/Quagga is present, I think it is a
> legitimate reason to recompute the route when a cable is unplugged,
> which should not be a frequent event unless when under error
> conditions.
There have to be real, demonstrable benefits for moving policy into the
kernel (i.e. the recent discussions concerning per-session
cgroups). Considering how long the Linux networking subsystem has
existed, I highly doubt there is a good reason to move this sort of
routing policy into the kernel that has not already been discussed.
Cheers,
- Ben
Ben Gamari wrote:
> On Wed, 24 Nov 2010 12:29:43 -0800 (PST), Mike Caoco <[email protected]> wrote:
>> So if you rely on NetworkManager or Connman or Quagga to remove the
>> route, the routing daemons will recompute the route table anyway. So
>> why cannot this be done in the kernel?
>
> This is policy. In the Linux world we generally strive to separate
> policy from mechanism, leaving the former to userspace. This allows
> (potentially complex) policy decisions to be made in user-space. The
> reason for this is two-fold: First, every line of kernel code introduces
> the potentially for a bug and error handling in the kernel is generally
> more complex than it is in user-space. Secondly, allowing user-space to
> handle policy allows users to do things with the kernel that kernel
> developers did not envision. This flexibility is one reason why the
> kernel is so suited for running on anything from your cell-phone to 4000
> processor big iron.
Secondly and a half, if you add a specific route you may really
mean it, and prefer not to send at all than use default.
Cheers,
Jarek P.