All,
I am experiencing issues with connecting two network adapters to the same
subnet, eg.
eth0 192.168.100.200
eth1 192.168.100.201
The task is to have redundant connections to two different hubs. In case one
link goes down the connection should go through the other. The driver
handles link events with netif_carrier_ok and netif_carrier_on from
linux/netdevice.h. These eventually send messages to networking stack with
netdev_change_state from net/core/dev.c
My question is: Does the kernel handle the interface state/routing tables
modifications due to link changing automatically or is there some external
daemon required to do that. Any links are greatly appreciated.
Sincerely,
Jyri P?ldre.
Hi Juri,
J?ri P?ldre wrote:
> I am experiencing issues with connecting two network adapters to the same
> subnet, eg.
I recently had similar problems (specifically with the e1000 driver).
When one hang goes down a send to the other interface will hang. This
was when we were using the same socket to send on both interfaces, what
resulted is that the socket buffer would fill up and cause the send to
block. If you aren't using the same socket to send out on both
interfaces you should be fine. the rather large hack that i used to get
this to work on my system is below. basically it will not enqueue a
packet when the interface is down. this may well do nasty things to tcp...
peter
---
--- linux-2.6.8-24.3/net/core/dev.c
+++ linux-2.6.8-24.3/net/core/dev.c
@@ -1379,6 +1379,11 @@
#ifdef CONFIG_NET_CLS_ACT
skb->tc_verd = SET_TC_AT(skb->tc_verd,AT_EGRESS);
#endif
+ if (!netif_carrier_ok(dev)) {
+ rc = NET_XMIT_DROP;
+ goto out_kfree_skb;
+ }
+
if (q->enqueue) {
/* Grab device queue */
spin_lock(&dev->queue_lock);
J?ri P?ldre wrote:
> All,
>
> I am experiencing issues with connecting two network adapters to the same
> subnet, eg.
>
> eth0 192.168.100.200
> eth1 192.168.100.201
>
> The task is to have redundant connections to two different hubs. In case one
> link goes down the connection should go through the other. The driver
> handles link events with netif_carrier_ok and netif_carrier_on from
> linux/netdevice.h. These eventually send messages to networking stack with
> netdev_change_state from net/core/dev.c
>
> My question is: Does the kernel handle the interface state/routing tables
> modifications due to link changing automatically or is there some external
> daemon required to do that. Any links are greatly appreciated.
As far as I know, you have to handle this sort of thing in user-space. You may
also have ARP issues with having two interfaces on the same subnet. Often people
will use two private, non-related IP addresses and then migrate a virtual IP back
and forth, placing it on the preferred (ie, link-ok) interface.
Ben
--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com
On Fri, 14 Jan 2005, Peter Buckingham wrote:
> I recently had similar problems (specifically with the e1000
> driver). When one hang goes down a send to the other interface will
> hang. This was when we were using the same socket to send on both
> interfaces, what resulted is that the socket buffer would fill up
> and cause the send to block. If you aren't using the same socket to
> send out on both interfaces you should be fine. the rather large
> hack that i used to get this to work on my system is below.
> basically it will not enqueue a packet when the interface is down.
> this may well do nasty things to tcp...
There's an old thread on netdev, occasionally resurrected, about this
behaviour btw.
> peter
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
Fortune:
I hear what you're saying but I just don't care.
On Fri, 14 Jan 2005, [iso-8859-4] J?ri P?ldre wrote:
> My question is: Does the kernel handle the interface state/routing
> tables modifications due to link changing automatically
Not completely.
The biggest problem is that kernel does not remove its "connected" or
"subnet" route while link is down. This means that even though kernel
knows link is down, it will still try route packets out that
interface.
> or is there some external daemon required to do that. Any links are
> greatly appreciated.
There is "Netplug" - part of the net-tools package (On fedora core 3
at least). You can use it to 'ip link set dev .... down' when carrier
is removed. However, you wont get notified of carrier being inserted
back - I dont know whether that holds true generally, or whether its
an e1000 bug. So it's one-shot.
We're looking at adding support to the 'zebra' daemon in Quagga to
remove connected routes while link is down and add them back when
link-up again, and hence deal with this properly. Amir is very
interested in this..
/me grumbles about why oh why the "make kernel add connected routes"
feature was *ever* added in 2.3 in the first place (cause now
userspace has "forgotten" how to manage them, so we cant simply undo
this brain-damage[1] without breaking networking for everyone, sigh)
1. In retrospect at least. Brain-damage really only becomes obvious
with carrier-sensing drivers, which were very rare back in 2.2 days,
dont think there was a unified way to report these events to
userspace either.
> Sincerely,
> Jyri P?ldre.
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
Fortune:
Sailors in ships, sail on! Even while we died, others rode out the storm.
Hi,
>> My question is: Does the kernel handle the interface state/routing
>> tables modifications due to link changing automatically
>
> Not completely.
>
> The biggest problem is that kernel does not remove its "connected" or
> "subnet" route while link is down. This means that even though kernel
> knows link is down, it will still try route packets out that interface.
I can confirm this.
>> or is there some external daemon required to do that. Any links are
>> greatly appreciated.
Try the shell script below. It works for me (tm).
> There is "Netplug" - part of the net-tools package (On fedora core 3 at
> least). You can use it to 'ip link set dev .... down' when carrier is
> removed. However, you wont get notified of carrier being inserted back -
> I dont know whether that holds true generally, or whether its an e1000
> bug. So it's one-shot.
>
> We're looking at adding support to the 'zebra' daemon in Quagga to
> remove connected routes while link is down and add them back when
> link-up again, and hence deal with this properly. Amir is very
> interested in this..
Yes, I am. And I found out the following. If I add the following
one-liner shell script daemon, I get it all working:
while /bin/true; do echo "Check of eth5 ..."; A=`ethtool eth5 | grep
'Link detected'`; if [ "$A" != "$O" ] ; then C=`echo "$A" | grep 'yes'`;
if [ -n "$C" ] ; then echo "Adding route"; route add -net 5.5.7.0
netmask 255.255.255.0 dev eth5; else echo "Removing route"; route del
-net 5.5.7.0 netmask 255.255.255.0 dev eth5; fi; O="$A"; fi; sleep 1; done
[I have to mention that eth5 is the interface where I pull and reinsert
the cable]
As you can see I simply try to remove and re-add the connected route
from user space. The above shell script works for me, and i can pull and
re-insert my network cable as often as I want while still being able to
reach the network on eth5 over different routes (received via ospf, I
run Quagga) while the cable is pulled out and over the local interface
while the cable is back in.
> /me grumbles about why oh why the "make kernel add connected routes"
> feature was *ever* added in 2.3 in the first place (cause now
> userspace has "forgotten" how to manage them, so we cant simply undo
> this brain-damage[1] without breaking networking for everyone, sigh)
True.
I'm /not/ able to recreate the route as it was before my 'route del' the
recreated route looks different from the route I remove at first (the
connected one). I'm nor able to recreate the 'src xxx' flags and
neighter is it a 'proto kernel' route ;)
So, in my eyes we need:
- A way to create connected routes from user space
or
- A kernel which removes the connected route on link loss and recreates
it later when the link gets connected again.
Regards
- Amir
--
Amir Guindehi, [email protected]
DataCore GmbH, Witikonerstrasse 289, 8053 Zurich, Switzerland
J?ri P?ldre wrote:
> My question is: Does the kernel handle the interface state/routing tables
> modifications due to link changing automatically or is there some external
> daemon required to do that. Any links are greatly appreciated.
You might want to have a look at ifplugd (from memory:
http://0pointer.de/~lennart/). It works very well for me (i ifup/ifdown
my interfaces with it, works fine at least as long as there is only one
cable plugged at a time :-)
Stefan
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
> All,
>
> I am experiencing issues with connecting two network adapters to the same
> subnet, eg.
>
> eth0 192.168.100.200
> eth1 192.168.100.201
>
> The task is to have redundant connections to two different hubs. In case
> one link goes down the connection should go through the other.
Use the bonding.o driver. This scenario is _exactly_ what it's designed for
(when mode=1)
Mark.
- --
Mark Watts
Senior Systems Engineer
QinetiQ Trusted Information Management
Trusted Solutions and Services group
GPG Public Key ID: 455420ED
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
iD8DBQFB641/Bn4EFUVUIO0RAsN6AKC3SdnVTlnJ8XDB+bn6yIUf563rNwCeJaGc
qwPKo+ucugBZsXtn0Olve2A=
=UQSS
-----END PGP SIGNATURE-----