I'm working on a mechanism to load balance across multiple 100 mbit interfaces
using the 2.4.20 linux kernel + custom server software. I have several
interfaces all on the same subnet with different IP addresses. They are
all connected to the same multi-port switch.
The intent is that a client can connect to any of the IP addresses
specifically and only burden that one. Some clients can connect to #1, others
to #2, etc.
The problem I ran into was the kernel's handling of ARP requests. What linux
does is each interface receives the arp request, and every single one
answers the request. So it becomes a race condition which response gets to
the client, and the client will have usually an incorrect mac address/ip
address arp entry.
I fixed this problem by modifying net/ipv4/arp.c:
//add to top of file in the #includes
#include <linux/inetdevice.h> // (DA) 20030515 to fix arp problem
//...then later in function arp_process()
if (addr_type == RTN_LOCAL) {
n = neigh_event_ns(&arp_tbl, sha, &sip, dev);
if (n) {
+ struct in_device *ind;
int dont_send = 0;
if (IN_DEV_ARPFILTER(in_dev))
dont_send |= arp_filter(sip,tip,dev);
+// (DA) 20030515 only send arp response if dev's IP address matches
+ if((ind=__in_dev_get(dev))) {
+ struct in_ifaddr *ifa;
+ ifa=ind->ifa_list;
+ while(ifa)
+ {
+ if(ifa->ifa_address==tip) break;
+ ifa=ifa->ifa_next;
+ }
+ if(!ifa) dont_send=1;
+ }
if (!dont_send)
arp_send(ARPOP_REPLY,ETH_P_ARP,sip,dev,tip,sha,dev->dev_addr,sha);
neigh_release(n);
}
goto out;
Before sending the arp response, the requested IP address is checked against
the interface's configured IP address, and only if there is a match will an
ARP response be sent.
I think the the check is harmless in any case. It's not clear to me if you'd
ever want each interface answering the ARP requests, and it is clear there
is a valid reason for wanting to wire up a computer this way. Enjoy!
-Dave
I think the below is good in theory, but it probably needs to be checked
against the proxy-arp behavior of some routing applications before it is
accepted as cannon.
(i.e. consider the bridging code...?)
I don't have such an application here, nor is this my best level of
abstraction. It just seems like the most likely scenario for colliding with
the below.
Rob.
-----Original Message-----
From: [email protected]
[mailto:[email protected]]On Behalf Of David Ashley
Sent: Friday, May 16, 2003 3:02 PM
To: [email protected]
Subject: Problem in ARP 2.4.20 kernel
I'm working on a mechanism to load balance across multiple 100 mbit
interfaces
using the 2.4.20 linux kernel + custom server software. I have several
interfaces all on the same subnet with different IP addresses. They are
all connected to the same multi-port switch.
The intent is that a client can connect to any of the IP addresses
specifically and only burden that one. Some clients can connect to #1,
others
to #2, etc.
The problem I ran into was the kernel's handling of ARP requests. What linux
does is each interface receives the arp request, and every single one
answers the request. So it becomes a race condition which response gets to
the client, and the client will have usually an incorrect mac address/ip
address arp entry.
I fixed this problem by modifying net/ipv4/arp.c:
//add to top of file in the #includes
#include <linux/inetdevice.h> // (DA) 20030515 to fix arp problem
//...then later in function arp_process()
if (addr_type == RTN_LOCAL) {
n = neigh_event_ns(&arp_tbl, sha, &sip, dev);
if (n) {
+ struct in_device *ind;
int dont_send = 0;
if (IN_DEV_ARPFILTER(in_dev))
dont_send |= arp_filter(sip,tip,dev);
+// (DA) 20030515 only send arp response if dev's IP address matches
+ if((ind=__in_dev_get(dev))) {
+ struct in_ifaddr *ifa;
+ ifa=ind->ifa_list;
+ while(ifa)
+ {
+ if(ifa->ifa_address==tip) break;
+ ifa=ifa->ifa_next;
+ }
+ if(!ifa) dont_send=1;
+ }
if (!dont_send)
arp_send(ARPOP_REPLY,ETH_P_ARP,sip,dev,tip,sha,dev->dev_addr,sha);
neigh_release(n);
}
goto out;
Before sending the arp response, the requested IP address is checked against
the interface's configured IP address, and only if there is a match will an
ARP response be sent.
I think the the check is harmless in any case. It's not clear to me if you'd
ever want each interface answering the ARP requests, and it is clear there
is a valid reason for wanting to wire up a computer this way. Enjoy!
-Dave
David Ashley wrote:
> The problem I ran into was the kernel's handling of ARP requests. What linux
> does is each interface receives the arp request, and every single one
> answers the request. So it becomes a race condition which response gets to
> the client, and the client will have usually an incorrect mac address/ip
> address arp entry.
This was hashed through on the list about a year back. You might try googling
next time...
This is actually standards compliant behaviour, as silly as it sounds. However,
if you want stricter arp behaviour, and are using source-based routing, the
following will fix it.
echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter
Here is an example setup on 2.4.18:
# ip add
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
link/ether 00:30:65:bf:46:ba brd ff:ff:ff:ff:ff:ff
inet 47.129.82.58/24 brd 47.129.82.255 scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
link/ether 00:50:ff:90:04:44 brd ff:ff:ff:ff:ff:ff
inet 47.129.82.107/24 brd 47.129.82.255 scope global eth1
# ip ru
0: from all lookup local
32764: from 47.129.82.107 lookup 101
32765: from 47.129.82.58 lookup 100
32766: from all lookup main
32767: from all lookup 253
# ip ro
47.129.82.0/24 dev eth0 proto kernel scope link src 47.129.82.58
47.129.82.0/24 dev eth1 proto kernel scope link src 47.129.82.107
127.0.0.0/8 dev lo scope link
default via 47.129.82.1 dev eth0
# ip ro li table 100
47.129.82.0/24 dev eth0 scope link
default via 47.129.82.1 dev eth0
# ip ro li table 101
47.129.82.0/24 dev eth1 scope link
default via 47.129.82.1 dev eth1
With this setup, arp requests for 47.129.82.58 will only be answered by eth0,
and similarly for 47.129.82.107 and eth1.
Chris
--
Chris Friesen | MailStop: 043/33/F10
Nortel Networks | work: (613) 765-0557
3500 Carling Avenue | fax: (613) 765-2986
Nepean, ON K2H 8E9 Canada | email: [email protected]