2001-04-16 20:49:54

by Sampsa Ranta

[permalink] [raw]
Subject: ARP responses broken!


Hello, I had a mystery with my Linux running 2.4.2 kernel with ARP packet
response.

I have two interfaces that share same subnet, I call eth0 194.29.192.37
and eth1 194.29.192.38. I have forwarding turned on, proxy arp is not
neighter are redirects.

When I flush local neighbor table in other machine I use to observe the
response and ping the router I get response like:

23:38:25.278848 > arp who-has 194.29.192.38 tell 194.29.192.10 (0:50:da:82:ae:9f)
23:38:25.278988 < arp reply 194.29.192.38 is-at 0:1:2:dc:d2:64 (0:50:da:82:ae:9f)
23:38:25.279009 < arp reply 194.29.192.38 is-at 0:1:2:dc:d2:6c (0:50:da:82:ae:9f)

The second one is the valid one, but both interfaces seem to answer to the
broadcasted packet with their own ARP addresses.

This came up when I wondered why I get responses to wrong interface with
Zebra.

- Sampsa Ranta
[email protected]


2001-04-16 21:27:09

by Eric Weigle

[permalink] [raw]
Subject: Re: ARP responses broken!

Hello-

This is a known 'feature' of the Linux kernel, and can help with load sharing
and fault tolerance. However, it can also cause problems (such as when one nic
in a multi-nic machine fails and you don't know right away).

There are three 'solutions' I know of:

* In recent 2.2 kernels, it was possible to fix this by doing the following as
root:
# Start the hiding interface functionality
echo 1 > /proc/sys/net/ipv4/conf/all/hidden
# Hide all addresses for this interface
echo 1 > /proc/sys/net/ipv4/conf/<interface_name>/hidden
but 2.4 doesn't have that option, for technical reasons.

* Use 'ifconfig -arp ...' to force an interface not to respond to ARP
requests. Hosts which want to send to that interface may need to manually add
the proper mac address to their ARP tables with 'arp -s'.

* Use a packet filtering tool (iptables arp filter module, for example) and
just filter the ARP requests and ARP replies so that only the proper set get
through, i.e. when an arp request for the mac address of an interface arrives,
filter out arp replies from all the other interfaces.

There have been a few threads on this on the linux-kernel mailing list. Search
your favorite archive for them.

-Eric

--------------------------------------------
Eric H. Weigle CCS-1, RADIANT team
[email protected] Los Alamos National Lab
(505) 665-4937 http://home.lanl.gov/ehw/
--------------------------------------------

2001-04-16 22:48:22

by Bernd Eckenfels

[permalink] [raw]
Subject: Re: ARP responses broken!

In article <[email protected]> you wrote:
> The second one is the valid one, but both interfaces seem to answer to the
> broadcasted packet with their own ARP addresses.

it is because the kernel does not know if both interfaces are on one subnet,
or not. The easisets thing to solve this is t use the MAC moduleof netfilter
and deny the incoming requests/responsnes based on the ip.

Greetings
Bernd

2001-04-17 13:56:22

by Eric Weigle

[permalink] [raw]
Subject: Re: ARP responses broken!

> oh great, now I wont be able to upgrade our kernels to 2.4 unless I find a
> utility to filter out the ARP requests?
"There's more than one way to do it" (see below)

> Why was this ability removed?
Apparently the decision was made to do it this way because it simplified the
fast path of the code; but I could be wrong. I have yet to achieve Linux Guru
status, right now I'm just some guy who has hit this problem and knows the
work-arounds.

> If I screw up and put it on the wrong card, I WANT the system to stop
> working...
I agree. I'm not making an argument for this implementation. Well, maybe I am;
Linux is intended as a desktop operating system and in that context most people
would rather have the failsafe than the failstop. Perhaps this method is
`inelegant' but it kept up a cluster of machines we have here a lot longer than
they might otherwise have functioned (the gige Acenic driver had some problems a
while back and about half the cards silently failed-- the 'broken' arp responses
meant that we could still talk to those boxes without reconfiguring IPs).

Anyway, Here's links to a discussion that occurred this January.

If you require the 'hidden' functionality, the first message cites the following
links; the patch apparently works with 2.4.x with some tweaking.
http://www.linuxvirtualserver.org/arp.html
http://www.linuxvirtualserver.org/hidden-2.3.41-1.diff

http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.3/0014.html
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.3/0020.html
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.3/0188.html
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.3/0213.html
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.3/0220.html
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.3/0268.html
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.3/0334.html
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.3/0410.html

Hope that helps
-Eric

--------------------------------------------
Eric H. Weigle CCS-1, RADIANT team
[email protected] Los Alamos National Lab
(505) 665-4937 http://home.lanl.gov/ehw/
--------------------------------------------

2001-04-17 14:13:35

by Sampsa Ranta

[permalink] [raw]
Subject: Broken ARP (was Re: ARP responses broken!)

On Mon, 16 Apr 2001, Eric Weigle wrote:

> Hello-
>
> This is a known 'feature' of the Linux kernel, and can help with load sharing
> and fault tolerance. However, it can also cause problems (such as when one nic
> in a multi-nic machine fails and you don't know right away).

I tought this for a while and this does not help load sharing neighter or
fault tolerance. Causes problem with router environment. I use different
cards to load the problem by assigning different addresses to these and by
pointing these addresses with routes so I use the IP to mark a device.

Only case where this would help with "fault tolerance" is if I
assign address to other device that is not marked as up, it would still
be possible to see the address via other device, and this goes way off.

> There are three 'solutions' I know of:
>
> * In recent 2.2 kernels, it was possible to fix this by doing the following as
> but 2.4 doesn't have that option, for technical reasons.

I don't see good valid reasons, I implemented this by myself to the kernel
with very little effort, only thing is that my patch makes it not
possible to have the same address on multible interfaces, this is
altought very easy to fix. FIB code lacks a way to detect if there are
multible interfaces for a single IP.

> * Use 'ifconfig -arp ...' to force an interface not to respond to ARP
> requests. Hosts which want to send to that interface may need to manually add
> the proper mac address to their ARP tables with 'arp -s'.

Manual ARP, uh, sounds as nasty as can be.

> * Use a packet filtering tool (iptables arp filter module, for example) and
> just filter the ARP requests and ARP replies so that only the proper set get
> through, i.e. when an arp request for the mac address of an interface arrives,
> filter out arp replies from all the other interfaces.

There is no ARP module for netfilter and this is such low level issue that
would see requires another way to be solved by all means.

It is not working correctly if there are two ARP replies for a ARP query,
another one from interface that DOES NOT have the address assigned to it.
Basically I should have a firewall rule that would instantly DROP all
packets that come to the interface with address that it does not have if I
don't have forwarding enabled.

> There have been a few threads on this on the linux-kernel mailing list. Search
> your favorite archive for them.

Well, this is not solved so we probaply need one more.

The code I used to do the trick at my network was as simple as this,
in function arp_rcv, the problem is ip_dev_find that does know if there
are other devices with same IP address.

--- job/arp.c Tue Apr 17 17:05:48 2001
+++ arp.c Tue Apr 17 17:05:24 2001
@@ -571,6 +571,7 @@
int addr_type;
struct in_device *in_dev = in_dev_get(dev);
struct neighbour *n;
+ struct net_device *devx = NULL;

/*
* The hardware length of the packet should match the hardware length
@@ -728,8 +729,10 @@

rt = (struct rtable*)skb->dst;
addr_type = rt->rt_type;

- if (addr_type == RTN_LOCAL) {
+ devx = ip_dev_find(tip);
+ if (addr_type == RTN_LOCAL &&
+ devx == dev) {
n = neigh_event_ns(&arp_tbl, sha, &sip, dev);
if (n) {
arp_send(ARPOP_REPLY,ETH_P_ARP,sip,dev,tip,sha,dev->dev_addr,sha);


- Sampsa Ranta
[email protected]


2001-04-17 14:19:46

by Andi Kleen

[permalink] [raw]
Subject: Re: ARP responses broken!

On Mon, Apr 16, 2001 at 03:26:19PM -0600, Eric Weigle wrote:
> Hello-
>
> This is a known 'feature' of the Linux kernel, and can help with load sharing
> and fault tolerance. However, it can also cause problems (such as when one nic
> in a multi-nic machine fails and you don't know right away).
>
> There are three 'solutions' I know of:
>
> * In recent 2.2 kernels, it was possible to fix this by doing the following as

Or use arpfilter in even newer 2.2 kernels; which filters based on the routing
table. "hidden" is quite a sledgehammer which often does more harm than good.


-Andi

2001-04-17 14:26:48

by Chris Friesen

[permalink] [raw]
Subject: Re: ARP responses broken!

Sampsa Ranta wrote:

> I have two interfaces that share same subnet, I call eth0 194.29.192.37
> and eth1 194.29.192.38. I have forwarding turned on, proxy arp is not
> neighter are redirects.
>
> When I flush local neighbor table in other machine I use to observe the
> response and ping the router I get response like:
>
> 23:38:25.278848 > arp who-has 194.29.192.38 tell 194.29.192.10 (0:50:da:82:ae:9f)
> 23:38:25.278988 < arp reply 194.29.192.38 is-at 0:1:2:dc:d2:64 (0:50:da:82:ae:9f)
> 23:38:25.279009 < arp reply 194.29.192.38 is-at 0:1:2:dc:d2:6c (0:50:da:82:ae:9f)
>
> The second one is the valid one, but both interfaces seem to answer to the
> broadcasted packet with their own ARP addresses.

This is the default Linux behaviour. It can be turned off by running the
following command as root:

echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter

This ensures that interfaces will only respond to arp requests for IP addresses
which are configured as belonging to that particular interface.

Chris

--
Chris Friesen | MailStop: 043/33/F10
Nortel Networks | work: (613) 765-0557
3500 Carling Avenue | fax: (613) 765-2986
Nepean, ON K2H 8E9 Canada | email: [email protected]

2001-04-17 14:31:56

by Bernd Eckenfels

[permalink] [raw]
Subject: Re: ARP responses broken!

> but why would you want it to reply for the IP of the other interface even if
> it was NOT on the same subnet?

Because Linux is always answering to all its local IP addresses, regardless
of the Network interface. Even if you tun off the IP Forwarding.

This is by Designs, there are situation where this is good and there are
situations where it is not so good. But it's an old tradition.

Greetings
Bernd
--
(OO) -- [email protected] --
( .. ) ecki@{inka.de,linux.de,debian.org} http://home.pages.de/~eckes/
o--o *plush* 2048/93600EFD eckes@irc +497257930613 BE5-RIPE
(O____O) When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl!

2001-04-17 14:54:16

by Martin Josefsson

[permalink] [raw]
Subject: Re: ARP responses broken!

On Tue, 17 Apr 2001, Andi Kleen wrote:

> On Mon, Apr 16, 2001 at 03:26:19PM -0600, Eric Weigle wrote:
> > Hello-
> >
> > This is a known 'feature' of the Linux kernel, and can help with load sharing
> > and fault tolerance. However, it can also cause problems (such as when one nic
> > in a multi-nic machine fails and you don't know right away).
> >
> > There are three 'solutions' I know of:
> >
> > * In recent 2.2 kernels, it was possible to fix this by doing the following as
>
> Or use arpfilter in even newer 2.2 kernels; which filters based on the routing
> table. "hidden" is quite a sledgehammer which often does more harm than good.

Does arpfilter exist in 2.4 kernels?

/Martin

2001-04-17 15:01:36

by Andi Kleen

[permalink] [raw]
Subject: Re: ARP responses broken!

On Tue, Apr 17, 2001 at 04:53:01PM +0200, Martin Josefsson wrote:
> On Tue, 17 Apr 2001, Andi Kleen wrote:
>
> > On Mon, Apr 16, 2001 at 03:26:19PM -0600, Eric Weigle wrote:
> > > Hello-
> > >
> > > This is a known 'feature' of the Linux kernel, and can help with load sharing
> > > and fault tolerance. However, it can also cause problems (such as when one nic
> > > in a multi-nic machine fails and you don't know right away).
> > >
> > > There are three 'solutions' I know of:
> > >
> > > * In recent 2.2 kernels, it was possible to fix this by doing the following as
> >
> > Or use arpfilter in even newer 2.2 kernels; which filters based on the routing
> > table. "hidden" is quite a sledgehammer which often does more harm than good.
>
> Does arpfilter exist in 2.4 kernels?

Not yet, will be merged very soon. I can send you a patch if you need it urgently.


-Andi

2001-04-17 15:07:56

by Martin Josefsson

[permalink] [raw]
Subject: Re: ARP responses broken!

On Tue, 17 Apr 2001, Andi Kleen wrote:

[snip]
> > Does arpfilter exist in 2.4 kernels?
>
> Not yet, will be merged very soon. I can send you a patch if you need it urgently.

No I don't need it urgently.
I was asking because I had this problem before (router with two cards
against one physical subnet) and arpwatch complained that the router kept
switching MACaddresses all the time.

/Martin

2001-04-17 15:22:00

by Eric Weigle

[permalink] [raw]
Subject: Re: Broken ARP (was Re: ARP responses broken!)

Ok, I was ignorant of the arp filter functionality in 2.2. I found an old
(probably painfully out-of-date) posting the patch Andi Kleen was referring to
in the archive, but I've not used it.
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.2/1198.html

> I tought this for a while and this does not help load sharing neighter or
> fault tolerance. Causes problem with router environment. I use different
> cards to load the problem by assigning different addresses to these and by
> pointing these addresses with routes so I use the IP to mark a device.
>
> Only case where this would help with "fault tolerance" is if I
> assign address to other device that is not marked as up, it would still
> be possible to see the address via other device, and this goes way off.

Indeed, the default behavior does cause problems in a router environment, but
this only happens when multiple nics are on the same subnet; in a 'true' router
each nic would be on a separate subnet. Regardless, I personally use FreeBSD
when I need a router (Horrors! ;)

What I meant by load sharing was implicit sharing rather than explicit sharing;
when an ARP request comes the reply the host gets may have the MAC address of
NICs other than the one explicitly bound to the given IP-- thus different hosts
will semi-randomly get different MAC addresses and thus send to different NICs;
this implicit sharing completely hoses explicit load sharing.

And as for fault tolerance, here's what happened to me, as I mentioned in
another message: We have a 8-node cluster with 2 nics, a eepro and a gig-e
acenic in each node. A while back the acenic driver had some problems and would
silently fail after a while; the arp reponse behavior allowed the cluster to
remain 'up' long enough to finish the jobs we assigned to it (although
performance sucked since all traffic went over the eepros). After we were done,
we could ifdown/ifup the interfaces and all was good. Again, this is a sort of
'implicit' fault tolerance, rather than a more explicit form where the card goes
down, we get some sort of notification, and it fails over to the other card
explicitly.

> The code I used to do the trick at my network was as simple as this,
> in function arp_rcv, the problem is ip_dev_find that does know if there
> are other devices with same IP address.
Well, Yes, but that's not really the issue. The problem is 'what is the proper
*default* behavior of the Linux ARP subsystem'... this code changes it, which is
probably more of a political than technical decision. Where you (and I) see
'broken' others see 'feature' :/

-Eric

--------------------------------------------
Eric H. Weigle CCS-1, RADIANT team
[email protected] Los Alamos National Lab
(505) 665-4937 http://home.lanl.gov/ehw/
--------------------------------------------

2001-04-17 15:32:30

by dean gaudet

[permalink] [raw]
Subject: Re: Broken ARP (was Re: ARP responses broken!)

On Tue, 17 Apr 2001, Eric Weigle wrote:

> Ok, I was ignorant of the arp filter functionality in 2.2. I found an old
> (probably painfully out-of-date) posting the patch Andi Kleen was referring to
> in the archive, but I've not used it.
> http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.2/1198.html

oh heck, why not post a third patch for this problem. here's the one i
wrote.

-dean

--- linux/net/ipv4/arp.c.badproxy Mon Feb 12 17:28:48 2001
+++ linux/net/ipv4/arp.c Tue Feb 13 20:06:37 2001
@@ -737,10 +737,12 @@
addr_type = rt->rt_type;

if (addr_type == RTN_LOCAL) {
+ if ((rt->rt_flags&RTCF_DIRECTSRC) || IN_DEV_PROXY_ARP(in_dev)) {
n = neigh_event_ns(&arp_tbl, sha, &sip, dev);
if (n) {
arp_send(ARPOP_REPLY,ETH_P_ARP,sip,dev,tip,sha,dev->dev_addr,sha);
neigh_release(n);
+ }
}
goto out;
} else if (IN_DEV_FORWARD(in_dev)) {

2001-04-17 16:05:02

by Alan

[permalink] [raw]
Subject: Re: ARP responses broken!

> I was asking because I had this problem before (router with two cards
> against one physical subnet) and arpwatch complained that the router kept
> switching MACaddresses all the time.

That sounds like a bug in arpwatch. A box can have multiple mac addresses. Its
probably a tricky one to handle but arpwatch I guess should spot and cope with
repeated transitions between the same set of addresses as one warning


Subject: RE: ARP responses broken!

Correction, that was on kernel v2.2.19

Sam

****** Forwarded Message Follows *******
>To: "'Christopher Friesen'" <[email protected]>, Sampsa Ranta

<[email protected]>
>From: "Bingner Sam J. Contractor RSIS" <[email protected]>
>Date: Tue, 17 Apr 2001 18:07:41 -0000
>
>I tested this with kernel version 2.2.18 and arp_filter appeared to be
>broken... I enabled it for /proc/sys/net/ipv4/conf/all/arp_filter,
>/proc/sys/net/ipv4/conf/eth0/arp_filter and
>/proc/sys/net/ipv4/conf/eth1/arp_filter and it did not change the arp
>behavior at all. I enabled hidden and it worked, is there a know problem
>with this functionality?
>
> Sam Bingner
> PACAF CSS/SCHE
> Contractor RSIS
> DSN 315 449-7889
> COMM 808 449-7889
>

Subject: RE: ARP responses broken!

I tested this with kernel version 2.2.18 and arp_filter appeared to be
broken... I enabled it for /proc/sys/net/ipv4/conf/all/arp_filter,
/proc/sys/net/ipv4/conf/eth0/arp_filter and
/proc/sys/net/ipv4/conf/eth1/arp_filter and it did not change the arp
behavior at all. I enabled hidden and it worked, is there a know problem
with this functionality?

Sam Bingner
PACAF CSS/SCHE
Contractor RSIS
DSN 315 449-7889
COMM 808 449-7889


-----Original Message-----
From: Christopher Friesen [mailto:[email protected]]
Sent: Tuesday, April 17, 2001 4:25 AM
To: Sampsa Ranta
Cc: linux-net; linux-kernel
Subject: Re: ARP responses broken!


Sampsa Ranta wrote:

> I have two interfaces that share same subnet, I call eth0 194.29.192.37
> and eth1 194.29.192.38. I have forwarding turned on, proxy arp is not
> neighter are redirects.
>
> When I flush local neighbor table in other machine I use to observe the
> response and ping the router I get response like:
>
> 23:38:25.278848 > arp who-has 194.29.192.38 tell 194.29.192.10
(0:50:da:82:ae:9f)
> 23:38:25.278988 < arp reply 194.29.192.38 is-at 0:1:2:dc:d2:64
(0:50:da:82:ae:9f)
> 23:38:25.279009 < arp reply 194.29.192.38 is-at 0:1:2:dc:d2:6c
(0:50:da:82:ae:9f)
>
> The second one is the valid one, but both interfaces seem to answer to the
> broadcasted packet with their own ARP addresses.

This is the default Linux behaviour. It can be turned off by running the
following command as root:

echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter

This ensures that interfaces will only respond to arp requests for IP
addresses
which are configured as belonging to that particular interface.

Chris

--
Chris Friesen | MailStop: 043/33/F10
Nortel Networks | work: (613) 765-0557
3500 Carling Avenue | fax: (613) 765-2986
Nepean, ON K2H 8E9 Canada | email: [email protected]

2001-04-17 21:43:47

by Rogier Wolff

[permalink] [raw]
Subject: Re: ARP responses broken!

Alan Cox wrote:
> > I was asking because I had this problem before (router with two cards
> > against one physical subnet) and arpwatch complained that the router kept
> > switching MACaddresses all the time.

> That sounds like a bug in arpwatch. A box can have multiple mac
> addresses. Its probably a tricky one to handle but arpwatch I guess
> should spot and cope with repeated transitions between the same set
> of addresses as one warning

Well, two. Or three.

- Hey, IP x changed from mac X to mac Y.
- Hey, IP x changed back again to X.
- Hmm. IP X seems to be using both Mac X and and Mac Y.
No further warnings will be issued about this.

If someone is taking over an IP address (which is especially what
arpwatch should be looking for), this is exactly what you'll see. Having
the issue be ignored after one warning is bad.

Oh, and I know people who swear that this would be an invalid
configuration, so that it is good for arpwatch to should loud and
clear about it...

Roger.

--
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots.
* There are also old, bald pilots.

2001-04-17 22:05:49

by Julian Anastasov

[permalink] [raw]
Subject: Re: ARP responses broken!


Hello,

Sampsa Ranta wrote:

> 23:38:25.278848 > arp who-has 194.29.192.38 tell 194.29.192.10 (0:50:da:82:ae:9f)
> 23:38:25.278988 < arp reply 194.29.192.38 is-at 0:1:2:dc:d2:64 (0:50:da:82:ae:9f)
> 23:38:25.279009 < arp reply 194.29.192.38 is-at 0:1:2:dc:d2:6c (0:50:da:82:ae:9f)
>
> The second one is the valid one, but both interfaces seem to answer to the
> broadcasted packet with their own ARP addresses.

arp_filter is not broken, it is simply not for your setup.
It depends what you want to achieve by defining two IP addresses in
different interfaces. Considering the fact you have two addresses
in one subnet you need the incoming traffic to come from the two
interfaces. In this case you need "hidden". For the outgoing traffic:
it is controlled only from the routing.

While in your setup arp_filter and rp_filter will ARP answer
from one card only, for the both addresses, hidden will answer from the
both cards, "correctly" in your eyes. Use arp_filter for different
nets only, i.e. when the ARP probes come from different nets in your
routing universe. hidden does not depend on nets/subnets. But may
be there are exceptions I'm missing and the other guys can correct me.

Regards

--
Julian Anastasov <[email protected]>

2001-04-17 22:44:25

by Julian Anastasov

[permalink] [raw]
Subject: Re: Broken ARP (was Re: ARP responses broken!)


Hello,

Sampsa Ranta wrote:

> The code I used to do the trick at my network was as simple as this,
> in function arp_rcv, the problem is ip_dev_find that does know if there
> are other devices with same IP address.

I don't think this is your problem. You patch is not correct.
In fact, you implement the same function as in "hidden" but you are
missing some things. Please, read the "hidden" flag description in
the kernel docs. You must solve the case where your ARP probes are sent
always through one device due to your routing (this is out traffic,
yes?). These probes soon or later will cause you problems because
they change the entry in the remote hosts' ARP tables. You so carefully
tried to advertise the address on specific interface and now the other
hosts again talk to one card only.

who-has 194.29.192.1 tell 194.29.192.38

and your are dead :)

So, please try "hidden" before going into more problems with
these patches (I see two in this thread, and I saw so many before).


Regards

--
Julian Anastasov <[email protected]>

2001-04-18 22:06:19

by Sampsa Ranta

[permalink] [raw]
Subject: Re: ARP responses broken!

On Wed, 18 Apr 2001, Julian Anastasov wrote:

>
> Hello,
>
> Sampsa Ranta wrote:
>
> > 23:38:25.278848 > arp who-has 194.29.192.38 tell 194.29.192.10 (0:50:da:82:ae:9f)
> > 23:38:25.278988 < arp reply 194.29.192.38 is-at 0:1:2:dc:d2:64 (0:50:da:82:ae:9f)
> > 23:38:25.279009 < arp reply 194.29.192.38 is-at 0:1:2:dc:d2:6c (0:50:da:82:ae:9f)
> >
> > The second one is the valid one, but both interfaces seem to answer to the
> > broadcasted packet with their own ARP addresses.
>
> arp_filter is not broken, it is simply not for your setup.
> It depends what you want to achieve by defining two IP addresses in
> different interfaces. Considering the fact you have two addresses
> in one subnet you need the incoming traffic to come from the two
> interfaces. In this case you need "hidden". For the outgoing traffic:
> it is controlled only from the routing.
>
> While in your setup arp_filter and rp_filter will ARP answer
> from one card only, for the both addresses, hidden will answer from the
> both cards, "correctly" in your eyes. Use arp_filter for different
> nets only, i.e. when the ARP probes come from different nets in your
> routing universe. hidden does not depend on nets/subnets. But may
> be there are exceptions I'm missing and the other guys can correct me.

Yes, I wan't that other routers only see the MAC address of the interface
I assigned the IP address for if someone asks it by ARP. I also control
outgoing traffic with routing. But how am I supposed to do this in 2.4
enviroment?

I also would want outgoing traffic to work from either of the interfaces,
not depending from which the incoming traffic did come, so outgoing ARP
requests would have the MAC address and IP address of the interface they
were asked from. For filtering IP traffic, the iptables is good enough so
I can filter just as I want, I don't need filtering to happen by routing
table at the moment.

Yeah, and I am aware that routing protocols won't make this very much
easier but OSPF for example communicates only via one interface per subnet
at a time so I can use the other for my static configuration. Zebra
also wants the packets to come from the interface it expects them to, and
that is the one it is assigned.

This is what my patch was supposed to deal with, but there was something
missing from it, right?

As far as I consider routing, the routers only determinate next-hop,
routing thinks the next-hop is IP. But actually the IP number given to
routing is nothing more than a key that is passed to ARP layer to
determinate the layer 2 address. And when I don't want to make stupid
subnets when I actually can work under one network, I would like to be
able to "mark" interface by giving it IP and when I point something to
that specific IP from other host, I would like to have the traffic to be
go via that interface. If I wanted it to go via two interfaces, I would
need to deal with load balancing issues on both ends, randomly taken
interface from ARP layer is nothing I call load balancing, and the
interface could even be the same to all.

Yeah, I could do this by defining ARP addresses by hand but I would rather
not do that because there are two many routers involved.

Btw. Kernel sets up two routes to the subnet, which one is selected and
should the other one be deleted?

Thanks,
Sampsa Ranta
[email protected]

2001-04-18 22:15:25

by Sampsa Ranta

[permalink] [raw]
Subject: Re: Broken ARP (was Re: ARP responses broken!)

On Wed, 18 Apr 2001, Julian Anastasov wrote:

>
> Hello,
>
> Sampsa Ranta wrote:
>
> > The code I used to do the trick at my network was as simple as this,
> > in function arp_rcv, the problem is ip_dev_find that does know if there
> > are other devices with same IP address.
>
> I don't think this is your problem. You patch is not correct.
> In fact, you implement the same function as in "hidden" but you are
> missing some things. Please, read the "hidden" flag description in
> the kernel docs. You must solve the case where your ARP probes are sent
> always through one device due to your routing (this is out traffic,
> yes?). These probes soon or later will cause you problems because
> they change the entry in the remote hosts' ARP tables. You so carefully
> tried to advertise the address on specific interface and now the other
> hosts again talk to one card only.
>
> who-has 194.29.192.1 tell 194.29.192.38
>
> and your are dead :)
>
> So, please try "hidden" before going into more problems with
> these patches (I see two in this thread, and I saw so many before).

The problem is there is no hidden in 2.4 and my patch did not modify
behavior of sending ARP requests and receiving the replies, are there
issues to be solved in these cases, too?

- Sampsa Ranta
[email protected]

2001-04-18 22:22:02

by Sampsa Ranta

[permalink] [raw]
Subject: Re: Broken ARP (was Re: ARP responses broken!)

On Wed, 18 Apr 2001, Julian Anastasov wrote:

>
> Hello,
>
> Sampsa Ranta wrote:
>
> > The code I used to do the trick at my network was as simple as this,
> > in function arp_rcv, the problem is ip_dev_find that does know if there
> > are other devices with same IP address.
>
> I don't think this is your problem. You patch is not correct.
> In fact, you implement the same function as in "hidden" but you are
> missing some things. Please, read the "hidden" flag description in
> the kernel docs. You must solve the case where your ARP probes are sent
> always through one device due to your routing (this is out traffic,
> yes?). These probes soon or later will cause you problems because
> they change the entry in the remote hosts' ARP tables. You so carefully
> tried to advertise the address on specific interface and now the other
> hosts again talk to one card only.
>
> who-has 194.29.192.1 tell 194.29.192.38
>
> and your are dead :)
>
> So, please try "hidden" before going into more problems with
> these patches (I see two in this thread, and I saw so many before).

Are you referring to the arp_solicit()?

if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL)
saddr = skb->nh.iph->saddr;
else
saddr = inet_select_addr(dev, target, RT_SCOPE_LINK);

Why is this hidden flag removed in 2.4 series with replacing no
functionality that would help to solve the problems?

- Sampsa Ranta
[email protected]

2001-04-18 22:34:42

by Sampsa Ranta

[permalink] [raw]
Subject: Re: Broken ARP (was Re: ARP responses broken!)


I have a rathar strange way to first ask and then try to find the answer
on my own. But what I found was:

from http://www.appwatch.com/lists/linux-kernel/Week-of-Mon-20010122/018588.html
> > am most curious about is how it ending up being removed from the kernel
> > in the first place. It must have been a decision that someone made.
> > Either, we don't need that any more since we can do it this way, or
> > we'll take it out since nobody uses it.
>
> It was only submitted to 2.2 a few months ago (=years after 2.3 branched), but
> never added to 2.4.

So I wonder if this hidden feature or alike should be brought to 2.4 tree
also?

- Sampsa Ranta
[email protected]

2001-04-18 22:47:05

by Julian Anastasov

[permalink] [raw]
Subject: Re: ARP responses broken!


Hello,

On Thu, 19 Apr 2001, Sampsa Ranta wrote:

> Yes, I wan't that other routers only see the MAC address of the interface
> I assigned the IP address for if someone asks it by ARP. I also control
> outgoing traffic with routing. But how am I supposed to do this in 2.4
> enviroment?

Eric Weigle exactly pointed out the solutions and the links.
There is still no standard support in the 2.4 kernel for your problem
but you can try http://www.linuxvirtualserver.org/hidden-2.3.41-1.diff
You can see some fails while applying but don't worry. You can try
the same 2.4.2 patch (I should make actual diff one day, for now may be
I'm trying to determine how long can live this patch) from
http://marc.theaimsgroup.com/?l=linux-virtual-server&m=98286657511673&w=2
This functionality is same as in Linux 2.2

> I also would want outgoing traffic to work from either of the interfaces,
> not depending from which the incoming traffic did come, so outgoing ARP
> requests would have the MAC address and IP address of the interface they
> were asked from. For filtering IP traffic, the iptables is good enough so

With "hidden" you will see that your ARP probes can go through
any device without problem, of course, if the targets listen everywhere.
If an address can't be announced from the outgoing device, it is replaced
with valid source address in the ARP probe. You still can need some
source routing rules if you want the other IP traffic with specific
source address to go through specific interface. In other case (routes
by destination, for example), your ARP probes and the IP traffic will
go through one device only.

> I can filter just as I want, I don't need filtering to happen by routing
> table at the moment.

Fine :) Because the "hidden" usage assumes secure environment
in some cases.

> Yeah, I could do this by defining ARP addresses by hand but I would rather
> not do that because there are two many routers involved.

The ARP usage is recommended :)

> Btw. Kernel sets up two routes to the subnet, which one is selected and
> should the other one be deleted?

You can determine it with:

ip route get DESTINATION_IP

Only the displayed route will be selected but you don't need
to delete the other.

> Thanks,
> Sampsa Ranta
> [email protected]


Regards

--
Julian Anastasov <[email protected]>

2001-04-18 23:11:47

by Julian Anastasov

[permalink] [raw]
Subject: Re: Broken ARP (was Re: ARP responses broken!)


Hello,

On Thu, 19 Apr 2001, Sampsa Ranta wrote:

> So I wonder if this hidden feature or alike should be brought to 2.4 tree
> also?

The three flags that can control the ARP behavior in 2.2
(arp_filter, hidden and rp_filter) cover almost everything without
breaking any RFC826 rule. You can always find the missing from Linux 2.4
functionality in the LVS site, where it is really used.

I hope the following posts/threads explain almost everything
related :)

http://marc.theaimsgroup.com/?l=linux-kernel&m=98032243112274&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=98042063530177&w=2
http://marc.theaimsgroup.com/?t=98019795800013&w=2&r=1
http://marc.theaimsgroup.com/?t=95743539800002&w=2&r=1


Regards

--
Julian Anastasov <[email protected]>