I've encountered what I believe to be an arp bug. Here's the situation:
I have quite a number of boxes, connected together via a number of networks.
(It's a routing testbed) In this case, there are two boxes in question, box A
and box B. Here's the relevent portion of the config of both boxes:
Box A: (netbsd)
ex0: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500
address: 00:b0:d0:99:e6:ab
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 10.131.10.22 netmask 0xff800000 broadcast 10.255.255.255
inet6 fe80::2b0:d0ff:fe99:e6ab%ex0 prefixlen 64 scopeid 0x5
inet6 fec0:0:0:30a::16 prefixlen 64
iso 47.0.5.80.ff.ff.0.0.0.6.0.0.0.0.0.a.83.a.16.0
netmask ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.0.0.0
fxp2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
address: 00:02:b3:03:4f:05
media: Ethernet autoselect (100baseTX)
status: active
inet 10.3.37.22 netmask 0xffffff80 broadcast 10.3.37.127
inet6 fe80::202:b3ff:fe03:4f05%fxp2 prefixlen 64 scopeid 0x3
inet6 fec0:0:0:325::16 prefixlen 64
iso 47.0.5.80.ff.ff.0.0.0.4.0.0.0.0.0.a.3.21.16.0
netmask ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.0.0.0
Box B: (linux 2.4.17)
eth0 Link encap:Ethernet HWaddr 00:B0:D0:99:E5:E6
inet addr:10.131.10.13 Bcast:10.255.255.255 Mask:255.128.0.0
inet6 addr: fe80::2b0:d0ff:fe99:e5e6/10 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7325 errors:0 dropped:0 overruns:1 frame:0
TX packets:416 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
Interrupt:5 Base address:0xe880
eth4 Link encap:Ethernet HWaddr 00:02:B3:03:4C:EE
inet addr:10.3.37.13 Bcast:10.3.37.127 Mask:255.255.255.128
inet6 addr: fe80::202:b3ff:fe03:4cee/10 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
Interrupt:11 Base address:0x6000
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
10.3.37.0 0.0.0.0 255.255.255.128 U 40 0 0 eth4
0.0.0.0 10.254.10.1 0.0.0.0 UG 40 0 0 eth0
If I now change the default route on box B from 10.254.10.1 to 10.3.37.22
(using route del/route add), then Box B sends the following arp, captured via
tcpdump on box A:
17:49:12.393126 arp who-has 10.3.37.22 tell 10.131.10.13
This triggers a bug in NetBSD, which is how I found out about this, but should
that be "tell 10.3.37.13"? If it was, this would work. As near as I can
tell, Linux does this type of thing whenever I change my default route from
it's initial one to one on another interface. This bug occures in (at least)
2.4.17, 2.4.5, and 2.2.20, so it's probably in all versions.
Thanks,
Daniel
---
Recursion n.:
See Recursion.
-- Random Shack Data Processing Dictionary
Hi Daniel,
I've seen a similar bug in 2.2.19 (and a lot of 2.2-s, I've not checked it in
the 2.4 series).
I have two boxes both connected to the same two nets (tr and eth in my case,
but it does not matter). A third machine is only connected to the ethernet.
--------------------------- 10.1.0.0 (tr)
|.1 |.2
BOX1 BOX2 BOX3
|.1 |.2 |.3
--------------------------- 10.2.0.0 (eth)
BOX3 has its default route set at 10.2.0.1 (BOX1). BOX3 opens a TCP connection
to 10.1.0.2 (BOX2's tr address). This connection setup packet goes thru
BOX1. BOX2 obviously answers thru the shorter path, directly over eth. BOX2
sends out a bogus arp query during this reply.
The problem lies in the ARP code itself. BOX2 fills out the arp query with the
following info:
- target: BOX3 IP address (correct)
- self: BOX2 eth MAC address (correct)
- self: BOX2 tr IP address (incorrect)
The arp code only checks, if the target IP address is reachable directly. If
it is, it takes its own IP address from the connection setup packet.
In my case BOX1 is a NetWare, which caches all the arp queries it sees, which
produced a very interesting arp table in BOX1...
The attached patch solved my problems.
Regards,
- Bela
--- net/ipv4/arp.c.orig Fri Apr 20 01:56:40 2001
+++ net/ipv4/arp.c Wed Feb 23 23:37:33 2000
@@ -67,6 +67,8 @@
* now it is in net/core/neighbour.c.
* Julian Anastasov: "hidden" flag: hide the
* interface and don't reply for it
+ * Bela Szekeres : Fixed bogus source IP address in ARP
+ * requests (multi-homed hosts only)
*/
/* RFC1122 Status:
@@ -318,7 +320,7 @@
if (skb &&
(dev2 = ip_dev_find(skb->nh.iph->saddr)) != NULL &&
(in_dev2 = dev2->ip_ptr) != NULL &&
- !IN_DEV_HIDDEN(in_dev2))
+ !IN_DEV_HIDDEN(in_dev2) && (dev2==dev))
saddr = skb->nh.iph->saddr;
else
saddr = inet_select_addr(dev, target, RT_SCOPE_LINK);
On Mon, 7 Jan 2002, Daniel Gryniewicz wrote:
> I've encountered what I believe to be an arp bug. Here's the situation:
>
> I have quite a number of boxes, connected together via a number of networks.
> (It's a routing testbed) In this case, there are two boxes in question, box A
> and box B. Here's the relevent portion of the config of both boxes:
>
> Box A: (netbsd)
> ex0: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> address: 00:b0:d0:99:e6:ab
> media: Ethernet autoselect (100baseTX full-duplex)
> status: active
> inet 10.131.10.22 netmask 0xff800000 broadcast 10.255.255.255
> inet6 fe80::2b0:d0ff:fe99:e6ab%ex0 prefixlen 64 scopeid 0x5
> inet6 fec0:0:0:30a::16 prefixlen 64
> iso 47.0.5.80.ff.ff.0.0.0.6.0.0.0.0.0.a.83.a.16.0
> netmask ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.0.0.0
> fxp2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
> address: 00:02:b3:03:4f:05
> media: Ethernet autoselect (100baseTX)
> status: active
> inet 10.3.37.22 netmask 0xffffff80 broadcast 10.3.37.127
> inet6 fe80::202:b3ff:fe03:4f05%fxp2 prefixlen 64 scopeid 0x3
> inet6 fec0:0:0:325::16 prefixlen 64
> iso 47.0.5.80.ff.ff.0.0.0.4.0.0.0.0.0.a.3.21.16.0
> netmask ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.ff.0.0.0
>
> Box B: (linux 2.4.17)
> eth0 Link encap:Ethernet HWaddr 00:B0:D0:99:E5:E6
> inet addr:10.131.10.13 Bcast:10.255.255.255 Mask:255.128.0.0
> inet6 addr: fe80::2b0:d0ff:fe99:e5e6/10 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:7325 errors:0 dropped:0 overruns:1 frame:0
> TX packets:416 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:100
> Interrupt:5 Base address:0xe880
>
> eth4 Link encap:Ethernet HWaddr 00:02:B3:03:4C:EE
> inet addr:10.3.37.13 Bcast:10.3.37.127 Mask:255.255.255.128
> inet6 addr: fe80::202:b3ff:fe03:4cee/10 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:100
> Interrupt:11 Base address:0x6000
>
> Kernel IP routing table
> Destination Gateway Genmask Flags MSS Window irtt Iface
> 10.3.37.0 0.0.0.0 255.255.255.128 U 40 0 0 eth4
> 0.0.0.0 10.254.10.1 0.0.0.0 UG 40 0 0 eth0
>
> If I now change the default route on box B from 10.254.10.1 to 10.3.37.22
> (using route del/route add), then Box B sends the following arp, captured via
> tcpdump on box A:
>
> 17:49:12.393126 arp who-has 10.3.37.22 tell 10.131.10.13
>
> This triggers a bug in NetBSD, which is how I found out about this, but should
> that be "tell 10.3.37.13"? If it was, this would work. As near as I can
> tell, Linux does this type of thing whenever I change my default route from
> it's initial one to one on another interface. This bug occures in (at least)
> 2.4.17, 2.4.5, and 2.2.20, so it's probably in all versions.
>
> Thanks,
> Daniel
> ---
> Recursion n.:
> See Recursion.
> -- Random Shack Data Processing Dictionary
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
Hello,
Szekeres Bela wrote:
> I've seen a similar bug in 2.2.19 (and a lot of 2.2-s, I've not checked it in
> the 2.4 series).
In your case
> --------------------------- 10.1.0.0 (tr)
> |.1 |.2
> BOX1 BOX2 BOX3
> |.1 |.2 |.3
> --------------------------- 10.2.0.0 (eth)
it is a feature, may be not fully utilized without using
alternative link routes. At least, I don't know for any standard
that is not followed from this feature. IMO, the following fix
is more correct:
01_arp_prefsrc-2.2.19-4.diff
or
01_arp_prefsrc-2.4.12-5.diff
from
http://www.linuxvirtualserver.org/~julian/#routes
its main usage is for devices attached to same medium but should
work for your case too. It always uses the preferred source IP
to the target because any local IP addresses announced in our probes
are updated in the remote ARP caches and in some cases it is not
the desired behavior. Even the Linux's rp_filter protection can't
avoid the ARP cache entry update. It could be a "problem" when BOX2 uses
devices attached to same medium. Other users are happy with such
feature because if eth0 fails may be eth1 still have link to the
same hub, for example.
> In my case BOX1 is a NetWare, which caches all the arp queries it sees, which
> produced a very interesting arp table in BOX1...
It seems NetWare is different than Linux in this handling.
Linux does not create new entries in the ARP cache. But even if they
are created (in NetWare) I can't believe they are used (I hope
the ARP entry is looked up by IP and outdev, not only by IP).
But note that:
- BOX2 is right to announce 10.1.0.2 through the eth device because
it can accept traffic to 10.1.0.2 from any device. At least, it
depends on the sender's address and the rp_filter values in BOX2.
You should be able to talk with 10.1.0.2 through the eth device
from any host attached to the same eth device.
- BOX1 can decide through which device to talk with BOX2, may be
in your case it is set to accept traffic from the both devices
but to send only through one of them (similar to rp_filter=0 in
Linux). You need alternative link routes to allow sending through
the both devices but may be that is not your goal.
So, where in fact is the problem without applying any patches?
May be there is no problem? Or NetWare has a problem to create
new ARP entries and even to use them?
As for the Daniel's problem I don't understand where is
the link route created for the 10.128.0.0/9 network ? Or may be the key
is to have the right preferred source IP address for all routes?:
ip route add ... src PREFSRC
Regards
--
Julian Anastasov <[email protected]>
[My apologies for getting into this thread rather late, but I just
came across this on a network I'm working with which is all on the
same switch/VLAN... I had 2 network cards per machine and was
puzzled why traffic destined for one interface went to the other]
Julian Anastasov <[email protected]> wrote:
> Szekeres Bela wrote:
>
> > I've seen a similar bug in 2.2.19 (and a lot of 2.2-s, I've not checked it in
> > the 2.4 series).
>
> In your case
>
> > --------------------------- 10.1.0.0 (tr)
> > |.1 |.2
> > BOX1 BOX2 BOX3
> > |.1 |.2 |.3
> > --------------------------- 10.2.0.0 (eth)
>
> it is a feature, may be not fully utilized without using
> alternative link routes. At least, I don't know for any standard
> that is not followed from this feature. IMO, the following fix
> is more correct:
>
> 01_arp_prefsrc-2.2.19-4.diff
> or
> 01_arp_prefsrc-2.4.12-5.diff
> from
> http://www.linuxvirtualserver.org/~julian/#routes
>
> its main usage is for devices attached to same medium but should
> work for your case too. It always uses the preferred source IP
> to the target because any local IP addresses announced in our probes
> are updated in the remote ARP caches and in some cases it is not
> the desired behavior. Even the Linux's rp_filter protection can't
> avoid the ARP cache entry update. It could be a "problem" when BOX2 uses
> devices attached to same medium. Other users are happy with such
> feature because if eth0 fails may be eth1 still have link to the
> same hub, for example.
This is *not* a feature, it is a major security hole.
Let's say you have a firewall running Linux. Oops, I can spoof the
external interface to accept traffic as if it's the internal one.
I may have screwed up my test, but it looks like it just worked on
my RedHat 7.2 based most recent 2.4.9-21 kernel on my firewall!!
This is Not Good.
You can imagine, I just added the proc command to shut it off to
my setup.
I'm not sure (haven't checked it yet), but I would guess that
there still may be a hole here (and in my config) where a machine
on the same network could craft packets that will get accepted
by my machine even though it doesn't respond to arps on that
interface any more.
I.e. the machine still may be accepting traffic destined for one
interface on another, even though it won't *advertise* that fact
any more.
If so, this is yet another bug which is dangerous in configs
like firewalls.
The fact that you have to manually turn this off is insane. I must
admit to being kind of angry that it was considered a "feature"
when discovered, and left in.
Am I the only one that saw this kind of scary hole?
These kind of "features" that cause weird behavior should always
be turned off by default, else corner cases like this sometimes
pop up.
Sorry about the mild flameage here, but this does make me wonder what
other "features" there are that might be exploitable.
--
Erich Stefan Boleyn <[email protected]> http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"
> Let's say you have a firewall running Linux. Oops, I can spoof the
> external interface to accept traffic as if it's the internal one.
ARP is irrelevant to security. You don't need the ARP layer to do any
attacks or routing at all. There are a million ways to get the mac
address of a box.
> I.e. the machine still may be accepting traffic destined for one
> interface on another, even though it won't *advertise* that fact
> any more.
Its supposed to accept any packet for that system. Thats correct behaviour
for the system.
Thats why you can have firewall rules. You'll find the standard Red Hat firewall
config tool sets up interface based rule sets. In fact in general rules should be
interface not address based.
Alan
Hello,
On Sat, 2 Mar 2002 [email protected] wrote:
> [My apologies for getting into this thread rather late, but I just
> came across this on a network I'm working with which is all on the
> same switch/VLAN... I had 2 network cards per machine and was
> puzzled why traffic destined for one interface went to the other]
If another host (spoofer) on the LAN is faster all
your devices lose the race.
> I.e. the machine still may be accepting traffic destined for one
> interface on another, even though it won't *advertise* that fact
> any more.
You mix two different issues:
1. ARP replying for same request through many devices
2. ARP probes advertising same source IP through many devices
Your statement is related to (1). There are some golden
rules on this issue:
- use rp_filter protection for all your external interface
- use different "external" interfaces for the different external
networks if you want to differentiate the traffic from/to them
and protect them with rp_filter
- use rp_filter protection for all "internal" interfaces
attached to same medium (hub) as the "external" interfaces
- no need to use rp_filter protection for the other "internal"
interfaces, it is recommended, though
- if you want to differentiate traffic by protocol or ports
use firewall rules (IPSec, stateful conntracking, etc)
As for (2) I don't see how this can be remotely
exploited but my opinion is that it should be fixed.
The current behavior is still valid: if you can talk
from one local IP to some remote IP through one device
then you should allow the reverse traffic to work. With
my fix we just want to reduce the set of the local IPs
that can be used for ARP announcement. It is only a local
problem - ARP uses scope link routes. Make sure (1) does
not lead to problems caused remotely.
> Am I the only one that saw this kind of scary hole?
The users prefer protection provided from firewall
rules, even for address spoofing, a matter of taste.
Regards
--
Julian Anastasov <[email protected]>
Alan Cox <[email protected]> wrote:
> > Let's say you have a firewall running Linux. Oops, I can spoof the
> > external interface to accept traffic as if it's the internal one.
>
> ARP is irrelevant to security. You don't need the ARP layer to do any
> attacks or routing at all. There are a million ways to get the mac
> address of a box.
Oh, agreed.
The security hole is in the complicity of the other layers in this, in
my mind. The ARP issue is mainly a bug that causes unexpected behavior
given this hole.
For example, in the testing situation that led me to looking into this
more, I was running 2 NICs, one 100Mbit and one Gigabit, in
each of 2 machines. The 100Mbit NICs were on normal DHCP for my
subnet, and the Gigabit NICs were on the same physical network switch,
and I didn't set up a VLAN because it didn't seem to matter and was
a bit of a pain. The traffic in one direction of my tests ended up
going into the 100Mbit inteface on the destination machine and being
throttled down to 100Mbit.
More below.
> > I.e. the machine still may be accepting traffic destined for one
> > interface on another, even though it won't *advertise* that fact
> > any more.
>
> Its supposed to accept any packet for that system. Thats correct
> behaviour for the system.
>
> Thats why you can have firewall rules. You'll find the standard Red
> Hat firewall config tool sets up interface based rule sets. In fact,
> in general rules should be interface not address based.
I would argue that this is a work-around to a problem, serves no
useful purpose, and in general this is violating the "principle of
least surprise".
To put it clearly, I am arguing that the two "features":
-- ARP responding on other interfaces.
-- IP stack accepting packets destined for one interface on another.
...should at least be disabled by default because:
1) It's a security hole as configured by default. Yet another
thing that confuses end-users using Linux who wouldn't expect
this behavior.
2) In my example above (and in fact any case of very asymmetric
bandwidth) it ends up causing weird and highly suboptimal
misbehavior.
3) I tested the idea of it being a "failover" kind of thing, as
suggested by someone else in the earlier thread, and it
didn't work because, at least for TCP, the TCP acks and
such had to go out of the original interface. I.e. it ends
up doing asymmetric routing. The outbound packets go from
one interface, and the inbound packets come into the other.
Removing the connection to the "correct" card killed the
connection. UDP would still work given no response was
necessary, admittedly. But a lot of UDP protocols use at
least some kind of response, so you're dead there too.
4) Given the people who want this consider it a "feature" know
about it or seek it out because they want something like it,
why not burden *them* with turning it on rather than burdening
those who don't know or care with the hill to climb of having
to discover it's existence and disabling it because it causes
strange effects/holes?
Can you give me an argument for why these should be present? (like
some kind of use for it?)
Is this a UNIX semantics legacy thing I'm not familiar with?? but
I'm pretty sure FreeBSD for example doesn't do this... at least not
the arp thing, because I saw warning messages from my FreeBSD boxen
about these weird arps.
I would think that making the IP stack, for each MAC/interface path
on reception, just check against the exact expected input address,
would actually be a performance improvement on machines with multiple
NICs.
As to the failover concept, Linux already has channel bonding, which
is the right way to do this and actually works when you take one
of them offline/unplug one. There are a few unusual kind of hacks
you can't do with channel bonding, say like having a card pretend it's
another one in some ways but still think it's part of another IP
interface... but that doesn't really happen in the current situation
anyway, so I again don't see any kind of argument that this is
actually useful in any way.
--
Erich Stefan Boleyn <[email protected]> http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"
> I would argue that this is a work-around to a problem, serves no
> useful purpose, and in general this is violating the "principle of
> least surprise".
I strongly disagree. The RFCs _are_ expected behaviour (with the odd
exception like URG which BSD redefined by force)
> 2) In my example above (and in fact any case of very asymmetric
> bandwidth) it ends up causing weird and highly suboptimal
> misbehavior.
Because you ran two different speed networks over the same cable without
any seperation ?
> Can you give me an argument for why these should be present? (like
> some kind of use for it?)
Internet standards. Along with the fact your perceived safety isnt there
in the first place. Consider a source routed frame. Consider the fact
most of your daemons bind to INADDR_ANY. It would be a false and misleading
appearance of security at best.
If you want a firewall use firewall rules. If an end user is not sure how to
set up a basic firewall they can run tools like gnome-lokkit and answer simple
questions.
> the arp thing, because I saw warning messages from my FreeBSD boxen
> about these weird arps.
FreeBSD binds arp to the specific interface, but accepts packets on any
(from memory). The warnings it issues about the ARPs are a bug because the
RFC's make no assertions about ARP replies being host or address based.
> on reception, just check against the exact expected input address,
> would actually be a performance improvement on machines with multiple
> NICs.
Hardly. You can have multiple addresses per nic anyway so its all the same
routing hashes. You can also have multiple nics with the same address. Its
already many<->many.
Alan
> As for (2) I don't see how this can be remotely
> exploited but my opinion is that it should be fixed.
"Fixed" ? How do you fix standards compliant behaviour ?
Alan Cox wrote:
>> As for (2) I don't see how this can be remotely
>>exploited but my opinion is that it should be fixed.
>>
>
> "Fixed" ? How do you fix standards compliant behaviour ?
Alan, Alan...you just have to learn to Innovate(tm) :)
--
Ben Greear <[email protected]> <Ben_Greear AT excite.com>
President of Candela Technologies Inc http://www.candelatech.com
ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear
Alan Cox <[email protected]> wrote:
> > I would argue that this is a work-around to a problem, serves no
> > useful purpose, and in general this is violating the "principle of
> > least surprise".
>
> I strongly disagree. The RFCs _are_ expected behaviour (with the odd
> exception like URG which BSD redefined by force)
Hmm.
My general contention is that the system should, by default, behave as
non-experts would expect, but this might be a point where we can't
agree.
It is, unfortunately, the cardinal rule when designing any usable
interfaces. I reference Donald Norman's "The Design of Everyday
Things". But I digress.
> > 2) In my example above (and in fact any case of very asymmetric
> > bandwidth) it ends up causing weird and highly suboptimal
> > misbehavior.
>
> Because you ran two different speed networks over the same cable without
> any seperation ?
I'll give up on this one relatively easily (though it was a switch,
so this is relevant)... but a more pertinent example would be
if you have IP aliasing going on with 2 cards on the same network
for failover capability, and they might be asymmetric. Again,
the expectation is that the "primary" one would be what is mainly
used.
> > Can you give me an argument for why these should be present? (like
> > some kind of use for it?)
>
> Internet standards. Along with the fact your perceived safety isnt there
> in the first place. Consider a source routed frame. Consider the fact
> most of your daemons bind to INADDR_ANY. It would be a false and misleading
> appearance of security at best.
>
> If you want a firewall use firewall rules. If an end user is not sure
> how to set up a basic firewall they can run tools like gnome-lokkit
> and answer simple questions.
The firewall rules aren't as fine-grained as you're implying. They
only accept packets or not, with the assumption that all programs
on the box are equally untrusted.
A good example is, you want a trivial high-security firewall, but
you want it to be a DNS and email server, knowing you only have
to be really careful about those 2 programs.
If the TCP/IP stack filters itself by what you've assigned it, then
you have the flexibility of using things like tcp wrappers (for
example) to allow one program but not another to accept things
from particular ports as well. Though tcp wrappers have their
own host of problems.
> > on reception, just check against the exact expected input address,
> > would actually be a performance improvement on machines with multiple
> > NICs.
>
> Hardly. You can have multiple addresses per nic anyway so its all the same
> routing hashes. You can also have multiple nics with the same address. Its
> already many<->many.
OK.
--
Erich Stefan Boleyn <[email protected]> http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"
> My general contention is that the system should, by default, behave as
> non-experts would expect, but this might be a point where we can't
> agree.
It does 8) You plug it into the network, you select the defaults in the
installation menu, you run mozilla and up comes a web site. Anything beyond
that is not non-expert.
If you want to go into the innards of the routing/arp stuff you might want
to move this to [email protected] - thats the network folks list and many
of them don't read linux-kernel.
Alan
Alan Cox <[email protected]> wrote:
> > My general contention is that the system should, by default, behave as
> > non-experts would expect, but this might be a point where we can't
> > agree.
>
> It does 8) You plug it into the network, you select the defaults in the
> installation menu, you run mozilla and up comes a web site. Anything beyond
> that is not non-expert.
Heh.
I can't say we agree here, but kudos to you for having a sense of
humor about these kind of things. It's refreshing.
> If you want to go into the innards of the routing/arp stuff you might want
> to move this to [email protected] - thats the network folks list and many
> of them don't read linux-kernel.
I will, thanks much.
--
Erich Stefan Boleyn <[email protected]> http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"
Hello,
On Sat, 2 Mar 2002, Alan Cox wrote:
> > As for (2) I don't see how this can be remotely
> > exploited but my opinion is that it should be fixed.
>
> "Fixed" ? How do you fix standards compliant behaviour ?
I still didn't found a good reason to keep any relation
between the iph->saddr and the address announced in the ARP probe.
Announcing the prefsrc should work for any setup while the current
behavior causes some problems for setups with rp_filter protection
and interfaces attached to same hub. If you want to find the reason
for this, here it is:
By definining local IP and direct route through one device
with rp_filter protection we apply the following policy:
1. we can receive traffic from this directly attached network
only from this device (rp_filter=1)
2. we can receive traffic to our local IPs through all interfaces
where (1) is OK (according to the used different remote source IPs),
i.e. we can receive traffic to this local IP from different
remote hosts, part from different subnets.
Do you see the assymetry?
So, here comes the restriction:
We have restrictions according to the remote IPs in one direction
but don't have them in the reverse one. In short, one local IP
can receive traffic from one remote IP only from one device with
rp_filter protection while at the same time ARP can announce the
local IP through many devices by using different routes (we resolve
different targets with same local IP). Put this on shared medium - the
other end does not know what MAC on what device is in our end.
The solution:
Bind the ARP probes strictly to the route, no variations for
the announced IPs, something like to deduce the local_ip that
works for ip route get from target to local_ip iif outdev.
Which local_ip works? Of course, the prefsrc to target is one of
the possibilities.
Simple/Stupid example where the plain kernel fails:
Cross-subnet talks (the anti-spoofing is not considered
a problem), we just want BOX1 to use the both NICs, so
we use the rp_filter
.---eth0 --. .------eth0-BOX2
BOX1 HUB
`---eth1 --' `------eth0-BOX3
BOX1:
ifconfig eth0 192.168.0.1, rp_filter on eth0
ifconfig eth1 192.168.1.1, rp_filter on eth1
BOX2:
ifconfig eth0 192.168.0.2
BOX3:
ifconfig eth0 192.168.1.3
0.2 talks with 0.1 and has 0.1 in its ARP cache.
Start the first talks from 192.168.0.1 to 192.168.1.3
or from 1.3 to 0.1 and boom, 0.1 is announced through eth1. BOX2
updates its ARP cache with new MAC for 0.1. BOX2 later will send
traffic 0.2->0.1 to the eth1's MAC on BOX1 where the rp_filter
drops it. The stupid app that uses bind() to addr does not work.
The standards don't explain it, right? :) We can
search the word "hub" in RFCs such as: ARP, VRRP. We will
not find this word there. So, that was how comes the problem
with these features when used with many devices attached to
same hub.
So, what changes the patch:
instead of announcing 0.1 when resolving 1.3 we should announce
1.1 because this is the prefsrc when talking to 1.0/24. We know
that we can send to 1.3 traffic with any local IP but we better
to use the best IP for ARP according to the used device - don't
announce one local IP through many devices attached to same
medium - that is. The problems can come if we use same prefsrc for
routes on different devices but it is not recommended without
adding alternative link routes to the kernel, so there is no
problem with this patch. Uh, long explanation :)
Regards
--
Julian Anastasov <[email protected]>
> behavior causes some problems for setups with rp_filter protection
> and interfaces attached to same hub. If you want to find the reason
> for this, here it is:
rp_filter is an add on - not exactly default standards behaviour. If you
want to make the case that rp_filter = 2 means apply a both way rule then
I've personally no problem with that argument
Hello,
On Sat, 2 Mar 2002, Alan Cox wrote:
> > behavior causes some problems for setups with rp_filter protection
> > and interfaces attached to same hub. If you want to find the reason
> > for this, here it is:
>
> rp_filter is an add on - not exactly default standards behaviour. If you
> want to make the case that rp_filter = 2 means apply a both way rule then
> I've personally no problem with that argument
The rp_filter value of 2 is not support from Linux and
after reading the "5.3.8 Source Address Validation" paragraph
from rfc1812 it seems rp_filter 1 covers it. What exactly do
you mean by value of 2? Note that the remote box does not want to
spoof, it was directed from BOX1 to a wrong MAC where the traffic is
spoofed, the remote hosts are not guilty. They connect to the MAC we
provide by broadcasts.
To Erich, rfc1812, 5.3.8 Source Address Validation:
If this feature is implemented, it MUST be disabled by default
Regards
--
Julian Anastasov <[email protected]>
> > rp_filter is an add on - not exactly default standards behaviour. If you
> > want to make the case that rp_filter = 2 means apply a both way rule then
> > I've personally no problem with that argument
>
> The rp_filter value of 2 is not support from Linux and
Language confusion - "if you want to make the case" = "if you want to argue
that a value of rp_filter = 2 should in future (after you implement it) mean
apply a both way rule - then I agree)
I'm glad about your RFC1812 cite btw - the number of problems I've seen with
one of the distros defaulting to rp_filter = 1 was large.
Alan
On Sat, Mar 02, 2002 at 12:31:48PM -0800, [email protected] wrote:
> My general contention is that the system should, by default, behave as
> non-experts would expect, but this might be a point where we can't
> agree.
>
> It is, unfortunately, the cardinal rule when designing any usable
> interfaces. I reference Donald Norman's "The Design of Everyday
> Things". But I digress.
I must agree with Alan. Low level technical interfaces should
behave according to standards, and should follow a consistent logic
understood by experts in the field (even if it is difficult for the
beginner). If people try to push "usability" (and I'm as much a fan
of that book as you) onto kernel interfaces, we'll wade into a swamp
and never get out.
Such interfaces need not be exposed to ordinary users. Indeed, by
keeping the low-level layer simple and orthogonal, it becomes easier
to build multiple user-facing layers (for different purposes, or for
comparison at the same purpose). I think this principle is much
more powerful than the one you advance.
Andrew
>>"Fixed" ? How do you fix standards compliant behaviour ?
Wouldn't that be an example of "Embrace and Extend".
Karl
[followons after this should probably go to "[email protected]" and not
the kernel list]
Julian Anastasov <[email protected]> wrote:
> On Sat, 2 Mar 2002, Alan Cox wrote:
>
> > > behavior causes some problems for setups with rp_filter protection
> > > and interfaces attached to same hub. If you want to find the reason
> > > for this, here it is:
> >
> > rp_filter is an add on - not exactly default standards behaviour. If you
> > want to make the case that rp_filter = 2 means apply a both way rule then
> > I've personally no problem with that argument
>
> The rp_filter value of 2 is not support from Linux and
> after reading the "5.3.8 Source Address Validation" paragraph
> from rfc1812 it seems rp_filter 1 covers it. What exactly do
> you mean by value of 2? Note that the remote box does not want to
> spoof, it was directed from BOX1 to a wrong MAC where the traffic is
> spoofed, the remote hosts are not guilty. They connect to the MAC we
> provide by broadcasts.
>
> To Erich, rfc1812, 5.3.8 Source Address Validation:
>
> If this feature is implemented, it MUST be disabled by default
That's not what I was talking about. I'm talking about
Destination Address Validation based on the network you're getting
the packet from, before it's passed on up to the protocol layers
to the application.
This is, frankly, the most important part for determining if you
want to firewall off a packet from the wrong place. And if you
don't have routing set up in your machine, I still kind of think
it's an end-user box.
The fact that the routing layer and application layers of Linux's
TCP/IP stack are one and the same is a difficulty here which the
IP firewalling code in Linux does not fix. I.e. if I wanted to
have routing as well, but not accept any packets internally *not*
destined for my interface, I'm not sure how to specify it without
something like TCP wrappers, as sleazy as they can be, and they
don't offer this kind of capability in general as is.
I have been sniffing through the RFCs (and my trusty copy of TCP/IP
Illustrated Vol 1), and while no guru on these issues, have found
nothing specififying that a non-router must accept packets that
have the wrong destination address.
In fact, there are several places that make it clear that hosts
(as opposed to routers) must not forward packets. I assume they're
talking about a host not accidentally confusing routers by moving
packets from one network to another without having routes programmed,
but that language is *still* relevant in the case when a packet is
received by the wrong interface. It should not be arbitrarily
forwarding them internally to different interfaces without being
told to.
I think that is a standards-based reason to change the default
behavior. If you need proof, I can look up the part saying that
hosts must never forward packets unless explicitly commanded to,
and the interpretation that if they are asked to they must refuse
should be sufficient.
Also, if I'm understanding this language correctly, hosts and
routers are distinctly not the same, and RFC 1812 is talking about
routers. It's possible I'm not understanding the language correctly
though, as I'm not deeply familiar with the IP RFCs by any means.
--
Erich Stefan Boleyn <[email protected]> http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"
On Sat, Mar 02, 2002 at 04:21:24PM -0800, [email protected] wrote:
> The fact that the routing layer and application layers of Linux's
> TCP/IP stack are one and the same is a difficulty here which the
> IP firewalling code in Linux does not fix. I.e. if I wanted to
> have routing as well, but not accept any packets internally *not*
> destined for my interface, I'm not sure how to specify it without
> something like TCP wrappers, as sleazy as they can be, and they
> don't offer this kind of capability in general as is.
Linux 2.4 netfilter:
Incoming Outgoing
interface interface
----+------------------- FORWARD -----------------+------->
| ^
v |
INPUT -------------> Application -----------> OUTPUT
The names in capitals are the names of the tables. You can control
packets that the local machine sees completely independently of what
gets routed through the machine with a kernel supporting iptables
by adding the appropriate rules to the input and forward tables.
--
Russell King ([email protected]) The developer of ARM Linux
http://www.arm.linux.org.uk/personal/aboutme.html
Hello,
On Sat, 2 Mar 2002, Alan Cox wrote:
> Language confusion - "if you want to make the case" = "if you want to argue
> that a value of rp_filter = 2 should in future (after you implement it) mean
> apply a both way rule - then I agree)
Yes, the arp_prefsrc feature can depend on rp_filter||arp_filter
but I prefer to keep it independent. And there is an agreement on netdev
that all ARP filtering issues (including the problem with shared IPs
in clusters) should be fixed by maintaining ARP hooks for universal
filtering. Then even the arp_prefsrc feature can be implemented with
proper rules. TODO. Until then, this is a temp solution.
> Alan
Regards
--
Julian Anastasov <[email protected]>
Russell King <[email protected]> wrote:
> On Sat, Mar 02, 2002 at 04:21:24PM -0800, [email protected] wrote:
> > The fact that the routing layer and application layers of Linux's
> > TCP/IP stack are one and the same is a difficulty here which the
> > IP firewalling code in Linux does not fix. I.e. if I wanted to
> > have routing as well, but not accept any packets internally *not*
> > destined for my interface, I'm not sure how to specify it without
> > something like TCP wrappers, as sleazy as they can be, and they
> > don't offer this kind of capability in general as is.
>
> Linux 2.4 netfilter:
>
> Incoming Outgoing
> interface interface
> ----+------------------- FORWARD -----------------+------->
> | ^
> v |
> INPUT -------------> Application -----------> OUTPUT
>
> The names in capitals are the names of the tables. You can control
> packets that the local machine sees completely independently of what
> gets routed through the machine with a kernel supporting iptables
> by adding the appropriate rules to the input and forward tables.
Hmm. This would seem to be false in the RH 7.2 kernel 2.4.9-21
kernel I'm working with.
My IP masquerading rule (which claims to be in the "forward"
chain, with target "MASQ"), was blocked when I did input address
masking.
I.e. Yes, I actually tested this before posting.
If you're calling it a bug, then so be it. But the result would be
a bit better than how my Linux system works now.
--
Erich Stefan Boleyn <[email protected]> http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"
[email protected] wrote;
> > Linux 2.4 netfilter:
> >
> > Incoming Outgoing
> > interface interface
> > ----+------------------- FORWARD -----------------+------->
> > | ^
> > v |
> > INPUT -------------> Application -----------> OUTPUT
> >
> > The names in capitals are the names of the tables. You can control
> > packets that the local machine sees completely independently of what
> > gets routed through the machine with a kernel supporting iptables
> > by adding the appropriate rules to the input and forward tables.
>
> Hmm. This would seem to be false in the RH 7.2 kernel 2.4.9-21
> kernel I'm working with.
>
> My IP masquerading rule (which claims to be in the "forward"
> chain, with target "MASQ"), was blocked when I did input address
> masking.
>
> I.e. Yes, I actually tested this before posting.
>
> If you're calling it a bug, then so be it. But the result would be
> a bit better than how my Linux system works now.
Whoops, I am apparently using "ipchains" and not "iptables", and
didn't note the distinction.
Sorry about the spurious bug report here. :/
--
Erich Stefan Boleyn <[email protected]> http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"
At 11:58 AM 3/2/2002 -0800, [email protected] wrote:
>I would think that making the IP stack, for each MAC/interface path
>on reception, just check against the exact expected input address,
>would actually be a performance improvement on machines with multiple
>NICs.
Two words: Broadcast addresses.
--
Stevie-O
Real programmers use COPY CON PROGRAM.EXE
Andrew Pimlott <[email protected]> wrote:
> On Sat, Mar 02, 2002 at 12:31:48PM -0800, [email protected] wrote:
> > My general contention is that the system should, by default, behave as
> > non-experts would expect, but this might be a point where we can't
> > agree.
> >
> > It is, unfortunately, the cardinal rule when designing any usable
> > interfaces. I reference Donald Norman's "The Design of Everyday
> > Things". But I digress.
>
> I must agree with Alan. Low level technical interfaces should
> behave according to standards, and should follow a consistent logic
> understood by experts in the field (even if it is difficult for the
> beginner). If people try to push "usability" (and I'm as much a fan
> of that book as you) onto kernel interfaces, we'll wade into a swamp
> and never get out.
>
> Such interfaces need not be exposed to ordinary users. Indeed, by
> keeping the low-level layer simple and orthogonal, it becomes easier
> to build multiple user-facing layers (for different purposes, or for
> comparison at the same purpose). I think this principle is much
> more powerful than the one you advance.
You get no disagreement from me with the concept, and I'm following
a similar one in a system I'm working on now. (as to the standards
conformance, look at my most recent email a few messages ago on
that... we're arguing it out ;-).
The Linux kernel at this point, however, is not so easy/orthogonal
as you claim, I think. The question always arises: If there is
no other easy way to do something than modify your low-level
technical interface, then what do you do?
--
Erich Stefan Boleyn <[email protected]> http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"
At 04:49 PM 3/2/2002 -0800, [email protected] wrote:
>Whoops, I am apparently using "ipchains" and not "iptables", and
>didn't note the distinction.
>
>Sorry about the spurious bug report here. :/
Yeah, I use 2.2.19 on my server (2.4.x is the most unstable 'stable series' i've ever seen..). ipchains is like this:
Incoming Outgoing
interface interface
----+ +------->
| ^
v +------------> forward -----------+ |
input | |----> output
+----------> Application ---------+
I actually like it that way, it makes it easier to block things from the dsl ether (eth0):
ipchains -a input -i eth0 -d ! 66.92.237.176 -j DENY -l
With iptables i'd need that on both the INPUT *and* FORWARD rules...
--
Stevie-O
Real programmers use COPY CON PROGRAM.EXE
[email protected] wrote:
>Alan Cox <[email protected]> wrote:
>
>>I strongly disagree. The RFCs _are_ expected behaviour (with the odd
>>exception like URG which BSD redefined by force)
>>
>
>Hmm.
>
>My general contention is that the system should, by default, behave as
>non-experts would expect, but this might be a point where we can't
>agree.
>
Microsoft has done this and we see where that has gone...
>I'll give up on this one relatively easily (though it was a switch,
>so this is relevant)... but a more pertinent example would be
>if you have IP aliasing going on with 2 cards on the same network
>for failover capability, and they might be asymmetric. Again,
>the expectation is that the "primary" one would be what is mainly
>used.
>
All of your givens can be tightly controlled using specific routing
combined with a routing daemon. Add a touch of correct firewalling and
all is constrained just like you please.
>>If you want a firewall use firewall rules. If an end user is not sure
>>how to set up a basic firewall they can run tools like gnome-lokkit
>>and answer simple questions.
>>
>
>The firewall rules aren't as fine-grained as you're implying. They
>only accept packets or not, with the assumption that all programs
>on the box are equally untrusted.
>
>A good example is, you want a trivial high-security firewall, but
>you want it to be a DNS and email server, knowing you only have
>to be really careful about those 2 programs.
>
>If the TCP/IP stack filters itself by what you've assigned it, then
>you have the flexibility of using things like tcp wrappers (for
>example) to allow one program but not another to accept things
>from particular ports as well. Though tcp wrappers have their
>own host of problems.
>
Netfilter (iptables) is very extendable, it is certainly not limited to
if(src=1.2.3.4) pass(); By configuring your DNS server suitably right
off the start, you eliminate a large group of possible threats. Email
is simple, inbound is one port and security for that is entirely
userland. You don't have to be psychotic paranoid about either email or
DNS if you apply some of the basic tenents of security; configuration,
ownership, and permissions. Add chroot if applicable.
Firewall rules can be very fine grained. What's left can be dealt with
in userland. If it can't be handled, drop it. In the event of
overload, it's always better to drop packets than to pass ambiguous data.
David
On Sat, 2 Mar 2002, Stevie O wrote:
> ipchains -a input -i eth0 -d ! 66.92.237.176 -j DENY -l
>
> With iptables i'd need that on both the INPUT *and* FORWARD rules...
create a 'filter' chain, attach it to INPUT -i eth0 and FORWARD -i
eth0.
maintain your 'filter' chain and you have the same as ipchains, but
with more flexibility for when you /do/ want rules to apply only to
local applications.
--paulj