2001-11-28 17:45:57

by Adrian Daminato

[permalink] [raw]
Subject: hiding arp for server farms

Okay, I've seen similar posts to this, but none of them provide a solution that
I can use.

I'm running several 2.2 machines behind a Radware load balancer, which uses
something called "local triangulation". Basically the Radware responds to ARP
requests for the IP of the farm, passes the packet to one of the servers, and
the server responds directly to the client. Each server has an aliased
interface on the loopback for the IP of the farm, and
/proc/sys/net/ipv4/conf/all/hidden and lo/hidden are set to 1. That works,
great, no problems.

Now, introduce an unpatched 2.4.x kernel. The hidden option no longer exists,
and for ease of operating a production environment, we prefer to use stock
kernels straight from kernel.org, no patches at all. I've tried many different
suggestion from the list:

1) ifconfig eth0 -arp
We have over 60 servers on the subnet these farms are on, and they need to
be able to communicate with each other. When I do this, I can't talk to other
servers on the network, and keeping an /etc/ethers file up to date is a daunting
task, and not practical.

2) arp_filter
I tried using it in a couple of ways, but there doesn't appear to be very
good documentation. I was hoping this would provide the same functionality of
the hidden in the 2.2 kernels for our current setup, but it doesn't appear to

3) I even tried adding the 'hidden' patch available, to put the hidden
functionality back in the 2.4.x kernel (currently I'm testing using a 2.4.9
kernel). It doesn't appear to work properly either, hosts on the local network
can't ping the server farm, and hosts outside the network although able to ping
the server farm, cannot ping the real IP of the host. It's kind of a weird
problem.

Is there any way to have this work on an unpatched 2.4.x kernel? Any
documentation/examples for arp_filter, how it works, how it can be implemented
for this?

Any help would be appreciated. Thanks.
--
Adrian Daminato
Tucows International Corp.
http://www.tucows.com
Tel: (416) 535-0123
Fax: (416) 531-5584

Beauty awakens the soul to act.
- Dante


2001-11-28 18:19:52

by Andi Kleen

[permalink] [raw]
Subject: Re: hiding arp for server farms

Adrian Daminato <[email protected]> writes:

> Now, introduce an unpatched 2.4.x kernel. The hidden option no longer exists,
> and for ease of operating a production environment, we prefer to use stock
> kernels straight from kernel.org, no patches at all. I've tried many different
> suggestion from the list:
>
> 1) ifconfig eth0 -arp

It'll make the stack no put any mac addresses into your packets,
which is likely not what you want.

>
> Is there any way to have this work on an unpatched 2.4.x kernel? Any
> documentation/examples for arp_filter, how it works, how it can be implemented
> for this?

arp_filter was not really designed to fix such a br^wweird setup.

It is possible to do it but a bit ugly. Basically you have to express a policy
filter rule/route that matches the outgoing ARP, but not the data and make the arp
route a blackhole route. The kernel unfortunately has no special key to select
ARP, so it has to be expressed in some other way (e.g. mark rules etc.), which is
usually possible, but ugly.

Your problems in (3) is that you asked for ARP to be turned off which
obviously breaks things if noone else (like your load balancing monstrosity) does
the ARP for you. IIRC the hidden guys usually work around this by using a
separate hidden virtual interface and only use that for load balancing purposes.
In the end it gets similarly ugly as the arp_filter setup.

-Andi

P.S.: I would not recommend 2.4.9 unpatched for any production setup.

2001-11-28 21:12:54

by ZipKid

[permalink] [raw]
Subject: Re: hiding arp for server farms

Adrian Daminato wrote:

>Okay, I've seen similar posts to this, but none of them provide a solution that
>I can use.
>
>I'm running several 2.2 machines behind a Radware load balancer, which uses
>something called "local triangulation". Basically the Radware responds to ARP
>requests for the IP of the farm, passes the packet to one of the servers, and
>the server responds directly to the client. Each server has an aliased
>interface on the loopback for the IP of the farm, and
>/proc/sys/net/ipv4/conf/all/hidden and lo/hidden are set to 1. That works,
>great, no problems.
>
>Now, introduce an unpatched 2.4.x kernel. The hidden option no longer exists,
>and for ease of operating a production environment, we prefer to use stock
>kernels straight from kernel.org, no patches at all. I've tried many different
>suggestion from the list:
>
>1) ifconfig eth0 -arp
> We have over 60 servers on the subnet these farms are on, and they need to
>be able to communicate with each other. When I do this, I can't talk to other
>servers on the network, and keeping an /etc/ethers file up to date is a daunting
>task, and not practical.
>
>2) arp_filter
> I tried using it in a couple of ways, but there doesn't appear to be very
>

I have tested this setup and had the same problems on a 2.4.12 kernel. I
tried out a few things
and could not resolve this issue. Fortunately for me the client is
running solaris and that does
not have this bug.
Sorry but I have no solution...

Stefan - ZipKid - Goethals


2001-11-29 00:07:03

by Julian Anastasov

[permalink] [raw]
Subject: Re: hiding arp for server farms


Hello,

Adrian Daminato wrote:

> 3) I even tried adding the 'hidden' patch available, to put the hidden
> functionality back in the 2.4.x kernel (currently I'm testing using a 2.4.9
> kernel). It doesn't appear to work properly either, hosts on the local network
> can't ping the server farm, and hosts outside the network although able to ping
> the server farm, cannot ping the real IP of the host. It's kind of a weird
> problem.

As this thread becomes too large I'm appending some
URLs. I don't know what patches you are using and what settings
you have. If you still have problems feel free to contact me directly.
The usage is simple: mark device "lo" as hidden and put there only
local addresses that must not advertised.

The "hidden" device's home page:
http://www.linuxvirtualserver.org/~julian/

Doc files:
http://www.linuxvirtualserver.org/~julian/hidden.txt
http://www.linuxvirtualserver.org/docs/arp.html

Regards

--
Julian Anastasov <[email protected]>