Hi,
I apologize for a vigorously off-topic post; Google groups just ain't
helping me this time.
Suppose I have two IP addresses from the same subnet on the same
interface, and this interface also happens to be what my default
gateway is on, like so:
/sbin/ifconfig eth1 213.168.178.209 netmask 255.255.255.192 \
broadcast 213.168.178.255
/sbin/ifconfig eth1:0 213.168.178.210 netmask 255.255.255.192 \
broadcast 213.168.178.255
/sbin/route add default gw 213.168.178.193
The question is, how does the IP stack decide what source IP address
it should use when there's a packet to be sent on the given subnet or
via the defaultgw? Is there any way to actually choose the source IP
address manually (say, per outgoing connection)?
I'm not interested in rewriting the source address with netfilter based
on destination and/or service; What I'm looking for is rather a way to
initiate two connections to the same destination host using the two
different source IP addresses.
--
Tomas Szepe <[email protected]>
On Thu, 5 Dec 2002, Tomas Szepe wrote:
> Hi,
>
> I apologize for a vigorously off-topic post; Google groups just ain't
> helping me this time.
>
> Suppose I have two IP addresses from the same subnet on the same
> interface, and this interface also happens to be what my default
> gateway is on, like so:
>
> /sbin/ifconfig eth1 213.168.178.209 netmask 255.255.255.192 \
> broadcast 213.168.178.255
> /sbin/ifconfig eth1:0 213.168.178.210 netmask 255.255.255.192 \
> broadcast 213.168.178.255
> /sbin/route add default gw 213.168.178.193
>
> The question is, how does the IP stack decide what source IP address
> it should use when there's a packet to be sent on the given subnet or
> via the defaultgw? Is there any way to actually choose the source IP
> address manually (say, per outgoing connection)?
>
> I'm not interested in rewriting the source address with netfilter based
> on destination and/or service; What I'm looking for is rather a way to
> initiate two connections to the same destination host using the two
> different source IP addresses.
>
The simple answer is that if you need a specific IP address
associated with a "multi-honed" host, that has only one interface,
then something is broken. And you get to keep the pieces.
The IP addresses assigned to a multi-honed host are the addresses
to which it will respond during ARP. The ARP (Address Resolution
Protocol) you remember, is the protocol used to get the "hardware"
or IEEE station address of the interface.
Any IP protocol will properly work with any IP address embedded in
the packet from the interface that responded to the ARP.
However, the IP address inside the data-gram will usually be
the IP address of the interface that first sent that packet.
The IP address used is the address of the interface that met
the necessary criteria for getting the data-gram onto the wire.
In other words, the net-mask and the network address are the
determining factors. If you have two or more IP addresses that
are capable of putting the data-gram on the wire, the first one,
i.e., the address used to initialize the interface first, will
be the one that is used in out-going packets.
Since you don't bind a socket to a specific IP address when
initiating connections, you can't chose what IP address will
be used for those connections. However, when setting up
a server that will accept connections, you bind that socket
to both an IP address and a port. Therefore, a server can
be created that accepts connections only to a specific IP
address of a multi-honed host.
If you insist upon using only a particular IP address for
a specific connection from a multi-honed host, you can set
up a host-route for that particular machine:
Script started on Thu Dec 5 14:57:26 2002
# ifconfig
eth0 Link encap:Ethernet HWaddr 00:50:DA:19:7A:7D
inet addr:10.100.2.224 Bcast:10.255.255.255 Mask:255.0.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:30718261 errors:72 dropped:0 overruns:0 frame:72
TX packets:3307230 errors:0 dropped:0 overruns:0 carrier:221
collisions:35578 txqueuelen:100
Interrupt:10 Base address:0xb800
eth0:1 Link encap:Ethernet HWaddr 00:50:DA:19:7A:7D
inet addr:10.106.100.167 Bcast:10.255.255.255 Mask:255.0.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:10 Base address:0xb800
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:59125 errors:0 dropped:0 overruns:0 frame:0
TX packets:59125 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
# route add -host skunkworks gw 10.106.100.167
# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.106.100.205 10.106.100.167 255.255.255.255 UGH 0 0 0 eth0
10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 10.100.1.1 0.0.0.0 UG 0 0 0 eth0
# exit
exit
Script done on Thu Dec 5 14:58:58 2002
You can verify with `tcpdump` that the source address to 10.106.100.205
has been set to the gateway address of 10.106.100.167.
Script started on Thu Dec 5 15:28:15 2002
# tcpdump -n
tcpdump: listening on eth0
15:28:26.334663 0:d0:b7:3c:df:26 ff:ff:ff:ff:ff:ff 886d 60:
0001 0001 2016 0c00 0100 0002 b32c 0bdb
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000
[SNIPPED lots of junk]
15:28:28.558563 arp who-has 10.106.100.205 tell 10.106.100.167
^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^
skunkworks second interface
513 packets received by filter
0 packets dropped by kernel
# exit
exit
Script done on Thu Dec 5 15:28:38 2002
This is just an artifact of a particular kernel and a particular
installation. It is not the "right way" to do this.
There is no RightWay(tm) because any attempt to choose a specific
IP to on the wire from a machine that has only one interface, but
is multi-honed, is broken at the start. However, you can chose where
Netmasks, associated with a particular address, "interlock" to get
a range of destination addresses to use a particular source address.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.
On Thu, Dec 05, 2002 at 08:00:54PM +0100, Tomas Szepe wrote:
> Suppose I have two IP addresses from the same subnet on the same
> interface, and this interface also happens to be what my default
> gateway is on, like so:
>
> /sbin/ifconfig eth1 213.168.178.209 netmask 255.255.255.192 \
> broadcast 213.168.178.255
> /sbin/ifconfig eth1:0 213.168.178.210 netmask 255.255.255.192 \
> broadcast 213.168.178.255
> /sbin/route add default gw 213.168.178.193
Honnestly, the simplest way to deal with aliases and source addresses is to
forget ifconfig/route and replace them with iproute2 (ftp.inr.ac.ru), which
lets you specify the source address among many other settings. Really a
wonderful and cleanly coded tool.
In your case, you'd call it this way :
ip addr add 213.168.178.209/26 dev eth0
ip addr add 213.168.178.210/26 dev eth0
ip route add default via 213.168.178.193
When several addresses can resolve for the source, the first one is used by
default, so in the above case, it will be .209. But it's easy to specify the
other one :
ip route add default via 213.168.178.193 src 213.168.178.210
In fact, you can even set the source of another interface, which is very useful
on point-to-point connections above private networks.
I really don't understand why modern distros don't use it by default, and still
prefer prehistoric ifconfig and route. These ones are extremely limited
regarding what the kernel can do.
Cheers,
Willy
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Fri, 6 Dec 2002 07:37, Richard B. Johnson wrote:
> > I'm not interested in rewriting the source address with netfilter based
> > on destination and/or service; What I'm looking for is rather a way to
> > initiate two connections to the same destination host using the two
> > different source IP addresses.
>
> The simple answer is that if you need a specific IP address
> associated with a "multi-honed" host, that has only one interface,
> then something is broken. And you get to keep the pieces.
There are some reasons why you might like to use a specific address on a
machine that is multi-homed, although normally the logic in the kernel works
fine. I won't try to explain it in detail at this stage, but link-local is a
personal favourite.
Probably all you want is to use the SO_BINDTODEVICE socket option.
Brad
- --
http://linux.conf.au. 22-25Jan2003. Perth, Aust. I'm registered. Are you?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE9771/W6pHgIdAuOMRAjZGAKC8anD6rum/sEuYJRX2XZNEtEOG2gCdFBUq
SOK8RUP9Ub2hX1HGej9vxhU=
=KHOl
-----END PGP SIGNATURE-----
[snip]
> determining factors. If you have two or more IP addresses that
> are capable of putting the data-gram on the wire, the first one,
> i.e., the address used to initialize the interface first, will
> be the one that is used in out-going packets.
Thanks for the clarification on this one.
--
Tomas Szepe <[email protected]>
> ip route add default via 213.168.178.193 src 213.168.178.210
That's just what I needed, thanks Willy.
--
Tomas Szepe <[email protected]>
Richard B. Johnson wrote:
> On Thu, 5 Dec 2002, Tomas Szepe wrote:
>>I'm not interested in rewriting the source address with netfilter based
>>on destination and/or service; What I'm looking for is rather a way to
>>initiate two connections to the same destination host using the two
>>different source IP addresses.
>>
>
>
> The simple answer is that if you need a specific IP address
> associated with a "multi-honed" host, that has only one interface,
> then something is broken. And you get to keep the pieces.
> The IP addresses assigned to a multi-honed host are the addresses
> to which it will respond during ARP. The ARP (Address Resolution
> Protocol) you remember, is the protocol used to get the "hardware"
> or IEEE station address of the interface.
>
> Any IP protocol will properly work with any IP address embedded in
> the packet from the interface that responded to the ARP.
>
> However, the IP address inside the data-gram will usually be
> the IP address of the interface that first sent that packet.
> The IP address used is the address of the interface that met
> the necessary criteria for getting the data-gram onto the wire.
> In other words, the net-mask and the network address are the
> determining factors. If you have two or more IP addresses that
> are capable of putting the data-gram on the wire, the first one,
> i.e., the address used to initialize the interface first, will
> be the one that is used in out-going packets.
You may be able to influence this with policy-based routing and
the arp-filter code.
>
> Since you don't bind a socket to a specific IP address when
> initiating connections, you can't chose what IP address will
> be used for those connections. However, when setting up
> a server that will accept connections, you bind that socket
> to both an IP address and a port. Therefore, a server can
> be created that accepts connections only to a specific IP
> address of a multi-honed host.
You certainly can bind to a specific IP and/or port when initiating
a connection. You can use the local IP to do source-based routing.
I have not done exactly the thing described here, but I have done
similar things, certainly binding to ports & ips on both server
and initiator side of an IP connection.
> There is no RightWay(tm) because any attempt to choose a specific
> IP to on the wire from a machine that has only one interface, but
> is multi-honed, is broken at the start. However, you can chose where
I think you presume too much about what other people might consider
broken or not. :)
--
Ben Greear <[email protected]> <Ben_Greear AT excite.com>
President of Candela Technologies Inc http://www.candelatech.com
ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear
--On Thursday, December 05, 2002 20:25:08 -0800 Ben Greear
<[email protected]> wrote:
> Richard B. Johnson wrote:
>> On Thu, 5 Dec 2002, Tomas Szepe wrote:
>
>>> I'm not interested in rewriting the source address with netfilter based
>>> on destination and/or service; What I'm looking for is rather a way to
>>> initiate two connections to the same destination host using the two
>>> different source IP addresses.
>>>
>>
>>
>> The simple answer is that if you need a specific IP address
>> associated with a "multi-honed" host, that has only one interface,
>> then something is broken. And you get to keep the pieces.
Not always so. I'm writing an app (userspace HIP, see http://www.hip4inter.net)
that needs this too. And it would be nice if there were a simple way to do
it. VoIP apps need this sometimes too.
>> The IP addresses assigned to a multi-honed host are the addresses
>> to which it will respond during ARP. The ARP (Address Resolution
>> Protocol) you remember, is the protocol used to get the "hardware"
>> or IEEE station address of the interface.
>>
>> Any IP protocol will properly work with any IP address embedded in
>> the packet from the interface that responded to the ARP.
Um. No, this isn't right. The interface in question could be PPP, or
something wierd like IP over FireWire, neither of which do ARP. ARP is
specifically for IPv4 over ethernet or something which emulates ethernet.
IPv6 doesn't use ARP even on ethernet, yet source address selection is even
more important in that case. Nearly every IPv6 application will need to do
it, or trust the kernel to do it right. Random is not an option.
And applications *do not* necessarily work with any old address in the
header; otherwise NAT would never be a problem.
>> However, the IP address inside the data-gram will usually be
>> the IP address of the interface that first sent that packet.
>> The IP address used is the address of the interface that met
>> the necessary criteria for getting the data-gram onto the wire.
>> In other words, the net-mask and the network address are the
>> determining factors. If you have two or more IP addresses that
>> are capable of putting the data-gram on the wire, the first one,
>> i.e., the address used to initialize the interface first, will
>> be the one that is used in out-going packets.
This just describes the usual behaviour of unix-like systems. But the most
recently configured address would be just as correct a default.
> You may be able to influence this with policy-based routing and
> the arp-filter code.
You can select what you want this way, yes. But there should be a way to
select source addresses from among those already configured without having
to have the ability to change the routing table or firewall rules! Hence:
> You certainly can bind to a specific IP and/or port when initiating
> a connection. You can use the local IP to do source-based routing.
> I have not done exactly the thing described here, but I have done
> similar things, certainly binding to ports & ips on both server
> and initiator side of an IP connection.
>
>
>> There is no RightWay(tm) because any attempt to choose a specific
>> IP to on the wire from a machine that has only one interface, but
>> is multi-honed, is broken at the start. However, you can chose where
>
> I think you presume too much about what other people might consider
> broken or not. :)
What's more, the specs actually say otherwise. That's something you MUST
be able to do (see the sockets API RFC). And in IPv6 sometimes only the
application can know what to do.
Andrew
On Thu, 5 Dec 2002, Tomas Szepe wrote about choosing a source ip.
If you are writing the application, it is in fact quite trivial.
I found that, although no networking or 'sockets' programming
books I have read ever bind a client's socket to an address, only
a server's, it can be done as the included code shows. It seems
to work fine (although aparently not well documented). I'm not
sure it's 'legal', but it works on Linux-2.4.18 and SunOS 5.5.1.
It does not work with windsock, but who cares.
#ifdef _MSC_VER
#define WINDOWS
#else
#define LINUX
#endif
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <errno.h>
#include <time.h>
#ifdef LINUX
#include <sys/socket.h>
#include <sys/time.h>
#include <netinet/in.h>
#include <netdb.h>
#include <unistd.h>
#endif
#ifdef WINDOWS
#include <winsock.h>
#include <io.h>
typedef unsigned long off_t;
#define ENONET 0x100
#define sleep(x) Sleep((x) * 1000L)
#endif
#define FAIL -1
#define ERRORS(s) { \
fprintf(stderr, "Error on line %d, file, %s, call %s (%s)\n", \
__LINE__,__FILE__,(s),strerror(errno)); \
}
int main(int argc, char *argv[]) {
struct sockaddr_in dst, src;
struct hostent *hp;
int s, on;
#ifdef WINDOWS
WSADATA wsadata;
(void)WSAStartup(MAKEWORD(2,0), &wsadata);
#endif
if(argc < 3) {
printf("Usage:\n %s source destination\n", argv[0]);
exit(EXIT_FAILURE);
}
memset(&src, 0x00, sizeof(src));
memset(&dst, 0x00, sizeof(dst));
if((hp = gethostbyname(argv[1])) == NULL)
{
errno = ENONET;
ERRORS("gethostbyname");
}
src.sin_addr.s_addr = *((unsigned long *) *hp->h_addr_list);
src.sin_family = AF_INET;
if((hp = gethostbyname(argv[2])) == NULL) {
errno = ENONET;
ERRORS("gethostbyname");
}
dst.sin_addr.s_addr = *((unsigned long *) *hp->h_addr_list);
dst.sin_family = AF_INET;
dst.sin_port = htons(IPPORT_TELNET);
if ((s = socket(AF_INET, SOCK_STREAM, 0)) == FAIL)
ERRORS("socket");
on = 1;
if(setsockopt(s, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) == FAIL)
ERRORS("setsockopt");
if(bind(s, &src, sizeof(src)) == FAIL)
ERRORS("bind");
if((connect(s, (struct sockaddr *) &dst, (int) sizeof(dst))) == FAIL)
ERRORS("connect");
fprintf(stdout,"Connected to %s, waiting...", argv[2]);
fflush(stdout);
pause();
(void)close(s);
return 0;
}
Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.
Tomas Szepe <[email protected]> [02/12/05 14:08]:
>
> What I'm looking for is rather a way to
> initiate two connections to the same destination host using the two
> different source IP addresses.
I believe you can just bind(2) each socket to a different address
before connect(2).
--
Fry, of all the friends I've had ... you're the first. - Bender