So, I have an application that works fine on Solaris, FreeBSD, etc. and
fails on Linux.
This application uses SO_REUSEADDR in conjunction with INADDR_ANY. What
it does is bind() to INADDR_ANY, then listen(). Then, it proceeds to
bind (but _not_ listen) various other specific addresses.
This works fine on Solaris, but the second, etc. bind fails on Linux.
I found a thread about this from February, 1999, starting here:
http://www.uwsg.indiana.edu/hypermail/linux/kernel/9902.1/0828.html
In this message, George Pajari ([email protected]) says:
The original security threat that lead to SO_REUSERADDR being broken in the
1.3.60 kernel (incorrect semantics which I think still persist) was the
fact that if process A bound to port X using INADDR_ANY and SO_REUSERADDR,
a second process could bind to the specific IP addresses and obtain
connections intended for process A.
So the kernel was changed to prevent the second process from completing the
bind() call.
Maybe I'm missing something, but it seems to me that having two binds is
not a security problem: what's really the problem is having two
_listens_. As long as you're only listening on the one, I don't see how
connections/packets could be stolen.
Isn't the correct behavior to fail the second listen(), not the second
bind()? This, anyway, is what Solaris and FreeBSD (and maybe others)
seem to do.
This app does this because it implements peer-to-peer, fully meshed
connections; it first sets up an INADDR_ANY and listens on it so that
any other instance of the app can reach it, then it proceeds to try to
bind to the other instances. Since we have no idea which one will start
first, or whether they'll all start together, we don't want to have a
hole where we might miss a connection from another instance. Thus, we
listen first, then proceed to attempt to connect to the other
instances.
Am I missing something here WRT SO_REUSEADDR vs. INADDR_ANY behavior?
Thanks.
PS. CC'ing me is helpful, but I'll follow along either way. Thx.
--
-------------------------------------------------------------------------------
Paul D. Smith <[email protected]> HASMAT--HA Software Methods & Tools
"Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
These are my opinions---Nortel Networks takes no responsibility for them.
> This application uses SO_REUSEADDR in conjunction with INADDR_ANY. What
> it does is bind() to INADDR_ANY, then listen(). Then, it proceeds to
> bind (but _not_ listen) various other specific addresses.
That should be ok if its setting SO_REUSEADDR
> not a security problem: what's really the problem is having two
> _listens_. As long as you're only listening on the one, I don't see how
> connections/packets could be stolen.
UDP.
In fact the classic exploit consisted of binding to port 2049 witha specific
connect address set on UDP and stealing NFS packets..
%% Alan Cox <[email protected]> writes:
>> This application uses SO_REUSEADDR in conjunction with INADDR_ANY. What
>> it does is bind() to INADDR_ANY, then listen(). Then, it proceeds to
>> bind (but _not_ listen) various other specific addresses.
ac> That should be ok if its setting SO_REUSEADDR
I agree, and so does Solaris/FreeBSD, but Linux doesn't. See below for
a test program. Maybe I'm doing something screwed up.
-------------------------8>< snip ><8-------------------------
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#define MY_PORT 10000
int
main(int argc, char *argv[])
{
int any_fd, this_fd;
int val = 1;
struct sockaddr_in addr;
if ((any_fd = socket(PF_INET, SOCK_STREAM, 0)) < 0
|| fcntl(any_fd, F_SETFL, O_NONBLOCK) < 0
|| setsockopt(any_fd, SOL_SOCKET, SO_REUSEADDR, &val, sizeof val) < 0) {
perror("setup(any)");
return 1;
}
if ((this_fd = socket(PF_INET, SOCK_STREAM, 0)) < 0
|| fcntl(this_fd, F_SETFL, O_NONBLOCK) < 0
|| setsockopt(this_fd, SOL_SOCKET, SO_REUSEADDR, &val, sizeof val) < 0) {
perror("setup(this)");
return 1;
}
addr.sin_family = AF_INET;
addr.sin_port = MY_PORT;
memset(addr.sin_zero, 0, sizeof (addr.sin_zero));
addr.sin_addr.s_addr = INADDR_ANY;
if (bind(any_fd, (struct sockaddr *)&addr, sizeof (addr)) < 0) {
perror("bind(any)");
return 1;
}
if (listen(any_fd, 10) < 0) {
perror("listen(any)");
return 1;
}
addr.sin_addr.s_addr = INADDR_LOOPBACK;
if (bind(this_fd, (struct sockaddr *)&addr, sizeof (addr)) < 0) {
perror("bind(this)");
return 1;
}
return 0;
}
-------------------------8>< snip ><8-------------------------
solaris$ gcc -o reuseaddr{,.c} -lsocket -lnsl
solaris$ ./reuseaddr
Works. Now:
linux$ gcc -o reuseaddr{,.c}
linux$ ./reuseaddr
bind(this): Cannot assign requested address
:( The real code doesn't use LOOPBACK, of course.
This is Linux 2.2.17, but I tried with 2.2.18 too. I haven't tried
2.4.x.
Thanks...
--
-------------------------------------------------------------------------------
Paul D. Smith <[email protected]> HASMAT--HA Software Methods & Tools
"Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
These are my opinions---Nortel Networks takes no responsibility for them.
Paul D. Smith writes:
> I agree, and so does Solaris/FreeBSD, but Linux doesn't. See below for
> a test program. Maybe I'm doing something screwed up.
I think I'm starting to get a hold on this. Can you do a test
for me?
Write a small test program, similar to your TCP one, which instead
uses UDP and let me know what FreeBSD and Solaris do in that case.
Of course, skip the listen part.
I think the test for port reuse in tcp_{v4,v6}_get_port is buggy.
I'll go study this and fix it up... but please do the test I asked
anyways.
Later,
David S. Miller
[email protected]
Paul D. Smith writes:
> %% Alan Cox <[email protected]> writes:
>
> >> This application uses SO_REUSEADDR in conjunction with INADDR_ANY. What
> >> it does is bind() to INADDR_ANY, then listen(). Then, it proceeds to
> >> bind (but _not_ listen) various other specific addresses.
>
> ac> That should be ok if its setting SO_REUSEADDR
>
> I agree, and so does Solaris/FreeBSD, but Linux doesn't.
After reading up some code, FreeBSD does do a bind() check which is
just as restrictive as Linux's except that they allow INADDR_ANY
combinations when the credentials of the user doing the bind() match
the credentials of all other sockets bound to that port.
I don't think we should change our behavior. Allowing the combination
in question only when the UIDs match between the socket owners is
dubious at best.
I actually went to the FreeBSD code because what Steven's showed
was extremely loose in what it let through. It allowed the nfs
port override trick Alan mentioned.
Later,
David S. Miller
[email protected]