2000-11-02 23:56:06

by Rob Landley

[permalink] [raw]
Subject: 255.255.255.255 won't broadcast to multiple NICs

Under 2.2.16, broadcast packets addressed to
255.255.255.255 do not go out to all interfaces in a
machine with multiple network cards. They're getting
routed out the default gateway's interface instead.

If I ifconfig eth1 down (which has the gateway behind
it), I start getting "no route to host", even though
the other subnet's still up and the default gateway's
cleaned out of the routing tables. Under no
circumstances can I get the broadcast packet to go out
more than one interface (I hate to say "like it does
under windows" but in this case, yes).

The packets aren't actually getting sent to the
gateway, they're just getting sent out the gateway's
interface. They're still broadcast packets. I.E. in
a machine with only one NIC, broadcasting
255.255.255.255 works fine.

Is there something I can echo into /proc somewhere to
make this work, or some magic combination of ifconfig
and route that will tell it to actually broadcast out
more than one interface? Should I mess around with
the ethernet bridging code? I don't know if any of
these will work. The problem seems to be conceptual:
when one packet goes into the stack, only one packet
comes out. Global broadcast means with multiple NICS,
multiple packets should come out (one per NIC), and
apparently there's no support for that.

Ummm... Help?

(I have config info and test code if you can't
reproduce this. I, unfortunately, have spent the
entire afternoon trying NOT to reproduce this.
Sigh...)

Rob

__________________________________________________
Do You Yahoo!?
>From homework help to love advice, Yahoo! Experts has your answer.
http://experts.yahoo.com/


2000-11-03 00:31:55

by Jeff Garzik

[permalink] [raw]
Subject: Re: 255.255.255.255 won't broadcast to multiple NICs

Rob Landley wrote:
> Under 2.2.16, broadcast packets addressed to
> 255.255.255.255 do not go out to all interfaces in a
> machine with multiple network cards. They're getting
> routed out the default gateway's interface instead.

Are the network cards on the same network?

--
Jeff Garzik | Dinner is ready when
Building 1024 | the smoke alarm goes off.
MandrakeSoft | -/usr/games/fortune

2000-11-03 01:23:15

by Rob Landley

[permalink] [raw]
Subject: Re: 255.255.255.255 won't broadcast to multiple NICs

--- Jeff Garzik <[email protected]> wrote:
> Rob Landley wrote:
> > Under 2.2.16, broadcast packets addressed to
> > 255.255.255.255 do not go out to all interfaces in
> a
> > machine with multiple network cards. They're
> getting
> > routed out the default gateway's interface
> instead.
>
> Are the network cards on the same network?

Two subnets. (both martians: 10.blah and
192.168.blah). Gateway's off of 10.blah (beyond which
lives the internet), the 192 thing is the small
cluster I'm putting together in my office to test the
software.

I take it this makes a difference? If there's some
kind of "don't do that" here, I might be happy just
documenting it. (In theory, I could iterate through
the NICs and send out a broadcast packet to each
interface's broadcast address (although for reasons
that are a bit complicated to go into right now unless
you really want to know, that's not easy to do in this
case).) But that's just a workaround to cover up the
fact that the IP stack isn't doing the obvious with
global broadcasts.

So the question is, is the stack's behavior right? If
not, what's involved in fixing it, and if so, is it
documented anywhere?

(I checked google rather a lot before coming here, and
linux/Documentation, and even glanced at the route.c
source code. ip_route_output_slow has several
explicit checks for "FFFFFFFF" which are easily
searched for, but the upshot is that the packet gets
mapped to a single interface anyway. Around line 1641
of my sources there's an #ifdef CONFIG_IP_MROUTE that
looked very interesting, but it turns out only to be
for multicast addresses and I don't know if
IN_DEV_FORWARD is forking the packet or not...)

At which point I came here. :)

Rob

__________________________________________________
Do You Yahoo!?
>From homework help to love advice, Yahoo! Experts has your answer.
http://experts.yahoo.com/

2000-11-03 02:16:15

by Philippe Troin

[permalink] [raw]
Subject: Re: 255.255.255.255 won't broadcast to multiple NICs

Rob Landley <[email protected]> writes:

> --- Jeff Garzik <[email protected]> wrote:
> > Rob Landley wrote:
> > > Under 2.2.16, broadcast packets addressed to
> > > 255.255.255.255 do not go out to all interfaces in
> > a
> > > machine with multiple network cards. They're
> > getting
> > > routed out the default gateway's interface
> > instead.
> >
> > Are the network cards on the same network?
>
> Two subnets. (both martians: 10.blah and
> 192.168.blah). Gateway's off of 10.blah (beyond which
> lives the internet), the 192 thing is the small
> cluster I'm putting together in my office to test the
> software.
>
> I take it this makes a difference? If there's some
> kind of "don't do that" here, I might be happy just
> documenting it. (In theory, I could iterate through
> the NICs and send out a broadcast packet to each
> interface's broadcast address (although for reasons
> that are a bit complicated to go into right now unless
> you really want to know, that's not easy to do in this
> case).) But that's just a workaround to cover up the
> fact that the IP stack isn't doing the obvious with
> global broadcasts.
>
> So the question is, is the stack's behavior right? If
> not, what's involved in fixing it, and if so, is it
> documented anywhere?

I think historically, BSD stacks were routing 255.255.255.255 to the
"primary interface" (whatever that means).

All the code I've encountered which actually needed to perform
broadcast on all interfaces was sending subnet-directed broadcasts by
hand on all interfaces.

Broadcast is ugly anyways, why don't you use multicast ?

Phil.

2000-11-03 04:17:53

by Rob Landley

[permalink] [raw]
Subject: Re: 255.255.255.255 won't broadcast to multiple NICs

--- "Richard B. Johnson" <[email protected]>
wrote:

> Using an IP packet of 255.255.255.255 doesn't mean
> it's a broadcast
> packet. It is going to your default gateway because
> it is outside
> your netmask, which guarantees that it is not a
> broadcast.

1) No, it's still a broadcast packet when it goes out.
That's the behavior of the current code. The packet
isn't redirected to the gateway, it retains its
255.255.255.255 address when heading out the other
interface. (Other computers on the subnet connected
to that interface see it, even though their address
isn't 255.255.255.255 and it doesn't match the address
or broadcast address of any of their interfaces.)

I just re-confirmed this. On 192.168.0.3 I moved the
default gateway to 192.168.0.99 (a non-existent
machine, but told it to still go out eth0), ran the
255.255.255.255 broadcaster on .3, ran a listener on
192.168.0.1, and the listener heard the packets from
the broadcaster (and confirmed their source address,
192.168.0.3). The broadcast address on both
interfaces (.3 and .1) is 192.168.0.255, with netmask
255.255.255.0.

So once again: when sending to 255.255.255.255,
Broadcast packets are spit out the gateway's interface
(AS broadcast packets), but not out the other
interface(s). The behavior I expected is the
broadcast packets getting sent out ALL the interfaces
this machine had. (Most gateways won't FORWARD
broadcast packets, which is why this doesn't flood the
whole internet. This is also WHY gateways have to go
out of their way not to forward broadcast packets,
because there IS a way of specifying a broadcast
address larger than a single subnet.)

2) Windows does it. (That's no defense of the
practice, but it is at least circumstantial evidence
that the fact linux is at least partially supporting
it is not just some strange accident.)

3) The support that's in there now has explicit code
implementing it. If it has no special meaning, why
does linux/net/ipv4/route.c treat "0xFFFFFFFF"
specially? Grep for it. Sample code snippet (from
route.c):

if (key.dst = 0xFFFFFFFF)
res.type = RTN_BROADCAST;

There's a half-dozen or more of those in there (2.2.16
on this box, I could check 2.2.18pre if you like but I
don't expect a difference)...

> To use a broadcast of 255.255.255.255, your netmask
> would have
> to be 0.0.0.0, which would gurantee that you have no
> default
> route.

I'm not making this up. There are a lot of precedents
of it being used. Look at cisco:

http://www.ieng.com/univercd/cc/td/doc/product/software/ios113ed/113ed_cr/np1_r/1ripadr.htm#xtocid587510

And of course bootp/dhcp use it for their queries when
they don't know WHO they are:

http://ctdp.tripod.com/os/linux/usersguide/linux_ugdhcp.html

I always thought it was the the global broadcast
address, propogated to all NICs. That's what it USED
to be, but I've been using it so long (and so
intermittently) I don't remember what documentation
that originally came from.

Yes, it could be a bad habit and maybe I should stop.
But if so, I'd like to see where that's in writing.
(Documented somewhere. Anywhere. Alan Cox saying
"this is so" is plenty authoritative enough for me.
Documentation counts as resolving the issue...)

> As an example, we 'own' a network from:
>
> 204.178.40.1 to 204.178.47.255.
>
> This means that the network address is 204.178.40.0,
> the netmask
> is 255.255.248.0, and the broadcast address is
> 204.178.47.255
>
> Anything outside the LAN, which means anything that
> won't 'fit'
> inside the netmask goes out the default route.
> That's how it
> works.

I know how that works. But the fact remains that
machines that aren't 255.255.255.255 know to receive
that address. Both Linux machines and windows
machines. The switching hub propogates it and the
stack receives it.

The behavior it has right now DOES come very close to
what I expect it to do, and way in the past (on a
mixed network of OS/2, windows, and macintoshes, I
suspect, don't remember who was server and who was
client, it was in java) using 255.255.255.255 did
exactly what I wanted it to do. I wasn't network
administrator on those boxes though, so I dunno if
this was an OS default or the way the NICs were
configured. (Come to think of it, I don't remember if
any of the boxes I was using HAD more than one NIC, it
was a while ago. They probably didn't.)

It's just not doing it here. I'm wondering if not
doing it here is documented and something the network
guys intend to keep, or if the half-support in there
now is going to be extended to full support.

> When you set a broadcast address, you are simply
> plugging in
> an IP, within your network, that is not otherwise in
> use.

That's the subnet broadcast, sure. But I remember
something about global broadcast addresses from way
back in college. Maybe I imagined it, but I know I
got tested on it.

> This means that when a HARDWARE address of all bits
> set is
> being sent out your Ethernet controller (as a
> destination),
> the IP address within the packet will be whatever
> you set it
> to be with : `ifconfig eth0 .... broadcast
> 204.178.47.255`.
> I could use ... broadcast 204.178.40.0 and, in
> fact,
> in the 'olden' days, that's what was used.

And here I thought the IP stack translated the
broadcastness down to the ethernet layer, where the
MAC address was what actually steered when
transporting packets from one machine to another
across a length of cat 5 (anybody mentioning OSI
models will be subjected to the "7 layer burrito"
page, assuming I can find a mirror of it that's still
up), and that there IS a MAC broadcast address (or
flag bit or some such).

Which could be why the IP stack needs to have a
broadcast flag set for the packets when it feeds them
to the interface...

> All machines on Ethernet LANs accept broadcast
> packets,
> multicast packets, and packets with a destination of
> their
> HARDWARE address. This is all done in HARDWARE! Your

Strangely, I was under the impression that the IP
stack is the layer of software that tells that
hardware (or tells the drivers for that hardware) what
to do. Again, I seem to be imagining all sorts of
strange things, thanks for clearing up the confusion.

> controller
> does not receive everything on the LAN. If it did,
> you'd be
> so bogged down getting ethernet interrupts and
> dumping the
> data on the floor, that you would have few CPU
> cycles to
> do any actual work.

Which would be why the MAC level addressing has a way
of designating packets broadcast, or addressing them
to a specific machine's address, generally supplied by
the vendor in an EPROM or some such. Which is also
what gives things such as switching hubs information
to go on... (Again, I vaguely remember this from
someplace, Perhaps from last month, when I taught the
networking chapter out of the textbook to my students
in the "intro to Unix" course I teach tuesday nights
at ACC...)

> When a machine receives a hardware broadcast packet,
> it may
> not even look at the destination address in the
> packet. That's
> why your broadcast address is a "throw-away". The
> receiver is
> only concerned with the sender's IP address, usually
> to
> resolve an address (ARP).

Okay, enlighten me. Is the hardware looking at the
MAC address and status bits of the passing frames, or
is it grabbing and inspecting each frame to see if the
TCP/IP headers in the data section of the frame
indicate it should be interested?

Have you ever seen a network that has more than one
(non-overlapping) IP subnet working through the same
ethernet hub? Those are fun. (And an area where your
stack shouldn't be lazy about simply accepting
broadcast packets without at least glancing at their
source address, at least according to the IBM guys
fiddling with the OS/2 TCP/IP stack back when I was
working on the install software for OS/2 4.0. :)

> Cheers,
> Dick Johnson

Rob

__________________________________________________
Do You Yahoo!?
>From homework help to love advice, Yahoo! Experts has your answer.
http://experts.yahoo.com/

2000-11-03 17:33:49

by Rob Landley

[permalink] [raw]
Subject: Re: 255.255.255.255 won't broadcast to multiple NICs

--- Philippe Troin <[email protected]> wrote:
> Rob Landley <[email protected]> writes:
> > So the question is, is the stack's behavior right?
> If
> > not, what's involved in fixing it, and if so, is
> it
> > documented anywhere?
>
> I think historically, BSD stacks were routing
> 255.255.255.255 to the
> "primary interface" (whatever that means).

Yeah, that maps with what I've seen.

Apparently, the stack is stricly "one packet in, one
packet out", and only concerns itself with where to
send that packet. So when a packet can go out more
than one interface (as in 255.255.255.255), the
broadcast nature of it doesn't actually cause the
packet to fork in the stack, it's treated like a load
balancing situation instead where one interface is
selected and the one packet is delivered. (The
multiple delivery aspect is left solely to the
ethernet layer. Despite being a broadcast packet, the
IP layer doesn't replicate the broadcast on an
interface level.)

I personally did not expect this behavior, I expected
the packet to go out all the interfaces. And I still
think the behavior I expected makes more sense.
However, if the behavior that's in the kernel now is
documented and what IP stacks the world over are
expected to do, I can certainly live with it. :)

It's entirely possible that the other platforms I
worked on before never had more than one NIC in a box
I actually used. The fact I wasn't suprised by the
behavior I saw elsewhere could simply be because I
didn't try stuff on a box where the behavior would be
suprising.

That said, I still think 255.255.255.255 "should" go
out all interfaces (since gateways don't forward
broadcast packets anyway, it's not going to flood the
network or anything). I suspect the "it works, leave
it alone" principle will apply here, but it'd be nice
to attract the attention of a networking guru like
Alan Cox, David S. Miller, or Donald Becker for a few
seconds to at least get shot down on this issue
decisively. :)

> All the code I've encountered which actually needed
> to perform
> broadcast on all interfaces was sending
> subnet-directed broadcasts by
> hand on all interfaces.

Unfortunately, I literally can't do that in this
instance. (Yes, I've tried.) The client and server
I'm writing are in Java. There's two problems here.

1) It's for a heterogeneous render farm. Linux boxes,
NT/2000 boxes, and SGI boxes at least. Maybe someday
macintoshes too, and who knows what else. (That's why
I'm writing it in Java in the first place.)

2) I'm trying to have it configure itself as much as
possible automatically. A lot of the people using
render farms went to art college and liked it. That's
why the broadcast packets (so the clients can find
their server and vice versa when everybody boots up,
without being told). I don't want them to have to
tell it anything, I want them to throw it on the
machine and run it (quite possibly running all the
nodes from a shared mount) and just have it work.
I've managed to avoid any node specific configuration
so far, and that makes things MUCH easier for my
expected user base.

3) Java sucks in many ways. Today's way is that it
never occurred to Sun that a machine might have more
than one IP address assigned to it, so
InetAddress.getLocalHost() returns exactly one
address. Unfortunately, just about EVERY machine has
two interfaces defined, the other one being loopback
on 127.0.0.1, and natrually the loopback is the one
that getLocalHost() returns. (Since it's the one that
we pretty much already know the address of anyway, and
querying it is therefore useless, that's the one it
queries. Thank you Sun.) There is no way to query
the current machine's interfaces without resorting to
native code.

Bind to a socket to a local port and query that
address you say? Nope, too easy. The address
returned when I query a socket (rather than a
connection) is 0.0.0.0 on any machine with multiple
interfaces (even loopback), since the socket is bound
to that port on ALL the interfaces. Each incoming or
outgoing connection does have a valid "from" IP
address, but I have to wait for a connection to come
in to get that. (Unless I explicitly specify which IP
to bind to when I create the socket, but if I knew
that I'd already be there.)

Nope, making my own connection to a port on the same
machine just means 127.0.0.1 is talking to 127.0.0.1.
Tried it. Didn't work.

Nope, feeding the loopback address to getAllByName()
doesn't help either. I tried that too, it just
returns a length 1 array containing just the loopback
address.

Now you know why I'm resorting to 255.255.255.255.
I'm sort of faking things: when the server broadcasts
to clients they know who it is, and when they
broadcast to it, it knows who THEY are (it says in the
UDP datagram header info). And the way I've written
it, that's all they really need to know (although when
we reply to each other we can each find out the info
we don't know: who WE are. But by that point, we no
longer need it. :)

I may just document "if you run this on a machine with
more than one network card, you have to specify the
broadcast addresses on the command line". It's
configuration, but the only machine likely to HAVE
multiple interfaces is the server (which could be
serving multiple subnets in a really BIG render farm),
so I suppose it's tolerable...

> Broadcast is ugly anyways, why don't you use
> multicast ?

I'm only passingly familiar with it, does it map well
to this problem?

The only data I'm trying to transmit is "where is
everybody", or "wake up". The broadcast packets are
only needed for clients to find the server on bootup
(and vice versa if the server is rebooted). They're
also used to wake up clients if they go to sleep
because the server has nothing for them to do at the
moment, but that second part's a convenience, really.
The server could loop through and address them
individually instead since it knows where they are by
that point.

The actual heavy lifting of data is done by TCP/IP
streams. UDP broadcast is just for figuring out where
to open the TCP/IP connections to.

> Phil.

Rob

__________________________________________________
Do You Yahoo!?
>From homework help to love advice, Yahoo! Experts has your answer.
http://experts.yahoo.com/

2000-11-03 18:14:10

by Paul Flinders

[permalink] [raw]
Subject: Re: 255.255.255.255 won't broadcast to multiple NICs


Rob Landley <[email protected]> writes:
> 3) Java sucks in many ways. Today's way is that it
> never occurred to Sun that a machine might have more
> than one IP address assigned to it, so
> InetAddress.getLocalHost() returns exactly one
> address. Unfortunately, just about EVERY machine has
> two interfaces defined, the other one being loopback
> on 127.0.0.1, and natrually the loopback is the one
> that getLocalHost() returns. (Since it's the one that
> we pretty much already know the address of anyway, and
> querying it is therefore useless, that's the one it
> queries. Thank you Sun.) There is no way to query
> the current machine's interfaces without resorting to
> native code.

I faced this problem a while ago - in the end I cheated and
put this bit of code in a shell script used to start the
application

NET_ADDRESSES=`/sbin/ifconfig | \
awk '/^[^ ].*HWaddr/ { HWaddr=$5; next;}
/^[^ ]/ { HWaddr=0; }
/^[ ].*addr:/ { if (HWaddr != 0) { printf("-Dethaddr.%s=%s -Dnetmask.%s=%s ",substr($2,6),HWaddr,substr($2,6),substr($4,6)); } } END {print;}'`

including ${NET_ADDRESSES} in the java command line sets
up a set of defines, one per interface. For example

-Dethaddr.172.16.1.1=00:00:0A:BC:CD:78 -Dnetmask.172.16.1.1=255.255.0.0

which you can use via System.getProperty() and System.getProperties()

2000-11-03 19:47:11

by Rob Landley

[permalink] [raw]
Subject: Re: 255.255.255.255 won't broadcast to multiple NICs

--- Paul Flinders <[email protected]> wrote:
>
> Rob Landley <[email protected]> writes:
> > 3) Java sucks in many ways. Today's way is that
...
> > There is no way to query the current machine's
> > interfaces without resorting to
> > native code.
>
> I faced this problem a while ago - in the end I
> cheated and put this bit of code in a shell script
> used to start the application

I've considered it. Counts as "native code", but
thanks for the script anyway. :)

For my current app, I've pretty much decided that for
the boxes where broadcasting 255.255.255.255, they
have to supply it on the command line. Maybe I'll
have the script supply it on the command line for them
since I'll probably offer an RPM or linux-specific tar
as an install option. I need to potentially install
the JRE for them (assuming licensing issues work out
ok there, which I'm 99% certain is fine but want to
double check), and should have a shell script to
encapsulate the "jre -cp myjar.jar runthisclass" part
anyway into "runclient" or "runserver". (Possibly
starting from the init scripts, or with a nice Gnome
icon. Depends how industrious I feel when I'm done,
and/or what my boss wants. :)

The larger question of "should the Linux kernel's IP
stack behavior be fixed, documented, or left alone" is
what I'm interested in now. If people agree that
255.255.255.255 should go out to multiple interfaces,
I'd be willing to try my hand at a patch to route.c
(be afraid, be very afraid), but I'm still waiting to
hear from on high (higher than me anyway) about
whether or not the current behavior is something
they're happy with. (My app will NOT require a custom
kernel to function properly, that's not an option. :)

> including ${NET_ADDRESSES} in the java command line
> sets
> up a set of defines, one per interface. For example
>
> -Dethaddr.172.16.1.1=00:00:0A:BC:CD:78
> -Dnetmask.172.16.1.1=255.255.0.0
>
> which you can use via System.getProperty() and
> System.getProperties()

If I go with a script I'll just have it spit the IP
broadcast addresses one after the other to stdout, and
then call it from the command line with back quotes as
some variant of:

./myprog broadcast `./findbroadcasts`

Encased in the platform-specific launch shell script,
of course. :)

Why on earth would my app need the ethernet address?
If the stack didn't abstract that away, there would be
a much bigger problem than global broadcasts not
really being global...

Rob

__________________________________________________
Do You Yahoo!?
>From homework help to love advice, Yahoo! Experts has your answer.
http://experts.yahoo.com/

2000-11-03 20:20:19

by Philippe Troin

[permalink] [raw]
Subject: Re: 255.255.255.255 won't broadcast to multiple NICs

Rob Landley <[email protected]> writes:
> --- Philippe Troin <[email protected]> wrote:
> > All the code I've encountered which actually needed
> > to perform
> > broadcast on all interfaces was sending
> > subnet-directed broadcasts by
> > hand on all interfaces.
>
> Bind to a socket to a local port and query that
> address you say? Nope, too easy. The address
> returned when I query a socket (rather than a
> connection) is 0.0.0.0 on any machine with multiple
> interfaces (even loopback), since the socket is bound
> to that port on ALL the interfaces. Each incoming or
> outgoing connection does have a valid "from" IP
> address, but I have to wait for a connection to come
> in to get that. (Unless I explicitly specify which IP
> to bind to when I create the socket, but if I knew
> that I'd already be there.)
>
> Nope, making my own connection to a port on the same
> machine just means 127.0.0.1 is talking to 127.0.0.1.
> Tried it. Didn't work.
>
> Nope, feeding the loopback address to getAllByName()
> doesn't help either. I tried that too, it just
> returns a length 1 array containing just the loopback
> address.

The source IP address (as returned by getsockname()) is only set when
the socket is connected... It follows the same logic: for a multihomed
machine, we know which interface will be used only when we know who
we'll be talking to...

> Now you know why I'm resorting to 255.255.255.255.
> I'm sort of faking things: when the server broadcasts
> to clients they know who it is, and when they
> broadcast to it, it knows who THEY are (it says in the
> UDP datagram header info). And the way I've written
> it, that's all they really need to know (although when
> we reply to each other we can each find out the info
> we don't know: who WE are. But by that point, we no
> longer need it. :)
>
> I may just document "if you run this on a machine with
> more than one network card, you have to specify the
> broadcast addresses on the command line". It's
> configuration, but the only machine likely to HAVE
> multiple interfaces is the server (which could be
> serving multiple subnets in a really BIG render farm),
> so I suppose it's tolerable...

You could use SIOGIFCONF (from C) to get the address list. I'm not
sure is java has the equivalent... Or maybe a very small native
method...

> > Broadcast is ugly anyways, why don't you use
> > multicast ?
>
> I'm only passingly familiar with it, does it map well
> to this problem?
>
> The only data I'm trying to transmit is "where is
> everybody", or "wake up". The broadcast packets are
> only needed for clients to find the server on bootup
> (and vice versa if the server is rebooted). They're
> also used to wake up clients if they go to sleep
> because the server has nothing for them to do at the
> moment, but that second part's a convenience, really.
> The server could loop through and address them
> individually instead since it knows where they are by
> that point.
>
> The actual heavy lifting of data is done by TCP/IP
> streams. UDP broadcast is just for figuring out where
> to open the TCP/IP connections to.

Sounds like a good job for multicast... It's fairly simple to use,
but:

1) I'm not sure if java gives you access to the required ioctls
(there's only five of them).

2) You may need to run mrouted on all your gateways.

Phil.

2000-11-04 03:16:09

by Rob Landley

[permalink] [raw]
Subject: Re: 255.255.255.255 won't broadcast to multiple NICs

--- Philippe Troin <[email protected]> wrote:
> Rob Landley <[email protected]> writes:

> The source IP address (as returned by getsockname())
> is only set when
> the socket is connected... It follows the same
> logic: for a multihomed
> machine, we know which interface will be used only
> when we know who
> we'll be talking to...

I know, it does make sense. I'm not really blaming
Java for that one. It's just yet another thing I
tried that didn't work. The lack of a way to query
what the current addresses are directly is the only
reason I tried that as a potential workaround. (If
it's going to return 0 unless you told it what the
address was in the first place, why do they have a
query address method on a server socket anyway? Does
it serve a purpose? Right...)

This isn't the most egregious oversight in java, by
the way. The list is pretty long, but my vote goes
for being unable to truncate a file before Java 1.2.
(There was no API for it. Couldn't be done. Period.
getLength() was there, but setLength() wasn't. And I
had to email them more than once before they'd admit
it was a bug...)

> You could use SIOGIFCONF (from C) to get the address
> list.

See "avoidance of native code" under the heading
"configuration details I don't want them to have to
deal with". When I make platform specific installers
(which there will be for both Linux and NT/2000), i'll
throw a shell around it and feed info in on the
command line. But a better solution is not having to
do it at all, which is the case for machines with one
NIC.

> I'm not sure is java has the equivalent...

It doesn't.

> Or maybe a very small native method...

Querying the system when it first runs and passing the
data onthe command line, maybe. Running platform
specific native code during execution, no.

> > > Broadcast is ugly anyways, why don't you use
> > > multicast ?

Having looked into this question a bit more, I can now
answer:

Because broadcast is, in this case, a more elegant
solution, which requires less configuration (even NOT
working the way I expected), and (as far as I can
tell) is actually more efficient (in this case) in
terms of utilizing network resources, and has no known
scalability problems as used here, either.
(Especially for the server, which is the most likely
bottleneck.)

The most common case here is that the server
broadcasts to all clients (either during boot or after
a period of inactivity). In either case, when this
happens all clients are interested in hearing what the
server has to say.

The only time where this isn't the case is when an
individual client is rebooted and has to look for its
server. This case is relatively rare (not a
scalability issue from a network traffic standpoint,
anyway, even Google only reboots around a hundred
boxen a day). Only the server would respond to the
traffic here, so there's no broadcast storm. (The
initial boot case is closer to that, but the response
is via TCP, not UDP.) And I fail to see how multicast
improves the client reboot case. If multicast ISN'T,
underneath it all, doing a broadcast at the MAC level,
then how is it supposed to find the server if it
doesn't know where it is? So what have we gained?

And in any case, as long as the broadcast mechanism's
already been implemented for the first paragraph,
implementing a gratuitous second mechanism for a
fairly rare case is a gratuitous source of complexity
and potential bugs for no apparent reason.

> Sounds like a good job for multicast... It's fairly
> simple to use,
> but:

I still don't understand why you consider broadcast a
bad solution here. (Other than aesthetic reasons.)
If multicast requires configuration and the whole
point of the broadcast packets in the first place was
to AVOID configuration... I missed something.

> 1) I'm not sure if java gives you access to the
> required ioctls
> (there's only five of them).

It does (or at least claims to), but you still haven't
explained WHY multicast is a more elegant solution in
this case than broadcast. Broadcast is actually the
same mechanism as sending targeted UDP packets to
hosts to wake them up, except the list I iterate
through is one address long. Multicast is a
completely different mechanism.

You want to explain how multicast works to me down at
the MAC layer? Is the server sending a small number
of broadcast MAC packets, or a large number of
individually MAC addressed packets to each host that
has registered itself as interested in these
broadcasts. If the former, what's really changed? If
the latter, how did they register themselves as
interested in the first place?

How about compatability? Will it work on the various
flavors of windows box? Will it work on an Irix box?
Would it conceivably work on a power Mac? I know
broadcast is at least theoretically present in every
networking stack there is, whereas multicast is an
option that has to be complied into the Linux kernel,
so I can't even count on it being there really.

(Art majors are NOT compliling kernels. We're not
going there. Maybe you'll say they'll never use a
distribution that doesn't have it. Should this be an
undocumented dependency, or a documented dependency
they'll never read until our tech support people point
out that paragraph to them on the phone?)

> 2) You may need to run mrouted on all your
> gateways.

That would shoot it down conclusively right there, if
it weren't for the fact they're all on the same subnet
in the first place anyway. (I'm NOT going to ask art
majors to reconfigure gateways. That's not
happening.)

By the way, the broadcast solution I've already
implemented wouldn't WORK if I had gateways in here.
Gateways don't forward broadcast packets. (Unless the
server's being run on the gateway between the subnets,
which is where the multiple interface problem came up
in testing in the first place.)

As for explicit configuration bridging subnets (which
is in the design already, although running the server
program on the gateway box was a way of avoiding that
configuration anyway), why not just tell MY program
the extra config info (list of IPs or whatever)
instead of setting up unrelated routing daemons?

I have a goal in mind and am looking for the shortest,
simplest path to it, with the least dependencies and
complications. I'm not promoting a specific tool
(multicast routing) and trying to see if the job can
be made to use that tool. I'm actually doing the
opposite, trying to AVOID as many things as I can get
away with.

> Phil.

__________________________________________________
Do You Yahoo!?
Thousands of Stores. Millions of Products. All in one Place.
http://shopping.yahoo.com/