2002-01-11 04:19:55

by Pei Zheng

[permalink] [raw]
Subject: strange kernel message when hacking the NIC driver

Hi,

I've done some simple hackings to the network device driver. Basically
i want to connect two NICs as a point-to-point link,ie, packets sending
out from one NIC will be redirected to another NIC, sort of direct-link
cable between these two NICs.
So in the drivers of these two NICs, before hart_start_xmit(), the skb's
header will be forcibly set to its peer's MAC address, thus each packey
sending out from one NIC will be directly diverted to its peer NIC. This
works fine most of the case since tcpdump shows that packets traverse in
this way. However, i found errors in system log:

Jan 10 18:02:40 em2 kernel: 1130: 09 => cb
Jan 10 18:02:40 em2 kernel: 388: f4 => 45
Jan 10 18:02:40 em2 kernel: 26: c0 => 7d
Jan 10 18:02:40 em2 kernel: 149: 1f => 98
Jan 10 18:02:40 em2 kernel: After error applied:
Jan 10 18:02:40 em2 kernel: 00 80 c8 b9 6b b6 00 80 c8 b9 6b b6 08 00
45 00
Jan 10 18:02:40 em2 kernel: 05 dc 73 cd 20 b9 3f 11 44 26 7d a8 0c 0a
c0 a8
Jan 10 18:02:40 em2 kernel: 10 0a e3 c1 4c 1f 6f 83 61 dc fc 5e 5c f4
66 17
Jan 10 18:02:40 em2 kernel: 2c 73 19 ce cc 2d 69 69 bd 43 33 6d 82 de
4a 87
Jan 10 18:02:40 em2 kernel: 8d 9d 81 f6 2e b2 8c 6a d5 e4 f2 e6 bc ab
5a 01
Jan 10 18:02:40 em2 kernel: 34 d5 39 f8 d9 5b 16 bc dc 95 bd b4 08 a9
5e 11
Jan 10 18:02:40 em2 kernel: df 38 80 8a ca d8 1c 53 65 97 91 4c 84 5a
a1 0e
Jan 10 18:02:40 em2 kernel: c2 5f d4 02 a1 9e 1c 35 d4 95 08 44 81 16
a3 e0

there are many this kind of messages. Don't understand what it is. the
beginning part of the data seems to be an icmp packet(00 80 c8 b9 6b
b6 is the MAC of one NIC). For some reason kernel thinks that this
packet is not correct. Any idea about this? If i want to modify a
packet's header before it goes to hard_strat_xmit(), what else should
i do except setting the skb header to the MAC of the NIC's peer only?
checksum stuff?

Any helps will be highly appreciated.

-Pei


2002-01-11 05:25:19

by Timothy Covell

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

On Thursday 10 January 2002 22:19, Pei Zheng wrote:
> Hi,
>
> I've done some simple hackings to the network device driver. Basically
> i want to connect two NICs as a point-to-point link,ie, packets sending
> out from one NIC will be redirected to another NIC, sort of direct-link
> cable between these two NICs.
> So in the drivers of these two NICs, before hart_start_xmit(), the skb's
> header will be forcibly set to its peer's MAC address, thus each packey
> sending out from one NIC will be directly diverted to its peer NIC. This
> works fine most of the case since tcpdump shows that packets traverse in
> this way. However, i found errors in system log:
>
> Jan 10 18:02:40 em2 kernel: 1130: 09 => cb
> Jan 10 18:02:40 em2 kernel: 388: f4 => 45
> Jan 10 18:02:40 em2 kernel: 26: c0 => 7d
> Jan 10 18:02:40 em2 kernel: 149: 1f => 98
> Jan 10 18:02:40 em2 kernel: After error applied:
> Jan 10 18:02:40 em2 kernel: 00 80 c8 b9 6b b6 00 80 c8 b9 6b b6 08 00
> 45 00
> Jan 10 18:02:40 em2 kernel: 05 dc 73 cd 20 b9 3f 11 44 26 7d a8 0c 0a
> c0 a8
> Jan 10 18:02:40 em2 kernel: 10 0a e3 c1 4c 1f 6f 83 61 dc fc 5e 5c f4
> 66 17
> Jan 10 18:02:40 em2 kernel: 2c 73 19 ce cc 2d 69 69 bd 43 33 6d 82 de
> 4a 87
> Jan 10 18:02:40 em2 kernel: 8d 9d 81 f6 2e b2 8c 6a d5 e4 f2 e6 bc ab
> 5a 01
> Jan 10 18:02:40 em2 kernel: 34 d5 39 f8 d9 5b 16 bc dc 95 bd b4 08 a9
> 5e 11
> Jan 10 18:02:40 em2 kernel: df 38 80 8a ca d8 1c 53 65 97 91 4c 84 5a
> a1 0e
> Jan 10 18:02:40 em2 kernel: c2 5f d4 02 a1 9e 1c 35 d4 95 08 44 81 16
> a3 e0
>
> there are many this kind of messages. Don't understand what it is. the
> beginning part of the data seems to be an icmp packet(00 80 c8 b9 6b
> b6 is the MAC of one NIC). For some reason kernel thinks that this
> packet is not correct. Any idea about this? If i want to modify a
> packet's header before it goes to hard_strat_xmit(), what else should
> i do except setting the skb header to the MAC of the NIC's peer only?
> checksum stuff?
>
> Any helps will be highly appreciated.
>
> -Pei


You have me confused about whether your trying to build some kind of
ethernet bridge or some kind of secret packet sniffer or some other
such thing. It looks like you're mixing up TCP/IP and Ethernet.

Ethernet is the link level protocol that addresses NICs by the MAC
address. There can only be one NIC on the network with the same
MAC address. TCP/IP uses IP addresses, not MAC addresses. And you
cannot have two IP addresses on the same link if they have the same
network and broadcast address unless you do multipathing.


I guess that I don't understand what you are trying to achieve.



--
[email protected].

2002-01-11 11:59:40

by Timothy Covell

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

On Thursday 10 January 2002 23:20, Timothy Covell wrote:


Let me clarify what I said earlier. You cannot have
identical MAC addresses on two different NICs. Indeed,
it is impossible w/o trying to fool the kernel into
redefining the NICs hardware based MAC address.

As concerns TCP/IP, you can define two NICs to have the
same IP address, but you will only end up confusing
your switch/HUB router which assumes a 1 to 1 mapping
of MACs to IPs. The whole point of ICMP is to
discover this mapping via ARP requests.

If you want to increase your aggregate bandwidth, then
there are several methods. One is to use the bonding
driver to make the NICs look like one but it needs to
be supported by the switch that you use or for point-to-point
connections, the other box. Another way is to use something
like OSPF or advanced iptable routing to effect multipathing.
Finally, a crude way to increase your effective bandwidth is
to run several instances of your server(s) on different IP
addresses/NICS but they cannot all have default routes on
Linux without resorting to OSPF and/or iptables.


--
[email protected].

2002-01-11 12:08:30

by David Miller

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

From: Timothy Covell <[email protected]>
Date: Fri, 11 Jan 2002 05:55:20 -0600

Let me clarify what I said earlier. You cannot have
identical MAC addresses on two different NICs.

There is nothing illegal about that at all. As long at
the NICs live on different subnets, it is perfectly fine.
In fact this is pretty common on Sun machines.

2002-01-11 12:25:17

by Timothy Covell

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

On Friday 11 January 2002 06:07, David S. Miller wrote:
> From: Timothy Covell <[email protected]>
> Date: Fri, 11 Jan 2002 05:55:20 -0600
>
> Let me clarify what I said earlier. You cannot have
> identical MAC addresses on two different NICs.
>
> There is nothing illegal about that at all. As long at
> the NICs live on different subnets, it is perfectly fine.
> In fact this is pretty common on Sun machines.

True. I was assuming that the context of the post was
that the NICs were on the same network link.

Solaris _defaults_ to using the MAC address from the
primary (hostname) NIC for the rest of them. IMHO, this
is a really stupid thing to do, and I disable it tout de suite
when given a choice. Of course, if you like it, then
why don't you try to convince Linus to change his mind
about it?

--
Surah II 120, Surah II 191-193, Surah V 45, Surah V 51
Surah VIII 12-18, Surah VIII 39-40, Surah VIII 65-69, etc.
--
[email protected].

2002-01-11 12:30:11

by David Miller

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

From: Timothy Covell <[email protected]>
Date: Fri, 11 Jan 2002 06:20:42 -0600

True. I was assuming that the context of the post was
that the NICs were on the same network link.

I didn't catch that bit.

Solaris _defaults_ to using the MAC address from the
primary (hostname) NIC for the rest of them.

This has nothing to do with Solaris, it has to do with
open firmware variable settings. If there is a 'local-mac-address'
property in the device node for the ethernet card, that is what
the drivers use. Otherwise they use the "host MAC".

If you look, this is what we do under Linux on Sparc too.

2002-01-11 12:59:56

by Alan

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

> Let me clarify what I said earlier. You cannot have
> identical MAC addresses on two different NICs. Indeed,
> it is impossible w/o trying to fool the kernel into
> redefining the NICs hardware based MAC address.

Wrong

A mac address is per system. Now in fact almost all systems do it per ethernet
card but that is not what the specifications guarantee. There are machines
out there and cards out there which use the same MAC on all interfaces.

2002-01-11 16:29:21

by Pei Zheng

[permalink] [raw]
Subject: RE: strange kernel message when hacking the NIC driver

Thank you for your reply. Sorry for bring confusion.
Let me clarify what I am trying to do. I am now using same MAC for two NICs.
Each NIC has a unique MAC in the context of my work.

To put it simple, routing simulation,ie.
to simulate(or emulate) 2 "routers" in one kernel. For example, 6 NICs exist
on a machine, 3 of them(eth1-eth3) consist "router A", the others(eth4-eth6)
"router B". Each router has a routing table associated with it. We are
trying to
set a route like entering eth1->kernel->out from eth2->ethernet switch->back
to eth4->kernel->out from eth5. We cannot setup a route directly like this
because the kernel will not send out packets whose nexthop is one of its
network
interfaces. So "ip route add from host_a to host_b via ip_of_eth4 dev eth2"
does
not work, packets never get out of the box before sending out from eth5.

Using iproutes, we can configure that packets entering eth1 is routed to
sending out from eth2 in "router A", and packets entering eth4 is routed to
sending out from eth5 in "router B". No next hop is specified here. Now
packets entering the box through eth2 first go through "router A", then
physically
get out of the box from eth2. We need to redirect them back to eth4 to go
through "router B". That is why I forcibly modify each packet's ethernet
frame
header and change the destination MAC addr to be the other end of NIC of
the link. For example, the dest_MAC_addr of a packet sending out from eth2
is set to MAC of eth4. This hack works. We can see packets come and go
like what we want. But I got some errors in system log like what i post
before.
It seems to be that some of the packets are "wrong" from the kernel's point
of view.

Please correct me if i misunderstand some points of ethernet, the kernel
and IP stack. Any comment to this approach, and how to do "router simulation
in one kernel" are highly appreciated. Thank you!

-Pei

> -----Original Message-----
> From: Timothy Covell [mailto:[email protected]]
> Sent: Thursday, January 10, 2002 9:21 PM
> To: Pei Zheng; [email protected]
> Subject: Re: strange kernel message when hacking the NIC driver
>
>
> On Thursday 10 January 2002 22:19, Pei Zheng wrote:
> > Hi,
> >
> > I've done some simple hackings to the network device driver. Basically
> > i want to connect two NICs as a point-to-point link,ie, packets sending
> > out from one NIC will be redirected to another NIC, sort of direct-link
> > cable between these two NICs.
> > So in the drivers of these two NICs, before hart_start_xmit(), the skb's
> > header will be forcibly set to its peer's MAC address, thus each packey
> > sending out from one NIC will be directly diverted to its peer NIC. This
> > works fine most of the case since tcpdump shows that packets traverse in
> > this way. However, i found errors in system log:
> >
> > Jan 10 18:02:40 em2 kernel: 1130: 09 => cb
> > Jan 10 18:02:40 em2 kernel: 388: f4 => 45
> > Jan 10 18:02:40 em2 kernel: 26: c0 => 7d
> > Jan 10 18:02:40 em2 kernel: 149: 1f => 98
> > Jan 10 18:02:40 em2 kernel: After error applied:
> > Jan 10 18:02:40 em2 kernel: 00 80 c8 b9 6b b6 00 80 c8 b9 6b b6 08 00
> > 45 00
> > Jan 10 18:02:40 em2 kernel: 05 dc 73 cd 20 b9 3f 11 44 26 7d a8 0c 0a
> > c0 a8
> > Jan 10 18:02:40 em2 kernel: 10 0a e3 c1 4c 1f 6f 83 61 dc fc 5e 5c f4
> > 66 17
> > Jan 10 18:02:40 em2 kernel: 2c 73 19 ce cc 2d 69 69 bd 43 33 6d 82 de
> > 4a 87
> > Jan 10 18:02:40 em2 kernel: 8d 9d 81 f6 2e b2 8c 6a d5 e4 f2 e6 bc ab
> > 5a 01
> > Jan 10 18:02:40 em2 kernel: 34 d5 39 f8 d9 5b 16 bc dc 95 bd b4 08 a9
> > 5e 11
> > Jan 10 18:02:40 em2 kernel: df 38 80 8a ca d8 1c 53 65 97 91 4c 84 5a
> > a1 0e
> > Jan 10 18:02:40 em2 kernel: c2 5f d4 02 a1 9e 1c 35 d4 95 08 44 81 16
> > a3 e0
> >
> > there are many this kind of messages. Don't understand what it is. the
> > beginning part of the data seems to be an icmp packet(00 80 c8 b9 6b
> > b6 is the MAC of one NIC). For some reason kernel thinks that this
> > packet is not correct. Any idea about this? If i want to modify a
> > packet's header before it goes to hard_strat_xmit(), what else should
> > i do except setting the skb header to the MAC of the NIC's peer only?
> > checksum stuff?
> >
> > Any helps will be highly appreciated.
> >
> > -Pei
>
>
> You have me confused about whether your trying to build some kind of
> ethernet bridge or some kind of secret packet sniffer or some other
> such thing. It looks like you're mixing up TCP/IP and Ethernet.
>
> Ethernet is the link level protocol that addresses NICs by the MAC
> address. There can only be one NIC on the network with the same
> MAC address. TCP/IP uses IP addresses, not MAC addresses. And you
> cannot have two IP addresses on the same link if they have the same
> network and broadcast address unless you do multipathing.
>
>
> I guess that I don't understand what you are trying to achieve.
>
>
>
> --
> [email protected].

2002-01-11 19:07:32

by Timothy Covell

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

On Friday 11 January 2002 07:11, Alan Cox wrote:
> > Let me clarify what I said earlier. You cannot have
> > identical MAC addresses on two different NICs. Indeed,
> > it is impossible w/o trying to fool the kernel into
> > redefining the NICs hardware based MAC address.
>
> Wrong
>
> A mac address is per system. Now in fact almost all systems do it per
> ethernet card but that is not what the specifications guarantee. There are
> machines out there and cards out there which use the same MAC on all
> interfaces. -

IMHO, they _should_ be unique except for multicast addressing and uses
such as in HSRP/VRRP and such. Network admins usually like to have
a firm grip concerning how traffic is routed. They don't want traffic on
one segment/subnet getting routed to another. IHMO, this is a vector
for viruses proliferation because the host vulnerable thinks that all
segments/subnets are the same.


I don't have the money to shell out for a copy of the IEEE 802.x standards,
but I can quote some other folks on this:


RFC 826:

However, 48.bit Ethernet addresses are supposed to be unique and fixed for
all time, so they shouldn't
change.


>From the Ethernet FAQ:

02.10Q: What is a MAC address?
A: It is the unique hexadecimal serial number assigned to each Ether-
net network device to identify it on the network. With Ethernet
devices (as with most other network types), this address is
permanently set at the time of manufacturer, though it can usually
be changed through software (though this is generally a Very
Bad Thing to do).

02.11Q: Why must the MAC address to be unique?

A: Each card has a unique MAC address, so that
it will be able to exclusively grab packets off the wire
meant for it. If MAC addresses are not unique, there is
no way to distinguish between two stations. Devices on the
network watch network traffic and look for their own MAC address
in each packet to determine whether they should decode it or not.
Special circumstances exist for broadcasting to every device.

04.01Q: What is a "segment"?
A: A piece of network wire bounded by bridges, routers, repeaters or
terminators.

04.02Q: What is a "subnet"?
A: Another overloaded term. It can mean, depending on the usage, a
segment, a set of machines grouped together by a specific protocol
feature (note that these machines do not have to be on the same
segment, but they could be) or a big nylon thing used to capture
enemy subs.







--
[email protected].

2002-01-11 19:24:38

by Paul Walmsley

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

On Fri, 11 Jan 2002, Timothy Covell wrote:

> I don't have the money to shell out for a copy of the IEEE 802.x standards,

The IEEE 802 standards can be downloaded for free from the IEEE.
<http://standards.ieee.org/getieee802/>


- Paul

2002-01-11 22:56:10

by Timothy Covell

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

On Friday 11 January 2002 13:02, Timothy Covell wrote:
> On Friday 11 January 2002 07:11, Alan Cox wrote:
> > > Let me clarify what I said earlier. You cannot have
> > > identical MAC addresses on two different NICs. Indeed,
> > > it is impossible w/o trying to fool the kernel into
> > > redefining the NICs hardware based MAC address.
> >
> > Wrong
> >
> > A mac address is per system. Now in fact almost all systems do it per
> > ethernet card but that is not what the specifications guarantee. There
> > are machines out there and cards out there which use the same MAC on all
> > interfaces. -
>
> IMHO, they _should_ be unique except for multicast addressing and uses
> such as in HSRP/VRRP and such. Network admins usually like to have
> a firm grip concerning how traffic is routed. They don't want traffic on
> one segment/subnet getting routed to another. IHMO, this is a vector
> for viruses proliferation because the host vulnerable thinks that all
> segments/subnets are the same.
>
>

OK, here are some snippets which I found from reading the IEEE 802.x
standards. As far as I can see, the only justification for your and Sun's
(OpenBootProm) method is to assume that all ports will be aggregated.

IEEE 802:
---------
However, due to the shared-medium nature of the IEEE 802 LANs, there is
always a MAC Sublayer.


IEEE 802:
---------

5.1 Organizationally Unique Identifier
5.1.1 Concept.

Organizationally Unique Identifiers allow a general means of assuring unique
identifiers for a
number of purposes. Currently, the IEEE assigns Organizationally Unique
Identifiers to be used
for generating LAN MAC addresses and protocol identifiers. Assuming correct
administration by
the assignee, the LAN MAC addresses and protocol identifiers will be
universally unique.



IEEE 802.1D
-----------

a) A Bridge is not directly addressed by communicating end stations, except
as an end station for
management purposes: frames transmitted between end stations carry the MAC
Address of the peer-end
station in their Destination Address field, not the MAC Address, if any, of
the Bridge.

b) All MAC Addresses must be unique within the Bridged LAN.

c) The MAC Addresses of end stations are not restricted by the topology and
configuration of the Bridged LAN.


IEEE 802.1G
-------------

Preservation of the MAC service The MAC service offered by a remotely bridged
LAN is
similar (see 5.3) to that offered by a single LAN. In consequence

a) A bridge is not directly addressed by communicating end stations, except
as an end station for management purposes: frames transmitted between end
stations carry the MAC address of the peer end station in their Destination
Address fields,
not the MAC address, if any, of a bridge.

b) All MAC addresses are unique and addressable within the bridged LAN.

c) The MAC addresses of end stations are not restricted by the topology and
configuration of the bridged LAN.


IEEE 802.1Q
----------------
B.1.2 Duplicate MAC Addresses

The simplest example of a need for Independent VLAN Learning occurs where
two (or more) distinct devices in different parts of the network reuse
the same individual MAC Address, or where a single device is connected to
multiple LAN segments, and all of its LAN interfaces use the same individual
MAC Address. This is shown in Figure B-3.


[picture ommitted for obvious reasons]

The example shows two clients with access to the same server; both clients
are using the same individual MAC Address, X. If the Bridge shares learning
between VLAN Red (which serves Client A) and VLAN Blue (which serves Client
B),
i.e., the Bridge uses the same FID for both VLANs, then Address X will appear
to
move between Ports 1 and 2 of the Bridge, depending upon which client has
most recently
trans-mitted a frame. Communication between these Clients and the server will
therefore
be seriously disrupted. Assignment of distinct FIDs for Red and Blue ensures
that
communication can take place correctly.




IEEE 802.3
----------
The only thing which I could find that supported you and Sun's
position is in using LACP. Then, one uses the SystemID
(the hostid from eeprom, or one of the port's MAC). So, if you are
assuming that all ports should belong to an aggregate group by
default, then I could understand this point. However, the vast
majority of system's NICs (aside from the QFE cards) are never
aggregated. So, it's a bit of a stretch to using LACP as a default.

Quotations:


i)Each port is assigned a unique,globally administered MAC address.This MAC
address
is used as the source address for frame exchanges that are initiated by
entities within
the Link Aggregation sublayer itself (i.e.,LACP and Marker protocol
exchanges). NOTE
The LACP and Marker protocols use a multicast destination address for all
exchanges,
and do not impose any requirement for a port to recognize more than one
unicast address
on received frames.

j)Each Aggregator is assigned a unique,globally administered MAC address;this
address is
used as the MAC address of the aggregation from the perspective of the MAC
Client, both as
a source address for transmitted frames and as the destination address for
received frames.
The MAC address of the Aggregator may be one of the MAC addresses of a port
in the associated
Link Aggregation Group (see 43.2.10).



43.2.8 Aggregator An Aggregator comprises an instance of a Frame Collection
function,
an instance of a Frame Distribution function and one or more instances of the
Aggregator
Parser/Multiplexer function for a Link Aggregation Group.A single Aggregator
is associated
with each Link Aggregation Group. An Aggregator offers a standard IEEE 802.3
MAC service
interface to its associated MAC Client;access to the MAC service by a MAC
Client is always
achieved via an Aggregator. An Aggregator can therefore be considered to be a
logical MAC ,
bound to one or more ports,through which the MAC client is provided access to
the MAC service.
A single,individual MAC address is associated with each Aggregator (see
43.2.10). An Aggregator is
available for use by the MAC Client if the following are all true:

a)It has one or more attached ports.
b)The Aggregator has not been set to a disabled state by administrative
action (see 30.7.1.1.13).
c)The collection and/or distribution function associated with one or more of
the attached ports is
enabled (see 30.7.1.1.14).

NOTE To simplify the modeling and description of the operation of Link
Aggregation,it is assumed that
there are as many Aggregators as there are ports in a given System; however,
this is not a requirement
of this standard.Aggregation of two or more ports consists of changing the
bindings between ports and
Aggregators such that more than one port is bound to a single Aggregator.The
creation of any aggregations
of two or more links will therefore result in one or more Aggregators that
are bound to more than one port,
and one or more Aggregators that are not bound to any port.An Aggregator that
is not bound to any port appears
to a MAC Client as a MAC interface to an inactive port.During times when the
bindings between ports and Aggregators a
re changing,or as a consequence of particular con guration choices, there
may be occasions when one or more ports
are not bound to any Aggregator.

43.2.10 Addressing Each IEEE 802.3 MAC has an associated globally-unique
individual MAC address,whether that MAC
is used for Link Aggregation or not (see Clause 4). Each Aggregator to which
one or more ports are attached has
an associated globally-unique individual MAC address (see 43.3.3).The MAC
address of the Aggregator may be the
globally-unique individual MAC addresses of one of the MACs in the associated
Link Aggregation Group,or it may
be a distinct MAC address.The manner in which such addresses are chosen is
not otherwise constrained by this standard.
Protocol entities sourcing frames from within the Link Aggregation sublayer
(e.g.,LACP and the Marker protocol)use the
MAC address of the MAC within an underlying port as the source address in
frames transmitted through that port.The MAC
Client sees only the Aggregator and not the underlying MACs,and therefore
uses the Aggregator s MAC address as the source
address in transmitted frames.If a MAC Client submits a frame to the
Aggregator for transmission without specifying a
source address,the Aggregator inserts its own MAC address as the source
address for transmitted frames.



RFC 826:
--------

However, 48.bit Ethernet addresses are supposed to be unique and fixed for
all time, so they shouldn't
change.


>From the Ethernet FAQ:
----------------------

02.10Q: What is a MAC address?
A: It is the unique hexadecimal serial number assigned to each Ether-
net network device to identify it on the network. With Ethernet
devices (as with most other network types), this address is
permanently set at the time of manufacturer, though it can usually
be changed through software (though this is generally a Very
Bad Thing to do).

02.11Q: Why must the MAC address to be unique?

A: Each card has a unique MAC address, so that
it will be able to exclusively grab packets off the wire
meant for it. If MAC addresses are not unique, there is
no way to distinguish between two stations. Devices on the
network watch network traffic and look for their own MAC address
in each packet to determine whether they should decode it or not.
Special circumstances exist for broadcasting to every device.

04.01Q: What is a "segment"?
A: A piece of network wire bounded by bridges, routers, repeaters or
terminators.

04.02Q: What is a "subnet"?
A: Another overloaded term. It can mean, depending on the usage, a
segment, a set of machines grouped together by a specific protocol
feature (note that these machines do not have to be on the same
segment, but they could be) or a big nylon thing used to capture
enemy subs.


--
[email protected].

2002-01-12 21:37:45

by Horst von Brand

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

Timothy Covell <[email protected]> said:
> On Thursday 10 January 2002 23:20, Timothy Covell wrote:

> Let me clarify what I said earlier. You cannot have
> identical MAC addresses on two different NICs.

You sure can. Look at your nearest Sun machine with two NICs for an
example.

> Indeed,
> it is impossible w/o trying to fool the kernel into
> redefining the NICs hardware based MAC address.

It is not "fooling", you can set the MAC address on some cards by software.
Kernel has nothing to do with it.

> As concerns TCP/IP, you can define two NICs to have the
> same IP address, but you will only end up confusing
> your switch/HUB router which assumes a 1 to 1 mapping
> of MACs to IPs. The whole point of ICMP is to
> discover this mapping via ARP requests.

TCP/IP is a 4(5) level protocol stack:

0 Hardware [Cribbed from ISO]
1 Device driver
2 IP [Includes ICMP]
3 TCP and UDP
4 Applications

A switch/hub works on Ethernet frames, i.e., at the hardware level. It has
absolutely no idea of mappings of MACs to IPs.

ICMP is part of the IP layer, the whole of the mapping of IPs to MACs is
(conceptually at least) part of the device driver layer. This includes ARP,
DHCP et al. ICMP has nothing to do with ARP.
--
Horst von Brand http://counter.li.org # 22616

2002-01-12 21:37:45

by Horst von Brand

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

Timothy Covell <[email protected]> said:
> On Friday 11 January 2002 06:07, David S. Miller wrote:
> > From: Timothy Covell <[email protected]>
> > Date: Fri, 11 Jan 2002 05:55:20 -0600
> >
> > Let me clarify what I said earlier. You cannot have
> > identical MAC addresses on two different NICs.
> >
> > There is nothing illegal about that at all. As long at
> > the NICs live on different subnets, it is perfectly fine.
> > In fact this is pretty common on Sun machines.
>
> True. I was assuming that the context of the post was
> that the NICs were on the same network link.

This is not the typical setup...

> Solaris _defaults_ to using the MAC address from the
> primary (hostname) NIC for the rest of them.

Sun sets the MAC from the machine ID on all network interfaces by
default. Not Solaris, AFAIU it is done by the PROM.

> IMHO, this
> is a really stupid thing to do, and I disable it tout de suite
> when given a choice. Of course, if you like it, then
> why don't you try to convince Linus to change his mind
> about it?

Why should DaveM convince Linus to get Sun to change their mind on NIC
setup?

Especially if it works just fine for 99.95% of Suns, and has the bonus that
you can track each machine by a _single_ MAC, even if you change NICs or
add more?
--
Horst von Brand http://counter.li.org # 22616

2002-01-12 21:49:16

by Timothy Covell

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

On Saturday 12 January 2002 15:37, Prof. Brand wrote:
> Timothy Covell <[email protected]> said:
> > On Friday 11 January 2002 06:07, David S. Miller wrote:
> > > From: Timothy Covell <[email protected]>
> > > Date: Fri, 11 Jan 2002 05:55:20 -0600
> > >
> > > Let me clarify what I said earlier. You cannot have
> > > identical MAC addresses on two different NICs.
> > >
> > > There is nothing illegal about that at all. As long at
> > > the NICs live on different subnets, it is perfectly fine.
> > > In fact this is pretty common on Sun machines.
> >
> > True. I was assuming that the context of the post was
> > that the NICs were on the same network link.
>
> This is not the typical setup...
>
> > Solaris _defaults_ to using the MAC address from the
> > primary (hostname) NIC for the rest of them.
>
> Sun sets the MAC from the machine ID on all network interfaces by
> default. Not Solaris, AFAIU it is done by the PROM.
>
> > IMHO, this
> > is a really stupid thing to do, and I disable it tout de suite
> > when given a choice. Of course, if you like it, then
> > why don't you try to convince Linus to change his mind
> > about it?
>
> Why should DaveM convince Linus to get Sun to change their mind on NIC
> setup?

You have it backwards. If you all seem to like the Sun way of doing things,
then Linus should change Linux to do likewise.

>
> Especially if it works just fine for 99.95% of Suns, and has the bonus that
> you can track each machine by a _single_ MAC, even if you change NICs or
> add more?

If you are already keeping a MAC <--> Host table, then it's not very hard to
expand it to keep track of other NICs. That is, of course, if you care about
tracking your inventory.

--
[email protected].

2002-01-12 22:02:36

by Timothy Covell

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

On Saturday 12 January 2002 15:37, Prof. Brand wrote:
> Timothy Covell <[email protected]> said:
> > On Thursday 10 January 2002 23:20, Timothy Covell wrote:
> >
> > Let me clarify what I said earlier. You cannot have
> > identical MAC addresses on two different NICs.
>
> You sure can. Look at your nearest Sun machine with two NICs for an
> example.

And why don't you connect those two NICs with the same MACs to
the same HUB and see how well that works.


>
> > Indeed,
> > it is impossible w/o trying to fool the kernel into
> > redefining the NICs hardware based MAC address.
>
> It is not "fooling", you can set the MAC address on some cards by software.
> Kernel has nothing to do with it.
>
> > As concerns TCP/IP, you can define two NICs to have the
> > same IP address, but you will only end up confusing
> > your switch/HUB router which assumes a 1 to 1 mapping
> > of MACs to IPs. The whole point of ICMP is to
> > discover this mapping via ARP requests.
>
> TCP/IP is a 4(5) level protocol stack:
>
> 0 Hardware [Cribbed from ISO]
> 1 Device driver
> 2 IP [Includes ICMP]
> 3 TCP and UDP
> 4 Applications
>
> A switch/hub works on Ethernet frames, i.e., at the hardware level. It has
> absolutely no idea of mappings of MACs to IPs.
>
> ICMP is part of the IP layer, the whole of the mapping of IPs to MACs is
> (conceptually at least) part of the device driver layer. This includes ARP,
> DHCP et al. ICMP has nothing to do with ARP.


If you have two NICs with the identical MAC address on the same segment,
your ethernet routing will become confused. The ARP table keeps a map
of MAC to IP address. This table assumes a 1 to 1 relationship. If you
try to reach a host but fail due to routing issues, then ICMP sends out
DESTINATION HOST UNKNOWN/HOST UNREACHABLE messages.
So, ICMP and very much to do with ARP.

The whole ISO Model is flawed because there really is not
a clean delineation among the layers. For example, BGP uses TCP
to route IP, so it's an ISO layer 3 process, but it makes use of TCP
which is layer 4. So, you need layer 4 to define layer 3. Crazy huh.
The same argument can be shifted to the Internet Model, which is
IHMO, an after-the-fact idea that came from trying to apply ISO to
Internet.


--
[email protected].

2002-01-14 06:34:21

by Zwane Mwaikambo

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

Timothy Covell <[email protected]> said:
>If you try to reach a host but fail due to routing issues, then ICMP sends out
>DESTINATION HOST UNKNOWN/HOST UNREACHABLE messages.
>So, ICMP and very much to do with ARP.

Using your argument... Everytime my feline decides to chew my ethernet
cable i lose my network connection, hence i get ICMP HOST UNREACHABLE ergo
my cat has very much to do with ARP.

Those are errors just filtering up the layers, same way an app doesn't
know anything about what device you're writing to, but will complain about
incomplete data writes if you run out of space.

Regards,
Zwane Mwaikambo


2002-01-14 07:41:29

by Bernd Eckenfels

[permalink] [raw]
Subject: Re: strange kernel message when hacking the NIC driver

In article <[email protected]> you wrote:
> Using your argument... Everytime my feline decides to chew my ethernet
> cable i lose my network connection, hence i get ICMP HOST UNREACHABLE ergo
> my cat has very much to do with ARP.

> Those are errors just filtering up the layers, same way an app doesn't
> know anything about what device you're writing to, but will complain about
> incomplete data writes if you run out of space.

Well actually, the neighbour alive discovery done by arp influences the
routing cache very much. Therefore recent Linux networking is actually able to
send "Host unreachable" messages based on the fact, that an host is down.

You are right, that it is not closely related, but it is something one should
not forget, since it is a behaviour introduced in 2.2 kernels, I think.

You can check it with "ip neig". Entries which have an unreachable host are
marked with "nud failed".

Greetings
Bernd