2003-03-28 21:59:37

by Dave Jones

[permalink] [raw]
Subject: NICs trading places ?

I just upgraded a box with 2 NICs in it to 2.5.66, and found
that what was eth0 in 2.4 is now eth1, and vice versa.
Is this phenomenon intentional ? documented ?
What caused it to do this ?

The box in question has a DEC Tulip and a 3com 3c905,
but I imagine this would affect any system with >1 NIC
of different vendors/drivers ?

Dave


2003-03-28 22:04:32

by jjs

[permalink] [raw]
Subject: Re: NICs trading places ?

That's bitten me several times, not
just 2.4 -> 2.5 but also within different
versions of 2.4-RHL :-(

I've head you can spell out pci IDs
in modules.conf to nail it down but
I haven't gotten around to trying it -

Joe

Dave Jones wrote:

>I just upgraded a box with 2 NICs in it to 2.5.66, and found
>that what was eth0 in 2.4 is now eth1, and vice versa.
>Is this phenomenon intentional ? documented ?
>What caused it to do this ?
>
>The box in question has a DEC Tulip and a 3com 3c905,
>but I imagine this would affect any system with >1 NIC
>of different vendors/drivers ?
>
> Dave
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to [email protected]
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at http://www.tux.org/lkml/
>
>
>


2003-03-28 22:03:35

by Jeff Garzik

[permalink] [raw]
Subject: Re: NICs trading places ?

On Fri, Mar 28, 2003 at 10:10:37PM +0000, Dave Jones wrote:
> I just upgraded a box with 2 NICs in it to 2.5.66, and found
> that what was eth0 in 2.4 is now eth1, and vice versa.
> Is this phenomenon intentional ? documented ?
> What caused it to do this ?

That's a bug that no one has tracked down yet.

Jamal reported it, too.

ethX numbering shouldn't be changing without a good reason...

Jeff



2003-03-28 22:16:40

by Joel Becker

[permalink] [raw]
Subject: Re: NICs trading places ?

On Fri, Mar 28, 2003 at 10:10:37PM +0000, Dave Jones wrote:
> I just upgraded a box with 2 NICs in it to 2.5.66, and found
> that what was eth0 in 2.4 is now eth1, and vice versa.
> Is this phenomenon intentional ? documented ?
> What caused it to do this ?

Is this a Red Hat system? I encountered the same thing on a
RHAS system. Basically, Anaconda had controlled the module load order
in /etc/modules.conf for 2.4. Because my network drivers were built in
in 2.5, they loaded in the order of the compile-in. This turned out to
be the reverse order.
Swapping eth0 and eth1 in /etc/modules.conf fixed the problem
for me. This is not to say it is "Red Hat's fault" or that this is
entirely the same situation, but I figured this would make a good
datapoint.

Joel

--

"It is not the function of our government to keep the citizen from
falling into error; it is the function of the citizen to keep the
government from falling into error."
- Robert H. Jackson

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: [email protected]
Phone: (650) 506-8127

2003-03-28 22:20:43

by Randy.Dunlap

[permalink] [raw]
Subject: Re: NICs trading places ?

On Fri, 28 Mar 2003 14:25:25 -0800 Joel Becker <[email protected]> wrote:

| On Fri, Mar 28, 2003 at 10:10:37PM +0000, Dave Jones wrote:
| > I just upgraded a box with 2 NICs in it to 2.5.66, and found
| > that what was eth0 in 2.4 is now eth1, and vice versa.
| > Is this phenomenon intentional ? documented ?
| > What caused it to do this ?
|
| Is this a Red Hat system? I encountered the same thing on a
| RHAS system. Basically, Anaconda had controlled the module load order
| in /etc/modules.conf for 2.4. Because my network drivers were built in
| in 2.5, they loaded in the order of the compile-in. This turned out to
| be the reverse order.
| Swapping eth0 and eth1 in /etc/modules.conf fixed the problem
| for me. This is not to say it is "Red Hat's fault" or that this is
| entirely the same situation, but I figured this would make a good
| datapoint.

I saw this same problem at home last night, not on a RH system,
so I think it's just a 2.5.lately thing.

--
~Randy

2003-03-28 22:23:49

by Dave Jones

[permalink] [raw]
Subject: Re: NICs trading places ?

On Fri, Mar 28, 2003 at 02:25:25PM -0800, Joel Becker wrote:
> Is this a Red Hat system? I encountered the same thing on a
> RHAS system.

I've seen the same issue on a SuSE box, and a Debian box.

> Basically, Anaconda had controlled the module load order
> in /etc/modules.conf for 2.4. Because my network drivers were built in
> in 2.5, they loaded in the order of the compile-in. This turned out to
> be the reverse order.

NICs built into kernel in both 2.4 and 2.5.

Dave

2003-03-28 22:37:34

by Andries Brouwer

[permalink] [raw]
Subject: Re: NICs trading places ?

On Fri, Mar 28, 2003 at 10:10:37PM +0000, Dave Jones wrote:

> I just upgraded a box with 2 NICs in it to 2.5.66, and found
> that what was eth0 in 2.4 is now eth1, and vice versa.
> Is this phenomenon intentional ? documented ?
> What caused it to do this ?
>
> The box in question has a DEC Tulip and a 3com 3c905,
> but I imagine this would affect any system with >1 NIC
> of different vendors/drivers ?

Intentional? No.
Documented? I suppose you can find complaints from others
mentioning the same thing.
Cause? eth discovery order is not well-defined.

Once or twice I have submitted patches to rectify.
They help for a while and then someone breaks things again.
I am not quite sure, apologies in case I misremember, but
maybe the most recent breakage was caused by Marc Zyngier
with EISA bus changes.

Google turns up
http://www.uwsg.iu.edu/hypermail/linux/kernel/0301.2/1139.html

Let me add the patch referred to there. Maybe Jeff likes it this time.

Andries


diff -u --recursive --new-file -X /linux/dontdiff a/drivers/net/3c59x.c b/drivers/net/3c59x.c
--- a/drivers/net/3c59x.c Sat Jan 18 23:54:39 2003
+++ b/drivers/net/3c59x.c Tue Jan 21 18:36:25 2003
@@ -1439,8 +1439,14 @@
acpi_set_WOL(dev);
}
retval = register_netdev(dev);
- if (retval == 0)
+ if (retval == 0) {
+ int i;
+ printk("%s: 3c59x, address", dev->name);
+ for (i = 0; i < 6; i++)
+ printk("%c%2.2x", i ? ':' : ' ', dev->dev_addr[i]);
+ printk("\n");
return 0;
+ }

free_ring:
pci_free_consistent(pdev,


2003-03-28 22:51:31

by Andrew Morton

[permalink] [raw]
Subject: Re: NICs trading places ?

Andries Brouwer <[email protected]> wrote:
>
> Cause? eth discovery order is not well-defined.

It's a continual irritation.

> + if (retval == 0) {
> + int i;
> + printk("%s: 3c59x, address", dev->name);
> + for (i = 0; i < 6; i++)
> + printk("%c%2.2x", i ? ':' : ' ', dev->dev_addr[i]);
> + printk("\n");
> return 0;
> + }

hm. typing `ifconfig' shows this information.

2003-03-28 23:09:13

by Andries Brouwer

[permalink] [raw]
Subject: Re: NICs trading places ?

On Fri, Mar 28, 2003 at 03:02:34PM -0800, Andrew Morton wrote:
> Andries Brouwer <[email protected]> wrote:
>
> > + if (retval == 0) {
> > + int i;
> > + printk("%s: 3c59x, address", dev->name);
> > + for (i = 0; i < 6; i++)
> > + printk("%c%2.2x", i ? ':' : ' ', dev->dev_addr[i]);
> > + printk("\n");
> > return 0;
> > + }
>
> hm. typing `ifconfig' shows this information.

I like uniformity.
Either all eth cards should have their address in dmesg, or none.
Almost all do.

And: ifconfig does not give the card types.
So presently one needs both boot messages and ifconfig.

And: in some situations the system does not boot because of
eth numbering mixup, and one never gets the opportunity to
ask ifconfig.

Andries

2003-03-28 23:19:06

by Dax Kelson

[permalink] [raw]
Subject: Re: NICs trading places ?

On Fri, 28 Mar 2003, Dave Jones wrote:

> I just upgraded a box with 2 NICs in it to 2.5.66, and found
> that what was eth0 in 2.4 is now eth1, and vice versa.
> Is this phenomenon intentional ? documented ?
> What caused it to do this ?

I've seen the same thing for 5+ years. Multiple things can
trigger it, eg:

Always having your NIC drivers compiled in, and then moving between
different kernel versions.

Going from modular NIC drivers to compiled in drivers can bite you.

Isn't life fun.

Dax

2003-03-28 23:31:15

by Greg KH

[permalink] [raw]
Subject: Re: NICs trading places ?

On Sat, Mar 29, 2003 at 12:20:22AM +0100, Andries Brouwer wrote:
>
> And: ifconfig does not give the card types.
> So presently one needs both boot messages and ifconfig.
>
> And: in some situations the system does not boot because of
> eth numbering mixup, and one never gets the opportunity to
> ask ifconfig.

ifconfig can bind cards to devices based on mac addresses.
/sbin/hotplug can also be used for this.

I recommend doing this for anyone with more than one nic card in their
machine.

thanks,

greg k-h

2003-03-28 23:31:36

by Sean Neakums

[permalink] [raw]
Subject: Re: NICs trading places ?

commence Dax Kelson quotation:

> On Fri, 28 Mar 2003, Dave Jones wrote:
>
>> I just upgraded a box with 2 NICs in it to 2.5.66, and found
>> that what was eth0 in 2.4 is now eth1, and vice versa.
>> Is this phenomenon intentional ? documented ?
>> What caused it to do this ?
>
> I've seen the same thing for 5+ years. Multiple things can
> trigger it, eg:
>
> Always having your NIC drivers compiled in, and then moving between
> different kernel versions.
>
> Going from modular NIC drivers to compiled in drivers can bite you.
>
> Isn't life fun.

Something I have been meaning to write as soon as I get bitten by this
myself is a script that uses the "ip link set X name Y" command
(possibly in conjuction with a MAC-to-name map, or else just a simple
sort) to name the interfaces consistently.

--
Sean Neakums - <[email protected]>

2003-03-28 23:44:54

by Jeff Garzik

[permalink] [raw]
Subject: Re: NICs trading places ?

Greg KH wrote:
> On Sat, Mar 29, 2003 at 12:20:22AM +0100, Andries Brouwer wrote:
>
>>And: ifconfig does not give the card types.
>>So presently one needs both boot messages and ifconfig.
>>
>>And: in some situations the system does not boot because of
>>eth numbering mixup, and one never gets the opportunity to
>>ask ifconfig.
>
>
> ifconfig can bind cards to devices based on mac addresses.
> /sbin/hotplug can also be used for this.
>
> I recommend doing this for anyone with more than one nic card in their
> machine.


Actually nameif(8) is the preferred way of doing this.

Jeff



2003-03-28 23:53:34

by Greg KH

[permalink] [raw]
Subject: Re: NICs trading places ?

On Fri, Mar 28, 2003 at 06:56:22PM -0500, Jeff Garzik wrote:
> Greg KH wrote:
> >On Sat, Mar 29, 2003 at 12:20:22AM +0100, Andries Brouwer wrote:
> >
> >>And: ifconfig does not give the card types.
> >>So presently one needs both boot messages and ifconfig.
> >>
> >>And: in some situations the system does not boot because of
> >>eth numbering mixup, and one never gets the opportunity to
> >>ask ifconfig.
> >
> >
> >ifconfig can bind cards to devices based on mac addresses.
> >/sbin/hotplug can also be used for this.
> >
> >I recommend doing this for anyone with more than one nic card in their
> >machine.
>
>
> Actually nameif(8) is the preferred way of doing this.

Oops, sorry, that's what I meant. Call nameif in a /sbin/hotplug script
when the network interface is registered.

greg k-h

2003-03-29 04:38:09

by Andi Kleen

[permalink] [raw]
Subject: Re: NICs trading places ?

Dave Jones <[email protected]> writes:

> I just upgraded a box with 2 NICs in it to 2.5.66, and found
> that what was eth0 in 2.4 is now eth1, and vice versa.
> Is this phenomenon intentional ? documented ?

Just assign mac addresses to names and run nameif early in boot.

-Andi

2003-03-29 09:13:19

by Marc Zyngier

[permalink] [raw]
Subject: Re: NICs trading places ?

>>>>> "Andries" == Andries Brouwer <[email protected]> writes:

Andries> I am not quite sure, apologies in case I misremember, but
Andries> maybe the most recent breakage was caused by Marc Zyngier
Andries> with EISA bus changes.

Well, it is not exactly related to EISA bus, although part of the same
patch.

The real problem is that some old drivers are still initialized from
Space.c (they simply do not know about init_etherdev (NULL, ...),
after all these years). So yes, I broke 3c509 in this respect. I also
broke znet if that matters. Oh, and the EISA part of depca in the AC
tree.

So the questions are :
- Should Space.c die ? I think so.
- Should 2.5 be the place and time to kill it ? I also think so.

I know it is a pain for most of us to have interfaces being
renumbered. But relying on something as static as Space.c is the wrong
answer IMHO.

M.
--
Places change, faces change. Life is so very strange.

2003-03-29 12:06:39

by bert hubert

[permalink] [raw]
Subject: NIC renaming does not rename /proc/sys/net/ipv4 Was: Re: NICs trading places ?

On Sat, Mar 29, 2003 at 05:47:17AM +0100, Andi Kleen wrote:
> Dave Jones <[email protected]> writes:
>
> > I just upgraded a box with 2 NICs in it to 2.5.66, and found
> > that what was eth0 in 2.4 is now eth1, and vice versa.
> > Is this phenomenon intentional ? documented ?
>
> Just assign mac addresses to names and run nameif early in boot.

A slight problem with that is that not all parts of /proc/sys get renamed
this way:

snapcount:/proc/sys/net/ipv4/conf# ifconfig lo down
snapcount:/proc/sys/net/ipv4/conf# ip link set name lo0 lo
snapcount:/proc/sys/net/ipv4/conf# ls -l
total 0
dr-xr-xr-x 2 root root 0 Mar 29 13:16 all
dr-xr-xr-x 2 root root 0 Mar 29 13:16 default
dr-xr-xr-x 2 root root 0 Mar 29 13:16 eth0
dr-xr-xr-x 2 root root 0 Mar 29 13:16 lo

Which can be very confusing. This problem exists in both 2.5 and 2.4.

Regards,

bert


--
http://www.PowerDNS.com Open source, database driven DNS Software
http://lartc.org Linux Advanced Routing & Traffic Control HOWTO

2003-03-29 12:18:04

by Andi Kleen

[permalink] [raw]
Subject: Re: NIC renaming does not rename /proc/sys/net/ipv4 Was: Re: NICs trading places ?

On Sat, 2003-03-29 at 13:17, bert hubert wrote:
> On Sat, Mar 29, 2003 at 05:47:17AM +0100, Andi Kleen wrote:
> > Dave Jones <[email protected]> writes:
> >
> > > I just upgraded a box with 2 NICs in it to 2.5.66, and found
> > > that what was eth0 in 2.4 is now eth1, and vice versa.
> > > Is this phenomenon intentional ? documented ?
> >
> > Just assign mac addresses to names and run nameif early in boot.
>
> A slight problem with that is that not all parts of /proc/sys get renamed
> this way:

Just rename at early boot before IP is set up. That is what i usually
do - set up /etc/mactab and run it very early at boot.

Running it later is usually flakey. e.g. it can also give confusing
effects with old style named ip aliases.

-Andi


2003-03-29 14:14:03

by Wichert Akkerman

[permalink] [raw]
Subject: Re: NIC renaming does not rename /proc/sys/net/ipv4 Was: Re: NICs trading places ?

Previously Andi Kleen wrote:
> Just rename at early boot before IP is set up. That is what i usually
> do - set up /etc/mactab and run it very early at boot.

How does that solve the problem of /proc/sys/net/*/conf/* not being
renamed?

Another problem is that nameif only supports ethernet devices currently,
making it an incomplete solution.

Wichert.

--
Wichert Akkerman <[email protected]> http://www.wiggy.net/
A random hacker

2003-03-29 17:39:25

by Andi Kleen

[permalink] [raw]
Subject: Re: NIC renaming does not rename /proc/sys/net/ipv4 Was: Re: NICs trading places ?

Wichert Akkerman <[email protected]> writes:

> > Just rename at early boot before IP is set up. That is what i usually
> > do - set up /etc/mactab and run it very early at boot.
>
> How does that solve the problem of /proc/sys/net/*/conf/* not being
> renamed?

They are only set up when the inet device structure is created.
That happens at the first ifconfig/ip, not on driver init.

As for non ethernet devices - if you find a reliably way to identify/number
them then the same principle can be used for them too.

-Andi