2002-03-23 01:24:21

by Shane Nay

[permalink] [raw]
Subject: Tyan S2466 MPX integrated ethernet interrupt happy

I have an S2466 with an integrated 3COM 3C905C ethernet controller.
What I've noticed when streaming tons of data accross my internal LAN
is that the ethernet driver appears to be sucking up tons of cycles.

A quick investigation of /proc/interrupts shows-
0: 195949 192414 IO-APIC-edge timer
1: 5126 5129 IO-APIC-edge keyboard
2: 0 0 XT-PIC cascade
5: 0 0 IO-APIC-level usb-ohci
8: 1 0 IO-APIC-edge rtc
9: 4248819 4245969 IO-APIC-level eth0
10: 144865 145243 IO-APIC-level usb-ohci, nvidia, EMU10K1
12: 48546 48865 IO-APIC-edge PS/2 Mouse
14: 122395 121652 IO-APIC-edge ide0
15: 26286 26498 IO-APIC-edge ide1
NMI: 0 0
LOC: 388204 388212
ERR: 0
MIS: 4


So, approximately 8.5 million ethernet interrupts. The system is
noticably slower when streaming ethernet data, and it's sucking up a
lot more processing time than on my other much slower other box.
This box is running a stock 2.4.18 kernel from kernel.org (i.e. no
custom hacks). It's running 2 1800+ XPs.

I'm wondering if anyone has seen this and has a quick solution. If
not I'll take a look through the 3com code when I get back from
vacation next Sunday. (Leaving in a few hours)

Thanks,
Shane Nay.


2002-03-23 01:57:30

by Andrew Morton

[permalink] [raw]
Subject: Re: Tyan S2466 MPX integrated ethernet interrupt happy

Shane Nay wrote:
>
> I have an S2466 with an integrated 3COM 3C905C ethernet controller.
> What I've noticed when streaming tons of data accross my internal LAN
> is that the ethernet driver appears to be sucking up tons of cycles.
>
> A quick investigation of /proc/interrupts shows-
> 0: 195949 192414 IO-APIC-edge timer
> 1: 5126 5129 IO-APIC-edge keyboard
> 2: 0 0 XT-PIC cascade
> 5: 0 0 IO-APIC-level usb-ohci
> 8: 1 0 IO-APIC-edge rtc
> 9: 4248819 4245969 IO-APIC-level eth0
> 10: 144865 145243 IO-APIC-level usb-ohci, nvidia, EMU10K1
> 12: 48546 48865 IO-APIC-edge PS/2 Mouse
> 14: 122395 121652 IO-APIC-edge ide0
> 15: 26286 26498 IO-APIC-edge ide1
> NMI: 0 0
> LOC: 388204 388212
> ERR: 0
> MIS: 4
>
> So, approximately 8.5 million ethernet interrupts. The system is
> noticably slower when streaming ethernet data, and it's sucking up a
> lot more processing time than on my other much slower other box.
> This box is running a stock 2.4.18 kernel from kernel.org (i.e. no
> custom hacks). It's running 2 1800+ XPs.

That all looks normal. Could you be more specific about
the performance problems? How much slower? Output from
`top' and `ps'? Any nasty messages in the system log?

Looking at the ethernet driver won't help, I expect - it's
as efficient as most any other driver. If there is a problem,
it lies elsewhere...

A kernel profile would be illuminating. Build the kernel with as few
modules as possible, boot the machine with `profile=1' and play
with readprofile.

-

2002-03-23 03:25:20

by Bill Davidsen

[permalink] [raw]
Subject: Re: Tyan S2466 MPX integrated ethernet interrupt happy

On Fri, 22 Mar 2002, Shane Nay wrote:

> I have an S2466 with an integrated 3COM 3C905C ethernet controller.
> What I've noticed when streaming tons of data accross my internal LAN
> is that the ethernet driver appears to be sucking up tons of cycles.

The table is just the number of ints, see below for one of my boxen which
is running 200-250 users and has idle of ~92% over three days.

> A quick investigation of /proc/interrupts shows-
> 0: 195949 192414 IO-APIC-edge timer
> 1: 5126 5129 IO-APIC-edge keyboard
> 2: 0 0 XT-PIC cascade
> 5: 0 0 IO-APIC-level usb-ohci
> 8: 1 0 IO-APIC-edge rtc
> 9: 4248819 4245969 IO-APIC-level eth0
> 10: 144865 145243 IO-APIC-level usb-ohci, nvidia, EMU10K1
> 12: 48546 48865 IO-APIC-edge PS/2 Mouse
> 14: 122395 121652 IO-APIC-edge ide0
> 15: 26286 26498 IO-APIC-edge ide1
> NMI: 0 0
> LOC: 388204 388212
> ERR: 0
> MIS: 4

newssvr17:twister$ cat /proc/interrupts
CPU0 CPU1
0: 14112007 14110727 IO-APIC-edge timer
1: 473 467 IO-APIC-edge keyboard
2: 0 0 XT-PIC cascade
8: 0 1 IO-APIC-edge rtc
12: 104 105 IO-APIC-edge PS/2 Mouse
14: 3 2 IO-APIC-edge ide0
18: 88268186 88280808 IO-APIC-level eth0
20: 6655721 6658714 IO-APIC-level ips
27: 74212944 74213812 IO-APIC-level PCnet/FAST III 79C975
28: 14 2 IO-APIC-level aic7xxx
29: 10 6 IO-APIC-level aic7xxx
NMI: 0 0
LOC: 28223102 28223059
ERR: 0
MIS: 0

--
bill davidsen <[email protected]>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

2002-03-23 13:44:48

by Shane Nay

[permalink] [raw]
Subject: Re: Tyan S2466 MPX integrated ethernet interrupt happy


> > So, approximately 8.5 million ethernet interrupts. The system is
> > noticably slower when streaming ethernet data, and it's sucking
> > up a lot more processing time than on my other much slower other
> > box. This box is running a stock 2.4.18 kernel from kernel.org
> > (i.e. no custom hacks). It's running 2 1800+ XPs.
>
> That all looks normal. Could you be more specific about
> the performance problems? How much slower? Output from
> `top' and `ps'? Any nasty messages in the system log?

No nasties in the syslog. System usage from top averages 50% per
CPU. (By system I mean kernel usage, bounces between 70 and 30 per
CPU)

> Looking at the ethernet driver won't help, I expect - it's
> as efficient as most any other driver. If there is a problem,
> it lies elsewhere...

Gotcha, yes that could definetly be. This was extraordinarily
unscientific. I was thinking maybe someone had seen something else
like this on this platform. So, I'll write a test that isolates
ethernet from other stuff like ide performance, etc.

> A kernel profile would be illuminating. Build the kernel with as
> few modules as possible, boot the machine with `profile=1' and play
> with readprofile.

Will do when I return.

Thanks,
Shane Nay.
(Someone else noted their /proc/interrupts for the same machine, but
my box had only been up for 1.5 hours :)