Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751529AbWAIG4r (ORCPT ); Mon, 9 Jan 2006 01:56:47 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751526AbWAIG4r (ORCPT ); Mon, 9 Jan 2006 01:56:47 -0500 Received: from relay02.mail-hub.dodo.com.au ([202.136.32.45]:60088 "EHLO relay02.mail-hub.dodo.com.au") by vger.kernel.org with ESMTP id S1751193AbWAIG4q (ORCPT ); Mon, 9 Jan 2006 01:56:46 -0500 From: Grant Coady To: Jesse Brandeburg Cc: Bernd Eckenfels , linux-kernel@vger.kernel.org, Willy Tarreau Subject: Re: Why is 2.4.32 four times faster than 2.6.14.6?? Date: Mon, 09 Jan 2006 17:56:42 +1100 Organization: http://bugsplatter.mine.nu/ Reply-To: gcoady@gmail.com Message-ID: References: <20060108095741.GH7142@w.ods.org> <4807377b0601081837u2c1d50b3w218d5ef9e3dc662@mail.gmail.com> In-Reply-To: <4807377b0601081837u2c1d50b3w218d5ef9e3dc662@mail.gmail.com> X-Mailer: Forte Agent 2.0/32.652 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 8298 Lines: 190 On Sun, 8 Jan 2006 18:37:52 -0800, Jesse Brandeburg wrote: Added Willy to Cc: -- I see a problem with 2 e100 NICs' accounting under 2.4.32... See near and of this report. >> Are rx checksums not turned on in 2.6' e100 driver? >> CPU is only pentium/mmx 233 > >Hey Grant, to answer your question, checksums are not offloaded with >the current e100 driver but that really shouldn't make that much of a >difference. I'm actually going to go with interrupt load due to e100 >being at least related to the problem. > >The netdev-2.6 git tree currently has a driver that supports microcode >loading for your rev 8 PRO/100 and that microcode may help your >interrupt load due to e100. however, it may already be loading. >Also, what do you have HZ set to? (250 is default in 2.6, 1000 in 2.4) >so you could try running your 2.6 kernel with HZ=1000 > >while you're running your test you could try (if you have sysstat) >sar -I 1 10 > >or a simpler version, 10 loops of cat /proc/interrupts; sleep 1; No sar, now I'm running a separate link from the other e100 eth1 from deltree to another box so measurement and test traffic are separated. I do everything via ssh so I can copy/paste from terminals ;) This run is 2.6.15a, with 100Hz and voluntary preempt: grant@deltree:~$ cat /proc/interrupts CPU0 0: 106221 XT-PIC timer 1: 8 XT-PIC i8042 2: 0 XT-PIC cascade 3: 11443 XT-PIC eth2 <<== ADSL modem 8: 1 XT-PIC rtc 11: 20402 XT-PIC eth0 <<== localnet 12: 21860 XT-PIC eth1 <<== spare -> test 14: 3260 XT-PIC ide0 NMI: 0 ERR: 0 grant@deltree:~$ grant@deltree:~$ while true; do grep eth1 /proc/interrupts; sleep 1; done 12: 26357 XT-PIC eth1 12: 26357 XT-PIC eth1 \ 12: 26573 XT-PIC eth1 | 12: 27039 XT-PIC eth1 > time grep -v 192\.168\. \ 12: 27514 XT-PIC eth1 | /var/log/apache/access_log \ 12: 28320 XT-PIC eth1 | | cut -c-96 12: 29090 XT-PIC eth1 | real 0m6.205s 12: 30017 XT-PIC eth1 | user 0m0.620s 12: 30434 XT-PIC eth1 / sys 0m0.730s 12: 30434 XT-PIC eth1 grant@deltree:~$ while true; do grep eth1 /proc/interrupts; sleep 1; done 12: 30650 XT-PIC eth1 12: 30651 XT-PIC eth1 \ 12: 30657 XT-PIC eth1 | 12: 30661 XT-PIC eth1 > time cat /var/log/apache/access_log 12: 30936 XT-PIC eth1 | real 0m2.383s 12: 31343 XT-PIC eth1 | user 0m0.010s 12: 31593 XT-PIC eth1 / sys 0m0.480s 12: 31593 XT-PIC eth1 This run is 2.6.15b, with 1000Hz and voluntary preempt: grant@deltree:~$ while true; do grep eth1 /proc/interrupts; sleep 1; done 12: 4386 XT-PIC eth1 12: 4386 XT-PIC eth1 \ 12: 4427 XT-PIC eth1 | 12: 4904 XT-PIC eth1 > time grep -v 192\.168\. \ 12: 5350 XT-PIC eth1 | /var/log/apache/access_log \ 12: 6065 XT-PIC eth1 | | cut -c-96 12: 6906 XT-PIC eth1 | real 0m6.649s 12: 7693 XT-PIC eth1 | user 0m0.841s 12: 8450 XT-PIC eth1 / sys 0m1.047s 12: 8548 XT-PIC eth1 12: 8548 XT-PIC eth1 ran above a few times to gauge repeatability, variation ~200ms in real. grant@deltree:~$ while true; do grep eth1 /proc/interrupts; sleep 1; done 12: 18347 XT-PIC eth1 12: 18348 XT-PIC eth1 \ 12: 18417 XT-PIC eth1 | 12: 18794 XT-PIC eth1 > time cat /var/log/apache/access_log 12: 19181 XT-PIC eth1 | real 0m2.573s 12: 19283 XT-PIC eth1 / user 0m0.005s 12: 19284 XT-PIC eth1 sys 0m0.547s No joy with 1000Hz, turn off preempt? This run is 2.6.15c, with 1000Hz and no preempt: grant@deltree:~$ while true; do grep eth1 /proc/interrupts; sleep 1; done 12: 4400 XT-PIC eth1 12: 4400 XT-PIC eth1 \ 12: 4614 XT-PIC eth1 | 12: 5053 XT-PIC eth1 > time grep -v 192\.168\. \ 12: 5495 XT-PIC eth1 | /var/log/apache/access_log \ 12: 6686 XT-PIC eth1 | | cut -c-96 12: 7394 XT-PIC eth1 | real 0m6.696s 12: 8258 XT-PIC eth1 | user 0m0.841s 12: 8456 XT-PIC eth1 / sys 0m0.994s 12: 8457 XT-PIC eth1 12: 8457 XT-PIC eth1 grant@deltree:~$ while true; do grep eth1 /proc/interrupts; sleep 1; done 12: 8544 XT-PIC eth1 \ 12: 8814 XT-PIC eth1 | time cat /var/log/apache/access_log 12: 9485 XT-PIC eth1 > real 0m2.511s 12: 9486 XT-PIC eth1 | user 0m0.004s 12: 9486 XT-PIC eth1 / sys 0m0.529s Still no joy? Confirm 2.4 timings, this is with 2.4.32-hf32.1: grant@deltree:~$ while true; do egrep 'eth0|eth1' /proc/interrupts; sleep 1; done 11: 6744 XT-PIC eth0 12: 4 XT-PIC eth1 11: 6746 XT-PIC eth0 \ 12: 4 XT-PIC eth1 | 11: 7178 XT-PIC eth0 > time grep -v 192\.168\. \ 12: 4 XT-PIC eth1 | /var/log/apache/access_log \ 11: 7552 XT-PIC eth0 | | cut -c-96 12: 4 XT-PIC eth1 / real 0m1.565s 11: 7554 XT-PIC eth0 user 0m0.510s 12: 4 XT-PIC eth1 sys 0m0.340s grant@deltree:~$ while true; do egrep 'eth0|eth1' /proc/interrupts; sleep 1; done 11: 9136 XT-PIC eth0 \ 12: 4 XT-PIC eth1 | 11: 9516 XT-PIC eth0 > time cat /var/log/apache/access_log 12: 4 XT-PIC eth1 | real 0m1.946s 11: 10146 XT-PIC eth0 | user 0m0.000s 12: 4 XT-PIC eth1 / sys 0m0.200s 11: 10321 XT-PIC eth0 12: 4 XT-PIC eth1 Odd, with 2.4, the two e100 NICs are not being accounted properly: root@deltree:~# ifconfig eth0 Link encap:Ethernet HWaddr 00:90:27:42:AA:77 inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4840 errors:0 dropped:0 overruns:0 frame:0 TX packets:8825 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:341812 (333.8 Kb) TX bytes:9931009 (9.4 Mb) Interrupt:11 Base address:0xdcc0 Memory:fd201000-fd201038 eth1 Link encap:Ethernet HWaddr 00:90:27:58:32:D4 inet addr:192.168.2.1 Bcast:192.168.2.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Interrupt:12 Base address:0xdc80 Memory:fd200000-fd200038 dmesg says: Intel(R) PRO/100 Network Driver - version 2.3.43-k1 Copyright (c) 2004 Intel Corporation e100: selftest OK. e100: eth0: Intel(R) PRO/100 Network Connection Hardware receive checksums enabled cpu cycle saver enabled e100: selftest OK. e100: eth1: Intel(R) PRO/100 Network Connection Hardware receive checksums enabled cpu cycle saver enabled smc-ultra.c:v2.02 2/3/98 Donald Becker (becker@cesdis.gsfc.nasa.gov) eth2: SMC Ultra at 0x280, 00 00 C0 5D 46 B5,assigned IRQ 3 memory 0xd0000-0xd3fff. e100: eth0 NIC Link is Up 100 Mbps Full duplex e100: eth1 NIC Link is Up 100 Mbps Full duplex Cheers, Grant. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/