Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757569Ab0AOM4Y (ORCPT ); Fri, 15 Jan 2010 07:56:24 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1757407Ab0AOM4X (ORCPT ); Fri, 15 Jan 2010 07:56:23 -0500 Received: from ernst.netinsight.se ([194.16.221.21]:8700 "HELO ernst.netinsight.se" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1757306Ab0AOM4W (ORCPT ); Fri, 15 Jan 2010 07:56:22 -0500 Date: Fri, 15 Jan 2010 13:56:13 +0100 (CET) Message-Id: <20100115.135613.1130385479658158300.anders@netinsight.net> To: xiong.huang@atheros.com, jie.yang@atheros.com, linux-kernel@vger.kernel.org Cc: ben@decadent.org.uk Subject: atl1e: TSO is broken From: Anders =?iso-8859-1?Q?Bostr=F6m?= X-Mailer: Mew version 6.3 on Emacs 23.1 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Short desription ---------------- TCP Segmentation Offload (TSO) result in broken IPv4-packets sent out from Atheros AR8121/AR8113/AR8114 with the atl1e driver. Work around ----------- Turn off TSO. Requires 2.6.33-rc1 or later. Long desription ---------------- When I run NFS over TCP (default options) and read large files from a server with Atheros AR8121/AR8113/AR8114 Ethernet chip, I only get ~25Mbyte/s performance. I get ~5000 retransmitted packets per GByte data, according to RetransSegs in /proc/net/snmp . wireshark in the client show that the server send out a sequence of frames. All but the last one are 1500 bytes IP-packets. The last one is shorter, but the IP-header still say 1500 byte. The client then requests retransmit, and the retransmitted frame arrives with correct IP-header. If I mount NFS using UDP instead, performance is ~110Mbyte/s. TCP Segmentation Offload (TSO) is default enabled in the atl1e Ethernet-driver. When I run a patched 2.6.30.10, enabling ethtool to turn off TSO (using ac936929092dc6a5409b627c4c67305ab9b626b3 by Ben Hutchings), and turn off TSO, the problem disappears. Performance is ~110Mbyte/s and no broken IP-packets arrive. Capture of 146-byte Ethernet frame with bad IP-header: No. Time Source Destination Protocol Info 98329 11.034129 flash.netinsight.se sid.netinsight.se RPC Continuation Frame 98329 (146 bytes on wire, 146 bytes captured) Arrival Time: Jan 15, 2010 13:35:16.224491000 [Time delta from previous captured frame: 0.000009000 seconds] [Time delta from previous displayed frame: 0.000009000 seconds] [Time since reference or first frame: 11.034129000 seconds] Frame Number: 98329 Frame Length: 146 bytes Capture Length: 146 bytes [Frame is marked: False] [Protocols in frame: eth:ip:tcp:rpc] [Coloring Rule Name: TCP] [Coloring Rule String: tcp] Ethernet II, Src: AsustekC_ae:69:6d (00:26:18:ae:69:6d), Dst: sid.netinsight.se (00:18:f3:52:22:3f) Internet Protocol, Src: flash.netinsight.se (10.100.0.88), Dst: sid.netinsight.se (10.100.1.25) Version: 4 Header length: 20 bytes Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00) Total Length: 1500 Identification: 0x331e (13086) Flags: 0x02 (Don't Fragment) Fragment offset: 0 Time to live: 64 Protocol: TCP (0x06) Header checksum: 0xebc5 [correct] Source: flash.netinsight.se (10.100.0.88) Destination: sid.netinsight.se (10.100.1.25) Transmission Control Protocol, Src Port: nfs (2049), Dst Port: accessbuilder (888), Seq: 93989617, Ack: 516997, Len: 80 Remote Procedure Call 0000 00 18 f3 52 22 3f 00 26 18 ae 69 6d 08 00 45 00 ...R"?.&..im..E. 0010 05 dc 33 1e 40 00 40 06 eb c5 0a 64 00 58 0a 64 ..3.@.@....d.X.d 0020 01 19 08 01 03 78 a4 07 57 23 e6 50 1f 1b 80 10 .....x..W#.P.... 0030 01 f5 dd 8d 00 00 01 01 08 0a 05 28 ca 7e 38 93 ...........(.~8. 0040 67 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 g............... 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0090 00 00 .. Software info: I've tested with Debian 2.6.26 (stable) and 2.6.30 (testing), as well as 2.6.30.10 from kernel.org. Same result. Hardware info: lspci -vvv: 03:00.0 Ethernet controller: Attansic Technology Corp. Atheros AR8121/AR8113/AR8114 PCI-E Ethernet Controller (rev b0) Subsystem: ASUSTeK Computer Inc. Device 831c Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR+ Capabilities: [180] Device Serial Number ff-18-26-00-6d-69-ae-ff Kernel driver in use: ATL1E Kernel modules: atl1e ethtool -i eth0: driver: ATL1E version: 1.0.0.7-NAPI firmware-version: L1e bus-info: 0000:03:00.0 / Anders -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/