2009-06-21 11:44:27

by Roger Willcocks

[permalink] [raw]
Subject: Re: [fuse-devel] Need Suggestions of packet overrun over 1 Gbps netwroktraffic in kernel 2.6.18

> Hi all,
>
> I have a high end server machine(multi core and large memory space) with
> 10
> Gbps NIC. With linux kernel 2.6.18 it is giving packet overrun as traffic
> goes beyond 1.5 Gbps. I need suggestions in tuning kernel so that it can
> handle higher traffic rates.
>
> I have a 10 Gbps test bed to test the capability of server, equipped with
> 10Gigwsitches and traffic generated through multi 1Gig machines. I have
> tested server machine by gradually increasing the incoming traffic rate
> and
> it's starting to do overrun as it crosses 1 Gbps.

I've seen a similar problem here with Myricom 10ge cards in multi-core
machines running CentOS 5.2. A point-to-point connection configured as per
http://www.myri.com/serve/cache/511.html worked fine (netperf > 9Gb/s) but
introducing an HP 6400 switch gave 'dropped_link_overflow' errors on the
receiving NIC, and the netperf figure dropped by about 90%.

The answer was to enable flow control on the switch. Interestingly an OSX
XServe box could absorb the full bandwidth even with flow control disabled.

Obviously you shouldn't need hardware flow control with TCP traffic (and
it's quite probably counter-productive) but the Myricom docs
(http://www.myri.com/scs/download-Myri10GE.html) say that "The 10GbE switch
to which the NICs are connected /must/ have flow control enabled", so that's
what I'd look at first.

--
Roger

----------------------------
FilmLight's looking for an experienced kernel / systems developer (central
London). An interest in parallel filesystems, image processing and low level
scary magic would be an advantage; email me for more details.