From: "M. Todd Smith" <todd@sohovfx.com>
Subject: Re: NFS tuning - high performance throughput.
Date: Tue, 14 Jun 2005 16:38:25 -0400
Message-ID: <42AF4041.8030504@sohovfx.com>
References: <482A3FA0050D21419C269D13989C611308539C89@lavender-fe.eng.netapp.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
To: nfs@lists.sourceforge.net
In-Reply-To: <482A3FA0050D21419C269D13989C611308539C89@lavender-fe.eng.netapp.com>
Sender: nfs-admin@lists.sourceforge.net
Errors-To: nfs-admin@lists.sourceforge.net

Charles,

I've tried mounting both TCP and UDP and with rwsizes varying from 8k to 
32k, 32k rwsizes are giving us the best performance thus far.

Indeed I did skip a level with the ping times didn't I, 200 microseconds 
it is.

Cheers
Todd


Lever, Charles wrote:

>first place i would look is mount options on the client.  are you
>mounting with NFS over TCP, or NFS over UDP?
>
>(btw, your ping time should be 200 microseconds, not 200 nanoseconds).
>
>  
>
>>-----Original Message-----
>>From: M. Todd Smith [mailto:todd@sohovfx.com] 
>>Sent: Tuesday, June 14, 2005 4:18 PM
>>To: nfs@lists.sourceforge.net
>>Subject: [NFS] NFS tuning - high performance throughput.
>>
>>
>>Hello all,
>>
>>I've been attempting to get a better hold over our NFS performance as 
>>our network grows and grows.
>>
>>We've recently upgraded our NFS server machine to a dual 3.2ghz Xeon 
>>running Fedora Core 3 (kernel 2.6.11-1.14_FC3smp) w/ 4Gb RAM. 
>> Coupled 
>>with this machine we have a 2 Broadcom NetExtreme 2 Port PCI-X NIC on 
>>its own PCI-X bus (133Mhz).  Attached to this machine is our fibre 
>>channel SAN, using Seagate fibre-channel drives and an LSI 
>>dual channel 
>>2gigabit fibre adapter on its own PCI-X bus.  Local RW is 
>>~135Mbytes/sec.
>> 
>>The 4 ports are trunked together using bonding (balanced round robin 
>>mode), and trunked together on our Extreme Networks Summit 
>>400i switch.  
>>All the test machines are attached to this switch, making everything 
>>within one hop and ping times of less than a 200 nanoseconds.
>>
>>Current test machine is running Suse 9.2 and has an Intel 100/1000 XT 
>>server adapter (e1000 driver) on a shared but not high 
>>traffic PCI bus.  
>>Other test machines include some Dell PowerEdge 1850's with onboard 
>>Intel NICs, and some Apple G4, G5 and Xserves.
>>
>>I have read most of the tuning guides I can find on the net and 
>>attempted just about everything I can get my hands on (I have 
>>not tried 
>>jumbo frames yet, still waiting for some downtime to attempt 
>>that).  My 
>>problem is that no matter how I tune the machines I can get at max 
>>45Mb/ps throughput on NFS.  This was the same throughput we 
>>were getting 
>>with our old server with PCI cards, moreover this throughput 
>>is roughly 
>>the same for every machine on our network.  Theoretically we 
>>should be 
>>able to get much higher values.
>>
>>Any idea as to why this is?  I can provide config files and such if 
>>needed, but I'm really at a loss as to where to start.
>>
>>Cheers
>>Todd
>>
>>-- 
>>Systems Administrator
>>----------------------------------
>>Soho VFX - Visual Effects Studio
>>99 Atlantic Avenue, Suite 303
>>Toronto, Ontario, M6K 3J8
>>(416) 516-7863 
>>http://www.sohovfx.com
>>----------------------------------
>>
>>
>> 
>>
>>
>>
>>-------------------------------------------------------
>>SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
>>from IBM. Find simple to follow Roadmaps, straightforward articles,
>>informative Webcasts and more! Get everything you need to get up to
>>speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
>>_______________________________________________
>>NFS maillist  -  NFS@lists.sourceforge.net
>>https://lists.sourceforge.net/lists/listinfo/nfs
>>
>>    
>>
>
>
>  
>


-- 
Systems Administrator
----------------------------------
Soho VFX - Visual Effects Studio
99 Atlantic Avenue, Suite 303
Toronto, Ontario, M6K 3J8
(416) 516-7863 
http://www.sohovfx.com
----------------------------------


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs