From: Neil Brown <neilb@cse.unsw.edu.au>
Subject: Re: How many nfsd's can I run?
Date: Wed, 22 Jun 2005 12:27:18 +1000
Message-ID: <17080.52358.24058.797844@cse.unsw.edu.au>
References: <42B8C69C.6010302@affymetrix.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: nfs@lists.sourceforge.net
To: Hugh Caley <Hugh_Caley@affymetrix.com>
In-Reply-To: message from Hugh Caley on Tuesday June 21
Sender: nfs-admin@lists.sourceforge.net
Errors-To: nfs-admin@lists.sourceforge.net

On Tuesday June 21, Hugh_Caley@affymetrix.com wrote:
> Often when a cluster run (100+ nodes) is started my NFS server will get 
> very slow; users with their home directories on the server will lock up 
> for short periods, and the load average crawls higher.
> 
> The output of cat /proc/net/rpc/nfsd:
> 
> th 125 264502 142948.997 38666.037 22620.179 16494.577 10072.689 
> 5700.113 2623.322 1094.785 474.752 1900.536
> 
> This would seem to mean that my nfsd threads are runnning at 100% for 
> much of the time, meaning I need more nfsd's, correct?  But is there a 
> limit to the amount of nfsd's I should run?  I had had it as high as 150 
> at one time, but it seemed to me this made the server less stable.  
> There may have been other reasons for this, however.
> 
> Anyway I have since doubled the ram on the server (to 8 gig).  Machine 
> is a dual Xeon 2.6 Ghz machine running Fedora Core 2, kernel version 
> 2.6.10-1.771_FC2smp, gigabit ethernet.  Can I run more nfsd's, or should 
> I be looking for other problems?

There should be very little cost in pushing the number of nfsd
substantially higher, but there might not be much gain.

So definitely run more nfsd - say 256 (you can just run "nfsd 256" and
it will increase the number of threads to that number).
But also look into how the filesystem and storage is performing.

What filesys and what storage are you using?

NeilBrown


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs