Date: Sat, 4 Mar 2006 19:47:17 +0530
From: Dipankar Sarma <dipankar@in.ibm.com>
To: "David S. Miller" <davem@davemloft.net>
Cc: linux-kernel@vger.kernel.org, torvalds@osdl.org, fabbione@ubuntu.com
Subject: Re: VFS nr_files accounting
Message-ID: <20060304141717.GA456@in.ibm.com>
Reply-To: dipankar@in.ibm.com
References: <20060304.022546.85833873.davem@davemloft.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20060304.022546.85833873.davem@davemloft.net>
User-Agent: Mutt/1.5.10i
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1531
Lines: 34

On Sat, Mar 04, 2006 at 02:25:46AM -0800, David S. Miller wrote:
> 
> I just wanted to report that I am hitting the "VFS: file-max limit xxx
> reached" problem quite easily on my 32-cpu Niagara machine with 16GB
> of ram with current 2.6.x GIT.
> 
> It seems far too easy to get a box into this state due to SLAB
> fragmentation and RCU.  And once you get a machine into this state it
> is totally unusable.
> 
> Our test case is usually a "make -j8192" kernel build along with a
> parallel bootstrap of gcc.  That puts about 256 processes on each
> cpu's runqueue, I doubt ksoftirqd can run much at all.

Dave, there is a set of patches in -mm that may handle this
better -

http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc5/2.6.16-rc5-mm2/broken-out/rcu-batch-tuning.patch
http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc5/2.6.16-rc5-mm2/broken-out/fix-file-counting.patch
http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc5/2.6.16-rc5-mm2/broken-out/fix-file-counting-fixes.patch

Could you please try this in your setup ?

The rcu-batch tuning patch provides automatic switching to
process as many RCUs as possible if too many of them are queued.
The file counting fixes count the file structures correctly.

Thanks
Dipankar
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/