Date: Sun, 29 Oct 2000 23:58:21 -0700
From: "Jeff V. Merkey" <jmerkey@vger.timpanogas.org>
To: Andi Kleen <ak@suse.de>
Cc: linux-kernel@vger.kernel.org
Subject: Re: 2.2.18Pre Lan Performance Rocks!
Message-ID: <20001029235821.A19076@vger.timpanogas.org>
In-Reply-To: <39FCB09E.93B657EC@timpanogas.org> <E13q2R7-0006S7-00@the-village.bc.nu> <20001029183531.A7155@vger.timpanogas.org> <20001030074700.A31783@gruyere.muc.suse.de>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
In-Reply-To: <20001030074700.A31783@gruyere.muc.suse.de>; from ak@suse.de on Mon, Oct 30, 2000 at 07:47:00AM +0100
Sender: linux-kernel-owner@vger.kernel.org

On Mon, Oct 30, 2000 at 07:47:00AM +0100, Andi Kleen wrote:
> On Sun, Oct 29, 2000 at 06:35:31PM -0700, Jeff V. Merkey wrote:
> > On Mon, Oct 30, 2000 at 12:04:23AM +0000, Alan Cox wrote:
> > > > It's still got some problems with NFS (I am seeing a few RPC timeout
> > > > errors) so I am backreving to 2.2.17 for the Ute-NWFS release next week,
> > > > but it's most impressive.
> > > 
> > > Can you send a summary of the NFS reports to nfs-devel@linux.kernel.org
> > 
> > Yes.  I just went home, so I am emailing from my house.  I'll post late 
> > tonight or in the morning.  Performance on 100Mbit with NFS going 
> > Linux->Linux is getting better throughput than IPX NetWare Client -> 
> > NetWare 5.x on the same network by @ 3%.  When you start loading up a 
> > Linux server, it drops off sharply and NetWare keeps scaling, however, 
> > this does indicate that the LAN code paths are equivalent relative to 
> > latency vs. MSM/TSM/HSM in NetWare.  NetWare does better caching 
> > (but we'll fix this in Linux next).  I think the ring transitions to 
> > user space daemons are what are causing the scaling problems 
> > Linux vs. NetWare.
> 
> There are no user space daemons involved in the knfsd fast path, only in slow paths
> like mounting.

So why is it spawning off nfsd servers in user space?  I did notice that all
the connect logic is down in the kernel with the RPC stuff in xprt.c, and this
looks to be pretty fast.  I know about the checksumming problem with TCPIP.
But cycles are cheap on todays processors, so even with this overhead, it 
could still get faster.  IPX uses small packets that are less wire efficient 
since the ratio of header size to payload is larger than what NFS in Linux
is doing, plus there's more of them for an equivalent data transfer, even
with packet burst.   IPX would tend to be faster if there were multiple 
routers involved, since the latency of smaller packets would be less and
IPX never has to deal with the problem of fragmentation.

Is there any CR3 reloading or tasking switching going on for each interrupt
that comes in you know of, BTW?  EMON stats show the server's bus utilization
to get disproportiantely higher in Linux than NetWare 5.x and when it hits
60% of total clock cycles, Linux starts dropping off.  NetWare 5.x is 1/8 
the bus utilitization, which would suggest clocks are being used for something
other than pushing packets in and out of the box (the checksumming is done 
in the tcp code when it copies the fragments, which is very low overhead).

Jeff  


> The main problem I think in knfsd are the numerous copies of the data (e.g. 2+checksumming for
> RX with fragments, upto 4 in some specific configurations). They're unfortunately 
> not trivial to fix. TX is a bit better, it does only one copy usually out of
> the page cache. For RX it also helps to have a network card that supports hardware
> checksumming.
> 
> 
> 
> -Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/