2004-07-01 21:18:31

by Bernd Schubert

[permalink] [raw]
Subject: client: 2.6.7 client much slower than 2.4.26

Hi,

I'm currently testing 2.6.7 on one of our clients and the nfs transfer is much
slower than when the same system is running 2.4.X

Well, the usual suggestion probably would be to run nfs over tcp now, but tcp
is already enabled (from /proc/mounts):

taylor:/worka /mnt/test nfs
rw,v3,rsize=32768,wsize=32768,hard,tcp,lock,addr=taylor 0 0

benchmarks:

copy a file to the server (2.6.7): ~ 3MB/s
copy a file to the server (2.4.26): ~ 9MB/s

copy a file from the server (2.6.7): ~ 3MB/s
copy a file from the server (2.4.26): ~ 11MB/s

The filesize was 300MB.


Here's the nfsstat output:

euklid:~# nfsstat
Client rpc stats:
calls retrans authrefrsh
385466 0 0
Client nfs v2:
null getattr setattr root lookup readlink
0 0% 27475 66% 231 0% 0 0% 3963 9% 161 0%
read wrcache write create remove rename
8303 20% 0 0% 560 1% 171 0% 198 0% 42 0%
link symlink mkdir rmdir readdir fsstat
45 0% 0 0% 3 0% 3 0% 129 0% 2 0%

Client nfs v3:
null getattr setattr lookup access readlink
0 0% 56851 16% 1047 0% 17711 5% 36670 10% 936 0%
read write create mkdir symlink mknod
73640 21% 149802 43% 629 0% 86 0% 1 0% 0 0%
remove rmdir rename link readdir readdirplus
701 0% 105 0% 279 0% 36 0% 464 0% 1557 0%
fsstat fsinfo pathconf commit
260 0% 18 0% 0 0% 3387 0%


I already tried to reduce rsize and wsize up to 1024, but this didn't change
anything.

This happens to two different servers, one server is running 2.6.7 and has
Gigabit connection to the switch. The other server is running 2.4.25 and only
has a 100MBit connection. All of our clients have 100MBit connections.

So I really thing this is a client problem. Well, the client is not the
fastest of our systems (PII 450), but with 2.4.X it has the usual speed. Its
also interesting top is showing that the rpciod is taking all cpu-time when
copying a file to the server.

Any ideas whats going on?

Thanks,
Bernd


Attachments:
(No filename) (2.15 kB)
(No filename) (189.00 B)
signature
Download all attachments

2004-07-01 21:43:15

by Trond Myklebust

[permalink] [raw]
Subject: Re: client: 2.6.7 client much slower than 2.4.26

P=E5 to , 01/07/2004 klokka 17:18, skreiv Bernd Schubert:

> benchmarks:
>=20
> copy a file to the server (2.6.7): ~ 3MB/s
> copy a file to the server (2.4.26): ~ 9MB/s
>=20
> copy a file from the server (2.6.7): ~ 3MB/s
> copy a file from the server (2.4.26): ~ 11MB/s

Looks like the wire speeds on read/write are the same. How does the
above compare to ttcp transfer speeds of a 300MB file?


> Here's the nfsstat output:

What's that for, 2.4.26 or 2.6.7? How does it compare?

> So I really thing this is a client problem. Well, the client is not the=20
> fastest of our systems (PII 450), but with 2.4.X it has the usual speed. =
Its=20
> also interesting top is showing that the rpciod is taking all cpu-time wh=
en=20
> copying a file to the server.

How much memory does this thing have? The main difference between 2.4.x
and 2.6.x is the fact that the 2.6.x caches a lot more data and for
longer (the VM is what decides when to flush the cache).
You might want to try twiddling around with the vm sysctl parameters to
see if that makes a difference.

Cheers,
Trond


-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 -
digital self defense, top technical experts, no vendor pitches,
unmatched networking opportunities. Visit http://www.blackhat.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2004-07-01 22:32:40

by Bernd Schubert

[permalink] [raw]
Subject: Re: client: 2.6.7 client much slower than 2.4.26

On Thursday 01 July 2004 23:43, you wrote:
> P=E5 to , 01/07/2004 klokka 17:18, skreiv Bernd Schubert:
> > benchmarks:
> >
> > copy a file to the server (2.6.7): ~ 3MB/s
> > copy a file to the server (2.4.26): ~ 9MB/s
> >
> > copy a file from the server (2.6.7): ~ 3MB/s
> > copy a file from the server (2.4.26): ~ 11MB/s
>
> Looks like the wire speeds on read/write are the same. How does the
> above compare to ttcp transfer speeds of a 300MB file?

Thanks, I didn't know about this tool and I think you are right, it seems t=
hat=20
the general network speed is much slower with 2.6.7:

2.4.26:
bernd@euklid bernd>nttcp -T taylor
Bytes Real s CPU s Real-MBit/s CPU-MBit/s Calls Real-C/s CPU-=
C/s
l 8388608 0.70 0.06 95.2480 1118.4811 2048 2906.74 3413=
3.3
1 8388608 0.71 0.03 94.1283 2485.7897 5793 8125.38 21457=
9.4

2.6.7:
bernd@euklid bernd>nttcp -T taylor
Bytes Real s CPU s Real-MBit/s CPU-MBit/s Calls Real-C/s CPU-=
C/s
l 8388608 2.67 2.64 25.1472 25.4046 2048 767.43 77=
5.3
1 8388608 2.67 0.03 25.1442 2237.3350 6145 2302.40 20486=
7.5

Its already too late, so I will do the 300MB test tomorrow, but I think the=
=20
numbers from the default 8MB test are already convincing.


> How much memory does this thing have? The main difference between 2.4.x

Somehow this system didn't like to have more than 2x128MB, when we added=20
another DIMM, the system crashed pretty often, though memtest86 showed no=20
problems.=20

> and 2.6.x is the fact that the 2.6.x caches a lot more data and for
> longer (the VM is what decides when to flush the cache).
> You might want to try twiddling around with the vm sysctl parameters to
> see if that makes a difference.
>

I will try to play with this over the weekend.


Thanks again a lot for your help!


Cheers,
Bernd


-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 -
digital self defense, top technical experts, no vendor pitches,
unmatched networking opportunities. Visit http://www.blackhat.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs