> I'm not sure if this is a bug or what, but I am experiencing severe=20
> performace problems between an IA64 NFS server and IA32 NFS client. I=20
> have tried every combination of socket buffer sizes as well as many=20
> other nfs options, but none of them seem to have any effect. I have=20
> tried RHEL21 to RHEL3U5 with all of these socket combos and still no=20
> difference. The command I am running is:
>=20
> time dd if=3D/dev/zero of=3D/scratch_dir_across_nfs bs=3D1M =
count=3D128
>=20
> Between two IA32 machines this takes 4-5 seconds. Between=20
> IA64 server and IA32 client this takes on the order of=20
> minutes. Network performance has been eliminated from the=20
> equation as after kernel tuning we have achieved consistent=20
> 94% of peak tcp throughput and 95.7% of peak udp throughput=20
> on our GigE network. This is both between IA32 and IA64 systems.
can you tell us about the file system on the ia64 server. block size,
fstype, etc?
what's the page size on that system?
have you run local performance tests on the server?
-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Tue, Aug 02, 2005 at 07:36:17AM -0700, Lever, Charles wrote:
> can you tell us about the file system on the ia64 server. block size,
> fstype, etc?
>
> what's the page size on that system?
This pretty much matches the symptoms we've seen with SGI servers
on 2.6.5. You may want to try a recent kernel on your clients, which
has support for write sizes larger than the client's PAGE_CACHE_SIZE.
Exactly these bad performance numbers made us backport the large
wsize/rsize patches to 2.6.5 for SLES.
Olaf
--
Olaf Kirch | --- o --- Nous sommes du soleil we love when we play
[email protected] | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax
-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Lever, Charles wrote:
>>I'm not sure if this is a bug or what, but I am experiencing severe
>>performace problems between an IA64 NFS server and IA32 NFS client. I
>>have tried every combination of socket buffer sizes as well as many
>>other nfs options, but none of them seem to have any effect. I have
>>tried RHEL21 to RHEL3U5 with all of these socket combos and still no
>>difference. The command I am running is:
>>
>>time dd if=/dev/zero of=/scratch_dir_across_nfs bs=1M count=128
>>
>>Between two IA32 machines this takes 4-5 seconds. Between
>>IA64 server and IA32 client this takes on the order of
>>minutes. Network performance has been eliminated from the
>>equation as after kernel tuning we have achieved consistent
>>94% of peak tcp throughput and 95.7% of peak udp throughput
>>on our GigE network. This is both between IA32 and IA64 systems.
>>
>>
>
>can you tell us about the file system on the ia64 server. block size,
>fstype, etc?
>
>what's the page size on that system?
>
>have you run local performance tests on the server?
>
>
>-------------------------------------------------------
>SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
>from IBM. Find simple to follow Roadmaps, straightforward articles,
>informative Webcasts and more! Get everything you need to get up to
>speed, fast. http://ads.osdn.com/?ad_idt77&alloc_id492&op=click
>_______________________________________________
>NFS maillist - [email protected]
>https://lists.sourceforge.net/lists/listinfo/nfs
>
>
Here's the info of the two filesystems from dumpe2fs:
On the Itanium:
Filesystem volume name: /
Last mounted on: <not available>
Filesystem UUID: 7f29192c-e37d-11d7-9483-d26f707daf6e
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal filetype needs_recovery
sparse_super large_file
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 3826368
Block count: 7662080
Reserved block count: 383104
Free blocks: 3603448
Free inodes: 3686174
First block: 0
Block size: 4096
Fragment size: 4096
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 16352
Inode blocks per group: 511
Last mount time: Fri Jul 15 16:55:05 2005
Last write time: Fri Jul 15 16:55:05 2005
Mount count: 30
Maximum mount count: -1
Last checked: Wed Sep 10 04:56:57 2003
Check interval: 0 (<none>)
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal UUID: <none>
Journal inode: 8
Journal device: 0x0000
First orphan inode: 229217
And on the Xeon:
Filesystem volume name: /
Last mounted on: <not available>
Filesystem UUID: 49140d68-f52a-11d9-832b-94d6e9be8214
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal filetype needs_recovery
sparse_super large_file
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 4627616
Block count: 9247415
Reserved block count: 462370
Free blocks: 6511674
Free inodes: 4466407
First block: 0
Block size: 4096
Fragment size: 4096
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 16352
Inode blocks per group: 511
Last mount time: Tue Aug 2 08:06:18 2005
Last write time: Tue Aug 2 08:06:18 2005
Mount count: 1
Maximum mount count: -1
Last checked: Tue Aug 2 08:06:10 2005
Check interval: 0 (<none>)
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal UUID: <none>
Journal inode: 8
Journal device: 0x0000
First orphan inode: 0
Unfortunately, this is not a performance issue (except with NFS of
course), as we can achieve peak throughput between two machines with
identical architecture. Both these machines are, however, using the 2.4
kernel currently, but tests have been performed with 2.6 as well with no
difference.
My understanding of NFS (correct me if I am wrong) is that it converts
the data representation to a format called XDR, which is then transfered
over the wire to the client/server who translates XDR into the
representation of the local filesystem. My gut instinct tells me that
something is being lost in translation.
Brett P
-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs