I'm seeing some 2.4.20rc1 client problems.
25 seconds to write a 1G file
13 minutes to read that file back
CLIENT 2.4.20rc1 NFSV3 UDP HIGHMEM SUPPORT ON
gigE card with jumbo frames enabled
Tested with both netgear and intel -- same result
SERVER
Netapps 825 ontap 6.3
gigE card with jumbo frames enabled
private point-to-point connection.
Is this a HIGHMEM problem? Or is it related to jumbo frames? I'll test
with both disabled.
I'm seeing network errors on the client -- dropped and overruns are both
the same.
Anything I should try?
eric
# time dd if=/dev/zero of=file1 bs=1G count=1
1+0 records in
1+0 records out
real 0m25.881s
user 0m0.000s
sys 0m4.220s
(run a program to calloc 1G of RAM and clear some buffer cache)
# time dd if=file1 of=/dev/null
2097152+0 records in
2097152+0 records out
real 13m7.445s
user 0m0.370s
sys 0m2.160s
# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:02:B3:AC:52:F5
inet addr:10.0.0.2 Bcast:10.255.255.255 Mask:255.0.0.0
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:1534585 errors:10240 dropped:10240 overruns:10240
frame:0
TX packets:5332748 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:2069590604 (1973.7 Mb) TX bytes:587855320 (560.6 Mb)
Interrupt:12 Base address:0xc000 Memory:ea020000-ea040000
-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing
your web site with SSL, click here to get a FREE TRIAL of a Thawte
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
I disabled jumbo frames and I'm seeing 70M/s reads (with a clean
cache)..
There must be a UDP or NFS client problem with the 9000 mtu packets.
eric
Eric Whiting wrote:
>
> I'm seeing some 2.4.20rc1 client problems.
>
> 25 seconds to write a 1G file
> 13 minutes to read that file back
>
> CLIENT 2.4.20rc1 NFSV3 UDP HIGHMEM SUPPORT ON
> gigE card with jumbo frames enabled
> Tested with both netgear and intel -- same result
>
> SERVER
> Netapps 825 ontap 6.3
> gigE card with jumbo frames enabled
>
> private point-to-point connection.
>
> Is this a HIGHMEM problem? Or is it related to jumbo frames? I'll test
> with both disabled.
>
> I'm seeing network errors on the client -- dropped and overruns are both
> the same.
>
> Anything I should try?
>
> eric
>
> # time dd if=/dev/zero of=file1 bs=1G count=1
> 1+0 records in
> 1+0 records out
>
> real 0m25.881s
> user 0m0.000s
> sys 0m4.220s
>
> (run a program to calloc 1G of RAM and clear some buffer cache)
>
> # time dd if=file1 of=/dev/null
>
> 2097152+0 records in
> 2097152+0 records out
>
> real 13m7.445s
> user 0m0.370s
> sys 0m2.160s
>
> # ifconfig eth0
> eth0 Link encap:Ethernet HWaddr 00:02:B3:AC:52:F5
> inet addr:10.0.0.2 Bcast:10.255.255.255 Mask:255.0.0.0
> UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
> RX packets:1534585 errors:10240 dropped:10240 overruns:10240
> frame:0
> TX packets:5332748 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:100
> RX bytes:2069590604 (1973.7 Mb) TX bytes:587855320 (560.6 Mb)
> Interrupt:12 Base address:0xc000 Memory:ea020000-ea040000
>
> -------------------------------------------------------
> This sf.net email is sponsored by: To learn the basics of securing
> your web site with SSL, click here to get a FREE TRIAL of a Thawte
> Server Certificate: http://www.gothawte.com/rd524.html
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs
-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing
your web site with SSL, click here to get a FREE TRIAL of a Thawte
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
>>>>> " " == Eric Whiting <[email protected]> writes:
> I disabled jumbo frames and I'm seeing 70M/s reads (with a
> clean cache)..
> There must be a UDP or NFS client problem with the 9000 mtu
> packets.
NFS neither knows nor cares what the MTU is: all it knows is that it
has a socket. I've suspected a bug in the UDP layer for a long time as
you are far from being the first person to report this problem when
enabling jumbo frames.
Cheers,
Trond
-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing
your web site with SSL, click here to get a FREE TRIAL of a Thawte
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
We had to set our MTUs to 8960 for things to work.
We've never followed through to see what the issue
was though.
We are using SysKonnect cards and 2.4.18 kernel. Note,
we haven't recently verified that the problem still
exists.
--Shane
-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing
your web site with SSL, click here to get a FREE TRIAL of a Thawte
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
I tried various MTU values headed gradually towards 9000. I found that a
MTUs between 1500 and 7500 give me the expected performance. MTU values
of anything much more than that causes problems given my setup.
eric
[email protected] wrote:
>
> We had to set our MTUs to 8960 for things to work.
> We've never followed through to see what the issue
> was though.
>
> We are using SysKonnect cards and 2.4.18 kernel. Note,
> we haven't recently verified that the problem still
> exists.
>
> --Shane
-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing
your web site with SSL, click here to get a FREE TRIAL of a Thawte
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, Nov 14, 2002 at 04:13:42PM -0700, Eric Whiting wrote:
> I tried various MTU values headed gradually towards 9000. I found that a
> MTUs between 1500 and 7500 give me the expected performance. MTU values
> of anything much more than that causes problems given my setup.
Which NIC? Some chipsets (like ns83820) only have an 8KB TX fifo and end
up doing two passes over the data for larger packet sizes. Also, running
gige with a standard 32bit/33MHz bus is pretty close to the limits.
-ben
-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing
your web site with SSL, click here to get a FREE TRIAL of a Thawte
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Both of these NICs seemed to have the same behavior.
<6>eth0: Intel(R) PRO/1000 Network Connection
<6>eth1: NetGear GA620 Gigabit Ethernet at 0xea060000, irq 11
Yes -- I'm running on a 32bit/33Mhz bus right now. I do have a Sun 280R
with a 64bit/66Mhz card in a 66Mhz slot that doesn't even seem to
support jumbo frames at all.
eric
Benjamin LaHaise wrote:
>
> On Thu, Nov 14, 2002 at 04:13:42PM -0700, Eric Whiting wrote:
> > I tried various MTU values headed gradually towards 9000. I found that a
> > MTUs between 1500 and 7500 give me the expected performance. MTU values
> > of anything much more than that causes problems given my setup.
>
> Which NIC? Some chipsets (like ns83820) only have an 8KB TX fifo and end
> up doing two passes over the data for larger packet sizes. Also, running
> gige with a standard 32bit/33MHz bus is pretty close to the limits.
>
> -ben
-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing
your web site with SSL, click here to get a FREE TRIAL of a Thawte
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, Nov 14, 2002 at 04:23:07PM -0700, Eric Whiting wrote:
> Both of these NICs seemed to have the same behavior.
> <6>eth0: Intel(R) PRO/1000 Network Connection
> <6>eth1: NetGear GA620 Gigabit Ethernet at 0xea060000, irq 11
>
> Yes -- I'm running on a 32bit/33Mhz bus right now. I do have a Sun 280R
> with a 64bit/66Mhz card in a 66Mhz slot that doesn't even seem to
> support jumbo frames at all.
I'd guess you're hitting the PCI bandwidth limitations then, as the larger
packets permit greater throughput, which in turn drives up bus utilizations.
If you've got any disk controllers on the same bus, that would definately
explain the problem (or any other devices on the bus that are in use).
-ben
-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing
your web site with SSL, click here to get a FREE TRIAL of a Thawte
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs