2005-12-14 02:15:09

by 陈欢

[permalink] [raw]
Subject: performance problem on nfs in kernel v2.4 and v2.6

nfs:

=09
=09I did some test on nfs in linux kernel v2.6 ,but there are some=
problem i can't solve.
=09I need your help!

=09I run iozone on nfs in kernel v2.4.21 and v2.6.11.12 ,the test=
result is as follows:
=09v2.4.21: write performance 106MB/s (wsize,rsize=3D8192)
=09v2.6.11.12: write performance 52MB/s ; (wsize,rsize=3D32768)

=09the iozone test command I use is :
=09=09iozone -c -e -f /mnt/nfs/file1 -s 1g -r 2m -i 0 -w

=09My question is why the performance is lower when using v2.6?

=09The test platform is :
=09=09client: 1G memory,
=09=09server: 4G memory, 146G disk. 2cpu.
=09=09net: 1000Mb

=09When i use the following command to chechk why v2.6 takes much=
more time, I found that the fsync operations takes much more=
time. the command is:
=09strace -o result -T write, close, fsync ./iozone -c -e -f=
/mnt/nfs/file1 -s 1g -r 2m -i 0 -w
=09the result of this command is that the time of write is=
6(ms),while the time of sync before close is 6 (s).

=09So what is wrong? how can i see performance improvement of nfs=
in v2.6. Is there any key value I can tune ? Or the problem=
relates to the removal of flushd while introducing pdflush in=
client memory management ?

=09Thank you! Waiting for your reply!


------------------------------------------------------------
National Research Center for Intelligent Computing Systems
Insitute of Computing Technology
Chinese Academy of Sciences=A1=A1=A1=A1=A1=A1
------------------------------------------------------------
=09=09

=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1
=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1huanchen@ncic.ac.cn
=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A12005-12-14




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs


2005-12-14 23:16:36

by Chuck Lever

[permalink] [raw]
Subject: Re: performance problem on nfs in kernel v2.4 and v2.6

huanchen-

you tell us which kernels you've tried, but it's not clear
whether you are switching the client or the server kernel.

also, what export options do you use on the server? and
what file system?



?»? wrote:
> nfs:
>
>
> I did some test on nfs in linux kernel v2.6 ,but there are some problem i can't solve.
> I need your help!
>
> I run iozone on nfs in kernel v2.4.21 and v2.6.11.12 ,the test result is as follows:
> v2.4.21: write performance 106MB/s (wsize,rsize=8192)
> v2.6.11.12: write performance 52MB/s ; (wsize,rsize=32768)
>
> the iozone test command I use is :
> iozone -c -e -f /mnt/nfs/file1 -s 1g -r 2m -i 0 -w
>
> My question is why the performance is lower when using v2.6?
>
> The test platform is :
> client: 1G memory,
> server: 4G memory, 146G disk. 2cpu.
> net: 1000Mb
>
> When i use the following command to chechk why v2.6 takes much more time, I found that the fsync operations takes much more time. the command is:
> strace -o result -T write, close, fsync ./iozone -c -e -f /mnt/nfs/file1 -s 1g -r 2m -i 0 -w
> the result of this command is that the time of write is 6(ms),while the time of sync before close is 6 (s).
>
> So what is wrong? how can i see performance improvement of nfs in v2.6. Is there any key value I can tune ? Or the problem relates to the removal of flushd while introducing pdflush in client memory management ?
>
> Thank you! Waiting for your reply!
>
>
> ------------------------------------------------------------
> National Research Center for Intelligent Computing Systems
> Insitute of Computing Technology
> Chinese Academy of Sciences??????
> ------------------------------------------------------------
>
>
> ????????????????
> [email protected]
> ????????????????????2005-12-14
>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
> for problems? Stop! Download the new AJAX search engine that makes
> searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
> http://ads.osdn.com/?ad_idv37&alloc_id865&op=click
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs


Attachments:
cel.vcf (264.00 B)

2005-12-15 01:12:29

by 陈欢

[permalink] [raw]
Subject: Re: Re: performance problem on nfs in kernel v2.4 and v2.6

Chuck Lever,=C4=FA=BA=C3=A3=A1

=09when i did test on kernel 2.4.21, both client and server are of=
that kernel version. So i did on kernel 2.6.11.12.

=09the export option i use is :=
=09(rw,async,wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_s=
quash,subtree_check,secure_locks,mapping=3Didentity,anonuid=3D-2,anon=
gid=3D-2)

=09the filesystem is ext2.

=09why i can't test the good performance i expected on v2.6.?

=3D=3D=3D=3D=3D=3D=3D 2005-12-14 15:16:00 =C4=FA=D4=DA=C0=B4=D0=C5=D6=D0=D0=B4=B5=C0=A3=BA=3D=3D=3D=3D=3D=3D=3D

>huanchen-
>
>you tell us which kernels you've tried, but it's not clear
>whether you are switching the client or the server kernel.
>
>also, what export options do you use on the server? and
>what file system?
>
>
>
>=B3=C2=BB=B6 wrote:
>> nfs:
>>
>> =09
>> =09I did some test on nfs in linux kernel v2.6 ,but there are=
some problem i can't solve.
>> =09I need your help!
>>
>> =09I run iozone on nfs in kernel v2.4.21 and v2.6.11.12 ,the=
test result is as follows:
>> =09v2.4.21: write performance 106MB/s (wsize,rsize=3D8192)
>> =09v2.6.11.12: write performance 52MB/s ; (wsize,rsize=3D32768)
>>
>> =09the iozone test command I use is :
>> =09=09iozone -c -e -f /mnt/nfs/file1 -s 1g -r 2m -i 0 -w
>>
>> =09My question is why the performance is lower when using v2.6?
>>
>> =09The test platform is :
>> =09=09client: 1G memory,
>> =09=09server: 4G memory, 146G disk. 2cpu.
>> =09=09net: 1000Mb
>>
>> =09When i use the following command to chechk why v2.6 takes=
much more time, I found that the fsync operations takes much=
more time. the command is:
>> =09strace -o result -T write, close, fsync ./iozone -c -e -f=
/mnt/nfs/file1 -s 1g -r 2m -i 0 -w
>> =09the result of this command is that the time of write is=
6(ms),while the time of sync before close is 6 (s).
>>
>> =09So what is wrong? how can i see performance improvement of=
nfs in v2.6. Is there any key value I can tune ? Or the=
problem relates to the removal of flushd while introducing =
pdflush in client memory management ?
>>
>> =09Thank you! Waiting for your reply!
>>
>>
>> ------------------------------------------------------------
>> National Research Center for Intelligent Computing Systems
>> Insitute of Computing Technology
>> Chinese Academy of Sciences=A1=A1=A1=A1=A1=A1
>> ------------------------------------------------------------
>> =09=09
>>
>> =A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1
>> =A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1huanchen@ncic.ac.cn
>> =A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A12005-12-14
>>
>>
>>
>>
>> -------------------------------------------------------
>> This SF.net email is sponsored by: Splunk Inc. Do you grep=
through log files
>> for problems? Stop! Download the new AJAX search engine that=
makes
>> searching your log files as easy as surfing the web. =
DOWNLOAD SPLUNK!
>> http://ads.osdn.com/?ad_idv37&alloc_id=16865&op=3Dclick
>> _______________________________________________
>> NFS maillist - [email protected]
>> https://lists.sourceforge.net/lists/listinfo/nfs

=3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D
=09=09=09

=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=D6=C2
=C0=F1=A3=A1

=09=09=09=09
=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=B3=C2=BB=B6
=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1huanchen@ncic.ac.cn
=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A12005-12-15





-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2005-12-15 09:08:38

by Jerome Warnier

[permalink] [raw]
Subject: Re: Re: performance problem on nfs in kernel v2.4 and v2.6

Le jeudi 15 d=C3=A9cembre 2005 =C3=A0 09:12 +0800, =E9=99=88=E6=AC=A2 a =C3=
=A9crit :
> Chuck Lever,=E6=82=A8=E5=A5=BD=EF=BC=81
>=20
> when i did test on kernel 2.4.21, both client and server are of that ker=
nel version. So i did on kernel 2.6.11.12.
>=20
> the export option i use is : (rw,async,wdelay,hide,nocrossmnt,secure,no=
_root_squash,no_all_squash,subtree_check,secure_locks,mapping=3Didentity,an=
onuid=3D-2,anongid=3D-2)
>=20
> the filesystem is ext2.
>=20
> why i can't test the good performance i expected on v2.6.?
Did you try playing with the "rsize" and "wsize" options?

[..]

--=20
J=C3=A9r=C3=B4me Warnier
FLOSS Consultant
http://beeznest.net



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2005-12-15 09:16:22

by 陈欢

[permalink] [raw]
Subject: Re: Re: Re: performance problem on nfs in kernel v2.4 and v2.6

Jerome Warnier,=C4=FA=BA=C3=A3=A1

=09yes, I did have do that .
=09in case 2.6.11, I tried rsize,wsize=3D8192/16384/32768, however,=
the results were bad.

=3D=3D=3D=3D=3D=3D=3D 2005-12-15 10:08:00 =C4=FA=D4=DA=C0=B4=D0=C5=D6=D0=D0=B4=B5=C0=A3=BA=3D=3D=3D=3D=3D=3D=3D

>Le jeudi 15 d=A8=A6cembre 2005 =A8=A4 09:12 +0800, =B3=C2=BB=B6 a =A8=A6crit :
>> Chuck Lever,=C4=FA=BA=C3=A3=A1
>>
>> =09when i did test on kernel 2.4.21, both client and server are=
of that kernel version. So i did on kernel 2.6.11.12.
>>
>> =09the export option i use is :=
=09(rw,async,wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_s=
quash,subtree_check,secure_locks,mapping=3Didentity,anonuid=3D-2,anon=
gid=3D-2)
>>
>> =09the filesystem is ext2.
>>
>> =09why i can't test the good performance i expected on v2.6.?
>Did you try playing with the "rsize" and "wsize" options?
>
>[..]
>
>--
>J=A8=A6r?me Warnier
>FLOSS Consultant
>http://beeznest.net
>
>
>
>-------------------------------------------------------
>This SF.net email is sponsored by: Splunk Inc. Do you grep=
through log files
>for problems? Stop! Download the new AJAX search engine that=
makes
>searching your log files as easy as surfing the web. DOWNLOAD=
SPLUNK!
>http://ads.osdn.com/?ad_idv37&alloc_id=16865&op=3Dclick
>_______________________________________________
>NFS maillist - [email protected]
>https://lists.sourceforge.net/lists/listinfo/nfs

=3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D =3D
=09=09=09

=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=D6=C2
=C0=F1=A3=A1

=09=09=09=09
=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=B3=C2=BB=B6
=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1huanchen@ncic.ac.cn
=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A1=A12005-12-15





-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2005-12-15 09:26:57

by Vincent Roqueta

[permalink] [raw]
Subject: Re: performance problem on nfs in kernel v2.4 and v2.6

Le Jeudi 15 D=C3=A9cembre 2005 10:16, =E9=99=88=E6=AC=A2 a =C3=A9crit=C2=A0:
> Jerome Warnier,=E6=82=A8=E5=A5=BD=EF=BC=81
>
> yes, I did have do that .
> in case 2.6.11, I tried rsize,wsize=3D8192/16384/32768, however, the res=
ults
> were bad.

Could you give me local filesystem performances of the server?


Vincent


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs