Hello there,
I run a small (10-node) diskless network in our company, and I've
been a user of Linux as server and client in it for many years now.
Most of the time my users are very happy with the system, and it
performs very reliably; however, I do get complaints that the network is
slow, and I always want to advocate that diskless should be as fast
enough as local disks (even though it really isn't!)
So a user today presented me with a real-world benchmark of an operation
that he does regularly. It's basically issueing a copy of a code checkout:
% cp -r gazpacho gazpacho-new
This is a 33MB tree with 2674 files. Network is a switched 100Mbit/s,
and you can netcat a file over HTTP at about 10MB/s from the server; I
think that suggests the network is fine.
Unscientific average times for variations of this benchmark are as
follows:
0:05 - Server: locally on RAID-5
0:06 - Client: copying from NFS mount to tmpfs
2:31 - Client: Copying from NFS mount to NFS mount
(i.e., the simple command above)
3:30 - Client: Copying from NFS mount to NFS mount over an existing
tree.
5:08 - Client: Copying from tmpfs to NFS mount
0:48 - Client: Deleting copy of tree on NFS mount
Now I know that file creation is synchronous, and that because of this,
the NFS copy will /always/ run significantly slower. But I was amazed at
just how much faster reading was than writing, and at how fast reading
actually is -- it's much faster than I expected NFS could be.
I also found it interesting that copying from NFS mount to NFS mount was
faster than copying from local filesystem to NFS mount; I imagine there
is a protocol optimization for that common scenario.
But what I'd really like to know if there is a way of making the
above operation run in significantly less time than it is today, or if
our expectations are just unrealistic given the complexity of network
filesystems. I'd be pretty happy with an operation that ran in, say, 40
seconds -- but 2:30 is a pretty long time. I've tried toying with rsize
and wsize, async and sync, and tcp rmem and wmem proc settings. None of
those changes makes a real difference. So my questions are:
Are there practical ways of speeding the operation up?
Would people with better hardware see significantly reduced times?
I appreciate any advice and discussion on the topic. Thanks!
--
Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 3376 0125
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Hi Christian,
> 0:05 - Server: locally on RAID-5
> 0:06 - Client: copying from NFS mount to tmpfs
> 2:31 - Client: Copying from NFS mount to NFS mount
> (i.e., the simple command above)
> 3:30 - Client: Copying from NFS mount to NFS mount over an existing
> tree.
> 5:08 - Client: Copying from tmpfs to NFS mount
> 0:48 - Client: Deleting copy of tree on NFS mount
I guess your exports has the "sync" option or your didn't set async or syn=
c =
at all (sync is then the default)? Can you repeat the test again using the =
async option? But please, also read carefully the explanations of "man 5 =
exports" about this topic. =
>
> Now I know that file creation is synchronous, and that because of this,
> the NFS copy will /always/ run significantly slower. But I was amazed at
> just how much faster reading was than writing, and at how fast reading
> actually is -- it's much faster than I expected NFS could be.
Just for fun, take ethereal and capture some data, once with the async and =
once with the sync option. Now have a look how much time each operation of =
the nfs client takes before it gets confirmed by the server. Especially loo=
k =
at the commit calls, once with sync and once with async option.
> But what I'd really like to know if there is a way of making the
> above operation run in significantly less time than it is today, or if
I guess if one could convince the nfs clients to reduce the number of commi=
t =
calls, write operations would be much much faster. Don't know if that is =
possible at all, though.
Cheers,
Bernd
-- =
Bernd Schubert
PCI / Theoretische Chemie
Universit=E4t Heidelberg
INF 229
69120 Heidelberg
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3DDE=
VDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thu, Jan 25, 2007 at 11:50:21PM +0100, Bernd Schubert wrote:
> Hi Christian,
>
> > 0:05 - Server: locally on RAID-5
> > 0:06 - Client: copying from NFS mount to tmpfs
> > 2:31 - Client: Copying from NFS mount to NFS mount
> > (i.e., the simple command above)
> > 3:30 - Client: Copying from NFS mount to NFS mount over an existing
> > tree.
> > 5:08 - Client: Copying from tmpfs to NFS mount
> > 0:48 - Client: Deleting copy of tree on NFS mount
>
> I guess your exports has the "sync" option or your didn't set async or sync
> at all (sync is then the default)? Can you repeat the test again using the
> async option? But please, also read carefully the explanations of "man 5
> exports" about this topic.
This /was/ using the async option. Using sync causes the times to worsen
by about 70% (so the NFS to NFS copy takes about 5 minutes).
> Just for fun, take ethereal and capture some data, once with the async and
> once with the sync option. Now have a look how much time each operation of
> the nfs client takes before it gets confirmed by the server. Especially look
> at the commit calls, once with sync and once with async option.
Yeah, that's a great idea; I should do that tomorrow.
> I guess if one could convince the nfs clients to reduce the number of commit
> calls, write operations would be much much faster. Don't know if that is
> possible at all, though.
I get the feeling that the real bottleneck is synchronous file
creation. What I would like to understand is whether this is done for
reliability or to preserve filesystem semantics.
--
Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 3376 0125
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs