I've searched/browsed the mail archive and the online documentation but am
still a bit clueless about where to start.
I am working on a scientific data analysis application which needs to
extract a little bit of data from around 10,000 files. The files
themselves are each around 1 MB in size. The application does:
for (i=0; i<10000; i++){
open() ith file
read() 32 bytes from beginning
lseek() to somewhere in the middle of the file
read() 620 bytes from the middle of the file
close() ith file
}
If I run this with the data set on a local disk, the total run time is
around 1 sec -- very reasonable since I am transferring around 6 MB of
data.
If I run this on an NFS-mounted disk the performance is 100 times worse.
Both client and server are stock RH 7.3. Note that if I run it a second
time, when the data is presumably cached, it takes just a few seconds to
run.
Is it obvious what needs to be changed/tuned to improve this performance?
Or if not, could someone suggest a tool and a strategy for tracing down
the cause of the poor performance?
Sincerely,
Bruce Allen
-------------------------------------------------------
This sf.net email is sponsored by:
With Great Power, Comes Great Responsibility
Learn to use your power at OSDN's High Performance Computing Channel
http://hpc.devchannel.org/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Hi Bruce (good name, btw),
It would help if you posted information about your network. Alternately,
you could read the "Optimizing NFS Performance" section of the Linux
NFS-HOWTO located at http://nfs.sourceforge.net/nfs-howto/
Regards,
---
Bruce Allan <[email protected]>
Software Engineer, Linux Technology Center
IBM Corporation, Beaverton OR
503-578-4187 IBM Tie-line 775-4187
Bruce Allen
<[email protected] To: [email protected]
.uwm.edu> cc: Bruce Allen <[email protected]>
Sent by: Subject: [NFS] slow NFS performance extracting bits from large files
[email protected]
ceforge.net
12/15/2002 11:02 PM
I've searched/browsed the mail archive and the online documentation but am
still a bit clueless about where to start.
I am working on a scientific data analysis application which needs to
extract a little bit of data from around 10,000 files. The files
themselves are each around 1 MB in size. The application does:
for (i=0; i<10000; i++){
open() ith file
read() 32 bytes from beginning
lseek() to somewhere in the middle of the file
read() 620 bytes from the middle of the file
close() ith file
}
If I run this with the data set on a local disk, the total run time is
around 1 sec -- very reasonable since I am transferring around 6 MB of
data.
If I run this on an NFS-mounted disk the performance is 100 times worse.
Both client and server are stock RH 7.3. Note that if I run it a second
time, when the data is presumably cached, it takes just a few seconds to
run.
Is it obvious what needs to be changed/tuned to improve this performance?
Or if not, could someone suggest a tool and a strategy for tracing down
the cause of the poor performance?
Sincerely,
Bruce Allen
-------------------------------------------------------
This sf.net email is sponsored by:
With Great Power, Comes Great Responsibility
Learn to use your power at OSDN's High Performance Computing Channel
http://hpc.devchannel.org/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
-------------------------------------------------------
This sf.net email is sponsored by:
With Great Power, Comes Great Responsibility
Learn to use your power at OSDN's High Performance Computing Channel
http://hpc.devchannel.org/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Hi Paul,
Thanks for the quick reply.
> > I am working on a scientific data analysis application which needs to
> > extract a little bit of data from around 10,000 files. The files
> > themselves are each around 1 MB in size. The application does:
> >
> > for (i=0; i<10000; i++){
> > open() ith file
> > read() 32 bytes from beginning
> > lseek() to somewhere in the middle of the file
> > read() 620 bytes from the middle of the file
> > close() ith file
> > }
>
> So that's 20,000 reads. Readahead wouldn't help much with this access
> pattern, so it looks like 20,000 reads and 20,000 seeks.
> > If I run this with the data set on a local disk, the total run time is
> > around 1 sec -- very reasonable since I am transferring around 6 MB of
> > data.
>
> Ok, 20,000 seeks per second, shows an average seek time of 50
> microseconds. You can't do that with a spinning disk. You've discovered
> how well disk caching works.
Thank you very much. This is a reasonable (obvious!) explanation. Let me
see if I undertand correctly.
I just looked up the specs for the local disk that I was using for the
comparison testing. It averages around 600 512-byte sectors (300 kB) per
track, and 1ms average track to track seek time. So (assuming little
fragmentation) a typical file is spread across 3 to 4 tracks of the disk.
Thus, if I seek to a data set in the middle of a file, and assuming that
the files are "more or less" contiguous on the disk, a typical iteration
of the loop above starting from the first read, should involve:
read 32 bytes
seek to middle (2 tracks) 2 msec
read 620 bytes
seek to next file (2 tracks) 2 msec
hence 4 msec/file x 10k files = 40 sec read time if the kernel disk cache
buffers are all dirty. Does the estimate above look reasonable?
I'll do the experiment that you suggest, using wc to clear the cache
first, and see how this compares to what's above.
> > If I run this on an NFS-mounted disk the performance is 100 times worse.
> > Both client and server are stock RH 7.3. Note that if I run it a second
> > time, when the data is presumably cached, it takes just a few seconds to
> > run.
>
> 20000 seeks in 100 seconds, that's an average seek time of 5 ms. That's
> possible on real hardware.
The files on the NFS server live on an ATA RAID array, made of 3-ware 7850
controller running in hardware RAID-5 with the data striped across 7 disks
(the 8th disk is a hot spare). I think the stripe size is configured to
the maximum value, but I have forgotten what that is. I have no idea what
the equivalent "seek time" on such a system is.
The client and the server are networked via gigabit ethernet, through a
Foundry Networks switch. Neither the switch nor the client or server are
oversubscribed.
> > Is it obvious what needs to be changed/tuned to improve this performance?
> > Or if not, could someone suggest a tool and a strategy for tracing down
> > the cause of the poor performance?
>
> Clear the cache in the system with the local disk and measure the
> performance. You can clear the cache by reading files bigger than all of
> RAM. I tend to use "wc *" in a directory with some large tar files. wc
> kindly tells me how many bytes I've read, so I know if I've read enough.
>
> With a clear cache you'll see local performance numbers that are closer to
> the NFS numbers.
OK, I'll try this. In retrospect, it's obvious.
> What's the ping time? The NFS performance should have a penalty of
> something like 3 network transactions per file, so your NFS times should
> be 30,000 packet round trips longer than the local case. (maybe 4 network
> transactions per file if there is also a stat call, a network trace would
> show all the transactions).
There's no stat call -- I've used strace to verify that I am only doing
the operations described in my pseudo-code above.
open("/scratch/xavi/CAL_L1_Data//CAL_SFT.714299460", O_RDONLY) = 3
read(3, "\0\0\0\0\0\0\360?DX\223*\0\0\0\0\0\0\0\0\0\0N@\270\v\0"..., 32) =
32
lseek(3, 460776, SEEK_CUR) = 460808
read(3, "\336\203z&\tM\216&/\342\227\244\307\333\352\245\233\27"...,
620) = 620
close(3) = 0
open("/scratch/xavi/CAL_L1_Data//CAL_SFT.714299520", O_RDONLY) = 3
There's no stat(). So I think it's 3 not 4. Here are the ping times:
[ballen@medusa-slave123 ballen]$ ping storage1
PING storage1.medusa.phys.uwm.edu (129.89.201.244) from 129.89.200.133
: 56(84) bytes of data.
64 bytes from storage1.medusa.phys.uwm.edu (129.89.201.244): icmp_seq=1
ttl=255 time=0.197 ms
64 bytes from storage1.medusa.phys.uwm.edu (129.89.201.244): icmp_seq=2
ttl=255 time=0.264 ms
....
--- storage1.medusa.phys.uwm.edu ping statistics ---
26 packets transmitted, 26 received, 0% loss, time 25250ms
rtt min/avg/max/mdev = 0.156/0.230/0.499/0.063 ms
so 230 usec (average) x 30,000 = 7 seconds
OK, so if all is well, then if I run the code locally on the NFS server
itself (with disk cache buffers all dirty) it should take around 7
seconds less time than if I run it on an NFS client.
I'll check the numbers and report back.
Thanks again for the help. I'd not thought carefully enough about what
the disk was really doing. The point being that reading a contiguous 6 MB
file from the local disk only involves ~20 track-to-track seeks, which is
negligable in comparison with what I am doing.
Cheers,
Bruce
-------------------------------------------------------
This sf.net email is sponsored by:
With Great Power, Comes Great Responsibility
Learn to use your power at OSDN's High Performance Computing Channel
http://hpc.devchannel.org/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Hi Bruce,
Problem solved. The point is that the code requires a lot of disks seeks
(20,000), which cost milliseconds each. This is the dominant cost, not
the read()s. Please see the thread with Paul Haas who nailed the problem
on his first reply.
Bruce Allen
On Mon, 16 Dec 2002, Bruce Allan wrote:
>
>
>
>
> Hi Bruce (good name, btw),
>
> It would help if you posted information about your network. Alternately,
> you could read the "Optimizing NFS Performance" section of the Linux
> NFS-HOWTO located at http://nfs.sourceforge.net/nfs-howto/
>
> Regards,
> ---
> Bruce Allan <[email protected]>
> Software Engineer, Linux Technology Center
> IBM Corporation, Beaverton OR
> 503-578-4187 IBM Tie-line 775-4187
>
>
>
>
> Bruce Allen
> <[email protected] To: [email protected]
> .uwm.edu> cc: Bruce Allen <[email protected]>
> Sent by: Subject: [NFS] slow NFS performance extracting bits from large files
> [email protected]
> ceforge.net
>
>
> 12/15/2002 11:02 PM
>
>
>
>
>
> I've searched/browsed the mail archive and the online documentation but am
> still a bit clueless about where to start.
>
> I am working on a scientific data analysis application which needs to
> extract a little bit of data from around 10,000 files. The files
> themselves are each around 1 MB in size. The application does:
>
> for (i=0; i<10000; i++){
> open() ith file
> read() 32 bytes from beginning
> lseek() to somewhere in the middle of the file
> read() 620 bytes from the middle of the file
> close() ith file
> }
>
> If I run this with the data set on a local disk, the total run time is
> around 1 sec -- very reasonable since I am transferring around 6 MB of
> data.
>
> If I run this on an NFS-mounted disk the performance is 100 times worse.
> Both client and server are stock RH 7.3. Note that if I run it a second
> time, when the data is presumably cached, it takes just a few seconds to
> run.
>
> Is it obvious what needs to be changed/tuned to improve this performance?
> Or if not, could someone suggest a tool and a strategy for tracing down
> the cause of the poor performance?
>
> Sincerely,
> Bruce Allen
>
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:
> With Great Power, Comes Great Responsibility
> Learn to use your power at OSDN's High Performance Computing Channel
> http://hpc.devchannel.org/
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs
>
>
>
>
-------------------------------------------------------
This sf.net email is sponsored by:
With Great Power, Comes Great Responsibility
Learn to use your power at OSDN's High Performance Computing Channel
http://hpc.devchannel.org/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Hi Paul,
Well, this thread is about to terminate. You nailed it the first
time. The results are (after dirtying the local disk cache):
On local disk: 2 m 45 s = 165 secs.
On NFS disk: 2 min 50 s = 170 secs.
In the long run we may want to reformat/redistribute the data so that
we're not seek-limited like this.
Thank you very much for explaning this to me. I was really puzzled about
what was going on.
Bruce Allen
On Mon, 16 Dec 2002, Bruce Allen wrote:
> Hi Paul,
>
> Thanks for the quick reply.
>
> > > I am working on a scientific data analysis application which needs to
> > > extract a little bit of data from around 10,000 files. The files
> > > themselves are each around 1 MB in size. The application does:
> > >
> > > for (i=0; i<10000; i++){
> > > open() ith file
> > > read() 32 bytes from beginning
> > > lseek() to somewhere in the middle of the file
> > > read() 620 bytes from the middle of the file
> > > close() ith file
> > > }
> >
> > So that's 20,000 reads. Readahead wouldn't help much with this access
> > pattern, so it looks like 20,000 reads and 20,000 seeks.
>
> > > If I run this with the data set on a local disk, the total run time is
> > > around 1 sec -- very reasonable since I am transferring around 6 MB of
> > > data.
> >
> > Ok, 20,000 seeks per second, shows an average seek time of 50
> > microseconds. You can't do that with a spinning disk. You've discovered
> > how well disk caching works.
>
> Thank you very much. This is a reasonable (obvious!) explanation. Let me
> see if I undertand correctly.
>
> I just looked up the specs for the local disk that I was using for the
> comparison testing. It averages around 600 512-byte sectors (300 kB) per
> track, and 1ms average track to track seek time. So (assuming little
> fragmentation) a typical file is spread across 3 to 4 tracks of the disk.
> Thus, if I seek to a data set in the middle of a file, and assuming that
> the files are "more or less" contiguous on the disk, a typical iteration
> of the loop above starting from the first read, should involve:
>
> read 32 bytes
> seek to middle (2 tracks) 2 msec
> read 620 bytes
> seek to next file (2 tracks) 2 msec
>
> hence 4 msec/file x 10k files = 40 sec read time if the kernel disk cache
> buffers are all dirty. Does the estimate above look reasonable?
>
> I'll do the experiment that you suggest, using wc to clear the cache
> first, and see how this compares to what's above.
>
> > > If I run this on an NFS-mounted disk the performance is 100 times worse.
> > > Both client and server are stock RH 7.3. Note that if I run it a second
> > > time, when the data is presumably cached, it takes just a few seconds to
> > > run.
> >
> > 20000 seeks in 100 seconds, that's an average seek time of 5 ms. That's
> > possible on real hardware.
>
> The files on the NFS server live on an ATA RAID array, made of 3-ware 7850
> controller running in hardware RAID-5 with the data striped across 7 disks
> (the 8th disk is a hot spare). I think the stripe size is configured to
> the maximum value, but I have forgotten what that is. I have no idea what
> the equivalent "seek time" on such a system is.
>
> The client and the server are networked via gigabit ethernet, through a
> Foundry Networks switch. Neither the switch nor the client or server are
> oversubscribed.
>
> > > Is it obvious what needs to be changed/tuned to improve this performance?
> > > Or if not, could someone suggest a tool and a strategy for tracing down
> > > the cause of the poor performance?
> >
> > Clear the cache in the system with the local disk and measure the
> > performance. You can clear the cache by reading files bigger than all of
> > RAM. I tend to use "wc *" in a directory with some large tar files. wc
> > kindly tells me how many bytes I've read, so I know if I've read enough.
> >
> > With a clear cache you'll see local performance numbers that are closer to
> > the NFS numbers.
>
> OK, I'll try this. In retrospect, it's obvious.
>
> > What's the ping time? The NFS performance should have a penalty of
> > something like 3 network transactions per file, so your NFS times should
> > be 30,000 packet round trips longer than the local case. (maybe 4 network
> > transactions per file if there is also a stat call, a network trace would
> > show all the transactions).
>
> There's no stat call -- I've used strace to verify that I am only doing
> the operations described in my pseudo-code above.
>
> open("/scratch/xavi/CAL_L1_Data//CAL_SFT.714299460", O_RDONLY) = 3
> read(3, "\0\0\0\0\0\0\360?DX\223*\0\0\0\0\0\0\0\0\0\0N@\270\v\0"..., 32) =
> 32
> lseek(3, 460776, SEEK_CUR) = 460808
> read(3, "\336\203z&\tM\216&/\342\227\244\307\333\352\245\233\27"...,
> 620) = 620
> close(3) = 0
> open("/scratch/xavi/CAL_L1_Data//CAL_SFT.714299520", O_RDONLY) = 3
>
> There's no stat(). So I think it's 3 not 4. Here are the ping times:
>
> [ballen@medusa-slave123 ballen]$ ping storage1
> PING storage1.medusa.phys.uwm.edu (129.89.201.244) from 129.89.200.133
> : 56(84) bytes of data.
> 64 bytes from storage1.medusa.phys.uwm.edu (129.89.201.244): icmp_seq=1
> ttl=255 time=0.197 ms
> 64 bytes from storage1.medusa.phys.uwm.edu (129.89.201.244): icmp_seq=2
> ttl=255 time=0.264 ms
> ....
> --- storage1.medusa.phys.uwm.edu ping statistics ---
> 26 packets transmitted, 26 received, 0% loss, time 25250ms
> rtt min/avg/max/mdev = 0.156/0.230/0.499/0.063 ms
>
> so 230 usec (average) x 30,000 = 7 seconds
>
> OK, so if all is well, then if I run the code locally on the NFS server
> itself (with disk cache buffers all dirty) it should take around 7
> seconds less time than if I run it on an NFS client.
>
> I'll check the numbers and report back.
>
> Thanks again for the help. I'd not thought carefully enough about what
> the disk was really doing. The point being that reading a contiguous 6 MB
> file from the local disk only involves ~20 track-to-track seeks, which is
> negligable in comparison with what I am doing.
>
> Cheers,
> Bruce
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:
> With Great Power, Comes Great Responsibility
> Learn to use your power at OSDN's High Performance Computing Channel
> http://hpc.devchannel.org/
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs
>
-------------------------------------------------------
This sf.net email is sponsored by:
With Great Power, Comes Great Responsibility
Learn to use your power at OSDN's High Performance Computing Channel
http://hpc.devchannel.org/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
> Thank you very much. This is a reasonable (obvious!) explanation. Let =
me
> see if I undertand correctly.
>=20
> I just looked up the specs for the local disk that I was using for the
> comparison testing. It averages around 600 512-byte sectors (300 kB) =
per
> track, and 1ms average track to track seek time. So (assuming little
> fragmentation) a typical file is spread across 3 to 4 tracks of the =
disk. =20
> Thus, if I seek to a data set in the middle of a file, and assuming =
that
> the files are "more or less" contiguous on the disk, a typical =
iteration
> of the loop above starting from the first read, should involve:
>=20
> read 32 bytes
> seek to middle (2 tracks) 2 msec
> read 620 bytes
> seek to next file (2 tracks) 2 msec
>=20
> hence 4 msec/file x 10k files =3D 40 sec read time if the kernel disk =
cache
> buffers are all dirty. Does the estimate above look reasonable?
>=20
> I'll do the experiment that you suggest, using wc to clear the cache
> first, and see how this compares to what's above.
>=20
Track to track does not really count. The average time per io is
1/2 of the rotational latency. If it is a 7200 rpm disk, it means
that even once you get to the proper track it takes 1/2 * 60/7200 =
seconds
for the proper data to come under the head on average. The number
for the above is 4.2ms and then you need to add in the track-to-track
number to get 5.2ms or so per operation. And probably you have a =
operation
also to open each file. And given that the filename entry data (name, =
and such) may
not be close, it may not be a short quick seek, it may take longer. =
Exactly
how many seeks you have depends on exactly how the filesystem lays
the data out on disk.
So you have open, read32b, read620b, each taking 5.2ms or so, so a =
total
of 15.6ms per file, times 10k files =3D 156 seconds, and that is in a =
perfect
world. You may also have a operation to update the file access time =
since
you have read the file, and each of the seeks could require a read to =
figure
out exactly where that block actually is on disk. So there could be as
many as 4 reads and a write going on on the far end. And then on top
of that you have the network latency of around a 1 ms or so to send
the packets back and forth for each operation. And my experience in
the past has been that opens by themselves are pretty expensive, so
there may be alot of operations going on in the background to make an
open happen. =20
> ttl=3D255 time=3D0.264 ms
> ....
> --- storage1.medusa.phys.uwm.edu ping statistics ---
> 26 packets transmitted, 26 received, 0% loss, time 25250ms
> rtt min/avg/max/mdev =3D 0.156/0.230/0.499/0.063 ms
>=20
> so 230 usec (average) x 30,000 =3D 7 seconds
>=20
> OK, so if all is well, then if I run the code locally on the NFS =
server
> itself (with disk cache buffers all dirty) it should take around 7
> seconds less time than if I run it on an NFS client.
>=20
> I'll check the numbers and report back.
>=20
>=20
On the ping, you really need to ping for at least an hour or more, my
experience is that if you are ping 1/second for an hour and you are
losing 1 of 3600 pings (losing 1 ping per hour) it is very obvious to=20
anything or anyone running that things are being affected, and this
needs to be fixed. A clean network will lose the pings at a rate
of less than 1 of 3600. And if you are losing 1 of 3600
packets this will have ugly results on UDP nfs and even on
other applications.
Roger
> --__--__--
>=20
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs
>=20
>=20
> End of NFS Digest
-------------------------------------------------------
This sf.net email is sponsored by:
With Great Power, Comes Great Responsibility
Learn to use your power at OSDN's High Performance Computing Channel
http://hpc.devchannel.org/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Hi Roger,
> > the files are "more or less" contiguous on the disk, a typical iteration
> > of the loop above starting from the first read, should involve:
> >
> > read 32 bytes
> > seek to middle (2 tracks) 2 msec
> > read 620 bytes
> > seek to next file (2 tracks) 2 msec
> >
> > hence 4 msec/file x 10k files = 40 sec read time if the kernel disk cache
> > buffers are all dirty. Does the estimate above look reasonable?
> >
> > I'll do the experiment that you suggest, using wc to clear the cache
> > first, and see how this compares to what's above.
> >
> Track to track does not really count. The average time per io is
> 1/2 of the rotational latency. If it is a 7200 rpm disk, it means
> that even once you get to the proper track it takes 1/2 * 60/7200 seconds
> for the proper data to come under the head on average. The number
> for the above is 4.2ms and then you need to add in the track-to-track
> number to get 5.2ms or so per operation.
I realized this myself a few hours after I sent my email. I completely
agree with the analysis. It also explains why my measured times where
about four times longer than I had predicted.
So I completely agree with this.
> And probably you have a operation
> also to open each file. And given that the filename entry data (name, and such) may
> not be close, it may not be a short quick seek, it may take longer. Exactly
> how many seeks you have depends on exactly how the filesystem
> lays
> the data out on disk.
It happens that all the files are in the same directory, and were created
at the same time during an otherwise quiescent period of operation. So in
fact I think that the directory is probably read at the very start, and
then sits in cache after that. If so, I think that the other operations
associated with the file access are probably all to/from cache.
> So you have open, read32b, read620b, each taking 5.2ms or so, so a total
> of 15.6ms per file, times 10k files = 156 seconds, and that is in a perfect
> world. You may also have a operation to update the file access time since
It's within 10% of what I measure. I think that the explanation is that
the directory info is all in cache. [I must say I find it quite satisfying
to be able to predict the code's performance this closely.]
> you have read the file, and each of the seeks could require a read to figure
> out exactly where that block actually is on disk. So there could be as
> many as 4 reads and a write going on on the far end. And then on top
> of that you have the network latency of around a 1 ms or so to send
> the packets back and forth for each operation. And my experience in
> the past has been that opens by themselves are pretty expensive, so
> there may be alot of operations going on in the background to make an
> open happen.
And indeed the additional time taken by the NFS-based accesses is also
very close to 30000*ping_time.
> > ttl=255 time=0.264 ms
> > ....
> > --- storage1.medusa.phys.uwm.edu ping statistics ---
> > 26 packets transmitted, 26 received, 0% loss, time 25250ms
> > rtt min/avg/max/mdev = 0.156/0.230/0.499/0.063 ms
> >
> > so 230 usec (average) x 30,000 = 7 seconds
> >
> > OK, so if all is well, then if I run the code locally on the NFS server
> > itself (with disk cache buffers all dirty) it should take around 7
> > seconds less time than if I run it on an NFS client.
> >
> > I'll check the numbers and report back.
> >
> >
> On the ping, you really need to ping for at least an hour or more, my
> experience is that if you are ping 1/second for an hour and you are
> losing 1 of 3600 pings (losing 1 ping per hour) it is very obvious to
> anything or anyone running that things are being affected, and this
> needs to be fixed. A clean network will lose the pings at a rate
> of less than 1 of 3600. And if you are losing 1 of 3600
> packets this will have ugly results on UDP nfs and even on
> other applications.
In our case, the network is completely under control. I'm running on our
research group's beowulf cluster, so it's a private network -- the only
packets on it come from our own applications. So again, I am highly
confident that at least while I am running the code, the round-trip ping
times I showed are reprsentative.
Thanks again for your comments. I completely agree about the
time-per-read above being at least 1msec plus half a rotation period.
Cheers,
Bruce Allen
-------------------------------------------------------
This sf.net email is sponsored by:
With Great Power, Comes Great Responsibility
Learn to use your power at OSDN's High Performance Computing Channel
http://hpc.devchannel.org/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs