Lorn Kay wrote:
>
> Is NFS a viable CFS? (I'm cross posting this due to a discussion on
> the the linux-ha list recently.)
Since there is not a really good cluster filesystem for Linux that is
not either "half baked" (IMHO - I'm probably going to get smacked over
that statement!) or cost an arm and a leg, this is exactly the route we
have taken.
> The NFS server is a high-quality dedicated machine (Net App, EMC,
> etc.)
We've had great success with just using SMP Linux servers. We do have
one EMC IP4700 in production, and it's a nice system, but I prefer the
Linux based alternative.
> Linux clients can use TCP instead of UDP.
Although I haven't had problems with this in our lab, I believe the NFS
authors still consider this experimental.
--
-[========================]-
-[ Brian Tinsley ]-
-[ Chief Systems Engineer ]-
-[ Emageon ]-
-[========================]-
-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
> -----Original Message-----
> From: Brian Tinsley [mailto:[email protected]]
> Sent: Thursday, January 09, 2003 4:11 PM
> To: Lorn Kay
> Cc: [email protected]; [email protected]
> Subject: [NFS] Re: NFS as a Cluster File System.
>
> > Linux clients can use TCP instead of UDP.
>
> Although I haven't had problems with this in our lab, I
> believe the NFS authors still consider this experimental.
the Linux NFS client support for TCP is not experimental.
perhaps less mature than UDP, but definitely not experimental.
the server, OTOH, sports brand new support for TCP. is that
what you were referring to?
-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
>>>>> " " == Lorn Kay <[email protected]> writes:
>> NFS V3 and before have problems with "cache coherency". That
>> is, the different nodes in the cluster are not guaranteed to
>> see the same contents.
> I think this can be resolved with the "noac" mount option
> (prior to V4).
Nope. It can only be resolved using file locking.
Cheers,
Trond
-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Lever, Charles wrote:
>>-----Original Message-----
>>From: Brian Tinsley [mailto:[email protected]]
>>Sent: Thursday, January 09, 2003 4:11 PM
>>To: Lorn Kay
>>Cc: [email protected]; [email protected]
>>Subject: [NFS] Re: NFS as a Cluster File System.
>>
>>> Linux clients can use TCP instead of UDP.
>>>
>>>
>>Although I haven't had problems with this in our lab, I
>>believe the NFS authors still consider this experimental.
>>
>>
>
>the Linux NFS client support for TCP is not experimental.
>perhaps less mature than UDP, but definitely not experimental.
>
>the server, OTOH, sports brand new support for TCP. is that
>what you were referring to?
>
>
Yes, sorry to confuse the issue.
--
-[========================]-
-[ Brian Tinsley ]-
-[ Chief Systems Engineer ]-
-[ Emageon ]-
-[========================]-
>
> >> NFS V3 and before have problems with "cache coherency". That
> >> is, the different nodes in the cluster are not guaranteed to
> >> see the same contents.
>
> > I think this can be resolved with the "noac" mount option
> > (prior to V4).
>
>Nope. It can only be resolved using file locking.
>
>Cheers,
> Trond
>
Meaning if you don't lock a file and just read it you may not see the what
another client has written to it, or is that not an issue because the other
client will have locked and then unlocked the file when it is done making
changes?
Thanks,
--K
_________________________________________________________________
MSN 8 with e-mail virus protection service: 2 months FREE*
http://join.msn.com/?page=features/virus
-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
> > I think this can be resolved with the "noac" mount option
> > (prior to V4).
>
> Nope. It can only be resolved using file locking.
There are 2 consistency problems: metadata and data.
Metadata is solved by noac. And yes, some MPI programs do things like
"node 0 writes out a bunch of files, then tells all the other nodes to
read one file each." This means that you have about 1/100 of a second
window between creation on one client and reading on a different client.
Data consistency is solved by file locking.
-- greg
-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
>>>>> " " == Greg Lindahl <[email protected]> writes:
> Metadata is solved by noac.
Nope. 'noac' doesn't even do that for you:
'noac' just increases the frequency of GETATTR calls. It will not
suffice to provide consistent metadata for those operations that
actually depend on the returned values, as there will be no atomicity
guarantees between the GETATTR call and the subsequent
READ/WRITE/READDIR/...
A common example where this is relevant: You have absolutely no
guarantee with 'noac' that open("foo",O_APPEND,O_WRONLY), will work as
expected.
So, again I repeat: File locking is the only way to ensure cache
coherency.
Cheers,
Trond
-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
>>>>> " " == Lorn Kay <[email protected]> writes:
> Meaning if you don't lock a file and just read it you may not
> see the what another client has written to it, or is that not
> an issue because the other client will have locked and then
> unlocked the file when it is done making changes?
Meaning that unless *all* clients lock the file prior to reading or
writing from it (and then unlock it when done), there will be
caching inconsistencies.
Cheers,
Trond
-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thursday 09 January 2003 03:11 pm, Brian Tinsley wrote:
> Lorn Kay wrote:
> > Is NFS a viable CFS? (I'm cross posting this due to a discussion on
> > the the linux-ha list recently.)
>
> Since there is not a really good cluster filesystem for Linux that is
> not either "half baked"
Hey, we're working on it ;)
--Brian Jackson
OpenGFS Project
> (IMHO - I'm probably going to get smacked over
> that statement!) or cost an arm and a leg, this is exactly the route we
> have taken.
>
> > The NFS server is a high-quality dedicated machine (Net App, EMC,
> > etc.)
>
> We've had great success with just using SMP Linux servers. We do have
> one EMC IP4700 in production, and it's a nice system, but I prefer the
> Linux based alternative.
>
> > Linux clients can use TCP instead of UDP.
>
> Although I haven't had problems with this in our lab, I believe the NFS
> authors still consider this experimental.
-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs