2005-04-07 17:14:47

by Vasiliy Boulytchev

[permalink] [raw]
Subject: NFS experince story, questions.


Ladies and Gents,
In my previous experience, I have never had good luck with NFS. We
used NFS to balance mail server queues (for spam and virus filtering)
between scanning nodes, and we ran into tons of problems, probably due
to lack of complete understanding and proper configuration on my part.
The performance was great, however, we kept getting stale mounts, and to
get things back, both sides needed to be rebooted, I could not just
unmount. Maybe we were not mounting properly, but who knows. Anyways,
as an alternative we looked into samba, and that solved all our issues.
We still run samba as the networking protocol, and are happy with it.

A new task has come up, and we are looking back at NFS, since samba
really does not allow us to have proper user arrangements. I really am
afraid of having stale mounts again. Is this a common newbie problem,
or is this the nature of NFS? When either of the nodes becomes
unstable, has to reboot, (or a failover occurs), does this mean the
other node will have a stale mount point and will need a reboot?

Please assure me that this is not the case, and I can safely go on
experimenting. What is the latest material I should get? Any good
O'reilly books come to mind?

Thanks for paying attention.

--
-------------
Vasiliy Boulytchev
Colorado Information Technologies Inc.
http://www.coinfotech.com



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs


2005-04-07 17:37:00

by Lever, Charles

[permalink] [raw]
Subject: RE: NFS experince story, questions.

hi vasiliy-

> A new task has come up, and we are looking back at NFS,=20
> since samba really does not allow us to have proper user=20
> arrangements. I really am afraid of having stale mounts=20
> again.

i'm not sure what you mean by "stale mount". can you describe the
problem in detail?

> Is this a common newbie problem, or is this the=20
> nature of NFS?

take a look at the Linux NFS FAQ to see if there are useful answers for
you:

http://nfs.sourceforge.net/

> Please assure me that this is not the case, and I can=20
> safely go on experimenting. What is the latest material I=20
> should get? Any good O'reilly books come to mind?

the FAQ (mentioned above) has links to some generic NFS references you
may find helpful.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2005-04-07 17:39:55

by Trond Myklebust

[permalink] [raw]
Subject: Re: NFS experince story, questions.

to den 07.04.2005 Klokka 11:14 (-0600) skreiv Vasiliy Boulytchev:
> A new task has come up, and we are looking back at NFS, since samba
> really does not allow us to have proper user arrangements. I really am
> afraid of having stale mounts again. Is this a common newbie problem,
> or is this the nature of NFS? When either of the nodes becomes
> unstable, has to reboot, (or a failover occurs), does this mean the
> other node will have a stale mount point and will need a reboot?

A server reboot should not require a client reboot, given a properly
configured system, however you do need to understand some of the
limitations of the NFS model.

NFS relies heavily on the use of "filehandles" in order to ensure that
we can retain posix behaviour in cases where a rename() of the file or
some element of the path occurs on the server.
In order to have the required properties, filehandles tend to have to
encode information such as device id (known as the "fsid" in NFS-lingo)
and file inode number. If one or more of those changes, then an ESTALE
is likely to occur.

One common cause of ESTALE on reboot is that the device id changes after
reboot (fibre channel being a notable culprit). In order to fix the
problem of the filehandles going stale, the Linux kernel server has an
export option "fsid" which allows you to fix the device id to some
arbitrary value. (See the "exports" manpage for more info)

The other common problem is that the filesystem may not support any
equivalent of the inode number in permanent storage (MS filesystems tend
to have a problem here). We currently have no good solution for that
problem on Linux (partly because the NFSv2/v3 protocols did not really
allow for it at all). We are certainly planning on hardening the client,
and allowing for some recovery, but the whole problem can be avoided, by
choosing a filesystem with good NFS support (ext2/ext3, XFS, reiserfs3
are among the list of good choices). The NFS FAQ has a more complete
section on this issue.

Cheers,
Trond
--
Trond Myklebust <[email protected]>



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2005-04-07 20:09:42

by Neil Horman

[permalink] [raw]
Subject: Re: NFS experince story, questions.

On Thu, Apr 07, 2005 at 11:14:41AM -0600, Vasiliy Boulytchev wrote:
>
> Ladies and Gents,
> In my previous experience, I have never had good luck with NFS. We
> used NFS to balance mail server queues (for spam and virus filtering)
> between scanning nodes, and we ran into tons of problems, probably due
> to lack of complete understanding and proper configuration on my part.
> The performance was great, however, we kept getting stale mounts, and to
> get things back, both sides needed to be rebooted, I could not just
> unmount. Maybe we were not mounting properly, but who knows. Anyways,
> as an alternative we looked into samba, and that solved all our issues.
> We still run samba as the networking protocol, and are happy with it.
>
> A new task has come up, and we are looking back at NFS, since samba
> really does not allow us to have proper user arrangements. I really am
> afraid of having stale mounts again. Is this a common newbie problem,
> or is this the nature of NFS? When either of the nodes becomes
> unstable, has to reboot, (or a failover occurs), does this mean the
> other node will have a stale mount point and will need a reboot?
>
When you say "stale mounts" do you mean file access operations are returning a
"Stale File handle" error?

Regards
Neil

--
/***************************************************
*Neil Horman
*Software Engineer
*Red Hat, Inc.
*[email protected]
*gpg keyid: 1024D / 0x92A74FA1
*http://pgp.mit.edu
***************************************************/


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2005-04-07 21:24:07

by Vasiliy Boulytchev

[permalink] [raw]
Subject: Re: NFS experince story, questions.


Thank you very much. I will read on!

On Thu, 2005-04-07 at 13:39 -0400, Trond Myklebust wrote:
> to den 07.04.2005 Klokka 11:14 (-0600) skreiv Vasiliy Boulytchev:
> > A new task has come up, and we are looking back at NFS, since samba
> > really does not allow us to have proper user arrangements. I really am
> > afraid of having stale mounts again. Is this a common newbie problem,
> > or is this the nature of NFS? When either of the nodes becomes
> > unstable, has to reboot, (or a failover occurs), does this mean the
> > other node will have a stale mount point and will need a reboot?
>
> A server reboot should not require a client reboot, given a properly
> configured system, however you do need to understand some of the
> limitations of the NFS model.
>
> NFS relies heavily on the use of "filehandles" in order to ensure that
> we can retain posix behaviour in cases where a rename() of the file or
> some element of the path occurs on the server.
> In order to have the required properties, filehandles tend to have to
> encode information such as device id (known as the "fsid" in NFS-lingo)
> and file inode number. If one or more of those changes, then an ESTALE
> is likely to occur.
>
> One common cause of ESTALE on reboot is that the device id changes after
> reboot (fibre channel being a notable culprit). In order to fix the
> problem of the filehandles going stale, the Linux kernel server has an
> export option "fsid" which allows you to fix the device id to some
> arbitrary value. (See the "exports" manpage for more info)
>
> The other common problem is that the filesystem may not support any
> equivalent of the inode number in permanent storage (MS filesystems tend
> to have a problem here). We currently have no good solution for that
> problem on Linux (partly because the NFSv2/v3 protocols did not really
> allow for it at all). We are certainly planning on hardening the client,
> and allowing for some recovery, but the whole problem can be avoided, by
> choosing a filesystem with good NFS support (ext2/ext3, XFS, reiserfs3
> are among the list of good choices). The NFS FAQ has a more complete
> section on this issue.
>
> Cheers,
> Trond
--

-------------
Vasiliy Boulytchev
Colorado Information Technologies Inc.
http://www.coinfotech.com



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs