2004-10-20 12:47:58

by Stuckless, Colin

[permalink] [raw]
Subject: Linux NFS server for heterogeneous production environment



I'm looking for case studies or reference sites that have moved their
production NFS server environments to Linux. We run a mix of Sun, Linux and
a few SGI clients and are currently running a Solaris 2.8 NFS server.

In looking at upgrade options and of course the debate between NAS or a
server based solution comes up. In my experience, NAS tends to take the
OS/hardware issues and hand them over to your NAS vendor (for better or for
worse). Given my location, there isn't a lot of presence from any of the
major NAS vendors, so I tend to prefer a server based solution that I can
manage more myself. Also, backup options for NAS (NDMP or backing up from
NFS mounts) are not as desirable as running our Legato agents directly on
the file server.

And that brings me to Linux. Inexpensive, reliable and well supported x86
hardware is in plentiful supply, but I'm balking at the maturity of the NFS
stack in Linux in suggesting a move in that direction, given our mix of
clients as I stated above. Has anyone out there done it? What did you use
(vanilla distribution from RedHat or Suse, or addons like Polyserve's Matrix
server)?

At the end of the day, what I would like is a dual server solution (for
scalable performance, and redundancy) with SAN attached storage that is able
to handle a mixture of NFS clients (Solaris and Linux predominantly) with
high speed (Gigabit) connections without breaking a sweat and without
needing a high amount of care and feeding (my goal is to keep storage
management as <5% of my workload as I have others things to do). If you have
that, and don't mind sharing, please email or followup to the list. I'll
promise to do a summary if I do get some replies.

I know with the work that is going into NFS on Linux that this is not a
question of "if", but a question of "when", and I'm wondering if the time is
right and if there are some references I can pull together to support our
own evaluation.


Thanks,
Colin Stuckless





********************

This email communication is intended as a private communication for the sole
use of the primary addressee and those individuals listed for copies in the
original message. The information contained in this email is private and
confidential and If you are not an intended recipient you are hereby
notified that copying, forwarding or other dissemination or distribution of
this communication by any means is prohibited. If you are not specifically
authorized to receive this email and if you believe that you received it in
error please notify the original sender immediately. We honour similar
requests relating to the privacy of email communications.

Cette communication par courrier electronique est une communication privee a
l'usage exclusif du destinataire principal ainsi que des personnes dont les
noms figurent en copie. Les renseignements contenus dans ce courriel sont
confidentiels et si vous n'etes pas le destinataire prevu, vous etes avise,
par les presentes que toute reproduction, transfert ou autre forme de
diffusion de cette communication par quelque moyen que ce soit est
interdite. Si vous n'etes pas specifiquement autorise a recevoir ce
courriel ou si vous croyez l'avoir recu par erreur, veuillez en aviser
l'expediteur original immediatement. Nous respectons les demandes
similaires qui touchent la confidentialite des communications par courrier
electronique.


2004-10-21 14:32:47

by David Dougall

[permalink] [raw]
Subject: Re: Linux NFS server for heterogeneous production environment

We transitioned a few years back from HPUX NFS servers to linux NFS
servers; both running on SAN and gigabit. The HPUX was slightly faster
for some things, but linux can certainly hold its own and has more
aggressive caching and can be faster in many cases.
As far as clients, we have lots of HPUX, linux, Irix, AIX and MAC.
Unfortunately, I have no experience with solaris clients. I have had
little to no problems with any of them except HPUX client locking code is
really flaky(although it was flaky even with HPUX as the server). MAC
also had some issues, but I think that was more because of the default
settings on the client.
Anyway, with good quality hardware, linux is a superb NFS server. My
experience would tell you to go ahead with a linux implementation.
--David Dougall


On Wed, 20 Oct 2004, Stuckless, Colin wrote:

>
>
> I'm looking for case studies or reference sites that have moved their
> production NFS server environments to Linux. We run a mix of Sun, Linux and
> a few SGI clients and are currently running a Solaris 2.8 NFS server.
>
> In looking at upgrade options and of course the debate between NAS or a
> server based solution comes up. In my experience, NAS tends to take the
> OS/hardware issues and hand them over to your NAS vendor (for better or for
> worse). Given my location, there isn't a lot of presence from any of the
> major NAS vendors, so I tend to prefer a server based solution that I can
> manage more myself. Also, backup options for NAS (NDMP or backing up from
> NFS mounts) are not as desirable as running our Legato agents directly on
> the file server.
>
> And that brings me to Linux. Inexpensive, reliable and well supported x86
> hardware is in plentiful supply, but I'm balking at the maturity of the NFS
> stack in Linux in suggesting a move in that direction, given our mix of
> clients as I stated above. Has anyone out there done it? What did you use
> (vanilla distribution from RedHat or Suse, or addons like Polyserve's Matrix
> server)?
>
> At the end of the day, what I would like is a dual server solution (for
> scalable performance, and redundancy) with SAN attached storage that is able
> to handle a mixture of NFS clients (Solaris and Linux predominantly) with
> high speed (Gigabit) connections without breaking a sweat and without
> needing a high amount of care and feeding (my goal is to keep storage
> management as <5% of my workload as I have others things to do). If you have
> that, and don't mind sharing, please email or followup to the list. I'll
> promise to do a summary if I do get some replies.
>
> I know with the work that is going into NFS on Linux that this is not a
> question of "if", but a question of "when", and I'm wondering if the time is
> right and if there are some references I can pull together to support our
> own evaluation.
>
>
> Thanks,
> Colin Stuckless
>
>
>
>
>
> ********************
>
> This email communication is intended as a private communication for the sole
> use of the primary addressee and those individuals listed for copies in the
> original message. The information contained in this email is private and
> confidential and If you are not an intended recipient you are hereby
> notified that copying, forwarding or other dissemination or distribution of
> this communication by any means is prohibited. If you are not specifically
> authorized to receive this email and if you believe that you received it in
> error please notify the original sender immediately. We honour similar
> requests relating to the privacy of email communications.
>
> Cette communication par courrier electronique est une communication privee a
> l'usage exclusif du destinataire principal ainsi que des personnes dont les
> noms figurent en copie. Les renseignements contenus dans ce courriel sont
> confidentiels et si vous n'etes pas le destinataire prevu, vous etes avise,
> par les presentes que toute reproduction, transfert ou autre forme de
> diffusion de cette communication par quelque moyen que ce soit est
> interdite. Si vous n'etes pas specifiquement autorise a recevoir ce
> courriel ou si vous croyez l'avoir recu par erreur, veuillez en aviser
> l'expediteur original immediatement. Nous respectons les demandes
> similaires qui touchent la confidentialite des communications par courrier
> electronique.
>


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2004-10-21 18:02:14

by Michael Richardson

[permalink] [raw]
Subject: Re: Linux NFS server for heterogeneous production environment

-----BEGIN PGP SIGNED MESSAGE-----


>>>>> "David" == David Dougall <[email protected]> writes:
David> faster in many cases. As far as clients, we have lots of
David> HPUX, linux, Irix, AIX and MAC. Unfortunately, I have no
David> experience with solaris clients. I have had little to no

Solaris clients are the most demanding of clients, protocol-wise.

David> problems with any of them except HPUX client locking code is
David> really flaky(although it was flaky even with HPUX as the

Yes, HPUX never did get the locking code right.
Were you using NFSv2 or v3? Did it vary by client?

Did you do lots of locking?
Did you have mailspool on NFS?
Any big, long running jobs? (e.g. Verilog simulations)

- --
] "Elmo went to the wrong fundraiser" - The Simpson | firewalls [
] Michael Richardson, Xelerance Corporation, Ottawa, ON |net architect[
] [email protected] http://www.sandelman.ottawa.on.ca/mcr/ |device driver[
] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Finger me for keys

iQCVAwUBQXf5VIqHRg3pndX9AQFEIAQAk0aSi6wJn3CAh5Rc+FbjzF78lkd5mWe1
IgNkdMEMa+Jle9HJIOYYvXwnbYapHWu80T4ydJ8Ihyfg+fYavGNituyDVoSp3CX5
d5V/+cAbz0O4NaqhTQBO1L4LST4oKAEWnWsvte8MmFyycIQ9wwbHPHPiw/6/84Kw
we0WeOIf1PA=
=YF6E
-----END PGP SIGNATURE-----


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2004-10-21 20:47:18

by David Dougall

[permalink] [raw]
Subject: Re: Linux NFS server for heterogeneous production environment

I had locking trouble with both nfsv2 and nfsv3. HPUX locking would work
based on whatever patchset was blessed by the gods. If I upgraded a patch
on client or server out of step with all the others, it would cease to
work, etc.
Anyway, are your questions referring to before or after we changed to
linux NFS? I assume after, so I will answer that.
We do lots of locking now. Our HPUX machines still have trouble at times.
We used to do mbox mailboxes over nfs. That was very ugly. We moved to
mbox mailboxes on a local drive, but we have since moved to maildir files
over nfs. It works without a hitch.
We have tons of simulations that take lots of time. Some program
simulations run for months on end over NFS. (Although some would argue
that if it wasn't over NFS it would take a lot less time :P)
--David Dougall


On Thu, 21 Oct 2004, Michael Richardson wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
>
>
> >>>>> "David" == David Dougall <[email protected]> writes:
> David> faster in many cases. As far as clients, we have lots of
> David> HPUX, linux, Irix, AIX and MAC. Unfortunately, I have no
> David> experience with solaris clients. I have had little to no
>
> Solaris clients are the most demanding of clients, protocol-wise.
>
> David> problems with any of them except HPUX client locking code is
> David> really flaky(although it was flaky even with HPUX as the
>
> Yes, HPUX never did get the locking code right.
> Were you using NFSv2 or v3? Did it vary by client?
>
> Did you do lots of locking?
> Did you have mailspool on NFS?
> Any big, long running jobs? (e.g. Verilog simulations)
>
> - --
> ] "Elmo went to the wrong fundraiser" - The Simpson | firewalls [
> ] Michael Richardson, Xelerance Corporation, Ottawa, ON |net architect[
> ] [email protected] http://www.sandelman.ottawa.on.ca/mcr/ |device driver[
> ] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.2 (GNU/Linux)
> Comment: Finger me for keys
>
> iQCVAwUBQXf5VIqHRg3pndX9AQFEIAQAk0aSi6wJn3CAh5Rc+FbjzF78lkd5mWe1
> IgNkdMEMa+Jle9HJIOYYvXwnbYapHWu80T4ydJ8Ihyfg+fYavGNituyDVoSp3CX5
> d5V/+cAbz0O4NaqhTQBO1L4LST4oKAEWnWsvte8MmFyycIQ9wwbHPHPiw/6/84Kw
> we0WeOIf1PA=
> =YF6E
> -----END PGP SIGNATURE-----
>
>
>


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

2004-10-21 22:32:04

by Michael Richardson

[permalink] [raw]
Subject: Re: Linux NFS server for heterogeneous production environment

-----BEGIN PGP SIGNED MESSAGE-----


>>>>> "David" == David Dougall <[email protected]> writes:
David> That was very ugly. We moved to mbox mailboxes on a local
David> drive, but we have since moved to maildir files over nfs. It
David> works without a hitch. We have tons of simulations that take
David> lots of time. Some program simulations run for months on end
David> over NFS. (Although some would argue that if it wasn't over
David> NFS it would take a lot less time :P)

So, it sounds like there have been definitely improvements in the past
3 years, since my previous company tried to do all of the above.
This is good news.

- --
] "Elmo went to the wrong fundraiser" - The Simpson | firewalls [
] Michael Richardson, Xelerance Corporation, Ottawa, ON |net architect[
] [email protected] http://www.sandelman.ottawa.on.ca/mcr/ |device driver[
] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Finger me for keys

iQCUAwUBQXg4nIqHRg3pndX9AQGGYwP3ct/0XgKDQpCdsKMox+Aobf7GUAg+Eo7K
JGagLq4zfgMwxSlO3gZUn6EaN0qwYjGeGB7rXivE9sBTfKZDIZkQJUdG0J/erMBY
dHuINl7mlDNmalkn1gftH8OlKrshn8UoaiwkhNfjP4/mjSxxvOofcgZlxN7Nih2L
hkQtPgbFSg==
=EE2N
-----END PGP SIGNATURE-----


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs