Hello,
Recently we turned on a RAID-backed NFS server that fell flat on its fa=
ce
as soon as it started to see even moderate activity (particularly IMAP
traffic from Courier-IMAP). Through help on the Courier mailing list, th=
e
problem appears to be very poor throughput to our hard drives (although
systems is not my strong point, so I hope there is not something else I'v=
e
overlooked).
Our choice to go with consumer-grade hardware seems to have bit us hard=
.=20
We are using the oft-maligned 3Ware 85xx series RAID5 controller and thre=
e
Hitachi Deskstar 7k250 (250GB) SATA drives. The NFS serves up the mailst=
ore
for a system that does about 60 mail messages/min, plus IMAP against the
mailstore (which is what brings the hurt), and some moderately (much more
than Joe Bob's homepage, but not Slashdot) trafficed web content. We wer=
e
able to buy some time by moving our (ext3) journal off of the same device=
,
fiddle with a couple NFS settings (nodiratime,noatime), but it is still
woefully slow.
Are we completely wrong to give up on this hardware? If anyone is
interested in giving advice in that realm, I am happy to provide more sta=
ts
(like the fact that the machine sits at near 100% iowait all day long).
But instead of sticking it out with that hardware, we are thinking abou=
t a
couple possibilities, but a look around Google just found too many "shopp=
ing"
sites, and not enough real information and comparisons. We are wanting
advice for proven performers: RAID controller cards and drives. We had b=
een
hoping not to have to pay the hefty costs that SCSI incurrs, but at this
point we are considering the possibility, so advice on both sides of the
fence is sought.
We are also quite curious about just running with software raid instead=
,
since there seems to be a growing movement that claims there is no reason=
it
cannot be as (or more) reliable and fast -- especially compared to the pi=
ece
of $@^!% that we apparently got. :)
Thoguhts and links highly appreciated!
=09
=09
__________________________________=20
Yahoo! Mail - PC Magazine Editors' Choice 2005=20
http://mail.yahoo.com
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
email builder wrote:
> Hello,
>
> Recently we turned on a RAID-backed NFS server that fell flat on its face
> as soon as it started to see even moderate activity (particularly IMAP
> traffic from Courier-IMAP). Through help on the Courier mailing list, the
> problem appears to be very poor throughput to our hard drives (although
> systems is not my strong point, so I hope there is not something else I've
> overlooked).
>
> Our choice to go with consumer-grade hardware seems to have bit us hard.
> We are using the oft-maligned 3Ware 85xx series RAID5 controller and three
> Hitachi Deskstar 7k250 (250GB) SATA drives. The NFS serves up the mailstore
> for a system that does about 60 mail messages/min, plus IMAP against the
> mailstore (which is what brings the hurt), and some moderately (much more
> than Joe Bob's homepage, but not Slashdot) trafficed web content. We were
> able to buy some time by moving our (ext3) journal off of the same device,
> fiddle with a couple NFS settings (nodiratime,noatime), but it is still
> woefully slow.
>
> Are we completely wrong to give up on this hardware? If anyone is
> interested in giving advice in that realm, I am happy to provide more stats
> (like the fact that the machine sits at near 100% iowait all day long).
Hi
We noticed similar issues which we never tracked down, even after help
from this list. There is a huge bug report at Redhat entitled:
"Extremely high iowait with 3Ware array and moderate disk activity"
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=121434
which may be of some help, but it never quite gets to the bottom
of things. 3ware cards/drivers seem to the common factor in most if
not all of the "me too's", but of course everyone with a 3ware problem
would find this bug report...
Cheers
Ian
--
Ian Thurlbeck http://www.stams.strath.ac.uk/
Statistics and Modelling Science, University of Strathclyde
Livingstone Tower, 26 Richmond Street, Glasgow, UK, G1 1XH
Tel: +44 (0)141 548 3667 Fax: +44 (0)141 552 2079
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
If you stay with RAID5, get something with a battery backed write cache.
That will make a huge difference in RAID5 performance.
Michael Haverkamp
[email protected] wrote on 10/27/2005 10:21:49 PM:
> Hello,
>
> Recently we turned on a RAID-backed NFS server that fell flat on its
face
> as soon as it started to see even moderate activity (particularly IMAP
> traffic from Courier-IMAP). Through help on the Courier mailing list,
the
> problem appears to be very poor throughput to our hard drives (although
> systems is not my strong point, so I hope there is not something else
I've
> overlooked).
>
> Our choice to go with consumer-grade hardware seems to have bit us
hard.
> We are using the oft-maligned 3Ware 85xx series RAID5 controller and
three
> Hitachi Deskstar 7k250 (250GB) SATA drives. The NFS serves up the
mailstore
> for a system that does about 60 mail messages/min, plus IMAP against the
> mailstore (which is what brings the hurt), and some moderately (much
more
> than Joe Bob's homepage, but not Slashdot) trafficed web content. We
were
> able to buy some time by moving our (ext3) journal off of the same
device,
> fiddle with a couple NFS settings (nodiratime,noatime), but it is still
> woefully slow.
>
> Are we completely wrong to give up on this hardware? If anyone is
> interested in giving advice in that realm, I am happy to provide more
stats
> (like the fact that the machine sits at near 100% iowait all day long).
>
> But instead of sticking it out with that hardware, we are thinking
about a
> couple possibilities, but a look around Google just found too many
"shopping"
> sites, and not enough real information and comparisons. We are wanting
> advice for proven performers: RAID controller cards and drives. We had
been
> hoping not to have to pay the hefty costs that SCSI incurrs, but at this
> point we are considering the possibility, so advice on both sides of the
> fence is sought.
>
> We are also quite curious about just running with software raid
instead,
> since there seems to be a growing movement that claims there is no
reason it
> cannot be as (or more) reliable and fast -- especially compared to the
piece
> of $@^!% that we apparently got. :)
>
> Thoguhts and links highly appreciated!
>
>
>
>
>
>
> __________________________________
> Yahoo! Mail - PC Magazine Editors' Choice 2005
> http://mail.yahoo.com
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by the JBoss Inc.
> Get Certified Today * Register for a JBoss Training Course
> Free Certification Exam for All Training Attendees Through End of 2005
> Visit http://www.jboss.com/services/certification for more information
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Hi List,
I was having some performance problems too, a while back. I don't know that
performance is now as good as it could possibly be, but it is substantially
better than it was. I'm not sure the problems I faced are related to those
described below, but I'll share them anyway in the hope that someone finds
them useful.
I have two NFS servers which are basically clones of each other, from a
hardware perspective. One was set up with Red Hat 8.0, and the other was
running Red Hat Enterprise 3. That becomes important later. Both have Intel
SE7500CW2 motherboards and 3Ware 7506 RAID controllers.
When I set up the first system, I noticed that I was having all sorts of
trouble with the gigabit card. I'll just paste in some of my notes:
# This next line comes to us from Intel. The SE7500CW2 motherboard which we
# are using has a bit of a problem, in combination with its Intel
# Gigabit card. When using any version of the e1000 driver newer than 4.3.15,
# we would get all sorts of messages in the syslog saying, "NETDEV WATCHDOG:
# eth0: transmit timed out". The card would then go offline for a number of
# seconds (10-15) while it reset itself. This happened quite frequently and
# was highly undesirable behavior in an NFS server. After much conferring with
# Intel ([email protected]), it was determined that there was a cache
# coherency issue with the ICH (Southbridge) device on our SE7500CW2, and that
# setting bit 13 in the 0x50 register of the ICH would alter its behavior with
# regard to prefetch flushing of the cache. Notes from Intel indicated that
# this device would probably be at Bus 0, Device 0x1E, Func 0, Vendor ID 8086,
# Device ID 244E. This was confirmed with "lspci" and "pcitweak -l". The
# original value in register 0x50 was 0x00001402. A bitwise OR with 0x00002000
# resulted in the new value, 0x00003402. This fix does not persist across
# reboots, which is why it had to be entered into this file. Special thanks
# goes to Colleen of Intel Support, who put in a lot of effort to help me get
# this fixed.
The other problem we noticed is that the server with RH8 had pretty decent
NFS performance, where the RHEL3 system seemed very slow. After a lot of
conferring with Red Hat on this, I was told that it was because of the change
from async to sync as the default for exports and that I should specify async
when exporting. I asked them about the safety of this (FAQ: B6), but don't
recall getting much by way of response; however, since performance *was*
better with async and since the complaints about performance were very loud,
I had little choice but to stick with that.
The other thing I've done to (hopefully) improve performance was to increase
the number of NFS threads being spawned, and give them each a bunch of memory
([rw]mem_{default,max}). I figure these boxes are NFS servers and they've
got the memory to spare, so I may as well dedicate that memory to what
they're supposed to be doing. I can't speak to the "Extremely high iowait
with 3Ware array and moderate disk activity", but if anyone hears anything
about that, I'd certainly be interested in hearing as well.
James
As always, standard disclaimers apply. I don't speak for my company, what
works for me may not work for you, etc.
On Fri, 28 Oct 2005, Ian Thurlbeck wrote:
> email builder wrote:
>> Hello,
>>
>> Recently we turned on a RAID-backed NFS server that fell flat on its face
>> as soon as it started to see even moderate activity (particularly IMAP
>> traffic from Courier-IMAP). Through help on the Courier mailing list, the
>> problem appears to be very poor throughput to our hard drives (although
>> systems is not my strong point, so I hope there is not something else I've
>> overlooked).
>>
>> Our choice to go with consumer-grade hardware seems to have bit us hard.
>> We are using the oft-maligned 3Ware 85xx series RAID5 controller and three
>> Hitachi Deskstar 7k250 (250GB) SATA drives. The NFS serves up the
>> mailstore
>> for a system that does about 60 mail messages/min, plus IMAP against the
>> mailstore (which is what brings the hurt), and some moderately (much more
>> than Joe Bob's homepage, but not Slashdot) trafficed web content. We were
>> able to buy some time by moving our (ext3) journal off of the same device,
>> fiddle with a couple NFS settings (nodiratime,noatime), but it is still
>> woefully slow.
>>
>> Are we completely wrong to give up on this hardware? If anyone is
>> interested in giving advice in that realm, I am happy to provide more stats
>> (like the fact that the machine sits at near 100% iowait all day long).
>
>
> Hi
>
> We noticed similar issues which we never tracked down, even after help from
> this list. There is a huge bug report at Redhat entitled:
>
> "Extremely high iowait with 3Ware array and moderate disk activity"
>
> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=121434
>
> which may be of some help, but it never quite gets to the bottom
> of things. 3ware cards/drivers seem to the common factor in most if
> not all of the "me too's", but of course everyone with a 3ware problem would
> find this bug report...
>
> Cheers
>
> Ian
>
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
> were
> > able to buy some time by moving our (ext3) journal off of the same
> device,
> > fiddle with a couple NFS settings (nodiratime,noatime), but it is sti=
ll
> > woefully slow.
>=20
> mounting the ext3 partition with 'data=3Djournal' can help NFS service,
> especially with the journal on a separate device.
OK, we must have missed this one in the noise. I just added it to the mo=
unt
where all the NFS server's exports are (ext3 defaults,data=3Djournal). E=
ven
rebooted, but don't see evidence that it took (cat /proc/mounts just show=
s
ext3 rw but nothing about the journaling mode, should it?). Where can I
verify what mode it is in?
The other thing that came as a strong recommendation to us was to put
"no_acl" on all of our NFS exports, but on our Fedora system (with up to =
date
nfs-utils), "no_acl" is treated as an unrecognized keyword. Is there any
support for no_acl under redhat at all???
=09
=09
__________________________________=20
Yahoo! Mail - PC Magazine Editors' Choice 2005=20
http://mail.yahoo.com
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
> We noticed similar issues which we never tracked down, even after help=20
> from this list. There is a huge bug report at Redhat entitled:
>=20
> "Extremely high iowait with 3Ware array and moderate disk activity"
>=20
> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=3D121434
>=20
> which may be of some help, but it never quite gets to the bottom
> of things. 3ware cards/drivers seem to the common factor in most if
Yes, we've combed this thread, much to our shagrin, none of the tips work=
ed
for us. I suppose I should have said that one of our other possibilities=
is
to go with FreeBSD to see if this issue is the source of the problem. Wh=
at
can you say after reading this thread that's been open for WELL over a ye=
ar?=20
RedHat loses a lot of points on this one.
> not all of the "me too's", but of course everyone with a 3ware problem=20
> would find this bug report...
=09
=09
__________________________________=20
Yahoo! Mail - PC Magazine Editors' Choice 2005=20
http://mail.yahoo.com
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
> If you stay with RAID5, get something with a battery backed write cache=
.=20
> That will make a huge difference in RAID5 performance.
Really? I was under the impression that this was just a safety precautio=
n or
similar... it is called a "backup" after all. :) How exactly does it
increase performance? =20
(And, yes, I have seen this mentioned in many places, so this would be a
criteria for our next purchase. :) )
=09
=09
__________________________________=20
Yahoo! Mail - PC Magazine Editors' Choice 2005=20
http://mail.yahoo.com
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
> I have two NFS servers which are basically clones of each other, from a=
=20
> hardware perspective. One was set up with Red Hat 8.0, and the other w=
as=20
> running Red Hat Enterprise 3. That becomes important later. Both have
> Intel=20
> SE7500CW2 motherboards and 3Ware 7506 RAID controllers.
>=20
> When I set up the first system, I noticed that I was having all sorts o=
f=20
> trouble with the gigabit card. I'll just paste in some of my notes:
<snip> we don't seem to have any actual eth problems
=20
> The other problem we noticed is that the server with RH8 had pretty dec=
ent=20
> NFS performance, where the RHEL3 system seemed very slow. After a lot =
of=20
> conferring with Red Hat on this, I was told that it was because of the
> change=20
> from async to sync as the default for exports and that I should specify
> async=20
> when exporting. I asked them about the safety of this (FAQ: B6), but d=
on't
>=20
> recall getting much by way of response; however, since performance *was=
*=20
> better with async and since the complaints about performance were very
> loud,=20
> I had little choice but to stick with that.
Ouch. I have spent many hours reading about this, and general consensus
seems to say this is not a good idea. This to me (along with their appar=
ent
inability to solve the issues with the 3Ware card) indicates that RedHat =
is a
bad choice for NFS servers.... I mean, if they just say you should do
something dangerous because it is the only way to get better performance =
but
then not back it up or have any kind of real response to the fact that th=
is
should not be the appropriate thing to do.... well, you get the picture.=
=20
Maybe we'll try FreeBSD before we change anything else or buy new hardwar=
e.
> The other thing I've done to (hopefully) improve performance was to
> increase=20
> the number of NFS threads being spawned, and give them each a bunch of
Yeah, we bumped ours to 128 just to be safe (beside the severe iowait, ou=
r
machine is lightly used). The default 8 were being absolutely swamped.
> memory=20
> ([rw]mem_{default,max}). I figure these boxes are NFS servers and they=
've=20
Where do you set their memory and what are people's recommended changes f=
or
getting more "umph" out of NFS?
Thanks a ton for sharing!
> got the memory to spare, so I may as well dedicate that memory to what=20
> they're supposed to be doing. I can't speak to the "Extremely high iow=
ait=20
> with 3Ware array and moderate disk activity", but if anyone hears anyth=
ing=20
> about that, I'd certainly be interested in hearing as well.
=09
=09
__________________________________=20
Yahoo! Mail - PC Magazine Editors' Choice 2005=20
http://mail.yahoo.com
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
[email protected] wrote on 10/28/2005 05:22:38 PM:
>
> > If you stay with RAID5, get something with a battery backed write
cache.
> > That will make a huge difference in RAID5 performance.
>
> Really? I was under the impression that this was just a safety
precaution or
> similar... it is called a "backup" after all. :) How exactly does it
> increase performance?
Without one, writes must be flushed to disk before the RAID controller can
report them as complete. This includes even trivial things like updating
file access times. With one, writes only need to get into the
controller's write cache before they are reported as complete. This
allows the controller to schedule the actual write to disk at an optimal
time.
A good way to show this is to untar a tar file full of small files (e.g.
kernel source). On RAID5 without write cache, this takes at least 10
times longer than even a single ATA disk.
>
> (And, yes, I have seen this mentioned in many places, so this would be a
> criteria for our next purchase. :) )
>
>
>
>
> __________________________________
> Yahoo! Mail - PC Magazine Editors' Choice 2005
> http://mail.yahoo.com
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by the JBoss Inc.
> Get Certified Today * Register for a JBoss Training Course
> Free Certification Exam for All Training Attendees Through End of 2005
> Visit http://www.jboss.com/services/certification for more information
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Most, if not all, of my notes pertaining to large storage are here:
http://dcs.nac.uci.edu/~strombrg/tech-tidbits-by-category.html#Large%20Storage
I think commodity hardware is generally a good way to go when you can,
but we also had bad experiences with some 3Ware RAID cards, though in
our case, it wasn't clear if the problem was best attributed to the
Maxtor disks, the 3Ware RAID cards, the Promicrosystems PC's, or the GFS
and Lustre we tried overtop of this hardware.
We eventually replaced all this with a Sun 3511 with a Sun V440 out
front of it as a NAS head, with a QFS filesystem to piece together the
slices (no SVM or other extra tack-them-together layer required, since
QFS does that -and- the filesystem). It's served out via a Solaris NFS
server to a handful of AIX NFS clients, and it works well. But that's
because we needed 15T of storage - had we required only 2T, we would've
had many more choices.
You -can- get good performance out of commodity stuff. As a simple
thought-experiment, consider what would happen if you had a huge number
of small disks accessed via distinct 100BaseT links, with your data
being striped across them all. Because of the huge amount of striping,
your aggregate throughput should be quite good despite the 100BaseT.
I believe Hitachi bought the Deskstar line of disks from IBM, shortly
after IBM had to replace tons of Deskstars being they were dying left
and right. The problems may or may not have been thoroughly worked out
since then.
Sometimes under linux and other OS's, a single process that's pound a
storage resource can pretty well mess up the performance for the other
processes requiring access to that resource. Sometimes deprioritising
that I/O hungry process can help the other processes on the system A
LOT:
http://dcs.nac.uci.edu/~strombrg/Keeping-backups-from-pounding-a-system-as-hard.html
HTH!
On Thu, 2005-10-27 at 20:21 -0700, email builder wrote:
> Hello,
>
> Recently we turned on a RAID-backed NFS server that fell flat on its face
> as soon as it started to see even moderate activity (particularly IMAP
> traffic from Courier-IMAP). Through help on the Courier mailing list, the
> problem appears to be very poor throughput to our hard drives (although
> systems is not my strong point, so I hope there is not something else I've
> overlooked).
>
> Our choice to go with consumer-grade hardware seems to have bit us hard.
> We are using the oft-maligned 3Ware 85xx series RAID5 controller and three
> Hitachi Deskstar 7k250 (250GB) SATA drives. The NFS serves up the mailstore
> for a system that does about 60 mail messages/min, plus IMAP against the
> mailstore (which is what brings the hurt), and some moderately (much more
> than Joe Bob's homepage, but not Slashdot) trafficed web content. We were
> able to buy some time by moving our (ext3) journal off of the same device,
> fiddle with a couple NFS settings (nodiratime,noatime), but it is still
> woefully slow.
>
> Are we completely wrong to give up on this hardware? If anyone is
> interested in giving advice in that realm, I am happy to provide more stats
> (like the fact that the machine sits at near 100% iowait all day long).
>
> But instead of sticking it out with that hardware, we are thinking about a
> couple possibilities, but a look around Google just found too many "shopping"
> sites, and not enough real information and comparisons. We are wanting
> advice for proven performers: RAID controller cards and drives. We had been
> hoping not to have to pay the hefty costs that SCSI incurrs, but at this
> point we are considering the possibility, so advice on both sides of the
> fence is sought.
>
> We are also quite curious about just running with software raid instead,
> since there seems to be a growing movement that claims there is no reason it
> cannot be as (or more) reliable and fast -- especially compared to the piece
> of $@^!% that we apparently got. :)
>
> Thoguhts and links highly appreciated!
>
>
>
>
>
>
> __________________________________
> Yahoo! Mail - PC Magazine Editors' Choice 2005
> http://mail.yahoo.com
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by the JBoss Inc.
> Get Certified Today * Register for a JBoss Training Course
> Free Certification Exam for All Training Attendees Through End of 2005
> Visit http://www.jboss.com/services/certification for more information
> _______________________________________________
> NFS maillist - [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfs
>
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On Thursday October 27, [email protected] wrote:
> Hello,
>
> Recently we turned on a RAID-backed NFS server that fell flat on its face
> as soon as it started to see even moderate activity (particularly IMAP
> traffic from Courier-IMAP). Through help on the Courier mailing list, the
> problem appears to be very poor throughput to our hard drives (although
> systems is not my strong point, so I hope there is not something else I've
> overlooked).
>
> Our choice to go with consumer-grade hardware seems to have bit us hard.
> We are using the oft-maligned 3Ware 85xx series RAID5 controller and three
> Hitachi Deskstar 7k250 (250GB) SATA drives. The NFS serves up the mailstore
> for a system that does about 60 mail messages/min, plus IMAP against the
> mailstore (which is what brings the hurt), and some moderately (much more
> than Joe Bob's homepage, but not Slashdot) trafficed web content. We were
> able to buy some time by moving our (ext3) journal off of the same device,
> fiddle with a couple NFS settings (nodiratime,noatime), but it is still
> woefully slow.
mounting the ext3 partition with 'data=journal' can help NFS service,
especially with the journal on a separate device.
>
> We are also quite curious about just running with software raid instead,
> since there seems to be a growing movement that claims there is no reason it
> cannot be as (or more) reliable and fast -- especially compared to the piece
> of $@^!% that we apparently got. :)
Certainly worth a try. It may not be as fast as some high-end raid
cards, it is unlikely to be slower than what you have.
NeilBrown
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
> > > If you stay with RAID5, get something with a battery backed write=20
> cache.=20
> > > That will make a huge difference in RAID5 performance.
> >=20
> > Really? I was under the impression that this was just a safety=20
> precaution or
> > similar... it is called a "backup" after all. :) How exactly does i=
t
> > increase performance?=20
>=20
> Without one, writes must be flushed to disk before the RAID controller =
can=20
> report them as complete. This includes even trivial things like updati=
ng=20
> file access times. With one, writes only need to get into the=20
> controller's write cache before they are reported as complete. This=20
> allows the controller to schedule the actual write to disk at an optima=
l=20
> time.
Thank you sincerely for this excellent explanation. We are currently loo=
king
at the new 3Ware SATA II card, the 9550SX:
http://3ware.com/products/serial_ata2-9000.asp
What is strange about these is that the battery backup is sold separately=
.=20
Does anyone have experience with the BBUs (battery backup units) on the 3=
Ware
9000 series cards?
- Do they really provide the bang for the extra $100?
- What kind of configuration/installation mess are the BBUs?
> A good way to show this is to untar a tar file full of small files (e.g=
.=20
> kernel source). On RAID5 without write cache, this takes at least 10=20
> times longer than even a single ATA disk.
>=20
> >=20
> > (And, yes, I have seen this mentioned in many places, so this would b=
e a
> > criteria for our next purchase. :) )
=09
__________________________________=20
Yahoo! FareChase: Search multiple travel sites in one click.
http://farechase.yahoo.com
-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. Downl=
oad
it for free - -and be entered to win a 42" plasma tv or your very own
Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
On 10/31/05, Dan Stromberg <[email protected]> wrote:
>
> Most, if not all, of my notes pertaining to large storage are here:
>
> http://dcs.nac.uci.edu/~strombrg/tech-tidbits-by-category.html#Large%20St=
orage
>
> I think commodity hardware is generally a good way to go when you can,
> but we also had bad experiences with some 3Ware RAID cards, though in
> our case, it wasn't clear if the problem was best attributed to the
> Maxtor disks, the 3Ware RAID cards, the Promicrosystems PC's, or the GFS
> and Lustre we tried overtop of this hardware.
I agree that commodity hardware is a good way to go if possible. I've
had great experience with 3ware cards on Redhat systems if they are
running the latest firmware/driver - horrible experiences otherwise.=20
Also had horrible luck with maxtor drives on 3ware controllers and
maxtor drives in generally - you should think twice about using them.
We see 12MB/sec from multiple clients all day long on RH systems with
this configuration - this is just ata raid on 75xx controllers,
nothing fancy.
Obviously the following options are huge for performance on clients as
well, tcp,rsize=3D8192,wsize=3D8192
The sync option also has a rather large impact on client write
performance and I have left it async for that reason.
-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. Download
it for free - -and be entered to win a 42" plasma tv or your very own
Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs