2006-08-24 03:34:29

by Jeff Garzik

[permalink] [raw]
Subject: Linux: Why software RAID?

Mark Perkel wrote:
> Running Linux on an AMD AM2 nVidia chip ser that supports Raid 0
> striping on the motherboard. Just wondering if hardware raid (SATA2) is
> going to be faster that software raid and why?


First, it sounds like you are confusing motherboard "RAID" with real
RAID. There's a FAQ for this sort of thing:

http://linux-ata.org/faq-sata-raid.html

In particular, your motherboard's Raid 0 striping (a) is not done in
hardware, and (b) has nothing to do with SATA2.

But anyway, to help answer the question of hardware vs. software RAID, I
wrote up a page:

http://linux.yyz.us/why-software-raid.html

Generally, you want software RAID unless your PCI bus (or more rarely,
your CPU) is getting saturated. With RAID-0, there is no duplication of
data, and so, PCI bus and CPU usage should be about the same for
hardware and software RAID.

Jeff



2006-08-24 05:20:58

by Chris Friesen

[permalink] [raw]
Subject: Re: Linux: Why software RAID?

Jeff Garzik wrote:

> But anyway, to help answer the question of hardware vs. software RAID, I
> wrote up a page:
>
> http://linux.yyz.us/why-software-raid.html

Just curious...with these guys
(http://www.bigfootnetworks.com/KillerOverview.aspx) putting linux on a
PCI NIC to allow them to bypass Windows' network stack, has anyone ever
considered doing "hardware" raid by using an embedded cpu running linux
software RAID, with battery-backed memory?

It would theoretically allow you to remain feature-compatible by
downloading new kernels to your RAID card.

Chris

2006-08-24 05:26:46

by H. Peter Anvin

[permalink] [raw]
Subject: Re: Linux: Why software RAID?

Chris Friesen wrote:
> Jeff Garzik wrote:
>
>> But anyway, to help answer the question of hardware vs. software RAID,
>> I wrote up a page:
>>
>> http://linux.yyz.us/why-software-raid.html
>
> Just curious...with these guys
> (http://www.bigfootnetworks.com/KillerOverview.aspx) putting linux on a
> PCI NIC to allow them to bypass Windows' network stack, has anyone ever
> considered doing "hardware" raid by using an embedded cpu running linux
> software RAID, with battery-backed memory?
>
> It would theoretically allow you to remain feature-compatible by
> downloading new kernels to your RAID card.
>

Yes. In fact, I have been told by several RAID chip vendors that their
customers are *strongly* demanding that their chips be able to run Linux
md (and still use whatever hardware offload features.)

So it's happening.

-hpa

2006-08-24 05:43:34

by Mattias Wadenstein

[permalink] [raw]
Subject: Re: Linux: Why software RAID?

On Wed, 23 Aug 2006, Chris Friesen wrote:

> Jeff Garzik wrote:
>
>> But anyway, to help answer the question of hardware vs. software RAID, I
>> wrote up a page:
>>
>> http://linux.yyz.us/why-software-raid.html
>
> Just curious...with these guys
> (http://www.bigfootnetworks.com/KillerOverview.aspx) putting linux on a PCI
> NIC to allow them to bypass Windows' network stack, has anyone ever
> considered doing "hardware" raid by using an embedded cpu running linux
> software RAID, with battery-backed memory?

I'd expect this to be the reason why md offload support to xor engines and
whatever turns up. It makes very little sense for a modern server/desktop
CPU, but for the embedded ones it does.

/Mattias Wadenstein

2006-08-24 12:44:27

by Adam Kropelin

[permalink] [raw]
Subject: Re: Linux: Why software RAID?

Jeff Garzik <[email protected]> wrote:
> But anyway, to help answer the question of hardware vs. software RAID, I
> wrote up a page:
>
> http://linux.yyz.us/why-software-raid.html
>
> Generally, you want software RAID unless your PCI bus (or more rarely,
> your CPU) is getting saturated. With RAID-0, there is no duplication of
> data, and so, PCI bus and CPU usage should be about the same for
> hardware and software RAID.

Hardware RAID can be (!= is) more tolerant of serious drive failures
where a single drive locks up the bus. A high-end hardware RAID card
may be designed with independent controllers so a single drive failure
cannot take other spindles down with it. The same can be accomplished
with sw RAID of course if the builder is careful to use multiple PCI
cards, etc. Sw RAID over your motherboard's onboard controllers leaves
you vulnerable.

--Adam

2006-08-24 13:01:20

by Alan

[permalink] [raw]
Subject: Re: Linux: Why software RAID?

Ar Iau, 2006-08-24 am 09:07 -0400, ysgrifennodd Adam Kropelin:
> Jeff Garzik <[email protected]> wrote:
> with sw RAID of course if the builder is careful to use multiple PCI
> cards, etc. Sw RAID over your motherboard's onboard controllers leaves
> you vulnerable.

Generally speaking the channels on onboard ATA are independant with any
vaguely modern card. And for newer systems well the motherboard tends to
be festooned with random SATA controllers, all separate!

2006-08-24 13:13:03

by Adam Kropelin

[permalink] [raw]
Subject: Re: Linux: Why software RAID?

On Thu, Aug 24, 2006 at 02:20:50PM +0100, Alan Cox wrote:
> Ar Iau, 2006-08-24 am 09:07 -0400, ysgrifennodd Adam Kropelin:
> > Jeff Garzik <[email protected]> wrote:
> > with sw RAID of course if the builder is careful to use multiple PCI
> > cards, etc. Sw RAID over your motherboard's onboard controllers leaves
> > you vulnerable.
>
> Generally speaking the channels on onboard ATA are independant with any
> vaguely modern card.

Ahh, I did not know that. Does this apply to master/slave connections on
the same PATA cable as well? I know zero about PATA, but I assumed from
the terminology that master and slave needed to cooperate rather closely.

> And for newer systems well the motherboard tends to
> be festooned with random SATA controllers, all separate!

And how. You can't swing a dead cat without hitting a half-dozen ATA
ports these days. And most of them are those infuriatingly insecure SATA
connectors that pop off when you look at them cross-eyed...

--Adam

2006-08-24 14:56:00

by Mark Lord

[permalink] [raw]
Subject: Re: Linux: Why software RAID?

Adam Kropelin wrote:
> On Thu, Aug 24, 2006 at 02:20:50PM +0100, Alan Cox wrote:
>> Generally speaking the channels on onboard ATA are independant with any
>> vaguely modern card.
>
> Ahh, I did not know that. Does this apply to master/slave connections on
> the same PATA cable as well?

No, it doesn't. Except for cards which use special cables,
such as the Pacific Digital ADMA cards (which can even run both
master and slave simultaneously on a cable, though not with
the current Linux drivers).

Cheers

2006-08-24 15:01:16

by Alan

[permalink] [raw]
Subject: Re: Linux: Why software RAID?

Ar Iau, 2006-08-24 am 07:31 -0700, ysgrifennodd Marc Perkel:
> So - the bottom line answer to my question is that unless you are
> running raid 5 and you have a high powered raid card with cache and
> battery backup that there is no significant speed increase to use
> hardware raid. For raid 0 there is no advantage.
>
If your raid is entirely on PCI plug in cards and you are doing RAID1
there is a speed up using hardware assisted raid because of the PCI bus
contention. If your controllers are PCI express, on internal high speed
busses (eg chipset controllers) or at least half of them are then
generally there is no win with hardware raid 0/1 and it is often slower.


2006-08-26 20:55:44

by Dan Williams

[permalink] [raw]
Subject: Re: Linux: Why software RAID?

On 8/23/06, H. Peter Anvin <[email protected]> wrote:
> Chris Friesen wrote:
> > Jeff Garzik wrote:
> >
> >> But anyway, to help answer the question of hardware vs. software RAID,
> >> I wrote up a page:
> >>
> >> http://linux.yyz.us/why-software-raid.html
> >
> > Just curious...with these guys
> > (http://www.bigfootnetworks.com/KillerOverview.aspx) putting linux on a
> > PCI NIC to allow them to bypass Windows' network stack, has anyone ever
> > considered doing "hardware" raid by using an embedded cpu running linux
> > software RAID, with battery-backed memory?
> >
> > It would theoretically allow you to remain feature-compatible by
> > downloading new kernels to your RAID card.
> >
>
> Yes. In fact, I have been told by several RAID chip vendors that their
> customers are *strongly* demanding that their chips be able to run Linux
> md (and still use whatever hardware offload features.)
>
> So it's happening.
Speaking of md with hardware offload features:

http://prdownloads.sourceforge.net/xscaleiop/ols_paper_2006.pdf?download

> -hpa

Dan

2006-09-04 17:26:27

by Bill Davidsen

[permalink] [raw]
Subject: Re: Linux: Why software RAID?

Alan Cox wrote:

>Ar Iau, 2006-08-24 am 07:31 -0700, ysgrifennodd Marc Perkel:
>
>
>>So - the bottom line answer to my question is that unless you are
>>running raid 5 and you have a high powered raid card with cache and
>>battery backup that there is no significant speed increase to use
>>hardware raid. For raid 0 there is no advantage.
>>
>>
>>
>If your raid is entirely on PCI plug in cards and you are doing RAID1
>there is a speed up using hardware assisted raid because of the PCI bus
>contention.
>

I would expect to see this with RAID5 as well, for the same reason...

--
bill davidsen <[email protected]>
CTO TMR Associates, Inc
Doing interesting things with small computers since 1979

2006-09-04 17:32:14

by Joel Jaeggli

[permalink] [raw]
Subject: Re: Linux: Why software RAID?



Bill Davidsen wrote:
> Alan Cox wrote:
>
>> Ar Iau, 2006-08-24 am 07:31 -0700, ysgrifennodd Marc Perkel:
>>
>>
>>> So - the bottom line answer to my question is that unless you are
>>> running raid 5 and you have a high powered raid card with cache and
>>> battery backup that there is no significant speed increase to use
>>> hardware raid. For raid 0 there is no advantage.
>>>
>>>
>> If your raid is entirely on PCI plug in cards and you are doing RAID1
>> there is a speed up using hardware assisted raid because of the PCI bus
>> contention.
>>
>
> I would expect to see this with RAID5 as well, for the same reason...

assuming you actually have lots of pci contention that might be a
consideration... if you're sitting on server class hardware with
multiple pci buses or using pci-express cards that won't be a
significant issue.

--
------------------------------------------------------------------------
Joel Jaeggli Unix Consulting [email protected]
GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2