Hello all. I've got a Promise SX8 SATA controller and the performance
seems a little off. I'm using a debian-built 2.6.9 kernel and the
read/write I/O seems to level off around 80-90 MB/s. When I use the same
hardware in Windows, sequential I/O goes up to 218 MB/s. (That's about
the limit of my motherboard's buses.) If I move some disks to the
on-board SATA controller, I can get the I/O to go higher but never as
high as in Windows.
Is this a problem with my expectations, my testing, or the driver?
Some Bonnie numbers for a raid 0 array of Seagate 7200.7's (2 GB file,
sequential, MB/s):
Disks/cntl Write Read
----- ----- ----
1 (sx8) 57 63
2 (sx8) 69 75
2 (ESB) 111 121
3 (sx8) 75 79
3 (1 sx8, 2 ESB) 141 125
4 (sx8) 89 80
4 (2 sx8, 2 ESB) 135 150
5 (sx8) 88 79
6 (sx8) 91 75
Some numbers from Sandra (ugh) in W2K Pro with the latest Promise
drivers and using software RAID 0 (sequential, MB/s):
Disks/cntl Write Read
----- ----- ----
1 (sx8) 56 62
2 (sx8) 106 122
3 (sx8) 151 176
4 (sx8) 178 209
5 (sx8) 180 218
6 (sx8) 179 218
6 (4 sx8, 2 ESB) 169 207
My full hardware:
Supermicro P4SCi motherboard (7210 chipset, 6300ESB SATA on-board)
3.0 GHz P4 Northwood w/ HT enabled
(2) 512MB DDR400 dual channel (non-ECC)
Promise SX8 SATA controller
(6) 200 GB Seagate 7200.7
(1) 40 GB Seagate Baraccuda IV (boot)
Adaptec 2940U
Hi,
Playing with a recently acquired Promise SX8 card, we've found
similar performance results to Matt's post to lkml a few months back
at http://marc.theaimsgroup.com/?l=linux-kernel&m=110175890323356&w=2
It appears that the driver is only submitting one command at a time
per port, which is at least one cause of the slowdowns. By raising
CARM_MAX_Q from 1 to 3 in drivers/block/sx8.c (it was 3 in an
earlier pre-merge incarnation of carmel.c), we're getting very
notable speed improvements, with no side effects just yet.
Knowing very little about what this change has actually done, I've a
few questions:
- Should this be considered dangerous?
- Why was it taken from 3 to 1?
- Is CARM_MAX_Q a number defined (or limited) by the hardware?
Thanks in advance,
Bernard.
--
Bernard Blackham <bernard at blackham dot com dot au>
Bernard Blackham wrote:
> Hi,
>
> Playing with a recently acquired Promise SX8 card, we've found
> similar performance results to Matt's post to lkml a few months back
> at http://marc.theaimsgroup.com/?l=linux-kernel&m=110175890323356&w=2
>
> It appears that the driver is only submitting one command at a time
> per port, which is at least one cause of the slowdowns. By raising
> CARM_MAX_Q from 1 to 3 in drivers/block/sx8.c (it was 3 in an
> earlier pre-merge incarnation of carmel.c), we're getting very
> notable speed improvements, with no side effects just yet.
>
> Knowing very little about what this change has actually done, I've a
> few questions:
>
> - Should this be considered dangerous?
> - Why was it taken from 3 to 1?
> - Is CARM_MAX_Q a number defined (or limited) by the hardware?
In multi-port stress tests, we couldn't get SX8 to function reliably
without locking up or corrupting data, with more than one outstanding
command.
Maybe a new firmware has solved this by now.
Jeff
Jeff Garzik wrote:
> Bernard Blackham wrote:
>
>> Hi,
>>
>> Playing with a recently acquired Promise SX8 card, we've found
>> similar performance results to Matt's post to lkml a few months back
>> at http://marc.theaimsgroup.com/?l=linux-kernel&m=110175890323356&w=2
>>
>> It appears that the driver is only submitting one command at a time
>> per port, which is at least one cause of the slowdowns. By raising
>> CARM_MAX_Q from 1 to 3 in drivers/block/sx8.c (it was 3 in an
>> earlier pre-merge incarnation of carmel.c), we're getting very
>> notable speed improvements, with no side effects just yet.
>>
>> Knowing very little about what this change has actually done, I've a
>> few questions:
>> - Should this be considered dangerous?
>> - Why was it taken from 3 to 1?
>> - Is CARM_MAX_Q a number defined (or limited) by the hardware?
>
>
> In multi-port stress tests, we couldn't get SX8 to function reliably
> without locking up or corrupting data, with more than one outstanding
> command.
>
> Maybe a new firmware has solved this by now.
>
> Jeff
>
>
>
Indeed there does seem to be new firmware out as of 2/23/05. I ran my
tests with the 9/10/04 firmware but I did not adjust the CARM_MAX_Q
value. Do either of you happen to know what firmware revisions you
tested under?
I've put the machine with the SX8 controller into production despite the
performance issues so I'm not going to be of any use for testing
revisions to the driver :(
Matt
Matthew Collins wrote:
> Jeff Garzik wrote:
>
>> Bernard Blackham wrote:
>>
>>> Hi,
>>>
>>> Playing with a recently acquired Promise SX8 card, we've found
>>> similar performance results to Matt's post to lkml a few months back
>>> at http://marc.theaimsgroup.com/?l=linux-kernel&m=110175890323356&w=2
>>>
>>> It appears that the driver is only submitting one command at a time
>>> per port, which is at least one cause of the slowdowns. By raising
>>> CARM_MAX_Q from 1 to 3 in drivers/block/sx8.c (it was 3 in an
>>> earlier pre-merge incarnation of carmel.c), we're getting very
>>> notable speed improvements, with no side effects just yet.
>>>
>>> Knowing very little about what this change has actually done, I've a
>>> few questions:
>>> - Should this be considered dangerous?
>>> - Why was it taken from 3 to 1?
>>> - Is CARM_MAX_Q a number defined (or limited) by the hardware?
>>
>>
>>
>> In multi-port stress tests, we couldn't get SX8 to function reliably
>> without locking up or corrupting data, with more than one outstanding
>> command.
>>
>> Maybe a new firmware has solved this by now.
>>
>> Jeff
>>
>>
>>
> Indeed there does seem to be new firmware out as of 2/23/05. I ran my
> tests with the 9/10/04 firmware but I did not adjust the CARM_MAX_Q
> value. Do either of you happen to know what firmware revisions you
> tested under?
>
> I've put the machine with the SX8 controller into production despite the
> performance issues so I'm not going to be of any use for testing
> revisions to the driver :(
The driver was developed on a pre-production board, so its entirely
possible Promise fixed this issue.
The driver should be solid for _at least_ CARM_MAX_Q==31, presuming that
the firmware doesn't choke.
Anybody willing to
(a) change CARM_MAX_Q
(b) use the latest firmware
(c) stress the card with fsx, badblocks, iozone, and other tools
on -multiple ports- simultaneously.
?
Jeff
On Thu, Mar 24, 2005 at 11:09:17PM -0500, Jeff Garzik wrote:
> >Indeed there does seem to be new firmware out as of 2/23/05. I ran my
> >tests with the 9/10/04 firmware but I did not adjust the CARM_MAX_Q
> >value. Do either of you happen to know what firmware revisions you
> >tested under?
BIOS boot messages didn't seem to indicate a firmware version, but
this card was only available in stock in early March, so its fairly
likely to have the latest firmware (2/23/05).
> The driver should be solid for _at least_ CARM_MAX_Q==31, presuming that
> the firmware doesn't choke.
We've run it here with CARM_MAX_Q = 32 with a custom stress testing
script and haven't had any incidents so far. CARM_MAX_Q = 33 goes
bad instantly.
> (c) stress the card with fsx, badblocks, iozone, and other tools
> on -multiple ports- simultaneously.
I'll give those a workout over the weekend and let you know the
results.
Thanks,
Bernard.
--
Bernard Blackham <bernard at blackham dot com dot au>
On Thu, Mar 24, 2005 at 11:09:17PM -0500, Jeff Garzik wrote:
> The driver was developed on a pre-production board, so its entirely
> possible Promise fixed this issue.
>
> The driver should be solid for _at least_ CARM_MAX_Q==31, presuming that
> the firmware doesn't choke.
Stressing this SX8 card broke with CARM_MAX_Q anything higher than
16. Tests included simultaneous iozones, bonnie++'s, fsx, badblocks,
and mkfs'ing, on 4 ports. Setting CARM_MAX_Q to 17 or above
resulted in errors such as:
Buffer I/O error on device sx80_0p1, logical block 108828
lost page write due to I/O error on sx80_0p1
end_request: I/O error, dev sx80_2, sector 389167
It's been thrashing away with various tests most of today with
CARM_MAX_Q = 16, and it hasn't missed a beat.
Thanks must go to Matt Johnston who actually did all the work of
giving the card and drives a hard time.
Bernard.
--
Bernard Blackham <bernard at blackham dot com dot au>