Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756945AbYFAJpu (ORCPT ); Sun, 1 Jun 2008 05:45:50 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1752931AbYFAJpl (ORCPT ); Sun, 1 Jun 2008 05:45:41 -0400 Received: from lucidpixels.com ([75.144.35.66]:37108 "EHLO lucidpixels.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753144AbYFAJpk (ORCPT ); Sun, 1 Jun 2008 05:45:40 -0400 Date: Sun, 1 Jun 2008 05:45:39 -0400 (EDT) From: Justin Piszcz To: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, xfs@oss.sgi.com Subject: Limits of the 965 chipset & 3 PCI-e cards/southbridge? ~774MiB/s peak for read, ~650MiB/s peak for write? Message-ID: User-Agent: Alpine 1.10 (DEB 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2613 Lines: 56 I have 12 enterprise-class seagate 1TiB disks on a 965 desktop board and it appears I have hit the limit, if I were able to get the maximum speed of all drives, ~70MiB/avg * 12 = 840MiB/s but it seems to stop aound 774 MiB/s (currently running badblocks on all drives).. I am testing some drives for someone and was curious to see how far one can push the disks/backplane to their theoretical limit. dstat output: ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- usr sys idl wai hiq siq| read writ| recv send| in out | int csw 1 12 0 83 3 2| - 0 | 0 0 | 0 0 | 13k 23k 1 11 0 84 2 2| 774M 0 |2100B 7373B| 0 0 | 13k 23k 1 12 0 83 3 2| 774M 0 | 0 0 | 0 0 | 13k 23k 1 11 0 82 4 2| 774M 0 |2030B 5178B| 0 0 | 13k 23k 1 11 0 83 4 2| 774M 0 | 0 0 | 0 0 | 13k 23k 1 11 0 83 3 2| 774M 0 |2264B 6225B| 0 0 | 13k 23k vmstat 1 output: ~$ vmstat 1 procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 0 12 124 3841772 8 13012 0 0 12595 163880 379 352 0 31 37 32 2 12 124 3841772 8 12992 0 0 791744 8 12796 23033 1 18 0 82 0 12 124 3841772 8 12992 0 0 792192 0 12677 22918 1 15 0 84 0 12 124 3841772 8 12992 0 0 792960 0 12894 22929 1 15 0 84 When I was writing to all of the drives it was maxing out around ~650 MiB/s. I also have 2 raptors on a PCI card (as I ran out of PCI-e cards) and: When I read from 1 of the raptors (w/ dd/example shown below) the speed drops: 1 13 0 79 5 3| 764M 0 |2240B 7105B| 0 0 | 12k 21k 1 13 0 80 5 2| 764M 0 | 0 0 | 0 0 | 12k 21k 1 11 0 82 5 2| 764M 0 |2170B 5446B| 0 0 | 12k 21k 1 12 0 81 5 2| 762M 0 | 0 0 | 0 0 | 12k 21k Does/has anyone done this with server intel board/would greater speeds be achievable? Also, how does AMD fair in this regard? Has anyone run similar tests? For instance if you have 12 disks in your host you could: dd if=/dev/disk1 of=/dev/null bs=1M dd if=/dev/disk2 of=/dev/null bs=1M What rate(s) do you get? Justin. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/