Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030958Ab0B1AJ5 (ORCPT ); Sat, 27 Feb 2010 19:09:57 -0500 Received: from mail.tmr.com ([64.65.253.246]:47377 "EHLO partygirl.tmr.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030928Ab0B1AJz (ORCPT ); Sat, 27 Feb 2010 19:09:55 -0500 Message-ID: <4B89B44A.70005@tmr.com> Date: Sat, 27 Feb 2010 19:09:46 -0500 From: Bill Davidsen Organization: TMR Associates Inc, Schenectady NY User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.21) Gecko/20090507 Fedora/1.1.16-1.fc9 NOT Firefox/3.0.11 SeaMonkey/1.1.16 MIME-Version: 1.0 To: Justin Piszcz CC: Neil Brown , linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, linux-ext4@vger.kernel.org, Alan Piszcz Subject: Re: mdadm software raid + ext4, capped at ~350MiB/s limitation/bug? References: <20100228080100.092c24c2@notabene.brown> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2273 Lines: 69 Justin Piszcz wrote: > > > On Sun, 28 Feb 2010, Neil Brown wrote: > >> On Sat, 27 Feb 2010 08:47:48 -0500 (EST) >> Justin Piszcz wrote: >> >>> Hello, >>> >>> I have two separate systems and with ext4 I cannot get speeds >>> greater than >>> ~350MiB/s when using ext4 as the filesystem on top of a raid5 or raid0. >>> It appears to be a bug with ext4 (or its just that ext4 is slower >>> for this >>> test)? >>> >>> Each system runs 2.6.33 x86_64. >> >> Could be related to the recent implementation of IO barriers in md. >> Can you try mounting your filesystem with >> -o barrier=0 >> >> and see how that changes the result. >> >> NeilBrown > > Hi Neil, > > Thanks for the suggestion, it has been used here: > http://lkml.org/lkml/2010/2/27/66 > > Looks like an EXT4 issue as XFS does ~600MiB/s..? > > Its strange though, on a single hard disk, I get approximately the > same speed for XFS and EXT4, but when it comes to scaling across > multiple disks, > in RAID-0 or RAID-5 (tested), there is a performance problem as it hits a > performance problem at ~350MiB/s. I tried multiple chunk sizes but > nothing > seemed to made a difference (whether 64KiB or 1024KiB), XFS performs at > 500-600MiB/s no matter what and EXT4 does not exceed ~350MiB/s. > > Is there anyone on any of the lists that gets > 350MiB/s on a mdadm/sw > raid > with EXT4? > > A single raw disk, no partitions: > p63:~# dd if=/dev/zero of=/dev/sdm bs=1M count=10240 > 10240+0 records in > 10240+0 records out > 10737418240 bytes (11 GB) copied, 92.4249 s, 116 MB/s I hate to say it, but I don't think this measures anything useful. When I was doing similar things I got great variabilty in my results until I learned about the fdatasync option so you measure the actual speed to the destination and not the disk cache. After that my results were far slower and reproducible. -- Bill Davidsen "We can't solve today's problems by using the same thinking we used in creating them." - Einstein -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/