Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755685Ab2HOR6E (ORCPT ); Wed, 15 Aug 2012 13:58:04 -0400 Received: from mail-we0-f174.google.com ([74.125.82.174]:53571 "EHLO mail-we0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753223Ab2HOR6D (ORCPT ); Wed, 15 Aug 2012 13:58:03 -0400 MIME-Version: 1.0 In-Reply-To: <502B8D1F.7030706@anonymous.org.uk> References: <502B8D1F.7030706@anonymous.org.uk> From: Andy Lutomirski Date: Wed, 15 Aug 2012 10:57:40 -0700 Message-ID: Subject: Re: O_DIRECT to md raid 6 is slow To: John Robinson Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org Content-Type: text/plain; charset=ISO-8859-1 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1112 Lines: 39 On Wed, Aug 15, 2012 at 4:50 AM, John Robinson wrote: > On 15/08/2012 01:49, Andy Lutomirski wrote: >> >> If I do: >> # dd if=/dev/zero of=/dev/md0p1 bs=8M > > [...] > >> It looks like md isn't recognizing that I'm writing whole stripes when >> I'm in O_DIRECT mode. > > > I see your md device is partitioned. Is the partition itself stripe-aligned? Crud. md0 : active raid6 sdg1[5] sdf1[4] sde1[3] sdd1[2] sdc1[1] sdb1[0] 11720536064 blocks super 1.2 level 6, 512k chunk, algorithm 2 [6/6] [UUUUUU] IIUC this means that I/O should be aligned on 2MB boundaries (512k chunk * 4 non-parity disks). gdisk put my partition on a 2048 sector (i.e. 1MB) boundary. Sadly, /sys/block/md0/md0p1/alignment_offset reports 0 (instead of 1MB). Fixing this has no effect, though. --Andy -- Andy Lutomirski AMA Capital Management, LLC -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/