Subject: Re: Higher than expected disk write(2) latency
From: Chris Mason <chris.mason@oracle.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Martin Sustrik <sustrik@fastmq.com>, Martin Lucina <mato@kotelna.sk>,
       linux-kernel@vger.kernel.org, linux-aio@kvack.org
In-Reply-To: <20080710011417.95532d51.akpm@linux-foundation.org>
References: <20080628121131.GA14181@nodbug.moloch.sk>
	 <20080709222701.8eab4924.akpm@linux-foundation.org>
	 <4875C45C.2010901@fastmq.com>
	 <20080710011417.95532d51.akpm@linux-foundation.org>
Content-Type: text/plain
Date: Thu, 10 Jul 2008 09:29:01 -0400
Message-Id: <1215696541.24425.214.camel@think.oraclecorp.com>
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1346
Lines: 31

On Thu, 2008-07-10 at 01:14 -0700, Andrew Morton wrote:
> On Thu, 10 Jul 2008 10:12:12 +0200 Martin Sustrik <sustrik@fastmq.com> wrote:
> 
> > Hi Andrew,
> > 
> > >> we're getting some rather high figures for write(2) latency when testing
> > >> synchronous writing to disk.  The test I'm running writes 2000 blocks of
> > >> contiguous data to a raw device, using O_DIRECT and various block sizes
> > >> down to a minimum of 512 bytes.  
> > >>
> > >> The disk is a Seagate ST380817AS SATA connected to an Intel ICH7
> > >> using ata_piix.  Write caching has been explicitly disabled on the
> > >> drive, and there is no other activity that should affect the test
> > >> results (all system filesystems are on a separate drive).  The system is
> > >> running Debian etch, with a 2.6.24 kernel.

Is NCQ enabled on the drive?  The basic way to fix this is to have
multiple requests in flight, which isn't going to happen on sata with
the cache off and with ncq off.

So, AIO + NCQ should be fast, as long as your application has multiple
requests in flight through aio (or multiple threads).

-chris


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/