DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type:content-transfer-encoding;
        b=oiW4LLRpDWwcfKf8dY4b3pKuxfJuvSCoSsgYtSjaZjm7XlS2HC1vNa5YpTjMWk98AF
         NTNNu0XGClWec/ynuANuDW8NG0e2ZIvwSJqd2Uy1HeYN2h8/D0pvNZzGUrwY1zqG3Y+g
         e/5FRlqO9lQ8+BqgsKkvgnJhVIhMe7JL4fnAM=
MIME-Version: 1.0
In-Reply-To: <40e92d5b0911111625sf692589u8678b03264c1b31d@mail.gmail.com>
References: <40e92d5b0911101608t4baf75bo83c5780f24761781@mail.gmail.com>
	 <4e5e476b0911110300nb25785eqf7998ad1fe3ed537@mail.gmail.com>
	 <40e92d5b0911111625sf692589u8678b03264c1b31d@mail.gmail.com>
Date: Thu, 12 Nov 2009 09:44:04 +0100
Message-ID: <4e5e476b0911120044j1c944b51me3b4d4a3653b7c5@mail.gmail.com>
Subject: Re: SATA NCQ - blessing for throughput, curse for latency?
From: Corrado Zoccolo <czoccolo@gmail.com>
To: =?UTF-8?Q?Przemys=C5=82aw_Pawe=C5=82czyk?= 
	<przemyslaw@pawelczyk.it>
Cc: lkml <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3438
Lines: 113

2009/11/12 Przemysław Pawełczyk <przemyslaw@pawelczyk.it>:
> 2009/11/11 Corrado Zoccolo <czoccolo@gmail.com>:
>> 2009/11/11 Przemysław Pawełczyk <przemyslaw@pawelczyk.it>:
>>> [snip] I wrote a script (that is pasted
>>> at the end), which using zcav from bonnie++ (and unbuffer from expect
>>> to work properly if output is redirected to a file) reads up to 64 GB
>>> of the disk, but after a first few gigabytes (depending on how many GB
>>> of memory you have) cats kernel sources with binary files up to the
>>> second level of directory depth. What is measured? The real time of
>>> the cat to /dev/null operation. Why not the kernel sources alone?
>>> Because if we build kernel, especially with -jX (where X > 1), then
>>> both inodes and contents of the corresponding files within these
>>> directories generally won't be consecutive. Time for the results with
>>> some background information. In my tests I have used 2.6.31.5 built
>>> with debuginfo (using debian's make-kpkg).
>>>
>>>[summarized the relevant numbers]
>>> *** NCQ turned on ***
>>> real 752.30
>>> user 0.20
>>> sys 1.83
>>>
>>>
>>> *** NCQ turned off *** (kernel booted with noncq parameter)
>>> real 62.30
>>> user 0.27
>>> sys 1.59
>>>
>>> [snip]
>>>
>>> Can anyone confirm analogous NCQ-dependant behavior with other
>>> disk/controller variants? Any suggestions for improvements other than
>>> turning off NCQ or switching to anticipatory (in typical cases cfq is
>>> better, so it's not the best option)?
>>
>> The NCQ latency problem is well known, and should be fixed in 2.6.32.
>> You can try building the latest rc.
>
> Thank you for this information. Could you provide fixing commit's sha?
> Just for the record.
It's a set of changes:
1d2235152dc745c6d94bedb550fea84cffdbf768 ..
61f0c1dcaaac71faabac6ef7c839b29f20204bea

>
> Bunch of numbers from my laptop, to show how it improved in my case:
>
> model name: Intel(R) Core(TM)2 Duo CPU     T7100  @ 1.80GHz
> mem total:  3087012 kB
>
> [    8.656966] scsi 0:0:0:0: Direct-Access     ATA      WDC
> WD3200BJKT-0 11.0 PQ: 0 ANSI: 5
> [    8.657771] sd 0:0:0:0: [sda] 625142448 512-byte logical blocks:
> (320 GB/298 GiB)
>
> 00:1f.2 SATA controller: Intel Corporation 82801HBM/HEM
> (ICH8M/ICH8M-E) SATA AHCI Controller (rev 03) (prog-if 01 [AHCI 1.0])
>        Kernel driver in use: ahci
>
> ** 2.6.31.5 NCQ **
>
> Did not end while was reading first 64 GB of the disk. Result:
> real 807.06
> user 0.14
> sys 2.06
>
> ** 2.6.32-rc6 NCQ **
>
> 0.00 82.33 12.437
> 1.00 82.47 12.417
> 2.00 82.51 12.411
> 3.00 82.31 12.441
> Concurrent read will start in 1 second...
> 4.00 57.30 17.870
> 5.00 41.49 24.680
> 6.00 38.76 26.421
> real 79.48
> user 0.24
> sys 1.43
> 7.00 53.64 19.092
> 8.00 82.51 12.411
>
> ** 2.6.32-rc6 noncq **
>
> 0.00 82.33 12.437
> 1.00 82.47 12.417
> 2.00 82.40 12.427
> 3.00 81.87 12.508
> Concurrent read will start in 1 second...
> 4.00 42.17 24.285
> 5.00 37.91 27.010
> real 62.94
> user 0.18
> sys 1.64
> 6.00 52.70 19.431
> 7.00 82.51 12.410
>
> With noncq second read is faster, but the difference is now acceptable I think.

Thanks for confirming.

Corrado
>
> Thanks.
>
> --
> Przemysław Pawełczyk
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/