Hi, I was wondering how IO schedulers such as as-iosched, deadline and
cfq behave on SSD
(that have virtually no seek time), from a theoretical point of view.
How do they affect
performance on these devices?
I heard that the noop scheduler is often chosen by owners of EeePcs
(with a SSD unit).
They report superior performance by using this (quite simple) scheduler.
Are there any scientific benchmarks around?
--
Lorenzo
On Fri, Jan 30 2009, Lorenzo Allegrucci wrote:
> Hi, I was wondering how IO schedulers such as as-iosched, deadline and
> cfq behave on SSD
> (that have virtually no seek time), from a theoretical point of view.
> How do they affect
> performance on these devices?
> I heard that the noop scheduler is often chosen by owners of EeePcs
> (with a SSD unit).
> They report superior performance by using this (quite simple) scheduler.
> Are there any scientific benchmarks around?
Just recently the io schedulers started checking for SSD devices, so
today it should not matter performance wise (throughput) whether you use
CFQ or NOOP on eg the eeepc. The io scheduler is still quite important
for providing fair access to the device, especially on the cheaper end
of the SSD segment.
--
Jens Axboe
Jens Axboe wrote:
> On Fri, Jan 30 2009, Lorenzo Allegrucci wrote:
>
>> Hi, I was wondering how IO schedulers such as as-iosched, deadline and
>> cfq behave on SSD
>> (that have virtually no seek time), from a theoretical point of view.
>> How do they affect
>> performance on these devices?
>> I heard that the noop scheduler is often chosen by owners of EeePcs
>> (with a SSD unit).
>> They report superior performance by using this (quite simple) scheduler.
>> Are there any scientific benchmarks around?
>>
>
> Just recently the io schedulers started checking for SSD devices, so
> today it should not matter performance wise (throughput) whether you use
> CFQ or NOOP on eg the eeepc.
Although not all SSDs identify themselves, including the one on my
eeepc. So to get the benefit you have to tell the kernel manually. The
ability to do this was merged into mainline yesterday. (/me resolves to
test it).
http://git.kernel.org/gitweb.cgi?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=1308835ffffe6d61ad1f48c5c381c9cc47f683ec
Anecdotally it was worth switching from CFQ to NOOP. CFQ caused
seconds-long hangs (with the SSD light solidly on); with NOOP this
happens far less often. I don't know or care if it halved throughput,
but I can't bear to use a system that hangs for seconds on end :-).
> The io scheduler is still quite important
> for providing fair access to the device, especially on the cheaper end
> of the SSD segment.
>
Hi,
I found that elevator=deadline performs much better than noop for
writes, and almost as well for reads, and started to wonder how a
combination of noop (for read) + deadline (for write) would work in
practice (since deadline still doesn't support the automatic ssd
detection).
I developed a simple hybrid scheduler (see attached), that implements
this idea, and it saved 1 s of my boot time compared to all other
schedulers (cfq, deadline and noop, each loses on some part of the
workload), that is a mixed read write workload, with writes that go
mainly to a very slow device (SDHC card, where I mount /var).
This proof of concept still doesn't support priorities, but I'm
willing to add them at least for read, in which the latency for
sync-reads could be improved.
Corrado
On Sat, Feb 7, 2009 at 6:58 PM, Jan Knutar <[email protected]> wrote:
> On Wednesday 04 February 2009, J.A. Magallón wrote:
>
>> Perhaps the reason is that, as the SSD is not so good, it behaves
>> more like a rotational drive ;).
>
> Do any other SSDs except Intel's exist that DON'T behave more like a
> rotational drive? I am guessing using something like LogFS would give
> the biggest boost on cheap SSDs and all memory cards.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
--
__________________________________________________________________________
dott. Corrado Zoccolo mailto:[email protected]
PhD - Department of Computer Science - University of Pisa, Italy
--------------------------------------------------------------------------
The self-confidence of a warrior is not the self-confidence of the average
man. The average man seeks certainty in the eyes of the onlooker and calls
that self-confidence. The warrior seeks impeccability in his own eyes and
calls that humbleness.
Tales of Power - C. Castaneda
On 08.04.2009, Corrado Zoccolo wrote:
> I found that elevator=deadline performs much better than noop for
> writes, and almost as well for reads
[....]
The DL elevator has slightly more throughput than cfq and anticipatory,
but is almost unusuable under load.
Running Theodore Ts'os "fsync-tester" while doing Linus' torture test
"while : ; do time sh -c "dd if=/dev/zero of=bigfile bs=8M count=256 ; sync; rm bigfile"; done"
shows it clearly:
mount: /dev/sda4 on /home type xfs (rw,noatime,logbsize=256k,logbufs=2,nobarrier)
Kernel 2.6.29.1 (vanilla)
with cfq:
htd@liesel:~/!> ./fsync-tester
fsync time: 0.7640
fsync time: 0.6166
fsync time: 1.2830
fsync time: 0.4273
fsync time: 1.1693
fsync time: 1.7466
fsync time: 1.2477
fsync time: 1.9411
fsync time: 1.9636
fsync time: 1.9065
fsync time: 1.1561
fsync time: 1.8267
fsync time: 0.2431
fsync time: 0.2898
fsync time: 0.2394
fsync time: 0.4309
fsync time: 1.5699
fsync time: 0.3742
fsync time: 1.3194
fsync time: 1.9442
fsync time: 1.0899
fsync time: 1.9443
fsync time: 1.0062
with dl:
fsync time: 10.5853
fsync time: 10.3339
fsync time: 5.3374
fsync time: 6.5707
fsync time: 10.6095
fsync time: 4.1154
fsync time: 4.9604
fsync time: 10.5325
fsync time: 10.4543
fsync time: 10.4970
fsync time: 10.5570
fsync time: 5.2717
fsync time: 10.5619
fsync time: 5.3058
fsync time: 3.1019
fsync time: 5.1504
fsync time: 5.7564
fsync time: 10.5998
fsync time: 4.0895
Regards, Heinz.
Well, that's not an usual workload for netbooks, where most SSDs are
currently deployed.
For usual workloads, that are mostly read, cfq has lower performance
both in throughput and in latency than deadline.
Corrado
2009/4/8 Heinz Diehl <[email protected]>:
> On 08.04.2009, Corrado Zoccolo wrote:
>
>> I found that elevator=deadline performs much better than noop for
>> writes, and almost as well for reads
> [....]
>
> The DL elevator has slightly more throughput than cfq and anticipatory,
> but is almost unusuable under load.
>
> Running Theodore Ts'os "fsync-tester" while doing Linus' torture test
> "while : ; do time sh -c "dd if=/dev/zero of=bigfile bs=8M count=256 ; sync; rm bigfile"; done"
> shows it clearly:
>
> mount: /dev/sda4 on /home type xfs (rw,noatime,logbsize=256k,logbufs=2,nobarrier)
> Kernel 2.6.29.1 (vanilla)
>
> with cfq:
>
> htd@liesel:~/!> ./fsync-tester
> fsync time: 0.7640
> fsync time: 0.6166
> fsync time: 1.2830
> fsync time: 0.4273
> fsync time: 1.1693
> fsync time: 1.7466
> fsync time: 1.2477
> fsync time: 1.9411
> fsync time: 1.9636
> fsync time: 1.9065
> fsync time: 1.1561
> fsync time: 1.8267
> fsync time: 0.2431
> fsync time: 0.2898
> fsync time: 0.2394
> fsync time: 0.4309
> fsync time: 1.5699
> fsync time: 0.3742
> fsync time: 1.3194
> fsync time: 1.9442
> fsync time: 1.0899
> fsync time: 1.9443
> fsync time: 1.0062
>
> with dl:
>
> fsync time: 10.5853
> fsync time: 10.3339
> fsync time: 5.3374
> fsync time: 6.5707
> fsync time: 10.6095
> fsync time: 4.1154
> fsync time: 4.9604
> fsync time: 10.5325
> fsync time: 10.4543
> fsync time: 10.4970
> fsync time: 10.5570
> fsync time: 5.2717
> fsync time: 10.5619
> fsync time: 5.3058
> fsync time: 3.1019
> fsync time: 5.1504
> fsync time: 5.7564
> fsync time: 10.5998
> fsync time: 4.0895
>
>
> Regards, Heinz.
>
>
--
__________________________________________________________________________
dott. Corrado Zoccolo mailto:[email protected]
PhD - Department of Computer Science - University of Pisa, Italy
--------------------------------------------------------------------------
The self-confidence of a warrior is not the self-confidence of the average
man. The average man seeks certainty in the eyes of the onlooker and calls
that self-confidence. The warrior seeks impeccability in his own eyes and
calls that humbleness.
Tales of Power - C. Castaneda
On 08.04.2009, Corrado Zoccolo wrote:
> Well, that's not an usual workload for netbooks, where most SSDs are
> currently deployed.
Yes, that's right.
> For usual workloads, that are mostly read, cfq has lower performance
> both in throughput and in latency than deadline.
I don't have a netbook myself, but a Notebook with a singlecore
Intel M-530 CPU and an SSD harddisk, hdparm says:
[....]
ATA device, with non-removable media
Model Number: OCZ SOLID_SSD
Serial Number: MK0708520E8AA000B
Firmware Revision: 02.10104
[....]
I did run a short test with fsync-tester, running 10 read-processes
on the disk at the same time. The results between CFQ and DL don't differ
visibly. Maybe I don't get the point, or my tests simply suck,
but with these results in mind, and considering the fact that when
the load gets gradually higher, DL will lead to hickups up to ca. 10 secs,
I would say that DL sucks _bigtime_ , compared to CFQ.
(Throughput doesn't differ that much either..).
CFQ:
fsync time: 0.0209
fsync time: 0.0204
fsync time: 0.2026
fsync time: 0.2053
fsync time: 0.2036
fsync time: 0.2348
fsync time: 0.2030
fsync time: 0.2051
fsync time: 0.2024
fsync time: 0.2108
fsync time: 0.2025
fsync time: 0.2025
fsync time: 0.2030
fsync time: 0.2006
fsync time: 0.2368
fsync time: 0.2070
fsync time: 0.2009
fsync time: 0.2033
fsync time: 0.2101
fsync time: 0.2054
fsync time: 0.2028
fsync time: 0.2031
fsync time: 0.2073
fsync time: 0.2100
fsync time: 0.2078
fsync time: 0.2093
fsync time: 0.0275
fsync time: 0.0217
fsync time: 0.0298
fsync time: 0.0206
fsync time: 0.0184
fsync time: 0.0201
fsync time: 0.0169
fsync time: 0.0202
fsync time: 0.0186
fsync time: 0.0224
fsync time: 0.0224
fsync time: 0.0214
fsync time: 0.0246
DL
fsync time: 0.0296
fsync time: 0.0223
fsync time: 0.0262
fsync time: 0.0232
fsync time: 0.0230
fsync time: 0.0235
fsync time: 0.0187
fsync time: 0.0284
fsync time: 0.0227
fsync time: 0.0314
fsync time: 0.0236
fsync time: 0.0251
fsync time: 0.0221
fsync time: 0.0279
fsync time: 0.0244
fsync time: 0.0217
fsync time: 0.0248
fsync time: 0.0241
fsync time: 0.0229
fsync time: 0.0212
fsync time: 0.0243
fsync time: 0.0227
fsync time: 0.0257
fsync time: 0.0206
fsync time: 0.0214
fsync time: 0.0255
fsync time: 0.0213
fsync time: 0.0212
fsync time: 0.0266
fsync time: 0.0221
fsync time: 0.0212
fsync time: 0.0246
fsync time: 0.0208
fsync time: 0.0267
fsync time: 0.0220
fsync time: 0.0213
fsync time: 0.0212
fsync time: 0.0264
htd@wildsau:~> bonnie++ -u htd:default -d /testing -s 4004m -m wildsau -n 16:100000:16:64
CFQ
Version 1.01d ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
wildsau 16016M 79619 45 78058 14 28841 7 98629 61 138596 14 1292 3
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16:100000:16/64 594 7 +++++ +++ 1309 6 556 6 +++++ +++ 449 4
DL
Version 1.01d ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
wildsau 16016M 80619 47 78123 14 27842 7 96317 59 135446 14 1383 4
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16:100000:16/64 601 8 +++++ +++ 1288 6 546 6 +++++ +++ 432 4
On 09.04.2009, Heinz Diehl wrote:
[....]
Quoting myself, I did copy in the wrong results from another machine,
sorry for that! The fsync-tester results were the right ones, though.
Here are the correct ones for the bonnie++ runs (the overall results doesn't differ
significantly either):
CFQ
Version 1.01d ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
wildsau 4004M 69133 48 67512 12 21810 8 94426 59 129977 13 1163 3
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16:100000:16/64 589 8 +++++ +++ 1211 6 544 7 +++++ +++ 421 4
DL
Version 1.01d ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
wildsau 4004M 71112 46 66155 15 22001 8 95132 62 130416 14 1191 3
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16:100000:16/64 565 9 +++++ +++ 1339 6 561 8 +++++ +++ 430 4
Heinz Diehl wrote:
> On 08.04.2009, Corrado Zoccolo wrote:
>
>> I found that elevator=deadline performs much better than noop for
>> writes, and almost as well for reads
> [....]
>
> The DL elevator has slightly more throughput than cfq and anticipatory,
> but is almost unusuable under load.
>
> Running Theodore Ts'os "fsync-tester" while doing Linus' torture test
> "while : ; do time sh -c "dd if=/dev/zero of=bigfile bs=8M count=256 ; sync; rm bigfile"; done"
> shows it clearly:
>
This is good information, and if I ever configure a netbook for run fsync-tester
I shall avoid the DL scheduler. ;-(
However... this test, and several others designed to find the ultimate
performance limits of disk io, don't mimic any typical use of most desktops and
virtually all netbooks.
Is there a benchmark which would return so useful data for typical use, doing
some mail, some browsing, and maybe some light presentation, spreadsheet, or
word processing. None of those uses are likely to generate this level of io,
this file size, etc. The number of users is one, it's not used as a server, and
probably most of the tuning done (if any) is aimed at battery life rather than
blinding speed with a three digit load average.
I don't think this is a useful benchmark for netbooks, and hopefully there is a
test available which will give more insight into the performance in typical use.
--
Bill Davidsen <[email protected]>
"We have more to fear from the bungling of the incompetent than from
the machinations of the wicked." - from Slashdot
On Thu, Apr 09, 2009 at 07:56:32PM -0400, Bill Davidsen wrote:
> This is good information, and if I ever configure a netbook for run
> fsync-tester I shall avoid the DL scheduler. ;-(
>
> However... this test, and several others designed to find the ultimate
> performance limits of disk io, don't mimic any typical use of most
> desktops and virtually all netbooks.
>
> Is there a benchmark which would return so useful data for typical use,
> doing some mail, some browsing, and maybe some light presentation,
> spreadsheet, or word processing. None of those uses are likely to
> generate this level of io, this file size, etc. The number of users is
> one, it's not used as a server, and probably most of the tuning done (if
> any) is aimed at battery life rather than blinding speed with a three
> digit load average.
As long as you don't believe a netbook user will ever try to type an
e-mail using a mail reader like alpine (which is what Linus uses),
while running "yum update" in the background, sure. But if you don't
think that is a normal use case, I'll let you argue with Linus on that
score. In any case, the big-file-write-and-flush plus fsync-tester
was designed to roughly replicate this scenario which Linus saw on his
desktop system.
- Ted
Theodore Tso wrote:
> On Thu, Apr 09, 2009 at 07:56:32PM -0400, Bill Davidsen wrote:
>
>> This is good information, and if I ever configure a netbook for run
>> fsync-tester I shall avoid the DL scheduler. ;-(
>>
>> However... this test, and several others designed to find the ultimate
>> performance limits of disk io, don't mimic any typical use of most
>> desktops and virtually all netbooks.
>>
>> Is there a benchmark which would return so useful data for typical use,
>> doing some mail, some browsing, and maybe some light presentation,
>> spreadsheet, or word processing. None of those uses are likely to
>> generate this level of io, this file size, etc. The number of users is
>> one, it's not used as a server, and probably most of the tuning done (if
>> any) is aimed at battery life rather than blinding speed with a three
>> digit load average.
>>
>
> As long as you don't believe a netbook user will ever try to type an
> e-mail using a mail reader like alpine (which is what Linus uses),
> while running "yum update" in the background, sure. But if you don't
> think that is a normal use case, I'll let you argue with Linus on that
> score. In any case, the big-file-write-and-flush plus fsync-tester
> was designed to roughly replicate this scenario which Linus saw on his
> desktop system.
>
I almost fell out of my chair reading this, because I was doing exactly
what you describe, applying the latest Fedora 9 security stuff and
reading mail. The difference is that I'm not on a netbook, with a puny
CPU, small memory and 5400 rpm drive, but a VM running on a host with
multi-core, 8GB RAM, and a raid array of fast TB drives. The way I would
use a netbook would be as a glorified PDA, and I still feel that a
useful benchmark should duplicate the typical use, rather than some case
which occurs a few times a month, unless that case is the critical load
and justified less optimal performance the majority of the time.
Thanks for the history lesson, but I want my netbook to handle best the
stuff I do most. If the modified scheduler performed better when I am
reading mail and need to open a spreadsheet attachment, I'd certainly
prefer that. Whether I'd bother to build my own patched kernel would
depend on how "faster" it ran, I'm reasonably happy with the way that
runs on what I have.
--
bill davidsen <[email protected]>
CTO TMR Associates, Inc
"You are disgraced professional losers. And by the way, give us our money back."
- Representative Earl Pomeroy, Democrat of North Dakota
on the A.I.G. executives who were paid bonuses after a federal bailout.