Per device dirty throttling patches
These patches aim to improve balance_dirty_pages() and directly address three
issues:
1) inter device starvation
2) stacked device deadlocks
3) inter process starvation
1 and 2 are a direct result from removing the global dirty limit and using
per device dirty limits. By giving each device its own dirty limit is will
no longer starve another device, and the cyclic dependancy on the dirty limit
is broken.
In order to efficiently distribute the dirty limit across the independant
devices a floating proportion is used, this will allocate a share of the total
limit proportional to the device's recent activity.
3 is done by also scaling the dirty limit proportional to the current task's
recent dirty rate.
Changes since -v9:
- cleaned up the perpcu_counter_init code
- little fixups
- fwd port to .23-rc4-mm1
Changes since -v8:
- cleanup of the proportion code
- fix percpu_counter_add(&counter, -(unsigned long))
- fix per task dirty rate code
- fwd port to .23-rc2-mm2
Peter> Per device dirty throttling patches These patches aim to
Peter> improve balance_dirty_pages() and directly address three
Peter> issues:
Peter> 1) inter device starvation
Peter> 2) stacked device deadlocks
Peter> 3) inter process starvation
Peter> 1 and 2 are a direct result from removing the global dirty
Peter> limit and using per device dirty limits. By giving each device
Peter> its own dirty limit is will no longer starve another device,
Peter> and the cyclic dependancy on the dirty limit is broken.
Ye haa! This should be a big improvement.
Peter> In order to efficiently distribute the dirty limit across the
Peter> independant devices a floating proportion is used, this will
Peter> allocate a share of the total limit proportional to the
Peter> device's recent activity.
I'm not sure I like or agree with this. Shouldn't we be limiting
based on the device's capability to sustain traffic? So if I have a
RAID device which can read/write a total of 100Mb/sec, while at the
same time I've got a CF device which can do 5Mb/sec, shouldn't we be
more strongly limiting the CF device, even if it is the only device
being written to?
Of course, I haven't read the patches yet, nor am I qualified to
comment on them in any meanginful way I think. Hopefully I'm just
missing something key here in the explanation.
Peter> 3 is done by also scaling the dirty limit proportional to the
Peter> current task's recent dirty rate.
Do you mean task or device here? I'm just wondering how well this
works with a bunch of devices with wildly varying speeds.
John
On Tue, 2007-09-11 at 22:31 -0400, John Stoffel wrote:
I hope the snipped questions were sufficiently answered in the other
mail. If not, holler :-)
> Peter> 3 is done by also scaling the dirty limit proportional to the
> Peter> current task's recent dirty rate.
>
> Do you mean task or device here? I'm just wondering how well this
> works with a bunch of devices with wildly varying speeds.
Task. What I do is modify the limit like this:
current_limit = dirty_limit * p(bdi_writeout) * (1 - p(task_dirty)/8)
Where the p() values are [0, 1].
By including the inverse of the task dirty rate one gets that tasks that
are more agressive dirtiers get throttled more aggressively, whereas
tasks that occasionally dirty a page get a little more room.
Peter,
I have been using your -v9 backport to 2.6.22 (thanks very much
btw) for the last few days on an amd64 system configured like this:
[/],[/home],[/backup],...
on top of
[lvm2]
using
[/dev/sdb]
which is an
[areca hw raid 6 device made up from 8 sata disks]
With your patch, performance was much improved. But still there are
some seemingly simple tasks which do not work well. Like
cp /home/bigfile.txt /backup/bigfile.txt
or also
rm -r /backup/big-directory
They both affect disk access times of other processes very badly.
As I said, things improved a quite a bit with your patch, but I wonder
if there is more that could be done for this particular setup?
A second thing. Since my box has 8GB memory, I am intrigued by the
idea of setting dirty_ratio not as a ratio but as an absolute
number of kilobytes as it was suggested by Neil Brown with the
dirty_kb patch a few months back. Would it make sense to forward
port this patch on top of yours ?
cheers
tobi
--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
http://it.oetiker.ch [email protected] ++41 62 213 9902