Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="US-ASCII"
Content-Transfer-Encoding: 8BIT
Subject: RE: Bandwidth Allocations under CFQ I/O Scheduler
Date: Tue, 17 Oct 2006 10:46:43 -0400
Message-ID: <CAEAF2308EEED149B26C2C164DFB20F4E7EF43@ALPMLVEM06.e2k.ad.ge.com>
In-Reply-To: <20061017132312.GD7854@kernel.dk>
Thread-Topic: Bandwidth Allocations under CFQ I/O Scheduler
Thread-Index: Acbx71OYqgBmFb6HQECtSuAdDzT38wABpvEw
From: "Phetteplace, Thad \(GE Healthcare, consultant\)" 
	<Thad.Phetteplace@ge.com>
To: "Jens Axboe" <jens.axboe@oracle.com>,
       "Arjan van de Ven" <arjan@infradead.org>
Cc: <linux-kernel@vger.kernel.org>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2770
Lines: 59

Jens Axboe wrote:
> Arjan van de Ven wrote:
> > 
> > it's a nice idea in theory. However... since IO bandwidth for seeks
is 
> > about 1% to 3% of that of sequential IO (on disks at least), which 
> > bandwidth do you want to allocate? "worst case" you need to use the 
> > all-seeks bandwidth, but that's so far away from "best case" that it

> > may well not be relevant in practice. Yet there are real world cases

> > where for a period of time you approach worst case behavior ;(
>
> Bandwidth reservation would have to be confined to special cases, you
> obviously cannot do it "in general" for the reasons Arjan lists above.
> So you absolutely have to limit any meta data io that would cause
seeks,
> and the file in question would have to be laid out in a closely
> sequential fashion. As long as the access pattern generated by the app
> asking for reservation is largely sequential, the kernel can do
whatever
> it needs to help you maintain the required bandwidth.
> 
> On a per-file basis the bandwidth reservation should be doable, to the
> extent that generic hardware allows.

I see bandwidth allocations coming in two flavors: floors and ceilings.
Floors (a guaranteed minimum) are indeed problematic because of the
danger of over-allocating bandwidth.  Seek latency reducing your total
available bandwidth in non-deterministic ways only complicates the
issue.
Ceilings are easier, as we are simply capping utilization even when
excess capacity is available.  Of course floors is probably what most
people are thinking of when they talk about allocations, but ceilings
have their place also.  In an embedded environment where very
deterministic behavior is the goal, I/O ceilings could be useful.  Also,
it could be useful for emulation of legacy hardware performance, perhaps
for regression testing or some such (admittedly an edge case).

If you over allocate bandwidth on a resource, the bandwidth allocation
would probably fall back to something more like the 'niceness' model
(with the higher bandwidth procs running with higher priority).  The
only real change then is the enforcing of bandwidth ceilings.  This
is probably not very useful in the general case (your main operating
system drives with many users/processes reading and writing), but it
can be very useful for managing the behavior of a limited set of apps
with excusive access to a drive.

There is a body of knowledge in the ISP/routing world we can draw on
here, though they don't have the same latency issues.

Later,

Thad Phetteplace
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/