Date: Tue, 22 Mar 2011 14:12:31 -0400
From: Vivek Goyal <vgoyal@redhat.com>
To: Chad Talbott <ctalbott@google.com>
Cc: jaxboe@fusionio.com, linux-kernel@vger.kernel.org, mrubin@google.com,
        teravest@google.com
Subject: Re: [PATCH 0/3] cfq-iosched: Fair cross-group preemption
Message-ID: <20110322181231.GJ3757@redhat.com>
References: <1300756245-12380-1-git-send-email-ctalbott@google.com>
 <20110322150905.GD3757@redhat.com>
 <AANLkTinTiEAFG1F1df380BiDtVFVr=nCsSqhM9__XdQ4@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <AANLkTinTiEAFG1F1df380BiDtVFVr=nCsSqhM9__XdQ4@mail.gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 4096
Lines: 82

On Tue, Mar 22, 2011 at 10:39:36AM -0700, Chad Talbott wrote:
> On Tue, Mar 22, 2011 at 8:09 AM, Vivek Goyal <vgoyal@redhat.com> wrote:
> > Why not just implement simply RT class groups and always allow an RT
> > group to preempt an BE class. Same thing we do for cfq queues. I will
> > not worry too much about a run away application consuming all the
> > bandwidth. If that's a concern we could use blkio controller to limit
> > the IO rate of a latency sensitive applicaiton to make sure it does
> > not starve BE applications.
> 
> That is not quite the same semantics.  This limited preemption patch
> is still work-conserving.  If the RT task in the only task on the
> system with IO, it will be able to use all available disk time.
> 

It is not same semantics but it feels like too much of special casing
for a single use case.

You are using the generic notion of a RT thread (which in general means
that it gets all the cpu or all the disk ahead of BE task). But you have
changed the definition of RT for this special use case. And also now
group RT is different from queue RT definition.

Why not have similar mechanism for cpu scheduler also then. This
application first should be able to get cpu bandwidth in same predictable
manner before it gets the disk bandwidth.

And I think your generation number patch should address this issue up
to great extent. Isn't it? If a latency sensitive task is not using
its fair quota, it will get a lower vdisktime and get to dispatch soon?

If that soon is not enough, then we could operate with reduce base slice
length so that we allocate smaller slices to groups and get better IO
latencies at the cost of total throughput. 

> > If RT starving BE is an issue, then it is an issue with plain cfq queue
> > also. First we shall have to fix it there.
> >
> > This definition that a latency sensitive task get prioritized only
> > till it is consuming its fair share and if task starts using more than
> > fair share then CFQ automatically stops prioritizing it sounds little
> > odd to me. If you are looking for predictability, then we lost it. We
> > shall have to very well know that task is not eating more than its
> > fair share before we can gurantee any kind of latencies to that task. And
> > if we know that task is not hogging the disk, there is anyway no risk
> > of it starving other groups/tasks completely.
> 
> In a shared environment, we have to be a little bit defensive.  We
> hope that a latency sensitive task is well characterized and won't
> exceed its share of the disk, and that we haven't over-committed the
> disk.  If the app does do more IO than expected, then we'd like them
> to bear the burden.  We have a choice of two outcomes.  A single job
> sometimes failing to achieve low disk latency when it's very busy.  Or
> all jobs on a disk sometimes being very slow when another (unrelated)
> job is very busy.  The first is easier to understand and debug.

To me you are trying to come up with a new scheduling class which is
not RT and you are trying to overload the meaning of RT for your use
case and that's the issue I have.

Coming up with a new scheduling class is also not desirable as that
will demand another service tree and we already have too many. Also
it should probably be also done for task and not just group otherwise
extending this concept to hierarchical setup will get complicated. Queues
and groups will just not gel well.

Frankly speaking, the problem you are having should be solved by your
generation number patch and by having smaller base slices. 

Or You could put latency sensitive applications in an RT class and 
then throttle them using blkio controller. That way you get good
latencies as well as you don't starve other tasks.

But I don't think overloading the meaning for RT or this specific use
case is a good idea.

Thanks
Vivek
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/