Hi all
I've used the corner cases described that cause a lot of the interactivity
problems to develop this patch. What it does different to default (without
nasty hacks) is the following:
The interactivity is based on the max sleep avg (10s) as previously, unless
the thread has been running for less than 10s. In that case it is based on
the duration it has been running, with a minimum of 10 jiffies to estimate
the interactivity (perhaps more might be appropriate here). This should
minimise tasks taking ages to be categorised as interactive.
If tasks sleep for a long time (>1s) they are no longer classified as
interactive, but idle - meaning they receive their static priority. However,
since they _may_ become interactive, the period over which the interactivity
is estimated is decreased (1sec, perhaps can be reduced) so they may become
interactive rapidly again. This should minimise idle tasks (like BASH) that
suddenly become cpu hungry from stalling everything.
I'm still working on something for the "xmms stalls if started during very
heavy load" as a different corner case.
This is only a compile tested patch (need sleep bad) but has been tested in
2.4 form and works nicely at avoiding those corner test cases.
Please comment and test if you can.
Con
At 02:09 AM 6/26/2003 +1000, Con Kolivas wrote:
>I'm still working on something for the "xmms stalls if started during very
>heavy load" as a different corner case.
One way to deal with that problem would be to create a very high priority
queue which is reserved for very light weight tasks, and heavily protected
against them going cpu hungry. If a forking task has a run history worthy
of trial, slip it straight into the high priority queue... and be prepared
to beat it into submission should it misbehave (short term run_avg ever
exceeds X, you're outta here buddy, and if this queue is consuming more
than Y, sorry, we're closed).
Another way would be to factor task age into the priority calculation, with
age becoming rapidly less important. Xmms' audio threads are light weight,
and will quickly be able to sustain their priority. Others would
(hopefully) rapidly fall down where they belong.
Just a couple random thoughts, both of which I can see problems with ;-)
-Mike
On Wed, 2003-06-25 at 15:00, Mike Galbraith wrote:
> At 02:09 AM 6/26/2003 +1000, Con Kolivas wrote:
>
> >I'm still working on something for the "xmms stalls if started during very
> >heavy load" as a different corner case.
>
<snip scheduler suggestion>
> Just a couple random thoughts, both of which I can see problems with ;-)
>
At least on 2.4 (I use 21-ck3), it appears to be I/O starvation that
gets xmms, not scheduler starvation. When xmms skips for me, there's
load, but there's also usually some idle time. The common thread seems
to be heavy I/O on the drive xmms is using, possibly combined with a
(formerly?) interactive process (evolution rebuilding my LKML index, for
example) doing the disk I/O. Because of the assorted I/O scheduler
changes in 2.5, this is unlikley to be the problem there.
Daniel
--
Daniel Gryniewicz <[email protected]>
At 03:34 PM 6/25/2003 -0400, Daniel Gryniewicz wrote:
>On Wed, 2003-06-25 at 15:00, Mike Galbraith wrote:
> > At 02:09 AM 6/26/2003 +1000, Con Kolivas wrote:
> >
> > >I'm still working on something for the "xmms stalls if started during very
> > >heavy load" as a different corner case.
> >
><snip scheduler suggestion>
> > Just a couple random thoughts, both of which I can see problems with ;-)
> >
>
>At least on 2.4 (I use 21-ck3), it appears to be I/O starvation that
>gets xmms, not scheduler starvation. When xmms skips for me, there's
>load, but there's also usually some idle time. The common thread seems
>to be heavy I/O on the drive xmms is using, possibly combined with a
>(formerly?) interactive process (evolution rebuilding my LKML index, for
>example) doing the disk I/O. Because of the assorted I/O scheduler
>changes in 2.5, this is unlikley to be the problem there.
Ahah. I thought Con was referring to the delay at new song, new task
starting at priority 20 while things higher are using the cpu. Yeah, your
skips sound like xmms is just running out of buffered data.
-Mike
On Thu, 26 Jun 2003 06:33, Mike Galbraith wrote:
> At 03:34 PM 6/25/2003 -0400, Daniel Gryniewicz wrote:
> >On Wed, 2003-06-25 at 15:00, Mike Galbraith wrote:
> > > At 02:09 AM 6/26/2003 +1000, Con Kolivas wrote:
> > > >I'm still working on something for the "xmms stalls if started during
> > > > very heavy load" as a different corner case.
> >
> ><snip scheduler suggestion>
> >
> > > Just a couple random thoughts, both of which I can see problems with
> > > ;-)
> >
> >At least on 2.4 (I use 21-ck3), it appears to be I/O starvation that
> >gets xmms, not scheduler starvation. When xmms skips for me, there's
> >load, but there's also usually some idle time. The common thread seems
> >to be heavy I/O on the drive xmms is using, possibly combined with a
> >(formerly?) interactive process (evolution rebuilding my LKML index, for
> >example) doing the disk I/O. Because of the assorted I/O scheduler
> >changes in 2.5, this is unlikley to be the problem there.
>
> Ahah. I thought Con was referring to the delay at new song, new task
> starting at priority 20 while things higher are using the cpu. Yeah, your
> skips sound like xmms is just running out of buffered data.
No, Mike's right. If you do a make -j32 and then start up xmms, xmms seems to
want to burn off some cpu when it first starts, and _then_ starts sleeping
regularly; so it starts as a cpu hog for a short while and then sleeps. This
makes is bloody hard for the scheduler to do the right thing to it since it
gets treated as a cpu hog initially, meaning it will take forever to burn off
that cpu time necessary when all other cpu hogs are also running.
Con
On Wed, 2003-06-25 at 18:09, Con Kolivas wrote:
> Hi all
>
> I've used the corner cases described that cause a lot of the interactivity
> problems to develop this patch. What it does different to default (without
> nasty hacks) is the following:
>
> The interactivity is based on the max sleep avg (10s) as previously, unless
> the thread has been running for less than 10s. In that case it is based on
> the duration it has been running, with a minimum of 10 jiffies to estimate
> the interactivity (perhaps more might be appropriate here). This should
> minimise tasks taking ages to be categorised as interactive.
>
> If tasks sleep for a long time (>1s) they are no longer classified as
> interactive, but idle - meaning they receive their static priority. However,
> since they _may_ become interactive, the period over which the interactivity
> is estimated is decreased (1sec, perhaps can be reduced) so they may become
> interactive rapidly again. This should minimise idle tasks (like BASH) that
> suddenly become cpu hungry from stalling everything.
>
> I'm still working on something for the "xmms stalls if started during very
> heavy load" as a different corner case.
>
> This is only a compile tested patch (need sleep bad) but has been tested in
> 2.4 form and works nicely at avoiding those corner test cases.
>
> Please comment and test if you can.
This patch is indeed much better than the ones posted before. In fact,
it's really, really hard for me to make XMMS skip audio. It feels much
better in general, but there are still some rough edges when the system
is under load. For example, the mouse cursor on an X session doesn't
move smoothly, and feels a little jumpy. It can be somewhat fixed by
renicing the X server to -20.
Anyways, this is the best I've seen until date, and I'm currently
running on it. We'll see if it's up to expectations... at least, I think
so :-)
>
> Con
On Wed, 2003-06-25 at 14:53, Felipe Alfaro Solana wrote:
> On Wed, 2003-06-25 at 18:09, Con Kolivas wrote:
> > Hi all
> >
> > I've used the corner cases described that cause a lot of the interactivity
> > problems to develop this patch.
> This patch is indeed much better than the ones posted before. In fact,
> it's really, really hard for me to make XMMS skip audio. It feels much
> better in general, but there are still some rough edges when the system
> is under load. For example, the mouse cursor on an X session doesn't
> move smoothly, and feels a little jumpy. It can be somewhat fixed by
> renicing the X server to -20.
I'm running with this patch on my dual-proc desktop right now.
I agree: with a make -j20 going, the mouse became non-responsive
for about 1 second at a time. Renicing the X server to -20 greatly
improved the response of my desktop with this patch under load.
I could switch virtual desktops (blackbox), move the mouse to focus on
an aterm and type a command (and get a response back), and not wait
too long for evolution to repaint or open a piece of email.
I could tell that something was grinding away on my system, but it was
still tolerable.
Andy
On Thu, 26 Jun 2003 09:10, Andy Pfiffer wrote:
> On Wed, 2003-06-25 at 14:53, Felipe Alfaro Solana wrote:
> > On Wed, 2003-06-25 at 18:09, Con Kolivas wrote:
> > > Hi all
> > >
> > > I've used the corner cases described that cause a lot of the
> > > interactivity problems to develop this patch.
> >
> > This patch is indeed much better than the ones posted before. In fact,
> > it's really, really hard for me to make XMMS skip audio. It feels much
> > better in general, but there are still some rough edges when the system
> > is under load. For example, the mouse cursor on an X session doesn't
> > move smoothly, and feels a little jumpy. It can be somewhat fixed by
> > renicing the X server to -20.
>
> I'm running with this patch on my dual-proc desktop right now.
>
> I agree: with a make -j20 going, the mouse became non-responsive
> for about 1 second at a time. Renicing the X server to -20 greatly
> improved the response of my desktop with this patch under load.
>
> I could switch virtual desktops (blackbox), move the mouse to focus on
> an aterm and type a command (and get a response back), and not wait
> too long for evolution to repaint or open a piece of email.
>
> I could tell that something was grinding away on my system, but it was
> still tolerable.
Thanks for testing this. The maximum interactive-non interactive difference
to tasks niced to 0 will be 10, so renice X to -11 should be the most you
need... -10 is what a lot of distributions already do by default.
Are there any other corner cases you've found with this patch? I have ideas
about the "starting xmms under extreme load" issue but I need to know if it's
a real life scenario, and whether building an algorithm around one
application in one setting is worthwhile.
Also to those actually looking at the code, would you like me to comment it in
detail? I kept it short on purpose, but adding comments would be simple
enough.
Con
On Thu, 26 Jun 2003 02:09:45 +1000
Con Kolivas <[email protected]> wrote:
> Please comment and test if you can.
Fixes the very bad behaviuor plain (or -mm) 2.5.73 behaviour has, when
for example, i check for new mail with sylpheed and it merges
the new messages in the 114 MB lmkl folder....(mouse isn't reponsive,
and this is a SMP box)