2006-11-24 16:09:31

by Larry Finger

[permalink] [raw]
Subject: 2.6.19-rc5-mm1 progression

Since at least the 2.6.18 development kernels, my interactive sessions have sometimes been very
sluggish whenever a cpu-intensive process is running. This has been so bad that the mouse is
unresponsive. The system behaved as if it were swapping; however, swapfile usage is always 0. The
system has 820 MB RAM. I have not reported this problem as I really couldn't quantify it due to its
intermittent nature.

Recently, I needed to run 2.6.19-rc5-mm1 to check out a problem reported by a bcm43xx user. I found
that even with a kernel build, a git pull, and a separate build running, the system was responsive
for interactive tasks. I have not yet identified which of the -mm1 patches "fixes" the problem, but
plan to do so. Is there the equivalent of 'git bisect' for the -mmX kernels?

Larry


2006-11-24 16:36:30

by Benoit Boissinot

[permalink] [raw]
Subject: Re: 2.6.19-rc5-mm1 progression

On 11/24/06, Larry Finger <[email protected]> wrote:
> Is there the equivalent of 'git bisect' for the -mmX kernels?
>
http://www.zip.com.au/~akpm/linux/patches/stuff/bisecting-mm-trees.txt

regards,

Benoit

2006-11-25 19:25:13

by Andrew Morton

[permalink] [raw]
Subject: Re: 2.6.19-rc5-mm1 progression

On Fri, 24 Nov 2006 17:36:27 +0100
"Benoit Boissinot" <[email protected]> wrote:

> On 11/24/06, Larry Finger <[email protected]> wrote:
> > Is there the equivalent of 'git bisect' for the -mmX kernels?
> >
> http://www.zip.com.au/~akpm/linux/patches/stuff/bisecting-mm-trees.txt
>

Please take the time to do that. Yours is an interesting report - I'm not
aware of anything in there which was expected to cause a change of this
mature.

2006-11-25 21:28:39

by Larry Finger

[permalink] [raw]
Subject: Re: 2.6.19-rc5-mm1 progression

Andrew Morton wrote:
> On Fri, 24 Nov 2006 17:36:27 +0100
> "Benoit Boissinot" <[email protected]> wrote:
>
>> On 11/24/06, Larry Finger <[email protected]> wrote:
>>> Is there the equivalent of 'git bisect' for the -mmX kernels?
>>>
>> http://www.zip.com.au/~akpm/linux/patches/stuff/bisecting-mm-trees.txt
>>
>
> Please take the time to do that. Yours is an interesting report - I'm not
> aware of anything in there which was expected to cause a change of this
> mature.
>

I am in the process of isolating the patch. I was already a user of quilt so that part of the
learning process was no problem.

Larry

2006-12-05 03:33:16

by Larry Finger

[permalink] [raw]
Subject: Re: 2.6.19-rc5-mm1 progression

Andrew Morton wrote:
> On Fri, 24 Nov 2006 17:36:27 +0100
> "Benoit Boissinot" <[email protected]> wrote:
>
>> On 11/24/06, Larry Finger <[email protected]> wrote:
>>> Is there the equivalent of 'git bisect' for the -mmX kernels?
>>>
>> http://www.zip.com.au/~akpm/linux/patches/stuff/bisecting-mm-trees.txt
>>
>
> Please take the time to do that. Yours is an interesting report - I'm not
> aware of anything in there which was expected to cause a change of this
> mature.
>

There are at least two patches in 2.6.19-rc5-mm2 that make my system much more responsive for
interactive jobs. The one that has the majority of the effect is:

radix-tree-rcu-lockless-readside.patch

I have not been able to isolate the second patch, which has the lesser effect. All I can say is that
it occurred before the above patch in patches/series. This patch was tested against 2.6.19 and fixed
most of the problem on that version.

Larry Finger


2006-12-06 12:19:59

by Peter Zijlstra

[permalink] [raw]
Subject: Re: 2.6.19-rc5-mm1 progression

On Mon, 2006-12-04 at 21:32 -0600, Larry Finger wrote:
> Andrew Morton wrote:
> > On Fri, 24 Nov 2006 17:36:27 +0100
> > "Benoit Boissinot" <[email protected]> wrote:
> >
> >> On 11/24/06, Larry Finger <[email protected]> wrote:
> >>> Is there the equivalent of 'git bisect' for the -mmX kernels?
> >>>
> >> http://www.zip.com.au/~akpm/linux/patches/stuff/bisecting-mm-trees.txt
> >>
> >
> > Please take the time to do that. Yours is an interesting report - I'm not
> > aware of anything in there which was expected to cause a change of this
> > mature.
> >
>
> There are at least two patches in 2.6.19-rc5-mm2 that make my system much more responsive for
> interactive jobs. The one that has the majority of the effect is:
>
> radix-tree-rcu-lockless-readside.patch
>
> I have not been able to isolate the second patch, which has the lesser effect. All I can say is that
> it occurred before the above patch in patches/series. This patch was tested against 2.6.19 and fixed
> most of the problem on that version.

Curious...

This patch introduces the direct pointer optimisation for single element
radix trees and makes the radix tree safe to read in a lock-less manner
which is not used -yet-. The only difference that that should have is
that the elements are freed using rcu callback instead of directly.

/me puzzled how this has a large effect on interactivity.

Nick?

2006-12-06 14:32:27

by Larry Finger

[permalink] [raw]
Subject: Re: 2.6.19-rc5-mm1 progression

Peter Zijlstra wrote:
> On Mon, 2006-12-04 at 21:32 -0600, Larry Finger wrote:
>> Andrew Morton wrote:
>>> On Fri, 24 Nov 2006 17:36:27 +0100
>>> "Benoit Boissinot" <[email protected]> wrote:
>>>
>>>> On 11/24/06, Larry Finger <[email protected]> wrote:
>>>>> Is there the equivalent of 'git bisect' for the -mmX kernels?
>>>>>
>>>> http://www.zip.com.au/~akpm/linux/patches/stuff/bisecting-mm-trees.txt
>>>>
>>> Please take the time to do that. Yours is an interesting report - I'm not
>>> aware of anything in there which was expected to cause a change of this
>>> mature.
>>>
>> There are at least two patches in 2.6.19-rc5-mm2 that make my system much more responsive for
>> interactive jobs. The one that has the majority of the effect is:
>>
>> radix-tree-rcu-lockless-readside.patch
>>
>> I have not been able to isolate the second patch, which has the lesser effect. All I can say is that
>> it occurred before the above patch in patches/series. This patch was tested against 2.6.19 and fixed
>> most of the problem on that version.
>
> Curious...
>
> This patch introduces the direct pointer optimisation for single element
> radix trees and makes the radix tree safe to read in a lock-less manner
> which is not used -yet-. The only difference that that should have is
> that the elements are freed using rcu callback instead of directly.
>
> /me puzzled how this has a large effect on interactivity.

I also wondered why this patch should have the effect as it didn't seem in an area that should be
important. I did test it on both 2.6.19-rc5-mm2 and vanilla 2.6.19. In both cases, it made
interactive response improve.

One single task that seems to have the biggest effect on my unpatched systems is the link step for a
new kernel. I am assuming that this process calls for lots of virtual memory. One other
characteristic of my system when it gets unresponsive is that kswapd is running regularly; however,
I have never observed any swap file usage.

Larry

2006-12-07 09:37:10

by Nick Piggin

[permalink] [raw]
Subject: Re: 2.6.19-rc5-mm1 progression

Peter Zijlstra wrote:
> On Mon, 2006-12-04 at 21:32 -0600, Larry Finger wrote:

>>There are at least two patches in 2.6.19-rc5-mm2 that make my system much more responsive for
>>interactive jobs. The one that has the majority of the effect is:
>>
>>radix-tree-rcu-lockless-readside.patch
>>
>>I have not been able to isolate the second patch, which has the lesser effect. All I can say is that
>>it occurred before the above patch in patches/series. This patch was tested against 2.6.19 and fixed
>>most of the problem on that version.
>
>
> Curious...
>
> This patch introduces the direct pointer optimisation for single element
> radix trees and makes the radix tree safe to read in a lock-less manner
> which is not used -yet-. The only difference that that should have is
> that the elements are freed using rcu callback instead of directly.
>
> /me puzzled how this has a large effect on interactivity.
>
> Nick?

I have already got the direct data optimisation upstream. It might be
possible that it is some interaction with the extra rcu callbacks going
off... I don't know :\

--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com