2003-11-26 23:27:39

by Adam Kropelin

[permalink] [raw]
Subject: Parallel build not working since -test6?

Lately I've noticed my kernel compilations taking longer than usual. Tonight
I finally realized the cause... Parallel building (i.e. make -jN) is no
longer working for me. I traced it back and the last kernel it worked in
was -test5. It ceased working in -test6.

Not to point fingers, but I do notice that the kbuild separate output
directory patch went into -test6...

Build environment details:
arch: i386
make: GNU Make version 3.79.1
gcc: gcc version 2.96 20000731 (Red Hat Linux 7.1 2.96-98)
output directory: default (i.e. source dir)

I can narrow it down to a -bk if that's useful or if someone has a patch to
back out the separate output dir kbuild changes I can give that a whirl...

--Adam


2003-11-28 20:25:52

by Sam Ravnborg

[permalink] [raw]
Subject: Re: Parallel build not working since -test6?

On Thursday 27 November 2003 00:27, Adam Kropelin wrote:
> Lately I've noticed my kernel compilations taking longer than usual.
> Tonight I finally realized the cause... Parallel building (i.e. make -jN)
> is no longer working for me. I traced it back and the last kernel it worked
> in was -test5. It ceased working in -test6.
It works for me, and for sure it works for most others. Otherwise I would
have seen lot of complaints like yours.
I recall one similar post, and the person in question used a homegrown
script that caused the problems.

Could you try to post:
a) Exact command used when building the kernel.
b) Output of kernel compile [first 100 lines] after a make clean

This may give me a hint of what is wrong.

Sam

2003-11-29 01:47:34

by Adam Kropelin

[permalink] [raw]
Subject: Re: Parallel build not working since -test6?

Sam Ravnborg <[email protected]> wrote:
> On Thursday 27 November 2003 00:27, Adam Kropelin wrote:
>> Lately I've noticed my kernel compilations taking longer than usual.
>> Tonight I finally realized the cause... Parallel building (i.e. make
>> -jN) is no longer working for me. I traced it back and the last
>> kernel it worked in was -test5. It ceased working in -test6.
> It works for me, and for sure it works for most others. Otherwise I
> would have seen lot of complaints like yours.
> I recall one similar post, and the person in question used a homegrown
> script that caused the problems.
>
> Could you try to post:
> a) Exact command used when building the kernel.

make oldconfig
make -j2 install

(Also tried 'make -j2 bzImage' with no change in behavior observed.)

I always do this directly from the command line, no special scripts.

> b) Output of kernel compile [first 100 lines] after a make clean

Attached since I'm sure this mail client will wrap it horribly otherwise. I
enabled verbose mode for the capture since I figured you probably wanted all
the details you could get. This particular compile is for -test6, the first
non-working version.

An additional item I just noticed...I do see 2 parallel builds for the very
first part of the session (when various items in the scripts/ directory are
being compiled). As soon as the scripts/ directory is finished, I only see
one process and my idle cpu usage jumps to about 40% (from 0%).

I'm happy to try anything else you wish...

--Adam


Attachments:
compile.out (32.58 kB)

2003-11-29 02:47:52

by Derek Foreman

[permalink] [raw]
Subject: Re: Parallel build not working since -test6?

On Fri, 28 Nov 2003, Sam Ravnborg wrote:

> On Thursday 27 November 2003 00:27, Adam Kropelin wrote:
> > Lately I've noticed my kernel compilations taking longer than usual.
> > Tonight I finally realized the cause... Parallel building (i.e. make -jN)
> > is no longer working for me. I traced it back and the last kernel it worked
> > in was -test5. It ceased working in -test6.
> It works for me, and for sure it works for most others. Otherwise I would
> have seen lot of complaints like yours.
> I recall one similar post, and the person in question used a homegrown
> script that caused the problems.

Well, this explains why 2.6.x builds so much slower here than it did a few
kernels ago.

make -j3 improves things. but currently, make -j2 doesn't use both my
cpus.

no scripts, just make -j2 bzImage

2003-11-29 03:40:45

by Adam Kropelin

[permalink] [raw]
Subject: Re: Parallel build not working since -test6?

Derek Foreman <[email protected]> wrote:
> On Fri, 28 Nov 2003, Sam Ravnborg wrote:
>
>> On Thursday 27 November 2003 00:27, Adam Kropelin wrote:
>>> Lately I've noticed my kernel compilations taking longer than usual.
>>> Tonight I finally realized the cause... Parallel building (i.e.
>>> make -jN) is no longer working for me. I traced it back and the
>>> last kernel it worked in was -test5. It ceased working in -test6.
>> It works for me, and for sure it works for most others. Otherwise I
>> would have seen lot of complaints like yours.
>> I recall one similar post, and the person in question used a
>> homegrown script that caused the problems.
>
> Well, this explains why 2.6.x builds so much slower here than it did
> a few kernels ago.
>
> make -j3 improves things. but currently, make -j2 doesn't use both my
> cpus.
>
> no scripts, just make -j2 bzImage

Ah, yes... -j3 gets me 2 cpp/cc pairs as well.

What I see with -j3 is 1 parent make process and two make child processes.
With -j2 I see one parent and one child. Leaving -j off entirely gives the same
as -j2.

-j3:
root 25314 15539 0 Nov27 tty1 00:00:00 make -j3 bzImage
root 25545 25314 0 Nov27 tty1 00:00:00 make -f scripts/Makefile.build obj=arch/i386/kernel
root 25814 25314 0 Nov27 tty1 00:00:00 make -f scripts/Makefile.build obj=kernel


-j2:
root 26391 15539 0 Nov27 tty1 00:00:00 make -j2 bzImage
root 26582 26391 0 Nov27 tty1 00:00:00 make -f scripts/Makefile.build obj=init


--Adam