From: Keith Owens <kaos@ocs.com.au>
To: Mike Touloumtzis <miket@bluemug.com>
Cc: Larry McVoy <lm@bitmover.com>, linux-kernel@vger.kernel.org,
        kbuild-devel@lists.sourceforge.net
Subject: Re: State of the new config & build system 
In-Reply-To: Your message of "Mon, 31 Dec 2001 20:03:59 -0800."
             <20011231200359.A22497@bluemug.com> 
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Date: Tue, 01 Jan 2002 19:26:59 +1100
Message-ID: <3680.1009873619@ocs3.intra.ocs.com.au>
Sender: linux-kernel-owner@vger.kernel.org

On Mon, 31 Dec 2001 20:03:59 -0800, 
Mike Touloumtzis <miket@bluemug.com> wrote:
>On Fri, Dec 28, 2001 at 12:57:58PM +1100, Keith Owens wrote:
>> 
>> Unlike the broken make dep, kbuild 2.5 extracts accurate dependencies
>> by using the -MD option of cpp and post processing the cpp list.  The
>> post processing code is slow because the current design requires every
>> compile to read a complete list of all the files, giving O(n^2)
>> effects.  Mark 2 of the core code will use a shared database with
>> concurrent update so post processing is limited to looking up just the
>> required files, instead of reading the complete list every time.
>
>Why not use '$(GCC) -c -Wp,-MD,foo.d foo.c' to generate the dependencies
>as a side effect of the regular compile step?  This enables you to skip
>the initial dependency preprocessing step entirely, and could lead to a
>speedup over even the current fastdep system.  You still have to massage
>the dependencies but you can do it based on the side-effect dependency
>output of the _previous_ build, to whatever degree that output exists.

That is exactly what kbuild 2.5 does.  The slowdown occurs when
massaging the -MD dependencies from absolute names to relative path
names.  To support separate source and object trees, renaming of trees,
different names in local and NFS mode etc., the massage code needs a
list of where all the files are before it can convert the absolute
dependencies produced by gcc.  Reading and indexing that file for every
compile is _slow_.

Larry McVoy has sent me the source code to an mmapped database (from
bitkeeper).  Using a shared mmapped database should speed the process
up considerably.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/