Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <CAHk-=wjmNOoX8iPtYsM8PVa+7DE1=5bv-XVe_egP0ZOiuT=7CQ@mail.gmail.com>
In-Reply-To: <CAHk-=wjmNOoX8iPtYsM8PVa+7DE1=5bv-XVe_egP0ZOiuT=7CQ@mail.gmail.com>
From:   Nick Desaulniers <ndesaulniers@google.com>
Date:   Thu, 29 Apr 2021 17:19:47 -0700
Message-ID: <CAKwvOdmMF_v9TzBtFn2S1qSS_yCDO8D-u3WhBehUM7gzjcdjUQ@mail.gmail.com>
Subject: Re: Very slow clang kernel config ..
To:     Linus Torvalds <torvalds@linux-foundation.org>
Cc:     Masahiro Yamada <masahiroy@kernel.org>,
        Nathan Chancellor <nathan@kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        clang-built-linux <clang-built-linux@googlegroups.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, Apr 29, 2021 at 2:53 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> I haven't looked into why this is so slow with clang, but it really is
> painfully slow:
>
>    time make CC=clang allmodconfig
>    real 0m2.667s
>
> vs the gcc case:
>
>     time make CC=gcc allmodconfig
>     real 0m0.903s

Hmmm...I seem to only be able to reproduce such a drastic difference
between the two if I:
1. make clean
2. time make CC=<either> allmodconfig
3. time make CC=<the other> allmodconfig

without doing another `make clean` in between 2 and 3; and regardless
which toolchain I use first vs second.  Otherwise I pretty
consistently get 1.49-1.62s for clang, 1.28-1.4s for gcc; that's a
build of clang with assertions enabled, too.

Can you confirm your observations with `make clean` between runs? Can
you provide info about your clang build such as the version string,
and whether this was built locally perhaps?

>
> Yeah, yeah, three seconds may sound like "not a lot of time, but
> considering that the subsequent full build (which for me is often
> empty) doesn't take all that much longer, that config time clang waste
> is actually quite noticeable.
>
> I actually don't do allmodconfig builds with clang, but I do my
> default kernel builds with it:

:)

>
>     time make oldconfig
>     real 0m2.748s
>
>     time sh -c "make -j128 > ../makes"
>     real 0m3.546s
>
> so that "make oldconfig" really is almost as slow as the whole
> "confirm build is done" thing. Its' quite noticeable in my workflow.
>
> The gcc config isn't super-fast either, but there's a big 3x
> difference, so the clang case really is doing something extra wrong.
>
> I've not actually looked into _why_. Except I do see that "clang" gets
> invoked with small (empty?) test files several times, probably to
> check for command line flags being valid.

There's probably more we can be doing to speed up the flag checking
case; Nathan had a good idea about using -fsyntax-only to stop the
compilation pipeline after flags have been validated.  I think we
should run some testing on that to see if it makes a measurable
impact; I'd imagine that being beneficial to both compilers.

>
> Sending this to relevant parties in the hope that somebody goes "Yeah,
> that's silly" and fixes it.
>
> This is on my F34 machine:
>
>      clang version 12.0.0 (Fedora 12.0.0-0.3.rc1.fc34)
>
> in case it matters (but I don't see why it should).
>
> Many many moons ago the promise for clang was faster build speeds.
> That didn't turn out to be true, but can we please at least try to
> make them not painfully much slower?

Ack. Forwarded that request directly up the chain of command. ;)

In the interest of build speed, have you tried LLD yet? `make LLVM=1
...` or `make LD=ld.lld ...` should do it; you'll find it's much
faster than the competition, especially when there's a large number of
cores on the host. Not going to help with the allmodconfig
configuration, but would definitely help incremental rebuilds.
-- 
Thanks,
~Nick Desaulniers