Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 24 Mar 2022 16:19:49 -0700
From:   Nick Desaulniers <ndesaulniers@google.com>
To:     Borislav Petkov <bp@alien8.de>
Cc:     Ingo Molnar <mingo@kernel.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Will Deacon <will@kernel.org>,
        Waiman Long <longman@redhat.com>,
        Boqun Feng <boqun.feng@gmail.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Andrew Morton <akpm@linux-foundation.org>,
        clang-built-linux <llvm@lists.linux.dev>,
        Sylvestre Ledru <sylvestre@mozilla.com>
Subject: Re: [GIT PULL] locking changes for v5.18
Message-ID: <Yjz8lQtnWimVeTp7@google.com>
References: <YjhdcJB4FaLfsoyO@gmail.com>
 <CAHk-=wjS6ptr5=JqmmyEb_qTjDz_68+S=h1o1bL1fEyArVOymA@mail.gmail.com>
 <CAHk-=whNBs07foho7=QxVbs_hnpDfPP3s2fwyXK-reNw67mU=g@mail.gmail.com>
 <YjwueBQoung7rFD+@gmail.com>
 <YjxFzlzfTF5Av3Zc@zn.tnic>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YjxFzlzfTF5Av3Zc@zn.tnic>
Precedence: bulk

On Thu, Mar 24, 2022 at 11:19:58AM +0100, Borislav Petkov wrote:
> On Thu, Mar 24, 2022 at 09:40:24AM +0100, Ingo Molnar wrote:
> > Only intermittently on my side - it only recently started working
> > reliably & it doubles the not inconsiderable test time :-/
> 
> True dat - last time I measured, clang builds take roughly double the
> time gcc builds with the same config do.

See also:
https://lore.kernel.org/lkml/CAHk-=whs8QZf3YnifdLv57+FhBi5_WeNTG1B-suOES=RcUSmQg@mail.gmail.com/

We do have to pay a penalty in that if these binaries can be built as
shared objects, the MUST be. (Actually, IDK if Debian has the same
policy as Fedora).

If you see libLLVM-*.so (and libclang-cpp.so) in your list of
`ldd $(which clang)`, or see multiple symbols from the dynamic linker
after running:

$ perf record -e cycles:pp --call-graph lbr make LLVM=1
$ perf report --no-children --sort=dso,cycles

Then dynamic linking is the main source of performance difference
between GCC and Clang on those distributions. So we're kind of fighting
with one hand behind our backs.

For Linux distros that have server costs, my guess is these policies
allow them to minimize their costs of serving prebuilt binaries, since
dynamic linking does allow for smaller binaries.

For short lived processes like most of the toolchain (and parts that
generally don't overlap, such as the serialization between compilation
THEN linkages), dynamic linking provides a ridiculous startup cost. CoW
gets us most of the benefits of dynamic linking in terms of shared
memory between processes. (I don't think ASLR hurts CoW in that way?)

On the flip side, I have been banging pots and pans around internally
trying to raise awareness of the issue (I'm but a peasant though, Mr.
bottom rung of the ladder); we do have budget finally allocated to focus
on compiler performance. Getting the checks written involves more red
tape and more effort than I expected.

I might start ignoring compiler bugs for a while, go off into the
wilderness, and not come back until I have statically linked images of
clang available to host on kernel.org though. There's a lot we could be
doing to hot rod the compiler to optimize for developer's (and CI's)
time.