Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <CAHk-=wjmNOoX8iPtYsM8PVa+7DE1=5bv-XVe_egP0ZOiuT=7CQ@mail.gmail.com>
 <CAKwvOdmMF_v9TzBtFn2S1qSS_yCDO8D-u3WhBehUM7gzjcdjUQ@mail.gmail.com>
 <CAKwvOdk+V2dc31guafFM=N2ez4SrwCmah+mimUG3MzPMx_2efQ@mail.gmail.com>
 <CAKwvOdn3uXniVedgtpD8QFAd-hdVuVjGPa4-n0h64PTxT4XhWg@mail.gmail.com>
 <CAKwvOdm3D=dqKw=kx46PLaiqfHOZJL3QFKGc8kxqJqpwdFFWqw@mail.gmail.com> <CAKwvOdkp_P8BCtFuKqDrtC_=A89ZfDf66Yr3FL2e=ojwv4KaMA@mail.gmail.com>
In-Reply-To: <CAKwvOdkp_P8BCtFuKqDrtC_=A89ZfDf66Yr3FL2e=ojwv4KaMA@mail.gmail.com>
From:   Linus Torvalds <torvalds@linux-foundation.org>
Date:   Fri, 30 Apr 2021 18:22:28 -0700
Message-ID: <CAHk-=wi1yiBBr3b3RbCEte6-yzAApsZN5zRdr3xoW8Av9jOX=Q@mail.gmail.com>
Subject: Re: Very slow clang kernel config ..
To:     Nick Desaulniers <ndesaulniers@google.com>
Cc:     Masahiro Yamada <masahiroy@kernel.org>,
        Nathan Chancellor <nathan@kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        clang-built-linux <clang-built-linux@googlegroups.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Fri, Apr 30, 2021 at 5:25 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> Ah, no, sorry, these are the runtime link editor/loader. So probably
> spending quite some time resolving symbols in large binaries.

Yeah. Appended is the profile I see when I profile that "make
oldconfig", so about 45% of all time seems to be spent in just symbol
lookup and relocation.

And a fair amount of time just creating and tearing down that huge
executable (with a lot of copy-on-write overhead too), with the kernel
side of that being another 15%. The cost of that is likely also fairly
directly linked to all the dynamic linking costs, which brings in all
that data.

Just to compare, btw, this is the symbol lookup overhead for the gcc case:

   1.43%  ld-2.33.so             do_lookup_x
   0.96%  ld-2.33.so             _dl_relocate_object
   0.69%  ld-2.33.so             _dl_lookup_symbol_x

so it really does seem to be something very odd going on with the clang binary.

Maybe the Fedora binary is built some odd way, but it's likely just
the default clang build.

             Linus

----
  23.59%  ld-2.33.so          _dl_lookup_symbol_x
  11.41%  ld-2.33.so          _dl_relocate_object
   9.95%  ld-2.33.so          do_lookup_x
   4.00%  [kernel.vmlinux]    copy_page
   3.98%  [kernel.vmlinux]    next_uptodate_page
   3.05%  [kernel.vmlinux]    zap_pte_range
   1.81%  [kernel.vmlinux]    clear_page_rep
   1.68%  [kernel.vmlinux]    asm_exc_page_fault
   1.33%  ld-2.33.so          strcmp
   1.33%  ld-2.33.so          check_match
   0.92%  libLLVM-12.so       llvm::StringMapImpl::LookupBucketFor
   0.83%  [kernel.vmlinux]    rmqueue_bulk
   0.77%  conf                yylex
   0.75%  libc-2.33.so        __gconv_transform_utf8_internal
   0.74%  libc-2.33.so        _int_malloc
   0.69%  libc-2.33.so        __strlen_avx2
   0.62%  [kernel.vmlinux]    pagecache_get_page
   0.58%  [kernel.vmlinux]    page_remove_rmap
   0.56%  [kernel.vmlinux]    __handle_mm_fault
   0.54%  [kernel.vmlinux]    filemap_map_pages
   0.54%  libc-2.33.so        __strcmp_avx2
   0.54%  [kernel.vmlinux]    __free_one_page
   0.52%  [kernel.vmlinux]    release_pages