Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220616104541.16289-1-jslaby@suse.cz> <CA+icZUW8O-HUSpw-656o6YZOiR2ZiCXjxsJwm2kctT6DHrs=4g@mail.gmail.com>
 <CA+icZUV6bM2_jxyROK5B4XRid6fv8oX6YYNEdHUX8e_1OAdQYA@mail.gmail.com>
 <CA+icZUUSTcrJqZB-gwNYt5objVg1J5+Ous6_hof0_A6eVCM-Kg@mail.gmail.com> <CA+icZUXDGdPrPKUnevt99LUpTRPe=ogqF33uHQRYrQ6Kh-iTAw@mail.gmail.com>
In-Reply-To: <CA+icZUXDGdPrPKUnevt99LUpTRPe=ogqF33uHQRYrQ6Kh-iTAw@mail.gmail.com>
From:   Masahiro Yamada <masahiroy@kernel.org>
Date:   Sat, 18 Jun 2022 00:21:28 +0900
Message-ID: <CAK7LNATHY88PbJ_=A6g7v8NMQnBcQ9g06k1+SCe+NM+xd5dLwA@mail.gmail.com>
Subject: Re: [PATCH] kbuild: pass jobserver to cmd_ld_vmlinux.o
To:     Sedat Dilek <sedat.dilek@gmail.com>
Cc:     Jiri Slaby <jslaby@suse.cz>,
        Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Michal Marek <michal.lkml@markovi.net>,
        Nick Desaulniers <ndesaulniers@google.com>,
        Nathan Chancellor <nathan@kernel.org>,
        Sami Tolvanen <samitolvanen@google.com>,
        clang-built-linux <llvm@lists.linux.dev>,
        Fangrui Song <maskray@google.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

(+LLVM list, Fangrui Song)

On Fri, Jun 17, 2022 at 7:41 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Fri, Jun 17, 2022 at 12:35 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > On Fri, Jun 17, 2022 at 12:53 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > >
> > > On Thu, Jun 16, 2022 at 4:09 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > > >
> > > > On Thu, Jun 16, 2022 at 12:45 PM Jiri Slaby <jslaby@suse.cz> wrote:
> > > > >
> > > > > Until the link-vmlinux.sh split (cf. the commit below), the linker was
> > > > > run with jobserver set in MAKEFLAGS. After the split, the command in
> > > > > Makefile.vmlinux_o is not prefixed by "+" anymore, so this information
> > > > > is lost.
> > > > >
> > > > > Restore it as linkers working in parallel (esp. the LTO ones) make a use
> > > > > of i

Hi Jiri,

Please let me clarify first.

Here, is it OK to assume you are talking about Clang LTO
instead of GCC LTO because the latter is not upstreamed ?


I tested this patch but I did not see any performance change for Clang LTO.


[1] CONFIG_CLANG_LTO_FULL

   lld always runs sequential.
   It never runs in parallel even if you pass -j option to Make


[2] CONFIG_CLANG_LTO_THIN

   lld always runs in parallel even if you do not pass -j option

   In my machine, lld always allocated 12 threads.
   This is irrespective of the Make parallelisms.


One more thing, if a program wants to participate in
Make's jobserver, it must parse MAKEFLAGS, and extract
file descriptors to be used to communicate to the jobserver.

As a code example in the kernel tree,
scripts/jobserver-exec parses "MAKEFLAGS" and "--jobserver".


I grepped the lld source code, but it does not contain
"MAKEFLAGS" or "jobserver".


masahiro@oscar:~/ref/lld$ git remote  show origin
* remote origin
  Fetch URL: https://github.com/llvm-mirror/lld.git
  Push  URL: https://github.com/llvm-mirror/lld.git
  HEAD branch: master
  Remote branches:
    master     tracked
    release_36 tracked
    release_37 tracked
    release_38 tracked
    release_39 tracked
    release_40 tracked
    release_50 tracked
    release_60 tracked
    release_70 tracked
    release_80 tracked
    release_90 tracked
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (up to date)
masahiro@oscar:~/ref/lld$ git grep MAKEFLAGS
masahiro@oscar:~/ref/lld$ git grep jobserver


So, in my research, LLD does not seem to support the jobserver.


If you are talking about GCC LTO, yes, the code
tries to parse "--jobserver-auth=" from the MAKEFLAGS
environment variable.  [1]

[1]:  https://github.com/gcc-mirror/gcc/blob/releases/gcc-12.1.0/gcc/lto-wrapper.cc#L1341


But, as you may know, GCC LTO works in a different way,
at least, we cannot do it before modpost.


-- 
Best Regards
Masahiro Yamada