Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Mon, 11 Jan 2021 11:28:31 -0700
From:   Nathan Chancellor <natechancellor@gmail.com>
To:     Sedat Dilek <sedat.dilek@gmail.com>
Cc:     Bill Wendling <morbo@google.com>, Jonathan Corbet <corbet@lwn.net>,
        Masahiro Yamada <masahiroy@kernel.org>,
        linux-doc@vger.kernel.org, LKML <linux-kernel@vger.kernel.org>,
        Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
        Clang-Built-Linux ML <clang-built-linux@googlegroups.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Nick Desaulniers <ndesaulniers@google.com>,
        Sami Tolvanen <samitolvanen@google.com>
Subject: Re: [PATCH] pgo: add clang's Profile Guided Optimization
 infrastructure
Message-ID: <20210111182831.GB758023@ubuntu-m3-large-x86>
References: <20210111081821.3041587-1-morbo@google.com>
 <CA+icZUUUTEdRNbmORy4X=UqMHLMEfD186o2mh23v_mXGBM8Anw@mail.gmail.com>
 <CAGG=3QUHWkuF0Q-=tNO5PztqtxuKYgyQy_EbBLyL0xNQ7TjgoQ@mail.gmail.com>
 <CA+icZUX0dtfOFLqyG_NOHgFnbwnOKEBP7Fo+RxgjhzuL3FPA9g@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CA+icZUX0dtfOFLqyG_NOHgFnbwnOKEBP7Fo+RxgjhzuL3FPA9g@mail.gmail.com>
Precedence: bulk

On Mon, Jan 11, 2021 at 10:57:35AM +0100, Sedat Dilek wrote:
> On Mon, Jan 11, 2021 at 10:17 AM Bill Wendling <morbo@google.com> wrote:
> >
> > On Mon, Jan 11, 2021 at 12:39 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > >
> > > On Mon, Jan 11, 2021 at 9:18 AM 'Bill Wendling' via Clang Built Linux
> > > <clang-built-linux@googlegroups.com> wrote:
> > > >
> > > > From: Sami Tolvanen <samitolvanen@google.com>
> > > >
> > > > Enable the use of clang's Profile-Guided Optimization[1]. To generate a
> > > > profile, the kernel is instrumented with PGO counters, a representative
> > > > workload is run, and the raw profile data is collected from
> > > > /sys/kernel/debug/pgo/profraw.
> > > >
> > > > The raw profile data must be processed by clang's "llvm-profdata" tool before
> > > > it can be used during recompilation:
> > > >
> > > >   $ cp /sys/kernel/debug/pgo/profraw vmlinux.profraw
> > > >   $ llvm-profdata merge --output=vmlinux.profdata vmlinux.profraw
> > > >
> > > > Multiple raw profiles may be merged during this step.
> > > >
> > > > The data can be used either by the compiler if LTO isn't enabled:
> > > >
> > > >     ... -fprofile-use=vmlinux.profdata ...
> > > >
> > > > or by LLD if LTO is enabled:
> > > >
> > > >     ... -lto-cs-profile-file=vmlinux.profdata ...
> > > >
> > > > This initial submission is restricted to x86, as that's the platform we know
> > > > works. This restriction can be lifted once other platforms have been verified
> > > > to work with PGO.
> > > >
> > > > Note that this method of profiling the kernel is clang-native and isn't
> > > > compatible with clang's gcov support in kernel/gcov.
> > > >
> > > > [1] https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization
> > > >
> > >
> > > Hi Bill and Sami,
> > >
> > > I have seen the pull-request in the CBL issue tracker and had some
> > > questions in mind.
> > >
> > > Good you send this.
> > >
> > > First of all, I like to fetch any development stuff easily from a Git
> > > repository.
> >
> > The version in the pull-request in the CBL issue tracker is roughly
> > the same as this patch. (There are some changes, but they aren't
> > functionality changes.)
> >
> > > Can you offer this, please?
> > > What is the base for your work?
> > > I hope this is (fresh released) Linux v5.11-rc3.
> > >
> > This patch (and the PR on the CBL issue tracker) are from top-of-tree Linux.
> >
> > > I myself had some experiences with a PGO + ThinLTO optimized LLVM
> > > toolchain built with the help of tc-build.
> > > Here it takes very long to build it.
> > >
> > > This means I have some profile-data archived.
> > > Can I use it?
> > >
> > LLVM is more tolerant of "stale" profile data than gcov, so it's
> > possible that your archived profile data would still work, but I can't
> > guarantee that it will be better than using new profile data.
> >
> > > Is an own PGO + ThinLTO optimized LLVM toolchain pre-requirement for
> > > this or not?
> > > That is one of my important questions.
> > >
> > Do you mean that the LLVM tools (clang, llc, etc.) are compiled with
> > PGO + ThinLTO?
> >
> 
> Yes.
> 
> - Sedat -

No, having an optimized LLVM toolchain is not a requirement of this
patchset. It will make compiling the kernel faster but it does nothing
more than that.

Cheers,
Nathan