Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20200527103236.148700-1-elver@google.com> <CAK8P3a1MFgRxm6=+9WZKNzN+Nc5fhrDso6orSNQaaa-0yqygYA@mail.gmail.com>
 <CA+icZUWtzu0ONUSy0E27Mq1BrdO79qNaY3Si-PDhHZyF8M4S5g@mail.gmail.com>
 <CAK8P3a04=mVQgSrvDhpVxQj50JEFDn_xMhYrvjmUnLYTWH3QXQ@mail.gmail.com>
 <CA+icZUXVSTxDYJwXLyAwZd91cjMPcPRpeAR72JKqkqa-wRNnWg@mail.gmail.com>
 <CAK8P3a3i0kPf8dRg7Ko-33hsb+LkP=P05uz2tGvg5B43O-hFvg@mail.gmail.com> <CA+icZUWr5xDz5ujBfsXjnDdiBuopaGE6xO5LJQP9_y=YoROb+Q@mail.gmail.com>
In-Reply-To: <CA+icZUWr5xDz5ujBfsXjnDdiBuopaGE6xO5LJQP9_y=YoROb+Q@mail.gmail.com>
From:   Marco Elver <elver@google.com>
Date:   Wed, 27 May 2020 15:30:08 +0200
Message-ID: <CANpmjNOtKQAB_3t1G5Da-J1k-9Dk6eQKP+xNozRbmHJXZqXGFw@mail.gmail.com>
Subject: Re: [PATCH -tip] compiler_types.h: Optimize __unqual_scalar_typeof
 compilation time
To:     sedat.dilek@gmail.com
Cc:     Arnd Bergmann <arnd@arndb.de>, Will Deacon <will@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Borislav Petkov <bp@alien8.de>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@kernel.org>,
        clang-built-linux <clang-built-linux@googlegroups.com>,
        "Paul E. McKenney" <paulmck@kernel.org>,
        Dmitry Vyukov <dvyukov@google.com>,
        Alexander Potapenko <glider@google.com>,
        Andrey Konovalov <andreyknvl@google.com>,
        kasan-dev <kasan-dev@googlegroups.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Nick Desaulniers <ndesaulniers@google.com>,
        Stephen Rothwell <sfr@canb.auug.org.au>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, 27 May 2020 at 15:11, Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Wed, May 27, 2020 at 2:50 PM Arnd Bergmann <arnd@arndb.de> wrote:
> >
> > On Wed, May 27, 2020 at 2:35 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > > On Wed, May 27, 2020 at 2:31 PM Arnd Bergmann <arnd@arndb.de> wrote:
> > > > On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > > > > On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <arnd@arndb.de> wrote:
> > > > > > On Wed, May 27, 2020 at 12:33 PM Marco Elver <elver@google.com> wrote:
> > > > > >
> > > > > > This gives us back 80% of the performance drop on clang, and 50%
> > > > > > of the drop I saw with gcc, compared to current mainline.
> > > > > >
> > > > > > Tested-by: Arnd Bergmann <arnd@arndb.de>
> > > > > >
> > > > >
> > > > > Hi Arnd,
> > > > >
> > > > > with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
> > > >
> > > > I meant v5.7.
> > > >
> > > > > I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
> > > > >
> > > > > Is there a speedup benefit also for Linux v5.7?
> > > > > Which patches do I need?
> > > >
> > > > v5.7-rc is the baseline and is the fastest I currently see. On certain files,
> > > > I saw an intermittent 10x slowdown that was already fixed earlier, now
> > > > linux-next
> > > > is more like 2x slowdown for me and 1.2x with this patch on top, so we're
> > > > almost back to the speed of linux-5.7.
> > > >
> > >
> > > Which clang version did you use - and have you set KCSAN kconfigs -
> > > AFAICS this needs clang-11?
> >
> > I'm currently using clang-11, but I see the same problem with older
> > versions, and both with and without KCSAN enabled. I think the issue
> > is mostly the deep nesting of macros that leads to code bloat.
> >
>
> Thanks.
>
> With clang-10:
>
> $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
>  BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> +HAVE_ARCH_KCSAN y

Clang 10 doesn't support KCSAN (HAVE_KCSAN_COMPILER unset).

> With clang-11:
>
> $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
>  BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
>  CLANG_VERSION 100001 -> 110000
> +CC_HAS_ASM_INLINE y
> +HAVE_ARCH_KCSAN y
> +HAVE_KCSAN_COMPILER y
> +KCSAN n
>
> Which KCSAN kconfigs did you enable?

To clarify: as said in [1], KCSAN (or any other instrumentation) is no
longer relevant to the issue here, and the compile-time regression is
observable with most configs. The problem is due to pre-processing and
parsing, which came about due to new READ_ONCE() and the
__unqual_scalar_typeof() macro (which this patch optimizes).

KCSAN and new ONCEs got tangled up because we first attempted to
annotate {READ,WRITE}_ONCE() with data_race(), but that turned out to
have all kinds of other issues (explanation in [2]). So we decided to
drop all the KCSAN-specific bits from ONCE, and require KCSAN to be
Clang 11. Those fixes were applied to the first version of new
{READ,WRITE}_ONCE() in -tip, which actually restored the new ONCEs to
the pre-KCSAN version (now that KCSAN can deal with them without
annotations).

Hope this makes more sense now.

[1] https://lore.kernel.org/lkml/CANpmjNOUdr2UG3F45=JaDa0zLwJ5ukPc1MMKujQtmYSmQnjcXg@mail.gmail.com/
[2] https://lore.kernel.org/lkml/20200521142047.169334-1-elver@google.com/

Thanks,
-- Marco