If the compiler supports C11's _Generic, use it to speed up compilation
times of __unqual_scalar_typeof(). GCC version 4.9 or later and
all supported versions of Clang support the feature (the oldest
supported compiler that doesn't support _Generic is GCC 4.8, for which
we use the slower alternative).
The non-_Generic variant relies on multiple expansions of
__pick_integer_type -> __pick_scalar_type -> __builtin_choose_expr,
which increases pre-processed code size, and can cause compile times to
increase in files with numerous expansions of READ_ONCE(), or other
users of __unqual_scalar_typeof().
Summary of compile-time benchmarking done by Arnd Bergmann [1]:
<baseline normalized time> clang-11 gcc-9
this patch 0.78 0.91
ideal 0.76 0.86
[1] https://lkml.kernel.org/r/CAK8P3a3UYQeXhiufUevz=rwe09WM_vSTCd9W+KvJHJcOeQyWVA@mail.gmail.com
Further compile-testing done with:
gcc 4.8, 4.9, 5.5, 6.4, 7.5, 8.4;
clang 9, 10.
Reported-by: Arnd Bergmann <[email protected]>
Signed-off-by: Marco Elver <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Nick Desaulniers <[email protected]>
Cc: Paul E. McKenney <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Stephen Rothwell <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Will Deacon <[email protected]>
Link: https://lkml.kernel.org/r/CAK8P3a0RJtbVi1JMsfik=jkHCNFv+DJn_FeDg-YLW+ueQW3tNg@mail.gmail.com
---
Same version as in:
https://lkml.kernel.org/r/[email protected]
---
include/linux/compiler_types.h | 22 +++++++++++++++++++++-
1 file changed, 21 insertions(+), 1 deletion(-)
diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index 5faf68eae204..a529fa263906 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -245,7 +245,9 @@ struct ftrace_likely_data {
/*
* __unqual_scalar_typeof(x) - Declare an unqualified scalar type, leaving
* non-scalar types unchanged.
- *
+ */
+#if defined(CONFIG_CC_IS_GCC) && CONFIG_GCC_VERSION < 40900
+/*
* We build this out of a couple of helper macros in a vain attempt to
* help you keep your lunch down while reading it.
*/
@@ -267,6 +269,24 @@ struct ftrace_likely_data {
__pick_integer_type(x, int, \
__pick_integer_type(x, long, \
__pick_integer_type(x, long long, x))))))
+#else
+/*
+ * If supported, prefer C11 _Generic for better compile-times. As above, 'char'
+ * is not type-compatible with 'signed char', and we define a separate case.
+ */
+#define __scalar_type_to_expr_cases(type) \
+ type: (type)0, unsigned type: (unsigned type)0
+
+#define __unqual_scalar_typeof(x) typeof( \
+ _Generic((x), \
+ __scalar_type_to_expr_cases(char), \
+ signed char: (signed char)0, \
+ __scalar_type_to_expr_cases(short), \
+ __scalar_type_to_expr_cases(int), \
+ __scalar_type_to_expr_cases(long), \
+ __scalar_type_to_expr_cases(long long), \
+ default: (x)))
+#endif
/* Is this type a native word size -- useful for atomic operations */
#define __native_word(t) \
--
2.27.0.rc0.183.gde8f92d652-goog
On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <[email protected]> wrote:
>
> On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
> >
> > If the compiler supports C11's _Generic, use it to speed up compilation
> > times of __unqual_scalar_typeof(). GCC version 4.9 or later and
> > all supported versions of Clang support the feature (the oldest
> > supported compiler that doesn't support _Generic is GCC 4.8, for which
> > we use the slower alternative).
> >
> > The non-_Generic variant relies on multiple expansions of
> > __pick_integer_type -> __pick_scalar_type -> __builtin_choose_expr,
> > which increases pre-processed code size, and can cause compile times to
> > increase in files with numerous expansions of READ_ONCE(), or other
> > users of __unqual_scalar_typeof().
> >
> > Summary of compile-time benchmarking done by Arnd Bergmann [1]:
> >
> > <baseline normalized time> clang-11 gcc-9
> > this patch 0.78 0.91
> > ideal 0.76 0.86
> >
> > [1] https://lkml.kernel.org/r/CAK8P3a3UYQeXhiufUevz=rwe09WM_vSTCd9W+KvJHJcOeQyWVA@mail.gmail.com
> >
> > Further compile-testing done with:
> > gcc 4.8, 4.9, 5.5, 6.4, 7.5, 8.4;
> > clang 9, 10.
> >
> > Reported-by: Arnd Bergmann <[email protected]>
> > Signed-off-by: Marco Elver <[email protected]>
>
> This gives us back 80% of the performance drop on clang, and 50%
> of the drop I saw with gcc, compared to current mainline.
>
> Tested-by: Arnd Bergmann <[email protected]>
>
Hi Arnd,
with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
Is there a speedup benefit also for Linux v5.7?
Which patches do I need?
Thanks.
Regards,
- Sedat -
On Wed, May 27, 2020 at 2:31 PM Arnd Bergmann <[email protected]> wrote:
>
> On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <[email protected]> wrote:
> > On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <[email protected]> wrote:
> > > On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
> > >
> > > This gives us back 80% of the performance drop on clang, and 50%
> > > of the drop I saw with gcc, compared to current mainline.
> > >
> > > Tested-by: Arnd Bergmann <[email protected]>
> > >
> >
> > Hi Arnd,
> >
> > with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
>
> I meant v5.7.
>
> > I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
> >
> > Is there a speedup benefit also for Linux v5.7?
> > Which patches do I need?
>
> v5.7-rc is the baseline and is the fastest I currently see. On certain files,
> I saw an intermittent 10x slowdown that was already fixed earlier, now
> linux-next
> is more like 2x slowdown for me and 1.2x with this patch on top, so we're
> almost back to the speed of linux-5.7.
>
Which clang version did you use - and have you set KCSAN kconfigs -
AFAICS this needs clang-11?
- Sedat -
On Wed, May 27, 2020 at 2:35 PM Sedat Dilek <[email protected]> wrote:
> On Wed, May 27, 2020 at 2:31 PM Arnd Bergmann <[email protected]> wrote:
> > On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <[email protected]> wrote:
> > > On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <[email protected]> wrote:
> > > > On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
> > > >
> > > > This gives us back 80% of the performance drop on clang, and 50%
> > > > of the drop I saw with gcc, compared to current mainline.
> > > >
> > > > Tested-by: Arnd Bergmann <[email protected]>
> > > >
> > >
> > > Hi Arnd,
> > >
> > > with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
> >
> > I meant v5.7.
> >
> > > I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
> > >
> > > Is there a speedup benefit also for Linux v5.7?
> > > Which patches do I need?
> >
> > v5.7-rc is the baseline and is the fastest I currently see. On certain files,
> > I saw an intermittent 10x slowdown that was already fixed earlier, now
> > linux-next
> > is more like 2x slowdown for me and 1.2x with this patch on top, so we're
> > almost back to the speed of linux-5.7.
> >
>
> Which clang version did you use - and have you set KCSAN kconfigs -
> AFAICS this needs clang-11?
I'm currently using clang-11, but I see the same problem with older
versions, and both with and without KCSAN enabled. I think the issue
is mostly the deep nesting of macros that leads to code bloat.
Arnd
On Wed, May 27, 2020 at 2:50 PM Arnd Bergmann <[email protected]> wrote:
>
> On Wed, May 27, 2020 at 2:35 PM Sedat Dilek <[email protected]> wrote:
> > On Wed, May 27, 2020 at 2:31 PM Arnd Bergmann <[email protected]> wrote:
> > > On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <[email protected]> wrote:
> > > > On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <[email protected]> wrote:
> > > > > On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
> > > > >
> > > > > This gives us back 80% of the performance drop on clang, and 50%
> > > > > of the drop I saw with gcc, compared to current mainline.
> > > > >
> > > > > Tested-by: Arnd Bergmann <[email protected]>
> > > > >
> > > >
> > > > Hi Arnd,
> > > >
> > > > with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
> > >
> > > I meant v5.7.
> > >
> > > > I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
> > > >
> > > > Is there a speedup benefit also for Linux v5.7?
> > > > Which patches do I need?
> > >
> > > v5.7-rc is the baseline and is the fastest I currently see. On certain files,
> > > I saw an intermittent 10x slowdown that was already fixed earlier, now
> > > linux-next
> > > is more like 2x slowdown for me and 1.2x with this patch on top, so we're
> > > almost back to the speed of linux-5.7.
> > >
> >
> > Which clang version did you use - and have you set KCSAN kconfigs -
> > AFAICS this needs clang-11?
>
> I'm currently using clang-11, but I see the same problem with older
> versions, and both with and without KCSAN enabled. I think the issue
> is mostly the deep nesting of macros that leads to code bloat.
>
Thanks.
With clang-10:
$ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
+HAVE_ARCH_KCSAN y
With clang-11:
$ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
CLANG_VERSION 100001 -> 110000
+CC_HAS_ASM_INLINE y
+HAVE_ARCH_KCSAN y
+HAVE_KCSAN_COMPILER y
+KCSAN n
Which KCSAN kconfigs did you enable?
- Sedat -
On Wed, 27 May 2020 at 15:11, Sedat Dilek <[email protected]> wrote:
>
> On Wed, May 27, 2020 at 2:50 PM Arnd Bergmann <[email protected]> wrote:
> >
> > On Wed, May 27, 2020 at 2:35 PM Sedat Dilek <[email protected]> wrote:
> > > On Wed, May 27, 2020 at 2:31 PM Arnd Bergmann <[email protected]> wrote:
> > > > On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <[email protected]> wrote:
> > > > > On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <[email protected]> wrote:
> > > > > > On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
> > > > > >
> > > > > > This gives us back 80% of the performance drop on clang, and 50%
> > > > > > of the drop I saw with gcc, compared to current mainline.
> > > > > >
> > > > > > Tested-by: Arnd Bergmann <[email protected]>
> > > > > >
> > > > >
> > > > > Hi Arnd,
> > > > >
> > > > > with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
> > > >
> > > > I meant v5.7.
> > > >
> > > > > I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
> > > > >
> > > > > Is there a speedup benefit also for Linux v5.7?
> > > > > Which patches do I need?
> > > >
> > > > v5.7-rc is the baseline and is the fastest I currently see. On certain files,
> > > > I saw an intermittent 10x slowdown that was already fixed earlier, now
> > > > linux-next
> > > > is more like 2x slowdown for me and 1.2x with this patch on top, so we're
> > > > almost back to the speed of linux-5.7.
> > > >
> > >
> > > Which clang version did you use - and have you set KCSAN kconfigs -
> > > AFAICS this needs clang-11?
> >
> > I'm currently using clang-11, but I see the same problem with older
> > versions, and both with and without KCSAN enabled. I think the issue
> > is mostly the deep nesting of macros that leads to code bloat.
> >
>
> Thanks.
>
> With clang-10:
>
> $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> +HAVE_ARCH_KCSAN y
Clang 10 doesn't support KCSAN (HAVE_KCSAN_COMPILER unset).
> With clang-11:
>
> $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> CLANG_VERSION 100001 -> 110000
> +CC_HAS_ASM_INLINE y
> +HAVE_ARCH_KCSAN y
> +HAVE_KCSAN_COMPILER y
> +KCSAN n
>
> Which KCSAN kconfigs did you enable?
To clarify: as said in [1], KCSAN (or any other instrumentation) is no
longer relevant to the issue here, and the compile-time regression is
observable with most configs. The problem is due to pre-processing and
parsing, which came about due to new READ_ONCE() and the
__unqual_scalar_typeof() macro (which this patch optimizes).
KCSAN and new ONCEs got tangled up because we first attempted to
annotate {READ,WRITE}_ONCE() with data_race(), but that turned out to
have all kinds of other issues (explanation in [2]). So we decided to
drop all the KCSAN-specific bits from ONCE, and require KCSAN to be
Clang 11. Those fixes were applied to the first version of new
{READ,WRITE}_ONCE() in -tip, which actually restored the new ONCEs to
the pre-KCSAN version (now that KCSAN can deal with them without
annotations).
Hope this makes more sense now.
[1] https://lore.kernel.org/lkml/CANpmjNOUdr2UG3F45=JaDa0zLwJ5ukPc1MMKujQtmYSmQnjcXg@mail.gmail.com/
[2] https://lore.kernel.org/lkml/[email protected]/
Thanks,
-- Marco
On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
>
> If the compiler supports C11's _Generic, use it to speed up compilation
> times of __unqual_scalar_typeof(). GCC version 4.9 or later and
> all supported versions of Clang support the feature (the oldest
> supported compiler that doesn't support _Generic is GCC 4.8, for which
> we use the slower alternative).
>
> The non-_Generic variant relies on multiple expansions of
> __pick_integer_type -> __pick_scalar_type -> __builtin_choose_expr,
> which increases pre-processed code size, and can cause compile times to
> increase in files with numerous expansions of READ_ONCE(), or other
> users of __unqual_scalar_typeof().
>
> Summary of compile-time benchmarking done by Arnd Bergmann [1]:
>
> <baseline normalized time> clang-11 gcc-9
> this patch 0.78 0.91
> ideal 0.76 0.86
>
> [1] https://lkml.kernel.org/r/CAK8P3a3UYQeXhiufUevz=rwe09WM_vSTCd9W+KvJHJcOeQyWVA@mail.gmail.com
>
> Further compile-testing done with:
> gcc 4.8, 4.9, 5.5, 6.4, 7.5, 8.4;
> clang 9, 10.
>
> Reported-by: Arnd Bergmann <[email protected]>
> Signed-off-by: Marco Elver <[email protected]>
This gives us back 80% of the performance drop on clang, and 50%
of the drop I saw with gcc, compared to current mainline.
Tested-by: Arnd Bergmann <[email protected]>
On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <[email protected]> wrote:
> On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <[email protected]> wrote:
> > On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
> >
> > This gives us back 80% of the performance drop on clang, and 50%
> > of the drop I saw with gcc, compared to current mainline.
> >
> > Tested-by: Arnd Bergmann <[email protected]>
> >
>
> Hi Arnd,
>
> with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
I meant v5.7.
> I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
>
> Is there a speedup benefit also for Linux v5.7?
> Which patches do I need?
v5.7-rc is the baseline and is the fastest I currently see. On certain files,
I saw an intermittent 10x slowdown that was already fixed earlier, now
linux-next
is more like 2x slowdown for me and 1.2x with this patch on top, so we're
almost back to the speed of linux-5.7.
Arnd
On Wed, May 27, 2020 at 3:30 PM Marco Elver <[email protected]> wrote:
>
> On Wed, 27 May 2020 at 15:11, Sedat Dilek <[email protected]> wrote:
> >
> > On Wed, May 27, 2020 at 2:50 PM Arnd Bergmann <[email protected]> wrote:
> > >
> > > On Wed, May 27, 2020 at 2:35 PM Sedat Dilek <[email protected]> wrote:
> > > > On Wed, May 27, 2020 at 2:31 PM Arnd Bergmann <[email protected]> wrote:
> > > > > On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <[email protected]> wrote:
> > > > > > On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <[email protected]> wrote:
> > > > > > > On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
> > > > > > >
> > > > > > > This gives us back 80% of the performance drop on clang, and 50%
> > > > > > > of the drop I saw with gcc, compared to current mainline.
> > > > > > >
> > > > > > > Tested-by: Arnd Bergmann <[email protected]>
> > > > > > >
> > > > > >
> > > > > > Hi Arnd,
> > > > > >
> > > > > > with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
> > > > >
> > > > > I meant v5.7.
> > > > >
> > > > > > I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
> > > > > >
> > > > > > Is there a speedup benefit also for Linux v5.7?
> > > > > > Which patches do I need?
> > > > >
> > > > > v5.7-rc is the baseline and is the fastest I currently see. On certain files,
> > > > > I saw an intermittent 10x slowdown that was already fixed earlier, now
> > > > > linux-next
> > > > > is more like 2x slowdown for me and 1.2x with this patch on top, so we're
> > > > > almost back to the speed of linux-5.7.
> > > > >
> > > >
> > > > Which clang version did you use - and have you set KCSAN kconfigs -
> > > > AFAICS this needs clang-11?
> > >
> > > I'm currently using clang-11, but I see the same problem with older
> > > versions, and both with and without KCSAN enabled. I think the issue
> > > is mostly the deep nesting of macros that leads to code bloat.
> > >
> >
> > Thanks.
> >
> > With clang-10:
> >
> > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > +HAVE_ARCH_KCSAN y
>
> Clang 10 doesn't support KCSAN (HAVE_KCSAN_COMPILER unset).
>
> > With clang-11:
> >
> > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > CLANG_VERSION 100001 -> 110000
> > +CC_HAS_ASM_INLINE y
> > +HAVE_ARCH_KCSAN y
> > +HAVE_KCSAN_COMPILER y
> > +KCSAN n
> >
> > Which KCSAN kconfigs did you enable?
>
> To clarify: as said in [1], KCSAN (or any other instrumentation) is no
> longer relevant to the issue here, and the compile-time regression is
> observable with most configs. The problem is due to pre-processing and
> parsing, which came about due to new READ_ONCE() and the
> __unqual_scalar_typeof() macro (which this patch optimizes).
>
> KCSAN and new ONCEs got tangled up because we first attempted to
> annotate {READ,WRITE}_ONCE() with data_race(), but that turned out to
> have all kinds of other issues (explanation in [2]). So we decided to
> drop all the KCSAN-specific bits from ONCE, and require KCSAN to be
> Clang 11. Those fixes were applied to the first version of new
> {READ,WRITE}_ONCE() in -tip, which actually restored the new ONCEs to
> the pre-KCSAN version (now that KCSAN can deal with them without
> annotations).
>
> Hope this makes more sense now.
>
> [1] https://lore.kernel.org/lkml/CANpmjNOUdr2UG3F45=JaDa0zLwJ5ukPc1MMKujQtmYSmQnjcXg@mail.gmail.com/
> [2] https://lore.kernel.org/lkml/[email protected]/
>
Thanks, Marco.
I pulled tip.git#locking/kcsan on top of Linux v5.7-rc7 and applied this patch.
Just wanted to try KCSAN for the first time and it will also be my
first building with clang-11.
That's why I asked.
- Sedat -
On Wed, 27 May 2020 at 15:37, Sedat Dilek <[email protected]> wrote:
>
> On Wed, May 27, 2020 at 3:30 PM Marco Elver <[email protected]> wrote:
> >
> > On Wed, 27 May 2020 at 15:11, Sedat Dilek <[email protected]> wrote:
> > >
> > > On Wed, May 27, 2020 at 2:50 PM Arnd Bergmann <[email protected]> wrote:
> > > >
> > > > On Wed, May 27, 2020 at 2:35 PM Sedat Dilek <[email protected]> wrote:
> > > > > On Wed, May 27, 2020 at 2:31 PM Arnd Bergmann <[email protected]> wrote:
> > > > > > On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <[email protected]> wrote:
> > > > > > > On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <[email protected]> wrote:
> > > > > > > > On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
> > > > > > > >
> > > > > > > > This gives us back 80% of the performance drop on clang, and 50%
> > > > > > > > of the drop I saw with gcc, compared to current mainline.
> > > > > > > >
> > > > > > > > Tested-by: Arnd Bergmann <[email protected]>
> > > > > > > >
> > > > > > >
> > > > > > > Hi Arnd,
> > > > > > >
> > > > > > > with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
> > > > > >
> > > > > > I meant v5.7.
> > > > > >
> > > > > > > I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
> > > > > > >
> > > > > > > Is there a speedup benefit also for Linux v5.7?
> > > > > > > Which patches do I need?
> > > > > >
> > > > > > v5.7-rc is the baseline and is the fastest I currently see. On certain files,
> > > > > > I saw an intermittent 10x slowdown that was already fixed earlier, now
> > > > > > linux-next
> > > > > > is more like 2x slowdown for me and 1.2x with this patch on top, so we're
> > > > > > almost back to the speed of linux-5.7.
> > > > > >
> > > > >
> > > > > Which clang version did you use - and have you set KCSAN kconfigs -
> > > > > AFAICS this needs clang-11?
> > > >
> > > > I'm currently using clang-11, but I see the same problem with older
> > > > versions, and both with and without KCSAN enabled. I think the issue
> > > > is mostly the deep nesting of macros that leads to code bloat.
> > > >
> > >
> > > Thanks.
> > >
> > > With clang-10:
> > >
> > > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > > +HAVE_ARCH_KCSAN y
> >
> > Clang 10 doesn't support KCSAN (HAVE_KCSAN_COMPILER unset).
> >
> > > With clang-11:
> > >
> > > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > > CLANG_VERSION 100001 -> 110000
> > > +CC_HAS_ASM_INLINE y
> > > +HAVE_ARCH_KCSAN y
> > > +HAVE_KCSAN_COMPILER y
> > > +KCSAN n
> > >
> > > Which KCSAN kconfigs did you enable?
> >
> > To clarify: as said in [1], KCSAN (or any other instrumentation) is no
> > longer relevant to the issue here, and the compile-time regression is
> > observable with most configs. The problem is due to pre-processing and
> > parsing, which came about due to new READ_ONCE() and the
> > __unqual_scalar_typeof() macro (which this patch optimizes).
> >
> > KCSAN and new ONCEs got tangled up because we first attempted to
> > annotate {READ,WRITE}_ONCE() with data_race(), but that turned out to
> > have all kinds of other issues (explanation in [2]). So we decided to
> > drop all the KCSAN-specific bits from ONCE, and require KCSAN to be
> > Clang 11. Those fixes were applied to the first version of new
> > {READ,WRITE}_ONCE() in -tip, which actually restored the new ONCEs to
> > the pre-KCSAN version (now that KCSAN can deal with them without
> > annotations).
> >
> > Hope this makes more sense now.
> >
> > [1] https://lore.kernel.org/lkml/CANpmjNOUdr2UG3F45=JaDa0zLwJ5ukPc1MMKujQtmYSmQnjcXg@mail.gmail.com/
> > [2] https://lore.kernel.org/lkml/[email protected]/
> >
>
> Thanks, Marco.
>
> I pulled tip.git#locking/kcsan on top of Linux v5.7-rc7 and applied this patch.
> Just wanted to try KCSAN for the first time and it will also be my
> first building with clang-11.
> That's why I asked.
In general, CONFIG_KCSAN=y and the defaults for the other KCSAN
options should be good. Depending on the size of your system, you
could also tweak KCSAN runtime performance:
https://lwn.net/Articles/816850/#Interacting%20with%20KCSAN%20at%20Runtime
-- the defaults should be good for most systems though.
Hope this helps. Any more questions, do let me know.
Thanks,
-- Marco
On Wed, May 27, 2020 at 12:32:36PM +0200, 'Marco Elver' via Clang Built Linux wrote:
> If the compiler supports C11's _Generic, use it to speed up compilation
> times of __unqual_scalar_typeof(). GCC version 4.9 or later and
> all supported versions of Clang support the feature (the oldest
> supported compiler that doesn't support _Generic is GCC 4.8, for which
> we use the slower alternative).
>
> The non-_Generic variant relies on multiple expansions of
> __pick_integer_type -> __pick_scalar_type -> __builtin_choose_expr,
> which increases pre-processed code size, and can cause compile times to
> increase in files with numerous expansions of READ_ONCE(), or other
> users of __unqual_scalar_typeof().
>
> Summary of compile-time benchmarking done by Arnd Bergmann [1]:
>
> <baseline normalized time> clang-11 gcc-9
> this patch 0.78 0.91
> ideal 0.76 0.86
>
> [1] https://lkml.kernel.org/r/CAK8P3a3UYQeXhiufUevz=rwe09WM_vSTCd9W+KvJHJcOeQyWVA@mail.gmail.com
>
> Further compile-testing done with:
> gcc 4.8, 4.9, 5.5, 6.4, 7.5, 8.4;
> clang 9, 10.
>
> Reported-by: Arnd Bergmann <[email protected]>
> Signed-off-by: Marco Elver <[email protected]>
> Cc: Borislav Petkov <[email protected]>
> Cc: Ingo Molnar <[email protected]>
> Cc: Nick Desaulniers <[email protected]>
> Cc: Paul E. McKenney <[email protected]>
> Cc: Peter Zijlstra <[email protected]>
> Cc: Stephen Rothwell <[email protected]>
> Cc: Thomas Gleixner <[email protected]>
> Cc: Will Deacon <[email protected]>
> Link: https://lkml.kernel.org/r/CAK8P3a0RJtbVi1JMsfik=jkHCNFv+DJn_FeDg-YLW+ueQW3tNg@mail.gmail.com
> ---
> Same version as in:
> https://lkml.kernel.org/r/[email protected]
> ---
> include/linux/compiler_types.h | 22 +++++++++++++++++++++-
> 1 file changed, 21 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
> index 5faf68eae204..a529fa263906 100644
> --- a/include/linux/compiler_types.h
> +++ b/include/linux/compiler_types.h
> @@ -245,7 +245,9 @@ struct ftrace_likely_data {
> /*
> * __unqual_scalar_typeof(x) - Declare an unqualified scalar type, leaving
> * non-scalar types unchanged.
> - *
> + */
> +#if defined(CONFIG_CC_IS_GCC) && CONFIG_GCC_VERSION < 40900
> +/*
> * We build this out of a couple of helper macros in a vain attempt to
> * help you keep your lunch down while reading it.
> */
> @@ -267,6 +269,24 @@ struct ftrace_likely_data {
> __pick_integer_type(x, int, \
> __pick_integer_type(x, long, \
> __pick_integer_type(x, long long, x))))))
> +#else
> +/*
> + * If supported, prefer C11 _Generic for better compile-times. As above, 'char'
> + * is not type-compatible with 'signed char', and we define a separate case.
> + */
> +#define __scalar_type_to_expr_cases(type) \
> + type: (type)0, unsigned type: (unsigned type)0
> +
> +#define __unqual_scalar_typeof(x) typeof( \
> + _Generic((x), \
> + __scalar_type_to_expr_cases(char), \
> + signed char: (signed char)0, \
> + __scalar_type_to_expr_cases(short), \
> + __scalar_type_to_expr_cases(int), \
> + __scalar_type_to_expr_cases(long), \
> + __scalar_type_to_expr_cases(long long), \
> + default: (x)))
> +#endif
>
> /* Is this type a native word size -- useful for atomic operations */
> #define __native_word(t) \
> --
> 2.27.0.rc0.183.gde8f92d652-goog
>
Reviewed-by: Nathan Chancellor <[email protected]>
Tested-by: Nathan Chancellor <[email protected]> # build
The following commit has been merged into the locking/kcsan branch of tip:
Commit-ID: a5dead405f6be1fb80555bdcb77c406bf133fdc8
Gitweb: https://git.kernel.org/tip/a5dead405f6be1fb80555bdcb77c406bf133fdc8
Author: Marco Elver <[email protected]>
AuthorDate: Wed, 27 May 2020 12:32:36 +02:00
Committer: Borislav Petkov <[email protected]>
CommitterDate: Wed, 27 May 2020 14:03:26 +02:00
compiler_types.h: Optimize __unqual_scalar_typeof compilation time
If the compiler supports C11's _Generic, use it to speed up compilation
times of __unqual_scalar_typeof(). GCC version 4.9 or later and
all supported versions of Clang support the feature (the oldest
supported compiler that doesn't support _Generic is GCC 4.8, for which
we use the slower alternative).
The non-_Generic variant relies on multiple expansions of
__pick_integer_type -> __pick_scalar_type -> __builtin_choose_expr,
which increases pre-processed code size, and can cause compile times to
increase in files with numerous expansions of READ_ONCE(), or other
users of __unqual_scalar_typeof().
Summary of compile-time benchmarking done by Arnd Bergmann:
<baseline normalized time> clang-11 gcc-9
this patch 0.78 0.91
ideal 0.76 0.86
See https://lkml.kernel.org/r/CAK8P3a3UYQeXhiufUevz=rwe09WM_vSTCd9W+KvJHJcOeQyWVA@mail.gmail.com
Further compile-testing done with:
gcc 4.8, 4.9, 5.5, 6.4, 7.5, 8.4;
clang 9, 10.
Reported-by: Arnd Bergmann <[email protected]>
Signed-off-by: Marco Elver <[email protected]>
Signed-off-by: Borislav Petkov <[email protected]>
Acked-by: Peter Zijlstra <[email protected]>
Tested-by: Arnd Bergmann <[email protected]>
Link: https://lkml.kernel.org/r/[email protected]
Link: https://lkml.kernel.org/r/CAK8P3a0RJtbVi1JMsfik=jkHCNFv+DJn_FeDg-YLW+ueQW3tNg@mail.gmail.com
---
include/linux/compiler_types.h | 22 +++++++++++++++++++++-
1 file changed, 21 insertions(+), 1 deletion(-)
diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index 5faf68e..a529fa2 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -245,7 +245,9 @@ struct ftrace_likely_data {
/*
* __unqual_scalar_typeof(x) - Declare an unqualified scalar type, leaving
* non-scalar types unchanged.
- *
+ */
+#if defined(CONFIG_CC_IS_GCC) && CONFIG_GCC_VERSION < 40900
+/*
* We build this out of a couple of helper macros in a vain attempt to
* help you keep your lunch down while reading it.
*/
@@ -267,6 +269,24 @@ struct ftrace_likely_data {
__pick_integer_type(x, int, \
__pick_integer_type(x, long, \
__pick_integer_type(x, long long, x))))))
+#else
+/*
+ * If supported, prefer C11 _Generic for better compile-times. As above, 'char'
+ * is not type-compatible with 'signed char', and we define a separate case.
+ */
+#define __scalar_type_to_expr_cases(type) \
+ type: (type)0, unsigned type: (unsigned type)0
+
+#define __unqual_scalar_typeof(x) typeof( \
+ _Generic((x), \
+ __scalar_type_to_expr_cases(char), \
+ signed char: (signed char)0, \
+ __scalar_type_to_expr_cases(short), \
+ __scalar_type_to_expr_cases(int), \
+ __scalar_type_to_expr_cases(long), \
+ __scalar_type_to_expr_cases(long long), \
+ default: (x)))
+#endif
/* Is this type a native word size -- useful for atomic operations */
#define __native_word(t) \
On Wed, May 27, 2020 at 3:57 PM Marco Elver <[email protected]> wrote:
>
> On Wed, 27 May 2020 at 15:37, Sedat Dilek <[email protected]> wrote:
> >
> > On Wed, May 27, 2020 at 3:30 PM Marco Elver <[email protected]> wrote:
> > >
> > > On Wed, 27 May 2020 at 15:11, Sedat Dilek <[email protected]> wrote:
> > > >
> > > > On Wed, May 27, 2020 at 2:50 PM Arnd Bergmann <[email protected]> wrote:
> > > > >
> > > > > On Wed, May 27, 2020 at 2:35 PM Sedat Dilek <[email protected]> wrote:
> > > > > > On Wed, May 27, 2020 at 2:31 PM Arnd Bergmann <[email protected]> wrote:
> > > > > > > On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <[email protected]> wrote:
> > > > > > > > On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <[email protected]> wrote:
> > > > > > > > > On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
> > > > > > > > >
> > > > > > > > > This gives us back 80% of the performance drop on clang, and 50%
> > > > > > > > > of the drop I saw with gcc, compared to current mainline.
> > > > > > > > >
> > > > > > > > > Tested-by: Arnd Bergmann <[email protected]>
> > > > > > > > >
> > > > > > > >
> > > > > > > > Hi Arnd,
> > > > > > > >
> > > > > > > > with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
> > > > > > >
> > > > > > > I meant v5.7.
> > > > > > >
> > > > > > > > I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
> > > > > > > >
> > > > > > > > Is there a speedup benefit also for Linux v5.7?
> > > > > > > > Which patches do I need?
> > > > > > >
> > > > > > > v5.7-rc is the baseline and is the fastest I currently see. On certain files,
> > > > > > > I saw an intermittent 10x slowdown that was already fixed earlier, now
> > > > > > > linux-next
> > > > > > > is more like 2x slowdown for me and 1.2x with this patch on top, so we're
> > > > > > > almost back to the speed of linux-5.7.
> > > > > > >
> > > > > >
> > > > > > Which clang version did you use - and have you set KCSAN kconfigs -
> > > > > > AFAICS this needs clang-11?
> > > > >
> > > > > I'm currently using clang-11, but I see the same problem with older
> > > > > versions, and both with and without KCSAN enabled. I think the issue
> > > > > is mostly the deep nesting of macros that leads to code bloat.
> > > > >
> > > >
> > > > Thanks.
> > > >
> > > > With clang-10:
> > > >
> > > > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > > > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > > > +HAVE_ARCH_KCSAN y
> > >
> > > Clang 10 doesn't support KCSAN (HAVE_KCSAN_COMPILER unset).
> > >
> > > > With clang-11:
> > > >
> > > > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > > > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > > > CLANG_VERSION 100001 -> 110000
> > > > +CC_HAS_ASM_INLINE y
> > > > +HAVE_ARCH_KCSAN y
> > > > +HAVE_KCSAN_COMPILER y
> > > > +KCSAN n
> > > >
> > > > Which KCSAN kconfigs did you enable?
> > >
> > > To clarify: as said in [1], KCSAN (or any other instrumentation) is no
> > > longer relevant to the issue here, and the compile-time regression is
> > > observable with most configs. The problem is due to pre-processing and
> > > parsing, which came about due to new READ_ONCE() and the
> > > __unqual_scalar_typeof() macro (which this patch optimizes).
> > >
> > > KCSAN and new ONCEs got tangled up because we first attempted to
> > > annotate {READ,WRITE}_ONCE() with data_race(), but that turned out to
> > > have all kinds of other issues (explanation in [2]). So we decided to
> > > drop all the KCSAN-specific bits from ONCE, and require KCSAN to be
> > > Clang 11. Those fixes were applied to the first version of new
> > > {READ,WRITE}_ONCE() in -tip, which actually restored the new ONCEs to
> > > the pre-KCSAN version (now that KCSAN can deal with them without
> > > annotations).
> > >
> > > Hope this makes more sense now.
> > >
> > > [1] https://lore.kernel.org/lkml/CANpmjNOUdr2UG3F45=JaDa0zLwJ5ukPc1MMKujQtmYSmQnjcXg@mail.gmail.com/
> > > [2] https://lore.kernel.org/lkml/[email protected]/
> > >
> >
> > Thanks, Marco.
> >
> > I pulled tip.git#locking/kcsan on top of Linux v5.7-rc7 and applied this patch.
> > Just wanted to try KCSAN for the first time and it will also be my
> > first building with clang-11.
> > That's why I asked.
>
> In general, CONFIG_KCSAN=y and the defaults for the other KCSAN
> options should be good. Depending on the size of your system, you
> could also tweak KCSAN runtime performance:
> https://lwn.net/Articles/816850/#Interacting%20with%20KCSAN%20at%20Runtime
> -- the defaults should be good for most systems though.
> Hope this helps. Any more questions, do let me know.
>
Which "projects" and packages do I need?
I have installed:
# LC_ALL=C apt-get install llvm-11 clang-11 lld-11
--no-install-recommends -t llvm-toolchain -y
# dpkg -l | grep
1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261 | awk
'/^ii/ {print $1 " " $2 " " $3}' | column -t
ii clang-11
1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
ii libclang-common-11-dev
1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
ii libclang-cpp11
1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
ii libclang1-11
1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
ii libllvm11:amd64
1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
ii lld-11
1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
ii llvm-11
1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
ii llvm-11-runtime
1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
Is that enough?
- Sedat -
On Wed, 27 May 2020 at 21:11, Sedat Dilek <[email protected]> wrote:
>
> On Wed, May 27, 2020 at 3:57 PM Marco Elver <[email protected]> wrote:
> >
> > On Wed, 27 May 2020 at 15:37, Sedat Dilek <[email protected]> wrote:
> > >
> > > On Wed, May 27, 2020 at 3:30 PM Marco Elver <[email protected]> wrote:
> > > >
> > > > On Wed, 27 May 2020 at 15:11, Sedat Dilek <[email protected]> wrote:
> > > > >
> > > > > On Wed, May 27, 2020 at 2:50 PM Arnd Bergmann <[email protected]> wrote:
> > > > > >
> > > > > > On Wed, May 27, 2020 at 2:35 PM Sedat Dilek <[email protected]> wrote:
> > > > > > > On Wed, May 27, 2020 at 2:31 PM Arnd Bergmann <[email protected]> wrote:
> > > > > > > > On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <[email protected]> wrote:
> > > > > > > > > On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <[email protected]> wrote:
> > > > > > > > > > On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
> > > > > > > > > >
> > > > > > > > > > This gives us back 80% of the performance drop on clang, and 50%
> > > > > > > > > > of the drop I saw with gcc, compared to current mainline.
> > > > > > > > > >
> > > > > > > > > > Tested-by: Arnd Bergmann <[email protected]>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Hi Arnd,
> > > > > > > > >
> > > > > > > > > with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
> > > > > > > >
> > > > > > > > I meant v5.7.
> > > > > > > >
> > > > > > > > > I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
> > > > > > > > >
> > > > > > > > > Is there a speedup benefit also for Linux v5.7?
> > > > > > > > > Which patches do I need?
> > > > > > > >
> > > > > > > > v5.7-rc is the baseline and is the fastest I currently see. On certain files,
> > > > > > > > I saw an intermittent 10x slowdown that was already fixed earlier, now
> > > > > > > > linux-next
> > > > > > > > is more like 2x slowdown for me and 1.2x with this patch on top, so we're
> > > > > > > > almost back to the speed of linux-5.7.
> > > > > > > >
> > > > > > >
> > > > > > > Which clang version did you use - and have you set KCSAN kconfigs -
> > > > > > > AFAICS this needs clang-11?
> > > > > >
> > > > > > I'm currently using clang-11, but I see the same problem with older
> > > > > > versions, and both with and without KCSAN enabled. I think the issue
> > > > > > is mostly the deep nesting of macros that leads to code bloat.
> > > > > >
> > > > >
> > > > > Thanks.
> > > > >
> > > > > With clang-10:
> > > > >
> > > > > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > > > > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > > > > +HAVE_ARCH_KCSAN y
> > > >
> > > > Clang 10 doesn't support KCSAN (HAVE_KCSAN_COMPILER unset).
> > > >
> > > > > With clang-11:
> > > > >
> > > > > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > > > > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > > > > CLANG_VERSION 100001 -> 110000
> > > > > +CC_HAS_ASM_INLINE y
> > > > > +HAVE_ARCH_KCSAN y
> > > > > +HAVE_KCSAN_COMPILER y
> > > > > +KCSAN n
> > > > >
> > > > > Which KCSAN kconfigs did you enable?
> > > >
> > > > To clarify: as said in [1], KCSAN (or any other instrumentation) is no
> > > > longer relevant to the issue here, and the compile-time regression is
> > > > observable with most configs. The problem is due to pre-processing and
> > > > parsing, which came about due to new READ_ONCE() and the
> > > > __unqual_scalar_typeof() macro (which this patch optimizes).
> > > >
> > > > KCSAN and new ONCEs got tangled up because we first attempted to
> > > > annotate {READ,WRITE}_ONCE() with data_race(), but that turned out to
> > > > have all kinds of other issues (explanation in [2]). So we decided to
> > > > drop all the KCSAN-specific bits from ONCE, and require KCSAN to be
> > > > Clang 11. Those fixes were applied to the first version of new
> > > > {READ,WRITE}_ONCE() in -tip, which actually restored the new ONCEs to
> > > > the pre-KCSAN version (now that KCSAN can deal with them without
> > > > annotations).
> > > >
> > > > Hope this makes more sense now.
> > > >
> > > > [1] https://lore.kernel.org/lkml/CANpmjNOUdr2UG3F45=JaDa0zLwJ5ukPc1MMKujQtmYSmQnjcXg@mail.gmail.com/
> > > > [2] https://lore.kernel.org/lkml/[email protected]/
> > > >
> > >
> > > Thanks, Marco.
> > >
> > > I pulled tip.git#locking/kcsan on top of Linux v5.7-rc7 and applied this patch.
> > > Just wanted to try KCSAN for the first time and it will also be my
> > > first building with clang-11.
> > > That's why I asked.
> >
> > In general, CONFIG_KCSAN=y and the defaults for the other KCSAN
> > options should be good. Depending on the size of your system, you
> > could also tweak KCSAN runtime performance:
> > https://lwn.net/Articles/816850/#Interacting%20with%20KCSAN%20at%20Runtime
> > -- the defaults should be good for most systems though.
> > Hope this helps. Any more questions, do let me know.
> >
>
> Which "projects" and packages do I need?
>
> I have installed:
>
> # LC_ALL=C apt-get install llvm-11 clang-11 lld-11
> --no-install-recommends -t llvm-toolchain -y
>
> # dpkg -l | grep
> 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261 | awk
> '/^ii/ {print $1 " " $2 " " $3}' | column -t
> ii clang-11
> 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> ii libclang-common-11-dev
> 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> ii libclang-cpp11
> 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> ii libclang1-11
> 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> ii libllvm11:amd64
> 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> ii lld-11
> 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> ii llvm-11
> 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> ii llvm-11-runtime
> 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
>
> Is that enough?
Just clang-11 (and its transitive dependencies) is enough. Unsure what
your installed binary is, likely "clang-11", so if you can do "make
CC=clang-11 defconfig" (and check for CONFIG_HAVE_KCSAN_COMPILER)
you're good to go.
Thanks,
-- Marco
On Wed, May 27, 2020 at 9:15 PM Marco Elver <[email protected]> wrote:
>
> On Wed, 27 May 2020 at 21:11, Sedat Dilek <[email protected]> wrote:
> >
> > On Wed, May 27, 2020 at 3:57 PM Marco Elver <[email protected]> wrote:
> > >
> > > On Wed, 27 May 2020 at 15:37, Sedat Dilek <[email protected]> wrote:
> > > >
> > > > On Wed, May 27, 2020 at 3:30 PM Marco Elver <[email protected]> wrote:
> > > > >
> > > > > On Wed, 27 May 2020 at 15:11, Sedat Dilek <[email protected]> wrote:
> > > > > >
> > > > > > On Wed, May 27, 2020 at 2:50 PM Arnd Bergmann <[email protected]> wrote:
> > > > > > >
> > > > > > > On Wed, May 27, 2020 at 2:35 PM Sedat Dilek <[email protected]> wrote:
> > > > > > > > On Wed, May 27, 2020 at 2:31 PM Arnd Bergmann <[email protected]> wrote:
> > > > > > > > > On Wed, May 27, 2020 at 1:36 PM Sedat Dilek <[email protected]> wrote:
> > > > > > > > > > On Wed, May 27, 2020 at 1:27 PM Arnd Bergmann <[email protected]> wrote:
> > > > > > > > > > > On Wed, May 27, 2020 at 12:33 PM Marco Elver <[email protected]> wrote:
> > > > > > > > > > >
> > > > > > > > > > > This gives us back 80% of the performance drop on clang, and 50%
> > > > > > > > > > > of the drop I saw with gcc, compared to current mainline.
> > > > > > > > > > >
> > > > > > > > > > > Tested-by: Arnd Bergmann <[email protected]>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi Arnd,
> > > > > > > > > >
> > > > > > > > > > with "mainline" you mean Linux-next aka Linux v5.8 - not v5.7?
> > > > > > > > >
> > > > > > > > > I meant v5.7.
> > > > > > > > >
> > > > > > > > > > I have not seen __unqual_scalar_typeof(x) in compiler_types.h in Linux v5.7.
> > > > > > > > > >
> > > > > > > > > > Is there a speedup benefit also for Linux v5.7?
> > > > > > > > > > Which patches do I need?
> > > > > > > > >
> > > > > > > > > v5.7-rc is the baseline and is the fastest I currently see. On certain files,
> > > > > > > > > I saw an intermittent 10x slowdown that was already fixed earlier, now
> > > > > > > > > linux-next
> > > > > > > > > is more like 2x slowdown for me and 1.2x with this patch on top, so we're
> > > > > > > > > almost back to the speed of linux-5.7.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Which clang version did you use - and have you set KCSAN kconfigs -
> > > > > > > > AFAICS this needs clang-11?
> > > > > > >
> > > > > > > I'm currently using clang-11, but I see the same problem with older
> > > > > > > versions, and both with and without KCSAN enabled. I think the issue
> > > > > > > is mostly the deep nesting of macros that leads to code bloat.
> > > > > > >
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > With clang-10:
> > > > > >
> > > > > > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > > > > > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > > > > > +HAVE_ARCH_KCSAN y
> > > > >
> > > > > Clang 10 doesn't support KCSAN (HAVE_KCSAN_COMPILER unset).
> > > > >
> > > > > > With clang-11:
> > > > > >
> > > > > > $ scripts/diffconfig /boot/config-5.7.0-rc7-2-amd64-clang .config
> > > > > > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > > > > > CLANG_VERSION 100001 -> 110000
> > > > > > +CC_HAS_ASM_INLINE y
> > > > > > +HAVE_ARCH_KCSAN y
> > > > > > +HAVE_KCSAN_COMPILER y
> > > > > > +KCSAN n
> > > > > >
> > > > > > Which KCSAN kconfigs did you enable?
> > > > >
> > > > > To clarify: as said in [1], KCSAN (or any other instrumentation) is no
> > > > > longer relevant to the issue here, and the compile-time regression is
> > > > > observable with most configs. The problem is due to pre-processing and
> > > > > parsing, which came about due to new READ_ONCE() and the
> > > > > __unqual_scalar_typeof() macro (which this patch optimizes).
> > > > >
> > > > > KCSAN and new ONCEs got tangled up because we first attempted to
> > > > > annotate {READ,WRITE}_ONCE() with data_race(), but that turned out to
> > > > > have all kinds of other issues (explanation in [2]). So we decided to
> > > > > drop all the KCSAN-specific bits from ONCE, and require KCSAN to be
> > > > > Clang 11. Those fixes were applied to the first version of new
> > > > > {READ,WRITE}_ONCE() in -tip, which actually restored the new ONCEs to
> > > > > the pre-KCSAN version (now that KCSAN can deal with them without
> > > > > annotations).
> > > > >
> > > > > Hope this makes more sense now.
> > > > >
> > > > > [1] https://lore.kernel.org/lkml/CANpmjNOUdr2UG3F45=JaDa0zLwJ5ukPc1MMKujQtmYSmQnjcXg@mail.gmail.com/
> > > > > [2] https://lore.kernel.org/lkml/[email protected]/
> > > > >
> > > >
> > > > Thanks, Marco.
> > > >
> > > > I pulled tip.git#locking/kcsan on top of Linux v5.7-rc7 and applied this patch.
> > > > Just wanted to try KCSAN for the first time and it will also be my
> > > > first building with clang-11.
> > > > That's why I asked.
> > >
> > > In general, CONFIG_KCSAN=y and the defaults for the other KCSAN
> > > options should be good. Depending on the size of your system, you
> > > could also tweak KCSAN runtime performance:
> > > https://lwn.net/Articles/816850/#Interacting%20with%20KCSAN%20at%20Runtime
> > > -- the defaults should be good for most systems though.
> > > Hope this helps. Any more questions, do let me know.
> > >
> >
> > Which "projects" and packages do I need?
> >
> > I have installed:
> >
> > # LC_ALL=C apt-get install llvm-11 clang-11 lld-11
> > --no-install-recommends -t llvm-toolchain -y
> >
> > # dpkg -l | grep
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261 | awk
> > '/^ii/ {print $1 " " $2 " " $3}' | column -t
> > ii clang-11
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii libclang-common-11-dev
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii libclang-cpp11
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii libclang1-11
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii libllvm11:amd64
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii lld-11
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii llvm-11
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > ii llvm-11-runtime
> > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> >
> > Is that enough?
>
> Just clang-11 (and its transitive dependencies) is enough. Unsure what
> your installed binary is, likely "clang-11", so if you can do "make
> CC=clang-11 defconfig" (and check for CONFIG_HAVE_KCSAN_COMPILER)
> you're good to go.
>
I was able to build with clang-11 from apt.llvm.org.
[ build-time ]
Normally, it takes me approx. 05:00 to build with clang-10
(10.0.1-rc1) and Linux v5.7-rc7.
This time start: 21:18 and stop: 03:45 means 06:27 - took 01:27 longer.
Samsung Ultrabook 2nd generation aka Intel Sandybridge CPU with 'make -j3'.
[ diffconfig ]
BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
CLANG_VERSION 100001 -> 110000
+CC_HAS_ASM_INLINE y
+HAVE_ARCH_KCSAN y
+HAVE_KCSAN_COMPILER y
+KCSAN y
+KCSAN_ASSUME_PLAIN_WRITES_ATOMIC y
+KCSAN_DEBUG n
+KCSAN_DELAY_RANDOMIZE y
+KCSAN_EARLY_ENABLE y
+KCSAN_IGNORE_ATOMICS n
+KCSAN_INTERRUPT_WATCHER n
+KCSAN_NUM_WATCHPOINTS 64
+KCSAN_REPORT_ONCE_IN_MS 3000
+KCSAN_REPORT_RACE_UNKNOWN_ORIGIN y
+KCSAN_REPORT_VALUE_CHANGE_ONLY y
+KCSAN_SELFTEST y
+KCSAN_SKIP_WATCH 4000
+KCSAN_SKIP_WATCH_RANDOMIZE y
+KCSAN_UDELAY_INTERRUPT 20
+KCSAN_UDELAY_TASK 80
I am seeing this data-races:
root@iniza:~# LC_ALL=C dmesg -T | grep 'BUG: KCSAN: data-race'
[Thu May 28 03:51:53 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:52:00 2020] BUG: KCSAN: data-race in mark_page_accessed
/ workingset_activation
[Thu May 28 03:52:02 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:52:08 2020] BUG: KCSAN: data-race in
blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
[Thu May 28 03:52:10 2020] BUG: KCSAN: data-race in dd_has_work /
dd_insert_requests
[Thu May 28 03:52:11 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:52:13 2020] BUG: KCSAN: data-race in
page_counter_try_charge / page_counter_try_charge
[Thu May 28 03:52:15 2020] BUG: KCSAN: data-race in ep_poll_callback /
ep_send_events_proc
[Thu May 28 03:52:21 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:52:25 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:52:26 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:52:31 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:52:38 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:52:53 2020] BUG: KCSAN: data-race in dd_has_work /
dd_insert_requests
[Thu May 28 03:52:56 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:52:59 2020] BUG: KCSAN: data-race in
blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
[Thu May 28 03:53:25 2020] BUG: KCSAN: data-race in
rwsem_spin_on_owner+0x102/0x1a0
[Thu May 28 03:53:25 2020] BUG: KCSAN: data-race in
page_counter_try_charge / page_counter_try_charge
[Thu May 28 03:53:39 2020] BUG: KCSAN: data-race in do_epoll_wait /
ep_poll_callback
[Thu May 28 03:53:39 2020] BUG: KCSAN: data-race in find_next_and_bit+0x30/0xd0
[Thu May 28 03:53:41 2020] BUG: KCSAN: data-race in dd_has_work /
dd_insert_requests
[Thu May 28 03:53:43 2020] BUG: KCSAN: data-race in do_epoll_wait /
ep_poll_callback
[Thu May 28 03:53:45 2020] BUG: KCSAN: data-race in dd_has_work /
dd_insert_requests
[Thu May 28 03:53:46 2020] BUG: KCSAN: data-race in
blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
[Thu May 28 03:53:47 2020] BUG: KCSAN: data-race in
rwsem_spin_on_owner+0x102/0x1a0
[Thu May 28 03:54:02 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:54:11 2020] BUG: KCSAN: data-race in find_next_and_bit+0x30/0xd0
[Thu May 28 03:54:19 2020] BUG: KCSAN: data-race in
rwsem_spin_on_owner+0x102/0x1a0
[Thu May 28 03:55:00 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:56:14 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:56:50 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:56:50 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:56:52 2020] BUG: KCSAN: data-race in
tick_nohz_next_event / tick_nohz_stop_tick
[Thu May 28 03:56:58 2020] BUG: KCSAN: data-race in
blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
[Thu May 28 03:57:58 2020] BUG: KCSAN: data-race in
blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
[Thu May 28 03:58:00 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 03:58:07 2020] BUG: KCSAN: data-race in
tick_nohz_next_event / tick_nohz_stop_tick
[Thu May 28 03:58:44 2020] BUG: KCSAN: data-race in
mutex_spin_on_owner+0xe0/0x1b0
[Thu May 28 03:58:49 2020] BUG: KCSAN: data-race in __bitmap_subset+0x38/0xd0
[Thu May 28 03:59:46 2020] BUG: KCSAN: data-race in
tick_nohz_next_event / tick_nohz_stop_tick
[Thu May 28 04:00:25 2020] BUG: KCSAN: data-race in dd_has_work /
deadline_remove_request
[Thu May 28 04:00:26 2020] BUG: KCSAN: data-race in
tick_nohz_next_event / tick_nohz_stop_tick
Full dmesg output and linux-config attached.
- Sedat -
On Thu, 28 May 2020 at 04:12, Sedat Dilek <[email protected]> wrote:
>
[...]
> > > >
> > > > In general, CONFIG_KCSAN=y and the defaults for the other KCSAN
> > > > options should be good. Depending on the size of your system, you
> > > > could also tweak KCSAN runtime performance:
> > > > https://lwn.net/Articles/816850/#Interacting%20with%20KCSAN%20at%20Runtime
> > > > -- the defaults should be good for most systems though.
> > > > Hope this helps. Any more questions, do let me know.
> > > >
> > >
> > > Which "projects" and packages do I need?
> > >
> > > I have installed:
> > >
> > > # LC_ALL=C apt-get install llvm-11 clang-11 lld-11
> > > --no-install-recommends -t llvm-toolchain -y
> > >
> > > # dpkg -l | grep
> > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261 | awk
> > > '/^ii/ {print $1 " " $2 " " $3}' | column -t
> > > ii clang-11
> > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > ii libclang-common-11-dev
> > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > ii libclang-cpp11
> > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > ii libclang1-11
> > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > ii libllvm11:amd64
> > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > ii lld-11
> > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > ii llvm-11
> > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > ii llvm-11-runtime
> > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > >
> > > Is that enough?
> >
> > Just clang-11 (and its transitive dependencies) is enough. Unsure what
> > your installed binary is, likely "clang-11", so if you can do "make
> > CC=clang-11 defconfig" (and check for CONFIG_HAVE_KCSAN_COMPILER)
> > you're good to go.
> >
>
> I was able to build with clang-11 from apt.llvm.org.
>
> [ build-time ]
>
> Normally, it takes me approx. 05:00 to build with clang-10
> (10.0.1-rc1) and Linux v5.7-rc7.
>
> This time start: 21:18 and stop: 03:45 means 06:27 - took 01:27 longer.
>
> Samsung Ultrabook 2nd generation aka Intel Sandybridge CPU with 'make -j3'.
>
> [ diffconfig ]
>
> BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> CLANG_VERSION 100001 -> 110000
> +CC_HAS_ASM_INLINE y
> +HAVE_ARCH_KCSAN y
> +HAVE_KCSAN_COMPILER y
> +KCSAN y
> +KCSAN_ASSUME_PLAIN_WRITES_ATOMIC y
> +KCSAN_DEBUG n
> +KCSAN_DELAY_RANDOMIZE y
> +KCSAN_EARLY_ENABLE y
> +KCSAN_IGNORE_ATOMICS n
> +KCSAN_INTERRUPT_WATCHER n
> +KCSAN_NUM_WATCHPOINTS 64
> +KCSAN_REPORT_ONCE_IN_MS 3000
> +KCSAN_REPORT_RACE_UNKNOWN_ORIGIN y
> +KCSAN_REPORT_VALUE_CHANGE_ONLY y
> +KCSAN_SELFTEST y
> +KCSAN_SKIP_WATCH 4000
> +KCSAN_SKIP_WATCH_RANDOMIZE y
> +KCSAN_UDELAY_INTERRUPT 20
> +KCSAN_UDELAY_TASK 80
>
> I am seeing this data-races:
>
> root@iniza:~# LC_ALL=C dmesg -T | grep 'BUG: KCSAN: data-race'
> [Thu May 28 03:51:53 2020] BUG: KCSAN: data-race in
> mutex_spin_on_owner+0xe0/0x1b0
> [Thu May 28 03:52:00 2020] BUG: KCSAN: data-race in mark_page_accessed
> / workingset_activation
> [Thu May 28 03:52:02 2020] BUG: KCSAN: data-race in
> mutex_spin_on_owner+0xe0/0x1b0
> [Thu May 28 03:52:08 2020] BUG: KCSAN: data-race in
> blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
> [Thu May 28 03:52:10 2020] BUG: KCSAN: data-race in dd_has_work /
> dd_insert_requests
> [Thu May 28 03:52:11 2020] BUG: KCSAN: data-race in
> mutex_spin_on_owner+0xe0/0x1b0
> [Thu May 28 03:52:13 2020] BUG: KCSAN: data-race in
> page_counter_try_charge / page_counter_try_charge
> [Thu May 28 03:52:15 2020] BUG: KCSAN: data-race in ep_poll_callback /
> ep_send_events_proc
> [Thu May 28 03:52:21 2020] BUG: KCSAN: data-race in
> mutex_spin_on_owner+0xe0/0x1b0
> [Thu May 28 03:52:25 2020] BUG: KCSAN: data-race in
> mutex_spin_on_owner+0xe0/0x1b0
> [Thu May 28 03:52:26 2020] BUG: KCSAN: data-race in dd_has_work /
> deadline_remove_request
> [Thu May 28 03:52:31 2020] BUG: KCSAN: data-race in dd_has_work /
> deadline_remove_request
> [Thu May 28 03:52:38 2020] BUG: KCSAN: data-race in dd_has_work /
> deadline_remove_request
> [Thu May 28 03:52:53 2020] BUG: KCSAN: data-race in dd_has_work /
> dd_insert_requests
> [Thu May 28 03:52:56 2020] BUG: KCSAN: data-race in dd_has_work /
> deadline_remove_request
> [Thu May 28 03:52:59 2020] BUG: KCSAN: data-race in
> blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
> [Thu May 28 03:53:25 2020] BUG: KCSAN: data-race in
> rwsem_spin_on_owner+0x102/0x1a0
> [Thu May 28 03:53:25 2020] BUG: KCSAN: data-race in
> page_counter_try_charge / page_counter_try_charge
> [Thu May 28 03:53:39 2020] BUG: KCSAN: data-race in do_epoll_wait /
> ep_poll_callback
> [Thu May 28 03:53:39 2020] BUG: KCSAN: data-race in find_next_and_bit+0x30/0xd0
> [Thu May 28 03:53:41 2020] BUG: KCSAN: data-race in dd_has_work /
> dd_insert_requests
> [Thu May 28 03:53:43 2020] BUG: KCSAN: data-race in do_epoll_wait /
> ep_poll_callback
> [Thu May 28 03:53:45 2020] BUG: KCSAN: data-race in dd_has_work /
> dd_insert_requests
> [Thu May 28 03:53:46 2020] BUG: KCSAN: data-race in
> blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
> [Thu May 28 03:53:47 2020] BUG: KCSAN: data-race in
> rwsem_spin_on_owner+0x102/0x1a0
> [Thu May 28 03:54:02 2020] BUG: KCSAN: data-race in dd_has_work /
> deadline_remove_request
> [Thu May 28 03:54:11 2020] BUG: KCSAN: data-race in find_next_and_bit+0x30/0xd0
> [Thu May 28 03:54:19 2020] BUG: KCSAN: data-race in
> rwsem_spin_on_owner+0x102/0x1a0
> [Thu May 28 03:55:00 2020] BUG: KCSAN: data-race in
> mutex_spin_on_owner+0xe0/0x1b0
> [Thu May 28 03:56:14 2020] BUG: KCSAN: data-race in dd_has_work /
> deadline_remove_request
> [Thu May 28 03:56:50 2020] BUG: KCSAN: data-race in dd_has_work /
> deadline_remove_request
> [Thu May 28 03:56:50 2020] BUG: KCSAN: data-race in dd_has_work /
> deadline_remove_request
> [Thu May 28 03:56:52 2020] BUG: KCSAN: data-race in
> tick_nohz_next_event / tick_nohz_stop_tick
> [Thu May 28 03:56:58 2020] BUG: KCSAN: data-race in
> blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
> [Thu May 28 03:57:58 2020] BUG: KCSAN: data-race in
> blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
> [Thu May 28 03:58:00 2020] BUG: KCSAN: data-race in dd_has_work /
> deadline_remove_request
> [Thu May 28 03:58:07 2020] BUG: KCSAN: data-race in
> tick_nohz_next_event / tick_nohz_stop_tick
> [Thu May 28 03:58:44 2020] BUG: KCSAN: data-race in
> mutex_spin_on_owner+0xe0/0x1b0
> [Thu May 28 03:58:49 2020] BUG: KCSAN: data-race in __bitmap_subset+0x38/0xd0
> [Thu May 28 03:59:46 2020] BUG: KCSAN: data-race in
> tick_nohz_next_event / tick_nohz_stop_tick
> [Thu May 28 04:00:25 2020] BUG: KCSAN: data-race in dd_has_work /
> deadline_remove_request
> [Thu May 28 04:00:26 2020] BUG: KCSAN: data-race in
> tick_nohz_next_event / tick_nohz_stop_tick
>
> Full dmesg output and linux-config attached.
Thank you for the report. There are a number of known data races. Note
that, we do not think it's wise to rush fixes for data races,
especially because each one requires careful analysis of what the
appropriate response is. In the meantime, also have a look at these 2
articles (if you haven't already), which describes the current state
of things:
1. https://lwn.net/Articles/816850/
2. https://lwn.net/Articles/816854/
Thanks,
-- Marco
On Thu, May 28, 2020 at 5:16 PM Marco Elver <[email protected]> wrote:
>
> On Thu, 28 May 2020 at 04:12, Sedat Dilek <[email protected]> wrote:
> >
> [...]
>
> > > > >
> > > > > In general, CONFIG_KCSAN=y and the defaults for the other KCSAN
> > > > > options should be good. Depending on the size of your system, you
> > > > > could also tweak KCSAN runtime performance:
> > > > > https://lwn.net/Articles/816850/#Interacting%20with%20KCSAN%20at%20Runtime
> > > > > -- the defaults should be good for most systems though.
> > > > > Hope this helps. Any more questions, do let me know.
> > > > >
> > > >
> > > > Which "projects" and packages do I need?
> > > >
> > > > I have installed:
> > > >
> > > > # LC_ALL=C apt-get install llvm-11 clang-11 lld-11
> > > > --no-install-recommends -t llvm-toolchain -y
> > > >
> > > > # dpkg -l | grep
> > > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261 | awk
> > > > '/^ii/ {print $1 " " $2 " " $3}' | column -t
> > > > ii clang-11
> > > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > > ii libclang-common-11-dev
> > > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > > ii libclang-cpp11
> > > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > > ii libclang1-11
> > > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > > ii libllvm11:amd64
> > > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > > ii lld-11
> > > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > > ii llvm-11
> > > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > > ii llvm-11-runtime
> > > > 1:11~++20200527111130+65030821d4a-1~exp1~20200527091804.3261
> > > >
> > > > Is that enough?
> > >
> > > Just clang-11 (and its transitive dependencies) is enough. Unsure what
> > > your installed binary is, likely "clang-11", so if you can do "make
> > > CC=clang-11 defconfig" (and check for CONFIG_HAVE_KCSAN_COMPILER)
> > > you're good to go.
> > >
> >
> > I was able to build with clang-11 from apt.llvm.org.
> >
> > [ build-time ]
> >
> > Normally, it takes me approx. 05:00 to build with clang-10
> > (10.0.1-rc1) and Linux v5.7-rc7.
> >
> > This time start: 21:18 and stop: 03:45 means 06:27 - took 01:27 longer.
> >
> > Samsung Ultrabook 2nd generation aka Intel Sandybridge CPU with 'make -j3'.
> >
> > [ diffconfig ]
> >
> > BUILD_SALT "5.7.0-rc7-2-amd64-clang" -> "5.7.0-rc7-3-amd64-clang"
> > CLANG_VERSION 100001 -> 110000
> > +CC_HAS_ASM_INLINE y
> > +HAVE_ARCH_KCSAN y
> > +HAVE_KCSAN_COMPILER y
> > +KCSAN y
> > +KCSAN_ASSUME_PLAIN_WRITES_ATOMIC y
> > +KCSAN_DEBUG n
> > +KCSAN_DELAY_RANDOMIZE y
> > +KCSAN_EARLY_ENABLE y
> > +KCSAN_IGNORE_ATOMICS n
> > +KCSAN_INTERRUPT_WATCHER n
> > +KCSAN_NUM_WATCHPOINTS 64
> > +KCSAN_REPORT_ONCE_IN_MS 3000
> > +KCSAN_REPORT_RACE_UNKNOWN_ORIGIN y
> > +KCSAN_REPORT_VALUE_CHANGE_ONLY y
> > +KCSAN_SELFTEST y
> > +KCSAN_SKIP_WATCH 4000
> > +KCSAN_SKIP_WATCH_RANDOMIZE y
> > +KCSAN_UDELAY_INTERRUPT 20
> > +KCSAN_UDELAY_TASK 80
> >
> > I am seeing this data-races:
> >
> > root@iniza:~# LC_ALL=C dmesg -T | grep 'BUG: KCSAN: data-race'
> > [Thu May 28 03:51:53 2020] BUG: KCSAN: data-race in
> > mutex_spin_on_owner+0xe0/0x1b0
> > [Thu May 28 03:52:00 2020] BUG: KCSAN: data-race in mark_page_accessed
> > / workingset_activation
> > [Thu May 28 03:52:02 2020] BUG: KCSAN: data-race in
> > mutex_spin_on_owner+0xe0/0x1b0
> > [Thu May 28 03:52:08 2020] BUG: KCSAN: data-race in
> > blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
> > [Thu May 28 03:52:10 2020] BUG: KCSAN: data-race in dd_has_work /
> > dd_insert_requests
> > [Thu May 28 03:52:11 2020] BUG: KCSAN: data-race in
> > mutex_spin_on_owner+0xe0/0x1b0
> > [Thu May 28 03:52:13 2020] BUG: KCSAN: data-race in
> > page_counter_try_charge / page_counter_try_charge
> > [Thu May 28 03:52:15 2020] BUG: KCSAN: data-race in ep_poll_callback /
> > ep_send_events_proc
> > [Thu May 28 03:52:21 2020] BUG: KCSAN: data-race in
> > mutex_spin_on_owner+0xe0/0x1b0
> > [Thu May 28 03:52:25 2020] BUG: KCSAN: data-race in
> > mutex_spin_on_owner+0xe0/0x1b0
> > [Thu May 28 03:52:26 2020] BUG: KCSAN: data-race in dd_has_work /
> > deadline_remove_request
> > [Thu May 28 03:52:31 2020] BUG: KCSAN: data-race in dd_has_work /
> > deadline_remove_request
> > [Thu May 28 03:52:38 2020] BUG: KCSAN: data-race in dd_has_work /
> > deadline_remove_request
> > [Thu May 28 03:52:53 2020] BUG: KCSAN: data-race in dd_has_work /
> > dd_insert_requests
> > [Thu May 28 03:52:56 2020] BUG: KCSAN: data-race in dd_has_work /
> > deadline_remove_request
> > [Thu May 28 03:52:59 2020] BUG: KCSAN: data-race in
> > blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
> > [Thu May 28 03:53:25 2020] BUG: KCSAN: data-race in
> > rwsem_spin_on_owner+0x102/0x1a0
> > [Thu May 28 03:53:25 2020] BUG: KCSAN: data-race in
> > page_counter_try_charge / page_counter_try_charge
> > [Thu May 28 03:53:39 2020] BUG: KCSAN: data-race in do_epoll_wait /
> > ep_poll_callback
> > [Thu May 28 03:53:39 2020] BUG: KCSAN: data-race in find_next_and_bit+0x30/0xd0
> > [Thu May 28 03:53:41 2020] BUG: KCSAN: data-race in dd_has_work /
> > dd_insert_requests
> > [Thu May 28 03:53:43 2020] BUG: KCSAN: data-race in do_epoll_wait /
> > ep_poll_callback
> > [Thu May 28 03:53:45 2020] BUG: KCSAN: data-race in dd_has_work /
> > dd_insert_requests
> > [Thu May 28 03:53:46 2020] BUG: KCSAN: data-race in
> > blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
> > [Thu May 28 03:53:47 2020] BUG: KCSAN: data-race in
> > rwsem_spin_on_owner+0x102/0x1a0
> > [Thu May 28 03:54:02 2020] BUG: KCSAN: data-race in dd_has_work /
> > deadline_remove_request
> > [Thu May 28 03:54:11 2020] BUG: KCSAN: data-race in find_next_and_bit+0x30/0xd0
> > [Thu May 28 03:54:19 2020] BUG: KCSAN: data-race in
> > rwsem_spin_on_owner+0x102/0x1a0
> > [Thu May 28 03:55:00 2020] BUG: KCSAN: data-race in
> > mutex_spin_on_owner+0xe0/0x1b0
> > [Thu May 28 03:56:14 2020] BUG: KCSAN: data-race in dd_has_work /
> > deadline_remove_request
> > [Thu May 28 03:56:50 2020] BUG: KCSAN: data-race in dd_has_work /
> > deadline_remove_request
> > [Thu May 28 03:56:50 2020] BUG: KCSAN: data-race in dd_has_work /
> > deadline_remove_request
> > [Thu May 28 03:56:52 2020] BUG: KCSAN: data-race in
> > tick_nohz_next_event / tick_nohz_stop_tick
> > [Thu May 28 03:56:58 2020] BUG: KCSAN: data-race in
> > blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
> > [Thu May 28 03:57:58 2020] BUG: KCSAN: data-race in
> > blk_mq_sched_dispatch_requests / blk_mq_sched_dispatch_requests
> > [Thu May 28 03:58:00 2020] BUG: KCSAN: data-race in dd_has_work /
> > deadline_remove_request
> > [Thu May 28 03:58:07 2020] BUG: KCSAN: data-race in
> > tick_nohz_next_event / tick_nohz_stop_tick
> > [Thu May 28 03:58:44 2020] BUG: KCSAN: data-race in
> > mutex_spin_on_owner+0xe0/0x1b0
> > [Thu May 28 03:58:49 2020] BUG: KCSAN: data-race in __bitmap_subset+0x38/0xd0
> > [Thu May 28 03:59:46 2020] BUG: KCSAN: data-race in
> > tick_nohz_next_event / tick_nohz_stop_tick
> > [Thu May 28 04:00:25 2020] BUG: KCSAN: data-race in dd_has_work /
> > deadline_remove_request
> > [Thu May 28 04:00:26 2020] BUG: KCSAN: data-race in
> > tick_nohz_next_event / tick_nohz_stop_tick
> >
> > Full dmesg output and linux-config attached.
>
> Thank you for the report. There are a number of known data races. Note
> that, we do not think it's wise to rush fixes for data races,
> especially because each one requires careful analysis of what the
> appropriate response is. In the meantime, also have a look at these 2
> articles (if you haven't already), which describes the current state
> of things:
>
> 1. https://lwn.net/Articles/816850/
> 2. https://lwn.net/Articles/816854/
>
Hi Marco,
thanks for your feedback.
The first article I have read already.
That does not mean I am a KCSAN expert now.
As you say each data-race needs an individual analysis.
Just one last number:
Building again a Linux v5.7-rc7 on a clang-11-compiled and
kcsan-enabled linux-kernel took me...
one hour longer (6 instead of 5 hours, start: 12:14 and stop: 18:15)
Regards,
- Sedat -