2020-01-17 17:28:06

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: linux-next: Tree for Dec 6 (objtool, lots in btrfs)

On Tue, Dec 17, 2019 at 04:25:11PM +0100, David Sterba wrote:
> On Fri, Dec 13, 2019 at 11:05:18PM -0800, Randy Dunlap wrote:
> > OK, that fixes most of them, but still leaves these 2:
> >
> > btrfs006.out:fs/btrfs/extent_io.o: warning: objtool: __set_extent_bit()+0x536: unreachable instruction
>
> Hard to read from the assembly what C statement is it referring to. I
> think there are also several functions inlined, I don't see anything
> suspicious inside __set_extent_bit itself.
>
> > btrfs006.out:fs/btrfs/relocation.o: warning: objtool: add_tree_block()+0x501: unreachable instruction
>
> Probably also heavily inlined, the function has like 50 lines, a few
> non-trivial function calls but the offset in the warning suggests a
> larger size.
>
> While browsing the callees I noticed that both have in common a function
> that is supposed to print and stop at fatal errors. They're
> extent_io_tree_panic (extent_io.c) and backref_tree_panic
> (relocation.c). Both call btrfs_panic which is a macro:
>
> 3239 #define btrfs_panic(fs_info, errno, fmt, args...) \
> 3240 do { \
> 3241 __btrfs_panic(fs_info, __func__, __LINE__, errno, fmt, ##args); \
> 3242 BUG(); \
> 3243 } while (0)
>
> There are no conditionals and BUG has the __noreturn annotation
> (unreachable()) so all is in place and I don't have better ideas what's
> causing the reports.

I think KCSAN is somehow disabling GCC's detection of implicit noreturn
functions -- or at least some calls to them. So GCC is inserting dead
code after the calls. BUG() uses __builtin_unreachable(), so GCC should
know better.

If this is specific to KCSAN then I might just disable these warnings
for KCSAN configs.

--
Josh


2020-01-17 20:29:49

by Marco Elver

[permalink] [raw]
Subject: Re: linux-next: Tree for Dec 6 (objtool, lots in btrfs)

On Fri, 17 Jan 2020 at 18:26, Josh Poimboeuf <[email protected]> wrote:
>
> On Tue, Dec 17, 2019 at 04:25:11PM +0100, David Sterba wrote:
> > On Fri, Dec 13, 2019 at 11:05:18PM -0800, Randy Dunlap wrote:
> > > OK, that fixes most of them, but still leaves these 2:
> > >
> > > btrfs006.out:fs/btrfs/extent_io.o: warning: objtool: __set_extent_bit()+0x536: unreachable instruction
> >
> > Hard to read from the assembly what C statement is it referring to. I
> > think there are also several functions inlined, I don't see anything
> > suspicious inside __set_extent_bit itself.
> >
> > > btrfs006.out:fs/btrfs/relocation.o: warning: objtool: add_tree_block()+0x501: unreachable instruction
> >
> > Probably also heavily inlined, the function has like 50 lines, a few
> > non-trivial function calls but the offset in the warning suggests a
> > larger size.
> >
> > While browsing the callees I noticed that both have in common a function
> > that is supposed to print and stop at fatal errors. They're
> > extent_io_tree_panic (extent_io.c) and backref_tree_panic
> > (relocation.c). Both call btrfs_panic which is a macro:
> >
> > 3239 #define btrfs_panic(fs_info, errno, fmt, args...) \
> > 3240 do { \
> > 3241 __btrfs_panic(fs_info, __func__, __LINE__, errno, fmt, ##args); \
> > 3242 BUG(); \
> > 3243 } while (0)
> >
> > There are no conditionals and BUG has the __noreturn annotation
> > (unreachable()) so all is in place and I don't have better ideas what's
> > causing the reports.
>
> I think KCSAN is somehow disabling GCC's detection of implicit noreturn
> functions -- or at least some calls to them. So GCC is inserting dead
> code after the calls. BUG() uses __builtin_unreachable(), so GCC should
> know better.
>
> If this is specific to KCSAN then I might just disable these warnings
> for KCSAN configs.

I noticed that this is also a CC_OPTIMIZE_FOR_SIZE config. I recently
sent some patches to turn some inlines into __always_inlines because
CC_OPTIMIZE_FOR_SIZE decides to not inline functions that should
always be inlined.

I noticed that 'assfail' is a 'static inline' function and you
mentioned earlier that GCC seems to not be able to determine if it
returns or not. If CC_OPTIMIZE_FOR_SIZE decides to not inline, then
maybe this could be a problem? It could also be the compiler having
some trouble here with the CC_OPTIMIZE_FOR_SIZE + KCSAN combination.

Thanks,
-- Marco

2020-01-17 21:28:49

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: linux-next: Tree for Dec 6 (objtool, lots in btrfs)

On Fri, Jan 17, 2020 at 09:28:27PM +0100, Marco Elver wrote:
> On Fri, 17 Jan 2020 at 18:26, Josh Poimboeuf <[email protected]> wrote:
> >
> > On Tue, Dec 17, 2019 at 04:25:11PM +0100, David Sterba wrote:
> > > On Fri, Dec 13, 2019 at 11:05:18PM -0800, Randy Dunlap wrote:
> > > > OK, that fixes most of them, but still leaves these 2:
> > > >
> > > > btrfs006.out:fs/btrfs/extent_io.o: warning: objtool: __set_extent_bit()+0x536: unreachable instruction
> > >
> > > Hard to read from the assembly what C statement is it referring to. I
> > > think there are also several functions inlined, I don't see anything
> > > suspicious inside __set_extent_bit itself.
> > >
> > > > btrfs006.out:fs/btrfs/relocation.o: warning: objtool: add_tree_block()+0x501: unreachable instruction
> > >
> > > Probably also heavily inlined, the function has like 50 lines, a few
> > > non-trivial function calls but the offset in the warning suggests a
> > > larger size.
> > >
> > > While browsing the callees I noticed that both have in common a function
> > > that is supposed to print and stop at fatal errors. They're
> > > extent_io_tree_panic (extent_io.c) and backref_tree_panic
> > > (relocation.c). Both call btrfs_panic which is a macro:
> > >
> > > 3239 #define btrfs_panic(fs_info, errno, fmt, args...) \
> > > 3240 do { \
> > > 3241 __btrfs_panic(fs_info, __func__, __LINE__, errno, fmt, ##args); \
> > > 3242 BUG(); \
> > > 3243 } while (0)
> > >
> > > There are no conditionals and BUG has the __noreturn annotation
> > > (unreachable()) so all is in place and I don't have better ideas what's
> > > causing the reports.
> >
> > I think KCSAN is somehow disabling GCC's detection of implicit noreturn
> > functions -- or at least some calls to them. So GCC is inserting dead
> > code after the calls. BUG() uses __builtin_unreachable(), so GCC should
> > know better.
> >
> > If this is specific to KCSAN then I might just disable these warnings
> > for KCSAN configs.
>
> I noticed that this is also a CC_OPTIMIZE_FOR_SIZE config. I recently
> sent some patches to turn some inlines into __always_inlines because
> CC_OPTIMIZE_FOR_SIZE decides to not inline functions that should
> always be inlined.
>
> I noticed that 'assfail' is a 'static inline' function and you
> mentioned earlier that GCC seems to not be able to determine if it
> returns or not. If CC_OPTIMIZE_FOR_SIZE decides to not inline, then
> maybe this could be a problem? It could also be the compiler having
> some trouble here with the CC_OPTIMIZE_FOR_SIZE + KCSAN combination.

Even for a non-inlined static function, GCC typically detects when it's
implicitly "noreturn", and optimizes the call sites accordingly. And
that has also been true even for CC_OPTIMIZE_FOR_SIZE in the past. So
something changed apparently. (KCSAN was just a guess.)

--
Josh

2020-01-17 22:23:29

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: linux-next: Tree for Dec 6 (objtool, lots in btrfs)

On Fri, Jan 17, 2020 at 03:26:49PM -0600, Josh Poimboeuf wrote:
> On Fri, Jan 17, 2020 at 09:28:27PM +0100, Marco Elver wrote:
> > On Fri, 17 Jan 2020 at 18:26, Josh Poimboeuf <[email protected]> wrote:
> > >
> > > On Tue, Dec 17, 2019 at 04:25:11PM +0100, David Sterba wrote:
> > > > On Fri, Dec 13, 2019 at 11:05:18PM -0800, Randy Dunlap wrote:
> > > > > OK, that fixes most of them, but still leaves these 2:
> > > > >
> > > > > btrfs006.out:fs/btrfs/extent_io.o: warning: objtool: __set_extent_bit()+0x536: unreachable instruction
> > > >
> > > > Hard to read from the assembly what C statement is it referring to. I
> > > > think there are also several functions inlined, I don't see anything
> > > > suspicious inside __set_extent_bit itself.
> > > >
> > > > > btrfs006.out:fs/btrfs/relocation.o: warning: objtool: add_tree_block()+0x501: unreachable instruction
> > > >
> > > > Probably also heavily inlined, the function has like 50 lines, a few
> > > > non-trivial function calls but the offset in the warning suggests a
> > > > larger size.
> > > >
> > > > While browsing the callees I noticed that both have in common a function
> > > > that is supposed to print and stop at fatal errors. They're
> > > > extent_io_tree_panic (extent_io.c) and backref_tree_panic
> > > > (relocation.c). Both call btrfs_panic which is a macro:
> > > >
> > > > 3239 #define btrfs_panic(fs_info, errno, fmt, args...) \
> > > > 3240 do { \
> > > > 3241 __btrfs_panic(fs_info, __func__, __LINE__, errno, fmt, ##args); \
> > > > 3242 BUG(); \
> > > > 3243 } while (0)
> > > >
> > > > There are no conditionals and BUG has the __noreturn annotation
> > > > (unreachable()) so all is in place and I don't have better ideas what's
> > > > causing the reports.
> > >
> > > I think KCSAN is somehow disabling GCC's detection of implicit noreturn
> > > functions -- or at least some calls to them. So GCC is inserting dead
> > > code after the calls. BUG() uses __builtin_unreachable(), so GCC should
> > > know better.
> > >
> > > If this is specific to KCSAN then I might just disable these warnings
> > > for KCSAN configs.
> >
> > I noticed that this is also a CC_OPTIMIZE_FOR_SIZE config. I recently
> > sent some patches to turn some inlines into __always_inlines because
> > CC_OPTIMIZE_FOR_SIZE decides to not inline functions that should
> > always be inlined.
> >
> > I noticed that 'assfail' is a 'static inline' function and you
> > mentioned earlier that GCC seems to not be able to determine if it
> > returns or not. If CC_OPTIMIZE_FOR_SIZE decides to not inline, then
> > maybe this could be a problem? It could also be the compiler having
> > some trouble here with the CC_OPTIMIZE_FOR_SIZE + KCSAN combination.
>
> Even for a non-inlined static function, GCC typically detects when it's
> implicitly "noreturn", and optimizes the call sites accordingly. And
> that has also been true even for CC_OPTIMIZE_FOR_SIZE in the past. So
> something changed apparently. (KCSAN was just a guess.)

I'm actually seeing this issue pop up recently in other places, without
KCSAN enabled. So it may just be a new GCC bug (albeit a very minor
one). Sorry for blaming KCSAN :-) I'll need to dig some more.

The easy fix would be something like:

diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index eb8bd0258360..4db39fef3b56 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -655,7 +655,7 @@ alloc_extent_state_atomic(struct extent_state *prealloc)
return prealloc;
}

-static void extent_io_tree_panic(struct extent_io_tree *tree, int err)
+static void __noreturn extent_io_tree_panic(struct extent_io_tree *tree, int err)
{
struct inode *inode = tree->private_data;

diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index d897a8e5e430..b7a94b1739ae 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -321,7 +321,7 @@ static struct rb_node *tree_search(struct rb_root *root, u64 bytenr)
return NULL;
}

-static void backref_tree_panic(struct rb_node *rb_node, int errno, u64 bytenr)
+static void __noreturn backref_tree_panic(struct rb_node *rb_node, int errno, u64 bytenr)
{

struct btrfs_fs_info *fs_info = NULL;