Hi all,
After merging the modules tree, today's linux-next build (powerpc
allyesconfig) failed like this:
In file included from include/linux/export.h:123,
from include/linux/linkage.h:7,
from arch/powerpc/include/asm/unistd.h:18,
from <stdin>:2:
./include/generated/autoksyms.h:5:9: warning: missing whitespace after the macro name
5 | #define __KSYM_.HT_update_self_and_peer_setting 1
| ^~~~~~~
./include/generated/autoksyms.h:6:9: warning: missing whitespace after the macro name
6 | #define __KSYM_.RemovePeerTS 1
| ^~~~~~~
./include/generated/autoksyms.h:6: warning: "__KSYM_" redefined
6 | #define __KSYM_.RemovePeerTS 1
|
./include/generated/autoksyms.h:5: note: this is the location of the previous definition
and on and on :-(
Caused by commit
367948220fce ("module: remove EXPORT_UNUSED_SYMBOL*")
I have reverted that commit for today.
--
Cheers,
Stephen Rothwell
+++ Stephen Rothwell [09/02/21 21:08 +1100]:
>Hi all,
>
>After merging the modules tree, today's linux-next build (powerpc
>allyesconfig) failed like this:
>
>In file included from include/linux/export.h:123,
> from include/linux/linkage.h:7,
> from arch/powerpc/include/asm/unistd.h:18,
> from <stdin>:2:
>./include/generated/autoksyms.h:5:9: warning: missing whitespace after the macro name
> 5 | #define __KSYM_.HT_update_self_and_peer_setting 1
> | ^~~~~~~
>./include/generated/autoksyms.h:6:9: warning: missing whitespace after the macro name
> 6 | #define __KSYM_.RemovePeerTS 1
> | ^~~~~~~
>./include/generated/autoksyms.h:6: warning: "__KSYM_" redefined
> 6 | #define __KSYM_.RemovePeerTS 1
> |
>./include/generated/autoksyms.h:5: note: this is the location of the previous definition
>
>and on and on :-(
>
>Caused by commit
>
> 367948220fce ("module: remove EXPORT_UNUSED_SYMBOL*")
>
>I have reverted that commit for today.
[ Adding Michael and Masahiro to CC ]
Hi Stephen,
Hmm, these errors don't look like it's related to that particular commit. I was
able to reproduce these weird autoksym errors even without any modules-next
patches applied, and on a clean v5.11-rc7 tree. To reproduce it,
CONFIG_TRIM_UNUSED_KSYMS needs to be enabled. I guess that's why we run into
these errors with allyesconfig. I used a gcc-7 ppc64le cross compiler and got
the same compiler warnings. It seems to not compile on powerpc properly because
it looks like some symbols have an extra dot "." prefix, for example in
kthread.o:
168: 0000000000000318 24 NOTYPE GLOBAL DEFAULT 6 kthread_create_worker
169: 0000000000001d90 104 FUNC GLOBAL DEFAULT 1 .kthread_create_worker
170: 0000000000000330 24 NOTYPE GLOBAL DEFAULT 6 kthread_create_worker_on_cpu
171: 0000000000001e00 88 FUNC GLOBAL DEFAULT 1 .kthread_create_worker_on_cpu
172: 0000000000000348 24 NOTYPE GLOBAL DEFAULT 6 kthread_queue_work
173: 0000000000001e60 228 FUNC GLOBAL DEFAULT 1 .kthread_queue_work
So I suppose this dot prefix is specific to powerpc. From the ppc64 elf abi docs:
Symbol names with a dot (.) prefix are reserved for holding entry point
addresses. The value of a symbol named ".FN", if it exists, is the entry point
of the function "FN".
I guess the presence of the extra dot symbols is confusing
scripts/gen_autoksyms.sh, so we get the dot symbols in autoksyms.h, which the
preprocessor doesn't like. I am wondering how this was never caught until now
and also now curious if this feature was ever functional on powerpc..
Thanks,
Jessica
Hi Jessica,
On Tue, 9 Feb 2021 16:16:20 +0100 Jessica Yu <[email protected]> wrote:
>
> Hmm, these errors don't look like it's related to that particular commit. I was
I found this commit by bisection and then tested by reverting it.
Before this commit, CONFIG_TRIM_UNUSED_KSYMS would not be set in the
allyesconfig build because CONFIG_UNUSED_SYMBOLS was set. After this
commit, CONFIG_TRIM_UNUSED_KSYMS will be set in the allyesconfig build.
> able to reproduce these weird autoksym errors even without any modules-next
> patches applied, and on a clean v5.11-rc7 tree. To reproduce it,
> CONFIG_TRIM_UNUSED_KSYMS needs to be enabled. I guess that's why we run into
> these errors with allyesconfig. I used a gcc-7 ppc64le cross compiler and got
> the same compiler warnings. It seems to not compile on powerpc properly because
> it looks like some symbols have an extra dot "." prefix, for example in
> kthread.o:
>
> 168: 0000000000000318 24 NOTYPE GLOBAL DEFAULT 6 kthread_create_worker
> 169: 0000000000001d90 104 FUNC GLOBAL DEFAULT 1 .kthread_create_worker
> 170: 0000000000000330 24 NOTYPE GLOBAL DEFAULT 6 kthread_create_worker_on_cpu
> 171: 0000000000001e00 88 FUNC GLOBAL DEFAULT 1 .kthread_create_worker_on_cpu
> 172: 0000000000000348 24 NOTYPE GLOBAL DEFAULT 6 kthread_queue_work
> 173: 0000000000001e60 228 FUNC GLOBAL DEFAULT 1 .kthread_queue_work
>
> So I suppose this dot prefix is specific to powerpc. From the ppc64 elf abi docs:
>
> Symbol names with a dot (.) prefix are reserved for holding entry point
> addresses. The value of a symbol named ".FN", if it exists, is the entry point
> of the function "FN".
>
> I guess the presence of the extra dot symbols is confusing
> scripts/gen_autoksyms.sh, so we get the dot symbols in autoksyms.h, which the
> preprocessor doesn't like. I am wondering how this was never caught until now
> and also now curious if this feature was ever functional on powerpc..
Which feature?
--
Cheers,
Stephen Rothwell
On Tue, Feb 09, 2021 at 04:16:20PM +0100, Jessica Yu wrote:
> Hmm, these errors don't look like it's related to that particular commit. I was
> able to reproduce these weird autoksym errors even without any modules-next
> patches applied, and on a clean v5.11-rc7 tree.
I OTOH can't reproduce it at all with either linux-next or module-next
and the powerpc allyesconfig..
+++ Stephen Rothwell [10/02/21 08:50 +1100]:
>Hi Jessica,
>
>On Tue, 9 Feb 2021 16:16:20 +0100 Jessica Yu <[email protected]> wrote:
>>
>> Hmm, these errors don't look like it's related to that particular commit. I was
>
>I found this commit by bisection and then tested by reverting it.
>
>Before this commit, CONFIG_TRIM_UNUSED_KSYMS would not be set in the
>allyesconfig build because CONFIG_UNUSED_SYMBOLS was set. After this
>commit, CONFIG_TRIM_UNUSED_KSYMS will be set in the allyesconfig build.
Ah, that makes sense then. I would get the error on powerpc whenever
CONFIG_TRIM_UNUSED_KSYMS was enabled.
>> able to reproduce these weird autoksym errors even without any modules-next
>> patches applied, and on a clean v5.11-rc7 tree. To reproduce it,
>> CONFIG_TRIM_UNUSED_KSYMS needs to be enabled. I guess that's why we run into
>> these errors with allyesconfig. I used a gcc-7 ppc64le cross compiler and got
>> the same compiler warnings. It seems to not compile on powerpc properly because
>> it looks like some symbols have an extra dot "." prefix, for example in
>> kthread.o:
>>
>> 168: 0000000000000318 24 NOTYPE GLOBAL DEFAULT 6 kthread_create_worker
>> 169: 0000000000001d90 104 FUNC GLOBAL DEFAULT 1 .kthread_create_worker
>> 170: 0000000000000330 24 NOTYPE GLOBAL DEFAULT 6 kthread_create_worker_on_cpu
>> 171: 0000000000001e00 88 FUNC GLOBAL DEFAULT 1 .kthread_create_worker_on_cpu
>> 172: 0000000000000348 24 NOTYPE GLOBAL DEFAULT 6 kthread_queue_work
>> 173: 0000000000001e60 228 FUNC GLOBAL DEFAULT 1 .kthread_queue_work
>>
>> So I suppose this dot prefix is specific to powerpc. From the ppc64 elf abi docs:
>>
>> Symbol names with a dot (.) prefix are reserved for holding entry point
>> addresses. The value of a symbol named ".FN", if it exists, is the entry point
>> of the function "FN".
>>
>> I guess the presence of the extra dot symbols is confusing
>> scripts/gen_autoksyms.sh, so we get the dot symbols in autoksyms.h, which the
>> preprocessor doesn't like. I am wondering how this was never caught until now
>> and also now curious if this feature was ever functional on powerpc..
>
>Which feature?
Sorry, by "feature" I meant CONFIG_TRIM_UNUSED_KSYMS. This config
option was introduced around v4.7. If simply enabling it produces
these compilation errors I was wondering if it ever built properly on
powerpc.
Thanks,
Jessica
On Wed, Feb 10, 2021 at 5:06 PM Jessica Yu <[email protected]> wrote:
>
> +++ Stephen Rothwell [10/02/21 08:50 +1100]:
> >Hi Jessica,
> >
> >On Tue, 9 Feb 2021 16:16:20 +0100 Jessica Yu <[email protected]> wrote:
> >>
> >> Hmm, these errors don't look like it's related to that particular commit. I was
> >
> >I found this commit by bisection and then tested by reverting it.
> >
> >Before this commit, CONFIG_TRIM_UNUSED_KSYMS would not be set in the
> >allyesconfig build because CONFIG_UNUSED_SYMBOLS was set. After this
> >commit, CONFIG_TRIM_UNUSED_KSYMS will be set in the allyesconfig build.
>
> Ah, that makes sense then. I would get the error on powerpc whenever
> CONFIG_TRIM_UNUSED_KSYMS was enabled.
>
> >> able to reproduce these weird autoksym errors even without any modules-next
> >> patches applied, and on a clean v5.11-rc7 tree. To reproduce it,
> >> CONFIG_TRIM_UNUSED_KSYMS needs to be enabled. I guess that's why we run into
> >> these errors with allyesconfig. I used a gcc-7 ppc64le cross compiler and got
> >> the same compiler warnings. It seems to not compile on powerpc properly because
> >> it looks like some symbols have an extra dot "." prefix, for example in
> >> kthread.o:
> >>
> >> 168: 0000000000000318 24 NOTYPE GLOBAL DEFAULT 6 kthread_create_worker
> >> 169: 0000000000001d90 104 FUNC GLOBAL DEFAULT 1 .kthread_create_worker
> >> 170: 0000000000000330 24 NOTYPE GLOBAL DEFAULT 6 kthread_create_worker_on_cpu
> >> 171: 0000000000001e00 88 FUNC GLOBAL DEFAULT 1 .kthread_create_worker_on_cpu
> >> 172: 0000000000000348 24 NOTYPE GLOBAL DEFAULT 6 kthread_queue_work
> >> 173: 0000000000001e60 228 FUNC GLOBAL DEFAULT 1 .kthread_queue_work
> >>
> >> So I suppose this dot prefix is specific to powerpc. From the ppc64 elf abi docs:
> >>
> >> Symbol names with a dot (.) prefix are reserved for holding entry point
> >> addresses. The value of a symbol named ".FN", if it exists, is the entry point
> >> of the function "FN".
> >>
> >> I guess the presence of the extra dot symbols is confusing
> >> scripts/gen_autoksyms.sh, so we get the dot symbols in autoksyms.h, which the
> >> preprocessor doesn't like. I am wondering how this was never caught until now
> >> and also now curious if this feature was ever functional on powerpc..
> >
> >Which feature?
>
> Sorry, by "feature" I meant CONFIG_TRIM_UNUSED_KSYMS. This config
> option was introduced around v4.7. If simply enabling it produces
> these compilation errors I was wondering if it ever built properly on
> powerpc.
>
> Thanks,
>
> Jessica
Thanks for the report.
I think the following will fix the issue,
but modpost needs fixing too.
diff --git a/scripts/gen_autoksyms.sh b/scripts/gen_autoksyms.sh
index 16c0b2ddaa4c..996a7109167b 100755
--- a/scripts/gen_autoksyms.sh
+++ b/scripts/gen_autoksyms.sh
@@ -44,7 +44,7 @@ sed 's/ko$/mod/' $modlist |
xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- |
cat - "$ksym_wl" |
sort -u |
-sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$output_file"
+sed -e 's/^\.\{,1\}\(.*\)/#define __KSYM_\1 1/' >> "$output_file"
# Special case for modversions (see modpost.c)
if [ -n "$CONFIG_MODVERSIONS" ]; then
m
I will post two patches with some commit log
after some testing.
(one for gen_autoksyms.sh and the other for modpost).
--
Best Regards
Masahiro Yamada
Hi Jessica,
On Wed, 10 Feb 2021 09:06:48 +0100 Jessica Yu <[email protected]> wrote:
>
> Sorry, by "feature" I meant CONFIG_TRIM_UNUSED_KSYMS. This config
> option was introduced around v4.7. If simply enabling it produces
> these compilation errors I was wondering if it ever built properly on
> powerpc.
Ah, of course. So for a quick fix, you could revert just the changes
to lib/Kconfig and all the defconfigs. That way all the UNUSED_SYMBOLS
infrastructure is still removed, but TRIM_UNUSED_KSYMS remains (un)set
whenever it used to be (un)set and that could then be cleaned up in a
followup patch set per architecture when we know it works.
--
Cheers,
Stephen Rothwell
+++ Stephen Rothwell [10/02/21 23:21 +1100]:
>Hi Jessica,
>
>On Wed, 10 Feb 2021 09:06:48 +0100 Jessica Yu <[email protected]> wrote:
>>
>> Sorry, by "feature" I meant CONFIG_TRIM_UNUSED_KSYMS. This config
>> option was introduced around v4.7. If simply enabling it produces
>> these compilation errors I was wondering if it ever built properly on
>> powerpc.
>
>Ah, of course. So for a quick fix, you could revert just the changes
>to lib/Kconfig and all the defconfigs. That way all the UNUSED_SYMBOLS
>infrastructure is still removed, but TRIM_UNUSED_KSYMS remains (un)set
>whenever it used to be (un)set and that could then be cleaned up in a
>followup patch set per architecture when we know it works.
Hi Stephen,
What's maybe simpler is marking this as BROKEN on PPC64 as a temporary
fix for linux-next until Masahiro's patch (the real fix) is posted. I
think the real problem is that TRIM_UNUSED_KSYMS was pretty much
hiding behind UNUSED_SYMBOLS=y until it finally got removed. Anyway
the main point is that TRIM_UNUSED_KSYMS=y shouldn't be breaking any
builds. I've pushed this change and hope that won't break linux-next now.
---
From 1fa67f8391acb88a54e7defe6b73f8f171450f4a Mon Sep 17 00:00:00 2001
From: Jessica Yu <[email protected]>
Date: Wed, 10 Feb 2021 16:59:53 +0100
Subject: [PATCH] module: mark TRIM_UNUSED_KSYMS as BROKEN on powerpc
Commit 367948220fce, which removed CONFIG_UNUSED_SYMBOLS, unraveled the
TRIM_UNUSED_KSYMS config option to be generally available without depending
on CONFIG_UNUSED_SYMBOLS=y. With CONFIG_UNUSED_SYMBOLS gone, this means
allyesconfig will now select this option without being "behind"
CONFIG_UNUSED_SYMBOLS. Unfortunately, this revealed some underlying build
issues on powerpc. When CONFIG_TRIM_UNUSED_KSYMS is enabled on powerpc, the
build would fail like this:
In file included from include/linux/export.h:123,
from include/linux/linkage.h:7,
from arch/powerpc/include/asm/unistd.h:18,
from <stdin>:2:
./include/generated/autoksyms.h:5:9: warning: missing whitespace after the
macro name
5 | #define __KSYM_.HT_update_self_and_peer_setting 1
| ^~~~~~~
./include/generated/autoksyms.h:6:9: warning: missing whitespace after the
macro name
6 | #define __KSYM_.RemovePeerTS 1
| ^~~~~~~
./include/generated/autoksyms.h:6: warning: "__KSYM_" redefined
6 | #define __KSYM_.RemovePeerTS 1
|
./include/generated/autoksyms.h:5: note: this is the location of the
previous definition
This is due to the addition of a dot "." prefix to some symbols that is
specific to powerpc, which confuses the gen_autoksyms script.
This is a temporary workaround for linux-next until gen_autoksyms and
modpost gets fixed.
Link: http://lore.kernel.org/r/[email protected]
Signed-off-by: Jessica Yu <[email protected]>
---
init/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/init/Kconfig b/init/Kconfig
index 11b803b45c19..e4504a04b601 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2264,6 +2264,7 @@ config MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
config TRIM_UNUSED_KSYMS
bool "Trim unused exported kernel symbols"
+ depends on !PPC64 || BROKEN
help
The kernel and some modules make many symbols available for
other modules to use via EXPORT_SYMBOL() and variants. Depending
On Wed, Feb 10, 2021 at 5:37 PM Masahiro Yamada <[email protected]> wrote:
>
> On Wed, Feb 10, 2021 at 5:06 PM Jessica Yu <[email protected]> wrote:
> >
> > +++ Stephen Rothwell [10/02/21 08:50 +1100]:
> > >Hi Jessica,
> > >
> > >On Tue, 9 Feb 2021 16:16:20 +0100 Jessica Yu <[email protected]> wrote:
> > >>
> > >> Hmm, these errors don't look like it's related to that particular commit. I was
> > >
> > >I found this commit by bisection and then tested by reverting it.
> > >
> > >Before this commit, CONFIG_TRIM_UNUSED_KSYMS would not be set in the
> > >allyesconfig build because CONFIG_UNUSED_SYMBOLS was set. After this
> > >commit, CONFIG_TRIM_UNUSED_KSYMS will be set in the allyesconfig build.
> >
> > Ah, that makes sense then. I would get the error on powerpc whenever
> > CONFIG_TRIM_UNUSED_KSYMS was enabled.
> >
> > >> able to reproduce these weird autoksym errors even without any modules-next
> > >> patches applied, and on a clean v5.11-rc7 tree. To reproduce it,
> > >> CONFIG_TRIM_UNUSED_KSYMS needs to be enabled. I guess that's why we run into
> > >> these errors with allyesconfig. I used a gcc-7 ppc64le cross compiler and got
> > >> the same compiler warnings. It seems to not compile on powerpc properly because
> > >> it looks like some symbols have an extra dot "." prefix, for example in
> > >> kthread.o:
> > >>
> > >> 168: 0000000000000318 24 NOTYPE GLOBAL DEFAULT 6 kthread_create_worker
> > >> 169: 0000000000001d90 104 FUNC GLOBAL DEFAULT 1 .kthread_create_worker
> > >> 170: 0000000000000330 24 NOTYPE GLOBAL DEFAULT 6 kthread_create_worker_on_cpu
> > >> 171: 0000000000001e00 88 FUNC GLOBAL DEFAULT 1 .kthread_create_worker_on_cpu
> > >> 172: 0000000000000348 24 NOTYPE GLOBAL DEFAULT 6 kthread_queue_work
> > >> 173: 0000000000001e60 228 FUNC GLOBAL DEFAULT 1 .kthread_queue_work
> > >>
> > >> So I suppose this dot prefix is specific to powerpc. From the ppc64 elf abi docs:
> > >>
> > >> Symbol names with a dot (.) prefix are reserved for holding entry point
> > >> addresses. The value of a symbol named ".FN", if it exists, is the entry point
> > >> of the function "FN".
> > >>
> > >> I guess the presence of the extra dot symbols is confusing
> > >> scripts/gen_autoksyms.sh, so we get the dot symbols in autoksyms.h, which the
> > >> preprocessor doesn't like. I am wondering how this was never caught until now
> > >> and also now curious if this feature was ever functional on powerpc..
> > >
> > >Which feature?
> >
> > Sorry, by "feature" I meant CONFIG_TRIM_UNUSED_KSYMS. This config
> > option was introduced around v4.7. If simply enabling it produces
> > these compilation errors I was wondering if it ever built properly on
> > powerpc.
> >
> > Thanks,
> >
> > Jessica
>
>
> Thanks for the report.
>
> I think the following will fix the issue,
> but modpost needs fixing too.
>
>
> diff --git a/scripts/gen_autoksyms.sh b/scripts/gen_autoksyms.sh
> index 16c0b2ddaa4c..996a7109167b 100755
> --- a/scripts/gen_autoksyms.sh
> +++ b/scripts/gen_autoksyms.sh
> @@ -44,7 +44,7 @@ sed 's/ko$/mod/' $modlist |
> xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- |
> cat - "$ksym_wl" |
> sort -u |
> -sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$output_file"
> +sed -e 's/^\.\{,1\}\(.*\)/#define __KSYM_\1 1/' >> "$output_file"
>
> # Special case for modversions (see modpost.c)
> if [ -n "$CONFIG_MODVERSIONS" ]; then
> m
After some more tests, I noticed the code above was not correct.
I still saw a lot of modpost errors.
ERROR: modpost: "_mcount" [arch/powerpc/oprofile/oprofile.ko] undefined!
ERROR: modpost: "._mcount" [arch/powerpc/oprofile/oprofile.ko] undefined!
I just posted a patch, which fixes the error as far as I tested.
https://patchwork.kernel.org/project/linux-kbuild/patch/[email protected]/
--
Best Regards
Masahiro Yamada