When build testing across architectures, I run into a build error on
all targets other than X86:
gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target
The problem is that 'hostprogs' get built with 'gcc' rather than
'$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.
To work around it, adding an X86 dependency gets randconfigs building
again on my box.
Clearly, this is not a good solution, since it should actually work fine
when building native kernels on other architectures but that is now
disabled, while cross building an x86 kernel on another host is still
broken after my patch.
What we probably want here is to try out if the compiler is able to build
executables for the target architecture and not build the helper otherwise,
at least when compile-testing. No idea how to do that though.
Link: http://www.kernel.org/pub/tools/crosstool/
Cc: Masahiro Yamada <[email protected]>
Cc: [email protected]
Signed-off-by: Arnd Bergmann <[email protected]>
---
net/bpfilter/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/net/bpfilter/Kconfig b/net/bpfilter/Kconfig
index 60725c5f79db..61cc4fcbb4d0 100644
--- a/net/bpfilter/Kconfig
+++ b/net/bpfilter/Kconfig
@@ -9,6 +9,7 @@ menuconfig BPFILTER
if BPFILTER
config BPFILTER_UMH
tristate "bpfilter kernel module with user mode helper"
+ depends on X86 # actually depends on native builds
default m
help
This builds bpfilter kernel module with embedded user mode helper
--
2.9.0
On Mon, May 28, 2018 at 05:31:01PM +0200, Arnd Bergmann wrote:
> When build testing across architectures, I run into a build error on
> all targets other than X86:
>
> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target
>
> The problem is that 'hostprogs' get built with 'gcc' rather than
> '$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.
>
> To work around it, adding an X86 dependency gets randconfigs building
> again on my box.
>
> Clearly, this is not a good solution, since it should actually work fine
> when building native kernels on other architectures but that is now
> disabled, while cross building an x86 kernel on another host is still
> broken after my patch.
>
> What we probably want here is to try out if the compiler is able to build
> executables for the target architecture and not build the helper otherwise,
> at least when compile-testing. No idea how to do that though.
>
> Link: http://www.kernel.org/pub/tools/crosstool/
> Cc: Masahiro Yamada <[email protected]>
> Cc: [email protected]
> Signed-off-by: Arnd Bergmann <[email protected]>
> ---
> net/bpfilter/Kconfig | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/net/bpfilter/Kconfig b/net/bpfilter/Kconfig
> index 60725c5f79db..61cc4fcbb4d0 100644
> --- a/net/bpfilter/Kconfig
> +++ b/net/bpfilter/Kconfig
> @@ -9,6 +9,7 @@ menuconfig BPFILTER
> if BPFILTER
> config BPFILTER_UMH
> tristate "bpfilter kernel module with user mode helper"
> + depends on X86 # actually depends on native builds
depends on X86 will break it on arm.
I think the better short term fix would be to test that HOSTCC == CC
It doesn't have to be the same compiler. HOSTCC's arch == kernel ARCH
Not sure how to hack makefile to do that.
Long term we need to get rid of HOSTCC dependency.
2018-05-31 0:17 GMT+09:00 Alexei Starovoitov <[email protected]>:
> On Mon, May 28, 2018 at 05:31:01PM +0200, Arnd Bergmann wrote:
>> When build testing across architectures, I run into a build error on
>> all targets other than X86:
>>
>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target
>>
>> The problem is that 'hostprogs' get built with 'gcc' rather than
>> '$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.
>>
>> To work around it, adding an X86 dependency gets randconfigs building
>> again on my box.
>>
>> Clearly, this is not a good solution, since it should actually work fine
>> when building native kernels on other architectures but that is now
>> disabled, while cross building an x86 kernel on another host is still
>> broken after my patch.
>>
>> What we probably want here is to try out if the compiler is able to build
>> executables for the target architecture and not build the helper otherwise,
>> at least when compile-testing. No idea how to do that though.
>>
>> Link: http://www.kernel.org/pub/tools/crosstool/
>> Cc: Masahiro Yamada <[email protected]>
>> Cc: [email protected]
>> Signed-off-by: Arnd Bergmann <[email protected]>
>> ---
>> net/bpfilter/Kconfig | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/net/bpfilter/Kconfig b/net/bpfilter/Kconfig
>> index 60725c5f79db..61cc4fcbb4d0 100644
>> --- a/net/bpfilter/Kconfig
>> +++ b/net/bpfilter/Kconfig
>> @@ -9,6 +9,7 @@ menuconfig BPFILTER
>> if BPFILTER
>> config BPFILTER_UMH
>> tristate "bpfilter kernel module with user mode helper"
>> + depends on X86 # actually depends on native builds
>
> depends on X86 will break it on arm.
> I think the better short term fix would be to test that HOSTCC == CC
> It doesn't have to be the same compiler. HOSTCC's arch == kernel ARCH
> Not sure how to hack makefile to do that.
> Long term we need to get rid of HOSTCC dependency.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Hmm.
For cross-compiling, we set 'ARCH' via the environment variable or the
command line.
ARCH is not explicitly set, the top-level Makefile sets it to $(SUBARCH)
ARCH ?= $(SUBARCH)
Maybe, we can assume the native build if $(ARCH) and $(SUBARCH) are the same?
--
Best Regards
Masahiro Yamada
On Thu, May 31, 2018 at 3:42 AM, Masahiro Yamada
<[email protected]> wrote:
> 2018-05-31 0:17 GMT+09:00 Alexei Starovoitov <[email protected]>:
>> On Mon, May 28, 2018 at 05:31:01PM +0200, Arnd Bergmann wrote:
> Hmm.
> For cross-compiling, we set 'ARCH' via the environment variable or the
> command line.
>
> ARCH is not explicitly set, the top-level Makefile sets it to $(SUBARCH)
>
>
> ARCH ?= $(SUBARCH)
>
>
> Maybe, we can assume the native build if $(ARCH) and $(SUBARCH) are the same?
>
SUBARCH is also used with a special meaning for arch/um where we build
with ARCH=um SUBARCH=x86, either on native (x86) or cross builds.
So doing that would still work in most but not all cases.
What is the reason for using HOSTCC rather than CC anyway? I think
the correct way to do this would be to check if CC is able to link binaries
and disallow the option if it's not.
Don't we already do something like that for tools/testing/selftest which
also needs to generate binaries with CC?
Arnd
On Fri, Jun 01, 2018 at 05:20:12PM +0200, Arnd Bergmann wrote:
> On Thu, May 31, 2018 at 3:42 AM, Masahiro Yamada
> <[email protected]> wrote:
> > 2018-05-31 0:17 GMT+09:00 Alexei Starovoitov <[email protected]>:
> >> On Mon, May 28, 2018 at 05:31:01PM +0200, Arnd Bergmann wrote:
>
> > Hmm.
> > For cross-compiling, we set 'ARCH' via the environment variable or the
> > command line.
> >
> > ARCH is not explicitly set, the top-level Makefile sets it to $(SUBARCH)
> >
> >
> > ARCH ?= $(SUBARCH)
> >
> >
> > Maybe, we can assume the native build if $(ARCH) and $(SUBARCH) are the same?
> >
>
> SUBARCH is also used with a special meaning for arch/um where we build
> with ARCH=um SUBARCH=x86, either on native (x86) or cross builds.
>
>
> So doing that would still work in most but not all cases.
>
> What is the reason for using HOSTCC rather than CC anyway? I think
> the correct way to do this would be to check if CC is able to link binaries
> and disallow the option if it's not.
that's a great idea. Let's do that.
> Don't we already do something like that for tools/testing/selftest which
> also needs to generate binaries with CC?
I couldn't find such makefile magic. Can you please help me with this?
Hi Arnd, Alexei,
On Mon, May 28, 2018 at 5:31 PM, Arnd Bergmann <[email protected]> wrote:
> When build testing across architectures, I run into a build error on
> all targets other than X86:
>
> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target
>
> The problem is that 'hostprogs' get built with 'gcc' rather than
> '$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.
>
> To work around it, adding an X86 dependency gets randconfigs building
> again on my box.
>
> Clearly, this is not a good solution, since it should actually work fine
> when building native kernels on other architectures but that is now
> disabled, while cross building an x86 kernel on another host is still
> broken after my patch.
>
> What we probably want here is to try out if the compiler is able to build
> executables for the target architecture and not build the helper otherwise,
> at least when compile-testing. No idea how to do that though.
So that was done in commit 819dd92b9c0bc7bc ("bpfilter: switch to CC
from HOSTCC"), but it is not sufficient:
GEN net/bpfilter/bpfilter_umh.o
Usage: m68k-linux-gnu-objcopy [option(s)] in-file [out-file]
Copies a binary file, possibly transforming it in the process
The options are:
[...]
net/bpfilter/Makefile:29: recipe for target 'net/bpfilter/bpfilter_umh.o' failed
make[5]: *** [net/bpfilter/bpfilter_umh.o] Error 1
> --- a/net/bpfilter/Kconfig
> +++ b/net/bpfilter/Kconfig
> @@ -9,6 +9,7 @@ menuconfig BPFILTER
> if BPFILTER
> config BPFILTER_UMH
> tristate "bpfilter kernel module with user mode helper"
> + depends on X86 # actually depends on native builds
No, (currently) it does depend on X86, due to its use of:
$(OBJCOPY) -I binary -O $(CONFIG_OUTPUT_FORMAT)
with CONFIG_OUTPUT_FORMAT being defined on X86 only...
> default m
> help
> This builds bpfilter kernel module with embedded user mode helper
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Hi Geert,
On 06/08/2018 10:57 AM, Geert Uytterhoeven wrote:
> On Mon, May 28, 2018 at 5:31 PM, Arnd Bergmann <[email protected]> wrote:
>> When build testing across architectures, I run into a build error on
>> all targets other than X86:
>>
>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target
>>
>> The problem is that 'hostprogs' get built with 'gcc' rather than
>> '$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.
>>
>> To work around it, adding an X86 dependency gets randconfigs building
>> again on my box.
>>
>> Clearly, this is not a good solution, since it should actually work fine
>> when building native kernels on other architectures but that is now
>> disabled, while cross building an x86 kernel on another host is still
>> broken after my patch.
>>
>> What we probably want here is to try out if the compiler is able to build
>> executables for the target architecture and not build the helper otherwise,
>> at least when compile-testing. No idea how to do that though.
>
> So that was done in commit 819dd92b9c0bc7bc ("bpfilter: switch to CC
> from HOSTCC"), but it is not sufficient:
>
> GEN net/bpfilter/bpfilter_umh.o
> Usage: m68k-linux-gnu-objcopy [option(s)] in-file [out-file]
> Copies a binary file, possibly transforming it in the process
> The options are:
> [...]
>
> net/bpfilter/Makefile:29: recipe for target 'net/bpfilter/bpfilter_umh.o' failed
> make[5]: *** [net/bpfilter/bpfilter_umh.o] Error 1
>
>> --- a/net/bpfilter/Kconfig
>> +++ b/net/bpfilter/Kconfig
>> @@ -9,6 +9,7 @@ menuconfig BPFILTER
>> if BPFILTER
>> config BPFILTER_UMH
>> tristate "bpfilter kernel module with user mode helper"
>> + depends on X86 # actually depends on native builds
>
> No, (currently) it does depend on X86, due to its use of:
>
> $(OBJCOPY) -I binary -O $(CONFIG_OUTPUT_FORMAT)
>
> with CONFIG_OUTPUT_FORMAT being defined on X86 only...
That hard dependency should have been fixed with the following patch in -net tree:
https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=8d97ca6b6755bf7ef57d323642ca9ee80d689782
Thanks,
Daniel
Hi Daniel,
On Fri, Jun 8, 2018 at 11:33 AM, Daniel Borkmann <[email protected]> wrote:
> On 06/08/2018 10:57 AM, Geert Uytterhoeven wrote:
>> On Mon, May 28, 2018 at 5:31 PM, Arnd Bergmann <[email protected]> wrote:
>>> When build testing across architectures, I run into a build error on
>>> all targets other than X86:
>>>
>>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
>>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target
>>>
>>> The problem is that 'hostprogs' get built with 'gcc' rather than
>>> '$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.
>>>
>>> To work around it, adding an X86 dependency gets randconfigs building
>>> again on my box.
>>>
>>> Clearly, this is not a good solution, since it should actually work fine
>>> when building native kernels on other architectures but that is now
>>> disabled, while cross building an x86 kernel on another host is still
>>> broken after my patch.
>>>
>>> What we probably want here is to try out if the compiler is able to build
>>> executables for the target architecture and not build the helper otherwise,
>>> at least when compile-testing. No idea how to do that though.
>>
>> So that was done in commit 819dd92b9c0bc7bc ("bpfilter: switch to CC
>> from HOSTCC"), but it is not sufficient:
>>
>> GEN net/bpfilter/bpfilter_umh.o
>> Usage: m68k-linux-gnu-objcopy [option(s)] in-file [out-file]
>> Copies a binary file, possibly transforming it in the process
>> The options are:
>> [...]
>>
>> net/bpfilter/Makefile:29: recipe for target 'net/bpfilter/bpfilter_umh.o' failed
>> make[5]: *** [net/bpfilter/bpfilter_umh.o] Error 1
>>
>>> --- a/net/bpfilter/Kconfig
>>> +++ b/net/bpfilter/Kconfig
>>> @@ -9,6 +9,7 @@ menuconfig BPFILTER
>>> if BPFILTER
>>> config BPFILTER_UMH
>>> tristate "bpfilter kernel module with user mode helper"
>>> + depends on X86 # actually depends on native builds
>>
>> No, (currently) it does depend on X86, due to its use of:
>>
>> $(OBJCOPY) -I binary -O $(CONFIG_OUTPUT_FORMAT)
>>
>> with CONFIG_OUTPUT_FORMAT being defined on X86 only...
>
> That hard dependency should have been fixed with the following patch in -net tree:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=8d97ca6b6755bf7ef57d323642ca9ee80d689782
Thanks, confirmed.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds