Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
resulting in build failures in hermetic environments where `cc` does not
exist. This includes both hostprogs and proc-macros.
Since we are setting the linker to `HOSTCC`, we set the linker flavor to
`gcc` explicitly.
Signed-off-by: Matthew Maurer <[email protected]>
---
Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS
should be respected as well.
I did not switch it to use HOSTLD for two reasons:
* That variable is not globally defined - it is only available in two
subdirectories of tools/
* C host scripts are linked by HOSTCC as well, even when linking a
collection of object files. It *prints* HOSTLD, but invokes HOSTCC.
See scripts/Makefile.host cmd_host-cmulti for an example.
rust/Makefile | 4 ++++
scripts/Makefile.host | 4 ++++
2 files changed, 8 insertions(+)
diff --git a/rust/Makefile b/rust/Makefile
index 87958e864be0..b60b7eb8c5a0 100644
--- a/rust/Makefile
+++ b/rust/Makefile
@@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
$(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
$(call if_changed,exports)
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
+
quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
cmd_rustc_procmacro = \
$(RUSTC_OR_CLIPPY) $(rust_common_flags) \
+ -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+ -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
--emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
--crate-type proc-macro \
--crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
diff --git a/scripts/Makefile.host b/scripts/Makefile.host
index 8f7f842b54f9..dc0410cae5ca 100644
--- a/scripts/Makefile.host
+++ b/scripts/Makefile.host
@@ -87,10 +87,14 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \
$(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
$(HOSTCXXFLAGS_$(target-stem).o)
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
+
# `--out-dir` is required to avoid temporaries being created by `rustc` in the
# current working directory, which may be not accessible in the out-of-tree
# modules case.
hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
+ -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+ -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
$(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
$(HOSTRUSTFLAGS_$(target-stem))
--
2.42.0.459.ge4e396fd5e-goog
On Tue, Sep 19, 2023 at 12:44 PM Nick Desaulniers
<[email protected]> wrote:
>
> On Mon, Sep 18, 2023 at 4:44 PM Matthew Maurer <[email protected]> wrote:
> >
> > Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
> > resulting in build failures in hermetic environments where `cc` does not
> > exist. This includes both hostprogs and proc-macros.
> >
> > Since we are setting the linker to `HOSTCC`, we set the linker flavor to
> > `gcc` explicitly.
> >
> > Signed-off-by: Matthew Maurer <[email protected]>
> > ---
> >
> > Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS
> > should be respected as well.
> >
> > I did not switch it to use HOSTLD for two reasons:
> > * That variable is not globally defined - it is only available in two
> > subdirectories of tools/
> > * C host scripts are linked by HOSTCC as well, even when linking a
> > collection of object files. It *prints* HOSTLD, but invokes HOSTCC.
> > See scripts/Makefile.host cmd_host-cmulti for an example.
>
> Sure, that makes sense to me, thanks for pointing that out.
>
> >
> > rust/Makefile | 4 ++++
> > scripts/Makefile.host | 4 ++++
> > 2 files changed, 8 insertions(+)
> >
> > diff --git a/rust/Makefile b/rust/Makefile
> > index 87958e864be0..b60b7eb8c5a0 100644
> > --- a/rust/Makefile
> > +++ b/rust/Makefile
> > @@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
> > $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
> > $(call if_changed,exports)
> >
> > +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
>
> I don't think we need to do this kind of escaping. If a linker flag is
> passed to the linker directly as the driver has spaces such as `-z
> relro`, if it is instead passed to the compiler as the driver will not
> contain spaces (`-Wl,-z,relro`). As such, I don't think we need this
> escaping (famous last words). Mind submitting a v3 without it?
Thanks for clarifying off list that:
```
It's not escaping the spaces, it's escaping any quotes
example: -Clink-args='-Wl,-z,relro -Wl,-linker-plugin-lto'
Without the quote, the second one would go to rustc rather than the linker
```
Sorry for misunderstanding what exactly we were escaping here and why;
thanks for implementing my suggestion of passing along
KBUILD_HOSTLDFLAGS though. That will avoid issues for Android
downstream.
Reviewed-by: Nick Desaulniers <[email protected]>
>
> > +
> > quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
> > cmd_rustc_procmacro = \
> > $(RUSTC_OR_CLIPPY) $(rust_common_flags) \
> > + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> > + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
> > --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
> > --crate-type proc-macro \
> > --crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
> > diff --git a/scripts/Makefile.host b/scripts/Makefile.host
> > index 8f7f842b54f9..dc0410cae5ca 100644
> > --- a/scripts/Makefile.host
> > +++ b/scripts/Makefile.host
> > @@ -87,10 +87,14 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \
> > $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
> > $(HOSTCXXFLAGS_$(target-stem).o)
> >
> > +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
> > +
> > # `--out-dir` is required to avoid temporaries being created by `rustc` in the
> > # current working directory, which may be not accessible in the out-of-tree
> > # modules case.
> > hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
> > + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> > + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
> > $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
> > $(HOSTRUSTFLAGS_$(target-stem))
> >
> > --
> > 2.42.0.459.ge4e396fd5e-goog
> >
>
>
> --
> Thanks,
> ~Nick Desaulniers
--
Thanks,
~Nick Desaulniers
On 9/18/23 20:43, Matthew Maurer wrote:
> Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
> resulting in build failures in hermetic environments where `cc` does not
> exist. This includes both hostprogs and proc-macros.
>
> Since we are setting the linker to `HOSTCC`, we set the linker flavor to
> `gcc` explicitly.
>
> Signed-off-by: Matthew Maurer <[email protected]>
> ---
>
> Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS
> should be respected as well.
>
> I did not switch it to use HOSTLD for two reasons:
> * That variable is not globally defined - it is only available in two
> subdirectories of tools/
> * C host scripts are linked by HOSTCC as well, even when linking a
> collection of object files. It *prints* HOSTLD, but invokes HOSTCC.
> See scripts/Makefile.host cmd_host-cmulti for an example.
> [...]
Reviewed-by: Martin Rodriguez Reboredo <[email protected]>
On Mon, Sep 18, 2023 at 4:44 PM Matthew Maurer <[email protected]> wrote:
>
> Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
> resulting in build failures in hermetic environments where `cc` does not
> exist. This includes both hostprogs and proc-macros.
>
> Since we are setting the linker to `HOSTCC`, we set the linker flavor to
> `gcc` explicitly.
>
> Signed-off-by: Matthew Maurer <[email protected]>
> ---
>
> Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS
> should be respected as well.
>
> I did not switch it to use HOSTLD for two reasons:
> * That variable is not globally defined - it is only available in two
> subdirectories of tools/
> * C host scripts are linked by HOSTCC as well, even when linking a
> collection of object files. It *prints* HOSTLD, but invokes HOSTCC.
> See scripts/Makefile.host cmd_host-cmulti for an example.
Sure, that makes sense to me, thanks for pointing that out.
>
> rust/Makefile | 4 ++++
> scripts/Makefile.host | 4 ++++
> 2 files changed, 8 insertions(+)
>
> diff --git a/rust/Makefile b/rust/Makefile
> index 87958e864be0..b60b7eb8c5a0 100644
> --- a/rust/Makefile
> +++ b/rust/Makefile
> @@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
> $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
> $(call if_changed,exports)
>
> +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
I don't think we need to do this kind of escaping. If a linker flag is
passed to the linker directly as the driver has spaces such as `-z
relro`, if it is instead passed to the compiler as the driver will not
contain spaces (`-Wl,-z,relro`). As such, I don't think we need this
escaping (famous last words). Mind submitting a v3 without it?
> +
> quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
> cmd_rustc_procmacro = \
> $(RUSTC_OR_CLIPPY) $(rust_common_flags) \
> + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
> --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
> --crate-type proc-macro \
> --crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
> diff --git a/scripts/Makefile.host b/scripts/Makefile.host
> index 8f7f842b54f9..dc0410cae5ca 100644
> --- a/scripts/Makefile.host
> +++ b/scripts/Makefile.host
> @@ -87,10 +87,14 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \
> $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
> $(HOSTCXXFLAGS_$(target-stem).o)
>
> +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
> +
> # `--out-dir` is required to avoid temporaries being created by `rustc` in the
> # current working directory, which may be not accessible in the out-of-tree
> # modules case.
> hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
> + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
> $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
> $(HOSTRUSTFLAGS_$(target-stem))
>
> --
> 2.42.0.459.ge4e396fd5e-goog
>
--
Thanks,
~Nick Desaulniers
Matthew Maurer <[email protected]> writes:
> Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
> resulting in build failures in hermetic environments where `cc` does not
> exist. This includes both hostprogs and proc-macros.
>
> Since we are setting the linker to `HOSTCC`, we set the linker flavor to
> `gcc` explicitly.
>
> Signed-off-by: Matthew Maurer <[email protected]>
Tested-by: Alice Ryhl <[email protected]>
On Tue, Sep 19, 2023 at 8:44 AM Matthew Maurer <[email protected]> wrote:
>
> Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined,
> resulting in build failures in hermetic environments where `cc` does not
> exist. This includes both hostprogs and proc-macros.
>
> Since we are setting the linker to `HOSTCC`, we set the linker flavor to
> `gcc` explicitly.
This sentence is unclear to me because it does not explain
why we need both despite the spec mentions
'linker-flavor' is inferred from the value of 'linker'.
The answer exists in this thread.
"
As mentioned earlier, we could pass $HOSTLD, but if the linker isn't
named something accurate (e.g. if the linker is not named lld, but is
lld), we need to know how to pass a flavor:
"
Maybe it is worthing menting that HOSTCC may not be properly named.
>
> Signed-off-by: Matthew Maurer <[email protected]>
> ---
>
> Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS
> should be respected as well.
>
> I did not switch it to use HOSTLD for two reasons:
> * That variable is not globally defined - it is only available in two
> subdirectories of tools/
> * C host scripts are linked by HOSTCC as well, even when linking a
> collection of object files. It *prints* HOSTLD, but invokes HOSTCC.
> See scripts/Makefile.host cmd_host-cmulti for an example.
>
> rust/Makefile | 4 ++++
> scripts/Makefile.host | 4 ++++
> 2 files changed, 8 insertions(+)
>
> diff --git a/rust/Makefile b/rust/Makefile
> index 87958e864be0..b60b7eb8c5a0 100644
> --- a/rust/Makefile
> +++ b/rust/Makefile
> @@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
> $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
> $(call if_changed,exports)
>
> +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
This is equivalent to a macro in scripts/Kbuild.include
escsq = $(subst $(squote),'\$(squote)',$1)
I would write it directly in cmd_rustc_procmacro.
> +
> quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
> cmd_rustc_procmacro = \
> $(RUSTC_OR_CLIPPY) $(rust_common_flags) \
> + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
> --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
> --crate-type proc-macro \
> --crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
> diff --git a/scripts/Makefile.host b/scripts/Makefile.host
> index 8f7f842b54f9..dc0410cae5ca 100644
> --- a/scripts/Makefile.host
> +++ b/scripts/Makefile.host
> @@ -87,10 +87,14 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \
> $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
> $(HOSTCXXFLAGS_$(target-stem).o)
>
> +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
> +
> # `--out-dir` is required to avoid temporaries being created by `rustc` in the
> # current working directory, which may be not accessible in the out-of-tree
> # modules case.
> hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
> + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
> + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
> $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
> $(HOSTRUSTFLAGS_$(target-stem))
>
> --
> 2.42.0.459.ge4e396fd5e-goog
>
--
Best Regards
Masahiro Yamada