2024-02-08 00:15:20

by Nathan Chancellor

Subject: [PATCH 00/11] s390: Support linking with ld.lld

Hi all,

This series allows the s390 kernel to be linked with ld.lld (support for
s390 is under review at [1]). This implicitly depends on [2], which was
created and sent before it was realized that this series was necessary.

The first chunk of this series enables support for
CONFIG_LD_ORPHAN_WARN, as it was discovered during testing that the
kernel fails to build with ld.lld due to differences in orphan section
handling, which would have been caught with the linker's orphan section
warnings ahead of the actual build error. There are no warnings when
building ARCH=s390 defconfig and allmodconfig with GCC 6 through 13 or
tip of tree Clang using ld.bfd or ld.lld

The final patch resolves a series of errors due to ld.lld having a
different default for checking for DT_TEXTREL ('-z text') vs ld.bfd,
which defaults to '-z notext' (but this is configurable at build time).

There is one outstanding issue due to something that ld.lld does not
support that the kernel relies on:

ld.lld: error: drivers/nvme/host/fc.o:(__bug_table): writable SHF_MERGE section is not supported

This was changed in the kernel in commit e21f8baf8d9a ("s390/bug: add
entry size to the __bug_table section"). Is this change truly necessary?
I selectively applied a revert on top of current mainline and I did not
observe any issues with either Clang or GCC.

diff --git a/arch/s390/include/asm/bug.h b/arch/s390/include/asm/bug.h
index aebe1e22c7be..c500d45fb465 100644
--- a/arch/s390/include/asm/bug.h
+++ b/arch/s390/include/asm/bug.h
@@ -14,7 +14,7 @@
".section .rodata.str,\"aMS\",@progbits,1\n" \
"1: .asciz \""__FILE__"\"\n" \
".previous\n" \
- ".section __bug_table,\"awM\",@progbits,%2\n" \
+ ".section __bug_table,\"aw\"\n" \
"2: .long 0b-.\n" \
" .long 1b-.\n" \
" .short %0,%1\n" \
@@ -30,7 +30,7 @@
#define __EMIT_BUG(x) do { \
asm_inline volatile( \
"0: mc 0,0\n" \
- ".section __bug_table,\"awM\",@progbits,%1\n" \
+ ".section __bug_table,\"aw\"\n" \
"1: .long 0b-.\n" \
" .short %0\n" \
" .org 1b+%1\n" \

If it is necessary, is there any way to work around this error? For
testing purposes, disabling CONFIG_BUG is easy enough but that is not
usable in the real world.

To test this series with ld.lld, you'll need to build ld.lld from the
pull request, which is easy to do following LLVM's instructions [3].
Here is a TL;DR version I tested that just builds LLD with minimal noise
during the build.

$ git clone https://github.com/llvm/llvm-project
$ cd llvm-project
$ git fetch https://github.com/llvm/llvm-project pull/75643/head
$ git switch -d FETCH_HEAD
$ cmake \
-B build \
-G Ninja \
-S llvm \
--log-level=NOTICE \
-Wno-dev \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_PROJECTS=lld \
-DLLVM_ENABLE_WARNINGS=OFF \
-DLLVM_TARGETS_TO_BUILD=SystemZ
$ ninja -C build lld
$ export PATH=$PWD/build/bin:$PATH

Then build the kernel with 'LD=ld.lld' in addition to whatever command
line you use (I tested both Clang and GCC). I can boot an ld.lld linked
kernel built with both compilers in QEMU with this series.

[ 1.386970] Linux version 6.8.0-rc3-00043-g05761ede85d6-dirty ([email protected]) (s390-linux-gcc (GCC) 13.2.0, ClangBuiltLinux LLD 19.0.0) #1 SMP Wed Feb 7 16:51:12 MST 2024

[ 0.871923] Linux version 6.8.0-rc3-00043-g05761ede85d6-dirty ([email protected]) (ClangBuiltLinux clang version 19.0.0git (https://github.com/llvm/llvm-project 417075e56aeba5a5b20301c7bfeba9c2a800982b), ClangBuiltLinux LLD 19.0.0) #1 SMP Wed Feb 7 17:01:22 MST 2024

[1]: https://github.com/llvm/llvm-project/pull/75643
[2]: https://lore.kernel.org/r/20240130-s390-vdso-drop-fpic-from-ldflags-v1-1-094ad104fc55@kernel.org/
[3]: https://llvm.org/docs/CMake.html

---
Nathan Chancellor (11):
s390: boot: Add support for CONFIG_LD_ORPHAN_WARN
s390: vmlinux.lds.S: Handle '.data.rel' sections explicitly
s390: vmlinux.lds.S: Explicitly handle '.got' and '.plt' sections
s390: vmlinux.lds.S: Discard unnecessary sections
s390/boot: vmlinux.lds.S: Handle '.init.text'
s390/boot: vmlinux.lds.S: Handle '.rela' sections
s390/boot: vmlinux.lds.S: Handle DWARF debug sections
s390/boot: vmlinux.lds.S: Handle ELF required sections
s390/boot: vmlinux.lds.S: Handle commonly discarded sections
s390: Select CONFIG_ARCH_WANT_LD_ORPHAN_WARN
s390: Link vmlinux with '-z notext'

arch/s390/Kconfig | 1 +
arch/s390/Makefile | 2 +-
arch/s390/boot/Makefile | 5 +++--
arch/s390/boot/vmlinux.lds.S | 28 ++++++++++++++++++++++++++++
arch/s390/kernel/vmlinux.lds.S | 28 +++++++++++++++++++++++++++-
5 files changed, 60 insertions(+), 4 deletions(-)
---
base-commit: 6613476e225e090cc9aad49be7fa504e290dd33d
change-id: 20240207-s390-lld-and-orphan-warn-d0ff4ff657b0

Best regards,
--
Nathan Chancellor <[email protected]>

2024-02-08 00:15:35

by Nathan Chancellor

Subject: [PATCH 01/11] s390: boot: Add support for CONFIG_LD_ORPHAN_WARN

arch/s390/boot/vmlinux uses a different linker script and build rules
than the main vmlinux, so the '--orphan-handling' flag is not applied to
it. Add support for '--orphan-handling' so that all sections are
properly described in the linker script, which helps eliminate bugs
between linker implementations having different orphan section
heuristics.

Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/s390/boot/Makefile | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/s390/boot/Makefile b/arch/s390/boot/Makefile
index c7c81e5f9218..ace0bda1ad24 100644
--- a/arch/s390/boot/Makefile
+++ b/arch/s390/boot/Makefile
@@ -73,11 +73,12 @@ $(obj)/bzImage: $(obj)/vmlinux $(obj)/section_cmp.boot.data $(obj)/section_cmp.b
$(obj)/section_cmp%: vmlinux $(obj)/vmlinux FORCE
$(call if_changed,section_cmp)

-LDFLAGS_vmlinux := --oformat $(LD_BFD) -e startup $(if $(CONFIG_VMLINUX_MAP),-Map=$(obj)/vmlinux.map) --build-id=sha1 -T
+LDFLAGS_vmlinux-$(CONFIG_LD_ORPHAN_WARN) := --orphan-handling=$(CONFIG_LD_ORPHAN_WARN_LEVEL)
+LDFLAGS_vmlinux := $(LDFLAGS_vmlinux-y) --oformat $(LD_BFD) -e startup $(if $(CONFIG_VMLINUX_MAP),-Map=$(obj)/vmlinux.map) --build-id=sha1 -T
$(obj)/vmlinux: $(obj)/vmlinux.lds $(OBJECTS_ALL) FORCE
$(call if_changed,ld)

-LDFLAGS_vmlinux.syms := --oformat $(LD_BFD) -e startup -T
+LDFLAGS_vmlinux.syms := $(LDFLAGS_vmlinux-y) --oformat $(LD_BFD) -e startup -T
$(obj)/vmlinux.syms: $(obj)/vmlinux.lds $(OBJECTS) FORCE
$(call if_changed,ld)


--
2.43.0

2024-02-08 00:15:35

by Nathan Chancellor

Subject: [PATCH 02/11] s390: vmlinux.lds.S: Handle '.data.rel' sections explicitly

When building with CONFIG_LD_ORPHAN_WARN after selecting
CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are a lot of warnings around
'.data.rel' sections:

s390-linux-ld: warning: orphan section `.data.rel' from `kernel/sched/build_utility.o' being placed in section `.data.rel'
s390-linux-ld: warning: orphan section `.data.rel.local' from `kernel/sched/build_utility.o' being placed in section `.data.rel.local'
s390-linux-ld: warning: orphan section `.data.rel.ro' from `kernel/sched/build_utility.o' being placed in section `.data.rel.ro'
s390-linux-ld: warning: orphan section `.data.rel.ro.local' from `kernel/sched/build_utility.o' being placed in section `.data.rel.ro.local'

Describe these in vmlinux.lds.S so there is no more warning and the
sections are placed consistently between linkers.

Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/s390/kernel/vmlinux.lds.S | 6 ++++++
1 file changed, 6 insertions(+)

diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S
index e32ef446f451..d231a3faf981 100644
--- a/arch/s390/kernel/vmlinux.lds.S
+++ b/arch/s390/kernel/vmlinux.lds.S
@@ -59,6 +59,9 @@ SECTIONS
} :text = 0x0700

RO_DATA(PAGE_SIZE)
+ .data.rel.ro : {
+ *(.data.rel.ro .data.rel.ro.*)
+ }

. = ALIGN(PAGE_SIZE);
_sdata = .; /* Start of data section */
@@ -73,6 +76,9 @@ SECTIONS
__end_ro_after_init = .;

RW_DATA(0x100, PAGE_SIZE, THREAD_SIZE)
+ .data.rel : {
+ *(.data.rel*)
+ }
BOOT_DATA_PRESERVED

. = ALIGN(8);

--
2.43.0

2024-02-08 00:15:45

by Nathan Chancellor

Subject: [PATCH 03/11] s390: vmlinux.lds.S: Explicitly handle '.got' and '.plt' sections

When building with CONFIG_LD_ORPHAN_WARN after selecting
CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are a lot of warnings around the
GOT and PLT sections:

s390-linux-ld: warning: orphan section `.plt' from `arch/s390/kernel/head64.o' being placed in section `.plt'
s390-linux-ld: warning: orphan section `.got' from `arch/s390/kernel/head64.o' being placed in section `.got'
s390-linux-ld: warning: orphan section `.got.plt' from `arch/s390/kernel/head64.o' being placed in section `.got.plt'
s390-linux-ld: warning: orphan section `.iplt' from `arch/s390/kernel/head64.o' being placed in section `.iplt'
s390-linux-ld: warning: orphan section `.igot.plt' from `arch/s390/kernel/head64.o' being placed in section `.igot.plt'

s390-linux-ld: warning: orphan section `.iplt' from `arch/s390/boot/head.o' being placed in section `.iplt'
s390-linux-ld: warning: orphan section `.igot.plt' from `arch/s390/boot/head.o' being placed in section `.igot.plt'
s390-linux-ld: warning: orphan section `.got' from `arch/s390/boot/head.o' being placed in section `.got'

Currently, only the '.got' section is actually emitted in the final
binary. In a manner similar to other architectures, put the '.got'
section near the '.data' section and coalesce the PLT sections,
checking that the final section is zero sized, which is a safe/tested
approach versus full discard.

Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/s390/boot/vmlinux.lds.S | 19 +++++++++++++++++++
arch/s390/kernel/vmlinux.lds.S | 19 +++++++++++++++++++
2 files changed, 38 insertions(+)

diff --git a/arch/s390/boot/vmlinux.lds.S b/arch/s390/boot/vmlinux.lds.S
index 389df0e0d9e5..4aa2f340c8d9 100644
--- a/arch/s390/boot/vmlinux.lds.S
+++ b/arch/s390/boot/vmlinux.lds.S
@@ -39,6 +39,9 @@ SECTIONS
*(.rodata.*)
_erodata = . ;
}
+ .got : {
+ *(.got)
+ }
NOTES
.data : {
_data = . ;
@@ -118,6 +121,22 @@ SECTIONS
}
_end = .;

+ /*
+ * Sections that should stay zero sized, which is safer to
+ * explicitly check instead of blindly discarding.
+ */
+ .got.plt : {
+ *(.got.plt)
+ }
+ ASSERT(SIZEOF(.got.plt) == 0, "Unexpected GOT/PLT entries detected!")
+ .plt : {
+ *(.plt)
+ *(.plt.*)
+ *(.iplt)
+ *(.igot .igot.plt)
+ }
+ ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!")
+
/* Sections to be discarded */
/DISCARD/ : {
*(.eh_frame)
diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S
index d231a3faf981..661a487a3048 100644
--- a/arch/s390/kernel/vmlinux.lds.S
+++ b/arch/s390/kernel/vmlinux.lds.S
@@ -62,6 +62,9 @@ SECTIONS
.data.rel.ro : {
*(.data.rel.ro .data.rel.ro.*)
}
+ .got : {
+ *(.got)
+ }

. = ALIGN(PAGE_SIZE);
_sdata = .; /* Start of data section */
@@ -241,6 +244,22 @@ SECTIONS
DWARF_DEBUG
ELF_DETAILS

+ /*
+ * Sections that should stay zero sized, which is safer to
+ * explicitly check instead of blindly discarding.
+ */
+ .got.plt : {
+ *(.got.plt)
+ }
+ ASSERT(SIZEOF(.got.plt) == 0, "Unexpected GOT/PLT entries detected!")
+ .plt : {
+ *(.plt)
+ *(.plt.*)
+ *(.iplt)
+ *(.igot .igot.plt)
+ }
+ ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!")
+
/* Sections to be discarded */
DISCARDS
/DISCARD/ : {

--
2.43.0

2024-02-08 00:16:11

by Nathan Chancellor

Subject: [PATCH 04/11] s390: vmlinux.lds.S: Discard unnecessary sections

When building with CONFIG_LD_ORPHAN_WARN after selecting
CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are some warnings around certain
ELF sections that are unnecessary for the kernel's purposes.

s390-linux-ld: warning: orphan section `.dynstr' from `arch/s390/kernel/head64.o' being placed in section `.dynstr'
s390-linux-ld: warning: orphan section `.dynamic' from `arch/s390/kernel/head64.o' being placed in section `.dynamic'
s390-linux-ld: warning: orphan section `.hash' from `arch/s390/kernel/head64.o' being placed in section `.hash'
s390-linux-ld: warning: orphan section `.gnu.hash' from `arch/s390/kernel/head64.o' being placed in section `.gnu.hash'

Add them to the discards to clear up the warnings, which matches other
architectures.

Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/s390/kernel/vmlinux.lds.S | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S
index 661a487a3048..35a6b3e6cc50 100644
--- a/arch/s390/kernel/vmlinux.lds.S
+++ b/arch/s390/kernel/vmlinux.lds.S
@@ -264,6 +264,7 @@ SECTIONS
DISCARDS
/DISCARD/ : {
*(.eh_frame)
- *(.interp)
+ *(.interp .dynamic)
+ *(.dynstr .hash .gnu.hash)
}
}

--
2.43.0

2024-02-08 00:16:30

by Nathan Chancellor

Subject: [PATCH 05/11] s390/boot: vmlinux.lds.S: Handle '.init.text'

When building with CONFIG_LD_ORPHAN_WARN after selecting
CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there is a warning about the presence of
an '.init.text' section in arch/s390/boot:

s390-linux-ld: warning: orphan section `.init.text' from `arch/s390/boot/sclp_early_core.o' being placed in section `.init.text'

arch/s390/boot/sclp_early_core.c includes a file from the main kernel
build, which picks up a usage of '__init' somewhere. For the
decompressed image, this section can just be coalesced into '.text'.

Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/s390/boot/vmlinux.lds.S | 1 +
1 file changed, 1 insertion(+)

diff --git a/arch/s390/boot/vmlinux.lds.S b/arch/s390/boot/vmlinux.lds.S
index 4aa2f340c8d9..2f0bc05664ed 100644
--- a/arch/s390/boot/vmlinux.lds.S
+++ b/arch/s390/boot/vmlinux.lds.S
@@ -31,6 +31,7 @@ SECTIONS
_text = .; /* Text */
*(.text)
*(.text.*)
+ INIT_TEXT
_etext = . ;
}
.rodata : {

--
2.43.0

2024-02-08 00:17:12

by Nathan Chancellor

Subject: [PATCH 08/11] s390/boot: vmlinux.lds.S: Handle ELF required sections

When building with CONFIG_LD_ORPHAN_WARN after selecting
CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there is a warning around the '.comment'
section for each file in arch/s390/boot

s390-linux-ld: warning: orphan section `.comment' from `arch/s390/boot/als.o' being placed in section `.comment'
s390-linux-ld: warning: orphan section `.comment' from `arch/s390/boot/startup.o' being placed in section `.comment'
s390-linux-ld: warning: orphan section `.comment' from `arch/s390/boot/physmem_info.o' being placed in section `.comment'

include/asm-generic/vmlinux.lds.h has a macro for required ELF sections
not related to debugging named ELF_DETAILS, use it to clear up the
warnings.

Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/s390/boot/vmlinux.lds.S | 1 +
1 file changed, 1 insertion(+)

diff --git a/arch/s390/boot/vmlinux.lds.S b/arch/s390/boot/vmlinux.lds.S
index 83af17bfe630..2bef450e84e9 100644
--- a/arch/s390/boot/vmlinux.lds.S
+++ b/arch/s390/boot/vmlinux.lds.S
@@ -123,6 +123,7 @@ SECTIONS
_end = .;

DWARF_DEBUG
+ ELF_DETAILS

/*
* Sections that should stay zero sized, which is safer to

--
2.43.0

2024-02-08 00:17:39

by Nathan Chancellor

Subject: [PATCH 06/11] s390/boot: vmlinux.lds.S: Handle '.rela' sections

When building with CONFIG_LD_ORPHAN_WARN after selecting
CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are several warnings from
arch/s390/boot/head.o due to the unhandled presence of '.rela' sections:

s390-linux-ld: warning: orphan section `.rela.iplt' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
s390-linux-ld: warning: orphan section `.rela.head.text' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
s390-linux-ld: warning: orphan section `.rela.got' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
s390-linux-ld: warning: orphan section `.rela.data' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
s390-linux-ld: warning: orphan section `.rela.data.rel.ro' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
s390-linux-ld: warning: orphan section `.rela.iplt' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
s390-linux-ld: warning: orphan section `.rela.head.text' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
s390-linux-ld: warning: orphan section `.rela.got' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
s390-linux-ld: warning: orphan section `.rela.data' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
s390-linux-ld: warning: orphan section `.rela.data.rel.ro' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'

These sections are unneeded for the decompressor and they are not
emitted in the binary currently. In a manner similar to other
architectures, coalesce the sections into '.rela.dyn' and ensure it is
zero sized, which is a safe/tested approach versus full discard.

Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/s390/boot/vmlinux.lds.S | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/arch/s390/boot/vmlinux.lds.S b/arch/s390/boot/vmlinux.lds.S
index 2f0bc05664ed..ff8c62d84f98 100644
--- a/arch/s390/boot/vmlinux.lds.S
+++ b/arch/s390/boot/vmlinux.lds.S
@@ -137,6 +137,10 @@ SECTIONS
*(.igot .igot.plt)
}
ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!")
+ .rela.dyn : {
+ *(.rela.*) *(.rela_*)
+ }
+ ASSERT(SIZEOF(.rela.dyn) == 0, "Unexpected run-time relocations (.rela) detected!")

/* Sections to be discarded */
/DISCARD/ : {

--
2.43.0

2024-02-08 00:17:59

by Nathan Chancellor

Subject: [PATCH 07/11] s390/boot: vmlinux.lds.S: Handle DWARF debug sections

When building with CONFIG_LD_ORPHAN_WARN after selecting
CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are several series of warnings for
each file in arch/s390/boot due to the boot linker script not handling
the DWARF debug sections:

s390-linux-ld: warning: orphan section `.debug_line' from `arch/s390/boot/head.o' being placed in section `.debug_line'
s390-linux-ld: warning: orphan section `.debug_info' from `arch/s390/boot/head.o' being placed in section `.debug_info'
s390-linux-ld: warning: orphan section `.debug_abbrev' from `arch/s390/boot/head.o' being placed in section `.debug_abbrev'
s390-linux-ld: warning: orphan section `.debug_aranges' from `arch/s390/boot/head.o' being placed in section `.debug_aranges'
s390-linux-ld: warning: orphan section `.debug_str' from `arch/s390/boot/head.o' being placed in section `.debug_str'

include/asm-generic/vmlinux.lds.h has a macro for DWARF debug sections
named DWARF_DEBUG, use it to clear up the warnings.

Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/s390/boot/vmlinux.lds.S | 2 ++
1 file changed, 2 insertions(+)

diff --git a/arch/s390/boot/vmlinux.lds.S b/arch/s390/boot/vmlinux.lds.S
index ff8c62d84f98..83af17bfe630 100644
--- a/arch/s390/boot/vmlinux.lds.S
+++ b/arch/s390/boot/vmlinux.lds.S
@@ -122,6 +122,8 @@ SECTIONS
}
_end = .;

+ DWARF_DEBUG
+
/*
* Sections that should stay zero sized, which is safer to
* explicitly check instead of blindly discarding.

--
2.43.0

2024-02-08 00:18:04

by Nathan Chancellor

Subject: [PATCH 11/11] s390: Link vmlinux with '-z notext'

ld.bfd defaults to '-z notext' (although it is customizable with the
'--enable-textrel-check' configure option) but ld.lld defaults to '-z
text', which causes issues with building the kernel due to the presence
of dynamic relocations in sections that are not writable.

ld.lld: error: relocation R_390_64 cannot be used against local symbol; recompile with -fPIC

Add '-z notext' to avoid these errors, as this is expected, which
matches other architectures.

Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/s390/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/s390/Makefile b/arch/s390/Makefile
index 73873e451686..994f9b3d575f 100644
--- a/arch/s390/Makefile
+++ b/arch/s390/Makefile
@@ -15,7 +15,7 @@ KBUILD_CFLAGS_MODULE += -fPIC
KBUILD_AFLAGS += -m64
KBUILD_CFLAGS += -m64
KBUILD_CFLAGS += -fPIE
-LDFLAGS_vmlinux := -pie
+LDFLAGS_vmlinux := -pie -z notext
aflags_dwarf := -Wa,-gdwarf-2
KBUILD_AFLAGS_DECOMPRESSOR := $(CLANG_FLAGS) -m64 -D__ASSEMBLY__
ifndef CONFIG_AS_IS_LLVM

--
2.43.0

2024-02-08 00:19:04

by Nathan Chancellor

Subject: [PATCH 09/11] s390/boot: vmlinux.lds.S: Handle commonly discarded sections

When building with CONFIG_LD_ORPHAN_WARN after selecting
CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are several series of warnings
from the various discardable sections that the kernel adds for build
purposes that are not needed at runtime:

s390-linux-ld: warning: orphan section `.export_symbol' from `arch/s390/boot/decompressor.o' being placed in section `.export_symbol'
s390-linux-ld: warning: orphan section `.discard.addressable' from `arch/s390/boot/decompressor.o' being placed in section `.discard.addressable'
s390-linux-ld: warning: orphan section `.modinfo' from `arch/s390/boot/decompressor.o' being placed in section `.modinfo'

include/asm-generic/vmlinux.lds.h has a macro for easily discarding
these sections across the kernel named COMMON_DISCARDS, use it to clear
up the warnings.

Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/s390/boot/vmlinux.lds.S | 1 +
1 file changed, 1 insertion(+)

diff --git a/arch/s390/boot/vmlinux.lds.S b/arch/s390/boot/vmlinux.lds.S
index 2bef450e84e9..806bca35e3f1 100644
--- a/arch/s390/boot/vmlinux.lds.S
+++ b/arch/s390/boot/vmlinux.lds.S
@@ -147,6 +147,7 @@ SECTIONS

/* Sections to be discarded */
/DISCARD/ : {
+ COMMON_DISCARDS
*(.eh_frame)
*(__ex_table)
*(*__ksymtab*)

--
2.43.0

2024-02-08 00:19:20

by Nathan Chancellor

Subject: [PATCH 10/11] s390: Select CONFIG_ARCH_WANT_LD_ORPHAN_WARN

Now that all sections have been properly accounted for in the s390
linker scripts, select CONFIG_ARCH_WANT_LD_ORPHAN_WARN so that
'--orphan-handling' is added to LDFLAGS to catch any future sections
that are added without being described in linker scripts.

Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/s390/Kconfig | 1 +
1 file changed, 1 insertion(+)

diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index fe565f3a3a91..771235aee6bf 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -127,6 +127,7 @@ config S390
select ARCH_WANT_DEFAULT_BPF_JIT
select ARCH_WANT_IPC_PARSE_VERSION
select ARCH_WANT_KERNEL_PMD_MKWRITE
+ select ARCH_WANT_LD_ORPHAN_WARN
select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP
select BUILDTIME_TABLE_SORT
select CLONE_BACKWARDS2

--
2.43.0

2024-02-08 23:14:25

by Justin Stitt

Subject: Re: [PATCH 00/11] s390: Support linking with ld.lld

Hi,

On Wed, Feb 07, 2024 at 05:14:52PM -0700, Nathan Chancellor wrote:
> Hi all,
>
> This series allows the s390 kernel to be linked with ld.lld (support for
> s390 is under review at [1]). This implicitly depends on [2], which was
> created and sent before it was realized that this series was necessary.
>
> The first chunk of this series enables support for
> CONFIG_LD_ORPHAN_WARN, as it was discovered during testing that the
> kernel fails to build with ld.lld due to differences in orphan section
> handling, which would have been caught with the linker's orphan section
> warnings ahead of the actual build error. There are no warnings when
> building ARCH=s390 defconfig and allmodconfig with GCC 6 through 13 or
> tip of tree Clang using ld.bfd or ld.lld
>
> The final patch resolves a series of errors due to ld.lld having a
> different default for checking for DT_TEXTREL ('-z text') vs ld.bfd,
> which defaults to '-z notext' (but this is configurable at build time).
>
> There is one outstanding issue due to something that ld.lld does not
> support that the kernel relies on:
>
> ld.lld: error: drivers/nvme/host/fc.o:(__bug_table): writable SHF_MERGE section is not supported
>
> This was changed in the kernel in commit e21f8baf8d9a ("s390/bug: add
> entry size to the __bug_table section"). Is this change truly necessary?
> I selectively applied a revert on top of current mainline and I did not
> observe any issues with either Clang or GCC.
>
> diff --git a/arch/s390/include/asm/bug.h b/arch/s390/include/asm/bug.h
> index aebe1e22c7be..c500d45fb465 100644
> --- a/arch/s390/include/asm/bug.h
> +++ b/arch/s390/include/asm/bug.h
> @@ -14,7 +14,7 @@
> ".section .rodata.str,\"aMS\",@progbits,1\n" \
> "1: .asciz \""__FILE__"\"\n" \
> ".previous\n" \
> - ".section __bug_table,\"awM\",@progbits,%2\n" \
> + ".section __bug_table,\"aw\"\n" \
> "2: .long 0b-.\n" \
> " .long 1b-.\n" \
> " .short %0,%1\n" \
> @@ -30,7 +30,7 @@
> #define __EMIT_BUG(x) do { \
> asm_inline volatile( \
> "0: mc 0,0\n" \
> - ".section __bug_table,\"awM\",@progbits,%1\n" \
> + ".section __bug_table,\"aw\"\n" \
> "1: .long 0b-.\n" \
> " .short %0\n" \
> " .org 1b+%1\n" \
>
> If it is necessary, is there any way to work around this error? For
> testing purposes, disabling CONFIG_BUG is easy enough but that is not
> usable in the real world.
>
> To test this series with ld.lld, you'll need to build ld.lld from the
> pull request, which is easy to do following LLVM's instructions [3].
> Here is a TL;DR version I tested that just builds LLD with minimal noise
> during the build.
>
> $ git clone https://github.com/llvm/llvm-project
> $ cd llvm-project
> $ git fetch https://github.com/llvm/llvm-project pull/75643/head
> $ git switch -d FETCH_HEAD
> $ cmake \
> -B build \
> -G Ninja \
> -S llvm \
> --log-level=NOTICE \
> -Wno-dev \
> -DCMAKE_BUILD_TYPE=Release \
> -DLLVM_ENABLE_PROJECTS=lld \
> -DLLVM_ENABLE_WARNINGS=OFF \
> -DLLVM_TARGETS_TO_BUILD=SystemZ
> $ ninja -C build lld
> $ export PATH=$PWD/build/bin:$PATH
>
> Then build the kernel with 'LD=ld.lld' in addition to whatever command
> line you use (I tested both Clang and GCC). I can boot an ld.lld linked
> kernel built with both compilers in QEMU with this series.

Yeah, this all works for me and I am able to boot. I did need to use the
diff present in 0/11 to remove the warnings regarding SHF_MERGE
sections. It should probably be a patch in this series instead of a
inlined diff?

>
> [ 1.386970] Linux version 6.8.0-rc3-00043-g05761ede85d6-dirty ([email protected]) (s390-linux-gcc (GCC) 13.2.0, ClangBuiltLinux LLD 19.0.0) #1 SMP Wed Feb 7 16:51:12 MST 2024
>
> [ 0.871923] Linux version 6.8.0-rc3-00043-g05761ede85d6-dirty ([email protected]) (ClangBuiltLinux clang version 19.0.0git (https://github.com/llvm/llvm-project 417075e56aeba5a5b20301c7bfeba9c2a800982b), ClangBuiltLinux LLD 19.0.0) #1 SMP Wed Feb 7 17:01:22 MST 2024
>
> [1]: https://github.com/llvm/llvm-project/pull/75643
> [2]: https://lore.kernel.org/r/20240130-s390-vdso-drop-fpic-from-ldflags-v1-1-094ad104fc55@kernel.org/
^^^^^^^^^
I needed this too, as I was getting a warnings about -fPIC being an
unknown option.


All in all, works great for me building on clang and booting with qemu.

Tested-by: Justin Stitt <[email protected]>

> [3]: https://llvm.org/docs/CMake.html
>
> ---
> Nathan Chancellor (11):
> s390: boot: Add support for CONFIG_LD_ORPHAN_WARN
> s390: vmlinux.lds.S: Handle '.data.rel' sections explicitly
> s390: vmlinux.lds.S: Explicitly handle '.got' and '.plt' sections
> s390: vmlinux.lds.S: Discard unnecessary sections
> s390/boot: vmlinux.lds.S: Handle '.init.text'
> s390/boot: vmlinux.lds.S: Handle '.rela' sections
> s390/boot: vmlinux.lds.S: Handle DWARF debug sections
> s390/boot: vmlinux.lds.S: Handle ELF required sections
> s390/boot: vmlinux.lds.S: Handle commonly discarded sections
> s390: Select CONFIG_ARCH_WANT_LD_ORPHAN_WARN
> s390: Link vmlinux with '-z notext'
>
> arch/s390/Kconfig | 1 +
> arch/s390/Makefile | 2 +-
> arch/s390/boot/Makefile | 5 +++--
> arch/s390/boot/vmlinux.lds.S | 28 ++++++++++++++++++++++++++++
> arch/s390/kernel/vmlinux.lds.S | 28 +++++++++++++++++++++++++++-
> 5 files changed, 60 insertions(+), 4 deletions(-)
> ---
> base-commit: 6613476e225e090cc9aad49be7fa504e290dd33d
> change-id: 20240207-s390-lld-and-orphan-warn-d0ff4ff657b0
>
> Best regards,
> --
> Nathan Chancellor <[email protected]>
>

Thanks
Justin

2024-02-09 11:23:24

by Heiko Carstens

Subject: Re: [PATCH 00/11] s390: Support linking with ld.lld

Hi Nathan,

> This series allows the s390 kernel to be linked with ld.lld (support for
> s390 is under review at [1]). This implicitly depends on [2], which was
> created and sent before it was realized that this series was necessary.
..
> There is one outstanding issue due to something that ld.lld does not
> support that the kernel relies on:
>
> ld.lld: error: drivers/nvme/host/fc.o:(__bug_table): writable SHF_MERGE section is not supported
>
> This was changed in the kernel in commit e21f8baf8d9a ("s390/bug: add
> entry size to the __bug_table section"). Is this change truly necessary?
> I selectively applied a revert on top of current mainline and I did not
> observe any issues with either Clang or GCC.

No it is not necessary. As the original patch stated this was a pre-req
patch for objtool, for which we still don't have support. This (or
something different) might be needed. But for now this can easily be
reverted.

> Then build the kernel with 'LD=ld.lld' in addition to whatever command
> line you use (I tested both Clang and GCC). I can boot an ld.lld linked
> kernel built with both compilers in QEMU with this series.
>
> [ 1.386970] Linux version 6.8.0-rc3-00043-g05761ede85d6-dirty ([email protected]) (s390-linux-gcc (GCC) 13.2.0, ClangBuiltLinux LLD 19.0.0) #1 SMP Wed Feb 7 16:51:12 MST 2024
>
> [ 0.871923] Linux version 6.8.0-rc3-00043-g05761ede85d6-dirty ([email protected]) (ClangBuiltLinux clang version 19.0.0git (https://github.com/llvm/llvm-project 417075e56aeba5a5b20301c7bfeba9c2a800982b), ClangBuiltLinux LLD 19.0.0) #1 SMP Wed Feb 7 17:01:22 MST 2024

Tested, and works for me. Thanks a lot for your work. This is highly
appreciated!

I applied this series internally to get some CI runs over the weekend, and
push it to our external repository beginning of next week. As suggested by
you I reverted the commit you mentioned above, and also removed ENTRY from
our vdso linker scripts, similar to what you did already for powerpc (see
patches below).

Please feel free to send patches for both of the issues, and I'll replace
my patches with your patches.

From 30a0a88d0e6c4802b748a942bb3f6f1b223f53ba Mon Sep 17 00:00:00 2001
From: Heiko Carstens <[email protected]>
Date: Fri, 9 Feb 2024 11:48:25 +0100
Subject: [PATCH 1/2] s390/bug: remove entry size from __bug_table section

Commit e21f8baf8d9a ("s390/bug: add entry size to the __bug_table section")
changed the __EMIT_BUG() inline assembly to emit mergeable __bug_table
entries. This is at least currently not needed, but causes problems with
the upcoming s390 ld.lld support:

ld.lld: error: drivers/nvme/host/fc.o:(__bug_table): writable SHF_MERGE section is not supported

Therefore revert the change for now.

Reported-by: Nathan Chancellor <[email protected]>
Closes: https://lore.kernel.org/all/[email protected]/
Suggested-by: Nathan Chancellor <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
---
arch/s390/include/asm/bug.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/s390/include/asm/bug.h b/arch/s390/include/asm/bug.h
index aebe1e22c7be..c500d45fb465 100644
--- a/arch/s390/include/asm/bug.h
+++ b/arch/s390/include/asm/bug.h
@@ -14,7 +14,7 @@
".section .rodata.str,\"aMS\",@progbits,1\n" \
"1: .asciz \""__FILE__"\"\n" \
".previous\n" \
- ".section __bug_table,\"awM\",@progbits,%2\n" \
+ ".section __bug_table,\"aw\"\n" \
"2: .long 0b-.\n" \
" .long 1b-.\n" \
" .short %0,%1\n" \
@@ -30,7 +30,7 @@
#define __EMIT_BUG(x) do { \
asm_inline volatile( \
"0: mc 0,0\n" \
- ".section __bug_table,\"awM\",@progbits,%1\n" \
+ ".section __bug_table,\"aw\"\n" \
"1: .long 0b-.\n" \
" .short %0\n" \
" .org 1b+%1\n" \
--
2.40.1

From bdca9b8dcf3f0884341f491d54502d4cbe660446 Mon Sep 17 00:00:00 2001
From: Heiko Carstens <[email protected]>
Date: Fri, 9 Feb 2024 11:54:01 +0100
Subject: [PATCH 2/2] s390/vdso: remove unused ENTRY in linker scripts

When linking vdso64.so.dbg with ld.lld, there is a warning about not
finding _start for the starting address:

ld.lld: warning: cannot find entry symbol _start; not setting start address

Fix this be removing the unused ENTRY in both vdso linker scripts. See
commit e247172854a5 ("powerpc/vdso: Remove unused ENTRY in linker
scripts"), which solved the same problem for powerpc, for further details.

Signed-off-by: Heiko Carstens <[email protected]>
---
arch/s390/kernel/vdso32/vdso32.lds.S | 1 -
arch/s390/kernel/vdso64/vdso64.lds.S | 1 -
2 files changed, 2 deletions(-)

diff --git a/arch/s390/kernel/vdso32/vdso32.lds.S b/arch/s390/kernel/vdso32/vdso32.lds.S
index edf5ff1debe1..65b9513a5a0e 100644
--- a/arch/s390/kernel/vdso32/vdso32.lds.S
+++ b/arch/s390/kernel/vdso32/vdso32.lds.S
@@ -9,7 +9,6 @@

OUTPUT_FORMAT("elf32-s390", "elf32-s390", "elf32-s390")
OUTPUT_ARCH(s390:31-bit)
-ENTRY(_start)

SECTIONS
{
diff --git a/arch/s390/kernel/vdso64/vdso64.lds.S b/arch/s390/kernel/vdso64/vdso64.lds.S
index 4461ea151e49..37e2a505e81d 100644
--- a/arch/s390/kernel/vdso64/vdso64.lds.S
+++ b/arch/s390/kernel/vdso64/vdso64.lds.S
@@ -9,7 +9,6 @@

OUTPUT_FORMAT("elf64-s390", "elf64-s390", "elf64-s390")
OUTPUT_ARCH(s390:64-bit)
-ENTRY(_start)

SECTIONS
{
--
2.40.1

2024-02-09 15:23:14

by Nathan Chancellor

Subject: Re: [PATCH 00/11] s390: Support linking with ld.lld

Hi Heiko,

On Fri, Feb 09, 2024 at 12:20:46PM +0100, Heiko Carstens wrote:
> > This series allows the s390 kernel to be linked with ld.lld (support for
> > s390 is under review at [1]). This implicitly depends on [2], which was
> > created and sent before it was realized that this series was necessary.
> ...
> > There is one outstanding issue due to something that ld.lld does not
> > support that the kernel relies on:
> >
> > ld.lld: error: drivers/nvme/host/fc.o:(__bug_table): writable SHF_MERGE section is not supported
> >
> > This was changed in the kernel in commit e21f8baf8d9a ("s390/bug: add
> > entry size to the __bug_table section"). Is this change truly necessary?
> > I selectively applied a revert on top of current mainline and I did not
> > observe any issues with either Clang or GCC.
>
> No it is not necessary. As the original patch stated this was a pre-req
> patch for objtool, for which we still don't have support. This (or
> something different) might be needed. But for now this can easily be
> reverted.
>
> > Then build the kernel with 'LD=ld.lld' in addition to whatever command
> > line you use (I tested both Clang and GCC). I can boot an ld.lld linked
> > kernel built with both compilers in QEMU with this series.
> >
> > [ 1.386970] Linux version 6.8.0-rc3-00043-g05761ede85d6-dirty ([email protected]) (s390-linux-gcc (GCC) 13.2.0, ClangBuiltLinux LLD 19.0.0) #1 SMP Wed Feb 7 16:51:12 MST 2024
> >
> > [ 0.871923] Linux version 6.8.0-rc3-00043-g05761ede85d6-dirty ([email protected]) (ClangBuiltLinux clang version 19.0.0git (https://github.com/llvm/llvm-project 417075e56aeba5a5b20301c7bfeba9c2a800982b), ClangBuiltLinux LLD 19.0.0) #1 SMP Wed Feb 7 17:01:22 MST 2024
>
> Tested, and works for me. Thanks a lot for your work. This is highly
> appreciated!
>
> I applied this series internally to get some CI runs over the weekend, and
> push it to our external repository beginning of next week. As suggested by

Excellent, please let me know if anything comes up from that!

> you I reverted the commit you mentioned above, and also removed ENTRY from
> our vdso linker scripts, similar to what you did already for powerpc (see
> patches below).

Ah thanks for spotting that, I was so focused on the errors I forgot to
clean up the warning too.

> Please feel free to send patches for both of the issues, and I'll replace
> my patches with your patches.

Honestly, I am not sure that I will write much better commit messages
than the ones you already have and those solutions seem acceptable to
me. Consider them:

Reviewed-by: Nathan Chancellor <[email protected]>

But if you would prefer patches from myself though, I can send them next
week.

Cheers,
Nathan

> From 30a0a88d0e6c4802b748a942bb3f6f1b223f53ba Mon Sep 17 00:00:00 2001
> From: Heiko Carstens <[email protected]>
> Date: Fri, 9 Feb 2024 11:48:25 +0100
> Subject: [PATCH 1/2] s390/bug: remove entry size from __bug_table section
>
> Commit e21f8baf8d9a ("s390/bug: add entry size to the __bug_table section")
> changed the __EMIT_BUG() inline assembly to emit mergeable __bug_table
> entries. This is at least currently not needed, but causes problems with
> the upcoming s390 ld.lld support:
>
> ld.lld: error: drivers/nvme/host/fc.o:(__bug_table): writable SHF_MERGE section is not supported
>
> Therefore revert the change for now.
>
> Reported-by: Nathan Chancellor <[email protected]>
> Closes: https://lore.kernel.org/all/[email protected]/
> Suggested-by: Nathan Chancellor <[email protected]>
> Signed-off-by: Heiko Carstens <[email protected]>
> ---
> arch/s390/include/asm/bug.h | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/s390/include/asm/bug.h b/arch/s390/include/asm/bug.h
> index aebe1e22c7be..c500d45fb465 100644
> --- a/arch/s390/include/asm/bug.h
> +++ b/arch/s390/include/asm/bug.h
> @@ -14,7 +14,7 @@
> ".section .rodata.str,\"aMS\",@progbits,1\n" \
> "1: .asciz \""__FILE__"\"\n" \
> ".previous\n" \
> - ".section __bug_table,\"awM\",@progbits,%2\n" \
> + ".section __bug_table,\"aw\"\n" \
> "2: .long 0b-.\n" \
> " .long 1b-.\n" \
> " .short %0,%1\n" \
> @@ -30,7 +30,7 @@
> #define __EMIT_BUG(x) do { \
> asm_inline volatile( \
> "0: mc 0,0\n" \
> - ".section __bug_table,\"awM\",@progbits,%1\n" \
> + ".section __bug_table,\"aw\"\n" \
> "1: .long 0b-.\n" \
> " .short %0\n" \
> " .org 1b+%1\n" \
> --
> 2.40.1
>
> From bdca9b8dcf3f0884341f491d54502d4cbe660446 Mon Sep 17 00:00:00 2001
> From: Heiko Carstens <[email protected]>
> Date: Fri, 9 Feb 2024 11:54:01 +0100
> Subject: [PATCH 2/2] s390/vdso: remove unused ENTRY in linker scripts
>
> When linking vdso64.so.dbg with ld.lld, there is a warning about not
> finding _start for the starting address:
>
> ld.lld: warning: cannot find entry symbol _start; not setting start address
>
> Fix this be removing the unused ENTRY in both vdso linker scripts. See
> commit e247172854a5 ("powerpc/vdso: Remove unused ENTRY in linker
> scripts"), which solved the same problem for powerpc, for further details.
>
> Signed-off-by: Heiko Carstens <[email protected]>
> ---
> arch/s390/kernel/vdso32/vdso32.lds.S | 1 -
> arch/s390/kernel/vdso64/vdso64.lds.S | 1 -
> 2 files changed, 2 deletions(-)
>
> diff --git a/arch/s390/kernel/vdso32/vdso32.lds.S b/arch/s390/kernel/vdso32/vdso32.lds.S
> index edf5ff1debe1..65b9513a5a0e 100644
> --- a/arch/s390/kernel/vdso32/vdso32.lds.S
> +++ b/arch/s390/kernel/vdso32/vdso32.lds.S
> @@ -9,7 +9,6 @@
>
> OUTPUT_FORMAT("elf32-s390", "elf32-s390", "elf32-s390")
> OUTPUT_ARCH(s390:31-bit)
> -ENTRY(_start)
>
> SECTIONS
> {
> diff --git a/arch/s390/kernel/vdso64/vdso64.lds.S b/arch/s390/kernel/vdso64/vdso64.lds.S
> index 4461ea151e49..37e2a505e81d 100644
> --- a/arch/s390/kernel/vdso64/vdso64.lds.S
> +++ b/arch/s390/kernel/vdso64/vdso64.lds.S
> @@ -9,7 +9,6 @@
>
> OUTPUT_FORMAT("elf64-s390", "elf64-s390", "elf64-s390")
> OUTPUT_ARCH(s390:64-bit)
> -ENTRY(_start)
>
> SECTIONS
> {
> --
> 2.40.1
>

2024-02-12 14:06:52

by Heiko Carstens

Subject: Re: [PATCH 04/11] s390: vmlinux.lds.S: Discard unnecessary sections

On Wed, Feb 07, 2024 at 05:14:56PM -0700, Nathan Chancellor wrote:
> When building with CONFIG_LD_ORPHAN_WARN after selecting
> CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are some warnings around certain
> ELF sections that are unnecessary for the kernel's purposes.
>
> s390-linux-ld: warning: orphan section `.dynstr' from `arch/s390/kernel/head64.o' being placed in section `.dynstr'
> s390-linux-ld: warning: orphan section `.dynamic' from `arch/s390/kernel/head64.o' being placed in section `.dynamic'
> s390-linux-ld: warning: orphan section `.hash' from `arch/s390/kernel/head64.o' being placed in section `.hash'
> s390-linux-ld: warning: orphan section `.gnu.hash' from `arch/s390/kernel/head64.o' being placed in section `.gnu.hash'
>
> Add them to the discards to clear up the warnings, which matches other
> architectures.
>
> Signed-off-by: Nathan Chancellor <[email protected]>
> ---
> arch/s390/kernel/vmlinux.lds.S | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
..
> - *(.interp)
> + *(.interp .dynamic)
> + *(.dynstr .hash .gnu.hash)

This seems to be wrong, since it leads to 1000s of error messages when
using the "crash" utility e.g. when looking into a live dump of system
with the generated debug info:

BFD: /usr/lib/debug/usr/lib/modules/6.8.0-20240211.rc3.git0.bdca9b8dcf3f.300.fc39.s390x/vmlinux: attempt to load strings from a non-string section (number 0)

I will change this commit to the below; it seems to work and is in
line with other architectures:

-----

When building with CONFIG_LD_ORPHAN_WARN after selecting
CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are some warnings around certain
ELF sections:

s390-linux-ld: warning: orphan section `.dynstr' from `arch/s390/kernel/head64.o' being placed in section `.dynstr'
s390-linux-ld: warning: orphan section `.dynamic' from `arch/s390/kernel/head64.o' being placed in section `.dynamic'
s390-linux-ld: warning: orphan section `.hash' from `arch/s390/kernel/head64.o' being placed in section `.hash'
s390-linux-ld: warning: orphan section `.gnu.hash' from `arch/s390/kernel/head64.o' being placed in section `.gnu.hash'

Explicitly keep those sections like other architectures when
CONFIG_RELOCATABLE is enabled, which is always true for s390.

[[email protected]: keep sections instead of discarding]
Signed-off-by: Nathan Chancellor <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Heiko Carstens <[email protected]>
---
arch/s390/kernel/vmlinux.lds.S | 15 +++++++++++++++
1 file changed, 15 insertions(+)

diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S
index 661a487a3048..d46e3c383952 100644
--- a/arch/s390/kernel/vmlinux.lds.S
+++ b/arch/s390/kernel/vmlinux.lds.S
@@ -200,6 +200,21 @@ SECTIONS
*(.rela*)
__rela_dyn_end = .;
}
+ .dynamic ALIGN(8) : {
+ *(.dynamic)
+ }
+ .dynsym ALIGN(8) : {
+ *(.dynsym)
+ }
+ .dynstr ALIGN(8) : {
+ *(.dynstr)
+ }
+ .hash ALIGN(8) : {
+ *(.hash)
+ }
+ .gnu.hash ALIGN(8) : {
+ *(.gnu.hash)
+ }

. = ALIGN(PAGE_SIZE);
__init_end = .; /* freed after init ends here */
--
2.40.1

2024-02-12 15:20:06

by Heiko Carstens

Subject: Re: [PATCH 04/11] s390: vmlinux.lds.S: Discard unnecessary sections

On Mon, Feb 12, 2024 at 02:55:11PM +0100, Heiko Carstens wrote:
> Explicitly keep those sections like other architectures when
> CONFIG_RELOCATABLE is enabled, which is always true for s390.
..
> + .dynstr ALIGN(8) : {
> + *(.dynstr)
> + }

Except for this, which is already present (copy-paste-error).

2024-02-13 00:16:46

by Nathan Chancellor

Subject: Re: [PATCH 04/11] s390: vmlinux.lds.S: Discard unnecessary sections

On Mon, Feb 12, 2024 at 02:55:11PM +0100, Heiko Carstens wrote:
> On Wed, Feb 07, 2024 at 05:14:56PM -0700, Nathan Chancellor wrote:
> > When building with CONFIG_LD_ORPHAN_WARN after selecting
> > CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are some warnings around certain
> > ELF sections that are unnecessary for the kernel's purposes.
> >
> > s390-linux-ld: warning: orphan section `.dynstr' from `arch/s390/kernel/head64.o' being placed in section `.dynstr'
> > s390-linux-ld: warning: orphan section `.dynamic' from `arch/s390/kernel/head64.o' being placed in section `.dynamic'
> > s390-linux-ld: warning: orphan section `.hash' from `arch/s390/kernel/head64.o' being placed in section `.hash'
> > s390-linux-ld: warning: orphan section `.gnu.hash' from `arch/s390/kernel/head64.o' being placed in section `.gnu.hash'
> >
> > Add them to the discards to clear up the warnings, which matches other
> > architectures.
> >
> > Signed-off-by: Nathan Chancellor <[email protected]>
> > ---
> > arch/s390/kernel/vmlinux.lds.S | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
> ...
> > - *(.interp)
> > + *(.interp .dynamic)
> > + *(.dynstr .hash .gnu.hash)
>
> This seems to be wrong, since it leads to 1000s of error messages when
> using the "crash" utility e.g. when looking into a live dump of system
> with the generated debug info:
>
> BFD: /usr/lib/debug/usr/lib/modules/6.8.0-20240211.rc3.git0.bdca9b8dcf3f.300.fc39.s390x/vmlinux: attempt to load strings from a non-string section (number 0)
>
> I will change this commit to the below; it seems to work and is in
> line with other architectures:

Thanks a lot for catching that, your final change seems good to me.
Here's to hoping I did not get anything else wrong :)

Cheers,
Nathan

> -----
>
> When building with CONFIG_LD_ORPHAN_WARN after selecting
> CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are some warnings around certain
> ELF sections:
>
> s390-linux-ld: warning: orphan section `.dynstr' from `arch/s390/kernel/head64.o' being placed in section `.dynstr'
> s390-linux-ld: warning: orphan section `.dynamic' from `arch/s390/kernel/head64.o' being placed in section `.dynamic'
> s390-linux-ld: warning: orphan section `.hash' from `arch/s390/kernel/head64.o' being placed in section `.hash'
> s390-linux-ld: warning: orphan section `.gnu.hash' from `arch/s390/kernel/head64.o' being placed in section `.gnu.hash'
>
> Explicitly keep those sections like other architectures when
> CONFIG_RELOCATABLE is enabled, which is always true for s390.
>
> [[email protected]: keep sections instead of discarding]
> Signed-off-by: Nathan Chancellor <[email protected]>
> Link: https://lore.kernel.org/r/[email protected]
> Signed-off-by: Heiko Carstens <[email protected]>
> ---
> arch/s390/kernel/vmlinux.lds.S | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S
> index 661a487a3048..d46e3c383952 100644
> --- a/arch/s390/kernel/vmlinux.lds.S
> +++ b/arch/s390/kernel/vmlinux.lds.S
> @@ -200,6 +200,21 @@ SECTIONS
> *(.rela*)
> __rela_dyn_end = .;
> }
> + .dynamic ALIGN(8) : {
> + *(.dynamic)
> + }
> + .dynsym ALIGN(8) : {
> + *(.dynsym)
> + }
> + .dynstr ALIGN(8) : {
> + *(.dynstr)
> + }
> + .hash ALIGN(8) : {
> + *(.hash)
> + }
> + .gnu.hash ALIGN(8) : {
> + *(.gnu.hash)
> + }
>
> . = ALIGN(PAGE_SIZE);
> __init_end = .; /* freed after init ends here */
> --
> 2.40.1
>

2024-02-13 05:18:59

by Fangrui Song

Subject: Re: [PATCH 06/11] s390/boot: vmlinux.lds.S: Handle '.rela' sections

On Wed, Feb 7, 2024 at 4:15 PM Nathan Chancellor <[email protected]> wrote:
>
> When building with CONFIG_LD_ORPHAN_WARN after selecting
> CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are several warnings from
> arch/s390/boot/head.o due to the unhandled presence of '.rela' sections:
>
> s390-linux-ld: warning: orphan section `.rela.iplt' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
> s390-linux-ld: warning: orphan section `.rela.head.text' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
> s390-linux-ld: warning: orphan section `.rela.got' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
> s390-linux-ld: warning: orphan section `.rela.data' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
> s390-linux-ld: warning: orphan section `.rela.data.rel.ro' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
> s390-linux-ld: warning: orphan section `.rela.iplt' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
> s390-linux-ld: warning: orphan section `.rela.head.text' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
> s390-linux-ld: warning: orphan section `.rela.got' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
> s390-linux-ld: warning: orphan section `.rela.data' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
> s390-linux-ld: warning: orphan section `.rela.data.rel.ro' from `arch/s390/boot/head.o' being placed in section `.rela.dyn'
>
> These sections are unneeded for the decompressor and they are not
> emitted in the binary currently. In a manner similar to other
> architectures, coalesce the sections into '.rela.dyn' and ensure it is
> zero sized, which is a safe/tested approach versus full discard.
>
> Signed-off-by: Nathan Chancellor <[email protected]>
> ---
> arch/s390/boot/vmlinux.lds.S | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/arch/s390/boot/vmlinux.lds.S b/arch/s390/boot/vmlinux.lds.S
> index 2f0bc05664ed..ff8c62d84f98 100644
> --- a/arch/s390/boot/vmlinux.lds.S
> +++ b/arch/s390/boot/vmlinux.lds.S
> @@ -137,6 +137,10 @@ SECTIONS
> *(.igot .igot.plt)
> }
> ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!")
> + .rela.dyn : {
> + *(.rela.*) *(.rela_*)
> + }
> + ASSERT(SIZEOF(.rela.dyn) == 0, "Unexpected run-time relocations (.rela) detected!")
>
> /* Sections to be discarded */
> /DISCARD/ : {
>
> --
> 2.43.0
>

Commit 5354e84598f264793265cc99b4be2a2295826c86 ("x86/build: Add
asserts for unwanted sections")
specifies `*(.rela.*) *(.rela_*)` but it's not clear why `.rela_*` is
included. We only need .rela.* (see also ld.bfd --verbose)

This patch LGTM with this changed.

--
宋方睿

2024-02-13 05:20:46

by Fangrui Song

Subject: Re: [PATCH 11/11] s390: Link vmlinux with '-z notext'

On Wed, Feb 7, 2024 at 4:15 PM Nathan Chancellor <[email protected]> wrote:
>
> ld.bfd defaults to '-z notext' (although it is customizable with the
> '--enable-textrel-check' configure option) but ld.lld defaults to '-z
> text', which causes issues with building the kernel due to the presence
> of dynamic relocations in sections that are not writable.
>
> ld.lld: error: relocation R_390_64 cannot be used against local symbol; recompile with -fPIC
>
> Add '-z notext' to avoid these errors, as this is expected, which
> matches other architectures.
>
> Signed-off-by: Nathan Chancellor <[email protected]>

This follows arm64/powerpc/loongarch/riscv.
LGTM.

Reviewed-by: Fangrui Song <[email protected]>

> ---
> arch/s390/Makefile | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/s390/Makefile b/arch/s390/Makefile
> index 73873e451686..994f9b3d575f 100644
> --- a/arch/s390/Makefile
> +++ b/arch/s390/Makefile
> @@ -15,7 +15,7 @@ KBUILD_CFLAGS_MODULE += -fPIC
> KBUILD_AFLAGS += -m64
> KBUILD_CFLAGS += -m64
> KBUILD_CFLAGS += -fPIE
> -LDFLAGS_vmlinux := -pie
> +LDFLAGS_vmlinux := -pie -z notext
> aflags_dwarf := -Wa,-gdwarf-2
> KBUILD_AFLAGS_DECOMPRESSOR := $(CLANG_FLAGS) -m64 -D__ASSEMBLY__
> ifndef CONFIG_AS_IS_LLVM
>
> --
> 2.43.0
>


--
宋方睿

2024-02-13 05:33:23

by Fangrui Song

Subject: Re: [PATCH 07/11] s390/boot: vmlinux.lds.S: Handle DWARF debug sections

On Wed, Feb 7, 2024 at 4:15 PM Nathan Chancellor <[email protected]> wrote:
>
> When building with CONFIG_LD_ORPHAN_WARN after selecting
> CONFIG_ARCH_HAS_LD_ORPHAN_WARN, there are several series of warnings for
> each file in arch/s390/boot due to the boot linker script not handling
> the DWARF debug sections:
>
> s390-linux-ld: warning: orphan section `.debug_line' from `arch/s390/boot/head.o' being placed in section `.debug_line'
> s390-linux-ld: warning: orphan section `.debug_info' from `arch/s390/boot/head.o' being placed in section `.debug_info'
> s390-linux-ld: warning: orphan section `.debug_abbrev' from `arch/s390/boot/head.o' being placed in section `.debug_abbrev'
> s390-linux-ld: warning: orphan section `.debug_aranges' from `arch/s390/boot/head.o' being placed in section `.debug_aranges'
> s390-linux-ld: warning: orphan section `.debug_str' from `arch/s390/boot/head.o' being placed in section `.debug_str'
>
> include/asm-generic/vmlinux.lds.h has a macro for DWARF debug sections
> named DWARF_DEBUG, use it to clear up the warnings.
>
> Signed-off-by: Nathan Chancellor <[email protected]>
> ---
> arch/s390/boot/vmlinux.lds.S | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/arch/s390/boot/vmlinux.lds.S b/arch/s390/boot/vmlinux.lds.S
> index ff8c62d84f98..83af17bfe630 100644
> --- a/arch/s390/boot/vmlinux.lds.S
> +++ b/arch/s390/boot/vmlinux.lds.S
> @@ -122,6 +122,8 @@ SECTIONS
> }
> _end = .;
>
> + DWARF_DEBUG
> +
> /*
> * Sections that should stay zero sized, which is safer to
> * explicitly check instead of blindly discarding.
>
> --
> 2.43.0
>

Reviewed-by: Fangrui Song <[email protected]>


--
宋方睿

2024-02-14 12:18:19

by Heiko Carstens

Subject: Re: [PATCH 06/11] s390/boot: vmlinux.lds.S: Handle '.rela' sections

On Mon, Feb 12, 2024 at 09:18:34PM -0800, Fangrui Song wrote:
> > ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!")
> > + .rela.dyn : {
> > + *(.rela.*) *(.rela_*)
> > + }
> > + ASSERT(SIZEOF(.rela.dyn) == 0, "Unexpected run-time relocations (.rela) detected!")
..
> Commit 5354e84598f264793265cc99b4be2a2295826c86 ("x86/build: Add
> asserts for unwanted sections")
> specifies `*(.rela.*) *(.rela_*)` but it's not clear why `.rela_*` is
> included. We only need .rela.* (see also ld.bfd --verbose)
>
> This patch LGTM with this changed.

I'll keep it as it is, just to be consistent with x86.

2024-02-14 12:26:50

by Heiko Carstens

Subject: Re: [PATCH 03/11] s390: vmlinux.lds.S: Explicitly handle '.got' and '.plt' sections

On Mon, Feb 12, 2024 at 09:31:53PM -0800, Fangrui Song wrote:
> On Wed, Feb 7, 2024 at 4:15 PM Nathan Chancellor <[email protected]> wrote:
> > + ASSERT(SIZEOF(.got.plt) == 0, "Unexpected GOT/PLT entries detected!")
> > + .plt : {
> > + *(.plt)
> > + *(.plt.*)
> > + *(.iplt)
> > + *(.igot .igot.plt)
> > + }
> > + ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!")
> > +
>
> It seems that arches that drop .plt typically place input section
> description on one line. This saves vertical space.
> It's shorter to use one input section description to match multiple
> sections, e.g.
>
> *(.plt .iplt)

Yes, I'll change Nathan's patch so it looks like arm64:

/*
* Sections that should stay zero sized, which is safer to
* explicitly check instead of blindly discarding.
*/
.plt : {
*(.plt) *(.plt.*) *(.iplt) *(.igot .igot.plt)
}

2024-02-14 13:43:52

by Heiko Carstens

Subject: Re: [PATCH 00/11] s390: Support linking with ld.lld

On Wed, Feb 07, 2024 at 05:14:52PM -0700, Nathan Chancellor wrote:
> Hi all,
>
> This series allows the s390 kernel to be linked with ld.lld (support for
> s390 is under review at [1]). This implicitly depends on [2], which was
> created and sent before it was realized that this series was necessary.
..
> Nathan Chancellor (11):
> s390: boot: Add support for CONFIG_LD_ORPHAN_WARN
> s390: vmlinux.lds.S: Handle '.data.rel' sections explicitly
> s390: vmlinux.lds.S: Explicitly handle '.got' and '.plt' sections
> s390: vmlinux.lds.S: Discard unnecessary sections
> s390/boot: vmlinux.lds.S: Handle '.init.text'
> s390/boot: vmlinux.lds.S: Handle '.rela' sections
> s390/boot: vmlinux.lds.S: Handle DWARF debug sections
> s390/boot: vmlinux.lds.S: Handle ELF required sections
> s390/boot: vmlinux.lds.S: Handle commonly discarded sections
> s390: Select CONFIG_ARCH_WANT_LD_ORPHAN_WARN
> s390: Link vmlinux with '-z notext'
>
> arch/s390/Kconfig | 1 +
> arch/s390/Makefile | 2 +-
> arch/s390/boot/Makefile | 5 +++--
> arch/s390/boot/vmlinux.lds.S | 28 ++++++++++++++++++++++++++++
> arch/s390/kernel/vmlinux.lds.S | 28 +++++++++++++++++++++++++++-
> 5 files changed, 60 insertions(+), 4 deletions(-)

Now available at:
https://git.kernel.org/pub/scm/linux/kernel/git/s390/linux.git/log/?h=features

And should be in linux-next soon.

Thanks a lot! :)

2024-02-14 19:48:46

by Nathan Chancellor

Subject: Re: [PATCH 03/11] s390: vmlinux.lds.S: Explicitly handle '.got' and '.plt' sections

On Wed, Feb 14, 2024 at 01:20:55PM +0100, Heiko Carstens wrote:
> On Mon, Feb 12, 2024 at 09:31:53PM -0800, Fangrui Song wrote:
> > On Wed, Feb 7, 2024 at 4:15 PM Nathan Chancellor <[email protected]> wrote:
> > > + ASSERT(SIZEOF(.got.plt) == 0, "Unexpected GOT/PLT entries detected!")
> > > + .plt : {
> > > + *(.plt)
> > > + *(.plt.*)
> > > + *(.iplt)
> > > + *(.igot .igot.plt)
> > > + }
> > > + ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!")
> > > +
> >
> > It seems that arches that drop .plt typically place input section
> > description on one line. This saves vertical space.
> > It's shorter to use one input section description to match multiple
> > sections, e.g.
> >
> > *(.plt .iplt)
>
> Yes, I'll change Nathan's patch so it looks like arm64:
>
> /*
> * Sections that should stay zero sized, which is safer to
> * explicitly check instead of blindly discarding.
> */
> .plt : {
> *(.plt) *(.plt.*) *(.iplt) *(.igot .igot.plt)
> }

Thanks a lot for changing this. I tried to be consistent with the rest
of the linker script but I guess there were not really any sections that
had this many lines to make it one lining it worth it prior to this
point. Stylistic expectations are hard to account for :)

Cheers,
Nathan

2024-02-14 20:33:17

by Nathan Chancellor

Subject: Re: [PATCH 00/11] s390: Support linking with ld.lld

On Wed, Feb 14, 2024 at 02:43:28PM +0100, Heiko Carstens wrote:
> On Wed, Feb 07, 2024 at 05:14:52PM -0700, Nathan Chancellor wrote:
> > Hi all,
> >
> > This series allows the s390 kernel to be linked with ld.lld (support for
> > s390 is under review at [1]). This implicitly depends on [2], which was
> > created and sent before it was realized that this series was necessary.
> ...
> > Nathan Chancellor (11):
> > s390: boot: Add support for CONFIG_LD_ORPHAN_WARN
> > s390: vmlinux.lds.S: Handle '.data.rel' sections explicitly
> > s390: vmlinux.lds.S: Explicitly handle '.got' and '.plt' sections
> > s390: vmlinux.lds.S: Discard unnecessary sections
> > s390/boot: vmlinux.lds.S: Handle '.init.text'
> > s390/boot: vmlinux.lds.S: Handle '.rela' sections
> > s390/boot: vmlinux.lds.S: Handle DWARF debug sections
> > s390/boot: vmlinux.lds.S: Handle ELF required sections
> > s390/boot: vmlinux.lds.S: Handle commonly discarded sections
> > s390: Select CONFIG_ARCH_WANT_LD_ORPHAN_WARN
> > s390: Link vmlinux with '-z notext'
> >
> > arch/s390/Kconfig | 1 +
> > arch/s390/Makefile | 2 +-
> > arch/s390/boot/Makefile | 5 +++--
> > arch/s390/boot/vmlinux.lds.S | 28 ++++++++++++++++++++++++++++
> > arch/s390/kernel/vmlinux.lds.S | 28 +++++++++++++++++++++++++++-
> > 5 files changed, 60 insertions(+), 4 deletions(-)
>
> Now available at:
> https://git.kernel.org/pub/scm/linux/kernel/git/s390/linux.git/log/?h=features
>
> And should be in linux-next soon.
>
> Thanks a lot! :)

Thank you for bearing with the issues that came up in the series and
getting it reviewed and accepted quickly!

The ld.lld pull request has been merged into main:

https://github.com/llvm/llvm-project/commit/fe3406e349884e4ef61480dd0607f1e237102c74

and Fangrui requested a backport to 18.1.0, so it is possible that
people will get access to this even quicker:

https://github.com/llvm/llvm-project/pull/81675

I did not CC you all on this one but it is needed to avoid a link error
with ld.lld when CONFIG_DEBUG_INFO_BTF is enabled but it is not specific
to s390 so it will go via the kbuild tree (hopefully for 6.8):

https://lore.kernel.org/20240212-fix-elf-type-btf-vmlinux-bin-o-big-endian-v2-1-22c0a6352069@kernel.org/

With that change on top of features, all my builds pass successfully
with ld.lld :) I'll be sending another change your way shortly for
something that came up during my testing with GCC + ld.lld.

Cheers,
Nathan

2024-02-14 21:56:59

by Fangrui Song

Subject: Re: [PATCH 06/11] s390/boot: vmlinux.lds.S: Handle '.rela' sections

On Wed, Feb 14, 2024 at 4:18 AM Heiko Carstens <[email protected]> wrote:
>
> On Mon, Feb 12, 2024 at 09:18:34PM -0800, Fangrui Song wrote:
> > > ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!")
> > > + .rela.dyn : {
> > > + *(.rela.*) *(.rela_*)
> > > + }
> > > + ASSERT(SIZEOF(.rela.dyn) == 0, "Unexpected run-time relocations (.rela) detected!")
> ...
> > Commit 5354e84598f264793265cc99b4be2a2295826c86 ("x86/build: Add
> > asserts for unwanted sections")
> > specifies `*(.rela.*) *(.rela_*)` but it's not clear why `.rela_*` is
> > included. We only need .rela.* (see also ld.bfd --verbose)
> >
> > This patch LGTM with this changed.
>
> I'll keep it as it is, just to be consistent with x86.

Sent https://lore.kernel.org/all/[email protected]/
("[PATCH] x86/build: Simplify patterns for unwanted section")
to simplify the patterns in x86 vmlinux.lds.S:)


--
宋方睿