The Linux build process on x86 roughly consists of compiling all input
files, statically linking them into a vmlinux ELF file, and then taking
and turning this file into an actual bzImage bootable file.
vmlinux has in this process two main purposes:
1) It is an intermediate build target on the way to produce the final
bootable image.
2) It is a file that is expected to be used by debuggers and standard
ELF tooling to work with the built kernel.
For the second purpose, a vmlinux file is typically collected by various
package build recipes, such as distribution spec files, including the
kernel's own binrpm-pkg target.
When building a kernel supporting KASLR with CONFIG_X86_NEED_RELOCS,
vmlinux contains also relocation information produced by using the
--emit-relocs linker option. This is utilized by subsequent build steps
to create vmlinux.relocs and produce a relocatable image. However, the
information is not needed by debuggers and other standard ELF tooling.
The issue is then that the collected vmlinux file and hence distribution
packages end up unnecessarily large because of this extra data. The
following is a size comparison of vmlinux v6.0-rc5 with and without the
relocation information:
| Configuration | With relocs | Stripped relocs |
| x86_64_defconfig | 70 MB | 43 MB |
| +CONFIG_DEBUG_INFO | 818 MB | 367 MB |
The patch optimizes a resulting vmlinux by adding a postlink step that
splits the relocation information into vmlinux.relocs and then strips it
from the vmlinux binary.
Signed-off-by: Petr Pavlu <[email protected]>
---
Notes:
The implemented approach has two shortcomings which is worth mentioning:
* The vmlinux file is rewritten in place. This creates a problem if the
build is interrupted when ld produced vmlinux but before the postlink
is started. A subsequent restart of the build would use the
incompletely processed vmlinux.
The current build logic already appears to have this problem as
scripts/link-vmlinux.sh rewrites vmlinux in place, for example, when
handling CONFIG_BUILDTIME_TABLE_SORT. It could be solved by using an
intermediate target and renaming it to vmlinux only once the file is
final.
* vmlinux.relocs is hidden from the Makefile workflow and Make is
additionally told about the file in arch/x86/boot/compressed/Makefile.
.gitignore | 1 +
arch/x86/Makefile.postlink | 41 +++++++++++++++++++++++++++++++
arch/x86/boot/compressed/Makefile | 10 +++-----
3 files changed, 46 insertions(+), 6 deletions(-)
create mode 100644 arch/x86/Makefile.postlink
diff --git a/.gitignore b/.gitignore
index 265959544978..cd4ef88584ea 100644
--- a/.gitignore
+++ b/.gitignore
@@ -37,6 +37,7 @@
*.o
*.o.*
*.patch
+*.relocs
*.s
*.so
*.so.dbg
diff --git a/arch/x86/Makefile.postlink b/arch/x86/Makefile.postlink
new file mode 100644
index 000000000000..4650aaf6d8b3
--- /dev/null
+++ b/arch/x86/Makefile.postlink
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: GPL-2.0
+# ===========================================================================
+# Post-link x86 pass
+# ===========================================================================
+#
+# 1. Separate relocations from vmlinux into vmlinux.relocs.
+# 2. Strip relocations from vmlinux.
+
+PHONY := __archpost
+__archpost:
+
+-include include/config/auto.conf
+include scripts/Kbuild.include
+
+CMD_RELOCS = arch/x86/tools/relocs
+quiet_cmd_relocs = RELOCS [email protected]
+ cmd_relocs = $(CMD_RELOCS) $@ > [email protected];$(CMD_RELOCS) --abs-relocs $@
+
+quiet_cmd_strip_relocs = RSTRIP $@
+ cmd_strip_relocs = objcopy --remove-relocations='*' $@
+
+# `@true` prevents complaint when there is nothing to be done
+
+vmlinux: FORCE
+ @true
+ifeq ($(CONFIG_X86_NEED_RELOCS),y)
+ $(call cmd,relocs)
+ $(call cmd,strip_relocs)
+endif
+
+%.ko: FORCE
+ @true
+
+clean:
+ @rm -f vmlinux.relocs
+
+PHONY += FORCE clean
+
+FORCE:
+
+.PHONY: $(PHONY)
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index 35ce1a64068b..eba7709d75ae 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -120,14 +120,12 @@ $(obj)/vmlinux.bin: vmlinux FORCE
targets += $(patsubst $(obj)/%,%,$(vmlinux-objs-y)) vmlinux.bin.all vmlinux.relocs
-CMD_RELOCS = arch/x86/tools/relocs
-quiet_cmd_relocs = RELOCS $@
- cmd_relocs = $(CMD_RELOCS) $< > $@;$(CMD_RELOCS) --abs-relocs $<
-$(obj)/vmlinux.relocs: vmlinux FORCE
- $(call if_changed,relocs)
+# vmlinux.relocs is created by the vmlinux postlink step.
+vmlinux.relocs: vmlinux
+ @true
vmlinux.bin.all-y := $(obj)/vmlinux.bin
-vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs
+vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += vmlinux.relocs
$(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE
$(call if_changed,gzip)
--
2.35.3
Hi Petr,
On Tue, Sep 13, 2022 at 03:29:11PM +0200, Petr Pavlu wrote:
> The Linux build process on x86 roughly consists of compiling all input
> files, statically linking them into a vmlinux ELF file, and then taking
> and turning this file into an actual bzImage bootable file.
>
> vmlinux has in this process two main purposes:
> 1) It is an intermediate build target on the way to produce the final
> bootable image.
> 2) It is a file that is expected to be used by debuggers and standard
> ELF tooling to work with the built kernel.
>
> For the second purpose, a vmlinux file is typically collected by various
> package build recipes, such as distribution spec files, including the
> kernel's own binrpm-pkg target.
>
> When building a kernel supporting KASLR with CONFIG_X86_NEED_RELOCS,
> vmlinux contains also relocation information produced by using the
> --emit-relocs linker option. This is utilized by subsequent build steps
> to create vmlinux.relocs and produce a relocatable image. However, the
> information is not needed by debuggers and other standard ELF tooling.
>
> The issue is then that the collected vmlinux file and hence distribution
> packages end up unnecessarily large because of this extra data. The
> following is a size comparison of vmlinux v6.0-rc5 with and without the
> relocation information:
> | Configuration | With relocs | Stripped relocs |
> | x86_64_defconfig | 70 MB | 43 MB |
> | +CONFIG_DEBUG_INFO | 818 MB | 367 MB |
>
> The patch optimizes a resulting vmlinux by adding a postlink step that
> splits the relocation information into vmlinux.relocs and then strips it
> from the vmlinux binary.
>
> Signed-off-by: Petr Pavlu <[email protected]>
> ---
>
> Notes:
> The implemented approach has two shortcomings which is worth mentioning:
> * The vmlinux file is rewritten in place. This creates a problem if the
> build is interrupted when ld produced vmlinux but before the postlink
> is started. A subsequent restart of the build would use the
> incompletely processed vmlinux.
>
> The current build logic already appears to have this problem as
> scripts/link-vmlinux.sh rewrites vmlinux in place, for example, when
> handling CONFIG_BUILDTIME_TABLE_SORT. It could be solved by using an
> intermediate target and renaming it to vmlinux only once the file is
> final.
>
> * vmlinux.relocs is hidden from the Makefile workflow and Make is
> additionally told about the file in arch/x86/boot/compressed/Makefile.
>
> .gitignore | 1 +
> arch/x86/Makefile.postlink | 41 +++++++++++++++++++++++++++++++
> arch/x86/boot/compressed/Makefile | 10 +++-----
> 3 files changed, 46 insertions(+), 6 deletions(-)
> create mode 100644 arch/x86/Makefile.postlink
>
> diff --git a/.gitignore b/.gitignore
> index 265959544978..cd4ef88584ea 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -37,6 +37,7 @@
> *.o
> *.o.*
> *.patch
> +*.relocs
> *.s
> *.so
> *.so.dbg
> diff --git a/arch/x86/Makefile.postlink b/arch/x86/Makefile.postlink
> new file mode 100644
> index 000000000000..4650aaf6d8b3
> --- /dev/null
> +++ b/arch/x86/Makefile.postlink
> @@ -0,0 +1,41 @@
> +# SPDX-License-Identifier: GPL-2.0
> +# ===========================================================================
> +# Post-link x86 pass
> +# ===========================================================================
> +#
> +# 1. Separate relocations from vmlinux into vmlinux.relocs.
> +# 2. Strip relocations from vmlinux.
> +
> +PHONY := __archpost
> +__archpost:
> +
> +-include include/config/auto.conf
> +include scripts/Kbuild.include
> +
> +CMD_RELOCS = arch/x86/tools/relocs
> +quiet_cmd_relocs = RELOCS [email protected]
> + cmd_relocs = $(CMD_RELOCS) $@ > [email protected];$(CMD_RELOCS) --abs-relocs $@
> +
> +quiet_cmd_strip_relocs = RSTRIP $@
> + cmd_strip_relocs = objcopy --remove-relocations='*' $@
Just a small drive by comment, prefer $(OBJCOPY) over objcopy so that
the user's choice of objcopy is respected (such as llvm-objcopy).
Unfortunately, llvm-objcopy does not appear to support
'--remove-relocations'. We can certainly file a feature request for this
upstream but is there a way to accomplish this in a different way? Or
perhaps this could be something that is controlled via Kconfig so it
is only selectable with GNU objcopy??
Cheers,
Nathan
> +
> +# `@true` prevents complaint when there is nothing to be done
> +
> +vmlinux: FORCE
> + @true
> +ifeq ($(CONFIG_X86_NEED_RELOCS),y)
> + $(call cmd,relocs)
> + $(call cmd,strip_relocs)
> +endif
> +
> +%.ko: FORCE
> + @true
> +
> +clean:
> + @rm -f vmlinux.relocs
> +
> +PHONY += FORCE clean
> +
> +FORCE:
> +
> +.PHONY: $(PHONY)
> diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
> index 35ce1a64068b..eba7709d75ae 100644
> --- a/arch/x86/boot/compressed/Makefile
> +++ b/arch/x86/boot/compressed/Makefile
> @@ -120,14 +120,12 @@ $(obj)/vmlinux.bin: vmlinux FORCE
>
> targets += $(patsubst $(obj)/%,%,$(vmlinux-objs-y)) vmlinux.bin.all vmlinux.relocs
>
> -CMD_RELOCS = arch/x86/tools/relocs
> -quiet_cmd_relocs = RELOCS $@
> - cmd_relocs = $(CMD_RELOCS) $< > $@;$(CMD_RELOCS) --abs-relocs $<
> -$(obj)/vmlinux.relocs: vmlinux FORCE
> - $(call if_changed,relocs)
> +# vmlinux.relocs is created by the vmlinux postlink step.
> +vmlinux.relocs: vmlinux
> + @true
>
> vmlinux.bin.all-y := $(obj)/vmlinux.bin
> -vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs
> +vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += vmlinux.relocs
>
> $(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE
> $(call if_changed,gzip)
> --
> 2.35.3
>
On 9/14/22 01:40, Nathan Chancellor wrote:
> [...]
>> diff --git a/arch/x86/Makefile.postlink b/arch/x86/Makefile.postlink
>> new file mode 100644
>> index 000000000000..4650aaf6d8b3
>> --- /dev/null
>> +++ b/arch/x86/Makefile.postlink
>> @@ -0,0 +1,41 @@
>> +# SPDX-License-Identifier: GPL-2.0
>> +# ===========================================================================
>> +# Post-link x86 pass
>> +# ===========================================================================
>> +#
>> +# 1. Separate relocations from vmlinux into vmlinux.relocs.
>> +# 2. Strip relocations from vmlinux.
>> +
>> +PHONY := __archpost
>> +__archpost:
>> +
>> +-include include/config/auto.conf
>> +include scripts/Kbuild.include
>> +
>> +CMD_RELOCS = arch/x86/tools/relocs
>> +quiet_cmd_relocs = RELOCS [email protected]
>> + cmd_relocs = $(CMD_RELOCS) $@ > [email protected];$(CMD_RELOCS) --abs-relocs $@
>> +
>> +quiet_cmd_strip_relocs = RSTRIP $@
>> + cmd_strip_relocs = objcopy --remove-relocations='*' $@
>
> Just a small drive by comment, prefer $(OBJCOPY) over objcopy so that
> the user's choice of objcopy is respected (such as llvm-objcopy).
Ok.
> Unfortunately, llvm-objcopy does not appear to support
> '--remove-relocations'. We can certainly file a feature request for this
> upstream but is there a way to accomplish this in a different way? Or
> perhaps this could be something that is controlled via Kconfig so it
> is only selectable with GNU objcopy??
An alternative is to use use --remove-section='.rel*' which has the same
effect.. or to be more careful, something as:
--remove-section='.rel.*' --remove-section='.rel__*' --remove-section='.rela.*' --remove-section='.rela__*'
Thanks,
Petr