From: Ard Biesheuvel <[email protected]>
On x86, the ordinary, position dependent 'small' and 'kernel' code models only
support placement of the executable in 32-bit addressable memory, due to
the use of 32-bit signed immediates to generate references to global
variables. For the kernel, this implies that all global variables must
reside in the top 2 GiB of the kernel virtual address space, where the
implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal
executables such as the kexec purgatory, which can be placed arbitrarily
in the physical address space, where its address may no longer be
representable as a sign extended 32-bit quantity. For this reason,
commit
e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
switched to the 'large' code model, which uses 64-bit immediates for all
symbol references, including function calls, in order to avoid relying
on any assumptions regarding proximity of symbols in the final
executable.
The large code model is rarely used, clunky and the least likely to
operate in a similar fashion when comparing GCC and Clang, so it is best
avoided. This is especially true now that Clang 18 has started to emit
executable code in two separate sections (.text and .ltext), which
triggers an issue in the kexec loading code at runtime.
Instead, use the position independent small code model, which makes no
assumptions about placement but only about proximity, where all
referenced symbols must be within -/+ 2 GiB, i.e., in range for a
RIP-relative reference. Use hidden visibility to suppress the use of a
GOT, which carries absolute addresses that are not covered by static ELF
relocations, and is therefore incompatible with the kexec loader's
relocation logic.
Cc: Nathan Chancellor <[email protected]>
Cc: Nick Desaulniers <[email protected]>
Cc: Bill Wendling <[email protected]>
Cc: Justin Stitt <[email protected]>
Cc: Song Liu <[email protected]>
Cc: Ricardo Ribalda <[email protected]>
Cc: Fangrui Song <[email protected]>
Cc: Arthur Eubanks <[email protected]>
Link: https://lore.kernel.org/all/[email protected]/
Cc: <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
---
arch/x86/purgatory/Makefile | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile
index bc31863c5ee6..a18591f6e6d9 100644
--- a/arch/x86/purgatory/Makefile
+++ b/arch/x86/purgatory/Makefile
@@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n
# make up the standalone purgatory.ro
PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel
-PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0
+PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0
+PURGATORY_CFLAGS += -fpic -fvisibility=hidden
PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING
PURGATORY_CFLAGS += -fno-stack-protector
--
2.44.0.769.g3c40516874-goog
On Thu, Apr 18, 2024 at 1:17 PM Ard Biesheuvel <[email protected]> wrote:
>
> From: Ard Biesheuvel <[email protected]>
>
> On x86, the ordinary, position dependent 'small' and 'kernel' code models only
> support placement of the executable in 32-bit addressable memory, due to
> the use of 32-bit signed immediates to generate references to global
> variables. For the kernel, this implies that all global variables must
> reside in the top 2 GiB of the kernel virtual address space, where the
> implicit address bits 63:32 are equal to sign bit 31.
>
> This means the kernel code model is not suitable for other bare metal
> executables such as the kexec purgatory, which can be placed arbitrarily
> in the physical address space, where its address may no longer be
> representable as a sign extended 32-bit quantity. For this reason,
> commit
>
> e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
>
> switched to the 'large' code model, which uses 64-bit immediates for all
> symbol references, including function calls, in order to avoid relying
> on any assumptions regarding proximity of symbols in the final
> executable.
>
> The large code model is rarely used, clunky and the least likely to
> operate in a similar fashion when comparing GCC and Clang, so it is best
> avoided. This is especially true now that Clang 18 has started to emit
> executable code in two separate sections (.text and .ltext), which
> triggers an issue in the kexec loading code at runtime.
>
> Instead, use the position independent small code model, which makes no
> assumptions about placement but only about proximity, where all
> referenced symbols must be within -/+ 2 GiB, i.e., in range for a
> RIP-relative reference. Use hidden visibility to suppress the use of a
> GOT, which carries absolute addresses that are not covered by static ELF
> relocations, and is therefore incompatible with the kexec loader's
> relocation logic.
>
> Cc: Nathan Chancellor <[email protected]>
> Cc: Nick Desaulniers <[email protected]>
Thanks Ard!
Acked-by: Nick Desaulniers <[email protected]>
Reported-by: ns <[email protected]>
Closes: https://github.com/ClangBuiltLinux/linux/issues/2016
Fixes: e16c2983fba0 ("x86/purgatory: Change compiler flags from
-mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
(I don't have a kexec setup ready to go; maybe someone that does can
help test it.)
> Cc: Bill Wendling <[email protected]>
> Cc: Justin Stitt <[email protected]>
> Cc: Song Liu <[email protected]>
> Cc: Ricardo Ribalda <[email protected]>
> Cc: Fangrui Song <[email protected]>
> Cc: Arthur Eubanks <[email protected]>
> Link: https://lore.kernel.org/all/[email protected]/
> Cc: <[email protected]>
> Signed-off-by: Ard Biesheuvel <[email protected]>
> ---
> arch/x86/purgatory/Makefile | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile
> index bc31863c5ee6..a18591f6e6d9 100644
> --- a/arch/x86/purgatory/Makefile
> +++ b/arch/x86/purgatory/Makefile
> @@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n
> # make up the standalone purgatory.ro
>
> PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel
> -PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0
> +PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0
> +PURGATORY_CFLAGS += -fpic -fvisibility=hidden
> PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING
> PURGATORY_CFLAGS += -fno-stack-protector
>
> --
> 2.44.0.769.g3c40516874-goog
>
--
Thanks,
~Nick Desaulniers
On Thu, Apr 18, 2024 at 10:17:06PM +0200, Ard Biesheuvel wrote:
> From: Ard Biesheuvel <[email protected]>
>
> On x86, the ordinary, position dependent 'small' and 'kernel' code models only
> support placement of the executable in 32-bit addressable memory, due to
> the use of 32-bit signed immediates to generate references to global
> variables. For the kernel, this implies that all global variables must
> reside in the top 2 GiB of the kernel virtual address space, where the
> implicit address bits 63:32 are equal to sign bit 31.
>
> This means the kernel code model is not suitable for other bare metal
> executables such as the kexec purgatory, which can be placed arbitrarily
> in the physical address space, where its address may no longer be
> representable as a sign extended 32-bit quantity. For this reason,
> commit
>
> e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
>
> switched to the 'large' code model, which uses 64-bit immediates for all
> symbol references, including function calls, in order to avoid relying
> on any assumptions regarding proximity of symbols in the final
> executable.
>
> The large code model is rarely used, clunky and the least likely to
> operate in a similar fashion when comparing GCC and Clang, so it is best
> avoided. This is especially true now that Clang 18 has started to emit
> executable code in two separate sections (.text and .ltext), which
> triggers an issue in the kexec loading code at runtime.
>
> Instead, use the position independent small code model, which makes no
> assumptions about placement but only about proximity, where all
> referenced symbols must be within -/+ 2 GiB, i.e., in range for a
> RIP-relative reference. Use hidden visibility to suppress the use of a
> GOT, which carries absolute addresses that are not covered by static ELF
> relocations, and is therefore incompatible with the kexec loader's
> relocation logic.
>
> Cc: Nathan Chancellor <[email protected]>
> Cc: Nick Desaulniers <[email protected]>
> Cc: Bill Wendling <[email protected]>
> Cc: Justin Stitt <[email protected]>
> Cc: Song Liu <[email protected]>
> Cc: Ricardo Ribalda <[email protected]>
> Cc: Fangrui Song <[email protected]>
> Cc: Arthur Eubanks <[email protected]>
> Link: https://lore.kernel.org/all/[email protected]/
> Cc: <[email protected]>
> Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Nathan Chancellor <[email protected]>
Tested-by: Nathan Chancellor <[email protected]>
This resolves the warning and relocation overflow error that I see with
LLVM 18.1.4 (the relocation error was fixed in LLVM but dropping
-mcmodel=large resolves it for toolchains without that fix).
> ---
> arch/x86/purgatory/Makefile | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile
> index bc31863c5ee6..a18591f6e6d9 100644
> --- a/arch/x86/purgatory/Makefile
> +++ b/arch/x86/purgatory/Makefile
> @@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n
> # make up the standalone purgatory.ro
>
> PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel
> -PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0
> +PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0
> +PURGATORY_CFLAGS += -fpic -fvisibility=hidden
> PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING
> PURGATORY_CFLAGS += -fno-stack-protector
>
> --
> 2.44.0.769.g3c40516874-goog
>
On 2024-04-18, Nick Desaulniers wrote:
>On Thu, Apr 18, 2024 at 1:17 PM Ard Biesheuvel <[email protected]> wrote:
>>
>> From: Ard Biesheuvel <[email protected]>
>>
>> On x86, the ordinary, position dependent 'small' and 'kernel' code models only
>> support placement of the executable in 32-bit addressable memory, due to
>> the use of 32-bit signed immediates to generate references to global
>> variables. For the kernel, this implies that all global variables must
>> reside in the top 2 GiB of the kernel virtual address space, where the
>> implicit address bits 63:32 are equal to sign bit 31.
>>
>> This means the kernel code model is not suitable for other bare metal
>> executables such as the kexec purgatory, which can be placed arbitrarily
>> in the physical address space, where its address may no longer be
>> representable as a sign extended 32-bit quantity. For this reason,
>> commit
>>
>> e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
>>
>> switched to the 'large' code model, which uses 64-bit immediates for all
>> symbol references, including function calls, in order to avoid relying
>> on any assumptions regarding proximity of symbols in the final
>> executable.
>>
>> The large code model is rarely used, clunky and the least likely to
>> operate in a similar fashion when comparing GCC and Clang, so it is best
>> avoided. This is especially true now that Clang 18 has started to emit
>> executable code in two separate sections (.text and .ltext), which
>> triggers an issue in the kexec loading code at runtime.
>>
>> Instead, use the position independent small code model, which makes no
>> assumptions about placement but only about proximity, where all
>> referenced symbols must be within -/+ 2 GiB, i.e., in range for a
>> RIP-relative reference. Use hidden visibility to suppress the use of a
>> GOT, which carries absolute addresses that are not covered by static ELF
>> relocations, and is therefore incompatible with the kexec loader's
>> relocation logic.
>>
>> Cc: Nathan Chancellor <[email protected]>
>> Cc: Nick Desaulniers <[email protected]>
>
>Thanks Ard!
>
>Acked-by: Nick Desaulniers <[email protected]>
>Reported-by: ns <[email protected]>
>Closes: https://github.com/ClangBuiltLinux/linux/issues/2016
>Fixes: e16c2983fba0 ("x86/purgatory: Change compiler flags from
>-mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
>
>(I don't have a kexec setup ready to go; maybe someone that does can
>help test it.)
LGTM.
Position-dependent small code model may generate R_X86_64_32S
relocations with a range of [0,2GiB) (the negative half cannot be used).
Position-independent small code model with hidden visibility will
generate R_X86_64_PC32 and can typically be quite larger than 2G without
hitting an overflow issue.
(I have some notes about R_X86_64_32S at
https://maskray.me/blog/2023-05-14-relocation-overflow-and-code-models#x86-64-linker-requirement)
Reviewed-by: Fangrui Song <[email protected]>
>> Cc: Bill Wendling <[email protected]>
>> Cc: Justin Stitt <[email protected]>
>> Cc: Song Liu <[email protected]>
>> Cc: Ricardo Ribalda <[email protected]>
>> Cc: Fangrui Song <[email protected]>
>> Cc: Arthur Eubanks <[email protected]>
>> Link: https://lore.kernel.org/all/[email protected]/
>> Cc: <[email protected]>
>> Signed-off-by: Ard Biesheuvel <[email protected]>
>> ---
>> arch/x86/purgatory/Makefile | 3 ++-
>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile
>> index bc31863c5ee6..a18591f6e6d9 100644
>> --- a/arch/x86/purgatory/Makefile
>> +++ b/arch/x86/purgatory/Makefile
>> @@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n
>> # make up the standalone purgatory.ro
>>
>> PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel
>> -PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0
>> +PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0
>> +PURGATORY_CFLAGS += -fpic -fvisibility=hidden
>> PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING
>> PURGATORY_CFLAGS += -fno-stack-protector
>>
>> --
>> 2.44.0.769.g3c40516874-goog
>>
>
>
>--
>Thanks,
>~Nick Desaulniers
On 18. 04. 24, 22:17, Ard Biesheuvel wrote:
> From: Ard Biesheuvel <[email protected]>
>
> On x86, the ordinary, position dependent 'small' and 'kernel' code models only
> support placement of the executable in 32-bit addressable memory, due to
> the use of 32-bit signed immediates to generate references to global
> variables. For the kernel, this implies that all global variables must
> reside in the top 2 GiB of the kernel virtual address space, where the
> implicit address bits 63:32 are equal to sign bit 31.
>
> This means the kernel code model is not suitable for other bare metal
> executables such as the kexec purgatory, which can be placed arbitrarily
> in the physical address space, where its address may no longer be
> representable as a sign extended 32-bit quantity. For this reason,
> commit
>
> e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
>
> switched to the 'large' code model, which uses 64-bit immediates for all
> symbol references, including function calls, in order to avoid relying
> on any assumptions regarding proximity of symbols in the final
> executable.
>
> The large code model is rarely used, clunky and the least likely to
> operate in a similar fashion when comparing GCC and Clang, so it is best
> avoided. This is especially true now that Clang 18 has started to emit
> executable code in two separate sections (.text and .ltext), which
> triggers an issue in the kexec loading code at runtime.
>
> Instead, use the position independent small code model, which makes no
> assumptions about placement but only about proximity, where all
> referenced symbols must be within -/+ 2 GiB, i.e., in range for a
> RIP-relative reference. Use hidden visibility to suppress the use of a
> GOT, which carries absolute addresses that are not covered by static ELF
> relocations, and is therefore incompatible with the kexec loader's
> relocation logic.
FWIW:
Fixes: https://bugzilla.suse.com/show_bug.cgi?id=1211853
thanks,
--
js
suse labs
On Fri, Apr 19, 2024 at 01:35:44PM +0200, Jiri Slaby wrote:
> On 18. 04. 24, 22:17, Ard Biesheuvel wrote:
> > From: Ard Biesheuvel <[email protected]>
> >
> > On x86, the ordinary, position dependent 'small' and 'kernel' code models only
> > support placement of the executable in 32-bit addressable memory, due to
> > the use of 32-bit signed immediates to generate references to global
> > variables. For the kernel, this implies that all global variables must
> > reside in the top 2 GiB of the kernel virtual address space, where the
> > implicit address bits 63:32 are equal to sign bit 31.
> >
> > This means the kernel code model is not suitable for other bare metal
> > executables such as the kexec purgatory, which can be placed arbitrarily
> > in the physical address space, where its address may no longer be
> > representable as a sign extended 32-bit quantity. For this reason,
> > commit
> >
> > e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
> >
> > switched to the 'large' code model, which uses 64-bit immediates for all
> > symbol references, including function calls, in order to avoid relying
> > on any assumptions regarding proximity of symbols in the final
> > executable.
> >
> > The large code model is rarely used, clunky and the least likely to
> > operate in a similar fashion when comparing GCC and Clang, so it is best
> > avoided. This is especially true now that Clang 18 has started to emit
> > executable code in two separate sections (.text and .ltext), which
> > triggers an issue in the kexec loading code at runtime.
> >
> > Instead, use the position independent small code model, which makes no
> > assumptions about placement but only about proximity, where all
> > referenced symbols must be within -/+ 2 GiB, i.e., in range for a
> > RIP-relative reference. Use hidden visibility to suppress the use of a
> > GOT, which carries absolute addresses that are not covered by static ELF
> > relocations, and is therefore incompatible with the kexec loader's
> > relocation logic.
>
> FWIW:
>
> Fixes: https://bugzilla.suse.com/show_bug.cgi?id=1211853
Interesting. I thought gcc doesn't have problems here yet and was
talking to Matz on Thu about it and it seems he's forgotten about his
statement too that "you should simply stop using -mcmodel=large. Noone
should use it." :-)
Ok then, lemme queue Ard's fix.
I guess we want it in stable too but as it is pretty much the case with
toolchain fixes I'd prefer if we queued it the normal path for 6.10 and
then backport it after it sees some testing and nothing breaks. And not
rush it to Linus now.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
On Thu, Apr 18, 2024 at 01:36:22PM -0700, Nick Desaulniers wrote:
> Reported-by: ns <[email protected]>
I don't know what the goal with this tag is but this'll keep bouncing:
<[email protected]>: Host or domain name not found. Name service
error for name=users.noreply.github.com type=AAAA: Host not found
Maybe use a non-void email...?
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
On Sat, 20 Apr 2024 at 17:30, Borislav Petkov <[email protected]> wrote:
>
> On Thu, Apr 18, 2024 at 01:36:22PM -0700, Nick Desaulniers wrote:
> > Reported-by: ns <[email protected]>
>
> I don't know what the goal with this tag is but this'll keep bouncing:
>
> <[email protected]>: Host or domain name not found. Name service
> error for name=users.noreply.github.com type=AAAA: Host not found
>
> Maybe use a non-void email...?
>
Let's just drop that.
If people cannot be bothered to use their real name, I don't think
they care about being credited either.
Hello,
On Sat, 20 Apr 2024, Borislav Petkov wrote:
> Interesting. I thought gcc doesn't have problems here yet and was
> talking to Matz on Thu about it and it seems he's forgotten about his
> statement too that "you should simply stop using -mcmodel=large. Noone
> should use it." :-)
It may be so ingrained in my brain that I'm not _always_ saying it when
talking about the large code model over a beer. And indeed I know of no
particular problems with it vis GCC, but that doesn't mean it's a good
idea to use :-)
So once again: "everyone should simply stop using -mcmodel=large. Noone
should use it."
So the patch goes strictly into the direction of betterment of the
universe. :)
Ciao,
Michael.
On Mon, Apr 29, 2024 at 02:05:12PM +0200, Michael Matz wrote:
> It may be so ingrained in my brain that I'm not _always_ saying it when
> talking about the large code model over a beer.
Doh, you should. This is what you start with! :-P
> And indeed I know of no particular problems with it vis GCC, but that
> doesn't mean it's a good idea to use :-)
>
> So once again: "everyone should simply stop using -mcmodel=large. Noone
> should use it."
>
> So the patch goes strictly into the direction of betterment of the
> universe. :)
Yeah, it is already on its way to every kernel near you. And looka here:
$ git grep mcmodel=large
arch/powerpc/Makefile:125: # 64bit relocation for this to work, hence -mcmodel=large.
arch/powerpc/Makefile:126: KBUILD_CFLAGS_MODULE += -mcmodel=large
arch/um/Makefile:34: KBUILD_CFLAGS += -mcmodel=large
x86 is all free of the large model now.
One less thing to worry about - gazillion more to go.
:-P
See ya on Thu.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
Hi,
On 29. 04. 24, 14:05, Michael Matz wrote:
> On Sat, 20 Apr 2024, Borislav Petkov wrote:
>
>> Interesting. I thought gcc doesn't have problems here yet and was
>> talking to Matz on Thu about it and it seems he's forgotten about his
>> statement too that "you should simply stop using -mcmodel=large. Noone
>> should use it." :-)
>
> It may be so ingrained in my brain that I'm not _always_ saying it when
> talking about the large code model over a beer. And indeed I know of no
> particular problems with it vis GCC,
Of course you do :). That bsc#1211853 I linked earlier. I.e. gcc-13 +
-fstrict-flex-arrays=3 + -mcmodel=large + some asm() expecting __FILE__
to be constant (not true with the large model).
regards,
--
js
suse labs
Hello,
On Tue, 30 Apr 2024, Jiri Slaby wrote:
> >> Interesting. I thought gcc doesn't have problems here yet and was
> >> talking to Matz on Thu about it and it seems he's forgotten about his
> >> statement too that "you should simply stop using -mcmodel=large. Noone
> >> should use it." :-)
> >
> > It may be so ingrained in my brain that I'm not _always_ saying it when
> > talking about the large code model over a beer. And indeed I know of no
> > particular problems with it vis GCC,
>
> Of course you do :).
:-P
> That bsc#1211853 I linked earlier. I.e. gcc-13 +
> -fstrict-flex-arrays=3 + -mcmodel=large + some asm() expecting __FILE__ to be
> constant (not true with the large model).
"asm() expecting $whatever" - clearly a user problem, not a GCC problem
;-)
Ciao,
Michael.