While testing -tip after the IBT bits were merged, our continuous
integration noticed a crash with clang-11:
https://builds.tuxbuild.com/26ZRIuAPHx1L802ExiYCuLuvMhK/build.log
This crash is a separate issue from the one that is already being
checked for with '-fcf-protection=branch -mfentry -pg'. This new crash
was fixed in clang-12:
https://github.com/llvm/llvm-project/commit/e0b89df2e0f0130881bf6c39bf31d7f6aac00e0f
Unfortunately, I have not been able to tease out a simplified reproducer
for this crash that always triggers. The reproducer in that commit
message and the reproducer commented in the LLVM test do not trigger all
the time (at least with the clang-11 that is in Debian/Ubuntu,
assertions might make the crash always happen but we cannot count on
that); even the full preprocessed files straight from the kernel source
do not always trigger a crash.
As a result, this series proposes just having a hard version check for
clang 14.0.0 and newer, which will cover this new clang-11 crash and the
'-fcf-protection=branch -mfentry -pg' crash.
Adding a check for a version of clang that has not been released yet is
a little riskier than dynamically testing the clang binary for an issue,
as it is possible that someone is using/testing a version of clang from
before a particular issue was fixed, while still having the same version
number. However, our policy for version checks in the kernel has always
been to use the final version of clang that will have the issue fixed,
as people should be upgrading their compilers often if they are using
prereleased versions.
In this particular instance, the '-fcf-protection=branch -mfentry -pg'
crash has been fixed for over three months, which is plenty of time to
upgrade:
https://github.com/llvm/llvm-project/commit/dfcf69770bc522b9e411c66454934a37c1f35332
At the moment, I only know of one version of clang that is fairly widely
used that will show this crash, which is Android's clang 14.0.2. We are
in the process of getting a newer version (14.0.3) deployed to minimize
the impact this change will have on various testing groups:
https://android-review.googlesource.com/c/kernel/common/+/2032664
https://android-review.googlesource.com/c/kernel/common/+/2032665
https://android-review.googlesource.com/c/kernel/common/+/2032666
https://gitlab.com/Linaro/tuxmake/-/merge_requests/244
If for some reason there are any reports of issues after this change,
feel free to direct them to this cover letter and tell them to upgrade
their compiler :)
The second patch just codifies the result of this discussion and issue:
https://lore.kernel.org/r/202202241627.EEF3D5D2@keescook/
https://github.com/ClangBuiltLinux/linux/issues/1606
This is based on -tip x86/core and it has survived an allmodconfig build
with clang-11 through clang-15 (CC=clang and LLVM=1).
Nathan Chancellor (2):
x86/Kconfig: Only enable CONFIG_CC_HAS_IBT for clang >= 14.0.0
x86/Kconfig: Only allow CONFIG_X86_KERNEL_IBT with ld.lld >= 14.0.0
arch/x86/Kconfig | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
base-commit: 2f35e67f621fffc636cb802a4f93fd168cf38274
--
2.35.1
With CONFIG_X86_KERNEL_IBT=y and a version of ld.lld prior to 14.0.0,
there are numerous objtool warnings along the lines of:
warning: objtool: .plt+0x6: indirect jump found in RETPOLINE build
This is a known issue that has been resolved in ld.lld 14.0.0. Prevent
CONFIG_X86_KERNEL_IBT from being selectable when using one of these
problematic ld.lld versions.
Link: https://github.com/llvm/llvm-project/commit/9d7001eba9c4cb311e03cd8cdc231f9e579f2d0f
Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/x86/Kconfig | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 921e4ebda564..87579264aa00 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1875,6 +1875,8 @@ config X86_KERNEL_IBT
prompt "Indirect Branch Tracking"
bool
depends on X86_64 && CC_HAS_IBT && STACK_VALIDATION
+ # https://github.com/llvm/llvm-project/commit/9d7001eba9c4cb311e03cd8cdc231f9e579f2d0f
+ depends on !LD_IS_LLD || LLD_VERSION >= 140000
help
Build the kernel with support for Indirect Branch Tracking, a
hardware support course-grain forward-edge Control Flow Integrity
--
2.35.1
Commit 156ff4a544ae ("x86/ibt: Base IBT bits") added a check for a crash
with 'clang -fcf-protection=branch -mfentry -pg', which intended to
exclude Clang versions older than 14.0.0 from selecting
CONFIG_X86_KERNEL_IBT.
clang-11 does not have the issue that the check is testing for, so
CONFIG_X86_KERNEL_IBT is selectable. Unfortunately, there is a different
crash in clang-11 that was fixed in clang-12. To make matters worse,
that crash does not appear to be entirely deterministic, as the same
input to the compiler will sometimes crash and other times not, which
makes dynamically checking for the crash like the '-pg' one unreliable.
To make everything work properly for all common versions of clang, use a
hard version check of 14.0.0, as that will be the first release upstream
that has both bugs properly fixed.
Link: https://github.com/llvm/llvm-project/commit/e0b89df2e0f0130881bf6c39bf31d7f6aac00e0f
Signed-off-by: Nathan Chancellor <[email protected]>
---
arch/x86/Kconfig | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 0f0672d2c816..921e4ebda564 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1865,9 +1865,10 @@ config CC_HAS_IBT
# GCC >= 9 and binutils >= 2.29
# Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654
# Clang/LLVM >= 14
- # fentry check to work around https://reviews.llvm.org/D111108
+ # https://github.com/llvm/llvm-project/commit/e0b89df2e0f0130881bf6c39bf31d7f6aac00e0f
+ # https://github.com/llvm/llvm-project/commit/dfcf69770bc522b9e411c66454934a37c1f35332
def_bool ((CC_IS_GCC && $(cc-option, -fcf-protection=branch -mindirect-branch-register)) || \
- (CC_IS_CLANG && $(success,echo "void a(void) {}" | $(CC) -Werror $(CLANG_FLAGS) -fcf-protection=branch -mfentry -pg -x c - -c -o /dev/null))) && \
+ (CC_IS_CLANG && CLANG_VERSION >= 140000)) && \
$(as-instr,endbr64)
config X86_KERNEL_IBT
--
2.35.1
The following commit has been merged into the x86/core branch of tip:
Commit-ID: 71a7580b17e9510e5f74c0687567f9d8b5d48f5d
Gitweb: https://git.kernel.org/tip/71a7580b17e9510e5f74c0687567f9d8b5d48f5d
Author: Nathan Chancellor <[email protected]>
AuthorDate: Fri, 18 Mar 2022 16:07:46 -07:00
Committer: Peter Zijlstra <[email protected]>
CommitterDate: Sat, 19 Mar 2022 00:47:12 +01:00
x86/Kconfig: Only enable CONFIG_CC_HAS_IBT for clang >= 14.0.0
Commit 156ff4a544ae ("x86/ibt: Base IBT bits") added a check for a crash
with 'clang -fcf-protection=branch -mfentry -pg', which intended to
exclude Clang versions older than 14.0.0 from selecting
CONFIG_X86_KERNEL_IBT.
clang-11 does not have the issue that the check is testing for, so
CONFIG_X86_KERNEL_IBT is selectable. Unfortunately, there is a different
crash in clang-11 that was fixed in clang-12. To make matters worse,
that crash does not appear to be entirely deterministic, as the same
input to the compiler will sometimes crash and other times not, which
makes dynamically checking for the crash like the '-pg' one unreliable.
To make everything work properly for all common versions of clang, use a
hard version check of 14.0.0, as that will be the first release upstream
that has both bugs properly fixed.
Signed-off-by: Nathan Chancellor <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
---
arch/x86/Kconfig | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 0f0672d..921e4eb 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1865,9 +1865,10 @@ config CC_HAS_IBT
# GCC >= 9 and binutils >= 2.29
# Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654
# Clang/LLVM >= 14
- # fentry check to work around https://reviews.llvm.org/D111108
+ # https://github.com/llvm/llvm-project/commit/e0b89df2e0f0130881bf6c39bf31d7f6aac00e0f
+ # https://github.com/llvm/llvm-project/commit/dfcf69770bc522b9e411c66454934a37c1f35332
def_bool ((CC_IS_GCC && $(cc-option, -fcf-protection=branch -mindirect-branch-register)) || \
- (CC_IS_CLANG && $(success,echo "void a(void) {}" | $(CC) -Werror $(CLANG_FLAGS) -fcf-protection=branch -mfentry -pg -x c - -c -o /dev/null))) && \
+ (CC_IS_CLANG && CLANG_VERSION >= 140000)) && \
$(as-instr,endbr64)
config X86_KERNEL_IBT
The following commit has been merged into the x86/core branch of tip:
Commit-ID: 46a5cf598a940e674a574664c32ad3518dc9ed6f
Gitweb: https://git.kernel.org/tip/46a5cf598a940e674a574664c32ad3518dc9ed6f
Author: Nathan Chancellor <[email protected]>
AuthorDate: Fri, 18 Mar 2022 16:07:47 -07:00
Committer: Peter Zijlstra <[email protected]>
CommitterDate: Sat, 19 Mar 2022 00:47:13 +01:00
x86/Kconfig: Only allow CONFIG_X86_KERNEL_IBT with ld.lld >= 14.0.0
With CONFIG_X86_KERNEL_IBT=y and a version of ld.lld prior to 14.0.0,
there are numerous objtool warnings along the lines of:
warning: objtool: .plt+0x6: indirect jump found in RETPOLINE build
This is a known issue that has been resolved in ld.lld 14.0.0. Prevent
CONFIG_X86_KERNEL_IBT from being selectable when using one of these
problematic ld.lld versions.
Signed-off-by: Nathan Chancellor <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
---
arch/x86/Kconfig | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 921e4eb..8757926 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1875,6 +1875,8 @@ config X86_KERNEL_IBT
prompt "Indirect Branch Tracking"
bool
depends on X86_64 && CC_HAS_IBT && STACK_VALIDATION
+ # https://github.com/llvm/llvm-project/commit/9d7001eba9c4cb311e03cd8cdc231f9e579f2d0f
+ depends on !LD_IS_LLD || LLD_VERSION >= 140000
help
Build the kernel with support for Indirect Branch Tracking, a
hardware support course-grain forward-edge Control Flow Integrity
The following commit has been merged into the x86/core branch of tip:
Commit-ID: 262448f3d18959d175b10e28a3b65f41d1d7313f
Gitweb: https://git.kernel.org/tip/262448f3d18959d175b10e28a3b65f41d1d7313f
Author: Nathan Chancellor <[email protected]>
AuthorDate: Fri, 18 Mar 2022 16:07:46 -07:00
Committer: Peter Zijlstra <[email protected]>
CommitterDate: Tue, 22 Mar 2022 21:12:13 +01:00
x86/Kconfig: Only enable CONFIG_CC_HAS_IBT for clang >= 14.0.0
Commit 156ff4a544ae ("x86/ibt: Base IBT bits") added a check for a crash
with 'clang -fcf-protection=branch -mfentry -pg', which intended to
exclude Clang versions older than 14.0.0 from selecting
CONFIG_X86_KERNEL_IBT.
clang-11 does not have the issue that the check is testing for, so
CONFIG_X86_KERNEL_IBT is selectable. Unfortunately, there is a different
crash in clang-11 that was fixed in clang-12. To make matters worse,
that crash does not appear to be entirely deterministic, as the same
input to the compiler will sometimes crash and other times not, which
makes dynamically checking for the crash like the '-pg' one unreliable.
To make everything work properly for all common versions of clang, use a
hard version check of 14.0.0, as that will be the first release upstream
that has both bugs properly fixed.
Signed-off-by: Nathan Chancellor <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
---
arch/x86/Kconfig | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 0f0672d..921e4eb 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1865,9 +1865,10 @@ config CC_HAS_IBT
# GCC >= 9 and binutils >= 2.29
# Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654
# Clang/LLVM >= 14
- # fentry check to work around https://reviews.llvm.org/D111108
+ # https://github.com/llvm/llvm-project/commit/e0b89df2e0f0130881bf6c39bf31d7f6aac00e0f
+ # https://github.com/llvm/llvm-project/commit/dfcf69770bc522b9e411c66454934a37c1f35332
def_bool ((CC_IS_GCC && $(cc-option, -fcf-protection=branch -mindirect-branch-register)) || \
- (CC_IS_CLANG && $(success,echo "void a(void) {}" | $(CC) -Werror $(CLANG_FLAGS) -fcf-protection=branch -mfentry -pg -x c - -c -o /dev/null))) && \
+ (CC_IS_CLANG && CLANG_VERSION >= 140000)) && \
$(as-instr,endbr64)
config X86_KERNEL_IBT
The following commit has been merged into the x86/core branch of tip:
Commit-ID: f6a2c2b2de817078ac5a7e58c10e746165e7825d
Gitweb: https://git.kernel.org/tip/f6a2c2b2de817078ac5a7e58c10e746165e7825d
Author: Nathan Chancellor <[email protected]>
AuthorDate: Fri, 18 Mar 2022 16:07:47 -07:00
Committer: Peter Zijlstra <[email protected]>
CommitterDate: Tue, 22 Mar 2022 21:12:13 +01:00
x86/Kconfig: Only allow CONFIG_X86_KERNEL_IBT with ld.lld >= 14.0.0
With CONFIG_X86_KERNEL_IBT=y and a version of ld.lld prior to 14.0.0,
there are numerous objtool warnings along the lines of:
warning: objtool: .plt+0x6: indirect jump found in RETPOLINE build
This is a known issue that has been resolved in ld.lld 14.0.0. Prevent
CONFIG_X86_KERNEL_IBT from being selectable when using one of these
problematic ld.lld versions.
Signed-off-by: Nathan Chancellor <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
---
arch/x86/Kconfig | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 921e4eb..8757926 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1875,6 +1875,8 @@ config X86_KERNEL_IBT
prompt "Indirect Branch Tracking"
bool
depends on X86_64 && CC_HAS_IBT && STACK_VALIDATION
+ # https://github.com/llvm/llvm-project/commit/9d7001eba9c4cb311e03cd8cdc231f9e579f2d0f
+ depends on !LD_IS_LLD || LLD_VERSION >= 140000
help
Build the kernel with support for Indirect Branch Tracking, a
hardware support course-grain forward-edge Control Flow Integrity