This series enables objtool to start doing stack validation and orc
generation on arm64 kernel builds.
Based on Julien's previous work(1)(2), Now I have finished most of work
for objtool enable on arm64. This series includes objtool part [1-13]
and arm64 support part [14-33], the second part is to make objtool run
correctly with no warning on arm64 so if necessary it can be taken apart
as two series.
ORC generation feature is implemented but not used because we don't have
an unwinder_orc on arm64, now it only be used to check whether objtool
has correct validation.
This series depends on series:
"objtool: Reorganize x86 arch-specific code"
(https://lkml.org/lkml/2022/6/22/463)
I moved some changes which work for all architectures to that series
because this one becomes too big now.
And two series are rebased to tip/objtool/core branch:
'commit 22682a07acc3 ("objtool: Fix objtool regression on x32 systems")'
I think the series is complete enough so I removed the RFC.
However now there are still (maybe only) two unsolved problems:
1. the switch optimization and dynamic jump
Although this problem has been troubling us for a long time and
temporarily resolved by -no-jump-table. Hopefully we have chance
to fix it. I'm trying for that and it will be updated in next version.
2. alternative_cb
alternative_cb in arm64 is an dynamic patch mechanism on arm64 which is
similar to alternative instruction but doesn't give instructions explicitly.
It cause trouble when there is a jump instructions inside it and causes some
branch that objtool can't reach.
I have solved some of them by adding UNWIND_HINT one macro
"mitigate_spectre_bhb_loop" can't be fixed because it is used in
different assembly files.
(1) https://lkml.org/lkml/2021/3/3/1135
(2) https://github.com/julien-thierry/linux.git
---
v6 Changes:
Fix conflicts makes patches can't be applied.
v5 Changes:
Compare to last RFC v4 series, this series does a lot of changes, including
code rebase, refactoring and solution for several warning in last series.
After refactoring and moving some patches out, the number of lines in this
series decrease a thousand.
Here are changes for each patch.
[1] Removed unnecessary part from insn.h and insn.c to avoid extra
dependencies and wired hack. So the patch to add head files also be removed.
[2] Rebase Makefile, clear sync-check for insn.h/c and add arm64 elf
relocation types in elf.h.
Remove other callee saved registers tracking except FP and LR. Beacause
in arm64 there are lots of branches store these CSRs in different order
or positions, tracking them will cause some false stack state mismatch
warnings.
[3-6] Refactor code. Add some macros to make code more clean.
For instructions load/store on stack, Simplifies the logic of moving SP
and accessing data because the order of ops makes no differences.
[7-8] Remove delete in record_invalid_insn because now we don't need to
handle special undecoded instructions.
[13] New patch to enable ORC generation.
[12, 14, 17] Rebase to current branch.
[19-21] Add annotations for trans_pgd-asm.S and some unreachable
branches.
[29] Moves kuser32.S and sigreturn32.S to rodata because they are VDSO.
[31] Remove ignores for hibernate.c.
[33] New patch to fix a fallthrough problem.
v4 Changes:
- fix EX_ENTRY_SIZE from 8 to 12.
- modify arm64 for supporting objtool, including annotation,
asm code modification, ignoring some validation, to make objtool
be enable to pass arm64 builds.
v3 Changes:
- rebase Julien's version to mainstream and solve conflicts.
- Merge dumplicate "*type = INSN_OTHER".
- When meeting unrecognized instructions such as datas
in .text code or 0x0 padding insns, last version used
"loc->ignorable" to mark and remove them from objtool insn list.
However there are two problems to do so:
1. when meeting insns can't be decoded or excluded, objtool will
just stop.
2. deleting every insn can cause problems in fellow procedure.
So I changed "record_invalid_insn" that we can delete one insn or
just set it ignored. Now check will throw an error and going on when
meeting undecodable instructions.
Also, to prevent the confusion between "loc->ignorable" and
"insn->ignore" I changed "ignore" to "delete".
v2 Changes:
- Drop gcc plugin in favor of -fno-jump-tables
- miscelaneous fixes and cleanups
___
Chen Zhongjin (33):
tools: arm64: Make aarch64 instruction decoder available to tools
objtool: arm64: Add base definition for arm64 backend
objtool: arm64: Decode add/sub instructions
objtool: arm64: Decode jump and call related instructions
objtool: arm64: Decode other system instructions
objtool: arm64: Decode load/store instructions
objtool: arm64: Decode LDR instructions
objtool: arm64: Accept non-instruction data in code sections
objtool: check: Support data in text section
objtool: arm64: Handle supported relocations in alternatives
objtool: arm64: Ignore replacement section for alternative callback
objtool: arm64: Enable stack validation for arm64
objtool: arm64: Enable ORC for arm64
objtool: arm64: Add annotate_reachable() for objtools
arm64: bug: Add reachable annotation to warning macros
arm64: kgdb: Add reachable annotation after kgdb brk
objtool: arm64: Add unwind_hint support
arm64: Change symbol type annotations
arm64: Annotate unwind_hint for symbols with empty stack
arm64: entry: Annotate unwind_hint for entry
arm64: kvm: Annotate unwind_hint for hyp entry
arm64: efi-header: Mark efi header as data
arm64: head: Mark constants as data
arm64: proc: Mark constant as data
arm64: crypto: Mark constant as data
arm64: crypto: Remove unnecessary stackframe
arm64: Set intra-function call annotations
arm64: sleep: Properly set frame pointer before call
arm64: compat: Move VDSO code to .rodata section
arm64: entry: Align stack size for alternative
arm64: kernel: Skip validation of proton-pack.c
arm64: irq-gic: Replace unreachable() with -EINVAL
objtool: revert c_file fallthrough detection for arm64
arch/arm64/Kconfig | 2 +
arch/arm64/Kconfig.debug | 31 ++
arch/arm64/Makefile | 4 +
arch/arm64/crypto/aes-neonbs-core.S | 14 +-
arch/arm64/crypto/crct10dif-ce-core.S | 5 +
arch/arm64/crypto/poly1305-armv8.pl | 4 +
arch/arm64/crypto/sha512-armv8.pl | 29 +-
arch/arm64/include/asm/assembler.h | 2 +
arch/arm64/include/asm/bug.h | 6 +-
arch/arm64/include/asm/kgdb.h | 1 +
arch/arm64/include/asm/module.h | 7 +
arch/arm64/include/asm/orc_types.h | 68 +++
arch/arm64/include/asm/unwind_hints.h | 26 +
arch/arm64/kernel/cpu-reset.S | 2 +
arch/arm64/kernel/efi-entry.S | 2 +
arch/arm64/kernel/efi-header.S | 2 +
arch/arm64/kernel/entry.S | 30 +-
arch/arm64/kernel/head.S | 76 +--
arch/arm64/kernel/hibernate-asm.S | 2 +
arch/arm64/kernel/kuser32.S | 1 +
arch/arm64/kernel/proton-pack.c | 2 +
arch/arm64/kernel/relocate_kernel.S | 2 +
arch/arm64/kernel/sigreturn32.S | 1 +
arch/arm64/kernel/sleep.S | 8 +-
arch/arm64/kernel/vmlinux.lds.S | 3 +
arch/arm64/kvm/hyp/entry.S | 9 +-
arch/arm64/kvm/hyp/hyp-entry.S | 4 +
arch/arm64/kvm/hyp/vgic-v3-sr.c | 7 +-
arch/arm64/mm/proc.S | 4 +-
arch/arm64/mm/trans_pgd-asm.S | 3 +
drivers/irqchip/irq-gic-v3.c | 2 +-
include/linux/compiler.h | 9 +
scripts/Makefile | 6 +-
tools/arch/arm64/include/asm/insn.h | 458 +++++++++++++++++
tools/arch/arm64/include/asm/orc_types.h | 68 +++
tools/arch/arm64/include/asm/unwind_hints.h | 26 +
tools/arch/arm64/lib/insn.c | 335 +++++++++++++
tools/objtool/Makefile | 5 +
tools/objtool/arch/arm64/Build | 9 +
tools/objtool/arch/arm64/decode.c | 465 ++++++++++++++++++
.../arch/arm64/include/arch/cfi_regs.h | 14 +
tools/objtool/arch/arm64/include/arch/elf.h | 12 +
.../arch/arm64/include/arch/endianness.h | 9 +
.../objtool/arch/arm64/include/arch/special.h | 22 +
tools/objtool/arch/arm64/orc.c | 117 +++++
tools/objtool/arch/arm64/special.c | 36 ++
tools/objtool/arch/x86/decode.c | 5 +
tools/objtool/check.c | 22 +-
tools/objtool/elf.c | 14 +
tools/objtool/include/objtool/arch.h | 2 +
tools/objtool/include/objtool/elf.h | 1 +
tools/objtool/include/objtool/objtool.h | 2 +-
tools/objtool/objtool.c | 1 +
53 files changed, 1924 insertions(+), 73 deletions(-)
create mode 100644 arch/arm64/include/asm/orc_types.h
create mode 100644 arch/arm64/include/asm/unwind_hints.h
create mode 100644 tools/arch/arm64/include/asm/insn.h
create mode 100644 tools/arch/arm64/include/asm/orc_types.h
create mode 100644 tools/arch/arm64/include/asm/unwind_hints.h
create mode 100644 tools/arch/arm64/lib/insn.c
create mode 100644 tools/objtool/arch/arm64/Build
create mode 100644 tools/objtool/arch/arm64/decode.c
create mode 100644 tools/objtool/arch/arm64/include/arch/cfi_regs.h
create mode 100644 tools/objtool/arch/arm64/include/arch/elf.h
create mode 100644 tools/objtool/arch/arm64/include/arch/endianness.h
create mode 100644 tools/objtool/arch/arm64/include/arch/special.h
create mode 100644 tools/objtool/arch/arm64/orc.c
create mode 100644 tools/objtool/arch/arm64/special.c
--
2.17.1
Decode branch, branch and link (aarch64's call) and return instructions.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/objtool/arch/arm64/decode.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/decode.c
index d8c32703874d..40ada17d0842 100644
--- a/tools/objtool/arch/arm64/decode.c
+++ b/tools/objtool/arch/arm64/decode.c
@@ -212,6 +212,27 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
}
}
break;
+ case AARCH64_INSN_CLS_BR_SYS:
+ if (aarch64_insn_is_ret(insn) &&
+ aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RN, insn)
+ == AARCH64_INSN_REG_LR) {
+ *type = INSN_RETURN;
+ } else if (aarch64_insn_is_bl(insn)) {
+ *type = INSN_CALL;
+ *immediate = aarch64_get_branch_offset(insn);
+ } else if (aarch64_insn_is_blr(insn)) {
+ *type = INSN_CALL_DYNAMIC;
+ } else if (aarch64_insn_is_b(insn)) {
+ *type = INSN_JUMP_UNCONDITIONAL;
+ *immediate = aarch64_get_branch_offset(insn);
+ } else if (aarch64_insn_is_br(insn)) {
+ *type = INSN_JUMP_DYNAMIC;
+ } else if (aarch64_insn_is_branch_imm(insn)) {
+ /* Remaining branch opcodes are conditional */
+ *type = INSN_JUMP_CONDITIONAL;
+ *immediate = aarch64_get_branch_offset(insn);
+ }
+ break;
default:
break;
}
--
2.17.1
x86 removed annotate_reachable and replaced it with ASM_REACHABLE
which is not suitable for arm64 micro because there are some cases
GCC will merge duplicate inline asm.
Re-add annotation_reachable() for arm64.
Signed-off-by: Chen Zhongjin <[email protected]>
---
include/linux/compiler.h | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 01ce94b58b42..c8bce9421fa7 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -117,6 +117,14 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
*/
#define __stringify_label(n) #n
+#define __annotate_reachable(c) ({ \
+ asm volatile(__stringify_label(c) ":\n\t" \
+ ".pushsection .discard.reachable\n\t" \
+ ".long " __stringify_label(c) "b - .\n\t" \
+ ".popsection\n\t"); \
+})
+#define annotate_reachable() __annotate_reachable(__COUNTER__)
+
#define __annotate_unreachable(c) ({ \
asm volatile(__stringify_label(c) ":\n\t" \
".pushsection .discard.unreachable\n\t" \
@@ -129,6 +137,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
#define __annotate_jump_table __section(".rodata..c_jump_table")
#else /* !CONFIG_OBJTOOL */
+#define annotate_reachable()
#define annotate_unreachable()
#define __annotate_jump_table
#endif /* CONFIG_OBJTOOL */
--
2.17.1
Add orc_type, orc build and ld options for arm64.
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/Kconfig.debug | 10 ++
arch/arm64/include/asm/module.h | 7 ++
arch/arm64/include/asm/orc_types.h | 68 +++++++++++++
arch/arm64/kernel/vmlinux.lds.S | 3 +
scripts/Makefile | 6 +-
tools/arch/arm64/include/asm/orc_types.h | 68 +++++++++++++
tools/objtool/Makefile | 1 +
tools/objtool/arch/arm64/Build | 1 +
tools/objtool/arch/arm64/orc.c | 117 +++++++++++++++++++++++
9 files changed, 280 insertions(+), 1 deletion(-)
create mode 100644 arch/arm64/include/asm/orc_types.h
create mode 100644 tools/arch/arm64/include/asm/orc_types.h
create mode 100644 tools/objtool/arch/arm64/orc.c
diff --git a/arch/arm64/Kconfig.debug b/arch/arm64/Kconfig.debug
index c2c68c6f7557..ec804a21c753 100644
--- a/arch/arm64/Kconfig.debug
+++ b/arch/arm64/Kconfig.debug
@@ -39,6 +39,16 @@ config UNWINDER_FRAME_POINTER
unwinder, but the kernel text size will grow by ~3% and the kernel's
overall performance will degrade by roughly 5-10%.
+config UNWINDER_ORC
+ bool "ORC unwinder"
+ select OBJTOOL
+ help
+ This option enables the ORC (Oops Rewind Capability) unwinder for
+ unwinding kernel stack traces. It uses a custom data format which is
+ a simplified version of the DWARF Call Frame Information standard.
+
+ The orc unwinder is not implemented on arm64 now, this option is only
+ used for testing orc data generation.
endchoice
source "drivers/hwtracing/coresight/Kconfig"
diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/module.h
index 4e7fa2623896..782ac8e120dd 100644
--- a/arch/arm64/include/asm/module.h
+++ b/arch/arm64/include/asm/module.h
@@ -6,6 +6,7 @@
#define __ASM_MODULE_H
#include <asm-generic/module.h>
+#include <asm/orc_types.h>
#ifdef CONFIG_ARM64_MODULE_PLTS
struct mod_plt_sec {
@@ -20,6 +21,12 @@ struct mod_arch_specific {
/* for CONFIG_DYNAMIC_FTRACE */
struct plt_entry *ftrace_trampolines;
+
+#ifdef CONFIG_UNWINDER_ORC
+ unsigned int num_orcs;
+ int *orc_unwind_ip;
+ struct orc_entry *orc_unwind;
+#endif
};
#endif
diff --git a/arch/arm64/include/asm/orc_types.h b/arch/arm64/include/asm/orc_types.h
new file mode 100644
index 000000000000..9c06e7a6ed55
--- /dev/null
+++ b/arch/arm64/include/asm/orc_types.h
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2017 Josh Poimboeuf <[email protected]>
+ */
+
+#ifndef _ORC_TYPES_H
+#define _ORC_TYPES_H
+
+#include <linux/types.h>
+#include <linux/compiler.h>
+
+/*
+ * The ORC_REG_* registers are base registers which are used to find other
+ * registers on the stack.
+ *
+ * ORC_REG_PREV_SP, also known as DWARF Call Frame Address (CFA), is the
+ * address of the previous frame: the caller's SP before it called the current
+ * function.
+ *
+ * ORC_REG_UNDEFINED means the corresponding register's value didn't change in
+ * the current frame.
+ *
+ * The most commonly used base registers are SP and BP -- which the previous SP
+ * is usually based on -- and PREV_SP and UNDEFINED -- which the previous BP is
+ * usually based on.
+ *
+ * The rest of the base registers are needed for special cases like entry code
+ * and GCC realigned stacks.
+ */
+#define ORC_REG_UNDEFINED 0
+#define ORC_REG_PREV_SP 1
+#define ORC_REG_BP 2
+#define ORC_REG_SP 3
+#define ORC_REG_BP_INDIRECT 4
+#define ORC_REG_SP_INDIRECT 5
+#define ORC_REG_MAX 6
+
+#ifndef __ASSEMBLY__
+#include <asm/byteorder.h>
+
+/*
+ * This struct is more or less a vastly simplified version of the DWARF Call
+ * Frame Information standard. It contains only the necessary parts of DWARF
+ * CFI, simplified for ease of access by the in-kernel unwinder. It tells the
+ * unwinder how to find the previous SP and BP (and sometimes entry regs) on
+ * the stack for a given code address. Each instance of the struct corresponds
+ * to one or more code locations.
+ */
+struct orc_entry {
+ s16 sp_offset;
+ s16 bp_offset;
+#if defined(__LITTLE_ENDIAN_BITFIELD)
+ unsigned sp_reg:4;
+ unsigned bp_reg:4;
+ unsigned type:2;
+ unsigned end:1;
+#elif defined(__BIG_ENDIAN_BITFIELD)
+ unsigned bp_reg:4;
+ unsigned sp_reg:4;
+ unsigned unused:5;
+ unsigned end:1;
+ unsigned type:2;
+#endif
+} __packed;
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ORC_TYPES_H */
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index edaf0faf766f..339cf3bf5ce2 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -61,6 +61,7 @@
#define RUNTIME_DISCARD_EXIT
#include <asm-generic/vmlinux.lds.h>
+#include <asm-generic/orc_lookup.h>
#include <asm/cache.h>
#include <asm/kernel-pgtable.h>
#include <asm/kexec.h>
@@ -306,6 +307,8 @@ SECTIONS
__pecoff_data_size = ABSOLUTE(. - __initdata_begin);
_end = .;
+ ORC_UNWIND_TABLE
+
STABS_DEBUG
DWARF_DEBUG
ELF_DETAILS
diff --git a/scripts/Makefile b/scripts/Makefile
index ce5aa9030b74..5be2552e4d02 100644
--- a/scripts/Makefile
+++ b/scripts/Makefile
@@ -20,8 +20,12 @@ HOSTLDLIBS_sign-file = $(shell pkg-config --libs libcrypto 2> /dev/null || echo
ifdef CONFIG_UNWINDER_ORC
ifeq ($(ARCH),x86_64)
ARCH := x86
-endif
HOSTCFLAGS_sorttable.o += -I$(srctree)/tools/arch/x86/include
+endif
+ifeq ($(ARCH),arm64)
+HOSTCFLAGS_sorttable.o += -I$(srctree)/tools/arch/arm64/include
+endif
+
HOSTCFLAGS_sorttable.o += -DUNWINDER_ORC_ENABLED
endif
diff --git a/tools/arch/arm64/include/asm/orc_types.h b/tools/arch/arm64/include/asm/orc_types.h
new file mode 100644
index 000000000000..9c06e7a6ed55
--- /dev/null
+++ b/tools/arch/arm64/include/asm/orc_types.h
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2017 Josh Poimboeuf <[email protected]>
+ */
+
+#ifndef _ORC_TYPES_H
+#define _ORC_TYPES_H
+
+#include <linux/types.h>
+#include <linux/compiler.h>
+
+/*
+ * The ORC_REG_* registers are base registers which are used to find other
+ * registers on the stack.
+ *
+ * ORC_REG_PREV_SP, also known as DWARF Call Frame Address (CFA), is the
+ * address of the previous frame: the caller's SP before it called the current
+ * function.
+ *
+ * ORC_REG_UNDEFINED means the corresponding register's value didn't change in
+ * the current frame.
+ *
+ * The most commonly used base registers are SP and BP -- which the previous SP
+ * is usually based on -- and PREV_SP and UNDEFINED -- which the previous BP is
+ * usually based on.
+ *
+ * The rest of the base registers are needed for special cases like entry code
+ * and GCC realigned stacks.
+ */
+#define ORC_REG_UNDEFINED 0
+#define ORC_REG_PREV_SP 1
+#define ORC_REG_BP 2
+#define ORC_REG_SP 3
+#define ORC_REG_BP_INDIRECT 4
+#define ORC_REG_SP_INDIRECT 5
+#define ORC_REG_MAX 6
+
+#ifndef __ASSEMBLY__
+#include <asm/byteorder.h>
+
+/*
+ * This struct is more or less a vastly simplified version of the DWARF Call
+ * Frame Information standard. It contains only the necessary parts of DWARF
+ * CFI, simplified for ease of access by the in-kernel unwinder. It tells the
+ * unwinder how to find the previous SP and BP (and sometimes entry regs) on
+ * the stack for a given code address. Each instance of the struct corresponds
+ * to one or more code locations.
+ */
+struct orc_entry {
+ s16 sp_offset;
+ s16 bp_offset;
+#if defined(__LITTLE_ENDIAN_BITFIELD)
+ unsigned sp_reg:4;
+ unsigned bp_reg:4;
+ unsigned type:2;
+ unsigned end:1;
+#elif defined(__BIG_ENDIAN_BITFIELD)
+ unsigned bp_reg:4;
+ unsigned sp_reg:4;
+ unsigned unused:5;
+ unsigned end:1;
+ unsigned type:2;
+#endif
+} __packed;
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ORC_TYPES_H */
diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile
index 3f7c7b54c741..e17c1fd90982 100644
--- a/tools/objtool/Makefile
+++ b/tools/objtool/Makefile
@@ -46,6 +46,7 @@ ifeq ($(SRCARCH),x86)
endif
ifeq ($(SRCARCH),arm64)
+ BUILD_ORC := y
CFLAGS += -Wno-nested-externs
endif
diff --git a/tools/objtool/arch/arm64/Build b/tools/objtool/arch/arm64/Build
index f3de3a50d541..00221087eefe 100644
--- a/tools/objtool/arch/arm64/Build
+++ b/tools/objtool/arch/arm64/Build
@@ -1,5 +1,6 @@
objtool-y += special.o
objtool-y += decode.o
+objtool-y += orc.o
objtool-y += libhweight.o
diff --git a/tools/objtool/arch/arm64/orc.c b/tools/objtool/arch/arm64/orc.c
new file mode 100644
index 000000000000..aa8404c482b6
--- /dev/null
+++ b/tools/objtool/arch/arm64/orc.c
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2017 Josh Poimboeuf <[email protected]>
+ */
+
+#include <stdlib.h>
+
+#include <linux/objtool.h>
+
+#include <objtool/orc.h>
+#include <objtool/warn.h>
+
+int init_orc_entry(struct orc_entry *orc, struct cfi_state *cfi,
+ struct instruction *insn)
+{
+ struct cfi_reg *bp = &cfi->regs[CFI_BP];
+
+ memset(orc, 0, sizeof(*orc));
+
+ if (!cfi) {
+ orc->end = 0;
+ orc->sp_reg = ORC_REG_UNDEFINED;
+ return 0;
+ }
+
+ orc->end = cfi->end;
+
+ if (cfi->cfa.base == CFI_UNDEFINED) {
+ orc->sp_reg = ORC_REG_UNDEFINED;
+ return 0;
+ }
+
+ switch (cfi->cfa.base) {
+ case CFI_SP:
+ orc->sp_reg = ORC_REG_SP;
+ break;
+ case CFI_SP_INDIRECT:
+ orc->sp_reg = ORC_REG_SP_INDIRECT;
+ break;
+ case CFI_BP:
+ orc->sp_reg = ORC_REG_BP;
+ break;
+ case CFI_BP_INDIRECT:
+ orc->sp_reg = ORC_REG_BP_INDIRECT;
+ break;
+ default:
+ WARN_FUNC("unknown CFA base reg %d",
+ insn->sec, insn->offset, cfi->cfa.base);
+ return -1;
+ }
+
+ switch (bp->base) {
+ case CFI_UNDEFINED:
+ orc->bp_reg = ORC_REG_UNDEFINED;
+ break;
+ case CFI_CFA:
+ orc->bp_reg = ORC_REG_PREV_SP;
+ break;
+ case CFI_BP:
+ orc->bp_reg = ORC_REG_BP;
+ break;
+ default:
+ WARN_FUNC("unknown BP base reg %d",
+ insn->sec, insn->offset, bp->base);
+ return -1;
+ }
+
+ orc->sp_offset = cfi->cfa.offset;
+ orc->bp_offset = bp->offset;
+ orc->type = cfi->type;
+
+ return 0;
+}
+
+static const char *reg_name(unsigned int reg)
+{
+ switch (reg) {
+ case ORC_REG_PREV_SP:
+ return "prevsp";
+ case ORC_REG_BP:
+ return "fp";
+ case ORC_REG_SP:
+ return "sp";
+ case ORC_REG_BP_INDIRECT:
+ return "fp(ind)";
+ case ORC_REG_SP_INDIRECT:
+ return "sp(ind)";
+ default:
+ return "?";
+ }
+}
+
+const char *orc_type_name(unsigned int type)
+{
+ switch (type) {
+ case UNWIND_HINT_TYPE_CALL:
+ return "call";
+ case UNWIND_HINT_TYPE_REGS:
+ return "regs";
+ case UNWIND_HINT_TYPE_REGS_PARTIAL:
+ return "regs (partial)";
+ default:
+ return "?";
+ }
+}
+
+void orc_print_reg(unsigned int reg, int offset)
+{
+ if (reg == ORC_REG_BP_INDIRECT)
+ printf("(fp%+d)", offset);
+ else if (reg == ORC_REG_SP_INDIRECT)
+ printf("(sp%+d)", offset);
+ else if (reg == ORC_REG_UNDEFINED)
+ printf("(und)");
+ else
+ printf("%s%+d", reg_name(reg), offset);
+}
--
2.17.1
Provide needed definitions for a new architecture instruction decoder.
No proper decoding is done yet.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/objtool/Makefile | 4 +
tools/objtool/arch/arm64/Build | 8 +
tools/objtool/arch/arm64/decode.c | 138 ++++++++++++++++++
.../arch/arm64/include/arch/cfi_regs.h | 14 ++
tools/objtool/arch/arm64/include/arch/elf.h | 12 ++
.../arch/arm64/include/arch/endianness.h | 9 ++
.../objtool/arch/arm64/include/arch/special.h | 22 +++
tools/objtool/arch/arm64/special.c | 21 +++
8 files changed, 228 insertions(+)
create mode 100644 tools/objtool/arch/arm64/Build
create mode 100644 tools/objtool/arch/arm64/decode.c
create mode 100644 tools/objtool/arch/arm64/include/arch/cfi_regs.h
create mode 100644 tools/objtool/arch/arm64/include/arch/elf.h
create mode 100644 tools/objtool/arch/arm64/include/arch/endianness.h
create mode 100644 tools/objtool/arch/arm64/include/arch/special.h
create mode 100644 tools/objtool/arch/arm64/special.c
diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile
index 2b97720ab608..3f7c7b54c741 100644
--- a/tools/objtool/Makefile
+++ b/tools/objtool/Makefile
@@ -45,6 +45,10 @@ ifeq ($(SRCARCH),x86)
BUILD_ORC := y
endif
+ifeq ($(SRCARCH),arm64)
+ CFLAGS += -Wno-nested-externs
+endif
+
export BUILD_ORC
export srctree OUTPUT CFLAGS SRCARCH AWK
include $(srctree)/tools/build/Makefile.include
diff --git a/tools/objtool/arch/arm64/Build b/tools/objtool/arch/arm64/Build
new file mode 100644
index 000000000000..f3de3a50d541
--- /dev/null
+++ b/tools/objtool/arch/arm64/Build
@@ -0,0 +1,8 @@
+objtool-y += special.o
+objtool-y += decode.o
+
+objtool-y += libhweight.o
+
+$(OUTPUT)arch/arm64/libhweight.o: ../lib/hweight.c FORCE
+ $(call rule_mkdir)
+ $(call if_changed_dep,cc_o_c)
diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/decode.c
new file mode 100644
index 000000000000..afe22c4593c8
--- /dev/null
+++ b/tools/objtool/arch/arm64/decode.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+
+#include <asm/insn.h>
+
+#include <objtool/check.h>
+#include <objtool/arch.h>
+#include <objtool/elf.h>
+#include <objtool/warn.h>
+#include <objtool/builtin.h>
+#include <arch/cfi_regs.h>
+
+#include "../../../arch/arm64/lib/insn.c"
+
+bool arch_callee_saved_reg(unsigned char reg)
+{
+ switch (reg) {
+ case AARCH64_INSN_REG_19:
+ case AARCH64_INSN_REG_20:
+ case AARCH64_INSN_REG_21:
+ case AARCH64_INSN_REG_22:
+ case AARCH64_INSN_REG_23:
+ case AARCH64_INSN_REG_24:
+ case AARCH64_INSN_REG_25:
+ case AARCH64_INSN_REG_26:
+ case AARCH64_INSN_REG_27:
+ case AARCH64_INSN_REG_28:
+ case AARCH64_INSN_REG_FP:
+ case AARCH64_INSN_REG_LR:
+ return true;
+ default:
+ return false;
+ }
+}
+
+void arch_initial_func_cfi_state(struct cfi_init_state *state)
+{
+ int i;
+
+ for (i = 0; i < CFI_NUM_REGS; i++) {
+ state->regs[i].base = CFI_UNDEFINED;
+ state->regs[i].offset = 0;
+ }
+
+ /* initial CFA (call frame address) */
+ state->cfa.base = CFI_SP;
+ state->cfa.offset = 0;
+}
+
+unsigned long arch_dest_reloc_offset(int addend)
+{
+ return addend;
+}
+
+unsigned long arch_jump_destination(struct instruction *insn)
+{
+ return insn->offset + insn->immediate;
+}
+
+const char *arch_nop_insn(int len)
+{
+ static u32 nop;
+
+ if (len != AARCH64_INSN_SIZE)
+ WARN("invalid NOP size: %d\n", len);
+
+ if (!nop)
+ nop = aarch64_insn_gen_nop();
+
+ return (const char *)&nop;
+}
+
+const char *arch_ret_insn(int len)
+{
+ static u32 ret;
+
+ if (len != AARCH64_INSN_SIZE)
+ WARN("invalid RET size: %d\n", len);
+
+ if (!ret) {
+ ret = aarch64_insn_gen_branch_reg(AARCH64_INSN_REG_LR,
+ AARCH64_INSN_BRANCH_RETURN);
+ }
+
+ return (const char *)&ret;
+}
+
+static int is_arm64(const struct elf *elf)
+{
+ switch (elf->ehdr.e_machine) {
+ case EM_AARCH64: //0xB7
+ return 1;
+ default:
+ WARN("unexpected ELF machine type %x",
+ elf->ehdr.e_machine);
+ return 0;
+ }
+}
+
+int arch_decode_hint_reg(u8 sp_reg, int *base)
+{
+ return -1;
+}
+
+int arch_decode_instruction(struct objtool_file *file, const struct section *sec,
+ unsigned long offset, unsigned int maxlen,
+ unsigned int *len, enum insn_type *type,
+ unsigned long *immediate,
+ struct list_head *ops_list)
+{
+ const struct elf *elf = file->elf;
+ u32 insn;
+
+ if (!is_arm64(elf))
+ return -1;
+
+ if (maxlen < AARCH64_INSN_SIZE)
+ return 0;
+
+ *len = AARCH64_INSN_SIZE;
+ *immediate = 0;
+ *type = INSN_OTHER;
+
+ insn = *(u32 *)(sec->data->d_buf + offset);
+
+ switch (aarch64_get_insn_class(insn)) {
+ case AARCH64_INSN_CLS_UNKNOWN:
+ WARN("can't decode instruction at %s:0x%lx", sec->name, offset);
+ return -1;
+ default:
+ break;
+ }
+
+ return 0;
+}
diff --git a/tools/objtool/arch/arm64/include/arch/cfi_regs.h b/tools/objtool/arch/arm64/include/arch/cfi_regs.h
new file mode 100644
index 000000000000..a5185649686b
--- /dev/null
+++ b/tools/objtool/arch/arm64/include/arch/cfi_regs.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef _OBJTOOL_CFI_REGS_H
+#define _OBJTOOL_CFI_REGS_H
+
+#include <asm/insn.h>
+
+#define CFI_BP AARCH64_INSN_REG_FP
+#define CFI_RA AARCH64_INSN_REG_LR
+#define CFI_SP AARCH64_INSN_REG_SP
+
+#define CFI_NUM_REGS 32
+
+#endif /* _OBJTOOL_CFI_REGS_H */
diff --git a/tools/objtool/arch/arm64/include/arch/elf.h b/tools/objtool/arch/arm64/include/arch/elf.h
new file mode 100644
index 000000000000..a59888a906b5
--- /dev/null
+++ b/tools/objtool/arch/arm64/include/arch/elf.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 */
+
+#ifndef _OBJTOOL_ARCH_ELF
+#define _OBJTOOL_ARCH_ELF
+
+#define R_NTYPE -1
+#define R_NONE R_AARCH64_NONE
+#define R_ABS64 R_AARCH64_ABS64
+#define R_REL32 R_AARCH64_PREL32
+#define R_PLT32 R_NTYPE
+
+#endif /* _OBJTOOL_ARCH_ELF */
diff --git a/tools/objtool/arch/arm64/include/arch/endianness.h b/tools/objtool/arch/arm64/include/arch/endianness.h
new file mode 100644
index 000000000000..7c362527da20
--- /dev/null
+++ b/tools/objtool/arch/arm64/include/arch/endianness.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef _ARCH_ENDIANNESS_H
+#define _ARCH_ENDIANNESS_H
+
+#include <endian.h>
+
+#define __TARGET_BYTE_ORDER __LITTLE_ENDIAN
+
+#endif /* _ARCH_ENDIANNESS_H */
diff --git a/tools/objtool/arch/arm64/include/arch/special.h b/tools/objtool/arch/arm64/include/arch/special.h
new file mode 100644
index 000000000000..63a705c622a4
--- /dev/null
+++ b/tools/objtool/arch/arm64/include/arch/special.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef _ARM64_ARCH_SPECIAL_H
+#define _ARM64_ARCH_SPECIAL_H
+
+#define EX_ENTRY_SIZE 12
+#define EX_ORIG_OFFSET 0
+#define EX_NEW_OFFSET 4
+
+#define JUMP_ENTRY_SIZE 16
+#define JUMP_ORIG_OFFSET 0
+#define JUMP_NEW_OFFSET 4
+#define JUMP_KEY_OFFSET 8
+
+#define ALT_ENTRY_SIZE 12
+#define ALT_ORIG_OFFSET 0
+#define ALT_NEW_OFFSET 4
+#define ALT_FEATURE_OFFSET 8
+#define ALT_ORIG_LEN_OFFSET 10
+#define ALT_NEW_LEN_OFFSET 11
+
+#endif /* _ARM64_ARCH_SPECIAL_H */
diff --git a/tools/objtool/arch/arm64/special.c b/tools/objtool/arch/arm64/special.c
new file mode 100644
index 000000000000..45f283283091
--- /dev/null
+++ b/tools/objtool/arch/arm64/special.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+#include <objtool/special.h>
+
+void arch_handle_alternative(unsigned short feature, struct special_alt *alt)
+{
+}
+
+bool arch_support_alt_relocation(struct special_alt *special_alt,
+ struct instruction *insn,
+ struct reloc *reloc)
+{
+ return false;
+}
+
+
+struct reloc *arch_find_switch_table(struct objtool_file *file,
+ struct instruction *insn)
+{
+ return NULL;
+}
--
2.17.1
Label __idmap_kpti_flag represents the location of a constant.
Mark it as data symbol.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/mm/proc.S | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 50bbed947bec..c34ea1778b1e 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -222,8 +222,8 @@ SYM_FUNC_END(idmap_cpu_replace_ttbr1)
*
* Called exactly once from stop_machine context by each CPU found during boot.
*/
-__idmap_kpti_flag:
- .long 1
+SYM_DATA_LOCAL(__idmap_kpti_flag, .long 1)
+
SYM_FUNC_START(idmap_kpti_install_ng_mappings)
cpu .req w0
num_cpus .req w1
--
2.17.1
The way sha256_block_neon restore the stackframe confuses objtool.
But it turns out this function is a leaf function and does not use
FP nor LR as scratch register.
Do not create a stackframe in this function as it is not necessary.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/crypto/sha512-armv8.pl | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/arch/arm64/crypto/sha512-armv8.pl b/arch/arm64/crypto/sha512-armv8.pl
index 1882c4110026..6e2a96e05c5a 100644
--- a/arch/arm64/crypto/sha512-armv8.pl
+++ b/arch/arm64/crypto/sha512-armv8.pl
@@ -648,8 +648,6 @@ $code.=<<___;
.align 4
sha256_block_neon:
.Lneon_entry:
- stp x29, x30, [sp, #-16]!
- mov x29, sp
sub sp,sp,#16*4
adr $Ktbl,K256
@@ -736,8 +734,7 @@ $code.=<<___;
mov $Xfer,sp
b.ne .L_00_48
- ldr x29,[x29]
- add sp,sp,#16*4+16
+ add sp,sp,#16*4
ret
.size sha256_block_neon,.-sha256_block_neon
___
--
2.17.1
Add data annotations to constants part of the image header.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/kernel/head.S | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 2a66d18091eb..71580eb1fd51 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -63,10 +63,11 @@
/*
* DO NOT MODIFY. Image header expected by Linux boot-loaders.
*/
- efi_signature_nop // special NOP to identity as PE/COFF executable
+SYM_DATA_LOCAL(efi_nop, efi_signature_nop) // special NOP to identity as PE/COFF executable
UNWIND_HINT_EMPTY
b primary_entry // branch to kernel start, magic
- .quad 0 // Image load offset from start of RAM, little-endian
+SYM_DATA_LOCAL(_zero_reserved, .quad 0) // Image load offset from start of RAM, little-endian
+SYM_DATA_START_LOCAL(_arm64_common_header)
le64sym _kernel_size_le // Effective size of kernel image, little-endian
le64sym _kernel_flags_le // Informative flags, little-endian
.quad 0 // reserved
@@ -74,6 +75,7 @@
.quad 0 // reserved
.ascii ARM64_IMAGE_MAGIC // Magic number
.long .Lpe_header_offset // Offset to the PE header.
+SYM_DATA_END(_arm64_common_header)
__EFI_PE_HEADER
--
2.17.1
Symbol __guest_enter and kvm_hyp_vector saved x0, x1 on stack.
Symbol __guest_exit and __guest_exit_panic is reached when resuming
EL2 execution, and the previous stack pointer gets restored.
Add adequate unwind hints.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/kvm/hyp/entry.S | 5 +++++
arch/arm64/kvm/hyp/hyp-entry.S | 3 +++
2 files changed, 8 insertions(+)
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index 045d4481c820..81e30d56bf28 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -15,6 +15,7 @@
#include <asm/kvm_mmu.h>
#include <asm/kvm_mte.h>
#include <asm/kvm_ptrauth.h>
+#include <asm/unwind_hints.h>
.text
@@ -22,6 +23,7 @@
* u64 __guest_enter(struct kvm_vcpu *vcpu);
*/
SYM_CODE_START(__guest_enter)
+ UNWIND_HINT_FUNC
// x0: vcpu
// x1-x17: clobbered by macros
// x29: guest context
@@ -88,6 +90,7 @@ SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL)
// vcpu x0-x1 on the stack
// If the hyp context is loaded, go straight to hyp_panic
+ UNWIND_HINT_FUNC
get_loaded_vcpu x0, x1
cbnz x0, 1f
b hyp_panic
@@ -110,6 +113,7 @@ SYM_INNER_LABEL(__guest_exit, SYM_L_GLOBAL)
// x1: vcpu
// x2-x29,lr: vcpu regs
// vcpu x0-x1 on the stack
+ UNWIND_HINT_FUNC sp_offset=16
add x1, x1, #VCPU_CONTEXT
@@ -199,6 +203,7 @@ abort_guest_exit_end:
msr daifset, #4 // Mask aborts
ret
+ UNWIND_HINT_FUNC
_kvm_extable abort_guest_exit_start, 9997f
_kvm_extable abort_guest_exit_end, 9997f
9997:
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index 4a65262a4f3a..d72143c59707 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -151,6 +151,7 @@ SYM_CODE_END(\label)
.macro valid_vect target
.align 7
+ UNWIND_HINT_FUNC
661:
esb
stp x0, x1, [sp, #-16]!
@@ -162,6 +163,7 @@ check_preamble_length 661b, 662b
.macro invalid_vect target
.align 7
+ UNWIND_HINT_FUNC
661:
nop
stp x0, x1, [sp, #-16]!
@@ -209,6 +211,7 @@ SYM_CODE_END(__kvm_hyp_vector)
.macro hyp_ventry indirect, spectrev2
.align 7
1: esb
+ UNWIND_HINT_FUNC
.if \spectrev2 != 0
spectrev2_smccc_wa1_smc
.else
--
2.17.1
Some assembly symbols contain code that might be executed with an
unspecified stack state (e.g. invalid stack pointer,
no stackframe, code after alt_cb, ...).
Annotate those symbol with UNWIND_HINT_EMPTY to let objtool be aware of
them.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/include/asm/assembler.h | 2 ++
arch/arm64/kernel/cpu-reset.S | 2 ++
arch/arm64/kernel/efi-entry.S | 2 ++
arch/arm64/kernel/entry.S | 7 +++++++
arch/arm64/kernel/head.S | 14 ++++++++++++++
arch/arm64/kernel/hibernate-asm.S | 2 ++
arch/arm64/kernel/relocate_kernel.S | 2 ++
arch/arm64/kernel/sleep.S | 3 +++
arch/arm64/kvm/hyp/hyp-entry.S | 1 +
arch/arm64/mm/trans_pgd-asm.S | 3 +++
10 files changed, 38 insertions(+)
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index 8c5a61aeaf8e..68db05428e4b 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -25,6 +25,7 @@
#include <asm/pgtable-hwdef.h>
#include <asm/ptrace.h>
#include <asm/thread_info.h>
+#include <asm/unwind_hints.h>
/*
* Provide a wxN alias for each wN register so what we can paste a xN
@@ -147,6 +148,7 @@ lr .req x30 // link register
*/
.macro ventry label
.align 7
+ UNWIND_HINT_EMPTY
b \label
.endm
diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S
index 48a8af97faa9..c9022042bdec 100644
--- a/arch/arm64/kernel/cpu-reset.S
+++ b/arch/arm64/kernel/cpu-reset.S
@@ -10,6 +10,7 @@
#include <linux/linkage.h>
#include <asm/assembler.h>
#include <asm/sysreg.h>
+#include <asm/unwind_hints.h>
#include <asm/virt.h>
.text
@@ -29,6 +30,7 @@
* flat identity mapping.
*/
SYM_CODE_START(cpu_soft_restart)
+ UNWIND_HINT_EMPTY
mov_q x12, INIT_SCTLR_EL1_MMU_OFF
pre_disable_mmu_workaround
/*
diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S
index 61a87fa1c305..9a1a94c3c4db 100644
--- a/arch/arm64/kernel/efi-entry.S
+++ b/arch/arm64/kernel/efi-entry.S
@@ -9,10 +9,12 @@
#include <linux/init.h>
#include <asm/assembler.h>
+#include <asm/unwind_hints.h>
__INIT
SYM_CODE_START(efi_enter_kernel)
+ UNWIND_HINT_EMPTY
/*
* efi_pe_entry() will have copied the kernel image if necessary and we
* end up here with device tree address in x1 and the kernel entry
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index c460ba2d009d..3bd11101e79d 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -28,6 +28,7 @@
#include <asm/thread_info.h>
#include <asm/asm-uaccess.h>
#include <asm/unistd.h>
+#include <asm/unwind_hints.h>
.macro clear_gp_regs
.irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29
@@ -37,6 +38,7 @@
.macro kernel_ventry, el:req, ht:req, regsize:req, label:req
.align 7
+ UNWIND_HINT_EMPTY
.Lventry_start\@:
.if \el == 0
/*
@@ -44,6 +46,7 @@
* skipped by the trampoline vectors, to trigger the cleanup.
*/
b .Lskip_tramp_vectors_cleanup\@
+ UNWIND_HINT_EMPTY
.if \regsize == 64
mrs x30, tpidrro_el0
msr tpidrro_el0, xzr
@@ -417,6 +420,7 @@ alternative_else_nop_endif
ldp x24, x25, [sp, #16 * 12]
ldp x26, x27, [sp, #16 * 13]
ldp x28, x29, [sp, #16 * 14]
+ UNWIND_HINT_EMPTY
.if \el == 0
alternative_if_not ARM64_UNMAP_KERNEL_AT_EL0
@@ -662,6 +666,7 @@ alternative_else_nop_endif
.macro tramp_ventry, vector_start, regsize, kpti, bhb
.align 7
+ UNWIND_HINT_EMPTY
1:
.if \regsize == 64
msr tpidrro_el0, x30 // Restored in kernel_ventry
@@ -687,6 +692,7 @@ alternative_else_nop_endif
* enter the full-fat kernel vectors.
*/
bl 2f
+ UNWIND_HINT_EMPTY
b .
2:
tramp_map_kernel x30
@@ -717,6 +723,7 @@ alternative_else_nop_endif
.endm
.macro tramp_exit, regsize = 64
+ UNWIND_HINT_EMPTY
tramp_data_read_var x30, this_cpu_vector
get_this_cpu_offset x29
ldr x30, [x30, x29]
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 6db9c3603bd8..2a66d18091eb 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -33,6 +33,7 @@
#include <asm/smp.h>
#include <asm/sysreg.h>
#include <asm/thread_info.h>
+#include <asm/unwind_hints.h>
#include <asm/virt.h>
#include "efi-header.S"
@@ -63,6 +64,7 @@
* DO NOT MODIFY. Image header expected by Linux boot-loaders.
*/
efi_signature_nop // special NOP to identity as PE/COFF executable
+ UNWIND_HINT_EMPTY
b primary_entry // branch to kernel start, magic
.quad 0 // Image load offset from start of RAM, little-endian
le64sym _kernel_size_le // Effective size of kernel image, little-endian
@@ -109,6 +111,7 @@ SYM_CODE_END(primary_entry)
* Preserve the arguments passed by the bootloader in x0 .. x3
*/
SYM_CODE_START_LOCAL(preserve_boot_args)
+ UNWIND_HINT_EMPTY
mov x21, x0 // x21=FDT
adr_l x0, boot_args // record the contents of
@@ -260,6 +263,7 @@ SYM_CODE_END(preserve_boot_args)
* been enabled
*/
SYM_CODE_START_LOCAL(__create_page_tables)
+ UNWIND_HINT_EMPTY
mov x28, lr
/*
@@ -494,6 +498,7 @@ EXPORT_SYMBOL(kimage_vaddr)
* booted in EL1 or EL2 respectively.
*/
SYM_CODE_START(init_kernel_el)
+ UNWIND_HINT_EMPTY
mrs x0, CurrentEL
cmp x0, #CurrentEL_EL2
b.eq init_el2
@@ -553,6 +558,7 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL)
eret
__cpu_stick_to_vhe:
+ UNWIND_HINT_EMPTY
mov x0, #HVC_VHE_RESTART
hvc #0
mov x0, #BOOT_CPU_MODE_EL2
@@ -564,6 +570,7 @@ SYM_CODE_END(init_kernel_el)
* in w0. See arch/arm64/include/asm/virt.h for more info.
*/
SYM_CODE_START_LOCAL(set_cpu_boot_mode_flag)
+ UNWIND_HINT_EMPTY
adr_l x1, __boot_cpu_mode
cmp w0, #BOOT_CPU_MODE_EL2
b.ne 1f
@@ -607,6 +614,7 @@ SYM_DATA_END(__early_cpu_boot_status)
* cores are held until we're ready for them to initialise.
*/
SYM_CODE_START(secondary_holding_pen)
+ UNWIND_HINT_EMPTY
bl init_kernel_el // w0=cpu_boot_mode
bl set_cpu_boot_mode_flag
mrs x0, mpidr_el1
@@ -625,6 +633,7 @@ SYM_CODE_END(secondary_holding_pen)
* be used where CPUs are brought online dynamically by the kernel.
*/
SYM_CODE_START(secondary_entry)
+ UNWIND_HINT_EMPTY
bl init_kernel_el // w0=cpu_boot_mode
bl set_cpu_boot_mode_flag
b secondary_startup
@@ -644,6 +653,7 @@ SYM_CODE_START_LOCAL(secondary_startup)
SYM_CODE_END(secondary_startup)
SYM_CODE_START_LOCAL(__secondary_switched)
+ UNWIND_HINT_EMPTY
adr_l x5, vectors
msr vbar_el1, x5
isb
@@ -663,6 +673,7 @@ SYM_CODE_START_LOCAL(__secondary_switched)
SYM_CODE_END(__secondary_switched)
SYM_CODE_START_LOCAL(__secondary_too_slow)
+ UNWIND_HINT_EMPTY
wfe
wfi
b __secondary_too_slow
@@ -699,6 +710,7 @@ SYM_CODE_END(__secondary_too_slow)
* If it isn't, park the CPU
*/
SYM_CODE_START(__enable_mmu)
+ UNWIND_HINT_EMPTY
mrs x2, ID_AA64MMFR0_EL1
ubfx x2, x2, #ID_AA64MMFR0_TGRAN_SHIFT, 4
cmp x2, #ID_AA64MMFR0_TGRAN_SUPPORTED_MIN
@@ -720,6 +732,7 @@ SYM_CODE_START(__enable_mmu)
SYM_CODE_END(__enable_mmu)
SYM_CODE_START_LOCAL(__cpu_secondary_check52bitva)
+ UNWIND_HINT_EMPTY
#ifdef CONFIG_ARM64_VA_BITS_52
ldr_l x0, vabits_actual
cmp x0, #52
@@ -751,6 +764,7 @@ SYM_CODE_END(__no_granule_support)
#ifdef CONFIG_RELOCATABLE
SYM_CODE_START_LOCAL(__relocate_kernel)
+ UNWIND_HINT_EMPTY
/*
* Iterate over each entry in the relocation table, and apply the
* relocations in place.
diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S
index 0e1d9c3c6a93..c0bec20bf0e0 100644
--- a/arch/arm64/kernel/hibernate-asm.S
+++ b/arch/arm64/kernel/hibernate-asm.S
@@ -13,6 +13,7 @@
#include <asm/cputype.h>
#include <asm/memory.h>
#include <asm/page.h>
+#include <asm/unwind_hints.h>
#include <asm/virt.h>
/*
@@ -46,6 +47,7 @@
*/
.pushsection ".hibernate_exit.text", "ax"
SYM_CODE_START(swsusp_arch_suspend_exit)
+ UNWIND_HINT_EMPTY
/*
* We execute from ttbr0, change ttbr1 to our copied linear map tables
* with a break-before-make via the zero page
diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S
index f0a3df9e18a3..f8cd8fcf2d4f 100644
--- a/arch/arm64/kernel/relocate_kernel.S
+++ b/arch/arm64/kernel/relocate_kernel.S
@@ -16,6 +16,7 @@
#include <asm/page.h>
#include <asm/sysreg.h>
#include <asm/virt.h>
+#include <asm/unwind_hints.h>
.macro turn_off_mmu tmp1, tmp2
mov_q \tmp1, INIT_SCTLR_EL1_MMU_OFF
@@ -37,6 +38,7 @@
* safe memory that has been set up to be preserved during the copy operation.
*/
SYM_CODE_START(arm64_relocate_new_kernel)
+ UNWIND_HINT_EMPTY
/* Setup the list loop variables. */
ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */
ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */
diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S
index f0087e8bcd28..799ec01b0649 100644
--- a/arch/arm64/kernel/sleep.S
+++ b/arch/arm64/kernel/sleep.S
@@ -4,6 +4,7 @@
#include <asm/asm-offsets.h>
#include <asm/assembler.h>
#include <asm/smp.h>
+#include <asm/unwind_hints.h>
.text
/*
@@ -99,6 +100,7 @@ SYM_FUNC_END(__cpu_suspend_enter)
.pushsection ".idmap.text", "awx"
SYM_CODE_START(cpu_resume)
+ UNWIND_HINT_EMPTY
bl init_kernel_el
bl switch_to_vhe
bl __cpu_setup
@@ -112,6 +114,7 @@ SYM_CODE_END(cpu_resume)
.popsection
SYM_CODE_START(_cpu_resume)
+ UNWIND_HINT_EMPTY
mrs x1, mpidr_el1
adr_l x8, mpidr_hash // x8 = struct mpidr_hash virt address
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index 7839d075729b..4a65262a4f3a 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -14,6 +14,7 @@
#include <asm/kvm_asm.h>
#include <asm/mmu.h>
#include <asm/spectre.h>
+#include <asm/unwind_hints.h>
.macro save_caller_saved_regs_vect
/* x0 and x1 were saved in the vector entry */
diff --git a/arch/arm64/mm/trans_pgd-asm.S b/arch/arm64/mm/trans_pgd-asm.S
index 021c31573bcb..148435248860 100644
--- a/arch/arm64/mm/trans_pgd-asm.S
+++ b/arch/arm64/mm/trans_pgd-asm.S
@@ -8,10 +8,12 @@
#include <linux/linkage.h>
#include <asm/assembler.h>
#include <asm/kvm_asm.h>
+#include <asm/unwind_hints.h>
.macro invalid_vector label
SYM_CODE_START_LOCAL(\label)
.align 7
+ UNWIND_HINT_EMPTY
b \label
SYM_CODE_END(\label)
.endm
@@ -19,6 +21,7 @@ SYM_CODE_END(\label)
.macro el1_sync_vector
SYM_CODE_START_LOCAL(el1_sync)
.align 7
+ UNWIND_HINT_EMPTY
cmp x0, #HVC_SET_VECTORS /* Called from hibernate */
b.ne 1f
msr vbar_el2, x1
--
2.17.1
Decode load/store operations and create corresponding stack_ops for
operations targeting SP or FP.
Operations storing/loading multiple registers are split into separate
stack_ops storing single registers.
Operations modifying the base register get an additional stack_op
for the register update. Since the atomic register(s) load/store + base
register update gets split into multiple operations, to make sure
objtool always sees a valid stack, consider store instruction to perform
stack allocations (i.e. modifying the base pointer before the storing)
and loads de-allocations (i.e. modifying the base pointer after the
load).
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/objtool/arch/arm64/decode.c | 112 ++++++++++++++++++++++++++++++
1 file changed, 112 insertions(+)
diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/decode.c
index 19840862f3aa..8ce9d91ff0db 100644
--- a/tools/objtool/arch/arm64/decode.c
+++ b/tools/objtool/arch/arm64/decode.c
@@ -132,6 +132,114 @@ static inline void make_add_op(enum aarch64_insn_register dest,
op->src.offset = val;
}
+static inline void make_store_op(enum aarch64_insn_register base,
+ enum aarch64_insn_register reg,
+ int offset, struct stack_op *op)
+{
+ op->dest.type = OP_DEST_REG_INDIRECT;
+ op->dest.reg = base;
+ op->dest.offset = offset;
+ op->src.type = OP_SRC_REG;
+ op->src.reg = reg;
+ op->src.offset = 0;
+}
+
+static inline void make_load_op(enum aarch64_insn_register base,
+ enum aarch64_insn_register reg,
+ int offset, struct stack_op *op)
+{
+ op->dest.type = OP_DEST_REG;
+ op->dest.reg = reg;
+ op->dest.offset = 0;
+ op->src.type = OP_SRC_REG_INDIRECT;
+ op->src.reg = base;
+ op->src.offset = offset;
+}
+
+static inline bool aarch64_insn_is_ldst_pre(u32 insn)
+{
+ return aarch64_insn_is_store_pre(insn) ||
+ aarch64_insn_is_load_pre(insn) ||
+ aarch64_insn_is_stp_pre(insn) ||
+ aarch64_insn_is_ldp_pre(insn);
+}
+
+static inline bool aarch64_insn_is_ldst_post(u32 insn)
+{
+ return aarch64_insn_is_store_post(insn) ||
+ aarch64_insn_is_load_post(insn) ||
+ aarch64_insn_is_stp_post(insn) ||
+ aarch64_insn_is_ldp_post(insn);
+}
+
+static int decode_load_store(u32 insn, unsigned long *immediate,
+ struct list_head *ops_list)
+{
+ enum aarch64_insn_register base;
+ enum aarch64_insn_register rt;
+ struct stack_op *op;
+ int size;
+ int offset;
+
+ if (aarch64_insn_is_store_single(insn) ||
+ aarch64_insn_is_load_single(insn))
+ size = 1 << ((insn & GENMASK(31, 30)) >> 30);
+ else
+ size = 4 << ((insn >> 31) & 1);
+
+ if (aarch64_insn_is_store_pair(insn) ||
+ aarch64_insn_is_load_pair(insn))
+ *immediate = size * sign_extend(aarch64_insn_decode_immediate(AARCH64_INSN_IMM_7,
+ insn), 7);
+ else if (aarch64_insn_is_store_imm(insn) ||
+ aarch64_insn_is_load_imm(insn))
+ *immediate = size * aarch64_insn_decode_immediate(AARCH64_INSN_IMM_12, insn);
+ else /* load/store_pre/post */
+ *immediate = sign_extend(aarch64_insn_decode_immediate(AARCH64_INSN_IMM_9,
+ insn), 9);
+
+ base = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RN, insn);
+ if (!is_SPFP(base))
+ return 0;
+
+ if (aarch64_insn_is_ldst_post(insn))
+ offset = 0;
+ else
+ offset = *immediate;
+
+ /* First register */
+ rt = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RT, insn);
+ ADD_OP(op) {
+ if (aarch64_insn_is_store_single(insn) ||
+ aarch64_insn_is_store_pair(insn))
+ make_store_op(base, rt, offset, op);
+ else
+ make_load_op(base, rt, offset, op);
+ }
+
+ /* Second register (if present) */
+ if (aarch64_insn_is_store_pair(insn) ||
+ aarch64_insn_is_load_pair(insn)) {
+ rt = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RT2,
+ insn);
+ ADD_OP(op) {
+ if (aarch64_insn_is_store_pair(insn))
+ make_store_op(base, rt, offset + size, op);
+ else
+ make_load_op(base, rt, offset + size, op);
+ }
+ }
+
+ if (aarch64_insn_is_ldst_pre(insn) ||
+ aarch64_insn_is_ldst_post(insn)) {
+ ADD_OP(op) {
+ make_add_op(base, base, *immediate, op);
+ }
+ }
+
+ return 0;
+}
+
static void decode_add_sub_imm(u32 instr, bool set_flags,
unsigned long *immediate,
struct stack_op *op)
@@ -241,6 +349,10 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
*immediate = aarch64_insn_decode_immediate(AARCH64_INSN_IMM_16, insn);
}
break;
+ case AARCH64_INSN_CLS_LDST:
+ {
+ return decode_load_store(insn, immediate, ops_list);
+ }
default:
break;
}
--
2.17.1
This file only contains a set of constants forming the efi header.
Make the constants part of a data symbol.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/kernel/efi-header.S | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/arm64/kernel/efi-header.S b/arch/arm64/kernel/efi-header.S
index 28d8a5dca5f1..3eacd27ab761 100644
--- a/arch/arm64/kernel/efi-header.S
+++ b/arch/arm64/kernel/efi-header.S
@@ -28,6 +28,7 @@
.macro __EFI_PE_HEADER
#ifdef CONFIG_EFI
.set .Lpe_header_offset, . - .L_head
+SYM_DATA_START_LOCAL(arm64_efi_header)
.long PE_MAGIC
.short IMAGE_FILE_MACHINE_ARM64 // Machine
.short .Lsection_count // NumberOfSections
@@ -160,6 +161,7 @@
.balign SEGMENT_ALIGN
.Lefi_header_end:
+SYM_DATA_END_LABEL(arm64_efi_header, SYM_L_LOCAL, efi_header_end)
#else
.set .Lpe_header_offset, 0x0
#endif
--
2.17.1
In the general use case, KGDB breakpoint handler should return normally
to the instruction following the brk.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/include/asm/kgdb.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/include/asm/kgdb.h b/arch/arm64/include/asm/kgdb.h
index 21fc85e9d2be..a8cb91d8d59b 100644
--- a/arch/arm64/include/asm/kgdb.h
+++ b/arch/arm64/include/asm/kgdb.h
@@ -19,6 +19,7 @@
static inline void arch_kgdb_breakpoint(void)
{
asm ("brk %0" : : "I" (KGDB_COMPILED_DBG_BRK_IMM));
+ annotate_reachable();
}
extern void kgdb_handle_bus_error(void);
--
2.17.1
VDSO code should be inside .rodata.
Now code in kuser32.S and sigreturn32.S are inside .text section and never
executed.
Move them to .rodata.
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/kernel/kuser32.S | 1 +
arch/arm64/kernel/sigreturn32.S | 1 +
2 files changed, 2 insertions(+)
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index 42bd8c0c60e0..692e9d2e31e5 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -15,6 +15,7 @@
#include <asm/unistd.h>
+ .section .rodata
.align 5
.globl __kuser_helper_start
__kuser_helper_start:
diff --git a/arch/arm64/kernel/sigreturn32.S b/arch/arm64/kernel/sigreturn32.S
index 475d30d471ac..ccbd4aab4ba4 100644
--- a/arch/arm64/kernel/sigreturn32.S
+++ b/arch/arm64/kernel/sigreturn32.S
@@ -15,6 +15,7 @@
#include <asm/unistd.h>
+ .section .rodata
.globl __aarch32_sigret_code_start
__aarch32_sigret_code_start:
--
2.17.1
Decode aarch64 additions and substractions and create stack_ops for
instructions interacting with SP or FP.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/objtool/arch/arm64/decode.c | 82 +++++++++++++++++++++++++++++++
1 file changed, 82 insertions(+)
diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/decode.c
index afe22c4593c8..d8c32703874d 100644
--- a/tools/objtool/arch/arm64/decode.c
+++ b/tools/objtool/arch/arm64/decode.c
@@ -15,6 +15,22 @@
#include "../../../arch/arm64/lib/insn.c"
+#define is_SP(reg) (reg == AARCH64_INSN_REG_SP)
+#define is_FP(reg) (reg == AARCH64_INSN_REG_FP)
+#define is_SPFP(reg) (reg == AARCH64_INSN_REG_SP || reg == AARCH64_INSN_REG_FP)
+
+#define ADD_OP(op) \
+ if (!(op = calloc(1, sizeof(*op)))) \
+ return -1; \
+ else for (list_add_tail(&op->list, ops_list); op; op = NULL)
+
+static unsigned long sign_extend(unsigned long x, int nbits)
+{
+ unsigned long sign_bit = (x >> (nbits - 1)) & 1;
+
+ return ((~0UL + (sign_bit ^ 1)) << nbits) | x;
+}
+
bool arch_callee_saved_reg(unsigned char reg)
{
switch (reg) {
@@ -105,6 +121,42 @@ int arch_decode_hint_reg(u8 sp_reg, int *base)
return -1;
}
+static inline void make_add_op(enum aarch64_insn_register dest,
+ enum aarch64_insn_register src,
+ int val, struct stack_op *op)
+{
+ op->dest.type = OP_DEST_REG;
+ op->dest.reg = dest;
+ op->src.reg = src;
+ op->src.type = val != 0 ? OP_SRC_ADD : OP_SRC_REG;
+ op->src.offset = val;
+}
+
+static void decode_add_sub_imm(u32 instr, bool set_flags,
+ unsigned long *immediate,
+ struct stack_op *op)
+{
+ u32 rd = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RD, instr);
+ u32 rn = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RN, instr);
+
+ *immediate = aarch64_insn_decode_immediate(AARCH64_INSN_IMM_12, instr);
+
+ if (instr & AARCH64_INSN_LSL_12)
+ *immediate <<= 12;
+
+ if ((!set_flags && is_SP(rd)) || is_FP(rd)
+ || is_SPFP(rn)) {
+ int value;
+
+ if (aarch64_insn_is_subs_imm(instr) || aarch64_insn_is_sub_imm(instr))
+ value = -*immediate;
+ else
+ value = *immediate;
+
+ make_add_op(rd, rn, value, op);
+ }
+}
+
int arch_decode_instruction(struct objtool_file *file, const struct section *sec,
unsigned long offset, unsigned int maxlen,
unsigned int *len, enum insn_type *type,
@@ -112,6 +164,7 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
struct list_head *ops_list)
{
const struct elf *elf = file->elf;
+ struct stack_op *op = NULL;
u32 insn;
if (!is_arm64(elf))
@@ -130,6 +183,35 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
case AARCH64_INSN_CLS_UNKNOWN:
WARN("can't decode instruction at %s:0x%lx", sec->name, offset);
return -1;
+ case AARCH64_INSN_CLS_DP_IMM:
+ /* Mov register to and from SP are aliases of add_imm */
+ if (aarch64_insn_is_add_imm(insn) ||
+ aarch64_insn_is_sub_imm(insn)) {
+ ADD_OP(op) {
+ decode_add_sub_imm(insn, false, immediate, op);
+ }
+ }
+ else if (aarch64_insn_is_adds_imm(insn) ||
+ aarch64_insn_is_subs_imm(insn)) {
+ ADD_OP(op) {
+ decode_add_sub_imm(insn, true, immediate, op);
+ }
+ }
+ break;
+ case AARCH64_INSN_CLS_DP_REG:
+ if (aarch64_insn_is_mov_reg(insn)) {
+ enum aarch64_insn_register rd;
+ enum aarch64_insn_register rm;
+
+ rd = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RD, insn);
+ rm = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RM, insn);
+ if (is_FP(rd) || is_FP(rm)) {
+ ADD_OP(op) {
+ make_add_op(rd, rm, 0, op);
+ }
+ }
+ }
+ break;
default:
break;
}
--
2.17.1
Code symbols not following the aarch64 procedure call convention should
be annotated with SYM_CODE_* instead of SYM_FUNC_*
Mark relevant symbols as generic code symbols.
Also replace SYM_INNER_LABEL for __swpan_entry_el0 becasuse
SYM_INNER_LABEL generates zero-size-label which can't be correctly loaded
in objtool.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/kernel/entry.S | 10 +++++--
arch/arm64/kernel/head.S | 56 +++++++++++++++++++-------------------
arch/arm64/kernel/sleep.S | 4 +--
arch/arm64/kvm/hyp/entry.S | 4 +--
4 files changed, 39 insertions(+), 35 deletions(-)
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index ede028dee81b..c460ba2d009d 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -460,11 +460,15 @@ SYM_CODE_START_LOCAL(__swpan_entry_el1)
orr x23, x23, #PSR_PAN_BIT // Set the emulated PAN in the saved SPSR
b.eq 1f // TTBR0 access already disabled
and x23, x23, #~PSR_PAN_BIT // Clear the emulated PAN in the saved SPSR
-SYM_INNER_LABEL(__swpan_entry_el0, SYM_L_LOCAL)
__uaccess_ttbr0_disable x21
1: ret
SYM_CODE_END(__swpan_entry_el1)
+SYM_CODE_START_LOCAL(__swpan_entry_el0)
+ __uaccess_ttbr0_disable x21
+1: ret
+SYM_CODE_END(__swpan_entry_el0)
+
/*
* Restore access to TTBR0_EL1. If returning to EL0, no need for SPSR
* PAN bit checking.
@@ -757,11 +761,11 @@ SYM_CODE_START_NOALIGN(tramp_vectors)
generate_tramp_vector kpti=1, bhb=BHB_MITIGATION_NONE
SYM_CODE_END(tramp_vectors)
-SYM_CODE_START(tramp_exit_native)
+SYM_CODE_START_LOCAL(tramp_exit_native)
tramp_exit
SYM_CODE_END(tramp_exit_native)
-SYM_CODE_START(tramp_exit_compat)
+SYM_CODE_START_LOCAL(tramp_exit_compat)
tramp_exit 32
SYM_CODE_END(tramp_exit_compat)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 6a98f1a38c29..6db9c3603bd8 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -259,7 +259,7 @@ SYM_CODE_END(preserve_boot_args)
* - first few MB of the kernel linear mapping to jump to once the MMU has
* been enabled
*/
-SYM_FUNC_START_LOCAL(__create_page_tables)
+SYM_CODE_START_LOCAL(__create_page_tables)
mov x28, lr
/*
@@ -389,7 +389,7 @@ SYM_FUNC_START_LOCAL(__create_page_tables)
bl dcache_inval_poc
ret x28
-SYM_FUNC_END(__create_page_tables)
+SYM_CODE_END(__create_page_tables)
/*
* Initialize CPU registers with task-specific and cpu-specific context.
@@ -422,7 +422,7 @@ SYM_FUNC_END(__create_page_tables)
*
* x0 = __PHYS_OFFSET
*/
-SYM_FUNC_START_LOCAL(__primary_switched)
+SYM_CODE_START_LOCAL(__primary_switched)
adr_l x4, init_task
init_cpu_task x4, x5, x6
@@ -467,7 +467,7 @@ SYM_FUNC_START_LOCAL(__primary_switched)
ldp x29, x30, [sp], #16
bl start_kernel
ASM_BUG()
-SYM_FUNC_END(__primary_switched)
+SYM_CODE_END(__primary_switched)
.pushsection ".rodata", "a"
SYM_DATA_START(kimage_vaddr)
@@ -493,7 +493,7 @@ EXPORT_SYMBOL(kimage_vaddr)
* Returns either BOOT_CPU_MODE_EL1 or BOOT_CPU_MODE_EL2 in w0 if
* booted in EL1 or EL2 respectively.
*/
-SYM_FUNC_START(init_kernel_el)
+SYM_CODE_START(init_kernel_el)
mrs x0, CurrentEL
cmp x0, #CurrentEL_EL2
b.eq init_el2
@@ -557,13 +557,13 @@ __cpu_stick_to_vhe:
hvc #0
mov x0, #BOOT_CPU_MODE_EL2
ret
-SYM_FUNC_END(init_kernel_el)
+SYM_CODE_END(init_kernel_el)
/*
* Sets the __boot_cpu_mode flag depending on the CPU boot mode passed
* in w0. See arch/arm64/include/asm/virt.h for more info.
*/
-SYM_FUNC_START_LOCAL(set_cpu_boot_mode_flag)
+SYM_CODE_START_LOCAL(set_cpu_boot_mode_flag)
adr_l x1, __boot_cpu_mode
cmp w0, #BOOT_CPU_MODE_EL2
b.ne 1f
@@ -572,7 +572,7 @@ SYM_FUNC_START_LOCAL(set_cpu_boot_mode_flag)
dmb sy
dc ivac, x1 // Invalidate potentially stale cache line
ret
-SYM_FUNC_END(set_cpu_boot_mode_flag)
+SYM_CODE_END(set_cpu_boot_mode_flag)
/*
* These values are written with the MMU off, but read with the MMU on.
@@ -606,7 +606,7 @@ SYM_DATA_END(__early_cpu_boot_status)
* This provides a "holding pen" for platforms to hold all secondary
* cores are held until we're ready for them to initialise.
*/
-SYM_FUNC_START(secondary_holding_pen)
+SYM_CODE_START(secondary_holding_pen)
bl init_kernel_el // w0=cpu_boot_mode
bl set_cpu_boot_mode_flag
mrs x0, mpidr_el1
@@ -618,19 +618,19 @@ pen: ldr x4, [x3]
b.eq secondary_startup
wfe
b pen
-SYM_FUNC_END(secondary_holding_pen)
+SYM_CODE_END(secondary_holding_pen)
/*
* Secondary entry point that jumps straight into the kernel. Only to
* be used where CPUs are brought online dynamically by the kernel.
*/
-SYM_FUNC_START(secondary_entry)
+SYM_CODE_START(secondary_entry)
bl init_kernel_el // w0=cpu_boot_mode
bl set_cpu_boot_mode_flag
b secondary_startup
-SYM_FUNC_END(secondary_entry)
+SYM_CODE_END(secondary_entry)
-SYM_FUNC_START_LOCAL(secondary_startup)
+SYM_CODE_START_LOCAL(secondary_startup)
/*
* Common entry point for secondary CPUs.
*/
@@ -641,9 +641,9 @@ SYM_FUNC_START_LOCAL(secondary_startup)
bl __enable_mmu
ldr x8, =__secondary_switched
br x8
-SYM_FUNC_END(secondary_startup)
+SYM_CODE_END(secondary_startup)
-SYM_FUNC_START_LOCAL(__secondary_switched)
+SYM_CODE_START_LOCAL(__secondary_switched)
adr_l x5, vectors
msr vbar_el1, x5
isb
@@ -660,13 +660,13 @@ SYM_FUNC_START_LOCAL(__secondary_switched)
bl secondary_start_kernel
ASM_BUG()
-SYM_FUNC_END(__secondary_switched)
+SYM_CODE_END(__secondary_switched)
-SYM_FUNC_START_LOCAL(__secondary_too_slow)
+SYM_CODE_START_LOCAL(__secondary_too_slow)
wfe
wfi
b __secondary_too_slow
-SYM_FUNC_END(__secondary_too_slow)
+SYM_CODE_END(__secondary_too_slow)
/*
* The booting CPU updates the failed status @__early_cpu_boot_status,
@@ -698,7 +698,7 @@ SYM_FUNC_END(__secondary_too_slow)
* Checks if the selected granule size is supported by the CPU.
* If it isn't, park the CPU
*/
-SYM_FUNC_START(__enable_mmu)
+SYM_CODE_START(__enable_mmu)
mrs x2, ID_AA64MMFR0_EL1
ubfx x2, x2, #ID_AA64MMFR0_TGRAN_SHIFT, 4
cmp x2, #ID_AA64MMFR0_TGRAN_SUPPORTED_MIN
@@ -717,9 +717,9 @@ SYM_FUNC_START(__enable_mmu)
set_sctlr_el1 x0
ret
-SYM_FUNC_END(__enable_mmu)
+SYM_CODE_END(__enable_mmu)
-SYM_FUNC_START(__cpu_secondary_check52bitva)
+SYM_CODE_START_LOCAL(__cpu_secondary_check52bitva)
#ifdef CONFIG_ARM64_VA_BITS_52
ldr_l x0, vabits_actual
cmp x0, #52
@@ -737,9 +737,9 @@ SYM_FUNC_START(__cpu_secondary_check52bitva)
#endif
2: ret
-SYM_FUNC_END(__cpu_secondary_check52bitva)
+SYM_CODE_END(__cpu_secondary_check52bitva)
-SYM_FUNC_START_LOCAL(__no_granule_support)
+SYM_CODE_START_LOCAL(__no_granule_support)
/* Indicate that this CPU can't boot and is stuck in the kernel */
update_early_cpu_boot_status \
CPU_STUCK_IN_KERNEL | CPU_STUCK_REASON_NO_GRAN, x1, x2
@@ -747,10 +747,10 @@ SYM_FUNC_START_LOCAL(__no_granule_support)
wfe
wfi
b 1b
-SYM_FUNC_END(__no_granule_support)
+SYM_CODE_END(__no_granule_support)
#ifdef CONFIG_RELOCATABLE
-SYM_FUNC_START_LOCAL(__relocate_kernel)
+SYM_CODE_START_LOCAL(__relocate_kernel)
/*
* Iterate over each entry in the relocation table, and apply the
* relocations in place.
@@ -852,10 +852,10 @@ SYM_FUNC_START_LOCAL(__relocate_kernel)
#endif
ret
-SYM_FUNC_END(__relocate_kernel)
+SYM_CODE_END(__relocate_kernel)
#endif
-SYM_FUNC_START_LOCAL(__primary_switch)
+SYM_CODE_START_LOCAL(__primary_switch)
#ifdef CONFIG_RANDOMIZE_BASE
mov x19, x0 // preserve new SCTLR_EL1 value
mrs x20, sctlr_el1 // preserve old SCTLR_EL1 value
@@ -895,4 +895,4 @@ SYM_FUNC_START_LOCAL(__primary_switch)
ldr x8, =__primary_switched
adrp x0, __PHYS_OFFSET
br x8
-SYM_FUNC_END(__primary_switch)
+SYM_CODE_END(__primary_switch)
diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S
index 4ea9392f86e0..f0087e8bcd28 100644
--- a/arch/arm64/kernel/sleep.S
+++ b/arch/arm64/kernel/sleep.S
@@ -111,7 +111,7 @@ SYM_CODE_END(cpu_resume)
.ltorg
.popsection
-SYM_FUNC_START(_cpu_resume)
+SYM_CODE_START(_cpu_resume)
mrs x1, mpidr_el1
adr_l x8, mpidr_hash // x8 = struct mpidr_hash virt address
@@ -147,4 +147,4 @@ SYM_FUNC_START(_cpu_resume)
ldp x29, lr, [x29]
mov x0, #0
ret
-SYM_FUNC_END(_cpu_resume)
+SYM_CODE_END(_cpu_resume)
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index 435346ea1504..045d4481c820 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -21,7 +21,7 @@
/*
* u64 __guest_enter(struct kvm_vcpu *vcpu);
*/
-SYM_FUNC_START(__guest_enter)
+SYM_CODE_START(__guest_enter)
// x0: vcpu
// x1-x17: clobbered by macros
// x29: guest context
@@ -212,4 +212,4 @@ abort_guest_exit_end:
msr spsr_el2, x4
orr x0, x0, x5
1: ret
-SYM_FUNC_END(__guest_enter)
+SYM_CODE_END(__guest_enter)
--
2.17.1
In __cpu_suspend_enter, the FP and LR are properly saved on the stack to
form a stack frame, but the frame pointer is not set afterwards.
Have the frame pointer point to the new frame.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/kernel/sleep.S | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S
index 799ec01b0649..7fd276f3c532 100644
--- a/arch/arm64/kernel/sleep.S
+++ b/arch/arm64/kernel/sleep.S
@@ -92,6 +92,7 @@ SYM_FUNC_START(__cpu_suspend_enter)
str x0, [x1]
add x0, x0, #SLEEP_STACK_DATA_SYSTEM_REGS
stp x29, lr, [sp, #-16]!
+ mov x29, sp
bl cpu_do_suspend
ldp x29, lr, [sp], #16
mov x0, #1
--
2.17.1
Add build option to run stack validation at compile time.
When requiring stack validation, jump tables are disabled as it
simplifies objtool analysis (without having to introduce unreliable
artifacs). In local testing, this does not appear to significaly
affect final binary size nor system performance.
Signed-off-by: Raphael Gault <[email protected]>
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/Kconfig | 2 ++
arch/arm64/Kconfig.debug | 21 +++++++++++++++++++++
arch/arm64/Makefile | 4 ++++
3 files changed, 27 insertions(+)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 57c4c995965f..30d3d549160f 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -201,6 +201,8 @@ config ARM64
select MMU_GATHER_RCU_TABLE_FREE
select HAVE_RSEQ
select HAVE_STACKPROTECTOR
+ select HAVE_OBJTOOL
+ select HAVE_STACK_VALIDATION
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
diff --git a/arch/arm64/Kconfig.debug b/arch/arm64/Kconfig.debug
index 265c4461031f..c2c68c6f7557 100644
--- a/arch/arm64/Kconfig.debug
+++ b/arch/arm64/Kconfig.debug
@@ -20,4 +20,25 @@ config ARM64_RELOC_TEST
depends on m
tristate "Relocation testing module"
+choice
+ prompt "Choose kernel unwinder"
+ default UNWINDER_FRAME_POINTER
+ help
+ This determines which method will be used for unwinding kernel stack
+ traces for panics, oopses, bugs, warnings, perf, /proc/<pid>/stack,
+ livepatch, lockdep, and more.
+
+config UNWINDER_FRAME_POINTER
+ bool "Frame pointer unwinder"
+ select FRAME_POINTER
+ help
+ This option enables the frame pointer unwinder for unwinding kernel
+ stack traces.
+
+ The unwinder itself is fast and it uses less RAM than the ORC
+ unwinder, but the kernel text size will grow by ~3% and the kernel's
+ overall performance will degrade by roughly 5-10%.
+
+endchoice
+
source "drivers/hwtracing/coresight/Kconfig"
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 2f1de88651e6..ad2f4a5e8f6c 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -133,6 +133,10 @@ ifeq ($(CONFIG_DYNAMIC_FTRACE_WITH_REGS),y)
CC_FLAGS_FTRACE := -fpatchable-function-entry=2
endif
+ifeq ($(CONFIG_STACK_VALIDATION),y)
+KBUILD_CFLAGS += -fno-jump-tables
+endif
+
# Default value
head-y := arch/arm64/kernel/head.o
--
2.17.1
ARM64_CB_PATCH doesn't have static replacement instructions. Skip
trying to validate the alternative section.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/objtool/arch/arm64/special.c | 11 +++++++++++
tools/objtool/check.c | 3 +++
2 files changed, 14 insertions(+)
diff --git a/tools/objtool/arch/arm64/special.c b/tools/objtool/arch/arm64/special.c
index a70b91e8bd7d..8bb1ebd2132a 100644
--- a/tools/objtool/arch/arm64/special.c
+++ b/tools/objtool/arch/arm64/special.c
@@ -4,6 +4,17 @@
void arch_handle_alternative(unsigned short feature, struct special_alt *alt)
{
+ /*
+ * ARM64_CB_PATCH has no alternative instruction.
+ * a callback is called at alternative replacement time
+ * to dynamically change the original instructions.
+ *
+ * ARM64_CB_PATCH is the last ARM64 feature, it's value changes
+ * every time a new feature is added. So the orig/alt region
+ * length are used to detect those alternatives
+ */
+ if (alt->orig_len && !alt->new_len)
+ alt->skip_alt = true;
}
bool arch_support_alt_relocation(struct special_alt *special_alt,
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index f19ea3fc1e6b..8ff7c30df513 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -1687,6 +1687,9 @@ static int add_special_section_alts(struct objtool_file *file)
continue;
}
+ if (special_alt->skip_alt && !special_alt->new_len)
+ continue;
+
ret = handle_group_alt(file, special_alt, orig_insn,
&new_insn);
if (ret)
--
2.17.1
In kernel_exit there is a alternative branch for KPTI which causes
stack size conflict for two instruction boundaries.
To fix that, make both branch move the sp and then revert it in
tramp_exit branch.
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/kernel/entry.S | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index d49bfbe81a0d..677e3be471bb 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -430,7 +430,11 @@ alternative_if_not ARM64_UNMAP_KERNEL_AT_EL0
ldr lr, [sp, #S_LR]
add sp, sp, #PT_REGS_SIZE // restore sp
eret
-alternative_else_nop_endif
+alternative_else
+ nop
+ add sp, sp, #PT_REGS_SIZE // restore sp
+ nop
+alternative_endif
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
bne 4f
msr far_el1, x29
@@ -729,6 +733,7 @@ alternative_else_nop_endif
.macro tramp_exit, regsize = 64
UNWIND_HINT_EMPTY
+ sub sp, sp, #PT_REGS_SIZE // revert sp
tramp_data_read_var x30, this_cpu_vector
get_this_cpu_offset x29
ldr x30, [x30, x29]
--
2.17.1
Use SYM_DATA_* macros to annotate data bytes in the middle of .text
sections.
For local symbols, ".L" prefix needs to be dropped as the assembler
exclude the symbols from the .o symbol table, making objtool unable
to see them.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/crypto/aes-neonbs-core.S | 14 +++++++-------
arch/arm64/crypto/poly1305-armv8.pl | 4 ++++
arch/arm64/crypto/sha512-armv8.pl | 24 ++++++++++++++----------
3 files changed, 25 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/crypto/aes-neonbs-core.S b/arch/arm64/crypto/aes-neonbs-core.S
index d427f4556b6e..fa161d1bf264 100644
--- a/arch/arm64/crypto/aes-neonbs-core.S
+++ b/arch/arm64/crypto/aes-neonbs-core.S
@@ -367,15 +367,15 @@
.align 6
-M0: .octa 0x0004080c0105090d02060a0e03070b0f
+SYM_DATA_LOCAL(M0, .octa 0x0004080c0105090d02060a0e03070b0f)
-M0SR: .octa 0x0004080c05090d010a0e02060f03070b
-SR: .octa 0x0f0e0d0c0a09080b0504070600030201
-SRM0: .octa 0x01060b0c0207080d0304090e00050a0f
+SYM_DATA_LOCAL(M0SR, .octa 0x0004080c05090d010a0e02060f03070b)
+SYM_DATA_LOCAL(SR, .octa 0x0f0e0d0c0a09080b0504070600030201)
+SYM_DATA_LOCAL(SRM0, .octa 0x01060b0c0207080d0304090e00050a0f)
-M0ISR: .octa 0x0004080c0d0105090a0e0206070b0f03
-ISR: .octa 0x0f0e0d0c080b0a090504070602010003
-ISRM0: .octa 0x0306090c00070a0d01040b0e0205080f
+SYM_DATA_LOCAL(M0ISR, .octa 0x0004080c0d0105090a0e0206070b0f03)
+SYM_DATA_LOCAL(ISR, .octa 0x0f0e0d0c080b0a090504070602010003)
+SYM_DATA_LOCAL(ISRM0, .octa 0x0306090c00070a0d01040b0e0205080f)
/*
* void aesbs_convert_key(u8 out[], u32 const rk[], int rounds)
diff --git a/arch/arm64/crypto/poly1305-armv8.pl b/arch/arm64/crypto/poly1305-armv8.pl
index cbc980fb02e3..f460f33c127a 100644
--- a/arch/arm64/crypto/poly1305-armv8.pl
+++ b/arch/arm64/crypto/poly1305-armv8.pl
@@ -47,6 +47,8 @@ my ($mac,$nonce)=($inp,$len);
my ($h0,$h1,$h2,$r0,$r1,$s1,$t0,$t1,$d0,$d1,$d2) = map("x$_",(4..14));
$code.=<<___;
+#include <linux/linkage.h>
+
#ifndef __KERNEL__
# include "arm_arch.h"
.extern OPENSSL_armcap_P
@@ -888,8 +890,10 @@ poly1305_blocks_neon:
.align 5
.Lzeros:
.long 0,0,0,0,0,0,0,0
+SYM_DATA_START_LOCAL(POLY1305_str)
.asciz "Poly1305 for ARMv8, CRYPTOGAMS by \@dot-asm"
.align 2
+SYM_DATA_END(POLY1305_str)
#if !defined(__KERNEL__) && !defined(_WIN64)
.comm OPENSSL_armcap_P,4,4
.hidden OPENSSL_armcap_P
diff --git a/arch/arm64/crypto/sha512-armv8.pl b/arch/arm64/crypto/sha512-armv8.pl
index 35ec9ae99fe1..1882c4110026 100644
--- a/arch/arm64/crypto/sha512-armv8.pl
+++ b/arch/arm64/crypto/sha512-armv8.pl
@@ -193,6 +193,8 @@ ___
}
$code.=<<___;
+#include <linux/linkage.h>
+
#ifndef __KERNEL__
# include "arm_arch.h"
#endif
@@ -208,11 +210,11 @@ ___
$code.=<<___ if ($SZ==4);
#ifndef __KERNEL__
# ifdef __ILP32__
- ldrsw x16,.LOPENSSL_armcap_P
+ ldrsw x16,OPENSSL_armcap_P_rel
# else
- ldr x16,.LOPENSSL_armcap_P
+ ldr x16,OPENSSL_armcap_P_rel
# endif
- adr x17,.LOPENSSL_armcap_P
+ adr x17,OPENSSL_armcap_P_rel
add x16,x16,x17
ldr w16,[x16]
tst w16,#ARMV8_SHA256
@@ -237,7 +239,7 @@ $code.=<<___;
ldp $E,$F,[$ctx,#4*$SZ]
add $num,$inp,$num,lsl#`log(16*$SZ)/log(2)` // end of input
ldp $G,$H,[$ctx,#6*$SZ]
- adr $Ktbl,.LK$BITS
+ adr $Ktbl,K$BITS
stp $ctx,$num,[x29,#96]
.Loop:
@@ -287,8 +289,7 @@ $code.=<<___;
.size $func,.-$func
.align 6
-.type .LK$BITS,%object
-.LK$BITS:
+SYM_DATA_START_LOCAL(K$BITS)
___
$code.=<<___ if ($SZ==8);
.quad 0x428a2f98d728ae22,0x7137449123ef65cd
@@ -353,18 +354,21 @@ $code.=<<___ if ($SZ==4);
.long 0 //terminator
___
$code.=<<___;
-.size .LK$BITS,.-.LK$BITS
+SYM_DATA_END(K$BITS)
#ifndef __KERNEL__
.align 3
-.LOPENSSL_armcap_P:
+SYM_DATA_START_LOCAL(OPENSSL_armcap_P_rel)
# ifdef __ILP32__
.long OPENSSL_armcap_P-.
# else
.quad OPENSSL_armcap_P-.
# endif
+SYM_DATA_END(OPENSSL_armcap_P_rel)
#endif
+SYM_DATA_START_LOCAL(OPENSSL_str)
.asciz "SHA$BITS block transform for ARMv8, CRYPTOGAMS by <appro\@openssl.org>"
.align 2
+SYM_DATA_END(OPENSSL_str)
___
if ($SZ==4) {
@@ -385,7 +389,7 @@ sha256_block_armv8:
add x29,sp,#0
ld1.32 {$ABCD,$EFGH},[$ctx]
- adr $Ktbl,.LK256
+ adr $Ktbl,K256
.Loop_hw:
ld1 {@MSG[0]-@MSG[3]},[$inp],#64
@@ -648,7 +652,7 @@ sha256_block_neon:
mov x29, sp
sub sp,sp,#16*4
- adr $Ktbl,.LK256
+ adr $Ktbl,K256
add $num,$inp,$num,lsl#6 // len to point at the end of inp
ld1.8 {@X[0]},[$inp], #16
--
2.17.1
Load literal instructions can generate constants inside code sections.
Record the locations of the constants in order to be able to remove
their corresponding "struct instruction".
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/objtool/arch/arm64/decode.c | 88 +++++++++++++++++++++++++++-
tools/objtool/arch/x86/decode.c | 5 ++
tools/objtool/check.c | 3 +
tools/objtool/include/objtool/arch.h | 2 +
4 files changed, 97 insertions(+), 1 deletion(-)
diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/decode.c
index 8ce9d91ff0db..30300d05c8f3 100644
--- a/tools/objtool/arch/arm64/decode.c
+++ b/tools/objtool/arch/arm64/decode.c
@@ -31,6 +31,64 @@ static unsigned long sign_extend(unsigned long x, int nbits)
return ((~0UL + (sign_bit ^ 1)) << nbits) | x;
}
+struct insn_loc {
+ const struct section *sec;
+ unsigned long offset;
+ struct hlist_node hnode;
+};
+
+DEFINE_HASHTABLE(invalid_insns, 16);
+
+static int record_invalid_insn(const struct section *sec,
+ unsigned long offset)
+{
+ struct insn_loc *loc;
+ struct hlist_head *l;
+
+ l = &invalid_insns[hash_min(offset, HASH_BITS(invalid_insns))];
+ if (!hlist_empty(l)) {
+ loc = hlist_entry(l->first, struct insn_loc, hnode);
+ return 0;
+ }
+
+ loc = malloc(sizeof(*loc));
+ if (!loc) {
+ WARN("malloc failed");
+ return -1;
+ }
+
+ loc->sec = sec;
+ loc->offset = offset;
+
+ hash_add(invalid_insns, &loc->hnode, loc->offset);
+
+ return 0;
+}
+
+int arch_post_process_instructions(struct objtool_file *file)
+{
+ struct hlist_node *tmp;
+ struct insn_loc *loc;
+ unsigned int bkt;
+ int res = 0;
+
+ hash_for_each_safe(invalid_insns, bkt, tmp, loc, hnode) {
+ struct instruction *insn;
+
+ insn = find_insn(file, (struct section *) loc->sec, loc->offset);
+ if (insn) {
+ list_del(&insn->list);
+ hash_del(&insn->hash);
+ free(insn);
+ }
+
+ hash_del(&loc->hnode);
+ free(loc);
+ }
+
+ return res;
+}
+
bool arch_callee_saved_reg(unsigned char reg)
{
switch (reg) {
@@ -351,7 +409,35 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
break;
case AARCH64_INSN_CLS_LDST:
{
- return decode_load_store(insn, immediate, ops_list);
+ int ret;
+
+ ret = decode_load_store(insn, immediate, ops_list);
+ if (ret <= 0)
+ return ret;
+
+ /*
+ * For LDR ops, assembler can generate the data to be
+ * loaded in the code section
+ * Record and remove these data because they
+ * are never excuted
+ */
+ if (aarch64_insn_is_ldr_lit(insn)) {
+ long pc_offset;
+
+ pc_offset = insn & GENMASK(23, 5);
+ /* Sign extend and multiply by 4 */
+ pc_offset = (pc_offset << (64 - 23));
+ pc_offset = ((pc_offset >> (64 - 23)) >> 5) << 2;
+
+ ret = record_invalid_insn(sec, offset + pc_offset);
+
+ /* 64-bit literal */
+ if (insn & BIT(30))
+ ret = record_invalid_insn(sec, offset + pc_offset + 4);
+
+ return ret;
+ }
+ break;
}
default:
break;
diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decode.c
index 8b990a52aada..081b5e72f8df 100644
--- a/tools/objtool/arch/x86/decode.c
+++ b/tools/objtool/arch/x86/decode.c
@@ -693,6 +693,11 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
return 0;
}
+int arch_post_process_instructions(struct objtool_file *file)
+{
+ return 0;
+}
+
void arch_initial_func_cfi_state(struct cfi_init_state *state)
{
int i;
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 35d0a1bc4279..c0feb6db7c6d 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -439,6 +439,9 @@ static int decode_instructions(struct objtool_file *file)
if (opts.stats)
printf("nr_insns: %lu\n", nr_insns);
+ if (arch_post_process_instructions(file))
+ return -1;
+
return 0;
err:
diff --git a/tools/objtool/include/objtool/arch.h b/tools/objtool/include/objtool/arch.h
index 9b19cc304195..651262af2655 100644
--- a/tools/objtool/include/objtool/arch.h
+++ b/tools/objtool/include/objtool/arch.h
@@ -77,6 +77,8 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
unsigned long *immediate,
struct list_head *ops_list);
+int arch_post_process_instructions(struct objtool_file *file);
+
bool arch_callee_saved_reg(unsigned char reg);
unsigned long arch_jump_destination(struct instruction *insn);
--
2.17.1
'commit 08feafe8d195 ("objtool: Fix function fallthrough detection for vmlinux")'
This commit canceled c_file which used to make fallthrough detection
only works on C objects.
However in arm64/crypto/aes-mods.S, there are cases that JUMP at the
end of function which make objtool wrongly detected them as fall through.
Revert c_file before this is fixed.
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/objtool/check.c | 3 +--
tools/objtool/include/objtool/objtool.h | 2 +-
tools/objtool/objtool.c | 1 +
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 8ff7c30df513..95cb88da4ed5 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -541,7 +541,6 @@ static struct instruction *find_last_insn(struct objtool_file *file,
struct instruction *insn = NULL;
unsigned int offset;
unsigned int end = (sec->sh.sh_size > 10) ? sec->sh.sh_size - 10 : 0;
-
for (offset = sec->sh.sh_size - 1; offset >= end && !insn; offset--)
insn = find_insn(file, sec, offset);
@@ -3220,7 +3219,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
while (1) {
next_insn = next_insn_to_validate(file, insn);
- if (func && insn->func && func != insn->func->pfunc) {
+ if (file->c_file && func && insn->func && func != insn->func->pfunc) {
WARN("%s() falls through to next function %s()",
func->name, insn->func->name);
return 1;
diff --git a/tools/objtool/include/objtool/objtool.h b/tools/objtool/include/objtool/objtool.h
index a6e72d916807..7a5c13a78f87 100644
--- a/tools/objtool/include/objtool/objtool.h
+++ b/tools/objtool/include/objtool/objtool.h
@@ -27,7 +27,7 @@ struct objtool_file {
struct list_head static_call_list;
struct list_head mcount_loc_list;
struct list_head endbr_list;
- bool ignore_unreachables, hints, rodata;
+ bool ignore_unreachables, c_file, hints, rodata;
unsigned int nr_endbr;
unsigned int nr_endbr_int;
diff --git a/tools/objtool/objtool.c b/tools/objtool/objtool.c
index 512669ce064c..d33620b1392d 100644
--- a/tools/objtool/objtool.c
+++ b/tools/objtool/objtool.c
@@ -105,6 +105,7 @@ struct objtool_file *objtool_open_read(const char *_objname)
INIT_LIST_HEAD(&file.static_call_list);
INIT_LIST_HEAD(&file.mcount_loc_list);
INIT_LIST_HEAD(&file.endbr_list);
+ file.c_file = !opts.link && find_section_by_name(file.elf, ".comment");
file.ignore_unreachables = opts.no_unreachable;
file.hints = false;
--
2.17.1
Stack validation requires BL instructions to an address,
within the symbol containing the BL should be annotated as intra-function
calls.
Make __pmull_p8_core normally set frame because there's a intra-function
call destinating middle of it and the caller have set the frame. When
analyzing the insns there will be a cfi state mismatch between normal-call
and intra-call.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/crypto/crct10dif-ce-core.S | 5 +++++
arch/arm64/kernel/entry.S | 2 ++
2 files changed, 7 insertions(+)
diff --git a/arch/arm64/crypto/crct10dif-ce-core.S b/arch/arm64/crypto/crct10dif-ce-core.S
index dce6dcebfca1..b3b8e56cb87d 100644
--- a/arch/arm64/crypto/crct10dif-ce-core.S
+++ b/arch/arm64/crypto/crct10dif-ce-core.S
@@ -63,6 +63,7 @@
//
#include <linux/linkage.h>
+#include <linux/objtool.h>
#include <asm/assembler.h>
.text
@@ -132,6 +133,8 @@
.endm
SYM_FUNC_START_LOCAL(__pmull_p8_core)
+ stp x29, x30, [sp, #-16]!
+ mov x29, sp
.L__pmull_p8_core:
ext t4.8b, ad.8b, ad.8b, #1 // A1
ext t5.8b, ad.8b, ad.8b, #2 // A2
@@ -193,6 +196,7 @@ SYM_FUNC_START_LOCAL(__pmull_p8_core)
eor t4.16b, t4.16b, t5.16b
eor t6.16b, t6.16b, t3.16b
+ ldp x29, x30, [sp], #16
ret
SYM_FUNC_END(__pmull_p8_core)
@@ -207,6 +211,7 @@ SYM_FUNC_END(__pmull_p8_core)
pmull2 \rq\().8h, \ad\().16b, \bd\().16b // D = A*B
.endif
+ ANNOTATE_INTRA_FUNCTION_CALL
bl .L__pmull_p8_core\i
eor \rq\().16b, \rq\().16b, t4.16b
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index bbc440379304..d49bfbe81a0d 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -10,6 +10,7 @@
#include <linux/arm-smccc.h>
#include <linux/init.h>
#include <linux/linkage.h>
+#include <linux/objtool.h>
#include <asm/alternative.h>
#include <asm/assembler.h>
@@ -694,6 +695,7 @@ alternative_else_nop_endif
* entry onto the return stack and using a RET instruction to
* enter the full-fat kernel vectors.
*/
+ ANNOTATE_INTRA_FUNCTION_CALL
bl 2f
UNWIND_HINT_EMPTY
b .
--
2.17.1
Add aarch64 encoder/decoder implementation under tools/.
The insn.h/.c are fixed version for user space tools. Some functions
(mainly about instructions generator code) and macros are deleted so that
it won't depend on too much head files.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/arch/arm64/include/asm/insn.h | 458 ++++++++++++++++++++++++++++
tools/arch/arm64/lib/insn.c | 335 ++++++++++++++++++++
2 files changed, 793 insertions(+)
create mode 100644 tools/arch/arm64/include/asm/insn.h
create mode 100644 tools/arch/arm64/lib/insn.c
diff --git a/tools/arch/arm64/include/asm/insn.h b/tools/arch/arm64/include/asm/insn.h
new file mode 100644
index 000000000000..839345692214
--- /dev/null
+++ b/tools/arch/arm64/include/asm/insn.h
@@ -0,0 +1,458 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2013 Huawei Ltd.
+ * Author: Jiang Liu <[email protected]>
+ *
+ * Copyright (C) 2014 Zi Shen Lim <[email protected]>
+ */
+#ifndef __ASM_INSN_H
+#define __ASM_INSN_H
+#include <linux/build_bug.h>
+#include <linux/types.h>
+
+/* A64 instructions are always 32 bits. */
+#define AARCH64_INSN_SIZE 4
+
+#ifndef __ASSEMBLY__
+/*
+ * ARM Architecture Reference Manual for ARMv8 Profile-A, Issue A.a
+ * Section C3.1 "A64 instruction index by encoding":
+ * AArch64 main encoding table
+ * Bit position
+ * 28 27 26 25 Encoding Group
+ * 0 0 - - Unallocated
+ * 1 0 0 - Data processing, immediate
+ * 1 0 1 - Branch, exception generation and system instructions
+ * - 1 - 0 Loads and stores
+ * - 1 0 1 Data processing - register
+ * 0 1 1 1 Data processing - SIMD and floating point
+ * 1 1 1 1 Data processing - SIMD and floating point
+ * "-" means "don't care"
+ */
+enum aarch64_insn_encoding_class {
+ AARCH64_INSN_CLS_UNKNOWN, /* UNALLOCATED */
+ AARCH64_INSN_CLS_SVE, /* SVE instructions */
+ AARCH64_INSN_CLS_DP_IMM, /* Data processing - immediate */
+ AARCH64_INSN_CLS_DP_REG, /* Data processing - register */
+ AARCH64_INSN_CLS_DP_FPSIMD, /* Data processing - SIMD and FP */
+ AARCH64_INSN_CLS_LDST, /* Loads and stores */
+ AARCH64_INSN_CLS_BR_SYS, /* Branch, exception generation and
+ * system instructions */
+};
+
+enum aarch64_insn_hint_cr_op {
+ AARCH64_INSN_HINT_NOP = 0x0 << 5,
+ AARCH64_INSN_HINT_YIELD = 0x1 << 5,
+ AARCH64_INSN_HINT_WFE = 0x2 << 5,
+ AARCH64_INSN_HINT_WFI = 0x3 << 5,
+ AARCH64_INSN_HINT_SEV = 0x4 << 5,
+ AARCH64_INSN_HINT_SEVL = 0x5 << 5,
+
+ AARCH64_INSN_HINT_XPACLRI = 0x07 << 5,
+ AARCH64_INSN_HINT_PACIA_1716 = 0x08 << 5,
+ AARCH64_INSN_HINT_PACIB_1716 = 0x0A << 5,
+ AARCH64_INSN_HINT_AUTIA_1716 = 0x0C << 5,
+ AARCH64_INSN_HINT_AUTIB_1716 = 0x0E << 5,
+ AARCH64_INSN_HINT_PACIAZ = 0x18 << 5,
+ AARCH64_INSN_HINT_PACIASP = 0x19 << 5,
+ AARCH64_INSN_HINT_PACIBZ = 0x1A << 5,
+ AARCH64_INSN_HINT_PACIBSP = 0x1B << 5,
+ AARCH64_INSN_HINT_AUTIAZ = 0x1C << 5,
+ AARCH64_INSN_HINT_AUTIASP = 0x1D << 5,
+ AARCH64_INSN_HINT_AUTIBZ = 0x1E << 5,
+ AARCH64_INSN_HINT_AUTIBSP = 0x1F << 5,
+
+ AARCH64_INSN_HINT_ESB = 0x10 << 5,
+ AARCH64_INSN_HINT_PSB = 0x11 << 5,
+ AARCH64_INSN_HINT_TSB = 0x12 << 5,
+ AARCH64_INSN_HINT_CSDB = 0x14 << 5,
+
+ AARCH64_INSN_HINT_BTI = 0x20 << 5,
+ AARCH64_INSN_HINT_BTIC = 0x22 << 5,
+ AARCH64_INSN_HINT_BTIJ = 0x24 << 5,
+ AARCH64_INSN_HINT_BTIJC = 0x26 << 5,
+};
+
+enum aarch64_insn_imm_type {
+ AARCH64_INSN_IMM_ADR,
+ AARCH64_INSN_IMM_26,
+ AARCH64_INSN_IMM_19,
+ AARCH64_INSN_IMM_16,
+ AARCH64_INSN_IMM_14,
+ AARCH64_INSN_IMM_12,
+ AARCH64_INSN_IMM_9,
+ AARCH64_INSN_IMM_7,
+ AARCH64_INSN_IMM_6,
+ AARCH64_INSN_IMM_S,
+ AARCH64_INSN_IMM_R,
+ AARCH64_INSN_IMM_N,
+ AARCH64_INSN_IMM_MAX
+};
+
+enum aarch64_insn_register_type {
+ AARCH64_INSN_REGTYPE_RT,
+ AARCH64_INSN_REGTYPE_RN,
+ AARCH64_INSN_REGTYPE_RT2,
+ AARCH64_INSN_REGTYPE_RM,
+ AARCH64_INSN_REGTYPE_RD,
+ AARCH64_INSN_REGTYPE_RA,
+ AARCH64_INSN_REGTYPE_RS,
+};
+
+enum aarch64_insn_register {
+ AARCH64_INSN_REG_0 = 0,
+ AARCH64_INSN_REG_1 = 1,
+ AARCH64_INSN_REG_2 = 2,
+ AARCH64_INSN_REG_3 = 3,
+ AARCH64_INSN_REG_4 = 4,
+ AARCH64_INSN_REG_5 = 5,
+ AARCH64_INSN_REG_6 = 6,
+ AARCH64_INSN_REG_7 = 7,
+ AARCH64_INSN_REG_8 = 8,
+ AARCH64_INSN_REG_9 = 9,
+ AARCH64_INSN_REG_10 = 10,
+ AARCH64_INSN_REG_11 = 11,
+ AARCH64_INSN_REG_12 = 12,
+ AARCH64_INSN_REG_13 = 13,
+ AARCH64_INSN_REG_14 = 14,
+ AARCH64_INSN_REG_15 = 15,
+ AARCH64_INSN_REG_16 = 16,
+ AARCH64_INSN_REG_17 = 17,
+ AARCH64_INSN_REG_18 = 18,
+ AARCH64_INSN_REG_19 = 19,
+ AARCH64_INSN_REG_20 = 20,
+ AARCH64_INSN_REG_21 = 21,
+ AARCH64_INSN_REG_22 = 22,
+ AARCH64_INSN_REG_23 = 23,
+ AARCH64_INSN_REG_24 = 24,
+ AARCH64_INSN_REG_25 = 25,
+ AARCH64_INSN_REG_26 = 26,
+ AARCH64_INSN_REG_27 = 27,
+ AARCH64_INSN_REG_28 = 28,
+ AARCH64_INSN_REG_29 = 29,
+ AARCH64_INSN_REG_FP = 29, /* Frame pointer */
+ AARCH64_INSN_REG_30 = 30,
+ AARCH64_INSN_REG_LR = 30, /* Link register */
+ AARCH64_INSN_REG_ZR = 31, /* Zero: as source register */
+ AARCH64_INSN_REG_SP = 31 /* Stack pointer: as load/store base reg */
+};
+
+enum aarch64_insn_special_register {
+ AARCH64_INSN_SPCLREG_SPSR_EL1 = 0xC200,
+ AARCH64_INSN_SPCLREG_ELR_EL1 = 0xC201,
+ AARCH64_INSN_SPCLREG_SP_EL0 = 0xC208,
+ AARCH64_INSN_SPCLREG_SPSEL = 0xC210,
+ AARCH64_INSN_SPCLREG_CURRENTEL = 0xC212,
+ AARCH64_INSN_SPCLREG_DAIF = 0xDA11,
+ AARCH64_INSN_SPCLREG_NZCV = 0xDA10,
+ AARCH64_INSN_SPCLREG_FPCR = 0xDA20,
+ AARCH64_INSN_SPCLREG_DSPSR_EL0 = 0xDA28,
+ AARCH64_INSN_SPCLREG_DLR_EL0 = 0xDA29,
+ AARCH64_INSN_SPCLREG_SPSR_EL2 = 0xE200,
+ AARCH64_INSN_SPCLREG_ELR_EL2 = 0xE201,
+ AARCH64_INSN_SPCLREG_SP_EL1 = 0xE208,
+ AARCH64_INSN_SPCLREG_SPSR_INQ = 0xE218,
+ AARCH64_INSN_SPCLREG_SPSR_ABT = 0xE219,
+ AARCH64_INSN_SPCLREG_SPSR_UND = 0xE21A,
+ AARCH64_INSN_SPCLREG_SPSR_FIQ = 0xE21B,
+ AARCH64_INSN_SPCLREG_SPSR_EL3 = 0xF200,
+ AARCH64_INSN_SPCLREG_ELR_EL3 = 0xF201,
+ AARCH64_INSN_SPCLREG_SP_EL2 = 0xF210
+};
+
+enum aarch64_insn_variant {
+ AARCH64_INSN_VARIANT_32BIT,
+ AARCH64_INSN_VARIANT_64BIT
+};
+
+enum aarch64_insn_condition {
+ AARCH64_INSN_COND_EQ = 0x0, /* == */
+ AARCH64_INSN_COND_NE = 0x1, /* != */
+ AARCH64_INSN_COND_CS = 0x2, /* unsigned >= */
+ AARCH64_INSN_COND_CC = 0x3, /* unsigned < */
+ AARCH64_INSN_COND_MI = 0x4, /* < 0 */
+ AARCH64_INSN_COND_PL = 0x5, /* >= 0 */
+ AARCH64_INSN_COND_VS = 0x6, /* overflow */
+ AARCH64_INSN_COND_VC = 0x7, /* no overflow */
+ AARCH64_INSN_COND_HI = 0x8, /* unsigned > */
+ AARCH64_INSN_COND_LS = 0x9, /* unsigned <= */
+ AARCH64_INSN_COND_GE = 0xa, /* signed >= */
+ AARCH64_INSN_COND_LT = 0xb, /* signed < */
+ AARCH64_INSN_COND_GT = 0xc, /* signed > */
+ AARCH64_INSN_COND_LE = 0xd, /* signed <= */
+ AARCH64_INSN_COND_AL = 0xe, /* always */
+};
+
+enum aarch64_insn_branch_type {
+ AARCH64_INSN_BRANCH_NOLINK,
+ AARCH64_INSN_BRANCH_LINK,
+ AARCH64_INSN_BRANCH_RETURN,
+ AARCH64_INSN_BRANCH_COMP_ZERO,
+ AARCH64_INSN_BRANCH_COMP_NONZERO,
+};
+
+enum aarch64_insn_size_type {
+ AARCH64_INSN_SIZE_8,
+ AARCH64_INSN_SIZE_16,
+ AARCH64_INSN_SIZE_32,
+ AARCH64_INSN_SIZE_64,
+};
+
+enum aarch64_insn_ldst_type {
+ AARCH64_INSN_LDST_LOAD_REG_OFFSET,
+ AARCH64_INSN_LDST_STORE_REG_OFFSET,
+ AARCH64_INSN_LDST_LOAD_PAIR_PRE_INDEX,
+ AARCH64_INSN_LDST_STORE_PAIR_PRE_INDEX,
+ AARCH64_INSN_LDST_LOAD_PAIR_POST_INDEX,
+ AARCH64_INSN_LDST_STORE_PAIR_POST_INDEX,
+ AARCH64_INSN_LDST_LOAD_EX,
+ AARCH64_INSN_LDST_STORE_EX,
+};
+
+enum aarch64_insn_adsb_type {
+ AARCH64_INSN_ADSB_ADD,
+ AARCH64_INSN_ADSB_SUB,
+ AARCH64_INSN_ADSB_ADD_SETFLAGS,
+ AARCH64_INSN_ADSB_SUB_SETFLAGS
+};
+
+enum aarch64_insn_movewide_type {
+ AARCH64_INSN_MOVEWIDE_ZERO,
+ AARCH64_INSN_MOVEWIDE_KEEP,
+ AARCH64_INSN_MOVEWIDE_INVERSE
+};
+
+enum aarch64_insn_bitfield_type {
+ AARCH64_INSN_BITFIELD_MOVE,
+ AARCH64_INSN_BITFIELD_MOVE_UNSIGNED,
+ AARCH64_INSN_BITFIELD_MOVE_SIGNED
+};
+
+enum aarch64_insn_data1_type {
+ AARCH64_INSN_DATA1_REVERSE_16,
+ AARCH64_INSN_DATA1_REVERSE_32,
+ AARCH64_INSN_DATA1_REVERSE_64,
+};
+
+enum aarch64_insn_data2_type {
+ AARCH64_INSN_DATA2_UDIV,
+ AARCH64_INSN_DATA2_SDIV,
+ AARCH64_INSN_DATA2_LSLV,
+ AARCH64_INSN_DATA2_LSRV,
+ AARCH64_INSN_DATA2_ASRV,
+ AARCH64_INSN_DATA2_RORV,
+};
+
+enum aarch64_insn_data3_type {
+ AARCH64_INSN_DATA3_MADD,
+ AARCH64_INSN_DATA3_MSUB,
+};
+
+enum aarch64_insn_logic_type {
+ AARCH64_INSN_LOGIC_AND,
+ AARCH64_INSN_LOGIC_BIC,
+ AARCH64_INSN_LOGIC_ORR,
+ AARCH64_INSN_LOGIC_ORN,
+ AARCH64_INSN_LOGIC_EOR,
+ AARCH64_INSN_LOGIC_EON,
+ AARCH64_INSN_LOGIC_AND_SETFLAGS,
+ AARCH64_INSN_LOGIC_BIC_SETFLAGS
+};
+
+enum aarch64_insn_prfm_type {
+ AARCH64_INSN_PRFM_TYPE_PLD,
+ AARCH64_INSN_PRFM_TYPE_PLI,
+ AARCH64_INSN_PRFM_TYPE_PST,
+};
+
+enum aarch64_insn_prfm_target {
+ AARCH64_INSN_PRFM_TARGET_L1,
+ AARCH64_INSN_PRFM_TARGET_L2,
+ AARCH64_INSN_PRFM_TARGET_L3,
+};
+
+enum aarch64_insn_prfm_policy {
+ AARCH64_INSN_PRFM_POLICY_KEEP,
+ AARCH64_INSN_PRFM_POLICY_STRM,
+};
+
+enum aarch64_insn_adr_type {
+ AARCH64_INSN_ADR_TYPE_ADRP,
+ AARCH64_INSN_ADR_TYPE_ADR,
+};
+
+#define __AARCH64_INSN_FUNCS(abbr, mask, val) \
+static __always_inline bool aarch64_insn_is_##abbr(u32 code) \
+{ \
+ BUILD_BUG_ON(~(mask) & (val)); \
+ return (code & (mask)) == (val); \
+} \
+static __always_inline u32 aarch64_insn_get_##abbr##_value(void) \
+{ \
+ return (val); \
+}
+
+__AARCH64_INSN_FUNCS(adr, 0x9F000000, 0x10000000)
+__AARCH64_INSN_FUNCS(adrp, 0x9F000000, 0x90000000)
+__AARCH64_INSN_FUNCS(prfm, 0x3FC00000, 0x39800000)
+__AARCH64_INSN_FUNCS(prfm_lit, 0xFF000000, 0xD8000000)
+__AARCH64_INSN_FUNCS(store_imm, 0x3FC00000, 0x39000000)
+__AARCH64_INSN_FUNCS(load_imm, 0x3FC00000, 0x39400000)
+__AARCH64_INSN_FUNCS(store_pre, 0x3FE00C00, 0x38000C00)
+__AARCH64_INSN_FUNCS(load_pre, 0x3FE00C00, 0x38400C00)
+__AARCH64_INSN_FUNCS(store_post, 0x3FE00C00, 0x38000400)
+__AARCH64_INSN_FUNCS(load_post, 0x3FE00C00, 0x38400400)
+__AARCH64_INSN_FUNCS(str_reg, 0x3FE0EC00, 0x38206800)
+__AARCH64_INSN_FUNCS(ldadd, 0x3F20FC00, 0x38200000)
+__AARCH64_INSN_FUNCS(ldr_reg, 0x3FE0EC00, 0x38606800)
+__AARCH64_INSN_FUNCS(ldr_lit, 0xBF000000, 0x18000000)
+__AARCH64_INSN_FUNCS(ldrsw_lit, 0xFF000000, 0x98000000)
+__AARCH64_INSN_FUNCS(exclusive, 0x3F800000, 0x08000000)
+__AARCH64_INSN_FUNCS(load_ex, 0x3F400000, 0x08400000)
+__AARCH64_INSN_FUNCS(store_ex, 0x3F400000, 0x08000000)
+__AARCH64_INSN_FUNCS(stp, 0x7FC00000, 0x29000000)
+__AARCH64_INSN_FUNCS(ldp, 0x7FC00000, 0x29400000)
+__AARCH64_INSN_FUNCS(stp_post, 0x7FC00000, 0x28800000)
+__AARCH64_INSN_FUNCS(ldp_post, 0x7FC00000, 0x28C00000)
+__AARCH64_INSN_FUNCS(stp_pre, 0x7FC00000, 0x29800000)
+__AARCH64_INSN_FUNCS(ldp_pre, 0x7FC00000, 0x29C00000)
+__AARCH64_INSN_FUNCS(add_imm, 0x7F000000, 0x11000000)
+__AARCH64_INSN_FUNCS(adds_imm, 0x7F000000, 0x31000000)
+__AARCH64_INSN_FUNCS(sub_imm, 0x7F000000, 0x51000000)
+__AARCH64_INSN_FUNCS(subs_imm, 0x7F000000, 0x71000000)
+__AARCH64_INSN_FUNCS(movn, 0x7F800000, 0x12800000)
+__AARCH64_INSN_FUNCS(sbfm, 0x7F800000, 0x13000000)
+__AARCH64_INSN_FUNCS(bfm, 0x7F800000, 0x33000000)
+__AARCH64_INSN_FUNCS(movz, 0x7F800000, 0x52800000)
+__AARCH64_INSN_FUNCS(ubfm, 0x7F800000, 0x53000000)
+__AARCH64_INSN_FUNCS(movk, 0x7F800000, 0x72800000)
+__AARCH64_INSN_FUNCS(add, 0x7F200000, 0x0B000000)
+__AARCH64_INSN_FUNCS(adds, 0x7F200000, 0x2B000000)
+__AARCH64_INSN_FUNCS(sub, 0x7F200000, 0x4B000000)
+__AARCH64_INSN_FUNCS(subs, 0x7F200000, 0x6B000000)
+__AARCH64_INSN_FUNCS(madd, 0x7FE08000, 0x1B000000)
+__AARCH64_INSN_FUNCS(msub, 0x7FE08000, 0x1B008000)
+__AARCH64_INSN_FUNCS(udiv, 0x7FE0FC00, 0x1AC00800)
+__AARCH64_INSN_FUNCS(sdiv, 0x7FE0FC00, 0x1AC00C00)
+__AARCH64_INSN_FUNCS(lslv, 0x7FE0FC00, 0x1AC02000)
+__AARCH64_INSN_FUNCS(lsrv, 0x7FE0FC00, 0x1AC02400)
+__AARCH64_INSN_FUNCS(asrv, 0x7FE0FC00, 0x1AC02800)
+__AARCH64_INSN_FUNCS(rorv, 0x7FE0FC00, 0x1AC02C00)
+__AARCH64_INSN_FUNCS(rev16, 0x7FFFFC00, 0x5AC00400)
+__AARCH64_INSN_FUNCS(rev32, 0x7FFFFC00, 0x5AC00800)
+__AARCH64_INSN_FUNCS(rev64, 0x7FFFFC00, 0x5AC00C00)
+__AARCH64_INSN_FUNCS(and, 0x7F200000, 0x0A000000)
+__AARCH64_INSN_FUNCS(bic, 0x7F200000, 0x0A200000)
+__AARCH64_INSN_FUNCS(orr, 0x7F200000, 0x2A000000)
+__AARCH64_INSN_FUNCS(mov_reg, 0x7FE0FFE0, 0x2A0003E0)
+__AARCH64_INSN_FUNCS(orn, 0x7F200000, 0x2A200000)
+__AARCH64_INSN_FUNCS(eor, 0x7F200000, 0x4A000000)
+__AARCH64_INSN_FUNCS(eon, 0x7F200000, 0x4A200000)
+__AARCH64_INSN_FUNCS(ands, 0x7F200000, 0x6A000000)
+__AARCH64_INSN_FUNCS(bics, 0x7F200000, 0x6A200000)
+__AARCH64_INSN_FUNCS(and_imm, 0x7F800000, 0x12000000)
+__AARCH64_INSN_FUNCS(orr_imm, 0x7F800000, 0x32000000)
+__AARCH64_INSN_FUNCS(eor_imm, 0x7F800000, 0x52000000)
+__AARCH64_INSN_FUNCS(ands_imm, 0x7F800000, 0x72000000)
+__AARCH64_INSN_FUNCS(extr, 0x7FA00000, 0x13800000)
+__AARCH64_INSN_FUNCS(b, 0xFC000000, 0x14000000)
+__AARCH64_INSN_FUNCS(bl, 0xFC000000, 0x94000000)
+__AARCH64_INSN_FUNCS(cbz, 0x7F000000, 0x34000000)
+__AARCH64_INSN_FUNCS(cbnz, 0x7F000000, 0x35000000)
+__AARCH64_INSN_FUNCS(tbz, 0x7F000000, 0x36000000)
+__AARCH64_INSN_FUNCS(tbnz, 0x7F000000, 0x37000000)
+__AARCH64_INSN_FUNCS(bcond, 0xFF000010, 0x54000000)
+__AARCH64_INSN_FUNCS(svc, 0xFFE0001F, 0xD4000001)
+__AARCH64_INSN_FUNCS(hvc, 0xFFE0001F, 0xD4000002)
+__AARCH64_INSN_FUNCS(smc, 0xFFE0001F, 0xD4000003)
+__AARCH64_INSN_FUNCS(brk, 0xFFE0001F, 0xD4200000)
+__AARCH64_INSN_FUNCS(exception, 0xFF000000, 0xD4000000)
+__AARCH64_INSN_FUNCS(hint, 0xFFFFF01F, 0xD503201F)
+__AARCH64_INSN_FUNCS(br, 0xFFFFFC1F, 0xD61F0000)
+__AARCH64_INSN_FUNCS(br_auth, 0xFEFFF800, 0xD61F0800)
+__AARCH64_INSN_FUNCS(blr, 0xFFFFFC1F, 0xD63F0000)
+__AARCH64_INSN_FUNCS(blr_auth, 0xFEFFF800, 0xD63F0800)
+__AARCH64_INSN_FUNCS(ret, 0xFFFFFC1F, 0xD65F0000)
+__AARCH64_INSN_FUNCS(ret_auth, 0xFFFFFBFF, 0xD65F0BFF)
+__AARCH64_INSN_FUNCS(eret, 0xFFFFFFFF, 0xD69F03E0)
+__AARCH64_INSN_FUNCS(eret_auth, 0xFFFFFBFF, 0xD69F0BFF)
+__AARCH64_INSN_FUNCS(mrs, 0xFFF00000, 0xD5300000)
+__AARCH64_INSN_FUNCS(msr_imm, 0xFFF8F01F, 0xD500401F)
+__AARCH64_INSN_FUNCS(msr_reg, 0xFFF00000, 0xD5100000)
+__AARCH64_INSN_FUNCS(dmb, 0xFFFFF0FF, 0xD50330BF)
+__AARCH64_INSN_FUNCS(dsb_base, 0xFFFFF0FF, 0xD503309F)
+__AARCH64_INSN_FUNCS(dsb_nxs, 0xFFFFF3FF, 0xD503323F)
+__AARCH64_INSN_FUNCS(isb, 0xFFFFF0FF, 0xD50330DF)
+__AARCH64_INSN_FUNCS(sb, 0xFFFFFFFF, 0xD50330FF)
+__AARCH64_INSN_FUNCS(clrex, 0xFFFFF0FF, 0xD503305F)
+__AARCH64_INSN_FUNCS(ssbb, 0xFFFFFFFF, 0xD503309F)
+__AARCH64_INSN_FUNCS(pssbb, 0xFFFFFFFF, 0xD503349F)
+
+#undef __AARCH64_INSN_FUNCS
+
+bool aarch64_insn_is_steppable_hint(u32 insn);
+bool aarch64_insn_is_branch_imm(u32 insn);
+
+static inline bool aarch64_insn_is_adr_adrp(u32 insn)
+{
+ return aarch64_insn_is_adr(insn) || aarch64_insn_is_adrp(insn);
+}
+
+static inline bool aarch64_insn_is_dsb(u32 insn)
+{
+ return (aarch64_insn_is_dsb_base(insn) && (insn & 0xb00)) ||
+ aarch64_insn_is_dsb_nxs(insn);
+}
+
+static inline bool aarch64_insn_is_barrier(u32 insn)
+{
+ return aarch64_insn_is_dmb(insn) || aarch64_insn_is_dsb(insn) ||
+ aarch64_insn_is_isb(insn) || aarch64_insn_is_sb(insn) ||
+ aarch64_insn_is_clrex(insn) || aarch64_insn_is_ssbb(insn) ||
+ aarch64_insn_is_pssbb(insn);
+}
+
+static inline bool aarch64_insn_is_store_single(u32 insn)
+{
+ return aarch64_insn_is_store_imm(insn) ||
+ aarch64_insn_is_store_pre(insn) ||
+ aarch64_insn_is_store_post(insn);
+}
+
+static inline bool aarch64_insn_is_store_pair(u32 insn)
+{
+ return aarch64_insn_is_stp(insn) ||
+ aarch64_insn_is_stp_pre(insn) ||
+ aarch64_insn_is_stp_post(insn);
+}
+
+static inline bool aarch64_insn_is_load_single(u32 insn)
+{
+ return aarch64_insn_is_load_imm(insn) ||
+ aarch64_insn_is_load_pre(insn) ||
+ aarch64_insn_is_load_post(insn);
+}
+
+static inline bool aarch64_insn_is_load_pair(u32 insn)
+{
+ return aarch64_insn_is_ldp(insn) ||
+ aarch64_insn_is_ldp_pre(insn) ||
+ aarch64_insn_is_ldp_post(insn);
+}
+
+enum aarch64_insn_encoding_class aarch64_get_insn_class(u32 insn);
+bool aarch64_insn_uses_literal(u32 insn);
+bool aarch64_insn_is_branch(u32 insn);
+u64 aarch64_insn_decode_immediate(enum aarch64_insn_imm_type type, u32 insn);
+u32 aarch64_insn_decode_register(enum aarch64_insn_register_type type,
+ u32 insn);
+u32 aarch64_insn_gen_hint(enum aarch64_insn_hint_cr_op op);
+u32 aarch64_insn_gen_nop(void);
+u32 aarch64_insn_gen_branch_reg(enum aarch64_insn_register reg,
+ enum aarch64_insn_branch_type type);
+s32 aarch64_get_branch_offset(u32 insn);
+s32 aarch64_insn_adrp_get_offset(u32 insn);
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_INSN_H */
diff --git a/tools/arch/arm64/lib/insn.c b/tools/arch/arm64/lib/insn.c
new file mode 100644
index 000000000000..b0cc984fcf6a
--- /dev/null
+++ b/tools/arch/arm64/lib/insn.c
@@ -0,0 +1,335 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2013 Huawei Ltd.
+ * Author: Jiang Liu <[email protected]>
+ *
+ * Copyright (C) 2014-2016 Zi Shen Lim <[email protected]>
+ */
+#include <linux/bitops.h>
+#include <linux/sizes.h>
+#include <linux/types.h>
+
+#include <asm/errno.h>
+#include <asm/insn.h>
+
+#define AARCH64_DECODE_FAULT 0xFFFFFFFF
+
+#define AARCH64_INSN_SF_BIT BIT(31)
+#define AARCH64_INSN_N_BIT BIT(22)
+#define AARCH64_INSN_LSL_12 BIT(22)
+
+static const int aarch64_insn_encoding_class[] = {
+ AARCH64_INSN_CLS_UNKNOWN,
+ AARCH64_INSN_CLS_UNKNOWN,
+ AARCH64_INSN_CLS_SVE,
+ AARCH64_INSN_CLS_UNKNOWN,
+ AARCH64_INSN_CLS_LDST,
+ AARCH64_INSN_CLS_DP_REG,
+ AARCH64_INSN_CLS_LDST,
+ AARCH64_INSN_CLS_DP_FPSIMD,
+ AARCH64_INSN_CLS_DP_IMM,
+ AARCH64_INSN_CLS_DP_IMM,
+ AARCH64_INSN_CLS_BR_SYS,
+ AARCH64_INSN_CLS_BR_SYS,
+ AARCH64_INSN_CLS_LDST,
+ AARCH64_INSN_CLS_DP_REG,
+ AARCH64_INSN_CLS_LDST,
+ AARCH64_INSN_CLS_DP_FPSIMD,
+};
+
+enum aarch64_insn_encoding_class aarch64_get_insn_class(u32 insn)
+{
+ return aarch64_insn_encoding_class[(insn >> 25) & 0xf];
+}
+
+bool aarch64_insn_is_steppable_hint(u32 insn)
+{
+ if (!aarch64_insn_is_hint(insn))
+ return false;
+
+ switch (insn & 0xFE0) {
+ case AARCH64_INSN_HINT_XPACLRI:
+ case AARCH64_INSN_HINT_PACIA_1716:
+ case AARCH64_INSN_HINT_PACIB_1716:
+ case AARCH64_INSN_HINT_PACIAZ:
+ case AARCH64_INSN_HINT_PACIASP:
+ case AARCH64_INSN_HINT_PACIBZ:
+ case AARCH64_INSN_HINT_PACIBSP:
+ case AARCH64_INSN_HINT_BTI:
+ case AARCH64_INSN_HINT_BTIC:
+ case AARCH64_INSN_HINT_BTIJ:
+ case AARCH64_INSN_HINT_BTIJC:
+ case AARCH64_INSN_HINT_NOP:
+ return true;
+ default:
+ return false;
+ }
+}
+
+bool aarch64_insn_is_branch_imm(u32 insn)
+{
+ return (aarch64_insn_is_b(insn) || aarch64_insn_is_bl(insn) ||
+ aarch64_insn_is_tbz(insn) || aarch64_insn_is_tbnz(insn) ||
+ aarch64_insn_is_cbz(insn) || aarch64_insn_is_cbnz(insn) ||
+ aarch64_insn_is_bcond(insn));
+}
+
+bool aarch64_insn_uses_literal(u32 insn)
+{
+ /* ldr/ldrsw (literal), prfm */
+
+ return aarch64_insn_is_ldr_lit(insn) ||
+ aarch64_insn_is_ldrsw_lit(insn) ||
+ aarch64_insn_is_adr_adrp(insn) ||
+ aarch64_insn_is_prfm_lit(insn);
+}
+
+bool aarch64_insn_is_branch(u32 insn)
+{
+ /* b, bl, cb*, tb*, ret*, b.cond, br*, blr* */
+
+ return aarch64_insn_is_b(insn) ||
+ aarch64_insn_is_bl(insn) ||
+ aarch64_insn_is_cbz(insn) ||
+ aarch64_insn_is_cbnz(insn) ||
+ aarch64_insn_is_tbz(insn) ||
+ aarch64_insn_is_tbnz(insn) ||
+ aarch64_insn_is_ret(insn) ||
+ aarch64_insn_is_ret_auth(insn) ||
+ aarch64_insn_is_br(insn) ||
+ aarch64_insn_is_br_auth(insn) ||
+ aarch64_insn_is_blr(insn) ||
+ aarch64_insn_is_blr_auth(insn) ||
+ aarch64_insn_is_bcond(insn);
+}
+
+static int aarch64_get_imm_shift_mask(enum aarch64_insn_imm_type type,
+ u32 *maskp, int *shiftp)
+{
+ u32 mask;
+ int shift;
+
+ switch (type) {
+ case AARCH64_INSN_IMM_26:
+ mask = BIT(26) - 1;
+ shift = 0;
+ break;
+ case AARCH64_INSN_IMM_19:
+ mask = BIT(19) - 1;
+ shift = 5;
+ break;
+ case AARCH64_INSN_IMM_16:
+ mask = BIT(16) - 1;
+ shift = 5;
+ break;
+ case AARCH64_INSN_IMM_14:
+ mask = BIT(14) - 1;
+ shift = 5;
+ break;
+ case AARCH64_INSN_IMM_12:
+ mask = BIT(12) - 1;
+ shift = 10;
+ break;
+ case AARCH64_INSN_IMM_9:
+ mask = BIT(9) - 1;
+ shift = 12;
+ break;
+ case AARCH64_INSN_IMM_7:
+ mask = BIT(7) - 1;
+ shift = 15;
+ break;
+ case AARCH64_INSN_IMM_6:
+ case AARCH64_INSN_IMM_S:
+ mask = BIT(6) - 1;
+ shift = 10;
+ break;
+ case AARCH64_INSN_IMM_R:
+ mask = BIT(6) - 1;
+ shift = 16;
+ break;
+ case AARCH64_INSN_IMM_N:
+ mask = 1;
+ shift = 22;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ *maskp = mask;
+ *shiftp = shift;
+
+ return 0;
+}
+
+#define ADR_IMM_HILOSPLIT 2
+#define ADR_IMM_SIZE SZ_2M
+#define ADR_IMM_LOMASK ((1 << ADR_IMM_HILOSPLIT) - 1)
+#define ADR_IMM_HIMASK ((ADR_IMM_SIZE >> ADR_IMM_HILOSPLIT) - 1)
+#define ADR_IMM_LOSHIFT 29
+#define ADR_IMM_HISHIFT 5
+
+u64 aarch64_insn_decode_immediate(enum aarch64_insn_imm_type type, u32 insn)
+{
+ u32 immlo, immhi, mask;
+ int shift;
+
+ switch (type) {
+ case AARCH64_INSN_IMM_ADR:
+ shift = 0;
+ immlo = (insn >> ADR_IMM_LOSHIFT) & ADR_IMM_LOMASK;
+ immhi = (insn >> ADR_IMM_HISHIFT) & ADR_IMM_HIMASK;
+ insn = (immhi << ADR_IMM_HILOSPLIT) | immlo;
+ mask = ADR_IMM_SIZE - 1;
+ break;
+ default:
+ if (aarch64_get_imm_shift_mask(type, &mask, &shift) < 0) {
+ WARN("aarch64_insn_decode_immediate: unknown immediate encoding %d\n",
+ type);
+ return AARCH64_DECODE_FAULT;
+ }
+ }
+
+ return (insn >> shift) & mask;
+}
+
+u32 aarch64_insn_decode_register(enum aarch64_insn_register_type type,
+ u32 insn)
+{
+ int shift;
+
+ switch (type) {
+ case AARCH64_INSN_REGTYPE_RT:
+ case AARCH64_INSN_REGTYPE_RD:
+ shift = 0;
+ break;
+ case AARCH64_INSN_REGTYPE_RN:
+ shift = 5;
+ break;
+ case AARCH64_INSN_REGTYPE_RT2:
+ case AARCH64_INSN_REGTYPE_RA:
+ shift = 10;
+ break;
+ case AARCH64_INSN_REGTYPE_RM:
+ shift = 16;
+ break;
+ default:
+ WARN("%s: unknown register type encoding %d\n", __func__,
+ type);
+ return AARCH64_DECODE_FAULT;
+ }
+
+ return (insn >> shift) & GENMASK(4, 0);
+}
+
+u32 aarch64_insn_gen_hint(enum aarch64_insn_hint_cr_op op)
+{
+ return aarch64_insn_get_hint_value() | op;
+}
+
+u32 aarch64_insn_gen_nop(void)
+{
+ return aarch64_insn_gen_hint(AARCH64_INSN_HINT_NOP);
+}
+
+static u32 aarch64_insn_encode_register(enum aarch64_insn_register_type type,
+ u32 insn,
+ enum aarch64_insn_register reg)
+{
+ int shift;
+
+ if (insn == AARCH64_DECODE_FAULT)
+ return AARCH64_DECODE_FAULT;
+
+ if (reg < AARCH64_INSN_REG_0 || reg > AARCH64_INSN_REG_SP) {
+ WARN("%s: unknown register encoding %d\n", __func__, reg);
+ return AARCH64_DECODE_FAULT;
+ }
+
+ switch (type) {
+ case AARCH64_INSN_REGTYPE_RT:
+ case AARCH64_INSN_REGTYPE_RD:
+ shift = 0;
+ break;
+ case AARCH64_INSN_REGTYPE_RN:
+ shift = 5;
+ break;
+ case AARCH64_INSN_REGTYPE_RT2:
+ case AARCH64_INSN_REGTYPE_RA:
+ shift = 10;
+ break;
+ case AARCH64_INSN_REGTYPE_RM:
+ case AARCH64_INSN_REGTYPE_RS:
+ shift = 16;
+ break;
+ default:
+ WARN("%s: unknown register type encoding %d\n", __func__,
+ type);
+ return AARCH64_DECODE_FAULT;
+ }
+
+ insn &= ~(GENMASK(4, 0) << shift);
+ insn |= reg << shift;
+
+ return insn;
+}
+
+u32 aarch64_insn_gen_branch_reg(enum aarch64_insn_register reg,
+ enum aarch64_insn_branch_type type)
+{
+ u32 insn;
+
+ switch (type) {
+ case AARCH64_INSN_BRANCH_NOLINK:
+ insn = aarch64_insn_get_br_value();
+ break;
+ case AARCH64_INSN_BRANCH_LINK:
+ insn = aarch64_insn_get_blr_value();
+ break;
+ case AARCH64_INSN_BRANCH_RETURN:
+ insn = aarch64_insn_get_ret_value();
+ break;
+ default:
+ WARN("%s: unknown branch encoding %d\n", __func__, type);
+ return AARCH64_DECODE_FAULT;
+ }
+
+ return aarch64_insn_encode_register(AARCH64_INSN_REGTYPE_RN, insn, reg);
+}
+
+/*
+ * Decode the imm field of a branch, and return the byte offset as a
+ * signed value (so it can be used when computing a new branch
+ * target).
+ */
+s32 aarch64_get_branch_offset(u32 insn)
+{
+ s32 imm;
+
+ if (aarch64_insn_is_b(insn) || aarch64_insn_is_bl(insn)) {
+ imm = aarch64_insn_decode_immediate(AARCH64_INSN_IMM_26, insn);
+ return (imm << 6) >> 4;
+ }
+
+ if (aarch64_insn_is_cbz(insn) || aarch64_insn_is_cbnz(insn) ||
+ aarch64_insn_is_bcond(insn)) {
+ imm = aarch64_insn_decode_immediate(AARCH64_INSN_IMM_19, insn);
+ return (imm << 13) >> 11;
+ }
+
+ if (aarch64_insn_is_tbz(insn) || aarch64_insn_is_tbnz(insn)) {
+ imm = aarch64_insn_decode_immediate(AARCH64_INSN_IMM_14, insn);
+ return (imm << 18) >> 16;
+ }
+
+ WARN("Unhandled instruction %x", insn);
+ return AARCH64_DECODE_FAULT;
+}
+
+s32 aarch64_insn_adrp_get_offset(u32 insn)
+{
+ if (!aarch64_insn_is_adrp(insn)) {
+ WARN("Unhandled instruction %x", insn);
+ return AARCH64_DECODE_FAULT;
+ }
+ return aarch64_insn_decode_immediate(AARCH64_INSN_IMM_ADR, insn) << 12;
+}
--
2.17.1
WARN* and BUG* both use brk #0x800 opcodes and the distinction is
provided by the contents of the bug table. This table is not accessible
to objtool, so add an annotation to WARN* macros to let objtool know
that brk handler will return an resume the execution of the instructions
following the WARN's brk.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/include/asm/bug.h | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/bug.h b/arch/arm64/include/asm/bug.h
index 28be048db3f6..9917429971d4 100644
--- a/arch/arm64/include/asm/bug.h
+++ b/arch/arm64/include/asm/bug.h
@@ -19,7 +19,11 @@
unreachable(); \
} while (0)
-#define __WARN_FLAGS(flags) __BUG_FLAGS(BUGFLAG_WARNING|(flags))
+#define __WARN_FLAGS(flags) \
+do { \
+ __BUG_FLAGS(BUGFLAG_WARNING|(flags)); \
+ annotate_reachable(); \
+} while (0)
#define HAVE_ARCH_BUG
--
2.17.1
Decode ERET, BRK and NOPs
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/objtool/arch/arm64/decode.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/decode.c
index 40ada17d0842..19840862f3aa 100644
--- a/tools/objtool/arch/arm64/decode.c
+++ b/tools/objtool/arch/arm64/decode.c
@@ -231,6 +231,14 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
/* Remaining branch opcodes are conditional */
*type = INSN_JUMP_CONDITIONAL;
*immediate = aarch64_get_branch_offset(insn);
+ } else if (aarch64_insn_is_eret(insn)) {
+ *type = INSN_CONTEXT_SWITCH;
+ } else if (aarch64_insn_is_hint(insn) ||
+ aarch64_insn_is_barrier(insn)) {
+ *type = INSN_NOP;
+ } else if (aarch64_insn_is_brk(insn)) {
+ *type = INSN_BUG;
+ *immediate = aarch64_insn_decode_immediate(AARCH64_INSN_IMM_16, insn);
}
break;
default:
--
2.17.1
When taking an exception/interrupt, add UNWIND_HINT_REGS to indicate
from which point the pt_regs is on stack.
Whether returning to userland or creating a new task, sp is
pointing to a pt_regs frame, add UNWIND_HINT_REGS after that.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/kernel/entry.S | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 3bd11101e79d..bbc440379304 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -120,6 +120,7 @@
alternative_cb spectre_v4_patch_fw_mitigation_enable
b .L__asm_ssbd_skip\@ // Patched to NOP
alternative_cb_end
+ UNWIND_HINT_REGS
ldr_this_cpu \tmp2, arm64_ssbd_callback_required, \tmp1
cbz \tmp2, .L__asm_ssbd_skip\@
ldr \tmp2, [tsk, #TSK_TI_FLAGS]
@@ -218,6 +219,7 @@ alternative_cb_end
stp x24, x25, [sp, #16 * 12]
stp x26, x27, [sp, #16 * 13]
stp x28, x29, [sp, #16 * 14]
+ UNWIND_HINT_REGS
.if \el == 0
clear_gp_regs
@@ -601,6 +603,7 @@ SYM_CODE_START_LOCAL(ret_to_kernel)
SYM_CODE_END(ret_to_kernel)
SYM_CODE_START_LOCAL(ret_to_user)
+ UNWIND_HINT_REGS
ldr x19, [tsk, #TSK_TI_FLAGS] // re-check for single-step
enable_step_tsk x19, x2
#ifdef CONFIG_GCC_PLUGIN_STACKLEAK
@@ -872,6 +875,7 @@ NOKPROBE(cpu_switch_to)
* This is how we return from a fork.
*/
SYM_CODE_START(ret_from_fork)
+ UNWIND_HINT_REGS
bl schedule_tail
cbz x19, 1f // not a kernel thread
mov x0, x20
--
2.17.1
Provide unwind hint defines for arm64 and objtool hint decoding.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/include/asm/unwind_hints.h | 26 +++++++++++++++++++++
tools/arch/arm64/include/asm/unwind_hints.h | 26 +++++++++++++++++++++
tools/objtool/arch/arm64/decode.c | 14 ++++++++++-
3 files changed, 65 insertions(+), 1 deletion(-)
create mode 100644 arch/arm64/include/asm/unwind_hints.h
create mode 100644 tools/arch/arm64/include/asm/unwind_hints.h
diff --git a/arch/arm64/include/asm/unwind_hints.h b/arch/arm64/include/asm/unwind_hints.h
new file mode 100644
index 000000000000..ffee9472b30d
--- /dev/null
+++ b/arch/arm64/include/asm/unwind_hints.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef __ASM_UNWIND_HINTS_H
+#define __ASM_UNWIND_HINTS_H
+
+#include <linux/objtool.h>
+
+#include "orc_types.h"
+
+#ifdef __ASSEMBLY__
+
+.macro UNWIND_HINT_EMPTY
+ UNWIND_HINT sp_reg=ORC_REG_UNDEFINED type=UNWIND_HINT_TYPE_CALL end=1
+.endm
+
+.macro UNWIND_HINT_FUNC sp_offset=0
+ UNWIND_HINT sp_reg=ORC_REG_SP sp_offset=\sp_offset type=UNWIND_HINT_TYPE_CALL
+.endm
+
+.macro UNWIND_HINT_REGS base=ORC_REG_SP offset=0
+ UNWIND_HINT sp_reg=\base sp_offset=\offset type=UNWIND_HINT_TYPE_REGS
+.endm
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_UNWIND_HINTS_H */
diff --git a/tools/arch/arm64/include/asm/unwind_hints.h b/tools/arch/arm64/include/asm/unwind_hints.h
new file mode 100644
index 000000000000..ffee9472b30d
--- /dev/null
+++ b/tools/arch/arm64/include/asm/unwind_hints.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef __ASM_UNWIND_HINTS_H
+#define __ASM_UNWIND_HINTS_H
+
+#include <linux/objtool.h>
+
+#include "orc_types.h"
+
+#ifdef __ASSEMBLY__
+
+.macro UNWIND_HINT_EMPTY
+ UNWIND_HINT sp_reg=ORC_REG_UNDEFINED type=UNWIND_HINT_TYPE_CALL end=1
+.endm
+
+.macro UNWIND_HINT_FUNC sp_offset=0
+ UNWIND_HINT sp_reg=ORC_REG_SP sp_offset=\sp_offset type=UNWIND_HINT_TYPE_CALL
+.endm
+
+.macro UNWIND_HINT_REGS base=ORC_REG_SP offset=0
+ UNWIND_HINT sp_reg=\base sp_offset=\offset type=UNWIND_HINT_TYPE_REGS
+.endm
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_UNWIND_HINTS_H */
diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/decode.c
index 771d37d872c8..21a7f8315465 100644
--- a/tools/objtool/arch/arm64/decode.c
+++ b/tools/objtool/arch/arm64/decode.c
@@ -5,6 +5,7 @@
#include <stdint.h>
#include <asm/insn.h>
+#include <asm/unwind_hints.h>
#include <objtool/check.h>
#include <objtool/arch.h>
@@ -176,7 +177,18 @@ static int is_arm64(const struct elf *elf)
int arch_decode_hint_reg(u8 sp_reg, int *base)
{
- return -1;
+ switch (sp_reg) {
+ case ORC_REG_UNDEFINED:
+ *base = CFI_UNDEFINED;
+ break;
+ case ORC_REG_SP:
+ *base = CFI_SP;
+ break;
+ default:
+ return -1;
+ }
+
+ return 0;
}
static inline void make_add_op(enum aarch64_insn_register dest,
--
2.17.1
Based on get_alt_insn() in arch/arm64/kernel/alternative.c, arm64
alternative code adapts offsets for static branches and adrp
instructions.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/objtool/arch/arm64/special.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/tools/objtool/arch/arm64/special.c b/tools/objtool/arch/arm64/special.c
index 45f283283091..a70b91e8bd7d 100644
--- a/tools/objtool/arch/arm64/special.c
+++ b/tools/objtool/arch/arm64/special.c
@@ -10,7 +10,11 @@ bool arch_support_alt_relocation(struct special_alt *special_alt,
struct instruction *insn,
struct reloc *reloc)
{
- return false;
+ u32 opcode = *(u32 *)(insn->sec->data->d_buf + insn->offset);
+
+ return aarch64_insn_is_branch_imm(opcode) ||
+ aarch64_insn_is_adrp(opcode) ||
+ !aarch64_insn_uses_literal(opcode);
}
--
2.17.1
The compiler can generate some '0x0' words in code sections to pad the
end of functions. Also some pesudo-instructions can generate data in
code sections. Mark them as INSN_NOP.
If there are other undecoded instructions, just record and remove them
from validation list.
These doesn't influence check and orc generation because these undecoded
instructions also won't be excuted.
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/objtool/arch/arm64/decode.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/tools/objtool/arch/arm64/decode.c b/tools/objtool/arch/arm64/decode.c
index 30300d05c8f3..771d37d872c8 100644
--- a/tools/objtool/arch/arm64/decode.c
+++ b/tools/objtool/arch/arm64/decode.c
@@ -347,8 +347,14 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
switch (aarch64_get_insn_class(insn)) {
case AARCH64_INSN_CLS_UNKNOWN:
- WARN("can't decode instruction at %s:0x%lx", sec->name, offset);
- return -1;
+ if (insn == 0x0) {
+ *type = INSN_NOP;
+ } else {
+ WARN("undecoded insn at %s:0x%lx", sec->name, offset);
+ return record_invalid_insn(sec, offset);
+ }
+
+ break;
case AARCH64_INSN_CLS_DP_IMM:
/* Mov register to and from SP are aliases of add_imm */
if (aarch64_insn_is_add_imm(insn) ||
--
2.17.1
Using unreachable() at default of switch generates an extra branch at
end of the function, and compiler won't generate a ret to close this
branch because it knows it's unreachable.
If there's no instruction in this branch, compiler will generate a NOP,
And it will confuse objtool to warn this NOP as a fall through branch.
In fact these branches are actually unreachable, so we can replace
unreachable() with returning a -EINVAL value.
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/kvm/hyp/vgic-v3-sr.c | 7 +++----
drivers/irqchip/irq-gic-v3.c | 2 +-
2 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index 4fb419f7b8b6..f3cee92c3038 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -6,7 +6,6 @@
#include <hyp/adjust_pc.h>
-#include <linux/compiler.h>
#include <linux/irqchip/arm-gic-v3.h>
#include <linux/kvm_host.h>
@@ -55,7 +54,7 @@ static u64 __gic_v3_get_lr(unsigned int lr)
return read_gicreg(ICH_LR15_EL2);
}
- unreachable();
+ return -EINVAL;
}
static void __gic_v3_set_lr(u64 val, int lr)
@@ -166,7 +165,7 @@ static u32 __vgic_v3_read_ap0rn(int n)
val = read_gicreg(ICH_AP0R3_EL2);
break;
default:
- unreachable();
+ val = -EINVAL;
}
return val;
@@ -190,7 +189,7 @@ static u32 __vgic_v3_read_ap1rn(int n)
val = read_gicreg(ICH_AP1R3_EL2);
break;
default:
- unreachable();
+ val = -EINVAL;
}
return val;
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index b252d5534547..2ef98e32d257 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -475,7 +475,7 @@ static u32 __gic_get_ppi_index(irq_hw_number_t hwirq)
case EPPI_RANGE:
return hwirq - EPPI_BASE_INTID + 16;
default:
- unreachable();
+ return -EINVAL;
}
}
--
2.17.1
qcom_link_stack_sanitisation() repeatly calls itself, but we can't
mark the asm code as intra-call so it should be marked as non_standard.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
arch/arm64/kernel/proton-pack.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/arm64/kernel/proton-pack.c b/arch/arm64/kernel/proton-pack.c
index 40be3a7c2c53..9439e62d4b57 100644
--- a/arch/arm64/kernel/proton-pack.c
+++ b/arch/arm64/kernel/proton-pack.c
@@ -22,6 +22,7 @@
#include <linux/cpu.h>
#include <linux/device.h>
#include <linux/nospec.h>
+#include <linux/objtool.h>
#include <linux/prctl.h>
#include <linux/sched/task_stack.h>
@@ -257,6 +258,7 @@ static noinstr void qcom_link_stack_sanitisation(void)
"mov x30, %0 \n"
: "=&r" (tmp));
}
+STACK_FRAME_NON_STANDARD(qcom_link_stack_sanitisation);
static bp_hardening_cb_t spectre_v2_get_sw_mitigation_cb(void)
{
--
2.17.1
Arm64 assembly code can mix code and data in text sections through the use
of SYM_DATA_*() macros. Skip the content of these symbols when decoding
instructions of text sections.
Signed-off-by: Julien Thierry <[email protected]>
Signed-off-by: Chen Zhongjin <[email protected]>
---
tools/objtool/check.c | 14 ++++++++++++--
tools/objtool/elf.c | 14 ++++++++++++++
tools/objtool/include/objtool/elf.h | 1 +
3 files changed, 27 insertions(+), 2 deletions(-)
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index c0feb6db7c6d..f19ea3fc1e6b 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -359,7 +359,7 @@ static int decode_instructions(struct objtool_file *file)
{
struct section *sec;
struct symbol *func;
- unsigned long offset;
+ unsigned long offset, next_offset;
struct instruction *insn;
int ret;
@@ -377,7 +377,15 @@ static int decode_instructions(struct objtool_file *file)
!strcmp(sec->name, ".entry.text"))
sec->noinstr = true;
- for (offset = 0; offset < sec->sh.sh_size; offset += insn->len) {
+ for (offset = 0; offset < sec->sh.sh_size; offset = next_offset) {
+ struct symbol *obj_sym = find_object_containing(sec, offset);
+
+ if (obj_sym) {
+ /* This is data in the middle of text section, skip it */
+ next_offset = obj_sym->offset + obj_sym->len;
+ continue;
+ }
+
insn = malloc(sizeof(*insn));
if (!insn) {
WARN("malloc failed");
@@ -410,6 +418,8 @@ static int decode_instructions(struct objtool_file *file)
hash_add(file->insn_hash, &insn->hash, sec_offset_hash(sec, insn->offset));
list_add_tail(&insn->list, &file->insn_list);
nr_insns++;
+
+ next_offset = offset + insn->len;
}
list_for_each_entry(func, &sec->symbol_list, list) {
diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
index c25e957c1e52..d7304e39f6b6 100644
--- a/tools/objtool/elf.c
+++ b/tools/objtool/elf.c
@@ -236,6 +236,20 @@ struct symbol *find_func_containing(struct section *sec, unsigned long offset)
return NULL;
}
+struct symbol *find_object_containing(const struct section *sec, unsigned long offset)
+{
+ struct rb_node *node;
+
+ rb_for_each(node, &offset, &sec->symbol_tree, symbol_by_offset) {
+ struct symbol *s = rb_entry(node, struct symbol, node);
+
+ if (s->type == STT_OBJECT)
+ return s;
+ }
+
+ return NULL;
+}
+
struct symbol *find_symbol_by_name(const struct elf *elf, const char *name)
{
struct symbol *sym;
diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
index adebfbc2b518..8b0896935af1 100644
--- a/tools/objtool/include/objtool/elf.h
+++ b/tools/objtool/include/objtool/elf.h
@@ -167,6 +167,7 @@ struct reloc *find_reloc_by_dest(const struct elf *elf, struct section *sec, uns
struct reloc *find_reloc_by_dest_range(const struct elf *elf, struct section *sec,
unsigned long offset, unsigned int len);
struct symbol *find_func_containing(struct section *sec, unsigned long offset);
+struct symbol *find_object_containing(const struct section *sec, unsigned long offset);
#define for_each_sec(file, sec) \
list_for_each_entry(sec, &file->elf->sections, list)
--
2.17.1
On 2022-06-23 02:49, Chen Zhongjin wrote:
> Using unreachable() at default of switch generates an extra branch at
> end of the function, and compiler won't generate a ret to close this
> branch because it knows it's unreachable.
>
> If there's no instruction in this branch, compiler will generate a NOP,
> And it will confuse objtool to warn this NOP as a fall through branch.
>
> In fact these branches are actually unreachable, so we can replace
> unreachable() with returning a -EINVAL value.
>
> Signed-off-by: Chen Zhongjin <[email protected]>
> ---
> arch/arm64/kvm/hyp/vgic-v3-sr.c | 7 +++----
> drivers/irqchip/irq-gic-v3.c | 2 +-
> 2 files changed, 4 insertions(+), 5 deletions(-)
Basic courtesy would have been to Cc the maintainers of this code.
>
> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c
> b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> index 4fb419f7b8b6..f3cee92c3038 100644
> --- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> @@ -6,7 +6,6 @@
>
> #include <hyp/adjust_pc.h>
>
> -#include <linux/compiler.h>
> #include <linux/irqchip/arm-gic-v3.h>
> #include <linux/kvm_host.h>
>
> @@ -55,7 +54,7 @@ static u64 __gic_v3_get_lr(unsigned int lr)
> return read_gicreg(ICH_LR15_EL2);
> }
>
> - unreachable();
> + return -EINVAL;
NAK. That's absolutely *wrong*, and will hide future bugs.
Nothing checks for -EINVAL, and we *never* expect to
reach this, hence the perfectly valid annotation.
If something needs fixing, it probably is your tooling.
M.
--
Jazz is not dead. It just smells funny...
Hi,
Thanks for your review and patient.
On 2022/6/23 16:13, Marc Zyngier wrote:
> On 2022-06-23 02:49, Chen Zhongjin wrote:
>> Using unreachable() at default of switch generates an extra branch at
>> end of the function, and compiler won't generate a ret to close this
>> branch because it knows it's unreachable.
>>
>> If there's no instruction in this branch, compiler will generate a NOP,
>> And it will confuse objtool to warn this NOP as a fall through branch.
>>
>> In fact these branches are actually unreachable, so we can replace
>> unreachable() with returning a -EINVAL value.
>>
>> Signed-off-by: Chen Zhongjin <[email protected]>
>> ---
>> arch/arm64/kvm/hyp/vgic-v3-sr.c | 7 +++----
>> drivers/irqchip/irq-gic-v3.c | 2 +-
>> 2 files changed, 4 insertions(+), 5 deletions(-)
>
> Basic courtesy would have been to Cc the maintainers of this code.
>
Sorry for that.
I'll cc everyone next time.
>>
>> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
>> index 4fb419f7b8b6..f3cee92c3038 100644
>> --- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
>> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
>> @@ -6,7 +6,6 @@
>>
>> #include <hyp/adjust_pc.h>
>>
>> -#include <linux/compiler.h>
>> #include <linux/irqchip/arm-gic-v3.h>
>> #include <linux/kvm_host.h>
>>
>> @@ -55,7 +54,7 @@ static u64 __gic_v3_get_lr(unsigned int lr)
>> return read_gicreg(ICH_LR15_EL2);
>> }
>>
>> - unreachable();
>> + return -EINVAL;
>
> NAK. That's absolutely *wrong*, and will hide future bugs.
> Nothing checks for -EINVAL, and we *never* expect to
> reach this, hence the perfectly valid annotation.
>
> If something needs fixing, it probably is your tooling.
>
> M.
You are right.
Essentially, this is because objtool does not anticipate that the compiler will
generate additional instructions when marking unreachable instructions.
I'll fix this problem or add a specific check for this state.
Best,
Chen
On Thu, Jun 23, 2022 at 09:49:02AM +0800, Chen Zhongjin wrote:
> Code symbols not following the aarch64 procedure call convention should
> be annotated with SYM_CODE_* instead of SYM_FUNC_*
>
> Mark relevant symbols as generic code symbols.
> -SYM_CODE_START(tramp_exit_native)
> +SYM_CODE_START_LOCAL(tramp_exit_native)
> tramp_exit
> SYM_CODE_END(tramp_exit_native)
>
> -SYM_CODE_START(tramp_exit_compat)
> +SYM_CODE_START_LOCAL(tramp_exit_compat)
The commit log says this is fixing things mistakenly lablelld SYM_FUNC
but this bit of the actual change is making some symbols local.
> -SYM_FUNC_START_LOCAL(__create_page_tables)
> +SYM_CODE_START_LOCAL(__create_page_tables)
> mov x28, lr
>
> /*
> @@ -389,7 +389,7 @@ SYM_FUNC_START_LOCAL(__create_page_tables)
> bl dcache_inval_poc
>
> ret x28
> -SYM_FUNC_END(__create_page_tables)
> +SYM_CODE_END(__create_page_tables)
This is removed by Ard's recent refactoring, the others that are still
present look valid enough (for things that don't use the stack IIRC they
could be seen as conforming but equally this is all running in non
standard environments).
On Thu, Jun 23, 2022 at 09:49:06AM +0800, Chen Zhongjin wrote:
> This file only contains a set of constants forming the efi header.
>
> Make the constants part of a data symbol.
Reviewed-by: Mark Brown <[email protected]>
On Thu, Jun 23, 2022 at 09:49:08AM +0800, Chen Zhongjin wrote:
> Label __idmap_kpti_flag represents the location of a constant.
> Mark it as data symbol.
Reviewed-by: Mark Brown <[email protected]>
On Thu, Jun 23, 2022 at 09:49:09AM +0800, Chen Zhongjin wrote:
> Use SYM_DATA_* macros to annotate data bytes in the middle of .text
> sections.
>
> For local symbols, ".L" prefix needs to be dropped as the assembler
> exclude the symbols from the .o symbol table, making objtool unable
> to see them.
Reviewed-by: Mark Brown <[email protected]>
Hi Mark,
Thanks for review!
On 2022/6/30 1:47, Mark Brown wrote:
> On Thu, Jun 23, 2022 at 09:49:02AM +0800, Chen Zhongjin wrote:
>> Code symbols not following the aarch64 procedure call convention should
>> be annotated with SYM_CODE_* instead of SYM_FUNC_*
>>
>> Mark relevant symbols as generic code symbols.
>
>> -SYM_CODE_START(tramp_exit_native)
>> +SYM_CODE_START_LOCAL(tramp_exit_native)
>> tramp_exit
>> SYM_CODE_END(tramp_exit_native)
>>
>> -SYM_CODE_START(tramp_exit_compat)
>> +SYM_CODE_START_LOCAL(tramp_exit_compat)
>
> The commit log says this is fixing things mistakenly lablelld SYM_FUNC
> but this bit of the actual change is making some symbols local.
>
It makes sense. I'll remove this because whether this symbol is global makes few
difference here.
>> -SYM_FUNC_START_LOCAL(__create_page_tables)
>> +SYM_CODE_START_LOCAL(__create_page_tables)
>> mov x28, lr
>>
>> /*
>> @@ -389,7 +389,7 @@ SYM_FUNC_START_LOCAL(__create_page_tables)
>> bl dcache_inval_poc
>>
>> ret x28
>> -SYM_FUNC_END(__create_page_tables)
>> +SYM_CODE_END(__create_page_tables)
>
> This is removed by Ard's recent refactoring, the others that are still
> present look valid enough (for things that don't use the stack IIRC they
> could be seen as conforming but equally this is all running in non
> standard environments).
You are right, for SYM_CODE_, objtool won't generate ORC automatically as other
normal functions, unless UNWIND_HINT is explicitly specified for non-standard
stack frames.