2022-02-09 19:58:44

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 00/15] Function Granular KASLR

From: Kristen Carlson Accardi <[email protected]>

Function Granular Kernel Address Space Layout Randomization (FG-KASLR)
---------------------------------------------------------------------

This is an implementation of finer grained kernel address space
randomization. It rearranges the kernel code at load time on a
per-function level granularity, with only around a second added
to boot time.

Background
----------
KASLR was merged into the kernel with the objective of increasing the
difficulty of code reuse attacks. Code reuse attacks reused existing code
snippets to get around existing memory protections. They exploit software
bugs which expose addresses of useful code snippets to control the flow of
execution for their own nefarious purposes. KASLR moves the entire kernel
code text as a unit at boot time in order to make addresses less
predictable.
The order of the code within the segment is unchanged - only the base
address is shifted. There are a few shortcomings to this algorithm.

1. Low Entropy - there are only so many locations the kernel can fit in.
This means an attacker could guess without too much trouble.
2. Knowledge of a single address can reveal the offset of the base address,
exposing all other locations for a published/known kernel image.
3. Info leaks abound.

Finer grained ASLR has been proposed as a way to make ASLR more resistant
to info leaks. It is not a new concept at all, and there are many
variations possible. Function reordering is an implementation of finer
grained ASLR which randomizes the layout of an address space on a function
level granularity. We use the term "fgkaslr" in this document to refer to
the technique of function reordering when used with KASLR, as well as finer
grained KASLR in general.

Proposed Improvement
--------------------
This patch set proposes adding function reordering on top of the existing
KASLR base address randomization. The over-arching objective is incremental
improvement over what we already have. It is designed to work in
combination with the existing solution. The implementation is really pretty
simple, and there are 2 main area where changes occur:

* Build time

GCC has had an option to place functions into individual .text sections for
many years now. This option can be used to implement function reordering at
load time. The final compiled vmlinux retains all the section headers,
which can be used to help find the address ranges of each function. Using
this information and an expanded table of relocation addresses, individual
text sections can be suffled immediately after decompression. Some data
tables inside the kernel that have assumptions about order require
re-sorting after being updated when applying relocations. In order to
modify these tables, a few key symbols are excluded from the objcopy symbol
stripping process for use after shuffling the text segments.

Some highlights from the build time changes to look for:

The top level kernel Makefile was modified to add the gcc flag if it
is supported. Currently, I am applying this flag to everything it is
possible to randomize. Anything that is written in C and not present in a
special input section is randomized. The final binary segment 0 retains a
consolidated .text section, as well as all the individual .text.* sections.
Future work could turn off this flags for selected files or even entire
subsystems, although obviously at the cost of security.

The relocs tool is updated to add relative relocations. This information
previously wasn't included because it wasn't necessary when moving the
entire .text segment as a unit.

A new file was created to contain a list of symbols that objcopy should
keep. We use those symbols at load time as described below.

* Load time

The boot kernel was modified to parse the vmlinux elf file after
decompression to check for our interesting symbols that we kept, and to
look for any .text.* sections to randomize. The consolidated .text section
is skipped and not moved. The sections are shuffled randomly, and copied
into memory following the .text section in a new random order. The existing
code which updated relocation addresses was modified to account for
not just a fixed delta from the load address, but the offset that the
function section was moved to. This requires inspection of each address to
see if it was impacted by a randomization. We use a bsearch to make this
less horrible on performance. Any tables that need to be modified with new
addresses or resorted are updated using the symbol addresses parsed from
the elf symbol table.

In order to hide our new layout, symbols reported through /proc/kallsyms
will be displayed in a random order.

Security Considerations
-----------------------
The objective of this patch set is to improve a technology that is already
merged into the kernel (KASLR). This code will not prevent all attacks,
but should instead be considered as one of several tools that can be used.
In particular, this code is meant to make KASLR more effective in the
presence of info leaks.

How much entropy we are adding to the existing entropy of standard KASLR
will depend on a few variables. Firstly and most obviously, the number of
functions that are randomized matters. This implementation keeps the
existing .text section for code that cannot be randomized - for example,
because it was assembly code. The less sections to randomize, the less
entropy. In addition, due to alignment (16 bytes for x86_64), the number
of bits in a address that the attacker needs to guess is reduced, as the
lower bits are identical.

Performance Impact
------------------
There are two areas where function reordering can impact performance: boot
time latency, and run time performance.

* Boot time latency
This implementation of finer grained KASLR impacts the boot time of the
kernel in several places. It requires additional parsing of the kernel ELF
file to obtain the section headers of the sections to be randomized. It
calls the random number generator for each section to be randomized to
determine that section's new memory location. It copies the decompressed
kernel into a new area of memory to avoid corruption when laying out the
newly randomized sections. It increases the number of relocations the
kernel has to perform at boot time vs. standard KASLR, and it also requires
a lookup on each address that needs to be relocated to see if it was in a
randomized section and needs to be adjusted by a new offset. Finally, it
re-sorts a few data tables that are required to be sorted by address.

Booting a test VM on a modern, well appointed system showed an increase in
latency of approximately 1 second.

* Run time
The performance impact at run-time of function reordering varies by
workload.
Using kcbench, a kernel compilation benchmark, the performance of a kernel
build with finer grained KASLR was about 1% slower than a kernel with
standard KASLR. Analysis with perf showed a slightly higher percentage of
L1-icache-load-misses. Other workloads were examined as well, with varied
results. Some workloads performed significantly worse under FGKASLR, while
others stayed the same or were mysteriously better. In general, it will
depend on the code flow whether or not finer grained KASLR will impact
your workload, and how the underlying code was designed. Because the layout
changes per boot, each time a system is rebooted the performance of a
workload may change.

Future work could identify hot areas that may not be randomized and either
leave them in the .text section or group them together into a single
section that may be randomized. If grouping things together helps, one
other thing to consider is that if we could identify text blobs that should
be grouped together to benefit a particular code flow, it could be
interesting to explore whether this security feature could be also be used
as a performance feature if you are interested in optimizing your kernel
layout for a particular workload at boot time. Optimizing function layout
for a particular workload has been researched and proven effective - for
more information read the Facebook paper "Optimizing Function Placement
for Large-Scale Data-Center Applications" (see references section below).

Image Size
----------
Adding additional section headers as a result of compiling with
-ffunction-sections will increase the size of the vmlinux ELF file.
With a standard distro config, the resulting vmlinux was increased by
about 3%. The compressed image is also increased due to the header files,
as well as the extra relocations that must be added. You can expect
fgkaslr to increase the size of the compressed image by about 15%.

Memory Usage
------------
fgkaslr increases the amount of heap that is required at boot time,
although this extra memory is released when the kernel has finished
decompression. As a result, it may not be appropriate to use this feature
on systems without much memory.

Building
--------
To enable fine grained KASLR, you need to have the following config options
set (including all the ones you would use to build normal KASLR)

CONFIG_FG_KASLR=y

In addition, fgkaslr is only supported for the X86_64 architecture.

Modules
-------
Modules are randomized similarly to the rest of the kernel by shuffling
the sections at load time prior to moving them into memory. The module
must also have been build with the -ffunction-sections compiler option.

Although fgkaslr for the kernel is only supported for the X86_64
architecture, it is possible to use fgkaslr with modules on other
architectures. To enable this feature, select

CONFIG_MODULE_FG_KASLR=y

This option is selected automatically for X86_64 when CONFIG_FG_KASLR is
set.

Disabling
---------
Disabling normal KASLR using the nokaslr command line option also disables
fgkaslr. It is also possible to disable fgkaslr separately by booting with
nofgkaslr on the commandline.

References
----------
There are a lot of academic papers which explore finer grained ASLR.
This paper in particular contributed the most to my implementation design
as well as my overall understanding of the problem space:

Selfrando: Securing the Tor Browser against De-anonymization Exploits,
M. Conti, S. Crane, T. Frassetto, et al.

For more information on how function layout impacts performance, see:

Optimizing Function Placement for Large-Scale Data-Center Applications,
G. Ottoni, B. Maher ([0]).

Alexander Lobakin:

Starting from v6, the project changed the main developer, please see
the changelog for details.

The actual revision has been compile-time and runtime tested on the
following setups with no issues:
- x86_64, GCC 11, Binutils 2.35;
- x86_64, Clang/LLVM 13, ClangLTO + ClangCFI (from Sami's tree).

Some numbers for comparison:

feat make -j65 boot vmlinux.o vmlinux bzImage bogoops/s
Relocatable 4m38.478s 24.440s 72014208 58579520 9396192 57640.39
KASLR 4m39.344s 24.204s 72020624 87805776 9740352 57393.80
FG-K 16 fps 6m16.493s 25.429s 83759856 87194160 10885632 57784.76
FG-K 8 fps 6m20.190s 25.094s 83759856 88741328 10985248 56625.84
FG-K 1 fps 7m09.611s 25.922s 83759856 95681128 11352192 56953.99

The legend:
* make -j65 -- the compilation time of a kernel tree with the named
option enabled (and -j$(($(nproc) + 1))) (with the build machine
running the same stock kernel for all entries), give to see mainly
how linkers choke on big LD scripts;
* boot -- time elapsed from starting the kernel by the bootloader
to login prompt, affected mostly by the main FG-KASLR preboot
loop which shuffles function sections;
* vmlinux.o -- the size of the final vmlinux.o, altered by relocs
and -ffunction-sections;
* vmlinux -- the size of the final vmlinux, depends directly on the
number of (function) sections;
* bzImage -- the size of the final compressed kernel, same as with
vmlinux;
* bogoops/s -- stress-ng -c$(nproc) results on the kernel with the
named feature enabled;
* fps -- the number of functions per section, controlled by
CONFIG_FG_KASLR_SHIFT and CONFIG_MODULE_FG_KASLR_SHIFT.
16 fps means shift = 4, 8 fps on shift = 2, 1 fps for shift = 0.

From v9 ([1]):
- rebase on top of 5.17-rc3 + tip/master, notably:
* drop .fixup (the section was removed in 5.17 cycle) and
.Lbad_gs (was converted from a symbol to a label) references
and handling;
* use `is_enabled` and sed in scripts/link-vmlinux.sh (doesn't
include auto.conf anymore);
- s/ASM/asm across the series (Boris);
- update the year across the series;
- restructure the cover letter to make it more readable and give
the key ideas earlier (Chris);
- 0001: expand the commitmsg with how this was reproduced, drop
"Stable:" to avoid accidental LTS issues (Boris);
- 0002: get back to plain resetting `sympos` to zero when
applicable (Mirek); rephrase and expand the commitsg (Boris);
- 0003: expand the subject and the commitmsg with a bit of crucial
details, invert a couple branches to drop a `goto` and save one
indent level (Boris);
avoid using `bool` type in a structure, revert the for-loop back
to the `while`;
- 0004: rename `to` argument in asm symbol macros to `sect` to make
it less confusing, elaborate on "asm function sections" in the
commitmsg (Boris);
- 0005: replace "aflags" with the actual "KBUILD_AFLAGS" in the
commitmsg (Boris);
- 0006: build the new arch/x86/lib/orc.c only for CONFIG_MODULES=y
as this code was formerly guarded with the corresponding #ifdefs;
- 0007: rephrase the commitmsg;
- 0009: rename `base` in fgkaslr.c to `kallsyms_base` to dodge
from `-Wshadow`;
fix "indecis" -> "indices" and reduce variable scope in
layout_image();
include lib/orc.c in utils.c rather than fgkaslr.c itself;
replace {,u}int{32,64}_t -> {s,u}{32,64};
- 0010: change obsolete references to the section alignment of 64
as starting from v8 only the alignment of 128+ is being taken
into account (with CONFIG_DEBUG_FORCE_FUNCTION_ALIGN_64B=y,
every function is aligned to 64, so it would blow up the kernel
image for no reason);
- 0012: rephrase the subject to give more clue.

From v8 ([2]):
- the list of vmlinux symbols needed by both objcopy and fgkaslr.c
is now being expanded automatically from a header file. For
objcopy plain text, a direct cpp call is used, in C file I define
a generator macro and then include the header (Peter);
- unify compare and adjust functions between ORC and non-ORC
symbols (Peter);
- place ORC sorting function in a separate file
(arch/x86/lib/orc.c) to be able to just include it and not repeat
the same code for the second time in the pre-boot environment
(Peter);
- turn ASM functions sections on by default, not by new macros.
This involves `--sectname-subst` GAS flag and a fistful of ASM
code tweaks (Peter, Nicolas Pitre);
- make the feature above optional for FG-KASLR, not a required one.
For sure, unrandomized blob of ASM .text is a hole, but better
than nothing. ASM function sections is here for x86 anyways;
- deduplicate lots of code apart from ORC sorting and vmlinux
symbols. Introduce a new common macro for shuffling an array
and use it all the way through (Peter);
- use `-z unique-symbol` linker flag to make position-based search
in livepatching code obsolete. This is now preferred and enabled
when available, and is a requirement for FG-KASLR where pos-based
search is impossible (Peter, Josh, HJL);
- always print kallsyms in random order for unpriviledged users,
not only when FG-KASLR is enabled. This allowed to simplify code,
and you can consider it as yet another hardening (Ard, Josh,
Peter);
- change ".lds" ext for module linker scripts to ".ko.lds" as ".lds"
can't be treated purely as of generated / build artifacts. There's
a bunch of LDSes inside the tree, and they all are valid. Since
it's not that easy to distinguish where is what on `make clean`
and stuff like ".mod.c" is being deleted using a call to `find`,
just pick ".ko.lds".
It makes it even more clear that this script is for the final
module, not any intermediate files.

From v7 (unreleased):
- rebase on top of 5.16-rc3, notably:
- drop 4 patches already taken in mainline;
- adopt to the new exception handlers logics;
- changed two new x86 ASM crypto module to generate function
sections. Also:
- improve generate_text_sections.pl script to address changes in
Clang 13 emitting __cfi_check_fail() only on final linking;
- retest on the latest stable Clang/LLVM stack (13);
- add missing .lds rule to the top .gitignore.

From v6 ([3]):
- rebase on top of 5.15-rc1 and commit db2b0c5d7b6f
("objtool: Support pv_opsindirect calls for noinstr")
from tip's objtool/core as there is plenty of counter-intuitive
conflicts between these two;
- change livepatch bit (#12) logics from forced override to exit
with errno and a error message to make it more clear to the users
(Miroslav);
- expand the cover letter a bit, add some build-time and runtime
numbers (Kees, Kristen).

The major differences since v5 [4]:
- one can now tune the number of functions per each section to
achieve the preferable vmlinux size or protection level. Default
is still as one section per function.
This can be handy for storage-constrained systems. 4-8 fps are
still strong, but reduce the size of the final vmlinu{x,z}
significantly (see the comparison below);
- don't use orphan sections anymore. It's not reliable at all /
may differ from linker to linker, and also conflicts with
CONFIG_LD_ORPHAN_WARN which is great for catching random bugs ->
- all the .text.* sections are now being described explicitly in the
linker script. A Perl script is used to take the original LDS, the
original object file, read a list of input sections from it and
generate the resulting LDS.
This costs a bit of linking time as LD tends to think hard when
processing scripts > 1 Mb (a subject for future BFD and LLD
patches). It adds about 60-80 seconds to the whole linking process
(BTF step, 2-3 kallsyms steps and the final step), but "better
safe than sorry".
In addition, that approach allows to reserve some space at the end
of text (8-12 Kb, no impact on vmlinux size as THP-aligned (2 Mb)
rodata goes right after it) and add some link-time assertions ->
- input .text section now must be empty, otherwise the linkage will
be stopped. This is implemented by the size assertion in the
resulting LD script and is designed to plug the potentional layout
leakage. This also means that ->
- "regular" ASM functions are now being placed into unique separate
functions the same way compiler does this for C functions. This is
achieved by hijacking the commonly used macros. The symbol name is
now being taken as a base for its new section name.
This gives a better opportunity to LTO, DCE and FG-KASLR, as ASM
code can now also be randomized or garbage-collected;
- it's now fully compatible with ClangLTO, ClangCFI,
CONFIG_LD_ORPHAN_WARN and some all the rest stuff landed since the
last revision has been published;
- `-z unique-symbol` linker flag is now used to ensure livepatching
works even with randomized sections. Position-based search is not
needed in this case;
- kallsyms are now being shuffled and displayed in random order not
only when FG-KASLR is enabled, but all the time (for unpriviledged
access);
- tons of code were improved and deduplicated all over the place.

Changes in v5:
--------------
* fixed a bug in the code which increases boot heap size for
CONFIG_FG_KASLR which prevented the boot heap from being increased
for CONFIG_FG_KASLR when using bzip2 compression. Thanks to Andy Lavr
for finding the problem and identifying the solution.
* changed the adjustment of the orc_unwind_ip table at boot time to
disregard relocs associated with this table, and instead inspect the
entries separately. Relocs are not able to be used since they are
no longer correct once the table is resorted at buildtime.
* changed how orc_unwind_ip addresses in randomized sections are identified
to include the byte immediately after the end of the section.
* updated module code to use kvmalloc/kvfree based on suggestions from
Evgenii Shatokhin <[email protected]>.
* changed kernel commandline to disable fgkaslr to simply "nofgkaslr" to
match the nokaslr option. fgkaslr="X" can be added at a later date
if it is needed.
* Added a patch to force livepatch to require symbols to be unique if
using while fgkaslr either for core or modules.

Changes in v4:
-------------
* dropped the patch to split out change to STATIC definition in
x86/boot/compressed/misc.c and replaced with a patch authored
by Kees Cook to avoid the duplicate malloc definitions
* Added a section to Documentation/admin-guide/kernel-parameters.txt
to document the fgkaslr boot option.
* redesigned the patch to hide the new layout when reading
/proc/kallsyms. The previous implementation utilized a dynamically
allocated linked list to display the kernel and module symbols
in alphabetical order. The new implementation uses a randomly
shuffled index array to display the kernel and module symbols
in a random order.

Changes in v3:
-------------
* Makefile changes to accommodate CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
* removal of extraneous ALIGN_PAGE from _etext changes
* changed variable names in x86/tools/relocs to be less confusing
* split out change to STATIC definition in x86/boot/compressed/misc.c
* Updates to Documentation to make it more clear what is preserved in .text
* much more detailed commit message for function granular KASLR patch
* minor tweaks and changes that make for more readable code
* this cover letter updated slightly to add additional details

Changes in v2:
--------------
* Fix to address i386 build failure
* Allow module reordering patch to be configured separately so that
arm (or other non-x86_64 arches) can take advantage of module function
reordering. This support has not be tested by me, but smoke tested by
Ard Biesheuvel <[email protected]> on arm.
* Fix build issue when building on arm as reported by
Ard Biesheuvel <[email protected]>

The series is also available here: [5]

[0] https://dl.acm.org/doi/pdf/10.5555/3049832.3049858
[1] https://lore.kernel.org/linux-hardening/[email protected]
[2] https://lore.kernel.org/linux-hardening/[email protected]
[3] https://lore.kernel.org/linux-hardening/[email protected]
[4] https://lore.kernel.org/kernel-hardening/[email protected]
[5] https://github.com/alobakin/linux/pull/3

Alexander Lobakin (9):
modpost: fix removing numeric suffixes
livepatch: avoid position-based search if `-z unique-symbol` is
available
arch: introduce asm function sections
x86: support asm function sections
x86: decouple ORC table sorting into a separate file
FG-KASLR: use a scripted approach to handle .text.* sections
x86/boot: allow FG-KASLR to be selected
module: use a scripted approach for FG-KASLR
maintainers: add MAINTAINERS entry for FG-KASLR

Kristen Carlson Accardi (6):
kallsyms: randomize /proc/kallsyms output order
Makefile: add config options and build scripts for FG-KASLR
x86/tools: Add relative relocs for randomized functions
x86: Add support for function granular KASLR
module: add arch-indep FG-KASLR for randomizing function layout
Documentation: add documentation for FG-KASLR

.gitignore | 1 +
.../admin-guide/kernel-parameters.txt | 6 +
Documentation/security/fgkaslr.rst | 172 ++++
Documentation/security/index.rst | 1 +
MAINTAINERS | 12 +
Makefile | 41 +-
arch/Kconfig | 10 +
arch/x86/Kconfig | 2 +
arch/x86/boot/Makefile | 1 +
arch/x86/boot/compressed/.gitignore | 1 +
arch/x86/boot/compressed/Makefile | 21 +-
arch/x86/boot/compressed/fgkaslr.c | 752 ++++++++++++++++++
arch/x86/boot/compressed/gen-symbols.h | 30 +
arch/x86/boot/compressed/head_32.S | 2 +-
arch/x86/boot/compressed/head_64.S | 32 +-
arch/x86/boot/compressed/misc.c | 144 +++-
arch/x86/boot/compressed/misc.h | 28 +
arch/x86/boot/compressed/utils.c | 16 +
arch/x86/boot/pmjump.S | 2 +-
arch/x86/crypto/aesni-intel_asm.S | 4 +-
arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 4 +
arch/x86/include/asm/boot.h | 13 +-
arch/x86/include/asm/orc_types.h | 7 +
arch/x86/include/asm/paravirt.h | 2 +
arch/x86/include/asm/qspinlock_paravirt.h | 2 +
arch/x86/kernel/head_32.S | 4 +-
arch/x86/kernel/head_64.S | 4 +-
arch/x86/kernel/kprobes/core.c | 2 +
arch/x86/kernel/kvm.c | 2 +
arch/x86/kernel/relocate_kernel_32.S | 10 +-
arch/x86/kernel/relocate_kernel_64.S | 12 +-
arch/x86/kernel/unwind_orc.c | 63 +-
arch/x86/kernel/vmlinux.lds.S | 8 +-
arch/x86/kvm/emulate.c | 7 +-
arch/x86/lib/Makefile | 3 +
arch/x86/lib/copy_user_64.S | 2 +-
arch/x86/lib/error-inject.c | 2 +
arch/x86/lib/getuser.S | 5 +-
arch/x86/lib/memcpy_64.S | 4 +-
arch/x86/lib/memmove_64.S | 5 +-
arch/x86/lib/memset_64.S | 5 +-
arch/x86/lib/orc.c | 78 ++
arch/x86/lib/putuser.S | 2 +-
arch/x86/power/hibernate_asm_32.S | 10 +-
arch/x86/power/hibernate_asm_64.S | 10 +-
arch/x86/tools/relocs.c | 32 +-
arch/x86/tools/relocs.h | 4 +-
arch/x86/tools/relocs_common.c | 14 +-
include/asm-generic/vmlinux.lds.h | 57 +-
include/linux/linkage.h | 121 ++-
include/linux/random.h | 16 +
include/uapi/linux/elf.h | 1 +
init/Kconfig | 69 ++
kernel/kallsyms.c | 94 ++-
kernel/livepatch/core.c | 17 +-
kernel/module.c | 73 +-
scripts/Makefile.modfinal | 20 +-
scripts/generate_text_sections.pl | 172 ++++
scripts/link-vmlinux.sh | 30 +-
scripts/mod/modpost.c | 48 +-
scripts/module.lds.S | 14 +-
scripts/sorttable.h | 5 -
tools/arch/x86/include/asm/orc_types.h | 7 +
63 files changed, 2122 insertions(+), 216 deletions(-)
create mode 100644 Documentation/security/fgkaslr.rst
create mode 100644 arch/x86/boot/compressed/fgkaslr.c
create mode 100644 arch/x86/boot/compressed/gen-symbols.h
create mode 100644 arch/x86/boot/compressed/utils.c
create mode 100644 arch/x86/lib/orc.c
create mode 100755 scripts/generate_text_sections.pl

--
2.34.1



2022-02-09 19:58:59

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 01/15] modpost: fix removing numeric suffixes

`-z unique-symbol` linker flag which is planned to use with FG-KASLR
to simplify livepatching (hopefully globally later on) triggers the
following:

ERROR: modpost: "param_set_uint.0" [vmlinux] is a static EXPORT_SYMBOL

The reason is that for now the condition from remove_dot():

if (m && (s[n + m] == '.' || s[n + m] == 0))

which was designed to test if it's a dot or a '\0' after the suffix
is never satisfied.
This is due to that `s[n + m]` always points to the last digit of a
numeric suffix, not on the symbol next to it (from a custom debug
print added to modpost):

param_set_uint.0, s[n + m] is '0', s[n + m + 1] is '\0'

So it's off-by-one and was like that since 2014.
Fix this for the sake of upcoming features, but don't bother
stable-backporting, as it's well hidden -- apart from that LD flag,
can be triggered only by GCC LTO which never landed upstream.

Fixes: fcd38ed0ff26 ("scripts: modpost: fix compilation warning")
Signed-off-by: Alexander Lobakin <[email protected]>
---
scripts/mod/modpost.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 6bfa33217914..4648b7afe5cc 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -1986,7 +1986,7 @@ static char *remove_dot(char *s)

if (n && s[n]) {
size_t m = strspn(s + n + 1, "0123456789");
- if (m && (s[n + m] == '.' || s[n + m] == 0))
+ if (m && (s[n + m + 1] == '.' || s[n + m + 1] == 0))
s[n] = 0;

/* strip trailing .lto */
--
2.34.1


2022-02-09 20:00:58

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 04/15] arch: introduce asm function sections

Sometimes it can be useful to create a separate section for every
function (symbol in general) to be able then to selectively merge
them back into one or several others. This is how Dead Code
Elimination (DCE, CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) and a part
of Link-Time Optimization (LTO, currently CONFIG_LTO_CLANG) work.
Currently, this can only be done for C functions as the compilers
are able to do this automatically when `-ffunction-sections` is
specified.

Add a basic infra for supporting asm function sections, which means
support for putting functions written in assembly into separate
sections. If any of the required build options (DCE, LTO, FG-KASLR
later) is on and the target architecture claims it supports them,
all asm functions and "code" will be placed into separate so-named
("current_section.function_name") sections by default.
This is achieved using `--sectname-subst` GAS flag which will then
substitute "%S" in a .pushsection or .section directive with the
name of the current section. So,

.section .entry.text # current section is .entry.text
SYM_FUNC_START(foo)
-> .pushsection %S.foo # now the section is .entry.text.foo
do_something
SYM_FUNC_END(foo)
-> .popsection # back to .entry.text

Now the function "foo" is placed into .entry.text.foo and can be
garbage-collected if there are no consumers for it.
Otherwise, the linker script will merge it back into .entry.text.

Since modpost is being run on vmlinux.o, i.e. before the final
linking, expand its okay-list to cover new potential sections
(which will get processed afterwards).

Suggested-by: Peter Zijlstra <[email protected]> # always do, then merge
Suggested-by: Nicolas Pitre <[email protected]> # --sectname-subst flag
Signed-off-by: Alexander Lobakin <[email protected]>
---
Makefile | 18 +++++
arch/Kconfig | 6 ++
include/asm-generic/vmlinux.lds.h | 21 +++---
include/linux/linkage.h | 119 +++++++++++++++++++++++++++++-
init/Kconfig | 10 +++
scripts/mod/modpost.c | 6 +-
6 files changed, 167 insertions(+), 13 deletions(-)

diff --git a/Makefile b/Makefile
index fa9f947c9839..fbe2d13028f4 100644
--- a/Makefile
+++ b/Makefile
@@ -877,6 +877,24 @@ ifeq ($(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL)$(CONFIG_LIVEPATCH),yy)
KBUILD_LDFLAGS += -z unique-symbol
endif

+# Allow ASM code to generate separate sections for each function. See
+# `include/linux/linkage.h` for explanation. This flag is to enable GAS to
+# insert the name of the previous section instead of `%S` inside .pushsection
+ifdef CONFIG_HAVE_ASM_FUNCTION_SECTIONS
+ifneq ($(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION)$(CONFIG_LTO_CLANG),)
+SECSUBST_AFLAGS := -Wa,--sectname-subst
+KBUILD_AFLAGS_KERNEL += $(SECSUBST_AFLAGS)
+KBUILD_CFLAGS_KERNEL += $(SECSUBST_AFLAGS)
+export SECSUBST_AFLAGS
+endif
+
+# Same for modules. LD DCE doesn't work for them, thus not checking for it
+ifneq ($(CONFIG_LTO_CLANG),)
+KBUILD_AFLAGS_MODULE += -Wa,--sectname-subst
+KBUILD_CFLAGS_MODULE += -Wa,--sectname-subst
+endif
+endif # CONFIG_HAVE_ASM_FUNCTION_SECTIONS
+
ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
KBUILD_CFLAGS_KERNEL += -ffunction-sections -fdata-sections
LDFLAGS_vmlinux += --gc-sections
diff --git a/arch/Kconfig b/arch/Kconfig
index 678a80713b21..550f0599e211 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1322,6 +1322,12 @@ config DYNAMIC_SIGFRAME
config HAVE_ARCH_NODE_DEV_GROUP
bool

+config ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS
+ bool
+ help
+ An arch should select this if it can be built and run with its
+ asm functions placed into separate sections to improve DCE and LTO.
+
source "kernel/gcov/Kconfig"

source "scripts/gcc-plugins/Kconfig"
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 42f3866bca69..e7b8a84e0e64 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -84,6 +84,9 @@
/* Align . to a 8 byte boundary equals to maximum function alignment. */
#define ALIGN_FUNCTION() . = ALIGN(8)

+/* This is useful for collecting individual sections back into one main */
+#define SECT_WILDCARD(sect) sect sect.[0-9a-zA-Z_]*
+
/*
* LD_DEAD_CODE_DATA_ELIMINATION option enables -fdata-sections, which
* generates .data.identifier sections, which need to be pulled in with
@@ -97,12 +100,12 @@
* sections to be brought in with rodata.
*/
#if defined(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) || defined(CONFIG_LTO_CLANG)
-#define TEXT_MAIN .text .text.[0-9a-zA-Z_]*
-#define DATA_MAIN .data .data.[0-9a-zA-Z_]* .data..L* .data..compoundliteral* .data.$__unnamed_* .data.$L*
-#define SDATA_MAIN .sdata .sdata.[0-9a-zA-Z_]*
-#define RODATA_MAIN .rodata .rodata.[0-9a-zA-Z_]* .rodata..L*
-#define BSS_MAIN .bss .bss.[0-9a-zA-Z_]* .bss..compoundliteral*
-#define SBSS_MAIN .sbss .sbss.[0-9a-zA-Z_]*
+#define TEXT_MAIN SECT_WILDCARD(.text)
+#define DATA_MAIN SECT_WILDCARD(.data) .data..L* .data..compoundliteral* .data.$__unnamed_* .data.$L*
+#define SDATA_MAIN SECT_WILDCARD(.sdata)
+#define RODATA_MAIN SECT_WILDCARD(.rodata) .rodata..L*
+#define BSS_MAIN SECT_WILDCARD(.bss) .bss..compoundliteral*
+#define SBSS_MAIN SECT_WILDCARD(.sbss)
#else
#define TEXT_MAIN .text
#define DATA_MAIN .data
@@ -564,7 +567,7 @@
#define NOINSTR_TEXT \
ALIGN_FUNCTION(); \
__noinstr_text_start = .; \
- *(.noinstr.text) \
+ *(SECT_WILDCARD(.noinstr.text)) \
__noinstr_text_end = .;

/*
@@ -621,7 +624,7 @@
#define ENTRY_TEXT \
ALIGN_FUNCTION(); \
__entry_text_start = .; \
- *(.entry.text) \
+ *(SECT_WILDCARD(.entry.text)) \
__entry_text_end = .;

#define IRQENTRY_TEXT \
@@ -643,7 +646,7 @@
__static_call_text_end = .;

/* Section used for early init (in .S files) */
-#define HEAD_TEXT KEEP(*(.head.text))
+#define HEAD_TEXT KEEP(*(SECT_WILDCARD(.head.text)))

#define HEAD_TEXT_SECTION \
.head.text : AT(ADDR(.head.text) - LOAD_OFFSET) { \
diff --git a/include/linux/linkage.h b/include/linux/linkage.h
index dbf8506decca..f3b966a6427e 100644
--- a/include/linux/linkage.h
+++ b/include/linux/linkage.h
@@ -73,6 +73,38 @@
#define __ALIGN_STR ".align 4,0x90"
#endif

+/*
+ * Allow ASM symbols to have their own unique sections if they are being
+ * generated by the compiler for C functions (DCE, LTO). Correlates with
+ * the presence of the `-ffunction-section` in KBUILD_CFLAGS.
+ */
+#if defined(CONFIG_HAVE_ASM_FUNCTION_SECTIONS) && \
+ ((defined(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) && !defined(MODULE)) || \
+ (defined(CONFIG_LTO_CLANG)))
+
+#define SYM_PUSH_SECTION(name) \
+ .pushsection %S.name, "ax"
+
+#define SYM_POP_SECTION() \
+ .popsection
+
+#define __ASM_PUSH_SECTION(name) \
+ ".pushsection %S." name ", \"ax\""
+
+#else /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || LTO)) */
+
+#define SYM_PUSH_SECTION(name)
+#define SYM_POP_SECTION()
+#define __ASM_PUSH_SECTION(name)
+
+#endif /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || LTO)) */
+
+#define ASM_PUSH_SECTION(name) \
+ __ASM_PUSH_SECTION(__stringify(name))
+
+#define ASM_POP_SECTION() \
+ __stringify(SYM_POP_SECTION())
+
#ifdef __ASSEMBLY__

/* SYM_T_FUNC -- type used by assembler to mark functions */
@@ -209,6 +241,15 @@
SYM_START(name, SYM_L_LOCAL, SYM_A_ALIGN)
#endif

+/*
+ * SYM_FUNC_START_WEAK_ALIAS -- use where there are two global names for one
+ * function, and one of them is weak
+ */
+#ifndef SYM_FUNC_START_WEAK_ALIAS
+#define SYM_FUNC_START_WEAK_ALIAS(name) \
+ SYM_START(name, SYM_L_WEAK, SYM_A_ALIGN)
+#endif
+
/*
* SYM_FUNC_START_ALIAS -- use where there are two global names for one
* function
@@ -225,12 +266,24 @@
* later.
*/
#define SYM_FUNC_START(name) \
+ SYM_PUSH_SECTION(name) ASM_NL \
+ SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN)
+#endif
+
+/*
+ * SYM_FUNC_START_SECT -- use for global functions, will be conditionally
+ * placed into a section specified in the second argument
+ */
+#ifndef SYM_FUNC_START_SECT
+#define SYM_FUNC_START_SECT(name, sect) \
+ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN)
#endif

/* SYM_FUNC_START_NOALIGN -- use for global functions, w/o alignment */
#ifndef SYM_FUNC_START_NOALIGN
#define SYM_FUNC_START_NOALIGN(name) \
+ SYM_PUSH_SECTION(name) ASM_NL \
SYM_START(name, SYM_L_GLOBAL, SYM_A_NONE)
#endif

@@ -238,24 +291,38 @@
#ifndef SYM_FUNC_START_LOCAL
/* the same as SYM_FUNC_START_LOCAL_ALIAS, see comment near SYM_FUNC_START */
#define SYM_FUNC_START_LOCAL(name) \
+ SYM_PUSH_SECTION(name) ASM_NL \
SYM_START(name, SYM_L_LOCAL, SYM_A_ALIGN)
#endif

/* SYM_FUNC_START_LOCAL_NOALIGN -- use for local functions, w/o alignment */
#ifndef SYM_FUNC_START_LOCAL_NOALIGN
#define SYM_FUNC_START_LOCAL_NOALIGN(name) \
+ SYM_PUSH_SECTION(name) ASM_NL \
+ SYM_START(name, SYM_L_LOCAL, SYM_A_NONE)
+#endif
+
+/*
+ * SYM_FUNC_START_LOCAL_NOALIGN_SECT -- use for local functions, w/o alignment,
+ * will be conditionally placed into a section specified in the second argument
+ */
+#ifndef SYM_FUNC_START_LOCAL_NOALIGN_SECT
+#define SYM_FUNC_START_LOCAL_NOALIGN_SECT(name, sect) \
+ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_LOCAL, SYM_A_NONE)
#endif

/* SYM_FUNC_START_WEAK -- use for weak functions */
#ifndef SYM_FUNC_START_WEAK
#define SYM_FUNC_START_WEAK(name) \
+ SYM_PUSH_SECTION(name) ASM_NL \
SYM_START(name, SYM_L_WEAK, SYM_A_ALIGN)
#endif

/* SYM_FUNC_START_WEAK_NOALIGN -- use for weak functions, w/o alignment */
#ifndef SYM_FUNC_START_WEAK_NOALIGN
#define SYM_FUNC_START_WEAK_NOALIGN(name) \
+ SYM_PUSH_SECTION(name) ASM_NL \
SYM_START(name, SYM_L_WEAK, SYM_A_NONE)
#endif

@@ -272,24 +339,59 @@
#ifndef SYM_FUNC_END
/* the same as SYM_FUNC_END_ALIAS, see comment near SYM_FUNC_START */
#define SYM_FUNC_END(name) \
- SYM_END(name, SYM_T_FUNC)
+ SYM_END(name, SYM_T_FUNC) ASM_NL \
+ SYM_POP_SECTION()
#endif

/* SYM_CODE_START -- use for non-C (special) functions */
#ifndef SYM_CODE_START
#define SYM_CODE_START(name) \
+ SYM_PUSH_SECTION(name) ASM_NL \
+ SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN)
+#endif
+
+/*
+ * SYM_CODE_START_SECT -- use for non-C (special) functions, will be
+ * conditionally placed into a section specified in the second argument
+ */
+#ifndef SYM_CODE_START_SECT
+#define SYM_CODE_START_SECT(name, sect) \
+ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN)
#endif

/* SYM_CODE_START_NOALIGN -- use for non-C (special) functions, w/o alignment */
#ifndef SYM_CODE_START_NOALIGN
#define SYM_CODE_START_NOALIGN(name) \
+ SYM_PUSH_SECTION(name) ASM_NL \
+ SYM_START(name, SYM_L_GLOBAL, SYM_A_NONE)
+#endif
+
+/*
+ * SYM_CODE_START_NOALIGN_SECT -- use for non-C (special) functions,
+ * w/o alignment, will be conditionally placed into a section specified
+ * in the second argument
+ */
+#ifndef SYM_CODE_START_NOALIGN_SECT
+#define SYM_CODE_START_NOALIGN_SECT(name, sect) \
+ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_GLOBAL, SYM_A_NONE)
#endif

/* SYM_CODE_START_LOCAL -- use for local non-C (special) functions */
#ifndef SYM_CODE_START_LOCAL
#define SYM_CODE_START_LOCAL(name) \
+ SYM_PUSH_SECTION(name) ASM_NL \
+ SYM_START(name, SYM_L_LOCAL, SYM_A_ALIGN)
+#endif
+
+/*
+ * SYM_CODE_START_LOCAL -- use for local non-C (special) functions, will
+ * be conditionally placing into a section specified in the second argument
+ */
+#ifndef SYM_CODE_START_LOCAL_SECT
+#define SYM_CODE_START_LOCAL_SECT(name, sect) \
+ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_LOCAL, SYM_A_ALIGN)
#endif

@@ -299,13 +401,26 @@
*/
#ifndef SYM_CODE_START_LOCAL_NOALIGN
#define SYM_CODE_START_LOCAL_NOALIGN(name) \
+ SYM_PUSH_SECTION(name) ASM_NL \
+ SYM_START(name, SYM_L_LOCAL, SYM_A_NONE)
+#endif
+
+/*
+ * SYM_CODE_START_LOCAL_NOALIGN_SECT -- use for local non-C (special)
+ * functions, w/o alignment, will be conditionally placed into a section
+ * specified in the second argument
+ */
+#ifndef SYM_CODE_START_LOCAL_NOALIGN_SECT
+#define SYM_CODE_START_LOCAL_NOALIGN_SECT(name, sect) \
+ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_LOCAL, SYM_A_NONE)
#endif

/* SYM_CODE_END -- the end of SYM_CODE_START_LOCAL, SYM_CODE_START, ... */
#ifndef SYM_CODE_END
#define SYM_CODE_END(name) \
- SYM_END(name, SYM_T_NONE)
+ SYM_END(name, SYM_T_NONE) ASM_NL \
+ SYM_POP_SECTION()
#endif

/* === data annotations === */
diff --git a/init/Kconfig b/init/Kconfig
index 8e900d17d42b..4acfc80f22df 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1386,6 +1386,16 @@ config CC_OPTIMIZE_FOR_SIZE

endchoice

+config HAVE_ASM_FUNCTION_SECTIONS
+ depends on ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS
+ depends on $(cc-option,-Wa$(comma)--sectname-subst)
+ def_bool y
+ help
+ This enables asm function sections if both architecture and
+ toolchain support it. It allows creating a separate section
+ for each function written in assembly in order to improve DCE
+ and LTO (works the same way as -ffunction-sections for C code).
+
config HAVE_LD_DEAD_CODE_DATA_ELIMINATION
bool
help
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index ec521ccebea6..84d2c44f9383 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -960,7 +960,9 @@ static void check_section(const char *modname, struct elf_info *elf,
".kprobes.text", ".cpuidle.text", ".noinstr.text"
#define OTHER_TEXT_SECTIONS ".ref.text", ".head.text", ".spinlock.text", \
".fixup", ".entry.text", ".exception.text", ".text.*", \
- ".coldtext", ".softirqentry.text"
+ ".coldtext", ".softirqentry.text", ".text.unlikely.*", \
+ ".noinstr.text.*", ".head.text.*", ".fixup.*", \
+ ".entry.text.*"

#define INIT_SECTIONS ".init.*"
#define MEM_INIT_SECTIONS ".meminit.*"
@@ -1041,7 +1043,7 @@ enum mismatch {
struct sectioncheck {
const char *fromsec[20];
const char *bad_tosec[20];
- const char *good_tosec[20];
+ const char *good_tosec[25];
enum mismatch mismatch;
const char *symbol_white_list[20];
void (*handler)(const char *modname, struct elf_info *elf,
--
2.34.1


2022-02-09 20:07:37

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 08/15] x86/tools: Add relative relocs for randomized functions

From: Kristen Carlson Accardi <[email protected]>

When reordering functions, the relative offsets for relocs that
are either in the randomized sections, or refer to the randomized
sections will need to be adjusted. Add code to detect whether a
reloc satisfies these cases, and if so, add them to the appropriate
reloc list.

Alexander Lobakin:

Don't split relocs' usage string across lines to ease grepping.

Signed-off-by: Kristen Carlson Accardi <[email protected]>
Reviewed-by: Tony Luck <[email protected]>
Tested-by: Tony Luck <[email protected]>
Reviewed-by: Kees Cook <[email protected]>
Signed-off-by: Alexander Lobakin <[email protected]>
---
arch/x86/boot/compressed/Makefile | 7 ++++++-
arch/x86/tools/relocs.c | 32 +++++++++++++++++++++++++++----
arch/x86/tools/relocs.h | 4 ++--
arch/x86/tools/relocs_common.c | 14 +++++++++-----
4 files changed, 45 insertions(+), 12 deletions(-)

diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index e7ee94d5f55a..b1f5817c5737 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -110,6 +110,11 @@ $(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE
$(call if_changed,ld)

OBJCOPYFLAGS_vmlinux.bin := -R .comment -S
+
+ifdef CONFIG_FG_KASLR
+RELOCS_ARGS += --fg-kaslr
+endif
+
$(obj)/vmlinux.bin: vmlinux FORCE
$(call if_changed,objcopy)

@@ -117,7 +122,7 @@ targets += $(patsubst $(obj)/%,%,$(vmlinux-objs-y)) vmlinux.bin.all vmlinux.relo

CMD_RELOCS = arch/x86/tools/relocs
quiet_cmd_relocs = RELOCS $@
- cmd_relocs = $(CMD_RELOCS) $< > $@;$(CMD_RELOCS) --abs-relocs $<
+ cmd_relocs = $(CMD_RELOCS) $(RELOCS_ARGS) $< > $@;$(CMD_RELOCS) $(RELOCS_ARGS) --abs-relocs $<
$(obj)/vmlinux.relocs: vmlinux FORCE
$(call if_changed,relocs)

diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c
index e2c5b296120d..96b6042fc76f 100644
--- a/arch/x86/tools/relocs.c
+++ b/arch/x86/tools/relocs.c
@@ -45,6 +45,8 @@ struct section {
};
static struct section *secs;

+static int fgkaslr_mode;
+
static const char * const sym_regex_kernel[S_NSYMTYPES] = {
/*
* Following symbols have been audited. There values are constant and do
@@ -823,6 +825,24 @@ static int is_percpu_sym(ElfW(Sym) *sym, const char *symname)
strncmp(symname, "init_per_cpu_", 13);
}

+static int is_function_section(struct section *sec)
+{
+ if (!fgkaslr_mode)
+ return 0;
+
+ return !strncmp(sec_name(sec->shdr.sh_info), ".text.", 6);
+}
+
+static int is_randomized_sym(ElfW(Sym) *sym)
+{
+ if (!fgkaslr_mode)
+ return 0;
+
+ if (sym->st_shndx > shnum)
+ return 0;
+
+ return !strncmp(sec_name(sym_index(sym)), ".text.", 6);
+}

static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym,
const char *symname)
@@ -848,12 +868,15 @@ static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym,
case R_X86_64_PC32:
case R_X86_64_PLT32:
/*
- * PC relative relocations don't need to be adjusted unless
- * referencing a percpu symbol.
+ * we need to keep pc relative relocations for sections which
+ * might be randomized, and for the percpu section.
+ * We also need to keep relocations for any offset which might
+ * reference an address in a section which has been randomized.
*
* NB: R_X86_64_PLT32 can be treated as R_X86_64_PC32.
*/
- if (is_percpu_sym(sym, symname))
+ if (is_function_section(sec) || is_randomized_sym(sym) ||
+ is_percpu_sym(sym, symname))
add_reloc(&relocs32neg, offset);
break;

@@ -1168,8 +1191,9 @@ static void print_reloc_info(void)

void process(FILE *fp, int use_real_mode, int as_text,
int show_absolute_syms, int show_absolute_relocs,
- int show_reloc_info)
+ int show_reloc_info, int fgkaslr)
{
+ fgkaslr_mode = fgkaslr;
regex_init(use_real_mode);
read_ehdr(fp);
read_shdrs(fp);
diff --git a/arch/x86/tools/relocs.h b/arch/x86/tools/relocs.h
index 4c49c82446eb..269db511b243 100644
--- a/arch/x86/tools/relocs.h
+++ b/arch/x86/tools/relocs.h
@@ -32,8 +32,8 @@ enum symtype {

void process_32(FILE *fp, int use_real_mode, int as_text,
int show_absolute_syms, int show_absolute_relocs,
- int show_reloc_info);
+ int show_reloc_info, int fgkaslr);
void process_64(FILE *fp, int use_real_mode, int as_text,
int show_absolute_syms, int show_absolute_relocs,
- int show_reloc_info);
+ int show_reloc_info, int fgkaslr);
#endif /* RELOCS_H */
diff --git a/arch/x86/tools/relocs_common.c b/arch/x86/tools/relocs_common.c
index 6634352a20bc..d6acda36575a 100644
--- a/arch/x86/tools/relocs_common.c
+++ b/arch/x86/tools/relocs_common.c
@@ -12,14 +12,13 @@ void die(char *fmt, ...)

static void usage(void)
{
- die("relocs [--abs-syms|--abs-relocs|--reloc-info|--text|--realmode]" \
- " vmlinux\n");
+ die("relocs [--abs-syms|--abs-relocs|--reloc-info|--text|--realmode|--fg-kaslr] vmlinux\n");
}

int main(int argc, char **argv)
{
int show_absolute_syms, show_absolute_relocs, show_reloc_info;
- int as_text, use_real_mode;
+ int as_text, use_real_mode, fgkaslr_opt;
const char *fname;
FILE *fp;
int i;
@@ -30,6 +29,7 @@ int main(int argc, char **argv)
show_reloc_info = 0;
as_text = 0;
use_real_mode = 0;
+ fgkaslr_opt = 0;
fname = NULL;
for (i = 1; i < argc; i++) {
char *arg = argv[i];
@@ -54,6 +54,10 @@ int main(int argc, char **argv)
use_real_mode = 1;
continue;
}
+ if (strcmp(arg, "--fg-kaslr") == 0) {
+ fgkaslr_opt = 1;
+ continue;
+ }
}
else if (!fname) {
fname = arg;
@@ -75,11 +79,11 @@ int main(int argc, char **argv)
if (e_ident[EI_CLASS] == ELFCLASS64)
process_64(fp, use_real_mode, as_text,
show_absolute_syms, show_absolute_relocs,
- show_reloc_info);
+ show_reloc_info, fgkaslr_opt);
else
process_32(fp, use_real_mode, as_text,
show_absolute_syms, show_absolute_relocs,
- show_reloc_info);
+ show_reloc_info, fgkaslr_opt);
fclose(fp);
return 0;
}
--
2.34.1


2022-02-09 20:13:21

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 13/15] module: use a scripted approach for FG-KASLR

Use the same methods and scripts to generate an LD script for every
module containing all the output text sections.
The only difference there is that we don't need to reserve any space
as the memory for every section is being allocated dynamically.

I picked ".ko.lds" extension since there's a fistful of ".lds" files
inside the tree, so I couldn't count all of them as generated /
build artifacts. OTOH, we're limited in heuristics when cleaning as
dotconfig doesn't get included and stuff like ".mod.c" is just being
wiped using `find`.

Signed-off-by: Alexander Lobakin <[email protected]>
---
.gitignore | 1 +
include/asm-generic/vmlinux.lds.h | 12 ++++++++++++
init/Kconfig | 15 ++++++++++++++-
scripts/Makefile.modfinal | 20 +++++++++++++++++---
scripts/generate_text_sections.pl | 9 ++++++++-
scripts/module.lds.S | 14 +++++++++++++-
6 files changed, 65 insertions(+), 6 deletions(-)

diff --git a/.gitignore b/.gitignore
index 7afd412dadd2..d1b48f01037a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -26,6 +26,7 @@
*.gz
*.i
*.ko
+*.ko.lds
*.lex.c
*.ll
*.lst
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index e63d5a69f1bc..9f67660ace18 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -130,6 +130,18 @@
#define TEXT_MAIN .text
#endif

+/*
+ * Same for modules. However, LD_DEAD_CODE_DATA_ELIMINATION doesn't touch
+ * them, so no need to check for it here.
+ */
+#if defined(CONFIG_LTO_CLANG) && !defined(CONFIG_MODULE_FG_KASLR)
+#define TEXT_MAIN_MODULE SECT_WILDCARD(.text)
+#elif defined(CONFIG_MODULE_FG_KASLR)
+#define TEXT_MAIN_MODULE .text.__unused__
+#else
+#define TEXT_MAIN_MODULE .text
+#endif
+
/*
* Used by scripts/generate_text_sections.pl to inject text sections,
* harmless if FG-KASLR is disabled.
diff --git a/init/Kconfig b/init/Kconfig
index 86a2d3fd6390..90951631aa03 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2391,7 +2391,6 @@ config MODULE_FG_KASLR
depends on $(cc-option,-ffunction-sections)
depends on LD_HAS_Z_UNIQUE_SYMBOL || !LIVEPATCH
default FG_KASLR
- depends on BROKEN
help
This option randomizes the module text section by reordering the text
section by function at module load time. In order to use this
@@ -2400,6 +2399,20 @@ config MODULE_FG_KASLR

If unsure, say N.

+config MODULE_FG_KASLR_SHIFT
+ int "Module FG-KASLR granularity (functions per section shift)"
+ depends on MODULE_FG_KASLR
+ range 0 16
+ default 0
+ help
+ This sets the number of functions that will be put in each section
+ as a power of two.
+ Decreasing the value increases the randomization, but also increases
+ the size of the final kernel module due to the amount of sections.
+ 0 means that a separate section will be created for each function.
+ 16 almost disables the randomization, leaving only the manual
+ separation.
+
endif # MODULES

config MODULES_TREE_LOOKUP
diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
index 7f39599e9fae..4ca9d8fc978d 100644
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -28,13 +28,25 @@ quiet_cmd_cc_o_c = CC [M] $@
%.mod.o: %.mod.c FORCE
$(call if_changed_dep,cc_o_c)

+ifdef CONFIG_MODULE_FG_KASLR
+quiet_cmd_gen_modules_lds = GEN [M] $@
+ cmd_gen_modules_lds = \
+ $(PERL) $(srctree)/scripts/generate_text_sections.pl \
+ $(if $(CONFIG_HAVE_ASM_FUNCTION_SECTIONS),-a) \
+ -s $(CONFIG_MODULE_FG_KASLR_SHIFT) $(filter %.o, $^) \
+ < $(filter %.lds, $^) > $@
+
+%.ko.lds: %$(mod-prelink-ext).o scripts/module.lds FORCE
+ $(call if_changed,gen_modules_lds)
+endif
+
ARCH_POSTLINK := $(wildcard $(srctree)/arch/$(SRCARCH)/Makefile.postlink)

quiet_cmd_ld_ko_o = LD [M] $@
cmd_ld_ko_o += \
$(LD) -r $(KBUILD_LDFLAGS) \
$(KBUILD_LDFLAGS_MODULE) $(LDFLAGS_MODULE) \
- -T scripts/module.lds -o $@ $(filter %.o, $^); \
+ -T $(filter %.lds, $^) -o $@ $(filter %.o, $^); \
$(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true)

quiet_cmd_btf_ko = BTF [M] $@
@@ -56,13 +68,15 @@ if_changed_except = $(if $(call newer_prereqs_except,$(2))$(cmd-check), \


# Re-generate module BTFs if either module's .ko or vmlinux changed
-$(modules): %.ko: %$(mod-prelink-ext).o %.mod.o scripts/module.lds $(if $(KBUILD_BUILTIN),vmlinux) FORCE
+$(modules): %.ko: %$(mod-prelink-ext).o %.mod.o
+$(modules): %.ko: $(if $(CONFIG_MODULE_FG_KASLR),%.ko.lds,scripts/module.lds)
+$(modules): %.ko: $(if $(KBUILD_BUILTIN),vmlinux) FORCE
+$(call if_changed_except,ld_ko_o,vmlinux)
ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+$(if $(newer-prereqs),$(call cmd,btf_ko))
endif

-targets += $(modules) $(modules:.ko=.mod.o)
+targets += $(modules) $(modules:.ko=.mod.o) $(modules:.ko=.ko.lds)

# Add FORCE to the prequisites of a target to force it to be always rebuilt.
# ---------------------------------------------------------------------------
diff --git a/scripts/generate_text_sections.pl b/scripts/generate_text_sections.pl
index 999e1b68181f..a05ae9fb0041 100755
--- a/scripts/generate_text_sections.pl
+++ b/scripts/generate_text_sections.pl
@@ -48,6 +48,7 @@ my $readelf = $ENV{'READELF'} || die "$0: ERROR: READELF not set?";
## text sections array
my @sections = ();
my $has_ccf = 0;
+my $vmlinux = 0;

## max alignment found to reserve some space. It would probably be
## better to start from 64, but CONFIG_DEBUG_FORCE_FUNCTION_ALIGN_64B
@@ -78,6 +79,12 @@ sub read_sections {
$has_ccf = 1;
}

+ ## If we're processing a module, don't reserve any space
+ ## at the end as its sections are being allocated separately.
+ if ($name eq ".sched.text") {
+ $vmlinux = 1;
+ }
+
if (!($name =~ /^\.text(\.(?!hot\.|unknown\.|unlikely\.|.san\.)[0-9a-zA-Z_]*){1,2}((\.constprop|\.isra|\.part)\.[0-9]){0,2}(|\.[0-9cfi]*)$/)) {
next;
}
@@ -141,7 +148,7 @@ sub print_reserve {
## If we have text sections aligned with 128 bytes or more, make
## sure we reserve some space for them to not overlap _etext
## while shuffling sections.
- if (!$count) {
+ if (!$vmlinux or !$count) {
return;
}

diff --git a/scripts/module.lds.S b/scripts/module.lds.S
index 1d0e1e4dc3d2..6e957aa614b1 100644
--- a/scripts/module.lds.S
+++ b/scripts/module.lds.S
@@ -3,6 +3,11 @@
* Archs are free to supply their own linker scripts. ld will
* combine them automatically.
*/
+
+#include <asm-generic/vmlinux.lds.h>
+
+#undef SANITIZER_DISCARDS
+
#ifdef CONFIG_CFI_CLANG
# include <asm/page.h>
# define ALIGN_CFI ALIGN(PAGE_SIZE)
@@ -58,9 +63,16 @@ SECTIONS {
*/
.text : ALIGN_CFI {
*(.text.__cfi_check)
- *(.text .text.[0-9a-zA-Z_]* .text..L.cfi*)
+ *(TEXT_MAIN_MODULE)
+ *(.text..L.cfi.jumptable .text..L.cfi.jumptable.*)
+ }
+#elif defined(CONFIG_MODULE_FG_KASLR)
+ .text : {
+ *(TEXT_MAIN_MODULE)
}
#endif
+
+ TEXT_FG_KASLR
}

/* bring in arch-specific sections */
--
2.34.1


2022-02-09 20:18:40

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 10/15] FG-KASLR: use a scripted approach to handle .text.* sections

Instead of relying on the linker and his heuristics about where to
place (orphan) .text.* section, use a script to read vmlinux.o and
generate a new .tmp_vmlinux.lds which will contain an entry for
each of them. It relies on a magic marker inside the preprocessed
vmlinux.lds (which is harmless in case FG-KASLR is disabled) and
injects a list of input text sections there.

As a bonus, this approach allows us to configure FG-KASLR in terms
of number of functions per each section. The zero value means one
section per each functions, it is the strongest choice, but the
resulting vmlinux also has the biggest size here, as well as the
total number of sections and the boottime delay (which is still
barely noticeable). The values of 4-8 are still strong enough and
allows to save some space, and so on.

We also keep tracking the maximum alignment we found while
traversing through the readelf output and the number of times we
spotted it. It's actual only for values >= 128 and is required to
reserve some space between the last .text.* section and the _etext
marker.
The reason is that e.g. x86 has at least 3 asm sections (4 with
ClangCFI) aligned to 4096, and when mixing them with the small
sections, we could go past the _etext and render the kernel
unbootable. This reserved space ensures this won't happen.

When CONFIG_HAVE_ASM_FUNCTION_SECTIONS=y, the generated script makes
sure you don't have anything in "plain" ".text" to not leak random
code. This assertion is omitted otherwise.

Signed-off-by: Alexander Lobakin <[email protected]>
---
arch/x86/kernel/vmlinux.lds.S | 4 +-
include/asm-generic/vmlinux.lds.h | 6 ++
init/Kconfig | 14 +++
scripts/generate_text_sections.pl | 165 ++++++++++++++++++++++++++++++
scripts/link-vmlinux.sh | 30 +++++-
5 files changed, 217 insertions(+), 2 deletions(-)
create mode 100755 scripts/generate_text_sections.pl

diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 54f16801e9d6..06ba33f5bc58 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -146,9 +146,11 @@ SECTIONS
#endif
} :text =0xcccc

+ TEXT_FG_KASLR
+
/* End of text section, which should occupy whole number of pages */
- _etext = .;
. = ALIGN(PAGE_SIZE);
+ _etext = .;

X86_ALIGN_RODATA_BEGIN
RO_DATA(PAGE_SIZE)
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 586465b2abb2..e63d5a69f1bc 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -130,6 +130,12 @@
#define TEXT_MAIN .text
#endif

+/*
+ * Used by scripts/generate_text_sections.pl to inject text sections,
+ * harmless if FG-KASLR is disabled.
+ */
+#define TEXT_FG_KASLR __fg_kaslr_magic = .;
+
/*
* GCC 4.5 and later have a 32 bytes section alignment for structures.
* Except GCC 4.9, that feels the need to align on 64 bytes.
diff --git a/init/Kconfig b/init/Kconfig
index 26f9a6e52dbd..5fbd1c294df4 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2076,6 +2076,20 @@ config FG_KASLR

If unsure, say N.

+config FG_KASLR_SHIFT
+ int "FG-KASLR granularity (number of functions per section shift)"
+ depends on FG_KASLR
+ range 0 16
+ default 0
+ help
+ This sets the number of functions that will be put in each section
+ as a power of two.
+ Decreasing the value increases the randomization, but also increases
+ the size of the final kernel/vmlinux due to the amount of sections.
+ 0 means that a separate section will be created for each function.
+ 16 almost disables the randomization, leaving only the manual
+ separation.
+
endmenu # General setup

source "arch/Kconfig"
diff --git a/scripts/generate_text_sections.pl b/scripts/generate_text_sections.pl
new file mode 100755
index 000000000000..999e1b68181f
--- /dev/null
+++ b/scripts/generate_text_sections.pl
@@ -0,0 +1,165 @@
+#!/usr/bin/env perl
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Generates a new LD script with every .text.* section described for FG-KASLR
+# to avoid orphan/heuristic section placement and double-checks we don't have
+# any symbols in plain .text section.
+#
+# Copyright (C) 2021-2022, Intel Corporation.
+# Author: Alexander Lobakin <[email protected]>
+#
+
+use strict;
+use warnings;
+
+## parameters
+my $add_assert = 0;
+my $expecting = 0;
+my $shift = 0;
+my $file;
+
+foreach (@ARGV) {
+ if ($_ eq '-a') {
+ $add_assert = 1;
+ } elsif ($_ eq '-s') {
+ $expecting = 1;
+ } elsif ($expecting) {
+ $shift = $_ + 0;
+ if ($shift < 0) {
+ $shift = 0;
+ } elsif ($shift > 16) {
+ $shift = 16;
+ }
+ $expecting = 0;
+ } elsif (!defined($file)) {
+ $file = $_;
+ } else {
+ die "$0: usage: $0 [-a] [-s shift] binary < linker script";
+ }
+}
+
+if (!defined($file)) {
+ die "$0: usage: $0 [-a] [-s shift] binary < linker script";
+}
+
+## environment
+my $readelf = $ENV{'READELF'} || die "$0: ERROR: READELF not set?";
+
+## text sections array
+my @sections = ();
+my $has_ccf = 0;
+
+## max alignment found to reserve some space. It would probably be
+## better to start from 64, but CONFIG_DEBUG_FORCE_FUNCTION_ALIGN_64B
+## (which aligns every function to 64b) would explode the $count then
+my $max_align = 128;
+my $count = 0;
+
+sub read_sections {
+ open(my $fh, "\"$readelf\" -SW \"$file\" 2>/dev/null |")
+ or die "$0: ERROR: failed to execute \"$readelf\": $!";
+
+ while (<$fh>) {
+ my $name;
+ my $align;
+ chomp;
+
+ ($name, $align) = $_ =~ /^\s*\[[\s0-9]*\]\s*(\.\S*)\s*[A-Z]*\s*[0-9a-f]{16}\s*[0-9a-f]*\s*[0-9a-f]*\s*[0-9a-f]*\s*[0-9a-f]{2}\s*[A-Z]{2}\s*[0-9]\s*[0-9]\s*([0-9]*)$/;
+
+ if (!defined($name)) {
+ next;
+ }
+
+ ## Clang 13 onwards emits __cfi_check_fail only on final
+ ## linking, so it won't appear in .o files and will be
+ ## missing in @sections. Add it manually to prevent
+ ## spawning orphans.
+ if ($name eq ".text.__cfi_check_fail") {
+ $has_ccf = 1;
+ }
+
+ if (!($name =~ /^\.text(\.(?!hot\.|unknown\.|unlikely\.|.san\.)[0-9a-zA-Z_]*){1,2}((\.constprop|\.isra|\.part)\.[0-9]){0,2}(|\.[0-9cfi]*)$/)) {
+ next;
+ }
+
+ if ($align > $max_align) {
+ $max_align = $align;
+ $count = 1;
+ } elsif ($align == $max_align) {
+ $count++;
+ }
+
+ push(@sections, $name);
+ }
+
+ close($fh);
+
+ if (!$has_ccf) {
+ push(@sections, ".text.__cfi_check_fail");
+ }
+
+ @sections = sort @sections;
+}
+
+sub print_sections {
+ my $fps = 1 << $shift;
+ my $counter = 1;
+
+ print "\t.text.0 : ALIGN(16) {\n";
+ print "\t\t*(.text)\n";
+ print "\t}\n";
+
+ ## If we have asm function sections, we shouldn't have anything
+ ## in here.
+ if ($add_assert) {
+ print "\tASSERT(SIZEOF(.text.0) == 0, \"Plain .text is not empty!\")\n\n";
+ }
+
+ if (!@sections) {
+ return;
+ }
+
+ while () {
+ print "\t.text.$counter : ALIGN(16) {\n";
+
+ my @a = (($counter - 1) * $fps .. ($counter * $fps) - 1);
+ for (@a) {
+ print "\t\t*($sections[$_])\n";
+
+ if ($sections[$_] eq $sections[-1]) {
+ print "\t}\n";
+ return;
+ }
+ }
+
+ print "\t}\n";
+ $counter++;
+ }
+}
+
+sub print_reserve {
+ ## If we have text sections aligned with 128 bytes or more, make
+ ## sure we reserve some space for them to not overlap _etext
+ ## while shuffling sections.
+ if (!$count) {
+ return;
+ }
+
+ print "\n\t. += $max_align * $count;\n";
+}
+
+sub print_lds {
+ while (<STDIN>) {
+ if ($_ =~ /^\s*__fg_kaslr_magic = \.;$/) {
+ print_sections();
+ print_reserve();
+ } else {
+ print $_;
+ }
+ }
+}
+
+## main
+
+read_sections();
+print_lds();
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 666f7bbc13eb..701cf540c12e 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -70,6 +70,23 @@ gen_symversions()
done
}

+# If CONFIG_FG_KASLR is selected, generate a linker script which will
+# declare all custom text sections for future boottime shuffling
+gen_text_sections()
+{
+ local shift=$(sed -n 's/^CONFIG_FG_KASLR_SHIFT=\(.*\)$/\1/p' include/config/auto.conf)
+ local assert=""
+
+ is_enabled CONFIG_HAVE_ASM_FUNCTION_SECTIONS && assert="-a"
+
+ info GEN .tmp_vmlinux.lds
+
+ ${PERL} ${srctree}/scripts/generate_text_sections.pl \
+ ${assert} -s "${shift}" vmlinux.o \
+ < "${objtree}/${KBUILD_LDS}" \
+ > .tmp_vmlinux.lds
+}
+
# Link of vmlinux.o used for section mismatch analysis
# ${1} output file
modpost_link()
@@ -162,12 +179,19 @@ vmlinux_link()
local ld
local ldflags
local ldlibs
+ local lds

info LD ${output}

# skip output file argument
shift

+ if is_enabled CONFIG_FG_KASLR; then
+ lds=".tmp_vmlinux.lds"
+ else
+ lds="${objtree}/${KBUILD_LDS}"
+ fi
+
if is_enabled CONFIG_LTO_CLANG; then
# Use vmlinux.o instead of performing the slow LTO link again.
objs=vmlinux.o
@@ -189,7 +213,7 @@ vmlinux_link()
ldlibs=
fi

- ldflags="${ldflags} ${wl}--script=${objtree}/${KBUILD_LDS}"
+ ldflags="${ldflags} ${wl}--script=${lds}"

# The kallsyms linking does not need debug symbols included.
if [ "$output" != "${output#.tmp_vmlinux.kallsyms}" ] ; then
@@ -346,6 +370,10 @@ info GEN modules.builtin
tr '\0' '\n' < modules.builtin.modinfo | sed -n 's/^[[:alnum:]:_]*\.file=//p' |
tr ' ' '\n' | uniq | sed -e 's:^:kernel/:' -e 's/$/.ko/' > modules.builtin

+if is_enabled CONFIG_FG_KASLR; then
+ gen_text_sections
+fi
+
btf_vmlinux_bin_o=""
if is_enabled CONFIG_DEBUG_INFO_BTF; then
btf_vmlinux_bin_o=.btf.vmlinux.bin.o
--
2.34.1


2022-02-09 20:19:15

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 05/15] x86: support asm function sections

Address places which need special care and enable
CONFIG_ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS.

Notably:
- propagate `--sectname-subst` to KBUILD_AFLAGS in
x86/boot/Makefile and x86/boot/compressed/Makefile as both
override them;
- symbols starting with a dot (like ".Lrelocated") should be
handled manually with SYM_*_START_SECT(.Lrelocated, relocated)
as "two dots" is a special (and CPP doesn't want to concatenate
two dots in general);
- some symbols explicitly need to reside in one section (like
kexec control code, hibernation page etc.);
- macros creating aliases for functions (like __memcpy() for
memcpy() etc.) should go after the main declaration (as
aliases should be declared in the same section and they
don't have SYM_PUSH_SECTION() inside);
- things like ".org", ".align" should be manually pushed to
the same section the next symbol goes to;
- expand indirect_thunk wildcards in vmlinux.lds.S to catch
symbols back into the "main" section;
- inline ASM functions like __raw_callee*() should be pushed
manually as well.

With these changes and `-ffunction-sections` enabled, "plain"
".text" section is empty which means that everything works
right as expected.

Signed-off-by: Alexander Lobakin <[email protected]>
---
arch/x86/Kconfig | 1 +
arch/x86/boot/Makefile | 1 +
arch/x86/boot/compressed/Makefile | 1 +
arch/x86/boot/compressed/head_32.S | 2 +-
arch/x86/boot/compressed/head_64.S | 32 ++++++++++++-------
arch/x86/boot/pmjump.S | 2 +-
arch/x86/crypto/aesni-intel_asm.S | 4 +--
arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 4 +++
arch/x86/include/asm/paravirt.h | 2 ++
arch/x86/include/asm/qspinlock_paravirt.h | 2 ++
arch/x86/kernel/head_32.S | 4 +--
arch/x86/kernel/head_64.S | 4 +--
arch/x86/kernel/kprobes/core.c | 2 ++
arch/x86/kernel/kvm.c | 2 ++
arch/x86/kernel/relocate_kernel_32.S | 10 +++---
arch/x86/kernel/relocate_kernel_64.S | 12 ++++---
arch/x86/kernel/vmlinux.lds.S | 2 +-
arch/x86/kvm/emulate.c | 7 +++-
arch/x86/lib/copy_user_64.S | 2 +-
arch/x86/lib/error-inject.c | 2 ++
arch/x86/lib/getuser.S | 5 ++-
arch/x86/lib/memcpy_64.S | 4 +--
arch/x86/lib/memmove_64.S | 5 ++-
arch/x86/lib/memset_64.S | 5 +--
arch/x86/lib/putuser.S | 2 +-
arch/x86/power/hibernate_asm_32.S | 10 +++---
arch/x86/power/hibernate_asm_64.S | 10 +++---
27 files changed, 89 insertions(+), 50 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 391c4cac8958..f6bb48d41349 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -102,6 +102,7 @@ config X86
select ARCH_MIGHT_HAVE_PC_SERIO
select ARCH_STACKWALK
select ARCH_SUPPORTS_ACPI
+ select ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS
select ARCH_SUPPORTS_ATOMIC_RMW
select ARCH_SUPPORTS_DEBUG_PAGEALLOC
select ARCH_SUPPORTS_PAGE_TABLE_CHECK if X86_64
diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile
index b5aecb524a8a..080990b09f06 100644
--- a/arch/x86/boot/Makefile
+++ b/arch/x86/boot/Makefile
@@ -68,6 +68,7 @@ targets += cpustr.h

KBUILD_CFLAGS := $(REALMODE_CFLAGS) -D_SETUP
KBUILD_AFLAGS := $(KBUILD_CFLAGS) -D__ASSEMBLY__
+KBUILD_AFLAGS += $(SECSUBST_AFLAGS)
KBUILD_CFLAGS += $(call cc-option,-fmacro-prefix-map=$(srctree)/=)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
GCOV_PROFILE := n
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index 6115274fe10f..e7ee94d5f55a 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -58,6 +58,7 @@ KBUILD_CFLAGS += -include $(srctree)/include/linux/hidden.h
CFLAGS_sev.o += -I$(objtree)/arch/x86/lib/

KBUILD_AFLAGS := $(KBUILD_CFLAGS) -D__ASSEMBLY__
+KBUILD_AFLAGS += $(SECSUBST_AFLAGS)
GCOV_PROFILE := n
UBSAN_SANITIZE :=n

diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S
index 659fad53ca82..a3a667f5e5cd 100644
--- a/arch/x86/boot/compressed/head_32.S
+++ b/arch/x86/boot/compressed/head_32.S
@@ -158,8 +158,8 @@ SYM_FUNC_START_ALIAS(efi_stub_entry)
call efi_main
/* efi_main returns the possibly relocated address of startup_32 */
jmp *%eax
-SYM_FUNC_END(efi32_stub_entry)
SYM_FUNC_END_ALIAS(efi_stub_entry)
+SYM_FUNC_END(efi32_stub_entry)
#endif

.text
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index fd9441f40457..b6fb27fd7abd 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -72,7 +72,7 @@
#define rva(X) ((X) - startup_32)

.code32
-SYM_FUNC_START(startup_32)
+SYM_FUNC_START_SECT(startup_32, startup)
/*
* 32bit entry is 0 and it is ABI so immutable!
* If we come here directly from a bootloader,
@@ -297,8 +297,10 @@ SYM_FUNC_START(startup_32)
SYM_FUNC_END(startup_32)

#ifdef CONFIG_EFI_MIXED
+SYM_PUSH_SECTION(startup)
.org 0x190
-SYM_FUNC_START(efi32_stub_entry)
+SYM_POP_SECTION()
+SYM_FUNC_START_SECT(efi32_stub_entry, startup)
add $0x4, %esp /* Discard return address */
popl %ecx
popl %edx
@@ -332,8 +334,10 @@ SYM_FUNC_END(efi32_stub_entry)
#endif

.code64
+SYM_PUSH_SECTION(startup)
.org 0x200
-SYM_CODE_START(startup_64)
+SYM_POP_SECTION()
+SYM_CODE_START_SECT(startup_64, startup)
/*
* 64bit entry is 0x200 and it is ABI so immutable!
* We come here either from startup_32 or directly from a
@@ -533,8 +537,10 @@ trampoline_return:
SYM_CODE_END(startup_64)

#ifdef CONFIG_EFI_STUB
+SYM_PUSH_SECTION(startup)
.org 0x390
-SYM_FUNC_START(efi64_stub_entry)
+SYM_POP_SECTION()
+SYM_FUNC_START_SECT(efi64_stub_entry, startup)
SYM_FUNC_START_ALIAS(efi_stub_entry)
and $~0xf, %rsp /* realign the stack */
movq %rdx, %rbx /* save boot_params pointer */
@@ -542,12 +548,12 @@ SYM_FUNC_START_ALIAS(efi_stub_entry)
movq %rbx,%rsi
leaq rva(startup_64)(%rax), %rax
jmp *%rax
-SYM_FUNC_END(efi64_stub_entry)
SYM_FUNC_END_ALIAS(efi_stub_entry)
+SYM_FUNC_END(efi64_stub_entry)
#endif

.text
-SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated)
+SYM_FUNC_START_LOCAL_NOALIGN_SECT(.Lrelocated, relocated)

/*
* Clear BSS (stack is currently empty)
@@ -670,20 +676,22 @@ SYM_CODE_START(trampoline_32bit_src)
SYM_CODE_END(trampoline_32bit_src)

.code64
-SYM_FUNC_START_LOCAL_NOALIGN(.Lpaging_enabled)
+SYM_FUNC_START_LOCAL_NOALIGN_SECT(.Lpaging_enabled, trampoline_32bit_src)
/* Return from the trampoline */
jmp *%rdi
SYM_FUNC_END(.Lpaging_enabled)

+SYM_PUSH_SECTION(trampoline_32bit_src)
/*
* The trampoline code has a size limit.
* Make sure we fail to compile if the trampoline code grows
* beyond TRAMPOLINE_32BIT_CODE_SIZE bytes.
*/
.org trampoline_32bit_src + TRAMPOLINE_32BIT_CODE_SIZE
+SYM_POP_SECTION()

.code32
-SYM_FUNC_START_LOCAL_NOALIGN(.Lno_longmode)
+SYM_FUNC_START_LOCAL_NOALIGN_SECT(.Lno_longmode, no_longmode)
/* This isn't an x86-64 CPU, so hang intentionally, we cannot continue */
1:
hlt
@@ -747,7 +755,7 @@ SYM_DATA(efi_is64, .byte 1)

__HEAD
.code32
-SYM_FUNC_START(efi32_pe_entry)
+SYM_FUNC_START_SECT(efi32_pe_entry, startup)
/*
* efi_status_t efi32_pe_entry(efi_handle_t image_handle,
* efi_system_table_32_t *sys_table)
@@ -839,7 +847,7 @@ SYM_DATA_END(loaded_image_proto)
*
* Physical offset is expected in %ebp
*/
-SYM_FUNC_START(startup32_set_idt_entry)
+SYM_FUNC_START_SECT(startup32_set_idt_entry, startup)
push %ebx
push %ecx

@@ -872,7 +880,7 @@ SYM_FUNC_START(startup32_set_idt_entry)
SYM_FUNC_END(startup32_set_idt_entry)
#endif

-SYM_FUNC_START(startup32_load_idt)
+SYM_FUNC_START_SECT(startup32_load_idt, startup)
#ifdef CONFIG_AMD_MEM_ENCRYPT
/* #VC handler */
leal rva(startup32_vc_handler)(%ebp), %eax
@@ -904,7 +912,7 @@ SYM_FUNC_END(startup32_load_idt)
* succeed. An incorrect C-bit position will map all memory unencrypted, so that
* the compare will use the encrypted random data and fail.
*/
-SYM_FUNC_START(startup32_check_sev_cbit)
+SYM_FUNC_START_SECT(startup32_check_sev_cbit, startup)
#ifdef CONFIG_AMD_MEM_ENCRYPT
pushl %eax
pushl %ebx
diff --git a/arch/x86/boot/pmjump.S b/arch/x86/boot/pmjump.S
index cbec8bd0841f..e647c17000a9 100644
--- a/arch/x86/boot/pmjump.S
+++ b/arch/x86/boot/pmjump.S
@@ -46,7 +46,7 @@ SYM_FUNC_END(protected_mode_jump)

.code32
.section ".text32","ax"
-SYM_FUNC_START_LOCAL_NOALIGN(.Lin_pm32)
+SYM_FUNC_START_LOCAL_NOALIGN_SECT(.Lin_pm32, in_pm32)
# Set up data segments for flat 32-bit mode
movl %ecx, %ds
movl %ecx, %es
diff --git a/arch/x86/crypto/aesni-intel_asm.S b/arch/x86/crypto/aesni-intel_asm.S
index 363699dd7220..6b92beb7820a 100644
--- a/arch/x86/crypto/aesni-intel_asm.S
+++ b/arch/x86/crypto/aesni-intel_asm.S
@@ -1752,8 +1752,8 @@ SYM_FUNC_END(aesni_gcm_finalize)
#endif


-SYM_FUNC_START_LOCAL_ALIAS(_key_expansion_128)
SYM_FUNC_START_LOCAL(_key_expansion_256a)
+SYM_FUNC_START_LOCAL_ALIAS(_key_expansion_128)
pshufd $0b11111111, %xmm1, %xmm1
shufps $0b00010000, %xmm0, %xmm4
pxor %xmm4, %xmm0
@@ -1763,8 +1763,8 @@ SYM_FUNC_START_LOCAL(_key_expansion_256a)
movaps %xmm0, (TKEYP)
add $0x10, TKEYP
RET
-SYM_FUNC_END(_key_expansion_256a)
SYM_FUNC_END_ALIAS(_key_expansion_128)
+SYM_FUNC_END(_key_expansion_256a)

SYM_FUNC_START_LOCAL(_key_expansion_192a)
pshufd $0b01010101, %xmm1, %xmm1
diff --git a/arch/x86/crypto/poly1305-x86_64-cryptogams.pl b/arch/x86/crypto/poly1305-x86_64-cryptogams.pl
index 71fae5a09e56..221a4596f390 100644
--- a/arch/x86/crypto/poly1305-x86_64-cryptogams.pl
+++ b/arch/x86/crypto/poly1305-x86_64-cryptogams.pl
@@ -421,6 +421,7 @@ my ($H0,$H1,$H2,$H3,$H4, $T0,$T1,$T2,$T3,$T4, $D0,$D1,$D2,$D3,$D4, $MASK) =
map("%xmm$_",(0..15));

$code.=<<___;
+SYM_PUSH_SECTION(__poly1305_block)
.type __poly1305_block,\@abi-omnipotent
.align 32
__poly1305_block:
@@ -431,7 +432,9 @@ $code.=<<___;
pop $ctx
ret
.size __poly1305_block,.-__poly1305_block
+SYM_POP_SECTION()

+SYM_PUSH_SECTION(__poly1305_init_avx)
.type __poly1305_init_avx,\@abi-omnipotent
.align 32
__poly1305_init_avx:
@@ -596,6 +599,7 @@ __poly1305_init_avx:
pop %rbp
ret
.size __poly1305_init_avx,.-__poly1305_init_avx
+SYM_POP_SECTION()
___

&declare_function("poly1305_blocks_avx", 32, 4);
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index 0d76502cc6f5..3efdac789bc4 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -663,6 +663,7 @@ bool __raw_callee_save___native_vcpu_is_preempted(long cpu);
extern typeof(func) __raw_callee_save_##func; \
\
asm(".pushsection " section ", \"ax\";" \
+ ASM_PUSH_SECTION(__raw_callee_save_##func) ";" \
".globl " PV_THUNK_NAME(func) ";" \
".type " PV_THUNK_NAME(func) ", @function;" \
PV_THUNK_NAME(func) ":" \
@@ -673,6 +674,7 @@ bool __raw_callee_save___native_vcpu_is_preempted(long cpu);
FRAME_END \
ASM_RET \
".size " PV_THUNK_NAME(func) ", .-" PV_THUNK_NAME(func) ";" \
+ ASM_POP_SECTION() ";" \
".popsection")

#define PV_CALLEE_SAVE_REGS_THUNK(func) \
diff --git a/arch/x86/include/asm/qspinlock_paravirt.h b/arch/x86/include/asm/qspinlock_paravirt.h
index 1474cf96251d..3fde623bbcb2 100644
--- a/arch/x86/include/asm/qspinlock_paravirt.h
+++ b/arch/x86/include/asm/qspinlock_paravirt.h
@@ -35,6 +35,7 @@ PV_CALLEE_SAVE_REGS_THUNK(__pv_queued_spin_unlock_slowpath);
* rdx = internal variable (set to 0)
*/
asm (".pushsection .text;"
+ ASM_PUSH_SECTION(__raw_callee_save___pv_queued_spin_unlock) ";"
".globl " PV_UNLOCK ";"
".type " PV_UNLOCK ", @function;"
".align 4,0x90;"
@@ -58,6 +59,7 @@ asm (".pushsection .text;"
FRAME_END
ASM_RET
".size " PV_UNLOCK ", .-" PV_UNLOCK ";"
+ ASM_POP_SECTION() ";"
".popsection");

#else /* CONFIG_64BIT */
diff --git a/arch/x86/kernel/head_32.S b/arch/x86/kernel/head_32.S
index eb8656bac99b..d58422148481 100644
--- a/arch/x86/kernel/head_32.S
+++ b/arch/x86/kernel/head_32.S
@@ -342,7 +342,7 @@ setup_once:
andl $0,setup_once_ref /* Once is enough, thanks */
RET

-SYM_FUNC_START(early_idt_handler_array)
+SYM_FUNC_START_SECT(early_idt_handler_array, early_idt_handler)
# 36(%esp) %eflags
# 32(%esp) %cs
# 28(%esp) %eip
@@ -359,7 +359,7 @@ SYM_FUNC_START(early_idt_handler_array)
.endr
SYM_FUNC_END(early_idt_handler_array)

-SYM_CODE_START_LOCAL(early_idt_handler_common)
+SYM_CODE_START_LOCAL_SECT(early_idt_handler_common, early_idt_handler)
/*
* The stack is the hardware frame, an error code or zero, and the
* vector number.
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index 9c63fc5988cd..a19b6fa2bf87 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -367,7 +367,7 @@ SYM_DATA(initial_stack, .quad init_thread_union + THREAD_SIZE - FRAME_SIZE)
__FINITDATA

__INIT
-SYM_CODE_START(early_idt_handler_array)
+SYM_CODE_START_SECT(early_idt_handler_array, early_idt_handler)
i = 0
.rept NUM_EXCEPTION_VECTORS
.if ((EXCEPTION_ERRCODE_MASK >> i) & 1) == 0
@@ -385,7 +385,7 @@ SYM_CODE_START(early_idt_handler_array)
UNWIND_HINT_IRET_REGS offset=16
SYM_CODE_END(early_idt_handler_array)

-SYM_CODE_START_LOCAL(early_idt_handler_common)
+SYM_CODE_START_LOCAL_SECT(early_idt_handler_common, early_idt_handler)
/*
* The stack is the hardware frame, an error code or zero, and the
* vector number.
diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c
index 6290712cb36d..e0be6bc76337 100644
--- a/arch/x86/kernel/kprobes/core.c
+++ b/arch/x86/kernel/kprobes/core.c
@@ -1019,6 +1019,7 @@ NOKPROBE_SYMBOL(kprobe_int3_handler);
*/
asm(
".text\n"
+ ASM_PUSH_SECTION(__kretprobe_trampoline) "\n"
".global __kretprobe_trampoline\n"
".type __kretprobe_trampoline, @function\n"
"__kretprobe_trampoline:\n"
@@ -1053,6 +1054,7 @@ asm(
#endif
ASM_RET
".size __kretprobe_trampoline, .-__kretprobe_trampoline\n"
+ ASM_POP_SECTION() "\n"
);
NOKPROBE_SYMBOL(__kretprobe_trampoline);
/*
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index a438217cbfac..6a497d5647e7 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -1021,6 +1021,7 @@ extern bool __raw_callee_save___kvm_vcpu_is_preempted(long);
*/
asm(
".pushsection .text;"
+ASM_PUSH_SECTION(__raw_callee_save___kvm_vcpu_is_preempted) ";"
".global __raw_callee_save___kvm_vcpu_is_preempted;"
".type __raw_callee_save___kvm_vcpu_is_preempted, @function;"
"__raw_callee_save___kvm_vcpu_is_preempted:"
@@ -1029,6 +1030,7 @@ asm(
"setne %al;"
"ret;"
".size __raw_callee_save___kvm_vcpu_is_preempted, .-__raw_callee_save___kvm_vcpu_is_preempted;"
+ASM_POP_SECTION() ";"
".popsection");

#endif
diff --git a/arch/x86/kernel/relocate_kernel_32.S b/arch/x86/kernel/relocate_kernel_32.S
index fcc8a7699103..ff36f21e665a 100644
--- a/arch/x86/kernel/relocate_kernel_32.S
+++ b/arch/x86/kernel/relocate_kernel_32.S
@@ -35,7 +35,7 @@
#define CP_PA_BACKUP_PAGES_MAP DATA(0x1c)

.text
-SYM_CODE_START_NOALIGN(relocate_kernel)
+SYM_CODE_START_NOALIGN_SECT(relocate_kernel, kexec_control_code)
/* Save the CPU context, used for jumping back */

pushl %ebx
@@ -94,7 +94,7 @@ SYM_CODE_START_NOALIGN(relocate_kernel)
RET
SYM_CODE_END(relocate_kernel)

-SYM_CODE_START_LOCAL_NOALIGN(identity_mapped)
+SYM_CODE_START_LOCAL_NOALIGN_SECT(identity_mapped, kexec_control_code)
/* set return address to 0 if not preserving context */
pushl $0
/* store the start address on the stack */
@@ -193,7 +193,7 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped)
RET
SYM_CODE_END(identity_mapped)

-SYM_CODE_START_LOCAL_NOALIGN(virtual_mapped)
+SYM_CODE_START_LOCAL_NOALIGN_SECT(virtual_mapped, kexec_control_code)
movl CR4(%edi), %eax
movl %eax, %cr4
movl CR3(%edi), %eax
@@ -212,7 +212,7 @@ SYM_CODE_START_LOCAL_NOALIGN(virtual_mapped)
SYM_CODE_END(virtual_mapped)

/* Do the copies */
-SYM_CODE_START_LOCAL_NOALIGN(swap_pages)
+SYM_CODE_START_LOCAL_NOALIGN_SECT(swap_pages, kexec_control_code)
movl 8(%esp), %edx
movl 4(%esp), %ecx
pushl %ebp
@@ -274,5 +274,7 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages)
RET
SYM_CODE_END(swap_pages)

+SYM_PUSH_SECTION(kexec_control_code)
.globl kexec_control_code_size
.set kexec_control_code_size, . - relocate_kernel
+SYM_POP_SECTION()
diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocate_kernel_64.S
index 399f075ccdc4..fb8ff461436e 100644
--- a/arch/x86/kernel/relocate_kernel_64.S
+++ b/arch/x86/kernel/relocate_kernel_64.S
@@ -38,9 +38,11 @@
#define CP_PA_BACKUP_PAGES_MAP DATA(0x30)

.text
+SYM_PUSH_SECTION(kexec_control_code)
.align PAGE_SIZE
.code64
-SYM_CODE_START_NOALIGN(relocate_kernel)
+SYM_POP_SECTION()
+SYM_CODE_START_NOALIGN_SECT(relocate_kernel, kexec_control_code)
UNWIND_HINT_EMPTY
/*
* %rdi indirection_page
@@ -107,7 +109,7 @@ SYM_CODE_START_NOALIGN(relocate_kernel)
RET
SYM_CODE_END(relocate_kernel)

-SYM_CODE_START_LOCAL_NOALIGN(identity_mapped)
+SYM_CODE_START_LOCAL_NOALIGN_SECT(identity_mapped, kexec_control_code)
UNWIND_HINT_EMPTY
/* set return address to 0 if not preserving context */
pushq $0
@@ -213,7 +215,7 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped)
RET
SYM_CODE_END(identity_mapped)

-SYM_CODE_START_LOCAL_NOALIGN(virtual_mapped)
+SYM_CODE_START_LOCAL_NOALIGN_SECT(virtual_mapped, kexec_control_code)
UNWIND_HINT_EMPTY
movq RSP(%r8), %rsp
movq CR4(%r8), %rax
@@ -235,7 +237,7 @@ SYM_CODE_START_LOCAL_NOALIGN(virtual_mapped)
SYM_CODE_END(virtual_mapped)

/* Do the copies */
-SYM_CODE_START_LOCAL_NOALIGN(swap_pages)
+SYM_CODE_START_LOCAL_NOALIGN_SECT(swap_pages, kexec_control_code)
UNWIND_HINT_EMPTY
movq %rdi, %rcx /* Put the page_list in %rcx */
xorl %edi, %edi
@@ -291,5 +293,7 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages)
RET
SYM_CODE_END(swap_pages)

+SYM_PUSH_SECTION(kexec_control_code)
.globl kexec_control_code_size
.set kexec_control_code_size, . - relocate_kernel
+SYM_POP_SECTION()
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 27f830345b6f..5550bd68f6e7 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -141,7 +141,7 @@ SECTIONS

#ifdef CONFIG_RETPOLINE
__indirect_thunk_start = .;
- *(.text.__x86.indirect_thunk)
+ *(SECT_WILDCARD(.text.__x86.indirect_thunk))
__indirect_thunk_end = .;
#endif
} :text =0xcccc
diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index 40da8c7f3019..85ce58b041ac 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -309,6 +309,7 @@ static void invalidate_registers(struct x86_emulate_ctxt *ctxt)
static int fastop(struct x86_emulate_ctxt *ctxt, fastop_t fop);

#define __FOP_FUNC(name) \
+ __ASM_PUSH_SECTION(name) "\n\t" \
".align " __stringify(FASTOP_SIZE) " \n\t" \
".type " name ", @function \n\t" \
name ":\n\t"
@@ -318,7 +319,8 @@ static int fastop(struct x86_emulate_ctxt *ctxt, fastop_t fop);

#define __FOP_RET(name) \
"11: " ASM_RET \
- ".size " name ", .-" name "\n\t"
+ ".size " name ", .-" name "\n\t" \
+ ASM_POP_SECTION() "\n\t"

#define FOP_RET(name) \
__FOP_RET(#name)
@@ -326,11 +328,13 @@ static int fastop(struct x86_emulate_ctxt *ctxt, fastop_t fop);
#define FOP_START(op) \
extern void em_##op(struct fastop *fake); \
asm(".pushsection .text, \"ax\" \n\t" \
+ ASM_PUSH_SECTION(em_##op) "\n\t" \
".global em_" #op " \n\t" \
".align " __stringify(FASTOP_SIZE) " \n\t" \
"em_" #op ":\n\t"

#define FOP_END \
+ ASM_POP_SECTION() "\n\t" \
".popsection")

#define __FOPNOP(name) \
@@ -430,6 +434,7 @@ static int fastop(struct x86_emulate_ctxt *ctxt, fastop_t fop);

/* Special case for SETcc - 1 instruction per cc */
#define FOP_SETCC(op) \
+ ASM_PUSH_SECTION(op) "\n\t" \
".align 4 \n\t" \
".type " #op ", @function \n\t" \
#op ": \n\t" \
diff --git a/arch/x86/lib/copy_user_64.S b/arch/x86/lib/copy_user_64.S
index 8ca5ecf16dc4..ddbd6b5b7b47 100644
--- a/arch/x86/lib/copy_user_64.S
+++ b/arch/x86/lib/copy_user_64.S
@@ -220,7 +220,7 @@ EXPORT_SYMBOL(copy_user_enhanced_fast_string)
* Output:
* eax uncopied bytes or 0 if successful.
*/
-SYM_CODE_START_LOCAL(.Lcopy_user_handle_tail)
+SYM_CODE_START_LOCAL_SECT(.Lcopy_user_handle_tail, copy_user_handle_tail)
cmp $X86_TRAP_MC,%eax
je 3f

diff --git a/arch/x86/lib/error-inject.c b/arch/x86/lib/error-inject.c
index 520897061ee0..720a73d4ee70 100644
--- a/arch/x86/lib/error-inject.c
+++ b/arch/x86/lib/error-inject.c
@@ -8,11 +8,13 @@ asmlinkage void just_return_func(void);

asm(
".text\n"
+ ASM_PUSH_SECTION(just_return_func) "\n"
".type just_return_func, @function\n"
".globl just_return_func\n"
"just_return_func:\n"
ASM_RET
".size just_return_func, .-just_return_func\n"
+ ASM_POP_SECTION() "\n"
);

void override_function_with_return(struct pt_regs *regs)
diff --git a/arch/x86/lib/getuser.S b/arch/x86/lib/getuser.S
index b70d98d79a9d..06d288909a68 100644
--- a/arch/x86/lib/getuser.S
+++ b/arch/x86/lib/getuser.S
@@ -163,8 +163,7 @@ SYM_FUNC_START(__get_user_nocheck_8)
SYM_FUNC_END(__get_user_nocheck_8)
EXPORT_SYMBOL(__get_user_nocheck_8)

-
-SYM_CODE_START_LOCAL(.Lbad_get_user_clac)
+SYM_CODE_START_LOCAL_SECT(.Lbad_get_user_clac, bad_get_user_clac)
ASM_CLAC
bad_get_user:
xor %edx,%edx
@@ -173,7 +172,7 @@ bad_get_user:
SYM_CODE_END(.Lbad_get_user_clac)

#ifdef CONFIG_X86_32
-SYM_CODE_START_LOCAL(.Lbad_get_user_8_clac)
+SYM_CODE_START_LOCAL_SECT(.Lbad_get_user_8_clac, bad_get_user_8_clac)
ASM_CLAC
bad_get_user_8:
xor %edx,%edx
diff --git a/arch/x86/lib/memcpy_64.S b/arch/x86/lib/memcpy_64.S
index 59cf2343f3d9..2dc6033e2932 100644
--- a/arch/x86/lib/memcpy_64.S
+++ b/arch/x86/lib/memcpy_64.S
@@ -27,8 +27,8 @@
* Output:
* rax original destination
*/
-SYM_FUNC_START_ALIAS(__memcpy)
SYM_FUNC_START_WEAK(memcpy)
+SYM_FUNC_START_ALIAS(__memcpy)
ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \
"jmp memcpy_erms", X86_FEATURE_ERMS

@@ -40,8 +40,8 @@ SYM_FUNC_START_WEAK(memcpy)
movl %edx, %ecx
rep movsb
RET
-SYM_FUNC_END(memcpy)
SYM_FUNC_END_ALIAS(__memcpy)
+SYM_FUNC_END(memcpy)
EXPORT_SYMBOL(memcpy)
EXPORT_SYMBOL(__memcpy)

diff --git a/arch/x86/lib/memmove_64.S b/arch/x86/lib/memmove_64.S
index 50ea390df712..0040ac38751b 100644
--- a/arch/x86/lib/memmove_64.S
+++ b/arch/x86/lib/memmove_64.S
@@ -24,9 +24,8 @@
* Output:
* rax: dest
*/
-SYM_FUNC_START_WEAK(memmove)
SYM_FUNC_START(__memmove)
-
+SYM_FUNC_START_WEAK_ALIAS(memmove)
mov %rdi, %rax

/* Decide forward/backward copy mode */
@@ -206,7 +205,7 @@ SYM_FUNC_START(__memmove)
movb %r11b, (%rdi)
13:
RET
-SYM_FUNC_END(__memmove)
SYM_FUNC_END_ALIAS(memmove)
+SYM_FUNC_END(__memmove)
EXPORT_SYMBOL(__memmove)
EXPORT_SYMBOL(memmove)
diff --git a/arch/x86/lib/memset_64.S b/arch/x86/lib/memset_64.S
index d624f2bc42f1..32cf147393e7 100644
--- a/arch/x86/lib/memset_64.S
+++ b/arch/x86/lib/memset_64.S
@@ -17,8 +17,9 @@
*
* rax original destination
*/
-SYM_FUNC_START_WEAK(memset)
+
SYM_FUNC_START(__memset)
+SYM_FUNC_START_WEAK_ALIAS(memset)
/*
* Some CPUs support enhanced REP MOVSB/STOSB feature. It is recommended
* to use it when possible. If not available, use fast string instructions.
@@ -41,8 +42,8 @@ SYM_FUNC_START(__memset)
rep stosb
movq %r9,%rax
RET
-SYM_FUNC_END(__memset)
SYM_FUNC_END_ALIAS(memset)
+SYM_FUNC_END(__memset)
EXPORT_SYMBOL(memset)
EXPORT_SYMBOL(__memset)

diff --git a/arch/x86/lib/putuser.S b/arch/x86/lib/putuser.S
index ecb2049c1273..be6b2dc0967c 100644
--- a/arch/x86/lib/putuser.S
+++ b/arch/x86/lib/putuser.S
@@ -102,7 +102,7 @@ SYM_FUNC_END(__put_user_8)
EXPORT_SYMBOL(__put_user_8)
EXPORT_SYMBOL(__put_user_nocheck_8)

-SYM_CODE_START_LOCAL(.Lbad_put_user_clac)
+SYM_CODE_START_LOCAL_SECT(.Lbad_put_user_clac, bad_put_user_clac)
ASM_CLAC
.Lbad_put_user:
movl $-EFAULT,%ecx
diff --git a/arch/x86/power/hibernate_asm_32.S b/arch/x86/power/hibernate_asm_32.S
index 5606a15cf9a1..06f666e3e28a 100644
--- a/arch/x86/power/hibernate_asm_32.S
+++ b/arch/x86/power/hibernate_asm_32.S
@@ -16,7 +16,7 @@

.text

-SYM_FUNC_START(swsusp_arch_suspend)
+SYM_FUNC_START_SECT(swsusp_arch_suspend, hibernate_page)
movl %esp, saved_context_esp
movl %ebx, saved_context_ebx
movl %ebp, saved_context_ebp
@@ -35,7 +35,7 @@ SYM_FUNC_START(swsusp_arch_suspend)
RET
SYM_FUNC_END(swsusp_arch_suspend)

-SYM_CODE_START(restore_image)
+SYM_CODE_START_SECT(restore_image, hibernate_page)
/* prepare to jump to the image kernel */
movl restore_jump_address, %ebx
movl restore_cr3, %ebp
@@ -48,7 +48,7 @@ SYM_CODE_START(restore_image)
SYM_CODE_END(restore_image)

/* code below has been relocated to a safe page */
-SYM_CODE_START(core_restore_code)
+SYM_CODE_START_SECT(core_restore_code, hibernate_page)
movl temp_pgt, %eax
movl %eax, %cr3

@@ -81,8 +81,10 @@ done:
SYM_CODE_END(core_restore_code)

/* code below belongs to the image kernel */
+SYM_PUSH_SECTION(hibernate_page)
.align PAGE_SIZE
-SYM_FUNC_START(restore_registers)
+SYM_POP_SECTION()
+SYM_FUNC_START_SECT(restore_registers, hibernate_page)
/* go back to the original page tables */
movl %ebp, %cr3
movl mmu_cr4_features, %ecx
diff --git a/arch/x86/power/hibernate_asm_64.S b/arch/x86/power/hibernate_asm_64.S
index 0a0539e1cc81..fdca1949ad9f 100644
--- a/arch/x86/power/hibernate_asm_64.S
+++ b/arch/x86/power/hibernate_asm_64.S
@@ -24,8 +24,10 @@
#include <asm/nospec-branch.h>

/* code below belongs to the image kernel */
+SYM_PUSH_SECTION(hibernate_page)
.align PAGE_SIZE
-SYM_FUNC_START(restore_registers)
+SYM_POP_SECTION()
+SYM_FUNC_START_SECT(restore_registers, hibernate_page)
/* go back to the original page tables */
movq %r9, %cr3

@@ -69,7 +71,7 @@ SYM_FUNC_START(restore_registers)
RET
SYM_FUNC_END(restore_registers)

-SYM_FUNC_START(swsusp_arch_suspend)
+SYM_FUNC_START_SECT(swsusp_arch_suspend, hibernate_page)
movq $saved_context, %rax
movq %rsp, pt_regs_sp(%rax)
movq %rbp, pt_regs_bp(%rax)
@@ -99,7 +101,7 @@ SYM_FUNC_START(swsusp_arch_suspend)
RET
SYM_FUNC_END(swsusp_arch_suspend)

-SYM_FUNC_START(restore_image)
+SYM_FUNC_START_SECT(restore_image, hibernate_page)
/* prepare to jump to the image kernel */
movq restore_jump_address(%rip), %r8
movq restore_cr3(%rip), %r9
@@ -118,7 +120,7 @@ SYM_FUNC_START(restore_image)
SYM_FUNC_END(restore_image)

/* code below has been relocated to a safe page */
-SYM_FUNC_START(core_restore_code)
+SYM_FUNC_START_SECT(core_restore_code, hibernate_page)
/* switch to temporary page tables */
movq %rax, %cr3
/* flush TLB */
--
2.34.1


2022-02-09 20:20:10

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 15/15] maintainers: add MAINTAINERS entry for FG-KASLR

Add an entry for FG-KASLR containing the maintainers, reviewers,
public mailing lists, files and so on.

Signed-off-by: Alexander Lobakin <[email protected]>
---
MAINTAINERS | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index f53d30463c21..49a0974d32a7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7925,6 +7925,18 @@ L: [email protected]
S: Maintained
F: drivers/platform/x86/fujitsu-tablet.c

+FUNCTION-GRAINED KASLR (FG-KASLR)
+M: Alexander Lobakin <[email protected]>
+R: Kristen Carlson Accardi <[email protected]>
+R: Kees Cook <[email protected]>
+L: [email protected]
+S: Supported
+F: Documentation/security/fgkaslr.rst
+F: arch/x86/boot/compressed/fgkaslr.c
+F: arch/x86/boot/compressed/gen-symbols.h
+F: arch/x86/boot/compressed/utils.c
+F: scripts/generate_text_sections.pl
+
FUSE: FILESYSTEM IN USERSPACE
M: Miklos Szeredi <[email protected]>
L: [email protected]
--
2.34.1


2022-02-09 20:25:16

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 07/15] Makefile: add config options and build scripts for FG-KASLR

From: Kristen Carlson Accardi <[email protected]>

Add Kconfig symbols CONFIG_ARCH_SUPPORTS_FG_KASLR and
CONFIG_FG_KASLR. The first is hidden and used to indicate that
a particular architecture supports it, the second allows a user
to enable FG-KASLR when the former is set to 'y'.
Make Kbuild not consolidate function sections back into `.text`
on linking if CONFIG_FG_KASLR is enabled (even with
CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y) as the feature itself
relies on functions still being separated in the final vmlinux.

Alexander Lobakin:

Improve KBUILD_CFLAGS{,_MODULE} management in the top Makefile:
don't turn on -f{data,function}-sections with ClangLTO as this is a
no-op provoking a full rebuild.
Add ".symtab_shndx" to the list of known sections since it is going
to be supported by the architecture-specific code. Otherwise LD
emits a warning when there are more than 64k sections and
CONFIG_LD_ORPHAN_WARN=y.
Turn ".text" LD script wildcard into ".text.__unused__" to make sure
all kernel code will land into our special sections.
Make FG-KASLR depend on `-z unique-symbol`. With every function being
in a separate section (randomly ordered each boot), position-based
search is impossible. This flag is likely to be widely available
(on non-LLD builds).

Signed-off-by: Kristen Carlson Accardi <[email protected]>
Suggested-by: Kees Cook <[email protected]> # coexistence with DCE
Reviewed-by: Kees Cook <[email protected]>
Reviewed-by: Tony Luck <[email protected]>
Tested-by: Tony Luck <[email protected]>
Co-developed-by: Alexander Lobakin <[email protected]>
Signed-off-by: Alexander Lobakin <[email protected]>
---
Makefile | 17 ++++++++++++++---
arch/Kconfig | 6 +++++-
include/asm-generic/vmlinux.lds.h | 20 ++++++++++++++++++--
include/linux/linkage.h | 9 +++++----
init/Kconfig | 19 +++++++++++++++++--
5 files changed, 59 insertions(+), 12 deletions(-)

diff --git a/Makefile b/Makefile
index fbe2d13028f4..4328d53d8b25 100644
--- a/Makefile
+++ b/Makefile
@@ -872,7 +872,7 @@ KBUILD_CFLAGS += -fno-inline-functions-called-once
endif

# Prefer linking with the `-z unique-symbol` if available, this eliminates
-# position-based search
+# position-based search. Also is a requirement for FG-KASLR
ifeq ($(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL)$(CONFIG_LIVEPATCH),yy)
KBUILD_LDFLAGS += -z unique-symbol
endif
@@ -881,7 +881,7 @@ endif
# `include/linux/linkage.h` for explanation. This flag is to enable GAS to
# insert the name of the previous section instead of `%S` inside .pushsection
ifdef CONFIG_HAVE_ASM_FUNCTION_SECTIONS
-ifneq ($(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION)$(CONFIG_LTO_CLANG),)
+ifneq ($(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION)$(CONFIG_LTO_CLANG)$(CONFIG_FG_KASLR),)
SECSUBST_AFLAGS := -Wa,--sectname-subst
KBUILD_AFLAGS_KERNEL += $(SECSUBST_AFLAGS)
KBUILD_CFLAGS_KERNEL += $(SECSUBST_AFLAGS)
@@ -895,8 +895,19 @@ KBUILD_CFLAGS_MODULE += -Wa,--sectname-subst
endif
endif # CONFIG_HAVE_ASM_FUNCTION_SECTIONS

+# ClangLTO implies `-ffunction-sections -fdata-sections`, no need
+# to specify them manually and trigger a pointless full rebuild
+ifndef CONFIG_LTO_CLANG
+ifneq ($(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION)$(CONFIG_FG_KASLR),)
+KBUILD_CFLAGS_KERNEL += -ffunction-sections
+endif
+
+ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
+KBUILD_CFLAGS_KERNEL += -fdata-sections
+endif
+endif # CONFIG_LTO_CLANG
+
ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
-KBUILD_CFLAGS_KERNEL += -ffunction-sections -fdata-sections
LDFLAGS_vmlinux += --gc-sections
endif

diff --git a/arch/Kconfig b/arch/Kconfig
index 550f0599e211..e06aeeea39f4 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1326,7 +1326,11 @@ config ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS
bool
help
An arch should select this if it can be built and run with its
- asm functions placed into separate sections to improve DCE and LTO.
+ asm functions placed into separate sections to improve DCE, LTO
+ and FG-KASLR.
+
+config ARCH_SUPPORTS_FG_KASLR
+ bool

source "kernel/gcov/Kconfig"

diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index e7b8a84e0e64..586465b2abb2 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -100,14 +100,12 @@
* sections to be brought in with rodata.
*/
#if defined(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) || defined(CONFIG_LTO_CLANG)
-#define TEXT_MAIN SECT_WILDCARD(.text)
#define DATA_MAIN SECT_WILDCARD(.data) .data..L* .data..compoundliteral* .data.$__unnamed_* .data.$L*
#define SDATA_MAIN SECT_WILDCARD(.sdata)
#define RODATA_MAIN SECT_WILDCARD(.rodata) .rodata..L*
#define BSS_MAIN SECT_WILDCARD(.bss) .bss..compoundliteral*
#define SBSS_MAIN SECT_WILDCARD(.sbss)
#else
-#define TEXT_MAIN .text
#define DATA_MAIN .data
#define SDATA_MAIN .sdata
#define RODATA_MAIN .rodata
@@ -115,6 +113,23 @@
#define SBSS_MAIN .sbss
#endif

+/*
+ * LTO_CLANG, LD_DEAD_CODE_DATA_ELIMINATION and FG_KASLR options enable
+ * -ffunction-sections, which produces separately named .text sections. In
+ * the case of CONFIG_FG_KASLR, they need to stay distict so they can be
+ * separately randomized. Without CONFIG_FG_KASLR, the separate .text
+ * sections can be collected back into a common section, which makes the
+ * resulting image slightly smaller
+ */
+#if (defined(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) || \
+ defined(CONFIG_LTO_CLANG)) && !defined(CONFIG_FG_KASLR)
+#define TEXT_MAIN SECT_WILDCARD(.text)
+#elif defined(CONFIG_FG_KASLR)
+#define TEXT_MAIN .text.__unused__
+#else
+#define TEXT_MAIN .text
+#endif
+
/*
* GCC 4.5 and later have a 32 bytes section alignment for structures.
* Except GCC 4.9, that feels the need to align on 64 bytes.
@@ -843,6 +858,7 @@
#define ELF_DETAILS \
.comment 0 : { *(.comment) } \
.symtab 0 : { *(.symtab) } \
+ .symtab_shndx 0 : { *(.symtab_shndx) } \
.strtab 0 : { *(.strtab) } \
.shstrtab 0 : { *(.shstrtab) }

diff --git a/include/linux/linkage.h b/include/linux/linkage.h
index f3b966a6427e..95ca162a868c 100644
--- a/include/linux/linkage.h
+++ b/include/linux/linkage.h
@@ -75,11 +75,12 @@

/*
* Allow ASM symbols to have their own unique sections if they are being
- * generated by the compiler for C functions (DCE, LTO). Correlates with
- * the presence of the `-ffunction-section` in KBUILD_CFLAGS.
+ * generated by the compiler for C functions (DCE, FG-KASLR, LTO). Correlates
+ * with the presence of the `-ffunction-section` in KBUILD_CFLAGS.
*/
#if defined(CONFIG_HAVE_ASM_FUNCTION_SECTIONS) && \
((defined(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) && !defined(MODULE)) || \
+ (defined(CONFIG_FG_KASLR) && !defined(MODULE)) || \
(defined(CONFIG_LTO_CLANG)))

#define SYM_PUSH_SECTION(name) \
@@ -91,13 +92,13 @@
#define __ASM_PUSH_SECTION(name) \
".pushsection %S." name ", \"ax\""

-#else /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || LTO)) */
+#else /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || FG_KASLR || LTO)) */

#define SYM_PUSH_SECTION(name)
#define SYM_POP_SECTION()
#define __ASM_PUSH_SECTION(name)

-#endif /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || LTO)) */
+#endif /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || FG_KASLR || LTO)) */

#define ASM_PUSH_SECTION(name) \
__ASM_PUSH_SECTION(__stringify(name))
diff --git a/init/Kconfig b/init/Kconfig
index 4acfc80f22df..26f9a6e52dbd 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1393,8 +1393,9 @@ config HAVE_ASM_FUNCTION_SECTIONS
help
This enables asm function sections if both architecture and
toolchain support it. It allows creating a separate section
- for each function written in assembly in order to improve DCE
- and LTO (works the same way as -ffunction-sections for C code).
+ for each function written in assembly in order to improve DCE,
+ LTO and FG-KASLR (works the same way as -ffunction-sections
+ for C code).

config HAVE_LD_DEAD_CODE_DATA_ELIMINATION
bool
@@ -2061,6 +2062,20 @@ config PROFILING
config TRACEPOINTS
bool

+config FG_KASLR
+ bool "Function Granular Kernel Address Space Layout Randomization"
+ depends on ARCH_SUPPORTS_FG_KASLR
+ depends on $(cc-option,-ffunction-sections)
+ depends on LD_HAS_Z_UNIQUE_SYMBOL || !LIVEPATCH
+ help
+ This option improves the randomness of the kernel text
+ over basic Kernel Address Space Layout Randomization (KASLR)
+ by reordering the kernel text at boot time. This feature
+ uses information generated at compile time to re-layout the
+ kernel text section at boot time at function level granularity.
+
+ If unsure, say N.
+
endmenu # General setup

source "arch/Kconfig"
--
2.34.1


2022-02-09 20:30:04

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 12/15] module: add arch-indep FG-KASLR for randomizing function layout

From: Kristen Carlson Accardi <[email protected]>

Introduce a new config option to allow modules to be re-ordered
by function. This option can be enabled independently of the
kernel text KASLR or FG_KASLR settings so that it can be used
by architectures that do not support either of these features.
This option will be selected by default if CONFIG_FG_KASLR is
selected.

If a module has functions split out into separate text sections
(i.e. compiled with the -ffunction-sections flag), reorder the
functions to provide some code diversification to modules.

Alexander Lobakin:

Make it work with ClangCFI -- in such builds, .text section must
always come first and be page-aligned. Exclude it from the shuffle
list and leave as it is.
Make this feature depend on `-z unique-symbol` as well, due to the
very same reasons as for FG-KASLR for vmlinux.
Use common shuffle_array() from <linux/random.h> instead of
open-coding it.

Signed-off-by: Kristen Carlson Accardi <[email protected]>
Reviewed-by: Kees Cook <[email protected]>
Acked-by: Ard Biesheuvel <[email protected]>
Tested-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Tony Luck <[email protected]>
Tested-by: Tony Luck <[email protected]>
Acked-by: Jessica Yu <[email protected]>
Tested-by: Jessica Yu <[email protected]>
Reported-by: kernel test robot <[email protected]> # swap.cocci
Co-developed-by: Alexander Lobakin <[email protected]>
Signed-off-by: Alexander Lobakin <[email protected]>
---
Makefile | 6 +++-
include/linux/linkage.h | 1 +
init/Kconfig | 14 ++++++++
kernel/module.c | 73 +++++++++++++++++++++++++++++++++++++++--
4 files changed, 91 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index 4328d53d8b25..cf7cf5cbdad9 100644
--- a/Makefile
+++ b/Makefile
@@ -889,7 +889,7 @@ export SECSUBST_AFLAGS
endif

# Same for modules. LD DCE doesn't work for them, thus not checking for it
-ifneq ($(CONFIG_LTO_CLANG),)
+ifneq ($(CONFIG_MODULE_FG_KASLR)$(CONFIG_LTO_CLANG),)
KBUILD_AFLAGS_MODULE += -Wa,--sectname-subst
KBUILD_CFLAGS_MODULE += -Wa,--sectname-subst
endif
@@ -898,6 +898,10 @@ endif # CONFIG_HAVE_ASM_FUNCTION_SECTIONS
# ClangLTO implies `-ffunction-sections -fdata-sections`, no need
# to specify them manually and trigger a pointless full rebuild
ifndef CONFIG_LTO_CLANG
+ifdef CONFIG_MODULE_FG_KASLR
+KBUILD_CFLAGS_MODULE += -ffunction-sections
+endif
+
ifneq ($(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION)$(CONFIG_FG_KASLR),)
KBUILD_CFLAGS_KERNEL += -ffunction-sections
endif
diff --git a/include/linux/linkage.h b/include/linux/linkage.h
index 95ca162a868c..12cf21f9d8ad 100644
--- a/include/linux/linkage.h
+++ b/include/linux/linkage.h
@@ -81,6 +81,7 @@
#if defined(CONFIG_HAVE_ASM_FUNCTION_SECTIONS) && \
((defined(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) && !defined(MODULE)) || \
(defined(CONFIG_FG_KASLR) && !defined(MODULE)) || \
+ (defined(CONFIG_MODULE_FG_KASLR) && defined(MODULE)) || \
(defined(CONFIG_LTO_CLANG)))

#define SYM_PUSH_SECTION(name) \
diff --git a/init/Kconfig b/init/Kconfig
index 5fbd1c294df4..86a2d3fd6390 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2386,6 +2386,20 @@ config UNUSED_KSYMS_WHITELIST
one per line. The path can be absolute, or relative to the kernel
source tree.

+config MODULE_FG_KASLR
+ bool "Module Function Granular Layout Randomization"
+ depends on $(cc-option,-ffunction-sections)
+ depends on LD_HAS_Z_UNIQUE_SYMBOL || !LIVEPATCH
+ default FG_KASLR
+ depends on BROKEN
+ help
+ This option randomizes the module text section by reordering the text
+ section by function at module load time. In order to use this
+ feature, the module must have been compiled with the
+ -ffunction-sections compiler flag.
+
+ If unsure, say N.
+
endif # MODULES

config MODULES_TREE_LOOKUP
diff --git a/kernel/module.c b/kernel/module.c
index 46a5c2ed1928..616a622953fa 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -57,6 +57,7 @@
#include <linux/bsearch.h>
#include <linux/dynamic_debug.h>
#include <linux/audit.h>
+#include <linux/random.h>
#include <uapi/linux/module.h>
#include "module-internal.h"

@@ -1526,7 +1527,7 @@ static void free_sect_attrs(struct module_sect_attrs *sect_attrs)

for (section = 0; section < sect_attrs->nsections; section++)
kfree(sect_attrs->attrs[section].battr.attr.name);
- kfree(sect_attrs);
+ kvfree(sect_attrs);
}

static void add_sect_attrs(struct module *mod, const struct load_info *info)
@@ -1543,7 +1544,7 @@ static void add_sect_attrs(struct module *mod, const struct load_info *info)
size[0] = ALIGN(struct_size(sect_attrs, attrs, nloaded),
sizeof(sect_attrs->grp.bin_attrs[0]));
size[1] = (nloaded + 1) * sizeof(sect_attrs->grp.bin_attrs[0]);
- sect_attrs = kzalloc(size[0] + size[1], GFP_KERNEL);
+ sect_attrs = kvzalloc(size[0] + size[1], GFP_KERNEL);
if (sect_attrs == NULL)
return;

@@ -2415,6 +2416,71 @@ static bool module_init_layout_section(const char *sname)
return module_init_section(sname);
}

+/*
+ * randomize_text()
+ * Look through the core section looking for executable code sections.
+ * Store sections in an array and then shuffle the sections
+ * to reorder the functions.
+ */
+static void randomize_text(struct module *mod, struct load_info *info)
+{
+ int max_sections = info->hdr->e_shnum;
+ int num_text_sections = 0;
+ Elf_Shdr **text_list;
+ int i, size;
+
+ text_list = kvmalloc_array(max_sections, sizeof(*text_list), GFP_KERNEL);
+ if (!text_list)
+ return;
+
+ for (i = 0; i < max_sections; i++) {
+ Elf_Shdr *shdr = &info->sechdrs[i];
+ const char *sname = info->secstrings + shdr->sh_name;
+
+ if (!(shdr->sh_flags & SHF_ALLOC) ||
+ !(shdr->sh_flags & SHF_EXECINSTR) ||
+ (shdr->sh_flags & ARCH_SHF_SMALL) ||
+ module_init_layout_section(sname))
+ continue;
+
+ /*
+ * With CONFIG_CFI_CLANG, .text with __cfi_check() must come
+ * before any other text sections, and be aligned to PAGE_SIZE.
+ * Don't include it in the shuffle list.
+ */
+ if (IS_ENABLED(CONFIG_CFI_CLANG) && !strcmp(sname, ".text"))
+ continue;
+
+ if (!num_text_sections)
+ size = shdr->sh_entsize;
+
+ text_list[num_text_sections] = shdr;
+ num_text_sections++;
+ }
+
+ if (!num_text_sections)
+ goto exit;
+
+ shuffle_array(text_list, num_text_sections);
+
+ for (i = 0; i < num_text_sections; i++) {
+ Elf_Shdr *shdr = text_list[i];
+
+ /*
+ * get_offset has a section index for it's last
+ * argument, that is only used by arch_mod_section_prepend(),
+ * which is only defined by parisc. Since this type
+ * of randomization isn't supported on parisc, we can
+ * safely pass in zero as the last argument, as it is
+ * ignored.
+ */
+ shdr->sh_entsize = get_offset(mod, &size, shdr, 0);
+ }
+
+exit:
+ kvfree(text_list);
+}
+
/*
* Lay out the SHF_ALLOC sections in a way not dissimilar to how ld
* might -- code, read-only data, read-write data, small data. Tally
@@ -2509,6 +2575,9 @@ static void layout_sections(struct module *mod, struct load_info *info)
break;
}
}
+
+ if (IS_ENABLED(CONFIG_MODULE_FG_KASLR))
+ randomize_text(mod, info);
}

static void set_license(struct module *mod, const char *license)
--
2.34.1


2022-02-09 20:33:35

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

Position-based search, which means that if there are several symbols
with the same name, the user needs to additionally provide the
"index" of a desired symbol, is fragile. For example, it breaks
when two symbols with the same name are located in different
sections.

Since a while, LD has a flag `-z unique-symbol` which appends
numeric suffixes to the functions with the same name (in symtab
and strtab). It can be used to effectively prevent from having
any ambiguity when referring to a symbol by its name.
Check for its availability and always prefer when the livepatching
is on. It can be used unconditionally later on after broader testing
on a wide variety of machines, but for now let's stick to the actual
CONFIG_LIVEPATCH=y case, which is true for most of distro configs
anyways.
This needs a little adjustment to the modpost to make it strip
suffixes before adding exports. depmod needs some treatment as well,
tho its false-positive warnings about unknown symbols are harmless
and don't alter the return code.

There is probably a bunch more livepatch code to optimize-out after
introducing this, leave it for later as well.

Suggested-by: H.J. Lu <[email protected]>
Suggested-by: Peter Zijlstra <[email protected]>
Suggested-by: Josh Poimboeuf <[email protected]>
Suggested-by: Miroslav Benes <[email protected]>
Signed-off-by: Alexander Lobakin <[email protected]>
---
Makefile | 6 ++++++
init/Kconfig | 3 +++
kernel/livepatch/core.c | 17 +++++++++++++----
scripts/mod/modpost.c | 42 ++++++++++++++++++++++-------------------
4 files changed, 45 insertions(+), 23 deletions(-)

diff --git a/Makefile b/Makefile
index ceb987e5c87b..fa9f947c9839 100644
--- a/Makefile
+++ b/Makefile
@@ -871,6 +871,12 @@ ifdef CONFIG_DEBUG_SECTION_MISMATCH
KBUILD_CFLAGS += -fno-inline-functions-called-once
endif

+# Prefer linking with the `-z unique-symbol` if available, this eliminates
+# position-based search
+ifeq ($(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL)$(CONFIG_LIVEPATCH),yy)
+KBUILD_LDFLAGS += -z unique-symbol
+endif
+
ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
KBUILD_CFLAGS_KERNEL += -ffunction-sections -fdata-sections
LDFLAGS_vmlinux += --gc-sections
diff --git a/init/Kconfig b/init/Kconfig
index e9119bf54b1f..8e900d17d42b 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -86,6 +86,9 @@ config CC_HAS_ASM_INLINE
config CC_HAS_NO_PROFILE_FN_ATTR
def_bool $(success,echo '__attribute__((no_profile_instrument_function)) int x();' | $(CC) -x c - -c -o /dev/null -Werror)

+config LD_HAS_Z_UNIQUE_SYMBOL
+ def_bool $(ld-option,-z unique-symbol)
+
config CONSTRUCTORS
bool

diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
index 585494ec464f..7a330465a8c7 100644
--- a/kernel/livepatch/core.c
+++ b/kernel/livepatch/core.c
@@ -143,11 +143,13 @@ static int klp_find_callback(void *data, const char *name,
args->count++;

/*
- * Finish the search when the symbol is found for the desired position
- * or the position is not defined for a non-unique symbol.
+ * Finish the search when unique symbol names are enabled
+ * or the symbol is found for the desired position or the
+ * position is not defined for a non-unique symbol.
*/
- if ((args->pos && (args->count == args->pos)) ||
- (!args->pos && (args->count > 1)))
+ if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL) ||
+ (args->pos && args->count == args->pos) ||
+ (!args->pos && args->count > 1))
return 1;

return 0;
@@ -169,6 +171,13 @@ static int klp_find_object_symbol(const char *objname, const char *name,
else
kallsyms_on_each_symbol(klp_find_callback, &args);

+ /*
+ * If the LD's `-z unique-symbol` flag is available and enabled,
+ * sympos checks are not relevant.
+ */
+ if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL))
+ sympos = 0;
+
/*
* Ensure an address was found. If sympos is 0, ensure symbol is unique;
* otherwise ensure the symbol position count matches sympos.
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 4648b7afe5cc..ec521ccebea6 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -689,11 +689,28 @@ static void handle_modversion(const struct module *mod,
sym_set_crc(symname, crc);
}

+static char *remove_dot(char *s)
+{
+ size_t n = strcspn(s, ".");
+
+ if (n && s[n]) {
+ size_t m = strspn(s + n + 1, "0123456789");
+
+ if (m && (s[n + m + 1] == '.' || s[n + m + 1] == 0))
+ s[n] = 0;
+
+ /* strip trailing .lto */
+ if (strends(s, ".lto"))
+ s[strlen(s) - 4] = '\0';
+ }
+
+ return s;
+}
+
static void handle_symbol(struct module *mod, struct elf_info *info,
const Elf_Sym *sym, const char *symname)
{
enum export export;
- const char *name;

if (strstarts(symname, "__ksymtab"))
export = export_from_secname(info, get_secindex(info, sym));
@@ -734,8 +751,11 @@ static void handle_symbol(struct module *mod, struct elf_info *info,
default:
/* All exported symbols */
if (strstarts(symname, "__ksymtab_")) {
- name = symname + strlen("__ksymtab_");
- sym_add_exported(name, mod, export);
+ char *name;
+
+ name = NOFAIL(strdup(symname + strlen("__ksymtab_")));
+ sym_add_exported(remove_dot(name), mod, export);
+ free(name);
}
if (strcmp(symname, "init_module") == 0)
mod->has_init = 1;
@@ -1980,22 +2000,6 @@ static void check_sec_ref(struct module *mod, const char *modname,
}
}

-static char *remove_dot(char *s)
-{
- size_t n = strcspn(s, ".");
-
- if (n && s[n]) {
- size_t m = strspn(s + n + 1, "0123456789");
- if (m && (s[n + m + 1] == '.' || s[n + m + 1] == 0))
- s[n] = 0;
-
- /* strip trailing .lto */
- if (strends(s, ".lto"))
- s[strlen(s) - 4] = '\0';
- }
- return s;
-}
-
static void read_symbols(const char *modname)
{
const char *symname;
--
2.34.1


2022-02-09 20:41:53

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH v10 00/15] Function Granular KASLR

From: Alexander Lobakin <[email protected]>
Date: Wed, 9 Feb 2022 19:57:37 +0100

> From: Kristen Carlson Accardi <[email protected]>
>
> Function Granular Kernel Address Space Layout Randomization (FG-KASLR)

--- 8< ---

> From v9 ([1]):
> - rebase on top of 5.17-rc3 + tip/master, notably:
> * drop .fixup (the section was removed in 5.17 cycle) and
> .Lbad_gs (was converted from a symbol to a label) references
> and handling;
> * use `is_enabled` and sed in scripts/link-vmlinux.sh (doesn't
> include auto.conf anymore);
> - s/ASM/asm across the series (Boris);
> - update the year across the series;
> - restructure the cover letter to make it more readable and give
> the key ideas earlier (Chris);
> - 0001: expand the commitmsg with how this was reproduced, drop
> "Stable:" to avoid accidental LTS issues (Boris);
> - 0002: get back to plain resetting `sympos` to zero when
> applicable (Mirek); rephrase and expand the commitsg (Boris);
> - 0003: expand the subject and the commitmsg with a bit of crucial
> details, invert a couple branches to drop a `goto` and save one
> indent level (Boris);
> avoid using `bool` type in a structure, revert the for-loop back
> to the `while`;
> - 0004: rename `to` argument in asm symbol macros to `sect` to make
> it less confusing, elaborate on "asm function sections" in the
> commitmsg (Boris);
> - 0005: replace "aflags" with the actual "KBUILD_AFLAGS" in the
> commitmsg (Boris);
> - 0006: build the new arch/x86/lib/orc.c only for CONFIG_MODULES=y
> as this code was formerly guarded with the corresponding #ifdefs;
> - 0007: rephrase the commitmsg;
> - 0009: rename `base` in fgkaslr.c to `kallsyms_base` to dodge
> from `-Wshadow`;
> fix "indecis" -> "indices" and reduce variable scope in
> layout_image();
> include lib/orc.c in utils.c rather than fgkaslr.c itself;
> replace {,u}int{32,64}_t -> {s,u}{32,64};
> - 0010: change obsolete references to the section alignment of 64
> as starting from v8 only the alignment of 128+ is being taken
> into account (with CONFIG_DEBUG_FORCE_FUNCTION_ALIGN_64B=y,
> every function is aligned to 64, so it would blow up the kernel
> image for no reason);
> - 0012: rephrase the subject to give more clue.

The diff between v9 and v10 to ease (or harden) the reviewing is
appended.
Thanks!

--- 8< ---

> --
> 2.34.1

Al

--
diff --git a/0001-modpost-fix-removing-numeric-suffixes.patch b/0001-modpost-fix-removing-numeric-suffixes.patch
index 27a8903455e7..39c03757fab4 100644
--- a/0001-modpost-fix-removing-numeric-suffixes.patch
+++ b/0001-modpost-fix-removing-numeric-suffixes.patch
@@ -1,39 +1,42 @@
-From 79b9b05458e9c42f4c998dbe69b76906def9c19a Mon Sep 17 00:00:00 2001
+From e2396ff76455bee6819ab9c192b60c2b4386756a Mon Sep 17 00:00:00 2001
From: Alexander Lobakin <[email protected]>
Date: Wed, 22 Dec 2021 16:44:07 +0100
-Subject: [PATCH v9 01/15] modpost: fix removing numeric suffixes
+Subject: [PATCH v10 01/15] modpost: fix removing numeric suffixes

-For now, that condition from remove_dot():
+`-z unique-symbol` linker flag which is planned to use with FG-KASLR
+to simplify livepatching (hopefully globally later on) triggers the
+following:
+
+ERROR: modpost: "param_set_uint.0" [vmlinux] is a static EXPORT_SYMBOL
+
+The reason is that for now the condition from remove_dot():

if (m && (s[n + m] == '.' || s[n + m] == 0))

-which was designed to test if it's a dot or a \0 after the suffix
+which was designed to test if it's a dot or a '\0' after the suffix
is never satisfied.
-This is due to that s[n + m] always points to the last digit of a
-numeric suffix, not on the symbol next to it:
+This is due to that `s[n + m]` always points to the last digit of a
+numeric suffix, not on the symbol next to it (from a custom debug
+print added to modpost):

param_set_uint.0, s[n + m] is '0', s[n + m + 1] is '\0'

-So it's off by one and was like that since 2014.
-
-`-z uniq-symbol` linker flag which we are planning to use to
-simplify livepatching brings numeric suffixes back, fix this.
-Otherwise:
-
-ERROR: modpost: "param_set_uint.0" [vmlinux] is a static EXPORT_SYMBOL
+So it's off-by-one and was like that since 2014.
+Fix this for the sake of upcoming features, but don't bother
+stable-backporting, as it's well hidden -- apart from that LD flag,
+can be triggered only by GCC LTO which never landed upstream.

Fixes: fcd38ed0ff26 ("scripts: modpost: fix compilation warning")
-Cc: [email protected] # 3.17+
Signed-off-by: Alexander Lobakin <[email protected]>
---
scripts/mod/modpost.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
-index cb8ab7d91d30..ccc6d35580f2 100644
+index 6bfa33217914..4648b7afe5cc 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
-@@ -1971,7 +1971,7 @@ static char *remove_dot(char *s)
+@@ -1986,7 +1986,7 @@ static char *remove_dot(char *s)

if (n && s[n]) {
size_t m = strspn(s + n + 1, "0123456789");
@@ -43,5 +46,5 @@ index cb8ab7d91d30..ccc6d35580f2 100644

/* strip trailing .lto */
--
-2.33.1
+2.34.1

diff --git a/0002-livepatch-use-z-unique-symbol-if-available-to-nuke-p.patch b/0002-livepatch-avoid-position-based-search-if-z-unique-sy.patch
similarity index 66%
rename from 0002-livepatch-use-z-unique-symbol-if-available-to-nuke-p.patch
rename to 0002-livepatch-avoid-position-based-search-if-z-unique-sy.patch
index 5af4a9ad1cac..a490bc08252d 100644
--- a/0002-livepatch-use-z-unique-symbol-if-available-to-nuke-p.patch
+++ b/0002-livepatch-avoid-position-based-search-if-z-unique-sy.patch
@@ -1,42 +1,49 @@
-From fef47fddee9dc9afad7dca89932fce8a996aa2df Mon Sep 17 00:00:00 2001
+From e5c211b10e86f7d3cd319b795adac5c725faf09c Mon Sep 17 00:00:00 2001
From: Alexander Lobakin <[email protected]>
Date: Wed, 22 Dec 2021 17:31:45 +0100
-Subject: [PATCH v9 02/15] livepatch: use `-z unique-symbol` if available to
- nuke pos-based search
+Subject: [PATCH v10 02/15] livepatch: avoid position-based search if `-z
+ unique-symbol` is available

-Position-based search, which means that if we have several symbols
-with the same name, we additionally need to provide an "index" of
-the desired symbol, is fragile. Par exemple, it breaks when two
-symbols with the same name are located in different sections.
+Position-based search, which means that if there are several symbols
+with the same name, the user needs to additionally provide the
+"index" of a desired symbol, is fragile. For example, it breaks
+when two symbols with the same name are located in different
+sections.

Since a while, LD has a flag `-z unique-symbol` which appends
numeric suffixes to the functions with the same name (in symtab
-and strtab).
+and strtab). It can be used to effectively prevent from having
+any ambiguity when referring to a symbol by its name.
Check for its availability and always prefer when the livepatching
-is on. This needs a little adjustment to the modpost to make it
-strip suffixes before adding exports.
+is on. It can be used unconditionally later on after broader testing
+on a wide variety of machines, but for now let's stick to the actual
+CONFIG_LIVEPATCH=y case, which is true for most of distro configs
+anyways.
+This needs a little adjustment to the modpost to make it strip
+suffixes before adding exports. depmod needs some treatment as well,
+tho its false-positive warnings about unknown symbols are harmless
+and don't alter the return code.

-depmod needs some treatment as well, tho its false-positibe warnings
-about unknown symbols are harmless and don't alter the return code.
-And there is a bunch more livepatch code to optimize-out after
-introducing this, but let's leave it for later.
+There is probably a bunch more livepatch code to optimize-out after
+introducing this, leave it for later as well.

Suggested-by: H.J. Lu <[email protected]>
Suggested-by: Peter Zijlstra <[email protected]>
Suggested-by: Josh Poimboeuf <[email protected]>
+Suggested-by: Miroslav Benes <[email protected]>
Signed-off-by: Alexander Lobakin <[email protected]>
---
Makefile | 6 ++++++
init/Kconfig | 3 +++
- kernel/livepatch/core.c | 20 +++++++++++++-------
+ kernel/livepatch/core.c | 17 +++++++++++++----
scripts/mod/modpost.c | 42 ++++++++++++++++++++++-------------------
- 4 files changed, 45 insertions(+), 26 deletions(-)
+ 4 files changed, 45 insertions(+), 23 deletions(-)

diff --git a/Makefile b/Makefile
-index d85f1ff79f5c..9dc15c67d132 100644
+index ceb987e5c87b..fa9f947c9839 100644
--- a/Makefile
+++ b/Makefile
-@@ -882,6 +882,12 @@ ifdef CONFIG_DEBUG_SECTION_MISMATCH
+@@ -871,6 +871,12 @@ ifdef CONFIG_DEBUG_SECTION_MISMATCH
KBUILD_CFLAGS += -fno-inline-functions-called-once
endif

@@ -50,7 +57,7 @@ index d85f1ff79f5c..9dc15c67d132 100644
KBUILD_CFLAGS_KERNEL += -ffunction-sections -fdata-sections
LDFLAGS_vmlinux += --gc-sections
diff --git a/init/Kconfig b/init/Kconfig
-index 4b7bac10c72d..37926d19a74a 100644
+index e9119bf54b1f..8e900d17d42b 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -86,6 +86,9 @@ config CC_HAS_ASM_INLINE
@@ -64,7 +71,7 @@ index 4b7bac10c72d..37926d19a74a 100644
bool

diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
-index 335d988bd811..b2c787297f85 100644
+index 585494ec464f..7a330465a8c7 100644
--- a/kernel/livepatch/core.c
+++ b/kernel/livepatch/core.c
@@ -143,11 +143,13 @@ static int klp_find_callback(void *data, const char *name,
@@ -85,33 +92,22 @@ index 335d988bd811..b2c787297f85 100644
return 1;

return 0;
-@@ -171,17 +173,21 @@ static int klp_find_object_symbol(const char *objname, const char *name,
+@@ -169,6 +171,13 @@ static int klp_find_object_symbol(const char *objname, const char *name,
+ else
+ kallsyms_on_each_symbol(klp_find_callback, &args);

++ /*
++ * If the LD's `-z unique-symbol` flag is available and enabled,
++ * sympos checks are not relevant.
++ */
++ if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL))
++ sympos = 0;
++
/*
* Ensure an address was found. If sympos is 0, ensure symbol is unique;
-- * otherwise ensure the symbol position count matches sympos.
-+ * otherwise ensure the symbol position count matches sympos. If the LD
-+ * `-z unique` flag is enabled, sympos checks are not relevant.
- */
-- if (args.addr == 0)
-+ if (args.addr == 0) {
- pr_err("symbol '%s' not found in symbol table\n", name);
-- else if (args.count > 1 && sympos == 0) {
-+ } else if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL)) {
-+ goto out_ok;
-+ } else if (args.count > 1 && sympos == 0) {
- pr_err("unresolvable ambiguity for symbol '%s' in object '%s'\n",
- name, objname);
- } else if (sympos != args.count && sympos > 0) {
- pr_err("symbol position %lu for symbol '%s' in object '%s' not found\n",
- sympos, name, objname ? objname : "vmlinux");
- } else {
-+out_ok:
- *addr = args.addr;
- return 0;
- }
+ * otherwise ensure the symbol position count matches sympos.
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
-index ccc6d35580f2..f39cc73a082c 100644
+index 4648b7afe5cc..ec521ccebea6 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -689,11 +689,28 @@ static void handle_modversion(const struct module *mod,
@@ -158,7 +154,7 @@ index ccc6d35580f2..f39cc73a082c 100644
}
if (strcmp(symname, "init_module") == 0)
mod->has_init = 1;
-@@ -1965,22 +1985,6 @@ static void check_sec_ref(struct module *mod, const char *modname,
+@@ -1980,22 +2000,6 @@ static void check_sec_ref(struct module *mod, const char *modname,
}
}

@@ -182,5 +178,5 @@ index ccc6d35580f2..f39cc73a082c 100644
{
const char *symname;
--
-2.33.1
+2.34.1

diff --git a/0003-kallsyms-Hide-layout.patch b/0003-kallsyms-randomize-proc-kallsyms-output-order.patch
similarity index 72%
rename from 0003-kallsyms-Hide-layout.patch
rename to 0003-kallsyms-randomize-proc-kallsyms-output-order.patch
index 7a4c211571e1..853ab3066f99 100644
--- a/0003-kallsyms-Hide-layout.patch
+++ b/0003-kallsyms-randomize-proc-kallsyms-output-order.patch
@@ -1,18 +1,29 @@
-From e6ebaa477467ce29918cee3f531c028a822aeb21 Mon Sep 17 00:00:00 2001
+From 00f254dededfeaf9e3845dc9aa64dbc275cd7226 Mon Sep 17 00:00:00 2001
From: Kristen Carlson Accardi <[email protected]>
Date: Fri, 27 Aug 2021 16:41:30 +0200
-Subject: [PATCH v9 03/15] kallsyms: Hide layout
+Subject: [PATCH v10 03/15] kallsyms: randomize /proc/kallsyms output order

-This patch makes /proc/kallsyms display in a random order, rather
-than sorted by address in order to hide the newly randomized address
-layout.
+For now, /proc/kallsyms printing code replaces the actual addresses
+with zeros for !CAP_SYSLOG accesses, but still emits the symbols
+sorted by address.
+This makes it easier for an attacker to determine the address of a
+desired function if the address of some other symbol is already
+known. It is especially critical for FG-KASLR as the text layout
+with it is random each boot and sorted /proc/kallsyms renders the
+entire feature useless.

-alobakin:
-Don't depend FG-KASLR and always do that for unpriviledged accesses
-as suggested by several folks.
-Also, introduce and use a shuffle_array() macro which shuffles an
-array using Fisher-Yates. We'll make use of it several more times
-later on.
+Make /proc/kallsyms display symbols in a random order to completely
+hide the kernel layout. Determine the full number of symbols prior
+to traversing through them and then shuffle the indexes, so the
+printing function will pick them randomly.
+
+Alexander Lobakin:
+
+Don't depend FG-KASLR and always do that for unprivileged accesses
+as suggested by several folks. Rename show_value to show_layout to
+reflect the new functionality.
+Also, introduce and use a generic shuffle_array() macro which
+shuffles an arbitrary array using a Fisher-Yates algorithm.

Signed-off-by: Kristen Carlson Accardi <[email protected]>
Reviewed-by: Tony Luck <[email protected]>
@@ -24,12 +35,12 @@ Suggested-by: Peter Zijlstra <[email protected]> # always do that, macro
Co-developed-by: Alexander Lobakin <[email protected]>
Signed-off-by: Alexander Lobakin <[email protected]>
---
- include/linux/random.h | 16 ++++++++
- kernel/kallsyms.c | 93 ++++++++++++++++++++++++++++++++++--------
- 2 files changed, 93 insertions(+), 16 deletions(-)
+ include/linux/random.h | 16 +++++++
+ kernel/kallsyms.c | 94 +++++++++++++++++++++++++++++++++++-------
+ 2 files changed, 95 insertions(+), 15 deletions(-)

diff --git a/include/linux/random.h b/include/linux/random.h
-index f45b8be3e3c4..c859a698089c 100644
+index c45b2693e51f..596b27423150 100644
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -110,6 +110,22 @@ declare_get_random_var_wait(long)
@@ -37,7 +48,7 @@ index f45b8be3e3c4..c859a698089c 100644
unsigned long randomize_page(unsigned long start, unsigned long range);

+/**
-+ * shuffle_array - use a Fisher Yates algorithm to shuffle an array.
++ * shuffle_array - use a Fisher-Yates algorithm to shuffle an array.
+ * @arr: pointer to the array
+ * @nents: the number of elements in the array
+ */
@@ -56,10 +67,10 @@ index f45b8be3e3c4..c859a698089c 100644
* This is designed to be standalone for just prandom
* users, but for now we include it from <linux/random.h>
diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c
-index 3011bc33a5ba..5d41b993113f 100644
+index 951c93216fc4..d991afa9b5bf 100644
--- a/kernel/kallsyms.c
+++ b/kernel/kallsyms.c
-@@ -574,13 +574,15 @@ struct kallsym_iter {
+@@ -575,13 +575,15 @@ struct kallsym_iter {
loff_t pos_mod_end;
loff_t pos_ftrace_mod_end;
loff_t pos_bpf_end;
@@ -71,12 +82,12 @@ index 3011bc33a5ba..5d41b993113f 100644
char module_name[MODULE_NAME_LEN];
int exported;
- int show_value;
-+ bool show_layout;
++ int show_layout:1;
+ loff_t shuffled_pos[];
};

int __weak arch_get_kallsym(unsigned int symnum, unsigned long *value,
-@@ -660,11 +662,19 @@ static int get_ksymbol_bpf(struct kallsym_iter *iter)
+@@ -661,11 +663,19 @@ static int get_ksymbol_bpf(struct kallsym_iter *iter)
*/
static int get_ksymbol_kprobe(struct kallsym_iter *iter)
{
@@ -99,12 +110,15 @@ index 3011bc33a5ba..5d41b993113f 100644
}

/* Returns space to next name. */
-@@ -687,11 +697,12 @@ static void reset_iter(struct kallsym_iter *iter, loff_t new_pos)
+@@ -688,11 +698,16 @@ static void reset_iter(struct kallsym_iter *iter, loff_t new_pos)
iter->name[0] = '\0';
iter->nameoff = get_symbol_offset(new_pos);
iter->pos = new_pos;
-- if (new_pos == 0) {
-+ if (iter->show_layout && new_pos == 0) {
++
++ if (!iter->show_layout)
++ return;
++
+ if (new_pos == 0) {
iter->pos_arch_end = 0;
iter->pos_mod_end = 0;
iter->pos_ftrace_mod_end = 0;
@@ -113,7 +127,7 @@ index 3011bc33a5ba..5d41b993113f 100644
}
}

-@@ -720,13 +731,23 @@ static int update_iter_mod(struct kallsym_iter *iter, loff_t pos)
+@@ -721,13 +736,23 @@ static int update_iter_mod(struct kallsym_iter *iter, loff_t pos)
get_ksymbol_bpf(iter))
return 1;

@@ -139,7 +153,7 @@ index 3011bc33a5ba..5d41b993113f 100644
if (pos >= kallsyms_num_syms)
return update_iter_mod(iter, pos);

-@@ -769,7 +790,7 @@ static int s_show(struct seq_file *m, void *p)
+@@ -770,7 +795,7 @@ static int s_show(struct seq_file *m, void *p)
if (!iter->name[0])
return 0;

@@ -148,7 +162,7 @@ index 3011bc33a5ba..5d41b993113f 100644

if (iter->module_name[0]) {
char type;
-@@ -806,9 +827,10 @@ static inline int kallsyms_for_perf(void)
+@@ -807,9 +832,10 @@ static inline int kallsyms_for_perf(void)
}

/*
@@ -162,7 +176,7 @@ index 3011bc33a5ba..5d41b993113f 100644
*
* Otherwise, require CAP_SYSLOG (assuming kptr_restrict isn't set to
* block even that).
-@@ -838,16 +860,54 @@ static int kallsyms_open(struct inode *inode, struct file *file)
+@@ -839,16 +865,53 @@ static int kallsyms_open(struct inode *inode, struct file *file)
* using get_symbol_offset for every symbol.
*/
struct kallsym_iter *iter;
@@ -188,15 +202,14 @@ index 3011bc33a5ba..5d41b993113f 100644
*/
- iter->show_value = kallsyms_show_value(file->f_cred);
+ fake.show_layout = kallsyms_show_value(file->f_cred);
-+ if (fake.show_layout)
-+ goto open;
-+
-+ for (pos = kallsyms_num_syms; update_iter_mod(&fake, pos); pos++)
-+ ;
++ if (!fake.show_layout) {
++ pos = kallsyms_num_syms;
++ while (update_iter_mod(&fake, pos))
++ pos++;
+
-+ size = struct_size(iter, shuffled_pos, fake.pos_end + 1);
++ size = struct_size(iter, shuffled_pos, fake.pos_end + 1);
++ }
+
-+open:
+ iter = __seq_open_private(file, &kallsyms_op, size);
+ if (!iter)
+ return -ENOMEM;
@@ -222,7 +235,7 @@ index 3011bc33a5ba..5d41b993113f 100644
return 0;
}

-@@ -858,6 +918,7 @@ const char *kdb_walk_kallsyms(loff_t *pos)
+@@ -859,6 +922,7 @@ const char *kdb_walk_kallsyms(loff_t *pos)
if (*pos == 0) {
memset(&kdb_walk_kallsyms_iter, 0,
sizeof(kdb_walk_kallsyms_iter));
@@ -231,5 +244,5 @@ index 3011bc33a5ba..5d41b993113f 100644
}
while (1) {
--
-2.33.1
+2.34.1

diff --git a/0004-arch-introduce-ASM-function-sections.patch b/0004-arch-introduce-asm-function-sections.patch
similarity index 80%
rename from 0004-arch-introduce-ASM-function-sections.patch
rename to 0004-arch-introduce-asm-function-sections.patch
index 1fc7bda06800..dc350b7b72bf 100644
--- a/0004-arch-introduce-ASM-function-sections.patch
+++ b/0004-arch-introduce-asm-function-sections.patch
@@ -1,20 +1,24 @@
-From 229aee37a21b49a0f1b0cddd993aefb71666e204 Mon Sep 17 00:00:00 2001
+From 45f20e8d29557c731b3eea5abedc3e852a7ca65e Mon Sep 17 00:00:00 2001
From: Alexander Lobakin <[email protected]>
Date: Wed, 22 Dec 2021 19:12:44 +0100
-Subject: [PATCH v9 04/15] arch: introduce ASM function sections
+Subject: [PATCH v10 04/15] arch: introduce asm function sections

-Sometimes it is useful to create a separate section for every
+Sometimes it can be useful to create a separate section for every
function (symbol in general) to be able then to selectively merge
-them back into on or several others. This is how DCE and a part of
-LTO work.
-Currently, only C functions are in scope and the compilers are able
-to do this automatically when `-ffunction-section` is specified.
+them back into one or several others. This is how Dead Code
+Elimination (DCE, CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) and a part
+of Link-Time Optimization (LTO, currently CONFIG_LTO_CLANG) work.
+Currently, this can only be done for C functions as the compilers
+are able to do this automatically when `-ffunction-sections` is
+specified.

-Add a basic infra for supporting ASM function sections. If any of
-the required build options (DCE, LTO, FG-KASLR later) is on and
-the target architecture claims it supports them, all ASM functions
-and "code" will be placed into separate named sections by default.
-This is achieved using --sectname-subst GAS flag which will then
+Add a basic infra for supporting asm function sections, which means
+support for putting functions written in assembly into separate
+sections. If any of the required build options (DCE, LTO, FG-KASLR
+later) is on and the target architecture claims it supports them,
+all asm functions and "code" will be placed into separate so-named
+("current_section.function_name") sections by default.
+This is achieved using `--sectname-subst` GAS flag which will then
substitute "%S" in a .pushsection or .section directive with the
name of the current section. So,

@@ -40,16 +44,16 @@ Signed-off-by: Alexander Lobakin <[email protected]>
Makefile | 18 +++++
arch/Kconfig | 6 ++
include/asm-generic/vmlinux.lds.h | 21 +++---
- include/linux/linkage.h | 118 +++++++++++++++++++++++++++++-
- init/Kconfig | 11 +++
+ include/linux/linkage.h | 119 +++++++++++++++++++++++++++++-
+ init/Kconfig | 10 +++
scripts/mod/modpost.c | 6 +-
6 files changed, 167 insertions(+), 13 deletions(-)

diff --git a/Makefile b/Makefile
-index 9dc15c67d132..b921b1fabf70 100644
+index fa9f947c9839..fbe2d13028f4 100644
--- a/Makefile
+++ b/Makefile
-@@ -888,6 +888,24 @@ ifeq ($(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL)$(CONFIG_LIVEPATCH),yy)
+@@ -877,6 +877,24 @@ ifeq ($(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL)$(CONFIG_LIVEPATCH),yy)
KBUILD_LDFLAGS += -z unique-symbol
endif

@@ -75,18 +79,18 @@ index 9dc15c67d132..b921b1fabf70 100644
KBUILD_CFLAGS_KERNEL += -ffunction-sections -fdata-sections
LDFLAGS_vmlinux += --gc-sections
diff --git a/arch/Kconfig b/arch/Kconfig
-index d3c4ab249e9c..b31a836bc252 100644
+index 678a80713b21..550f0599e211 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
-@@ -1312,6 +1312,12 @@ config ARCH_HAS_PARANOID_L1D_FLUSH
- config DYNAMIC_SIGFRAME
+@@ -1322,6 +1322,12 @@ config DYNAMIC_SIGFRAME
+ config HAVE_ARCH_NODE_DEV_GROUP
bool

+config ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS
+ bool
+ help
+ An arch should select this if it can be built and run with its
-+ ASM functions placed into separate sections to improve DCE and LTO.
++ asm functions placed into separate sections to improve DCE and LTO.
+
source "kernel/gcov/Kconfig"

@@ -152,16 +156,17 @@ index 42f3866bca69..e7b8a84e0e64 100644
#define HEAD_TEXT_SECTION \
.head.text : AT(ADDR(.head.text) - LOAD_OFFSET) { \
diff --git a/include/linux/linkage.h b/include/linux/linkage.h
-index dbf8506decca..0c0ddf4429dc 100644
+index dbf8506decca..f3b966a6427e 100644
--- a/include/linux/linkage.h
+++ b/include/linux/linkage.h
-@@ -73,6 +73,37 @@
+@@ -73,6 +73,38 @@
#define __ALIGN_STR ".align 4,0x90"
#endif

+/*
+ * Allow ASM symbols to have their own unique sections if they are being
-+ * generated by the compiler for C functions (DCE, LTO).
++ * generated by the compiler for C functions (DCE, LTO). Correlates with
++ * the presence of the `-ffunction-section` in KBUILD_CFLAGS.
+ */
+#if defined(CONFIG_HAVE_ASM_FUNCTION_SECTIONS) && \
+ ((defined(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) && !defined(MODULE)) || \
@@ -176,13 +181,13 @@ index dbf8506decca..0c0ddf4429dc 100644
+#define __ASM_PUSH_SECTION(name) \
+ ".pushsection %S." name ", \"ax\""
+
-+#else /* Just .text */
++#else /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || LTO)) */
+
+#define SYM_PUSH_SECTION(name)
+#define SYM_POP_SECTION()
+#define __ASM_PUSH_SECTION(name)
+
-+#endif /* Just .text */
++#endif /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || LTO)) */
+
+#define ASM_PUSH_SECTION(name) \
+ __ASM_PUSH_SECTION(__stringify(name))
@@ -193,12 +198,12 @@ index dbf8506decca..0c0ddf4429dc 100644
#ifdef __ASSEMBLY__

/* SYM_T_FUNC -- type used by assembler to mark functions */
-@@ -209,6 +240,15 @@
+@@ -209,6 +241,15 @@
SYM_START(name, SYM_L_LOCAL, SYM_A_ALIGN)
#endif

+/*
-+ * SYM_FUNC_START_WEAK -- use where there are two global names for one
++ * SYM_FUNC_START_WEAK_ALIAS -- use where there are two global names for one
+ * function, and one of them is weak
+ */
+#ifndef SYM_FUNC_START_WEAK_ALIAS
@@ -209,7 +214,7 @@ index dbf8506decca..0c0ddf4429dc 100644
/*
* SYM_FUNC_START_ALIAS -- use where there are two global names for one
* function
-@@ -225,12 +265,24 @@
+@@ -225,12 +266,24 @@
* later.
*/
#define SYM_FUNC_START(name) \
@@ -222,8 +227,8 @@ index dbf8506decca..0c0ddf4429dc 100644
+ * placed into a section specified in the second argument
+ */
+#ifndef SYM_FUNC_START_SECT
-+#define SYM_FUNC_START_SECT(name, to) \
-+ SYM_PUSH_SECTION(to) ASM_NL \
++#define SYM_FUNC_START_SECT(name, sect) \
++ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN)
#endif

@@ -234,7 +239,7 @@ index dbf8506decca..0c0ddf4429dc 100644
SYM_START(name, SYM_L_GLOBAL, SYM_A_NONE)
#endif

-@@ -238,24 +290,38 @@
+@@ -238,24 +291,38 @@
#ifndef SYM_FUNC_START_LOCAL
/* the same as SYM_FUNC_START_LOCAL_ALIAS, see comment near SYM_FUNC_START */
#define SYM_FUNC_START_LOCAL(name) \
@@ -254,8 +259,8 @@ index dbf8506decca..0c0ddf4429dc 100644
+ * will be conditionally placed into a section specified in the second argument
+ */
+#ifndef SYM_FUNC_START_LOCAL_NOALIGN_SECT
-+#define SYM_FUNC_START_LOCAL_NOALIGN_SECT(name, to) \
-+ SYM_PUSH_SECTION(to) ASM_NL \
++#define SYM_FUNC_START_LOCAL_NOALIGN_SECT(name, sect) \
++ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_LOCAL, SYM_A_NONE)
#endif

@@ -273,7 +278,7 @@ index dbf8506decca..0c0ddf4429dc 100644
SYM_START(name, SYM_L_WEAK, SYM_A_NONE)
#endif

-@@ -272,24 +338,59 @@
+@@ -272,24 +339,59 @@
#ifndef SYM_FUNC_END
/* the same as SYM_FUNC_END_ALIAS, see comment near SYM_FUNC_START */
#define SYM_FUNC_END(name) \
@@ -294,8 +299,8 @@ index dbf8506decca..0c0ddf4429dc 100644
+ * conditionally placed into a section specified in the second argument
+ */
+#ifndef SYM_CODE_START_SECT
-+#define SYM_CODE_START_SECT(name, to) \
-+ SYM_PUSH_SECTION(to) ASM_NL \
++#define SYM_CODE_START_SECT(name, sect) \
++ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN)
#endif

@@ -312,8 +317,8 @@ index dbf8506decca..0c0ddf4429dc 100644
+ * in the second argument
+ */
+#ifndef SYM_CODE_START_NOALIGN_SECT
-+#define SYM_CODE_START_NOALIGN_SECT(name, to) \
-+ SYM_PUSH_SECTION(to) ASM_NL \
++#define SYM_CODE_START_NOALIGN_SECT(name, sect) \
++ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_GLOBAL, SYM_A_NONE)
#endif

@@ -329,12 +334,12 @@ index dbf8506decca..0c0ddf4429dc 100644
+ * be conditionally placing into a section specified in the second argument
+ */
+#ifndef SYM_CODE_START_LOCAL_SECT
-+#define SYM_CODE_START_LOCAL_SECT(name, to) \
-+ SYM_PUSH_SECTION(to) ASM_NL \
++#define SYM_CODE_START_LOCAL_SECT(name, sect) \
++ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_LOCAL, SYM_A_ALIGN)
#endif

-@@ -299,13 +400,26 @@
+@@ -299,13 +401,26 @@
*/
#ifndef SYM_CODE_START_LOCAL_NOALIGN
#define SYM_CODE_START_LOCAL_NOALIGN(name) \
@@ -348,8 +353,8 @@ index dbf8506decca..0c0ddf4429dc 100644
+ * specified in the second argument
+ */
+#ifndef SYM_CODE_START_LOCAL_NOALIGN_SECT
-+#define SYM_CODE_START_LOCAL_NOALIGN_SECT(name, to) \
-+ SYM_PUSH_SECTION(to) ASM_NL \
++#define SYM_CODE_START_LOCAL_NOALIGN_SECT(name, sect) \
++ SYM_PUSH_SECTION(sect) ASM_NL \
SYM_START(name, SYM_L_LOCAL, SYM_A_NONE)
#endif

@@ -363,10 +368,10 @@ index dbf8506decca..0c0ddf4429dc 100644

/* === data annotations === */
diff --git a/init/Kconfig b/init/Kconfig
-index 37926d19a74a..3babc0aeac61 100644
+index 8e900d17d42b..4acfc80f22df 100644
--- a/init/Kconfig
+++ b/init/Kconfig
-@@ -1386,6 +1386,17 @@ config CC_OPTIMIZE_FOR_SIZE
+@@ -1386,6 +1386,16 @@ config CC_OPTIMIZE_FOR_SIZE

endchoice

@@ -375,17 +380,16 @@ index 37926d19a74a..3babc0aeac61 100644
+ depends on $(cc-option,-Wa$(comma)--sectname-subst)
+ def_bool y
+ help
-+ This enables ASM function sections if both architecture
-+ and toolchain supports that. It allows creating a separate
-+ .text section for each ASM function in order to improve
-+ DCE and LTO (works the same way as -ffunction-sections for
-+ C code).
++ This enables asm function sections if both architecture and
++ toolchain support it. It allows creating a separate section
++ for each function written in assembly in order to improve DCE
++ and LTO (works the same way as -ffunction-sections for C code).
+
config HAVE_LD_DEAD_CODE_DATA_ELIMINATION
bool
help
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
-index f39cc73a082c..a6e9e75ff3e5 100644
+index ec521ccebea6..84d2c44f9383 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -960,7 +960,9 @@ static void check_section(const char *modname, struct elf_info *elf,
@@ -409,5 +413,5 @@ index f39cc73a082c..a6e9e75ff3e5 100644
const char *symbol_white_list[20];
void (*handler)(const char *modname, struct elf_info *elf,
--
-2.33.1
+2.34.1

diff --git a/0005-x86-support-ASM-function-sections.patch b/0005-x86-support-asm-function-sections.patch
similarity index 88%
rename from 0005-x86-support-ASM-function-sections.patch
rename to 0005-x86-support-asm-function-sections.patch
index feb5fb741943..4c6889d57782 100644
--- a/0005-x86-support-ASM-function-sections.patch
+++ b/0005-x86-support-asm-function-sections.patch
@@ -1,18 +1,19 @@
-From 645835128b5bbf6aabe83383b7075974229506b9 Mon Sep 17 00:00:00 2001
+From 1c8cf0c01c1e62a345807d0e877afcace2f6a555 Mon Sep 17 00:00:00 2001
From: Alexander Lobakin <[email protected]>
Date: Wed, 22 Dec 2021 20:15:03 +0100
-Subject: [PATCH v9 05/15] x86: support ASM function sections
+Subject: [PATCH v10 05/15] x86: support asm function sections

Address places which need special care and enable
CONFIG_ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS.

Notably:
- - propagate --sectname-subst to aflags in x86/boot/Makefile and
- x86/boot/compressed/Makefile as both override aflags;
- - symbols starting with a dot (like ".Lbad_gs") should be handled
- manually with SYM_*_START_SECT(.Lbad_gs, bad_gs) as "two dots"
- is a special (and CPP doesn't want to concatenate two dots in
- general);
+ - propagate `--sectname-subst` to KBUILD_AFLAGS in
+ x86/boot/Makefile and x86/boot/compressed/Makefile as both
+ override them;
+ - symbols starting with a dot (like ".Lrelocated") should be
+ handled manually with SYM_*_START_SECT(.Lrelocated, relocated)
+ as "two dots" is a special (and CPP doesn't want to concatenate
+ two dots in general);
- some symbols explicitly need to reside in one section (like
kexec control code, hibernation page etc.);
- macros creating aliases for functions (like __memcpy() for
@@ -21,12 +22,12 @@ Notably:
don't have SYM_PUSH_SECTION() inside);
- things like ".org", ".align" should be manually pushed to
the same section the next symbol goes to;
- - expand indirect_thunk and .fixup wildcards in vmlinux.lds.S
- to catch symbols back into the "main" section;
+ - expand indirect_thunk wildcards in vmlinux.lds.S to catch
+ symbols back into the "main" section;
- inline ASM functions like __raw_callee*() should be pushed
manually as well.

-With these changes and `-ffunction-sections enabled`, "plain"
+With these changes and `-ffunction-sections` enabled, "plain"
".text" section is empty which means that everything works
right as expected.

@@ -40,7 +41,6 @@ Signed-off-by: Alexander Lobakin <[email protected]>
arch/x86/boot/pmjump.S | 2 +-
arch/x86/crypto/aesni-intel_asm.S | 4 +--
arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 4 +++
- arch/x86/entry/entry_64.S | 2 +-
arch/x86/include/asm/paravirt.h | 2 ++
arch/x86/include/asm/qspinlock_paravirt.h | 2 ++
arch/x86/kernel/head_32.S | 4 +--
@@ -49,7 +49,7 @@ Signed-off-by: Alexander Lobakin <[email protected]>
arch/x86/kernel/kvm.c | 2 ++
arch/x86/kernel/relocate_kernel_32.S | 10 +++---
arch/x86/kernel/relocate_kernel_64.S | 12 ++++---
- arch/x86/kernel/vmlinux.lds.S | 4 +--
+ arch/x86/kernel/vmlinux.lds.S | 2 +-
arch/x86/kvm/emulate.c | 7 +++-
arch/x86/lib/copy_user_64.S | 2 +-
arch/x86/lib/error-inject.c | 2 ++
@@ -60,10 +60,10 @@ Signed-off-by: Alexander Lobakin <[email protected]>
arch/x86/lib/putuser.S | 2 +-
arch/x86/power/hibernate_asm_32.S | 10 +++---
arch/x86/power/hibernate_asm_64.S | 10 +++---
- 28 files changed, 91 insertions(+), 52 deletions(-)
+ 27 files changed, 89 insertions(+), 50 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
-index 5c2ccb85f2ef..3e4ea355147b 100644
+index 391c4cac8958..f6bb48d41349 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -102,6 +102,7 @@ config X86
@@ -73,7 +73,7 @@ index 5c2ccb85f2ef..3e4ea355147b 100644
+ select ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS
select ARCH_SUPPORTS_ATOMIC_RMW
select ARCH_SUPPORTS_DEBUG_PAGEALLOC
- select ARCH_SUPPORTS_NUMA_BALANCING if X86_64
+ select ARCH_SUPPORTS_PAGE_TABLE_CHECK if X86_64
diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile
index b5aecb524a8a..080990b09f06 100644
--- a/arch/x86/boot/Makefile
@@ -87,10 +87,10 @@ index b5aecb524a8a..080990b09f06 100644
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
GCOV_PROFILE := n
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
-index 431bf7f846c3..4bf32db56010 100644
+index 6115274fe10f..e7ee94d5f55a 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
-@@ -55,6 +55,7 @@ KBUILD_CFLAGS += $(CLANG_FLAGS)
+@@ -58,6 +58,7 @@ KBUILD_CFLAGS += -include $(srctree)/include/linux/hidden.h
CFLAGS_sev.o += -I$(objtree)/arch/x86/lib/

KBUILD_AFLAGS := $(KBUILD_CFLAGS) -D__ASSEMBLY__
@@ -113,7 +113,7 @@ index 659fad53ca82..a3a667f5e5cd 100644

.text
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
-index 572c535cf45b..c496dab184ef 100644
+index fd9441f40457..b6fb27fd7abd 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -72,7 +72,7 @@
@@ -251,7 +251,7 @@ index cbec8bd0841f..e647c17000a9 100644
movl %ecx, %ds
movl %ecx, %es
diff --git a/arch/x86/crypto/aesni-intel_asm.S b/arch/x86/crypto/aesni-intel_asm.S
-index 4e3972570916..d3d511058520 100644
+index 363699dd7220..6b92beb7820a 100644
--- a/arch/x86/crypto/aesni-intel_asm.S
+++ b/arch/x86/crypto/aesni-intel_asm.S
@@ -1752,8 +1752,8 @@ SYM_FUNC_END(aesni_gcm_finalize)
@@ -267,7 +267,7 @@ index 4e3972570916..d3d511058520 100644
@@ -1763,8 +1763,8 @@ SYM_FUNC_START_LOCAL(_key_expansion_256a)
movaps %xmm0, (TKEYP)
add $0x10, TKEYP
- ret
+ RET
-SYM_FUNC_END(_key_expansion_256a)
SYM_FUNC_END_ALIAS(_key_expansion_128)
+SYM_FUNC_END(_key_expansion_256a)
@@ -304,21 +304,8 @@ index 71fae5a09e56..221a4596f390 100644
___

&declare_function("poly1305_blocks_avx", 32, 4);
-diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
-index 97b1f84bb53f..15eae25753fb 100644
---- a/arch/x86/entry/entry_64.S
-+++ b/arch/x86/entry/entry_64.S
-@@ -745,7 +745,7 @@ EXPORT_SYMBOL(asm_load_gs_index)
- _ASM_EXTABLE(.Lgs_change, .Lbad_gs)
- .section .fixup, "ax"
- /* running with kernelgs */
--SYM_CODE_START_LOCAL_NOALIGN(.Lbad_gs)
-+SYM_CODE_START_LOCAL_NOALIGN_SECT(.Lbad_gs, bad_gs)
- swapgs /* switch back to user gs */
- .macro ZAP_GS
- /* This can't be a string because the preprocessor needs to see it. */
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
-index 21c4a694ca11..f045af533161 100644
+index 0d76502cc6f5..3efdac789bc4 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -663,6 +663,7 @@ bool __raw_callee_save___native_vcpu_is_preempted(long cpu);
@@ -331,14 +318,14 @@ index 21c4a694ca11..f045af533161 100644
PV_THUNK_NAME(func) ":" \
@@ -673,6 +674,7 @@ bool __raw_callee_save___native_vcpu_is_preempted(long cpu);
FRAME_END \
- "ret;" \
+ ASM_RET \
".size " PV_THUNK_NAME(func) ", .-" PV_THUNK_NAME(func) ";" \
+ ASM_POP_SECTION() ";" \
".popsection")

#define PV_CALLEE_SAVE_REGS_THUNK(func) \
diff --git a/arch/x86/include/asm/qspinlock_paravirt.h b/arch/x86/include/asm/qspinlock_paravirt.h
-index 159622ee0674..209f610dda18 100644
+index 1474cf96251d..3fde623bbcb2 100644
--- a/arch/x86/include/asm/qspinlock_paravirt.h
+++ b/arch/x86/include/asm/qspinlock_paravirt.h
@@ -35,6 +35,7 @@ PV_CALLEE_SAVE_REGS_THUNK(__pv_queued_spin_unlock_slowpath);
@@ -351,19 +338,19 @@ index 159622ee0674..209f610dda18 100644
".align 4,0x90;"
@@ -58,6 +59,7 @@ asm (".pushsection .text;"
FRAME_END
- "ret;"
+ ASM_RET
".size " PV_UNLOCK ", .-" PV_UNLOCK ";"
+ ASM_POP_SECTION() ";"
".popsection");

#else /* CONFIG_64BIT */
diff --git a/arch/x86/kernel/head_32.S b/arch/x86/kernel/head_32.S
-index d8c64dab0efe..aa7a99876397 100644
+index eb8656bac99b..d58422148481 100644
--- a/arch/x86/kernel/head_32.S
+++ b/arch/x86/kernel/head_32.S
@@ -342,7 +342,7 @@ setup_once:
andl $0,setup_once_ref /* Once is enough, thanks */
- ret
+ RET

-SYM_FUNC_START(early_idt_handler_array)
+SYM_FUNC_START_SECT(early_idt_handler_array, early_idt_handler)
@@ -380,10 +367,10 @@ index d8c64dab0efe..aa7a99876397 100644
* The stack is the hardware frame, an error code or zero, and the
* vector number.
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
-index d8b3ebd2bb85..1bc081c92775 100644
+index 9c63fc5988cd..a19b6fa2bf87 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
-@@ -350,7 +350,7 @@ SYM_DATA(initial_stack, .quad init_thread_union + THREAD_SIZE - FRAME_SIZE)
+@@ -367,7 +367,7 @@ SYM_DATA(initial_stack, .quad init_thread_union + THREAD_SIZE - FRAME_SIZE)
__FINITDATA

__INIT
@@ -392,7 +379,7 @@ index d8b3ebd2bb85..1bc081c92775 100644
i = 0
.rept NUM_EXCEPTION_VECTORS
.if ((EXCEPTION_ERRCODE_MASK >> i) & 1) == 0
-@@ -368,7 +368,7 @@ SYM_CODE_START(early_idt_handler_array)
+@@ -385,7 +385,7 @@ SYM_CODE_START(early_idt_handler_array)
UNWIND_HINT_IRET_REGS offset=16
SYM_CODE_END(early_idt_handler_array)

@@ -402,7 +389,7 @@ index d8b3ebd2bb85..1bc081c92775 100644
* The stack is the hardware frame, an error code or zero, and the
* vector number.
diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c
-index fce99e249d61..6938dccfa4a7 100644
+index 6290712cb36d..e0be6bc76337 100644
--- a/arch/x86/kernel/kprobes/core.c
+++ b/arch/x86/kernel/kprobes/core.c
@@ -1019,6 +1019,7 @@ NOKPROBE_SYMBOL(kprobe_int3_handler);
@@ -415,14 +402,14 @@ index fce99e249d61..6938dccfa4a7 100644
"__kretprobe_trampoline:\n"
@@ -1053,6 +1054,7 @@ asm(
#endif
- " ret\n"
+ ASM_RET
".size __kretprobe_trampoline, .-__kretprobe_trampoline\n"
+ ASM_POP_SECTION() "\n"
);
NOKPROBE_SYMBOL(__kretprobe_trampoline);
/*
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
-index 59abbdad7729..25a08ba473a0 100644
+index a438217cbfac..6a497d5647e7 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -1021,6 +1021,7 @@ extern bool __raw_callee_save___kvm_vcpu_is_preempted(long);
@@ -442,7 +429,7 @@ index 59abbdad7729..25a08ba473a0 100644

#endif
diff --git a/arch/x86/kernel/relocate_kernel_32.S b/arch/x86/kernel/relocate_kernel_32.S
-index f469153eca8a..9077aa9367ca 100644
+index fcc8a7699103..ff36f21e665a 100644
--- a/arch/x86/kernel/relocate_kernel_32.S
+++ b/arch/x86/kernel/relocate_kernel_32.S
@@ -35,7 +35,7 @@
@@ -455,7 +442,7 @@ index f469153eca8a..9077aa9367ca 100644

pushl %ebx
@@ -94,7 +94,7 @@ SYM_CODE_START_NOALIGN(relocate_kernel)
- ret
+ RET
SYM_CODE_END(relocate_kernel)

-SYM_CODE_START_LOCAL_NOALIGN(identity_mapped)
@@ -464,7 +451,7 @@ index f469153eca8a..9077aa9367ca 100644
pushl $0
/* store the start address on the stack */
@@ -193,7 +193,7 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped)
- ret
+ RET
SYM_CODE_END(identity_mapped)

-SYM_CODE_START_LOCAL_NOALIGN(virtual_mapped)
@@ -482,7 +469,7 @@ index f469153eca8a..9077aa9367ca 100644
movl 4(%esp), %ecx
pushl %ebp
@@ -274,5 +274,7 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages)
- ret
+ RET
SYM_CODE_END(swap_pages)

+SYM_PUSH_SECTION(kexec_control_code)
@@ -490,7 +477,7 @@ index f469153eca8a..9077aa9367ca 100644
.set kexec_control_code_size, . - relocate_kernel
+SYM_POP_SECTION()
diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocate_kernel_64.S
-index c8fe74a28143..91336d71dd2d 100644
+index 399f075ccdc4..fb8ff461436e 100644
--- a/arch/x86/kernel/relocate_kernel_64.S
+++ b/arch/x86/kernel/relocate_kernel_64.S
@@ -38,9 +38,11 @@
@@ -507,7 +494,7 @@ index c8fe74a28143..91336d71dd2d 100644
/*
* %rdi indirection_page
@@ -107,7 +109,7 @@ SYM_CODE_START_NOALIGN(relocate_kernel)
- ret
+ RET
SYM_CODE_END(relocate_kernel)

-SYM_CODE_START_LOCAL_NOALIGN(identity_mapped)
@@ -516,7 +503,7 @@ index c8fe74a28143..91336d71dd2d 100644
/* set return address to 0 if not preserving context */
pushq $0
@@ -213,7 +215,7 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped)
- ret
+ RET
SYM_CODE_END(identity_mapped)

-SYM_CODE_START_LOCAL_NOALIGN(virtual_mapped)
@@ -534,7 +521,7 @@ index c8fe74a28143..91336d71dd2d 100644
movq %rdi, %rcx /* Put the page_list in %rcx */
xorl %edi, %edi
@@ -291,5 +293,7 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages)
- ret
+ RET
SYM_CODE_END(swap_pages)

+SYM_PUSH_SECTION(kexec_control_code)
@@ -542,16 +529,10 @@ index c8fe74a28143..91336d71dd2d 100644
.set kexec_control_code_size, . - relocate_kernel
+SYM_POP_SECTION()
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
-index 3d6dc12d198f..6f026400261e 100644
+index 27f830345b6f..5550bd68f6e7 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
-@@ -137,12 +137,12 @@ SECTIONS
- ALIGN_ENTRY_TEXT_END
- SOFTIRQENTRY_TEXT
- STATIC_CALL_TEXT
-- *(.fixup)
-+ *(SECT_WILDCARD(.fixup))
- *(.gnu.warning)
+@@ -141,7 +141,7 @@ SECTIONS

#ifdef CONFIG_RETPOLINE
__indirect_thunk_start = .;
@@ -561,10 +542,10 @@ index 3d6dc12d198f..6f026400261e 100644
#endif
} :text =0xcccc
diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
-index 28b1a4e57827..00466f15c4ed 100644
+index 40da8c7f3019..85ce58b041ac 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
-@@ -307,6 +307,7 @@ static void invalidate_registers(struct x86_emulate_ctxt *ctxt)
+@@ -309,6 +309,7 @@ static void invalidate_registers(struct x86_emulate_ctxt *ctxt)
static int fastop(struct x86_emulate_ctxt *ctxt, fastop_t fop);

#define __FOP_FUNC(name) \
@@ -572,17 +553,17 @@ index 28b1a4e57827..00466f15c4ed 100644
".align " __stringify(FASTOP_SIZE) " \n\t" \
".type " name ", @function \n\t" \
name ":\n\t"
-@@ -316,7 +317,8 @@ static int fastop(struct x86_emulate_ctxt *ctxt, fastop_t fop);
+@@ -318,7 +319,8 @@ static int fastop(struct x86_emulate_ctxt *ctxt, fastop_t fop);

#define __FOP_RET(name) \
- "ret \n\t" \
+ "11: " ASM_RET \
- ".size " name ", .-" name "\n\t"
+ ".size " name ", .-" name "\n\t" \
+ ASM_POP_SECTION() "\n\t"

#define FOP_RET(name) \
__FOP_RET(#name)
-@@ -324,11 +326,13 @@ static int fastop(struct x86_emulate_ctxt *ctxt, fastop_t fop);
+@@ -326,11 +328,13 @@ static int fastop(struct x86_emulate_ctxt *ctxt, fastop_t fop);
#define FOP_START(op) \
extern void em_##op(struct fastop *fake); \
asm(".pushsection .text, \"ax\" \n\t" \
@@ -596,7 +577,7 @@ index 28b1a4e57827..00466f15c4ed 100644
".popsection")

#define __FOPNOP(name) \
-@@ -428,6 +432,7 @@ static int fastop(struct x86_emulate_ctxt *ctxt, fastop_t fop);
+@@ -430,6 +434,7 @@ static int fastop(struct x86_emulate_ctxt *ctxt, fastop_t fop);

/* Special case for SETcc - 1 instruction per cc */
#define FOP_SETCC(op) \
@@ -605,23 +586,23 @@ index 28b1a4e57827..00466f15c4ed 100644
".type " #op ", @function \n\t" \
#op ": \n\t" \
diff --git a/arch/x86/lib/copy_user_64.S b/arch/x86/lib/copy_user_64.S
-index 2797e630b9b1..d351e1288a23 100644
+index 8ca5ecf16dc4..ddbd6b5b7b47 100644
--- a/arch/x86/lib/copy_user_64.S
+++ b/arch/x86/lib/copy_user_64.S
-@@ -232,7 +232,7 @@ EXPORT_SYMBOL(copy_user_enhanced_fast_string)
+@@ -220,7 +220,7 @@ EXPORT_SYMBOL(copy_user_enhanced_fast_string)
* Output:
* eax uncopied bytes or 0 if successful.
*/
-SYM_CODE_START_LOCAL(.Lcopy_user_handle_tail)
+SYM_CODE_START_LOCAL_SECT(.Lcopy_user_handle_tail, copy_user_handle_tail)
- movl %edx,%ecx
- 1: rep movsb
- 2: mov %ecx,%eax
+ cmp $X86_TRAP_MC,%eax
+ je 3f
+
diff --git a/arch/x86/lib/error-inject.c b/arch/x86/lib/error-inject.c
-index be5b5fb1598b..e3ecab588a4c 100644
+index 520897061ee0..720a73d4ee70 100644
--- a/arch/x86/lib/error-inject.c
+++ b/arch/x86/lib/error-inject.c
-@@ -7,11 +7,13 @@ asmlinkage void just_return_func(void);
+@@ -8,11 +8,13 @@ asmlinkage void just_return_func(void);

asm(
".text\n"
@@ -629,14 +610,14 @@ index be5b5fb1598b..e3ecab588a4c 100644
".type just_return_func, @function\n"
".globl just_return_func\n"
"just_return_func:\n"
- " ret\n"
+ ASM_RET
".size just_return_func, .-just_return_func\n"
+ ASM_POP_SECTION() "\n"
);

void override_function_with_return(struct pt_regs *regs)
diff --git a/arch/x86/lib/getuser.S b/arch/x86/lib/getuser.S
-index fa1bc2104b32..35d93412605a 100644
+index b70d98d79a9d..06d288909a68 100644
--- a/arch/x86/lib/getuser.S
+++ b/arch/x86/lib/getuser.S
@@ -163,8 +163,7 @@ SYM_FUNC_START(__get_user_nocheck_8)
@@ -659,7 +640,7 @@ index fa1bc2104b32..35d93412605a 100644
bad_get_user_8:
xor %edx,%edx
diff --git a/arch/x86/lib/memcpy_64.S b/arch/x86/lib/memcpy_64.S
-index 1cc9da6e29c7..1414c62f7c32 100644
+index 59cf2343f3d9..2dc6033e2932 100644
--- a/arch/x86/lib/memcpy_64.S
+++ b/arch/x86/lib/memcpy_64.S
@@ -27,8 +27,8 @@
@@ -675,7 +656,7 @@ index 1cc9da6e29c7..1414c62f7c32 100644
@@ -40,8 +40,8 @@ SYM_FUNC_START_WEAK(memcpy)
movl %edx, %ecx
rep movsb
- ret
+ RET
-SYM_FUNC_END(memcpy)
SYM_FUNC_END_ALIAS(__memcpy)
+SYM_FUNC_END(memcpy)
@@ -683,7 +664,7 @@ index 1cc9da6e29c7..1414c62f7c32 100644
EXPORT_SYMBOL(__memcpy)

diff --git a/arch/x86/lib/memmove_64.S b/arch/x86/lib/memmove_64.S
-index 64801010d312..e397791ec067 100644
+index 50ea390df712..0040ac38751b 100644
--- a/arch/x86/lib/memmove_64.S
+++ b/arch/x86/lib/memmove_64.S
@@ -24,9 +24,8 @@
@@ -700,14 +681,14 @@ index 64801010d312..e397791ec067 100644
@@ -206,7 +205,7 @@ SYM_FUNC_START(__memmove)
movb %r11b, (%rdi)
13:
- retq
+ RET
-SYM_FUNC_END(__memmove)
SYM_FUNC_END_ALIAS(memmove)
+SYM_FUNC_END(__memmove)
EXPORT_SYMBOL(__memmove)
EXPORT_SYMBOL(memmove)
diff --git a/arch/x86/lib/memset_64.S b/arch/x86/lib/memset_64.S
-index 9827ae267f96..5fa7f64edc9c 100644
+index d624f2bc42f1..32cf147393e7 100644
--- a/arch/x86/lib/memset_64.S
+++ b/arch/x86/lib/memset_64.S
@@ -17,8 +17,9 @@
@@ -724,7 +705,7 @@ index 9827ae267f96..5fa7f64edc9c 100644
@@ -41,8 +42,8 @@ SYM_FUNC_START(__memset)
rep stosb
movq %r9,%rax
- ret
+ RET
-SYM_FUNC_END(__memset)
SYM_FUNC_END_ALIAS(memset)
+SYM_FUNC_END(__memset)
@@ -732,7 +713,7 @@ index 9827ae267f96..5fa7f64edc9c 100644
EXPORT_SYMBOL(__memset)

diff --git a/arch/x86/lib/putuser.S b/arch/x86/lib/putuser.S
-index 0ea344c5ea43..a7e1be50841a 100644
+index ecb2049c1273..be6b2dc0967c 100644
--- a/arch/x86/lib/putuser.S
+++ b/arch/x86/lib/putuser.S
@@ -102,7 +102,7 @@ SYM_FUNC_END(__put_user_8)
@@ -745,7 +726,7 @@ index 0ea344c5ea43..a7e1be50841a 100644
.Lbad_put_user:
movl $-EFAULT,%ecx
diff --git a/arch/x86/power/hibernate_asm_32.S b/arch/x86/power/hibernate_asm_32.S
-index 8786653ad3c0..492dd2bcce4b 100644
+index 5606a15cf9a1..06f666e3e28a 100644
--- a/arch/x86/power/hibernate_asm_32.S
+++ b/arch/x86/power/hibernate_asm_32.S
@@ -16,7 +16,7 @@
@@ -758,7 +739,7 @@ index 8786653ad3c0..492dd2bcce4b 100644
movl %ebx, saved_context_ebx
movl %ebp, saved_context_ebp
@@ -35,7 +35,7 @@ SYM_FUNC_START(swsusp_arch_suspend)
- ret
+ RET
SYM_FUNC_END(swsusp_arch_suspend)

-SYM_CODE_START(restore_image)
@@ -788,7 +769,7 @@ index 8786653ad3c0..492dd2bcce4b 100644
movl %ebp, %cr3
movl mmu_cr4_features, %ecx
diff --git a/arch/x86/power/hibernate_asm_64.S b/arch/x86/power/hibernate_asm_64.S
-index d9bed596d849..225be7a985d5 100644
+index 0a0539e1cc81..fdca1949ad9f 100644
--- a/arch/x86/power/hibernate_asm_64.S
+++ b/arch/x86/power/hibernate_asm_64.S
@@ -24,8 +24,10 @@
@@ -804,7 +785,7 @@ index d9bed596d849..225be7a985d5 100644
movq %r9, %cr3

@@ -69,7 +71,7 @@ SYM_FUNC_START(restore_registers)
- ret
+ RET
SYM_FUNC_END(restore_registers)

-SYM_FUNC_START(swsusp_arch_suspend)
@@ -813,7 +794,7 @@ index d9bed596d849..225be7a985d5 100644
movq %rsp, pt_regs_sp(%rax)
movq %rbp, pt_regs_bp(%rax)
@@ -99,7 +101,7 @@ SYM_FUNC_START(swsusp_arch_suspend)
- ret
+ RET
SYM_FUNC_END(swsusp_arch_suspend)

-SYM_FUNC_START(restore_image)
@@ -831,5 +812,5 @@ index d9bed596d849..225be7a985d5 100644
movq %rax, %cr3
/* flush TLB */
--
-2.33.1
+2.34.1

diff --git a/0006-x86-decouple-ORC-table-sorting-into-a-separate-file.patch b/0006-x86-decouple-ORC-table-sorting-into-a-separate-file.patch
index 15cebf05e392..8d19203e6f3d 100644
--- a/0006-x86-decouple-ORC-table-sorting-into-a-separate-file.patch
+++ b/0006-x86-decouple-ORC-table-sorting-into-a-separate-file.patch
@@ -1,7 +1,8 @@
-From 3a67355398231e6c81343069cb4682e6e3404766 Mon Sep 17 00:00:00 2001
+From 51fa39c629ffb032548911b411b3fd4dc6b48788 Mon Sep 17 00:00:00 2001
From: Alexander Lobakin <[email protected]>
Date: Wed, 22 Dec 2021 21:36:33 +0100
-Subject: [PATCH v9 06/15] x86: decouple ORC table sorting into a separate file
+Subject: [PATCH v10 06/15] x86: decouple ORC table sorting into a separate
+ file

In order to be able to sort ORC entries from both the kernel and
the pre-boot compressed environment, place ORC sorting function
@@ -18,11 +19,11 @@ Signed-off-by: Alexander Lobakin <[email protected]>
---
arch/x86/include/asm/orc_types.h | 7 +++
arch/x86/kernel/unwind_orc.c | 63 +--------------------
- arch/x86/lib/Makefile | 1 +
- arch/x86/lib/orc.c | 76 ++++++++++++++++++++++++++
+ arch/x86/lib/Makefile | 3 +
+ arch/x86/lib/orc.c | 78 ++++++++++++++++++++++++++
scripts/sorttable.h | 5 --
tools/arch/x86/include/asm/orc_types.h | 7 +++
- 6 files changed, 92 insertions(+), 67 deletions(-)
+ 6 files changed, 96 insertions(+), 67 deletions(-)
create mode 100644 arch/x86/lib/orc.c

diff --git a/arch/x86/include/asm/orc_types.h b/arch/x86/include/asm/orc_types.h
@@ -132,23 +133,25 @@ index 2de3c8c5eba9..e5748bf15966 100644
mod->arch.orc_unwind_ip = orc_ip;
mod->arch.orc_unwind = orc;
diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile
-index c6506c6a7092..4335518adcaf 100644
+index f76747862bd2..ff094cecebc4 100644
--- a/arch/x86/lib/Makefile
+++ b/arch/x86/lib/Makefile
-@@ -50,6 +50,7 @@ lib-$(CONFIG_INSTRUCTION_DECODER) += insn.o inat.o insn-eval.o
+@@ -50,6 +50,9 @@ lib-$(CONFIG_INSTRUCTION_DECODER) += insn.o inat.o insn-eval.o
lib-$(CONFIG_RANDOMIZE_BASE) += kaslr.o
lib-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
lib-$(CONFIG_RETPOLINE) += retpoline.o
++ifdef CONFIG_MODULES
+lib-$(CONFIG_UNWINDER_ORC) += orc.o
++endif

obj-y += msr.o msr-reg.o msr-reg-export.o hweight.o
obj-y += iomem.o
diff --git a/arch/x86/lib/orc.c b/arch/x86/lib/orc.c
new file mode 100644
-index 000000000000..5c37494bbbb5
+index 000000000000..6a42842bf4ef
--- /dev/null
+++ b/arch/x86/lib/orc.c
-@@ -0,0 +1,76 @@
+@@ -0,0 +1,78 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * ORC sorting shared by the compressed boot code and ORC module
@@ -208,6 +211,7 @@ index 000000000000..5c37494bbbb5
+ * whitelisted .o files which didn't get objtool generation.
+ */
+ orc_a = cur_orc_table + (a - cur_orc_ip_table);
++
+ return orc_a->sp_reg == ORC_REG_UNDEFINED && !orc_a->end ? -1 : 1;
+}
+
@@ -219,17 +223,18 @@ index 000000000000..5c37494bbbb5
+ * .orc_unwind entry so they can both be swapped.
+ */
+ sort_mutex_lock();
++
+ cur_orc_ip_table = ip_table;
+ cur_orc_table = orc_table;
-+ sort(ip_table, num_orcs, sizeof(int), orc_sort_cmp,
-+ orc_sort_swap);
++ sort(ip_table, num_orcs, sizeof(int), orc_sort_cmp, orc_sort_swap);
++
+ sort_mutex_unlock();
+}
diff --git a/scripts/sorttable.h b/scripts/sorttable.h
-index a2baa2fefb13..44f8d7d654ff 100644
+index deb7c1d3e979..a6bb46f36854 100644
--- a/scripts/sorttable.h
+++ b/scripts/sorttable.h
-@@ -96,11 +96,6 @@ struct orc_entry *g_orc_table;
+@@ -103,11 +103,6 @@ struct orc_entry *g_orc_table;

pthread_t orc_sort_thread;

@@ -260,5 +265,5 @@ index 5a2baf28a1dc..7708548713c4 100644

#endif /* _ORC_TYPES_H */
--
-2.33.1
+2.34.1

diff --git a/0007-Makefile-Add-build-and-config-option-for-CONFIG_FG_K.patch b/0007-Makefile-add-config-options-and-build-scripts-for-FG.patch
similarity index 65%
rename from 0007-Makefile-Add-build-and-config-option-for-CONFIG_FG_K.patch
rename to 0007-Makefile-add-config-options-and-build-scripts-for-FG.patch
index c798a0b89456..8b8445ae9c96 100644
--- a/0007-Makefile-Add-build-and-config-option-for-CONFIG_FG_K.patch
+++ b/0007-Makefile-add-config-options-and-build-scripts-for-FG.patch
@@ -1,35 +1,38 @@
-From 259a6e21bc646c8e0a5e1f3928ca389064246d1b Mon Sep 17 00:00:00 2001
+From 4dabd4d7cc343fc2c0af0a3a9326dbbc06922808 Mon Sep 17 00:00:00 2001
From: Kristen Carlson Accardi <[email protected]>
Date: Fri, 27 Aug 2021 15:57:37 +0200
-Subject: [PATCH v9 07/15] Makefile: Add build and config option for
- CONFIG_FG_KASLR
+Subject: [PATCH v10 07/15] Makefile: add config options and build scripts for
+ FG-KASLR

-Allow user to select CONFIG_FG_KASLR if dependencies are met. Change
-the make file to build with -ffunction-sections if CONFIG_FG_KASLR.
+Add Kconfig symbols CONFIG_ARCH_SUPPORTS_FG_KASLR and
+CONFIG_FG_KASLR. The first is hidden and used to indicate that
+a particular architecture supports it, the second allows a user
+to enable FG-KASLR when the former is set to 'y'.
+Make Kbuild not consolidate function sections back into `.text`
+on linking if CONFIG_FG_KASLR is enabled (even with
+CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y) as the feature itself
+relies on functions still being separated in the final vmlinux.

-While the only architecture that supports CONFIG_FG_KASLR does not
-currently enable HAVE_LD_DEAD_CODE_DATA_ELIMINATION, make sure these
-2 features play nicely together for the future by ensuring that if
-CONFIG_LD_DEAD_CODE_DATA_ELIMINATION is selected when used with
-CONFIG_FG_KASLR the function sections will not be consolidated back
-into .text. Thanks to Kees Cook for the dead code elimination changes.
+Alexander Lobakin:

-alobakin:
-Improve cflags management in the top Makefile: don't turn on
--f{data,function}-sections with ClangLTO as this is a no-op
-provoking a full rebuild.
-Add ".symtab_shndx" to the list of known sections since we are going
-to support it. Otherwise LD will emit a warning when there are more
-than 64k sections and CONFIG_LD_ORPHAN_WARN=y.
+Improve KBUILD_CFLAGS{,_MODULE} management in the top Makefile:
+don't turn on -f{data,function}-sections with ClangLTO as this is a
+no-op provoking a full rebuild.
+Add ".symtab_shndx" to the list of known sections since it is going
+to be supported by the architecture-specific code. Otherwise LD
+emits a warning when there are more than 64k sections and
+CONFIG_LD_ORPHAN_WARN=y.
Turn ".text" LD script wildcard into ".text.__unused__" to make sure
all kernel code will land into our special sections.
Make FG-KASLR depend on `-z unique-symbol`. With every function being
in a separate section (randomly ordered each boot), position-based
-search is impossible. This flag is likely to be widely available.
+search is impossible. This flag is likely to be widely available
+(on non-LLD builds).

Signed-off-by: Kristen Carlson Accardi <[email protected]>
-Reviewed-by: Tony Luck <[email protected]>
+Suggested-by: Kees Cook <[email protected]> # coexistence with DCE
Reviewed-by: Kees Cook <[email protected]>
+Reviewed-by: Tony Luck <[email protected]>
Tested-by: Tony Luck <[email protected]>
Co-developed-by: Alexander Lobakin <[email protected]>
Signed-off-by: Alexander Lobakin <[email protected]>
@@ -37,15 +40,15 @@ Signed-off-by: Alexander Lobakin <[email protected]>
Makefile | 17 ++++++++++++++---
arch/Kconfig | 6 +++++-
include/asm-generic/vmlinux.lds.h | 20 ++++++++++++++++++--
- include/linux/linkage.h | 3 ++-
- init/Kconfig | 18 ++++++++++++++++--
- 5 files changed, 55 insertions(+), 9 deletions(-)
+ include/linux/linkage.h | 9 +++++----
+ init/Kconfig | 19 +++++++++++++++++--
+ 5 files changed, 59 insertions(+), 12 deletions(-)

diff --git a/Makefile b/Makefile
-index b921b1fabf70..3346269341d4 100644
+index fbe2d13028f4..4328d53d8b25 100644
--- a/Makefile
+++ b/Makefile
-@@ -883,7 +883,7 @@ KBUILD_CFLAGS += -fno-inline-functions-called-once
+@@ -872,7 +872,7 @@ KBUILD_CFLAGS += -fno-inline-functions-called-once
endif

# Prefer linking with the `-z unique-symbol` if available, this eliminates
@@ -54,7 +57,7 @@ index b921b1fabf70..3346269341d4 100644
ifeq ($(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL)$(CONFIG_LIVEPATCH),yy)
KBUILD_LDFLAGS += -z unique-symbol
endif
-@@ -892,7 +892,7 @@ endif
+@@ -881,7 +881,7 @@ endif
# `include/linux/linkage.h` for explanation. This flag is to enable GAS to
# insert the name of the previous section instead of `%S` inside .pushsection
ifdef CONFIG_HAVE_ASM_FUNCTION_SECTIONS
@@ -63,7 +66,7 @@ index b921b1fabf70..3346269341d4 100644
SECSUBST_AFLAGS := -Wa,--sectname-subst
KBUILD_AFLAGS_KERNEL += $(SECSUBST_AFLAGS)
KBUILD_CFLAGS_KERNEL += $(SECSUBST_AFLAGS)
-@@ -906,8 +906,19 @@ KBUILD_CFLAGS_MODULE += -Wa,--sectname-subst
+@@ -895,8 +895,19 @@ KBUILD_CFLAGS_MODULE += -Wa,--sectname-subst
endif
endif # CONFIG_HAVE_ASM_FUNCTION_SECTIONS

@@ -85,15 +88,15 @@ index b921b1fabf70..3346269341d4 100644
endif

diff --git a/arch/Kconfig b/arch/Kconfig
-index b31a836bc252..01c026d090d4 100644
+index 550f0599e211..e06aeeea39f4 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
-@@ -1316,7 +1316,11 @@ config ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS
+@@ -1326,7 +1326,11 @@ config ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS
bool
help
An arch should select this if it can be built and run with its
-- ASM functions placed into separate sections to improve DCE and LTO.
-+ ASM functions placed into separate sections to improve DCE, LTO
+- asm functions placed into separate sections to improve DCE and LTO.
++ asm functions placed into separate sections to improve DCE, LTO
+ and FG-KASLR.
+
+config ARCH_SUPPORTS_FG_KASLR
@@ -153,15 +156,17 @@ index e7b8a84e0e64..586465b2abb2 100644
.shstrtab 0 : { *(.shstrtab) }

diff --git a/include/linux/linkage.h b/include/linux/linkage.h
-index 0c0ddf4429dc..f3c96fb6a534 100644
+index f3b966a6427e..95ca162a868c 100644
--- a/include/linux/linkage.h
+++ b/include/linux/linkage.h
-@@ -75,10 +75,11 @@
+@@ -75,11 +75,12 @@

/*
* Allow ASM symbols to have their own unique sections if they are being
-- * generated by the compiler for C functions (DCE, LTO).
-+ * generated by the compiler for C functions (DCE, FG-KASLR, LTO).
+- * generated by the compiler for C functions (DCE, LTO). Correlates with
+- * the presence of the `-ffunction-section` in KBUILD_CFLAGS.
++ * generated by the compiler for C functions (DCE, FG-KASLR, LTO). Correlates
++ * with the presence of the `-ffunction-section` in KBUILD_CFLAGS.
*/
#if defined(CONFIG_HAVE_ASM_FUNCTION_SECTIONS) && \
((defined(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) && !defined(MODULE)) || \
@@ -169,22 +174,39 @@ index 0c0ddf4429dc..f3c96fb6a534 100644
(defined(CONFIG_LTO_CLANG)))

#define SYM_PUSH_SECTION(name) \
+@@ -91,13 +92,13 @@
+ #define __ASM_PUSH_SECTION(name) \
+ ".pushsection %S." name ", \"ax\""
+
+-#else /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || LTO)) */
++#else /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || FG_KASLR || LTO)) */
+
+ #define SYM_PUSH_SECTION(name)
+ #define SYM_POP_SECTION()
+ #define __ASM_PUSH_SECTION(name)
+
+-#endif /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || LTO)) */
++#endif /* !(CONFIG_HAVE_ASM_FUNCTION_SECTIONS && (DCE || FG_KASLR || LTO)) */
+
+ #define ASM_PUSH_SECTION(name) \
+ __ASM_PUSH_SECTION(__stringify(name))
diff --git a/init/Kconfig b/init/Kconfig
-index 3babc0aeac61..a74b3c3acb49 100644
+index 4acfc80f22df..26f9a6e52dbd 100644
--- a/init/Kconfig
+++ b/init/Kconfig
-@@ -1394,8 +1394,8 @@ config HAVE_ASM_FUNCTION_SECTIONS
- This enables ASM function sections if both architecture
- and toolchain supports that. It allows creating a separate
- .text section for each ASM function in order to improve
-- DCE and LTO (works the same way as -ffunction-sections for
-- C code).
-+ DCE, LTO and FG-KASLR (works the same way as -ffunction-sections
+@@ -1393,8 +1393,9 @@ config HAVE_ASM_FUNCTION_SECTIONS
+ help
+ This enables asm function sections if both architecture and
+ toolchain support it. It allows creating a separate section
+- for each function written in assembly in order to improve DCE
+- and LTO (works the same way as -ffunction-sections for C code).
++ for each function written in assembly in order to improve DCE,
++ LTO and FG-KASLR (works the same way as -ffunction-sections
+ for C code).

config HAVE_LD_DEAD_CODE_DATA_ELIMINATION
bool
-@@ -2065,6 +2065,20 @@ config PROFILING
+@@ -2061,6 +2062,20 @@ config PROFILING
config TRACEPOINTS
bool

@@ -206,5 +228,5 @@ index 3babc0aeac61..a74b3c3acb49 100644

source "arch/Kconfig"
--
-2.33.1
+2.34.1

diff --git a/0008-x86-tools-Add-relative-relocs-for-randomized-functio.patch b/0008-x86-tools-Add-relative-relocs-for-randomized-functio.patch
index 240a2efd1df8..197364848196 100644
--- a/0008-x86-tools-Add-relative-relocs-for-randomized-functio.patch
+++ b/0008-x86-tools-Add-relative-relocs-for-randomized-functio.patch
@@ -1,7 +1,7 @@
-From 052c4d03eca1ae7c3925d4b829c2d25ad998588f Mon Sep 17 00:00:00 2001
+From 786423924d3f28a659978af6f2eeb96d050a6793 Mon Sep 17 00:00:00 2001
From: Kristen Carlson Accardi <[email protected]>
Date: Fri, 27 Aug 2021 16:08:09 +0200
-Subject: [PATCH v9 08/15] x86/tools: Add relative relocs for randomized
+Subject: [PATCH v10 08/15] x86/tools: Add relative relocs for randomized
functions

When reordering functions, the relative offsets for relocs that
@@ -10,11 +10,14 @@ sections will need to be adjusted. Add code to detect whether a
reloc satisfies these cases, and if so, add them to the appropriate
reloc list.

+Alexander Lobakin:
+
+Don't split relocs' usage string across lines to ease grepping.
+
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Reviewed-by: Tony Luck <[email protected]>
Tested-by: Tony Luck <[email protected]>
Reviewed-by: Kees Cook <[email protected]>
-[ alobakin: don't split relocs' usage string across lines ]
Signed-off-by: Alexander Lobakin <[email protected]>
---
arch/x86/boot/compressed/Makefile | 7 ++++++-
@@ -24,10 +27,10 @@ Signed-off-by: Alexander Lobakin <[email protected]>
4 files changed, 45 insertions(+), 12 deletions(-)

diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
-index 4bf32db56010..2cf809f5c642 100644
+index e7ee94d5f55a..b1f5817c5737 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
-@@ -107,6 +107,11 @@ $(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE
+@@ -110,6 +110,11 @@ $(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE
$(call if_changed,ld)

OBJCOPYFLAGS_vmlinux.bin := -R .comment -S
@@ -39,7 +42,7 @@ index 4bf32db56010..2cf809f5c642 100644
$(obj)/vmlinux.bin: vmlinux FORCE
$(call if_changed,objcopy)

-@@ -114,7 +119,7 @@ targets += $(patsubst $(obj)/%,%,$(vmlinux-objs-y)) vmlinux.bin.all vmlinux.relo
+@@ -117,7 +122,7 @@ targets += $(patsubst $(obj)/%,%,$(vmlinux-objs-y)) vmlinux.bin.all vmlinux.relo

CMD_RELOCS = arch/x86/tools/relocs
quiet_cmd_relocs = RELOCS $@
@@ -49,7 +52,7 @@ index 4bf32db56010..2cf809f5c642 100644
$(call if_changed,relocs)

diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c
-index c736cf2ac76b..8aa1f39be561 100644
+index e2c5b296120d..96b6042fc76f 100644
--- a/arch/x86/tools/relocs.c
+++ b/arch/x86/tools/relocs.c
@@ -45,6 +45,8 @@ struct section {
@@ -186,5 +189,5 @@ index 6634352a20bc..d6acda36575a 100644
return 0;
}
--
-2.33.1
+2.34.1

diff --git a/0009-x86-Add-support-for-function-granular-KASLR.patch b/0009-x86-Add-support-for-function-granular-KASLR.patch
index 0ebc0dedf4fe..bc167354f4b0 100644
--- a/0009-x86-Add-support-for-function-granular-KASLR.patch
+++ b/0009-x86-Add-support-for-function-granular-KASLR.patch
@@ -1,11 +1,10 @@
-From 332cdd61cec4b2315b1fa440947f014e8cd77f3f Mon Sep 17 00:00:00 2001
+From 0f075ed9b0a747d78e0cb74df5c9f90ae1273061 Mon Sep 17 00:00:00 2001
From: Kristen Carlson Accardi <[email protected]>
Date: Fri, 27 Aug 2021 16:13:21 +0200
-Subject: [PATCH v9 09/15] x86: Add support for function granular KASLR
+Subject: [PATCH v10 09/15] x86: Add support for function granular KASLR

-This commit contains the changes required to re-layout the kernel text
-sections generated by -ffunction-sections shortly after decompression.
-Documentation of the feature is also added.
+Add the x86-specific bits to re-layout the kernel text sections
+generated by -ffunction-sections shortly after decompression.

After decompression, the decompressed image's elf headers are parsed.
In order to manually update certain data structures that are built with
@@ -37,7 +36,8 @@ with new symbol locations, and then re-sorted by the new address. The
orc table will have been updated as part of applying relocations, but since
it is expected to be sorted by address, it will need to be resorted.

-alobakin:
+Alexander Lobakin:
+
Handle .altinstr_replacement relocations. Add vmlinux symbols to be
able to determine if a code location belongs to them, and treat this
code as text.
@@ -45,11 +45,10 @@ Use the new "gen-symbols.h" header file to ad hoc create a list of
symbols needed for both objcopy (as plain text) and C source
(fgkaslr.c, twice). With this approach, it's easier to add new
sections if needed.
-Make use of our "arch/x86/lib/orc.c" introduced a bunch commits
-earlier to sort ORC entries at pre-boot time -- include it directly,
-similarly to extable.c et al.
-Finally, use the new shuffle_array() macro landed in the kallsyms
-commit to randomize our section list.
+Make use of the "arch/x86/lib/orc.c" to sort ORC entries at pre-boot
+time -- include it directly, similarly to extable.c et al.
+Finally, use the shuffle_array() macro instead of open-coding it
+to randomize the section list.

Signed-off-by: Kristen Carlson Accardi <[email protected]>
Reviewed-by: Tony Luck <[email protected]>
@@ -62,15 +61,15 @@ Signed-off-by: Alexander Lobakin <[email protected]>
---
arch/x86/boot/compressed/.gitignore | 1 +
arch/x86/boot/compressed/Makefile | 15 +-
- arch/x86/boot/compressed/fgkaslr.c | 755 +++++++++++++++++++++++++
+ arch/x86/boot/compressed/fgkaslr.c | 752 +++++++++++++++++++++++++
arch/x86/boot/compressed/gen-symbols.h | 30 +
- arch/x86/boot/compressed/misc.c | 153 ++++-
+ arch/x86/boot/compressed/misc.c | 144 ++++-
arch/x86/boot/compressed/misc.h | 28 +
- arch/x86/boot/compressed/utils.c | 13 +
+ arch/x86/boot/compressed/utils.c | 16 +
arch/x86/include/asm/boot.h | 13 +-
arch/x86/kernel/vmlinux.lds.S | 2 +
include/uapi/linux/elf.h | 1 +
- 10 files changed, 982 insertions(+), 29 deletions(-)
+ 10 files changed, 975 insertions(+), 27 deletions(-)
create mode 100644 arch/x86/boot/compressed/fgkaslr.c
create mode 100644 arch/x86/boot/compressed/gen-symbols.h
create mode 100644 arch/x86/boot/compressed/utils.c
@@ -87,10 +86,10 @@ index 25805199a506..bc5f8436be1d 100644
mkpiggy
piggy.S
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
-index 2cf809f5c642..6f8dcbbbad80 100644
+index b1f5817c5737..caa7d120f0eb 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
-@@ -90,6 +90,7 @@ vmlinux-objs-y := $(obj)/vmlinux.lds $(obj)/kernel_info.o $(obj)/head_$(BITS).o
+@@ -93,6 +93,7 @@ vmlinux-objs-y := $(obj)/vmlinux.lds $(obj)/kernel_info.o $(obj)/head_$(BITS).o

vmlinux-objs-$(CONFIG_EARLY_PRINTK) += $(obj)/early_serial_console.o
vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/kaslr.o
@@ -98,7 +97,7 @@ index 2cf809f5c642..6f8dcbbbad80 100644
ifdef CONFIG_X86_64
vmlinux-objs-y += $(obj)/ident_map_64.o
vmlinux-objs-y += $(obj)/idt_64.o $(obj)/idt_handlers_64.o
-@@ -108,11 +109,21 @@ $(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE
+@@ -111,11 +112,21 @@ $(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE

OBJCOPYFLAGS_vmlinux.bin := -R .comment -S

@@ -124,16 +123,16 @@ index 2cf809f5c642..6f8dcbbbad80 100644
targets += $(patsubst $(obj)/%,%,$(vmlinux-objs-y)) vmlinux.bin.all vmlinux.relocs
diff --git a/arch/x86/boot/compressed/fgkaslr.c b/arch/x86/boot/compressed/fgkaslr.c
new file mode 100644
-index 000000000000..91b3b6ea3a93
+index 000000000000..85eb1ef574a2
--- /dev/null
+++ b/arch/x86/boot/compressed/fgkaslr.c
-@@ -0,0 +1,755 @@
+@@ -0,0 +1,752 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * This contains the routines needed to reorder the kernel text section
+ * at boot time.
+ *
-+ * Copyright (C) 2020-2021, Intel Corporation.
++ * Copyright (C) 2020-2022, Intel Corporation.
+ * Author: Kristen Carlson Accardi <[email protected]>
+ */
+
@@ -169,14 +168,14 @@ index 000000000000..91b3b6ea3a93
+#undef GEN
+
+/* addresses in mapped address space */
-+static int *base;
++static int *kallsyms_base;
+static u8 *names;
+static unsigned long relative_base;
+static unsigned int *markers_addr;
+
+struct kallsyms_name {
+ u8 len;
-+ u8 indecis[256];
++ u8 indices[256];
+};
+
+static struct kallsyms_name *names_table;
@@ -324,9 +323,9 @@ index 000000000000..91b3b6ea3a93
+ int idx1, idx2;
+
+ /* Determine our index into the array. */
-+ idx1 = (const int *)a - base;
-+ idx2 = (const int *)b - base;
-+ swap(base[idx1], base[idx2]);
++ idx1 = (const int *)a - kallsyms_base;
++ idx2 = (const int *)b - kallsyms_base;
++ swap(kallsyms_base[idx1], kallsyms_base[idx2]);
+
+ /* Swap the names table. */
+ memcpy(&name_a, &names_table[idx1], sizeof(name_a));
@@ -380,7 +379,7 @@ index 000000000000..91b3b6ea3a93
+ names_table[i].len = names[offset];
+ offset++;
+ for (j = 0; j < names_table[i].len; j++) {
-+ names_table[i].indecis[j] = names[offset];
++ names_table[i].indices[j] = names[offset];
+ offset++;
+ }
+ }
@@ -412,7 +411,7 @@ index 000000000000..91b3b6ea3a93
+ names[offset] = (u8)names_table[i].len;
+ offset++;
+ for (j = 0; j < names_table[i].len; j++) {
-+ names[offset] = (u8)names_table[i].indecis[j];
++ names[offset] = names_table[i].indices[j];
+ offset++;
+ }
+ }
@@ -433,7 +432,7 @@ index 000000000000..91b3b6ea3a93
+ debug_putstr("\nRe-sorting kallsyms...\n");
+
+ num_syms = *(int *)(addr_kallsyms_num_syms + map);
-+ base = (int *)(addr_kallsyms_offsets + map);
++ kallsyms_base = (int *)(addr_kallsyms_offsets + map);
+ relative_base = *(unsigned long *)(addr_kallsyms_relative_base + map);
+ markers_addr = (unsigned int *)(addr_kallsyms_markers + map);
+ names = (u8 *)(addr_kallsyms_names + map);
@@ -452,14 +451,14 @@ index 000000000000..91b3b6ea3a93
+ * according to kernel/kallsyms.c, positive offsets are absolute
+ * values and negative offsets are relative to the base.
+ */
-+ if (base[i] >= 0)
-+ addr = base[i];
++ if (kallsyms_base[i] >= 0)
++ addr = kallsyms_base[i];
+ else
-+ addr = relative_base - 1 - base[i];
++ addr = relative_base - 1 - kallsyms_base[i];
+
+ if (adjust_address(&addr))
+ /* here we need to recalcuate the offset */
-+ base[i] = relative_base - 1 - addr;
++ kallsyms_base[i] = relative_base - 1 - addr;
+ }
+
+ /*
@@ -468,7 +467,7 @@ index 000000000000..91b3b6ea3a93
+ */
+ deal_with_names(num_syms);
+
-+ sort(base, num_syms, sizeof(int), kallsyms_cmp, kallsyms_swp);
++ sort(kallsyms_base, num_syms, sizeof(int), kallsyms_cmp, kallsyms_swp);
+
+ /* write the newly sorted names table over the old one */
+ write_sorted_names(num_syms);
@@ -528,9 +527,6 @@ index 000000000000..91b3b6ea3a93
+ sort_extable(start_ex_table, stop_ex_table);
+}
+
-+#define ORC_COMPRESSED_BOOT
-+#include "../../lib/orc.c"
-+
+static void update_orc_table(unsigned long map)
+{
+ int *ip_table = (int *)(addr___start_orc_unwind_ip + map);
@@ -885,7 +881,7 @@ index 000000000000..91b3b6ea3a93
+}
diff --git a/arch/x86/boot/compressed/gen-symbols.h b/arch/x86/boot/compressed/gen-symbols.h
new file mode 100644
-index 000000000000..6828c80dd19f
+index 000000000000..15b7ddec2762
--- /dev/null
+++ b/arch/x86/boot/compressed/gen-symbols.h
@@ -0,0 +1,30 @@
@@ -894,7 +890,7 @@ index 000000000000..6828c80dd19f
+ * List of symbols needed for both C code and objcopy when FG-KASLR is on.
+ * We declare them once and then just use GEN() definition.
+ *
-+ * Copyright (C) 2021, Intel Corporation.
++ * Copyright (C) 2021-2022, Intel Corporation.
+ * Author: Alexander Lobakin <[email protected]>
+ */
+
@@ -920,27 +916,23 @@ index 000000000000..6828c80dd19f
+GEN(kallsyms_token_table)
+#endif /* GEN */
diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c
-index a4339cb2d247..bd733e1abc75 100644
+index a4339cb2d247..838c1b2b721a 100644
--- a/arch/x86/boot/compressed/misc.c
+++ b/arch/x86/boot/compressed/misc.c
-@@ -207,10 +207,21 @@ static void handle_relocations(void *output, unsigned long output_len,
+@@ -207,10 +207,19 @@ static void handle_relocations(void *output, unsigned long output_len,
if (IS_ENABLED(CONFIG_X86_64))
delta = virt_addr - LOAD_PHYSICAL_ADDR;

- if (!delta) {
-- debug_putstr("No relocation needed... ");
-- return;
+ /*
+ * it is possible to have delta be zero and still have enabled
-+ * fg kaslr. We need to perform relocations for fgkaslr regardless
++ * FG-KASLR. We need to perform relocations for it regardless
+ * of whether the base address has moved.
+ */
-+ if (!IS_ENABLED(CONFIG_FG_KASLR) ||
-+ cmdline_find_option_bool("nokaslr")) {
-+ if (!delta) {
-+ debug_putstr("No relocation needed... ");
-+ return;
-+ }
++ if ((cmdline_find_option_bool("nokaslr") ||
++ !IS_ENABLED(CONFIG_FG_KASLR)) && !delta) {
+ debug_putstr("No relocation needed... ");
+ return;
}
+
+ pre_relocations_cleanup(map);
@@ -948,7 +940,7 @@ index a4339cb2d247..bd733e1abc75 100644
debug_putstr("Performing relocations... ");

/*
-@@ -234,35 +245,105 @@ static void handle_relocations(void *output, unsigned long output_len,
+@@ -234,35 +243,103 @@ static void handle_relocations(void *output, unsigned long output_len,
*/
for (reloc = output + output_len - sizeof(*reloc); *reloc; reloc--) {
long extended = *reloc;
@@ -968,18 +960,17 @@ index a4339cb2d247..bd733e1abc75 100644
error("32-bit relocation outside of kernel!\n");

- *(uint32_t *)ptr += delta;
-+ value = *(int32_t *)ptr;
++ value = *(u32 *)ptr;
+
+ /*
-+ * If using fgkaslr, the value of the relocation
++ * If using FG-KASLR, the value of the relocation
+ * might need to be changed because it referred
+ * to an address that has moved.
+ */
+ adjust_address(&value);
+
+ value += delta;
-+
-+ *(uint32_t *)ptr = value;
++ *(u32 *)ptr = value;
}
#ifdef CONFIG_X86_64
while (*--reloc) {
@@ -988,8 +979,8 @@ index a4339cb2d247..bd733e1abc75 100644
+ Elf64_Shdr *s;
+
+ /*
-+ * if using fgkaslr, we might have moved the address
-+ * of the relocation. Check it to see if it needs adjusting
++ * if using FG-KASLR, we might have moved the address of
++ * the relocation. Check it to see if it needs adjusting
+ * from the original address.
+ */
+ s = adjust_address(&extended);
@@ -1001,7 +992,7 @@ index a4339cb2d247..bd733e1abc75 100644
error("inverse 32-bit relocation outside of kernel!\n");

- *(int32_t *)ptr -= delta;
-+ value = *(int32_t *)ptr;
++ value = *(s32 *)ptr;
+ oldvalue = value;
+
+ /*
@@ -1014,22 +1005,22 @@ index a4339cb2d247..bd733e1abc75 100644
+
+ /*
+ * only percpu symbols need to have their values adjusted for
-+ * base address kaslr since relative offsets within the .text
++ * base address KASLR since relative offsets within the .text
+ * and .text.* sections are ok wrt each other.
+ */
+ if (is_percpu_addr(*reloc, oldvalue))
+ value -= delta;
+
-+ *(int32_t *)ptr = value;
++ *(s32 *)ptr = value;
}
for (reloc--; *reloc; reloc--) {
long extended = *reloc;
+ long value;
+
+ /*
-+ * if using fgkaslr, we might have moved the address
-+ * of the relocation. Check it to see if it needs adjusting
-+ * from the original address.
++ * if using FG-KASLR, we might have moved the address of the
++ * relocation. Check it to see if it needs adjusting from the
++ * original address.
+ */
+ adjust_address(&extended);
+
@@ -1040,7 +1031,7 @@ index a4339cb2d247..bd733e1abc75 100644
error("64-bit relocation outside of kernel!\n");

- *(uint64_t *)ptr += delta;
-+ value = *(int64_t *)ptr;
++ value = *(u64 *)ptr;
+
+ /*
+ * If using fgkaslr, the value of the relocation
@@ -1050,25 +1041,23 @@ index a4339cb2d247..bd733e1abc75 100644
+ adjust_address(&value);
+
+ value += delta;
-+
-+ *(uint64_t *)ptr = value;
++ *(u64 *)ptr = value;
}
+ post_relocations_cleanup(map);
#endif
}
#else
-@@ -271,6 +352,35 @@ static inline void handle_relocations(void *output, unsigned long output_len,
+@@ -271,6 +348,34 @@ static inline void handle_relocations(void *output, unsigned long output_len,
{ }
#endif

+static void layout_image(void *output, Elf_Ehdr *ehdr, Elf_Phdr *phdrs)
+{
-+ int i;
-+ void *dest;
-+ Elf_Phdr *phdr;
++ u32 i;
+
+ for (i = 0; i < ehdr->e_phnum; i++) {
-+ phdr = &phdrs[i];
++ const Elf_Phdr *phdr = &phdrs[i];
++ void *dest;
+
+ switch (phdr->p_type) {
+ case PT_LOAD:
@@ -1093,15 +1082,15 @@ index a4339cb2d247..bd733e1abc75 100644
static void parse_elf(void *output)
{
#ifdef CONFIG_X86_64
-@@ -282,6 +392,7 @@ static void parse_elf(void *output)
+@@ -280,6 +385,7 @@ static void parse_elf(void *output)
+ Elf32_Ehdr ehdr;
+ Elf32_Phdr *phdrs, *phdr;
#endif
++ int nokaslr;
void *dest;
int i;
-+ int nokaslr;

- memcpy(&ehdr, output, sizeof(ehdr));
- if (ehdr.e_ident[EI_MAG0] != ELFMAG0 ||
-@@ -292,6 +403,12 @@ static void parse_elf(void *output)
+@@ -292,6 +398,12 @@ static void parse_elf(void *output)
return;
}

@@ -1114,7 +1103,7 @@ index a4339cb2d247..bd733e1abc75 100644
debug_putstr("Parsing ELF... ");

phdrs = malloc(sizeof(*phdrs) * ehdr.e_phnum);
-@@ -300,26 +417,10 @@ static void parse_elf(void *output)
+@@ -300,26 +412,10 @@ static void parse_elf(void *output)

memcpy(phdrs, output + ehdr.e_phoff, sizeof(*phdrs) * ehdr.e_phnum);

@@ -1186,15 +1175,15 @@ index 16ed360b6692..1315a101c1c9 100644
void choose_random_location(unsigned long input,
diff --git a/arch/x86/boot/compressed/utils.c b/arch/x86/boot/compressed/utils.c
new file mode 100644
-index 000000000000..7c3c745f6251
+index 000000000000..0fbc2c18d0b9
--- /dev/null
+++ b/arch/x86/boot/compressed/utils.c
-@@ -0,0 +1,13 @@
+@@ -0,0 +1,16 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * This contains various libraries that are needed for FG-KASLR.
+ *
-+ * Copyright (C) 2020-2021, Intel Corporation.
++ * Copyright (C) 2020-2022, Intel Corporation.
+ * Author: Kristen Carlson Accardi <[email protected]>
+ */
+
@@ -1203,8 +1192,11 @@ index 000000000000..7c3c745f6251
+
+#include "../../../../lib/sort.c"
+#include "../../../../lib/bsearch.c"
++
++#define ORC_COMPRESSED_BOOT
++#include "../../lib/orc.c"
diff --git a/arch/x86/include/asm/boot.h b/arch/x86/include/asm/boot.h
-index 9191280d9ea3..ce5fdee49046 100644
+index 9191280d9ea3..a37be0dd3179 100644
--- a/arch/x86/include/asm/boot.h
+++ b/arch/x86/include/asm/boot.h
@@ -24,7 +24,18 @@
@@ -1214,7 +1206,7 @@ index 9191280d9ea3..ce5fdee49046 100644
-#if defined(CONFIG_KERNEL_BZIP2)
+#ifdef CONFIG_FG_KASLR
+/*
-+ * We need extra boot heap when using fgkaslr because we make a copy
++ * We need extra boot heap when using FG-KASLR because we make a copy
+ * of the original decompressed kernel to avoid issues with writing
+ * over ourselves when shuffling the sections. We also need extra
+ * space for resorting kallsyms after shuffling. This value could
@@ -1228,10 +1220,10 @@ index 9191280d9ea3..ce5fdee49046 100644
#elif defined(CONFIG_KERNEL_ZSTD)
/*
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
-index 6f026400261e..6620f069b7ef 100644
+index 5550bd68f6e7..54f16801e9d6 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
-@@ -304,7 +304,9 @@ SECTIONS
+@@ -303,7 +303,9 @@ SECTIONS
* get the address and the length of them to patch the kernel safely.
*/
.altinstr_replacement : AT(ADDR(.altinstr_replacement) - LOAD_OFFSET) {
@@ -1254,5 +1246,5 @@ index 61bf4774b8f2..1c74d9594919 100644

typedef struct elf32_shdr {
--
-2.33.1
+2.34.1

diff --git a/0010-FG-KASLR-use-a-scripted-approach-to-handle-.text.-se.patch b/0010-FG-KASLR-use-a-scripted-approach-to-handle-.text.-se.patch
index d6daea5c55da..d8a0674d2d15 100644
--- a/0010-FG-KASLR-use-a-scripted-approach-to-handle-.text.-se.patch
+++ b/0010-FG-KASLR-use-a-scripted-approach-to-handle-.text.-se.patch
@@ -1,7 +1,7 @@
-From 9bed576c3e177e0d5d9b7c4d79469110845101ea Mon Sep 17 00:00:00 2001
+From 148870a73d8c955a514cf2362a04fa531bf82ff3 Mon Sep 17 00:00:00 2001
From: Alexander Lobakin <[email protected]>
Date: Fri, 27 Aug 2021 16:29:40 +0200
-Subject: [PATCH v9 10/15] FG-KASLR: use a scripted approach to handle .text.*
+Subject: [PATCH v10 10/15] FG-KASLR: use a scripted approach to handle .text.*
sections

Instead of relying on the linker and his heuristics about where to
@@ -19,12 +19,12 @@ total number of sections and the boottime delay (which is still
barely noticeable). The values of 4-8 are still strong enough and
allows to save some space, and so on.

-We also keep tracking the maximal alignment we found while
+We also keep tracking the maximum alignment we found while
traversing through the readelf output and the number of times we
-spotted it. It's actual only for values >= 64 and is required to
+spotted it. It's actual only for values >= 128 and is required to
reserve some space between the last .text.* section and the _etext
marker.
-The reason is that e.g. x86 has at least 3 ASM sections (4 with
+The reason is that e.g. x86 has at least 3 asm sections (4 with
ClangCFI) aligned to 4096, and when mixing them with the small
sections, we could go past the _etext and render the kernel
unbootable. This reserved space ensures this won't happen.
@@ -39,15 +39,15 @@ Signed-off-by: Alexander Lobakin <[email protected]>
include/asm-generic/vmlinux.lds.h | 6 ++
init/Kconfig | 14 +++
scripts/generate_text_sections.pl | 165 ++++++++++++++++++++++++++++++
- scripts/link-vmlinux.sh | 29 +++++-
- 5 files changed, 216 insertions(+), 2 deletions(-)
+ scripts/link-vmlinux.sh | 30 +++++-
+ 5 files changed, 217 insertions(+), 2 deletions(-)
create mode 100755 scripts/generate_text_sections.pl

diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
-index 6620f069b7ef..6f039bf9de34 100644
+index 54f16801e9d6..06ba33f5bc58 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
-@@ -147,9 +147,11 @@ SECTIONS
+@@ -146,9 +146,11 @@ SECTIONS
#endif
} :text =0xcccc

@@ -78,10 +78,10 @@ index 586465b2abb2..e63d5a69f1bc 100644
* GCC 4.5 and later have a 32 bytes section alignment for structures.
* Except GCC 4.9, that feels the need to align on 64 bytes.
diff --git a/init/Kconfig b/init/Kconfig
-index a74b3c3acb49..381b063b4925 100644
+index 26f9a6e52dbd..5fbd1c294df4 100644
--- a/init/Kconfig
+++ b/init/Kconfig
-@@ -2079,6 +2079,20 @@ config FG_KASLR
+@@ -2076,6 +2076,20 @@ config FG_KASLR

If unsure, say N.

@@ -104,7 +104,7 @@ index a74b3c3acb49..381b063b4925 100644
source "arch/Kconfig"
diff --git a/scripts/generate_text_sections.pl b/scripts/generate_text_sections.pl
new file mode 100755
-index 000000000000..c95b9be28920
+index 000000000000..999e1b68181f
--- /dev/null
+++ b/scripts/generate_text_sections.pl
@@ -0,0 +1,165 @@
@@ -115,7 +115,7 @@ index 000000000000..c95b9be28920
+# to avoid orphan/heuristic section placement and double-checks we don't have
+# any symbols in plain .text section.
+#
-+# Copyright (C) 2021, Intel Corporation.
++# Copyright (C) 2021-2022, Intel Corporation.
+# Author: Alexander Lobakin <[email protected]>
+#
+
@@ -161,7 +161,7 @@ index 000000000000..c95b9be28920
+
+## max alignment found to reserve some space. It would probably be
+## better to start from 64, but CONFIG_DEBUG_FORCE_FUNCTION_ALIGN_64B
-+## (which aligns every function to 64b) would kill us then
++## (which aligns every function to 64b) would explode the $count then
+my $max_align = 128;
+my $count = 0;
+
@@ -219,7 +219,7 @@ index 000000000000..c95b9be28920
+ print "\t\t*(.text)\n";
+ print "\t}\n";
+
-+ ## If we have ASM function sections, we shouldn't have anything
++ ## If we have asm function sections, we shouldn't have anything
+ ## in here.
+ if ($add_assert) {
+ print "\tASSERT(SIZEOF(.text.0) == 0, \"Plain .text is not empty!\")\n\n";
@@ -248,7 +248,7 @@ index 000000000000..c95b9be28920
+}
+
+sub print_reserve {
-+ ## If we have text sections aligned with 64 bytes or more, make
++ ## If we have text sections aligned with 128 bytes or more, make
+ ## sure we reserve some space for them to not overlap _etext
+ ## while shuffling sections.
+ if (!$count) {
@@ -274,10 +274,10 @@ index 000000000000..c95b9be28920
+read_sections();
+print_lds();
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
-index 5cdd9bc5c385..9d8894cb1c21 100755
+index 666f7bbc13eb..701cf540c12e 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
-@@ -66,6 +66,22 @@ gen_symversions()
+@@ -70,6 +70,23 @@ gen_symversions()
done
}

@@ -285,14 +285,15 @@ index 5cdd9bc5c385..9d8894cb1c21 100755
+# declare all custom text sections for future boottime shuffling
+gen_text_sections()
+{
-+ local a=""
++ local shift=$(sed -n 's/^CONFIG_FG_KASLR_SHIFT=\(.*\)$/\1/p' include/config/auto.conf)
++ local assert=""
+
-+ [ -n "${CONFIG_HAVE_ASM_FUNCTION_SECTIONS}" ] && a="-a"
++ is_enabled CONFIG_HAVE_ASM_FUNCTION_SECTIONS && assert="-a"
+
+ info GEN .tmp_vmlinux.lds
+
+ ${PERL} ${srctree}/scripts/generate_text_sections.pl \
-+ ${a} -s "${CONFIG_FG_KASLR_SHIFT}" vmlinux.o \
++ ${assert} -s "${shift}" vmlinux.o \
+ < "${objtree}/${KBUILD_LDS}" \
+ > .tmp_vmlinux.lds
+}
@@ -300,7 +301,7 @@ index 5cdd9bc5c385..9d8894cb1c21 100755
# Link of vmlinux.o used for section mismatch analysis
# ${1} output file
modpost_link()
-@@ -155,12 +171,19 @@ vmlinux_link()
+@@ -162,12 +179,19 @@ vmlinux_link()
local ld
local ldflags
local ldlibs
@@ -311,16 +312,16 @@ index 5cdd9bc5c385..9d8894cb1c21 100755
# skip output file argument
shift

-+ if [ -n "${CONFIG_FG_KASLR}" ]; then
++ if is_enabled CONFIG_FG_KASLR; then
+ lds=".tmp_vmlinux.lds"
+ else
+ lds="${objtree}/${KBUILD_LDS}"
+ fi
+
- if [ -n "${CONFIG_LTO_CLANG}" ]; then
+ if is_enabled CONFIG_LTO_CLANG; then
# Use vmlinux.o instead of performing the slow LTO link again.
objs=vmlinux.o
-@@ -182,7 +205,7 @@ vmlinux_link()
+@@ -189,7 +213,7 @@ vmlinux_link()
ldlibs=
fi

@@ -329,17 +330,17 @@ index 5cdd9bc5c385..9d8894cb1c21 100755

# The kallsyms linking does not need debug symbols included.
if [ "$output" != "${output#.tmp_vmlinux.kallsyms}" ] ; then
-@@ -342,6 +365,10 @@ info GEN modules.builtin
+@@ -346,6 +370,10 @@ info GEN modules.builtin
tr '\0' '\n' < modules.builtin.modinfo | sed -n 's/^[[:alnum:]:_]*\.file=//p' |
tr ' ' '\n' | uniq | sed -e 's:^:kernel/:' -e 's/$/.ko/' > modules.builtin

-+if [ -n "${CONFIG_FG_KASLR}" ]; then
++if is_enabled CONFIG_FG_KASLR; then
+ gen_text_sections
+fi
+
btf_vmlinux_bin_o=""
- if [ -n "${CONFIG_DEBUG_INFO_BTF}" ]; then
+ if is_enabled CONFIG_DEBUG_INFO_BTF; then
btf_vmlinux_bin_o=.btf.vmlinux.bin.o
--
-2.33.1
+2.34.1

diff --git a/0011-x86-boot-allow-FG-KASLR-to-be-selected.patch b/0011-x86-boot-allow-FG-KASLR-to-be-selected.patch
index c38f1de36de4..3675cc78d97f 100644
--- a/0011-x86-boot-allow-FG-KASLR-to-be-selected.patch
+++ b/0011-x86-boot-allow-FG-KASLR-to-be-selected.patch
@@ -1,11 +1,11 @@
-From f6e70105a68e1d772107bc9edc39a6c43e4c0b1d Mon Sep 17 00:00:00 2001
+From 007d007823bf3395e6a086f6f37383b80302a2e5 Mon Sep 17 00:00:00 2001
From: Alexander Lobakin <[email protected]>
Date: Fri, 27 Aug 2021 16:43:53 +0200
-Subject: [PATCH v9 11/15] x86/boot: allow FG-KASLR to be selected
+Subject: [PATCH v10 11/15] x86/boot: allow FG-KASLR to be selected

-Now that we have full support of FG-KASLR from both kernel core
-and x86 code, allow FG-KASLR to be enabled for x86_64 if the
-"regular" KASLR is also turned on.
+Now that the kernel has full support of FG-KASLR from both core and
+x86 code, allow FG-KASLR to be enabled for x86_64 if the "regular"
+KASLR is also turned on.

Signed-off-by: Alexander Lobakin <[email protected]>
---
@@ -13,7 +13,7 @@ Signed-off-by: Alexander Lobakin <[email protected]>
1 file changed, 1 insertion(+)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
-index 3e4ea355147b..f7472528f3dd 100644
+index f6bb48d41349..d5442107a915 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -105,6 +105,7 @@ config X86
@@ -21,9 +21,9 @@ index 3e4ea355147b..f7472528f3dd 100644
select ARCH_SUPPORTS_ATOMIC_RMW
select ARCH_SUPPORTS_DEBUG_PAGEALLOC
+ select ARCH_SUPPORTS_FG_KASLR if X86_64 && RANDOMIZE_BASE
+ select ARCH_SUPPORTS_PAGE_TABLE_CHECK if X86_64
select ARCH_SUPPORTS_NUMA_BALANCING if X86_64
select ARCH_SUPPORTS_KMAP_LOCAL_FORCE_MAP if NR_CPUS <= 4096
- select ARCH_SUPPORTS_LTO_CLANG
--
-2.33.1
+2.34.1

diff --git a/0012-module-Reorder-functions.patch b/0012-module-add-arch-indep-FG-KASLR-for-randomizing-funct.patch
similarity index 87%
rename from 0012-module-Reorder-functions.patch
rename to 0012-module-add-arch-indep-FG-KASLR-for-randomizing-funct.patch
index 707f4c8b305e..2461865b59f8 100644
--- a/0012-module-Reorder-functions.patch
+++ b/0012-module-add-arch-indep-FG-KASLR-for-randomizing-funct.patch
@@ -1,7 +1,8 @@
-From f5644bd95ba176c0851073958047eb715a7d30cd Mon Sep 17 00:00:00 2001
+From 85ec1552ebe35533ca5cde2dd1cae081a924e7f5 Mon Sep 17 00:00:00 2001
From: Kristen Carlson Accardi <[email protected]>
Date: Fri, 27 Aug 2021 16:51:49 +0200
-Subject: [PATCH v9 12/15] module: Reorder functions
+Subject: [PATCH v10 12/15] module: add arch-indep FG-KASLR for randomizing
+ function layout

Introduce a new config option to allow modules to be re-ordered
by function. This option can be enabled independently of the
@@ -14,13 +15,15 @@ If a module has functions split out into separate text sections
(i.e. compiled with the -ffunction-sections flag), reorder the
functions to provide some code diversification to modules.

-alobakin:
+Alexander Lobakin:
+
Make it work with ClangCFI -- in such builds, .text section must
always come first and be page-aligned. Exclude it from the shuffle
list and leave as it is.
Make this feature depend on `-z unique-symbol` as well, due to the
-very same reasons.
-Traditionally, use common shuffle_array() from <linux/random.h>.
+very same reasons as for FG-KASLR for vmlinux.
+Use common shuffle_array() from <linux/random.h> instead of
+open-coding it.

Signed-off-by: Kristen Carlson Accardi <[email protected]>
Reviewed-by: Kees Cook <[email protected]>
@@ -41,10 +44,10 @@ Signed-off-by: Alexander Lobakin <[email protected]>
4 files changed, 91 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
-index 3346269341d4..74d270c77d96 100644
+index 4328d53d8b25..cf7cf5cbdad9 100644
--- a/Makefile
+++ b/Makefile
-@@ -900,7 +900,7 @@ export SECSUBST_AFLAGS
+@@ -889,7 +889,7 @@ export SECSUBST_AFLAGS
endif

# Same for modules. LD DCE doesn't work for them, thus not checking for it
@@ -53,7 +56,7 @@ index 3346269341d4..74d270c77d96 100644
KBUILD_AFLAGS_MODULE += -Wa,--sectname-subst
KBUILD_CFLAGS_MODULE += -Wa,--sectname-subst
endif
-@@ -909,6 +909,10 @@ endif # CONFIG_HAVE_ASM_FUNCTION_SECTIONS
+@@ -898,6 +898,10 @@ endif # CONFIG_HAVE_ASM_FUNCTION_SECTIONS
# ClangLTO implies `-ffunction-sections -fdata-sections`, no need
# to specify them manually and trigger a pointless full rebuild
ifndef CONFIG_LTO_CLANG
@@ -65,10 +68,10 @@ index 3346269341d4..74d270c77d96 100644
KBUILD_CFLAGS_KERNEL += -ffunction-sections
endif
diff --git a/include/linux/linkage.h b/include/linux/linkage.h
-index f3c96fb6a534..deb26069278a 100644
+index 95ca162a868c..12cf21f9d8ad 100644
--- a/include/linux/linkage.h
+++ b/include/linux/linkage.h
-@@ -80,6 +80,7 @@
+@@ -81,6 +81,7 @@
#if defined(CONFIG_HAVE_ASM_FUNCTION_SECTIONS) && \
((defined(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) && !defined(MODULE)) || \
(defined(CONFIG_FG_KASLR) && !defined(MODULE)) || \
@@ -77,10 +80,10 @@ index f3c96fb6a534..deb26069278a 100644

#define SYM_PUSH_SECTION(name) \
diff --git a/init/Kconfig b/init/Kconfig
-index 381b063b4925..38c82e21efd7 100644
+index 5fbd1c294df4..86a2d3fd6390 100644
--- a/init/Kconfig
+++ b/init/Kconfig
-@@ -2376,6 +2376,20 @@ config UNUSED_KSYMS_WHITELIST
+@@ -2386,6 +2386,20 @@ config UNUSED_KSYMS_WHITELIST
one per line. The path can be absolute, or relative to the kernel
source tree.

@@ -102,7 +105,7 @@ index 381b063b4925..38c82e21efd7 100644

config MODULES_TREE_LOOKUP
diff --git a/kernel/module.c b/kernel/module.c
-index 84a9141a5e15..802e1098eaf4 100644
+index 46a5c2ed1928..616a622953fa 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -57,6 +57,7 @@
@@ -113,7 +116,7 @@ index 84a9141a5e15..802e1098eaf4 100644
#include <uapi/linux/module.h>
#include "module-internal.h"

-@@ -1527,7 +1528,7 @@ static void free_sect_attrs(struct module_sect_attrs *sect_attrs)
+@@ -1526,7 +1527,7 @@ static void free_sect_attrs(struct module_sect_attrs *sect_attrs)

for (section = 0; section < sect_attrs->nsections; section++)
kfree(sect_attrs->attrs[section].battr.attr.name);
@@ -122,7 +125,7 @@ index 84a9141a5e15..802e1098eaf4 100644
}

static void add_sect_attrs(struct module *mod, const struct load_info *info)
-@@ -1544,7 +1545,7 @@ static void add_sect_attrs(struct module *mod, const struct load_info *info)
+@@ -1543,7 +1544,7 @@ static void add_sect_attrs(struct module *mod, const struct load_info *info)
size[0] = ALIGN(struct_size(sect_attrs, attrs, nloaded),
sizeof(sect_attrs->grp.bin_attrs[0]));
size[1] = (nloaded + 1) * sizeof(sect_attrs->grp.bin_attrs[0]);
@@ -131,7 +134,7 @@ index 84a9141a5e15..802e1098eaf4 100644
if (sect_attrs == NULL)
return;

-@@ -2416,6 +2417,71 @@ static bool module_init_layout_section(const char *sname)
+@@ -2415,6 +2416,71 @@ static bool module_init_layout_section(const char *sname)
return module_init_section(sname);
}

@@ -203,7 +206,7 @@ index 84a9141a5e15..802e1098eaf4 100644
/*
* Lay out the SHF_ALLOC sections in a way not dissimilar to how ld
* might -- code, read-only data, read-write data, small data. Tally
-@@ -2510,6 +2576,9 @@ static void layout_sections(struct module *mod, struct load_info *info)
+@@ -2509,6 +2575,9 @@ static void layout_sections(struct module *mod, struct load_info *info)
break;
}
}
@@ -214,5 +217,5 @@ index 84a9141a5e15..802e1098eaf4 100644

static void set_license(struct module *mod, const char *license)
--
-2.33.1
+2.34.1

diff --git a/0013-module-use-a-scripted-approach-for-FG-KASLR.patch b/0013-module-use-a-scripted-approach-for-FG-KASLR.patch
index 4d372dd63639..0a45b214e453 100644
--- a/0013-module-use-a-scripted-approach-for-FG-KASLR.patch
+++ b/0013-module-use-a-scripted-approach-for-FG-KASLR.patch
@@ -1,7 +1,7 @@
-From d9657682acf698c3ff18c17435ad16f19b5743ea Mon Sep 17 00:00:00 2001
+From 5a60699c43cc6a9e7f7014bf7c6ae918d1fb1db1 Mon Sep 17 00:00:00 2001
From: Alexander Lobakin <[email protected]>
Date: Thu, 23 Dec 2021 00:09:25 +0100
-Subject: [PATCH v9 13/15] module: use a scripted approach for FG-KASLR
+Subject: [PATCH v10 13/15] module: use a scripted approach for FG-KASLR

Use the same methods and scripts to generate an LD script for every
module containing all the output text sections.
@@ -60,10 +60,10 @@ index e63d5a69f1bc..9f67660ace18 100644
* Used by scripts/generate_text_sections.pl to inject text sections,
* harmless if FG-KASLR is disabled.
diff --git a/init/Kconfig b/init/Kconfig
-index 38c82e21efd7..d4000885b27d 100644
+index 86a2d3fd6390..90951631aa03 100644
--- a/init/Kconfig
+++ b/init/Kconfig
-@@ -2381,7 +2381,6 @@ config MODULE_FG_KASLR
+@@ -2391,7 +2391,6 @@ config MODULE_FG_KASLR
depends on $(cc-option,-ffunction-sections)
depends on LD_HAS_Z_UNIQUE_SYMBOL || !LIVEPATCH
default FG_KASLR
@@ -71,7 +71,7 @@ index 38c82e21efd7..d4000885b27d 100644
help
This option randomizes the module text section by reordering the text
section by function at module load time. In order to use this
-@@ -2390,6 +2389,20 @@ config MODULE_FG_KASLR
+@@ -2400,6 +2399,20 @@ config MODULE_FG_KASLR

If unsure, say N.

@@ -142,7 +142,7 @@ index 7f39599e9fae..4ca9d8fc978d 100644
# Add FORCE to the prequisites of a target to force it to be always rebuilt.
# ---------------------------------------------------------------------------
diff --git a/scripts/generate_text_sections.pl b/scripts/generate_text_sections.pl
-index c95b9be28920..79a37a3c079f 100755
+index 999e1b68181f..a05ae9fb0041 100755
--- a/scripts/generate_text_sections.pl
+++ b/scripts/generate_text_sections.pl
@@ -48,6 +48,7 @@ my $readelf = $ENV{'READELF'} || die "$0: ERROR: READELF not set?";
@@ -167,7 +167,7 @@ index c95b9be28920..79a37a3c079f 100755
next;
}
@@ -141,7 +148,7 @@ sub print_reserve {
- ## If we have text sections aligned with 64 bytes or more, make
+ ## If we have text sections aligned with 128 bytes or more, make
## sure we reserve some space for them to not overlap _etext
## while shuffling sections.
- if (!$count) {
@@ -210,5 +210,5 @@ index 1d0e1e4dc3d2..6e957aa614b1 100644

/* bring in arch-specific sections */
--
-2.33.1
+2.34.1

diff --git a/0014-Documentation-add-documentation-for-FG-KASLR.patch b/0014-Documentation-add-documentation-for-FG-KASLR.patch
index 811e0e75de79..018778c38f4c 100644
--- a/0014-Documentation-add-documentation-for-FG-KASLR.patch
+++ b/0014-Documentation-add-documentation-for-FG-KASLR.patch
@@ -1,7 +1,7 @@
-From 612099530af493bdf26798897acf6b922c55b0ae Mon Sep 17 00:00:00 2001
+From e18582a0728127ad8607f1cebbd23314d95e398b Mon Sep 17 00:00:00 2001
From: Kristen Carlson Accardi <[email protected]>
Date: Fri, 27 Aug 2021 16:56:23 +0200
-Subject: [PATCH v9 14/15] Documentation: add documentation for FG-KASLR
+Subject: [PATCH v10 14/15] Documentation: add documentation for FG-KASLR

Describe the main principles behind the FG-KASLR hardening feature
in a new doc section.
@@ -16,10 +16,10 @@ Signed-off-by: Alexander Lobakin <[email protected]>
create mode 100644 Documentation/security/fgkaslr.rst

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
-index fc34332c8d9a..b79423c5a41a 100644
+index f5a27f067db9..bb8804c5fa5c 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
-@@ -2224,6 +2224,12 @@
+@@ -2235,6 +2235,12 @@
kernel and module base offset ASLR (Address Space
Layout Randomization).

@@ -223,5 +223,5 @@ index 16335de04e8c..41444124090f 100644
lsm
lsm-development
--
-2.33.1
+2.34.1

diff --git a/0015-maintainers-add-MAINTAINERS-entry-for-FG-KASLR.patch b/0015-maintainers-add-MAINTAINERS-entry-for-FG-KASLR.patch
index be939d69282a..61c615cd2352 100644
--- a/0015-maintainers-add-MAINTAINERS-entry-for-FG-KASLR.patch
+++ b/0015-maintainers-add-MAINTAINERS-entry-for-FG-KASLR.patch
@@ -1,7 +1,7 @@
-From 07533a2b6703a950bde7e4a3fce7a752ed17e180 Mon Sep 17 00:00:00 2001
+From 22aaf043900d66d1f0b17a064774aca1d22b81d2 Mon Sep 17 00:00:00 2001
From: Alexander Lobakin <[email protected]>
Date: Fri, 27 Aug 2021 16:58:35 +0200
-Subject: [PATCH v9 15/15] maintainers: add MAINTAINERS entry for FG-KASLR
+Subject: [PATCH v10 15/15] maintainers: add MAINTAINERS entry for FG-KASLR

Add an entry for FG-KASLR containing the maintainers, reviewers,
public mailing lists, files and so on.
@@ -12,10 +12,10 @@ Signed-off-by: Alexander Lobakin <[email protected]>
1 file changed, 12 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
-index 8912b2c1260c..efdb313b6813 100644
+index f53d30463c21..49a0974d32a7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
-@@ -7853,6 +7853,18 @@ L: [email protected]
+@@ -7925,6 +7925,18 @@ L: [email protected]
S: Maintained
F: drivers/platform/x86/fujitsu-tablet.c

@@ -35,5 +35,5 @@ index 8912b2c1260c..efdb313b6813 100644
M: Miklos Szeredi <[email protected]>
L: [email protected]
--
-2.33.1
+2.34.1
--
2.34.1

2022-02-09 23:36:08

by Alexander Lobakin

[permalink] [raw]
Subject: [PATCH v10 06/15] x86: decouple ORC table sorting into a separate file

In order to be able to sort ORC entries from both the kernel and
the pre-boot compressed environment, place ORC sorting function
into the new file arch/x86/lib/orc.c.
It can be then included directly by the pre-boot code, and
ORC_COMPRESSED_BOOT definition guards out the sort_mutex which
is unavailable and unneeded in that environment.

Placing orc_ip() into `include/asm/orc_types.h` wipes out
sorttable.h's little code dup.

Suggested-by: Peter Zijlstra <[email protected]>
Signed-off-by: Alexander Lobakin <[email protected]>
---
arch/x86/include/asm/orc_types.h | 7 +++
arch/x86/kernel/unwind_orc.c | 63 +--------------------
arch/x86/lib/Makefile | 3 +
arch/x86/lib/orc.c | 78 ++++++++++++++++++++++++++
scripts/sorttable.h | 5 --
tools/arch/x86/include/asm/orc_types.h | 7 +++
6 files changed, 96 insertions(+), 67 deletions(-)
create mode 100644 arch/x86/lib/orc.c

diff --git a/arch/x86/include/asm/orc_types.h b/arch/x86/include/asm/orc_types.h
index 5a2baf28a1dc..7708548713c4 100644
--- a/arch/x86/include/asm/orc_types.h
+++ b/arch/x86/include/asm/orc_types.h
@@ -67,6 +67,13 @@ struct orc_entry {
#endif
} __packed;

+static inline unsigned long orc_ip(const int *ip)
+{
+ return (unsigned long)ip + *ip;
+}
+
+void orc_sort(int *ip_table, struct orc_entry *orc_table, u32 num_orcs);
+
#endif /* __ASSEMBLY__ */

#endif /* _ORC_TYPES_H */
diff --git a/arch/x86/kernel/unwind_orc.c b/arch/x86/kernel/unwind_orc.c
index 2de3c8c5eba9..e5748bf15966 100644
--- a/arch/x86/kernel/unwind_orc.c
+++ b/arch/x86/kernel/unwind_orc.c
@@ -25,11 +25,6 @@ extern struct orc_entry __stop_orc_unwind[];
static bool orc_init __ro_after_init;
static unsigned int lookup_num_blocks __ro_after_init;

-static inline unsigned long orc_ip(const int *ip)
-{
- return (unsigned long)ip + *ip;
-}
-
static struct orc_entry *__orc_find(int *ip_table, struct orc_entry *u_table,
unsigned int num_entries, unsigned long ip)
{
@@ -188,53 +183,6 @@ static struct orc_entry *orc_find(unsigned long ip)
}

#ifdef CONFIG_MODULES
-
-static DEFINE_MUTEX(sort_mutex);
-static int *cur_orc_ip_table = __start_orc_unwind_ip;
-static struct orc_entry *cur_orc_table = __start_orc_unwind;
-
-static void orc_sort_swap(void *_a, void *_b, int size)
-{
- struct orc_entry *orc_a, *orc_b;
- struct orc_entry orc_tmp;
- int *a = _a, *b = _b, tmp;
- int delta = _b - _a;
-
- /* Swap the .orc_unwind_ip entries: */
- tmp = *a;
- *a = *b + delta;
- *b = tmp - delta;
-
- /* Swap the corresponding .orc_unwind entries: */
- orc_a = cur_orc_table + (a - cur_orc_ip_table);
- orc_b = cur_orc_table + (b - cur_orc_ip_table);
- orc_tmp = *orc_a;
- *orc_a = *orc_b;
- *orc_b = orc_tmp;
-}
-
-static int orc_sort_cmp(const void *_a, const void *_b)
-{
- struct orc_entry *orc_a;
- const int *a = _a, *b = _b;
- unsigned long a_val = orc_ip(a);
- unsigned long b_val = orc_ip(b);
-
- if (a_val > b_val)
- return 1;
- if (a_val < b_val)
- return -1;
-
- /*
- * The "weak" section terminator entries need to always be on the left
- * to ensure the lookup code skips them in favor of real entries.
- * These terminator entries exist to handle any gaps created by
- * whitelisted .o files which didn't get objtool generation.
- */
- orc_a = cur_orc_table + (a - cur_orc_ip_table);
- return orc_a->sp_reg == ORC_REG_UNDEFINED && !orc_a->end ? -1 : 1;
-}
-
void unwind_module_init(struct module *mod, void *_orc_ip, size_t orc_ip_size,
void *_orc, size_t orc_size)
{
@@ -246,16 +194,7 @@ void unwind_module_init(struct module *mod, void *_orc_ip, size_t orc_ip_size,
orc_size % sizeof(*orc) != 0 ||
num_entries != orc_size / sizeof(*orc));

- /*
- * The 'cur_orc_*' globals allow the orc_sort_swap() callback to
- * associate an .orc_unwind_ip table entry with its corresponding
- * .orc_unwind entry so they can both be swapped.
- */
- mutex_lock(&sort_mutex);
- cur_orc_ip_table = orc_ip;
- cur_orc_table = orc;
- sort(orc_ip, num_entries, sizeof(int), orc_sort_cmp, orc_sort_swap);
- mutex_unlock(&sort_mutex);
+ orc_sort(orc_ip, orc, num_entries);

mod->arch.orc_unwind_ip = orc_ip;
mod->arch.orc_unwind = orc;
diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile
index f76747862bd2..ff094cecebc4 100644
--- a/arch/x86/lib/Makefile
+++ b/arch/x86/lib/Makefile
@@ -50,6 +50,9 @@ lib-$(CONFIG_INSTRUCTION_DECODER) += insn.o inat.o insn-eval.o
lib-$(CONFIG_RANDOMIZE_BASE) += kaslr.o
lib-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
lib-$(CONFIG_RETPOLINE) += retpoline.o
+ifdef CONFIG_MODULES
+lib-$(CONFIG_UNWINDER_ORC) += orc.o
+endif

obj-y += msr.o msr-reg.o msr-reg-export.o hweight.o
obj-y += iomem.o
diff --git a/arch/x86/lib/orc.c b/arch/x86/lib/orc.c
new file mode 100644
index 000000000000..6a42842bf4ef
--- /dev/null
+++ b/arch/x86/lib/orc.c
@@ -0,0 +1,78 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * ORC sorting shared by the compressed boot code and ORC module
+ * support.
+ */
+
+#include <asm/orc_types.h>
+#include <linux/mutex.h>
+#include <linux/sort.h>
+
+#ifndef ORC_COMPRESSED_BOOT
+static DEFINE_MUTEX(sort_mutex);
+
+#define sort_mutex_lock() mutex_lock(&sort_mutex)
+#define sort_mutex_unlock() mutex_unlock(&sort_mutex)
+#else /* ORC_COMPRESSED_BOOT */
+#define sort_mutex_lock()
+#define sort_mutex_unlock()
+#endif /* ORC_COMPRESSED_BOOT */
+
+static int *cur_orc_ip_table;
+static struct orc_entry *cur_orc_table;
+
+static void orc_sort_swap(void *_a, void *_b, int size)
+{
+ struct orc_entry *orc_a, *orc_b;
+ int *a = _a, *b = _b, tmp;
+ int delta = _b - _a;
+
+ /* Swap the .orc_unwind_ip entries: */
+ tmp = *a;
+ *a = *b + delta;
+ *b = tmp - delta;
+
+ /* Swap the corresponding .orc_unwind entries: */
+ orc_a = cur_orc_table + (a - cur_orc_ip_table);
+ orc_b = cur_orc_table + (b - cur_orc_ip_table);
+ swap(*orc_a, *orc_b);
+}
+
+static int orc_sort_cmp(const void *_a, const void *_b)
+{
+ const int *a = _a, *b = _b;
+ unsigned long a_val = orc_ip(a);
+ unsigned long b_val = orc_ip(b);
+ struct orc_entry *orc_a;
+
+ if (a_val > b_val)
+ return 1;
+ if (a_val < b_val)
+ return -1;
+
+ /*
+ * The "weak" section terminator entries need to always be on the left
+ * to ensure the lookup code skips them in favor of real entries.
+ * These terminator entries exist to handle any gaps created by
+ * whitelisted .o files which didn't get objtool generation.
+ */
+ orc_a = cur_orc_table + (a - cur_orc_ip_table);
+
+ return orc_a->sp_reg == ORC_REG_UNDEFINED && !orc_a->end ? -1 : 1;
+}
+
+void orc_sort(int *ip_table, struct orc_entry *orc_table, u32 num_orcs)
+{
+ /*
+ * The 'cur_orc_*' globals allow the orc_sort_swap() callback to
+ * associate an .orc_unwind_ip table entry with its corresponding
+ * .orc_unwind entry so they can both be swapped.
+ */
+ sort_mutex_lock();
+
+ cur_orc_ip_table = ip_table;
+ cur_orc_table = orc_table;
+ sort(ip_table, num_orcs, sizeof(int), orc_sort_cmp, orc_sort_swap);
+
+ sort_mutex_unlock();
+}
diff --git a/scripts/sorttable.h b/scripts/sorttable.h
index deb7c1d3e979..a6bb46f36854 100644
--- a/scripts/sorttable.h
+++ b/scripts/sorttable.h
@@ -103,11 +103,6 @@ struct orc_entry *g_orc_table;

pthread_t orc_sort_thread;

-static inline unsigned long orc_ip(const int *ip)
-{
- return (unsigned long)ip + *ip;
-}
-
static int orc_sort_cmp(const void *_a, const void *_b)
{
struct orc_entry *orc_a;
diff --git a/tools/arch/x86/include/asm/orc_types.h b/tools/arch/x86/include/asm/orc_types.h
index 5a2baf28a1dc..7708548713c4 100644
--- a/tools/arch/x86/include/asm/orc_types.h
+++ b/tools/arch/x86/include/asm/orc_types.h
@@ -67,6 +67,13 @@ struct orc_entry {
#endif
} __packed;

+static inline unsigned long orc_ip(const int *ip)
+{
+ return (unsigned long)ip + *ip;
+}
+
+void orc_sort(int *ip_table, struct orc_entry *orc_table, u32 num_orcs);
+
#endif /* __ASSEMBLY__ */

#endif /* _ORC_TYPES_H */
--
2.34.1


2022-02-11 16:10:33

by Peter Zijlstra

[permalink] [raw]
Subject: Re: [PATCH v10 10/15] FG-KASLR: use a scripted approach to handle .text.* sections

On Wed, Feb 09, 2022 at 07:57:47PM +0100, Alexander Lobakin wrote:
> +sub read_sections {
> + open(my $fh, "\"$readelf\" -SW \"$file\" 2>/dev/null |")
> + or die "$0: ERROR: failed to execute \"$readelf\": $!";
> +
> + while (<$fh>) {
> + my $name;
> + my $align;
> + chomp;
> +
> + ($name, $align) = $_ =~ /^\s*\[[\s0-9]*\]\s*(\.\S*)\s*[A-Z]*\s*[0-9a-f]{16}\s*[0-9a-f]*\s*[0-9a-f]*\s*[0-9a-f]*\s*[0-9a-f]{2}\s*[A-Z]{2}\s*[0-9]\s*[0-9]\s*([0-9]*)$/;

Is there really no readable way to write this?

> +
> + if (!defined($name)) {
> + next;
> + }
> +
> + ## Clang 13 onwards emits __cfi_check_fail only on final
> + ## linking, so it won't appear in .o files and will be
> + ## missing in @sections. Add it manually to prevent
> + ## spawning orphans.
> + if ($name eq ".text.__cfi_check_fail") {
> + $has_ccf = 1;
> + }

How is that relevant, x86-64 doesn't and won't do clang-cfi.

2022-02-11 20:06:43

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

On Wed, Feb 09, 2022 at 07:57:39PM +0100, Alexander Lobakin wrote:
> Position-based search, which means that if there are several symbols
> with the same name, the user needs to additionally provide the
> "index" of a desired symbol, is fragile. For example, it breaks
> when two symbols with the same name are located in different
> sections.
>
> Since a while, LD has a flag `-z unique-symbol` which appends
> numeric suffixes to the functions with the same name (in symtab
> and strtab). It can be used to effectively prevent from having
> any ambiguity when referring to a symbol by its name.

In the patch description can you also give the version of binutils (and
possibly other linkers) which have the flag?

> Check for its availability and always prefer when the livepatching
> is on. It can be used unconditionally later on after broader testing
> on a wide variety of machines, but for now let's stick to the actual
> CONFIG_LIVEPATCH=y case, which is true for most of distro configs
> anyways.

Has anybody objected to just enabling it for *all* configs, not just for
livepatch?

I'd much prefer that: the less "special" livepatch is (and the distros
which enable it), the better. And I think having unique symbols would
benefit some other components.

> +++ b/kernel/livepatch/core.c
> @@ -143,11 +143,13 @@ static int klp_find_callback(void *data, const char *name,
> args->count++;
>
> /*
> - * Finish the search when the symbol is found for the desired position
> - * or the position is not defined for a non-unique symbol.
> + * Finish the search when unique symbol names are enabled
> + * or the symbol is found for the desired position or the
> + * position is not defined for a non-unique symbol.
> */
> - if ((args->pos && (args->count == args->pos)) ||
> - (!args->pos && (args->count > 1)))
> + if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL) ||
> + (args->pos && args->count == args->pos) ||
> + (!args->pos && args->count > 1))
> return 1;

There's no real need to do this. The code already works as-is, even if
there are no unique symbols.

Even if there are no duplicates, there's little harm in going through
all the symbols anyway, to check for errors just in case something
unexpected happened with the linking (unexpected duplicate) or the patch
creation (unexpected sympos). It's not a hot path, so performance isn't
really a concern.

When the old linker versions eventually age out, we can then go strip
out all the sympos stuff.

> @@ -169,6 +171,13 @@ static int klp_find_object_symbol(const char *objname, const char *name,
> else
> kallsyms_on_each_symbol(klp_find_callback, &args);
>
> + /*
> + * If the LD's `-z unique-symbol` flag is available and enabled,
> + * sympos checks are not relevant.
> + */
> + if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL))
> + sympos = 0;
> +

Similarly, I don't see a need for this. If the patch is legit then
sympos should already be zero. If not, an error gets reported and the
patch fails to load.

--
Josh

2022-02-11 20:36:12

by Peter Zijlstra

[permalink] [raw]
Subject: Re: [PATCH v10 05/15] x86: support asm function sections

On Wed, Feb 09, 2022 at 07:57:42PM +0100, Alexander Lobakin wrote:
> Address places which need special care and enable
> CONFIG_ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS.
>
> Notably:
> - propagate `--sectname-subst` to KBUILD_AFLAGS in
> x86/boot/Makefile and x86/boot/compressed/Makefile as both
> override them;
> - symbols starting with a dot (like ".Lrelocated") should be
> handled manually with SYM_*_START_SECT(.Lrelocated, relocated)
> as "two dots" is a special (and CPP doesn't want to concatenate
> two dots in general);
> - some symbols explicitly need to reside in one section (like
> kexec control code, hibernation page etc.);
> - macros creating aliases for functions (like __memcpy() for
> memcpy() etc.) should go after the main declaration (as
> aliases should be declared in the same section and they
> don't have SYM_PUSH_SECTION() inside);
> - things like ".org", ".align" should be manually pushed to
> the same section the next symbol goes to;
> - expand indirect_thunk wildcards in vmlinux.lds.S to catch
> symbols back into the "main" section;
> - inline ASM functions like __raw_callee*() should be pushed
> manually as well.
>
> With these changes and `-ffunction-sections` enabled, "plain"
> ".text" section is empty which means that everything works
> right as expected.
>
> Signed-off-by: Alexander Lobakin <[email protected]>
> ---
> arch/x86/Kconfig | 1 +
> arch/x86/boot/Makefile | 1 +
> arch/x86/boot/compressed/Makefile | 1 +
> arch/x86/boot/compressed/head_32.S | 2 +-
> arch/x86/boot/compressed/head_64.S | 32 ++++++++++++-------
> arch/x86/boot/pmjump.S | 2 +-
> arch/x86/crypto/aesni-intel_asm.S | 4 +--
> arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 4 +++
> arch/x86/include/asm/paravirt.h | 2 ++
> arch/x86/include/asm/qspinlock_paravirt.h | 2 ++
> arch/x86/kernel/head_32.S | 4 +--
> arch/x86/kernel/head_64.S | 4 +--
> arch/x86/kernel/kprobes/core.c | 2 ++
> arch/x86/kernel/kvm.c | 2 ++
> arch/x86/kernel/relocate_kernel_32.S | 10 +++---
> arch/x86/kernel/relocate_kernel_64.S | 12 ++++---
> arch/x86/kernel/vmlinux.lds.S | 2 +-
> arch/x86/kvm/emulate.c | 7 +++-
> arch/x86/lib/copy_user_64.S | 2 +-
> arch/x86/lib/error-inject.c | 2 ++
> arch/x86/lib/getuser.S | 5 ++-
> arch/x86/lib/memcpy_64.S | 4 +--
> arch/x86/lib/memmove_64.S | 5 ++-
> arch/x86/lib/memset_64.S | 5 +--
> arch/x86/lib/putuser.S | 2 +-
> arch/x86/power/hibernate_asm_32.S | 10 +++---
> arch/x86/power/hibernate_asm_64.S | 10 +++---
> 27 files changed, 89 insertions(+), 50 deletions(-)

Urgh, how much of that can you avoid by (ab)using __DISABLE_EXPORTS
like:

https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/commit/?h=x86/wip.ibt&id=ab74f54f2b1f6cfeaf2b3ba6999bde7cabada9ca

2022-02-11 20:52:11

by Fangrui Song

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

On Fri, Feb 11, 2022 at 9:41 AM Josh Poimboeuf <[email protected]> wrote:
>
> On Wed, Feb 09, 2022 at 07:57:39PM +0100, Alexander Lobakin wrote:
> > Position-based search, which means that if there are several symbols
> > with the same name, the user needs to additionally provide the
> > "index" of a desired symbol, is fragile. For example, it breaks
> > when two symbols with the same name are located in different
> > sections.
> >
> > Since a while, LD has a flag `-z unique-symbol` which appends
> > numeric suffixes to the functions with the same name (in symtab
> > and strtab). It can be used to effectively prevent from having
> > any ambiguity when referring to a symbol by its name.
>
> In the patch description can you also give the version of binutils (and
> possibly other linkers) which have the flag?

GNU ld>=2.36 supports -z unique-symbol. ld.lld doesn't support -z unique-symbol.

I subscribe to [email protected] and happen to notice this message
(can't keep up with the changes...)
I am a bit concerned with this option and replied last time on
https://lore.kernel.org/r/[email protected]

My full reasoning is on
https://maskray.me/blog/2020-11-15-explain-gnu-linker-options#z-unique-symbol

> > Check for its availability and always prefer when the livepatching
> > is on. It can be used unconditionally later on after broader testing
> > on a wide variety of machines, but for now let's stick to the actual
> > CONFIG_LIVEPATCH=y case, which is true for most of distro configs
> > anyways.
>
> Has anybody objected to just enabling it for *all* configs, not just for
> livepatch?
>
> I'd much prefer that: the less "special" livepatch is (and the distros
> which enable it), the better. And I think having unique symbols would
> benefit some other components.
>
> > +++ b/kernel/livepatch/core.c
> > @@ -143,11 +143,13 @@ static int klp_find_callback(void *data, const char *name,
> > args->count++;
> >
> > /*
> > - * Finish the search when the symbol is found for the desired position
> > - * or the position is not defined for a non-unique symbol.
> > + * Finish the search when unique symbol names are enabled
> > + * or the symbol is found for the desired position or the
> > + * position is not defined for a non-unique symbol.
> > */
> > - if ((args->pos && (args->count == args->pos)) ||
> > - (!args->pos && (args->count > 1)))
> > + if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL) ||
> > + (args->pos && args->count == args->pos) ||
> > + (!args->pos && args->count > 1))
> > return 1;
>
> There's no real need to do this. The code already works as-is, even if
> there are no unique symbols.
>
> Even if there are no duplicates, there's little harm in going through
> all the symbols anyway, to check for errors just in case something
> unexpected happened with the linking (unexpected duplicate) or the patch
> creation (unexpected sympos). It's not a hot path, so performance isn't
> really a concern.
>
> When the old linker versions eventually age out, we can then go strip
> out all the sympos stuff.
>
> > @@ -169,6 +171,13 @@ static int klp_find_object_symbol(const char *objname, const char *name,
> > else
> > kallsyms_on_each_symbol(klp_find_callback, &args);
> >
> > + /*
> > + * If the LD's `-z unique-symbol` flag is available and enabled,
> > + * sympos checks are not relevant.
> > + */
> > + if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL))
> > + sympos = 0;
> > +
>
> Similarly, I don't see a need for this. If the patch is legit then
> sympos should already be zero. If not, an error gets reported and the
> patch fails to load.
>
> --
> Josh
>
>


--
宋方睿

2022-02-14 13:44:52

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

From: Josh Poimboeuf <[email protected]>
Date: Fri, 11 Feb 2022 10:35:29 -0800

> On Fri, Feb 11, 2022 at 10:05:02AM -0800, Fāng-ruì Sòng wrote:
> > On Fri, Feb 11, 2022 at 9:41 AM Josh Poimboeuf <[email protected]> wrote:
> > >
> > > On Wed, Feb 09, 2022 at 07:57:39PM +0100, Alexander Lobakin wrote:
> > > > Position-based search, which means that if there are several symbols
> > > > with the same name, the user needs to additionally provide the
> > > > "index" of a desired symbol, is fragile. For example, it breaks
> > > > when two symbols with the same name are located in different
> > > > sections.
> > > >
> > > > Since a while, LD has a flag `-z unique-symbol` which appends
> > > > numeric suffixes to the functions with the same name (in symtab
> > > > and strtab). It can be used to effectively prevent from having
> > > > any ambiguity when referring to a symbol by its name.
> > >
> > > In the patch description can you also give the version of binutils (and
> > > possibly other linkers) which have the flag?
> >
> > GNU ld>=2.36 supports -z unique-symbol. ld.lld doesn't support -z unique-symbol.
> >
> > I subscribe to [email protected] and happen to notice this message
> > (can't keep up with the changes...)
> > I am a bit concerned with this option and replied last time on
> > https://lore.kernel.org/r/[email protected]
> >
> > My full reasoning is on
> > https://maskray.me/blog/2020-11-15-explain-gnu-linker-options#z-unique-symbol
>
> Ah, right. Also discussed here:
>
> https://lore.kernel.org/all/[email protected]/T/#u
> https://lore.kernel.org/all/20210125172124.awabevkpvq4poqxf@treble/
>
> I'm not qualified to comment on LTO/PGO stability issues, but it doesn't
> sound good. And we want to support livepatch for LTO kernels.
>
> Also I realized that this flag would have a negative effect on
> kpatch-build, as it currently does its analysis on .o files. So it
> would have to figure out how to properly detect function renames, to
> avoid patching the wrong function for example.
>
> And if LLD doesn't plan to support the flag then it will be a headache
> for livepatch (and the kernel in general) to deal with the divergent
> configs.

I'm always down with replacing any of the parts, I'm just not
familiar with any other ways of approaching this without huge diffs.
I've read Fāng-ruì's blogpost previously and there's a possible
replacement described there, but I dunno how to approach it.
And them Miroslav just told me that unique-symbol should work just
fine and I can go with it.
So I asked here prevously and ask once again for any hints regarding
some other ways :p

>
> One idea I mentioned before, it may be worth exploring changing the "F"
> in FGKASLR to "File" instead of "Function". In other words, only
> shuffle at an object-file granularity. Then, even with duplicates, the
> <file+function> symbol pair doesn't change in the symbol table. And as
> a bonus, it should help FGKASLR i-cache performance, significantly.

Yeah, I keep that in mind. However, this wouldn't solve the
duplicate static function names problem, right?
Let's say you have a static function f() in file1 and f() in file2,
then the layout each boot can be

.text.file1 or .text.file2
f() f()
.text.file2 .text.file1
f() f()

and position-based search won't work anyway, right?

>
> --
> Josh

Thanks,
Al

2022-02-14 18:20:48

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

On Mon, Feb 14, 2022 at 01:24:33PM +0100, Alexander Lobakin wrote:
> > One idea I mentioned before, it may be worth exploring changing the "F"
> > in FGKASLR to "File" instead of "Function". In other words, only
> > shuffle at an object-file granularity. Then, even with duplicates, the
> > <file+function> symbol pair doesn't change in the symbol table. And as
> > a bonus, it should help FGKASLR i-cache performance, significantly.
>
> Yeah, I keep that in mind. However, this wouldn't solve the
> duplicate static function names problem, right?
> Let's say you have a static function f() in file1 and f() in file2,
> then the layout each boot can be
>
> .text.file1 or .text.file2
> f() f()
> .text.file2 .text.file1
> f() f()
>
> and position-based search won't work anyway, right?

Right, so we'd have to abandon position-based search in favor of
file+func based search.

It's not perfect because there are still a few file+func duplicates.
But it might be good enough. We would presumably just refuse to patch a
duplicate. Or we could remove them (and enforce their continued removal
with tooling-based warnings).

Another variant of this which I described here

https://lore.kernel.org/all/20210125172124.awabevkpvq4poqxf@treble/

would be to keep it function-granular, but have kallsyms keep track of
what file each func belongs to. Then livepatch could still do the
file+func based search.

--
Josh

2022-02-14 19:16:18

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

From: Josh Poimboeuf <[email protected]>
Date: Fri, 11 Feb 2022 09:41:30 -0800

> On Wed, Feb 09, 2022 at 07:57:39PM +0100, Alexander Lobakin wrote:
> > Position-based search, which means that if there are several symbols
> > with the same name, the user needs to additionally provide the
> > "index" of a desired symbol, is fragile. For example, it breaks
> > when two symbols with the same name are located in different
> > sections.
> >
> > Since a while, LD has a flag `-z unique-symbol` which appends
> > numeric suffixes to the functions with the same name (in symtab
> > and strtab). It can be used to effectively prevent from having
> > any ambiguity when referring to a symbol by its name.
>
> In the patch description can you also give the version of binutils (and
> possibly other linkers) which have the flag?

Yeah, sure.

>
> > Check for its availability and always prefer when the livepatching
> > is on. It can be used unconditionally later on after broader testing
> > on a wide variety of machines, but for now let's stick to the actual
> > CONFIG_LIVEPATCH=y case, which is true for most of distro configs
> > anyways.
>
> Has anybody objected to just enabling it for *all* configs, not just for
> livepatch?

A few folks previously.

>
> I'd much prefer that: the less "special" livepatch is (and the distros
> which enable it), the better. And I think having unique symbols would
> benefit some other components.

Agree, I just want this series to be as least invasive for
non-FG-KASLR builds as possible. And currently this flag make depmod
emit a bunch of harmless false-positive warnings, so I'd wait until
at least the series is accepted / I post a patch for depmod and it
gets accepted.

>
> > +++ b/kernel/livepatch/core.c
> > @@ -143,11 +143,13 @@ static int klp_find_callback(void *data, const char *name,
> > args->count++;
> >
> > /*
> > - * Finish the search when the symbol is found for the desired position
> > - * or the position is not defined for a non-unique symbol.
> > + * Finish the search when unique symbol names are enabled
> > + * or the symbol is found for the desired position or the
> > + * position is not defined for a non-unique symbol.
> > */
> > - if ((args->pos && (args->count == args->pos)) ||
> > - (!args->pos && (args->count > 1)))
> > + if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL) ||
> > + (args->pos && args->count == args->pos) ||
> > + (!args->pos && args->count > 1))
> > return 1;
>
> There's no real need to do this. The code already works as-is, even if
> there are no unique symbols.
>
> Even if there are no duplicates, there's little harm in going through
> all the symbols anyway, to check for errors just in case something
> unexpected happened with the linking (unexpected duplicate) or the patch
> creation (unexpected sympos). It's not a hot path, so performance isn't
> really a concern.
>
> When the old linker versions eventually age out, we can then go strip
> out all the sympos stuff.
>
> > @@ -169,6 +171,13 @@ static int klp_find_object_symbol(const char *objname, const char *name,
> > else
> > kallsyms_on_each_symbol(klp_find_callback, &args);
> >
> > + /*
> > + * If the LD's `-z unique-symbol` flag is available and enabled,
> > + * sympos checks are not relevant.
> > + */
> > + if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL))
> > + sympos = 0;
> > +
>
> Similarly, I don't see a need for this. If the patch is legit then
> sympos should already be zero. If not, an error gets reported and the
> patch fails to load.

Right, but for both those chunks the main idea is to let the
compiler optimize-out the code non-actual for unique-symbol builds:

add/remove: 0/0 grow/shrink: 1/2 up/down: 3/-80 (-77)
Function old new delta
klp_find_callback 139 142 +3
klp_find_object_symbol.cold 85 48 -37
klp_find_object_symbol 168 125 -43

>
> --
> Josh

Thanks,
Al

2022-02-14 19:18:04

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH v10 05/15] x86: support asm function sections

From: Peter Zijlstra <[email protected]>
Date: Fri, 11 Feb 2022 16:45:24 +0100

> On Wed, Feb 09, 2022 at 07:57:42PM +0100, Alexander Lobakin wrote:
> > Address places which need special care and enable
> > CONFIG_ARCH_SUPPORTS_ASM_FUNCTION_SECTIONS.
> >
> > Notably:
> > - propagate `--sectname-subst` to KBUILD_AFLAGS in
> > x86/boot/Makefile and x86/boot/compressed/Makefile as both
> > override them;
> > - symbols starting with a dot (like ".Lrelocated") should be
> > handled manually with SYM_*_START_SECT(.Lrelocated, relocated)
> > as "two dots" is a special (and CPP doesn't want to concatenate
> > two dots in general);
> > - some symbols explicitly need to reside in one section (like
> > kexec control code, hibernation page etc.);
> > - macros creating aliases for functions (like __memcpy() for
> > memcpy() etc.) should go after the main declaration (as
> > aliases should be declared in the same section and they
> > don't have SYM_PUSH_SECTION() inside);
> > - things like ".org", ".align" should be manually pushed to
> > the same section the next symbol goes to;
> > - expand indirect_thunk wildcards in vmlinux.lds.S to catch
> > symbols back into the "main" section;
> > - inline ASM functions like __raw_callee*() should be pushed
> > manually as well.
> >
> > With these changes and `-ffunction-sections` enabled, "plain"
> > ".text" section is empty which means that everything works
> > right as expected.
> >
> > Signed-off-by: Alexander Lobakin <[email protected]>
> > ---
> > arch/x86/Kconfig | 1 +
> > arch/x86/boot/Makefile | 1 +
> > arch/x86/boot/compressed/Makefile | 1 +
> > arch/x86/boot/compressed/head_32.S | 2 +-
> > arch/x86/boot/compressed/head_64.S | 32 ++++++++++++-------
> > arch/x86/boot/pmjump.S | 2 +-
> > arch/x86/crypto/aesni-intel_asm.S | 4 +--
> > arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 4 +++
> > arch/x86/include/asm/paravirt.h | 2 ++
> > arch/x86/include/asm/qspinlock_paravirt.h | 2 ++
> > arch/x86/kernel/head_32.S | 4 +--
> > arch/x86/kernel/head_64.S | 4 +--
> > arch/x86/kernel/kprobes/core.c | 2 ++
> > arch/x86/kernel/kvm.c | 2 ++
> > arch/x86/kernel/relocate_kernel_32.S | 10 +++---
> > arch/x86/kernel/relocate_kernel_64.S | 12 ++++---
> > arch/x86/kernel/vmlinux.lds.S | 2 +-
> > arch/x86/kvm/emulate.c | 7 +++-
> > arch/x86/lib/copy_user_64.S | 2 +-
> > arch/x86/lib/error-inject.c | 2 ++
> > arch/x86/lib/getuser.S | 5 ++-
> > arch/x86/lib/memcpy_64.S | 4 +--
> > arch/x86/lib/memmove_64.S | 5 ++-
> > arch/x86/lib/memset_64.S | 5 +--
> > arch/x86/lib/putuser.S | 2 +-
> > arch/x86/power/hibernate_asm_32.S | 10 +++---
> > arch/x86/power/hibernate_asm_64.S | 10 +++---
> > 27 files changed, 89 insertions(+), 50 deletions(-)
>
> Urgh, how much of that can you avoid by (ab)using __DISABLE_EXPORTS
> like:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/commit/?h=x86/wip.ibt&id=ab74f54f2b1f6cfeaf2b3ba6999bde7cabada9ca

Oh, never thought on that, looks like at least 3 of 27 files and
35 of 139 lines, nice!
I'll redo this one in the meantime whilst waiting for more comments
here.

Thanks!
Al

2022-02-14 19:43:12

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH v10 10/15] FG-KASLR: use a scripted approach to handle .text.* sections

From: Peter Zijlstra <[email protected]>
Date: Fri, 11 Feb 2022 16:37:06 +0100

> On Wed, Feb 09, 2022 at 07:57:47PM +0100, Alexander Lobakin wrote:
> > +sub read_sections {
> > + open(my $fh, "\"$readelf\" -SW \"$file\" 2>/dev/null |")
> > + or die "$0: ERROR: failed to execute \"$readelf\": $!";
> > +
> > + while (<$fh>) {
> > + my $name;
> > + my $align;
> > + chomp;
> > +
> > + ($name, $align) = $_ =~ /^\s*\[[\s0-9]*\]\s*(\.\S*)\s*[A-Z]*\s*[0-9a-f]{16}\s*[0-9a-f]*\s*[0-9a-f]*\s*[0-9a-f]*\s*[0-9a-f]{2}\s*[A-Z]{2}\s*[0-9]\s*[0-9]\s*([0-9]*)$/;
>
> Is there really no readable way to write this?

I'm no regexp master, so I'd be glad if someone could help improve
this :D I only tried to optimize it using online tools, and they
eventually didn't help.
It could probably be replaced with a pipe to `cut` or `tr`, I'll
take a look a bit later if there won't be any other proposals.

>
> > +
> > + if (!defined($name)) {
> > + next;
> > + }
> > +
> > + ## Clang 13 onwards emits __cfi_check_fail only on final
> > + ## linking, so it won't appear in .o files and will be
> > + ## missing in @sections. Add it manually to prevent
> > + ## spawning orphans.
> > + if ($name eq ".text.__cfi_check_fail") {
> > + $has_ccf = 1;
> > + }
>
> How is that relevant, x86-64 doesn't and won't do clang-cfi.

1. One of my test cases was based on Sami's ClangCFI x86 series from
LKML (the one that was getting patched by objtool :P).
2. ClangCFI is present on ARM64 and it can receive FG-KASLR in a
future, why not make it work together from the start?

Re "won't do" -- sorry for trying to hijack this thread a bit, but
did I miss something? The last comments I've read were that LLVM
tools need to change their approach for CFI on x86, and Sami went
redo it, but I can't recall any "life-time" nacks.

Thanks,
Al

2022-02-14 19:44:06

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

On Fri, Feb 11, 2022 at 10:05:02AM -0800, Fāng-ruì Sòng wrote:
> On Fri, Feb 11, 2022 at 9:41 AM Josh Poimboeuf <[email protected]> wrote:
> >
> > On Wed, Feb 09, 2022 at 07:57:39PM +0100, Alexander Lobakin wrote:
> > > Position-based search, which means that if there are several symbols
> > > with the same name, the user needs to additionally provide the
> > > "index" of a desired symbol, is fragile. For example, it breaks
> > > when two symbols with the same name are located in different
> > > sections.
> > >
> > > Since a while, LD has a flag `-z unique-symbol` which appends
> > > numeric suffixes to the functions with the same name (in symtab
> > > and strtab). It can be used to effectively prevent from having
> > > any ambiguity when referring to a symbol by its name.
> >
> > In the patch description can you also give the version of binutils (and
> > possibly other linkers) which have the flag?
>
> GNU ld>=2.36 supports -z unique-symbol. ld.lld doesn't support -z unique-symbol.
>
> I subscribe to [email protected] and happen to notice this message
> (can't keep up with the changes...)
> I am a bit concerned with this option and replied last time on
> https://lore.kernel.org/r/[email protected]
>
> My full reasoning is on
> https://maskray.me/blog/2020-11-15-explain-gnu-linker-options#z-unique-symbol

Ah, right. Also discussed here:

https://lore.kernel.org/all/[email protected]/T/#u
https://lore.kernel.org/all/20210125172124.awabevkpvq4poqxf@treble/

I'm not qualified to comment on LTO/PGO stability issues, but it doesn't
sound good. And we want to support livepatch for LTO kernels.

Also I realized that this flag would have a negative effect on
kpatch-build, as it currently does its analysis on .o files. So it
would have to figure out how to properly detect function renames, to
avoid patching the wrong function for example.

And if LLD doesn't plan to support the flag then it will be a headache
for livepatch (and the kernel in general) to deal with the divergent
configs.

One idea I mentioned before, it may be worth exploring changing the "F"
in FGKASLR to "File" instead of "Function". In other words, only
shuffle at an object-file granularity. Then, even with duplicates, the
<file+function> symbol pair doesn't change in the symbol table. And as
a bonus, it should help FGKASLR i-cache performance, significantly.

--
Josh

2022-02-14 20:46:27

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH v10 10/15] FG-KASLR: use a scripted approach to handle .text.* sections

From: Peter Zijlstra <[email protected]>
Date: Mon, 14 Feb 2022 12:59:35 +0100

> On Mon, Feb 14, 2022 at 12:34:34PM +0100, Alexander Lobakin wrote:
>
> > Re "won't do" -- sorry for trying to hijack this thread a bit, but
> > did I miss something? The last comments I've read were that LLVM
> > tools need to change their approach for CFI on x86, and Sami went
> > redo it, but I can't recall any "life-time" nacks.
>
> Won't as in the lclang-cfi as it exists today. And I've understood that
> this CFI model is a keeper. It is true that Sami has been working on an
> alternative KCFI, but the little I can make of this proposal, it
> still needs serious work. Also see here:
>
> https://lkml.kernel.org/r/[email protected]
>
> Specifically, I object to the existence of any __*cfi_check_fail symbol
> on the grounds that it will bloat the code (and makes thinking about the
> whole speculation angle more painful than it needs to be).

Ah, I see, thanks! I've been tracking your IBT works, but missed
LKML thread for some reason.
I have no problems in dropping the related lines from my patch.

Al

2022-02-14 20:48:20

by Peter Zijlstra

[permalink] [raw]
Subject: Re: [PATCH v10 10/15] FG-KASLR: use a scripted approach to handle .text.* sections

On Mon, Feb 14, 2022 at 12:34:34PM +0100, Alexander Lobakin wrote:

> Re "won't do" -- sorry for trying to hijack this thread a bit, but
> did I miss something? The last comments I've read were that LLVM
> tools need to change their approach for CFI on x86, and Sami went
> redo it, but I can't recall any "life-time" nacks.

Won't as in the lclang-cfi as it exists today. And I've understood that
this CFI model is a keeper. It is true that Sami has been working on an
alternative KCFI, but the little I can make of this proposal, it
still needs serious work. Also see here:

https://lkml.kernel.org/r/[email protected]

Specifically, I object to the existence of any __*cfi_check_fail symbol
on the grounds that it will bloat the code (and makes thinking about the
whole speculation angle more painful than it needs to be).

2022-02-14 21:22:40

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

NOTE: Maybe -zunique-symbol won't get used after all, based on maskray's
objections. Regardless, I'm replying below, because the rest of the
approach in this patch seems all wrong.

On Mon, Feb 14, 2022 at 01:14:47PM +0100, Alexander Lobakin wrote:
> From: Josh Poimboeuf <[email protected]>
> Date: Fri, 11 Feb 2022 09:41:30 -0800
>
> > On Wed, Feb 09, 2022 at 07:57:39PM +0100, Alexander Lobakin wrote:
> > > Position-based search, which means that if there are several symbols
> > > with the same name, the user needs to additionally provide the
> > > "index" of a desired symbol, is fragile. For example, it breaks
> > > when two symbols with the same name are located in different
> > > sections.
> > >
> > > Since a while, LD has a flag `-z unique-symbol` which appends
> > > numeric suffixes to the functions with the same name (in symtab
> > > and strtab). It can be used to effectively prevent from having
> > > any ambiguity when referring to a symbol by its name.
> >
> > In the patch description can you also give the version of binutils (and
> > possibly other linkers) which have the flag?
>
> Yeah, sure.

> > > Check for its availability and always prefer when the livepatching
> > > is on. It can be used unconditionally later on after broader testing
> > > on a wide variety of machines, but for now let's stick to the actual
> > > CONFIG_LIVEPATCH=y case, which is true for most of distro configs
> > > anyways.
> >
> > Has anybody objected to just enabling it for *all* configs, not just for
> > livepatch?
>
> A few folks previously.

Why? It would be good to document that here.

> > I'd much prefer that: the less "special" livepatch is (and the distros
> > which enable it), the better. And I think having unique symbols would
> > benefit some other components.
>
> Agree, I just want this series to be as least invasive for
> non-FG-KASLR builds as possible.

But in a very real sense, this patch is making the series *more*
invasive by complexifying the config space.

Adding -zunique-symbols could have kernel-wide implications. If there
were bugs, we'd want to root them out, not hide them behind obscure
config combinations we hope nobody uses. Effectively this is
destabilizing CONFIG_LIVEPATCH.

Beyond "least invasive", we also need to consider:

- What makes fgkaslr most compatible with other features?
- What makes fgkaslr most palatable for wide use?
- What's best for the kernel as a whole?

It's much better to integrate new features properly with the kernel,
rather than just grafting them on to the side. Otherwise it just adds
technical debt, with no benefit to the rest of the kernel. Then it
might as well just remain an out-of-tree patch set.

> > > + if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL))
> > > + sympos = 0;
> > > +
> >
> > Similarly, I don't see a need for this. If the patch is legit then
> > sympos should already be zero. If not, an error gets reported and the
> > patch fails to load.
>
> Right, but for both those chunks the main idea is to let the
> compiler optimize-out the code non-actual for unique-symbol builds:
>
> add/remove: 0/0 grow/shrink: 1/2 up/down: 3/-80 (-77)
> Function old new delta
> klp_find_callback 139 142 +3
> klp_find_object_symbol.cold 85 48 -37
> klp_find_object_symbol 168 125 -43

As I said, it's not a hot path, so there's no need to complicate the
code with edge cases, and remove useful error checking in the process.

--
Josh

2022-02-16 16:24:45

by Miroslav Benes

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

> > +++ b/kernel/livepatch/core.c
> > @@ -143,11 +143,13 @@ static int klp_find_callback(void *data, const char *name,
> > args->count++;
> >
> > /*
> > - * Finish the search when the symbol is found for the desired position
> > - * or the position is not defined for a non-unique symbol.
> > + * Finish the search when unique symbol names are enabled
> > + * or the symbol is found for the desired position or the
> > + * position is not defined for a non-unique symbol.
> > */
> > - if ((args->pos && (args->count == args->pos)) ||
> > - (!args->pos && (args->count > 1)))
> > + if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL) ||
> > + (args->pos && args->count == args->pos) ||
> > + (!args->pos && args->count > 1))
> > return 1;
>
> There's no real need to do this. The code already works as-is, even if
> there are no unique symbols.
>
> Even if there are no duplicates, there's little harm in going through
> all the symbols anyway, to check for errors just in case something
> unexpected happened with the linking (unexpected duplicate) or the patch
> creation (unexpected sympos). It's not a hot path, so performance isn't
> really a concern.

Correct.

> When the old linker versions eventually age out, we can then go strip
> out all the sympos stuff.

Yes.

> > @@ -169,6 +171,13 @@ static int klp_find_object_symbol(const char *objname, const char *name,
> > else
> > kallsyms_on_each_symbol(klp_find_callback, &args);
> >
> > + /*
> > + * If the LD's `-z unique-symbol` flag is available and enabled,
> > + * sympos checks are not relevant.
> > + */
> > + if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL))
> > + sympos = 0;
> > +
>
> Similarly, I don't see a need for this. If the patch is legit then
> sympos should already be zero. If not, an error gets reported and the
> patch fails to load.

My concern was that if the patch is not legit (that is, sympos is > 0 for
some reason), the error would be really cryptic and would not help the
user at all. So zeroing sympos seems to be a good idea to me. There is no
harm and the change is very small and compact.

On the other hand, I do not insist on this.

Regards,
Miroslav

2022-02-16 16:40:21

by Miroslav Benes

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

On Fri, 11 Feb 2022, Josh Poimboeuf wrote:

> On Fri, Feb 11, 2022 at 10:05:02AM -0800, Fāng-ruì Sòng wrote:
> > On Fri, Feb 11, 2022 at 9:41 AM Josh Poimboeuf <[email protected]> wrote:
> > >
> > > On Wed, Feb 09, 2022 at 07:57:39PM +0100, Alexander Lobakin wrote:
> > > > Position-based search, which means that if there are several symbols
> > > > with the same name, the user needs to additionally provide the
> > > > "index" of a desired symbol, is fragile. For example, it breaks
> > > > when two symbols with the same name are located in different
> > > > sections.
> > > >
> > > > Since a while, LD has a flag `-z unique-symbol` which appends
> > > > numeric suffixes to the functions with the same name (in symtab
> > > > and strtab). It can be used to effectively prevent from having
> > > > any ambiguity when referring to a symbol by its name.
> > >
> > > In the patch description can you also give the version of binutils (and
> > > possibly other linkers) which have the flag?
> >
> > GNU ld>=2.36 supports -z unique-symbol. ld.lld doesn't support -z unique-symbol.
> >
> > I subscribe to [email protected] and happen to notice this message
> > (can't keep up with the changes...)
> > I am a bit concerned with this option and replied last time on
> > https://lore.kernel.org/r/[email protected]
> >
> > My full reasoning is on
> > https://maskray.me/blog/2020-11-15-explain-gnu-linker-options#z-unique-symbol
>
> Ah, right. Also discussed here:
>
> https://lore.kernel.org/all/[email protected]/T/#u
> https://lore.kernel.org/all/20210125172124.awabevkpvq4poqxf@treble/
>
> I'm not qualified to comment on LTO/PGO stability issues, but it doesn't
> sound good. And we want to support livepatch for LTO kernels.

Hm, bear with me, because I am very likely missing something which is
clear to everyone else...

Is the stability really a problem for the live patching (and I am talking
about the live patching only here. It may be a problem elsewhere, but I am
just trying to understand.)? I understand that two different kernel builds
could have a different name mapping between the original symbols and their
unique renames. Not nice. But we can prepare two different live patches
for these two different kernels. Something one would like to avoid if
possible, but it is not impossible. Am I missing something?

> Also I realized that this flag would have a negative effect on
> kpatch-build, as it currently does its analysis on .o files. So it
> would have to figure out how to properly detect function renames, to
> avoid patching the wrong function for example.

Yes, that is unfortunate. And not only for kpatch-build.

> And if LLD doesn't plan to support the flag then it will be a headache
> for livepatch (and the kernel in general) to deal with the divergent
> configs.

True.

The position-based approach clearly shows its limits. I like <file+func>
approach based on kallsyms tracking, that you proposed elsewhere in the
thread, more.

Miroslav

2022-02-16 20:35:11

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

On Wed, Feb 16, 2022 at 04:15:20PM +0100, Miroslav Benes wrote:
> > > I subscribe to [email protected] and happen to notice this message
> > > (can't keep up with the changes...)
> > > I am a bit concerned with this option and replied last time on
> > > https://lore.kernel.org/r/[email protected]
> > >
> > > My full reasoning is on
> > > https://maskray.me/blog/2020-11-15-explain-gnu-linker-options#z-unique-symbol
> >
> > Ah, right. Also discussed here:
> >
> > https://lore.kernel.org/all/[email protected]/T/#u
> > https://lore.kernel.org/all/20210125172124.awabevkpvq4poqxf@treble/
> >
> > I'm not qualified to comment on LTO/PGO stability issues, but it doesn't
> > sound good. And we want to support livepatch for LTO kernels.
>
> Hm, bear with me, because I am very likely missing something which is
> clear to everyone else...
>
> Is the stability really a problem for the live patching (and I am talking
> about the live patching only here. It may be a problem elsewhere, but I am
> just trying to understand.)? I understand that two different kernel builds
> could have a different name mapping between the original symbols and their
> unique renames. Not nice. But we can prepare two different live patches
> for these two different kernels. Something one would like to avoid if
> possible, but it is not impossible. Am I missing something?

Maybe Fāng-ruì can clarify, but my understanding was that the stability
issue affects the kernel in general (particularly if LTO or PGO is
enabled) and isn't necessarily specific to livepatch itself.

--
Josh

2022-02-16 21:20:02

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

On Wed, Feb 16, 2022 at 04:06:24PM +0100, Miroslav Benes wrote:
> > > + /*
> > > + * If the LD's `-z unique-symbol` flag is available and enabled,
> > > + * sympos checks are not relevant.
> > > + */
> > > + if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL))
> > > + sympos = 0;
> > > +
> >
> > Similarly, I don't see a need for this. If the patch is legit then
> > sympos should already be zero. If not, an error gets reported and the
> > patch fails to load.
>
> My concern was that if the patch is not legit (that is, sympos is > 0 for
> some reason), the error would be really cryptic and would not help the
> user at all. So zeroing sympos seems to be a good idea to me. There is no
> harm and the change is very small and compact.

But wouldn't a cryptic error be better than no error at all? A bad
sympos might be indicative of some larger issue, like the wrong symbol
getting patched.

--
Josh

2022-02-16 21:57:49

by Joe Lawrence

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

On Mon, Feb 14, 2022 at 10:10:00AM -0800, Josh Poimboeuf wrote:
> On Mon, Feb 14, 2022 at 01:24:33PM +0100, Alexander Lobakin wrote:
> > > One idea I mentioned before, it may be worth exploring changing the "F"
> > > in FGKASLR to "File" instead of "Function". In other words, only
> > > shuffle at an object-file granularity. Then, even with duplicates, the
> > > <file+function> symbol pair doesn't change in the symbol table. And as
> > > a bonus, it should help FGKASLR i-cache performance, significantly.
> >
> > Yeah, I keep that in mind. However, this wouldn't solve the
> > duplicate static function names problem, right?
> > Let's say you have a static function f() in file1 and f() in file2,
> > then the layout each boot can be
> >
> > .text.file1 or .text.file2
> > f() f()
> > .text.file2 .text.file1
> > f() f()
> >
> > and position-based search won't work anyway, right?
>
> Right, so we'd have to abandon position-based search in favor of
> file+func based search.
>
> It's not perfect because there are still a few file+func duplicates.
> But it might be good enough. We would presumably just refuse to patch a
> duplicate. Or we could remove them (and enforce their continued removal
> with tooling-based warnings).
>

You're talking about duplicate file+func combinations as stored in the
symbol table?

From a recent rhel-9 development kernel:

$ readelf --wide --symbols vmlinux | \
awk '$4=="FILE" { f=$NF } $4=="OBJECT" || $4=="FUNC" { print $4 " " f "::" $NF }' | \
sort | uniq -c | sort -n | awk '$1 != 1'

2 FUNC bus.c::new_id_store
2 FUNC core.c::native_read_msr
2 FUNC core.c::type_show
2 FUNC diag.c::sk_diag_fill.constprop.0
2 FUNC hid-core.c::hid_exit
2 FUNC hid-core.c::hid_init
2 FUNC inode.c::remove_one
2 FUNC iommu.c::__list_del_entry
2 FUNC msr.c::msr_init
2 FUNC msr.c::msr_read
2 FUNC proc.c::c_next
2 FUNC proc.c::c_start
2 FUNC proc.c::c_stop
2 FUNC raw.c::copy_overflow
2 FUNC raw.c::dst_output
2 FUNC route.c::dst_discard
2 FUNC sysfs.c::name_show
2 FUNC udp.c::copy_overflow
2 FUNC udp.c::udp_lib_close
2 FUNC udp.c::udp_lib_hash
2 FUNC udp.c::udplite_getfrag
2 FUNC udplite.c::udp_lib_close
2 FUNC udplite.c::udp_lib_hash
2 FUNC udplite.c::udplite_sk_init
2 OBJECT acpi.c::__func__.0
2 OBJECT amd.c::__already_done.10
2 OBJECT amd.c::__func__.4
2 OBJECT amd.c::__func__.5
2 OBJECT bus.c::driver_attr_new_id
2 OBJECT bus.c::__func__.1
2 OBJECT bus.c::__func__.2
2 OBJECT bus.c::__func__.3
2 OBJECT bus.c::__func__.4
2 OBJECT bus.c::__func__.5
2 OBJECT bus.c::__func__.6
2 OBJECT bus.c::__func__.7
2 OBJECT bus.c::__func__.8
2 OBJECT bus.c::__func__.9
2 OBJECT cgroup.c::__func__.0
2 OBJECT class.c::__func__.0
2 OBJECT class.c::__func__.1
2 OBJECT class.c::__func__.3
2 OBJECT class.c::__func__.5
2 OBJECT class.c::__key.0
2 OBJECT class.c::__key.1
2 OBJECT class.c::__key.4
2 OBJECT core.c::__already_done.18
2 OBJECT core.c::__already_done.19
2 OBJECT core.c::__already_done.3
2 OBJECT core.c::dev_attr_size
2 OBJECT core.c::dev_attr_start
2 OBJECT core.c::dev_attr_type
2 OBJECT core.c::empty_attrs
2 OBJECT core.c::__func__.10
2 OBJECT core.c::__func__.14
2 OBJECT core.c::__func__.7
2 OBJECT core.c::__func__.9
2 OBJECT core.c::__key.0
2 OBJECT core.c::__key.2
2 OBJECT core.c::__key.3
2 OBJECT dev.c::__func__.0
2 OBJECT dir.c::__func__.3
2 OBJECT driver.c::__func__.0
2 OBJECT fib_rules.c::__msg.0
2 OBJECT file.c::__func__.2
2 OBJECT file.c::__key.1
2 OBJECT file.c::__key.2
2 OBJECT hpet.c::__func__.4
2 OBJECT icmp.c::__func__.1
2 OBJECT inode.c::__func__.1
2 OBJECT inode.c::__func__.3
2 OBJECT intel.c::__already_done.10
2 OBJECT intel.c::__already_done.11
2 OBJECT intel.c::__already_done.13
2 OBJECT ioctl.c::__func__.0
2 OBJECT iommu.c::__already_done.15
2 OBJECT iommu.c::__func__.10
2 OBJECT iommu.c::__func__.2
2 OBJECT iommu.c::_rs.13
2 OBJECT iommu.c::_rs.5
2 OBJECT iommu.c::_rs.9
2 OBJECT irq.c::__func__.0
2 OBJECT irq.c::__func__.2
2 OBJECT irqdomain.c::__func__.0
2 OBJECT irqdomain.c::__func__.1
2 OBJECT irqdomain.c::__func__.3
2 OBJECT main.c::__func__.10
2 OBJECT main.c::__func__.11
2 OBJECT main.c::__func__.3
2 OBJECT main.c::__func__.4
2 OBJECT main.c::__func__.5
2 OBJECT manage.c::__func__.1
2 OBJECT mount.c::__func__.0
2 OBJECT msr.c::__func__.0
2 OBJECT ping.c::__func__.1
2 OBJECT property.c::__func__.3
2 OBJECT qos.c::__func__.0
2 OBJECT qos.c::__func__.2
2 OBJECT resource.c::__func__.1
2 OBJECT route.c::__key.0
2 OBJECT route.c::__msg.1
2 OBJECT route.c::__msg.2
2 OBJECT route.c::__msg.3
2 OBJECT route.c::__msg.4
2 OBJECT route.c::__msg.5
2 OBJECT route.c::__msg.6
2 OBJECT swap.c::__func__.0
2 OBJECT syncookies.c::___done.1
2 OBJECT syncookies.c::msstab
2 OBJECT syncookies.c::___once_key.2
2 OBJECT sysfs.c::dev_attr_name
2 OBJECT sysfs.c::__key.1
2 OBJECT sysfs.c::power_attrs
2 OBJECT udp.c::descriptor.12
2 OBJECT udp.c::descriptor.13
2 OBJECT udp.c::__func__.2
2 OBJECT udp.c::__func__.3
2 OBJECT udp.c::__func__.4
2 OBJECT utils.c::__func__.5
3 FUNC core.c::cmask_show
3 FUNC core.c::edge_show
3 FUNC core.c::event_show
3 FUNC core.c::inv_show
3 FUNC core.c::umask_show
3 FUNC inode.c::init_once
3 OBJECT acpi.c::__func__.1
3 OBJECT core.c::format_attr_cmask
3 OBJECT core.c::format_attr_edge
3 OBJECT core.c::format_attr_event
3 OBJECT core.c::format_attr_inv
3 OBJECT core.c::format_attr_umask
3 OBJECT core.c::__func__.6
3 OBJECT core.c::__func__.8
3 OBJECT file.c::__key.3
3 OBJECT generic.c::__func__.0
3 OBJECT iommu.c::__func__.0
3 OBJECT iommu.c::__func__.1
3 OBJECT iommu.c::__func__.8
3 OBJECT main.c::__func__.0
3 OBJECT main.c::__func__.1
3 OBJECT main.c::__func__.6
3 OBJECT quirks.c::__func__.0
3 OBJECT sysfs.c::__func__.0
4 OBJECT core.c::__func__.4
5 OBJECT inode.c::tokens
6 OBJECT core.c::__func__.3
6 OBJECT core.c::__func__.5
7 OBJECT core.c::__func__.1
8 OBJECT core.c::__func__.0
8 OBJECT core.c::__func__.2

We could probably minimize the FUNC duplicates with unique names, but
I'm not as optimistic about the OBJECTs as most are created via macros
like __already_done.X. Unless clever macro magic?

Next question: what are the odds that these entries, at least the ones
we can't easily rename, need disambiguity for livepatching? or
kpatch-build for related purposes?

-- Joe

2022-02-17 10:13:30

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

On Wed, Feb 16, 2022 at 03:32:41PM -0500, Joe Lawrence wrote:
> > Right, so we'd have to abandon position-based search in favor of
> > file+func based search.
> >
> > It's not perfect because there are still a few file+func duplicates.
> > But it might be good enough. We would presumably just refuse to patch a
> > duplicate. Or we could remove them (and enforce their continued removal
> > with tooling-based warnings).
> >
>
> You're talking about duplicate file+func combinations as stored in the
> symbol table?

Right.

> ...
> 6 OBJECT core.c::__func__.3
> 6 OBJECT core.c::__func__.5
> 7 OBJECT core.c::__func__.1
> 8 OBJECT core.c::__func__.0
> 8 OBJECT core.c::__func__.2
>
> We could probably minimize the FUNC duplicates with unique names, but
> I'm not as optimistic about the OBJECTs as most are created via macros
> like __already_done.X. Unless clever macro magic?

Good point about objects, as we rely on disambiguating them for klp
relocations. Luckily, the fact that most of them are created by macros
is largely a good thing. We consider most of those to be "special"
static locals, which don't actually need to be correlated or referenced
with a klp reloc.

For example:

- '__func__' is just the function name. The patched function shouldn't
need to reference the original function's function name string.

- '__already_done' is used for printk_once(); no harm in making a new
variable initialized to false and printing it again; or converting
printk_once() to just printk() to avoid an extra print.

- '__key' is used by lockdep to track lock usage and validate locking
order. It probably makes sense to use a new key in the patched
function, since the new function might have different locking
behavior.

> Next question: what are the odds that these entries, at least the ones
> we can't easily rename, need disambiguity for livepatching? or
> kpatch-build for related purposes?

I would guess the odds are rather low, given the fact that there are so
few functions, and we don't care about most of the objects on the list.

If duplicates were to become problematic then we could consider adding
tooling which warns on a duplicate file:sym pair with the goal of
eliminating duplicates (exculding the "special" objects).

--
Josh

2022-02-17 10:47:54

by Miroslav Benes

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

On Wed, 16 Feb 2022, Josh Poimboeuf wrote:

> On Wed, Feb 16, 2022 at 04:06:24PM +0100, Miroslav Benes wrote:
> > > > + /*
> > > > + * If the LD's `-z unique-symbol` flag is available and enabled,
> > > > + * sympos checks are not relevant.
> > > > + */
> > > > + if (IS_ENABLED(CONFIG_LD_HAS_Z_UNIQUE_SYMBOL))
> > > > + sympos = 0;
> > > > +
> > >
> > > Similarly, I don't see a need for this. If the patch is legit then
> > > sympos should already be zero. If not, an error gets reported and the
> > > patch fails to load.
> >
> > My concern was that if the patch is not legit (that is, sympos is > 0 for
> > some reason), the error would be really cryptic and would not help the
> > user at all. So zeroing sympos seems to be a good idea to me. There is no
> > harm and the change is very small and compact.
>
> But wouldn't a cryptic error be better than no error at all? A bad
> sympos might be indicative of some larger issue, like the wrong symbol
> getting patched.

Maybe you are right. I do not feel confident enough to decide it. So
either way would be fine, I guess.

Miroslav

2022-02-18 16:42:57

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

From: Miroslav Benes <[email protected]>
Date: Wed, 16 Feb 2022 16:15:20 +0100 (CET)

> On Fri, 11 Feb 2022, Josh Poimboeuf wrote:
>
> > On Fri, Feb 11, 2022 at 10:05:02AM -0800, Fāng-ruì Sòng wrote:
> > > On Fri, Feb 11, 2022 at 9:41 AM Josh Poimboeuf <[email protected]> wrote:
> > > >
> > > > On Wed, Feb 09, 2022 at 07:57:39PM +0100, Alexander Lobakin wrote:
> > > > > Position-based search, which means that if there are several symbols
> > > > > with the same name, the user needs to additionally provide the
> > > > > "index" of a desired symbol, is fragile. For example, it breaks
> > > > > when two symbols with the same name are located in different
> > > > > sections.
> > > > >
> > > > > Since a while, LD has a flag `-z unique-symbol` which appends
> > > > > numeric suffixes to the functions with the same name (in symtab
> > > > > and strtab). It can be used to effectively prevent from having
> > > > > any ambiguity when referring to a symbol by its name.
> > > >
> > > > In the patch description can you also give the version of binutils (and
> > > > possibly other linkers) which have the flag?
> > >
> > > GNU ld>=2.36 supports -z unique-symbol. ld.lld doesn't support -z unique-symbol.
> > >
> > > I subscribe to [email protected] and happen to notice this message
> > > (can't keep up with the changes...)
> > > I am a bit concerned with this option and replied last time on
> > > https://lore.kernel.org/r/[email protected]
> > >
> > > My full reasoning is on
> > > https://maskray.me/blog/2020-11-15-explain-gnu-linker-options#z-unique-symbol
> >
> > Ah, right. Also discussed here:
> >
> > https://lore.kernel.org/all/[email protected]/T/#u
> > https://lore.kernel.org/all/20210125172124.awabevkpvq4poqxf@treble/
> >
> > I'm not qualified to comment on LTO/PGO stability issues, but it doesn't
> > sound good. And we want to support livepatch for LTO kernels.
>
> Hm, bear with me, because I am very likely missing something which is
> clear to everyone else...
>
> Is the stability really a problem for the live patching (and I am talking
> about the live patching only here. It may be a problem elsewhere, but I am
> just trying to understand.)? I understand that two different kernel builds
> could have a different name mapping between the original symbols and their
> unique renames. Not nice. But we can prepare two different live patches
> for these two different kernels. Something one would like to avoid if
> possible, but it is not impossible. Am I missing something?
>
> > Also I realized that this flag would have a negative effect on
> > kpatch-build, as it currently does its analysis on .o files. So it
> > would have to figure out how to properly detect function renames, to
> > avoid patching the wrong function for example.
>
> Yes, that is unfortunate. And not only for kpatch-build.
>
> > And if LLD doesn't plan to support the flag then it will be a headache
> > for livepatch (and the kernel in general) to deal with the divergent
> > configs.
>
> True.
>
> The position-based approach clearly shows its limits. I like <file+func>
> approach based on kallsyms tracking, that you proposed elsewhere in the
> thread, more.

Hmm, same.

For FG-KASLR part, `-ffunction-sections` has no options, it only
appends the function name to the name of a function, i.e. it can
be only ".text.dup".
However, LD scripts allow to specify a particular input file for
the section being described, i.e.:

.text.dup { .text.file1_dup {
(.text.dup) -> file1.o(.text.dup)
} }
.text.file2_dup {
file2.o(.text.dup)
}

But the problem is that currently vmlinux is being linked from
vmlinux.o solely, so there are no input files apart from vmlinux.o.
I could probably (not 100% sure, I'm not deep into the details of
thin archives) create a temporary linker script for vmlinux.o
itself to process duplicates. Then vmlinux.o will always have only
unique section names right from the start.
It may not worth it: I don't mind that random functions with the
same name go into one section, it's not a big deal and/or security
risk, and it doesn't help livepatch which operates with symbol
names, not sections.

Re livepatch, the best option would probably be storing relative
paths to the object files in kallsyms. By relative I mean starting
from $srctree -- this would keep their versatility (no abspaths),
but provide needed uniquity:

dup() main.o:dup() init/main.o:dup() /mnt/init/main.o:dup()
dup() main.o:dup() foo/bar/main.o:dup() /mnt/foo/bar/main.o:dup()

^^^^^^ here ^^^^^^

The problem is that kallsyms are being generated at the moment of
(re)linking vmlinux already and no earlier.
If I could catch STT_FILE (can't say for sure now), it would provide
only filenames, so wouldn't be enough.
...oh wait, kallsyms rely on `nm` output. I checked nm's `-l` which
tries to find a file corresponding to each symbol and got a nice
output:

ffffffff8109ad00 T switch_mm_irqs_off /home/alobakin/Documents/work/xdp_hints/linux/arch/x86/mm/tlb.c:488

So this could be parsed with no issues nto:

name: switch_mm_irqs_off
addr: 0x9ad00 (rel)
file: arch/x86/mm/tlb.c

This solves a lot. One problem is that

> time nm -ln vmlinux > ~/Documents/tmp/nml
nm -ln vmlinux > ~/Documents/tmp/nml 120.80s user 1.77s system 99% cpu 2:02.94 total

it took 2 minutes to generate the whole map (instead of a split
second) (on 64-core CPU, but I guess nm runs in one thread).
I guess it can be optimized? I'm no a binutils master (will take a
look after sending this), is there a way to do it manually skipping
this nm lag or maybe make nm emit filenames without such delays?

>
> Miroslav

Thanks,
Al

2022-02-18 21:43:12

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH v10 02/15] livepatch: avoid position-based search if `-z unique-symbol` is available

On Fri, Feb 18, 2022 at 05:31:11PM +0100, Alexander Lobakin wrote:
> it took 2 minutes to generate the whole map (instead of a split
> second) (on 64-core CPU, but I guess nm runs in one thread).
> I guess it can be optimized? I'm no a binutils master (will take a
> look after sending this), is there a way to do it manually skipping
> this nm lag or maybe make nm emit filenames without such delays?

Hm, yeah, adding 2 minutes to the link time isn't going to fly ;-) It
probably takes a while to parse all the DWARF.

Based on ther other discussions I think just using the basename (main.o)
in STT_FILE would be good enough. Some duplicates are probably ok.

--
Josh

2022-05-03 01:35:56

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v10 01/15] modpost: fix removing numeric suffixes

On Thu, Feb 10, 2022 at 3:59 AM Alexander Lobakin
<[email protected]> wrote:
>
> `-z unique-symbol` linker flag which is planned to use with FG-KASLR
> to simplify livepatching (hopefully globally later on) triggers the
> following:
>
> ERROR: modpost: "param_set_uint.0" [vmlinux] is a static EXPORT_SYMBOL
>
> The reason is that for now the condition from remove_dot():
>
> if (m && (s[n + m] == '.' || s[n + m] == 0))
>
> which was designed to test if it's a dot or a '\0' after the suffix
> is never satisfied.
> This is due to that `s[n + m]` always points to the last digit of a
> numeric suffix, not on the symbol next to it (from a custom debug
> print added to modpost):
>
> param_set_uint.0, s[n + m] is '0', s[n + m + 1] is '\0'
>
> So it's off-by-one and was like that since 2014.
> Fix this for the sake of upcoming features, but don't bother
> stable-backporting, as it's well hidden -- apart from that LD flag,
> can be triggered only by GCC LTO which never landed upstream.
>
> Fixes: fcd38ed0ff26 ("scripts: modpost: fix compilation warning")
> Signed-off-by: Alexander Lobakin <[email protected]>
> ---


Acked-by: Masahiro Yamada <[email protected]>



> scripts/mod/modpost.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
> index 6bfa33217914..4648b7afe5cc 100644
> --- a/scripts/mod/modpost.c
> +++ b/scripts/mod/modpost.c
> @@ -1986,7 +1986,7 @@ static char *remove_dot(char *s)
>
> if (n && s[n]) {
> size_t m = strspn(s + n + 1, "0123456789");
> - if (m && (s[n + m] == '.' || s[n + m] == 0))
> + if (m && (s[n + m + 1] == '.' || s[n + m + 1] == 0))
> s[n] = 0;
>
> /* strip trailing .lto */
> --
> 2.34.1
>


--
Best Regards
Masahiro Yamada

2022-05-03 12:34:05

by Petr Mladek

[permalink] [raw]
Subject: Re: [PATCH v10 01/15] modpost: fix removing numeric suffixes

On Wed 2022-02-09 19:57:38, Alexander Lobakin wrote:
> `-z unique-symbol` linker flag which is planned to use with FG-KASLR
> to simplify livepatching (hopefully globally later on) triggers the
> following:
>
> ERROR: modpost: "param_set_uint.0" [vmlinux] is a static EXPORT_SYMBOL
>
> The reason is that for now the condition from remove_dot():
>
> if (m && (s[n + m] == '.' || s[n + m] == 0))
>
> which was designed to test if it's a dot or a '\0' after the suffix
> is never satisfied.
> This is due to that `s[n + m]` always points to the last digit of a
> numeric suffix, not on the symbol next to it (from a custom debug
> print added to modpost):
>
> param_set_uint.0, s[n + m] is '0', s[n + m + 1] is '\0'

Yup, the + 1 is for the '.' between the symbol name and the number.
In the order of apperance it would be: n + 1 + m

> So it's off-by-one and was like that since 2014.
> Fix this for the sake of upcoming features, but don't bother
> stable-backporting, as it's well hidden -- apart from that LD flag,
> can be triggered only by GCC LTO which never landed upstream.
>
> Fixes: fcd38ed0ff26 ("scripts: modpost: fix compilation warning")
> Signed-off-by: Alexander Lobakin <[email protected]>

Reviewed-by: Petr Mladek <[email protected]>

Best Regards,
Petr

2022-05-23 18:45:47

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v10 01/15] modpost: fix removing numeric suffixes

On Thu, Feb 10, 2022 at 3:59 AM Alexander Lobakin
<[email protected]> wrote:
>
> `-z unique-symbol` linker flag which is planned to use with FG-KASLR
> to simplify livepatching (hopefully globally later on) triggers the
> following:
>
> ERROR: modpost: "param_set_uint.0" [vmlinux] is a static EXPORT_SYMBOL
>
> The reason is that for now the condition from remove_dot():
>
> if (m && (s[n + m] == '.' || s[n + m] == 0))
>
> which was designed to test if it's a dot or a '\0' after the suffix
> is never satisfied.
> This is due to that `s[n + m]` always points to the last digit of a
> numeric suffix, not on the symbol next to it (from a custom debug
> print added to modpost):
>
> param_set_uint.0, s[n + m] is '0', s[n + m + 1] is '\0'
>
> So it's off-by-one and was like that since 2014.
> Fix this for the sake of upcoming features, but don't bother
> stable-backporting, as it's well hidden -- apart from that LD flag,
> can be triggered only by GCC LTO which never landed upstream.
>
> Fixes: fcd38ed0ff26 ("scripts: modpost: fix compilation warning")
> Signed-off-by: Alexander Lobakin <[email protected]>
> ---
> scripts/mod/modpost.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
> index 6bfa33217914..4648b7afe5cc 100644
> --- a/scripts/mod/modpost.c
> +++ b/scripts/mod/modpost.c
> @@ -1986,7 +1986,7 @@ static char *remove_dot(char *s)
>
> if (n && s[n]) {
> size_t m = strspn(s + n + 1, "0123456789");
> - if (m && (s[n + m] == '.' || s[n + m] == 0))
> + if (m && (s[n + m + 1] == '.' || s[n + m + 1] == 0))
> s[n] = 0;
>
> /* strip trailing .lto */
> --
> 2.34.1
>

This trivial patch has not been picked up yet.

I can apply this to my tree, if you want.

Please let me know your thoughts.


--
Best Regards
Masahiro Yamada

2022-05-25 01:18:03

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v10 01/15] modpost: fix removing numeric suffixes

On Tue, May 24, 2022 at 8:34 PM Alexander Lobakin
<[email protected]> wrote:
>
> From: Masahiro Yamada <[email protected]>
> Date: Tue, 24 May 2022 03:04:00 +0900
>
> > On Thu, Feb 10, 2022 at 3:59 AM Alexander Lobakin
> > <[email protected]> wrote:
> > >
> > > `-z unique-symbol` linker flag which is planned to use with FG-KASLR
> > > to simplify livepatching (hopefully globally later on) triggers the
> > > following:
> > >
> > > ERROR: modpost: "param_set_uint.0" [vmlinux] is a static EXPORT_SYMBOL
> > >
> > > The reason is that for now the condition from remove_dot():
> > >
> > > if (m && (s[n + m] == '.' || s[n + m] == 0))
> > >
> > > which was designed to test if it's a dot or a '\0' after the suffix
> > > is never satisfied.
> > > This is due to that `s[n + m]` always points to the last digit of a
> > > numeric suffix, not on the symbol next to it (from a custom debug
> > > print added to modpost):
> > >
> > > param_set_uint.0, s[n + m] is '0', s[n + m + 1] is '\0'
> > >
> > > So it's off-by-one and was like that since 2014.
> > > Fix this for the sake of upcoming features, but don't bother
> > > stable-backporting, as it's well hidden -- apart from that LD flag,
> > > can be triggered only by GCC LTO which never landed upstream.
> > >
> > > Fixes: fcd38ed0ff26 ("scripts: modpost: fix compilation warning")
> > > Signed-off-by: Alexander Lobakin <[email protected]>
> > > ---
> > > scripts/mod/modpost.c | 2 +-
> > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
> > > index 6bfa33217914..4648b7afe5cc 100644
> > > --- a/scripts/mod/modpost.c
> > > +++ b/scripts/mod/modpost.c
> > > @@ -1986,7 +1986,7 @@ static char *remove_dot(char *s)
> > >
> > > if (n && s[n]) {
> > > size_t m = strspn(s + n + 1, "0123456789");
> > > - if (m && (s[n + m] == '.' || s[n + m] == 0))
> > > + if (m && (s[n + m + 1] == '.' || s[n + m + 1] == 0))
> > > s[n] = 0;
> > >
> > > /* strip trailing .lto */
> > > --
> > > 2.34.1
> > >
> >
> > This trivial patch has not been picked up yet.
> >
> > I can apply this to my tree, if you want.
>
> It's a good idea, I'd like to!
> I don't use `-z unique-symbol` for FG-KALSR anymore*, but this fix
> is not directly related to it and can be taken independently.
> Should I change the commit message or it's ok to take it as it is?


I am fine with either way.

If you want to resubmit this with a fresh commit log,
please send it to:
[email protected]

Then, I will take care of it in this MW.

Thanks.




> >
> > Please let me know your thoughts.
> >
> >
> > --
> > Best Regards
> > Masahiro Yamada
>
> * I'm planning to submit a new rev of FG-KASLR series soon, but
> since I'm too busy with XDP for now, it will happen no sooner than
> in a couple months =\
>
> Thanks!
> Al



--
Best Regards
Masahiro Yamada

2022-05-25 01:20:14

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH v10 01/15] modpost: fix removing numeric suffixes

From: Masahiro Yamada <[email protected]>
Date: Tue, 24 May 2022 03:04:00 +0900

> On Thu, Feb 10, 2022 at 3:59 AM Alexander Lobakin
> <[email protected]> wrote:
> >
> > `-z unique-symbol` linker flag which is planned to use with FG-KASLR
> > to simplify livepatching (hopefully globally later on) triggers the
> > following:
> >
> > ERROR: modpost: "param_set_uint.0" [vmlinux] is a static EXPORT_SYMBOL
> >
> > The reason is that for now the condition from remove_dot():
> >
> > if (m && (s[n + m] == '.' || s[n + m] == 0))
> >
> > which was designed to test if it's a dot or a '\0' after the suffix
> > is never satisfied.
> > This is due to that `s[n + m]` always points to the last digit of a
> > numeric suffix, not on the symbol next to it (from a custom debug
> > print added to modpost):
> >
> > param_set_uint.0, s[n + m] is '0', s[n + m + 1] is '\0'
> >
> > So it's off-by-one and was like that since 2014.
> > Fix this for the sake of upcoming features, but don't bother
> > stable-backporting, as it's well hidden -- apart from that LD flag,
> > can be triggered only by GCC LTO which never landed upstream.
> >
> > Fixes: fcd38ed0ff26 ("scripts: modpost: fix compilation warning")
> > Signed-off-by: Alexander Lobakin <[email protected]>
> > ---
> > scripts/mod/modpost.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
> > index 6bfa33217914..4648b7afe5cc 100644
> > --- a/scripts/mod/modpost.c
> > +++ b/scripts/mod/modpost.c
> > @@ -1986,7 +1986,7 @@ static char *remove_dot(char *s)
> >
> > if (n && s[n]) {
> > size_t m = strspn(s + n + 1, "0123456789");
> > - if (m && (s[n + m] == '.' || s[n + m] == 0))
> > + if (m && (s[n + m + 1] == '.' || s[n + m + 1] == 0))
> > s[n] = 0;
> >
> > /* strip trailing .lto */
> > --
> > 2.34.1
> >
>
> This trivial patch has not been picked up yet.
>
> I can apply this to my tree, if you want.

It's a good idea, I'd like to!
I don't use `-z unique-symbol` for FG-KALSR anymore*, but this fix
is not directly related to it and can be taken independently.
Should I change the commit message or it's ok to take it as it is?

>
> Please let me know your thoughts.
>
>
> --
> Best Regards
> Masahiro Yamada

* I'm planning to submit a new rev of FG-KASLR series soon, but
since I'm too busy with XDP for now, it will happen no sooner than
in a couple months =\

Thanks!
Al