Introduce '.hyp.data..percpu' as part of ongoing effort to make nVHE
hyp code self-contained and independent of the rest of the kernel.
Main benefits:
* independent nVHE per-CPU data section that can be unmapped from host,
* more robust linking of nVHE hyp code,
* no need for hyp-specific macros to access per-CPU variables.
The series is structured as follows:
- patch 1: Improve existing hyp build rules. This could be sent and merged
independently of per-CPU but this series builds on it.
- patches 2-3: Minor cleanups.
- patches 4-5: Replace hyp helpers for accessing per-CPU variables
with common helpers modified to work correctly in hyp. Per-CPU
variables can now be accessed with one API anywhere.
- patches 6-8: Where VHE and nVHE use per-CPU variables defined in
kernel proper, move their definitions to hyp/ where they are
duplicated and owned by VHE/nVHE, respectively. Non-VHE hyp code
now refers only to per-CPU variables defined in its source files.
Helpers are added so that kernel proper can continue to access
nVHE hyp variables, same way as it does with other nVHE symbols.
- patches 9-11: Introduce '.hyp.data..percpu' ELF section and allocate
memory for every CPU core during KVM init. All nVHE per-CPU state
is now grouped together in ELF and in memory. Introducing a new
per-CPU variable does not require adding new memory mappings any
more. nVHE hyp code cannot accidentally refer to kernel-proper
per-CPU data as it only has the pointer to its own per-CPU memory.
Patches are rebased on v5.9-rc5 and available in branch 'topic/percpu-v3' at:
https://android-kvm.googlesource.com/linux
For maintainers: In case of interest, there are patches that remove the need
for redefining macros under DEBUG_PREEMPT available at the same repo, branch
'topic/percpu-v3-debug-preempt'. Since they are non-trivial, I am not going
to post them here so late in the 5.10 window. I plan to post them for 5.11
when they will also be useful for other patches.
Changes v2 -> v3:
* Use PERCPU_INPUT in hyp.ld instead of modifying PERCPU_SECTION
* Only pass linker script once to LD (fix error message)
* Renamed '.hyp.o' to '.nvhe.o'
* Use __KVM_VHE_HYPERVISOR__ to select TPIDR_EL2 instead of alternatives
* Move all prefixing-related macros to hyp_image.h
Changes v1 -> v2:
* partially link hyp code, add linker script
David Brazdil (11):
kvm: arm64: Partially link nVHE hyp code, simplify HYPCOPY
kvm: arm64: Move nVHE hyp namespace macros to hyp_image.h
kvm: arm64: Only define __kvm_ex_table for CONFIG_KVM
kvm: arm64: Remove __hyp_this_cpu_read
kvm: arm64: Remove hyp_adr/ldr_this_cpu
kvm: arm64: Add helpers for accessing nVHE hyp per-cpu vars
kvm: arm64: Duplicate arm64_ssbd_callback_required for nVHE hyp
kvm: arm64: Create separate instances of kvm_host_data for VHE/nVHE
kvm: arm64: Mark hyp stack pages reserved
kvm: arm64: Set up hyp percpu data for nVHE
kvm: arm64: Remove unnecessary hyp mappings
arch/arm64/include/asm/assembler.h | 27 ++++--
arch/arm64/include/asm/hyp_image.h | 36 +++++++
arch/arm64/include/asm/kvm_asm.h | 82 ++++++++--------
arch/arm64/include/asm/kvm_host.h | 2 +-
arch/arm64/include/asm/kvm_mmu.h | 19 ++--
arch/arm64/include/asm/percpu.h | 33 ++++++-
arch/arm64/include/asm/sections.h | 1 +
arch/arm64/kernel/image-vars.h | 4 -
arch/arm64/kernel/vmlinux.lds.S | 13 +++
arch/arm64/kvm/arm.c | 109 +++++++++++++++++++---
arch/arm64/kvm/hyp/hyp-entry.S | 2 +-
arch/arm64/kvm/hyp/include/hyp/debug-sr.h | 4 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 8 +-
arch/arm64/kvm/hyp/nvhe/Makefile | 60 ++++++------
arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 19 ++++
arch/arm64/kvm/hyp/nvhe/switch.c | 8 +-
arch/arm64/kvm/hyp/vhe/switch.c | 5 +-
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 4 +-
arch/arm64/kvm/pmu.c | 13 ++-
19 files changed, 320 insertions(+), 129 deletions(-)
create mode 100644 arch/arm64/include/asm/hyp_image.h
create mode 100644 arch/arm64/kvm/hyp/nvhe/hyp.lds.S
--
2.28.0.618.gf4bc123cb7-goog