LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V.
LoongArch includes a reduced 32-bit version (LA32R), a standard 32-bit
version (LA32S) and a 64-bit version (LA64). LoongArch use ACPI as its
boot protocol LoongArch-specific interrupt controllers (similar to APIC)
are already added in the next revision of ACPI Specification (current
revision is 6.4).
This patchset is adding basic LoongArch support in mainline kernel, we
can see a complete snapshot here:
https://github.com/loongson/linux/tree/loongarch-next
https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git/log/?h=loongarch-next
Cross-compile tool chain to build kernel:
https://github.com/loongson/build-tools/releases/download/2021.12.21/loongarch64-clfs-2022-03-03-cross-tools-gcc-glibc.tar.xz
A CLFS-based Linux distro:
https://github.com/loongson/build-tools/releases/download/2021.12.21/loongarch64-clfs-system-2022-03-03.tar.bz2
Open-source tool chain which is under review (Binutils and Gcc are already upstream):
https://github.com/loongson/binutils-gdb/tree/upstream_v3.1
https://github.com/loongson/gcc/tree/loongarch_upstream_v6.3
https://github.com/loongson/glibc/tree/loongarch_2_35_dev_v2.2
Loongson and LoongArch documentations:
https://github.com/loongson/LoongArch-Documentation
LoongArch-specific interrupt controllers:
https://mantis.uefi.org/mantis/view.php?id=2203
https://mantis.uefi.org/mantis/view.php?id=2313
V1 -> V2:
1, Add documentation patches;
2, Restore copyright statements;
3, Split the big header patch;
4, Cleanup signal-related headers;
5, Cleanup incomplete 32-bit support;
6, Move the major PCI work to drivers/pci;
7, Rework Loongson64 platform support;
8, Rework lpj and __udelay()/__ndelay();
9, Rework page table layout config options;
10, Rework syscall/exception/interrupt with generic entry framework;
11, Simplify the VDSO/VSYSCALL implementation;
12, Use generic I/O access macros and functions;
13, Remove unaligned access emulation at present;
14, Keep clocksource code in arch since it is the "native clocksource";
15, Some other minor fixes and improvements.
V2 -> V3:
1, Rebased on 5.15-rc1;
2, Cleanup PCI code on V2;
3, Support multiple msi domain;
4, Support cacheable ioremap();
5, Use irq stack for interrupt handling;
6, Adjust struct ucontext and rt_sigframe;
7, Some other minor fixes and improvements.
V3 -> V4:
1, Rebased on 5.15-rc3;
2, Rework SMP support and remove legacy IPI;
3, Rework signal support and remove loongarch_abi;
4, Simplify phys_to_dma() and dma_to_phys();
5, Remove unused sys_mmap2() implementation;
6, Remove unused strncpy_user.S and strnlen_user.S;
7, Some other minor fixes and improvements.
V4 -> V5:
1, Rebased on 5.15-rc4;
2, Fix a _PAGE_CHG_MASK bug;
3, Fix vdso_base() calculation;
4, Adjust syscall and ptrace code;
5, Use generic bitops implementation;
6, Avoid syscall restart handling in sys_rt_sigreturn();
7, Update commit messages.
V5 -> V6:
1, Rebased on 5.17-rc4;
2, Use GENERIC_IRQ_MIGRATION;
3, Improve sigcontext definition;
4, Improve numa_default_distance();
5, Increse MINSIGSTKSZ and SIGSTKSZ;
6, Restruct pt_regs and user_pt_regs;
7, Fix a corner case of protection_map;
8, Fix some corner cases of system calls;
9, Separate module region and vmalloc region;
10, Rename registers to match official documents.
V6 -> V7:
1, Rebased on 5.17-rc6;
2, Refactor do_page_fault();
3, Adjust memblock initialization;
4, Use -mstrict-align to build kernel;
5, Reimplement elf_read_implies_exec() as other archs;
6, Some other minor fixes and improvements.
V7 -> V8:
1, Rebased on 5.17-rc8;
2, Remove useless abidefs.h;
3, Remove useless HAVE_ARCH_NODEDATA_EXTENSION;
4, Fix and simplify uaccess.h;
5, Fix bugs after pt_regs restruction;
6, Use generic copy_from_user_page()/copy_to_user_page();
7, Some other minor fixes and improvements.
V8 -> V9:
1, Rebased on 5.18-rc4;
2, Fix 4-level page tables;
3, Always use 16KB kernel stack;
4, Add efistub and zboot support;
5, Remove useless HAVE_FUTEX_CMPXCHG;
6, Optimize module machanism for size;
7, Define copy_user_page() to avoid build errors;
8, Some other minor fixes and improvements.
V9 -> V10:
1, Rebased on 5.18-rc6;
2, Use generic efi stub;
3, Use generic string library;
4, Use generic ticket spinlock;
5, Use more meaningful macro naming;
6, Remove the zboot patch;
7, Fix commit message and documentations;
8, Some other minor fixes and improvements.
V10 -> V11:
1, Rebased on asm-generic tree;
2, Fix fpreg macros definition;
3, Fix ELF ABI macros definition;
4, Fix magic number definition in efi header;
5, Remove unneeded swab.h, bitfield.h and rtc.c;
6, Remove __ARCH_WANT_NEW_STAT (glibc need update);
7, Improve documents as WANG Xuerui suggested;
8, Some other minor fixes and improvements.
V11 -> V12:
1, Rebased on mainline;
2, Remove ACPI, PCI and IRQCHIP dependency;
3, Remove unneeded register_gop_device();
4, Fix show_code() and __arch_get_hw_counter();
5, Add a MAINTAINERS entry;
6, Rewrite documents as WANG Xuerui suggested;
7, Some other small adjustment.
Huacai Chen(24):
irqchip: Adjust Kconfig for Loongson.
irqchip/loongson-liointc: Fix build error for LoongArch.
Documentation: LoongArch: Add basic documentations.
Documentation/zh_CN: Add basic LoongArch documentations.
LoongArch: Add elf-related definitions.
LoongArch: Add writecombine support for drm.
LoongArch: Add build infrastructure.
LoongArch: Add CPU definition headers.
LoongArch: Add atomic/locking headers.
LoongArch: Add other common headers.
LoongArch: Add boot and setup routines.
LoongArch: Add exception/interrupt handling.
LoongArch: Add process management.
LoongArch: Add memory management.
LoongArch: Add system call support.
LoongArch: Add signal handling support.
LoongArch: Add elf and module support.
LoongArch: Add misc common routines.
LoongArch: Add some library functions.
LoongArch: Add PCI controller support.
LoongArch: Add VDSO and VSYSCALL support.
LoongArch: Add multi-processor (SMP) support.
LoongArch: Add Non-Uniform Memory Access (NUMA) support.
LoongArch: Add Loongson-3 default config file.
Signed-off-by: Huacai Chen <[email protected]>
---
Documentation/arch.rst | 1 +
Documentation/loongarch/features.rst | 3 +
Documentation/loongarch/index.rst | 21 +
Documentation/loongarch/introduction.rst | 387 +++++
Documentation/loongarch/irq-chip-model.rst | 168 +++
Documentation/translations/zh_CN/index.rst | 1 +
.../translations/zh_CN/loongarch/features.rst | 8 +
.../translations/zh_CN/loongarch/index.rst | 26 +
.../translations/zh_CN/loongarch/introduction.rst | 351 +++++
.../zh_CN/loongarch/irq-chip-model.rst | 167 +++
MAINTAINERS | 10 +
arch/loongarch/Kbuild | 6 +
arch/loongarch/Kconfig | 447 ++++++
arch/loongarch/Kconfig.debug | 0
arch/loongarch/Makefile | 105 ++
arch/loongarch/boot/.gitignore | 2 +
arch/loongarch/boot/Makefile | 20 +
arch/loongarch/boot/dts/Makefile | 4 +
arch/loongarch/configs/loongson3_defconfig | 771 ++++++++++
arch/loongarch/include/asm/Kbuild | 30 +
arch/loongarch/include/asm/acenv.h | 18 +
arch/loongarch/include/asm/acpi.h | 38 +
arch/loongarch/include/asm/addrspace.h | 112 ++
arch/loongarch/include/asm/asm-offsets.h | 5 +
arch/loongarch/include/asm/asm-prototypes.h | 7 +
arch/loongarch/include/asm/asm.h | 191 +++
arch/loongarch/include/asm/asmmacro.h | 289 ++++
arch/loongarch/include/asm/atomic.h | 362 +++++
arch/loongarch/include/asm/barrier.h | 159 ++
arch/loongarch/include/asm/bitops.h | 33 +
arch/loongarch/include/asm/bitrev.h | 34 +
arch/loongarch/include/asm/bootinfo.h | 43 +
arch/loongarch/include/asm/branch.h | 21 +
arch/loongarch/include/asm/bug.h | 23 +
arch/loongarch/include/asm/cache.h | 13 +
arch/loongarch/include/asm/cacheflush.h | 80 ++
arch/loongarch/include/asm/cacheops.h | 37 +
arch/loongarch/include/asm/clocksource.h | 12 +
arch/loongarch/include/asm/cmpxchg.h | 123 ++
arch/loongarch/include/asm/compiler.h | 15 +
arch/loongarch/include/asm/cpu-features.h | 73 +
arch/loongarch/include/asm/cpu-info.h | 116 ++
arch/loongarch/include/asm/cpu.h | 127 ++
arch/loongarch/include/asm/cpufeature.h | 24 +
arch/loongarch/include/asm/delay.h | 26 +
arch/loongarch/include/asm/dma-direct.h | 11 +
arch/loongarch/include/asm/dmi.h | 24 +
arch/loongarch/include/asm/efi.h | 41 +
arch/loongarch/include/asm/elf.h | 301 ++++
arch/loongarch/include/asm/entry-common.h | 13 +
arch/loongarch/include/asm/exec.h | 10 +
arch/loongarch/include/asm/fb.h | 23 +
arch/loongarch/include/asm/fixmap.h | 13 +
arch/loongarch/include/asm/fpregdef.h | 53 +
arch/loongarch/include/asm/fpu.h | 129 ++
arch/loongarch/include/asm/futex.h | 108 ++
arch/loongarch/include/asm/hardirq.h | 26 +
arch/loongarch/include/asm/hugetlb.h | 83 ++
arch/loongarch/include/asm/hw_irq.h | 17 +
arch/loongarch/include/asm/idle.h | 9 +
arch/loongarch/include/asm/inst.h | 117 ++
arch/loongarch/include/asm/io.h | 129 ++
arch/loongarch/include/asm/irq.h | 132 ++
arch/loongarch/include/asm/irq_regs.h | 27 +
arch/loongarch/include/asm/irqflags.h | 78 +
arch/loongarch/include/asm/kdebug.h | 23 +
arch/loongarch/include/asm/linkage.h | 36 +
arch/loongarch/include/asm/local.h | 138 ++
arch/loongarch/include/asm/loongarch.h | 1516 ++++++++++++++++++++
arch/loongarch/include/asm/loongson.h | 153 ++
arch/loongarch/include/asm/mmu.h | 16 +
arch/loongarch/include/asm/mmu_context.h | 152 ++
arch/loongarch/include/asm/mmzone.h | 18 +
arch/loongarch/include/asm/module.h | 80 ++
arch/loongarch/include/asm/module.lds.h | 7 +
arch/loongarch/include/asm/numa.h | 69 +
arch/loongarch/include/asm/page.h | 115 ++
arch/loongarch/include/asm/percpu.h | 214 +++
arch/loongarch/include/asm/perf_event.h | 10 +
arch/loongarch/include/asm/pgalloc.h | 103 ++
arch/loongarch/include/asm/pgtable-bits.h | 131 ++
arch/loongarch/include/asm/pgtable.h | 565 ++++++++
arch/loongarch/include/asm/prefetch.h | 29 +
arch/loongarch/include/asm/processor.h | 209 +++
arch/loongarch/include/asm/ptrace.h | 152 ++
arch/loongarch/include/asm/reboot.h | 10 +
arch/loongarch/include/asm/regdef.h | 41 +
arch/loongarch/include/asm/seccomp.h | 20 +
arch/loongarch/include/asm/serial.h | 11 +
arch/loongarch/include/asm/setup.h | 21 +
arch/loongarch/include/asm/shmparam.h | 12 +
arch/loongarch/include/asm/smp.h | 124 ++
arch/loongarch/include/asm/sparsemem.h | 23 +
arch/loongarch/include/asm/stackframe.h | 219 +++
arch/loongarch/include/asm/stacktrace.h | 74 +
arch/loongarch/include/asm/string.h | 12 +
arch/loongarch/include/asm/switch_to.h | 37 +
arch/loongarch/include/asm/syscall.h | 74 +
arch/loongarch/include/asm/thread_info.h | 106 ++
arch/loongarch/include/asm/time.h | 50 +
arch/loongarch/include/asm/timex.h | 33 +
arch/loongarch/include/asm/tlb.h | 180 +++
arch/loongarch/include/asm/tlbflush.h | 48 +
arch/loongarch/include/asm/topology.h | 41 +
arch/loongarch/include/asm/types.h | 19 +
arch/loongarch/include/asm/uaccess.h | 269 ++++
arch/loongarch/include/asm/unistd.h | 11 +
arch/loongarch/include/asm/vdso.h | 38 +
arch/loongarch/include/asm/vdso/clocksource.h | 8 +
arch/loongarch/include/asm/vdso/gettimeofday.h | 99 ++
arch/loongarch/include/asm/vdso/processor.h | 14 +
arch/loongarch/include/asm/vdso/vdso.h | 30 +
arch/loongarch/include/asm/vdso/vsyscall.h | 27 +
arch/loongarch/include/asm/vermagic.h | 19 +
arch/loongarch/include/asm/vmalloc.h | 4 +
arch/loongarch/include/uapi/asm/Kbuild | 2 +
arch/loongarch/include/uapi/asm/auxvec.h | 17 +
arch/loongarch/include/uapi/asm/bitsperlong.h | 9 +
arch/loongarch/include/uapi/asm/break.h | 23 +
arch/loongarch/include/uapi/asm/byteorder.h | 13 +
arch/loongarch/include/uapi/asm/hwcap.h | 20 +
arch/loongarch/include/uapi/asm/ptrace.h | 52 +
arch/loongarch/include/uapi/asm/reg.h | 59 +
arch/loongarch/include/uapi/asm/sigcontext.h | 44 +
arch/loongarch/include/uapi/asm/signal.h | 13 +
arch/loongarch/include/uapi/asm/ucontext.h | 35 +
arch/loongarch/include/uapi/asm/unistd.h | 5 +
arch/loongarch/kernel/.gitignore | 2 +
arch/loongarch/kernel/Makefile | 25 +
arch/loongarch/kernel/access-helper.h | 13 +
arch/loongarch/kernel/acpi.c | 333 +++++
arch/loongarch/kernel/asm-offsets.c | 264 ++++
arch/loongarch/kernel/cacheinfo.c | 122 ++
arch/loongarch/kernel/cpu-probe.c | 292 ++++
arch/loongarch/kernel/dma.c | 40 +
arch/loongarch/kernel/efi-header.S | 104 ++
arch/loongarch/kernel/efi.c | 226 +++
arch/loongarch/kernel/elf.c | 30 +
arch/loongarch/kernel/entry.S | 89 ++
arch/loongarch/kernel/env.c | 70 +
arch/loongarch/kernel/fpu.S | 261 ++++
arch/loongarch/kernel/genex.S | 95 ++
arch/loongarch/kernel/head.S | 129 ++
arch/loongarch/kernel/idle.c | 16 +
arch/loongarch/kernel/image-vars.h | 29 +
arch/loongarch/kernel/inst.c | 40 +
arch/loongarch/kernel/io.c | 94 ++
arch/loongarch/kernel/irq.c | 88 ++
arch/loongarch/kernel/mem.c | 64 +
arch/loongarch/kernel/module-sections.c | 121 ++
arch/loongarch/kernel/module.c | 375 +++++
arch/loongarch/kernel/numa.c | 466 ++++++
arch/loongarch/kernel/proc.c | 127 ++
arch/loongarch/kernel/process.c | 267 ++++
arch/loongarch/kernel/ptrace.c | 431 ++++++
arch/loongarch/kernel/reset.c | 102 ++
arch/loongarch/kernel/setup.c | 374 +++++
arch/loongarch/kernel/signal.c | 566 ++++++++
arch/loongarch/kernel/smp.c | 751 ++++++++++
arch/loongarch/kernel/switch.S | 35 +
arch/loongarch/kernel/syscall.c | 63 +
arch/loongarch/kernel/time.c | 214 +++
arch/loongarch/kernel/topology.c | 52 +
arch/loongarch/kernel/traps.c | 725 ++++++++++
arch/loongarch/kernel/vdso.c | 138 ++
arch/loongarch/kernel/vmlinux.lds.S | 121 ++
arch/loongarch/lib/Makefile | 6 +
arch/loongarch/lib/clear_user.S | 43 +
arch/loongarch/lib/copy_user.S | 47 +
arch/loongarch/lib/delay.c | 43 +
arch/loongarch/lib/dump_tlb.c | 111 ++
arch/loongarch/mm/Makefile | 9 +
arch/loongarch/mm/cache.c | 141 ++
arch/loongarch/mm/extable.c | 22 +
arch/loongarch/mm/fault.c | 261 ++++
arch/loongarch/mm/hugetlbpage.c | 87 ++
arch/loongarch/mm/init.c | 178 +++
arch/loongarch/mm/ioremap.c | 27 +
arch/loongarch/mm/maccess.c | 10 +
arch/loongarch/mm/mmap.c | 125 ++
arch/loongarch/mm/page.S | 84 ++
arch/loongarch/mm/pgtable.c | 130 ++
arch/loongarch/mm/tlb.c | 305 ++++
arch/loongarch/mm/tlbex.S | 546 +++++++
arch/loongarch/pci/Makefile | 7 +
arch/loongarch/vdso/.gitignore | 2 +
arch/loongarch/vdso/Makefile | 96 ++
arch/loongarch/vdso/elf.S | 15 +
arch/loongarch/vdso/gen_vdso_offsets.sh | 13 +
arch/loongarch/vdso/sigreturn.S | 24 +
arch/loongarch/vdso/vdso.S | 22 +
arch/loongarch/vdso/vdso.lds.S | 72 +
arch/loongarch/vdso/vgettimeofday.c | 25 +
drivers/firmware/efi/Kconfig | 2 +-
drivers/firmware/efi/libstub/Makefile | 10 +
drivers/firmware/efi/libstub/efi-stub-helper.c | 2 +-
drivers/firmware/efi/libstub/efi-stub.c | 2 +-
drivers/firmware/efi/libstub/loongarch-stub.c | 88 ++
drivers/gpu/drm/drm_vm.c | 2 +-
drivers/gpu/drm/ttm/ttm_module.c | 2 +-
drivers/irqchip/Kconfig | 6 +-
drivers/irqchip/irq-loongson-liointc.c | 6 +-
include/drm/drm_cache.h | 8 +
include/linux/cpuhotplug.h | 1 +
include/linux/efi.h | 1 +
include/linux/pe.h | 2 +
include/uapi/linux/audit.h | 2 +
include/uapi/linux/elf-em.h | 1 +
include/uapi/linux/elf.h | 5 +
include/uapi/linux/kexec.h | 1 +
scripts/sorttable.c | 5 +
scripts/subarch.include | 2 +-
tools/include/uapi/asm/bitsperlong.h | 2 +
213 files changed, 21500 insertions(+), 10 deletions(-)
create mode 100644 Documentation/loongarch/features.rst
create mode 100644 Documentation/loongarch/index.rst
create mode 100644 Documentation/loongarch/introduction.rst
create mode 100644 Documentation/loongarch/irq-chip-model.rst
create mode 100644 Documentation/translations/zh_CN/loongarch/features.rst
create mode 100644 Documentation/translations/zh_CN/loongarch/index.rst
create mode 100644 Documentation/translations/zh_CN/loongarch/introduction.rst
create mode 100644 Documentation/translations/zh_CN/loongarch/irq-chip-model.rst
create mode 100644 arch/loongarch/Kbuild
create mode 100644 arch/loongarch/Kconfig
create mode 100644 arch/loongarch/Kconfig.debug
create mode 100644 arch/loongarch/Makefile
create mode 100644 arch/loongarch/boot/.gitignore
create mode 100644 arch/loongarch/boot/Makefile
create mode 100644 arch/loongarch/boot/dts/Makefile
create mode 100644 arch/loongarch/configs/loongson3_defconfig
create mode 100644 arch/loongarch/include/asm/Kbuild
create mode 100644 arch/loongarch/include/asm/acenv.h
create mode 100644 arch/loongarch/include/asm/acpi.h
create mode 100644 arch/loongarch/include/asm/addrspace.h
create mode 100644 arch/loongarch/include/asm/asm-offsets.h
create mode 100644 arch/loongarch/include/asm/asm-prototypes.h
create mode 100644 arch/loongarch/include/asm/asm.h
create mode 100644 arch/loongarch/include/asm/asmmacro.h
create mode 100644 arch/loongarch/include/asm/atomic.h
create mode 100644 arch/loongarch/include/asm/barrier.h
create mode 100644 arch/loongarch/include/asm/bitops.h
create mode 100644 arch/loongarch/include/asm/bitrev.h
create mode 100644 arch/loongarch/include/asm/bootinfo.h
create mode 100644 arch/loongarch/include/asm/branch.h
create mode 100644 arch/loongarch/include/asm/bug.h
create mode 100644 arch/loongarch/include/asm/cache.h
create mode 100644 arch/loongarch/include/asm/cacheflush.h
create mode 100644 arch/loongarch/include/asm/cacheops.h
create mode 100644 arch/loongarch/include/asm/clocksource.h
create mode 100644 arch/loongarch/include/asm/cmpxchg.h
create mode 100644 arch/loongarch/include/asm/compiler.h
create mode 100644 arch/loongarch/include/asm/cpu-features.h
create mode 100644 arch/loongarch/include/asm/cpu-info.h
create mode 100644 arch/loongarch/include/asm/cpu.h
create mode 100644 arch/loongarch/include/asm/cpufeature.h
create mode 100644 arch/loongarch/include/asm/delay.h
create mode 100644 arch/loongarch/include/asm/dma-direct.h
create mode 100644 arch/loongarch/include/asm/dmi.h
create mode 100644 arch/loongarch/include/asm/efi.h
create mode 100644 arch/loongarch/include/asm/elf.h
create mode 100644 arch/loongarch/include/asm/entry-common.h
create mode 100644 arch/loongarch/include/asm/exec.h
create mode 100644 arch/loongarch/include/asm/fb.h
create mode 100644 arch/loongarch/include/asm/fixmap.h
create mode 100644 arch/loongarch/include/asm/fpregdef.h
create mode 100644 arch/loongarch/include/asm/fpu.h
create mode 100644 arch/loongarch/include/asm/futex.h
create mode 100644 arch/loongarch/include/asm/hardirq.h
create mode 100644 arch/loongarch/include/asm/hugetlb.h
create mode 100644 arch/loongarch/include/asm/hw_irq.h
create mode 100644 arch/loongarch/include/asm/idle.h
create mode 100644 arch/loongarch/include/asm/inst.h
create mode 100644 arch/loongarch/include/asm/io.h
create mode 100644 arch/loongarch/include/asm/irq.h
create mode 100644 arch/loongarch/include/asm/irq_regs.h
create mode 100644 arch/loongarch/include/asm/irqflags.h
create mode 100644 arch/loongarch/include/asm/kdebug.h
create mode 100644 arch/loongarch/include/asm/linkage.h
create mode 100644 arch/loongarch/include/asm/local.h
create mode 100644 arch/loongarch/include/asm/loongarch.h
create mode 100644 arch/loongarch/include/asm/loongson.h
create mode 100644 arch/loongarch/include/asm/mmu.h
create mode 100644 arch/loongarch/include/asm/mmu_context.h
create mode 100644 arch/loongarch/include/asm/mmzone.h
create mode 100644 arch/loongarch/include/asm/module.h
create mode 100644 arch/loongarch/include/asm/module.lds.h
create mode 100644 arch/loongarch/include/asm/numa.h
create mode 100644 arch/loongarch/include/asm/page.h
create mode 100644 arch/loongarch/include/asm/percpu.h
create mode 100644 arch/loongarch/include/asm/perf_event.h
create mode 100644 arch/loongarch/include/asm/pgalloc.h
create mode 100644 arch/loongarch/include/asm/pgtable-bits.h
create mode 100644 arch/loongarch/include/asm/pgtable.h
create mode 100644 arch/loongarch/include/asm/prefetch.h
create mode 100644 arch/loongarch/include/asm/processor.h
create mode 100644 arch/loongarch/include/asm/ptrace.h
create mode 100644 arch/loongarch/include/asm/reboot.h
create mode 100644 arch/loongarch/include/asm/regdef.h
create mode 100644 arch/loongarch/include/asm/seccomp.h
create mode 100644 arch/loongarch/include/asm/serial.h
create mode 100644 arch/loongarch/include/asm/setup.h
create mode 100644 arch/loongarch/include/asm/shmparam.h
create mode 100644 arch/loongarch/include/asm/smp.h
create mode 100644 arch/loongarch/include/asm/sparsemem.h
create mode 100644 arch/loongarch/include/asm/stackframe.h
create mode 100644 arch/loongarch/include/asm/stacktrace.h
create mode 100644 arch/loongarch/include/asm/string.h
create mode 100644 arch/loongarch/include/asm/switch_to.h
create mode 100644 arch/loongarch/include/asm/syscall.h
create mode 100644 arch/loongarch/include/asm/thread_info.h
create mode 100644 arch/loongarch/include/asm/time.h
create mode 100644 arch/loongarch/include/asm/timex.h
create mode 100644 arch/loongarch/include/asm/tlb.h
create mode 100644 arch/loongarch/include/asm/tlbflush.h
create mode 100644 arch/loongarch/include/asm/topology.h
create mode 100644 arch/loongarch/include/asm/types.h
create mode 100644 arch/loongarch/include/asm/uaccess.h
create mode 100644 arch/loongarch/include/asm/unistd.h
create mode 100644 arch/loongarch/include/asm/vdso.h
create mode 100644 arch/loongarch/include/asm/vdso/clocksource.h
create mode 100644 arch/loongarch/include/asm/vdso/gettimeofday.h
create mode 100644 arch/loongarch/include/asm/vdso/processor.h
create mode 100644 arch/loongarch/include/asm/vdso/vdso.h
create mode 100644 arch/loongarch/include/asm/vdso/vsyscall.h
create mode 100644 arch/loongarch/include/asm/vermagic.h
create mode 100644 arch/loongarch/include/asm/vmalloc.h
create mode 100644 arch/loongarch/include/uapi/asm/Kbuild
create mode 100644 arch/loongarch/include/uapi/asm/auxvec.h
create mode 100644 arch/loongarch/include/uapi/asm/bitsperlong.h
create mode 100644 arch/loongarch/include/uapi/asm/break.h
create mode 100644 arch/loongarch/include/uapi/asm/byteorder.h
create mode 100644 arch/loongarch/include/uapi/asm/hwcap.h
create mode 100644 arch/loongarch/include/uapi/asm/ptrace.h
create mode 100644 arch/loongarch/include/uapi/asm/reg.h
create mode 100644 arch/loongarch/include/uapi/asm/sigcontext.h
create mode 100644 arch/loongarch/include/uapi/asm/signal.h
create mode 100644 arch/loongarch/include/uapi/asm/ucontext.h
create mode 100644 arch/loongarch/include/uapi/asm/unistd.h
create mode 100644 arch/loongarch/kernel/.gitignore
create mode 100644 arch/loongarch/kernel/Makefile
create mode 100644 arch/loongarch/kernel/access-helper.h
create mode 100644 arch/loongarch/kernel/acpi.c
create mode 100644 arch/loongarch/kernel/asm-offsets.c
create mode 100644 arch/loongarch/kernel/cacheinfo.c
create mode 100644 arch/loongarch/kernel/cpu-probe.c
create mode 100644 arch/loongarch/kernel/dma.c
create mode 100644 arch/loongarch/kernel/efi-header.S
create mode 100644 arch/loongarch/kernel/efi.c
create mode 100644 arch/loongarch/kernel/elf.c
create mode 100644 arch/loongarch/kernel/entry.S
create mode 100644 arch/loongarch/kernel/env.c
create mode 100644 arch/loongarch/kernel/fpu.S
create mode 100644 arch/loongarch/kernel/genex.S
create mode 100644 arch/loongarch/kernel/head.S
create mode 100644 arch/loongarch/kernel/idle.c
create mode 100644 arch/loongarch/kernel/image-vars.h
create mode 100644 arch/loongarch/kernel/inst.c
create mode 100644 arch/loongarch/kernel/io.c
create mode 100644 arch/loongarch/kernel/irq.c
create mode 100644 arch/loongarch/kernel/mem.c
create mode 100644 arch/loongarch/kernel/module-sections.c
create mode 100644 arch/loongarch/kernel/module.c
create mode 100644 arch/loongarch/kernel/numa.c
create mode 100644 arch/loongarch/kernel/proc.c
create mode 100644 arch/loongarch/kernel/process.c
create mode 100644 arch/loongarch/kernel/ptrace.c
create mode 100644 arch/loongarch/kernel/reset.c
create mode 100644 arch/loongarch/kernel/setup.c
create mode 100644 arch/loongarch/kernel/signal.c
create mode 100644 arch/loongarch/kernel/smp.c
create mode 100644 arch/loongarch/kernel/switch.S
create mode 100644 arch/loongarch/kernel/syscall.c
create mode 100644 arch/loongarch/kernel/time.c
create mode 100644 arch/loongarch/kernel/topology.c
create mode 100644 arch/loongarch/kernel/traps.c
create mode 100644 arch/loongarch/kernel/vdso.c
create mode 100644 arch/loongarch/kernel/vmlinux.lds.S
create mode 100644 arch/loongarch/lib/Makefile
create mode 100644 arch/loongarch/lib/clear_user.S
create mode 100644 arch/loongarch/lib/copy_user.S
create mode 100644 arch/loongarch/lib/delay.c
create mode 100644 arch/loongarch/lib/dump_tlb.c
create mode 100644 arch/loongarch/mm/Makefile
create mode 100644 arch/loongarch/mm/cache.c
create mode 100644 arch/loongarch/mm/extable.c
create mode 100644 arch/loongarch/mm/fault.c
create mode 100644 arch/loongarch/mm/hugetlbpage.c
create mode 100644 arch/loongarch/mm/init.c
create mode 100644 arch/loongarch/mm/ioremap.c
create mode 100644 arch/loongarch/mm/maccess.c
create mode 100644 arch/loongarch/mm/mmap.c
create mode 100644 arch/loongarch/mm/page.S
create mode 100644 arch/loongarch/mm/pgtable.c
create mode 100644 arch/loongarch/mm/tlb.c
create mode 100644 arch/loongarch/mm/tlbex.S
create mode 100644 arch/loongarch/pci/Makefile
create mode 100644 arch/loongarch/vdso/.gitignore
create mode 100644 arch/loongarch/vdso/Makefile
create mode 100644 arch/loongarch/vdso/elf.S
create mode 100755 arch/loongarch/vdso/gen_vdso_offsets.sh
create mode 100644 arch/loongarch/vdso/sigreturn.S
create mode 100644 arch/loongarch/vdso/vdso.S
create mode 100644 arch/loongarch/vdso/vdso.lds.S
create mode 100644 arch/loongarch/vdso/vgettimeofday.c
create mode 100644 drivers/firmware/efi/libstub/loongarch-stub.c
--
2.27.0
在2022年6月1日六月 上午10:59，Huacai Chen写道：
> LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V.
> LoongArch includes a reduced 32-bit version (LA32R), a standard 32-bit
> version (LA32S) and a 64-bit version (LA64). LoongArch use ACPI as its
> boot protocol LoongArch-specific interrupt controllers (similar to APIC)
> are already added in the next revision of ACPI Specification (current
> revision is 6.4).
>
I’ve been reviewing all LA changes in past week and now I’m giving out R-b
for every patch I had reviewed in detail. (I don’t really now anything about
mm and processes so I just leave them).
I also tried to run the kernel on my machine with Huacai’s next tree and
Xuerui’s BPI patch.
I watched the “New World” of LoongArch grow up from scratch. And I must
say it’s a epic work showing the collaboration between community and Loongson
company. Especially Xuerui who invested numerous days and nights without any
return.
Thanks to all people involved.
- Jiaxun
Add some basic documentation for LoongArch. LoongArch is a new RISC ISA,
which is a bit like MIPS or RISC-V. LoongArch includes a reduced 32-bit
version (LA32R), a standard 32-bit version (LA32S) and a 64-bit version
(LA64).
Co-developed-by: WANG Xuerui <[email protected]>
Signed-off-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
Documentation/arch.rst | 1 +
Documentation/loongarch/features.rst | 3 +
Documentation/loongarch/index.rst | 21 ++
Documentation/loongarch/introduction.rst | 387 +++++++++++++++++++++
Documentation/loongarch/irq-chip-model.rst | 168 +++++++++
5 files changed, 580 insertions(+)
create mode 100644 Documentation/loongarch/features.rst
create mode 100644 Documentation/loongarch/index.rst
create mode 100644 Documentation/loongarch/introduction.rst
create mode 100644 Documentation/loongarch/irq-chip-model.rst
diff --git a/Documentation/arch.rst b/Documentation/arch.rst
index 14bcd8294b93..41a66a8b38e4 100644
--- a/Documentation/arch.rst
+++ b/Documentation/arch.rst
@@ -13,6 +13,7 @@ implementation.
arm/index
arm64/index
ia64/index
+ loongarch/index
m68k/index
mips/index
nios2/index
diff --git a/Documentation/loongarch/features.rst b/Documentation/loongarch/features.rst
new file mode 100644
index 000000000000..ebacade3ea45
--- /dev/null
+++ b/Documentation/loongarch/features.rst
@@ -0,0 +1,3 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. kernel-feat:: $srctree/Documentation/features loongarch
diff --git a/Documentation/loongarch/index.rst b/Documentation/loongarch/index.rst
new file mode 100644
index 000000000000..aaba648db907
--- /dev/null
+++ b/Documentation/loongarch/index.rst
@@ -0,0 +1,21 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======================
+LoongArch Architecture
+======================
+
+.. toctree::
+ :maxdepth: 2
+ :numbered:
+
+ introduction
+ irq-chip-model
+
+ features
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
diff --git a/Documentation/loongarch/introduction.rst b/Documentation/loongarch/introduction.rst
new file mode 100644
index 000000000000..b4df5b459677
--- /dev/null
+++ b/Documentation/loongarch/introduction.rst
@@ -0,0 +1,387 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================
+Introduction to LoongArch
+=========================
+
+LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V. There are
+currently 3 variants: a reduced 32-bit version (LA32R), a standard 32-bit
+version (LA32S) and a 64-bit version (LA64). There are 4 privilege levels
+(PLVs) defined in LoongArch: PLV0~PLV3, from high to low. Kernel runs at PLV0
+while applications run at PLV3. This document introduces the registers, basic
+instruction set, virtual memory and some other topics of LoongArch.
+
+Registers
+=========
+
+LoongArch registers include general purpose registers (GPRs), floating point
+registers (FPRs), vector registers (VRs) and control status registers (CSRs)
+used in privileged mode (PLV0).
+
+GPRs
+----
+
+LoongArch has 32 GPRs ( ``$r0`` ~ ``$r31`` ); each one is 32-bit wide in LA32
+and 64-bit wide in LA64. ``$r0`` is hard-wired to zero, and the other registers
+are not architecturally special. (Except ``$r1``, which is hard-wired as the
+link register of the BL instruction.)
+
+The kernel uses a variant of the LoongArch register convention, as described in
+the LoongArch ELF psABI spec, in :ref:`References <loongarch-references>`:
+
+================= =============== =================== ============
+Name Alias Usage Preserved
+ across calls
+================= =============== =================== ============
+``$r0`` ``$zero`` Constant zero Unused
+``$r1`` ``$ra`` Return address No
+``$r2`` ``$tp`` TLS/Thread pointer Unused
+``$r3`` ``$sp`` Stack pointer Yes
+``$r4``-``$r11`` ``$a0``-``$a7`` Argument registers No
+``$r4``-``$r5`` ``$v0``-``$v1`` Return value No
+``$r12``-``$r20`` ``$t0``-``$t8`` Temp registers No
+``$r21`` ``$u0`` Percpu base address Unused
+``$r22`` ``$fp`` Frame pointer Yes
+``$r23``-``$r31`` ``$s0``-``$s8`` Static registers Yes
+================= =============== =================== ============
+
+Note: The register ``$r21`` is reserved in the ELF psABI, but used by the Linux
+kernel for storing the percpu base address. It normally has no ABI name, but is
+called ``$u0`` in the kernel. You may also see ``$v0`` or ``$v1`` in some old code,
+they are deprecated aliases of ``$a0`` and ``$a1`` respectively.
+
+FPRs
+----
+
+LoongArch has 32 FPRs ( ``$f0`` ~ ``$f31`` ) when FPU is present. Each one is
+64-bit wide on the LA64 cores.
+
+The floating-point register convention is the same as described in the
+LoongArch ELF psABI spec:
+
+================= ================== =================== ============
+Name Alias Usage Preserved
+ across calls
+================= ================== =================== ============
+``$f0``-``$f7`` ``$fa0``-``$fa7`` Argument registers No
+``$f0``-``$f1`` ``$fv0``-``$fv1`` Return value No
+``$f8``-``$f23`` ``$ft0``-``$ft15`` Temp registers No
+``$f24``-``$f31`` ``$fs0``-``$fs7`` Static registers Yes
+================= ================== =================== ============
+
+Note: You may see ``$fv0`` or ``$fv1`` in some old code, they are deprecated
+aliases of ``$fa0`` and ``$fa1`` respectively.
+
+VRs
+----
+
+There are currently 2 vector extensions to LoongArch:
+
+- LSX (Loongson SIMD eXtension) with 128-bit vectors,
+- LASX (Loongson Advanced SIMD eXtension) with 256-bit vectors.
+
+LSX brings ``$v0`` ~ ``$v31`` while LASX brings ``$x0`` ~ ``$x31`` as the vector
+registers.
+
+The VRs overlap with FPRs: for example, on a core implementing LSX and LASX,
+the lower 128 bits of ``$x0`` is shared with ``$v0``, and the lower 64 bits of
+``$v0`` is shared with ``$f0``; same with all other VRs.
+
+CSRs
+----
+
+CSRs can only be accessed from privileged mode (PLV0):
+
+================= ===================================== ==============
+Address Full Name Abbrev Name
+================= ===================================== ==============
+0x0 Current Mode Information CRMD
+0x1 Pre-exception Mode Information PRMD
+0x2 Extension Unit Enable EUEN
+0x3 Miscellaneous Control MISC
+0x4 Exception Configuration ECFG
+0x5 Exception Status ESTAT
+0x6 Exception Return Address ERA
+0x7 Bad (Faulting) Virtual Address BADV
+0x8 Bad (Faulting) Instruction Word BADI
+0xC Exception Entrypoint Address EENTRY
+0x10 TLB Index TLBIDX
+0x11 TLB Entry High-order Bits TLBEHI
+0x12 TLB Entry Low-order Bits 0 TLBELO0
+0x13 TLB Entry Low-order Bits 1 TLBELO1
+0x18 Address Space Identifier ASID
+0x19 Page Global Directory Address for PGDL
+ Lower-half Address Space
+0x1A Page Global Directory Address for PGDH
+ Higher-half Address Space
+0x1B Page Global Directory Address PGD
+0x1C Page Walk Control for Lower- PWCL
+ half Address Space
+0x1D Page Walk Control for Higher- PWCH
+ half Address Space
+0x1E STLB Page Size STLBPS
+0x1F Reduced Virtual Address Configuration RVACFG
+0x20 CPU Identifier CPUID
+0x21 Privileged Resource Configuration 1 PRCFG1
+0x22 Privileged Resource Configuration 2 PRCFG2
+0x23 Privileged Resource Configuration 3 PRCFG3
+0x30+n (0≤n≤15) Saved Data register SAVEn
+0x40 Timer Identifier TID
+0x41 Timer Configuration TCFG
+0x42 Timer Value TVAL
+0x43 Compensation of Timer Count CNTC
+0x44 Timer Interrupt Clearing TICLR
+0x60 LLBit Control LLBCTL
+0x80 Implementation-specific Control 1 IMPCTL1
+0x81 Implementation-specific Control 2 IMPCTL2
+0x88 TLB Refill Exception Entrypoint TLBRENTRY
+ Address
+0x89 TLB Refill Exception BAD (Faulting) TLBRBADV
+ Virtual Address
+0x8A TLB Refill Exception Return Address TLBRERA
+0x8B TLB Refill Exception Saved Data TLBRSAVE
+ Register
+0x8C TLB Refill Exception Entry Low-order TLBRELO0
+ Bits 0
+0x8D TLB Refill Exception Entry Low-order TLBRELO1
+ Bits 1
+0x8E TLB Refill Exception Entry High-order TLBEHI
+ Bits
+0x8F TLB Refill Exception Pre-exception TLBRPRMD
+ Mode Information
+0x90 Machine Error Control MERRCTL
+0x91 Machine Error Information 1 MERRINFO1
+0x92 Machine Error Information 2 MERRINFO2
+0x93 Machine Error Exception Entrypoint MERRENTRY
+ Address
+0x94 Machine Error Exception Return MERRERA
+ Address
+0x95 Machine Error Exception Saved Data MERRSAVE
+ Register
+0x98 Cache TAGs CTAG
+0x180+n (0≤n≤3) Direct Mapping Configuration Window n DMWn
+0x200+2n (0≤n≤31) Performance Monitor Configuration n PMCFGn
+0x201+2n (0≤n≤31) Performance Monitor Overall Counter n PMCNTn
+0x300 Memory Load/Store WatchPoint MWPC
+ Overall Control
+0x301 Memory Load/Store WatchPoint MWPS
+ Overall Status
+0x310+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG1
+ Configuration 1
+0x311+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG2
+ Configuration 2
+0x312+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG3
+ Configuration 3
+0x313+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG4
+ Configuration 4
+0x380 Instruction Fetch WatchPoint FWPC
+ Overall Control
+0x381 Instruction Fetch WatchPoint FWPS
+ Overall Status
+0x390+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG1
+ Configuration 1
+0x391+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG2
+ Configuration 2
+0x392+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG3
+ Configuration 3
+0x393+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG4
+ Configuration 4
+0x500 Debug Register DBG
+0x501 Debug Exception Return Address DERA
+0x502 Debug Exception Saved Data Register DSAVE
+================= ===================================== ==============
+
+ERA, TLBRERA, MERRERA and DERA are sometimes also known as EPC, TLBREPC, MERREPC
+and DEPC respectively.
+
+Basic Instruction Set
+=====================
+
+Instruction formats
+-------------------
+
+LoongArch instructions are 32 bits wide, belonging to 9 basic instruction
+formats (and variants of them):
+
+=========== ==========================
+Format name Composition
+=========== ==========================
+2R Opcode + Rj + Rd
+3R Opcode + Rk + Rj + Rd
+4R Opcode + Ra + Rk + Rj + Rd
+2RI8 Opcode + I8 + Rj + Rd
+2RI12 Opcode + I12 + Rj + Rd
+2RI14 Opcode + I14 + Rj + Rd
+2RI16 Opcode + I16 + Rj + Rd
+1RI21 Opcode + I21L + Rj + I21H
+I26 Opcode + I26L + I26H
+=========== ==========================
+
+Rd is the destination register operand, while Rj, Rk and Ra ("a" stands for
+"additional") are the source register operands. I8/I12/I16/I21/I26 are
+immediate operands of respective width. The longer I21 and I26 are stored
+in separate higher and lower parts in the instruction word, denoted by the "L"
+and "H" suffixes.
+
+List of Instructions
+--------------------
+
+For brevity, only instruction names (mnemonics) are listed here; please see the
+:ref:`References <loongarch-references>` for details.
+
+
+1. Arithmetic Instructions::
+
+ ADD.W SUB.W ADDI.W ADD.D SUB.D ADDI.D
+ SLT SLTU SLTI SLTUI
+ AND OR NOR XOR ANDN ORN ANDI ORI XORI
+ MUL.W MULH.W MULH.WU DIV.W DIV.WU MOD.W MOD.WU
+ MUL.D MULH.D MULH.DU DIV.D DIV.DU MOD.D MOD.DU
+ PCADDI PCADDU12I PCADDU18I
+ LU12I.W LU32I.D LU52I.D ADDU16I.D
+
+2. Bit-shift Instructions::
+
+ SLL.W SRL.W SRA.W ROTR.W SLLI.W SRLI.W SRAI.W ROTRI.W
+ SLL.D SRL.D SRA.D ROTR.D SLLI.D SRLI.D SRAI.D ROTRI.D
+
+3. Bit-manipulation Instructions::
+
+ EXT.W.B EXT.W.H CLO.W CLO.D SLZ.W CLZ.D CTO.W CTO.D CTZ.W CTZ.D
+ BYTEPICK.W BYTEPICK.D BSTRINS.W BSTRINS.D BSTRPICK.W BSTRPICK.D
+ REVB.2H REVB.4H REVB.2W REVB.D REVH.2W REVH.D BITREV.4B BITREV.8B BITREV.W BITREV.D
+ MASKEQZ MASKNEZ
+
+4. Branch Instructions::
+
+ BEQ BNE BLT BGE BLTU BGEU BEQZ BNEZ B BL JIRL
+
+5. Load/Store Instructions::
+
+ LD.B LD.BU LD.H LD.HU LD.W LD.WU LD.D ST.B ST.H ST.W ST.D
+ LDX.B LDX.BU LDX.H LDX.HU LDX.W LDX.WU LDX.D STX.B STX.H STX.W STX.D
+ LDPTR.W LDPTR.D STPTR.W STPTR.D
+ PRELD PRELDX
+
+6. Atomic Operation Instructions::
+
+ LL.W SC.W LL.D SC.D
+ AMSWAP.W AMSWAP.D AMADD.W AMADD.D AMAND.W AMAND.D AMOR.W AMOR.D AMXOR.W AMXOR.D
+ AMMAX.W AMMAX.D AMMIN.W AMMIN.D
+
+7. Barrier Instructions::
+
+ IBAR DBAR
+
+8. Special Instructions::
+
+ SYSCALL BREAK CPUCFG NOP IDLE ERTN(ERET) DBCL(DBGCALL) RDTIMEL.W RDTIMEH.W RDTIME.D
+ ASRTLE.D ASRTGT.D
+
+9. Privileged Instructions::
+
+ CSRRD CSRWR CSRXCHG
+ IOCSRRD.B IOCSRRD.H IOCSRRD.W IOCSRRD.D IOCSRWR.B IOCSRWR.H IOCSRWR.W IOCSRWR.D
+ CACOP TLBP(TLBSRCH) TLBRD TLBWR TLBFILL TLBCLR TLBFLUSH INVTLB LDDIR LDPTE
+
+Virtual Memory
+==============
+
+LoongArch supports direct-mapped virtual memory and page-mapped virtual memory.
+
+Direct-mapped virtual memory is configured by CSR.DMWn (n=0~3), it has a simple
+relationship between virtual address (VA) and physical address (PA)::
+
+ VA = PA + FixedOffset
+
+Page-mapped virtual memory has arbitrary relationship between VA and PA, which
+is recorded in TLB and page tables. LoongArch's TLB includes a fully-associative
+MTLB (Multiple Page Size TLB) and set-associative STLB (Single Page Size TLB).
+
+By default, the whole virtual address space of LA32 is configured like this:
+
+============ =========================== =============================
+Name Address Range Attributes
+============ =========================== =============================
+``UVRANGE`` ``0x00000000 - 0x7FFFFFFF`` Page-mapped, Cached, PLV0~3
+``KPRANGE0`` ``0x80000000 - 0x9FFFFFFF`` Direct-mapped, Uncached, PLV0
+``KPRANGE1`` ``0xA0000000 - 0xBFFFFFFF`` Direct-mapped, Cached, PLV0
+``KVRANGE`` ``0xC0000000 - 0xFFFFFFFF`` Page-mapped, Cached, PLV0
+============ =========================== =============================
+
+User mode (PLV3) can only access UVRANGE. For direct-mapped KPRANGE0 and
+KPRANGE1, PA is equal to VA with bit30~31 cleared. For example, the uncached
+direct-mapped VA of 0x00001000 is 0x80001000, and the cached direct-mapped
+VA of 0x00001000 is 0xA0001000.
+
+By default, the whole virtual address space of LA64 is configured like this:
+
+============ ====================== ======================================
+Name Address Range Attributes
+============ ====================== ======================================
+``XUVRANGE`` ``0x0000000000000000 - Page-mapped, Cached, PLV0~3
+ 0x3FFFFFFFFFFFFFFF``
+``XSPRANGE`` ``0x4000000000000000 - Direct-mapped, Cached / Uncached, PLV0
+ 0x7FFFFFFFFFFFFFFF``
+``XKPRANGE`` ``0x8000000000000000 - Direct-mapped, Cached / Uncached, PLV0
+ 0xBFFFFFFFFFFFFFFF``
+``XKVRANGE`` ``0xC000000000000000 - Page-mapped, Cached, PLV0
+ 0xFFFFFFFFFFFFFFFF``
+============ ====================== ======================================
+
+User mode (PLV3) can only access XUVRANGE. For direct-mapped XSPRANGE and
+XKPRANGE, PA is equal to VA with bits 60~63 cleared, and the cache attribute
+is configured by bits 60~61 in VA: 0 is for strongly-ordered uncached, 1 is
+for coherent cached, and 2 is for weakly-ordered uncached.
+
+Currently we only use XKPRANGE for direct mapping and XSPRANGE is reserved.
+
+To put this in action: the strongly-ordered uncached direct-mapped VA (in
+XKPRANGE) of 0x00000000_00001000 is 0x80000000_00001000, the coherent cached
+direct-mapped VA (in XKPRANGE) of 0x00000000_00001000 is 0x90000000_00001000,
+and the weakly-ordered uncached direct-mapped VA (in XKPRANGE) of 0x00000000
+_00001000 is 0xA0000000_00001000.
+
+Relationship of Loongson and LoongArch
+======================================
+
+LoongArch is a RISC ISA which is different from any other existing ones, while
+Loongson is a family of processors. Loongson includes 3 series: Loongson-1 is
+the 32-bit processor series, Loongson-2 is the low-end 64-bit processor series,
+and Loongson-3 is the high-end 64-bit processor series. Old Loongson is based on
+MIPS, while New Loongson is based on LoongArch. Take Loongson-3 as an example:
+Loongson-3A1000/3B1500/3A2000/3A3000/3A4000 are MIPS-compatible, while Loongson-
+3A5000 (and future revisions) are all based on LoongArch.
+
+.. _loongarch-references:
+
+References
+==========
+
+Official web site of Loongson Technology Corp. Ltd.:
+
+ http://www.loongson.cn/
+
+Developer web site of Loongson and LoongArch (Software and Documentation):
+
+ http://www.loongnix.cn/
+
+ https://github.com/loongson/
+
+ https://loongson.github.io/LoongArch-Documentation/
+
+Documentation of LoongArch ISA:
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.00-CN.pdf (in Chinese)
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.00-EN.pdf (in English)
+
+Documentation of LoongArch ELF psABI:
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v1.00-CN.pdf (in Chinese)
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v1.00-EN.pdf (in English)
+
+Linux kernel repository of Loongson and LoongArch:
+
+ https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git
diff --git a/Documentation/loongarch/irq-chip-model.rst b/Documentation/loongarch/irq-chip-model.rst
new file mode 100644
index 000000000000..35c962991283
--- /dev/null
+++ b/Documentation/loongarch/irq-chip-model.rst
@@ -0,0 +1,168 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=======================================
+IRQ chip model (hierarchy) of LoongArch
+=======================================
+
+Currently, LoongArch based processors (e.g. Loongson-3A5000) can only work together
+with LS7A chipsets. The irq chips in LoongArch computers include CPUINTC (CPU Core
+Interrupt Controller), LIOINTC (Legacy I/O Interrupt Controller), EIOINTC (Extended
+I/O Interrupt Controller), HTVECINTC (Hyper-Transport Vector Interrupt Controller),
+PCH-PIC (Main Interrupt Controller in LS7A chipset), PCH-LPC (LPC Interrupt Controller
+in LS7A chipset) and PCH-MSI (MSI Interrupt Controller).
+
+CPUINTC is a per-core controller (in CPU), LIOINTC/EIOINTC/HTVECINTC are per-package
+controllers (in CPU), while PCH-PIC/PCH-LPC/PCH-MSI are controllers out of CPU (i.e.,
+in chipsets). These controllers (in other words, irqchips) are linked in a hierarchy,
+and there are two models of hierarchy (legacy model and extended model).
+
+Legacy IRQ model
+================
+
+In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer interrupt go
+to CPUINTC directly, CPU UARTS interrupts go to LIOINTC, while all other devices
+interrupts go to PCH-PIC/PCH-LPC/PCH-MSI and gathered by HTVECINTC, and then go
+to LIOINTC, and then CPUINTC.
+
+ +---------------------------------------------+
+ |:: |
+ | |
+ | +-----+ +---------+ +-------+ |
+ | | IPI | --> | CPUINTC | <-- | Timer | |
+ | +-----+ +---------+ +-------+ |
+ | ^ |
+ | | |
+ | +---------+ +-------+ |
+ | | LIOINTC | <-- | UARTs | |
+ | +---------+ +-------+ |
+ | ^ |
+ | | |
+ | +-----------+ |
+ | | HTVECINTC | |
+ | +-----------+ |
+ | ^ ^ |
+ | | | |
+ | +---------+ +---------+ |
+ | | PCH-PIC | | PCH-MSI | |
+ | +---------+ +---------+ |
+ | ^ ^ ^ |
+ | | | | |
+ | +---------+ +---------+ +---------+ |
+ | | PCH-LPC | | Devices | | Devices | |
+ | +---------+ +---------+ +---------+ |
+ | ^ |
+ | | |
+ | +---------+ |
+ | | Devices | |
+ | +---------+ |
+ | |
+ | |
+ +---------------------------------------------+
+
+Extended IRQ model
+==================
+
+In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer interrupt go
+to CPUINTC directly, CPU UARTS interrupts go to LIOINTC, while all other devices
+interrupts go to PCH-PIC/PCH-LPC/PCH-MSI and gathered by EIOINTC, and then go to
+to CPUINTC directly.
+
+ +--------------------------------------------------------+
+ |:: |
+ | |
+ | +-----+ +---------+ +-------+ |
+ | | IPI | --> | CPUINTC | <-- | Timer | |
+ | +-----+ +---------+ +-------+ |
+ | ^ ^ |
+ | | | |
+ | +---------+ +---------+ +-------+ |
+ | | EIOINTC | | LIOINTC | <-- | UARTs | |
+ | +---------+ +---------+ +-------+ |
+ | ^ ^ |
+ | | | |
+ | +---------+ +---------+ |
+ | | PCH-PIC | | PCH-MSI | |
+ | +---------+ +---------+ |
+ | ^ ^ ^ |
+ | | | | |
+ | +---------+ +---------+ +---------+ |
+ | | PCH-LPC | | Devices | | Devices | |
+ | +---------+ +---------+ +---------+ |
+ | ^ |
+ | | |
+ | +---------+ |
+ | | Devices | |
+ | +---------+ |
+ | |
+ | |
+ +--------------------------------------------------------+
+
+ACPI-related definitions
+========================
+
+CPUINTC::
+
+ ACPI_MADT_TYPE_CORE_PIC;
+ struct acpi_madt_core_pic;
+ enum acpi_madt_core_pic_version;
+
+LIOINTC::
+
+ ACPI_MADT_TYPE_LIO_PIC;
+ struct acpi_madt_lio_pic;
+ enum acpi_madt_lio_pic_version;
+
+EIOINTC::
+
+ ACPI_MADT_TYPE_EIO_PIC;
+ struct acpi_madt_eio_pic;
+ enum acpi_madt_eio_pic_version;
+
+HTVECINTC::
+
+ ACPI_MADT_TYPE_HT_PIC;
+ struct acpi_madt_ht_pic;
+ enum acpi_madt_ht_pic_version;
+
+PCH-PIC::
+
+ ACPI_MADT_TYPE_BIO_PIC;
+ struct acpi_madt_bio_pic;
+ enum acpi_madt_bio_pic_version;
+
+PCH-MSI::
+
+ ACPI_MADT_TYPE_MSI_PIC;
+ struct acpi_madt_msi_pic;
+ enum acpi_madt_msi_pic_version;
+
+PCH-LPC::
+
+ ACPI_MADT_TYPE_LPC_PIC;
+ struct acpi_madt_lpc_pic;
+ enum acpi_madt_lpc_pic_version;
+
+References
+==========
+
+Documentation of Loongson-3A5000:
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-3A5000-usermanual-1.02-CN.pdf (in Chinese)
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-3A5000-usermanual-1.02-EN.pdf (in English)
+
+Documentation of Loongson's LS7A chipset:
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-7A1000-usermanual-2.00-CN.pdf (in Chinese)
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-7A1000-usermanual-2.00-EN.pdf (in English)
+
+Note: CPUINTC is CSR.ECFG/CSR.ESTAT and its interrupt controller described
+in Section 7.4 of "LoongArch Reference Manual, Vol 1"; LIOINTC is "Legacy I/O
+Interrupts" described in Section 11.1 of "Loongson 3A5000 Processor Reference
+Manual"; EIOINTC is "Extended I/O Interrupts" described in Section 11.2 of
+"Loongson 3A5000 Processor Reference Manual"; HTVECINTC is "HyperTransport
+Interrupts" described in Section 14.3 of "Loongson 3A5000 Processor Reference
+Manual"; PCH-PIC/PCH-MSI is "Interrupt Controller" described in Section 5 of
+"Loongson 7A1000 Bridge User Manual"; PCH-LPC is "LPC Interrupts" described in
+Section 24.3 of "Loongson 7A1000 Bridge User Manual".
--
2.27.0
On 6/1/22 17:59, Huacai Chen wrote:
> <snip>
>
> Huacai Chen(24):
> irqchip: Adjust Kconfig for Loongson.
> irqchip/loongson-liointc: Fix build error for LoongArch.
> Documentation: LoongArch: Add basic documentations.
> Documentation/zh_CN: Add basic LoongArch documentations.
> LoongArch: Add elf-related definitions.
> LoongArch: Add writecombine support for drm.
> LoongArch: Add build infrastructure.
> LoongArch: Add CPU definition headers.
> LoongArch: Add atomic/locking headers.
> LoongArch: Add other common headers.
> LoongArch: Add boot and setup routines.
> LoongArch: Add exception/interrupt handling.
> LoongArch: Add process management.
> LoongArch: Add memory management.
> LoongArch: Add system call support.
> LoongArch: Add signal handling support.
> LoongArch: Add elf and module support.
> LoongArch: Add misc common routines.
> LoongArch: Add some library functions.
> LoongArch: Add PCI controller support.
> LoongArch: Add VDSO and VSYSCALL support.
> LoongArch: Add multi-processor (SMP) support.
> LoongArch: Add Non-Uniform Memory Access (NUMA) support.
> LoongArch: Add Loongson-3 default config file.
>
> Signed-off-by: Huacai Chen <[email protected]>
I have gone through every of these patches, and they now all look good
to me. Thanks for all the revisions and cooperation!
I remember several more people have left comments on the v11 code; Eric
commented on the signal handling bits and Ard may have something to say
on the UEFI part. The first two cleanup patches to irqchip are
previously sent in a separate series, and they are already trivial
enough, but Marc may want to give the ack here too, for the patches to
go in via the asm-generic tree.
At this time, although the port currently cannot work on any real
hardware, the value is mostly for unblocking downstream development
starting from the libc's, as the userspace ABI has been stable for
several months already and there's probably no point in delaying even
further. I've been working on the Gentoo/LoongArch port since August
2021 and experienced every ABI break since then, so I probably know
enough to make this statement.
I think, with the userspace ABI effectively stable already, and other
implementation details available for continued improvement, we could
make it into this merge window after all. People, please take a final
quick look at this; your opinions and/or review and acks are appreciated.
On Wed, Jun 1, 2022 at 8:44 PM Jiaxun Yang <[email protected]> wrote:
>
>
>
> 在2022年6月1日六月 上午10:59，Huacai Chen写道：
> > LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V.
> > LoongArch includes a reduced 32-bit version (LA32R), a standard 32-bit
> > version (LA32S) and a 64-bit version (LA64). LoongArch use ACPI as its
> > boot protocol LoongArch-specific interrupt controllers (similar to APIC)
> > are already added in the next revision of ACPI Specification (current
> > revision is 6.4).
> >
>
> I’ve been reviewing all LA changes in past week and now I’m giving out R-b
> for every patch I had reviewed in detail. (I don’t really now anything about
> mm and processes so I just leave them).
>
> I also tried to run the kernel on my machine with Huacai’s next tree and
> Xuerui’s BPI patch.
>
> I watched the “New World” of LoongArch grow up from scratch. And I must
> say it’s a epic work showing the collaboration between community and Loongson
> company. Especially Xuerui who invested numerous days and nights without any
> return.
>
> Thanks to all people involved.
Great job, best wishes to all of you.
>
> - Jiaxun
>
--
Best Regards
Guo Ren
ML: https://lore.kernel.org/linux-csky/
Add VDSO and VSYSCALL support (sigreturn, gettimeofday and its friends)
for LoongArch.
Reviewed-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/vdso.h | 38 +++++
arch/loongarch/include/asm/vdso/clocksource.h | 8 +
.../loongarch/include/asm/vdso/gettimeofday.h | 99 +++++++++++++
arch/loongarch/include/asm/vdso/processor.h | 14 ++
arch/loongarch/include/asm/vdso/vdso.h | 30 ++++
arch/loongarch/include/asm/vdso/vsyscall.h | 27 ++++
arch/loongarch/kernel/time.c | 1 +
arch/loongarch/kernel/vdso.c | 138 ++++++++++++++++++
arch/loongarch/vdso/Makefile | 96 ++++++++++++
arch/loongarch/vdso/elf.S | 15 ++
arch/loongarch/vdso/gen_vdso_offsets.sh | 13 ++
arch/loongarch/vdso/sigreturn.S | 24 +++
arch/loongarch/vdso/vdso.S | 22 +++
arch/loongarch/vdso/vdso.lds.S | 72 +++++++++
arch/loongarch/vdso/vgettimeofday.c | 25 ++++
15 files changed, 622 insertions(+)
create mode 100644 arch/loongarch/include/asm/vdso.h
create mode 100644 arch/loongarch/include/asm/vdso/clocksource.h
create mode 100644 arch/loongarch/include/asm/vdso/gettimeofday.h
create mode 100644 arch/loongarch/include/asm/vdso/processor.h
create mode 100644 arch/loongarch/include/asm/vdso/vdso.h
create mode 100644 arch/loongarch/include/asm/vdso/vsyscall.h
create mode 100644 arch/loongarch/kernel/vdso.c
create mode 100644 arch/loongarch/vdso/Makefile
create mode 100644 arch/loongarch/vdso/elf.S
create mode 100755 arch/loongarch/vdso/gen_vdso_offsets.sh
create mode 100644 arch/loongarch/vdso/sigreturn.S
create mode 100644 arch/loongarch/vdso/vdso.S
create mode 100644 arch/loongarch/vdso/vdso.lds.S
create mode 100644 arch/loongarch/vdso/vgettimeofday.c
diff --git a/arch/loongarch/include/asm/vdso.h b/arch/loongarch/include/asm/vdso.h
new file mode 100644
index 000000000000..8f8a0f9a4953
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASM_VDSO_H
+#define __ASM_VDSO_H
+
+#include <linux/mm_types.h>
+#include <vdso/datapage.h>
+
+#include <asm/barrier.h>
+
+/*
+ * struct loongarch_vdso_info - Details of a VDSO image.
+ * @vdso: Pointer to VDSO image (page-aligned).
+ * @size: Size of the VDSO image (page-aligned).
+ * @off_rt_sigreturn: Offset of the rt_sigreturn() trampoline.
+ * @code_mapping: Special mapping structure for vdso code.
+ * @code_mapping: Special mapping structure for vdso data.
+ *
+ * This structure contains details of a VDSO image, including the image data
+ * and offsets of certain symbols required by the kernel. It is generated as
+ * part of the VDSO build process, aside from the mapping page array, which is
+ * populated at runtime.
+ */
+struct loongarch_vdso_info {
+ void *vdso;
+ unsigned long size;
+ unsigned long offset_sigreturn;
+ struct vm_special_mapping code_mapping;
+ struct vm_special_mapping data_mapping;
+};
+
+extern struct loongarch_vdso_info vdso_info;
+
+#endif /* __ASM_VDSO_H */
diff --git a/arch/loongarch/include/asm/vdso/clocksource.h b/arch/loongarch/include/asm/vdso/clocksource.h
new file mode 100644
index 000000000000..13cd580d406d
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso/clocksource.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef __ASM_VDSOCLOCKSOURCE_H
+#define __ASM_VDSOCLOCKSOURCE_H
+
+#define VDSO_ARCH_CLOCKMODES \
+ VDSO_CLOCKMODE_CPU
+
+#endif /* __ASM_VDSOCLOCKSOURCE_H */
diff --git a/arch/loongarch/include/asm/vdso/gettimeofday.h b/arch/loongarch/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..7b2cd37641e2
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,99 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <asm/vdso/vdso.h>
+
+#define VDSO_HAS_CLOCK_GETRES 1
+
+static __always_inline long gettimeofday_fallback(
+ struct __kernel_old_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct __kernel_old_timeval *tv asm("a0") = _tv;
+ register struct timezone *tz asm("a1") = _tz;
+ register long nr asm("a7") = __NR_gettimeofday;
+ register long ret asm("a0");
+
+ asm volatile(
+ " syscall 0\n"
+ : "+r" (ret)
+ : "r" (nr), "r" (tv), "r" (tz)
+ : "$t0", "$t1", "$t2", "$t3", "$t4", "$t5", "$t6", "$t7",
+ "$t8", "memory");
+
+ return ret;
+}
+
+static __always_inline long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __kernel_timespec *_ts)
+{
+ register clockid_t clkid asm("a0") = _clkid;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
+ register long nr asm("a7") = __NR_clock_gettime;
+ register long ret asm("a0");
+
+ asm volatile(
+ " syscall 0\n"
+ : "+r" (ret)
+ : "r" (nr), "r" (clkid), "r" (ts)
+ : "$t0", "$t1", "$t2", "$t3", "$t4", "$t5", "$t6", "$t7",
+ "$t8", "memory");
+
+ return ret;
+}
+
+static __always_inline int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __kernel_timespec *_ts)
+{
+ register clockid_t clkid asm("a0") = _clkid;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
+ register long nr asm("a7") = __NR_clock_getres;
+ register long ret asm("a0");
+
+ asm volatile(
+ " syscall 0\n"
+ : "+r" (ret)
+ : "r" (nr), "r" (clkid), "r" (ts)
+ : "$t0", "$t1", "$t2", "$t3", "$t4", "$t5", "$t6", "$t7",
+ "$t8", "memory");
+
+ return ret;
+}
+
+static __always_inline u64 __arch_get_hw_counter(s32 clock_mode,
+ const struct vdso_data *vd)
+{
+ uint64_t count;
+
+ __asm__ __volatile__(
+ " rdtime.d %0, $zero\n"
+ : "=r" (count));
+
+ return count;
+}
+
+static inline bool loongarch_vdso_hres_capable(void)
+{
+ return true;
+}
+#define __arch_vdso_hres_capable loongarch_vdso_hres_capable
+
+static __always_inline const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return get_vdso_data();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/loongarch/include/asm/vdso/processor.h b/arch/loongarch/include/asm/vdso/processor.h
new file mode 100644
index 000000000000..ef5770b343a0
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso/processor.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_VDSO_PROCESSOR_H
+#define __ASM_VDSO_PROCESSOR_H
+
+#ifndef __ASSEMBLY__
+
+#define cpu_relax() barrier()
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_PROCESSOR_H */
diff --git a/arch/loongarch/include/asm/vdso/vdso.h b/arch/loongarch/include/asm/vdso/vdso.h
new file mode 100644
index 000000000000..5a01643a65b3
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso/vdso.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASSEMBLY__
+
+#include <asm/asm.h>
+#include <asm/page.h>
+
+static inline unsigned long get_vdso_base(void)
+{
+ unsigned long addr;
+
+ __asm__(
+ " la.pcrel %0, _start\n"
+ : "=r" (addr)
+ :
+ :);
+
+ return addr;
+}
+
+static inline const struct vdso_data *get_vdso_data(void)
+{
+ return (const struct vdso_data *)(get_vdso_base() - PAGE_SIZE);
+}
+
+#endif /* __ASSEMBLY__ */
diff --git a/arch/loongarch/include/asm/vdso/vsyscall.h b/arch/loongarch/include/asm/vdso/vsyscall.h
new file mode 100644
index 000000000000..5de615383a22
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso/vsyscall.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ASM_VDSO_VSYSCALL_H
+#define __ASM_VDSO_VSYSCALL_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/timekeeper_internal.h>
+#include <vdso/datapage.h>
+
+extern struct vdso_data *vdso_data;
+
+/*
+ * Update the vDSO data page to keep in sync with kernel timekeeping.
+ */
+static __always_inline
+struct vdso_data *__loongarch_get_k_vdso_data(void)
+{
+ return vdso_data;
+}
+#define __arch_get_k_vdso_data __loongarch_get_k_vdso_data
+
+/* The asm-generic header needs to be included after the definitions above */
+#include <asm-generic/vdso/vsyscall.h>
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_VSYSCALL_H */
diff --git a/arch/loongarch/kernel/time.c b/arch/loongarch/kernel/time.c
index b2bb14844f01..fe6823875895 100644
--- a/arch/loongarch/kernel/time.c
+++ b/arch/loongarch/kernel/time.c
@@ -183,6 +183,7 @@ static struct clocksource clocksource_const = {
.read = read_const_counter,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
+ .vdso_clock_mode = VDSO_CLOCKMODE_CPU,
};
int __init constant_clocksource_init(void)
diff --git a/arch/loongarch/kernel/vdso.c b/arch/loongarch/kernel/vdso.c
new file mode 100644
index 000000000000..e20c8ca87473
--- /dev/null
+++ b/arch/loongarch/kernel/vdso.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/binfmts.h>
+#include <linux/elf.h>
+#include <linux/err.h>
+#include <linux/init.h>
+#include <linux/ioport.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/random.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/timekeeper_internal.h>
+
+#include <asm/page.h>
+#include <asm/vdso.h>
+#include <vdso/helpers.h>
+#include <vdso/vsyscall.h>
+#include <generated/vdso-offsets.h>
+
+extern char vdso_start[], vdso_end[];
+
+/* Kernel-provided data used by the VDSO. */
+static union loongarch_vdso_data {
+ u8 page[PAGE_SIZE];
+ struct vdso_data data[CS_BASES];
+} loongarch_vdso_data __page_aligned_data;
+struct vdso_data *vdso_data = loongarch_vdso_data.data;
+static struct page *vdso_pages[] = { NULL };
+
+static int vdso_mremap(const struct vm_special_mapping *sm, struct vm_area_struct *new_vma)
+{
+ current->mm->context.vdso = (void *)(new_vma->vm_start);
+
+ return 0;
+}
+
+struct loongarch_vdso_info vdso_info = {
+ .vdso = vdso_start,
+ .size = PAGE_SIZE,
+ .code_mapping = {
+ .name = "[vdso]",
+ .pages = vdso_pages,
+ .mremap = vdso_mremap,
+ },
+ .data_mapping = {
+ .name = "[vvar]",
+ },
+ .offset_sigreturn = vdso_offset_sigreturn,
+};
+
+static int __init init_vdso(void)
+{
+ unsigned long i, pfn;
+
+ BUG_ON(!PAGE_ALIGNED(vdso_info.vdso));
+ BUG_ON(!PAGE_ALIGNED(vdso_info.size));
+
+ pfn = __phys_to_pfn(__pa_symbol(vdso_info.vdso));
+ for (i = 0; i < vdso_info.size / PAGE_SIZE; i++)
+ vdso_info.code_mapping.pages[i] = pfn_to_page(pfn + i);
+
+ return 0;
+}
+subsys_initcall(init_vdso);
+
+static unsigned long vdso_base(void)
+{
+ unsigned long base = STACK_TOP;
+
+ if (current->flags & PF_RANDOMIZE) {
+ base += get_random_int() & (VDSO_RANDOMIZE_SIZE - 1);
+ base = PAGE_ALIGN(base);
+ }
+
+ return base;
+}
+
+int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
+{
+ int ret;
+ unsigned long vvar_size, size, data_addr, vdso_addr;
+ struct mm_struct *mm = current->mm;
+ struct vm_area_struct *vma;
+ struct loongarch_vdso_info *info = current->thread.vdso;
+
+ if (mmap_write_lock_killable(mm))
+ return -EINTR;
+
+ /*
+ * Determine total area size. This includes the VDSO data itself
+ * and the data page.
+ */
+ vvar_size = PAGE_SIZE;
+ size = vvar_size + info->size;
+
+ data_addr = get_unmapped_area(NULL, vdso_base(), size, 0, 0);
+ if (IS_ERR_VALUE(data_addr)) {
+ ret = data_addr;
+ goto out;
+ }
+ vdso_addr = data_addr + PAGE_SIZE;
+
+ vma = _install_special_mapping(mm, data_addr, vvar_size,
+ VM_READ | VM_MAYREAD,
+ &info->data_mapping);
+ if (IS_ERR(vma)) {
+ ret = PTR_ERR(vma);
+ goto out;
+ }
+
+ /* Map VDSO data page. */
+ ret = remap_pfn_range(vma, data_addr,
+ virt_to_phys(vdso_data) >> PAGE_SHIFT,
+ PAGE_SIZE, PAGE_READONLY);
+ if (ret)
+ goto out;
+
+ /* Map VDSO code page. */
+ vma = _install_special_mapping(mm, vdso_addr, info->size,
+ VM_READ | VM_EXEC | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC,
+ &info->code_mapping);
+ if (IS_ERR(vma)) {
+ ret = PTR_ERR(vma);
+ goto out;
+ }
+
+ mm->context.vdso = (void *)vdso_addr;
+ ret = 0;
+
+out:
+ mmap_write_unlock(mm);
+ return ret;
+}
diff --git a/arch/loongarch/vdso/Makefile b/arch/loongarch/vdso/Makefile
new file mode 100644
index 000000000000..6b6e16732c60
--- /dev/null
+++ b/arch/loongarch/vdso/Makefile
@@ -0,0 +1,96 @@
+# SPDX-License-Identifier: GPL-2.0
+# Objects to go into the VDSO.
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_LARCH_32|R_LARCH_64|R_LARCH_MARK_LA|R_LARCH_JUMP_SLOT
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso-y := elf.o vgettimeofday.o sigreturn.o
+
+# Common compiler flags between ABIs.
+ccflags-vdso := \
+ $(filter -I%,$(KBUILD_CFLAGS)) \
+ $(filter -E%,$(KBUILD_CFLAGS)) \
+ $(filter -march=%,$(KBUILD_CFLAGS)) \
+ $(filter -m%-float,$(KBUILD_CFLAGS)) \
+ -D__VDSO__
+
+ifeq ($(cc-name),clang)
+ccflags-vdso += $(filter --target=%,$(KBUILD_CFLAGS))
+endif
+
+cflags-vdso := $(ccflags-vdso) \
+ $(filter -W%,$(filter-out -Wa$(comma)%,$(KBUILD_CFLAGS))) \
+ -O2 -g -fno-strict-aliasing -fno-common -fno-builtin -G0 \
+ -fno-stack-protector -fno-jump-tables -DDISABLE_BRANCH_PROFILING \
+ $(call cc-option, -fno-asynchronous-unwind-tables) \
+ $(call cc-option, -fno-stack-protector)
+aflags-vdso := $(ccflags-vdso) \
+ -D__ASSEMBLY__ -Wa,-gdwarf-2
+
+ifneq ($(c-gettimeofday-y),)
+ CFLAGS_vgettimeofday.o += -include $(c-gettimeofday-y)
+endif
+
+# VDSO linker flags.
+ldflags-y := -Bsymbolic --no-undefined -soname=linux-vdso.so.1 \
+ $(filter -E%,$(KBUILD_CFLAGS)) -nostdlib -shared \
+ --hash-style=sysv --build-id -T
+
+GCOV_PROFILE := n
+
+#
+# Shared build commands.
+#
+
+quiet_cmd_vdsold_and_vdso_check = LD $@
+ cmd_vdsold_and_vdso_check = $(cmd_ld); $(cmd_vdso_check)
+
+quiet_cmd_vdsoas_o_S = AS $@
+ cmd_vdsoas_o_S = $(CC) $(a_flags) -c -o $@ $<
+
+# Generate VDSO offsets using helper script
+gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh
+quiet_cmd_vdsosym = VDSOSYM $@
+ cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
+
+include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,vdsosym)
+
+#
+# Build native VDSO.
+#
+
+native-abi := $(filter -mabi=%,$(KBUILD_CFLAGS))
+
+targets += $(obj-vdso-y)
+targets += vdso.lds vdso.so.dbg vdso.so
+
+obj-vdso := $(obj-vdso-y:%.o=$(obj)/%.o)
+
+$(obj-vdso): KBUILD_CFLAGS := $(cflags-vdso) $(native-abi)
+$(obj-vdso): KBUILD_AFLAGS := $(aflags-vdso) $(native-abi)
+
+$(obj)/vdso.lds: KBUILD_CPPFLAGS := $(ccflags-vdso) $(native-abi)
+
+$(obj)/vdso.so.dbg: $(obj)/vdso.lds $(obj-vdso) FORCE
+ $(call if_changed,vdsold_and_vdso_check)
+
+$(obj)/vdso.so: OBJCOPYFLAGS := -S
+$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,objcopy)
+
+obj-y += vdso.o
+
+$(obj)/vdso.o : $(obj)/vdso.so
+
+# install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/[email protected] $(MODLIB)/vdso/$@
+
+vdso.so: $(obj)/vdso.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso_install: vdso.so
diff --git a/arch/loongarch/vdso/elf.S b/arch/loongarch/vdso/elf.S
new file mode 100644
index 000000000000..9bb21b9f9583
--- /dev/null
+++ b/arch/loongarch/vdso/elf.S
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/vdso/vdso.h>
+
+#include <linux/elfnote.h>
+#include <linux/version.h>
+
+ELFNOTE_START(Linux, 0, "a")
+ .long LINUX_VERSION_CODE
+ELFNOTE_END
diff --git a/arch/loongarch/vdso/gen_vdso_offsets.sh b/arch/loongarch/vdso/gen_vdso_offsets.sh
new file mode 100755
index 000000000000..1bb4e12642ff
--- /dev/null
+++ b/arch/loongarch/vdso/gen_vdso_offsets.sh
@@ -0,0 +1,13 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+
+#
+# Derived from RISC-V and ARM64:
+# Author: Will Deacon <[email protected]>
+#
+# Match symbols in the DSO that look like VDSO_*; produce a header file
+# of constant offsets into the shared object.
+#
+
+LC_ALL=C sed -n -e 's/^00*/0/' -e \
+'s/^\([0-9a-fA-F]*\) . VDSO_\([a-zA-Z0-9_]*\)$/\#define vdso_offset_\2\t0x\1/p'
diff --git a/arch/loongarch/vdso/sigreturn.S b/arch/loongarch/vdso/sigreturn.S
new file mode 100644
index 000000000000..9cb3c58fad03
--- /dev/null
+++ b/arch/loongarch/vdso/sigreturn.S
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/vdso/vdso.h>
+
+#include <linux/linkage.h>
+#include <uapi/asm/unistd.h>
+
+#include <asm/regdef.h>
+#include <asm/asm.h>
+
+ .section .text
+ .cfi_sections .debug_frame
+
+SYM_FUNC_START(__vdso_rt_sigreturn)
+
+ li.w a7, __NR_rt_sigreturn
+ syscall 0
+
+SYM_FUNC_END(__vdso_rt_sigreturn)
diff --git a/arch/loongarch/vdso/vdso.S b/arch/loongarch/vdso/vdso.S
new file mode 100644
index 000000000000..46789bade6ff
--- /dev/null
+++ b/arch/loongarch/vdso/vdso.S
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from RISC-V:
+ * Copyright (C) 2014 Regents of the University of California
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <asm/page.h>
+
+ __PAGE_ALIGNED_DATA
+
+ .globl vdso_start, vdso_end
+ .balign PAGE_SIZE
+vdso_start:
+ .incbin "arch/loongarch/vdso/vdso.so"
+ .balign PAGE_SIZE
+vdso_end:
+
+ .previous
diff --git a/arch/loongarch/vdso/vdso.lds.S b/arch/loongarch/vdso/vdso.lds.S
new file mode 100644
index 000000000000..955f02de4a2d
--- /dev/null
+++ b/arch/loongarch/vdso/vdso.lds.S
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+OUTPUT_FORMAT("elf64-loongarch", "elf64-loongarch", "elf64-loongarch")
+
+OUTPUT_ARCH(loongarch)
+
+SECTIONS
+{
+ PROVIDE(_start = .);
+ . = SIZEOF_HEADERS;
+
+ .hash : { *(.hash) } :text
+ .gnu.hash : { *(.gnu.hash) }
+ .dynsym : { *(.dynsym) }
+ .dynstr : { *(.dynstr) }
+ .gnu.version : { *(.gnu.version) }
+ .gnu.version_d : { *(.gnu.version_d) }
+ .gnu.version_r : { *(.gnu.version_r) }
+
+ .note : { *(.note.*) } :text :note
+
+ .text : { *(.text*) } :text
+ PROVIDE (__etext = .);
+ PROVIDE (_etext = .);
+ PROVIDE (etext = .);
+
+ .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr
+ .eh_frame : { KEEP (*(.eh_frame)) } :text
+
+ .dynamic : { *(.dynamic) } :text :dynamic
+
+ .rodata : { *(.rodata*) } :text
+
+ _end = .;
+ PROVIDE(end = .);
+
+ /DISCARD/ : {
+ *(.gnu.attributes)
+ *(.note.GNU-stack)
+ *(.data .data.* .gnu.linkonce.d.* .sdata*)
+ *(.bss .sbss .dynbss .dynsbss)
+ }
+}
+
+PHDRS
+{
+ text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
+ dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
+ note PT_NOTE FLAGS(4); /* PF_R */
+ eh_frame_hdr PT_GNU_EH_FRAME;
+}
+
+VERSION
+{
+ LINUX_5.10 {
+ global:
+ __vdso_clock_getres;
+ __vdso_clock_gettime;
+ __vdso_gettimeofday;
+ __vdso_rt_sigreturn;
+ local: *;
+ };
+}
+
+/*
+ * Make the sigreturn code visible to the kernel.
+ */
+VDSO_sigreturn = __vdso_rt_sigreturn;
diff --git a/arch/loongarch/vdso/vgettimeofday.c b/arch/loongarch/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..b1f4548dae92
--- /dev/null
+++ b/arch/loongarch/vdso/vgettimeofday.c
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * LoongArch userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/types.h>
+
+int __vdso_clock_gettime(clockid_t clock,
+ struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+int __vdso_clock_getres(clockid_t clock_id,
+ struct __kernel_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
--
2.27.0
在2022年6月1日六月 上午10:59，Huacai Chen写道：
> Add some basic documentation for LoongArch. LoongArch is a new RISC ISA,
> which is a bit like MIPS or RISC-V. LoongArch includes a reduced 32-bit
> version (LA32R), a standard 32-bit version (LA32S) and a 64-bit version
> (LA64).
>
> Co-developed-by: WANG Xuerui <[email protected]>
> Signed-off-by: WANG Xuerui <[email protected]>
> Signed-off-by: Huacai Chen <[email protected]>
Reviewed-by: Jiaxun Yang <[email protected]>
Did proofread, perfect!
Thanks.
> ---
> Documentation/arch.rst | 1 +
> Documentation/loongarch/features.rst | 3 +
> Documentation/loongarch/index.rst | 21 ++
> Documentation/loongarch/introduction.rst | 387 +++++++++++++++++++++
> Documentation/loongarch/irq-chip-model.rst | 168 +++++++++
> 5 files changed, 580 insertions(+)
> create mode 100644 Documentation/loongarch/features.rst
> create mode 100644 Documentation/loongarch/index.rst
> create mode 100644 Documentation/loongarch/introduction.rst
> create mode 100644 Documentation/loongarch/irq-chip-model.rst
>
> diff --git a/Documentation/arch.rst b/Documentation/arch.rst
> index 14bcd8294b93..41a66a8b38e4 100644
> --- a/Documentation/arch.rst
> +++ b/Documentation/arch.rst
> @@ -13,6 +13,7 @@ implementation.
> arm/index
> arm64/index
> ia64/index
> + loongarch/index
> m68k/index
> mips/index
> nios2/index
> diff --git a/Documentation/loongarch/features.rst
> b/Documentation/loongarch/features.rst
> new file mode 100644
> index 000000000000..ebacade3ea45
> --- /dev/null
> +++ b/Documentation/loongarch/features.rst
> @@ -0,0 +1,3 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +.. kernel-feat:: $srctree/Documentation/features loongarch
> diff --git a/Documentation/loongarch/index.rst
> b/Documentation/loongarch/index.rst
> new file mode 100644
> index 000000000000..aaba648db907
> --- /dev/null
> +++ b/Documentation/loongarch/index.rst
> @@ -0,0 +1,21 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +======================
> +LoongArch Architecture
> +======================
> +
> +.. toctree::
> + :maxdepth: 2
> + :numbered:
> +
> + introduction
> + irq-chip-model
> +
> + features
> +
> +.. only:: subproject and html
> +
> + Indices
> + =======
> +
> + * :ref:`genindex`
> diff --git a/Documentation/loongarch/introduction.rst
> b/Documentation/loongarch/introduction.rst
> new file mode 100644
> index 000000000000..b4df5b459677
> --- /dev/null
> +++ b/Documentation/loongarch/introduction.rst
> @@ -0,0 +1,387 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=========================
> +Introduction to LoongArch
> +=========================
> +
> +LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V. There
> are
> +currently 3 variants: a reduced 32-bit version (LA32R), a standard
> 32-bit
> +version (LA32S) and a 64-bit version (LA64). There are 4 privilege
> levels
> +(PLVs) defined in LoongArch: PLV0~PLV3, from high to low. Kernel runs
> at PLV0
> +while applications run at PLV3. This document introduces the
> registers, basic
> +instruction set, virtual memory and some other topics of LoongArch.
> +
> +Registers
> +=========
> +
> +LoongArch registers include general purpose registers (GPRs), floating
> point
> +registers (FPRs), vector registers (VRs) and control status registers
> (CSRs)
> +used in privileged mode (PLV0).
> +
> +GPRs
> +----
> +
> +LoongArch has 32 GPRs ( ``$r0`` ~ ``$r31`` ); each one is 32-bit wide
> in LA32
> +and 64-bit wide in LA64. ``$r0`` is hard-wired to zero, and the other
> registers
> +are not architecturally special. (Except ``$r1``, which is hard-wired
> as the
> +link register of the BL instruction.)
> +
> +The kernel uses a variant of the LoongArch register convention, as
> described in
> +the LoongArch ELF psABI spec, in :ref:`References
> <loongarch-references>`:
> +
> +================= =============== =================== ============
> +Name Alias Usage Preserved
> + across calls
> +================= =============== =================== ============
> +``$r0`` ``$zero`` Constant zero Unused
> +``$r1`` ``$ra`` Return address No
> +``$r2`` ``$tp`` TLS/Thread pointer Unused
> +``$r3`` ``$sp`` Stack pointer Yes
> +``$r4``-``$r11`` ``$a0``-``$a7`` Argument registers No
> +``$r4``-``$r5`` ``$v0``-``$v1`` Return value No
> +``$r12``-``$r20`` ``$t0``-``$t8`` Temp registers No
> +``$r21`` ``$u0`` Percpu base address Unused
> +``$r22`` ``$fp`` Frame pointer Yes
> +``$r23``-``$r31`` ``$s0``-``$s8`` Static registers Yes
> +================= =============== =================== ============
> +
> +Note: The register ``$r21`` is reserved in the ELF psABI, but used by
> the Linux
> +kernel for storing the percpu base address. It normally has no ABI
> name, but is
> +called ``$u0`` in the kernel. You may also see ``$v0`` or ``$v1`` in
> some old code,
> +they are deprecated aliases of ``$a0`` and ``$a1`` respectively.
> +
> +FPRs
> +----
> +
> +LoongArch has 32 FPRs ( ``$f0`` ~ ``$f31`` ) when FPU is present. Each
> one is
> +64-bit wide on the LA64 cores.
> +
> +The floating-point register convention is the same as described in the
> +LoongArch ELF psABI spec:
> +
> +================= ================== =================== ============
> +Name Alias Usage Preserved
> + across calls
> +================= ================== =================== ============
> +``$f0``-``$f7`` ``$fa0``-``$fa7`` Argument registers No
> +``$f0``-``$f1`` ``$fv0``-``$fv1`` Return value No
> +``$f8``-``$f23`` ``$ft0``-``$ft15`` Temp registers No
> +``$f24``-``$f31`` ``$fs0``-``$fs7`` Static registers Yes
> +================= ================== =================== ============
> +
> +Note: You may see ``$fv0`` or ``$fv1`` in some old code, they are
> deprecated
> +aliases of ``$fa0`` and ``$fa1`` respectively.
> +
> +VRs
> +----
> +
> +There are currently 2 vector extensions to LoongArch:
> +
> +- LSX (Loongson SIMD eXtension) with 128-bit vectors,
> +- LASX (Loongson Advanced SIMD eXtension) with 256-bit vectors.
> +
> +LSX brings ``$v0`` ~ ``$v31`` while LASX brings ``$x0`` ~ ``$x31`` as
> the vector
> +registers.
> +
> +The VRs overlap with FPRs: for example, on a core implementing LSX and
> LASX,
> +the lower 128 bits of ``$x0`` is shared with ``$v0``, and the lower 64
> bits of
> +``$v0`` is shared with ``$f0``; same with all other VRs.
> +
> +CSRs
> +----
> +
> +CSRs can only be accessed from privileged mode (PLV0):
> +
> +================= ===================================== ==============
> +Address Full Name Abbrev Name
> +================= ===================================== ==============
> +0x0 Current Mode Information CRMD
> +0x1 Pre-exception Mode Information PRMD
> +0x2 Extension Unit Enable EUEN
> +0x3 Miscellaneous Control MISC
> +0x4 Exception Configuration ECFG
> +0x5 Exception Status ESTAT
> +0x6 Exception Return Address ERA
> +0x7 Bad (Faulting) Virtual Address BADV
> +0x8 Bad (Faulting) Instruction Word BADI
> +0xC Exception Entrypoint Address EENTRY
> +0x10 TLB Index TLBIDX
> +0x11 TLB Entry High-order Bits TLBEHI
> +0x12 TLB Entry Low-order Bits 0 TLBELO0
> +0x13 TLB Entry Low-order Bits 1 TLBELO1
> +0x18 Address Space Identifier ASID
> +0x19 Page Global Directory Address for PGDL
> + Lower-half Address Space
> +0x1A Page Global Directory Address for PGDH
> + Higher-half Address Space
> +0x1B Page Global Directory Address PGD
> +0x1C Page Walk Control for Lower- PWCL
> + half Address Space
> +0x1D Page Walk Control for Higher- PWCH
> + half Address Space
> +0x1E STLB Page Size STLBPS
> +0x1F Reduced Virtual Address Configuration RVACFG
> +0x20 CPU Identifier CPUID
> +0x21 Privileged Resource Configuration 1 PRCFG1
> +0x22 Privileged Resource Configuration 2 PRCFG2
> +0x23 Privileged Resource Configuration 3 PRCFG3
> +0x30+n (0≤n≤15) Saved Data register SAVEn
> +0x40 Timer Identifier TID
> +0x41 Timer Configuration TCFG
> +0x42 Timer Value TVAL
> +0x43 Compensation of Timer Count CNTC
> +0x44 Timer Interrupt Clearing TICLR
> +0x60 LLBit Control LLBCTL
> +0x80 Implementation-specific Control 1 IMPCTL1
> +0x81 Implementation-specific Control 2 IMPCTL2
> +0x88 TLB Refill Exception Entrypoint TLBRENTRY
> + Address
> +0x89 TLB Refill Exception BAD (Faulting) TLBRBADV
> + Virtual Address
> +0x8A TLB Refill Exception Return Address TLBRERA
> +0x8B TLB Refill Exception Saved Data TLBRSAVE
> + Register
> +0x8C TLB Refill Exception Entry Low-order TLBRELO0
> + Bits 0
> +0x8D TLB Refill Exception Entry Low-order TLBRELO1
> + Bits 1
> +0x8E TLB Refill Exception Entry High-order TLBEHI
> + Bits
> +0x8F TLB Refill Exception Pre-exception TLBRPRMD
> + Mode Information
> +0x90 Machine Error Control MERRCTL
> +0x91 Machine Error Information 1 MERRINFO1
> +0x92 Machine Error Information 2 MERRINFO2
> +0x93 Machine Error Exception Entrypoint MERRENTRY
> + Address
> +0x94 Machine Error Exception Return MERRERA
> + Address
> +0x95 Machine Error Exception Saved Data MERRSAVE
> + Register
> +0x98 Cache TAGs CTAG
> +0x180+n (0≤n≤3) Direct Mapping Configuration Window n DMWn
> +0x200+2n (0≤n≤31) Performance Monitor Configuration n PMCFGn
> +0x201+2n (0≤n≤31) Performance Monitor Overall Counter n PMCNTn
> +0x300 Memory Load/Store WatchPoint MWPC
> + Overall Control
> +0x301 Memory Load/Store WatchPoint MWPS
> + Overall Status
> +0x310+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG1
> + Configuration 1
> +0x311+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG2
> + Configuration 2
> +0x312+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG3
> + Configuration 3
> +0x313+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG4
> + Configuration 4
> +0x380 Instruction Fetch WatchPoint FWPC
> + Overall Control
> +0x381 Instruction Fetch WatchPoint FWPS
> + Overall Status
> +0x390+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG1
> + Configuration 1
> +0x391+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG2
> + Configuration 2
> +0x392+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG3
> + Configuration 3
> +0x393+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG4
> + Configuration 4
> +0x500 Debug Register DBG
> +0x501 Debug Exception Return Address DERA
> +0x502 Debug Exception Saved Data Register DSAVE
> +================= ===================================== ==============
> +
> +ERA, TLBRERA, MERRERA and DERA are sometimes also known as EPC,
> TLBREPC, MERREPC
> +and DEPC respectively.
> +
> +Basic Instruction Set
> +=====================
> +
> +Instruction formats
> +-------------------
> +
> +LoongArch instructions are 32 bits wide, belonging to 9 basic
> instruction
> +formats (and variants of them):
> +
> +=========== ==========================
> +Format name Composition
> +=========== ==========================
> +2R Opcode + Rj + Rd
> +3R Opcode + Rk + Rj + Rd
> +4R Opcode + Ra + Rk + Rj + Rd
> +2RI8 Opcode + I8 + Rj + Rd
> +2RI12 Opcode + I12 + Rj + Rd
> +2RI14 Opcode + I14 + Rj + Rd
> +2RI16 Opcode + I16 + Rj + Rd
> +1RI21 Opcode + I21L + Rj + I21H
> +I26 Opcode + I26L + I26H
> +=========== ==========================
> +
> +Rd is the destination register operand, while Rj, Rk and Ra ("a"
> stands for
> +"additional") are the source register operands. I8/I12/I16/I21/I26 are
> +immediate operands of respective width. The longer I21 and I26 are
> stored
> +in separate higher and lower parts in the instruction word, denoted by
> the "L"
> +and "H" suffixes.
> +
> +List of Instructions
> +--------------------
> +
> +For brevity, only instruction names (mnemonics) are listed here;
> please see the
> +:ref:`References <loongarch-references>` for details.
> +
> +
> +1. Arithmetic Instructions::
> +
> + ADD.W SUB.W ADDI.W ADD.D SUB.D ADDI.D
> + SLT SLTU SLTI SLTUI
> + AND OR NOR XOR ANDN ORN ANDI ORI XORI
> + MUL.W MULH.W MULH.WU DIV.W DIV.WU MOD.W MOD.WU
> + MUL.D MULH.D MULH.DU DIV.D DIV.DU MOD.D MOD.DU
> + PCADDI PCADDU12I PCADDU18I
> + LU12I.W LU32I.D LU52I.D ADDU16I.D
> +
> +2. Bit-shift Instructions::
> +
> + SLL.W SRL.W SRA.W ROTR.W SLLI.W SRLI.W SRAI.W ROTRI.W
> + SLL.D SRL.D SRA.D ROTR.D SLLI.D SRLI.D SRAI.D ROTRI.D
> +
> +3. Bit-manipulation Instructions::
> +
> + EXT.W.B EXT.W.H CLO.W CLO.D SLZ.W CLZ.D CTO.W CTO.D CTZ.W CTZ.D
> + BYTEPICK.W BYTEPICK.D BSTRINS.W BSTRINS.D BSTRPICK.W BSTRPICK.D
> + REVB.2H REVB.4H REVB.2W REVB.D REVH.2W REVH.D BITREV.4B BITREV.8B
> BITREV.W BITREV.D
> + MASKEQZ MASKNEZ
> +
> +4. Branch Instructions::
> +
> + BEQ BNE BLT BGE BLTU BGEU BEQZ BNEZ B BL JIRL
> +
> +5. Load/Store Instructions::
> +
> + LD.B LD.BU LD.H LD.HU LD.W LD.WU LD.D ST.B ST.H ST.W ST.D
> + LDX.B LDX.BU LDX.H LDX.HU LDX.W LDX.WU LDX.D STX.B STX.H STX.W
> STX.D
> + LDPTR.W LDPTR.D STPTR.W STPTR.D
> + PRELD PRELDX
> +
> +6. Atomic Operation Instructions::
> +
> + LL.W SC.W LL.D SC.D
> + AMSWAP.W AMSWAP.D AMADD.W AMADD.D AMAND.W AMAND.D AMOR.W AMOR.D
> AMXOR.W AMXOR.D
> + AMMAX.W AMMAX.D AMMIN.W AMMIN.D
> +
> +7. Barrier Instructions::
> +
> + IBAR DBAR
> +
> +8. Special Instructions::
> +
> + SYSCALL BREAK CPUCFG NOP IDLE ERTN(ERET) DBCL(DBGCALL) RDTIMEL.W
> RDTIMEH.W RDTIME.D
> + ASRTLE.D ASRTGT.D
> +
> +9. Privileged Instructions::
> +
> + CSRRD CSRWR CSRXCHG
> + IOCSRRD.B IOCSRRD.H IOCSRRD.W IOCSRRD.D IOCSRWR.B IOCSRWR.H
> IOCSRWR.W IOCSRWR.D
> + CACOP TLBP(TLBSRCH) TLBRD TLBWR TLBFILL TLBCLR TLBFLUSH INVTLB
> LDDIR LDPTE
> +
> +Virtual Memory
> +==============
> +
> +LoongArch supports direct-mapped virtual memory and page-mapped
> virtual memory.
> +
> +Direct-mapped virtual memory is configured by CSR.DMWn (n=0~3), it has
> a simple
> +relationship between virtual address (VA) and physical address (PA)::
> +
> + VA = PA + FixedOffset
> +
> +Page-mapped virtual memory has arbitrary relationship between VA and
> PA, which
> +is recorded in TLB and page tables. LoongArch's TLB includes a
> fully-associative
> +MTLB (Multiple Page Size TLB) and set-associative STLB (Single Page
> Size TLB).
> +
> +By default, the whole virtual address space of LA32 is configured like
> this:
> +
> +============ =========================== =============================
> +Name Address Range Attributes
> +============ =========================== =============================
> +``UVRANGE`` ``0x00000000 - 0x7FFFFFFF`` Page-mapped, Cached, PLV0~3
> +``KPRANGE0`` ``0x80000000 - 0x9FFFFFFF`` Direct-mapped, Uncached, PLV0
> +``KPRANGE1`` ``0xA0000000 - 0xBFFFFFFF`` Direct-mapped, Cached, PLV0
> +``KVRANGE`` ``0xC0000000 - 0xFFFFFFFF`` Page-mapped, Cached, PLV0
> +============ =========================== =============================
> +
> +User mode (PLV3) can only access UVRANGE. For direct-mapped KPRANGE0
> and
> +KPRANGE1, PA is equal to VA with bit30~31 cleared. For example, the
> uncached
> +direct-mapped VA of 0x00001000 is 0x80001000, and the cached
> direct-mapped
> +VA of 0x00001000 is 0xA0001000.
> +
> +By default, the whole virtual address space of LA64 is configured like
> this:
> +
> +============ ======================
> ======================================
> +Name Address Range Attributes
> +============ ======================
> ======================================
> +``XUVRANGE`` ``0x0000000000000000 - Page-mapped, Cached, PLV0~3
> + 0x3FFFFFFFFFFFFFFF``
> +``XSPRANGE`` ``0x4000000000000000 - Direct-mapped, Cached / Uncached,
> PLV0
> + 0x7FFFFFFFFFFFFFFF``
> +``XKPRANGE`` ``0x8000000000000000 - Direct-mapped, Cached / Uncached,
> PLV0
> + 0xBFFFFFFFFFFFFFFF``
> +``XKVRANGE`` ``0xC000000000000000 - Page-mapped, Cached, PLV0
> + 0xFFFFFFFFFFFFFFFF``
> +============ ======================
> ======================================
> +
> +User mode (PLV3) can only access XUVRANGE. For direct-mapped XSPRANGE
> and
> +XKPRANGE, PA is equal to VA with bits 60~63 cleared, and the cache
> attribute
> +is configured by bits 60~61 in VA: 0 is for strongly-ordered uncached,
> 1 is
> +for coherent cached, and 2 is for weakly-ordered uncached.
> +
> +Currently we only use XKPRANGE for direct mapping and XSPRANGE is
> reserved.
> +
> +To put this in action: the strongly-ordered uncached direct-mapped VA
> (in
> +XKPRANGE) of 0x00000000_00001000 is 0x80000000_00001000, the coherent
> cached
> +direct-mapped VA (in XKPRANGE) of 0x00000000_00001000 is
> 0x90000000_00001000,
> +and the weakly-ordered uncached direct-mapped VA (in XKPRANGE) of
> 0x00000000
> +_00001000 is 0xA0000000_00001000.
> +
> +Relationship of Loongson and LoongArch
> +======================================
> +
> +LoongArch is a RISC ISA which is different from any other existing
> ones, while
> +Loongson is a family of processors. Loongson includes 3 series:
> Loongson-1 is
> +the 32-bit processor series, Loongson-2 is the low-end 64-bit
> processor series,
> +and Loongson-3 is the high-end 64-bit processor series. Old Loongson
> is based on
> +MIPS, while New Loongson is based on LoongArch. Take Loongson-3 as an
> example:
> +Loongson-3A1000/3B1500/3A2000/3A3000/3A4000 are MIPS-compatible, while
> Loongson-
> +3A5000 (and future revisions) are all based on LoongArch.
> +
> +.. _loongarch-references:
> +
> +References
> +==========
> +
> +Official web site of Loongson Technology Corp. Ltd.:
> +
> + http://www.loongson.cn/
> +
> +Developer web site of Loongson and LoongArch (Software and
> Documentation):
> +
> + http://www.loongnix.cn/
> +
> + https://github.com/loongson/
> +
> + https://loongson.github.io/LoongArch-Documentation/
> +
> +Documentation of LoongArch ISA:
> +
> +
> https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.00-CN.pdf
> (in Chinese)
> +
> +
> https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.00-EN.pdf
> (in English)
> +
> +Documentation of LoongArch ELF psABI:
> +
> +
> https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v1.00-CN.pdf
> (in Chinese)
> +
> +
> https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v1.00-EN.pdf
> (in English)
> +
> +Linux kernel repository of Loongson and LoongArch:
> +
> +
> https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git
> diff --git a/Documentation/loongarch/irq-chip-model.rst
> b/Documentation/loongarch/irq-chip-model.rst
> new file mode 100644
> index 000000000000..35c962991283
> --- /dev/null
> +++ b/Documentation/loongarch/irq-chip-model.rst
> @@ -0,0 +1,168 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=======================================
> +IRQ chip model (hierarchy) of LoongArch
> +=======================================
> +
> +Currently, LoongArch based processors (e.g. Loongson-3A5000) can only
> work together
> +with LS7A chipsets. The irq chips in LoongArch computers include
> CPUINTC (CPU Core
> +Interrupt Controller), LIOINTC (Legacy I/O Interrupt Controller),
> EIOINTC (Extended
> +I/O Interrupt Controller), HTVECINTC (Hyper-Transport Vector Interrupt
> Controller),
> +PCH-PIC (Main Interrupt Controller in LS7A chipset), PCH-LPC (LPC
> Interrupt Controller
> +in LS7A chipset) and PCH-MSI (MSI Interrupt Controller).
> +
> +CPUINTC is a per-core controller (in CPU), LIOINTC/EIOINTC/HTVECINTC
> are per-package
> +controllers (in CPU), while PCH-PIC/PCH-LPC/PCH-MSI are controllers
> out of CPU (i.e.,
> +in chipsets). These controllers (in other words, irqchips) are linked
> in a hierarchy,
> +and there are two models of hierarchy (legacy model and extended
> model).
> +
> +Legacy IRQ model
> +================
> +
> +In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer
> interrupt go
> +to CPUINTC directly, CPU UARTS interrupts go to LIOINTC, while all
> other devices
> +interrupts go to PCH-PIC/PCH-LPC/PCH-MSI and gathered by HTVECINTC,
> and then go
> +to LIOINTC, and then CPUINTC.
> +
> + +---------------------------------------------+
> + |:: |
> + | |
> + | +-----+ +---------+ +-------+ |
> + | | IPI | --> | CPUINTC | <-- | Timer | |
> + | +-----+ +---------+ +-------+ |
> + | ^ |
> + | | |
> + | +---------+ +-------+ |
> + | | LIOINTC | <-- | UARTs | |
> + | +---------+ +-------+ |
> + | ^ |
> + | | |
> + | +-----------+ |
> + | | HTVECINTC | |
> + | +-----------+ |
> + | ^ ^ |
> + | | | |
> + | +---------+ +---------+ |
> + | | PCH-PIC | | PCH-MSI | |
> + | +---------+ +---------+ |
> + | ^ ^ ^ |
> + | | | | |
> + | +---------+ +---------+ +---------+ |
> + | | PCH-LPC | | Devices | | Devices | |
> + | +---------+ +---------+ +---------+ |
> + | ^ |
> + | | |
> + | +---------+ |
> + | | Devices | |
> + | +---------+ |
> + | |
> + | |
> + +---------------------------------------------+
> +
> +Extended IRQ model
> +==================
> +
> +In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer
> interrupt go
> +to CPUINTC directly, CPU UARTS interrupts go to LIOINTC, while all
> other devices
> +interrupts go to PCH-PIC/PCH-LPC/PCH-MSI and gathered by EIOINTC, and
> then go to
> +to CPUINTC directly.
> +
> + +--------------------------------------------------------+
> + |:: |
> + | |
> + | +-----+ +---------+ +-------+ |
> + | | IPI | --> | CPUINTC | <-- | Timer | |
> + | +-----+ +---------+ +-------+ |
> + | ^ ^ |
> + | | | |
> + | +---------+ +---------+ +-------+ |
> + | | EIOINTC | | LIOINTC | <-- | UARTs | |
> + | +---------+ +---------+ +-------+ |
> + | ^ ^ |
> + | | | |
> + | +---------+ +---------+ |
> + | | PCH-PIC | | PCH-MSI | |
> + | +---------+ +---------+ |
> + | ^ ^ ^ |
> + | | | | |
> + | +---------+ +---------+ +---------+ |
> + | | PCH-LPC | | Devices | | Devices | |
> + | +---------+ +---------+ +---------+ |
> + | ^ |
> + | | |
> + | +---------+ |
> + | | Devices | |
> + | +---------+ |
> + | |
> + | |
> + +--------------------------------------------------------+
> +
> +ACPI-related definitions
> +========================
> +
> +CPUINTC::
> +
> + ACPI_MADT_TYPE_CORE_PIC;
> + struct acpi_madt_core_pic;
> + enum acpi_madt_core_pic_version;
> +
> +LIOINTC::
> +
> + ACPI_MADT_TYPE_LIO_PIC;
> + struct acpi_madt_lio_pic;
> + enum acpi_madt_lio_pic_version;
> +
> +EIOINTC::
> +
> + ACPI_MADT_TYPE_EIO_PIC;
> + struct acpi_madt_eio_pic;
> + enum acpi_madt_eio_pic_version;
> +
> +HTVECINTC::
> +
> + ACPI_MADT_TYPE_HT_PIC;
> + struct acpi_madt_ht_pic;
> + enum acpi_madt_ht_pic_version;
> +
> +PCH-PIC::
> +
> + ACPI_MADT_TYPE_BIO_PIC;
> + struct acpi_madt_bio_pic;
> + enum acpi_madt_bio_pic_version;
> +
> +PCH-MSI::
> +
> + ACPI_MADT_TYPE_MSI_PIC;
> + struct acpi_madt_msi_pic;
> + enum acpi_madt_msi_pic_version;
> +
> +PCH-LPC::
> +
> + ACPI_MADT_TYPE_LPC_PIC;
> + struct acpi_madt_lpc_pic;
> + enum acpi_madt_lpc_pic_version;
> +
> +References
> +==========
> +
> +Documentation of Loongson-3A5000:
> +
> +
> https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-3A5000-usermanual-1.02-CN.pdf
> (in Chinese)
> +
> +
> https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-3A5000-usermanual-1.02-EN.pdf
> (in English)
> +
> +Documentation of Loongson's LS7A chipset:
> +
> +
> https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-7A1000-usermanual-2.00-CN.pdf
> (in Chinese)
> +
> +
> https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-7A1000-usermanual-2.00-EN.pdf
> (in English)
> +
> +Note: CPUINTC is CSR.ECFG/CSR.ESTAT and its interrupt controller
> described
> +in Section 7.4 of "LoongArch Reference Manual, Vol 1"; LIOINTC is
> "Legacy I/O
> +Interrupts" described in Section 11.1 of "Loongson 3A5000 Processor
> Reference
> +Manual"; EIOINTC is "Extended I/O Interrupts" described in Section
> 11.2 of
> +"Loongson 3A5000 Processor Reference Manual"; HTVECINTC is
> "HyperTransport
> +Interrupts" described in Section 14.3 of "Loongson 3A5000 Processor
> Reference
> +Manual"; PCH-PIC/PCH-MSI is "Interrupt Controller" described in
> Section 5 of
> +"Loongson 7A1000 Bridge User Manual"; PCH-LPC is "LPC Interrupts"
> described in
> +Section 24.3 of "Loongson 7A1000 Bridge User Manual".
> --
> 2.27.0
--
- Jiaxun
LoongArch maintains cache coherency in hardware, but its WUC attribute
(Weak-ordered UnCached, which is similar to WC) is out of the scope of
cache coherency machanism. This means WUC can only used for write-only
memory regions.
Cc: Daniel Vetter <[email protected]>
Cc: [email protected]
Reviewed-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
drivers/gpu/drm/drm_vm.c | 2 +-
drivers/gpu/drm/ttm/ttm_module.c | 2 +-
include/drm/drm_cache.h | 8 ++++++++
3 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/drm_vm.c b/drivers/gpu/drm/drm_vm.c
index e957d4851dc0..f024dc93939e 100644
--- a/drivers/gpu/drm/drm_vm.c
+++ b/drivers/gpu/drm/drm_vm.c
@@ -69,7 +69,7 @@ static pgprot_t drm_io_prot(struct drm_local_map *map,
pgprot_t tmp = vm_get_page_prot(vma->vm_flags);
#if defined(__i386__) || defined(__x86_64__) || defined(__powerpc__) || \
- defined(__mips__)
+ defined(__mips__) || defined(__loongarch__)
if (map->type == _DRM_REGISTERS && !(map->flags & _DRM_WRITE_COMBINING))
tmp = pgprot_noncached(tmp);
else
diff --git a/drivers/gpu/drm/ttm/ttm_module.c b/drivers/gpu/drm/ttm/ttm_module.c
index a3ad7c9736ec..b3fffe7b5062 100644
--- a/drivers/gpu/drm/ttm/ttm_module.c
+++ b/drivers/gpu/drm/ttm/ttm_module.c
@@ -74,7 +74,7 @@ pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp)
#endif /* CONFIG_UML */
#endif /* __i386__ || __x86_64__ */
#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \
- defined(__powerpc__) || defined(__mips__)
+ defined(__powerpc__) || defined(__mips__) || defined(__loongarch__)
if (caching == ttm_write_combined)
tmp = pgprot_writecombine(tmp);
else
diff --git a/include/drm/drm_cache.h b/include/drm/drm_cache.h
index 22deb216b59c..08e0e3ffad13 100644
--- a/include/drm/drm_cache.h
+++ b/include/drm/drm_cache.h
@@ -67,6 +67,14 @@ static inline bool drm_arch_can_wc_memory(void)
* optimization entirely for ARM and arm64.
*/
return false;
+#elif defined(CONFIG_LOONGARCH)
+ /*
+ * LoongArch maintains cache coherency in hardware, but its WUC attribute
+ * (Weak-ordered UnCached, which is similar to WC) is out of the scope of
+ * cache coherency machanism. This means WUC can only used for write-only
+ * memory regions.
+ */
+ return false;
#else
return true;
#endif
--
2.27.0
Add memory management support for LoongArch, including: cache and tlb
management, page fault handling and ioremap/mmap support.
Reviewed-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/cache.h | 13 +
arch/loongarch/include/asm/cacheflush.h | 80 ++++
arch/loongarch/include/asm/cacheops.h | 37 ++
arch/loongarch/include/asm/fixmap.h | 13 +
arch/loongarch/include/asm/hugetlb.h | 83 ++++
arch/loongarch/include/asm/page.h | 115 +++++
arch/loongarch/include/asm/pgalloc.h | 103 +++++
arch/loongarch/include/asm/pgtable-bits.h | 131 ++++++
arch/loongarch/include/asm/pgtable.h | 532 ++++++++++++++++++++++
arch/loongarch/include/asm/shmparam.h | 12 +
arch/loongarch/include/asm/sparsemem.h | 23 +
arch/loongarch/include/asm/tlb.h | 180 ++++++++
arch/loongarch/include/asm/tlbflush.h | 35 ++
arch/loongarch/include/asm/vmalloc.h | 4 +
arch/loongarch/mm/cache.c | 141 ++++++
arch/loongarch/mm/extable.c | 22 +
arch/loongarch/mm/fault.c | 261 +++++++++++
arch/loongarch/mm/hugetlbpage.c | 87 ++++
arch/loongarch/mm/init.c | 165 +++++++
arch/loongarch/mm/ioremap.c | 27 ++
arch/loongarch/mm/maccess.c | 10 +
arch/loongarch/mm/mmap.c | 125 +++++
arch/loongarch/mm/page.S | 84 ++++
arch/loongarch/mm/pgtable.c | 130 ++++++
arch/loongarch/mm/tlb.c | 282 ++++++++++++
arch/loongarch/mm/tlbex.S | 477 +++++++++++++++++++
26 files changed, 3172 insertions(+)
create mode 100644 arch/loongarch/include/asm/cache.h
create mode 100644 arch/loongarch/include/asm/cacheflush.h
create mode 100644 arch/loongarch/include/asm/cacheops.h
create mode 100644 arch/loongarch/include/asm/fixmap.h
create mode 100644 arch/loongarch/include/asm/hugetlb.h
create mode 100644 arch/loongarch/include/asm/page.h
create mode 100644 arch/loongarch/include/asm/pgalloc.h
create mode 100644 arch/loongarch/include/asm/pgtable-bits.h
create mode 100644 arch/loongarch/include/asm/pgtable.h
create mode 100644 arch/loongarch/include/asm/shmparam.h
create mode 100644 arch/loongarch/include/asm/sparsemem.h
create mode 100644 arch/loongarch/include/asm/tlb.h
create mode 100644 arch/loongarch/include/asm/tlbflush.h
create mode 100644 arch/loongarch/include/asm/vmalloc.h
create mode 100644 arch/loongarch/mm/cache.c
create mode 100644 arch/loongarch/mm/extable.c
create mode 100644 arch/loongarch/mm/fault.c
create mode 100644 arch/loongarch/mm/hugetlbpage.c
create mode 100644 arch/loongarch/mm/init.c
create mode 100644 arch/loongarch/mm/ioremap.c
create mode 100644 arch/loongarch/mm/maccess.c
create mode 100644 arch/loongarch/mm/mmap.c
create mode 100644 arch/loongarch/mm/page.S
create mode 100644 arch/loongarch/mm/pgtable.c
create mode 100644 arch/loongarch/mm/tlb.c
create mode 100644 arch/loongarch/mm/tlbex.S
diff --git a/arch/loongarch/include/asm/cache.h b/arch/loongarch/include/asm/cache.h
new file mode 100644
index 000000000000..1b6d09617199
--- /dev/null
+++ b/arch/loongarch/include/asm/cache.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_CACHE_H
+#define _ASM_CACHE_H
+
+#define L1_CACHE_SHIFT CONFIG_L1_CACHE_SHIFT
+#define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT)
+
+#define __read_mostly __section(".data..read_mostly")
+
+#endif /* _ASM_CACHE_H */
diff --git a/arch/loongarch/include/asm/cacheflush.h b/arch/loongarch/include/asm/cacheflush.h
new file mode 100644
index 000000000000..670900141b7c
--- /dev/null
+++ b/arch/loongarch/include/asm/cacheflush.h
@@ -0,0 +1,80 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_CACHEFLUSH_H
+#define _ASM_CACHEFLUSH_H
+
+#include <linux/mm.h>
+#include <asm/cpu-features.h>
+#include <asm/cacheops.h>
+
+extern void local_flush_icache_range(unsigned long start, unsigned long end);
+
+#define flush_icache_range local_flush_icache_range
+#define flush_icache_user_range local_flush_icache_range
+
+#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
+
+#define flush_cache_all() do { } while (0)
+#define flush_cache_mm(mm) do { } while (0)
+#define flush_cache_dup_mm(mm) do { } while (0)
+#define flush_cache_range(vma, start, end) do { } while (0)
+#define flush_cache_page(vma, vmaddr, pfn) do { } while (0)
+#define flush_cache_vmap(start, end) do { } while (0)
+#define flush_cache_vunmap(start, end) do { } while (0)
+#define flush_icache_page(vma, page) do { } while (0)
+#define flush_icache_user_page(vma, page, addr, len) do { } while (0)
+#define flush_dcache_page(page) do { } while (0)
+#define flush_dcache_mmap_lock(mapping) do { } while (0)
+#define flush_dcache_mmap_unlock(mapping) do { } while (0)
+
+#define cache_op(op, addr) \
+ __asm__ __volatile__( \
+ " cacop %0, %1 \n" \
+ : \
+ : "i" (op), "ZC" (*(unsigned char *)(addr)))
+
+static inline void flush_icache_line_indexed(unsigned long addr)
+{
+ cache_op(Index_Invalidate_I, addr);
+}
+
+static inline void flush_dcache_line_indexed(unsigned long addr)
+{
+ cache_op(Index_Writeback_Inv_D, addr);
+}
+
+static inline void flush_vcache_line_indexed(unsigned long addr)
+{
+ cache_op(Index_Writeback_Inv_V, addr);
+}
+
+static inline void flush_scache_line_indexed(unsigned long addr)
+{
+ cache_op(Index_Writeback_Inv_S, addr);
+}
+
+static inline void flush_icache_line(unsigned long addr)
+{
+ cache_op(Hit_Invalidate_I, addr);
+}
+
+static inline void flush_dcache_line(unsigned long addr)
+{
+ cache_op(Hit_Writeback_Inv_D, addr);
+}
+
+static inline void flush_vcache_line(unsigned long addr)
+{
+ cache_op(Hit_Writeback_Inv_V, addr);
+}
+
+static inline void flush_scache_line(unsigned long addr)
+{
+ cache_op(Hit_Writeback_Inv_S, addr);
+}
+
+#include <asm-generic/cacheflush.h>
+
+#endif /* _ASM_CACHEFLUSH_H */
diff --git a/arch/loongarch/include/asm/cacheops.h b/arch/loongarch/include/asm/cacheops.h
new file mode 100644
index 000000000000..dc280efecebd
--- /dev/null
+++ b/arch/loongarch/include/asm/cacheops.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Cache operations for the cache instruction.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_CACHEOPS_H
+#define __ASM_CACHEOPS_H
+
+/*
+ * Most cache ops are split into a 2 bit field identifying the cache, and a 3
+ * bit field identifying the cache operation.
+ */
+#define CacheOp_Cache 0x03
+#define CacheOp_Op 0x1c
+
+#define Cache_I 0x00
+#define Cache_D 0x01
+#define Cache_V 0x02
+#define Cache_S 0x03
+
+#define Index_Invalidate 0x08
+#define Index_Writeback_Inv 0x08
+#define Hit_Invalidate 0x10
+#define Hit_Writeback_Inv 0x10
+#define CacheOp_User_Defined 0x18
+
+#define Index_Invalidate_I (Cache_I | Index_Invalidate)
+#define Index_Writeback_Inv_D (Cache_D | Index_Writeback_Inv)
+#define Index_Writeback_Inv_V (Cache_V | Index_Writeback_Inv)
+#define Index_Writeback_Inv_S (Cache_S | Index_Writeback_Inv)
+#define Hit_Invalidate_I (Cache_I | Hit_Invalidate)
+#define Hit_Writeback_Inv_D (Cache_D | Hit_Writeback_Inv)
+#define Hit_Writeback_Inv_V (Cache_V | Hit_Writeback_Inv)
+#define Hit_Writeback_Inv_S (Cache_S | Hit_Writeback_Inv)
+
+#endif /* __ASM_CACHEOPS_H */
diff --git a/arch/loongarch/include/asm/fixmap.h b/arch/loongarch/include/asm/fixmap.h
new file mode 100644
index 000000000000..b3541dfa2013
--- /dev/null
+++ b/arch/loongarch/include/asm/fixmap.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * fixmap.h: compile-time virtual memory allocation
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef _ASM_FIXMAP_H
+#define _ASM_FIXMAP_H
+
+#define NR_FIX_BTMAPS 64
+
+#endif
diff --git a/arch/loongarch/include/asm/hugetlb.h b/arch/loongarch/include/asm/hugetlb.h
new file mode 100644
index 000000000000..aa44b3fe43dd
--- /dev/null
+++ b/arch/loongarch/include/asm/hugetlb.h
@@ -0,0 +1,83 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASM_HUGETLB_H
+#define __ASM_HUGETLB_H
+
+#include <asm/page.h>
+
+uint64_t pmd_to_entrylo(unsigned long pmd_val);
+
+#define __HAVE_ARCH_PREPARE_HUGEPAGE_RANGE
+static inline int prepare_hugepage_range(struct file *file,
+ unsigned long addr,
+ unsigned long len)
+{
+ unsigned long task_size = STACK_TOP;
+ struct hstate *h = hstate_file(file);
+
+ if (len & ~huge_page_mask(h))
+ return -EINVAL;
+ if (addr & ~huge_page_mask(h))
+ return -EINVAL;
+ if (len > task_size)
+ return -ENOMEM;
+ if (task_size - len < addr)
+ return -EINVAL;
+ return 0;
+}
+
+#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR
+static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep)
+{
+ pte_t clear;
+ pte_t pte = *ptep;
+
+ pte_val(clear) = (unsigned long)invalid_pte_table;
+ set_pte_at(mm, addr, ptep, clear);
+ return pte;
+}
+
+#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH
+static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep)
+{
+ pte_t pte;
+
+ pte = huge_ptep_get_and_clear(vma->vm_mm, addr, ptep);
+ flush_tlb_page(vma, addr);
+ return pte;
+}
+
+#define __HAVE_ARCH_HUGE_PTE_NONE
+static inline int huge_pte_none(pte_t pte)
+{
+ unsigned long val = pte_val(pte) & ~_PAGE_GLOBAL;
+ return !val || (val == (unsigned long)invalid_pte_table);
+}
+
+#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS
+static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma,
+ unsigned long addr,
+ pte_t *ptep, pte_t pte,
+ int dirty)
+{
+ int changed = !pte_same(*ptep, pte);
+
+ if (changed) {
+ set_pte_at(vma->vm_mm, addr, ptep, pte);
+ /*
+ * There could be some standard sized pages in there,
+ * get them all.
+ */
+ flush_tlb_range(vma, addr, addr + HPAGE_SIZE);
+ }
+ return changed;
+}
+
+#include <asm-generic/hugetlb.h>
+
+#endif /* __ASM_HUGETLB_H */
diff --git a/arch/loongarch/include/asm/page.h b/arch/loongarch/include/asm/page.h
new file mode 100644
index 000000000000..3dba4986f6c9
--- /dev/null
+++ b/arch/loongarch/include/asm/page.h
@@ -0,0 +1,115 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_PAGE_H
+#define _ASM_PAGE_H
+
+#include <linux/const.h>
+
+/*
+ * PAGE_SHIFT determines the page size
+ */
+#ifdef CONFIG_PAGE_SIZE_4KB
+#define PAGE_SHIFT 12
+#endif
+#ifdef CONFIG_PAGE_SIZE_16KB
+#define PAGE_SHIFT 14
+#endif
+#ifdef CONFIG_PAGE_SIZE_64KB
+#define PAGE_SHIFT 16
+#endif
+#define PAGE_SIZE (_AC(1, UL) << PAGE_SHIFT)
+#define PAGE_MASK (~(PAGE_SIZE - 1))
+
+#define HPAGE_SHIFT (PAGE_SHIFT + PAGE_SHIFT - 3)
+#define HPAGE_SIZE (_AC(1, UL) << HPAGE_SHIFT)
+#define HPAGE_MASK (~(HPAGE_SIZE - 1))
+#define HUGETLB_PAGE_ORDER (HPAGE_SHIFT - PAGE_SHIFT)
+
+#ifndef __ASSEMBLY__
+
+#include <linux/kernel.h>
+#include <linux/pfn.h>
+
+#define MAX_DMA32_PFN (1UL << (32 - PAGE_SHIFT))
+
+/*
+ * It's normally defined only for FLATMEM config but it's
+ * used in our early mem init code for all memory models.
+ * So always define it.
+ */
+#define ARCH_PFN_OFFSET PFN_UP(PHYS_OFFSET)
+
+extern void clear_page(void *page);
+extern void copy_page(void *to, void *from);
+
+#define clear_user_page(page, vaddr, pg) clear_page(page)
+#define copy_user_page(to, from, vaddr, pg) copy_page(to, from)
+
+extern unsigned long shm_align_mask;
+
+struct page;
+struct vm_area_struct;
+void copy_user_highpage(struct page *to, struct page *from,
+ unsigned long vaddr, struct vm_area_struct *vma);
+
+#define __HAVE_ARCH_COPY_USER_HIGHPAGE
+
+typedef struct { unsigned long pte; } pte_t;
+#define pte_val(x) ((x).pte)
+#define __pte(x) ((pte_t) { (x) })
+typedef struct page *pgtable_t;
+
+typedef struct { unsigned long pgd; } pgd_t;
+#define pgd_val(x) ((x).pgd)
+#define __pgd(x) ((pgd_t) { (x) })
+
+/*
+ * Manipulate page protection bits
+ */
+typedef struct { unsigned long pgprot; } pgprot_t;
+#define pgprot_val(x) ((x).pgprot)
+#define __pgprot(x) ((pgprot_t) { (x) })
+#define pte_pgprot(x) __pgprot(pte_val(x) & ~_PFN_MASK)
+
+#define ptep_buddy(x) ((pte_t *)((unsigned long)(x) ^ sizeof(pte_t)))
+
+/*
+ * __pa()/__va() should be used only during mem init.
+ */
+#define __pa(x) PHYSADDR(x)
+#define __va(x) ((void *)((unsigned long)(x) + PAGE_OFFSET - PHYS_OFFSET))
+
+#define pfn_to_kaddr(pfn) __va((pfn) << PAGE_SHIFT)
+
+#ifdef CONFIG_FLATMEM
+
+static inline int pfn_valid(unsigned long pfn)
+{
+ /* avoid <linux/mm.h> include hell */
+ extern unsigned long max_mapnr;
+ unsigned long pfn_offset = ARCH_PFN_OFFSET;
+
+ return pfn >= pfn_offset && pfn < max_mapnr;
+}
+
+#endif
+
+#define virt_to_pfn(kaddr) PFN_DOWN(virt_to_phys((void *)(kaddr)))
+#define virt_to_page(kaddr) pfn_to_page(virt_to_pfn(kaddr))
+
+extern int __virt_addr_valid(volatile void *kaddr);
+#define virt_addr_valid(kaddr) __virt_addr_valid((volatile void *)(kaddr))
+
+#define VM_DATA_DEFAULT_FLAGS \
+ (VM_READ | VM_WRITE | \
+ ((current->personality & READ_IMPLIES_EXEC) ? VM_EXEC : 0) | \
+ VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
+
+#include <asm-generic/memory_model.h>
+#include <asm-generic/getorder.h>
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* _ASM_PAGE_H */
diff --git a/arch/loongarch/include/asm/pgalloc.h b/arch/loongarch/include/asm/pgalloc.h
new file mode 100644
index 000000000000..b0a57b25c131
--- /dev/null
+++ b/arch/loongarch/include/asm/pgalloc.h
@@ -0,0 +1,103 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_PGALLOC_H
+#define _ASM_PGALLOC_H
+
+#include <linux/mm.h>
+#include <linux/sched.h>
+
+#define __HAVE_ARCH_PMD_ALLOC_ONE
+#define __HAVE_ARCH_PUD_ALLOC_ONE
+#include <asm-generic/pgalloc.h>
+
+static inline void pmd_populate_kernel(struct mm_struct *mm,
+ pmd_t *pmd, pte_t *pte)
+{
+ set_pmd(pmd, __pmd((unsigned long)pte));
+}
+
+static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd, pgtable_t pte)
+{
+ set_pmd(pmd, __pmd((unsigned long)page_address(pte)));
+}
+
+#ifndef __PAGETABLE_PMD_FOLDED
+
+static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
+{
+ set_pud(pud, __pud((unsigned long)pmd));
+}
+#endif
+
+#ifndef __PAGETABLE_PUD_FOLDED
+
+static inline void p4d_populate(struct mm_struct *mm, p4d_t *p4d, pud_t *pud)
+{
+ set_p4d(p4d, __p4d((unsigned long)pud));
+}
+
+#endif /* __PAGETABLE_PUD_FOLDED */
+
+extern void pagetable_init(void);
+
+/*
+ * Initialize a new pmd table with invalid pointers.
+ */
+extern void pmd_init(unsigned long page, unsigned long pagetable);
+
+/*
+ * Initialize a new pgd / pmd table with invalid pointers.
+ */
+extern void pgd_init(unsigned long page);
+extern pgd_t *pgd_alloc(struct mm_struct *mm);
+
+#define __pte_free_tlb(tlb, pte, address) \
+do { \
+ pgtable_pte_page_dtor(pte); \
+ tlb_remove_page((tlb), pte); \
+} while (0)
+
+#ifndef __PAGETABLE_PMD_FOLDED
+
+static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
+{
+ pmd_t *pmd;
+ struct page *pg;
+
+ pg = alloc_pages(GFP_KERNEL_ACCOUNT, PMD_ORDER);
+ if (!pg)
+ return NULL;
+
+ if (!pgtable_pmd_page_ctor(pg)) {
+ __free_pages(pg, PMD_ORDER);
+ return NULL;
+ }
+
+ pmd = (pmd_t *)page_address(pg);
+ pmd_init((unsigned long)pmd, (unsigned long)invalid_pte_table);
+ return pmd;
+}
+
+#define __pmd_free_tlb(tlb, x, addr) pmd_free((tlb)->mm, x)
+
+#endif
+
+#ifndef __PAGETABLE_PUD_FOLDED
+
+static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long address)
+{
+ pud_t *pud;
+
+ pud = (pud_t *) __get_free_pages(GFP_KERNEL, PUD_ORDER);
+ if (pud)
+ pud_init((unsigned long)pud, (unsigned long)invalid_pmd_table);
+ return pud;
+}
+
+#define __pud_free_tlb(tlb, x, addr) pud_free((tlb)->mm, x)
+
+#endif /* __PAGETABLE_PUD_FOLDED */
+
+#endif /* _ASM_PGALLOC_H */
diff --git a/arch/loongarch/include/asm/pgtable-bits.h b/arch/loongarch/include/asm/pgtable-bits.h
new file mode 100644
index 000000000000..3badd112d9ab
--- /dev/null
+++ b/arch/loongarch/include/asm/pgtable-bits.h
@@ -0,0 +1,131 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_PGTABLE_BITS_H
+#define _ASM_PGTABLE_BITS_H
+
+/* Page table bits */
+#define _PAGE_VALID_SHIFT 0
+#define _PAGE_ACCESSED_SHIFT 0 /* Reuse Valid for Accessed */
+#define _PAGE_DIRTY_SHIFT 1
+#define _PAGE_PLV_SHIFT 2 /* 2~3, two bits */
+#define _CACHE_SHIFT 4 /* 4~5, two bits */
+#define _PAGE_GLOBAL_SHIFT 6
+#define _PAGE_HUGE_SHIFT 6 /* HUGE is a PMD bit */
+#define _PAGE_PRESENT_SHIFT 7
+#define _PAGE_WRITE_SHIFT 8
+#define _PAGE_MODIFIED_SHIFT 9
+#define _PAGE_PROTNONE_SHIFT 10
+#define _PAGE_SPECIAL_SHIFT 11
+#define _PAGE_HGLOBAL_SHIFT 12 /* HGlobal is a PMD bit */
+#define _PAGE_PFN_SHIFT 12
+#define _PAGE_PFN_END_SHIFT 48
+#define _PAGE_NO_READ_SHIFT 61
+#define _PAGE_NO_EXEC_SHIFT 62
+#define _PAGE_RPLV_SHIFT 63
+
+/* Used by software */
+#define _PAGE_PRESENT (_ULCAST_(1) << _PAGE_PRESENT_SHIFT)
+#define _PAGE_WRITE (_ULCAST_(1) << _PAGE_WRITE_SHIFT)
+#define _PAGE_ACCESSED (_ULCAST_(1) << _PAGE_ACCESSED_SHIFT)
+#define _PAGE_MODIFIED (_ULCAST_(1) << _PAGE_MODIFIED_SHIFT)
+#define _PAGE_PROTNONE (_ULCAST_(1) << _PAGE_PROTNONE_SHIFT)
+#define _PAGE_SPECIAL (_ULCAST_(1) << _PAGE_SPECIAL_SHIFT)
+
+/* Used by TLB hardware (placed in EntryLo*) */
+#define _PAGE_VALID (_ULCAST_(1) << _PAGE_VALID_SHIFT)
+#define _PAGE_DIRTY (_ULCAST_(1) << _PAGE_DIRTY_SHIFT)
+#define _PAGE_PLV (_ULCAST_(3) << _PAGE_PLV_SHIFT)
+#define _PAGE_GLOBAL (_ULCAST_(1) << _PAGE_GLOBAL_SHIFT)
+#define _PAGE_HUGE (_ULCAST_(1) << _PAGE_HUGE_SHIFT)
+#define _PAGE_HGLOBAL (_ULCAST_(1) << _PAGE_HGLOBAL_SHIFT)
+#define _PAGE_NO_READ (_ULCAST_(1) << _PAGE_NO_READ_SHIFT)
+#define _PAGE_NO_EXEC (_ULCAST_(1) << _PAGE_NO_EXEC_SHIFT)
+#define _PAGE_RPLV (_ULCAST_(1) << _PAGE_RPLV_SHIFT)
+#define _CACHE_MASK (_ULCAST_(3) << _CACHE_SHIFT)
+#define _PFN_SHIFT (PAGE_SHIFT - 12 + _PAGE_PFN_SHIFT)
+
+#define _PAGE_USER (PLV_USER << _PAGE_PLV_SHIFT)
+#define _PAGE_KERN (PLV_KERN << _PAGE_PLV_SHIFT)
+
+#define _PFN_MASK (~((_ULCAST_(1) << (_PFN_SHIFT)) - 1) & \
+ ((_ULCAST_(1) << (_PAGE_PFN_END_SHIFT)) - 1))
+
+/*
+ * Cache attributes
+ */
+#ifndef _CACHE_SUC
+#define _CACHE_SUC (0<<_CACHE_SHIFT) /* Strong-ordered UnCached */
+#endif
+#ifndef _CACHE_CC
+#define _CACHE_CC (1<<_CACHE_SHIFT) /* Coherent Cached */
+#endif
+#ifndef _CACHE_WUC
+#define _CACHE_WUC (2<<_CACHE_SHIFT) /* Weak-ordered UnCached */
+#endif
+
+#define __READABLE (_PAGE_VALID)
+#define __WRITEABLE (_PAGE_DIRTY | _PAGE_WRITE)
+
+#define _PAGE_CHG_MASK (_PAGE_MODIFIED | _PAGE_SPECIAL | _PFN_MASK | _CACHE_MASK | _PAGE_PLV)
+#define _HPAGE_CHG_MASK (_PAGE_MODIFIED | _PAGE_SPECIAL | _PFN_MASK | _CACHE_MASK | _PAGE_PLV | _PAGE_HUGE)
+
+#define PAGE_NONE __pgprot(_PAGE_PROTNONE | _PAGE_NO_READ | \
+ _PAGE_USER | _CACHE_CC)
+#define PAGE_SHARED __pgprot(_PAGE_PRESENT | _PAGE_WRITE | \
+ _PAGE_USER | _CACHE_CC)
+#define PAGE_READONLY __pgprot(_PAGE_PRESENT | _PAGE_USER | _CACHE_CC)
+
+#define PAGE_KERNEL __pgprot(_PAGE_PRESENT | __READABLE | __WRITEABLE | \
+ _PAGE_GLOBAL | _PAGE_KERN | _CACHE_CC)
+#define PAGE_KERNEL_SUC __pgprot(_PAGE_PRESENT | __READABLE | __WRITEABLE | \
+ _PAGE_GLOBAL | _PAGE_KERN | _CACHE_SUC)
+#define PAGE_KERNEL_WUC __pgprot(_PAGE_PRESENT | __READABLE | __WRITEABLE | \
+ _PAGE_GLOBAL | _PAGE_KERN | _CACHE_WUC)
+
+#define __P000 __pgprot(_CACHE_CC | _PAGE_USER | _PAGE_PROTNONE | _PAGE_NO_EXEC | _PAGE_NO_READ)
+#define __P001 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC)
+#define __P010 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC)
+#define __P011 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC)
+#define __P100 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+#define __P101 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+#define __P110 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+#define __P111 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+
+#define __S000 __pgprot(_CACHE_CC | _PAGE_USER | _PAGE_PROTNONE | _PAGE_NO_EXEC | _PAGE_NO_READ)
+#define __S001 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC)
+#define __S010 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC | _PAGE_WRITE)
+#define __S011 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC | _PAGE_WRITE)
+#define __S100 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+#define __S101 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+#define __S110 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_WRITE)
+#define __S111 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_WRITE)
+
+#ifndef __ASSEMBLY__
+
+#define pgprot_noncached pgprot_noncached
+
+static inline pgprot_t pgprot_noncached(pgprot_t _prot)
+{
+ unsigned long prot = pgprot_val(_prot);
+
+ prot = (prot & ~_CACHE_MASK) | _CACHE_SUC;
+
+ return __pgprot(prot);
+}
+
+#define pgprot_writecombine pgprot_writecombine
+
+static inline pgprot_t pgprot_writecombine(pgprot_t _prot)
+{
+ unsigned long prot = pgprot_val(_prot);
+
+ prot = (prot & ~_CACHE_MASK) | _CACHE_WUC;
+
+ return __pgprot(prot);
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* _ASM_PGTABLE_BITS_H */
diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h
new file mode 100644
index 000000000000..8920dd8b297b
--- /dev/null
+++ b/arch/loongarch/include/asm/pgtable.h
@@ -0,0 +1,532 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1994, 95, 96, 97, 98, 99, 2000, 2003 Ralf Baechle
+ * Copyright (C) 1999, 2000, 2001 Silicon Graphics, Inc.
+ */
+#ifndef _ASM_PGTABLE_H
+#define _ASM_PGTABLE_H
+
+#include <linux/compiler.h>
+#include <asm/addrspace.h>
+#include <asm/pgtable-bits.h>
+
+#if CONFIG_PGTABLE_LEVELS == 2
+#include <asm-generic/pgtable-nopmd.h>
+#elif CONFIG_PGTABLE_LEVELS == 3
+#include <asm-generic/pgtable-nopud.h>
+#else
+#include <asm-generic/pgtable-nop4d.h>
+#endif
+
+#define PGD_ORDER 0
+#define PUD_ORDER 0
+#define PMD_ORDER 0
+#define PTE_ORDER 0
+
+#if CONFIG_PGTABLE_LEVELS == 2
+#define PGDIR_SHIFT (PAGE_SHIFT + (PAGE_SHIFT + PTE_ORDER - 3))
+#elif CONFIG_PGTABLE_LEVELS == 3
+#define PMD_SHIFT (PAGE_SHIFT + (PAGE_SHIFT + PTE_ORDER - 3))
+#define PMD_SIZE (1UL << PMD_SHIFT)
+#define PMD_MASK (~(PMD_SIZE-1))
+#define PGDIR_SHIFT (PMD_SHIFT + (PAGE_SHIFT + PMD_ORDER - 3))
+#elif CONFIG_PGTABLE_LEVELS == 4
+#define PMD_SHIFT (PAGE_SHIFT + (PAGE_SHIFT + PTE_ORDER - 3))
+#define PMD_SIZE (1UL << PMD_SHIFT)
+#define PMD_MASK (~(PMD_SIZE-1))
+#define PUD_SHIFT (PMD_SHIFT + (PAGE_SHIFT + PMD_ORDER - 3))
+#define PUD_SIZE (1UL << PUD_SHIFT)
+#define PUD_MASK (~(PUD_SIZE-1))
+#define PGDIR_SHIFT (PUD_SHIFT + (PAGE_SHIFT + PUD_ORDER - 3))
+#endif
+
+#define PGDIR_SIZE (1UL << PGDIR_SHIFT)
+#define PGDIR_MASK (~(PGDIR_SIZE-1))
+
+#define VA_BITS (PGDIR_SHIFT + (PAGE_SHIFT + PGD_ORDER - 3))
+
+#define PTRS_PER_PGD ((PAGE_SIZE << PGD_ORDER) >> 3)
+#if CONFIG_PGTABLE_LEVELS > 3
+#define PTRS_PER_PUD ((PAGE_SIZE << PUD_ORDER) >> 3)
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+#define PTRS_PER_PMD ((PAGE_SIZE << PMD_ORDER) >> 3)
+#endif
+#define PTRS_PER_PTE ((PAGE_SIZE << PTE_ORDER) >> 3)
+
+#define USER_PTRS_PER_PGD ((TASK_SIZE64 / PGDIR_SIZE)?(TASK_SIZE64 / PGDIR_SIZE):1)
+
+#ifndef __ASSEMBLY__
+
+#include <linux/mm_types.h>
+#include <linux/mmzone.h>
+#include <asm/fixmap.h>
+#include <asm/io.h>
+
+struct mm_struct;
+struct vm_area_struct;
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero; used
+ * for zero-mapped memory areas etc..
+ */
+
+extern unsigned long empty_zero_page;
+extern unsigned long zero_page_mask;
+
+#define ZERO_PAGE(vaddr) \
+ (virt_to_page((void *)(empty_zero_page + (((unsigned long)(vaddr)) & zero_page_mask))))
+#define __HAVE_COLOR_ZERO_PAGE
+
+/*
+ * TLB refill handlers may also map the vmalloc area into xkvrange.
+ * Avoid the first couple of pages so NULL pointer dereferences will
+ * still reliably trap.
+ */
+#define MODULES_VADDR (vm_map_base + PCI_IOSIZE + (2 * PAGE_SIZE))
+#define MODULES_END (MODULES_VADDR + SZ_256M)
+
+#define VMALLOC_START MODULES_END
+#define VMALLOC_END \
+ (vm_map_base + \
+ min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE, (1UL << cpu_vabits)) - PMD_SIZE)
+
+#define pte_ERROR(e) \
+ pr_err("%s:%d: bad pte %016lx.\n", __FILE__, __LINE__, pte_val(e))
+#ifndef __PAGETABLE_PMD_FOLDED
+#define pmd_ERROR(e) \
+ pr_err("%s:%d: bad pmd %016lx.\n", __FILE__, __LINE__, pmd_val(e))
+#endif
+#ifndef __PAGETABLE_PUD_FOLDED
+#define pud_ERROR(e) \
+ pr_err("%s:%d: bad pud %016lx.\n", __FILE__, __LINE__, pud_val(e))
+#endif
+#define pgd_ERROR(e) \
+ pr_err("%s:%d: bad pgd %016lx.\n", __FILE__, __LINE__, pgd_val(e))
+
+extern pte_t invalid_pte_table[PTRS_PER_PTE];
+
+#ifndef __PAGETABLE_PUD_FOLDED
+
+typedef struct { unsigned long pud; } pud_t;
+#define pud_val(x) ((x).pud)
+#define __pud(x) ((pud_t) { (x) })
+
+extern pud_t invalid_pud_table[PTRS_PER_PUD];
+
+/*
+ * Empty pgd/p4d entries point to the invalid_pud_table.
+ */
+static inline int p4d_none(p4d_t p4d)
+{
+ return p4d_val(p4d) == (unsigned long)invalid_pud_table;
+}
+
+static inline int p4d_bad(p4d_t p4d)
+{
+ return p4d_val(p4d) & ~PAGE_MASK;
+}
+
+static inline int p4d_present(p4d_t p4d)
+{
+ return p4d_val(p4d) != (unsigned long)invalid_pud_table;
+}
+
+static inline void p4d_clear(p4d_t *p4dp)
+{
+ p4d_val(*p4dp) = (unsigned long)invalid_pud_table;
+}
+
+static inline pud_t *p4d_pgtable(p4d_t p4d)
+{
+ return (pud_t *)p4d_val(p4d);
+}
+
+static inline void set_p4d(p4d_t *p4d, p4d_t p4dval)
+{
+ *p4d = p4dval;
+}
+
+#define p4d_phys(p4d) virt_to_phys((void *)p4d_val(p4d))
+#define p4d_page(p4d) (pfn_to_page(p4d_phys(p4d) >> PAGE_SHIFT))
+
+#endif
+
+#ifndef __PAGETABLE_PMD_FOLDED
+
+typedef struct { unsigned long pmd; } pmd_t;
+#define pmd_val(x) ((x).pmd)
+#define __pmd(x) ((pmd_t) { (x) })
+
+extern pmd_t invalid_pmd_table[PTRS_PER_PMD];
+
+/*
+ * Empty pud entries point to the invalid_pmd_table.
+ */
+static inline int pud_none(pud_t pud)
+{
+ return pud_val(pud) == (unsigned long)invalid_pmd_table;
+}
+
+static inline int pud_bad(pud_t pud)
+{
+ return pud_val(pud) & ~PAGE_MASK;
+}
+
+static inline int pud_present(pud_t pud)
+{
+ return pud_val(pud) != (unsigned long)invalid_pmd_table;
+}
+
+static inline void pud_clear(pud_t *pudp)
+{
+ pud_val(*pudp) = ((unsigned long)invalid_pmd_table);
+}
+
+static inline pmd_t *pud_pgtable(pud_t pud)
+{
+ return (pmd_t *)pud_val(pud);
+}
+
+#define set_pud(pudptr, pudval) do { *(pudptr) = (pudval); } while (0)
+
+#define pud_phys(pud) virt_to_phys((void *)pud_val(pud))
+#define pud_page(pud) (pfn_to_page(pud_phys(pud) >> PAGE_SHIFT))
+
+#endif
+
+/*
+ * Empty pmd entries point to the invalid_pte_table.
+ */
+static inline int pmd_none(pmd_t pmd)
+{
+ return pmd_val(pmd) == (unsigned long)invalid_pte_table;
+}
+
+static inline int pmd_bad(pmd_t pmd)
+{
+ return (pmd_val(pmd) & ~PAGE_MASK);
+}
+
+static inline int pmd_present(pmd_t pmd)
+{
+ if (unlikely(pmd_val(pmd) & _PAGE_HUGE))
+ return !!(pmd_val(pmd) & (_PAGE_PRESENT | _PAGE_PROTNONE));
+
+ return pmd_val(pmd) != (unsigned long)invalid_pte_table;
+}
+
+static inline void pmd_clear(pmd_t *pmdp)
+{
+ pmd_val(*pmdp) = ((unsigned long)invalid_pte_table);
+}
+
+#define set_pmd(pmdptr, pmdval) do { *(pmdptr) = (pmdval); } while (0)
+
+#define pmd_phys(pmd) virt_to_phys((void *)pmd_val(pmd))
+
+#ifndef CONFIG_TRANSPARENT_HUGEPAGE
+#define pmd_page(pmd) (pfn_to_page(pmd_phys(pmd) >> PAGE_SHIFT))
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+
+#define pmd_page_vaddr(pmd) pmd_val(pmd)
+
+extern pmd_t mk_pmd(struct page *page, pgprot_t prot);
+extern void set_pmd_at(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, pmd_t pmd);
+
+#define pte_page(x) pfn_to_page(pte_pfn(x))
+#define pte_pfn(x) ((unsigned long)(((x).pte & _PFN_MASK) >> _PFN_SHIFT))
+#define pfn_pte(pfn, prot) __pte(((pfn) << _PFN_SHIFT) | pgprot_val(prot))
+#define pfn_pmd(pfn, prot) __pmd(((pfn) << _PFN_SHIFT) | pgprot_val(prot))
+
+/*
+ * Initialize a new pgd / pmd table with invalid pointers.
+ */
+extern void pgd_init(unsigned long page);
+extern void pud_init(unsigned long page, unsigned long pagetable);
+extern void pmd_init(unsigned long page, unsigned long pagetable);
+
+/*
+ * Non-present pages: high 40 bits are offset, next 8 bits type,
+ * low 16 bits zero.
+ */
+static inline pte_t mk_swap_pte(unsigned long type, unsigned long offset)
+{ pte_t pte; pte_val(pte) = (type << 16) | (offset << 24); return pte; }
+
+#define __swp_type(x) (((x).val >> 16) & 0xff)
+#define __swp_offset(x) ((x).val >> 24)
+#define __swp_entry(type, offset) ((swp_entry_t) { pte_val(mk_swap_pte((type), (offset))) })
+#define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) })
+#define __swp_entry_to_pte(x) ((pte_t) { (x).val })
+#define __pmd_to_swp_entry(pmd) ((swp_entry_t) { pmd_val(pmd) })
+#define __swp_entry_to_pmd(x) ((pmd_t) { (x).val | _PAGE_HUGE })
+
+extern void paging_init(void);
+
+#define pte_none(pte) (!(pte_val(pte) & ~_PAGE_GLOBAL))
+#define pte_present(pte) (pte_val(pte) & (_PAGE_PRESENT | _PAGE_PROTNONE))
+#define pte_no_exec(pte) (pte_val(pte) & _PAGE_NO_EXEC)
+
+static inline void set_pte(pte_t *ptep, pte_t pteval)
+{
+ *ptep = pteval;
+ if (pte_val(pteval) & _PAGE_GLOBAL) {
+ pte_t *buddy = ptep_buddy(ptep);
+ /*
+ * Make sure the buddy is global too (if it's !none,
+ * it better already be global)
+ */
+ if (pte_none(*buddy))
+ pte_val(*buddy) = pte_val(*buddy) | _PAGE_GLOBAL;
+ }
+}
+
+static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pteval)
+{
+ set_pte(ptep, pteval);
+}
+
+static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep)
+{
+ /* Preserve global status for the pair */
+ if (pte_val(*ptep_buddy(ptep)) & _PAGE_GLOBAL)
+ set_pte_at(mm, addr, ptep, __pte(_PAGE_GLOBAL));
+ else
+ set_pte_at(mm, addr, ptep, __pte(0));
+}
+
+#define PGD_T_LOG2 (__builtin_ffs(sizeof(pgd_t)) - 1)
+#define PMD_T_LOG2 (__builtin_ffs(sizeof(pmd_t)) - 1)
+#define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1)
+
+extern pgd_t swapper_pg_dir[];
+extern pgd_t invalid_pg_dir[];
+
+/*
+ * The following only work if pte_present() is true.
+ * Undefined behaviour if not..
+ */
+static inline int pte_write(pte_t pte) { return pte_val(pte) & _PAGE_WRITE; }
+static inline int pte_young(pte_t pte) { return pte_val(pte) & _PAGE_ACCESSED; }
+static inline int pte_dirty(pte_t pte) { return pte_val(pte) & _PAGE_MODIFIED; }
+
+static inline pte_t pte_mkold(pte_t pte)
+{
+ pte_val(pte) &= ~_PAGE_ACCESSED;
+ return pte;
+}
+
+static inline pte_t pte_mkyoung(pte_t pte)
+{
+ pte_val(pte) |= _PAGE_ACCESSED;
+ return pte;
+}
+
+static inline pte_t pte_mkclean(pte_t pte)
+{
+ pte_val(pte) &= ~(_PAGE_DIRTY | _PAGE_MODIFIED);
+ return pte;
+}
+
+static inline pte_t pte_mkdirty(pte_t pte)
+{
+ pte_val(pte) |= (_PAGE_DIRTY | _PAGE_MODIFIED);
+ return pte;
+}
+
+static inline pte_t pte_mkwrite(pte_t pte)
+{
+ pte_val(pte) |= (_PAGE_WRITE | _PAGE_DIRTY);
+ return pte;
+}
+
+static inline pte_t pte_wrprotect(pte_t pte)
+{
+ pte_val(pte) &= ~(_PAGE_WRITE | _PAGE_DIRTY);
+ return pte;
+}
+
+static inline int pte_huge(pte_t pte) { return pte_val(pte) & _PAGE_HUGE; }
+
+static inline pte_t pte_mkhuge(pte_t pte)
+{
+ pte_val(pte) |= _PAGE_HUGE;
+ return pte;
+}
+
+#if defined(CONFIG_ARCH_HAS_PTE_SPECIAL)
+static inline int pte_special(pte_t pte) { return pte_val(pte) & _PAGE_SPECIAL; }
+static inline pte_t pte_mkspecial(pte_t pte) { pte_val(pte) |= _PAGE_SPECIAL; return pte; }
+#endif /* CONFIG_ARCH_HAS_PTE_SPECIAL */
+
+#define pte_accessible pte_accessible
+static inline unsigned long pte_accessible(struct mm_struct *mm, pte_t a)
+{
+ if (pte_val(a) & _PAGE_PRESENT)
+ return true;
+
+ if ((pte_val(a) & _PAGE_PROTNONE) &&
+ atomic_read(&mm->tlb_flush_pending))
+ return true;
+
+ return false;
+}
+
+/*
+ * Conversion functions: convert a page and protection to a page entry,
+ * and a page entry and page directory to the page they refer to.
+ */
+#define mk_pte(page, pgprot) pfn_pte(page_to_pfn(page), (pgprot))
+
+static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
+{
+ return __pte((pte_val(pte) & _PAGE_CHG_MASK) |
+ (pgprot_val(newprot) & ~_PAGE_CHG_MASK));
+}
+
+extern void __update_tlb(struct vm_area_struct *vma,
+ unsigned long address, pte_t *ptep);
+
+static inline void update_mmu_cache(struct vm_area_struct *vma,
+ unsigned long address, pte_t *ptep)
+{
+ __update_tlb(vma, address, ptep);
+}
+
+static inline void update_mmu_cache_pmd(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp)
+{
+ __update_tlb(vma, address, (pte_t *)pmdp);
+}
+
+#define kern_addr_valid(addr) (1)
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+
+/* We don't have hardware dirty/accessed bits, generic_pmdp_establish is fine.*/
+#define pmdp_establish generic_pmdp_establish
+
+static inline int pmd_trans_huge(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_HUGE) && pmd_present(pmd);
+}
+
+static inline pmd_t pmd_mkhuge(pmd_t pmd)
+{
+ pmd_val(pmd) = (pmd_val(pmd) & ~(_PAGE_GLOBAL)) |
+ ((pmd_val(pmd) & _PAGE_GLOBAL) << (_PAGE_HGLOBAL_SHIFT - _PAGE_GLOBAL_SHIFT));
+ pmd_val(pmd) |= _PAGE_HUGE;
+
+ return pmd;
+}
+
+#define pmd_write pmd_write
+static inline int pmd_write(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_WRITE);
+}
+
+static inline pmd_t pmd_mkwrite(pmd_t pmd)
+{
+ pmd_val(pmd) |= (_PAGE_WRITE | _PAGE_DIRTY);
+ return pmd;
+}
+
+static inline pmd_t pmd_wrprotect(pmd_t pmd)
+{
+ pmd_val(pmd) &= ~(_PAGE_WRITE | _PAGE_DIRTY);
+ return pmd;
+}
+
+static inline int pmd_dirty(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_MODIFIED);
+}
+
+static inline pmd_t pmd_mkclean(pmd_t pmd)
+{
+ pmd_val(pmd) &= ~(_PAGE_DIRTY | _PAGE_MODIFIED);
+ return pmd;
+}
+
+static inline pmd_t pmd_mkdirty(pmd_t pmd)
+{
+ pmd_val(pmd) |= (_PAGE_DIRTY | _PAGE_MODIFIED);
+ return pmd;
+}
+
+static inline int pmd_young(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_ACCESSED);
+}
+
+static inline pmd_t pmd_mkold(pmd_t pmd)
+{
+ pmd_val(pmd) &= ~_PAGE_ACCESSED;
+ return pmd;
+}
+
+static inline pmd_t pmd_mkyoung(pmd_t pmd)
+{
+ pmd_val(pmd) |= _PAGE_ACCESSED;
+ return pmd;
+}
+
+static inline unsigned long pmd_pfn(pmd_t pmd)
+{
+ return (pmd_val(pmd) & _PFN_MASK) >> _PFN_SHIFT;
+}
+
+static inline struct page *pmd_page(pmd_t pmd)
+{
+ if (pmd_trans_huge(pmd))
+ return pfn_to_page(pmd_pfn(pmd));
+
+ return pfn_to_page(pmd_phys(pmd) >> PAGE_SHIFT);
+}
+
+static inline pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot)
+{
+ pmd_val(pmd) = (pmd_val(pmd) & _HPAGE_CHG_MASK) |
+ (pgprot_val(newprot) & ~_HPAGE_CHG_MASK);
+ return pmd;
+}
+
+static inline pmd_t pmd_mkinvalid(pmd_t pmd)
+{
+ pmd_val(pmd) &= ~(_PAGE_PRESENT | _PAGE_VALID | _PAGE_DIRTY | _PAGE_PROTNONE);
+
+ return pmd;
+}
+
+/*
+ * The generic version pmdp_huge_get_and_clear uses a version of pmd_clear() with a
+ * different prototype.
+ */
+#define __HAVE_ARCH_PMDP_HUGE_GET_AND_CLEAR
+static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
+ unsigned long address, pmd_t *pmdp)
+{
+ pmd_t old = *pmdp;
+
+ pmd_clear(pmdp);
+
+ return old;
+}
+
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+
+/*
+ * We provide our own get_unmapped area to cope with the virtual aliasing
+ * constraints placed on us by the cache architecture.
+ */
+#define HAVE_ARCH_UNMAPPED_AREA
+#define HAVE_ARCH_UNMAPPED_AREA_TOPDOWN
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* _ASM_PGTABLE_H */
diff --git a/arch/loongarch/include/asm/shmparam.h b/arch/loongarch/include/asm/shmparam.h
new file mode 100644
index 000000000000..c9554f48d2df
--- /dev/null
+++ b/arch/loongarch/include/asm/shmparam.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_SHMPARAM_H
+#define _ASM_SHMPARAM_H
+
+#define __ARCH_FORCE_SHMLBA 1
+
+#define SHMLBA SZ_64K /* attach addr a multiple of this */
+
+#endif /* _ASM_SHMPARAM_H */
diff --git a/arch/loongarch/include/asm/sparsemem.h b/arch/loongarch/include/asm/sparsemem.h
new file mode 100644
index 000000000000..3d18cdf1b069
--- /dev/null
+++ b/arch/loongarch/include/asm/sparsemem.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LOONGARCH_SPARSEMEM_H
+#define _LOONGARCH_SPARSEMEM_H
+
+#ifdef CONFIG_SPARSEMEM
+
+/*
+ * SECTION_SIZE_BITS 2^N: how big each section will be
+ * MAX_PHYSMEM_BITS 2^N: how much memory we can have in that space
+ */
+#define SECTION_SIZE_BITS 29 /* 2^29 = Largest Huge Page Size */
+#define MAX_PHYSMEM_BITS 48
+
+#endif /* CONFIG_SPARSEMEM */
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+int memory_add_physaddr_to_nid(u64 addr);
+#define memory_add_physaddr_to_nid memory_add_physaddr_to_nid
+#endif
+
+#define INIT_MEMBLOCK_RESERVED_REGIONS (INIT_MEMBLOCK_REGIONS + NR_CPUS)
+
+#endif /* _LOONGARCH_SPARSEMEM_H */
diff --git a/arch/loongarch/include/asm/tlb.h b/arch/loongarch/include/asm/tlb.h
new file mode 100644
index 000000000000..4f629ae9d5a9
--- /dev/null
+++ b/arch/loongarch/include/asm/tlb.h
@@ -0,0 +1,180 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_TLB_H
+#define __ASM_TLB_H
+
+#include <linux/mm_types.h>
+#include <asm/cpu-features.h>
+#include <asm/loongarch.h>
+
+/*
+ * TLB Invalidate Flush
+ */
+static inline void tlbclr(void)
+{
+ __asm__ __volatile__("tlbclr");
+}
+
+static inline void tlbflush(void)
+{
+ __asm__ __volatile__("tlbflush");
+}
+
+/*
+ * TLB R/W operations.
+ */
+static inline void tlb_probe(void)
+{
+ __asm__ __volatile__("tlbsrch");
+}
+
+static inline void tlb_read(void)
+{
+ __asm__ __volatile__("tlbrd");
+}
+
+static inline void tlb_write_indexed(void)
+{
+ __asm__ __volatile__("tlbwr");
+}
+
+static inline void tlb_write_random(void)
+{
+ __asm__ __volatile__("tlbfill");
+}
+
+enum invtlb_ops {
+ /* Invalid all tlb */
+ INVTLB_ALL = 0x0,
+ /* Invalid current tlb */
+ INVTLB_CURRENT_ALL = 0x1,
+ /* Invalid all global=1 lines in current tlb */
+ INVTLB_CURRENT_GTRUE = 0x2,
+ /* Invalid all global=0 lines in current tlb */
+ INVTLB_CURRENT_GFALSE = 0x3,
+ /* Invalid global=0 and matched asid lines in current tlb */
+ INVTLB_GFALSE_AND_ASID = 0x4,
+ /* Invalid addr with global=0 and matched asid in current tlb */
+ INVTLB_ADDR_GFALSE_AND_ASID = 0x5,
+ /* Invalid addr with global=1 or matched asid in current tlb */
+ INVTLB_ADDR_GTRUE_OR_ASID = 0x6,
+ /* Invalid matched gid in guest tlb */
+ INVGTLB_GID = 0x9,
+ /* Invalid global=1, matched gid in guest tlb */
+ INVGTLB_GID_GTRUE = 0xa,
+ /* Invalid global=0, matched gid in guest tlb */
+ INVGTLB_GID_GFALSE = 0xb,
+ /* Invalid global=0, matched gid and asid in guest tlb */
+ INVGTLB_GID_GFALSE_ASID = 0xc,
+ /* Invalid global=0 , matched gid, asid and addr in guest tlb */
+ INVGTLB_GID_GFALSE_ASID_ADDR = 0xd,
+ /* Invalid global=1 , matched gid, asid and addr in guest tlb */
+ INVGTLB_GID_GTRUE_ASID_ADDR = 0xe,
+ /* Invalid all gid gva-->gpa guest tlb */
+ INVGTLB_ALLGID_GVA_TO_GPA = 0x10,
+ /* Invalid all gid gpa-->hpa tlb */
+ INVTLB_ALLGID_GPA_TO_HPA = 0x11,
+ /* Invalid all gid tlb, including gva-->gpa and gpa-->hpa */
+ INVTLB_ALLGID = 0x12,
+ /* Invalid matched gid gva-->gpa guest tlb */
+ INVGTLB_GID_GVA_TO_GPA = 0x13,
+ /* Invalid matched gid gpa-->hpa tlb */
+ INVTLB_GID_GPA_TO_HPA = 0x14,
+ /* Invalid matched gid tlb,including gva-->gpa and gpa-->hpa */
+ INVTLB_GID_ALL = 0x15,
+ /* Invalid matched gid and addr gpa-->hpa tlb */
+ INVTLB_GID_ADDR = 0x16,
+};
+
+/*
+ * invtlb op info addr
+ * (0x1 << 26) | (0x24 << 20) | (0x13 << 15) |
+ * (addr << 10) | (info << 5) | op
+ */
+static inline void invtlb(u32 op, u32 info, u64 addr)
+{
+ __asm__ __volatile__(
+ "parse_r addr,%0\n\t"
+ "parse_r info,%1\n\t"
+ ".word ((0x6498000) | (addr << 10) | (info << 5) | %2)\n\t"
+ :
+ : "r"(addr), "r"(info), "i"(op)
+ :
+ );
+}
+
+static inline void invtlb_addr(u32 op, u32 info, u64 addr)
+{
+ __asm__ __volatile__(
+ "parse_r addr,%0\n\t"
+ ".word ((0x6498000) | (addr << 10) | (0 << 5) | %1)\n\t"
+ :
+ : "r"(addr), "i"(op)
+ :
+ );
+}
+
+static inline void invtlb_info(u32 op, u32 info, u64 addr)
+{
+ __asm__ __volatile__(
+ "parse_r info,%0\n\t"
+ ".word ((0x6498000) | (0 << 10) | (info << 5) | %1)\n\t"
+ :
+ : "r"(info), "i"(op)
+ :
+ );
+}
+
+static inline void invtlb_all(u32 op, u32 info, u64 addr)
+{
+ __asm__ __volatile__(
+ ".word ((0x6498000) | (0 << 10) | (0 << 5) | %0)\n\t"
+ :
+ : "i"(op)
+ :
+ );
+}
+
+/*
+ * LoongArch doesn't need any special per-pte or per-vma handling, except
+ * we need to flush cache for area to be unmapped.
+ */
+#define tlb_start_vma(tlb, vma) \
+ do { \
+ if (!(tlb)->fullmm) \
+ flush_cache_range(vma, vma->vm_start, vma->vm_end); \
+ } while (0)
+#define tlb_end_vma(tlb, vma) do { } while (0)
+#define __tlb_remove_tlb_entry(tlb, ptep, address) do { } while (0)
+
+static void tlb_flush(struct mmu_gather *tlb);
+
+#define tlb_flush tlb_flush
+#include <asm-generic/tlb.h>
+
+static inline void tlb_flush(struct mmu_gather *tlb)
+{
+ struct vm_area_struct vma;
+
+ vma.vm_mm = tlb->mm;
+ vma.vm_flags = 0;
+ if (tlb->fullmm) {
+ flush_tlb_mm(tlb->mm);
+ return;
+ }
+
+ flush_tlb_range(&vma, tlb->start, tlb->end);
+}
+
+extern void handle_tlb_load(void);
+extern void handle_tlb_store(void);
+extern void handle_tlb_modify(void);
+extern void handle_tlb_refill(void);
+extern void handle_tlb_protect(void);
+
+extern void dump_tlb_all(void);
+extern void dump_tlb_regs(void);
+
+#endif /* __ASM_TLB_H */
diff --git a/arch/loongarch/include/asm/tlbflush.h b/arch/loongarch/include/asm/tlbflush.h
new file mode 100644
index 000000000000..36bd6d11dc2d
--- /dev/null
+++ b/arch/loongarch/include/asm/tlbflush.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_TLBFLUSH_H
+#define __ASM_TLBFLUSH_H
+
+#include <linux/mm.h>
+
+/*
+ * TLB flushing:
+ *
+ * - flush_tlb_all() flushes all processes TLB entries
+ * - flush_tlb_mm(mm) flushes the specified mm context TLB entries
+ * - flush_tlb_page(vma, vmaddr) flushes one page
+ * - flush_tlb_range(vma, start, end) flushes a range of pages
+ * - flush_tlb_kernel_range(start, end) flushes a range of kernel pages
+ */
+extern void local_flush_tlb_all(void);
+extern void local_flush_tlb_user(void);
+extern void local_flush_tlb_kernel(void);
+extern void local_flush_tlb_mm(struct mm_struct *mm);
+extern void local_flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end);
+extern void local_flush_tlb_kernel_range(unsigned long start, unsigned long end);
+extern void local_flush_tlb_page(struct vm_area_struct *vma, unsigned long page);
+extern void local_flush_tlb_one(unsigned long vaddr);
+
+#define flush_tlb_all() local_flush_tlb_all()
+#define flush_tlb_mm(mm) local_flush_tlb_mm(mm)
+#define flush_tlb_range(vma, vmaddr, end) local_flush_tlb_range(vma, vmaddr, end)
+#define flush_tlb_kernel_range(vmaddr, end) local_flush_tlb_kernel_range(vmaddr, end)
+#define flush_tlb_page(vma, page) local_flush_tlb_page(vma, page)
+#define flush_tlb_one(vaddr) local_flush_tlb_one(vaddr)
+
+#endif /* __ASM_TLBFLUSH_H */
diff --git a/arch/loongarch/include/asm/vmalloc.h b/arch/loongarch/include/asm/vmalloc.h
new file mode 100644
index 000000000000..965a0d41ac2d
--- /dev/null
+++ b/arch/loongarch/include/asm/vmalloc.h
@@ -0,0 +1,4 @@
+#ifndef _ASM_LOONGARCH_VMALLOC_H
+#define _ASM_LOONGARCH_VMALLOC_H
+
+#endif /* _ASM_LOONGARCH_VMALLOC_H */
diff --git a/arch/loongarch/mm/cache.c b/arch/loongarch/mm/cache.c
new file mode 100644
index 000000000000..9e5ce5aa73f7
--- /dev/null
+++ b/arch/loongarch/mm/cache.c
@@ -0,0 +1,141 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1994 - 2003, 06, 07 by Ralf Baechle ([email protected])
+ * Copyright (C) 2007 MIPS Technologies, Inc.
+ */
+#include <linux/export.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <linux/highmem.h>
+#include <linux/kernel.h>
+#include <linux/linkage.h>
+#include <linux/mm.h>
+#include <linux/sched.h>
+#include <linux/syscalls.h>
+
+#include <asm/cacheflush.h>
+#include <asm/cpu.h>
+#include <asm/cpu-features.h>
+#include <asm/dma.h>
+#include <asm/loongarch.h>
+#include <asm/processor.h>
+#include <asm/setup.h>
+
+/*
+ * LoongArch maintains ICache/DCache coherency by hardware,
+ * we just need "ibar" to avoid instruction hazard here.
+ */
+void local_flush_icache_range(unsigned long start, unsigned long end)
+{
+ asm volatile ("\tibar 0\n"::);
+}
+EXPORT_SYMBOL(local_flush_icache_range);
+
+void cache_error_setup(void)
+{
+ extern char __weak except_vec_cex;
+ set_merr_handler(0x0, &except_vec_cex, 0x80);
+}
+
+static unsigned long icache_size __read_mostly;
+static unsigned long dcache_size __read_mostly;
+static unsigned long vcache_size __read_mostly;
+static unsigned long scache_size __read_mostly;
+
+static char *way_string[] = { NULL, "direct mapped", "2-way",
+ "3-way", "4-way", "5-way", "6-way", "7-way", "8-way",
+ "9-way", "10-way", "11-way", "12-way",
+ "13-way", "14-way", "15-way", "16-way",
+};
+
+static void probe_pcache(void)
+{
+ struct cpuinfo_loongarch *c = ¤t_cpu_data;
+ unsigned int lsize, sets, ways;
+ unsigned int config;
+
+ config = read_cpucfg(LOONGARCH_CPUCFG17);
+ lsize = 1 << ((config & CPUCFG17_L1I_SIZE_M) >> CPUCFG17_L1I_SIZE);
+ sets = 1 << ((config & CPUCFG17_L1I_SETS_M) >> CPUCFG17_L1I_SETS);
+ ways = ((config & CPUCFG17_L1I_WAYS_M) >> CPUCFG17_L1I_WAYS) + 1;
+
+ c->icache.linesz = lsize;
+ c->icache.sets = sets;
+ c->icache.ways = ways;
+ icache_size = sets * ways * lsize;
+ c->icache.waysize = icache_size / c->icache.ways;
+
+ config = read_cpucfg(LOONGARCH_CPUCFG18);
+ lsize = 1 << ((config & CPUCFG18_L1D_SIZE_M) >> CPUCFG18_L1D_SIZE);
+ sets = 1 << ((config & CPUCFG18_L1D_SETS_M) >> CPUCFG18_L1D_SETS);
+ ways = ((config & CPUCFG18_L1D_WAYS_M) >> CPUCFG18_L1D_WAYS) + 1;
+
+ c->dcache.linesz = lsize;
+ c->dcache.sets = sets;
+ c->dcache.ways = ways;
+ dcache_size = sets * ways * lsize;
+ c->dcache.waysize = dcache_size / c->dcache.ways;
+
+ c->options |= LOONGARCH_CPU_PREFETCH;
+
+ pr_info("Primary instruction cache %ldkB, %s, %s, linesize %d bytes.\n",
+ icache_size >> 10, way_string[c->icache.ways], "VIPT", c->icache.linesz);
+
+ pr_info("Primary data cache %ldkB, %s, %s, %s, linesize %d bytes\n",
+ dcache_size >> 10, way_string[c->dcache.ways], "VIPT", "no aliases", c->dcache.linesz);
+}
+
+static void probe_vcache(void)
+{
+ struct cpuinfo_loongarch *c = ¤t_cpu_data;
+ unsigned int lsize, sets, ways;
+ unsigned int config;
+
+ config = read_cpucfg(LOONGARCH_CPUCFG19);
+ lsize = 1 << ((config & CPUCFG19_L2_SIZE_M) >> CPUCFG19_L2_SIZE);
+ sets = 1 << ((config & CPUCFG19_L2_SETS_M) >> CPUCFG19_L2_SETS);
+ ways = ((config & CPUCFG19_L2_WAYS_M) >> CPUCFG19_L2_WAYS) + 1;
+
+ c->vcache.linesz = lsize;
+ c->vcache.sets = sets;
+ c->vcache.ways = ways;
+ vcache_size = lsize * sets * ways;
+ c->vcache.waysize = vcache_size / c->vcache.ways;
+
+ pr_info("Unified victim cache %ldkB %s, linesize %d bytes.\n",
+ vcache_size >> 10, way_string[c->vcache.ways], c->vcache.linesz);
+}
+
+static void probe_scache(void)
+{
+ struct cpuinfo_loongarch *c = ¤t_cpu_data;
+ unsigned int lsize, sets, ways;
+ unsigned int config;
+
+ config = read_cpucfg(LOONGARCH_CPUCFG20);
+ lsize = 1 << ((config & CPUCFG20_L3_SIZE_M) >> CPUCFG20_L3_SIZE);
+ sets = 1 << ((config & CPUCFG20_L3_SETS_M) >> CPUCFG20_L3_SETS);
+ ways = ((config & CPUCFG20_L3_WAYS_M) >> CPUCFG20_L3_WAYS) + 1;
+
+ c->scache.linesz = lsize;
+ c->scache.sets = sets;
+ c->scache.ways = ways;
+ /* 4 cores. scaches are shared */
+ scache_size = lsize * sets * ways;
+ c->scache.waysize = scache_size / c->scache.ways;
+
+ pr_info("Unified secondary cache %ldkB %s, linesize %d bytes.\n",
+ scache_size >> 10, way_string[c->scache.ways], c->scache.linesz);
+}
+
+void cpu_cache_init(void)
+{
+ probe_pcache();
+ probe_vcache();
+ probe_scache();
+
+ shm_align_mask = PAGE_SIZE - 1;
+}
diff --git a/arch/loongarch/mm/extable.c b/arch/loongarch/mm/extable.c
new file mode 100644
index 000000000000..bc20988f2b87
--- /dev/null
+++ b/arch/loongarch/mm/extable.c
@@ -0,0 +1,22 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/extable.h>
+#include <linux/spinlock.h>
+#include <asm/branch.h>
+#include <linux/uaccess.h>
+
+int fixup_exception(struct pt_regs *regs)
+{
+ const struct exception_table_entry *fixup;
+
+ fixup = search_exception_tables(exception_era(regs));
+ if (fixup) {
+ regs->csr_era = fixup->fixup;
+
+ return 1;
+ }
+
+ return 0;
+}
diff --git a/arch/loongarch/mm/fault.c b/arch/loongarch/mm/fault.c
new file mode 100644
index 000000000000..605579b19a00
--- /dev/null
+++ b/arch/loongarch/mm/fault.c
@@ -0,0 +1,261 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1995 - 2000 by Ralf Baechle
+ */
+#include <linux/context_tracking.h>
+#include <linux/signal.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/entry-common.h>
+#include <linux/errno.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/ptrace.h>
+#include <linux/ratelimit.h>
+#include <linux/mman.h>
+#include <linux/mm.h>
+#include <linux/smp.h>
+#include <linux/kdebug.h>
+#include <linux/kprobes.h>
+#include <linux/perf_event.h>
+#include <linux/uaccess.h>
+
+#include <asm/branch.h>
+#include <asm/mmu_context.h>
+#include <asm/ptrace.h>
+
+int show_unhandled_signals = 1;
+
+static void __kprobes no_context(struct pt_regs *regs, unsigned long address)
+{
+ const int field = sizeof(unsigned long) * 2;
+
+ /* Are we prepared to handle this kernel fault? */
+ if (fixup_exception(regs))
+ return;
+
+ /*
+ * Oops. The kernel tried to access some bad page. We'll have to
+ * terminate things with extreme prejudice.
+ */
+ bust_spinlocks(1);
+
+ pr_alert("CPU %d Unable to handle kernel paging request at "
+ "virtual address %0*lx, era == %0*lx, ra == %0*lx\n",
+ raw_smp_processor_id(), field, address, field, regs->csr_era,
+ field, regs->regs[1]);
+ die("Oops", regs);
+}
+
+static void __kprobes do_out_of_memory(struct pt_regs *regs, unsigned long address)
+{
+ /*
+ * We ran out of memory, call the OOM killer, and return the userspace
+ * (which will retry the fault, or kill us if we got oom-killed).
+ */
+ if (!user_mode(regs)) {
+ no_context(regs, address);
+ return;
+ }
+ pagefault_out_of_memory();
+}
+
+static void __kprobes do_sigbus(struct pt_regs *regs,
+ unsigned long write, unsigned long address, int si_code)
+{
+ /* Kernel mode? Handle exceptions or die */
+ if (!user_mode(regs)) {
+ no_context(regs, address);
+ return;
+ }
+
+ /*
+ * Send a sigbus, regardless of whether we were in kernel
+ * or user mode.
+ */
+ current->thread.csr_badvaddr = address;
+ current->thread.trap_nr = read_csr_excode();
+ force_sig_fault(SIGBUS, BUS_ADRERR, (void __user *)address);
+}
+
+static void __kprobes do_sigsegv(struct pt_regs *regs,
+ unsigned long write, unsigned long address, int si_code)
+{
+ const int field = sizeof(unsigned long) * 2;
+ static DEFINE_RATELIMIT_STATE(ratelimit_state, 5 * HZ, 10);
+
+ /* Kernel mode? Handle exceptions or die */
+ if (!user_mode(regs)) {
+ no_context(regs, address);
+ return;
+ }
+
+ /* User mode accesses just cause a SIGSEGV */
+ current->thread.csr_badvaddr = address;
+ if (!write)
+ current->thread.error_code = 1;
+ else
+ current->thread.error_code = 2;
+ current->thread.trap_nr = read_csr_excode();
+
+ if (show_unhandled_signals &&
+ unhandled_signal(current, SIGSEGV) && __ratelimit(&ratelimit_state)) {
+ pr_info("do_page_fault(): sending SIGSEGV to %s for invalid %s %0*lx\n",
+ current->comm,
+ write ? "write access to" : "read access from",
+ field, address);
+ pr_info("era = %0*lx in", field,
+ (unsigned long) regs->csr_era);
+ print_vma_addr(KERN_CONT " ", regs->csr_era);
+ pr_cont("\n");
+ pr_info("ra = %0*lx in", field,
+ (unsigned long) regs->regs[1]);
+ print_vma_addr(KERN_CONT " ", regs->regs[1]);
+ pr_cont("\n");
+ }
+ force_sig_fault(SIGSEGV, si_code, (void __user *)address);
+}
+
+/*
+ * This routine handles page faults. It determines the address,
+ * and the problem, and then passes it off to one of the appropriate
+ * routines.
+ */
+static void __kprobes __do_page_fault(struct pt_regs *regs,
+ unsigned long write, unsigned long address)
+{
+ int si_code = SEGV_MAPERR;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
+ struct task_struct *tsk = current;
+ struct mm_struct *mm = tsk->mm;
+ struct vm_area_struct *vma = NULL;
+ vm_fault_t fault;
+
+ /*
+ * We fault-in kernel-space virtual memory on-demand. The
+ * 'reference' page table is init_mm.pgd.
+ *
+ * NOTE! We MUST NOT take any locks for this case. We may
+ * be in an interrupt or a critical region, and should
+ * only copy the information from the master page table,
+ * nothing more.
+ */
+ if (address & __UA_LIMIT) {
+ if (!user_mode(regs))
+ no_context(regs, address);
+ else
+ do_sigsegv(regs, write, address, si_code);
+ return;
+ }
+
+ /*
+ * If we're in an interrupt or have no user
+ * context, we must not take the fault..
+ */
+ if (faulthandler_disabled() || !mm) {
+ do_sigsegv(regs, write, address, si_code);
+ return;
+ }
+
+ if (user_mode(regs))
+ flags |= FAULT_FLAG_USER;
+
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address);
+retry:
+ mmap_read_lock(mm);
+ vma = find_vma(mm, address);
+ if (!vma)
+ goto bad_area;
+ if (vma->vm_start <= address)
+ goto good_area;
+ if (!(vma->vm_flags & VM_GROWSDOWN))
+ goto bad_area;
+ if (!expand_stack(vma, address))
+ goto good_area;
+/*
+ * Something tried to access memory that isn't in our memory map..
+ * Fix it, but check if it's kernel or user first..
+ */
+bad_area:
+ mmap_read_unlock(mm);
+ do_sigsegv(regs, write, address, si_code);
+ return;
+
+/*
+ * Ok, we have a good vm_area for this memory access, so
+ * we can handle it..
+ */
+good_area:
+ si_code = SEGV_ACCERR;
+
+ if (write) {
+ flags |= FAULT_FLAG_WRITE;
+ if (!(vma->vm_flags & VM_WRITE))
+ goto bad_area;
+ } else {
+ if (!(vma->vm_flags & VM_READ) && address != exception_era(regs))
+ goto bad_area;
+ if (!(vma->vm_flags & VM_EXEC) && address == exception_era(regs))
+ goto bad_area;
+ }
+
+ /*
+ * If for any reason at all we couldn't handle the fault,
+ * make sure we exit gracefully rather than endlessly redo
+ * the fault.
+ */
+ fault = handle_mm_fault(vma, address, flags, regs);
+
+ if (fault_signal_pending(fault, regs)) {
+ if (!user_mode(regs))
+ no_context(regs, address);
+ return;
+ }
+
+ if (unlikely(fault & VM_FAULT_RETRY)) {
+ flags |= FAULT_FLAG_TRIED;
+
+ /*
+ * No need to mmap_read_unlock(mm) as we would
+ * have already released it in __lock_page_or_retry
+ * in mm/filemap.c.
+ */
+ goto retry;
+ }
+ if (unlikely(fault & VM_FAULT_ERROR)) {
+ mmap_read_unlock(mm);
+ if (fault & VM_FAULT_OOM) {
+ do_out_of_memory(regs, address);
+ return;
+ } else if (fault & VM_FAULT_SIGSEGV) {
+ do_sigsegv(regs, write, address, si_code);
+ return;
+ } else if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|VM_FAULT_HWPOISON_LARGE)) {
+ do_sigbus(regs, write, address, si_code);
+ return;
+ }
+ BUG();
+ }
+
+ mmap_read_unlock(mm);
+}
+
+asmlinkage void __kprobes do_page_fault(struct pt_regs *regs,
+ unsigned long write, unsigned long address)
+{
+ irqentry_state_t state = irqentry_enter(regs);
+
+ /* Enable interrupt if enabled in parent context */
+ if (likely(regs->csr_prmd & CSR_PRMD_PIE))
+ local_irq_enable();
+
+ __do_page_fault(regs, write, address);
+
+ local_irq_disable();
+
+ irqentry_exit(regs, state);
+}
diff --git a/arch/loongarch/mm/hugetlbpage.c b/arch/loongarch/mm/hugetlbpage.c
new file mode 100644
index 000000000000..ba138117b124
--- /dev/null
+++ b/arch/loongarch/mm/hugetlbpage.c
@@ -0,0 +1,87 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/fs.h>
+#include <linux/mm.h>
+#include <linux/hugetlb.h>
+#include <linux/pagemap.h>
+#include <linux/err.h>
+#include <linux/sysctl.h>
+#include <asm/mman.h>
+#include <asm/tlb.h>
+#include <asm/tlbflush.h>
+
+pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
+ unsigned long addr, unsigned long sz)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pte_t *pte = NULL;
+
+ pgd = pgd_offset(mm, addr);
+ p4d = p4d_alloc(mm, pgd, addr);
+ pud = pud_alloc(mm, p4d, addr);
+ if (pud)
+ pte = (pte_t *)pmd_alloc(mm, pud, addr);
+
+ return pte;
+}
+
+pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr,
+ unsigned long sz)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd = NULL;
+
+ pgd = pgd_offset(mm, addr);
+ if (pgd_present(*pgd)) {
+ p4d = p4d_offset(pgd, addr);
+ if (p4d_present(*p4d)) {
+ pud = pud_offset(p4d, addr);
+ if (pud_present(*pud))
+ pmd = pmd_offset(pud, addr);
+ }
+ }
+ return (pte_t *) pmd;
+}
+
+/*
+ * This function checks for proper alignment of input addr and len parameters.
+ */
+int is_aligned_hugepage_range(unsigned long addr, unsigned long len)
+{
+ if (len & ~HPAGE_MASK)
+ return -EINVAL;
+ if (addr & ~HPAGE_MASK)
+ return -EINVAL;
+ return 0;
+}
+
+int pmd_huge(pmd_t pmd)
+{
+ return (pmd_val(pmd) & _PAGE_HUGE) != 0;
+}
+
+int pud_huge(pud_t pud)
+{
+ return (pud_val(pud) & _PAGE_HUGE) != 0;
+}
+
+uint64_t pmd_to_entrylo(unsigned long pmd_val)
+{
+ uint64_t val;
+ /* PMD as PTE. Must be huge page */
+ if (!pmd_huge(__pmd(pmd_val)))
+ panic("%s", __func__);
+
+ val = pmd_val ^ _PAGE_HUGE;
+ val |= ((val & _PAGE_HGLOBAL) >>
+ (_PAGE_HGLOBAL_SHIFT - _PAGE_GLOBAL_SHIFT));
+
+ return val;
+}
diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c
new file mode 100644
index 000000000000..afd6634ce171
--- /dev/null
+++ b/arch/loongarch/mm/init.c
@@ -0,0 +1,165 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/init.h>
+#include <linux/export.h>
+#include <linux/signal.h>
+#include <linux/sched.h>
+#include <linux/smp.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/pagemap.h>
+#include <linux/memblock.h>
+#include <linux/memremap.h>
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/highmem.h>
+#include <linux/swap.h>
+#include <linux/proc_fs.h>
+#include <linux/pfn.h>
+#include <linux/hardirq.h>
+#include <linux/gfp.h>
+#include <linux/initrd.h>
+#include <linux/mmzone.h>
+
+#include <asm/asm-offsets.h>
+#include <asm/bootinfo.h>
+#include <asm/cpu.h>
+#include <asm/dma.h>
+#include <asm/mmu_context.h>
+#include <asm/sections.h>
+#include <asm/pgtable.h>
+#include <asm/pgalloc.h>
+#include <asm/tlb.h>
+
+/*
+ * We have up to 8 empty zeroed pages so we can map one of the right colour
+ * when needed. Since page is never written to after the initialization we
+ * don't have to care about aliases on other CPUs.
+ */
+unsigned long empty_zero_page, zero_page_mask;
+EXPORT_SYMBOL_GPL(empty_zero_page);
+EXPORT_SYMBOL(zero_page_mask);
+
+void setup_zero_pages(void)
+{
+ unsigned int order, i;
+ struct page *page;
+
+ order = 0;
+
+ empty_zero_page = __get_free_pages(GFP_KERNEL | __GFP_ZERO, order);
+ if (!empty_zero_page)
+ panic("Oh boy, that early out of memory?");
+
+ page = virt_to_page((void *)empty_zero_page);
+ split_page(page, order);
+ for (i = 0; i < (1 << order); i++, page++)
+ mark_page_reserved(page);
+
+ zero_page_mask = ((PAGE_SIZE << order) - 1) & PAGE_MASK;
+}
+
+void copy_user_highpage(struct page *to, struct page *from,
+ unsigned long vaddr, struct vm_area_struct *vma)
+{
+ void *vfrom, *vto;
+
+ vto = kmap_atomic(to);
+ vfrom = kmap_atomic(from);
+ copy_page(vto, vfrom);
+ kunmap_atomic(vfrom);
+ kunmap_atomic(vto);
+ /* Make sure this page is cleared on other CPU's too before using it */
+ smp_wmb();
+}
+
+int __ref page_is_ram(unsigned long pfn)
+{
+ unsigned long addr = PFN_PHYS(pfn);
+
+ return memblock_is_memory(addr) && !memblock_is_reserved(addr);
+}
+
+void __init paging_init(void)
+{
+ unsigned long max_zone_pfns[MAX_NR_ZONES];
+
+#ifdef CONFIG_ZONE_DMA
+ max_zone_pfns[ZONE_DMA] = MAX_DMA_PFN;
+#endif
+#ifdef CONFIG_ZONE_DMA32
+ max_zone_pfns[ZONE_DMA32] = MAX_DMA32_PFN;
+#endif
+ max_zone_pfns[ZONE_NORMAL] = max_low_pfn;
+
+ free_area_init(max_zone_pfns);
+}
+
+void __init mem_init(void)
+{
+ max_mapnr = max_low_pfn;
+ high_memory = (void *) __va(max_low_pfn << PAGE_SHIFT);
+
+ memblock_free_all();
+ setup_zero_pages(); /* Setup zeroed pages. */
+}
+
+void __ref free_initmem(void)
+{
+ free_initmem_default(POISON_FREE_INITMEM);
+}
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+int arch_add_memory(int nid, u64 start, u64 size, struct mhp_params *params)
+{
+ unsigned long start_pfn = start >> PAGE_SHIFT;
+ unsigned long nr_pages = size >> PAGE_SHIFT;
+ int ret;
+
+ ret = __add_pages(nid, start_pfn, nr_pages, params);
+
+ if (ret)
+ pr_warn("%s: Problem encountered in __add_pages() as ret=%d\n",
+ __func__, ret);
+
+ return ret;
+}
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+void arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap)
+{
+ unsigned long start_pfn = start >> PAGE_SHIFT;
+ unsigned long nr_pages = size >> PAGE_SHIFT;
+ struct page *page = pfn_to_page(start_pfn);
+
+ /* With altmap the first mapped page is offset from @start */
+ if (altmap)
+ page += vmem_altmap_offset(altmap);
+ __remove_pages(start_pfn, nr_pages, altmap);
+}
+#endif
+#endif
+
+/*
+ * Align swapper_pg_dir in to 64K, allows its address to be loaded
+ * with a single LUI instruction in the TLB handlers. If we used
+ * __aligned(64K), its size would get rounded up to the alignment
+ * size, and waste space. So we place it in its own section and align
+ * it in the linker script.
+ */
+pgd_t swapper_pg_dir[_PTRS_PER_PGD] __section(".bss..swapper_pg_dir");
+
+pgd_t invalid_pg_dir[_PTRS_PER_PGD] __page_aligned_bss;
+#ifndef __PAGETABLE_PUD_FOLDED
+pud_t invalid_pud_table[PTRS_PER_PUD] __page_aligned_bss;
+#endif
+#ifndef __PAGETABLE_PMD_FOLDED
+pmd_t invalid_pmd_table[PTRS_PER_PMD] __page_aligned_bss;
+EXPORT_SYMBOL_GPL(invalid_pmd_table);
+#endif
+pte_t invalid_pte_table[PTRS_PER_PTE] __page_aligned_bss;
+EXPORT_SYMBOL(invalid_pte_table);
diff --git a/arch/loongarch/mm/ioremap.c b/arch/loongarch/mm/ioremap.c
new file mode 100644
index 000000000000..73b0980ab6f5
--- /dev/null
+++ b/arch/loongarch/mm/ioremap.c
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/io.h>
+
+void __init __iomem *early_ioremap(u64 phys_addr, unsigned long size)
+{
+ return ((void __iomem *)TO_CACHE(phys_addr));
+}
+
+void __init early_iounmap(void __iomem *addr, unsigned long size)
+{
+
+}
+
+void *early_memremap_ro(resource_size_t phys_addr, unsigned long size)
+{
+ return early_memremap(phys_addr, size);
+}
+
+void *early_memremap_prot(resource_size_t phys_addr, unsigned long size,
+ unsigned long prot_val)
+{
+ return early_memremap(phys_addr, size);
+}
diff --git a/arch/loongarch/mm/maccess.c b/arch/loongarch/mm/maccess.c
new file mode 100644
index 000000000000..58173842c6be
--- /dev/null
+++ b/arch/loongarch/mm/maccess.c
@@ -0,0 +1,10 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+#include <linux/uaccess.h>
+#include <linux/kernel.h>
+
+bool copy_from_kernel_nofault_allowed(const void *unsafe_src, size_t size)
+{
+ /* highest bit set means kernel space */
+ return (unsigned long)unsafe_src >> (BITS_PER_LONG - 1);
+}
diff --git a/arch/loongarch/mm/mmap.c b/arch/loongarch/mm/mmap.c
new file mode 100644
index 000000000000..52e40f0ba732
--- /dev/null
+++ b/arch/loongarch/mm/mmap.c
@@ -0,0 +1,125 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/compiler.h>
+#include <linux/elf-randomize.h>
+#include <linux/errno.h>
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/export.h>
+#include <linux/personality.h>
+#include <linux/random.h>
+#include <linux/sched/signal.h>
+#include <linux/sched/mm.h>
+
+unsigned long shm_align_mask = PAGE_SIZE - 1; /* Sane caches */
+EXPORT_SYMBOL(shm_align_mask);
+
+#define COLOUR_ALIGN(addr, pgoff) \
+ ((((addr) + shm_align_mask) & ~shm_align_mask) + \
+ (((pgoff) << PAGE_SHIFT) & shm_align_mask))
+
+enum mmap_allocation_direction {UP, DOWN};
+
+static unsigned long arch_get_unmapped_area_common(struct file *filp,
+ unsigned long addr0, unsigned long len, unsigned long pgoff,
+ unsigned long flags, enum mmap_allocation_direction dir)
+{
+ struct mm_struct *mm = current->mm;
+ struct vm_area_struct *vma;
+ unsigned long addr = addr0;
+ int do_color_align;
+ struct vm_unmapped_area_info info;
+
+ if (unlikely(len > TASK_SIZE))
+ return -ENOMEM;
+
+ if (flags & MAP_FIXED) {
+ /* Even MAP_FIXED mappings must reside within TASK_SIZE */
+ if (TASK_SIZE - len < addr)
+ return -EINVAL;
+
+ /*
+ * We do not accept a shared mapping if it would violate
+ * cache aliasing constraints.
+ */
+ if ((flags & MAP_SHARED) &&
+ ((addr - (pgoff << PAGE_SHIFT)) & shm_align_mask))
+ return -EINVAL;
+ return addr;
+ }
+
+ do_color_align = 0;
+ if (filp || (flags & MAP_SHARED))
+ do_color_align = 1;
+
+ /* requesting a specific address */
+ if (addr) {
+ if (do_color_align)
+ addr = COLOUR_ALIGN(addr, pgoff);
+ else
+ addr = PAGE_ALIGN(addr);
+
+ vma = find_vma(mm, addr);
+ if (TASK_SIZE - len >= addr &&
+ (!vma || addr + len <= vm_start_gap(vma)))
+ return addr;
+ }
+
+ info.length = len;
+ info.align_mask = do_color_align ? (PAGE_MASK & shm_align_mask) : 0;
+ info.align_offset = pgoff << PAGE_SHIFT;
+
+ if (dir == DOWN) {
+ info.flags = VM_UNMAPPED_AREA_TOPDOWN;
+ info.low_limit = PAGE_SIZE;
+ info.high_limit = mm->mmap_base;
+ addr = vm_unmapped_area(&info);
+
+ if (!(addr & ~PAGE_MASK))
+ return addr;
+
+ /*
+ * A failed mmap() very likely causes application failure,
+ * so fall back to the bottom-up function here. This scenario
+ * can happen with large stack limits and large mmap()
+ * allocations.
+ */
+ }
+
+ info.flags = 0;
+ info.low_limit = mm->mmap_base;
+ info.high_limit = TASK_SIZE;
+ return vm_unmapped_area(&info);
+}
+
+unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr0,
+ unsigned long len, unsigned long pgoff, unsigned long flags)
+{
+ return arch_get_unmapped_area_common(filp,
+ addr0, len, pgoff, flags, UP);
+}
+
+/*
+ * There is no need to export this but sched.h declares the function as
+ * extern so making it static here results in an error.
+ */
+unsigned long arch_get_unmapped_area_topdown(struct file *filp,
+ unsigned long addr0, unsigned long len, unsigned long pgoff,
+ unsigned long flags)
+{
+ return arch_get_unmapped_area_common(filp,
+ addr0, len, pgoff, flags, DOWN);
+}
+
+int __virt_addr_valid(volatile void *kaddr)
+{
+ unsigned long vaddr = (unsigned long)kaddr;
+
+ if ((vaddr < PAGE_OFFSET) || (vaddr >= vm_map_base))
+ return 0;
+
+ return pfn_valid(PFN_DOWN(virt_to_phys(kaddr)));
+}
+EXPORT_SYMBOL_GPL(__virt_addr_valid);
diff --git a/arch/loongarch/mm/page.S b/arch/loongarch/mm/page.S
new file mode 100644
index 000000000000..ddc78ab33c7b
--- /dev/null
+++ b/arch/loongarch/mm/page.S
@@ -0,0 +1,84 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/linkage.h>
+#include <asm/asm.h>
+#include <asm/export.h>
+#include <asm/page.h>
+#include <asm/regdef.h>
+
+ .align 5
+SYM_FUNC_START(clear_page)
+ lu12i.w t0, 1 << (PAGE_SHIFT - 12)
+ add.d t0, t0, a0
+1:
+ st.d zero, a0, 0
+ st.d zero, a0, 8
+ st.d zero, a0, 16
+ st.d zero, a0, 24
+ st.d zero, a0, 32
+ st.d zero, a0, 40
+ st.d zero, a0, 48
+ st.d zero, a0, 56
+ addi.d a0, a0, 128
+ st.d zero, a0, -64
+ st.d zero, a0, -56
+ st.d zero, a0, -48
+ st.d zero, a0, -40
+ st.d zero, a0, -32
+ st.d zero, a0, -24
+ st.d zero, a0, -16
+ st.d zero, a0, -8
+ bne t0, a0, 1b
+
+ jirl $r0, ra, 0
+SYM_FUNC_END(clear_page)
+EXPORT_SYMBOL(clear_page)
+
+.align 5
+SYM_FUNC_START(copy_page)
+ lu12i.w t8, 1 << (PAGE_SHIFT - 12)
+ add.d t8, t8, a0
+1:
+ ld.d t0, a1, 0
+ ld.d t1, a1, 8
+ ld.d t2, a1, 16
+ ld.d t3, a1, 24
+ ld.d t4, a1, 32
+ ld.d t5, a1, 40
+ ld.d t6, a1, 48
+ ld.d t7, a1, 56
+
+ st.d t0, a0, 0
+ st.d t1, a0, 8
+ ld.d t0, a1, 64
+ ld.d t1, a1, 72
+ st.d t2, a0, 16
+ st.d t3, a0, 24
+ ld.d t2, a1, 80
+ ld.d t3, a1, 88
+ st.d t4, a0, 32
+ st.d t5, a0, 40
+ ld.d t4, a1, 96
+ ld.d t5, a1, 104
+ st.d t6, a0, 48
+ st.d t7, a0, 56
+ ld.d t6, a1, 112
+ ld.d t7, a1, 120
+ addi.d a0, a0, 128
+ addi.d a1, a1, 128
+
+ st.d t0, a0, -64
+ st.d t1, a0, -56
+ st.d t2, a0, -48
+ st.d t3, a0, -40
+ st.d t4, a0, -32
+ st.d t5, a0, -24
+ st.d t6, a0, -16
+ st.d t7, a0, -8
+
+ bne t8, a0, 1b
+ jirl $r0, ra, 0
+SYM_FUNC_END(copy_page)
+EXPORT_SYMBOL(copy_page)
diff --git a/arch/loongarch/mm/pgtable.c b/arch/loongarch/mm/pgtable.c
new file mode 100644
index 000000000000..0569647152e9
--- /dev/null
+++ b/arch/loongarch/mm/pgtable.c
@@ -0,0 +1,130 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/init.h>
+#include <linux/export.h>
+#include <linux/mm.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+#include <asm/tlbflush.h>
+
+pgd_t *pgd_alloc(struct mm_struct *mm)
+{
+ pgd_t *ret, *init;
+
+ ret = (pgd_t *) __get_free_pages(GFP_KERNEL, PGD_ORDER);
+ if (ret) {
+ init = pgd_offset(&init_mm, 0UL);
+ pgd_init((unsigned long)ret);
+ memcpy(ret + USER_PTRS_PER_PGD, init + USER_PTRS_PER_PGD,
+ (PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t));
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(pgd_alloc);
+
+void pgd_init(unsigned long page)
+{
+ unsigned long *p, *end;
+ unsigned long entry;
+
+#if !defined(__PAGETABLE_PUD_FOLDED)
+ entry = (unsigned long)invalid_pud_table;
+#elif !defined(__PAGETABLE_PMD_FOLDED)
+ entry = (unsigned long)invalid_pmd_table;
+#else
+ entry = (unsigned long)invalid_pte_table;
+#endif
+
+ p = (unsigned long *) page;
+ end = p + PTRS_PER_PGD;
+
+ do {
+ p[0] = entry;
+ p[1] = entry;
+ p[2] = entry;
+ p[3] = entry;
+ p[4] = entry;
+ p += 8;
+ p[-3] = entry;
+ p[-2] = entry;
+ p[-1] = entry;
+ } while (p != end);
+}
+EXPORT_SYMBOL_GPL(pgd_init);
+
+#ifndef __PAGETABLE_PMD_FOLDED
+void pmd_init(unsigned long addr, unsigned long pagetable)
+{
+ unsigned long *p, *end;
+
+ p = (unsigned long *) addr;
+ end = p + PTRS_PER_PMD;
+
+ do {
+ p[0] = pagetable;
+ p[1] = pagetable;
+ p[2] = pagetable;
+ p[3] = pagetable;
+ p[4] = pagetable;
+ p += 8;
+ p[-3] = pagetable;
+ p[-2] = pagetable;
+ p[-1] = pagetable;
+ } while (p != end);
+}
+EXPORT_SYMBOL_GPL(pmd_init);
+#endif
+
+#ifndef __PAGETABLE_PUD_FOLDED
+void pud_init(unsigned long addr, unsigned long pagetable)
+{
+ unsigned long *p, *end;
+
+ p = (unsigned long *)addr;
+ end = p + PTRS_PER_PUD;
+
+ do {
+ p[0] = pagetable;
+ p[1] = pagetable;
+ p[2] = pagetable;
+ p[3] = pagetable;
+ p[4] = pagetable;
+ p += 8;
+ p[-3] = pagetable;
+ p[-2] = pagetable;
+ p[-1] = pagetable;
+ } while (p != end);
+}
+#endif
+
+pmd_t mk_pmd(struct page *page, pgprot_t prot)
+{
+ pmd_t pmd;
+
+ pmd_val(pmd) = (page_to_pfn(page) << _PFN_SHIFT) | pgprot_val(prot);
+
+ return pmd;
+}
+
+void set_pmd_at(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, pmd_t pmd)
+{
+ *pmdp = pmd;
+ flush_tlb_all();
+}
+
+void __init pagetable_init(void)
+{
+ /* Initialize the entire pgd. */
+ pgd_init((unsigned long)swapper_pg_dir);
+ pgd_init((unsigned long)invalid_pg_dir);
+#ifndef __PAGETABLE_PUD_FOLDED
+ pud_init((unsigned long)invalid_pud_table, (unsigned long)invalid_pmd_table);
+#endif
+#ifndef __PAGETABLE_PMD_FOLDED
+ pmd_init((unsigned long)invalid_pmd_table, (unsigned long)invalid_pte_table);
+#endif
+}
diff --git a/arch/loongarch/mm/tlb.c b/arch/loongarch/mm/tlb.c
new file mode 100644
index 000000000000..78dd328cec77
--- /dev/null
+++ b/arch/loongarch/mm/tlb.c
@@ -0,0 +1,282 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/init.h>
+#include <linux/sched.h>
+#include <linux/smp.h>
+#include <linux/mm.h>
+#include <linux/hugetlb.h>
+#include <linux/export.h>
+
+#include <asm/cpu.h>
+#include <asm/bootinfo.h>
+#include <asm/mmu_context.h>
+#include <asm/pgtable.h>
+#include <asm/tlb.h>
+
+void local_flush_tlb_all(void)
+{
+ invtlb_all(INVTLB_CURRENT_ALL, 0, 0);
+}
+EXPORT_SYMBOL(local_flush_tlb_all);
+
+void local_flush_tlb_user(void)
+{
+ invtlb_all(INVTLB_CURRENT_GFALSE, 0, 0);
+}
+EXPORT_SYMBOL(local_flush_tlb_user);
+
+void local_flush_tlb_kernel(void)
+{
+ invtlb_all(INVTLB_CURRENT_GTRUE, 0, 0);
+}
+EXPORT_SYMBOL(local_flush_tlb_kernel);
+
+/*
+ * All entries common to a mm share an asid. To effectively flush
+ * these entries, we just bump the asid.
+ */
+void local_flush_tlb_mm(struct mm_struct *mm)
+{
+ int cpu;
+
+ preempt_disable();
+
+ cpu = smp_processor_id();
+
+ if (asid_valid(mm, cpu))
+ drop_mmu_context(mm, cpu);
+ else
+ cpumask_clear_cpu(cpu, mm_cpumask(mm));
+
+ preempt_enable();
+}
+
+void local_flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
+ unsigned long end)
+{
+ struct mm_struct *mm = vma->vm_mm;
+ int cpu = smp_processor_id();
+
+ if (asid_valid(mm, cpu)) {
+ unsigned long size, flags;
+
+ local_irq_save(flags);
+ start = round_down(start, PAGE_SIZE << 1);
+ end = round_up(end, PAGE_SIZE << 1);
+ size = (end - start) >> (PAGE_SHIFT + 1);
+ if (size <= (current_cpu_data.tlbsizestlbsets ?
+ current_cpu_data.tlbsize / 8 :
+ current_cpu_data.tlbsize / 2)) {
+ int asid = cpu_asid(cpu, mm);
+
+ while (start < end) {
+ invtlb(INVTLB_ADDR_GFALSE_AND_ASID, asid, start);
+ start += (PAGE_SIZE << 1);
+ }
+ } else {
+ drop_mmu_context(mm, cpu);
+ }
+ local_irq_restore(flags);
+ } else {
+ cpumask_clear_cpu(cpu, mm_cpumask(mm));
+ }
+}
+
+void local_flush_tlb_kernel_range(unsigned long start, unsigned long end)
+{
+ unsigned long size, flags;
+
+ local_irq_save(flags);
+ size = (end - start + (PAGE_SIZE - 1)) >> PAGE_SHIFT;
+ size = (size + 1) >> 1;
+ if (size <= (current_cpu_data.tlbsizestlbsets ?
+ current_cpu_data.tlbsize / 8 :
+ current_cpu_data.tlbsize / 2)) {
+
+ start &= (PAGE_MASK << 1);
+ end += ((PAGE_SIZE << 1) - 1);
+ end &= (PAGE_MASK << 1);
+
+ while (start < end) {
+ invtlb_addr(INVTLB_ADDR_GTRUE_OR_ASID, 0, start);
+ start += (PAGE_SIZE << 1);
+ }
+ } else {
+ local_flush_tlb_kernel();
+ }
+ local_irq_restore(flags);
+}
+
+void local_flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
+{
+ int cpu = smp_processor_id();
+
+ if (asid_valid(vma->vm_mm, cpu)) {
+ int newpid;
+
+ newpid = cpu_asid(cpu, vma->vm_mm);
+ page &= (PAGE_MASK << 1);
+ invtlb(INVTLB_ADDR_GFALSE_AND_ASID, newpid, page);
+ } else {
+ cpumask_clear_cpu(cpu, mm_cpumask(vma->vm_mm));
+ }
+}
+
+/*
+ * This one is only used for pages with the global bit set so we don't care
+ * much about the ASID.
+ */
+void local_flush_tlb_one(unsigned long page)
+{
+ page &= (PAGE_MASK << 1);
+ invtlb_addr(INVTLB_ADDR_GTRUE_OR_ASID, 0, page);
+}
+
+static void __update_hugetlb(struct vm_area_struct *vma, unsigned long address, pte_t *ptep)
+{
+#ifdef CONFIG_HUGETLB_PAGE
+ int idx;
+ unsigned long lo;
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ address &= (PAGE_MASK << 1);
+ write_csr_entryhi(address);
+ tlb_probe();
+ idx = read_csr_tlbidx();
+ write_csr_pagesize(PS_HUGE_SIZE);
+ lo = pmd_to_entrylo(pte_val(*ptep));
+ write_csr_entrylo0(lo);
+ write_csr_entrylo1(lo + (HPAGE_SIZE >> 1));
+
+ if (idx < 0)
+ tlb_write_random();
+ else
+ tlb_write_indexed();
+ write_csr_pagesize(PS_DEFAULT_SIZE);
+
+ local_irq_restore(flags);
+#endif
+}
+
+void __update_tlb(struct vm_area_struct *vma, unsigned long address, pte_t *ptep)
+{
+ int idx;
+ unsigned long flags;
+
+ /*
+ * Handle debugger faulting in for debugee.
+ */
+ if (current->active_mm != vma->vm_mm)
+ return;
+
+ if (pte_val(*ptep) & _PAGE_HUGE) {
+ __update_hugetlb(vma, address, ptep);
+ return;
+ }
+
+ local_irq_save(flags);
+
+ if ((unsigned long)ptep & sizeof(pte_t))
+ ptep--;
+
+ address &= (PAGE_MASK << 1);
+ write_csr_entryhi(address);
+ tlb_probe();
+ idx = read_csr_tlbidx();
+ write_csr_pagesize(PS_DEFAULT_SIZE);
+ write_csr_entrylo0(pte_val(*ptep++));
+ write_csr_entrylo1(pte_val(*ptep));
+ if (idx < 0)
+ tlb_write_random();
+ else
+ tlb_write_indexed();
+
+ local_irq_restore(flags);
+}
+
+static void setup_ptwalker(void)
+{
+ unsigned long pwctl0, pwctl1;
+ unsigned long pgd_i = 0, pgd_w = 0;
+ unsigned long pud_i = 0, pud_w = 0;
+ unsigned long pmd_i = 0, pmd_w = 0;
+ unsigned long pte_i = 0, pte_w = 0;
+
+ pgd_i = PGDIR_SHIFT;
+ pgd_w = PAGE_SHIFT - 3;
+#if CONFIG_PGTABLE_LEVELS > 3
+ pud_i = PUD_SHIFT;
+ pud_w = PAGE_SHIFT - 3;
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+ pmd_i = PMD_SHIFT;
+ pmd_w = PAGE_SHIFT - 3;
+#endif
+ pte_i = PAGE_SHIFT;
+ pte_w = PAGE_SHIFT - 3;
+
+ pwctl0 = pte_i | pte_w << 5 | pmd_i << 10 | pmd_w << 15 | pud_i << 20 | pud_w << 25;
+ pwctl1 = pgd_i | pgd_w << 6;
+
+ csr_write64(pwctl0, LOONGARCH_CSR_PWCTL0);
+ csr_write64(pwctl1, LOONGARCH_CSR_PWCTL1);
+ csr_write64((long)swapper_pg_dir, LOONGARCH_CSR_PGDH);
+ csr_write64((long)invalid_pg_dir, LOONGARCH_CSR_PGDL);
+ csr_write64((long)smp_processor_id(), LOONGARCH_CSR_TMID);
+}
+
+static void output_pgtable_bits_defines(void)
+{
+#define pr_define(fmt, ...) \
+ pr_debug("#define " fmt, ##__VA_ARGS__)
+
+ pr_debug("#include <asm/asm.h>\n");
+ pr_debug("#include <asm/regdef.h>\n");
+ pr_debug("\n");
+
+ pr_define("_PAGE_VALID_SHIFT %d\n", _PAGE_VALID_SHIFT);
+ pr_define("_PAGE_DIRTY_SHIFT %d\n", _PAGE_DIRTY_SHIFT);
+ pr_define("_PAGE_HUGE_SHIFT %d\n", _PAGE_HUGE_SHIFT);
+ pr_define("_PAGE_GLOBAL_SHIFT %d\n", _PAGE_GLOBAL_SHIFT);
+ pr_define("_PAGE_PRESENT_SHIFT %d\n", _PAGE_PRESENT_SHIFT);
+ pr_define("_PAGE_WRITE_SHIFT %d\n", _PAGE_WRITE_SHIFT);
+ pr_define("_PAGE_NO_READ_SHIFT %d\n", _PAGE_NO_READ_SHIFT);
+ pr_define("_PAGE_NO_EXEC_SHIFT %d\n", _PAGE_NO_EXEC_SHIFT);
+ pr_define("_PFN_SHIFT %d\n", _PFN_SHIFT);
+ pr_debug("\n");
+}
+
+void setup_tlb_handler(void)
+{
+ static int run_once = 0;
+
+ setup_ptwalker();
+ output_pgtable_bits_defines();
+
+ /* The tlb handlers are generated only once */
+ if (!run_once) {
+ memcpy((void *)tlbrentry, handle_tlb_refill, 0x80);
+ local_flush_icache_range(tlbrentry, tlbrentry + 0x80);
+ set_handler(EXCCODE_TLBI * VECSIZE, handle_tlb_load, VECSIZE);
+ set_handler(EXCCODE_TLBL * VECSIZE, handle_tlb_load, VECSIZE);
+ set_handler(EXCCODE_TLBS * VECSIZE, handle_tlb_store, VECSIZE);
+ set_handler(EXCCODE_TLBM * VECSIZE, handle_tlb_modify, VECSIZE);
+ set_handler(EXCCODE_TLBNR * VECSIZE, handle_tlb_protect, VECSIZE);
+ set_handler(EXCCODE_TLBNX * VECSIZE, handle_tlb_protect, VECSIZE);
+ set_handler(EXCCODE_TLBPE * VECSIZE, handle_tlb_protect, VECSIZE);
+ run_once++;
+ }
+}
+
+void tlb_init(void)
+{
+ write_csr_pagesize(PS_DEFAULT_SIZE);
+ write_csr_stlbpgsize(PS_DEFAULT_SIZE);
+ write_csr_tlbrefill_pagesize(PS_DEFAULT_SIZE);
+ setup_tlb_handler();
+ local_flush_tlb_all();
+}
diff --git a/arch/loongarch/mm/tlbex.S b/arch/loongarch/mm/tlbex.S
new file mode 100644
index 000000000000..bef740710a3b
--- /dev/null
+++ b/arch/loongarch/mm/tlbex.S
@@ -0,0 +1,477 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <asm/asm.h>
+#include <asm/export.h>
+#include <asm/loongarch.h>
+#include <asm/page.h>
+#include <asm/pgtable.h>
+#include <asm/regdef.h>
+#include <asm/stackframe.h>
+
+ .macro tlb_do_page_fault, write
+ SYM_FUNC_START(tlb_do_page_fault_\write)
+ SAVE_ALL
+ csrrd a2, LOONGARCH_CSR_BADV
+ move a0, sp
+ REG_S a2, sp, PT_BVADDR
+ li.w a1, \write
+ la.abs t0, do_page_fault
+ jirl ra, t0, 0
+ RESTORE_ALL_AND_RET
+ SYM_FUNC_END(tlb_do_page_fault_\write)
+ .endm
+
+ tlb_do_page_fault 0
+ tlb_do_page_fault 1
+
+SYM_FUNC_START(handle_tlb_protect)
+ BACKUP_T0T1
+ SAVE_ALL
+ move a0, sp
+ move a1, zero
+ csrrd a2, LOONGARCH_CSR_BADV
+ REG_S a2, sp, PT_BVADDR
+ la.abs t0, do_page_fault
+ jirl ra, t0, 0
+ RESTORE_ALL_AND_RET
+SYM_FUNC_END(handle_tlb_protect)
+
+SYM_FUNC_START(handle_tlb_load)
+ csrwr t0, EXCEPTION_KS0
+ csrwr t1, EXCEPTION_KS1
+ csrwr ra, EXCEPTION_KS2
+
+ /*
+ * The vmalloc handling is not in the hotpath.
+ */
+ csrrd t0, LOONGARCH_CSR_BADV
+ blt t0, $r0, vmalloc_load
+ csrrd t1, LOONGARCH_CSR_PGDL
+
+vmalloc_done_load:
+ /* Get PGD offset in bytes */
+ srli.d t0, t0, PGDIR_SHIFT
+ andi t0, t0, (PTRS_PER_PGD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#if CONFIG_PGTABLE_LEVELS > 3
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PUD_SHIFT
+ andi t0, t0, (PTRS_PER_PUD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PMD_SHIFT
+ andi t0, t0, (PTRS_PER_PMD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+ ld.d ra, t1, 0
+
+ /*
+ * For huge tlb entries, pmde doesn't contain an address but
+ * instead contains the tlb pte. Check the PAGE_HUGE bit and
+ * see if we need to jump to huge tlb processing.
+ */
+ andi t0, ra, _PAGE_HUGE
+ bne t0, $r0, tlb_huge_update_load
+
+ csrrd t0, LOONGARCH_CSR_BADV
+ srli.d t0, t0, (PAGE_SHIFT + PTE_ORDER)
+ andi t0, t0, (PTRS_PER_PTE - 1)
+ slli.d t0, t0, _PTE_T_LOG2
+ add.d t1, ra, t0
+
+ ld.d t0, t1, 0
+ tlbsrch
+
+ srli.d ra, t0, _PAGE_PRESENT_SHIFT
+ andi ra, ra, 1
+ beq ra, $r0, nopage_tlb_load
+
+ ori t0, t0, _PAGE_VALID
+ st.d t0, t1, 0
+ ori t1, t1, 8
+ xori t1, t1, 8
+ ld.d t0, t1, 0
+ ld.d t1, t1, 8
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ csrwr t1, LOONGARCH_CSR_TLBELO1
+ tlbwr
+leave_load:
+ csrrd t0, EXCEPTION_KS0
+ csrrd t1, EXCEPTION_KS1
+ csrrd ra, EXCEPTION_KS2
+ ertn
+#ifdef CONFIG_64BIT
+vmalloc_load:
+ la.abs t1, swapper_pg_dir
+ b vmalloc_done_load
+#endif
+
+ /*
+ * This is the entry point when build_tlbchange_handler_head
+ * spots a huge page.
+ */
+tlb_huge_update_load:
+ ld.d t0, t1, 0
+ srli.d ra, t0, _PAGE_PRESENT_SHIFT
+ andi ra, ra, 1
+ beq ra, $r0, nopage_tlb_load
+ tlbsrch
+
+ ori t0, t0, _PAGE_VALID
+ st.d t0, t1, 0
+ addu16i.d t1, $r0, -(CSR_TLBIDX_EHINV >> 16)
+ addi.d ra, t1, 0
+ csrxchg ra, t1, LOONGARCH_CSR_TLBIDX
+ tlbwr
+
+ csrxchg $r0, t1, LOONGARCH_CSR_TLBIDX
+
+ /*
+ * A huge PTE describes an area the size of the
+ * configured huge page size. This is twice the
+ * of the large TLB entry size we intend to use.
+ * A TLB entry half the size of the configured
+ * huge page size is configured into entrylo0
+ * and entrylo1 to cover the contiguous huge PTE
+ * address space.
+ */
+ /* Huge page: Move Global bit */
+ xori t0, t0, _PAGE_HUGE
+ lu12i.w t1, _PAGE_HGLOBAL >> 12
+ and t1, t0, t1
+ srli.d t1, t1, (_PAGE_HGLOBAL_SHIFT - _PAGE_GLOBAL_SHIFT)
+ or t0, t0, t1
+
+ addi.d ra, t0, 0
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ addi.d t0, ra, 0
+
+ /* Convert to entrylo1 */
+ addi.d t1, $r0, 1
+ slli.d t1, t1, (HPAGE_SHIFT - 1)
+ add.d t0, t0, t1
+ csrwr t0, LOONGARCH_CSR_TLBELO1
+
+ /* Set huge page tlb entry size */
+ addu16i.d t0, $r0, (CSR_TLBIDX_PS >> 16)
+ addu16i.d t1, $r0, (PS_HUGE_SIZE << (CSR_TLBIDX_PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+ tlbfill
+
+ addu16i.d t0, $r0, (CSR_TLBIDX_PS >> 16)
+ addu16i.d t1, $r0, (PS_DEFAULT_SIZE << (CSR_TLBIDX_PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+nopage_tlb_load:
+ csrrd ra, EXCEPTION_KS2
+ la.abs t0, tlb_do_page_fault_0
+ jirl $r0, t0, 0
+SYM_FUNC_END(handle_tlb_load)
+
+SYM_FUNC_START(handle_tlb_store)
+ csrwr t0, EXCEPTION_KS0
+ csrwr t1, EXCEPTION_KS1
+ csrwr ra, EXCEPTION_KS2
+
+ /*
+ * The vmalloc handling is not in the hotpath.
+ */
+ csrrd t0, LOONGARCH_CSR_BADV
+ blt t0, $r0, vmalloc_store
+ csrrd t1, LOONGARCH_CSR_PGDL
+
+vmalloc_done_store:
+ /* Get PGD offset in bytes */
+ srli.d t0, t0, PGDIR_SHIFT
+ andi t0, t0, (PTRS_PER_PGD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+
+#if CONFIG_PGTABLE_LEVELS > 3
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PUD_SHIFT
+ andi t0, t0, (PTRS_PER_PUD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PMD_SHIFT
+ andi t0, t0, (PTRS_PER_PMD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+ ld.d ra, t1, 0
+
+ /*
+ * For huge tlb entries, pmde doesn't contain an address but
+ * instead contains the tlb pte. Check the PAGE_HUGE bit and
+ * see if we need to jump to huge tlb processing.
+ */
+ andi t0, ra, _PAGE_HUGE
+ bne t0, $r0, tlb_huge_update_store
+
+ csrrd t0, LOONGARCH_CSR_BADV
+ srli.d t0, t0, (PAGE_SHIFT + PTE_ORDER)
+ andi t0, t0, (PTRS_PER_PTE - 1)
+ slli.d t0, t0, _PTE_T_LOG2
+ add.d t1, ra, t0
+
+ ld.d t0, t1, 0
+ tlbsrch
+
+ srli.d ra, t0, _PAGE_PRESENT_SHIFT
+ andi ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
+ xori ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
+ bne ra, $r0, nopage_tlb_store
+
+ ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+ st.d t0, t1, 0
+
+ ori t1, t1, 8
+ xori t1, t1, 8
+ ld.d t0, t1, 0
+ ld.d t1, t1, 8
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ csrwr t1, LOONGARCH_CSR_TLBELO1
+ tlbwr
+leave_store:
+ csrrd t0, EXCEPTION_KS0
+ csrrd t1, EXCEPTION_KS1
+ csrrd ra, EXCEPTION_KS2
+ ertn
+#ifdef CONFIG_64BIT
+vmalloc_store:
+ la.abs t1, swapper_pg_dir
+ b vmalloc_done_store
+#endif
+
+ /*
+ * This is the entry point when build_tlbchange_handler_head
+ * spots a huge page.
+ */
+tlb_huge_update_store:
+ ld.d t0, t1, 0
+ srli.d ra, t0, _PAGE_PRESENT_SHIFT
+ andi ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
+ xori ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
+ bne ra, $r0, nopage_tlb_store
+
+ tlbsrch
+ ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+
+ st.d t0, t1, 0
+ addu16i.d t1, $r0, -(CSR_TLBIDX_EHINV >> 16)
+ addi.d ra, t1, 0
+ csrxchg ra, t1, LOONGARCH_CSR_TLBIDX
+ tlbwr
+
+ csrxchg $r0, t1, LOONGARCH_CSR_TLBIDX
+ /*
+ * A huge PTE describes an area the size of the
+ * configured huge page size. This is twice the
+ * of the large TLB entry size we intend to use.
+ * A TLB entry half the size of the configured
+ * huge page size is configured into entrylo0
+ * and entrylo1 to cover the contiguous huge PTE
+ * address space.
+ */
+ /* Huge page: Move Global bit */
+ xori t0, t0, _PAGE_HUGE
+ lu12i.w t1, _PAGE_HGLOBAL >> 12
+ and t1, t0, t1
+ srli.d t1, t1, (_PAGE_HGLOBAL_SHIFT - _PAGE_GLOBAL_SHIFT)
+ or t0, t0, t1
+
+ addi.d ra, t0, 0
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ addi.d t0, ra, 0
+
+ /* Convert to entrylo1 */
+ addi.d t1, $r0, 1
+ slli.d t1, t1, (HPAGE_SHIFT - 1)
+ add.d t0, t0, t1
+ csrwr t0, LOONGARCH_CSR_TLBELO1
+
+ /* Set huge page tlb entry size */
+ addu16i.d t0, $r0, (CSR_TLBIDX_PS >> 16)
+ addu16i.d t1, $r0, (PS_HUGE_SIZE << (CSR_TLBIDX_PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+ tlbfill
+
+ /* Reset default page size */
+ addu16i.d t0, $r0, (CSR_TLBIDX_PS >> 16)
+ addu16i.d t1, $r0, (PS_DEFAULT_SIZE << (CSR_TLBIDX_PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+nopage_tlb_store:
+ csrrd ra, EXCEPTION_KS2
+ la.abs t0, tlb_do_page_fault_1
+ jirl $r0, t0, 0
+SYM_FUNC_END(handle_tlb_store)
+
+SYM_FUNC_START(handle_tlb_modify)
+ csrwr t0, EXCEPTION_KS0
+ csrwr t1, EXCEPTION_KS1
+ csrwr ra, EXCEPTION_KS2
+
+ /*
+ * The vmalloc handling is not in the hotpath.
+ */
+ csrrd t0, LOONGARCH_CSR_BADV
+ blt t0, $r0, vmalloc_modify
+ csrrd t1, LOONGARCH_CSR_PGDL
+
+vmalloc_done_modify:
+ /* Get PGD offset in bytes */
+ srli.d t0, t0, PGDIR_SHIFT
+ andi t0, t0, (PTRS_PER_PGD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#if CONFIG_PGTABLE_LEVELS > 3
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PUD_SHIFT
+ andi t0, t0, (PTRS_PER_PUD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PMD_SHIFT
+ andi t0, t0, (PTRS_PER_PMD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+ ld.d ra, t1, 0
+
+ /*
+ * For huge tlb entries, pmde doesn't contain an address but
+ * instead contains the tlb pte. Check the PAGE_HUGE bit and
+ * see if we need to jump to huge tlb processing.
+ */
+ andi t0, ra, _PAGE_HUGE
+ bne t0, $r0, tlb_huge_update_modify
+
+ csrrd t0, LOONGARCH_CSR_BADV
+ srli.d t0, t0, (PAGE_SHIFT + PTE_ORDER)
+ andi t0, t0, (PTRS_PER_PTE - 1)
+ slli.d t0, t0, _PTE_T_LOG2
+ add.d t1, ra, t0
+
+ ld.d t0, t1, 0
+ tlbsrch
+
+ srli.d ra, t0, _PAGE_WRITE_SHIFT
+ andi ra, ra, 1
+ beq ra, $r0, nopage_tlb_modify
+
+ ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+ st.d t0, t1, 0
+ ori t1, t1, 8
+ xori t1, t1, 8
+ ld.d t0, t1, 0
+ ld.d t1, t1, 8
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ csrwr t1, LOONGARCH_CSR_TLBELO1
+ tlbwr
+leave_modify:
+ csrrd t0, EXCEPTION_KS0
+ csrrd t1, EXCEPTION_KS1
+ csrrd ra, EXCEPTION_KS2
+ ertn
+#ifdef CONFIG_64BIT
+vmalloc_modify:
+ la.abs t1, swapper_pg_dir
+ b vmalloc_done_modify
+#endif
+
+ /*
+ * This is the entry point when
+ * build_tlbchange_handler_head spots a huge page.
+ */
+tlb_huge_update_modify:
+ ld.d t0, t1, 0
+
+ srli.d ra, t0, _PAGE_WRITE_SHIFT
+ andi ra, ra, 1
+ beq ra, $r0, nopage_tlb_modify
+
+ tlbsrch
+ ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+
+ st.d t0, t1, 0
+ /*
+ * A huge PTE describes an area the size of the
+ * configured huge page size. This is twice the
+ * of the large TLB entry size we intend to use.
+ * A TLB entry half the size of the configured
+ * huge page size is configured into entrylo0
+ * and entrylo1 to cover the contiguous huge PTE
+ * address space.
+ */
+ /* Huge page: Move Global bit */
+ xori t0, t0, _PAGE_HUGE
+ lu12i.w t1, _PAGE_HGLOBAL >> 12
+ and t1, t0, t1
+ srli.d t1, t1, (_PAGE_HGLOBAL_SHIFT - _PAGE_GLOBAL_SHIFT)
+ or t0, t0, t1
+
+ addi.d ra, t0, 0
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ addi.d t0, ra, 0
+
+ /* Convert to entrylo1 */
+ addi.d t1, $r0, 1
+ slli.d t1, t1, (HPAGE_SHIFT - 1)
+ add.d t0, t0, t1
+ csrwr t0, LOONGARCH_CSR_TLBELO1
+
+ /* Set huge page tlb entry size */
+ addu16i.d t0, $r0, (CSR_TLBIDX_PS >> 16)
+ addu16i.d t1, $r0, (PS_HUGE_SIZE << (CSR_TLBIDX_PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+ tlbwr
+
+ /* Reset default page size */
+ addu16i.d t0, $r0, (CSR_TLBIDX_PS >> 16)
+ addu16i.d t1, $r0, (PS_DEFAULT_SIZE << (CSR_TLBIDX_PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+nopage_tlb_modify:
+ csrrd ra, EXCEPTION_KS2
+ la.abs t0, tlb_do_page_fault_1
+ jirl $r0, t0, 0
+SYM_FUNC_END(handle_tlb_modify)
+
+SYM_FUNC_START(handle_tlb_refill)
+ csrwr t0, LOONGARCH_CSR_TLBRSAVE
+ csrrd t0, LOONGARCH_CSR_PGD
+ lddir t0, t0, 3
+#if CONFIG_PGTABLE_LEVELS > 3
+ lddir t0, t0, 2
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+ lddir t0, t0, 1
+#endif
+ ldpte t0, 0
+ ldpte t0, 1
+ tlbfill
+ csrrd t0, LOONGARCH_CSR_TLBRSAVE
+ ertn
+SYM_FUNC_END(handle_tlb_refill)
--
2.27.0
Add some other common headers for basic LoongArch support.
Reviewed-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/asm-prototypes.h | 7 +
arch/loongarch/include/asm/asm.h | 191 ++++++++++++
arch/loongarch/include/asm/asmmacro.h | 289 ++++++++++++++++++
arch/loongarch/include/asm/clocksource.h | 12 +
arch/loongarch/include/asm/compiler.h | 15 +
arch/loongarch/include/asm/inst.h | 117 +++++++
arch/loongarch/include/asm/linkage.h | 36 +++
arch/loongarch/include/asm/perf_event.h | 10 +
arch/loongarch/include/asm/prefetch.h | 29 ++
arch/loongarch/include/asm/serial.h | 11 +
arch/loongarch/include/asm/time.h | 50 +++
arch/loongarch/include/asm/timex.h | 33 ++
arch/loongarch/include/asm/topology.h | 15 +
arch/loongarch/include/asm/types.h | 19 ++
arch/loongarch/include/uapi/asm/bitsperlong.h | 9 +
arch/loongarch/include/uapi/asm/byteorder.h | 13 +
arch/loongarch/include/uapi/asm/reg.h | 59 ++++
tools/include/uapi/asm/bitsperlong.h | 2 +
18 files changed, 917 insertions(+)
create mode 100644 arch/loongarch/include/asm/asm-prototypes.h
create mode 100644 arch/loongarch/include/asm/asm.h
create mode 100644 arch/loongarch/include/asm/asmmacro.h
create mode 100644 arch/loongarch/include/asm/clocksource.h
create mode 100644 arch/loongarch/include/asm/compiler.h
create mode 100644 arch/loongarch/include/asm/inst.h
create mode 100644 arch/loongarch/include/asm/linkage.h
create mode 100644 arch/loongarch/include/asm/perf_event.h
create mode 100644 arch/loongarch/include/asm/prefetch.h
create mode 100644 arch/loongarch/include/asm/serial.h
create mode 100644 arch/loongarch/include/asm/time.h
create mode 100644 arch/loongarch/include/asm/timex.h
create mode 100644 arch/loongarch/include/asm/topology.h
create mode 100644 arch/loongarch/include/asm/types.h
create mode 100644 arch/loongarch/include/uapi/asm/bitsperlong.h
create mode 100644 arch/loongarch/include/uapi/asm/byteorder.h
create mode 100644 arch/loongarch/include/uapi/asm/reg.h
diff --git a/arch/loongarch/include/asm/asm-prototypes.h b/arch/loongarch/include/asm/asm-prototypes.h
new file mode 100644
index 000000000000..ed06d3997420
--- /dev/null
+++ b/arch/loongarch/include/asm/asm-prototypes.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/uaccess.h>
+#include <asm/fpu.h>
+#include <asm/mmu_context.h>
+#include <asm/page.h>
+#include <asm/ftrace.h>
+#include <asm-generic/asm-prototypes.h>
diff --git a/arch/loongarch/include/asm/asm.h b/arch/loongarch/include/asm/asm.h
new file mode 100644
index 000000000000..40eea6aa469e
--- /dev/null
+++ b/arch/loongarch/include/asm/asm.h
@@ -0,0 +1,191 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Some useful macros for LoongArch assembler code
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1995, 1996, 1997, 1999, 2001 by Ralf Baechle
+ * Copyright (C) 1999 by Silicon Graphics, Inc.
+ * Copyright (C) 2001 MIPS Technologies, Inc.
+ * Copyright (C) 2002 Maciej W. Rozycki
+ */
+#ifndef __ASM_ASM_H
+#define __ASM_ASM_H
+
+/* LoongArch pref instruction. */
+#ifdef CONFIG_CPU_HAS_PREFETCH
+
+#define PREF(hint, addr, offs) \
+ preld hint, addr, offs; \
+
+#define PREFX(hint, addr, index) \
+ preldx hint, addr, index; \
+
+#else /* !CONFIG_CPU_HAS_PREFETCH */
+
+#define PREF(hint, addr, offs)
+#define PREFX(hint, addr, index)
+
+#endif /* !CONFIG_CPU_HAS_PREFETCH */
+
+/*
+ * Stack alignment
+ */
+#define STACK_ALIGN ~(0xf)
+
+/*
+ * Macros to handle different pointer/register sizes for 32/64-bit code
+ */
+
+/*
+ * Size of a register
+ */
+#ifndef __loongarch64
+#define SZREG 4
+#else
+#define SZREG 8
+#endif
+
+/*
+ * Use the following macros in assemblercode to load/store registers,
+ * pointers etc.
+ */
+#if (SZREG == 4)
+#define REG_L ld.w
+#define REG_S st.w
+#define REG_ADD add.w
+#define REG_SUB sub.w
+#else /* SZREG == 8 */
+#define REG_L ld.d
+#define REG_S st.d
+#define REG_ADD add.d
+#define REG_SUB sub.d
+#endif
+
+/*
+ * How to add/sub/load/store/shift C int variables.
+ */
+#if (__SIZEOF_INT__ == 4)
+#define INT_ADD add.w
+#define INT_ADDI addi.w
+#define INT_SUB sub.w
+#define INT_L ld.w
+#define INT_S st.w
+#define INT_SLL slli.w
+#define INT_SLLV sll.w
+#define INT_SRL srli.w
+#define INT_SRLV srl.w
+#define INT_SRA srai.w
+#define INT_SRAV sra.w
+#endif
+
+#if (__SIZEOF_INT__ == 8)
+#define INT_ADD add.d
+#define INT_ADDI addi.d
+#define INT_SUB sub.d
+#define INT_L ld.d
+#define INT_S st.d
+#define INT_SLL slli.d
+#define INT_SLLV sll.d
+#define INT_SRL srli.d
+#define INT_SRLV srl.d
+#define INT_SRA srai.d
+#define INT_SRAV sra.d
+#endif
+
+/*
+ * How to add/sub/load/store/shift C long variables.
+ */
+#if (__SIZEOF_LONG__ == 4)
+#define LONG_ADD add.w
+#define LONG_ADDI addi.w
+#define LONG_SUB sub.w
+#define LONG_L ld.w
+#define LONG_S st.w
+#define LONG_SLL slli.w
+#define LONG_SLLV sll.w
+#define LONG_SRL srli.w
+#define LONG_SRLV srl.w
+#define LONG_SRA srai.w
+#define LONG_SRAV sra.w
+
+#ifdef __ASSEMBLY__
+#define LONG .word
+#endif
+#define LONGSIZE 4
+#define LONGMASK 3
+#define LONGLOG 2
+#endif
+
+#if (__SIZEOF_LONG__ == 8)
+#define LONG_ADD add.d
+#define LONG_ADDI addi.d
+#define LONG_SUB sub.d
+#define LONG_L ld.d
+#define LONG_S st.d
+#define LONG_SLL slli.d
+#define LONG_SLLV sll.d
+#define LONG_SRL srli.d
+#define LONG_SRLV srl.d
+#define LONG_SRA srai.d
+#define LONG_SRAV sra.d
+
+#ifdef __ASSEMBLY__
+#define LONG .dword
+#endif
+#define LONGSIZE 8
+#define LONGMASK 7
+#define LONGLOG 3
+#endif
+
+/*
+ * How to add/sub/load/store/shift pointers.
+ */
+#if (__SIZEOF_POINTER__ == 4)
+#define PTR_ADD add.w
+#define PTR_ADDI addi.w
+#define PTR_SUB sub.w
+#define PTR_L ld.w
+#define PTR_S st.w
+#define PTR_LI li.w
+#define PTR_SLL slli.w
+#define PTR_SLLV sll.w
+#define PTR_SRL srli.w
+#define PTR_SRLV srl.w
+#define PTR_SRA srai.w
+#define PTR_SRAV sra.w
+
+#define PTR_SCALESHIFT 2
+
+#ifdef __ASSEMBLY__
+#define PTR .word
+#endif
+#define PTRSIZE 4
+#define PTRLOG 2
+#endif
+
+#if (__SIZEOF_POINTER__ == 8)
+#define PTR_ADD add.d
+#define PTR_ADDI addi.d
+#define PTR_SUB sub.d
+#define PTR_L ld.d
+#define PTR_S st.d
+#define PTR_LI li.d
+#define PTR_SLL slli.d
+#define PTR_SLLV sll.d
+#define PTR_SRL srli.d
+#define PTR_SRLV srl.d
+#define PTR_SRA srai.d
+#define PTR_SRAV sra.d
+
+#define PTR_SCALESHIFT 3
+
+#ifdef __ASSEMBLY__
+#define PTR .dword
+#endif
+#define PTRSIZE 8
+#define PTRLOG 3
+#endif
+
+#endif /* __ASM_ASM_H */
diff --git a/arch/loongarch/include/asm/asmmacro.h b/arch/loongarch/include/asm/asmmacro.h
new file mode 100644
index 000000000000..a1a04083bd67
--- /dev/null
+++ b/arch/loongarch/include/asm/asmmacro.h
@@ -0,0 +1,289 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_ASMMACRO_H
+#define _ASM_ASMMACRO_H
+
+#include <asm/asm-offsets.h>
+#include <asm/regdef.h>
+#include <asm/fpregdef.h>
+#include <asm/loongarch.h>
+
+ .macro parse_v var val
+ \var = \val
+ .endm
+
+ .macro parse_r var r
+ \var = -1
+ .ifc \r, $r0
+ \var = 0
+ .endif
+ .ifc \r, $r1
+ \var = 1
+ .endif
+ .ifc \r, $r2
+ \var = 2
+ .endif
+ .ifc \r, $r3
+ \var = 3
+ .endif
+ .ifc \r, $r4
+ \var = 4
+ .endif
+ .ifc \r, $r5
+ \var = 5
+ .endif
+ .ifc \r, $r6
+ \var = 6
+ .endif
+ .ifc \r, $r7
+ \var = 7
+ .endif
+ .ifc \r, $r8
+ \var = 8
+ .endif
+ .ifc \r, $r9
+ \var = 9
+ .endif
+ .ifc \r, $r10
+ \var = 10
+ .endif
+ .ifc \r, $r11
+ \var = 11
+ .endif
+ .ifc \r, $r12
+ \var = 12
+ .endif
+ .ifc \r, $r13
+ \var = 13
+ .endif
+ .ifc \r, $r14
+ \var = 14
+ .endif
+ .ifc \r, $r15
+ \var = 15
+ .endif
+ .ifc \r, $r16
+ \var = 16
+ .endif
+ .ifc \r, $r17
+ \var = 17
+ .endif
+ .ifc \r, $r18
+ \var = 18
+ .endif
+ .ifc \r, $r19
+ \var = 19
+ .endif
+ .ifc \r, $r20
+ \var = 20
+ .endif
+ .ifc \r, $r21
+ \var = 21
+ .endif
+ .ifc \r, $r22
+ \var = 22
+ .endif
+ .ifc \r, $r23
+ \var = 23
+ .endif
+ .ifc \r, $r24
+ \var = 24
+ .endif
+ .ifc \r, $r25
+ \var = 25
+ .endif
+ .ifc \r, $r26
+ \var = 26
+ .endif
+ .ifc \r, $r27
+ \var = 27
+ .endif
+ .ifc \r, $r28
+ \var = 28
+ .endif
+ .ifc \r, $r29
+ \var = 29
+ .endif
+ .ifc \r, $r30
+ \var = 30
+ .endif
+ .ifc \r, $r31
+ \var = 31
+ .endif
+ .iflt \var
+ .error "Unable to parse register name \r"
+ .endif
+ .endm
+
+ .macro cpu_save_nonscratch thread
+ stptr.d s0, \thread, THREAD_REG23
+ stptr.d s1, \thread, THREAD_REG24
+ stptr.d s2, \thread, THREAD_REG25
+ stptr.d s3, \thread, THREAD_REG26
+ stptr.d s4, \thread, THREAD_REG27
+ stptr.d s5, \thread, THREAD_REG28
+ stptr.d s6, \thread, THREAD_REG29
+ stptr.d s7, \thread, THREAD_REG30
+ stptr.d s8, \thread, THREAD_REG31
+ stptr.d sp, \thread, THREAD_REG03
+ stptr.d fp, \thread, THREAD_REG22
+ .endm
+
+ .macro cpu_restore_nonscratch thread
+ ldptr.d s0, \thread, THREAD_REG23
+ ldptr.d s1, \thread, THREAD_REG24
+ ldptr.d s2, \thread, THREAD_REG25
+ ldptr.d s3, \thread, THREAD_REG26
+ ldptr.d s4, \thread, THREAD_REG27
+ ldptr.d s5, \thread, THREAD_REG28
+ ldptr.d s6, \thread, THREAD_REG29
+ ldptr.d s7, \thread, THREAD_REG30
+ ldptr.d s8, \thread, THREAD_REG31
+ ldptr.d ra, \thread, THREAD_REG01
+ ldptr.d sp, \thread, THREAD_REG03
+ ldptr.d fp, \thread, THREAD_REG22
+ .endm
+
+ .macro fpu_save_csr thread tmp
+ movfcsr2gr \tmp, fcsr0
+ stptr.w \tmp, \thread, THREAD_FCSR
+ .endm
+
+ .macro fpu_restore_csr thread tmp
+ ldptr.w \tmp, \thread, THREAD_FCSR
+ movgr2fcsr fcsr0, \tmp
+ .endm
+
+ .macro fpu_save_cc thread tmp0 tmp1
+ movcf2gr \tmp0, $fcc0
+ move \tmp1, \tmp0
+ movcf2gr \tmp0, $fcc1
+ bstrins.d \tmp1, \tmp0, 15, 8
+ movcf2gr \tmp0, $fcc2
+ bstrins.d \tmp1, \tmp0, 23, 16
+ movcf2gr \tmp0, $fcc3
+ bstrins.d \tmp1, \tmp0, 31, 24
+ movcf2gr \tmp0, $fcc4
+ bstrins.d \tmp1, \tmp0, 39, 32
+ movcf2gr \tmp0, $fcc5
+ bstrins.d \tmp1, \tmp0, 47, 40
+ movcf2gr \tmp0, $fcc6
+ bstrins.d \tmp1, \tmp0, 55, 48
+ movcf2gr \tmp0, $fcc7
+ bstrins.d \tmp1, \tmp0, 63, 56
+ stptr.d \tmp1, \thread, THREAD_FCC
+ .endm
+
+ .macro fpu_restore_cc thread tmp0 tmp1
+ ldptr.d \tmp0, \thread, THREAD_FCC
+ bstrpick.d \tmp1, \tmp0, 7, 0
+ movgr2cf $fcc0, \tmp1
+ bstrpick.d \tmp1, \tmp0, 15, 8
+ movgr2cf $fcc1, \tmp1
+ bstrpick.d \tmp1, \tmp0, 23, 16
+ movgr2cf $fcc2, \tmp1
+ bstrpick.d \tmp1, \tmp0, 31, 24
+ movgr2cf $fcc3, \tmp1
+ bstrpick.d \tmp1, \tmp0, 39, 32
+ movgr2cf $fcc4, \tmp1
+ bstrpick.d \tmp1, \tmp0, 47, 40
+ movgr2cf $fcc5, \tmp1
+ bstrpick.d \tmp1, \tmp0, 55, 48
+ movgr2cf $fcc6, \tmp1
+ bstrpick.d \tmp1, \tmp0, 63, 56
+ movgr2cf $fcc7, \tmp1
+ .endm
+
+ .macro fpu_save_double thread tmp
+ li.w \tmp, THREAD_FPR0
+ PTR_ADD \tmp, \tmp, \thread
+ fst.d $f0, \tmp, THREAD_FPR0 - THREAD_FPR0
+ fst.d $f1, \tmp, THREAD_FPR1 - THREAD_FPR0
+ fst.d $f2, \tmp, THREAD_FPR2 - THREAD_FPR0
+ fst.d $f3, \tmp, THREAD_FPR3 - THREAD_FPR0
+ fst.d $f4, \tmp, THREAD_FPR4 - THREAD_FPR0
+ fst.d $f5, \tmp, THREAD_FPR5 - THREAD_FPR0
+ fst.d $f6, \tmp, THREAD_FPR6 - THREAD_FPR0
+ fst.d $f7, \tmp, THREAD_FPR7 - THREAD_FPR0
+ fst.d $f8, \tmp, THREAD_FPR8 - THREAD_FPR0
+ fst.d $f9, \tmp, THREAD_FPR9 - THREAD_FPR0
+ fst.d $f10, \tmp, THREAD_FPR10 - THREAD_FPR0
+ fst.d $f11, \tmp, THREAD_FPR11 - THREAD_FPR0
+ fst.d $f12, \tmp, THREAD_FPR12 - THREAD_FPR0
+ fst.d $f13, \tmp, THREAD_FPR13 - THREAD_FPR0
+ fst.d $f14, \tmp, THREAD_FPR14 - THREAD_FPR0
+ fst.d $f15, \tmp, THREAD_FPR15 - THREAD_FPR0
+ fst.d $f16, \tmp, THREAD_FPR16 - THREAD_FPR0
+ fst.d $f17, \tmp, THREAD_FPR17 - THREAD_FPR0
+ fst.d $f18, \tmp, THREAD_FPR18 - THREAD_FPR0
+ fst.d $f19, \tmp, THREAD_FPR19 - THREAD_FPR0
+ fst.d $f20, \tmp, THREAD_FPR20 - THREAD_FPR0
+ fst.d $f21, \tmp, THREAD_FPR21 - THREAD_FPR0
+ fst.d $f22, \tmp, THREAD_FPR22 - THREAD_FPR0
+ fst.d $f23, \tmp, THREAD_FPR23 - THREAD_FPR0
+ fst.d $f24, \tmp, THREAD_FPR24 - THREAD_FPR0
+ fst.d $f25, \tmp, THREAD_FPR25 - THREAD_FPR0
+ fst.d $f26, \tmp, THREAD_FPR26 - THREAD_FPR0
+ fst.d $f27, \tmp, THREAD_FPR27 - THREAD_FPR0
+ fst.d $f28, \tmp, THREAD_FPR28 - THREAD_FPR0
+ fst.d $f29, \tmp, THREAD_FPR29 - THREAD_FPR0
+ fst.d $f30, \tmp, THREAD_FPR30 - THREAD_FPR0
+ fst.d $f31, \tmp, THREAD_FPR31 - THREAD_FPR0
+ .endm
+
+ .macro fpu_restore_double thread tmp
+ li.w \tmp, THREAD_FPR0
+ PTR_ADD \tmp, \tmp, \thread
+ fld.d $f0, \tmp, THREAD_FPR0 - THREAD_FPR0
+ fld.d $f1, \tmp, THREAD_FPR1 - THREAD_FPR0
+ fld.d $f2, \tmp, THREAD_FPR2 - THREAD_FPR0
+ fld.d $f3, \tmp, THREAD_FPR3 - THREAD_FPR0
+ fld.d $f4, \tmp, THREAD_FPR4 - THREAD_FPR0
+ fld.d $f5, \tmp, THREAD_FPR5 - THREAD_FPR0
+ fld.d $f6, \tmp, THREAD_FPR6 - THREAD_FPR0
+ fld.d $f7, \tmp, THREAD_FPR7 - THREAD_FPR0
+ fld.d $f8, \tmp, THREAD_FPR8 - THREAD_FPR0
+ fld.d $f9, \tmp, THREAD_FPR9 - THREAD_FPR0
+ fld.d $f10, \tmp, THREAD_FPR10 - THREAD_FPR0
+ fld.d $f11, \tmp, THREAD_FPR11 - THREAD_FPR0
+ fld.d $f12, \tmp, THREAD_FPR12 - THREAD_FPR0
+ fld.d $f13, \tmp, THREAD_FPR13 - THREAD_FPR0
+ fld.d $f14, \tmp, THREAD_FPR14 - THREAD_FPR0
+ fld.d $f15, \tmp, THREAD_FPR15 - THREAD_FPR0
+ fld.d $f16, \tmp, THREAD_FPR16 - THREAD_FPR0
+ fld.d $f17, \tmp, THREAD_FPR17 - THREAD_FPR0
+ fld.d $f18, \tmp, THREAD_FPR18 - THREAD_FPR0
+ fld.d $f19, \tmp, THREAD_FPR19 - THREAD_FPR0
+ fld.d $f20, \tmp, THREAD_FPR20 - THREAD_FPR0
+ fld.d $f21, \tmp, THREAD_FPR21 - THREAD_FPR0
+ fld.d $f22, \tmp, THREAD_FPR22 - THREAD_FPR0
+ fld.d $f23, \tmp, THREAD_FPR23 - THREAD_FPR0
+ fld.d $f24, \tmp, THREAD_FPR24 - THREAD_FPR0
+ fld.d $f25, \tmp, THREAD_FPR25 - THREAD_FPR0
+ fld.d $f26, \tmp, THREAD_FPR26 - THREAD_FPR0
+ fld.d $f27, \tmp, THREAD_FPR27 - THREAD_FPR0
+ fld.d $f28, \tmp, THREAD_FPR28 - THREAD_FPR0
+ fld.d $f29, \tmp, THREAD_FPR29 - THREAD_FPR0
+ fld.d $f30, \tmp, THREAD_FPR30 - THREAD_FPR0
+ fld.d $f31, \tmp, THREAD_FPR31 - THREAD_FPR0
+ .endm
+
+.macro not dst src
+ nor \dst, \src, zero
+.endm
+
+.macro bgt r0 r1 label
+ blt \r1, \r0, \label
+.endm
+
+.macro bltz r0 label
+ blt \r0, zero, \label
+.endm
+
+.macro bgez r0 label
+ bge \r0, zero, \label
+.endm
+
+#endif /* _ASM_ASMMACRO_H */
diff --git a/arch/loongarch/include/asm/clocksource.h b/arch/loongarch/include/asm/clocksource.h
new file mode 100644
index 000000000000..58e64aa05d26
--- /dev/null
+++ b/arch/loongarch/include/asm/clocksource.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASM_CLOCKSOURCE_H
+#define __ASM_CLOCKSOURCE_H
+
+#include <asm/vdso/clocksource.h>
+
+#endif /* __ASM_CLOCKSOURCE_H */
diff --git a/arch/loongarch/include/asm/compiler.h b/arch/loongarch/include/asm/compiler.h
new file mode 100644
index 000000000000..657cebe70ace
--- /dev/null
+++ b/arch/loongarch/include/asm/compiler.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_COMPILER_H
+#define _ASM_COMPILER_H
+
+#define GCC_OFF_SMALL_ASM() "ZC"
+
+#define LOONGARCH_ISA_LEVEL "loongarch"
+#define LOONGARCH_ISA_ARCH_LEVEL "arch=loongarch"
+#define LOONGARCH_ISA_LEVEL_RAW loongarch
+#define LOONGARCH_ISA_ARCH_LEVEL_RAW LOONGARCH_ISA_LEVEL_RAW
+
+#endif /* _ASM_COMPILER_H */
diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm/inst.h
new file mode 100644
index 000000000000..575d1bb66ffb
--- /dev/null
+++ b/arch/loongarch/include/asm/inst.h
@@ -0,0 +1,117 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_INST_H
+#define _ASM_INST_H
+
+#include <linux/types.h>
+#include <asm/asm.h>
+
+#define ADDR_IMMMASK_LU52ID 0xFFF0000000000000
+#define ADDR_IMMMASK_LU32ID 0x000FFFFF00000000
+#define ADDR_IMMMASK_ADDU16ID 0x00000000FFFF0000
+
+#define ADDR_IMMSHIFT_LU52ID 52
+#define ADDR_IMMSHIFT_LU32ID 32
+#define ADDR_IMMSHIFT_ADDU16ID 16
+
+#define ADDR_IMM(addr, INSN) ((addr & ADDR_IMMMASK_##INSN) >> ADDR_IMMSHIFT_##INSN)
+
+enum reg1i20_op {
+ lu12iw_op = 0x0a,
+ lu32id_op = 0x0b,
+};
+
+enum reg2i12_op {
+ lu52id_op = 0x0c,
+};
+
+enum reg2i16_op {
+ jirl_op = 0x13,
+};
+
+struct reg0i26_format {
+ unsigned int immediate_h : 10;
+ unsigned int immediate_l : 16;
+ unsigned int opcode : 6;
+};
+
+struct reg1i20_format {
+ unsigned int rd : 5;
+ unsigned int immediate : 20;
+ unsigned int opcode : 7;
+};
+
+struct reg1i21_format {
+ unsigned int immediate_h : 5;
+ unsigned int rj : 5;
+ unsigned int immediate_l : 16;
+ unsigned int opcode : 6;
+};
+
+struct reg2i12_format {
+ unsigned int rd : 5;
+ unsigned int rj : 5;
+ unsigned int immediate : 12;
+ unsigned int opcode : 10;
+};
+
+struct reg2i16_format {
+ unsigned int rd : 5;
+ unsigned int rj : 5;
+ unsigned int immediate : 16;
+ unsigned int opcode : 6;
+};
+
+union loongarch_instruction {
+ unsigned int word;
+ struct reg0i26_format reg0i26_format;
+ struct reg1i20_format reg1i20_format;
+ struct reg1i21_format reg1i21_format;
+ struct reg2i12_format reg2i12_format;
+ struct reg2i16_format reg2i16_format;
+};
+
+#define LOONGARCH_INSN_SIZE sizeof(union loongarch_instruction)
+
+enum loongarch_gpr {
+ LOONGARCH_GPR_ZERO = 0,
+ LOONGARCH_GPR_RA = 1,
+ LOONGARCH_GPR_TP = 2,
+ LOONGARCH_GPR_SP = 3,
+ LOONGARCH_GPR_A0 = 4, /* Reused as V0 for return value */
+ LOONGARCH_GPR_A1, /* Reused as V1 for return value */
+ LOONGARCH_GPR_A2,
+ LOONGARCH_GPR_A3,
+ LOONGARCH_GPR_A4,
+ LOONGARCH_GPR_A5,
+ LOONGARCH_GPR_A6,
+ LOONGARCH_GPR_A7,
+ LOONGARCH_GPR_T0 = 12,
+ LOONGARCH_GPR_T1,
+ LOONGARCH_GPR_T2,
+ LOONGARCH_GPR_T3,
+ LOONGARCH_GPR_T4,
+ LOONGARCH_GPR_T5,
+ LOONGARCH_GPR_T6,
+ LOONGARCH_GPR_T7,
+ LOONGARCH_GPR_T8,
+ LOONGARCH_GPR_FP = 22,
+ LOONGARCH_GPR_S0 = 23,
+ LOONGARCH_GPR_S1,
+ LOONGARCH_GPR_S2,
+ LOONGARCH_GPR_S3,
+ LOONGARCH_GPR_S4,
+ LOONGARCH_GPR_S5,
+ LOONGARCH_GPR_S6,
+ LOONGARCH_GPR_S7,
+ LOONGARCH_GPR_S8,
+ LOONGARCH_GPR_MAX
+};
+
+u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm);
+u32 larch_insn_gen_lu52id(enum loongarch_gpr rd, enum loongarch_gpr rj, int imm);
+u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, unsigned long pc, unsigned long dest);
+
+#endif /* _ASM_INST_H */
diff --git a/arch/loongarch/include/asm/linkage.h b/arch/loongarch/include/asm/linkage.h
new file mode 100644
index 000000000000..81b0c4cfbf4f
--- /dev/null
+++ b/arch/loongarch/include/asm/linkage.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ASM_LINKAGE_H
+#define __ASM_LINKAGE_H
+
+#define __ALIGN .align 2
+#define __ALIGN_STR __stringify(__ALIGN)
+
+#define SYM_FUNC_START(name) \
+ SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN) \
+ .cfi_startproc;
+
+#define SYM_FUNC_START_NOALIGN(name) \
+ SYM_START(name, SYM_L_GLOBAL, SYM_A_NONE) \
+ .cfi_startproc;
+
+#define SYM_FUNC_START_LOCAL(name) \
+ SYM_START(name, SYM_L_LOCAL, SYM_A_ALIGN) \
+ .cfi_startproc;
+
+#define SYM_FUNC_START_LOCAL_NOALIGN(name) \
+ SYM_START(name, SYM_L_LOCAL, SYM_A_NONE) \
+ .cfi_startproc;
+
+#define SYM_FUNC_START_WEAK(name) \
+ SYM_START(name, SYM_L_WEAK, SYM_A_ALIGN) \
+ .cfi_startproc;
+
+#define SYM_FUNC_START_WEAK_NOALIGN(name) \
+ SYM_START(name, SYM_L_WEAK, SYM_A_NONE) \
+ .cfi_startproc;
+
+#define SYM_FUNC_END(name) \
+ .cfi_endproc; \
+ SYM_END(name, SYM_T_FUNC)
+
+#endif
diff --git a/arch/loongarch/include/asm/perf_event.h b/arch/loongarch/include/asm/perf_event.h
new file mode 100644
index 000000000000..dcb3b17053a8
--- /dev/null
+++ b/arch/loongarch/include/asm/perf_event.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __LOONGARCH_PERF_EVENT_H__
+#define __LOONGARCH_PERF_EVENT_H__
+/* Nothing to show here; the file is required by linux/perf_event.h. */
+#endif /* __LOONGARCH_PERF_EVENT_H__ */
diff --git a/arch/loongarch/include/asm/prefetch.h b/arch/loongarch/include/asm/prefetch.h
new file mode 100644
index 000000000000..1672262a5e2e
--- /dev/null
+++ b/arch/loongarch/include/asm/prefetch.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_PREFETCH_H
+#define __ASM_PREFETCH_H
+
+#define Pref_Load 0
+#define Pref_Store 8
+
+#ifdef __ASSEMBLY__
+
+ .macro __pref hint addr
+#ifdef CONFIG_CPU_HAS_PREFETCH
+ preld \hint, \addr, 0
+#endif
+ .endm
+
+ .macro pref_load addr
+ __pref Pref_Load, \addr
+ .endm
+
+ .macro pref_store addr
+ __pref Pref_Store, \addr
+ .endm
+
+#endif
+
+#endif /* __ASM_PREFETCH_H */
diff --git a/arch/loongarch/include/asm/serial.h b/arch/loongarch/include/asm/serial.h
new file mode 100644
index 000000000000..3fb550eb9115
--- /dev/null
+++ b/arch/loongarch/include/asm/serial.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM__SERIAL_H
+#define __ASM__SERIAL_H
+
+#define BASE_BAUD 0
+#define STD_COM_FLAGS (ASYNC_BOOT_AUTOCONF | ASYNC_SKIP_TEST)
+
+#endif /* __ASM__SERIAL_H */
diff --git a/arch/loongarch/include/asm/time.h b/arch/loongarch/include/asm/time.h
new file mode 100644
index 000000000000..2eae219301d0
--- /dev/null
+++ b/arch/loongarch/include/asm/time.h
@@ -0,0 +1,50 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_TIME_H
+#define _ASM_TIME_H
+
+#include <linux/clockchips.h>
+#include <linux/clocksource.h>
+#include <asm/loongarch.h>
+
+extern u64 cpu_clock_freq;
+extern u64 const_clock_freq;
+
+extern void sync_counter(void);
+
+static inline unsigned int calc_const_freq(void)
+{
+ unsigned int res;
+ unsigned int base_freq;
+ unsigned int cfm, cfd;
+
+ res = read_cpucfg(LOONGARCH_CPUCFG2);
+ if (!(res & CPUCFG2_LLFTP))
+ return 0;
+
+ base_freq = read_cpucfg(LOONGARCH_CPUCFG4);
+ res = read_cpucfg(LOONGARCH_CPUCFG5);
+ cfm = res & 0xffff;
+ cfd = (res >> 16) & 0xffff;
+
+ if (!base_freq || !cfm || !cfd)
+ return 0;
+
+ return (base_freq * cfm / cfd);
+}
+
+/*
+ * Initialize the calling CPU's timer interrupt as clockevent device
+ */
+extern int constant_clockevent_init(void);
+extern int constant_clocksource_init(void);
+
+static inline void clockevent_set_clock(struct clock_event_device *cd,
+ unsigned int clock)
+{
+ clockevents_calc_mult_shift(cd, clock, 4);
+}
+
+#endif /* _ASM_TIME_H */
diff --git a/arch/loongarch/include/asm/timex.h b/arch/loongarch/include/asm/timex.h
new file mode 100644
index 000000000000..d3ed99a4fdbd
--- /dev/null
+++ b/arch/loongarch/include/asm/timex.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_TIMEX_H
+#define _ASM_TIMEX_H
+
+#ifdef __KERNEL__
+
+#include <linux/compiler.h>
+
+#include <asm/cpu.h>
+#include <asm/cpu-features.h>
+
+/*
+ * Standard way to access the cycle counter.
+ * Currently only used on SMP for scheduling.
+ *
+ * We know that all SMP capable CPUs have cycle counters.
+ */
+
+typedef unsigned long cycles_t;
+
+#define get_cycles get_cycles
+
+static inline cycles_t get_cycles(void)
+{
+ return drdtime();
+}
+
+#endif /* __KERNEL__ */
+
+#endif /* _ASM_TIMEX_H */
diff --git a/arch/loongarch/include/asm/topology.h b/arch/loongarch/include/asm/topology.h
new file mode 100644
index 000000000000..9ac71a25207a
--- /dev/null
+++ b/arch/loongarch/include/asm/topology.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_TOPOLOGY_H
+#define __ASM_TOPOLOGY_H
+
+#include <linux/smp.h>
+
+#define cpu_logical_map(cpu) 0
+
+#include <asm-generic/topology.h>
+
+static inline void arch_fix_phys_package_id(int num, u32 slot) { }
+#endif /* __ASM_TOPOLOGY_H */
diff --git a/arch/loongarch/include/asm/types.h b/arch/loongarch/include/asm/types.h
new file mode 100644
index 000000000000..baf15a0dcf8b
--- /dev/null
+++ b/arch/loongarch/include/asm/types.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_TYPES_H
+#define _ASM_TYPES_H
+
+#include <asm-generic/int-ll64.h>
+#include <uapi/asm/types.h>
+
+#ifdef __ASSEMBLY__
+#define _ULCAST_
+#define _U64CAST_
+#else
+#define _ULCAST_ (unsigned long)
+#define _U64CAST_ (u64)
+#endif
+
+#endif /* _ASM_TYPES_H */
diff --git a/arch/loongarch/include/uapi/asm/bitsperlong.h b/arch/loongarch/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..00b4ba1e5cdf
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __ASM_LOONGARCH_BITSPERLONG_H
+#define __ASM_LOONGARCH_BITSPERLONG_H
+
+#define __BITS_PER_LONG (__SIZEOF_LONG__ * 8)
+
+#include <asm-generic/bitsperlong.h>
+
+#endif /* __ASM_LOONGARCH_BITSPERLONG_H */
diff --git a/arch/loongarch/include/uapi/asm/byteorder.h b/arch/loongarch/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..b1722d890deb
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/byteorder.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
+/*
+ * Author: Hanlu Li <[email protected]>
+ * Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_BYTEORDER_H
+#define _ASM_BYTEORDER_H
+
+#include <linux/byteorder/little_endian.h>
+
+#endif /* _ASM_BYTEORDER_H */
diff --git a/arch/loongarch/include/uapi/asm/reg.h b/arch/loongarch/include/uapi/asm/reg.h
new file mode 100644
index 000000000000..90ad910c60eb
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/reg.h
@@ -0,0 +1,59 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Various register offset definitions for debuggers, core file
+ * examiners and whatnot.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __UAPI_ASM_LOONGARCH_REG_H
+#define __UAPI_ASM_LOONGARCH_REG_H
+
+#define LOONGARCH_EF_R0 0
+#define LOONGARCH_EF_R1 1
+#define LOONGARCH_EF_R2 2
+#define LOONGARCH_EF_R3 3
+#define LOONGARCH_EF_R4 4
+#define LOONGARCH_EF_R5 5
+#define LOONGARCH_EF_R6 6
+#define LOONGARCH_EF_R7 7
+#define LOONGARCH_EF_R8 8
+#define LOONGARCH_EF_R9 9
+#define LOONGARCH_EF_R10 10
+#define LOONGARCH_EF_R11 11
+#define LOONGARCH_EF_R12 12
+#define LOONGARCH_EF_R13 13
+#define LOONGARCH_EF_R14 14
+#define LOONGARCH_EF_R15 15
+#define LOONGARCH_EF_R16 16
+#define LOONGARCH_EF_R17 17
+#define LOONGARCH_EF_R18 18
+#define LOONGARCH_EF_R19 19
+#define LOONGARCH_EF_R20 20
+#define LOONGARCH_EF_R21 21
+#define LOONGARCH_EF_R22 22
+#define LOONGARCH_EF_R23 23
+#define LOONGARCH_EF_R24 24
+#define LOONGARCH_EF_R25 25
+#define LOONGARCH_EF_R26 26
+#define LOONGARCH_EF_R27 27
+#define LOONGARCH_EF_R28 28
+#define LOONGARCH_EF_R29 29
+#define LOONGARCH_EF_R30 30
+#define LOONGARCH_EF_R31 31
+
+/*
+ * Saved special registers
+ */
+#define LOONGARCH_EF_ORIG_A0 32
+#define LOONGARCH_EF_CSR_ERA 33
+#define LOONGARCH_EF_CSR_BADV 34
+#define LOONGARCH_EF_CSR_CRMD 35
+#define LOONGARCH_EF_CSR_PRMD 36
+#define LOONGARCH_EF_CSR_EUEN 37
+#define LOONGARCH_EF_CSR_ECFG 38
+#define LOONGARCH_EF_CSR_ESTAT 39
+
+#define LOONGARCH_EF_SIZE 320 /* size in bytes */
+
+#endif /* __UAPI_ASM_LOONGARCH_REG_H */
diff --git a/tools/include/uapi/asm/bitsperlong.h b/tools/include/uapi/asm/bitsperlong.h
index edba4d93e9e6..da5206517158 100644
--- a/tools/include/uapi/asm/bitsperlong.h
+++ b/tools/include/uapi/asm/bitsperlong.h
@@ -17,6 +17,8 @@
#include "../../../arch/riscv/include/uapi/asm/bitsperlong.h"
#elif defined(__alpha__)
#include "../../../arch/alpha/include/uapi/asm/bitsperlong.h"
+#elif defined(__loongarch__)
+#include "../../../arch/loongarch/include/uapi/asm/bitsperlong.h"
#else
#include <asm-generic/bitsperlong.h>
#endif
--
2.27.0
Add basic boot, setup and reset routines for LoongArch. Now, LoongArch
machines use UEFI-based firmware. The firmware passes configuration
information to the kernel via ACPI and DMI/SMBIOS.
Currently an existing interface between the kernel and the bootloader
is implemented. Kernel gets 2 values from the bootloader, passed in
registers a0 and a1; a0 is an "EFI boot flag" distinguishing UEFI and
non-UEFI firmware, while a1 is a pointer to an FDT with systable,
memmap, cmdline and initrd information.
This interface is used by existing bootloaders for booting kernels in
raw ELF format. However, the standard UEFI boot protocol (EFISTUB) is
preferred.
ECR for adding LoongArch support in ACPI:
https://mantis.uefi.org/mantis/view.php?id=2203
ECR for adding LoongArch support in ACPI (version update):
https://mantis.uefi.org/mantis/view.php?id=2268
ECR for adding LoongArch support in UEFI:
https://mantis.uefi.org/mantis/view.php?id=2313
ACPI changes of LoongArch have been approved in the last year, but the
new version of ACPI SPEC hasn't been made public yet. And UEFI changes
of LoongArch are under review now.
Cc: [email protected]
Cc: Ard Biesheuvel <[email protected]>
Reviewed-by: WANG Xuerui <[email protected]>
Co-developed-by: Yun Liu <[email protected]>
Signed-off-by: Yun Liu <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/acenv.h | 18 +
arch/loongarch/include/asm/acpi.h | 38 ++
arch/loongarch/include/asm/bootinfo.h | 41 +++
arch/loongarch/include/asm/dmi.h | 24 ++
arch/loongarch/include/asm/efi.h | 41 +++
arch/loongarch/include/asm/reboot.h | 10 +
arch/loongarch/include/asm/setup.h | 21 ++
arch/loongarch/kernel/acpi.c | 169 +++++++++
arch/loongarch/kernel/cacheinfo.c | 122 +++++++
arch/loongarch/kernel/cpu-probe.c | 292 +++++++++++++++
arch/loongarch/kernel/efi-header.S | 104 ++++++
arch/loongarch/kernel/efi.c | 226 ++++++++++++
arch/loongarch/kernel/env.c | 70 ++++
arch/loongarch/kernel/head.S | 99 +++++
arch/loongarch/kernel/image-vars.h | 29 ++
arch/loongarch/kernel/mem.c | 64 ++++
arch/loongarch/kernel/reset.c | 90 +++++
arch/loongarch/kernel/setup.c | 341 ++++++++++++++++++
arch/loongarch/kernel/time.c | 213 +++++++++++
arch/loongarch/kernel/topology.c | 13 +
drivers/firmware/efi/Kconfig | 2 +-
drivers/firmware/efi/libstub/Makefile | 10 +
.../firmware/efi/libstub/efi-stub-helper.c | 2 +-
drivers/firmware/efi/libstub/efi-stub.c | 2 +-
drivers/firmware/efi/libstub/loongarch-stub.c | 88 +++++
include/linux/efi.h | 1 +
include/linux/pe.h | 2 +
27 files changed, 2129 insertions(+), 3 deletions(-)
create mode 100644 arch/loongarch/include/asm/acenv.h
create mode 100644 arch/loongarch/include/asm/acpi.h
create mode 100644 arch/loongarch/include/asm/bootinfo.h
create mode 100644 arch/loongarch/include/asm/dmi.h
create mode 100644 arch/loongarch/include/asm/efi.h
create mode 100644 arch/loongarch/include/asm/reboot.h
create mode 100644 arch/loongarch/include/asm/setup.h
create mode 100644 arch/loongarch/kernel/acpi.c
create mode 100644 arch/loongarch/kernel/cacheinfo.c
create mode 100644 arch/loongarch/kernel/cpu-probe.c
create mode 100644 arch/loongarch/kernel/efi-header.S
create mode 100644 arch/loongarch/kernel/efi.c
create mode 100644 arch/loongarch/kernel/env.c
create mode 100644 arch/loongarch/kernel/head.S
create mode 100644 arch/loongarch/kernel/image-vars.h
create mode 100644 arch/loongarch/kernel/mem.c
create mode 100644 arch/loongarch/kernel/reset.c
create mode 100644 arch/loongarch/kernel/setup.c
create mode 100644 arch/loongarch/kernel/time.c
create mode 100644 arch/loongarch/kernel/topology.c
create mode 100644 drivers/firmware/efi/libstub/loongarch-stub.c
diff --git a/arch/loongarch/include/asm/acenv.h b/arch/loongarch/include/asm/acenv.h
new file mode 100644
index 000000000000..52f298f7293b
--- /dev/null
+++ b/arch/loongarch/include/asm/acenv.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * LoongArch specific ACPICA environments and implementation
+ *
+ * Author: Jianmin Lv <[email protected]>
+ * Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef _ASM_LOONGARCH_ACENV_H
+#define _ASM_LOONGARCH_ACENV_H
+
+/*
+ * This header is required by ACPI core, but we have nothing to fill in
+ * right now. Will be updated later when needed.
+ */
+
+#endif /* _ASM_LOONGARCH_ACENV_H */
diff --git a/arch/loongarch/include/asm/acpi.h b/arch/loongarch/include/asm/acpi.h
new file mode 100644
index 000000000000..62044cd5b7bc
--- /dev/null
+++ b/arch/loongarch/include/asm/acpi.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Jianmin Lv <[email protected]>
+ * Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef _ASM_LOONGARCH_ACPI_H
+#define _ASM_LOONGARCH_ACPI_H
+
+#ifdef CONFIG_ACPI
+extern int acpi_strict;
+extern int acpi_disabled;
+extern int acpi_pci_disabled;
+extern int acpi_noirq;
+
+#define acpi_os_ioremap acpi_os_ioremap
+void __init __iomem *acpi_os_ioremap(acpi_physical_address phys, acpi_size size);
+
+static inline void disable_acpi(void)
+{
+ acpi_disabled = 1;
+ acpi_pci_disabled = 1;
+ acpi_noirq = 1;
+}
+
+static inline bool acpi_has_cpu_in_madt(void)
+{
+ return true;
+}
+
+extern struct list_head acpi_wakeup_device_list;
+
+#endif /* !CONFIG_ACPI */
+
+#define ACPI_TABLE_UPGRADE_MAX_PHYS ARCH_LOW_ADDRESS_LIMIT
+
+#endif /* _ASM_LOONGARCH_ACPI_H */
diff --git a/arch/loongarch/include/asm/bootinfo.h b/arch/loongarch/include/asm/bootinfo.h
new file mode 100644
index 000000000000..7b60f202faaf
--- /dev/null
+++ b/arch/loongarch/include/asm/bootinfo.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_BOOTINFO_H
+#define _ASM_BOOTINFO_H
+
+#include <linux/types.h>
+#include <asm/setup.h>
+
+const char *get_system_type(void);
+
+extern void init_environ(void);
+extern void memblock_init(void);
+extern void platform_init(void);
+
+struct loongson_board_info {
+ int bios_size;
+ const char *bios_vendor;
+ const char *bios_version;
+ const char *bios_release_date;
+ const char *board_name;
+ const char *board_vendor;
+};
+
+struct loongson_system_configuration {
+ int nr_cpus;
+ int nr_nodes;
+ int nr_io_pics;
+ int boot_cpu_id;
+ int cores_per_node;
+ int cores_per_package;
+ const char *cpuname;
+};
+
+extern u64 efi_system_table;
+extern unsigned long fw_arg0, fw_arg1;
+extern struct loongson_board_info b_info;
+extern struct loongson_system_configuration loongson_sysconf;
+
+#endif /* _ASM_BOOTINFO_H */
diff --git a/arch/loongarch/include/asm/dmi.h b/arch/loongarch/include/asm/dmi.h
new file mode 100644
index 000000000000..605493417753
--- /dev/null
+++ b/arch/loongarch/include/asm/dmi.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_DMI_H
+#define _ASM_DMI_H
+
+#include <linux/io.h>
+#include <linux/memblock.h>
+
+#define dmi_early_remap(x, l) dmi_remap(x, l)
+#define dmi_early_unmap(x, l) dmi_unmap(x)
+#define dmi_alloc(l) memblock_alloc(l, PAGE_SIZE)
+
+static inline void *dmi_remap(u64 phys_addr, unsigned long size)
+{
+ return ((void *)TO_CACHE(phys_addr));
+}
+
+static inline void dmi_unmap(void *addr)
+{
+}
+
+#endif /* _ASM_DMI_H */
diff --git a/arch/loongarch/include/asm/efi.h b/arch/loongarch/include/asm/efi.h
new file mode 100644
index 000000000000..0127d84d5e1d
--- /dev/null
+++ b/arch/loongarch/include/asm/efi.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_LOONGARCH_EFI_H
+#define _ASM_LOONGARCH_EFI_H
+
+#include <linux/efi.h>
+
+void __init efi_init(void);
+void __init efi_runtime_init(void);
+void efifb_setup_from_dmi(struct screen_info *si, const char *opt);
+
+#define ARCH_EFI_IRQ_FLAGS_MASK 0x00000004 /* Bit 2: CSR.CRMD.IE */
+
+#define arch_efi_call_virt_setup() \
+({ \
+})
+
+#define arch_efi_call_virt(p, f, args...) \
+({ \
+ efi_##f##_t * __f; \
+ __f = p->f; \
+ __f(args); \
+})
+
+#define arch_efi_call_virt_teardown() \
+({ \
+})
+
+#define EFI_ALLOC_ALIGN SZ_64K
+
+struct screen_info *alloc_screen_info(void);
+void free_screen_info(struct screen_info *si);
+
+static inline unsigned long efi_get_max_initrd_addr(unsigned long image_addr)
+{
+ return ULONG_MAX;
+}
+
+#endif /* _ASM_LOONGARCH_EFI_H */
diff --git a/arch/loongarch/include/asm/reboot.h b/arch/loongarch/include/asm/reboot.h
new file mode 100644
index 000000000000..51151749d8f0
--- /dev/null
+++ b/arch/loongarch/include/asm/reboot.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_REBOOT_H
+#define _ASM_REBOOT_H
+
+extern void (*pm_restart)(void);
+
+#endif /* _ASM_REBOOT_H */
diff --git a/arch/loongarch/include/asm/setup.h b/arch/loongarch/include/asm/setup.h
new file mode 100644
index 000000000000..6d7d2a3e23dd
--- /dev/null
+++ b/arch/loongarch/include/asm/setup.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef _LOONGARCH_SETUP_H
+#define _LOONGARCH_SETUP_H
+
+#include <linux/types.h>
+#include <uapi/asm/setup.h>
+
+#define VECSIZE 0x200
+
+extern unsigned long eentry;
+extern unsigned long tlbrentry;
+extern void cpu_cache_init(void);
+extern void per_cpu_trap_init(int cpu);
+extern void set_handler(unsigned long offset, void *addr, unsigned long len);
+extern void set_merr_handler(unsigned long offset, void *addr, unsigned long len);
+
+#endif /* __SETUP_H */
diff --git a/arch/loongarch/kernel/acpi.c b/arch/loongarch/kernel/acpi.c
new file mode 100644
index 000000000000..a644220bb426
--- /dev/null
+++ b/arch/loongarch/kernel/acpi.c
@@ -0,0 +1,169 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * acpi.c - Architecture-Specific Low-Level ACPI Boot Support
+ *
+ * Author: Jianmin Lv <[email protected]>
+ * Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/init.h>
+#include <linux/acpi.h>
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
+#include <linux/memblock.h>
+#include <linux/serial_core.h>
+#include <asm/io.h>
+#include <asm/loongson.h>
+
+int acpi_disabled;
+EXPORT_SYMBOL(acpi_disabled);
+int acpi_noirq;
+int acpi_pci_disabled;
+EXPORT_SYMBOL(acpi_pci_disabled);
+int acpi_strict = 1; /* We have no workarounds on LoongArch */
+int num_processors;
+int disabled_cpus;
+enum acpi_irq_model_id acpi_irq_model = ACPI_IRQ_MODEL_PLATFORM;
+
+u64 acpi_saved_sp;
+
+#define MAX_CORE_PIC 256
+
+#define PREFIX "ACPI: "
+
+int acpi_gsi_to_irq(u32 gsi, unsigned int *irqp)
+{
+ if (irqp != NULL)
+ *irqp = acpi_register_gsi(NULL, gsi, -1, -1);
+ return (*irqp >= 0) ? 0 : -EINVAL;
+}
+EXPORT_SYMBOL_GPL(acpi_gsi_to_irq);
+
+int acpi_isa_irq_to_gsi(unsigned int isa_irq, u32 *gsi)
+{
+ if (gsi)
+ *gsi = isa_irq;
+ return 0;
+}
+
+/*
+ * success: return IRQ number (>=0)
+ * failure: return < 0
+ */
+int acpi_register_gsi(struct device *dev, u32 gsi, int trigger, int polarity)
+{
+ struct irq_fwspec fwspec;
+
+ switch (gsi) {
+ case GSI_MIN_CPU_IRQ ... GSI_MAX_CPU_IRQ:
+ fwspec.fwnode = liointc_domain->fwnode;
+ fwspec.param[0] = gsi - GSI_MIN_CPU_IRQ;
+ fwspec.param_count = 1;
+
+ return irq_create_fwspec_mapping(&fwspec);
+
+ case GSI_MIN_LPC_IRQ ... GSI_MAX_LPC_IRQ:
+ if (!pch_lpc_domain)
+ return -EINVAL;
+
+ fwspec.fwnode = pch_lpc_domain->fwnode;
+ fwspec.param[0] = gsi - GSI_MIN_LPC_IRQ;
+ fwspec.param[1] = acpi_dev_get_irq_type(trigger, polarity);
+ fwspec.param_count = 2;
+
+ return irq_create_fwspec_mapping(&fwspec);
+
+ case GSI_MIN_PCH_IRQ ... GSI_MAX_PCH_IRQ:
+ if (!pch_pic_domain[0])
+ return -EINVAL;
+
+ fwspec.fwnode = pch_pic_domain[0]->fwnode;
+ fwspec.param[0] = gsi - GSI_MIN_PCH_IRQ;
+ fwspec.param[1] = IRQ_TYPE_LEVEL_HIGH;
+ fwspec.param_count = 2;
+
+ return irq_create_fwspec_mapping(&fwspec);
+ }
+
+ return -EINVAL;
+}
+EXPORT_SYMBOL_GPL(acpi_register_gsi);
+
+void acpi_unregister_gsi(u32 gsi)
+{
+
+}
+EXPORT_SYMBOL_GPL(acpi_unregister_gsi);
+
+void __init __iomem * __acpi_map_table(unsigned long phys, unsigned long size)
+{
+
+ if (!phys || !size)
+ return NULL;
+
+ return early_memremap(phys, size);
+}
+void __init __acpi_unmap_table(void __iomem *map, unsigned long size)
+{
+ if (!map || !size)
+ return;
+
+ early_memunmap(map, size);
+}
+
+void __init __iomem *acpi_os_ioremap(acpi_physical_address phys, acpi_size size)
+{
+ if (!memblock_is_memory(phys))
+ return ioremap(phys, size);
+ else
+ return ioremap_cache(phys, size);
+}
+
+void __init acpi_boot_table_init(void)
+{
+ /*
+ * If acpi_disabled, bail out
+ */
+ if (acpi_disabled)
+ return;
+
+ /*
+ * Initialize the ACPI boot-time table parser.
+ */
+ if (acpi_table_init()) {
+ disable_acpi();
+ return;
+ }
+}
+
+static void __init acpi_process_madt(void)
+{
+ loongson_sysconf.nr_cpus = num_processors;
+}
+
+int __init acpi_boot_init(void)
+{
+ /*
+ * If acpi_disabled, bail out
+ */
+ if (acpi_disabled)
+ return -1;
+
+ loongson_sysconf.boot_cpu_id = read_csr_cpuid();
+
+ /*
+ * Process the Multiple APIC Description Table (MADT), if present
+ */
+ acpi_process_madt();
+
+ /* Do not enable ACPI SPCR console by default */
+ acpi_parse_spcr(earlycon_acpi_spcr_enable, false);
+
+ return 0;
+}
+
+void __init arch_reserve_mem_area(acpi_physical_address addr, size_t size)
+{
+ memblock_reserve(addr, size);
+}
diff --git a/arch/loongarch/kernel/cacheinfo.c b/arch/loongarch/kernel/cacheinfo.c
new file mode 100644
index 000000000000..8c9fe29e98f0
--- /dev/null
+++ b/arch/loongarch/kernel/cacheinfo.c
@@ -0,0 +1,122 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * LoongArch cacheinfo support
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/cacheinfo.h>
+
+/* Populates leaf and increments to next leaf */
+#define populate_cache(cache, leaf, c_level, c_type) \
+do { \
+ leaf->type = c_type; \
+ leaf->level = c_level; \
+ leaf->coherency_line_size = c->cache.linesz; \
+ leaf->number_of_sets = c->cache.sets; \
+ leaf->ways_of_associativity = c->cache.ways; \
+ leaf->size = c->cache.linesz * c->cache.sets * \
+ c->cache.ways; \
+ leaf++; \
+} while (0)
+
+int init_cache_level(unsigned int cpu)
+{
+ struct cpuinfo_loongarch *c = ¤t_cpu_data;
+ struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
+ int levels = 0, leaves = 0;
+
+ /*
+ * If Dcache is not set, we assume the cache structures
+ * are not properly initialized.
+ */
+ if (c->dcache.waysize)
+ levels += 1;
+ else
+ return -ENOENT;
+
+
+ leaves += (c->icache.waysize) ? 2 : 1;
+
+ if (c->vcache.waysize) {
+ levels++;
+ leaves++;
+ }
+
+ if (c->scache.waysize) {
+ levels++;
+ leaves++;
+ }
+
+ if (c->tcache.waysize) {
+ levels++;
+ leaves++;
+ }
+
+ this_cpu_ci->num_levels = levels;
+ this_cpu_ci->num_leaves = leaves;
+ return 0;
+}
+
+static inline bool cache_leaves_are_shared(struct cacheinfo *this_leaf,
+ struct cacheinfo *sib_leaf)
+{
+ return !((this_leaf->level == 1) || (this_leaf->level == 2));
+}
+
+static void cache_cpumap_setup(unsigned int cpu)
+{
+ struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
+ struct cacheinfo *this_leaf, *sib_leaf;
+ unsigned int index;
+
+ for (index = 0; index < this_cpu_ci->num_leaves; index++) {
+ unsigned int i;
+
+ this_leaf = this_cpu_ci->info_list + index;
+ /* skip if shared_cpu_map is already populated */
+ if (!cpumask_empty(&this_leaf->shared_cpu_map))
+ continue;
+
+ cpumask_set_cpu(cpu, &this_leaf->shared_cpu_map);
+ for_each_online_cpu(i) {
+ struct cpu_cacheinfo *sib_cpu_ci = get_cpu_cacheinfo(i);
+
+ if (i == cpu || !sib_cpu_ci->info_list)
+ continue;/* skip if itself or no cacheinfo */
+ sib_leaf = sib_cpu_ci->info_list + index;
+ if (cache_leaves_are_shared(this_leaf, sib_leaf)) {
+ cpumask_set_cpu(cpu, &sib_leaf->shared_cpu_map);
+ cpumask_set_cpu(i, &this_leaf->shared_cpu_map);
+ }
+ }
+ }
+}
+
+int populate_cache_leaves(unsigned int cpu)
+{
+ int level = 1;
+ struct cpuinfo_loongarch *c = ¤t_cpu_data;
+ struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
+ struct cacheinfo *this_leaf = this_cpu_ci->info_list;
+
+ if (c->icache.waysize) {
+ populate_cache(dcache, this_leaf, level, CACHE_TYPE_DATA);
+ populate_cache(icache, this_leaf, level++, CACHE_TYPE_INST);
+ } else {
+ populate_cache(dcache, this_leaf, level++, CACHE_TYPE_UNIFIED);
+ }
+
+ if (c->vcache.waysize)
+ populate_cache(vcache, this_leaf, level++, CACHE_TYPE_UNIFIED);
+
+ if (c->scache.waysize)
+ populate_cache(scache, this_leaf, level++, CACHE_TYPE_UNIFIED);
+
+ if (c->tcache.waysize)
+ populate_cache(tcache, this_leaf, level++, CACHE_TYPE_UNIFIED);
+
+ cache_cpumap_setup(cpu);
+ this_cpu_ci->cpu_map_populated = true;
+
+ return 0;
+}
diff --git a/arch/loongarch/kernel/cpu-probe.c b/arch/loongarch/kernel/cpu-probe.c
new file mode 100644
index 000000000000..6c87ea36b257
--- /dev/null
+++ b/arch/loongarch/kernel/cpu-probe.c
@@ -0,0 +1,292 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Processor capabilities determination functions.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/ptrace.h>
+#include <linux/smp.h>
+#include <linux/stddef.h>
+#include <linux/export.h>
+#include <linux/printk.h>
+#include <linux/uaccess.h>
+
+#include <asm/cpu-features.h>
+#include <asm/elf.h>
+#include <asm/fpu.h>
+#include <asm/loongarch.h>
+#include <asm/pgtable-bits.h>
+#include <asm/setup.h>
+
+/* Hardware capabilities */
+unsigned int elf_hwcap __read_mostly;
+EXPORT_SYMBOL_GPL(elf_hwcap);
+
+/*
+ * Determine the FCSR mask for FPU hardware.
+ */
+static inline void cpu_set_fpu_fcsr_mask(struct cpuinfo_loongarch *c)
+{
+ unsigned long sr, mask, fcsr, fcsr0, fcsr1;
+
+ fcsr = c->fpu_csr0;
+ mask = FPU_CSR_ALL_X | FPU_CSR_ALL_E | FPU_CSR_ALL_S | FPU_CSR_RM;
+
+ sr = read_csr_euen();
+ enable_fpu();
+
+ fcsr0 = fcsr & mask;
+ write_fcsr(LOONGARCH_FCSR0, fcsr0);
+ fcsr0 = read_fcsr(LOONGARCH_FCSR0);
+
+ fcsr1 = fcsr | ~mask;
+ write_fcsr(LOONGARCH_FCSR0, fcsr1);
+ fcsr1 = read_fcsr(LOONGARCH_FCSR0);
+
+ write_fcsr(LOONGARCH_FCSR0, fcsr);
+
+ write_csr_euen(sr);
+
+ c->fpu_mask = ~(fcsr0 ^ fcsr1) & ~mask;
+}
+
+static inline void set_elf_platform(int cpu, const char *plat)
+{
+ if (cpu == 0)
+ __elf_platform = plat;
+}
+
+/* MAP BASE */
+unsigned long vm_map_base;
+EXPORT_SYMBOL_GPL(vm_map_base);
+
+static void cpu_probe_addrbits(struct cpuinfo_loongarch *c)
+{
+#ifdef __NEED_ADDRBITS_PROBE
+ c->pabits = (read_cpucfg(LOONGARCH_CPUCFG1) & CPUCFG1_PABITS) >> 4;
+ c->vabits = (read_cpucfg(LOONGARCH_CPUCFG1) & CPUCFG1_VABITS) >> 12;
+ vm_map_base = 0UL - (1UL << c->vabits);
+#endif
+}
+
+static void set_isa(struct cpuinfo_loongarch *c, unsigned int isa)
+{
+ switch (isa) {
+ case LOONGARCH_CPU_ISA_LA64:
+ c->isa_level |= LOONGARCH_CPU_ISA_LA64;
+ fallthrough;
+ case LOONGARCH_CPU_ISA_LA32S:
+ c->isa_level |= LOONGARCH_CPU_ISA_LA32S;
+ fallthrough;
+ case LOONGARCH_CPU_ISA_LA32R:
+ c->isa_level |= LOONGARCH_CPU_ISA_LA32R;
+ break;
+ }
+}
+
+static void cpu_probe_common(struct cpuinfo_loongarch *c)
+{
+ unsigned int config;
+ unsigned long asid_mask;
+
+ c->options = LOONGARCH_CPU_CPUCFG | LOONGARCH_CPU_CSR |
+ LOONGARCH_CPU_TLB | LOONGARCH_CPU_VINT | LOONGARCH_CPU_WATCH;
+
+ elf_hwcap |= HWCAP_LOONGARCH_CRC32;
+
+ config = read_cpucfg(LOONGARCH_CPUCFG1);
+ if (config & CPUCFG1_UAL) {
+ c->options |= LOONGARCH_CPU_UAL;
+ elf_hwcap |= HWCAP_LOONGARCH_UAL;
+ }
+
+ config = read_cpucfg(LOONGARCH_CPUCFG2);
+ if (config & CPUCFG2_LAM) {
+ c->options |= LOONGARCH_CPU_LAM;
+ elf_hwcap |= HWCAP_LOONGARCH_LAM;
+ }
+ if (config & CPUCFG2_FP) {
+ c->options |= LOONGARCH_CPU_FPU;
+ elf_hwcap |= HWCAP_LOONGARCH_FPU;
+ }
+ if (config & CPUCFG2_COMPLEX) {
+ c->options |= LOONGARCH_CPU_COMPLEX;
+ elf_hwcap |= HWCAP_LOONGARCH_COMPLEX;
+ }
+ if (config & CPUCFG2_CRYPTO) {
+ c->options |= LOONGARCH_CPU_CRYPTO;
+ elf_hwcap |= HWCAP_LOONGARCH_CRYPTO;
+ }
+ if (config & CPUCFG2_LVZP) {
+ c->options |= LOONGARCH_CPU_LVZ;
+ elf_hwcap |= HWCAP_LOONGARCH_LVZ;
+ }
+
+ config = read_cpucfg(LOONGARCH_CPUCFG6);
+ if (config & CPUCFG6_PMP)
+ c->options |= LOONGARCH_CPU_PMP;
+
+ config = iocsr_read32(LOONGARCH_IOCSR_FEATURES);
+ if (config & IOCSRF_CSRIPI)
+ c->options |= LOONGARCH_CPU_CSRIPI;
+ if (config & IOCSRF_EXTIOI)
+ c->options |= LOONGARCH_CPU_EXTIOI;
+ if (config & IOCSRF_FREQSCALE)
+ c->options |= LOONGARCH_CPU_SCALEFREQ;
+ if (config & IOCSRF_FLATMODE)
+ c->options |= LOONGARCH_CPU_FLATMODE;
+ if (config & IOCSRF_EIODECODE)
+ c->options |= LOONGARCH_CPU_EIODECODE;
+ if (config & IOCSRF_VM)
+ c->options |= LOONGARCH_CPU_HYPERVISOR;
+
+ config = csr_read32(LOONGARCH_CSR_ASID);
+ config = (config & CSR_ASID_BIT) >> CSR_ASID_BIT_SHIFT;
+ asid_mask = GENMASK(config - 1, 0);
+ set_cpu_asid_mask(c, asid_mask);
+
+ config = read_csr_prcfg1();
+ c->ksave_mask = GENMASK((config & CSR_CONF1_KSNUM) - 1, 0);
+ c->ksave_mask &= ~(EXC_KSAVE_MASK | PERCPU_KSAVE_MASK | KVM_KSAVE_MASK);
+
+ config = read_csr_prcfg3();
+ switch (config & CSR_CONF3_TLBTYPE) {
+ case 0:
+ c->tlbsizemtlb = 0;
+ c->tlbsizestlbsets = 0;
+ c->tlbsizestlbways = 0;
+ c->tlbsize = 0;
+ break;
+ case 1:
+ c->tlbsizemtlb = ((config & CSR_CONF3_MTLBSIZE) >> CSR_CONF3_MTLBSIZE_SHIFT) + 1;
+ c->tlbsizestlbsets = 0;
+ c->tlbsizestlbways = 0;
+ c->tlbsize = c->tlbsizemtlb + c->tlbsizestlbsets * c->tlbsizestlbways;
+ break;
+ case 2:
+ c->tlbsizemtlb = ((config & CSR_CONF3_MTLBSIZE) >> CSR_CONF3_MTLBSIZE_SHIFT) + 1;
+ c->tlbsizestlbsets = 1 << ((config & CSR_CONF3_STLBIDX) >> CSR_CONF3_STLBIDX_SHIFT);
+ c->tlbsizestlbways = ((config & CSR_CONF3_STLBWAYS) >> CSR_CONF3_STLBWAYS_SHIFT) + 1;
+ c->tlbsize = c->tlbsizemtlb + c->tlbsizestlbsets * c->tlbsizestlbways;
+ break;
+ default:
+ pr_warn("Warning: unknown TLB type\n");
+ }
+}
+
+#define MAX_NAME_LEN 32
+#define VENDOR_OFFSET 0
+#define CPUNAME_OFFSET 9
+
+static char cpu_full_name[MAX_NAME_LEN] = " - ";
+
+static inline void cpu_probe_loongson(struct cpuinfo_loongarch *c, unsigned int cpu)
+{
+ uint64_t *vendor = (void *)(&cpu_full_name[VENDOR_OFFSET]);
+ uint64_t *cpuname = (void *)(&cpu_full_name[CPUNAME_OFFSET]);
+
+ __cpu_full_name[cpu] = cpu_full_name;
+ *vendor = iocsr_read64(LOONGARCH_IOCSR_VENDOR);
+ *cpuname = iocsr_read64(LOONGARCH_IOCSR_CPUNAME);
+
+ switch (c->processor_id & PRID_SERIES_MASK) {
+ case PRID_SERIES_LA132:
+ c->cputype = CPU_LOONGSON32;
+ set_isa(c, LOONGARCH_CPU_ISA_LA32S);
+ __cpu_family[cpu] = "Loongson-32bit";
+ pr_info("32-bit Loongson Processor probed (LA132 Core)\n");
+ break;
+ case PRID_SERIES_LA264:
+ c->cputype = CPU_LOONGSON64;
+ set_isa(c, LOONGARCH_CPU_ISA_LA64);
+ __cpu_family[cpu] = "Loongson-64bit";
+ pr_info("64-bit Loongson Processor probed (LA264 Core)\n");
+ break;
+ case PRID_SERIES_LA364:
+ c->cputype = CPU_LOONGSON64;
+ set_isa(c, LOONGARCH_CPU_ISA_LA64);
+ __cpu_family[cpu] = "Loongson-64bit";
+ pr_info("64-bit Loongson Processor probed (LA364 Core)\n");
+ break;
+ case PRID_SERIES_LA464:
+ c->cputype = CPU_LOONGSON64;
+ set_isa(c, LOONGARCH_CPU_ISA_LA64);
+ __cpu_family[cpu] = "Loongson-64bit";
+ pr_info("64-bit Loongson Processor probed (LA464 Core)\n");
+ break;
+ case PRID_SERIES_LA664:
+ c->cputype = CPU_LOONGSON64;
+ set_isa(c, LOONGARCH_CPU_ISA_LA64);
+ __cpu_family[cpu] = "Loongson-64bit";
+ pr_info("64-bit Loongson Processor probed (LA664 Core)\n");
+ break;
+ default: /* Default to 64 bit */
+ c->cputype = CPU_LOONGSON64;
+ set_isa(c, LOONGARCH_CPU_ISA_LA64);
+ __cpu_family[cpu] = "Loongson-64bit";
+ pr_info("64-bit Loongson Processor probed (Unknown Core)\n");
+ }
+}
+
+#ifdef CONFIG_64BIT
+/* For use by uaccess.h */
+u64 __ua_limit;
+EXPORT_SYMBOL(__ua_limit);
+#endif
+
+const char *__cpu_family[NR_CPUS];
+const char *__cpu_full_name[NR_CPUS];
+const char *__elf_platform;
+
+static void cpu_report(void)
+{
+ struct cpuinfo_loongarch *c = ¤t_cpu_data;
+
+ pr_info("CPU%d revision is: %08x (%s)\n",
+ smp_processor_id(), c->processor_id, cpu_family_string());
+ if (c->options & LOONGARCH_CPU_FPU)
+ pr_info("FPU%d revision is: %08x\n", smp_processor_id(), c->fpu_vers);
+}
+
+void cpu_probe(void)
+{
+ unsigned int cpu = smp_processor_id();
+ struct cpuinfo_loongarch *c = ¤t_cpu_data;
+
+ /*
+ * Set a default ELF platform, cpu probe may later
+ * overwrite it with a more precise value
+ */
+ set_elf_platform(cpu, "loongarch");
+
+ c->cputype = CPU_UNKNOWN;
+ c->processor_id = read_cpucfg(LOONGARCH_CPUCFG0);
+ c->fpu_vers = (read_cpucfg(LOONGARCH_CPUCFG2) >> 3) & 0x3;
+
+ c->fpu_csr0 = FPU_CSR_RN;
+ c->fpu_mask = FPU_CSR_RSVD;
+
+ cpu_probe_common(c);
+
+ per_cpu_trap_init(cpu);
+
+ switch (c->processor_id & PRID_COMP_MASK) {
+ case PRID_COMP_LOONGSON:
+ cpu_probe_loongson(c, cpu);
+ break;
+ }
+
+ BUG_ON(!__cpu_family[cpu]);
+ BUG_ON(c->cputype == CPU_UNKNOWN);
+
+ cpu_probe_addrbits(c);
+
+#ifdef CONFIG_64BIT
+ if (cpu == 0)
+ __ua_limit = ~((1ull << cpu_vabits) - 1);
+#endif
+
+ cpu_report();
+}
diff --git a/arch/loongarch/kernel/efi-header.S b/arch/loongarch/kernel/efi-header.S
new file mode 100644
index 000000000000..1d9c735fe6d4
--- /dev/null
+++ b/arch/loongarch/kernel/efi-header.S
@@ -0,0 +1,104 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/pe.h>
+#include <linux/sizes.h>
+
+ .macro __EFI_PE_HEADER
+ .long PE_MAGIC
+.Lcoff_header:
+#ifdef CONFIG_32BIT
+ .short IMAGE_FILE_MACHINE_LOONGARCH32 /* Machine */
+#else
+ .short IMAGE_FILE_MACHINE_LOONGARCH64 /* Machine */
+#endif
+ .short .Lsection_count /* NumberOfSections */
+ .long 0 /* TimeDateStamp */
+ .long 0 /* PointerToSymbolTable */
+ .long 0 /* NumberOfSymbols */
+ .short .Lsection_table - .Loptional_header /* SizeOfOptionalHeader */
+ .short IMAGE_FILE_DEBUG_STRIPPED | \
+ IMAGE_FILE_EXECUTABLE_IMAGE | \
+ IMAGE_FILE_LINE_NUMS_STRIPPED /* Characteristics */
+
+.Loptional_header:
+ .short PE_OPT_MAGIC_PE32PLUS /* PE32+ format */
+ .byte 0x02 /* MajorLinkerVersion */
+ .byte 0x14 /* MinorLinkerVersion */
+ .long __inittext_end - .Lefi_header_end /* SizeOfCode */
+ .long _end - __initdata_begin /* SizeOfInitializedData */
+ .long 0 /* SizeOfUninitializedData */
+ .long __efistub_efi_pe_entry - _head /* AddressOfEntryPoint */
+ .long .Lefi_header_end - _head /* BaseOfCode */
+
+.Lextra_header_fields:
+ .quad 0 /* ImageBase */
+ .long PECOFF_SEGMENT_ALIGN /* SectionAlignment */
+ .long PECOFF_FILE_ALIGN /* FileAlignment */
+ .short 0 /* MajorOperatingSystemVersion */
+ .short 0 /* MinorOperatingSystemVersion */
+ .short LINUX_EFISTUB_MAJOR_VERSION /* MajorImageVersion */
+ .short LINUX_EFISTUB_MINOR_VERSION /* MinorImageVersion */
+ .short 0 /* MajorSubsystemVersion */
+ .short 0 /* MinorSubsystemVersion */
+ .long 0 /* Win32VersionValue */
+
+ .long _end - _head /* SizeOfImage */
+
+ /* Everything before the kernel image is considered part of the header */
+ .long .Lefi_header_end - _head /* SizeOfHeaders */
+ .long 0 /* CheckSum */
+ .short IMAGE_SUBSYSTEM_EFI_APPLICATION /* Subsystem */
+ .short 0 /* DllCharacteristics */
+ .quad 0 /* SizeOfStackReserve */
+ .quad 0 /* SizeOfStackCommit */
+ .quad 0 /* SizeOfHeapReserve */
+ .quad 0 /* SizeOfHeapCommit */
+ .long 0 /* LoaderFlags */
+ .long (.Lsection_table - .) / 8 /* NumberOfRvaAndSizes */
+
+ .quad 0 /* ExportTable */
+ .quad 0 /* ImportTable */
+ .quad 0 /* ResourceTable */
+ .quad 0 /* ExceptionTable */
+ .quad 0 /* CertificationTable */
+ .quad 0 /* BaseRelocationTable */
+
+ /* Section table */
+.Lsection_table:
+ .ascii ".text\0\0\0"
+ .long __inittext_end - .Lefi_header_end /* VirtualSize */
+ .long .Lefi_header_end - _head /* VirtualAddress */
+ .long __inittext_end - .Lefi_header_end /* SizeOfRawData */
+ .long .Lefi_header_end - _head /* PointerToRawData */
+
+ .long 0 /* PointerToRelocations */
+ .long 0 /* PointerToLineNumbers */
+ .short 0 /* NumberOfRelocations */
+ .short 0 /* NumberOfLineNumbers */
+ .long IMAGE_SCN_CNT_CODE | \
+ IMAGE_SCN_MEM_READ | \
+ IMAGE_SCN_MEM_EXECUTE /* Characteristics */
+
+ .ascii ".data\0\0\0"
+ .long _end - __initdata_begin /* VirtualSize */
+ .long __initdata_begin - _head /* VirtualAddress */
+ .long _edata - __initdata_begin /* SizeOfRawData */
+ .long __initdata_begin - _head /* PointerToRawData */
+
+ .long 0 /* PointerToRelocations */
+ .long 0 /* PointerToLineNumbers */
+ .short 0 /* NumberOfRelocations */
+ .short 0 /* NumberOfLineNumbers */
+ .long IMAGE_SCN_CNT_INITIALIZED_DATA | \
+ IMAGE_SCN_MEM_READ | \
+ IMAGE_SCN_MEM_WRITE /* Characteristics */
+
+ .org 0x20e
+ .word kernel_version - 512 - _head
+
+ .set .Lsection_count, (. - .Lsection_table) / 40
+.Lefi_header_end:
+ .endm
diff --git a/arch/loongarch/kernel/efi.c b/arch/loongarch/kernel/efi.c
new file mode 100644
index 000000000000..d36a325a990c
--- /dev/null
+++ b/arch/loongarch/kernel/efi.c
@@ -0,0 +1,226 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * EFI initialization
+ *
+ * Author: Jianmin Lv <[email protected]>
+ * Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/acpi.h>
+#include <linux/efi.h>
+#include <linux/efi-bgrt.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/export.h>
+#include <linux/io.h>
+#include <linux/kobject.h>
+#include <linux/memblock.h>
+#include <linux/reboot.h>
+#include <linux/uaccess.h>
+
+#include <asm/early_ioremap.h>
+#include <asm/efi.h>
+#include <asm/tlb.h>
+#include <asm/loongson.h>
+
+static unsigned long efi_nr_tables;
+static unsigned long efi_config_table;
+static unsigned long screen_info_table __initdata = EFI_INVALID_TABLE_ADDR;
+
+static efi_system_table_t *efi_systab;
+static efi_config_table_type_t arch_tables[] __initdata = {
+ {LINUX_EFI_LARCH_SCREEN_INFO_TABLE_GUID, &screen_info_table, "SINFO"},
+ {},
+};
+
+static void __init init_screen_info(void)
+{
+ struct screen_info *si;
+
+ if (screen_info_table == EFI_INVALID_TABLE_ADDR)
+ return;
+
+ si = early_memremap_ro(screen_info_table, sizeof(*si));
+ if (!si) {
+ pr_err("Could not map screen_info config table\n");
+ return;
+ }
+ screen_info = *si;
+ early_memunmap(si, sizeof(*si));
+
+ if (screen_info.orig_video_isVGA == VIDEO_TYPE_EFI)
+ memblock_reserve(screen_info.lfb_base, screen_info.lfb_size);
+}
+
+static void __init create_tlb(u32 index, u64 vppn, u32 ps, u32 mat)
+{
+ unsigned long tlblo0, tlblo1;
+
+ write_csr_pagesize(ps);
+
+ tlblo0 = vppn | CSR_TLBLO0_V | CSR_TLBLO0_WE |
+ CSR_TLBLO0_GLOBAL | (mat << CSR_TLBLO0_CCA_SHIFT);
+ tlblo1 = tlblo0 + (1 << ps);
+
+ csr_write64(vppn, LOONGARCH_CSR_TLBEHI);
+ csr_write64(tlblo0, LOONGARCH_CSR_TLBELO0);
+ csr_write64(tlblo1, LOONGARCH_CSR_TLBELO1);
+ csr_xchg32(0, CSR_TLBIDX_EHINV, LOONGARCH_CSR_TLBIDX);
+ csr_xchg32(index, CSR_TLBIDX_IDX, LOONGARCH_CSR_TLBIDX);
+
+ tlb_write_indexed();
+}
+
+#define MTLB_ENTRY_INDEX 0x800
+
+/* Create VA == PA mapping as UEFI */
+static void __init fix_efi_mapping(void)
+{
+ unsigned int index = MTLB_ENTRY_INDEX;
+ unsigned int tlbnr = boot_cpu_data.tlbsizemtlb - 2;
+ unsigned long i, vppn;
+
+ /* Low Memory, Cached */
+ create_tlb(index++, 0x00000000, PS_128M, 1);
+ /* MMIO Registers, Uncached */
+ create_tlb(index++, 0x10000000, PS_128M, 0);
+
+ /* High Memory, Cached */
+ for (i = 0; i < tlbnr; i++) {
+ vppn = 0x80000000ULL + (i * SZ_2G);
+ create_tlb(index++, vppn, PS_1G, 1);
+ }
+}
+
+/*
+ * set_virtual_map() - create a virtual mapping for the EFI memory map and call
+ * efi_set_virtual_address_map enter virtual for runtime service
+ *
+ * This function populates the virt_addr fields of all memory region descriptors
+ * in @memory_map whose EFI_MEMORY_RUNTIME attribute is set. Those descriptors
+ * are also copied to @runtime_map, and their total count is returned in @count.
+ */
+static int __init set_virtual_map(void)
+{
+ int count = 0;
+ unsigned int size;
+ unsigned long attr;
+ efi_status_t status;
+ efi_runtime_services_t *rt;
+ efi_set_virtual_address_map_t *svam;
+ efi_memory_desc_t *in, runtime_map[32];
+
+ size = sizeof(efi_memory_desc_t);
+
+ for_each_efi_memory_desc(in) {
+ attr = in->attribute;
+ if (!(attr & EFI_MEMORY_RUNTIME))
+ continue;
+
+ if (attr & (EFI_MEMORY_WB | EFI_MEMORY_WT))
+ in->virt_addr = TO_CACHE(in->phys_addr);
+ else
+ in->virt_addr = TO_UNCACHE(in->phys_addr);
+
+ memcpy(&runtime_map[count++], in, size);
+ }
+
+ rt = early_memremap_ro((unsigned long)efi_systab->runtime, sizeof(*rt));
+
+ /* Install the new virtual address map */
+ svam = rt->set_virtual_address_map;
+
+ fix_efi_mapping();
+
+ status = svam(size * count, size, efi.memmap.desc_version,
+ (efi_memory_desc_t *)TO_PHYS((unsigned long)runtime_map));
+
+ local_flush_tlb_all();
+ write_csr_pagesize(PS_DEFAULT_SIZE);
+
+ return 0;
+}
+
+void __init efi_runtime_init(void)
+{
+ efi_status_t status;
+
+ if (!efi_enabled(EFI_BOOT))
+ return;
+
+ if (!efi_systab->runtime)
+ return;
+
+ status = set_virtual_map();
+ if (status < 0)
+ return;
+
+ if (efi_runtime_disabled()) {
+ pr_info("EFI runtime services will be disabled.\n");
+ return;
+ }
+
+ efi.runtime = (efi_runtime_services_t *)efi_systab->runtime;
+ efi.runtime_version = (unsigned int)efi.runtime->hdr.revision;
+
+ efi_native_runtime_setup();
+ set_bit(EFI_RUNTIME_SERVICES, &efi.flags);
+}
+
+void __init efi_init(void)
+{
+ int size;
+ void *config_tables;
+
+ if (!efi_system_table)
+ return;
+
+ efi_systab = (efi_system_table_t *)early_memremap_ro(efi_system_table, sizeof(*efi_systab));
+ if (!efi_systab) {
+ pr_err("Can't find EFI system table.\n");
+ return;
+ }
+
+ set_bit(EFI_64BIT, &efi.flags);
+ efi_nr_tables = efi_systab->nr_tables;
+ efi_config_table = (unsigned long)efi_systab->tables;
+
+ size = sizeof(efi_config_table_t);
+ config_tables = early_memremap(efi_config_table, efi_nr_tables * size);
+ efi_config_parse_tables(config_tables, efi_systab->nr_tables, arch_tables);
+ early_memunmap(config_tables, efi_nr_tables * size);
+
+ init_screen_info();
+}
+
+static ssize_t boardinfo_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return sprintf(buf,
+ "BIOS Information\n"
+ "Vendor\t\t\t: %s\n"
+ "Version\t\t\t: %s\n"
+ "ROM Size\t\t: %d KB\n"
+ "Release Date\t\t: %s\n\n"
+ "Board Information\n"
+ "Manufacturer\t\t: %s\n"
+ "Board Name\t\t: %s\n"
+ "Family\t\t\t: LOONGSON64\n\n",
+ b_info.bios_vendor, b_info.bios_version,
+ b_info.bios_size, b_info.bios_release_date,
+ b_info.board_vendor, b_info.board_name);
+}
+
+static struct kobj_attribute boardinfo_attr = __ATTR(boardinfo, 0444,
+ boardinfo_show, NULL);
+
+static int __init boardinfo_init(void)
+{
+ if (!efi_kobj)
+ return -EINVAL;
+
+ return sysfs_create_file(efi_kobj, &boardinfo_attr.attr);
+}
+late_initcall(boardinfo_init);
diff --git a/arch/loongarch/kernel/env.c b/arch/loongarch/kernel/env.c
new file mode 100644
index 000000000000..bf2593f1e536
--- /dev/null
+++ b/arch/loongarch/kernel/env.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Author: Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/acpi.h>
+#include <linux/efi.h>
+#include <linux/export.h>
+#include <linux/memblock.h>
+#include <linux/of_fdt.h>
+#include <asm/early_ioremap.h>
+#include <asm/bootinfo.h>
+#include <asm/loongson.h>
+
+u64 efi_system_table;
+struct loongson_system_configuration loongson_sysconf;
+EXPORT_SYMBOL(loongson_sysconf);
+
+u64 loongson_chipcfg[MAX_PACKAGES];
+u64 loongson_chiptemp[MAX_PACKAGES];
+u64 loongson_freqctrl[MAX_PACKAGES];
+unsigned long long smp_group[MAX_PACKAGES];
+
+static void __init register_addrs_set(u64 *registers, const u64 addr, int num)
+{
+ u64 i;
+
+ for (i = 0; i < num; i++) {
+ *registers = (i << 44) | addr;
+ registers++;
+ }
+}
+
+void __init init_environ(void)
+{
+ int efi_boot = fw_arg0;
+ struct efi_memory_map_data data;
+ void *fdt_ptr = early_memremap_ro(fw_arg1, SZ_64K);
+
+ if (efi_boot)
+ set_bit(EFI_BOOT, &efi.flags);
+ else
+ clear_bit(EFI_BOOT, &efi.flags);
+
+ early_init_dt_scan(fdt_ptr);
+ early_init_fdt_reserve_self();
+ efi_system_table = efi_get_fdt_params(&data);
+
+ efi_memmap_init_early(&data);
+ memblock_reserve(data.phys_map & PAGE_MASK,
+ PAGE_ALIGN(data.size + (data.phys_map & ~PAGE_MASK)));
+
+ register_addrs_set(smp_group, TO_UNCACHE(0x1fe01000), 16);
+ register_addrs_set(loongson_chipcfg, TO_UNCACHE(0x1fe00180), 16);
+ register_addrs_set(loongson_chiptemp, TO_UNCACHE(0x1fe0019c), 16);
+ register_addrs_set(loongson_freqctrl, TO_UNCACHE(0x1fe001d0), 16);
+}
+
+static int __init init_cpu_fullname(void)
+{
+ int cpu;
+
+ if (loongson_sysconf.cpuname && !strncmp(loongson_sysconf.cpuname, "Loongson", 8)) {
+ for (cpu = 0; cpu < NR_CPUS; cpu++)
+ __cpu_full_name[cpu] = loongson_sysconf.cpuname;
+ }
+ return 0;
+}
+arch_initcall(init_cpu_fullname);
diff --git a/arch/loongarch/kernel/head.S b/arch/loongarch/kernel/head.S
new file mode 100644
index 000000000000..4cb79e9c70f5
--- /dev/null
+++ b/arch/loongarch/kernel/head.S
@@ -0,0 +1,99 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/init.h>
+#include <linux/threads.h>
+
+#include <asm/addrspace.h>
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/regdef.h>
+#include <asm/loongarch.h>
+#include <asm/stackframe.h>
+#include <generated/compile.h>
+#include <generated/utsrelease.h>
+
+#ifdef CONFIG_EFI_STUB
+
+#include "efi-header.S"
+
+ __HEAD
+
+_head:
+ .word MZ_MAGIC /* "MZ", MS-DOS header */
+ .org 0x38
+#ifdef CONFIG_32BIT
+ .ascii "LA32" /* Magic number for BootLoader */
+#else
+ .ascii "LA64" /* Magic number for BootLoader */
+#endif
+ .org 0x3c
+ .long pe_header - _head /* Offset to the PE header */
+
+pe_header:
+ __EFI_PE_HEADER
+
+SYM_DATA(kernel_asize, .long _end - _text);
+SYM_DATA(kernel_fsize, .long _edata - _text);
+SYM_DATA(kernel_offset, .long kernel_offset - _text);
+
+kernel_version:
+ .ascii UTS_RELEASE " (" LINUX_COMPILE_BY "@" LINUX_COMPILE_HOST ") " UTS_VERSION "\0"
+
+#endif
+
+ __REF
+
+SYM_ENTRY(_stext, SYM_L_GLOBAL, SYM_A_NONE)
+
+SYM_CODE_START(kernel_entry) # kernel entry point
+
+ /* Config direct window and set PG */
+ li.d t0, CSR_DMW0_INIT # UC, PLV0, 0x8000 xxxx xxxx xxxx
+ csrwr t0, LOONGARCH_CSR_DMWIN0
+ li.d t0, CSR_DMW1_INIT # CA, PLV0, 0x9000 xxxx xxxx xxxx
+ csrwr t0, LOONGARCH_CSR_DMWIN1
+ /* Enable PG */
+ li.w t0, 0xb0 # PLV=0, IE=0, PG=1
+ csrwr t0, LOONGARCH_CSR_CRMD
+ li.w t0, 0x04 # PLV=0, PIE=1, PWE=0
+ csrwr t0, LOONGARCH_CSR_PRMD
+ li.w t0, 0x00 # FPE=0, SXE=0, ASXE=0, BTE=0
+ csrwr t0, LOONGARCH_CSR_EUEN
+
+ /* We might not get launched at the address the kernel is linked to,
+ so we jump there. */
+ la.abs t0, 0f
+ jirl zero, t0, 0
+0:
+ la t0, __bss_start # clear .bss
+ st.d zero, t0, 0
+ la t1, __bss_stop - LONGSIZE
+1:
+ addi.d t0, t0, LONGSIZE
+ st.d zero, t0, 0
+ bne t0, t1, 1b
+
+ la t0, fw_arg0
+ st.d a0, t0, 0 # firmware arguments
+ la t0, fw_arg1
+ st.d a1, t0, 0
+
+ /* KSave3 used for percpu base, initialized as 0 */
+ csrwr zero, PERCPU_BASE_KS
+ /* GPR21 used for percpu base (runtime), initialized as 0 */
+ or u0, zero, zero
+
+ la tp, init_thread_union
+ /* Set the SP after an empty pt_regs. */
+ PTR_LI sp, (_THREAD_SIZE - 32 - PT_SIZE)
+ PTR_ADD sp, sp, tp
+ set_saved_sp sp, t0, t1
+ PTR_ADDI sp, sp, -4 * SZREG # init stack pointer
+
+ bl start_kernel
+
+SYM_CODE_END(kernel_entry)
+
+SYM_ENTRY(kernel_entry_end, SYM_L_GLOBAL, SYM_A_NONE)
diff --git a/arch/loongarch/kernel/image-vars.h b/arch/loongarch/kernel/image-vars.h
new file mode 100644
index 000000000000..104e9f0e97fe
--- /dev/null
+++ b/arch/loongarch/kernel/image-vars.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __LOONGARCH_KERNEL_IMAGE_VARS_H
+#define __LOONGARCH_KERNEL_IMAGE_VARS_H
+
+#ifdef CONFIG_EFI_STUB
+
+__efistub_memcmp = memcmp;
+__efistub_memchr = memchr;
+__efistub_memcpy = memcpy;
+__efistub_memmove = memmove;
+__efistub_memset = memset;
+__efistub_strcat = strcat;
+__efistub_strcmp = strcmp;
+__efistub_strlen = strlen;
+__efistub_strncat = strncat;
+__efistub_strnstr = strnstr;
+__efistub_strnlen = strnlen;
+__efistub_strrchr = strrchr;
+__efistub_kernel_entry = kernel_entry;
+__efistub_kernel_asize = kernel_asize;
+__efistub_kernel_fsize = kernel_fsize;
+__efistub_kernel_offset = kernel_offset;
+
+#endif
+
+#endif /* __LOONGARCH_KERNEL_IMAGE_VARS_H */
diff --git a/arch/loongarch/kernel/mem.c b/arch/loongarch/kernel/mem.c
new file mode 100644
index 000000000000..7423361b0ebc
--- /dev/null
+++ b/arch/loongarch/kernel/mem.c
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/efi.h>
+#include <linux/initrd.h>
+#include <linux/memblock.h>
+
+#include <asm/bootinfo.h>
+#include <asm/loongson.h>
+#include <asm/sections.h>
+
+void __init memblock_init(void)
+{
+ u32 mem_type;
+ u64 mem_start, mem_end, mem_size;
+ efi_memory_desc_t *md;
+
+ /* Parse memory information */
+ for_each_efi_memory_desc(md) {
+ mem_type = md->type;
+ mem_start = md->phys_addr;
+ mem_size = md->num_pages << EFI_PAGE_SHIFT;
+ mem_end = mem_start + mem_size;
+
+ switch (mem_type) {
+ case EFI_LOADER_CODE:
+ case EFI_LOADER_DATA:
+ case EFI_BOOT_SERVICES_CODE:
+ case EFI_BOOT_SERVICES_DATA:
+ case EFI_PERSISTENT_MEMORY:
+ case EFI_CONVENTIONAL_MEMORY:
+ memblock_add(mem_start, mem_size);
+ if (max_low_pfn < (mem_end >> PAGE_SHIFT))
+ max_low_pfn = mem_end >> PAGE_SHIFT;
+ break;
+ case EFI_PAL_CODE:
+ case EFI_UNUSABLE_MEMORY:
+ case EFI_ACPI_RECLAIM_MEMORY:
+ memblock_add(mem_start, mem_size);
+ fallthrough;
+ case EFI_RESERVED_TYPE:
+ case EFI_RUNTIME_SERVICES_CODE:
+ case EFI_RUNTIME_SERVICES_DATA:
+ case EFI_MEMORY_MAPPED_IO:
+ case EFI_MEMORY_MAPPED_IO_PORT_SPACE:
+ memblock_reserve(mem_start, mem_size);
+ break;
+ }
+ }
+
+ memblock_set_current_limit(PFN_PHYS(max_low_pfn));
+ memblock_set_node(0, PHYS_ADDR_MAX, &memblock.memory, 0);
+
+ /* Reserve the first 2MB */
+ memblock_reserve(PHYS_OFFSET, 0x200000);
+
+ /* Reserve the kernel text/data/bss */
+ memblock_reserve(__pa_symbol(&_text),
+ __pa_symbol(&_end) - __pa_symbol(&_text));
+
+ /* Reserve the initrd */
+ reserve_initrd_mem();
+}
diff --git a/arch/loongarch/kernel/reset.c b/arch/loongarch/kernel/reset.c
new file mode 100644
index 000000000000..ef484ce43c5c
--- /dev/null
+++ b/arch/loongarch/kernel/reset.c
@@ -0,0 +1,90 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/kernel.h>
+#include <linux/acpi.h>
+#include <linux/efi.h>
+#include <linux/export.h>
+#include <linux/pm.h>
+#include <linux/types.h>
+#include <linux/reboot.h>
+#include <linux/delay.h>
+#include <linux/console.h>
+
+#include <acpi/reboot.h>
+#include <asm/compiler.h>
+#include <asm/idle.h>
+#include <asm/loongarch.h>
+#include <asm/reboot.h>
+
+static void default_halt(void)
+{
+ local_irq_disable();
+ clear_csr_ecfg(ECFG0_IM);
+
+ pr_notice("\n\n** You can safely turn off the power now **\n\n");
+ console_flush_on_panic(CONSOLE_FLUSH_PENDING);
+
+ while (true) {
+ __arch_cpu_idle();
+ }
+}
+
+static void default_poweroff(void)
+{
+#ifdef CONFIG_EFI
+ efi.reset_system(EFI_RESET_SHUTDOWN, EFI_SUCCESS, 0, NULL);
+#endif
+ while (true) {
+ __arch_cpu_idle();
+ }
+}
+
+static void default_restart(void)
+{
+#ifdef CONFIG_EFI
+ if (efi_capsule_pending(NULL))
+ efi_reboot(REBOOT_WARM, NULL);
+ else
+ efi_reboot(REBOOT_COLD, NULL);
+#endif
+ if (!acpi_disabled)
+ acpi_reboot();
+
+ while (true) {
+ __arch_cpu_idle();
+ }
+}
+
+void (*pm_restart)(void);
+EXPORT_SYMBOL(pm_restart);
+
+void (*pm_power_off)(void);
+EXPORT_SYMBOL(pm_power_off);
+
+void machine_halt(void)
+{
+ default_halt();
+}
+
+void machine_power_off(void)
+{
+ pm_power_off();
+}
+
+void machine_restart(char *command)
+{
+ do_kernel_restart(command);
+ pm_restart();
+}
+
+static int __init loongarch_reboot_setup(void)
+{
+ pm_restart = default_restart;
+ pm_power_off = default_poweroff;
+
+ return 0;
+}
+
+arch_initcall(loongarch_reboot_setup);
diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c
new file mode 100644
index 000000000000..29f3b82cd0a5
--- /dev/null
+++ b/arch/loongarch/kernel/setup.c
@@ -0,0 +1,341 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1995 Linus Torvalds
+ * Copyright (C) 1995 Waldorf Electronics
+ * Copyright (C) 1994, 95, 96, 97, 98, 99, 2000, 01, 02, 03 Ralf Baechle
+ * Copyright (C) 1996 Stoned Elipot
+ * Copyright (C) 1999 Silicon Graphics, Inc.
+ * Copyright (C) 2000, 2001, 2002, 2007 Maciej W. Rozycki
+ */
+#include <linux/init.h>
+#include <linux/acpi.h>
+#include <linux/dmi.h>
+#include <linux/efi.h>
+#include <linux/export.h>
+#include <linux/screen_info.h>
+#include <linux/memblock.h>
+#include <linux/initrd.h>
+#include <linux/ioport.h>
+#include <linux/root_dev.h>
+#include <linux/console.h>
+#include <linux/pfn.h>
+#include <linux/platform_device.h>
+#include <linux/sizes.h>
+#include <linux/device.h>
+#include <linux/dma-map-ops.h>
+#include <linux/swiotlb.h>
+
+#include <asm/addrspace.h>
+#include <asm/bootinfo.h>
+#include <asm/cache.h>
+#include <asm/cpu.h>
+#include <asm/dma.h>
+#include <asm/efi.h>
+#include <asm/loongson.h>
+#include <asm/pgalloc.h>
+#include <asm/sections.h>
+#include <asm/setup.h>
+#include <asm/time.h>
+
+#define SMBIOS_BIOSSIZE_OFFSET 0x09
+#define SMBIOS_BIOSEXTERN_OFFSET 0x13
+#define SMBIOS_FREQLOW_OFFSET 0x16
+#define SMBIOS_FREQHIGH_OFFSET 0x17
+#define SMBIOS_FREQLOW_MASK 0xFF
+#define SMBIOS_CORE_PACKAGE_OFFSET 0x23
+#define LOONGSON_EFI_ENABLE (1 << 3)
+
+#ifdef CONFIG_VT
+struct screen_info screen_info;
+#endif
+
+unsigned long fw_arg0, fw_arg1;
+DEFINE_PER_CPU(unsigned long, kernelsp);
+struct cpuinfo_loongarch cpu_data[NR_CPUS] __read_mostly;
+
+EXPORT_SYMBOL(cpu_data);
+
+struct loongson_board_info b_info;
+static const char dmi_empty_string[] = " ";
+
+/*
+ * Setup information
+ *
+ * These are initialized so they are in the .data section
+ */
+
+static int num_standard_resources;
+static struct resource *standard_resources;
+
+static struct resource code_resource = { .name = "Kernel code", };
+static struct resource data_resource = { .name = "Kernel data", };
+static struct resource bss_resource = { .name = "Kernel bss", };
+
+const char *get_system_type(void)
+{
+ return "generic-loongson-machine";
+}
+
+static const char *dmi_string_parse(const struct dmi_header *dm, u8 s)
+{
+ const u8 *bp = ((u8 *) dm) + dm->length;
+
+ if (s) {
+ s--;
+ while (s > 0 && *bp) {
+ bp += strlen(bp) + 1;
+ s--;
+ }
+
+ if (*bp != 0) {
+ size_t len = strlen(bp)+1;
+ size_t cmp_len = len > 8 ? 8 : len;
+
+ if (!memcmp(bp, dmi_empty_string, cmp_len))
+ return dmi_empty_string;
+
+ return bp;
+ }
+ }
+
+ return "";
+}
+
+static void __init parse_cpu_table(const struct dmi_header *dm)
+{
+ long freq_temp = 0;
+ char *dmi_data = (char *)dm;
+
+ freq_temp = ((*(dmi_data + SMBIOS_FREQHIGH_OFFSET) << 8) +
+ ((*(dmi_data + SMBIOS_FREQLOW_OFFSET)) & SMBIOS_FREQLOW_MASK));
+ cpu_clock_freq = freq_temp * 1000000;
+
+ loongson_sysconf.cpuname = (void *)dmi_string_parse(dm, dmi_data[16]);
+ loongson_sysconf.cores_per_package = *(dmi_data + SMBIOS_CORE_PACKAGE_OFFSET);
+
+ pr_info("CpuClock = %llu\n", cpu_clock_freq);
+}
+
+static void __init parse_bios_table(const struct dmi_header *dm)
+{
+ int bios_extern;
+ char *dmi_data = (char *)dm;
+
+ bios_extern = *(dmi_data + SMBIOS_BIOSEXTERN_OFFSET);
+ b_info.bios_size = *(dmi_data + SMBIOS_BIOSSIZE_OFFSET);
+
+ if (bios_extern & LOONGSON_EFI_ENABLE)
+ set_bit(EFI_BOOT, &efi.flags);
+ else
+ clear_bit(EFI_BOOT, &efi.flags);
+}
+
+static void __init find_tokens(const struct dmi_header *dm, void *dummy)
+{
+ switch (dm->type) {
+ case 0x0: /* Extern BIOS */
+ parse_bios_table(dm);
+ break;
+ case 0x4: /* Calling interface */
+ parse_cpu_table(dm);
+ break;
+ }
+}
+static void __init smbios_parse(void)
+{
+ b_info.bios_vendor = (void *)dmi_get_system_info(DMI_BIOS_VENDOR);
+ b_info.bios_version = (void *)dmi_get_system_info(DMI_BIOS_VERSION);
+ b_info.bios_release_date = (void *)dmi_get_system_info(DMI_BIOS_DATE);
+ b_info.board_vendor = (void *)dmi_get_system_info(DMI_BOARD_VENDOR);
+ b_info.board_name = (void *)dmi_get_system_info(DMI_BOARD_NAME);
+ dmi_walk(find_tokens, NULL);
+}
+
+static int usermem __initdata;
+
+static int __init early_parse_mem(char *p)
+{
+ phys_addr_t start, size;
+
+ if (!p) {
+ pr_err("mem parameter is empty, do nothing\n");
+ return -EINVAL;
+ }
+
+ /*
+ * If a user specifies memory size, we
+ * blow away any automatically generated
+ * size.
+ */
+ if (usermem == 0) {
+ usermem = 1;
+ memblock_remove(memblock_start_of_DRAM(),
+ memblock_end_of_DRAM() - memblock_start_of_DRAM());
+ }
+ start = 0;
+ size = memparse(p, &p);
+ if (*p == '@')
+ start = memparse(p + 1, &p);
+ else {
+ pr_err("Invalid format!\n");
+ return -EINVAL;
+ }
+
+ memblock_add(start, size);
+
+ return 0;
+}
+early_param("mem", early_parse_mem);
+
+void __init platform_init(void)
+{
+ efi_init();
+#ifdef CONFIG_ACPI_TABLE_UPGRADE
+ acpi_table_upgrade();
+#endif
+#ifdef CONFIG_ACPI
+ acpi_gbl_use_default_register_widths = false;
+ acpi_boot_table_init();
+ acpi_boot_init();
+#endif
+
+ dmi_setup();
+ smbios_parse();
+ pr_info("The BIOS Version: %s\n", b_info.bios_version);
+
+ efi_runtime_init();
+}
+
+static void __init check_kernel_sections_mem(void)
+{
+ phys_addr_t start = __pa_symbol(&_text);
+ phys_addr_t size = __pa_symbol(&_end) - start;
+
+ if (!memblock_is_region_memory(start, size)) {
+ pr_info("Kernel sections are not in the memory maps\n");
+ memblock_add(start, size);
+ }
+}
+
+/*
+ * arch_mem_init - initialize memory management subsystem
+ */
+static void __init arch_mem_init(char **cmdline_p)
+{
+ if (usermem)
+ pr_info("User-defined physical RAM map overwrite\n");
+
+ check_kernel_sections_mem();
+
+ /*
+ * In order to reduce the possibility of kernel panic when failed to
+ * get IO TLB memory under CONFIG_SWIOTLB, it is better to allocate
+ * low memory as small as possible before plat_swiotlb_setup(), so
+ * make sparse_init() using top-down allocation.
+ */
+ memblock_set_bottom_up(false);
+ sparse_init();
+ memblock_set_bottom_up(true);
+
+ swiotlb_init(true, SWIOTLB_VERBOSE);
+
+ dma_contiguous_reserve(PFN_PHYS(max_low_pfn));
+
+ memblock_dump_all();
+
+ early_memtest(PFN_PHYS(ARCH_PFN_OFFSET), PFN_PHYS(max_low_pfn));
+}
+
+static void __init resource_init(void)
+{
+ long i = 0;
+ size_t res_size;
+ struct resource *res;
+ struct memblock_region *region;
+
+ code_resource.start = __pa_symbol(&_text);
+ code_resource.end = __pa_symbol(&_etext) - 1;
+ data_resource.start = __pa_symbol(&_etext);
+ data_resource.end = __pa_symbol(&_edata) - 1;
+ bss_resource.start = __pa_symbol(&__bss_start);
+ bss_resource.end = __pa_symbol(&__bss_stop) - 1;
+
+ num_standard_resources = memblock.memory.cnt;
+ res_size = num_standard_resources * sizeof(*standard_resources);
+ standard_resources = memblock_alloc(res_size, SMP_CACHE_BYTES);
+
+ for_each_mem_region(region) {
+ res = &standard_resources[i++];
+ if (!memblock_is_nomap(region)) {
+ res->name = "System RAM";
+ res->flags = IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY;
+ res->start = __pfn_to_phys(memblock_region_memory_base_pfn(region));
+ res->end = __pfn_to_phys(memblock_region_memory_end_pfn(region)) - 1;
+ } else {
+ res->name = "Reserved";
+ res->flags = IORESOURCE_MEM;
+ res->start = __pfn_to_phys(memblock_region_reserved_base_pfn(region));
+ res->end = __pfn_to_phys(memblock_region_reserved_end_pfn(region)) - 1;
+ }
+
+ request_resource(&iomem_resource, res);
+
+ /*
+ * We don't know which RAM region contains kernel data,
+ * so we try it repeatedly and let the resource manager
+ * test it.
+ */
+ request_resource(res, &code_resource);
+ request_resource(res, &data_resource);
+ request_resource(res, &bss_resource);
+ }
+}
+
+static int __init reserve_memblock_reserved_regions(void)
+{
+ u64 i, j;
+
+ for (i = 0; i < num_standard_resources; ++i) {
+ struct resource *mem = &standard_resources[i];
+ phys_addr_t r_start, r_end, mem_size = resource_size(mem);
+
+ if (!memblock_is_region_reserved(mem->start, mem_size))
+ continue;
+
+ for_each_reserved_mem_range(j, &r_start, &r_end) {
+ resource_size_t start, end;
+
+ start = max(PFN_PHYS(PFN_DOWN(r_start)), mem->start);
+ end = min(PFN_PHYS(PFN_UP(r_end)) - 1, mem->end);
+
+ if (start > mem->end || end < mem->start)
+ continue;
+
+ reserve_region_with_split(mem, start, end, "Reserved");
+ }
+ }
+
+ return 0;
+}
+arch_initcall(reserve_memblock_reserved_regions);
+
+void __init setup_arch(char **cmdline_p)
+{
+ cpu_probe();
+ *cmdline_p = boot_command_line;
+
+ init_environ();
+ memblock_init();
+ parse_early_param();
+
+ platform_init();
+ pagetable_init();
+ arch_mem_init(cmdline_p);
+
+ resource_init();
+
+ paging_init();
+}
diff --git a/arch/loongarch/kernel/time.c b/arch/loongarch/kernel/time.c
new file mode 100644
index 000000000000..b2bb14844f01
--- /dev/null
+++ b/arch/loongarch/kernel/time.c
@@ -0,0 +1,213 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Common time service routines for LoongArch machines.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/clockchips.h>
+#include <linux/delay.h>
+#include <linux/export.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/sched_clock.h>
+#include <linux/spinlock.h>
+
+#include <asm/cpu-features.h>
+#include <asm/loongarch.h>
+#include <asm/time.h>
+
+u64 cpu_clock_freq;
+EXPORT_SYMBOL(cpu_clock_freq);
+u64 const_clock_freq;
+EXPORT_SYMBOL(const_clock_freq);
+
+static DEFINE_RAW_SPINLOCK(state_lock);
+static DEFINE_PER_CPU(struct clock_event_device, constant_clockevent_device);
+
+static void constant_event_handler(struct clock_event_device *dev)
+{
+}
+
+irqreturn_t constant_timer_interrupt(int irq, void *data)
+{
+ int cpu = smp_processor_id();
+ struct clock_event_device *cd;
+
+ /* Clear Timer Interrupt */
+ write_csr_tintclear(CSR_TINTCLR_TI);
+ cd = &per_cpu(constant_clockevent_device, cpu);
+ cd->event_handler(cd);
+
+ return IRQ_HANDLED;
+}
+
+static int constant_set_state_oneshot(struct clock_event_device *evt)
+{
+ unsigned long timer_config;
+
+ raw_spin_lock(&state_lock);
+
+ timer_config = csr_read64(LOONGARCH_CSR_TCFG);
+ timer_config |= CSR_TCFG_EN;
+ timer_config &= ~CSR_TCFG_PERIOD;
+ csr_write64(timer_config, LOONGARCH_CSR_TCFG);
+
+ raw_spin_unlock(&state_lock);
+
+ return 0;
+}
+
+static int constant_set_state_oneshot_stopped(struct clock_event_device *evt)
+{
+ unsigned long timer_config;
+
+ raw_spin_lock(&state_lock);
+
+ timer_config = csr_read64(LOONGARCH_CSR_TCFG);
+ timer_config &= ~CSR_TCFG_EN;
+ csr_write64(timer_config, LOONGARCH_CSR_TCFG);
+
+ raw_spin_unlock(&state_lock);
+
+ return 0;
+}
+
+static int constant_set_state_periodic(struct clock_event_device *evt)
+{
+ unsigned long period;
+ unsigned long timer_config;
+
+ raw_spin_lock(&state_lock);
+
+ period = const_clock_freq / HZ;
+ timer_config = period & CSR_TCFG_VAL;
+ timer_config |= (CSR_TCFG_PERIOD | CSR_TCFG_EN);
+ csr_write64(timer_config, LOONGARCH_CSR_TCFG);
+
+ raw_spin_unlock(&state_lock);
+
+ return 0;
+}
+
+static int constant_set_state_shutdown(struct clock_event_device *evt)
+{
+ return 0;
+}
+
+static int constant_timer_next_event(unsigned long delta, struct clock_event_device *evt)
+{
+ unsigned long timer_config;
+
+ delta &= CSR_TCFG_VAL;
+ timer_config = delta | CSR_TCFG_EN;
+ csr_write64(timer_config, LOONGARCH_CSR_TCFG);
+
+ return 0;
+}
+
+static unsigned long __init get_loops_per_jiffy(void)
+{
+ unsigned long lpj = (unsigned long)const_clock_freq;
+
+ do_div(lpj, HZ);
+
+ return lpj;
+}
+
+static long init_timeval;
+
+void sync_counter(void)
+{
+ /* Ensure counter begin at 0 */
+ csr_write64(-init_timeval, LOONGARCH_CSR_CNTC);
+}
+
+int constant_clockevent_init(void)
+{
+ unsigned int irq;
+ unsigned int cpu = smp_processor_id();
+ unsigned long min_delta = 0x600;
+ unsigned long max_delta = (1UL << 48) - 1;
+ struct clock_event_device *cd;
+ static int timer_irq_installed = 0;
+
+ irq = EXCCODE_TIMER - EXCCODE_INT_START;
+
+ cd = &per_cpu(constant_clockevent_device, cpu);
+
+ cd->name = "Constant";
+ cd->features = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_PERCPU;
+
+ cd->irq = irq;
+ cd->rating = 320;
+ cd->cpumask = cpumask_of(cpu);
+ cd->set_state_oneshot = constant_set_state_oneshot;
+ cd->set_state_oneshot_stopped = constant_set_state_oneshot_stopped;
+ cd->set_state_periodic = constant_set_state_periodic;
+ cd->set_state_shutdown = constant_set_state_shutdown;
+ cd->set_next_event = constant_timer_next_event;
+ cd->event_handler = constant_event_handler;
+
+ clockevents_config_and_register(cd, const_clock_freq, min_delta, max_delta);
+
+ if (timer_irq_installed)
+ return 0;
+
+ timer_irq_installed = 1;
+
+ sync_counter();
+
+ if (request_irq(irq, constant_timer_interrupt, IRQF_PERCPU | IRQF_TIMER, "timer", NULL))
+ pr_err("Failed to request irq %d (timer)\n", irq);
+
+ lpj_fine = get_loops_per_jiffy();
+ pr_info("Constant clock event device register\n");
+
+ return 0;
+}
+
+static u64 read_const_counter(struct clocksource *clk)
+{
+ return drdtime();
+}
+
+static u64 native_sched_clock(void)
+{
+ return read_const_counter(NULL);
+}
+
+static struct clocksource clocksource_const = {
+ .name = "Constant",
+ .rating = 400,
+ .read = read_const_counter,
+ .mask = CLOCKSOURCE_MASK(64),
+ .flags = CLOCK_SOURCE_IS_CONTINUOUS,
+};
+
+int __init constant_clocksource_init(void)
+{
+ int res;
+ unsigned long freq = const_clock_freq;
+
+ res = clocksource_register_hz(&clocksource_const, freq);
+
+ sched_clock_register(native_sched_clock, 64, freq);
+
+ pr_info("Constant clock source device register\n");
+
+ return res;
+}
+
+void __init time_init(void)
+{
+ if (!cpu_has_cpucfg)
+ const_clock_freq = cpu_clock_freq;
+ else
+ const_clock_freq = calc_const_freq();
+
+ init_timeval = drdtime() - csr_read64(LOONGARCH_CSR_CNTC);
+
+ constant_clockevent_init();
+ constant_clocksource_init();
+}
diff --git a/arch/loongarch/kernel/topology.c b/arch/loongarch/kernel/topology.c
new file mode 100644
index 000000000000..3b2cbb95875b
--- /dev/null
+++ b/arch/loongarch/kernel/topology.c
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/cpu.h>
+#include <linux/init.h>
+#include <linux/percpu.h>
+
+static struct cpu cpu_device;
+
+static int __init topology_init(void)
+{
+ return register_cpu(&cpu_device, 0);
+}
+
+subsys_initcall(topology_init);
diff --git a/drivers/firmware/efi/Kconfig b/drivers/firmware/efi/Kconfig
index 4720ba98cec3..ba5d3d946f61 100644
--- a/drivers/firmware/efi/Kconfig
+++ b/drivers/firmware/efi/Kconfig
@@ -118,7 +118,7 @@ config EFI_GENERIC_STUB
config EFI_ARMSTUB_DTB_LOADER
bool "Enable the DTB loader"
- depends on EFI_GENERIC_STUB && !RISCV
+ depends on EFI_GENERIC_STUB && !RISCV && !LOONGARCH
default y
help
Select this config option to add support for the dtb= command
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index d0537573501e..1588c61939e7 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -26,6 +26,8 @@ cflags-$(CONFIG_ARM) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
$(call cc-option,-mno-single-pic-base)
cflags-$(CONFIG_RISCV) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
-fpic
+cflags-$(CONFIG_LOONGARCH) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
+ -fpic
cflags-$(CONFIG_EFI_GENERIC_STUB) += -I$(srctree)/scripts/dtc/libfdt
@@ -70,6 +72,8 @@ lib-$(CONFIG_ARM) += arm32-stub.o
lib-$(CONFIG_ARM64) += arm64-stub.o
lib-$(CONFIG_X86) += x86-stub.o
lib-$(CONFIG_RISCV) += riscv-stub.o
+lib-$(CONFIG_LOONGARCH) += loongarch-stub.o
+
CFLAGS_arm32-stub.o := -DTEXT_OFFSET=$(TEXT_OFFSET)
# Even when -mbranch-protection=none is set, Clang will generate a
@@ -125,6 +129,12 @@ STUBCOPY_FLAGS-$(CONFIG_RISCV) += --prefix-alloc-sections=.init \
--prefix-symbols=__efistub_
STUBCOPY_RELOC-$(CONFIG_RISCV) := R_RISCV_HI20
+# For LoongArch, keep all the symbols in .init section and make sure that no
+# absolute symbols references doesn't exist.
+STUBCOPY_FLAGS-$(CONFIG_LOONGARCH) += --prefix-alloc-sections=.init \
+ --prefix-symbols=__efistub_
+STUBCOPY_RELOC-$(CONFIG_LOONGARCH) := R_LARCH_MARK_LA
+
$(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,stubcopy)
diff --git a/drivers/firmware/efi/libstub/efi-stub-helper.c b/drivers/firmware/efi/libstub/efi-stub-helper.c
index 3d972061c1b0..f612cfceda22 100644
--- a/drivers/firmware/efi/libstub/efi-stub-helper.c
+++ b/drivers/firmware/efi/libstub/efi-stub-helper.c
@@ -21,7 +21,7 @@
bool efi_nochunk;
bool efi_nokaslr = !IS_ENABLED(CONFIG_RANDOMIZE_BASE);
int efi_loglevel = CONSOLE_LOGLEVEL_DEFAULT;
-bool efi_novamap;
+bool efi_novamap = IS_ENABLED(CONFIG_LOONGARCH); /* LoongArch call svam() in kernel */
static bool efi_noinitrd;
static bool efi_nosoftreserve;
diff --git a/drivers/firmware/efi/libstub/efi-stub.c b/drivers/firmware/efi/libstub/efi-stub.c
index f515394cce6e..f228fce398cc 100644
--- a/drivers/firmware/efi/libstub/efi-stub.c
+++ b/drivers/firmware/efi/libstub/efi-stub.c
@@ -40,7 +40,7 @@
#ifdef CONFIG_ARM64
# define EFI_RT_VIRTUAL_LIMIT DEFAULT_MAP_WINDOW_64
-#elif defined(CONFIG_RISCV)
+#elif defined(CONFIG_RISCV) || defined(CONFIG_LOONGARCH)
# define EFI_RT_VIRTUAL_LIMIT TASK_SIZE_MIN
#else
# define EFI_RT_VIRTUAL_LIMIT TASK_SIZE
diff --git a/drivers/firmware/efi/libstub/loongarch-stub.c b/drivers/firmware/efi/libstub/loongarch-stub.c
new file mode 100644
index 000000000000..beee086d9950
--- /dev/null
+++ b/drivers/firmware/efi/libstub/loongarch-stub.c
@@ -0,0 +1,88 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Author: Yun Liu <[email protected]>
+ * Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/efi.h>
+#include <asm/efi.h>
+#include <asm/addrspace.h>
+#include "efistub.h"
+
+typedef void __noreturn (*kernel_entry_t)(bool efi, unsigned long fdt);
+
+extern int kernel_asize;
+extern int kernel_fsize;
+extern int kernel_offset;
+extern kernel_entry_t kernel_entry;
+
+static efi_guid_t screen_info_guid = LINUX_EFI_LARCH_SCREEN_INFO_TABLE_GUID;
+
+struct screen_info *alloc_screen_info(void)
+{
+ efi_status_t status;
+ struct screen_info *si;
+
+ status = efi_bs_call(allocate_pool,
+ EFI_RUNTIME_SERVICES_DATA, sizeof(*si), (void **)&si);
+ if (status != EFI_SUCCESS)
+ return NULL;
+
+ status = efi_bs_call(install_configuration_table, &screen_info_guid, si);
+ if (status == EFI_SUCCESS)
+ return si;
+
+ efi_bs_call(free_pool, si);
+
+ return NULL;
+}
+
+void free_screen_info(struct screen_info *si)
+{
+ if (!si)
+ return;
+
+ efi_bs_call(install_configuration_table, &screen_info_guid, NULL);
+ efi_bs_call(free_pool, si);
+}
+
+efi_status_t check_platform_features(void)
+{
+ /* Config Direct Mapping */
+ csr_write64(CSR_DMW0_INIT, LOONGARCH_CSR_DMWIN0);
+ csr_write64(CSR_DMW1_INIT, LOONGARCH_CSR_DMWIN1);
+
+ return EFI_SUCCESS;
+}
+
+efi_status_t handle_kernel_image(unsigned long *image_addr,
+ unsigned long *image_size,
+ unsigned long *reserve_addr,
+ unsigned long *reserve_size,
+ efi_loaded_image_t *image,
+ efi_handle_t image_handle)
+{
+ efi_status_t status;
+ unsigned long kernel_addr = 0;
+
+ kernel_addr = (unsigned long)&kernel_offset - kernel_offset;
+
+ status = efi_relocate_kernel(&kernel_addr, kernel_fsize, kernel_asize,
+ PHYSADDR(VMLINUX_LOAD_ADDRESS), SZ_2M, 0x0);
+
+ *image_addr = kernel_addr;
+ *image_size = kernel_asize;
+
+ return status;
+}
+
+void __noreturn efi_enter_kernel(unsigned long entrypoint, unsigned long fdt, unsigned long fdt_size)
+{
+ kernel_entry_t real_kernel_entry;
+
+ real_kernel_entry = (kernel_entry_t)
+ ((unsigned long)&kernel_entry - entrypoint + VMLINUX_LOAD_ADDRESS);
+
+ real_kernel_entry(true, fdt);
+}
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 7d9b0bb47eb3..adc43641ef8c 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -401,6 +401,7 @@ void efi_native_runtime_setup(void);
* associated with ConOut
*/
#define LINUX_EFI_ARM_SCREEN_INFO_TABLE_GUID EFI_GUID(0xe03fc20a, 0x85dc, 0x406e, 0xb9, 0x0e, 0x4a, 0xb5, 0x02, 0x37, 0x1d, 0x95)
+#define LINUX_EFI_LARCH_SCREEN_INFO_TABLE_GUID EFI_GUID(0x07fd51a6, 0x9532, 0x926f, 0x51, 0xdc, 0x6a, 0x63, 0x60, 0x2f, 0x84, 0xb4)
#define LINUX_EFI_ARM_CPU_STATE_TABLE_GUID EFI_GUID(0xef79e4aa, 0x3c3d, 0x4989, 0xb9, 0x02, 0x07, 0xa9, 0x43, 0xe5, 0x50, 0xd2)
#define LINUX_EFI_LOADER_ENTRY_GUID EFI_GUID(0x4a67b082, 0x0a4c, 0x41cf, 0xb6, 0xc7, 0x44, 0x0b, 0x29, 0xbb, 0x8c, 0x4f)
#define LINUX_EFI_RANDOM_SEED_TABLE_GUID EFI_GUID(0x1ce1e5bc, 0x7ceb, 0x42f2, 0x81, 0xe5, 0x8a, 0xad, 0xf1, 0x80, 0xf5, 0x7b)
diff --git a/include/linux/pe.h b/include/linux/pe.h
index daf09ffffe38..1d3836ef9d92 100644
--- a/include/linux/pe.h
+++ b/include/linux/pe.h
@@ -65,6 +65,8 @@
#define IMAGE_FILE_MACHINE_SH5 0x01a8
#define IMAGE_FILE_MACHINE_THUMB 0x01c2
#define IMAGE_FILE_MACHINE_WCEMIPSV2 0x0169
+#define IMAGE_FILE_MACHINE_LOONGARCH32 0x6232
+#define IMAGE_FILE_MACHINE_LOONGARCH64 0x6264
/* flags */
#define IMAGE_FILE_RELOCS_STRIPPED 0x0001
--
2.27.0
LoongArch-based procesors have 4, 8 or 16 cores per package. This patch
adds multi-processor (SMP) support for LoongArch.
Reviewed-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/Kconfig | 40 +-
arch/loongarch/include/asm/atomic.h | 4 +
arch/loongarch/include/asm/barrier.h | 108 ++++
arch/loongarch/include/asm/cmpxchg.h | 1 +
arch/loongarch/include/asm/futex.h | 1 +
arch/loongarch/include/asm/hardirq.h | 2 +
arch/loongarch/include/asm/irq.h | 2 +
arch/loongarch/include/asm/percpu.h | 194 +++++++
arch/loongarch/include/asm/pgtable.h | 21 +
arch/loongarch/include/asm/smp.h | 124 ++++
arch/loongarch/include/asm/stackframe.h | 17 +-
arch/loongarch/include/asm/tlbflush.h | 13 +
arch/loongarch/include/asm/topology.h | 7 +-
arch/loongarch/kernel/Makefile | 2 +
arch/loongarch/kernel/acpi.c | 69 +++
arch/loongarch/kernel/asm-offsets.c | 10 +
arch/loongarch/kernel/head.S | 30 +
arch/loongarch/kernel/irq.c | 13 +-
arch/loongarch/kernel/proc.c | 5 +
arch/loongarch/kernel/process.c | 7 +
arch/loongarch/kernel/reset.c | 12 +
arch/loongarch/kernel/setup.c | 26 +
arch/loongarch/kernel/smp.c | 735 ++++++++++++++++++++++++
arch/loongarch/kernel/topology.c | 43 +-
arch/loongarch/kernel/vmlinux.lds.S | 4 +
arch/loongarch/mm/tlbex.S | 69 +++
include/linux/cpuhotplug.h | 1 +
27 files changed, 1550 insertions(+), 10 deletions(-)
create mode 100644 arch/loongarch/include/asm/smp.h
create mode 100644 arch/loongarch/kernel/smp.c
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index d6ac80cf3922..b252a51946e3 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -64,6 +64,7 @@ config LOONGARCH
select GENERIC_LIB_UCMPDI2
select GENERIC_PCI_IOMAP
select GENERIC_SCHED_CLOCK
+ select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
select GPIOLIB
select HAVE_ARCH_AUDITSYSCALL
@@ -92,7 +93,7 @@ config LOONGARCH
select HAVE_RSEQ
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_TIF_NOHZ
- select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HAVE_VIRT_CPU_ACCOUNTING_GEN if !SMP
select IRQ_FORCED_THREADING
select IRQ_LOONGARCH_CPU
select MODULES_USE_ELF_RELA if MODULES
@@ -306,6 +307,43 @@ config EFI_STUB
This kernel feature allows the kernel to be loaded directly by
EFI firmware without the use of a bootloader.
+config SMP
+ bool "Multi-Processing support"
+ help
+ This enables support for systems with more than one CPU. If you have
+ a system with only one CPU, say N. If you have a system with more
+ than one CPU, say Y.
+
+ If you say N here, the kernel will run on uni- and multiprocessor
+ machines, but will use only one CPU of a multiprocessor machine. If
+ you say Y here, the kernel will run on many, but not all,
+ uniprocessor machines. On a uniprocessor machine, the kernel
+ will run faster if you say N here.
+
+ See also the SMP-HOWTO available at <http://www.tldp.org/docs.html#howto>.
+
+ If you don't know what to do here, say N.
+
+config HOTPLUG_CPU
+ bool "Support for hot-pluggable CPUs"
+ depends on SMP
+ select GENERIC_IRQ_MIGRATION
+ help
+ Say Y here to allow turning CPUs off and on. CPUs can be
+ controlled through /sys/devices/system/cpu.
+ (Note: power management support will enable this option
+ automatically on SMP systems. )
+ Say N if you want to disable CPU hotplug.
+
+config NR_CPUS
+ int "Maximum number of CPUs (2-256)"
+ range 2 256
+ depends on SMP
+ default "64"
+ help
+ This allows you to specify the maximum number of CPUs which this
+ kernel will support.
+
config FORCE_MAX_ZONEORDER
int "Maximum zone order"
range 14 64 if PAGE_SIZE_64KB
diff --git a/arch/loongarch/include/asm/atomic.h b/arch/loongarch/include/asm/atomic.h
index 932352342b12..979367ad4e2c 100644
--- a/arch/loongarch/include/asm/atomic.h
+++ b/arch/loongarch/include/asm/atomic.h
@@ -162,6 +162,7 @@ static inline int arch_atomic_sub_if_positive(int i, atomic_t *v)
" sc.w %1, %2 \n"
" beq $zero, %1, 1b \n"
"2: \n"
+ __WEAK_LLSC_MB
: "=&r" (result), "=&r" (temp),
"+" GCC_OFF_SMALL_ASM() (v->counter)
: "I" (-i));
@@ -174,6 +175,7 @@ static inline int arch_atomic_sub_if_positive(int i, atomic_t *v)
" sc.w %1, %2 \n"
" beq $zero, %1, 1b \n"
"2: \n"
+ __WEAK_LLSC_MB
: "=&r" (result), "=&r" (temp),
"+" GCC_OFF_SMALL_ASM() (v->counter)
: "r" (i));
@@ -323,6 +325,7 @@ static inline long arch_atomic64_sub_if_positive(long i, atomic64_t *v)
" sc.d %1, %2 \n"
" beq %1, $zero, 1b \n"
"2: \n"
+ __WEAK_LLSC_MB
: "=&r" (result), "=&r" (temp),
"+" GCC_OFF_SMALL_ASM() (v->counter)
: "I" (-i));
@@ -335,6 +338,7 @@ static inline long arch_atomic64_sub_if_positive(long i, atomic64_t *v)
" sc.d %1, %2 \n"
" beq %1, $zero, 1b \n"
"2: \n"
+ __WEAK_LLSC_MB
: "=&r" (result), "=&r" (temp),
"+" GCC_OFF_SMALL_ASM() (v->counter)
: "r" (i));
diff --git a/arch/loongarch/include/asm/barrier.h b/arch/loongarch/include/asm/barrier.h
index e57571bcaf4f..b6517eeeb141 100644
--- a/arch/loongarch/include/asm/barrier.h
+++ b/arch/loongarch/include/asm/barrier.h
@@ -18,6 +18,19 @@
#define mb() fast_mb()
#define iob() fast_iob()
+#define __smp_mb() __asm__ __volatile__("dbar 0" : : : "memory")
+#define __smp_rmb() __asm__ __volatile__("dbar 0" : : : "memory")
+#define __smp_wmb() __asm__ __volatile__("dbar 0" : : : "memory")
+
+#ifdef CONFIG_SMP
+#define __WEAK_LLSC_MB " dbar 0 \n"
+#else
+#define __WEAK_LLSC_MB " \n"
+#endif
+
+#define __smp_mb__before_atomic() barrier()
+#define __smp_mb__after_atomic() barrier()
+
/**
* array_index_mask_nospec() - generate a ~0 mask when index < size, 0 otherwise
* @index: array element index
@@ -46,6 +59,101 @@ static inline unsigned long array_index_mask_nospec(unsigned long index,
return mask;
}
+#define __smp_load_acquire(p) \
+({ \
+ union { typeof(*p) __val; char __c[1]; } __u; \
+ unsigned long __tmp = 0; \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 1: \
+ *(__u8 *)__u.__c = *(volatile __u8 *)p; \
+ __smp_mb(); \
+ break; \
+ case 2: \
+ *(__u16 *)__u.__c = *(volatile __u16 *)p; \
+ __smp_mb(); \
+ break; \
+ case 4: \
+ __asm__ __volatile__( \
+ "amor_db.w %[val], %[tmp], %[mem] \n" \
+ : [val] "=&r" (*(__u32 *)__u.__c) \
+ : [mem] "ZB" (*(u32 *) p), [tmp] "r" (__tmp) \
+ : "memory"); \
+ break; \
+ case 8: \
+ __asm__ __volatile__( \
+ "amor_db.d %[val], %[tmp], %[mem] \n" \
+ : [val] "=&r" (*(__u64 *)__u.__c) \
+ : [mem] "ZB" (*(u64 *) p), [tmp] "r" (__tmp) \
+ : "memory"); \
+ break; \
+ } \
+ (typeof(*p))__u.__val; \
+})
+
+#define __smp_store_release(p, v) \
+do { \
+ union { typeof(*p) __val; char __c[1]; } __u = \
+ { .__val = (__force typeof(*p)) (v) }; \
+ unsigned long __tmp; \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 1: \
+ __smp_mb(); \
+ *(volatile __u8 *)p = *(__u8 *)__u.__c; \
+ break; \
+ case 2: \
+ __smp_mb(); \
+ *(volatile __u16 *)p = *(__u16 *)__u.__c; \
+ break; \
+ case 4: \
+ __asm__ __volatile__( \
+ "amswap_db.w %[tmp], %[val], %[mem] \n" \
+ : [mem] "+ZB" (*(u32 *)p), [tmp] "=&r" (__tmp) \
+ : [val] "r" (*(__u32 *)__u.__c) \
+ : ); \
+ break; \
+ case 8: \
+ __asm__ __volatile__( \
+ "amswap_db.d %[tmp], %[val], %[mem] \n" \
+ : [mem] "+ZB" (*(u64 *)p), [tmp] "=&r" (__tmp) \
+ : [val] "r" (*(__u64 *)__u.__c) \
+ : ); \
+ break; \
+ } \
+} while (0)
+
+#define __smp_store_mb(p, v) \
+do { \
+ union { typeof(p) __val; char __c[1]; } __u = \
+ { .__val = (__force typeof(p)) (v) }; \
+ unsigned long __tmp; \
+ switch (sizeof(p)) { \
+ case 1: \
+ *(volatile __u8 *)&p = *(__u8 *)__u.__c; \
+ __smp_mb(); \
+ break; \
+ case 2: \
+ *(volatile __u16 *)&p = *(__u16 *)__u.__c; \
+ __smp_mb(); \
+ break; \
+ case 4: \
+ __asm__ __volatile__( \
+ "amswap_db.w %[tmp], %[val], %[mem] \n" \
+ : [mem] "+ZB" (*(u32 *)&p), [tmp] "=&r" (__tmp) \
+ : [val] "r" (*(__u32 *)__u.__c) \
+ : ); \
+ break; \
+ case 8: \
+ __asm__ __volatile__( \
+ "amswap_db.d %[tmp], %[val], %[mem] \n" \
+ : [mem] "+ZB" (*(u64 *)&p), [tmp] "=&r" (__tmp) \
+ : [val] "r" (*(__u64 *)__u.__c) \
+ : ); \
+ break; \
+ } \
+} while (0)
+
#include <asm-generic/barrier.h>
#endif /* __ASM_BARRIER_H */
diff --git a/arch/loongarch/include/asm/cmpxchg.h b/arch/loongarch/include/asm/cmpxchg.h
index 48613b872bc8..75b3a4478652 100644
--- a/arch/loongarch/include/asm/cmpxchg.h
+++ b/arch/loongarch/include/asm/cmpxchg.h
@@ -59,6 +59,7 @@ static inline unsigned long __xchg(volatile void *ptr, unsigned long x,
" " st " $t0, %1 \n" \
" beq $zero, $t0, 1b \n" \
"2: \n" \
+ __WEAK_LLSC_MB \
: "=&r" (__ret), "=ZB"(*m) \
: "ZB"(*m), "Jr" (old), "Jr" (new) \
: "t0", "memory"); \
diff --git a/arch/loongarch/include/asm/futex.h b/arch/loongarch/include/asm/futex.h
index b27d55f92db7..9de8231694ec 100644
--- a/arch/loongarch/include/asm/futex.h
+++ b/arch/loongarch/include/asm/futex.h
@@ -86,6 +86,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, u32 oldval, u32 newv
"2: sc.w $t0, %2 \n"
" beq $zero, $t0, 1b \n"
"3: \n"
+ __WEAK_LLSC_MB
" .section .fixup,\"ax\" \n"
"4: li.d %0, %6 \n"
" b 3b \n"
diff --git a/arch/loongarch/include/asm/hardirq.h b/arch/loongarch/include/asm/hardirq.h
index d32f83938880..befe8184aa08 100644
--- a/arch/loongarch/include/asm/hardirq.h
+++ b/arch/loongarch/include/asm/hardirq.h
@@ -21,4 +21,6 @@ typedef struct {
DECLARE_PER_CPU_ALIGNED(irq_cpustat_t, irq_stat);
+#define __ARCH_IRQ_STAT
+
#endif /* _ASM_HARDIRQ_H */
diff --git a/arch/loongarch/include/asm/irq.h b/arch/loongarch/include/asm/irq.h
index f3f1baf7027c..ace3ea6da72e 100644
--- a/arch/loongarch/include/asm/irq.h
+++ b/arch/loongarch/include/asm/irq.h
@@ -125,6 +125,8 @@ extern struct irq_domain *pch_lpc_domain;
extern struct irq_domain *pch_msi_domain[MAX_IO_PICS];
extern struct irq_domain *pch_pic_domain[MAX_IO_PICS];
+extern irqreturn_t loongson3_ipi_interrupt(int irq, void *dev);
+
#include <asm-generic/irq.h>
#endif /* _ASM_IRQ_H */
diff --git a/arch/loongarch/include/asm/percpu.h b/arch/loongarch/include/asm/percpu.h
index b03d8f8b9fd3..34f15a6fb1e7 100644
--- a/arch/loongarch/include/asm/percpu.h
+++ b/arch/loongarch/include/asm/percpu.h
@@ -5,6 +5,8 @@
#ifndef __ASM_PERCPU_H
#define __ASM_PERCPU_H
+#include <asm/cmpxchg.h>
+
/* Use r21 for fast access */
register unsigned long __my_cpu_offset __asm__("$r21");
@@ -15,6 +17,198 @@ static inline void set_my_cpu_offset(unsigned long off)
}
#define __my_cpu_offset __my_cpu_offset
+#define PERCPU_OP(op, asm_op, c_op) \
+static inline unsigned long __percpu_##op(void *ptr, \
+ unsigned long val, int size) \
+{ \
+ unsigned long ret; \
+ \
+ switch (size) { \
+ case 4: \
+ __asm__ __volatile__( \
+ "am"#asm_op".w" " %[ret], %[val], %[ptr] \n" \
+ : [ret] "=&r" (ret), [ptr] "+ZB"(*(u32 *)ptr) \
+ : [val] "r" (val)); \
+ break; \
+ case 8: \
+ __asm__ __volatile__( \
+ "am"#asm_op".d" " %[ret], %[val], %[ptr] \n" \
+ : [ret] "=&r" (ret), [ptr] "+ZB"(*(u64 *)ptr) \
+ : [val] "r" (val)); \
+ break; \
+ default: \
+ ret = 0; \
+ BUILD_BUG(); \
+ } \
+ \
+ return ret c_op val; \
+}
+
+PERCPU_OP(add, add, +)
+PERCPU_OP(and, and, &)
+PERCPU_OP(or, or, |)
+#undef PERCPU_OP
+
+static inline unsigned long __percpu_read(void *ptr, int size)
+{
+ unsigned long ret;
+
+ switch (size) {
+ case 1:
+ __asm__ __volatile__ ("ldx.b %[ret], $r21, %[ptr] \n"
+ : [ret] "=&r"(ret)
+ : [ptr] "r"(ptr)
+ : "memory");
+ break;
+ case 2:
+ __asm__ __volatile__ ("ldx.h %[ret], $r21, %[ptr] \n"
+ : [ret] "=&r"(ret)
+ : [ptr] "r"(ptr)
+ : "memory");
+ break;
+ case 4:
+ __asm__ __volatile__ ("ldx.w %[ret], $r21, %[ptr] \n"
+ : [ret] "=&r"(ret)
+ : [ptr] "r"(ptr)
+ : "memory");
+ break;
+ case 8:
+ __asm__ __volatile__ ("ldx.d %[ret], $r21, %[ptr] \n"
+ : [ret] "=&r"(ret)
+ : [ptr] "r"(ptr)
+ : "memory");
+ break;
+ default:
+ ret = 0;
+ BUILD_BUG();
+ }
+
+ return ret;
+}
+
+static inline void __percpu_write(void *ptr, unsigned long val, int size)
+{
+ switch (size) {
+ case 1:
+ __asm__ __volatile__("stx.b %[val], $r21, %[ptr] \n"
+ :
+ : [val] "r" (val), [ptr] "r" (ptr)
+ : "memory");
+ break;
+ case 2:
+ __asm__ __volatile__("stx.h %[val], $r21, %[ptr] \n"
+ :
+ : [val] "r" (val), [ptr] "r" (ptr)
+ : "memory");
+ break;
+ case 4:
+ __asm__ __volatile__("stx.w %[val], $r21, %[ptr] \n"
+ :
+ : [val] "r" (val), [ptr] "r" (ptr)
+ : "memory");
+ break;
+ case 8:
+ __asm__ __volatile__("stx.d %[val], $r21, %[ptr] \n"
+ :
+ : [val] "r" (val), [ptr] "r" (ptr)
+ : "memory");
+ break;
+ default:
+ BUILD_BUG();
+ }
+}
+
+static inline unsigned long __percpu_xchg(void *ptr, unsigned long val,
+ int size)
+{
+ switch (size) {
+ case 4:
+ return __xchg_asm("amswap.w", (volatile u32 *)ptr, (u32)val);
+
+ case 8:
+ return __xchg_asm("amswap.d", (volatile u64 *)ptr, (u64)val);
+
+ default:
+ BUILD_BUG();
+ }
+
+ return 0;
+}
+
+/* this_cpu_cmpxchg */
+#define _protect_cmpxchg_local(pcp, o, n) \
+({ \
+ typeof(*raw_cpu_ptr(&(pcp))) __ret; \
+ preempt_disable_notrace(); \
+ __ret = cmpxchg_local(raw_cpu_ptr(&(pcp)), o, n); \
+ preempt_enable_notrace(); \
+ __ret; \
+})
+
+#define _percpu_read(pcp) \
+({ \
+ typeof(pcp) __retval; \
+ __retval = (typeof(pcp))__percpu_read(&(pcp), sizeof(pcp)); \
+ __retval; \
+})
+
+#define _percpu_write(pcp, val) \
+do { \
+ __percpu_write(&(pcp), (unsigned long)(val), sizeof(pcp)); \
+} while (0) \
+
+#define _pcp_protect(operation, pcp, val) \
+({ \
+ typeof(pcp) __retval; \
+ preempt_disable_notrace(); \
+ __retval = (typeof(pcp))operation(raw_cpu_ptr(&(pcp)), \
+ (val), sizeof(pcp)); \
+ preempt_enable_notrace(); \
+ __retval; \
+})
+
+#define _percpu_add(pcp, val) \
+ _pcp_protect(__percpu_add, pcp, val)
+
+#define _percpu_add_return(pcp, val) _percpu_add(pcp, val)
+
+#define _percpu_and(pcp, val) \
+ _pcp_protect(__percpu_and, pcp, val)
+
+#define _percpu_or(pcp, val) \
+ _pcp_protect(__percpu_or, pcp, val)
+
+#define _percpu_xchg(pcp, val) ((typeof(pcp)) \
+ _pcp_protect(__percpu_xchg, pcp, (unsigned long)(val)))
+
+#define this_cpu_add_4(pcp, val) _percpu_add(pcp, val)
+#define this_cpu_add_8(pcp, val) _percpu_add(pcp, val)
+
+#define this_cpu_add_return_4(pcp, val) _percpu_add_return(pcp, val)
+#define this_cpu_add_return_8(pcp, val) _percpu_add_return(pcp, val)
+
+#define this_cpu_and_4(pcp, val) _percpu_and(pcp, val)
+#define this_cpu_and_8(pcp, val) _percpu_and(pcp, val)
+
+#define this_cpu_or_4(pcp, val) _percpu_or(pcp, val)
+#define this_cpu_or_8(pcp, val) _percpu_or(pcp, val)
+
+#define this_cpu_read_1(pcp) _percpu_read(pcp)
+#define this_cpu_read_2(pcp) _percpu_read(pcp)
+#define this_cpu_read_4(pcp) _percpu_read(pcp)
+#define this_cpu_read_8(pcp) _percpu_read(pcp)
+
+#define this_cpu_write_1(pcp, val) _percpu_write(pcp, val)
+#define this_cpu_write_2(pcp, val) _percpu_write(pcp, val)
+#define this_cpu_write_4(pcp, val) _percpu_write(pcp, val)
+#define this_cpu_write_8(pcp, val) _percpu_write(pcp, val)
+
+#define this_cpu_xchg_4(pcp, val) _percpu_xchg(pcp, val)
+#define this_cpu_xchg_8(pcp, val) _percpu_xchg(pcp, val)
+
+#define this_cpu_cmpxchg_4(ptr, o, n) _protect_cmpxchg_local(ptr, o, n)
+#define this_cpu_cmpxchg_8(ptr, o, n) _protect_cmpxchg_local(ptr, o, n)
+
#include <asm-generic/percpu.h>
#endif /* __ASM_PERCPU_H */
diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h
index 8920dd8b297b..5e33987d0a13 100644
--- a/arch/loongarch/include/asm/pgtable.h
+++ b/arch/loongarch/include/asm/pgtable.h
@@ -279,8 +279,29 @@ static inline void set_pte(pte_t *ptep, pte_t pteval)
* Make sure the buddy is global too (if it's !none,
* it better already be global)
*/
+#ifdef CONFIG_SMP
+ /*
+ * For SMP, multiple CPUs can race, so we need to do
+ * this atomically.
+ */
+ unsigned long page_global = _PAGE_GLOBAL;
+ unsigned long tmp;
+
+ __asm__ __volatile__ (
+ "1:" __LL "%[tmp], %[buddy] \n"
+ " bnez %[tmp], 2f \n"
+ " or %[tmp], %[tmp], %[global] \n"
+ __SC "%[tmp], %[buddy] \n"
+ " beqz %[tmp], 1b \n"
+ " nop \n"
+ "2: \n"
+ __WEAK_LLSC_MB
+ : [buddy] "+m" (buddy->pte), [tmp] "=&r" (tmp)
+ : [global] "r" (page_global));
+#else /* !CONFIG_SMP */
if (pte_none(*buddy))
pte_val(*buddy) = pte_val(*buddy) | _PAGE_GLOBAL;
+#endif /* CONFIG_SMP */
}
}
diff --git a/arch/loongarch/include/asm/smp.h b/arch/loongarch/include/asm/smp.h
new file mode 100644
index 000000000000..551e1f37c705
--- /dev/null
+++ b/arch/loongarch/include/asm/smp.h
@@ -0,0 +1,124 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_SMP_H
+#define __ASM_SMP_H
+
+#include <linux/atomic.h>
+#include <linux/bitops.h>
+#include <linux/linkage.h>
+#include <linux/smp.h>
+#include <linux/threads.h>
+#include <linux/cpumask.h>
+
+void loongson3_smp_setup(void);
+void loongson3_prepare_cpus(unsigned int max_cpus);
+void loongson3_boot_secondary(int cpu, struct task_struct *idle);
+void loongson3_init_secondary(void);
+void loongson3_smp_finish(void);
+void loongson3_send_ipi_single(int cpu, unsigned int action);
+void loongson3_send_ipi_mask(const struct cpumask *mask, unsigned int action);
+#ifdef CONFIG_HOTPLUG_CPU
+int loongson3_cpu_disable(void);
+void loongson3_cpu_die(unsigned int cpu);
+#endif
+
+#ifdef CONFIG_SMP
+
+static inline void plat_smp_setup(void)
+{
+ loongson3_smp_setup();
+}
+
+#else /* !CONFIG_SMP */
+
+static inline void plat_smp_setup(void) { }
+
+#endif /* !CONFIG_SMP */
+
+extern int smp_num_siblings;
+extern int num_processors;
+extern int disabled_cpus;
+extern cpumask_t cpu_sibling_map[];
+extern cpumask_t cpu_core_map[];
+extern cpumask_t cpu_foreign_map[];
+
+static inline int raw_smp_processor_id(void)
+{
+#if defined(__VDSO__)
+ extern int vdso_smp_processor_id(void)
+ __compiletime_error("VDSO should not call smp_processor_id()");
+ return vdso_smp_processor_id();
+#else
+ return current_thread_info()->cpu;
+#endif
+}
+#define raw_smp_processor_id raw_smp_processor_id
+
+/* Map from cpu id to sequential logical cpu number. This will only
+ * not be idempotent when cpus failed to come on-line. */
+extern int __cpu_number_map[NR_CPUS];
+#define cpu_number_map(cpu) __cpu_number_map[cpu]
+
+/* The reverse map from sequential logical cpu number to cpu id. */
+extern int __cpu_logical_map[NR_CPUS];
+#define cpu_logical_map(cpu) __cpu_logical_map[cpu]
+
+#define cpu_physical_id(cpu) cpu_logical_map(cpu)
+
+#define SMP_BOOT_CPU 0x1
+#define SMP_RESCHEDULE 0x2
+#define SMP_CALL_FUNCTION 0x4
+
+struct secondary_data {
+ unsigned long stack;
+ unsigned long thread_info;
+};
+extern struct secondary_data cpuboot_data;
+
+extern asmlinkage void smpboot_entry(void);
+
+extern void calculate_cpu_foreign_map(void);
+
+/*
+ * Generate IPI list text
+ */
+extern void show_ipi_list(struct seq_file *p, int prec);
+
+/*
+ * This function sends a 'reschedule' IPI to another CPU.
+ * it goes straight through and wastes no time serializing
+ * anything. Worst case is that we lose a reschedule ...
+ */
+static inline void smp_send_reschedule(int cpu)
+{
+ loongson3_send_ipi_single(cpu, SMP_RESCHEDULE);
+}
+
+static inline void arch_send_call_function_single_ipi(int cpu)
+{
+ loongson3_send_ipi_single(cpu, SMP_CALL_FUNCTION);
+}
+
+static inline void arch_send_call_function_ipi_mask(const struct cpumask *mask)
+{
+ loongson3_send_ipi_mask(mask, SMP_CALL_FUNCTION);
+}
+
+#ifdef CONFIG_HOTPLUG_CPU
+static inline int __cpu_disable(void)
+{
+ return loongson3_cpu_disable();
+}
+
+static inline void __cpu_die(unsigned int cpu)
+{
+ loongson3_cpu_die(cpu);
+}
+
+extern void play_dead(void);
+#endif
+
+#endif /* __ASM_SMP_H */
diff --git a/arch/loongarch/include/asm/stackframe.h b/arch/loongarch/include/asm/stackframe.h
index 44151b878d00..4ca953062b5b 100644
--- a/arch/loongarch/include/asm/stackframe.h
+++ b/arch/loongarch/include/asm/stackframe.h
@@ -77,17 +77,24 @@
* new value in sp.
*/
.macro get_saved_sp docfi=0
- la.abs t1, kernelsp
- move t0, sp
+ la.abs t1, kernelsp
+#ifdef CONFIG_SMP
+ csrrd t0, PERCPU_BASE_KS
+ LONG_ADD t1, t1, t0
+#endif
+ move t0, sp
.if \docfi
.cfi_register sp, t0
.endif
- LONG_L sp, t1, 0
+ LONG_L sp, t1, 0
.endm
.macro set_saved_sp stackp temp temp2
- la.abs \temp, kernelsp
- LONG_S \stackp, \temp, 0
+ la.abs \temp, kernelsp
+#ifdef CONFIG_SMP
+ LONG_ADD \temp, \temp, u0
+#endif
+ LONG_S \stackp, \temp, 0
.endm
.macro SAVE_SOME docfi=0
diff --git a/arch/loongarch/include/asm/tlbflush.h b/arch/loongarch/include/asm/tlbflush.h
index 36bd6d11dc2d..a0785e590681 100644
--- a/arch/loongarch/include/asm/tlbflush.h
+++ b/arch/loongarch/include/asm/tlbflush.h
@@ -25,6 +25,17 @@ extern void local_flush_tlb_kernel_range(unsigned long start, unsigned long end)
extern void local_flush_tlb_page(struct vm_area_struct *vma, unsigned long page);
extern void local_flush_tlb_one(unsigned long vaddr);
+#ifdef CONFIG_SMP
+
+extern void flush_tlb_all(void);
+extern void flush_tlb_mm(struct mm_struct *);
+extern void flush_tlb_range(struct vm_area_struct *vma, unsigned long, unsigned long);
+extern void flush_tlb_kernel_range(unsigned long, unsigned long);
+extern void flush_tlb_page(struct vm_area_struct *, unsigned long);
+extern void flush_tlb_one(unsigned long vaddr);
+
+#else /* CONFIG_SMP */
+
#define flush_tlb_all() local_flush_tlb_all()
#define flush_tlb_mm(mm) local_flush_tlb_mm(mm)
#define flush_tlb_range(vma, vmaddr, end) local_flush_tlb_range(vma, vmaddr, end)
@@ -32,4 +43,6 @@ extern void local_flush_tlb_one(unsigned long vaddr);
#define flush_tlb_page(vma, page) local_flush_tlb_page(vma, page)
#define flush_tlb_one(vaddr) local_flush_tlb_one(vaddr)
+#endif /* CONFIG_SMP */
+
#endif /* __ASM_TLBFLUSH_H */
diff --git a/arch/loongarch/include/asm/topology.h b/arch/loongarch/include/asm/topology.h
index 9ac71a25207a..da135841e5b1 100644
--- a/arch/loongarch/include/asm/topology.h
+++ b/arch/loongarch/include/asm/topology.h
@@ -7,7 +7,12 @@
#include <linux/smp.h>
-#define cpu_logical_map(cpu) 0
+#ifdef CONFIG_SMP
+#define topology_physical_package_id(cpu) (cpu_data[cpu].package)
+#define topology_core_id(cpu) (cpu_data[cpu].core)
+#define topology_core_cpumask(cpu) (&cpu_core_map[cpu])
+#define topology_sibling_cpumask(cpu) (&cpu_sibling_map[cpu])
+#endif
#include <asm-generic/topology.h>
diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
index e5a3b2fb9961..2cb6f698716a 100644
--- a/arch/loongarch/kernel/Makefile
+++ b/arch/loongarch/kernel/Makefile
@@ -18,4 +18,6 @@ obj-$(CONFIG_MODULES) += module.o module-sections.o
obj-$(CONFIG_PROC_FS) += proc.o
+obj-$(CONFIG_SMP) += smp.o
+
CPPFLAGS_vmlinux.lds := $(KBUILD_CFLAGS)
diff --git a/arch/loongarch/kernel/acpi.c b/arch/loongarch/kernel/acpi.c
index a644220bb426..181c59493b63 100644
--- a/arch/loongarch/kernel/acpi.c
+++ b/arch/loongarch/kernel/acpi.c
@@ -137,8 +137,44 @@ void __init acpi_boot_table_init(void)
}
}
+static int set_processor_mask(u32 id, u32 flags)
+{
+
+ int cpu, cpuid = id;
+
+ if (num_processors >= nr_cpu_ids) {
+ pr_warn(PREFIX "nr_cpus/possible_cpus limit of %i reached."
+ " processor 0x%x ignored.\n", nr_cpu_ids, cpuid);
+
+ return -ENODEV;
+
+ }
+ if (cpuid == loongson_sysconf.boot_cpu_id)
+ cpu = 0;
+ else
+ cpu = cpumask_next_zero(-1, cpu_present_mask);
+
+ if (flags & ACPI_MADT_ENABLED) {
+ num_processors++;
+ set_cpu_possible(cpu, true);
+ set_cpu_present(cpu, true);
+ __cpu_number_map[cpuid] = cpu;
+ __cpu_logical_map[cpu] = cpuid;
+ } else
+ disabled_cpus++;
+
+ return cpu;
+}
+
static void __init acpi_process_madt(void)
{
+ int i;
+
+ for (i = 0; i < NR_CPUS; i++) {
+ __cpu_number_map[i] = -1;
+ __cpu_logical_map[i] = -1;
+ }
+
loongson_sysconf.nr_cpus = num_processors;
}
@@ -167,3 +203,36 @@ void __init arch_reserve_mem_area(acpi_physical_address addr, size_t size)
{
memblock_reserve(addr, size);
}
+
+#ifdef CONFIG_ACPI_HOTPLUG_CPU
+
+#include <acpi/processor.h>
+
+int acpi_map_cpu(acpi_handle handle, phys_cpuid_t physid, u32 acpi_id, int *pcpu)
+{
+ int cpu;
+
+ cpu = set_processor_mask(physid, ACPI_MADT_ENABLED);
+ if (cpu < 0) {
+ pr_info(PREFIX "Unable to map lapic to logical cpu number\n");
+ return cpu;
+ }
+
+ *pcpu = cpu;
+
+ return 0;
+}
+EXPORT_SYMBOL(acpi_map_cpu);
+
+int acpi_unmap_cpu(int cpu)
+{
+ set_cpu_present(cpu, false);
+ num_processors--;
+
+ pr_info("cpu%d hot remove!\n", cpu);
+
+ return 0;
+}
+EXPORT_SYMBOL(acpi_unmap_cpu);
+
+#endif /* CONFIG_ACPI_HOTPLUG_CPU */
diff --git a/arch/loongarch/kernel/asm-offsets.c b/arch/loongarch/kernel/asm-offsets.c
index 3531e3c60a6e..bfb65eb2844f 100644
--- a/arch/loongarch/kernel/asm-offsets.c
+++ b/arch/loongarch/kernel/asm-offsets.c
@@ -252,3 +252,13 @@ void output_signal_defines(void)
DEFINE(_SIGXFSZ, SIGXFSZ);
BLANK();
}
+
+#ifdef CONFIG_SMP
+void output_smpboot_defines(void)
+{
+ COMMENT("Linux smp cpu boot offsets.");
+ OFFSET(CPU_BOOT_STACK, secondary_data, stack);
+ OFFSET(CPU_BOOT_TINFO, secondary_data, thread_info);
+ BLANK();
+}
+#endif
diff --git a/arch/loongarch/kernel/head.S b/arch/loongarch/kernel/head.S
index 4cb79e9c70f5..8ca45cf17433 100644
--- a/arch/loongarch/kernel/head.S
+++ b/arch/loongarch/kernel/head.S
@@ -96,4 +96,34 @@ SYM_CODE_START(kernel_entry) # kernel entry point
SYM_CODE_END(kernel_entry)
+#ifdef CONFIG_SMP
+
+/*
+ * SMP slave cpus entry point. Board specific code for bootstrap calls this
+ * function after setting up the stack and tp registers.
+ */
+SYM_CODE_START(smpboot_entry)
+ li.d t0, CSR_DMW0_INIT # UC, PLV0
+ csrwr t0, LOONGARCH_CSR_DMWIN0
+ li.d t0, CSR_DMW1_INIT # CA, PLV0
+ csrwr t0, LOONGARCH_CSR_DMWIN1
+ li.w t0, 0xb0 # PLV=0, IE=0, PG=1
+ csrwr t0, LOONGARCH_CSR_CRMD
+ li.w t0, 0x04 # PLV=0, PIE=1, PWE=0
+ csrwr t0, LOONGARCH_CSR_PRMD
+ li.w t0, 0x00 # FPE=0, SXE=0, ASXE=0, BTE=0
+ csrwr t0, LOONGARCH_CSR_EUEN
+
+ la.abs t0, cpuboot_data
+ ld.d sp, t0, CPU_BOOT_STACK
+ ld.d tp, t0, CPU_BOOT_TINFO
+
+ la.abs t0, 0f
+ jirl zero, t0, 0
+0:
+ bl start_secondary
+SYM_CODE_END(smpboot_entry)
+
+#endif /* CONFIG_SMP */
+
SYM_ENTRY(kernel_entry_end, SYM_L_GLOBAL, SYM_A_NONE)
diff --git a/arch/loongarch/kernel/irq.c b/arch/loongarch/kernel/irq.c
index 9bd07edf0bce..4b671d305ede 100644
--- a/arch/loongarch/kernel/irq.c
+++ b/arch/loongarch/kernel/irq.c
@@ -47,13 +47,17 @@ asmlinkage void spurious_interrupt(void)
int arch_show_interrupts(struct seq_file *p, int prec)
{
+#ifdef CONFIG_SMP
+ show_ipi_list(p, prec);
+#endif
seq_printf(p, "%*s: %10u\n", prec, "ERR", atomic_read(&irq_err_count));
return 0;
}
void __init init_IRQ(void)
{
- int i;
+ int i, r, ipi_irq;
+ static int ipi_dummy_dev;
unsigned int order = get_order(IRQ_STACK_SIZE);
struct page *page;
@@ -61,6 +65,13 @@ void __init init_IRQ(void)
clear_csr_estat(ESTATF_IP);
irqchip_init();
+#ifdef CONFIG_SMP
+ ipi_irq = EXCCODE_IPI - EXCCODE_INT_START;
+ irq_set_percpu_devid(ipi_irq);
+ r = request_percpu_irq(ipi_irq, loongson3_ipi_interrupt, "IPI", &ipi_dummy_dev);
+ if (r < 0)
+ panic("IPI IRQ request failed\n");
+#endif
for (i = 0; i < NR_IRQS; i++)
irq_set_noprobe(i);
diff --git a/arch/loongarch/kernel/proc.c b/arch/loongarch/kernel/proc.c
index d25592a29196..1effc73850fe 100644
--- a/arch/loongarch/kernel/proc.c
+++ b/arch/loongarch/kernel/proc.c
@@ -35,6 +35,11 @@ static int show_cpuinfo(struct seq_file *m, void *v)
unsigned int fp_version = cpu_data[n].fpu_vers;
struct proc_cpuinfo_notifier_args proc_cpuinfo_notifier_args;
+#ifdef CONFIG_SMP
+ if (!cpu_online(n))
+ return 0;
+#endif
+
/*
* For the first processor also print the system type
*/
diff --git a/arch/loongarch/kernel/process.c b/arch/loongarch/kernel/process.c
index 54020223068f..6d944d65f600 100644
--- a/arch/loongarch/kernel/process.c
+++ b/arch/loongarch/kernel/process.c
@@ -53,6 +53,13 @@
unsigned long boot_option_idle_override = IDLE_NO_OVERRIDE;
EXPORT_SYMBOL(boot_option_idle_override);
+#ifdef CONFIG_HOTPLUG_CPU
+void arch_cpu_idle_dead(void)
+{
+ play_dead();
+}
+#endif
+
asmlinkage void ret_from_fork(void);
asmlinkage void ret_from_kernel_thread(void);
diff --git a/arch/loongarch/kernel/reset.c b/arch/loongarch/kernel/reset.c
index ef484ce43c5c..2b86469e4718 100644
--- a/arch/loongarch/kernel/reset.c
+++ b/arch/loongarch/kernel/reset.c
@@ -65,16 +65,28 @@ EXPORT_SYMBOL(pm_power_off);
void machine_halt(void)
{
+#ifdef CONFIG_SMP
+ preempt_disable();
+ smp_send_stop();
+#endif
default_halt();
}
void machine_power_off(void)
{
+#ifdef CONFIG_SMP
+ preempt_disable();
+ smp_send_stop();
+#endif
pm_power_off();
}
void machine_restart(char *command)
{
+#ifdef CONFIG_SMP
+ preempt_disable();
+ smp_send_stop();
+#endif
do_kernel_restart(command);
pm_restart();
}
diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c
index 29f3b82cd0a5..34a3011f679e 100644
--- a/arch/loongarch/kernel/setup.c
+++ b/arch/loongarch/kernel/setup.c
@@ -38,6 +38,7 @@
#include <asm/pgalloc.h>
#include <asm/sections.h>
#include <asm/setup.h>
+#include <asm/smp.h>
#include <asm/time.h>
#define SMBIOS_BIOSSIZE_OFFSET 0x09
@@ -322,6 +323,29 @@ static int __init reserve_memblock_reserved_regions(void)
}
arch_initcall(reserve_memblock_reserved_regions);
+#ifdef CONFIG_SMP
+static void __init prefill_possible_map(void)
+{
+ int i, possible;
+
+ possible = num_processors + disabled_cpus;
+ if (possible > nr_cpu_ids)
+ possible = nr_cpu_ids;
+
+ pr_info("SMP: Allowing %d CPUs, %d hotplug CPUs\n",
+ possible, max((possible - num_processors), 0));
+
+ for (i = 0; i < possible; i++)
+ set_cpu_possible(i, true);
+ for (; i < NR_CPUS; i++)
+ set_cpu_possible(i, false);
+
+ nr_cpu_ids = possible;
+}
+#else
+static inline void prefill_possible_map(void) {}
+#endif
+
void __init setup_arch(char **cmdline_p)
{
cpu_probe();
@@ -336,6 +360,8 @@ void __init setup_arch(char **cmdline_p)
arch_mem_init(cmdline_p);
resource_init();
+ plat_smp_setup();
+ prefill_possible_map();
paging_init();
}
diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c
new file mode 100644
index 000000000000..99ba7a56edf9
--- /dev/null
+++ b/arch/loongarch/kernel/smp.c
@@ -0,0 +1,735 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 2000, 2001 Kanoj Sarcar
+ * Copyright (C) 2000, 2001 Ralf Baechle
+ * Copyright (C) 2000, 2001 Silicon Graphics, Inc.
+ * Copyright (C) 2000, 2001, 2003 Broadcom Corporation
+ */
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/seq_file.h>
+#include <linux/smp.h>
+#include <linux/threads.h>
+#include <linux/export.h>
+#include <linux/time.h>
+#include <linux/tracepoint.h>
+#include <linux/sched/hotplug.h>
+#include <linux/sched/task_stack.h>
+
+#include <asm/cpu.h>
+#include <asm/idle.h>
+#include <asm/loongson.h>
+#include <asm/mmu_context.h>
+#include <asm/processor.h>
+#include <asm/setup.h>
+#include <asm/time.h>
+
+int __cpu_number_map[NR_CPUS]; /* Map physical to logical */
+EXPORT_SYMBOL(__cpu_number_map);
+
+int __cpu_logical_map[NR_CPUS]; /* Map logical to physical */
+EXPORT_SYMBOL(__cpu_logical_map);
+
+/* Number of threads (siblings) per CPU core */
+int smp_num_siblings = 1;
+EXPORT_SYMBOL(smp_num_siblings);
+
+/* Representing the threads (siblings) of each logical CPU */
+cpumask_t cpu_sibling_map[NR_CPUS] __read_mostly;
+EXPORT_SYMBOL(cpu_sibling_map);
+
+/* Representing the core map of multi-core chips of each logical CPU */
+cpumask_t cpu_core_map[NR_CPUS] __read_mostly;
+EXPORT_SYMBOL(cpu_core_map);
+
+static DECLARE_COMPLETION(cpu_starting);
+static DECLARE_COMPLETION(cpu_running);
+
+/*
+ * A logcal cpu mask containing only one VPE per core to
+ * reduce the number of IPIs on large MT systems.
+ */
+cpumask_t cpu_foreign_map[NR_CPUS] __read_mostly;
+EXPORT_SYMBOL(cpu_foreign_map);
+
+/* representing cpus for which sibling maps can be computed */
+static cpumask_t cpu_sibling_setup_map;
+
+/* representing cpus for which core maps can be computed */
+static cpumask_t cpu_core_setup_map;
+
+struct secondary_data cpuboot_data;
+static DEFINE_PER_CPU(int, cpu_state);
+DEFINE_PER_CPU_SHARED_ALIGNED(irq_cpustat_t, irq_stat);
+EXPORT_PER_CPU_SYMBOL(irq_stat);
+
+enum ipi_msg_type {
+ IPI_RESCHEDULE,
+ IPI_CALL_FUNCTION,
+};
+
+static const char *ipi_types[NR_IPI] __tracepoint_string = {
+ [IPI_RESCHEDULE] = "Rescheduling interrupts",
+ [IPI_CALL_FUNCTION] = "Function call interrupts",
+};
+
+void show_ipi_list(struct seq_file *p, int prec)
+{
+ unsigned int cpu, i;
+
+ for (i = 0; i < NR_IPI; i++) {
+ seq_printf(p, "%*s%u:%s", prec - 1, "IPI", i, prec >= 4 ? " " : "");
+ for_each_online_cpu(cpu)
+ seq_printf(p, "%10u ", per_cpu(irq_stat, cpu).ipi_irqs[i]);
+ seq_printf(p, " LoongArch %d %s\n", i + 1, ipi_types[i]);
+ }
+}
+
+/* Send mailbox buffer via Mail_Send */
+static void csr_mail_send(uint64_t data, int cpu, int mailbox)
+{
+ uint64_t val;
+
+ /* Send high 32 bits */
+ val = IOCSR_MBUF_SEND_BLOCKING;
+ val |= (IOCSR_MBUF_SEND_BOX_HI(mailbox) << IOCSR_MBUF_SEND_BOX_SHIFT);
+ val |= (cpu << IOCSR_MBUF_SEND_CPU_SHIFT);
+ val |= (data & IOCSR_MBUF_SEND_H32_MASK);
+ iocsr_write64(val, LOONGARCH_IOCSR_MBUF_SEND);
+
+ /* Send low 32 bits */
+ val = IOCSR_MBUF_SEND_BLOCKING;
+ val |= (IOCSR_MBUF_SEND_BOX_LO(mailbox) << IOCSR_MBUF_SEND_BOX_SHIFT);
+ val |= (cpu << IOCSR_MBUF_SEND_CPU_SHIFT);
+ val |= (data << IOCSR_MBUF_SEND_BUF_SHIFT);
+ iocsr_write64(val, LOONGARCH_IOCSR_MBUF_SEND);
+};
+
+static u32 ipi_read_clear(int cpu)
+{
+ u32 action;
+
+ /* Load the ipi register to figure out what we're supposed to do */
+ action = iocsr_read32(LOONGARCH_IOCSR_IPI_STATUS);
+ /* Clear the ipi register to clear the interrupt */
+ iocsr_write32(action, LOONGARCH_IOCSR_IPI_CLEAR);
+ smp_mb();
+
+ return action;
+}
+
+static void ipi_write_action(int cpu, u32 action)
+{
+ unsigned int irq = 0;
+
+ while ((irq = ffs(action))) {
+ uint32_t val = IOCSR_IPI_SEND_BLOCKING;
+
+ val |= (irq - 1);
+ val |= (cpu << IOCSR_IPI_SEND_CPU_SHIFT);
+ iocsr_write32(val, LOONGARCH_IOCSR_IPI_SEND);
+ action &= ~BIT(irq - 1);
+ }
+}
+
+void loongson3_send_ipi_single(int cpu, unsigned int action)
+{
+ ipi_write_action(cpu_logical_map(cpu), (u32)action);
+}
+
+void loongson3_send_ipi_mask(const struct cpumask *mask, unsigned int action)
+{
+ unsigned int i;
+
+ for_each_cpu(i, mask)
+ ipi_write_action(cpu_logical_map(i), (u32)action);
+}
+
+irqreturn_t loongson3_ipi_interrupt(int irq, void *dev)
+{
+ unsigned int action;
+ unsigned int cpu = smp_processor_id();
+
+ action = ipi_read_clear(cpu_logical_map(cpu));
+
+ if (action & SMP_RESCHEDULE) {
+ scheduler_ipi();
+ per_cpu(irq_stat, cpu).ipi_irqs[IPI_RESCHEDULE]++;
+ }
+
+ if (action & SMP_CALL_FUNCTION) {
+ generic_smp_call_function_interrupt();
+ per_cpu(irq_stat, cpu).ipi_irqs[IPI_CALL_FUNCTION]++;
+ }
+
+ return IRQ_HANDLED;
+}
+
+void __init loongson3_smp_setup(void)
+{
+ cpu_data[0].core = cpu_logical_map(0) % loongson_sysconf.cores_per_package;
+ cpu_data[0].package = cpu_logical_map(0) / loongson_sysconf.cores_per_package;
+
+ iocsr_write32(0xffffffff, LOONGARCH_IOCSR_IPI_EN);
+ pr_info("Detected %i available CPU(s)\n", loongson_sysconf.nr_cpus);
+}
+
+void __init loongson3_prepare_cpus(unsigned int max_cpus)
+{
+ int i = 0;
+
+ for (i = 0; i < loongson_sysconf.nr_cpus; i++) {
+ set_cpu_present(i, true);
+ csr_mail_send(0, __cpu_logical_map[i], 0);
+ }
+
+ per_cpu(cpu_state, smp_processor_id()) = CPU_ONLINE;
+}
+
+/*
+ * Setup the PC, SP, and TP of a secondary processor and start it running!
+ */
+void loongson3_boot_secondary(int cpu, struct task_struct *idle)
+{
+ unsigned long entry;
+
+ pr_info("Booting CPU#%d...\n", cpu);
+
+ entry = __pa_symbol((unsigned long)&smpboot_entry);
+ cpuboot_data.stack = (unsigned long)__KSTK_TOS(idle);
+ cpuboot_data.thread_info = (unsigned long)task_thread_info(idle);
+
+ csr_mail_send(entry, cpu_logical_map(cpu), 0);
+
+ loongson3_send_ipi_single(cpu, SMP_BOOT_CPU);
+}
+
+/*
+ * SMP init and finish on secondary CPUs
+ */
+void loongson3_init_secondary(void)
+{
+ unsigned int cpu = smp_processor_id();
+ unsigned int imask = ECFGF_IP0 | ECFGF_IP1 | ECFGF_IP2 |
+ ECFGF_IPI | ECFGF_PMC | ECFGF_TIMER;
+
+ change_csr_ecfg(ECFG0_IM, imask);
+
+ iocsr_write32(0xffffffff, LOONGARCH_IOCSR_IPI_EN);
+
+ per_cpu(cpu_state, cpu) = CPU_ONLINE;
+ cpu_data[cpu].core =
+ cpu_logical_map(cpu) % loongson_sysconf.cores_per_package;
+ cpu_data[cpu].package =
+ cpu_logical_map(cpu) / loongson_sysconf.cores_per_package;
+}
+
+void loongson3_smp_finish(void)
+{
+ local_irq_enable();
+ iocsr_write64(0, LOONGARCH_IOCSR_MBUF0);
+ pr_info("CPU#%d finished\n", smp_processor_id());
+}
+
+#ifdef CONFIG_HOTPLUG_CPU
+
+static bool io_master(int cpu)
+{
+ if (cpu == 0)
+ return true;
+
+ return false;
+}
+
+int loongson3_cpu_disable(void)
+{
+ unsigned long flags;
+ unsigned int cpu = smp_processor_id();
+
+ if (io_master(cpu))
+ return -EBUSY;
+
+ set_cpu_online(cpu, false);
+ calculate_cpu_foreign_map();
+ local_irq_save(flags);
+ irq_migrate_all_off_this_cpu();
+ clear_csr_ecfg(ECFG0_IM);
+ local_irq_restore(flags);
+ local_flush_tlb_all();
+
+ return 0;
+}
+
+void loongson3_cpu_die(unsigned int cpu)
+{
+ while (per_cpu(cpu_state, cpu) != CPU_DEAD)
+ cpu_relax();
+
+ mb();
+}
+
+/*
+ * The target CPU should go to XKPRANGE (uncached area) and flush
+ * ICache/DCache/VCache before the control CPU can safely disable its clock.
+ */
+static void loongson3_play_dead(int *state_addr)
+{
+ register int val;
+ register void *addr;
+ register void (*init_fn)(void);
+
+ __asm__ __volatile__(
+ " li.d %[addr], 0x8000000000000000\n"
+ "1: cacop 0x8, %[addr], 0 \n" /* flush ICache */
+ " cacop 0x8, %[addr], 1 \n"
+ " cacop 0x8, %[addr], 2 \n"
+ " cacop 0x8, %[addr], 3 \n"
+ " cacop 0x9, %[addr], 0 \n" /* flush DCache */
+ " cacop 0x9, %[addr], 1 \n"
+ " cacop 0x9, %[addr], 2 \n"
+ " cacop 0x9, %[addr], 3 \n"
+ " addi.w %[sets], %[sets], -1 \n"
+ " addi.d %[addr], %[addr], 0x40 \n"
+ " bnez %[sets], 1b \n"
+ " li.d %[addr], 0x8000000000000000\n"
+ "2: cacop 0xa, %[addr], 0 \n" /* flush VCache */
+ " cacop 0xa, %[addr], 1 \n"
+ " cacop 0xa, %[addr], 2 \n"
+ " cacop 0xa, %[addr], 3 \n"
+ " cacop 0xa, %[addr], 4 \n"
+ " cacop 0xa, %[addr], 5 \n"
+ " cacop 0xa, %[addr], 6 \n"
+ " cacop 0xa, %[addr], 7 \n"
+ " cacop 0xa, %[addr], 8 \n"
+ " cacop 0xa, %[addr], 9 \n"
+ " cacop 0xa, %[addr], 10 \n"
+ " cacop 0xa, %[addr], 11 \n"
+ " cacop 0xa, %[addr], 12 \n"
+ " cacop 0xa, %[addr], 13 \n"
+ " cacop 0xa, %[addr], 14 \n"
+ " cacop 0xa, %[addr], 15 \n"
+ " addi.w %[vsets], %[vsets], -1 \n"
+ " addi.d %[addr], %[addr], 0x40 \n"
+ " bnez %[vsets], 2b \n"
+ " li.w %[val], 0x7 \n" /* *state_addr = CPU_DEAD; */
+ " st.w %[val], %[state_addr], 0 \n"
+ " dbar 0 \n"
+ " cacop 0x11, %[state_addr], 0 \n" /* flush entry of *state_addr */
+ : [addr] "=&r" (addr), [val] "=&r" (val)
+ : [state_addr] "r" (state_addr),
+ [sets] "r" (cpu_data[smp_processor_id()].dcache.sets),
+ [vsets] "r" (cpu_data[smp_processor_id()].vcache.sets));
+
+ local_irq_enable();
+ change_csr_ecfg(ECFG0_IM, ECFGF_IPI);
+
+ __asm__ __volatile__(
+ " idle 0 \n"
+ " li.w $t0, 0x1020 \n"
+ " iocsrrd.d %[init_fn], $t0 \n" /* Get init PC */
+ : [init_fn] "=&r" (addr)
+ : /* No Input */
+ : "a0");
+ init_fn = __va(addr);
+
+ init_fn();
+ unreachable();
+}
+
+void play_dead(void)
+{
+ int *state_addr;
+ unsigned int cpu = smp_processor_id();
+ void (*play_dead_uncached)(int *s);
+
+ idle_task_exit();
+ play_dead_uncached = (void *)TO_UNCACHE(__pa((unsigned long)loongson3_play_dead));
+ state_addr = &per_cpu(cpu_state, cpu);
+ mb();
+ play_dead_uncached(state_addr);
+}
+
+static int loongson3_enable_clock(unsigned int cpu)
+{
+ uint64_t core_id = cpu_data[cpu].core;
+ uint64_t package_id = cpu_data[cpu].package;
+
+ LOONGSON_FREQCTRL(package_id) |= 1 << (core_id * 4 + 3);
+
+ return 0;
+}
+
+static int loongson3_disable_clock(unsigned int cpu)
+{
+ uint64_t core_id = cpu_data[cpu].core;
+ uint64_t package_id = cpu_data[cpu].package;
+
+ LOONGSON_FREQCTRL(package_id) &= ~(1 << (core_id * 4 + 3));
+
+ return 0;
+}
+
+static int register_loongson3_notifier(void)
+{
+ return cpuhp_setup_state_nocalls(CPUHP_LOONGARCH_SOC_PREPARE,
+ "loongarch/loongson:prepare",
+ loongson3_enable_clock,
+ loongson3_disable_clock);
+}
+early_initcall(register_loongson3_notifier);
+
+#endif
+
+/*
+ * Power management
+ */
+#ifdef CONFIG_PM
+
+static int loongson3_ipi_suspend(void)
+{
+ return 0;
+}
+
+static void loongson3_ipi_resume(void)
+{
+ iocsr_write32(0xffffffff, LOONGARCH_IOCSR_IPI_EN);
+}
+
+static struct syscore_ops loongson3_ipi_syscore_ops = {
+ .resume = loongson3_ipi_resume,
+ .suspend = loongson3_ipi_suspend,
+};
+
+/*
+ * Enable boot cpu ipi before enabling nonboot cpus
+ * during syscore_resume.
+ */
+static int __init ipi_pm_init(void)
+{
+ register_syscore_ops(&loongson3_ipi_syscore_ops);
+ return 0;
+}
+
+core_initcall(ipi_pm_init);
+#endif
+
+static inline void set_cpu_sibling_map(int cpu)
+{
+ int i;
+
+ cpumask_set_cpu(cpu, &cpu_sibling_setup_map);
+
+ if (smp_num_siblings <= 1)
+ cpumask_set_cpu(cpu, &cpu_sibling_map[cpu]);
+ else {
+ for_each_cpu(i, &cpu_sibling_setup_map) {
+ if (cpus_are_siblings(cpu, i)) {
+ cpumask_set_cpu(i, &cpu_sibling_map[cpu]);
+ cpumask_set_cpu(cpu, &cpu_sibling_map[i]);
+ }
+ }
+ }
+}
+
+static inline void set_cpu_core_map(int cpu)
+{
+ int i;
+
+ cpumask_set_cpu(cpu, &cpu_core_setup_map);
+
+ for_each_cpu(i, &cpu_core_setup_map) {
+ if (cpu_data[cpu].package == cpu_data[i].package) {
+ cpumask_set_cpu(i, &cpu_core_map[cpu]);
+ cpumask_set_cpu(cpu, &cpu_core_map[i]);
+ }
+ }
+}
+
+/*
+ * Calculate a new cpu_foreign_map mask whenever a
+ * new cpu appears or disappears.
+ */
+void calculate_cpu_foreign_map(void)
+{
+ int i, k, core_present;
+ cpumask_t temp_foreign_map;
+
+ /* Re-calculate the mask */
+ cpumask_clear(&temp_foreign_map);
+ for_each_online_cpu(i) {
+ core_present = 0;
+ for_each_cpu(k, &temp_foreign_map)
+ if (cpus_are_siblings(i, k))
+ core_present = 1;
+ if (!core_present)
+ cpumask_set_cpu(i, &temp_foreign_map);
+ }
+
+ for_each_online_cpu(i)
+ cpumask_andnot(&cpu_foreign_map[i],
+ &temp_foreign_map, &cpu_sibling_map[i]);
+}
+
+/* Preload SMP state for boot cpu */
+void smp_prepare_boot_cpu(void)
+{
+ unsigned int cpu;
+
+ set_cpu_possible(0, true);
+ set_cpu_online(0, true);
+ set_my_cpu_offset(per_cpu_offset(0));
+
+ for_each_possible_cpu(cpu)
+ set_cpu_numa_node(cpu, 0);
+}
+
+/* called from main before smp_init() */
+void __init smp_prepare_cpus(unsigned int max_cpus)
+{
+ init_new_context(current, &init_mm);
+ current_thread_info()->cpu = 0;
+ loongson3_prepare_cpus(max_cpus);
+ set_cpu_sibling_map(0);
+ set_cpu_core_map(0);
+ calculate_cpu_foreign_map();
+#ifndef CONFIG_HOTPLUG_CPU
+ init_cpu_present(cpu_possible_mask);
+#endif
+}
+
+int __cpu_up(unsigned int cpu, struct task_struct *tidle)
+{
+ loongson3_boot_secondary(cpu, tidle);
+
+ /* Wait for CPU to start and be ready to sync counters */
+ if (!wait_for_completion_timeout(&cpu_starting,
+ msecs_to_jiffies(5000))) {
+ pr_crit("CPU%u: failed to start\n", cpu);
+ return -EIO;
+ }
+
+ /* Wait for CPU to finish startup & mark itself online before return */
+ wait_for_completion(&cpu_running);
+
+ return 0;
+}
+
+/*
+ * First C code run on the secondary CPUs after being started up by
+ * the master.
+ */
+asmlinkage void start_secondary(void)
+{
+ unsigned int cpu;
+
+ sync_counter();
+ cpu = smp_processor_id();
+ set_my_cpu_offset(per_cpu_offset(cpu));
+
+ cpu_probe();
+ constant_clockevent_init();
+ loongson3_init_secondary();
+
+ set_cpu_sibling_map(cpu);
+ set_cpu_core_map(cpu);
+
+ notify_cpu_starting(cpu);
+
+ /* Notify boot CPU that we're starting */
+ complete(&cpu_starting);
+
+ /* The CPU is running, now mark it online */
+ set_cpu_online(cpu, true);
+
+ calculate_cpu_foreign_map();
+
+ /*
+ * Notify boot CPU that we're up & online and it can safely return
+ * from __cpu_up()
+ */
+ complete(&cpu_running);
+
+ /*
+ * irq will be enabled in loongson3_smp_finish(), enabling it too
+ * early is dangerous.
+ */
+ WARN_ON_ONCE(!irqs_disabled());
+ loongson3_smp_finish();
+
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
+}
+
+void __init smp_cpus_done(unsigned int max_cpus)
+{
+}
+
+static void stop_this_cpu(void *dummy)
+{
+ set_cpu_online(smp_processor_id(), false);
+ calculate_cpu_foreign_map();
+ local_irq_disable();
+ while (true);
+}
+
+void smp_send_stop(void)
+{
+ smp_call_function(stop_this_cpu, NULL, 0);
+}
+
+int setup_profiling_timer(unsigned int multiplier)
+{
+ return 0;
+}
+
+static void flush_tlb_all_ipi(void *info)
+{
+ local_flush_tlb_all();
+}
+
+void flush_tlb_all(void)
+{
+ on_each_cpu(flush_tlb_all_ipi, NULL, 1);
+}
+
+static void flush_tlb_mm_ipi(void *mm)
+{
+ local_flush_tlb_mm((struct mm_struct *)mm);
+}
+
+void flush_tlb_mm(struct mm_struct *mm)
+{
+ if (atomic_read(&mm->mm_users) == 0)
+ return; /* happens as a result of exit_mmap() */
+
+ preempt_disable();
+
+ if ((atomic_read(&mm->mm_users) != 1) || (current->mm != mm)) {
+ on_each_cpu_mask(mm_cpumask(mm), flush_tlb_mm_ipi, mm, 1);
+ } else {
+ unsigned int cpu;
+
+ for_each_online_cpu(cpu) {
+ if (cpu != smp_processor_id() && cpu_context(cpu, mm))
+ cpu_context(cpu, mm) = 0;
+ }
+ local_flush_tlb_mm(mm);
+ }
+
+ preempt_enable();
+}
+
+struct flush_tlb_data {
+ struct vm_area_struct *vma;
+ unsigned long addr1;
+ unsigned long addr2;
+};
+
+static void flush_tlb_range_ipi(void *info)
+{
+ struct flush_tlb_data *fd = info;
+
+ local_flush_tlb_range(fd->vma, fd->addr1, fd->addr2);
+}
+
+void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end)
+{
+ struct mm_struct *mm = vma->vm_mm;
+
+ preempt_disable();
+ if ((atomic_read(&mm->mm_users) != 1) || (current->mm != mm)) {
+ struct flush_tlb_data fd = {
+ .vma = vma,
+ .addr1 = start,
+ .addr2 = end,
+ };
+
+ on_each_cpu_mask(mm_cpumask(mm), flush_tlb_range_ipi, &fd, 1);
+ } else {
+ unsigned int cpu;
+ int exec = vma->vm_flags & VM_EXEC;
+
+ for_each_online_cpu(cpu) {
+ /*
+ * flush_cache_range() will only fully flush icache if
+ * the VMA is executable, otherwise we must invalidate
+ * ASID without it appearing to has_valid_asid() as if
+ * mm has been completely unused by that CPU.
+ */
+ if (cpu != smp_processor_id() && cpu_context(cpu, mm))
+ cpu_context(cpu, mm) = !exec;
+ }
+ local_flush_tlb_range(vma, start, end);
+ }
+ preempt_enable();
+}
+
+static void flush_tlb_kernel_range_ipi(void *info)
+{
+ struct flush_tlb_data *fd = info;
+
+ local_flush_tlb_kernel_range(fd->addr1, fd->addr2);
+}
+
+void flush_tlb_kernel_range(unsigned long start, unsigned long end)
+{
+ struct flush_tlb_data fd = {
+ .addr1 = start,
+ .addr2 = end,
+ };
+
+ on_each_cpu(flush_tlb_kernel_range_ipi, &fd, 1);
+}
+
+static void flush_tlb_page_ipi(void *info)
+{
+ struct flush_tlb_data *fd = info;
+
+ local_flush_tlb_page(fd->vma, fd->addr1);
+}
+
+void flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
+{
+ preempt_disable();
+ if ((atomic_read(&vma->vm_mm->mm_users) != 1) || (current->mm != vma->vm_mm)) {
+ struct flush_tlb_data fd = {
+ .vma = vma,
+ .addr1 = page,
+ };
+
+ on_each_cpu_mask(mm_cpumask(vma->vm_mm), flush_tlb_page_ipi, &fd, 1);
+ } else {
+ unsigned int cpu;
+
+ for_each_online_cpu(cpu) {
+ /*
+ * flush_cache_page() only does partial flushes, so
+ * invalidate ASID without it appearing to
+ * has_valid_asid() as if mm has been completely unused
+ * by that CPU.
+ */
+ if (cpu != smp_processor_id() && cpu_context(cpu, vma->vm_mm))
+ cpu_context(cpu, vma->vm_mm) = 1;
+ }
+ local_flush_tlb_page(vma, page);
+ }
+ preempt_enable();
+}
+EXPORT_SYMBOL(flush_tlb_page);
+
+static void flush_tlb_one_ipi(void *info)
+{
+ unsigned long vaddr = (unsigned long) info;
+
+ local_flush_tlb_one(vaddr);
+}
+
+void flush_tlb_one(unsigned long vaddr)
+{
+ on_each_cpu(flush_tlb_one_ipi, (void *)vaddr, 1);
+}
+EXPORT_SYMBOL(flush_tlb_one);
diff --git a/arch/loongarch/kernel/topology.c b/arch/loongarch/kernel/topology.c
index 3b2cbb95875b..ab1a75c0b5a6 100644
--- a/arch/loongarch/kernel/topology.c
+++ b/arch/loongarch/kernel/topology.c
@@ -1,13 +1,52 @@
// SPDX-License-Identifier: GPL-2.0
#include <linux/cpu.h>
+#include <linux/cpumask.h>
#include <linux/init.h>
+#include <linux/node.h>
+#include <linux/nodemask.h>
#include <linux/percpu.h>
-static struct cpu cpu_device;
+static DEFINE_PER_CPU(struct cpu, cpu_devices);
+
+#ifdef CONFIG_HOTPLUG_CPU
+int arch_register_cpu(int cpu)
+{
+ int ret;
+ struct cpu *c = &per_cpu(cpu_devices, cpu);
+
+ c->hotpluggable = 1;
+ ret = register_cpu(c, cpu);
+ if (ret < 0)
+ pr_warn("register_cpu %d failed (%d)\n", cpu, ret);
+
+ return ret;
+}
+EXPORT_SYMBOL(arch_register_cpu);
+
+void arch_unregister_cpu(int cpu)
+{
+ struct cpu *c = &per_cpu(cpu_devices, cpu);
+
+ c->hotpluggable = 0;
+ unregister_cpu(c);
+}
+EXPORT_SYMBOL(arch_unregister_cpu);
+#endif
static int __init topology_init(void)
{
- return register_cpu(&cpu_device, 0);
+ int i, ret;
+
+ for_each_present_cpu(i) {
+ struct cpu *c = &per_cpu(cpu_devices, i);
+
+ c->hotpluggable = !!i;
+ ret = register_cpu(c, i);
+ if (ret < 0)
+ pr_warn("topology_init: register_cpu %d failed (%d)\n", i, ret);
+ }
+
+ return 0;
}
subsys_initcall(topology_init);
diff --git a/arch/loongarch/kernel/vmlinux.lds.S b/arch/loongarch/kernel/vmlinux.lds.S
index 7da4c4d7c50d..006cbb1bd5c6 100644
--- a/arch/loongarch/kernel/vmlinux.lds.S
+++ b/arch/loongarch/kernel/vmlinux.lds.S
@@ -73,6 +73,10 @@ SECTIONS
EXIT_DATA
}
+#ifdef CONFIG_SMP
+ PERCPU_SECTION(1 << CONFIG_L1_CACHE_SHIFT)
+#endif
+
.init.bss : {
*(.init.bss)
}
diff --git a/arch/loongarch/mm/tlbex.S b/arch/loongarch/mm/tlbex.S
index bef740710a3b..7eee40271577 100644
--- a/arch/loongarch/mm/tlbex.S
+++ b/arch/loongarch/mm/tlbex.S
@@ -88,7 +88,14 @@ vmalloc_done_load:
slli.d t0, t0, _PTE_T_LOG2
add.d t1, ra, t0
+#ifdef CONFIG_SMP
+smp_pgtable_change_load:
+#endif
+#ifdef CONFIG_SMP
+ ll.d t0, t1, 0
+#else
ld.d t0, t1, 0
+#endif
tlbsrch
srli.d ra, t0, _PAGE_PRESENT_SHIFT
@@ -96,7 +103,12 @@ vmalloc_done_load:
beq ra, $r0, nopage_tlb_load
ori t0, t0, _PAGE_VALID
+#ifdef CONFIG_SMP
+ sc.d t0, t1, 0
+ beq t0, $r0, smp_pgtable_change_load
+#else
st.d t0, t1, 0
+#endif
ori t1, t1, 8
xori t1, t1, 8
ld.d t0, t1, 0
@@ -120,14 +132,24 @@ vmalloc_load:
* spots a huge page.
*/
tlb_huge_update_load:
+#ifdef CONFIG_SMP
+ ll.d t0, t1, 0
+#else
ld.d t0, t1, 0
+#endif
srli.d ra, t0, _PAGE_PRESENT_SHIFT
andi ra, ra, 1
beq ra, $r0, nopage_tlb_load
tlbsrch
ori t0, t0, _PAGE_VALID
+#ifdef CONFIG_SMP
+ sc.d t0, t1, 0
+ beq t0, $r0, tlb_huge_update_load
+ ld.d t0, t1, 0
+#else
st.d t0, t1, 0
+#endif
addu16i.d t1, $r0, -(CSR_TLBIDX_EHINV >> 16)
addi.d ra, t1, 0
csrxchg ra, t1, LOONGARCH_CSR_TLBIDX
@@ -173,6 +195,7 @@ tlb_huge_update_load:
csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
nopage_tlb_load:
+ dbar 0
csrrd ra, EXCEPTION_KS2
la.abs t0, tlb_do_page_fault_0
jirl $r0, t0, 0
@@ -229,7 +252,14 @@ vmalloc_done_store:
slli.d t0, t0, _PTE_T_LOG2
add.d t1, ra, t0
+#ifdef CONFIG_SMP
+smp_pgtable_change_store:
+#endif
+#ifdef CONFIG_SMP
+ ll.d t0, t1, 0
+#else
ld.d t0, t1, 0
+#endif
tlbsrch
srli.d ra, t0, _PAGE_PRESENT_SHIFT
@@ -238,7 +268,12 @@ vmalloc_done_store:
bne ra, $r0, nopage_tlb_store
ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+#ifdef CONFIG_SMP
+ sc.d t0, t1, 0
+ beq t0, $r0, smp_pgtable_change_store
+#else
st.d t0, t1, 0
+#endif
ori t1, t1, 8
xori t1, t1, 8
@@ -263,7 +298,11 @@ vmalloc_store:
* spots a huge page.
*/
tlb_huge_update_store:
+#ifdef CONFIG_SMP
+ ll.d t0, t1, 0
+#else
ld.d t0, t1, 0
+#endif
srli.d ra, t0, _PAGE_PRESENT_SHIFT
andi ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
xori ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
@@ -272,7 +311,13 @@ tlb_huge_update_store:
tlbsrch
ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+#ifdef CONFIG_SMP
+ sc.d t0, t1, 0
+ beq t0, $r0, tlb_huge_update_store
+ ld.d t0, t1, 0
+#else
st.d t0, t1, 0
+#endif
addu16i.d t1, $r0, -(CSR_TLBIDX_EHINV >> 16)
addi.d ra, t1, 0
csrxchg ra, t1, LOONGARCH_CSR_TLBIDX
@@ -318,6 +363,7 @@ tlb_huge_update_store:
csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
nopage_tlb_store:
+ dbar 0
csrrd ra, EXCEPTION_KS2
la.abs t0, tlb_do_page_fault_1
jirl $r0, t0, 0
@@ -373,7 +419,14 @@ vmalloc_done_modify:
slli.d t0, t0, _PTE_T_LOG2
add.d t1, ra, t0
+#ifdef CONFIG_SMP
+smp_pgtable_change_modify:
+#endif
+#ifdef CONFIG_SMP
+ ll.d t0, t1, 0
+#else
ld.d t0, t1, 0
+#endif
tlbsrch
srli.d ra, t0, _PAGE_WRITE_SHIFT
@@ -381,7 +434,12 @@ vmalloc_done_modify:
beq ra, $r0, nopage_tlb_modify
ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+#ifdef CONFIG_SMP
+ sc.d t0, t1, 0
+ beq t0, $r0, smp_pgtable_change_modify
+#else
st.d t0, t1, 0
+#endif
ori t1, t1, 8
xori t1, t1, 8
ld.d t0, t1, 0
@@ -405,7 +463,11 @@ vmalloc_modify:
* build_tlbchange_handler_head spots a huge page.
*/
tlb_huge_update_modify:
+#ifdef CONFIG_SMP
+ ll.d t0, t1, 0
+#else
ld.d t0, t1, 0
+#endif
srli.d ra, t0, _PAGE_WRITE_SHIFT
andi ra, ra, 1
@@ -414,7 +476,13 @@ tlb_huge_update_modify:
tlbsrch
ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+#ifdef CONFIG_SMP
+ sc.d t0, t1, 0
+ beq t0, $r0, tlb_huge_update_modify
+ ld.d t0, t1, 0
+#else
st.d t0, t1, 0
+#endif
/*
* A huge PTE describes an area the size of the
* configured huge page size. This is twice the
@@ -454,6 +522,7 @@ tlb_huge_update_modify:
csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
nopage_tlb_modify:
+ dbar 0
csrrd ra, EXCEPTION_KS2
la.abs t0, tlb_do_page_fault_1
jirl $r0, t0, 0
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index b66c5f389159..19f0dbfdd7fe 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -130,6 +130,7 @@ enum cpuhp_state {
CPUHP_ZCOMP_PREPARE,
CPUHP_TIMERS_PREPARE,
CPUHP_MIPS_SOC_PREPARE,
+ CPUHP_LOONGARCH_SOC_PREPARE,
CPUHP_BP_PREPARE_DYN,
CPUHP_BP_PREPARE_DYN_END = CPUHP_BP_PREPARE_DYN + 20,
CPUHP_BRINGUP_CPU,
--
2.27.0
在2022年6月1日六月 上午10:59，Huacai Chen写道：
> Add basic boot, setup and reset routines for LoongArch. Now, LoongArch
> machines use UEFI-based firmware. The firmware passes configuration
> information to the kernel via ACPI and DMI/SMBIOS.
>
> Currently an existing interface between the kernel and the bootloader
> is implemented. Kernel gets 2 values from the bootloader, passed in
> registers a0 and a1; a0 is an "EFI boot flag" distinguishing UEFI and
> non-UEFI firmware, while a1 is a pointer to an FDT with systable,
> memmap, cmdline and initrd information.
>
> This interface is used by existing bootloaders for booting kernels in
> raw ELF format. However, the standard UEFI boot protocol (EFISTUB) is
> preferred.
>
> ECR for adding LoongArch support in ACPI:
> https://mantis.uefi.org/mantis/view.php?id=2203
>
> ECR for adding LoongArch support in ACPI (version update):
> https://mantis.uefi.org/mantis/view.php?id=2268
>
> ECR for adding LoongArch support in UEFI:
> https://mantis.uefi.org/mantis/view.php?id=2313
>
> ACPI changes of LoongArch have been approved in the last year, but the
> new version of ACPI SPEC hasn't been made public yet. And UEFI changes
> of LoongArch are under review now.
>
> Cc: [email protected]
> Cc: Ard Biesheuvel <[email protected]>
> Reviewed-by: WANG Xuerui <[email protected]>
> Co-developed-by: Yun Liu <[email protected]>
> Signed-off-by: Yun Liu <[email protected]>
> Signed-off-by: Huacai Chen <[email protected]>
Reviewed-by: Jiaxun Yang <[email protected]>
So generic efistub is definitely a great thing.
Throwing away LEFI is a wise choice as well :-)
> ---
> arch/loongarch/include/asm/acenv.h | 18 +
> arch/loongarch/include/asm/acpi.h | 38 ++
> arch/loongarch/include/asm/bootinfo.h | 41 +++
> arch/loongarch/include/asm/dmi.h | 24 ++
> arch/loongarch/include/asm/efi.h | 41 +++
> arch/loongarch/include/asm/reboot.h | 10 +
> arch/loongarch/include/asm/setup.h | 21 ++
> arch/loongarch/kernel/acpi.c | 169 +++++++++
> arch/loongarch/kernel/cacheinfo.c | 122 +++++++
> arch/loongarch/kernel/cpu-probe.c | 292 +++++++++++++++
> arch/loongarch/kernel/efi-header.S | 104 ++++++
> arch/loongarch/kernel/efi.c | 226 ++++++++++++
> arch/loongarch/kernel/env.c | 70 ++++
> arch/loongarch/kernel/head.S | 99 +++++
> arch/loongarch/kernel/image-vars.h | 29 ++
> arch/loongarch/kernel/mem.c | 64 ++++
> arch/loongarch/kernel/reset.c | 90 +++++
> arch/loongarch/kernel/setup.c | 341 ++++++++++++++++++
> arch/loongarch/kernel/time.c | 213 +++++++++++
> arch/loongarch/kernel/topology.c | 13 +
> drivers/firmware/efi/Kconfig | 2 +-
> drivers/firmware/efi/libstub/Makefile | 10 +
> .../firmware/efi/libstub/efi-stub-helper.c | 2 +-
> drivers/firmware/efi/libstub/efi-stub.c | 2 +-
> drivers/firmware/efi/libstub/loongarch-stub.c | 88 +++++
> include/linux/efi.h | 1 +
> include/linux/pe.h | 2 +
> 27 files changed, 2129 insertions(+), 3 deletions(-)
> create mode 100644 arch/loongarch/include/asm/acenv.h
> create mode 100644 arch/loongarch/include/asm/acpi.h
> create mode 100644 arch/loongarch/include/asm/bootinfo.h
> create mode 100644 arch/loongarch/include/asm/dmi.h
> create mode 100644 arch/loongarch/include/asm/efi.h
> create mode 100644 arch/loongarch/include/asm/reboot.h
> create mode 100644 arch/loongarch/include/asm/setup.h
> create mode 100644 arch/loongarch/kernel/acpi.c
> create mode 100644 arch/loongarch/kernel/cacheinfo.c
> create mode 100644 arch/loongarch/kernel/cpu-probe.c
> create mode 100644 arch/loongarch/kernel/efi-header.S
> create mode 100644 arch/loongarch/kernel/efi.c
> create mode 100644 arch/loongarch/kernel/env.c
> create mode 100644 arch/loongarch/kernel/head.S
> create mode 100644 arch/loongarch/kernel/image-vars.h
> create mode 100644 arch/loongarch/kernel/mem.c
> create mode 100644 arch/loongarch/kernel/reset.c
> create mode 100644 arch/loongarch/kernel/setup.c
> create mode 100644 arch/loongarch/kernel/time.c
> create mode 100644 arch/loongarch/kernel/topology.c
> create mode 100644 drivers/firmware/efi/libstub/loongarch-stub.c
>
> diff --git a/arch/loongarch/include/asm/acenv.h
> b/arch/loongarch/include/asm/acenv.h
> new file mode 100644
> index 000000000000..52f298f7293b
> --- /dev/null
> +++ b/arch/loongarch/include/asm/acenv.h
> @@ -0,0 +1,18 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * LoongArch specific ACPICA environments and implementation
> + *
> + * Author: Jianmin Lv <[email protected]>
> + * Huacai Chen <[email protected]>
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +
> +#ifndef _ASM_LOONGARCH_ACENV_H
> +#define _ASM_LOONGARCH_ACENV_H
> +
> +/*
> + * This header is required by ACPI core, but we have nothing to fill in
> + * right now. Will be updated later when needed.
> + */
> +
> +#endif /* _ASM_LOONGARCH_ACENV_H */
> diff --git a/arch/loongarch/include/asm/acpi.h
> b/arch/loongarch/include/asm/acpi.h
> new file mode 100644
> index 000000000000..62044cd5b7bc
> --- /dev/null
> +++ b/arch/loongarch/include/asm/acpi.h
> @@ -0,0 +1,38 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Author: Jianmin Lv <[email protected]>
> + * Huacai Chen <[email protected]>
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +
> +#ifndef _ASM_LOONGARCH_ACPI_H
> +#define _ASM_LOONGARCH_ACPI_H
> +
> +#ifdef CONFIG_ACPI
> +extern int acpi_strict;
> +extern int acpi_disabled;
> +extern int acpi_pci_disabled;
> +extern int acpi_noirq;
> +
> +#define acpi_os_ioremap acpi_os_ioremap
> +void __init __iomem *acpi_os_ioremap(acpi_physical_address phys,
> acpi_size size);
> +
> +static inline void disable_acpi(void)
> +{
> + acpi_disabled = 1;
> + acpi_pci_disabled = 1;
> + acpi_noirq = 1;
> +}
> +
> +static inline bool acpi_has_cpu_in_madt(void)
> +{
> + return true;
> +}
> +
> +extern struct list_head acpi_wakeup_device_list;
> +
> +#endif /* !CONFIG_ACPI */
> +
> +#define ACPI_TABLE_UPGRADE_MAX_PHYS ARCH_LOW_ADDRESS_LIMIT
> +
> +#endif /* _ASM_LOONGARCH_ACPI_H */
> diff --git a/arch/loongarch/include/asm/bootinfo.h
> b/arch/loongarch/include/asm/bootinfo.h
> new file mode 100644
> index 000000000000..7b60f202faaf
> --- /dev/null
> +++ b/arch/loongarch/include/asm/bootinfo.h
> @@ -0,0 +1,41 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef _ASM_BOOTINFO_H
> +#define _ASM_BOOTINFO_H
> +
> +#include <linux/types.h>
> +#include <asm/setup.h>
> +
> +const char *get_system_type(void);
> +
> +extern void init_environ(void);
> +extern void memblock_init(void);
> +extern void platform_init(void);
> +
> +struct loongson_board_info {
> + int bios_size;
> + const char *bios_vendor;
> + const char *bios_version;
> + const char *bios_release_date;
> + const char *board_name;
> + const char *board_vendor;
> +};
> +
> +struct loongson_system_configuration {
> + int nr_cpus;
> + int nr_nodes;
> + int nr_io_pics;
> + int boot_cpu_id;
> + int cores_per_node;
> + int cores_per_package;
> + const char *cpuname;
> +};
> +
> +extern u64 efi_system_table;
> +extern unsigned long fw_arg0, fw_arg1;
> +extern struct loongson_board_info b_info;
> +extern struct loongson_system_configuration loongson_sysconf;
> +
> +#endif /* _ASM_BOOTINFO_H */
> diff --git a/arch/loongarch/include/asm/dmi.h
> b/arch/loongarch/include/asm/dmi.h
> new file mode 100644
> index 000000000000..605493417753
> --- /dev/null
> +++ b/arch/loongarch/include/asm/dmi.h
> @@ -0,0 +1,24 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef _ASM_DMI_H
> +#define _ASM_DMI_H
> +
> +#include <linux/io.h>
> +#include <linux/memblock.h>
> +
> +#define dmi_early_remap(x, l) dmi_remap(x, l)
> +#define dmi_early_unmap(x, l) dmi_unmap(x)
> +#define dmi_alloc(l) memblock_alloc(l, PAGE_SIZE)
> +
> +static inline void *dmi_remap(u64 phys_addr, unsigned long size)
> +{
> + return ((void *)TO_CACHE(phys_addr));
> +}
> +
> +static inline void dmi_unmap(void *addr)
> +{
> +}
> +
> +#endif /* _ASM_DMI_H */
> diff --git a/arch/loongarch/include/asm/efi.h
> b/arch/loongarch/include/asm/efi.h
> new file mode 100644
> index 000000000000..0127d84d5e1d
> --- /dev/null
> +++ b/arch/loongarch/include/asm/efi.h
> @@ -0,0 +1,41 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef _ASM_LOONGARCH_EFI_H
> +#define _ASM_LOONGARCH_EFI_H
> +
> +#include <linux/efi.h>
> +
> +void __init efi_init(void);
> +void __init efi_runtime_init(void);
> +void efifb_setup_from_dmi(struct screen_info *si, const char *opt);
> +
> +#define ARCH_EFI_IRQ_FLAGS_MASK 0x00000004 /* Bit 2: CSR.CRMD.IE */
> +
> +#define arch_efi_call_virt_setup() \
> +({ \
> +})
> +
> +#define arch_efi_call_virt(p, f, args...) \
> +({ \
> + efi_##f##_t * __f; \
> + __f = p->f; \
> + __f(args); \
> +})
> +
> +#define arch_efi_call_virt_teardown() \
> +({ \
> +})
> +
> +#define EFI_ALLOC_ALIGN SZ_64K
> +
> +struct screen_info *alloc_screen_info(void);
> +void free_screen_info(struct screen_info *si);
> +
> +static inline unsigned long efi_get_max_initrd_addr(unsigned long
> image_addr)
> +{
> + return ULONG_MAX;
> +}
> +
> +#endif /* _ASM_LOONGARCH_EFI_H */
> diff --git a/arch/loongarch/include/asm/reboot.h
> b/arch/loongarch/include/asm/reboot.h
> new file mode 100644
> index 000000000000..51151749d8f0
> --- /dev/null
> +++ b/arch/loongarch/include/asm/reboot.h
> @@ -0,0 +1,10 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef _ASM_REBOOT_H
> +#define _ASM_REBOOT_H
> +
> +extern void (*pm_restart)(void);
> +
> +#endif /* _ASM_REBOOT_H */
> diff --git a/arch/loongarch/include/asm/setup.h
> b/arch/loongarch/include/asm/setup.h
> new file mode 100644
> index 000000000000..6d7d2a3e23dd
> --- /dev/null
> +++ b/arch/loongarch/include/asm/setup.h
> @@ -0,0 +1,21 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +
> +#ifndef _LOONGARCH_SETUP_H
> +#define _LOONGARCH_SETUP_H
> +
> +#include <linux/types.h>
> +#include <uapi/asm/setup.h>
> +
> +#define VECSIZE 0x200
> +
> +extern unsigned long eentry;
> +extern unsigned long tlbrentry;
> +extern void cpu_cache_init(void);
> +extern void per_cpu_trap_init(int cpu);
> +extern void set_handler(unsigned long offset, void *addr, unsigned
> long len);
> +extern void set_merr_handler(unsigned long offset, void *addr,
> unsigned long len);
> +
> +#endif /* __SETUP_H */
> diff --git a/arch/loongarch/kernel/acpi.c b/arch/loongarch/kernel/acpi.c
> new file mode 100644
> index 000000000000..a644220bb426
> --- /dev/null
> +++ b/arch/loongarch/kernel/acpi.c
> @@ -0,0 +1,169 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * acpi.c - Architecture-Specific Low-Level ACPI Boot Support
> + *
> + * Author: Jianmin Lv <[email protected]>
> + * Huacai Chen <[email protected]>
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +
> +#include <linux/init.h>
> +#include <linux/acpi.h>
> +#include <linux/irq.h>
> +#include <linux/irqdomain.h>
> +#include <linux/memblock.h>
> +#include <linux/serial_core.h>
> +#include <asm/io.h>
> +#include <asm/loongson.h>
> +
> +int acpi_disabled;
> +EXPORT_SYMBOL(acpi_disabled);
> +int acpi_noirq;
> +int acpi_pci_disabled;
> +EXPORT_SYMBOL(acpi_pci_disabled);
> +int acpi_strict = 1; /* We have no workarounds on LoongArch */
> +int num_processors;
> +int disabled_cpus;
> +enum acpi_irq_model_id acpi_irq_model = ACPI_IRQ_MODEL_PLATFORM;
> +
> +u64 acpi_saved_sp;
> +
> +#define MAX_CORE_PIC 256
> +
> +#define PREFIX "ACPI: "
> +
> +int acpi_gsi_to_irq(u32 gsi, unsigned int *irqp)
> +{
> + if (irqp != NULL)
> + *irqp = acpi_register_gsi(NULL, gsi, -1, -1);
> + return (*irqp >= 0) ? 0 : -EINVAL;
> +}
> +EXPORT_SYMBOL_GPL(acpi_gsi_to_irq);
> +
> +int acpi_isa_irq_to_gsi(unsigned int isa_irq, u32 *gsi)
> +{
> + if (gsi)
> + *gsi = isa_irq;
> + return 0;
> +}
> +
> +/*
> + * success: return IRQ number (>=0)
> + * failure: return < 0
> + */
> +int acpi_register_gsi(struct device *dev, u32 gsi, int trigger, int
> polarity)
> +{
> + struct irq_fwspec fwspec;
> +
> + switch (gsi) {
> + case GSI_MIN_CPU_IRQ ... GSI_MAX_CPU_IRQ:
> + fwspec.fwnode = liointc_domain->fwnode;
> + fwspec.param[0] = gsi - GSI_MIN_CPU_IRQ;
> + fwspec.param_count = 1;
> +
> + return irq_create_fwspec_mapping(&fwspec);
> +
> + case GSI_MIN_LPC_IRQ ... GSI_MAX_LPC_IRQ:
> + if (!pch_lpc_domain)
> + return -EINVAL;
> +
> + fwspec.fwnode = pch_lpc_domain->fwnode;
> + fwspec.param[0] = gsi - GSI_MIN_LPC_IRQ;
> + fwspec.param[1] = acpi_dev_get_irq_type(trigger, polarity);
> + fwspec.param_count = 2;
> +
> + return irq_create_fwspec_mapping(&fwspec);
> +
> + case GSI_MIN_PCH_IRQ ... GSI_MAX_PCH_IRQ:
> + if (!pch_pic_domain[0])
> + return -EINVAL;
> +
> + fwspec.fwnode = pch_pic_domain[0]->fwnode;
> + fwspec.param[0] = gsi - GSI_MIN_PCH_IRQ;
> + fwspec.param[1] = IRQ_TYPE_LEVEL_HIGH;
> + fwspec.param_count = 2;
> +
> + return irq_create_fwspec_mapping(&fwspec);
> + }
> +
> + return -EINVAL;
> +}
> +EXPORT_SYMBOL_GPL(acpi_register_gsi);
> +
> +void acpi_unregister_gsi(u32 gsi)
> +{
> +
> +}
> +EXPORT_SYMBOL_GPL(acpi_unregister_gsi);
> +
> +void __init __iomem * __acpi_map_table(unsigned long phys, unsigned
> long size)
> +{
> +
> + if (!phys || !size)
> + return NULL;
> +
> + return early_memremap(phys, size);
> +}
> +void __init __acpi_unmap_table(void __iomem *map, unsigned long size)
> +{
> + if (!map || !size)
> + return;
> +
> + early_memunmap(map, size);
> +}
> +
> +void __init __iomem *acpi_os_ioremap(acpi_physical_address phys,
> acpi_size size)
> +{
> + if (!memblock_is_memory(phys))
> + return ioremap(phys, size);
> + else
> + return ioremap_cache(phys, size);
> +}
> +
> +void __init acpi_boot_table_init(void)
> +{
> + /*
> + * If acpi_disabled, bail out
> + */
> + if (acpi_disabled)
> + return;
> +
> + /*
> + * Initialize the ACPI boot-time table parser.
> + */
> + if (acpi_table_init()) {
> + disable_acpi();
> + return;
> + }
> +}
> +
> +static void __init acpi_process_madt(void)
> +{
> + loongson_sysconf.nr_cpus = num_processors;
> +}
> +
> +int __init acpi_boot_init(void)
> +{
> + /*
> + * If acpi_disabled, bail out
> + */
> + if (acpi_disabled)
> + return -1;
> +
> + loongson_sysconf.boot_cpu_id = read_csr_cpuid();
> +
> + /*
> + * Process the Multiple APIC Description Table (MADT), if present
> + */
> + acpi_process_madt();
> +
> + /* Do not enable ACPI SPCR console by default */
> + acpi_parse_spcr(earlycon_acpi_spcr_enable, false);
> +
> + return 0;
> +}
> +
> +void __init arch_reserve_mem_area(acpi_physical_address addr, size_t
> size)
> +{
> + memblock_reserve(addr, size);
> +}
> diff --git a/arch/loongarch/kernel/cacheinfo.c
> b/arch/loongarch/kernel/cacheinfo.c
> new file mode 100644
> index 000000000000..8c9fe29e98f0
> --- /dev/null
> +++ b/arch/loongarch/kernel/cacheinfo.c
> @@ -0,0 +1,122 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * LoongArch cacheinfo support
> + *
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#include <linux/cacheinfo.h>
> +
> +/* Populates leaf and increments to next leaf */
> +#define populate_cache(cache, leaf, c_level, c_type) \
> +do { \
> + leaf->type = c_type; \
> + leaf->level = c_level; \
> + leaf->coherency_line_size = c->cache.linesz; \
> + leaf->number_of_sets = c->cache.sets; \
> + leaf->ways_of_associativity = c->cache.ways; \
> + leaf->size = c->cache.linesz * c->cache.sets * \
> + c->cache.ways; \
> + leaf++; \
> +} while (0)
> +
> +int init_cache_level(unsigned int cpu)
> +{
> + struct cpuinfo_loongarch *c = ¤t_cpu_data;
> + struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
> + int levels = 0, leaves = 0;
> +
> + /*
> + * If Dcache is not set, we assume the cache structures
> + * are not properly initialized.
> + */
> + if (c->dcache.waysize)
> + levels += 1;
> + else
> + return -ENOENT;
> +
> +
> + leaves += (c->icache.waysize) ? 2 : 1;
> +
> + if (c->vcache.waysize) {
> + levels++;
> + leaves++;
> + }
> +
> + if (c->scache.waysize) {
> + levels++;
> + leaves++;
> + }
> +
> + if (c->tcache.waysize) {
> + levels++;
> + leaves++;
> + }
> +
> + this_cpu_ci->num_levels = levels;
> + this_cpu_ci->num_leaves = leaves;
> + return 0;
> +}
> +
> +static inline bool cache_leaves_are_shared(struct cacheinfo *this_leaf,
> + struct cacheinfo *sib_leaf)
> +{
> + return !((this_leaf->level == 1) || (this_leaf->level == 2));
> +}
> +
> +static void cache_cpumap_setup(unsigned int cpu)
> +{
> + struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
> + struct cacheinfo *this_leaf, *sib_leaf;
> + unsigned int index;
> +
> + for (index = 0; index < this_cpu_ci->num_leaves; index++) {
> + unsigned int i;
> +
> + this_leaf = this_cpu_ci->info_list + index;
> + /* skip if shared_cpu_map is already populated */
> + if (!cpumask_empty(&this_leaf->shared_cpu_map))
> + continue;
> +
> + cpumask_set_cpu(cpu, &this_leaf->shared_cpu_map);
> + for_each_online_cpu(i) {
> + struct cpu_cacheinfo *sib_cpu_ci = get_cpu_cacheinfo(i);
> +
> + if (i == cpu || !sib_cpu_ci->info_list)
> + continue;/* skip if itself or no cacheinfo */
> + sib_leaf = sib_cpu_ci->info_list + index;
> + if (cache_leaves_are_shared(this_leaf, sib_leaf)) {
> + cpumask_set_cpu(cpu, &sib_leaf->shared_cpu_map);
> + cpumask_set_cpu(i, &this_leaf->shared_cpu_map);
> + }
> + }
> + }
> +}
> +
> +int populate_cache_leaves(unsigned int cpu)
> +{
> + int level = 1;
> + struct cpuinfo_loongarch *c = ¤t_cpu_data;
> + struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
> + struct cacheinfo *this_leaf = this_cpu_ci->info_list;
> +
> + if (c->icache.waysize) {
> + populate_cache(dcache, this_leaf, level, CACHE_TYPE_DATA);
> + populate_cache(icache, this_leaf, level++, CACHE_TYPE_INST);
> + } else {
> + populate_cache(dcache, this_leaf, level++, CACHE_TYPE_UNIFIED);
> + }
> +
> + if (c->vcache.waysize)
> + populate_cache(vcache, this_leaf, level++, CACHE_TYPE_UNIFIED);
> +
> + if (c->scache.waysize)
> + populate_cache(scache, this_leaf, level++, CACHE_TYPE_UNIFIED);
> +
> + if (c->tcache.waysize)
> + populate_cache(tcache, this_leaf, level++, CACHE_TYPE_UNIFIED);
> +
> + cache_cpumap_setup(cpu);
> + this_cpu_ci->cpu_map_populated = true;
> +
> + return 0;
> +}
> diff --git a/arch/loongarch/kernel/cpu-probe.c
> b/arch/loongarch/kernel/cpu-probe.c
> new file mode 100644
> index 000000000000..6c87ea36b257
> --- /dev/null
> +++ b/arch/loongarch/kernel/cpu-probe.c
> @@ -0,0 +1,292 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Processor capabilities determination functions.
> + *
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#include <linux/init.h>
> +#include <linux/kernel.h>
> +#include <linux/ptrace.h>
> +#include <linux/smp.h>
> +#include <linux/stddef.h>
> +#include <linux/export.h>
> +#include <linux/printk.h>
> +#include <linux/uaccess.h>
> +
> +#include <asm/cpu-features.h>
> +#include <asm/elf.h>
> +#include <asm/fpu.h>
> +#include <asm/loongarch.h>
> +#include <asm/pgtable-bits.h>
> +#include <asm/setup.h>
> +
> +/* Hardware capabilities */
> +unsigned int elf_hwcap __read_mostly;
> +EXPORT_SYMBOL_GPL(elf_hwcap);
> +
> +/*
> + * Determine the FCSR mask for FPU hardware.
> + */
> +static inline void cpu_set_fpu_fcsr_mask(struct cpuinfo_loongarch *c)
> +{
> + unsigned long sr, mask, fcsr, fcsr0, fcsr1;
> +
> + fcsr = c->fpu_csr0;
> + mask = FPU_CSR_ALL_X | FPU_CSR_ALL_E | FPU_CSR_ALL_S | FPU_CSR_RM;
> +
> + sr = read_csr_euen();
> + enable_fpu();
> +
> + fcsr0 = fcsr & mask;
> + write_fcsr(LOONGARCH_FCSR0, fcsr0);
> + fcsr0 = read_fcsr(LOONGARCH_FCSR0);
> +
> + fcsr1 = fcsr | ~mask;
> + write_fcsr(LOONGARCH_FCSR0, fcsr1);
> + fcsr1 = read_fcsr(LOONGARCH_FCSR0);
> +
> + write_fcsr(LOONGARCH_FCSR0, fcsr);
> +
> + write_csr_euen(sr);
> +
> + c->fpu_mask = ~(fcsr0 ^ fcsr1) & ~mask;
> +}
> +
> +static inline void set_elf_platform(int cpu, const char *plat)
> +{
> + if (cpu == 0)
> + __elf_platform = plat;
> +}
> +
> +/* MAP BASE */
> +unsigned long vm_map_base;
> +EXPORT_SYMBOL_GPL(vm_map_base);
> +
> +static void cpu_probe_addrbits(struct cpuinfo_loongarch *c)
> +{
> +#ifdef __NEED_ADDRBITS_PROBE
> + c->pabits = (read_cpucfg(LOONGARCH_CPUCFG1) & CPUCFG1_PABITS) >> 4;
> + c->vabits = (read_cpucfg(LOONGARCH_CPUCFG1) & CPUCFG1_VABITS) >> 12;
> + vm_map_base = 0UL - (1UL << c->vabits);
> +#endif
> +}
> +
> +static void set_isa(struct cpuinfo_loongarch *c, unsigned int isa)
> +{
> + switch (isa) {
> + case LOONGARCH_CPU_ISA_LA64:
> + c->isa_level |= LOONGARCH_CPU_ISA_LA64;
> + fallthrough;
> + case LOONGARCH_CPU_ISA_LA32S:
> + c->isa_level |= LOONGARCH_CPU_ISA_LA32S;
> + fallthrough;
> + case LOONGARCH_CPU_ISA_LA32R:
> + c->isa_level |= LOONGARCH_CPU_ISA_LA32R;
> + break;
> + }
> +}
> +
> +static void cpu_probe_common(struct cpuinfo_loongarch *c)
> +{
> + unsigned int config;
> + unsigned long asid_mask;
> +
> + c->options = LOONGARCH_CPU_CPUCFG | LOONGARCH_CPU_CSR |
> + LOONGARCH_CPU_TLB | LOONGARCH_CPU_VINT | LOONGARCH_CPU_WATCH;
> +
> + elf_hwcap |= HWCAP_LOONGARCH_CRC32;
> +
> + config = read_cpucfg(LOONGARCH_CPUCFG1);
> + if (config & CPUCFG1_UAL) {
> + c->options |= LOONGARCH_CPU_UAL;
> + elf_hwcap |= HWCAP_LOONGARCH_UAL;
> + }
> +
> + config = read_cpucfg(LOONGARCH_CPUCFG2);
> + if (config & CPUCFG2_LAM) {
> + c->options |= LOONGARCH_CPU_LAM;
> + elf_hwcap |= HWCAP_LOONGARCH_LAM;
> + }
> + if (config & CPUCFG2_FP) {
> + c->options |= LOONGARCH_CPU_FPU;
> + elf_hwcap |= HWCAP_LOONGARCH_FPU;
> + }
> + if (config & CPUCFG2_COMPLEX) {
> + c->options |= LOONGARCH_CPU_COMPLEX;
> + elf_hwcap |= HWCAP_LOONGARCH_COMPLEX;
> + }
> + if (config & CPUCFG2_CRYPTO) {
> + c->options |= LOONGARCH_CPU_CRYPTO;
> + elf_hwcap |= HWCAP_LOONGARCH_CRYPTO;
> + }
> + if (config & CPUCFG2_LVZP) {
> + c->options |= LOONGARCH_CPU_LVZ;
> + elf_hwcap |= HWCAP_LOONGARCH_LVZ;
> + }
> +
> + config = read_cpucfg(LOONGARCH_CPUCFG6);
> + if (config & CPUCFG6_PMP)
> + c->options |= LOONGARCH_CPU_PMP;
> +
> + config = iocsr_read32(LOONGARCH_IOCSR_FEATURES);
> + if (config & IOCSRF_CSRIPI)
> + c->options |= LOONGARCH_CPU_CSRIPI;
> + if (config & IOCSRF_EXTIOI)
> + c->options |= LOONGARCH_CPU_EXTIOI;
> + if (config & IOCSRF_FREQSCALE)
> + c->options |= LOONGARCH_CPU_SCALEFREQ;
> + if (config & IOCSRF_FLATMODE)
> + c->options |= LOONGARCH_CPU_FLATMODE;
> + if (config & IOCSRF_EIODECODE)
> + c->options |= LOONGARCH_CPU_EIODECODE;
> + if (config & IOCSRF_VM)
> + c->options |= LOONGARCH_CPU_HYPERVISOR;
> +
> + config = csr_read32(LOONGARCH_CSR_ASID);
> + config = (config & CSR_ASID_BIT) >> CSR_ASID_BIT_SHIFT;
> + asid_mask = GENMASK(config - 1, 0);
> + set_cpu_asid_mask(c, asid_mask);
> +
> + config = read_csr_prcfg1();
> + c->ksave_mask = GENMASK((config & CSR_CONF1_KSNUM) - 1, 0);
> + c->ksave_mask &= ~(EXC_KSAVE_MASK | PERCPU_KSAVE_MASK |
> KVM_KSAVE_MASK);
> +
> + config = read_csr_prcfg3();
> + switch (config & CSR_CONF3_TLBTYPE) {
> + case 0:
> + c->tlbsizemtlb = 0;
> + c->tlbsizestlbsets = 0;
> + c->tlbsizestlbways = 0;
> + c->tlbsize = 0;
> + break;
> + case 1:
> + c->tlbsizemtlb = ((config & CSR_CONF3_MTLBSIZE) >>
> CSR_CONF3_MTLBSIZE_SHIFT) + 1;
> + c->tlbsizestlbsets = 0;
> + c->tlbsizestlbways = 0;
> + c->tlbsize = c->tlbsizemtlb + c->tlbsizestlbsets *
> c->tlbsizestlbways;
> + break;
> + case 2:
> + c->tlbsizemtlb = ((config & CSR_CONF3_MTLBSIZE) >>
> CSR_CONF3_MTLBSIZE_SHIFT) + 1;
> + c->tlbsizestlbsets = 1 << ((config & CSR_CONF3_STLBIDX) >>
> CSR_CONF3_STLBIDX_SHIFT);
> + c->tlbsizestlbways = ((config & CSR_CONF3_STLBWAYS) >>
> CSR_CONF3_STLBWAYS_SHIFT) + 1;
> + c->tlbsize = c->tlbsizemtlb + c->tlbsizestlbsets *
> c->tlbsizestlbways;
> + break;
> + default:
> + pr_warn("Warning: unknown TLB type\n");
> + }
> +}
> +
> +#define MAX_NAME_LEN 32
> +#define VENDOR_OFFSET 0
> +#define CPUNAME_OFFSET 9
> +
> +static char cpu_full_name[MAX_NAME_LEN] = " - ";
> +
> +static inline void cpu_probe_loongson(struct cpuinfo_loongarch *c,
> unsigned int cpu)
> +{
> + uint64_t *vendor = (void *)(&cpu_full_name[VENDOR_OFFSET]);
> + uint64_t *cpuname = (void *)(&cpu_full_name[CPUNAME_OFFSET]);
> +
> + __cpu_full_name[cpu] = cpu_full_name;
> + *vendor = iocsr_read64(LOONGARCH_IOCSR_VENDOR);
> + *cpuname = iocsr_read64(LOONGARCH_IOCSR_CPUNAME);
> +
> + switch (c->processor_id & PRID_SERIES_MASK) {
> + case PRID_SERIES_LA132:
> + c->cputype = CPU_LOONGSON32;
> + set_isa(c, LOONGARCH_CPU_ISA_LA32S);
> + __cpu_family[cpu] = "Loongson-32bit";
> + pr_info("32-bit Loongson Processor probed (LA132 Core)\n");
> + break;
> + case PRID_SERIES_LA264:
> + c->cputype = CPU_LOONGSON64;
> + set_isa(c, LOONGARCH_CPU_ISA_LA64);
> + __cpu_family[cpu] = "Loongson-64bit";
> + pr_info("64-bit Loongson Processor probed (LA264 Core)\n");
> + break;
> + case PRID_SERIES_LA364:
> + c->cputype = CPU_LOONGSON64;
> + set_isa(c, LOONGARCH_CPU_ISA_LA64);
> + __cpu_family[cpu] = "Loongson-64bit";
> + pr_info("64-bit Loongson Processor probed (LA364 Core)\n");
> + break;
> + case PRID_SERIES_LA464:
> + c->cputype = CPU_LOONGSON64;
> + set_isa(c, LOONGARCH_CPU_ISA_LA64);
> + __cpu_family[cpu] = "Loongson-64bit";
> + pr_info("64-bit Loongson Processor probed (LA464 Core)\n");
> + break;
> + case PRID_SERIES_LA664:
> + c->cputype = CPU_LOONGSON64;
> + set_isa(c, LOONGARCH_CPU_ISA_LA64);
> + __cpu_family[cpu] = "Loongson-64bit";
> + pr_info("64-bit Loongson Processor probed (LA664 Core)\n");
> + break;
> + default: /* Default to 64 bit */
> + c->cputype = CPU_LOONGSON64;
> + set_isa(c, LOONGARCH_CPU_ISA_LA64);
> + __cpu_family[cpu] = "Loongson-64bit";
> + pr_info("64-bit Loongson Processor probed (Unknown Core)\n");
> + }
> +}
> +
> +#ifdef CONFIG_64BIT
> +/* For use by uaccess.h */
> +u64 __ua_limit;
> +EXPORT_SYMBOL(__ua_limit);
> +#endif
> +
> +const char *__cpu_family[NR_CPUS];
> +const char *__cpu_full_name[NR_CPUS];
> +const char *__elf_platform;
> +
> +static void cpu_report(void)
> +{
> + struct cpuinfo_loongarch *c = ¤t_cpu_data;
> +
> + pr_info("CPU%d revision is: %08x (%s)\n",
> + smp_processor_id(), c->processor_id, cpu_family_string());
> + if (c->options & LOONGARCH_CPU_FPU)
> + pr_info("FPU%d revision is: %08x\n", smp_processor_id(),
> c->fpu_vers);
> +}
> +
> +void cpu_probe(void)
> +{
> + unsigned int cpu = smp_processor_id();
> + struct cpuinfo_loongarch *c = ¤t_cpu_data;
> +
> + /*
> + * Set a default ELF platform, cpu probe may later
> + * overwrite it with a more precise value
> + */
> + set_elf_platform(cpu, "loongarch");
> +
> + c->cputype = CPU_UNKNOWN;
> + c->processor_id = read_cpucfg(LOONGARCH_CPUCFG0);
> + c->fpu_vers = (read_cpucfg(LOONGARCH_CPUCFG2) >> 3) & 0x3;
> +
> + c->fpu_csr0 = FPU_CSR_RN;
> + c->fpu_mask = FPU_CSR_RSVD;
> +
> + cpu_probe_common(c);
> +
> + per_cpu_trap_init(cpu);
> +
> + switch (c->processor_id & PRID_COMP_MASK) {
> + case PRID_COMP_LOONGSON:
> + cpu_probe_loongson(c, cpu);
> + break;
> + }
> +
> + BUG_ON(!__cpu_family[cpu]);
> + BUG_ON(c->cputype == CPU_UNKNOWN);
> +
> + cpu_probe_addrbits(c);
> +
> +#ifdef CONFIG_64BIT
> + if (cpu == 0)
> + __ua_limit = ~((1ull << cpu_vabits) - 1);
> +#endif
> +
> + cpu_report();
> +}
> diff --git a/arch/loongarch/kernel/efi-header.S
> b/arch/loongarch/kernel/efi-header.S
> new file mode 100644
> index 000000000000..1d9c735fe6d4
> --- /dev/null
> +++ b/arch/loongarch/kernel/efi-header.S
> @@ -0,0 +1,104 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +
> +#include <linux/pe.h>
> +#include <linux/sizes.h>
> +
> + .macro __EFI_PE_HEADER
> + .long PE_MAGIC
> +.Lcoff_header:
> +#ifdef CONFIG_32BIT
> + .short IMAGE_FILE_MACHINE_LOONGARCH32 /* Machine */
> +#else
> + .short IMAGE_FILE_MACHINE_LOONGARCH64 /* Machine */
> +#endif
> + .short .Lsection_count /* NumberOfSections */
> + .long 0 /* TimeDateStamp */
> + .long 0 /* PointerToSymbolTable */
> + .long 0 /* NumberOfSymbols */
> + .short .Lsection_table - .Loptional_header /* SizeOfOptionalHeader */
> + .short IMAGE_FILE_DEBUG_STRIPPED | \
> + IMAGE_FILE_EXECUTABLE_IMAGE | \
> + IMAGE_FILE_LINE_NUMS_STRIPPED /* Characteristics */
> +
> +.Loptional_header:
> + .short PE_OPT_MAGIC_PE32PLUS /* PE32+ format */
> + .byte 0x02 /* MajorLinkerVersion */
> + .byte 0x14 /* MinorLinkerVersion */
> + .long __inittext_end - .Lefi_header_end /* SizeOfCode */
> + .long _end - __initdata_begin /* SizeOfInitializedData */
> + .long 0 /* SizeOfUninitializedData */
> + .long __efistub_efi_pe_entry - _head /* AddressOfEntryPoint */
> + .long .Lefi_header_end - _head /* BaseOfCode */
> +
> +.Lextra_header_fields:
> + .quad 0 /* ImageBase */
> + .long PECOFF_SEGMENT_ALIGN /* SectionAlignment */
> + .long PECOFF_FILE_ALIGN /* FileAlignment */
> + .short 0 /* MajorOperatingSystemVersion */
> + .short 0 /* MinorOperatingSystemVersion */
> + .short LINUX_EFISTUB_MAJOR_VERSION /* MajorImageVersion */
> + .short LINUX_EFISTUB_MINOR_VERSION /* MinorImageVersion */
> + .short 0 /* MajorSubsystemVersion */
> + .short 0 /* MinorSubsystemVersion */
> + .long 0 /* Win32VersionValue */
> +
> + .long _end - _head /* SizeOfImage */
> +
> + /* Everything before the kernel image is considered part of the
> header */
> + .long .Lefi_header_end - _head /* SizeOfHeaders */
> + .long 0 /* CheckSum */
> + .short IMAGE_SUBSYSTEM_EFI_APPLICATION /* Subsystem */
> + .short 0 /* DllCharacteristics */
> + .quad 0 /* SizeOfStackReserve */
> + .quad 0 /* SizeOfStackCommit */
> + .quad 0 /* SizeOfHeapReserve */
> + .quad 0 /* SizeOfHeapCommit */
> + .long 0 /* LoaderFlags */
> + .long (.Lsection_table - .) / 8 /* NumberOfRvaAndSizes */
> +
> + .quad 0 /* ExportTable */
> + .quad 0 /* ImportTable */
> + .quad 0 /* ResourceTable */
> + .quad 0 /* ExceptionTable */
> + .quad 0 /* CertificationTable */
> + .quad 0 /* BaseRelocationTable */
> +
> + /* Section table */
> +.Lsection_table:
> + .ascii ".text\0\0\0"
> + .long __inittext_end - .Lefi_header_end /* VirtualSize */
> + .long .Lefi_header_end - _head /* VirtualAddress */
> + .long __inittext_end - .Lefi_header_end /* SizeOfRawData */
> + .long .Lefi_header_end - _head /* PointerToRawData */
> +
> + .long 0 /* PointerToRelocations */
> + .long 0 /* PointerToLineNumbers */
> + .short 0 /* NumberOfRelocations */
> + .short 0 /* NumberOfLineNumbers */
> + .long IMAGE_SCN_CNT_CODE | \
> + IMAGE_SCN_MEM_READ | \
> + IMAGE_SCN_MEM_EXECUTE /* Characteristics */
> +
> + .ascii ".data\0\0\0"
> + .long _end - __initdata_begin /* VirtualSize */
> + .long __initdata_begin - _head /* VirtualAddress */
> + .long _edata - __initdata_begin /* SizeOfRawData */
> + .long __initdata_begin - _head /* PointerToRawData */
> +
> + .long 0 /* PointerToRelocations */
> + .long 0 /* PointerToLineNumbers */
> + .short 0 /* NumberOfRelocations */
> + .short 0 /* NumberOfLineNumbers */
> + .long IMAGE_SCN_CNT_INITIALIZED_DATA | \
> + IMAGE_SCN_MEM_READ | \
> + IMAGE_SCN_MEM_WRITE /* Characteristics */
> +
> + .org 0x20e
> + .word kernel_version - 512 - _head
> +
> + .set .Lsection_count, (. - .Lsection_table) / 40
> +.Lefi_header_end:
> + .endm
> diff --git a/arch/loongarch/kernel/efi.c b/arch/loongarch/kernel/efi.c
> new file mode 100644
> index 000000000000..d36a325a990c
> --- /dev/null
> +++ b/arch/loongarch/kernel/efi.c
> @@ -0,0 +1,226 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * EFI initialization
> + *
> + * Author: Jianmin Lv <[email protected]>
> + * Huacai Chen <[email protected]>
> + *
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +
> +#include <linux/acpi.h>
> +#include <linux/efi.h>
> +#include <linux/efi-bgrt.h>
> +#include <linux/init.h>
> +#include <linux/kernel.h>
> +#include <linux/export.h>
> +#include <linux/io.h>
> +#include <linux/kobject.h>
> +#include <linux/memblock.h>
> +#include <linux/reboot.h>
> +#include <linux/uaccess.h>
> +
> +#include <asm/early_ioremap.h>
> +#include <asm/efi.h>
> +#include <asm/tlb.h>
> +#include <asm/loongson.h>
> +
> +static unsigned long efi_nr_tables;
> +static unsigned long efi_config_table;
> +static unsigned long screen_info_table __initdata =
> EFI_INVALID_TABLE_ADDR;
> +
> +static efi_system_table_t *efi_systab;
> +static efi_config_table_type_t arch_tables[] __initdata = {
> + {LINUX_EFI_LARCH_SCREEN_INFO_TABLE_GUID, &screen_info_table, "SINFO"},
> + {},
> +};
> +
> +static void __init init_screen_info(void)
> +{
> + struct screen_info *si;
> +
> + if (screen_info_table == EFI_INVALID_TABLE_ADDR)
> + return;
> +
> + si = early_memremap_ro(screen_info_table, sizeof(*si));
> + if (!si) {
> + pr_err("Could not map screen_info config table\n");
> + return;
> + }
> + screen_info = *si;
> + early_memunmap(si, sizeof(*si));
> +
> + if (screen_info.orig_video_isVGA == VIDEO_TYPE_EFI)
> + memblock_reserve(screen_info.lfb_base, screen_info.lfb_size);
> +}
> +
> +static void __init create_tlb(u32 index, u64 vppn, u32 ps, u32 mat)
> +{
> + unsigned long tlblo0, tlblo1;
> +
> + write_csr_pagesize(ps);
> +
> + tlblo0 = vppn | CSR_TLBLO0_V | CSR_TLBLO0_WE |
> + CSR_TLBLO0_GLOBAL | (mat << CSR_TLBLO0_CCA_SHIFT);
> + tlblo1 = tlblo0 + (1 << ps);
> +
> + csr_write64(vppn, LOONGARCH_CSR_TLBEHI);
> + csr_write64(tlblo0, LOONGARCH_CSR_TLBELO0);
> + csr_write64(tlblo1, LOONGARCH_CSR_TLBELO1);
> + csr_xchg32(0, CSR_TLBIDX_EHINV, LOONGARCH_CSR_TLBIDX);
> + csr_xchg32(index, CSR_TLBIDX_IDX, LOONGARCH_CSR_TLBIDX);
> +
> + tlb_write_indexed();
> +}
> +
> +#define MTLB_ENTRY_INDEX 0x800
> +
> +/* Create VA == PA mapping as UEFI */
> +static void __init fix_efi_mapping(void)
> +{
> + unsigned int index = MTLB_ENTRY_INDEX;
> + unsigned int tlbnr = boot_cpu_data.tlbsizemtlb - 2;
> + unsigned long i, vppn;
> +
> + /* Low Memory, Cached */
> + create_tlb(index++, 0x00000000, PS_128M, 1);
> + /* MMIO Registers, Uncached */
> + create_tlb(index++, 0x10000000, PS_128M, 0);
> +
> + /* High Memory, Cached */
> + for (i = 0; i < tlbnr; i++) {
> + vppn = 0x80000000ULL + (i * SZ_2G);
> + create_tlb(index++, vppn, PS_1G, 1);
> + }
> +}
> +
> +/*
> + * set_virtual_map() - create a virtual mapping for the EFI memory map
> and call
> + * efi_set_virtual_address_map enter virtual for runtime service
> + *
> + * This function populates the virt_addr fields of all memory region
> descriptors
> + * in @memory_map whose EFI_MEMORY_RUNTIME attribute is set. Those
> descriptors
> + * are also copied to @runtime_map, and their total count is returned
> in @count.
> + */
> +static int __init set_virtual_map(void)
> +{
> + int count = 0;
> + unsigned int size;
> + unsigned long attr;
> + efi_status_t status;
> + efi_runtime_services_t *rt;
> + efi_set_virtual_address_map_t *svam;
> + efi_memory_desc_t *in, runtime_map[32];
> +
> + size = sizeof(efi_memory_desc_t);
> +
> + for_each_efi_memory_desc(in) {
> + attr = in->attribute;
> + if (!(attr & EFI_MEMORY_RUNTIME))
> + continue;
> +
> + if (attr & (EFI_MEMORY_WB | EFI_MEMORY_WT))
> + in->virt_addr = TO_CACHE(in->phys_addr);
> + else
> + in->virt_addr = TO_UNCACHE(in->phys_addr);
> +
> + memcpy(&runtime_map[count++], in, size);
> + }
> +
> + rt = early_memremap_ro((unsigned long)efi_systab->runtime,
> sizeof(*rt));
> +
> + /* Install the new virtual address map */
> + svam = rt->set_virtual_address_map;
> +
> + fix_efi_mapping();
> +
> + status = svam(size * count, size, efi.memmap.desc_version,
> + (efi_memory_desc_t *)TO_PHYS((unsigned long)runtime_map));
> +
> + local_flush_tlb_all();
> + write_csr_pagesize(PS_DEFAULT_SIZE);
> +
> + return 0;
> +}
> +
> +void __init efi_runtime_init(void)
> +{
> + efi_status_t status;
> +
> + if (!efi_enabled(EFI_BOOT))
> + return;
> +
> + if (!efi_systab->runtime)
> + return;
> +
> + status = set_virtual_map();
> + if (status < 0)
> + return;
> +
> + if (efi_runtime_disabled()) {
> + pr_info("EFI runtime services will be disabled.\n");
> + return;
> + }
> +
> + efi.runtime = (efi_runtime_services_t *)efi_systab->runtime;
> + efi.runtime_version = (unsigned int)efi.runtime->hdr.revision;
> +
> + efi_native_runtime_setup();
> + set_bit(EFI_RUNTIME_SERVICES, &efi.flags);
> +}
> +
> +void __init efi_init(void)
> +{
> + int size;
> + void *config_tables;
> +
> + if (!efi_system_table)
> + return;
> +
> + efi_systab = (efi_system_table_t
> *)early_memremap_ro(efi_system_table, sizeof(*efi_systab));
> + if (!efi_systab) {
> + pr_err("Can't find EFI system table.\n");
> + return;
> + }
> +
> + set_bit(EFI_64BIT, &efi.flags);
> + efi_nr_tables = efi_systab->nr_tables;
> + efi_config_table = (unsigned long)efi_systab->tables;
> +
> + size = sizeof(efi_config_table_t);
> + config_tables = early_memremap(efi_config_table, efi_nr_tables *
> size);
> + efi_config_parse_tables(config_tables, efi_systab->nr_tables,
> arch_tables);
> + early_memunmap(config_tables, efi_nr_tables * size);
> +
> + init_screen_info();
> +}
> +
> +static ssize_t boardinfo_show(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + return sprintf(buf,
> + "BIOS Information\n"
> + "Vendor\t\t\t: %s\n"
> + "Version\t\t\t: %s\n"
> + "ROM Size\t\t: %d KB\n"
> + "Release Date\t\t: %s\n\n"
> + "Board Information\n"
> + "Manufacturer\t\t: %s\n"
> + "Board Name\t\t: %s\n"
> + "Family\t\t\t: LOONGSON64\n\n",
> + b_info.bios_vendor, b_info.bios_version,
> + b_info.bios_size, b_info.bios_release_date,
> + b_info.board_vendor, b_info.board_name);
> +}
> +
> +static struct kobj_attribute boardinfo_attr = __ATTR(boardinfo, 0444,
> + boardinfo_show, NULL);
> +
> +static int __init boardinfo_init(void)
> +{
> + if (!efi_kobj)
> + return -EINVAL;
> +
> + return sysfs_create_file(efi_kobj, &boardinfo_attr.attr);
> +}
> +late_initcall(boardinfo_init);
> diff --git a/arch/loongarch/kernel/env.c b/arch/loongarch/kernel/env.c
> new file mode 100644
> index 000000000000..bf2593f1e536
> --- /dev/null
> +++ b/arch/loongarch/kernel/env.c
> @@ -0,0 +1,70 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Author: Huacai Chen <[email protected]>
> + *
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#include <linux/acpi.h>
> +#include <linux/efi.h>
> +#include <linux/export.h>
> +#include <linux/memblock.h>
> +#include <linux/of_fdt.h>
> +#include <asm/early_ioremap.h>
> +#include <asm/bootinfo.h>
> +#include <asm/loongson.h>
> +
> +u64 efi_system_table;
> +struct loongson_system_configuration loongson_sysconf;
> +EXPORT_SYMBOL(loongson_sysconf);
> +
> +u64 loongson_chipcfg[MAX_PACKAGES];
> +u64 loongson_chiptemp[MAX_PACKAGES];
> +u64 loongson_freqctrl[MAX_PACKAGES];
> +unsigned long long smp_group[MAX_PACKAGES];
> +
> +static void __init register_addrs_set(u64 *registers, const u64 addr,
> int num)
> +{
> + u64 i;
> +
> + for (i = 0; i < num; i++) {
> + *registers = (i << 44) | addr;
> + registers++;
> + }
> +}
> +
> +void __init init_environ(void)
> +{
> + int efi_boot = fw_arg0;
> + struct efi_memory_map_data data;
> + void *fdt_ptr = early_memremap_ro(fw_arg1, SZ_64K);
> +
> + if (efi_boot)
> + set_bit(EFI_BOOT, &efi.flags);
> + else
> + clear_bit(EFI_BOOT, &efi.flags);
> +
> + early_init_dt_scan(fdt_ptr);
> + early_init_fdt_reserve_self();
> + efi_system_table = efi_get_fdt_params(&data);
> +
> + efi_memmap_init_early(&data);
> + memblock_reserve(data.phys_map & PAGE_MASK,
> + PAGE_ALIGN(data.size + (data.phys_map & ~PAGE_MASK)));
> +
> + register_addrs_set(smp_group, TO_UNCACHE(0x1fe01000), 16);
> + register_addrs_set(loongson_chipcfg, TO_UNCACHE(0x1fe00180), 16);
> + register_addrs_set(loongson_chiptemp, TO_UNCACHE(0x1fe0019c), 16);
> + register_addrs_set(loongson_freqctrl, TO_UNCACHE(0x1fe001d0), 16);
> +}
> +
> +static int __init init_cpu_fullname(void)
> +{
> + int cpu;
> +
> + if (loongson_sysconf.cpuname && !strncmp(loongson_sysconf.cpuname,
> "Loongson", 8)) {
> + for (cpu = 0; cpu < NR_CPUS; cpu++)
> + __cpu_full_name[cpu] = loongson_sysconf.cpuname;
> + }
> + return 0;
> +}
> +arch_initcall(init_cpu_fullname);
> diff --git a/arch/loongarch/kernel/head.S b/arch/loongarch/kernel/head.S
> new file mode 100644
> index 000000000000..4cb79e9c70f5
> --- /dev/null
> +++ b/arch/loongarch/kernel/head.S
> @@ -0,0 +1,99 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#include <linux/init.h>
> +#include <linux/threads.h>
> +
> +#include <asm/addrspace.h>
> +#include <asm/asm.h>
> +#include <asm/asmmacro.h>
> +#include <asm/regdef.h>
> +#include <asm/loongarch.h>
> +#include <asm/stackframe.h>
> +#include <generated/compile.h>
> +#include <generated/utsrelease.h>
> +
> +#ifdef CONFIG_EFI_STUB
> +
> +#include "efi-header.S"
> +
> + __HEAD
> +
> +_head:
> + .word MZ_MAGIC /* "MZ", MS-DOS header */
> + .org 0x38
> +#ifdef CONFIG_32BIT
> + .ascii "LA32" /* Magic number for BootLoader */
> +#else
> + .ascii "LA64" /* Magic number for BootLoader */
> +#endif
> + .org 0x3c
> + .long pe_header - _head /* Offset to the PE header */
> +
> +pe_header:
> + __EFI_PE_HEADER
> +
> +SYM_DATA(kernel_asize, .long _end - _text);
> +SYM_DATA(kernel_fsize, .long _edata - _text);
> +SYM_DATA(kernel_offset, .long kernel_offset - _text);
> +
> +kernel_version:
> + .ascii UTS_RELEASE " (" LINUX_COMPILE_BY "@" LINUX_COMPILE_HOST ") "
> UTS_VERSION "\0"
> +
> +#endif
> +
> + __REF
> +
> +SYM_ENTRY(_stext, SYM_L_GLOBAL, SYM_A_NONE)
> +
> +SYM_CODE_START(kernel_entry) # kernel entry point
> +
> + /* Config direct window and set PG */
> + li.d t0, CSR_DMW0_INIT # UC, PLV0, 0x8000 xxxx xxxx xxxx
> + csrwr t0, LOONGARCH_CSR_DMWIN0
> + li.d t0, CSR_DMW1_INIT # CA, PLV0, 0x9000 xxxx xxxx xxxx
> + csrwr t0, LOONGARCH_CSR_DMWIN1
> + /* Enable PG */
> + li.w t0, 0xb0 # PLV=0, IE=0, PG=1
> + csrwr t0, LOONGARCH_CSR_CRMD
> + li.w t0, 0x04 # PLV=0, PIE=1, PWE=0
> + csrwr t0, LOONGARCH_CSR_PRMD
> + li.w t0, 0x00 # FPE=0, SXE=0, ASXE=0, BTE=0
> + csrwr t0, LOONGARCH_CSR_EUEN
> +
> + /* We might not get launched at the address the kernel is linked to,
> + so we jump there. */
> + la.abs t0, 0f
> + jirl zero, t0, 0
> +0:
> + la t0, __bss_start # clear .bss
> + st.d zero, t0, 0
> + la t1, __bss_stop - LONGSIZE
> +1:
> + addi.d t0, t0, LONGSIZE
> + st.d zero, t0, 0
> + bne t0, t1, 1b
> +
> + la t0, fw_arg0
> + st.d a0, t0, 0 # firmware arguments
> + la t0, fw_arg1
> + st.d a1, t0, 0
> +
> + /* KSave3 used for percpu base, initialized as 0 */
> + csrwr zero, PERCPU_BASE_KS
> + /* GPR21 used for percpu base (runtime), initialized as 0 */
> + or u0, zero, zero
> +
> + la tp, init_thread_union
> + /* Set the SP after an empty pt_regs. */
> + PTR_LI sp, (_THREAD_SIZE - 32 - PT_SIZE)
> + PTR_ADD sp, sp, tp
> + set_saved_sp sp, t0, t1
> + PTR_ADDI sp, sp, -4 * SZREG # init stack pointer
> +
> + bl start_kernel
> +
> +SYM_CODE_END(kernel_entry)
> +
> +SYM_ENTRY(kernel_entry_end, SYM_L_GLOBAL, SYM_A_NONE)
> diff --git a/arch/loongarch/kernel/image-vars.h
> b/arch/loongarch/kernel/image-vars.h
> new file mode 100644
> index 000000000000..104e9f0e97fe
> --- /dev/null
> +++ b/arch/loongarch/kernel/image-vars.h
> @@ -0,0 +1,29 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef __LOONGARCH_KERNEL_IMAGE_VARS_H
> +#define __LOONGARCH_KERNEL_IMAGE_VARS_H
> +
> +#ifdef CONFIG_EFI_STUB
> +
> +__efistub_memcmp = memcmp;
> +__efistub_memchr = memchr;
> +__efistub_memcpy = memcpy;
> +__efistub_memmove = memmove;
> +__efistub_memset = memset;
> +__efistub_strcat = strcat;
> +__efistub_strcmp = strcmp;
> +__efistub_strlen = strlen;
> +__efistub_strncat = strncat;
> +__efistub_strnstr = strnstr;
> +__efistub_strnlen = strnlen;
> +__efistub_strrchr = strrchr;
> +__efistub_kernel_entry = kernel_entry;
> +__efistub_kernel_asize = kernel_asize;
> +__efistub_kernel_fsize = kernel_fsize;
> +__efistub_kernel_offset = kernel_offset;
> +
> +#endif
> +
> +#endif /* __LOONGARCH_KERNEL_IMAGE_VARS_H */
> diff --git a/arch/loongarch/kernel/mem.c b/arch/loongarch/kernel/mem.c
> new file mode 100644
> index 000000000000..7423361b0ebc
> --- /dev/null
> +++ b/arch/loongarch/kernel/mem.c
> @@ -0,0 +1,64 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#include <linux/efi.h>
> +#include <linux/initrd.h>
> +#include <linux/memblock.h>
> +
> +#include <asm/bootinfo.h>
> +#include <asm/loongson.h>
> +#include <asm/sections.h>
> +
> +void __init memblock_init(void)
> +{
> + u32 mem_type;
> + u64 mem_start, mem_end, mem_size;
> + efi_memory_desc_t *md;
> +
> + /* Parse memory information */
> + for_each_efi_memory_desc(md) {
> + mem_type = md->type;
> + mem_start = md->phys_addr;
> + mem_size = md->num_pages << EFI_PAGE_SHIFT;
> + mem_end = mem_start + mem_size;
> +
> + switch (mem_type) {
> + case EFI_LOADER_CODE:
> + case EFI_LOADER_DATA:
> + case EFI_BOOT_SERVICES_CODE:
> + case EFI_BOOT_SERVICES_DATA:
> + case EFI_PERSISTENT_MEMORY:
> + case EFI_CONVENTIONAL_MEMORY:
> + memblock_add(mem_start, mem_size);
> + if (max_low_pfn < (mem_end >> PAGE_SHIFT))
> + max_low_pfn = mem_end >> PAGE_SHIFT;
> + break;
> + case EFI_PAL_CODE:
> + case EFI_UNUSABLE_MEMORY:
> + case EFI_ACPI_RECLAIM_MEMORY:
> + memblock_add(mem_start, mem_size);
> + fallthrough;
> + case EFI_RESERVED_TYPE:
> + case EFI_RUNTIME_SERVICES_CODE:
> + case EFI_RUNTIME_SERVICES_DATA:
> + case EFI_MEMORY_MAPPED_IO:
> + case EFI_MEMORY_MAPPED_IO_PORT_SPACE:
> + memblock_reserve(mem_start, mem_size);
> + break;
> + }
> + }
> +
> + memblock_set_current_limit(PFN_PHYS(max_low_pfn));
> + memblock_set_node(0, PHYS_ADDR_MAX, &memblock.memory, 0);
> +
> + /* Reserve the first 2MB */
> + memblock_reserve(PHYS_OFFSET, 0x200000);
> +
> + /* Reserve the kernel text/data/bss */
> + memblock_reserve(__pa_symbol(&_text),
> + __pa_symbol(&_end) - __pa_symbol(&_text));
> +
> + /* Reserve the initrd */
> + reserve_initrd_mem();
> +}
> diff --git a/arch/loongarch/kernel/reset.c
> b/arch/loongarch/kernel/reset.c
> new file mode 100644
> index 000000000000..ef484ce43c5c
> --- /dev/null
> +++ b/arch/loongarch/kernel/reset.c
> @@ -0,0 +1,90 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#include <linux/kernel.h>
> +#include <linux/acpi.h>
> +#include <linux/efi.h>
> +#include <linux/export.h>
> +#include <linux/pm.h>
> +#include <linux/types.h>
> +#include <linux/reboot.h>
> +#include <linux/delay.h>
> +#include <linux/console.h>
> +
> +#include <acpi/reboot.h>
> +#include <asm/compiler.h>
> +#include <asm/idle.h>
> +#include <asm/loongarch.h>
> +#include <asm/reboot.h>
> +
> +static void default_halt(void)
> +{
> + local_irq_disable();
> + clear_csr_ecfg(ECFG0_IM);
> +
> + pr_notice("\n\n** You can safely turn off the power now **\n\n");
> + console_flush_on_panic(CONSOLE_FLUSH_PENDING);
> +
> + while (true) {
> + __arch_cpu_idle();
> + }
> +}
> +
> +static void default_poweroff(void)
> +{
> +#ifdef CONFIG_EFI
> + efi.reset_system(EFI_RESET_SHUTDOWN, EFI_SUCCESS, 0, NULL);
> +#endif
> + while (true) {
> + __arch_cpu_idle();
> + }
> +}
> +
> +static void default_restart(void)
> +{
> +#ifdef CONFIG_EFI
> + if (efi_capsule_pending(NULL))
> + efi_reboot(REBOOT_WARM, NULL);
> + else
> + efi_reboot(REBOOT_COLD, NULL);
> +#endif
> + if (!acpi_disabled)
> + acpi_reboot();
> +
> + while (true) {
> + __arch_cpu_idle();
> + }
> +}
> +
> +void (*pm_restart)(void);
> +EXPORT_SYMBOL(pm_restart);
> +
> +void (*pm_power_off)(void);
> +EXPORT_SYMBOL(pm_power_off);
> +
> +void machine_halt(void)
> +{
> + default_halt();
> +}
> +
> +void machine_power_off(void)
> +{
> + pm_power_off();
> +}
> +
> +void machine_restart(char *command)
> +{
> + do_kernel_restart(command);
> + pm_restart();
> +}
> +
> +static int __init loongarch_reboot_setup(void)
> +{
> + pm_restart = default_restart;
> + pm_power_off = default_poweroff;
> +
> + return 0;
> +}
> +
> +arch_initcall(loongarch_reboot_setup);
> diff --git a/arch/loongarch/kernel/setup.c
> b/arch/loongarch/kernel/setup.c
> new file mode 100644
> index 000000000000..29f3b82cd0a5
> --- /dev/null
> +++ b/arch/loongarch/kernel/setup.c
> @@ -0,0 +1,341 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + *
> + * Derived from MIPS:
> + * Copyright (C) 1995 Linus Torvalds
> + * Copyright (C) 1995 Waldorf Electronics
> + * Copyright (C) 1994, 95, 96, 97, 98, 99, 2000, 01, 02, 03 Ralf
> Baechle
> + * Copyright (C) 1996 Stoned Elipot
> + * Copyright (C) 1999 Silicon Graphics, Inc.
> + * Copyright (C) 2000, 2001, 2002, 2007 Maciej W. Rozycki
> + */
> +#include <linux/init.h>
> +#include <linux/acpi.h>
> +#include <linux/dmi.h>
> +#include <linux/efi.h>
> +#include <linux/export.h>
> +#include <linux/screen_info.h>
> +#include <linux/memblock.h>
> +#include <linux/initrd.h>
> +#include <linux/ioport.h>
> +#include <linux/root_dev.h>
> +#include <linux/console.h>
> +#include <linux/pfn.h>
> +#include <linux/platform_device.h>
> +#include <linux/sizes.h>
> +#include <linux/device.h>
> +#include <linux/dma-map-ops.h>
> +#include <linux/swiotlb.h>
> +
> +#include <asm/addrspace.h>
> +#include <asm/bootinfo.h>
> +#include <asm/cache.h>
> +#include <asm/cpu.h>
> +#include <asm/dma.h>
> +#include <asm/efi.h>
> +#include <asm/loongson.h>
> +#include <asm/pgalloc.h>
> +#include <asm/sections.h>
> +#include <asm/setup.h>
> +#include <asm/time.h>
> +
> +#define SMBIOS_BIOSSIZE_OFFSET 0x09
> +#define SMBIOS_BIOSEXTERN_OFFSET 0x13
> +#define SMBIOS_FREQLOW_OFFSET 0x16
> +#define SMBIOS_FREQHIGH_OFFSET 0x17
> +#define SMBIOS_FREQLOW_MASK 0xFF
> +#define SMBIOS_CORE_PACKAGE_OFFSET 0x23
> +#define LOONGSON_EFI_ENABLE (1 << 3)
> +
> +#ifdef CONFIG_VT
> +struct screen_info screen_info;
> +#endif
> +
> +unsigned long fw_arg0, fw_arg1;
> +DEFINE_PER_CPU(unsigned long, kernelsp);
> +struct cpuinfo_loongarch cpu_data[NR_CPUS] __read_mostly;
> +
> +EXPORT_SYMBOL(cpu_data);
> +
> +struct loongson_board_info b_info;
> +static const char dmi_empty_string[] = " ";
> +
> +/*
> + * Setup information
> + *
> + * These are initialized so they are in the .data section
> + */
> +
> +static int num_standard_resources;
> +static struct resource *standard_resources;
> +
> +static struct resource code_resource = { .name = "Kernel code", };
> +static struct resource data_resource = { .name = "Kernel data", };
> +static struct resource bss_resource = { .name = "Kernel bss", };
> +
> +const char *get_system_type(void)
> +{
> + return "generic-loongson-machine";
> +}
> +
> +static const char *dmi_string_parse(const struct dmi_header *dm, u8 s)
> +{
> + const u8 *bp = ((u8 *) dm) + dm->length;
> +
> + if (s) {
> + s--;
> + while (s > 0 && *bp) {
> + bp += strlen(bp) + 1;
> + s--;
> + }
> +
> + if (*bp != 0) {
> + size_t len = strlen(bp)+1;
> + size_t cmp_len = len > 8 ? 8 : len;
> +
> + if (!memcmp(bp, dmi_empty_string, cmp_len))
> + return dmi_empty_string;
> +
> + return bp;
> + }
> + }
> +
> + return "";
> +}
> +
> +static void __init parse_cpu_table(const struct dmi_header *dm)
> +{
> + long freq_temp = 0;
> + char *dmi_data = (char *)dm;
> +
> + freq_temp = ((*(dmi_data + SMBIOS_FREQHIGH_OFFSET) << 8) +
> + ((*(dmi_data + SMBIOS_FREQLOW_OFFSET)) & SMBIOS_FREQLOW_MASK));
> + cpu_clock_freq = freq_temp * 1000000;
> +
> + loongson_sysconf.cpuname = (void *)dmi_string_parse(dm, dmi_data[16]);
> + loongson_sysconf.cores_per_package = *(dmi_data +
> SMBIOS_CORE_PACKAGE_OFFSET);
> +
> + pr_info("CpuClock = %llu\n", cpu_clock_freq);
> +}
> +
> +static void __init parse_bios_table(const struct dmi_header *dm)
> +{
> + int bios_extern;
> + char *dmi_data = (char *)dm;
> +
> + bios_extern = *(dmi_data + SMBIOS_BIOSEXTERN_OFFSET);
> + b_info.bios_size = *(dmi_data + SMBIOS_BIOSSIZE_OFFSET);
> +
> + if (bios_extern & LOONGSON_EFI_ENABLE)
> + set_bit(EFI_BOOT, &efi.flags);
> + else
> + clear_bit(EFI_BOOT, &efi.flags);
> +}
> +
> +static void __init find_tokens(const struct dmi_header *dm, void
> *dummy)
> +{
> + switch (dm->type) {
> + case 0x0: /* Extern BIOS */
> + parse_bios_table(dm);
> + break;
> + case 0x4: /* Calling interface */
> + parse_cpu_table(dm);
> + break;
> + }
> +}
> +static void __init smbios_parse(void)
> +{
> + b_info.bios_vendor = (void *)dmi_get_system_info(DMI_BIOS_VENDOR);
> + b_info.bios_version = (void *)dmi_get_system_info(DMI_BIOS_VERSION);
> + b_info.bios_release_date = (void *)dmi_get_system_info(DMI_BIOS_DATE);
> + b_info.board_vendor = (void *)dmi_get_system_info(DMI_BOARD_VENDOR);
> + b_info.board_name = (void *)dmi_get_system_info(DMI_BOARD_NAME);
> + dmi_walk(find_tokens, NULL);
> +}
> +
> +static int usermem __initdata;
> +
> +static int __init early_parse_mem(char *p)
> +{
> + phys_addr_t start, size;
> +
> + if (!p) {
> + pr_err("mem parameter is empty, do nothing\n");
> + return -EINVAL;
> + }
> +
> + /*
> + * If a user specifies memory size, we
> + * blow away any automatically generated
> + * size.
> + */
> + if (usermem == 0) {
> + usermem = 1;
> + memblock_remove(memblock_start_of_DRAM(),
> + memblock_end_of_DRAM() - memblock_start_of_DRAM());
> + }
> + start = 0;
> + size = memparse(p, &p);
> + if (*p == '@')
> + start = memparse(p + 1, &p);
> + else {
> + pr_err("Invalid format!\n");
> + return -EINVAL;
> + }
> +
> + memblock_add(start, size);
> +
> + return 0;
> +}
> +early_param("mem", early_parse_mem);
> +
> +void __init platform_init(void)
> +{
> + efi_init();
> +#ifdef CONFIG_ACPI_TABLE_UPGRADE
> + acpi_table_upgrade();
> +#endif
> +#ifdef CONFIG_ACPI
> + acpi_gbl_use_default_register_widths = false;
> + acpi_boot_table_init();
> + acpi_boot_init();
> +#endif
> +
> + dmi_setup();
> + smbios_parse();
> + pr_info("The BIOS Version: %s\n", b_info.bios_version);
> +
> + efi_runtime_init();
> +}
> +
> +static void __init check_kernel_sections_mem(void)
> +{
> + phys_addr_t start = __pa_symbol(&_text);
> + phys_addr_t size = __pa_symbol(&_end) - start;
> +
> + if (!memblock_is_region_memory(start, size)) {
> + pr_info("Kernel sections are not in the memory maps\n");
> + memblock_add(start, size);
> + }
> +}
> +
> +/*
> + * arch_mem_init - initialize memory management subsystem
> + */
> +static void __init arch_mem_init(char **cmdline_p)
> +{
> + if (usermem)
> + pr_info("User-defined physical RAM map overwrite\n");
> +
> + check_kernel_sections_mem();
> +
> + /*
> + * In order to reduce the possibility of kernel panic when failed to
> + * get IO TLB memory under CONFIG_SWIOTLB, it is better to allocate
> + * low memory as small as possible before plat_swiotlb_setup(), so
> + * make sparse_init() using top-down allocation.
> + */
> + memblock_set_bottom_up(false);
> + sparse_init();
> + memblock_set_bottom_up(true);
> +
> + swiotlb_init(true, SWIOTLB_VERBOSE);
> +
> + dma_contiguous_reserve(PFN_PHYS(max_low_pfn));
> +
> + memblock_dump_all();
> +
> + early_memtest(PFN_PHYS(ARCH_PFN_OFFSET), PFN_PHYS(max_low_pfn));
> +}
> +
> +static void __init resource_init(void)
> +{
> + long i = 0;
> + size_t res_size;
> + struct resource *res;
> + struct memblock_region *region;
> +
> + code_resource.start = __pa_symbol(&_text);
> + code_resource.end = __pa_symbol(&_etext) - 1;
> + data_resource.start = __pa_symbol(&_etext);
> + data_resource.end = __pa_symbol(&_edata) - 1;
> + bss_resource.start = __pa_symbol(&__bss_start);
> + bss_resource.end = __pa_symbol(&__bss_stop) - 1;
> +
> + num_standard_resources = memblock.memory.cnt;
> + res_size = num_standard_resources * sizeof(*standard_resources);
> + standard_resources = memblock_alloc(res_size, SMP_CACHE_BYTES);
> +
> + for_each_mem_region(region) {
> + res = &standard_resources[i++];
> + if (!memblock_is_nomap(region)) {
> + res->name = "System RAM";
> + res->flags = IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY;
> + res->start = __pfn_to_phys(memblock_region_memory_base_pfn(region));
> + res->end = __pfn_to_phys(memblock_region_memory_end_pfn(region)) -
> 1;
> + } else {
> + res->name = "Reserved";
> + res->flags = IORESOURCE_MEM;
> + res->start =
> __pfn_to_phys(memblock_region_reserved_base_pfn(region));
> + res->end = __pfn_to_phys(memblock_region_reserved_end_pfn(region))
> - 1;
> + }
> +
> + request_resource(&iomem_resource, res);
> +
> + /*
> + * We don't know which RAM region contains kernel data,
> + * so we try it repeatedly and let the resource manager
> + * test it.
> + */
> + request_resource(res, &code_resource);
> + request_resource(res, &data_resource);
> + request_resource(res, &bss_resource);
> + }
> +}
> +
> +static int __init reserve_memblock_reserved_regions(void)
> +{
> + u64 i, j;
> +
> + for (i = 0; i < num_standard_resources; ++i) {
> + struct resource *mem = &standard_resources[i];
> + phys_addr_t r_start, r_end, mem_size = resource_size(mem);
> +
> + if (!memblock_is_region_reserved(mem->start, mem_size))
> + continue;
> +
> + for_each_reserved_mem_range(j, &r_start, &r_end) {
> + resource_size_t start, end;
> +
> + start = max(PFN_PHYS(PFN_DOWN(r_start)), mem->start);
> + end = min(PFN_PHYS(PFN_UP(r_end)) - 1, mem->end);
> +
> + if (start > mem->end || end < mem->start)
> + continue;
> +
> + reserve_region_with_split(mem, start, end, "Reserved");
> + }
> + }
> +
> + return 0;
> +}
> +arch_initcall(reserve_memblock_reserved_regions);
> +
> +void __init setup_arch(char **cmdline_p)
> +{
> + cpu_probe();
> + *cmdline_p = boot_command_line;
> +
> + init_environ();
> + memblock_init();
> + parse_early_param();
> +
> + platform_init();
> + pagetable_init();
> + arch_mem_init(cmdline_p);
> +
> + resource_init();
> +
> + paging_init();
> +}
> diff --git a/arch/loongarch/kernel/time.c b/arch/loongarch/kernel/time.c
> new file mode 100644
> index 000000000000..b2bb14844f01
> --- /dev/null
> +++ b/arch/loongarch/kernel/time.c
> @@ -0,0 +1,213 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Common time service routines for LoongArch machines.
> + *
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#include <linux/clockchips.h>
> +#include <linux/delay.h>
> +#include <linux/export.h>
> +#include <linux/init.h>
> +#include <linux/interrupt.h>
> +#include <linux/kernel.h>
> +#include <linux/sched_clock.h>
> +#include <linux/spinlock.h>
> +
> +#include <asm/cpu-features.h>
> +#include <asm/loongarch.h>
> +#include <asm/time.h>
> +
> +u64 cpu_clock_freq;
> +EXPORT_SYMBOL(cpu_clock_freq);
> +u64 const_clock_freq;
> +EXPORT_SYMBOL(const_clock_freq);
> +
> +static DEFINE_RAW_SPINLOCK(state_lock);
> +static DEFINE_PER_CPU(struct clock_event_device,
> constant_clockevent_device);
> +
> +static void constant_event_handler(struct clock_event_device *dev)
> +{
> +}
> +
> +irqreturn_t constant_timer_interrupt(int irq, void *data)
> +{
> + int cpu = smp_processor_id();
> + struct clock_event_device *cd;
> +
> + /* Clear Timer Interrupt */
> + write_csr_tintclear(CSR_TINTCLR_TI);
> + cd = &per_cpu(constant_clockevent_device, cpu);
> + cd->event_handler(cd);
> +
> + return IRQ_HANDLED;
> +}
> +
> +static int constant_set_state_oneshot(struct clock_event_device *evt)
> +{
> + unsigned long timer_config;
> +
> + raw_spin_lock(&state_lock);
> +
> + timer_config = csr_read64(LOONGARCH_CSR_TCFG);
> + timer_config |= CSR_TCFG_EN;
> + timer_config &= ~CSR_TCFG_PERIOD;
> + csr_write64(timer_config, LOONGARCH_CSR_TCFG);
> +
> + raw_spin_unlock(&state_lock);
> +
> + return 0;
> +}
> +
> +static int constant_set_state_oneshot_stopped(struct
> clock_event_device *evt)
> +{
> + unsigned long timer_config;
> +
> + raw_spin_lock(&state_lock);
> +
> + timer_config = csr_read64(LOONGARCH_CSR_TCFG);
> + timer_config &= ~CSR_TCFG_EN;
> + csr_write64(timer_config, LOONGARCH_CSR_TCFG);
> +
> + raw_spin_unlock(&state_lock);
> +
> + return 0;
> +}
> +
> +static int constant_set_state_periodic(struct clock_event_device *evt)
> +{
> + unsigned long period;
> + unsigned long timer_config;
> +
> + raw_spin_lock(&state_lock);
> +
> + period = const_clock_freq / HZ;
> + timer_config = period & CSR_TCFG_VAL;
> + timer_config |= (CSR_TCFG_PERIOD | CSR_TCFG_EN);
> + csr_write64(timer_config, LOONGARCH_CSR_TCFG);
> +
> + raw_spin_unlock(&state_lock);
> +
> + return 0;
> +}
> +
> +static int constant_set_state_shutdown(struct clock_event_device *evt)
> +{
> + return 0;
> +}
> +
> +static int constant_timer_next_event(unsigned long delta, struct
> clock_event_device *evt)
> +{
> + unsigned long timer_config;
> +
> + delta &= CSR_TCFG_VAL;
> + timer_config = delta | CSR_TCFG_EN;
> + csr_write64(timer_config, LOONGARCH_CSR_TCFG);
> +
> + return 0;
> +}
> +
> +static unsigned long __init get_loops_per_jiffy(void)
> +{
> + unsigned long lpj = (unsigned long)const_clock_freq;
> +
> + do_div(lpj, HZ);
> +
> + return lpj;
> +}
> +
> +static long init_timeval;
> +
> +void sync_counter(void)
> +{
> + /* Ensure counter begin at 0 */
> + csr_write64(-init_timeval, LOONGARCH_CSR_CNTC);
> +}
> +
> +int constant_clockevent_init(void)
> +{
> + unsigned int irq;
> + unsigned int cpu = smp_processor_id();
> + unsigned long min_delta = 0x600;
> + unsigned long max_delta = (1UL << 48) - 1;
> + struct clock_event_device *cd;
> + static int timer_irq_installed = 0;
> +
> + irq = EXCCODE_TIMER - EXCCODE_INT_START;
> +
> + cd = &per_cpu(constant_clockevent_device, cpu);
> +
> + cd->name = "Constant";
> + cd->features = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_PERIODIC |
> CLOCK_EVT_FEAT_PERCPU;
> +
> + cd->irq = irq;
> + cd->rating = 320;
> + cd->cpumask = cpumask_of(cpu);
> + cd->set_state_oneshot = constant_set_state_oneshot;
> + cd->set_state_oneshot_stopped = constant_set_state_oneshot_stopped;
> + cd->set_state_periodic = constant_set_state_periodic;
> + cd->set_state_shutdown = constant_set_state_shutdown;
> + cd->set_next_event = constant_timer_next_event;
> + cd->event_handler = constant_event_handler;
> +
> + clockevents_config_and_register(cd, const_clock_freq, min_delta,
> max_delta);
> +
> + if (timer_irq_installed)
> + return 0;
> +
> + timer_irq_installed = 1;
> +
> + sync_counter();
> +
> + if (request_irq(irq, constant_timer_interrupt, IRQF_PERCPU |
> IRQF_TIMER, "timer", NULL))
> + pr_err("Failed to request irq %d (timer)\n", irq);
> +
> + lpj_fine = get_loops_per_jiffy();
> + pr_info("Constant clock event device register\n");
> +
> + return 0;
> +}
> +
> +static u64 read_const_counter(struct clocksource *clk)
> +{
> + return drdtime();
> +}
> +
> +static u64 native_sched_clock(void)
> +{
> + return read_const_counter(NULL);
> +}
> +
> +static struct clocksource clocksource_const = {
> + .name = "Constant",
> + .rating = 400,
> + .read = read_const_counter,
> + .mask = CLOCKSOURCE_MASK(64),
> + .flags = CLOCK_SOURCE_IS_CONTINUOUS,
> +};
> +
> +int __init constant_clocksource_init(void)
> +{
> + int res;
> + unsigned long freq = const_clock_freq;
> +
> + res = clocksource_register_hz(&clocksource_const, freq);
> +
> + sched_clock_register(native_sched_clock, 64, freq);
> +
> + pr_info("Constant clock source device register\n");
> +
> + return res;
> +}
> +
> +void __init time_init(void)
> +{
> + if (!cpu_has_cpucfg)
> + const_clock_freq = cpu_clock_freq;
> + else
> + const_clock_freq = calc_const_freq();
> +
> + init_timeval = drdtime() - csr_read64(LOONGARCH_CSR_CNTC);
> +
> + constant_clockevent_init();
> + constant_clocksource_init();
> +}
> diff --git a/arch/loongarch/kernel/topology.c
> b/arch/loongarch/kernel/topology.c
> new file mode 100644
> index 000000000000..3b2cbb95875b
> --- /dev/null
> +++ b/arch/loongarch/kernel/topology.c
> @@ -0,0 +1,13 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <linux/cpu.h>
> +#include <linux/init.h>
> +#include <linux/percpu.h>
> +
> +static struct cpu cpu_device;
> +
> +static int __init topology_init(void)
> +{
> + return register_cpu(&cpu_device, 0);
> +}
> +
> +subsys_initcall(topology_init);
> diff --git a/drivers/firmware/efi/Kconfig b/drivers/firmware/efi/Kconfig
> index 4720ba98cec3..ba5d3d946f61 100644
> --- a/drivers/firmware/efi/Kconfig
> +++ b/drivers/firmware/efi/Kconfig
> @@ -118,7 +118,7 @@ config EFI_GENERIC_STUB
>
> config EFI_ARMSTUB_DTB_LOADER
> bool "Enable the DTB loader"
> - depends on EFI_GENERIC_STUB && !RISCV
> + depends on EFI_GENERIC_STUB && !RISCV && !LOONGARCH
> default y
> help
> Select this config option to add support for the dtb= command
> diff --git a/drivers/firmware/efi/libstub/Makefile
> b/drivers/firmware/efi/libstub/Makefile
> index d0537573501e..1588c61939e7 100644
> --- a/drivers/firmware/efi/libstub/Makefile
> +++ b/drivers/firmware/efi/libstub/Makefile
> @@ -26,6 +26,8 @@ cflags-$(CONFIG_ARM) := $(subst
> $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
> $(call cc-option,-mno-single-pic-base)
> cflags-$(CONFIG_RISCV) := $(subst
> $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
> -fpic
> +cflags-$(CONFIG_LOONGARCH) := $(subst
> $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
> + -fpic
>
> cflags-$(CONFIG_EFI_GENERIC_STUB) += -I$(srctree)/scripts/dtc/libfdt
>
> @@ -70,6 +72,8 @@ lib-$(CONFIG_ARM) += arm32-stub.o
> lib-$(CONFIG_ARM64) += arm64-stub.o
> lib-$(CONFIG_X86) += x86-stub.o
> lib-$(CONFIG_RISCV) += riscv-stub.o
> +lib-$(CONFIG_LOONGARCH) += loongarch-stub.o
> +
> CFLAGS_arm32-stub.o := -DTEXT_OFFSET=$(TEXT_OFFSET)
>
> # Even when -mbranch-protection=none is set, Clang will generate a
> @@ -125,6 +129,12 @@ STUBCOPY_FLAGS-$(CONFIG_RISCV) +=
> --prefix-alloc-sections=.init \
> --prefix-symbols=__efistub_
> STUBCOPY_RELOC-$(CONFIG_RISCV) := R_RISCV_HI20
>
> +# For LoongArch, keep all the symbols in .init section and make sure that no
> +# absolute symbols references doesn't exist.
> +STUBCOPY_FLAGS-$(CONFIG_LOONGARCH) += --prefix-alloc-sections=.init \
> + --prefix-symbols=__efistub_
> +STUBCOPY_RELOC-$(CONFIG_LOONGARCH) := R_LARCH_MARK_LA
> +
> $(obj)/%.stub.o: $(obj)/%.o FORCE
> $(call if_changed,stubcopy)
>
> diff --git a/drivers/firmware/efi/libstub/efi-stub-helper.c
> b/drivers/firmware/efi/libstub/efi-stub-helper.c
> index 3d972061c1b0..f612cfceda22 100644
> --- a/drivers/firmware/efi/libstub/efi-stub-helper.c
> +++ b/drivers/firmware/efi/libstub/efi-stub-helper.c
> @@ -21,7 +21,7 @@
> bool efi_nochunk;
> bool efi_nokaslr = !IS_ENABLED(CONFIG_RANDOMIZE_BASE);
> int efi_loglevel = CONSOLE_LOGLEVEL_DEFAULT;
> -bool efi_novamap;
> +bool efi_novamap = IS_ENABLED(CONFIG_LOONGARCH); /* LoongArch call
> svam() in kernel */
>
> static bool efi_noinitrd;
> static bool efi_nosoftreserve;
> diff --git a/drivers/firmware/efi/libstub/efi-stub.c
> b/drivers/firmware/efi/libstub/efi-stub.c
> index f515394cce6e..f228fce398cc 100644
> --- a/drivers/firmware/efi/libstub/efi-stub.c
> +++ b/drivers/firmware/efi/libstub/efi-stub.c
> @@ -40,7 +40,7 @@
>
> #ifdef CONFIG_ARM64
> # define EFI_RT_VIRTUAL_LIMIT DEFAULT_MAP_WINDOW_64
> -#elif defined(CONFIG_RISCV)
> +#elif defined(CONFIG_RISCV) || defined(CONFIG_LOONGARCH)
> # define EFI_RT_VIRTUAL_LIMIT TASK_SIZE_MIN
> #else
> # define EFI_RT_VIRTUAL_LIMIT TASK_SIZE
> diff --git a/drivers/firmware/efi/libstub/loongarch-stub.c
> b/drivers/firmware/efi/libstub/loongarch-stub.c
> new file mode 100644
> index 000000000000..beee086d9950
> --- /dev/null
> +++ b/drivers/firmware/efi/libstub/loongarch-stub.c
> @@ -0,0 +1,88 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Author: Yun Liu <[email protected]>
> + * Huacai Chen <[email protected]>
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +
> +#include <linux/efi.h>
> +#include <asm/efi.h>
> +#include <asm/addrspace.h>
> +#include "efistub.h"
> +
> +typedef void __noreturn (*kernel_entry_t)(bool efi, unsigned long fdt);
> +
> +extern int kernel_asize;
> +extern int kernel_fsize;
> +extern int kernel_offset;
> +extern kernel_entry_t kernel_entry;
> +
> +static efi_guid_t screen_info_guid =
> LINUX_EFI_LARCH_SCREEN_INFO_TABLE_GUID;
> +
> +struct screen_info *alloc_screen_info(void)
> +{
> + efi_status_t status;
> + struct screen_info *si;
> +
> + status = efi_bs_call(allocate_pool,
> + EFI_RUNTIME_SERVICES_DATA, sizeof(*si), (void **)&si);
> + if (status != EFI_SUCCESS)
> + return NULL;
> +
> + status = efi_bs_call(install_configuration_table, &screen_info_guid,
> si);
> + if (status == EFI_SUCCESS)
> + return si;
> +
> + efi_bs_call(free_pool, si);
> +
> + return NULL;
> +}
> +
> +void free_screen_info(struct screen_info *si)
> +{
> + if (!si)
> + return;
> +
> + efi_bs_call(install_configuration_table, &screen_info_guid, NULL);
> + efi_bs_call(free_pool, si);
> +}
> +
> +efi_status_t check_platform_features(void)
> +{
> + /* Config Direct Mapping */
> + csr_write64(CSR_DMW0_INIT, LOONGARCH_CSR_DMWIN0);
> + csr_write64(CSR_DMW1_INIT, LOONGARCH_CSR_DMWIN1);
> +
> + return EFI_SUCCESS;
> +}
> +
> +efi_status_t handle_kernel_image(unsigned long *image_addr,
> + unsigned long *image_size,
> + unsigned long *reserve_addr,
> + unsigned long *reserve_size,
> + efi_loaded_image_t *image,
> + efi_handle_t image_handle)
> +{
> + efi_status_t status;
> + unsigned long kernel_addr = 0;
> +
> + kernel_addr = (unsigned long)&kernel_offset - kernel_offset;
> +
> + status = efi_relocate_kernel(&kernel_addr, kernel_fsize, kernel_asize,
> + PHYSADDR(VMLINUX_LOAD_ADDRESS), SZ_2M, 0x0);
> +
> + *image_addr = kernel_addr;
> + *image_size = kernel_asize;
> +
> + return status;
> +}
> +
> +void __noreturn efi_enter_kernel(unsigned long entrypoint, unsigned
> long fdt, unsigned long fdt_size)
> +{
> + kernel_entry_t real_kernel_entry;
> +
> + real_kernel_entry = (kernel_entry_t)
> + ((unsigned long)&kernel_entry - entrypoint + VMLINUX_LOAD_ADDRESS);
> +
> + real_kernel_entry(true, fdt);
> +}
> diff --git a/include/linux/efi.h b/include/linux/efi.h
> index 7d9b0bb47eb3..adc43641ef8c 100644
> --- a/include/linux/efi.h
> +++ b/include/linux/efi.h
> @@ -401,6 +401,7 @@ void efi_native_runtime_setup(void);
> * associated with ConOut
> */
> #define LINUX_EFI_ARM_SCREEN_INFO_TABLE_GUID EFI_GUID(0xe03fc20a,
> 0x85dc, 0x406e, 0xb9, 0x0e, 0x4a, 0xb5, 0x02, 0x37, 0x1d, 0x95)
> +#define LINUX_EFI_LARCH_SCREEN_INFO_TABLE_GUID EFI_GUID(0x07fd51a6,
> 0x9532, 0x926f, 0x51, 0xdc, 0x6a, 0x63, 0x60, 0x2f, 0x84, 0xb4)
> #define LINUX_EFI_ARM_CPU_STATE_TABLE_GUID EFI_GUID(0xef79e4aa,
> 0x3c3d, 0x4989, 0xb9, 0x02, 0x07, 0xa9, 0x43, 0xe5, 0x50, 0xd2)
> #define LINUX_EFI_LOADER_ENTRY_GUID EFI_GUID(0x4a67b082, 0x0a4c,
> 0x41cf, 0xb6, 0xc7, 0x44, 0x0b, 0x29, 0xbb, 0x8c, 0x4f)
> #define LINUX_EFI_RANDOM_SEED_TABLE_GUID EFI_GUID(0x1ce1e5bc, 0x7ceb,
> 0x42f2, 0x81, 0xe5, 0x8a, 0xad, 0xf1, 0x80, 0xf5, 0x7b)
> diff --git a/include/linux/pe.h b/include/linux/pe.h
> index daf09ffffe38..1d3836ef9d92 100644
> --- a/include/linux/pe.h
> +++ b/include/linux/pe.h
> @@ -65,6 +65,8 @@
> #define IMAGE_FILE_MACHINE_SH5 0x01a8
> #define IMAGE_FILE_MACHINE_THUMB 0x01c2
> #define IMAGE_FILE_MACHINE_WCEMIPSV2 0x0169
> +#define IMAGE_FILE_MACHINE_LOONGARCH32 0x6232
> +#define IMAGE_FILE_MACHINE_LOONGARCH64 0x6264
>
> /* flags */
> #define IMAGE_FILE_RELOCS_STRIPPED 0x0001
> --
> 2.27.0
--
- Jiaxun
Add ELF-related definition and module relocation code for basic
LoongArch support.
Cc: Jessica Yu <[email protected]>
Reviewed-by: WANG Xuerui <[email protected]>
Reviewed-by: Luis Chamberlain <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/cpufeature.h | 24 ++
arch/loongarch/include/asm/elf.h | 301 ++++++++++++++++++
arch/loongarch/include/asm/exec.h | 10 +
arch/loongarch/include/asm/module.h | 80 +++++
arch/loongarch/include/asm/module.lds.h | 7 +
arch/loongarch/include/asm/vermagic.h | 19 ++
arch/loongarch/include/uapi/asm/auxvec.h | 17 ++
arch/loongarch/include/uapi/asm/hwcap.h | 20 ++
arch/loongarch/kernel/elf.c | 30 ++
arch/loongarch/kernel/inst.c | 40 +++
arch/loongarch/kernel/module-sections.c | 121 ++++++++
arch/loongarch/kernel/module.c | 374 +++++++++++++++++++++++
12 files changed, 1043 insertions(+)
create mode 100644 arch/loongarch/include/asm/cpufeature.h
create mode 100644 arch/loongarch/include/asm/elf.h
create mode 100644 arch/loongarch/include/asm/exec.h
create mode 100644 arch/loongarch/include/asm/module.h
create mode 100644 arch/loongarch/include/asm/module.lds.h
create mode 100644 arch/loongarch/include/asm/vermagic.h
create mode 100644 arch/loongarch/include/uapi/asm/auxvec.h
create mode 100644 arch/loongarch/include/uapi/asm/hwcap.h
create mode 100644 arch/loongarch/kernel/elf.c
create mode 100644 arch/loongarch/kernel/inst.c
create mode 100644 arch/loongarch/kernel/module-sections.c
create mode 100644 arch/loongarch/kernel/module.c
diff --git a/arch/loongarch/include/asm/cpufeature.h b/arch/loongarch/include/asm/cpufeature.h
new file mode 100644
index 000000000000..4da22a8e63de
--- /dev/null
+++ b/arch/loongarch/include/asm/cpufeature.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * CPU feature definitions for module loading, used by
+ * module_cpu_feature_match(), see uapi/asm/hwcap.h for LoongArch CPU features.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASM_CPUFEATURE_H
+#define __ASM_CPUFEATURE_H
+
+#include <uapi/asm/hwcap.h>
+#include <asm/elf.h>
+
+#define MAX_CPU_FEATURES (8 * sizeof(elf_hwcap))
+
+#define cpu_feature(x) ilog2(HWCAP_ ## x)
+
+static inline bool cpu_have_feature(unsigned int num)
+{
+ return elf_hwcap & (1UL << num);
+}
+
+#endif /* __ASM_CPUFEATURE_H */
diff --git a/arch/loongarch/include/asm/elf.h b/arch/loongarch/include/asm/elf.h
new file mode 100644
index 000000000000..f3960b18a90e
--- /dev/null
+++ b/arch/loongarch/include/asm/elf.h
@@ -0,0 +1,301 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_ELF_H
+#define _ASM_ELF_H
+
+#include <linux/auxvec.h>
+#include <linux/fs.h>
+#include <uapi/linux/elf.h>
+
+#include <asm/current.h>
+#include <asm/vdso.h>
+
+/* The ABI of a file. */
+#define EF_LOONGARCH_ABI_LP64_SOFT_FLOAT 0x1
+#define EF_LOONGARCH_ABI_LP64_SINGLE_FLOAT 0x2
+#define EF_LOONGARCH_ABI_LP64_DOUBLE_FLOAT 0x3
+
+#define EF_LOONGARCH_ABI_ILP32_SOFT_FLOAT 0x5
+#define EF_LOONGARCH_ABI_ILP32_SINGLE_FLOAT 0x6
+#define EF_LOONGARCH_ABI_ILP32_DOUBLE_FLOAT 0x7
+
+/* LoongArch relocation types used by the dynamic linker */
+#define R_LARCH_NONE 0
+#define R_LARCH_32 1
+#define R_LARCH_64 2
+#define R_LARCH_RELATIVE 3
+#define R_LARCH_COPY 4
+#define R_LARCH_JUMP_SLOT 5
+#define R_LARCH_TLS_DTPMOD32 6
+#define R_LARCH_TLS_DTPMOD64 7
+#define R_LARCH_TLS_DTPREL32 8
+#define R_LARCH_TLS_DTPREL64 9
+#define R_LARCH_TLS_TPREL32 10
+#define R_LARCH_TLS_TPREL64 11
+#define R_LARCH_IRELATIVE 12
+#define R_LARCH_MARK_LA 20
+#define R_LARCH_MARK_PCREL 21
+#define R_LARCH_SOP_PUSH_PCREL 22
+#define R_LARCH_SOP_PUSH_ABSOLUTE 23
+#define R_LARCH_SOP_PUSH_DUP 24
+#define R_LARCH_SOP_PUSH_GPREL 25
+#define R_LARCH_SOP_PUSH_TLS_TPREL 26
+#define R_LARCH_SOP_PUSH_TLS_GOT 27
+#define R_LARCH_SOP_PUSH_TLS_GD 28
+#define R_LARCH_SOP_PUSH_PLT_PCREL 29
+#define R_LARCH_SOP_ASSERT 30
+#define R_LARCH_SOP_NOT 31
+#define R_LARCH_SOP_SUB 32
+#define R_LARCH_SOP_SL 33
+#define R_LARCH_SOP_SR 34
+#define R_LARCH_SOP_ADD 35
+#define R_LARCH_SOP_AND 36
+#define R_LARCH_SOP_IF_ELSE 37
+#define R_LARCH_SOP_POP_32_S_10_5 38
+#define R_LARCH_SOP_POP_32_U_10_12 39
+#define R_LARCH_SOP_POP_32_S_10_12 40
+#define R_LARCH_SOP_POP_32_S_10_16 41
+#define R_LARCH_SOP_POP_32_S_10_16_S2 42
+#define R_LARCH_SOP_POP_32_S_5_20 43
+#define R_LARCH_SOP_POP_32_S_0_5_10_16_S2 44
+#define R_LARCH_SOP_POP_32_S_0_10_10_16_S2 45
+#define R_LARCH_SOP_POP_32_U 46
+#define R_LARCH_ADD8 47
+#define R_LARCH_ADD16 48
+#define R_LARCH_ADD24 49
+#define R_LARCH_ADD32 50
+#define R_LARCH_ADD64 51
+#define R_LARCH_SUB8 52
+#define R_LARCH_SUB16 53
+#define R_LARCH_SUB24 54
+#define R_LARCH_SUB32 55
+#define R_LARCH_SUB64 56
+#define R_LARCH_GNU_VTINHERIT 57
+#define R_LARCH_GNU_VTENTRY 58
+
+#ifndef ELF_ARCH
+
+/* ELF register definitions */
+
+/*
+ * General purpose have the following registers:
+ * Register Number
+ * GPRs 32
+ * ORIG_A0 1
+ * ERA 1
+ * BADVADDR 1
+ * CRMD 1
+ * PRMD 1
+ * EUEN 1
+ * ECFG 1
+ * ESTAT 1
+ * Reserved 5
+ */
+#define ELF_NGREG 45
+
+/*
+ * Floating point have the following registers:
+ * Register Number
+ * FPR 32
+ * FCC 1
+ * FCSR 1
+ */
+#define ELF_NFPREG 34
+
+typedef unsigned long elf_greg_t;
+typedef elf_greg_t elf_gregset_t[ELF_NGREG];
+
+typedef double elf_fpreg_t;
+typedef elf_fpreg_t elf_fpregset_t[ELF_NFPREG];
+
+void loongarch_dump_regs64(u64 *uregs, const struct pt_regs *regs);
+
+#ifdef CONFIG_32BIT
+/*
+ * This is used to ensure we don't load something for the wrong architecture.
+ */
+#define elf_check_arch elf32_check_arch
+
+/*
+ * These are used to set parameters in the core dumps.
+ */
+#define ELF_CLASS ELFCLASS32
+
+#define ELF_CORE_COPY_REGS(dest, regs) \
+ loongarch_dump_regs32((u32 *)&(dest), (regs));
+
+#endif /* CONFIG_32BIT */
+
+#ifdef CONFIG_64BIT
+/*
+ * This is used to ensure we don't load something for the wrong architecture.
+ */
+#define elf_check_arch elf64_check_arch
+
+/*
+ * These are used to set parameters in the core dumps.
+ */
+#define ELF_CLASS ELFCLASS64
+
+#define ELF_CORE_COPY_REGS(dest, regs) \
+ loongarch_dump_regs64((u64 *)&(dest), (regs));
+
+#endif /* CONFIG_64BIT */
+
+/*
+ * These are used to set parameters in the core dumps.
+ */
+#define ELF_DATA ELFDATA2LSB
+#define ELF_ARCH EM_LOONGARCH
+
+#endif /* !defined(ELF_ARCH) */
+
+#define loongarch_elf_check_machine(x) ((x)->e_machine == EM_LOONGARCH)
+
+#define vmcore_elf32_check_arch loongarch_elf_check_machine
+#define vmcore_elf64_check_arch loongarch_elf_check_machine
+
+/*
+ * Return non-zero if HDR identifies an 32bit ELF binary.
+ */
+#define elf32_check_arch(hdr) \
+({ \
+ int __res = 1; \
+ struct elfhdr *__h = (hdr); \
+ \
+ if (!loongarch_elf_check_machine(__h)) \
+ __res = 0; \
+ if (__h->e_ident[EI_CLASS] != ELFCLASS32) \
+ __res = 0; \
+ \
+ __res; \
+})
+
+/*
+ * Return non-zero if HDR identifies an 64bit ELF binary.
+ */
+#define elf64_check_arch(hdr) \
+({ \
+ int __res = 1; \
+ struct elfhdr *__h = (hdr); \
+ \
+ if (!loongarch_elf_check_machine(__h)) \
+ __res = 0; \
+ if (__h->e_ident[EI_CLASS] != ELFCLASS64) \
+ __res = 0; \
+ \
+ __res; \
+})
+
+#ifdef CONFIG_32BIT
+
+#define SET_PERSONALITY2(ex, state) \
+do { \
+ current->thread.vdso = &vdso_info; \
+ \
+ loongarch_set_personality_fcsr(state); \
+ \
+ if (personality(current->personality) != PER_LINUX) \
+ set_personality(PER_LINUX); \
+} while (0)
+
+#endif /* CONFIG_32BIT */
+
+#ifdef CONFIG_64BIT
+
+#define SET_PERSONALITY2(ex, state) \
+do { \
+ unsigned int p; \
+ \
+ clear_thread_flag(TIF_32BIT_REGS); \
+ clear_thread_flag(TIF_32BIT_ADDR); \
+ \
+ current->thread.vdso = &vdso_info; \
+ loongarch_set_personality_fcsr(state); \
+ \
+ p = personality(current->personality); \
+ if (p != PER_LINUX32 && p != PER_LINUX) \
+ set_personality(PER_LINUX); \
+} while (0)
+
+#endif /* CONFIG_64BIT */
+
+#define CORE_DUMP_USE_REGSET
+#define ELF_EXEC_PAGESIZE PAGE_SIZE
+
+/*
+ * This yields a mask that user programs can use to figure out what
+ * instruction set this cpu supports. This could be done in userspace,
+ * but it's not easy, and we've already done it here.
+ */
+
+#define ELF_HWCAP (elf_hwcap)
+extern unsigned int elf_hwcap;
+#include <asm/hwcap.h>
+
+/*
+ * This yields a string that ld.so will use to load implementation
+ * specific libraries for optimization. This is more specific in
+ * intent than poking at uname or /proc/cpuinfo.
+ */
+
+#define ELF_PLATFORM __elf_platform
+extern const char *__elf_platform;
+
+#define ELF_PLAT_INIT(_r, load_addr) do { \
+ _r->regs[1] = _r->regs[2] = _r->regs[3] = _r->regs[4] = 0; \
+ _r->regs[5] = _r->regs[6] = _r->regs[7] = _r->regs[8] = 0; \
+ _r->regs[9] = _r->regs[10] = _r->regs[11] = _r->regs[12] = 0; \
+ _r->regs[13] = _r->regs[14] = _r->regs[15] = _r->regs[16] = 0; \
+ _r->regs[17] = _r->regs[18] = _r->regs[19] = _r->regs[20] = 0; \
+ _r->regs[21] = _r->regs[22] = _r->regs[23] = _r->regs[24] = 0; \
+ _r->regs[25] = _r->regs[26] = _r->regs[27] = _r->regs[28] = 0; \
+ _r->regs[29] = _r->regs[30] = _r->regs[31] = 0; \
+} while (0)
+
+/*
+ * This is the location that an ET_DYN program is loaded if exec'ed. Typical
+ * use of this is to invoke "./ld.so someprog" to test out a new version of
+ * the loader. We need to make sure that it is out of the way of the program
+ * that it will "exec", and that there is sufficient room for the brk.
+ */
+
+#define ELF_ET_DYN_BASE (TASK_SIZE / 3 * 2)
+
+/* update AT_VECTOR_SIZE_ARCH if the number of NEW_AUX_ENT entries changes */
+#define ARCH_DLINFO \
+do { \
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, \
+ (unsigned long)current->mm->context.vdso); \
+} while (0)
+
+#define ARCH_HAS_SETUP_ADDITIONAL_PAGES 1
+struct linux_binprm;
+extern int arch_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
+
+struct arch_elf_state {
+ int fp_abi;
+ int interp_fp_abi;
+};
+
+#define LOONGARCH_ABI_FP_ANY (0)
+
+#define INIT_ARCH_ELF_STATE { \
+ .fp_abi = LOONGARCH_ABI_FP_ANY, \
+ .interp_fp_abi = LOONGARCH_ABI_FP_ANY, \
+}
+
+#define elf_read_implies_exec(ex, exec_stk) (exec_stk == EXSTACK_DEFAULT)
+
+extern int arch_elf_pt_proc(void *ehdr, void *phdr, struct file *elf,
+ bool is_interp, struct arch_elf_state *state);
+
+extern int arch_check_elf(void *ehdr, bool has_interpreter, void *interp_ehdr,
+ struct arch_elf_state *state);
+
+extern void loongarch_set_personality_fcsr(struct arch_elf_state *state);
+
+#endif /* _ASM_ELF_H */
diff --git a/arch/loongarch/include/asm/exec.h b/arch/loongarch/include/asm/exec.h
new file mode 100644
index 000000000000..ba0220812ebb
--- /dev/null
+++ b/arch/loongarch/include/asm/exec.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_EXEC_H
+#define _ASM_EXEC_H
+
+extern unsigned long arch_align_stack(unsigned long sp);
+
+#endif /* _ASM_EXEC_H */
diff --git a/arch/loongarch/include/asm/module.h b/arch/loongarch/include/asm/module.h
new file mode 100644
index 000000000000..9f6718df1854
--- /dev/null
+++ b/arch/loongarch/include/asm/module.h
@@ -0,0 +1,80 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_MODULE_H
+#define _ASM_MODULE_H
+
+#include <asm/inst.h>
+#include <asm-generic/module.h>
+
+#define RELA_STACK_DEPTH 16
+
+struct mod_section {
+ Elf_Shdr *shdr;
+ int num_entries;
+ int max_entries;
+};
+
+struct mod_arch_specific {
+ struct mod_section plt;
+ struct mod_section plt_idx;
+};
+
+struct plt_entry {
+ u32 inst_lu12iw;
+ u32 inst_lu32id;
+ u32 inst_lu52id;
+ u32 inst_jirl;
+};
+
+struct plt_idx_entry {
+ unsigned long symbol_addr;
+};
+
+Elf_Addr module_emit_plt_entry(struct module *mod, unsigned long val);
+
+static inline struct plt_entry emit_plt_entry(unsigned long val)
+{
+ u32 lu12iw, lu32id, lu52id, jirl;
+
+ lu12iw = (lu12iw_op << 25 | (((val >> 12) & 0xfffff) << 5) | LOONGARCH_GPR_T1);
+ lu32id = larch_insn_gen_lu32id(LOONGARCH_GPR_T1, ADDR_IMM(val, LU32ID));
+ lu52id = larch_insn_gen_lu52id(LOONGARCH_GPR_T1, LOONGARCH_GPR_T1, ADDR_IMM(val, LU52ID));
+ jirl = larch_insn_gen_jirl(0, LOONGARCH_GPR_T1, 0, (val & 0xfff));
+
+ return (struct plt_entry) { lu12iw, lu32id, lu52id, jirl };
+}
+
+static inline struct plt_idx_entry emit_plt_idx_entry(unsigned long val)
+{
+ return (struct plt_idx_entry) { val };
+}
+
+static inline int get_plt_idx(unsigned long val, const struct mod_section *sec)
+{
+ int i;
+ struct plt_idx_entry *plt_idx = (struct plt_idx_entry *)sec->shdr->sh_addr;
+
+ for (i = 0; i < sec->num_entries; i++) {
+ if (plt_idx[i].symbol_addr == val)
+ return i;
+ }
+
+ return -1;
+}
+
+static inline struct plt_entry *get_plt_entry(unsigned long val,
+ const struct mod_section *sec_plt,
+ const struct mod_section *sec_plt_idx)
+{
+ int plt_idx = get_plt_idx(val, sec_plt_idx);
+ struct plt_entry *plt = (struct plt_entry *)sec_plt->shdr->sh_addr;
+
+ if (plt_idx < 0)
+ return NULL;
+
+ return plt + plt_idx;
+}
+
+#endif /* _ASM_MODULE_H */
diff --git a/arch/loongarch/include/asm/module.lds.h b/arch/loongarch/include/asm/module.lds.h
new file mode 100644
index 000000000000..31c1c0db11a3
--- /dev/null
+++ b/arch/loongarch/include/asm/module.lds.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2020-2022 Loongson Technology Corporation Limited */
+SECTIONS {
+ . = ALIGN(4);
+ .plt : { BYTE(0) }
+ .plt.idx : { BYTE(0) }
+}
diff --git a/arch/loongarch/include/asm/vermagic.h b/arch/loongarch/include/asm/vermagic.h
new file mode 100644
index 000000000000..8b47ccfe3aad
--- /dev/null
+++ b/arch/loongarch/include/asm/vermagic.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_VERMAGIC_H
+#define _ASM_VERMAGIC_H
+
+#define MODULE_PROC_FAMILY "LOONGARCH "
+
+#ifdef CONFIG_32BIT
+#define MODULE_KERNEL_TYPE "32BIT "
+#elif defined CONFIG_64BIT
+#define MODULE_KERNEL_TYPE "64BIT "
+#endif
+
+#define MODULE_ARCH_VERMAGIC \
+ MODULE_PROC_FAMILY MODULE_KERNEL_TYPE
+
+#endif /* _ASM_VERMAGIC_H */
diff --git a/arch/loongarch/include/uapi/asm/auxvec.h b/arch/loongarch/include/uapi/asm/auxvec.h
new file mode 100644
index 000000000000..922d9e6b5058
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/auxvec.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
+/*
+ * Author: Hanlu Li <[email protected]>
+ * Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASM_AUXVEC_H
+#define __ASM_AUXVEC_H
+
+/* Location of VDSO image. */
+#define AT_SYSINFO_EHDR 33
+
+#define AT_VECTOR_SIZE_ARCH 1 /* entries in ARCH_DLINFO */
+
+#endif /* __ASM_AUXVEC_H */
diff --git a/arch/loongarch/include/uapi/asm/hwcap.h b/arch/loongarch/include/uapi/asm/hwcap.h
new file mode 100644
index 000000000000..8840b72fa8e8
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/hwcap.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _UAPI_ASM_HWCAP_H
+#define _UAPI_ASM_HWCAP_H
+
+/* HWCAP flags */
+#define HWCAP_LOONGARCH_CPUCFG (1 << 0)
+#define HWCAP_LOONGARCH_LAM (1 << 1)
+#define HWCAP_LOONGARCH_UAL (1 << 2)
+#define HWCAP_LOONGARCH_FPU (1 << 3)
+#define HWCAP_LOONGARCH_LSX (1 << 4)
+#define HWCAP_LOONGARCH_LASX (1 << 5)
+#define HWCAP_LOONGARCH_CRC32 (1 << 6)
+#define HWCAP_LOONGARCH_COMPLEX (1 << 7)
+#define HWCAP_LOONGARCH_CRYPTO (1 << 8)
+#define HWCAP_LOONGARCH_LVZ (1 << 9)
+#define HWCAP_LOONGARCH_LBT_X86 (1 << 10)
+#define HWCAP_LOONGARCH_LBT_ARM (1 << 11)
+#define HWCAP_LOONGARCH_LBT_MIPS (1 << 12)
+
+#endif /* _UAPI_ASM_HWCAP_H */
diff --git a/arch/loongarch/kernel/elf.c b/arch/loongarch/kernel/elf.c
new file mode 100644
index 000000000000..183e94fc9c69
--- /dev/null
+++ b/arch/loongarch/kernel/elf.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/binfmts.h>
+#include <linux/elf.h>
+#include <linux/export.h>
+#include <linux/sched.h>
+
+#include <asm/cpu-features.h>
+#include <asm/cpu-info.h>
+
+int arch_elf_pt_proc(void *_ehdr, void *_phdr, struct file *elf,
+ bool is_interp, struct arch_elf_state *state)
+{
+ return 0;
+}
+
+int arch_check_elf(void *_ehdr, bool has_interpreter, void *_interp_ehdr,
+ struct arch_elf_state *state)
+{
+ return 0;
+}
+
+void loongarch_set_personality_fcsr(struct arch_elf_state *state)
+{
+ current->thread.fpu.fcsr = boot_cpu_data.fpu_csr0;
+}
diff --git a/arch/loongarch/kernel/inst.c b/arch/loongarch/kernel/inst.c
new file mode 100644
index 000000000000..b1df0ec34bd1
--- /dev/null
+++ b/arch/loongarch/kernel/inst.c
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <asm/inst.h>
+
+u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm)
+{
+ union loongarch_instruction insn;
+
+ insn.reg1i20_format.opcode = lu32id_op;
+ insn.reg1i20_format.rd = rd;
+ insn.reg1i20_format.immediate = imm;
+
+ return insn.word;
+}
+
+u32 larch_insn_gen_lu52id(enum loongarch_gpr rd, enum loongarch_gpr rj, int imm)
+{
+ union loongarch_instruction insn;
+
+ insn.reg2i12_format.opcode = lu52id_op;
+ insn.reg2i12_format.rd = rd;
+ insn.reg2i12_format.rj = rj;
+ insn.reg2i12_format.immediate = imm;
+
+ return insn.word;
+}
+
+u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, unsigned long pc, unsigned long dest)
+{
+ union loongarch_instruction insn;
+
+ insn.reg2i16_format.opcode = jirl_op;
+ insn.reg2i16_format.rd = rd;
+ insn.reg2i16_format.rj = rj;
+ insn.reg2i16_format.immediate = (dest - pc) >> 2;
+
+ return insn.word;
+}
diff --git a/arch/loongarch/kernel/module-sections.c b/arch/loongarch/kernel/module-sections.c
new file mode 100644
index 000000000000..6d498288977d
--- /dev/null
+++ b/arch/loongarch/kernel/module-sections.c
@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/elf.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+Elf_Addr module_emit_plt_entry(struct module *mod, unsigned long val)
+{
+ int nr;
+ struct mod_section *plt_sec = &mod->arch.plt;
+ struct mod_section *plt_idx_sec = &mod->arch.plt_idx;
+ struct plt_entry *plt = get_plt_entry(val, plt_sec, plt_idx_sec);
+ struct plt_idx_entry *plt_idx;
+
+ if (plt)
+ return (Elf_Addr)plt;
+
+ nr = plt_sec->num_entries;
+
+ /* There is no duplicate entry, create a new one */
+ plt = (struct plt_entry *)plt_sec->shdr->sh_addr;
+ plt[nr] = emit_plt_entry(val);
+ plt_idx = (struct plt_idx_entry *)plt_idx_sec->shdr->sh_addr;
+ plt_idx[nr] = emit_plt_idx_entry(val);
+
+ plt_sec->num_entries++;
+ plt_idx_sec->num_entries++;
+ BUG_ON(plt_sec->num_entries > plt_sec->max_entries);
+
+ return (Elf_Addr)&plt[nr];
+}
+
+static int is_rela_equal(const Elf_Rela *x, const Elf_Rela *y)
+{
+ return x->r_info == y->r_info && x->r_addend == y->r_addend;
+}
+
+static bool duplicate_rela(const Elf_Rela *rela, int idx)
+{
+ int i;
+
+ for (i = 0; i < idx; i++) {
+ if (is_rela_equal(&rela[i], &rela[idx]))
+ return true;
+ }
+
+ return false;
+}
+
+static void count_max_entries(Elf_Rela *relas, int num, unsigned int *plts)
+{
+ unsigned int i, type;
+
+ for (i = 0; i < num; i++) {
+ type = ELF_R_TYPE(relas[i].r_info);
+ if (type == R_LARCH_SOP_PUSH_PLT_PCREL) {
+ if (!duplicate_rela(relas, i))
+ (*plts)++;
+ }
+ }
+}
+
+int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
+ char *secstrings, struct module *mod)
+{
+ unsigned int i, num_plts = 0;
+
+ /*
+ * Find the empty .plt sections.
+ */
+ for (i = 0; i < ehdr->e_shnum; i++) {
+ if (!strcmp(secstrings + sechdrs[i].sh_name, ".plt"))
+ mod->arch.plt.shdr = sechdrs + i;
+ else if (!strcmp(secstrings + sechdrs[i].sh_name, ".plt.idx"))
+ mod->arch.plt_idx.shdr = sechdrs + i;
+ }
+
+ if (!mod->arch.plt.shdr) {
+ pr_err("%s: module PLT section(s) missing\n", mod->name);
+ return -ENOEXEC;
+ }
+ if (!mod->arch.plt_idx.shdr) {
+ pr_err("%s: module PLT.IDX section(s) missing\n", mod->name);
+ return -ENOEXEC;
+ }
+
+ /* Calculate the maxinum number of entries */
+ for (i = 0; i < ehdr->e_shnum; i++) {
+ int num_rela = sechdrs[i].sh_size / sizeof(Elf_Rela);
+ Elf_Rela *relas = (void *)ehdr + sechdrs[i].sh_offset;
+ Elf_Shdr *dst_sec = sechdrs + sechdrs[i].sh_info;
+
+ if (sechdrs[i].sh_type != SHT_RELA)
+ continue;
+
+ /* ignore relocations that operate on non-exec sections */
+ if (!(dst_sec->sh_flags & SHF_EXECINSTR))
+ continue;
+
+ count_max_entries(relas, num_rela, &num_plts);
+ }
+
+ mod->arch.plt.shdr->sh_type = SHT_NOBITS;
+ mod->arch.plt.shdr->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
+ mod->arch.plt.shdr->sh_addralign = L1_CACHE_BYTES;
+ mod->arch.plt.shdr->sh_size = (num_plts + 1) * sizeof(struct plt_entry);
+ mod->arch.plt.num_entries = 0;
+ mod->arch.plt.max_entries = num_plts;
+
+ mod->arch.plt_idx.shdr->sh_type = SHT_NOBITS;
+ mod->arch.plt_idx.shdr->sh_flags = SHF_ALLOC;
+ mod->arch.plt_idx.shdr->sh_addralign = L1_CACHE_BYTES;
+ mod->arch.plt_idx.shdr->sh_size = (num_plts + 1) * sizeof(struct plt_idx_entry);
+ mod->arch.plt_idx.num_entries = 0;
+ mod->arch.plt_idx.max_entries = num_plts;
+
+ return 0;
+}
diff --git a/arch/loongarch/kernel/module.c b/arch/loongarch/kernel/module.c
new file mode 100644
index 000000000000..87b3768f1eef
--- /dev/null
+++ b/arch/loongarch/kernel/module.c
@@ -0,0 +1,374 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Author: Hanlu Li <[email protected]>
+ * Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#define pr_fmt(fmt) "kmod: " fmt
+
+#include <linux/moduleloader.h>
+#include <linux/elf.h>
+#include <linux/mm.h>
+#include <linux/vmalloc.h>
+#include <linux/slab.h>
+#include <linux/fs.h>
+#include <linux/string.h>
+#include <linux/kernel.h>
+
+static inline bool signed_imm_check(long val, unsigned int bit)
+{
+ return -(1L << (bit - 1)) <= val && val < (1L << (bit - 1));
+}
+
+static inline bool unsigned_imm_check(unsigned long val, unsigned int bit)
+{
+ return val < (1UL << bit);
+}
+
+static int rela_stack_push(s64 stack_value, s64 *rela_stack, size_t *rela_stack_top)
+{
+ if (*rela_stack_top >= RELA_STACK_DEPTH)
+ return -ENOEXEC;
+
+ rela_stack[(*rela_stack_top)++] = stack_value;
+ pr_debug("%s stack_value = 0x%llx\n", __func__, stack_value);
+
+ return 0;
+}
+
+static int rela_stack_pop(s64 *stack_value, s64 *rela_stack, size_t *rela_stack_top)
+{
+ if (*rela_stack_top == 0)
+ return -ENOEXEC;
+
+ *stack_value = rela_stack[--(*rela_stack_top)];
+ pr_debug("%s stack_value = 0x%llx\n", __func__, *stack_value);
+
+ return 0;
+}
+
+static int apply_r_larch_none(struct module *mod, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
+{
+ return 0;
+}
+
+static int apply_r_larch_error(struct module *me, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
+{
+ pr_err("%s: Unsupport relocation type %u, please add its support.\n", me->name, type);
+ return -EINVAL;
+}
+
+static int apply_r_larch_32(struct module *mod, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
+{
+ *location = v;
+ return 0;
+}
+
+static int apply_r_larch_64(struct module *mod, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
+{
+ *(Elf_Addr *)location = v;
+ return 0;
+}
+
+static int apply_r_larch_sop_push_pcrel(struct module *mod, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
+{
+ return rela_stack_push(v - (u64)location, rela_stack, rela_stack_top);
+}
+
+static int apply_r_larch_sop_push_absolute(struct module *mod, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
+{
+ return rela_stack_push(v, rela_stack, rela_stack_top);
+}
+
+static int apply_r_larch_sop_push_dup(struct module *mod, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
+{
+ int err = 0;
+ s64 opr1;
+
+ err = rela_stack_pop(&opr1, rela_stack, rela_stack_top);
+ if (err)
+ return err;
+ err = rela_stack_push(opr1, rela_stack, rela_stack_top);
+ if (err)
+ return err;
+ err = rela_stack_push(opr1, rela_stack, rela_stack_top);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int apply_r_larch_sop_push_plt_pcrel(struct module *mod, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
+{
+ ptrdiff_t offset = (void *)v - (void *)location;
+
+ if (offset >= SZ_128M)
+ v = module_emit_plt_entry(mod, v);
+
+ if (offset < -SZ_128M)
+ v = module_emit_plt_entry(mod, v);
+
+ return apply_r_larch_sop_push_pcrel(mod, location, v, rela_stack, rela_stack_top, type);
+}
+
+static int apply_r_larch_sop(struct module *mod, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
+{
+ int err = 0;
+ s64 opr1, opr2, opr3;
+
+ if (type == R_LARCH_SOP_IF_ELSE) {
+ err = rela_stack_pop(&opr3, rela_stack, rela_stack_top);
+ if (err)
+ return err;
+ }
+
+ err = rela_stack_pop(&opr2, rela_stack, rela_stack_top);
+ if (err)
+ return err;
+ err = rela_stack_pop(&opr1, rela_stack, rela_stack_top);
+ if (err)
+ return err;
+
+ switch (type) {
+ case R_LARCH_SOP_AND:
+ err = rela_stack_push(opr1 & opr2, rela_stack, rela_stack_top);
+ break;
+ case R_LARCH_SOP_ADD:
+ err = rela_stack_push(opr1 + opr2, rela_stack, rela_stack_top);
+ break;
+ case R_LARCH_SOP_SUB:
+ err = rela_stack_push(opr1 - opr2, rela_stack, rela_stack_top);
+ break;
+ case R_LARCH_SOP_SL:
+ err = rela_stack_push(opr1 << opr2, rela_stack, rela_stack_top);
+ break;
+ case R_LARCH_SOP_SR:
+ err = rela_stack_push(opr1 >> opr2, rela_stack, rela_stack_top);
+ break;
+ case R_LARCH_SOP_IF_ELSE:
+ err = rela_stack_push(opr1 ? opr2 : opr3, rela_stack, rela_stack_top);
+ break;
+ default:
+ pr_err("%s: Unsupport relocation type %u\n", mod->name, type);
+ return -EINVAL;
+ }
+
+ return err;
+}
+
+static int apply_r_larch_sop_imm_field(struct module *mod, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
+{
+ int err = 0;
+ s64 opr1;
+ union loongarch_instruction *insn = (union loongarch_instruction *)location;
+
+ err = rela_stack_pop(&opr1, rela_stack, rela_stack_top);
+ if (err)
+ return err;
+
+ switch (type) {
+ case R_LARCH_SOP_POP_32_U_10_12:
+ if (!unsigned_imm_check(opr1, 12))
+ goto overflow;
+
+ /* (*(uint32_t *) PC) [21 ... 10] = opr [11 ... 0] */
+ insn->reg2i12_format.immediate = opr1 & 0xfff;
+ return 0;
+ case R_LARCH_SOP_POP_32_S_10_12:
+ if (!signed_imm_check(opr1, 12))
+ goto overflow;
+
+ insn->reg2i12_format.immediate = opr1 & 0xfff;
+ return 0;
+ case R_LARCH_SOP_POP_32_S_10_16:
+ if (!signed_imm_check(opr1, 16))
+ goto overflow;
+
+ insn->reg2i16_format.immediate = opr1 & 0xffff;
+ return 0;
+ case R_LARCH_SOP_POP_32_S_10_16_S2:
+ if (opr1 % 4)
+ goto unaligned;
+
+ if (!signed_imm_check(opr1, 18))
+ goto overflow;
+
+ insn->reg2i16_format.immediate = (opr1 >> 2) & 0xffff;
+ return 0;
+ case R_LARCH_SOP_POP_32_S_5_20:
+ if (!signed_imm_check(opr1, 20))
+ goto overflow;
+
+ insn->reg1i20_format.immediate = (opr1) & 0xfffff;
+ return 0;
+ case R_LARCH_SOP_POP_32_S_0_5_10_16_S2:
+ if (opr1 % 4)
+ goto unaligned;
+
+ if (!signed_imm_check(opr1, 23))
+ goto overflow;
+
+ opr1 >>= 2;
+ insn->reg1i21_format.immediate_l = opr1 & 0xffff;
+ insn->reg1i21_format.immediate_h = (opr1 >> 16) & 0x1f;
+ return 0;
+ case R_LARCH_SOP_POP_32_S_0_10_10_16_S2:
+ if (opr1 % 4)
+ goto unaligned;
+
+ if (!signed_imm_check(opr1, 28))
+ goto overflow;
+
+ opr1 >>= 2;
+ insn->reg0i26_format.immediate_l = opr1 & 0xffff;
+ insn->reg0i26_format.immediate_h = (opr1 >> 16) & 0x3ff;
+ return 0;
+ case R_LARCH_SOP_POP_32_U:
+ if (!unsigned_imm_check(opr1, 32))
+ goto overflow;
+
+ /* (*(uint32_t *) PC) = opr */
+ *location = (u32)opr1;
+ return 0;
+ default:
+ pr_err("%s: Unsupport relocation type %u\n", mod->name, type);
+ return -EINVAL;
+ }
+
+overflow:
+ pr_err("module %s: opr1 = 0x%llx overflow! dangerous %s (%u) relocation\n",
+ mod->name, opr1, __func__, type);
+ return -ENOEXEC;
+
+unaligned:
+ pr_err("module %s: opr1 = 0x%llx unaligned! dangerous %s (%u) relocation\n",
+ mod->name, opr1, __func__, type);
+ return -ENOEXEC;
+}
+
+static int apply_r_larch_add_sub(struct module *mod, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
+{
+ switch (type) {
+ case R_LARCH_ADD32:
+ *(s32 *)location += v;
+ return 0;
+ case R_LARCH_ADD64:
+ *(s64 *)location += v;
+ return 0;
+ case R_LARCH_SUB32:
+ *(s32 *)location -= v;
+ return 0;
+ case R_LARCH_SUB64:
+ *(s64 *)location -= v;
+ return 0;
+ default:
+ pr_err("%s: Unsupport relocation type %u\n", mod->name, type);
+ return -EINVAL;
+ }
+}
+
+/*
+ * reloc_handlers_rela() - Apply a particular relocation to a module
+ * @mod: the module to apply the reloc to
+ * @location: the address at which the reloc is to be applied
+ * @v: the value of the reloc, with addend for RELA-style
+ * @rela_stack: the stack used for store relocation info, LOCAL to THIS module
+ * @rela_stac_top: where the stack operation(pop/push) applies to
+ *
+ * Return: 0 upon success, else -ERRNO
+ */
+typedef int (*reloc_rela_handler)(struct module *mod, u32 *location, Elf_Addr v,
+ s64 *rela_stack, size_t *rela_stack_top, unsigned int type);
+
+/* The handlers for known reloc types */
+static reloc_rela_handler reloc_rela_handlers[] = {
+ [R_LARCH_NONE ... R_LARCH_SUB64] = apply_r_larch_error,
+
+ [R_LARCH_NONE] = apply_r_larch_none,
+ [R_LARCH_32] = apply_r_larch_32,
+ [R_LARCH_64] = apply_r_larch_64,
+ [R_LARCH_MARK_LA] = apply_r_larch_none,
+ [R_LARCH_MARK_PCREL] = apply_r_larch_none,
+ [R_LARCH_SOP_PUSH_PCREL] = apply_r_larch_sop_push_pcrel,
+ [R_LARCH_SOP_PUSH_ABSOLUTE] = apply_r_larch_sop_push_absolute,
+ [R_LARCH_SOP_PUSH_DUP] = apply_r_larch_sop_push_dup,
+ [R_LARCH_SOP_PUSH_PLT_PCREL] = apply_r_larch_sop_push_plt_pcrel,
+ [R_LARCH_SOP_SUB ... R_LARCH_SOP_IF_ELSE] = apply_r_larch_sop,
+ [R_LARCH_SOP_POP_32_S_10_5 ... R_LARCH_SOP_POP_32_U] = apply_r_larch_sop_imm_field,
+ [R_LARCH_ADD32 ... R_LARCH_SUB64] = apply_r_larch_add_sub,
+};
+
+int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab,
+ unsigned int symindex, unsigned int relsec,
+ struct module *mod)
+{
+ int i, err;
+ unsigned int type;
+ s64 rela_stack[RELA_STACK_DEPTH];
+ size_t rela_stack_top = 0;
+ reloc_rela_handler handler;
+ void *location;
+ Elf_Addr v;
+ Elf_Sym *sym;
+ Elf_Rela *rel = (void *) sechdrs[relsec].sh_addr;
+
+ pr_debug("%s: Applying relocate section %u to %u\n", __func__, relsec,
+ sechdrs[relsec].sh_info);
+
+ rela_stack_top = 0;
+ for (i = 0; i < sechdrs[relsec].sh_size / sizeof(*rel); i++) {
+ /* This is where to make the change */
+ location = (void *)sechdrs[sechdrs[relsec].sh_info].sh_addr + rel[i].r_offset;
+ /* This is the symbol it is referring to */
+ sym = (Elf_Sym *)sechdrs[symindex].sh_addr + ELF_R_SYM(rel[i].r_info);
+ if (IS_ERR_VALUE(sym->st_value)) {
+ /* Ignore unresolved weak symbol */
+ if (ELF_ST_BIND(sym->st_info) == STB_WEAK)
+ continue;
+ pr_warn("%s: Unknown symbol %s\n", mod->name, strtab + sym->st_name);
+ return -ENOENT;
+ }
+
+ type = ELF_R_TYPE(rel[i].r_info);
+
+ if (type < ARRAY_SIZE(reloc_rela_handlers))
+ handler = reloc_rela_handlers[type];
+ else
+ handler = NULL;
+
+ if (!handler) {
+ pr_err("%s: Unknown relocation type %u\n", mod->name, type);
+ return -EINVAL;
+ }
+
+ pr_debug("type %d st_value %llx r_addend %llx loc %llx\n",
+ (int)ELF_R_TYPE(rel[i].r_info),
+ sym->st_value, rel[i].r_addend, (u64)location);
+
+ v = sym->st_value + rel[i].r_addend;
+ err = handler(mod, location, v, rela_stack, &rela_stack_top, type);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+void *module_alloc(unsigned long size)
+{
+ return __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END,
+ GFP_KERNEL, PAGE_KERNEL, 0, NUMA_NO_NODE, __builtin_return_address(0));
+}
--
2.27.0
在2022年6月1日六月 上午11:00，Huacai Chen写道：
> LoongArch-based procesors have 4, 8 or 16 cores per package. This patch
> adds multi-processor (SMP) support for LoongArch.
>
> Reviewed-by: WANG Xuerui <[email protected]>
> Signed-off-by: Huacai Chen <[email protected]>
Reviewed-by: Jiaxun Yang <[email protected]>
CSR IPI makes things much more easily than MIPS era.
Thanks.
> ---
> arch/loongarch/Kconfig | 40 +-
> arch/loongarch/include/asm/atomic.h | 4 +
> arch/loongarch/include/asm/barrier.h | 108 ++++
> arch/loongarch/include/asm/cmpxchg.h | 1 +
> arch/loongarch/include/asm/futex.h | 1 +
> arch/loongarch/include/asm/hardirq.h | 2 +
> arch/loongarch/include/asm/irq.h | 2 +
> arch/loongarch/include/asm/percpu.h | 194 +++++++
> arch/loongarch/include/asm/pgtable.h | 21 +
> arch/loongarch/include/asm/smp.h | 124 ++++
> arch/loongarch/include/asm/stackframe.h | 17 +-
> arch/loongarch/include/asm/tlbflush.h | 13 +
> arch/loongarch/include/asm/topology.h | 7 +-
> arch/loongarch/kernel/Makefile | 2 +
> arch/loongarch/kernel/acpi.c | 69 +++
> arch/loongarch/kernel/asm-offsets.c | 10 +
> arch/loongarch/kernel/head.S | 30 +
> arch/loongarch/kernel/irq.c | 13 +-
> arch/loongarch/kernel/proc.c | 5 +
> arch/loongarch/kernel/process.c | 7 +
> arch/loongarch/kernel/reset.c | 12 +
> arch/loongarch/kernel/setup.c | 26 +
> arch/loongarch/kernel/smp.c | 735 ++++++++++++++++++++++++
> arch/loongarch/kernel/topology.c | 43 +-
> arch/loongarch/kernel/vmlinux.lds.S | 4 +
> arch/loongarch/mm/tlbex.S | 69 +++
> include/linux/cpuhotplug.h | 1 +
> 27 files changed, 1550 insertions(+), 10 deletions(-)
> create mode 100644 arch/loongarch/include/asm/smp.h
> create mode 100644 arch/loongarch/kernel/smp.c
>
> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
> index d6ac80cf3922..b252a51946e3 100644
> --- a/arch/loongarch/Kconfig
> +++ b/arch/loongarch/Kconfig
> @@ -64,6 +64,7 @@ config LOONGARCH
> select GENERIC_LIB_UCMPDI2
> select GENERIC_PCI_IOMAP
> select GENERIC_SCHED_CLOCK
> + select GENERIC_SMP_IDLE_THREAD
> select GENERIC_TIME_VSYSCALL
> select GPIOLIB
> select HAVE_ARCH_AUDITSYSCALL
> @@ -92,7 +93,7 @@ config LOONGARCH
> select HAVE_RSEQ
> select HAVE_SYSCALL_TRACEPOINTS
> select HAVE_TIF_NOHZ
> - select HAVE_VIRT_CPU_ACCOUNTING_GEN
> + select HAVE_VIRT_CPU_ACCOUNTING_GEN if !SMP
> select IRQ_FORCED_THREADING
> select IRQ_LOONGARCH_CPU
> select MODULES_USE_ELF_RELA if MODULES
> @@ -306,6 +307,43 @@ config EFI_STUB
> This kernel feature allows the kernel to be loaded directly by
> EFI firmware without the use of a bootloader.
>
> +config SMP
> + bool "Multi-Processing support"
> + help
> + This enables support for systems with more than one CPU. If you have
> + a system with only one CPU, say N. If you have a system with more
> + than one CPU, say Y.
> +
> + If you say N here, the kernel will run on uni- and multiprocessor
> + machines, but will use only one CPU of a multiprocessor machine. If
> + you say Y here, the kernel will run on many, but not all,
> + uniprocessor machines. On a uniprocessor machine, the kernel
> + will run faster if you say N here.
> +
> + See also the SMP-HOWTO available at
> <http://www.tldp.org/docs.html#howto>.
> +
> + If you don't know what to do here, say N.
> +
> +config HOTPLUG_CPU
> + bool "Support for hot-pluggable CPUs"
> + depends on SMP
> + select GENERIC_IRQ_MIGRATION
> + help
> + Say Y here to allow turning CPUs off and on. CPUs can be
> + controlled through /sys/devices/system/cpu.
> + (Note: power management support will enable this option
> + automatically on SMP systems. )
> + Say N if you want to disable CPU hotplug.
> +
> +config NR_CPUS
> + int "Maximum number of CPUs (2-256)"
> + range 2 256
> + depends on SMP
> + default "64"
> + help
> + This allows you to specify the maximum number of CPUs which this
> + kernel will support.
> +
> config FORCE_MAX_ZONEORDER
> int "Maximum zone order"
> range 14 64 if PAGE_SIZE_64KB
> diff --git a/arch/loongarch/include/asm/atomic.h
> b/arch/loongarch/include/asm/atomic.h
> index 932352342b12..979367ad4e2c 100644
> --- a/arch/loongarch/include/asm/atomic.h
> +++ b/arch/loongarch/include/asm/atomic.h
> @@ -162,6 +162,7 @@ static inline int arch_atomic_sub_if_positive(int
> i, atomic_t *v)
> " sc.w %1, %2 \n"
> " beq $zero, %1, 1b \n"
> "2: \n"
> + __WEAK_LLSC_MB
> : "=&r" (result), "=&r" (temp),
> "+" GCC_OFF_SMALL_ASM() (v->counter)
> : "I" (-i));
> @@ -174,6 +175,7 @@ static inline int arch_atomic_sub_if_positive(int
> i, atomic_t *v)
> " sc.w %1, %2 \n"
> " beq $zero, %1, 1b \n"
> "2: \n"
> + __WEAK_LLSC_MB
> : "=&r" (result), "=&r" (temp),
> "+" GCC_OFF_SMALL_ASM() (v->counter)
> : "r" (i));
> @@ -323,6 +325,7 @@ static inline long
> arch_atomic64_sub_if_positive(long i, atomic64_t *v)
> " sc.d %1, %2 \n"
> " beq %1, $zero, 1b \n"
> "2: \n"
> + __WEAK_LLSC_MB
> : "=&r" (result), "=&r" (temp),
> "+" GCC_OFF_SMALL_ASM() (v->counter)
> : "I" (-i));
> @@ -335,6 +338,7 @@ static inline long
> arch_atomic64_sub_if_positive(long i, atomic64_t *v)
> " sc.d %1, %2 \n"
> " beq %1, $zero, 1b \n"
> "2: \n"
> + __WEAK_LLSC_MB
> : "=&r" (result), "=&r" (temp),
> "+" GCC_OFF_SMALL_ASM() (v->counter)
> : "r" (i));
> diff --git a/arch/loongarch/include/asm/barrier.h
> b/arch/loongarch/include/asm/barrier.h
> index e57571bcaf4f..b6517eeeb141 100644
> --- a/arch/loongarch/include/asm/barrier.h
> +++ b/arch/loongarch/include/asm/barrier.h
> @@ -18,6 +18,19 @@
> #define mb() fast_mb()
> #define iob() fast_iob()
>
> +#define __smp_mb() __asm__ __volatile__("dbar 0" : : : "memory")
> +#define __smp_rmb() __asm__ __volatile__("dbar 0" : : : "memory")
> +#define __smp_wmb() __asm__ __volatile__("dbar 0" : : : "memory")
> +
> +#ifdef CONFIG_SMP
> +#define __WEAK_LLSC_MB " dbar 0 \n"
> +#else
> +#define __WEAK_LLSC_MB " \n"
> +#endif
> +
> +#define __smp_mb__before_atomic() barrier()
> +#define __smp_mb__after_atomic() barrier()
> +
> /**
> * array_index_mask_nospec() - generate a ~0 mask when index < size, 0
> otherwise
> * @index: array element index
> @@ -46,6 +59,101 @@ static inline unsigned long
> array_index_mask_nospec(unsigned long index,
> return mask;
> }
>
> +#define __smp_load_acquire(p) \
> +({ \
> + union { typeof(*p) __val; char __c[1]; } __u; \
> + unsigned long __tmp = 0; \
> + compiletime_assert_atomic_type(*p); \
> + switch (sizeof(*p)) { \
> + case 1: \
> + *(__u8 *)__u.__c = *(volatile __u8 *)p; \
> + __smp_mb(); \
> + break; \
> + case 2: \
> + *(__u16 *)__u.__c = *(volatile __u16 *)p; \
> + __smp_mb(); \
> + break; \
> + case 4: \
> + __asm__ __volatile__( \
> + "amor_db.w %[val], %[tmp], %[mem] \n" \
> + : [val] "=&r" (*(__u32 *)__u.__c) \
> + : [mem] "ZB" (*(u32 *) p), [tmp] "r" (__tmp) \
> + : "memory"); \
> + break; \
> + case 8: \
> + __asm__ __volatile__( \
> + "amor_db.d %[val], %[tmp], %[mem] \n" \
> + : [val] "=&r" (*(__u64 *)__u.__c) \
> + : [mem] "ZB" (*(u64 *) p), [tmp] "r" (__tmp) \
> + : "memory"); \
> + break; \
> + } \
> + (typeof(*p))__u.__val; \
> +})
> +
> +#define __smp_store_release(p, v) \
> +do { \
> + union { typeof(*p) __val; char __c[1]; } __u = \
> + { .__val = (__force typeof(*p)) (v) }; \
> + unsigned long __tmp; \
> + compiletime_assert_atomic_type(*p); \
> + switch (sizeof(*p)) { \
> + case 1: \
> + __smp_mb(); \
> + *(volatile __u8 *)p = *(__u8 *)__u.__c; \
> + break; \
> + case 2: \
> + __smp_mb(); \
> + *(volatile __u16 *)p = *(__u16 *)__u.__c; \
> + break; \
> + case 4: \
> + __asm__ __volatile__( \
> + "amswap_db.w %[tmp], %[val], %[mem] \n" \
> + : [mem] "+ZB" (*(u32 *)p), [tmp] "=&r" (__tmp) \
> + : [val] "r" (*(__u32 *)__u.__c) \
> + : ); \
> + break; \
> + case 8: \
> + __asm__ __volatile__( \
> + "amswap_db.d %[tmp], %[val], %[mem] \n" \
> + : [mem] "+ZB" (*(u64 *)p), [tmp] "=&r" (__tmp) \
> + : [val] "r" (*(__u64 *)__u.__c) \
> + : ); \
> + break; \
> + } \
> +} while (0)
> +
> +#define __smp_store_mb(p, v) \
> +do { \
> + union { typeof(p) __val; char __c[1]; } __u = \
> + { .__val = (__force typeof(p)) (v) }; \
> + unsigned long __tmp; \
> + switch (sizeof(p)) { \
> + case 1: \
> + *(volatile __u8 *)&p = *(__u8 *)__u.__c; \
> + __smp_mb(); \
> + break; \
> + case 2: \
> + *(volatile __u16 *)&p = *(__u16 *)__u.__c; \
> + __smp_mb(); \
> + break; \
> + case 4: \
> + __asm__ __volatile__( \
> + "amswap_db.w %[tmp], %[val], %[mem] \n" \
> + : [mem] "+ZB" (*(u32 *)&p), [tmp] "=&r" (__tmp) \
> + : [val] "r" (*(__u32 *)__u.__c) \
> + : ); \
> + break; \
> + case 8: \
> + __asm__ __volatile__( \
> + "amswap_db.d %[tmp], %[val], %[mem] \n" \
> + : [mem] "+ZB" (*(u64 *)&p), [tmp] "=&r" (__tmp) \
> + : [val] "r" (*(__u64 *)__u.__c) \
> + : ); \
> + break; \
> + } \
> +} while (0)
> +
> #include <asm-generic/barrier.h>
>
> #endif /* __ASM_BARRIER_H */
> diff --git a/arch/loongarch/include/asm/cmpxchg.h
> b/arch/loongarch/include/asm/cmpxchg.h
> index 48613b872bc8..75b3a4478652 100644
> --- a/arch/loongarch/include/asm/cmpxchg.h
> +++ b/arch/loongarch/include/asm/cmpxchg.h
> @@ -59,6 +59,7 @@ static inline unsigned long __xchg(volatile void
> *ptr, unsigned long x,
> " " st " $t0, %1 \n" \
> " beq $zero, $t0, 1b \n" \
> "2: \n" \
> + __WEAK_LLSC_MB \
> : "=&r" (__ret), "=ZB"(*m) \
> : "ZB"(*m), "Jr" (old), "Jr" (new) \
> : "t0", "memory"); \
> diff --git a/arch/loongarch/include/asm/futex.h
> b/arch/loongarch/include/asm/futex.h
> index b27d55f92db7..9de8231694ec 100644
> --- a/arch/loongarch/include/asm/futex.h
> +++ b/arch/loongarch/include/asm/futex.h
> @@ -86,6 +86,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user
> *uaddr, u32 oldval, u32 newv
> "2: sc.w $t0, %2 \n"
> " beq $zero, $t0, 1b \n"
> "3: \n"
> + __WEAK_LLSC_MB
> " .section .fixup,\"ax\" \n"
> "4: li.d %0, %6 \n"
> " b 3b \n"
> diff --git a/arch/loongarch/include/asm/hardirq.h
> b/arch/loongarch/include/asm/hardirq.h
> index d32f83938880..befe8184aa08 100644
> --- a/arch/loongarch/include/asm/hardirq.h
> +++ b/arch/loongarch/include/asm/hardirq.h
> @@ -21,4 +21,6 @@ typedef struct {
>
> DECLARE_PER_CPU_ALIGNED(irq_cpustat_t, irq_stat);
>
> +#define __ARCH_IRQ_STAT
> +
> #endif /* _ASM_HARDIRQ_H */
> diff --git a/arch/loongarch/include/asm/irq.h b/arch/loongarch/include/asm/irq.h
> index f3f1baf7027c..ace3ea6da72e 100644
> --- a/arch/loongarch/include/asm/irq.h
> +++ b/arch/loongarch/include/asm/irq.h
> @@ -125,6 +125,8 @@ extern struct irq_domain *pch_lpc_domain;
> extern struct irq_domain *pch_msi_domain[MAX_IO_PICS];
> extern struct irq_domain *pch_pic_domain[MAX_IO_PICS];
>
> +extern irqreturn_t loongson3_ipi_interrupt(int irq, void *dev);
> +
> #include <asm-generic/irq.h>
>
> #endif /* _ASM_IRQ_H */
> diff --git a/arch/loongarch/include/asm/percpu.h
> b/arch/loongarch/include/asm/percpu.h
> index b03d8f8b9fd3..34f15a6fb1e7 100644
> --- a/arch/loongarch/include/asm/percpu.h
> +++ b/arch/loongarch/include/asm/percpu.h
> @@ -5,6 +5,8 @@
> #ifndef __ASM_PERCPU_H
> #define __ASM_PERCPU_H
>
> +#include <asm/cmpxchg.h>
> +
> /* Use r21 for fast access */
> register unsigned long __my_cpu_offset __asm__("$r21");
>
> @@ -15,6 +17,198 @@ static inline void set_my_cpu_offset(unsigned long off)
> }
> #define __my_cpu_offset __my_cpu_offset
>
> +#define PERCPU_OP(op, asm_op, c_op) \
> +static inline unsigned long __percpu_##op(void *ptr, \
> + unsigned long val, int size) \
> +{ \
> + unsigned long ret; \
> + \
> + switch (size) { \
> + case 4: \
> + __asm__ __volatile__( \
> + "am"#asm_op".w" " %[ret], %[val], %[ptr] \n" \
> + : [ret] "=&r" (ret), [ptr] "+ZB"(*(u32 *)ptr) \
> + : [val] "r" (val)); \
> + break; \
> + case 8: \
> + __asm__ __volatile__( \
> + "am"#asm_op".d" " %[ret], %[val], %[ptr] \n" \
> + : [ret] "=&r" (ret), [ptr] "+ZB"(*(u64 *)ptr) \
> + : [val] "r" (val)); \
> + break; \
> + default: \
> + ret = 0; \
> + BUILD_BUG(); \
> + } \
> + \
> + return ret c_op val; \
> +}
> +
> +PERCPU_OP(add, add, +)
> +PERCPU_OP(and, and, &)
> +PERCPU_OP(or, or, |)
> +#undef PERCPU_OP
> +
> +static inline unsigned long __percpu_read(void *ptr, int size)
> +{
> + unsigned long ret;
> +
> + switch (size) {
> + case 1:
> + __asm__ __volatile__ ("ldx.b %[ret], $r21, %[ptr] \n"
> + : [ret] "=&r"(ret)
> + : [ptr] "r"(ptr)
> + : "memory");
> + break;
> + case 2:
> + __asm__ __volatile__ ("ldx.h %[ret], $r21, %[ptr] \n"
> + : [ret] "=&r"(ret)
> + : [ptr] "r"(ptr)
> + : "memory");
> + break;
> + case 4:
> + __asm__ __volatile__ ("ldx.w %[ret], $r21, %[ptr] \n"
> + : [ret] "=&r"(ret)
> + : [ptr] "r"(ptr)
> + : "memory");
> + break;
> + case 8:
> + __asm__ __volatile__ ("ldx.d %[ret], $r21, %[ptr] \n"
> + : [ret] "=&r"(ret)
> + : [ptr] "r"(ptr)
> + : "memory");
> + break;
> + default:
> + ret = 0;
> + BUILD_BUG();
> + }
> +
> + return ret;
> +}
> +
> +static inline void __percpu_write(void *ptr, unsigned long val, int size)
> +{
> + switch (size) {
> + case 1:
> + __asm__ __volatile__("stx.b %[val], $r21, %[ptr] \n"
> + :
> + : [val] "r" (val), [ptr] "r" (ptr)
> + : "memory");
> + break;
> + case 2:
> + __asm__ __volatile__("stx.h %[val], $r21, %[ptr] \n"
> + :
> + : [val] "r" (val), [ptr] "r" (ptr)
> + : "memory");
> + break;
> + case 4:
> + __asm__ __volatile__("stx.w %[val], $r21, %[ptr] \n"
> + :
> + : [val] "r" (val), [ptr] "r" (ptr)
> + : "memory");
> + break;
> + case 8:
> + __asm__ __volatile__("stx.d %[val], $r21, %[ptr] \n"
> + :
> + : [val] "r" (val), [ptr] "r" (ptr)
> + : "memory");
> + break;
> + default:
> + BUILD_BUG();
> + }
> +}
> +
> +static inline unsigned long __percpu_xchg(void *ptr, unsigned long val,
> + int size)
> +{
> + switch (size) {
> + case 4:
> + return __xchg_asm("amswap.w", (volatile u32 *)ptr, (u32)val);
> +
> + case 8:
> + return __xchg_asm("amswap.d", (volatile u64 *)ptr, (u64)val);
> +
> + default:
> + BUILD_BUG();
> + }
> +
> + return 0;
> +}
> +
> +/* this_cpu_cmpxchg */
> +#define _protect_cmpxchg_local(pcp, o, n) \
> +({ \
> + typeof(*raw_cpu_ptr(&(pcp))) __ret; \
> + preempt_disable_notrace(); \
> + __ret = cmpxchg_local(raw_cpu_ptr(&(pcp)), o, n); \
> + preempt_enable_notrace(); \
> + __ret; \
> +})
> +
> +#define _percpu_read(pcp) \
> +({ \
> + typeof(pcp) __retval; \
> + __retval = (typeof(pcp))__percpu_read(&(pcp), sizeof(pcp)); \
> + __retval; \
> +})
> +
> +#define _percpu_write(pcp, val) \
> +do { \
> + __percpu_write(&(pcp), (unsigned long)(val), sizeof(pcp)); \
> +} while (0) \
> +
> +#define _pcp_protect(operation, pcp, val) \
> +({ \
> + typeof(pcp) __retval; \
> + preempt_disable_notrace(); \
> + __retval = (typeof(pcp))operation(raw_cpu_ptr(&(pcp)), \
> + (val), sizeof(pcp)); \
> + preempt_enable_notrace(); \
> + __retval; \
> +})
> +
> +#define _percpu_add(pcp, val) \
> + _pcp_protect(__percpu_add, pcp, val)
> +
> +#define _percpu_add_return(pcp, val) _percpu_add(pcp, val)
> +
> +#define _percpu_and(pcp, val) \
> + _pcp_protect(__percpu_and, pcp, val)
> +
> +#define _percpu_or(pcp, val) \
> + _pcp_protect(__percpu_or, pcp, val)
> +
> +#define _percpu_xchg(pcp, val) ((typeof(pcp)) \
> + _pcp_protect(__percpu_xchg, pcp, (unsigned long)(val)))
> +
> +#define this_cpu_add_4(pcp, val) _percpu_add(pcp, val)
> +#define this_cpu_add_8(pcp, val) _percpu_add(pcp, val)
> +
> +#define this_cpu_add_return_4(pcp, val) _percpu_add_return(pcp, val)
> +#define this_cpu_add_return_8(pcp, val) _percpu_add_return(pcp, val)
> +
> +#define this_cpu_and_4(pcp, val) _percpu_and(pcp, val)
> +#define this_cpu_and_8(pcp, val) _percpu_and(pcp, val)
> +
> +#define this_cpu_or_4(pcp, val) _percpu_or(pcp, val)
> +#define this_cpu_or_8(pcp, val) _percpu_or(pcp, val)
> +
> +#define this_cpu_read_1(pcp) _percpu_read(pcp)
> +#define this_cpu_read_2(pcp) _percpu_read(pcp)
> +#define this_cpu_read_4(pcp) _percpu_read(pcp)
> +#define this_cpu_read_8(pcp) _percpu_read(pcp)
> +
> +#define this_cpu_write_1(pcp, val) _percpu_write(pcp, val)
> +#define this_cpu_write_2(pcp, val) _percpu_write(pcp, val)
> +#define this_cpu_write_4(pcp, val) _percpu_write(pcp, val)
> +#define this_cpu_write_8(pcp, val) _percpu_write(pcp, val)
> +
> +#define this_cpu_xchg_4(pcp, val) _percpu_xchg(pcp, val)
> +#define this_cpu_xchg_8(pcp, val) _percpu_xchg(pcp, val)
> +
> +#define this_cpu_cmpxchg_4(ptr, o, n) _protect_cmpxchg_local(ptr, o, n)
> +#define this_cpu_cmpxchg_8(ptr, o, n) _protect_cmpxchg_local(ptr, o, n)
> +
> #include <asm-generic/percpu.h>
>
> #endif /* __ASM_PERCPU_H */
> diff --git a/arch/loongarch/include/asm/pgtable.h
> b/arch/loongarch/include/asm/pgtable.h
> index 8920dd8b297b..5e33987d0a13 100644
> --- a/arch/loongarch/include/asm/pgtable.h
> +++ b/arch/loongarch/include/asm/pgtable.h
> @@ -279,8 +279,29 @@ static inline void set_pte(pte_t *ptep, pte_t
> pteval)
> * Make sure the buddy is global too (if it's !none,
> * it better already be global)
> */
> +#ifdef CONFIG_SMP
> + /*
> + * For SMP, multiple CPUs can race, so we need to do
> + * this atomically.
> + */
> + unsigned long page_global = _PAGE_GLOBAL;
> + unsigned long tmp;
> +
> + __asm__ __volatile__ (
> + "1:" __LL "%[tmp], %[buddy] \n"
> + " bnez %[tmp], 2f \n"
> + " or %[tmp], %[tmp], %[global] \n"
> + __SC "%[tmp], %[buddy] \n"
> + " beqz %[tmp], 1b \n"
> + " nop \n"
> + "2: \n"
> + __WEAK_LLSC_MB
> + : [buddy] "+m" (buddy->pte), [tmp] "=&r" (tmp)
> + : [global] "r" (page_global));
> +#else /* !CONFIG_SMP */
> if (pte_none(*buddy))
> pte_val(*buddy) = pte_val(*buddy) | _PAGE_GLOBAL;
> +#endif /* CONFIG_SMP */
> }
> }
>
> diff --git a/arch/loongarch/include/asm/smp.h
> b/arch/loongarch/include/asm/smp.h
> new file mode 100644
> index 000000000000..551e1f37c705
> --- /dev/null
> +++ b/arch/loongarch/include/asm/smp.h
> @@ -0,0 +1,124 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Author: Huacai Chen <[email protected]>
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef __ASM_SMP_H
> +#define __ASM_SMP_H
> +
> +#include <linux/atomic.h>
> +#include <linux/bitops.h>
> +#include <linux/linkage.h>
> +#include <linux/smp.h>
> +#include <linux/threads.h>
> +#include <linux/cpumask.h>
> +
> +void loongson3_smp_setup(void);
> +void loongson3_prepare_cpus(unsigned int max_cpus);
> +void loongson3_boot_secondary(int cpu, struct task_struct *idle);
> +void loongson3_init_secondary(void);
> +void loongson3_smp_finish(void);
> +void loongson3_send_ipi_single(int cpu, unsigned int action);
> +void loongson3_send_ipi_mask(const struct cpumask *mask, unsigned int
> action);
> +#ifdef CONFIG_HOTPLUG_CPU
> +int loongson3_cpu_disable(void);
> +void loongson3_cpu_die(unsigned int cpu);
> +#endif
> +
> +#ifdef CONFIG_SMP
> +
> +static inline void plat_smp_setup(void)
> +{
> + loongson3_smp_setup();
> +}
> +
> +#else /* !CONFIG_SMP */
> +
> +static inline void plat_smp_setup(void) { }
> +
> +#endif /* !CONFIG_SMP */
> +
> +extern int smp_num_siblings;
> +extern int num_processors;
> +extern int disabled_cpus;
> +extern cpumask_t cpu_sibling_map[];
> +extern cpumask_t cpu_core_map[];
> +extern cpumask_t cpu_foreign_map[];
> +
> +static inline int raw_smp_processor_id(void)
> +{
> +#if defined(__VDSO__)
> + extern int vdso_smp_processor_id(void)
> + __compiletime_error("VDSO should not call smp_processor_id()");
> + return vdso_smp_processor_id();
> +#else
> + return current_thread_info()->cpu;
> +#endif
> +}
> +#define raw_smp_processor_id raw_smp_processor_id
> +
> +/* Map from cpu id to sequential logical cpu number. This will only
> + * not be idempotent when cpus failed to come on-line. */
> +extern int __cpu_number_map[NR_CPUS];
> +#define cpu_number_map(cpu) __cpu_number_map[cpu]
> +
> +/* The reverse map from sequential logical cpu number to cpu id. */
> +extern int __cpu_logical_map[NR_CPUS];
> +#define cpu_logical_map(cpu) __cpu_logical_map[cpu]
> +
> +#define cpu_physical_id(cpu) cpu_logical_map(cpu)
> +
> +#define SMP_BOOT_CPU 0x1
> +#define SMP_RESCHEDULE 0x2
> +#define SMP_CALL_FUNCTION 0x4
> +
> +struct secondary_data {
> + unsigned long stack;
> + unsigned long thread_info;
> +};
> +extern struct secondary_data cpuboot_data;
> +
> +extern asmlinkage void smpboot_entry(void);
> +
> +extern void calculate_cpu_foreign_map(void);
> +
> +/*
> + * Generate IPI list text
> + */
> +extern void show_ipi_list(struct seq_file *p, int prec);
> +
> +/*
> + * This function sends a 'reschedule' IPI to another CPU.
> + * it goes straight through and wastes no time serializing
> + * anything. Worst case is that we lose a reschedule ...
> + */
> +static inline void smp_send_reschedule(int cpu)
> +{
> + loongson3_send_ipi_single(cpu, SMP_RESCHEDULE);
> +}
> +
> +static inline void arch_send_call_function_single_ipi(int cpu)
> +{
> + loongson3_send_ipi_single(cpu, SMP_CALL_FUNCTION);
> +}
> +
> +static inline void arch_send_call_function_ipi_mask(const struct
> cpumask *mask)
> +{
> + loongson3_send_ipi_mask(mask, SMP_CALL_FUNCTION);
> +}
> +
> +#ifdef CONFIG_HOTPLUG_CPU
> +static inline int __cpu_disable(void)
> +{
> + return loongson3_cpu_disable();
> +}
> +
> +static inline void __cpu_die(unsigned int cpu)
> +{
> + loongson3_cpu_die(cpu);
> +}
> +
> +extern void play_dead(void);
> +#endif
> +
> +#endif /* __ASM_SMP_H */
> diff --git a/arch/loongarch/include/asm/stackframe.h
> b/arch/loongarch/include/asm/stackframe.h
> index 44151b878d00..4ca953062b5b 100644
> --- a/arch/loongarch/include/asm/stackframe.h
> +++ b/arch/loongarch/include/asm/stackframe.h
> @@ -77,17 +77,24 @@
> * new value in sp.
> */
> .macro get_saved_sp docfi=0
> - la.abs t1, kernelsp
> - move t0, sp
> + la.abs t1, kernelsp
> +#ifdef CONFIG_SMP
> + csrrd t0, PERCPU_BASE_KS
> + LONG_ADD t1, t1, t0
> +#endif
> + move t0, sp
> .if \docfi
> .cfi_register sp, t0
> .endif
> - LONG_L sp, t1, 0
> + LONG_L sp, t1, 0
> .endm
>
> .macro set_saved_sp stackp temp temp2
> - la.abs \temp, kernelsp
> - LONG_S \stackp, \temp, 0
> + la.abs \temp, kernelsp
> +#ifdef CONFIG_SMP
> + LONG_ADD \temp, \temp, u0
> +#endif
> + LONG_S \stackp, \temp, 0
> .endm
>
> .macro SAVE_SOME docfi=0
> diff --git a/arch/loongarch/include/asm/tlbflush.h
> b/arch/loongarch/include/asm/tlbflush.h
> index 36bd6d11dc2d..a0785e590681 100644
> --- a/arch/loongarch/include/asm/tlbflush.h
> +++ b/arch/loongarch/include/asm/tlbflush.h
> @@ -25,6 +25,17 @@ extern void local_flush_tlb_kernel_range(unsigned
> long start, unsigned long end)
> extern void local_flush_tlb_page(struct vm_area_struct *vma, unsigned
> long page);
> extern void local_flush_tlb_one(unsigned long vaddr);
>
> +#ifdef CONFIG_SMP
> +
> +extern void flush_tlb_all(void);
> +extern void flush_tlb_mm(struct mm_struct *);
> +extern void flush_tlb_range(struct vm_area_struct *vma, unsigned long,
> unsigned long);
> +extern void flush_tlb_kernel_range(unsigned long, unsigned long);
> +extern void flush_tlb_page(struct vm_area_struct *, unsigned long);
> +extern void flush_tlb_one(unsigned long vaddr);
> +
> +#else /* CONFIG_SMP */
> +
> #define flush_tlb_all() local_flush_tlb_all()
> #define flush_tlb_mm(mm) local_flush_tlb_mm(mm)
> #define flush_tlb_range(vma, vmaddr, end) local_flush_tlb_range(vma,
> vmaddr, end)
> @@ -32,4 +43,6 @@ extern void local_flush_tlb_one(unsigned long vaddr);
> #define flush_tlb_page(vma, page) local_flush_tlb_page(vma, page)
> #define flush_tlb_one(vaddr) local_flush_tlb_one(vaddr)
>
> +#endif /* CONFIG_SMP */
> +
> #endif /* __ASM_TLBFLUSH_H */
> diff --git a/arch/loongarch/include/asm/topology.h
> b/arch/loongarch/include/asm/topology.h
> index 9ac71a25207a..da135841e5b1 100644
> --- a/arch/loongarch/include/asm/topology.h
> +++ b/arch/loongarch/include/asm/topology.h
> @@ -7,7 +7,12 @@
>
> #include <linux/smp.h>
>
> -#define cpu_logical_map(cpu) 0
> +#ifdef CONFIG_SMP
> +#define topology_physical_package_id(cpu) (cpu_data[cpu].package)
> +#define topology_core_id(cpu) (cpu_data[cpu].core)
> +#define topology_core_cpumask(cpu) (&cpu_core_map[cpu])
> +#define topology_sibling_cpumask(cpu) (&cpu_sibling_map[cpu])
> +#endif
>
> #include <asm-generic/topology.h>
>
> diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
> index e5a3b2fb9961..2cb6f698716a 100644
> --- a/arch/loongarch/kernel/Makefile
> +++ b/arch/loongarch/kernel/Makefile
> @@ -18,4 +18,6 @@ obj-$(CONFIG_MODULES) += module.o module-sections.o
>
> obj-$(CONFIG_PROC_FS) += proc.o
>
> +obj-$(CONFIG_SMP) += smp.o
> +
> CPPFLAGS_vmlinux.lds := $(KBUILD_CFLAGS)
> diff --git a/arch/loongarch/kernel/acpi.c b/arch/loongarch/kernel/acpi.c
> index a644220bb426..181c59493b63 100644
> --- a/arch/loongarch/kernel/acpi.c
> +++ b/arch/loongarch/kernel/acpi.c
> @@ -137,8 +137,44 @@ void __init acpi_boot_table_init(void)
> }
> }
>
> +static int set_processor_mask(u32 id, u32 flags)
> +{
> +
> + int cpu, cpuid = id;
> +
> + if (num_processors >= nr_cpu_ids) {
> + pr_warn(PREFIX "nr_cpus/possible_cpus limit of %i reached."
> + " processor 0x%x ignored.\n", nr_cpu_ids, cpuid);
> +
> + return -ENODEV;
> +
> + }
> + if (cpuid == loongson_sysconf.boot_cpu_id)
> + cpu = 0;
> + else
> + cpu = cpumask_next_zero(-1, cpu_present_mask);
> +
> + if (flags & ACPI_MADT_ENABLED) {
> + num_processors++;
> + set_cpu_possible(cpu, true);
> + set_cpu_present(cpu, true);
> + __cpu_number_map[cpuid] = cpu;
> + __cpu_logical_map[cpu] = cpuid;
> + } else
> + disabled_cpus++;
> +
> + return cpu;
> +}
> +
> static void __init acpi_process_madt(void)
> {
> + int i;
> +
> + for (i = 0; i < NR_CPUS; i++) {
> + __cpu_number_map[i] = -1;
> + __cpu_logical_map[i] = -1;
> + }
> +
> loongson_sysconf.nr_cpus = num_processors;
> }
>
> @@ -167,3 +203,36 @@ void __init
> arch_reserve_mem_area(acpi_physical_address addr, size_t size)
> {
> memblock_reserve(addr, size);
> }
> +
> +#ifdef CONFIG_ACPI_HOTPLUG_CPU
> +
> +#include <acpi/processor.h>
> +
> +int acpi_map_cpu(acpi_handle handle, phys_cpuid_t physid, u32 acpi_id,
> int *pcpu)
> +{
> + int cpu;
> +
> + cpu = set_processor_mask(physid, ACPI_MADT_ENABLED);
> + if (cpu < 0) {
> + pr_info(PREFIX "Unable to map lapic to logical cpu number\n");
> + return cpu;
> + }
> +
> + *pcpu = cpu;
> +
> + return 0;
> +}
> +EXPORT_SYMBOL(acpi_map_cpu);
> +
> +int acpi_unmap_cpu(int cpu)
> +{
> + set_cpu_present(cpu, false);
> + num_processors--;
> +
> + pr_info("cpu%d hot remove!\n", cpu);
> +
> + return 0;
> +}
> +EXPORT_SYMBOL(acpi_unmap_cpu);
> +
> +#endif /* CONFIG_ACPI_HOTPLUG_CPU */
> diff --git a/arch/loongarch/kernel/asm-offsets.c
> b/arch/loongarch/kernel/asm-offsets.c
> index 3531e3c60a6e..bfb65eb2844f 100644
> --- a/arch/loongarch/kernel/asm-offsets.c
> +++ b/arch/loongarch/kernel/asm-offsets.c
> @@ -252,3 +252,13 @@ void output_signal_defines(void)
> DEFINE(_SIGXFSZ, SIGXFSZ);
> BLANK();
> }
> +
> +#ifdef CONFIG_SMP
> +void output_smpboot_defines(void)
> +{
> + COMMENT("Linux smp cpu boot offsets.");
> + OFFSET(CPU_BOOT_STACK, secondary_data, stack);
> + OFFSET(CPU_BOOT_TINFO, secondary_data, thread_info);
> + BLANK();
> +}
> +#endif
> diff --git a/arch/loongarch/kernel/head.S b/arch/loongarch/kernel/head.S
> index 4cb79e9c70f5..8ca45cf17433 100644
> --- a/arch/loongarch/kernel/head.S
> +++ b/arch/loongarch/kernel/head.S
> @@ -96,4 +96,34 @@ SYM_CODE_START(kernel_entry) # kernel entry point
>
> SYM_CODE_END(kernel_entry)
>
> +#ifdef CONFIG_SMP
> +
> +/*
> + * SMP slave cpus entry point. Board specific code for bootstrap calls this
> + * function after setting up the stack and tp registers.
> + */
> +SYM_CODE_START(smpboot_entry)
> + li.d t0, CSR_DMW0_INIT # UC, PLV0
> + csrwr t0, LOONGARCH_CSR_DMWIN0
> + li.d t0, CSR_DMW1_INIT # CA, PLV0
> + csrwr t0, LOONGARCH_CSR_DMWIN1
> + li.w t0, 0xb0 # PLV=0, IE=0, PG=1
> + csrwr t0, LOONGARCH_CSR_CRMD
> + li.w t0, 0x04 # PLV=0, PIE=1, PWE=0
> + csrwr t0, LOONGARCH_CSR_PRMD
> + li.w t0, 0x00 # FPE=0, SXE=0, ASXE=0, BTE=0
> + csrwr t0, LOONGARCH_CSR_EUEN
> +
> + la.abs t0, cpuboot_data
> + ld.d sp, t0, CPU_BOOT_STACK
> + ld.d tp, t0, CPU_BOOT_TINFO
> +
> + la.abs t0, 0f
> + jirl zero, t0, 0
> +0:
> + bl start_secondary
> +SYM_CODE_END(smpboot_entry)
> +
> +#endif /* CONFIG_SMP */
> +
> SYM_ENTRY(kernel_entry_end, SYM_L_GLOBAL, SYM_A_NONE)
> diff --git a/arch/loongarch/kernel/irq.c b/arch/loongarch/kernel/irq.c
> index 9bd07edf0bce..4b671d305ede 100644
> --- a/arch/loongarch/kernel/irq.c
> +++ b/arch/loongarch/kernel/irq.c
> @@ -47,13 +47,17 @@ asmlinkage void spurious_interrupt(void)
>
> int arch_show_interrupts(struct seq_file *p, int prec)
> {
> +#ifdef CONFIG_SMP
> + show_ipi_list(p, prec);
> +#endif
> seq_printf(p, "%*s: %10u\n", prec, "ERR", atomic_read(&irq_err_count));
> return 0;
> }
>
> void __init init_IRQ(void)
> {
> - int i;
> + int i, r, ipi_irq;
> + static int ipi_dummy_dev;
> unsigned int order = get_order(IRQ_STACK_SIZE);
> struct page *page;
>
> @@ -61,6 +65,13 @@ void __init init_IRQ(void)
> clear_csr_estat(ESTATF_IP);
>
> irqchip_init();
> +#ifdef CONFIG_SMP
> + ipi_irq = EXCCODE_IPI - EXCCODE_INT_START;
> + irq_set_percpu_devid(ipi_irq);
> + r = request_percpu_irq(ipi_irq, loongson3_ipi_interrupt, "IPI",
> &ipi_dummy_dev);
> + if (r < 0)
> + panic("IPI IRQ request failed\n");
> +#endif
>
> for (i = 0; i < NR_IRQS; i++)
> irq_set_noprobe(i);
> diff --git a/arch/loongarch/kernel/proc.c b/arch/loongarch/kernel/proc.c
> index d25592a29196..1effc73850fe 100644
> --- a/arch/loongarch/kernel/proc.c
> +++ b/arch/loongarch/kernel/proc.c
> @@ -35,6 +35,11 @@ static int show_cpuinfo(struct seq_file *m, void *v)
> unsigned int fp_version = cpu_data[n].fpu_vers;
> struct proc_cpuinfo_notifier_args proc_cpuinfo_notifier_args;
>
> +#ifdef CONFIG_SMP
> + if (!cpu_online(n))
> + return 0;
> +#endif
> +
> /*
> * For the first processor also print the system type
> */
> diff --git a/arch/loongarch/kernel/process.c b/arch/loongarch/kernel/process.c
> index 54020223068f..6d944d65f600 100644
> --- a/arch/loongarch/kernel/process.c
> +++ b/arch/loongarch/kernel/process.c
> @@ -53,6 +53,13 @@
> unsigned long boot_option_idle_override = IDLE_NO_OVERRIDE;
> EXPORT_SYMBOL(boot_option_idle_override);
>
> +#ifdef CONFIG_HOTPLUG_CPU
> +void arch_cpu_idle_dead(void)
> +{
> + play_dead();
> +}
> +#endif
> +
> asmlinkage void ret_from_fork(void);
> asmlinkage void ret_from_kernel_thread(void);
>
> diff --git a/arch/loongarch/kernel/reset.c b/arch/loongarch/kernel/reset.c
> index ef484ce43c5c..2b86469e4718 100644
> --- a/arch/loongarch/kernel/reset.c
> +++ b/arch/loongarch/kernel/reset.c
> @@ -65,16 +65,28 @@ EXPORT_SYMBOL(pm_power_off);
>
> void machine_halt(void)
> {
> +#ifdef CONFIG_SMP
> + preempt_disable();
> + smp_send_stop();
> +#endif
> default_halt();
> }
>
> void machine_power_off(void)
> {
> +#ifdef CONFIG_SMP
> + preempt_disable();
> + smp_send_stop();
> +#endif
> pm_power_off();
> }
>
> void machine_restart(char *command)
> {
> +#ifdef CONFIG_SMP
> + preempt_disable();
> + smp_send_stop();
> +#endif
> do_kernel_restart(command);
> pm_restart();
> }
> diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c
> index 29f3b82cd0a5..34a3011f679e 100644
> --- a/arch/loongarch/kernel/setup.c
> +++ b/arch/loongarch/kernel/setup.c
> @@ -38,6 +38,7 @@
> #include <asm/pgalloc.h>
> #include <asm/sections.h>
> #include <asm/setup.h>
> +#include <asm/smp.h>
> #include <asm/time.h>
>
> #define SMBIOS_BIOSSIZE_OFFSET 0x09
> @@ -322,6 +323,29 @@ static int __init reserve_memblock_reserved_regions(void)
> }
> arch_initcall(reserve_memblock_reserved_regions);
>
> +#ifdef CONFIG_SMP
> +static void __init prefill_possible_map(void)
> +{
> + int i, possible;
> +
> + possible = num_processors + disabled_cpus;
> + if (possible > nr_cpu_ids)
> + possible = nr_cpu_ids;
> +
> + pr_info("SMP: Allowing %d CPUs, %d hotplug CPUs\n",
> + possible, max((possible - num_processors), 0));
> +
> + for (i = 0; i < possible; i++)
> + set_cpu_possible(i, true);
> + for (; i < NR_CPUS; i++)
> + set_cpu_possible(i, false);
> +
> + nr_cpu_ids = possible;
> +}
> +#else
> +static inline void prefill_possible_map(void) {}
> +#endif
> +
> void __init setup_arch(char **cmdline_p)
> {
> cpu_probe();
> @@ -336,6 +360,8 @@ void __init setup_arch(char **cmdline_p)
> arch_mem_init(cmdline_p);
>
> resource_init();
> + plat_smp_setup();
> + prefill_possible_map();
>
> paging_init();
> }
> diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c
> new file mode 100644
> index 000000000000..99ba7a56edf9
> --- /dev/null
> +++ b/arch/loongarch/kernel/smp.c
> @@ -0,0 +1,735 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + *
> + * Derived from MIPS:
> + * Copyright (C) 2000, 2001 Kanoj Sarcar
> + * Copyright (C) 2000, 2001 Ralf Baechle
> + * Copyright (C) 2000, 2001 Silicon Graphics, Inc.
> + * Copyright (C) 2000, 2001, 2003 Broadcom Corporation
> + */
> +#include <linux/cpu.h>
> +#include <linux/cpumask.h>
> +#include <linux/init.h>
> +#include <linux/interrupt.h>
> +#include <linux/seq_file.h>
> +#include <linux/smp.h>
> +#include <linux/threads.h>
> +#include <linux/export.h>
> +#include <linux/time.h>
> +#include <linux/tracepoint.h>
> +#include <linux/sched/hotplug.h>
> +#include <linux/sched/task_stack.h>
> +
> +#include <asm/cpu.h>
> +#include <asm/idle.h>
> +#include <asm/loongson.h>
> +#include <asm/mmu_context.h>
> +#include <asm/processor.h>
> +#include <asm/setup.h>
> +#include <asm/time.h>
> +
> +int __cpu_number_map[NR_CPUS]; /* Map physical to logical */
> +EXPORT_SYMBOL(__cpu_number_map);
> +
> +int __cpu_logical_map[NR_CPUS]; /* Map logical to physical */
> +EXPORT_SYMBOL(__cpu_logical_map);
> +
> +/* Number of threads (siblings) per CPU core */
> +int smp_num_siblings = 1;
> +EXPORT_SYMBOL(smp_num_siblings);
> +
> +/* Representing the threads (siblings) of each logical CPU */
> +cpumask_t cpu_sibling_map[NR_CPUS] __read_mostly;
> +EXPORT_SYMBOL(cpu_sibling_map);
> +
> +/* Representing the core map of multi-core chips of each logical CPU */
> +cpumask_t cpu_core_map[NR_CPUS] __read_mostly;
> +EXPORT_SYMBOL(cpu_core_map);
> +
> +static DECLARE_COMPLETION(cpu_starting);
> +static DECLARE_COMPLETION(cpu_running);
> +
> +/*
> + * A logcal cpu mask containing only one VPE per core to
> + * reduce the number of IPIs on large MT systems.
> + */
> +cpumask_t cpu_foreign_map[NR_CPUS] __read_mostly;
> +EXPORT_SYMBOL(cpu_foreign_map);
> +
> +/* representing cpus for which sibling maps can be computed */
> +static cpumask_t cpu_sibling_setup_map;
> +
> +/* representing cpus for which core maps can be computed */
> +static cpumask_t cpu_core_setup_map;
> +
> +struct secondary_data cpuboot_data;
> +static DEFINE_PER_CPU(int, cpu_state);
> +DEFINE_PER_CPU_SHARED_ALIGNED(irq_cpustat_t, irq_stat);
> +EXPORT_PER_CPU_SYMBOL(irq_stat);
> +
> +enum ipi_msg_type {
> + IPI_RESCHEDULE,
> + IPI_CALL_FUNCTION,
> +};
> +
> +static const char *ipi_types[NR_IPI] __tracepoint_string = {
> + [IPI_RESCHEDULE] = "Rescheduling interrupts",
> + [IPI_CALL_FUNCTION] = "Function call interrupts",
> +};
> +
> +void show_ipi_list(struct seq_file *p, int prec)
> +{
> + unsigned int cpu, i;
> +
> + for (i = 0; i < NR_IPI; i++) {
> + seq_printf(p, "%*s%u:%s", prec - 1, "IPI", i, prec >= 4 ? " " : "");
> + for_each_online_cpu(cpu)
> + seq_printf(p, "%10u ", per_cpu(irq_stat, cpu).ipi_irqs[i]);
> + seq_printf(p, " LoongArch %d %s\n", i + 1, ipi_types[i]);
> + }
> +}
> +
> +/* Send mailbox buffer via Mail_Send */
> +static void csr_mail_send(uint64_t data, int cpu, int mailbox)
> +{
> + uint64_t val;
> +
> + /* Send high 32 bits */
> + val = IOCSR_MBUF_SEND_BLOCKING;
> + val |= (IOCSR_MBUF_SEND_BOX_HI(mailbox) << IOCSR_MBUF_SEND_BOX_SHIFT);
> + val |= (cpu << IOCSR_MBUF_SEND_CPU_SHIFT);
> + val |= (data & IOCSR_MBUF_SEND_H32_MASK);
> + iocsr_write64(val, LOONGARCH_IOCSR_MBUF_SEND);
> +
> + /* Send low 32 bits */
> + val = IOCSR_MBUF_SEND_BLOCKING;
> + val |= (IOCSR_MBUF_SEND_BOX_LO(mailbox) << IOCSR_MBUF_SEND_BOX_SHIFT);
> + val |= (cpu << IOCSR_MBUF_SEND_CPU_SHIFT);
> + val |= (data << IOCSR_MBUF_SEND_BUF_SHIFT);
> + iocsr_write64(val, LOONGARCH_IOCSR_MBUF_SEND);
> +};
> +
> +static u32 ipi_read_clear(int cpu)
> +{
> + u32 action;
> +
> + /* Load the ipi register to figure out what we're supposed to do */
> + action = iocsr_read32(LOONGARCH_IOCSR_IPI_STATUS);
> + /* Clear the ipi register to clear the interrupt */
> + iocsr_write32(action, LOONGARCH_IOCSR_IPI_CLEAR);
> + smp_mb();
> +
> + return action;
> +}
> +
> +static void ipi_write_action(int cpu, u32 action)
> +{
> + unsigned int irq = 0;
> +
> + while ((irq = ffs(action))) {
> + uint32_t val = IOCSR_IPI_SEND_BLOCKING;
> +
> + val |= (irq - 1);
> + val |= (cpu << IOCSR_IPI_SEND_CPU_SHIFT);
> + iocsr_write32(val, LOONGARCH_IOCSR_IPI_SEND);
> + action &= ~BIT(irq - 1);
> + }
> +}
> +
> +void loongson3_send_ipi_single(int cpu, unsigned int action)
> +{
> + ipi_write_action(cpu_logical_map(cpu), (u32)action);
> +}
> +
> +void loongson3_send_ipi_mask(const struct cpumask *mask, unsigned int
> action)
> +{
> + unsigned int i;
> +
> + for_each_cpu(i, mask)
> + ipi_write_action(cpu_logical_map(i), (u32)action);
> +}
> +
> +irqreturn_t loongson3_ipi_interrupt(int irq, void *dev)
> +{
> + unsigned int action;
> + unsigned int cpu = smp_processor_id();
> +
> + action = ipi_read_clear(cpu_logical_map(cpu));
> +
> + if (action & SMP_RESCHEDULE) {
> + scheduler_ipi();
> + per_cpu(irq_stat, cpu).ipi_irqs[IPI_RESCHEDULE]++;
> + }
> +
> + if (action & SMP_CALL_FUNCTION) {
> + generic_smp_call_function_interrupt();
> + per_cpu(irq_stat, cpu).ipi_irqs[IPI_CALL_FUNCTION]++;
> + }
> +
> + return IRQ_HANDLED;
> +}
> +
> +void __init loongson3_smp_setup(void)
> +{
> + cpu_data[0].core = cpu_logical_map(0) %
> loongson_sysconf.cores_per_package;
> + cpu_data[0].package = cpu_logical_map(0) /
> loongson_sysconf.cores_per_package;
> +
> + iocsr_write32(0xffffffff, LOONGARCH_IOCSR_IPI_EN);
> + pr_info("Detected %i available CPU(s)\n", loongson_sysconf.nr_cpus);
> +}
> +
> +void __init loongson3_prepare_cpus(unsigned int max_cpus)
> +{
> + int i = 0;
> +
> + for (i = 0; i < loongson_sysconf.nr_cpus; i++) {
> + set_cpu_present(i, true);
> + csr_mail_send(0, __cpu_logical_map[i], 0);
> + }
> +
> + per_cpu(cpu_state, smp_processor_id()) = CPU_ONLINE;
> +}
> +
> +/*
> + * Setup the PC, SP, and TP of a secondary processor and start it
> running!
> + */
> +void loongson3_boot_secondary(int cpu, struct task_struct *idle)
> +{
> + unsigned long entry;
> +
> + pr_info("Booting CPU#%d...\n", cpu);
> +
> + entry = __pa_symbol((unsigned long)&smpboot_entry);
> + cpuboot_data.stack = (unsigned long)__KSTK_TOS(idle);
> + cpuboot_data.thread_info = (unsigned long)task_thread_info(idle);
> +
> + csr_mail_send(entry, cpu_logical_map(cpu), 0);
> +
> + loongson3_send_ipi_single(cpu, SMP_BOOT_CPU);
> +}
> +
> +/*
> + * SMP init and finish on secondary CPUs
> + */
> +void loongson3_init_secondary(void)
> +{
> + unsigned int cpu = smp_processor_id();
> + unsigned int imask = ECFGF_IP0 | ECFGF_IP1 | ECFGF_IP2 |
> + ECFGF_IPI | ECFGF_PMC | ECFGF_TIMER;
> +
> + change_csr_ecfg(ECFG0_IM, imask);
> +
> + iocsr_write32(0xffffffff, LOONGARCH_IOCSR_IPI_EN);
> +
> + per_cpu(cpu_state, cpu) = CPU_ONLINE;
> + cpu_data[cpu].core =
> + cpu_logical_map(cpu) % loongson_sysconf.cores_per_package;
> + cpu_data[cpu].package =
> + cpu_logical_map(cpu) / loongson_sysconf.cores_per_package;
> +}
> +
> +void loongson3_smp_finish(void)
> +{
> + local_irq_enable();
> + iocsr_write64(0, LOONGARCH_IOCSR_MBUF0);
> + pr_info("CPU#%d finished\n", smp_processor_id());
> +}
> +
> +#ifdef CONFIG_HOTPLUG_CPU
> +
> +static bool io_master(int cpu)
> +{
> + if (cpu == 0)
> + return true;
> +
> + return false;
> +}
> +
> +int loongson3_cpu_disable(void)
> +{
> + unsigned long flags;
> + unsigned int cpu = smp_processor_id();
> +
> + if (io_master(cpu))
> + return -EBUSY;
> +
> + set_cpu_online(cpu, false);
> + calculate_cpu_foreign_map();
> + local_irq_save(flags);
> + irq_migrate_all_off_this_cpu();
> + clear_csr_ecfg(ECFG0_IM);
> + local_irq_restore(flags);
> + local_flush_tlb_all();
> +
> + return 0;
> +}
> +
> +void loongson3_cpu_die(unsigned int cpu)
> +{
> + while (per_cpu(cpu_state, cpu) != CPU_DEAD)
> + cpu_relax();
> +
> + mb();
> +}
> +
> +/*
> + * The target CPU should go to XKPRANGE (uncached area) and flush
> + * ICache/DCache/VCache before the control CPU can safely disable its
> clock.
> + */
> +static void loongson3_play_dead(int *state_addr)
> +{
> + register int val;
> + register void *addr;
> + register void (*init_fn)(void);
> +
> + __asm__ __volatile__(
> + " li.d %[addr], 0x8000000000000000\n"
> + "1: cacop 0x8, %[addr], 0 \n" /* flush ICache */
> + " cacop 0x8, %[addr], 1 \n"
> + " cacop 0x8, %[addr], 2 \n"
> + " cacop 0x8, %[addr], 3 \n"
> + " cacop 0x9, %[addr], 0 \n" /* flush DCache */
> + " cacop 0x9, %[addr], 1 \n"
> + " cacop 0x9, %[addr], 2 \n"
> + " cacop 0x9, %[addr], 3 \n"
> + " addi.w %[sets], %[sets], -1 \n"
> + " addi.d %[addr], %[addr], 0x40 \n"
> + " bnez %[sets], 1b \n"
> + " li.d %[addr], 0x8000000000000000\n"
> + "2: cacop 0xa, %[addr], 0 \n" /* flush VCache */
> + " cacop 0xa, %[addr], 1 \n"
> + " cacop 0xa, %[addr], 2 \n"
> + " cacop 0xa, %[addr], 3 \n"
> + " cacop 0xa, %[addr], 4 \n"
> + " cacop 0xa, %[addr], 5 \n"
> + " cacop 0xa, %[addr], 6 \n"
> + " cacop 0xa, %[addr], 7 \n"
> + " cacop 0xa, %[addr], 8 \n"
> + " cacop 0xa, %[addr], 9 \n"
> + " cacop 0xa, %[addr], 10 \n"
> + " cacop 0xa, %[addr], 11 \n"
> + " cacop 0xa, %[addr], 12 \n"
> + " cacop 0xa, %[addr], 13 \n"
> + " cacop 0xa, %[addr], 14 \n"
> + " cacop 0xa, %[addr], 15 \n"
> + " addi.w %[vsets], %[vsets], -1 \n"
> + " addi.d %[addr], %[addr], 0x40 \n"
> + " bnez %[vsets], 2b \n"
> + " li.w %[val], 0x7 \n" /* *state_addr = CPU_DEAD; */
> + " st.w %[val], %[state_addr], 0 \n"
> + " dbar 0 \n"
> + " cacop 0x11, %[state_addr], 0 \n" /* flush entry of
> *state_addr */
> + : [addr] "=&r" (addr), [val] "=&r" (val)
> + : [state_addr] "r" (state_addr),
> + [sets] "r" (cpu_data[smp_processor_id()].dcache.sets),
> + [vsets] "r" (cpu_data[smp_processor_id()].vcache.sets));
> +
> + local_irq_enable();
> + change_csr_ecfg(ECFG0_IM, ECFGF_IPI);
> +
> + __asm__ __volatile__(
> + " idle 0 \n"
> + " li.w $t0, 0x1020 \n"
> + " iocsrrd.d %[init_fn], $t0 \n" /* Get init PC */
> + : [init_fn] "=&r" (addr)
> + : /* No Input */
> + : "a0");
> + init_fn = __va(addr);
> +
> + init_fn();
> + unreachable();
> +}
> +
> +void play_dead(void)
> +{
> + int *state_addr;
> + unsigned int cpu = smp_processor_id();
> + void (*play_dead_uncached)(int *s);
> +
> + idle_task_exit();
> + play_dead_uncached = (void *)TO_UNCACHE(__pa((unsigned
> long)loongson3_play_dead));
> + state_addr = &per_cpu(cpu_state, cpu);
> + mb();
> + play_dead_uncached(state_addr);
> +}
> +
> +static int loongson3_enable_clock(unsigned int cpu)
> +{
> + uint64_t core_id = cpu_data[cpu].core;
> + uint64_t package_id = cpu_data[cpu].package;
> +
> + LOONGSON_FREQCTRL(package_id) |= 1 << (core_id * 4 + 3);
> +
> + return 0;
> +}
> +
> +static int loongson3_disable_clock(unsigned int cpu)
> +{
> + uint64_t core_id = cpu_data[cpu].core;
> + uint64_t package_id = cpu_data[cpu].package;
> +
> + LOONGSON_FREQCTRL(package_id) &= ~(1 << (core_id * 4 + 3));
> +
> + return 0;
> +}
> +
> +static int register_loongson3_notifier(void)
> +{
> + return cpuhp_setup_state_nocalls(CPUHP_LOONGARCH_SOC_PREPARE,
> + "loongarch/loongson:prepare",
> + loongson3_enable_clock,
> + loongson3_disable_clock);
> +}
> +early_initcall(register_loongson3_notifier);
> +
> +#endif
> +
> +/*
> + * Power management
> + */
> +#ifdef CONFIG_PM
> +
> +static int loongson3_ipi_suspend(void)
> +{
> + return 0;
> +}
> +
> +static void loongson3_ipi_resume(void)
> +{
> + iocsr_write32(0xffffffff, LOONGARCH_IOCSR_IPI_EN);
> +}
> +
> +static struct syscore_ops loongson3_ipi_syscore_ops = {
> + .resume = loongson3_ipi_resume,
> + .suspend = loongson3_ipi_suspend,
> +};
> +
> +/*
> + * Enable boot cpu ipi before enabling nonboot cpus
> + * during syscore_resume.
> + */
> +static int __init ipi_pm_init(void)
> +{
> + register_syscore_ops(&loongson3_ipi_syscore_ops);
> + return 0;
> +}
> +
> +core_initcall(ipi_pm_init);
> +#endif
> +
> +static inline void set_cpu_sibling_map(int cpu)
> +{
> + int i;
> +
> + cpumask_set_cpu(cpu, &cpu_sibling_setup_map);
> +
> + if (smp_num_siblings <= 1)
> + cpumask_set_cpu(cpu, &cpu_sibling_map[cpu]);
> + else {
> + for_each_cpu(i, &cpu_sibling_setup_map) {
> + if (cpus_are_siblings(cpu, i)) {
> + cpumask_set_cpu(i, &cpu_sibling_map[cpu]);
> + cpumask_set_cpu(cpu, &cpu_sibling_map[i]);
> + }
> + }
> + }
> +}
> +
> +static inline void set_cpu_core_map(int cpu)
> +{
> + int i;
> +
> + cpumask_set_cpu(cpu, &cpu_core_setup_map);
> +
> + for_each_cpu(i, &cpu_core_setup_map) {
> + if (cpu_data[cpu].package == cpu_data[i].package) {
> + cpumask_set_cpu(i, &cpu_core_map[cpu]);
> + cpumask_set_cpu(cpu, &cpu_core_map[i]);
> + }
> + }
> +}
> +
> +/*
> + * Calculate a new cpu_foreign_map mask whenever a
> + * new cpu appears or disappears.
> + */
> +void calculate_cpu_foreign_map(void)
> +{
> + int i, k, core_present;
> + cpumask_t temp_foreign_map;
> +
> + /* Re-calculate the mask */
> + cpumask_clear(&temp_foreign_map);
> + for_each_online_cpu(i) {
> + core_present = 0;
> + for_each_cpu(k, &temp_foreign_map)
> + if (cpus_are_siblings(i, k))
> + core_present = 1;
> + if (!core_present)
> + cpumask_set_cpu(i, &temp_foreign_map);
> + }
> +
> + for_each_online_cpu(i)
> + cpumask_andnot(&cpu_foreign_map[i],
> + &temp_foreign_map, &cpu_sibling_map[i]);
> +}
> +
> +/* Preload SMP state for boot cpu */
> +void smp_prepare_boot_cpu(void)
> +{
> + unsigned int cpu;
> +
> + set_cpu_possible(0, true);
> + set_cpu_online(0, true);
> + set_my_cpu_offset(per_cpu_offset(0));
> +
> + for_each_possible_cpu(cpu)
> + set_cpu_numa_node(cpu, 0);
> +}
> +
> +/* called from main before smp_init() */
> +void __init smp_prepare_cpus(unsigned int max_cpus)
> +{
> + init_new_context(current, &init_mm);
> + current_thread_info()->cpu = 0;
> + loongson3_prepare_cpus(max_cpus);
> + set_cpu_sibling_map(0);
> + set_cpu_core_map(0);
> + calculate_cpu_foreign_map();
> +#ifndef CONFIG_HOTPLUG_CPU
> + init_cpu_present(cpu_possible_mask);
> +#endif
> +}
> +
> +int __cpu_up(unsigned int cpu, struct task_struct *tidle)
> +{
> + loongson3_boot_secondary(cpu, tidle);
> +
> + /* Wait for CPU to start and be ready to sync counters */
> + if (!wait_for_completion_timeout(&cpu_starting,
> + msecs_to_jiffies(5000))) {
> + pr_crit("CPU%u: failed to start\n", cpu);
> + return -EIO;
> + }
> +
> + /* Wait for CPU to finish startup & mark itself online before return
> */
> + wait_for_completion(&cpu_running);
> +
> + return 0;
> +}
> +
> +/*
> + * First C code run on the secondary CPUs after being started up by
> + * the master.
> + */
> +asmlinkage void start_secondary(void)
> +{
> + unsigned int cpu;
> +
> + sync_counter();
> + cpu = smp_processor_id();
> + set_my_cpu_offset(per_cpu_offset(cpu));
> +
> + cpu_probe();
> + constant_clockevent_init();
> + loongson3_init_secondary();
> +
> + set_cpu_sibling_map(cpu);
> + set_cpu_core_map(cpu);
> +
> + notify_cpu_starting(cpu);
> +
> + /* Notify boot CPU that we're starting */
> + complete(&cpu_starting);
> +
> + /* The CPU is running, now mark it online */
> + set_cpu_online(cpu, true);
> +
> + calculate_cpu_foreign_map();
> +
> + /*
> + * Notify boot CPU that we're up & online and it can safely return
> + * from __cpu_up()
> + */
> + complete(&cpu_running);
> +
> + /*
> + * irq will be enabled in loongson3_smp_finish(), enabling it too
> + * early is dangerous.
> + */
> + WARN_ON_ONCE(!irqs_disabled());
> + loongson3_smp_finish();
> +
> + cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
> +}
> +
> +void __init smp_cpus_done(unsigned int max_cpus)
> +{
> +}
> +
> +static void stop_this_cpu(void *dummy)
> +{
> + set_cpu_online(smp_processor_id(), false);
> + calculate_cpu_foreign_map();
> + local_irq_disable();
> + while (true);
> +}
> +
> +void smp_send_stop(void)
> +{
> + smp_call_function(stop_this_cpu, NULL, 0);
> +}
> +
> +int setup_profiling_timer(unsigned int multiplier)
> +{
> + return 0;
> +}
> +
> +static void flush_tlb_all_ipi(void *info)
> +{
> + local_flush_tlb_all();
> +}
> +
> +void flush_tlb_all(void)
> +{
> + on_each_cpu(flush_tlb_all_ipi, NULL, 1);
> +}
> +
> +static void flush_tlb_mm_ipi(void *mm)
> +{
> + local_flush_tlb_mm((struct mm_struct *)mm);
> +}
> +
> +void flush_tlb_mm(struct mm_struct *mm)
> +{
> + if (atomic_read(&mm->mm_users) == 0)
> + return; /* happens as a result of exit_mmap() */
> +
> + preempt_disable();
> +
> + if ((atomic_read(&mm->mm_users) != 1) || (current->mm != mm)) {
> + on_each_cpu_mask(mm_cpumask(mm), flush_tlb_mm_ipi, mm, 1);
> + } else {
> + unsigned int cpu;
> +
> + for_each_online_cpu(cpu) {
> + if (cpu != smp_processor_id() && cpu_context(cpu, mm))
> + cpu_context(cpu, mm) = 0;
> + }
> + local_flush_tlb_mm(mm);
> + }
> +
> + preempt_enable();
> +}
> +
> +struct flush_tlb_data {
> + struct vm_area_struct *vma;
> + unsigned long addr1;
> + unsigned long addr2;
> +};
> +
> +static void flush_tlb_range_ipi(void *info)
> +{
> + struct flush_tlb_data *fd = info;
> +
> + local_flush_tlb_range(fd->vma, fd->addr1, fd->addr2);
> +}
> +
> +void flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
> unsigned long end)
> +{
> + struct mm_struct *mm = vma->vm_mm;
> +
> + preempt_disable();
> + if ((atomic_read(&mm->mm_users) != 1) || (current->mm != mm)) {
> + struct flush_tlb_data fd = {
> + .vma = vma,
> + .addr1 = start,
> + .addr2 = end,
> + };
> +
> + on_each_cpu_mask(mm_cpumask(mm), flush_tlb_range_ipi, &fd, 1);
> + } else {
> + unsigned int cpu;
> + int exec = vma->vm_flags & VM_EXEC;
> +
> + for_each_online_cpu(cpu) {
> + /*
> + * flush_cache_range() will only fully flush icache if
> + * the VMA is executable, otherwise we must invalidate
> + * ASID without it appearing to has_valid_asid() as if
> + * mm has been completely unused by that CPU.
> + */
> + if (cpu != smp_processor_id() && cpu_context(cpu, mm))
> + cpu_context(cpu, mm) = !exec;
> + }
> + local_flush_tlb_range(vma, start, end);
> + }
> + preempt_enable();
> +}
> +
> +static void flush_tlb_kernel_range_ipi(void *info)
> +{
> + struct flush_tlb_data *fd = info;
> +
> + local_flush_tlb_kernel_range(fd->addr1, fd->addr2);
> +}
> +
> +void flush_tlb_kernel_range(unsigned long start, unsigned long end)
> +{
> + struct flush_tlb_data fd = {
> + .addr1 = start,
> + .addr2 = end,
> + };
> +
> + on_each_cpu(flush_tlb_kernel_range_ipi, &fd, 1);
> +}
> +
> +static void flush_tlb_page_ipi(void *info)
> +{
> + struct flush_tlb_data *fd = info;
> +
> + local_flush_tlb_page(fd->vma, fd->addr1);
> +}
> +
> +void flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
> +{
> + preempt_disable();
> + if ((atomic_read(&vma->vm_mm->mm_users) != 1) || (current->mm !=
> vma->vm_mm)) {
> + struct flush_tlb_data fd = {
> + .vma = vma,
> + .addr1 = page,
> + };
> +
> + on_each_cpu_mask(mm_cpumask(vma->vm_mm), flush_tlb_page_ipi, &fd, 1);
> + } else {
> + unsigned int cpu;
> +
> + for_each_online_cpu(cpu) {
> + /*
> + * flush_cache_page() only does partial flushes, so
> + * invalidate ASID without it appearing to
> + * has_valid_asid() as if mm has been completely unused
> + * by that CPU.
> + */
> + if (cpu != smp_processor_id() && cpu_context(cpu, vma->vm_mm))
> + cpu_context(cpu, vma->vm_mm) = 1;
> + }
> + local_flush_tlb_page(vma, page);
> + }
> + preempt_enable();
> +}
> +EXPORT_SYMBOL(flush_tlb_page);
> +
> +static void flush_tlb_one_ipi(void *info)
> +{
> + unsigned long vaddr = (unsigned long) info;
> +
> + local_flush_tlb_one(vaddr);
> +}
> +
> +void flush_tlb_one(unsigned long vaddr)
> +{
> + on_each_cpu(flush_tlb_one_ipi, (void *)vaddr, 1);
> +}
> +EXPORT_SYMBOL(flush_tlb_one);
> diff --git a/arch/loongarch/kernel/topology.c
> b/arch/loongarch/kernel/topology.c
> index 3b2cbb95875b..ab1a75c0b5a6 100644
> --- a/arch/loongarch/kernel/topology.c
> +++ b/arch/loongarch/kernel/topology.c
> @@ -1,13 +1,52 @@
> // SPDX-License-Identifier: GPL-2.0
> #include <linux/cpu.h>
> +#include <linux/cpumask.h>
> #include <linux/init.h>
> +#include <linux/node.h>
> +#include <linux/nodemask.h>
> #include <linux/percpu.h>
>
> -static struct cpu cpu_device;
> +static DEFINE_PER_CPU(struct cpu, cpu_devices);
> +
> +#ifdef CONFIG_HOTPLUG_CPU
> +int arch_register_cpu(int cpu)
> +{
> + int ret;
> + struct cpu *c = &per_cpu(cpu_devices, cpu);
> +
> + c->hotpluggable = 1;
> + ret = register_cpu(c, cpu);
> + if (ret < 0)
> + pr_warn("register_cpu %d failed (%d)\n", cpu, ret);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL(arch_register_cpu);
> +
> +void arch_unregister_cpu(int cpu)
> +{
> + struct cpu *c = &per_cpu(cpu_devices, cpu);
> +
> + c->hotpluggable = 0;
> + unregister_cpu(c);
> +}
> +EXPORT_SYMBOL(arch_unregister_cpu);
> +#endif
>
> static int __init topology_init(void)
> {
> - return register_cpu(&cpu_device, 0);
> + int i, ret;
> +
> + for_each_present_cpu(i) {
> + struct cpu *c = &per_cpu(cpu_devices, i);
> +
> + c->hotpluggable = !!i;
> + ret = register_cpu(c, i);
> + if (ret < 0)
> + pr_warn("topology_init: register_cpu %d failed (%d)\n", i, ret);
> + }
> +
> + return 0;
> }
>
> subsys_initcall(topology_init);
> diff --git a/arch/loongarch/kernel/vmlinux.lds.S
> b/arch/loongarch/kernel/vmlinux.lds.S
> index 7da4c4d7c50d..006cbb1bd5c6 100644
> --- a/arch/loongarch/kernel/vmlinux.lds.S
> +++ b/arch/loongarch/kernel/vmlinux.lds.S
> @@ -73,6 +73,10 @@ SECTIONS
> EXIT_DATA
> }
>
> +#ifdef CONFIG_SMP
> + PERCPU_SECTION(1 << CONFIG_L1_CACHE_SHIFT)
> +#endif
> +
> .init.bss : {
> *(.init.bss)
> }
> diff --git a/arch/loongarch/mm/tlbex.S b/arch/loongarch/mm/tlbex.S
> index bef740710a3b..7eee40271577 100644
> --- a/arch/loongarch/mm/tlbex.S
> +++ b/arch/loongarch/mm/tlbex.S
> @@ -88,7 +88,14 @@ vmalloc_done_load:
> slli.d t0, t0, _PTE_T_LOG2
> add.d t1, ra, t0
>
> +#ifdef CONFIG_SMP
> +smp_pgtable_change_load:
> +#endif
> +#ifdef CONFIG_SMP
> + ll.d t0, t1, 0
> +#else
> ld.d t0, t1, 0
> +#endif
> tlbsrch
>
> srli.d ra, t0, _PAGE_PRESENT_SHIFT
> @@ -96,7 +103,12 @@ vmalloc_done_load:
> beq ra, $r0, nopage_tlb_load
>
> ori t0, t0, _PAGE_VALID
> +#ifdef CONFIG_SMP
> + sc.d t0, t1, 0
> + beq t0, $r0, smp_pgtable_change_load
> +#else
> st.d t0, t1, 0
> +#endif
> ori t1, t1, 8
> xori t1, t1, 8
> ld.d t0, t1, 0
> @@ -120,14 +132,24 @@ vmalloc_load:
> * spots a huge page.
> */
> tlb_huge_update_load:
> +#ifdef CONFIG_SMP
> + ll.d t0, t1, 0
> +#else
> ld.d t0, t1, 0
> +#endif
> srli.d ra, t0, _PAGE_PRESENT_SHIFT
> andi ra, ra, 1
> beq ra, $r0, nopage_tlb_load
> tlbsrch
>
> ori t0, t0, _PAGE_VALID
> +#ifdef CONFIG_SMP
> + sc.d t0, t1, 0
> + beq t0, $r0, tlb_huge_update_load
> + ld.d t0, t1, 0
> +#else
> st.d t0, t1, 0
> +#endif
> addu16i.d t1, $r0, -(CSR_TLBIDX_EHINV >> 16)
> addi.d ra, t1, 0
> csrxchg ra, t1, LOONGARCH_CSR_TLBIDX
> @@ -173,6 +195,7 @@ tlb_huge_update_load:
> csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
>
> nopage_tlb_load:
> + dbar 0
> csrrd ra, EXCEPTION_KS2
> la.abs t0, tlb_do_page_fault_0
> jirl $r0, t0, 0
> @@ -229,7 +252,14 @@ vmalloc_done_store:
> slli.d t0, t0, _PTE_T_LOG2
> add.d t1, ra, t0
>
> +#ifdef CONFIG_SMP
> +smp_pgtable_change_store:
> +#endif
> +#ifdef CONFIG_SMP
> + ll.d t0, t1, 0
> +#else
> ld.d t0, t1, 0
> +#endif
> tlbsrch
>
> srli.d ra, t0, _PAGE_PRESENT_SHIFT
> @@ -238,7 +268,12 @@ vmalloc_done_store:
> bne ra, $r0, nopage_tlb_store
>
> ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
> +#ifdef CONFIG_SMP
> + sc.d t0, t1, 0
> + beq t0, $r0, smp_pgtable_change_store
> +#else
> st.d t0, t1, 0
> +#endif
>
> ori t1, t1, 8
> xori t1, t1, 8
> @@ -263,7 +298,11 @@ vmalloc_store:
> * spots a huge page.
> */
> tlb_huge_update_store:
> +#ifdef CONFIG_SMP
> + ll.d t0, t1, 0
> +#else
> ld.d t0, t1, 0
> +#endif
> srli.d ra, t0, _PAGE_PRESENT_SHIFT
> andi ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
> xori ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
> @@ -272,7 +311,13 @@ tlb_huge_update_store:
> tlbsrch
> ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
>
> +#ifdef CONFIG_SMP
> + sc.d t0, t1, 0
> + beq t0, $r0, tlb_huge_update_store
> + ld.d t0, t1, 0
> +#else
> st.d t0, t1, 0
> +#endif
> addu16i.d t1, $r0, -(CSR_TLBIDX_EHINV >> 16)
> addi.d ra, t1, 0
> csrxchg ra, t1, LOONGARCH_CSR_TLBIDX
> @@ -318,6 +363,7 @@ tlb_huge_update_store:
> csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
>
> nopage_tlb_store:
> + dbar 0
> csrrd ra, EXCEPTION_KS2
> la.abs t0, tlb_do_page_fault_1
> jirl $r0, t0, 0
> @@ -373,7 +419,14 @@ vmalloc_done_modify:
> slli.d t0, t0, _PTE_T_LOG2
> add.d t1, ra, t0
>
> +#ifdef CONFIG_SMP
> +smp_pgtable_change_modify:
> +#endif
> +#ifdef CONFIG_SMP
> + ll.d t0, t1, 0
> +#else
> ld.d t0, t1, 0
> +#endif
> tlbsrch
>
> srli.d ra, t0, _PAGE_WRITE_SHIFT
> @@ -381,7 +434,12 @@ vmalloc_done_modify:
> beq ra, $r0, nopage_tlb_modify
>
> ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
> +#ifdef CONFIG_SMP
> + sc.d t0, t1, 0
> + beq t0, $r0, smp_pgtable_change_modify
> +#else
> st.d t0, t1, 0
> +#endif
> ori t1, t1, 8
> xori t1, t1, 8
> ld.d t0, t1, 0
> @@ -405,7 +463,11 @@ vmalloc_modify:
> * build_tlbchange_handler_head spots a huge page.
> */
> tlb_huge_update_modify:
> +#ifdef CONFIG_SMP
> + ll.d t0, t1, 0
> +#else
> ld.d t0, t1, 0
> +#endif
>
> srli.d ra, t0, _PAGE_WRITE_SHIFT
> andi ra, ra, 1
> @@ -414,7 +476,13 @@ tlb_huge_update_modify:
> tlbsrch
> ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
>
> +#ifdef CONFIG_SMP
> + sc.d t0, t1, 0
> + beq t0, $r0, tlb_huge_update_modify
> + ld.d t0, t1, 0
> +#else
> st.d t0, t1, 0
> +#endif
> /*
> * A huge PTE describes an area the size of the
> * configured huge page size. This is twice the
> @@ -454,6 +522,7 @@ tlb_huge_update_modify:
> csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
>
> nopage_tlb_modify:
> + dbar 0
> csrrd ra, EXCEPTION_KS2
> la.abs t0, tlb_do_page_fault_1
> jirl $r0, t0, 0
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index b66c5f389159..19f0dbfdd7fe 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -130,6 +130,7 @@ enum cpuhp_state {
> CPUHP_ZCOMP_PREPARE,
> CPUHP_TIMERS_PREPARE,
> CPUHP_MIPS_SOC_PREPARE,
> + CPUHP_LOONGARCH_SOC_PREPARE,
> CPUHP_BP_PREPARE_DYN,
> CPUHP_BP_PREPARE_DYN_END = CPUHP_BP_PREPARE_DYN + 20,
> CPUHP_BRINGUP_CPU,
> --
> 2.27.0
--
- Jiaxun
Hi, Bagas,
On Thu, Jun 2, 2022 at 5:13 PM Bagas Sanjaya <[email protected]> wrote:
>
> On Wed, Jun 01, 2022 at 05:59:44PM +0800, Huacai Chen wrote:
>
> > +Note: The register ``$r21`` is reserved in the ELF psABI, but used by the Linux
> > +kernel for storing the percpu base address. It normally has no ABI name, but is
> > +called ``$u0`` in the kernel. You may also see ``$v0`` or ``$v1`` in some old code,
> > +they are deprecated aliases of ``$a0`` and ``$a1`` respectively.
>
> A nitpick: instead of just comma (,), also use "however" conjunction, that
> is "You may also see ..., however they ... ."
OK, thanks.
Huacai
>
> > +
> > +Note: You may see ``$fv0`` or ``$fv1`` in some old code, they are deprecated
> > +aliases of ``$fa0`` and ``$fa1`` respectively.
> > +
>
> The nitpick above also applies here, too.
>
> Otherwise, htmldocs built successfully without any new warnings.
>
> Tested-by: Bagas Sanjaya <[email protected]>
>
> --
> An old man doll... just what I always wanted! - Clara
On Wed, Jun 01, 2022 at 05:59:44PM +0800, Huacai Chen wrote:
> +Note: The register ``$r21`` is reserved in the ELF psABI, but used by the Linux
> +kernel for storing the percpu base address. It normally has no ABI name, but is
> +called ``$u0`` in the kernel. You may also see ``$v0`` or ``$v1`` in some old code,
> +they are deprecated aliases of ``$a0`` and ``$a1`` respectively.
A nitpick: instead of just comma (,), also use "however" conjunction, that
is "You may also see ..., however they ... ."
> +
> +Note: You may see ``$fv0`` or ``$fv1`` in some old code, they are deprecated
> +aliases of ``$fa0`` and ``$fa1`` respectively.
> +
The nitpick above also applies here, too.
Otherwise, htmldocs built successfully without any new warnings.
Tested-by: Bagas Sanjaya <[email protected]>
--
An old man doll... just what I always wanted! - Clara