2022-05-18 09:32:57

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 00/22] arch: Add basic LoongArch support

LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V.
LoongArch includes a reduced 32-bit version (LA32R), a standard 32-bit
version (LA32S) and a 64-bit version (LA64). LoongArch use ACPI as its
boot protocol LoongArch-specific interrupt controllers (similar to APIC)
are already added in the next revision of ACPI Specification (current
revision is 6.4).

This patchset is adding basic LoongArch support in mainline kernel, we
can see a complete snapshot here:
https://github.com/loongson/linux/tree/loongarch-next
https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git/log/?h=loongarch-next

Cross-compile tool chain to build kernel:
https://github.com/loongson/build-tools/releases/download/2021.12.21/loongarch64-clfs-2022-03-03-cross-tools-gcc-glibc.tar.xz

A CLFS-based Linux distro:
https://github.com/loongson/build-tools/releases/download/2021.12.21/loongarch64-clfs-system-2022-03-03.tar.bz2

Open-source tool chain which is under review (Binutils and Gcc are already upstream):
https://github.com/loongson/binutils-gdb/tree/upstream_v3.1
https://github.com/loongson/gcc/tree/loongarch_upstream_v6.3
https://github.com/loongson/glibc/tree/loongarch_2_35_dev_v2.2

Loongson and LoongArch documentations:
https://github.com/loongson/LoongArch-Documentation

LoongArch-specific interrupt controllers:
https://mantis.uefi.org/mantis/view.php?id=2203
https://mantis.uefi.org/mantis/view.php?id=2313

V1 -> V2:
1, Add documentation patches;
2, Restore copyright statements;
3, Split the big header patch;
4, Cleanup signal-related headers;
5, Cleanup incomplete 32-bit support;
6, Move the major PCI work to drivers/pci;
7, Rework Loongson64 platform support;
8, Rework lpj and __udelay()/__ndelay();
9, Rework page table layout config options;
10, Rework syscall/exception/interrupt with generic entry framework;
11, Simplify the VDSO/VSYSCALL implementation;
12, Use generic I/O access macros and functions;
13, Remove unaligned access emulation at present;
14, Keep clocksource code in arch since it is the "native clocksource";
15, Some other minor fixes and improvements.

V2 -> V3:
1, Rebased on 5.15-rc1;
2, Cleanup PCI code on V2;
3, Support multiple msi domain;
4, Support cacheable ioremap();
5, Use irq stack for interrupt handling;
6, Adjust struct ucontext and rt_sigframe;
7, Some other minor fixes and improvements.

V3 -> V4:
1, Rebased on 5.15-rc3;
2, Rework SMP support and remove legacy IPI;
3, Rework signal support and remove loongarch_abi;
4, Simplify phys_to_dma() and dma_to_phys();
5, Remove unused sys_mmap2() implementation;
6, Remove unused strncpy_user.S and strnlen_user.S;
7, Some other minor fixes and improvements.

V4 -> V5:
1, Rebased on 5.15-rc4;
2, Fix a _PAGE_CHG_MASK bug;
3, Fix vdso_base() calculation;
4, Adjust syscall and ptrace code;
5, Use generic bitops implementation;
6, Avoid syscall restart handling in sys_rt_sigreturn();
7, Update commit messages.

V5 -> V6:
1, Rebased on 5.17-rc4;
2, Use GENERIC_IRQ_MIGRATION;
3, Improve sigcontext definition;
4, Improve numa_default_distance();
5, Increse MINSIGSTKSZ and SIGSTKSZ;
6, Restruct pt_regs and user_pt_regs;
7, Fix a corner case of protection_map;
8, Fix some corner cases of system calls;
9, Separate module region and vmalloc region;
10, Rename registers to match official documents.

V6 -> V7:
1, Rebased on 5.17-rc6;
2, Refactor do_page_fault();
3, Adjust memblock initialization;
4, Use -mstrict-align to build kernel;
5, Reimplement elf_read_implies_exec() as other archs;
6, Some other minor fixes and improvements.

V7 -> V8:
1, Rebased on 5.17-rc8;
2, Remove useless abidefs.h;
3, Remove useless HAVE_ARCH_NODEDATA_EXTENSION;
4, Fix and simplify uaccess.h;
5, Fix bugs after pt_regs restruction;
6, Use generic copy_from_user_page()/copy_to_user_page();
7, Some other minor fixes and improvements.

V8 -> V9:
1, Rebased on 5.18-rc4;
2, Fix 4-level page tables;
3, Always use 16KB kernel stack;
4, Add efistub and zboot support;
5, Remove useless HAVE_FUTEX_CMPXCHG;
6, Optimize module machanism for size;
7, Define copy_user_page() to avoid build errors;
8, Some other minor fixes and improvements.

V9 -> V10:
1, Rebased on 5.18-rc6;
2, Use generic efi stub;
3, Use generic string library;
4, Use generic ticket spinlock;
5, Use more meaningful macro naming;
6, Remove the zboot patch;
7, Fix commit message and documentations;
8, Some other minor fixes and improvements.

V10 -> V11:
1, Rebased on asm-generic tree;
2, Fix fpreg macros definition;
3, Fix ELF ABI macros definition;
4, Fix magic number definition in efi header;
5, Remove unneeded swab.h, bitfield.h and rtc.c;
6, Remove __ARCH_WANT_NEW_STAT (glibc need update);
7, Improve documents as WANG Xuerui suggested;
8, Some other minor fixes and improvements.

Huacai Chen(22):
Documentation: LoongArch: Add basic documentations.
Documentation/zh_CN: Add basic LoongArch documentations.
LoongArch: Add elf-related definitions.
LoongArch: Add writecombine support for drm.
LoongArch: Add build infrastructure.
LoongArch: Add CPU definition headers.
LoongArch: Add atomic/locking headers.
LoongArch: Add other common headers.
LoongArch: Add boot and setup routines.
LoongArch: Add exception/interrupt handling.
LoongArch: Add process management.
LoongArch: Add memory management.
LoongArch: Add system call support.
LoongArch: Add signal handling support.
LoongArch: Add elf and module support.
LoongArch: Add misc common routines.
LoongArch: Add some library functions.
LoongArch: Add PCI controller support.
LoongArch: Add VDSO and VSYSCALL support.
LoongArch: Add multi-processor (SMP) support.
LoongArch: Add Non-Uniform Memory Access (NUMA) support.
LoongArch: Add Loongson-3 default config file.

Signed-off-by: Huacai Chen <[email protected]>
---
Documentation/arch.rst | 1 +
Documentation/loongarch/features.rst | 3 +
Documentation/loongarch/index.rst | 21 +
Documentation/loongarch/introduction.rst | 345 +++++
Documentation/loongarch/irq-chip-model.rst | 168 +++
Documentation/translations/zh_CN/index.rst | 1 +
.../translations/zh_CN/loongarch/features.rst | 8 +
.../translations/zh_CN/loongarch/index.rst | 26 +
.../translations/zh_CN/loongarch/introduction.rst | 318 ++++
.../zh_CN/loongarch/irq-chip-model.rst | 167 +++
arch/loongarch/.gitignore | 9 +
arch/loongarch/Kbuild | 6 +
arch/loongarch/Kconfig | 434 ++++++
arch/loongarch/Kconfig.debug | 0
arch/loongarch/Makefile | 131 ++
arch/loongarch/boot/Makefile | 78 +
arch/loongarch/boot/boot.lds.S | 64 +
arch/loongarch/boot/decompress.c | 98 ++
arch/loongarch/boot/string.c | 166 +++
arch/loongarch/boot/zheader.S | 100 ++
arch/loongarch/boot/zkernel.S | 99 ++
arch/loongarch/configs/loongson3_defconfig | 770 ++++++++++
arch/loongarch/include/asm/Kbuild | 29 +
arch/loongarch/include/asm/acenv.h | 17 +
arch/loongarch/include/asm/acpi.h | 38 +
arch/loongarch/include/asm/addrspace.h | 110 ++
arch/loongarch/include/asm/asm-offsets.h | 5 +
arch/loongarch/include/asm/asm-prototypes.h | 7 +
arch/loongarch/include/asm/asm.h | 190 +++
arch/loongarch/include/asm/asmmacro.h | 294 ++++
arch/loongarch/include/asm/atomic.h | 362 +++++
arch/loongarch/include/asm/barrier.h | 159 ++
arch/loongarch/include/asm/bitops.h | 33 +
arch/loongarch/include/asm/bitrev.h | 34 +
arch/loongarch/include/asm/boot_param.h | 97 ++
arch/loongarch/include/asm/bootinfo.h | 30 +
arch/loongarch/include/asm/branch.h | 21 +
arch/loongarch/include/asm/bug.h | 23 +
arch/loongarch/include/asm/cache.h | 13 +
arch/loongarch/include/asm/cacheflush.h | 80 +
arch/loongarch/include/asm/cacheops.h | 37 +
arch/loongarch/include/asm/clocksource.h | 12 +
arch/loongarch/include/asm/cmpxchg.h | 136 ++
arch/loongarch/include/asm/compiler.h | 15 +
arch/loongarch/include/asm/cpu-features.h | 69 +
arch/loongarch/include/asm/cpu-info.h | 136 ++
arch/loongarch/include/asm/cpu.h | 127 ++
arch/loongarch/include/asm/cpufeature.h | 24 +
arch/loongarch/include/asm/delay.h | 26 +
arch/loongarch/include/asm/dma-direct.h | 11 +
arch/loongarch/include/asm/dma.h | 13 +
arch/loongarch/include/asm/dmi.h | 24 +
arch/loongarch/include/asm/efi.h | 33 +
arch/loongarch/include/asm/elf.h | 299 ++++
arch/loongarch/include/asm/entry-common.h | 13 +
arch/loongarch/include/asm/exec.h | 10 +
arch/loongarch/include/asm/fb.h | 23 +
arch/loongarch/include/asm/fixmap.h | 13 +
arch/loongarch/include/asm/fpregdef.h | 49 +
arch/loongarch/include/asm/fpu.h | 129 ++
arch/loongarch/include/asm/futex.h | 108 ++
arch/loongarch/include/asm/fw.h | 18 +
arch/loongarch/include/asm/hardirq.h | 26 +
arch/loongarch/include/asm/hugetlb.h | 79 +
arch/loongarch/include/asm/hw_irq.h | 17 +
arch/loongarch/include/asm/idle.h | 9 +
arch/loongarch/include/asm/inst.h | 63 +
arch/loongarch/include/asm/io.h | 129 ++
arch/loongarch/include/asm/irq.h | 133 ++
arch/loongarch/include/asm/irq_regs.h | 27 +
arch/loongarch/include/asm/irqflags.h | 78 +
arch/loongarch/include/asm/kdebug.h | 23 +
arch/loongarch/include/asm/linkage.h | 36 +
arch/loongarch/include/asm/local.h | 138 ++
arch/loongarch/include/asm/loongarch.h | 1528 ++++++++++++++++++++
arch/loongarch/include/asm/loongson.h | 159 ++
arch/loongarch/include/asm/mmu.h | 16 +
arch/loongarch/include/asm/mmu_context.h | 152 ++
arch/loongarch/include/asm/mmzone.h | 18 +
arch/loongarch/include/asm/module.h | 80 +
arch/loongarch/include/asm/module.lds.h | 7 +
arch/loongarch/include/asm/numa.h | 69 +
arch/loongarch/include/asm/page.h | 113 ++
arch/loongarch/include/asm/pci.h | 40 +
arch/loongarch/include/asm/percpu.h | 222 +++
arch/loongarch/include/asm/perf_event.h | 10 +
arch/loongarch/include/asm/pgalloc.h | 103 ++
arch/loongarch/include/asm/pgtable-bits.h | 139 ++
arch/loongarch/include/asm/pgtable.h | 572 ++++++++
arch/loongarch/include/asm/prefetch.h | 29 +
arch/loongarch/include/asm/processor.h | 207 +++
arch/loongarch/include/asm/ptrace.h | 152 ++
arch/loongarch/include/asm/reboot.h | 10 +
arch/loongarch/include/asm/regdef.h | 43 +
arch/loongarch/include/asm/seccomp.h | 20 +
arch/loongarch/include/asm/serial.h | 11 +
arch/loongarch/include/asm/setup.h | 21 +
arch/loongarch/include/asm/shmparam.h | 12 +
arch/loongarch/include/asm/smp.h | 124 ++
arch/loongarch/include/asm/sparsemem.h | 23 +
arch/loongarch/include/asm/spinlock.h | 12 +
arch/loongarch/include/asm/spinlock_types.h | 11 +
arch/loongarch/include/asm/stackframe.h | 219 +++
arch/loongarch/include/asm/stacktrace.h | 74 +
arch/loongarch/include/asm/string.h | 17 +
arch/loongarch/include/asm/switch_to.h | 37 +
arch/loongarch/include/asm/syscall.h | 74 +
arch/loongarch/include/asm/thread_info.h | 106 ++
arch/loongarch/include/asm/time.h | 50 +
arch/loongarch/include/asm/timex.h | 31 +
arch/loongarch/include/asm/tlb.h | 216 +++
arch/loongarch/include/asm/tlbflush.h | 48 +
arch/loongarch/include/asm/topology.h | 41 +
arch/loongarch/include/asm/types.h | 33 +
arch/loongarch/include/asm/uaccess.h | 270 ++++
arch/loongarch/include/asm/unistd.h | 11 +
arch/loongarch/include/asm/vdso.h | 38 +
arch/loongarch/include/asm/vdso/clocksource.h | 8 +
arch/loongarch/include/asm/vdso/gettimeofday.h | 99 ++
arch/loongarch/include/asm/vdso/processor.h | 14 +
arch/loongarch/include/asm/vdso/vdso.h | 30 +
arch/loongarch/include/asm/vdso/vsyscall.h | 27 +
arch/loongarch/include/asm/vermagic.h | 19 +
arch/loongarch/include/asm/vmalloc.h | 4 +
arch/loongarch/include/uapi/asm/Kbuild | 2 +
arch/loongarch/include/uapi/asm/auxvec.h | 17 +
arch/loongarch/include/uapi/asm/bitfield.h | 15 +
arch/loongarch/include/uapi/asm/bitsperlong.h | 9 +
arch/loongarch/include/uapi/asm/break.h | 23 +
arch/loongarch/include/uapi/asm/byteorder.h | 13 +
arch/loongarch/include/uapi/asm/hwcap.h | 20 +
arch/loongarch/include/uapi/asm/inst.h | 57 +
arch/loongarch/include/uapi/asm/ptrace.h | 52 +
arch/loongarch/include/uapi/asm/reg.h | 59 +
arch/loongarch/include/uapi/asm/sigcontext.h | 63 +
arch/loongarch/include/uapi/asm/signal.h | 13 +
arch/loongarch/include/uapi/asm/swab.h | 52 +
arch/loongarch/include/uapi/asm/ucontext.h | 35 +
arch/loongarch/include/uapi/asm/unistd.h | 6 +
arch/loongarch/kernel/Makefile | 26 +
arch/loongarch/kernel/access-helper.h | 13 +
arch/loongarch/kernel/acpi.c | 501 +++++++
arch/loongarch/kernel/asm-offsets.c | 262 ++++
arch/loongarch/kernel/cacheinfo.c | 122 ++
arch/loongarch/kernel/cmdline.c | 31 +
arch/loongarch/kernel/cmpxchg.c | 105 ++
arch/loongarch/kernel/cpu-probe.c | 305 ++++
arch/loongarch/kernel/dma.c | 40 +
arch/loongarch/kernel/efi-header.S | 100 ++
arch/loongarch/kernel/efi.c | 235 +++
arch/loongarch/kernel/elf.c | 30 +
arch/loongarch/kernel/entry.S | 89 ++
arch/loongarch/kernel/env.c | 176 +++
arch/loongarch/kernel/fpu.S | 264 ++++
arch/loongarch/kernel/genex.S | 95 ++
arch/loongarch/kernel/head.S | 144 ++
arch/loongarch/kernel/idle.c | 16 +
arch/loongarch/kernel/image-vars.h | 30 +
arch/loongarch/kernel/inst.c | 40 +
arch/loongarch/kernel/io.c | 94 ++
arch/loongarch/kernel/irq.c | 140 ++
arch/loongarch/kernel/mem.c | 83 ++
arch/loongarch/kernel/module-sections.c | 121 ++
arch/loongarch/kernel/module.c | 385 +++++
arch/loongarch/kernel/numa.c | 461 ++++++
arch/loongarch/kernel/proc.c | 127 ++
arch/loongarch/kernel/process.c | 267 ++++
arch/loongarch/kernel/ptrace.c | 431 ++++++
arch/loongarch/kernel/reset.c | 102 ++
arch/loongarch/kernel/rtc.c | 36 +
arch/loongarch/kernel/setup.c | 464 ++++++
arch/loongarch/kernel/signal.c | 568 ++++++++
arch/loongarch/kernel/smp.c | 762 ++++++++++
arch/loongarch/kernel/switch.S | 35 +
arch/loongarch/kernel/syscall.c | 63 +
arch/loongarch/kernel/time.c | 220 +++
arch/loongarch/kernel/topology.c | 52 +
arch/loongarch/kernel/traps.c | 753 ++++++++++
arch/loongarch/kernel/vdso.c | 138 ++
arch/loongarch/kernel/vmlinux.lds.S | 121 ++
arch/loongarch/lib/Makefile | 7 +
arch/loongarch/lib/clear_user.S | 43 +
arch/loongarch/lib/copy_user.S | 47 +
arch/loongarch/lib/delay.c | 43 +
arch/loongarch/lib/dump_tlb.c | 111 ++
arch/loongarch/lib/memcpy.S | 32 +
arch/loongarch/lib/memmove.S | 45 +
arch/loongarch/lib/memset.S | 30 +
arch/loongarch/mm/Makefile | 9 +
arch/loongarch/mm/cache.c | 140 ++
arch/loongarch/mm/extable.c | 22 +
arch/loongarch/mm/fault.c | 261 ++++
arch/loongarch/mm/hugetlbpage.c | 87 ++
arch/loongarch/mm/init.c | 178 +++
arch/loongarch/mm/ioremap.c | 27 +
arch/loongarch/mm/maccess.c | 10 +
arch/loongarch/mm/mmap.c | 125 ++
arch/loongarch/mm/page.S | 84 ++
arch/loongarch/mm/pgtable.c | 130 ++
arch/loongarch/mm/tlb.c | 303 ++++
arch/loongarch/mm/tlbex.S | 546 +++++++
arch/loongarch/pci/Makefile | 7 +
arch/loongarch/pci/acpi.c | 175 +++
arch/loongarch/pci/pci.c | 98 ++
arch/loongarch/tools/Makefile | 15 +
arch/loongarch/tools/calc_vmlinuz_load_addr.c | 51 +
arch/loongarch/tools/elf-entry.c | 66 +
arch/loongarch/vdso/Makefile | 96 ++
arch/loongarch/vdso/elf.S | 15 +
arch/loongarch/vdso/gen_vdso_offsets.sh | 13 +
arch/loongarch/vdso/sigreturn.S | 24 +
arch/loongarch/vdso/vdso.S | 22 +
arch/loongarch/vdso/vdso.lds.S | 72 +
arch/loongarch/vdso/vgettimeofday.c | 25 +
drivers/firmware/efi/Kconfig | 4 +-
drivers/firmware/efi/libstub/Makefile | 14 +-
drivers/firmware/efi/libstub/loongarch-stub.c | 425 ++++++
drivers/gpu/drm/drm_vm.c | 2 +-
drivers/gpu/drm/ttm/ttm_module.c | 2 +-
include/drm/drm_cache.h | 8 +
include/linux/cpuhotplug.h | 1 +
include/linux/efi.h | 1 +
include/linux/pe.h | 1 +
include/uapi/linux/audit.h | 2 +
include/uapi/linux/elf-em.h | 1 +
include/uapi/linux/elf.h | 5 +
include/uapi/linux/kexec.h | 1 +
scripts/sorttable.c | 5 +
scripts/subarch.include | 2 +-
tools/include/uapi/asm/bitsperlong.h | 2 +
230 files changed, 23918 insertions(+), 7 deletions(-)
create mode 100644 Documentation/loongarch/features.rst
create mode 100644 Documentation/loongarch/index.rst
create mode 100644 Documentation/loongarch/introduction.rst
create mode 100644 Documentation/loongarch/irq-chip-model.rst
create mode 100644 Documentation/translations/zh_CN/loongarch/features.rst
create mode 100644 Documentation/translations/zh_CN/loongarch/index.rst
create mode 100644 Documentation/translations/zh_CN/loongarch/introduction.rst
create mode 100644 Documentation/translations/zh_CN/loongarch/irq-chip-model.rst
create mode 100644 arch/loongarch/.gitignore
create mode 100644 arch/loongarch/Kbuild
create mode 100644 arch/loongarch/Kconfig
create mode 100644 arch/loongarch/Kconfig.debug
create mode 100644 arch/loongarch/Makefile
create mode 100644 arch/loongarch/boot/Makefile
create mode 100644 arch/loongarch/boot/boot.lds.S
create mode 100644 arch/loongarch/boot/decompress.c
create mode 100644 arch/loongarch/boot/string.c
create mode 100644 arch/loongarch/boot/zheader.S
create mode 100644 arch/loongarch/boot/zkernel.S
create mode 100644 arch/loongarch/configs/loongson3_defconfig
create mode 100644 arch/loongarch/include/asm/Kbuild
create mode 100644 arch/loongarch/include/asm/acenv.h
create mode 100644 arch/loongarch/include/asm/acpi.h
create mode 100644 arch/loongarch/include/asm/addrspace.h
create mode 100644 arch/loongarch/include/asm/asm-offsets.h
create mode 100644 arch/loongarch/include/asm/asm-prototypes.h
create mode 100644 arch/loongarch/include/asm/asm.h
create mode 100644 arch/loongarch/include/asm/asmmacro.h
create mode 100644 arch/loongarch/include/asm/atomic.h
create mode 100644 arch/loongarch/include/asm/barrier.h
create mode 100644 arch/loongarch/include/asm/bitops.h
create mode 100644 arch/loongarch/include/asm/bitrev.h
create mode 100644 arch/loongarch/include/asm/boot_param.h
create mode 100644 arch/loongarch/include/asm/bootinfo.h
create mode 100644 arch/loongarch/include/asm/branch.h
create mode 100644 arch/loongarch/include/asm/bug.h
create mode 100644 arch/loongarch/include/asm/cache.h
create mode 100644 arch/loongarch/include/asm/cacheflush.h
create mode 100644 arch/loongarch/include/asm/cacheops.h
create mode 100644 arch/loongarch/include/asm/clocksource.h
create mode 100644 arch/loongarch/include/asm/cmpxchg.h
create mode 100644 arch/loongarch/include/asm/compiler.h
create mode 100644 arch/loongarch/include/asm/cpu-features.h
create mode 100644 arch/loongarch/include/asm/cpu-info.h
create mode 100644 arch/loongarch/include/asm/cpu.h
create mode 100644 arch/loongarch/include/asm/cpufeature.h
create mode 100644 arch/loongarch/include/asm/delay.h
create mode 100644 arch/loongarch/include/asm/dma-direct.h
create mode 100644 arch/loongarch/include/asm/dma.h
create mode 100644 arch/loongarch/include/asm/dmi.h
create mode 100644 arch/loongarch/include/asm/efi.h
create mode 100644 arch/loongarch/include/asm/elf.h
create mode 100644 arch/loongarch/include/asm/entry-common.h
create mode 100644 arch/loongarch/include/asm/exec.h
create mode 100644 arch/loongarch/include/asm/fb.h
create mode 100644 arch/loongarch/include/asm/fixmap.h
create mode 100644 arch/loongarch/include/asm/fpregdef.h
create mode 100644 arch/loongarch/include/asm/fpu.h
create mode 100644 arch/loongarch/include/asm/futex.h
create mode 100644 arch/loongarch/include/asm/fw.h
create mode 100644 arch/loongarch/include/asm/hardirq.h
create mode 100644 arch/loongarch/include/asm/hugetlb.h
create mode 100644 arch/loongarch/include/asm/hw_irq.h
create mode 100644 arch/loongarch/include/asm/idle.h
create mode 100644 arch/loongarch/include/asm/inst.h
create mode 100644 arch/loongarch/include/asm/io.h
create mode 100644 arch/loongarch/include/asm/irq.h
create mode 100644 arch/loongarch/include/asm/irq_regs.h
create mode 100644 arch/loongarch/include/asm/irqflags.h
create mode 100644 arch/loongarch/include/asm/kdebug.h
create mode 100644 arch/loongarch/include/asm/linkage.h
create mode 100644 arch/loongarch/include/asm/local.h
create mode 100644 arch/loongarch/include/asm/loongarch.h
create mode 100644 arch/loongarch/include/asm/loongson.h
create mode 100644 arch/loongarch/include/asm/mmu.h
create mode 100644 arch/loongarch/include/asm/mmu_context.h
create mode 100644 arch/loongarch/include/asm/mmzone.h
create mode 100644 arch/loongarch/include/asm/module.h
create mode 100644 arch/loongarch/include/asm/module.lds.h
create mode 100644 arch/loongarch/include/asm/numa.h
create mode 100644 arch/loongarch/include/asm/page.h
create mode 100644 arch/loongarch/include/asm/pci.h
create mode 100644 arch/loongarch/include/asm/percpu.h
create mode 100644 arch/loongarch/include/asm/perf_event.h
create mode 100644 arch/loongarch/include/asm/pgalloc.h
create mode 100644 arch/loongarch/include/asm/pgtable-bits.h
create mode 100644 arch/loongarch/include/asm/pgtable.h
create mode 100644 arch/loongarch/include/asm/prefetch.h
create mode 100644 arch/loongarch/include/asm/processor.h
create mode 100644 arch/loongarch/include/asm/ptrace.h
create mode 100644 arch/loongarch/include/asm/reboot.h
create mode 100644 arch/loongarch/include/asm/regdef.h
create mode 100644 arch/loongarch/include/asm/seccomp.h
create mode 100644 arch/loongarch/include/asm/serial.h
create mode 100644 arch/loongarch/include/asm/setup.h
create mode 100644 arch/loongarch/include/asm/shmparam.h
create mode 100644 arch/loongarch/include/asm/smp.h
create mode 100644 arch/loongarch/include/asm/sparsemem.h
create mode 100644 arch/loongarch/include/asm/spinlock.h
create mode 100644 arch/loongarch/include/asm/spinlock_types.h
create mode 100644 arch/loongarch/include/asm/stackframe.h
create mode 100644 arch/loongarch/include/asm/stacktrace.h
create mode 100644 arch/loongarch/include/asm/string.h
create mode 100644 arch/loongarch/include/asm/switch_to.h
create mode 100644 arch/loongarch/include/asm/syscall.h
create mode 100644 arch/loongarch/include/asm/thread_info.h
create mode 100644 arch/loongarch/include/asm/time.h
create mode 100644 arch/loongarch/include/asm/timex.h
create mode 100644 arch/loongarch/include/asm/tlb.h
create mode 100644 arch/loongarch/include/asm/tlbflush.h
create mode 100644 arch/loongarch/include/asm/topology.h
create mode 100644 arch/loongarch/include/asm/types.h
create mode 100644 arch/loongarch/include/asm/uaccess.h
create mode 100644 arch/loongarch/include/asm/unistd.h
create mode 100644 arch/loongarch/include/asm/vdso.h
create mode 100644 arch/loongarch/include/asm/vdso/clocksource.h
create mode 100644 arch/loongarch/include/asm/vdso/gettimeofday.h
create mode 100644 arch/loongarch/include/asm/vdso/processor.h
create mode 100644 arch/loongarch/include/asm/vdso/vdso.h
create mode 100644 arch/loongarch/include/asm/vdso/vsyscall.h
create mode 100644 arch/loongarch/include/asm/vermagic.h
create mode 100644 arch/loongarch/include/asm/vmalloc.h
create mode 100644 arch/loongarch/include/uapi/asm/Kbuild
create mode 100644 arch/loongarch/include/uapi/asm/auxvec.h
create mode 100644 arch/loongarch/include/uapi/asm/bitfield.h
create mode 100644 arch/loongarch/include/uapi/asm/bitsperlong.h
create mode 100644 arch/loongarch/include/uapi/asm/break.h
create mode 100644 arch/loongarch/include/uapi/asm/byteorder.h
create mode 100644 arch/loongarch/include/uapi/asm/hwcap.h
create mode 100644 arch/loongarch/include/uapi/asm/inst.h
create mode 100644 arch/loongarch/include/uapi/asm/ptrace.h
create mode 100644 arch/loongarch/include/uapi/asm/reg.h
create mode 100644 arch/loongarch/include/uapi/asm/sigcontext.h
create mode 100644 arch/loongarch/include/uapi/asm/signal.h
create mode 100644 arch/loongarch/include/uapi/asm/swab.h
create mode 100644 arch/loongarch/include/uapi/asm/ucontext.h
create mode 100644 arch/loongarch/include/uapi/asm/unistd.h
create mode 100644 arch/loongarch/kernel/Makefile
create mode 100644 arch/loongarch/kernel/access-helper.h
create mode 100644 arch/loongarch/kernel/acpi.c
create mode 100644 arch/loongarch/kernel/asm-offsets.c
create mode 100644 arch/loongarch/kernel/cacheinfo.c
create mode 100644 arch/loongarch/kernel/cmdline.c
create mode 100644 arch/loongarch/kernel/cmpxchg.c
create mode 100644 arch/loongarch/kernel/cpu-probe.c
create mode 100644 arch/loongarch/kernel/dma.c
create mode 100644 arch/loongarch/kernel/efi-header.S
create mode 100644 arch/loongarch/kernel/efi.c
create mode 100644 arch/loongarch/kernel/elf.c
create mode 100644 arch/loongarch/kernel/entry.S
create mode 100644 arch/loongarch/kernel/env.c
create mode 100644 arch/loongarch/kernel/fpu.S
create mode 100644 arch/loongarch/kernel/genex.S
create mode 100644 arch/loongarch/kernel/head.S
create mode 100644 arch/loongarch/kernel/idle.c
create mode 100644 arch/loongarch/kernel/image-vars.h
create mode 100644 arch/loongarch/kernel/inst.c
create mode 100644 arch/loongarch/kernel/io.c
create mode 100644 arch/loongarch/kernel/irq.c
create mode 100644 arch/loongarch/kernel/mem.c
create mode 100644 arch/loongarch/kernel/module-sections.c
create mode 100644 arch/loongarch/kernel/module.c
create mode 100644 arch/loongarch/kernel/numa.c
create mode 100644 arch/loongarch/kernel/proc.c
create mode 100644 arch/loongarch/kernel/process.c
create mode 100644 arch/loongarch/kernel/ptrace.c
create mode 100644 arch/loongarch/kernel/reset.c
create mode 100644 arch/loongarch/kernel/rtc.c
create mode 100644 arch/loongarch/kernel/setup.c
create mode 100644 arch/loongarch/kernel/signal.c
create mode 100644 arch/loongarch/kernel/smp.c
create mode 100644 arch/loongarch/kernel/switch.S
create mode 100644 arch/loongarch/kernel/syscall.c
create mode 100644 arch/loongarch/kernel/time.c
create mode 100644 arch/loongarch/kernel/topology.c
create mode 100644 arch/loongarch/kernel/traps.c
create mode 100644 arch/loongarch/kernel/vdso.c
create mode 100644 arch/loongarch/kernel/vmlinux.lds.S
create mode 100644 arch/loongarch/lib/Makefile
create mode 100644 arch/loongarch/lib/clear_user.S
create mode 100644 arch/loongarch/lib/copy_user.S
create mode 100644 arch/loongarch/lib/delay.c
create mode 100644 arch/loongarch/lib/dump_tlb.c
create mode 100644 arch/loongarch/lib/memcpy.S
create mode 100644 arch/loongarch/lib/memmove.S
create mode 100644 arch/loongarch/lib/memset.S
create mode 100644 arch/loongarch/mm/Makefile
create mode 100644 arch/loongarch/mm/cache.c
create mode 100644 arch/loongarch/mm/extable.c
create mode 100644 arch/loongarch/mm/fault.c
create mode 100644 arch/loongarch/mm/hugetlbpage.c
create mode 100644 arch/loongarch/mm/init.c
create mode 100644 arch/loongarch/mm/ioremap.c
create mode 100644 arch/loongarch/mm/maccess.c
create mode 100644 arch/loongarch/mm/mmap.c
create mode 100644 arch/loongarch/mm/page.S
create mode 100644 arch/loongarch/mm/pgtable.c
create mode 100644 arch/loongarch/mm/tlb.c
create mode 100644 arch/loongarch/mm/tlbex.S
create mode 100644 arch/loongarch/pci/Makefile
create mode 100644 arch/loongarch/pci/acpi.c
create mode 100644 arch/loongarch/pci/pci.c
create mode 100644 arch/loongarch/tools/Makefile
create mode 100644 arch/loongarch/tools/calc_vmlinuz_load_addr.c
create mode 100644 arch/loongarch/tools/elf-entry.c
create mode 100644 arch/loongarch/vdso/Makefile
create mode 100644 arch/loongarch/vdso/elf.S
create mode 100755 arch/loongarch/vdso/gen_vdso_offsets.sh
create mode 100644 arch/loongarch/vdso/sigreturn.S
create mode 100644 arch/loongarch/vdso/vdso.S
create mode 100644 arch/loongarch/vdso/vdso.lds.S
create mode 100644 arch/loongarch/vdso/vgettimeofday.c
create mode 100644 drivers/firmware/efi/libstub/loongarch-stub.c
--
2.27.0



2022-05-18 09:33:41

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 01/22] Documentation: LoongArch: Add basic documentations

Add some basic documentation for LoongArch. LoongArch is a new RISC ISA,
which is a bit like MIPS or RISC-V. LoongArch includes a reduced 32-bit
version (LA32R), a standard 32-bit version (LA32S) and a 64-bit version
(LA64).

Signed-off-by: Huacai Chen <[email protected]>
---
Documentation/arch.rst | 1 +
Documentation/loongarch/features.rst | 3 +
Documentation/loongarch/index.rst | 21 ++
Documentation/loongarch/introduction.rst | 387 +++++++++++++++++++++
Documentation/loongarch/irq-chip-model.rst | 168 +++++++++
5 files changed, 580 insertions(+)
create mode 100644 Documentation/loongarch/features.rst
create mode 100644 Documentation/loongarch/index.rst
create mode 100644 Documentation/loongarch/introduction.rst
create mode 100644 Documentation/loongarch/irq-chip-model.rst

diff --git a/Documentation/arch.rst b/Documentation/arch.rst
index 14bcd8294b93..41a66a8b38e4 100644
--- a/Documentation/arch.rst
+++ b/Documentation/arch.rst
@@ -13,6 +13,7 @@ implementation.
arm/index
arm64/index
ia64/index
+ loongarch/index
m68k/index
mips/index
nios2/index
diff --git a/Documentation/loongarch/features.rst b/Documentation/loongarch/features.rst
new file mode 100644
index 000000000000..ebacade3ea45
--- /dev/null
+++ b/Documentation/loongarch/features.rst
@@ -0,0 +1,3 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. kernel-feat:: $srctree/Documentation/features loongarch
diff --git a/Documentation/loongarch/index.rst b/Documentation/loongarch/index.rst
new file mode 100644
index 000000000000..aaba648db907
--- /dev/null
+++ b/Documentation/loongarch/index.rst
@@ -0,0 +1,21 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======================
+LoongArch Architecture
+======================
+
+.. toctree::
+ :maxdepth: 2
+ :numbered:
+
+ introduction
+ irq-chip-model
+
+ features
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
diff --git a/Documentation/loongarch/introduction.rst b/Documentation/loongarch/introduction.rst
new file mode 100644
index 000000000000..4bf912b3ed02
--- /dev/null
+++ b/Documentation/loongarch/introduction.rst
@@ -0,0 +1,387 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================
+Introduction to LoongArch
+=========================
+
+LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V. There are
+currently has 3 variants: a reduced 32-bit version (LA32R), a standard 32-bit
+version (LA32S) and a 64-bit version (LA64). There are 4 privilege levels
+(PLVs) defined in LoongArch: PLV0~PLV3, from high to low. Kernel runs at PLV0
+while applications run at PLV3. This document introduces the registers, basic
+instruction set, virtual memory and some other topics of LoongArch.
+
+Registers
+=========
+
+LoongArch registers include general purpose registers (GPRs), floating point
+registers (FPRs), vector registers (VRs) and control status registers (CSRs)
+used in privileged mode (PLV0).
+
+GPRs
+----
+
+LoongArch has 32 GPRs ( ``$r0`` ~ ``$r31`` ); each one is 32-bit wide in LA32
+and 64-bit wide in LA64. ``$r0`` is hard-wired to zero, and the other registers
+are not architecturally special. (Except ``$r1``, which is hard-wired as the
+link register of the BL instruction.)
+
+The kernel uses a variant of the LoongArch register convention, as described in
+the LoongArch ELF psABI spec, in :ref:`References <loongarch-references>`:
+
+================= =============== =================== ============
+Name Alias Usage Preserved
+ across calls
+================= =============== =================== ============
+``$r0`` ``$zero`` Constant zero Unused
+``$r1`` ``$ra`` Return address No
+``$r2`` ``$tp`` TLS/Thread pointer Unused
+``$r3`` ``$sp`` Stack pointer Yes
+``$r4``-``$r11`` ``$a0``-``$a7`` Argument registers No
+``$r4``-``$r5`` ``$v0``-``$v1`` Return value No
+``$r12``-``$r20`` ``$t0``-``$t8`` Temp registers No
+``$r21`` ``$u0`` Percpu base address Unused
+``$r22`` ``$fp`` Frame pointer Yes
+``$r23``-``$r31`` ``$s0``-``$s8`` Static registers Yes
+================= =============== =================== ============
+
+Note: The register ``$r21`` is reserved in the ELF psABI, but used by the Linux
+kernel for storing the percpu base address. It normally has no ABI name, but is
+called ``$u0`` in the kernel. You may also see ``$v0`` or ``$v1`` in some old code,
+they are deprecated aliases of ``$a0`` and ``$a1`` respectively.
+
+FPRs
+----
+
+LoongArch has 32 FPRs ( ``$f0`` ~ ``$f31`` ) when FPU is present. Each one is
+64-bit wide on the LA64 cores.
+
+The floating-point register convention is the same as described in the
+LoongArch ELF psABI spec:
+
+================= ================== =================== ============
+Name Alias Usage Preserved
+ across calls
+================= ================== =================== ============
+``$f0``-``$f7`` ``$fa0``-``$fa7`` Argument registers No
+``$f0``-``$f1`` ``$fv0``-``$fv1`` Return value No
+``$f8``-``$f23`` ``$ft0``-``$ft15`` Temp registers No
+``$f24``-``$f31`` ``$fs0``-``$fs7`` Static registers Yes
+================= ================== =================== ============
+
+Note: You may see ``$fv0`` or ``$fv1`` in some old code, they are deprecated
+aliases of ``$fa0`` and ``$fa1`` respectively.
+
+VRs
+----
+
+There are currently 2 vector extensions to LoongArch:
+
+- LSX (Loongson SIMD eXtension) with 128-bit vectors,
+- LASX (Loongson Advanced SIMD eXtension) with 256-bit vectors.
+
+LSX brings ``$v0`` ~ ``$v31`` while LASX brings ``$x0`` ~ ``$x31`` as the vector
+registers.
+
+The VRs overlap with FPRs: for example, on a core implementing LSX and LASX,
+the lower 128 bits of ``$x0`` is shared with ``$v0``, and the lower 64 bits of
+``$v0`` is shared with ``$f0``; same with all other VRs.
+
+CSRs
+----
+
+CSRs can only be accessed from in privileged mode (PLV0):
+
+================= ===================================== ==============
+Address Full Name Abbrev Name
+================= ===================================== ==============
+0x0 Current Mode Information CRMD
+0x1 Pre-exception Mode Information PRMD
+0x2 Extension Unit Enable EUEN
+0x3 Miscellaneous Control MISC
+0x4 Exception Configuration ECFG
+0x5 Exception Status ESTAT
+0x6 Exception Return Address ERA
+0x7 Bad (Faulting) Virtual Address BADV
+0x8 Bad (Faulting) Instruction Word BADI
+0xC Exception Entrypoint Address EENTRY
+0x10 TLB Index TLBIDX
+0x11 TLB Entry High-order Bits TLBEHI
+0x12 TLB Entry Low-order Bits 0 TLBELO0
+0x13 TLB Entry Low-order Bits 1 TLBELO1
+0x18 Address Space Identifier ASID
+0x19 Page Global Directory Address for PGDL
+ Lower-half Address Space
+0x1A Page Global Directory Address for PGDH
+ Higher-half Address Space
+0x1B Page Global Directory Address PGD
+0x1C Page Walk Control for Lower- PWCL
+ half Address Space
+0x1D Page Walk Control for Higher- PWCH
+ half Address Space
+0x1E STLB Page Size STLBPS
+0x1F Reduced Virtual Address Configuration RVACFG
+0x20 CPU Identifier CPUID
+0x21 Privileged Resource Configuration 1 PRCFG1
+0x22 Privileged Resource Configuration 2 PRCFG2
+0x23 Privileged Resource Configuration 3 PRCFG3
+0x30+n (0≤n≤15) Saved Data register SAVEn
+0x40 Timer Identifier TID
+0x41 Timer Configuration TCFG
+0x42 Timer Value TVAL
+0x43 Compensation of Timer Count CNTC
+0x44 Timer Interrupt Clearing TICLR
+0x60 LLBit Control LLBCTL
+0x80 Implementation-specific Control 1 IMPCTL1
+0x81 Implementation-specific Control 2 IMPCTL2
+0x88 TLB Refill Exception Entrypoint TLBRENTRY
+ Address
+0x89 TLB Refill Exception BAD (Faulting) TLBRBADV
+ Virtual Address
+0x8A TLB Refill Exception Return Address TLBRERA
+0x8B TLB Refill Exception Saved Data TLBRSAVE
+ Register
+0x8C TLB Refill Exception Entry Low-order TLBRELO0
+ Bits 0
+0x8D TLB Refill Exception Entry Low-order TLBRELO1
+ Bits 1
+0x8E TLB Refill Exception Entry High-order TLBEHI
+ Bits
+0x8F TLB Refill Exception Pre-exception TLBRPRMD
+ Mode Information
+0x90 Machine Error Control MERRCTL
+0x91 Machine Error Information 1 MERRINFO1
+0x92 Machine Error Information 2 MERRINFO2
+0x93 Machine Error Exception Entrypoint MERRENTRY
+ Address
+0x94 Machine Error Exception Return MERRERA
+ Address
+0x95 Machine Error Exception Saved Data MERRSAVE
+ Register
+0x98 Cache TAGs CTAG
+0x180+n (0≤n≤3) Direct Mapping Configuration Window n DMWn
+0x200+2n (0≤n≤31) Performance Monitor Configuration n PMCFGn
+0x201+2n (0≤n≤31) Performance Monitor Overall Counter n PMCNTn
+0x300 Memory Load/Store WatchPoint MWPC
+ Overall Control
+0x301 Memory Load/Store WatchPoint MWPS
+ Overall Status
+0x310+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG1
+ Configuration 1
+0x311+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG2
+ Configuration 2
+0x312+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG3
+ Configuration 3
+0x313+8n (0≤n≤7) Memory Load/Store WatchPoint n MWPnCFG4
+ Configuration 4
+0x380 Instruction Fetch WatchPoint FWPC
+ Overall Control
+0x381 Instruction Fetch WatchPoint FWPS
+ Overall Status
+0x390+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG1
+ Configuration 1
+0x391+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG2
+ Configuration 2
+0x392+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG3
+ Configuration 3
+0x393+8n (0≤n≤7) Instruction Fetch WatchPoint n FWPnCFG4
+ Configuration 4
+0x500 Debug Register DBG
+0x501 Debug Exception Return Address DERA
+0x502 Debug Exception Saved Data Register DSAVE
+================= ===================================== ==============
+
+ERA, TLBRERA, MERRERA and DERA are sometimes also known as EPC, TLBREPC, MERREPC
+and DEPC respectively.
+
+Basic Instruction Set
+=====================
+
+Instruction formats
+-------------------
+
+LoongArch instructions are 32 bits wide, belonging to 9 basic instruction
+formats (and variants of them):
+
+=========== ==========================
+Format name Composition
+=========== ==========================
+2R Opcode + Rj + Rd
+3R Opcode + Rk + Rj + Rd
+4R Opcode + Ra + Rk + Rj + Rd
+2RI8 Opcode + I8 + Rj + Rd
+2RI12 Opcode + I12 + Rj + Rd
+2RI14 Opcode + I14 + Rj + Rd
+2RI16 Opcode + I16 + Rj + Rd
+1RI21 Opcode + I21L + Rj + I21H
+I26 Opcode + I26L + I26H
+=========== ==========================
+
+Rd is the destination register operand, while Rj, Rk and Ra ("a" stands for
+"additional") are the source register operands. I8/I12/I16/I21/I26 are
+immediate operands of respective width. The longer I21 and I26 are stored
+in separate higher and lower parts in the instruction word, denoted by the "L"
+and "H" suffixes.
+
+List of Instructions
+--------------------
+
+For brevity, only instruction names (mnemonics) are listed here; please see the
+:ref:`References <loongarch-references>` for details.
+
+
+1. Arithmetic Instructions::
+
+ ADD.W SUB.W ADDI.W ADD.D SUB.D ADDI.D
+ SLT SLTU SLTI SLTUI
+ AND OR NOR XOR ANDN ORN ANDI ORI XORI
+ MUL.W MULH.W MULH.WU DIV.W DIV.WU MOD.W MOD.WU
+ MUL.D MULH.D MULH.DU DIV.D DIV.DU MOD.D MOD.DU
+ PCADDI PCADDU12I PCADDU18I
+ LU12I.W LU32I.D LU52I.D ADDU16I.D
+
+2. Bit-shift Instructions::
+
+ SLL.W SRL.W SRA.W ROTR.W SLLI.W SRLI.W SRAI.W ROTRI.W
+ SLL.D SRL.D SRA.D ROTR.D SLLI.D SRLI.D SRAI.D ROTRI.D
+
+3. Bit-manipulation Instructions::
+
+ EXT.W.B EXT.W.H CLO.W CLO.D SLZ.W CLZ.D CTO.W CTO.D CTZ.W CTZ.D
+ BYTEPICK.W BYTEPICK.D BSTRINS.W BSTRINS.D BSTRPICK.W BSTRPICK.D
+ REVB.2H REVB.4H REVB.2W REVB.D REVH.2W REVH.D BITREV.4B BITREV.8B BITREV.W BITREV.D
+ MASKEQZ MASKNEZ
+
+4. Branch Instructions::
+
+ BEQ BNE BLT BGE BLTU BGEU BEQZ BNEZ B BL JIRL
+
+5. Load/Store Instructions::
+
+ LD.B LD.BU LD.H LD.HU LD.W LD.WU LD.D ST.B ST.H ST.W ST.D
+ LDX.B LDX.BU LDX.H LDX.HU LDX.W LDX.WU LDX.D STX.B STX.H STX.W STX.D
+ LDPTR.W LDPTR.D STPTR.W STPTR.D
+ PRELD PRELDX
+
+6. Atomic Operation Instructions::
+
+ LL.W SC.W LL.D SC.D
+ AMSWAP.W AMSWAP.D AMADD.W AMADD.D AMAND.W AMAND.D AMOR.W AMOR.D AMXOR.W AMXOR.D
+ AMMAX.W AMMAX.D AMMIN.W AMMIN.D
+
+7. Barrier Instructions::
+
+ IBAR DBAR
+
+8. Special Instructions::
+
+ SYSCALL BREAK CPUCFG NOP IDLE ERTN(ERET) DBCL(DBGCALL) RDTIMEL.W RDTIMEH.W RDTIME.D
+ ASRTLE.D ASRTGT.D
+
+9. Privileged Instructions::
+
+ CSRRD CSRWR CSRXCHG
+ IOCSRRD.B IOCSRRD.H IOCSRRD.W IOCSRRD.D IOCSRWR.B IOCSRWR.H IOCSRWR.W IOCSRWR.D
+ CACOP TLBP(TLBSRCH) TLBRD TLBWR TLBFILL TLBCLR TLBFLUSH INVTLB LDDIR LDPTE
+
+Virtual Memory
+==============
+
+LoongArch supports direct-mapped virtual memory and page-mapped virtual memory.
+
+Direct-mapped virtual memory is configured by CSR.DMWn (n=0~3), it has a simple
+relationship between virtual address (VA) and physical address (PA)::
+
+ VA = PA + FixedOffset
+
+Page-mapped virtual memory has arbitrary relationship between VA and PA, which
+is recorded in TLB and page tables. LoongArch's TLB includes a fully-associative
+MTLB (Multiple Page Size TLB) and set-associative STLB (Single Page Size TLB).
+
+By default, the whole virtual address space of LA32 is configured like this:
+
+============ =========================== =============================
+Name Address Range Attributes
+============ =========================== =============================
+``UVRANGE`` ``0x00000000 - 0x7FFFFFFF`` Page-mapped, Cached, PLV0~3
+``KPRANGE0`` ``0x80000000 - 0x9FFFFFFF`` Direct-mapped, Uncached, PLV0
+``KPRANGE1`` ``0xA0000000 - 0xBFFFFFFF`` Direct-mapped, Cached, PLV0
+``KVRANGE`` ``0xC0000000 - 0xFFFFFFFF`` Page-mapped, Cached, PLV0
+============ =========================== =============================
+
+User mode (PLV3) can only access UVRANGE. For direct-mapped KPRANGE0 and
+KPRANGE1, PA is equal to VA with bit30~31 cleared. For example, the uncached
+direct-mapped VA of 0x00001000 is 0x80001000, and the cached direct-mapped
+VA of 0x00001000 is 0xA0001000.
+
+By default, the whole virtual address space of LA64 is configured like this:
+
+============ ====================== ======================================
+Name Address Range Attributes
+============ ====================== ======================================
+``XUVRANGE`` ``0x0000000000000000 - Page-mapped, Cached, PLV0~3
+ 0x3FFFFFFFFFFFFFFF``
+``XSPRANGE`` ``0x4000000000000000 - Direct-mapped, Cached / Uncached, PLV0
+ 0x7FFFFFFFFFFFFFFF``
+``XKPRANGE`` ``0x8000000000000000 - Direct-mapped, Cached / Uncached, PLV0
+ 0xBFFFFFFFFFFFFFFF``
+``XKVRANGE`` ``0xC000000000000000 - Page-mapped, Cached, PLV0
+ 0xFFFFFFFFFFFFFFFF``
+============ ====================== ======================================
+
+User mode (PLV3) can only access XUVRANGE. For direct-mapped XSPRANGE and
+XKPRANGE, PA is equal to VA with bits 60~63 cleared, and the cache attribute
+is configured by bits 60~61 in VA: 0 is for strongly-ordered uncached, 1 is
+for coherent cached, and 2 is for weakly-ordered uncached.
+
+Currently we only use XKPRANGE for direct mapping and XSPRANGE is reserved.
+
+To put this in action: the strongly-ordered uncached direct-mapped VA (in
+XKPRANGE) of 0x00000000_00001000 is 0x80000000_00001000, the coherent cached
+direct-mapped VA (in XKPRANGE) of 0x00000000_00001000 is 0x90000000_00001000,
+and the weakly-ordered uncached direct-mapped VA (in XKPRANGE) of 0x00000000
+_00001000 is 0xA0000000_00001000.
+
+Relationship of Loongson and LoongArch
+======================================
+
+LoongArch is a RISC ISA which is different from any other existing ones, while
+Loongson is a family of processors. Loongson includes 3 series: Loongson-1 is
+the 32-bit processor series, Loongson-2 is the low-end 64-bit processor series,
+and Loongson-3 is the high-end 64-bit processor series. Old Loongson is based on
+MIPS, while New Loongson is based on LoongArch. Take Loongson-3 as an example:
+Loongson-3A1000/3B1500/3A2000/3A3000/3A4000 are MIPS-compatible, while Loongson-
+3A5000 (and future revisions) are all based on LoongArch.
+
+.. _loongarch-references:
+
+References
+==========
+
+Official web site of Loongson Technology Corp. Ltd.:
+
+ http://www.loongson.cn/
+
+Developer web site of Loongson and LoongArch (Software and Documentation):
+
+ http://www.loongnix.cn/
+
+ https://github.com/loongson/
+
+ https://loongson.github.io/LoongArch-Documentation/
+
+Documentation of LoongArch ISA:
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.00-CN.pdf (in Chinese)
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.00-EN.pdf (in English)
+
+Documentation of LoongArch ELF psABI:
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v1.00-CN.pdf (in Chinese)
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v1.00-EN.pdf (in English)
+
+Linux kernel repository of Loongson and LoongArch:
+
+ https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git
diff --git a/Documentation/loongarch/irq-chip-model.rst b/Documentation/loongarch/irq-chip-model.rst
new file mode 100644
index 000000000000..35c962991283
--- /dev/null
+++ b/Documentation/loongarch/irq-chip-model.rst
@@ -0,0 +1,168 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=======================================
+IRQ chip model (hierarchy) of LoongArch
+=======================================
+
+Currently, LoongArch based processors (e.g. Loongson-3A5000) can only work together
+with LS7A chipsets. The irq chips in LoongArch computers include CPUINTC (CPU Core
+Interrupt Controller), LIOINTC (Legacy I/O Interrupt Controller), EIOINTC (Extended
+I/O Interrupt Controller), HTVECINTC (Hyper-Transport Vector Interrupt Controller),
+PCH-PIC (Main Interrupt Controller in LS7A chipset), PCH-LPC (LPC Interrupt Controller
+in LS7A chipset) and PCH-MSI (MSI Interrupt Controller).
+
+CPUINTC is a per-core controller (in CPU), LIOINTC/EIOINTC/HTVECINTC are per-package
+controllers (in CPU), while PCH-PIC/PCH-LPC/PCH-MSI are controllers out of CPU (i.e.,
+in chipsets). These controllers (in other words, irqchips) are linked in a hierarchy,
+and there are two models of hierarchy (legacy model and extended model).
+
+Legacy IRQ model
+================
+
+In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer interrupt go
+to CPUINTC directly, CPU UARTS interrupts go to LIOINTC, while all other devices
+interrupts go to PCH-PIC/PCH-LPC/PCH-MSI and gathered by HTVECINTC, and then go
+to LIOINTC, and then CPUINTC.
+
+ +---------------------------------------------+
+ |:: |
+ | |
+ | +-----+ +---------+ +-------+ |
+ | | IPI | --> | CPUINTC | <-- | Timer | |
+ | +-----+ +---------+ +-------+ |
+ | ^ |
+ | | |
+ | +---------+ +-------+ |
+ | | LIOINTC | <-- | UARTs | |
+ | +---------+ +-------+ |
+ | ^ |
+ | | |
+ | +-----------+ |
+ | | HTVECINTC | |
+ | +-----------+ |
+ | ^ ^ |
+ | | | |
+ | +---------+ +---------+ |
+ | | PCH-PIC | | PCH-MSI | |
+ | +---------+ +---------+ |
+ | ^ ^ ^ |
+ | | | | |
+ | +---------+ +---------+ +---------+ |
+ | | PCH-LPC | | Devices | | Devices | |
+ | +---------+ +---------+ +---------+ |
+ | ^ |
+ | | |
+ | +---------+ |
+ | | Devices | |
+ | +---------+ |
+ | |
+ | |
+ +---------------------------------------------+
+
+Extended IRQ model
+==================
+
+In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer interrupt go
+to CPUINTC directly, CPU UARTS interrupts go to LIOINTC, while all other devices
+interrupts go to PCH-PIC/PCH-LPC/PCH-MSI and gathered by EIOINTC, and then go to
+to CPUINTC directly.
+
+ +--------------------------------------------------------+
+ |:: |
+ | |
+ | +-----+ +---------+ +-------+ |
+ | | IPI | --> | CPUINTC | <-- | Timer | |
+ | +-----+ +---------+ +-------+ |
+ | ^ ^ |
+ | | | |
+ | +---------+ +---------+ +-------+ |
+ | | EIOINTC | | LIOINTC | <-- | UARTs | |
+ | +---------+ +---------+ +-------+ |
+ | ^ ^ |
+ | | | |
+ | +---------+ +---------+ |
+ | | PCH-PIC | | PCH-MSI | |
+ | +---------+ +---------+ |
+ | ^ ^ ^ |
+ | | | | |
+ | +---------+ +---------+ +---------+ |
+ | | PCH-LPC | | Devices | | Devices | |
+ | +---------+ +---------+ +---------+ |
+ | ^ |
+ | | |
+ | +---------+ |
+ | | Devices | |
+ | +---------+ |
+ | |
+ | |
+ +--------------------------------------------------------+
+
+ACPI-related definitions
+========================
+
+CPUINTC::
+
+ ACPI_MADT_TYPE_CORE_PIC;
+ struct acpi_madt_core_pic;
+ enum acpi_madt_core_pic_version;
+
+LIOINTC::
+
+ ACPI_MADT_TYPE_LIO_PIC;
+ struct acpi_madt_lio_pic;
+ enum acpi_madt_lio_pic_version;
+
+EIOINTC::
+
+ ACPI_MADT_TYPE_EIO_PIC;
+ struct acpi_madt_eio_pic;
+ enum acpi_madt_eio_pic_version;
+
+HTVECINTC::
+
+ ACPI_MADT_TYPE_HT_PIC;
+ struct acpi_madt_ht_pic;
+ enum acpi_madt_ht_pic_version;
+
+PCH-PIC::
+
+ ACPI_MADT_TYPE_BIO_PIC;
+ struct acpi_madt_bio_pic;
+ enum acpi_madt_bio_pic_version;
+
+PCH-MSI::
+
+ ACPI_MADT_TYPE_MSI_PIC;
+ struct acpi_madt_msi_pic;
+ enum acpi_madt_msi_pic_version;
+
+PCH-LPC::
+
+ ACPI_MADT_TYPE_LPC_PIC;
+ struct acpi_madt_lpc_pic;
+ enum acpi_madt_lpc_pic_version;
+
+References
+==========
+
+Documentation of Loongson-3A5000:
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-3A5000-usermanual-1.02-CN.pdf (in Chinese)
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-3A5000-usermanual-1.02-EN.pdf (in English)
+
+Documentation of Loongson's LS7A chipset:
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-7A1000-usermanual-2.00-CN.pdf (in Chinese)
+
+ https://github.com/loongson/LoongArch-Documentation/releases/latest/download/Loongson-7A1000-usermanual-2.00-EN.pdf (in English)
+
+Note: CPUINTC is CSR.ECFG/CSR.ESTAT and its interrupt controller described
+in Section 7.4 of "LoongArch Reference Manual, Vol 1"; LIOINTC is "Legacy I/O
+Interrupts" described in Section 11.1 of "Loongson 3A5000 Processor Reference
+Manual"; EIOINTC is "Extended I/O Interrupts" described in Section 11.2 of
+"Loongson 3A5000 Processor Reference Manual"; HTVECINTC is "HyperTransport
+Interrupts" described in Section 14.3 of "Loongson 3A5000 Processor Reference
+Manual"; PCH-PIC/PCH-MSI is "Interrupt Controller" described in Section 5 of
+"Loongson 7A1000 Bridge User Manual"; PCH-LPC is "LPC Interrupts" described in
+Section 24.3 of "Loongson 7A1000 Bridge User Manual".
--
2.27.0


2022-05-18 09:41:49

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 04/22] LoongArch: Add writecombine support for drm

LoongArch maintains cache coherency in hardware, but its WUC attribute
(Weak-ordered UnCached, which is similar to WC) is out of the scope of
cache coherency machanism. This means WUC can only used for write-only
memory regions.

Cc: [email protected]
Reviewed-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
drivers/gpu/drm/drm_vm.c | 2 +-
drivers/gpu/drm/ttm/ttm_module.c | 2 +-
include/drm/drm_cache.h | 8 ++++++++
3 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/drm_vm.c b/drivers/gpu/drm/drm_vm.c
index e957d4851dc0..f024dc93939e 100644
--- a/drivers/gpu/drm/drm_vm.c
+++ b/drivers/gpu/drm/drm_vm.c
@@ -69,7 +69,7 @@ static pgprot_t drm_io_prot(struct drm_local_map *map,
pgprot_t tmp = vm_get_page_prot(vma->vm_flags);

#if defined(__i386__) || defined(__x86_64__) || defined(__powerpc__) || \
- defined(__mips__)
+ defined(__mips__) || defined(__loongarch__)
if (map->type == _DRM_REGISTERS && !(map->flags & _DRM_WRITE_COMBINING))
tmp = pgprot_noncached(tmp);
else
diff --git a/drivers/gpu/drm/ttm/ttm_module.c b/drivers/gpu/drm/ttm/ttm_module.c
index a3ad7c9736ec..b3fffe7b5062 100644
--- a/drivers/gpu/drm/ttm/ttm_module.c
+++ b/drivers/gpu/drm/ttm/ttm_module.c
@@ -74,7 +74,7 @@ pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp)
#endif /* CONFIG_UML */
#endif /* __i386__ || __x86_64__ */
#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \
- defined(__powerpc__) || defined(__mips__)
+ defined(__powerpc__) || defined(__mips__) || defined(__loongarch__)
if (caching == ttm_write_combined)
tmp = pgprot_writecombine(tmp);
else
diff --git a/include/drm/drm_cache.h b/include/drm/drm_cache.h
index 22deb216b59c..08e0e3ffad13 100644
--- a/include/drm/drm_cache.h
+++ b/include/drm/drm_cache.h
@@ -67,6 +67,14 @@ static inline bool drm_arch_can_wc_memory(void)
* optimization entirely for ARM and arm64.
*/
return false;
+#elif defined(CONFIG_LOONGARCH)
+ /*
+ * LoongArch maintains cache coherency in hardware, but its WUC attribute
+ * (Weak-ordered UnCached, which is similar to WC) is out of the scope of
+ * cache coherency machanism. This means WUC can only used for write-only
+ * memory regions.
+ */
+ return false;
#else
return true;
#endif
--
2.27.0


2022-05-18 09:44:01

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 05/22] LoongArch: Add build infrastructure

Add Kbuild, Makefile, Kconfig and link script for LoongArch build
infrastructure.

Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/Kbuild | 6 +
arch/loongarch/Kconfig | 395 +++++++++++++++++++++++++
arch/loongarch/Kconfig.debug | 0
arch/loongarch/Makefile | 105 +++++++
arch/loongarch/boot/.gitignore | 2 +
arch/loongarch/boot/Makefile | 20 ++
arch/loongarch/boot/dts/Makefile | 4 +
arch/loongarch/include/asm/Kbuild | 30 ++
arch/loongarch/include/uapi/asm/Kbuild | 2 +
arch/loongarch/kernel/.gitignore | 2 +
arch/loongarch/kernel/Makefile | 21 ++
arch/loongarch/kernel/vmlinux.lds.S | 117 ++++++++
arch/loongarch/lib/Makefile | 6 +
arch/loongarch/mm/Makefile | 9 +
arch/loongarch/pci/Makefile | 7 +
arch/loongarch/vdso/.gitignore | 2 +
scripts/subarch.include | 2 +-
17 files changed, 729 insertions(+), 1 deletion(-)
create mode 100644 arch/loongarch/Kbuild
create mode 100644 arch/loongarch/Kconfig
create mode 100644 arch/loongarch/Kconfig.debug
create mode 100644 arch/loongarch/Makefile
create mode 100644 arch/loongarch/boot/.gitignore
create mode 100644 arch/loongarch/boot/Makefile
create mode 100644 arch/loongarch/boot/dts/Makefile
create mode 100644 arch/loongarch/include/asm/Kbuild
create mode 100644 arch/loongarch/include/uapi/asm/Kbuild
create mode 100644 arch/loongarch/kernel/.gitignore
create mode 100644 arch/loongarch/kernel/Makefile
create mode 100644 arch/loongarch/kernel/vmlinux.lds.S
create mode 100644 arch/loongarch/lib/Makefile
create mode 100644 arch/loongarch/mm/Makefile
create mode 100644 arch/loongarch/pci/Makefile
create mode 100644 arch/loongarch/vdso/.gitignore

diff --git a/arch/loongarch/Kbuild b/arch/loongarch/Kbuild
new file mode 100644
index 000000000000..ab5373d0a24f
--- /dev/null
+++ b/arch/loongarch/Kbuild
@@ -0,0 +1,6 @@
+obj-y += kernel/
+obj-y += mm/
+obj-y += vdso/
+
+# for cleaning
+subdir- += boot
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
new file mode 100644
index 000000000000..09902903bba8
--- /dev/null
+++ b/arch/loongarch/Kconfig
@@ -0,0 +1,395 @@
+# SPDX-License-Identifier: GPL-2.0
+config LOONGARCH
+ bool
+ default y
+ select ACPI_MCFG if ACPI
+ select ACPI_SYSTEM_POWER_STATES_SUPPORT if ACPI
+ select ARCH_BINFMT_ELF_STATE
+ select ARCH_ENABLE_MEMORY_HOTPLUG
+ select ARCH_ENABLE_MEMORY_HOTREMOVE
+ select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI
+ select ARCH_HAS_PTE_SPECIAL
+ select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
+ select ARCH_INLINE_READ_LOCK if !PREEMPTION
+ select ARCH_INLINE_READ_LOCK_BH if !PREEMPTION
+ select ARCH_INLINE_READ_LOCK_IRQ if !PREEMPTION
+ select ARCH_INLINE_READ_LOCK_IRQSAVE if !PREEMPTION
+ select ARCH_INLINE_READ_UNLOCK if !PREEMPTION
+ select ARCH_INLINE_READ_UNLOCK_BH if !PREEMPTION
+ select ARCH_INLINE_READ_UNLOCK_IRQ if !PREEMPTION
+ select ARCH_INLINE_READ_UNLOCK_IRQRESTORE if !PREEMPTION
+ select ARCH_INLINE_WRITE_LOCK if !PREEMPTION
+ select ARCH_INLINE_WRITE_LOCK_BH if !PREEMPTION
+ select ARCH_INLINE_WRITE_LOCK_IRQ if !PREEMPTION
+ select ARCH_INLINE_WRITE_LOCK_IRQSAVE if !PREEMPTION
+ select ARCH_INLINE_WRITE_UNLOCK if !PREEMPTION
+ select ARCH_INLINE_WRITE_UNLOCK_BH if !PREEMPTION
+ select ARCH_INLINE_WRITE_UNLOCK_IRQ if !PREEMPTION
+ select ARCH_INLINE_WRITE_UNLOCK_IRQRESTORE if !PREEMPTION
+ select ARCH_INLINE_SPIN_TRYLOCK if !PREEMPTION
+ select ARCH_INLINE_SPIN_TRYLOCK_BH if !PREEMPTION
+ select ARCH_INLINE_SPIN_LOCK if !PREEMPTION
+ select ARCH_INLINE_SPIN_LOCK_BH if !PREEMPTION
+ select ARCH_INLINE_SPIN_LOCK_IRQ if !PREEMPTION
+ select ARCH_INLINE_SPIN_LOCK_IRQSAVE if !PREEMPTION
+ select ARCH_INLINE_SPIN_UNLOCK if !PREEMPTION
+ select ARCH_INLINE_SPIN_UNLOCK_BH if !PREEMPTION
+ select ARCH_INLINE_SPIN_UNLOCK_IRQ if !PREEMPTION
+ select ARCH_INLINE_SPIN_UNLOCK_IRQRESTORE if !PREEMPTION
+ select ARCH_MIGHT_HAVE_PC_PARPORT
+ select ARCH_MIGHT_HAVE_PC_SERIO
+ select ARCH_SPARSEMEM_ENABLE
+ select ARCH_SUPPORTS_ACPI
+ select ARCH_SUPPORTS_ATOMIC_RMW
+ select ARCH_SUPPORTS_HUGETLBFS
+ select ARCH_USE_BUILTIN_BSWAP
+ select ARCH_USE_CMPXCHG_LOCKREF
+ select ARCH_USE_QUEUED_RWLOCKS
+ select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT
+ select ARCH_WANTS_NO_INSTR
+ select BUILDTIME_TABLE_SORT
+ select COMMON_CLK
+ select GENERIC_CLOCKEVENTS
+ select GENERIC_CMOS_UPDATE
+ select GENERIC_CPU_AUTOPROBE
+ select GENERIC_ENTRY
+ select GENERIC_FIND_FIRST_BIT
+ select GENERIC_GETTIMEOFDAY
+ select GENERIC_IRQ_MULTI_HANDLER
+ select GENERIC_IRQ_PROBE
+ select GENERIC_IRQ_SHOW
+ select GENERIC_LIB_ASHLDI3
+ select GENERIC_LIB_ASHRDI3
+ select GENERIC_LIB_CMPDI2
+ select GENERIC_LIB_LSHRDI3
+ select GENERIC_LIB_UCMPDI2
+ select GENERIC_PCI_IOMAP
+ select GENERIC_SCHED_CLOCK
+ select GENERIC_TIME_VSYSCALL
+ select GPIOLIB
+ select HAVE_ARCH_AUDITSYSCALL
+ select HAVE_ARCH_COMPILER_H
+ select HAVE_ARCH_MMAP_RND_BITS if MMU
+ select HAVE_ARCH_SECCOMP_FILTER
+ select HAVE_ARCH_TRACEHOOK
+ select HAVE_ARCH_TRANSPARENT_HUGEPAGE
+ select HAVE_ASM_MODVERSIONS
+ select HAVE_CONTEXT_TRACKING
+ select HAVE_COPY_THREAD_TLS
+ select HAVE_DEBUG_KMEMLEAK
+ select HAVE_DEBUG_STACKOVERFLOW
+ select HAVE_DMA_CONTIGUOUS
+ select HAVE_EXIT_THREAD
+ select HAVE_FAST_GUP
+ select HAVE_GENERIC_VDSO
+ select HAVE_IOREMAP_PROT
+ select HAVE_IRQ_EXIT_ON_IRQ_STACK
+ select HAVE_IRQ_TIME_ACCOUNTING
+ select HAVE_MEMBLOCK
+ select HAVE_MEMBLOCK_NODE_MAP
+ select HAVE_MOD_ARCH_SPECIFIC
+ select HAVE_NMI
+ select HAVE_PCI
+ select HAVE_PERF_EVENTS
+ select HAVE_REGS_AND_STACK_ACCESS_API
+ select HAVE_RSEQ
+ select HAVE_SYSCALL_TRACEPOINTS
+ select HAVE_TIF_NOHZ
+ select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select IRQ_FORCED_THREADING
+ select IRQ_LOONGARCH_CPU
+ select MODULES_USE_ELF_RELA if MODULES
+ select OF
+ select OF_EARLY_FLATTREE
+ select PCI
+ select PCI_DOMAINS_GENERIC
+ select PCI_ECAM if ACPI
+ select PCI_MSI_ARCH_FALLBACKS
+ select PERF_USE_VMALLOC
+ select RTC_LIB
+ select SPARSE_IRQ
+ select SYSCTL_EXCEPTION_TRACE
+ select SWIOTLB
+ select TRACE_IRQFLAGS_SUPPORT
+ select ZONE_DMA32
+
+config 32BIT
+ bool
+
+config 64BIT
+ def_bool y
+
+config CPU_HAS_FPU
+ bool
+ default y
+
+config CPU_HAS_PREFETCH
+ bool
+ default y
+
+config GENERIC_CALIBRATE_DELAY
+ def_bool y
+
+config GENERIC_CSUM
+ def_bool y
+
+config GENERIC_HWEIGHT
+ def_bool y
+
+config L1_CACHE_SHIFT
+ int
+ default "6"
+
+config LOCKDEP_SUPPORT
+ bool
+ default y
+
+# MACH_LOONGSON32 and MACH_LOONGSON64 are delibrately carried over from the
+# MIPS Loongson code, to preserve Loongson-specific code paths in drivers that
+# are shared between architectures, and specifically expecting the symbols.
+config MACH_LOONGSON32
+ def_bool 32BIT
+
+config MACH_LOONGSON64
+ def_bool 64BIT
+
+config PAGE_SIZE_4KB
+ bool
+
+config PAGE_SIZE_16KB
+ bool
+
+config PAGE_SIZE_64KB
+ bool
+
+config PGTABLE_2LEVEL
+ bool
+
+config PGTABLE_3LEVEL
+ bool
+
+config PGTABLE_4LEVEL
+ bool
+
+config PGTABLE_LEVELS
+ int
+ default 2 if PGTABLE_2LEVEL
+ default 3 if PGTABLE_3LEVEL
+ default 4 if PGTABLE_4LEVEL
+
+config SCHED_OMIT_FRAME_POINTER
+ bool
+ default y
+
+menu "Kernel type and options"
+
+source "kernel/Kconfig.hz"
+
+choice
+ prompt "Page Table Layout"
+ default 16KB_2LEVEL if 32BIT
+ default 16KB_3LEVEL if 64BIT
+ help
+ Allows choosing the page table layout, which is a combination
+ of page size and page table levels. The size of virtual memory
+ address space are determined by the page table layout.
+
+config 4KB_3LEVEL
+ bool "4KB with 3 levels"
+ select PAGE_SIZE_4KB
+ select PGTABLE_3LEVEL
+ help
+ This option selects 4KB page size with 3 level page tables, which
+ support a maximum of 39 bits of application virtual memory.
+
+config 4KB_4LEVEL
+ bool "4KB with 4 levels"
+ select PAGE_SIZE_4KB
+ select PGTABLE_4LEVEL
+ help
+ This option selects 4KB page size with 4 level page tables, which
+ support a maximum of 48 bits of application virtual memory.
+
+config 16KB_2LEVEL
+ bool "16KB with 2 levels"
+ select PAGE_SIZE_16KB
+ select PGTABLE_2LEVEL
+ help
+ This option selects 16KB page size with 2 level page tables, which
+ support a maximum of 36 bits of application virtual memory.
+
+config 16KB_3LEVEL
+ bool "16KB with 3 levels"
+ select PAGE_SIZE_16KB
+ select PGTABLE_3LEVEL
+ help
+ This option selects 16KB page size with 3 level page tables, which
+ support a maximum of 47 bits of application virtual memory.
+
+config 64KB_2LEVEL
+ bool "64KB with 2 levels"
+ select PAGE_SIZE_64KB
+ select PGTABLE_2LEVEL
+ help
+ This option selects 64KB page size with 2 level page tables, which
+ support a maximum of 42 bits of application virtual memory.
+
+config 64KB_3LEVEL
+ bool "64KB with 3 levels"
+ select PAGE_SIZE_64KB
+ select PGTABLE_3LEVEL
+ help
+ This option selects 64KB page size with 3 level page tables, which
+ support a maximum of 55 bits of application virtual memory.
+
+endchoice
+
+config CMDLINE
+ string "Built-in kernel command line"
+ help
+ For most platforms, the arguments for the kernel's command line
+ are provided at run-time, during boot. However, there are cases
+ where either no arguments are being provided or the provided
+ arguments are insufficient or even invalid.
+
+ When that occurs, it is possible to define a built-in command
+ line here and choose how the kernel should use it later on.
+
+choice
+ prompt "Kernel command line type"
+ default CMDLINE_BOOTLOADER
+ help
+ Choose how the kernel will handle the provided built-in command
+ line.
+
+config CMDLINE_BOOTLOADER
+ bool "Use bootloader kernel arguments if available"
+ help
+ Prefer the command-line passed by the boot loader if available.
+ Use the built-in command line as fallback in case we get nothing
+ during boot. This is the default behaviour.
+
+config CMDLINE_EXTEND
+ bool "Use built-in to extend bootloader kernel arguments"
+ help
+ The command-line arguments provided during boot will be
+ appended to the built-in command line. This is useful in
+ cases where the provided arguments are insufficient and
+ you don't want to or cannot modify them.
+
+config CMDLINE_FORCE
+ bool "Always use the built-in kernel command string"
+ help
+ Always use the built-in command line, even if we get one during
+ boot. This is useful in case you need to override the provided
+ command line on systems where you don't have or want control
+ over it.
+
+endchoice
+
+config DMI
+ bool "Enable DMI scanning"
+ select DMI_SCAN_MACHINE_NON_EFI_FALLBACK
+ default y
+ help
+ This enables SMBIOS/DMI feature for systems, and scanning of
+ DMI to identify machine quirks.
+
+config EFI
+ bool "EFI runtime service support"
+ select UCS2_STRING
+ select EFI_PARAMS_FROM_FDT
+ select EFI_RUNTIME_WRAPPERS
+ help
+ This enables the kernel to use EFI runtime services that are
+ available (such as the EFI variable services).
+
+config EFI_STUB
+ bool "EFI boot stub support"
+ default y
+ depends on EFI
+ select EFI_GENERIC_STUB
+ help
+ This kernel feature allows the kernel to be loaded directly by
+ EFI firmware without the use of a bootloader.
+
+config FORCE_MAX_ZONEORDER
+ int "Maximum zone order"
+ range 14 64 if PAGE_SIZE_64KB
+ default "14" if PAGE_SIZE_64KB
+ range 12 64 if PAGE_SIZE_16KB
+ default "12" if PAGE_SIZE_16KB
+ range 11 64
+ default "11"
+ help
+ The kernel memory allocator divides physically contiguous memory
+ blocks into "zones", where each zone is a power of two number of
+ pages. This option selects the largest power of two that the kernel
+ keeps in the memory allocator. If you need to allocate very large
+ blocks of physically contiguous memory, then you may need to
+ increase this value.
+
+ This config option is actually maximum order plus one. For example,
+ a value of 11 means that the largest free memory block is 2^10 pages.
+
+ The page size is not necessarily 4KB. Keep this in mind
+ when choosing a value for this option.
+
+config SECCOMP
+ bool "Enable seccomp to safely compute untrusted bytecode"
+ depends on PROC_FS
+ default y
+ help
+ This kernel feature is useful for number crunching applications
+ that may need to compute untrusted bytecode during their
+ execution. By using pipes or other transports made available to
+ the process as file descriptors supporting the read/write
+ syscalls, it's possible to isolate those applications in
+ their own address space using seccomp. Once seccomp is
+ enabled via /proc/<pid>/seccomp, it cannot be disabled
+ and the task is only allowed to execute a few safe syscalls
+ defined by each seccomp mode.
+
+ If unsure, say Y. Only embedded should say N here.
+
+endmenu
+
+config ARCH_SELECT_MEMORY_MODEL
+ def_bool y
+
+config ARCH_FLATMEM_ENABLE
+ def_bool y
+
+config ARCH_SPARSEMEM_ENABLE
+ def_bool y
+ help
+ Say Y to support efficient handling of sparse physical memory,
+ for architectures which are either NUMA (Non-Uniform Memory Access)
+ or have huge holes in the physical address space for other reasons.
+ See <file:Documentation/vm/numa.rst> for more.
+
+config ARCH_ENABLE_THP_MIGRATION
+ def_bool y
+ depends on TRANSPARENT_HUGEPAGE
+
+config ARCH_MEMORY_PROBE
+ def_bool y
+ depends on MEMORY_HOTPLUG
+
+config MMU
+ bool
+ default y
+
+config ARCH_MMAP_RND_BITS_MIN
+ default 12
+
+config ARCH_MMAP_RND_BITS_MAX
+ default 18
+
+menu "Power management options"
+
+source "drivers/acpi/Kconfig"
+
+endmenu
+
+source "drivers/firmware/Kconfig"
diff --git a/arch/loongarch/Kconfig.debug b/arch/loongarch/Kconfig.debug
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile
new file mode 100644
index 000000000000..6dcc0495bec9
--- /dev/null
+++ b/arch/loongarch/Makefile
@@ -0,0 +1,105 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Author: Huacai Chen <[email protected]>
+# Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+
+boot := arch/loongarch/boot
+
+ifndef CONFIG_EFI_STUB
+KBUILD_IMAGE = $(boot)/vmlinux
+else
+KBUILD_IMAGE = $(boot)/vmlinux.efi
+endif
+
+#
+# Select the object file format to substitute into the linker script.
+#
+64bit-tool-archpref = loongarch64
+32bit-bfd = elf32-loongarch
+64bit-bfd = elf64-loongarch
+32bit-emul = elf32loongarch
+64bit-emul = elf64loongarch
+
+ifdef CONFIG_64BIT
+tool-archpref = $(64bit-tool-archpref)
+UTS_MACHINE := loongarch64
+endif
+
+ifneq ($(SUBARCH),$(ARCH))
+ ifeq ($(CROSS_COMPILE),)
+ CROSS_COMPILE := $(call cc-cross-prefix, $(tool-archpref)-linux- $(tool-archpref)-linux-gnu- $(tool-archpref)-unknown-linux-gnu-)
+ endif
+endif
+
+ifdef CONFIG_64BIT
+ld-emul = $(64bit-emul)
+cflags-y += -mabi=lp64s
+endif
+
+cflags-y += -G0 -pipe -msoft-float
+LDFLAGS_vmlinux += -G0 -static -n -nostdlib
+KBUILD_AFLAGS_KERNEL += -Wa,-mla-global-with-pcrel
+KBUILD_CFLAGS_KERNEL += -Wa,-mla-global-with-pcrel
+KBUILD_AFLAGS_MODULE += -Wa,-mla-global-with-abs
+KBUILD_CFLAGS_MODULE += -fplt -Wa,-mla-global-with-abs,-mla-local-with-abs
+
+cflags-y += -ffreestanding
+cflags-y += $(call cc-option, -mno-check-zero-division)
+
+load-y = 0x9000000000200000
+bootvars-y = VMLINUX_LOAD_ADDRESS=$(load-y)
+
+drivers-$(CONFIG_PCI) += arch/loongarch/pci/
+
+KBUILD_AFLAGS += $(cflags-y)
+KBUILD_CFLAGS += $(cflags-y)
+KBUILD_CPPFLAGS += -DVMLINUX_LOAD_ADDRESS=$(load-y)
+
+# This is required to get dwarf unwinding tables into .debug_frame
+# instead of .eh_frame so we don't discard them.
+KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
+
+# Don't emit unaligned accesses.
+# Not all LoongArch cores support unaligned access, and as kernel we can't
+# rely on others to provide emulation for these accesses.
+KBUILD_CFLAGS += $(call cc-option,-mstrict-align)
+
+KBUILD_CFLAGS += -isystem $(shell $(CC) -print-file-name=include)
+
+KBUILD_LDFLAGS += -m $(ld-emul)
+
+ifdef CONFIG_LOONGARCH
+CHECKFLAGS += $(shell $(CC) $(KBUILD_CFLAGS) -dM -E -x c /dev/null | \
+ egrep -vw '__GNUC_(MINOR_|PATCHLEVEL_)?_' | \
+ sed -e "s/^\#define /-D'/" -e "s/ /'='/" -e "s/$$/'/" -e 's/\$$/&&/g')
+endif
+
+head-y := arch/loongarch/kernel/head.o
+
+libs-y += arch/loongarch/lib/
+libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
+
+ifeq ($(KBUILD_EXTMOD),)
+prepare: vdso_prepare
+vdso_prepare: prepare0
+ $(Q)$(MAKE) $(build)=arch/loongarch/vdso include/generated/vdso-offsets.h
+endif
+
+PHONY += vdso_install
+vdso_install:
+ $(Q)$(MAKE) $(build)=arch/loongarch/vdso $@
+
+all: $(KBUILD_IMAGE)
+
+$(KBUILD_IMAGE): vmlinux
+ $(Q)$(MAKE) $(build)=$(boot) $(bootvars-y) $@
+
+install:
+ $(Q)install -D -m 755 $(KBUILD_IMAGE) $(INSTALL_PATH)/vmlinux-$(KERNELRELEASE)
+ $(Q)install -D -m 644 .config $(INSTALL_PATH)/config-$(KERNELRELEASE)
+ $(Q)install -D -m 644 System.map $(INSTALL_PATH)/System.map-$(KERNELRELEASE)
+
+define archhelp
+ echo ' install - install kernel into $(INSTALL_PATH)'
+ echo
+endef
diff --git a/arch/loongarch/boot/.gitignore b/arch/loongarch/boot/.gitignore
new file mode 100644
index 000000000000..49423ee96ef3
--- /dev/null
+++ b/arch/loongarch/boot/.gitignore
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+vmlinux*
diff --git a/arch/loongarch/boot/Makefile b/arch/loongarch/boot/Makefile
new file mode 100644
index 000000000000..b39d50a7a3df
--- /dev/null
+++ b/arch/loongarch/boot/Makefile
@@ -0,0 +1,20 @@
+#
+# arch/loongarch/boot/Makefile
+#
+# Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+#
+
+drop-sections := .comment .note .options .note.gnu.build-id
+strip-flags := $(addprefix --remove-section=,$(drop-sections)) -S
+OBJCOPYFLAGS_vmlinux.efi := -O binary $(strip-flags)
+
+targets := vmlinux
+quiet_cmd_strip = STRIP $@
+ cmd_strip = $(STRIP) -s -o $@ $<
+
+$(obj)/vmlinux: vmlinux FORCE
+ $(call if_changed,strip)
+
+targets += vmlinux.efi
+$(obj)/vmlinux.efi: $(obj)/vmlinux FORCE
+ $(call if_changed,objcopy)
diff --git a/arch/loongarch/boot/dts/Makefile b/arch/loongarch/boot/dts/Makefile
new file mode 100644
index 000000000000..5f1f55e911ad
--- /dev/null
+++ b/arch/loongarch/boot/dts/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0-only
+dtstree := $(srctree)/$(src)
+
+dtb-y := $(patsubst $(dtstree)/%.dts,%.dtb, $(wildcard $(dtstree)/*.dts))
diff --git a/arch/loongarch/include/asm/Kbuild b/arch/loongarch/include/asm/Kbuild
new file mode 100644
index 000000000000..83bc0681e72b
--- /dev/null
+++ b/arch/loongarch/include/asm/Kbuild
@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: GPL-2.0
+generic-y += dma-contiguous.h
+generic-y += export.h
+generic-y += parport.h
+generic-y += early_ioremap.h
+generic-y += qrwlock.h
+generic-y += qrwlock_types.h
+generic-y += spinlock.h
+generic-y += spinlock_types.h
+generic-y += rwsem.h
+generic-y += segment.h
+generic-y += user.h
+generic-y += stat.h
+generic-y += fcntl.h
+generic-y += ioctl.h
+generic-y += ioctls.h
+generic-y += mman.h
+generic-y += msgbuf.h
+generic-y += sembuf.h
+generic-y += shmbuf.h
+generic-y += statfs.h
+generic-y += socket.h
+generic-y += sockios.h
+generic-y += termios.h
+generic-y += termbits.h
+generic-y += poll.h
+generic-y += param.h
+generic-y += posix_types.h
+generic-y += resource.h
+generic-y += kvm_para.h
diff --git a/arch/loongarch/include/uapi/asm/Kbuild b/arch/loongarch/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..4aa680ca2e5f
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/Kbuild
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+generic-y += kvm_para.h
diff --git a/arch/loongarch/kernel/.gitignore b/arch/loongarch/kernel/.gitignore
new file mode 100644
index 000000000000..bbb90f92d051
--- /dev/null
+++ b/arch/loongarch/kernel/.gitignore
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+vmlinux.lds
diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
new file mode 100644
index 000000000000..e5a3b2fb9961
--- /dev/null
+++ b/arch/loongarch/kernel/Makefile
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Linux/LoongArch kernel.
+#
+
+extra-y := head.o vmlinux.lds
+
+obj-y += cpu-probe.o cacheinfo.o env.o setup.o entry.o genex.o \
+ traps.o irq.o idle.o process.o dma.o mem.o io.o reset.o switch.o \
+ elf.o syscall.o signal.o time.o topology.o inst.o ptrace.o vdso.o
+
+obj-$(CONFIG_ACPI) += acpi.o
+obj-$(CONFIG_EFI) += efi.o
+
+obj-$(CONFIG_CPU_HAS_FPU) += fpu.o
+
+obj-$(CONFIG_MODULES) += module.o module-sections.o
+
+obj-$(CONFIG_PROC_FS) += proc.o
+
+CPPFLAGS_vmlinux.lds := $(KBUILD_CFLAGS)
diff --git a/arch/loongarch/kernel/vmlinux.lds.S b/arch/loongarch/kernel/vmlinux.lds.S
new file mode 100644
index 000000000000..7da4c4d7c50d
--- /dev/null
+++ b/arch/loongarch/kernel/vmlinux.lds.S
@@ -0,0 +1,117 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/sizes.h>
+#include <asm/asm-offsets.h>
+#include <asm/thread_info.h>
+
+#define PAGE_SIZE _PAGE_SIZE
+
+/*
+ * Put .bss..swapper_pg_dir as the first thing in .bss. This will
+ * ensure that it has .bss alignment (64K).
+ */
+#define BSS_FIRST_SECTIONS *(.bss..swapper_pg_dir)
+
+#include <asm-generic/vmlinux.lds.h>
+#include "image-vars.h"
+
+/*
+ * Max avaliable Page Size is 64K, so we set SectionAlignment
+ * field of EFI application to 64K.
+ */
+PECOFF_FILE_ALIGN = 0x200;
+PECOFF_SEGMENT_ALIGN = 0x10000;
+
+OUTPUT_ARCH(loongarch)
+ENTRY(kernel_entry)
+PHDRS {
+ text PT_LOAD FLAGS(7); /* RWX */
+ note PT_NOTE FLAGS(4); /* R__ */
+}
+
+jiffies = jiffies_64;
+
+SECTIONS
+{
+ . = VMLINUX_LOAD_ADDRESS;
+
+ _text = .;
+ HEAD_TEXT_SECTION
+
+ . = ALIGN(PECOFF_SEGMENT_ALIGN);
+ .text : {
+ TEXT_TEXT
+ SCHED_TEXT
+ CPUIDLE_TEXT
+ LOCK_TEXT
+ KPROBES_TEXT
+ IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
+ *(.fixup)
+ *(.gnu.warning)
+ } :text = 0
+ . = ALIGN(PECOFF_SEGMENT_ALIGN);
+ _etext = .;
+
+ EXCEPTION_TABLE(16)
+
+ . = ALIGN(PECOFF_SEGMENT_ALIGN);
+ __init_begin = .;
+ __inittext_begin = .;
+
+ INIT_TEXT_SECTION(PAGE_SIZE)
+ .exit.text : {
+ EXIT_TEXT
+ }
+
+ . = ALIGN(PECOFF_SEGMENT_ALIGN);
+ __inittext_end = .;
+
+ __initdata_begin = .;
+
+ INIT_DATA_SECTION(16)
+ .exit.data : {
+ EXIT_DATA
+ }
+
+ .init.bss : {
+ *(.init.bss)
+ }
+ . = ALIGN(PECOFF_SEGMENT_ALIGN);
+ __initdata_end = .;
+
+ __init_end = .;
+
+ _sdata = .;
+ RO_DATA(4096)
+ RW_DATA(1 << CONFIG_L1_CACHE_SHIFT, PAGE_SIZE, THREAD_SIZE)
+
+ .sdata : {
+ *(.sdata)
+ }
+ .edata_padding : { BYTE(0); . = ALIGN(PECOFF_FILE_ALIGN); }
+ _edata = .;
+
+ BSS_SECTION(0, SZ_64K, 8)
+ . = ALIGN(PECOFF_SEGMENT_ALIGN);
+
+ _end = .;
+
+ STABS_DEBUG
+ DWARF_DEBUG
+
+ .gptab.sdata : {
+ *(.gptab.data)
+ *(.gptab.sdata)
+ }
+ .gptab.sbss : {
+ *(.gptab.bss)
+ *(.gptab.sbss)
+ }
+
+ DISCARDS
+ /DISCARD/ : {
+ *(.gnu.attributes)
+ *(.options)
+ *(.eh_frame)
+ }
+}
diff --git a/arch/loongarch/lib/Makefile b/arch/loongarch/lib/Makefile
new file mode 100644
index 000000000000..e36635fccb69
--- /dev/null
+++ b/arch/loongarch/lib/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for LoongArch-specific library files.
+#
+
+lib-y += delay.o clear_user.o copy_user.o dump_tlb.o
diff --git a/arch/loongarch/mm/Makefile b/arch/loongarch/mm/Makefile
new file mode 100644
index 000000000000..8ffc6383f836
--- /dev/null
+++ b/arch/loongarch/mm/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Linux/LoongArch-specific parts of the memory manager.
+#
+
+obj-y += init.o cache.o tlb.o tlbex.o extable.o \
+ fault.o ioremap.o maccess.o mmap.o pgtable.o page.o
+
+obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
diff --git a/arch/loongarch/pci/Makefile b/arch/loongarch/pci/Makefile
new file mode 100644
index 000000000000..8101ef3df71c
--- /dev/null
+++ b/arch/loongarch/pci/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the PCI specific kernel interface routines under Linux.
+#
+
+obj-y += pci.o
+obj-$(CONFIG_ACPI) += acpi.o
diff --git a/arch/loongarch/vdso/.gitignore b/arch/loongarch/vdso/.gitignore
new file mode 100644
index 000000000000..652e31d82582
--- /dev/null
+++ b/arch/loongarch/vdso/.gitignore
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+vdso.lds
diff --git a/scripts/subarch.include b/scripts/subarch.include
index 776849a3c500..4bd327d0ae42 100644
--- a/scripts/subarch.include
+++ b/scripts/subarch.include
@@ -10,4 +10,4 @@ SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \
-e s/s390x/s390/ \
-e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
-e s/sh[234].*/sh/ -e s/aarch64.*/arm64/ \
- -e s/riscv.*/riscv/)
+ -e s/riscv.*/riscv/ -e s/loongarch.*/loongarch/)
--
2.27.0


2022-05-18 09:44:18

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 06/22] LoongArch: Add CPU definition headers

Add common headers (CPU definition and address space layout) for basic
LoongArch support.

Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/addrspace.h | 112 ++
arch/loongarch/include/asm/cpu-features.h | 73 +
arch/loongarch/include/asm/cpu-info.h | 116 ++
arch/loongarch/include/asm/cpu.h | 127 ++
arch/loongarch/include/asm/fpregdef.h | 53 +
arch/loongarch/include/asm/loongarch.h | 1519 +++++++++++++++++++++
arch/loongarch/include/asm/loongson.h | 153 +++
arch/loongarch/include/asm/regdef.h | 41 +
8 files changed, 2194 insertions(+)
create mode 100644 arch/loongarch/include/asm/addrspace.h
create mode 100644 arch/loongarch/include/asm/cpu-features.h
create mode 100644 arch/loongarch/include/asm/cpu-info.h
create mode 100644 arch/loongarch/include/asm/cpu.h
create mode 100644 arch/loongarch/include/asm/fpregdef.h
create mode 100644 arch/loongarch/include/asm/loongarch.h
create mode 100644 arch/loongarch/include/asm/loongson.h
create mode 100644 arch/loongarch/include/asm/regdef.h

diff --git a/arch/loongarch/include/asm/addrspace.h b/arch/loongarch/include/asm/addrspace.h
new file mode 100644
index 000000000000..b91e0733b2e5
--- /dev/null
+++ b/arch/loongarch/include/asm/addrspace.h
@@ -0,0 +1,112 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1996, 99 Ralf Baechle
+ * Copyright (C) 2000, 2002 Maciej W. Rozycki
+ * Copyright (C) 1990, 1999 by Silicon Graphics, Inc.
+ */
+#ifndef _ASM_ADDRSPACE_H
+#define _ASM_ADDRSPACE_H
+
+#include <linux/const.h>
+
+#include <asm/loongarch.h>
+
+/*
+ * This gives the physical RAM offset.
+ */
+#ifndef __ASSEMBLY__
+#ifndef PHYS_OFFSET
+#define PHYS_OFFSET _AC(0, UL)
+#endif
+extern unsigned long vm_map_base;
+#endif /* __ASSEMBLY__ */
+
+#ifndef IO_BASE
+#define IO_BASE CSR_DMW0_BASE
+#endif
+
+#ifndef CACHE_BASE
+#define CACHE_BASE CSR_DMW1_BASE
+#endif
+
+#ifndef UNCACHE_BASE
+#define UNCACHE_BASE CSR_DMW0_BASE
+#endif
+
+#define DMW_PABITS 48
+#define TO_PHYS_MASK ((1ULL << DMW_PABITS) - 1)
+
+/*
+ * Memory above this physical address will be considered highmem.
+ */
+#ifndef HIGHMEM_START
+#define HIGHMEM_START (_AC(1, UL) << _AC(DMW_PABITS, UL))
+#endif
+
+#define TO_PHYS(x) ( ((x) & TO_PHYS_MASK))
+#define TO_CACHE(x) (CACHE_BASE | ((x) & TO_PHYS_MASK))
+#define TO_UNCACHE(x) (UNCACHE_BASE | ((x) & TO_PHYS_MASK))
+
+/*
+ * This handles the memory map.
+ */
+#ifndef PAGE_OFFSET
+#define PAGE_OFFSET (CACHE_BASE + PHYS_OFFSET)
+#endif
+
+#ifndef FIXADDR_TOP
+#define FIXADDR_TOP ((unsigned long)(long)(int)0xfffe0000)
+#endif
+
+#ifdef __ASSEMBLY__
+#define _ATYPE_
+#define _ATYPE32_
+#define _ATYPE64_
+#define _CONST64_(x) x
+#else
+#define _ATYPE_ __PTRDIFF_TYPE__
+#define _ATYPE32_ int
+#define _ATYPE64_ __s64
+#ifdef CONFIG_64BIT
+#define _CONST64_(x) x ## L
+#else
+#define _CONST64_(x) x ## LL
+#endif
+#endif
+
+/*
+ * 32/64-bit LoongArch address spaces
+ */
+#ifdef __ASSEMBLY__
+#define _ACAST32_
+#define _ACAST64_
+#else
+#define _ACAST32_ (_ATYPE_)(_ATYPE32_) /* widen if necessary */
+#define _ACAST64_ (_ATYPE64_) /* do _not_ narrow */
+#endif
+
+#ifdef CONFIG_32BIT
+
+#define UVRANGE 0x00000000
+#define KPRANGE0 0x80000000
+#define KPRANGE1 0xa0000000
+#define KVRANGE 0xc0000000
+
+#else
+
+#define XUVRANGE _CONST64_(0x0000000000000000)
+#define XSPRANGE _CONST64_(0x4000000000000000)
+#define XKPRANGE _CONST64_(0x8000000000000000)
+#define XKVRANGE _CONST64_(0xc000000000000000)
+
+#endif
+
+/*
+ * Returns the physical address of a KPRANGEx / XKPRANGE address
+ */
+#define PHYSADDR(a) ((_ACAST64_(a)) & TO_PHYS_MASK)
+
+#endif /* _ASM_ADDRSPACE_H */
diff --git a/arch/loongarch/include/asm/cpu-features.h b/arch/loongarch/include/asm/cpu-features.h
new file mode 100644
index 000000000000..a8d87c40a0eb
--- /dev/null
+++ b/arch/loongarch/include/asm/cpu-features.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 2003, 2004 Ralf Baechle
+ * Copyright (C) 2004 Maciej W. Rozycki
+ */
+#ifndef __ASM_CPU_FEATURES_H
+#define __ASM_CPU_FEATURES_H
+
+#include <asm/cpu.h>
+#include <asm/cpu-info.h>
+
+#define cpu_opt(opt) (cpu_data[0].options & (opt))
+#define cpu_has(feat) (cpu_data[0].options & BIT_ULL(feat))
+
+#define cpu_has_loongarch (cpu_has_loongarch32 | cpu_has_loongarch64)
+#define cpu_has_loongarch32 (cpu_data[0].isa_level & LOONGARCH_CPU_ISA_32BIT)
+#define cpu_has_loongarch64 (cpu_data[0].isa_level & LOONGARCH_CPU_ISA_64BIT)
+
+#define cpu_icache_line_size() cpu_data[0].icache.linesz
+#define cpu_dcache_line_size() cpu_data[0].dcache.linesz
+#define cpu_vcache_line_size() cpu_data[0].vcache.linesz
+#define cpu_scache_line_size() cpu_data[0].scache.linesz
+
+#ifdef CONFIG_32BIT
+# define cpu_has_64bits (cpu_data[0].isa_level & LOONGARCH_CPU_ISA_64BIT)
+# define cpu_vabits 31
+# define cpu_pabits 31
+#endif
+
+#ifdef CONFIG_64BIT
+# define cpu_has_64bits 1
+# define cpu_vabits cpu_data[0].vabits
+# define cpu_pabits cpu_data[0].pabits
+# define __NEED_ADDRBITS_PROBE
+#endif
+
+/*
+ * SMP assumption: Options of CPU 0 are a superset of all processors.
+ * This is true for all known LoongArch systems.
+ */
+#define cpu_has_cpucfg cpu_opt(LOONGARCH_CPU_CPUCFG)
+#define cpu_has_lam cpu_opt(LOONGARCH_CPU_LAM)
+#define cpu_has_ual cpu_opt(LOONGARCH_CPU_UAL)
+#define cpu_has_fpu cpu_opt(LOONGARCH_CPU_FPU)
+#define cpu_has_lsx cpu_opt(LOONGARCH_CPU_LSX)
+#define cpu_has_lasx cpu_opt(LOONGARCH_CPU_LASX)
+#define cpu_has_complex cpu_opt(LOONGARCH_CPU_COMPLEX)
+#define cpu_has_crypto cpu_opt(LOONGARCH_CPU_CRYPTO)
+#define cpu_has_lvz cpu_opt(LOONGARCH_CPU_LVZ)
+#define cpu_has_lbt_x86 cpu_opt(LOONGARCH_CPU_LBT_X86)
+#define cpu_has_lbt_arm cpu_opt(LOONGARCH_CPU_LBT_ARM)
+#define cpu_has_lbt_mips cpu_opt(LOONGARCH_CPU_LBT_MIPS)
+#define cpu_has_lbt (cpu_has_lbt_x86|cpu_has_lbt_arm|cpu_has_lbt_mips)
+#define cpu_has_csr cpu_opt(LOONGARCH_CPU_CSR)
+#define cpu_has_tlb cpu_opt(LOONGARCH_CPU_TLB)
+#define cpu_has_watch cpu_opt(LOONGARCH_CPU_WATCH)
+#define cpu_has_vint cpu_opt(LOONGARCH_CPU_VINT)
+#define cpu_has_csripi cpu_opt(LOONGARCH_CPU_CSRIPI)
+#define cpu_has_extioi cpu_opt(LOONGARCH_CPU_EXTIOI)
+#define cpu_has_prefetch cpu_opt(LOONGARCH_CPU_PREFETCH)
+#define cpu_has_pmp cpu_opt(LOONGARCH_CPU_PMP)
+#define cpu_has_perf cpu_opt(LOONGARCH_CPU_PMP)
+#define cpu_has_scalefreq cpu_opt(LOONGARCH_CPU_SCALEFREQ)
+#define cpu_has_flatmode cpu_opt(LOONGARCH_CPU_FLATMODE)
+#define cpu_has_eiodecode cpu_opt(LOONGARCH_CPU_EIODECODE)
+#define cpu_has_guestid cpu_opt(LOONGARCH_CPU_GUESTID)
+#define cpu_has_hypervisor cpu_opt(LOONGARCH_CPU_HYPERVISOR)
+
+
+#endif /* __ASM_CPU_FEATURES_H */
diff --git a/arch/loongarch/include/asm/cpu-info.h b/arch/loongarch/include/asm/cpu-info.h
new file mode 100644
index 000000000000..b6c4f96079df
--- /dev/null
+++ b/arch/loongarch/include/asm/cpu-info.h
@@ -0,0 +1,116 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_CPU_INFO_H
+#define __ASM_CPU_INFO_H
+
+#include <linux/cache.h>
+#include <linux/types.h>
+
+#include <asm/loongarch.h>
+
+/*
+ * Descriptor for a cache
+ */
+struct cache_desc {
+ unsigned int waysize; /* Bytes per way */
+ unsigned short sets; /* Number of lines per set */
+ unsigned char ways; /* Number of ways */
+ unsigned char linesz; /* Size of line in bytes */
+ unsigned char waybit; /* Bits to select in a cache set */
+ unsigned char flags; /* Flags describing cache properties */
+};
+
+struct cpuinfo_loongarch {
+ u64 asid_cache;
+ unsigned long asid_mask;
+
+ /*
+ * Capability and feature descriptor structure for LoongArch CPU
+ */
+ unsigned long long options;
+ unsigned int processor_id;
+ unsigned int fpu_vers;
+ unsigned int fpu_csr0;
+ unsigned int fpu_mask;
+ unsigned int cputype;
+ int isa_level;
+ int tlbsize;
+ int tlbsizemtlb;
+ int tlbsizestlbsets;
+ int tlbsizestlbways;
+ struct cache_desc icache; /* Primary I-cache */
+ struct cache_desc dcache; /* Primary D or combined I/D cache */
+ struct cache_desc vcache; /* Victim cache, between pcache and scache */
+ struct cache_desc scache; /* Secondary cache */
+ struct cache_desc tcache; /* Tertiary/split secondary cache */
+ int core; /* physical core number in package */
+ int package;/* physical package number */
+ int vabits; /* Virtual Address size in bits */
+ int pabits; /* Physical Address size in bits */
+ unsigned int ksave_mask; /* Usable KSave mask. */
+ unsigned int watch_dreg_count; /* Number data breakpoints */
+ unsigned int watch_ireg_count; /* Number instruction breakpoints */
+ unsigned int watch_reg_use_cnt; /* min(NUM_WATCH_REGS, watch_dreg_count + watch_ireg_count), Usable by ptrace */
+} __aligned(SMP_CACHE_BYTES);
+
+extern struct cpuinfo_loongarch cpu_data[];
+#define boot_cpu_data cpu_data[0]
+#define current_cpu_data cpu_data[smp_processor_id()]
+#define raw_current_cpu_data cpu_data[raw_smp_processor_id()]
+
+extern void cpu_probe(void);
+
+extern const char *__cpu_family[];
+extern const char *__cpu_full_name[];
+#define cpu_family_string() __cpu_family[raw_smp_processor_id()]
+#define cpu_full_name_string() __cpu_full_name[raw_smp_processor_id()]
+
+struct seq_file;
+struct notifier_block;
+
+extern int register_proc_cpuinfo_notifier(struct notifier_block *nb);
+extern int proc_cpuinfo_notifier_call_chain(unsigned long val, void *v);
+
+#define proc_cpuinfo_notifier(fn, pri) \
+({ \
+ static struct notifier_block fn##_nb = { \
+ .notifier_call = fn, \
+ .priority = pri \
+ }; \
+ \
+ register_proc_cpuinfo_notifier(&fn##_nb); \
+})
+
+struct proc_cpuinfo_notifier_args {
+ struct seq_file *m;
+ unsigned long n;
+};
+
+static inline bool cpus_are_siblings(int cpua, int cpub)
+{
+ struct cpuinfo_loongarch *infoa = &cpu_data[cpua];
+ struct cpuinfo_loongarch *infob = &cpu_data[cpub];
+
+ if (infoa->package != infob->package)
+ return false;
+
+ if (infoa->core != infob->core)
+ return false;
+
+ return true;
+}
+
+static inline unsigned long cpu_asid_mask(struct cpuinfo_loongarch *cpuinfo)
+{
+ return cpuinfo->asid_mask;
+}
+
+static inline void set_cpu_asid_mask(struct cpuinfo_loongarch *cpuinfo,
+ unsigned long asid_mask)
+{
+ cpuinfo->asid_mask = asid_mask;
+}
+
+#endif /* __ASM_CPU_INFO_H */
diff --git a/arch/loongarch/include/asm/cpu.h b/arch/loongarch/include/asm/cpu.h
new file mode 100644
index 000000000000..754f28506791
--- /dev/null
+++ b/arch/loongarch/include/asm/cpu.h
@@ -0,0 +1,127 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * cpu.h: Values of the PRID register used to match up
+ * various LoongArch CPU types.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_CPU_H
+#define _ASM_CPU_H
+
+/*
+ * As described in LoongArch specs from Loongson Technology, the PRID register
+ * (CPUCFG.00) has the following layout:
+ *
+ * +---------------+----------------+------------+--------------------+
+ * | Reserved | Company ID | Series ID | Product ID |
+ * +---------------+----------------+------------+--------------------+
+ * 31 24 23 16 15 12 11 0
+ */
+
+/*
+ * Assigned Company values for bits 23:16 of the PRID register.
+ */
+
+#define PRID_COMP_MASK 0xff0000
+
+#define PRID_COMP_LOONGSON 0x140000
+
+/*
+ * Assigned Series ID values for bits 15:12 of the PRID register. In order
+ * to detect a certain CPU type exactly eventually additional registers may
+ * need to be examined.
+ */
+
+#define PRID_SERIES_MASK 0xf000
+
+#define PRID_SERIES_LA132 0x8000 /* Loongson 32bit */
+#define PRID_SERIES_LA264 0xa000 /* Loongson 64bit, 2-issue */
+#define PRID_SERIES_LA364 0xb000 /* Loongson 64bit,3-issue */
+#define PRID_SERIES_LA464 0xc000 /* Loongson 64bit, 4-issue */
+#define PRID_SERIES_LA664 0xd000 /* Loongson 64bit, 6-issue */
+
+/*
+ * Particular Product ID values for bits 11:0 of the PRID register.
+ */
+
+#define PRID_PRODUCT_MASK 0x0fff
+
+#if !defined(__ASSEMBLY__)
+
+enum cpu_type_enum {
+ CPU_UNKNOWN,
+ CPU_LOONGSON32,
+ CPU_LOONGSON64,
+ CPU_LAST
+};
+
+#endif /* !__ASSEMBLY */
+
+/*
+ * ISA Level encodings
+ *
+ */
+
+#define LOONGARCH_CPU_ISA_LA32R 0x00000001
+#define LOONGARCH_CPU_ISA_LA32S 0x00000002
+#define LOONGARCH_CPU_ISA_LA64 0x00000004
+
+#define LOONGARCH_CPU_ISA_32BIT (LOONGARCH_CPU_ISA_LA32R | LOONGARCH_CPU_ISA_LA32S)
+#define LOONGARCH_CPU_ISA_64BIT LOONGARCH_CPU_ISA_LA64
+
+/*
+ * CPU Option encodings
+ */
+#define CPU_FEATURE_CPUCFG 0 /* CPU has CPUCFG */
+#define CPU_FEATURE_LAM 1 /* CPU has Atomic instructions */
+#define CPU_FEATURE_UAL 2 /* CPU supports unaligned access */
+#define CPU_FEATURE_FPU 3 /* CPU has FPU */
+#define CPU_FEATURE_LSX 4 /* CPU has LSX (128-bit SIMD) */
+#define CPU_FEATURE_LASX 5 /* CPU has LASX (256-bit SIMD) */
+#define CPU_FEATURE_COMPLEX 6 /* CPU has Complex instructions */
+#define CPU_FEATURE_CRYPTO 7 /* CPU has Crypto instructions */
+#define CPU_FEATURE_LVZ 8 /* CPU has Virtualization extension */
+#define CPU_FEATURE_LBT_X86 9 /* CPU has X86 Binary Translation */
+#define CPU_FEATURE_LBT_ARM 10 /* CPU has ARM Binary Translation */
+#define CPU_FEATURE_LBT_MIPS 11 /* CPU has MIPS Binary Translation */
+#define CPU_FEATURE_TLB 12 /* CPU has TLB */
+#define CPU_FEATURE_CSR 13 /* CPU has CSR */
+#define CPU_FEATURE_WATCH 14 /* CPU has watchpoint registers */
+#define CPU_FEATURE_VINT 15 /* CPU has vectored interrupts */
+#define CPU_FEATURE_CSRIPI 16 /* CPU has CSR-IPI */
+#define CPU_FEATURE_EXTIOI 17 /* CPU has EXT-IOI */
+#define CPU_FEATURE_PREFETCH 18 /* CPU has prefetch instructions */
+#define CPU_FEATURE_PMP 19 /* CPU has perfermance counter */
+#define CPU_FEATURE_SCALEFREQ 20 /* CPU supports cpufreq scaling */
+#define CPU_FEATURE_FLATMODE 21 /* CPU has flat mode */
+#define CPU_FEATURE_EIODECODE 22 /* CPU has EXTIOI interrupt pin decode mode */
+#define CPU_FEATURE_GUESTID 23 /* CPU has GuestID feature */
+#define CPU_FEATURE_HYPERVISOR 24 /* CPU has hypervisor (running in VM) */
+
+#define LOONGARCH_CPU_CPUCFG BIT_ULL(CPU_FEATURE_CPUCFG)
+#define LOONGARCH_CPU_LAM BIT_ULL(CPU_FEATURE_LAM)
+#define LOONGARCH_CPU_UAL BIT_ULL(CPU_FEATURE_UAL)
+#define LOONGARCH_CPU_FPU BIT_ULL(CPU_FEATURE_FPU)
+#define LOONGARCH_CPU_LSX BIT_ULL(CPU_FEATURE_LSX)
+#define LOONGARCH_CPU_LASX BIT_ULL(CPU_FEATURE_LASX)
+#define LOONGARCH_CPU_COMPLEX BIT_ULL(CPU_FEATURE_COMPLEX)
+#define LOONGARCH_CPU_CRYPTO BIT_ULL(CPU_FEATURE_CRYPTO)
+#define LOONGARCH_CPU_LVZ BIT_ULL(CPU_FEATURE_LVZ)
+#define LOONGARCH_CPU_LBT_X86 BIT_ULL(CPU_FEATURE_LBT_X86)
+#define LOONGARCH_CPU_LBT_ARM BIT_ULL(CPU_FEATURE_LBT_ARM)
+#define LOONGARCH_CPU_LBT_MIPS BIT_ULL(CPU_FEATURE_LBT_MIPS)
+#define LOONGARCH_CPU_TLB BIT_ULL(CPU_FEATURE_TLB)
+#define LOONGARCH_CPU_CSR BIT_ULL(CPU_FEATURE_CSR)
+#define LOONGARCH_CPU_WATCH BIT_ULL(CPU_FEATURE_WATCH)
+#define LOONGARCH_CPU_VINT BIT_ULL(CPU_FEATURE_VINT)
+#define LOONGARCH_CPU_CSRIPI BIT_ULL(CPU_FEATURE_CSRIPI)
+#define LOONGARCH_CPU_EXTIOI BIT_ULL(CPU_FEATURE_EXTIOI)
+#define LOONGARCH_CPU_PREFETCH BIT_ULL(CPU_FEATURE_PREFETCH)
+#define LOONGARCH_CPU_PMP BIT_ULL(CPU_FEATURE_PMP)
+#define LOONGARCH_CPU_SCALEFREQ BIT_ULL(CPU_FEATURE_SCALEFREQ)
+#define LOONGARCH_CPU_FLATMODE BIT_ULL(CPU_FEATURE_FLATMODE)
+#define LOONGARCH_CPU_EIODECODE BIT_ULL(CPU_FEATURE_EIODECODE)
+#define LOONGARCH_CPU_GUESTID BIT_ULL(CPU_FEATURE_GUESTID)
+#define LOONGARCH_CPU_HYPERVISOR BIT_ULL(CPU_FEATURE_HYPERVISOR)
+
+#endif /* _ASM_CPU_H */
diff --git a/arch/loongarch/include/asm/fpregdef.h b/arch/loongarch/include/asm/fpregdef.h
new file mode 100644
index 000000000000..adb16e4b43b0
--- /dev/null
+++ b/arch/loongarch/include/asm/fpregdef.h
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Definitions for the FPU register names
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_FPREGDEF_H
+#define _ASM_FPREGDEF_H
+
+#define fa0 $f0 /* argument registers, fa0/fa1 reused as fv0/fv1 for return value */
+#define fa1 $f1
+#define fa2 $f2
+#define fa3 $f3
+#define fa4 $f4
+#define fa5 $f5
+#define fa6 $f6
+#define fa7 $f7
+#define ft0 $f8 /* caller saved */
+#define ft1 $f9
+#define ft2 $f10
+#define ft3 $f11
+#define ft4 $f12
+#define ft5 $f13
+#define ft6 $f14
+#define ft7 $f15
+#define ft8 $f16
+#define ft9 $f17
+#define ft10 $f18
+#define ft11 $f19
+#define ft12 $f20
+#define ft13 $f21
+#define ft14 $f22
+#define ft15 $f23
+#define fs0 $f24 /* callee saved */
+#define fs1 $f25
+#define fs2 $f26
+#define fs3 $f27
+#define fs4 $f28
+#define fs5 $f29
+#define fs6 $f30
+#define fs7 $f31
+
+/*
+ * Current binutils expects *GPRs* at FCSR position for the FCSR
+ * operation instructions, so define aliases for those used.
+ */
+#define fcsr0 $r0
+#define fcsr1 $r1
+#define fcsr2 $r2
+#define fcsr3 $r3
+#define vcsr16 $r16
+
+#endif /* _ASM_FPREGDEF_H */
diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h
new file mode 100644
index 000000000000..2be0efe93875
--- /dev/null
+++ b/arch/loongarch/include/asm/loongarch.h
@@ -0,0 +1,1519 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_LOONGARCH_H
+#define _ASM_LOONGARCH_H
+
+#include <linux/bits.h>
+#include <linux/linkage.h>
+#include <linux/types.h>
+
+#ifndef __ASSEMBLY__
+#include <larchintrin.h>
+
+/*
+ * parse_r var, r - Helper assembler macro for parsing register names.
+ *
+ * This converts the register name in $n form provided in \r to the
+ * corresponding register number, which is assigned to the variable \var. It is
+ * needed to allow explicit encoding of instructions in inline assembly where
+ * registers are chosen by the compiler in $n form, allowing us to avoid using
+ * fixed register numbers.
+ *
+ * It also allows newer instructions (not implemented by the assembler) to be
+ * transparently implemented using assembler macros, instead of needing separate
+ * cases depending on toolchain support.
+ *
+ * Simple usage example:
+ * __asm__ __volatile__("parse_r addr, %0\n\t"
+ * "#invtlb op, 0, %0\n\t"
+ * ".word ((0x6498000) | (addr << 10) | (0 << 5) | op)"
+ * : "=r" (status);
+ */
+
+/* Match an individual register number and assign to \var */
+#define _IFC_REG(n) \
+ ".ifc \\r, $r" #n "\n\t" \
+ "\\var = " #n "\n\t" \
+ ".endif\n\t"
+
+__asm__(".macro parse_r var r\n\t"
+ "\\var = -1\n\t"
+ _IFC_REG(0) _IFC_REG(1) _IFC_REG(2) _IFC_REG(3)
+ _IFC_REG(4) _IFC_REG(5) _IFC_REG(6) _IFC_REG(7)
+ _IFC_REG(8) _IFC_REG(9) _IFC_REG(10) _IFC_REG(11)
+ _IFC_REG(12) _IFC_REG(13) _IFC_REG(14) _IFC_REG(15)
+ _IFC_REG(16) _IFC_REG(17) _IFC_REG(18) _IFC_REG(19)
+ _IFC_REG(20) _IFC_REG(21) _IFC_REG(22) _IFC_REG(23)
+ _IFC_REG(24) _IFC_REG(25) _IFC_REG(26) _IFC_REG(27)
+ _IFC_REG(28) _IFC_REG(29) _IFC_REG(30) _IFC_REG(31)
+ ".iflt \\var\n\t"
+ ".error \"Unable to parse register name \\r\"\n\t"
+ ".endif\n\t"
+ ".endm");
+
+#undef _IFC_REG
+
+/* CPUCFG */
+static inline u32 read_cpucfg(u32 reg)
+{
+ return __cpucfg(reg);
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#ifdef __ASSEMBLY__
+
+/* LoongArch Registers */
+#define REG_ZERO 0x0
+#define REG_RA 0x1
+#define REG_TP 0x2
+#define REG_SP 0x3
+#define REG_A0 0x4 /* Reused as V0 for return value */
+#define REG_A1 0x5 /* Reused as V1 for return value */
+#define REG_A2 0x6
+#define REG_A3 0x7
+#define REG_A4 0x8
+#define REG_A5 0x9
+#define REG_A6 0xa
+#define REG_A7 0xb
+#define REG_T0 0xc
+#define REG_T1 0xd
+#define REG_T2 0xe
+#define REG_T3 0xf
+#define REG_T4 0x10
+#define REG_T5 0x11
+#define REG_T6 0x12
+#define REG_T7 0x13
+#define REG_T8 0x14
+#define REG_U0 0x15 /* Kernel uses it as percpu base */
+#define REG_FP 0x16
+#define REG_S0 0x17
+#define REG_S1 0x18
+#define REG_S2 0x19
+#define REG_S3 0x1a
+#define REG_S4 0x1b
+#define REG_S5 0x1c
+#define REG_S6 0x1d
+#define REG_S7 0x1e
+#define REG_S8 0x1f
+
+#endif /* __ASSEMBLY__ */
+
+/* Bit fields for CPUCFG registers */
+#define LOONGARCH_CPUCFG0 0x0
+#define CPUCFG0_PRID GENMASK(31, 0)
+
+#define LOONGARCH_CPUCFG1 0x1
+#define CPUCFG1_ISGR32 BIT(0)
+#define CPUCFG1_ISGR64 BIT(1)
+#define CPUCFG1_PAGING BIT(2)
+#define CPUCFG1_IOCSR BIT(3)
+#define CPUCFG1_PABITS GENMASK(11, 4)
+#define CPUCFG1_VABITS GENMASK(19, 12)
+#define CPUCFG1_UAL BIT(20)
+#define CPUCFG1_RI BIT(21)
+#define CPUCFG1_EP BIT(22)
+#define CPUCFG1_RPLV BIT(23)
+#define CPUCFG1_HUGEPG BIT(24)
+#define CPUCFG1_IOCSRBRD BIT(25)
+#define CPUCFG1_MSGINT BIT(26)
+
+#define LOONGARCH_CPUCFG2 0x2
+#define CPUCFG2_FP BIT(0)
+#define CPUCFG2_FPSP BIT(1)
+#define CPUCFG2_FPDP BIT(2)
+#define CPUCFG2_FPVERS GENMASK(5, 3)
+#define CPUCFG2_LSX BIT(6)
+#define CPUCFG2_LASX BIT(7)
+#define CPUCFG2_COMPLEX BIT(8)
+#define CPUCFG2_CRYPTO BIT(9)
+#define CPUCFG2_LVZP BIT(10)
+#define CPUCFG2_LVZVER GENMASK(13, 11)
+#define CPUCFG2_LLFTP BIT(14)
+#define CPUCFG2_LLFTPREV GENMASK(17, 15)
+#define CPUCFG2_X86BT BIT(18)
+#define CPUCFG2_ARMBT BIT(19)
+#define CPUCFG2_MIPSBT BIT(20)
+#define CPUCFG2_LSPW BIT(21)
+#define CPUCFG2_LAM BIT(22)
+
+#define LOONGARCH_CPUCFG3 0x3
+#define CPUCFG3_CCDMA BIT(0)
+#define CPUCFG3_SFB BIT(1)
+#define CPUCFG3_UCACC BIT(2)
+#define CPUCFG3_LLEXC BIT(3)
+#define CPUCFG3_SCDLY BIT(4)
+#define CPUCFG3_LLDBAR BIT(5)
+#define CPUCFG3_ITLBT BIT(6)
+#define CPUCFG3_ICACHET BIT(7)
+#define CPUCFG3_SPW_LVL GENMASK(10, 8)
+#define CPUCFG3_SPW_HG_HF BIT(11)
+#define CPUCFG3_RVA BIT(12)
+#define CPUCFG3_RVAMAX GENMASK(16, 13)
+
+#define LOONGARCH_CPUCFG4 0x4
+#define CPUCFG4_CCFREQ GENMASK(31, 0)
+
+#define LOONGARCH_CPUCFG5 0x5
+#define CPUCFG5_CCMUL GENMASK(15, 0)
+#define CPUCFG5_CCDIV GENMASK(31, 16)
+
+#define LOONGARCH_CPUCFG6 0x6
+#define CPUCFG6_PMP BIT(0)
+#define CPUCFG6_PAMVER GENMASK(3, 1)
+#define CPUCFG6_PMNUM GENMASK(7, 4)
+#define CPUCFG6_PMBITS GENMASK(13, 8)
+#define CPUCFG6_UPM BIT(14)
+
+#define LOONGARCH_CPUCFG16 0x10
+#define CPUCFG16_L1_IUPRE BIT(0)
+#define CPUCFG16_L1_IUUNIFY BIT(1)
+#define CPUCFG16_L1_DPRE BIT(2)
+#define CPUCFG16_L2_IUPRE BIT(3)
+#define CPUCFG16_L2_IUUNIFY BIT(4)
+#define CPUCFG16_L2_IUPRIV BIT(5)
+#define CPUCFG16_L2_IUINCL BIT(6)
+#define CPUCFG16_L2_DPRE BIT(7)
+#define CPUCFG16_L2_DPRIV BIT(8)
+#define CPUCFG16_L2_DINCL BIT(9)
+#define CPUCFG16_L3_IUPRE BIT(10)
+#define CPUCFG16_L3_IUUNIFY BIT(11)
+#define CPUCFG16_L3_IUPRIV BIT(12)
+#define CPUCFG16_L3_IUINCL BIT(13)
+#define CPUCFG16_L3_DPRE BIT(14)
+#define CPUCFG16_L3_DPRIV BIT(15)
+#define CPUCFG16_L3_DINCL BIT(16)
+
+#define LOONGARCH_CPUCFG17 0x11
+#define CPUCFG17_L1I_WAYS_M GENMASK(15, 0)
+#define CPUCFG17_L1I_SETS_M GENMASK(23, 16)
+#define CPUCFG17_L1I_SIZE_M GENMASK(30, 24)
+#define CPUCFG17_L1I_WAYS 0
+#define CPUCFG17_L1I_SETS 16
+#define CPUCFG17_L1I_SIZE 24
+
+#define LOONGARCH_CPUCFG18 0x12
+#define CPUCFG18_L1D_WAYS_M GENMASK(15, 0)
+#define CPUCFG18_L1D_SETS_M GENMASK(23, 16)
+#define CPUCFG18_L1D_SIZE_M GENMASK(30, 24)
+#define CPUCFG18_L1D_WAYS 0
+#define CPUCFG18_L1D_SETS 16
+#define CPUCFG18_L1D_SIZE 24
+
+#define LOONGARCH_CPUCFG19 0x13
+#define CPUCFG19_L2_WAYS_M GENMASK(15, 0)
+#define CPUCFG19_L2_SETS_M GENMASK(23, 16)
+#define CPUCFG19_L2_SIZE_M GENMASK(30, 24)
+#define CPUCFG19_L2_WAYS 0
+#define CPUCFG19_L2_SETS 16
+#define CPUCFG19_L2_SIZE 24
+
+#define LOONGARCH_CPUCFG20 0x14
+#define CPUCFG20_L3_WAYS_M GENMASK(15, 0)
+#define CPUCFG20_L3_SETS_M GENMASK(23, 16)
+#define CPUCFG20_L3_SIZE_M GENMASK(30, 24)
+#define CPUCFG20_L3_WAYS 0
+#define CPUCFG20_L3_SETS 16
+#define CPUCFG20_L3_SIZE 24
+
+#define LOONGARCH_CPUCFG48 0x30
+#define CPUCFG48_MCSR_LCK BIT(0)
+#define CPUCFG48_NAP_EN BIT(1)
+#define CPUCFG48_VFPU_CG BIT(2)
+#define CPUCFG48_RAM_CG BIT(3)
+
+#ifndef __ASSEMBLY__
+
+/* CSR */
+static __always_inline u32 csr_readl(u32 reg)
+{
+ return __csrrd_w(reg);
+}
+
+static __always_inline u64 csr_readq(u32 reg)
+{
+ return __csrrd_d(reg);
+}
+
+static __always_inline void csr_writel(u32 val, u32 reg)
+{
+ __csrwr_w(val, reg);
+}
+
+static __always_inline void csr_writeq(u64 val, u32 reg)
+{
+ __csrwr_d(val, reg);
+}
+
+static __always_inline u32 csr_xchgl(u32 val, u32 mask, u32 reg)
+{
+ return __csrxchg_w(val, mask, reg);
+}
+
+static __always_inline u64 csr_xchgq(u64 val, u64 mask, u32 reg)
+{
+ return __csrxchg_d(val, mask, reg);
+}
+
+/* IOCSR */
+static __always_inline u32 iocsr_readl(u32 reg)
+{
+ return __iocsrrd_w(reg);
+}
+
+static __always_inline u64 iocsr_readq(u32 reg)
+{
+ return __iocsrrd_d(reg);
+}
+
+static __always_inline void iocsr_writel(u32 val, u32 reg)
+{
+ __iocsrwr_w(val, reg);
+}
+
+static __always_inline void iocsr_writeq(u64 val, u32 reg)
+{
+ __iocsrwr_d(val, reg);
+}
+
+#endif /* !__ASSEMBLY__ */
+
+/* CSR register number */
+
+/* Basic CSR registers */
+#define LOONGARCH_CSR_CRMD 0x0 /* Current mode info */
+#define CSR_CRMD_WE_SHIFT 9
+#define CSR_CRMD_WE (_ULCAST_(0x1) << CSR_CRMD_WE_SHIFT)
+#define CSR_CRMD_DACM_SHIFT 7
+#define CSR_CRMD_DACM_WIDTH 2
+#define CSR_CRMD_DACM (_ULCAST_(0x3) << CSR_CRMD_DACM_SHIFT)
+#define CSR_CRMD_DACF_SHIFT 5
+#define CSR_CRMD_DACF_WIDTH 2
+#define CSR_CRMD_DACF (_ULCAST_(0x3) << CSR_CRMD_DACF_SHIFT)
+#define CSR_CRMD_PG_SHIFT 4
+#define CSR_CRMD_PG (_ULCAST_(0x1) << CSR_CRMD_PG_SHIFT)
+#define CSR_CRMD_DA_SHIFT 3
+#define CSR_CRMD_DA (_ULCAST_(0x1) << CSR_CRMD_DA_SHIFT)
+#define CSR_CRMD_IE_SHIFT 2
+#define CSR_CRMD_IE (_ULCAST_(0x1) << CSR_CRMD_IE_SHIFT)
+#define CSR_CRMD_PLV_SHIFT 0
+#define CSR_CRMD_PLV_WIDTH 2
+#define CSR_CRMD_PLV (_ULCAST_(0x3) << CSR_CRMD_PLV_SHIFT)
+
+#define PLV_KERN 0
+#define PLV_USER 3
+#define PLV_MASK 0x3
+
+#define LOONGARCH_CSR_PRMD 0x1 /* Prev-exception mode info */
+#define CSR_PRMD_PWE_SHIFT 3
+#define CSR_PRMD_PWE (_ULCAST_(0x1) << CSR_PRMD_PWE_SHIFT)
+#define CSR_PRMD_PIE_SHIFT 2
+#define CSR_PRMD_PIE (_ULCAST_(0x1) << CSR_PRMD_PIE_SHIFT)
+#define CSR_PRMD_PPLV_SHIFT 0
+#define CSR_PRMD_PPLV_WIDTH 2
+#define CSR_PRMD_PPLV (_ULCAST_(0x3) << CSR_PRMD_PPLV_SHIFT)
+
+#define LOONGARCH_CSR_EUEN 0x2 /* Extended unit enable */
+#define CSR_EUEN_LBTEN_SHIFT 3
+#define CSR_EUEN_LBTEN (_ULCAST_(0x1) << CSR_EUEN_LBTEN_SHIFT)
+#define CSR_EUEN_LASXEN_SHIFT 2
+#define CSR_EUEN_LASXEN (_ULCAST_(0x1) << CSR_EUEN_LASXEN_SHIFT)
+#define CSR_EUEN_LSXEN_SHIFT 1
+#define CSR_EUEN_LSXEN (_ULCAST_(0x1) << CSR_EUEN_LSXEN_SHIFT)
+#define CSR_EUEN_FPEN_SHIFT 0
+#define CSR_EUEN_FPEN (_ULCAST_(0x1) << CSR_EUEN_FPEN_SHIFT)
+
+#define LOONGARCH_CSR_MISC 0x3 /* Misc config */
+
+#define LOONGARCH_CSR_ECFG 0x4 /* Exception config */
+#define CSR_ECFG_VS_SHIFT 16
+#define CSR_ECFG_VS_WIDTH 3
+#define CSR_ECFG_VS (_ULCAST_(0x7) << CSR_ECFG_VS_SHIFT)
+#define CSR_ECFG_IM_SHIFT 0
+#define CSR_ECFG_IM_WIDTH 13
+#define CSR_ECFG_IM (_ULCAST_(0x1fff) << CSR_ECFG_IM_SHIFT)
+
+#define LOONGARCH_CSR_ESTAT 0x5 /* Exception status */
+#define CSR_ESTAT_ESUBCODE_SHIFT 22
+#define CSR_ESTAT_ESUBCODE_WIDTH 9
+#define CSR_ESTAT_ESUBCODE (_ULCAST_(0x1ff) << CSR_ESTAT_ESUBCODE_SHIFT)
+#define CSR_ESTAT_EXC_SHIFT 16
+#define CSR_ESTAT_EXC_WIDTH 6
+#define CSR_ESTAT_EXC (_ULCAST_(0x3f) << CSR_ESTAT_EXC_SHIFT)
+#define CSR_ESTAT_IS_SHIFT 0
+#define CSR_ESTAT_IS_WIDTH 15
+#define CSR_ESTAT_IS (_ULCAST_(0x7fff) << CSR_ESTAT_IS_SHIFT)
+
+#define LOONGARCH_CSR_ERA 0x6 /* ERA */
+
+#define LOONGARCH_CSR_BADV 0x7 /* Bad virtual address */
+
+#define LOONGARCH_CSR_BADI 0x8 /* Bad instruction */
+
+#define LOONGARCH_CSR_EENTRY 0xc /* Exception entry */
+
+/* TLB related CSR registers */
+#define LOONGARCH_CSR_TLBIDX 0x10 /* TLB Index, EHINV, PageSize, NP */
+#define CSR_TLBIDX_EHINV_SHIFT 31
+#define CSR_TLBIDX_EHINV (_ULCAST_(1) << CSR_TLBIDX_EHINV_SHIFT)
+#define CSR_TLBIDX_PS_SHIFT 24
+#define CSR_TLBIDX_PS_WIDTH 6
+#define CSR_TLBIDX_PS (_ULCAST_(0x3f) << CSR_TLBIDX_PS_SHIFT)
+#define CSR_TLBIDX_IDX_SHIFT 0
+#define CSR_TLBIDX_IDX_WIDTH 12
+#define CSR_TLBIDX_IDX (_ULCAST_(0xfff) << CSR_TLBIDX_IDX_SHIFT)
+#define CSR_TLBIDX_SIZEM 0x3f000000
+#define CSR_TLBIDX_SIZE CSR_TLBIDX_PS_SHIFT
+#define CSR_TLBIDX_IDXM 0xfff
+#define CSR_INVALID_ENTRY(e) (CSR_TLBIDX_EHINV | e)
+
+#define LOONGARCH_CSR_TLBEHI 0x11 /* TLB EntryHi */
+
+#define LOONGARCH_CSR_TLBELO0 0x12 /* TLB EntryLo0 */
+#define CSR_TLBLO0_RPLV_SHIFT 63
+#define CSR_TLBLO0_RPLV (_ULCAST_(0x1) << CSR_TLBLO0_RPLV_SHIFT)
+#define CSR_TLBLO0_NX_SHIFT 62
+#define CSR_TLBLO0_NX (_ULCAST_(0x1) << CSR_TLBLO0_NX_SHIFT)
+#define CSR_TLBLO0_NR_SHIFT 61
+#define CSR_TLBLO0_NR (_ULCAST_(0x1) << CSR_TLBLO0_NR_SHIFT)
+#define CSR_TLBLO0_PFN_SHIFT 12
+#define CSR_TLBLO0_PFN_WIDTH 36
+#define CSR_TLBLO0_PFN (_ULCAST_(0xfffffffff) << CSR_TLBLO0_PFN_SHIFT)
+#define CSR_TLBLO0_GLOBAL_SHIFT 6
+#define CSR_TLBLO0_GLOBAL (_ULCAST_(0x1) << CSR_TLBLO0_GLOBAL_SHIFT)
+#define CSR_TLBLO0_CCA_SHIFT 4
+#define CSR_TLBLO0_CCA_WIDTH 2
+#define CSR_TLBLO0_CCA (_ULCAST_(0x3) << CSR_TLBLO0_CCA_SHIFT)
+#define CSR_TLBLO0_PLV_SHIFT 2
+#define CSR_TLBLO0_PLV_WIDTH 2
+#define CSR_TLBLO0_PLV (_ULCAST_(0x3) << CSR_TLBLO0_PLV_SHIFT)
+#define CSR_TLBLO0_WE_SHIFT 1
+#define CSR_TLBLO0_WE (_ULCAST_(0x1) << CSR_TLBLO0_WE_SHIFT)
+#define CSR_TLBLO0_V_SHIFT 0
+#define CSR_TLBLO0_V (_ULCAST_(0x1) << CSR_TLBLO0_V_SHIFT)
+
+#define LOONGARCH_CSR_TLBELO1 0x13 /* TLB EntryLo1 */
+#define CSR_TLBLO1_RPLV_SHIFT 63
+#define CSR_TLBLO1_RPLV (_ULCAST_(0x1) << CSR_TLBLO1_RPLV_SHIFT)
+#define CSR_TLBLO1_NX_SHIFT 62
+#define CSR_TLBLO1_NX (_ULCAST_(0x1) << CSR_TLBLO1_NX_SHIFT)
+#define CSR_TLBLO1_NR_SHIFT 61
+#define CSR_TLBLO1_NR (_ULCAST_(0x1) << CSR_TLBLO1_NR_SHIFT)
+#define CSR_TLBLO1_PFN_SHIFT 12
+#define CSR_TLBLO1_PFN_WIDTH 36
+#define CSR_TLBLO1_PFN (_ULCAST_(0xfffffffff) << CSR_TLBLO1_PFN_SHIFT)
+#define CSR_TLBLO1_GLOBAL_SHIFT 6
+#define CSR_TLBLO1_GLOBAL (_ULCAST_(0x1) << CSR_TLBLO1_GLOBAL_SHIFT)
+#define CSR_TLBLO1_CCA_SHIFT 4
+#define CSR_TLBLO1_CCA_WIDTH 2
+#define CSR_TLBLO1_CCA (_ULCAST_(0x3) << CSR_TLBLO1_CCA_SHIFT)
+#define CSR_TLBLO1_PLV_SHIFT 2
+#define CSR_TLBLO1_PLV_WIDTH 2
+#define CSR_TLBLO1_PLV (_ULCAST_(0x3) << CSR_TLBLO1_PLV_SHIFT)
+#define CSR_TLBLO1_WE_SHIFT 1
+#define CSR_TLBLO1_WE (_ULCAST_(0x1) << CSR_TLBLO1_WE_SHIFT)
+#define CSR_TLBLO1_V_SHIFT 0
+#define CSR_TLBLO1_V (_ULCAST_(0x1) << CSR_TLBLO1_V_SHIFT)
+
+#define LOONGARCH_CSR_GTLBC 0x15 /* Guest TLB control */
+#define CSR_GTLBC_RID_SHIFT 16
+#define CSR_GTLBC_RID_WIDTH 8
+#define CSR_GTLBC_RID (_ULCAST_(0xff) << CSR_GTLBC_RID_SHIFT)
+#define CSR_GTLBC_TOTI_SHIFT 13
+#define CSR_GTLBC_TOTI (_ULCAST_(0x1) << CSR_GTLBC_TOTI_SHIFT)
+#define CSR_GTLBC_USERID_SHIFT 12
+#define CSR_GTLBC_USERID (_ULCAST_(0x1) << CSR_GTLBC_USERID_SHIFT)
+#define CSR_GTLBC_GMTLBSZ_SHIFT 0
+#define CSR_GTLBC_GMTLBSZ_WIDTH 6
+#define CSR_GTLBC_GMTLBSZ (_ULCAST_(0x3f) << CSR_GTLBC_GMTLBSZ_SHIFT)
+
+#define LOONGARCH_CSR_TRGP 0x16 /* TLBR read guest info */
+#define CSR_TRGP_RID_SHIFT 16
+#define CSR_TRGP_RID_WIDTH 8
+#define CSR_TRGP_RID (_ULCAST_(0xff) << CSR_TRGP_RID_SHIFT)
+#define CSR_TRGP_GTLB_SHIFT 0
+#define CSR_TRGP_GTLB (1 << CSR_TRGP_GTLB_SHIFT)
+
+#define LOONGARCH_CSR_ASID 0x18 /* ASID */
+#define CSR_ASID_BIT_SHIFT 16 /* ASIDBits */
+#define CSR_ASID_BIT_WIDTH 8
+#define CSR_ASID_BIT (_ULCAST_(0xff) << CSR_ASID_BIT_SHIFT)
+#define CSR_ASID_ASID_SHIFT 0
+#define CSR_ASID_ASID_WIDTH 10
+#define CSR_ASID_ASID (_ULCAST_(0x3ff) << CSR_ASID_ASID_SHIFT)
+
+#define LOONGARCH_CSR_PGDL 0x19 /* Page table base address when VA[47] = 0 */
+
+#define LOONGARCH_CSR_PGDH 0x1a /* Page table base address when VA[47] = 1 */
+
+#define LOONGARCH_CSR_PGD 0x1b /* Page table base */
+
+#define LOONGARCH_CSR_PWCTL0 0x1c /* PWCtl0 */
+#define CSR_PWCTL0_PTEW_SHIFT 30
+#define CSR_PWCTL0_PTEW_WIDTH 2
+#define CSR_PWCTL0_PTEW (_ULCAST_(0x3) << CSR_PWCTL0_PTEW_SHIFT)
+#define CSR_PWCTL0_DIR1WIDTH_SHIFT 25
+#define CSR_PWCTL0_DIR1WIDTH_WIDTH 5
+#define CSR_PWCTL0_DIR1WIDTH (_ULCAST_(0x1f) << CSR_PWCTL0_DIR1WIDTH_SHIFT)
+#define CSR_PWCTL0_DIR1BASE_SHIFT 20
+#define CSR_PWCTL0_DIR1BASE_WIDTH 5
+#define CSR_PWCTL0_DIR1BASE (_ULCAST_(0x1f) << CSR_PWCTL0_DIR1BASE_SHIFT)
+#define CSR_PWCTL0_DIR0WIDTH_SHIFT 15
+#define CSR_PWCTL0_DIR0WIDTH_WIDTH 5
+#define CSR_PWCTL0_DIR0WIDTH (_ULCAST_(0x1f) << CSR_PWCTL0_DIR0WIDTH_SHIFT)
+#define CSR_PWCTL0_DIR0BASE_SHIFT 10
+#define CSR_PWCTL0_DIR0BASE_WIDTH 5
+#define CSR_PWCTL0_DIR0BASE (_ULCAST_(0x1f) << CSR_PWCTL0_DIR0BASE_SHIFT)
+#define CSR_PWCTL0_PTWIDTH_SHIFT 5
+#define CSR_PWCTL0_PTWIDTH_WIDTH 5
+#define CSR_PWCTL0_PTWIDTH (_ULCAST_(0x1f) << CSR_PWCTL0_PTWIDTH_SHIFT)
+#define CSR_PWCTL0_PTBASE_SHIFT 0
+#define CSR_PWCTL0_PTBASE_WIDTH 5
+#define CSR_PWCTL0_PTBASE (_ULCAST_(0x1f) << CSR_PWCTL0_PTBASE_SHIFT)
+
+#define LOONGARCH_CSR_PWCTL1 0x1d /* PWCtl1 */
+#define CSR_PWCTL1_DIR3WIDTH_SHIFT 18
+#define CSR_PWCTL1_DIR3WIDTH_WIDTH 5
+#define CSR_PWCTL1_DIR3WIDTH (_ULCAST_(0x1f) << CSR_PWCTL1_DIR3WIDTH_SHIFT)
+#define CSR_PWCTL1_DIR3BASE_SHIFT 12
+#define CSR_PWCTL1_DIR3BASE_WIDTH 5
+#define CSR_PWCTL1_DIR3BASE (_ULCAST_(0x1f) << CSR_PWCTL0_DIR3BASE_SHIFT)
+#define CSR_PWCTL1_DIR2WIDTH_SHIFT 6
+#define CSR_PWCTL1_DIR2WIDTH_WIDTH 5
+#define CSR_PWCTL1_DIR2WIDTH (_ULCAST_(0x1f) << CSR_PWCTL1_DIR2WIDTH_SHIFT)
+#define CSR_PWCTL1_DIR2BASE_SHIFT 0
+#define CSR_PWCTL1_DIR2BASE_WIDTH 5
+#define CSR_PWCTL1_DIR2BASE (_ULCAST_(0x1f) << CSR_PWCTL0_DIR2BASE_SHIFT)
+
+#define LOONGARCH_CSR_STLBPGSIZE 0x1e
+#define CSR_STLBPGSIZE_PS_WIDTH 6
+#define CSR_STLBPGSIZE_PS (_ULCAST_(0x3f))
+
+#define LOONGARCH_CSR_RVACFG 0x1f
+#define CSR_RVACFG_RDVA_WIDTH 4
+#define CSR_RVACFG_RDVA (_ULCAST_(0xf))
+
+/* Config CSR registers */
+#define LOONGARCH_CSR_CPUID 0x20 /* CPU core id */
+#define CSR_CPUID_COREID_WIDTH 9
+#define CSR_CPUID_COREID _ULCAST_(0x1ff)
+
+#define LOONGARCH_CSR_PRCFG1 0x21 /* Config1 */
+#define CSR_CONF1_VSMAX_SHIFT 12
+#define CSR_CONF1_VSMAX_WIDTH 3
+#define CSR_CONF1_VSMAX (_ULCAST_(7) << CSR_CONF1_VSMAX_SHIFT)
+#define CSR_CONF1_TMRBITS_SHIFT 4
+#define CSR_CONF1_TMRBITS_WIDTH 8
+#define CSR_CONF1_TMRBITS (_ULCAST_(0xff) << CSR_CONF1_TMRBITS_SHIFT)
+#define CSR_CONF1_KSNUM_WIDTH 4
+#define CSR_CONF1_KSNUM _ULCAST_(0xf)
+
+#define LOONGARCH_CSR_PRCFG2 0x22 /* Config2 */
+#define CSR_CONF2_PGMASK_SUPP 0x3ffff000
+
+#define LOONGARCH_CSR_PRCFG3 0x23 /* Config3 */
+#define CSR_CONF3_STLBIDX_SHIFT 20
+#define CSR_CONF3_STLBIDX_WIDTH 6
+#define CSR_CONF3_STLBIDX (_ULCAST_(0x3f) << CSR_CONF3_STLBIDX_SHIFT)
+#define CSR_CONF3_STLBWAYS_SHIFT 12
+#define CSR_CONF3_STLBWAYS_WIDTH 8
+#define CSR_CONF3_STLBWAYS (_ULCAST_(0xff) << CSR_CONF3_STLBWAYS_SHIFT)
+#define CSR_CONF3_MTLBSIZE_SHIFT 4
+#define CSR_CONF3_MTLBSIZE_WIDTH 8
+#define CSR_CONF3_MTLBSIZE (_ULCAST_(0xff) << CSR_CONF3_MTLBSIZE_SHIFT)
+#define CSR_CONF3_TLBTYPE_SHIFT 0
+#define CSR_CONF3_TLBTYPE_WIDTH 4
+#define CSR_CONF3_TLBTYPE (_ULCAST_(0xf) << CSR_CONF3_TLBTYPE_SHIFT)
+
+/* KSave registers */
+#define LOONGARCH_CSR_KS0 0x30
+#define LOONGARCH_CSR_KS1 0x31
+#define LOONGARCH_CSR_KS2 0x32
+#define LOONGARCH_CSR_KS3 0x33
+#define LOONGARCH_CSR_KS4 0x34
+#define LOONGARCH_CSR_KS5 0x35
+#define LOONGARCH_CSR_KS6 0x36
+#define LOONGARCH_CSR_KS7 0x37
+#define LOONGARCH_CSR_KS8 0x38
+
+/* Exception allocated KS0, KS1 and KS2 statically */
+#define EXCEPTION_KS0 LOONGARCH_CSR_KS0
+#define EXCEPTION_KS1 LOONGARCH_CSR_KS1
+#define EXCEPTION_KS2 LOONGARCH_CSR_KS2
+#define EXC_KSAVE_MASK (1 << 0 | 1 << 1 | 1 << 2)
+
+/* Percpu-data base allocated KS3 statically */
+#define PERCPU_BASE_KS LOONGARCH_CSR_KS3
+#define PERCPU_KSAVE_MASK (1 << 3)
+
+/* KVM allocated KS4 and KS5 statically */
+#define KVM_VCPU_KS LOONGARCH_CSR_KS4
+#define KVM_TEMP_KS LOONGARCH_CSR_KS5
+#define KVM_KSAVE_MASK (1 << 4 | 1 << 5)
+
+/* Timer registers */
+#define LOONGARCH_CSR_TMID 0x40 /* Timer ID */
+
+#define LOONGARCH_CSR_TCFG 0x41 /* Timer config */
+#define CSR_TCFG_VAL_SHIFT 2
+#define CSR_TCFG_VAL_WIDTH 48
+#define CSR_TCFG_VAL (_ULCAST_(0x3fffffffffff) << CSR_TCFG_VAL_SHIFT)
+#define CSR_TCFG_PERIOD_SHIFT 1
+#define CSR_TCFG_PERIOD (_ULCAST_(0x1) << CSR_TCFG_PERIOD_SHIFT)
+#define CSR_TCFG_EN (_ULCAST_(0x1))
+
+#define LOONGARCH_CSR_TVAL 0x42 /* Timer value */
+
+#define LOONGARCH_CSR_CNTC 0x43 /* Timer offset */
+
+#define LOONGARCH_CSR_TINTCLR 0x44 /* Timer interrupt clear */
+#define CSR_TINTCLR_TI_SHIFT 0
+#define CSR_TINTCLR_TI (1 << CSR_TINTCLR_TI_SHIFT)
+
+/* Guest registers */
+#define LOONGARCH_CSR_GSTAT 0x50 /* Guest status */
+#define CSR_GSTAT_GID_SHIFT 16
+#define CSR_GSTAT_GID_WIDTH 8
+#define CSR_GSTAT_GID (_ULCAST_(0xff) << CSR_GSTAT_GID_SHIFT)
+#define CSR_GSTAT_GIDBIT_SHIFT 4
+#define CSR_GSTAT_GIDBIT_WIDTH 6
+#define CSR_GSTAT_GIDBIT (_ULCAST_(0x3f) << CSR_GSTAT_GIDBIT_SHIFT)
+#define CSR_GSTAT_PVM_SHIFT 1
+#define CSR_GSTAT_PVM (_ULCAST_(0x1) << CSR_GSTAT_PVM_SHIFT)
+#define CSR_GSTAT_VM_SHIFT 0
+#define CSR_GSTAT_VM (_ULCAST_(0x1) << CSR_GSTAT_VM_SHIFT)
+
+#define LOONGARCH_CSR_GCFG 0x51 /* Guest config */
+#define CSR_GCFG_GPERF_SHIFT 24
+#define CSR_GCFG_GPERF_WIDTH 3
+#define CSR_GCFG_GPERF (_ULCAST_(0x7) << CSR_GCFG_GPERF_SHIFT)
+#define CSR_GCFG_GCI_SHIFT 20
+#define CSR_GCFG_GCI_WIDTH 2
+#define CSR_GCFG_GCI (_ULCAST_(0x3) << CSR_GCFG_GCI_SHIFT)
+#define CSR_GCFG_GCI_ALL (_ULCAST_(0x0) << CSR_GCFG_GCI_SHIFT)
+#define CSR_GCFG_GCI_HIT (_ULCAST_(0x1) << CSR_GCFG_GCI_SHIFT)
+#define CSR_GCFG_GCI_SECURE (_ULCAST_(0x2) << CSR_GCFG_GCI_SHIFT)
+#define CSR_GCFG_GCIP_SHIFT 16
+#define CSR_GCFG_GCIP (_ULCAST_(0xf) << CSR_GCFG_GCIP_SHIFT)
+#define CSR_GCFG_GCIP_ALL (_ULCAST_(0x1) << CSR_GCFG_GCIP_SHIFT)
+#define CSR_GCFG_GCIP_HIT (_ULCAST_(0x1) << (CSR_GCFG_GCIP_SHIFT + 1))
+#define CSR_GCFG_GCIP_SECURE (_ULCAST_(0x1) << (CSR_GCFG_GCIP_SHIFT + 2))
+#define CSR_GCFG_TORU_SHIFT 15
+#define CSR_GCFG_TORU (_ULCAST_(0x1) << CSR_GCFG_TORU_SHIFT)
+#define CSR_GCFG_TORUP_SHIFT 14
+#define CSR_GCFG_TORUP (_ULCAST_(0x1) << CSR_GCFG_TORUP_SHIFT)
+#define CSR_GCFG_TOP_SHIFT 13
+#define CSR_GCFG_TOP (_ULCAST_(0x1) << CSR_GCFG_TOP_SHIFT)
+#define CSR_GCFG_TOPP_SHIFT 12
+#define CSR_GCFG_TOPP (_ULCAST_(0x1) << CSR_GCFG_TOPP_SHIFT)
+#define CSR_GCFG_TOE_SHIFT 11
+#define CSR_GCFG_TOE (_ULCAST_(0x1) << CSR_GCFG_TOE_SHIFT)
+#define CSR_GCFG_TOEP_SHIFT 10
+#define CSR_GCFG_TOEP (_ULCAST_(0x1) << CSR_GCFG_TOEP_SHIFT)
+#define CSR_GCFG_TIT_SHIFT 9
+#define CSR_GCFG_TIT (_ULCAST_(0x1) << CSR_GCFG_TIT_SHIFT)
+#define CSR_GCFG_TITP_SHIFT 8
+#define CSR_GCFG_TITP (_ULCAST_(0x1) << CSR_GCFG_TITP_SHIFT)
+#define CSR_GCFG_SIT_SHIFT 7
+#define CSR_GCFG_SIT (_ULCAST_(0x1) << CSR_GCFG_SIT_SHIFT)
+#define CSR_GCFG_SITP_SHIFT 6
+#define CSR_GCFG_SITP (_ULCAST_(0x1) << CSR_GCFG_SITP_SHIFT)
+#define CSR_GCFG_MATC_SHITF 4
+#define CSR_GCFG_MATC_WIDTH 2
+#define CSR_GCFG_MATC_MASK (_ULCAST_(0x3) << CSR_GCFG_MATC_SHITF)
+#define CSR_GCFG_MATC_GUEST (_ULCAST_(0x0) << CSR_GCFG_MATC_SHITF)
+#define CSR_GCFG_MATC_ROOT (_ULCAST_(0x1) << CSR_GCFG_MATC_SHITF)
+#define CSR_GCFG_MATC_NEST (_ULCAST_(0x2) << CSR_GCFG_MATC_SHITF)
+
+#define LOONGARCH_CSR_GINTC 0x52 /* Guest interrupt control */
+#define CSR_GINTC_HC_SHIFT 16
+#define CSR_GINTC_HC_WIDTH 8
+#define CSR_GINTC_HC (_ULCAST_(0xff) << CSR_GINTC_HC_SHIFT)
+#define CSR_GINTC_PIP_SHIFT 8
+#define CSR_GINTC_PIP_WIDTH 8
+#define CSR_GINTC_PIP (_ULCAST_(0xff) << CSR_GINTC_PIP_SHIFT)
+#define CSR_GINTC_VIP_SHIFT 0
+#define CSR_GINTC_VIP_WIDTH 8
+#define CSR_GINTC_VIP (_ULCAST_(0xff))
+
+#define LOONGARCH_CSR_GCNTC 0x53 /* Guest timer offset */
+
+/* LLBCTL register */
+#define LOONGARCH_CSR_LLBCTL 0x60 /* LLBit control */
+#define CSR_LLBCTL_ROLLB_SHIFT 0
+#define CSR_LLBCTL_ROLLB (_ULCAST_(1) << CSR_LLBCTL_ROLLB_SHIFT)
+#define CSR_LLBCTL_WCLLB_SHIFT 1
+#define CSR_LLBCTL_WCLLB (_ULCAST_(1) << CSR_LLBCTL_WCLLB_SHIFT)
+#define CSR_LLBCTL_KLO_SHIFT 2
+#define CSR_LLBCTL_KLO (_ULCAST_(1) << CSR_LLBCTL_KLO_SHIFT)
+
+/* Implement dependent */
+#define LOONGARCH_CSR_IMPCTL1 0x80 /* Loongson config1 */
+#define CSR_MISPEC_SHIFT 20
+#define CSR_MISPEC_WIDTH 8
+#define CSR_MISPEC (_ULCAST_(0xff) << CSR_MISPEC_SHIFT)
+#define CSR_SSEN_SHIFT 18
+#define CSR_SSEN (_ULCAST_(1) << CSR_SSEN_SHIFT)
+#define CSR_SCRAND_SHIFT 17
+#define CSR_SCRAND (_ULCAST_(1) << CSR_SCRAND_SHIFT)
+#define CSR_LLEXCL_SHIFT 16
+#define CSR_LLEXCL (_ULCAST_(1) << CSR_LLEXCL_SHIFT)
+#define CSR_DISVC_SHIFT 15
+#define CSR_DISVC (_ULCAST_(1) << CSR_DISVC_SHIFT)
+#define CSR_VCLRU_SHIFT 14
+#define CSR_VCLRU (_ULCAST_(1) << CSR_VCLRU_SHIFT)
+#define CSR_DCLRU_SHIFT 13
+#define CSR_DCLRU (_ULCAST_(1) << CSR_DCLRU_SHIFT)
+#define CSR_FASTLDQ_SHIFT 12
+#define CSR_FASTLDQ (_ULCAST_(1) << CSR_FASTLDQ_SHIFT)
+#define CSR_USERCAC_SHIFT 11
+#define CSR_USERCAC (_ULCAST_(1) << CSR_USERCAC_SHIFT)
+#define CSR_ANTI_MISPEC_SHIFT 10
+#define CSR_ANTI_MISPEC (_ULCAST_(1) << CSR_ANTI_MISPEC_SHIFT)
+#define CSR_AUTO_FLUSHSFB_SHIFT 9
+#define CSR_AUTO_FLUSHSFB (_ULCAST_(1) << CSR_AUTO_FLUSHSFB_SHIFT)
+#define CSR_STFILL_SHIFT 8
+#define CSR_STFILL (_ULCAST_(1) << CSR_STFILL_SHIFT)
+#define CSR_LIFEP_SHIFT 7
+#define CSR_LIFEP (_ULCAST_(1) << CSR_LIFEP_SHIFT)
+#define CSR_LLSYNC_SHIFT 6
+#define CSR_LLSYNC (_ULCAST_(1) << CSR_LLSYNC_SHIFT)
+#define CSR_BRBTDIS_SHIFT 5
+#define CSR_BRBTDIS (_ULCAST_(1) << CSR_BRBTDIS_SHIFT)
+#define CSR_RASDIS_SHIFT 4
+#define CSR_RASDIS (_ULCAST_(1) << CSR_RASDIS_SHIFT)
+#define CSR_STPRE_SHIFT 2
+#define CSR_STPRE_WIDTH 2
+#define CSR_STPRE (_ULCAST_(3) << CSR_STPRE_SHIFT)
+#define CSR_INSTPRE_SHIFT 1
+#define CSR_INSTPRE (_ULCAST_(1) << CSR_INSTPRE_SHIFT)
+#define CSR_DATAPRE_SHIFT 0
+#define CSR_DATAPRE (_ULCAST_(1) << CSR_DATAPRE_SHIFT)
+
+#define LOONGARCH_CSR_IMPCTL2 0x81 /* Loongson config2 */
+#define CSR_FLUSH_MTLB_SHIFT 0
+#define CSR_FLUSH_MTLB (_ULCAST_(1) << CSR_FLUSH_MTLB_SHIFT)
+#define CSR_FLUSH_STLB_SHIFT 1
+#define CSR_FLUSH_STLB (_ULCAST_(1) << CSR_FLUSH_STLB_SHIFT)
+#define CSR_FLUSH_DTLB_SHIFT 2
+#define CSR_FLUSH_DTLB (_ULCAST_(1) << CSR_FLUSH_DTLB_SHIFT)
+#define CSR_FLUSH_ITLB_SHIFT 3
+#define CSR_FLUSH_ITLB (_ULCAST_(1) << CSR_FLUSH_ITLB_SHIFT)
+#define CSR_FLUSH_BTAC_SHIFT 4
+#define CSR_FLUSH_BTAC (_ULCAST_(1) << CSR_FLUSH_BTAC_SHIFT)
+
+#define LOONGARCH_CSR_GNMI 0x82
+
+/* TLB Refill registers */
+#define LOONGARCH_CSR_TLBRENTRY 0x88 /* TLB refill exception entry */
+#define LOONGARCH_CSR_TLBRBADV 0x89 /* TLB refill badvaddr */
+#define LOONGARCH_CSR_TLBRERA 0x8a /* TLB refill ERA */
+#define LOONGARCH_CSR_TLBRSAVE 0x8b /* KSave for TLB refill exception */
+#define LOONGARCH_CSR_TLBRELO0 0x8c /* TLB refill entrylo0 */
+#define LOONGARCH_CSR_TLBRELO1 0x8d /* TLB refill entrylo1 */
+#define LOONGARCH_CSR_TLBREHI 0x8e /* TLB refill entryhi */
+#define CSR_TLBREHI_PS_SHIFT 0
+#define CSR_TLBREHI_PS (_ULCAST_(0x3f) << CSR_TLBREHI_PS_SHIFT)
+#define LOONGARCH_CSR_TLBRPRMD 0x8f /* TLB refill mode info */
+
+/* Machine Error registers */
+#define LOONGARCH_CSR_MERRCTL 0x90 /* MERRCTL */
+#define LOONGARCH_CSR_MERRINFO1 0x91 /* MError info1 */
+#define LOONGARCH_CSR_MERRINFO2 0x92 /* MError info2 */
+#define LOONGARCH_CSR_MERRENTRY 0x93 /* MError exception entry */
+#define LOONGARCH_CSR_MERRERA 0x94 /* MError exception ERA */
+#define LOONGARCH_CSR_MERRSAVE 0x95 /* KSave for machine error exception */
+
+#define LOONGARCH_CSR_CTAG 0x98 /* TagLo + TagHi */
+
+#define LOONGARCH_CSR_PRID 0xc0
+
+/* Shadow MCSR : 0xc0 ~ 0xff */
+#define LOONGARCH_CSR_MCSR0 0xc0 /* CPUCFG0 and CPUCFG1 */
+#define MCSR0_INT_IMPL_SHIFT 58
+#define MCSR0_INT_IMPL 0
+#define MCSR0_IOCSR_BRD_SHIFT 57
+#define MCSR0_IOCSR_BRD (_ULCAST_(1) << MCSR0_IOCSR_BRD_SHIFT)
+#define MCSR0_HUGEPG_SHIFT 56
+#define MCSR0_HUGEPG (_ULCAST_(1) << MCSR0_HUGEPG_SHIFT)
+#define MCSR0_RPLMTLB_SHIFT 55
+#define MCSR0_RPLMTLB (_ULCAST_(1) << MCSR0_RPLMTLB_SHIFT)
+#define MCSR0_EP_SHIFT 54
+#define MCSR0_EP (_ULCAST_(1) << MCSR0_EP_SHIFT)
+#define MCSR0_RI_SHIFT 53
+#define MCSR0_RI (_ULCAST_(1) << MCSR0_RI_SHIFT)
+#define MCSR0_UAL_SHIFT 52
+#define MCSR0_UAL (_ULCAST_(1) << MCSR0_UAL_SHIFT)
+#define MCSR0_VABIT_SHIFT 44
+#define MCSR0_VABIT_WIDTH 8
+#define MCSR0_VABIT (_ULCAST_(0xff) << MCSR0_VABIT_SHIFT)
+#define VABIT_DEFAULT 0x2f
+#define MCSR0_PABIT_SHIFT 36
+#define MCSR0_PABIT_WIDTH 8
+#define MCSR0_PABIT (_ULCAST_(0xff) << MCSR0_PABIT_SHIFT)
+#define PABIT_DEFAULT 0x2f
+#define MCSR0_IOCSR_SHIFT 35
+#define MCSR0_IOCSR (_ULCAST_(1) << MCSR0_IOCSR_SHIFT)
+#define MCSR0_PAGING_SHIFT 34
+#define MCSR0_PAGING (_ULCAST_(1) << MCSR0_PAGING_SHIFT)
+#define MCSR0_GR64_SHIFT 33
+#define MCSR0_GR64 (_ULCAST_(1) << MCSR0_GR64_SHIFT)
+#define GR64_DEFAULT 1
+#define MCSR0_GR32_SHIFT 32
+#define MCSR0_GR32 (_ULCAST_(1) << MCSR0_GR32_SHIFT)
+#define GR32_DEFAULT 0
+#define MCSR0_PRID_WIDTH 32
+#define MCSR0_PRID 0x14C010
+
+#define LOONGARCH_CSR_MCSR1 0xc1 /* CPUCFG2 and CPUCFG3 */
+#define MCSR1_HPFOLD_SHIFT 43
+#define MCSR1_HPFOLD (_ULCAST_(1) << MCSR1_HPFOLD_SHIFT)
+#define MCSR1_SPW_LVL_SHIFT 40
+#define MCSR1_SPW_LVL_WIDTH 3
+#define MCSR1_SPW_LVL (_ULCAST_(7) << MCSR1_SPW_LVL_SHIFT)
+#define MCSR1_ICACHET_SHIFT 39
+#define MCSR1_ICACHET (_ULCAST_(1) << MCSR1_ICACHET_SHIFT)
+#define MCSR1_ITLBT_SHIFT 38
+#define MCSR1_ITLBT (_ULCAST_(1) << MCSR1_ITLBT_SHIFT)
+#define MCSR1_LLDBAR_SHIFT 37
+#define MCSR1_LLDBAR (_ULCAST_(1) << MCSR1_LLDBAR_SHIFT)
+#define MCSR1_SCDLY_SHIFT 36
+#define MCSR1_SCDLY (_ULCAST_(1) << MCSR1_SCDLY_SHIFT)
+#define MCSR1_LLEXC_SHIFT 35
+#define MCSR1_LLEXC (_ULCAST_(1) << MCSR1_LLEXC_SHIFT)
+#define MCSR1_UCACC_SHIFT 34
+#define MCSR1_UCACC (_ULCAST_(1) << MCSR1_UCACC_SHIFT)
+#define MCSR1_SFB_SHIFT 33
+#define MCSR1_SFB (_ULCAST_(1) << MCSR1_SFB_SHIFT)
+#define MCSR1_CCDMA_SHIFT 32
+#define MCSR1_CCDMA (_ULCAST_(1) << MCSR1_CCDMA_SHIFT)
+#define MCSR1_LAMO_SHIFT 22
+#define MCSR1_LAMO (_ULCAST_(1) << MCSR1_LAMO_SHIFT)
+#define MCSR1_LSPW_SHIFT 21
+#define MCSR1_LSPW (_ULCAST_(1) << MCSR1_LSPW_SHIFT)
+#define MCSR1_MIPSBT_SHIFT 20
+#define MCSR1_MIPSBT (_ULCAST_(1) << MCSR1_MIPSBT_SHIFT)
+#define MCSR1_ARMBT_SHIFT 19
+#define MCSR1_ARMBT (_ULCAST_(1) << MCSR1_ARMBT_SHIFT)
+#define MCSR1_X86BT_SHIFT 18
+#define MCSR1_X86BT (_ULCAST_(1) << MCSR1_X86BT_SHIFT)
+#define MCSR1_LLFTPVERS_SHIFT 15
+#define MCSR1_LLFTPVERS_WIDTH 3
+#define MCSR1_LLFTPVERS (_ULCAST_(7) << MCSR1_LLFTPVERS_SHIFT)
+#define MCSR1_LLFTP_SHIFT 14
+#define MCSR1_LLFTP (_ULCAST_(1) << MCSR1_LLFTP_SHIFT)
+#define MCSR1_VZVERS_SHIFT 11
+#define MCSR1_VZVERS_WIDTH 3
+#define MCSR1_VZVERS (_ULCAST_(7) << MCSR1_VZVERS_SHIFT)
+#define MCSR1_VZ_SHIFT 10
+#define MCSR1_VZ (_ULCAST_(1) << MCSR1_VZ_SHIFT)
+#define MCSR1_CRYPTO_SHIFT 9
+#define MCSR1_CRYPTO (_ULCAST_(1) << MCSR1_CRYPTO_SHIFT)
+#define MCSR1_COMPLEX_SHIFT 8
+#define MCSR1_COMPLEX (_ULCAST_(1) << MCSR1_COMPLEX_SHIFT)
+#define MCSR1_LASX_SHIFT 7
+#define MCSR1_LASX (_ULCAST_(1) << MCSR1_LASX_SHIFT)
+#define MCSR1_LSX_SHIFT 6
+#define MCSR1_LSX (_ULCAST_(1) << MCSR1_LSX_SHIFT)
+#define MCSR1_FPVERS_SHIFT 3
+#define MCSR1_FPVERS_WIDTH 3
+#define MCSR1_FPVERS (_ULCAST_(7) << MCSR1_FPVERS_SHIFT)
+#define MCSR1_FPDP_SHIFT 2
+#define MCSR1_FPDP (_ULCAST_(1) << MCSR1_FPDP_SHIFT)
+#define MCSR1_FPSP_SHIFT 1
+#define MCSR1_FPSP (_ULCAST_(1) << MCSR1_FPSP_SHIFT)
+#define MCSR1_FP_SHIFT 0
+#define MCSR1_FP (_ULCAST_(1) << MCSR1_FP_SHIFT)
+
+#define LOONGARCH_CSR_MCSR2 0xc2 /* CPUCFG4 and CPUCFG5 */
+#define MCSR2_CCDIV_SHIFT 48
+#define MCSR2_CCDIV_WIDTH 16
+#define MCSR2_CCDIV (_ULCAST_(0xffff) << MCSR2_CCDIV_SHIFT)
+#define MCSR2_CCMUL_SHIFT 32
+#define MCSR2_CCMUL_WIDTH 16
+#define MCSR2_CCMUL (_ULCAST_(0xffff) << MCSR2_CCMUL_SHIFT)
+#define MCSR2_CCFREQ_WIDTH 32
+#define MCSR2_CCFREQ (_ULCAST_(0xffffffff))
+#define CCFREQ_DEFAULT 0x5f5e100 /* 100MHz */
+
+#define LOONGARCH_CSR_MCSR3 0xc3 /* CPUCFG6 */
+#define MCSR3_UPM_SHIFT 14
+#define MCSR3_UPM (_ULCAST_(1) << MCSR3_UPM_SHIFT)
+#define MCSR3_PMBITS_SHIFT 8
+#define MCSR3_PMBITS_WIDTH 6
+#define MCSR3_PMBITS (_ULCAST_(0x3f) << MCSR3_PMBITS_SHIFT)
+#define PMBITS_DEFAULT 0x40
+#define MCSR3_PMNUM_SHIFT 4
+#define MCSR3_PMNUM_WIDTH 4
+#define MCSR3_PMNUM (_ULCAST_(0xf) << MCSR3_PMNUM_SHIFT)
+#define MCSR3_PAMVER_SHIFT 1
+#define MCSR3_PAMVER_WIDTH 3
+#define MCSR3_PAMVER (_ULCAST_(0x7) << MCSR3_PAMVER_SHIFT)
+#define MCSR3_PMP_SHIFT 0
+#define MCSR3_PMP (_ULCAST_(1) << MCSR3_PMP_SHIFT)
+
+#define LOONGARCH_CSR_MCSR8 0xc8 /* CPUCFG16 and CPUCFG17 */
+#define MCSR8_L1I_SIZE_SHIFT 56
+#define MCSR8_L1I_SIZE_WIDTH 7
+#define MCSR8_L1I_SIZE (_ULCAST_(0x7f) << MCSR8_L1I_SIZE_SHIFT)
+#define MCSR8_L1I_IDX_SHIFT 48
+#define MCSR8_L1I_IDX_WIDTH 8
+#define MCSR8_L1I_IDX (_ULCAST_(0xff) << MCSR8_L1I_IDX_SHIFT)
+#define MCSR8_L1I_WAY_SHIFT 32
+#define MCSR8_L1I_WAY_WIDTH 16
+#define MCSR8_L1I_WAY (_ULCAST_(0xffff) << MCSR8_L1I_WAY_SHIFT)
+#define MCSR8_L3DINCL_SHIFT 16
+#define MCSR8_L3DINCL (_ULCAST_(1) << MCSR8_L3DINCL_SHIFT)
+#define MCSR8_L3DPRIV_SHIFT 15
+#define MCSR8_L3DPRIV (_ULCAST_(1) << MCSR8_L3DPRIV_SHIFT)
+#define MCSR8_L3DPRE_SHIFT 14
+#define MCSR8_L3DPRE (_ULCAST_(1) << MCSR8_L3DPRE_SHIFT)
+#define MCSR8_L3IUINCL_SHIFT 13
+#define MCSR8_L3IUINCL (_ULCAST_(1) << MCSR8_L3IUINCL_SHIFT)
+#define MCSR8_L3IUPRIV_SHIFT 12
+#define MCSR8_L3IUPRIV (_ULCAST_(1) << MCSR8_L3IUPRIV_SHIFT)
+#define MCSR8_L3IUUNIFY_SHIFT 11
+#define MCSR8_L3IUUNIFY (_ULCAST_(1) << MCSR8_L3IUUNIFY_SHIFT)
+#define MCSR8_L3IUPRE_SHIFT 10
+#define MCSR8_L3IUPRE (_ULCAST_(1) << MCSR8_L3IUPRE_SHIFT)
+#define MCSR8_L2DINCL_SHIFT 9
+#define MCSR8_L2DINCL (_ULCAST_(1) << MCSR8_L2DINCL_SHIFT)
+#define MCSR8_L2DPRIV_SHIFT 8
+#define MCSR8_L2DPRIV (_ULCAST_(1) << MCSR8_L2DPRIV_SHIFT)
+#define MCSR8_L2DPRE_SHIFT 7
+#define MCSR8_L2DPRE (_ULCAST_(1) << MCSR8_L2DPRE_SHIFT)
+#define MCSR8_L2IUINCL_SHIFT 6
+#define MCSR8_L2IUINCL (_ULCAST_(1) << MCSR8_L2IUINCL_SHIFT)
+#define MCSR8_L2IUPRIV_SHIFT 5
+#define MCSR8_L2IUPRIV (_ULCAST_(1) << MCSR8_L2IUPRIV_SHIFT)
+#define MCSR8_L2IUUNIFY_SHIFT 4
+#define MCSR8_L2IUUNIFY (_ULCAST_(1) << MCSR8_L2IUUNIFY_SHIFT)
+#define MCSR8_L2IUPRE_SHIFT 3
+#define MCSR8_L2IUPRE (_ULCAST_(1) << MCSR8_L2IUPRE_SHIFT)
+#define MCSR8_L1DPRE_SHIFT 2
+#define MCSR8_L1DPRE (_ULCAST_(1) << MCSR8_L1DPRE_SHIFT)
+#define MCSR8_L1IUUNIFY_SHIFT 1
+#define MCSR8_L1IUUNIFY (_ULCAST_(1) << MCSR8_L1IUUNIFY_SHIFT)
+#define MCSR8_L1IUPRE_SHIFT 0
+#define MCSR8_L1IUPRE (_ULCAST_(1) << MCSR8_L1IUPRE_SHIFT)
+
+#define LOONGARCH_CSR_MCSR9 0xc9 /* CPUCFG18 and CPUCFG19 */
+#define MCSR9_L2U_SIZE_SHIFT 56
+#define MCSR9_L2U_SIZE_WIDTH 7
+#define MCSR9_L2U_SIZE (_ULCAST_(0x7f) << MCSR9_L2U_SIZE_SHIFT)
+#define MCSR9_L2U_IDX_SHIFT 48
+#define MCSR9_L2U_IDX_WIDTH 8
+#define MCSR9_L2U_IDX (_ULCAST_(0xff) << MCSR9_IDX_LOG_SHIFT)
+#define MCSR9_L2U_WAY_SHIFT 32
+#define MCSR9_L2U_WAY_WIDTH 16
+#define MCSR9_L2U_WAY (_ULCAST_(0xffff) << MCSR9_L2U_WAY_SHIFT)
+#define MCSR9_L1D_SIZE_SHIFT 24
+#define MCSR9_L1D_SIZE_WIDTH 7
+#define MCSR9_L1D_SIZE (_ULCAST_(0x7f) << MCSR9_L1D_SIZE_SHIFT)
+#define MCSR9_L1D_IDX_SHIFT 16
+#define MCSR9_L1D_IDX_WIDTH 8
+#define MCSR9_L1D_IDX (_ULCAST_(0xff) << MCSR9_L1D_IDX_SHIFT)
+#define MCSR9_L1D_WAY_SHIFT 0
+#define MCSR9_L1D_WAY_WIDTH 16
+#define MCSR9_L1D_WAY (_ULCAST_(0xffff) << MCSR9_L1D_WAY_SHIFT)
+
+#define LOONGARCH_CSR_MCSR10 0xca /* CPUCFG20 */
+#define MCSR10_L3U_SIZE_SHIFT 24
+#define MCSR10_L3U_SIZE_WIDTH 7
+#define MCSR10_L3U_SIZE (_ULCAST_(0x7f) << MCSR10_L3U_SIZE_SHIFT)
+#define MCSR10_L3U_IDX_SHIFT 16
+#define MCSR10_L3U_IDX_WIDTH 8
+#define MCSR10_L3U_IDX (_ULCAST_(0xff) << MCSR10_L3U_IDX_SHIFT)
+#define MCSR10_L3U_WAY_SHIFT 0
+#define MCSR10_L3U_WAY_WIDTH 16
+#define MCSR10_L3U_WAY (_ULCAST_(0xffff) << MCSR10_L3U_WAY_SHIFT)
+
+#define LOONGARCH_CSR_MCSR24 0xf0 /* cpucfg48 */
+#define MCSR24_RAMCG_SHIFT 3
+#define MCSR24_RAMCG (_ULCAST_(1) << MCSR24_RAMCG_SHIFT)
+#define MCSR24_VFPUCG_SHIFT 2
+#define MCSR24_VFPUCG (_ULCAST_(1) << MCSR24_VFPUCG_SHIFT)
+#define MCSR24_NAPEN_SHIFT 1
+#define MCSR24_NAPEN (_ULCAST_(1) << MCSR24_NAPEN_SHIFT)
+#define MCSR24_MCSRLOCK_SHIFT 0
+#define MCSR24_MCSRLOCK (_ULCAST_(1) << MCSR24_MCSRLOCK_SHIFT)
+
+/* Uncached accelerate windows registers */
+#define LOONGARCH_CSR_UCAWIN 0x100
+#define LOONGARCH_CSR_UCAWIN0_LO 0x102
+#define LOONGARCH_CSR_UCAWIN0_HI 0x103
+#define LOONGARCH_CSR_UCAWIN1_LO 0x104
+#define LOONGARCH_CSR_UCAWIN1_HI 0x105
+#define LOONGARCH_CSR_UCAWIN2_LO 0x106
+#define LOONGARCH_CSR_UCAWIN2_HI 0x107
+#define LOONGARCH_CSR_UCAWIN3_LO 0x108
+#define LOONGARCH_CSR_UCAWIN3_HI 0x109
+
+/* Direct Map windows registers */
+#define LOONGARCH_CSR_DMWIN0 0x180 /* 64 direct map win0: MEM & IF */
+#define LOONGARCH_CSR_DMWIN1 0x181 /* 64 direct map win1: MEM & IF */
+#define LOONGARCH_CSR_DMWIN2 0x182 /* 64 direct map win2: MEM */
+#define LOONGARCH_CSR_DMWIN3 0x183 /* 64 direct map win3: MEM */
+
+/* Direct Map window 0/1 */
+#define CSR_DMW0_PLV0 _CONST64_(1 << 0)
+#define CSR_DMW0_VSEG _CONST64_(0x8000)
+#define CSR_DMW0_BASE (CSR_DMW0_VSEG << DMW_PABITS)
+#define CSR_DMW0_INIT (CSR_DMW0_BASE | CSR_DMW0_PLV0)
+
+#define CSR_DMW1_PLV0 _CONST64_(1 << 0)
+#define CSR_DMW1_MAT _CONST64_(1 << 4)
+#define CSR_DMW1_VSEG _CONST64_(0x9000)
+#define CSR_DMW1_BASE (CSR_DMW1_VSEG << DMW_PABITS)
+#define CSR_DMW1_INIT (CSR_DMW1_BASE | CSR_DMW1_MAT | CSR_DMW1_PLV0)
+
+/* Performance Counter registers */
+#define LOONGARCH_CSR_PERFCTRL0 0x200 /* 32 perf event 0 config */
+#define LOONGARCH_CSR_PERFCNTR0 0x201 /* 64 perf event 0 count value */
+#define LOONGARCH_CSR_PERFCTRL1 0x202 /* 32 perf event 1 config */
+#define LOONGARCH_CSR_PERFCNTR1 0x203 /* 64 perf event 1 count value */
+#define LOONGARCH_CSR_PERFCTRL2 0x204 /* 32 perf event 2 config */
+#define LOONGARCH_CSR_PERFCNTR2 0x205 /* 64 perf event 2 count value */
+#define LOONGARCH_CSR_PERFCTRL3 0x206 /* 32 perf event 3 config */
+#define LOONGARCH_CSR_PERFCNTR3 0x207 /* 64 perf event 3 count value */
+#define CSR_PERFCTRL_PLV0 (_ULCAST_(1) << 16)
+#define CSR_PERFCTRL_PLV1 (_ULCAST_(1) << 17)
+#define CSR_PERFCTRL_PLV2 (_ULCAST_(1) << 18)
+#define CSR_PERFCTRL_PLV3 (_ULCAST_(1) << 19)
+#define CSR_PERFCTRL_IE (_ULCAST_(1) << 20)
+#define CSR_PERFCTRL_EVENT 0x3ff
+
+/* Debug registers */
+#define LOONGARCH_CSR_MWPC 0x300 /* data breakpoint config */
+#define LOONGARCH_CSR_MWPS 0x301 /* data breakpoint status */
+
+#define LOONGARCH_CSR_DB0ADDR 0x310 /* data breakpoint 0 address */
+#define LOONGARCH_CSR_DB0MASK 0x311 /* data breakpoint 0 mask */
+#define LOONGARCH_CSR_DB0CTL 0x312 /* data breakpoint 0 control */
+#define LOONGARCH_CSR_DB0ASID 0x313 /* data breakpoint 0 asid */
+
+#define LOONGARCH_CSR_DB1ADDR 0x318 /* data breakpoint 1 address */
+#define LOONGARCH_CSR_DB1MASK 0x319 /* data breakpoint 1 mask */
+#define LOONGARCH_CSR_DB1CTL 0x31a /* data breakpoint 1 control */
+#define LOONGARCH_CSR_DB1ASID 0x31b /* data breakpoint 1 asid */
+
+#define LOONGARCH_CSR_DB2ADDR 0x320 /* data breakpoint 2 address */
+#define LOONGARCH_CSR_DB2MASK 0x321 /* data breakpoint 2 mask */
+#define LOONGARCH_CSR_DB2CTL 0x322 /* data breakpoint 2 control */
+#define LOONGARCH_CSR_DB2ASID 0x323 /* data breakpoint 2 asid */
+
+#define LOONGARCH_CSR_DB3ADDR 0x328 /* data breakpoint 3 address */
+#define LOONGARCH_CSR_DB3MASK 0x329 /* data breakpoint 3 mask */
+#define LOONGARCH_CSR_DB3CTL 0x32a /* data breakpoint 3 control */
+#define LOONGARCH_CSR_DB3ASID 0x32b /* data breakpoint 3 asid */
+
+#define LOONGARCH_CSR_DB4ADDR 0x330 /* data breakpoint 4 address */
+#define LOONGARCH_CSR_DB4MASK 0x331 /* data breakpoint 4 maks */
+#define LOONGARCH_CSR_DB4CTL 0x332 /* data breakpoint 4 control */
+#define LOONGARCH_CSR_DB4ASID 0x333 /* data breakpoint 4 asid */
+
+#define LOONGARCH_CSR_DB5ADDR 0x338 /* data breakpoint 5 address */
+#define LOONGARCH_CSR_DB5MASK 0x339 /* data breakpoint 5 mask */
+#define LOONGARCH_CSR_DB5CTL 0x33a /* data breakpoint 5 control */
+#define LOONGARCH_CSR_DB5ASID 0x33b /* data breakpoint 5 asid */
+
+#define LOONGARCH_CSR_DB6ADDR 0x340 /* data breakpoint 6 address */
+#define LOONGARCH_CSR_DB6MASK 0x341 /* data breakpoint 6 mask */
+#define LOONGARCH_CSR_DB6CTL 0x342 /* data breakpoint 6 control */
+#define LOONGARCH_CSR_DB6ASID 0x343 /* data breakpoint 6 asid */
+
+#define LOONGARCH_CSR_DB7ADDR 0x348 /* data breakpoint 7 address */
+#define LOONGARCH_CSR_DB7MASK 0x349 /* data breakpoint 7 mask */
+#define LOONGARCH_CSR_DB7CTL 0x34a /* data breakpoint 7 control */
+#define LOONGARCH_CSR_DB7ASID 0x34b /* data breakpoint 7 asid */
+
+#define LOONGARCH_CSR_FWPC 0x380 /* instruction breakpoint config */
+#define LOONGARCH_CSR_FWPS 0x381 /* instruction breakpoint status */
+
+#define LOONGARCH_CSR_IB0ADDR 0x390 /* inst breakpoint 0 address */
+#define LOONGARCH_CSR_IB0MASK 0x391 /* inst breakpoint 0 mask */
+#define LOONGARCH_CSR_IB0CTL 0x392 /* inst breakpoint 0 control */
+#define LOONGARCH_CSR_IB0ASID 0x393 /* inst breakpoint 0 asid */
+
+#define LOONGARCH_CSR_IB1ADDR 0x398 /* inst breakpoint 1 address */
+#define LOONGARCH_CSR_IB1MASK 0x399 /* inst breakpoint 1 mask */
+#define LOONGARCH_CSR_IB1CTL 0x39a /* inst breakpoint 1 control */
+#define LOONGARCH_CSR_IB1ASID 0x39b /* inst breakpoint 1 asid */
+
+#define LOONGARCH_CSR_IB2ADDR 0x3a0 /* inst breakpoint 2 address */
+#define LOONGARCH_CSR_IB2MASK 0x3a1 /* inst breakpoint 2 mask */
+#define LOONGARCH_CSR_IB2CTL 0x3a2 /* inst breakpoint 2 control */
+#define LOONGARCH_CSR_IB2ASID 0x3a3 /* inst breakpoint 2 asid */
+
+#define LOONGARCH_CSR_IB3ADDR 0x3a8 /* inst breakpoint 3 address */
+#define LOONGARCH_CSR_IB3MASK 0x3a9 /* breakpoint 3 mask */
+#define LOONGARCH_CSR_IB3CTL 0x3aa /* inst breakpoint 3 control */
+#define LOONGARCH_CSR_IB3ASID 0x3ab /* inst breakpoint 3 asid */
+
+#define LOONGARCH_CSR_IB4ADDR 0x3b0 /* inst breakpoint 4 address */
+#define LOONGARCH_CSR_IB4MASK 0x3b1 /* inst breakpoint 4 mask */
+#define LOONGARCH_CSR_IB4CTL 0x3b2 /* inst breakpoint 4 control */
+#define LOONGARCH_CSR_IB4ASID 0x3b3 /* inst breakpoint 4 asid */
+
+#define LOONGARCH_CSR_IB5ADDR 0x3b8 /* inst breakpoint 5 address */
+#define LOONGARCH_CSR_IB5MASK 0x3b9 /* inst breakpoint 5 mask */
+#define LOONGARCH_CSR_IB5CTL 0x3ba /* inst breakpoint 5 control */
+#define LOONGARCH_CSR_IB5ASID 0x3bb /* inst breakpoint 5 asid */
+
+#define LOONGARCH_CSR_IB6ADDR 0x3c0 /* inst breakpoint 6 address */
+#define LOONGARCH_CSR_IB6MASK 0x3c1 /* inst breakpoint 6 mask */
+#define LOONGARCH_CSR_IB6CTL 0x3c2 /* inst breakpoint 6 control */
+#define LOONGARCH_CSR_IB6ASID 0x3c3 /* inst breakpoint 6 asid */
+
+#define LOONGARCH_CSR_IB7ADDR 0x3c8 /* inst breakpoint 7 address */
+#define LOONGARCH_CSR_IB7MASK 0x3c9 /* inst breakpoint 7 mask */
+#define LOONGARCH_CSR_IB7CTL 0x3ca /* inst breakpoint 7 control */
+#define LOONGARCH_CSR_IB7ASID 0x3cb /* inst breakpoint 7 asid */
+
+#define LOONGARCH_CSR_DEBUG 0x500 /* debug config */
+#define LOONGARCH_CSR_DERA 0x501 /* debug era */
+#define LOONGARCH_CSR_DESAVE 0x502 /* debug save */
+
+/*
+ * CSR_ECFG IM
+ */
+#define ECFG0_IM 0x00001fff
+#define ECFGB_SIP0 0
+#define ECFGF_SIP0 (_ULCAST_(1) << ECFGB_SIP0)
+#define ECFGB_SIP1 1
+#define ECFGF_SIP1 (_ULCAST_(1) << ECFGB_SIP1)
+#define ECFGB_IP0 2
+#define ECFGF_IP0 (_ULCAST_(1) << ECFGB_IP0)
+#define ECFGB_IP1 3
+#define ECFGF_IP1 (_ULCAST_(1) << ECFGB_IP1)
+#define ECFGB_IP2 4
+#define ECFGF_IP2 (_ULCAST_(1) << ECFGB_IP2)
+#define ECFGB_IP3 5
+#define ECFGF_IP3 (_ULCAST_(1) << ECFGB_IP3)
+#define ECFGB_IP4 6
+#define ECFGF_IP4 (_ULCAST_(1) << ECFGB_IP4)
+#define ECFGB_IP5 7
+#define ECFGF_IP5 (_ULCAST_(1) << ECFGB_IP5)
+#define ECFGB_IP6 8
+#define ECFGF_IP6 (_ULCAST_(1) << ECFGB_IP6)
+#define ECFGB_IP7 9
+#define ECFGF_IP7 (_ULCAST_(1) << ECFGB_IP7)
+#define ECFGB_PMC 10
+#define ECFGF_PMC (_ULCAST_(1) << ECFGB_PMC)
+#define ECFGB_TIMER 11
+#define ECFGF_TIMER (_ULCAST_(1) << ECFGB_TIMER)
+#define ECFGB_IPI 12
+#define ECFGF_IPI (_ULCAST_(1) << ECFGB_IPI)
+#define ECFGF(hwirq) (_ULCAST_(1) << hwirq)
+
+#define ESTATF_IP 0x00001fff
+
+#define LOONGARCH_IOCSR_FEATURES 0x8
+#define IOCSRF_TEMP BIT_ULL(0)
+#define IOCSRF_NODECNT BIT_ULL(1)
+#define IOCSRF_MSI BIT_ULL(2)
+#define IOCSRF_EXTIOI BIT_ULL(3)
+#define IOCSRF_CSRIPI BIT_ULL(4)
+#define IOCSRF_FREQCSR BIT_ULL(5)
+#define IOCSRF_FREQSCALE BIT_ULL(6)
+#define IOCSRF_DVFSV1 BIT_ULL(7)
+#define IOCSRF_EIODECODE BIT_ULL(9)
+#define IOCSRF_FLATMODE BIT_ULL(10)
+#define IOCSRF_VM BIT_ULL(11)
+
+#define LOONGARCH_IOCSR_VENDOR 0x10
+
+#define LOONGARCH_IOCSR_CPUNAME 0x20
+
+#define LOONGARCH_IOCSR_NODECNT 0x408
+
+#define LOONGARCH_IOCSR_MISC_FUNC 0x420
+#define IOCSR_MISC_FUNC_TIMER_RESET BIT_ULL(21)
+#define IOCSR_MISC_FUNC_EXT_IOI_EN BIT_ULL(48)
+
+#define LOONGARCH_IOCSR_CPUTEMP 0x428
+
+/* PerCore CSR, only accessible by local cores */
+#define LOONGARCH_IOCSR_IPI_STATUS 0x1000
+#define LOONGARCH_IOCSR_IPI_EN 0x1004
+#define LOONGARCH_IOCSR_IPI_SET 0x1008
+#define LOONGARCH_IOCSR_IPI_CLEAR 0x100c
+#define LOONGARCH_IOCSR_MBUF0 0x1020
+#define LOONGARCH_IOCSR_MBUF1 0x1028
+#define LOONGARCH_IOCSR_MBUF2 0x1030
+#define LOONGARCH_IOCSR_MBUF3 0x1038
+
+#define LOONGARCH_IOCSR_IPI_SEND 0x1040
+#define IOCSR_IPI_SEND_IP_SHIFT 0
+#define IOCSR_IPI_SEND_CPU_SHIFT 16
+#define IOCSR_IPI_SEND_BLOCKING BIT(31)
+
+#define LOONGARCH_IOCSR_MBUF_SEND 0x1048
+#define IOCSR_MBUF_SEND_BLOCKING BIT_ULL(31)
+#define IOCSR_MBUF_SEND_BOX_SHIFT 2
+#define IOCSR_MBUF_SEND_BOX_LO(box) (box << 1)
+#define IOCSR_MBUF_SEND_BOX_HI(box) ((box << 1) + 1)
+#define IOCSR_MBUF_SEND_CPU_SHIFT 16
+#define IOCSR_MBUF_SEND_BUF_SHIFT 32
+#define IOCSR_MBUF_SEND_H32_MASK 0xFFFFFFFF00000000ULL
+
+#define LOONGARCH_IOCSR_ANY_SEND 0x1158
+#define IOCSR_ANY_SEND_BLOCKING BIT_ULL(31)
+#define IOCSR_ANY_SEND_CPU_SHIFT 16
+#define IOCSR_ANY_SEND_MASK_SHIFT 27
+#define IOCSR_ANY_SEND_BUF_SHIFT 32
+#define IOCSR_ANY_SEND_H32_MASK 0xFFFFFFFF00000000ULL
+
+/* Register offset and bit definition for CSR access */
+#define LOONGARCH_IOCSR_TIMER_CFG 0x1060
+#define LOONGARCH_IOCSR_TIMER_TICK 0x1070
+#define IOCSR_TIMER_CFG_RESERVED (_ULCAST_(1) << 63)
+#define IOCSR_TIMER_CFG_PERIODIC (_ULCAST_(1) << 62)
+#define IOCSR_TIMER_CFG_EN (_ULCAST_(1) << 61)
+#define IOCSR_TIMER_MASK 0x0ffffffffffffULL
+#define IOCSR_TIMER_INITVAL_RST (_ULCAST_(0xffff) << 48)
+
+#define LOONGARCH_IOCSR_EXTIOI_NODEMAP_BASE 0x14a0
+#define LOONGARCH_IOCSR_EXTIOI_IPMAP_BASE 0x14c0
+#define LOONGARCH_IOCSR_EXTIOI_EN_BASE 0x1600
+#define LOONGARCH_IOCSR_EXTIOI_BOUNCE_BASE 0x1680
+#define LOONGARCH_IOCSR_EXTIOI_ISR_BASE 0x1800
+#define LOONGARCH_IOCSR_EXTIOI_ROUTE_BASE 0x1c00
+#define IOCSR_EXTIOI_VECTOR_NUM 256
+
+#ifndef __ASSEMBLY__
+
+static inline u64 drdtime(void)
+{
+ int rID = 0;
+ u64 val = 0;
+
+ __asm__ __volatile__(
+ "rdtime.d %0, %1 \n\t"
+ : "=r"(val), "=r"(rID)
+ :
+ );
+ return val;
+}
+
+static inline unsigned int get_csr_cpuid(void)
+{
+ return csr_readl(LOONGARCH_CSR_CPUID);
+}
+
+static inline void csr_any_send(unsigned int addr, unsigned int data,
+ unsigned int data_mask, unsigned int cpu)
+{
+ uint64_t val = 0;
+
+ val = IOCSR_ANY_SEND_BLOCKING | addr;
+ val |= (cpu << IOCSR_ANY_SEND_CPU_SHIFT);
+ val |= (data_mask << IOCSR_ANY_SEND_MASK_SHIFT);
+ val |= ((uint64_t)data << IOCSR_ANY_SEND_BUF_SHIFT);
+ iocsr_writeq(val, LOONGARCH_IOCSR_ANY_SEND);
+}
+
+static inline unsigned int read_csr_excode(void)
+{
+ return (csr_readl(LOONGARCH_CSR_ESTAT) & CSR_ESTAT_EXC) >> CSR_ESTAT_EXC_SHIFT;
+}
+
+static inline void write_csr_index(unsigned int idx)
+{
+ csr_xchgl(idx, CSR_TLBIDX_IDXM, LOONGARCH_CSR_TLBIDX);
+}
+
+static inline unsigned int read_csr_pagesize(void)
+{
+ return (csr_readl(LOONGARCH_CSR_TLBIDX) & CSR_TLBIDX_SIZEM) >> CSR_TLBIDX_SIZE;
+}
+
+static inline void write_csr_pagesize(unsigned int size)
+{
+ csr_xchgl(size << CSR_TLBIDX_SIZE, CSR_TLBIDX_SIZEM, LOONGARCH_CSR_TLBIDX);
+}
+
+static inline unsigned int read_csr_tlbrefill_pagesize(void)
+{
+ return (csr_readq(LOONGARCH_CSR_TLBREHI) & CSR_TLBREHI_PS) >> CSR_TLBREHI_PS_SHIFT;
+}
+
+static inline void write_csr_tlbrefill_pagesize(unsigned int size)
+{
+ csr_xchgq(size << CSR_TLBREHI_PS_SHIFT, CSR_TLBREHI_PS, LOONGARCH_CSR_TLBREHI);
+}
+
+#define read_csr_asid() csr_readl(LOONGARCH_CSR_ASID)
+#define write_csr_asid(val) csr_writel(val, LOONGARCH_CSR_ASID)
+#define read_csr_entryhi() csr_readq(LOONGARCH_CSR_TLBEHI)
+#define write_csr_entryhi(val) csr_writeq(val, LOONGARCH_CSR_TLBEHI)
+#define read_csr_entrylo0() csr_readq(LOONGARCH_CSR_TLBELO0)
+#define write_csr_entrylo0(val) csr_writeq(val, LOONGARCH_CSR_TLBELO0)
+#define read_csr_entrylo1() csr_readq(LOONGARCH_CSR_TLBELO1)
+#define write_csr_entrylo1(val) csr_writeq(val, LOONGARCH_CSR_TLBELO1)
+#define read_csr_ecfg() csr_readl(LOONGARCH_CSR_ECFG)
+#define write_csr_ecfg(val) csr_writel(val, LOONGARCH_CSR_ECFG)
+#define read_csr_estat() csr_readl(LOONGARCH_CSR_ESTAT)
+#define write_csr_estat(val) csr_writel(val, LOONGARCH_CSR_ESTAT)
+#define read_csr_tlbidx() csr_readl(LOONGARCH_CSR_TLBIDX)
+#define write_csr_tlbidx(val) csr_writel(val, LOONGARCH_CSR_TLBIDX)
+#define read_csr_euen() csr_readl(LOONGARCH_CSR_EUEN)
+#define write_csr_euen(val) csr_writel(val, LOONGARCH_CSR_EUEN)
+#define read_csr_cpuid() csr_readl(LOONGARCH_CSR_CPUID)
+#define read_csr_prcfg1() csr_readq(LOONGARCH_CSR_PRCFG1)
+#define write_csr_prcfg1(val) csr_writeq(val, LOONGARCH_CSR_PRCFG1)
+#define read_csr_prcfg2() csr_readq(LOONGARCH_CSR_PRCFG2)
+#define write_csr_prcfg2(val) csr_writeq(val, LOONGARCH_CSR_PRCFG2)
+#define read_csr_prcfg3() csr_readq(LOONGARCH_CSR_PRCFG3)
+#define write_csr_prcfg3(val) csr_writeq(val, LOONGARCH_CSR_PRCFG3)
+#define read_csr_stlbpgsize() csr_readl(LOONGARCH_CSR_STLBPGSIZE)
+#define write_csr_stlbpgsize(val) csr_writel(val, LOONGARCH_CSR_STLBPGSIZE)
+#define read_csr_rvacfg() csr_readl(LOONGARCH_CSR_RVACFG)
+#define write_csr_rvacfg(val) csr_writel(val, LOONGARCH_CSR_RVACFG)
+#define write_csr_tintclear(val) csr_writel(val, LOONGARCH_CSR_TINTCLR)
+#define read_csr_impctl1() csr_readq(LOONGARCH_CSR_IMPCTL1)
+#define write_csr_impctl1(val) csr_writeq(val, LOONGARCH_CSR_IMPCTL1)
+#define write_csr_impctl2(val) csr_writeq(val, LOONGARCH_CSR_IMPCTL2)
+
+#define read_csr_perfctrl0() csr_readq(LOONGARCH_CSR_PERFCTRL0)
+#define read_csr_perfcntr0() csr_readq(LOONGARCH_CSR_PERFCNTR0)
+#define read_csr_perfctrl1() csr_readq(LOONGARCH_CSR_PERFCTRL1)
+#define read_csr_perfcntr1() csr_readq(LOONGARCH_CSR_PERFCNTR1)
+#define read_csr_perfctrl2() csr_readq(LOONGARCH_CSR_PERFCTRL2)
+#define read_csr_perfcntr2() csr_readq(LOONGARCH_CSR_PERFCNTR2)
+#define read_csr_perfctrl3() csr_readq(LOONGARCH_CSR_PERFCTRL3)
+#define read_csr_perfcntr3() csr_readq(LOONGARCH_CSR_PERFCNTR3)
+#define write_csr_perfctrl0(val) csr_writeq(val, LOONGARCH_CSR_PERFCTRL0)
+#define write_csr_perfcntr0(val) csr_writeq(val, LOONGARCH_CSR_PERFCNTR0)
+#define write_csr_perfctrl1(val) csr_writeq(val, LOONGARCH_CSR_PERFCTRL1)
+#define write_csr_perfcntr1(val) csr_writeq(val, LOONGARCH_CSR_PERFCNTR1)
+#define write_csr_perfctrl2(val) csr_writeq(val, LOONGARCH_CSR_PERFCTRL2)
+#define write_csr_perfcntr2(val) csr_writeq(val, LOONGARCH_CSR_PERFCNTR2)
+#define write_csr_perfctrl3(val) csr_writeq(val, LOONGARCH_CSR_PERFCTRL3)
+#define write_csr_perfcntr3(val) csr_writeq(val, LOONGARCH_CSR_PERFCNTR3)
+
+/*
+ * Manipulate bits in a register.
+ */
+#define __BUILD_CSR_COMMON(name) \
+static inline unsigned long \
+set_##name(unsigned long set) \
+{ \
+ unsigned long res, new; \
+ \
+ res = read_##name(); \
+ new = res | set; \
+ write_##name(new); \
+ \
+ return res; \
+} \
+ \
+static inline unsigned long \
+clear_##name(unsigned long clear) \
+{ \
+ unsigned long res, new; \
+ \
+ res = read_##name(); \
+ new = res & ~clear; \
+ write_##name(new); \
+ \
+ return res; \
+} \
+ \
+static inline unsigned long \
+change_##name(unsigned long change, unsigned long val) \
+{ \
+ unsigned long res, new; \
+ \
+ res = read_##name(); \
+ new = res & ~change; \
+ new |= (val & change); \
+ write_##name(new); \
+ \
+ return res; \
+}
+
+#define __BUILD_CSR_OP(name) __BUILD_CSR_COMMON(csr_##name)
+
+__BUILD_CSR_OP(euen)
+__BUILD_CSR_OP(ecfg)
+__BUILD_CSR_OP(tlbidx)
+
+#define set_csr_estat(val) \
+ csr_xchgl(val, val, LOONGARCH_CSR_ESTAT)
+#define clear_csr_estat(val) \
+ csr_xchgl(~(val), val, LOONGARCH_CSR_ESTAT)
+
+#endif /* __ASSEMBLY__ */
+
+/* Generic EntryLo bit definitions */
+#define ENTRYLO_V (_ULCAST_(1) << 0)
+#define ENTRYLO_D (_ULCAST_(1) << 1)
+#define ENTRYLO_PLV_SHIFT 2
+#define ENTRYLO_PLV (_ULCAST_(3) << ENTRYLO_PLV_SHIFT)
+#define ENTRYLO_C_SHIFT 4
+#define ENTRYLO_C (_ULCAST_(3) << ENTRYLO_C_SHIFT)
+#define ENTRYLO_G (_ULCAST_(1) << 6)
+#define ENTRYLO_NR (_ULCAST_(1) << 61)
+#define ENTRYLO_NX (_ULCAST_(1) << 62)
+
+/* Values for PageSize register */
+#define PS_4K 0x0000000c
+#define PS_8K 0x0000000d
+#define PS_16K 0x0000000e
+#define PS_32K 0x0000000f
+#define PS_64K 0x00000010
+#define PS_128K 0x00000011
+#define PS_256K 0x00000012
+#define PS_512K 0x00000013
+#define PS_1M 0x00000014
+#define PS_2M 0x00000015
+#define PS_4M 0x00000016
+#define PS_8M 0x00000017
+#define PS_16M 0x00000018
+#define PS_32M 0x00000019
+#define PS_64M 0x0000001a
+#define PS_128M 0x0000001b
+#define PS_256M 0x0000001c
+#define PS_512M 0x0000001d
+#define PS_1G 0x0000001e
+
+#define PS_MASK 0x3f000000
+#define PS_SHIFT 24
+
+/* Default page size for a given kernel configuration */
+#ifdef CONFIG_PAGE_SIZE_4KB
+#define PS_DEFAULT_SIZE PS_4K
+#elif defined(CONFIG_PAGE_SIZE_16KB)
+#define PS_DEFAULT_SIZE PS_16K
+#elif defined(CONFIG_PAGE_SIZE_64KB)
+#define PS_DEFAULT_SIZE PS_64K
+#else
+#error Bad page size configuration!
+#endif
+
+/* Default huge tlb size for a given kernel configuration */
+#ifdef CONFIG_PAGE_SIZE_4KB
+#define PS_HUGE_SIZE PS_1M
+#elif defined(CONFIG_PAGE_SIZE_16KB)
+#define PS_HUGE_SIZE PS_16M
+#elif defined(CONFIG_PAGE_SIZE_64KB)
+#define PS_HUGE_SIZE PS_256M
+#else
+#error Bad page size configuration for hugetlbfs!
+#endif
+
+/* ExStatus.ExcCode */
+#define EXCCODE_RSV 0 /* Reserved */
+#define EXCCODE_TLBL 1 /* TLB miss on a load */
+#define EXCCODE_TLBS 2 /* TLB miss on a store */
+#define EXCCODE_TLBI 3 /* TLB miss on a ifetch */
+#define EXCCODE_TLBM 4 /* TLB modified fault */
+#define EXCCODE_TLBNR 5 /* TLB Read-Inhibit exception */
+#define EXCCODE_TLBNX 6 /* TLB Execution-Inhibit exception */
+#define EXCCODE_TLBPE 7 /* TLB Privilege Error */
+#define EXCCODE_ADE 8 /* Address Error */
+ #define EXSUBCODE_ADEF 0 /* Fetch Instruction */
+ #define EXSUBCODE_ADEM 1 /* Access Memory*/
+#define EXCCODE_ALE 9 /* Unalign Access */
+#define EXCCODE_OOB 10 /* Out of bounds */
+#define EXCCODE_SYS 11 /* System call */
+#define EXCCODE_BP 12 /* Breakpoint */
+#define EXCCODE_INE 13 /* Inst. Not Exist */
+#define EXCCODE_IPE 14 /* Inst. Privileged Error */
+#define EXCCODE_FPDIS 15 /* FPU Disabled */
+#define EXCCODE_LSXDIS 16 /* LSX Disabled */
+#define EXCCODE_LASXDIS 17 /* LASX Disabled */
+#define EXCCODE_FPE 18 /* Floating Point Exception */
+ #define EXCSUBCODE_FPE 0 /* Floating Point Exception */
+ #define EXCSUBCODE_VFPE 1 /* Vector Exception */
+#define EXCCODE_WATCH 19 /* Watch address reference */
+#define EXCCODE_BTDIS 20 /* Binary Trans. Disabled */
+#define EXCCODE_BTE 21 /* Binary Trans. Exception */
+#define EXCCODE_PSI 22 /* Guest Privileged Error */
+#define EXCCODE_HYP 23 /* Hypercall */
+#define EXCCODE_GCM 24 /* Guest CSR modified */
+ #define EXCSUBCODE_GCSC 0 /* Software caused */
+ #define EXCSUBCODE_GCHC 1 /* Hardware caused */
+#define EXCCODE_SE 25 /* Security */
+
+#define EXCCODE_INT_START 64
+#define EXCCODE_SIP0 64
+#define EXCCODE_SIP1 65
+#define EXCCODE_IP0 66
+#define EXCCODE_IP1 67
+#define EXCCODE_IP2 68
+#define EXCCODE_IP3 69
+#define EXCCODE_IP4 70
+#define EXCCODE_IP5 71
+#define EXCCODE_IP6 72
+#define EXCCODE_IP7 73
+#define EXCCODE_PMC 74 /* Performance Counter */
+#define EXCCODE_TIMER 75
+#define EXCCODE_IPI 76
+#define EXCCODE_NMI 77
+#define EXCCODE_INT_END 78
+#define EXCCODE_INT_NUM (EXCCODE_INT_END - EXCCODE_INT_START)
+
+/* FPU register names */
+#define LOONGARCH_FCSR0 $r0
+#define LOONGARCH_FCSR1 $r1
+#define LOONGARCH_FCSR2 $r2
+#define LOONGARCH_FCSR3 $r3
+
+/* FPU Status Register Values */
+#define FPU_CSR_RSVD 0xe0e0fce0
+
+/*
+ * X the exception cause indicator
+ * E the exception enable
+ * S the sticky/flag bit
+ */
+#define FPU_CSR_ALL_X 0x1f000000
+#define FPU_CSR_INV_X 0x10000000
+#define FPU_CSR_DIV_X 0x08000000
+#define FPU_CSR_OVF_X 0x04000000
+#define FPU_CSR_UDF_X 0x02000000
+#define FPU_CSR_INE_X 0x01000000
+
+#define FPU_CSR_ALL_S 0x001f0000
+#define FPU_CSR_INV_S 0x00100000
+#define FPU_CSR_DIV_S 0x00080000
+#define FPU_CSR_OVF_S 0x00040000
+#define FPU_CSR_UDF_S 0x00020000
+#define FPU_CSR_INE_S 0x00010000
+
+#define FPU_CSR_ALL_E 0x0000001f
+#define FPU_CSR_INV_E 0x00000010
+#define FPU_CSR_DIV_E 0x00000008
+#define FPU_CSR_OVF_E 0x00000004
+#define FPU_CSR_UDF_E 0x00000002
+#define FPU_CSR_INE_E 0x00000001
+
+/* Bits 8 and 9 of FPU Status Register specify the rounding mode */
+#define FPU_CSR_RM 0x300
+#define FPU_CSR_RN 0x000 /* nearest */
+#define FPU_CSR_RZ 0x100 /* towards zero */
+#define FPU_CSR_RU 0x200 /* towards +Infinity */
+#define FPU_CSR_RD 0x300 /* towards -Infinity */
+
+#define read_fcsr(source) \
+({ \
+ unsigned int __res; \
+\
+ __asm__ __volatile__( \
+ " movfcsr2gr %0, "__stringify(source)" \n" \
+ : "=r" (__res)); \
+ __res; \
+})
+
+#define write_fcsr(dest, val) \
+do { \
+ __asm__ __volatile__( \
+ " movgr2fcsr %0, "__stringify(dest)" \n" \
+ : : "r" (val)); \
+} while (0)
+
+#endif /* _ASM_LOONGARCH_H */
diff --git a/arch/loongarch/include/asm/loongson.h b/arch/loongarch/include/asm/loongson.h
new file mode 100644
index 000000000000..db4992ae6a6c
--- /dev/null
+++ b/arch/loongarch/include/asm/loongson.h
@@ -0,0 +1,153 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASM_LOONGSON_H
+#define __ASM_LOONGSON_H
+
+#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/irq.h>
+#include <linux/pci.h>
+#include <asm/addrspace.h>
+#include <asm/bootinfo.h>
+
+extern const struct plat_smp_ops loongson3_smp_ops;
+
+#define LOONGSON_REG(x) \
+ (*(volatile u32 *)((char *)TO_UNCACHE(LOONGSON_REG_BASE) + (x)))
+
+#define LOONGSON_LIO_BASE 0x18000000
+#define LOONGSON_LIO_SIZE 0x00100000 /* 1M */
+#define LOONGSON_LIO_TOP (LOONGSON_LIO_BASE+LOONGSON_LIO_SIZE-1)
+
+#define LOONGSON_BOOT_BASE 0x1c000000
+#define LOONGSON_BOOT_SIZE 0x02000000 /* 32M */
+#define LOONGSON_BOOT_TOP (LOONGSON_BOOT_BASE+LOONGSON_BOOT_SIZE-1)
+
+#define LOONGSON_REG_BASE 0x1fe00000
+#define LOONGSON_REG_SIZE 0x00100000 /* 1M */
+#define LOONGSON_REG_TOP (LOONGSON_REG_BASE+LOONGSON_REG_SIZE-1)
+
+/* GPIO Regs - r/w */
+
+#define LOONGSON_GPIODATA LOONGSON_REG(0x11c)
+#define LOONGSON_GPIOIE LOONGSON_REG(0x120)
+#define LOONGSON_REG_GPIO_BASE (LOONGSON_REG_BASE + 0x11c)
+
+#define MAX_PACKAGES 16
+
+/* Chip Config registor of each physical cpu package */
+extern u64 loongson_chipcfg[MAX_PACKAGES];
+#define LOONGSON_CHIPCFG(id) (*(volatile u32 *)(loongson_chipcfg[id]))
+
+/* Chip Temperature registor of each physical cpu package */
+extern u64 loongson_chiptemp[MAX_PACKAGES];
+#define LOONGSON_CHIPTEMP(id) (*(volatile u32 *)(loongson_chiptemp[id]))
+
+/* Freq Control register of each physical cpu package */
+extern u64 loongson_freqctrl[MAX_PACKAGES];
+#define LOONGSON_FREQCTRL(id) (*(volatile u32 *)(loongson_freqctrl[id]))
+
+#define xconf_readl(addr) readl(addr)
+#define xconf_readq(addr) readq(addr)
+
+static inline void xconf_writel(u32 val, volatile void __iomem *addr)
+{
+ asm volatile (
+ " st.w %[v], %[hw], 0 \n"
+ " ld.b $r0, %[hw], 0 \n"
+ :
+ : [hw] "r" (addr), [v] "r" (val)
+ );
+}
+
+static inline void xconf_writeq(u64 val64, volatile void __iomem *addr)
+{
+ asm volatile (
+ " st.d %[v], %[hw], 0 \n"
+ " ld.b $r0, %[hw], 0 \n"
+ :
+ : [hw] "r" (addr), [v] "r" (val64)
+ );
+}
+
+/* ============== LS7A registers =============== */
+#define LS7A_PCH_REG_BASE 0x10000000UL
+/* LPC regs */
+#define LS7A_LPC_REG_BASE (LS7A_PCH_REG_BASE + 0x00002000)
+/* CHIPCFG regs */
+#define LS7A_CHIPCFG_REG_BASE (LS7A_PCH_REG_BASE + 0x00010000)
+/* MISC reg base */
+#define LS7A_MISC_REG_BASE (LS7A_PCH_REG_BASE + 0x00080000)
+/* ACPI regs */
+#define LS7A_ACPI_REG_BASE (LS7A_MISC_REG_BASE + 0x00050000)
+/* RTC regs */
+#define LS7A_RTC_REG_BASE (LS7A_MISC_REG_BASE + 0x00050100)
+
+#define LS7A_DMA_CFG (volatile void *)TO_UNCACHE(LS7A_CHIPCFG_REG_BASE + 0x041c)
+#define LS7A_DMA_NODE_SHF 8
+#define LS7A_DMA_NODE_MASK 0x1F00
+
+#define LS7A_INT_MASK_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x020)
+#define LS7A_INT_EDGE_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x060)
+#define LS7A_INT_CLEAR_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x080)
+#define LS7A_INT_HTMSI_EN_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x040)
+#define LS7A_INT_ROUTE_ENTRY_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x100)
+#define LS7A_INT_HTMSI_VEC_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x200)
+#define LS7A_INT_STATUS_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x3a0)
+#define LS7A_INT_POL_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x3e0)
+#define LS7A_LPC_INT_CTL (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2000)
+#define LS7A_LPC_INT_ENA (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2004)
+#define LS7A_LPC_INT_STS (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2008)
+#define LS7A_LPC_INT_CLR (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x200c)
+#define LS7A_LPC_INT_POL (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2010)
+
+#define LS7A_PMCON_SOC_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x000)
+#define LS7A_PMCON_RESUME_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x004)
+#define LS7A_PMCON_RTC_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x008)
+#define LS7A_PM1_EVT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x00c)
+#define LS7A_PM1_ENA_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x010)
+#define LS7A_PM1_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x014)
+#define LS7A_PM1_TMR_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x018)
+#define LS7A_P_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x01c)
+#define LS7A_GPE0_STS_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x028)
+#define LS7A_GPE0_ENA_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x02c)
+#define LS7A_RST_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x030)
+#define LS7A_WD_SET_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x034)
+#define LS7A_WD_TIMER_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x038)
+#define LS7A_THSENS_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x04c)
+#define LS7A_GEN_RTC_1_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x050)
+#define LS7A_GEN_RTC_2_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x054)
+#define LS7A_DPM_CFG_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x400)
+#define LS7A_DPM_STS_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x404)
+#define LS7A_DPM_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x408)
+
+typedef enum {
+ ACPI_PCI_HOTPLUG_STATUS = 1 << 1,
+ ACPI_CPU_HOTPLUG_STATUS = 1 << 2,
+ ACPI_MEM_HOTPLUG_STATUS = 1 << 3,
+ ACPI_POWERBUTTON_STATUS = 1 << 8,
+ ACPI_RTC_WAKE_STATUS = 1 << 10,
+ ACPI_PCI_WAKE_STATUS = 1 << 14,
+ ACPI_ANY_WAKE_STATUS = 1 << 15,
+} AcpiEventStatusBits;
+
+#define HT1LO_OFFSET 0xe0000000000UL
+
+/* PCI Configuration Space Base */
+#define MCFG_EXT_PCICFG_BASE 0xefe00000000UL
+
+/* REG ACCESS*/
+#define ls7a_readb(addr) (*(volatile unsigned char *)TO_UNCACHE(addr))
+#define ls7a_readw(addr) (*(volatile unsigned short *)TO_UNCACHE(addr))
+#define ls7a_readl(addr) (*(volatile unsigned int *)TO_UNCACHE(addr))
+#define ls7a_readq(addr) (*(volatile unsigned long *)TO_UNCACHE(addr))
+#define ls7a_writeb(val, addr) *(volatile unsigned char *)TO_UNCACHE(addr) = (val)
+#define ls7a_writew(val, addr) *(volatile unsigned short *)TO_UNCACHE(addr) = (val)
+#define ls7a_writel(val, addr) *(volatile unsigned int *)TO_UNCACHE(addr) = (val)
+#define ls7a_writeq(val, addr) *(volatile unsigned long *)TO_UNCACHE(addr) = (val)
+
+#endif /* __ASM_LOONGSON_H */
diff --git a/arch/loongarch/include/asm/regdef.h b/arch/loongarch/include/asm/regdef.h
new file mode 100644
index 000000000000..49a374c2612c
--- /dev/null
+++ b/arch/loongarch/include/asm/regdef.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_REGDEF_H
+#define _ASM_REGDEF_H
+
+#define zero $r0 /* wired zero */
+#define ra $r1 /* return address */
+#define tp $r2
+#define sp $r3 /* stack pointer */
+#define a0 $r4 /* argument registers, a0/a1 reused as v0/v1 for return value */
+#define a1 $r5
+#define a2 $r6
+#define a3 $r7
+#define a4 $r8
+#define a5 $r9
+#define a6 $r10
+#define a7 $r11
+#define t0 $r12 /* caller saved */
+#define t1 $r13
+#define t2 $r14
+#define t3 $r15
+#define t4 $r16
+#define t5 $r17
+#define t6 $r18
+#define t7 $r19
+#define t8 $r20
+#define u0 $r21
+#define fp $r22 /* frame pointer */
+#define s0 $r23 /* callee saved */
+#define s1 $r24
+#define s2 $r25
+#define s3 $r26
+#define s4 $r27
+#define s5 $r28
+#define s6 $r29
+#define s7 $r30
+#define s8 $r31
+
+#endif /* _ASM_REGDEF_H */
--
2.27.0


2022-05-18 09:45:41

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 07/22] LoongArch: Add atomic/locking headers

Add common headers (atomic, bitops, barrier and locking) for basic
LoongArch support.

Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/atomic.h | 358 +++++++++++++++++++++++++++
arch/loongarch/include/asm/barrier.h | 51 ++++
arch/loongarch/include/asm/bitops.h | 33 +++
arch/loongarch/include/asm/bitrev.h | 34 +++
arch/loongarch/include/asm/cmpxchg.h | 121 +++++++++
arch/loongarch/include/asm/local.h | 138 +++++++++++
arch/loongarch/include/asm/percpu.h | 20 ++
7 files changed, 755 insertions(+)
create mode 100644 arch/loongarch/include/asm/atomic.h
create mode 100644 arch/loongarch/include/asm/barrier.h
create mode 100644 arch/loongarch/include/asm/bitops.h
create mode 100644 arch/loongarch/include/asm/bitrev.h
create mode 100644 arch/loongarch/include/asm/cmpxchg.h
create mode 100644 arch/loongarch/include/asm/local.h
create mode 100644 arch/loongarch/include/asm/percpu.h

diff --git a/arch/loongarch/include/asm/atomic.h b/arch/loongarch/include/asm/atomic.h
new file mode 100644
index 000000000000..932352342b12
--- /dev/null
+++ b/arch/loongarch/include/asm/atomic.h
@@ -0,0 +1,358 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Atomic operations.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_ATOMIC_H
+#define _ASM_ATOMIC_H
+
+#include <linux/types.h>
+#include <asm/barrier.h>
+#include <asm/cmpxchg.h>
+#include <asm/compiler.h>
+
+#if __SIZEOF_LONG__ == 4
+#define __LL "ll.w "
+#define __SC "sc.w "
+#define __AMADD "amadd.w "
+#define __AMAND_DB "amand_db.w "
+#define __AMOR_DB "amor_db.w "
+#define __AMXOR_DB "amxor_db.w "
+#elif __SIZEOF_LONG__ == 8
+#define __LL "ll.d "
+#define __SC "sc.d "
+#define __AMADD "amadd.d "
+#define __AMAND_DB "amand_db.d "
+#define __AMOR_DB "amor_db.d "
+#define __AMXOR_DB "amxor_db.d "
+#endif
+
+#define ATOMIC_INIT(i) { (i) }
+
+/*
+ * arch_atomic_read - read atomic variable
+ * @v: pointer of type atomic_t
+ *
+ * Atomically reads the value of @v.
+ */
+#define arch_atomic_read(v) READ_ONCE((v)->counter)
+
+/*
+ * arch_atomic_set - set atomic variable
+ * @v: pointer of type atomic_t
+ * @i: required value
+ *
+ * Atomically sets the value of @v to @i.
+ */
+#define arch_atomic_set(v, i) WRITE_ONCE((v)->counter, (i))
+
+#define ATOMIC_OP(op, I, asm_op) \
+static inline void arch_atomic_##op(int i, atomic_t *v) \
+{ \
+ __asm__ __volatile__( \
+ "am"#asm_op"_db.w" " $zero, %1, %0 \n" \
+ : "+ZB" (v->counter) \
+ : "r" (I) \
+ : "memory"); \
+}
+
+#define ATOMIC_OP_RETURN(op, I, asm_op, c_op) \
+static inline int arch_atomic_##op##_return_relaxed(int i, atomic_t *v) \
+{ \
+ int result; \
+ \
+ __asm__ __volatile__( \
+ "am"#asm_op"_db.w" " %1, %2, %0 \n" \
+ : "+ZB" (v->counter), "=&r" (result) \
+ : "r" (I) \
+ : "memory"); \
+ \
+ return result c_op I; \
+}
+
+#define ATOMIC_FETCH_OP(op, I, asm_op) \
+static inline int arch_atomic_fetch_##op##_relaxed(int i, atomic_t *v) \
+{ \
+ int result; \
+ \
+ __asm__ __volatile__( \
+ "am"#asm_op"_db.w" " %1, %2, %0 \n" \
+ : "+ZB" (v->counter), "=&r" (result) \
+ : "r" (I) \
+ : "memory"); \
+ \
+ return result; \
+}
+
+#define ATOMIC_OPS(op, I, asm_op, c_op) \
+ ATOMIC_OP(op, I, asm_op) \
+ ATOMIC_OP_RETURN(op, I, asm_op, c_op) \
+ ATOMIC_FETCH_OP(op, I, asm_op)
+
+ATOMIC_OPS(add, i, add, +)
+ATOMIC_OPS(sub, -i, add, +)
+
+#define arch_atomic_add_return_relaxed arch_atomic_add_return_relaxed
+#define arch_atomic_sub_return_relaxed arch_atomic_sub_return_relaxed
+#define arch_atomic_fetch_add_relaxed arch_atomic_fetch_add_relaxed
+#define arch_atomic_fetch_sub_relaxed arch_atomic_fetch_sub_relaxed
+
+#undef ATOMIC_OPS
+
+#define ATOMIC_OPS(op, I, asm_op) \
+ ATOMIC_OP(op, I, asm_op) \
+ ATOMIC_FETCH_OP(op, I, asm_op)
+
+ATOMIC_OPS(and, i, and)
+ATOMIC_OPS(or, i, or)
+ATOMIC_OPS(xor, i, xor)
+
+#define arch_atomic_fetch_and_relaxed arch_atomic_fetch_and_relaxed
+#define arch_atomic_fetch_or_relaxed arch_atomic_fetch_or_relaxed
+#define arch_atomic_fetch_xor_relaxed arch_atomic_fetch_xor_relaxed
+
+#undef ATOMIC_OPS
+#undef ATOMIC_FETCH_OP
+#undef ATOMIC_OP_RETURN
+#undef ATOMIC_OP
+
+static inline int arch_atomic_fetch_add_unless(atomic_t *v, int a, int u)
+{
+ int prev, rc;
+
+ __asm__ __volatile__ (
+ "0: ll.w %[p], %[c]\n"
+ " beq %[p], %[u], 1f\n"
+ " add.w %[rc], %[p], %[a]\n"
+ " sc.w %[rc], %[c]\n"
+ " beqz %[rc], 0b\n"
+ " b 2f\n"
+ "1:\n"
+ __WEAK_LLSC_MB
+ "2:\n"
+ : [p]"=&r" (prev), [rc]"=&r" (rc),
+ [c]"=ZB" (v->counter)
+ : [a]"r" (a), [u]"r" (u)
+ : "memory");
+
+ return prev;
+}
+#define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless
+
+/*
+ * arch_atomic_sub_if_positive - conditionally subtract integer from atomic variable
+ * @i: integer value to subtract
+ * @v: pointer of type atomic_t
+ *
+ * Atomically test @v and subtract @i if @v is greater or equal than @i.
+ * The function returns the old value of @v minus @i.
+ */
+static inline int arch_atomic_sub_if_positive(int i, atomic_t *v)
+{
+ int result;
+ int temp;
+
+ if (__builtin_constant_p(i)) {
+ __asm__ __volatile__(
+ "1: ll.w %1, %2 # atomic_sub_if_positive\n"
+ " addi.w %0, %1, %3 \n"
+ " or %1, %0, $zero \n"
+ " blt %0, $zero, 2f \n"
+ " sc.w %1, %2 \n"
+ " beq $zero, %1, 1b \n"
+ "2: \n"
+ : "=&r" (result), "=&r" (temp),
+ "+" GCC_OFF_SMALL_ASM() (v->counter)
+ : "I" (-i));
+ } else {
+ __asm__ __volatile__(
+ "1: ll.w %1, %2 # atomic_sub_if_positive\n"
+ " sub.w %0, %1, %3 \n"
+ " or %1, %0, $zero \n"
+ " blt %0, $zero, 2f \n"
+ " sc.w %1, %2 \n"
+ " beq $zero, %1, 1b \n"
+ "2: \n"
+ : "=&r" (result), "=&r" (temp),
+ "+" GCC_OFF_SMALL_ASM() (v->counter)
+ : "r" (i));
+ }
+
+ return result;
+}
+
+#define arch_atomic_cmpxchg(v, o, n) (arch_cmpxchg(&((v)->counter), (o), (n)))
+#define arch_atomic_xchg(v, new) (arch_xchg(&((v)->counter), (new)))
+
+/*
+ * arch_atomic_dec_if_positive - decrement by 1 if old value positive
+ * @v: pointer of type atomic_t
+ */
+#define arch_atomic_dec_if_positive(v) arch_atomic_sub_if_positive(1, v)
+
+#ifdef CONFIG_64BIT
+
+#define ATOMIC64_INIT(i) { (i) }
+
+/*
+ * arch_atomic64_read - read atomic variable
+ * @v: pointer of type atomic64_t
+ *
+ */
+#define arch_atomic64_read(v) READ_ONCE((v)->counter)
+
+/*
+ * arch_atomic64_set - set atomic variable
+ * @v: pointer of type atomic64_t
+ * @i: required value
+ */
+#define arch_atomic64_set(v, i) WRITE_ONCE((v)->counter, (i))
+
+#define ATOMIC64_OP(op, I, asm_op) \
+static inline void arch_atomic64_##op(long i, atomic64_t *v) \
+{ \
+ __asm__ __volatile__( \
+ "am"#asm_op"_db.d " " $zero, %1, %0 \n" \
+ : "+ZB" (v->counter) \
+ : "r" (I) \
+ : "memory"); \
+}
+
+#define ATOMIC64_OP_RETURN(op, I, asm_op, c_op) \
+static inline long arch_atomic64_##op##_return_relaxed(long i, atomic64_t *v) \
+{ \
+ long result; \
+ __asm__ __volatile__( \
+ "am"#asm_op"_db.d " " %1, %2, %0 \n" \
+ : "+ZB" (v->counter), "=&r" (result) \
+ : "r" (I) \
+ : "memory"); \
+ \
+ return result c_op I; \
+}
+
+#define ATOMIC64_FETCH_OP(op, I, asm_op) \
+static inline long arch_atomic64_fetch_##op##_relaxed(long i, atomic64_t *v) \
+{ \
+ long result; \
+ \
+ __asm__ __volatile__( \
+ "am"#asm_op"_db.d " " %1, %2, %0 \n" \
+ : "+ZB" (v->counter), "=&r" (result) \
+ : "r" (I) \
+ : "memory"); \
+ \
+ return result; \
+}
+
+#define ATOMIC64_OPS(op, I, asm_op, c_op) \
+ ATOMIC64_OP(op, I, asm_op) \
+ ATOMIC64_OP_RETURN(op, I, asm_op, c_op) \
+ ATOMIC64_FETCH_OP(op, I, asm_op)
+
+ATOMIC64_OPS(add, i, add, +)
+ATOMIC64_OPS(sub, -i, add, +)
+
+#define arch_atomic64_add_return_relaxed arch_atomic64_add_return_relaxed
+#define arch_atomic64_sub_return_relaxed arch_atomic64_sub_return_relaxed
+#define arch_atomic64_fetch_add_relaxed arch_atomic64_fetch_add_relaxed
+#define arch_atomic64_fetch_sub_relaxed arch_atomic64_fetch_sub_relaxed
+
+#undef ATOMIC64_OPS
+
+#define ATOMIC64_OPS(op, I, asm_op) \
+ ATOMIC64_OP(op, I, asm_op) \
+ ATOMIC64_FETCH_OP(op, I, asm_op)
+
+ATOMIC64_OPS(and, i, and)
+ATOMIC64_OPS(or, i, or)
+ATOMIC64_OPS(xor, i, xor)
+
+#define arch_atomic64_fetch_and_relaxed arch_atomic64_fetch_and_relaxed
+#define arch_atomic64_fetch_or_relaxed arch_atomic64_fetch_or_relaxed
+#define arch_atomic64_fetch_xor_relaxed arch_atomic64_fetch_xor_relaxed
+
+#undef ATOMIC64_OPS
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP_RETURN
+#undef ATOMIC64_OP
+
+static inline long arch_atomic64_fetch_add_unless(atomic64_t *v, long a, long u)
+{
+ long prev, rc;
+
+ __asm__ __volatile__ (
+ "0: ll.d %[p], %[c]\n"
+ " beq %[p], %[u], 1f\n"
+ " add.d %[rc], %[p], %[a]\n"
+ " sc.d %[rc], %[c]\n"
+ " beqz %[rc], 0b\n"
+ " b 2f\n"
+ "1:\n"
+ __WEAK_LLSC_MB
+ "2:\n"
+ : [p]"=&r" (prev), [rc]"=&r" (rc),
+ [c] "=ZB" (v->counter)
+ : [a]"r" (a), [u]"r" (u)
+ : "memory");
+
+ return prev;
+}
+#define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless
+
+/*
+ * arch_atomic64_sub_if_positive - conditionally subtract integer from atomic variable
+ * @i: integer value to subtract
+ * @v: pointer of type atomic64_t
+ *
+ * Atomically test @v and subtract @i if @v is greater or equal than @i.
+ * The function returns the old value of @v minus @i.
+ */
+static inline long arch_atomic64_sub_if_positive(long i, atomic64_t *v)
+{
+ long result;
+ long temp;
+
+ if (__builtin_constant_p(i)) {
+ __asm__ __volatile__(
+ "1: ll.d %1, %2 # atomic64_sub_if_positive \n"
+ " addi.d %0, %1, %3 \n"
+ " or %1, %0, $zero \n"
+ " blt %0, $zero, 2f \n"
+ " sc.d %1, %2 \n"
+ " beq %1, $zero, 1b \n"
+ "2: \n"
+ : "=&r" (result), "=&r" (temp),
+ "+" GCC_OFF_SMALL_ASM() (v->counter)
+ : "I" (-i));
+ } else {
+ __asm__ __volatile__(
+ "1: ll.d %1, %2 # atomic64_sub_if_positive \n"
+ " sub.d %0, %1, %3 \n"
+ " or %1, %0, $zero \n"
+ " blt %0, $zero, 2f \n"
+ " sc.d %1, %2 \n"
+ " beq %1, $zero, 1b \n"
+ "2: \n"
+ : "=&r" (result), "=&r" (temp),
+ "+" GCC_OFF_SMALL_ASM() (v->counter)
+ : "r" (i));
+ }
+
+ return result;
+}
+
+#define arch_atomic64_cmpxchg(v, o, n) \
+ ((__typeof__((v)->counter))arch_cmpxchg(&((v)->counter), (o), (n)))
+#define arch_atomic64_xchg(v, new) (arch_xchg(&((v)->counter), (new)))
+
+/*
+ * arch_atomic64_dec_if_positive - decrement by 1 if old value positive
+ * @v: pointer of type atomic64_t
+ */
+#define arch_atomic64_dec_if_positive(v) arch_atomic64_sub_if_positive(1, v)
+
+#endif /* CONFIG_64BIT */
+
+#endif /* _ASM_ATOMIC_H */
diff --git a/arch/loongarch/include/asm/barrier.h b/arch/loongarch/include/asm/barrier.h
new file mode 100644
index 000000000000..e57571bcaf4f
--- /dev/null
+++ b/arch/loongarch/include/asm/barrier.h
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_BARRIER_H
+#define __ASM_BARRIER_H
+
+#define __sync() __asm__ __volatile__("dbar 0" : : : "memory")
+
+#define fast_wmb() __sync()
+#define fast_rmb() __sync()
+#define fast_mb() __sync()
+#define fast_iob() __sync()
+#define wbflush() __sync()
+
+#define wmb() fast_wmb()
+#define rmb() fast_rmb()
+#define mb() fast_mb()
+#define iob() fast_iob()
+
+/**
+ * array_index_mask_nospec() - generate a ~0 mask when index < size, 0 otherwise
+ * @index: array element index
+ * @size: number of elements in array
+ *
+ * Returns:
+ * 0 - (@index < @size)
+ */
+#define array_index_mask_nospec array_index_mask_nospec
+static inline unsigned long array_index_mask_nospec(unsigned long index,
+ unsigned long size)
+{
+ unsigned long mask;
+
+ __asm__ __volatile__(
+ "sltu %0, %1, %2\n\t"
+#if (__SIZEOF_LONG__ == 4)
+ "sub.w %0, $r0, %0\n\t"
+#elif (__SIZEOF_LONG__ == 8)
+ "sub.d %0, $r0, %0\n\t"
+#endif
+ : "=r" (mask)
+ : "r" (index), "r" (size)
+ :);
+
+ return mask;
+}
+
+#include <asm-generic/barrier.h>
+
+#endif /* __ASM_BARRIER_H */
diff --git a/arch/loongarch/include/asm/bitops.h b/arch/loongarch/include/asm/bitops.h
new file mode 100644
index 000000000000..69e00f8d8034
--- /dev/null
+++ b/arch/loongarch/include/asm/bitops.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_BITOPS_H
+#define _ASM_BITOPS_H
+
+#include <linux/compiler.h>
+
+#ifndef _LINUX_BITOPS_H
+#error only <linux/bitops.h> can be included directly
+#endif
+
+#include <asm/barrier.h>
+
+#include <asm-generic/bitops/builtin-ffs.h>
+#include <asm-generic/bitops/builtin-fls.h>
+#include <asm-generic/bitops/builtin-__ffs.h>
+#include <asm-generic/bitops/builtin-__fls.h>
+
+#include <asm-generic/bitops/ffz.h>
+#include <asm-generic/bitops/fls64.h>
+
+#include <asm-generic/bitops/sched.h>
+#include <asm-generic/bitops/hweight.h>
+
+#include <asm-generic/bitops/atomic.h>
+#include <asm-generic/bitops/non-atomic.h>
+#include <asm-generic/bitops/lock.h>
+#include <asm-generic/bitops/le.h>
+#include <asm-generic/bitops/ext2-atomic.h>
+
+#endif /* _ASM_BITOPS_H */
diff --git a/arch/loongarch/include/asm/bitrev.h b/arch/loongarch/include/asm/bitrev.h
new file mode 100644
index 000000000000..46f275b9cdf7
--- /dev/null
+++ b/arch/loongarch/include/asm/bitrev.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __LOONGARCH_ASM_BITREV_H__
+#define __LOONGARCH_ASM_BITREV_H__
+
+#include <linux/swab.h>
+
+static __always_inline __attribute_const__ u32 __arch_bitrev32(u32 x)
+{
+ u32 ret;
+
+ asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(__swab32(x)));
+ return ret;
+}
+
+static __always_inline __attribute_const__ u16 __arch_bitrev16(u16 x)
+{
+ u16 ret;
+
+ asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(__swab16(x)));
+ return ret;
+}
+
+static __always_inline __attribute_const__ u8 __arch_bitrev8(u8 x)
+{
+ u8 ret;
+
+ asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(x));
+ return ret;
+}
+
+#endif /* __LOONGARCH_ASM_BITREV_H__ */
diff --git a/arch/loongarch/include/asm/cmpxchg.h b/arch/loongarch/include/asm/cmpxchg.h
new file mode 100644
index 000000000000..699925b8ff27
--- /dev/null
+++ b/arch/loongarch/include/asm/cmpxchg.h
@@ -0,0 +1,121 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_CMPXCHG_H
+#define __ASM_CMPXCHG_H
+
+#include <linux/build_bug.h>
+
+#define __xchg_asm(amswap_db, m, val) \
+({ \
+ __typeof(val) __ret; \
+ \
+ __asm__ __volatile__ ( \
+ " "amswap_db" %1, %z2, %0 \n" \
+ : "+ZB" (*m), "=&r" (__ret) \
+ : "Jr" (val) \
+ : "memory"); \
+ \
+ __ret; \
+})
+
+static inline unsigned long __xchg(volatile void *ptr, unsigned long x,
+ int size)
+{
+ switch (size) {
+ case 4:
+ return __xchg_asm("amswap_db.w", (volatile u32 *)ptr, (u32)x);
+
+ case 8:
+ return __xchg_asm("amswap_db.d", (volatile u64 *)ptr, (u64)x);
+
+ default:
+ BUILD_BUG();
+ }
+
+ return 0;
+}
+
+#define arch_xchg(ptr, x) \
+({ \
+ __typeof__(*(ptr)) __res; \
+ \
+ __res = (__typeof__(*(ptr))) \
+ __xchg((ptr), (unsigned long)(x), sizeof(*(ptr))); \
+ \
+ __res; \
+})
+
+#define __cmpxchg_asm(ld, st, m, old, new) \
+({ \
+ __typeof(old) __ret; \
+ \
+ __asm__ __volatile__( \
+ "1: " ld " %0, %2 # __cmpxchg_asm \n" \
+ " bne %0, %z3, 2f \n" \
+ " or $t0, %z4, $zero \n" \
+ " " st " $t0, %1 \n" \
+ " beq $zero, $t0, 1b \n" \
+ "2: \n" \
+ : "=&r" (__ret), "=ZB"(*m) \
+ : "ZB"(*m), "Jr" (old), "Jr" (new) \
+ : "t0", "memory"); \
+ \
+ __ret; \
+})
+
+static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
+ unsigned long new, unsigned int size)
+{
+ switch (size) {
+ case 4:
+ return __cmpxchg_asm("ll.w", "sc.w", (volatile u32 *)ptr,
+ (u32)old, new);
+
+ case 8:
+ return __cmpxchg_asm("ll.d", "sc.d", (volatile u64 *)ptr,
+ (u64)old, new);
+
+ default:
+ BUILD_BUG();
+ }
+
+ return 0;
+}
+
+#define arch_cmpxchg_local(ptr, old, new) \
+ ((__typeof__(*(ptr))) \
+ __cmpxchg((ptr), \
+ (unsigned long)(__typeof__(*(ptr)))(old), \
+ (unsigned long)(__typeof__(*(ptr)))(new), \
+ sizeof(*(ptr))))
+
+#define arch_cmpxchg(ptr, old, new) \
+({ \
+ __typeof__(*(ptr)) __res; \
+ \
+ __res = arch_cmpxchg_local((ptr), (old), (new)); \
+ \
+ __res; \
+})
+
+#ifdef CONFIG_64BIT
+#define arch_cmpxchg64_local(ptr, o, n) \
+ ({ \
+ BUILD_BUG_ON(sizeof(*(ptr)) != 8); \
+ arch_cmpxchg_local((ptr), (o), (n)); \
+ })
+
+#define arch_cmpxchg64(ptr, o, n) \
+ ({ \
+ BUILD_BUG_ON(sizeof(*(ptr)) != 8); \
+ arch_cmpxchg((ptr), (o), (n)); \
+ })
+#else
+#include <asm-generic/cmpxchg-local.h>
+#define arch_cmpxchg64_local(ptr, o, n) __generic_cmpxchg64_local((ptr), (o), (n))
+#define arch_cmpxchg64(ptr, o, n) arch_cmpxchg64_local((ptr), (o), (n))
+#endif
+
+#endif /* __ASM_CMPXCHG_H */
diff --git a/arch/loongarch/include/asm/local.h b/arch/loongarch/include/asm/local.h
new file mode 100644
index 000000000000..2052a2267337
--- /dev/null
+++ b/arch/loongarch/include/asm/local.h
@@ -0,0 +1,138 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ARCH_LOONGARCH_LOCAL_H
+#define _ARCH_LOONGARCH_LOCAL_H
+
+#include <linux/percpu.h>
+#include <linux/bitops.h>
+#include <linux/atomic.h>
+#include <asm/cmpxchg.h>
+#include <asm/compiler.h>
+
+typedef struct {
+ atomic_long_t a;
+} local_t;
+
+#define LOCAL_INIT(i) { ATOMIC_LONG_INIT(i) }
+
+#define local_read(l) atomic_long_read(&(l)->a)
+#define local_set(l, i) atomic_long_set(&(l)->a, (i))
+
+#define local_add(i, l) atomic_long_add((i), (&(l)->a))
+#define local_sub(i, l) atomic_long_sub((i), (&(l)->a))
+#define local_inc(l) atomic_long_inc(&(l)->a)
+#define local_dec(l) atomic_long_dec(&(l)->a)
+
+/*
+ * Same as above, but return the result value
+ */
+static inline long local_add_return(long i, local_t *l)
+{
+ unsigned long result;
+
+ __asm__ __volatile__(
+ " " __AMADD " %1, %2, %0 \n"
+ : "+ZB" (l->a.counter), "=&r" (result)
+ : "r" (i)
+ : "memory");
+ result = result + i;
+
+ return result;
+}
+
+static inline long local_sub_return(long i, local_t *l)
+{
+ unsigned long result;
+
+ __asm__ __volatile__(
+ " " __AMADD "%1, %2, %0 \n"
+ : "+ZB" (l->a.counter), "=&r" (result)
+ : "r" (-i)
+ : "memory");
+
+ result = result - i;
+
+ return result;
+}
+
+#define local_cmpxchg(l, o, n) \
+ ((long)cmpxchg_local(&((l)->a.counter), (o), (n)))
+#define local_xchg(l, n) (atomic_long_xchg((&(l)->a), (n)))
+
+/**
+ * local_add_unless - add unless the number is a given value
+ * @l: pointer of type local_t
+ * @a: the amount to add to l...
+ * @u: ...unless l is equal to u.
+ *
+ * Atomically adds @a to @l, so long as it was not @u.
+ * Returns non-zero if @l was not @u, and zero otherwise.
+ */
+#define local_add_unless(l, a, u) \
+({ \
+ long c, old; \
+ c = local_read(l); \
+ while (c != (u) && (old = local_cmpxchg((l), c, c + (a))) != c) \
+ c = old; \
+ c != (u); \
+})
+#define local_inc_not_zero(l) local_add_unless((l), 1, 0)
+
+#define local_dec_return(l) local_sub_return(1, (l))
+#define local_inc_return(l) local_add_return(1, (l))
+
+/*
+ * local_sub_and_test - subtract value from variable and test result
+ * @i: integer value to subtract
+ * @l: pointer of type local_t
+ *
+ * Atomically subtracts @i from @l and returns
+ * true if the result is zero, or false for all
+ * other cases.
+ */
+#define local_sub_and_test(i, l) (local_sub_return((i), (l)) == 0)
+
+/*
+ * local_inc_and_test - increment and test
+ * @l: pointer of type local_t
+ *
+ * Atomically increments @l by 1
+ * and returns true if the result is zero, or false for all
+ * other cases.
+ */
+#define local_inc_and_test(l) (local_inc_return(l) == 0)
+
+/*
+ * local_dec_and_test - decrement by 1 and test
+ * @l: pointer of type local_t
+ *
+ * Atomically decrements @l by 1 and
+ * returns true if the result is 0, or false for all other
+ * cases.
+ */
+#define local_dec_and_test(l) (local_sub_return(1, (l)) == 0)
+
+/*
+ * local_add_negative - add and test if negative
+ * @l: pointer of type local_t
+ * @i: integer value to add
+ *
+ * Atomically adds @i to @l and returns true
+ * if the result is negative, or false when
+ * result is greater than or equal to zero.
+ */
+#define local_add_negative(i, l) (local_add_return(i, (l)) < 0)
+
+/* Use these for per-cpu local_t variables: on some archs they are
+ * much more efficient than these naive implementations. Note they take
+ * a variable, not an address.
+ */
+
+#define __local_inc(l) ((l)->a.counter++)
+#define __local_dec(l) ((l)->a.counter++)
+#define __local_add(i, l) ((l)->a.counter += (i))
+#define __local_sub(i, l) ((l)->a.counter -= (i))
+
+#endif /* _ARCH_LOONGARCH_LOCAL_H */
diff --git a/arch/loongarch/include/asm/percpu.h b/arch/loongarch/include/asm/percpu.h
new file mode 100644
index 000000000000..7d5b22ebd834
--- /dev/null
+++ b/arch/loongarch/include/asm/percpu.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_PERCPU_H
+#define __ASM_PERCPU_H
+
+/* Use r21 for fast access */
+register unsigned long __my_cpu_offset __asm__("$r21");
+
+static inline void set_my_cpu_offset(unsigned long off)
+{
+ __my_cpu_offset = off;
+ csr_writeq(off, PERCPU_BASE_KS);
+}
+#define __my_cpu_offset __my_cpu_offset
+
+#include <asm-generic/percpu.h>
+
+#endif /* __ASM_PERCPU_H */
--
2.27.0


2022-05-18 09:58:16

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 12/22] LoongArch: Add memory management

Add memory management support for LoongArch, including: cache and tlb
management, page fault handling and ioremap/mmap support.

Reviewed-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/cache.h | 13 +
arch/loongarch/include/asm/cacheflush.h | 80 ++++
arch/loongarch/include/asm/cacheops.h | 37 ++
arch/loongarch/include/asm/fixmap.h | 13 +
arch/loongarch/include/asm/hugetlb.h | 79 ++++
arch/loongarch/include/asm/page.h | 113 +++++
arch/loongarch/include/asm/pgalloc.h | 103 +++++
arch/loongarch/include/asm/pgtable-bits.h | 131 ++++++
arch/loongarch/include/asm/pgtable.h | 532 ++++++++++++++++++++++
arch/loongarch/include/asm/shmparam.h | 12 +
arch/loongarch/include/asm/sparsemem.h | 23 +
arch/loongarch/include/asm/tlb.h | 180 ++++++++
arch/loongarch/include/asm/tlbflush.h | 35 ++
arch/loongarch/include/asm/vmalloc.h | 4 +
arch/loongarch/mm/cache.c | 140 ++++++
arch/loongarch/mm/extable.c | 22 +
arch/loongarch/mm/fault.c | 261 +++++++++++
arch/loongarch/mm/hugetlbpage.c | 87 ++++
arch/loongarch/mm/init.c | 165 +++++++
arch/loongarch/mm/ioremap.c | 27 ++
arch/loongarch/mm/maccess.c | 10 +
arch/loongarch/mm/mmap.c | 125 +++++
arch/loongarch/mm/page.S | 84 ++++
arch/loongarch/mm/pgtable.c | 130 ++++++
arch/loongarch/mm/tlb.c | 282 ++++++++++++
arch/loongarch/mm/tlbex.S | 477 +++++++++++++++++++
26 files changed, 3165 insertions(+)
create mode 100644 arch/loongarch/include/asm/cache.h
create mode 100644 arch/loongarch/include/asm/cacheflush.h
create mode 100644 arch/loongarch/include/asm/cacheops.h
create mode 100644 arch/loongarch/include/asm/fixmap.h
create mode 100644 arch/loongarch/include/asm/hugetlb.h
create mode 100644 arch/loongarch/include/asm/page.h
create mode 100644 arch/loongarch/include/asm/pgalloc.h
create mode 100644 arch/loongarch/include/asm/pgtable-bits.h
create mode 100644 arch/loongarch/include/asm/pgtable.h
create mode 100644 arch/loongarch/include/asm/shmparam.h
create mode 100644 arch/loongarch/include/asm/sparsemem.h
create mode 100644 arch/loongarch/include/asm/tlb.h
create mode 100644 arch/loongarch/include/asm/tlbflush.h
create mode 100644 arch/loongarch/include/asm/vmalloc.h
create mode 100644 arch/loongarch/mm/cache.c
create mode 100644 arch/loongarch/mm/extable.c
create mode 100644 arch/loongarch/mm/fault.c
create mode 100644 arch/loongarch/mm/hugetlbpage.c
create mode 100644 arch/loongarch/mm/init.c
create mode 100644 arch/loongarch/mm/ioremap.c
create mode 100644 arch/loongarch/mm/maccess.c
create mode 100644 arch/loongarch/mm/mmap.c
create mode 100644 arch/loongarch/mm/page.S
create mode 100644 arch/loongarch/mm/pgtable.c
create mode 100644 arch/loongarch/mm/tlb.c
create mode 100644 arch/loongarch/mm/tlbex.S

diff --git a/arch/loongarch/include/asm/cache.h b/arch/loongarch/include/asm/cache.h
new file mode 100644
index 000000000000..1b6d09617199
--- /dev/null
+++ b/arch/loongarch/include/asm/cache.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_CACHE_H
+#define _ASM_CACHE_H
+
+#define L1_CACHE_SHIFT CONFIG_L1_CACHE_SHIFT
+#define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT)
+
+#define __read_mostly __section(".data..read_mostly")
+
+#endif /* _ASM_CACHE_H */
diff --git a/arch/loongarch/include/asm/cacheflush.h b/arch/loongarch/include/asm/cacheflush.h
new file mode 100644
index 000000000000..670900141b7c
--- /dev/null
+++ b/arch/loongarch/include/asm/cacheflush.h
@@ -0,0 +1,80 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_CACHEFLUSH_H
+#define _ASM_CACHEFLUSH_H
+
+#include <linux/mm.h>
+#include <asm/cpu-features.h>
+#include <asm/cacheops.h>
+
+extern void local_flush_icache_range(unsigned long start, unsigned long end);
+
+#define flush_icache_range local_flush_icache_range
+#define flush_icache_user_range local_flush_icache_range
+
+#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
+
+#define flush_cache_all() do { } while (0)
+#define flush_cache_mm(mm) do { } while (0)
+#define flush_cache_dup_mm(mm) do { } while (0)
+#define flush_cache_range(vma, start, end) do { } while (0)
+#define flush_cache_page(vma, vmaddr, pfn) do { } while (0)
+#define flush_cache_vmap(start, end) do { } while (0)
+#define flush_cache_vunmap(start, end) do { } while (0)
+#define flush_icache_page(vma, page) do { } while (0)
+#define flush_icache_user_page(vma, page, addr, len) do { } while (0)
+#define flush_dcache_page(page) do { } while (0)
+#define flush_dcache_mmap_lock(mapping) do { } while (0)
+#define flush_dcache_mmap_unlock(mapping) do { } while (0)
+
+#define cache_op(op, addr) \
+ __asm__ __volatile__( \
+ " cacop %0, %1 \n" \
+ : \
+ : "i" (op), "ZC" (*(unsigned char *)(addr)))
+
+static inline void flush_icache_line_indexed(unsigned long addr)
+{
+ cache_op(Index_Invalidate_I, addr);
+}
+
+static inline void flush_dcache_line_indexed(unsigned long addr)
+{
+ cache_op(Index_Writeback_Inv_D, addr);
+}
+
+static inline void flush_vcache_line_indexed(unsigned long addr)
+{
+ cache_op(Index_Writeback_Inv_V, addr);
+}
+
+static inline void flush_scache_line_indexed(unsigned long addr)
+{
+ cache_op(Index_Writeback_Inv_S, addr);
+}
+
+static inline void flush_icache_line(unsigned long addr)
+{
+ cache_op(Hit_Invalidate_I, addr);
+}
+
+static inline void flush_dcache_line(unsigned long addr)
+{
+ cache_op(Hit_Writeback_Inv_D, addr);
+}
+
+static inline void flush_vcache_line(unsigned long addr)
+{
+ cache_op(Hit_Writeback_Inv_V, addr);
+}
+
+static inline void flush_scache_line(unsigned long addr)
+{
+ cache_op(Hit_Writeback_Inv_S, addr);
+}
+
+#include <asm-generic/cacheflush.h>
+
+#endif /* _ASM_CACHEFLUSH_H */
diff --git a/arch/loongarch/include/asm/cacheops.h b/arch/loongarch/include/asm/cacheops.h
new file mode 100644
index 000000000000..dc280efecebd
--- /dev/null
+++ b/arch/loongarch/include/asm/cacheops.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Cache operations for the cache instruction.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_CACHEOPS_H
+#define __ASM_CACHEOPS_H
+
+/*
+ * Most cache ops are split into a 2 bit field identifying the cache, and a 3
+ * bit field identifying the cache operation.
+ */
+#define CacheOp_Cache 0x03
+#define CacheOp_Op 0x1c
+
+#define Cache_I 0x00
+#define Cache_D 0x01
+#define Cache_V 0x02
+#define Cache_S 0x03
+
+#define Index_Invalidate 0x08
+#define Index_Writeback_Inv 0x08
+#define Hit_Invalidate 0x10
+#define Hit_Writeback_Inv 0x10
+#define CacheOp_User_Defined 0x18
+
+#define Index_Invalidate_I (Cache_I | Index_Invalidate)
+#define Index_Writeback_Inv_D (Cache_D | Index_Writeback_Inv)
+#define Index_Writeback_Inv_V (Cache_V | Index_Writeback_Inv)
+#define Index_Writeback_Inv_S (Cache_S | Index_Writeback_Inv)
+#define Hit_Invalidate_I (Cache_I | Hit_Invalidate)
+#define Hit_Writeback_Inv_D (Cache_D | Hit_Writeback_Inv)
+#define Hit_Writeback_Inv_V (Cache_V | Hit_Writeback_Inv)
+#define Hit_Writeback_Inv_S (Cache_S | Hit_Writeback_Inv)
+
+#endif /* __ASM_CACHEOPS_H */
diff --git a/arch/loongarch/include/asm/fixmap.h b/arch/loongarch/include/asm/fixmap.h
new file mode 100644
index 000000000000..b3541dfa2013
--- /dev/null
+++ b/arch/loongarch/include/asm/fixmap.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * fixmap.h: compile-time virtual memory allocation
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef _ASM_FIXMAP_H
+#define _ASM_FIXMAP_H
+
+#define NR_FIX_BTMAPS 64
+
+#endif
diff --git a/arch/loongarch/include/asm/hugetlb.h b/arch/loongarch/include/asm/hugetlb.h
new file mode 100644
index 000000000000..960ee06d7ffd
--- /dev/null
+++ b/arch/loongarch/include/asm/hugetlb.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASM_HUGETLB_H
+#define __ASM_HUGETLB_H
+
+#include <asm/page.h>
+
+uint64_t pmd_to_entrylo(unsigned long pmd_val);
+
+#define __HAVE_ARCH_PREPARE_HUGEPAGE_RANGE
+static inline int prepare_hugepage_range(struct file *file,
+ unsigned long addr,
+ unsigned long len)
+{
+ unsigned long task_size = STACK_TOP;
+ struct hstate *h = hstate_file(file);
+
+ if (len & ~huge_page_mask(h))
+ return -EINVAL;
+ if (addr & ~huge_page_mask(h))
+ return -EINVAL;
+ if (len > task_size)
+ return -ENOMEM;
+ if (task_size - len < addr)
+ return -EINVAL;
+ return 0;
+}
+
+#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR
+static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep)
+{
+ pte_t clear;
+ pte_t pte = *ptep;
+
+ pte_val(clear) = (unsigned long)invalid_pte_table;
+ set_pte_at(mm, addr, ptep, clear);
+ return pte;
+}
+
+#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH
+static inline void huge_ptep_clear_flush(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep)
+{
+ flush_tlb_page(vma, addr & huge_page_mask(hstate_vma(vma)));
+}
+
+#define __HAVE_ARCH_HUGE_PTE_NONE
+static inline int huge_pte_none(pte_t pte)
+{
+ unsigned long val = pte_val(pte) & ~_PAGE_GLOBAL;
+ return !val || (val == (unsigned long)invalid_pte_table);
+}
+
+#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS
+static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma,
+ unsigned long addr,
+ pte_t *ptep, pte_t pte,
+ int dirty)
+{
+ int changed = !pte_same(*ptep, pte);
+
+ if (changed) {
+ set_pte_at(vma->vm_mm, addr, ptep, pte);
+ /*
+ * There could be some standard sized pages in there,
+ * get them all.
+ */
+ flush_tlb_range(vma, addr, addr + HPAGE_SIZE);
+ }
+ return changed;
+}
+
+#include <asm-generic/hugetlb.h>
+
+#endif /* __ASM_HUGETLB_H */
diff --git a/arch/loongarch/include/asm/page.h b/arch/loongarch/include/asm/page.h
new file mode 100644
index 000000000000..5bd2330d9576
--- /dev/null
+++ b/arch/loongarch/include/asm/page.h
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_PAGE_H
+#define _ASM_PAGE_H
+
+#include <linux/const.h>
+
+/*
+ * PAGE_SHIFT determines the page size
+ */
+#ifdef CONFIG_PAGE_SIZE_4KB
+#define PAGE_SHIFT 12
+#endif
+#ifdef CONFIG_PAGE_SIZE_16KB
+#define PAGE_SHIFT 14
+#endif
+#ifdef CONFIG_PAGE_SIZE_64KB
+#define PAGE_SHIFT 16
+#endif
+#define PAGE_SIZE (_AC(1, UL) << PAGE_SHIFT)
+#define PAGE_MASK (~(PAGE_SIZE - 1))
+
+#define HPAGE_SHIFT (PAGE_SHIFT + PAGE_SHIFT - 3)
+#define HPAGE_SIZE (_AC(1, UL) << HPAGE_SHIFT)
+#define HPAGE_MASK (~(HPAGE_SIZE - 1))
+#define HUGETLB_PAGE_ORDER (HPAGE_SHIFT - PAGE_SHIFT)
+
+#ifndef __ASSEMBLY__
+
+#include <linux/kernel.h>
+#include <linux/pfn.h>
+
+/*
+ * It's normally defined only for FLATMEM config but it's
+ * used in our early mem init code for all memory models.
+ * So always define it.
+ */
+#define ARCH_PFN_OFFSET PFN_UP(PHYS_OFFSET)
+
+extern void clear_page(void *page);
+extern void copy_page(void *to, void *from);
+
+#define clear_user_page(page, vaddr, pg) clear_page(page)
+#define copy_user_page(to, from, vaddr, pg) copy_page(to, from)
+
+extern unsigned long shm_align_mask;
+
+struct page;
+struct vm_area_struct;
+void copy_user_highpage(struct page *to, struct page *from,
+ unsigned long vaddr, struct vm_area_struct *vma);
+
+#define __HAVE_ARCH_COPY_USER_HIGHPAGE
+
+typedef struct { unsigned long pte; } pte_t;
+#define pte_val(x) ((x).pte)
+#define __pte(x) ((pte_t) { (x) })
+typedef struct page *pgtable_t;
+
+typedef struct { unsigned long pgd; } pgd_t;
+#define pgd_val(x) ((x).pgd)
+#define __pgd(x) ((pgd_t) { (x) })
+
+/*
+ * Manipulate page protection bits
+ */
+typedef struct { unsigned long pgprot; } pgprot_t;
+#define pgprot_val(x) ((x).pgprot)
+#define __pgprot(x) ((pgprot_t) { (x) })
+#define pte_pgprot(x) __pgprot(pte_val(x) & ~_PFN_MASK)
+
+#define ptep_buddy(x) ((pte_t *)((unsigned long)(x) ^ sizeof(pte_t)))
+
+/*
+ * __pa()/__va() should be used only during mem init.
+ */
+#define __pa(x) PHYSADDR(x)
+#define __va(x) ((void *)((unsigned long)(x) + PAGE_OFFSET - PHYS_OFFSET))
+
+#define pfn_to_kaddr(pfn) __va((pfn) << PAGE_SHIFT)
+
+#ifdef CONFIG_FLATMEM
+
+static inline int pfn_valid(unsigned long pfn)
+{
+ /* avoid <linux/mm.h> include hell */
+ extern unsigned long max_mapnr;
+ unsigned long pfn_offset = ARCH_PFN_OFFSET;
+
+ return pfn >= pfn_offset && pfn < max_mapnr;
+}
+
+#endif
+
+#define virt_to_pfn(kaddr) PFN_DOWN(virt_to_phys((void *)(kaddr)))
+#define virt_to_page(kaddr) pfn_to_page(virt_to_pfn(kaddr))
+
+extern int __virt_addr_valid(volatile void *kaddr);
+#define virt_addr_valid(kaddr) __virt_addr_valid((volatile void *)(kaddr))
+
+#define VM_DATA_DEFAULT_FLAGS \
+ (VM_READ | VM_WRITE | \
+ ((current->personality & READ_IMPLIES_EXEC) ? VM_EXEC : 0) | \
+ VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
+
+#include <asm-generic/memory_model.h>
+#include <asm-generic/getorder.h>
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* _ASM_PAGE_H */
diff --git a/arch/loongarch/include/asm/pgalloc.h b/arch/loongarch/include/asm/pgalloc.h
new file mode 100644
index 000000000000..b0a57b25c131
--- /dev/null
+++ b/arch/loongarch/include/asm/pgalloc.h
@@ -0,0 +1,103 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_PGALLOC_H
+#define _ASM_PGALLOC_H
+
+#include <linux/mm.h>
+#include <linux/sched.h>
+
+#define __HAVE_ARCH_PMD_ALLOC_ONE
+#define __HAVE_ARCH_PUD_ALLOC_ONE
+#include <asm-generic/pgalloc.h>
+
+static inline void pmd_populate_kernel(struct mm_struct *mm,
+ pmd_t *pmd, pte_t *pte)
+{
+ set_pmd(pmd, __pmd((unsigned long)pte));
+}
+
+static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd, pgtable_t pte)
+{
+ set_pmd(pmd, __pmd((unsigned long)page_address(pte)));
+}
+
+#ifndef __PAGETABLE_PMD_FOLDED
+
+static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
+{
+ set_pud(pud, __pud((unsigned long)pmd));
+}
+#endif
+
+#ifndef __PAGETABLE_PUD_FOLDED
+
+static inline void p4d_populate(struct mm_struct *mm, p4d_t *p4d, pud_t *pud)
+{
+ set_p4d(p4d, __p4d((unsigned long)pud));
+}
+
+#endif /* __PAGETABLE_PUD_FOLDED */
+
+extern void pagetable_init(void);
+
+/*
+ * Initialize a new pmd table with invalid pointers.
+ */
+extern void pmd_init(unsigned long page, unsigned long pagetable);
+
+/*
+ * Initialize a new pgd / pmd table with invalid pointers.
+ */
+extern void pgd_init(unsigned long page);
+extern pgd_t *pgd_alloc(struct mm_struct *mm);
+
+#define __pte_free_tlb(tlb, pte, address) \
+do { \
+ pgtable_pte_page_dtor(pte); \
+ tlb_remove_page((tlb), pte); \
+} while (0)
+
+#ifndef __PAGETABLE_PMD_FOLDED
+
+static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
+{
+ pmd_t *pmd;
+ struct page *pg;
+
+ pg = alloc_pages(GFP_KERNEL_ACCOUNT, PMD_ORDER);
+ if (!pg)
+ return NULL;
+
+ if (!pgtable_pmd_page_ctor(pg)) {
+ __free_pages(pg, PMD_ORDER);
+ return NULL;
+ }
+
+ pmd = (pmd_t *)page_address(pg);
+ pmd_init((unsigned long)pmd, (unsigned long)invalid_pte_table);
+ return pmd;
+}
+
+#define __pmd_free_tlb(tlb, x, addr) pmd_free((tlb)->mm, x)
+
+#endif
+
+#ifndef __PAGETABLE_PUD_FOLDED
+
+static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long address)
+{
+ pud_t *pud;
+
+ pud = (pud_t *) __get_free_pages(GFP_KERNEL, PUD_ORDER);
+ if (pud)
+ pud_init((unsigned long)pud, (unsigned long)invalid_pmd_table);
+ return pud;
+}
+
+#define __pud_free_tlb(tlb, x, addr) pud_free((tlb)->mm, x)
+
+#endif /* __PAGETABLE_PUD_FOLDED */
+
+#endif /* _ASM_PGALLOC_H */
diff --git a/arch/loongarch/include/asm/pgtable-bits.h b/arch/loongarch/include/asm/pgtable-bits.h
new file mode 100644
index 000000000000..3badd112d9ab
--- /dev/null
+++ b/arch/loongarch/include/asm/pgtable-bits.h
@@ -0,0 +1,131 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_PGTABLE_BITS_H
+#define _ASM_PGTABLE_BITS_H
+
+/* Page table bits */
+#define _PAGE_VALID_SHIFT 0
+#define _PAGE_ACCESSED_SHIFT 0 /* Reuse Valid for Accessed */
+#define _PAGE_DIRTY_SHIFT 1
+#define _PAGE_PLV_SHIFT 2 /* 2~3, two bits */
+#define _CACHE_SHIFT 4 /* 4~5, two bits */
+#define _PAGE_GLOBAL_SHIFT 6
+#define _PAGE_HUGE_SHIFT 6 /* HUGE is a PMD bit */
+#define _PAGE_PRESENT_SHIFT 7
+#define _PAGE_WRITE_SHIFT 8
+#define _PAGE_MODIFIED_SHIFT 9
+#define _PAGE_PROTNONE_SHIFT 10
+#define _PAGE_SPECIAL_SHIFT 11
+#define _PAGE_HGLOBAL_SHIFT 12 /* HGlobal is a PMD bit */
+#define _PAGE_PFN_SHIFT 12
+#define _PAGE_PFN_END_SHIFT 48
+#define _PAGE_NO_READ_SHIFT 61
+#define _PAGE_NO_EXEC_SHIFT 62
+#define _PAGE_RPLV_SHIFT 63
+
+/* Used by software */
+#define _PAGE_PRESENT (_ULCAST_(1) << _PAGE_PRESENT_SHIFT)
+#define _PAGE_WRITE (_ULCAST_(1) << _PAGE_WRITE_SHIFT)
+#define _PAGE_ACCESSED (_ULCAST_(1) << _PAGE_ACCESSED_SHIFT)
+#define _PAGE_MODIFIED (_ULCAST_(1) << _PAGE_MODIFIED_SHIFT)
+#define _PAGE_PROTNONE (_ULCAST_(1) << _PAGE_PROTNONE_SHIFT)
+#define _PAGE_SPECIAL (_ULCAST_(1) << _PAGE_SPECIAL_SHIFT)
+
+/* Used by TLB hardware (placed in EntryLo*) */
+#define _PAGE_VALID (_ULCAST_(1) << _PAGE_VALID_SHIFT)
+#define _PAGE_DIRTY (_ULCAST_(1) << _PAGE_DIRTY_SHIFT)
+#define _PAGE_PLV (_ULCAST_(3) << _PAGE_PLV_SHIFT)
+#define _PAGE_GLOBAL (_ULCAST_(1) << _PAGE_GLOBAL_SHIFT)
+#define _PAGE_HUGE (_ULCAST_(1) << _PAGE_HUGE_SHIFT)
+#define _PAGE_HGLOBAL (_ULCAST_(1) << _PAGE_HGLOBAL_SHIFT)
+#define _PAGE_NO_READ (_ULCAST_(1) << _PAGE_NO_READ_SHIFT)
+#define _PAGE_NO_EXEC (_ULCAST_(1) << _PAGE_NO_EXEC_SHIFT)
+#define _PAGE_RPLV (_ULCAST_(1) << _PAGE_RPLV_SHIFT)
+#define _CACHE_MASK (_ULCAST_(3) << _CACHE_SHIFT)
+#define _PFN_SHIFT (PAGE_SHIFT - 12 + _PAGE_PFN_SHIFT)
+
+#define _PAGE_USER (PLV_USER << _PAGE_PLV_SHIFT)
+#define _PAGE_KERN (PLV_KERN << _PAGE_PLV_SHIFT)
+
+#define _PFN_MASK (~((_ULCAST_(1) << (_PFN_SHIFT)) - 1) & \
+ ((_ULCAST_(1) << (_PAGE_PFN_END_SHIFT)) - 1))
+
+/*
+ * Cache attributes
+ */
+#ifndef _CACHE_SUC
+#define _CACHE_SUC (0<<_CACHE_SHIFT) /* Strong-ordered UnCached */
+#endif
+#ifndef _CACHE_CC
+#define _CACHE_CC (1<<_CACHE_SHIFT) /* Coherent Cached */
+#endif
+#ifndef _CACHE_WUC
+#define _CACHE_WUC (2<<_CACHE_SHIFT) /* Weak-ordered UnCached */
+#endif
+
+#define __READABLE (_PAGE_VALID)
+#define __WRITEABLE (_PAGE_DIRTY | _PAGE_WRITE)
+
+#define _PAGE_CHG_MASK (_PAGE_MODIFIED | _PAGE_SPECIAL | _PFN_MASK | _CACHE_MASK | _PAGE_PLV)
+#define _HPAGE_CHG_MASK (_PAGE_MODIFIED | _PAGE_SPECIAL | _PFN_MASK | _CACHE_MASK | _PAGE_PLV | _PAGE_HUGE)
+
+#define PAGE_NONE __pgprot(_PAGE_PROTNONE | _PAGE_NO_READ | \
+ _PAGE_USER | _CACHE_CC)
+#define PAGE_SHARED __pgprot(_PAGE_PRESENT | _PAGE_WRITE | \
+ _PAGE_USER | _CACHE_CC)
+#define PAGE_READONLY __pgprot(_PAGE_PRESENT | _PAGE_USER | _CACHE_CC)
+
+#define PAGE_KERNEL __pgprot(_PAGE_PRESENT | __READABLE | __WRITEABLE | \
+ _PAGE_GLOBAL | _PAGE_KERN | _CACHE_CC)
+#define PAGE_KERNEL_SUC __pgprot(_PAGE_PRESENT | __READABLE | __WRITEABLE | \
+ _PAGE_GLOBAL | _PAGE_KERN | _CACHE_SUC)
+#define PAGE_KERNEL_WUC __pgprot(_PAGE_PRESENT | __READABLE | __WRITEABLE | \
+ _PAGE_GLOBAL | _PAGE_KERN | _CACHE_WUC)
+
+#define __P000 __pgprot(_CACHE_CC | _PAGE_USER | _PAGE_PROTNONE | _PAGE_NO_EXEC | _PAGE_NO_READ)
+#define __P001 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC)
+#define __P010 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC)
+#define __P011 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC)
+#define __P100 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+#define __P101 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+#define __P110 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+#define __P111 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+
+#define __S000 __pgprot(_CACHE_CC | _PAGE_USER | _PAGE_PROTNONE | _PAGE_NO_EXEC | _PAGE_NO_READ)
+#define __S001 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC)
+#define __S010 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC | _PAGE_WRITE)
+#define __S011 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_NO_EXEC | _PAGE_WRITE)
+#define __S100 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+#define __S101 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT)
+#define __S110 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_WRITE)
+#define __S111 __pgprot(_CACHE_CC | _PAGE_VALID | _PAGE_USER | _PAGE_PRESENT | _PAGE_WRITE)
+
+#ifndef __ASSEMBLY__
+
+#define pgprot_noncached pgprot_noncached
+
+static inline pgprot_t pgprot_noncached(pgprot_t _prot)
+{
+ unsigned long prot = pgprot_val(_prot);
+
+ prot = (prot & ~_CACHE_MASK) | _CACHE_SUC;
+
+ return __pgprot(prot);
+}
+
+#define pgprot_writecombine pgprot_writecombine
+
+static inline pgprot_t pgprot_writecombine(pgprot_t _prot)
+{
+ unsigned long prot = pgprot_val(_prot);
+
+ prot = (prot & ~_CACHE_MASK) | _CACHE_WUC;
+
+ return __pgprot(prot);
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* _ASM_PGTABLE_BITS_H */
diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h
new file mode 100644
index 000000000000..8920dd8b297b
--- /dev/null
+++ b/arch/loongarch/include/asm/pgtable.h
@@ -0,0 +1,532 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1994, 95, 96, 97, 98, 99, 2000, 2003 Ralf Baechle
+ * Copyright (C) 1999, 2000, 2001 Silicon Graphics, Inc.
+ */
+#ifndef _ASM_PGTABLE_H
+#define _ASM_PGTABLE_H
+
+#include <linux/compiler.h>
+#include <asm/addrspace.h>
+#include <asm/pgtable-bits.h>
+
+#if CONFIG_PGTABLE_LEVELS == 2
+#include <asm-generic/pgtable-nopmd.h>
+#elif CONFIG_PGTABLE_LEVELS == 3
+#include <asm-generic/pgtable-nopud.h>
+#else
+#include <asm-generic/pgtable-nop4d.h>
+#endif
+
+#define PGD_ORDER 0
+#define PUD_ORDER 0
+#define PMD_ORDER 0
+#define PTE_ORDER 0
+
+#if CONFIG_PGTABLE_LEVELS == 2
+#define PGDIR_SHIFT (PAGE_SHIFT + (PAGE_SHIFT + PTE_ORDER - 3))
+#elif CONFIG_PGTABLE_LEVELS == 3
+#define PMD_SHIFT (PAGE_SHIFT + (PAGE_SHIFT + PTE_ORDER - 3))
+#define PMD_SIZE (1UL << PMD_SHIFT)
+#define PMD_MASK (~(PMD_SIZE-1))
+#define PGDIR_SHIFT (PMD_SHIFT + (PAGE_SHIFT + PMD_ORDER - 3))
+#elif CONFIG_PGTABLE_LEVELS == 4
+#define PMD_SHIFT (PAGE_SHIFT + (PAGE_SHIFT + PTE_ORDER - 3))
+#define PMD_SIZE (1UL << PMD_SHIFT)
+#define PMD_MASK (~(PMD_SIZE-1))
+#define PUD_SHIFT (PMD_SHIFT + (PAGE_SHIFT + PMD_ORDER - 3))
+#define PUD_SIZE (1UL << PUD_SHIFT)
+#define PUD_MASK (~(PUD_SIZE-1))
+#define PGDIR_SHIFT (PUD_SHIFT + (PAGE_SHIFT + PUD_ORDER - 3))
+#endif
+
+#define PGDIR_SIZE (1UL << PGDIR_SHIFT)
+#define PGDIR_MASK (~(PGDIR_SIZE-1))
+
+#define VA_BITS (PGDIR_SHIFT + (PAGE_SHIFT + PGD_ORDER - 3))
+
+#define PTRS_PER_PGD ((PAGE_SIZE << PGD_ORDER) >> 3)
+#if CONFIG_PGTABLE_LEVELS > 3
+#define PTRS_PER_PUD ((PAGE_SIZE << PUD_ORDER) >> 3)
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+#define PTRS_PER_PMD ((PAGE_SIZE << PMD_ORDER) >> 3)
+#endif
+#define PTRS_PER_PTE ((PAGE_SIZE << PTE_ORDER) >> 3)
+
+#define USER_PTRS_PER_PGD ((TASK_SIZE64 / PGDIR_SIZE)?(TASK_SIZE64 / PGDIR_SIZE):1)
+
+#ifndef __ASSEMBLY__
+
+#include <linux/mm_types.h>
+#include <linux/mmzone.h>
+#include <asm/fixmap.h>
+#include <asm/io.h>
+
+struct mm_struct;
+struct vm_area_struct;
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero; used
+ * for zero-mapped memory areas etc..
+ */
+
+extern unsigned long empty_zero_page;
+extern unsigned long zero_page_mask;
+
+#define ZERO_PAGE(vaddr) \
+ (virt_to_page((void *)(empty_zero_page + (((unsigned long)(vaddr)) & zero_page_mask))))
+#define __HAVE_COLOR_ZERO_PAGE
+
+/*
+ * TLB refill handlers may also map the vmalloc area into xkvrange.
+ * Avoid the first couple of pages so NULL pointer dereferences will
+ * still reliably trap.
+ */
+#define MODULES_VADDR (vm_map_base + PCI_IOSIZE + (2 * PAGE_SIZE))
+#define MODULES_END (MODULES_VADDR + SZ_256M)
+
+#define VMALLOC_START MODULES_END
+#define VMALLOC_END \
+ (vm_map_base + \
+ min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE, (1UL << cpu_vabits)) - PMD_SIZE)
+
+#define pte_ERROR(e) \
+ pr_err("%s:%d: bad pte %016lx.\n", __FILE__, __LINE__, pte_val(e))
+#ifndef __PAGETABLE_PMD_FOLDED
+#define pmd_ERROR(e) \
+ pr_err("%s:%d: bad pmd %016lx.\n", __FILE__, __LINE__, pmd_val(e))
+#endif
+#ifndef __PAGETABLE_PUD_FOLDED
+#define pud_ERROR(e) \
+ pr_err("%s:%d: bad pud %016lx.\n", __FILE__, __LINE__, pud_val(e))
+#endif
+#define pgd_ERROR(e) \
+ pr_err("%s:%d: bad pgd %016lx.\n", __FILE__, __LINE__, pgd_val(e))
+
+extern pte_t invalid_pte_table[PTRS_PER_PTE];
+
+#ifndef __PAGETABLE_PUD_FOLDED
+
+typedef struct { unsigned long pud; } pud_t;
+#define pud_val(x) ((x).pud)
+#define __pud(x) ((pud_t) { (x) })
+
+extern pud_t invalid_pud_table[PTRS_PER_PUD];
+
+/*
+ * Empty pgd/p4d entries point to the invalid_pud_table.
+ */
+static inline int p4d_none(p4d_t p4d)
+{
+ return p4d_val(p4d) == (unsigned long)invalid_pud_table;
+}
+
+static inline int p4d_bad(p4d_t p4d)
+{
+ return p4d_val(p4d) & ~PAGE_MASK;
+}
+
+static inline int p4d_present(p4d_t p4d)
+{
+ return p4d_val(p4d) != (unsigned long)invalid_pud_table;
+}
+
+static inline void p4d_clear(p4d_t *p4dp)
+{
+ p4d_val(*p4dp) = (unsigned long)invalid_pud_table;
+}
+
+static inline pud_t *p4d_pgtable(p4d_t p4d)
+{
+ return (pud_t *)p4d_val(p4d);
+}
+
+static inline void set_p4d(p4d_t *p4d, p4d_t p4dval)
+{
+ *p4d = p4dval;
+}
+
+#define p4d_phys(p4d) virt_to_phys((void *)p4d_val(p4d))
+#define p4d_page(p4d) (pfn_to_page(p4d_phys(p4d) >> PAGE_SHIFT))
+
+#endif
+
+#ifndef __PAGETABLE_PMD_FOLDED
+
+typedef struct { unsigned long pmd; } pmd_t;
+#define pmd_val(x) ((x).pmd)
+#define __pmd(x) ((pmd_t) { (x) })
+
+extern pmd_t invalid_pmd_table[PTRS_PER_PMD];
+
+/*
+ * Empty pud entries point to the invalid_pmd_table.
+ */
+static inline int pud_none(pud_t pud)
+{
+ return pud_val(pud) == (unsigned long)invalid_pmd_table;
+}
+
+static inline int pud_bad(pud_t pud)
+{
+ return pud_val(pud) & ~PAGE_MASK;
+}
+
+static inline int pud_present(pud_t pud)
+{
+ return pud_val(pud) != (unsigned long)invalid_pmd_table;
+}
+
+static inline void pud_clear(pud_t *pudp)
+{
+ pud_val(*pudp) = ((unsigned long)invalid_pmd_table);
+}
+
+static inline pmd_t *pud_pgtable(pud_t pud)
+{
+ return (pmd_t *)pud_val(pud);
+}
+
+#define set_pud(pudptr, pudval) do { *(pudptr) = (pudval); } while (0)
+
+#define pud_phys(pud) virt_to_phys((void *)pud_val(pud))
+#define pud_page(pud) (pfn_to_page(pud_phys(pud) >> PAGE_SHIFT))
+
+#endif
+
+/*
+ * Empty pmd entries point to the invalid_pte_table.
+ */
+static inline int pmd_none(pmd_t pmd)
+{
+ return pmd_val(pmd) == (unsigned long)invalid_pte_table;
+}
+
+static inline int pmd_bad(pmd_t pmd)
+{
+ return (pmd_val(pmd) & ~PAGE_MASK);
+}
+
+static inline int pmd_present(pmd_t pmd)
+{
+ if (unlikely(pmd_val(pmd) & _PAGE_HUGE))
+ return !!(pmd_val(pmd) & (_PAGE_PRESENT | _PAGE_PROTNONE));
+
+ return pmd_val(pmd) != (unsigned long)invalid_pte_table;
+}
+
+static inline void pmd_clear(pmd_t *pmdp)
+{
+ pmd_val(*pmdp) = ((unsigned long)invalid_pte_table);
+}
+
+#define set_pmd(pmdptr, pmdval) do { *(pmdptr) = (pmdval); } while (0)
+
+#define pmd_phys(pmd) virt_to_phys((void *)pmd_val(pmd))
+
+#ifndef CONFIG_TRANSPARENT_HUGEPAGE
+#define pmd_page(pmd) (pfn_to_page(pmd_phys(pmd) >> PAGE_SHIFT))
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+
+#define pmd_page_vaddr(pmd) pmd_val(pmd)
+
+extern pmd_t mk_pmd(struct page *page, pgprot_t prot);
+extern void set_pmd_at(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, pmd_t pmd);
+
+#define pte_page(x) pfn_to_page(pte_pfn(x))
+#define pte_pfn(x) ((unsigned long)(((x).pte & _PFN_MASK) >> _PFN_SHIFT))
+#define pfn_pte(pfn, prot) __pte(((pfn) << _PFN_SHIFT) | pgprot_val(prot))
+#define pfn_pmd(pfn, prot) __pmd(((pfn) << _PFN_SHIFT) | pgprot_val(prot))
+
+/*
+ * Initialize a new pgd / pmd table with invalid pointers.
+ */
+extern void pgd_init(unsigned long page);
+extern void pud_init(unsigned long page, unsigned long pagetable);
+extern void pmd_init(unsigned long page, unsigned long pagetable);
+
+/*
+ * Non-present pages: high 40 bits are offset, next 8 bits type,
+ * low 16 bits zero.
+ */
+static inline pte_t mk_swap_pte(unsigned long type, unsigned long offset)
+{ pte_t pte; pte_val(pte) = (type << 16) | (offset << 24); return pte; }
+
+#define __swp_type(x) (((x).val >> 16) & 0xff)
+#define __swp_offset(x) ((x).val >> 24)
+#define __swp_entry(type, offset) ((swp_entry_t) { pte_val(mk_swap_pte((type), (offset))) })
+#define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) })
+#define __swp_entry_to_pte(x) ((pte_t) { (x).val })
+#define __pmd_to_swp_entry(pmd) ((swp_entry_t) { pmd_val(pmd) })
+#define __swp_entry_to_pmd(x) ((pmd_t) { (x).val | _PAGE_HUGE })
+
+extern void paging_init(void);
+
+#define pte_none(pte) (!(pte_val(pte) & ~_PAGE_GLOBAL))
+#define pte_present(pte) (pte_val(pte) & (_PAGE_PRESENT | _PAGE_PROTNONE))
+#define pte_no_exec(pte) (pte_val(pte) & _PAGE_NO_EXEC)
+
+static inline void set_pte(pte_t *ptep, pte_t pteval)
+{
+ *ptep = pteval;
+ if (pte_val(pteval) & _PAGE_GLOBAL) {
+ pte_t *buddy = ptep_buddy(ptep);
+ /*
+ * Make sure the buddy is global too (if it's !none,
+ * it better already be global)
+ */
+ if (pte_none(*buddy))
+ pte_val(*buddy) = pte_val(*buddy) | _PAGE_GLOBAL;
+ }
+}
+
+static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pteval)
+{
+ set_pte(ptep, pteval);
+}
+
+static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep)
+{
+ /* Preserve global status for the pair */
+ if (pte_val(*ptep_buddy(ptep)) & _PAGE_GLOBAL)
+ set_pte_at(mm, addr, ptep, __pte(_PAGE_GLOBAL));
+ else
+ set_pte_at(mm, addr, ptep, __pte(0));
+}
+
+#define PGD_T_LOG2 (__builtin_ffs(sizeof(pgd_t)) - 1)
+#define PMD_T_LOG2 (__builtin_ffs(sizeof(pmd_t)) - 1)
+#define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1)
+
+extern pgd_t swapper_pg_dir[];
+extern pgd_t invalid_pg_dir[];
+
+/*
+ * The following only work if pte_present() is true.
+ * Undefined behaviour if not..
+ */
+static inline int pte_write(pte_t pte) { return pte_val(pte) & _PAGE_WRITE; }
+static inline int pte_young(pte_t pte) { return pte_val(pte) & _PAGE_ACCESSED; }
+static inline int pte_dirty(pte_t pte) { return pte_val(pte) & _PAGE_MODIFIED; }
+
+static inline pte_t pte_mkold(pte_t pte)
+{
+ pte_val(pte) &= ~_PAGE_ACCESSED;
+ return pte;
+}
+
+static inline pte_t pte_mkyoung(pte_t pte)
+{
+ pte_val(pte) |= _PAGE_ACCESSED;
+ return pte;
+}
+
+static inline pte_t pte_mkclean(pte_t pte)
+{
+ pte_val(pte) &= ~(_PAGE_DIRTY | _PAGE_MODIFIED);
+ return pte;
+}
+
+static inline pte_t pte_mkdirty(pte_t pte)
+{
+ pte_val(pte) |= (_PAGE_DIRTY | _PAGE_MODIFIED);
+ return pte;
+}
+
+static inline pte_t pte_mkwrite(pte_t pte)
+{
+ pte_val(pte) |= (_PAGE_WRITE | _PAGE_DIRTY);
+ return pte;
+}
+
+static inline pte_t pte_wrprotect(pte_t pte)
+{
+ pte_val(pte) &= ~(_PAGE_WRITE | _PAGE_DIRTY);
+ return pte;
+}
+
+static inline int pte_huge(pte_t pte) { return pte_val(pte) & _PAGE_HUGE; }
+
+static inline pte_t pte_mkhuge(pte_t pte)
+{
+ pte_val(pte) |= _PAGE_HUGE;
+ return pte;
+}
+
+#if defined(CONFIG_ARCH_HAS_PTE_SPECIAL)
+static inline int pte_special(pte_t pte) { return pte_val(pte) & _PAGE_SPECIAL; }
+static inline pte_t pte_mkspecial(pte_t pte) { pte_val(pte) |= _PAGE_SPECIAL; return pte; }
+#endif /* CONFIG_ARCH_HAS_PTE_SPECIAL */
+
+#define pte_accessible pte_accessible
+static inline unsigned long pte_accessible(struct mm_struct *mm, pte_t a)
+{
+ if (pte_val(a) & _PAGE_PRESENT)
+ return true;
+
+ if ((pte_val(a) & _PAGE_PROTNONE) &&
+ atomic_read(&mm->tlb_flush_pending))
+ return true;
+
+ return false;
+}
+
+/*
+ * Conversion functions: convert a page and protection to a page entry,
+ * and a page entry and page directory to the page they refer to.
+ */
+#define mk_pte(page, pgprot) pfn_pte(page_to_pfn(page), (pgprot))
+
+static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
+{
+ return __pte((pte_val(pte) & _PAGE_CHG_MASK) |
+ (pgprot_val(newprot) & ~_PAGE_CHG_MASK));
+}
+
+extern void __update_tlb(struct vm_area_struct *vma,
+ unsigned long address, pte_t *ptep);
+
+static inline void update_mmu_cache(struct vm_area_struct *vma,
+ unsigned long address, pte_t *ptep)
+{
+ __update_tlb(vma, address, ptep);
+}
+
+static inline void update_mmu_cache_pmd(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp)
+{
+ __update_tlb(vma, address, (pte_t *)pmdp);
+}
+
+#define kern_addr_valid(addr) (1)
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+
+/* We don't have hardware dirty/accessed bits, generic_pmdp_establish is fine.*/
+#define pmdp_establish generic_pmdp_establish
+
+static inline int pmd_trans_huge(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_HUGE) && pmd_present(pmd);
+}
+
+static inline pmd_t pmd_mkhuge(pmd_t pmd)
+{
+ pmd_val(pmd) = (pmd_val(pmd) & ~(_PAGE_GLOBAL)) |
+ ((pmd_val(pmd) & _PAGE_GLOBAL) << (_PAGE_HGLOBAL_SHIFT - _PAGE_GLOBAL_SHIFT));
+ pmd_val(pmd) |= _PAGE_HUGE;
+
+ return pmd;
+}
+
+#define pmd_write pmd_write
+static inline int pmd_write(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_WRITE);
+}
+
+static inline pmd_t pmd_mkwrite(pmd_t pmd)
+{
+ pmd_val(pmd) |= (_PAGE_WRITE | _PAGE_DIRTY);
+ return pmd;
+}
+
+static inline pmd_t pmd_wrprotect(pmd_t pmd)
+{
+ pmd_val(pmd) &= ~(_PAGE_WRITE | _PAGE_DIRTY);
+ return pmd;
+}
+
+static inline int pmd_dirty(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_MODIFIED);
+}
+
+static inline pmd_t pmd_mkclean(pmd_t pmd)
+{
+ pmd_val(pmd) &= ~(_PAGE_DIRTY | _PAGE_MODIFIED);
+ return pmd;
+}
+
+static inline pmd_t pmd_mkdirty(pmd_t pmd)
+{
+ pmd_val(pmd) |= (_PAGE_DIRTY | _PAGE_MODIFIED);
+ return pmd;
+}
+
+static inline int pmd_young(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_ACCESSED);
+}
+
+static inline pmd_t pmd_mkold(pmd_t pmd)
+{
+ pmd_val(pmd) &= ~_PAGE_ACCESSED;
+ return pmd;
+}
+
+static inline pmd_t pmd_mkyoung(pmd_t pmd)
+{
+ pmd_val(pmd) |= _PAGE_ACCESSED;
+ return pmd;
+}
+
+static inline unsigned long pmd_pfn(pmd_t pmd)
+{
+ return (pmd_val(pmd) & _PFN_MASK) >> _PFN_SHIFT;
+}
+
+static inline struct page *pmd_page(pmd_t pmd)
+{
+ if (pmd_trans_huge(pmd))
+ return pfn_to_page(pmd_pfn(pmd));
+
+ return pfn_to_page(pmd_phys(pmd) >> PAGE_SHIFT);
+}
+
+static inline pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot)
+{
+ pmd_val(pmd) = (pmd_val(pmd) & _HPAGE_CHG_MASK) |
+ (pgprot_val(newprot) & ~_HPAGE_CHG_MASK);
+ return pmd;
+}
+
+static inline pmd_t pmd_mkinvalid(pmd_t pmd)
+{
+ pmd_val(pmd) &= ~(_PAGE_PRESENT | _PAGE_VALID | _PAGE_DIRTY | _PAGE_PROTNONE);
+
+ return pmd;
+}
+
+/*
+ * The generic version pmdp_huge_get_and_clear uses a version of pmd_clear() with a
+ * different prototype.
+ */
+#define __HAVE_ARCH_PMDP_HUGE_GET_AND_CLEAR
+static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
+ unsigned long address, pmd_t *pmdp)
+{
+ pmd_t old = *pmdp;
+
+ pmd_clear(pmdp);
+
+ return old;
+}
+
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+
+/*
+ * We provide our own get_unmapped area to cope with the virtual aliasing
+ * constraints placed on us by the cache architecture.
+ */
+#define HAVE_ARCH_UNMAPPED_AREA
+#define HAVE_ARCH_UNMAPPED_AREA_TOPDOWN
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* _ASM_PGTABLE_H */
diff --git a/arch/loongarch/include/asm/shmparam.h b/arch/loongarch/include/asm/shmparam.h
new file mode 100644
index 000000000000..c9554f48d2df
--- /dev/null
+++ b/arch/loongarch/include/asm/shmparam.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_SHMPARAM_H
+#define _ASM_SHMPARAM_H
+
+#define __ARCH_FORCE_SHMLBA 1
+
+#define SHMLBA SZ_64K /* attach addr a multiple of this */
+
+#endif /* _ASM_SHMPARAM_H */
diff --git a/arch/loongarch/include/asm/sparsemem.h b/arch/loongarch/include/asm/sparsemem.h
new file mode 100644
index 000000000000..3d18cdf1b069
--- /dev/null
+++ b/arch/loongarch/include/asm/sparsemem.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LOONGARCH_SPARSEMEM_H
+#define _LOONGARCH_SPARSEMEM_H
+
+#ifdef CONFIG_SPARSEMEM
+
+/*
+ * SECTION_SIZE_BITS 2^N: how big each section will be
+ * MAX_PHYSMEM_BITS 2^N: how much memory we can have in that space
+ */
+#define SECTION_SIZE_BITS 29 /* 2^29 = Largest Huge Page Size */
+#define MAX_PHYSMEM_BITS 48
+
+#endif /* CONFIG_SPARSEMEM */
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+int memory_add_physaddr_to_nid(u64 addr);
+#define memory_add_physaddr_to_nid memory_add_physaddr_to_nid
+#endif
+
+#define INIT_MEMBLOCK_RESERVED_REGIONS (INIT_MEMBLOCK_REGIONS + NR_CPUS)
+
+#endif /* _LOONGARCH_SPARSEMEM_H */
diff --git a/arch/loongarch/include/asm/tlb.h b/arch/loongarch/include/asm/tlb.h
new file mode 100644
index 000000000000..4f629ae9d5a9
--- /dev/null
+++ b/arch/loongarch/include/asm/tlb.h
@@ -0,0 +1,180 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_TLB_H
+#define __ASM_TLB_H
+
+#include <linux/mm_types.h>
+#include <asm/cpu-features.h>
+#include <asm/loongarch.h>
+
+/*
+ * TLB Invalidate Flush
+ */
+static inline void tlbclr(void)
+{
+ __asm__ __volatile__("tlbclr");
+}
+
+static inline void tlbflush(void)
+{
+ __asm__ __volatile__("tlbflush");
+}
+
+/*
+ * TLB R/W operations.
+ */
+static inline void tlb_probe(void)
+{
+ __asm__ __volatile__("tlbsrch");
+}
+
+static inline void tlb_read(void)
+{
+ __asm__ __volatile__("tlbrd");
+}
+
+static inline void tlb_write_indexed(void)
+{
+ __asm__ __volatile__("tlbwr");
+}
+
+static inline void tlb_write_random(void)
+{
+ __asm__ __volatile__("tlbfill");
+}
+
+enum invtlb_ops {
+ /* Invalid all tlb */
+ INVTLB_ALL = 0x0,
+ /* Invalid current tlb */
+ INVTLB_CURRENT_ALL = 0x1,
+ /* Invalid all global=1 lines in current tlb */
+ INVTLB_CURRENT_GTRUE = 0x2,
+ /* Invalid all global=0 lines in current tlb */
+ INVTLB_CURRENT_GFALSE = 0x3,
+ /* Invalid global=0 and matched asid lines in current tlb */
+ INVTLB_GFALSE_AND_ASID = 0x4,
+ /* Invalid addr with global=0 and matched asid in current tlb */
+ INVTLB_ADDR_GFALSE_AND_ASID = 0x5,
+ /* Invalid addr with global=1 or matched asid in current tlb */
+ INVTLB_ADDR_GTRUE_OR_ASID = 0x6,
+ /* Invalid matched gid in guest tlb */
+ INVGTLB_GID = 0x9,
+ /* Invalid global=1, matched gid in guest tlb */
+ INVGTLB_GID_GTRUE = 0xa,
+ /* Invalid global=0, matched gid in guest tlb */
+ INVGTLB_GID_GFALSE = 0xb,
+ /* Invalid global=0, matched gid and asid in guest tlb */
+ INVGTLB_GID_GFALSE_ASID = 0xc,
+ /* Invalid global=0 , matched gid, asid and addr in guest tlb */
+ INVGTLB_GID_GFALSE_ASID_ADDR = 0xd,
+ /* Invalid global=1 , matched gid, asid and addr in guest tlb */
+ INVGTLB_GID_GTRUE_ASID_ADDR = 0xe,
+ /* Invalid all gid gva-->gpa guest tlb */
+ INVGTLB_ALLGID_GVA_TO_GPA = 0x10,
+ /* Invalid all gid gpa-->hpa tlb */
+ INVTLB_ALLGID_GPA_TO_HPA = 0x11,
+ /* Invalid all gid tlb, including gva-->gpa and gpa-->hpa */
+ INVTLB_ALLGID = 0x12,
+ /* Invalid matched gid gva-->gpa guest tlb */
+ INVGTLB_GID_GVA_TO_GPA = 0x13,
+ /* Invalid matched gid gpa-->hpa tlb */
+ INVTLB_GID_GPA_TO_HPA = 0x14,
+ /* Invalid matched gid tlb,including gva-->gpa and gpa-->hpa */
+ INVTLB_GID_ALL = 0x15,
+ /* Invalid matched gid and addr gpa-->hpa tlb */
+ INVTLB_GID_ADDR = 0x16,
+};
+
+/*
+ * invtlb op info addr
+ * (0x1 << 26) | (0x24 << 20) | (0x13 << 15) |
+ * (addr << 10) | (info << 5) | op
+ */
+static inline void invtlb(u32 op, u32 info, u64 addr)
+{
+ __asm__ __volatile__(
+ "parse_r addr,%0\n\t"
+ "parse_r info,%1\n\t"
+ ".word ((0x6498000) | (addr << 10) | (info << 5) | %2)\n\t"
+ :
+ : "r"(addr), "r"(info), "i"(op)
+ :
+ );
+}
+
+static inline void invtlb_addr(u32 op, u32 info, u64 addr)
+{
+ __asm__ __volatile__(
+ "parse_r addr,%0\n\t"
+ ".word ((0x6498000) | (addr << 10) | (0 << 5) | %1)\n\t"
+ :
+ : "r"(addr), "i"(op)
+ :
+ );
+}
+
+static inline void invtlb_info(u32 op, u32 info, u64 addr)
+{
+ __asm__ __volatile__(
+ "parse_r info,%0\n\t"
+ ".word ((0x6498000) | (0 << 10) | (info << 5) | %1)\n\t"
+ :
+ : "r"(info), "i"(op)
+ :
+ );
+}
+
+static inline void invtlb_all(u32 op, u32 info, u64 addr)
+{
+ __asm__ __volatile__(
+ ".word ((0x6498000) | (0 << 10) | (0 << 5) | %0)\n\t"
+ :
+ : "i"(op)
+ :
+ );
+}
+
+/*
+ * LoongArch doesn't need any special per-pte or per-vma handling, except
+ * we need to flush cache for area to be unmapped.
+ */
+#define tlb_start_vma(tlb, vma) \
+ do { \
+ if (!(tlb)->fullmm) \
+ flush_cache_range(vma, vma->vm_start, vma->vm_end); \
+ } while (0)
+#define tlb_end_vma(tlb, vma) do { } while (0)
+#define __tlb_remove_tlb_entry(tlb, ptep, address) do { } while (0)
+
+static void tlb_flush(struct mmu_gather *tlb);
+
+#define tlb_flush tlb_flush
+#include <asm-generic/tlb.h>
+
+static inline void tlb_flush(struct mmu_gather *tlb)
+{
+ struct vm_area_struct vma;
+
+ vma.vm_mm = tlb->mm;
+ vma.vm_flags = 0;
+ if (tlb->fullmm) {
+ flush_tlb_mm(tlb->mm);
+ return;
+ }
+
+ flush_tlb_range(&vma, tlb->start, tlb->end);
+}
+
+extern void handle_tlb_load(void);
+extern void handle_tlb_store(void);
+extern void handle_tlb_modify(void);
+extern void handle_tlb_refill(void);
+extern void handle_tlb_protect(void);
+
+extern void dump_tlb_all(void);
+extern void dump_tlb_regs(void);
+
+#endif /* __ASM_TLB_H */
diff --git a/arch/loongarch/include/asm/tlbflush.h b/arch/loongarch/include/asm/tlbflush.h
new file mode 100644
index 000000000000..36bd6d11dc2d
--- /dev/null
+++ b/arch/loongarch/include/asm/tlbflush.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_TLBFLUSH_H
+#define __ASM_TLBFLUSH_H
+
+#include <linux/mm.h>
+
+/*
+ * TLB flushing:
+ *
+ * - flush_tlb_all() flushes all processes TLB entries
+ * - flush_tlb_mm(mm) flushes the specified mm context TLB entries
+ * - flush_tlb_page(vma, vmaddr) flushes one page
+ * - flush_tlb_range(vma, start, end) flushes a range of pages
+ * - flush_tlb_kernel_range(start, end) flushes a range of kernel pages
+ */
+extern void local_flush_tlb_all(void);
+extern void local_flush_tlb_user(void);
+extern void local_flush_tlb_kernel(void);
+extern void local_flush_tlb_mm(struct mm_struct *mm);
+extern void local_flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end);
+extern void local_flush_tlb_kernel_range(unsigned long start, unsigned long end);
+extern void local_flush_tlb_page(struct vm_area_struct *vma, unsigned long page);
+extern void local_flush_tlb_one(unsigned long vaddr);
+
+#define flush_tlb_all() local_flush_tlb_all()
+#define flush_tlb_mm(mm) local_flush_tlb_mm(mm)
+#define flush_tlb_range(vma, vmaddr, end) local_flush_tlb_range(vma, vmaddr, end)
+#define flush_tlb_kernel_range(vmaddr, end) local_flush_tlb_kernel_range(vmaddr, end)
+#define flush_tlb_page(vma, page) local_flush_tlb_page(vma, page)
+#define flush_tlb_one(vaddr) local_flush_tlb_one(vaddr)
+
+#endif /* __ASM_TLBFLUSH_H */
diff --git a/arch/loongarch/include/asm/vmalloc.h b/arch/loongarch/include/asm/vmalloc.h
new file mode 100644
index 000000000000..965a0d41ac2d
--- /dev/null
+++ b/arch/loongarch/include/asm/vmalloc.h
@@ -0,0 +1,4 @@
+#ifndef _ASM_LOONGARCH_VMALLOC_H
+#define _ASM_LOONGARCH_VMALLOC_H
+
+#endif /* _ASM_LOONGARCH_VMALLOC_H */
diff --git a/arch/loongarch/mm/cache.c b/arch/loongarch/mm/cache.c
new file mode 100644
index 000000000000..7f65f750c776
--- /dev/null
+++ b/arch/loongarch/mm/cache.c
@@ -0,0 +1,140 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1994 - 2003, 06, 07 by Ralf Baechle ([email protected])
+ * Copyright (C) 2007 MIPS Technologies, Inc.
+ */
+#include <linux/export.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <linux/highmem.h>
+#include <linux/kernel.h>
+#include <linux/linkage.h>
+#include <linux/mm.h>
+#include <linux/sched.h>
+#include <linux/syscalls.h>
+
+#include <asm/cacheflush.h>
+#include <asm/cpu.h>
+#include <asm/cpu-features.h>
+#include <asm/dma.h>
+#include <asm/loongarch.h>
+#include <asm/processor.h>
+#include <asm/setup.h>
+
+/*
+ * LoongArch maintains ICache/DCache coherency by hardware,
+ * we just need "ibar" to avoid instruction hazard here.
+ */
+void local_flush_icache_range(unsigned long start, unsigned long end)
+{
+ asm volatile ("\tibar 0\n"::);
+}
+
+void cache_error_setup(void)
+{
+ extern char __weak except_vec_cex;
+ set_merr_handler(0x0, &except_vec_cex, 0x80);
+}
+
+static unsigned long icache_size __read_mostly;
+static unsigned long dcache_size __read_mostly;
+static unsigned long vcache_size __read_mostly;
+static unsigned long scache_size __read_mostly;
+
+static char *way_string[] = { NULL, "direct mapped", "2-way",
+ "3-way", "4-way", "5-way", "6-way", "7-way", "8-way",
+ "9-way", "10-way", "11-way", "12-way",
+ "13-way", "14-way", "15-way", "16-way",
+};
+
+static void probe_pcache(void)
+{
+ struct cpuinfo_loongarch *c = &current_cpu_data;
+ unsigned int lsize, sets, ways;
+ unsigned int config;
+
+ config = read_cpucfg(LOONGARCH_CPUCFG17);
+ lsize = 1 << ((config & CPUCFG17_L1I_SIZE_M) >> CPUCFG17_L1I_SIZE);
+ sets = 1 << ((config & CPUCFG17_L1I_SETS_M) >> CPUCFG17_L1I_SETS);
+ ways = ((config & CPUCFG17_L1I_WAYS_M) >> CPUCFG17_L1I_WAYS) + 1;
+
+ c->icache.linesz = lsize;
+ c->icache.sets = sets;
+ c->icache.ways = ways;
+ icache_size = sets * ways * lsize;
+ c->icache.waysize = icache_size / c->icache.ways;
+
+ config = read_cpucfg(LOONGARCH_CPUCFG18);
+ lsize = 1 << ((config & CPUCFG18_L1D_SIZE_M) >> CPUCFG18_L1D_SIZE);
+ sets = 1 << ((config & CPUCFG18_L1D_SETS_M) >> CPUCFG18_L1D_SETS);
+ ways = ((config & CPUCFG18_L1D_WAYS_M) >> CPUCFG18_L1D_WAYS) + 1;
+
+ c->dcache.linesz = lsize;
+ c->dcache.sets = sets;
+ c->dcache.ways = ways;
+ dcache_size = sets * ways * lsize;
+ c->dcache.waysize = dcache_size / c->dcache.ways;
+
+ c->options |= LOONGARCH_CPU_PREFETCH;
+
+ pr_info("Primary instruction cache %ldkB, %s, %s, linesize %d bytes.\n",
+ icache_size >> 10, way_string[c->icache.ways], "VIPT", c->icache.linesz);
+
+ pr_info("Primary data cache %ldkB, %s, %s, %s, linesize %d bytes\n",
+ dcache_size >> 10, way_string[c->dcache.ways], "VIPT", "no aliases", c->dcache.linesz);
+}
+
+static void probe_vcache(void)
+{
+ struct cpuinfo_loongarch *c = &current_cpu_data;
+ unsigned int lsize, sets, ways;
+ unsigned int config;
+
+ config = read_cpucfg(LOONGARCH_CPUCFG19);
+ lsize = 1 << ((config & CPUCFG19_L2_SIZE_M) >> CPUCFG19_L2_SIZE);
+ sets = 1 << ((config & CPUCFG19_L2_SETS_M) >> CPUCFG19_L2_SETS);
+ ways = ((config & CPUCFG19_L2_WAYS_M) >> CPUCFG19_L2_WAYS) + 1;
+
+ c->vcache.linesz = lsize;
+ c->vcache.sets = sets;
+ c->vcache.ways = ways;
+ vcache_size = lsize * sets * ways;
+ c->vcache.waysize = vcache_size / c->vcache.ways;
+
+ pr_info("Unified victim cache %ldkB %s, linesize %d bytes.\n",
+ vcache_size >> 10, way_string[c->vcache.ways], c->vcache.linesz);
+}
+
+static void probe_scache(void)
+{
+ struct cpuinfo_loongarch *c = &current_cpu_data;
+ unsigned int lsize, sets, ways;
+ unsigned int config;
+
+ config = read_cpucfg(LOONGARCH_CPUCFG20);
+ lsize = 1 << ((config & CPUCFG20_L3_SIZE_M) >> CPUCFG20_L3_SIZE);
+ sets = 1 << ((config & CPUCFG20_L3_SETS_M) >> CPUCFG20_L3_SETS);
+ ways = ((config & CPUCFG20_L3_WAYS_M) >> CPUCFG20_L3_WAYS) + 1;
+
+ c->scache.linesz = lsize;
+ c->scache.sets = sets;
+ c->scache.ways = ways;
+ /* 4 cores. scaches are shared */
+ scache_size = lsize * sets * ways;
+ c->scache.waysize = scache_size / c->scache.ways;
+
+ pr_info("Unified secondary cache %ldkB %s, linesize %d bytes.\n",
+ scache_size >> 10, way_string[c->scache.ways], c->scache.linesz);
+}
+
+void cpu_cache_init(void)
+{
+ probe_pcache();
+ probe_vcache();
+ probe_scache();
+
+ shm_align_mask = PAGE_SIZE - 1;
+}
diff --git a/arch/loongarch/mm/extable.c b/arch/loongarch/mm/extable.c
new file mode 100644
index 000000000000..bc20988f2b87
--- /dev/null
+++ b/arch/loongarch/mm/extable.c
@@ -0,0 +1,22 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/extable.h>
+#include <linux/spinlock.h>
+#include <asm/branch.h>
+#include <linux/uaccess.h>
+
+int fixup_exception(struct pt_regs *regs)
+{
+ const struct exception_table_entry *fixup;
+
+ fixup = search_exception_tables(exception_era(regs));
+ if (fixup) {
+ regs->csr_era = fixup->fixup;
+
+ return 1;
+ }
+
+ return 0;
+}
diff --git a/arch/loongarch/mm/fault.c b/arch/loongarch/mm/fault.c
new file mode 100644
index 000000000000..605579b19a00
--- /dev/null
+++ b/arch/loongarch/mm/fault.c
@@ -0,0 +1,261 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1995 - 2000 by Ralf Baechle
+ */
+#include <linux/context_tracking.h>
+#include <linux/signal.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/entry-common.h>
+#include <linux/errno.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/ptrace.h>
+#include <linux/ratelimit.h>
+#include <linux/mman.h>
+#include <linux/mm.h>
+#include <linux/smp.h>
+#include <linux/kdebug.h>
+#include <linux/kprobes.h>
+#include <linux/perf_event.h>
+#include <linux/uaccess.h>
+
+#include <asm/branch.h>
+#include <asm/mmu_context.h>
+#include <asm/ptrace.h>
+
+int show_unhandled_signals = 1;
+
+static void __kprobes no_context(struct pt_regs *regs, unsigned long address)
+{
+ const int field = sizeof(unsigned long) * 2;
+
+ /* Are we prepared to handle this kernel fault? */
+ if (fixup_exception(regs))
+ return;
+
+ /*
+ * Oops. The kernel tried to access some bad page. We'll have to
+ * terminate things with extreme prejudice.
+ */
+ bust_spinlocks(1);
+
+ pr_alert("CPU %d Unable to handle kernel paging request at "
+ "virtual address %0*lx, era == %0*lx, ra == %0*lx\n",
+ raw_smp_processor_id(), field, address, field, regs->csr_era,
+ field, regs->regs[1]);
+ die("Oops", regs);
+}
+
+static void __kprobes do_out_of_memory(struct pt_regs *regs, unsigned long address)
+{
+ /*
+ * We ran out of memory, call the OOM killer, and return the userspace
+ * (which will retry the fault, or kill us if we got oom-killed).
+ */
+ if (!user_mode(regs)) {
+ no_context(regs, address);
+ return;
+ }
+ pagefault_out_of_memory();
+}
+
+static void __kprobes do_sigbus(struct pt_regs *regs,
+ unsigned long write, unsigned long address, int si_code)
+{
+ /* Kernel mode? Handle exceptions or die */
+ if (!user_mode(regs)) {
+ no_context(regs, address);
+ return;
+ }
+
+ /*
+ * Send a sigbus, regardless of whether we were in kernel
+ * or user mode.
+ */
+ current->thread.csr_badvaddr = address;
+ current->thread.trap_nr = read_csr_excode();
+ force_sig_fault(SIGBUS, BUS_ADRERR, (void __user *)address);
+}
+
+static void __kprobes do_sigsegv(struct pt_regs *regs,
+ unsigned long write, unsigned long address, int si_code)
+{
+ const int field = sizeof(unsigned long) * 2;
+ static DEFINE_RATELIMIT_STATE(ratelimit_state, 5 * HZ, 10);
+
+ /* Kernel mode? Handle exceptions or die */
+ if (!user_mode(regs)) {
+ no_context(regs, address);
+ return;
+ }
+
+ /* User mode accesses just cause a SIGSEGV */
+ current->thread.csr_badvaddr = address;
+ if (!write)
+ current->thread.error_code = 1;
+ else
+ current->thread.error_code = 2;
+ current->thread.trap_nr = read_csr_excode();
+
+ if (show_unhandled_signals &&
+ unhandled_signal(current, SIGSEGV) && __ratelimit(&ratelimit_state)) {
+ pr_info("do_page_fault(): sending SIGSEGV to %s for invalid %s %0*lx\n",
+ current->comm,
+ write ? "write access to" : "read access from",
+ field, address);
+ pr_info("era = %0*lx in", field,
+ (unsigned long) regs->csr_era);
+ print_vma_addr(KERN_CONT " ", regs->csr_era);
+ pr_cont("\n");
+ pr_info("ra = %0*lx in", field,
+ (unsigned long) regs->regs[1]);
+ print_vma_addr(KERN_CONT " ", regs->regs[1]);
+ pr_cont("\n");
+ }
+ force_sig_fault(SIGSEGV, si_code, (void __user *)address);
+}
+
+/*
+ * This routine handles page faults. It determines the address,
+ * and the problem, and then passes it off to one of the appropriate
+ * routines.
+ */
+static void __kprobes __do_page_fault(struct pt_regs *regs,
+ unsigned long write, unsigned long address)
+{
+ int si_code = SEGV_MAPERR;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
+ struct task_struct *tsk = current;
+ struct mm_struct *mm = tsk->mm;
+ struct vm_area_struct *vma = NULL;
+ vm_fault_t fault;
+
+ /*
+ * We fault-in kernel-space virtual memory on-demand. The
+ * 'reference' page table is init_mm.pgd.
+ *
+ * NOTE! We MUST NOT take any locks for this case. We may
+ * be in an interrupt or a critical region, and should
+ * only copy the information from the master page table,
+ * nothing more.
+ */
+ if (address & __UA_LIMIT) {
+ if (!user_mode(regs))
+ no_context(regs, address);
+ else
+ do_sigsegv(regs, write, address, si_code);
+ return;
+ }
+
+ /*
+ * If we're in an interrupt or have no user
+ * context, we must not take the fault..
+ */
+ if (faulthandler_disabled() || !mm) {
+ do_sigsegv(regs, write, address, si_code);
+ return;
+ }
+
+ if (user_mode(regs))
+ flags |= FAULT_FLAG_USER;
+
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address);
+retry:
+ mmap_read_lock(mm);
+ vma = find_vma(mm, address);
+ if (!vma)
+ goto bad_area;
+ if (vma->vm_start <= address)
+ goto good_area;
+ if (!(vma->vm_flags & VM_GROWSDOWN))
+ goto bad_area;
+ if (!expand_stack(vma, address))
+ goto good_area;
+/*
+ * Something tried to access memory that isn't in our memory map..
+ * Fix it, but check if it's kernel or user first..
+ */
+bad_area:
+ mmap_read_unlock(mm);
+ do_sigsegv(regs, write, address, si_code);
+ return;
+
+/*
+ * Ok, we have a good vm_area for this memory access, so
+ * we can handle it..
+ */
+good_area:
+ si_code = SEGV_ACCERR;
+
+ if (write) {
+ flags |= FAULT_FLAG_WRITE;
+ if (!(vma->vm_flags & VM_WRITE))
+ goto bad_area;
+ } else {
+ if (!(vma->vm_flags & VM_READ) && address != exception_era(regs))
+ goto bad_area;
+ if (!(vma->vm_flags & VM_EXEC) && address == exception_era(regs))
+ goto bad_area;
+ }
+
+ /*
+ * If for any reason at all we couldn't handle the fault,
+ * make sure we exit gracefully rather than endlessly redo
+ * the fault.
+ */
+ fault = handle_mm_fault(vma, address, flags, regs);
+
+ if (fault_signal_pending(fault, regs)) {
+ if (!user_mode(regs))
+ no_context(regs, address);
+ return;
+ }
+
+ if (unlikely(fault & VM_FAULT_RETRY)) {
+ flags |= FAULT_FLAG_TRIED;
+
+ /*
+ * No need to mmap_read_unlock(mm) as we would
+ * have already released it in __lock_page_or_retry
+ * in mm/filemap.c.
+ */
+ goto retry;
+ }
+ if (unlikely(fault & VM_FAULT_ERROR)) {
+ mmap_read_unlock(mm);
+ if (fault & VM_FAULT_OOM) {
+ do_out_of_memory(regs, address);
+ return;
+ } else if (fault & VM_FAULT_SIGSEGV) {
+ do_sigsegv(regs, write, address, si_code);
+ return;
+ } else if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|VM_FAULT_HWPOISON_LARGE)) {
+ do_sigbus(regs, write, address, si_code);
+ return;
+ }
+ BUG();
+ }
+
+ mmap_read_unlock(mm);
+}
+
+asmlinkage void __kprobes do_page_fault(struct pt_regs *regs,
+ unsigned long write, unsigned long address)
+{
+ irqentry_state_t state = irqentry_enter(regs);
+
+ /* Enable interrupt if enabled in parent context */
+ if (likely(regs->csr_prmd & CSR_PRMD_PIE))
+ local_irq_enable();
+
+ __do_page_fault(regs, write, address);
+
+ local_irq_disable();
+
+ irqentry_exit(regs, state);
+}
diff --git a/arch/loongarch/mm/hugetlbpage.c b/arch/loongarch/mm/hugetlbpage.c
new file mode 100644
index 000000000000..ba138117b124
--- /dev/null
+++ b/arch/loongarch/mm/hugetlbpage.c
@@ -0,0 +1,87 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/fs.h>
+#include <linux/mm.h>
+#include <linux/hugetlb.h>
+#include <linux/pagemap.h>
+#include <linux/err.h>
+#include <linux/sysctl.h>
+#include <asm/mman.h>
+#include <asm/tlb.h>
+#include <asm/tlbflush.h>
+
+pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
+ unsigned long addr, unsigned long sz)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pte_t *pte = NULL;
+
+ pgd = pgd_offset(mm, addr);
+ p4d = p4d_alloc(mm, pgd, addr);
+ pud = pud_alloc(mm, p4d, addr);
+ if (pud)
+ pte = (pte_t *)pmd_alloc(mm, pud, addr);
+
+ return pte;
+}
+
+pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr,
+ unsigned long sz)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd = NULL;
+
+ pgd = pgd_offset(mm, addr);
+ if (pgd_present(*pgd)) {
+ p4d = p4d_offset(pgd, addr);
+ if (p4d_present(*p4d)) {
+ pud = pud_offset(p4d, addr);
+ if (pud_present(*pud))
+ pmd = pmd_offset(pud, addr);
+ }
+ }
+ return (pte_t *) pmd;
+}
+
+/*
+ * This function checks for proper alignment of input addr and len parameters.
+ */
+int is_aligned_hugepage_range(unsigned long addr, unsigned long len)
+{
+ if (len & ~HPAGE_MASK)
+ return -EINVAL;
+ if (addr & ~HPAGE_MASK)
+ return -EINVAL;
+ return 0;
+}
+
+int pmd_huge(pmd_t pmd)
+{
+ return (pmd_val(pmd) & _PAGE_HUGE) != 0;
+}
+
+int pud_huge(pud_t pud)
+{
+ return (pud_val(pud) & _PAGE_HUGE) != 0;
+}
+
+uint64_t pmd_to_entrylo(unsigned long pmd_val)
+{
+ uint64_t val;
+ /* PMD as PTE. Must be huge page */
+ if (!pmd_huge(__pmd(pmd_val)))
+ panic("%s", __func__);
+
+ val = pmd_val ^ _PAGE_HUGE;
+ val |= ((val & _PAGE_HGLOBAL) >>
+ (_PAGE_HGLOBAL_SHIFT - _PAGE_GLOBAL_SHIFT));
+
+ return val;
+}
diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c
new file mode 100644
index 000000000000..afd6634ce171
--- /dev/null
+++ b/arch/loongarch/mm/init.c
@@ -0,0 +1,165 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/init.h>
+#include <linux/export.h>
+#include <linux/signal.h>
+#include <linux/sched.h>
+#include <linux/smp.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/pagemap.h>
+#include <linux/memblock.h>
+#include <linux/memremap.h>
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/highmem.h>
+#include <linux/swap.h>
+#include <linux/proc_fs.h>
+#include <linux/pfn.h>
+#include <linux/hardirq.h>
+#include <linux/gfp.h>
+#include <linux/initrd.h>
+#include <linux/mmzone.h>
+
+#include <asm/asm-offsets.h>
+#include <asm/bootinfo.h>
+#include <asm/cpu.h>
+#include <asm/dma.h>
+#include <asm/mmu_context.h>
+#include <asm/sections.h>
+#include <asm/pgtable.h>
+#include <asm/pgalloc.h>
+#include <asm/tlb.h>
+
+/*
+ * We have up to 8 empty zeroed pages so we can map one of the right colour
+ * when needed. Since page is never written to after the initialization we
+ * don't have to care about aliases on other CPUs.
+ */
+unsigned long empty_zero_page, zero_page_mask;
+EXPORT_SYMBOL_GPL(empty_zero_page);
+EXPORT_SYMBOL(zero_page_mask);
+
+void setup_zero_pages(void)
+{
+ unsigned int order, i;
+ struct page *page;
+
+ order = 0;
+
+ empty_zero_page = __get_free_pages(GFP_KERNEL | __GFP_ZERO, order);
+ if (!empty_zero_page)
+ panic("Oh boy, that early out of memory?");
+
+ page = virt_to_page((void *)empty_zero_page);
+ split_page(page, order);
+ for (i = 0; i < (1 << order); i++, page++)
+ mark_page_reserved(page);
+
+ zero_page_mask = ((PAGE_SIZE << order) - 1) & PAGE_MASK;
+}
+
+void copy_user_highpage(struct page *to, struct page *from,
+ unsigned long vaddr, struct vm_area_struct *vma)
+{
+ void *vfrom, *vto;
+
+ vto = kmap_atomic(to);
+ vfrom = kmap_atomic(from);
+ copy_page(vto, vfrom);
+ kunmap_atomic(vfrom);
+ kunmap_atomic(vto);
+ /* Make sure this page is cleared on other CPU's too before using it */
+ smp_wmb();
+}
+
+int __ref page_is_ram(unsigned long pfn)
+{
+ unsigned long addr = PFN_PHYS(pfn);
+
+ return memblock_is_memory(addr) && !memblock_is_reserved(addr);
+}
+
+void __init paging_init(void)
+{
+ unsigned long max_zone_pfns[MAX_NR_ZONES];
+
+#ifdef CONFIG_ZONE_DMA
+ max_zone_pfns[ZONE_DMA] = MAX_DMA_PFN;
+#endif
+#ifdef CONFIG_ZONE_DMA32
+ max_zone_pfns[ZONE_DMA32] = MAX_DMA32_PFN;
+#endif
+ max_zone_pfns[ZONE_NORMAL] = max_low_pfn;
+
+ free_area_init(max_zone_pfns);
+}
+
+void __init mem_init(void)
+{
+ max_mapnr = max_low_pfn;
+ high_memory = (void *) __va(max_low_pfn << PAGE_SHIFT);
+
+ memblock_free_all();
+ setup_zero_pages(); /* Setup zeroed pages. */
+}
+
+void __ref free_initmem(void)
+{
+ free_initmem_default(POISON_FREE_INITMEM);
+}
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+int arch_add_memory(int nid, u64 start, u64 size, struct mhp_params *params)
+{
+ unsigned long start_pfn = start >> PAGE_SHIFT;
+ unsigned long nr_pages = size >> PAGE_SHIFT;
+ int ret;
+
+ ret = __add_pages(nid, start_pfn, nr_pages, params);
+
+ if (ret)
+ pr_warn("%s: Problem encountered in __add_pages() as ret=%d\n",
+ __func__, ret);
+
+ return ret;
+}
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+void arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap)
+{
+ unsigned long start_pfn = start >> PAGE_SHIFT;
+ unsigned long nr_pages = size >> PAGE_SHIFT;
+ struct page *page = pfn_to_page(start_pfn);
+
+ /* With altmap the first mapped page is offset from @start */
+ if (altmap)
+ page += vmem_altmap_offset(altmap);
+ __remove_pages(start_pfn, nr_pages, altmap);
+}
+#endif
+#endif
+
+/*
+ * Align swapper_pg_dir in to 64K, allows its address to be loaded
+ * with a single LUI instruction in the TLB handlers. If we used
+ * __aligned(64K), its size would get rounded up to the alignment
+ * size, and waste space. So we place it in its own section and align
+ * it in the linker script.
+ */
+pgd_t swapper_pg_dir[_PTRS_PER_PGD] __section(".bss..swapper_pg_dir");
+
+pgd_t invalid_pg_dir[_PTRS_PER_PGD] __page_aligned_bss;
+#ifndef __PAGETABLE_PUD_FOLDED
+pud_t invalid_pud_table[PTRS_PER_PUD] __page_aligned_bss;
+#endif
+#ifndef __PAGETABLE_PMD_FOLDED
+pmd_t invalid_pmd_table[PTRS_PER_PMD] __page_aligned_bss;
+EXPORT_SYMBOL_GPL(invalid_pmd_table);
+#endif
+pte_t invalid_pte_table[PTRS_PER_PTE] __page_aligned_bss;
+EXPORT_SYMBOL(invalid_pte_table);
diff --git a/arch/loongarch/mm/ioremap.c b/arch/loongarch/mm/ioremap.c
new file mode 100644
index 000000000000..73b0980ab6f5
--- /dev/null
+++ b/arch/loongarch/mm/ioremap.c
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/io.h>
+
+void __init __iomem *early_ioremap(u64 phys_addr, unsigned long size)
+{
+ return ((void __iomem *)TO_CACHE(phys_addr));
+}
+
+void __init early_iounmap(void __iomem *addr, unsigned long size)
+{
+
+}
+
+void *early_memremap_ro(resource_size_t phys_addr, unsigned long size)
+{
+ return early_memremap(phys_addr, size);
+}
+
+void *early_memremap_prot(resource_size_t phys_addr, unsigned long size,
+ unsigned long prot_val)
+{
+ return early_memremap(phys_addr, size);
+}
diff --git a/arch/loongarch/mm/maccess.c b/arch/loongarch/mm/maccess.c
new file mode 100644
index 000000000000..58173842c6be
--- /dev/null
+++ b/arch/loongarch/mm/maccess.c
@@ -0,0 +1,10 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+#include <linux/uaccess.h>
+#include <linux/kernel.h>
+
+bool copy_from_kernel_nofault_allowed(const void *unsafe_src, size_t size)
+{
+ /* highest bit set means kernel space */
+ return (unsigned long)unsafe_src >> (BITS_PER_LONG - 1);
+}
diff --git a/arch/loongarch/mm/mmap.c b/arch/loongarch/mm/mmap.c
new file mode 100644
index 000000000000..52e40f0ba732
--- /dev/null
+++ b/arch/loongarch/mm/mmap.c
@@ -0,0 +1,125 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/compiler.h>
+#include <linux/elf-randomize.h>
+#include <linux/errno.h>
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/export.h>
+#include <linux/personality.h>
+#include <linux/random.h>
+#include <linux/sched/signal.h>
+#include <linux/sched/mm.h>
+
+unsigned long shm_align_mask = PAGE_SIZE - 1; /* Sane caches */
+EXPORT_SYMBOL(shm_align_mask);
+
+#define COLOUR_ALIGN(addr, pgoff) \
+ ((((addr) + shm_align_mask) & ~shm_align_mask) + \
+ (((pgoff) << PAGE_SHIFT) & shm_align_mask))
+
+enum mmap_allocation_direction {UP, DOWN};
+
+static unsigned long arch_get_unmapped_area_common(struct file *filp,
+ unsigned long addr0, unsigned long len, unsigned long pgoff,
+ unsigned long flags, enum mmap_allocation_direction dir)
+{
+ struct mm_struct *mm = current->mm;
+ struct vm_area_struct *vma;
+ unsigned long addr = addr0;
+ int do_color_align;
+ struct vm_unmapped_area_info info;
+
+ if (unlikely(len > TASK_SIZE))
+ return -ENOMEM;
+
+ if (flags & MAP_FIXED) {
+ /* Even MAP_FIXED mappings must reside within TASK_SIZE */
+ if (TASK_SIZE - len < addr)
+ return -EINVAL;
+
+ /*
+ * We do not accept a shared mapping if it would violate
+ * cache aliasing constraints.
+ */
+ if ((flags & MAP_SHARED) &&
+ ((addr - (pgoff << PAGE_SHIFT)) & shm_align_mask))
+ return -EINVAL;
+ return addr;
+ }
+
+ do_color_align = 0;
+ if (filp || (flags & MAP_SHARED))
+ do_color_align = 1;
+
+ /* requesting a specific address */
+ if (addr) {
+ if (do_color_align)
+ addr = COLOUR_ALIGN(addr, pgoff);
+ else
+ addr = PAGE_ALIGN(addr);
+
+ vma = find_vma(mm, addr);
+ if (TASK_SIZE - len >= addr &&
+ (!vma || addr + len <= vm_start_gap(vma)))
+ return addr;
+ }
+
+ info.length = len;
+ info.align_mask = do_color_align ? (PAGE_MASK & shm_align_mask) : 0;
+ info.align_offset = pgoff << PAGE_SHIFT;
+
+ if (dir == DOWN) {
+ info.flags = VM_UNMAPPED_AREA_TOPDOWN;
+ info.low_limit = PAGE_SIZE;
+ info.high_limit = mm->mmap_base;
+ addr = vm_unmapped_area(&info);
+
+ if (!(addr & ~PAGE_MASK))
+ return addr;
+
+ /*
+ * A failed mmap() very likely causes application failure,
+ * so fall back to the bottom-up function here. This scenario
+ * can happen with large stack limits and large mmap()
+ * allocations.
+ */
+ }
+
+ info.flags = 0;
+ info.low_limit = mm->mmap_base;
+ info.high_limit = TASK_SIZE;
+ return vm_unmapped_area(&info);
+}
+
+unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr0,
+ unsigned long len, unsigned long pgoff, unsigned long flags)
+{
+ return arch_get_unmapped_area_common(filp,
+ addr0, len, pgoff, flags, UP);
+}
+
+/*
+ * There is no need to export this but sched.h declares the function as
+ * extern so making it static here results in an error.
+ */
+unsigned long arch_get_unmapped_area_topdown(struct file *filp,
+ unsigned long addr0, unsigned long len, unsigned long pgoff,
+ unsigned long flags)
+{
+ return arch_get_unmapped_area_common(filp,
+ addr0, len, pgoff, flags, DOWN);
+}
+
+int __virt_addr_valid(volatile void *kaddr)
+{
+ unsigned long vaddr = (unsigned long)kaddr;
+
+ if ((vaddr < PAGE_OFFSET) || (vaddr >= vm_map_base))
+ return 0;
+
+ return pfn_valid(PFN_DOWN(virt_to_phys(kaddr)));
+}
+EXPORT_SYMBOL_GPL(__virt_addr_valid);
diff --git a/arch/loongarch/mm/page.S b/arch/loongarch/mm/page.S
new file mode 100644
index 000000000000..ddc78ab33c7b
--- /dev/null
+++ b/arch/loongarch/mm/page.S
@@ -0,0 +1,84 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/linkage.h>
+#include <asm/asm.h>
+#include <asm/export.h>
+#include <asm/page.h>
+#include <asm/regdef.h>
+
+ .align 5
+SYM_FUNC_START(clear_page)
+ lu12i.w t0, 1 << (PAGE_SHIFT - 12)
+ add.d t0, t0, a0
+1:
+ st.d zero, a0, 0
+ st.d zero, a0, 8
+ st.d zero, a0, 16
+ st.d zero, a0, 24
+ st.d zero, a0, 32
+ st.d zero, a0, 40
+ st.d zero, a0, 48
+ st.d zero, a0, 56
+ addi.d a0, a0, 128
+ st.d zero, a0, -64
+ st.d zero, a0, -56
+ st.d zero, a0, -48
+ st.d zero, a0, -40
+ st.d zero, a0, -32
+ st.d zero, a0, -24
+ st.d zero, a0, -16
+ st.d zero, a0, -8
+ bne t0, a0, 1b
+
+ jirl $r0, ra, 0
+SYM_FUNC_END(clear_page)
+EXPORT_SYMBOL(clear_page)
+
+.align 5
+SYM_FUNC_START(copy_page)
+ lu12i.w t8, 1 << (PAGE_SHIFT - 12)
+ add.d t8, t8, a0
+1:
+ ld.d t0, a1, 0
+ ld.d t1, a1, 8
+ ld.d t2, a1, 16
+ ld.d t3, a1, 24
+ ld.d t4, a1, 32
+ ld.d t5, a1, 40
+ ld.d t6, a1, 48
+ ld.d t7, a1, 56
+
+ st.d t0, a0, 0
+ st.d t1, a0, 8
+ ld.d t0, a1, 64
+ ld.d t1, a1, 72
+ st.d t2, a0, 16
+ st.d t3, a0, 24
+ ld.d t2, a1, 80
+ ld.d t3, a1, 88
+ st.d t4, a0, 32
+ st.d t5, a0, 40
+ ld.d t4, a1, 96
+ ld.d t5, a1, 104
+ st.d t6, a0, 48
+ st.d t7, a0, 56
+ ld.d t6, a1, 112
+ ld.d t7, a1, 120
+ addi.d a0, a0, 128
+ addi.d a1, a1, 128
+
+ st.d t0, a0, -64
+ st.d t1, a0, -56
+ st.d t2, a0, -48
+ st.d t3, a0, -40
+ st.d t4, a0, -32
+ st.d t5, a0, -24
+ st.d t6, a0, -16
+ st.d t7, a0, -8
+
+ bne t8, a0, 1b
+ jirl $r0, ra, 0
+SYM_FUNC_END(copy_page)
+EXPORT_SYMBOL(copy_page)
diff --git a/arch/loongarch/mm/pgtable.c b/arch/loongarch/mm/pgtable.c
new file mode 100644
index 000000000000..0569647152e9
--- /dev/null
+++ b/arch/loongarch/mm/pgtable.c
@@ -0,0 +1,130 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/init.h>
+#include <linux/export.h>
+#include <linux/mm.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+#include <asm/tlbflush.h>
+
+pgd_t *pgd_alloc(struct mm_struct *mm)
+{
+ pgd_t *ret, *init;
+
+ ret = (pgd_t *) __get_free_pages(GFP_KERNEL, PGD_ORDER);
+ if (ret) {
+ init = pgd_offset(&init_mm, 0UL);
+ pgd_init((unsigned long)ret);
+ memcpy(ret + USER_PTRS_PER_PGD, init + USER_PTRS_PER_PGD,
+ (PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t));
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(pgd_alloc);
+
+void pgd_init(unsigned long page)
+{
+ unsigned long *p, *end;
+ unsigned long entry;
+
+#if !defined(__PAGETABLE_PUD_FOLDED)
+ entry = (unsigned long)invalid_pud_table;
+#elif !defined(__PAGETABLE_PMD_FOLDED)
+ entry = (unsigned long)invalid_pmd_table;
+#else
+ entry = (unsigned long)invalid_pte_table;
+#endif
+
+ p = (unsigned long *) page;
+ end = p + PTRS_PER_PGD;
+
+ do {
+ p[0] = entry;
+ p[1] = entry;
+ p[2] = entry;
+ p[3] = entry;
+ p[4] = entry;
+ p += 8;
+ p[-3] = entry;
+ p[-2] = entry;
+ p[-1] = entry;
+ } while (p != end);
+}
+EXPORT_SYMBOL_GPL(pgd_init);
+
+#ifndef __PAGETABLE_PMD_FOLDED
+void pmd_init(unsigned long addr, unsigned long pagetable)
+{
+ unsigned long *p, *end;
+
+ p = (unsigned long *) addr;
+ end = p + PTRS_PER_PMD;
+
+ do {
+ p[0] = pagetable;
+ p[1] = pagetable;
+ p[2] = pagetable;
+ p[3] = pagetable;
+ p[4] = pagetable;
+ p += 8;
+ p[-3] = pagetable;
+ p[-2] = pagetable;
+ p[-1] = pagetable;
+ } while (p != end);
+}
+EXPORT_SYMBOL_GPL(pmd_init);
+#endif
+
+#ifndef __PAGETABLE_PUD_FOLDED
+void pud_init(unsigned long addr, unsigned long pagetable)
+{
+ unsigned long *p, *end;
+
+ p = (unsigned long *)addr;
+ end = p + PTRS_PER_PUD;
+
+ do {
+ p[0] = pagetable;
+ p[1] = pagetable;
+ p[2] = pagetable;
+ p[3] = pagetable;
+ p[4] = pagetable;
+ p += 8;
+ p[-3] = pagetable;
+ p[-2] = pagetable;
+ p[-1] = pagetable;
+ } while (p != end);
+}
+#endif
+
+pmd_t mk_pmd(struct page *page, pgprot_t prot)
+{
+ pmd_t pmd;
+
+ pmd_val(pmd) = (page_to_pfn(page) << _PFN_SHIFT) | pgprot_val(prot);
+
+ return pmd;
+}
+
+void set_pmd_at(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, pmd_t pmd)
+{
+ *pmdp = pmd;
+ flush_tlb_all();
+}
+
+void __init pagetable_init(void)
+{
+ /* Initialize the entire pgd. */
+ pgd_init((unsigned long)swapper_pg_dir);
+ pgd_init((unsigned long)invalid_pg_dir);
+#ifndef __PAGETABLE_PUD_FOLDED
+ pud_init((unsigned long)invalid_pud_table, (unsigned long)invalid_pmd_table);
+#endif
+#ifndef __PAGETABLE_PMD_FOLDED
+ pmd_init((unsigned long)invalid_pmd_table, (unsigned long)invalid_pte_table);
+#endif
+}
diff --git a/arch/loongarch/mm/tlb.c b/arch/loongarch/mm/tlb.c
new file mode 100644
index 000000000000..5201a87937fe
--- /dev/null
+++ b/arch/loongarch/mm/tlb.c
@@ -0,0 +1,282 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/init.h>
+#include <linux/sched.h>
+#include <linux/smp.h>
+#include <linux/mm.h>
+#include <linux/hugetlb.h>
+#include <linux/export.h>
+
+#include <asm/cpu.h>
+#include <asm/bootinfo.h>
+#include <asm/mmu_context.h>
+#include <asm/pgtable.h>
+#include <asm/tlb.h>
+
+void local_flush_tlb_all(void)
+{
+ invtlb_all(INVTLB_CURRENT_ALL, 0, 0);
+}
+EXPORT_SYMBOL(local_flush_tlb_all);
+
+void local_flush_tlb_user(void)
+{
+ invtlb_all(INVTLB_CURRENT_GFALSE, 0, 0);
+}
+EXPORT_SYMBOL(local_flush_tlb_user);
+
+void local_flush_tlb_kernel(void)
+{
+ invtlb_all(INVTLB_CURRENT_GTRUE, 0, 0);
+}
+EXPORT_SYMBOL(local_flush_tlb_kernel);
+
+/*
+ * All entries common to a mm share an asid. To effectively flush
+ * these entries, we just bump the asid.
+ */
+void local_flush_tlb_mm(struct mm_struct *mm)
+{
+ int cpu;
+
+ preempt_disable();
+
+ cpu = smp_processor_id();
+
+ if (asid_valid(mm, cpu))
+ drop_mmu_context(mm, cpu);
+ else
+ cpumask_clear_cpu(cpu, mm_cpumask(mm));
+
+ preempt_enable();
+}
+
+void local_flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
+ unsigned long end)
+{
+ struct mm_struct *mm = vma->vm_mm;
+ int cpu = smp_processor_id();
+
+ if (asid_valid(mm, cpu)) {
+ unsigned long size, flags;
+
+ local_irq_save(flags);
+ start = round_down(start, PAGE_SIZE << 1);
+ end = round_up(end, PAGE_SIZE << 1);
+ size = (end - start) >> (PAGE_SHIFT + 1);
+ if (size <= (current_cpu_data.tlbsizestlbsets ?
+ current_cpu_data.tlbsize / 8 :
+ current_cpu_data.tlbsize / 2)) {
+ int asid = cpu_asid(cpu, mm);
+
+ while (start < end) {
+ invtlb(INVTLB_ADDR_GFALSE_AND_ASID, asid, start);
+ start += (PAGE_SIZE << 1);
+ }
+ } else {
+ drop_mmu_context(mm, cpu);
+ }
+ local_irq_restore(flags);
+ } else {
+ cpumask_clear_cpu(cpu, mm_cpumask(mm));
+ }
+}
+
+void local_flush_tlb_kernel_range(unsigned long start, unsigned long end)
+{
+ unsigned long size, flags;
+
+ local_irq_save(flags);
+ size = (end - start + (PAGE_SIZE - 1)) >> PAGE_SHIFT;
+ size = (size + 1) >> 1;
+ if (size <= (current_cpu_data.tlbsizestlbsets ?
+ current_cpu_data.tlbsize / 8 :
+ current_cpu_data.tlbsize / 2)) {
+
+ start &= (PAGE_MASK << 1);
+ end += ((PAGE_SIZE << 1) - 1);
+ end &= (PAGE_MASK << 1);
+
+ while (start < end) {
+ invtlb_addr(INVTLB_ADDR_GTRUE_OR_ASID, 0, start);
+ start += (PAGE_SIZE << 1);
+ }
+ } else {
+ local_flush_tlb_kernel();
+ }
+ local_irq_restore(flags);
+}
+
+void local_flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
+{
+ int cpu = smp_processor_id();
+
+ if (asid_valid(vma->vm_mm, cpu)) {
+ int newpid;
+
+ newpid = cpu_asid(cpu, vma->vm_mm);
+ page &= (PAGE_MASK << 1);
+ invtlb(INVTLB_ADDR_GFALSE_AND_ASID, newpid, page);
+ } else {
+ cpumask_clear_cpu(cpu, mm_cpumask(vma->vm_mm));
+ }
+}
+
+/*
+ * This one is only used for pages with the global bit set so we don't care
+ * much about the ASID.
+ */
+void local_flush_tlb_one(unsigned long page)
+{
+ page &= (PAGE_MASK << 1);
+ invtlb_addr(INVTLB_ADDR_GTRUE_OR_ASID, 0, page);
+}
+
+static void __update_hugetlb(struct vm_area_struct *vma, unsigned long address, pte_t *ptep)
+{
+#ifdef CONFIG_HUGETLB_PAGE
+ int idx;
+ unsigned long lo;
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ address &= (PAGE_MASK << 1);
+ write_csr_entryhi(address);
+ tlb_probe();
+ idx = read_csr_tlbidx();
+ write_csr_pagesize(PS_HUGE_SIZE);
+ lo = pmd_to_entrylo(pte_val(*ptep));
+ write_csr_entrylo0(lo);
+ write_csr_entrylo1(lo + (HPAGE_SIZE >> 1));
+
+ if (idx < 0)
+ tlb_write_random();
+ else
+ tlb_write_indexed();
+ write_csr_pagesize(PS_DEFAULT_SIZE);
+
+ local_irq_restore(flags);
+#endif
+}
+
+void __update_tlb(struct vm_area_struct *vma, unsigned long address, pte_t *ptep)
+{
+ int idx;
+ unsigned long flags;
+
+ /*
+ * Handle debugger faulting in for debugee.
+ */
+ if (current->active_mm != vma->vm_mm)
+ return;
+
+ if (pte_val(*ptep) & _PAGE_HUGE) {
+ __update_hugetlb(vma, address, ptep);
+ return;
+ }
+
+ local_irq_save(flags);
+
+ if ((unsigned long)ptep & sizeof(pte_t))
+ ptep--;
+
+ address &= (PAGE_MASK << 1);
+ write_csr_entryhi(address);
+ tlb_probe();
+ idx = read_csr_tlbidx();
+ write_csr_pagesize(PS_DEFAULT_SIZE);
+ write_csr_entrylo0(pte_val(*ptep++));
+ write_csr_entrylo1(pte_val(*ptep));
+ if (idx < 0)
+ tlb_write_random();
+ else
+ tlb_write_indexed();
+
+ local_irq_restore(flags);
+}
+
+static void setup_ptwalker(void)
+{
+ unsigned long pwctl0, pwctl1;
+ unsigned long pgd_i = 0, pgd_w = 0;
+ unsigned long pud_i = 0, pud_w = 0;
+ unsigned long pmd_i = 0, pmd_w = 0;
+ unsigned long pte_i = 0, pte_w = 0;
+
+ pgd_i = PGDIR_SHIFT;
+ pgd_w = PAGE_SHIFT - 3;
+#if CONFIG_PGTABLE_LEVELS > 3
+ pud_i = PUD_SHIFT;
+ pud_w = PAGE_SHIFT - 3;
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+ pmd_i = PMD_SHIFT;
+ pmd_w = PAGE_SHIFT - 3;
+#endif
+ pte_i = PAGE_SHIFT;
+ pte_w = PAGE_SHIFT - 3;
+
+ pwctl0 = pte_i | pte_w << 5 | pmd_i << 10 | pmd_w << 15 | pud_i << 20 | pud_w << 25;
+ pwctl1 = pgd_i | pgd_w << 6;
+
+ csr_writeq(pwctl0, LOONGARCH_CSR_PWCTL0);
+ csr_writeq(pwctl1, LOONGARCH_CSR_PWCTL1);
+ csr_writeq((long)swapper_pg_dir, LOONGARCH_CSR_PGDH);
+ csr_writeq((long)invalid_pg_dir, LOONGARCH_CSR_PGDL);
+ csr_writeq((long)smp_processor_id(), LOONGARCH_CSR_TMID);
+}
+
+static void output_pgtable_bits_defines(void)
+{
+#define pr_define(fmt, ...) \
+ pr_debug("#define " fmt, ##__VA_ARGS__)
+
+ pr_debug("#include <asm/asm.h>\n");
+ pr_debug("#include <asm/regdef.h>\n");
+ pr_debug("\n");
+
+ pr_define("_PAGE_VALID_SHIFT %d\n", _PAGE_VALID_SHIFT);
+ pr_define("_PAGE_DIRTY_SHIFT %d\n", _PAGE_DIRTY_SHIFT);
+ pr_define("_PAGE_HUGE_SHIFT %d\n", _PAGE_HUGE_SHIFT);
+ pr_define("_PAGE_GLOBAL_SHIFT %d\n", _PAGE_GLOBAL_SHIFT);
+ pr_define("_PAGE_PRESENT_SHIFT %d\n", _PAGE_PRESENT_SHIFT);
+ pr_define("_PAGE_WRITE_SHIFT %d\n", _PAGE_WRITE_SHIFT);
+ pr_define("_PAGE_NO_READ_SHIFT %d\n", _PAGE_NO_READ_SHIFT);
+ pr_define("_PAGE_NO_EXEC_SHIFT %d\n", _PAGE_NO_EXEC_SHIFT);
+ pr_define("_PFN_SHIFT %d\n", _PFN_SHIFT);
+ pr_debug("\n");
+}
+
+void setup_tlb_handler(void)
+{
+ static int run_once = 0;
+
+ setup_ptwalker();
+ output_pgtable_bits_defines();
+
+ /* The tlb handlers are generated only once */
+ if (!run_once) {
+ memcpy((void *)tlbrentry, handle_tlb_refill, 0x80);
+ local_flush_icache_range(tlbrentry, tlbrentry + 0x80);
+ set_handler(EXCCODE_TLBI * VECSIZE, handle_tlb_load, VECSIZE);
+ set_handler(EXCCODE_TLBL * VECSIZE, handle_tlb_load, VECSIZE);
+ set_handler(EXCCODE_TLBS * VECSIZE, handle_tlb_store, VECSIZE);
+ set_handler(EXCCODE_TLBM * VECSIZE, handle_tlb_modify, VECSIZE);
+ set_handler(EXCCODE_TLBNR * VECSIZE, handle_tlb_protect, VECSIZE);
+ set_handler(EXCCODE_TLBNX * VECSIZE, handle_tlb_protect, VECSIZE);
+ set_handler(EXCCODE_TLBPE * VECSIZE, handle_tlb_protect, VECSIZE);
+ run_once++;
+ }
+}
+
+void tlb_init(void)
+{
+ write_csr_pagesize(PS_DEFAULT_SIZE);
+ write_csr_stlbpgsize(PS_DEFAULT_SIZE);
+ write_csr_tlbrefill_pagesize(PS_DEFAULT_SIZE);
+ setup_tlb_handler();
+ local_flush_tlb_all();
+}
diff --git a/arch/loongarch/mm/tlbex.S b/arch/loongarch/mm/tlbex.S
new file mode 100644
index 000000000000..a4ca4e507ee8
--- /dev/null
+++ b/arch/loongarch/mm/tlbex.S
@@ -0,0 +1,477 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <asm/asm.h>
+#include <asm/export.h>
+#include <asm/loongarch.h>
+#include <asm/page.h>
+#include <asm/pgtable.h>
+#include <asm/regdef.h>
+#include <asm/stackframe.h>
+
+ .macro tlb_do_page_fault, write
+ SYM_FUNC_START(tlb_do_page_fault_\write)
+ SAVE_ALL
+ csrrd a2, LOONGARCH_CSR_BADV
+ move a0, sp
+ REG_S a2, sp, PT_BVADDR
+ li.w a1, \write
+ la.abs t0, do_page_fault
+ jirl ra, t0, 0
+ RESTORE_ALL_AND_RET
+ SYM_FUNC_END(tlb_do_page_fault_\write)
+ .endm
+
+ tlb_do_page_fault 0
+ tlb_do_page_fault 1
+
+SYM_FUNC_START(handle_tlb_protect)
+ BACKUP_T0T1
+ SAVE_ALL
+ move a0, sp
+ move a1, zero
+ csrrd a2, LOONGARCH_CSR_BADV
+ REG_S a2, sp, PT_BVADDR
+ la.abs t0, do_page_fault
+ jirl ra, t0, 0
+ RESTORE_ALL_AND_RET
+SYM_FUNC_END(handle_tlb_protect)
+
+SYM_FUNC_START(handle_tlb_load)
+ csrwr t0, EXCEPTION_KS0
+ csrwr t1, EXCEPTION_KS1
+ csrwr ra, EXCEPTION_KS2
+
+ /*
+ * The vmalloc handling is not in the hotpath.
+ */
+ csrrd t0, LOONGARCH_CSR_BADV
+ blt t0, $r0, vmalloc_load
+ csrrd t1, LOONGARCH_CSR_PGDL
+
+vmalloc_done_load:
+ /* Get PGD offset in bytes */
+ srli.d t0, t0, PGDIR_SHIFT
+ andi t0, t0, (PTRS_PER_PGD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#if CONFIG_PGTABLE_LEVELS > 3
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PUD_SHIFT
+ andi t0, t0, (PTRS_PER_PUD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PMD_SHIFT
+ andi t0, t0, (PTRS_PER_PMD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+ ld.d ra, t1, 0
+
+ /*
+ * For huge tlb entries, pmde doesn't contain an address but
+ * instead contains the tlb pte. Check the PAGE_HUGE bit and
+ * see if we need to jump to huge tlb processing.
+ */
+ andi t0, ra, _PAGE_HUGE
+ bne t0, $r0, tlb_huge_update_load
+
+ csrrd t0, LOONGARCH_CSR_BADV
+ srli.d t0, t0, (PAGE_SHIFT + PTE_ORDER)
+ andi t0, t0, (PTRS_PER_PTE - 1)
+ slli.d t0, t0, _PTE_T_LOG2
+ add.d t1, ra, t0
+
+ ld.d t0, t1, 0
+ tlbsrch
+
+ srli.d ra, t0, _PAGE_PRESENT_SHIFT
+ andi ra, ra, 1
+ beq ra, $r0, nopage_tlb_load
+
+ ori t0, t0, _PAGE_VALID
+ st.d t0, t1, 0
+ ori t1, t1, 8
+ xori t1, t1, 8
+ ld.d t0, t1, 0
+ ld.d t1, t1, 8
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ csrwr t1, LOONGARCH_CSR_TLBELO1
+ tlbwr
+leave_load:
+ csrrd t0, EXCEPTION_KS0
+ csrrd t1, EXCEPTION_KS1
+ csrrd ra, EXCEPTION_KS2
+ ertn
+#ifdef CONFIG_64BIT
+vmalloc_load:
+ la.abs t1, swapper_pg_dir
+ b vmalloc_done_load
+#endif
+
+ /*
+ * This is the entry point when build_tlbchange_handler_head
+ * spots a huge page.
+ */
+tlb_huge_update_load:
+ ld.d t0, t1, 0
+ srli.d ra, t0, _PAGE_PRESENT_SHIFT
+ andi ra, ra, 1
+ beq ra, $r0, nopage_tlb_load
+ tlbsrch
+
+ ori t0, t0, _PAGE_VALID
+ st.d t0, t1, 0
+ addu16i.d t1, $r0, -(CSR_TLBIDX_EHINV >> 16)
+ addi.d ra, t1, 0
+ csrxchg ra, t1, LOONGARCH_CSR_TLBIDX
+ tlbwr
+
+ csrxchg $r0, t1, LOONGARCH_CSR_TLBIDX
+
+ /*
+ * A huge PTE describes an area the size of the
+ * configured huge page size. This is twice the
+ * of the large TLB entry size we intend to use.
+ * A TLB entry half the size of the configured
+ * huge page size is configured into entrylo0
+ * and entrylo1 to cover the contiguous huge PTE
+ * address space.
+ */
+ /* Huge page: Move Global bit */
+ xori t0, t0, _PAGE_HUGE
+ lu12i.w t1, _PAGE_HGLOBAL >> 12
+ and t1, t0, t1
+ srli.d t1, t1, (_PAGE_HGLOBAL_SHIFT - _PAGE_GLOBAL_SHIFT)
+ or t0, t0, t1
+
+ addi.d ra, t0, 0
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ addi.d t0, ra, 0
+
+ /* Convert to entrylo1 */
+ addi.d t1, $r0, 1
+ slli.d t1, t1, (HPAGE_SHIFT - 1)
+ add.d t0, t0, t1
+ csrwr t0, LOONGARCH_CSR_TLBELO1
+
+ /* Set huge page tlb entry size */
+ addu16i.d t0, $r0, (PS_MASK >> 16)
+ addu16i.d t1, $r0, (PS_HUGE_SIZE << (PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+ tlbfill
+
+ addu16i.d t0, $r0, (PS_MASK >> 16)
+ addu16i.d t1, $r0, (PS_DEFAULT_SIZE << (PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+nopage_tlb_load:
+ csrrd ra, EXCEPTION_KS2
+ la.abs t0, tlb_do_page_fault_0
+ jirl $r0, t0, 0
+SYM_FUNC_END(handle_tlb_load)
+
+SYM_FUNC_START(handle_tlb_store)
+ csrwr t0, EXCEPTION_KS0
+ csrwr t1, EXCEPTION_KS1
+ csrwr ra, EXCEPTION_KS2
+
+ /*
+ * The vmalloc handling is not in the hotpath.
+ */
+ csrrd t0, LOONGARCH_CSR_BADV
+ blt t0, $r0, vmalloc_store
+ csrrd t1, LOONGARCH_CSR_PGDL
+
+vmalloc_done_store:
+ /* Get PGD offset in bytes */
+ srli.d t0, t0, PGDIR_SHIFT
+ andi t0, t0, (PTRS_PER_PGD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+
+#if CONFIG_PGTABLE_LEVELS > 3
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PUD_SHIFT
+ andi t0, t0, (PTRS_PER_PUD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PMD_SHIFT
+ andi t0, t0, (PTRS_PER_PMD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+ ld.d ra, t1, 0
+
+ /*
+ * For huge tlb entries, pmde doesn't contain an address but
+ * instead contains the tlb pte. Check the PAGE_HUGE bit and
+ * see if we need to jump to huge tlb processing.
+ */
+ andi t0, ra, _PAGE_HUGE
+ bne t0, $r0, tlb_huge_update_store
+
+ csrrd t0, LOONGARCH_CSR_BADV
+ srli.d t0, t0, (PAGE_SHIFT + PTE_ORDER)
+ andi t0, t0, (PTRS_PER_PTE - 1)
+ slli.d t0, t0, _PTE_T_LOG2
+ add.d t1, ra, t0
+
+ ld.d t0, t1, 0
+ tlbsrch
+
+ srli.d ra, t0, _PAGE_PRESENT_SHIFT
+ andi ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
+ xori ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
+ bne ra, $r0, nopage_tlb_store
+
+ ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+ st.d t0, t1, 0
+
+ ori t1, t1, 8
+ xori t1, t1, 8
+ ld.d t0, t1, 0
+ ld.d t1, t1, 8
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ csrwr t1, LOONGARCH_CSR_TLBELO1
+ tlbwr
+leave_store:
+ csrrd t0, EXCEPTION_KS0
+ csrrd t1, EXCEPTION_KS1
+ csrrd ra, EXCEPTION_KS2
+ ertn
+#ifdef CONFIG_64BIT
+vmalloc_store:
+ la.abs t1, swapper_pg_dir
+ b vmalloc_done_store
+#endif
+
+ /*
+ * This is the entry point when build_tlbchange_handler_head
+ * spots a huge page.
+ */
+tlb_huge_update_store:
+ ld.d t0, t1, 0
+ srli.d ra, t0, _PAGE_PRESENT_SHIFT
+ andi ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
+ xori ra, ra, ((_PAGE_PRESENT | _PAGE_WRITE) >> _PAGE_PRESENT_SHIFT)
+ bne ra, $r0, nopage_tlb_store
+
+ tlbsrch
+ ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+
+ st.d t0, t1, 0
+ addu16i.d t1, $r0, -(CSR_TLBIDX_EHINV >> 16)
+ addi.d ra, t1, 0
+ csrxchg ra, t1, LOONGARCH_CSR_TLBIDX
+ tlbwr
+
+ csrxchg $r0, t1, LOONGARCH_CSR_TLBIDX
+ /*
+ * A huge PTE describes an area the size of the
+ * configured huge page size. This is twice the
+ * of the large TLB entry size we intend to use.
+ * A TLB entry half the size of the configured
+ * huge page size is configured into entrylo0
+ * and entrylo1 to cover the contiguous huge PTE
+ * address space.
+ */
+ /* Huge page: Move Global bit */
+ xori t0, t0, _PAGE_HUGE
+ lu12i.w t1, _PAGE_HGLOBAL >> 12
+ and t1, t0, t1
+ srli.d t1, t1, (_PAGE_HGLOBAL_SHIFT - _PAGE_GLOBAL_SHIFT)
+ or t0, t0, t1
+
+ addi.d ra, t0, 0
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ addi.d t0, ra, 0
+
+ /* Convert to entrylo1 */
+ addi.d t1, $r0, 1
+ slli.d t1, t1, (HPAGE_SHIFT - 1)
+ add.d t0, t0, t1
+ csrwr t0, LOONGARCH_CSR_TLBELO1
+
+ /* Set huge page tlb entry size */
+ addu16i.d t0, $r0, (PS_MASK >> 16)
+ addu16i.d t1, $r0, (PS_HUGE_SIZE << (PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+ tlbfill
+
+ /* Reset default page size */
+ addu16i.d t0, $r0, (PS_MASK >> 16)
+ addu16i.d t1, $r0, (PS_DEFAULT_SIZE << (PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+nopage_tlb_store:
+ csrrd ra, EXCEPTION_KS2
+ la.abs t0, tlb_do_page_fault_1
+ jirl $r0, t0, 0
+SYM_FUNC_END(handle_tlb_store)
+
+SYM_FUNC_START(handle_tlb_modify)
+ csrwr t0, EXCEPTION_KS0
+ csrwr t1, EXCEPTION_KS1
+ csrwr ra, EXCEPTION_KS2
+
+ /*
+ * The vmalloc handling is not in the hotpath.
+ */
+ csrrd t0, LOONGARCH_CSR_BADV
+ blt t0, $r0, vmalloc_modify
+ csrrd t1, LOONGARCH_CSR_PGDL
+
+vmalloc_done_modify:
+ /* Get PGD offset in bytes */
+ srli.d t0, t0, PGDIR_SHIFT
+ andi t0, t0, (PTRS_PER_PGD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#if CONFIG_PGTABLE_LEVELS > 3
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PUD_SHIFT
+ andi t0, t0, (PTRS_PER_PUD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+ csrrd t0, LOONGARCH_CSR_BADV
+ ld.d t1, t1, 0
+ srli.d t0, t0, PMD_SHIFT
+ andi t0, t0, (PTRS_PER_PMD - 1)
+ slli.d t0, t0, 3
+ add.d t1, t1, t0
+#endif
+ ld.d ra, t1, 0
+
+ /*
+ * For huge tlb entries, pmde doesn't contain an address but
+ * instead contains the tlb pte. Check the PAGE_HUGE bit and
+ * see if we need to jump to huge tlb processing.
+ */
+ andi t0, ra, _PAGE_HUGE
+ bne t0, $r0, tlb_huge_update_modify
+
+ csrrd t0, LOONGARCH_CSR_BADV
+ srli.d t0, t0, (PAGE_SHIFT + PTE_ORDER)
+ andi t0, t0, (PTRS_PER_PTE - 1)
+ slli.d t0, t0, _PTE_T_LOG2
+ add.d t1, ra, t0
+
+ ld.d t0, t1, 0
+ tlbsrch
+
+ srli.d ra, t0, _PAGE_WRITE_SHIFT
+ andi ra, ra, 1
+ beq ra, $r0, nopage_tlb_modify
+
+ ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+ st.d t0, t1, 0
+ ori t1, t1, 8
+ xori t1, t1, 8
+ ld.d t0, t1, 0
+ ld.d t1, t1, 8
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ csrwr t1, LOONGARCH_CSR_TLBELO1
+ tlbwr
+leave_modify:
+ csrrd t0, EXCEPTION_KS0
+ csrrd t1, EXCEPTION_KS1
+ csrrd ra, EXCEPTION_KS2
+ ertn
+#ifdef CONFIG_64BIT
+vmalloc_modify:
+ la.abs t1, swapper_pg_dir
+ b vmalloc_done_modify
+#endif
+
+ /*
+ * This is the entry point when
+ * build_tlbchange_handler_head spots a huge page.
+ */
+tlb_huge_update_modify:
+ ld.d t0, t1, 0
+
+ srli.d ra, t0, _PAGE_WRITE_SHIFT
+ andi ra, ra, 1
+ beq ra, $r0, nopage_tlb_modify
+
+ tlbsrch
+ ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
+
+ st.d t0, t1, 0
+ /*
+ * A huge PTE describes an area the size of the
+ * configured huge page size. This is twice the
+ * of the large TLB entry size we intend to use.
+ * A TLB entry half the size of the configured
+ * huge page size is configured into entrylo0
+ * and entrylo1 to cover the contiguous huge PTE
+ * address space.
+ */
+ /* Huge page: Move Global bit */
+ xori t0, t0, _PAGE_HUGE
+ lu12i.w t1, _PAGE_HGLOBAL >> 12
+ and t1, t0, t1
+ srli.d t1, t1, (_PAGE_HGLOBAL_SHIFT - _PAGE_GLOBAL_SHIFT)
+ or t0, t0, t1
+
+ addi.d ra, t0, 0
+ csrwr t0, LOONGARCH_CSR_TLBELO0
+ addi.d t0, ra, 0
+
+ /* Convert to entrylo1 */
+ addi.d t1, $r0, 1
+ slli.d t1, t1, (HPAGE_SHIFT - 1)
+ add.d t0, t0, t1
+ csrwr t0, LOONGARCH_CSR_TLBELO1
+
+ /* Set huge page tlb entry size */
+ addu16i.d t0, $r0, (PS_MASK >> 16)
+ addu16i.d t1, $r0, (PS_HUGE_SIZE << (PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+ tlbwr
+
+ /* Reset default page size */
+ addu16i.d t0, $r0, (PS_MASK >> 16)
+ addu16i.d t1, $r0, (PS_DEFAULT_SIZE << (PS_SHIFT - 16))
+ csrxchg t1, t0, LOONGARCH_CSR_TLBIDX
+
+nopage_tlb_modify:
+ csrrd ra, EXCEPTION_KS2
+ la.abs t0, tlb_do_page_fault_1
+ jirl $r0, t0, 0
+SYM_FUNC_END(handle_tlb_modify)
+
+SYM_FUNC_START(handle_tlb_refill)
+ csrwr t0, LOONGARCH_CSR_TLBRSAVE
+ csrrd t0, LOONGARCH_CSR_PGD
+ lddir t0, t0, 3
+#if CONFIG_PGTABLE_LEVELS > 3
+ lddir t0, t0, 2
+#endif
+#if CONFIG_PGTABLE_LEVELS > 2
+ lddir t0, t0, 1
+#endif
+ ldpte t0, 0
+ ldpte t0, 1
+ tlbfill
+ csrrd t0, LOONGARCH_CSR_TLBRSAVE
+ ertn
+SYM_FUNC_END(handle_tlb_refill)
--
2.27.0


2022-05-18 10:02:51

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 14/22] LoongArch: Add signal handling support

Add ucontext/sigcontext definition and signal handling support for
LoongArch.

Cc: Eric Biederman <[email protected]>
Cc: Al Viro <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/uapi/asm/sigcontext.h | 44 ++
arch/loongarch/include/uapi/asm/signal.h | 13 +
arch/loongarch/include/uapi/asm/ucontext.h | 35 ++
arch/loongarch/kernel/signal.c | 566 +++++++++++++++++++
4 files changed, 658 insertions(+)
create mode 100644 arch/loongarch/include/uapi/asm/sigcontext.h
create mode 100644 arch/loongarch/include/uapi/asm/signal.h
create mode 100644 arch/loongarch/include/uapi/asm/ucontext.h
create mode 100644 arch/loongarch/kernel/signal.c

diff --git a/arch/loongarch/include/uapi/asm/sigcontext.h b/arch/loongarch/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..be3d3c6ac83e
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/sigcontext.h
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
+/*
+ * Author: Hanlu Li <[email protected]>
+ * Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _UAPI_ASM_SIGCONTEXT_H
+#define _UAPI_ASM_SIGCONTEXT_H
+
+#include <linux/types.h>
+#include <linux/posix_types.h>
+
+/* FP context was used */
+#define SC_USED_FP (1 << 0)
+/* Address error was due to memory load */
+#define SC_ADDRERR_RD (1 << 30)
+/* Address error was due to memory store */
+#define SC_ADDRERR_WR (1 << 31)
+
+struct sigcontext {
+ __u64 sc_pc;
+ __u64 sc_regs[32];
+ __u32 sc_flags;
+ __u64 sc_extcontext[0] __attribute__((__aligned__(16)));
+};
+
+#define CONTEXT_INFO_ALIGN 16
+struct _ctxinfo {
+ __u32 magic;
+ __u32 size;
+ __u64 padding; /* padding to 16 bytes */
+};
+
+/* FPU context */
+#define FPU_CTX_MAGIC 0x46505501
+#define FPU_CTX_ALIGN 8
+struct fpu_context {
+ __u64 regs[32];
+ __u64 fcc;
+ __u32 fcsr;
+};
+
+#endif /* _UAPI_ASM_SIGCONTEXT_H */
diff --git a/arch/loongarch/include/uapi/asm/signal.h b/arch/loongarch/include/uapi/asm/signal.h
new file mode 100644
index 000000000000..992d965aa13f
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/signal.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _UAPI_ASM_SIGNAL_H
+#define _UAPI_ASM_SIGNAL_H
+
+#define MINSIGSTKSZ 4096
+#define SIGSTKSZ 16384
+
+#include <asm-generic/signal.h>
+
+#endif
diff --git a/arch/loongarch/include/uapi/asm/ucontext.h b/arch/loongarch/include/uapi/asm/ucontext.h
new file mode 100644
index 000000000000..12577e22b1c7
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/ucontext.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __LOONGARCH_UAPI_ASM_UCONTEXT_H
+#define __LOONGARCH_UAPI_ASM_UCONTEXT_H
+
+/**
+ * struct ucontext - user context structure
+ * @uc_flags:
+ * @uc_link:
+ * @uc_stack:
+ * @uc_mcontext: holds basic processor state
+ * @uc_sigmask:
+ * @uc_extcontext: holds extended processor state
+ */
+struct ucontext {
+ unsigned long uc_flags;
+ struct ucontext *uc_link;
+ stack_t uc_stack;
+ sigset_t uc_sigmask;
+ /* There's some padding here to allow sigset_t to be expanded in the
+ * future. Though this is unlikely, other architectures put uc_sigmask
+ * at the end of this structure and explicitly state it can be
+ * expanded, so we didn't want to box ourselves in here. */
+ __u8 __unused[1024 / 8 - sizeof(sigset_t)];
+ /* We can't put uc_sigmask at the end of this structure because we need
+ * to be able to expand sigcontext in the future. For example, the
+ * vector ISA extension will almost certainly add ISA state. We want
+ * to ensure all user-visible ISA state can be saved and restored via a
+ * ucontext, so we're putting this at the end in order to allow for
+ * infinite extensibility. Since we know this will be extended and we
+ * assume sigset_t won't be extended an extreme amount, we're
+ * prioritizing this. */
+ struct sigcontext uc_mcontext;
+};
+
+#endif /* __LOONGARCH_UAPI_ASM_UCONTEXT_H */
diff --git a/arch/loongarch/kernel/signal.c b/arch/loongarch/kernel/signal.c
new file mode 100644
index 000000000000..7ffc77d56988
--- /dev/null
+++ b/arch/loongarch/kernel/signal.c
@@ -0,0 +1,566 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Author: Hanlu Li <[email protected]>
+ * Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1991, 1992 Linus Torvalds
+ * Copyright (C) 1994 - 2000 Ralf Baechle
+ * Copyright (C) 1999, 2000 Silicon Graphics, Inc.
+ * Copyright (C) 2014, Imagination Technologies Ltd.
+ */
+#include <linux/audit.h>
+#include <linux/cache.h>
+#include <linux/context_tracking.h>
+#include <linux/irqflags.h>
+#include <linux/sched.h>
+#include <linux/mm.h>
+#include <linux/personality.h>
+#include <linux/smp.h>
+#include <linux/kernel.h>
+#include <linux/signal.h>
+#include <linux/errno.h>
+#include <linux/wait.h>
+#include <linux/ptrace.h>
+#include <linux/unistd.h>
+#include <linux/compiler.h>
+#include <linux/syscalls.h>
+#include <linux/uaccess.h>
+
+#include <asm/asm.h>
+#include <asm/cacheflush.h>
+#include <asm/cpu-features.h>
+#include <asm/fpu.h>
+#include <asm/ucontext.h>
+#include <asm/vdso.h>
+
+#ifdef DEBUG_SIG
+# define DEBUGP(fmt, args...) printk("%s: " fmt, __func__, ##args)
+#else
+# define DEBUGP(fmt, args...)
+#endif
+
+/* Make sure we will not lose FPU ownership */
+#define lock_fpu_owner() ({ preempt_disable(); pagefault_disable(); })
+#define unlock_fpu_owner() ({ pagefault_enable(); preempt_enable(); })
+
+/* Assembly functions to move context to/from the FPU */
+extern asmlinkage int
+_save_fp_context(void __user *fpregs, void __user *fcc, void __user *csr);
+extern asmlinkage int
+_restore_fp_context(void __user *fpregs, void __user *fcc, void __user *csr);
+
+struct rt_sigframe {
+ struct siginfo rs_info;
+ struct ucontext rs_uctx;
+};
+
+struct _ctx_layout {
+ struct _ctxinfo *addr;
+ unsigned int size;
+};
+
+struct extctx_layout {
+ unsigned long size;
+ unsigned int flags;
+ struct _ctx_layout fpu;
+ struct _ctx_layout end;
+};
+
+static void __user *get_ctx_through_ctxinfo(struct _ctxinfo *info)
+{
+ return (void __user *)((char *)info + sizeof(struct _ctxinfo));
+}
+
+/*
+ * Thread saved context copy to/from a signal context presumed to be on the
+ * user stack, and therefore accessed with appropriate macros from uaccess.h.
+ */
+static int copy_fpu_to_sigcontext(struct fpu_context __user *ctx)
+{
+ int i;
+ int err = 0;
+ uint64_t __user *regs = (uint64_t *)&ctx->regs;
+ uint64_t __user *fcc = &ctx->fcc;
+ uint32_t __user *fcsr = &ctx->fcsr;
+
+ for (i = 0; i < NUM_FPU_REGS; i++) {
+ err |=
+ __put_user(get_fpr64(&current->thread.fpu.fpr[i], 0),
+ &regs[i]);
+ }
+ err |= __put_user(current->thread.fpu.fcc, fcc);
+ err |= __put_user(current->thread.fpu.fcsr, fcsr);
+
+ return err;
+}
+
+static int copy_fpu_from_sigcontext(struct fpu_context __user *ctx)
+{
+ int i;
+ int err = 0;
+ u64 fpr_val;
+ uint64_t __user *regs = (uint64_t *)&ctx->regs;
+ uint64_t __user *fcc = &ctx->fcc;
+ uint32_t __user *fcsr = &ctx->fcsr;
+
+ for (i = 0; i < NUM_FPU_REGS; i++) {
+ err |= __get_user(fpr_val, &regs[i]);
+ set_fpr64(&current->thread.fpu.fpr[i], 0, fpr_val);
+ }
+ err |= __get_user(current->thread.fpu.fcc, fcc);
+ err |= __get_user(current->thread.fpu.fcsr, fcsr);
+
+ return err;
+}
+
+/*
+ * Wrappers for the assembly _{save,restore}_fp_context functions.
+ */
+static int save_hw_fpu_context(struct fpu_context __user *ctx)
+{
+ uint64_t __user *regs = (uint64_t *)&ctx->regs;
+ uint64_t __user *fcc = &ctx->fcc;
+ uint32_t __user *fcsr = &ctx->fcsr;
+
+ return _save_fp_context(regs, fcc, fcsr);
+}
+
+static int restore_hw_fpu_context(struct fpu_context __user *ctx)
+{
+ uint64_t __user *regs = (uint64_t *)&ctx->regs;
+ uint64_t __user *fcc = &ctx->fcc;
+ uint32_t __user *fcsr = &ctx->fcsr;
+
+ return _restore_fp_context(regs, fcc, fcsr);
+}
+
+static int fcsr_pending(unsigned int __user *fcsr)
+{
+ int err, sig = 0;
+ unsigned int csr, enabled;
+
+ err = __get_user(csr, fcsr);
+ enabled = ((csr & FPU_CSR_ALL_E) << 24);
+ /*
+ * If the signal handler set some FPU exceptions, clear it and
+ * send SIGFPE.
+ */
+ if (csr & enabled) {
+ csr &= ~enabled;
+ err |= __put_user(csr, fcsr);
+ sig = SIGFPE;
+ }
+ return err ?: sig;
+}
+
+/*
+ * Helper routines
+ */
+static int protected_save_fpu_context(struct extctx_layout *extctx)
+{
+ int err = 0;
+ struct _ctxinfo __user *info = extctx->fpu.addr;
+ struct fpu_context __user *fpu_ctx = (struct fpu_context *)get_ctx_through_ctxinfo(info);
+ uint64_t __user *regs = (uint64_t *)&fpu_ctx->regs;
+ uint64_t __user *fcc = &fpu_ctx->fcc;
+ uint32_t __user *fcsr = &fpu_ctx->fcsr;
+
+ while (1) {
+ lock_fpu_owner();
+ if (is_fpu_owner())
+ err = save_hw_fpu_context(fpu_ctx);
+ else
+ err = copy_fpu_to_sigcontext(fpu_ctx);
+ unlock_fpu_owner();
+
+ err |= __put_user(FPU_CTX_MAGIC, &info->magic);
+ err |= __put_user(extctx->fpu.size, &info->size);
+
+ if (likely(!err))
+ break;
+ /* Touch the FPU context and try again */
+ err = __put_user(0, &regs[0]) |
+ __put_user(0, &regs[31]) |
+ __put_user(0, fcc) |
+ __put_user(0, fcsr);
+ if (err)
+ return err; /* really bad sigcontext */
+ }
+
+ return err;
+}
+
+static int protected_restore_fpu_context(struct extctx_layout *extctx)
+{
+ int err = 0, sig = 0, tmp __maybe_unused;
+ struct _ctxinfo __user *info = extctx->fpu.addr;
+ struct fpu_context __user *fpu_ctx = (struct fpu_context *)get_ctx_through_ctxinfo(info);
+ uint64_t __user *regs = (uint64_t *)&fpu_ctx->regs;
+ uint64_t __user *fcc = &fpu_ctx->fcc;
+ uint32_t __user *fcsr = &fpu_ctx->fcsr;
+
+ err = sig = fcsr_pending(fcsr);
+ if (err < 0)
+ return err;
+
+ while (1) {
+ lock_fpu_owner();
+ if (is_fpu_owner())
+ err = restore_hw_fpu_context(fpu_ctx);
+ else
+ err = copy_fpu_from_sigcontext(fpu_ctx);
+ unlock_fpu_owner();
+
+ if (likely(!err))
+ break;
+ /* Touch the FPU context and try again */
+ err = __get_user(tmp, &regs[0]) |
+ __get_user(tmp, &regs[31]) |
+ __get_user(tmp, fcc) |
+ __get_user(tmp, fcsr);
+ if (err)
+ break; /* really bad sigcontext */
+ }
+
+ return err ?: sig;
+}
+
+static int setup_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc,
+ struct extctx_layout *extctx)
+{
+ int i, err = 0;
+ struct _ctxinfo __user *info;
+
+ err |= __put_user(regs->csr_era, &sc->sc_pc);
+ err |= __put_user(extctx->flags, &sc->sc_flags);
+
+ err |= __put_user(0, &sc->sc_regs[0]);
+ for (i = 1; i < 32; i++)
+ err |= __put_user(regs->regs[i], &sc->sc_regs[i]);
+
+ if (extctx->fpu.addr)
+ err |= protected_save_fpu_context(extctx);
+
+ /* Set the "end" magic */
+ info = (struct _ctxinfo *)extctx->end.addr;
+ err |= __put_user(0, &info->magic);
+ err |= __put_user(0, &info->size);
+
+ return err;
+}
+
+static int parse_extcontext(struct sigcontext __user *sc, struct extctx_layout *extctx)
+{
+ int err = 0;
+ unsigned int magic, size;
+ struct _ctxinfo __user *info = (struct _ctxinfo __user *)&sc->sc_extcontext;
+
+ while(1) {
+ err |= __get_user(magic, &info->magic);
+ err |= __get_user(size, &info->size);
+ if (err)
+ return err;
+
+ switch (magic) {
+ case 0: /* END */
+ goto done;
+
+ case FPU_CTX_MAGIC:
+ if (size < (sizeof(struct _ctxinfo) +
+ sizeof(struct fpu_context)))
+ goto invalid;
+ extctx->fpu.addr = info;
+ break;
+
+ default:
+ goto invalid;
+ }
+
+ info = (struct _ctxinfo *)((char *)info + size);
+ }
+
+done:
+ return 0;
+
+invalid:
+ return -EINVAL;
+}
+
+static int restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc)
+{
+ int i, err = 0;
+ struct extctx_layout extctx;
+
+ memset(&extctx, 0, sizeof(struct extctx_layout));
+
+ err = __get_user(extctx.flags, &sc->sc_flags);
+ if (err)
+ goto bad;
+
+ err = parse_extcontext(sc, &extctx);
+ if (err)
+ goto bad;
+
+ conditional_used_math(extctx.flags & SC_USED_FP);
+
+ /*
+ * The signal handler may have used FPU; give it up if the program
+ * doesn't want it following sigreturn.
+ */
+ if (!(extctx.flags & SC_USED_FP))
+ lose_fpu(0);
+
+ /* Always make any pending restarted system calls return -EINTR */
+ current->restart_block.fn = do_no_restart_syscall;
+
+ err |= __get_user(regs->csr_era, &sc->sc_pc);
+ for (i = 1; i < 32; i++)
+ err |= __get_user(regs->regs[i], &sc->sc_regs[i]);
+
+ if (extctx.fpu.addr)
+ err |= protected_restore_fpu_context(&extctx);
+
+bad:
+ return err;
+}
+
+static unsigned int handle_flags(void)
+{
+ unsigned int flags = 0;
+
+ flags = used_math() ? SC_USED_FP : 0;
+
+ switch (current->thread.error_code) {
+ case 1:
+ flags |= SC_ADDRERR_RD;
+ break;
+ case 2:
+ flags |= SC_ADDRERR_WR;
+ break;
+ }
+
+ return flags;
+}
+
+static unsigned long extframe_alloc(struct extctx_layout *extctx,
+ struct _ctx_layout *layout,
+ size_t size, unsigned int align, unsigned long base)
+{
+ unsigned long new_base = base - size;
+
+ new_base = round_down(new_base, (align < 16 ? 16 : align));
+ new_base -= sizeof(struct _ctxinfo);
+
+ layout->addr = (void *)new_base;
+ layout->size = (unsigned int)(base - new_base);
+ extctx->size += layout->size;
+
+ return new_base;
+}
+
+static unsigned long setup_extcontext(struct extctx_layout *extctx, unsigned long sp)
+{
+ unsigned long new_sp = sp;
+
+ memset(extctx, 0, sizeof(struct extctx_layout));
+
+ extctx->flags = handle_flags();
+
+ /* Grow down, alloc "end" context info first. */
+ new_sp -= sizeof(struct _ctxinfo);
+ extctx->end.addr = (void *)new_sp;
+ extctx->end.size = (unsigned int)sizeof(struct _ctxinfo);
+ extctx->size += extctx->end.size;
+
+ if (extctx->flags & SC_USED_FP) {
+ if (cpu_has_fpu)
+ new_sp = extframe_alloc(extctx, &extctx->fpu,
+ sizeof(struct fpu_context), FPU_CTX_ALIGN, new_sp);
+ }
+
+ return new_sp;
+}
+
+void __user *get_sigframe(struct ksignal *ksig, struct pt_regs *regs,
+ struct extctx_layout *extctx)
+{
+ unsigned long sp;
+
+ /* Default to using normal stack */
+ sp = regs->regs[3];
+
+ /*
+ * If we are on the alternate signal stack and would overflow it, don't.
+ * Return an always-bogus address instead so we will die with SIGSEGV.
+ */
+ if (on_sig_stack(sp) &&
+ !likely(on_sig_stack(sp - sizeof(struct rt_sigframe))))
+ return (void __user __force *)(-1UL);
+
+ sp = sigsp(sp, ksig);
+ sp = round_down(sp, 16);
+ sp = setup_extcontext(extctx, sp);
+ sp -= sizeof(struct rt_sigframe);
+
+ if (!IS_ALIGNED(sp, 16))
+ BUG();
+
+ return (void __user *)sp;
+}
+
+/*
+ * Atomically swap in the new signal mask, and wait for a signal.
+ */
+
+asmlinkage long sys_rt_sigreturn(void)
+{
+ int sig;
+ sigset_t set;
+ struct pt_regs *regs;
+ struct rt_sigframe __user *frame;
+
+ regs = current_pt_regs();
+ frame = (struct rt_sigframe __user *)regs->regs[3];
+ if (!access_ok(frame, sizeof(*frame)))
+ goto badframe;
+ if (__copy_from_user(&set, &frame->rs_uctx.uc_sigmask, sizeof(set)))
+ goto badframe;
+
+ set_current_blocked(&set);
+
+ sig = restore_sigcontext(regs, &frame->rs_uctx.uc_mcontext);
+ if (sig < 0)
+ goto badframe;
+ else if (sig)
+ force_sig(sig);
+
+ regs->regs[0] = 0; /* No syscall restarting */
+ if (restore_altstack(&frame->rs_uctx.uc_stack))
+ goto badframe;
+
+ return regs->regs[4];
+
+badframe:
+ force_sig(SIGSEGV);
+ return 0;
+}
+
+static int setup_rt_frame(void *sig_return, struct ksignal *ksig,
+ struct pt_regs *regs, sigset_t *set)
+{
+ int err = 0;
+ struct extctx_layout extctx;
+ struct rt_sigframe __user *frame;
+
+ frame = get_sigframe(ksig, regs, &extctx);
+ if (!access_ok(frame, sizeof(*frame) + extctx.size))
+ return -EFAULT;
+
+ /* Create siginfo. */
+ err |= copy_siginfo_to_user(&frame->rs_info, &ksig->info);
+
+ /* Create the ucontext. */
+ err |= __put_user(0, &frame->rs_uctx.uc_flags);
+ err |= __put_user(NULL, &frame->rs_uctx.uc_link);
+ err |= __save_altstack(&frame->rs_uctx.uc_stack, regs->regs[3]);
+ err |= setup_sigcontext(regs, &frame->rs_uctx.uc_mcontext, &extctx);
+ err |= __copy_to_user(&frame->rs_uctx.uc_sigmask, set, sizeof(*set));
+
+ if (err)
+ return -EFAULT;
+
+ /*
+ * Arguments to signal handler:
+ *
+ * a0 = signal number
+ * a1 = pointer to siginfo
+ * a2 = pointer to ucontext
+ *
+ * c0_era point to the signal handler, $r3 (sp) points to
+ * the struct rt_sigframe.
+ */
+ regs->regs[4] = ksig->sig;
+ regs->regs[5] = (unsigned long) &frame->rs_info;
+ regs->regs[6] = (unsigned long) &frame->rs_uctx;
+ regs->regs[3] = (unsigned long) frame;
+ regs->regs[1] = (unsigned long) sig_return;
+ regs->csr_era = (unsigned long) ksig->ka.sa.sa_handler;
+
+ DEBUGP("SIG deliver (%s:%d): sp=0x%p pc=0x%lx ra=0x%lx\n",
+ current->comm, current->pid,
+ frame, regs->csr_era, regs->regs[1]);
+
+ return 0;
+}
+
+static void handle_signal(struct ksignal *ksig, struct pt_regs *regs)
+{
+ int ret;
+ sigset_t *oldset = sigmask_to_save();
+ void *vdso = current->mm->context.vdso;
+
+ /* Are we from a system call? */
+ if (regs->regs[0]) {
+ switch (regs->regs[4]) {
+ case -ERESTART_RESTARTBLOCK:
+ case -ERESTARTNOHAND:
+ regs->regs[4] = -EINTR;
+ break;
+ case -ERESTARTSYS:
+ if (!(ksig->ka.sa.sa_flags & SA_RESTART)) {
+ regs->regs[4] = -EINTR;
+ break;
+ }
+ fallthrough;
+ case -ERESTARTNOINTR:
+ regs->regs[4] = regs->orig_a0;
+ regs->csr_era -= 4;
+ }
+
+ regs->regs[0] = 0; /* Don't deal with this again. */
+ }
+
+ rseq_signal_deliver(ksig, regs);
+
+ ret = setup_rt_frame(vdso + current->thread.vdso->offset_sigreturn, ksig, regs, oldset);
+
+ signal_setup_done(ret, ksig, 0);
+}
+
+void arch_do_signal_or_restart(struct pt_regs *regs, bool has_signal)
+{
+ struct ksignal ksig;
+
+ if (has_signal && get_signal(&ksig)) {
+ /* Whee! Actually deliver the signal. */
+ handle_signal(&ksig, regs);
+ return;
+ }
+
+ /* Are we from a system call? */
+ if (regs->regs[0]) {
+ switch (regs->regs[4]) {
+ case -ERESTARTNOHAND:
+ case -ERESTARTSYS:
+ case -ERESTARTNOINTR:
+ regs->regs[4] = regs->orig_a0;
+ regs->csr_era -= 4;
+ break;
+
+ case -ERESTART_RESTARTBLOCK:
+ regs->regs[4] = regs->orig_a0;
+ regs->regs[11] = __NR_restart_syscall;
+ regs->csr_era -= 4;
+ break;
+ }
+ regs->regs[0] = 0; /* Don't deal with this again. */
+ }
+
+ /*
+ * If there's no signal to deliver, we just put the saved sigmask
+ * back
+ */
+ restore_saved_sigmask();
+}
--
2.27.0


2022-05-18 10:08:33

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 11/22] LoongArch: Add process management

Add process management support for LoongArch, including: thread info
definition, context switch and process tracing.

Reviewed-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/fpu.h | 129 +++++++
arch/loongarch/include/asm/idle.h | 9 +
arch/loongarch/include/asm/mmu.h | 16 +
arch/loongarch/include/asm/mmu_context.h | 152 ++++++++
arch/loongarch/include/asm/processor.h | 209 +++++++++++
arch/loongarch/include/asm/ptrace.h | 152 ++++++++
arch/loongarch/include/asm/switch_to.h | 37 ++
arch/loongarch/include/asm/thread_info.h | 106 ++++++
arch/loongarch/include/uapi/asm/ptrace.h | 52 +++
arch/loongarch/kernel/fpu.S | 261 ++++++++++++++
arch/loongarch/kernel/idle.c | 16 +
arch/loongarch/kernel/process.c | 260 ++++++++++++++
arch/loongarch/kernel/ptrace.c | 431 +++++++++++++++++++++++
arch/loongarch/kernel/switch.S | 35 ++
14 files changed, 1865 insertions(+)
create mode 100644 arch/loongarch/include/asm/fpu.h
create mode 100644 arch/loongarch/include/asm/idle.h
create mode 100644 arch/loongarch/include/asm/mmu.h
create mode 100644 arch/loongarch/include/asm/mmu_context.h
create mode 100644 arch/loongarch/include/asm/processor.h
create mode 100644 arch/loongarch/include/asm/ptrace.h
create mode 100644 arch/loongarch/include/asm/switch_to.h
create mode 100644 arch/loongarch/include/asm/thread_info.h
create mode 100644 arch/loongarch/include/uapi/asm/ptrace.h
create mode 100644 arch/loongarch/kernel/fpu.S
create mode 100644 arch/loongarch/kernel/idle.c
create mode 100644 arch/loongarch/kernel/process.c
create mode 100644 arch/loongarch/kernel/ptrace.c
create mode 100644 arch/loongarch/kernel/switch.S

diff --git a/arch/loongarch/include/asm/fpu.h b/arch/loongarch/include/asm/fpu.h
new file mode 100644
index 000000000000..3e1c6f01aec5
--- /dev/null
+++ b/arch/loongarch/include/asm/fpu.h
@@ -0,0 +1,129 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_FPU_H
+#define _ASM_FPU_H
+
+#include <linux/sched.h>
+#include <linux/sched/task_stack.h>
+#include <linux/ptrace.h>
+#include <linux/thread_info.h>
+#include <linux/bitops.h>
+
+#include <asm/cpu.h>
+#include <asm/cpu-features.h>
+#include <asm/current.h>
+#include <asm/loongarch.h>
+#include <asm/processor.h>
+#include <asm/ptrace.h>
+
+struct sigcontext;
+
+extern void _init_fpu(unsigned int);
+extern void _save_fp(struct loongarch_fpu *);
+extern void _restore_fp(struct loongarch_fpu *);
+
+/*
+ * Mask the FCSR Cause bits according to the Enable bits, observing
+ * that Unimplemented is always enabled.
+ */
+static inline unsigned long mask_fcsr_x(unsigned long fcsr)
+{
+ return fcsr & ((fcsr & FPU_CSR_ALL_E) <<
+ (ffs(FPU_CSR_ALL_X) - ffs(FPU_CSR_ALL_E)));
+}
+
+static inline int is_fp_enabled(void)
+{
+ return (csr_readl(LOONGARCH_CSR_EUEN) & CSR_EUEN_FPEN) ?
+ 1 : 0;
+}
+
+#define enable_fpu() set_csr_euen(CSR_EUEN_FPEN)
+
+#define disable_fpu() clear_csr_euen(CSR_EUEN_FPEN)
+
+#define clear_fpu_owner() clear_thread_flag(TIF_USEDFPU)
+
+static inline int is_fpu_owner(void)
+{
+ return test_thread_flag(TIF_USEDFPU);
+}
+
+static inline void __own_fpu(void)
+{
+ enable_fpu();
+ set_thread_flag(TIF_USEDFPU);
+ KSTK_EUEN(current) |= CSR_EUEN_FPEN;
+}
+
+static inline void own_fpu_inatomic(int restore)
+{
+ if (cpu_has_fpu && !is_fpu_owner()) {
+ __own_fpu();
+ if (restore)
+ _restore_fp(&current->thread.fpu);
+ }
+}
+
+static inline void own_fpu(int restore)
+{
+ preempt_disable();
+ own_fpu_inatomic(restore);
+ preempt_enable();
+}
+
+static inline void lose_fpu_inatomic(int save, struct task_struct *tsk)
+{
+ if (is_fpu_owner()) {
+ if (save)
+ _save_fp(&tsk->thread.fpu);
+ disable_fpu();
+ clear_tsk_thread_flag(tsk, TIF_USEDFPU);
+ }
+ KSTK_EUEN(tsk) &= ~(CSR_EUEN_FPEN | CSR_EUEN_LSXEN | CSR_EUEN_LASXEN);
+}
+
+static inline void lose_fpu(int save)
+{
+ preempt_disable();
+ lose_fpu_inatomic(save, current);
+ preempt_enable();
+}
+
+static inline void init_fpu(void)
+{
+ unsigned int fcsr = current->thread.fpu.fcsr;
+
+ __own_fpu();
+ _init_fpu(fcsr);
+ set_used_math();
+}
+
+static inline void save_fp(struct task_struct *tsk)
+{
+ if (cpu_has_fpu)
+ _save_fp(&tsk->thread.fpu);
+}
+
+static inline void restore_fp(struct task_struct *tsk)
+{
+ if (cpu_has_fpu)
+ _restore_fp(&tsk->thread.fpu);
+}
+
+static inline union fpureg *get_fpu_regs(struct task_struct *tsk)
+{
+ if (tsk == current) {
+ preempt_disable();
+ if (is_fpu_owner())
+ _save_fp(&current->thread.fpu);
+ preempt_enable();
+ }
+
+ return tsk->thread.fpu.fpr;
+}
+
+#endif /* _ASM_FPU_H */
diff --git a/arch/loongarch/include/asm/idle.h b/arch/loongarch/include/asm/idle.h
new file mode 100644
index 000000000000..f7f2b7dbf958
--- /dev/null
+++ b/arch/loongarch/include/asm/idle.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ASM_IDLE_H
+#define __ASM_IDLE_H
+
+#include <linux/linkage.h>
+
+extern asmlinkage void __arch_cpu_idle(void);
+
+#endif /* __ASM_IDLE_H */
diff --git a/arch/loongarch/include/asm/mmu.h b/arch/loongarch/include/asm/mmu.h
new file mode 100644
index 000000000000..0cc2d0803537
--- /dev/null
+++ b/arch/loongarch/include/asm/mmu.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_MMU_H
+#define __ASM_MMU_H
+
+#include <linux/atomic.h>
+#include <linux/spinlock.h>
+
+typedef struct {
+ u64 asid[NR_CPUS];
+ void *vdso;
+} mm_context_t;
+
+#endif /* __ASM_MMU_H */
diff --git a/arch/loongarch/include/asm/mmu_context.h b/arch/loongarch/include/asm/mmu_context.h
new file mode 100644
index 000000000000..ccd3c8a47cad
--- /dev/null
+++ b/arch/loongarch/include/asm/mmu_context.h
@@ -0,0 +1,152 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Switch a MMU context.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_MMU_CONTEXT_H
+#define _ASM_MMU_CONTEXT_H
+
+#include <linux/errno.h>
+#include <linux/sched.h>
+#include <linux/mm_types.h>
+#include <linux/smp.h>
+#include <linux/slab.h>
+
+#include <asm/cacheflush.h>
+#include <asm/tlbflush.h>
+#include <asm-generic/mm_hooks.h>
+
+/*
+ * All unused by hardware upper bits will be considered
+ * as a software asid extension.
+ */
+static inline u64 asid_version_mask(unsigned int cpu)
+{
+ return ~(u64)(cpu_asid_mask(&cpu_data[cpu]));
+}
+
+static inline u64 asid_first_version(unsigned int cpu)
+{
+ return cpu_asid_mask(&cpu_data[cpu]) + 1;
+}
+
+#define cpu_context(cpu, mm) ((mm)->context.asid[cpu])
+#define asid_cache(cpu) (cpu_data[cpu].asid_cache)
+#define cpu_asid(cpu, mm) (cpu_context((cpu), (mm)) & cpu_asid_mask(&cpu_data[cpu]))
+
+static inline int asid_valid(struct mm_struct *mm, unsigned int cpu)
+{
+ if ((cpu_context(cpu, mm) ^ asid_cache(cpu)) & asid_version_mask(cpu))
+ return 0;
+
+ return 1;
+}
+
+static inline void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk)
+{
+}
+
+/* Normal, classic get_new_mmu_context */
+static inline void
+get_new_mmu_context(struct mm_struct *mm, unsigned long cpu)
+{
+ u64 asid = asid_cache(cpu);
+
+ if (!((++asid) & cpu_asid_mask(&cpu_data[cpu])))
+ local_flush_tlb_user(); /* start new asid cycle */
+
+ cpu_context(cpu, mm) = asid_cache(cpu) = asid;
+}
+
+/*
+ * Initialize the context related info for a new mm_struct
+ * instance.
+ */
+static inline int
+init_new_context(struct task_struct *tsk, struct mm_struct *mm)
+{
+ int i;
+
+ for_each_possible_cpu(i)
+ cpu_context(i, mm) = 0;
+
+ return 0;
+}
+
+static inline void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
+ struct task_struct *tsk)
+{
+ unsigned int cpu = smp_processor_id();
+
+ /* Check if our ASID is of an older version and thus invalid */
+ if (!asid_valid(next, cpu))
+ get_new_mmu_context(next, cpu);
+
+ write_csr_asid(cpu_asid(cpu, next));
+
+ if (next != &init_mm)
+ csr_writeq((unsigned long)next->pgd, LOONGARCH_CSR_PGDL);
+ else
+ csr_writeq((unsigned long)invalid_pg_dir, LOONGARCH_CSR_PGDL);
+
+ /*
+ * Mark current->active_mm as not "active" anymore.
+ * We don't want to mislead possible IPI tlb flush routines.
+ */
+ cpumask_set_cpu(cpu, mm_cpumask(next));
+}
+
+#define switch_mm_irqs_off switch_mm_irqs_off
+
+static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next,
+ struct task_struct *tsk)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+ switch_mm_irqs_off(prev, next, tsk);
+ local_irq_restore(flags);
+}
+
+/*
+ * Destroy context related info for an mm_struct that is about
+ * to be put to rest.
+ */
+static inline void destroy_context(struct mm_struct *mm)
+{
+}
+
+#define activate_mm(prev, next) switch_mm(prev, next, current)
+#define deactivate_mm(task, mm) do { } while (0)
+
+/*
+ * If mm is currently active, we can't really drop it.
+ * Instead, we will get a new one for it.
+ */
+static inline void
+drop_mmu_context(struct mm_struct *mm, unsigned int cpu)
+{
+ int asid;
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ asid = read_csr_asid() & cpu_asid_mask(&current_cpu_data);
+
+ if (asid == cpu_asid(cpu, mm)) {
+ if (!current->mm || (current->mm == mm)) {
+ get_new_mmu_context(mm, cpu);
+ write_csr_asid(cpu_asid(cpu, mm));
+ goto out;
+ }
+ }
+
+ /* Will get a new context next time */
+ cpu_context(cpu, mm) = 0;
+ cpumask_clear_cpu(cpu, mm_cpumask(mm));
+out:
+ local_irq_restore(flags);
+}
+
+#endif /* _ASM_MMU_CONTEXT_H */
diff --git a/arch/loongarch/include/asm/processor.h b/arch/loongarch/include/asm/processor.h
new file mode 100644
index 000000000000..1d63c934b289
--- /dev/null
+++ b/arch/loongarch/include/asm/processor.h
@@ -0,0 +1,209 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_PROCESSOR_H
+#define _ASM_PROCESSOR_H
+
+#include <linux/atomic.h>
+#include <linux/cpumask.h>
+#include <linux/sizes.h>
+
+#include <asm/cpu.h>
+#include <asm/cpu-info.h>
+#include <asm/loongarch.h>
+#include <asm/vdso/processor.h>
+#include <uapi/asm/ptrace.h>
+#include <uapi/asm/sigcontext.h>
+
+#ifdef CONFIG_32BIT
+
+#define TASK_SIZE 0x80000000UL
+#define TASK_SIZE_MIN TASK_SIZE
+#define STACK_TOP_MAX TASK_SIZE
+
+#define TASK_IS_32BIT_ADDR 1
+
+#endif
+
+#ifdef CONFIG_64BIT
+
+#define TASK_SIZE32 0x100000000UL
+#define TASK_SIZE64 (0x1UL << ((cpu_vabits > VA_BITS) ? VA_BITS : cpu_vabits))
+
+#define TASK_SIZE (test_thread_flag(TIF_32BIT_ADDR) ? TASK_SIZE32 : TASK_SIZE64)
+#define TASK_SIZE_MIN TASK_SIZE32
+#define STACK_TOP_MAX TASK_SIZE64
+
+#define TASK_SIZE_OF(tsk) \
+ (test_tsk_thread_flag(tsk, TIF_32BIT_ADDR) ? TASK_SIZE32 : TASK_SIZE64)
+
+#define TASK_IS_32BIT_ADDR test_thread_flag(TIF_32BIT_ADDR)
+
+#endif
+
+#define VDSO_RANDOMIZE_SIZE (TASK_IS_32BIT_ADDR ? SZ_1M : SZ_64M)
+
+unsigned long stack_top(void);
+#define STACK_TOP stack_top()
+
+/*
+ * This decides where the kernel will search for a free chunk of vm
+ * space during mmap's.
+ */
+#define TASK_UNMAPPED_BASE PAGE_ALIGN(TASK_SIZE / 3)
+
+#define FPU_REG_WIDTH 256
+#define FPU_ALIGN __attribute__((aligned(32)))
+
+union fpureg {
+ __u32 val32[FPU_REG_WIDTH / 32];
+ __u64 val64[FPU_REG_WIDTH / 64];
+};
+
+#define FPR_IDX(width, idx) (idx)
+
+#define BUILD_FPR_ACCESS(width) \
+static inline u##width get_fpr##width(union fpureg *fpr, unsigned idx) \
+{ \
+ return fpr->val##width[FPR_IDX(width, idx)]; \
+} \
+ \
+static inline void set_fpr##width(union fpureg *fpr, unsigned int idx, \
+ u##width val) \
+{ \
+ fpr->val##width[FPR_IDX(width, idx)] = val; \
+}
+
+BUILD_FPR_ACCESS(32)
+BUILD_FPR_ACCESS(64)
+
+struct loongarch_fpu {
+ unsigned int fcsr;
+ unsigned int vcsr;
+ uint64_t fcc; /* 8x8 */
+ union fpureg fpr[NUM_FPU_REGS];
+};
+
+#define INIT_CPUMASK { \
+ {0,} \
+}
+
+#define ARCH_MIN_TASKALIGN 32
+
+struct loongarch_vdso_info;
+
+/*
+ * If you change thread_struct remember to change the #defines below too!
+ */
+struct thread_struct {
+ /* Main processor registers. */
+ unsigned long reg01, reg03, reg22; /* ra sp fp */
+ unsigned long reg23, reg24, reg25, reg26; /* s0-s3 */
+ unsigned long reg27, reg28, reg29, reg30, reg31; /* s4-s8 */
+
+ /* CSR registers */
+ unsigned long csr_prmd;
+ unsigned long csr_crmd;
+ unsigned long csr_euen;
+ unsigned long csr_ecfg;
+ unsigned long csr_badvaddr; /* Last user fault */
+
+ /* Scratch registers */
+ unsigned long scr0;
+ unsigned long scr1;
+ unsigned long scr2;
+ unsigned long scr3;
+
+ /* Eflags register */
+ unsigned long eflags;
+
+ /* Other stuff associated with the thread. */
+ unsigned long trap_nr;
+ unsigned long error_code;
+ struct loongarch_vdso_info *vdso;
+
+ /*
+ * FPU & vector registers, must be at last because
+ * they are conditionally copied at fork().
+ */
+ struct loongarch_fpu fpu FPU_ALIGN;
+};
+
+#define INIT_THREAD { \
+ /* \
+ * Main processor registers \
+ */ \
+ .reg01 = 0, \
+ .reg03 = 0, \
+ .reg22 = 0, \
+ .reg23 = 0, \
+ .reg24 = 0, \
+ .reg25 = 0, \
+ .reg26 = 0, \
+ .reg27 = 0, \
+ .reg28 = 0, \
+ .reg29 = 0, \
+ .reg30 = 0, \
+ .reg31 = 0, \
+ .csr_crmd = 0, \
+ .csr_prmd = 0, \
+ .csr_euen = 0, \
+ .csr_ecfg = 0, \
+ .csr_badvaddr = 0, \
+ /* \
+ * Other stuff associated with the process \
+ */ \
+ .trap_nr = 0, \
+ .error_code = 0, \
+ /* \
+ * FPU & vector registers \
+ */ \
+ .fpu = { \
+ .fcsr = 0, \
+ .vcsr = 0, \
+ .fcc = 0, \
+ .fpr = {{{0,},},}, \
+ }, \
+}
+
+struct task_struct;
+
+/* Free all resources held by a thread. */
+#define release_thread(thread) do { } while (0)
+
+enum idle_boot_override {IDLE_NO_OVERRIDE = 0, IDLE_HALT, IDLE_NOMWAIT, IDLE_POLL};
+
+extern unsigned long boot_option_idle_override;
+/*
+ * Do necessary setup to start up a newly executed thread.
+ */
+extern void start_thread(struct pt_regs *regs, unsigned long pc, unsigned long sp);
+
+static inline void flush_thread(void)
+{
+}
+
+unsigned long __get_wchan(struct task_struct *p);
+
+#define __KSTK_TOS(tsk) ((unsigned long)task_stack_page(tsk) + \
+ THREAD_SIZE - 32 - sizeof(struct pt_regs))
+#define task_pt_regs(tsk) ((struct pt_regs *)__KSTK_TOS(tsk))
+#define KSTK_EIP(tsk) (task_pt_regs(tsk)->csr_era)
+#define KSTK_ESP(tsk) (task_pt_regs(tsk)->regs[3])
+#define KSTK_EUEN(tsk) (task_pt_regs(tsk)->csr_euen)
+#define KSTK_ECFG(tsk) (task_pt_regs(tsk)->csr_ecfg)
+
+#define return_address() ({__asm__ __volatile__("":::"$1"); __builtin_return_address(0);})
+
+#ifdef CONFIG_CPU_HAS_PREFETCH
+
+#define ARCH_HAS_PREFETCH
+#define prefetch(x) __builtin_prefetch((x), 0, 1)
+
+#define ARCH_HAS_PREFETCHW
+#define prefetchw(x) __builtin_prefetch((x), 1, 1)
+
+#endif
+
+#endif /* _ASM_PROCESSOR_H */
diff --git a/arch/loongarch/include/asm/ptrace.h b/arch/loongarch/include/asm/ptrace.h
new file mode 100644
index 000000000000..17838c6b7ccd
--- /dev/null
+++ b/arch/loongarch/include/asm/ptrace.h
@@ -0,0 +1,152 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_PTRACE_H
+#define _ASM_PTRACE_H
+
+#include <asm/page.h>
+#include <asm/thread_info.h>
+#include <uapi/asm/ptrace.h>
+
+/*
+ * This struct defines the way the registers are stored on the stack during
+ * a system call/exception. If you add a register here, please also add it to
+ * regoffset_table[] in arch/loongarch/kernel/ptrace.c.
+ */
+struct pt_regs {
+ /* Main processor registers. */
+ unsigned long regs[32];
+
+ /* Original syscall arg0. */
+ unsigned long orig_a0;
+
+ /* Special CSR registers. */
+ unsigned long csr_era;
+ unsigned long csr_badvaddr;
+ unsigned long csr_crmd;
+ unsigned long csr_prmd;
+ unsigned long csr_euen;
+ unsigned long csr_ecfg;
+ unsigned long csr_estat;
+ unsigned long __last[0];
+} __aligned(8);
+
+static inline int regs_irqs_disabled(struct pt_regs *regs)
+{
+ return arch_irqs_disabled_flags(regs->csr_prmd);
+}
+
+static inline unsigned long kernel_stack_pointer(struct pt_regs *regs)
+{
+ return regs->regs[3];
+}
+
+/*
+ * Don't use asm-generic/ptrace.h it defines FP accessors that don't make
+ * sense on LoongArch. We rather want an error if they get invoked.
+ */
+
+static inline void instruction_pointer_set(struct pt_regs *regs, unsigned long val)
+{
+ regs->csr_era = val;
+}
+
+/* Query offset/name of register from its name/offset */
+extern int regs_query_register_offset(const char *name);
+#define MAX_REG_OFFSET (offsetof(struct pt_regs, __last))
+
+/**
+ * regs_get_register() - get register value from its offset
+ * @regs: pt_regs from which register value is gotten.
+ * @offset: offset number of the register.
+ *
+ * regs_get_register returns the value of a register. The @offset is the
+ * offset of the register in struct pt_regs address which specified by @regs.
+ * If @offset is bigger than MAX_REG_OFFSET, this returns 0.
+ */
+static inline unsigned long regs_get_register(struct pt_regs *regs, unsigned int offset)
+{
+ if (unlikely(offset > MAX_REG_OFFSET))
+ return 0;
+
+ return *(unsigned long *)((unsigned long)regs + offset);
+}
+
+/**
+ * regs_within_kernel_stack() - check the address in the stack
+ * @regs: pt_regs which contains kernel stack pointer.
+ * @addr: address which is checked.
+ *
+ * regs_within_kernel_stack() checks @addr is within the kernel stack page(s).
+ * If @addr is within the kernel stack, it returns true. If not, returns false.
+ */
+static inline int regs_within_kernel_stack(struct pt_regs *regs, unsigned long addr)
+{
+ return ((addr & ~(THREAD_SIZE - 1)) ==
+ (kernel_stack_pointer(regs) & ~(THREAD_SIZE - 1)));
+}
+
+/**
+ * regs_get_kernel_stack_nth() - get Nth entry of the stack
+ * @regs: pt_regs which contains kernel stack pointer.
+ * @n: stack entry number.
+ *
+ * regs_get_kernel_stack_nth() returns @n th entry of the kernel stack which
+ * is specified by @regs. If the @n th entry is NOT in the kernel stack,
+ * this returns 0.
+ */
+static inline unsigned long regs_get_kernel_stack_nth(struct pt_regs *regs, unsigned int n)
+{
+ unsigned long *addr = (unsigned long *)kernel_stack_pointer(regs);
+
+ addr += n;
+ if (regs_within_kernel_stack(regs, (unsigned long)addr))
+ return *addr;
+ else
+ return 0;
+}
+
+struct task_struct;
+
+/*
+ * Does the process account for user or for system time?
+ */
+#define user_mode(regs) (((regs)->csr_prmd & PLV_MASK) == PLV_USER)
+
+static inline long regs_return_value(struct pt_regs *regs)
+{
+ return regs->regs[4];
+}
+
+#define instruction_pointer(regs) ((regs)->csr_era)
+#define profile_pc(regs) instruction_pointer(regs)
+
+extern void die(const char *, struct pt_regs *) __noreturn;
+
+static inline void die_if_kernel(const char *str, struct pt_regs *regs)
+{
+ if (unlikely(!user_mode(regs)))
+ die(str, regs);
+}
+
+#define current_pt_regs() \
+({ \
+ unsigned long sp = (unsigned long)__builtin_frame_address(0); \
+ (struct pt_regs *)((sp | (THREAD_SIZE - 1)) + 1 - 32) - 1; \
+})
+
+/* Helpers for working with the user stack pointer */
+
+static inline unsigned long user_stack_pointer(struct pt_regs *regs)
+{
+ return regs->regs[3];
+}
+
+static inline void user_stack_pointer_set(struct pt_regs *regs,
+ unsigned long val)
+{
+ regs->regs[3] = val;
+}
+
+#endif /* _ASM_PTRACE_H */
diff --git a/arch/loongarch/include/asm/switch_to.h b/arch/loongarch/include/asm/switch_to.h
new file mode 100644
index 000000000000..2a8d04375574
--- /dev/null
+++ b/arch/loongarch/include/asm/switch_to.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_SWITCH_TO_H
+#define _ASM_SWITCH_TO_H
+
+#include <asm/cpu-features.h>
+#include <asm/fpu.h>
+
+struct task_struct;
+
+/**
+ * __switch_to - switch execution of a task
+ * @prev: The task previously executed.
+ * @next: The task to begin executing.
+ * @next_ti: task_thread_info(next).
+ *
+ * This function is used whilst scheduling to save the context of prev & load
+ * the context of next. Returns prev.
+ */
+extern asmlinkage struct task_struct *__switch_to(struct task_struct *prev,
+ struct task_struct *next, struct thread_info *next_ti);
+
+/*
+ * For newly created kernel threads switch_to() will return to
+ * ret_from_kernel_thread, newly created user threads to ret_from_fork.
+ * That is, everything following __switch_to() will be skipped for new threads.
+ * So everything that matters to new threads should be placed before __switch_to().
+ */
+#define switch_to(prev, next, last) \
+do { \
+ lose_fpu_inatomic(1, prev); \
+ (last) = __switch_to(prev, next, task_thread_info(next)); \
+} while (0)
+
+#endif /* _ASM_SWITCH_TO_H */
diff --git a/arch/loongarch/include/asm/thread_info.h b/arch/loongarch/include/asm/thread_info.h
new file mode 100644
index 000000000000..99beb11c2fa8
--- /dev/null
+++ b/arch/loongarch/include/asm/thread_info.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * thread_info.h: LoongArch low-level thread information
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef _ASM_THREAD_INFO_H
+#define _ASM_THREAD_INFO_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <asm/processor.h>
+
+/*
+ * low level task data that entry.S needs immediate access to
+ * - this struct should fit entirely inside of one cache line
+ * - this struct shares the supervisor stack pages
+ * - if the contents of this structure are changed, the assembly constants
+ * must also be changed
+ */
+struct thread_info {
+ struct task_struct *task; /* main task structure */
+ unsigned long flags; /* low level flags */
+ unsigned long tp_value; /* thread pointer */
+ __u32 cpu; /* current CPU */
+ int preempt_count; /* 0 => preemptible, <0 => BUG */
+ struct pt_regs *regs;
+ unsigned long syscall; /* syscall number */
+ unsigned long syscall_work; /* SYSCALL_WORK_ flags */
+};
+
+/*
+ * macros/functions for gaining access to the thread information structure
+ */
+#define INIT_THREAD_INFO(tsk) \
+{ \
+ .task = &tsk, \
+ .flags = 0, \
+ .cpu = 0, \
+ .preempt_count = INIT_PREEMPT_COUNT, \
+}
+
+/* How to get the thread information struct from C. */
+register struct thread_info *__current_thread_info __asm__("$r2");
+
+static inline struct thread_info *current_thread_info(void)
+{
+ return __current_thread_info;
+}
+
+register unsigned long current_stack_pointer __asm__("$r3");
+
+#endif /* !__ASSEMBLY__ */
+
+/* thread information allocation */
+#define THREAD_SIZE SZ_16K
+#define THREAD_MASK (THREAD_SIZE - 1UL)
+#define THREAD_SIZE_ORDER ilog2(THREAD_SIZE / PAGE_SIZE)
+/*
+ * thread information flags
+ * - these are process state flags that various assembly files may need to
+ * access
+ * - pending work-to-be-done flags are in LSW
+ * - other flags in MSW
+ */
+#define TIF_SIGPENDING 1 /* signal pending */
+#define TIF_NEED_RESCHED 2 /* rescheduling necessary */
+#define TIF_NOTIFY_RESUME 3 /* callback before returning to user */
+#define TIF_NOTIFY_SIGNAL 4 /* signal notifications exist */
+#define TIF_RESTORE_SIGMASK 5 /* restore signal mask in do_signal() */
+#define TIF_NOHZ 6 /* in adaptive nohz mode */
+#define TIF_UPROBE 7 /* breakpointed or singlestepping */
+#define TIF_USEDFPU 8 /* FPU was used by this task this quantum (SMP) */
+#define TIF_USEDSIMD 9 /* SIMD has been used this quantum */
+#define TIF_MEMDIE 10 /* is terminating due to OOM killer */
+#define TIF_FIXADE 11 /* Fix address errors in software */
+#define TIF_LOGADE 12 /* Log address errors to syslog */
+#define TIF_32BIT_REGS 13 /* 32-bit general purpose registers */
+#define TIF_32BIT_ADDR 14 /* 32-bit address space */
+#define TIF_LOAD_WATCH 15 /* If set, load watch registers */
+#define TIF_SINGLESTEP 16 /* Single Step */
+#define TIF_LSX_CTX_LIVE 17 /* LSX context must be preserved */
+#define TIF_LASX_CTX_LIVE 18 /* LASX context must be preserved */
+
+#define _TIF_SIGPENDING (1<<TIF_SIGPENDING)
+#define _TIF_NEED_RESCHED (1<<TIF_NEED_RESCHED)
+#define _TIF_NOTIFY_RESUME (1<<TIF_NOTIFY_RESUME)
+#define _TIF_NOTIFY_SIGNAL (1<<TIF_NOTIFY_SIGNAL)
+#define _TIF_NOHZ (1<<TIF_NOHZ)
+#define _TIF_UPROBE (1<<TIF_UPROBE)
+#define _TIF_USEDFPU (1<<TIF_USEDFPU)
+#define _TIF_USEDSIMD (1<<TIF_USEDSIMD)
+#define _TIF_FIXADE (1<<TIF_FIXADE)
+#define _TIF_LOGADE (1<<TIF_LOGADE)
+#define _TIF_32BIT_REGS (1<<TIF_32BIT_REGS)
+#define _TIF_32BIT_ADDR (1<<TIF_32BIT_ADDR)
+#define _TIF_LOAD_WATCH (1<<TIF_LOAD_WATCH)
+#define _TIF_SINGLESTEP (1<<TIF_SINGLESTEP)
+#define _TIF_LSX_CTX_LIVE (1<<TIF_LSX_CTX_LIVE)
+#define _TIF_LASX_CTX_LIVE (1<<TIF_LASX_CTX_LIVE)
+
+#endif /* __KERNEL__ */
+#endif /* _ASM_THREAD_INFO_H */
diff --git a/arch/loongarch/include/uapi/asm/ptrace.h b/arch/loongarch/include/uapi/asm/ptrace.h
new file mode 100644
index 000000000000..083193f4a5d5
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/ptrace.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
+/*
+ * Author: Hanlu Li <[email protected]>
+ * Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _UAPI_ASM_PTRACE_H
+#define _UAPI_ASM_PTRACE_H
+
+#include <linux/types.h>
+
+#ifndef __KERNEL__
+#include <stdint.h>
+#endif
+
+/*
+ * For PTRACE_{POKE,PEEK}USR. 0 - 31 are GPRs,
+ * 32 is syscall's original ARG0, 33 is PC, 34 is BADVADDR.
+ */
+#define GPR_BASE 0
+#define GPR_NUM 32
+#define GPR_END (GPR_BASE + GPR_NUM - 1)
+#define ARG0 (GPR_END + 1)
+#define PC (GPR_END + 2)
+#define BADVADDR (GPR_END + 3)
+
+#define NUM_FPU_REGS 32
+
+struct user_pt_regs {
+ /* Main processor registers. */
+ unsigned long regs[32];
+
+ /* Original syscall arg0. */
+ unsigned long orig_a0;
+
+ /* Special CSR registers. */
+ unsigned long csr_era;
+ unsigned long csr_badv;
+ unsigned long reserved[10];
+} __attribute__((aligned(8)));
+
+struct user_fp_state {
+ uint64_t fpr[32];
+ uint64_t fcc;
+ uint32_t fcsr;
+};
+
+#define PTRACE_SYSEMU 0x1f
+#define PTRACE_SYSEMU_SINGLESTEP 0x20
+
+#endif /* _UAPI_ASM_PTRACE_H */
diff --git a/arch/loongarch/kernel/fpu.S b/arch/loongarch/kernel/fpu.S
new file mode 100644
index 000000000000..75c6ce0682a2
--- /dev/null
+++ b/arch/loongarch/kernel/fpu.S
@@ -0,0 +1,261 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Lu Zeng <[email protected]>
+ * Pei Huang <[email protected]>
+ * Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/asm-offsets.h>
+#include <asm/errno.h>
+#include <asm/export.h>
+#include <asm/fpregdef.h>
+#include <asm/loongarch.h>
+#include <asm/regdef.h>
+
+#define FPU_REG_WIDTH 8
+#define LSX_REG_WIDTH 16
+#define LASX_REG_WIDTH 32
+
+ .macro EX insn, reg, src, offs
+.ex\@: \insn \reg, \src, \offs
+ .section __ex_table,"a"
+ PTR .ex\@, fault
+ .previous
+ .endm
+
+ .macro sc_save_fp base
+ EX fst.d $f0, \base, (0 * FPU_REG_WIDTH)
+ EX fst.d $f1, \base, (1 * FPU_REG_WIDTH)
+ EX fst.d $f2, \base, (2 * FPU_REG_WIDTH)
+ EX fst.d $f3, \base, (3 * FPU_REG_WIDTH)
+ EX fst.d $f4, \base, (4 * FPU_REG_WIDTH)
+ EX fst.d $f5, \base, (5 * FPU_REG_WIDTH)
+ EX fst.d $f6, \base, (6 * FPU_REG_WIDTH)
+ EX fst.d $f7, \base, (7 * FPU_REG_WIDTH)
+ EX fst.d $f8, \base, (8 * FPU_REG_WIDTH)
+ EX fst.d $f9, \base, (9 * FPU_REG_WIDTH)
+ EX fst.d $f10, \base, (10 * FPU_REG_WIDTH)
+ EX fst.d $f11, \base, (11 * FPU_REG_WIDTH)
+ EX fst.d $f12, \base, (12 * FPU_REG_WIDTH)
+ EX fst.d $f13, \base, (13 * FPU_REG_WIDTH)
+ EX fst.d $f14, \base, (14 * FPU_REG_WIDTH)
+ EX fst.d $f15, \base, (15 * FPU_REG_WIDTH)
+ EX fst.d $f16, \base, (16 * FPU_REG_WIDTH)
+ EX fst.d $f17, \base, (17 * FPU_REG_WIDTH)
+ EX fst.d $f18, \base, (18 * FPU_REG_WIDTH)
+ EX fst.d $f19, \base, (19 * FPU_REG_WIDTH)
+ EX fst.d $f20, \base, (20 * FPU_REG_WIDTH)
+ EX fst.d $f21, \base, (21 * FPU_REG_WIDTH)
+ EX fst.d $f22, \base, (22 * FPU_REG_WIDTH)
+ EX fst.d $f23, \base, (23 * FPU_REG_WIDTH)
+ EX fst.d $f24, \base, (24 * FPU_REG_WIDTH)
+ EX fst.d $f25, \base, (25 * FPU_REG_WIDTH)
+ EX fst.d $f26, \base, (26 * FPU_REG_WIDTH)
+ EX fst.d $f27, \base, (27 * FPU_REG_WIDTH)
+ EX fst.d $f28, \base, (28 * FPU_REG_WIDTH)
+ EX fst.d $f29, \base, (29 * FPU_REG_WIDTH)
+ EX fst.d $f30, \base, (30 * FPU_REG_WIDTH)
+ EX fst.d $f31, \base, (31 * FPU_REG_WIDTH)
+ .endm
+
+ .macro sc_restore_fp base
+ EX fld.d $f0, \base, (0 * FPU_REG_WIDTH)
+ EX fld.d $f1, \base, (1 * FPU_REG_WIDTH)
+ EX fld.d $f2, \base, (2 * FPU_REG_WIDTH)
+ EX fld.d $f3, \base, (3 * FPU_REG_WIDTH)
+ EX fld.d $f4, \base, (4 * FPU_REG_WIDTH)
+ EX fld.d $f5, \base, (5 * FPU_REG_WIDTH)
+ EX fld.d $f6, \base, (6 * FPU_REG_WIDTH)
+ EX fld.d $f7, \base, (7 * FPU_REG_WIDTH)
+ EX fld.d $f8, \base, (8 * FPU_REG_WIDTH)
+ EX fld.d $f9, \base, (9 * FPU_REG_WIDTH)
+ EX fld.d $f10, \base, (10 * FPU_REG_WIDTH)
+ EX fld.d $f11, \base, (11 * FPU_REG_WIDTH)
+ EX fld.d $f12, \base, (12 * FPU_REG_WIDTH)
+ EX fld.d $f13, \base, (13 * FPU_REG_WIDTH)
+ EX fld.d $f14, \base, (14 * FPU_REG_WIDTH)
+ EX fld.d $f15, \base, (15 * FPU_REG_WIDTH)
+ EX fld.d $f16, \base, (16 * FPU_REG_WIDTH)
+ EX fld.d $f17, \base, (17 * FPU_REG_WIDTH)
+ EX fld.d $f18, \base, (18 * FPU_REG_WIDTH)
+ EX fld.d $f19, \base, (19 * FPU_REG_WIDTH)
+ EX fld.d $f20, \base, (20 * FPU_REG_WIDTH)
+ EX fld.d $f21, \base, (21 * FPU_REG_WIDTH)
+ EX fld.d $f22, \base, (22 * FPU_REG_WIDTH)
+ EX fld.d $f23, \base, (23 * FPU_REG_WIDTH)
+ EX fld.d $f24, \base, (24 * FPU_REG_WIDTH)
+ EX fld.d $f25, \base, (25 * FPU_REG_WIDTH)
+ EX fld.d $f26, \base, (26 * FPU_REG_WIDTH)
+ EX fld.d $f27, \base, (27 * FPU_REG_WIDTH)
+ EX fld.d $f28, \base, (28 * FPU_REG_WIDTH)
+ EX fld.d $f29, \base, (29 * FPU_REG_WIDTH)
+ EX fld.d $f30, \base, (30 * FPU_REG_WIDTH)
+ EX fld.d $f31, \base, (31 * FPU_REG_WIDTH)
+ .endm
+
+ .macro sc_save_fcc base, tmp0, tmp1
+ movcf2gr \tmp0, $fcc0
+ move \tmp1, \tmp0
+ movcf2gr \tmp0, $fcc1
+ bstrins.d \tmp1, \tmp0, 15, 8
+ movcf2gr \tmp0, $fcc2
+ bstrins.d \tmp1, \tmp0, 23, 16
+ movcf2gr \tmp0, $fcc3
+ bstrins.d \tmp1, \tmp0, 31, 24
+ movcf2gr \tmp0, $fcc4
+ bstrins.d \tmp1, \tmp0, 39, 32
+ movcf2gr \tmp0, $fcc5
+ bstrins.d \tmp1, \tmp0, 47, 40
+ movcf2gr \tmp0, $fcc6
+ bstrins.d \tmp1, \tmp0, 55, 48
+ movcf2gr \tmp0, $fcc7
+ bstrins.d \tmp1, \tmp0, 63, 56
+ EX st.d \tmp1, \base, 0
+ .endm
+
+ .macro sc_restore_fcc base, tmp0, tmp1
+ EX ld.d \tmp0, \base, 0
+ bstrpick.d \tmp1, \tmp0, 7, 0
+ movgr2cf $fcc0, \tmp1
+ bstrpick.d \tmp1, \tmp0, 15, 8
+ movgr2cf $fcc1, \tmp1
+ bstrpick.d \tmp1, \tmp0, 23, 16
+ movgr2cf $fcc2, \tmp1
+ bstrpick.d \tmp1, \tmp0, 31, 24
+ movgr2cf $fcc3, \tmp1
+ bstrpick.d \tmp1, \tmp0, 39, 32
+ movgr2cf $fcc4, \tmp1
+ bstrpick.d \tmp1, \tmp0, 47, 40
+ movgr2cf $fcc5, \tmp1
+ bstrpick.d \tmp1, \tmp0, 55, 48
+ movgr2cf $fcc6, \tmp1
+ bstrpick.d \tmp1, \tmp0, 63, 56
+ movgr2cf $fcc7, \tmp1
+ .endm
+
+ .macro sc_save_fcsr base, tmp0
+ movfcsr2gr \tmp0, fcsr0
+ EX st.w \tmp0, \base, 0
+ .endm
+
+ .macro sc_restore_fcsr base, tmp0
+ EX ld.w \tmp0, \base, 0
+ movgr2fcsr fcsr0, \tmp0
+ .endm
+
+ .macro sc_save_vcsr base, tmp0
+ movfcsr2gr \tmp0, vcsr16
+ EX st.w \tmp0, \base, 0
+ .endm
+
+ .macro sc_restore_vcsr base, tmp0
+ EX ld.w \tmp0, \base, 0
+ movgr2fcsr vcsr16, \tmp0
+ .endm
+
+/*
+ * Save a thread's fp context.
+ */
+SYM_FUNC_START(_save_fp)
+ fpu_save_csr a0 t1
+ fpu_save_double a0 t1 # clobbers t1
+ fpu_save_cc a0 t1 t2 # clobbers t1, t2
+ jirl zero, ra, 0
+SYM_FUNC_END(_save_fp)
+EXPORT_SYMBOL(_save_fp)
+
+/*
+ * Restore a thread's fp context.
+ */
+SYM_FUNC_START(_restore_fp)
+ fpu_restore_double a0 t1 # clobbers t1
+ fpu_restore_csr a0 t1
+ fpu_restore_cc a0 t1 t2 # clobbers t1, t2
+ jirl zero, ra, 0
+SYM_FUNC_END(_restore_fp)
+
+/*
+ * Load the FPU with signalling NANS. This bit pattern we're using has
+ * the property that no matter whether considered as single or as double
+ * precision represents signaling NANS.
+ *
+ * The value to initialize fcsr0 to comes in $a0.
+ */
+
+SYM_FUNC_START(_init_fpu)
+ li.w t1, CSR_EUEN_FPEN
+ csrxchg t1, t1, LOONGARCH_CSR_EUEN
+
+ movgr2fcsr fcsr0, a0
+
+ li.w t1, -1 # SNaN
+
+ movgr2fr.d $f0, t1
+ movgr2fr.d $f1, t1
+ movgr2fr.d $f2, t1
+ movgr2fr.d $f3, t1
+ movgr2fr.d $f4, t1
+ movgr2fr.d $f5, t1
+ movgr2fr.d $f6, t1
+ movgr2fr.d $f7, t1
+ movgr2fr.d $f8, t1
+ movgr2fr.d $f9, t1
+ movgr2fr.d $f10, t1
+ movgr2fr.d $f11, t1
+ movgr2fr.d $f12, t1
+ movgr2fr.d $f13, t1
+ movgr2fr.d $f14, t1
+ movgr2fr.d $f15, t1
+ movgr2fr.d $f16, t1
+ movgr2fr.d $f17, t1
+ movgr2fr.d $f18, t1
+ movgr2fr.d $f19, t1
+ movgr2fr.d $f20, t1
+ movgr2fr.d $f21, t1
+ movgr2fr.d $f22, t1
+ movgr2fr.d $f23, t1
+ movgr2fr.d $f24, t1
+ movgr2fr.d $f25, t1
+ movgr2fr.d $f26, t1
+ movgr2fr.d $f27, t1
+ movgr2fr.d $f28, t1
+ movgr2fr.d $f29, t1
+ movgr2fr.d $f30, t1
+ movgr2fr.d $f31, t1
+
+ jirl zero, ra, 0
+SYM_FUNC_END(_init_fpu)
+
+/*
+ * a0: fpregs
+ * a1: fcc
+ * a2: fcsr
+ */
+SYM_FUNC_START(_save_fp_context)
+ sc_save_fcc a1 t1 t2
+ sc_save_fcsr a2 t1
+ sc_save_fp a0
+ li.w a0, 0 # success
+ jirl zero, ra, 0
+SYM_FUNC_END(_save_fp_context)
+
+/*
+ * a0: fpregs
+ * a1: fcc
+ * a2: fcsr
+ */
+SYM_FUNC_START(_restore_fp_context)
+ sc_restore_fp a0
+ sc_restore_fcc a1 t1 t2
+ sc_restore_fcsr a2 t1
+ li.w a0, 0 # success
+ jirl zero, ra, 0
+SYM_FUNC_END(_restore_fp_context)
+
+SYM_FUNC_START(fault)
+ li.w a0, -EFAULT # failure
+ jirl zero, ra, 0
+SYM_FUNC_END(fault)
diff --git a/arch/loongarch/kernel/idle.c b/arch/loongarch/kernel/idle.c
new file mode 100644
index 000000000000..1a65d0527d25
--- /dev/null
+++ b/arch/loongarch/kernel/idle.c
@@ -0,0 +1,16 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * LoongArch idle loop support.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/cpu.h>
+#include <linux/irqflags.h>
+#include <asm/cpu.h>
+#include <asm/idle.h>
+
+void __cpuidle arch_cpu_idle(void)
+{
+ raw_local_irq_enable();
+ __arch_cpu_idle(); /* idle instruction needs irq enabled */
+}
diff --git a/arch/loongarch/kernel/process.c b/arch/loongarch/kernel/process.c
new file mode 100644
index 000000000000..667b365d909d
--- /dev/null
+++ b/arch/loongarch/kernel/process.c
@@ -0,0 +1,260 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1994 - 1999, 2000 by Ralf Baechle and others.
+ * Copyright (C) 2005, 2006 by Ralf Baechle ([email protected])
+ * Copyright (C) 1999, 2000 Silicon Graphics, Inc.
+ * Copyright (C) 2004 Thiemo Seufer
+ * Copyright (C) 2013 Imagination Technologies Ltd.
+ */
+#include <linux/cpu.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/sched.h>
+#include <linux/sched/debug.h>
+#include <linux/sched/task.h>
+#include <linux/sched/task_stack.h>
+#include <linux/mm.h>
+#include <linux/stddef.h>
+#include <linux/unistd.h>
+#include <linux/export.h>
+#include <linux/ptrace.h>
+#include <linux/mman.h>
+#include <linux/personality.h>
+#include <linux/sys.h>
+#include <linux/completion.h>
+#include <linux/kallsyms.h>
+#include <linux/random.h>
+#include <linux/prctl.h>
+#include <linux/nmi.h>
+
+#include <asm/asm.h>
+#include <asm/bootinfo.h>
+#include <asm/cpu.h>
+#include <asm/elf.h>
+#include <asm/fpu.h>
+#include <asm/io.h>
+#include <asm/irq.h>
+#include <asm/irq_regs.h>
+#include <asm/loongarch.h>
+#include <asm/pgtable.h>
+#include <asm/processor.h>
+#include <asm/reg.h>
+#include <asm/vdso.h>
+
+/*
+ * Idle related variables and functions
+ */
+
+unsigned long boot_option_idle_override = IDLE_NO_OVERRIDE;
+EXPORT_SYMBOL(boot_option_idle_override);
+
+asmlinkage void ret_from_fork(void);
+asmlinkage void ret_from_kernel_thread(void);
+
+void start_thread(struct pt_regs *regs, unsigned long pc, unsigned long sp)
+{
+ unsigned long crmd;
+ unsigned long prmd;
+ unsigned long euen;
+
+ /* New thread loses kernel privileges. */
+ crmd = regs->csr_crmd & ~(PLV_MASK);
+ crmd |= PLV_USER;
+ regs->csr_crmd = crmd;
+
+ prmd = regs->csr_prmd & ~(PLV_MASK);
+ prmd |= PLV_USER;
+ regs->csr_prmd = prmd;
+
+ euen = regs->csr_euen & ~(CSR_EUEN_FPEN);
+ regs->csr_euen = euen;
+ lose_fpu(0);
+
+ clear_thread_flag(TIF_LSX_CTX_LIVE);
+ clear_thread_flag(TIF_LASX_CTX_LIVE);
+ clear_used_math();
+ regs->csr_era = pc;
+ regs->regs[3] = sp;
+}
+
+void exit_thread(struct task_struct *tsk)
+{
+}
+
+int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
+{
+ /*
+ * Save any process state which is live in hardware registers to the
+ * parent context prior to duplication. This prevents the new child
+ * state becoming stale if the parent is preempted before copy_thread()
+ * gets a chance to save the parent's live hardware registers to the
+ * child context.
+ */
+ preempt_disable();
+
+ if (is_fpu_owner())
+ save_fp(current);
+
+ preempt_enable();
+
+ if (used_math())
+ memcpy(dst, src, sizeof(struct task_struct));
+ else
+ memcpy(dst, src, offsetof(struct task_struct, thread.fpu.fpr));
+
+ return 0;
+}
+
+/*
+ * Copy architecture-specific thread state
+ */
+int copy_thread(unsigned long clone_flags, unsigned long usp,
+ unsigned long kthread_arg, struct task_struct *p, unsigned long tls)
+{
+ unsigned long childksp;
+ struct pt_regs *childregs, *regs = current_pt_regs();
+
+ childksp = (unsigned long)task_stack_page(p) + THREAD_SIZE - 32;
+
+ /* set up new TSS. */
+ childregs = (struct pt_regs *) childksp - 1;
+ /* Put the stack after the struct pt_regs. */
+ childksp = (unsigned long) childregs;
+ p->thread.csr_euen = 0;
+ p->thread.csr_crmd = csr_readl(LOONGARCH_CSR_CRMD);
+ p->thread.csr_prmd = csr_readl(LOONGARCH_CSR_PRMD);
+ p->thread.csr_ecfg = csr_readl(LOONGARCH_CSR_ECFG);
+ if (unlikely(p->flags & (PF_KTHREAD | PF_IO_WORKER))) {
+ /* kernel thread */
+ p->thread.reg23 = usp; /* fn */
+ p->thread.reg24 = kthread_arg;
+ p->thread.reg03 = childksp;
+ p->thread.reg01 = (unsigned long) ret_from_kernel_thread;
+ memset(childregs, 0, sizeof(struct pt_regs));
+ childregs->csr_euen = p->thread.csr_euen;
+ childregs->csr_crmd = p->thread.csr_crmd;
+ childregs->csr_prmd = p->thread.csr_prmd;
+ childregs->csr_ecfg = p->thread.csr_ecfg;
+ return 0;
+ }
+
+ /* user thread */
+ *childregs = *regs;
+ childregs->regs[4] = 0; /* Child gets zero as return value */
+ if (usp)
+ childregs->regs[3] = usp;
+
+ p->thread.reg03 = (unsigned long) childregs;
+ p->thread.reg01 = (unsigned long) ret_from_fork;
+
+ /*
+ * New tasks lose permission to use the fpu. This accelerates context
+ * switching for most programs since they don't use the fpu.
+ */
+ childregs->csr_euen = 0;
+
+ clear_tsk_thread_flag(p, TIF_USEDFPU);
+ clear_tsk_thread_flag(p, TIF_USEDSIMD);
+ clear_tsk_thread_flag(p, TIF_LSX_CTX_LIVE);
+ clear_tsk_thread_flag(p, TIF_LASX_CTX_LIVE);
+
+ if (clone_flags & CLONE_SETTLS)
+ childregs->regs[2] = tls;
+
+ return 0;
+}
+
+unsigned long __get_wchan(struct task_struct *task)
+{
+ return 0;
+}
+
+unsigned long stack_top(void)
+{
+ unsigned long top = TASK_SIZE & PAGE_MASK;
+
+ /* Space for the VDSO & data page */
+ top -= PAGE_ALIGN(current->thread.vdso->size);
+ top -= PAGE_SIZE;
+
+ /* Space to randomize the VDSO base */
+ if (current->flags & PF_RANDOMIZE)
+ top -= VDSO_RANDOMIZE_SIZE;
+
+ return top;
+}
+
+/*
+ * Don't forget that the stack pointer must be aligned on a 8 bytes
+ * boundary for 32-bits ABI and 16 bytes for 64-bits ABI.
+ */
+unsigned long arch_align_stack(unsigned long sp)
+{
+ if (!(current->personality & ADDR_NO_RANDOMIZE) && randomize_va_space)
+ sp -= get_random_int() & ~PAGE_MASK;
+
+ return sp & STACK_ALIGN;
+}
+
+static DEFINE_PER_CPU(call_single_data_t, backtrace_csd);
+static struct cpumask backtrace_csd_busy;
+
+static void handle_backtrace(void *info)
+{
+ nmi_cpu_backtrace(get_irq_regs());
+ cpumask_clear_cpu(smp_processor_id(), &backtrace_csd_busy);
+}
+
+static void raise_backtrace(cpumask_t *mask)
+{
+ call_single_data_t *csd;
+ int cpu;
+
+ for_each_cpu(cpu, mask) {
+ /*
+ * If we previously sent an IPI to the target CPU & it hasn't
+ * cleared its bit in the busy cpumask then it didn't handle
+ * our previous IPI & it's not safe for us to reuse the
+ * call_single_data_t.
+ */
+ if (cpumask_test_and_set_cpu(cpu, &backtrace_csd_busy)) {
+ pr_warn("Unable to send backtrace IPI to CPU%u - perhaps it hung?\n",
+ cpu);
+ continue;
+ }
+
+ csd = &per_cpu(backtrace_csd, cpu);
+ csd->func = handle_backtrace;
+ smp_call_function_single_async(cpu, csd);
+ }
+}
+
+void arch_trigger_cpumask_backtrace(const cpumask_t *mask, bool exclude_self)
+{
+ nmi_trigger_cpumask_backtrace(mask, exclude_self, raise_backtrace);
+}
+
+#ifdef CONFIG_64BIT
+void loongarch_dump_regs64(u64 *uregs, const struct pt_regs *regs)
+{
+ unsigned int i;
+
+ for (i = LOONGARCH_EF_R1; i <= LOONGARCH_EF_R31; i++) {
+ uregs[i] = regs->regs[i - LOONGARCH_EF_R0];
+ }
+
+ uregs[LOONGARCH_EF_ORIG_A0] = regs->orig_a0;
+ uregs[LOONGARCH_EF_CSR_ERA] = regs->csr_era;
+ uregs[LOONGARCH_EF_CSR_BADV] = regs->csr_badvaddr;
+ uregs[LOONGARCH_EF_CSR_CRMD] = regs->csr_crmd;
+ uregs[LOONGARCH_EF_CSR_PRMD] = regs->csr_prmd;
+ uregs[LOONGARCH_EF_CSR_EUEN] = regs->csr_euen;
+ uregs[LOONGARCH_EF_CSR_ECFG] = regs->csr_ecfg;
+ uregs[LOONGARCH_EF_CSR_ESTAT] = regs->csr_estat;
+}
+#endif /* CONFIG_64BIT */
diff --git a/arch/loongarch/kernel/ptrace.c b/arch/loongarch/kernel/ptrace.c
new file mode 100644
index 000000000000..e6ab87948e1d
--- /dev/null
+++ b/arch/loongarch/kernel/ptrace.c
@@ -0,0 +1,431 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Author: Hanlu Li <[email protected]>
+ * Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1992 Ross Biro
+ * Copyright (C) Linus Torvalds
+ * Copyright (C) 1994, 95, 96, 97, 98, 2000 Ralf Baechle
+ * Copyright (C) 1996 David S. Miller
+ * Kevin D. Kissell, [email protected] and Carsten Langgaard, [email protected]
+ * Copyright (C) 1999 MIPS Technologies, Inc.
+ * Copyright (C) 2000 Ulf Carlsson
+ */
+#include <linux/kernel.h>
+#include <linux/audit.h>
+#include <linux/compiler.h>
+#include <linux/context_tracking.h>
+#include <linux/elf.h>
+#include <linux/errno.h>
+#include <linux/mm.h>
+#include <linux/ptrace.h>
+#include <linux/regset.h>
+#include <linux/sched.h>
+#include <linux/sched/task_stack.h>
+#include <linux/security.h>
+#include <linux/smp.h>
+#include <linux/stddef.h>
+#include <linux/seccomp.h>
+#include <linux/uaccess.h>
+
+#include <asm/byteorder.h>
+#include <asm/cpu.h>
+#include <asm/cpu-info.h>
+#include <asm/fpu.h>
+#include <asm/loongarch.h>
+#include <asm/page.h>
+#include <asm/pgtable.h>
+#include <asm/processor.h>
+#include <asm/reg.h>
+#include <asm/syscall.h>
+
+static void init_fp_ctx(struct task_struct *target)
+{
+ /* The target already has context */
+ if (tsk_used_math(target))
+ return;
+
+ /* Begin with data registers set to all 1s... */
+ memset(&target->thread.fpu.fpr, ~0, sizeof(target->thread.fpu.fpr));
+ set_stopped_child_used_math(target);
+}
+
+/*
+ * Called by kernel/ptrace.c when detaching..
+ *
+ * Make sure single step bits etc are not set.
+ */
+void ptrace_disable(struct task_struct *child)
+{
+ /* Don't load the watchpoint registers for the ex-child. */
+ clear_tsk_thread_flag(child, TIF_LOAD_WATCH);
+ clear_tsk_thread_flag(child, TIF_SINGLESTEP);
+}
+
+/* regset get/set implementations */
+
+static int gpr_get(struct task_struct *target,
+ const struct user_regset *regset,
+ struct membuf to)
+{
+ int r;
+ struct pt_regs *regs = task_pt_regs(target);
+
+ r = membuf_write(&to, &regs->regs, sizeof(u64) * GPR_NUM);
+ r = membuf_write(&to, &regs->orig_a0, sizeof(u64));
+ r = membuf_write(&to, &regs->csr_era, sizeof(u64));
+ r = membuf_write(&to, &regs->csr_badvaddr, sizeof(u64));
+
+ return r;
+}
+
+static int gpr_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int err;
+ int a0_start = sizeof(u64) * GPR_NUM;
+ int era_start = a0_start + sizeof(u64);
+ int badvaddr_start = era_start + sizeof(u64);
+ struct pt_regs *regs = task_pt_regs(target);
+
+ err = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ &regs->regs,
+ 0, a0_start);
+ err |= user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ &regs->orig_a0,
+ a0_start, a0_start + sizeof(u64));
+ err |= user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ &regs->csr_era,
+ era_start, era_start + sizeof(u64));
+ err |= user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ &regs->csr_badvaddr,
+ badvaddr_start, badvaddr_start + sizeof(u64));
+
+ return err;
+}
+
+
+/*
+ * Get the general floating-point registers.
+ */
+static int gfpr_get(struct task_struct *target, struct membuf *to)
+{
+ return membuf_write(to, &target->thread.fpu.fpr,
+ sizeof(elf_fpreg_t) * NUM_FPU_REGS);
+}
+
+static int gfpr_get_simd(struct task_struct *target, struct membuf *to)
+{
+ int i, r;
+ u64 fpr_val;
+
+ BUILD_BUG_ON(sizeof(fpr_val) != sizeof(elf_fpreg_t));
+ for (i = 0; i < NUM_FPU_REGS; i++) {
+ fpr_val = get_fpr64(&target->thread.fpu.fpr[i], 0);
+ r = membuf_write(to, &fpr_val, sizeof(elf_fpreg_t));
+ }
+
+ return r;
+}
+
+/*
+ * Choose the appropriate helper for general registers, and then copy
+ * the FCC and FCSR registers separately.
+ */
+static int fpr_get(struct task_struct *target,
+ const struct user_regset *regset,
+ struct membuf to)
+{
+ int r;
+
+ if (sizeof(target->thread.fpu.fpr[0]) == sizeof(elf_fpreg_t))
+ r = gfpr_get(target, &to);
+ else
+ r = gfpr_get_simd(target, &to);
+
+ r = membuf_write(&to, &target->thread.fpu.fcc, sizeof(target->thread.fpu.fcc));
+ r = membuf_write(&to, &target->thread.fpu.fcsr, sizeof(target->thread.fpu.fcsr));
+
+ return r;
+}
+
+static int gfpr_set(struct task_struct *target,
+ unsigned int *pos, unsigned int *count,
+ const void **kbuf, const void __user **ubuf)
+{
+ return user_regset_copyin(pos, count, kbuf, ubuf,
+ &target->thread.fpu.fpr,
+ 0, NUM_FPU_REGS * sizeof(elf_fpreg_t));
+}
+
+static int gfpr_set_simd(struct task_struct *target,
+ unsigned int *pos, unsigned int *count,
+ const void **kbuf, const void __user **ubuf)
+{
+ int i, err;
+ u64 fpr_val;
+
+ BUILD_BUG_ON(sizeof(fpr_val) != sizeof(elf_fpreg_t));
+ for (i = 0; i < NUM_FPU_REGS && *count > 0; i++) {
+ err = user_regset_copyin(pos, count, kbuf, ubuf,
+ &fpr_val, i * sizeof(elf_fpreg_t),
+ (i + 1) * sizeof(elf_fpreg_t));
+ if (err)
+ return err;
+ set_fpr64(&target->thread.fpu.fpr[i], 0, fpr_val);
+ }
+
+ return 0;
+}
+
+/*
+ * Choose the appropriate helper for general registers, and then copy
+ * the FCC register separately.
+ */
+static int fpr_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ const int fcc_start = NUM_FPU_REGS * sizeof(elf_fpreg_t);
+ const int fcc_end = fcc_start + sizeof(u64);
+ int err;
+
+ BUG_ON(count % sizeof(elf_fpreg_t));
+ if (pos + count > sizeof(elf_fpregset_t))
+ return -EIO;
+
+ init_fp_ctx(target);
+
+ if (sizeof(target->thread.fpu.fpr[0]) == sizeof(elf_fpreg_t))
+ err = gfpr_set(target, &pos, &count, &kbuf, &ubuf);
+ else
+ err = gfpr_set_simd(target, &pos, &count, &kbuf, &ubuf);
+ if (err)
+ return err;
+
+ if (count > 0)
+ err |= user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ &target->thread.fpu.fcc,
+ fcc_start, fcc_end);
+
+ return err;
+}
+
+static int cfg_get(struct task_struct *target,
+ const struct user_regset *regset,
+ struct membuf to)
+{
+ int i, r;
+ u32 cfg_val;
+
+ i = 0;
+ while (to.left > 0) {
+ cfg_val = read_cpucfg(i++);
+ r = membuf_write(&to, &cfg_val, sizeof(u32));
+ }
+
+ return r;
+}
+
+/*
+ * CFG registers are read-only.
+ */
+static int cfg_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ return 0;
+}
+
+struct pt_regs_offset {
+ const char *name;
+ int offset;
+};
+
+#define REG_OFFSET_NAME(n, r) {.name = #n, .offset = offsetof(struct pt_regs, r)}
+#define REG_OFFSET_END {.name = NULL, .offset = 0}
+
+static const struct pt_regs_offset regoffset_table[] = {
+ REG_OFFSET_NAME(r0, regs[0]),
+ REG_OFFSET_NAME(r1, regs[1]),
+ REG_OFFSET_NAME(r2, regs[2]),
+ REG_OFFSET_NAME(r3, regs[3]),
+ REG_OFFSET_NAME(r4, regs[4]),
+ REG_OFFSET_NAME(r5, regs[5]),
+ REG_OFFSET_NAME(r6, regs[6]),
+ REG_OFFSET_NAME(r7, regs[7]),
+ REG_OFFSET_NAME(r8, regs[8]),
+ REG_OFFSET_NAME(r9, regs[9]),
+ REG_OFFSET_NAME(r10, regs[10]),
+ REG_OFFSET_NAME(r11, regs[11]),
+ REG_OFFSET_NAME(r12, regs[12]),
+ REG_OFFSET_NAME(r13, regs[13]),
+ REG_OFFSET_NAME(r14, regs[14]),
+ REG_OFFSET_NAME(r15, regs[15]),
+ REG_OFFSET_NAME(r16, regs[16]),
+ REG_OFFSET_NAME(r17, regs[17]),
+ REG_OFFSET_NAME(r18, regs[18]),
+ REG_OFFSET_NAME(r19, regs[19]),
+ REG_OFFSET_NAME(r20, regs[20]),
+ REG_OFFSET_NAME(r21, regs[21]),
+ REG_OFFSET_NAME(r22, regs[22]),
+ REG_OFFSET_NAME(r23, regs[23]),
+ REG_OFFSET_NAME(r24, regs[24]),
+ REG_OFFSET_NAME(r25, regs[25]),
+ REG_OFFSET_NAME(r26, regs[26]),
+ REG_OFFSET_NAME(r27, regs[27]),
+ REG_OFFSET_NAME(r28, regs[28]),
+ REG_OFFSET_NAME(r29, regs[29]),
+ REG_OFFSET_NAME(r30, regs[30]),
+ REG_OFFSET_NAME(r31, regs[31]),
+ REG_OFFSET_NAME(orig_a0, orig_a0),
+ REG_OFFSET_NAME(csr_era, csr_era),
+ REG_OFFSET_NAME(csr_badvaddr, csr_badvaddr),
+ REG_OFFSET_NAME(csr_crmd, csr_crmd),
+ REG_OFFSET_NAME(csr_prmd, csr_prmd),
+ REG_OFFSET_NAME(csr_euen, csr_euen),
+ REG_OFFSET_NAME(csr_ecfg, csr_ecfg),
+ REG_OFFSET_NAME(csr_estat, csr_estat),
+ REG_OFFSET_END,
+};
+
+/**
+ * regs_query_register_offset() - query register offset from its name
+ * @name: the name of a register
+ *
+ * regs_query_register_offset() returns the offset of a register in struct
+ * pt_regs from its name. If the name is invalid, this returns -EINVAL;
+ */
+int regs_query_register_offset(const char *name)
+{
+ const struct pt_regs_offset *roff;
+
+ for (roff = regoffset_table; roff->name != NULL; roff++)
+ if (!strcmp(roff->name, name))
+ return roff->offset;
+ return -EINVAL;
+}
+
+enum loongarch_regset {
+ REGSET_GPR,
+ REGSET_FPR,
+ REGSET_CPUCFG,
+};
+
+static const struct user_regset loongarch64_regsets[] = {
+ [REGSET_GPR] = {
+ .core_note_type = NT_PRSTATUS,
+ .n = ELF_NGREG,
+ .size = sizeof(elf_greg_t),
+ .align = sizeof(elf_greg_t),
+ .regset_get = gpr_get,
+ .set = gpr_set,
+ },
+ [REGSET_FPR] = {
+ .core_note_type = NT_PRFPREG,
+ .n = ELF_NFPREG,
+ .size = sizeof(elf_fpreg_t),
+ .align = sizeof(elf_fpreg_t),
+ .regset_get = fpr_get,
+ .set = fpr_set,
+ },
+ [REGSET_CPUCFG] = {
+ .core_note_type = NT_LOONGARCH_CPUCFG,
+ .n = 64,
+ .size = sizeof(u32),
+ .align = sizeof(u32),
+ .regset_get = cfg_get,
+ .set = cfg_set,
+ },
+};
+
+static const struct user_regset_view user_loongarch64_view = {
+ .name = "loongarch64",
+ .e_machine = ELF_ARCH,
+ .regsets = loongarch64_regsets,
+ .n = ARRAY_SIZE(loongarch64_regsets),
+};
+
+
+const struct user_regset_view *task_user_regset_view(struct task_struct *task)
+{
+ return &user_loongarch64_view;
+}
+
+static inline int read_user(struct task_struct *target, unsigned long addr,
+ unsigned long __user *data)
+{
+ unsigned long tmp = 0;
+
+ switch (addr) {
+ case 0 ... 31:
+ tmp = task_pt_regs(target)->regs[addr];
+ break;
+ case ARG0:
+ tmp = task_pt_regs(target)->orig_a0;
+ break;
+ case PC:
+ tmp = task_pt_regs(target)->csr_era;
+ break;
+ case BADVADDR:
+ tmp = task_pt_regs(target)->csr_badvaddr;
+ break;
+ default:
+ return -EIO;
+ }
+
+ return put_user(tmp, data);
+}
+
+static inline int write_user(struct task_struct *target, unsigned long addr,
+ unsigned long data)
+{
+ switch (addr) {
+ case 0 ... 31:
+ task_pt_regs(target)->regs[addr] = data;
+ break;
+ case ARG0:
+ task_pt_regs(target)->orig_a0 = data;
+ break;
+ case PC:
+ task_pt_regs(target)->csr_era = data;
+ break;
+ case BADVADDR:
+ task_pt_regs(target)->csr_badvaddr = data;
+ break;
+ default:
+ return -EIO;
+ }
+
+ return 0;
+}
+
+long arch_ptrace(struct task_struct *child, long request,
+ unsigned long addr, unsigned long data)
+{
+ int ret;
+ unsigned long __user *datap = (void __user *) data;
+
+ switch (request) {
+ case PTRACE_PEEKUSR:
+ ret = read_user(child, addr, datap);
+ break;
+
+ case PTRACE_POKEUSR:
+ ret = write_user(child, addr, data);
+ break;
+
+ default:
+ ret = ptrace_request(child, request, addr, data);
+ break;
+ }
+
+ return ret;
+}
diff --git a/arch/loongarch/kernel/switch.S b/arch/loongarch/kernel/switch.S
new file mode 100644
index 000000000000..53e2fa8e580e
--- /dev/null
+++ b/arch/loongarch/kernel/switch.S
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/asm-offsets.h>
+#include <asm/loongarch.h>
+#include <asm/regdef.h>
+#include <asm/stackframe.h>
+#include <asm/thread_info.h>
+
+/*
+ * task_struct *__switch_to(task_struct *prev, task_struct *next,
+ * struct thread_info *next_ti)
+ */
+ .align 5
+SYM_FUNC_START(__switch_to)
+ csrrd t1, LOONGARCH_CSR_PRMD
+ stptr.d t1, a0, THREAD_CSRPRMD
+
+ cpu_save_nonscratch a0
+ stptr.d ra, a0, THREAD_REG01
+ move tp, a2
+ cpu_restore_nonscratch a1
+
+ li.w t0, _THREAD_SIZE - 32
+ PTR_ADD t0, t0, tp
+ set_saved_sp t0, t1, t2
+
+ ldptr.d t1, a1, THREAD_CSRPRMD
+ csrwr t1, LOONGARCH_CSR_PRMD
+
+ jr ra
+SYM_FUNC_END(__switch_to)
--
2.27.0


2022-05-18 10:16:05

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 17/22] LoongArch: Add some library functions

Add some library functions for LoongArch, including: delay, memset,
memcpy, memmove, copy_user, strncpy_user, strnlen_user and tlb dump
functions.

Reviewed-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/delay.h | 26 +++++++
arch/loongarch/include/asm/string.h | 12 +++
arch/loongarch/lib/clear_user.S | 43 +++++++++++
arch/loongarch/lib/copy_user.S | 47 ++++++++++++
arch/loongarch/lib/delay.c | 43 +++++++++++
arch/loongarch/lib/dump_tlb.c | 111 ++++++++++++++++++++++++++++
6 files changed, 282 insertions(+)
create mode 100644 arch/loongarch/include/asm/delay.h
create mode 100644 arch/loongarch/include/asm/string.h
create mode 100644 arch/loongarch/lib/clear_user.S
create mode 100644 arch/loongarch/lib/copy_user.S
create mode 100644 arch/loongarch/lib/delay.c
create mode 100644 arch/loongarch/lib/dump_tlb.c

diff --git a/arch/loongarch/include/asm/delay.h b/arch/loongarch/include/asm/delay.h
new file mode 100644
index 000000000000..36d775191310
--- /dev/null
+++ b/arch/loongarch/include/asm/delay.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_DELAY_H
+#define _ASM_DELAY_H
+
+#include <linux/param.h>
+
+extern void __delay(unsigned long cycles);
+extern void __ndelay(unsigned long ns);
+extern void __udelay(unsigned long us);
+
+#define ndelay(ns) __ndelay(ns)
+#define udelay(us) __udelay(us)
+
+/* make sure "usecs *= ..." in udelay do not overflow. */
+#if HZ >= 1000
+#define MAX_UDELAY_MS 1
+#elif HZ <= 200
+#define MAX_UDELAY_MS 5
+#else
+#define MAX_UDELAY_MS (1000 / HZ)
+#endif
+
+#endif /* _ASM_DELAY_H */
diff --git a/arch/loongarch/include/asm/string.h b/arch/loongarch/include/asm/string.h
new file mode 100644
index 000000000000..b07e60ded957
--- /dev/null
+++ b/arch/loongarch/include/asm/string.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_STRING_H
+#define _ASM_STRING_H
+
+extern void *memset(void *__s, int __c, size_t __count);
+extern void *memcpy(void *__to, __const__ void *__from, size_t __n);
+extern void *memmove(void *__dest, __const__ void *__src, size_t __n);
+
+#endif /* _ASM_STRING_H */
diff --git a/arch/loongarch/lib/clear_user.S b/arch/loongarch/lib/clear_user.S
new file mode 100644
index 000000000000..25d9be5fbb19
--- /dev/null
+++ b/arch/loongarch/lib/clear_user.S
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/export.h>
+#include <asm/regdef.h>
+
+.macro fixup_ex from, to, offset, fix
+.if \fix
+ .section .fixup, "ax"
+\to: addi.d a0, a1, \offset
+ jr ra
+ .previous
+.endif
+ .section __ex_table, "a"
+ PTR \from\()b, \to\()b
+ .previous
+.endm
+
+/*
+ * unsigned long __clear_user(void *addr, size_t size)
+ *
+ * a0: addr
+ * a1: size
+ */
+SYM_FUNC_START(__clear_user)
+ beqz a1, 2f
+
+1: st.b zero, a0, 0
+ addi.d a0, a0, 1
+ addi.d a1, a1, -1
+ bgt a1, zero, 1b
+
+2: move a0, a1
+ jr ra
+
+ fixup_ex 1, 3, 0, 1
+SYM_FUNC_END(__clear_user)
+
+EXPORT_SYMBOL(__clear_user)
diff --git a/arch/loongarch/lib/copy_user.S b/arch/loongarch/lib/copy_user.S
new file mode 100644
index 000000000000..9ae507f851b5
--- /dev/null
+++ b/arch/loongarch/lib/copy_user.S
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/export.h>
+#include <asm/regdef.h>
+
+.macro fixup_ex from, to, offset, fix
+.if \fix
+ .section .fixup, "ax"
+\to: addi.d a0, a2, \offset
+ jr ra
+ .previous
+.endif
+ .section __ex_table, "a"
+ PTR \from\()b, \to\()b
+ .previous
+.endm
+
+/*
+ * unsigned long __copy_user(void *to, const void *from, size_t n)
+ *
+ * a0: to
+ * a1: from
+ * a2: n
+ */
+SYM_FUNC_START(__copy_user)
+ beqz a2, 3f
+
+1: ld.b t0, a1, 0
+2: st.b t0, a0, 0
+ addi.d a0, a0, 1
+ addi.d a1, a1, 1
+ addi.d a2, a2, -1
+ bgt a2, zero, 1b
+
+3: move a0, a2
+ jr ra
+
+ fixup_ex 1, 4, 0, 1
+ fixup_ex 2, 4, 0, 0
+SYM_FUNC_END(__copy_user)
+
+EXPORT_SYMBOL(__copy_user)
diff --git a/arch/loongarch/lib/delay.c b/arch/loongarch/lib/delay.c
new file mode 100644
index 000000000000..5d856694fcfe
--- /dev/null
+++ b/arch/loongarch/lib/delay.c
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/delay.h>
+#include <linux/export.h>
+#include <linux/smp.h>
+#include <linux/timex.h>
+
+#include <asm/compiler.h>
+#include <asm/processor.h>
+
+void __delay(unsigned long cycles)
+{
+ u64 t0 = get_cycles();
+
+ while ((unsigned long)(get_cycles() - t0) < cycles)
+ cpu_relax();
+}
+EXPORT_SYMBOL(__delay);
+
+/*
+ * Division by multiplication: you don't have to worry about
+ * loss of precision.
+ *
+ * Use only for very small delays ( < 1 msec). Should probably use a
+ * lookup table, really, as the multiplications take much too long with
+ * short delays. This is a "reasonable" implementation, though (and the
+ * first constant multiplications gets optimized away if the delay is
+ * a constant)
+ */
+
+void __udelay(unsigned long us)
+{
+ __delay((us * 0x000010c7ull * HZ * lpj_fine) >> 32);
+}
+EXPORT_SYMBOL(__udelay);
+
+void __ndelay(unsigned long ns)
+{
+ __delay((ns * 0x00000005ull * HZ * lpj_fine) >> 32);
+}
+EXPORT_SYMBOL(__ndelay);
diff --git a/arch/loongarch/lib/dump_tlb.c b/arch/loongarch/lib/dump_tlb.c
new file mode 100644
index 000000000000..cda2c6bc7f09
--- /dev/null
+++ b/arch/loongarch/lib/dump_tlb.c
@@ -0,0 +1,111 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1994, 1995 by Waldorf Electronics, written by Ralf Baechle.
+ * Copyright (C) 1999 by Silicon Graphics, Inc.
+ */
+#include <linux/kernel.h>
+#include <linux/mm.h>
+
+#include <asm/loongarch.h>
+#include <asm/page.h>
+#include <asm/pgtable.h>
+#include <asm/tlb.h>
+
+void dump_tlb_regs(void)
+{
+ const int field = 2 * sizeof(unsigned long);
+
+ pr_info("Index : %0x\n", read_csr_tlbidx());
+ pr_info("PageSize : %0x\n", read_csr_pagesize());
+ pr_info("EntryHi : %0*llx\n", field, read_csr_entryhi());
+ pr_info("EntryLo0 : %0*llx\n", field, read_csr_entrylo0());
+ pr_info("EntryLo1 : %0*llx\n", field, read_csr_entrylo1());
+}
+
+static void dump_tlb(int first, int last)
+{
+ unsigned long s_entryhi, entryhi, asid;
+ unsigned long long entrylo0, entrylo1, pa;
+ unsigned int index;
+ unsigned int s_index, s_asid;
+ unsigned int pagesize, c0, c1, i;
+ unsigned long asidmask = cpu_asid_mask(&current_cpu_data);
+ int pwidth = 11;
+ int vwidth = 11;
+ int asidwidth = DIV_ROUND_UP(ilog2(asidmask) + 1, 4);
+
+ s_entryhi = read_csr_entryhi();
+ s_index = read_csr_tlbidx();
+ s_asid = read_csr_asid();
+
+ for (i = first; i <= last; i++) {
+ write_csr_index(i);
+ tlb_read();
+ pagesize = read_csr_pagesize();
+ entryhi = read_csr_entryhi();
+ entrylo0 = read_csr_entrylo0();
+ entrylo1 = read_csr_entrylo1();
+ index = read_csr_tlbidx();
+ asid = read_csr_asid();
+
+ /* EHINV bit marks entire entry as invalid */
+ if (index & CSR_TLBIDX_EHINV)
+ continue;
+ /*
+ * ASID takes effect in absence of G (global) bit.
+ */
+ if (!((entrylo0 | entrylo1) & ENTRYLO_G) &&
+ asid != s_asid)
+ continue;
+
+ /*
+ * Only print entries in use
+ */
+ pr_info("Index: %2d pgsize=%x ", i, (1 << pagesize));
+
+ c0 = (entrylo0 & ENTRYLO_C) >> ENTRYLO_C_SHIFT;
+ c1 = (entrylo1 & ENTRYLO_C) >> ENTRYLO_C_SHIFT;
+
+ pr_cont("va=%0*lx asid=%0*lx",
+ vwidth, (entryhi & ~0x1fffUL), asidwidth, asid & asidmask);
+
+ /* NR/NX are in awkward places, so mask them off separately */
+ pa = entrylo0 & ~(ENTRYLO_NR | ENTRYLO_NX);
+ pa = pa & PAGE_MASK;
+ pr_cont("\n\t[");
+ pr_cont("ri=%d xi=%d ",
+ (entrylo0 & ENTRYLO_NR) ? 1 : 0,
+ (entrylo0 & ENTRYLO_NX) ? 1 : 0);
+ pr_cont("pa=%0*llx c=%d d=%d v=%d g=%d plv=%lld] [",
+ pwidth, pa, c0,
+ (entrylo0 & ENTRYLO_D) ? 1 : 0,
+ (entrylo0 & ENTRYLO_V) ? 1 : 0,
+ (entrylo0 & ENTRYLO_G) ? 1 : 0,
+ (entrylo0 & ENTRYLO_PLV) >> ENTRYLO_PLV_SHIFT);
+ /* NR/NX are in awkward places, so mask them off separately */
+ pa = entrylo1 & ~(ENTRYLO_NR | ENTRYLO_NX);
+ pa = pa & PAGE_MASK;
+ pr_cont("ri=%d xi=%d ",
+ (entrylo1 & ENTRYLO_NR) ? 1 : 0,
+ (entrylo1 & ENTRYLO_NX) ? 1 : 0);
+ pr_cont("pa=%0*llx c=%d d=%d v=%d g=%d plv=%lld]\n",
+ pwidth, pa, c1,
+ (entrylo1 & ENTRYLO_D) ? 1 : 0,
+ (entrylo1 & ENTRYLO_V) ? 1 : 0,
+ (entrylo1 & ENTRYLO_G) ? 1 : 0,
+ (entrylo1 & ENTRYLO_PLV) >> ENTRYLO_PLV_SHIFT);
+ }
+ pr_info("\n");
+
+ write_csr_entryhi(s_entryhi);
+ write_csr_tlbidx(s_index);
+ write_csr_asid(s_asid);
+}
+
+void dump_tlb_all(void)
+{
+ dump_tlb(0, current_cpu_data.tlbsize - 1);
+}
--
2.27.0


2022-05-18 10:19:00

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 19/22] LoongArch: Add VDSO and VSYSCALL support

Add VDSO and VSYSCALL support (sigreturn, gettimeofday and its friends)
for LoongArch.

Reviewed-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/vdso.h | 38 +++++
arch/loongarch/include/asm/vdso/clocksource.h | 8 +
.../loongarch/include/asm/vdso/gettimeofday.h | 99 +++++++++++++
arch/loongarch/include/asm/vdso/processor.h | 14 ++
arch/loongarch/include/asm/vdso/vdso.h | 30 ++++
arch/loongarch/include/asm/vdso/vsyscall.h | 27 ++++
arch/loongarch/kernel/vdso.c | 138 ++++++++++++++++++
arch/loongarch/vdso/Makefile | 96 ++++++++++++
arch/loongarch/vdso/elf.S | 15 ++
arch/loongarch/vdso/gen_vdso_offsets.sh | 13 ++
arch/loongarch/vdso/sigreturn.S | 24 +++
arch/loongarch/vdso/vdso.S | 22 +++
arch/loongarch/vdso/vdso.lds.S | 72 +++++++++
arch/loongarch/vdso/vgettimeofday.c | 25 ++++
14 files changed, 621 insertions(+)
create mode 100644 arch/loongarch/include/asm/vdso.h
create mode 100644 arch/loongarch/include/asm/vdso/clocksource.h
create mode 100644 arch/loongarch/include/asm/vdso/gettimeofday.h
create mode 100644 arch/loongarch/include/asm/vdso/processor.h
create mode 100644 arch/loongarch/include/asm/vdso/vdso.h
create mode 100644 arch/loongarch/include/asm/vdso/vsyscall.h
create mode 100644 arch/loongarch/kernel/vdso.c
create mode 100644 arch/loongarch/vdso/Makefile
create mode 100644 arch/loongarch/vdso/elf.S
create mode 100755 arch/loongarch/vdso/gen_vdso_offsets.sh
create mode 100644 arch/loongarch/vdso/sigreturn.S
create mode 100644 arch/loongarch/vdso/vdso.S
create mode 100644 arch/loongarch/vdso/vdso.lds.S
create mode 100644 arch/loongarch/vdso/vgettimeofday.c

diff --git a/arch/loongarch/include/asm/vdso.h b/arch/loongarch/include/asm/vdso.h
new file mode 100644
index 000000000000..996bddae12dc
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASM_VDSO_H
+#define __ASM_VDSO_H
+
+#include <linux/mm_types.h>
+#include <vdso/datapage.h>
+
+#include <asm/barrier.h>
+
+/**
+ * struct loongarch_vdso_info - Details of a VDSO image.
+ * @vdso: Pointer to VDSO image (page-aligned).
+ * @size: Size of the VDSO image (page-aligned).
+ * @off_rt_sigreturn: Offset of the rt_sigreturn() trampoline.
+ * @code_mapping: Special mapping structure for vdso code.
+ * @code_mapping: Special mapping structure for vdso data.
+ *
+ * This structure contains details of a VDSO image, including the image data
+ * and offsets of certain symbols required by the kernel. It is generated as
+ * part of the VDSO build process, aside from the mapping page array, which is
+ * populated at runtime.
+ */
+struct loongarch_vdso_info {
+ void *vdso;
+ unsigned long size;
+ unsigned long offset_sigreturn;
+ struct vm_special_mapping code_mapping;
+ struct vm_special_mapping data_mapping;
+};
+
+extern struct loongarch_vdso_info vdso_info;
+
+#endif /* __ASM_VDSO_H */
diff --git a/arch/loongarch/include/asm/vdso/clocksource.h b/arch/loongarch/include/asm/vdso/clocksource.h
new file mode 100644
index 000000000000..13cd580d406d
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso/clocksource.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef __ASM_VDSOCLOCKSOURCE_H
+#define __ASM_VDSOCLOCKSOURCE_H
+
+#define VDSO_ARCH_CLOCKMODES \
+ VDSO_CLOCKMODE_CPU
+
+#endif /* __ASM_VDSOCLOCKSOURCE_H */
diff --git a/arch/loongarch/include/asm/vdso/gettimeofday.h b/arch/loongarch/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..b88e09b343ff
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,99 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <asm/vdso/vdso.h>
+
+#define VDSO_HAS_CLOCK_GETRES 1
+
+static __always_inline long gettimeofday_fallback(
+ struct __kernel_old_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct __kernel_old_timeval *tv asm("a0") = _tv;
+ register struct timezone *tz asm("a1") = _tz;
+ register long nr asm("a7") = __NR_gettimeofday;
+ register long ret asm("a0");
+
+ asm volatile(
+ " syscall 0\n"
+ : "+r" (ret)
+ : "r" (nr), "r" (tv), "r" (tz)
+ : "$t0", "$t1", "$t2", "$t3", "$t4", "$t5", "$t6", "$t7",
+ "$t8", "memory");
+
+ return ret;
+}
+
+static __always_inline long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __kernel_timespec *_ts)
+{
+ register clockid_t clkid asm("a0") = _clkid;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
+ register long nr asm("a7") = __NR_clock_gettime;
+ register long ret asm("a0");
+
+ asm volatile(
+ " syscall 0\n"
+ : "+r" (ret)
+ : "r" (nr), "r" (clkid), "r" (ts)
+ : "$t0", "$t1", "$t2", "$t3", "$t4", "$t5", "$t6", "$t7",
+ "$t8", "memory");
+
+ return ret;
+}
+
+static __always_inline int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __kernel_timespec *_ts)
+{
+ register clockid_t clkid asm("a0") = _clkid;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
+ register long nr asm("a7") = __NR_clock_getres;
+ register long ret asm("a0");
+
+ asm volatile(
+ " syscall 0\n"
+ : "+r" (ret)
+ : "r" (nr), "r" (clkid), "r" (ts)
+ : "$t0", "$t1", "$t2", "$t3", "$t4", "$t5", "$t6", "$t7",
+ "$t8", "memory");
+
+ return ret;
+}
+
+static __always_inline u64 __arch_get_hw_counter(s32 clock_mode,
+ const struct vdso_data *vd)
+{
+ unsigned int count;
+
+ __asm__ __volatile__(
+ " rdtime.d %0, $zero\n"
+ : "=r" (count));
+
+ return count;
+}
+
+static inline bool loongarch_vdso_hres_capable(void)
+{
+ return true;
+}
+#define __arch_vdso_hres_capable loongarch_vdso_hres_capable
+
+static __always_inline const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return get_vdso_data();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/loongarch/include/asm/vdso/processor.h b/arch/loongarch/include/asm/vdso/processor.h
new file mode 100644
index 000000000000..ef5770b343a0
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso/processor.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef __ASM_VDSO_PROCESSOR_H
+#define __ASM_VDSO_PROCESSOR_H
+
+#ifndef __ASSEMBLY__
+
+#define cpu_relax() barrier()
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_PROCESSOR_H */
diff --git a/arch/loongarch/include/asm/vdso/vdso.h b/arch/loongarch/include/asm/vdso/vdso.h
new file mode 100644
index 000000000000..5a01643a65b3
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso/vdso.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASSEMBLY__
+
+#include <asm/asm.h>
+#include <asm/page.h>
+
+static inline unsigned long get_vdso_base(void)
+{
+ unsigned long addr;
+
+ __asm__(
+ " la.pcrel %0, _start\n"
+ : "=r" (addr)
+ :
+ :);
+
+ return addr;
+}
+
+static inline const struct vdso_data *get_vdso_data(void)
+{
+ return (const struct vdso_data *)(get_vdso_base() - PAGE_SIZE);
+}
+
+#endif /* __ASSEMBLY__ */
diff --git a/arch/loongarch/include/asm/vdso/vsyscall.h b/arch/loongarch/include/asm/vdso/vsyscall.h
new file mode 100644
index 000000000000..5de615383a22
--- /dev/null
+++ b/arch/loongarch/include/asm/vdso/vsyscall.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ASM_VDSO_VSYSCALL_H
+#define __ASM_VDSO_VSYSCALL_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/timekeeper_internal.h>
+#include <vdso/datapage.h>
+
+extern struct vdso_data *vdso_data;
+
+/*
+ * Update the vDSO data page to keep in sync with kernel timekeeping.
+ */
+static __always_inline
+struct vdso_data *__loongarch_get_k_vdso_data(void)
+{
+ return vdso_data;
+}
+#define __arch_get_k_vdso_data __loongarch_get_k_vdso_data
+
+/* The asm-generic header needs to be included after the definitions above */
+#include <asm-generic/vdso/vsyscall.h>
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_VSYSCALL_H */
diff --git a/arch/loongarch/kernel/vdso.c b/arch/loongarch/kernel/vdso.c
new file mode 100644
index 000000000000..e20c8ca87473
--- /dev/null
+++ b/arch/loongarch/kernel/vdso.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/binfmts.h>
+#include <linux/elf.h>
+#include <linux/err.h>
+#include <linux/init.h>
+#include <linux/ioport.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/random.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/timekeeper_internal.h>
+
+#include <asm/page.h>
+#include <asm/vdso.h>
+#include <vdso/helpers.h>
+#include <vdso/vsyscall.h>
+#include <generated/vdso-offsets.h>
+
+extern char vdso_start[], vdso_end[];
+
+/* Kernel-provided data used by the VDSO. */
+static union loongarch_vdso_data {
+ u8 page[PAGE_SIZE];
+ struct vdso_data data[CS_BASES];
+} loongarch_vdso_data __page_aligned_data;
+struct vdso_data *vdso_data = loongarch_vdso_data.data;
+static struct page *vdso_pages[] = { NULL };
+
+static int vdso_mremap(const struct vm_special_mapping *sm, struct vm_area_struct *new_vma)
+{
+ current->mm->context.vdso = (void *)(new_vma->vm_start);
+
+ return 0;
+}
+
+struct loongarch_vdso_info vdso_info = {
+ .vdso = vdso_start,
+ .size = PAGE_SIZE,
+ .code_mapping = {
+ .name = "[vdso]",
+ .pages = vdso_pages,
+ .mremap = vdso_mremap,
+ },
+ .data_mapping = {
+ .name = "[vvar]",
+ },
+ .offset_sigreturn = vdso_offset_sigreturn,
+};
+
+static int __init init_vdso(void)
+{
+ unsigned long i, pfn;
+
+ BUG_ON(!PAGE_ALIGNED(vdso_info.vdso));
+ BUG_ON(!PAGE_ALIGNED(vdso_info.size));
+
+ pfn = __phys_to_pfn(__pa_symbol(vdso_info.vdso));
+ for (i = 0; i < vdso_info.size / PAGE_SIZE; i++)
+ vdso_info.code_mapping.pages[i] = pfn_to_page(pfn + i);
+
+ return 0;
+}
+subsys_initcall(init_vdso);
+
+static unsigned long vdso_base(void)
+{
+ unsigned long base = STACK_TOP;
+
+ if (current->flags & PF_RANDOMIZE) {
+ base += get_random_int() & (VDSO_RANDOMIZE_SIZE - 1);
+ base = PAGE_ALIGN(base);
+ }
+
+ return base;
+}
+
+int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
+{
+ int ret;
+ unsigned long vvar_size, size, data_addr, vdso_addr;
+ struct mm_struct *mm = current->mm;
+ struct vm_area_struct *vma;
+ struct loongarch_vdso_info *info = current->thread.vdso;
+
+ if (mmap_write_lock_killable(mm))
+ return -EINTR;
+
+ /*
+ * Determine total area size. This includes the VDSO data itself
+ * and the data page.
+ */
+ vvar_size = PAGE_SIZE;
+ size = vvar_size + info->size;
+
+ data_addr = get_unmapped_area(NULL, vdso_base(), size, 0, 0);
+ if (IS_ERR_VALUE(data_addr)) {
+ ret = data_addr;
+ goto out;
+ }
+ vdso_addr = data_addr + PAGE_SIZE;
+
+ vma = _install_special_mapping(mm, data_addr, vvar_size,
+ VM_READ | VM_MAYREAD,
+ &info->data_mapping);
+ if (IS_ERR(vma)) {
+ ret = PTR_ERR(vma);
+ goto out;
+ }
+
+ /* Map VDSO data page. */
+ ret = remap_pfn_range(vma, data_addr,
+ virt_to_phys(vdso_data) >> PAGE_SHIFT,
+ PAGE_SIZE, PAGE_READONLY);
+ if (ret)
+ goto out;
+
+ /* Map VDSO code page. */
+ vma = _install_special_mapping(mm, vdso_addr, info->size,
+ VM_READ | VM_EXEC | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC,
+ &info->code_mapping);
+ if (IS_ERR(vma)) {
+ ret = PTR_ERR(vma);
+ goto out;
+ }
+
+ mm->context.vdso = (void *)vdso_addr;
+ ret = 0;
+
+out:
+ mmap_write_unlock(mm);
+ return ret;
+}
diff --git a/arch/loongarch/vdso/Makefile b/arch/loongarch/vdso/Makefile
new file mode 100644
index 000000000000..6b6e16732c60
--- /dev/null
+++ b/arch/loongarch/vdso/Makefile
@@ -0,0 +1,96 @@
+# SPDX-License-Identifier: GPL-2.0
+# Objects to go into the VDSO.
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_LARCH_32|R_LARCH_64|R_LARCH_MARK_LA|R_LARCH_JUMP_SLOT
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso-y := elf.o vgettimeofday.o sigreturn.o
+
+# Common compiler flags between ABIs.
+ccflags-vdso := \
+ $(filter -I%,$(KBUILD_CFLAGS)) \
+ $(filter -E%,$(KBUILD_CFLAGS)) \
+ $(filter -march=%,$(KBUILD_CFLAGS)) \
+ $(filter -m%-float,$(KBUILD_CFLAGS)) \
+ -D__VDSO__
+
+ifeq ($(cc-name),clang)
+ccflags-vdso += $(filter --target=%,$(KBUILD_CFLAGS))
+endif
+
+cflags-vdso := $(ccflags-vdso) \
+ $(filter -W%,$(filter-out -Wa$(comma)%,$(KBUILD_CFLAGS))) \
+ -O2 -g -fno-strict-aliasing -fno-common -fno-builtin -G0 \
+ -fno-stack-protector -fno-jump-tables -DDISABLE_BRANCH_PROFILING \
+ $(call cc-option, -fno-asynchronous-unwind-tables) \
+ $(call cc-option, -fno-stack-protector)
+aflags-vdso := $(ccflags-vdso) \
+ -D__ASSEMBLY__ -Wa,-gdwarf-2
+
+ifneq ($(c-gettimeofday-y),)
+ CFLAGS_vgettimeofday.o += -include $(c-gettimeofday-y)
+endif
+
+# VDSO linker flags.
+ldflags-y := -Bsymbolic --no-undefined -soname=linux-vdso.so.1 \
+ $(filter -E%,$(KBUILD_CFLAGS)) -nostdlib -shared \
+ --hash-style=sysv --build-id -T
+
+GCOV_PROFILE := n
+
+#
+# Shared build commands.
+#
+
+quiet_cmd_vdsold_and_vdso_check = LD $@
+ cmd_vdsold_and_vdso_check = $(cmd_ld); $(cmd_vdso_check)
+
+quiet_cmd_vdsoas_o_S = AS $@
+ cmd_vdsoas_o_S = $(CC) $(a_flags) -c -o $@ $<
+
+# Generate VDSO offsets using helper script
+gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh
+quiet_cmd_vdsosym = VDSOSYM $@
+ cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
+
+include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,vdsosym)
+
+#
+# Build native VDSO.
+#
+
+native-abi := $(filter -mabi=%,$(KBUILD_CFLAGS))
+
+targets += $(obj-vdso-y)
+targets += vdso.lds vdso.so.dbg vdso.so
+
+obj-vdso := $(obj-vdso-y:%.o=$(obj)/%.o)
+
+$(obj-vdso): KBUILD_CFLAGS := $(cflags-vdso) $(native-abi)
+$(obj-vdso): KBUILD_AFLAGS := $(aflags-vdso) $(native-abi)
+
+$(obj)/vdso.lds: KBUILD_CPPFLAGS := $(ccflags-vdso) $(native-abi)
+
+$(obj)/vdso.so.dbg: $(obj)/vdso.lds $(obj-vdso) FORCE
+ $(call if_changed,vdsold_and_vdso_check)
+
+$(obj)/vdso.so: OBJCOPYFLAGS := -S
+$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,objcopy)
+
+obj-y += vdso.o
+
+$(obj)/vdso.o : $(obj)/vdso.so
+
+# install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/[email protected] $(MODLIB)/vdso/$@
+
+vdso.so: $(obj)/vdso.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso_install: vdso.so
diff --git a/arch/loongarch/vdso/elf.S b/arch/loongarch/vdso/elf.S
new file mode 100644
index 000000000000..9bb21b9f9583
--- /dev/null
+++ b/arch/loongarch/vdso/elf.S
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/vdso/vdso.h>
+
+#include <linux/elfnote.h>
+#include <linux/version.h>
+
+ELFNOTE_START(Linux, 0, "a")
+ .long LINUX_VERSION_CODE
+ELFNOTE_END
diff --git a/arch/loongarch/vdso/gen_vdso_offsets.sh b/arch/loongarch/vdso/gen_vdso_offsets.sh
new file mode 100755
index 000000000000..1bb4e12642ff
--- /dev/null
+++ b/arch/loongarch/vdso/gen_vdso_offsets.sh
@@ -0,0 +1,13 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+
+#
+# Derived from RISC-V and ARM64:
+# Author: Will Deacon <[email protected]>
+#
+# Match symbols in the DSO that look like VDSO_*; produce a header file
+# of constant offsets into the shared object.
+#
+
+LC_ALL=C sed -n -e 's/^00*/0/' -e \
+'s/^\([0-9a-fA-F]*\) . VDSO_\([a-zA-Z0-9_]*\)$/\#define vdso_offset_\2\t0x\1/p'
diff --git a/arch/loongarch/vdso/sigreturn.S b/arch/loongarch/vdso/sigreturn.S
new file mode 100644
index 000000000000..9cb3c58fad03
--- /dev/null
+++ b/arch/loongarch/vdso/sigreturn.S
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/vdso/vdso.h>
+
+#include <linux/linkage.h>
+#include <uapi/asm/unistd.h>
+
+#include <asm/regdef.h>
+#include <asm/asm.h>
+
+ .section .text
+ .cfi_sections .debug_frame
+
+SYM_FUNC_START(__vdso_rt_sigreturn)
+
+ li.w a7, __NR_rt_sigreturn
+ syscall 0
+
+SYM_FUNC_END(__vdso_rt_sigreturn)
diff --git a/arch/loongarch/vdso/vdso.S b/arch/loongarch/vdso/vdso.S
new file mode 100644
index 000000000000..46789bade6ff
--- /dev/null
+++ b/arch/loongarch/vdso/vdso.S
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from RISC-V:
+ * Copyright (C) 2014 Regents of the University of California
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <asm/page.h>
+
+ __PAGE_ALIGNED_DATA
+
+ .globl vdso_start, vdso_end
+ .balign PAGE_SIZE
+vdso_start:
+ .incbin "arch/loongarch/vdso/vdso.so"
+ .balign PAGE_SIZE
+vdso_end:
+
+ .previous
diff --git a/arch/loongarch/vdso/vdso.lds.S b/arch/loongarch/vdso/vdso.lds.S
new file mode 100644
index 000000000000..955f02de4a2d
--- /dev/null
+++ b/arch/loongarch/vdso/vdso.lds.S
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Author: Huacai Chen <[email protected]>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+OUTPUT_FORMAT("elf64-loongarch", "elf64-loongarch", "elf64-loongarch")
+
+OUTPUT_ARCH(loongarch)
+
+SECTIONS
+{
+ PROVIDE(_start = .);
+ . = SIZEOF_HEADERS;
+
+ .hash : { *(.hash) } :text
+ .gnu.hash : { *(.gnu.hash) }
+ .dynsym : { *(.dynsym) }
+ .dynstr : { *(.dynstr) }
+ .gnu.version : { *(.gnu.version) }
+ .gnu.version_d : { *(.gnu.version_d) }
+ .gnu.version_r : { *(.gnu.version_r) }
+
+ .note : { *(.note.*) } :text :note
+
+ .text : { *(.text*) } :text
+ PROVIDE (__etext = .);
+ PROVIDE (_etext = .);
+ PROVIDE (etext = .);
+
+ .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr
+ .eh_frame : { KEEP (*(.eh_frame)) } :text
+
+ .dynamic : { *(.dynamic) } :text :dynamic
+
+ .rodata : { *(.rodata*) } :text
+
+ _end = .;
+ PROVIDE(end = .);
+
+ /DISCARD/ : {
+ *(.gnu.attributes)
+ *(.note.GNU-stack)
+ *(.data .data.* .gnu.linkonce.d.* .sdata*)
+ *(.bss .sbss .dynbss .dynsbss)
+ }
+}
+
+PHDRS
+{
+ text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
+ dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
+ note PT_NOTE FLAGS(4); /* PF_R */
+ eh_frame_hdr PT_GNU_EH_FRAME;
+}
+
+VERSION
+{
+ LINUX_5.10 {
+ global:
+ __vdso_clock_getres;
+ __vdso_clock_gettime;
+ __vdso_gettimeofday;
+ __vdso_rt_sigreturn;
+ local: *;
+ };
+}
+
+/*
+ * Make the sigreturn code visible to the kernel.
+ */
+VDSO_sigreturn = __vdso_rt_sigreturn;
diff --git a/arch/loongarch/vdso/vgettimeofday.c b/arch/loongarch/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..b1f4548dae92
--- /dev/null
+++ b/arch/loongarch/vdso/vgettimeofday.c
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * LoongArch userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/types.h>
+
+int __vdso_clock_gettime(clockid_t clock,
+ struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+int __vdso_clock_getres(clockid_t clock_id,
+ struct __kernel_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
--
2.27.0


2022-05-18 10:34:09

by Huacai Chen

[permalink] [raw]
Subject: [PATCH V11 13/22] LoongArch: Add system call support

Add system call support and related uaccess.h for LoongArch.

Q: Why keep _ARCH_WANT_SYS_CLONE definition while there is clone3:
A: The latest glibc release has some basic support for clone3 but it is
not complete. E.g., pthread_create() and spawni() have converted to
use clone3 but fork() will still use clone. Moreover, some seccomp
related applications can still not work perfectly with clone3. E.g.,
Chromium sandbox cannot work at all and there is no solution for it,
which is more terrible than the fork() story [1].

[1] https://chromium-review.googlesource.com/c/chromium/src/+/2936184

Reviewed-by: WANG Xuerui <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
---
arch/loongarch/include/asm/seccomp.h | 20 ++
arch/loongarch/include/asm/syscall.h | 74 +++++++
arch/loongarch/include/asm/uaccess.h | 269 +++++++++++++++++++++++
arch/loongarch/include/asm/unistd.h | 11 +
arch/loongarch/include/uapi/asm/unistd.h | 5 +
arch/loongarch/kernel/entry.S | 89 ++++++++
arch/loongarch/kernel/syscall.c | 63 ++++++
7 files changed, 531 insertions(+)
create mode 100644 arch/loongarch/include/asm/seccomp.h
create mode 100644 arch/loongarch/include/asm/syscall.h
create mode 100644 arch/loongarch/include/asm/uaccess.h
create mode 100644 arch/loongarch/include/asm/unistd.h
create mode 100644 arch/loongarch/include/uapi/asm/unistd.h
create mode 100644 arch/loongarch/kernel/entry.S
create mode 100644 arch/loongarch/kernel/syscall.c

diff --git a/arch/loongarch/include/asm/seccomp.h b/arch/loongarch/include/asm/seccomp.h
new file mode 100644
index 000000000000..31d6ab42e43e
--- /dev/null
+++ b/arch/loongarch/include/asm/seccomp.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _ASM_SECCOMP_H
+#define _ASM_SECCOMP_H
+
+#include <asm/unistd.h>
+
+#include <asm-generic/seccomp.h>
+
+#ifdef CONFIG_32BIT
+# define SECCOMP_ARCH_NATIVE AUDIT_ARCH_LOONGARCH32
+# define SECCOMP_ARCH_NATIVE_NR NR_syscalls
+# define SECCOMP_ARCH_NATIVE_NAME "loongarch32"
+#else
+# define SECCOMP_ARCH_NATIVE AUDIT_ARCH_LOONGARCH64
+# define SECCOMP_ARCH_NATIVE_NR NR_syscalls
+# define SECCOMP_ARCH_NATIVE_NAME "loongarch64"
+#endif
+
+#endif /* _ASM_SECCOMP_H */
diff --git a/arch/loongarch/include/asm/syscall.h b/arch/loongarch/include/asm/syscall.h
new file mode 100644
index 000000000000..e286dc58476e
--- /dev/null
+++ b/arch/loongarch/include/asm/syscall.h
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Author: Hanlu Li <[email protected]>
+ * Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASM_LOONGARCH_SYSCALL_H
+#define __ASM_LOONGARCH_SYSCALL_H
+
+#include <linux/compiler.h>
+#include <uapi/linux/audit.h>
+#include <linux/elf-em.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/uaccess.h>
+#include <asm/ptrace.h>
+#include <asm/unistd.h>
+
+extern void *sys_call_table[];
+
+static inline long syscall_get_nr(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ return regs->regs[11];
+}
+
+static inline void syscall_rollback(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ regs->regs[4] = regs->orig_a0;
+}
+
+static inline long syscall_get_error(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ unsigned long error = regs->regs[4];
+
+ return IS_ERR_VALUE(error) ? error : 0;
+}
+
+static inline long syscall_get_return_value(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ return regs->regs[4];
+}
+
+static inline void syscall_set_return_value(struct task_struct *task,
+ struct pt_regs *regs,
+ int error, long val)
+{
+ regs->regs[4] = (long) error ? error : val;
+}
+
+static inline void syscall_get_arguments(struct task_struct *task,
+ struct pt_regs *regs,
+ unsigned long *args)
+{
+ args[0] = regs->orig_a0;
+ memcpy(&args[1], &regs->regs[5], 5 * sizeof(long));
+}
+
+static inline int syscall_get_arch(struct task_struct *task)
+{
+ return AUDIT_ARCH_LOONGARCH64;
+}
+
+static inline bool arch_syscall_is_vdso_sigreturn(struct pt_regs *regs)
+{
+ return false;
+}
+
+#endif /* __ASM_LOONGARCH_SYSCALL_H */
diff --git a/arch/loongarch/include/asm/uaccess.h b/arch/loongarch/include/asm/uaccess.h
new file mode 100644
index 000000000000..217c6a3727b1
--- /dev/null
+++ b/arch/loongarch/include/asm/uaccess.h
@@ -0,0 +1,269 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1996, 1997, 1998, 1999, 2000, 03, 04 by Ralf Baechle
+ * Copyright (C) 1999, 2000 Silicon Graphics, Inc.
+ * Copyright (C) 2007 Maciej W. Rozycki
+ * Copyright (C) 2014, Imagination Technologies Ltd.
+ */
+#ifndef _ASM_UACCESS_H
+#define _ASM_UACCESS_H
+
+#include <linux/kernel.h>
+#include <linux/string.h>
+#include <linux/extable.h>
+#include <asm/pgtable.h>
+#include <asm-generic/extable.h>
+#include <asm-generic/access_ok.h>
+
+extern u64 __ua_limit;
+
+#define __UA_ADDR ".dword"
+#define __UA_LA "la.abs"
+#define __UA_LIMIT __ua_limit
+
+/*
+ * get_user: - Get a simple variable from user space.
+ * @x: Variable to store result.
+ * @ptr: Source address, in user space.
+ *
+ * Context: User context only. This function may sleep if pagefaults are
+ * enabled.
+ *
+ * This macro copies a single simple variable from user space to kernel
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and the result of
+ * dereferencing @ptr must be assignable to @x without a cast.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ * On error, the variable @x is set to zero.
+ */
+#define get_user(x, ptr) \
+({ \
+ const __typeof__(*(ptr)) __user *__p = (ptr); \
+ \
+ might_fault(); \
+ access_ok(__p, sizeof(*__p)) ? __get_user((x), __p) : \
+ ((x) = 0, -EFAULT); \
+})
+
+/*
+ * put_user: - Write a simple value into user space.
+ * @x: Value to copy to user space.
+ * @ptr: Destination address, in user space.
+ *
+ * Context: User context only. This function may sleep if pagefaults are
+ * enabled.
+ *
+ * This macro copies a single simple value from kernel space to user
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and @x must be assignable
+ * to the result of dereferencing @ptr.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ */
+#define put_user(x, ptr) \
+({ \
+ __typeof__(*(ptr)) __user *__p = (ptr); \
+ \
+ might_fault(); \
+ access_ok(__p, sizeof(*__p)) ? __put_user((x), __p) : -EFAULT; \
+})
+
+/*
+ * __get_user: - Get a simple variable from user space, with less checking.
+ * @x: Variable to store result.
+ * @ptr: Source address, in user space.
+ *
+ * Context: User context only. This function may sleep if pagefaults are
+ * enabled.
+ *
+ * This macro copies a single simple variable from user space to kernel
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and the result of
+ * dereferencing @ptr must be assignable to @x without a cast.
+ *
+ * Caller must check the pointer with access_ok() before calling this
+ * function.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ * On error, the variable @x is set to zero.
+ */
+#define __get_user(x, ptr) \
+({ \
+ int __gu_err = 0; \
+ \
+ __chk_user_ptr(ptr); \
+ __get_user_common((x), sizeof(*(ptr)), ptr); \
+ __gu_err; \
+})
+
+/*
+ * __put_user: - Write a simple value into user space, with less checking.
+ * @x: Value to copy to user space.
+ * @ptr: Destination address, in user space.
+ *
+ * Context: User context only. This function may sleep if pagefaults are
+ * enabled.
+ *
+ * This macro copies a single simple value from kernel space to user
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and @x must be assignable
+ * to the result of dereferencing @ptr.
+ *
+ * Caller must check the pointer with access_ok() before calling this
+ * function.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ */
+#define __put_user(x, ptr) \
+({ \
+ int __pu_err = 0; \
+ __typeof__(*(ptr)) __pu_val; \
+ \
+ __pu_val = (x); \
+ __chk_user_ptr(ptr); \
+ __put_user_common(ptr, sizeof(*(ptr))); \
+ __pu_err; \
+})
+
+struct __large_struct { unsigned long buf[100]; };
+#define __m(x) (*(struct __large_struct __user *)(x))
+
+#define __get_user_common(val, size, ptr) \
+do { \
+ switch (size) { \
+ case 1: __get_data_asm(val, "ld.b", ptr); break; \
+ case 2: __get_data_asm(val, "ld.h", ptr); break; \
+ case 4: __get_data_asm(val, "ld.w", ptr); break; \
+ case 8: __get_data_asm(val, "ld.d", ptr); break; \
+ default: BUILD_BUG(); break; \
+ } \
+} while (0)
+
+#define __get_kernel_common(val, size, ptr) __get_user_common(val, size, ptr)
+
+#define __get_data_asm(val, insn, ptr) \
+{ \
+ long __gu_tmp; \
+ \
+ __asm__ __volatile__( \
+ "1: " insn " %1, %2 \n" \
+ "2: \n" \
+ " .section .fixup,\"ax\" \n" \
+ "3: li.w %0, %3 \n" \
+ " or %1, $r0, $r0 \n" \
+ " b 2b \n" \
+ " .previous \n" \
+ " .section __ex_table,\"a\" \n" \
+ " "__UA_ADDR "\t1b, 3b \n" \
+ " .previous \n" \
+ : "+r" (__gu_err), "=r" (__gu_tmp) \
+ : "m" (__m(ptr)), "i" (-EFAULT)); \
+ \
+ (val) = (__typeof__(*(ptr))) __gu_tmp; \
+}
+
+#define __put_user_common(ptr, size) \
+do { \
+ switch (size) { \
+ case 1: __put_data_asm("st.b", ptr); break; \
+ case 2: __put_data_asm("st.h", ptr); break; \
+ case 4: __put_data_asm("st.w", ptr); break; \
+ case 8: __put_data_asm("st.d", ptr); break; \
+ default: BUILD_BUG(); break; \
+ } \
+} while (0)
+
+#define __put_kernel_common(ptr, size) __put_user_common(ptr, size)
+
+#define __put_data_asm(insn, ptr) \
+{ \
+ __asm__ __volatile__( \
+ "1: " insn " %z2, %1 # __put_user_asm\n" \
+ "2: \n" \
+ " .section .fixup,\"ax\" \n" \
+ "3: li.w %0, %3 \n" \
+ " b 2b \n" \
+ " .previous \n" \
+ " .section __ex_table,\"a\" \n" \
+ " " __UA_ADDR " 1b, 3b \n" \
+ " .previous \n" \
+ : "+r" (__pu_err), "=m" (__m(ptr)) \
+ : "Jr" (__pu_val), "i" (-EFAULT)); \
+}
+
+#define __get_kernel_nofault(dst, src, type, err_label) \
+do { \
+ int __gu_err = 0; \
+ \
+ __get_kernel_common(*((type *)(dst)), sizeof(type), \
+ (__force type *)(src)); \
+ if (unlikely(__gu_err)) \
+ goto err_label; \
+} while (0)
+
+#define __put_kernel_nofault(dst, src, type, err_label) \
+do { \
+ type __pu_val; \
+ int __pu_err = 0; \
+ \
+ __pu_val = *(__force type *)(src); \
+ __put_kernel_common(((type *)(dst)), sizeof(type)); \
+ if (unlikely(__pu_err)) \
+ goto err_label; \
+} while (0)
+
+extern unsigned long __copy_user(void *to, const void *from, __kernel_size_t n);
+
+static inline unsigned long __must_check
+raw_copy_from_user(void *to, const void __user *from, unsigned long n)
+{
+ return __copy_user(to, from, n);
+}
+
+static inline unsigned long __must_check
+raw_copy_to_user(void __user *to, const void *from, unsigned long n)
+{
+ return __copy_user(to, from, n);
+}
+
+#define INLINE_COPY_FROM_USER
+#define INLINE_COPY_TO_USER
+
+/*
+ * __clear_user: - Zero a block of memory in user space, with less checking.
+ * @addr: Destination address, in user space.
+ * @size: Number of bytes to zero.
+ *
+ * Zero a block of memory in user space. Caller must check
+ * the specified block with access_ok() before calling this function.
+ *
+ * Returns number of bytes that could not be cleared.
+ * On success, this will be zero.
+ */
+extern unsigned long __clear_user(void __user *addr, __kernel_size_t size);
+
+#define clear_user(addr, n) \
+({ \
+ void __user *__cl_addr = (addr); \
+ unsigned long __cl_size = (n); \
+ if (__cl_size && access_ok(__cl_addr, __cl_size)) \
+ __cl_size = __clear_user(__cl_addr, __cl_size); \
+ __cl_size; \
+})
+
+extern long strncpy_from_user(char *to, const char __user *from, long n);
+extern long strnlen_user(const char __user *str, long n);
+
+#endif /* _ASM_UACCESS_H */
diff --git a/arch/loongarch/include/asm/unistd.h b/arch/loongarch/include/asm/unistd.h
new file mode 100644
index 000000000000..cfddb0116a8c
--- /dev/null
+++ b/arch/loongarch/include/asm/unistd.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Author: Hanlu Li <[email protected]>
+ * Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <uapi/asm/unistd.h>
+
+#define NR_syscalls (__NR_syscalls)
diff --git a/arch/loongarch/include/uapi/asm/unistd.h b/arch/loongarch/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..fcb668984f03
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/unistd.h
@@ -0,0 +1,5 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#define __ARCH_WANT_SYS_CLONE
+#define __ARCH_WANT_SYS_CLONE3
+
+#include <asm-generic/unistd.h>
diff --git a/arch/loongarch/kernel/entry.S b/arch/loongarch/kernel/entry.S
new file mode 100644
index 000000000000..d5b3dbcf5425
--- /dev/null
+++ b/arch/loongarch/kernel/entry.S
@@ -0,0 +1,89 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1994 - 2000, 2001, 2003 Ralf Baechle
+ * Copyright (C) 1999, 2000 Silicon Graphics, Inc.
+ * Copyright (C) 2001 MIPS Technologies, Inc.
+ */
+
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/loongarch.h>
+#include <asm/regdef.h>
+#include <asm/stackframe.h>
+#include <asm/thread_info.h>
+
+ .text
+ .cfi_sections .debug_frame
+ .align 5
+SYM_FUNC_START(handle_syscall)
+ csrrd t0, PERCPU_BASE_KS
+ la.abs t1, kernelsp
+ add.d t1, t1, t0
+ move t2, sp
+ ld.d sp, t1, 0
+
+ addi.d sp, sp, -PT_SIZE
+ cfi_st t2, PT_R3
+ cfi_rel_offset sp, PT_R3
+ st.d zero, sp, PT_R0
+ csrrd t2, LOONGARCH_CSR_PRMD
+ st.d t2, sp, PT_PRMD
+ csrrd t2, LOONGARCH_CSR_CRMD
+ st.d t2, sp, PT_CRMD
+ csrrd t2, LOONGARCH_CSR_EUEN
+ st.d t2, sp, PT_EUEN
+ csrrd t2, LOONGARCH_CSR_ECFG
+ st.d t2, sp, PT_ECFG
+ csrrd t2, LOONGARCH_CSR_ESTAT
+ st.d t2, sp, PT_ESTAT
+ cfi_st ra, PT_R1
+ cfi_st a0, PT_R4
+ cfi_st a1, PT_R5
+ cfi_st a2, PT_R6
+ cfi_st a3, PT_R7
+ cfi_st a4, PT_R8
+ cfi_st a5, PT_R9
+ cfi_st a6, PT_R10
+ cfi_st a7, PT_R11
+ csrrd ra, LOONGARCH_CSR_ERA
+ st.d ra, sp, PT_ERA
+ cfi_rel_offset ra, PT_ERA
+
+ cfi_st tp, PT_R2
+ cfi_st u0, PT_R21
+ cfi_st fp, PT_R22
+
+ SAVE_STATIC
+
+ move u0, t0
+ li.d tp, ~_THREAD_MASK
+ and tp, tp, sp
+
+ move a0, sp
+ bl do_syscall
+
+ RESTORE_ALL_AND_RET
+SYM_FUNC_END(handle_syscall)
+
+SYM_CODE_START(ret_from_fork)
+ bl schedule_tail # a0 = struct task_struct *prev
+ move a0, sp
+ bl syscall_exit_to_user_mode
+ RESTORE_STATIC
+ RESTORE_SOME
+ RESTORE_SP_AND_RET
+SYM_CODE_END(ret_from_fork)
+
+SYM_CODE_START(ret_from_kernel_thread)
+ bl schedule_tail # a0 = struct task_struct *prev
+ move a0, s1
+ jirl ra, s0, 0
+ move a0, sp
+ bl syscall_exit_to_user_mode
+ RESTORE_STATIC
+ RESTORE_SOME
+ RESTORE_SP_AND_RET
+SYM_CODE_END(ret_from_kernel_thread)
diff --git a/arch/loongarch/kernel/syscall.c b/arch/loongarch/kernel/syscall.c
new file mode 100644
index 000000000000..3fc4211db989
--- /dev/null
+++ b/arch/loongarch/kernel/syscall.c
@@ -0,0 +1,63 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Author: Hanlu Li <[email protected]>
+ * Huacai Chen <[email protected]>
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/capability.h>
+#include <linux/entry-common.h>
+#include <linux/errno.h>
+#include <linux/linkage.h>
+#include <linux/syscalls.h>
+#include <linux/unistd.h>
+
+#include <asm/asm.h>
+#include <asm/signal.h>
+#include <asm/switch_to.h>
+#include <asm-generic/syscalls.h>
+
+#undef __SYSCALL
+#define __SYSCALL(nr, call) [nr] = (call),
+
+SYSCALL_DEFINE6(mmap, unsigned long, addr, unsigned long, len, unsigned long,
+ prot, unsigned long, flags, unsigned long, fd, off_t, offset)
+{
+ if (offset & ~PAGE_MASK)
+ return -EINVAL;
+
+ return ksys_mmap_pgoff(addr, len, prot, flags, fd, offset >> PAGE_SHIFT);
+}
+
+void *sys_call_table[__NR_syscalls] = {
+ [0 ... __NR_syscalls - 1] = sys_ni_syscall,
+#include <asm/unistd.h>
+};
+
+typedef long (*sys_call_fn)(unsigned long, unsigned long,
+ unsigned long, unsigned long, unsigned long, unsigned long);
+
+void noinstr do_syscall(struct pt_regs *regs)
+{
+ unsigned long nr;
+ sys_call_fn syscall_fn;
+
+ nr = regs->regs[11];
+ /* Set for syscall restarting */
+ if (nr < NR_syscalls)
+ regs->regs[0] = nr + 1;
+
+ regs->csr_era += 4;
+ regs->orig_a0 = regs->regs[4];
+ regs->regs[4] = -ENOSYS;
+
+ nr = syscall_enter_from_user_mode(regs, nr);
+
+ if (nr < NR_syscalls) {
+ syscall_fn = sys_call_table[nr];
+ regs->regs[4] = syscall_fn(regs->orig_a0, regs->regs[5], regs->regs[6],
+ regs->regs[7], regs->regs[8], regs->regs[9]);
+ }
+
+ syscall_exit_to_user_mode(regs);
+}
--
2.27.0


2022-05-18 13:43:40

by Arnd Bergmann

[permalink] [raw]
Subject: Re: [PATCH V11 00/22] arch: Add basic LoongArch support

On Wed, May 18, 2022 at 10:25 AM Huacai Chen <[email protected]> wrote:
>
> LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V.
> LoongArch includes a reduced 32-bit version (LA32R), a standard 32-bit
> version (LA32S) and a 64-bit version (LA64). LoongArch use ACPI as its
> boot protocol LoongArch-specific interrupt controllers (similar to APIC)
> are already added in the next revision of ACPI Specification (current
> revision is 6.4).
>
> This patchset is adding basic LoongArch support in mainline kernel, we
> can see a complete snapshot here:
> https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git/log/?h=loongarch-next

Stephen, can you add this branch to the linux-next tree?

I see there are still comments coming in, but at some point this has
to just be considered good enough that any further changes can be addressed
with patches on top rather than rebasing.

> V10 -> V11:
> 1, Rebased on asm-generic tree;

I was expecting that you'd base this on just the spinlock changes from Palmer's
tree that are part of the asm-generic tree rather than all of what I have.

Can you rebase it once more? If there are conflicts against the h8300 removal
series that is also in asm-generic, leaving it on top of that may be
easier though.

Arnd

2022-05-18 15:25:25

by WANG Xuerui

[permalink] [raw]
Subject: Re: [PATCH V11 05/22] LoongArch: Add build infrastructure

On 5/18/22 17:26, Huacai Chen wrote:
> Add Kbuild, Makefile, Kconfig and link script for LoongArch build
> infrastructure.
>
> Signed-off-by: Huacai Chen <[email protected]>
> ---
> arch/loongarch/Kbuild | 6 +
> arch/loongarch/Kconfig | 395 +++++++++++++++++++++++++
> arch/loongarch/Kconfig.debug | 0
> arch/loongarch/Makefile | 105 +++++++
> arch/loongarch/boot/.gitignore | 2 +
> arch/loongarch/boot/Makefile | 20 ++
> arch/loongarch/boot/dts/Makefile | 4 +
> arch/loongarch/include/asm/Kbuild | 30 ++
> arch/loongarch/include/uapi/asm/Kbuild | 2 +
> arch/loongarch/kernel/.gitignore | 2 +
> arch/loongarch/kernel/Makefile | 21 ++
> arch/loongarch/kernel/vmlinux.lds.S | 117 ++++++++
> arch/loongarch/lib/Makefile | 6 +
> arch/loongarch/mm/Makefile | 9 +
> arch/loongarch/pci/Makefile | 7 +
> arch/loongarch/vdso/.gitignore | 2 +
> scripts/subarch.include | 2 +-
> 17 files changed, 729 insertions(+), 1 deletion(-)
> create mode 100644 arch/loongarch/Kbuild
> create mode 100644 arch/loongarch/Kconfig
> create mode 100644 arch/loongarch/Kconfig.debug
> create mode 100644 arch/loongarch/Makefile
> create mode 100644 arch/loongarch/boot/.gitignore
> create mode 100644 arch/loongarch/boot/Makefile
> create mode 100644 arch/loongarch/boot/dts/Makefile
> create mode 100644 arch/loongarch/include/asm/Kbuild
> create mode 100644 arch/loongarch/include/uapi/asm/Kbuild
> create mode 100644 arch/loongarch/kernel/.gitignore
> create mode 100644 arch/loongarch/kernel/Makefile
> create mode 100644 arch/loongarch/kernel/vmlinux.lds.S
> create mode 100644 arch/loongarch/lib/Makefile
> create mode 100644 arch/loongarch/mm/Makefile
> create mode 100644 arch/loongarch/pci/Makefile
> create mode 100644 arch/loongarch/vdso/.gitignore

Seems most (if not all) my comments from v10 are addressed; and there
seems to have no obvious mistake.

Reviewed-by: WANG Xuerui <[email protected]>


2022-05-18 15:36:45

by WANG Xuerui

[permalink] [raw]
Subject: Re: [PATCH V11 01/22] Documentation: LoongArch: Add basic documentations

On 5/18/22 17:25, Huacai Chen wrote:
> Add some basic documentation for LoongArch. LoongArch is a new RISC ISA,
> which is a bit like MIPS or RISC-V. LoongArch includes a reduced 32-bit
> version (LA32R), a standard 32-bit version (LA32S) and a 64-bit version
> (LA64).
>
> Signed-off-by: Huacai Chen <[email protected]>
> ---
> Documentation/arch.rst | 1 +
> Documentation/loongarch/features.rst | 3 +
> Documentation/loongarch/index.rst | 21 ++
> Documentation/loongarch/introduction.rst | 387 +++++++++++++++++++++
> Documentation/loongarch/irq-chip-model.rst | 168 +++++++++
> 5 files changed, 580 insertions(+)
> create mode 100644 Documentation/loongarch/features.rst
> create mode 100644 Documentation/loongarch/index.rst
> create mode 100644 Documentation/loongarch/introduction.rst
> create mode 100644 Documentation/loongarch/irq-chip-model.rst

This feels much better now; I feel it's acceptable from a random
community member's perspective. Thanks for incorporating my comments and
tweaks!

Reviewed-by: WANG Xuerui <[email protected]>


2022-05-18 15:52:17

by WANG Xuerui

[permalink] [raw]
Subject: Re: [PATCH V11 06/22] LoongArch: Add CPU definition headers

On 5/18/22 17:26, Huacai Chen wrote:
> Add common headers (CPU definition and address space layout) for basic
> LoongArch support.
>
> Signed-off-by: Huacai Chen <[email protected]>
> ---
> arch/loongarch/include/asm/addrspace.h | 112 ++
> arch/loongarch/include/asm/cpu-features.h | 73 +
> arch/loongarch/include/asm/cpu-info.h | 116 ++
> arch/loongarch/include/asm/cpu.h | 127 ++
> arch/loongarch/include/asm/fpregdef.h | 53 +
> arch/loongarch/include/asm/loongarch.h | 1519 +++++++++++++++++++++
> arch/loongarch/include/asm/loongson.h | 153 +++
> arch/loongarch/include/asm/regdef.h | 41 +
> 8 files changed, 2194 insertions(+)
> create mode 100644 arch/loongarch/include/asm/addrspace.h
> create mode 100644 arch/loongarch/include/asm/cpu-features.h
> create mode 100644 arch/loongarch/include/asm/cpu-info.h
> create mode 100644 arch/loongarch/include/asm/cpu.h
> create mode 100644 arch/loongarch/include/asm/fpregdef.h
> create mode 100644 arch/loongarch/include/asm/loongarch.h
> create mode 100644 arch/loongarch/include/asm/loongson.h
> create mode 100644 arch/loongarch/include/asm/regdef.h
>
> [snip]
>
> diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h
> new file mode 100644
> index 000000000000..2be0efe93875
> --- /dev/null
> +++ b/arch/loongarch/include/asm/loongarch.h
> @@ -0,0 +1,1519 @@
>
> [snip]
>
> +
> +/* FPU register names */
> +#define LOONGARCH_FCSR0 $r0
> +#define LOONGARCH_FCSR1 $r1
> +#define LOONGARCH_FCSR2 $r2
> +#define LOONGARCH_FCSR3 $r3
Why not reuse the definitions in <asm/fpregdef.h>?
> +
> +/* FPU Status Register Values */
> +#define FPU_CSR_RSVD 0xe0e0fce0
> +
> +/*
> + * X the exception cause indicator
> + * E the exception enable
> + * S the sticky/flag bit
> + */
> +#define FPU_CSR_ALL_X 0x1f000000
> +#define FPU_CSR_INV_X 0x10000000
> +#define FPU_CSR_DIV_X 0x08000000
> +#define FPU_CSR_OVF_X 0x04000000
> +#define FPU_CSR_UDF_X 0x02000000
> +#define FPU_CSR_INE_X 0x01000000
> +
> +#define FPU_CSR_ALL_S 0x001f0000
> +#define FPU_CSR_INV_S 0x00100000
> +#define FPU_CSR_DIV_S 0x00080000
> +#define FPU_CSR_OVF_S 0x00040000
> +#define FPU_CSR_UDF_S 0x00020000
> +#define FPU_CSR_INE_S 0x00010000
> +
> +#define FPU_CSR_ALL_E 0x0000001f
> +#define FPU_CSR_INV_E 0x00000010
> +#define FPU_CSR_DIV_E 0x00000008
> +#define FPU_CSR_OVF_E 0x00000004
> +#define FPU_CSR_UDF_E 0x00000002
> +#define FPU_CSR_INE_E 0x00000001
> +
> +/* Bits 8 and 9 of FPU Status Register specify the rounding mode */
> +#define FPU_CSR_RM 0x300
> +#define FPU_CSR_RN 0x000 /* nearest */
> +#define FPU_CSR_RZ 0x100 /* towards zero */
> +#define FPU_CSR_RU 0x200 /* towards +Infinity */
> +#define FPU_CSR_RD 0x300 /* towards -Infinity */
> +
> +#define read_fcsr(source) \
> +({ \
> + unsigned int __res; \
> +\
> + __asm__ __volatile__( \
> + " movfcsr2gr %0, "__stringify(source)" \n" \
> + : "=r" (__res)); \
> + __res; \
> +})
> +
> +#define write_fcsr(dest, val) \
> +do { \
> + __asm__ __volatile__( \
> + " movgr2fcsr %0, "__stringify(dest)" \n" \
> + : : "r" (val)); \
> +} while (0)
> +
> +#endif /* _ASM_LOONGARCH_H */
> diff --git a/arch/loongarch/include/asm/loongson.h b/arch/loongarch/include/asm/loongson.h
> new file mode 100644
> index 000000000000..db4992ae6a6c
> --- /dev/null
> +++ b/arch/loongarch/include/asm/loongson.h
> @@ -0,0 +1,153 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Author: Huacai Chen <[email protected]>
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +
> +#ifndef __ASM_LOONGSON_H
> +#define __ASM_LOONGSON_H
> +
> +#include <linux/init.h>
> +#include <linux/io.h>
> +#include <linux/irq.h>
> +#include <linux/pci.h>
> +#include <asm/addrspace.h>
> +#include <asm/bootinfo.h>
> +
> +extern const struct plat_smp_ops loongson3_smp_ops;
> +
> +#define LOONGSON_REG(x) \
> + (*(volatile u32 *)((char *)TO_UNCACHE(LOONGSON_REG_BASE) + (x)))
> +
> +#define LOONGSON_LIO_BASE 0x18000000
> +#define LOONGSON_LIO_SIZE 0x00100000 /* 1M */
> +#define LOONGSON_LIO_TOP (LOONGSON_LIO_BASE+LOONGSON_LIO_SIZE-1)
> +
> +#define LOONGSON_BOOT_BASE 0x1c000000
> +#define LOONGSON_BOOT_SIZE 0x02000000 /* 32M */
> +#define LOONGSON_BOOT_TOP (LOONGSON_BOOT_BASE+LOONGSON_BOOT_SIZE-1)
> +
> +#define LOONGSON_REG_BASE 0x1fe00000
> +#define LOONGSON_REG_SIZE 0x00100000 /* 1M */
> +#define LOONGSON_REG_TOP (LOONGSON_REG_BASE+LOONGSON_REG_SIZE-1)
> +
> +/* GPIO Regs - r/w */
> +
> +#define LOONGSON_GPIODATA LOONGSON_REG(0x11c)
> +#define LOONGSON_GPIOIE LOONGSON_REG(0x120)
> +#define LOONGSON_REG_GPIO_BASE (LOONGSON_REG_BASE + 0x11c)
> +
> +#define MAX_PACKAGES 16
> +
> +/* Chip Config registor of each physical cpu package */
"register" -- same below
> +extern u64 loongson_chipcfg[MAX_PACKAGES];
> +#define LOONGSON_CHIPCFG(id) (*(volatile u32 *)(loongson_chipcfg[id]))
> +
> +/* Chip Temperature registor of each physical cpu package */
> +extern u64 loongson_chiptemp[MAX_PACKAGES];
> +#define LOONGSON_CHIPTEMP(id) (*(volatile u32 *)(loongson_chiptemp[id]))
> +
> +/* Freq Control register of each physical cpu package */
> +extern u64 loongson_freqctrl[MAX_PACKAGES];
> +#define LOONGSON_FREQCTRL(id) (*(volatile u32 *)(loongson_freqctrl[id]))
> +
> +#define xconf_readl(addr) readl(addr)
> +#define xconf_readq(addr) readq(addr)
> +
> +static inline void xconf_writel(u32 val, volatile void __iomem *addr)
> +{
> + asm volatile (
> + " st.w %[v], %[hw], 0 \n"
> + " ld.b $r0, %[hw], 0 \n"
> + :
> + : [hw] "r" (addr), [v] "r" (val)
> + );
> +}
> +
> +static inline void xconf_writeq(u64 val64, volatile void __iomem *addr)
> +{
> + asm volatile (
> + " st.d %[v], %[hw], 0 \n"
> + " ld.b $r0, %[hw], 0 \n"
> + :
> + : [hw] "r" (addr), [v] "r" (val64)
> + );
> +}
> +
> +/* ============== LS7A registers =============== */
> +#define LS7A_PCH_REG_BASE 0x10000000UL
> +/* LPC regs */
> +#define LS7A_LPC_REG_BASE (LS7A_PCH_REG_BASE + 0x00002000)
> +/* CHIPCFG regs */
> +#define LS7A_CHIPCFG_REG_BASE (LS7A_PCH_REG_BASE + 0x00010000)
> +/* MISC reg base */
> +#define LS7A_MISC_REG_BASE (LS7A_PCH_REG_BASE + 0x00080000)
> +/* ACPI regs */
> +#define LS7A_ACPI_REG_BASE (LS7A_MISC_REG_BASE + 0x00050000)
> +/* RTC regs */
> +#define LS7A_RTC_REG_BASE (LS7A_MISC_REG_BASE + 0x00050100)
> +
> +#define LS7A_DMA_CFG (volatile void *)TO_UNCACHE(LS7A_CHIPCFG_REG_BASE + 0x041c)
> +#define LS7A_DMA_NODE_SHF 8
> +#define LS7A_DMA_NODE_MASK 0x1F00
> +
> +#define LS7A_INT_MASK_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x020)
> +#define LS7A_INT_EDGE_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x060)
> +#define LS7A_INT_CLEAR_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x080)
> +#define LS7A_INT_HTMSI_EN_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x040)
> +#define LS7A_INT_ROUTE_ENTRY_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x100)
> +#define LS7A_INT_HTMSI_VEC_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x200)
> +#define LS7A_INT_STATUS_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x3a0)
> +#define LS7A_INT_POL_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x3e0)
> +#define LS7A_LPC_INT_CTL (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2000)
> +#define LS7A_LPC_INT_ENA (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2004)
> +#define LS7A_LPC_INT_STS (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2008)
> +#define LS7A_LPC_INT_CLR (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x200c)
> +#define LS7A_LPC_INT_POL (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2010)
> +
> +#define LS7A_PMCON_SOC_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x000)
> +#define LS7A_PMCON_RESUME_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x004)
> +#define LS7A_PMCON_RTC_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x008)
> +#define LS7A_PM1_EVT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x00c)
> +#define LS7A_PM1_ENA_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x010)
> +#define LS7A_PM1_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x014)
> +#define LS7A_PM1_TMR_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x018)
> +#define LS7A_P_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x01c)
> +#define LS7A_GPE0_STS_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x028)
> +#define LS7A_GPE0_ENA_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x02c)
> +#define LS7A_RST_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x030)
> +#define LS7A_WD_SET_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x034)
> +#define LS7A_WD_TIMER_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x038)
> +#define LS7A_THSENS_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x04c)
> +#define LS7A_GEN_RTC_1_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x050)
> +#define LS7A_GEN_RTC_2_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x054)
> +#define LS7A_DPM_CFG_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x400)
> +#define LS7A_DPM_STS_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x404)
> +#define LS7A_DPM_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x408)
> +
> +typedef enum {
> + ACPI_PCI_HOTPLUG_STATUS = 1 << 1,
> + ACPI_CPU_HOTPLUG_STATUS = 1 << 2,
> + ACPI_MEM_HOTPLUG_STATUS = 1 << 3,
> + ACPI_POWERBUTTON_STATUS = 1 << 8,
> + ACPI_RTC_WAKE_STATUS = 1 << 10,
> + ACPI_PCI_WAKE_STATUS = 1 << 14,
> + ACPI_ANY_WAKE_STATUS = 1 << 15,
> +} AcpiEventStatusBits;
> +
> +#define HT1LO_OFFSET 0xe0000000000UL
> +
> +/* PCI Configuration Space Base */
> +#define MCFG_EXT_PCICFG_BASE 0xefe00000000UL
> +
> +/* REG ACCESS*/
> +#define ls7a_readb(addr) (*(volatile unsigned char *)TO_UNCACHE(addr))
> +#define ls7a_readw(addr) (*(volatile unsigned short *)TO_UNCACHE(addr))
> +#define ls7a_readl(addr) (*(volatile unsigned int *)TO_UNCACHE(addr))
> +#define ls7a_readq(addr) (*(volatile unsigned long *)TO_UNCACHE(addr))
> +#define ls7a_writeb(val, addr) *(volatile unsigned char *)TO_UNCACHE(addr) = (val)
> +#define ls7a_writew(val, addr) *(volatile unsigned short *)TO_UNCACHE(addr) = (val)
> +#define ls7a_writel(val, addr) *(volatile unsigned int *)TO_UNCACHE(addr) = (val)
> +#define ls7a_writeq(val, addr) *(volatile unsigned long *)TO_UNCACHE(addr) = (val)
> +
> +#endif /* __ASM_LOONGSON_H */
> diff --git a/arch/loongarch/include/asm/regdef.h b/arch/loongarch/include/asm/regdef.h
> new file mode 100644
> index 000000000000..49a374c2612c
> --- /dev/null
> +++ b/arch/loongarch/include/asm/regdef.h
> @@ -0,0 +1,41 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef _ASM_REGDEF_H
> +#define _ASM_REGDEF_H
> +
> +#define zero $r0 /* wired zero */
> +#define ra $r1 /* return address */
> +#define tp $r2
> +#define sp $r3 /* stack pointer */
> +#define a0 $r4 /* argument registers, a0/a1 reused as v0/v1 for return value */
> +#define a1 $r5
> +#define a2 $r6
> +#define a3 $r7
> +#define a4 $r8
> +#define a5 $r9
> +#define a6 $r10
> +#define a7 $r11
> +#define t0 $r12 /* caller saved */
> +#define t1 $r13
> +#define t2 $r14
> +#define t3 $r15
> +#define t4 $r16
> +#define t5 $r17
> +#define t6 $r18
> +#define t7 $r19
> +#define t8 $r20
> +#define u0 $r21
> +#define fp $r22 /* frame pointer */
> +#define s0 $r23 /* callee saved */
> +#define s1 $r24
> +#define s2 $r25
> +#define s3 $r26
> +#define s4 $r27
> +#define s5 $r28
> +#define s6 $r29
> +#define s7 $r30
> +#define s8 $r31
> +
> +#endif /* _ASM_REGDEF_H */

Apart from the minor nits:

Reviewed-by: WANG Xuerui <[email protected]>


2022-05-18 15:55:43

by WANG Xuerui

[permalink] [raw]
Subject: Re: [PATCH V11 07/22] LoongArch: Add atomic/locking headers

On 5/18/22 17:26, Huacai Chen wrote:
> Add common headers (atomic, bitops, barrier and locking) for basic
> LoongArch support.
>
> Signed-off-by: Huacai Chen <[email protected]>
> ---
> arch/loongarch/include/asm/atomic.h | 358 +++++++++++++++++++++++++++
> arch/loongarch/include/asm/barrier.h | 51 ++++
> arch/loongarch/include/asm/bitops.h | 33 +++
> arch/loongarch/include/asm/bitrev.h | 34 +++
> arch/loongarch/include/asm/cmpxchg.h | 121 +++++++++
> arch/loongarch/include/asm/local.h | 138 +++++++++++
> arch/loongarch/include/asm/percpu.h | 20 ++
> 7 files changed, 755 insertions(+)
> create mode 100644 arch/loongarch/include/asm/atomic.h
> create mode 100644 arch/loongarch/include/asm/barrier.h
> create mode 100644 arch/loongarch/include/asm/bitops.h
> create mode 100644 arch/loongarch/include/asm/bitrev.h
> create mode 100644 arch/loongarch/include/asm/cmpxchg.h
> create mode 100644 arch/loongarch/include/asm/local.h
> create mode 100644 arch/loongarch/include/asm/percpu.h

No major problems here so:

Reviewed-by: WANG Xuerui <[email protected]>


2022-05-18 16:16:33

by Huacai Chen

[permalink] [raw]
Subject: Re: [PATCH V11 06/22] LoongArch: Add CPU definition headers

Hi, Xuerui,

On Wed, May 18, 2022 at 11:45 PM WANG Xuerui <[email protected]> wrote:
>
> On 5/18/22 17:26, Huacai Chen wrote:
> > Add common headers (CPU definition and address space layout) for basic
> > LoongArch support.
> >
> > Signed-off-by: Huacai Chen <[email protected]>
> > ---
> > arch/loongarch/include/asm/addrspace.h | 112 ++
> > arch/loongarch/include/asm/cpu-features.h | 73 +
> > arch/loongarch/include/asm/cpu-info.h | 116 ++
> > arch/loongarch/include/asm/cpu.h | 127 ++
> > arch/loongarch/include/asm/fpregdef.h | 53 +
> > arch/loongarch/include/asm/loongarch.h | 1519 +++++++++++++++++++++
> > arch/loongarch/include/asm/loongson.h | 153 +++
> > arch/loongarch/include/asm/regdef.h | 41 +
> > 8 files changed, 2194 insertions(+)
> > create mode 100644 arch/loongarch/include/asm/addrspace.h
> > create mode 100644 arch/loongarch/include/asm/cpu-features.h
> > create mode 100644 arch/loongarch/include/asm/cpu-info.h
> > create mode 100644 arch/loongarch/include/asm/cpu.h
> > create mode 100644 arch/loongarch/include/asm/fpregdef.h
> > create mode 100644 arch/loongarch/include/asm/loongarch.h
> > create mode 100644 arch/loongarch/include/asm/loongson.h
> > create mode 100644 arch/loongarch/include/asm/regdef.h
> >
> > [snip]
> >
> > diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h
> > new file mode 100644
> > index 000000000000..2be0efe93875
> > --- /dev/null
> > +++ b/arch/loongarch/include/asm/loongarch.h
> > @@ -0,0 +1,1519 @@
> >
> > [snip]
> >
> > +
> > +/* FPU register names */
> > +#define LOONGARCH_FCSR0 $r0
> > +#define LOONGARCH_FCSR1 $r1
> > +#define LOONGARCH_FCSR2 $r2
> > +#define LOONGARCH_FCSR3 $r3
> Why not reuse the definitions in <asm/fpregdef.h>?
> > +
> > +/* FPU Status Register Values */
> > +#define FPU_CSR_RSVD 0xe0e0fce0
> > +
> > +/*
> > + * X the exception cause indicator
> > + * E the exception enable
> > + * S the sticky/flag bit
> > + */
> > +#define FPU_CSR_ALL_X 0x1f000000
> > +#define FPU_CSR_INV_X 0x10000000
> > +#define FPU_CSR_DIV_X 0x08000000
> > +#define FPU_CSR_OVF_X 0x04000000
> > +#define FPU_CSR_UDF_X 0x02000000
> > +#define FPU_CSR_INE_X 0x01000000
> > +
> > +#define FPU_CSR_ALL_S 0x001f0000
> > +#define FPU_CSR_INV_S 0x00100000
> > +#define FPU_CSR_DIV_S 0x00080000
> > +#define FPU_CSR_OVF_S 0x00040000
> > +#define FPU_CSR_UDF_S 0x00020000
> > +#define FPU_CSR_INE_S 0x00010000
> > +
> > +#define FPU_CSR_ALL_E 0x0000001f
> > +#define FPU_CSR_INV_E 0x00000010
> > +#define FPU_CSR_DIV_E 0x00000008
> > +#define FPU_CSR_OVF_E 0x00000004
> > +#define FPU_CSR_UDF_E 0x00000002
> > +#define FPU_CSR_INE_E 0x00000001
> > +
> > +/* Bits 8 and 9 of FPU Status Register specify the rounding mode */
> > +#define FPU_CSR_RM 0x300
> > +#define FPU_CSR_RN 0x000 /* nearest */
> > +#define FPU_CSR_RZ 0x100 /* towards zero */
> > +#define FPU_CSR_RU 0x200 /* towards +Infinity */
> > +#define FPU_CSR_RD 0x300 /* towards -Infinity */
> > +
> > +#define read_fcsr(source) \
> > +({ \
> > + unsigned int __res; \
> > +\
> > + __asm__ __volatile__( \
> > + " movfcsr2gr %0, "__stringify(source)" \n" \
> > + : "=r" (__res)); \
> > + __res; \
> > +})
> > +
> > +#define write_fcsr(dest, val) \
> > +do { \
> > + __asm__ __volatile__( \
> > + " movgr2fcsr %0, "__stringify(dest)" \n" \
> > + : : "r" (val)); \
> > +} while (0)
> > +
> > +#endif /* _ASM_LOONGARCH_H */
> > diff --git a/arch/loongarch/include/asm/loongson.h b/arch/loongarch/include/asm/loongson.h
> > new file mode 100644
> > index 000000000000..db4992ae6a6c
> > --- /dev/null
> > +++ b/arch/loongarch/include/asm/loongson.h
> > @@ -0,0 +1,153 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Author: Huacai Chen <[email protected]>
> > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > + */
> > +
> > +#ifndef __ASM_LOONGSON_H
> > +#define __ASM_LOONGSON_H
> > +
> > +#include <linux/init.h>
> > +#include <linux/io.h>
> > +#include <linux/irq.h>
> > +#include <linux/pci.h>
> > +#include <asm/addrspace.h>
> > +#include <asm/bootinfo.h>
> > +
> > +extern const struct plat_smp_ops loongson3_smp_ops;
> > +
> > +#define LOONGSON_REG(x) \
> > + (*(volatile u32 *)((char *)TO_UNCACHE(LOONGSON_REG_BASE) + (x)))
> > +
> > +#define LOONGSON_LIO_BASE 0x18000000
> > +#define LOONGSON_LIO_SIZE 0x00100000 /* 1M */
> > +#define LOONGSON_LIO_TOP (LOONGSON_LIO_BASE+LOONGSON_LIO_SIZE-1)
> > +
> > +#define LOONGSON_BOOT_BASE 0x1c000000
> > +#define LOONGSON_BOOT_SIZE 0x02000000 /* 32M */
> > +#define LOONGSON_BOOT_TOP (LOONGSON_BOOT_BASE+LOONGSON_BOOT_SIZE-1)
> > +
> > +#define LOONGSON_REG_BASE 0x1fe00000
> > +#define LOONGSON_REG_SIZE 0x00100000 /* 1M */
> > +#define LOONGSON_REG_TOP (LOONGSON_REG_BASE+LOONGSON_REG_SIZE-1)
> > +
> > +/* GPIO Regs - r/w */
> > +
> > +#define LOONGSON_GPIODATA LOONGSON_REG(0x11c)
> > +#define LOONGSON_GPIOIE LOONGSON_REG(0x120)
> > +#define LOONGSON_REG_GPIO_BASE (LOONGSON_REG_BASE + 0x11c)
> > +
> > +#define MAX_PACKAGES 16
> > +
> > +/* Chip Config registor of each physical cpu package */
> "register" -- same below
> > +extern u64 loongson_chipcfg[MAX_PACKAGES];
> > +#define LOONGSON_CHIPCFG(id) (*(volatile u32 *)(loongson_chipcfg[id]))
> > +
> > +/* Chip Temperature registor of each physical cpu package */
> > +extern u64 loongson_chiptemp[MAX_PACKAGES];
> > +#define LOONGSON_CHIPTEMP(id) (*(volatile u32 *)(loongson_chiptemp[id]))
> > +
> > +/* Freq Control register of each physical cpu package */
> > +extern u64 loongson_freqctrl[MAX_PACKAGES];
> > +#define LOONGSON_FREQCTRL(id) (*(volatile u32 *)(loongson_freqctrl[id]))
> > +
> > +#define xconf_readl(addr) readl(addr)
> > +#define xconf_readq(addr) readq(addr)
> > +
> > +static inline void xconf_writel(u32 val, volatile void __iomem *addr)
> > +{
> > + asm volatile (
> > + " st.w %[v], %[hw], 0 \n"
> > + " ld.b $r0, %[hw], 0 \n"
> > + :
> > + : [hw] "r" (addr), [v] "r" (val)
> > + );
> > +}
> > +
> > +static inline void xconf_writeq(u64 val64, volatile void __iomem *addr)
> > +{
> > + asm volatile (
> > + " st.d %[v], %[hw], 0 \n"
> > + " ld.b $r0, %[hw], 0 \n"
> > + :
> > + : [hw] "r" (addr), [v] "r" (val64)
> > + );
> > +}
> > +
> > +/* ============== LS7A registers =============== */
> > +#define LS7A_PCH_REG_BASE 0x10000000UL
> > +/* LPC regs */
> > +#define LS7A_LPC_REG_BASE (LS7A_PCH_REG_BASE + 0x00002000)
> > +/* CHIPCFG regs */
> > +#define LS7A_CHIPCFG_REG_BASE (LS7A_PCH_REG_BASE + 0x00010000)
> > +/* MISC reg base */
> > +#define LS7A_MISC_REG_BASE (LS7A_PCH_REG_BASE + 0x00080000)
> > +/* ACPI regs */
> > +#define LS7A_ACPI_REG_BASE (LS7A_MISC_REG_BASE + 0x00050000)
> > +/* RTC regs */
> > +#define LS7A_RTC_REG_BASE (LS7A_MISC_REG_BASE + 0x00050100)
> > +
> > +#define LS7A_DMA_CFG (volatile void *)TO_UNCACHE(LS7A_CHIPCFG_REG_BASE + 0x041c)
> > +#define LS7A_DMA_NODE_SHF 8
> > +#define LS7A_DMA_NODE_MASK 0x1F00
> > +
> > +#define LS7A_INT_MASK_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x020)
> > +#define LS7A_INT_EDGE_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x060)
> > +#define LS7A_INT_CLEAR_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x080)
> > +#define LS7A_INT_HTMSI_EN_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x040)
> > +#define LS7A_INT_ROUTE_ENTRY_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x100)
> > +#define LS7A_INT_HTMSI_VEC_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x200)
> > +#define LS7A_INT_STATUS_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x3a0)
> > +#define LS7A_INT_POL_REG (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x3e0)
> > +#define LS7A_LPC_INT_CTL (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2000)
> > +#define LS7A_LPC_INT_ENA (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2004)
> > +#define LS7A_LPC_INT_STS (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2008)
> > +#define LS7A_LPC_INT_CLR (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x200c)
> > +#define LS7A_LPC_INT_POL (volatile void *)TO_UNCACHE(LS7A_PCH_REG_BASE + 0x2010)
> > +
> > +#define LS7A_PMCON_SOC_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x000)
> > +#define LS7A_PMCON_RESUME_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x004)
> > +#define LS7A_PMCON_RTC_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x008)
> > +#define LS7A_PM1_EVT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x00c)
> > +#define LS7A_PM1_ENA_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x010)
> > +#define LS7A_PM1_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x014)
> > +#define LS7A_PM1_TMR_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x018)
> > +#define LS7A_P_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x01c)
> > +#define LS7A_GPE0_STS_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x028)
> > +#define LS7A_GPE0_ENA_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x02c)
> > +#define LS7A_RST_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x030)
> > +#define LS7A_WD_SET_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x034)
> > +#define LS7A_WD_TIMER_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x038)
> > +#define LS7A_THSENS_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x04c)
> > +#define LS7A_GEN_RTC_1_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x050)
> > +#define LS7A_GEN_RTC_2_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x054)
> > +#define LS7A_DPM_CFG_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x400)
> > +#define LS7A_DPM_STS_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x404)
> > +#define LS7A_DPM_CNT_REG (volatile void *)TO_UNCACHE(LS7A_ACPI_REG_BASE + 0x408)
> > +
> > +typedef enum {
> > + ACPI_PCI_HOTPLUG_STATUS = 1 << 1,
> > + ACPI_CPU_HOTPLUG_STATUS = 1 << 2,
> > + ACPI_MEM_HOTPLUG_STATUS = 1 << 3,
> > + ACPI_POWERBUTTON_STATUS = 1 << 8,
> > + ACPI_RTC_WAKE_STATUS = 1 << 10,
> > + ACPI_PCI_WAKE_STATUS = 1 << 14,
> > + ACPI_ANY_WAKE_STATUS = 1 << 15,
> > +} AcpiEventStatusBits;
> > +
> > +#define HT1LO_OFFSET 0xe0000000000UL
> > +
> > +/* PCI Configuration Space Base */
> > +#define MCFG_EXT_PCICFG_BASE 0xefe00000000UL
> > +
> > +/* REG ACCESS*/
> > +#define ls7a_readb(addr) (*(volatile unsigned char *)TO_UNCACHE(addr))
> > +#define ls7a_readw(addr) (*(volatile unsigned short *)TO_UNCACHE(addr))
> > +#define ls7a_readl(addr) (*(volatile unsigned int *)TO_UNCACHE(addr))
> > +#define ls7a_readq(addr) (*(volatile unsigned long *)TO_UNCACHE(addr))
> > +#define ls7a_writeb(val, addr) *(volatile unsigned char *)TO_UNCACHE(addr) = (val)
> > +#define ls7a_writew(val, addr) *(volatile unsigned short *)TO_UNCACHE(addr) = (val)
> > +#define ls7a_writel(val, addr) *(volatile unsigned int *)TO_UNCACHE(addr) = (val)
> > +#define ls7a_writeq(val, addr) *(volatile unsigned long *)TO_UNCACHE(addr) = (val)
> > +
> > +#endif /* __ASM_LOONGSON_H */
> > diff --git a/arch/loongarch/include/asm/regdef.h b/arch/loongarch/include/asm/regdef.h
> > new file mode 100644
> > index 000000000000..49a374c2612c
> > --- /dev/null
> > +++ b/arch/loongarch/include/asm/regdef.h
> > @@ -0,0 +1,41 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > + */
> > +#ifndef _ASM_REGDEF_H
> > +#define _ASM_REGDEF_H
> > +
> > +#define zero $r0 /* wired zero */
> > +#define ra $r1 /* return address */
> > +#define tp $r2
> > +#define sp $r3 /* stack pointer */
> > +#define a0 $r4 /* argument registers, a0/a1 reused as v0/v1 for return value */
> > +#define a1 $r5
> > +#define a2 $r6
> > +#define a3 $r7
> > +#define a4 $r8
> > +#define a5 $r9
> > +#define a6 $r10
> > +#define a7 $r11
> > +#define t0 $r12 /* caller saved */
> > +#define t1 $r13
> > +#define t2 $r14
> > +#define t3 $r15
> > +#define t4 $r16
> > +#define t5 $r17
> > +#define t6 $r18
> > +#define t7 $r19
> > +#define t8 $r20
> > +#define u0 $r21
> > +#define fp $r22 /* frame pointer */
> > +#define s0 $r23 /* callee saved */
> > +#define s1 $r24
> > +#define s2 $r25
> > +#define s3 $r26
> > +#define s4 $r27
> > +#define s5 $r28
> > +#define s6 $r29
> > +#define s7 $r30
> > +#define s8 $r31
> > +
> > +#endif /* _ASM_REGDEF_H */
>
> Apart from the minor nits:
Thanks, typos will be fixed.

Huacai
>
> Reviewed-by: WANG Xuerui <[email protected]>
>

2022-05-18 16:16:37

by WANG Xuerui

[permalink] [raw]
Subject: Re: [PATCH V11 14/22] LoongArch: Add signal handling support

On 5/18/22 17:26, Huacai Chen wrote:
> Add ucontext/sigcontext definition and signal handling support for
> LoongArch.
>
> Cc: Eric Biederman <[email protected]>
> Cc: Al Viro <[email protected]>
> Signed-off-by: Huacai Chen <[email protected]>
> ---
> arch/loongarch/include/uapi/asm/sigcontext.h | 44 ++
> arch/loongarch/include/uapi/asm/signal.h | 13 +
> arch/loongarch/include/uapi/asm/ucontext.h | 35 ++
> arch/loongarch/kernel/signal.c | 566 +++++++++++++++++++
> 4 files changed, 658 insertions(+)
> create mode 100644 arch/loongarch/include/uapi/asm/sigcontext.h
> create mode 100644 arch/loongarch/include/uapi/asm/signal.h
> create mode 100644 arch/loongarch/include/uapi/asm/ucontext.h
> create mode 100644 arch/loongarch/kernel/signal.c
>
> diff --git a/arch/loongarch/include/uapi/asm/sigcontext.h b/arch/loongarch/include/uapi/asm/sigcontext.h
> new file mode 100644
> index 000000000000..be3d3c6ac83e
> --- /dev/null
> +++ b/arch/loongarch/include/uapi/asm/sigcontext.h
> @@ -0,0 +1,44 @@
> +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
> +/*
> + * Author: Hanlu Li <[email protected]>
> + * Huacai Chen <[email protected]>
> + *
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef _UAPI_ASM_SIGCONTEXT_H
> +#define _UAPI_ASM_SIGCONTEXT_H
> +
> +#include <linux/types.h>
> +#include <linux/posix_types.h>
> +
> +/* FP context was used */
> +#define SC_USED_FP (1 << 0)
> +/* Address error was due to memory load */
> +#define SC_ADDRERR_RD (1 << 30)
> +/* Address error was due to memory store */
> +#define SC_ADDRERR_WR (1 << 31)

Looks nice.

BTW: following some communication it appears that the SC_ADDRERR_*
constants are not tightly coupled to BUS_ADRERR after all, so the
spelling is "fixed".

I don't have anything more to comment, because the rest are either
reviewed or non-userspace-ABI that don't exactly require getting right
from the beginning. (Not to say they're flawed.)

Reviewed-by: WANG Xuerui <[email protected]>


2022-05-18 16:42:07

by Eric W. Biederman

[permalink] [raw]
Subject: Re: [PATCH V11 14/22] LoongArch: Add signal handling support

Huacai Chen <[email protected]> writes:

> Add ucontext/sigcontext definition and signal handling support for
> LoongArch.
>
> Cc: Eric Biederman <[email protected]>
> Cc: Al Viro <[email protected]>
> Signed-off-by: Huacai Chen <[email protected]>
> ---
> arch/loongarch/include/uapi/asm/sigcontext.h | 44 ++
> arch/loongarch/include/uapi/asm/signal.h | 13 +
> arch/loongarch/include/uapi/asm/ucontext.h | 35 ++
> arch/loongarch/kernel/signal.c | 566 +++++++++++++++++++
> 4 files changed, 658 insertions(+)
> create mode 100644 arch/loongarch/include/uapi/asm/sigcontext.h
> create mode 100644 arch/loongarch/include/uapi/asm/signal.h
> create mode 100644 arch/loongarch/include/uapi/asm/ucontext.h
> create mode 100644 arch/loongarch/kernel/signal.c
>
> diff --git a/arch/loongarch/include/uapi/asm/sigcontext.h b/arch/loongarch/include/uapi/asm/sigcontext.h
> new file mode 100644
> index 000000000000..be3d3c6ac83e
> --- /dev/null
> +++ b/arch/loongarch/include/uapi/asm/sigcontext.h
> @@ -0,0 +1,44 @@
> +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
> +/*
> + * Author: Hanlu Li <[email protected]>
> + * Huacai Chen <[email protected]>
> + *
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef _UAPI_ASM_SIGCONTEXT_H
> +#define _UAPI_ASM_SIGCONTEXT_H
> +
> +#include <linux/types.h>
> +#include <linux/posix_types.h>
> +
> +/* FP context was used */
> +#define SC_USED_FP (1 << 0)
> +/* Address error was due to memory load */
> +#define SC_ADDRERR_RD (1 << 30)
> +/* Address error was due to memory store */
> +#define SC_ADDRERR_WR (1 << 31)
> +
> +struct sigcontext {
> + __u64 sc_pc;
> + __u64 sc_regs[32];
> + __u32 sc_flags;
> + __u64 sc_extcontext[0] __attribute__((__aligned__(16)));
> +};
> +
> +#define CONTEXT_INFO_ALIGN 16
> +struct _ctxinfo {
> + __u32 magic;
> + __u32 size;
> + __u64 padding; /* padding to 16 bytes */
> +};

This is probably something I a missing but what is struct _ctxinfo and
why is it in a uapi header?

I don't see anything else in the uapi implementation using it.

Symbols that start with an underscore "_" are reserved and should not
be used in general, and especially not in uapi header files.

Eric

2022-05-18 17:00:53

by Huacai Chen

[permalink] [raw]
Subject: Re: [PATCH V11 14/22] LoongArch: Add signal handling support

Hi, Eric,

On Thu, May 19, 2022 at 12:40 AM Eric W. Biederman
<[email protected]> wrote:
>
> Huacai Chen <[email protected]> writes:
>
> > Add ucontext/sigcontext definition and signal handling support for
> > LoongArch.
> >
> > Cc: Eric Biederman <[email protected]>
> > Cc: Al Viro <[email protected]>
> > Signed-off-by: Huacai Chen <[email protected]>
> > ---
> > arch/loongarch/include/uapi/asm/sigcontext.h | 44 ++
> > arch/loongarch/include/uapi/asm/signal.h | 13 +
> > arch/loongarch/include/uapi/asm/ucontext.h | 35 ++
> > arch/loongarch/kernel/signal.c | 566 +++++++++++++++++++
> > 4 files changed, 658 insertions(+)
> > create mode 100644 arch/loongarch/include/uapi/asm/sigcontext.h
> > create mode 100644 arch/loongarch/include/uapi/asm/signal.h
> > create mode 100644 arch/loongarch/include/uapi/asm/ucontext.h
> > create mode 100644 arch/loongarch/kernel/signal.c
> >
> > diff --git a/arch/loongarch/include/uapi/asm/sigcontext.h b/arch/loongarch/include/uapi/asm/sigcontext.h
> > new file mode 100644
> > index 000000000000..be3d3c6ac83e
> > --- /dev/null
> > +++ b/arch/loongarch/include/uapi/asm/sigcontext.h
> > @@ -0,0 +1,44 @@
> > +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
> > +/*
> > + * Author: Hanlu Li <[email protected]>
> > + * Huacai Chen <[email protected]>
> > + *
> > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > + */
> > +#ifndef _UAPI_ASM_SIGCONTEXT_H
> > +#define _UAPI_ASM_SIGCONTEXT_H
> > +
> > +#include <linux/types.h>
> > +#include <linux/posix_types.h>
> > +
> > +/* FP context was used */
> > +#define SC_USED_FP (1 << 0)
> > +/* Address error was due to memory load */
> > +#define SC_ADDRERR_RD (1 << 30)
> > +/* Address error was due to memory store */
> > +#define SC_ADDRERR_WR (1 << 31)
> > +
> > +struct sigcontext {
> > + __u64 sc_pc;
> > + __u64 sc_regs[32];
> > + __u32 sc_flags;
> > + __u64 sc_extcontext[0] __attribute__((__aligned__(16)));
> > +};
> > +
> > +#define CONTEXT_INFO_ALIGN 16
> > +struct _ctxinfo {
> > + __u32 magic;
> > + __u32 size;
> > + __u64 padding; /* padding to 16 bytes */
> > +};
>
> This is probably something I a missing but what is struct _ctxinfo and
> why is it in a uapi header?
>
> I don't see anything else in the uapi implementation using it.
This is used by get_ctx_through_ctxinfo() in signal.c and I think
similar function is also needed by userspace.

Its name is once before called context_info but conflict with another
software, then I want to use ctx_info but conflict with another kernel
struct. :(
>
> Symbols that start with an underscore "_" are reserved and should not
> be used in general, and especially not in uapi header files.
Then, maybe we can use sctx_info here?

Huacai
>
> Eric

2022-05-18 17:19:36

by WANG Xuerui

[permalink] [raw]
Subject: Re: [PATCH V11 14/22] LoongArch: Add signal handling support

On 5/19/22 00:54, Huacai Chen wrote:
> Hi, Eric,
>
> On Thu, May 19, 2022 at 12:40 AM Eric W. Biederman
> <[email protected]> wrote:
>> Huacai Chen <[email protected]> writes:
>>
>>> Add ucontext/sigcontext definition and signal handling support for
>>> LoongArch.
>>>
>>> Cc: Eric Biederman <[email protected]>
>>> Cc: Al Viro <[email protected]>
>>> Signed-off-by: Huacai Chen <[email protected]>
>>> ---
>>> arch/loongarch/include/uapi/asm/sigcontext.h | 44 ++
>>> arch/loongarch/include/uapi/asm/signal.h | 13 +
>>> arch/loongarch/include/uapi/asm/ucontext.h | 35 ++
>>> arch/loongarch/kernel/signal.c | 566 +++++++++++++++++++
>>> 4 files changed, 658 insertions(+)
>>> create mode 100644 arch/loongarch/include/uapi/asm/sigcontext.h
>>> create mode 100644 arch/loongarch/include/uapi/asm/signal.h
>>> create mode 100644 arch/loongarch/include/uapi/asm/ucontext.h
>>> create mode 100644 arch/loongarch/kernel/signal.c
>>>
>>> diff --git a/arch/loongarch/include/uapi/asm/sigcontext.h b/arch/loongarch/include/uapi/asm/sigcontext.h
>>> new file mode 100644
>>> index 000000000000..be3d3c6ac83e
>>> --- /dev/null
>>> +++ b/arch/loongarch/include/uapi/asm/sigcontext.h
>>> @@ -0,0 +1,44 @@
>>> +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
>>> +/*
>>> + * Author: Hanlu Li <[email protected]>
>>> + * Huacai Chen <[email protected]>
>>> + *
>>> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
>>> + */
>>> +#ifndef _UAPI_ASM_SIGCONTEXT_H
>>> +#define _UAPI_ASM_SIGCONTEXT_H
>>> +
>>> +#include <linux/types.h>
>>> +#include <linux/posix_types.h>
>>> +
>>> +/* FP context was used */
>>> +#define SC_USED_FP (1 << 0)
>>> +/* Address error was due to memory load */
>>> +#define SC_ADDRERR_RD (1 << 30)
>>> +/* Address error was due to memory store */
>>> +#define SC_ADDRERR_WR (1 << 31)
>>> +
>>> +struct sigcontext {
>>> + __u64 sc_pc;
>>> + __u64 sc_regs[32];
>>> + __u32 sc_flags;
>>> + __u64 sc_extcontext[0] __attribute__((__aligned__(16)));
>>> +};
>>> +
>>> +#define CONTEXT_INFO_ALIGN 16
>>> +struct _ctxinfo {
>>> + __u32 magic;
>>> + __u32 size;
>>> + __u64 padding; /* padding to 16 bytes */
>>> +};
>> This is probably something I a missing but what is struct _ctxinfo and
>> why is it in a uapi header?
>>
>> I don't see anything else in the uapi implementation using it.
> This is used by get_ctx_through_ctxinfo() in signal.c and I think
> similar function is also needed by userspace.
>
> Its name is once before called context_info but conflict with another
> software, then I want to use ctx_info but conflict with another kernel
> struct. :(
>> Symbols that start with an underscore "_" are reserved and should not
>> be used in general, and especially not in uapi header files.
> Then, maybe we can use sctx_info here?

Actually it seems to me that this struct is the header of each
sc_extcontext; i.e. one struct sigcontext with 0~N trailing
"sc_extcontext", and each "sc_extcontext" consists of one "struct
_ctxinfo" header plus the real content. For now there's only the FPU
context but the LSX/LASX/LBT contexts will come later. The "magic" and
"size" fields also matches the definitions following.

So I'd suggest something like "struct sc_extcontext_head" while naming
the individual contexts like "struct sc_extcontext_foo", e.g. "struct
sc_extcontext_fpu". The "sc_extcontext" part could use some further
abbreviation but the end result should be something intuitive.

>
> Huacai
>> Eric

2022-05-18 17:19:43

by Huacai Chen

[permalink] [raw]
Subject: Re: [PATCH V11 00/22] arch: Add basic LoongArch support

Hi, Arnd,

On Wed, May 18, 2022 at 9:42 PM Arnd Bergmann <[email protected]> wrote:
>
> On Wed, May 18, 2022 at 10:25 AM Huacai Chen <[email protected]> wrote:
> >
> > LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V.
> > LoongArch includes a reduced 32-bit version (LA32R), a standard 32-bit
> > version (LA32S) and a 64-bit version (LA64). LoongArch use ACPI as its
> > boot protocol LoongArch-specific interrupt controllers (similar to APIC)
> > are already added in the next revision of ACPI Specification (current
> > revision is 6.4).
> >
> > This patchset is adding basic LoongArch support in mainline kernel, we
> > can see a complete snapshot here:
> > https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git/log/?h=loongarch-next
>
> Stephen, can you add this branch to the linux-next tree?
>
> I see there are still comments coming in, but at some point this has
> to just be considered good enough that any further changes can be addressed
> with patches on top rather than rebasing.
>
> > V10 -> V11:
> > 1, Rebased on asm-generic tree;
>
> I was expecting that you'd base this on just the spinlock changes from Palmer's
> tree that are part of the asm-generic tree rather than all of what I have.
>
> Can you rebase it once more? If there are conflicts against the h8300 removal
> series that is also in asm-generic, leaving it on top of that may be
> easier though.
Thank you very much, I have rebased the below tree on asm-generic
tree's spinlock commit 9282d0996936c5fbf877c0d096a3feb45 again, and
fixed some small problems from Eric and WANG Xuerui's comments in V11.
https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git/log/?h=loongarch-next

Huacai
>
> Arnd

2022-05-18 22:58:50

by Stephen Rothwell

[permalink] [raw]
Subject: Re: [PATCH V11 00/22] arch: Add basic LoongArch support

Hi Arnd,

On Wed, 18 May 2022 14:42:07 +0100 Arnd Bergmann <[email protected]> wrote:
>
> On Wed, May 18, 2022 at 10:25 AM Huacai Chen <[email protected]> wrote:
> >
> > LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V.
> > LoongArch includes a reduced 32-bit version (LA32R), a standard 32-bit
> > version (LA32S) and a 64-bit version (LA64). LoongArch use ACPI as its
> > boot protocol LoongArch-specific interrupt controllers (similar to APIC)
> > are already added in the next revision of ACPI Specification (current
> > revision is 6.4).
> >
> > This patchset is adding basic LoongArch support in mainline kernel, we
> > can see a complete snapshot here:
> > https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git/log/?h=loongarch-next
>
> Stephen, can you add this branch to the linux-next tree?

Added from today. I have called it "loongarch" and it will be merged
among the other architecture trees after the asm-generic tree.
Currently Huacai Chen is the only contact listed.

I note that there is no new entry in MAINTAINERS in this tree.

> I see there are still comments coming in, but at some point this has
> to just be considered good enough that any further changes can be addressed
> with patches on top rather than rebasing.

That should be fine.

Thanks for adding your subsystem tree as a participant of linux-next. As
you may know, this is not a judgement of your code. The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window.

You will need to ensure that the patches/commits in your tree/series have
been:
* submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
* posted to the relevant mailing list,
* reviewed by you (or another maintainer of your subsystem tree),
* successfully unit tested, and
* destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch). It is allowed to be rebased if you deem it necessary.

--
Cheers,
Stephen Rothwell
[email protected]

--
Cheers,
Stephen Rothwell


Attachments:
(No filename) (499.00 B)
OpenPGP digital signature

2022-05-21 11:13:19

by Huacai Chen

[permalink] [raw]
Subject: Re: [PATCH V11 07/22] LoongArch: Add atomic/locking headers

Hi, Ren,

On Fri, May 20, 2022 at 3:54 PM Guo Ren <[email protected]> wrote:
>
> On Wed, May 18, 2022 at 5:34 PM Huacai Chen <[email protected]> wrote:
> >
> > Add common headers (atomic, bitops, barrier and locking) for basic
> > LoongArch support.
> >
> > Signed-off-by: Huacai Chen <[email protected]>
> > ---
> > arch/loongarch/include/asm/atomic.h | 358 +++++++++++++++++++++++++++
> > arch/loongarch/include/asm/barrier.h | 51 ++++
> > arch/loongarch/include/asm/bitops.h | 33 +++
> > arch/loongarch/include/asm/bitrev.h | 34 +++
> > arch/loongarch/include/asm/cmpxchg.h | 121 +++++++++
> > arch/loongarch/include/asm/local.h | 138 +++++++++++
> > arch/loongarch/include/asm/percpu.h | 20 ++
> > 7 files changed, 755 insertions(+)
> > create mode 100644 arch/loongarch/include/asm/atomic.h
> > create mode 100644 arch/loongarch/include/asm/barrier.h
> > create mode 100644 arch/loongarch/include/asm/bitops.h
> > create mode 100644 arch/loongarch/include/asm/bitrev.h
> > create mode 100644 arch/loongarch/include/asm/cmpxchg.h
> > create mode 100644 arch/loongarch/include/asm/local.h
> > create mode 100644 arch/loongarch/include/asm/percpu.h
> >
> > diff --git a/arch/loongarch/include/asm/atomic.h b/arch/loongarch/include/asm/atomic.h
> > new file mode 100644
> > index 000000000000..932352342b12
> > --- /dev/null
> > +++ b/arch/loongarch/include/asm/atomic.h
> > @@ -0,0 +1,358 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Atomic operations.
> > + *
> > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > + */
> > +#ifndef _ASM_ATOMIC_H
> > +#define _ASM_ATOMIC_H
> > +
> > +#include <linux/types.h>
> > +#include <asm/barrier.h>
> > +#include <asm/cmpxchg.h>
> > +#include <asm/compiler.h>
> > +
> > +#if __SIZEOF_LONG__ == 4
> > +#define __LL "ll.w "
> > +#define __SC "sc.w "
> > +#define __AMADD "amadd.w "
> > +#define __AMAND_DB "amand_db.w "
> > +#define __AMOR_DB "amor_db.w "
> > +#define __AMXOR_DB "amxor_db.w "
> > +#elif __SIZEOF_LONG__ == 8
> > +#define __LL "ll.d "
> > +#define __SC "sc.d "
> > +#define __AMADD "amadd.d "
> > +#define __AMAND_DB "amand_db.d "
> > +#define __AMOR_DB "amor_db.d "
> > +#define __AMXOR_DB "amxor_db.d "
> > +#endif
> > +
> > +#define ATOMIC_INIT(i) { (i) }
> > +
> > +/*
> > + * arch_atomic_read - read atomic variable
> > + * @v: pointer of type atomic_t
> > + *
> > + * Atomically reads the value of @v.
> > + */
> > +#define arch_atomic_read(v) READ_ONCE((v)->counter)
> > +
> > +/*
> > + * arch_atomic_set - set atomic variable
> > + * @v: pointer of type atomic_t
> > + * @i: required value
> > + *
> > + * Atomically sets the value of @v to @i.
> > + */
> > +#define arch_atomic_set(v, i) WRITE_ONCE((v)->counter, (i))
> > +
> > +#define ATOMIC_OP(op, I, asm_op) \
> > +static inline void arch_atomic_##op(int i, atomic_t *v) \
> > +{ \
> > + __asm__ __volatile__( \
> > + "am"#asm_op"_db.w" " $zero, %1, %0 \n" \
> > + : "+ZB" (v->counter) \
> > + : "r" (I) \
> > + : "memory"); \
> > +}
> > +
> > +#define ATOMIC_OP_RETURN(op, I, asm_op, c_op) \
> > +static inline int arch_atomic_##op##_return_relaxed(int i, atomic_t *v) \
> > +{ \
> > + int result; \
> > + \
> > + __asm__ __volatile__( \
> > + "am"#asm_op"_db.w" " %1, %2, %0 \n" \
> > + : "+ZB" (v->counter), "=&r" (result) \
> > + : "r" (I) \
> > + : "memory"); \
> > + \
> > + return result c_op I; \
> > +}
> > +
> > +#define ATOMIC_FETCH_OP(op, I, asm_op) \
> > +static inline int arch_atomic_fetch_##op##_relaxed(int i, atomic_t *v) \
> > +{ \
> > + int result; \
> > + \
> > + __asm__ __volatile__( \
> > + "am"#asm_op"_db.w" " %1, %2, %0 \n" \
> > + : "+ZB" (v->counter), "=&r" (result) \
> > + : "r" (I) \
> > + : "memory"); \
> > + \
> > + return result; \
> > +}
> > +
> > +#define ATOMIC_OPS(op, I, asm_op, c_op) \
> > + ATOMIC_OP(op, I, asm_op) \
> > + ATOMIC_OP_RETURN(op, I, asm_op, c_op) \
> > + ATOMIC_FETCH_OP(op, I, asm_op)
> > +
> > +ATOMIC_OPS(add, i, add, +)
> > +ATOMIC_OPS(sub, -i, add, +)
> > +
> > +#define arch_atomic_add_return_relaxed arch_atomic_add_return_relaxed
> > +#define arch_atomic_sub_return_relaxed arch_atomic_sub_return_relaxed
> > +#define arch_atomic_fetch_add_relaxed arch_atomic_fetch_add_relaxed
> > +#define arch_atomic_fetch_sub_relaxed arch_atomic_fetch_sub_relaxed
> > +
> > +#undef ATOMIC_OPS
> > +
> > +#define ATOMIC_OPS(op, I, asm_op) \
> > + ATOMIC_OP(op, I, asm_op) \
> > + ATOMIC_FETCH_OP(op, I, asm_op)
> > +
> > +ATOMIC_OPS(and, i, and)
> > +ATOMIC_OPS(or, i, or)
> > +ATOMIC_OPS(xor, i, xor)
> > +
> > +#define arch_atomic_fetch_and_relaxed arch_atomic_fetch_and_relaxed
> > +#define arch_atomic_fetch_or_relaxed arch_atomic_fetch_or_relaxed
> > +#define arch_atomic_fetch_xor_relaxed arch_atomic_fetch_xor_relaxed
> > +
> > +#undef ATOMIC_OPS
> > +#undef ATOMIC_FETCH_OP
> > +#undef ATOMIC_OP_RETURN
> > +#undef ATOMIC_OP
> > +
> > +static inline int arch_atomic_fetch_add_unless(atomic_t *v, int a, int u)
> > +{
> > + int prev, rc;
> > +
> > + __asm__ __volatile__ (
> We need __WEAK_LLSC_MB here, to prevent previous access after sc.w.
LoongArch's ll and sc has implied memory barrier. :)

>
> > + "0: ll.w %[p], %[c]\n"
> > + " beq %[p], %[u], 1f\n"
> > + " add.w %[rc], %[p], %[a]\n"
> > + " sc.w %[rc], %[c]\n"
> > + " beqz %[rc], 0b\n"
> > + " b 2f\n"
> > + "1:\n"
> > + __WEAK_LLSC_MB
> > + "2:\n"
> > + : [p]"=&r" (prev), [rc]"=&r" (rc),
> > + [c]"=ZB" (v->counter)
> > + : [a]"r" (a), [u]"r" (u)
> > + : "memory");
> > +
> > + return prev;
> > +}
> > +#define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless
> > +
> > +/*
> > + * arch_atomic_sub_if_positive - conditionally subtract integer from atomic variable
> > + * @i: integer value to subtract
> > + * @v: pointer of type atomic_t
> > + *
> > + * Atomically test @v and subtract @i if @v is greater or equal than @i.
> > + * The function returns the old value of @v minus @i.
> > + */
> > +static inline int arch_atomic_sub_if_positive(int i, atomic_t *v)
> > +{
> > + int result;
> > + int temp;
> > +
> > + if (__builtin_constant_p(i)) {
> > + __asm__ __volatile__(
> Ditto.
> > + "1: ll.w %1, %2 # atomic_sub_if_positive\n"
> > + " addi.w %0, %1, %3 \n"
> > + " or %1, %0, $zero \n"
> > + " blt %0, $zero, 2f \n"
> > + " sc.w %1, %2 \n"
> > + " beq $zero, %1, 1b \n"
> Need __WEAK_LLSC_MB ?
> > + "2: \n"
> > + : "=&r" (result), "=&r" (temp),
> > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > + : "I" (-i));
> > + } else {
> > + __asm__ __volatile__(
> Ditto.
> > + "1: ll.w %1, %2 # atomic_sub_if_positive\n"
> > + " sub.w %0, %1, %3 \n"
> > + " or %1, %0, $zero \n"
> > + " blt %0, $zero, 2f \n"
> > + " sc.w %1, %2 \n"
> Need __WEAK_LLSC_MB ?
> > + " beq $zero, %1, 1b \n"
> > + "2: \n"
> > + : "=&r" (result), "=&r" (temp),
> > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > + : "r" (i));
> > + }
> > +
> > + return result;
> > +}
> > +
> > +#define arch_atomic_cmpxchg(v, o, n) (arch_cmpxchg(&((v)->counter), (o), (n)))
> > +#define arch_atomic_xchg(v, new) (arch_xchg(&((v)->counter), (new)))
> > +
> > +/*
> > + * arch_atomic_dec_if_positive - decrement by 1 if old value positive
> > + * @v: pointer of type atomic_t
> > + */
> > +#define arch_atomic_dec_if_positive(v) arch_atomic_sub_if_positive(1, v)
> > +
> > +#ifdef CONFIG_64BIT
> > +
> > +#define ATOMIC64_INIT(i) { (i) }
> > +
> > +/*
> > + * arch_atomic64_read - read atomic variable
> > + * @v: pointer of type atomic64_t
> > + *
> > + */
> > +#define arch_atomic64_read(v) READ_ONCE((v)->counter)
> > +
> > +/*
> > + * arch_atomic64_set - set atomic variable
> > + * @v: pointer of type atomic64_t
> > + * @i: required value
> > + */
> > +#define arch_atomic64_set(v, i) WRITE_ONCE((v)->counter, (i))
> > +
> > +#define ATOMIC64_OP(op, I, asm_op) \
> > +static inline void arch_atomic64_##op(long i, atomic64_t *v) \
> > +{ \
> > + __asm__ __volatile__( \
> > + "am"#asm_op"_db.d " " $zero, %1, %0 \n" \
> > + : "+ZB" (v->counter) \
> > + : "r" (I) \
> > + : "memory"); \
> > +}
> > +
> > +#define ATOMIC64_OP_RETURN(op, I, asm_op, c_op) \
> > +static inline long arch_atomic64_##op##_return_relaxed(long i, atomic64_t *v) \
> > +{ \
> > + long result; \
> > + __asm__ __volatile__( \
> > + "am"#asm_op"_db.d " " %1, %2, %0 \n" \
> > + : "+ZB" (v->counter), "=&r" (result) \
> > + : "r" (I) \
> > + : "memory"); \
> > + \
> > + return result c_op I; \
> > +}
> > +
> > +#define ATOMIC64_FETCH_OP(op, I, asm_op) \
> > +static inline long arch_atomic64_fetch_##op##_relaxed(long i, atomic64_t *v) \
> > +{ \
> > + long result; \
> > + \
> > + __asm__ __volatile__( \
> > + "am"#asm_op"_db.d " " %1, %2, %0 \n" \
> > + : "+ZB" (v->counter), "=&r" (result) \
> > + : "r" (I) \
> > + : "memory"); \
> > + \
> > + return result; \
> > +}
> > +
> > +#define ATOMIC64_OPS(op, I, asm_op, c_op) \
> > + ATOMIC64_OP(op, I, asm_op) \
> > + ATOMIC64_OP_RETURN(op, I, asm_op, c_op) \
> > + ATOMIC64_FETCH_OP(op, I, asm_op)
> > +
> > +ATOMIC64_OPS(add, i, add, +)
> > +ATOMIC64_OPS(sub, -i, add, +)
> > +
> > +#define arch_atomic64_add_return_relaxed arch_atomic64_add_return_relaxed
> > +#define arch_atomic64_sub_return_relaxed arch_atomic64_sub_return_relaxed
> > +#define arch_atomic64_fetch_add_relaxed arch_atomic64_fetch_add_relaxed
> > +#define arch_atomic64_fetch_sub_relaxed arch_atomic64_fetch_sub_relaxed
> > +
> > +#undef ATOMIC64_OPS
> > +
> > +#define ATOMIC64_OPS(op, I, asm_op) \
> > + ATOMIC64_OP(op, I, asm_op) \
> > + ATOMIC64_FETCH_OP(op, I, asm_op)
> > +
> > +ATOMIC64_OPS(and, i, and)
> > +ATOMIC64_OPS(or, i, or)
> > +ATOMIC64_OPS(xor, i, xor)
> > +
> > +#define arch_atomic64_fetch_and_relaxed arch_atomic64_fetch_and_relaxed
> > +#define arch_atomic64_fetch_or_relaxed arch_atomic64_fetch_or_relaxed
> > +#define arch_atomic64_fetch_xor_relaxed arch_atomic64_fetch_xor_relaxed
> > +
> > +#undef ATOMIC64_OPS
> > +#undef ATOMIC64_FETCH_OP
> > +#undef ATOMIC64_OP_RETURN
> > +#undef ATOMIC64_OP
> > +
> > +static inline long arch_atomic64_fetch_add_unless(atomic64_t *v, long a, long u)
> > +{
> > + long prev, rc;
> > +
> > + __asm__ __volatile__ (
> Ditto
> > + "0: ll.d %[p], %[c]\n"
> > + " beq %[p], %[u], 1f\n"
> > + " add.d %[rc], %[p], %[a]\n"
> > + " sc.d %[rc], %[c]\n"
> > + " beqz %[rc], 0b\n"
> Ditto
> > + " b 2f\n"
> > + "1:\n"
> > + __WEAK_LLSC_MB
> > + "2:\n"
> > + : [p]"=&r" (prev), [rc]"=&r" (rc),
> > + [c] "=ZB" (v->counter)
> > + : [a]"r" (a), [u]"r" (u)
> > + : "memory");
> > +
> > + return prev;
> > +}
> > +#define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless
> > +
> > +/*
> > + * arch_atomic64_sub_if_positive - conditionally subtract integer from atomic variable
> > + * @i: integer value to subtract
> > + * @v: pointer of type atomic64_t
> > + *
> > + * Atomically test @v and subtract @i if @v is greater or equal than @i.
> > + * The function returns the old value of @v minus @i.
> > + */
> > +static inline long arch_atomic64_sub_if_positive(long i, atomic64_t *v)
> > +{
> > + long result;
> > + long temp;
> > +
> > + if (__builtin_constant_p(i)) {
> > + __asm__ __volatile__(
> Ditto
> > + "1: ll.d %1, %2 # atomic64_sub_if_positive \n"
> > + " addi.d %0, %1, %3 \n"
> > + " or %1, %0, $zero \n"
> > + " blt %0, $zero, 2f \n"
> > + " sc.d %1, %2 \n"
> Ditto
> > + " beq %1, $zero, 1b \n"
> > + "2: \n"
> > + : "=&r" (result), "=&r" (temp),
> > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > + : "I" (-i));
> > + } else {
> > + __asm__ __volatile__(
> Ditto
> > + "1: ll.d %1, %2 # atomic64_sub_if_positive \n"
> > + " sub.d %0, %1, %3 \n"
> > + " or %1, %0, $zero \n"
> > + " blt %0, $zero, 2f \n"
> > + " sc.d %1, %2 \n"
> Ditto
> > + " beq %1, $zero, 1b \n"
> > + "2: \n"
> > + : "=&r" (result), "=&r" (temp),
> > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > + : "r" (i));
> > + }
> > +
> > + return result;
> > +}
> > +
> > +#define arch_atomic64_cmpxchg(v, o, n) \
> > + ((__typeof__((v)->counter))arch_cmpxchg(&((v)->counter), (o), (n)))
> > +#define arch_atomic64_xchg(v, new) (arch_xchg(&((v)->counter), (new)))
> > +
> > +/*
> > + * arch_atomic64_dec_if_positive - decrement by 1 if old value positive
> > + * @v: pointer of type atomic64_t
> > + */
> > +#define arch_atomic64_dec_if_positive(v) arch_atomic64_sub_if_positive(1, v)
> > +
> > +#endif /* CONFIG_64BIT */
> > +
> > +#endif /* _ASM_ATOMIC_H */
> > diff --git a/arch/loongarch/include/asm/barrier.h b/arch/loongarch/include/asm/barrier.h
> > new file mode 100644
> > index 000000000000..e57571bcaf4f
> > --- /dev/null
> > +++ b/arch/loongarch/include/asm/barrier.h
> > @@ -0,0 +1,51 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > + */
> > +#ifndef __ASM_BARRIER_H
> > +#define __ASM_BARRIER_H
> > +
> > +#define __sync() __asm__ __volatile__("dbar 0" : : : "memory")
> > +
> > +#define fast_wmb() __sync()
> > +#define fast_rmb() __sync()
> > +#define fast_mb() __sync()
> > +#define fast_iob() __sync()
> > +#define wbflush() __sync()
> > +
> > +#define wmb() fast_wmb()
> > +#define rmb() fast_rmb()
> > +#define mb() fast_mb()
> > +#define iob() fast_iob()
> > +
> > +/**
> > + * array_index_mask_nospec() - generate a ~0 mask when index < size, 0 otherwise
> > + * @index: array element index
> > + * @size: number of elements in array
> > + *
> > + * Returns:
> > + * 0 - (@index < @size)
> > + */
> > +#define array_index_mask_nospec array_index_mask_nospec
> > +static inline unsigned long array_index_mask_nospec(unsigned long index,
> > + unsigned long size)
> > +{
> > + unsigned long mask;
> > +
> > + __asm__ __volatile__(
> > + "sltu %0, %1, %2\n\t"
> > +#if (__SIZEOF_LONG__ == 4)
> > + "sub.w %0, $r0, %0\n\t"
> > +#elif (__SIZEOF_LONG__ == 8)
> > + "sub.d %0, $r0, %0\n\t"
> > +#endif
> > + : "=r" (mask)
> > + : "r" (index), "r" (size)
> > + :);
> > +
> > + return mask;
> > +}
> > +
> > +#include <asm-generic/barrier.h>
> > +
> > +#endif /* __ASM_BARRIER_H */
> > diff --git a/arch/loongarch/include/asm/bitops.h b/arch/loongarch/include/asm/bitops.h
> > new file mode 100644
> > index 000000000000..69e00f8d8034
> > --- /dev/null
> > +++ b/arch/loongarch/include/asm/bitops.h
> > @@ -0,0 +1,33 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > + */
> > +#ifndef _ASM_BITOPS_H
> > +#define _ASM_BITOPS_H
> > +
> > +#include <linux/compiler.h>
> > +
> > +#ifndef _LINUX_BITOPS_H
> > +#error only <linux/bitops.h> can be included directly
> > +#endif
> > +
> > +#include <asm/barrier.h>
> > +
> > +#include <asm-generic/bitops/builtin-ffs.h>
> > +#include <asm-generic/bitops/builtin-fls.h>
> > +#include <asm-generic/bitops/builtin-__ffs.h>
> > +#include <asm-generic/bitops/builtin-__fls.h>
> > +
> > +#include <asm-generic/bitops/ffz.h>
> > +#include <asm-generic/bitops/fls64.h>
> > +
> > +#include <asm-generic/bitops/sched.h>
> > +#include <asm-generic/bitops/hweight.h>
> > +
> > +#include <asm-generic/bitops/atomic.h>
> > +#include <asm-generic/bitops/non-atomic.h>
> > +#include <asm-generic/bitops/lock.h>
> > +#include <asm-generic/bitops/le.h>
> > +#include <asm-generic/bitops/ext2-atomic.h>
> > +
> > +#endif /* _ASM_BITOPS_H */
> > diff --git a/arch/loongarch/include/asm/bitrev.h b/arch/loongarch/include/asm/bitrev.h
> > new file mode 100644
> > index 000000000000..46f275b9cdf7
> > --- /dev/null
> > +++ b/arch/loongarch/include/asm/bitrev.h
> > @@ -0,0 +1,34 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > + */
> > +#ifndef __LOONGARCH_ASM_BITREV_H__
> > +#define __LOONGARCH_ASM_BITREV_H__
> > +
> > +#include <linux/swab.h>
> > +
> > +static __always_inline __attribute_const__ u32 __arch_bitrev32(u32 x)
> > +{
> > + u32 ret;
> > +
> > + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(__swab32(x)));
> > + return ret;
> > +}
> > +
> > +static __always_inline __attribute_const__ u16 __arch_bitrev16(u16 x)
> > +{
> > + u16 ret;
> > +
> > + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(__swab16(x)));
> > + return ret;
> > +}
> > +
> > +static __always_inline __attribute_const__ u8 __arch_bitrev8(u8 x)
> > +{
> > + u8 ret;
> > +
> > + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(x));
> > + return ret;
> > +}
> > +
> > +#endif /* __LOONGARCH_ASM_BITREV_H__ */
> > diff --git a/arch/loongarch/include/asm/cmpxchg.h b/arch/loongarch/include/asm/cmpxchg.h
> > new file mode 100644
> > index 000000000000..699925b8ff27
> > --- /dev/null
> > +++ b/arch/loongarch/include/asm/cmpxchg.h
> > @@ -0,0 +1,121 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > + */
> > +#ifndef __ASM_CMPXCHG_H
> > +#define __ASM_CMPXCHG_H
> > +
> > +#include <linux/build_bug.h>
> > +
> > +#define __xchg_asm(amswap_db, m, val) \
> > +({ \
> > + __typeof(val) __ret; \
> > + \
> > + __asm__ __volatile__ ( \
> > + " "amswap_db" %1, %z2, %0 \n" \
> > + : "+ZB" (*m), "=&r" (__ret) \
> > + : "Jr" (val) \
> > + : "memory"); \
> > + \
> > + __ret; \
> > +})
> > +
> > +static inline unsigned long __xchg(volatile void *ptr, unsigned long x,
> relaxed ?
>
> > + int size)
> > +{
> > + switch (size) {
> > + case 4:
> > + return __xchg_asm("amswap_db.w", (volatile u32 *)ptr, (u32)x);
> Your amswap_db don't have any implicit acquire/release, right?
amswap don't have implicit acquire/release, but amswap_db has.

Huacai
>
> > +
> > + case 8:
> > + return __xchg_asm("amswap_db.d", (volatile u64 *)ptr, (u64)x);
> > +
> > + default:
> > + BUILD_BUG();
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +#define arch_xchg(ptr, x) \
> > +({ \
> > + __typeof__(*(ptr)) __res; \
> > + \
> > + __res = (__typeof__(*(ptr))) \
> > + __xchg((ptr), (unsigned long)(x), sizeof(*(ptr))); \
> > + \
> > + __res; \
> > +})
> > +
> > +#define __cmpxchg_asm(ld, st, m, old, new) \
> > +({ \
> > + __typeof(old) __ret; \
> > + \
> > + __asm__ __volatile__( \
> > + "1: " ld " %0, %2 # __cmpxchg_asm \n" \
> > + " bne %0, %z3, 2f \n" \
> > + " or $t0, %z4, $zero \n" \
> > + " " st " $t0, %1 \n" \
> > + " beq $zero, $t0, 1b \n" \
> > + "2: \n" \
> > + : "=&r" (__ret), "=ZB"(*m) \
> > + : "ZB"(*m), "Jr" (old), "Jr" (new) \
> > + : "t0", "memory"); \
> > + \
> > + __ret; \
> > +})
> > +
> > +static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
> > + unsigned long new, unsigned int size)
> > +{
> > + switch (size) {
> > + case 4:
> > + return __cmpxchg_asm("ll.w", "sc.w", (volatile u32 *)ptr,
> > + (u32)old, new);
> > +
> > + case 8:
> > + return __cmpxchg_asm("ll.d", "sc.d", (volatile u64 *)ptr,
> > + (u64)old, new);
> > +
> > + default:
> > + BUILD_BUG();
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +#define arch_cmpxchg_local(ptr, old, new) \
> > + ((__typeof__(*(ptr))) \
> > + __cmpxchg((ptr), \
> > + (unsigned long)(__typeof__(*(ptr)))(old), \
> > + (unsigned long)(__typeof__(*(ptr)))(new), \
> > + sizeof(*(ptr))))
> > +
> > +#define arch_cmpxchg(ptr, old, new) \
> Ditto, it's a relaxed implementation, right?
> > +({ \
> > + __typeof__(*(ptr)) __res; \
> > + \
> > + __res = arch_cmpxchg_local((ptr), (old), (new)); \
> > + \
> > + __res; \
> > +})
> > +
> > +#ifdef CONFIG_64BIT
> > +#define arch_cmpxchg64_local(ptr, o, n) \
> > + ({ \
> > + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \
> > + arch_cmpxchg_local((ptr), (o), (n)); \
> > + })
> > +
> > +#define arch_cmpxchg64(ptr, o, n) \
> > + ({ \
> > + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \
> > + arch_cmpxchg((ptr), (o), (n)); \
> > + })
> > +#else
> > +#include <asm-generic/cmpxchg-local.h>
> > +#define arch_cmpxchg64_local(ptr, o, n) __generic_cmpxchg64_local((ptr), (o), (n))
> > +#define arch_cmpxchg64(ptr, o, n) arch_cmpxchg64_local((ptr), (o), (n))
> > +#endif
> > +
> > +#endif /* __ASM_CMPXCHG_H */
> > diff --git a/arch/loongarch/include/asm/local.h b/arch/loongarch/include/asm/local.h
> > new file mode 100644
> > index 000000000000..2052a2267337
> > --- /dev/null
> > +++ b/arch/loongarch/include/asm/local.h
> > @@ -0,0 +1,138 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > + */
> > +#ifndef _ARCH_LOONGARCH_LOCAL_H
> > +#define _ARCH_LOONGARCH_LOCAL_H
> > +
> > +#include <linux/percpu.h>
> > +#include <linux/bitops.h>
> > +#include <linux/atomic.h>
> > +#include <asm/cmpxchg.h>
> > +#include <asm/compiler.h>
> > +
> > +typedef struct {
> > + atomic_long_t a;
> > +} local_t;
> > +
> > +#define LOCAL_INIT(i) { ATOMIC_LONG_INIT(i) }
> > +
> > +#define local_read(l) atomic_long_read(&(l)->a)
> > +#define local_set(l, i) atomic_long_set(&(l)->a, (i))
> > +
> > +#define local_add(i, l) atomic_long_add((i), (&(l)->a))
> > +#define local_sub(i, l) atomic_long_sub((i), (&(l)->a))
> > +#define local_inc(l) atomic_long_inc(&(l)->a)
> > +#define local_dec(l) atomic_long_dec(&(l)->a)
> > +
> > +/*
> > + * Same as above, but return the result value
> > + */
> > +static inline long local_add_return(long i, local_t *l)
> > +{
> > + unsigned long result;
> > +
> > + __asm__ __volatile__(
> > + " " __AMADD " %1, %2, %0 \n"
> > + : "+ZB" (l->a.counter), "=&r" (result)
> > + : "r" (i)
> > + : "memory");
> > + result = result + i;
> > +
> > + return result;
> > +}
> > +
> > +static inline long local_sub_return(long i, local_t *l)
> > +{
> > + unsigned long result;
> > +
> > + __asm__ __volatile__(
> > + " " __AMADD "%1, %2, %0 \n"
> > + : "+ZB" (l->a.counter), "=&r" (result)
> > + : "r" (-i)
> > + : "memory");
> > +
> > + result = result - i;
> > +
> > + return result;
> > +}
> > +
> > +#define local_cmpxchg(l, o, n) \
> > + ((long)cmpxchg_local(&((l)->a.counter), (o), (n)))
> > +#define local_xchg(l, n) (atomic_long_xchg((&(l)->a), (n)))
> > +
> > +/**
> > + * local_add_unless - add unless the number is a given value
> > + * @l: pointer of type local_t
> > + * @a: the amount to add to l...
> > + * @u: ...unless l is equal to u.
> > + *
> > + * Atomically adds @a to @l, so long as it was not @u.
> > + * Returns non-zero if @l was not @u, and zero otherwise.
> > + */
> > +#define local_add_unless(l, a, u) \
> > +({ \
> > + long c, old; \
> > + c = local_read(l); \
> > + while (c != (u) && (old = local_cmpxchg((l), c, c + (a))) != c) \
> > + c = old; \
> > + c != (u); \
> > +})
> > +#define local_inc_not_zero(l) local_add_unless((l), 1, 0)
> > +
> > +#define local_dec_return(l) local_sub_return(1, (l))
> > +#define local_inc_return(l) local_add_return(1, (l))
> > +
> > +/*
> > + * local_sub_and_test - subtract value from variable and test result
> > + * @i: integer value to subtract
> > + * @l: pointer of type local_t
> > + *
> > + * Atomically subtracts @i from @l and returns
> > + * true if the result is zero, or false for all
> > + * other cases.
> > + */
> > +#define local_sub_and_test(i, l) (local_sub_return((i), (l)) == 0)
> > +
> > +/*
> > + * local_inc_and_test - increment and test
> > + * @l: pointer of type local_t
> > + *
> > + * Atomically increments @l by 1
> > + * and returns true if the result is zero, or false for all
> > + * other cases.
> > + */
> > +#define local_inc_and_test(l) (local_inc_return(l) == 0)
> > +
> > +/*
> > + * local_dec_and_test - decrement by 1 and test
> > + * @l: pointer of type local_t
> > + *
> > + * Atomically decrements @l by 1 and
> > + * returns true if the result is 0, or false for all other
> > + * cases.
> > + */
> > +#define local_dec_and_test(l) (local_sub_return(1, (l)) == 0)
> > +
> > +/*
> > + * local_add_negative - add and test if negative
> > + * @l: pointer of type local_t
> > + * @i: integer value to add
> > + *
> > + * Atomically adds @i to @l and returns true
> > + * if the result is negative, or false when
> > + * result is greater than or equal to zero.
> > + */
> > +#define local_add_negative(i, l) (local_add_return(i, (l)) < 0)
> > +
> > +/* Use these for per-cpu local_t variables: on some archs they are
> > + * much more efficient than these naive implementations. Note they take
> > + * a variable, not an address.
> > + */
> > +
> > +#define __local_inc(l) ((l)->a.counter++)
> > +#define __local_dec(l) ((l)->a.counter++)
> > +#define __local_add(i, l) ((l)->a.counter += (i))
> > +#define __local_sub(i, l) ((l)->a.counter -= (i))
> > +
> > +#endif /* _ARCH_LOONGARCH_LOCAL_H */
> > diff --git a/arch/loongarch/include/asm/percpu.h b/arch/loongarch/include/asm/percpu.h
> > new file mode 100644
> > index 000000000000..7d5b22ebd834
> > --- /dev/null
> > +++ b/arch/loongarch/include/asm/percpu.h
> > @@ -0,0 +1,20 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > + */
> > +#ifndef __ASM_PERCPU_H
> > +#define __ASM_PERCPU_H
> > +
> > +/* Use r21 for fast access */
> > +register unsigned long __my_cpu_offset __asm__("$r21");
> > +
> > +static inline void set_my_cpu_offset(unsigned long off)
> > +{
> > + __my_cpu_offset = off;
> > + csr_writeq(off, PERCPU_BASE_KS);
> > +}
> > +#define __my_cpu_offset __my_cpu_offset
> > +
> > +#include <asm-generic/percpu.h>
> > +
> > +#endif /* __ASM_PERCPU_H */
> > --
> > 2.27.0
> >
>
>
> --
> Best Regards
> Guo Ren
>
> ML: https://lore.kernel.org/linux-csky/

2022-05-21 11:14:15

by Guo Ren

[permalink] [raw]
Subject: Re: [PATCH V11 07/22] LoongArch: Add atomic/locking headers

On Wed, May 18, 2022 at 5:34 PM Huacai Chen <[email protected]> wrote:
>
> Add common headers (atomic, bitops, barrier and locking) for basic
> LoongArch support.
>
> Signed-off-by: Huacai Chen <[email protected]>
> ---
> arch/loongarch/include/asm/atomic.h | 358 +++++++++++++++++++++++++++
> arch/loongarch/include/asm/barrier.h | 51 ++++
> arch/loongarch/include/asm/bitops.h | 33 +++
> arch/loongarch/include/asm/bitrev.h | 34 +++
> arch/loongarch/include/asm/cmpxchg.h | 121 +++++++++
> arch/loongarch/include/asm/local.h | 138 +++++++++++
> arch/loongarch/include/asm/percpu.h | 20 ++
> 7 files changed, 755 insertions(+)
> create mode 100644 arch/loongarch/include/asm/atomic.h
> create mode 100644 arch/loongarch/include/asm/barrier.h
> create mode 100644 arch/loongarch/include/asm/bitops.h
> create mode 100644 arch/loongarch/include/asm/bitrev.h
> create mode 100644 arch/loongarch/include/asm/cmpxchg.h
> create mode 100644 arch/loongarch/include/asm/local.h
> create mode 100644 arch/loongarch/include/asm/percpu.h
>
> diff --git a/arch/loongarch/include/asm/atomic.h b/arch/loongarch/include/asm/atomic.h
> new file mode 100644
> index 000000000000..932352342b12
> --- /dev/null
> +++ b/arch/loongarch/include/asm/atomic.h
> @@ -0,0 +1,358 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Atomic operations.
> + *
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef _ASM_ATOMIC_H
> +#define _ASM_ATOMIC_H
> +
> +#include <linux/types.h>
> +#include <asm/barrier.h>
> +#include <asm/cmpxchg.h>
> +#include <asm/compiler.h>
> +
> +#if __SIZEOF_LONG__ == 4
> +#define __LL "ll.w "
> +#define __SC "sc.w "
> +#define __AMADD "amadd.w "
> +#define __AMAND_DB "amand_db.w "
> +#define __AMOR_DB "amor_db.w "
> +#define __AMXOR_DB "amxor_db.w "
> +#elif __SIZEOF_LONG__ == 8
> +#define __LL "ll.d "
> +#define __SC "sc.d "
> +#define __AMADD "amadd.d "
> +#define __AMAND_DB "amand_db.d "
> +#define __AMOR_DB "amor_db.d "
> +#define __AMXOR_DB "amxor_db.d "
> +#endif
> +
> +#define ATOMIC_INIT(i) { (i) }
> +
> +/*
> + * arch_atomic_read - read atomic variable
> + * @v: pointer of type atomic_t
> + *
> + * Atomically reads the value of @v.
> + */
> +#define arch_atomic_read(v) READ_ONCE((v)->counter)
> +
> +/*
> + * arch_atomic_set - set atomic variable
> + * @v: pointer of type atomic_t
> + * @i: required value
> + *
> + * Atomically sets the value of @v to @i.
> + */
> +#define arch_atomic_set(v, i) WRITE_ONCE((v)->counter, (i))
> +
> +#define ATOMIC_OP(op, I, asm_op) \
> +static inline void arch_atomic_##op(int i, atomic_t *v) \
> +{ \
> + __asm__ __volatile__( \
> + "am"#asm_op"_db.w" " $zero, %1, %0 \n" \
> + : "+ZB" (v->counter) \
> + : "r" (I) \
> + : "memory"); \
> +}
> +
> +#define ATOMIC_OP_RETURN(op, I, asm_op, c_op) \
> +static inline int arch_atomic_##op##_return_relaxed(int i, atomic_t *v) \
> +{ \
> + int result; \
> + \
> + __asm__ __volatile__( \
> + "am"#asm_op"_db.w" " %1, %2, %0 \n" \
> + : "+ZB" (v->counter), "=&r" (result) \
> + : "r" (I) \
> + : "memory"); \
> + \
> + return result c_op I; \
> +}
> +
> +#define ATOMIC_FETCH_OP(op, I, asm_op) \
> +static inline int arch_atomic_fetch_##op##_relaxed(int i, atomic_t *v) \
> +{ \
> + int result; \
> + \
> + __asm__ __volatile__( \
> + "am"#asm_op"_db.w" " %1, %2, %0 \n" \
> + : "+ZB" (v->counter), "=&r" (result) \
> + : "r" (I) \
> + : "memory"); \
> + \
> + return result; \
> +}
> +
> +#define ATOMIC_OPS(op, I, asm_op, c_op) \
> + ATOMIC_OP(op, I, asm_op) \
> + ATOMIC_OP_RETURN(op, I, asm_op, c_op) \
> + ATOMIC_FETCH_OP(op, I, asm_op)
> +
> +ATOMIC_OPS(add, i, add, +)
> +ATOMIC_OPS(sub, -i, add, +)
> +
> +#define arch_atomic_add_return_relaxed arch_atomic_add_return_relaxed
> +#define arch_atomic_sub_return_relaxed arch_atomic_sub_return_relaxed
> +#define arch_atomic_fetch_add_relaxed arch_atomic_fetch_add_relaxed
> +#define arch_atomic_fetch_sub_relaxed arch_atomic_fetch_sub_relaxed
> +
> +#undef ATOMIC_OPS
> +
> +#define ATOMIC_OPS(op, I, asm_op) \
> + ATOMIC_OP(op, I, asm_op) \
> + ATOMIC_FETCH_OP(op, I, asm_op)
> +
> +ATOMIC_OPS(and, i, and)
> +ATOMIC_OPS(or, i, or)
> +ATOMIC_OPS(xor, i, xor)
> +
> +#define arch_atomic_fetch_and_relaxed arch_atomic_fetch_and_relaxed
> +#define arch_atomic_fetch_or_relaxed arch_atomic_fetch_or_relaxed
> +#define arch_atomic_fetch_xor_relaxed arch_atomic_fetch_xor_relaxed
> +
> +#undef ATOMIC_OPS
> +#undef ATOMIC_FETCH_OP
> +#undef ATOMIC_OP_RETURN
> +#undef ATOMIC_OP
> +
> +static inline int arch_atomic_fetch_add_unless(atomic_t *v, int a, int u)
> +{
> + int prev, rc;
> +
> + __asm__ __volatile__ (
We need __WEAK_LLSC_MB here, to prevent previous access after sc.w.

> + "0: ll.w %[p], %[c]\n"
> + " beq %[p], %[u], 1f\n"
> + " add.w %[rc], %[p], %[a]\n"
> + " sc.w %[rc], %[c]\n"
> + " beqz %[rc], 0b\n"
> + " b 2f\n"
> + "1:\n"
> + __WEAK_LLSC_MB
> + "2:\n"
> + : [p]"=&r" (prev), [rc]"=&r" (rc),
> + [c]"=ZB" (v->counter)
> + : [a]"r" (a), [u]"r" (u)
> + : "memory");
> +
> + return prev;
> +}
> +#define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless
> +
> +/*
> + * arch_atomic_sub_if_positive - conditionally subtract integer from atomic variable
> + * @i: integer value to subtract
> + * @v: pointer of type atomic_t
> + *
> + * Atomically test @v and subtract @i if @v is greater or equal than @i.
> + * The function returns the old value of @v minus @i.
> + */
> +static inline int arch_atomic_sub_if_positive(int i, atomic_t *v)
> +{
> + int result;
> + int temp;
> +
> + if (__builtin_constant_p(i)) {
> + __asm__ __volatile__(
Ditto.
> + "1: ll.w %1, %2 # atomic_sub_if_positive\n"
> + " addi.w %0, %1, %3 \n"
> + " or %1, %0, $zero \n"
> + " blt %0, $zero, 2f \n"
> + " sc.w %1, %2 \n"
> + " beq $zero, %1, 1b \n"
Need __WEAK_LLSC_MB ?
> + "2: \n"
> + : "=&r" (result), "=&r" (temp),
> + "+" GCC_OFF_SMALL_ASM() (v->counter)
> + : "I" (-i));
> + } else {
> + __asm__ __volatile__(
Ditto.
> + "1: ll.w %1, %2 # atomic_sub_if_positive\n"
> + " sub.w %0, %1, %3 \n"
> + " or %1, %0, $zero \n"
> + " blt %0, $zero, 2f \n"
> + " sc.w %1, %2 \n"
Need __WEAK_LLSC_MB ?
> + " beq $zero, %1, 1b \n"
> + "2: \n"
> + : "=&r" (result), "=&r" (temp),
> + "+" GCC_OFF_SMALL_ASM() (v->counter)
> + : "r" (i));
> + }
> +
> + return result;
> +}
> +
> +#define arch_atomic_cmpxchg(v, o, n) (arch_cmpxchg(&((v)->counter), (o), (n)))
> +#define arch_atomic_xchg(v, new) (arch_xchg(&((v)->counter), (new)))
> +
> +/*
> + * arch_atomic_dec_if_positive - decrement by 1 if old value positive
> + * @v: pointer of type atomic_t
> + */
> +#define arch_atomic_dec_if_positive(v) arch_atomic_sub_if_positive(1, v)
> +
> +#ifdef CONFIG_64BIT
> +
> +#define ATOMIC64_INIT(i) { (i) }
> +
> +/*
> + * arch_atomic64_read - read atomic variable
> + * @v: pointer of type atomic64_t
> + *
> + */
> +#define arch_atomic64_read(v) READ_ONCE((v)->counter)
> +
> +/*
> + * arch_atomic64_set - set atomic variable
> + * @v: pointer of type atomic64_t
> + * @i: required value
> + */
> +#define arch_atomic64_set(v, i) WRITE_ONCE((v)->counter, (i))
> +
> +#define ATOMIC64_OP(op, I, asm_op) \
> +static inline void arch_atomic64_##op(long i, atomic64_t *v) \
> +{ \
> + __asm__ __volatile__( \
> + "am"#asm_op"_db.d " " $zero, %1, %0 \n" \
> + : "+ZB" (v->counter) \
> + : "r" (I) \
> + : "memory"); \
> +}
> +
> +#define ATOMIC64_OP_RETURN(op, I, asm_op, c_op) \
> +static inline long arch_atomic64_##op##_return_relaxed(long i, atomic64_t *v) \
> +{ \
> + long result; \
> + __asm__ __volatile__( \
> + "am"#asm_op"_db.d " " %1, %2, %0 \n" \
> + : "+ZB" (v->counter), "=&r" (result) \
> + : "r" (I) \
> + : "memory"); \
> + \
> + return result c_op I; \
> +}
> +
> +#define ATOMIC64_FETCH_OP(op, I, asm_op) \
> +static inline long arch_atomic64_fetch_##op##_relaxed(long i, atomic64_t *v) \
> +{ \
> + long result; \
> + \
> + __asm__ __volatile__( \
> + "am"#asm_op"_db.d " " %1, %2, %0 \n" \
> + : "+ZB" (v->counter), "=&r" (result) \
> + : "r" (I) \
> + : "memory"); \
> + \
> + return result; \
> +}
> +
> +#define ATOMIC64_OPS(op, I, asm_op, c_op) \
> + ATOMIC64_OP(op, I, asm_op) \
> + ATOMIC64_OP_RETURN(op, I, asm_op, c_op) \
> + ATOMIC64_FETCH_OP(op, I, asm_op)
> +
> +ATOMIC64_OPS(add, i, add, +)
> +ATOMIC64_OPS(sub, -i, add, +)
> +
> +#define arch_atomic64_add_return_relaxed arch_atomic64_add_return_relaxed
> +#define arch_atomic64_sub_return_relaxed arch_atomic64_sub_return_relaxed
> +#define arch_atomic64_fetch_add_relaxed arch_atomic64_fetch_add_relaxed
> +#define arch_atomic64_fetch_sub_relaxed arch_atomic64_fetch_sub_relaxed
> +
> +#undef ATOMIC64_OPS
> +
> +#define ATOMIC64_OPS(op, I, asm_op) \
> + ATOMIC64_OP(op, I, asm_op) \
> + ATOMIC64_FETCH_OP(op, I, asm_op)
> +
> +ATOMIC64_OPS(and, i, and)
> +ATOMIC64_OPS(or, i, or)
> +ATOMIC64_OPS(xor, i, xor)
> +
> +#define arch_atomic64_fetch_and_relaxed arch_atomic64_fetch_and_relaxed
> +#define arch_atomic64_fetch_or_relaxed arch_atomic64_fetch_or_relaxed
> +#define arch_atomic64_fetch_xor_relaxed arch_atomic64_fetch_xor_relaxed
> +
> +#undef ATOMIC64_OPS
> +#undef ATOMIC64_FETCH_OP
> +#undef ATOMIC64_OP_RETURN
> +#undef ATOMIC64_OP
> +
> +static inline long arch_atomic64_fetch_add_unless(atomic64_t *v, long a, long u)
> +{
> + long prev, rc;
> +
> + __asm__ __volatile__ (
Ditto
> + "0: ll.d %[p], %[c]\n"
> + " beq %[p], %[u], 1f\n"
> + " add.d %[rc], %[p], %[a]\n"
> + " sc.d %[rc], %[c]\n"
> + " beqz %[rc], 0b\n"
Ditto
> + " b 2f\n"
> + "1:\n"
> + __WEAK_LLSC_MB
> + "2:\n"
> + : [p]"=&r" (prev), [rc]"=&r" (rc),
> + [c] "=ZB" (v->counter)
> + : [a]"r" (a), [u]"r" (u)
> + : "memory");
> +
> + return prev;
> +}
> +#define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless
> +
> +/*
> + * arch_atomic64_sub_if_positive - conditionally subtract integer from atomic variable
> + * @i: integer value to subtract
> + * @v: pointer of type atomic64_t
> + *
> + * Atomically test @v and subtract @i if @v is greater or equal than @i.
> + * The function returns the old value of @v minus @i.
> + */
> +static inline long arch_atomic64_sub_if_positive(long i, atomic64_t *v)
> +{
> + long result;
> + long temp;
> +
> + if (__builtin_constant_p(i)) {
> + __asm__ __volatile__(
Ditto
> + "1: ll.d %1, %2 # atomic64_sub_if_positive \n"
> + " addi.d %0, %1, %3 \n"
> + " or %1, %0, $zero \n"
> + " blt %0, $zero, 2f \n"
> + " sc.d %1, %2 \n"
Ditto
> + " beq %1, $zero, 1b \n"
> + "2: \n"
> + : "=&r" (result), "=&r" (temp),
> + "+" GCC_OFF_SMALL_ASM() (v->counter)
> + : "I" (-i));
> + } else {
> + __asm__ __volatile__(
Ditto
> + "1: ll.d %1, %2 # atomic64_sub_if_positive \n"
> + " sub.d %0, %1, %3 \n"
> + " or %1, %0, $zero \n"
> + " blt %0, $zero, 2f \n"
> + " sc.d %1, %2 \n"
Ditto
> + " beq %1, $zero, 1b \n"
> + "2: \n"
> + : "=&r" (result), "=&r" (temp),
> + "+" GCC_OFF_SMALL_ASM() (v->counter)
> + : "r" (i));
> + }
> +
> + return result;
> +}
> +
> +#define arch_atomic64_cmpxchg(v, o, n) \
> + ((__typeof__((v)->counter))arch_cmpxchg(&((v)->counter), (o), (n)))
> +#define arch_atomic64_xchg(v, new) (arch_xchg(&((v)->counter), (new)))
> +
> +/*
> + * arch_atomic64_dec_if_positive - decrement by 1 if old value positive
> + * @v: pointer of type atomic64_t
> + */
> +#define arch_atomic64_dec_if_positive(v) arch_atomic64_sub_if_positive(1, v)
> +
> +#endif /* CONFIG_64BIT */
> +
> +#endif /* _ASM_ATOMIC_H */
> diff --git a/arch/loongarch/include/asm/barrier.h b/arch/loongarch/include/asm/barrier.h
> new file mode 100644
> index 000000000000..e57571bcaf4f
> --- /dev/null
> +++ b/arch/loongarch/include/asm/barrier.h
> @@ -0,0 +1,51 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef __ASM_BARRIER_H
> +#define __ASM_BARRIER_H
> +
> +#define __sync() __asm__ __volatile__("dbar 0" : : : "memory")
> +
> +#define fast_wmb() __sync()
> +#define fast_rmb() __sync()
> +#define fast_mb() __sync()
> +#define fast_iob() __sync()
> +#define wbflush() __sync()
> +
> +#define wmb() fast_wmb()
> +#define rmb() fast_rmb()
> +#define mb() fast_mb()
> +#define iob() fast_iob()
> +
> +/**
> + * array_index_mask_nospec() - generate a ~0 mask when index < size, 0 otherwise
> + * @index: array element index
> + * @size: number of elements in array
> + *
> + * Returns:
> + * 0 - (@index < @size)
> + */
> +#define array_index_mask_nospec array_index_mask_nospec
> +static inline unsigned long array_index_mask_nospec(unsigned long index,
> + unsigned long size)
> +{
> + unsigned long mask;
> +
> + __asm__ __volatile__(
> + "sltu %0, %1, %2\n\t"
> +#if (__SIZEOF_LONG__ == 4)
> + "sub.w %0, $r0, %0\n\t"
> +#elif (__SIZEOF_LONG__ == 8)
> + "sub.d %0, $r0, %0\n\t"
> +#endif
> + : "=r" (mask)
> + : "r" (index), "r" (size)
> + :);
> +
> + return mask;
> +}
> +
> +#include <asm-generic/barrier.h>
> +
> +#endif /* __ASM_BARRIER_H */
> diff --git a/arch/loongarch/include/asm/bitops.h b/arch/loongarch/include/asm/bitops.h
> new file mode 100644
> index 000000000000..69e00f8d8034
> --- /dev/null
> +++ b/arch/loongarch/include/asm/bitops.h
> @@ -0,0 +1,33 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef _ASM_BITOPS_H
> +#define _ASM_BITOPS_H
> +
> +#include <linux/compiler.h>
> +
> +#ifndef _LINUX_BITOPS_H
> +#error only <linux/bitops.h> can be included directly
> +#endif
> +
> +#include <asm/barrier.h>
> +
> +#include <asm-generic/bitops/builtin-ffs.h>
> +#include <asm-generic/bitops/builtin-fls.h>
> +#include <asm-generic/bitops/builtin-__ffs.h>
> +#include <asm-generic/bitops/builtin-__fls.h>
> +
> +#include <asm-generic/bitops/ffz.h>
> +#include <asm-generic/bitops/fls64.h>
> +
> +#include <asm-generic/bitops/sched.h>
> +#include <asm-generic/bitops/hweight.h>
> +
> +#include <asm-generic/bitops/atomic.h>
> +#include <asm-generic/bitops/non-atomic.h>
> +#include <asm-generic/bitops/lock.h>
> +#include <asm-generic/bitops/le.h>
> +#include <asm-generic/bitops/ext2-atomic.h>
> +
> +#endif /* _ASM_BITOPS_H */
> diff --git a/arch/loongarch/include/asm/bitrev.h b/arch/loongarch/include/asm/bitrev.h
> new file mode 100644
> index 000000000000..46f275b9cdf7
> --- /dev/null
> +++ b/arch/loongarch/include/asm/bitrev.h
> @@ -0,0 +1,34 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef __LOONGARCH_ASM_BITREV_H__
> +#define __LOONGARCH_ASM_BITREV_H__
> +
> +#include <linux/swab.h>
> +
> +static __always_inline __attribute_const__ u32 __arch_bitrev32(u32 x)
> +{
> + u32 ret;
> +
> + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(__swab32(x)));
> + return ret;
> +}
> +
> +static __always_inline __attribute_const__ u16 __arch_bitrev16(u16 x)
> +{
> + u16 ret;
> +
> + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(__swab16(x)));
> + return ret;
> +}
> +
> +static __always_inline __attribute_const__ u8 __arch_bitrev8(u8 x)
> +{
> + u8 ret;
> +
> + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(x));
> + return ret;
> +}
> +
> +#endif /* __LOONGARCH_ASM_BITREV_H__ */
> diff --git a/arch/loongarch/include/asm/cmpxchg.h b/arch/loongarch/include/asm/cmpxchg.h
> new file mode 100644
> index 000000000000..699925b8ff27
> --- /dev/null
> +++ b/arch/loongarch/include/asm/cmpxchg.h
> @@ -0,0 +1,121 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef __ASM_CMPXCHG_H
> +#define __ASM_CMPXCHG_H
> +
> +#include <linux/build_bug.h>
> +
> +#define __xchg_asm(amswap_db, m, val) \
> +({ \
> + __typeof(val) __ret; \
> + \
> + __asm__ __volatile__ ( \
> + " "amswap_db" %1, %z2, %0 \n" \
> + : "+ZB" (*m), "=&r" (__ret) \
> + : "Jr" (val) \
> + : "memory"); \
> + \
> + __ret; \
> +})
> +
> +static inline unsigned long __xchg(volatile void *ptr, unsigned long x,
relaxed ?

> + int size)
> +{
> + switch (size) {
> + case 4:
> + return __xchg_asm("amswap_db.w", (volatile u32 *)ptr, (u32)x);
Your amswap_db don't have any implicit acquire/release, right?

> +
> + case 8:
> + return __xchg_asm("amswap_db.d", (volatile u64 *)ptr, (u64)x);
> +
> + default:
> + BUILD_BUG();
> + }
> +
> + return 0;
> +}
> +
> +#define arch_xchg(ptr, x) \
> +({ \
> + __typeof__(*(ptr)) __res; \
> + \
> + __res = (__typeof__(*(ptr))) \
> + __xchg((ptr), (unsigned long)(x), sizeof(*(ptr))); \
> + \
> + __res; \
> +})
> +
> +#define __cmpxchg_asm(ld, st, m, old, new) \
> +({ \
> + __typeof(old) __ret; \
> + \
> + __asm__ __volatile__( \
> + "1: " ld " %0, %2 # __cmpxchg_asm \n" \
> + " bne %0, %z3, 2f \n" \
> + " or $t0, %z4, $zero \n" \
> + " " st " $t0, %1 \n" \
> + " beq $zero, $t0, 1b \n" \
> + "2: \n" \
> + : "=&r" (__ret), "=ZB"(*m) \
> + : "ZB"(*m), "Jr" (old), "Jr" (new) \
> + : "t0", "memory"); \
> + \
> + __ret; \
> +})
> +
> +static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
> + unsigned long new, unsigned int size)
> +{
> + switch (size) {
> + case 4:
> + return __cmpxchg_asm("ll.w", "sc.w", (volatile u32 *)ptr,
> + (u32)old, new);
> +
> + case 8:
> + return __cmpxchg_asm("ll.d", "sc.d", (volatile u64 *)ptr,
> + (u64)old, new);
> +
> + default:
> + BUILD_BUG();
> + }
> +
> + return 0;
> +}
> +
> +#define arch_cmpxchg_local(ptr, old, new) \
> + ((__typeof__(*(ptr))) \
> + __cmpxchg((ptr), \
> + (unsigned long)(__typeof__(*(ptr)))(old), \
> + (unsigned long)(__typeof__(*(ptr)))(new), \
> + sizeof(*(ptr))))
> +
> +#define arch_cmpxchg(ptr, old, new) \
Ditto, it's a relaxed implementation, right?
> +({ \
> + __typeof__(*(ptr)) __res; \
> + \
> + __res = arch_cmpxchg_local((ptr), (old), (new)); \
> + \
> + __res; \
> +})
> +
> +#ifdef CONFIG_64BIT
> +#define arch_cmpxchg64_local(ptr, o, n) \
> + ({ \
> + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \
> + arch_cmpxchg_local((ptr), (o), (n)); \
> + })
> +
> +#define arch_cmpxchg64(ptr, o, n) \
> + ({ \
> + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \
> + arch_cmpxchg((ptr), (o), (n)); \
> + })
> +#else
> +#include <asm-generic/cmpxchg-local.h>
> +#define arch_cmpxchg64_local(ptr, o, n) __generic_cmpxchg64_local((ptr), (o), (n))
> +#define arch_cmpxchg64(ptr, o, n) arch_cmpxchg64_local((ptr), (o), (n))
> +#endif
> +
> +#endif /* __ASM_CMPXCHG_H */
> diff --git a/arch/loongarch/include/asm/local.h b/arch/loongarch/include/asm/local.h
> new file mode 100644
> index 000000000000..2052a2267337
> --- /dev/null
> +++ b/arch/loongarch/include/asm/local.h
> @@ -0,0 +1,138 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef _ARCH_LOONGARCH_LOCAL_H
> +#define _ARCH_LOONGARCH_LOCAL_H
> +
> +#include <linux/percpu.h>
> +#include <linux/bitops.h>
> +#include <linux/atomic.h>
> +#include <asm/cmpxchg.h>
> +#include <asm/compiler.h>
> +
> +typedef struct {
> + atomic_long_t a;
> +} local_t;
> +
> +#define LOCAL_INIT(i) { ATOMIC_LONG_INIT(i) }
> +
> +#define local_read(l) atomic_long_read(&(l)->a)
> +#define local_set(l, i) atomic_long_set(&(l)->a, (i))
> +
> +#define local_add(i, l) atomic_long_add((i), (&(l)->a))
> +#define local_sub(i, l) atomic_long_sub((i), (&(l)->a))
> +#define local_inc(l) atomic_long_inc(&(l)->a)
> +#define local_dec(l) atomic_long_dec(&(l)->a)
> +
> +/*
> + * Same as above, but return the result value
> + */
> +static inline long local_add_return(long i, local_t *l)
> +{
> + unsigned long result;
> +
> + __asm__ __volatile__(
> + " " __AMADD " %1, %2, %0 \n"
> + : "+ZB" (l->a.counter), "=&r" (result)
> + : "r" (i)
> + : "memory");
> + result = result + i;
> +
> + return result;
> +}
> +
> +static inline long local_sub_return(long i, local_t *l)
> +{
> + unsigned long result;
> +
> + __asm__ __volatile__(
> + " " __AMADD "%1, %2, %0 \n"
> + : "+ZB" (l->a.counter), "=&r" (result)
> + : "r" (-i)
> + : "memory");
> +
> + result = result - i;
> +
> + return result;
> +}
> +
> +#define local_cmpxchg(l, o, n) \
> + ((long)cmpxchg_local(&((l)->a.counter), (o), (n)))
> +#define local_xchg(l, n) (atomic_long_xchg((&(l)->a), (n)))
> +
> +/**
> + * local_add_unless - add unless the number is a given value
> + * @l: pointer of type local_t
> + * @a: the amount to add to l...
> + * @u: ...unless l is equal to u.
> + *
> + * Atomically adds @a to @l, so long as it was not @u.
> + * Returns non-zero if @l was not @u, and zero otherwise.
> + */
> +#define local_add_unless(l, a, u) \
> +({ \
> + long c, old; \
> + c = local_read(l); \
> + while (c != (u) && (old = local_cmpxchg((l), c, c + (a))) != c) \
> + c = old; \
> + c != (u); \
> +})
> +#define local_inc_not_zero(l) local_add_unless((l), 1, 0)
> +
> +#define local_dec_return(l) local_sub_return(1, (l))
> +#define local_inc_return(l) local_add_return(1, (l))
> +
> +/*
> + * local_sub_and_test - subtract value from variable and test result
> + * @i: integer value to subtract
> + * @l: pointer of type local_t
> + *
> + * Atomically subtracts @i from @l and returns
> + * true if the result is zero, or false for all
> + * other cases.
> + */
> +#define local_sub_and_test(i, l) (local_sub_return((i), (l)) == 0)
> +
> +/*
> + * local_inc_and_test - increment and test
> + * @l: pointer of type local_t
> + *
> + * Atomically increments @l by 1
> + * and returns true if the result is zero, or false for all
> + * other cases.
> + */
> +#define local_inc_and_test(l) (local_inc_return(l) == 0)
> +
> +/*
> + * local_dec_and_test - decrement by 1 and test
> + * @l: pointer of type local_t
> + *
> + * Atomically decrements @l by 1 and
> + * returns true if the result is 0, or false for all other
> + * cases.
> + */
> +#define local_dec_and_test(l) (local_sub_return(1, (l)) == 0)
> +
> +/*
> + * local_add_negative - add and test if negative
> + * @l: pointer of type local_t
> + * @i: integer value to add
> + *
> + * Atomically adds @i to @l and returns true
> + * if the result is negative, or false when
> + * result is greater than or equal to zero.
> + */
> +#define local_add_negative(i, l) (local_add_return(i, (l)) < 0)
> +
> +/* Use these for per-cpu local_t variables: on some archs they are
> + * much more efficient than these naive implementations. Note they take
> + * a variable, not an address.
> + */
> +
> +#define __local_inc(l) ((l)->a.counter++)
> +#define __local_dec(l) ((l)->a.counter++)
> +#define __local_add(i, l) ((l)->a.counter += (i))
> +#define __local_sub(i, l) ((l)->a.counter -= (i))
> +
> +#endif /* _ARCH_LOONGARCH_LOCAL_H */
> diff --git a/arch/loongarch/include/asm/percpu.h b/arch/loongarch/include/asm/percpu.h
> new file mode 100644
> index 000000000000..7d5b22ebd834
> --- /dev/null
> +++ b/arch/loongarch/include/asm/percpu.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> + */
> +#ifndef __ASM_PERCPU_H
> +#define __ASM_PERCPU_H
> +
> +/* Use r21 for fast access */
> +register unsigned long __my_cpu_offset __asm__("$r21");
> +
> +static inline void set_my_cpu_offset(unsigned long off)
> +{
> + __my_cpu_offset = off;
> + csr_writeq(off, PERCPU_BASE_KS);
> +}
> +#define __my_cpu_offset __my_cpu_offset
> +
> +#include <asm-generic/percpu.h>
> +
> +#endif /* __ASM_PERCPU_H */
> --
> 2.27.0
>


--
Best Regards
Guo Ren

ML: https://lore.kernel.org/linux-csky/

2022-05-22 14:24:06

by Huacai Chen

[permalink] [raw]
Subject: Re: [PATCH V11 07/22] LoongArch: Add atomic/locking headers

Hi, Ren,

On Sat, May 21, 2022 at 2:15 AM Guo Ren <[email protected]> wrote:
>
> On Fri, May 20, 2022 at 5:50 PM Huacai Chen <[email protected]> wrote:
> >
> > Hi, Ren,
> >
> > On Fri, May 20, 2022 at 3:54 PM Guo Ren <[email protected]> wrote:
> > >
> > > On Wed, May 18, 2022 at 5:34 PM Huacai Chen <[email protected]> wrote:
> > > >
> > > > Add common headers (atomic, bitops, barrier and locking) for basic
> > > > LoongArch support.
> > > >
> > > > Signed-off-by: Huacai Chen <[email protected]>
> > > > ---
> > > > arch/loongarch/include/asm/atomic.h | 358 +++++++++++++++++++++++++++
> > > > arch/loongarch/include/asm/barrier.h | 51 ++++
> > > > arch/loongarch/include/asm/bitops.h | 33 +++
> > > > arch/loongarch/include/asm/bitrev.h | 34 +++
> > > > arch/loongarch/include/asm/cmpxchg.h | 121 +++++++++
> > > > arch/loongarch/include/asm/local.h | 138 +++++++++++
> > > > arch/loongarch/include/asm/percpu.h | 20 ++
> > > > 7 files changed, 755 insertions(+)
> > > > create mode 100644 arch/loongarch/include/asm/atomic.h
> > > > create mode 100644 arch/loongarch/include/asm/barrier.h
> > > > create mode 100644 arch/loongarch/include/asm/bitops.h
> > > > create mode 100644 arch/loongarch/include/asm/bitrev.h
> > > > create mode 100644 arch/loongarch/include/asm/cmpxchg.h
> > > > create mode 100644 arch/loongarch/include/asm/local.h
> > > > create mode 100644 arch/loongarch/include/asm/percpu.h
> > > >
> > > > diff --git a/arch/loongarch/include/asm/atomic.h b/arch/loongarch/include/asm/atomic.h
> > > > new file mode 100644
> > > > index 000000000000..932352342b12
> > > > --- /dev/null
> > > > +++ b/arch/loongarch/include/asm/atomic.h
> > > > @@ -0,0 +1,358 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > +/*
> > > > + * Atomic operations.
> > > > + *
> > > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > > + */
> > > > +#ifndef _ASM_ATOMIC_H
> > > > +#define _ASM_ATOMIC_H
> > > > +
> > > > +#include <linux/types.h>
> > > > +#include <asm/barrier.h>
> > > > +#include <asm/cmpxchg.h>
> > > > +#include <asm/compiler.h>
> > > > +
> > > > +#if __SIZEOF_LONG__ == 4
> > > > +#define __LL "ll.w "
> > > > +#define __SC "sc.w "
> > > > +#define __AMADD "amadd.w "
> > > > +#define __AMAND_DB "amand_db.w "
> > > > +#define __AMOR_DB "amor_db.w "
> > > > +#define __AMXOR_DB "amxor_db.w "
> > > > +#elif __SIZEOF_LONG__ == 8
> > > > +#define __LL "ll.d "
> > > > +#define __SC "sc.d "
> > > > +#define __AMADD "amadd.d "
> > > > +#define __AMAND_DB "amand_db.d "
> > > > +#define __AMOR_DB "amor_db.d "
> > > > +#define __AMXOR_DB "amxor_db.d "
> > > > +#endif
> > > > +
> > > > +#define ATOMIC_INIT(i) { (i) }
> > > > +
> > > > +/*
> > > > + * arch_atomic_read - read atomic variable
> > > > + * @v: pointer of type atomic_t
> > > > + *
> > > > + * Atomically reads the value of @v.
> > > > + */
> > > > +#define arch_atomic_read(v) READ_ONCE((v)->counter)
> > > > +
> > > > +/*
> > > > + * arch_atomic_set - set atomic variable
> > > > + * @v: pointer of type atomic_t
> > > > + * @i: required value
> > > > + *
> > > > + * Atomically sets the value of @v to @i.
> > > > + */
> > > > +#define arch_atomic_set(v, i) WRITE_ONCE((v)->counter, (i))
> > > > +
> > > > +#define ATOMIC_OP(op, I, asm_op) \
> > > > +static inline void arch_atomic_##op(int i, atomic_t *v) \
> > > > +{ \
> > > > + __asm__ __volatile__( \
> > > > + "am"#asm_op"_db.w" " $zero, %1, %0 \n" \
> > > > + : "+ZB" (v->counter) \
> > > > + : "r" (I) \
> > > > + : "memory"); \
> > > > +}
> > > > +
> > > > +#define ATOMIC_OP_RETURN(op, I, asm_op, c_op) \
> > > > +static inline int arch_atomic_##op##_return_relaxed(int i, atomic_t *v) \
> > > > +{ \
> > > > + int result; \
> > > > + \
> > > > + __asm__ __volatile__( \
> > > > + "am"#asm_op"_db.w" " %1, %2, %0 \n" \
> > > > + : "+ZB" (v->counter), "=&r" (result) \
> > > > + : "r" (I) \
> > > > + : "memory"); \
> > > > + \
> > > > + return result c_op I; \
> > > > +}
> > > > +
> > > > +#define ATOMIC_FETCH_OP(op, I, asm_op) \
> > > > +static inline int arch_atomic_fetch_##op##_relaxed(int i, atomic_t *v) \
> > > > +{ \
> > > > + int result; \
> > > > + \
> > > > + __asm__ __volatile__( \
> > > > + "am"#asm_op"_db.w" " %1, %2, %0 \n" \
> > > > + : "+ZB" (v->counter), "=&r" (result) \
> > > > + : "r" (I) \
> > > > + : "memory"); \
> > > > + \
> > > > + return result; \
> > > > +}
> > > > +
> > > > +#define ATOMIC_OPS(op, I, asm_op, c_op) \
> > > > + ATOMIC_OP(op, I, asm_op) \
> > > > + ATOMIC_OP_RETURN(op, I, asm_op, c_op) \
> > > > + ATOMIC_FETCH_OP(op, I, asm_op)
> > > > +
> > > > +ATOMIC_OPS(add, i, add, +)
> > > > +ATOMIC_OPS(sub, -i, add, +)
> > > > +
> > > > +#define arch_atomic_add_return_relaxed arch_atomic_add_return_relaxed
> > > > +#define arch_atomic_sub_return_relaxed arch_atomic_sub_return_relaxed
> > > > +#define arch_atomic_fetch_add_relaxed arch_atomic_fetch_add_relaxed
> > > > +#define arch_atomic_fetch_sub_relaxed arch_atomic_fetch_sub_relaxed
> > > > +
> > > > +#undef ATOMIC_OPS
> > > > +
> > > > +#define ATOMIC_OPS(op, I, asm_op) \
> > > > + ATOMIC_OP(op, I, asm_op) \
> > > > + ATOMIC_FETCH_OP(op, I, asm_op)
> > > > +
> > > > +ATOMIC_OPS(and, i, and)
> > > > +ATOMIC_OPS(or, i, or)
> > > > +ATOMIC_OPS(xor, i, xor)
> > > > +
> > > > +#define arch_atomic_fetch_and_relaxed arch_atomic_fetch_and_relaxed
> > > > +#define arch_atomic_fetch_or_relaxed arch_atomic_fetch_or_relaxed
> > > > +#define arch_atomic_fetch_xor_relaxed arch_atomic_fetch_xor_relaxed
> > > > +
> > > > +#undef ATOMIC_OPS
> > > > +#undef ATOMIC_FETCH_OP
> > > > +#undef ATOMIC_OP_RETURN
> > > > +#undef ATOMIC_OP
> > > > +
> > > > +static inline int arch_atomic_fetch_add_unless(atomic_t *v, int a, int u)
> > > > +{
> > > > + int prev, rc;
> > > > +
> > > > + __asm__ __volatile__ (
> > > We need __WEAK_LLSC_MB here, to prevent previous access after sc.w.
> > LoongArch's ll and sc has implied memory barrier. :)
>
> But why do you put __WEAK_LLSC_MB after sc ?
We don't need a barrier before ll, and don't need a barrier after sc,
but we need a barrier when the code goes out of the ll-sc region.

Huacai
>
> >
> > >
> > > > + "0: ll.w %[p], %[c]\n"
> > > > + " beq %[p], %[u], 1f\n"
> > > > + " add.w %[rc], %[p], %[a]\n"
> > > > + " sc.w %[rc], %[c]\n"
> > > > + " beqz %[rc], 0b\n"
> > > > + " b 2f\n"
> > > > + "1:\n"
> > > > + __WEAK_LLSC_MB
> > > > + "2:\n"
> > > > + : [p]"=&r" (prev), [rc]"=&r" (rc),
> > > > + [c]"=ZB" (v->counter)
> > > > + : [a]"r" (a), [u]"r" (u)
> > > > + : "memory");
> > > > +
> > > > + return prev;
> > > > +}
> > > > +#define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless
> > > > +
> > > > +/*
> > > > + * arch_atomic_sub_if_positive - conditionally subtract integer from atomic variable
> > > > + * @i: integer value to subtract
> > > > + * @v: pointer of type atomic_t
> > > > + *
> > > > + * Atomically test @v and subtract @i if @v is greater or equal than @i.
> > > > + * The function returns the old value of @v minus @i.
> > > > + */
> > > > +static inline int arch_atomic_sub_if_positive(int i, atomic_t *v)
> > > > +{
> > > > + int result;
> > > > + int temp;
> > > > +
> > > > + if (__builtin_constant_p(i)) {
> > > > + __asm__ __volatile__(
> > > Ditto.
> > > > + "1: ll.w %1, %2 # atomic_sub_if_positive\n"
> > > > + " addi.w %0, %1, %3 \n"
> > > > + " or %1, %0, $zero \n"
> > > > + " blt %0, $zero, 2f \n"
> > > > + " sc.w %1, %2 \n"
> > > > + " beq $zero, %1, 1b \n"
> > > Need __WEAK_LLSC_MB ?
> > > > + "2: \n"
> > > > + : "=&r" (result), "=&r" (temp),
> > > > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > > > + : "I" (-i));
> > > > + } else {
> > > > + __asm__ __volatile__(
> > > Ditto.
> > > > + "1: ll.w %1, %2 # atomic_sub_if_positive\n"
> > > > + " sub.w %0, %1, %3 \n"
> > > > + " or %1, %0, $zero \n"
> > > > + " blt %0, $zero, 2f \n"
> > > > + " sc.w %1, %2 \n"
> > > Need __WEAK_LLSC_MB ?
> > > > + " beq $zero, %1, 1b \n"
> > > > + "2: \n"
> > > > + : "=&r" (result), "=&r" (temp),
> > > > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > > > + : "r" (i));
> > > > + }
> > > > +
> > > > + return result;
> > > > +}
> > > > +
> > > > +#define arch_atomic_cmpxchg(v, o, n) (arch_cmpxchg(&((v)->counter), (o), (n)))
> > > > +#define arch_atomic_xchg(v, new) (arch_xchg(&((v)->counter), (new)))
> > > > +
> > > > +/*
> > > > + * arch_atomic_dec_if_positive - decrement by 1 if old value positive
> > > > + * @v: pointer of type atomic_t
> > > > + */
> > > > +#define arch_atomic_dec_if_positive(v) arch_atomic_sub_if_positive(1, v)
> > > > +
> > > > +#ifdef CONFIG_64BIT
> > > > +
> > > > +#define ATOMIC64_INIT(i) { (i) }
> > > > +
> > > > +/*
> > > > + * arch_atomic64_read - read atomic variable
> > > > + * @v: pointer of type atomic64_t
> > > > + *
> > > > + */
> > > > +#define arch_atomic64_read(v) READ_ONCE((v)->counter)
> > > > +
> > > > +/*
> > > > + * arch_atomic64_set - set atomic variable
> > > > + * @v: pointer of type atomic64_t
> > > > + * @i: required value
> > > > + */
> > > > +#define arch_atomic64_set(v, i) WRITE_ONCE((v)->counter, (i))
> > > > +
> > > > +#define ATOMIC64_OP(op, I, asm_op) \
> > > > +static inline void arch_atomic64_##op(long i, atomic64_t *v) \
> > > > +{ \
> > > > + __asm__ __volatile__( \
> > > > + "am"#asm_op"_db.d " " $zero, %1, %0 \n" \
> > > > + : "+ZB" (v->counter) \
> > > > + : "r" (I) \
> > > > + : "memory"); \
> > > > +}
> > > > +
> > > > +#define ATOMIC64_OP_RETURN(op, I, asm_op, c_op) \
> > > > +static inline long arch_atomic64_##op##_return_relaxed(long i, atomic64_t *v) \
> > > > +{ \
> > > > + long result; \
> > > > + __asm__ __volatile__( \
> > > > + "am"#asm_op"_db.d " " %1, %2, %0 \n" \
> > > > + : "+ZB" (v->counter), "=&r" (result) \
> > > > + : "r" (I) \
> > > > + : "memory"); \
> > > > + \
> > > > + return result c_op I; \
> > > > +}
> > > > +
> > > > +#define ATOMIC64_FETCH_OP(op, I, asm_op) \
> > > > +static inline long arch_atomic64_fetch_##op##_relaxed(long i, atomic64_t *v) \
> > > > +{ \
> > > > + long result; \
> > > > + \
> > > > + __asm__ __volatile__( \
> > > > + "am"#asm_op"_db.d " " %1, %2, %0 \n" \
> > > > + : "+ZB" (v->counter), "=&r" (result) \
> > > > + : "r" (I) \
> > > > + : "memory"); \
> > > > + \
> > > > + return result; \
> > > > +}
> > > > +
> > > > +#define ATOMIC64_OPS(op, I, asm_op, c_op) \
> > > > + ATOMIC64_OP(op, I, asm_op) \
> > > > + ATOMIC64_OP_RETURN(op, I, asm_op, c_op) \
> > > > + ATOMIC64_FETCH_OP(op, I, asm_op)
> > > > +
> > > > +ATOMIC64_OPS(add, i, add, +)
> > > > +ATOMIC64_OPS(sub, -i, add, +)
> > > > +
> > > > +#define arch_atomic64_add_return_relaxed arch_atomic64_add_return_relaxed
> > > > +#define arch_atomic64_sub_return_relaxed arch_atomic64_sub_return_relaxed
> > > > +#define arch_atomic64_fetch_add_relaxed arch_atomic64_fetch_add_relaxed
> > > > +#define arch_atomic64_fetch_sub_relaxed arch_atomic64_fetch_sub_relaxed
> > > > +
> > > > +#undef ATOMIC64_OPS
> > > > +
> > > > +#define ATOMIC64_OPS(op, I, asm_op) \
> > > > + ATOMIC64_OP(op, I, asm_op) \
> > > > + ATOMIC64_FETCH_OP(op, I, asm_op)
> > > > +
> > > > +ATOMIC64_OPS(and, i, and)
> > > > +ATOMIC64_OPS(or, i, or)
> > > > +ATOMIC64_OPS(xor, i, xor)
> > > > +
> > > > +#define arch_atomic64_fetch_and_relaxed arch_atomic64_fetch_and_relaxed
> > > > +#define arch_atomic64_fetch_or_relaxed arch_atomic64_fetch_or_relaxed
> > > > +#define arch_atomic64_fetch_xor_relaxed arch_atomic64_fetch_xor_relaxed
> > > > +
> > > > +#undef ATOMIC64_OPS
> > > > +#undef ATOMIC64_FETCH_OP
> > > > +#undef ATOMIC64_OP_RETURN
> > > > +#undef ATOMIC64_OP
> > > > +
> > > > +static inline long arch_atomic64_fetch_add_unless(atomic64_t *v, long a, long u)
> > > > +{
> > > > + long prev, rc;
> > > > +
> > > > + __asm__ __volatile__ (
> > > Ditto
> > > > + "0: ll.d %[p], %[c]\n"
> > > > + " beq %[p], %[u], 1f\n"
> > > > + " add.d %[rc], %[p], %[a]\n"
> > > > + " sc.d %[rc], %[c]\n"
> > > > + " beqz %[rc], 0b\n"
> > > Ditto
> > > > + " b 2f\n"
> > > > + "1:\n"
> > > > + __WEAK_LLSC_MB
> > > > + "2:\n"
> > > > + : [p]"=&r" (prev), [rc]"=&r" (rc),
> > > > + [c] "=ZB" (v->counter)
> > > > + : [a]"r" (a), [u]"r" (u)
> > > > + : "memory");
> > > > +
> > > > + return prev;
> > > > +}
> > > > +#define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless
> > > > +
> > > > +/*
> > > > + * arch_atomic64_sub_if_positive - conditionally subtract integer from atomic variable
> > > > + * @i: integer value to subtract
> > > > + * @v: pointer of type atomic64_t
> > > > + *
> > > > + * Atomically test @v and subtract @i if @v is greater or equal than @i.
> > > > + * The function returns the old value of @v minus @i.
> > > > + */
> > > > +static inline long arch_atomic64_sub_if_positive(long i, atomic64_t *v)
> > > > +{
> > > > + long result;
> > > > + long temp;
> > > > +
> > > > + if (__builtin_constant_p(i)) {
> > > > + __asm__ __volatile__(
> > > Ditto
> > > > + "1: ll.d %1, %2 # atomic64_sub_if_positive \n"
> > > > + " addi.d %0, %1, %3 \n"
> > > > + " or %1, %0, $zero \n"
> > > > + " blt %0, $zero, 2f \n"
> > > > + " sc.d %1, %2 \n"
> > > Ditto
> > > > + " beq %1, $zero, 1b \n"
> > > > + "2: \n"
> > > > + : "=&r" (result), "=&r" (temp),
> > > > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > > > + : "I" (-i));
> > > > + } else {
> > > > + __asm__ __volatile__(
> > > Ditto
> > > > + "1: ll.d %1, %2 # atomic64_sub_if_positive \n"
> > > > + " sub.d %0, %1, %3 \n"
> > > > + " or %1, %0, $zero \n"
> > > > + " blt %0, $zero, 2f \n"
> > > > + " sc.d %1, %2 \n"
> > > Ditto
> > > > + " beq %1, $zero, 1b \n"
> > > > + "2: \n"
> > > > + : "=&r" (result), "=&r" (temp),
> > > > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > > > + : "r" (i));
> > > > + }
> > > > +
> > > > + return result;
> > > > +}
> > > > +
> > > > +#define arch_atomic64_cmpxchg(v, o, n) \
> > > > + ((__typeof__((v)->counter))arch_cmpxchg(&((v)->counter), (o), (n)))
> > > > +#define arch_atomic64_xchg(v, new) (arch_xchg(&((v)->counter), (new)))
> > > > +
> > > > +/*
> > > > + * arch_atomic64_dec_if_positive - decrement by 1 if old value positive
> > > > + * @v: pointer of type atomic64_t
> > > > + */
> > > > +#define arch_atomic64_dec_if_positive(v) arch_atomic64_sub_if_positive(1, v)
> > > > +
> > > > +#endif /* CONFIG_64BIT */
> > > > +
> > > > +#endif /* _ASM_ATOMIC_H */
> > > > diff --git a/arch/loongarch/include/asm/barrier.h b/arch/loongarch/include/asm/barrier.h
> > > > new file mode 100644
> > > > index 000000000000..e57571bcaf4f
> > > > --- /dev/null
> > > > +++ b/arch/loongarch/include/asm/barrier.h
> > > > @@ -0,0 +1,51 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > +/*
> > > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > > + */
> > > > +#ifndef __ASM_BARRIER_H
> > > > +#define __ASM_BARRIER_H
> > > > +
> > > > +#define __sync() __asm__ __volatile__("dbar 0" : : : "memory")
> > > > +
> > > > +#define fast_wmb() __sync()
> > > > +#define fast_rmb() __sync()
> > > > +#define fast_mb() __sync()
> > > > +#define fast_iob() __sync()
> > > > +#define wbflush() __sync()
> > > > +
> > > > +#define wmb() fast_wmb()
> > > > +#define rmb() fast_rmb()
> > > > +#define mb() fast_mb()
> > > > +#define iob() fast_iob()
> > > > +
> > > > +/**
> > > > + * array_index_mask_nospec() - generate a ~0 mask when index < size, 0 otherwise
> > > > + * @index: array element index
> > > > + * @size: number of elements in array
> > > > + *
> > > > + * Returns:
> > > > + * 0 - (@index < @size)
> > > > + */
> > > > +#define array_index_mask_nospec array_index_mask_nospec
> > > > +static inline unsigned long array_index_mask_nospec(unsigned long index,
> > > > + unsigned long size)
> > > > +{
> > > > + unsigned long mask;
> > > > +
> > > > + __asm__ __volatile__(
> > > > + "sltu %0, %1, %2\n\t"
> > > > +#if (__SIZEOF_LONG__ == 4)
> > > > + "sub.w %0, $r0, %0\n\t"
> > > > +#elif (__SIZEOF_LONG__ == 8)
> > > > + "sub.d %0, $r0, %0\n\t"
> > > > +#endif
> > > > + : "=r" (mask)
> > > > + : "r" (index), "r" (size)
> > > > + :);
> > > > +
> > > > + return mask;
> > > > +}
> > > > +
> > > > +#include <asm-generic/barrier.h>
> > > > +
> > > > +#endif /* __ASM_BARRIER_H */
> > > > diff --git a/arch/loongarch/include/asm/bitops.h b/arch/loongarch/include/asm/bitops.h
> > > > new file mode 100644
> > > > index 000000000000..69e00f8d8034
> > > > --- /dev/null
> > > > +++ b/arch/loongarch/include/asm/bitops.h
> > > > @@ -0,0 +1,33 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > +/*
> > > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > > + */
> > > > +#ifndef _ASM_BITOPS_H
> > > > +#define _ASM_BITOPS_H
> > > > +
> > > > +#include <linux/compiler.h>
> > > > +
> > > > +#ifndef _LINUX_BITOPS_H
> > > > +#error only <linux/bitops.h> can be included directly
> > > > +#endif
> > > > +
> > > > +#include <asm/barrier.h>
> > > > +
> > > > +#include <asm-generic/bitops/builtin-ffs.h>
> > > > +#include <asm-generic/bitops/builtin-fls.h>
> > > > +#include <asm-generic/bitops/builtin-__ffs.h>
> > > > +#include <asm-generic/bitops/builtin-__fls.h>
> > > > +
> > > > +#include <asm-generic/bitops/ffz.h>
> > > > +#include <asm-generic/bitops/fls64.h>
> > > > +
> > > > +#include <asm-generic/bitops/sched.h>
> > > > +#include <asm-generic/bitops/hweight.h>
> > > > +
> > > > +#include <asm-generic/bitops/atomic.h>
> > > > +#include <asm-generic/bitops/non-atomic.h>
> > > > +#include <asm-generic/bitops/lock.h>
> > > > +#include <asm-generic/bitops/le.h>
> > > > +#include <asm-generic/bitops/ext2-atomic.h>
> > > > +
> > > > +#endif /* _ASM_BITOPS_H */
> > > > diff --git a/arch/loongarch/include/asm/bitrev.h b/arch/loongarch/include/asm/bitrev.h
> > > > new file mode 100644
> > > > index 000000000000..46f275b9cdf7
> > > > --- /dev/null
> > > > +++ b/arch/loongarch/include/asm/bitrev.h
> > > > @@ -0,0 +1,34 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > +/*
> > > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > > + */
> > > > +#ifndef __LOONGARCH_ASM_BITREV_H__
> > > > +#define __LOONGARCH_ASM_BITREV_H__
> > > > +
> > > > +#include <linux/swab.h>
> > > > +
> > > > +static __always_inline __attribute_const__ u32 __arch_bitrev32(u32 x)
> > > > +{
> > > > + u32 ret;
> > > > +
> > > > + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(__swab32(x)));
> > > > + return ret;
> > > > +}
> > > > +
> > > > +static __always_inline __attribute_const__ u16 __arch_bitrev16(u16 x)
> > > > +{
> > > > + u16 ret;
> > > > +
> > > > + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(__swab16(x)));
> > > > + return ret;
> > > > +}
> > > > +
> > > > +static __always_inline __attribute_const__ u8 __arch_bitrev8(u8 x)
> > > > +{
> > > > + u8 ret;
> > > > +
> > > > + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(x));
> > > > + return ret;
> > > > +}
> > > > +
> > > > +#endif /* __LOONGARCH_ASM_BITREV_H__ */
> > > > diff --git a/arch/loongarch/include/asm/cmpxchg.h b/arch/loongarch/include/asm/cmpxchg.h
> > > > new file mode 100644
> > > > index 000000000000..699925b8ff27
> > > > --- /dev/null
> > > > +++ b/arch/loongarch/include/asm/cmpxchg.h
> > > > @@ -0,0 +1,121 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > +/*
> > > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > > + */
> > > > +#ifndef __ASM_CMPXCHG_H
> > > > +#define __ASM_CMPXCHG_H
> > > > +
> > > > +#include <linux/build_bug.h>
> > > > +
> > > > +#define __xchg_asm(amswap_db, m, val) \
> > > > +({ \
> > > > + __typeof(val) __ret; \
> > > > + \
> > > > + __asm__ __volatile__ ( \
> > > > + " "amswap_db" %1, %z2, %0 \n" \
> > > > + : "+ZB" (*m), "=&r" (__ret) \
> > > > + : "Jr" (val) \
> > > > + : "memory"); \
> > > > + \
> > > > + __ret; \
> > > > +})
> > > > +
> > > > +static inline unsigned long __xchg(volatile void *ptr, unsigned long x,
> > > relaxed ?
> > >
> > > > + int size)
> > > > +{
> > > > + switch (size) {
> > > > + case 4:
> > > > + return __xchg_asm("amswap_db.w", (volatile u32 *)ptr, (u32)x);
> > > Your amswap_db don't have any implicit acquire/release, right?
> > amswap don't have implicit acquire/release, but amswap_db has.
> >
> > Huacai
> > >
> > > > +
> > > > + case 8:
> > > > + return __xchg_asm("amswap_db.d", (volatile u64 *)ptr, (u64)x);
> > > > +
> > > > + default:
> > > > + BUILD_BUG();
> > > > + }
> > > > +
> > > > + return 0;
> > > > +}
> > > > +
> > > > +#define arch_xchg(ptr, x) \
> > > > +({ \
> > > > + __typeof__(*(ptr)) __res; \
> > > > + \
> > > > + __res = (__typeof__(*(ptr))) \
> > > > + __xchg((ptr), (unsigned long)(x), sizeof(*(ptr))); \
> > > > + \
> > > > + __res; \
> > > > +})
> > > > +
> > > > +#define __cmpxchg_asm(ld, st, m, old, new) \
> > > > +({ \
> > > > + __typeof(old) __ret; \
> > > > + \
> > > > + __asm__ __volatile__( \
> > > > + "1: " ld " %0, %2 # __cmpxchg_asm \n" \
> > > > + " bne %0, %z3, 2f \n" \
> > > > + " or $t0, %z4, $zero \n" \
> > > > + " " st " $t0, %1 \n" \
> > > > + " beq $zero, $t0, 1b \n" \
> > > > + "2: \n" \
> > > > + : "=&r" (__ret), "=ZB"(*m) \
> > > > + : "ZB"(*m), "Jr" (old), "Jr" (new) \
> > > > + : "t0", "memory"); \
> > > > + \
> > > > + __ret; \
> > > > +})
> > > > +
> > > > +static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
> > > > + unsigned long new, unsigned int size)
> > > > +{
> > > > + switch (size) {
> > > > + case 4:
> > > > + return __cmpxchg_asm("ll.w", "sc.w", (volatile u32 *)ptr,
> > > > + (u32)old, new);
> > > > +
> > > > + case 8:
> > > > + return __cmpxchg_asm("ll.d", "sc.d", (volatile u64 *)ptr,
> > > > + (u64)old, new);
> > > > +
> > > > + default:
> > > > + BUILD_BUG();
> > > > + }
> > > > +
> > > > + return 0;
> > > > +}
> > > > +
> > > > +#define arch_cmpxchg_local(ptr, old, new) \
> > > > + ((__typeof__(*(ptr))) \
> > > > + __cmpxchg((ptr), \
> > > > + (unsigned long)(__typeof__(*(ptr)))(old), \
> > > > + (unsigned long)(__typeof__(*(ptr)))(new), \
> > > > + sizeof(*(ptr))))
> > > > +
> > > > +#define arch_cmpxchg(ptr, old, new) \
> > > Ditto, it's a relaxed implementation, right?
> > > > +({ \
> > > > + __typeof__(*(ptr)) __res; \
> > > > + \
> > > > + __res = arch_cmpxchg_local((ptr), (old), (new)); \
> > > > + \
> > > > + __res; \
> > > > +})
> > > > +
> > > > +#ifdef CONFIG_64BIT
> > > > +#define arch_cmpxchg64_local(ptr, o, n) \
> > > > + ({ \
> > > > + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \
> > > > + arch_cmpxchg_local((ptr), (o), (n)); \
> > > > + })
> > > > +
> > > > +#define arch_cmpxchg64(ptr, o, n) \
> > > > + ({ \
> > > > + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \
> > > > + arch_cmpxchg((ptr), (o), (n)); \
> > > > + })
> > > > +#else
> > > > +#include <asm-generic/cmpxchg-local.h>
> > > > +#define arch_cmpxchg64_local(ptr, o, n) __generic_cmpxchg64_local((ptr), (o), (n))
> > > > +#define arch_cmpxchg64(ptr, o, n) arch_cmpxchg64_local((ptr), (o), (n))
> > > > +#endif
> > > > +
> > > > +#endif /* __ASM_CMPXCHG_H */
> > > > diff --git a/arch/loongarch/include/asm/local.h b/arch/loongarch/include/asm/local.h
> > > > new file mode 100644
> > > > index 000000000000..2052a2267337
> > > > --- /dev/null
> > > > +++ b/arch/loongarch/include/asm/local.h
> > > > @@ -0,0 +1,138 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > +/*
> > > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > > + */
> > > > +#ifndef _ARCH_LOONGARCH_LOCAL_H
> > > > +#define _ARCH_LOONGARCH_LOCAL_H
> > > > +
> > > > +#include <linux/percpu.h>
> > > > +#include <linux/bitops.h>
> > > > +#include <linux/atomic.h>
> > > > +#include <asm/cmpxchg.h>
> > > > +#include <asm/compiler.h>
> > > > +
> > > > +typedef struct {
> > > > + atomic_long_t a;
> > > > +} local_t;
> > > > +
> > > > +#define LOCAL_INIT(i) { ATOMIC_LONG_INIT(i) }
> > > > +
> > > > +#define local_read(l) atomic_long_read(&(l)->a)
> > > > +#define local_set(l, i) atomic_long_set(&(l)->a, (i))
> > > > +
> > > > +#define local_add(i, l) atomic_long_add((i), (&(l)->a))
> > > > +#define local_sub(i, l) atomic_long_sub((i), (&(l)->a))
> > > > +#define local_inc(l) atomic_long_inc(&(l)->a)
> > > > +#define local_dec(l) atomic_long_dec(&(l)->a)
> > > > +
> > > > +/*
> > > > + * Same as above, but return the result value
> > > > + */
> > > > +static inline long local_add_return(long i, local_t *l)
> > > > +{
> > > > + unsigned long result;
> > > > +
> > > > + __asm__ __volatile__(
> > > > + " " __AMADD " %1, %2, %0 \n"
> > > > + : "+ZB" (l->a.counter), "=&r" (result)
> > > > + : "r" (i)
> > > > + : "memory");
> > > > + result = result + i;
> > > > +
> > > > + return result;
> > > > +}
> > > > +
> > > > +static inline long local_sub_return(long i, local_t *l)
> > > > +{
> > > > + unsigned long result;
> > > > +
> > > > + __asm__ __volatile__(
> > > > + " " __AMADD "%1, %2, %0 \n"
> > > > + : "+ZB" (l->a.counter), "=&r" (result)
> > > > + : "r" (-i)
> > > > + : "memory");
> > > > +
> > > > + result = result - i;
> > > > +
> > > > + return result;
> > > > +}
> > > > +
> > > > +#define local_cmpxchg(l, o, n) \
> > > > + ((long)cmpxchg_local(&((l)->a.counter), (o), (n)))
> > > > +#define local_xchg(l, n) (atomic_long_xchg((&(l)->a), (n)))
> > > > +
> > > > +/**
> > > > + * local_add_unless - add unless the number is a given value
> > > > + * @l: pointer of type local_t
> > > > + * @a: the amount to add to l...
> > > > + * @u: ...unless l is equal to u.
> > > > + *
> > > > + * Atomically adds @a to @l, so long as it was not @u.
> > > > + * Returns non-zero if @l was not @u, and zero otherwise.
> > > > + */
> > > > +#define local_add_unless(l, a, u) \
> > > > +({ \
> > > > + long c, old; \
> > > > + c = local_read(l); \
> > > > + while (c != (u) && (old = local_cmpxchg((l), c, c + (a))) != c) \
> > > > + c = old; \
> > > > + c != (u); \
> > > > +})
> > > > +#define local_inc_not_zero(l) local_add_unless((l), 1, 0)
> > > > +
> > > > +#define local_dec_return(l) local_sub_return(1, (l))
> > > > +#define local_inc_return(l) local_add_return(1, (l))
> > > > +
> > > > +/*
> > > > + * local_sub_and_test - subtract value from variable and test result
> > > > + * @i: integer value to subtract
> > > > + * @l: pointer of type local_t
> > > > + *
> > > > + * Atomically subtracts @i from @l and returns
> > > > + * true if the result is zero, or false for all
> > > > + * other cases.
> > > > + */
> > > > +#define local_sub_and_test(i, l) (local_sub_return((i), (l)) == 0)
> > > > +
> > > > +/*
> > > > + * local_inc_and_test - increment and test
> > > > + * @l: pointer of type local_t
> > > > + *
> > > > + * Atomically increments @l by 1
> > > > + * and returns true if the result is zero, or false for all
> > > > + * other cases.
> > > > + */
> > > > +#define local_inc_and_test(l) (local_inc_return(l) == 0)
> > > > +
> > > > +/*
> > > > + * local_dec_and_test - decrement by 1 and test
> > > > + * @l: pointer of type local_t
> > > > + *
> > > > + * Atomically decrements @l by 1 and
> > > > + * returns true if the result is 0, or false for all other
> > > > + * cases.
> > > > + */
> > > > +#define local_dec_and_test(l) (local_sub_return(1, (l)) == 0)
> > > > +
> > > > +/*
> > > > + * local_add_negative - add and test if negative
> > > > + * @l: pointer of type local_t
> > > > + * @i: integer value to add
> > > > + *
> > > > + * Atomically adds @i to @l and returns true
> > > > + * if the result is negative, or false when
> > > > + * result is greater than or equal to zero.
> > > > + */
> > > > +#define local_add_negative(i, l) (local_add_return(i, (l)) < 0)
> > > > +
> > > > +/* Use these for per-cpu local_t variables: on some archs they are
> > > > + * much more efficient than these naive implementations. Note they take
> > > > + * a variable, not an address.
> > > > + */
> > > > +
> > > > +#define __local_inc(l) ((l)->a.counter++)
> > > > +#define __local_dec(l) ((l)->a.counter++)
> > > > +#define __local_add(i, l) ((l)->a.counter += (i))
> > > > +#define __local_sub(i, l) ((l)->a.counter -= (i))
> > > > +
> > > > +#endif /* _ARCH_LOONGARCH_LOCAL_H */
> > > > diff --git a/arch/loongarch/include/asm/percpu.h b/arch/loongarch/include/asm/percpu.h
> > > > new file mode 100644
> > > > index 000000000000..7d5b22ebd834
> > > > --- /dev/null
> > > > +++ b/arch/loongarch/include/asm/percpu.h
> > > > @@ -0,0 +1,20 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > +/*
> > > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > > + */
> > > > +#ifndef __ASM_PERCPU_H
> > > > +#define __ASM_PERCPU_H
> > > > +
> > > > +/* Use r21 for fast access */
> > > > +register unsigned long __my_cpu_offset __asm__("$r21");
> > > > +
> > > > +static inline void set_my_cpu_offset(unsigned long off)
> > > > +{
> > > > + __my_cpu_offset = off;
> > > > + csr_writeq(off, PERCPU_BASE_KS);
> > > > +}
> > > > +#define __my_cpu_offset __my_cpu_offset
> > > > +
> > > > +#include <asm-generic/percpu.h>
> > > > +
> > > > +#endif /* __ASM_PERCPU_H */
> > > > --
> > > > 2.27.0
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
> > >
> > > ML: https://lore.kernel.org/linux-csky/
>
>
>
> --
> Best Regards
> Guo Ren
>
> ML: https://lore.kernel.org/linux-csky/

2022-05-23 06:33:41

by WANG Xuerui

[permalink] [raw]
Subject: Re: [PATCH V11 00/22] arch: Add basic LoongArch support

On 5/18/22 17:25, Huacai Chen wrote:
> [snip]
>
> V10 -> V11:
> 1, Rebased on asm-generic tree;
> 2, Fix fpreg macros definition;
> 3, Fix ELF ABI macros definition;
> 4, Fix magic number definition in efi header;
> 5, Remove unneeded swab.h, bitfield.h and rtc.c;
> 6, Remove __ARCH_WANT_NEW_STAT (glibc need update);

Regarding the syscall ABI change taking out fstat and newfstatat, I've
done the following to ensure a clean path forward:

- Sent glibc patch [1] for upstream review;
- Filed [2] on Loongson's glibc fork for them to test, and incorporate
the 2nd patch in their port;
- Updated my tool [3] for the small number of end users already on the
previous ABI (me included, actually), to easily check if their systems
are compatible before moving to newer kernels.

[1]: https://sourceware.org/pipermail/libc-alpha/2022-May/138958.html
[2]: https://github.com/loongson/glibc/pull/29
[3]: https://github.com/xen0n/shengloong

> 7, Improve documents as WANG Xuerui suggested;
> 8, Some other minor fixes and improvements.

2022-05-23 07:00:59

by Guo Ren

[permalink] [raw]
Subject: Re: [PATCH V11 07/22] LoongArch: Add atomic/locking headers

On Fri, May 20, 2022 at 5:50 PM Huacai Chen <[email protected]> wrote:
>
> Hi, Ren,
>
> On Fri, May 20, 2022 at 3:54 PM Guo Ren <[email protected]> wrote:
> >
> > On Wed, May 18, 2022 at 5:34 PM Huacai Chen <[email protected]> wrote:
> > >
> > > Add common headers (atomic, bitops, barrier and locking) for basic
> > > LoongArch support.
> > >
> > > Signed-off-by: Huacai Chen <[email protected]>
> > > ---
> > > arch/loongarch/include/asm/atomic.h | 358 +++++++++++++++++++++++++++
> > > arch/loongarch/include/asm/barrier.h | 51 ++++
> > > arch/loongarch/include/asm/bitops.h | 33 +++
> > > arch/loongarch/include/asm/bitrev.h | 34 +++
> > > arch/loongarch/include/asm/cmpxchg.h | 121 +++++++++
> > > arch/loongarch/include/asm/local.h | 138 +++++++++++
> > > arch/loongarch/include/asm/percpu.h | 20 ++
> > > 7 files changed, 755 insertions(+)
> > > create mode 100644 arch/loongarch/include/asm/atomic.h
> > > create mode 100644 arch/loongarch/include/asm/barrier.h
> > > create mode 100644 arch/loongarch/include/asm/bitops.h
> > > create mode 100644 arch/loongarch/include/asm/bitrev.h
> > > create mode 100644 arch/loongarch/include/asm/cmpxchg.h
> > > create mode 100644 arch/loongarch/include/asm/local.h
> > > create mode 100644 arch/loongarch/include/asm/percpu.h
> > >
> > > diff --git a/arch/loongarch/include/asm/atomic.h b/arch/loongarch/include/asm/atomic.h
> > > new file mode 100644
> > > index 000000000000..932352342b12
> > > --- /dev/null
> > > +++ b/arch/loongarch/include/asm/atomic.h
> > > @@ -0,0 +1,358 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +/*
> > > + * Atomic operations.
> > > + *
> > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > + */
> > > +#ifndef _ASM_ATOMIC_H
> > > +#define _ASM_ATOMIC_H
> > > +
> > > +#include <linux/types.h>
> > > +#include <asm/barrier.h>
> > > +#include <asm/cmpxchg.h>
> > > +#include <asm/compiler.h>
> > > +
> > > +#if __SIZEOF_LONG__ == 4
> > > +#define __LL "ll.w "
> > > +#define __SC "sc.w "
> > > +#define __AMADD "amadd.w "
> > > +#define __AMAND_DB "amand_db.w "
> > > +#define __AMOR_DB "amor_db.w "
> > > +#define __AMXOR_DB "amxor_db.w "
> > > +#elif __SIZEOF_LONG__ == 8
> > > +#define __LL "ll.d "
> > > +#define __SC "sc.d "
> > > +#define __AMADD "amadd.d "
> > > +#define __AMAND_DB "amand_db.d "
> > > +#define __AMOR_DB "amor_db.d "
> > > +#define __AMXOR_DB "amxor_db.d "
> > > +#endif
> > > +
> > > +#define ATOMIC_INIT(i) { (i) }
> > > +
> > > +/*
> > > + * arch_atomic_read - read atomic variable
> > > + * @v: pointer of type atomic_t
> > > + *
> > > + * Atomically reads the value of @v.
> > > + */
> > > +#define arch_atomic_read(v) READ_ONCE((v)->counter)
> > > +
> > > +/*
> > > + * arch_atomic_set - set atomic variable
> > > + * @v: pointer of type atomic_t
> > > + * @i: required value
> > > + *
> > > + * Atomically sets the value of @v to @i.
> > > + */
> > > +#define arch_atomic_set(v, i) WRITE_ONCE((v)->counter, (i))
> > > +
> > > +#define ATOMIC_OP(op, I, asm_op) \
> > > +static inline void arch_atomic_##op(int i, atomic_t *v) \
> > > +{ \
> > > + __asm__ __volatile__( \
> > > + "am"#asm_op"_db.w" " $zero, %1, %0 \n" \
> > > + : "+ZB" (v->counter) \
> > > + : "r" (I) \
> > > + : "memory"); \
> > > +}
> > > +
> > > +#define ATOMIC_OP_RETURN(op, I, asm_op, c_op) \
> > > +static inline int arch_atomic_##op##_return_relaxed(int i, atomic_t *v) \
> > > +{ \
> > > + int result; \
> > > + \
> > > + __asm__ __volatile__( \
> > > + "am"#asm_op"_db.w" " %1, %2, %0 \n" \
> > > + : "+ZB" (v->counter), "=&r" (result) \
> > > + : "r" (I) \
> > > + : "memory"); \
> > > + \
> > > + return result c_op I; \
> > > +}
> > > +
> > > +#define ATOMIC_FETCH_OP(op, I, asm_op) \
> > > +static inline int arch_atomic_fetch_##op##_relaxed(int i, atomic_t *v) \
> > > +{ \
> > > + int result; \
> > > + \
> > > + __asm__ __volatile__( \
> > > + "am"#asm_op"_db.w" " %1, %2, %0 \n" \
> > > + : "+ZB" (v->counter), "=&r" (result) \
> > > + : "r" (I) \
> > > + : "memory"); \
> > > + \
> > > + return result; \
> > > +}
> > > +
> > > +#define ATOMIC_OPS(op, I, asm_op, c_op) \
> > > + ATOMIC_OP(op, I, asm_op) \
> > > + ATOMIC_OP_RETURN(op, I, asm_op, c_op) \
> > > + ATOMIC_FETCH_OP(op, I, asm_op)
> > > +
> > > +ATOMIC_OPS(add, i, add, +)
> > > +ATOMIC_OPS(sub, -i, add, +)
> > > +
> > > +#define arch_atomic_add_return_relaxed arch_atomic_add_return_relaxed
> > > +#define arch_atomic_sub_return_relaxed arch_atomic_sub_return_relaxed
> > > +#define arch_atomic_fetch_add_relaxed arch_atomic_fetch_add_relaxed
> > > +#define arch_atomic_fetch_sub_relaxed arch_atomic_fetch_sub_relaxed
> > > +
> > > +#undef ATOMIC_OPS
> > > +
> > > +#define ATOMIC_OPS(op, I, asm_op) \
> > > + ATOMIC_OP(op, I, asm_op) \
> > > + ATOMIC_FETCH_OP(op, I, asm_op)
> > > +
> > > +ATOMIC_OPS(and, i, and)
> > > +ATOMIC_OPS(or, i, or)
> > > +ATOMIC_OPS(xor, i, xor)
> > > +
> > > +#define arch_atomic_fetch_and_relaxed arch_atomic_fetch_and_relaxed
> > > +#define arch_atomic_fetch_or_relaxed arch_atomic_fetch_or_relaxed
> > > +#define arch_atomic_fetch_xor_relaxed arch_atomic_fetch_xor_relaxed
> > > +
> > > +#undef ATOMIC_OPS
> > > +#undef ATOMIC_FETCH_OP
> > > +#undef ATOMIC_OP_RETURN
> > > +#undef ATOMIC_OP
> > > +
> > > +static inline int arch_atomic_fetch_add_unless(atomic_t *v, int a, int u)
> > > +{
> > > + int prev, rc;
> > > +
> > > + __asm__ __volatile__ (
> > We need __WEAK_LLSC_MB here, to prevent previous access after sc.w.
> LoongArch's ll and sc has implied memory barrier. :)

But why do you put __WEAK_LLSC_MB after sc ?

>
> >
> > > + "0: ll.w %[p], %[c]\n"
> > > + " beq %[p], %[u], 1f\n"
> > > + " add.w %[rc], %[p], %[a]\n"
> > > + " sc.w %[rc], %[c]\n"
> > > + " beqz %[rc], 0b\n"
> > > + " b 2f\n"
> > > + "1:\n"
> > > + __WEAK_LLSC_MB
> > > + "2:\n"
> > > + : [p]"=&r" (prev), [rc]"=&r" (rc),
> > > + [c]"=ZB" (v->counter)
> > > + : [a]"r" (a), [u]"r" (u)
> > > + : "memory");
> > > +
> > > + return prev;
> > > +}
> > > +#define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless
> > > +
> > > +/*
> > > + * arch_atomic_sub_if_positive - conditionally subtract integer from atomic variable
> > > + * @i: integer value to subtract
> > > + * @v: pointer of type atomic_t
> > > + *
> > > + * Atomically test @v and subtract @i if @v is greater or equal than @i.
> > > + * The function returns the old value of @v minus @i.
> > > + */
> > > +static inline int arch_atomic_sub_if_positive(int i, atomic_t *v)
> > > +{
> > > + int result;
> > > + int temp;
> > > +
> > > + if (__builtin_constant_p(i)) {
> > > + __asm__ __volatile__(
> > Ditto.
> > > + "1: ll.w %1, %2 # atomic_sub_if_positive\n"
> > > + " addi.w %0, %1, %3 \n"
> > > + " or %1, %0, $zero \n"
> > > + " blt %0, $zero, 2f \n"
> > > + " sc.w %1, %2 \n"
> > > + " beq $zero, %1, 1b \n"
> > Need __WEAK_LLSC_MB ?
> > > + "2: \n"
> > > + : "=&r" (result), "=&r" (temp),
> > > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > > + : "I" (-i));
> > > + } else {
> > > + __asm__ __volatile__(
> > Ditto.
> > > + "1: ll.w %1, %2 # atomic_sub_if_positive\n"
> > > + " sub.w %0, %1, %3 \n"
> > > + " or %1, %0, $zero \n"
> > > + " blt %0, $zero, 2f \n"
> > > + " sc.w %1, %2 \n"
> > Need __WEAK_LLSC_MB ?
> > > + " beq $zero, %1, 1b \n"
> > > + "2: \n"
> > > + : "=&r" (result), "=&r" (temp),
> > > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > > + : "r" (i));
> > > + }
> > > +
> > > + return result;
> > > +}
> > > +
> > > +#define arch_atomic_cmpxchg(v, o, n) (arch_cmpxchg(&((v)->counter), (o), (n)))
> > > +#define arch_atomic_xchg(v, new) (arch_xchg(&((v)->counter), (new)))
> > > +
> > > +/*
> > > + * arch_atomic_dec_if_positive - decrement by 1 if old value positive
> > > + * @v: pointer of type atomic_t
> > > + */
> > > +#define arch_atomic_dec_if_positive(v) arch_atomic_sub_if_positive(1, v)
> > > +
> > > +#ifdef CONFIG_64BIT
> > > +
> > > +#define ATOMIC64_INIT(i) { (i) }
> > > +
> > > +/*
> > > + * arch_atomic64_read - read atomic variable
> > > + * @v: pointer of type atomic64_t
> > > + *
> > > + */
> > > +#define arch_atomic64_read(v) READ_ONCE((v)->counter)
> > > +
> > > +/*
> > > + * arch_atomic64_set - set atomic variable
> > > + * @v: pointer of type atomic64_t
> > > + * @i: required value
> > > + */
> > > +#define arch_atomic64_set(v, i) WRITE_ONCE((v)->counter, (i))
> > > +
> > > +#define ATOMIC64_OP(op, I, asm_op) \
> > > +static inline void arch_atomic64_##op(long i, atomic64_t *v) \
> > > +{ \
> > > + __asm__ __volatile__( \
> > > + "am"#asm_op"_db.d " " $zero, %1, %0 \n" \
> > > + : "+ZB" (v->counter) \
> > > + : "r" (I) \
> > > + : "memory"); \
> > > +}
> > > +
> > > +#define ATOMIC64_OP_RETURN(op, I, asm_op, c_op) \
> > > +static inline long arch_atomic64_##op##_return_relaxed(long i, atomic64_t *v) \
> > > +{ \
> > > + long result; \
> > > + __asm__ __volatile__( \
> > > + "am"#asm_op"_db.d " " %1, %2, %0 \n" \
> > > + : "+ZB" (v->counter), "=&r" (result) \
> > > + : "r" (I) \
> > > + : "memory"); \
> > > + \
> > > + return result c_op I; \
> > > +}
> > > +
> > > +#define ATOMIC64_FETCH_OP(op, I, asm_op) \
> > > +static inline long arch_atomic64_fetch_##op##_relaxed(long i, atomic64_t *v) \
> > > +{ \
> > > + long result; \
> > > + \
> > > + __asm__ __volatile__( \
> > > + "am"#asm_op"_db.d " " %1, %2, %0 \n" \
> > > + : "+ZB" (v->counter), "=&r" (result) \
> > > + : "r" (I) \
> > > + : "memory"); \
> > > + \
> > > + return result; \
> > > +}
> > > +
> > > +#define ATOMIC64_OPS(op, I, asm_op, c_op) \
> > > + ATOMIC64_OP(op, I, asm_op) \
> > > + ATOMIC64_OP_RETURN(op, I, asm_op, c_op) \
> > > + ATOMIC64_FETCH_OP(op, I, asm_op)
> > > +
> > > +ATOMIC64_OPS(add, i, add, +)
> > > +ATOMIC64_OPS(sub, -i, add, +)
> > > +
> > > +#define arch_atomic64_add_return_relaxed arch_atomic64_add_return_relaxed
> > > +#define arch_atomic64_sub_return_relaxed arch_atomic64_sub_return_relaxed
> > > +#define arch_atomic64_fetch_add_relaxed arch_atomic64_fetch_add_relaxed
> > > +#define arch_atomic64_fetch_sub_relaxed arch_atomic64_fetch_sub_relaxed
> > > +
> > > +#undef ATOMIC64_OPS
> > > +
> > > +#define ATOMIC64_OPS(op, I, asm_op) \
> > > + ATOMIC64_OP(op, I, asm_op) \
> > > + ATOMIC64_FETCH_OP(op, I, asm_op)
> > > +
> > > +ATOMIC64_OPS(and, i, and)
> > > +ATOMIC64_OPS(or, i, or)
> > > +ATOMIC64_OPS(xor, i, xor)
> > > +
> > > +#define arch_atomic64_fetch_and_relaxed arch_atomic64_fetch_and_relaxed
> > > +#define arch_atomic64_fetch_or_relaxed arch_atomic64_fetch_or_relaxed
> > > +#define arch_atomic64_fetch_xor_relaxed arch_atomic64_fetch_xor_relaxed
> > > +
> > > +#undef ATOMIC64_OPS
> > > +#undef ATOMIC64_FETCH_OP
> > > +#undef ATOMIC64_OP_RETURN
> > > +#undef ATOMIC64_OP
> > > +
> > > +static inline long arch_atomic64_fetch_add_unless(atomic64_t *v, long a, long u)
> > > +{
> > > + long prev, rc;
> > > +
> > > + __asm__ __volatile__ (
> > Ditto
> > > + "0: ll.d %[p], %[c]\n"
> > > + " beq %[p], %[u], 1f\n"
> > > + " add.d %[rc], %[p], %[a]\n"
> > > + " sc.d %[rc], %[c]\n"
> > > + " beqz %[rc], 0b\n"
> > Ditto
> > > + " b 2f\n"
> > > + "1:\n"
> > > + __WEAK_LLSC_MB
> > > + "2:\n"
> > > + : [p]"=&r" (prev), [rc]"=&r" (rc),
> > > + [c] "=ZB" (v->counter)
> > > + : [a]"r" (a), [u]"r" (u)
> > > + : "memory");
> > > +
> > > + return prev;
> > > +}
> > > +#define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless
> > > +
> > > +/*
> > > + * arch_atomic64_sub_if_positive - conditionally subtract integer from atomic variable
> > > + * @i: integer value to subtract
> > > + * @v: pointer of type atomic64_t
> > > + *
> > > + * Atomically test @v and subtract @i if @v is greater or equal than @i.
> > > + * The function returns the old value of @v minus @i.
> > > + */
> > > +static inline long arch_atomic64_sub_if_positive(long i, atomic64_t *v)
> > > +{
> > > + long result;
> > > + long temp;
> > > +
> > > + if (__builtin_constant_p(i)) {
> > > + __asm__ __volatile__(
> > Ditto
> > > + "1: ll.d %1, %2 # atomic64_sub_if_positive \n"
> > > + " addi.d %0, %1, %3 \n"
> > > + " or %1, %0, $zero \n"
> > > + " blt %0, $zero, 2f \n"
> > > + " sc.d %1, %2 \n"
> > Ditto
> > > + " beq %1, $zero, 1b \n"
> > > + "2: \n"
> > > + : "=&r" (result), "=&r" (temp),
> > > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > > + : "I" (-i));
> > > + } else {
> > > + __asm__ __volatile__(
> > Ditto
> > > + "1: ll.d %1, %2 # atomic64_sub_if_positive \n"
> > > + " sub.d %0, %1, %3 \n"
> > > + " or %1, %0, $zero \n"
> > > + " blt %0, $zero, 2f \n"
> > > + " sc.d %1, %2 \n"
> > Ditto
> > > + " beq %1, $zero, 1b \n"
> > > + "2: \n"
> > > + : "=&r" (result), "=&r" (temp),
> > > + "+" GCC_OFF_SMALL_ASM() (v->counter)
> > > + : "r" (i));
> > > + }
> > > +
> > > + return result;
> > > +}
> > > +
> > > +#define arch_atomic64_cmpxchg(v, o, n) \
> > > + ((__typeof__((v)->counter))arch_cmpxchg(&((v)->counter), (o), (n)))
> > > +#define arch_atomic64_xchg(v, new) (arch_xchg(&((v)->counter), (new)))
> > > +
> > > +/*
> > > + * arch_atomic64_dec_if_positive - decrement by 1 if old value positive
> > > + * @v: pointer of type atomic64_t
> > > + */
> > > +#define arch_atomic64_dec_if_positive(v) arch_atomic64_sub_if_positive(1, v)
> > > +
> > > +#endif /* CONFIG_64BIT */
> > > +
> > > +#endif /* _ASM_ATOMIC_H */
> > > diff --git a/arch/loongarch/include/asm/barrier.h b/arch/loongarch/include/asm/barrier.h
> > > new file mode 100644
> > > index 000000000000..e57571bcaf4f
> > > --- /dev/null
> > > +++ b/arch/loongarch/include/asm/barrier.h
> > > @@ -0,0 +1,51 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +/*
> > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > + */
> > > +#ifndef __ASM_BARRIER_H
> > > +#define __ASM_BARRIER_H
> > > +
> > > +#define __sync() __asm__ __volatile__("dbar 0" : : : "memory")
> > > +
> > > +#define fast_wmb() __sync()
> > > +#define fast_rmb() __sync()
> > > +#define fast_mb() __sync()
> > > +#define fast_iob() __sync()
> > > +#define wbflush() __sync()
> > > +
> > > +#define wmb() fast_wmb()
> > > +#define rmb() fast_rmb()
> > > +#define mb() fast_mb()
> > > +#define iob() fast_iob()
> > > +
> > > +/**
> > > + * array_index_mask_nospec() - generate a ~0 mask when index < size, 0 otherwise
> > > + * @index: array element index
> > > + * @size: number of elements in array
> > > + *
> > > + * Returns:
> > > + * 0 - (@index < @size)
> > > + */
> > > +#define array_index_mask_nospec array_index_mask_nospec
> > > +static inline unsigned long array_index_mask_nospec(unsigned long index,
> > > + unsigned long size)
> > > +{
> > > + unsigned long mask;
> > > +
> > > + __asm__ __volatile__(
> > > + "sltu %0, %1, %2\n\t"
> > > +#if (__SIZEOF_LONG__ == 4)
> > > + "sub.w %0, $r0, %0\n\t"
> > > +#elif (__SIZEOF_LONG__ == 8)
> > > + "sub.d %0, $r0, %0\n\t"
> > > +#endif
> > > + : "=r" (mask)
> > > + : "r" (index), "r" (size)
> > > + :);
> > > +
> > > + return mask;
> > > +}
> > > +
> > > +#include <asm-generic/barrier.h>
> > > +
> > > +#endif /* __ASM_BARRIER_H */
> > > diff --git a/arch/loongarch/include/asm/bitops.h b/arch/loongarch/include/asm/bitops.h
> > > new file mode 100644
> > > index 000000000000..69e00f8d8034
> > > --- /dev/null
> > > +++ b/arch/loongarch/include/asm/bitops.h
> > > @@ -0,0 +1,33 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +/*
> > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > + */
> > > +#ifndef _ASM_BITOPS_H
> > > +#define _ASM_BITOPS_H
> > > +
> > > +#include <linux/compiler.h>
> > > +
> > > +#ifndef _LINUX_BITOPS_H
> > > +#error only <linux/bitops.h> can be included directly
> > > +#endif
> > > +
> > > +#include <asm/barrier.h>
> > > +
> > > +#include <asm-generic/bitops/builtin-ffs.h>
> > > +#include <asm-generic/bitops/builtin-fls.h>
> > > +#include <asm-generic/bitops/builtin-__ffs.h>
> > > +#include <asm-generic/bitops/builtin-__fls.h>
> > > +
> > > +#include <asm-generic/bitops/ffz.h>
> > > +#include <asm-generic/bitops/fls64.h>
> > > +
> > > +#include <asm-generic/bitops/sched.h>
> > > +#include <asm-generic/bitops/hweight.h>
> > > +
> > > +#include <asm-generic/bitops/atomic.h>
> > > +#include <asm-generic/bitops/non-atomic.h>
> > > +#include <asm-generic/bitops/lock.h>
> > > +#include <asm-generic/bitops/le.h>
> > > +#include <asm-generic/bitops/ext2-atomic.h>
> > > +
> > > +#endif /* _ASM_BITOPS_H */
> > > diff --git a/arch/loongarch/include/asm/bitrev.h b/arch/loongarch/include/asm/bitrev.h
> > > new file mode 100644
> > > index 000000000000..46f275b9cdf7
> > > --- /dev/null
> > > +++ b/arch/loongarch/include/asm/bitrev.h
> > > @@ -0,0 +1,34 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +/*
> > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > + */
> > > +#ifndef __LOONGARCH_ASM_BITREV_H__
> > > +#define __LOONGARCH_ASM_BITREV_H__
> > > +
> > > +#include <linux/swab.h>
> > > +
> > > +static __always_inline __attribute_const__ u32 __arch_bitrev32(u32 x)
> > > +{
> > > + u32 ret;
> > > +
> > > + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(__swab32(x)));
> > > + return ret;
> > > +}
> > > +
> > > +static __always_inline __attribute_const__ u16 __arch_bitrev16(u16 x)
> > > +{
> > > + u16 ret;
> > > +
> > > + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(__swab16(x)));
> > > + return ret;
> > > +}
> > > +
> > > +static __always_inline __attribute_const__ u8 __arch_bitrev8(u8 x)
> > > +{
> > > + u8 ret;
> > > +
> > > + asm("bitrev.4b %0, %1" : "=r"(ret) : "r"(x));
> > > + return ret;
> > > +}
> > > +
> > > +#endif /* __LOONGARCH_ASM_BITREV_H__ */
> > > diff --git a/arch/loongarch/include/asm/cmpxchg.h b/arch/loongarch/include/asm/cmpxchg.h
> > > new file mode 100644
> > > index 000000000000..699925b8ff27
> > > --- /dev/null
> > > +++ b/arch/loongarch/include/asm/cmpxchg.h
> > > @@ -0,0 +1,121 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +/*
> > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > + */
> > > +#ifndef __ASM_CMPXCHG_H
> > > +#define __ASM_CMPXCHG_H
> > > +
> > > +#include <linux/build_bug.h>
> > > +
> > > +#define __xchg_asm(amswap_db, m, val) \
> > > +({ \
> > > + __typeof(val) __ret; \
> > > + \
> > > + __asm__ __volatile__ ( \
> > > + " "amswap_db" %1, %z2, %0 \n" \
> > > + : "+ZB" (*m), "=&r" (__ret) \
> > > + : "Jr" (val) \
> > > + : "memory"); \
> > > + \
> > > + __ret; \
> > > +})
> > > +
> > > +static inline unsigned long __xchg(volatile void *ptr, unsigned long x,
> > relaxed ?
> >
> > > + int size)
> > > +{
> > > + switch (size) {
> > > + case 4:
> > > + return __xchg_asm("amswap_db.w", (volatile u32 *)ptr, (u32)x);
> > Your amswap_db don't have any implicit acquire/release, right?
> amswap don't have implicit acquire/release, but amswap_db has.
>
> Huacai
> >
> > > +
> > > + case 8:
> > > + return __xchg_asm("amswap_db.d", (volatile u64 *)ptr, (u64)x);
> > > +
> > > + default:
> > > + BUILD_BUG();
> > > + }
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +#define arch_xchg(ptr, x) \
> > > +({ \
> > > + __typeof__(*(ptr)) __res; \
> > > + \
> > > + __res = (__typeof__(*(ptr))) \
> > > + __xchg((ptr), (unsigned long)(x), sizeof(*(ptr))); \
> > > + \
> > > + __res; \
> > > +})
> > > +
> > > +#define __cmpxchg_asm(ld, st, m, old, new) \
> > > +({ \
> > > + __typeof(old) __ret; \
> > > + \
> > > + __asm__ __volatile__( \
> > > + "1: " ld " %0, %2 # __cmpxchg_asm \n" \
> > > + " bne %0, %z3, 2f \n" \
> > > + " or $t0, %z4, $zero \n" \
> > > + " " st " $t0, %1 \n" \
> > > + " beq $zero, $t0, 1b \n" \
> > > + "2: \n" \
> > > + : "=&r" (__ret), "=ZB"(*m) \
> > > + : "ZB"(*m), "Jr" (old), "Jr" (new) \
> > > + : "t0", "memory"); \
> > > + \
> > > + __ret; \
> > > +})
> > > +
> > > +static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
> > > + unsigned long new, unsigned int size)
> > > +{
> > > + switch (size) {
> > > + case 4:
> > > + return __cmpxchg_asm("ll.w", "sc.w", (volatile u32 *)ptr,
> > > + (u32)old, new);
> > > +
> > > + case 8:
> > > + return __cmpxchg_asm("ll.d", "sc.d", (volatile u64 *)ptr,
> > > + (u64)old, new);
> > > +
> > > + default:
> > > + BUILD_BUG();
> > > + }
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +#define arch_cmpxchg_local(ptr, old, new) \
> > > + ((__typeof__(*(ptr))) \
> > > + __cmpxchg((ptr), \
> > > + (unsigned long)(__typeof__(*(ptr)))(old), \
> > > + (unsigned long)(__typeof__(*(ptr)))(new), \
> > > + sizeof(*(ptr))))
> > > +
> > > +#define arch_cmpxchg(ptr, old, new) \
> > Ditto, it's a relaxed implementation, right?
> > > +({ \
> > > + __typeof__(*(ptr)) __res; \
> > > + \
> > > + __res = arch_cmpxchg_local((ptr), (old), (new)); \
> > > + \
> > > + __res; \
> > > +})
> > > +
> > > +#ifdef CONFIG_64BIT
> > > +#define arch_cmpxchg64_local(ptr, o, n) \
> > > + ({ \
> > > + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \
> > > + arch_cmpxchg_local((ptr), (o), (n)); \
> > > + })
> > > +
> > > +#define arch_cmpxchg64(ptr, o, n) \
> > > + ({ \
> > > + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \
> > > + arch_cmpxchg((ptr), (o), (n)); \
> > > + })
> > > +#else
> > > +#include <asm-generic/cmpxchg-local.h>
> > > +#define arch_cmpxchg64_local(ptr, o, n) __generic_cmpxchg64_local((ptr), (o), (n))
> > > +#define arch_cmpxchg64(ptr, o, n) arch_cmpxchg64_local((ptr), (o), (n))
> > > +#endif
> > > +
> > > +#endif /* __ASM_CMPXCHG_H */
> > > diff --git a/arch/loongarch/include/asm/local.h b/arch/loongarch/include/asm/local.h
> > > new file mode 100644
> > > index 000000000000..2052a2267337
> > > --- /dev/null
> > > +++ b/arch/loongarch/include/asm/local.h
> > > @@ -0,0 +1,138 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +/*
> > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > + */
> > > +#ifndef _ARCH_LOONGARCH_LOCAL_H
> > > +#define _ARCH_LOONGARCH_LOCAL_H
> > > +
> > > +#include <linux/percpu.h>
> > > +#include <linux/bitops.h>
> > > +#include <linux/atomic.h>
> > > +#include <asm/cmpxchg.h>
> > > +#include <asm/compiler.h>
> > > +
> > > +typedef struct {
> > > + atomic_long_t a;
> > > +} local_t;
> > > +
> > > +#define LOCAL_INIT(i) { ATOMIC_LONG_INIT(i) }
> > > +
> > > +#define local_read(l) atomic_long_read(&(l)->a)
> > > +#define local_set(l, i) atomic_long_set(&(l)->a, (i))
> > > +
> > > +#define local_add(i, l) atomic_long_add((i), (&(l)->a))
> > > +#define local_sub(i, l) atomic_long_sub((i), (&(l)->a))
> > > +#define local_inc(l) atomic_long_inc(&(l)->a)
> > > +#define local_dec(l) atomic_long_dec(&(l)->a)
> > > +
> > > +/*
> > > + * Same as above, but return the result value
> > > + */
> > > +static inline long local_add_return(long i, local_t *l)
> > > +{
> > > + unsigned long result;
> > > +
> > > + __asm__ __volatile__(
> > > + " " __AMADD " %1, %2, %0 \n"
> > > + : "+ZB" (l->a.counter), "=&r" (result)
> > > + : "r" (i)
> > > + : "memory");
> > > + result = result + i;
> > > +
> > > + return result;
> > > +}
> > > +
> > > +static inline long local_sub_return(long i, local_t *l)
> > > +{
> > > + unsigned long result;
> > > +
> > > + __asm__ __volatile__(
> > > + " " __AMADD "%1, %2, %0 \n"
> > > + : "+ZB" (l->a.counter), "=&r" (result)
> > > + : "r" (-i)
> > > + : "memory");
> > > +
> > > + result = result - i;
> > > +
> > > + return result;
> > > +}
> > > +
> > > +#define local_cmpxchg(l, o, n) \
> > > + ((long)cmpxchg_local(&((l)->a.counter), (o), (n)))
> > > +#define local_xchg(l, n) (atomic_long_xchg((&(l)->a), (n)))
> > > +
> > > +/**
> > > + * local_add_unless - add unless the number is a given value
> > > + * @l: pointer of type local_t
> > > + * @a: the amount to add to l...
> > > + * @u: ...unless l is equal to u.
> > > + *
> > > + * Atomically adds @a to @l, so long as it was not @u.
> > > + * Returns non-zero if @l was not @u, and zero otherwise.
> > > + */
> > > +#define local_add_unless(l, a, u) \
> > > +({ \
> > > + long c, old; \
> > > + c = local_read(l); \
> > > + while (c != (u) && (old = local_cmpxchg((l), c, c + (a))) != c) \
> > > + c = old; \
> > > + c != (u); \
> > > +})
> > > +#define local_inc_not_zero(l) local_add_unless((l), 1, 0)
> > > +
> > > +#define local_dec_return(l) local_sub_return(1, (l))
> > > +#define local_inc_return(l) local_add_return(1, (l))
> > > +
> > > +/*
> > > + * local_sub_and_test - subtract value from variable and test result
> > > + * @i: integer value to subtract
> > > + * @l: pointer of type local_t
> > > + *
> > > + * Atomically subtracts @i from @l and returns
> > > + * true if the result is zero, or false for all
> > > + * other cases.
> > > + */
> > > +#define local_sub_and_test(i, l) (local_sub_return((i), (l)) == 0)
> > > +
> > > +/*
> > > + * local_inc_and_test - increment and test
> > > + * @l: pointer of type local_t
> > > + *
> > > + * Atomically increments @l by 1
> > > + * and returns true if the result is zero, or false for all
> > > + * other cases.
> > > + */
> > > +#define local_inc_and_test(l) (local_inc_return(l) == 0)
> > > +
> > > +/*
> > > + * local_dec_and_test - decrement by 1 and test
> > > + * @l: pointer of type local_t
> > > + *
> > > + * Atomically decrements @l by 1 and
> > > + * returns true if the result is 0, or false for all other
> > > + * cases.
> > > + */
> > > +#define local_dec_and_test(l) (local_sub_return(1, (l)) == 0)
> > > +
> > > +/*
> > > + * local_add_negative - add and test if negative
> > > + * @l: pointer of type local_t
> > > + * @i: integer value to add
> > > + *
> > > + * Atomically adds @i to @l and returns true
> > > + * if the result is negative, or false when
> > > + * result is greater than or equal to zero.
> > > + */
> > > +#define local_add_negative(i, l) (local_add_return(i, (l)) < 0)
> > > +
> > > +/* Use these for per-cpu local_t variables: on some archs they are
> > > + * much more efficient than these naive implementations. Note they take
> > > + * a variable, not an address.
> > > + */
> > > +
> > > +#define __local_inc(l) ((l)->a.counter++)
> > > +#define __local_dec(l) ((l)->a.counter++)
> > > +#define __local_add(i, l) ((l)->a.counter += (i))
> > > +#define __local_sub(i, l) ((l)->a.counter -= (i))
> > > +
> > > +#endif /* _ARCH_LOONGARCH_LOCAL_H */
> > > diff --git a/arch/loongarch/include/asm/percpu.h b/arch/loongarch/include/asm/percpu.h
> > > new file mode 100644
> > > index 000000000000..7d5b22ebd834
> > > --- /dev/null
> > > +++ b/arch/loongarch/include/asm/percpu.h
> > > @@ -0,0 +1,20 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +/*
> > > + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> > > + */
> > > +#ifndef __ASM_PERCPU_H
> > > +#define __ASM_PERCPU_H
> > > +
> > > +/* Use r21 for fast access */
> > > +register unsigned long __my_cpu_offset __asm__("$r21");
> > > +
> > > +static inline void set_my_cpu_offset(unsigned long off)
> > > +{
> > > + __my_cpu_offset = off;
> > > + csr_writeq(off, PERCPU_BASE_KS);
> > > +}
> > > +#define __my_cpu_offset __my_cpu_offset
> > > +
> > > +#include <asm-generic/percpu.h>
> > > +
> > > +#endif /* __ASM_PERCPU_H */
> > > --
> > > 2.27.0
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
> > ML: https://lore.kernel.org/linux-csky/



--
Best Regards
Guo Ren

ML: https://lore.kernel.org/linux-csky/

2022-05-23 08:07:56

by Guo Ren

[permalink] [raw]
Subject: Re: [PATCH V11 05/22] LoongArch: Add build infrastructure

Reviewed-by: Guo Ren <[email protected]>

On Wed, May 18, 2022 at 5:32 PM Huacai Chen <[email protected]> wrote:
>
> Add Kbuild, Makefile, Kconfig and link script for LoongArch build
> infrastructure.
>
> Signed-off-by: Huacai Chen <[email protected]>
> ---
> arch/loongarch/Kbuild | 6 +
> arch/loongarch/Kconfig | 395 +++++++++++++++++++++++++
> arch/loongarch/Kconfig.debug | 0
> arch/loongarch/Makefile | 105 +++++++
> arch/loongarch/boot/.gitignore | 2 +
> arch/loongarch/boot/Makefile | 20 ++
> arch/loongarch/boot/dts/Makefile | 4 +
> arch/loongarch/include/asm/Kbuild | 30 ++
> arch/loongarch/include/uapi/asm/Kbuild | 2 +
> arch/loongarch/kernel/.gitignore | 2 +
> arch/loongarch/kernel/Makefile | 21 ++
> arch/loongarch/kernel/vmlinux.lds.S | 117 ++++++++
> arch/loongarch/lib/Makefile | 6 +
> arch/loongarch/mm/Makefile | 9 +
> arch/loongarch/pci/Makefile | 7 +
> arch/loongarch/vdso/.gitignore | 2 +
> scripts/subarch.include | 2 +-
> 17 files changed, 729 insertions(+), 1 deletion(-)
> create mode 100644 arch/loongarch/Kbuild
> create mode 100644 arch/loongarch/Kconfig
> create mode 100644 arch/loongarch/Kconfig.debug
> create mode 100644 arch/loongarch/Makefile
> create mode 100644 arch/loongarch/boot/.gitignore
> create mode 100644 arch/loongarch/boot/Makefile
> create mode 100644 arch/loongarch/boot/dts/Makefile
> create mode 100644 arch/loongarch/include/asm/Kbuild
> create mode 100644 arch/loongarch/include/uapi/asm/Kbuild
> create mode 100644 arch/loongarch/kernel/.gitignore
> create mode 100644 arch/loongarch/kernel/Makefile
> create mode 100644 arch/loongarch/kernel/vmlinux.lds.S
> create mode 100644 arch/loongarch/lib/Makefile
> create mode 100644 arch/loongarch/mm/Makefile
> create mode 100644 arch/loongarch/pci/Makefile
> create mode 100644 arch/loongarch/vdso/.gitignore
>
> diff --git a/arch/loongarch/Kbuild b/arch/loongarch/Kbuild
> new file mode 100644
> index 000000000000..ab5373d0a24f
> --- /dev/null
> +++ b/arch/loongarch/Kbuild
> @@ -0,0 +1,6 @@
> +obj-y += kernel/
> +obj-y += mm/
> +obj-y += vdso/
> +
> +# for cleaning
> +subdir- += boot
> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
> new file mode 100644
> index 000000000000..09902903bba8
> --- /dev/null
> +++ b/arch/loongarch/Kconfig
> @@ -0,0 +1,395 @@
> +# SPDX-License-Identifier: GPL-2.0
> +config LOONGARCH
> + bool
> + default y
> + select ACPI_MCFG if ACPI
> + select ACPI_SYSTEM_POWER_STATES_SUPPORT if ACPI
> + select ARCH_BINFMT_ELF_STATE
> + select ARCH_ENABLE_MEMORY_HOTPLUG
> + select ARCH_ENABLE_MEMORY_HOTREMOVE
> + select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI
> + select ARCH_HAS_PTE_SPECIAL
> + select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
> + select ARCH_INLINE_READ_LOCK if !PREEMPTION
> + select ARCH_INLINE_READ_LOCK_BH if !PREEMPTION
> + select ARCH_INLINE_READ_LOCK_IRQ if !PREEMPTION
> + select ARCH_INLINE_READ_LOCK_IRQSAVE if !PREEMPTION
> + select ARCH_INLINE_READ_UNLOCK if !PREEMPTION
> + select ARCH_INLINE_READ_UNLOCK_BH if !PREEMPTION
> + select ARCH_INLINE_READ_UNLOCK_IRQ if !PREEMPTION
> + select ARCH_INLINE_READ_UNLOCK_IRQRESTORE if !PREEMPTION
> + select ARCH_INLINE_WRITE_LOCK if !PREEMPTION
> + select ARCH_INLINE_WRITE_LOCK_BH if !PREEMPTION
> + select ARCH_INLINE_WRITE_LOCK_IRQ if !PREEMPTION
> + select ARCH_INLINE_WRITE_LOCK_IRQSAVE if !PREEMPTION
> + select ARCH_INLINE_WRITE_UNLOCK if !PREEMPTION
> + select ARCH_INLINE_WRITE_UNLOCK_BH if !PREEMPTION
> + select ARCH_INLINE_WRITE_UNLOCK_IRQ if !PREEMPTION
> + select ARCH_INLINE_WRITE_UNLOCK_IRQRESTORE if !PREEMPTION
> + select ARCH_INLINE_SPIN_TRYLOCK if !PREEMPTION
> + select ARCH_INLINE_SPIN_TRYLOCK_BH if !PREEMPTION
> + select ARCH_INLINE_SPIN_LOCK if !PREEMPTION
> + select ARCH_INLINE_SPIN_LOCK_BH if !PREEMPTION
> + select ARCH_INLINE_SPIN_LOCK_IRQ if !PREEMPTION
> + select ARCH_INLINE_SPIN_LOCK_IRQSAVE if !PREEMPTION
> + select ARCH_INLINE_SPIN_UNLOCK if !PREEMPTION
> + select ARCH_INLINE_SPIN_UNLOCK_BH if !PREEMPTION
> + select ARCH_INLINE_SPIN_UNLOCK_IRQ if !PREEMPTION
> + select ARCH_INLINE_SPIN_UNLOCK_IRQRESTORE if !PREEMPTION
> + select ARCH_MIGHT_HAVE_PC_PARPORT
> + select ARCH_MIGHT_HAVE_PC_SERIO
> + select ARCH_SPARSEMEM_ENABLE
> + select ARCH_SUPPORTS_ACPI
> + select ARCH_SUPPORTS_ATOMIC_RMW
> + select ARCH_SUPPORTS_HUGETLBFS
> + select ARCH_USE_BUILTIN_BSWAP
> + select ARCH_USE_CMPXCHG_LOCKREF
> + select ARCH_USE_QUEUED_RWLOCKS
> + select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT
> + select ARCH_WANTS_NO_INSTR
> + select BUILDTIME_TABLE_SORT
> + select COMMON_CLK
> + select GENERIC_CLOCKEVENTS
> + select GENERIC_CMOS_UPDATE
> + select GENERIC_CPU_AUTOPROBE
> + select GENERIC_ENTRY
> + select GENERIC_FIND_FIRST_BIT
> + select GENERIC_GETTIMEOFDAY
> + select GENERIC_IRQ_MULTI_HANDLER
> + select GENERIC_IRQ_PROBE
> + select GENERIC_IRQ_SHOW
> + select GENERIC_LIB_ASHLDI3
> + select GENERIC_LIB_ASHRDI3
> + select GENERIC_LIB_CMPDI2
> + select GENERIC_LIB_LSHRDI3
> + select GENERIC_LIB_UCMPDI2
> + select GENERIC_PCI_IOMAP
> + select GENERIC_SCHED_CLOCK
> + select GENERIC_TIME_VSYSCALL
> + select GPIOLIB
> + select HAVE_ARCH_AUDITSYSCALL
> + select HAVE_ARCH_COMPILER_H
> + select HAVE_ARCH_MMAP_RND_BITS if MMU
> + select HAVE_ARCH_SECCOMP_FILTER
> + select HAVE_ARCH_TRACEHOOK
> + select HAVE_ARCH_TRANSPARENT_HUGEPAGE
> + select HAVE_ASM_MODVERSIONS
> + select HAVE_CONTEXT_TRACKING
> + select HAVE_COPY_THREAD_TLS
> + select HAVE_DEBUG_KMEMLEAK
> + select HAVE_DEBUG_STACKOVERFLOW
> + select HAVE_DMA_CONTIGUOUS
> + select HAVE_EXIT_THREAD
> + select HAVE_FAST_GUP
> + select HAVE_GENERIC_VDSO
> + select HAVE_IOREMAP_PROT
> + select HAVE_IRQ_EXIT_ON_IRQ_STACK
> + select HAVE_IRQ_TIME_ACCOUNTING
> + select HAVE_MEMBLOCK
> + select HAVE_MEMBLOCK_NODE_MAP
> + select HAVE_MOD_ARCH_SPECIFIC
> + select HAVE_NMI
> + select HAVE_PCI
> + select HAVE_PERF_EVENTS
> + select HAVE_REGS_AND_STACK_ACCESS_API
> + select HAVE_RSEQ
> + select HAVE_SYSCALL_TRACEPOINTS
> + select HAVE_TIF_NOHZ
> + select HAVE_VIRT_CPU_ACCOUNTING_GEN
> + select IRQ_FORCED_THREADING
> + select IRQ_LOONGARCH_CPU
> + select MODULES_USE_ELF_RELA if MODULES
> + select OF
> + select OF_EARLY_FLATTREE
> + select PCI
> + select PCI_DOMAINS_GENERIC
> + select PCI_ECAM if ACPI
> + select PCI_MSI_ARCH_FALLBACKS
> + select PERF_USE_VMALLOC
> + select RTC_LIB
> + select SPARSE_IRQ
> + select SYSCTL_EXCEPTION_TRACE
> + select SWIOTLB
> + select TRACE_IRQFLAGS_SUPPORT
> + select ZONE_DMA32
> +
> +config 32BIT
> + bool
> +
> +config 64BIT
> + def_bool y
> +
> +config CPU_HAS_FPU
> + bool
> + default y
> +
> +config CPU_HAS_PREFETCH
> + bool
> + default y
> +
> +config GENERIC_CALIBRATE_DELAY
> + def_bool y
> +
> +config GENERIC_CSUM
> + def_bool y
> +
> +config GENERIC_HWEIGHT
> + def_bool y
> +
> +config L1_CACHE_SHIFT
> + int
> + default "6"
> +
> +config LOCKDEP_SUPPORT
> + bool
> + default y
> +
> +# MACH_LOONGSON32 and MACH_LOONGSON64 are delibrately carried over from the
> +# MIPS Loongson code, to preserve Loongson-specific code paths in drivers that
> +# are shared between architectures, and specifically expecting the symbols.
> +config MACH_LOONGSON32
> + def_bool 32BIT
> +
> +config MACH_LOONGSON64
> + def_bool 64BIT
> +
> +config PAGE_SIZE_4KB
> + bool
> +
> +config PAGE_SIZE_16KB
> + bool
> +
> +config PAGE_SIZE_64KB
> + bool
> +
> +config PGTABLE_2LEVEL
> + bool
> +
> +config PGTABLE_3LEVEL
> + bool
> +
> +config PGTABLE_4LEVEL
> + bool
> +
> +config PGTABLE_LEVELS
> + int
> + default 2 if PGTABLE_2LEVEL
> + default 3 if PGTABLE_3LEVEL
> + default 4 if PGTABLE_4LEVEL
> +
> +config SCHED_OMIT_FRAME_POINTER
> + bool
> + default y
> +
> +menu "Kernel type and options"
> +
> +source "kernel/Kconfig.hz"
> +
> +choice
> + prompt "Page Table Layout"
> + default 16KB_2LEVEL if 32BIT
> + default 16KB_3LEVEL if 64BIT
> + help
> + Allows choosing the page table layout, which is a combination
> + of page size and page table levels. The size of virtual memory
> + address space are determined by the page table layout.
> +
> +config 4KB_3LEVEL
> + bool "4KB with 3 levels"
> + select PAGE_SIZE_4KB
> + select PGTABLE_3LEVEL
> + help
> + This option selects 4KB page size with 3 level page tables, which
> + support a maximum of 39 bits of application virtual memory.
> +
> +config 4KB_4LEVEL
> + bool "4KB with 4 levels"
> + select PAGE_SIZE_4KB
> + select PGTABLE_4LEVEL
> + help
> + This option selects 4KB page size with 4 level page tables, which
> + support a maximum of 48 bits of application virtual memory.
> +
> +config 16KB_2LEVEL
> + bool "16KB with 2 levels"
> + select PAGE_SIZE_16KB
> + select PGTABLE_2LEVEL
> + help
> + This option selects 16KB page size with 2 level page tables, which
> + support a maximum of 36 bits of application virtual memory.
> +
> +config 16KB_3LEVEL
> + bool "16KB with 3 levels"
> + select PAGE_SIZE_16KB
> + select PGTABLE_3LEVEL
> + help
> + This option selects 16KB page size with 3 level page tables, which
> + support a maximum of 47 bits of application virtual memory.
> +
> +config 64KB_2LEVEL
> + bool "64KB with 2 levels"
> + select PAGE_SIZE_64KB
> + select PGTABLE_2LEVEL
> + help
> + This option selects 64KB page size with 2 level page tables, which
> + support a maximum of 42 bits of application virtual memory.
> +
> +config 64KB_3LEVEL
> + bool "64KB with 3 levels"
> + select PAGE_SIZE_64KB
> + select PGTABLE_3LEVEL
> + help
> + This option selects 64KB page size with 3 level page tables, which
> + support a maximum of 55 bits of application virtual memory.
> +
> +endchoice
> +
> +config CMDLINE
> + string "Built-in kernel command line"
> + help
> + For most platforms, the arguments for the kernel's command line
> + are provided at run-time, during boot. However, there are cases
> + where either no arguments are being provided or the provided
> + arguments are insufficient or even invalid.
> +
> + When that occurs, it is possible to define a built-in command
> + line here and choose how the kernel should use it later on.
> +
> +choice
> + prompt "Kernel command line type"
> + default CMDLINE_BOOTLOADER
> + help
> + Choose how the kernel will handle the provided built-in command
> + line.
> +
> +config CMDLINE_BOOTLOADER
> + bool "Use bootloader kernel arguments if available"
> + help
> + Prefer the command-line passed by the boot loader if available.
> + Use the built-in command line as fallback in case we get nothing
> + during boot. This is the default behaviour.
> +
> +config CMDLINE_EXTEND
> + bool "Use built-in to extend bootloader kernel arguments"
> + help
> + The command-line arguments provided during boot will be
> + appended to the built-in command line. This is useful in
> + cases where the provided arguments are insufficient and
> + you don't want to or cannot modify them.
> +
> +config CMDLINE_FORCE
> + bool "Always use the built-in kernel command string"
> + help
> + Always use the built-in command line, even if we get one during
> + boot. This is useful in case you need to override the provided
> + command line on systems where you don't have or want control
> + over it.
> +
> +endchoice
> +
> +config DMI
> + bool "Enable DMI scanning"
> + select DMI_SCAN_MACHINE_NON_EFI_FALLBACK
> + default y
> + help
> + This enables SMBIOS/DMI feature for systems, and scanning of
> + DMI to identify machine quirks.
> +
> +config EFI
> + bool "EFI runtime service support"
> + select UCS2_STRING
> + select EFI_PARAMS_FROM_FDT
> + select EFI_RUNTIME_WRAPPERS
> + help
> + This enables the kernel to use EFI runtime services that are
> + available (such as the EFI variable services).
> +
> +config EFI_STUB
> + bool "EFI boot stub support"
> + default y
> + depends on EFI
> + select EFI_GENERIC_STUB
> + help
> + This kernel feature allows the kernel to be loaded directly by
> + EFI firmware without the use of a bootloader.
> +
> +config FORCE_MAX_ZONEORDER
> + int "Maximum zone order"
> + range 14 64 if PAGE_SIZE_64KB
> + default "14" if PAGE_SIZE_64KB
> + range 12 64 if PAGE_SIZE_16KB
> + default "12" if PAGE_SIZE_16KB
> + range 11 64
> + default "11"
> + help
> + The kernel memory allocator divides physically contiguous memory
> + blocks into "zones", where each zone is a power of two number of
> + pages. This option selects the largest power of two that the kernel
> + keeps in the memory allocator. If you need to allocate very large
> + blocks of physically contiguous memory, then you may need to
> + increase this value.
> +
> + This config option is actually maximum order plus one. For example,
> + a value of 11 means that the largest free memory block is 2^10 pages.
> +
> + The page size is not necessarily 4KB. Keep this in mind
> + when choosing a value for this option.
> +
> +config SECCOMP
> + bool "Enable seccomp to safely compute untrusted bytecode"
> + depends on PROC_FS
> + default y
> + help
> + This kernel feature is useful for number crunching applications
> + that may need to compute untrusted bytecode during their
> + execution. By using pipes or other transports made available to
> + the process as file descriptors supporting the read/write
> + syscalls, it's possible to isolate those applications in
> + their own address space using seccomp. Once seccomp is
> + enabled via /proc/<pid>/seccomp, it cannot be disabled
> + and the task is only allowed to execute a few safe syscalls
> + defined by each seccomp mode.
> +
> + If unsure, say Y. Only embedded should say N here.
> +
> +endmenu
> +
> +config ARCH_SELECT_MEMORY_MODEL
> + def_bool y
> +
> +config ARCH_FLATMEM_ENABLE
> + def_bool y
> +
> +config ARCH_SPARSEMEM_ENABLE
> + def_bool y
> + help
> + Say Y to support efficient handling of sparse physical memory,
> + for architectures which are either NUMA (Non-Uniform Memory Access)
> + or have huge holes in the physical address space for other reasons.
> + See <file:Documentation/vm/numa.rst> for more.
> +
> +config ARCH_ENABLE_THP_MIGRATION
> + def_bool y
> + depends on TRANSPARENT_HUGEPAGE
> +
> +config ARCH_MEMORY_PROBE
> + def_bool y
> + depends on MEMORY_HOTPLUG
> +
> +config MMU
> + bool
> + default y
> +
> +config ARCH_MMAP_RND_BITS_MIN
> + default 12
> +
> +config ARCH_MMAP_RND_BITS_MAX
> + default 18
> +
> +menu "Power management options"
> +
> +source "drivers/acpi/Kconfig"
> +
> +endmenu
> +
> +source "drivers/firmware/Kconfig"
> diff --git a/arch/loongarch/Kconfig.debug b/arch/loongarch/Kconfig.debug
> new file mode 100644
> index 000000000000..e69de29bb2d1
> diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile
> new file mode 100644
> index 000000000000..6dcc0495bec9
> --- /dev/null
> +++ b/arch/loongarch/Makefile
> @@ -0,0 +1,105 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Author: Huacai Chen <[email protected]>
> +# Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> +
> +boot := arch/loongarch/boot
> +
> +ifndef CONFIG_EFI_STUB
> +KBUILD_IMAGE = $(boot)/vmlinux
> +else
> +KBUILD_IMAGE = $(boot)/vmlinux.efi
> +endif
> +
> +#
> +# Select the object file format to substitute into the linker script.
> +#
> +64bit-tool-archpref = loongarch64
> +32bit-bfd = elf32-loongarch
> +64bit-bfd = elf64-loongarch
> +32bit-emul = elf32loongarch
> +64bit-emul = elf64loongarch
> +
> +ifdef CONFIG_64BIT
> +tool-archpref = $(64bit-tool-archpref)
> +UTS_MACHINE := loongarch64
> +endif
> +
> +ifneq ($(SUBARCH),$(ARCH))
> + ifeq ($(CROSS_COMPILE),)
> + CROSS_COMPILE := $(call cc-cross-prefix, $(tool-archpref)-linux- $(tool-archpref)-linux-gnu- $(tool-archpref)-unknown-linux-gnu-)
> + endif
> +endif
> +
> +ifdef CONFIG_64BIT
> +ld-emul = $(64bit-emul)
> +cflags-y += -mabi=lp64s
> +endif
> +
> +cflags-y += -G0 -pipe -msoft-float
> +LDFLAGS_vmlinux += -G0 -static -n -nostdlib
> +KBUILD_AFLAGS_KERNEL += -Wa,-mla-global-with-pcrel
> +KBUILD_CFLAGS_KERNEL += -Wa,-mla-global-with-pcrel
> +KBUILD_AFLAGS_MODULE += -Wa,-mla-global-with-abs
> +KBUILD_CFLAGS_MODULE += -fplt -Wa,-mla-global-with-abs,-mla-local-with-abs
> +
> +cflags-y += -ffreestanding
> +cflags-y += $(call cc-option, -mno-check-zero-division)
> +
> +load-y = 0x9000000000200000
> +bootvars-y = VMLINUX_LOAD_ADDRESS=$(load-y)
> +
> +drivers-$(CONFIG_PCI) += arch/loongarch/pci/
> +
> +KBUILD_AFLAGS += $(cflags-y)
> +KBUILD_CFLAGS += $(cflags-y)
> +KBUILD_CPPFLAGS += -DVMLINUX_LOAD_ADDRESS=$(load-y)
> +
> +# This is required to get dwarf unwinding tables into .debug_frame
> +# instead of .eh_frame so we don't discard them.
> +KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
> +
> +# Don't emit unaligned accesses.
> +# Not all LoongArch cores support unaligned access, and as kernel we can't
> +# rely on others to provide emulation for these accesses.
> +KBUILD_CFLAGS += $(call cc-option,-mstrict-align)
> +
> +KBUILD_CFLAGS += -isystem $(shell $(CC) -print-file-name=include)
> +
> +KBUILD_LDFLAGS += -m $(ld-emul)
> +
> +ifdef CONFIG_LOONGARCH
> +CHECKFLAGS += $(shell $(CC) $(KBUILD_CFLAGS) -dM -E -x c /dev/null | \
> + egrep -vw '__GNUC_(MINOR_|PATCHLEVEL_)?_' | \
> + sed -e "s/^\#define /-D'/" -e "s/ /'='/" -e "s/$$/'/" -e 's/\$$/&&/g')
> +endif
> +
> +head-y := arch/loongarch/kernel/head.o
> +
> +libs-y += arch/loongarch/lib/
> +libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
> +
> +ifeq ($(KBUILD_EXTMOD),)
> +prepare: vdso_prepare
> +vdso_prepare: prepare0
> + $(Q)$(MAKE) $(build)=arch/loongarch/vdso include/generated/vdso-offsets.h
> +endif
> +
> +PHONY += vdso_install
> +vdso_install:
> + $(Q)$(MAKE) $(build)=arch/loongarch/vdso $@
> +
> +all: $(KBUILD_IMAGE)
> +
> +$(KBUILD_IMAGE): vmlinux
> + $(Q)$(MAKE) $(build)=$(boot) $(bootvars-y) $@
> +
> +install:
> + $(Q)install -D -m 755 $(KBUILD_IMAGE) $(INSTALL_PATH)/vmlinux-$(KERNELRELEASE)
> + $(Q)install -D -m 644 .config $(INSTALL_PATH)/config-$(KERNELRELEASE)
> + $(Q)install -D -m 644 System.map $(INSTALL_PATH)/System.map-$(KERNELRELEASE)
> +
> +define archhelp
> + echo ' install - install kernel into $(INSTALL_PATH)'
> + echo
> +endef
> diff --git a/arch/loongarch/boot/.gitignore b/arch/loongarch/boot/.gitignore
> new file mode 100644
> index 000000000000..49423ee96ef3
> --- /dev/null
> +++ b/arch/loongarch/boot/.gitignore
> @@ -0,0 +1,2 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +vmlinux*
> diff --git a/arch/loongarch/boot/Makefile b/arch/loongarch/boot/Makefile
> new file mode 100644
> index 000000000000..b39d50a7a3df
> --- /dev/null
> +++ b/arch/loongarch/boot/Makefile
> @@ -0,0 +1,20 @@
> +#
> +# arch/loongarch/boot/Makefile
> +#
> +# Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> +#
> +
> +drop-sections := .comment .note .options .note.gnu.build-id
> +strip-flags := $(addprefix --remove-section=,$(drop-sections)) -S
> +OBJCOPYFLAGS_vmlinux.efi := -O binary $(strip-flags)
> +
> +targets := vmlinux
> +quiet_cmd_strip = STRIP $@
> + cmd_strip = $(STRIP) -s -o $@ $<
> +
> +$(obj)/vmlinux: vmlinux FORCE
> + $(call if_changed,strip)
> +
> +targets += vmlinux.efi
> +$(obj)/vmlinux.efi: $(obj)/vmlinux FORCE
> + $(call if_changed,objcopy)
> diff --git a/arch/loongarch/boot/dts/Makefile b/arch/loongarch/boot/dts/Makefile
> new file mode 100644
> index 000000000000..5f1f55e911ad
> --- /dev/null
> +++ b/arch/loongarch/boot/dts/Makefile
> @@ -0,0 +1,4 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +dtstree := $(srctree)/$(src)
> +
> +dtb-y := $(patsubst $(dtstree)/%.dts,%.dtb, $(wildcard $(dtstree)/*.dts))
> diff --git a/arch/loongarch/include/asm/Kbuild b/arch/loongarch/include/asm/Kbuild
> new file mode 100644
> index 000000000000..83bc0681e72b
> --- /dev/null
> +++ b/arch/loongarch/include/asm/Kbuild
> @@ -0,0 +1,30 @@
> +# SPDX-License-Identifier: GPL-2.0
> +generic-y += dma-contiguous.h
> +generic-y += export.h
> +generic-y += parport.h
> +generic-y += early_ioremap.h
> +generic-y += qrwlock.h
> +generic-y += qrwlock_types.h
> +generic-y += spinlock.h
> +generic-y += spinlock_types.h
> +generic-y += rwsem.h
> +generic-y += segment.h
> +generic-y += user.h
> +generic-y += stat.h
> +generic-y += fcntl.h
> +generic-y += ioctl.h
> +generic-y += ioctls.h
> +generic-y += mman.h
> +generic-y += msgbuf.h
> +generic-y += sembuf.h
> +generic-y += shmbuf.h
> +generic-y += statfs.h
> +generic-y += socket.h
> +generic-y += sockios.h
> +generic-y += termios.h
> +generic-y += termbits.h
> +generic-y += poll.h
> +generic-y += param.h
> +generic-y += posix_types.h
> +generic-y += resource.h
> +generic-y += kvm_para.h
> diff --git a/arch/loongarch/include/uapi/asm/Kbuild b/arch/loongarch/include/uapi/asm/Kbuild
> new file mode 100644
> index 000000000000..4aa680ca2e5f
> --- /dev/null
> +++ b/arch/loongarch/include/uapi/asm/Kbuild
> @@ -0,0 +1,2 @@
> +# SPDX-License-Identifier: GPL-2.0
> +generic-y += kvm_para.h
> diff --git a/arch/loongarch/kernel/.gitignore b/arch/loongarch/kernel/.gitignore
> new file mode 100644
> index 000000000000..bbb90f92d051
> --- /dev/null
> +++ b/arch/loongarch/kernel/.gitignore
> @@ -0,0 +1,2 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +vmlinux.lds
> diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
> new file mode 100644
> index 000000000000..e5a3b2fb9961
> --- /dev/null
> +++ b/arch/loongarch/kernel/Makefile
> @@ -0,0 +1,21 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for the Linux/LoongArch kernel.
> +#
> +
> +extra-y := head.o vmlinux.lds
> +
> +obj-y += cpu-probe.o cacheinfo.o env.o setup.o entry.o genex.o \
> + traps.o irq.o idle.o process.o dma.o mem.o io.o reset.o switch.o \
> + elf.o syscall.o signal.o time.o topology.o inst.o ptrace.o vdso.o
> +
> +obj-$(CONFIG_ACPI) += acpi.o
> +obj-$(CONFIG_EFI) += efi.o
> +
> +obj-$(CONFIG_CPU_HAS_FPU) += fpu.o
> +
> +obj-$(CONFIG_MODULES) += module.o module-sections.o
> +
> +obj-$(CONFIG_PROC_FS) += proc.o
> +
> +CPPFLAGS_vmlinux.lds := $(KBUILD_CFLAGS)
> diff --git a/arch/loongarch/kernel/vmlinux.lds.S b/arch/loongarch/kernel/vmlinux.lds.S
> new file mode 100644
> index 000000000000..7da4c4d7c50d
> --- /dev/null
> +++ b/arch/loongarch/kernel/vmlinux.lds.S
> @@ -0,0 +1,117 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#include <linux/sizes.h>
> +#include <asm/asm-offsets.h>
> +#include <asm/thread_info.h>
> +
> +#define PAGE_SIZE _PAGE_SIZE
> +
> +/*
> + * Put .bss..swapper_pg_dir as the first thing in .bss. This will
> + * ensure that it has .bss alignment (64K).
> + */
> +#define BSS_FIRST_SECTIONS *(.bss..swapper_pg_dir)
> +
> +#include <asm-generic/vmlinux.lds.h>
> +#include "image-vars.h"
> +
> +/*
> + * Max avaliable Page Size is 64K, so we set SectionAlignment
> + * field of EFI application to 64K.
> + */
> +PECOFF_FILE_ALIGN = 0x200;
> +PECOFF_SEGMENT_ALIGN = 0x10000;
> +
> +OUTPUT_ARCH(loongarch)
> +ENTRY(kernel_entry)
> +PHDRS {
> + text PT_LOAD FLAGS(7); /* RWX */
> + note PT_NOTE FLAGS(4); /* R__ */
> +}
> +
> +jiffies = jiffies_64;
> +
> +SECTIONS
> +{
> + . = VMLINUX_LOAD_ADDRESS;
> +
> + _text = .;
> + HEAD_TEXT_SECTION
> +
> + . = ALIGN(PECOFF_SEGMENT_ALIGN);
> + .text : {
> + TEXT_TEXT
> + SCHED_TEXT
> + CPUIDLE_TEXT
> + LOCK_TEXT
> + KPROBES_TEXT
> + IRQENTRY_TEXT
> + SOFTIRQENTRY_TEXT
> + *(.fixup)
> + *(.gnu.warning)
> + } :text = 0
> + . = ALIGN(PECOFF_SEGMENT_ALIGN);
> + _etext = .;
> +
> + EXCEPTION_TABLE(16)
> +
> + . = ALIGN(PECOFF_SEGMENT_ALIGN);
> + __init_begin = .;
> + __inittext_begin = .;
> +
> + INIT_TEXT_SECTION(PAGE_SIZE)
> + .exit.text : {
> + EXIT_TEXT
> + }
> +
> + . = ALIGN(PECOFF_SEGMENT_ALIGN);
> + __inittext_end = .;
> +
> + __initdata_begin = .;
> +
> + INIT_DATA_SECTION(16)
> + .exit.data : {
> + EXIT_DATA
> + }
> +
> + .init.bss : {
> + *(.init.bss)
> + }
> + . = ALIGN(PECOFF_SEGMENT_ALIGN);
> + __initdata_end = .;
> +
> + __init_end = .;
> +
> + _sdata = .;
> + RO_DATA(4096)
> + RW_DATA(1 << CONFIG_L1_CACHE_SHIFT, PAGE_SIZE, THREAD_SIZE)
> +
> + .sdata : {
> + *(.sdata)
> + }
> + .edata_padding : { BYTE(0); . = ALIGN(PECOFF_FILE_ALIGN); }
> + _edata = .;
> +
> + BSS_SECTION(0, SZ_64K, 8)
> + . = ALIGN(PECOFF_SEGMENT_ALIGN);
> +
> + _end = .;
> +
> + STABS_DEBUG
> + DWARF_DEBUG
> +
> + .gptab.sdata : {
> + *(.gptab.data)
> + *(.gptab.sdata)
> + }
> + .gptab.sbss : {
> + *(.gptab.bss)
> + *(.gptab.sbss)
> + }
> +
> + DISCARDS
> + /DISCARD/ : {
> + *(.gnu.attributes)
> + *(.options)
> + *(.eh_frame)
> + }
> +}
> diff --git a/arch/loongarch/lib/Makefile b/arch/loongarch/lib/Makefile
> new file mode 100644
> index 000000000000..e36635fccb69
> --- /dev/null
> +++ b/arch/loongarch/lib/Makefile
> @@ -0,0 +1,6 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for LoongArch-specific library files.
> +#
> +
> +lib-y += delay.o clear_user.o copy_user.o dump_tlb.o
> diff --git a/arch/loongarch/mm/Makefile b/arch/loongarch/mm/Makefile
> new file mode 100644
> index 000000000000..8ffc6383f836
> --- /dev/null
> +++ b/arch/loongarch/mm/Makefile
> @@ -0,0 +1,9 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for the Linux/LoongArch-specific parts of the memory manager.
> +#
> +
> +obj-y += init.o cache.o tlb.o tlbex.o extable.o \
> + fault.o ioremap.o maccess.o mmap.o pgtable.o page.o
> +
> +obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
> diff --git a/arch/loongarch/pci/Makefile b/arch/loongarch/pci/Makefile
> new file mode 100644
> index 000000000000..8101ef3df71c
> --- /dev/null
> +++ b/arch/loongarch/pci/Makefile
> @@ -0,0 +1,7 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for the PCI specific kernel interface routines under Linux.
> +#
> +
> +obj-y += pci.o
> +obj-$(CONFIG_ACPI) += acpi.o
> diff --git a/arch/loongarch/vdso/.gitignore b/arch/loongarch/vdso/.gitignore
> new file mode 100644
> index 000000000000..652e31d82582
> --- /dev/null
> +++ b/arch/loongarch/vdso/.gitignore
> @@ -0,0 +1,2 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +vdso.lds
> diff --git a/scripts/subarch.include b/scripts/subarch.include
> index 776849a3c500..4bd327d0ae42 100644
> --- a/scripts/subarch.include
> +++ b/scripts/subarch.include
> @@ -10,4 +10,4 @@ SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \
> -e s/s390x/s390/ \
> -e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
> -e s/sh[234].*/sh/ -e s/aarch64.*/arm64/ \
> - -e s/riscv.*/riscv/)
> + -e s/riscv.*/riscv/ -e s/loongarch.*/loongarch/)
> --
> 2.27.0
>


--
Best Regards
Guo Ren

ML: https://lore.kernel.org/linux-csky/