This series adds KVM LoongArch support. Loongson 3A5000 supports hardware
assisted virtualization. With cpu virtualization, there are separate
hw-supported user mode and kernel mode in guest mode. With memory
virtualization, there are two-level hw mmu table for guest mode and host
mode. Also there is separate hw cpu timer with consant frequency in
guest mode, so that vm can migrate between hosts with different freq.
Currently, we are able to boot LoongArch Linux Guests.
Few key aspects of KVM LoongArch added by this series are:
1. Enable kvm hardware function when kvm module is loaded.
2. Implement VM and vcpu related ioctl interface such as vcpu create,
vcpu run etc. GET_ONE_REG/SET_ONE_REG ioctl commands are use to
get general registers one by one.
3. Hardware access about MMU, timer and csr are emulated in kernel.
4. Hardwares such as mmio and iocsr device are emulated in user space
such as APIC, IPI, pci devices etc.
The running environment of LoongArch virt machine:
1. Cross tools to build kernel and uefi:
$ wget https://github.com/loongson/build-tools/releases/download/2022.09.06/loongarch64-clfs-6.3-cross-tools-gcc-glibc.tar.xz
tar -vxf loongarch64-clfs-6.3-cross-tools-gcc-glibc.tar.xz -C /opt
export PATH=/opt/cross-tools/bin:$PATH
export LD_LIBRARY_PATH=/opt/cross-tools/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/cross-tools/loongarch64-unknown-linux-gnu/lib/:$LD_LIBRARY_PATH
2. This series is based on the linux source code:
https://github.com/loongson/linux-loongarch-kvm
Build command:
git checkout kvm-loongarch
make ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu- loongson3_defconfig
make ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu-
3. QEMU hypervisor with LoongArch supported:
https://github.com/loongson/qemu
Build command:
git checkout kvm-loongarch
./configure --target-list="loongarch64-softmmu" --enable-kvm
make
4. Uefi bios of LoongArch virt machine:
Link: https://github.com/tianocore/edk2-platforms/tree/master/Platform/Loongson/LoongArchQemuPkg#readme
5. you can also access the binary files we have already build:
https://github.com/yangxiaojuan-loongson/qemu-binary
The command to boot loongarch virt machine:
$ qemu-system-loongarch64 -machine virt -m 4G -cpu la464 \
-smp 1 -bios QEMU_EFI.fd -kernel vmlinuz.efi -initrd ramdisk \
-serial stdio -monitor telnet:localhost:4495,server,nowait \
-append "root=/dev/ram rdinit=/sbin/init console=ttyS0,115200" \
--nographic
Changes for v13:
1. Remove patch-28 "Implement probe virtualization when cpu init", as the
virtualization information about FPU,PMP,LSX in guest.options,options_dyn
is not used and the gcfg reg value can be read in kvm_hardware_enable, so
remove the previous cpu_probe_lvz function.
2. Fix vcpu_enable_cap interface, it should return -EINVAL directly, as
FPU cap is enable by default, and do not support any other caps now.
3. Simplify the jirl instruction with jr when without return addr,
simplify case HW0 ... HW7 statment in interrupt.c
4. Rename host_stack,host_gp in kvm_vcpu_arch to host_sp,host_tp.
5. Remove 'cpu' parameter in _kvm_check_requests, as 'cpu' is not used,
and remove 'cpu' parameter in kvm_check_vmid function, as it can get
cpu number by itself.
Changes for v12:
1. Improve the gcsr write/read/xchg interface to avoid the previous
instruction statment like parse_r and make the code easy understanding,
they are implemented in asm/insn-def.h and the instructions consistent
of "opcode" "rj" "rd" "simm14" arguments.
2. Fix the maintainers list of LoongArch KVM.
Changes for v11:
1. Add maintainers for LoongArch KVM.
Changes for v10:
1. Fix grammatical problems in LoongArch documentation.
2. It is not necessary to save or restore the LOONGARCH_CSR_PGD when
vcpu put and vcpu load, so we remove it.
Changes for v9:
1. Apply the new defined interrupt number macros in loongarch.h to kvm,
such as INT_SWI0, INT_HWI0, INT_TI, INT_IPI, etc. And remove the
previous unused macros.
2. Remove unused variables in kvm_vcpu_arch, and reorder the variables
to make them more standard.
Changes for v8:
1. Adjust the cpu_data.guest.options structure, add the ases flag into
it, and remove the previous guest.ases. We do this to keep consistent
with host cpu_data.options structure.
2. Remove the "#include <asm/kvm_host.h>" in some files which also
include the "<linux/kvm_host.h>". As linux/kvm_host.h already include
the asm/kvm_host.h.
3. Fix some unstandard spelling and grammar errors in comments, and
improve a little code format to make it easier and standard.
Changes for v7:
1. Fix the kvm_save/restore_hw_gcsr compiling warnings reported by
kernel test robot. The report link is:
https://lore.kernel.org/oe-kbuild-all/[email protected]/
2. Fix loongarch kvm trace related compiling problems.
Changes for v6:
1. Fix the Documentation/virt/kvm/api.rst compile warning about
loongarch parts.
Changes for v5:
1. Implement get/set mp_state ioctl interface, and only the
KVM_MP_STATE_RUNNABLE state is supported now, and other states
will be completed in the future. The state is also used when vcpu
run idle instruction, if vcpu state is changed to RUNNABLE, the
vcpu will have the possibility to be woken up.
2. Supplement kvm document about loongarch-specific part, such as add
api introduction for GET/SET_ONE_REG, GET/SET_FPU, GET/SET_MP_STATE,
etc.
3. Improve the kvm_switch_to_guest function in switch.S, remove the
previous tmp,tmp1 arguments and replace it with t0,t1 reg.
Changes for v4:
1. Add a csr_need_update flag in _vcpu_put, as most csr registers keep
unchanged during process context switch, so we need not to update it
every time. We can do this only if the soft csr is different form hardware.
That is to say all of csrs should update after vcpu enter guest, as for
set_csr_ioctl, we have written soft csr to keep consistent with hardware.
2. Improve get/set_csr_ioctl interface, we set SW or HW or INVALID flag
for all csrs according to it's features when kvm init. In get/set_csr_ioctl,
if csr is HW, we use gcsrrd/ gcsrwr instruction to access it, else if csr is
SW, we use software to emulate it, and others return false.
3. Add set_hw_gcsr function in csr_ops.S, and it is used in set_csr_ioctl.
We have splited hw gcsr into three parts, so we can calculate the code offset
by gcsrid and jump here to run the gcsrwr instruction. We use this function to
make the code easier and avoid to use the previous SET_HW_GCSR(XXX) interface.
4. Improve kvm mmu functions, such as flush page table and make clean page table
interface.
Changes for v3:
1. Remove the vpid array list in kvm_vcpu_arch and use a vpid variable here,
because a vpid will never be recycled if a vCPU migrates from physical CPU A
to B and back to A.
2. Make some constant variables in kvm_context to global such as vpid_mask,
guest_eentry, enter_guest, etc.
3. Add some new tracepoints, such as kvm_trace_idle, kvm_trace_cache,
kvm_trace_gspr, etc.
4. There are some duplicate codes in kvm_handle_exit and kvm_vcpu_run,
so we move it to a new function kvm_pre_enter_guest.
5. Change the RESUME_HOST, RESUME_GUEST value, return 1 for resume guest
and "<= 0" for resume host.
6. Fcsr and fpu registers are saved/restored together.
Changes for v2:
1. Seprate the original patch-01 and patch-03 into small patches, and the
patches mainly contain kvm module init, module exit, vcpu create, vcpu run,
etc.
2. Remove the original KVM_{GET,SET}_CSRS ioctl in the kvm uapi header,
and we use the common KVM_{GET,SET}_ONE_REG to access register.
3. Use BIT(x) to replace the "1 << n_bits" statement.
Tianrui Zhao (30):
LoongArch: KVM: Add kvm related header files
LoongArch: KVM: Implement kvm module related interface
LoongArch: KVM: Implement kvm hardware enable, disable interface
LoongArch: KVM: Implement VM related functions
LoongArch: KVM: Add vcpu related header files
LoongArch: KVM: Implement vcpu create and destroy interface
LoongArch: KVM: Implement vcpu run interface
LoongArch: KVM: Implement vcpu handle exit interface
LoongArch: KVM: Implement vcpu get, vcpu set registers
LoongArch: KVM: Implement vcpu ENABLE_CAP ioctl interface
LoongArch: KVM: Implement fpu related operations for vcpu
LoongArch: KVM: Implement vcpu interrupt operations
LoongArch: KVM: Implement misc vcpu related interfaces
LoongArch: KVM: Implement vcpu load and vcpu put operations
LoongArch: KVM: Implement vcpu status description
LoongArch: KVM: Implement update VM id function
LoongArch: KVM: Implement virtual machine tlb operations
LoongArch: KVM: Implement vcpu timer operations
LoongArch: KVM: Implement kvm mmu operations
LoongArch: KVM: Implement handle csr excption
LoongArch: KVM: Implement handle iocsr exception
LoongArch: KVM: Implement handle idle exception
LoongArch: KVM: Implement handle gspr exception
LoongArch: KVM: Implement handle mmio exception
LoongArch: KVM: Implement handle fpu exception
LoongArch: KVM: Implement kvm exception vector
LoongArch: KVM: Implement vcpu world switch
LoongArch: KVM: Enable kvm config and add the makefile
LoongArch: KVM: Supplement kvm document about LoongArch-specific part
LoongArch: KVM: Add maintainers for LoongArch KVM
Documentation/virt/kvm/api.rst | 71 +-
MAINTAINERS | 12 +
arch/loongarch/Kbuild | 1 +
arch/loongarch/Kconfig | 2 +
arch/loongarch/configs/loongson3_defconfig | 2 +
arch/loongarch/include/asm/insn-def.h | 55 ++
arch/loongarch/include/asm/inst.h | 16 +
arch/loongarch/include/asm/kvm_csr.h | 231 ++++++
arch/loongarch/include/asm/kvm_host.h | 253 ++++++
arch/loongarch/include/asm/kvm_types.h | 11 +
arch/loongarch/include/asm/kvm_vcpu.h | 97 +++
arch/loongarch/include/asm/loongarch.h | 20 +-
arch/loongarch/include/uapi/asm/kvm.h | 106 +++
arch/loongarch/kernel/asm-offsets.c | 32 +
arch/loongarch/kvm/Kconfig | 38 +
arch/loongarch/kvm/Makefile | 22 +
arch/loongarch/kvm/csr_ops.S | 76 ++
arch/loongarch/kvm/exit.c | 707 +++++++++++++++++
arch/loongarch/kvm/interrupt.c | 113 +++
arch/loongarch/kvm/main.c | 347 ++++++++
arch/loongarch/kvm/mmu.c | 725 +++++++++++++++++
arch/loongarch/kvm/switch.S | 301 +++++++
arch/loongarch/kvm/timer.c | 266 +++++++
arch/loongarch/kvm/tlb.c | 32 +
arch/loongarch/kvm/trace.h | 168 ++++
arch/loongarch/kvm/vcpu.c | 869 +++++++++++++++++++++
arch/loongarch/kvm/vm.c | 76 ++
arch/loongarch/kvm/vmid.c | 66 ++
include/uapi/linux/kvm.h | 9 +
29 files changed, 4710 insertions(+), 14 deletions(-)
create mode 100644 arch/loongarch/include/asm/insn-def.h
create mode 100644 arch/loongarch/include/asm/kvm_csr.h
create mode 100644 arch/loongarch/include/asm/kvm_host.h
create mode 100644 arch/loongarch/include/asm/kvm_types.h
create mode 100644 arch/loongarch/include/asm/kvm_vcpu.h
create mode 100644 arch/loongarch/include/uapi/asm/kvm.h
create mode 100644 arch/loongarch/kvm/Kconfig
create mode 100644 arch/loongarch/kvm/Makefile
create mode 100644 arch/loongarch/kvm/csr_ops.S
create mode 100644 arch/loongarch/kvm/exit.c
create mode 100644 arch/loongarch/kvm/interrupt.c
create mode 100644 arch/loongarch/kvm/main.c
create mode 100644 arch/loongarch/kvm/mmu.c
create mode 100644 arch/loongarch/kvm/switch.S
create mode 100644 arch/loongarch/kvm/timer.c
create mode 100644 arch/loongarch/kvm/tlb.c
create mode 100644 arch/loongarch/kvm/trace.h
create mode 100644 arch/loongarch/kvm/vcpu.c
create mode 100644 arch/loongarch/kvm/vm.c
create mode 100644 arch/loongarch/kvm/vmid.c
--
2.39.1
Implement vcpu handle exit interface, getting the exit code by ESTAT
register and using kvm exception vector to handle it.
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/vcpu.c | 45 +++++++++++++++++++++++++++++++++++++++
1 file changed, 45 insertions(+)
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index eba5c07b8be3..a45e9d9efe5b 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -52,6 +52,51 @@ static void kvm_pre_enter_guest(struct kvm_vcpu *vcpu)
vcpu->arch.aux_inuse &= ~KVM_LARCH_CSR;
}
+/*
+ * Return 1 for resume guest and "<= 0" for resume host.
+ */
+static int _kvm_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu)
+{
+ unsigned long exst = vcpu->arch.host_estat;
+ u32 intr = exst & 0x1fff; /* ignore NMI */
+ u32 exccode = (exst & CSR_ESTAT_EXC) >> CSR_ESTAT_EXC_SHIFT;
+ int ret = RESUME_GUEST;
+
+ vcpu->mode = OUTSIDE_GUEST_MODE;
+
+ /* Set a default exit reason */
+ run->exit_reason = KVM_EXIT_UNKNOWN;
+
+ local_irq_enable();
+ guest_state_exit_irqoff();
+
+ trace_kvm_exit(vcpu, exccode);
+ if (exccode) {
+ ret = _kvm_handle_fault(vcpu, exccode);
+ } else {
+ WARN(!intr, "vm exiting with suspicious irq\n");
+ ++vcpu->stat.int_exits;
+ }
+
+ cond_resched();
+ local_irq_disable();
+
+ if (ret == RESUME_HOST)
+ return ret;
+
+ /* Only check for signals if not already exiting to userspace */
+ if (signal_pending(current)) {
+ vcpu->run->exit_reason = KVM_EXIT_INTR;
+ ++vcpu->stat.signal_exits;
+ return -EINTR;
+ }
+
+ kvm_pre_enter_guest(vcpu);
+ trace_kvm_reenter(vcpu);
+ guest_state_enter_irqoff();
+ return RESUME_GUEST;
+}
+
int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
{
unsigned long timer_hz;
--
2.39.1
Implement LoongArch virtual machine tlb operations such as flush tlb by
specific gpa parameter and flush all of the virt machines tlb.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/tlb.c | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
create mode 100644 arch/loongarch/kvm/tlb.c
diff --git a/arch/loongarch/kvm/tlb.c b/arch/loongarch/kvm/tlb.c
new file mode 100644
index 000000000000..3475b76492ad
--- /dev/null
+++ b/arch/loongarch/kvm/tlb.c
@@ -0,0 +1,32 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#include <linux/kvm_host.h>
+#include <asm/tlb.h>
+#include <asm/kvm_csr.h>
+
+int kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa)
+{
+ preempt_disable();
+ gpa &= (PAGE_MASK << 1);
+ invtlb(INVTLB_GID_ADDR, read_csr_gstat() & CSR_GSTAT_GID, gpa);
+ preempt_enable();
+ return 0;
+}
+
+/**
+ * kvm_flush_tlb_all() - Flush all root TLB entries for
+ * guests.
+ *
+ * Invalidate all entries including GVA-->GPA and GPA-->HPA mappings.
+ */
+void kvm_flush_tlb_all(void)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+ invtlb_all(INVTLB_ALLGID, 0, 0);
+ local_irq_restore(flags);
+}
--
2.39.1
Implement kvm handle vcpu iocsr exception, setting the iocsr info into
vcpu_run and return to user space to handle it.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/include/asm/inst.h | 16 ++++++
arch/loongarch/kvm/exit.c | 92 +++++++++++++++++++++++++++++++
2 files changed, 108 insertions(+)
diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm/inst.h
index b09887ffcd15..db5857796432 100644
--- a/arch/loongarch/include/asm/inst.h
+++ b/arch/loongarch/include/asm/inst.h
@@ -56,6 +56,14 @@ enum reg2_op {
revbd_op = 0x0f,
revh2w_op = 0x10,
revhd_op = 0x11,
+ iocsrrdb_op = 0x19200,
+ iocsrrdh_op = 0x19201,
+ iocsrrdw_op = 0x19202,
+ iocsrrdd_op = 0x19203,
+ iocsrwrb_op = 0x19204,
+ iocsrwrh_op = 0x19205,
+ iocsrwrw_op = 0x19206,
+ iocsrwrd_op = 0x19207,
};
enum reg2i5_op {
@@ -298,6 +306,13 @@ struct reg3sa2_format {
unsigned int opcode : 15;
};
+struct reg2csr_format {
+ unsigned int rd : 5;
+ unsigned int rj : 5;
+ unsigned int csr : 14;
+ unsigned int opcode : 8;
+};
+
union loongarch_instruction {
unsigned int word;
struct reg0i15_format reg0i15_format;
@@ -313,6 +328,7 @@ union loongarch_instruction {
struct reg2bstrd_format reg2bstrd_format;
struct reg3_format reg3_format;
struct reg3sa2_format reg3sa2_format;
+ struct reg2csr_format reg2csr_format;
};
#define LOONGARCH_INSN_SIZE sizeof(union loongarch_instruction)
diff --git a/arch/loongarch/kvm/exit.c b/arch/loongarch/kvm/exit.c
index 18635333fc9a..32edd915ebcb 100644
--- a/arch/loongarch/kvm/exit.c
+++ b/arch/loongarch/kvm/exit.c
@@ -96,3 +96,95 @@ static int _kvm_handle_csr(struct kvm_vcpu *vcpu, larch_inst inst)
return EMULATE_DONE;
}
+
+int _kvm_emu_iocsr(larch_inst inst, struct kvm_run *run, struct kvm_vcpu *vcpu)
+{
+ u32 rd, rj, opcode;
+ u32 addr;
+ unsigned long val;
+ int ret;
+
+ /*
+ * Each IOCSR with different opcode
+ */
+ rd = inst.reg2_format.rd;
+ rj = inst.reg2_format.rj;
+ opcode = inst.reg2_format.opcode;
+ addr = vcpu->arch.gprs[rj];
+ ret = EMULATE_DO_IOCSR;
+ run->iocsr_io.phys_addr = addr;
+ run->iocsr_io.is_write = 0;
+
+ /* LoongArch is Little endian */
+ switch (opcode) {
+ case iocsrrdb_op:
+ run->iocsr_io.len = 1;
+ break;
+ case iocsrrdh_op:
+ run->iocsr_io.len = 2;
+ break;
+ case iocsrrdw_op:
+ run->iocsr_io.len = 4;
+ break;
+ case iocsrrdd_op:
+ run->iocsr_io.len = 8;
+ break;
+ case iocsrwrb_op:
+ run->iocsr_io.len = 1;
+ run->iocsr_io.is_write = 1;
+ break;
+ case iocsrwrh_op:
+ run->iocsr_io.len = 2;
+ run->iocsr_io.is_write = 1;
+ break;
+ case iocsrwrw_op:
+ run->iocsr_io.len = 4;
+ run->iocsr_io.is_write = 1;
+ break;
+ case iocsrwrd_op:
+ run->iocsr_io.len = 8;
+ run->iocsr_io.is_write = 1;
+ break;
+ default:
+ ret = EMULATE_FAIL;
+ break;
+ }
+
+ if (ret == EMULATE_DO_IOCSR) {
+ if (run->iocsr_io.is_write) {
+ val = vcpu->arch.gprs[rd];
+ memcpy(run->iocsr_io.data, &val, run->iocsr_io.len);
+ }
+ vcpu->arch.io_gpr = rd;
+ }
+
+ return ret;
+}
+
+int _kvm_complete_iocsr_read(struct kvm_vcpu *vcpu, struct kvm_run *run)
+{
+ unsigned long *gpr = &vcpu->arch.gprs[vcpu->arch.io_gpr];
+ enum emulation_result er = EMULATE_DONE;
+
+ switch (run->iocsr_io.len) {
+ case 8:
+ *gpr = *(s64 *)run->iocsr_io.data;
+ break;
+ case 4:
+ *gpr = *(int *)run->iocsr_io.data;
+ break;
+ case 2:
+ *gpr = *(short *)run->iocsr_io.data;
+ break;
+ case 1:
+ *gpr = *(char *) run->iocsr_io.data;
+ break;
+ default:
+ kvm_err("Bad IOCSR length: %d,addr is 0x%lx",
+ run->iocsr_io.len, vcpu->arch.badv);
+ er = EMULATE_FAIL;
+ break;
+ }
+
+ return er;
+}
--
2.39.1
Implement LoongArch VM operations: Init and destroy vm interface,
allocating memory page to save the vm pgd when init vm. Implement
vm check extension, such as getting vcpu number info, memory slots
info, and fpu info. And implement vm status description.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/vm.c | 76 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 76 insertions(+)
create mode 100644 arch/loongarch/kvm/vm.c
diff --git a/arch/loongarch/kvm/vm.c b/arch/loongarch/kvm/vm.c
new file mode 100644
index 000000000000..84554988b4fd
--- /dev/null
+++ b/arch/loongarch/kvm/vm.c
@@ -0,0 +1,76 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#include <linux/kvm_host.h>
+
+const struct _kvm_stats_desc kvm_vm_stats_desc[] = {
+ KVM_GENERIC_VM_STATS(),
+};
+
+const struct kvm_stats_header kvm_vm_stats_header = {
+ .name_size = KVM_STATS_NAME_SIZE,
+ .num_desc = ARRAY_SIZE(kvm_vm_stats_desc),
+ .id_offset = sizeof(struct kvm_stats_header),
+ .desc_offset = sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE,
+ .data_offset = sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE +
+ sizeof(kvm_vm_stats_desc),
+};
+
+int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
+{
+ /* Allocate page table to map GPA -> RPA */
+ kvm->arch.gpa_mm.pgd = kvm_pgd_alloc();
+ if (!kvm->arch.gpa_mm.pgd)
+ return -ENOMEM;
+
+ kvm_init_vmcs(kvm);
+ kvm->arch.gpa_size = BIT(cpu_vabits - 1);
+ return 0;
+}
+
+void kvm_arch_destroy_vm(struct kvm *kvm)
+{
+ kvm_destroy_vcpus(kvm);
+ _kvm_destroy_mm(kvm);
+}
+
+int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
+{
+ int r;
+
+ switch (ext) {
+ case KVM_CAP_ONE_REG:
+ case KVM_CAP_ENABLE_CAP:
+ case KVM_CAP_READONLY_MEM:
+ case KVM_CAP_SYNC_MMU:
+ case KVM_CAP_IMMEDIATE_EXIT:
+ case KVM_CAP_IOEVENTFD:
+ case KVM_CAP_MP_STATE:
+ r = 1;
+ break;
+ case KVM_CAP_NR_VCPUS:
+ r = num_online_cpus();
+ break;
+ case KVM_CAP_MAX_VCPUS:
+ r = KVM_MAX_VCPUS;
+ break;
+ case KVM_CAP_MAX_VCPU_ID:
+ r = KVM_MAX_VCPU_IDS;
+ break;
+ case KVM_CAP_NR_MEMSLOTS:
+ r = KVM_USER_MEM_SLOTS;
+ break;
+ default:
+ r = 0;
+ break;
+ }
+
+ return r;
+}
+
+int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
+{
+ return -ENOIOCTLCMD;
+}
--
2.39.1
Implement LoongArch vcpu KVM_ENABLE_CAP ioctl interface.
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/vcpu.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index b0cce413762d..da97b77da8eb 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -186,6 +186,16 @@ int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
return 0;
}
+static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
+ struct kvm_enable_cap *cap)
+{
+ /*
+ * FPU is enable by default, do not support any other caps,
+ * and later we will support such as LSX cap.
+ */
+ return -EINVAL;
+}
+
long kvm_arch_vcpu_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -209,6 +219,15 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
r = _kvm_get_reg(vcpu, ®);
break;
}
+ case KVM_ENABLE_CAP: {
+ struct kvm_enable_cap cap;
+
+ r = -EFAULT;
+ if (copy_from_user(&cap, argp, sizeof(cap)))
+ break;
+ r = kvm_vcpu_ioctl_enable_cap(vcpu, &cap);
+ break;
+ }
default:
r = -ENOIOCTLCMD;
break;
--
2.39.1
Implement kvm hardware enable, disable interface, setting
the guest config register to enable virtualization features
when called the interface.
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/main.c | 64 +++++++++++++++++++++++++++++++++++++++
1 file changed, 64 insertions(+)
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index f98c1619725f..5ebae1ea7565 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -195,6 +195,70 @@ static void _kvm_init_gcsr_flag(void)
set_gcsr_sw_flag(LOONGARCH_CSR_PERFCNTR3);
}
+void kvm_init_vmcs(struct kvm *kvm)
+{
+ kvm->arch.vmcs = vmcs;
+}
+
+long kvm_arch_dev_ioctl(struct file *filp,
+ unsigned int ioctl, unsigned long arg)
+{
+ return -ENOIOCTLCMD;
+}
+
+#ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING
+int kvm_arch_hardware_enable(void)
+{
+ unsigned long env, gcfg = 0;
+
+ env = read_csr_gcfg();
+ /* First init gtlbc, gcfg, gstat, gintc. All guest use the same config */
+ clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI);
+ write_csr_gcfg(0);
+ write_csr_gstat(0);
+ write_csr_gintc(0);
+
+ /*
+ * Enable virtualization features granting guest direct control of
+ * certain features:
+ * GCI=2: Trap on init or unimplement cache instruction.
+ * TORU=0: Trap on Root Unimplement.
+ * CACTRL=1: Root control cache.
+ * TOP=0: Trap on Previlege.
+ * TOE=0: Trap on Exception.
+ * TIT=0: Trap on Timer.
+ */
+ if (env & CSR_GCFG_GCIP_ALL)
+ gcfg |= CSR_GCFG_GCI_SECURE;
+ if (env & CSR_GCFG_MATC_ROOT)
+ gcfg |= CSR_GCFG_MATC_ROOT;
+
+ gcfg |= CSR_GCFG_TIT;
+ write_csr_gcfg(gcfg);
+
+ kvm_flush_tlb_all();
+
+ /* Enable using TGID */
+ set_csr_gtlbc(CSR_GTLBC_USETGID);
+ kvm_debug("gtlbc:%llx gintc:%llx gstat:%llx gcfg:%llx",
+ read_csr_gtlbc(), read_csr_gintc(),
+ read_csr_gstat(), read_csr_gcfg());
+
+ return 0;
+}
+
+void kvm_arch_hardware_disable(void)
+{
+ clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI);
+ write_csr_gcfg(0);
+ write_csr_gstat(0);
+ write_csr_gintc(0);
+
+ /* Flush any remaining guest TLB entries */
+ kvm_flush_tlb_all();
+}
+#endif
+
static int kvm_loongarch_env_init(void)
{
struct kvm_context *context;
--
2.39.1
Implement LoongArch kvm module init, module exit interface,
using kvm context to save the vpid info and vcpu world switch
interface pointer.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/main.c | 283 ++++++++++++++++++++++++++++++++++++++
1 file changed, 283 insertions(+)
create mode 100644 arch/loongarch/kvm/main.c
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
new file mode 100644
index 000000000000..f98c1619725f
--- /dev/null
+++ b/arch/loongarch/kvm/main.c
@@ -0,0 +1,283 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/kvm_host.h>
+#include <asm/cacheflush.h>
+#include <asm/kvm_csr.h>
+
+static struct kvm_context __percpu *vmcs;
+struct kvm_world_switch *kvm_loongarch_ops;
+unsigned long vpid_mask;
+static int gcsr_flag[CSR_MAX_NUMS];
+
+int get_gcsr_flag(int csr)
+{
+ if (csr < CSR_MAX_NUMS)
+ return gcsr_flag[csr];
+
+ return INVALID_GCSR;
+}
+
+static inline void set_gcsr_sw_flag(int csr)
+{
+ if (csr < CSR_MAX_NUMS)
+ gcsr_flag[csr] |= SW_GCSR;
+}
+
+static inline void set_gcsr_hw_flag(int csr)
+{
+ if (csr < CSR_MAX_NUMS)
+ gcsr_flag[csr] |= HW_GCSR;
+}
+
+/*
+ * The default value of gcsr_flag[CSR] is 0, and we use this
+ * function to set the flag to 1(SW_GCSR) or 2(HW_GCSR) if the
+ * gcsr is software or hardware. It will be used by get/set_gcsr,
+ * if gcsr_flag is HW we should use gcsrrd/gcsrwr to access it,
+ * else use sw csr to emulate it.
+ */
+static void _kvm_init_gcsr_flag(void)
+{
+ set_gcsr_hw_flag(LOONGARCH_CSR_CRMD);
+ set_gcsr_hw_flag(LOONGARCH_CSR_PRMD);
+ set_gcsr_hw_flag(LOONGARCH_CSR_EUEN);
+ set_gcsr_hw_flag(LOONGARCH_CSR_MISC);
+ set_gcsr_hw_flag(LOONGARCH_CSR_ECFG);
+ set_gcsr_hw_flag(LOONGARCH_CSR_ESTAT);
+ set_gcsr_hw_flag(LOONGARCH_CSR_ERA);
+ set_gcsr_hw_flag(LOONGARCH_CSR_BADV);
+ set_gcsr_hw_flag(LOONGARCH_CSR_BADI);
+ set_gcsr_hw_flag(LOONGARCH_CSR_EENTRY);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBIDX);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBEHI);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBELO0);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBELO1);
+ set_gcsr_hw_flag(LOONGARCH_CSR_ASID);
+ set_gcsr_hw_flag(LOONGARCH_CSR_PGDL);
+ set_gcsr_hw_flag(LOONGARCH_CSR_PGDH);
+ set_gcsr_hw_flag(LOONGARCH_CSR_PWCTL0);
+ set_gcsr_hw_flag(LOONGARCH_CSR_PWCTL1);
+ set_gcsr_hw_flag(LOONGARCH_CSR_STLBPGSIZE);
+ set_gcsr_hw_flag(LOONGARCH_CSR_RVACFG);
+ set_gcsr_hw_flag(LOONGARCH_CSR_CPUID);
+ set_gcsr_hw_flag(LOONGARCH_CSR_PRCFG1);
+ set_gcsr_hw_flag(LOONGARCH_CSR_PRCFG2);
+ set_gcsr_hw_flag(LOONGARCH_CSR_PRCFG3);
+ set_gcsr_hw_flag(LOONGARCH_CSR_KS0);
+ set_gcsr_hw_flag(LOONGARCH_CSR_KS1);
+ set_gcsr_hw_flag(LOONGARCH_CSR_KS2);
+ set_gcsr_hw_flag(LOONGARCH_CSR_KS3);
+ set_gcsr_hw_flag(LOONGARCH_CSR_KS4);
+ set_gcsr_hw_flag(LOONGARCH_CSR_KS5);
+ set_gcsr_hw_flag(LOONGARCH_CSR_KS6);
+ set_gcsr_hw_flag(LOONGARCH_CSR_KS7);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TMID);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TCFG);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TVAL);
+ set_gcsr_hw_flag(LOONGARCH_CSR_CNTC);
+ set_gcsr_hw_flag(LOONGARCH_CSR_LLBCTL);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBRENTRY);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBRBADV);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBRERA);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBRSAVE);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBRELO0);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBRELO1);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBREHI);
+ set_gcsr_hw_flag(LOONGARCH_CSR_TLBRPRMD);
+ set_gcsr_hw_flag(LOONGARCH_CSR_DMWIN0);
+ set_gcsr_hw_flag(LOONGARCH_CSR_DMWIN1);
+ set_gcsr_hw_flag(LOONGARCH_CSR_DMWIN2);
+ set_gcsr_hw_flag(LOONGARCH_CSR_DMWIN3);
+ set_gcsr_hw_flag(LOONGARCH_CSR_MWPS);
+ set_gcsr_hw_flag(LOONGARCH_CSR_FWPS);
+
+ set_gcsr_sw_flag(LOONGARCH_CSR_IMPCTL1);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IMPCTL2);
+ set_gcsr_sw_flag(LOONGARCH_CSR_MERRCTL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_MERRINFO1);
+ set_gcsr_sw_flag(LOONGARCH_CSR_MERRINFO2);
+ set_gcsr_sw_flag(LOONGARCH_CSR_MERRENTRY);
+ set_gcsr_sw_flag(LOONGARCH_CSR_MERRERA);
+ set_gcsr_sw_flag(LOONGARCH_CSR_MERRSAVE);
+ set_gcsr_sw_flag(LOONGARCH_CSR_CTAG);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DEBUG);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DERA);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DESAVE);
+ set_gcsr_sw_flag(LOONGARCH_CSR_PRCFG1);
+ set_gcsr_sw_flag(LOONGARCH_CSR_PRCFG2);
+ set_gcsr_sw_flag(LOONGARCH_CSR_PRCFG3);
+ set_gcsr_sw_flag(LOONGARCH_CSR_PGD);
+ set_gcsr_sw_flag(LOONGARCH_CSR_TINTCLR);
+
+ set_gcsr_sw_flag(LOONGARCH_CSR_FWPS);
+ set_gcsr_sw_flag(LOONGARCH_CSR_FWPC);
+ set_gcsr_sw_flag(LOONGARCH_CSR_MWPS);
+ set_gcsr_sw_flag(LOONGARCH_CSR_MWPC);
+
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB0ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB0MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB0CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB0ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB1ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB1MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB1CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB1ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB2ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB2MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB2CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB2ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB3ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB3MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB3CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB3ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB4ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB4MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB4CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB4ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB5ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB5MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB5CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB5ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB6ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB6MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB6CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB6ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB7ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB7MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB7CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_DB7ASID);
+
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB0ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB0MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB0CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB0ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB1ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB1MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB1CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB1ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB2ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB2MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB2CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB2ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB3ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB3MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB3CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB3ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB4ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB4MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB4CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB4ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB5ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB5MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB5CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB5ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB6ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB6MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB6CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB6ASID);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB7ADDR);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB7MASK);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB7CTRL);
+ set_gcsr_sw_flag(LOONGARCH_CSR_IB7ASID);
+
+ set_gcsr_sw_flag(LOONGARCH_CSR_PERFCTRL0);
+ set_gcsr_sw_flag(LOONGARCH_CSR_PERFCNTR0);
+ set_gcsr_sw_flag(LOONGARCH_CSR_PERFCTRL1);
+ set_gcsr_sw_flag(LOONGARCH_CSR_PERFCNTR1);
+ set_gcsr_sw_flag(LOONGARCH_CSR_PERFCTRL2);
+ set_gcsr_sw_flag(LOONGARCH_CSR_PERFCNTR2);
+ set_gcsr_sw_flag(LOONGARCH_CSR_PERFCTRL3);
+ set_gcsr_sw_flag(LOONGARCH_CSR_PERFCNTR3);
+}
+
+static int kvm_loongarch_env_init(void)
+{
+ struct kvm_context *context;
+ int cpu, order;
+ void *addr;
+
+ vmcs = alloc_percpu(struct kvm_context);
+ if (!vmcs) {
+ pr_err("kvm: failed to allocate percpu kvm_context\n");
+ return -ENOMEM;
+ }
+
+ kvm_loongarch_ops = kzalloc(sizeof(*kvm_loongarch_ops), GFP_KERNEL);
+ if (!kvm_loongarch_ops)
+ return -ENOMEM;
+
+ /*
+ * There will be problem in world switch code if there
+ * is page fault reenter, since pgd register is shared
+ * between root kernel and kvm hypervisor. World switch
+ * entry need be unmapped area, cannot be tlb mapped area.
+ * In future if hw pagetable walking is supported, or there
+ * is separate pgd registers between root kernel and kvm
+ * hypervisor, copying about world switch code will not be used.
+ */
+
+ order = get_order(kvm_vector_size + kvm_enter_guest_size);
+ addr = (void *)__get_free_pages(GFP_KERNEL, order);
+ if (!addr) {
+ free_percpu(vmcs);
+ return -ENOMEM;
+ }
+
+ memcpy(addr, kvm_vector_entry, kvm_vector_size);
+ memcpy(addr + kvm_vector_size, kvm_enter_guest, kvm_enter_guest_size);
+ flush_icache_range((unsigned long)addr, (unsigned long)addr +
+ kvm_vector_size + kvm_enter_guest_size);
+ kvm_loongarch_ops->guest_eentry = addr;
+ kvm_loongarch_ops->enter_guest = addr + kvm_vector_size;
+ kvm_loongarch_ops->page_order = order;
+
+ vpid_mask = read_csr_gstat();
+ vpid_mask = (vpid_mask & CSR_GSTAT_GIDBIT) >> CSR_GSTAT_GIDBIT_SHIFT;
+ if (vpid_mask)
+ vpid_mask = GENMASK(vpid_mask - 1, 0);
+
+ for_each_possible_cpu(cpu) {
+ context = per_cpu_ptr(vmcs, cpu);
+ context->vpid_cache = vpid_mask + 1;
+ context->last_vcpu = NULL;
+ }
+
+ _kvm_init_fault();
+ _kvm_init_gcsr_flag();
+
+ return 0;
+}
+
+static void kvm_loongarch_env_exit(void)
+{
+ free_pages((unsigned long)kvm_loongarch_ops->guest_eentry, kvm_loongarch_ops->page_order);
+ free_percpu(vmcs);
+}
+
+static int kvm_loongarch_init(void)
+{
+ int r;
+
+ if (!cpu_has_lvz)
+ return 0;
+
+ r = kvm_loongarch_env_init();
+ if (r)
+ return r;
+
+ return kvm_init(sizeof(struct kvm_vcpu), 0, THIS_MODULE);
+}
+
+static void kvm_loongarch_exit(void)
+{
+ kvm_exit();
+ kvm_loongarch_env_exit();
+}
+
+module_init(kvm_loongarch_init);
+module_exit(kvm_loongarch_exit);
--
2.39.1
Implement LoongArch vcpu status description such as idle exits counter,
signal exits counter, cpucfg exits counter, etc.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/vcpu.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index 012b0738a06d..2a8d9a76b134 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -12,6 +12,23 @@
#define CREATE_TRACE_POINTS
#include "trace.h"
+const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = {
+ KVM_GENERIC_VCPU_STATS(),
+ STATS_DESC_COUNTER(VCPU, idle_exits),
+ STATS_DESC_COUNTER(VCPU, signal_exits),
+ STATS_DESC_COUNTER(VCPU, int_exits),
+ STATS_DESC_COUNTER(VCPU, cpucfg_exits),
+};
+
+const struct kvm_stats_header kvm_vcpu_stats_header = {
+ .name_size = KVM_STATS_NAME_SIZE,
+ .num_desc = ARRAY_SIZE(kvm_vcpu_stats_desc),
+ .id_offset = sizeof(struct kvm_stats_header),
+ .desc_offset = sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE,
+ .data_offset = sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE +
+ sizeof(kvm_vcpu_stats_desc),
+};
+
int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
{
return !!(vcpu->arch.irq_pending) &&
--
2.39.1
Implement kvm check vmid and update vmid, the vmid should be checked before
vcpu enter guest.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/vmid.c | 66 +++++++++++++++++++++++++++++++++++++++
1 file changed, 66 insertions(+)
create mode 100644 arch/loongarch/kvm/vmid.c
diff --git a/arch/loongarch/kvm/vmid.c b/arch/loongarch/kvm/vmid.c
new file mode 100644
index 000000000000..380c25c5d12b
--- /dev/null
+++ b/arch/loongarch/kvm/vmid.c
@@ -0,0 +1,66 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#include <linux/kvm_host.h>
+#include "trace.h"
+
+static void _kvm_update_vpid(struct kvm_vcpu *vcpu, int cpu)
+{
+ struct kvm_context *context;
+ unsigned long vpid;
+
+ context = per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu);
+ vpid = context->vpid_cache + 1;
+ if (!(vpid & vpid_mask)) {
+ /* finish round of 64 bit loop */
+ if (unlikely(!vpid))
+ vpid = vpid_mask + 1;
+
+ /* vpid 0 reserved for root */
+ ++vpid;
+
+ /* start new vpid cycle */
+ kvm_flush_tlb_all();
+ }
+
+ context->vpid_cache = vpid;
+ vcpu->arch.vpid = vpid;
+}
+
+void _kvm_check_vmid(struct kvm_vcpu *vcpu)
+{
+ struct kvm_context *context;
+ bool migrated;
+ unsigned long ver, old, vpid;
+ int cpu;
+
+ cpu = smp_processor_id();
+ /*
+ * Are we entering guest context on a different CPU to last time?
+ * If so, the vCPU's guest TLB state on this CPU may be stale.
+ */
+ context = per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu);
+ migrated = (vcpu->arch.last_exec_cpu != cpu);
+ vcpu->arch.last_exec_cpu = cpu;
+
+ /*
+ * Check if our vpid is of an older version
+ *
+ * We also discard the stored vpid if we've executed on
+ * another CPU, as the guest mappings may have changed without
+ * hypervisor knowledge.
+ */
+ ver = vcpu->arch.vpid & ~vpid_mask;
+ old = context->vpid_cache & ~vpid_mask;
+ if (migrated || (ver != old)) {
+ _kvm_update_vpid(vcpu, cpu);
+ trace_kvm_vpid_change(vcpu, vcpu->arch.vpid);
+ }
+
+ /* Restore GSTAT(0x50).vpid */
+ vpid = (vcpu->arch.vpid & vpid_mask)
+ << CSR_GSTAT_GID_SHIFT;
+ change_csr_gstat(vpid_mask << CSR_GSTAT_GID_SHIFT, vpid);
+}
--
2.39.1
Implement kvm handle gspr exception interface, including emulate the
reading and writing of cpucfg, csr, iocsr resource.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/exit.c | 115 ++++++++++++++++++++++++++++++++++++++
1 file changed, 115 insertions(+)
diff --git a/arch/loongarch/kvm/exit.c b/arch/loongarch/kvm/exit.c
index 30748238c78c..04e4339cd78d 100644
--- a/arch/loongarch/kvm/exit.c
+++ b/arch/loongarch/kvm/exit.c
@@ -208,3 +208,118 @@ int _kvm_emu_idle(struct kvm_vcpu *vcpu)
return EMULATE_DONE;
}
+
+static int _kvm_trap_handle_gspr(struct kvm_vcpu *vcpu)
+{
+ enum emulation_result er = EMULATE_DONE;
+ struct kvm_run *run = vcpu->run;
+ larch_inst inst;
+ unsigned long curr_pc;
+ int rd, rj;
+ unsigned int index;
+
+ /*
+ * Fetch the instruction.
+ */
+ inst.word = vcpu->arch.badi;
+ curr_pc = vcpu->arch.pc;
+ update_pc(&vcpu->arch);
+
+ trace_kvm_exit_gspr(vcpu, inst.word);
+ er = EMULATE_FAIL;
+ switch (((inst.word >> 24) & 0xff)) {
+ case 0x0:
+ /* cpucfg GSPR */
+ if (inst.reg2_format.opcode == 0x1B) {
+ rd = inst.reg2_format.rd;
+ rj = inst.reg2_format.rj;
+ ++vcpu->stat.cpucfg_exits;
+ index = vcpu->arch.gprs[rj];
+
+ vcpu->arch.gprs[rd] = read_cpucfg(index);
+ /* Nested KVM is not supported */
+ if (index == 2)
+ vcpu->arch.gprs[rd] &= ~CPUCFG2_LVZP;
+ if (index == 6)
+ vcpu->arch.gprs[rd] &= ~CPUCFG6_PMP;
+ er = EMULATE_DONE;
+ }
+ break;
+ case 0x4:
+ /* csr GSPR */
+ er = _kvm_handle_csr(vcpu, inst);
+ break;
+ case 0x6:
+ /* iocsr,cache,idle GSPR */
+ switch (((inst.word >> 22) & 0x3ff)) {
+ case 0x18:
+ /* cache GSPR */
+ er = EMULATE_DONE;
+ trace_kvm_exit_cache(vcpu, KVM_TRACE_EXIT_CACHE);
+ break;
+ case 0x19:
+ /* iocsr/idle GSPR */
+ switch (((inst.word >> 15) & 0x1ffff)) {
+ case 0xc90:
+ /* iocsr GSPR */
+ er = _kvm_emu_iocsr(inst, run, vcpu);
+ break;
+ case 0xc91:
+ /* idle GSPR */
+ er = _kvm_emu_idle(vcpu);
+ break;
+ default:
+ er = EMULATE_FAIL;
+ break;
+ }
+ break;
+ default:
+ er = EMULATE_FAIL;
+ break;
+ }
+ break;
+ default:
+ er = EMULATE_FAIL;
+ break;
+ }
+
+ /* Rollback PC only if emulation was unsuccessful */
+ if (er == EMULATE_FAIL) {
+ kvm_err("[%#lx]%s: unsupported gspr instruction 0x%08x\n",
+ curr_pc, __func__, inst.word);
+
+ kvm_arch_vcpu_dump_regs(vcpu);
+ vcpu->arch.pc = curr_pc;
+ }
+ return er;
+}
+
+/*
+ * Execute cpucfg instruction will tirggerGSPR,
+ * Also the access to unimplemented csrs 0x15
+ * 0x16, 0x50~0x53, 0x80, 0x81, 0x90~0x95, 0x98
+ * 0xc0~0xff, 0x100~0x109, 0x500~0x502,
+ * cache_op, idle_op iocsr ops the same
+ */
+static int _kvm_handle_gspr(struct kvm_vcpu *vcpu)
+{
+ enum emulation_result er = EMULATE_DONE;
+ int ret = RESUME_GUEST;
+
+ er = _kvm_trap_handle_gspr(vcpu);
+
+ if (er == EMULATE_DONE) {
+ ret = RESUME_GUEST;
+ } else if (er == EMULATE_DO_MMIO) {
+ vcpu->run->exit_reason = KVM_EXIT_MMIO;
+ ret = RESUME_HOST;
+ } else if (er == EMULATE_DO_IOCSR) {
+ vcpu->run->exit_reason = KVM_EXIT_LOONGARCH_IOCSR;
+ ret = RESUME_HOST;
+ } else {
+ kvm_err("%s internal error\n", __func__);
+ vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
+ ret = RESUME_HOST;
+ }
+ return ret;
+}
--
2.39.1
Implement vcpu run interface, handling mmio, iocsr reading fault
and deliver interrupt, lose fpu before vcpu enter guest.
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/vcpu.c | 83 +++++++++++++++++++++++++++++++++++++++
1 file changed, 83 insertions(+)
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index 24b5b00266a1..eba5c07b8be3 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -17,6 +17,41 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
return 0;
}
+/* Returns 1 if the guest TLB may be clobbered */
+static int _kvm_check_requests(struct kvm_vcpu *vcpu)
+{
+ int ret = 0;
+
+ if (!kvm_request_pending(vcpu))
+ return 0;
+
+ if (kvm_check_request(KVM_REQ_TLB_FLUSH, vcpu)) {
+ /* Drop vpid for this vCPU */
+ vcpu->arch.vpid = 0;
+ /* This will clobber guest TLB contents too */
+ ret = 1;
+ }
+
+ return ret;
+}
+
+static void kvm_pre_enter_guest(struct kvm_vcpu *vcpu)
+{
+ /*
+ * handle vcpu timer, interrupts, check requests and
+ * check vmid before vcpu enter guest
+ */
+ kvm_acquire_timer(vcpu);
+ _kvm_deliver_intr(vcpu);
+ /* make sure the vcpu mode has been written */
+ smp_store_mb(vcpu->mode, IN_GUEST_MODE);
+ _kvm_check_requests(vcpu);
+ _kvm_check_vmid(vcpu);
+ vcpu->arch.host_eentry = csr_read64(LOONGARCH_CSR_EENTRY);
+ /* clear KVM_LARCH_CSR as csr will change when enter guest */
+ vcpu->arch.aux_inuse &= ~KVM_LARCH_CSR;
+}
+
int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
{
unsigned long timer_hz;
@@ -86,3 +121,51 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
context->last_vcpu = NULL;
}
}
+
+int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
+{
+ int r = -EINTR;
+ struct kvm_run *run = vcpu->run;
+
+ vcpu_load(vcpu);
+
+ kvm_sigset_activate(vcpu);
+
+ if (vcpu->mmio_needed) {
+ if (!vcpu->mmio_is_write)
+ _kvm_complete_mmio_read(vcpu, run);
+ vcpu->mmio_needed = 0;
+ }
+
+ if (run->exit_reason == KVM_EXIT_LOONGARCH_IOCSR) {
+ if (!run->iocsr_io.is_write)
+ _kvm_complete_iocsr_read(vcpu, run);
+ }
+
+ /* clear exit_reason */
+ run->exit_reason = KVM_EXIT_UNKNOWN;
+ if (run->immediate_exit)
+ goto out;
+
+ lose_fpu(1);
+
+ local_irq_disable();
+ guest_timing_enter_irqoff();
+
+ kvm_pre_enter_guest(vcpu);
+ trace_kvm_enter(vcpu);
+
+ guest_state_enter_irqoff();
+ r = kvm_loongarch_ops->enter_guest(run, vcpu);
+
+ /* guest_state_exit_irqoff() already done. */
+ trace_kvm_out(vcpu);
+ guest_timing_exit_irqoff();
+ local_irq_enable();
+
+out:
+ kvm_sigset_deactivate(vcpu);
+
+ vcpu_put(vcpu);
+ return r;
+}
--
2.39.1
Implement LoongArch vcpu get registers and set registers operations, it
is called when user space use the ioctl interface to get or set regs.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/csr_ops.S | 76 +++++++++++++
arch/loongarch/kvm/vcpu.c | 206 +++++++++++++++++++++++++++++++++++
2 files changed, 282 insertions(+)
create mode 100644 arch/loongarch/kvm/csr_ops.S
diff --git a/arch/loongarch/kvm/csr_ops.S b/arch/loongarch/kvm/csr_ops.S
new file mode 100644
index 000000000000..8a149af3241b
--- /dev/null
+++ b/arch/loongarch/kvm/csr_ops.S
@@ -0,0 +1,76 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#include <asm/regdef.h>
+#include <linux/linkage.h>
+ .text
+ .section .text
+ .cfi_sections .debug_frame
+/*
+ * we have splited hw gcsr into three parts, so we can
+ * calculate the code offset by gcsrid and jump here to
+ * run the gcsrwr instruction.
+ */
+SYM_FUNC_START(set_hw_gcsr)
+ addi.d t0, a0, 0
+ addi.w t1, zero, 96
+ bltu t1, t0, 1f
+ la.pcrel t0, 10f
+ alsl.d t0, a0, t0, 3
+ jr t0
+1:
+ addi.w t1, a0, -128
+ addi.w t2, zero, 15
+ bltu t2, t1, 2f
+ la.pcrel t0, 11f
+ alsl.d t0, t1, t0, 3
+ jr t0
+2:
+ addi.w t1, a0, -384
+ addi.w t2, zero, 3
+ bltu t2, t1, 3f
+ la.pcrel t0, 12f
+ alsl.d t0, t1, t0, 3
+ jr t0
+3:
+ addi.w a0, zero, -1
+ jr ra
+/*
+ * write guest csr
+ * 0x05000000 | (LOONGARCH_CSR_XXX << 10) | 1 << 5 | a1
+ * range from 0x0(KVM_CSR_CRMD) to 0x60 (KVM_CSR_LLBCTL)
+ */
+10:
+ csrnum = 0
+ .rept 0x61
+ .word 0x05000020 | csrnum << 10 | 5
+ jr ra
+ csrnum = csrnum + 1
+ .endr
+/*
+ * write guest csr
+ * 0x05000000 | (LOONGARCH_CSR_XXX << 10) | 1<<5 | a1
+ * range from 0x80 (KVM_CSR_IMPCTL1) to 0x8f (KVM_CSR_TLBRPRMD)
+ */
+11:
+ csrnum = 0x80
+ .rept 0x10
+ .word 0x05000020 | csrnum << 10 | 5
+ jr ra
+ csrnum = csrnum + 1
+ .endr
+/*
+ * write guest csr
+ * 0x05000000 | (LOONGARCH_CSR_XXX << 10) | 1<<5 | a1
+ * range from 0x180(KVM_CSR_DMWIN0) to 0x183(KVM_CSR_DMWIN3)
+ */
+12:
+ csrnum = 0x180
+ .rept 0x4
+ .word 0x05000020 | csrnum << 10 | 5
+ jr ra
+ csrnum = csrnum + 1
+ .endr
+SYM_FUNC_END(set_hw_gcsr)
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index a45e9d9efe5b..b0cce413762d 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -12,6 +12,212 @@
#define CREATE_TRACE_POINTS
#include "trace.h"
+int _kvm_getcsr(struct kvm_vcpu *vcpu, unsigned int id, u64 *v)
+{
+ unsigned long val;
+ struct loongarch_csrs *csr = vcpu->arch.csr;
+
+ if (get_gcsr_flag(id) & INVALID_GCSR)
+ return -EINVAL;
+
+ if (id == LOONGARCH_CSR_ESTAT) {
+ /* interrupt status IP0 -- IP7 from GINTC */
+ val = kvm_read_sw_gcsr(csr, LOONGARCH_CSR_GINTC) & 0xff;
+ *v = kvm_read_sw_gcsr(csr, id) | (val << 2);
+ return 0;
+ }
+
+ /*
+ * get software csr state if csrid is valid, since software
+ * csr state is consistent with hardware
+ */
+ *v = kvm_read_sw_gcsr(csr, id);
+
+ return 0;
+}
+
+int _kvm_setcsr(struct kvm_vcpu *vcpu, unsigned int id, u64 val)
+{
+ struct loongarch_csrs *csr = vcpu->arch.csr;
+ int ret = 0, gintc;
+
+ if (get_gcsr_flag(id) & INVALID_GCSR)
+ return -EINVAL;
+
+ if (id == LOONGARCH_CSR_ESTAT) {
+ /* estat IP0~IP7 inject through guestexcept */
+ gintc = (val >> 2) & 0xff;
+ write_csr_gintc(gintc);
+ kvm_set_sw_gcsr(csr, LOONGARCH_CSR_GINTC, gintc);
+
+ gintc = val & ~(0xffUL << 2);
+ write_gcsr_estat(gintc);
+ kvm_set_sw_gcsr(csr, LOONGARCH_CSR_ESTAT, gintc);
+
+ return ret;
+ }
+
+ if (get_gcsr_flag(id) & HW_GCSR) {
+ set_hw_gcsr(id, val);
+ /* write sw gcsr to keep consistent with hardware */
+ kvm_write_sw_gcsr(csr, id, val);
+ } else
+ kvm_write_sw_gcsr(csr, id, val);
+
+ return ret;
+}
+
+static int _kvm_get_one_reg(struct kvm_vcpu *vcpu,
+ const struct kvm_one_reg *reg, s64 *v)
+{
+ int reg_idx, ret = 0;
+
+ if ((reg->id & KVM_REG_LOONGARCH_MASK) == KVM_REG_LOONGARCH_CSR) {
+ reg_idx = KVM_GET_IOC_CSRIDX(reg->id);
+ ret = _kvm_getcsr(vcpu, reg_idx, v);
+ } else if (reg->id == KVM_REG_LOONGARCH_COUNTER)
+ *v = drdtime() + vcpu->kvm->arch.time_offset;
+ else
+ ret = -EINVAL;
+
+ return ret;
+}
+
+static int _kvm_get_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
+{
+ int ret = -EINVAL;
+ s64 v;
+
+ if ((reg->id & KVM_REG_SIZE_MASK) != KVM_REG_SIZE_U64)
+ return ret;
+
+ if (_kvm_get_one_reg(vcpu, reg, &v))
+ return ret;
+
+ return put_user(v, (u64 __user *)(long)reg->addr);
+}
+
+static int _kvm_set_one_reg(struct kvm_vcpu *vcpu,
+ const struct kvm_one_reg *reg,
+ s64 v)
+{
+ int ret = 0;
+ unsigned long flags;
+ u64 val;
+ int reg_idx;
+
+ val = v;
+ if ((reg->id & KVM_REG_LOONGARCH_MASK) == KVM_REG_LOONGARCH_CSR) {
+ reg_idx = KVM_GET_IOC_CSRIDX(reg->id);
+ ret = _kvm_setcsr(vcpu, reg_idx, val);
+ } else if (reg->id == KVM_REG_LOONGARCH_COUNTER) {
+ local_irq_save(flags);
+ /*
+ * gftoffset is relative with board, not vcpu
+ * only set for the first time for smp system
+ */
+ if (vcpu->vcpu_id == 0)
+ vcpu->kvm->arch.time_offset = (signed long)(v - drdtime());
+ write_csr_gcntc((ulong)vcpu->kvm->arch.time_offset);
+ local_irq_restore(flags);
+ } else if (reg->id == KVM_REG_LOONGARCH_VCPU_RESET) {
+ kvm_reset_timer(vcpu);
+ memset(&vcpu->arch.irq_pending, 0, sizeof(vcpu->arch.irq_pending));
+ memset(&vcpu->arch.irq_clear, 0, sizeof(vcpu->arch.irq_clear));
+ } else
+ ret = -EINVAL;
+
+ return ret;
+}
+
+static int _kvm_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
+{
+ s64 v;
+ int ret = -EINVAL;
+
+ if ((reg->id & KVM_REG_SIZE_MASK) != KVM_REG_SIZE_U64)
+ return ret;
+
+ if (get_user(v, (u64 __user *)(long)reg->addr))
+ return ret;
+
+ return _kvm_set_one_reg(vcpu, reg, v);
+}
+
+int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu,
+ struct kvm_sregs *sregs)
+{
+ return -ENOIOCTLCMD;
+}
+
+int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
+ struct kvm_sregs *sregs)
+{
+ return -ENOIOCTLCMD;
+}
+
+int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
+{
+ int i;
+
+ vcpu_load(vcpu);
+
+ for (i = 0; i < ARRAY_SIZE(vcpu->arch.gprs); i++)
+ regs->gpr[i] = vcpu->arch.gprs[i];
+
+ regs->pc = vcpu->arch.pc;
+
+ vcpu_put(vcpu);
+ return 0;
+}
+
+int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
+{
+ int i;
+
+ vcpu_load(vcpu);
+
+ for (i = 1; i < ARRAY_SIZE(vcpu->arch.gprs); i++)
+ vcpu->arch.gprs[i] = regs->gpr[i];
+ vcpu->arch.gprs[0] = 0; /* zero is special, and cannot be set. */
+ vcpu->arch.pc = regs->pc;
+
+ vcpu_put(vcpu);
+ return 0;
+}
+
+long kvm_arch_vcpu_ioctl(struct file *filp,
+ unsigned int ioctl, unsigned long arg)
+{
+ struct kvm_vcpu *vcpu = filp->private_data;
+ void __user *argp = (void __user *)arg;
+ long r;
+
+ vcpu_load(vcpu);
+
+ switch (ioctl) {
+ case KVM_SET_ONE_REG:
+ case KVM_GET_ONE_REG: {
+ struct kvm_one_reg reg;
+
+ r = -EFAULT;
+ if (copy_from_user(®, argp, sizeof(reg)))
+ break;
+ if (ioctl == KVM_SET_ONE_REG)
+ r = _kvm_set_reg(vcpu, ®);
+ else
+ r = _kvm_get_reg(vcpu, ®);
+ break;
+ }
+ default:
+ r = -ENOIOCTLCMD;
+ break;
+ }
+
+ vcpu_put(vcpu);
+ return r;
+}
+
int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
{
return 0;
--
2.39.1
Implement LoongArch kvm mmu, it is used to switch gpa to hpa when
guest exit because of address translation exception. This patch
implement allocate gpa page table, search gpa from it and flush guest
gpa in the table.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/mmu.c | 725 +++++++++++++++++++++++++++++++++++++++
1 file changed, 725 insertions(+)
create mode 100644 arch/loongarch/kvm/mmu.c
diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c
new file mode 100644
index 000000000000..d75446139546
--- /dev/null
+++ b/arch/loongarch/kvm/mmu.c
@@ -0,0 +1,725 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#include <linux/highmem.h>
+#include <linux/page-flags.h>
+#include <linux/kvm_host.h>
+#include <linux/uaccess.h>
+#include <asm/mmu_context.h>
+#include <asm/pgalloc.h>
+#include <asm/tlb.h>
+
+/*
+ * KVM_MMU_CACHE_MIN_PAGES is the number of GPA page table translation levels
+ * for which pages need to be cached.
+ */
+#define KVM_MMU_CACHE_MIN_PAGES (CONFIG_PGTABLE_LEVELS - 1)
+
+/**
+ * kvm_pgd_alloc() - Allocate and initialise a KVM GPA page directory.
+ *
+ * Allocate a blank KVM GPA page directory (PGD) for representing guest physical
+ * to host physical page mappings.
+ *
+ * Returns: Pointer to new KVM GPA page directory.
+ * NULL on allocation failure.
+ */
+pgd_t *kvm_pgd_alloc(void)
+{
+ pgd_t *pgd;
+
+ pgd = (pgd_t *)__get_free_pages(GFP_KERNEL, 0);
+ if (pgd)
+ pgd_init((void *)pgd);
+
+ return pgd;
+}
+
+/**
+ * kvm_walk_pgd() - Walk page table with optional allocation.
+ * @pgd: Page directory pointer.
+ * @addr: Address to index page table using.
+ * @cache: MMU page cache to allocate new page tables from, or NULL.
+ *
+ * Walk the page tables pointed to by @pgd to find the PTE corresponding to the
+ * address @addr. If page tables don't exist for @addr, they will be created
+ * from the MMU cache if @cache is not NULL.
+ *
+ * Returns: Pointer to pte_t corresponding to @addr.
+ * NULL if a page table doesn't exist for @addr and !@cache.
+ * NULL if a page table allocation failed.
+ */
+static pte_t *kvm_walk_pgd(pgd_t *pgd, struct kvm_mmu_memory_cache *cache,
+ unsigned long addr)
+{
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+
+ pgd += pgd_index(addr);
+ if (pgd_none(*pgd)) {
+ /* Not used yet */
+ BUG();
+ return NULL;
+ }
+ p4d = p4d_offset(pgd, addr);
+ pud = pud_offset(p4d, addr);
+ if (pud_none(*pud)) {
+ pmd_t *new_pmd;
+
+ if (!cache)
+ return NULL;
+ new_pmd = kvm_mmu_memory_cache_alloc(cache);
+ pmd_init((void *)new_pmd);
+ pud_populate(NULL, pud, new_pmd);
+ }
+ pmd = pmd_offset(pud, addr);
+ if (pmd_none(*pmd)) {
+ pte_t *new_pte;
+
+ if (!cache)
+ return NULL;
+ new_pte = kvm_mmu_memory_cache_alloc(cache);
+ clear_page(new_pte);
+ pmd_populate_kernel(NULL, pmd, new_pte);
+ }
+ return pte_offset_kernel(pmd, addr);
+}
+
+/* Caller must hold kvm->mm_lock */
+static pte_t *kvm_pte_for_gpa(struct kvm *kvm,
+ struct kvm_mmu_memory_cache *cache,
+ unsigned long addr)
+{
+ return kvm_walk_pgd(kvm->arch.gpa_mm.pgd, cache, addr);
+}
+
+/*
+ * level2_flush_{pte,pmd,pud,pgd,pt}.
+ * Flush a range of guest physical address space from the VM's GPA page tables.
+ */
+static int level2_flush_pte(pmd_t *pmd, unsigned long addr, unsigned long end)
+{
+ pte_t *pte;
+ unsigned long next, start;
+ int ret;
+
+ ret = 0;
+ start = addr;
+ pte = pte_offset_kernel(pmd, addr);
+ do {
+ next = addr + PAGE_SIZE;
+ if (!pte_present(*pte))
+ continue;
+
+ set_pte(pte, __pte(0));
+ ret = 1;
+ } while (pte++, addr = next, addr != end);
+
+ if (start + PMD_SIZE == end) {
+ pte = pte_offset_kernel(pmd, 0);
+ pmd_clear(pmd);
+ pte_free_kernel(NULL, pte);
+ }
+ return ret;
+}
+
+static int level2_flush_pmd(pud_t *pud, unsigned long addr, unsigned long end)
+{
+ pmd_t *pmd;
+ unsigned long next, start;
+ int ret;
+
+ ret = 0;
+ start = addr;
+ pmd = pmd_offset(pud, addr);
+ do {
+ next = pmd_addr_end(addr, end);
+ if (!pmd_present(*pmd))
+ continue;
+
+ ret |= level2_flush_pte(pmd, addr, next);
+ } while (pmd++, addr = next, addr != end);
+
+ if (start + PUD_SIZE == end) {
+ pmd = pmd_offset(pud, 0);
+ pud_clear(pud);
+ pmd_free(NULL, pmd);
+ }
+ return ret;
+}
+
+static int level2_flush_pud(pgd_t *pgd, unsigned long addr, unsigned long end)
+{
+ p4d_t *p4d;
+ pud_t *pud;
+ unsigned long next, start;
+ int ret;
+
+ ret = 0;
+ start = addr;
+ p4d = p4d_offset(pgd, addr);
+ pud = pud_offset(p4d, addr);
+ do {
+ next = pud_addr_end(addr, end);
+ if (!pud_present(*pud))
+ continue;
+
+ ret |= level2_flush_pmd(pud, addr, next);
+ } while (pud++, addr = next, addr != end);
+
+ if (start + PGDIR_SIZE == end) {
+ pud = pud_offset(p4d, 0);
+ pgd_clear(pgd);
+ pud_free(NULL, pud);
+ }
+ return ret;
+}
+
+static int level2_flush_pgd(pgd_t *pgd, unsigned long addr, unsigned long end)
+{
+ unsigned long next;
+ int ret;
+
+ ret = 0;
+ if (addr > end - 1)
+ return ret;
+ pgd = pgd + pgd_index(addr);
+ do {
+ next = pgd_addr_end(addr, end);
+ if (!pgd_present(*pgd))
+ continue;
+
+ ret |= level2_flush_pud(pgd, addr, next);
+ } while (pgd++, addr = next, addr != end);
+
+ return ret;
+}
+
+/**
+ * level2_flush_range() - Flush a range of guest physical addresses.
+ * @kvm: KVM pointer.
+ * @start_gfn: Guest frame number of first page in GPA range to flush.
+ * @end_gfn: Guest frame number of last page in GPA range to flush.
+ *
+ * Flushes a range of GPA mappings from the GPA page tables.
+ *
+ * The caller must hold the @kvm->mmu_lock spinlock.
+ *
+ * Returns: Whether its safe to remove the top level page directory because
+ * all lower levels have been removed.
+ */
+static bool level2_flush_range(struct kvm *kvm, gfn_t start_gfn, gfn_t end_gfn)
+{
+ return level2_flush_pgd(kvm->arch.gpa_mm.pgd, start_gfn << PAGE_SHIFT,
+ end_gfn << PAGE_SHIFT);
+}
+
+typedef int (*level2_pte_ops)(void *pte);
+/*
+ * level2_mkclean_pte
+ * Mark a range of guest physical address space clean (writes fault) in the VM's
+ * GPA page table to allow dirty page tracking.
+ */
+static int level2_mkclean_pte(void *pte)
+{
+ pte_t val;
+
+ val = *(pte_t *)pte;
+ if (pte_dirty(val)) {
+ *(pte_t *)pte = pte_mkclean(val);
+ return 1;
+ }
+ return 0;
+}
+
+static int level2_ptw_pte(pmd_t *pmd, unsigned long addr, unsigned long end,
+ level2_pte_ops func)
+{
+ pte_t *pte;
+ unsigned long next;
+ int ret;
+
+ ret = 0;
+ pte = pte_offset_kernel(pmd, addr);
+ do {
+ next = addr + PAGE_SIZE;
+ if (!pte_present(*pte))
+ continue;
+
+ ret |= func(pte);
+ } while (pte++, addr = next, addr != end);
+
+ return ret;
+}
+
+static int level2_ptw_pmd(pud_t *pud, unsigned long addr, unsigned long end,
+ level2_pte_ops func)
+{
+ pmd_t *pmd;
+ unsigned long next;
+ int ret;
+
+ ret = 0;
+ pmd = pmd_offset(pud, addr);
+ do {
+ next = pmd_addr_end(addr, end);
+ if (!pmd_present(*pmd))
+ continue;
+
+ ret |= level2_ptw_pte(pmd, addr, next, func);
+ } while (pmd++, addr = next, addr != end);
+
+ return ret;
+}
+
+static int level2_ptw_pud(pgd_t *pgd, unsigned long addr, unsigned long end,
+ level2_pte_ops func)
+{
+ p4d_t *p4d;
+ pud_t *pud;
+ unsigned long next;
+ int ret;
+
+ ret = 0;
+ p4d = p4d_offset(pgd, addr);
+ pud = pud_offset(p4d, addr);
+ do {
+ next = pud_addr_end(addr, end);
+ if (!pud_present(*pud))
+ continue;
+
+ ret |= level2_ptw_pmd(pud, addr, next, func);
+ } while (pud++, addr = next, addr != end);
+
+ return ret;
+}
+
+static int level2_ptw_pgd(pgd_t *pgd, unsigned long addr, unsigned long end,
+ level2_pte_ops func)
+{
+ unsigned long next;
+ int ret;
+
+ ret = 0;
+ if (addr > end - 1)
+ return ret;
+ pgd = pgd + pgd_index(addr);
+ do {
+ next = pgd_addr_end(addr, end);
+ if (!pgd_present(*pgd))
+ continue;
+
+ ret |= level2_ptw_pud(pgd, addr, next, func);
+ } while (pgd++, addr = next, addr != end);
+
+ return ret;
+}
+
+/*
+ * kvm_mkclean_gpa_pt() - Make a range of guest physical addresses clean.
+ * @kvm: KVM pointer.
+ * @start_gfn: Guest frame number of first page in GPA range to flush.
+ * @end_gfn: Guest frame number of last page in GPA range to flush.
+ *
+ * Make a range of GPA mappings clean so that guest writes will fault and
+ * trigger dirty page logging.
+ *
+ * The caller must hold the @kvm->mmu_lock spinlock.
+ *
+ * Returns: Whether any GPA mappings were modified, which would require
+ * derived mappings (GVA page tables & TLB enties) to be
+ * invalidated.
+ */
+static int kvm_mkclean_gpa_pt(struct kvm *kvm, gfn_t start_gfn, gfn_t end_gfn)
+{
+ return level2_ptw_pgd(kvm->arch.gpa_mm.pgd, start_gfn << PAGE_SHIFT,
+ end_gfn << PAGE_SHIFT, level2_mkclean_pte);
+}
+
+/*
+ * kvm_arch_mmu_enable_log_dirty_pt_masked() - write protect dirty pages
+ * @kvm: The KVM pointer
+ * @slot: The memory slot associated with mask
+ * @gfn_offset: The gfn offset in memory slot
+ * @mask: The mask of dirty pages at offset 'gfn_offset' in this memory
+ * slot to be write protected
+ *
+ * Walks bits set in mask write protects the associated pte's. Caller must
+ * acquire @kvm->mmu_lock.
+ */
+void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,
+ struct kvm_memory_slot *slot,
+ gfn_t gfn_offset, unsigned long mask)
+{
+ gfn_t base_gfn = slot->base_gfn + gfn_offset;
+ gfn_t start = base_gfn + __ffs(mask);
+ gfn_t end = base_gfn + __fls(mask) + 1;
+
+ kvm_mkclean_gpa_pt(kvm, start, end);
+}
+
+void kvm_arch_commit_memory_region(struct kvm *kvm,
+ struct kvm_memory_slot *old,
+ const struct kvm_memory_slot *new,
+ enum kvm_mr_change change)
+{
+ int needs_flush;
+
+ /*
+ * If dirty page logging is enabled, write protect all pages in the slot
+ * ready for dirty logging.
+ *
+ * There is no need to do this in any of the following cases:
+ * CREATE: No dirty mappings will already exist.
+ * MOVE/DELETE: The old mappings will already have been cleaned up by
+ * kvm_arch_flush_shadow_memslot()
+ */
+ if (change == KVM_MR_FLAGS_ONLY &&
+ (!(old->flags & KVM_MEM_LOG_DIRTY_PAGES) &&
+ new->flags & KVM_MEM_LOG_DIRTY_PAGES)) {
+ spin_lock(&kvm->mmu_lock);
+ /* Write protect GPA page table entries */
+ needs_flush = kvm_mkclean_gpa_pt(kvm, new->base_gfn,
+ new->base_gfn + new->npages);
+ if (needs_flush)
+ kvm_flush_remote_tlbs(kvm);
+ spin_unlock(&kvm->mmu_lock);
+ }
+}
+
+void kvm_arch_flush_shadow_all(struct kvm *kvm)
+{
+ /* Flush whole GPA */
+ level2_flush_range(kvm, 0, kvm->arch.gpa_size >> PAGE_SHIFT);
+ /* Flush vpid for each vCPU individually */
+ kvm_flush_remote_tlbs(kvm);
+}
+
+void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
+ struct kvm_memory_slot *slot)
+{
+ int ret;
+
+ /*
+ * The slot has been made invalid (ready for moving or deletion), so we
+ * need to ensure that it can no longer be accessed by any guest vCPUs.
+ */
+ spin_lock(&kvm->mmu_lock);
+ /* Flush slot from GPA */
+ ret = level2_flush_range(kvm, slot->base_gfn,
+ slot->base_gfn + slot->npages);
+ /* Let implementation do the rest */
+ if (ret)
+ kvm_flush_remote_tlbs(kvm);
+ spin_unlock(&kvm->mmu_lock);
+}
+
+void _kvm_destroy_mm(struct kvm *kvm)
+{
+ /* It should always be safe to remove after flushing the whole range */
+ level2_flush_range(kvm, 0, kvm->arch.gpa_size >> PAGE_SHIFT);
+ pgd_free(NULL, kvm->arch.gpa_mm.pgd);
+ kvm->arch.gpa_mm.pgd = NULL;
+}
+
+/*
+ * Mark a range of guest physical address space old (all accesses fault) in the
+ * VM's GPA page table to allow detection of commonly used pages.
+ */
+static int level2_mkold_pte(void *pte)
+{
+ pte_t val;
+
+ val = *(pte_t *)pte;
+ if (pte_young(val)) {
+ *(pte_t *)pte = pte_mkold(val);
+ return 1;
+ }
+ return 0;
+}
+
+bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range)
+{
+ return level2_flush_range(kvm, range->start, range->end);
+}
+
+bool kvm_set_spte_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
+{
+ gpa_t gpa = range->start << PAGE_SHIFT;
+ pte_t hva_pte = range->pte;
+ pte_t *ptep = kvm_pte_for_gpa(kvm, NULL, gpa);
+ pte_t old_pte;
+
+ if (!ptep)
+ return false;
+
+ /* Mapping may need adjusting depending on memslot flags */
+ old_pte = *ptep;
+ if (range->slot->flags & KVM_MEM_LOG_DIRTY_PAGES && !pte_dirty(old_pte))
+ hva_pte = pte_mkclean(hva_pte);
+ else if (range->slot->flags & KVM_MEM_READONLY)
+ hva_pte = pte_wrprotect(hva_pte);
+
+ set_pte(ptep, hva_pte);
+
+ /* Replacing an absent or old page doesn't need flushes */
+ if (!pte_present(old_pte) || !pte_young(old_pte))
+ return false;
+
+ /* Pages swapped, aged, moved, or cleaned require flushes */
+ return !pte_present(hva_pte) ||
+ !pte_young(hva_pte) ||
+ pte_pfn(old_pte) != pte_pfn(hva_pte) ||
+ (pte_dirty(old_pte) && !pte_dirty(hva_pte));
+}
+
+bool kvm_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
+{
+ return level2_ptw_pgd(kvm->arch.gpa_mm.pgd, range->start << PAGE_SHIFT,
+ range->end << PAGE_SHIFT, level2_mkold_pte);
+}
+
+bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
+{
+ gpa_t gpa = range->start << PAGE_SHIFT;
+ pte_t *ptep = kvm_pte_for_gpa(kvm, NULL, gpa);
+
+ if (ptep && pte_present(*ptep) && pte_young(*ptep))
+ return true;
+
+ return false;
+}
+
+/**
+ * kvm_map_page_fast() - Fast path GPA fault handler.
+ * @vcpu: vCPU pointer.
+ * @gpa: Guest physical address of fault.
+ * @write: Whether the fault was due to a write.
+ *
+ * Perform fast path GPA fault handling, doing all that can be done without
+ * calling into KVM. This handles marking old pages young (for idle page
+ * tracking), and dirtying of clean pages (for dirty page logging).
+ *
+ * Returns: 0 on success, in which case we can update derived mappings and
+ * resume guest execution.
+ * -EFAULT on failure due to absent GPA mapping or write to
+ * read-only page, in which case KVM must be consulted.
+ */
+static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa,
+ bool write)
+{
+ struct kvm *kvm = vcpu->kvm;
+ gfn_t gfn = gpa >> PAGE_SHIFT;
+ pte_t *ptep;
+ kvm_pfn_t pfn = 0;
+ bool pfn_valid = false;
+ int ret = 0;
+
+ spin_lock(&kvm->mmu_lock);
+
+ /* Fast path - just check GPA page table for an existing entry */
+ ptep = kvm_pte_for_gpa(kvm, NULL, gpa);
+ if (!ptep || !pte_present(*ptep)) {
+ ret = -EFAULT;
+ goto out;
+ }
+
+ /* Track access to pages marked old */
+ if (!pte_young(*ptep)) {
+ set_pte(ptep, pte_mkyoung(*ptep));
+ pfn = pte_pfn(*ptep);
+ pfn_valid = true;
+ /* call kvm_set_pfn_accessed() after unlock */
+ }
+ if (write && !pte_dirty(*ptep)) {
+ if (!pte_write(*ptep)) {
+ ret = -EFAULT;
+ goto out;
+ }
+
+ /* Track dirtying of writeable pages */
+ set_pte(ptep, pte_mkdirty(*ptep));
+ pfn = pte_pfn(*ptep);
+ mark_page_dirty(kvm, gfn);
+ kvm_set_pfn_dirty(pfn);
+ }
+
+out:
+ spin_unlock(&kvm->mmu_lock);
+ if (pfn_valid)
+ kvm_set_pfn_accessed(pfn);
+ return ret;
+}
+
+/**
+ * kvm_map_page() - Map a guest physical page.
+ * @vcpu: vCPU pointer.
+ * @gpa: Guest physical address of fault.
+ * @write: Whether the fault was due to a write.
+ *
+ * Handle GPA faults by creating a new GPA mapping (or updating an existing
+ * one).
+ *
+ * This takes care of marking pages young or dirty (idle/dirty page tracking),
+ * asking KVM for the corresponding PFN, and creating a mapping in the GPA page
+ * tables. Derived mappings (GVA page tables and TLBs) must be handled by the
+ * caller.
+ *
+ * Returns: 0 on success
+ * -EFAULT if there is no memory region at @gpa or a write was
+ * attempted to a read-only memory region. This is usually handled
+ * as an MMIO access.
+ */
+static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write)
+{
+ bool writeable;
+ int srcu_idx, err = 0, retry_no = 0;
+ unsigned long hva;
+ unsigned long mmu_seq;
+ unsigned long prot_bits;
+ pte_t *ptep, new_pte;
+ kvm_pfn_t pfn;
+ gfn_t gfn = gpa >> PAGE_SHIFT;
+ struct vm_area_struct *vma;
+ struct kvm *kvm = vcpu->kvm;
+ struct kvm_memory_slot *memslot;
+ struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache;
+
+ /* Try the fast path to handle old / clean pages */
+ srcu_idx = srcu_read_lock(&kvm->srcu);
+ err = kvm_map_page_fast(vcpu, gpa, write);
+ if (!err)
+ goto out;
+
+ memslot = gfn_to_memslot(kvm, gfn);
+ hva = gfn_to_hva_memslot_prot(memslot, gfn, &writeable);
+ if (kvm_is_error_hva(hva) || (write && !writeable))
+ goto out;
+
+ mmap_read_lock(current->mm);
+ vma = find_vma_intersection(current->mm, hva, hva + 1);
+ if (unlikely(!vma)) {
+ kvm_err("Failed to find VMA for hva 0x%lx\n", hva);
+ mmap_read_unlock(current->mm);
+ err = -EFAULT;
+ goto out;
+ }
+ mmap_read_unlock(current->mm);
+
+ /* We need a minimum of cached pages ready for page table creation */
+ err = kvm_mmu_topup_memory_cache(memcache, KVM_MMU_CACHE_MIN_PAGES);
+ if (err)
+ goto out;
+
+retry:
+ /*
+ * Used to check for invalidations in progress, of the pfn that is
+ * returned by pfn_to_pfn_prot below.
+ */
+ mmu_seq = kvm->mmu_invalidate_seq;
+ /*
+ * Ensure the read of mmu_invalidate_seq isn't reordered with PTE reads in
+ * gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't
+ * risk the page we get a reference to getting unmapped before we have a
+ * chance to grab the mmu_lock without mmu_invalidate_retry() noticing.
+ *
+ * This smp_rmb() pairs with the effective smp_wmb() of the combination
+ * of the pte_unmap_unlock() after the PTE is zapped, and the
+ * spin_lock() in kvm_mmu_invalidate_invalidate_<page|range_end>() before
+ * mmu_invalidate_seq is incremented.
+ */
+ smp_rmb();
+
+ /* Slow path - ask KVM core whether we can access this GPA */
+ pfn = gfn_to_pfn_prot(kvm, gfn, write, &writeable);
+ if (is_error_noslot_pfn(pfn)) {
+ err = -EFAULT;
+ goto out;
+ }
+
+ spin_lock(&kvm->mmu_lock);
+ /* Check if an invalidation has taken place since we got pfn */
+ if (mmu_invalidate_retry(kvm, mmu_seq)) {
+ /*
+ * This can happen when mappings are changed asynchronously, but
+ * also synchronously if a COW is triggered by
+ * gfn_to_pfn_prot().
+ */
+ spin_unlock(&kvm->mmu_lock);
+ kvm_set_pfn_accessed(pfn);
+ kvm_release_pfn_clean(pfn);
+ if (retry_no > 100) {
+ retry_no = 0;
+ schedule();
+ }
+ retry_no++;
+ goto retry;
+ }
+
+ /*
+ * For emulated devices such virtio device, actual cache attribute is
+ * determined by physical machine.
+ * For pass through physical device, it should be uncachable
+ */
+ prot_bits = _PAGE_PRESENT | __READABLE;
+ if (vma->vm_flags & (VM_IO | VM_PFNMAP))
+ prot_bits |= _CACHE_SUC;
+ else
+ prot_bits |= _CACHE_CC;
+
+ if (writeable) {
+ prot_bits |= _PAGE_WRITE;
+ if (write) {
+ prot_bits |= __WRITEABLE;
+ mark_page_dirty(kvm, gfn);
+ kvm_set_pfn_dirty(pfn);
+ }
+ }
+
+ /* Ensure page tables are allocated */
+ ptep = kvm_pte_for_gpa(kvm, memcache, gpa);
+ new_pte = pfn_pte(pfn, __pgprot(prot_bits));
+ set_pte(ptep, new_pte);
+
+ err = 0;
+ spin_unlock(&kvm->mmu_lock);
+ kvm_release_pfn_clean(pfn);
+ kvm_set_pfn_accessed(pfn);
+out:
+ srcu_read_unlock(&kvm->srcu, srcu_idx);
+ return err;
+}
+
+int kvm_handle_mm_fault(struct kvm_vcpu *vcpu, unsigned long gpa, bool write)
+{
+ int ret;
+
+ ret = kvm_map_page(vcpu, gpa, write);
+ if (ret)
+ return ret;
+
+ /* Invalidate this entry in the TLB */
+ return kvm_flush_tlb_gpa(vcpu, gpa);
+}
+
+void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot)
+{
+
+}
+
+int kvm_arch_prepare_memory_region(struct kvm *kvm,
+ const struct kvm_memory_slot *old,
+ struct kvm_memory_slot *new,
+ enum kvm_mr_change change)
+{
+ return 0;
+}
+
+void kvm_arch_flush_remote_tlbs_memslot(struct kvm *kvm,
+ const struct kvm_memory_slot *memslot)
+{
+ kvm_flush_remote_tlbs(kvm);
+}
--
2.39.1
Implement LoongArch vcpu world switch, including vcpu enter guest and
vcpu exit from guest, both operations need to save or restore the host
and guest registers.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kernel/asm-offsets.c | 32 +++
arch/loongarch/kvm/switch.S | 301 ++++++++++++++++++++++++++++
2 files changed, 333 insertions(+)
create mode 100644 arch/loongarch/kvm/switch.S
diff --git a/arch/loongarch/kernel/asm-offsets.c b/arch/loongarch/kernel/asm-offsets.c
index 4bdb203fc66e..cb6c5a5afea9 100644
--- a/arch/loongarch/kernel/asm-offsets.c
+++ b/arch/loongarch/kernel/asm-offsets.c
@@ -9,6 +9,7 @@
#include <linux/mm.h>
#include <linux/kbuild.h>
#include <linux/suspend.h>
+#include <linux/kvm_host.h>
#include <asm/cpu-info.h>
#include <asm/ptrace.h>
#include <asm/processor.h>
@@ -272,3 +273,34 @@ void output_pbe_defines(void)
BLANK();
}
#endif
+
+static void __used output_kvm_defines(void)
+{
+ COMMENT(" KVM/LOONGARCH Specific offsets. ");
+
+ OFFSET(VCPU_FCSR0, kvm_vcpu_arch, fpu.fcsr);
+ OFFSET(VCPU_FCC, kvm_vcpu_arch, fpu.fcc);
+ BLANK();
+
+ OFFSET(KVM_VCPU_ARCH, kvm_vcpu, arch);
+ OFFSET(KVM_VCPU_KVM, kvm_vcpu, kvm);
+ OFFSET(KVM_VCPU_RUN, kvm_vcpu, run);
+ BLANK();
+
+ OFFSET(KVM_ARCH_HSP, kvm_vcpu_arch, host_sp);
+ OFFSET(KVM_ARCH_HTP, kvm_vcpu_arch, host_tp);
+ OFFSET(KVM_ARCH_HANDLE_EXIT, kvm_vcpu_arch, handle_exit);
+ OFFSET(KVM_ARCH_HPGD, kvm_vcpu_arch, host_pgd);
+ OFFSET(KVM_ARCH_GEENTRY, kvm_vcpu_arch, guest_eentry);
+ OFFSET(KVM_ARCH_GPC, kvm_vcpu_arch, pc);
+ OFFSET(KVM_ARCH_GGPR, kvm_vcpu_arch, gprs);
+ OFFSET(KVM_ARCH_HESTAT, kvm_vcpu_arch, host_estat);
+ OFFSET(KVM_ARCH_HBADV, kvm_vcpu_arch, badv);
+ OFFSET(KVM_ARCH_HBADI, kvm_vcpu_arch, badi);
+ OFFSET(KVM_ARCH_HECFG, kvm_vcpu_arch, host_ecfg);
+ OFFSET(KVM_ARCH_HEENTRY, kvm_vcpu_arch, host_eentry);
+ OFFSET(KVM_ARCH_HPERCPU, kvm_vcpu_arch, host_percpu);
+
+ OFFSET(KVM_GPGD, kvm, arch.gpa_mm.pgd);
+ BLANK();
+}
diff --git a/arch/loongarch/kvm/switch.S b/arch/loongarch/kvm/switch.S
new file mode 100644
index 000000000000..f9f6e0707cd7
--- /dev/null
+++ b/arch/loongarch/kvm/switch.S
@@ -0,0 +1,301 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#include <linux/linkage.h>
+#include <asm/stackframe.h>
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/regdef.h>
+#include <asm/loongarch.h>
+#include <asm/export.h>
+
+#define PT_GPR_OFFSET(x) (PT_R0 + 8*x)
+ .text
+
+.macro kvm_save_host_gpr base
+ .irp n,1,2,3,22,23,24,25,26,27,28,29,30,31
+ st.d $r\n, \base, PT_GPR_OFFSET(\n)
+ .endr
+.endm
+
+.macro kvm_restore_host_gpr base
+ .irp n,1,2,3,22,23,24,25,26,27,28,29,30,31
+ ld.d $r\n, \base, PT_GPR_OFFSET(\n)
+ .endr
+.endm
+
+/*
+ * prepare switch to guest, save host reg and restore guest reg.
+ * a2: kvm_vcpu_arch, don't touch it until 'ertn'
+ * t0, t1: temp register
+ */
+.macro kvm_switch_to_guest
+ /* set host excfg.VS=0, all exceptions share one exception entry */
+ csrrd t0, LOONGARCH_CSR_ECFG
+ bstrins.w t0, zero, CSR_ECFG_VS_SHIFT_END, CSR_ECFG_VS_SHIFT
+ csrwr t0, LOONGARCH_CSR_ECFG
+
+ /* Load up the new EENTRY */
+ ld.d t0, a2, KVM_ARCH_GEENTRY
+ csrwr t0, LOONGARCH_CSR_EENTRY
+
+ /* Set Guest ERA */
+ ld.d t0, a2, KVM_ARCH_GPC
+ csrwr t0, LOONGARCH_CSR_ERA
+
+ /* Save host PGDL */
+ csrrd t0, LOONGARCH_CSR_PGDL
+ st.d t0, a2, KVM_ARCH_HPGD
+
+ /* Switch to kvm */
+ ld.d t1, a2, KVM_VCPU_KVM - KVM_VCPU_ARCH
+
+ /* Load guest PGDL */
+ lu12i.w t0, KVM_GPGD
+ srli.w t0, t0, 12
+ ldx.d t0, t1, t0
+ csrwr t0, LOONGARCH_CSR_PGDL
+
+ /* Mix GID and RID */
+ csrrd t1, LOONGARCH_CSR_GSTAT
+ bstrpick.w t1, t1, CSR_GSTAT_GID_SHIFT_END, CSR_GSTAT_GID_SHIFT
+ csrrd t0, LOONGARCH_CSR_GTLBC
+ bstrins.w t0, t1, CSR_GTLBC_TGID_SHIFT_END, CSR_GTLBC_TGID_SHIFT
+ csrwr t0, LOONGARCH_CSR_GTLBC
+
+ /*
+ * Switch to guest:
+ * GSTAT.PGM = 1, ERRCTL.ISERR = 0, TLBRPRMD.ISTLBR = 0
+ * ertn
+ */
+
+ /*
+ * Enable intr in root mode with future ertn so that host interrupt
+ * can be responsed during VM runs
+ * guest crmd comes from separate gcsr_CRMD register
+ */
+ ori t0, zero, CSR_PRMD_PIE
+ csrxchg t0, t0, LOONGARCH_CSR_PRMD
+
+ /* Set PVM bit to setup ertn to guest context */
+ ori t0, zero, CSR_GSTAT_PVM
+ csrxchg t0, t0, LOONGARCH_CSR_GSTAT
+
+ /* Load Guest gprs */
+ ld.d ra, a2, (KVM_ARCH_GGPR + 8 * REG_RA)
+ ld.d tp, a2, (KVM_ARCH_GGPR + 8 * REG_TP)
+ ld.d sp, a2, (KVM_ARCH_GGPR + 8 * REG_SP)
+ ld.d a0, a2, (KVM_ARCH_GGPR + 8 * REG_A0)
+ ld.d a1, a2, (KVM_ARCH_GGPR + 8 * REG_A1)
+ ld.d a3, a2, (KVM_ARCH_GGPR + 8 * REG_A3)
+ ld.d a4, a2, (KVM_ARCH_GGPR + 8 * REG_A4)
+ ld.d a5, a2, (KVM_ARCH_GGPR + 8 * REG_A5)
+ ld.d a6, a2, (KVM_ARCH_GGPR + 8 * REG_A6)
+ ld.d a7, a2, (KVM_ARCH_GGPR + 8 * REG_A7)
+ ld.d t0, a2, (KVM_ARCH_GGPR + 8 * REG_T0)
+ ld.d t1, a2, (KVM_ARCH_GGPR + 8 * REG_T1)
+ ld.d t2, a2, (KVM_ARCH_GGPR + 8 * REG_T2)
+ ld.d t3, a2, (KVM_ARCH_GGPR + 8 * REG_T3)
+ ld.d t4, a2, (KVM_ARCH_GGPR + 8 * REG_T4)
+ ld.d t5, a2, (KVM_ARCH_GGPR + 8 * REG_T5)
+ ld.d t6, a2, (KVM_ARCH_GGPR + 8 * REG_T6)
+ ld.d t7, a2, (KVM_ARCH_GGPR + 8 * REG_T7)
+ ld.d t8, a2, (KVM_ARCH_GGPR + 8 * REG_T8)
+ ld.d u0, a2, (KVM_ARCH_GGPR + 8 * REG_U0)
+ ld.d fp, a2, (KVM_ARCH_GGPR + 8 * REG_FP)
+ ld.d s0, a2, (KVM_ARCH_GGPR + 8 * REG_S0)
+ ld.d s1, a2, (KVM_ARCH_GGPR + 8 * REG_S1)
+ ld.d s2, a2, (KVM_ARCH_GGPR + 8 * REG_S2)
+ ld.d s3, a2, (KVM_ARCH_GGPR + 8 * REG_S3)
+ ld.d s4, a2, (KVM_ARCH_GGPR + 8 * REG_S4)
+ ld.d s5, a2, (KVM_ARCH_GGPR + 8 * REG_S5)
+ ld.d s6, a2, (KVM_ARCH_GGPR + 8 * REG_S6)
+ ld.d s7, a2, (KVM_ARCH_GGPR + 8 * REG_S7)
+ ld.d s8, a2, (KVM_ARCH_GGPR + 8 * REG_S8)
+ /* Load KVM_ARCH register */
+ ld.d a2, a2, (KVM_ARCH_GGPR + 8 * REG_A2)
+
+ ertn
+.endm
+
+ /*
+ * exception entry for general exception from guest mode
+ * - IRQ is disabled
+ * - kernel privilege in root mode
+ * - page mode keep unchanged from previous prmd in root mode
+ * - Fixme: tlb exception cannot happen since registers relative with TLB
+ * - is still in guest mode, such as pgd table/vmid registers etc,
+ * - will fix with hw page walk enabled in future
+ * load kvm_vcpu from reserved CSR KVM_VCPU_KS, and save a2 to KVM_TEMP_KS
+ */
+ .section .text
+ .cfi_sections .debug_frame
+SYM_CODE_START(kvm_vector_entry)
+ csrwr a2, KVM_TEMP_KS
+ csrrd a2, KVM_VCPU_KS
+ addi.d a2, a2, KVM_VCPU_ARCH
+
+ /* After save gprs, free to use any gpr */
+ st.d ra, a2, (KVM_ARCH_GGPR + 8 * REG_RA)
+ st.d tp, a2, (KVM_ARCH_GGPR + 8 * REG_TP)
+ st.d sp, a2, (KVM_ARCH_GGPR + 8 * REG_SP)
+ st.d a0, a2, (KVM_ARCH_GGPR + 8 * REG_A0)
+ st.d a1, a2, (KVM_ARCH_GGPR + 8 * REG_A1)
+ st.d a3, a2, (KVM_ARCH_GGPR + 8 * REG_A3)
+ st.d a4, a2, (KVM_ARCH_GGPR + 8 * REG_A4)
+ st.d a5, a2, (KVM_ARCH_GGPR + 8 * REG_A5)
+ st.d a6, a2, (KVM_ARCH_GGPR + 8 * REG_A6)
+ st.d a7, a2, (KVM_ARCH_GGPR + 8 * REG_A7)
+ st.d t0, a2, (KVM_ARCH_GGPR + 8 * REG_T0)
+ st.d t1, a2, (KVM_ARCH_GGPR + 8 * REG_T1)
+ st.d t2, a2, (KVM_ARCH_GGPR + 8 * REG_T2)
+ st.d t3, a2, (KVM_ARCH_GGPR + 8 * REG_T3)
+ st.d t4, a2, (KVM_ARCH_GGPR + 8 * REG_T4)
+ st.d t5, a2, (KVM_ARCH_GGPR + 8 * REG_T5)
+ st.d t6, a2, (KVM_ARCH_GGPR + 8 * REG_T6)
+ st.d t7, a2, (KVM_ARCH_GGPR + 8 * REG_T7)
+ st.d t8, a2, (KVM_ARCH_GGPR + 8 * REG_T8)
+ st.d u0, a2, (KVM_ARCH_GGPR + 8 * REG_U0)
+ st.d fp, a2, (KVM_ARCH_GGPR + 8 * REG_FP)
+ st.d s0, a2, (KVM_ARCH_GGPR + 8 * REG_S0)
+ st.d s1, a2, (KVM_ARCH_GGPR + 8 * REG_S1)
+ st.d s2, a2, (KVM_ARCH_GGPR + 8 * REG_S2)
+ st.d s3, a2, (KVM_ARCH_GGPR + 8 * REG_S3)
+ st.d s4, a2, (KVM_ARCH_GGPR + 8 * REG_S4)
+ st.d s5, a2, (KVM_ARCH_GGPR + 8 * REG_S5)
+ st.d s6, a2, (KVM_ARCH_GGPR + 8 * REG_S6)
+ st.d s7, a2, (KVM_ARCH_GGPR + 8 * REG_S7)
+ st.d s8, a2, (KVM_ARCH_GGPR + 8 * REG_S8)
+ /* Save guest a2 */
+ csrrd t0, KVM_TEMP_KS
+ st.d t0, a2, (KVM_ARCH_GGPR + 8 * REG_A2)
+
+ /* a2: kvm_vcpu_arch, a1 is free to use */
+ csrrd s1, KVM_VCPU_KS
+ ld.d s0, s1, KVM_VCPU_RUN
+
+ csrrd t0, LOONGARCH_CSR_ESTAT
+ st.d t0, a2, KVM_ARCH_HESTAT
+ csrrd t0, LOONGARCH_CSR_ERA
+ st.d t0, a2, KVM_ARCH_GPC
+ csrrd t0, LOONGARCH_CSR_BADV
+ st.d t0, a2, KVM_ARCH_HBADV
+ csrrd t0, LOONGARCH_CSR_BADI
+ st.d t0, a2, KVM_ARCH_HBADI
+
+ /* Restore host excfg.VS */
+ csrrd t0, LOONGARCH_CSR_ECFG
+ ld.d t1, a2, KVM_ARCH_HECFG
+ or t0, t0, t1
+ csrwr t0, LOONGARCH_CSR_ECFG
+
+ /* Restore host eentry */
+ ld.d t0, a2, KVM_ARCH_HEENTRY
+ csrwr t0, LOONGARCH_CSR_EENTRY
+
+ /* restore host pgd table */
+ ld.d t0, a2, KVM_ARCH_HPGD
+ csrwr t0, LOONGARCH_CSR_PGDL
+
+ /*
+ * Disable PGM bit to enter root mode by default with next ertn
+ */
+ ori t0, zero, CSR_GSTAT_PVM
+ csrxchg zero, t0, LOONGARCH_CSR_GSTAT
+ /*
+ * Clear GTLBC.TGID field
+ * 0: for root tlb update in future tlb instr
+ * others: for guest tlb update like gpa to hpa in future tlb instr
+ */
+ csrrd t0, LOONGARCH_CSR_GTLBC
+ bstrins.w t0, zero, CSR_GTLBC_TGID_SHIFT_END, CSR_GTLBC_TGID_SHIFT
+ csrwr t0, LOONGARCH_CSR_GTLBC
+ ld.d tp, a2, KVM_ARCH_HTP
+ ld.d sp, a2, KVM_ARCH_HSP
+ /* restore per cpu register */
+ ld.d u0, a2, KVM_ARCH_HPERCPU
+ addi.d sp, sp, -PT_SIZE
+
+ /* Prepare handle exception */
+ or a0, s0, zero
+ or a1, s1, zero
+ ld.d t8, a2, KVM_ARCH_HANDLE_EXIT
+ jirl ra, t8, 0
+
+ or a2, s1, zero
+ addi.d a2, a2, KVM_VCPU_ARCH
+
+ /* resume host when ret <= 0 */
+ bge zero, a0, ret_to_host
+
+ /*
+ * return to guest
+ * save per cpu register again, maybe switched to another cpu
+ */
+ st.d u0, a2, KVM_ARCH_HPERCPU
+
+ /* Save kvm_vcpu to kscratch */
+ csrwr s1, KVM_VCPU_KS
+ kvm_switch_to_guest
+
+ret_to_host:
+ ld.d a2, a2, KVM_ARCH_HSP
+ addi.d a2, a2, -PT_SIZE
+ kvm_restore_host_gpr a2
+ jr ra
+SYM_CODE_END(kvm_vector_entry)
+kvm_vector_entry_end:
+
+/*
+ * int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu)
+ *
+ * @register_param:
+ * a0: kvm_run* run
+ * a1: kvm_vcpu* vcpu
+ */
+SYM_FUNC_START(kvm_enter_guest)
+ /* allocate space in stack bottom */
+ addi.d a2, sp, -PT_SIZE
+ /* save host gprs */
+ kvm_save_host_gpr a2
+
+ /* save host crmd,prmd csr to stack */
+ csrrd a3, LOONGARCH_CSR_CRMD
+ st.d a3, a2, PT_CRMD
+ csrrd a3, LOONGARCH_CSR_PRMD
+ st.d a3, a2, PT_PRMD
+
+ addi.d a2, a1, KVM_VCPU_ARCH
+ st.d sp, a2, KVM_ARCH_HSP
+ st.d tp, a2, KVM_ARCH_HTP
+ /* Save per cpu register */
+ st.d u0, a2, KVM_ARCH_HPERCPU
+
+ /* Save kvm_vcpu to kscratch */
+ csrwr a1, KVM_VCPU_KS
+ kvm_switch_to_guest
+SYM_FUNC_END(kvm_enter_guest)
+kvm_enter_guest_end:
+
+ .section ".rodata"
+SYM_DATA(kvm_vector_size,
+ .quad kvm_vector_entry_end - kvm_vector_entry)
+SYM_DATA(kvm_enter_guest_size,
+ .quad kvm_enter_guest_end - kvm_enter_guest)
+
+
+SYM_FUNC_START(kvm_save_fpu)
+ fpu_save_csr a0 t1
+ fpu_save_double a0 t1
+ fpu_save_cc a0 t1 t2
+ jr ra
+SYM_FUNC_END(kvm_save_fpu)
+
+SYM_FUNC_START(kvm_restore_fpu)
+ fpu_restore_double a0 t1
+ fpu_restore_csr a0 t1
+ fpu_restore_cc a0 t1 t2
+ jr ra
+SYM_FUNC_END(kvm_restore_fpu)
--
2.39.1
Implement LoongArch fpu related interface for vcpu, such as get fpu, set
fpu, own fpu and lose fpu, etc.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/vcpu.c | 60 +++++++++++++++++++++++++++++++++++++++
1 file changed, 60 insertions(+)
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index da97b77da8eb..71f5e0fb96e5 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -237,6 +237,66 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
return r;
}
+int kvm_arch_vcpu_ioctl_get_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu)
+{
+ int i = 0;
+
+ /* no need vcpu_load and vcpu_put */
+ fpu->fcsr = vcpu->arch.fpu.fcsr;
+ fpu->fcc = vcpu->arch.fpu.fcc;
+ for (i = 0; i < NUM_FPU_REGS; i++)
+ memcpy(&fpu->fpr[i], &vcpu->arch.fpu.fpr[i], FPU_REG_WIDTH / 64);
+
+ return 0;
+}
+
+int kvm_arch_vcpu_ioctl_set_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu)
+{
+ int i = 0;
+
+ /* no need vcpu_load and vcpu_put */
+ vcpu->arch.fpu.fcsr = fpu->fcsr;
+ vcpu->arch.fpu.fcc = fpu->fcc;
+ for (i = 0; i < NUM_FPU_REGS; i++)
+ memcpy(&vcpu->arch.fpu.fpr[i], &fpu->fpr[i], FPU_REG_WIDTH / 64);
+
+ return 0;
+}
+
+/* Enable FPU for guest and restore context */
+void kvm_own_fpu(struct kvm_vcpu *vcpu)
+{
+ preempt_disable();
+
+ /*
+ * Enable FPU for guest
+ */
+ set_csr_euen(CSR_EUEN_FPEN);
+
+ kvm_restore_fpu(&vcpu->arch.fpu);
+ vcpu->arch.aux_inuse |= KVM_LARCH_FPU;
+ trace_kvm_aux(vcpu, KVM_TRACE_AUX_RESTORE, KVM_TRACE_AUX_FPU);
+
+ preempt_enable();
+}
+
+/* Save and disable FPU */
+void kvm_lose_fpu(struct kvm_vcpu *vcpu)
+{
+ preempt_disable();
+
+ if (vcpu->arch.aux_inuse & KVM_LARCH_FPU) {
+ kvm_save_fpu(&vcpu->arch.fpu);
+ vcpu->arch.aux_inuse &= ~KVM_LARCH_FPU;
+ trace_kvm_aux(vcpu, KVM_TRACE_AUX_SAVE, KVM_TRACE_AUX_FPU);
+
+ /* Disable FPU */
+ clear_csr_euen(CSR_EUEN_FPEN);
+ }
+
+ preempt_enable();
+}
+
int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
{
return 0;
--
2.39.1
Supplement kvm document about LoongArch-specific part, such as add
api introduction for GET/SET_ONE_REG, GET/SET_FPU, GET/SET_MP_STATE,
etc.
Signed-off-by: Tianrui Zhao <[email protected]>
---
Documentation/virt/kvm/api.rst | 71 +++++++++++++++++++++++++++++-----
1 file changed, 62 insertions(+), 9 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index add067793b90..ad8e13eab48d 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -416,6 +416,12 @@ Reads the general purpose registers from the vcpu.
__u64 pc;
};
+ /* LoongArch */
+ struct kvm_regs {
+ unsigned long gpr[32];
+ unsigned long pc;
+ };
+
4.12 KVM_SET_REGS
-----------------
@@ -506,7 +512,7 @@ translation mode.
------------------
:Capability: basic
-:Architectures: x86, ppc, mips, riscv
+:Architectures: x86, ppc, mips, riscv, loongarch
:Type: vcpu ioctl
:Parameters: struct kvm_interrupt (in)
:Returns: 0 on success, negative on failure.
@@ -592,6 +598,14 @@ b) KVM_INTERRUPT_UNSET
This is an asynchronous vcpu ioctl and can be invoked from any thread.
+LOONGARCH:
+^^^^^^^^^^
+
+Queues an external interrupt to be injected into the virtual CPU. A negative
+interrupt number dequeues the interrupt.
+
+This is an asynchronous vcpu ioctl and can be invoked from any thread.
+
4.17 KVM_DEBUG_GUEST
--------------------
@@ -737,7 +751,7 @@ signal mask.
----------------
:Capability: basic
-:Architectures: x86
+:Architectures: x86, loongarch
:Type: vcpu ioctl
:Parameters: struct kvm_fpu (out)
:Returns: 0 on success, -1 on error
@@ -746,7 +760,7 @@ Reads the floating point state from the vcpu.
::
- /* for KVM_GET_FPU and KVM_SET_FPU */
+ /* x86: for KVM_GET_FPU and KVM_SET_FPU */
struct kvm_fpu {
__u8 fpr[8][16];
__u16 fcw;
@@ -761,12 +775,22 @@ Reads the floating point state from the vcpu.
__u32 pad2;
};
+ /* LoongArch: for KVM_GET_FPU and KVM_SET_FPU */
+ struct kvm_fpu {
+ __u32 fcsr;
+ __u32 none;
+ __u64 fcc;
+ struct kvm_fpureg {
+ __u64 val64[4];
+ }fpr[32];
+ };
+
4.23 KVM_SET_FPU
----------------
:Capability: basic
-:Architectures: x86
+:Architectures: x86, loongarch
:Type: vcpu ioctl
:Parameters: struct kvm_fpu (in)
:Returns: 0 on success, -1 on error
@@ -775,7 +799,7 @@ Writes the floating point state to the vcpu.
::
- /* for KVM_GET_FPU and KVM_SET_FPU */
+ /* x86: for KVM_GET_FPU and KVM_SET_FPU */
struct kvm_fpu {
__u8 fpr[8][16];
__u16 fcw;
@@ -790,6 +814,16 @@ Writes the floating point state to the vcpu.
__u32 pad2;
};
+ /* LoongArch: for KVM_GET_FPU and KVM_SET_FPU */
+ struct kvm_fpu {
+ __u32 fcsr;
+ __u32 none;
+ __u64 fcc;
+ struct kvm_fpureg {
+ __u64 val64[4];
+ }fpr[32];
+ };
+
4.24 KVM_CREATE_IRQCHIP
-----------------------
@@ -1387,7 +1421,7 @@ documentation when it pops into existence).
-------------------
:Capability: KVM_CAP_ENABLE_CAP
-:Architectures: mips, ppc, s390, x86
+:Architectures: mips, ppc, s390, x86, loongarch
:Type: vcpu ioctl
:Parameters: struct kvm_enable_cap (in)
:Returns: 0 on success; -1 on error
@@ -1442,7 +1476,7 @@ for vm-wide capabilities.
---------------------
:Capability: KVM_CAP_MP_STATE
-:Architectures: x86, s390, arm64, riscv
+:Architectures: x86, s390, arm64, riscv, loongarch
:Type: vcpu ioctl
:Parameters: struct kvm_mp_state (out)
:Returns: 0 on success; -1 on error
@@ -1460,7 +1494,7 @@ Possible values are:
========================== ===============================================
KVM_MP_STATE_RUNNABLE the vcpu is currently running
- [x86,arm64,riscv]
+ [x86,arm64,riscv,loongarch]
KVM_MP_STATE_UNINITIALIZED the vcpu is an application processor (AP)
which has not yet received an INIT signal [x86]
KVM_MP_STATE_INIT_RECEIVED the vcpu has received an INIT signal, and is
@@ -1516,11 +1550,14 @@ For riscv:
The only states that are valid are KVM_MP_STATE_STOPPED and
KVM_MP_STATE_RUNNABLE which reflect if the vcpu is paused or not.
+On LoongArch, only the KVM_MP_STATE_RUNNABLE state is used to reflect
+whether the vcpu is runnable.
+
4.39 KVM_SET_MP_STATE
---------------------
:Capability: KVM_CAP_MP_STATE
-:Architectures: x86, s390, arm64, riscv
+:Architectures: x86, s390, arm64, riscv, loongarch
:Type: vcpu ioctl
:Parameters: struct kvm_mp_state (in)
:Returns: 0 on success; -1 on error
@@ -1538,6 +1575,9 @@ For arm64/riscv:
The only states that are valid are KVM_MP_STATE_STOPPED and
KVM_MP_STATE_RUNNABLE which reflect if the vcpu should be paused or not.
+On LoongArch, only the KVM_MP_STATE_RUNNABLE state is used to reflect
+whether the vcpu is runnable.
+
4.40 KVM_SET_IDENTITY_MAP_ADDR
------------------------------
@@ -2839,6 +2879,19 @@ Following are the RISC-V D-extension registers:
0x8020 0000 0600 0020 fcsr Floating point control and status register
======================= ========= =============================================
+LoongArch registers are mapped using the lower 32 bits. The upper 16 bits of
+that is the register group type.
+
+LoongArch csr registers are used to control guest cpu or get status of guest
+cpu, and they have the following id bit patterns::
+
+ 0x9030 0000 0001 00 <reg:5> <sel:3> (64-bit)
+
+LoongArch KVM control registers are used to implement some new defined functions
+such as set vcpu counter or reset vcpu, and they have the following id bit patterns::
+
+ 0x9030 0000 0002 <reg:16>
+
4.69 KVM_GET_ONE_REG
--------------------
--
2.39.1
Implement vcpu create and destroy interface, saving some info
into vcpu arch structure such as vcpu exception entrance, vcpu
enter guest pointer, etc. Init vcpu timer and set address
translation mode when vcpu create.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/vcpu.c | 88 +++++++++++++++++++++++++++++++++++++++
1 file changed, 88 insertions(+)
create mode 100644 arch/loongarch/kvm/vcpu.c
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
new file mode 100644
index 000000000000..24b5b00266a1
--- /dev/null
+++ b/arch/loongarch/kvm/vcpu.c
@@ -0,0 +1,88 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#include <linux/kvm_host.h>
+#include <asm/fpu.h>
+#include <asm/loongarch.h>
+#include <asm/setup.h>
+#include <asm/time.h>
+
+#define CREATE_TRACE_POINTS
+#include "trace.h"
+
+int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
+{
+ return 0;
+}
+
+int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
+{
+ unsigned long timer_hz;
+ struct loongarch_csrs *csr;
+
+ vcpu->arch.vpid = 0;
+
+ hrtimer_init(&vcpu->arch.swtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_PINNED);
+ vcpu->arch.swtimer.function = kvm_swtimer_wakeup;
+ vcpu->kvm->arch.online_vcpus = vcpu->vcpu_id + 1;
+
+ vcpu->arch.guest_eentry = (unsigned long)kvm_loongarch_ops->guest_eentry;
+ vcpu->arch.handle_exit = _kvm_handle_exit;
+ vcpu->arch.csr = kzalloc(sizeof(struct loongarch_csrs), GFP_KERNEL);
+ if (!vcpu->arch.csr)
+ return -ENOMEM;
+
+ /*
+ * kvm all exceptions share one exception entry, and host <-> guest switch
+ * also switch excfg.VS field, keep host excfg.VS info here
+ */
+ vcpu->arch.host_ecfg = (read_csr_ecfg() & CSR_ECFG_VS);
+
+ /* Init */
+ vcpu->arch.last_sched_cpu = -1;
+ vcpu->arch.last_exec_cpu = -1;
+
+ /*
+ * Initialize guest register state to valid architectural reset state.
+ */
+ timer_hz = calc_const_freq();
+ kvm_init_timer(vcpu, timer_hz);
+
+ /* Set Initialize mode for GUEST */
+ csr = vcpu->arch.csr;
+ kvm_write_sw_gcsr(csr, LOONGARCH_CSR_CRMD, CSR_CRMD_DA);
+
+ /* Set cpuid */
+ kvm_write_sw_gcsr(csr, LOONGARCH_CSR_TMID, vcpu->vcpu_id);
+
+ /* start with no pending virtual guest interrupts */
+ csr->csrs[LOONGARCH_CSR_GINTC] = 0;
+
+ return 0;
+}
+
+void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
+{
+}
+
+void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
+{
+ int cpu;
+ struct kvm_context *context;
+
+ hrtimer_cancel(&vcpu->arch.swtimer);
+ kvm_mmu_free_memory_cache(&vcpu->arch.mmu_page_cache);
+ kfree(vcpu->arch.csr);
+
+ /*
+ * If the vCPU is freed and reused as another vCPU, we don't want the
+ * matching pointer wrongly hanging around in last_vcpu.
+ */
+ for_each_possible_cpu(cpu) {
+ context = per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu);
+ if (context->last_vcpu == vcpu)
+ context->last_vcpu = NULL;
+ }
+}
--
2.39.1
Implement handle mmio exception, setting the mmio info into vcpu_run and
return to user space to handle it.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/exit.c | 308 ++++++++++++++++++++++++++++++++++++++
1 file changed, 308 insertions(+)
diff --git a/arch/loongarch/kvm/exit.c b/arch/loongarch/kvm/exit.c
index 04e4339cd78d..36a838607218 100644
--- a/arch/loongarch/kvm/exit.c
+++ b/arch/loongarch/kvm/exit.c
@@ -209,6 +209,265 @@ int _kvm_emu_idle(struct kvm_vcpu *vcpu)
return EMULATE_DONE;
}
+int _kvm_emu_mmio_write(struct kvm_vcpu *vcpu, larch_inst inst)
+{
+ struct kvm_run *run = vcpu->run;
+ unsigned int rd, op8, opcode;
+ unsigned long rd_val = 0;
+ void *data = run->mmio.data;
+ unsigned long curr_pc;
+ int ret;
+
+ /*
+ * Update PC and hold onto current PC in case there is
+ * an error and we want to rollback the PC
+ */
+ curr_pc = vcpu->arch.pc;
+ update_pc(&vcpu->arch);
+
+ op8 = (inst.word >> 24) & 0xff;
+ run->mmio.phys_addr = vcpu->arch.badv;
+ ret = EMULATE_DO_MMIO;
+ if (op8 < 0x28) {
+ /* stptrw/d process */
+ rd = inst.reg2i14_format.rd;
+ opcode = inst.reg2i14_format.opcode;
+
+ switch (opcode) {
+ case stptrd_op:
+ run->mmio.len = 8;
+ *(unsigned long *)data = vcpu->arch.gprs[rd];
+ break;
+ case stptrw_op:
+ run->mmio.len = 4;
+ *(unsigned int *)data = vcpu->arch.gprs[rd];
+ break;
+ default:
+ ret = EMULATE_FAIL;
+ break;
+ }
+ } else if (op8 < 0x30) {
+ /* st.b/h/w/d process */
+ rd = inst.reg2i12_format.rd;
+ opcode = inst.reg2i12_format.opcode;
+ rd_val = vcpu->arch.gprs[rd];
+
+ switch (opcode) {
+ case std_op:
+ run->mmio.len = 8;
+ *(unsigned long *)data = rd_val;
+ break;
+ case stw_op:
+ run->mmio.len = 4;
+ *(unsigned int *)data = rd_val;
+ break;
+ case sth_op:
+ run->mmio.len = 2;
+ *(unsigned short *)data = rd_val;
+ break;
+ case stb_op:
+ run->mmio.len = 1;
+ *(unsigned char *)data = rd_val;
+ break;
+ default:
+ ret = EMULATE_FAIL;
+ break;
+ }
+ } else if (op8 == 0x38) {
+ /* stxb/h/w/d process */
+ rd = inst.reg3_format.rd;
+ opcode = inst.reg3_format.opcode;
+
+ switch (opcode) {
+ case stxb_op:
+ run->mmio.len = 1;
+ *(unsigned char *)data = vcpu->arch.gprs[rd];
+ break;
+ case stxh_op:
+ run->mmio.len = 2;
+ *(unsigned short *)data = vcpu->arch.gprs[rd];
+ break;
+ case stxw_op:
+ run->mmio.len = 4;
+ *(unsigned int *)data = vcpu->arch.gprs[rd];
+ break;
+ case stxd_op:
+ run->mmio.len = 8;
+ *(unsigned long *)data = vcpu->arch.gprs[rd];
+ break;
+ default:
+ ret = EMULATE_FAIL;
+ break;
+ }
+ } else
+ ret = EMULATE_FAIL;
+
+ if (ret == EMULATE_DO_MMIO) {
+ run->mmio.is_write = 1;
+ vcpu->mmio_needed = 1;
+ vcpu->mmio_is_write = 1;
+ } else {
+ vcpu->arch.pc = curr_pc;
+ kvm_err("Write not supporded inst=0x%08x @%lx BadVaddr:%#lx\n",
+ inst.word, vcpu->arch.pc, vcpu->arch.badv);
+ kvm_arch_vcpu_dump_regs(vcpu);
+ /* Rollback PC if emulation was unsuccessful */
+ }
+
+ return ret;
+}
+
+int _kvm_emu_mmio_read(struct kvm_vcpu *vcpu, larch_inst inst)
+{
+ unsigned int op8, opcode, rd;
+ struct kvm_run *run = vcpu->run;
+ int ret;
+
+ run->mmio.phys_addr = vcpu->arch.badv;
+ vcpu->mmio_needed = 2; /* signed */
+ op8 = (inst.word >> 24) & 0xff;
+ ret = EMULATE_DO_MMIO;
+
+ if (op8 < 0x28) {
+ /* ldptr.w/d process */
+ rd = inst.reg2i14_format.rd;
+ opcode = inst.reg2i14_format.opcode;
+
+ switch (opcode) {
+ case ldptrd_op:
+ run->mmio.len = 8;
+ break;
+ case ldptrw_op:
+ run->mmio.len = 4;
+ break;
+ default:
+ break;
+ }
+ } else if (op8 < 0x2f) {
+ /* ld.b/h/w/d, ld.bu/hu/wu process */
+ rd = inst.reg2i12_format.rd;
+ opcode = inst.reg2i12_format.opcode;
+
+ switch (opcode) {
+ case ldd_op:
+ run->mmio.len = 8;
+ break;
+ case ldwu_op:
+ vcpu->mmio_needed = 1; /* unsigned */
+ run->mmio.len = 4;
+ break;
+ case ldw_op:
+ run->mmio.len = 4;
+ break;
+ case ldhu_op:
+ vcpu->mmio_needed = 1; /* unsigned */
+ run->mmio.len = 2;
+ break;
+ case ldh_op:
+ run->mmio.len = 2;
+ break;
+ case ldbu_op:
+ vcpu->mmio_needed = 1; /* unsigned */
+ run->mmio.len = 1;
+ break;
+ case ldb_op:
+ run->mmio.len = 1;
+ break;
+ default:
+ ret = EMULATE_FAIL;
+ break;
+ }
+ } else if (op8 == 0x38) {
+ /* ldxb/h/w/d, ldxb/h/wu, ldgtb/h/w/d, ldleb/h/w/d process */
+ rd = inst.reg3_format.rd;
+ opcode = inst.reg3_format.opcode;
+
+ switch (opcode) {
+ case ldxb_op:
+ run->mmio.len = 1;
+ break;
+ case ldxbu_op:
+ run->mmio.len = 1;
+ vcpu->mmio_needed = 1; /* unsigned */
+ break;
+ case ldxh_op:
+ run->mmio.len = 2;
+ break;
+ case ldxhu_op:
+ run->mmio.len = 2;
+ vcpu->mmio_needed = 1; /* unsigned */
+ break;
+ case ldxw_op:
+ run->mmio.len = 4;
+ break;
+ case ldxwu_op:
+ run->mmio.len = 4;
+ vcpu->mmio_needed = 1; /* unsigned */
+ break;
+ case ldxd_op:
+ run->mmio.len = 8;
+ break;
+ default:
+ ret = EMULATE_FAIL;
+ break;
+ }
+ } else
+ ret = EMULATE_FAIL;
+
+ if (ret == EMULATE_DO_MMIO) {
+ /* Set for _kvm_complete_mmio_read use */
+ vcpu->arch.io_gpr = rd;
+ run->mmio.is_write = 0;
+ vcpu->mmio_is_write = 0;
+ } else {
+ kvm_err("Load not supporded inst=0x%08x @%lx BadVaddr:%#lx\n",
+ inst.word, vcpu->arch.pc, vcpu->arch.badv);
+ kvm_arch_vcpu_dump_regs(vcpu);
+ vcpu->mmio_needed = 0;
+ }
+ return ret;
+}
+
+int _kvm_complete_mmio_read(struct kvm_vcpu *vcpu, struct kvm_run *run)
+{
+ unsigned long *gpr = &vcpu->arch.gprs[vcpu->arch.io_gpr];
+ enum emulation_result er = EMULATE_DONE;
+
+ /* update with new PC */
+ update_pc(&vcpu->arch);
+ switch (run->mmio.len) {
+ case 8:
+ *gpr = *(s64 *)run->mmio.data;
+ break;
+ case 4:
+ if (vcpu->mmio_needed == 2)
+ *gpr = *(int *)run->mmio.data;
+ else
+ *gpr = *(unsigned int *)run->mmio.data;
+ break;
+ case 2:
+ if (vcpu->mmio_needed == 2)
+ *gpr = *(short *) run->mmio.data;
+ else
+ *gpr = *(unsigned short *)run->mmio.data;
+
+ break;
+ case 1:
+ if (vcpu->mmio_needed == 2)
+ *gpr = *(char *) run->mmio.data;
+ else
+ *gpr = *(unsigned char *) run->mmio.data;
+ break;
+ default:
+ kvm_err("Bad MMIO length: %d,addr is 0x%lx",
+ run->mmio.len, vcpu->arch.badv);
+ er = EMULATE_FAIL;
+ break;
+ }
+
+ return er;
+}
+
static int _kvm_trap_handle_gspr(struct kvm_vcpu *vcpu)
{
enum emulation_result er = EMULATE_DONE;
@@ -323,3 +582,52 @@ static int _kvm_handle_gspr(struct kvm_vcpu *vcpu)
}
return ret;
}
+
+static int _kvm_handle_mmu_fault(struct kvm_vcpu *vcpu, bool write)
+{
+ struct kvm_run *run = vcpu->run;
+ unsigned long badv = vcpu->arch.badv;
+ larch_inst inst;
+ enum emulation_result er = EMULATE_DONE;
+ int ret;
+
+ ret = kvm_handle_mm_fault(vcpu, badv, write);
+ if (ret) {
+ /* Treat as MMIO */
+ inst.word = vcpu->arch.badi;
+ if (write) {
+ er = _kvm_emu_mmio_write(vcpu, inst);
+ } else {
+ /* A code fetch fault doesn't count as an MMIO */
+ if (kvm_is_ifetch_fault(&vcpu->arch)) {
+ kvm_err("%s ifetch error addr:%lx\n", __func__, badv);
+ run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
+ return RESUME_HOST;
+ }
+
+ er = _kvm_emu_mmio_read(vcpu, inst);
+ }
+ }
+
+ if (er == EMULATE_DONE) {
+ ret = RESUME_GUEST;
+ } else if (er == EMULATE_DO_MMIO) {
+ run->exit_reason = KVM_EXIT_MMIO;
+ ret = RESUME_HOST;
+ } else {
+ run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
+ ret = RESUME_HOST;
+ }
+
+ return ret;
+}
+
+static int _kvm_handle_write_fault(struct kvm_vcpu *vcpu)
+{
+ return _kvm_handle_mmu_fault(vcpu, true);
+}
+
+static int _kvm_handle_read_fault(struct kvm_vcpu *vcpu)
+{
+ return _kvm_handle_mmu_fault(vcpu, false);
+}
--
2.39.1
Implement kvm handle LoongArch vcpu exit caused by reading and
writing csr. Using csr structure to emulate the registers.
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/exit.c | 98 +++++++++++++++++++++++++++++++++++++++
1 file changed, 98 insertions(+)
create mode 100644 arch/loongarch/kvm/exit.c
diff --git a/arch/loongarch/kvm/exit.c b/arch/loongarch/kvm/exit.c
new file mode 100644
index 000000000000..18635333fc9a
--- /dev/null
+++ b/arch/loongarch/kvm/exit.c
@@ -0,0 +1,98 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#include <linux/errno.h>
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/preempt.h>
+#include <linux/vmalloc.h>
+#include <asm/fpu.h>
+#include <asm/inst.h>
+#include <asm/time.h>
+#include <asm/tlb.h>
+#include <asm/loongarch.h>
+#include <asm/numa.h>
+#include <asm/kvm_vcpu.h>
+#include <asm/kvm_csr.h>
+#include <linux/kvm_host.h>
+#include <asm/mmzone.h>
+#include "trace.h"
+
+static unsigned long _kvm_emu_read_csr(struct kvm_vcpu *vcpu, int csrid)
+{
+ struct loongarch_csrs *csr = vcpu->arch.csr;
+ unsigned long val = 0;
+
+ if (get_gcsr_flag(csrid) & SW_GCSR)
+ val = kvm_read_sw_gcsr(csr, csrid);
+ else
+ pr_warn_once("Unsupport csrread 0x%x with pc %lx\n",
+ csrid, vcpu->arch.pc);
+ return val;
+}
+
+static void _kvm_emu_write_csr(struct kvm_vcpu *vcpu, int csrid,
+ unsigned long val)
+{
+ struct loongarch_csrs *csr = vcpu->arch.csr;
+
+ if (get_gcsr_flag(csrid) & SW_GCSR)
+ kvm_write_sw_gcsr(csr, csrid, val);
+ else
+ pr_warn_once("Unsupport csrwrite 0x%x with pc %lx\n",
+ csrid, vcpu->arch.pc);
+}
+
+static void _kvm_emu_xchg_csr(struct kvm_vcpu *vcpu, int csrid,
+ unsigned long csr_mask, unsigned long val)
+{
+ struct loongarch_csrs *csr = vcpu->arch.csr;
+
+ if (get_gcsr_flag(csrid) & SW_GCSR) {
+ unsigned long orig;
+
+ orig = kvm_read_sw_gcsr(csr, csrid);
+ orig &= ~csr_mask;
+ orig |= val & csr_mask;
+ kvm_write_sw_gcsr(csr, csrid, orig);
+ } else
+ pr_warn_once("Unsupport csrxchg 0x%x with pc %lx\n",
+ csrid, vcpu->arch.pc);
+}
+
+static int _kvm_handle_csr(struct kvm_vcpu *vcpu, larch_inst inst)
+{
+ unsigned int rd, rj, csrid;
+ unsigned long csr_mask;
+ unsigned long val = 0;
+
+ /*
+ * CSR value mask imm
+ * rj = 0 means csrrd
+ * rj = 1 means csrwr
+ * rj != 0,1 means csrxchg
+ */
+ rd = inst.reg2csr_format.rd;
+ rj = inst.reg2csr_format.rj;
+ csrid = inst.reg2csr_format.csr;
+
+ /* Process CSR ops */
+ if (rj == 0) {
+ /* process csrrd */
+ val = _kvm_emu_read_csr(vcpu, csrid);
+ vcpu->arch.gprs[rd] = val;
+ } else if (rj == 1) {
+ /* process csrwr */
+ val = vcpu->arch.gprs[rd];
+ _kvm_emu_write_csr(vcpu, csrid, val);
+ } else {
+ /* process csrxchg */
+ val = vcpu->arch.gprs[rd];
+ csr_mask = vcpu->arch.gprs[rj];
+ _kvm_emu_xchg_csr(vcpu, csrid, csr_mask, val);
+ }
+
+ return EMULATE_DONE;
+}
--
2.39.1
Enable LoongArch kvm config and add the makefile to support build kvm
module.
Reviewed-by: Bibo Mao <[email protected]>
Reported-by: kernel test robot <[email protected]>
Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/Kbuild | 1 +
arch/loongarch/Kconfig | 2 ++
arch/loongarch/configs/loongson3_defconfig | 2 ++
arch/loongarch/kvm/Kconfig | 38 ++++++++++++++++++++++
arch/loongarch/kvm/Makefile | 22 +++++++++++++
5 files changed, 65 insertions(+)
create mode 100644 arch/loongarch/kvm/Kconfig
create mode 100644 arch/loongarch/kvm/Makefile
diff --git a/arch/loongarch/Kbuild b/arch/loongarch/Kbuild
index b01f5cdb27e0..40be8a1696f9 100644
--- a/arch/loongarch/Kbuild
+++ b/arch/loongarch/Kbuild
@@ -2,6 +2,7 @@ obj-y += kernel/
obj-y += mm/
obj-y += net/
obj-y += vdso/
+obj-y += kvm/
# for cleaning
subdir- += boot
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index d38b066fc931..16ddf4f6f9bd 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -113,6 +113,7 @@ config LOONGARCH
select HAVE_KPROBES
select HAVE_KPROBES_ON_FTRACE
select HAVE_KRETPROBES
+ select HAVE_KVM
select HAVE_MOD_ARCH_SPECIFIC
select HAVE_NMI
select HAVE_PCI
@@ -604,3 +605,4 @@ source "drivers/acpi/Kconfig"
endmenu
source "drivers/firmware/Kconfig"
+source "arch/loongarch/kvm/Kconfig"
diff --git a/arch/loongarch/configs/loongson3_defconfig b/arch/loongarch/configs/loongson3_defconfig
index 6cd26dd3c134..3c0cbe6ad08f 100644
--- a/arch/loongarch/configs/loongson3_defconfig
+++ b/arch/loongarch/configs/loongson3_defconfig
@@ -63,6 +63,8 @@ CONFIG_EFI_ZBOOT=y
CONFIG_EFI_GENERIC_STUB_INITRD_CMDLINE_LOADER=y
CONFIG_EFI_CAPSULE_LOADER=m
CONFIG_EFI_TEST=m
+CONFIG_VIRTUALIZATION=y
+CONFIG_KVM=m
CONFIG_MODULES=y
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
diff --git a/arch/loongarch/kvm/Kconfig b/arch/loongarch/kvm/Kconfig
new file mode 100644
index 000000000000..8a999b4c0232
--- /dev/null
+++ b/arch/loongarch/kvm/Kconfig
@@ -0,0 +1,38 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# KVM configuration
+#
+
+source "virt/kvm/Kconfig"
+
+menuconfig VIRTUALIZATION
+ bool "Virtualization"
+ help
+ Say Y here to get to see options for using your Linux host to run
+ other operating systems inside virtual machines (guests).
+ This option alone does not add any kernel code.
+
+ If you say N, all options in this submenu will be skipped and
+ disabled.
+
+if VIRTUALIZATION
+
+config KVM
+ tristate "Kernel-based Virtual Machine (KVM) support"
+ depends on HAVE_KVM
+ select MMU_NOTIFIER
+ select ANON_INODES
+ select PREEMPT_NOTIFIERS
+ select KVM_MMIO
+ select KVM_GENERIC_DIRTYLOG_READ_PROTECT
+ select HAVE_KVM_VCPU_ASYNC_IOCTL
+ select HAVE_KVM_EVENTFD
+ select SRCU
+ help
+ Support hosting virtualized guest machines using hardware
+ virtualization extensions. You will need a fairly processor
+ equipped with virtualization extensions.
+
+ If unsure, say N.
+
+endif # VIRTUALIZATION
diff --git a/arch/loongarch/kvm/Makefile b/arch/loongarch/kvm/Makefile
new file mode 100644
index 000000000000..2335e873a6ef
--- /dev/null
+++ b/arch/loongarch/kvm/Makefile
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for LOONGARCH KVM support
+#
+
+ccflags-y += -I $(srctree)/$(src)
+
+include $(srctree)/virt/kvm/Makefile.kvm
+
+obj-$(CONFIG_KVM) += kvm.o
+
+kvm-y += main.o
+kvm-y += vm.o
+kvm-y += vmid.o
+kvm-y += tlb.o
+kvm-y += mmu.o
+kvm-y += vcpu.o
+kvm-y += exit.o
+kvm-y += interrupt.o
+kvm-y += timer.o
+kvm-y += switch.o
+kvm-y += csr_ops.o
--
2.39.1
Add maintainers for LoongArch KVM.
Signed-off-by: Tianrui Zhao <[email protected]>
---
MAINTAINERS | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 27ef11624748..c2fbfd6ad4e5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -11357,6 +11357,18 @@ F: include/kvm/arm_*
F: tools/testing/selftests/kvm/*/aarch64/
F: tools/testing/selftests/kvm/aarch64/
+KERNEL VIRTUAL MACHINE FOR LOONGARCH (KVM/LoongArch)
+M: Tianrui Zhao <[email protected]>
+M: Bibo Mao <[email protected]>
+M: Huacai Chen <[email protected]>
+L: [email protected]
+L: [email protected]
+S: Maintained
+T: git https://github.com/loongson/linux-loongarch-kvm
+F: arch/loongarch/include/asm/kvm*
+F: arch/loongarch/include/uapi/asm/kvm*
+F: arch/loongarch/kvm/
+
KERNEL VIRTUAL MACHINE FOR MIPS (KVM/mips)
M: Huacai Chen <[email protected]>
M: Aleksandar Markovic <[email protected]>
--
2.39.1
Implement kvm handle LoongArch vcpu idle exception, using kvm_vcpu_block
to emulate it.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/exit.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/arch/loongarch/kvm/exit.c b/arch/loongarch/kvm/exit.c
index 32edd915ebcb..30748238c78c 100644
--- a/arch/loongarch/kvm/exit.c
+++ b/arch/loongarch/kvm/exit.c
@@ -188,3 +188,23 @@ int _kvm_complete_iocsr_read(struct kvm_vcpu *vcpu, struct kvm_run *run)
return er;
}
+
+int _kvm_emu_idle(struct kvm_vcpu *vcpu)
+{
+ ++vcpu->stat.idle_exits;
+ trace_kvm_exit_idle(vcpu, KVM_TRACE_EXIT_IDLE);
+
+ if (!kvm_arch_vcpu_runnable(vcpu)) {
+ /*
+ * Switch to the software timer before halt-polling/blocking as
+ * the guest's timer may be a break event for the vCPU, and the
+ * hypervisor timer runs only when the CPU is in guest mode.
+ * Switch before halt-polling so that KVM recognizes an expired
+ * timer before blocking.
+ */
+ kvm_save_timer(vcpu);
+ kvm_vcpu_block(vcpu);
+ }
+
+ return EMULATE_DONE;
+}
--
2.39.1
Implement some misc vcpu relaterd interfaces, such as vcpu runnable,
vcpu should kick, vcpu dump regs, etc.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/vcpu.c | 108 ++++++++++++++++++++++++++++++++++++++
1 file changed, 108 insertions(+)
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index be4a6a2aeb44..26585e1661b7 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -12,6 +12,114 @@
#define CREATE_TRACE_POINTS
#include "trace.h"
+int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
+{
+ return !!(vcpu->arch.irq_pending) &&
+ vcpu->arch.mp_state.mp_state == KVM_MP_STATE_RUNNABLE;
+}
+
+int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
+{
+ return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE;
+}
+
+bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu)
+{
+ return false;
+}
+
+vm_fault_t kvm_arch_vcpu_fault(struct kvm_vcpu *vcpu, struct vm_fault *vmf)
+{
+ return VM_FAULT_SIGBUS;
+}
+
+int kvm_arch_vcpu_ioctl_translate(struct kvm_vcpu *vcpu,
+ struct kvm_translation *tr)
+{
+ return -EINVAL;
+}
+
+int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
+{
+ return _kvm_pending_timer(vcpu) ||
+ kvm_read_hw_gcsr(LOONGARCH_CSR_ESTAT) &
+ (1 << INT_TI);
+}
+
+int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu)
+{
+ int i;
+
+ if (!vcpu)
+ return -1;
+
+ kvm_debug("vCPU Register Dump:\n");
+ kvm_debug("\tpc = 0x%08lx\n", vcpu->arch.pc);
+ kvm_debug("\texceptions: %08lx\n", vcpu->arch.irq_pending);
+
+ for (i = 0; i < 32; i += 4) {
+ kvm_debug("\tgpr%02d: %08lx %08lx %08lx %08lx\n", i,
+ vcpu->arch.gprs[i],
+ vcpu->arch.gprs[i + 1],
+ vcpu->arch.gprs[i + 2], vcpu->arch.gprs[i + 3]);
+ }
+
+ kvm_debug("\tCRMOD: 0x%08lx, exst: 0x%08lx\n",
+ kvm_read_hw_gcsr(LOONGARCH_CSR_CRMD),
+ kvm_read_hw_gcsr(LOONGARCH_CSR_ESTAT));
+
+ kvm_debug("\tERA: 0x%08lx\n", kvm_read_hw_gcsr(LOONGARCH_CSR_ERA));
+
+ return 0;
+}
+
+int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu,
+ struct kvm_mp_state *mp_state)
+{
+ *mp_state = vcpu->arch.mp_state;
+
+ return 0;
+}
+
+int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
+ struct kvm_mp_state *mp_state)
+{
+ int ret = 0;
+
+ switch (mp_state->mp_state) {
+ case KVM_MP_STATE_RUNNABLE:
+ vcpu->arch.mp_state = *mp_state;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
+ struct kvm_guest_debug *dbg)
+{
+ return -EINVAL;
+}
+
+/**
+ * kvm_migrate_count() - Migrate timer.
+ * @vcpu: Virtual CPU.
+ *
+ * Migrate hrtimer to the current CPU by cancelling and restarting it
+ * if it was running prior to being cancelled.
+ *
+ * Must be called when the vCPU is migrated to a different CPU to ensure that
+ * timer expiry during guest execution interrupts the guest and causes the
+ * interrupt to be delivered in a timely manner.
+ */
+static void kvm_migrate_count(struct kvm_vcpu *vcpu)
+{
+ if (hrtimer_cancel(&vcpu->arch.swtimer))
+ hrtimer_restart(&vcpu->arch.swtimer);
+}
+
int _kvm_getcsr(struct kvm_vcpu *vcpu, unsigned int id, u64 *v)
{
unsigned long val;
--
2.39.1
Implement handle fpu exception, using kvm_own_fpu to enable fpu for
guest.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/exit.c | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/arch/loongarch/kvm/exit.c b/arch/loongarch/kvm/exit.c
index 36a838607218..17e94ecec140 100644
--- a/arch/loongarch/kvm/exit.c
+++ b/arch/loongarch/kvm/exit.c
@@ -631,3 +631,29 @@ static int _kvm_handle_read_fault(struct kvm_vcpu *vcpu)
{
return _kvm_handle_mmu_fault(vcpu, false);
}
+
+/**
+ * _kvm_handle_fpu_disabled() - Guest used fpu however it is disabled at host
+ * @vcpu: Virtual CPU context.
+ *
+ * Handle when the guest attempts to use fpu which hasn't been allowed
+ * by the root context.
+ */
+static int _kvm_handle_fpu_disabled(struct kvm_vcpu *vcpu)
+{
+ struct kvm_run *run = vcpu->run;
+
+ /*
+ * If guest FPU not present, the FPU operation should have been
+ * treated as a reserved instruction!
+ * If FPU already in use, we shouldn't get this at all.
+ */
+ if (WARN_ON(vcpu->arch.aux_inuse & KVM_LARCH_FPU)) {
+ kvm_err("%s internal error\n", __func__);
+ run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
+ return RESUME_HOST;
+ }
+
+ kvm_own_fpu(vcpu);
+ return RESUME_GUEST;
+}
--
2.39.1
Add LoongArch KVM related header files, including kvm.h,
kvm_host.h, kvm_types.h. All of those are about LoongArch
virtualization features and kvm interfaces.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/include/asm/kvm_host.h | 253 +++++++++++++++++++++++++
arch/loongarch/include/asm/kvm_types.h | 11 ++
arch/loongarch/include/uapi/asm/kvm.h | 106 +++++++++++
include/uapi/linux/kvm.h | 9 +
4 files changed, 379 insertions(+)
create mode 100644 arch/loongarch/include/asm/kvm_host.h
create mode 100644 arch/loongarch/include/asm/kvm_types.h
create mode 100644 arch/loongarch/include/uapi/asm/kvm.h
diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h
new file mode 100644
index 000000000000..a8ff3ef9cd55
--- /dev/null
+++ b/arch/loongarch/include/asm/kvm_host.h
@@ -0,0 +1,253 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#ifndef __ASM_LOONGARCH_KVM_HOST_H__
+#define __ASM_LOONGARCH_KVM_HOST_H__
+
+#include <linux/cpumask.h>
+#include <linux/mutex.h>
+#include <linux/hrtimer.h>
+#include <linux/interrupt.h>
+#include <linux/types.h>
+#include <linux/kvm.h>
+#include <linux/kvm_types.h>
+#include <linux/threads.h>
+#include <linux/spinlock.h>
+
+#include <asm/inst.h>
+#include <asm/loongarch.h>
+
+/* Loongarch KVM register ids */
+#define LOONGARCH_CSR_32(_R, _S) \
+ (KVM_REG_LOONGARCH_CSR | KVM_REG_SIZE_U32 | (8 * (_R) + (_S)))
+
+#define LOONGARCH_CSR_64(_R, _S) \
+ (KVM_REG_LOONGARCH_CSR | KVM_REG_SIZE_U64 | (8 * (_R) + (_S)))
+
+#define KVM_IOC_CSRID(id) LOONGARCH_CSR_64(id, 0)
+#define KVM_GET_IOC_CSRIDX(id) ((id & KVM_CSR_IDX_MASK) >> 3)
+
+#define KVM_MAX_VCPUS 256
+/* memory slots that does not exposed to userspace */
+#define KVM_PRIVATE_MEM_SLOTS 0
+
+#define KVM_HALT_POLL_NS_DEFAULT 500000
+
+struct kvm_vm_stat {
+ struct kvm_vm_stat_generic generic;
+};
+
+struct kvm_vcpu_stat {
+ struct kvm_vcpu_stat_generic generic;
+ u64 idle_exits;
+ u64 signal_exits;
+ u64 int_exits;
+ u64 cpucfg_exits;
+};
+
+struct kvm_arch_memory_slot {
+};
+
+struct kvm_context {
+ unsigned long vpid_cache;
+ struct kvm_vcpu *last_vcpu;
+};
+
+struct kvm_world_switch {
+ int (*guest_eentry)(void);
+ int (*enter_guest)(struct kvm_run *run, struct kvm_vcpu *vcpu);
+ unsigned long page_order;
+};
+
+struct kvm_arch {
+ /* Guest physical mm */
+ struct mm_struct gpa_mm;
+ /* Mask of CPUs needing GPA ASID flush */
+ cpumask_t asid_flush_mask;
+
+ unsigned char online_vcpus;
+ unsigned char is_migrate;
+ s64 time_offset;
+ struct kvm_context __percpu *vmcs;
+ unsigned long gpa_size;
+};
+
+#define CSR_MAX_NUMS 0x800
+
+struct loongarch_csrs {
+ unsigned long csrs[CSR_MAX_NUMS];
+};
+
+/* Resume Flags */
+#define RESUME_GUEST 1
+#define RESUME_HOST 0
+
+enum emulation_result {
+ EMULATE_DONE, /* no further processing */
+ EMULATE_DO_MMIO, /* kvm_run filled with MMIO request */
+ EMULATE_FAIL, /* can't emulate this instruction */
+ EMULATE_WAIT, /* WAIT instruction */
+ EMULATE_EXCEPT, /* A guest exception has been generated */
+ EMULATE_DO_IOCSR, /* handle IOCSR request */
+};
+
+#define KVM_LARCH_FPU (0x1 << 0)
+#define KVM_LARCH_CSR (0x1 << 1)
+
+struct kvm_vcpu_arch {
+ /*
+ * Switch pointer-to-function type to unsigned long
+ * for loading the value into register directly.
+ */
+ unsigned long guest_eentry;
+ unsigned long host_eentry;
+
+ /* Pointers stored here for easy accessing from assembly code */
+ int (*handle_exit)(struct kvm_run *run, struct kvm_vcpu *vcpu);
+
+ /* Host registers preserved across guest mode execution */
+ unsigned long host_sp;
+ unsigned long host_tp;
+ unsigned long host_pgd;
+
+ /* Host CSRs are used when handling exits from guest */
+ unsigned long badi;
+ unsigned long badv;
+ unsigned long host_ecfg;
+ unsigned long host_estat;
+ unsigned long host_percpu;
+
+ /* GPRs */
+ unsigned long gprs[32];
+ unsigned long pc;
+
+ /* FPU state */
+ struct loongarch_fpu fpu FPU_ALIGN;
+ /* Which auxiliary state is loaded (KVM_LOONGARCH_AUX_*) */
+ unsigned int aux_inuse;
+
+ /* CSR state */
+ struct loongarch_csrs *csr;
+
+ /* GPR used as IO source/target */
+ u32 io_gpr;
+
+ struct hrtimer swtimer;
+ /* KVM register to control count timer */
+ u32 count_ctl;
+
+ /* Bitmask of exceptions that are pending */
+ unsigned long irq_pending;
+ /* Bitmask of pending exceptions to be cleared */
+ unsigned long irq_clear;
+
+ /* Cache for pages needed inside spinlock regions */
+ struct kvm_mmu_memory_cache mmu_page_cache;
+
+ /* vcpu's vpid */
+ u64 vpid;
+
+ /* Period of stable timer tick in ns */
+ u64 timer_period_ns;
+ /* Frequency of stable timer in Hz */
+ u64 timer_mhz;
+ /* Stable bias from the raw time */
+ u64 timer_bias;
+ /* Dynamic nanosecond bias (multiple of timer_period_ns) to avoid overflow */
+ s64 timer_dyn_bias;
+
+ ktime_t stable_ktime_saved;
+
+ u64 core_ext_ioisr[4];
+
+ /* Last CPU the vCPU state was loaded on */
+ int last_sched_cpu;
+ /* Last CPU the vCPU actually executed guest code on */
+ int last_exec_cpu;
+ /* mp state */
+ struct kvm_mp_state mp_state;
+};
+
+static inline unsigned long readl_sw_gcsr(struct loongarch_csrs *csr, int reg)
+{
+ return csr->csrs[reg];
+}
+
+static inline void writel_sw_gcsr(struct loongarch_csrs *csr, int reg,
+ unsigned long val)
+{
+ csr->csrs[reg] = val;
+}
+
+/* Helpers */
+static inline bool _kvm_guest_has_fpu(struct kvm_vcpu_arch *arch)
+{
+ return cpu_has_fpu;
+}
+
+void _kvm_init_fault(void);
+
+/* Debug: dump vcpu state */
+int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu);
+
+/* MMU handling */
+int kvm_handle_mm_fault(struct kvm_vcpu *vcpu, unsigned long badv, bool write);
+void kvm_flush_tlb_all(void);
+void _kvm_destroy_mm(struct kvm *kvm);
+pgd_t *kvm_pgd_alloc(void);
+
+#define KVM_ARCH_WANT_MMU_NOTIFIER
+int kvm_unmap_hva_range(struct kvm *kvm,
+ unsigned long start, unsigned long end, bool blockable);
+void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte);
+int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end);
+int kvm_test_age_hva(struct kvm *kvm, unsigned long hva);
+
+static inline void update_pc(struct kvm_vcpu_arch *arch)
+{
+ arch->pc += 4;
+}
+
+/**
+ * kvm_is_ifetch_fault() - Find whether a TLBL exception is due to ifetch fault.
+ * @vcpu: Virtual CPU.
+ *
+ * Returns: Whether the TLBL exception was likely due to an instruction
+ * fetch fault rather than a data load fault.
+ */
+static inline bool kvm_is_ifetch_fault(struct kvm_vcpu_arch *arch)
+{
+ return arch->pc == arch->badv;
+}
+
+/* Misc */
+static inline void kvm_arch_hardware_unsetup(void) {}
+static inline void kvm_arch_sync_events(struct kvm *kvm) {}
+static inline void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen) {}
+static inline void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) {}
+static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) {}
+static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}
+static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
+static inline void kvm_arch_free_memslot(struct kvm *kvm,
+ struct kvm_memory_slot *slot) {}
+void _kvm_check_vmid(struct kvm_vcpu *vcpu);
+enum hrtimer_restart kvm_swtimer_wakeup(struct hrtimer *timer);
+int kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa);
+void kvm_arch_flush_remote_tlbs_memslot(struct kvm *kvm,
+ const struct kvm_memory_slot *memslot);
+void kvm_init_vmcs(struct kvm *kvm);
+void kvm_vector_entry(void);
+int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu);
+extern const unsigned long kvm_vector_size;
+extern const unsigned long kvm_enter_guest_size;
+extern unsigned long vpid_mask;
+extern struct kvm_world_switch *kvm_loongarch_ops;
+
+#define SW_GCSR (1 << 0)
+#define HW_GCSR (1 << 1)
+#define INVALID_GCSR (1 << 2)
+int get_gcsr_flag(int csr);
+extern void set_hw_gcsr(int csr_id, unsigned long val);
+#endif /* __ASM_LOONGARCH_KVM_HOST_H__ */
diff --git a/arch/loongarch/include/asm/kvm_types.h b/arch/loongarch/include/asm/kvm_types.h
new file mode 100644
index 000000000000..060647b5fe2e
--- /dev/null
+++ b/arch/loongarch/include/asm/kvm_types.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#ifndef _ASM_LOONGARCH_KVM_TYPES_H
+#define _ASM_LOONGARCH_KVM_TYPES_H
+
+#define KVM_ARCH_NR_OBJS_PER_MEMORY_CACHE 4
+
+#endif /* _ASM_LOONGARCH_KVM_TYPES_H */
diff --git a/arch/loongarch/include/uapi/asm/kvm.h b/arch/loongarch/include/uapi/asm/kvm.h
new file mode 100644
index 000000000000..3ccadb73ad8d
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/kvm.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
+ */
+
+#ifndef __UAPI_ASM_LOONGARCH_KVM_H
+#define __UAPI_ASM_LOONGARCH_KVM_H
+
+#include <linux/types.h>
+
+/*
+ * KVM Loongarch specific structures and definitions.
+ *
+ * Some parts derived from the x86 version of this file.
+ */
+
+#define __KVM_HAVE_READONLY_MEM
+
+#define KVM_COALESCED_MMIO_PAGE_OFFSET 1
+
+/*
+ * for KVM_GET_REGS and KVM_SET_REGS
+ */
+struct kvm_regs {
+ /* out (KVM_GET_REGS) / in (KVM_SET_REGS) */
+ __u64 gpr[32];
+ __u64 pc;
+};
+
+/*
+ * for KVM_GET_FPU and KVM_SET_FPU
+ */
+struct kvm_fpu {
+ __u32 fcsr;
+ __u64 fcc; /* 8x8 */
+ struct kvm_fpureg {
+ __u64 val64[4];
+ } fpr[32];
+};
+
+/*
+ * For LoongArch, we use KVM_SET_ONE_REG and KVM_GET_ONE_REG to access various
+ * registers. The id field is broken down as follows:
+ *
+ * bits[63..52] - As per linux/kvm.h
+ * bits[51..32] - Must be zero.
+ * bits[31..16] - Register set.
+ *
+ * Register set = 0: GP registers from kvm_regs (see definitions below).
+ *
+ * Register set = 1: CSR registers.
+ *
+ * Register set = 2: KVM specific registers (see definitions below).
+ *
+ * Register set = 3: FPU / SIMD registers (see definitions below).
+ *
+ * Other sets registers may be added in the future. Each set would
+ * have its own identifier in bits[31..16].
+ */
+
+#define KVM_REG_LOONGARCH_GP (KVM_REG_LOONGARCH | 0x00000ULL)
+#define KVM_REG_LOONGARCH_CSR (KVM_REG_LOONGARCH | 0x10000ULL)
+#define KVM_REG_LOONGARCH_KVM (KVM_REG_LOONGARCH | 0x20000ULL)
+#define KVM_REG_LOONGARCH_FPU (KVM_REG_LOONGARCH | 0x30000ULL)
+#define KVM_REG_LOONGARCH_MASK (KVM_REG_LOONGARCH | 0x30000ULL)
+#define KVM_CSR_IDX_MASK (0x10000 - 1)
+
+/*
+ * KVM_REG_LOONGARCH_KVM - KVM specific control registers.
+ */
+
+#define KVM_REG_LOONGARCH_COUNTER (KVM_REG_LOONGARCH_KVM | KVM_REG_SIZE_U64 | 3)
+#define KVM_REG_LOONGARCH_VCPU_RESET (KVM_REG_LOONGARCH_KVM | KVM_REG_SIZE_U64 | 4)
+
+struct kvm_debug_exit_arch {
+};
+
+/* for KVM_SET_GUEST_DEBUG */
+struct kvm_guest_debug_arch {
+};
+
+/* definition of registers in kvm_run */
+struct kvm_sync_regs {
+};
+
+/* dummy definition */
+struct kvm_sregs {
+};
+
+struct kvm_iocsr_entry {
+ __u32 addr;
+ __u32 pad;
+ __u64 data;
+};
+
+struct kvm_loongarch_interrupt {
+ /* in */
+ __u32 cpu;
+ __u32 irq;
+};
+
+#define KVM_NR_IRQCHIPS 1
+#define KVM_IRQCHIP_NUM_PINS 64
+#define KVM_MAX_CORES 256
+
+#endif /* __UAPI_ASM_LOONGARCH_KVM_H */
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 737318b1c1d9..74d9766277e7 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -264,6 +264,7 @@ struct kvm_xen_exit {
#define KVM_EXIT_RISCV_SBI 35
#define KVM_EXIT_RISCV_CSR 36
#define KVM_EXIT_NOTIFY 37
+#define KVM_EXIT_LOONGARCH_IOCSR 38
/* For KVM_EXIT_INTERNAL_ERROR */
/* Emulate instruction failed. */
@@ -336,6 +337,13 @@ struct kvm_run {
__u32 len;
__u8 is_write;
} mmio;
+ /* KVM_EXIT_LOONGARCH_IOCSR */
+ struct {
+ __u64 phys_addr;
+ __u8 data[8];
+ __u32 len;
+ __u8 is_write;
+ } iocsr_io;
/* KVM_EXIT_HYPERCALL */
struct {
__u64 nr;
@@ -1360,6 +1368,7 @@ struct kvm_dirty_tlb {
#define KVM_REG_ARM64 0x6000000000000000ULL
#define KVM_REG_MIPS 0x7000000000000000ULL
#define KVM_REG_RISCV 0x8000000000000000ULL
+#define KVM_REG_LOONGARCH 0x9000000000000000ULL
#define KVM_REG_SIZE_SHIFT 52
#define KVM_REG_SIZE_MASK 0x00f0000000000000ULL
--
2.39.1
Implement kvm exception vector, using _kvm_fault_tables array to save
the handle function pointer and it is used when vcpu handle exit.
Reviewed-by: Bibo Mao <[email protected]>
Signed-off-by: Tianrui Zhao <[email protected]>
---
arch/loongarch/kvm/exit.c | 48 +++++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/arch/loongarch/kvm/exit.c b/arch/loongarch/kvm/exit.c
index 17e94ecec140..4332dc9ce2d6 100644
--- a/arch/loongarch/kvm/exit.c
+++ b/arch/loongarch/kvm/exit.c
@@ -657,3 +657,51 @@ static int _kvm_handle_fpu_disabled(struct kvm_vcpu *vcpu)
kvm_own_fpu(vcpu);
return RESUME_GUEST;
}
+
+/*
+ * Loongarch KVM callback handling for not implemented guest exiting
+ */
+static int _kvm_fault_ni(struct kvm_vcpu *vcpu)
+{
+ unsigned long estat, badv;
+ unsigned int exccode, inst;
+
+ /*
+ * Fetch the instruction.
+ */
+ badv = vcpu->arch.badv;
+ estat = vcpu->arch.host_estat;
+ exccode = (estat & CSR_ESTAT_EXC) >> CSR_ESTAT_EXC_SHIFT;
+ inst = vcpu->arch.badi;
+ kvm_err("Exccode: %d PC=%#lx inst=0x%08x BadVaddr=%#lx estat=%#lx\n",
+ exccode, vcpu->arch.pc, inst, badv, read_gcsr_estat());
+ kvm_arch_vcpu_dump_regs(vcpu);
+ vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
+
+ return RESUME_HOST;
+}
+
+static exit_handle_fn _kvm_fault_tables[EXCCODE_INT_START] = {
+ [EXCCODE_TLBL] = _kvm_handle_read_fault,
+ [EXCCODE_TLBI] = _kvm_handle_read_fault,
+ [EXCCODE_TLBNR] = _kvm_handle_read_fault,
+ [EXCCODE_TLBNX] = _kvm_handle_read_fault,
+ [EXCCODE_TLBS] = _kvm_handle_write_fault,
+ [EXCCODE_TLBM] = _kvm_handle_write_fault,
+ [EXCCODE_FPDIS] = _kvm_handle_fpu_disabled,
+ [EXCCODE_GSPR] = _kvm_handle_gspr,
+};
+
+void _kvm_init_fault(void)
+{
+ int i;
+
+ for (i = 0; i < EXCCODE_INT_START; i++)
+ if (!_kvm_fault_tables[i])
+ _kvm_fault_tables[i] = _kvm_fault_ni;
+}
+
+int _kvm_handle_fault(struct kvm_vcpu *vcpu, int fault)
+{
+ return _kvm_fault_tables[fault](vcpu);
+}
--
2.39.1
在 2023/6/9 17:08, Tianrui Zhao 写道:
> Implement kvm hardware enable, disable interface, setting
> the guest config register to enable virtualization features
> when called the interface.
>
> Signed-off-by: Tianrui Zhao <[email protected]>
> ---
> arch/loongarch/kvm/main.c | 64 +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 64 insertions(+)
>
> diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
> index f98c1619725f..5ebae1ea7565 100644
> --- a/arch/loongarch/kvm/main.c
> +++ b/arch/loongarch/kvm/main.c
> @@ -195,6 +195,70 @@ static void _kvm_init_gcsr_flag(void)
> set_gcsr_sw_flag(LOONGARCH_CSR_PERFCNTR3);
> }
>
> +void kvm_init_vmcs(struct kvm *kvm)
> +{
> + kvm->arch.vmcs = vmcs;
> +}
> +
> +long kvm_arch_dev_ioctl(struct file *filp,
> + unsigned int ioctl, unsigned long arg)
> +{
> + return -ENOIOCTLCMD;
> +}
> +
> +#ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING
Can we remove macro CONFIG_KVM_GENERIC_HARDWARE_ENABLING here since it
is always enabled? The other looks good to me.
Reviewed-by: Bibo Mao <[email protected]>
Regards
Bibo, Mao
> +int kvm_arch_hardware_enable(void)
> +{
> + unsigned long env, gcfg = 0;
> +
> + env = read_csr_gcfg();
> + /* First init gtlbc, gcfg, gstat, gintc. All guest use the same config */
> + clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI);
> + write_csr_gcfg(0);
> + write_csr_gstat(0);
> + write_csr_gintc(0);
> +
> + /*
> + * Enable virtualization features granting guest direct control of
> + * certain features:
> + * GCI=2: Trap on init or unimplement cache instruction.
> + * TORU=0: Trap on Root Unimplement.
> + * CACTRL=1: Root control cache.
> + * TOP=0: Trap on Previlege.
> + * TOE=0: Trap on Exception.
> + * TIT=0: Trap on Timer.
> + */
> + if (env & CSR_GCFG_GCIP_ALL)
> + gcfg |= CSR_GCFG_GCI_SECURE;
> + if (env & CSR_GCFG_MATC_ROOT)
> + gcfg |= CSR_GCFG_MATC_ROOT;
> +
> + gcfg |= CSR_GCFG_TIT;
> + write_csr_gcfg(gcfg);
> +
> + kvm_flush_tlb_all();
> +
> + /* Enable using TGID */
> + set_csr_gtlbc(CSR_GTLBC_USETGID);
> + kvm_debug("gtlbc:%llx gintc:%llx gstat:%llx gcfg:%llx",
> + read_csr_gtlbc(), read_csr_gintc(),
> + read_csr_gstat(), read_csr_gcfg());
> +
> + return 0;
> +}
> +
> +void kvm_arch_hardware_disable(void)
> +{
> + clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI);
> + write_csr_gcfg(0);
> + write_csr_gstat(0);
> + write_csr_gintc(0);
> +
> + /* Flush any remaining guest TLB entries */
> + kvm_flush_tlb_all();
> +}
> +#endif
> +
> static int kvm_loongarch_env_init(void)
> {
> struct kvm_context *context;
Reviewed-by: Bibo Mao <[email protected]>
Regards
Bibo, Mao
在 2023/6/9 17:08, Tianrui Zhao 写道:
> Implement vcpu handle exit interface, getting the exit code by ESTAT
> register and using kvm exception vector to handle it.
>
> Signed-off-by: Tianrui Zhao <[email protected]>
> ---
> arch/loongarch/kvm/vcpu.c | 45 +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 45 insertions(+)
>
> diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
> index eba5c07b8be3..a45e9d9efe5b 100644
> --- a/arch/loongarch/kvm/vcpu.c
> +++ b/arch/loongarch/kvm/vcpu.c
> @@ -52,6 +52,51 @@ static void kvm_pre_enter_guest(struct kvm_vcpu *vcpu)
> vcpu->arch.aux_inuse &= ~KVM_LARCH_CSR;
> }
>
> +/*
> + * Return 1 for resume guest and "<= 0" for resume host.
> + */
> +static int _kvm_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu)
> +{
> + unsigned long exst = vcpu->arch.host_estat;
> + u32 intr = exst & 0x1fff; /* ignore NMI */
> + u32 exccode = (exst & CSR_ESTAT_EXC) >> CSR_ESTAT_EXC_SHIFT;
> + int ret = RESUME_GUEST;
> +
> + vcpu->mode = OUTSIDE_GUEST_MODE;
> +
> + /* Set a default exit reason */
> + run->exit_reason = KVM_EXIT_UNKNOWN;
> +
> + local_irq_enable();
> + guest_state_exit_irqoff();
> +
> + trace_kvm_exit(vcpu, exccode);
> + if (exccode) {
> + ret = _kvm_handle_fault(vcpu, exccode);
> + } else {
> + WARN(!intr, "vm exiting with suspicious irq\n");
> + ++vcpu->stat.int_exits;
> + }
> +
> + cond_resched();
> + local_irq_disable();
> +
> + if (ret == RESUME_HOST)
> + return ret;
> +
> + /* Only check for signals if not already exiting to userspace */
> + if (signal_pending(current)) {
> + vcpu->run->exit_reason = KVM_EXIT_INTR;
> + ++vcpu->stat.signal_exits;
> + return -EINTR;
> + }
> +
> + kvm_pre_enter_guest(vcpu);
> + trace_kvm_reenter(vcpu);
> + guest_state_enter_irqoff();
> + return RESUME_GUEST;
> +}
> +
> int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
> {
> unsigned long timer_hz;
Reviewed-by: Bibo Mao <[email protected]>
Regards
Bibo, Mao
在 2023/6/9 17:08, Tianrui Zhao 写道:
> Implement LoongArch vcpu KVM_ENABLE_CAP ioctl interface.
>
> Signed-off-by: Tianrui Zhao <[email protected]>
> ---
> arch/loongarch/kvm/vcpu.c | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
> index b0cce413762d..da97b77da8eb 100644
> --- a/arch/loongarch/kvm/vcpu.c
> +++ b/arch/loongarch/kvm/vcpu.c
> @@ -186,6 +186,16 @@ int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
> return 0;
> }
>
> +static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
> + struct kvm_enable_cap *cap)
> +{
> + /*
> + * FPU is enable by default, do not support any other caps,
> + * and later we will support such as LSX cap.
> + */
> + return -EINVAL;
> +}
> +
> long kvm_arch_vcpu_ioctl(struct file *filp,
> unsigned int ioctl, unsigned long arg)
> {
> @@ -209,6 +219,15 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
> r = _kvm_get_reg(vcpu, ®);
> break;
> }
> + case KVM_ENABLE_CAP: {
> + struct kvm_enable_cap cap;
> +
> + r = -EFAULT;
> + if (copy_from_user(&cap, argp, sizeof(cap)))
> + break;
> + r = kvm_vcpu_ioctl_enable_cap(vcpu, &cap);
> + break;
> + }
> default:
> r = -ENOIOCTLCMD;
> break;
Reviewed-by: Bibo Mao <[email protected]>
Regards
Bibo, Mao
在 2023/6/9 17:08, Tianrui Zhao 写道:
> Implement vcpu run interface, handling mmio, iocsr reading fault
> and deliver interrupt, lose fpu before vcpu enter guest.
>
> Signed-off-by: Tianrui Zhao <[email protected]>
> ---
> arch/loongarch/kvm/vcpu.c | 83 +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 83 insertions(+)
>
> diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
> index 24b5b00266a1..eba5c07b8be3 100644
> --- a/arch/loongarch/kvm/vcpu.c
> +++ b/arch/loongarch/kvm/vcpu.c
> @@ -17,6 +17,41 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
> return 0;
> }
>
> +/* Returns 1 if the guest TLB may be clobbered */
> +static int _kvm_check_requests(struct kvm_vcpu *vcpu)
> +{
> + int ret = 0;
> +
> + if (!kvm_request_pending(vcpu))
> + return 0;
> +
> + if (kvm_check_request(KVM_REQ_TLB_FLUSH, vcpu)) {
> + /* Drop vpid for this vCPU */
> + vcpu->arch.vpid = 0;
> + /* This will clobber guest TLB contents too */
> + ret = 1;
> + }
> +
> + return ret;
> +}
> +
> +static void kvm_pre_enter_guest(struct kvm_vcpu *vcpu)
> +{
> + /*
> + * handle vcpu timer, interrupts, check requests and
> + * check vmid before vcpu enter guest
> + */
> + kvm_acquire_timer(vcpu);
> + _kvm_deliver_intr(vcpu);
> + /* make sure the vcpu mode has been written */
> + smp_store_mb(vcpu->mode, IN_GUEST_MODE);
> + _kvm_check_requests(vcpu);
> + _kvm_check_vmid(vcpu);
> + vcpu->arch.host_eentry = csr_read64(LOONGARCH_CSR_EENTRY);
> + /* clear KVM_LARCH_CSR as csr will change when enter guest */
> + vcpu->arch.aux_inuse &= ~KVM_LARCH_CSR;
> +}
> +
> int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
> {
> unsigned long timer_hz;
> @@ -86,3 +121,51 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
> context->last_vcpu = NULL;
> }
> }
> +
> +int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
> +{
> + int r = -EINTR;
> + struct kvm_run *run = vcpu->run;
> +
> + vcpu_load(vcpu);
> +
> + kvm_sigset_activate(vcpu);
> +
> + if (vcpu->mmio_needed) {
> + if (!vcpu->mmio_is_write)
> + _kvm_complete_mmio_read(vcpu, run);
> + vcpu->mmio_needed = 0;
> + }
> +
> + if (run->exit_reason == KVM_EXIT_LOONGARCH_IOCSR) {
> + if (!run->iocsr_io.is_write)
> + _kvm_complete_iocsr_read(vcpu, run);
> + }
> +
> + /* clear exit_reason */
> + run->exit_reason = KVM_EXIT_UNKNOWN;
> + if (run->immediate_exit)
> + goto out;
> +
> + lose_fpu(1);
> +
> + local_irq_disable();
> + guest_timing_enter_irqoff();
> +
> + kvm_pre_enter_guest(vcpu);
> + trace_kvm_enter(vcpu);
> +
> + guest_state_enter_irqoff();
> + r = kvm_loongarch_ops->enter_guest(run, vcpu);
> +
> + /* guest_state_exit_irqoff() already done. */
> + trace_kvm_out(vcpu);
> + guest_timing_exit_irqoff();
> + local_irq_enable();
> +
> +out:
> + kvm_sigset_deactivate(vcpu);
> +
> + vcpu_put(vcpu);
> + return r;
> +}
Reviewed-by: Bibo Mao <[email protected]>
Regards
Bibo, Mao
在 2023/6/9 17:08, Tianrui Zhao 写道:
> Implement kvm handle LoongArch vcpu exit caused by reading and
> writing csr. Using csr structure to emulate the registers.
>
> Signed-off-by: Tianrui Zhao <[email protected]>
> ---
> arch/loongarch/kvm/exit.c | 98 +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 98 insertions(+)
> create mode 100644 arch/loongarch/kvm/exit.c
>
> diff --git a/arch/loongarch/kvm/exit.c b/arch/loongarch/kvm/exit.c
> new file mode 100644
> index 000000000000..18635333fc9a
> --- /dev/null
> +++ b/arch/loongarch/kvm/exit.c
> @@ -0,0 +1,98 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
> + */
> +
> +#include <linux/errno.h>
> +#include <linux/err.h>
> +#include <linux/module.h>
> +#include <linux/preempt.h>
> +#include <linux/vmalloc.h>
> +#include <asm/fpu.h>
> +#include <asm/inst.h>
> +#include <asm/time.h>
> +#include <asm/tlb.h>
> +#include <asm/loongarch.h>
> +#include <asm/numa.h>
> +#include <asm/kvm_vcpu.h>
> +#include <asm/kvm_csr.h>
> +#include <linux/kvm_host.h>
> +#include <asm/mmzone.h>
> +#include "trace.h"
> +
> +static unsigned long _kvm_emu_read_csr(struct kvm_vcpu *vcpu, int csrid)
> +{
> + struct loongarch_csrs *csr = vcpu->arch.csr;
> + unsigned long val = 0;
> +
> + if (get_gcsr_flag(csrid) & SW_GCSR)
> + val = kvm_read_sw_gcsr(csr, csrid);
> + else
> + pr_warn_once("Unsupport csrread 0x%x with pc %lx\n",
> + csrid, vcpu->arch.pc);
> + return val;
> +}
> +
> +static void _kvm_emu_write_csr(struct kvm_vcpu *vcpu, int csrid,
> + unsigned long val)
> +{
> + struct loongarch_csrs *csr = vcpu->arch.csr;
> +
> + if (get_gcsr_flag(csrid) & SW_GCSR)
> + kvm_write_sw_gcsr(csr, csrid, val);
> + else
> + pr_warn_once("Unsupport csrwrite 0x%x with pc %lx\n",
> + csrid, vcpu->arch.pc);
> +}
> +
> +static void _kvm_emu_xchg_csr(struct kvm_vcpu *vcpu, int csrid,
> + unsigned long csr_mask, unsigned long val)
> +{
> + struct loongarch_csrs *csr = vcpu->arch.csr;
> +
> + if (get_gcsr_flag(csrid) & SW_GCSR) {
> + unsigned long orig;
> +
> + orig = kvm_read_sw_gcsr(csr, csrid);
> + orig &= ~csr_mask;
> + orig |= val & csr_mask;
> + kvm_write_sw_gcsr(csr, csrid, orig);
> + } else
> + pr_warn_once("Unsupport csrxchg 0x%x with pc %lx\n",
> + csrid, vcpu->arch.pc);
> +}
> +
> +static int _kvm_handle_csr(struct kvm_vcpu *vcpu, larch_inst inst)
> +{
> + unsigned int rd, rj, csrid;
> + unsigned long csr_mask;
> + unsigned long val = 0;
> +
> + /*
> + * CSR value mask imm
> + * rj = 0 means csrrd
> + * rj = 1 means csrwr
> + * rj != 0,1 means csrxchg
> + */
> + rd = inst.reg2csr_format.rd;
> + rj = inst.reg2csr_format.rj;
> + csrid = inst.reg2csr_format.csr;
> +
> + /* Process CSR ops */
> + if (rj == 0) {
> + /* process csrrd */
> + val = _kvm_emu_read_csr(vcpu, csrid);
> + vcpu->arch.gprs[rd] = val;
> + } else if (rj == 1) {
> + /* process csrwr */
> + val = vcpu->arch.gprs[rd];
> + _kvm_emu_write_csr(vcpu, csrid, val);
> + } else {
> + /* process csrxchg */
> + val = vcpu->arch.gprs[rd];
> + csr_mask = vcpu->arch.gprs[rj];
> + _kvm_emu_xchg_csr(vcpu, csrid, csr_mask, val);
> + }
> +
> + return EMULATE_DONE;
> +}
Hi, Tianrui,
On Fri, Jun 9, 2023 at 5:09 PM Tianrui Zhao <[email protected]> wrote:
>
> Supplement kvm document about LoongArch-specific part, such as add
> api introduction for GET/SET_ONE_REG, GET/SET_FPU, GET/SET_MP_STATE,
> etc.
>
> Signed-off-by: Tianrui Zhao <[email protected]>
> ---
> Documentation/virt/kvm/api.rst | 71 +++++++++++++++++++++++++++++-----
> 1 file changed, 62 insertions(+), 9 deletions(-)
>
> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> index add067793b90..ad8e13eab48d 100644
> --- a/Documentation/virt/kvm/api.rst
> +++ b/Documentation/virt/kvm/api.rst
> @@ -416,6 +416,12 @@ Reads the general purpose registers from the vcpu.
> __u64 pc;
> };
>
> + /* LoongArch */
> + struct kvm_regs {
Add a " /* out (KVM_GET_REGS) / in (KVM_SET_REGS) */" line as others here?
> + unsigned long gpr[32];
> + unsigned long pc;
> + };
> +
>
> 4.12 KVM_SET_REGS
> -----------------
> @@ -506,7 +512,7 @@ translation mode.
> ------------------
>
> :Capability: basic
> -:Architectures: x86, ppc, mips, riscv
> +:Architectures: x86, ppc, mips, riscv, loongarch
> :Type: vcpu ioctl
> :Parameters: struct kvm_interrupt (in)
> :Returns: 0 on success, negative on failure.
> @@ -592,6 +598,14 @@ b) KVM_INTERRUPT_UNSET
>
> This is an asynchronous vcpu ioctl and can be invoked from any thread.
>
> +LOONGARCH:
> +^^^^^^^^^^
> +
> +Queues an external interrupt to be injected into the virtual CPU. A negative
> +interrupt number dequeues the interrupt.
> +
> +This is an asynchronous vcpu ioctl and can be invoked from any thread.
> +
>
> 4.17 KVM_DEBUG_GUEST
> --------------------
> @@ -737,7 +751,7 @@ signal mask.
> ----------------
>
> :Capability: basic
> -:Architectures: x86
> +:Architectures: x86, loongarch
> :Type: vcpu ioctl
> :Parameters: struct kvm_fpu (out)
> :Returns: 0 on success, -1 on error
> @@ -746,7 +760,7 @@ Reads the floating point state from the vcpu.
>
> ::
>
> - /* for KVM_GET_FPU and KVM_SET_FPU */
> + /* x86: for KVM_GET_FPU and KVM_SET_FPU */
> struct kvm_fpu {
> __u8 fpr[8][16];
> __u16 fcw;
> @@ -761,12 +775,22 @@ Reads the floating point state from the vcpu.
> __u32 pad2;
> };
>
> + /* LoongArch: for KVM_GET_FPU and KVM_SET_FPU */
> + struct kvm_fpu {
> + __u32 fcsr;
> + __u32 none;
Maybe use pad1 as x86 is better.
Huacai
> + __u64 fcc;
> + struct kvm_fpureg {
> + __u64 val64[4];
> + }fpr[32];
> + };
> +
>
> 4.23 KVM_SET_FPU
> ----------------
>
> :Capability: basic
> -:Architectures: x86
> +:Architectures: x86, loongarch
> :Type: vcpu ioctl
> :Parameters: struct kvm_fpu (in)
> :Returns: 0 on success, -1 on error
> @@ -775,7 +799,7 @@ Writes the floating point state to the vcpu.
>
> ::
>
> - /* for KVM_GET_FPU and KVM_SET_FPU */
> + /* x86: for KVM_GET_FPU and KVM_SET_FPU */
> struct kvm_fpu {
> __u8 fpr[8][16];
> __u16 fcw;
> @@ -790,6 +814,16 @@ Writes the floating point state to the vcpu.
> __u32 pad2;
> };
>
> + /* LoongArch: for KVM_GET_FPU and KVM_SET_FPU */
> + struct kvm_fpu {
> + __u32 fcsr;
> + __u32 none;
> + __u64 fcc;
> + struct kvm_fpureg {
> + __u64 val64[4];
> + }fpr[32];
> + };
> +
>
> 4.24 KVM_CREATE_IRQCHIP
> -----------------------
> @@ -1387,7 +1421,7 @@ documentation when it pops into existence).
> -------------------
>
> :Capability: KVM_CAP_ENABLE_CAP
> -:Architectures: mips, ppc, s390, x86
> +:Architectures: mips, ppc, s390, x86, loongarch
> :Type: vcpu ioctl
> :Parameters: struct kvm_enable_cap (in)
> :Returns: 0 on success; -1 on error
> @@ -1442,7 +1476,7 @@ for vm-wide capabilities.
> ---------------------
>
> :Capability: KVM_CAP_MP_STATE
> -:Architectures: x86, s390, arm64, riscv
> +:Architectures: x86, s390, arm64, riscv, loongarch
> :Type: vcpu ioctl
> :Parameters: struct kvm_mp_state (out)
> :Returns: 0 on success; -1 on error
> @@ -1460,7 +1494,7 @@ Possible values are:
>
> ========================== ===============================================
> KVM_MP_STATE_RUNNABLE the vcpu is currently running
> - [x86,arm64,riscv]
> + [x86,arm64,riscv,loongarch]
> KVM_MP_STATE_UNINITIALIZED the vcpu is an application processor (AP)
> which has not yet received an INIT signal [x86]
> KVM_MP_STATE_INIT_RECEIVED the vcpu has received an INIT signal, and is
> @@ -1516,11 +1550,14 @@ For riscv:
> The only states that are valid are KVM_MP_STATE_STOPPED and
> KVM_MP_STATE_RUNNABLE which reflect if the vcpu is paused or not.
>
> +On LoongArch, only the KVM_MP_STATE_RUNNABLE state is used to reflect
> +whether the vcpu is runnable.
> +
> 4.39 KVM_SET_MP_STATE
> ---------------------
>
> :Capability: KVM_CAP_MP_STATE
> -:Architectures: x86, s390, arm64, riscv
> +:Architectures: x86, s390, arm64, riscv, loongarch
> :Type: vcpu ioctl
> :Parameters: struct kvm_mp_state (in)
> :Returns: 0 on success; -1 on error
> @@ -1538,6 +1575,9 @@ For arm64/riscv:
> The only states that are valid are KVM_MP_STATE_STOPPED and
> KVM_MP_STATE_RUNNABLE which reflect if the vcpu should be paused or not.
>
> +On LoongArch, only the KVM_MP_STATE_RUNNABLE state is used to reflect
> +whether the vcpu is runnable.
> +
> 4.40 KVM_SET_IDENTITY_MAP_ADDR
> ------------------------------
>
> @@ -2839,6 +2879,19 @@ Following are the RISC-V D-extension registers:
> 0x8020 0000 0600 0020 fcsr Floating point control and status register
> ======================= ========= =============================================
>
> +LoongArch registers are mapped using the lower 32 bits. The upper 16 bits of
> +that is the register group type.
> +
> +LoongArch csr registers are used to control guest cpu or get status of guest
> +cpu, and they have the following id bit patterns::
> +
> + 0x9030 0000 0001 00 <reg:5> <sel:3> (64-bit)
> +
> +LoongArch KVM control registers are used to implement some new defined functions
> +such as set vcpu counter or reset vcpu, and they have the following id bit patterns::
> +
> + 0x9030 0000 0002 <reg:16>
> +
>
> 4.69 KVM_GET_ONE_REG
> --------------------
> --
> 2.39.1
>
>
在 2023/6/15 下午5:42, Huacai Chen 写道:
> Hi, Tianrui,
>
> On Fri, Jun 9, 2023 at 5:09 PM Tianrui Zhao <[email protected]> wrote:
>> Supplement kvm document about LoongArch-specific part, such as add
>> api introduction for GET/SET_ONE_REG, GET/SET_FPU, GET/SET_MP_STATE,
>> etc.
>>
>> Signed-off-by: Tianrui Zhao <[email protected]>
>> ---
>> Documentation/virt/kvm/api.rst | 71 +++++++++++++++++++++++++++++-----
>> 1 file changed, 62 insertions(+), 9 deletions(-)
>>
>> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
>> index add067793b90..ad8e13eab48d 100644
>> --- a/Documentation/virt/kvm/api.rst
>> +++ b/Documentation/virt/kvm/api.rst
>> @@ -416,6 +416,12 @@ Reads the general purpose registers from the vcpu.
>> __u64 pc;
>> };
>>
>> + /* LoongArch */
>> + struct kvm_regs {
> Add a " /* out (KVM_GET_REGS) / in (KVM_SET_REGS) */" line as others here?
Thanks, I will add this comment here.
Tianrui Zhao
>> + unsigned long gpr[32];
>> + unsigned long pc;
>> + };
>> +
>>
>> 4.12 KVM_SET_REGS
>> -----------------
>> @@ -506,7 +512,7 @@ translation mode.
>> ------------------
>>
>> :Capability: basic
>> -:Architectures: x86, ppc, mips, riscv
>> +:Architectures: x86, ppc, mips, riscv, loongarch
>> :Type: vcpu ioctl
>> :Parameters: struct kvm_interrupt (in)
>> :Returns: 0 on success, negative on failure.
>> @@ -592,6 +598,14 @@ b) KVM_INTERRUPT_UNSET
>>
>> This is an asynchronous vcpu ioctl and can be invoked from any thread.
>>
>> +LOONGARCH:
>> +^^^^^^^^^^
>> +
>> +Queues an external interrupt to be injected into the virtual CPU. A negative
>> +interrupt number dequeues the interrupt.
>> +
>> +This is an asynchronous vcpu ioctl and can be invoked from any thread.
>> +
>>
>> 4.17 KVM_DEBUG_GUEST
>> --------------------
>> @@ -737,7 +751,7 @@ signal mask.
>> ----------------
>>
>> :Capability: basic
>> -:Architectures: x86
>> +:Architectures: x86, loongarch
>> :Type: vcpu ioctl
>> :Parameters: struct kvm_fpu (out)
>> :Returns: 0 on success, -1 on error
>> @@ -746,7 +760,7 @@ Reads the floating point state from the vcpu.
>>
>> ::
>>
>> - /* for KVM_GET_FPU and KVM_SET_FPU */
>> + /* x86: for KVM_GET_FPU and KVM_SET_FPU */
>> struct kvm_fpu {
>> __u8 fpr[8][16];
>> __u16 fcw;
>> @@ -761,12 +775,22 @@ Reads the floating point state from the vcpu.
>> __u32 pad2;
>> };
>>
>> + /* LoongArch: for KVM_GET_FPU and KVM_SET_FPU */
>> + struct kvm_fpu {
>> + __u32 fcsr;
>> + __u32 none;
> Maybe use pad1 as x86 is better.
>
> Huacai
Thanks, this 'none' variable is meaningless, and I will remove it. As
this have already been removed in the actually kvm_fpu structure.
Tianrui Zhao
>> + __u64 fcc;
>> + struct kvm_fpureg {
>> + __u64 val64[4];
>> + }fpr[32];
>> + };
>> +
>>
>> 4.23 KVM_SET_FPU
>> ----------------
>>
>> :Capability: basic
>> -:Architectures: x86
>> +:Architectures: x86, loongarch
>> :Type: vcpu ioctl
>> :Parameters: struct kvm_fpu (in)
>> :Returns: 0 on success, -1 on error
>> @@ -775,7 +799,7 @@ Writes the floating point state to the vcpu.
>>
>> ::
>>
>> - /* for KVM_GET_FPU and KVM_SET_FPU */
>> + /* x86: for KVM_GET_FPU and KVM_SET_FPU */
>> struct kvm_fpu {
>> __u8 fpr[8][16];
>> __u16 fcw;
>> @@ -790,6 +814,16 @@ Writes the floating point state to the vcpu.
>> __u32 pad2;
>> };
>>
>> + /* LoongArch: for KVM_GET_FPU and KVM_SET_FPU */
>> + struct kvm_fpu {
>> + __u32 fcsr;
>> + __u32 none;
I will also remove this 'none' variable.
Tianrui Zhao
>> + __u64 fcc;
>> + struct kvm_fpureg {
>> + __u64 val64[4];
>> + }fpr[32];
>> + };
>> +
>>
>> 4.24 KVM_CREATE_IRQCHIP
>> -----------------------
>> @@ -1387,7 +1421,7 @@ documentation when it pops into existence).
>> -------------------
>>
>> :Capability: KVM_CAP_ENABLE_CAP
>> -:Architectures: mips, ppc, s390, x86
>> +:Architectures: mips, ppc, s390, x86, loongarch
>> :Type: vcpu ioctl
>> :Parameters: struct kvm_enable_cap (in)
>> :Returns: 0 on success; -1 on error
>> @@ -1442,7 +1476,7 @@ for vm-wide capabilities.
>> ---------------------
>>
>> :Capability: KVM_CAP_MP_STATE
>> -:Architectures: x86, s390, arm64, riscv
>> +:Architectures: x86, s390, arm64, riscv, loongarch
>> :Type: vcpu ioctl
>> :Parameters: struct kvm_mp_state (out)
>> :Returns: 0 on success; -1 on error
>> @@ -1460,7 +1494,7 @@ Possible values are:
>>
>> ========================== ===============================================
>> KVM_MP_STATE_RUNNABLE the vcpu is currently running
>> - [x86,arm64,riscv]
>> + [x86,arm64,riscv,loongarch]
>> KVM_MP_STATE_UNINITIALIZED the vcpu is an application processor (AP)
>> which has not yet received an INIT signal [x86]
>> KVM_MP_STATE_INIT_RECEIVED the vcpu has received an INIT signal, and is
>> @@ -1516,11 +1550,14 @@ For riscv:
>> The only states that are valid are KVM_MP_STATE_STOPPED and
>> KVM_MP_STATE_RUNNABLE which reflect if the vcpu is paused or not.
>>
>> +On LoongArch, only the KVM_MP_STATE_RUNNABLE state is used to reflect
>> +whether the vcpu is runnable.
>> +
>> 4.39 KVM_SET_MP_STATE
>> ---------------------
>>
>> :Capability: KVM_CAP_MP_STATE
>> -:Architectures: x86, s390, arm64, riscv
>> +:Architectures: x86, s390, arm64, riscv, loongarch
>> :Type: vcpu ioctl
>> :Parameters: struct kvm_mp_state (in)
>> :Returns: 0 on success; -1 on error
>> @@ -1538,6 +1575,9 @@ For arm64/riscv:
>> The only states that are valid are KVM_MP_STATE_STOPPED and
>> KVM_MP_STATE_RUNNABLE which reflect if the vcpu should be paused or not.
>>
>> +On LoongArch, only the KVM_MP_STATE_RUNNABLE state is used to reflect
>> +whether the vcpu is runnable.
>> +
>> 4.40 KVM_SET_IDENTITY_MAP_ADDR
>> ------------------------------
>>
>> @@ -2839,6 +2879,19 @@ Following are the RISC-V D-extension registers:
>> 0x8020 0000 0600 0020 fcsr Floating point control and status register
>> ======================= ========= =============================================
>>
>> +LoongArch registers are mapped using the lower 32 bits. The upper 16 bits of
>> +that is the register group type.
>> +
>> +LoongArch csr registers are used to control guest cpu or get status of guest
>> +cpu, and they have the following id bit patterns::
>> +
>> + 0x9030 0000 0001 00 <reg:5> <sel:3> (64-bit)
>> +
>> +LoongArch KVM control registers are used to implement some new defined functions
>> +such as set vcpu counter or reset vcpu, and they have the following id bit patterns::
>> +
>> + 0x9030 0000 0002 <reg:16>
>> +
>>
>> 4.69 KVM_GET_ONE_REG
>> --------------------
>> --
>> 2.39.1
>>
>>
On Fri, Jun 16, 2023 at 10:39 AM zhaotianrui <[email protected]> wrote:
>
>
> 在 2023/6/15 下午5:42, Huacai Chen 写道:
> > Hi, Tianrui,
> >
> > On Fri, Jun 9, 2023 at 5:09 PM Tianrui Zhao <[email protected]> wrote:
> >> Supplement kvm document about LoongArch-specific part, such as add
> >> api introduction for GET/SET_ONE_REG, GET/SET_FPU, GET/SET_MP_STATE,
> >> etc.
> >>
> >> Signed-off-by: Tianrui Zhao <[email protected]>
> >> ---
> >> Documentation/virt/kvm/api.rst | 71 +++++++++++++++++++++++++++++-----
> >> 1 file changed, 62 insertions(+), 9 deletions(-)
> >>
> >> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> >> index add067793b90..ad8e13eab48d 100644
> >> --- a/Documentation/virt/kvm/api.rst
> >> +++ b/Documentation/virt/kvm/api.rst
> >> @@ -416,6 +416,12 @@ Reads the general purpose registers from the vcpu.
> >> __u64 pc;
> >> };
> >>
> >> + /* LoongArch */
> >> + struct kvm_regs {
> > Add a " /* out (KVM_GET_REGS) / in (KVM_SET_REGS) */" line as others here?
>
> Thanks, I will add this comment here.
>
> Tianrui Zhao
>
> >> + unsigned long gpr[32];
> >> + unsigned long pc;
> >> + };
> >> +
> >>
> >> 4.12 KVM_SET_REGS
> >> -----------------
> >> @@ -506,7 +512,7 @@ translation mode.
> >> ------------------
> >>
> >> :Capability: basic
> >> -:Architectures: x86, ppc, mips, riscv
> >> +:Architectures: x86, ppc, mips, riscv, loongarch
> >> :Type: vcpu ioctl
> >> :Parameters: struct kvm_interrupt (in)
> >> :Returns: 0 on success, negative on failure.
> >> @@ -592,6 +598,14 @@ b) KVM_INTERRUPT_UNSET
> >>
> >> This is an asynchronous vcpu ioctl and can be invoked from any thread.
> >>
> >> +LOONGARCH:
> >> +^^^^^^^^^^
> >> +
> >> +Queues an external interrupt to be injected into the virtual CPU. A negative
> >> +interrupt number dequeues the interrupt.
> >> +
> >> +This is an asynchronous vcpu ioctl and can be invoked from any thread.
> >> +
> >>
> >> 4.17 KVM_DEBUG_GUEST
> >> --------------------
> >> @@ -737,7 +751,7 @@ signal mask.
> >> ----------------
> >>
> >> :Capability: basic
> >> -:Architectures: x86
> >> +:Architectures: x86, loongarch
> >> :Type: vcpu ioctl
> >> :Parameters: struct kvm_fpu (out)
> >> :Returns: 0 on success, -1 on error
> >> @@ -746,7 +760,7 @@ Reads the floating point state from the vcpu.
> >>
> >> ::
> >>
> >> - /* for KVM_GET_FPU and KVM_SET_FPU */
> >> + /* x86: for KVM_GET_FPU and KVM_SET_FPU */
> >> struct kvm_fpu {
> >> __u8 fpr[8][16];
> >> __u16 fcw;
> >> @@ -761,12 +775,22 @@ Reads the floating point state from the vcpu.
> >> __u32 pad2;
> >> };
> >>
> >> + /* LoongArch: for KVM_GET_FPU and KVM_SET_FPU */
> >> + struct kvm_fpu {
> >> + __u32 fcsr;
> >> + __u32 none;
> > Maybe use pad1 as x86 is better.
> >
> > Huacai
>
> Thanks, this 'none' variable is meaningless, and I will remove it. As
> this have already been removed in the actually kvm_fpu structure.
Why remove? I think it is for alignment, just like pad1 for x86.
Huacai
>
> Tianrui Zhao
>
> >> + __u64 fcc;
> >> + struct kvm_fpureg {
> >> + __u64 val64[4];
> >> + }fpr[32];
> >> + };
> >> +
> >>
> >> 4.23 KVM_SET_FPU
> >> ----------------
> >>
> >> :Capability: basic
> >> -:Architectures: x86
> >> +:Architectures: x86, loongarch
> >> :Type: vcpu ioctl
> >> :Parameters: struct kvm_fpu (in)
> >> :Returns: 0 on success, -1 on error
> >> @@ -775,7 +799,7 @@ Writes the floating point state to the vcpu.
> >>
> >> ::
> >>
> >> - /* for KVM_GET_FPU and KVM_SET_FPU */
> >> + /* x86: for KVM_GET_FPU and KVM_SET_FPU */
> >> struct kvm_fpu {
> >> __u8 fpr[8][16];
> >> __u16 fcw;
> >> @@ -790,6 +814,16 @@ Writes the floating point state to the vcpu.
> >> __u32 pad2;
> >> };
> >>
> >> + /* LoongArch: for KVM_GET_FPU and KVM_SET_FPU */
> >> + struct kvm_fpu {
> >> + __u32 fcsr;
> >> + __u32 none;
>
> I will also remove this 'none' variable.
>
> Tianrui Zhao
>
> >> + __u64 fcc;
> >> + struct kvm_fpureg {
> >> + __u64 val64[4];
> >> + }fpr[32];
> >> + };
> >> +
> >>
> >> 4.24 KVM_CREATE_IRQCHIP
> >> -----------------------
> >> @@ -1387,7 +1421,7 @@ documentation when it pops into existence).
> >> -------------------
> >>
> >> :Capability: KVM_CAP_ENABLE_CAP
> >> -:Architectures: mips, ppc, s390, x86
> >> +:Architectures: mips, ppc, s390, x86, loongarch
> >> :Type: vcpu ioctl
> >> :Parameters: struct kvm_enable_cap (in)
> >> :Returns: 0 on success; -1 on error
> >> @@ -1442,7 +1476,7 @@ for vm-wide capabilities.
> >> ---------------------
> >>
> >> :Capability: KVM_CAP_MP_STATE
> >> -:Architectures: x86, s390, arm64, riscv
> >> +:Architectures: x86, s390, arm64, riscv, loongarch
> >> :Type: vcpu ioctl
> >> :Parameters: struct kvm_mp_state (out)
> >> :Returns: 0 on success; -1 on error
> >> @@ -1460,7 +1494,7 @@ Possible values are:
> >>
> >> ========================== ===============================================
> >> KVM_MP_STATE_RUNNABLE the vcpu is currently running
> >> - [x86,arm64,riscv]
> >> + [x86,arm64,riscv,loongarch]
> >> KVM_MP_STATE_UNINITIALIZED the vcpu is an application processor (AP)
> >> which has not yet received an INIT signal [x86]
> >> KVM_MP_STATE_INIT_RECEIVED the vcpu has received an INIT signal, and is
> >> @@ -1516,11 +1550,14 @@ For riscv:
> >> The only states that are valid are KVM_MP_STATE_STOPPED and
> >> KVM_MP_STATE_RUNNABLE which reflect if the vcpu is paused or not.
> >>
> >> +On LoongArch, only the KVM_MP_STATE_RUNNABLE state is used to reflect
> >> +whether the vcpu is runnable.
> >> +
> >> 4.39 KVM_SET_MP_STATE
> >> ---------------------
> >>
> >> :Capability: KVM_CAP_MP_STATE
> >> -:Architectures: x86, s390, arm64, riscv
> >> +:Architectures: x86, s390, arm64, riscv, loongarch
> >> :Type: vcpu ioctl
> >> :Parameters: struct kvm_mp_state (in)
> >> :Returns: 0 on success; -1 on error
> >> @@ -1538,6 +1575,9 @@ For arm64/riscv:
> >> The only states that are valid are KVM_MP_STATE_STOPPED and
> >> KVM_MP_STATE_RUNNABLE which reflect if the vcpu should be paused or not.
> >>
> >> +On LoongArch, only the KVM_MP_STATE_RUNNABLE state is used to reflect
> >> +whether the vcpu is runnable.
> >> +
> >> 4.40 KVM_SET_IDENTITY_MAP_ADDR
> >> ------------------------------
> >>
> >> @@ -2839,6 +2879,19 @@ Following are the RISC-V D-extension registers:
> >> 0x8020 0000 0600 0020 fcsr Floating point control and status register
> >> ======================= ========= =============================================
> >>
> >> +LoongArch registers are mapped using the lower 32 bits. The upper 16 bits of
> >> +that is the register group type.
> >> +
> >> +LoongArch csr registers are used to control guest cpu or get status of guest
> >> +cpu, and they have the following id bit patterns::
> >> +
> >> + 0x9030 0000 0001 00 <reg:5> <sel:3> (64-bit)
> >> +
> >> +LoongArch KVM control registers are used to implement some new defined functions
> >> +such as set vcpu counter or reset vcpu, and they have the following id bit patterns::
> >> +
> >> + 0x9030 0000 0002 <reg:16>
> >> +
> >>
> >> 4.69 KVM_GET_ONE_REG
> >> --------------------
> >> --
> >> 2.39.1
> >>
> >>
>
在 2023/6/16 上午10:59, Huacai Chen 写道:
> On Fri, Jun 16, 2023 at 10:39 AM zhaotianrui <[email protected]> wrote:
>>
>> 在 2023/6/15 下午5:42, Huacai Chen 写道:
>>> Hi, Tianrui,
>>>
>>> On Fri, Jun 9, 2023 at 5:09 PM Tianrui Zhao <[email protected]> wrote:
>>>> Supplement kvm document about LoongArch-specific part, such as add
>>>> api introduction for GET/SET_ONE_REG, GET/SET_FPU, GET/SET_MP_STATE,
>>>> etc.
>>>>
>>>> Signed-off-by: Tianrui Zhao <[email protected]>
>>>> ---
>>>> Documentation/virt/kvm/api.rst | 71 +++++++++++++++++++++++++++++-----
>>>> 1 file changed, 62 insertions(+), 9 deletions(-)
>>>>
>>>> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
>>>> index add067793b90..ad8e13eab48d 100644
>>>> --- a/Documentation/virt/kvm/api.rst
>>>> +++ b/Documentation/virt/kvm/api.rst
>>>> @@ -416,6 +416,12 @@ Reads the general purpose registers from the vcpu.
>>>> __u64 pc;
>>>> };
>>>>
>>>> + /* LoongArch */
>>>> + struct kvm_regs {
>>> Add a " /* out (KVM_GET_REGS) / in (KVM_SET_REGS) */" line as others here?
>> Thanks, I will add this comment here.
>>
>> Tianrui Zhao
>>
>>>> + unsigned long gpr[32];
>>>> + unsigned long pc;
>>>> + };
>>>> +
>>>>
>>>> 4.12 KVM_SET_REGS
>>>> -----------------
>>>> @@ -506,7 +512,7 @@ translation mode.
>>>> ------------------
>>>>
>>>> :Capability: basic
>>>> -:Architectures: x86, ppc, mips, riscv
>>>> +:Architectures: x86, ppc, mips, riscv, loongarch
>>>> :Type: vcpu ioctl
>>>> :Parameters: struct kvm_interrupt (in)
>>>> :Returns: 0 on success, negative on failure.
>>>> @@ -592,6 +598,14 @@ b) KVM_INTERRUPT_UNSET
>>>>
>>>> This is an asynchronous vcpu ioctl and can be invoked from any thread.
>>>>
>>>> +LOONGARCH:
>>>> +^^^^^^^^^^
>>>> +
>>>> +Queues an external interrupt to be injected into the virtual CPU. A negative
>>>> +interrupt number dequeues the interrupt.
>>>> +
>>>> +This is an asynchronous vcpu ioctl and can be invoked from any thread.
>>>> +
>>>>
>>>> 4.17 KVM_DEBUG_GUEST
>>>> --------------------
>>>> @@ -737,7 +751,7 @@ signal mask.
>>>> ----------------
>>>>
>>>> :Capability: basic
>>>> -:Architectures: x86
>>>> +:Architectures: x86, loongarch
>>>> :Type: vcpu ioctl
>>>> :Parameters: struct kvm_fpu (out)
>>>> :Returns: 0 on success, -1 on error
>>>> @@ -746,7 +760,7 @@ Reads the floating point state from the vcpu.
>>>>
>>>> ::
>>>>
>>>> - /* for KVM_GET_FPU and KVM_SET_FPU */
>>>> + /* x86: for KVM_GET_FPU and KVM_SET_FPU */
>>>> struct kvm_fpu {
>>>> __u8 fpr[8][16];
>>>> __u16 fcw;
>>>> @@ -761,12 +775,22 @@ Reads the floating point state from the vcpu.
>>>> __u32 pad2;
>>>> };
>>>>
>>>> + /* LoongArch: for KVM_GET_FPU and KVM_SET_FPU */
>>>> + struct kvm_fpu {
>>>> + __u32 fcsr;
>>>> + __u32 none;
>>> Maybe use pad1 as x86 is better.
>>>
>>> Huacai
>> Thanks, this 'none' variable is meaningless, and I will remove it. As
>> this have already been removed in the actually kvm_fpu structure.
> Why remove? I think it is for alignment, just like pad1 for x86.
>
> Huacai
This 'none' variable is not for alignment but a unused register named
vcsr in kvm_fpu, so I think it could be removed.
Tianrui Zhao
>> Tianrui Zhao
>>
>>>> + __u64 fcc;
>>>> + struct kvm_fpureg {
>>>> + __u64 val64[4];
>>>> + }fpr[32];
>>>> + };
>>>> +
>>>>
>>>> 4.23 KVM_SET_FPU
>>>> ----------------
>>>>
>>>> :Capability: basic
>>>> -:Architectures: x86
>>>> +:Architectures: x86, loongarch
>>>> :Type: vcpu ioctl
>>>> :Parameters: struct kvm_fpu (in)
>>>> :Returns: 0 on success, -1 on error
>>>> @@ -775,7 +799,7 @@ Writes the floating point state to the vcpu.
>>>>
>>>> ::
>>>>
>>>> - /* for KVM_GET_FPU and KVM_SET_FPU */
>>>> + /* x86: for KVM_GET_FPU and KVM_SET_FPU */
>>>> struct kvm_fpu {
>>>> __u8 fpr[8][16];
>>>> __u16 fcw;
>>>> @@ -790,6 +814,16 @@ Writes the floating point state to the vcpu.
>>>> __u32 pad2;
>>>> };
>>>>
>>>> + /* LoongArch: for KVM_GET_FPU and KVM_SET_FPU */
>>>> + struct kvm_fpu {
>>>> + __u32 fcsr;
>>>> + __u32 none;
>> I will also remove this 'none' variable.
>>
>> Tianrui Zhao
>>
>>>> + __u64 fcc;
>>>> + struct kvm_fpureg {
>>>> + __u64 val64[4];
>>>> + }fpr[32];
>>>> + };
>>>> +
>>>>
>>>> 4.24 KVM_CREATE_IRQCHIP
>>>> -----------------------
>>>> @@ -1387,7 +1421,7 @@ documentation when it pops into existence).
>>>> -------------------
>>>>
>>>> :Capability: KVM_CAP_ENABLE_CAP
>>>> -:Architectures: mips, ppc, s390, x86
>>>> +:Architectures: mips, ppc, s390, x86, loongarch
>>>> :Type: vcpu ioctl
>>>> :Parameters: struct kvm_enable_cap (in)
>>>> :Returns: 0 on success; -1 on error
>>>> @@ -1442,7 +1476,7 @@ for vm-wide capabilities.
>>>> ---------------------
>>>>
>>>> :Capability: KVM_CAP_MP_STATE
>>>> -:Architectures: x86, s390, arm64, riscv
>>>> +:Architectures: x86, s390, arm64, riscv, loongarch
>>>> :Type: vcpu ioctl
>>>> :Parameters: struct kvm_mp_state (out)
>>>> :Returns: 0 on success; -1 on error
>>>> @@ -1460,7 +1494,7 @@ Possible values are:
>>>>
>>>> ========================== ===============================================
>>>> KVM_MP_STATE_RUNNABLE the vcpu is currently running
>>>> - [x86,arm64,riscv]
>>>> + [x86,arm64,riscv,loongarch]
>>>> KVM_MP_STATE_UNINITIALIZED the vcpu is an application processor (AP)
>>>> which has not yet received an INIT signal [x86]
>>>> KVM_MP_STATE_INIT_RECEIVED the vcpu has received an INIT signal, and is
>>>> @@ -1516,11 +1550,14 @@ For riscv:
>>>> The only states that are valid are KVM_MP_STATE_STOPPED and
>>>> KVM_MP_STATE_RUNNABLE which reflect if the vcpu is paused or not.
>>>>
>>>> +On LoongArch, only the KVM_MP_STATE_RUNNABLE state is used to reflect
>>>> +whether the vcpu is runnable.
>>>> +
>>>> 4.39 KVM_SET_MP_STATE
>>>> ---------------------
>>>>
>>>> :Capability: KVM_CAP_MP_STATE
>>>> -:Architectures: x86, s390, arm64, riscv
>>>> +:Architectures: x86, s390, arm64, riscv, loongarch
>>>> :Type: vcpu ioctl
>>>> :Parameters: struct kvm_mp_state (in)
>>>> :Returns: 0 on success; -1 on error
>>>> @@ -1538,6 +1575,9 @@ For arm64/riscv:
>>>> The only states that are valid are KVM_MP_STATE_STOPPED and
>>>> KVM_MP_STATE_RUNNABLE which reflect if the vcpu should be paused or not.
>>>>
>>>> +On LoongArch, only the KVM_MP_STATE_RUNNABLE state is used to reflect
>>>> +whether the vcpu is runnable.
>>>> +
>>>> 4.40 KVM_SET_IDENTITY_MAP_ADDR
>>>> ------------------------------
>>>>
>>>> @@ -2839,6 +2879,19 @@ Following are the RISC-V D-extension registers:
>>>> 0x8020 0000 0600 0020 fcsr Floating point control and status register
>>>> ======================= ========= =============================================
>>>>
>>>> +LoongArch registers are mapped using the lower 32 bits. The upper 16 bits of
>>>> +that is the register group type.
>>>> +
>>>> +LoongArch csr registers are used to control guest cpu or get status of guest
>>>> +cpu, and they have the following id bit patterns::
>>>> +
>>>> + 0x9030 0000 0001 00 <reg:5> <sel:3> (64-bit)
>>>> +
>>>> +LoongArch KVM control registers are used to implement some new defined functions
>>>> +such as set vcpu counter or reset vcpu, and they have the following id bit patterns::
>>>> +
>>>> + 0x9030 0000 0002 <reg:16>
>>>> +
>>>>
>>>> 4.69 KVM_GET_ONE_REG
>>>> --------------------
>>>> --
>>>> 2.39.1
>>>>
>>>>
在 2023/6/13 下午8:49, bibo, mao 写道:
>
>
> 在 2023/6/9 17:08, Tianrui Zhao 写道:
>> Implement kvm hardware enable, disable interface, setting
>> the guest config register to enable virtualization features
>> when called the interface.
>>
>> Signed-off-by: Tianrui Zhao <[email protected]>
>> ---
>> arch/loongarch/kvm/main.c | 64 +++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 64 insertions(+)
>>
>> diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
>> index f98c1619725f..5ebae1ea7565 100644
>> --- a/arch/loongarch/kvm/main.c
>> +++ b/arch/loongarch/kvm/main.c
>> @@ -195,6 +195,70 @@ static void _kvm_init_gcsr_flag(void)
>> set_gcsr_sw_flag(LOONGARCH_CSR_PERFCNTR3);
>> }
>> +void kvm_init_vmcs(struct kvm *kvm)
>> +{
>> + kvm->arch.vmcs = vmcs;
>> +}
>> +
>> +long kvm_arch_dev_ioctl(struct file *filp,
>> + unsigned int ioctl, unsigned long arg)
>> +{
>> + return -ENOIOCTLCMD;
>> +}
>> +
>> +#ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING
> Can we remove macro CONFIG_KVM_GENERIC_HARDWARE_ENABLING here since it
> is always enabled? The other looks good to me.
>
> Reviewed-by: Bibo Mao <[email protected]>
>
> Regards
> Bibo, Mao
Thanks, this CONFIG_KVM_GENERIC_HARDWARE_ENABLING is meaningless, and I
will remove it.
Tianrui Zhao
>> +int kvm_arch_hardware_enable(void)
>> +{
>> + unsigned long env, gcfg = 0;
>> +
>> + env = read_csr_gcfg();
>> + /* First init gtlbc, gcfg, gstat, gintc. All guest use the same
>> config */
>> + clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI);
>> + write_csr_gcfg(0);
>> + write_csr_gstat(0);
>> + write_csr_gintc(0);
>> +
>> + /*
>> + * Enable virtualization features granting guest direct control of
>> + * certain features:
>> + * GCI=2: Trap on init or unimplement cache instruction.
>> + * TORU=0: Trap on Root Unimplement.
>> + * CACTRL=1: Root control cache.
>> + * TOP=0: Trap on Previlege.
>> + * TOE=0: Trap on Exception.
>> + * TIT=0: Trap on Timer.
>> + */
>> + if (env & CSR_GCFG_GCIP_ALL)
>> + gcfg |= CSR_GCFG_GCI_SECURE;
>> + if (env & CSR_GCFG_MATC_ROOT)
>> + gcfg |= CSR_GCFG_MATC_ROOT;
>> +
>> + gcfg |= CSR_GCFG_TIT;
>> + write_csr_gcfg(gcfg);
>> +
>> + kvm_flush_tlb_all();
>> +
>> + /* Enable using TGID */
>> + set_csr_gtlbc(CSR_GTLBC_USETGID);
>> + kvm_debug("gtlbc:%llx gintc:%llx gstat:%llx gcfg:%llx",
>> + read_csr_gtlbc(), read_csr_gintc(),
>> + read_csr_gstat(), read_csr_gcfg());
>> +
>> + return 0;
>> +}
>> +
>> +void kvm_arch_hardware_disable(void)
>> +{
>> + clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI);
>> + write_csr_gcfg(0);
>> + write_csr_gstat(0);
>> + write_csr_gintc(0);
>> +
>> + /* Flush any remaining guest TLB entries */
>> + kvm_flush_tlb_all();
>> +}
>> +#endif
>> +
>> static int kvm_loongarch_env_init(void)
>> {
>> struct kvm_context *context;