2021-09-08 17:49:57

by Greentime Hu

Subject: [RFC PATCH v8 00/21] riscv: Add vector ISA support

This patchset is implemented based on vector 1.0-rc1 spec to add vector
support in riscv Linux kernel. To make this happen, we defined a new
structure __riscv_v_state to save the vector related registers. It is used
for both kernel space and user space.

- In kernel space, the datap pointer in __riscv_v_state will be allocated
dynamically to save vector registers.
- In user space,
- In signal handler of user space, datap will point to the address
of the __riscv_v_state data structure to save vector
registers in stack. We also create a __reserved[] array for
future extensions.
- In ptrace, the data will be put in ubuf in which we use
riscv_vr_get()/riscv_vr_set() to get or set the
__riscv_v_state data structure from/to it, datap pointer
would be zeroed and vector registers will be copied to the
address right after the __riscv_v_state structure in ubuf.

This patchset also adds support for kernel mode vector, kernel XOR
implementation with vector ISA and includes several bug fixes and code
refinement.

This patchset is rebased to v5.14 and it is tested by running several
vector programs simultaneously. It also can get the correct ucontext_t in
signal handler and restore correct context after sigreturn. It is also
tested with ptrace() syscall to use PTRACE_GETREGSET/PTRACE_SETREGSET to
get/set the vector registers. I have tested vlen=128 and vlen=256 cases in
qemu-system-riscv64 provided by Frank Chang.

We have sent patches to glibc mailing list for ifunc support and sigcontext
changes. We will send patches for vector support to glibc mailing list
recently.

[1] https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc

---
Changelog V8
- Rebase to v5.14
- Refine struct __riscv_v_state with struct __riscv_ctx_hdr
- Refine has_vector into a static key
- Defined __reserved space in struct sigcontext for vector and future extensions

Changelog V7
- Add support for kernel mode vector
- Add vector extension XOR implementation
- Optimize task switch codes of vector
- Allocate space for vector registers in start_thread()
- Fix an illegal instruction exception when accessing vlenb
- Optimize vector registers initialization
- Initialize vector registers with proper vsetvli then it can work normally
- Refine ptrace porting due to generic API changed
- Code clean up

Changelog V6
- Replace vle.v/vse.v instructions with vle8.v/vse8.v based on 0.9 spec
- Add comments based on mailinglist feedback
- Fix rv32 build error

Changelog V5
- Using regset_size() correctly in generic ptrace
- Fix the ptrace porting
- Fix compile warning

Changelog V4
- Support dynamic vlen
- Fix bugs: lazy save/resotre, not saving vtype
- Update VS bit offset based on latest vector spec
- Add new vector csr based on latest vector spec
- Code refine and removed unused macros

Changelog V3
- Rebase linux-5.6-rc3 and tested with qemu
- Seperate patches with Anup's advice
- Give out a ABI puzzle with unlimited vlen

Changelog V2
- Fixup typo "vecotr, fstate_save->vstate_save".
- Fixup wrong saved registers' length in vector.S.
- Seperate unrelated patches from this one.

Greentime Hu (15):
riscv: Add new csr defines related to vector extension
riscv: Add has_vector/riscv_vsize to save vector features.
riscv: Add vector struct and assembler definitions
riscv: Add task switch support for vector
riscv: Add ptrace vector support
riscv: Add sigcontext save/restore for vector
riscv: Add support for kernel mode vector
riscv: Use CSR_STATUS to replace sstatus in vector.S
riscv: Add vector extension XOR implementation
riscv: Initialize vector registers with proper vsetvli then it can
work normally
riscv: Optimize vector registers initialization
riscv: Fix an illegal instruction exception when accessing vlenb
without enable vector first
riscv: Allocate space for vector registers in start_thread()
riscv: Optimize task switch codes of vector
riscv: Turn has_vector into a static key if VECTOR=y

Guo Ren (5):
riscv: Separate patch for cflags and aflags
riscv: Rename __switch_to_aux -> fpu
riscv: Extending cpufeature.c to detect V-extension
riscv: Add vector feature to compile
riscv: Reset vector register

Vincent Chen (1):
riscv: signal: Report signal frame size to userspace via auxv

arch/riscv/Kconfig | 9 ++
arch/riscv/Makefile | 19 ++-
arch/riscv/include/asm/csr.h | 16 ++-
arch/riscv/include/asm/elf.h | 41 +++---
arch/riscv/include/asm/processor.h | 3 +
arch/riscv/include/asm/switch_to.h | 71 +++++++++-
arch/riscv/include/asm/vector.h | 16 +++
arch/riscv/include/asm/xor.h | 74 ++++++++++
arch/riscv/include/uapi/asm/auxvec.h | 1 +
arch/riscv/include/uapi/asm/hwcap.h | 1 +
arch/riscv/include/uapi/asm/ptrace.h | 25 ++++
arch/riscv/include/uapi/asm/sigcontext.h | 24 ++++
arch/riscv/kernel/Makefile | 7 +
arch/riscv/kernel/asm-offsets.c | 8 ++
arch/riscv/kernel/cpufeature.c | 16 +++
arch/riscv/kernel/entry.S | 6 +-
arch/riscv/kernel/head.S | 22 ++-
arch/riscv/kernel/kernel_mode_vector.c | 158 +++++++++++++++++++++
arch/riscv/kernel/process.c | 49 +++++++
arch/riscv/kernel/ptrace.c | 71 ++++++++++
arch/riscv/kernel/setup.c | 4 +
arch/riscv/kernel/signal.c | 172 ++++++++++++++++++++++-
arch/riscv/kernel/vector.S | 81 +++++++++++
arch/riscv/lib/Makefile | 1 +
arch/riscv/lib/xor.S | 81 +++++++++++
include/uapi/linux/elf.h | 1 +
26 files changed, 941 insertions(+), 36 deletions(-)
create mode 100644 arch/riscv/include/asm/vector.h
create mode 100644 arch/riscv/include/asm/xor.h
create mode 100644 arch/riscv/kernel/kernel_mode_vector.c
create mode 100644 arch/riscv/kernel/vector.S
create mode 100644 arch/riscv/lib/xor.S

--
2.31.1

2021-09-08 17:50:01

by Greentime Hu

Subject: [RFC PATCH v8 21/21] riscv: Turn has_vector into a static key if VECTOR=y

Just like fpu, we can use static key for has_vector.
The has_vector check sits at hot code path: switch_to(). Currently,
has_vector is a bool variable if VECTOR=y, switch_to() checks it each time,
we can optimize out this check by turning the has_vector into a static key.

Signed-off-by: Greentime Hu <[email protected]>
---
arch/riscv/include/asm/switch_to.h | 10 +++++++---
arch/riscv/kernel/cpufeature.c | 4 ++--
arch/riscv/kernel/kernel_mode_vector.c | 4 ++--
arch/riscv/kernel/process.c | 8 ++++----
arch/riscv/kernel/signal.c | 6 +++---
5 files changed, 18 insertions(+), 14 deletions(-)

diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h
index b48c9c974564..576204217e0f 100644
--- a/arch/riscv/include/asm/switch_to.h
+++ b/arch/riscv/include/asm/switch_to.h
@@ -71,7 +71,11 @@ static __always_inline bool has_fpu(void) { return false; }
#endif

#ifdef CONFIG_VECTOR
-extern bool has_vector;
+extern struct static_key_false cpu_hwcap_vector;
+static __always_inline bool has_vector(void)
+{
+ return static_branch_likely(&cpu_hwcap_vector);
+}
extern unsigned long riscv_vsize;
extern void __vstate_save(struct __riscv_v_state *save_to, void *datap);
extern void __vstate_restore(struct __riscv_v_state *restore_from, void *datap);
@@ -120,7 +124,7 @@ static inline void __switch_to_vector(struct task_struct *prev,
}

#else
-#define has_vector false
+static __always_inline bool has_vector(void) { return false; }
#define riscv_vsize (0)
#define vstate_save(task, regs) do { } while (0)
#define vstate_restore(task, regs) do { } while (0)
@@ -136,7 +140,7 @@ do { \
struct task_struct *__next = (next); \
if (has_fpu()) \
__switch_to_fpu(__prev, __next); \
- if (has_vector) \
+ if (has_vector()) \
__switch_to_vector(__prev, __next); \
((last) = __switch_to(__prev, __next)); \
} while (0)
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index af984f875f60..0139ec20adce 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -23,7 +23,7 @@ __ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_fpu);
#endif
#ifdef CONFIG_VECTOR
#include <asm/vector.h>
-bool has_vector __read_mostly;
+__ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_vector);
unsigned long riscv_vsize __read_mostly;
#endif

@@ -157,7 +157,7 @@ void __init riscv_fill_hwcap(void)

#ifdef CONFIG_VECTOR
if (elf_hwcap & COMPAT_HWCAP_ISA_V) {
- has_vector = true;
+ static_branch_enable(&cpu_hwcap_vector);
/* There are 32 vector registers with vlenb length. */
rvv_enable();
riscv_vsize = csr_read(CSR_VLENB) * 32;
diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c
index 0d990bd8b8dd..0d08954c30af 100644
--- a/arch/riscv/kernel/kernel_mode_vector.c
+++ b/arch/riscv/kernel/kernel_mode_vector.c
@@ -110,7 +110,7 @@ static void vector_flush_cpu_state(void)
*/
void kernel_rvv_begin(void)
{
- if (WARN_ON(!has_vector))
+ if (WARN_ON(!has_vector()))
return;

WARN_ON(!may_use_vector());
@@ -140,7 +140,7 @@ EXPORT_SYMBOL(kernel_rvv_begin);
*/
void kernel_rvv_end(void)
{
- if (WARN_ON(!has_vector))
+ if (WARN_ON(!has_vector()))
return;

/* Invalidate vector regs */
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index 05ff5f934e7e..62540815ba1c 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -96,7 +96,7 @@ void start_thread(struct pt_regs *regs, unsigned long pc,
fstate_restore(current, regs);
}

- if (has_vector) {
+ if (has_vector()) {
struct __riscv_v_state *vstate = &(current->thread.vstate);

/* Enable vector and allocate memory for vector registers. */
@@ -141,11 +141,11 @@ void flush_thread(void)
int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
{
fstate_save(src, task_pt_regs(src));
- if (has_vector)
+ if (has_vector())
/* To make sure every dirty vector context is saved. */
vstate_save(src, task_pt_regs(src));
*dst = *src;
- if (has_vector) {
+ if (has_vector()) {
/* Copy vector context to the forked task from parent. */
if ((task_pt_regs(src)->status & SR_VS) != SR_VS_OFF) {
dst->thread.vstate.datap = kzalloc(riscv_vsize, GFP_KERNEL);
@@ -164,7 +164,7 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
void arch_release_task_struct(struct task_struct *tsk)
{
/* Free the vector context of datap. */
- if (has_vector)
+ if (has_vector())
kfree(tsk->thread.vstate.datap);
}

diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c
index d30a3b588156..6a19b4b7b206 100644
--- a/arch/riscv/kernel/signal.c
+++ b/arch/riscv/kernel/signal.c
@@ -192,7 +192,7 @@ static long restore_sigcontext(struct pt_regs *regs,
goto invalid;
goto done;
case RVV_MAGIC:
- if (!has_vector)
+ if (!has_vector())
goto invalid;
if (size != rvv_sc_size)
goto invalid;
@@ -221,7 +221,7 @@ static size_t cal_rt_frame_size(void)

frame_size = sizeof(*frame);

- if (has_vector)
+ if (has_vector())
total_context_size += rvv_sc_size;
/* Preserved a __riscv_ctx_hdr for END signal context header. */
total_context_size += sizeof(struct __riscv_ctx_hdr);
@@ -288,7 +288,7 @@ static long setup_sigcontext(struct rt_sigframe __user *frame,
if (has_fpu())
err |= save_fp_state(regs, &sc->sc_fpregs);
/* Save the vector state. */
- if (has_vector)
+ if (has_vector())
err |= save_v_state(regs, &sc_reserved_free_ptr);

/* Put END __riscv_ctx_hdr at the end. */
--
2.31.1

2021-09-08 17:50:33

by Greentime Hu

Subject: [RFC PATCH v8 20/21] riscv: Optimize task switch codes of vector

This patch replacees 2 instructions with 1 instruction to do the same thing
. rs1=x0 with rd != x0 is a special form of the instruction that sets vl to
MAXVL.

Suggested-by: Andrew Waterman <[email protected]>
Co-developed-by: Vincent Chen <[email protected]>
Signed-off-by: Vincent Chen <[email protected]>
Signed-off-by: Greentime Hu <[email protected]>
---
arch/riscv/kernel/vector.S | 9 +++------
1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/arch/riscv/kernel/vector.S b/arch/riscv/kernel/vector.S
index 4f0c5a166e4e..f7223c81b11a 100644
--- a/arch/riscv/kernel/vector.S
+++ b/arch/riscv/kernel/vector.S
@@ -27,8 +27,7 @@
#define x_vl t2
#define x_vcsr t3
#define incr t4
-#define m_one t5
-#define status t6
+#define status t5

ENTRY(__vstate_save)
li status, SR_VS
@@ -38,8 +37,7 @@ ENTRY(__vstate_save)
csrr x_vtype, CSR_VTYPE
csrr x_vl, CSR_VL
csrr x_vcsr, CSR_VCSR
- li m_one, -1
- vsetvli incr, m_one, e8, m8
+ vsetvli incr, x0, e8, m8
vse8.v v0, (datap)
add datap, datap, incr
vse8.v v8, (datap)
@@ -61,8 +59,7 @@ ENTRY(__vstate_restore)
li status, SR_VS
csrs CSR_STATUS, status

- li m_one, -1
- vsetvli incr, m_one, e8, m8
+ vsetvli incr, x0, e8, m8
vle8.v v0, (datap)
add datap, datap, incr
vle8.v v8, (datap)
--
2.31.1

2021-09-08 17:50:42

by Greentime Hu

Subject: [RFC PATCH v8 13/21] riscv: Add support for kernel mode vector

Add <asm/vector.h> containing kernel_rvv_begin()/kernel_rvv_end() function
declarations and corresponding definitions in kernel_mode_vector.c

These are needed to wrap uses of vector in kernel mode.

Signed-off-by: Greentime Hu <[email protected]>
Co-developed-by: Vincent Chen <[email protected]>
Signed-off-by: Vincent Chen <[email protected]>
---
arch/riscv/include/asm/vector.h | 14 ++
arch/riscv/kernel/Makefile | 6 +
arch/riscv/kernel/kernel_mode_vector.c | 184 +++++++++++++++++++++++++
3 files changed, 204 insertions(+)
create mode 100644 arch/riscv/include/asm/vector.h
create mode 100644 arch/riscv/kernel/kernel_mode_vector.c

diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h
new file mode 100644
index 000000000000..5d7f14453f68
--- /dev/null
+++ b/arch/riscv/include/asm/vector.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2020 SiFive
+ */
+
+#ifndef __ASM_RISCV_VECTOR_H
+#define __ASM_RISCV_VECTOR_H
+
+#include <linux/types.h>
+
+void kernel_rvv_begin(void);
+void kernel_rvv_end(void);
+
+#endif /* ! __ASM_RISCV_VECTOR_H */
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index 344078080839..a2efd3646cd8 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -41,6 +41,12 @@ obj-$(CONFIG_MMU) += vdso.o vdso/
obj-$(CONFIG_RISCV_M_MODE) += traps_misaligned.o
obj-$(CONFIG_FPU) += fpu.o
obj-$(CONFIG_VECTOR) += vector.o
+obj-$(CONFIG_VECTOR) += kernel_mode_vector.o
+riscv-march-cflags-$(CONFIG_ARCH_RV32I) := rv32ima
+riscv-march-cflags-$(CONFIG_ARCH_RV64I) := rv64ima
+riscv-march-cflags-$(CONFIG_RISCV_ISA_C) := $(riscv-march-cflags-y)c
+riscv-march-cflags-$(CONFIG_VECTOR) := $(riscv-march-cflags-y)v
+CFLAGS_kernel_mode_vector.o += -march=$(riscv-march-cflags-y)
obj-$(CONFIG_SMP) += smpboot.o
obj-$(CONFIG_SMP) += smp.o
obj-$(CONFIG_SMP) += cpu_ops.o
diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c
new file mode 100644
index 000000000000..108cfafe7496
--- /dev/null
+++ b/arch/riscv/kernel/kernel_mode_vector.c
@@ -0,0 +1,184 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ * Author: Catalin Marinas <[email protected]>
+ * Copyright (C) 2017 Linaro Ltd. <[email protected]>
+ * Copyright (C) 2020 SiFive
+ */
+#include <linux/compiler.h>
+#include <linux/irqflags.h>
+#include <linux/percpu.h>
+#include <linux/preempt.h>
+#include <linux/types.h>
+
+#include <asm/vector.h>
+#include <asm/switch_to.h>
+
+DECLARE_PER_CPU(bool, vector_context_busy);
+DEFINE_PER_CPU(bool, vector_context_busy);
+
+/*
+ * may_use_vector - whether it is allowable at this time to issue vector
+ * instructions or access the vector register file
+ *
+ * Callers must not assume that the result remains true beyond the next
+ * preempt_enable() or return from softirq context.
+ */
+static __must_check inline bool may_use_vector(void)
+{
+ /*
+ * vector_context_busy is only set while preemption is disabled,
+ * and is clear whenever preemption is enabled. Since
+ * this_cpu_read() is atomic w.r.t. preemption, vector_context_busy
+ * cannot change under our feet -- if it's set we cannot be
+ * migrated, and if it's clear we cannot be migrated to a CPU
+ * where it is set.
+ */
+ return !in_irq() && !irqs_disabled() && !in_nmi() &&
+ !this_cpu_read(vector_context_busy);
+}
+
+
+
+/*
+ * Claim ownership of the CPU vector context for use by the calling context.
+ *
+ * The caller may freely manipulate the vector context metadata until
+ * put_cpu_vector_context() is called.
+ */
+static void get_cpu_vector_context(void)
+{
+ bool busy;
+
+ preempt_disable();
+ busy = __this_cpu_xchg(vector_context_busy, true);
+
+ WARN_ON(busy);
+}
+
+/*
+ * Release the CPU vector context.
+ *
+ * Must be called from a context in which get_cpu_vector_context() was
+ * previously called, with no call to put_cpu_vector_context() in the
+ * meantime.
+ */
+static void put_cpu_vector_context(void)
+{
+ bool busy = __this_cpu_xchg(vector_context_busy, false);
+
+ WARN_ON(!busy);
+ preempt_enable();
+}
+
+static void rvv_enable(void)
+{
+ csr_set(CSR_STATUS, SR_VS);
+}
+
+static void rvv_disable(void)
+{
+ csr_clear(CSR_STATUS, SR_VS);
+}
+
+static void vector_flush_cpu_state(void)
+{
+ long tmp;
+
+ __asm__ __volatile__ (
+ "vsetvli %0, x0, e8, m1\n"
+ "vmv.v.i v0, 0\n"
+ "vmv.v.i v1, 0\n"
+ "vmv.v.i v2, 0\n"
+ "vmv.v.i v3, 0\n"
+ "vmv.v.i v4, 0\n"
+ "vmv.v.i v5, 0\n"
+ "vmv.v.i v6, 0\n"
+ "vmv.v.i v7, 0\n"
+ "vmv.v.i v8, 0\n"
+ "vmv.v.i v9, 0\n"
+ "vmv.v.i v10, 0\n"
+ "vmv.v.i v11, 0\n"
+ "vmv.v.i v12, 0\n"
+ "vmv.v.i v13, 0\n"
+ "vmv.v.i v14, 0\n"
+ "vmv.v.i v15, 0\n"
+ "vmv.v.i v16, 0\n"
+ "vmv.v.i v17, 0\n"
+ "vmv.v.i v18, 0\n"
+ "vmv.v.i v19, 0\n"
+ "vmv.v.i v20, 0\n"
+ "vmv.v.i v21, 0\n"
+ "vmv.v.i v22, 0\n"
+ "vmv.v.i v23, 0\n"
+ "vmv.v.i v24, 0\n"
+ "vmv.v.i v25, 0\n"
+ "vmv.v.i v26, 0\n"
+ "vmv.v.i v27, 0\n"
+ "vmv.v.i v28, 0\n"
+ "vmv.v.i v29, 0\n"
+ "vmv.v.i v30, 0\n"
+ "vmv.v.i v31, 0\n":"=r"(tmp)::);
+}
+
+/*
+ * kernel_rvv_begin(): obtain the CPU vector registers for use by the calling
+ * context
+ *
+ * Must not be called unless may_use_vector() returns true.
+ * Task context in the vector registers is saved back to memory as necessary.
+ *
+ * A matching call to kernel_rvv_end() must be made before returning from the
+ * calling context.
+ *
+ * The caller may freely use the vector registers until kernel_rvv_end() is
+ * called.
+ */
+void kernel_rvv_begin(void)
+{
+ if (WARN_ON(!has_vector))
+ return;
+
+ WARN_ON(!may_use_vector());
+
+ /* Acquire kernel mode vector */
+ get_cpu_vector_context();
+
+ /* Save vector state, if any */
+ vstate_save(current, task_pt_regs(current));
+
+ /* Enable vector */
+ rvv_enable();
+
+ /* Invalidate vector regs */
+ vector_flush_cpu_state();
+}
+EXPORT_SYMBOL(kernel_rvv_begin);
+
+/*
+ * kernel_rvv_end(): give the CPU vector registers back to the current task
+ *
+ * Must be called from a context in which kernel_rvv_begin() was previously
+ * called, with no call to kernel_rvv_end() in the meantime.
+ *
+ * The caller must not use the vector registers after this function is called,
+ * unless kernel_rvv_begin() is called again in the meantime.
+ */
+void kernel_rvv_end(void)
+{
+ if (WARN_ON(!has_vector))
+ return;
+
+ /* Invalidate vector regs */
+ vector_flush_cpu_state();
+
+ /* Restore vector state, if any */
+ vstate_restore(current, task_pt_regs(current));
+
+ /* disable vector */
+ rvv_disable();
+
+ /* release kernel mode vector */
+ put_cpu_vector_context();
+}
+EXPORT_SYMBOL(kernel_rvv_end);
--
2.31.1

2021-09-08 17:51:02

by Greentime Hu

Subject: [RFC PATCH v8 17/21] riscv: Optimize vector registers initialization

This patch optimizes the initialization or invalidation of vector
registers. It can reduce the code sizes of vector_flush_cpu_state()
and reset_regs().

Co-developed-by: Han-Kuan Chen <[email protected]>
Signed-off-by: Han-Kuan Chen <[email protected]>
Signed-off-by: Greentime Hu <[email protected]>
---
arch/riscv/kernel/head.S | 30 +-----------------------
arch/riscv/kernel/kernel_mode_vector.c | 32 ++------------------------
2 files changed, 3 insertions(+), 59 deletions(-)

diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
index 42eb3203fa77..8362d7458c6c 100644
--- a/arch/riscv/kernel/head.S
+++ b/arch/riscv/kernel/head.S
@@ -442,39 +442,11 @@ ENTRY(reset_regs)

li t1, SR_VS
csrs CSR_STATUS, t1
- vsetvli t1, x0, e8, m1
+ vsetvli t1, x0, e8, m8
vmv.v.i v0, 0
- vmv.v.i v1, 0
- vmv.v.i v2, 0
- vmv.v.i v3, 0
- vmv.v.i v4, 0
- vmv.v.i v5, 0
- vmv.v.i v6, 0
- vmv.v.i v7, 0
vmv.v.i v8, 0
- vmv.v.i v9, 0
- vmv.v.i v10, 0
- vmv.v.i v11, 0
- vmv.v.i v12, 0
- vmv.v.i v13, 0
- vmv.v.i v14, 0
- vmv.v.i v15, 0
vmv.v.i v16, 0
- vmv.v.i v17, 0
- vmv.v.i v18, 0
- vmv.v.i v19, 0
- vmv.v.i v20, 0
- vmv.v.i v21, 0
- vmv.v.i v22, 0
- vmv.v.i v23, 0
vmv.v.i v24, 0
- vmv.v.i v25, 0
- vmv.v.i v26, 0
- vmv.v.i v27, 0
- vmv.v.i v28, 0
- vmv.v.i v29, 0
- vmv.v.i v30, 0
- vmv.v.i v31, 0
/* note that the caller must clear SR_VS */
#endif /* CONFIG_VECTOR */

diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c
index 108cfafe7496..b84618630edf 100644
--- a/arch/riscv/kernel/kernel_mode_vector.c
+++ b/arch/riscv/kernel/kernel_mode_vector.c
@@ -86,39 +86,11 @@ static void vector_flush_cpu_state(void)
long tmp;

__asm__ __volatile__ (
- "vsetvli %0, x0, e8, m1\n"
+ "vsetvli %0, x0, e8, m8\n"
"vmv.v.i v0, 0\n"
- "vmv.v.i v1, 0\n"
- "vmv.v.i v2, 0\n"
- "vmv.v.i v3, 0\n"
- "vmv.v.i v4, 0\n"
- "vmv.v.i v5, 0\n"
- "vmv.v.i v6, 0\n"
- "vmv.v.i v7, 0\n"
"vmv.v.i v8, 0\n"
- "vmv.v.i v9, 0\n"
- "vmv.v.i v10, 0\n"
- "vmv.v.i v11, 0\n"
- "vmv.v.i v12, 0\n"
- "vmv.v.i v13, 0\n"
- "vmv.v.i v14, 0\n"
- "vmv.v.i v15, 0\n"
"vmv.v.i v16, 0\n"
- "vmv.v.i v17, 0\n"
- "vmv.v.i v18, 0\n"
- "vmv.v.i v19, 0\n"
- "vmv.v.i v20, 0\n"
- "vmv.v.i v21, 0\n"
- "vmv.v.i v22, 0\n"
- "vmv.v.i v23, 0\n"
- "vmv.v.i v24, 0\n"
- "vmv.v.i v25, 0\n"
- "vmv.v.i v26, 0\n"
- "vmv.v.i v27, 0\n"
- "vmv.v.i v28, 0\n"
- "vmv.v.i v29, 0\n"
- "vmv.v.i v30, 0\n"
- "vmv.v.i v31, 0\n":"=r"(tmp)::);
+ "vmv.v.i v24, 0\n":"=r"(tmp)::);
}

/*
--
2.31.1

2021-09-08 18:15:05

by Greentime Hu

Subject: [RFC PATCH v8 14/21] riscv: Use CSR_STATUS to replace sstatus in vector.S

It should use the same logic here in both m-mode and s-mode.

Signed-off-by: Greentime Hu <[email protected]>
Co-developed-by: Vincent Chen <[email protected]>
Signed-off-by: Vincent Chen <[email protected]>
---
arch/riscv/kernel/vector.S | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/riscv/kernel/vector.S b/arch/riscv/kernel/vector.S
index 4c880b1c32aa..4f0c5a166e4e 100644
--- a/arch/riscv/kernel/vector.S
+++ b/arch/riscv/kernel/vector.S
@@ -32,7 +32,7 @@

ENTRY(__vstate_save)
li status, SR_VS
- csrs sstatus, status
+ csrs CSR_STATUS, status

csrr x_vstart, CSR_VSTART
csrr x_vtype, CSR_VTYPE
@@ -53,13 +53,13 @@ ENTRY(__vstate_save)
REG_S x_vl, RISCV_V_STATE_VL(vstatep)
REG_S x_vcsr, RISCV_V_STATE_VCSR(vstatep)

- csrc sstatus, status
+ csrc CSR_STATUS, status
ret
ENDPROC(__vstate_save)

ENTRY(__vstate_restore)
li status, SR_VS
- csrs sstatus, status
+ csrs CSR_STATUS, status

li m_one, -1
vsetvli incr, m_one, e8, m8
@@ -79,6 +79,6 @@ ENTRY(__vstate_restore)
csrw CSR_VSTART, x_vstart
csrw CSR_VCSR, x_vcsr

- csrc sstatus, status
+ csrc CSR_STATUS, status
ret
ENDPROC(__vstate_restore)
--
2.31.1

2021-09-08 18:15:09

by Greentime Hu

Subject: [RFC PATCH v8 16/21] riscv: Initialize vector registers with proper vsetvli then it can work normally

It may cause an illegal instruction exception if it doesn't use vsetvli
before vmv.v.i v0, 0.

Co-developed-by: Han-Kuan Chen <[email protected]>
Signed-off-by: Han-Kuan Chen <[email protected]>
Signed-off-by: Greentime Hu <[email protected]>
---
arch/riscv/kernel/head.S | 1 +
1 file changed, 1 insertion(+)

diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
index cf331f138142..42eb3203fa77 100644
--- a/arch/riscv/kernel/head.S
+++ b/arch/riscv/kernel/head.S
@@ -442,6 +442,7 @@ ENTRY(reset_regs)

li t1, SR_VS
csrs CSR_STATUS, t1
+ vsetvli t1, x0, e8, m1
vmv.v.i v0, 0
vmv.v.i v1, 0
vmv.v.i v2, 0
--
2.31.1

2021-09-08 18:15:31

by Greentime Hu

Subject: [RFC PATCH v8 12/21] riscv: signal: Report signal frame size to userspace via auxv

From: Vincent Chen <[email protected]>

The vector register belongs to the signal context. They need to be stored
and restored as entering and leaving the signal handler. According to the
V-extension specification, the maximum length of the vector registers can
be 2^(XLEN-1). Hence, if userspace refers to the MINSIGSTKSZ to create a
sigframe, it may not be enough. To resolve this problem, this patch refers
to the commit 94b07c1f8c39c
("arm64: signal: Report signal frame size to userspace via auxv") to enable
userspace to know the minimum required sigframe size through the auxiliary
vector and use it to allocate enough memory for signal context.

Signed-off-by: Greentime Hu <[email protected]>
Signed-off-by: Vincent Chen <[email protected]>
---
arch/riscv/include/asm/elf.h | 41 +++++++++++++++++-----------
arch/riscv/include/asm/processor.h | 2 ++
arch/riscv/include/uapi/asm/auxvec.h | 1 +
arch/riscv/kernel/signal.c | 8 ++++++
4 files changed, 36 insertions(+), 16 deletions(-)

diff --git a/arch/riscv/include/asm/elf.h b/arch/riscv/include/asm/elf.h
index f4b490cd0e5d..1102052aa593 100644
--- a/arch/riscv/include/asm/elf.h
+++ b/arch/riscv/include/asm/elf.h
@@ -58,22 +58,31 @@ extern unsigned long elf_hwcap;
#define ELF_PLATFORM (NULL)

#ifdef CONFIG_MMU
-#define ARCH_DLINFO \
-do { \
- NEW_AUX_ENT(AT_SYSINFO_EHDR, \
- (elf_addr_t)current->mm->context.vdso); \
- NEW_AUX_ENT(AT_L1I_CACHESIZE, \
- get_cache_size(1, CACHE_TYPE_INST)); \
- NEW_AUX_ENT(AT_L1I_CACHEGEOMETRY, \
- get_cache_geometry(1, CACHE_TYPE_INST)); \
- NEW_AUX_ENT(AT_L1D_CACHESIZE, \
- get_cache_size(1, CACHE_TYPE_DATA)); \
- NEW_AUX_ENT(AT_L1D_CACHEGEOMETRY, \
- get_cache_geometry(1, CACHE_TYPE_DATA)); \
- NEW_AUX_ENT(AT_L2_CACHESIZE, \
- get_cache_size(2, CACHE_TYPE_UNIFIED)); \
- NEW_AUX_ENT(AT_L2_CACHEGEOMETRY, \
- get_cache_geometry(2, CACHE_TYPE_UNIFIED)); \
+#define ARCH_DLINFO \
+do { \
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, \
+ (elf_addr_t)current->mm->context.vdso); \
+ NEW_AUX_ENT(AT_L1I_CACHESIZE, \
+ get_cache_size(1, CACHE_TYPE_INST)); \
+ NEW_AUX_ENT(AT_L1I_CACHEGEOMETRY, \
+ get_cache_geometry(1, CACHE_TYPE_INST)); \
+ NEW_AUX_ENT(AT_L1D_CACHESIZE, \
+ get_cache_size(1, CACHE_TYPE_DATA)); \
+ NEW_AUX_ENT(AT_L1D_CACHEGEOMETRY, \
+ get_cache_geometry(1, CACHE_TYPE_DATA)); \
+ NEW_AUX_ENT(AT_L2_CACHESIZE, \
+ get_cache_size(2, CACHE_TYPE_UNIFIED)); \
+ NEW_AUX_ENT(AT_L2_CACHEGEOMETRY, \
+ get_cache_geometry(2, CACHE_TYPE_UNIFIED)); \
+ /* \
+ * Should always be nonzero unless there's a kernel bug. \
+ * If we haven't determined a sensible value to give to \
+ * userspace, omit the entry: \
+ */ \
+ if (likely(signal_minsigstksz)) \
+ NEW_AUX_ENT(AT_MINSIGSTKSZ, signal_minsigstksz); \
+ else \
+ NEW_AUX_ENT(AT_IGNORE, 0); \
} while (0)
#define ARCH_HAS_SETUP_ADDITIONAL_PAGES
struct linux_binprm;
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index 1b037c69d311..62c75645c606 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -7,6 +7,7 @@
#define _ASM_RISCV_PROCESSOR_H

#include <linux/const.h>
+#include <linux/cache.h>

#include <vdso/processor.h>

@@ -74,6 +75,7 @@ int riscv_of_parent_hartid(struct device_node *node);
extern void riscv_fill_hwcap(void);
extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src);

+extern unsigned long signal_minsigstksz __ro_after_init;
#endif /* __ASSEMBLY__ */

#endif /* _ASM_RISCV_PROCESSOR_H */
diff --git a/arch/riscv/include/uapi/asm/auxvec.h b/arch/riscv/include/uapi/asm/auxvec.h
index 32c73ba1d531..6610d24e6662 100644
--- a/arch/riscv/include/uapi/asm/auxvec.h
+++ b/arch/riscv/include/uapi/asm/auxvec.h
@@ -33,5 +33,6 @@

/* entries in ARCH_DLINFO */
#define AT_VECTOR_SIZE_ARCH 7
+#define AT_MINSIGSTKSZ 51

#endif /* _UAPI_ASM_RISCV_AUXVEC_H */
diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c
index 6938cfa16b45..d30a3b588156 100644
--- a/arch/riscv/kernel/signal.c
+++ b/arch/riscv/kernel/signal.c
@@ -470,8 +470,16 @@ asmlinkage __visible void do_notify_resume(struct pt_regs *regs,
tracehook_notify_resume(regs);
}

+unsigned long __ro_after_init signal_minsigstksz;
+
void init_rt_signal_env(void);
void __init init_rt_signal_env(void)
{
rvv_sc_size = sizeof(struct __sc_riscv_v_state) + riscv_vsize;
+ /*
+ * Determine the stack space required for guaranteed signal delivery.
+ * The signal_minsigstksz will be populated into the AT_MINSIGSTKSZ entry
+ * in the auxiliary array at process startup.
+ */
+ signal_minsigstksz = cal_rt_frame_size();
}
--
2.31.1

2021-09-08 18:15:31

by Greentime Hu

Subject: [RFC PATCH v8 01/21] riscv: Separate patch for cflags and aflags

From: Guo Ren <[email protected]>

Use "subst fd" in Makefile is a hack way and it's not convenient
to add new ISA feature. Just separate them into riscv-march-cflags
and riscv-march-aflags.

Signed-off-by: Guo Ren <[email protected]>
Signed-off-by: Guo Ren <[email protected]>
Signed-off-by: Greentime Hu <[email protected]>
---
arch/riscv/Makefile | 18 ++++++++++++------
1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index bc74afdbf31e..428bd3bc202f 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -48,12 +48,18 @@ endif
endif

# ISA string setting
-riscv-march-$(CONFIG_ARCH_RV32I) := rv32ima
-riscv-march-$(CONFIG_ARCH_RV64I) := rv64ima
-riscv-march-$(CONFIG_FPU) := $(riscv-march-y)fd
-riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c
-KBUILD_CFLAGS += -march=$(subst fd,,$(riscv-march-y))
-KBUILD_AFLAGS += -march=$(riscv-march-y)
+riscv-march-cflags-$(CONFIG_ARCH_RV32I) := rv32ima
+riscv-march-cflags-$(CONFIG_ARCH_RV64I) := rv64ima
+riscv-march-$(CONFIG_FPU) := $(riscv-march-y)fd
+riscv-march-cflags-$(CONFIG_RISCV_ISA_C) := $(riscv-march-cflags-y)c
+
+riscv-march-aflags-$(CONFIG_ARCH_RV32I) := rv32ima
+riscv-march-aflags-$(CONFIG_ARCH_RV64I) := rv64ima
+riscv-march-aflags-$(CONFIG_FPU) := $(riscv-march-aflags-y)fd
+riscv-march-aflags-$(CONFIG_RISCV_ISA_C) := $(riscv-march-aflags-y)c
+
+KBUILD_CFLAGS += -march=$(riscv-march-cflags-y)
+KBUILD_AFLAGS += -march=$(riscv-march-aflags-y)

KBUILD_CFLAGS += -mno-save-restore
KBUILD_CFLAGS += -DCONFIG_PAGE_OFFSET=$(CONFIG_PAGE_OFFSET)
--
2.31.1

2021-09-08 18:15:32

by Greentime Hu

Subject: [RFC PATCH v8 06/21] riscv: Add has_vector/riscv_vsize to save vector features.

This patch is used to detect vector support status of CPU and use
riscv_vsize to save the size of all the vector registers. It assumes
all harts has the same capabilities in SMP system.

[[email protected]: add has_vector checking]
Signed-off-by: Greentime Hu <[email protected]>
Co-developed-by: Guo Ren <[email protected]>
Signed-off-by: Guo Ren <[email protected]>
Co-developed-by: Vincent Chen <[email protected]>
Signed-off-by: Vincent Chen <[email protected]>
---
arch/riscv/kernel/cpufeature.c | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index 7069e55335d0..7265d947d981 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -21,6 +21,10 @@ static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly;
#ifdef CONFIG_FPU
__ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_fpu);
#endif
+#ifdef CONFIG_VECTOR
+bool has_vector __read_mostly;
+unsigned long riscv_vsize __read_mostly;
+#endif

/**
* riscv_isa_extension_base() - Get base extension word
@@ -149,4 +153,12 @@ void __init riscv_fill_hwcap(void)
if (elf_hwcap & (COMPAT_HWCAP_ISA_F | COMPAT_HWCAP_ISA_D))
static_branch_enable(&cpu_hwcap_fpu);
#endif
+
+#ifdef CONFIG_VECTOR
+ if (elf_hwcap & COMPAT_HWCAP_ISA_V) {
+ has_vector = true;
+ /* There are 32 vector registers with vlenb length. */
+ riscv_vsize = csr_read(CSR_VLENB) * 32;
+ }
+#endif
}
--
2.31.1

2021-09-08 18:15:32

by Greentime Hu

Subject: [RFC PATCH v8 10/21] riscv: Add ptrace vector support

This patch adds ptrace support for riscv vector. The vector registers will
be saved in datap pointer of __riscv_v_state. This pointer will be set
right after the __riscv_v_state data structure then it will be put in ubuf
for ptrace system call to get or set. It will check if the datap got from
ubuf is set to the correct address or not when the ptrace system call is
trying to set the vector registers.

Co-developed-by: Vincent Chen <[email protected]>
Signed-off-by: Vincent Chen <[email protected]>
Signed-off-by: Greentime Hu <[email protected]>
---
arch/riscv/include/uapi/asm/ptrace.h | 14 ++++++
arch/riscv/kernel/ptrace.c | 71 ++++++++++++++++++++++++++++
include/uapi/linux/elf.h | 1 +
3 files changed, 86 insertions(+)

diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h
index bd3b8a710246..c3760395236c 100644
--- a/arch/riscv/include/uapi/asm/ptrace.h
+++ b/arch/riscv/include/uapi/asm/ptrace.h
@@ -83,11 +83,25 @@ struct __riscv_v_state {
unsigned long vtype;
unsigned long vcsr;
void *datap;
+ /*
+ * In signal handler, datap will be set a correct user stack offset
+ * and vector registers will be copied to the address of datap
+ * pointer.
+ *
+ * In ptrace syscall, datap will be set to zero and the vector
+ * registers will be copied to the address right after this
+ * structure.
+ */
#if __riscv_xlen == 32
__u32 __padding;
#endif
};

+/*
+ * To define a practical maximum vlenb for ptrace and it may need to be
+ * extended someday.
+ */
+#define RISCV_MAX_VLENB (16384)
#endif /* __ASSEMBLY__ */

#endif /* _UAPI_ASM_RISCV_PTRACE_H */
diff --git a/arch/riscv/kernel/ptrace.c b/arch/riscv/kernel/ptrace.c
index 9c0511119bad..0bc11a70090c 100644
--- a/arch/riscv/kernel/ptrace.c
+++ b/arch/riscv/kernel/ptrace.c
@@ -27,6 +27,9 @@ enum riscv_regset {
#ifdef CONFIG_FPU
REGSET_F,
#endif
+#ifdef CONFIG_VECTOR
+ REGSET_V,
+#endif
};

static int riscv_gpr_get(struct task_struct *target,
@@ -85,6 +88,64 @@ static int riscv_fpr_set(struct task_struct *target,
}
#endif

+#ifdef CONFIG_VECTOR
+static int riscv_vr_get(struct task_struct *target,
+ const struct user_regset *regset,
+ struct membuf to)
+{
+ struct __riscv_v_state *vstate = &target->thread.vstate;
+
+ /*
+ * Ensure the vector registers have been saved to the memory before
+ * copying them to membuf.
+ */
+ if (target == current)
+ vstate_save(current, task_pt_regs(current));
+
+ /* Copy vector header from vstate. */
+ membuf_write(&to, vstate, RISCV_V_STATE_DATAP);
+ membuf_zero(&to, sizeof(void *));
+#if __riscv_xlen == 32
+ membuf_zero(&to, sizeof(__u32));
+#endif
+
+ /* Copy all the vector registers from vstate. */
+ return membuf_write(&to, vstate->datap, riscv_vsize);
+}
+
+static int riscv_vr_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret, size;
+ struct __riscv_v_state *vstate = &target->thread.vstate;
+
+ /* Copy rest of the vstate except datap and __padding. */
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, vstate, 0,
+ RISCV_V_STATE_DATAP);
+ if (unlikely(ret))
+ return ret;
+
+ /* Skip copy datap. */
+ size = sizeof(vstate->datap);
+ count -= size;
+ ubuf += size;
+#if __riscv_xlen == 32
+ /* Skip copy _padding. */
+ size = sizeof(vstate->__padding);
+ count -= size;
+ ubuf += size;
+#endif
+
+ /* Copy all the vector registers. */
+ pos = 0;
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, vstate->datap,
+ 0, riscv_vsize);
+ return ret;
+}
+#endif
+
static const struct user_regset riscv_user_regset[] = {
[REGSET_X] = {
.core_note_type = NT_PRSTATUS,
@@ -104,6 +165,16 @@ static const struct user_regset riscv_user_regset[] = {
.set = riscv_fpr_set,
},
#endif
+#ifdef CONFIG_VECTOR
+ [REGSET_V] = {
+ .core_note_type = NT_RISCV_VECTOR,
+ .align = 16,
+ .n = (32 * RISCV_MAX_VLENB)/sizeof(__u32),
+ .size = sizeof(__u32),
+ .regset_get = riscv_vr_get,
+ .set = riscv_vr_set,
+ },
+#endif
};

static const struct user_regset_view riscv_user_native_view = {
diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
index 61bf4774b8f2..60c5b873a8f6 100644
--- a/include/uapi/linux/elf.h
+++ b/include/uapi/linux/elf.h
@@ -432,6 +432,7 @@ typedef struct elf64_shdr {
#define NT_MIPS_DSP 0x800 /* MIPS DSP ASE registers */
#define NT_MIPS_FP_MODE 0x801 /* MIPS floating-point mode */
#define NT_MIPS_MSA 0x802 /* MIPS SIMD registers */
+#define NT_RISCV_VECTOR 0x900 /* RISC-V vector registers */

/* Note types with note name "GNU" */
#define NT_GNU_PROPERTY_TYPE_0 5
--
2.31.1

2021-09-08 18:15:32

by Greentime Hu

Subject: [RFC PATCH v8 04/21] riscv: Add new csr defines related to vector extension

Follow the riscv vector spec to add new csr numbers.

[[email protected]: first porting for new vector related csr]
Signed-off-by: Greentime Hu <[email protected]>
Acked-by: Guo Ren <[email protected]>
Co-developed-by: Guo Ren <[email protected]>
Signed-off-by: Guo Ren <[email protected]>
Co-developed-by: Vincent Chen <[email protected]>
Signed-off-by: Vincent Chen <[email protected]>
---
arch/riscv/include/asm/csr.h | 16 ++++++++++++++--
1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h
index 87ac65696871..069743102fac 100644
--- a/arch/riscv/include/asm/csr.h
+++ b/arch/riscv/include/asm/csr.h
@@ -24,6 +24,12 @@
#define SR_FS_CLEAN _AC(0x00004000, UL)
#define SR_FS_DIRTY _AC(0x00006000, UL)

+#define SR_VS _AC(0x00000600, UL) /* Vector Status */
+#define SR_VS_OFF _AC(0x00000000, UL)
+#define SR_VS_INITIAL _AC(0x00000200, UL)
+#define SR_VS_CLEAN _AC(0x00000400, UL)
+#define SR_VS_DIRTY _AC(0x00000600, UL)
+
#define SR_XS _AC(0x00018000, UL) /* Extension Status */
#define SR_XS_OFF _AC(0x00000000, UL)
#define SR_XS_INITIAL _AC(0x00008000, UL)
@@ -31,9 +37,9 @@
#define SR_XS_DIRTY _AC(0x00018000, UL)

#ifndef CONFIG_64BIT
-#define SR_SD _AC(0x80000000, UL) /* FS/XS dirty */
+#define SR_SD _AC(0x80000000, UL) /* FS/VS/XS dirty */
#else
-#define SR_SD _AC(0x8000000000000000, UL) /* FS/XS dirty */
+#define SR_SD _AC(0x8000000000000000, UL) /* FS/VS/XS dirty */
#endif

/* SATP flags */
@@ -120,6 +126,12 @@
#define CSR_MIMPID 0xf13
#define CSR_MHARTID 0xf14

+#define CSR_VSTART 0x8
+#define CSR_VCSR 0xf
+#define CSR_VL 0xc20
+#define CSR_VTYPE 0xc21
+#define CSR_VLENB 0xc22
+
#ifdef CONFIG_RISCV_M_MODE
# define CSR_STATUS CSR_MSTATUS
# define CSR_IE CSR_MIE
--
2.31.1

2021-09-08 18:15:32

by Greentime Hu

Subject: [RFC PATCH v8 08/21] riscv: Add vector struct and assembler definitions

Add vector state context struct in struct thread and asm-offsets.c
definitions.

The vector registers will be saved in datap pointer of __riscv_v_state. It
will be dynamically allocated in kernel space. It will be put right after
the __riscv_v_state data structure in user space.

Co-developed-by: Vincent Chen <[email protected]>
Signed-off-by: Vincent Chen <[email protected]>
Signed-off-by: Greentime Hu <[email protected]>
---
arch/riscv/include/asm/processor.h | 1 +
arch/riscv/include/uapi/asm/ptrace.h | 11 +++++++++++
arch/riscv/kernel/asm-offsets.c | 6 ++++++
3 files changed, 18 insertions(+)

diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index 021ed64ee608..1b037c69d311 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -35,6 +35,7 @@ struct thread_struct {
unsigned long s[12]; /* s[0]: frame pointer */
struct __riscv_d_ext_state fstate;
unsigned long bad_cause;
+ struct __riscv_v_state vstate;
};

#define INIT_THREAD { \
diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h
index 882547f6bd5c..bd3b8a710246 100644
--- a/arch/riscv/include/uapi/asm/ptrace.h
+++ b/arch/riscv/include/uapi/asm/ptrace.h
@@ -77,6 +77,17 @@ union __riscv_fp_state {
struct __riscv_q_ext_state q;
};

+struct __riscv_v_state {
+ unsigned long vstart;
+ unsigned long vl;
+ unsigned long vtype;
+ unsigned long vcsr;
+ void *datap;
+#if __riscv_xlen == 32
+ __u32 __padding;
+#endif
+};
+
#endif /* __ASSEMBLY__ */

#endif /* _UAPI_ASM_RISCV_PTRACE_H */
diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c
index 90f8ce64fa6f..34f43c84723a 100644
--- a/arch/riscv/kernel/asm-offsets.c
+++ b/arch/riscv/kernel/asm-offsets.c
@@ -72,6 +72,12 @@ void asm_offsets(void)
OFFSET(TSK_STACK_CANARY, task_struct, stack_canary);
#endif

+ OFFSET(RISCV_V_STATE_VSTART, __riscv_v_state, vstart);
+ OFFSET(RISCV_V_STATE_VL, __riscv_v_state, vl);
+ OFFSET(RISCV_V_STATE_VTYPE, __riscv_v_state, vtype);
+ OFFSET(RISCV_V_STATE_VCSR, __riscv_v_state, vcsr);
+ OFFSET(RISCV_V_STATE_DATAP, __riscv_v_state, datap);
+
DEFINE(PT_SIZE, sizeof(struct pt_regs));
OFFSET(PT_EPC, pt_regs, epc);
OFFSET(PT_RA, pt_regs, ra);
--
2.31.1

2021-09-08 19:03:11

by Greentime Hu

Subject: [RFC PATCH v8 18/21] riscv: Fix an illegal instruction exception when accessing vlenb without enable vector first

It triggered an illegal instruction exception when accessing vlenb CSR
without enable vector first. To fix this issue, we should enable vector
before using it and disable vector after using it.

Co-developed-by: Vincent Chen <[email protected]>
Signed-off-by: Vincent Chen <[email protected]>
Signed-off-by: Greentime Hu <[email protected]>
---
arch/riscv/include/asm/vector.h | 2 ++
arch/riscv/kernel/cpufeature.c | 3 +++
arch/riscv/kernel/kernel_mode_vector.c | 6 ++++--
3 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h
index 5d7f14453f68..ca063c8f47f2 100644
--- a/arch/riscv/include/asm/vector.h
+++ b/arch/riscv/include/asm/vector.h
@@ -8,6 +8,8 @@

#include <linux/types.h>

+void rvv_enable(void);
+void rvv_disable(void);
void kernel_rvv_begin(void);
void kernel_rvv_end(void);

diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index 7265d947d981..af984f875f60 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -22,6 +22,7 @@ static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly;
__ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_fpu);
#endif
#ifdef CONFIG_VECTOR
+#include <asm/vector.h>
bool has_vector __read_mostly;
unsigned long riscv_vsize __read_mostly;
#endif
@@ -158,7 +159,9 @@ void __init riscv_fill_hwcap(void)
if (elf_hwcap & COMPAT_HWCAP_ISA_V) {
has_vector = true;
/* There are 32 vector registers with vlenb length. */
+ rvv_enable();
riscv_vsize = csr_read(CSR_VLENB) * 32;
+ rvv_disable();
}
#endif
}
diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c
index b84618630edf..0d990bd8b8dd 100644
--- a/arch/riscv/kernel/kernel_mode_vector.c
+++ b/arch/riscv/kernel/kernel_mode_vector.c
@@ -71,15 +71,17 @@ static void put_cpu_vector_context(void)
preempt_enable();
}

-static void rvv_enable(void)
+void rvv_enable(void)
{
csr_set(CSR_STATUS, SR_VS);
}
+EXPORT_SYMBOL(rvv_enable);

-static void rvv_disable(void)
+void rvv_disable(void)
{
csr_clear(CSR_STATUS, SR_VS);
}
+EXPORT_SYMBOL(rvv_disable);

static void vector_flush_cpu_state(void)
{
--
2.31.1

2021-09-08 19:03:14

by Greentime Hu

Subject: [RFC PATCH v8 05/21] riscv: Add vector feature to compile

From: Guo Ren <[email protected]>

This patch adds a new config option which could enable assembler's
vector feature.

Signed-off-by: Greentime Hu <[email protected]>
Signed-off-by: Guo Ren <[email protected]>
Reviewed-by: Greentime Hu <[email protected]>
---
arch/riscv/Kconfig | 9 +++++++++
arch/riscv/Makefile | 1 +
2 files changed, 10 insertions(+)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 4f7b70ae7c31..619cfc370ee5 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -380,6 +380,15 @@ config FPU

If you don't know what to do here, say Y.

+config VECTOR
+ bool "VECTOR support"
+ default n
+ help
+ Say N here if you want to disable all vector related procedure
+ in the kernel.
+
+ If you don't know what to do here, say Y.
+
endmenu

menu "Kernel features"
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 428bd3bc202f..1450bdde5288 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -57,6 +57,7 @@ riscv-march-aflags-$(CONFIG_ARCH_RV32I) := rv32ima
riscv-march-aflags-$(CONFIG_ARCH_RV64I) := rv64ima
riscv-march-aflags-$(CONFIG_FPU) := $(riscv-march-aflags-y)fd
riscv-march-aflags-$(CONFIG_RISCV_ISA_C) := $(riscv-march-aflags-y)c
+riscv-march-aflags-$(CONFIG_VECTOR) := $(riscv-march-aflags-y)v

KBUILD_CFLAGS += -march=$(riscv-march-cflags-y)
KBUILD_AFLAGS += -march=$(riscv-march-aflags-y)
--
2.31.1

2021-09-08 19:03:15

by Greentime Hu

Subject: [RFC PATCH v8 07/21] riscv: Reset vector register

From: Guo Ren <[email protected]>

Reset vector registers at boot-time and disable vector instructions
execution for kernel mode.

[[email protected]: add comments]
Signed-off-by: Guo Ren <[email protected]>
Co-developed-by: Vincent Chen <[email protected]>
Signed-off-by: Vincent Chen <[email protected]>
Co-developed-by: Greentime Hu <[email protected]>
Signed-off-by: Greentime Hu <[email protected]>
---
arch/riscv/kernel/entry.S | 6 ++---
arch/riscv/kernel/head.S | 49 +++++++++++++++++++++++++++++++++++++--
2 files changed, 50 insertions(+), 5 deletions(-)

diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 98f502654edd..ad0fa80ada81 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -77,10 +77,10 @@ _save_context:
* Disable user-mode memory access as it should only be set in the
* actual user copy routines.
*
- * Disable the FPU to detect illegal usage of floating point in kernel
- * space.
+ * Disable the FPU/Vector to detect illegal usage of floating point
+ * or vector in kernel space.
*/
- li t0, SR_SUM | SR_FS
+ li t0, SR_SUM | SR_FS | SR_VS

REG_L s0, TASK_TI_USER_SP(tp)
csrrc s1, CSR_STATUS, t0
diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
index fce5184b22c3..cf331f138142 100644
--- a/arch/riscv/kernel/head.S
+++ b/arch/riscv/kernel/head.S
@@ -241,10 +241,10 @@ pmp_done:
.option pop

/*
- * Disable FPU to detect illegal usage of
+ * Disable FPU & VECTOR to detect illegal usage of
* floating point in kernel space
*/
- li t0, SR_FS
+ li t0, SR_FS | SR_VS
csrc CSR_STATUS, t0

#ifdef CONFIG_SMP
@@ -432,6 +432,51 @@ ENTRY(reset_regs)
csrw fcsr, 0
/* note that the caller must clear SR_FS */
#endif /* CONFIG_FPU */
+
+#ifdef CONFIG_VECTOR
+ csrr t0, CSR_MISA
+ li t1, (COMPAT_HWCAP_ISA_V >> 16)
+ slli t1, t1, 16
+ and t0, t0, t1
+ beqz t0, .Lreset_regs_done
+
+ li t1, SR_VS
+ csrs CSR_STATUS, t1
+ vmv.v.i v0, 0
+ vmv.v.i v1, 0
+ vmv.v.i v2, 0
+ vmv.v.i v3, 0
+ vmv.v.i v4, 0
+ vmv.v.i v5, 0
+ vmv.v.i v6, 0
+ vmv.v.i v7, 0
+ vmv.v.i v8, 0
+ vmv.v.i v9, 0
+ vmv.v.i v10, 0
+ vmv.v.i v11, 0
+ vmv.v.i v12, 0
+ vmv.v.i v13, 0
+ vmv.v.i v14, 0
+ vmv.v.i v15, 0
+ vmv.v.i v16, 0
+ vmv.v.i v17, 0
+ vmv.v.i v18, 0
+ vmv.v.i v19, 0
+ vmv.v.i v20, 0
+ vmv.v.i v21, 0
+ vmv.v.i v22, 0
+ vmv.v.i v23, 0
+ vmv.v.i v24, 0
+ vmv.v.i v25, 0
+ vmv.v.i v26, 0
+ vmv.v.i v27, 0
+ vmv.v.i v28, 0
+ vmv.v.i v29, 0
+ vmv.v.i v30, 0
+ vmv.v.i v31, 0
+ /* note that the caller must clear SR_VS */
+#endif /* CONFIG_VECTOR */
+
.Lreset_regs_done:
ret
END(reset_regs)
--
2.31.1

2021-09-08 19:03:32

by Greentime Hu

Subject: [RFC PATCH v8 19/21] riscv: Allocate space for vector registers in start_thread()

It allocates memory space for vector registers in start_thread() instead of
allocating in vstate_restore() in this patch. We can allocate memory here
so that it will be more readable.

Co-developed-by: Vincent Chen <[email protected]>
Signed-off-by: Vincent Chen <[email protected]>
Signed-off-by: Greentime Hu <[email protected]>
---
arch/riscv/include/asm/switch_to.h | 7 +------
arch/riscv/kernel/process.c | 15 +++++++++++++--
2 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h
index de0573dad78f..b48c9c974564 100644
--- a/arch/riscv/include/asm/switch_to.h
+++ b/arch/riscv/include/asm/switch_to.h
@@ -103,12 +103,6 @@ static inline void vstate_restore(struct task_struct *task,
{
if ((regs->status & SR_VS) != SR_VS_OFF) {
struct __riscv_v_state *vstate = &(task->thread.vstate);
-
- /* Allocate space for vector registers. */
- if (!vstate->datap) {
- vstate->datap = kzalloc(riscv_vsize, GFP_ATOMIC);
- vstate->size = riscv_vsize;
- }
__vstate_restore(vstate, vstate->datap);
__vstate_clean(regs);
}
@@ -127,6 +121,7 @@ static inline void __switch_to_vector(struct task_struct *prev,

#else
#define has_vector false
+#define riscv_vsize (0)
#define vstate_save(task, regs) do { } while (0)
#define vstate_restore(task, regs) do { } while (0)
#define __switch_to_vector(__prev, __next) do { } while (0)
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index 0b86e9e531c9..05ff5f934e7e 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -97,7 +97,16 @@ void start_thread(struct pt_regs *regs, unsigned long pc,
}

if (has_vector) {
+ struct __riscv_v_state *vstate = &(current->thread.vstate);
+
+ /* Enable vector and allocate memory for vector registers. */
+ if (!vstate->datap) {
+ vstate->datap = kzalloc(riscv_vsize, GFP_KERNEL);
+ if (WARN_ON(!vstate->datap))
+ return;
+ }
regs->status |= SR_VS_INITIAL;
+
/*
* Restore the initial value to the vector register
* before starting the user program.
@@ -121,9 +130,11 @@ void flush_thread(void)
memset(&current->thread.fstate, 0, sizeof(current->thread.fstate));
#endif
#ifdef CONFIG_VECTOR
- /* Reset vector state */
+ /* Reset vector state and keep datap pointer. */
vstate_off(current, task_pt_regs(current));
- memset(&current->thread.vstate, 0, sizeof(current->thread.vstate));
+ memset(&current->thread.vstate, 0, RISCV_V_STATE_DATAP);
+ if (current->thread.vstate.datap)
+ memset(current->thread.vstate.datap, 0, riscv_vsize);
#endif
}

--
2.31.1

2021-09-09 06:19:20

by Christoph Hellwig

Subject: Re: [RFC PATCH v8 13/21] riscv: Add support for kernel mode vector

On Thu, Sep 09, 2021 at 01:45:25AM +0800, Greentime Hu wrote:
> +obj-$(CONFIG_VECTOR) += kernel_mode_vector.o
> +riscv-march-cflags-$(CONFIG_ARCH_RV32I) := rv32ima
> +riscv-march-cflags-$(CONFIG_ARCH_RV64I) := rv64ima
> +riscv-march-cflags-$(CONFIG_RISCV_ISA_C) := $(riscv-march-cflags-y)c
> +riscv-march-cflags-$(CONFIG_VECTOR) := $(riscv-march-cflags-y)v
> +CFLAGS_kernel_mode_vector.o += -march=$(riscv-march-cflags-y)

Do we need a helper in arch/riscv/Makefile to define the vector flags
instead of open coding them where used? Also I think the variable
name should include vector in it.


> +EXPORT_SYMBOL(kernel_rvv_begin);

> +EXPORT_SYMBOL(kernel_rvv_end);

This needs to be EXPORT_SYMBOL_GPL just like x86 kernel_fpu_begin/end

2021-09-13 01:57:53

by Vincent Chen

Subject: Re: [RFC PATCH v8 00/21] riscv: Add vector ISA support

Hi all,

The associated Glibc vector patches could be found here
https://sourceware.org/pipermail/libc-alpha/2021-September/130897.html
Thanks

On Thu, Sep 9, 2021 at 1:45 AM Greentime Hu <[email protected]> wrote:
>
> This patchset is implemented based on vector 1.0-rc1 spec to add vector
> support in riscv Linux kernel. To make this happen, we defined a new
> structure __riscv_v_state to save the vector related registers. It is used
> for both kernel space and user space.
>
> - In kernel space, the datap pointer in __riscv_v_state will be allocated
> dynamically to save vector registers.
> - In user space,
> - In signal handler of user space, datap will point to the address
> of the __riscv_v_state data structure to save vector
> registers in stack. We also create a __reserved[] array for
> future extensions.
> - In ptrace, the data will be put in ubuf in which we use
> riscv_vr_get()/riscv_vr_set() to get or set the
> __riscv_v_state data structure from/to it, datap pointer
> would be zeroed and vector registers will be copied to the
> address right after the __riscv_v_state structure in ubuf.
>
> This patchset also adds support for kernel mode vector, kernel XOR
> implementation with vector ISA and includes several bug fixes and code
> refinement.
>
> This patchset is rebased to v5.14 and it is tested by running several
> vector programs simultaneously. It also can get the correct ucontext_t in
> signal handler and restore correct context after sigreturn. It is also
> tested with ptrace() syscall to use PTRACE_GETREGSET/PTRACE_SETREGSET to
> get/set the vector registers. I have tested vlen=128 and vlen=256 cases in
> qemu-system-riscv64 provided by Frank Chang.
>
> We have sent patches to glibc mailing list for ifunc support and sigcontext
> changes. We will send patches for vector support to glibc mailing list
> recently.
>
> [1] https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc
>
> ---
> Changelog V8
> - Rebase to v5.14
> - Refine struct __riscv_v_state with struct __riscv_ctx_hdr
> - Refine has_vector into a static key
> - Defined __reserved space in struct sigcontext for vector and future extensions
>
> Changelog V7
> - Add support for kernel mode vector
> - Add vector extension XOR implementation
> - Optimize task switch codes of vector
> - Allocate space for vector registers in start_thread()
> - Fix an illegal instruction exception when accessing vlenb
> - Optimize vector registers initialization
> - Initialize vector registers with proper vsetvli then it can work normally
> - Refine ptrace porting due to generic API changed
> - Code clean up
>
> Changelog V6
> - Replace vle.v/vse.v instructions with vle8.v/vse8.v based on 0.9 spec
> - Add comments based on mailinglist feedback
> - Fix rv32 build error
>
> Changelog V5
> - Using regset_size() correctly in generic ptrace
> - Fix the ptrace porting
> - Fix compile warning
>
> Changelog V4
> - Support dynamic vlen
> - Fix bugs: lazy save/resotre, not saving vtype
> - Update VS bit offset based on latest vector spec
> - Add new vector csr based on latest vector spec
> - Code refine and removed unused macros
>
> Changelog V3
> - Rebase linux-5.6-rc3 and tested with qemu
> - Seperate patches with Anup's advice
> - Give out a ABI puzzle with unlimited vlen
>
> Changelog V2
> - Fixup typo "vecotr, fstate_save->vstate_save".
> - Fixup wrong saved registers' length in vector.S.
> - Seperate unrelated patches from this one.
>
> Greentime Hu (15):
> riscv: Add new csr defines related to vector extension
> riscv: Add has_vector/riscv_vsize to save vector features.
> riscv: Add vector struct and assembler definitions
> riscv: Add task switch support for vector
> riscv: Add ptrace vector support
> riscv: Add sigcontext save/restore for vector
> riscv: Add support for kernel mode vector
> riscv: Use CSR_STATUS to replace sstatus in vector.S
> riscv: Add vector extension XOR implementation
> riscv: Initialize vector registers with proper vsetvli then it can
> work normally
> riscv: Optimize vector registers initialization
> riscv: Fix an illegal instruction exception when accessing vlenb
> without enable vector first
> riscv: Allocate space for vector registers in start_thread()
> riscv: Optimize task switch codes of vector
> riscv: Turn has_vector into a static key if VECTOR=y
>
> Guo Ren (5):
> riscv: Separate patch for cflags and aflags
> riscv: Rename __switch_to_aux -> fpu
> riscv: Extending cpufeature.c to detect V-extension
> riscv: Add vector feature to compile
> riscv: Reset vector register
>
> Vincent Chen (1):
> riscv: signal: Report signal frame size to userspace via auxv
>
> arch/riscv/Kconfig | 9 ++
> arch/riscv/Makefile | 19 ++-
> arch/riscv/include/asm/csr.h | 16 ++-
> arch/riscv/include/asm/elf.h | 41 +++---
> arch/riscv/include/asm/processor.h | 3 +
> arch/riscv/include/asm/switch_to.h | 71 +++++++++-
> arch/riscv/include/asm/vector.h | 16 +++
> arch/riscv/include/asm/xor.h | 74 ++++++++++
> arch/riscv/include/uapi/asm/auxvec.h | 1 +
> arch/riscv/include/uapi/asm/hwcap.h | 1 +
> arch/riscv/include/uapi/asm/ptrace.h | 25 ++++
> arch/riscv/include/uapi/asm/sigcontext.h | 24 ++++
> arch/riscv/kernel/Makefile | 7 +
> arch/riscv/kernel/asm-offsets.c | 8 ++
> arch/riscv/kernel/cpufeature.c | 16 +++
> arch/riscv/kernel/entry.S | 6 +-
> arch/riscv/kernel/head.S | 22 ++-
> arch/riscv/kernel/kernel_mode_vector.c | 158 +++++++++++++++++++++
> arch/riscv/kernel/process.c | 49 +++++++
> arch/riscv/kernel/ptrace.c | 71 ++++++++++
> arch/riscv/kernel/setup.c | 4 +
> arch/riscv/kernel/signal.c | 172 ++++++++++++++++++++++-
> arch/riscv/kernel/vector.S | 81 +++++++++++
> arch/riscv/lib/Makefile | 1 +
> arch/riscv/lib/xor.S | 81 +++++++++++
> include/uapi/linux/elf.h | 1 +
> 26 files changed, 941 insertions(+), 36 deletions(-)
> create mode 100644 arch/riscv/include/asm/vector.h
> create mode 100644 arch/riscv/include/asm/xor.h
> create mode 100644 arch/riscv/kernel/kernel_mode_vector.c
> create mode 100644 arch/riscv/kernel/vector.S
> create mode 100644 arch/riscv/lib/xor.S
>
> --
> 2.31.1
>

2021-09-13 17:22:05

by Vineet Gupta

Subject: Re: [RFC PATCH v8 00/21] riscv: Add vector ISA support

On 9/8/21 10:45 AM, Greentime Hu wrote:
> This patchset is implemented based on vector 1.0-rc1 spec to add vector
> support in riscv Linux kernel. To make this happen, we defined a new
> structure __riscv_v_state to save the vector related registers. It is used
> for both kernel space and user space.
>
> - In kernel space, the datap pointer in __riscv_v_state will be allocated
> dynamically to save vector registers.
> - In user space,
> - In signal handler of user space, datap will point to the address
> of the __riscv_v_state data structure to save vector
> registers in stack. We also create a __reserved[] array for
> future extensions.
> - In ptrace, the data will be put in ubuf in which we use
> riscv_vr_get()/riscv_vr_set() to get or set the
> __riscv_v_state data structure from/to it, datap pointer
> would be zeroed and vector registers will be copied to the
> address right after the __riscv_v_state structure in ubuf.
>
> This patchset also adds support for kernel mode vector, kernel XOR
> implementation with vector ISA and includes several bug fixes and code
> refinement.
>
> This patchset is rebased to v5.14 and it is tested by running several
> vector programs simultaneously. It also can get the correct ucontext_t in
> signal handler and restore correct context after sigreturn. It is also
> tested with ptrace() syscall to use PTRACE_GETREGSET/PTRACE_SETREGSET to
> get/set the vector registers. I have tested vlen=128 and vlen=256 cases in
> qemu-system-riscv64 provided by Frank Chang.

Are QEMU/Spike changes available somewhere publicly for people to try this ?

Thx,
-Vineet

>
> We have sent patches to glibc mailing list for ifunc support and sigcontext
> changes. We will send patches for vector support to glibc mailing list
> recently.
>
> [1] https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc
>
> ---
> Changelog V8
> - Rebase to v5.14
> - Refine struct __riscv_v_state with struct __riscv_ctx_hdr
> - Refine has_vector into a static key
> - Defined __reserved space in struct sigcontext for vector and future extensions
>
> Changelog V7
> - Add support for kernel mode vector
> - Add vector extension XOR implementation
> - Optimize task switch codes of vector
> - Allocate space for vector registers in start_thread()
> - Fix an illegal instruction exception when accessing vlenb
> - Optimize vector registers initialization
> - Initialize vector registers with proper vsetvli then it can work normally
> - Refine ptrace porting due to generic API changed
> - Code clean up
>
> Changelog V6
> - Replace vle.v/vse.v instructions with vle8.v/vse8.v based on 0.9 spec
> - Add comments based on mailinglist feedback
> - Fix rv32 build error
>
> Changelog V5
> - Using regset_size() correctly in generic ptrace
> - Fix the ptrace porting
> - Fix compile warning
>
> Changelog V4
> - Support dynamic vlen
> - Fix bugs: lazy save/resotre, not saving vtype
> - Update VS bit offset based on latest vector spec
> - Add new vector csr based on latest vector spec
> - Code refine and removed unused macros
>
> Changelog V3
> - Rebase linux-5.6-rc3 and tested with qemu
> - Seperate patches with Anup's advice
> - Give out a ABI puzzle with unlimited vlen
>
> Changelog V2
> - Fixup typo "vecotr, fstate_save->vstate_save".
> - Fixup wrong saved registers' length in vector.S.
> - Seperate unrelated patches from this one.
>
> Greentime Hu (15):
> riscv: Add new csr defines related to vector extension
> riscv: Add has_vector/riscv_vsize to save vector features.
> riscv: Add vector struct and assembler definitions
> riscv: Add task switch support for vector
> riscv: Add ptrace vector support
> riscv: Add sigcontext save/restore for vector
> riscv: Add support for kernel mode vector
> riscv: Use CSR_STATUS to replace sstatus in vector.S
> riscv: Add vector extension XOR implementation
> riscv: Initialize vector registers with proper vsetvli then it can
> work normally
> riscv: Optimize vector registers initialization
> riscv: Fix an illegal instruction exception when accessing vlenb
> without enable vector first
> riscv: Allocate space for vector registers in start_thread()
> riscv: Optimize task switch codes of vector
> riscv: Turn has_vector into a static key if VECTOR=y
>
> Guo Ren (5):
> riscv: Separate patch for cflags and aflags
> riscv: Rename __switch_to_aux -> fpu
> riscv: Extending cpufeature.c to detect V-extension
> riscv: Add vector feature to compile
> riscv: Reset vector register
>
> Vincent Chen (1):
> riscv: signal: Report signal frame size to userspace via auxv
>
> arch/riscv/Kconfig | 9 ++
> arch/riscv/Makefile | 19 ++-
> arch/riscv/include/asm/csr.h | 16 ++-
> arch/riscv/include/asm/elf.h | 41 +++---
> arch/riscv/include/asm/processor.h | 3 +
> arch/riscv/include/asm/switch_to.h | 71 +++++++++-
> arch/riscv/include/asm/vector.h | 16 +++
> arch/riscv/include/asm/xor.h | 74 ++++++++++
> arch/riscv/include/uapi/asm/auxvec.h | 1 +
> arch/riscv/include/uapi/asm/hwcap.h | 1 +
> arch/riscv/include/uapi/asm/ptrace.h | 25 ++++
> arch/riscv/include/uapi/asm/sigcontext.h | 24 ++++
> arch/riscv/kernel/Makefile | 7 +
> arch/riscv/kernel/asm-offsets.c | 8 ++
> arch/riscv/kernel/cpufeature.c | 16 +++
> arch/riscv/kernel/entry.S | 6 +-
> arch/riscv/kernel/head.S | 22 ++-
> arch/riscv/kernel/kernel_mode_vector.c | 158 +++++++++++++++++++++
> arch/riscv/kernel/process.c | 49 +++++++
> arch/riscv/kernel/ptrace.c | 71 ++++++++++
> arch/riscv/kernel/setup.c | 4 +
> arch/riscv/kernel/signal.c | 172 ++++++++++++++++++++++-
> arch/riscv/kernel/vector.S | 81 +++++++++++
> arch/riscv/lib/Makefile | 1 +
> arch/riscv/lib/xor.S | 81 +++++++++++
> include/uapi/linux/elf.h | 1 +
> 26 files changed, 941 insertions(+), 36 deletions(-)
> create mode 100644 arch/riscv/include/asm/vector.h
> create mode 100644 arch/riscv/include/asm/xor.h
> create mode 100644 arch/riscv/kernel/kernel_mode_vector.c
> create mode 100644 arch/riscv/kernel/vector.S
> create mode 100644 arch/riscv/lib/xor.S
>

2021-09-15 14:33:46

by Jisheng Zhang

Subject: Re: [RFC PATCH v8 21/21] riscv: Turn has_vector into a static key if VECTOR=y

On Thu, 9 Sep 2021 01:45:33 +0800
Greentime Hu <[email protected]> wrote:

> Just like fpu, we can use static key for has_vector.
> The has_vector check sits at hot code path: switch_to(). Currently,
> has_vector is a bool variable if VECTOR=y, switch_to() checks it each time,
> we can optimize out this check by turning the has_vector into a static key.
>

has_vector is newly introduced in this patch set so I believe this patch can
be folded into has_vector introducing patch, I.E patch 6


> Signed-off-by: Greentime Hu <[email protected]>
> ---
> arch/riscv/include/asm/switch_to.h | 10 +++++++---
> arch/riscv/kernel/cpufeature.c | 4 ++--
> arch/riscv/kernel/kernel_mode_vector.c | 4 ++--
> arch/riscv/kernel/process.c | 8 ++++----
> arch/riscv/kernel/signal.c | 6 +++---
> 5 files changed, 18 insertions(+), 14 deletions(-)
>
> diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h
> index b48c9c974564..576204217e0f 100644
> --- a/arch/riscv/include/asm/switch_to.h
> +++ b/arch/riscv/include/asm/switch_to.h
> @@ -71,7 +71,11 @@ static __always_inline bool has_fpu(void) { return false; }
> #endif
>
> #ifdef CONFIG_VECTOR
> -extern bool has_vector;
> +extern struct static_key_false cpu_hwcap_vector;
> +static __always_inline bool has_vector(void)
> +{
> + return static_branch_likely(&cpu_hwcap_vector);
> +}
> extern unsigned long riscv_vsize;
> extern void __vstate_save(struct __riscv_v_state *save_to, void *datap);
> extern void __vstate_restore(struct __riscv_v_state *restore_from, void *datap);
> @@ -120,7 +124,7 @@ static inline void __switch_to_vector(struct task_struct *prev,
> }
>
> #else
> -#define has_vector false
> +static __always_inline bool has_vector(void) { return false; }
> #define riscv_vsize (0)
> #define vstate_save(task, regs) do { } while (0)
> #define vstate_restore(task, regs) do { } while (0)
> @@ -136,7 +140,7 @@ do { \
> struct task_struct *__next = (next); \
> if (has_fpu()) \
> __switch_to_fpu(__prev, __next); \
> - if (has_vector) \
> + if (has_vector()) \
> __switch_to_vector(__prev, __next); \
> ((last) = __switch_to(__prev, __next)); \
> } while (0)
> diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> index af984f875f60..0139ec20adce 100644
> --- a/arch/riscv/kernel/cpufeature.c
> +++ b/arch/riscv/kernel/cpufeature.c
> @@ -23,7 +23,7 @@ __ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_fpu);
> #endif
> #ifdef CONFIG_VECTOR
> #include <asm/vector.h>
> -bool has_vector __read_mostly;
> +__ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_vector);
> unsigned long riscv_vsize __read_mostly;
> #endif
>
> @@ -157,7 +157,7 @@ void __init riscv_fill_hwcap(void)
>
> #ifdef CONFIG_VECTOR
> if (elf_hwcap & COMPAT_HWCAP_ISA_V) {
> - has_vector = true;
> + static_branch_enable(&cpu_hwcap_vector);
> /* There are 32 vector registers with vlenb length. */
> rvv_enable();
> riscv_vsize = csr_read(CSR_VLENB) * 32;
> diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c
> index 0d990bd8b8dd..0d08954c30af 100644
> --- a/arch/riscv/kernel/kernel_mode_vector.c
> +++ b/arch/riscv/kernel/kernel_mode_vector.c
> @@ -110,7 +110,7 @@ static void vector_flush_cpu_state(void)
> */
> void kernel_rvv_begin(void)
> {
> - if (WARN_ON(!has_vector))
> + if (WARN_ON(!has_vector()))
> return;
>
> WARN_ON(!may_use_vector());
> @@ -140,7 +140,7 @@ EXPORT_SYMBOL(kernel_rvv_begin);
> */
> void kernel_rvv_end(void)
> {
> - if (WARN_ON(!has_vector))
> + if (WARN_ON(!has_vector()))
> return;
>
> /* Invalidate vector regs */
> diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
> index 05ff5f934e7e..62540815ba1c 100644
> --- a/arch/riscv/kernel/process.c
> +++ b/arch/riscv/kernel/process.c
> @@ -96,7 +96,7 @@ void start_thread(struct pt_regs *regs, unsigned long pc,
> fstate_restore(current, regs);
> }
>
> - if (has_vector) {
> + if (has_vector()) {
> struct __riscv_v_state *vstate = &(current->thread.vstate);
>
> /* Enable vector and allocate memory for vector registers. */
> @@ -141,11 +141,11 @@ void flush_thread(void)
> int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> {
> fstate_save(src, task_pt_regs(src));
> - if (has_vector)
> + if (has_vector())
> /* To make sure every dirty vector context is saved. */
> vstate_save(src, task_pt_regs(src));
> *dst = *src;
> - if (has_vector) {
> + if (has_vector()) {
> /* Copy vector context to the forked task from parent. */
> if ((task_pt_regs(src)->status & SR_VS) != SR_VS_OFF) {
> dst->thread.vstate.datap = kzalloc(riscv_vsize, GFP_KERNEL);
> @@ -164,7 +164,7 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> void arch_release_task_struct(struct task_struct *tsk)
> {
> /* Free the vector context of datap. */
> - if (has_vector)
> + if (has_vector())
> kfree(tsk->thread.vstate.datap);
> }
>
> diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c
> index d30a3b588156..6a19b4b7b206 100644
> --- a/arch/riscv/kernel/signal.c
> +++ b/arch/riscv/kernel/signal.c
> @@ -192,7 +192,7 @@ static long restore_sigcontext(struct pt_regs *regs,
> goto invalid;
> goto done;
> case RVV_MAGIC:
> - if (!has_vector)
> + if (!has_vector())
> goto invalid;
> if (size != rvv_sc_size)
> goto invalid;
> @@ -221,7 +221,7 @@ static size_t cal_rt_frame_size(void)
>
> frame_size = sizeof(*frame);
>
> - if (has_vector)
> + if (has_vector())
> total_context_size += rvv_sc_size;
> /* Preserved a __riscv_ctx_hdr for END signal context header. */
> total_context_size += sizeof(struct __riscv_ctx_hdr);
> @@ -288,7 +288,7 @@ static long setup_sigcontext(struct rt_sigframe __user *frame,
> if (has_fpu())
> err |= save_fp_state(regs, &sc->sc_fpregs);
> /* Save the vector state. */
> - if (has_vector)
> + if (has_vector())
> err |= save_v_state(regs, &sc_reserved_free_ptr);
>
> /* Put END __riscv_ctx_hdr at the end. */

2021-09-15 14:41:15

by Jisheng Zhang

Subject: Re: [RFC PATCH v8 20/21] riscv: Optimize task switch codes of vector

On Thu, 9 Sep 2021 01:45:32 +0800
Greentime Hu <[email protected]> wrote:

> This patch replacees 2 instructions with 1 instruction to do the same thing
> . rs1=x0 with rd != x0 is a special form of the instruction that sets vl to
> MAXVL.

Similarly, the vector.S is newly introduced in this patch set, so could
this optimization be folded into the __vstate_save and __vstate_restore
introduction patch? Or it's better to keep this optimizaion in commit log?

>
> Suggested-by: Andrew Waterman <[email protected]>
> Co-developed-by: Vincent Chen <[email protected]>
> Signed-off-by: Vincent Chen <[email protected]>
> Signed-off-by: Greentime Hu <[email protected]>
> ---
> arch/riscv/kernel/vector.S | 9 +++------
> 1 file changed, 3 insertions(+), 6 deletions(-)
>
> diff --git a/arch/riscv/kernel/vector.S b/arch/riscv/kernel/vector.S
> index 4f0c5a166e4e..f7223c81b11a 100644
> --- a/arch/riscv/kernel/vector.S
> +++ b/arch/riscv/kernel/vector.S
> @@ -27,8 +27,7 @@
> #define x_vl t2
> #define x_vcsr t3
> #define incr t4
> -#define m_one t5
> -#define status t6
> +#define status t5
>
> ENTRY(__vstate_save)
> li status, SR_VS
> @@ -38,8 +37,7 @@ ENTRY(__vstate_save)
> csrr x_vtype, CSR_VTYPE
> csrr x_vl, CSR_VL
> csrr x_vcsr, CSR_VCSR
> - li m_one, -1
> - vsetvli incr, m_one, e8, m8
> + vsetvli incr, x0, e8, m8
> vse8.v v0, (datap)
> add datap, datap, incr
> vse8.v v8, (datap)
> @@ -61,8 +59,7 @@ ENTRY(__vstate_restore)
> li status, SR_VS
> csrs CSR_STATUS, status
>
> - li m_one, -1
> - vsetvli incr, m_one, e8, m8
> + vsetvli incr, x0, e8, m8
> vle8.v v0, (datap)
> add datap, datap, incr
> vle8.v v8, (datap)

2021-10-04 20:57:29

by Greentime Hu

Subject: Re: [RFC PATCH v8 20/21] riscv: Optimize task switch codes of vector

Jisheng Zhang <[email protected]> 於 2021年9月15日 週三 下午10:36寫道：
>
> On Thu, 9 Sep 2021 01:45:32 +0800
> Greentime Hu <[email protected]> wrote:
>
> > This patch replacees 2 instructions with 1 instruction to do the same thing
> > . rs1=x0 with rd != x0 is a special form of the instruction that sets vl to
> > MAXVL.
>
> Similarly, the vector.S is newly introduced in this patch set, so could
> this optimization be folded into the __vstate_save and __vstate_restore
> introduction patch? Or it's better to keep this optimizaion in commit log?
>
Yeah, I wanted to keep the optimization commit log before, but it's ok
if you think merge code is easier to read.
I'll merge this patch in the next version.

> >
> > Suggested-by: Andrew Waterman <[email protected]>
> > Co-developed-by: Vincent Chen <[email protected]>
> > Signed-off-by: Vincent Chen <[email protected]>
> > Signed-off-by: Greentime Hu <[email protected]>
> > ---
> > arch/riscv/kernel/vector.S | 9 +++------
> > 1 file changed, 3 insertions(+), 6 deletions(-)
> >
> > diff --git a/arch/riscv/kernel/vector.S b/arch/riscv/kernel/vector.S
> > index 4f0c5a166e4e..f7223c81b11a 100644
> > --- a/arch/riscv/kernel/vector.S
> > +++ b/arch/riscv/kernel/vector.S
> > @@ -27,8 +27,7 @@
> > #define x_vl t2
> > #define x_vcsr t3
> > #define incr t4
> > -#define m_one t5
> > -#define status t6
> > +#define status t5
> >
> > ENTRY(__vstate_save)
> > li status, SR_VS
> > @@ -38,8 +37,7 @@ ENTRY(__vstate_save)
> > csrr x_vtype, CSR_VTYPE
> > csrr x_vl, CSR_VL
> > csrr x_vcsr, CSR_VCSR
> > - li m_one, -1
> > - vsetvli incr, m_one, e8, m8
> > + vsetvli incr, x0, e8, m8
> > vse8.v v0, (datap)
> > add datap, datap, incr
> > vse8.v v8, (datap)
> > @@ -61,8 +59,7 @@ ENTRY(__vstate_restore)
> > li status, SR_VS
> > csrs CSR_STATUS, status
> >
> > - li m_one, -1
> > - vsetvli incr, m_one, e8, m8
> > + vsetvli incr, x0, e8, m8
> > vle8.v v0, (datap)
> > add datap, datap, incr
> > vle8.v v8, (datap)
>
>

2021-10-04 21:08:59

by Greentime Hu

Subject: Re: [RFC PATCH v8 21/21] riscv: Turn has_vector into a static key if VECTOR=y

Jisheng Zhang <[email protected]> 於 2021年9月15日 週三 下午10:31寫道：
>
> On Thu, 9 Sep 2021 01:45:33 +0800
> Greentime Hu <[email protected]> wrote:
>
> > Just like fpu, we can use static key for has_vector.
> > The has_vector check sits at hot code path: switch_to(). Currently,
> > has_vector is a bool variable if VECTOR=y, switch_to() checks it each time,
> > we can optimize out this check by turning the has_vector into a static key.
> >
>
> has_vector is newly introduced in this patch set so I believe this patch can
> be folded into has_vector introducing patch, I.E patch 6
>
ok, I'll split this patch and merge these codes into its related patches.

>
> > Signed-off-by: Greentime Hu <[email protected]>
> > ---
> > arch/riscv/include/asm/switch_to.h | 10 +++++++---
> > arch/riscv/kernel/cpufeature.c | 4 ++--
> > arch/riscv/kernel/kernel_mode_vector.c | 4 ++--
> > arch/riscv/kernel/process.c | 8 ++++----
> > arch/riscv/kernel/signal.c | 6 +++---
> > 5 files changed, 18 insertions(+), 14 deletions(-)
> >
> > diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h
> > index b48c9c974564..576204217e0f 100644
> > --- a/arch/riscv/include/asm/switch_to.h
> > +++ b/arch/riscv/include/asm/switch_to.h
> > @@ -71,7 +71,11 @@ static __always_inline bool has_fpu(void) { return false; }
> > #endif
> >
> > #ifdef CONFIG_VECTOR
> > -extern bool has_vector;
> > +extern struct static_key_false cpu_hwcap_vector;
> > +static __always_inline bool has_vector(void)
> > +{
> > + return static_branch_likely(&cpu_hwcap_vector);
> > +}
> > extern unsigned long riscv_vsize;
> > extern void __vstate_save(struct __riscv_v_state *save_to, void *datap);
> > extern void __vstate_restore(struct __riscv_v_state *restore_from, void *datap);
> > @@ -120,7 +124,7 @@ static inline void __switch_to_vector(struct task_struct *prev,
> > }
> >
> > #else
> > -#define has_vector false
> > +static __always_inline bool has_vector(void) { return false; }
> > #define riscv_vsize (0)
> > #define vstate_save(task, regs) do { } while (0)
> > #define vstate_restore(task, regs) do { } while (0)
> > @@ -136,7 +140,7 @@ do { \
> > struct task_struct *__next = (next); \
> > if (has_fpu()) \
> > __switch_to_fpu(__prev, __next); \
> > - if (has_vector) \
> > + if (has_vector()) \
> > __switch_to_vector(__prev, __next); \
> > ((last) = __switch_to(__prev, __next)); \
> > } while (0)
> > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > index af984f875f60..0139ec20adce 100644
> > --- a/arch/riscv/kernel/cpufeature.c
> > +++ b/arch/riscv/kernel/cpufeature.c
> > @@ -23,7 +23,7 @@ __ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_fpu);
> > #endif
> > #ifdef CONFIG_VECTOR
> > #include <asm/vector.h>
> > -bool has_vector __read_mostly;
> > +__ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_vector);
> > unsigned long riscv_vsize __read_mostly;
> > #endif
> >
> > @@ -157,7 +157,7 @@ void __init riscv_fill_hwcap(void)
> >
> > #ifdef CONFIG_VECTOR
> > if (elf_hwcap & COMPAT_HWCAP_ISA_V) {
> > - has_vector = true;
> > + static_branch_enable(&cpu_hwcap_vector);
> > /* There are 32 vector registers with vlenb length. */
> > rvv_enable();
> > riscv_vsize = csr_read(CSR_VLENB) * 32;
> > diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c
> > index 0d990bd8b8dd..0d08954c30af 100644
> > --- a/arch/riscv/kernel/kernel_mode_vector.c
> > +++ b/arch/riscv/kernel/kernel_mode_vector.c
> > @@ -110,7 +110,7 @@ static void vector_flush_cpu_state(void)
> > */
> > void kernel_rvv_begin(void)
> > {
> > - if (WARN_ON(!has_vector))
> > + if (WARN_ON(!has_vector()))
> > return;
> >
> > WARN_ON(!may_use_vector());
> > @@ -140,7 +140,7 @@ EXPORT_SYMBOL(kernel_rvv_begin);
> > */
> > void kernel_rvv_end(void)
> > {
> > - if (WARN_ON(!has_vector))
> > + if (WARN_ON(!has_vector()))
> > return;
> >
> > /* Invalidate vector regs */
> > diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
> > index 05ff5f934e7e..62540815ba1c 100644
> > --- a/arch/riscv/kernel/process.c
> > +++ b/arch/riscv/kernel/process.c
> > @@ -96,7 +96,7 @@ void start_thread(struct pt_regs *regs, unsigned long pc,
> > fstate_restore(current, regs);
> > }
> >
> > - if (has_vector) {
> > + if (has_vector()) {
> > struct __riscv_v_state *vstate = &(current->thread.vstate);
> >
> > /* Enable vector and allocate memory for vector registers. */
> > @@ -141,11 +141,11 @@ void flush_thread(void)
> > int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> > {
> > fstate_save(src, task_pt_regs(src));
> > - if (has_vector)
> > + if (has_vector())
> > /* To make sure every dirty vector context is saved. */
> > vstate_save(src, task_pt_regs(src));
> > *dst = *src;
> > - if (has_vector) {
> > + if (has_vector()) {
> > /* Copy vector context to the forked task from parent. */
> > if ((task_pt_regs(src)->status & SR_VS) != SR_VS_OFF) {
> > dst->thread.vstate.datap = kzalloc(riscv_vsize, GFP_KERNEL);
> > @@ -164,7 +164,7 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> > void arch_release_task_struct(struct task_struct *tsk)
> > {
> > /* Free the vector context of datap. */
> > - if (has_vector)
> > + if (has_vector())
> > kfree(tsk->thread.vstate.datap);
> > }
> >
> > diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c
> > index d30a3b588156..6a19b4b7b206 100644
> > --- a/arch/riscv/kernel/signal.c
> > +++ b/arch/riscv/kernel/signal.c
> > @@ -192,7 +192,7 @@ static long restore_sigcontext(struct pt_regs *regs,
> > goto invalid;
> > goto done;
> > case RVV_MAGIC:
> > - if (!has_vector)
> > + if (!has_vector())
> > goto invalid;
> > if (size != rvv_sc_size)
> > goto invalid;
> > @@ -221,7 +221,7 @@ static size_t cal_rt_frame_size(void)
> >
> > frame_size = sizeof(*frame);
> >
> > - if (has_vector)
> > + if (has_vector())
> > total_context_size += rvv_sc_size;
> > /* Preserved a __riscv_ctx_hdr for END signal context header. */
> > total_context_size += sizeof(struct __riscv_ctx_hdr);
> > @@ -288,7 +288,7 @@ static long setup_sigcontext(struct rt_sigframe __user *frame,
> > if (has_fpu())
> > err |= save_fp_state(regs, &sc->sc_fpregs);
> > /* Save the vector state. */
> > - if (has_vector)
> > + if (has_vector())
> > err |= save_v_state(regs, &sc_reserved_free_ptr);
> >
> > /* Put END __riscv_ctx_hdr at the end. */
>
>