2022-01-22 01:57:49

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 00/14] riscv: support for svpbmt and D1 memory types

So this is my try at implementing svpbmt (and the diverging D1 memory
types using the alternatives framework).

This includes a number of changes to the alternatives mechanism itself.
The biggest one being the move to a more central location, as I expect
in the future, nearly every chip needing some sort of patching, be it
either for erratas or for optional features (svpbmt or others).

The dt-binding for svpbmt itself is of course not finished and is still
using the binding introduced in previous versions, as where to put
a svpbmt-property in the devicetree is still under dicussion.
Atish seems to be working on a framework for extensions [0],

The series also introduces support for the memory types of the D1
which are implemented differently to svpbmt. But when patching anyway
it's pretty clean to add the D1 variant via ALTERNATIVE_2 to the same
location.

The only slightly bigger difference is that the "normal" type is not 0
as with svpbmt, so kernel patches for this PMA type need to be applied
even before the MMU is brought up, so the series introduces a separate
stage for that.


In theory this series is 3 parts:
- sbi cache-flush / null-ptr
- alternatives improvements
- svpbmt+d1

So expecially patches from the first 2 areas could be applied when
deemed ready, I just thought to keep it together to show-case where
the end-goal is and not requiring jumping between different series.


The sbi cache-flush patch is based on Atish's sparse-hartid patch [1],
as it touches a similar area in mm/cacheflush.c


I picked the recipient list from the previous version, hopefully
I didn't forget anybody.


[0] https://lore.kernel.org/r/[email protected]
[1] https://lore.kernel.org/r/[email protected]


Heiko Stuebner (12):
riscv: only use IPIs to handle cache-flushes on remote cpus
riscv: integrate alternatives better into the main architecture
riscv: allow different stages with alternatives
riscv: implement module alternatives
riscv: implement ALTERNATIVE_2 macro
riscv: extend concatenated alternatives-lines to the same length
riscv: prevent compressed instructions in alternatives
riscv: move boot alternatives to a slightly earlier position
riscv: Fix accessing pfn bits in PTEs for non-32bit variants
riscv: add cpufeature handling via alternatives
riscv: remove FIXMAP_PAGE_IO and fall back to its default value
riscv: add memory-type errata for T-Head

Wei Fu (2):
dt-bindings: riscv: add MMU Standard Extensions support for Svpbmt
riscv: add RISC-V Svpbmt extension supports

.../devicetree/bindings/riscv/cpus.yaml | 10 ++
arch/riscv/Kconfig.erratas | 29 ++--
arch/riscv/Kconfig.socs | 1 -
arch/riscv/Makefile | 2 +-
arch/riscv/errata/Makefile | 2 +-
arch/riscv/errata/sifive/errata.c | 10 +-
arch/riscv/errata/thead/Makefile | 1 +
arch/riscv/errata/thead/errata.c | 85 +++++++++++
arch/riscv/include/asm/alternative-macros.h | 114 ++++++++-------
arch/riscv/include/asm/alternative.h | 16 ++-
arch/riscv/include/asm/errata_list.h | 52 +++++++
arch/riscv/include/asm/fixmap.h | 2 -
arch/riscv/include/asm/pgtable-32.h | 17 +++
arch/riscv/include/asm/pgtable-64.h | 79 +++++++++-
arch/riscv/include/asm/pgtable-bits.h | 10 --
arch/riscv/include/asm/pgtable.h | 53 +++++--
arch/riscv/include/asm/vendorid_list.h | 1 +
arch/riscv/kernel/Makefile | 1 +
arch/riscv/{errata => kernel}/alternative.c | 47 +++++-
arch/riscv/kernel/cpufeature.c | 136 +++++++++++++++++-
arch/riscv/kernel/head.S | 2 +
arch/riscv/kernel/module.c | 29 ++++
arch/riscv/kernel/smpboot.c | 4 -
arch/riscv/kernel/traps.c | 2 +-
arch/riscv/mm/cacheflush.c | 8 +-
arch/riscv/mm/init.c | 2 +
26 files changed, 599 insertions(+), 116 deletions(-)
create mode 100644 arch/riscv/errata/thead/Makefile
create mode 100644 arch/riscv/errata/thead/errata.c
rename arch/riscv/{errata => kernel}/alternative.c (60%)

--
2.30.2


2022-01-22 01:57:53

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 06/14] riscv: extend concatenated alternatives-lines to the same length

ALT_NEW_CONTENT already uses same-length assembler lines, so
extend this to the other elements as well.

This makes it more readable when these elements need to be extended
in the future.

Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/include/asm/alternative-macros.h | 30 ++++++++++-----------
1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
index baf649293288..c0fb11fad631 100644
--- a/arch/riscv/include/asm/alternative-macros.h
+++ b/arch/riscv/include/asm/alternative-macros.h
@@ -56,14 +56,14 @@
#include <asm/asm.h>
#include <linux/stringify.h>

-#define ALT_ENTRY(oldptr, newptr, vendor_id, errata_id, newlen) \
- RISCV_PTR " " oldptr "\n" \
- RISCV_PTR " " newptr "\n" \
- REG_ASM " " vendor_id "\n" \
- REG_ASM " " newlen "\n" \
+#define ALT_ENTRY(oldptr, newptr, vendor_id, errata_id, newlen) \
+ RISCV_PTR " " oldptr "\n" \
+ RISCV_PTR " " newptr "\n" \
+ REG_ASM " " vendor_id "\n" \
+ REG_ASM " " newlen "\n" \
".word " errata_id "\n"

-#define ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c) \
+#define ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c) \
".if " __stringify(enable) " == 1\n" \
".pushsection .alternative, \"a\"\n" \
ALT_ENTRY("886b", "888f", __stringify(vendor_id), __stringify(errata_id), "889f - 888f") \
@@ -77,21 +77,21 @@
".org . - (889b - 888b) + (887b - 886b)\n" \
".endif\n"

-#define __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, enable) \
- "886 :\n" \
- old_c "\n" \
- "887 :\n" \
+#define __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, enable) \
+ "886 :\n" \
+ old_c "\n" \
+ "887 :\n" \
ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c)

-#define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
+#define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
__ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, IS_ENABLED(CONFIG_k))

#define __ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
new_c_2, vendor_id_2, errata_id_2, enable_2) \
- "886 :\n" \
- old_c "\n" \
- "887 :\n" \
- ALT_NEW_CONTENT(vendor_id_1, errata_id_1, enable_1, new_c_1) \
+ "886 :\n" \
+ old_c "\n" \
+ "887 :\n" \
+ ALT_NEW_CONTENT(vendor_id_1, errata_id_1, enable_1, new_c_1) \
ALT_NEW_CONTENT(vendor_id_2, errata_id_2, enable_2, new_c_2)

#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
--
2.30.2

2022-01-22 01:58:01

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 09/14] riscv: Fix accessing pfn bits in PTEs for non-32bit variants

On rv32 the PFN part of PTEs is defined to use bits [xlen-1:10]
while on rv64 it is defined to use bits [53:10], leaving [63:54]
as reserved.

With upcoming optional extensions like svpbmt these previously
reserved bits will get used so simply right-shifting the PTE
to get the PFN won't be enough.

So introduce a _PAGE_PFN_MASK constant to mask the correct bits
for both rv32 and rv64 before shifting.

Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/include/asm/pgtable-32.h | 8 ++++++++
arch/riscv/include/asm/pgtable-64.h | 14 +++++++++++---
arch/riscv/include/asm/pgtable-bits.h | 6 ------
arch/riscv/include/asm/pgtable.h | 6 +++---
4 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/arch/riscv/include/asm/pgtable-32.h b/arch/riscv/include/asm/pgtable-32.h
index 5b2e79e5bfa5..e266a4fe7f43 100644
--- a/arch/riscv/include/asm/pgtable-32.h
+++ b/arch/riscv/include/asm/pgtable-32.h
@@ -7,6 +7,7 @@
#define _ASM_RISCV_PGTABLE_32_H

#include <asm-generic/pgtable-nopmd.h>
+#include <linux/bits.h>
#include <linux/const.h>

/* Size of region mapped by a page global directory */
@@ -16,4 +17,11 @@

#define MAX_POSSIBLE_PHYSMEM_BITS 34

+/*
+ * rv32 PTE format:
+ * | XLEN-1 10 | 9 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
+ * PFN reserved for SW D A G U X W R V
+ */
+#define _PAGE_PFN_MASK GENMASK(31, 10)
+
#endif /* _ASM_RISCV_PGTABLE_32_H */
diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index 228261aa9628..706f0897437d 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -6,6 +6,7 @@
#ifndef _ASM_RISCV_PGTABLE_64_H
#define _ASM_RISCV_PGTABLE_64_H

+#include <linux/bits.h>
#include <linux/const.h>

#define PGDIR_SHIFT 30
@@ -28,6 +29,13 @@ typedef struct {

#define PTRS_PER_PMD (PAGE_SIZE / sizeof(pmd_t))

+/*
+ * rv64 PTE format:
+ * | 63 | 62 61 | 60 54 | 53 10 | 9 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
+ * N MT RSV PFN reserved for SW D A G U X W R V
+ */
+#define _PAGE_PFN_MASK GENMASK(53, 10)
+
static inline int pud_present(pud_t pud)
{
return (pud_val(pud) & _PAGE_PRESENT);
@@ -61,12 +69,12 @@ static inline void pud_clear(pud_t *pudp)

static inline pmd_t *pud_pgtable(pud_t pud)
{
- return (pmd_t *)pfn_to_virt(pud_val(pud) >> _PAGE_PFN_SHIFT);
+ return (pmd_t *)pfn_to_virt((pud_val(pud) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT);
}

static inline struct page *pud_page(pud_t pud)
{
- return pfn_to_page(pud_val(pud) >> _PAGE_PFN_SHIFT);
+ return pfn_to_page((pud_val(pud) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT);
}

static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
@@ -76,7 +84,7 @@ static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)

static inline unsigned long _pmd_pfn(pmd_t pmd)
{
- return pmd_val(pmd) >> _PAGE_PFN_SHIFT;
+ return (pmd_val(pmd) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT;
}

#define mk_pmd(page, prot) pfn_pmd(page_to_pfn(page), prot)
diff --git a/arch/riscv/include/asm/pgtable-bits.h b/arch/riscv/include/asm/pgtable-bits.h
index 2ee413912926..4831122d559d 100644
--- a/arch/riscv/include/asm/pgtable-bits.h
+++ b/arch/riscv/include/asm/pgtable-bits.h
@@ -6,12 +6,6 @@
#ifndef _ASM_RISCV_PGTABLE_BITS_H
#define _ASM_RISCV_PGTABLE_BITS_H

-/*
- * PTE format:
- * | XLEN-1 10 | 9 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
- * PFN reserved for SW D A G U X W R V
- */
-
#define _PAGE_ACCESSED_OFFSET 6

#define _PAGE_PRESENT (1 << 0)
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index bf204e7c1f74..5cf7465a96c5 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -232,12 +232,12 @@ static inline unsigned long _pgd_pfn(pgd_t pgd)

static inline struct page *pmd_page(pmd_t pmd)
{
- return pfn_to_page(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
+ return pfn_to_page((pmd_val(pmd) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT);
}

static inline unsigned long pmd_page_vaddr(pmd_t pmd)
{
- return (unsigned long)pfn_to_virt(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
+ return (unsigned long)pfn_to_virt((pmd_val(pmd) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT);
}

static inline pte_t pmd_pte(pmd_t pmd)
@@ -253,7 +253,7 @@ static inline pte_t pud_pte(pud_t pud)
/* Yields the page frame number (PFN) of a page table entry */
static inline unsigned long pte_pfn(pte_t pte)
{
- return (pte_val(pte) >> _PAGE_PFN_SHIFT);
+ return ((pte_val(pte) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT);
}

#define pte_page(x) pfn_to_page(pte_pfn(x))
--
2.30.2

2022-01-22 01:58:06

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 13/14] riscv: remove FIXMAP_PAGE_IO and fall back to its default value

If not defined in the arch, FIXMAP_PAGE_IO defaults to PAGE_KERNEL_IO,
which we defined when adding the svpbmt implementation.

So drop the FIXMAP_PAGE_IO riscv define.

Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/include/asm/fixmap.h | 2 --
1 file changed, 2 deletions(-)

diff --git a/arch/riscv/include/asm/fixmap.h b/arch/riscv/include/asm/fixmap.h
index 54cbf07fb4e9..899b59bdb9eb 100644
--- a/arch/riscv/include/asm/fixmap.h
+++ b/arch/riscv/include/asm/fixmap.h
@@ -43,8 +43,6 @@ enum fixed_addresses {
__end_of_fixed_addresses
};

-#define FIXMAP_PAGE_IO PAGE_KERNEL
-
#define __early_set_fixmap __set_fixmap

#define __late_set_fixmap __set_fixmap
--
2.30.2

2022-01-22 01:58:16

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 11/14] dt-bindings: riscv: add MMU Standard Extensions support for Svpbmt

From: Wei Fu <[email protected]>

Previous patch has added svpbmt in arch/riscv and add "riscv,svpmbt"
in the DT mmu node. Update dt-bindings related property here.

Signed-off-by: Wei Fu <[email protected]>
Co-developed-by: Guo Ren <[email protected]>
Signed-off-by: Guo Ren <[email protected]>
Signed-off-by: Heiko Stuebner <[email protected]>
Cc: Anup Patel <[email protected]>
Cc: Palmer Dabbelt <[email protected]>
Cc: Rob Herring <[email protected]>
---
Documentation/devicetree/bindings/riscv/cpus.yaml | 10 ++++++++++
1 file changed, 10 insertions(+)

diff --git a/Documentation/devicetree/bindings/riscv/cpus.yaml b/Documentation/devicetree/bindings/riscv/cpus.yaml
index aa5fb64d57eb..3ad2593f1400 100644
--- a/Documentation/devicetree/bindings/riscv/cpus.yaml
+++ b/Documentation/devicetree/bindings/riscv/cpus.yaml
@@ -63,6 +63,16 @@ properties:
- riscv,sv48
- riscv,none

+ mmu:
+ description:
+ Describes the CPU's MMU Standard Extensions support.
+ These values originate from the RISC-V Privileged
+ Specification document, available from
+ https://riscv.org/specifications/
+ $ref: '/schemas/types.yaml#/definitions/string'
+ enum:
+ - riscv,svpbmt
+
riscv,isa:
description:
Identifies the specific RISC-V instruction set architecture
--
2.30.2

2022-01-22 01:58:28

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 05/14] riscv: implement ALTERNATIVE_2 macro

When the alternatives were added the commit already provided a template
on how to implement 2 different alternatives for one piece of code.

Make this usable.

Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/include/asm/alternative-macros.h | 52 +++++++++++++--------
1 file changed, 32 insertions(+), 20 deletions(-)

diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
index 92da6b3920a3..baf649293288 100644
--- a/arch/riscv/include/asm/alternative-macros.h
+++ b/arch/riscv/include/asm/alternative-macros.h
@@ -37,6 +37,20 @@
#define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
__ALTERNATIVE_CFG old_c, new_c, vendor_id, errata_id, IS_ENABLED(CONFIG_k)

+.macro __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
+ new_c_2, vendor_id_2, errata_id_2, enable_2
+886 :
+ \old_c
+887 :
+ ALT_NEW_CONTENT \vendor_id_1, \errata_id_1, \enable_1, \new_c_1
+ ALT_NEW_CONTENT \vendor_id_2, \errata_id_2, \enable_2, \new_c_2
+.endm
+
+#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
+ new_c_2, vendor_id_2, errata_id_2, CONFIG_k_2) \
+ __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, IS_ENABLED(CONFIG_k_1), \
+ new_c_2, vendor_id_2, errata_id_2, IS_ENABLED(CONFIG_k_2)
+
#else /* !__ASSEMBLY__ */

#include <asm/asm.h>
@@ -72,6 +86,19 @@
#define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
__ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, IS_ENABLED(CONFIG_k))

+#define __ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
+ new_c_2, vendor_id_2, errata_id_2, enable_2) \
+ "886 :\n" \
+ old_c "\n" \
+ "887 :\n" \
+ ALT_NEW_CONTENT(vendor_id_1, errata_id_1, enable_1, new_c_1) \
+ ALT_NEW_CONTENT(vendor_id_2, errata_id_2, enable_2, new_c_2)
+
+#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
+ new_c_2, vendor_id_2, errata_id_2, CONFIG_k_2) \
+ __ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, IS_ENABLED(CONFIG_k_1), \
+ new_c_2, vendor_id_2, errata_id_2, IS_ENABLED(CONFIG_k_2))
+
#endif /* __ASSEMBLY__ */

/*
@@ -96,25 +123,10 @@
* this case, this vendor can create a new macro ALTERNATIVE_2() based
* on the following sample code and then replace ALTERNATIVE() with
* ALTERNATIVE_2() to append its customized content.
- *
- * .macro __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
- * new_c_2, vendor_id_2, errata_id_2, enable_2
- * 886 :
- * \old_c
- * 887 :
- * ALT_NEW_CONTENT \vendor_id_1, \errata_id_1, \enable_1, \new_c_1
- * ALT_NEW_CONTENT \vendor_id_2, \errata_id_2, \enable_2, \new_c_2
- * .endm
- *
- * #define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
- * new_c_2, vendor_id_2, errata_id_2, CONFIG_k_2) \
- * __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, IS_ENABLED(CONFIG_k_1), \
- * new_c_2, vendor_id_2, errata_id_2, IS_ENABLED(CONFIG_k_2) \
- *
- * #define ALTERNATIVE_2(old_content, new_content_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
- * new_content_2, vendor_id_2, errata_id_2, CONFIG_k_2) \
- * _ALTERNATIVE_CFG_2(old_content, new_content_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
- * new_content_2, vendor_id_2, errata_id_2, CONFIG_k_2)
- *
*/
+#define ALTERNATIVE_2(old_content, new_content_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
+ new_content_2, vendor_id_2, errata_id_2, CONFIG_k_2) \
+ _ALTERNATIVE_CFG_2(old_content, new_content_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
+ new_content_2, vendor_id_2, errata_id_2, CONFIG_k_2)
+
#endif
--
2.30.2

2022-01-22 01:58:29

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 12/14] riscv: add RISC-V Svpbmt extension supports

From: Wei Fu <[email protected]>

This patch follows the standard pure RISC-V Svpbmt extension in
privilege spec to solve the non-coherent SOC dma synchronization
issues.

Here is the svpbmt PTE format:
| 63 | 62-61 | 60-8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
N MT RSW D A G U X W R V
^

Of the Reserved bits [63:54] in a leaf PTE, the high bit is already
allocated (as the N bit), so bits [62:61] are used as the MT (aka
MemType) field. This field specifies one of three memory types that
are close equivalents (or equivalent in effect) to the three main x86
and ARMv8 memory types - as shown in the following table.

RISC-V
Encoding &
MemType RISC-V Description
---------- ------------------------------------------------
00 - PMA Normal Cacheable, No change to implied PMA memory type
01 - NC Non-cacheable, idempotent, weakly-ordered Main Memory
10 - IO Non-cacheable, non-idempotent, strongly-ordered I/O memory
11 - Rsvd Reserved for future standard use

The standard protection_map[] needn't be modified because the "PMA"
type keeps the highest bits zero. The implementation is limited to
arch/riscv/* as it uses an inline function and the alternatives
mechanism to generate the _PAGE_MTMASK/IO/NOCACHE values for
pgprot_noncached (& writecombine) in pgtable.h on both svpbmt and
non-svpbmt-capable chips.

Signed-off-by: Wei Fu <[email protected]>
Co-developed-by: Liu Shaohua <[email protected]>
Signed-off-by: Liu Shaohua <[email protected]>
Co-developed-by: Guo Ren <[email protected]>
Signed-off-by: Guo Ren <[email protected]>
[moved to use the alternatives mechanism]
Co-developed-by: Heiko Stuebner <[email protected]>
Signed-off-by: Heiko Stuebner <[email protected]>
Cc: Palmer Dabbelt <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Anup Patel <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: Atish Patra <[email protected]>
Cc: Drew Fustini <[email protected]>
Cc: Wei Fu <[email protected]>
Cc: Wei Wu <[email protected]>
Cc: Chen-Yu Tsai <[email protected]>
Cc: Maxime Ripard <[email protected]>
Cc: Daniel Lustig <[email protected]>
Cc: Greg Favor <[email protected]>
Cc: Andrea Mondelli <[email protected]>
Cc: Jonathan Behrens <[email protected]>
Cc: Xinhaoqu (Freddie) <[email protected]>
Cc: Bill Huffman <[email protected]>
Cc: Nick Kossifidis <[email protected]>
Cc: Allen Baum <[email protected]>
Cc: Josh Scheid <[email protected]>
Cc: Richard Trauben <[email protected]>
---
arch/riscv/include/asm/errata_list.h | 13 ++++-
arch/riscv/include/asm/pgtable-32.h | 9 +++
arch/riscv/include/asm/pgtable-64.h | 47 ++++++++++++++++
arch/riscv/include/asm/pgtable-bits.h | 4 --
arch/riscv/include/asm/pgtable.h | 29 ++++++++--
arch/riscv/kernel/cpufeature.c | 79 +++++++++++++++++++++++++++
6 files changed, 171 insertions(+), 10 deletions(-)

diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
index 6b95bd9aee82..a4a9b0842922 100644
--- a/arch/riscv/include/asm/errata_list.h
+++ b/arch/riscv/include/asm/errata_list.h
@@ -14,7 +14,8 @@
#define ERRATA_SIFIVE_NUMBER 2
#endif

-#define CPUFEATURE_NUMBER 0
+#define CPUFEATURE_SVPBMT 0
+#define CPUFEATURE_NUMBER 1

#ifdef __ASSEMBLY__

@@ -36,6 +37,16 @@ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \
ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \
: : "r" (addr) : "memory")

+/*
+ * _val is marked as "will be overwritten", so need to set it to 0
+ * in the default case.
+ */
+#define ALT_SVPBMT_SHIFT 61
+#define ALT_SVPBMT(_val, prot) \
+asm(ALTERNATIVE("li %0, 0\t\nnop", "li %0, %1\t\nslli %0,%0,%2", 0, \
+ CPUFEATURE_SVPBMT, CONFIG_64BIT) \
+ : "=r"(_val) : "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT), "I"(ALT_SVPBMT_SHIFT))
+
#endif /* __ASSEMBLY__ */

#endif
diff --git a/arch/riscv/include/asm/pgtable-32.h b/arch/riscv/include/asm/pgtable-32.h
index e266a4fe7f43..59ba1fbaf784 100644
--- a/arch/riscv/include/asm/pgtable-32.h
+++ b/arch/riscv/include/asm/pgtable-32.h
@@ -24,4 +24,13 @@
*/
#define _PAGE_PFN_MASK GENMASK(31, 10)

+#define _PAGE_NOCACHE 0
+#define _PAGE_IO 0
+#define _PAGE_MTMASK 0
+
+/* Set of bits to preserve across pte_modify() */
+#define _PAGE_CHG_MASK (~(unsigned long)(_PAGE_PRESENT | _PAGE_READ | \
+ _PAGE_WRITE | _PAGE_EXEC | \
+ _PAGE_USER | _PAGE_GLOBAL))
+
#endif /* _ASM_RISCV_PGTABLE_32_H */
diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index 706f0897437d..58b34ef74acb 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -8,6 +8,7 @@

#include <linux/bits.h>
#include <linux/const.h>
+#include <asm/errata_list.h>

#define PGDIR_SHIFT 30
/* Size of region mapped by a page global directory */
@@ -36,6 +37,52 @@ typedef struct {
*/
#define _PAGE_PFN_MASK GENMASK(53, 10)

+/*
+ * [62:61] Svpbmt Memory Type definitions:
+ *
+ * 00 - PMA Normal Cacheable, No change to implied PMA memory type
+ * 01 - NC Non-cacheable, idempotent, weakly-ordered Main Memory
+ * 10 - IO Non-cacheable, non-idempotent, strongly-ordered I/O memory
+ * 11 - Rsvd Reserved for future standard use
+ */
+#define _PAGE_NOCACHE_SVPBMT (1UL << 61)
+#define _PAGE_IO_SVPBMT (1UL << 62)
+#define _PAGE_MTMASK_SVPBMT (_PAGE_NOCACHE_SVPBMT | _PAGE_IO_SVPBMT)
+
+static inline u64 riscv_page_mtmask(void)
+{
+ u64 val;
+
+ ALT_SVPBMT(val, _PAGE_MTMASK);
+ return val;
+}
+
+static inline u64 riscv_page_nocache(void)
+{
+ u64 val;
+
+ ALT_SVPBMT(val, _PAGE_NOCACHE);
+ return val;
+}
+
+static inline u64 riscv_page_io(void)
+{
+ u64 val;
+
+ ALT_SVPBMT(val, _PAGE_IO);
+ return val;
+}
+
+#define _PAGE_NOCACHE riscv_page_nocache()
+#define _PAGE_IO riscv_page_io()
+#define _PAGE_MTMASK riscv_page_mtmask()
+
+/* Set of bits to preserve across pte_modify() */
+#define _PAGE_CHG_MASK (~(unsigned long)(_PAGE_PRESENT | _PAGE_READ | \
+ _PAGE_WRITE | _PAGE_EXEC | \
+ _PAGE_USER | _PAGE_GLOBAL | \
+ _PAGE_MTMASK))
+
static inline int pud_present(pud_t pud)
{
return (pud_val(pud) & _PAGE_PRESENT);
diff --git a/arch/riscv/include/asm/pgtable-bits.h b/arch/riscv/include/asm/pgtable-bits.h
index 4831122d559d..ee908a142f3c 100644
--- a/arch/riscv/include/asm/pgtable-bits.h
+++ b/arch/riscv/include/asm/pgtable-bits.h
@@ -29,10 +29,6 @@

#define _PAGE_PFN_SHIFT 10

-/* Set of bits to preserve across pte_modify() */
-#define _PAGE_CHG_MASK (~(unsigned long)(_PAGE_PRESENT | _PAGE_READ | \
- _PAGE_WRITE | _PAGE_EXEC | \
- _PAGE_USER | _PAGE_GLOBAL))
/*
* when all of R/W/X are zero, the PTE is a pointer to the next level
* of the page table; otherwise, it is a leaf PTE.
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 5cf7465a96c5..a189a21a7c8a 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -148,11 +148,8 @@

#define PAGE_TABLE __pgprot(_PAGE_TABLE)

-/*
- * The RISC-V ISA doesn't yet specify how to query or modify PMAs, so we can't
- * change the properties of memory regions.
- */
-#define _PAGE_IOREMAP _PAGE_KERNEL
+#define _PAGE_IOREMAP ((_PAGE_KERNEL & ~_PAGE_MTMASK) | _PAGE_IO)
+#define PAGE_KERNEL_IO __pgprot(_PAGE_IOREMAP)

extern pgd_t swapper_pg_dir[];

@@ -492,6 +489,28 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma,
return ptep_test_and_clear_young(vma, address, ptep);
}

+#define pgprot_noncached pgprot_noncached
+static inline pgprot_t pgprot_noncached(pgprot_t _prot)
+{
+ unsigned long prot = pgprot_val(_prot);
+
+ prot &= ~_PAGE_MTMASK;
+ prot |= _PAGE_IO;
+
+ return __pgprot(prot);
+}
+
+#define pgprot_writecombine pgprot_writecombine
+static inline pgprot_t pgprot_writecombine(pgprot_t _prot)
+{
+ unsigned long prot = pgprot_val(_prot);
+
+ prot &= ~_PAGE_MTMASK;
+ prot |= _PAGE_NOCACHE;
+
+ return __pgprot(prot);
+}
+
/*
* THP functions
*/
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index fef804346d75..d14289fba055 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -7,6 +7,7 @@
*/

#include <linux/bitmap.h>
+#include <linux/libfdt.h>
#include <linux/of.h>
#include <asm/alternative.h>
#include <asm/errata_list.h>
@@ -159,7 +160,85 @@ struct cpufeature_info {
bool (*check_func)(unsigned int stage);
};

+#if defined(CONFIG_MMU) && defined(CONFIG_64BIT)
+static bool cpufeature_svpbmt_check_fdt(void)
+{
+ const void *fdt = dtb_early_va;
+ const char *str;
+ int offset;
+
+ offset = fdt_path_offset(fdt, "/cpus");
+ if (offset < 0)
+ return false;
+
+ for (offset = fdt_next_node(fdt, offset, NULL); offset >= 0;
+ offset = fdt_next_node(fdt, offset, NULL)) {
+ str = fdt_getprop(fdt, offset, "device_type", NULL);
+ if (!str || strcmp(str, "cpu"))
+ break;
+
+ str = fdt_getprop(fdt, offset, "mmu-type", NULL);
+ if (!str)
+ continue;
+
+ if (!strncmp(str + 6, "none", 4))
+ continue;
+
+ str = fdt_getprop(fdt, offset, "mmu", NULL);
+ if (!str)
+ continue;
+
+ if (!strncmp(str + 6, "svpbmt", 6))
+ return true;
+ }
+
+ return false;
+}
+
+static bool cpufeature_svpbmt_check_of(void)
+{
+ struct device_node *node;
+ const char *str;
+
+ for_each_of_cpu_node(node) {
+ if (of_property_read_string(node, "mmu-type", &str))
+ continue;
+
+ if (!strncmp(str + 6, "none", 4))
+ continue;
+
+ if (of_property_read_string(node, "mmu", &str))
+ continue;
+
+ if (!strncmp(str + 6, "svpbmt", 6))
+ return true;
+ }
+
+ return false;
+}
+#endif
+
+static bool cpufeature_svpbmt_check_func(unsigned int stage)
+{
+ bool ret = false;
+
+#if defined(CONFIG_MMU) && defined(CONFIG_64BIT)
+ switch (stage) {
+ case RISCV_ALTERNATIVES_BOOT:
+ return cpufeature_svpbmt_check_fdt();
+ default:
+ return cpufeature_svpbmt_check_of();
+ }
+#endif
+
+ return ret;
+}
+
static const struct cpufeature_info cpufeature_list[CPUFEATURE_NUMBER] = {
+ {
+ .name = "svpbmt",
+ .check_func = cpufeature_svpbmt_check_func
+ },
};

static u32 __init cpufeature_probe(unsigned int stage)
--
2.30.2

2022-01-22 01:58:32

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 10/14] riscv: add cpufeature handling via alternatives

Some cpufeatures should be handled via the alternatives mechanism
to not incur penalties on unsupporting variants.

So add a mechanism to handle these similar to cpu erratas.

Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/include/asm/alternative.h | 3 ++
arch/riscv/include/asm/errata_list.h | 2 +
arch/riscv/kernel/alternative.c | 2 +
arch/riscv/kernel/cpufeature.c | 55 +++++++++++++++++++++++++++-
4 files changed, 61 insertions(+), 1 deletion(-)

diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
index f0657b1b3174..cf3b22173834 100644
--- a/arch/riscv/include/asm/alternative.h
+++ b/arch/riscv/include/asm/alternative.h
@@ -39,5 +39,8 @@ struct errata_checkfunc_id {
void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
unsigned long archid, unsigned long impid,
unsigned int stage);
+
+void riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end,
+ unsigned int stage);
#endif
#endif
diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
index 5f1046e82d9f..6b95bd9aee82 100644
--- a/arch/riscv/include/asm/errata_list.h
+++ b/arch/riscv/include/asm/errata_list.h
@@ -14,6 +14,8 @@
#define ERRATA_SIFIVE_NUMBER 2
#endif

+#define CPUFEATURE_NUMBER 0
+
#ifdef __ASSEMBLY__

#define ALT_INSN_FAULT(x) \
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index e1849faa2fa1..54913857b682 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -63,6 +63,8 @@ static void __init_or_module _apply_alternatives(struct alt_entry *begin,
struct alt_entry *end,
unsigned int stage)
{
+ riscv_cpufeature_patch_func(begin, end, stage);
+
if (!vendor_patch_func)
return;

diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index d959d207a40d..fef804346d75 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -8,8 +8,12 @@

#include <linux/bitmap.h>
#include <linux/of.h>
-#include <asm/processor.h>
+#include <asm/alternative.h>
+#include <asm/errata_list.h>
#include <asm/hwcap.h>
+#include <asm/patch.h>
+#include <asm/pgtable.h>
+#include <asm/processor.h>
#include <asm/smp.h>
#include <asm/switch_to.h>

@@ -149,3 +153,52 @@ void __init riscv_fill_hwcap(void)
static_branch_enable(&cpu_hwcap_fpu);
#endif
}
+
+struct cpufeature_info {
+ char name[ERRATA_STRING_LENGTH_MAX];
+ bool (*check_func)(unsigned int stage);
+};
+
+static const struct cpufeature_info cpufeature_list[CPUFEATURE_NUMBER] = {
+};
+
+static u32 __init cpufeature_probe(unsigned int stage)
+{
+ const struct cpufeature_info *info;
+ u32 cpu_req_feature = 0;
+ int idx;
+
+ for (idx = 0; idx < CPUFEATURE_NUMBER; idx++) {
+ info = &cpufeature_list[idx];
+
+ if (info->check_func(stage))
+ cpu_req_feature |= (1U << idx);
+ }
+
+ return cpu_req_feature;
+}
+
+void riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end,
+ unsigned int stage)
+{
+ u32 cpu_req_feature = cpufeature_probe(stage);
+ u32 cpu_apply_feature = 0;
+ struct alt_entry *alt;
+ u32 tmp;
+
+ for (alt = begin; alt < end; alt++) {
+ if (alt->vendor_id != 0)
+ continue;
+ if (alt->errata_id >= CPUFEATURE_NUMBER) {
+ WARN(1, "This feature id:%d is not in kernel cpufeature list",
+ alt->errata_id);
+ continue;
+ }
+
+ tmp = (1U << alt->errata_id);
+ if (cpu_req_feature & tmp) {
+ patch_text_nosync(alt->old_ptr, alt->alt_ptr, alt->alt_len);
+ cpu_apply_feature |= tmp;
+ }
+ }
+}
--
2.30.2

2022-01-22 01:58:33

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 02/14] riscv: integrate alternatives better into the main architecture

Right now the alternatives need to be explicitly enabled and
erratas are limited to SiFive ones.

Over time with more SoCs and additional RiscV extensions, many more
erratas or other patch-worthy features will emerge, so it doesn't
really make sense to have the core alternatives able to get
deactivated.

So make it part of the core RiscV kernel and drop the main
RISCV_ERRATA_ALTERNATIVES config symbol.

This also mimics how arm64 handles its alternatives implementation.

Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/Kconfig.erratas | 10 ----------
arch/riscv/Kconfig.socs | 1 -
arch/riscv/Makefile | 2 +-
arch/riscv/errata/Makefile | 1 -
arch/riscv/include/asm/alternative-macros.h | 22 ---------------------
arch/riscv/kernel/Makefile | 1 +
arch/riscv/{errata => kernel}/alternative.c | 0
arch/riscv/kernel/smpboot.c | 2 --
arch/riscv/kernel/traps.c | 2 +-
9 files changed, 3 insertions(+), 38 deletions(-)
rename arch/riscv/{errata => kernel}/alternative.c (100%)

diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas
index b44d6ecdb46e..d18be8ff0245 100644
--- a/arch/riscv/Kconfig.erratas
+++ b/arch/riscv/Kconfig.erratas
@@ -1,17 +1,7 @@
menu "CPU errata selection"

-config RISCV_ERRATA_ALTERNATIVE
- bool "RISC-V alternative scheme"
- default y
- help
- This Kconfig allows the kernel to automatically patch the
- errata required by the execution platform at run time. The
- code patching is performed once in the boot stages. It means
- that the overhead from this mechanism is just taken once.
-
config ERRATA_SIFIVE
bool "SiFive errata"
- depends on RISCV_ERRATA_ALTERNATIVE
help
All SiFive errata Kconfig depend on this Kconfig. Disabling
this Kconfig will disable all SiFive errata. Please say "Y"
diff --git a/arch/riscv/Kconfig.socs b/arch/riscv/Kconfig.socs
index 6ec44a22278a..3df7f7ed0d81 100644
--- a/arch/riscv/Kconfig.socs
+++ b/arch/riscv/Kconfig.socs
@@ -14,7 +14,6 @@ config SOC_SIFIVE
select CLK_SIFIVE
select CLK_SIFIVE_PRCI
select SIFIVE_PLIC
- select RISCV_ERRATA_ALTERNATIVE
select ERRATA_SIFIVE
help
This enables support for SiFive SoC platform hardware.
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 8a107ed18b0d..194220889969 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -97,7 +97,7 @@ endif

head-y := arch/riscv/kernel/head.o

-core-$(CONFIG_RISCV_ERRATA_ALTERNATIVE) += arch/riscv/errata/
+core-y += arch/riscv/errata/
core-$(CONFIG_KVM) += arch/riscv/kvm/

libs-y += arch/riscv/lib/
diff --git a/arch/riscv/errata/Makefile b/arch/riscv/errata/Makefile
index b8f8740a3e44..0ca1c5281a2d 100644
--- a/arch/riscv/errata/Makefile
+++ b/arch/riscv/errata/Makefile
@@ -1,2 +1 @@
-obj-y += alternative.o
obj-$(CONFIG_ERRATA_SIFIVE) += sifive/
diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
index 67406c376389..92da6b3920a3 100644
--- a/arch/riscv/include/asm/alternative-macros.h
+++ b/arch/riscv/include/asm/alternative-macros.h
@@ -2,8 +2,6 @@
#ifndef __ASM_ALTERNATIVE_MACROS_H
#define __ASM_ALTERNATIVE_MACROS_H

-#ifdef CONFIG_RISCV_ERRATA_ALTERNATIVE
-
#ifdef __ASSEMBLY__

.macro ALT_ENTRY oldptr newptr vendor_id errata_id new_len
@@ -76,26 +74,6 @@

#endif /* __ASSEMBLY__ */

-#else /* !CONFIG_RISCV_ERRATA_ALTERNATIVE*/
-#ifdef __ASSEMBLY__
-
-.macro __ALTERNATIVE_CFG old_c
- \old_c
-.endm
-
-#define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
- __ALTERNATIVE_CFG old_c
-
-#else /* !__ASSEMBLY__ */
-
-#define __ALTERNATIVE_CFG(old_c) \
- old_c "\n"
-
-#define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
- __ALTERNATIVE_CFG(old_c)
-
-#endif /* __ASSEMBLY__ */
-#endif /* CONFIG_RISCV_ERRATA_ALTERNATIVE */
/*
* Usage:
* ALTERNATIVE(old_content, new_content, vendor_id, errata_id, CONFIG_k)
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index fb63b462ff85..62d6860af574 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -18,6 +18,7 @@ extra-y += head.o
extra-y += vmlinux.lds

obj-y += soc.o
+obj-y += alternative.o
obj-y += cpu.o
obj-y += cpufeature.o
obj-y += entry.o
diff --git a/arch/riscv/errata/alternative.c b/arch/riscv/kernel/alternative.c
similarity index 100%
rename from arch/riscv/errata/alternative.c
rename to arch/riscv/kernel/alternative.c
diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
index 622f226454d5..a6d13dca1403 100644
--- a/arch/riscv/kernel/smpboot.c
+++ b/arch/riscv/kernel/smpboot.c
@@ -41,9 +41,7 @@ static DECLARE_COMPLETION(cpu_running);
void __init smp_prepare_boot_cpu(void)
{
init_cpu_topology();
-#ifdef CONFIG_RISCV_ERRATA_ALTERNATIVE
apply_boot_alternatives();
-#endif
}

void __init smp_prepare_cpus(unsigned int max_cpus)
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index 0daaa3e4630d..18869912d23d 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -86,7 +86,7 @@ static void do_trap_error(struct pt_regs *regs, int signo, int code,
}
}

-#if defined (CONFIG_XIP_KERNEL) && defined (CONFIG_RISCV_ERRATA_ALTERNATIVE)
+#if defined (CONFIG_XIP_KERNEL)
#define __trap_section __section(".xip.traps")
#else
#define __trap_section
--
2.30.2

2022-01-22 01:58:45

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 08/14] riscv: move boot alternatives to a slightly earlier position

Move the application of boot alternatives to soc_early_init().
This allows to catch more generic cases of code needing patches
than doing it in smp_prepare_boot_cpu() and also makes it actually
work if CONFIG_SMP is disabled for whatever reason.

The position is chosen mainly as it is before the actual soc early
init runs but also already allows accessing the devicetree
via fdt_* functions.

Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/kernel/head.S | 2 ++
arch/riscv/kernel/smpboot.c | 2 --
2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
index db062279d9ed..432fe7b3f897 100644
--- a/arch/riscv/kernel/head.S
+++ b/arch/riscv/kernel/head.S
@@ -10,6 +10,7 @@
#include <asm/thread_info.h>
#include <asm/page.h>
#include <asm/pgtable.h>
+#include <asm/alternative.h>
#include <asm/csr.h>
#include <asm/cpu_ops_sbi.h>
#include <asm/hwcap.h>
@@ -341,6 +342,7 @@ clear_bss_done:
call kasan_early_init
#endif
/* Start the kernel */
+ call apply_boot_alternatives
call soc_early_init
tail start_kernel

diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
index a6d13dca1403..f1e4948a4b52 100644
--- a/arch/riscv/kernel/smpboot.c
+++ b/arch/riscv/kernel/smpboot.c
@@ -32,7 +32,6 @@
#include <asm/sections.h>
#include <asm/sbi.h>
#include <asm/smp.h>
-#include <asm/alternative.h>

#include "head.h"

@@ -41,7 +40,6 @@ static DECLARE_COMPLETION(cpu_running);
void __init smp_prepare_boot_cpu(void)
{
init_cpu_topology();
- apply_boot_alternatives();
}

void __init smp_prepare_cpus(unsigned int max_cpus)
--
2.30.2

2022-01-22 01:58:53

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 01/14] riscv: only use IPIs to handle cache-flushes on remote cpus

Right now, the flush_icache functions always use the SBI remote-fence
when SBI is available, leaving using IPIs as a fallback mechanism.

IPIs on the other hand are more flexible, as the ipi_ops are initially
set to go through SBI but later will be overwritten to go through the
ACLINT/CLINT.

In a discussion we had, Nick was of the opinion that "In general we
should prefer doing IPIs on S-mode through CLINT instead of going
through SBI/M-mode, so IMHO we should only be using
on_each_cpu_mask(ipi_remote_fence_i) on flush_icache_all()/
flush_icache_mm() and remove any explicit calls to sbi_remote_fence_i(),
because this way we continue using SBI for doing remote fences even after
CLINT/ACLINT driver is registered, instead of using direct IPIs through
CLINT/ACLINT."

So follow this suggestion and just do ipi calls to have the proper kernel
parts do them,

This also fixes the null-ptr dereference happening when flush_icache_all()
is called before sbi_init().

Suggested-by: Nick Kossifidis <[email protected]>
Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/mm/cacheflush.c | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c
index 6cb7d96ad9c7..c35375cd52ec 100644
--- a/arch/riscv/mm/cacheflush.c
+++ b/arch/riscv/mm/cacheflush.c
@@ -17,11 +17,7 @@ static void ipi_remote_fence_i(void *info)
void flush_icache_all(void)
{
local_flush_icache_all();
-
- if (IS_ENABLED(CONFIG_RISCV_SBI))
- sbi_remote_fence_i(NULL);
- else
- on_each_cpu(ipi_remote_fence_i, NULL, 1);
+ on_each_cpu(ipi_remote_fence_i, NULL, 1);
}
EXPORT_SYMBOL(flush_icache_all);

@@ -66,8 +62,6 @@ void flush_icache_mm(struct mm_struct *mm, bool local)
* with flush_icache_deferred().
*/
smp_mb();
- } else if (IS_ENABLED(CONFIG_RISCV_SBI)) {
- sbi_remote_fence_i(&others);
} else {
on_each_cpu_mask(&others, ipi_remote_fence_i, NULL, 1);
}
--
2.30.2

2022-01-22 01:58:56

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 03/14] riscv: allow different stages with alternatives

Future features may need to be applied at a different
time during boot, so allow defining stages for alternatives
and handling them differently depending on the stage.

Also make the alternatives-location more flexible so that
future stages may provide their own location.

Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/errata/sifive/errata.c | 3 ++-
arch/riscv/include/asm/alternative.h | 5 ++++-
arch/riscv/kernel/alternative.c | 25 +++++++++++++++++--------
3 files changed, 23 insertions(+), 10 deletions(-)

diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c
index f5e5ae70e829..4fe03ac41fd7 100644
--- a/arch/riscv/errata/sifive/errata.c
+++ b/arch/riscv/errata/sifive/errata.c
@@ -80,7 +80,8 @@ static void __init warn_miss_errata(u32 miss_errata)
}

void __init sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
- unsigned long archid, unsigned long impid)
+ unsigned long archid, unsigned long impid,
+ unsigned int stage)
{
struct alt_entry *alt;
u32 cpu_req_errata = sifive_errata_probe(archid, impid);
diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
index e625d3cafbed..811bdd8027db 100644
--- a/arch/riscv/include/asm/alternative.h
+++ b/arch/riscv/include/asm/alternative.h
@@ -17,6 +17,8 @@
#include <linux/stddef.h>
#include <asm/hwcap.h>

+#define RISCV_ALTERNATIVES_BOOT 0 /* alternatives applied during regular boot */
+
void __init apply_boot_alternatives(void);

struct alt_entry {
@@ -33,7 +35,8 @@ struct errata_checkfunc_id {
};

void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
- unsigned long archid, unsigned long impid);
+ unsigned long archid, unsigned long impid,
+ unsigned int stage);

#endif
#endif
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index 3b15885db70b..ead00c508266 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -22,7 +22,8 @@ static struct cpu_manufacturer_info_t {
} cpu_mfr_info;

static void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
- unsigned long archid, unsigned long impid);
+ unsigned long archid, unsigned long impid,
+ unsigned int stage);

static inline void __init riscv_fill_cpu_mfr_info(void)
{
@@ -57,6 +58,18 @@ static void __init init_alternative(void)
* a feature detect on the boot CPU). No need to worry about other CPUs
* here.
*/
+static void __init _apply_alternatives(struct alt_entry *begin,
+ struct alt_entry *end,
+ unsigned int stage)
+{
+ if (!vendor_patch_func)
+ return;
+
+ vendor_patch_func(begin, end,
+ cpu_mfr_info.arch_id, cpu_mfr_info.imp_id,
+ stage);
+}
+
void __init apply_boot_alternatives(void)
{
/* If called on non-boot cpu things could go wrong */
@@ -64,11 +77,7 @@ void __init apply_boot_alternatives(void)

init_alternative();

- if (!vendor_patch_func)
- return;
-
- vendor_patch_func((struct alt_entry *)__alt_start,
- (struct alt_entry *)__alt_end,
- cpu_mfr_info.arch_id, cpu_mfr_info.imp_id);
+ _apply_alternatives((struct alt_entry *)__alt_start,
+ (struct alt_entry *)__alt_end,
+ RISCV_ALTERNATIVES_BOOT);
}
-
--
2.30.2

2022-01-22 01:59:02

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 07/14] riscv: prevent compressed instructions in alternatives

Instructions are opportunistically compressed by the RISC-V assembler
when possible, but in alternatives-blocks both the old and new content
need to be the same size, so having the toolchain do somewhat random
optimizations will cause strange side-effects like
"attempt to move .org backwards" compile-time errors.

Already a simple "and" used in alternatives assembly will cause these
mismatched code sizes.

So prevent compressed instructions to be generated in alternatives-
code and use option-push and -pop to limit this to the relevant
code blocks

Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/include/asm/alternative-macros.h | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)

diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
index c0fb11fad631..3a52884bf23d 100644
--- a/arch/riscv/include/asm/alternative-macros.h
+++ b/arch/riscv/include/asm/alternative-macros.h
@@ -19,7 +19,10 @@
.popsection
.subsection 1
888 :
+ .option push
+ .option norvc
\new_c
+ .option pop
889 :
.previous
.org . - (889b - 888b) + (887b - 886b)
@@ -29,7 +32,10 @@

.macro __ALTERNATIVE_CFG old_c, new_c, vendor_id, errata_id, enable
886 :
+ .option push
+ .option norvc
\old_c
+ .option pop
887 :
ALT_NEW_CONTENT \vendor_id, \errata_id, \enable, \new_c
.endm
@@ -40,7 +46,10 @@
.macro __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
new_c_2, vendor_id_2, errata_id_2, enable_2
886 :
+ .option push
+ .option norvc
\old_c
+ .option pop
887 :
ALT_NEW_CONTENT \vendor_id_1, \errata_id_1, \enable_1, \new_c_1
ALT_NEW_CONTENT \vendor_id_2, \errata_id_2, \enable_2, \new_c_2
@@ -70,7 +79,10 @@
".popsection\n" \
".subsection 1\n" \
"888 :\n" \
+ ".option push\n" \
+ ".option norvc\n" \
new_c "\n" \
+ ".option pop\n" \
"889 :\n" \
".previous\n" \
".org . - (887b - 886b) + (889b - 888b)\n" \
@@ -79,7 +91,10 @@

#define __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, enable) \
"886 :\n" \
+ ".option push\n" \
+ ".option norvc\n" \
old_c "\n" \
+ ".option pop\n" \
"887 :\n" \
ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c)

@@ -89,7 +104,10 @@
#define __ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
new_c_2, vendor_id_2, errata_id_2, enable_2) \
"886 :\n" \
+ ".option push\n" \
+ ".option norvc\n" \
old_c "\n" \
+ ".option pop\n" \
"887 :\n" \
ALT_NEW_CONTENT(vendor_id_1, errata_id_1, enable_1, new_c_1) \
ALT_NEW_CONTENT(vendor_id_2, errata_id_2, enable_2, new_c_2)
--
2.30.2

2022-01-22 01:59:10

by Heiko Stuebner

[permalink] [raw]
Subject: [PATCH v5 14/14] riscv: add memory-type errata for T-Head

Some current cpus based on T-Head cores implement memory-types
way different than described in the svpbmt spec even going
so far as using PTE bits marked as reserved.

Add the T-Head vendor-id and necessary errata code to
replace the affected instructions.

Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/Kconfig.erratas | 19 ++++++
arch/riscv/errata/Makefile | 1 +
arch/riscv/errata/sifive/errata.c | 7 ++-
arch/riscv/errata/thead/Makefile | 1 +
arch/riscv/errata/thead/errata.c | 85 ++++++++++++++++++++++++++
arch/riscv/include/asm/alternative.h | 5 ++
arch/riscv/include/asm/errata_list.h | 47 ++++++++++++--
arch/riscv/include/asm/pgtable-64.h | 18 +++++-
arch/riscv/include/asm/pgtable.h | 18 +++++-
arch/riscv/include/asm/vendorid_list.h | 1 +
arch/riscv/kernel/alternative.c | 14 +++++
arch/riscv/kernel/cpufeature.c | 2 +
arch/riscv/mm/init.c | 2 +
13 files changed, 211 insertions(+), 9 deletions(-)
create mode 100644 arch/riscv/errata/thead/Makefile
create mode 100644 arch/riscv/errata/thead/errata.c

diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas
index d18be8ff0245..380ec039c3dc 100644
--- a/arch/riscv/Kconfig.erratas
+++ b/arch/riscv/Kconfig.erratas
@@ -31,4 +31,23 @@ config ERRATA_SIFIVE_CIP_1200

If you don't know what to do here, say "Y".

+config ERRATA_THEAD
+ bool "T-HEAD errata"
+ help
+ All T-HEAD errata Kconfig depend on this Kconfig. Disabling
+ this Kconfig will disable all T-HEAD errata. Please say "Y"
+ here if your platform uses T-HEAD CPU cores.
+
+ If you don't know what to do here, say "Y".
+
+config ERRATA_THEAD_PBMT
+ bool "Apply T-Head memory type errata"
+ depends on ERRATA_THEAD && 64BIT
+ default y
+ help
+ This will apply the memory type errata to handle the non-standard
+ memory type bits in page-table-entries on T-Head SoCs.
+
+ If you don't know what to do here, say "Y".
+
endmenu
diff --git a/arch/riscv/errata/Makefile b/arch/riscv/errata/Makefile
index 0ca1c5281a2d..a1055965fbee 100644
--- a/arch/riscv/errata/Makefile
+++ b/arch/riscv/errata/Makefile
@@ -1 +1,2 @@
obj-$(CONFIG_ERRATA_SIFIVE) += sifive/
+obj-$(CONFIG_ERRATA_THEAD) += thead/
diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c
index 4fe03ac41fd7..f933d6cdf304 100644
--- a/arch/riscv/errata/sifive/errata.c
+++ b/arch/riscv/errata/sifive/errata.c
@@ -84,10 +84,15 @@ void __init sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *
unsigned int stage)
{
struct alt_entry *alt;
- u32 cpu_req_errata = sifive_errata_probe(archid, impid);
+ u32 cpu_req_errata;
u32 cpu_apply_errata = 0;
u32 tmp;

+ if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
+ return;
+
+ cpu_req_errata = sifive_errata_probe(archid, impid);
+
for (alt = begin; alt < end; alt++) {
if (alt->vendor_id != SIFIVE_VENDOR_ID)
continue;
diff --git a/arch/riscv/errata/thead/Makefile b/arch/riscv/errata/thead/Makefile
new file mode 100644
index 000000000000..2d644e19caef
--- /dev/null
+++ b/arch/riscv/errata/thead/Makefile
@@ -0,0 +1 @@
+obj-y += errata.o
diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
new file mode 100644
index 000000000000..fd8e0538a3f0
--- /dev/null
+++ b/arch/riscv/errata/thead/errata.c
@@ -0,0 +1,85 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021 Heiko Stuebner <[email protected]>
+ */
+
+#include <linux/bug.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/string.h>
+#include <linux/uaccess.h>
+#include <asm/alternative.h>
+#include <asm/cacheflush.h>
+#include <asm/errata_list.h>
+#include <asm/patch.h>
+#include <asm/vendorid_list.h>
+
+struct errata_info {
+ char name[ERRATA_STRING_LENGTH_MAX];
+ bool (*check_func)(unsigned long arch_id, unsigned long impid);
+ unsigned int stage;
+};
+
+static bool errata_mt_check_func(unsigned long arch_id, unsigned long impid)
+{
+ if (arch_id != 0 || impid != 0)
+ return false;
+ return true;
+}
+
+static const struct errata_info errata_list[ERRATA_THEAD_NUMBER] = {
+ {
+ .name = "memory-types",
+ .stage = RISCV_ALTERNATIVES_EARLY_BOOT,
+ .check_func = errata_mt_check_func
+ },
+};
+
+static u32 thead_errata_probe(unsigned int stage, unsigned long archid, unsigned long impid)
+{
+ const struct errata_info *info;
+ u32 cpu_req_errata = 0;
+ int idx;
+
+ for (idx = 0; idx < ERRATA_THEAD_NUMBER; idx++) {
+ info = &errata_list[idx];
+
+ if ((stage == RISCV_ALTERNATIVES_MODULE ||
+ info->stage == stage) && info->check_func(archid, impid))
+ cpu_req_errata |= (1U << idx);
+ }
+
+ return cpu_req_errata;
+}
+
+void __init_or_module thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
+ unsigned long archid, unsigned long impid,
+ unsigned int stage)
+{
+ struct alt_entry *alt;
+ u32 cpu_req_errata = thead_errata_probe(stage, archid, impid);
+ u32 cpu_apply_errata = 0;
+ u32 tmp;
+
+ for (alt = begin; alt < end; alt++) {
+ if (alt->vendor_id != THEAD_VENDOR_ID)
+ continue;
+ if (alt->errata_id >= ERRATA_THEAD_NUMBER)
+ continue;
+
+ tmp = (1U << alt->errata_id);
+ if (cpu_req_errata & tmp) {
+ /* On vm-alternatives, the mmu isn't running yet */
+ if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
+ memcpy((void *)__pa_symbol(alt->old_ptr),
+ (void *)__pa_symbol(alt->alt_ptr), alt->alt_len);
+ else
+ patch_text_nosync(alt->old_ptr, alt->alt_ptr, alt->alt_len);
+
+ cpu_apply_errata |= tmp;
+ }
+ }
+
+ if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
+ local_flush_icache_all();
+}
diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
index cf3b22173834..d1154c91ab03 100644
--- a/arch/riscv/include/asm/alternative.h
+++ b/arch/riscv/include/asm/alternative.h
@@ -19,8 +19,10 @@

#define RISCV_ALTERNATIVES_BOOT 0 /* alternatives applied during regular boot */
#define RISCV_ALTERNATIVES_MODULE 1 /* alternatives applied during module-init */
+#define RISCV_ALTERNATIVES_EARLY_BOOT 2 /* alternatives applied before mmu start */

void __init apply_boot_alternatives(void);
+void __init apply_early_boot_alternatives(void);
void apply_module_alternatives(void *start, size_t length);

struct alt_entry {
@@ -39,6 +41,9 @@ struct errata_checkfunc_id {
void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
unsigned long archid, unsigned long impid,
unsigned int stage);
+void thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
+ unsigned long archid, unsigned long impid,
+ unsigned int stage);

void riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end,
unsigned int stage);
diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
index a4a9b0842922..4fac46b82c16 100644
--- a/arch/riscv/include/asm/errata_list.h
+++ b/arch/riscv/include/asm/errata_list.h
@@ -14,6 +14,11 @@
#define ERRATA_SIFIVE_NUMBER 2
#endif

+#ifdef CONFIG_ERRATA_THEAD
+#define ERRATA_THEAD_PBMT 0
+#define ERRATA_THEAD_NUMBER 1
+#endif
+
#define CPUFEATURE_SVPBMT 0
#define CPUFEATURE_NUMBER 1

@@ -42,10 +47,44 @@ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \
* in the default case.
*/
#define ALT_SVPBMT_SHIFT 61
-#define ALT_SVPBMT(_val, prot) \
-asm(ALTERNATIVE("li %0, 0\t\nnop", "li %0, %1\t\nslli %0,%0,%2", 0, \
- CPUFEATURE_SVPBMT, CONFIG_64BIT) \
- : "=r"(_val) : "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT), "I"(ALT_SVPBMT_SHIFT))
+#define ALT_THEAD_PBMT_SHIFT 59
+#define ALT_SVPBMT(_val, prot) \
+asm(ALTERNATIVE_2("li %0, 0\t\nnop", \
+ "li %0, %1\t\nslli %0,%0,%3", 0, CPUFEATURE_SVPBMT, CONFIG_64BIT, \
+ "li %0, %2\t\nslli %0,%0,%4", THEAD_VENDOR_ID, ERRATA_THEAD_PBMT, \
+ CONFIG_ERRATA_THEAD_PBMT) \
+ : "=r"(_val) : "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT), \
+ "I"(prot##_THEAD >> ALT_THEAD_PBMT_SHIFT), \
+ "I"(ALT_SVPBMT_SHIFT), "I"(ALT_THEAD_PBMT_SHIFT))
+
+#ifdef CONFIG_ERRATA_THEAD_PBMT
+/*
+ * IO/NOCACHE memory types are handled together with svpbmt,
+ * so on T-Head chips, check if no other memory type is set,
+ * and set the non-0 PMA type if applicable.
+ */
+#define ALT_THEAD_PMA(_val) \
+asm volatile(ALTERNATIVE( \
+ "nop\n\t" \
+ "nop\n\t" \
+ "nop\n\t" \
+ "nop\n\t" \
+ "nop\n\t" \
+ "nop\n\t" \
+ "nop", \
+ "li t3, %2\n\t" \
+ "slli t3, t3, %4\n\t" \
+ "and t3, %0, t3\n\t" \
+ "bne t3, zero, 2f\n\t" \
+ "li t3, %3\n\t" \
+ "slli t3, t3, %4\n\t" \
+ "or %0, %0, t3\n\t" \
+ "2:", THEAD_VENDOR_ID, ERRATA_THEAD_PBMT, CONFIG_ERRATA_THEAD_PBMT) \
+ : "+r"(_val) : "0"(_val), "I"(_PAGE_MTMASK_THEAD >> ALT_THEAD_PBMT_SHIFT), \
+ "I"(_PAGE_PMA_THEAD >> ALT_THEAD_PBMT_SHIFT), "I"(ALT_THEAD_PBMT_SHIFT))
+#else
+#define ALT_THEAD_PMA(_val)
+#endif

#endif /* __ASSEMBLY__ */

diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index 58b34ef74acb..e7ce01cdc235 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -49,6 +49,18 @@ typedef struct {
#define _PAGE_IO_SVPBMT (1UL << 62)
#define _PAGE_MTMASK_SVPBMT (_PAGE_NOCACHE_SVPBMT | _PAGE_IO_SVPBMT)

+/*
+ * [63:59] T-Head Memory Type definitions:
+ *
+ * 00000 - NC Weakly-ordered, Non-cacheable, Non-bufferable, Non-shareable, Non-trustable
+ * 01110 - PMA Weakly-ordered, Cacheable, Bufferable, Shareable, Non-trustable
+ * 10000 - IO Strongly-ordered, Non-cacheable, Non-bufferable, Non-shareable, Non-trustable
+ */
+#define _PAGE_PMA_THEAD ((1UL << 62) | (1UL << 61) | (1UL << 60))
+#define _PAGE_NOCACHE_THEAD 0UL
+#define _PAGE_IO_THEAD (1UL << 63)
+#define _PAGE_MTMASK_THEAD (_PAGE_PMA_THEAD | _PAGE_IO_THEAD | (1UL << 59))
+
static inline u64 riscv_page_mtmask(void)
{
u64 val;
@@ -126,7 +138,11 @@ static inline struct page *pud_page(pud_t pud)

static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
{
- return __pmd((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
+ unsigned long prot_val = pgprot_val(prot);
+
+ ALT_THEAD_PMA(prot_val);
+
+ return __pmd((pfn << _PAGE_PFN_SHIFT) | prot_val);
}

static inline unsigned long _pmd_pfn(pmd_t pmd)
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index a189a21a7c8a..c3b9220b2523 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -219,7 +219,11 @@ static inline void pmd_clear(pmd_t *pmdp)

static inline pgd_t pfn_pgd(unsigned long pfn, pgprot_t prot)
{
- return __pgd((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
+ unsigned long prot_val = pgprot_val(prot);
+
+ ALT_THEAD_PMA(prot_val);
+
+ return __pgd((pfn << _PAGE_PFN_SHIFT) | prot_val);
}

static inline unsigned long _pgd_pfn(pgd_t pgd)
@@ -258,7 +262,11 @@ static inline unsigned long pte_pfn(pte_t pte)
/* Constructs a page table entry */
static inline pte_t pfn_pte(unsigned long pfn, pgprot_t prot)
{
- return __pte((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
+ unsigned long prot_val = pgprot_val(prot);
+
+ ALT_THEAD_PMA(prot_val);
+
+ return __pte((pfn << _PAGE_PFN_SHIFT) | prot_val);
}

#define mk_pte(page, prot) pfn_pte(page_to_pfn(page), prot)
@@ -367,7 +375,11 @@ static inline int pmd_protnone(pmd_t pmd)
/* Modify page protection bits */
static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
{
- return __pte((pte_val(pte) & _PAGE_CHG_MASK) | pgprot_val(newprot));
+ unsigned long newprot_val = pgprot_val(newprot);
+
+ ALT_THEAD_PMA(newprot_val);
+
+ return __pte((pte_val(pte) & _PAGE_CHG_MASK) | newprot_val);
}

#define pgd_ERROR(e) \
diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
index 9d934215b3c8..cb89af3f0704 100644
--- a/arch/riscv/include/asm/vendorid_list.h
+++ b/arch/riscv/include/asm/vendorid_list.h
@@ -6,5 +6,6 @@
#define ASM_VENDOR_LIST_H

#define SIFIVE_VENDOR_ID 0x489
+#define THEAD_VENDOR_ID 0x5b7

#endif
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index 54913857b682..e272599f96ef 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -48,6 +48,11 @@ static void __init init_alternative(void)
case SIFIVE_VENDOR_ID:
vendor_patch_func = sifive_errata_patch_func;
break;
+#endif
+#ifdef CONFIG_ERRATA_THEAD
+ case THEAD_VENDOR_ID:
+ vendor_patch_func = thead_errata_patch_func;
+ break;
#endif
default:
vendor_patch_func = NULL;
@@ -85,6 +90,15 @@ void __init apply_boot_alternatives(void)
RISCV_ALTERNATIVES_BOOT);
}

+void __init apply_early_boot_alternatives(void)
+{
+ init_alternative();
+
+ _apply_alternatives((struct alt_entry *)__alt_start,
+ (struct alt_entry *)__alt_end,
+ RISCV_ALTERNATIVES_EARLY_BOOT);
+}
+
#ifdef CONFIG_MODULES
void apply_module_alternatives(void *start, size_t length)
{
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index d14289fba055..4ed18981d94d 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -224,6 +224,8 @@ static bool cpufeature_svpbmt_check_func(unsigned int stage)

#if defined(CONFIG_MMU) && defined(CONFIG_64BIT)
switch (stage) {
+ case RISCV_ALTERNATIVES_EARLY_BOOT:
+ return false;
case RISCV_ALTERNATIVES_BOOT:
return cpufeature_svpbmt_check_fdt();
default:
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index 24b2b8044602..063d5f022a05 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -624,6 +624,8 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa)
BUG_ON((kernel_map.virt_addr + kernel_map.size) > ADDRESS_SPACE_END - SZ_4K);
#endif

+ apply_early_boot_alternatives();
+
pt_ops.alloc_pte = alloc_pte_early;
pt_ops.get_pte_virt = get_pte_virt_early;
#ifndef __PAGETABLE_PMD_FOLDED
--
2.30.2

2022-01-23 00:14:48

by Atish Patra

[permalink] [raw]
Subject: Re: [PATCH v5 01/14] riscv: only use IPIs to handle cache-flushes on remote cpus

On Fri, Jan 21, 2022 at 8:37 AM Heiko Stuebner <[email protected]> wrote:
>
> Right now, the flush_icache functions always use the SBI remote-fence
> when SBI is available, leaving using IPIs as a fallback mechanism.
>
> IPIs on the other hand are more flexible, as the ipi_ops are initially
> set to go through SBI but later will be overwritten to go through the
> ACLINT/CLINT.
>
> In a discussion we had, Nick was of the opinion that "In general we
> should prefer doing IPIs on S-mode through CLINT instead of going
> through SBI/M-mode,

Yes. Once Anup's ACLINT drivers are merged, that should be the
preferred approach.

https://github.com/avpatel/linux/commit/416c667fd77d6f1fc310cbf727ec127aaf96cae2

>so IMHO we should only be using
> on_each_cpu_mask(ipi_remote_fence_i) on flush_icache_all()/
> flush_icache_mm() and remove any explicit calls to sbi_remote_fence_i(),

That's a bit confusing because we will be using SBI calls for all other fences
while using IPIs for fence.i

> because this way we continue using SBI for doing remote fences even after
> CLINT/ACLINT driver is registered, instead of using direct IPIs through
> CLINT/ACLINT."
>
> So follow this suggestion and just do ipi calls to have the proper kernel
> parts do them,
>
> This also fixes the null-ptr dereference happening when flush_icache_all()
> is called before sbi_init().
>

IMHO, this series should only fix the null-ptr dereference issue.
The IPI based fence (for all) should only be disabled along with the
ACLINT driver
that actually enables S-mode IPIs.

> Suggested-by: Nick Kossifidis <[email protected]>
> Signed-off-by: Heiko Stuebner <[email protected]>
> ---
> arch/riscv/mm/cacheflush.c | 8 +-------
> 1 file changed, 1 insertion(+), 7 deletions(-)
>
> diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c
> index 6cb7d96ad9c7..c35375cd52ec 100644
> --- a/arch/riscv/mm/cacheflush.c
> +++ b/arch/riscv/mm/cacheflush.c
> @@ -17,11 +17,7 @@ static void ipi_remote_fence_i(void *info)
> void flush_icache_all(void)
> {
> local_flush_icache_all();
> -
> - if (IS_ENABLED(CONFIG_RISCV_SBI))
> - sbi_remote_fence_i(NULL);
> - else
> - on_each_cpu(ipi_remote_fence_i, NULL, 1);
> + on_each_cpu(ipi_remote_fence_i, NULL, 1);
> }
> EXPORT_SYMBOL(flush_icache_all);
>
> @@ -66,8 +62,6 @@ void flush_icache_mm(struct mm_struct *mm, bool local)
> * with flush_icache_deferred().
> */
> smp_mb();
> - } else if (IS_ENABLED(CONFIG_RISCV_SBI)) {
> - sbi_remote_fence_i(&others);
> } else {
> on_each_cpu_mask(&others, ipi_remote_fence_i, NULL, 1);
> }
> --
> 2.30.2
>
>
> _______________________________________________
> linux-riscv mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-riscv



--
Regards,
Atish

2022-01-23 00:16:24

by Anup Patel

[permalink] [raw]
Subject: Re: [PATCH v5 01/14] riscv: only use IPIs to handle cache-flushes on remote cpus

On Fri, Jan 21, 2022 at 10:07 PM Heiko Stuebner <[email protected]> wrote:
>
> Right now, the flush_icache functions always use the SBI remote-fence
> when SBI is available, leaving using IPIs as a fallback mechanism.
>
> IPIs on the other hand are more flexible, as the ipi_ops are initially
> set to go through SBI but later will be overwritten to go through the
> ACLINT/CLINT.
>
> In a discussion we had, Nick was of the opinion that "In general we
> should prefer doing IPIs on S-mode through CLINT instead of going
> through SBI/M-mode, so IMHO we should only be using
> on_each_cpu_mask(ipi_remote_fence_i) on flush_icache_all()/
> flush_icache_mm() and remove any explicit calls to sbi_remote_fence_i(),
> because this way we continue using SBI for doing remote fences even after
> CLINT/ACLINT driver is registered, instead of using direct IPIs through
> CLINT/ACLINT."
>
> So follow this suggestion and just do ipi calls to have the proper kernel
> parts do them,
>
> This also fixes the null-ptr dereference happening when flush_icache_all()
> is called before sbi_init().
>
> Suggested-by: Nick Kossifidis <[email protected]>
> Signed-off-by: Heiko Stuebner <[email protected]>

For Guest/VM, only virtual IMSIC provides faster IPIs so in absence of
virtual IMSIC, the SBI IPI based IPIs are the best approach.

Like Atish mentioned, please base this work on the ACLINT series
because the ACLINT series adds required IPI infrastructure which helps
us select SBI IPI versus direct S-mode IPI based on hardware capability.

Regards,
Anup

> ---
> arch/riscv/mm/cacheflush.c | 8 +-------
> 1 file changed, 1 insertion(+), 7 deletions(-)
>
> diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c
> index 6cb7d96ad9c7..c35375cd52ec 100644
> --- a/arch/riscv/mm/cacheflush.c
> +++ b/arch/riscv/mm/cacheflush.c
> @@ -17,11 +17,7 @@ static void ipi_remote_fence_i(void *info)
> void flush_icache_all(void)
> {
> local_flush_icache_all();
> -
> - if (IS_ENABLED(CONFIG_RISCV_SBI))
> - sbi_remote_fence_i(NULL);
> - else
> - on_each_cpu(ipi_remote_fence_i, NULL, 1);
> + on_each_cpu(ipi_remote_fence_i, NULL, 1);
> }
> EXPORT_SYMBOL(flush_icache_all);
>
> @@ -66,8 +62,6 @@ void flush_icache_mm(struct mm_struct *mm, bool local)
> * with flush_icache_deferred().
> */
> smp_mb();
> - } else if (IS_ENABLED(CONFIG_RISCV_SBI)) {
> - sbi_remote_fence_i(&others);
> } else {
> on_each_cpu_mask(&others, ipi_remote_fence_i, NULL, 1);
> }
> --
> 2.30.2
>

2022-01-24 18:04:41

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH v5 00/14] riscv: support for svpbmt and D1 memory types

On Fri, Jan 21, 2022 at 05:36:04PM +0100, Heiko Stuebner wrote:
> So this is my try at implementing svpbmt (and the diverging D1 memory
> types using the alternatives framework).

Please spell out what it does. While I can remember the original single
letter extensions very well, the new priviliged extension naming is
just letter salad.

You might also want to explain in more detail what it does and why you
want to support it.

2022-01-24 19:12:51

by Heiko Stuebner

[permalink] [raw]
Subject: Re: [PATCH v5 01/14] riscv: only use IPIs to handle cache-flushes on remote cpus

Hi Atish,

Am Samstag, 22. Januar 2022, 04:45:52 CET schrieb Atish Patra:
> On Fri, Jan 21, 2022 at 8:37 AM Heiko Stuebner <[email protected]> wrote:
> >
> > Right now, the flush_icache functions always use the SBI remote-fence
> > when SBI is available, leaving using IPIs as a fallback mechanism.
> >
> > IPIs on the other hand are more flexible, as the ipi_ops are initially
> > set to go through SBI but later will be overwritten to go through the
> > ACLINT/CLINT.
> >
> > In a discussion we had, Nick was of the opinion that "In general we
> > should prefer doing IPIs on S-mode through CLINT instead of going
> > through SBI/M-mode,
>
> Yes. Once Anup's ACLINT drivers are merged, that should be the
> preferred approach.
>
> https://github.com/avpatel/linux/commit/416c667fd77d6f1fc310cbf727ec127aaf96cae2
>
> >so IMHO we should only be using
> > on_each_cpu_mask(ipi_remote_fence_i) on flush_icache_all()/
> > flush_icache_mm() and remove any explicit calls to sbi_remote_fence_i(),
>
> That's a bit confusing because we will be using SBI calls for all other fences
> while using IPIs for fence.i
>
> > because this way we continue using SBI for doing remote fences even after
> > CLINT/ACLINT driver is registered, instead of using direct IPIs through
> > CLINT/ACLINT."
> >
> > So follow this suggestion and just do ipi calls to have the proper kernel
> > parts do them,
> >
> > This also fixes the null-ptr dereference happening when flush_icache_all()
> > is called before sbi_init().
> >
>
> IMHO, this series should only fix the null-ptr dereference issue.
> The IPI based fence (for all) should only be disabled along with the
> ACLINT driver
> that actually enables S-mode IPIs.

ok, I'll roll this back to simply fixing the null-ptr issue.

Meanwhile I even found a nicer solution without actually touching
the cachflush code.

Without sbi_init() we can assume that we're still before smp bringup,
so the local_flush_icache_all() in flush_icache_all() will do the trick
just fine and sbi_remote_fence_i() simply just needs to be an empty
function until sbi is initialized.


Heiko

>
> > Suggested-by: Nick Kossifidis <[email protected]>
> > Signed-off-by: Heiko Stuebner <[email protected]>
> > ---
> > arch/riscv/mm/cacheflush.c | 8 +-------
> > 1 file changed, 1 insertion(+), 7 deletions(-)
> >
> > diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c
> > index 6cb7d96ad9c7..c35375cd52ec 100644
> > --- a/arch/riscv/mm/cacheflush.c
> > +++ b/arch/riscv/mm/cacheflush.c
> > @@ -17,11 +17,7 @@ static void ipi_remote_fence_i(void *info)
> > void flush_icache_all(void)
> > {
> > local_flush_icache_all();
> > -
> > - if (IS_ENABLED(CONFIG_RISCV_SBI))
> > - sbi_remote_fence_i(NULL);
> > - else
> > - on_each_cpu(ipi_remote_fence_i, NULL, 1);
> > + on_each_cpu(ipi_remote_fence_i, NULL, 1);
> > }
> > EXPORT_SYMBOL(flush_icache_all);
> >
> > @@ -66,8 +62,6 @@ void flush_icache_mm(struct mm_struct *mm, bool local)
> > * with flush_icache_deferred().
> > */
> > smp_mb();
> > - } else if (IS_ENABLED(CONFIG_RISCV_SBI)) {
> > - sbi_remote_fence_i(&others);
> > } else {
> > on_each_cpu_mask(&others, ipi_remote_fence_i, NULL, 1);
> > }
> > --
> > 2.30.2
> >
> >
> > _______________________________________________
> > linux-riscv mailing list
> > [email protected]
> > http://lists.infradead.org/mailman/listinfo/linux-riscv
>
>
>
>




2022-02-09 07:21:42

by Heiko Stuebner

[permalink] [raw]
Subject: Re: [PATCH v5 11/14] dt-bindings: riscv: add MMU Standard Extensions support for Svpbmt

Am Freitag, 4. Februar 2022, 23:33:42 CET schrieb Rob Herring:
> On Fri, Jan 21, 2022 at 05:36:15PM +0100, Heiko Stuebner wrote:
> > From: Wei Fu <[email protected]>
> >
> > Previous patch has added svpbmt in arch/riscv and add "riscv,svpmbt"
> > in the DT mmu node. Update dt-bindings related property here.
> >
> > Signed-off-by: Wei Fu <[email protected]>
> > Co-developed-by: Guo Ren <[email protected]>
> > Signed-off-by: Guo Ren <[email protected]>
> > Signed-off-by: Heiko Stuebner <[email protected]>
> > Cc: Anup Patel <[email protected]>
> > Cc: Palmer Dabbelt <[email protected]>
> > Cc: Rob Herring <[email protected]>
> > ---
> > Documentation/devicetree/bindings/riscv/cpus.yaml | 10 ++++++++++
> > 1 file changed, 10 insertions(+)
> >
> > diff --git a/Documentation/devicetree/bindings/riscv/cpus.yaml b/Documentation/devicetree/bindings/riscv/cpus.yaml
> > index aa5fb64d57eb..3ad2593f1400 100644
> > --- a/Documentation/devicetree/bindings/riscv/cpus.yaml
> > +++ b/Documentation/devicetree/bindings/riscv/cpus.yaml
> > @@ -63,6 +63,16 @@ properties:
> > - riscv,sv48
> > - riscv,none
> >
> > + mmu:
>
> riscv,mmu
>
> > + description:
> > + Describes the CPU's MMU Standard Extensions support.
> > + These values originate from the RISC-V Privileged
> > + Specification document, available from
> > + https://riscv.org/specifications/
> > + $ref: '/schemas/types.yaml#/definitions/string'
> > + enum:
> > + - riscv,svpbmt
>
> Are there per vendor MMU extensions? If not, drop the 'riscv,' part.

Judging by the somewhat wild-west nature, I guess there might already
be non-riscv extensions existing somewhere, or at least the probability
is quite high that there will be in the future ;-)

>
> > +
> > riscv,isa:
> > description:
> > Identifies the specific RISC-V instruction set architecture
>