Svpbmt is an extension defining "Supervisor-mode: page-based memory types"
for things like non-cacheable pages or I/O memory pages.
So this is my 2nd try at implementing Svpbmt (and the diverging D1 memory
types) using the alternatives framework.
This includes a number of changes to the alternatives mechanism itself.
The biggest one being the move to a more central location, as I expect
in the future, nearly every chip needing some sort of patching, be it
either for erratas or for optional features (svpbmt or others).
Detection of the svpbmt functionality is done via Atish's isa extension
handling series [0] and thus does not need any dt-parsing of its own
anymore.
The series also introduces support for the memory types of the D1
which are implemented differently to svpbmt. But when patching anyway
it's pretty clean to add the D1 variant via ALTERNATIVE_2 to the same
location.
The only slightly bigger difference is that the "normal" type is not 0
as with svpbmt, so kernel patches for this PMA type need to be applied
even before the MMU is brought up, so the series introduces a separate
stage for that.
In theory this series is 3 parts:
- sbi cache-flush / null-ptr
- alternatives improvements
- svpbmt+d1
So expecially patches from the first 2 areas could be applied when
deemed ready, I just thought to keep it together to show-case where
the end-goal is and not requiring jumping between different series.
I picked the recipient list from the previous versions, hopefully
I didn't forget anybody.
changes in v8:
- rebase onto 5.17-final + isa extension series
We're halfway through the merge-window, so this series
should be merge after that
- adapt to fix limiting alternatives to non-xip-kernels
- add .norelax option for alternatives
- fix unused cpu_apply_errata in thead errata
- don't use static globals to store cpu-manufacturer data
as it makes machines hang if done too early
changes in v7:
- fix typo in patch1 (Atish)
- moved to Atish's isa-extension framework
- and therefore move regular boot-alternatives directly behind fill_hwcaps
- change T-Head errata Kconfig text (Atish)
changes in v6:
- rebase onto 5.17-rc1
- handle sbi null-ptr differently
- improve commit messages
- use riscv,mmu as property name
changes in v5:
- move to use alternatives for runtime-patching
- add D1 variant
[0] https://lore.kernel.org/r/[email protected]
Heiko Stuebner (13):
riscv: prevent null-pointer dereference with sbi_remote_fence_i
riscv: integrate alternatives better into the main architecture
riscv: allow different stages with alternatives
riscv: implement module alternatives
riscv: implement ALTERNATIVE_2 macro
riscv: extend concatenated alternatives-lines to the same length
riscv: prevent compressed instructions in alternatives
riscv: move boot alternatives to after fill_hwcap
riscv: Fix accessing pfn bits in PTEs for non-32bit variants
riscv: add cpufeature handling via alternatives
riscv: remove FIXMAP_PAGE_IO and fall back to its default value
riscv: don't use global static vars to store alternative data
riscv: add memory-type errata for T-Head
Wei Fu (1):
riscv: add RISC-V Svpbmt extension support
arch/riscv/Kconfig.erratas | 31 +++--
arch/riscv/Kconfig.socs | 1 -
arch/riscv/Makefile | 2 +-
arch/riscv/errata/Makefile | 2 +-
arch/riscv/errata/alternative.c | 75 ------------
arch/riscv/errata/sifive/errata.c | 17 ++-
arch/riscv/errata/thead/Makefile | 1 +
arch/riscv/errata/thead/errata.c | 82 +++++++++++++
arch/riscv/include/asm/alternative-macros.h | 121 ++++++++++++++------
arch/riscv/include/asm/alternative.h | 16 ++-
arch/riscv/include/asm/errata_list.h | 52 +++++++++
arch/riscv/include/asm/fixmap.h | 2 -
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/pgtable-32.h | 17 +++
arch/riscv/include/asm/pgtable-64.h | 79 ++++++++++++-
arch/riscv/include/asm/pgtable-bits.h | 10 --
arch/riscv/include/asm/pgtable.h | 53 +++++++--
arch/riscv/include/asm/vendorid_list.h | 1 +
arch/riscv/kernel/Makefile | 1 +
arch/riscv/kernel/alternative.c | 104 +++++++++++++++++
arch/riscv/kernel/cpu.c | 1 +
arch/riscv/kernel/cpufeature.c | 80 ++++++++++++-
arch/riscv/kernel/module.c | 29 +++++
arch/riscv/kernel/sbi.c | 10 +-
arch/riscv/kernel/setup.c | 2 +
arch/riscv/kernel/smpboot.c | 4 -
arch/riscv/kernel/traps.c | 2 +-
arch/riscv/mm/init.c | 1 +
28 files changed, 629 insertions(+), 168 deletions(-)
delete mode 100644 arch/riscv/errata/alternative.c
create mode 100644 arch/riscv/errata/thead/Makefile
create mode 100644 arch/riscv/errata/thead/errata.c
create mode 100644 arch/riscv/kernel/alternative.c
--
2.35.1
Move the application of boot alternatives to after the hw-capabilities
are populated. This allows to check for available extensions when
determining which alternatives to apply and also makes it actually
work if CONFIG_SMP is disabled for whatever reason.
Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/kernel/setup.c | 2 ++
arch/riscv/kernel/smpboot.c | 2 --
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 339ceb595b38..b4879c942b42 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -21,6 +21,7 @@
#include <linux/efi.h>
#include <linux/crash_dump.h>
+#include <asm/alternative.h>
#include <asm/cpu_ops.h>
#include <asm/early_ioremap.h>
#include <asm/pgtable.h>
@@ -295,6 +296,7 @@ void __init setup_arch(char **cmdline_p)
#endif
riscv_fill_hwcap();
+ apply_boot_alternatives();
}
static int __init topology_init(void)
diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
index a6d13dca1403..f1e4948a4b52 100644
--- a/arch/riscv/kernel/smpboot.c
+++ b/arch/riscv/kernel/smpboot.c
@@ -32,7 +32,6 @@
#include <asm/sections.h>
#include <asm/sbi.h>
#include <asm/smp.h>
-#include <asm/alternative.h>
#include "head.h"
@@ -41,7 +40,6 @@ static DECLARE_COMPLETION(cpu_running);
void __init smp_prepare_boot_cpu(void)
{
init_cpu_topology();
- apply_boot_alternatives();
}
void __init smp_prepare_cpus(unsigned int max_cpus)
--
2.35.1
From: Wei Fu <[email protected]>
Svpbmt (the S should be capitalized) is the
"Supervisor-mode: page-based memory types" extension
that specifies attributes for cacheability, idempotency
and ordering.
The relevant settings are done in special bits in PTEs:
Here is the svpbmt PTE format:
| 63 | 62-61 | 60-8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
N MT RSW D A G U X W R V
^
Of the Reserved bits [63:54] in a leaf PTE, the high bit is already
allocated (as the N bit), so bits [62:61] are used as the MT (aka
MemType) field. This field specifies one of three memory types that
are close equivalents (or equivalent in effect) to the three main x86
and ARMv8 memory types - as shown in the following table.
RISC-V
Encoding &
MemType RISC-V Description
---------- ------------------------------------------------
00 - PMA Normal Cacheable, No change to implied PMA memory type
01 - NC Non-cacheable, idempotent, weakly-ordered Main Memory
10 - IO Non-cacheable, non-idempotent, strongly-ordered I/O memory
11 - Rsvd Reserved for future standard use
As the extension will not be present on all implementations use
alternatives to check its existence at runtime and if needed
adapt the code to use memory-types.
Signed-off-by: Wei Fu <[email protected]>
Co-developed-by: Liu Shaohua <[email protected]>
Signed-off-by: Liu Shaohua <[email protected]>
Co-developed-by: Guo Ren <[email protected]>
Signed-off-by: Guo Ren <[email protected]>
[moved to use the alternatives mechanism]
Co-developed-by: Heiko Stuebner <[email protected]>
Signed-off-by: Heiko Stuebner <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: Drew Fustini <[email protected]>
Cc: Wei Fu <[email protected]>
Cc: Wei Wu <[email protected]>
Cc: Chen-Yu Tsai <[email protected]>
Cc: Maxime Ripard <[email protected]>
Cc: Daniel Lustig <[email protected]>
Cc: Greg Favor <[email protected]>
Cc: Andrea Mondelli <[email protected]>
Cc: Jonathan Behrens <[email protected]>
Cc: Xinhaoqu (Freddie) <[email protected]>
Cc: Bill Huffman <[email protected]>
Cc: Nick Kossifidis <[email protected]>
Cc: Allen Baum <[email protected]>
Cc: Josh Scheid <[email protected]>
Cc: Richard Trauben <[email protected]>
---
arch/riscv/include/asm/errata_list.h | 13 +++++++-
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/pgtable-32.h | 9 +++++
arch/riscv/include/asm/pgtable-64.h | 47 +++++++++++++++++++++++++++
arch/riscv/include/asm/pgtable-bits.h | 4 ---
arch/riscv/include/asm/pgtable.h | 29 ++++++++++++++---
arch/riscv/kernel/cpu.c | 1 +
arch/riscv/kernel/cpufeature.c | 18 ++++++++++
8 files changed, 112 insertions(+), 10 deletions(-)
diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
index 6b95bd9aee82..a4a9b0842922 100644
--- a/arch/riscv/include/asm/errata_list.h
+++ b/arch/riscv/include/asm/errata_list.h
@@ -14,7 +14,8 @@
#define ERRATA_SIFIVE_NUMBER 2
#endif
-#define CPUFEATURE_NUMBER 0
+#define CPUFEATURE_SVPBMT 0
+#define CPUFEATURE_NUMBER 1
#ifdef __ASSEMBLY__
@@ -36,6 +37,16 @@ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \
ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \
: : "r" (addr) : "memory")
+/*
+ * _val is marked as "will be overwritten", so need to set it to 0
+ * in the default case.
+ */
+#define ALT_SVPBMT_SHIFT 61
+#define ALT_SVPBMT(_val, prot) \
+asm(ALTERNATIVE("li %0, 0\t\nnop", "li %0, %1\t\nslli %0,%0,%2", 0, \
+ CPUFEATURE_SVPBMT, CONFIG_64BIT) \
+ : "=r"(_val) : "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT), "I"(ALT_SVPBMT_SHIFT))
+
#endif /* __ASSEMBLY__ */
#endif
diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
index 691fc9c8099b..656cd626eb1a 100644
--- a/arch/riscv/include/asm/hwcap.h
+++ b/arch/riscv/include/asm/hwcap.h
@@ -51,6 +51,7 @@ extern unsigned long elf_hwcap;
* available logical extension id.
*/
enum riscv_isa_ext_id {
+ RISCV_ISA_EXT_SVPBMT = RISCV_ISA_EXT_BASE,
RISCV_ISA_EXT_ID_MAX = RISCV_ISA_EXT_MAX,
};
diff --git a/arch/riscv/include/asm/pgtable-32.h b/arch/riscv/include/asm/pgtable-32.h
index e266a4fe7f43..59ba1fbaf784 100644
--- a/arch/riscv/include/asm/pgtable-32.h
+++ b/arch/riscv/include/asm/pgtable-32.h
@@ -24,4 +24,13 @@
*/
#define _PAGE_PFN_MASK GENMASK(31, 10)
+#define _PAGE_NOCACHE 0
+#define _PAGE_IO 0
+#define _PAGE_MTMASK 0
+
+/* Set of bits to preserve across pte_modify() */
+#define _PAGE_CHG_MASK (~(unsigned long)(_PAGE_PRESENT | _PAGE_READ | \
+ _PAGE_WRITE | _PAGE_EXEC | \
+ _PAGE_USER | _PAGE_GLOBAL))
+
#endif /* _ASM_RISCV_PGTABLE_32_H */
diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index 9412c6157c88..07ba3416cb19 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -8,6 +8,7 @@
#include <linux/bits.h>
#include <linux/const.h>
+#include <asm/errata_list.h>
extern bool pgtable_l4_enabled;
@@ -56,6 +57,52 @@ typedef struct {
*/
#define _PAGE_PFN_MASK GENMASK(53, 10)
+/*
+ * [62:61] Svpbmt Memory Type definitions:
+ *
+ * 00 - PMA Normal Cacheable, No change to implied PMA memory type
+ * 01 - NC Non-cacheable, idempotent, weakly-ordered Main Memory
+ * 10 - IO Non-cacheable, non-idempotent, strongly-ordered I/O memory
+ * 11 - Rsvd Reserved for future standard use
+ */
+#define _PAGE_NOCACHE_SVPBMT (1UL << 61)
+#define _PAGE_IO_SVPBMT (1UL << 62)
+#define _PAGE_MTMASK_SVPBMT (_PAGE_NOCACHE_SVPBMT | _PAGE_IO_SVPBMT)
+
+static inline u64 riscv_page_mtmask(void)
+{
+ u64 val;
+
+ ALT_SVPBMT(val, _PAGE_MTMASK);
+ return val;
+}
+
+static inline u64 riscv_page_nocache(void)
+{
+ u64 val;
+
+ ALT_SVPBMT(val, _PAGE_NOCACHE);
+ return val;
+}
+
+static inline u64 riscv_page_io(void)
+{
+ u64 val;
+
+ ALT_SVPBMT(val, _PAGE_IO);
+ return val;
+}
+
+#define _PAGE_NOCACHE riscv_page_nocache()
+#define _PAGE_IO riscv_page_io()
+#define _PAGE_MTMASK riscv_page_mtmask()
+
+/* Set of bits to preserve across pte_modify() */
+#define _PAGE_CHG_MASK (~(unsigned long)(_PAGE_PRESENT | _PAGE_READ | \
+ _PAGE_WRITE | _PAGE_EXEC | \
+ _PAGE_USER | _PAGE_GLOBAL | \
+ _PAGE_MTMASK))
+
static inline int pud_present(pud_t pud)
{
return (pud_val(pud) & _PAGE_PRESENT);
diff --git a/arch/riscv/include/asm/pgtable-bits.h b/arch/riscv/include/asm/pgtable-bits.h
index e571fa954afc..b9e13a8fe2b7 100644
--- a/arch/riscv/include/asm/pgtable-bits.h
+++ b/arch/riscv/include/asm/pgtable-bits.h
@@ -29,10 +29,6 @@
#define _PAGE_PFN_SHIFT 10
-/* Set of bits to preserve across pte_modify() */
-#define _PAGE_CHG_MASK (~(unsigned long)(_PAGE_PRESENT | _PAGE_READ | \
- _PAGE_WRITE | _PAGE_EXEC | \
- _PAGE_USER | _PAGE_GLOBAL))
/*
* when all of R/W/X are zero, the PTE is a pointer to the next level
* of the page table; otherwise, it is a leaf PTE.
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 6d31489818cd..009a17f94b86 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -175,11 +175,8 @@ extern struct pt_alloc_ops pt_ops __initdata;
#define PAGE_TABLE __pgprot(_PAGE_TABLE)
-/*
- * The RISC-V ISA doesn't yet specify how to query or modify PMAs, so we can't
- * change the properties of memory regions.
- */
-#define _PAGE_IOREMAP _PAGE_KERNEL
+#define _PAGE_IOREMAP ((_PAGE_KERNEL & ~_PAGE_MTMASK) | _PAGE_IO)
+#define PAGE_KERNEL_IO __pgprot(_PAGE_IOREMAP)
extern pgd_t swapper_pg_dir[];
@@ -519,6 +516,28 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma,
return ptep_test_and_clear_young(vma, address, ptep);
}
+#define pgprot_noncached pgprot_noncached
+static inline pgprot_t pgprot_noncached(pgprot_t _prot)
+{
+ unsigned long prot = pgprot_val(_prot);
+
+ prot &= ~_PAGE_MTMASK;
+ prot |= _PAGE_IO;
+
+ return __pgprot(prot);
+}
+
+#define pgprot_writecombine pgprot_writecombine
+static inline pgprot_t pgprot_writecombine(pgprot_t _prot)
+{
+ unsigned long prot = pgprot_val(_prot);
+
+ prot &= ~_PAGE_MTMASK;
+ prot |= _PAGE_NOCACHE;
+
+ return __pgprot(prot);
+}
+
/*
* THP functions
*/
diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
index fc115e307ef5..01a4799e7e1c 100644
--- a/arch/riscv/kernel/cpu.c
+++ b/arch/riscv/kernel/cpu.c
@@ -87,6 +87,7 @@ int riscv_of_parent_hartid(struct device_node *node)
* extensions by an underscore.
*/
static struct riscv_isa_ext_data isa_ext_arr[] = {
+ __RISCV_ISA_EXT_DATA("svpbmt", RISCV_ISA_EXT_SVPBMT),
__RISCV_ISA_EXT_DATA("", RISCV_ISA_EXT_MAX),
};
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index c0ffc26c3ef3..fab9c5367949 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -8,6 +8,7 @@
#include <linux/bitmap.h>
#include <linux/ctype.h>
+#include <linux/libfdt.h>
#include <linux/module.h>
#include <linux/of.h>
#include <asm/alternative.h>
@@ -195,6 +196,8 @@ void __init riscv_fill_hwcap(void)
if (!ext_long) {
this_hwcap |= isa2hwcap[(unsigned char)(*ext)];
set_bit(*ext - 'a', this_isa);
+ } else {
+ SET_ISA_EXT_MAP("svpbmt", RISCV_ISA_EXT_SVPBMT);
}
#undef SET_ISA_EXT_MAP
}
@@ -246,7 +249,22 @@ struct cpufeature_info {
bool (*check_func)(unsigned int stage);
};
+static bool __init_or_module cpufeature_svpbmt_check_func(unsigned int stage)
+{
+ bool ret = false;
+
+#if defined(CONFIG_MMU) && defined(CONFIG_64BIT)
+ return riscv_isa_extension_available(NULL, SVPBMT);
+#endif
+
+ return ret;
+}
+
static const struct cpufeature_info __initdata_or_module cpufeature_list[CPUFEATURE_NUMBER] = {
+ {
+ .name = "svpbmt",
+ .check_func = cpufeature_svpbmt_check_func
+ },
};
static u32 __init_or_module cpufeature_probe(unsigned int stage)
--
2.35.1
Some current cpus based on T-Head cores implement memory-types
way different than described in the svpbmt spec even going
so far as using PTE bits marked as reserved.
Add the T-Head vendor-id and necessary errata code to
replace the affected instructions.
Signed-off-by: Heiko Stuebner <[email protected]>
Tested-by: Samuel Holland <[email protected]>
---
arch/riscv/Kconfig.erratas | 19 ++++++
arch/riscv/errata/Makefile | 1 +
arch/riscv/errata/sifive/errata.c | 7 ++-
arch/riscv/errata/thead/Makefile | 1 +
arch/riscv/errata/thead/errata.c | 82 ++++++++++++++++++++++++++
arch/riscv/include/asm/alternative.h | 5 ++
arch/riscv/include/asm/errata_list.h | 47 +++++++++++++--
arch/riscv/include/asm/pgtable-64.h | 18 +++++-
arch/riscv/include/asm/pgtable.h | 18 +++++-
arch/riscv/include/asm/vendorid_list.h | 1 +
arch/riscv/kernel/alternative.c | 12 ++++
arch/riscv/kernel/cpufeature.c | 7 ++-
arch/riscv/mm/init.c | 1 +
13 files changed, 209 insertions(+), 10 deletions(-)
create mode 100644 arch/riscv/errata/thead/Makefile
create mode 100644 arch/riscv/errata/thead/errata.c
diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas
index 15cdeeb4151f..8221c5df8da8 100644
--- a/arch/riscv/Kconfig.erratas
+++ b/arch/riscv/Kconfig.erratas
@@ -32,4 +32,23 @@ config ERRATA_SIFIVE_CIP_1200
If you don't know what to do here, say "Y".
+config ERRATA_THEAD
+ bool "T-HEAD errata"
+ help
+ All T-HEAD errata Kconfig depend on this Kconfig. Disabling
+ this Kconfig will disable all T-HEAD errata. Please say "Y"
+ here if your platform uses T-HEAD CPU cores.
+
+ Otherwise, please say "N" here to avoid unnecessary overhead.
+
+config ERRATA_THEAD_PBMT
+ bool "Apply T-Head memory type errata"
+ depends on ERRATA_THEAD && 64BIT
+ default y
+ help
+ This will apply the memory type errata to handle the non-standard
+ memory type bits in page-table-entries on T-Head SoCs.
+
+ If you don't know what to do here, say "Y".
+
endmenu
diff --git a/arch/riscv/errata/Makefile b/arch/riscv/errata/Makefile
index 0ca1c5281a2d..a1055965fbee 100644
--- a/arch/riscv/errata/Makefile
+++ b/arch/riscv/errata/Makefile
@@ -1 +1,2 @@
obj-$(CONFIG_ERRATA_SIFIVE) += sifive/
+obj-$(CONFIG_ERRATA_THEAD) += thead/
diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c
index 42457efcaaa3..0b14d5475573 100644
--- a/arch/riscv/errata/sifive/errata.c
+++ b/arch/riscv/errata/sifive/errata.c
@@ -85,10 +85,15 @@ void __init_or_module sifive_errata_patch_func(struct alt_entry *begin, struct a
unsigned int stage)
{
struct alt_entry *alt;
- u32 cpu_req_errata = sifive_errata_probe(archid, impid);
+ u32 cpu_req_errata;
u32 cpu_apply_errata = 0;
u32 tmp;
+ if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
+ return;
+
+ cpu_req_errata = sifive_errata_probe(archid, impid);
+
for (alt = begin; alt < end; alt++) {
if (alt->vendor_id != SIFIVE_VENDOR_ID)
continue;
diff --git a/arch/riscv/errata/thead/Makefile b/arch/riscv/errata/thead/Makefile
new file mode 100644
index 000000000000..2d644e19caef
--- /dev/null
+++ b/arch/riscv/errata/thead/Makefile
@@ -0,0 +1 @@
+obj-y += errata.o
diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
new file mode 100644
index 000000000000..e5d75270b99c
--- /dev/null
+++ b/arch/riscv/errata/thead/errata.c
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021 Heiko Stuebner <[email protected]>
+ */
+
+#include <linux/bug.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/string.h>
+#include <linux/uaccess.h>
+#include <asm/alternative.h>
+#include <asm/cacheflush.h>
+#include <asm/errata_list.h>
+#include <asm/patch.h>
+#include <asm/vendorid_list.h>
+
+struct errata_info {
+ char name[ERRATA_STRING_LENGTH_MAX];
+ bool (*check_func)(unsigned long arch_id, unsigned long impid);
+ unsigned int stage;
+};
+
+static bool errata_mt_check_func(unsigned long arch_id, unsigned long impid)
+{
+ if (arch_id != 0 || impid != 0)
+ return false;
+ return true;
+}
+
+static const struct errata_info errata_list[ERRATA_THEAD_NUMBER] = {
+ {
+ .name = "memory-types",
+ .stage = RISCV_ALTERNATIVES_EARLY_BOOT,
+ .check_func = errata_mt_check_func
+ },
+};
+
+static u32 thead_errata_probe(unsigned int stage, unsigned long archid, unsigned long impid)
+{
+ const struct errata_info *info;
+ u32 cpu_req_errata = 0;
+ int idx;
+
+ for (idx = 0; idx < ERRATA_THEAD_NUMBER; idx++) {
+ info = &errata_list[idx];
+
+ if ((stage == RISCV_ALTERNATIVES_MODULE ||
+ info->stage == stage) && info->check_func(archid, impid))
+ cpu_req_errata |= (1U << idx);
+ }
+
+ return cpu_req_errata;
+}
+
+void __init_or_module thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
+ unsigned long archid, unsigned long impid,
+ unsigned int stage)
+{
+ struct alt_entry *alt;
+ u32 cpu_req_errata = thead_errata_probe(stage, archid, impid);
+ u32 tmp;
+
+ for (alt = begin; alt < end; alt++) {
+ if (alt->vendor_id != THEAD_VENDOR_ID)
+ continue;
+ if (alt->errata_id >= ERRATA_THEAD_NUMBER)
+ continue;
+
+ tmp = (1U << alt->errata_id);
+ if (cpu_req_errata & tmp) {
+ /* On vm-alternatives, the mmu isn't running yet */
+ if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
+ memcpy((void *)__pa_symbol(alt->old_ptr),
+ (void *)__pa_symbol(alt->alt_ptr), alt->alt_len);
+ else
+ patch_text_nosync(alt->old_ptr, alt->alt_ptr, alt->alt_len);
+ }
+ }
+
+ if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
+ local_flush_icache_all();
+}
diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
index cf3b22173834..d1154c91ab03 100644
--- a/arch/riscv/include/asm/alternative.h
+++ b/arch/riscv/include/asm/alternative.h
@@ -19,8 +19,10 @@
#define RISCV_ALTERNATIVES_BOOT 0 /* alternatives applied during regular boot */
#define RISCV_ALTERNATIVES_MODULE 1 /* alternatives applied during module-init */
+#define RISCV_ALTERNATIVES_EARLY_BOOT 2 /* alternatives applied before mmu start */
void __init apply_boot_alternatives(void);
+void __init apply_early_boot_alternatives(void);
void apply_module_alternatives(void *start, size_t length);
struct alt_entry {
@@ -39,6 +41,9 @@ struct errata_checkfunc_id {
void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
unsigned long archid, unsigned long impid,
unsigned int stage);
+void thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
+ unsigned long archid, unsigned long impid,
+ unsigned int stage);
void riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end,
unsigned int stage);
diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
index a4a9b0842922..4fac46b82c16 100644
--- a/arch/riscv/include/asm/errata_list.h
+++ b/arch/riscv/include/asm/errata_list.h
@@ -14,6 +14,11 @@
#define ERRATA_SIFIVE_NUMBER 2
#endif
+#ifdef CONFIG_ERRATA_THEAD
+#define ERRATA_THEAD_PBMT 0
+#define ERRATA_THEAD_NUMBER 1
+#endif
+
#define CPUFEATURE_SVPBMT 0
#define CPUFEATURE_NUMBER 1
@@ -42,10 +47,44 @@ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \
* in the default case.
*/
#define ALT_SVPBMT_SHIFT 61
-#define ALT_SVPBMT(_val, prot) \
-asm(ALTERNATIVE("li %0, 0\t\nnop", "li %0, %1\t\nslli %0,%0,%2", 0, \
- CPUFEATURE_SVPBMT, CONFIG_64BIT) \
- : "=r"(_val) : "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT), "I"(ALT_SVPBMT_SHIFT))
+#define ALT_THEAD_PBMT_SHIFT 59
+#define ALT_SVPBMT(_val, prot) \
+asm(ALTERNATIVE_2("li %0, 0\t\nnop", \
+ "li %0, %1\t\nslli %0,%0,%3", 0, CPUFEATURE_SVPBMT, CONFIG_64BIT, \
+ "li %0, %2\t\nslli %0,%0,%4", THEAD_VENDOR_ID, ERRATA_THEAD_PBMT, \
+ CONFIG_ERRATA_THEAD_PBMT) \
+ : "=r"(_val) : "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT), \
+ "I"(prot##_THEAD >> ALT_THEAD_PBMT_SHIFT), \
+ "I"(ALT_SVPBMT_SHIFT), "I"(ALT_THEAD_PBMT_SHIFT))
+
+#ifdef CONFIG_ERRATA_THEAD_PBMT
+/*
+ * IO/NOCACHE memory types are handled together with svpbmt,
+ * so on T-Head chips, check if no other memory type is set,
+ * and set the non-0 PMA type if applicable.
+ */
+#define ALT_THEAD_PMA(_val) \
+asm volatile(ALTERNATIVE( \
+ "nop\n\t" \
+ "nop\n\t" \
+ "nop\n\t" \
+ "nop\n\t" \
+ "nop\n\t" \
+ "nop\n\t" \
+ "nop", \
+ "li t3, %2\n\t" \
+ "slli t3, t3, %4\n\t" \
+ "and t3, %0, t3\n\t" \
+ "bne t3, zero, 2f\n\t" \
+ "li t3, %3\n\t" \
+ "slli t3, t3, %4\n\t" \
+ "or %0, %0, t3\n\t" \
+ "2:", THEAD_VENDOR_ID, ERRATA_THEAD_PBMT, CONFIG_ERRATA_THEAD_PBMT) \
+ : "+r"(_val) : "0"(_val), "I"(_PAGE_MTMASK_THEAD >> ALT_THEAD_PBMT_SHIFT), \
+ "I"(_PAGE_PMA_THEAD >> ALT_THEAD_PBMT_SHIFT), "I"(ALT_THEAD_PBMT_SHIFT))
+#else
+#define ALT_THEAD_PMA(_val)
+#endif
#endif /* __ASSEMBLY__ */
diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index 07ba3416cb19..6d59e4695200 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -69,6 +69,18 @@ typedef struct {
#define _PAGE_IO_SVPBMT (1UL << 62)
#define _PAGE_MTMASK_SVPBMT (_PAGE_NOCACHE_SVPBMT | _PAGE_IO_SVPBMT)
+/*
+ * [63:59] T-Head Memory Type definitions:
+ *
+ * 00000 - NC Weakly-ordered, Non-cacheable, Non-bufferable, Non-shareable, Non-trustable
+ * 01110 - PMA Weakly-ordered, Cacheable, Bufferable, Shareable, Non-trustable
+ * 10000 - IO Strongly-ordered, Non-cacheable, Non-bufferable, Non-shareable, Non-trustable
+ */
+#define _PAGE_PMA_THEAD ((1UL << 62) | (1UL << 61) | (1UL << 60))
+#define _PAGE_NOCACHE_THEAD 0UL
+#define _PAGE_IO_THEAD (1UL << 63)
+#define _PAGE_MTMASK_THEAD (_PAGE_PMA_THEAD | _PAGE_IO_THEAD | (1UL << 59))
+
static inline u64 riscv_page_mtmask(void)
{
u64 val;
@@ -167,7 +179,11 @@ static inline bool mm_pud_folded(struct mm_struct *mm)
static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
{
- return __pmd((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
+ unsigned long prot_val = pgprot_val(prot);
+
+ ALT_THEAD_PMA(prot_val);
+
+ return __pmd((pfn << _PAGE_PFN_SHIFT) | prot_val);
}
static inline unsigned long _pmd_pfn(pmd_t pmd)
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 009a17f94b86..e2658a25f06d 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -246,7 +246,11 @@ static inline void pmd_clear(pmd_t *pmdp)
static inline pgd_t pfn_pgd(unsigned long pfn, pgprot_t prot)
{
- return __pgd((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
+ unsigned long prot_val = pgprot_val(prot);
+
+ ALT_THEAD_PMA(prot_val);
+
+ return __pgd((pfn << _PAGE_PFN_SHIFT) | prot_val);
}
static inline unsigned long _pgd_pfn(pgd_t pgd)
@@ -285,7 +289,11 @@ static inline unsigned long pte_pfn(pte_t pte)
/* Constructs a page table entry */
static inline pte_t pfn_pte(unsigned long pfn, pgprot_t prot)
{
- return __pte((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
+ unsigned long prot_val = pgprot_val(prot);
+
+ ALT_THEAD_PMA(prot_val);
+
+ return __pte((pfn << _PAGE_PFN_SHIFT) | prot_val);
}
#define mk_pte(page, prot) pfn_pte(page_to_pfn(page), prot)
@@ -394,7 +402,11 @@ static inline int pmd_protnone(pmd_t pmd)
/* Modify page protection bits */
static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
{
- return __pte((pte_val(pte) & _PAGE_CHG_MASK) | pgprot_val(newprot));
+ unsigned long newprot_val = pgprot_val(newprot);
+
+ ALT_THEAD_PMA(newprot_val);
+
+ return __pte((pte_val(pte) & _PAGE_CHG_MASK) | newprot_val);
}
#define pgd_ERROR(e) \
diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
index 9d934215b3c8..cb89af3f0704 100644
--- a/arch/riscv/include/asm/vendorid_list.h
+++ b/arch/riscv/include/asm/vendorid_list.h
@@ -6,5 +6,6 @@
#define ASM_VENDOR_LIST_H
#define SIFIVE_VENDOR_ID 0x489
+#define THEAD_VENDOR_ID 0x5b7
#endif
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index 27f722ae452b..e7980bf17e8b 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -42,6 +42,11 @@ static void __init_or_module riscv_fill_cpu_mfr_info(struct cpu_manufacturer_inf
case SIFIVE_VENDOR_ID:
cpu_mfr_info->vendor_patch_func = sifive_errata_patch_func;
break;
+#endif
+#ifdef CONFIG_ERRATA_THEAD
+ case THEAD_VENDOR_ID:
+ cpu_mfr_info->vendor_patch_func = thead_errata_patch_func;
+ break;
#endif
default:
cpu_mfr_info->vendor_patch_func = NULL;
@@ -82,6 +87,13 @@ void __init apply_boot_alternatives(void)
RISCV_ALTERNATIVES_BOOT);
}
+void __init apply_early_boot_alternatives(void)
+{
+ _apply_alternatives((struct alt_entry *)__alt_start,
+ (struct alt_entry *)__alt_end,
+ RISCV_ALTERNATIVES_EARLY_BOOT);
+}
+
#ifdef CONFIG_MODULES
void apply_module_alternatives(void *start, size_t length)
{
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index fab9c5367949..564c1917c8aa 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -254,7 +254,12 @@ static bool __init_or_module cpufeature_svpbmt_check_func(unsigned int stage)
bool ret = false;
#if defined(CONFIG_MMU) && defined(CONFIG_64BIT)
- return riscv_isa_extension_available(NULL, SVPBMT);
+ switch (stage) {
+ case RISCV_ALTERNATIVES_EARLY_BOOT:
+ return false;
+ default:
+ return riscv_isa_extension_available(NULL, SVPBMT);
+ }
#endif
return ret;
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index 0d588032d6e6..245d54a3616c 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -821,6 +821,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa)
BUG_ON((kernel_map.virt_addr + kernel_map.size) > ADDRESS_SPACE_END - SZ_4K);
#endif
+ apply_early_boot_alternatives();
pt_ops_set_early();
/* Setup early PGD for fixmap */
--
2.35.1
Right now the alternatives need to be explicitly enabled and
erratas are limited to SiFive ones.
Over time with more SoCs and additional RiscV extensions, many more
erratas or other patch-worthy features will emerge, so it doesn't
really make sense to have the core alternatives able to get
deactivated.
So make it part of the core RiscV kernel and drop the main
RISCV_ERRATA_ALTERNATIVES config symbol.
This mimics how other architectures like for example arm64 handle
their alternatives implementation.
Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/Kconfig.erratas | 12 +-----------
arch/riscv/Kconfig.socs | 1 -
arch/riscv/Makefile | 2 +-
arch/riscv/errata/Makefile | 1 -
arch/riscv/include/asm/alternative-macros.h | 15 ++++++++-------
arch/riscv/kernel/Makefile | 1 +
arch/riscv/{errata => kernel}/alternative.c | 0
arch/riscv/kernel/smpboot.c | 2 --
arch/riscv/kernel/traps.c | 2 +-
9 files changed, 12 insertions(+), 24 deletions(-)
rename arch/riscv/{errata => kernel}/alternative.c (100%)
diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas
index 0aacd7052585..15cdeeb4151f 100644
--- a/arch/riscv/Kconfig.erratas
+++ b/arch/riscv/Kconfig.erratas
@@ -1,18 +1,8 @@
menu "CPU errata selection"
-config RISCV_ERRATA_ALTERNATIVE
- bool "RISC-V alternative scheme"
- depends on !XIP_KERNEL
- default y
- help
- This Kconfig allows the kernel to automatically patch the
- errata required by the execution platform at run time. The
- code patching is performed once in the boot stages. It means
- that the overhead from this mechanism is just taken once.
-
config ERRATA_SIFIVE
bool "SiFive errata"
- depends on RISCV_ERRATA_ALTERNATIVE
+ depends on !XIP_KERNEL
help
All SiFive errata Kconfig depend on this Kconfig. Disabling
this Kconfig will disable all SiFive errata. Please say "Y"
diff --git a/arch/riscv/Kconfig.socs b/arch/riscv/Kconfig.socs
index c112ab2a9052..236d7d46a6f1 100644
--- a/arch/riscv/Kconfig.socs
+++ b/arch/riscv/Kconfig.socs
@@ -14,7 +14,6 @@ config SOC_SIFIVE
select CLK_SIFIVE
select CLK_SIFIVE_PRCI
select SIFIVE_PLIC
- select RISCV_ERRATA_ALTERNATIVE if !XIP_KERNEL
select ERRATA_SIFIVE if !XIP_KERNEL
help
This enables support for SiFive SoC platform hardware.
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 7d81102cffd4..a7ed47ce9311 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -103,7 +103,7 @@ endif
head-y := arch/riscv/kernel/head.o
-core-$(CONFIG_RISCV_ERRATA_ALTERNATIVE) += arch/riscv/errata/
+core-y += arch/riscv/errata/
core-$(CONFIG_KVM) += arch/riscv/kvm/
libs-y += arch/riscv/lib/
diff --git a/arch/riscv/errata/Makefile b/arch/riscv/errata/Makefile
index b8f8740a3e44..0ca1c5281a2d 100644
--- a/arch/riscv/errata/Makefile
+++ b/arch/riscv/errata/Makefile
@@ -1,2 +1 @@
-obj-y += alternative.o
obj-$(CONFIG_ERRATA_SIFIVE) += sifive/
diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
index 67406c376389..e33d4bc54019 100644
--- a/arch/riscv/include/asm/alternative-macros.h
+++ b/arch/riscv/include/asm/alternative-macros.h
@@ -2,7 +2,7 @@
#ifndef __ASM_ALTERNATIVE_MACROS_H
#define __ASM_ALTERNATIVE_MACROS_H
-#ifdef CONFIG_RISCV_ERRATA_ALTERNATIVE
+#ifndef CONFIG_XIP_KERNEL
#ifdef __ASSEMBLY__
@@ -76,26 +76,27 @@
#endif /* __ASSEMBLY__ */
-#else /* !CONFIG_RISCV_ERRATA_ALTERNATIVE*/
+#else /* CONFIG_XIP_KERNEL */
#ifdef __ASSEMBLY__
.macro __ALTERNATIVE_CFG old_c
- \old_c
+ \old_c
.endm
#define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
- __ALTERNATIVE_CFG old_c
+ __ALTERNATIVE_CFG old_c
#else /* !__ASSEMBLY__ */
#define __ALTERNATIVE_CFG(old_c) \
- old_c "\n"
+ old_c "\n"
#define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
- __ALTERNATIVE_CFG(old_c)
+ __ALTERNATIVE_CFG(old_c)
#endif /* __ASSEMBLY__ */
-#endif /* CONFIG_RISCV_ERRATA_ALTERNATIVE */
+#endif /* CONFIG_XIP_KERNEL */
+
/*
* Usage:
* ALTERNATIVE(old_content, new_content, vendor_id, errata_id, CONFIG_k)
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index ffc87e76b1dd..b02142517f7f 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -18,6 +18,7 @@ extra-y += head.o
extra-y += vmlinux.lds
obj-y += soc.o
+obj-y += alternative.o
obj-y += cpu.o
obj-y += cpufeature.o
obj-y += entry.o
diff --git a/arch/riscv/errata/alternative.c b/arch/riscv/kernel/alternative.c
similarity index 100%
rename from arch/riscv/errata/alternative.c
rename to arch/riscv/kernel/alternative.c
diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
index 622f226454d5..a6d13dca1403 100644
--- a/arch/riscv/kernel/smpboot.c
+++ b/arch/riscv/kernel/smpboot.c
@@ -41,9 +41,7 @@ static DECLARE_COMPLETION(cpu_running);
void __init smp_prepare_boot_cpu(void)
{
init_cpu_topology();
-#ifdef CONFIG_RISCV_ERRATA_ALTERNATIVE
apply_boot_alternatives();
-#endif
}
void __init smp_prepare_cpus(unsigned int max_cpus)
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index fe92e119e6a3..9984c8622c3b 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -86,7 +86,7 @@ static void do_trap_error(struct pt_regs *regs, int signo, int code,
}
}
-#if defined (CONFIG_XIP_KERNEL) && defined (CONFIG_RISCV_ERRATA_ALTERNATIVE)
+#if defined (CONFIG_XIP_KERNEL)
#define __trap_section __section(".xip.traps")
#else
#define __trap_section
--
2.35.1
This allows alternatives to also be applied when loading modules
and follows the implementation of other architectures (e.g. arm64).
Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/errata/sifive/errata.c | 7 ++++---
arch/riscv/include/asm/alternative.h | 3 ++-
arch/riscv/kernel/alternative.c | 18 +++++++++++++----
arch/riscv/kernel/module.c | 29 ++++++++++++++++++++++++++++
4 files changed, 49 insertions(+), 8 deletions(-)
diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c
index 4fe03ac41fd7..42457efcaaa3 100644
--- a/arch/riscv/errata/sifive/errata.c
+++ b/arch/riscv/errata/sifive/errata.c
@@ -4,6 +4,7 @@
*/
#include <linux/kernel.h>
+#include <linux/module.h>
#include <linux/string.h>
#include <linux/bug.h>
#include <asm/patch.h>
@@ -54,7 +55,7 @@ static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
},
};
-static u32 __init sifive_errata_probe(unsigned long archid, unsigned long impid)
+static u32 __init_or_module sifive_errata_probe(unsigned long archid, unsigned long impid)
{
int idx;
u32 cpu_req_errata = 0;
@@ -66,7 +67,7 @@ static u32 __init sifive_errata_probe(unsigned long archid, unsigned long impid)
return cpu_req_errata;
}
-static void __init warn_miss_errata(u32 miss_errata)
+static void __init_or_module warn_miss_errata(u32 miss_errata)
{
int i;
@@ -79,7 +80,7 @@ static void __init warn_miss_errata(u32 miss_errata)
pr_warn("----------------------------------------------------------------\n");
}
-void __init sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
+void __init_or_module sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
unsigned long archid, unsigned long impid,
unsigned int stage)
{
diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
index 811bdd8027db..f0657b1b3174 100644
--- a/arch/riscv/include/asm/alternative.h
+++ b/arch/riscv/include/asm/alternative.h
@@ -18,8 +18,10 @@
#include <asm/hwcap.h>
#define RISCV_ALTERNATIVES_BOOT 0 /* alternatives applied during regular boot */
+#define RISCV_ALTERNATIVES_MODULE 1 /* alternatives applied during module-init */
void __init apply_boot_alternatives(void);
+void apply_module_alternatives(void *start, size_t length);
struct alt_entry {
void *old_ptr; /* address of original instruciton or data */
@@ -37,6 +39,5 @@ struct errata_checkfunc_id {
void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
unsigned long archid, unsigned long impid,
unsigned int stage);
-
#endif
#endif
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index 02db62f55bac..223770b3945c 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -7,6 +7,7 @@
*/
#include <linux/init.h>
+#include <linux/module.h>
#include <linux/cpu.h>
#include <linux/uaccess.h>
#include <asm/alternative.h>
@@ -23,7 +24,7 @@ static struct cpu_manufacturer_info_t {
static void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
unsigned long archid, unsigned long impid,
- unsigned int stage) __initdata;
+ unsigned int stage) __initdata_or_module;
static inline void __init riscv_fill_cpu_mfr_info(void)
{
@@ -58,9 +59,9 @@ static void __init init_alternative(void)
* a feature detect on the boot CPU). No need to worry about other CPUs
* here.
*/
-static void __init _apply_alternatives(struct alt_entry *begin,
- struct alt_entry *end,
- unsigned int stage)
+static void __init_or_module _apply_alternatives(struct alt_entry *begin,
+ struct alt_entry *end,
+ unsigned int stage)
{
if (!vendor_patch_func)
return;
@@ -81,3 +82,12 @@ void __init apply_boot_alternatives(void)
(struct alt_entry *)__alt_end,
RISCV_ALTERNATIVES_BOOT);
}
+
+#ifdef CONFIG_MODULES
+void apply_module_alternatives(void *start, size_t length)
+{
+ _apply_alternatives((struct alt_entry *)start,
+ (struct alt_entry *)(start + length),
+ RISCV_ALTERNATIVES_MODULE);
+}
+#endif
diff --git a/arch/riscv/kernel/module.c b/arch/riscv/kernel/module.c
index 4a48287513c3..823b7ad47c9a 100644
--- a/arch/riscv/kernel/module.c
+++ b/arch/riscv/kernel/module.c
@@ -11,6 +11,7 @@
#include <linux/vmalloc.h>
#include <linux/sizes.h>
#include <linux/pgtable.h>
+#include <asm/alternative.h>
#include <asm/sections.h>
/*
@@ -427,3 +428,31 @@ void *module_alloc(unsigned long size)
__builtin_return_address(0));
}
#endif
+
+static const Elf_Shdr *find_section(const Elf_Ehdr *hdr,
+ const Elf_Shdr *sechdrs,
+ const char *name)
+{
+ const Elf_Shdr *s, *se;
+ const char *secstrs = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset;
+
+ for (s = sechdrs, se = sechdrs + hdr->e_shnum; s < se; s++) {
+ if (strcmp(name, secstrs + s->sh_name) == 0)
+ return s;
+ }
+
+ return NULL;
+}
+
+int module_finalize(const Elf_Ehdr *hdr,
+ const Elf_Shdr *sechdrs,
+ struct module *me)
+{
+ const Elf_Shdr *s;
+
+ s = find_section(hdr, sechdrs, ".alternative");
+ if (s)
+ apply_module_alternatives((void *)s->sh_addr, s->sh_size);
+
+ return 0;
+}
--
2.35.1
Future features may need to be applied at a different
time during boot, so allow defining stages for alternatives
and handling them differently depending on the stage.
Also make the alternatives-location more flexible so that
future stages may provide their own location.
Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/errata/sifive/errata.c | 3 ++-
arch/riscv/include/asm/alternative.h | 5 ++++-
arch/riscv/kernel/alternative.c | 26 +++++++++++++++++---------
3 files changed, 23 insertions(+), 11 deletions(-)
diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c
index f5e5ae70e829..4fe03ac41fd7 100644
--- a/arch/riscv/errata/sifive/errata.c
+++ b/arch/riscv/errata/sifive/errata.c
@@ -80,7 +80,8 @@ static void __init warn_miss_errata(u32 miss_errata)
}
void __init sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
- unsigned long archid, unsigned long impid)
+ unsigned long archid, unsigned long impid,
+ unsigned int stage)
{
struct alt_entry *alt;
u32 cpu_req_errata = sifive_errata_probe(archid, impid);
diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
index e625d3cafbed..811bdd8027db 100644
--- a/arch/riscv/include/asm/alternative.h
+++ b/arch/riscv/include/asm/alternative.h
@@ -17,6 +17,8 @@
#include <linux/stddef.h>
#include <asm/hwcap.h>
+#define RISCV_ALTERNATIVES_BOOT 0 /* alternatives applied during regular boot */
+
void __init apply_boot_alternatives(void);
struct alt_entry {
@@ -33,7 +35,8 @@ struct errata_checkfunc_id {
};
void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
- unsigned long archid, unsigned long impid);
+ unsigned long archid, unsigned long impid,
+ unsigned int stage);
#endif
#endif
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index e8b4a0fe488c..02db62f55bac 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -22,8 +22,8 @@ static struct cpu_manufacturer_info_t {
} cpu_mfr_info;
static void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
- unsigned long archid,
- unsigned long impid) __initdata;
+ unsigned long archid, unsigned long impid,
+ unsigned int stage) __initdata;
static inline void __init riscv_fill_cpu_mfr_info(void)
{
@@ -58,6 +58,18 @@ static void __init init_alternative(void)
* a feature detect on the boot CPU). No need to worry about other CPUs
* here.
*/
+static void __init _apply_alternatives(struct alt_entry *begin,
+ struct alt_entry *end,
+ unsigned int stage)
+{
+ if (!vendor_patch_func)
+ return;
+
+ vendor_patch_func(begin, end,
+ cpu_mfr_info.arch_id, cpu_mfr_info.imp_id,
+ stage);
+}
+
void __init apply_boot_alternatives(void)
{
/* If called on non-boot cpu things could go wrong */
@@ -65,11 +77,7 @@ void __init apply_boot_alternatives(void)
init_alternative();
- if (!vendor_patch_func)
- return;
-
- vendor_patch_func((struct alt_entry *)__alt_start,
- (struct alt_entry *)__alt_end,
- cpu_mfr_info.arch_id, cpu_mfr_info.imp_id);
+ _apply_alternatives((struct alt_entry *)__alt_start,
+ (struct alt_entry *)__alt_end,
+ RISCV_ALTERNATIVES_BOOT);
}
-
--
2.35.1
Right now the code uses a global struct to store vendor-ids
and another global variable to store the vendor-patch-function.
There exist specific cases where we'll need to patch the kernel
at an even earlier stage, where trying to write to a static
variable might actually result in hangs.
Also collecting the vendor-information consists of 3 sbi-ecalls
(or csr-reads) which is pretty negligible in the context of
booting a kernel.
So rework the code to not rely on static variables and instead
collect the vendor-information when a round of alternatives is
to be applied.
Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/kernel/alternative.c | 51 ++++++++++++++++-----------------
1 file changed, 24 insertions(+), 27 deletions(-)
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index e6c9de9f9ba6..27f722ae452b 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -16,41 +16,35 @@
#include <asm/sbi.h>
#include <asm/csr.h>
-static struct cpu_manufacturer_info_t {
+struct cpu_manufacturer_info_t {
unsigned long vendor_id;
unsigned long arch_id;
unsigned long imp_id;
-} cpu_mfr_info;
+ void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
+ unsigned long archid, unsigned long impid,
+ unsigned int stage);
+};
-static void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
- unsigned long archid, unsigned long impid,
- unsigned int stage) __initdata_or_module;
-
-static inline void __init riscv_fill_cpu_mfr_info(void)
+static void __init_or_module riscv_fill_cpu_mfr_info(struct cpu_manufacturer_info_t *cpu_mfr_info)
{
#ifdef CONFIG_RISCV_M_MODE
- cpu_mfr_info.vendor_id = csr_read(CSR_MVENDORID);
- cpu_mfr_info.arch_id = csr_read(CSR_MARCHID);
- cpu_mfr_info.imp_id = csr_read(CSR_MIMPID);
+ cpu_mfr_info->vendor_id = csr_read(CSR_MVENDORID);
+ cpu_mfr_info->arch_id = csr_read(CSR_MARCHID);
+ cpu_mfr_info->imp_id = csr_read(CSR_MIMPID);
#else
- cpu_mfr_info.vendor_id = sbi_get_mvendorid();
- cpu_mfr_info.arch_id = sbi_get_marchid();
- cpu_mfr_info.imp_id = sbi_get_mimpid();
+ cpu_mfr_info->vendor_id = sbi_get_mvendorid();
+ cpu_mfr_info->arch_id = sbi_get_marchid();
+ cpu_mfr_info->imp_id = sbi_get_mimpid();
#endif
-}
-
-static void __init init_alternative(void)
-{
- riscv_fill_cpu_mfr_info();
- switch (cpu_mfr_info.vendor_id) {
+ switch (cpu_mfr_info->vendor_id) {
#ifdef CONFIG_ERRATA_SIFIVE
case SIFIVE_VENDOR_ID:
- vendor_patch_func = sifive_errata_patch_func;
+ cpu_mfr_info->vendor_patch_func = sifive_errata_patch_func;
break;
#endif
default:
- vendor_patch_func = NULL;
+ cpu_mfr_info->vendor_patch_func = NULL;
}
}
@@ -63,14 +57,19 @@ static void __init_or_module _apply_alternatives(struct alt_entry *begin,
struct alt_entry *end,
unsigned int stage)
{
+ struct cpu_manufacturer_info_t cpu_mfr_info;
+
+ riscv_fill_cpu_mfr_info(&cpu_mfr_info);
+
riscv_cpufeature_patch_func(begin, end, stage);
- if (!vendor_patch_func)
+ if (!cpu_mfr_info.vendor_patch_func)
return;
- vendor_patch_func(begin, end,
- cpu_mfr_info.arch_id, cpu_mfr_info.imp_id,
- stage);
+ cpu_mfr_info.vendor_patch_func(begin, end,
+ cpu_mfr_info.arch_id,
+ cpu_mfr_info.imp_id,
+ stage);
}
void __init apply_boot_alternatives(void)
@@ -78,8 +77,6 @@ void __init apply_boot_alternatives(void)
/* If called on non-boot cpu things could go wrong */
WARN_ON(smp_processor_id() != 0);
- init_alternative();
-
_apply_alternatives((struct alt_entry *)__alt_start,
(struct alt_entry *)__alt_end,
RISCV_ALTERNATIVES_BOOT);
--
2.35.1
The callback used inside sbi_remote_fence_i is set at sbi probe time
to the needed variant. Before that it is a NULL pointer.
Some users like the flush_icache_*() functions suggest a generic
functionality, that doesn't depend on a specific boot-stage but
uses sbi_remote_fence_i as one option to flush other cpu cores.
So they definitely shouldn't run into null-pointer dereference
issues when called "too early" during boot.
So introduce an empty function to be the standard for the __sbi_rfence
function pointer until sbi_init has run.
Users of sbi_remote_fence_i will have separate code for the local
cpu and sbi_init() is called before other cpus are brought up.
So there are no other cpus present at the time when the issue
might happen.
Signed-off-by: Heiko Stuebner <[email protected]>
Reviewed-by: Atish Patra <[email protected]>
---
arch/riscv/kernel/sbi.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
index 775d3322b422..5a60a458c0b7 100644
--- a/arch/riscv/kernel/sbi.c
+++ b/arch/riscv/kernel/sbi.c
@@ -16,11 +16,19 @@
unsigned long sbi_spec_version __ro_after_init = SBI_SPEC_VERSION_DEFAULT;
EXPORT_SYMBOL(sbi_spec_version);
+static int __sbi_rfence_none(int fid, const struct cpumask *cpu_mask,
+ unsigned long start, unsigned long size,
+ unsigned long arg4, unsigned long arg5)
+{
+ return -EOPNOTSUPP;
+}
+
static void (*__sbi_set_timer)(uint64_t stime) __ro_after_init;
static int (*__sbi_send_ipi)(const struct cpumask *cpu_mask) __ro_after_init;
static int (*__sbi_rfence)(int fid, const struct cpumask *cpu_mask,
unsigned long start, unsigned long size,
- unsigned long arg4, unsigned long arg5) __ro_after_init;
+ unsigned long arg4, unsigned long arg5)
+ __ro_after_init = __sbi_rfence_none;
struct sbiret sbi_ecall(int ext, int fid, unsigned long arg0,
unsigned long arg1, unsigned long arg2,
--
2.35.1
If not defined in the arch, FIXMAP_PAGE_IO defaults to PAGE_KERNEL_IO,
which we defined when adding the svpbmt implementation.
So drop the FIXMAP_PAGE_IO riscv define.
Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/include/asm/fixmap.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/arch/riscv/include/asm/fixmap.h b/arch/riscv/include/asm/fixmap.h
index 58a718573ad6..692e25e56d8e 100644
--- a/arch/riscv/include/asm/fixmap.h
+++ b/arch/riscv/include/asm/fixmap.h
@@ -44,8 +44,6 @@ enum fixed_addresses {
__end_of_fixed_addresses
};
-#define FIXMAP_PAGE_IO PAGE_KERNEL
-
#define __early_set_fixmap __set_fixmap
#define __late_set_fixmap __set_fixmap
--
2.35.1
ALT_NEW_CONTENT already uses same-length assembler lines, so
extend this to the other elements as well.
This makes it more readable when these elements need to be extended
in the future.
Signed-off-by: Heiko Stuebner <[email protected]>
---
arch/riscv/include/asm/alternative-macros.h | 30 ++++++++++-----------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
index 259d8246d612..25b4559d743b 100644
--- a/arch/riscv/include/asm/alternative-macros.h
+++ b/arch/riscv/include/asm/alternative-macros.h
@@ -58,14 +58,14 @@
#include <asm/asm.h>
#include <linux/stringify.h>
-#define ALT_ENTRY(oldptr, newptr, vendor_id, errata_id, newlen) \
- RISCV_PTR " " oldptr "\n" \
- RISCV_PTR " " newptr "\n" \
- REG_ASM " " vendor_id "\n" \
- REG_ASM " " newlen "\n" \
+#define ALT_ENTRY(oldptr, newptr, vendor_id, errata_id, newlen) \
+ RISCV_PTR " " oldptr "\n" \
+ RISCV_PTR " " newptr "\n" \
+ REG_ASM " " vendor_id "\n" \
+ REG_ASM " " newlen "\n" \
".word " errata_id "\n"
-#define ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c) \
+#define ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c) \
".if " __stringify(enable) " == 1\n" \
".pushsection .alternative, \"a\"\n" \
ALT_ENTRY("886b", "888f", __stringify(vendor_id), __stringify(errata_id), "889f - 888f") \
@@ -79,21 +79,21 @@
".org . - (889b - 888b) + (887b - 886b)\n" \
".endif\n"
-#define __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, enable) \
- "886 :\n" \
- old_c "\n" \
- "887 :\n" \
+#define __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, enable) \
+ "886 :\n" \
+ old_c "\n" \
+ "887 :\n" \
ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c)
-#define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
+#define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
__ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, IS_ENABLED(CONFIG_k))
#define __ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
new_c_2, vendor_id_2, errata_id_2, enable_2) \
- "886 :\n" \
- old_c "\n" \
- "887 :\n" \
- ALT_NEW_CONTENT(vendor_id_1, errata_id_1, enable_1, new_c_1) \
+ "886 :\n" \
+ old_c "\n" \
+ "887 :\n" \
+ ALT_NEW_CONTENT(vendor_id_1, errata_id_1, enable_1, new_c_1) \
ALT_NEW_CONTENT(vendor_id_2, errata_id_2, enable_2, new_c_2)
#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
--
2.35.1
On Wed, 23 Mar 2022 17:06:56 PDT (-0700), [email protected] wrote:
> Svpbmt is an extension defining "Supervisor-mode: page-based memory types"
> for things like non-cacheable pages or I/O memory pages.
>
>
> So this is my 2nd try at implementing Svpbmt (and the diverging D1 memory
> types) using the alternatives framework.
>
> This includes a number of changes to the alternatives mechanism itself.
> The biggest one being the move to a more central location, as I expect
> in the future, nearly every chip needing some sort of patching, be it
> either for erratas or for optional features (svpbmt or others).
>
> Detection of the svpbmt functionality is done via Atish's isa extension
> handling series [0] and thus does not need any dt-parsing of its own
> anymore.
>
> The series also introduces support for the memory types of the D1
> which are implemented differently to svpbmt. But when patching anyway
> it's pretty clean to add the D1 variant via ALTERNATIVE_2 to the same
> location.
>
> The only slightly bigger difference is that the "normal" type is not 0
> as with svpbmt, so kernel patches for this PMA type need to be applied
> even before the MMU is brought up, so the series introduces a separate
> stage for that.
>
>
> In theory this series is 3 parts:
> - sbi cache-flush / null-ptr
> - alternatives improvements
> - svpbmt+d1
>
> So expecially patches from the first 2 areas could be applied when
> deemed ready, I just thought to keep it together to show-case where
> the end-goal is and not requiring jumping between different series.
>
>
> I picked the recipient list from the previous versions, hopefully
> I didn't forget anybody.
>
> changes in v8:
> - rebase onto 5.17-final + isa extension series
> We're halfway through the merge-window, so this series
> should be merge after that
> - adapt to fix limiting alternatives to non-xip-kernels
> - add .norelax option for alternatives
> - fix unused cpu_apply_errata in thead errata
> - don't use static globals to store cpu-manufacturer data
> as it makes machines hang if done too early
I'm still seeing a very similar failure mode. There were a few merge
conflicts when I tried to put this on top of my merged first PR, I've
put what I'm testing at riscv-d1. Not sure if I'm missing something.
>
> changes in v7:
> - fix typo in patch1 (Atish)
> - moved to Atish's isa-extension framework
> - and therefore move regular boot-alternatives directly behind fill_hwcaps
> - change T-Head errata Kconfig text (Atish)
>
> changes in v6:
> - rebase onto 5.17-rc1
> - handle sbi null-ptr differently
> - improve commit messages
> - use riscv,mmu as property name
>
> changes in v5:
> - move to use alternatives for runtime-patching
> - add D1 variant
>
>
> [0] https://lore.kernel.org/r/[email protected]
>
>
> Heiko Stuebner (13):
> riscv: prevent null-pointer dereference with sbi_remote_fence_i
> riscv: integrate alternatives better into the main architecture
> riscv: allow different stages with alternatives
> riscv: implement module alternatives
> riscv: implement ALTERNATIVE_2 macro
> riscv: extend concatenated alternatives-lines to the same length
> riscv: prevent compressed instructions in alternatives
> riscv: move boot alternatives to after fill_hwcap
> riscv: Fix accessing pfn bits in PTEs for non-32bit variants
> riscv: add cpufeature handling via alternatives
> riscv: remove FIXMAP_PAGE_IO and fall back to its default value
> riscv: don't use global static vars to store alternative data
> riscv: add memory-type errata for T-Head
>
> Wei Fu (1):
> riscv: add RISC-V Svpbmt extension support
>
> arch/riscv/Kconfig.erratas | 31 +++--
> arch/riscv/Kconfig.socs | 1 -
> arch/riscv/Makefile | 2 +-
> arch/riscv/errata/Makefile | 2 +-
> arch/riscv/errata/alternative.c | 75 ------------
> arch/riscv/errata/sifive/errata.c | 17 ++-
> arch/riscv/errata/thead/Makefile | 1 +
> arch/riscv/errata/thead/errata.c | 82 +++++++++++++
> arch/riscv/include/asm/alternative-macros.h | 121 ++++++++++++++------
> arch/riscv/include/asm/alternative.h | 16 ++-
> arch/riscv/include/asm/errata_list.h | 52 +++++++++
> arch/riscv/include/asm/fixmap.h | 2 -
> arch/riscv/include/asm/hwcap.h | 1 +
> arch/riscv/include/asm/pgtable-32.h | 17 +++
> arch/riscv/include/asm/pgtable-64.h | 79 ++++++++++++-
> arch/riscv/include/asm/pgtable-bits.h | 10 --
> arch/riscv/include/asm/pgtable.h | 53 +++++++--
> arch/riscv/include/asm/vendorid_list.h | 1 +
> arch/riscv/kernel/Makefile | 1 +
> arch/riscv/kernel/alternative.c | 104 +++++++++++++++++
> arch/riscv/kernel/cpu.c | 1 +
> arch/riscv/kernel/cpufeature.c | 80 ++++++++++++-
> arch/riscv/kernel/module.c | 29 +++++
> arch/riscv/kernel/sbi.c | 10 +-
> arch/riscv/kernel/setup.c | 2 +
> arch/riscv/kernel/smpboot.c | 4 -
> arch/riscv/kernel/traps.c | 2 +-
> arch/riscv/mm/init.c | 1 +
> 28 files changed, 629 insertions(+), 168 deletions(-)
> delete mode 100644 arch/riscv/errata/alternative.c
> create mode 100644 arch/riscv/errata/thead/Makefile
> create mode 100644 arch/riscv/errata/thead/errata.c
> create mode 100644 arch/riscv/kernel/alternative.c
On Thu, Mar 24, 2022 at 01:06:57AM +0100, Heiko Stuebner wrote:
> The callback used inside sbi_remote_fence_i is set at sbi probe time
> to the needed variant. Before that it is a NULL pointer.
>
> Some users like the flush_icache_*() functions suggest a generic
> functionality, that doesn't depend on a specific boot-stage but
> uses sbi_remote_fence_i as one option to flush other cpu cores.
>
> So they definitely shouldn't run into null-pointer dereference
> issues when called "too early" during boot.
>
> So introduce an empty function to be the standard for the __sbi_rfence
> function pointer until sbi_init has run.
>
> Users of sbi_remote_fence_i will have separate code for the local
> cpu and sbi_init() is called before other cpus are brought up.
> So there are no other cpus present at the time when the issue
> might happen.
I don't really understand this changelog. If flush_icache_* or
other routines using SBI calls are called too early they won't
do what they are asked to do, which implies a bug in the code.
So crashing absolutely is the right thing to do here as we don't
really have any other error reporting method available.
So unless I'm totally misunderstanding what you are saying here:
Nacked-by: Christoph Hellwig <[email protected]>
On Thu, Mar 24, 2022 at 01:06:58AM +0100, Heiko Stuebner wrote:
> Right now the alternatives need to be explicitly enabled and
> erratas are limited to SiFive ones.
>
> Over time with more SoCs and additional RiscV extensions, many more
> erratas or other patch-worthy features will emerge, so it doesn't
> really make sense to have the core alternatives able to get
> deactivated.
>
> So make it part of the core RiscV kernel and drop the main
> RISCV_ERRATA_ALTERNATIVES config symbol.
>
> This mimics how other architectures like for example arm64 handle
> their alternatives implementation.
For minimal kernels like the k210 it would be really good to be
able to avoid any not strictly neeed code. So I'd much rather
have the alternatives mechanism only built when it actually is needed,
not (semi-)unconditional.
On Thu, Mar 24, 2022 at 01:07:04AM +0100, Heiko Stuebner wrote:
> Move the application of boot alternatives to after the hw-capabilities
> are populated. This allows to check for available extensions when
> determining which alternatives to apply and also makes it actually
> work if CONFIG_SMP is disabled for whatever reason.
Looks good:
Reviewed-by: Christoph Hellwig <[email protected]>
Hi Christoph,
Am Donnerstag, 31. M?rz 2022, 12:03:37 CEST schrieb Christoph Hellwig:
>
looks like this mail somehow lost its content.
At first I assumed some issue on my end, but lore [0] agrees
that this mail somehow was empty.
Heiko
[0] https://lore.kernel.org/all/[email protected]/
On Thu, Mar 24, 2022 at 01:07:08AM +0100, Heiko Stuebner wrote:
> If not defined in the arch, FIXMAP_PAGE_IO defaults to PAGE_KERNEL_IO,
> which we defined when adding the svpbmt implementation.
>
> So drop the FIXMAP_PAGE_IO riscv define.
Looks good:
Reviewed-by: Christoph Hellwig <[email protected]>
On Thu, Mar 31, 2022 at 02:28:06PM +0200, Heiko St?bner wrote:
> so essentially flushes the _local_ icache first and then tries to flush
> caches on other cores, either via an ipi or via sbi.
>
> The remote-fence callback is set correctly during sbi_init().
> The other cores are only brought up after sbi-init is done.
>
> So it's not really about error reporting but making sure that flush_icache_all()
> does something sane even when still running on the first core.
> As I assume the "all" means on all available cores (which would be the
> core the system booted on).
>
> Does this make it clearer what this tries to solve?
A little. Whatever code calls this early still seems broken to me
and just just do a local flush, though.
On Thu, Mar 31, 2022 at 02:19:15PM +0200, Heiko St?bner wrote:
> Hi Christoph,
>
> Am Donnerstag, 31. M?rz 2022, 12:03:37 CEST schrieb Christoph Hellwig:
> >
>
> looks like this mail somehow lost its content.
>
> At first I assumed some issue on my end, but lore [0] agrees
> that this mail somehow was empty.
It also is empty in my sent mail folder, and I have no idea why.
On Thu, Mar 24, 2022 at 01:07:07AM +0100, Heiko Stuebner wrote:
> +asm(ALTERNATIVE("li %0, 0\t\nnop", "li %0, %1\t\nslli %0,%0,%2", 0, \
> + CPUFEATURE_SVPBMT, CONFIG_64BIT) \
> + : "=r"(_val) : "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT), "I"(ALT_SVPBMT_SHIFT))
This would be much more readable as;
asm(ALTERNATIVE("li %0, 0\t\nnop", "li %0, %1\t\nslli %0,%0,%2", 0, \
CPUFEATURE_SVPBMT, CONFIG_64BIT) \
: "=r"(_val) \
: "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT), \
"I"(ALT_SVPBMT_SHIFT))
Otherwise this looks good to me:
Reviewed-by: Christoph Hellwig <[email protected]>
Hi,
Am Donnerstag, 31. M?rz 2022, 11:51:55 CEST schrieb Christoph Hellwig:
> On Thu, Mar 24, 2022 at 01:06:57AM +0100, Heiko Stuebner wrote:
> > The callback used inside sbi_remote_fence_i is set at sbi probe time
> > to the needed variant. Before that it is a NULL pointer.
> >
> > Some users like the flush_icache_*() functions suggest a generic
> > functionality, that doesn't depend on a specific boot-stage but
> > uses sbi_remote_fence_i as one option to flush other cpu cores.
> >
> > So they definitely shouldn't run into null-pointer dereference
> > issues when called "too early" during boot.
> >
> > So introduce an empty function to be the standard for the __sbi_rfence
> > function pointer until sbi_init has run.
> >
> > Users of sbi_remote_fence_i will have separate code for the local
> > cpu and sbi_init() is called before other cpus are brought up.
> > So there are no other cpus present at the time when the issue
> > might happen.
>
> I don't really understand this changelog. If flush_icache_* or
> other routines using SBI calls are called too early they won't
> do what they are asked to do, which implies a bug in the code.
>
> So crashing absolutely is the right thing to do here as we don't
> really have any other error reporting method available.
>
> So unless I'm totally misunderstanding what you are saying here:
>
> Nacked-by: Christoph Hellwig <[email protected]>
The function is defined as
void flush_icache_all(void)
{
local_flush_icache_all();
if (IS_ENABLED(CONFIG_RISCV_SBI))
sbi_remote_fence_i(NULL);
else
on_each_cpu(ipi_remote_fence_i, NULL, 1);
}
so essentially flushes the _local_ icache first and then tries to flush
caches on other cores, either via an ipi or via sbi.
The remote-fence callback is set correctly during sbi_init().
The other cores are only brought up after sbi-init is done.
So it's not really about error reporting but making sure that flush_icache_all()
does something sane even when still running on the first core.
As I assume the "all" means on all available cores (which would be the
core the system booted on).
Does this make it clearer what this tries to solve?
Heiko
Am Donnerstag, 31. M?rz 2022, 14:33:22 CEST schrieb Christoph Hellwig:
> On Thu, Mar 31, 2022 at 02:28:06PM +0200, Heiko St?bner wrote:
> > so essentially flushes the _local_ icache first and then tries to flush
> > caches on other cores, either via an ipi or via sbi.
> >
> > The remote-fence callback is set correctly during sbi_init().
> > The other cores are only brought up after sbi-init is done.
> >
> > So it's not really about error reporting but making sure that flush_icache_all()
> > does something sane even when still running on the first core.
> > As I assume the "all" means on all available cores (which would be the
> > core the system booted on).
> >
> > Does this make it clearer what this tries to solve?
>
> A little. Whatever code calls this early still seems broken to me
> and just just do a local flush, though.
Looking at this again made me realize that we don't need this patch
anymore at all.
In a previous revision the "main" alternatives apply-point was earlier,
triggering the issue when the generic patching-functions wanted to
flush the cache, but with the alternatives getting applied where they
are now, the issue is not triggered anymore.
And the "early" alternatives use a local flush anyway.