Currently, riscv has several features why may not be supported on all
riscv platforms, for example, FPU, SV48 and so on. To support unified
kernel Image style, we need to check whether the feature is suportted
or not. If the check sits at hot code path, then performance will be
impacted a lot. static key can be used to solve the issue. In the
past FPU support has been converted to use static key mechanism. I
believe we will have similar cases in the future. For example, the
SV48 support can take advantage of static key[1].
patch1 introduces an unified mechanism to use static key for riscv cpu
features.
patch2 converts has_cpu() to use the mechanism.
patch3 uses the mechanism to optimize pgtable_l4_enabled.
[1] http://lists.infradead.org/pipermail/linux-riscv/2021-December/011164.html
Jisheng Zhang (3):
riscv: introduce unified static key mechanism for CPU features
riscv: replace has_fpu() with system_supports_fpu()
riscv: convert pgtable_l4_enabled to static key
arch/riscv/Makefile | 3 +
arch/riscv/include/asm/cpufeature.h | 105 ++++++++++++++++++++++++++++
arch/riscv/include/asm/pgalloc.h | 8 +--
arch/riscv/include/asm/pgtable-64.h | 21 +++---
arch/riscv/include/asm/pgtable.h | 3 +-
arch/riscv/include/asm/switch_to.h | 9 +--
arch/riscv/kernel/cpu.c | 2 +-
arch/riscv/kernel/cpufeature.c | 29 ++++++--
arch/riscv/kernel/process.c | 2 +-
arch/riscv/kernel/signal.c | 4 +-
arch/riscv/mm/init.c | 23 +++---
arch/riscv/mm/kasan_init.c | 6 +-
arch/riscv/tools/Makefile | 22 ++++++
arch/riscv/tools/cpucaps | 6 ++
arch/riscv/tools/gen-cpucaps.awk | 40 +++++++++++
15 files changed, 234 insertions(+), 49 deletions(-)
create mode 100644 arch/riscv/include/asm/cpufeature.h
create mode 100644 arch/riscv/tools/Makefile
create mode 100644 arch/riscv/tools/cpucaps
create mode 100755 arch/riscv/tools/gen-cpucaps.awk
--
2.34.1
Currently, riscv has several features why may not be supported on all
riscv platforms, for example, FPU, SV48 and so on. To support unified
kernel Image style, we need to check whether the feature is suportted
or not. If the check sits at hot code path, then performance will be
impacted a lot. static key can be used to solve the issue. In the past
FPU support has been converted to use static key mechanism. I believe
we will have similar cases in the future.
Similar as arm64 does(in fact, some code is borrowed from arm64), this
patch tries to add an unified mechanism to use static keys for all
the cpu features by implementing an array of default-false static keys
and enabling them when detected. The cpus_have_*_cap() check uses the
static keys if riscv_const_caps_ready is finalized, otherwise the
compiler generates the bitmap test.
Signed-off-by: Jisheng Zhang <[email protected]>
---
arch/riscv/Makefile | 3 +
arch/riscv/include/asm/cpufeature.h | 94 +++++++++++++++++++++++++++++
arch/riscv/kernel/cpufeature.c | 23 +++++++
arch/riscv/tools/Makefile | 22 +++++++
arch/riscv/tools/cpucaps | 5 ++
arch/riscv/tools/gen-cpucaps.awk | 40 ++++++++++++
6 files changed, 187 insertions(+)
create mode 100644 arch/riscv/include/asm/cpufeature.h
create mode 100644 arch/riscv/tools/Makefile
create mode 100644 arch/riscv/tools/cpucaps
create mode 100755 arch/riscv/tools/gen-cpucaps.awk
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 8a107ed18b0d..65c63023c8a8 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -148,3 +148,6 @@ PHONY += rv64_randconfig
rv64_randconfig:
$(Q)$(MAKE) KCONFIG_ALLCONFIG=$(srctree)/arch/riscv/configs/64-bit.config \
-f $(srctree)/Makefile randconfig
+
+archprepare:
+ $(Q)$(MAKE) $(build)=arch/riscv/tools kapi
diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h
new file mode 100644
index 000000000000..d80ddd2f3b49
--- /dev/null
+++ b/arch/riscv/include/asm/cpufeature.h
@@ -0,0 +1,94 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2014 Linaro Ltd. <[email protected]>
+ * Copyright (C) 2022 Jisheng Zhang <[email protected]>
+ */
+
+#ifndef __ASM_CPUFEATURE_H
+#define __ASM_CPUFEATURE_H
+
+#include <asm/cpucaps.h>
+
+#include <linux/bug.h>
+#include <linux/jump_label.h>
+#include <linux/kernel.h>
+
+extern DECLARE_BITMAP(cpu_hwcaps, RISCV_NCAPS);
+extern struct static_key_false cpu_hwcap_keys[RISCV_NCAPS];
+extern struct static_key_false riscv_const_caps_ready;
+
+static __always_inline bool system_capabilities_finalized(void)
+{
+ return static_branch_likely(&riscv_const_caps_ready);
+}
+
+/*
+ * Test for a capability with a runtime check.
+ *
+ * Before the capability is detected, this returns false.
+ */
+static inline bool cpus_have_cap(unsigned int num)
+{
+ if (num >= RISCV_NCAPS)
+ return false;
+ return test_bit(num, cpu_hwcaps);
+}
+
+/*
+ * Test for a capability without a runtime check.
+ *
+ * Before capabilities are finalized, this returns false.
+ * After capabilities are finalized, this is patched to avoid a runtime check.
+ *
+ * @num must be a compile-time constant.
+ */
+static __always_inline bool __cpus_have_const_cap(int num)
+{
+ if (num >= RISCV_NCAPS)
+ return false;
+ return static_branch_unlikely(&cpu_hwcap_keys[num]);
+}
+
+/*
+ * Test for a capability without a runtime check.
+ *
+ * Before capabilities are finalized, this will BUG().
+ * After capabilities are finalized, this is patched to avoid a runtime check.
+ *
+ * @num must be a compile-time constant.
+ */
+static __always_inline bool cpus_have_final_cap(int num)
+{
+ if (system_capabilities_finalized())
+ return __cpus_have_const_cap(num);
+ else
+ BUG();
+}
+
+/*
+ * Test for a capability, possibly with a runtime check.
+ *
+ * Before capabilities are finalized, this behaves as cpus_have_cap().
+ * After capabilities are finalized, this is patched to avoid a runtime check.
+ *
+ * @num must be a compile-time constant.
+ */
+static __always_inline bool cpus_have_const_cap(int num)
+{
+ if (system_capabilities_finalized())
+ return __cpus_have_const_cap(num);
+ else
+ return cpus_have_cap(num);
+}
+
+static inline void cpus_set_cap(unsigned int num)
+{
+ if (num >= RISCV_NCAPS) {
+ pr_warn("Attempt to set an illegal CPU capability (%d >= %d)\n",
+ num, RISCV_NCAPS);
+ } else {
+ __set_bit(num, cpu_hwcaps);
+ }
+}
+
+#endif
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index d959d207a40d..09331abfa70c 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -8,6 +8,7 @@
#include <linux/bitmap.h>
#include <linux/of.h>
+#include <asm/cpufeature.h>
#include <asm/processor.h>
#include <asm/hwcap.h>
#include <asm/smp.h>
@@ -22,6 +23,15 @@ static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly;
__ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_fpu);
#endif
+DECLARE_BITMAP(cpu_hwcaps, RISCV_NCAPS);
+EXPORT_SYMBOL(cpu_hwcaps);
+
+DEFINE_STATIC_KEY_ARRAY_FALSE(cpu_hwcap_keys, RISCV_NCAPS);
+EXPORT_SYMBOL(cpu_hwcap_keys);
+
+DEFINE_STATIC_KEY_FALSE(riscv_const_caps_ready);
+EXPORT_SYMBOL(riscv_const_caps_ready);
+
/**
* riscv_isa_extension_base() - Get base extension word
*
@@ -59,6 +69,17 @@ bool __riscv_isa_extension_available(const unsigned long *isa_bitmap, int bit)
}
EXPORT_SYMBOL_GPL(__riscv_isa_extension_available);
+static void __init enable_cpu_capabilities(void)
+{
+ int i;
+
+ for (i = 0; i < RISCV_NCAPS; i++) {
+ if (!cpus_have_cap(i))
+ continue;
+ static_branch_enable(&cpu_hwcap_keys[i]);
+ }
+}
+
void __init riscv_fill_hwcap(void)
{
struct device_node *node;
@@ -148,4 +169,6 @@ void __init riscv_fill_hwcap(void)
if (elf_hwcap & (COMPAT_HWCAP_ISA_F | COMPAT_HWCAP_ISA_D))
static_branch_enable(&cpu_hwcap_fpu);
#endif
+ enable_cpu_capabilities();
+ static_branch_enable(&riscv_const_caps_ready);
}
diff --git a/arch/riscv/tools/Makefile b/arch/riscv/tools/Makefile
new file mode 100644
index 000000000000..932b4fe5c768
--- /dev/null
+++ b/arch/riscv/tools/Makefile
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0
+
+gen := arch/$(ARCH)/include/generated
+kapi := $(gen)/asm
+
+kapi-hdrs-y := $(kapi)/cpucaps.h
+
+targets += $(addprefix ../../../,$(gen-y) $(kapi-hdrs-y))
+
+PHONY += kapi
+
+kapi: $(kapi-hdrs-y) $(gen-y)
+
+# Create output directory if not already present
+_dummy := $(shell [ -d '$(kapi)' ] || mkdir -p '$(kapi)')
+
+quiet_cmd_gen_cpucaps = GEN $@
+ cmd_gen_cpucaps = mkdir -p $(dir $@) && \
+ $(AWK) -f $(filter-out $(PHONY),$^) > $@
+
+$(kapi)/cpucaps.h: $(src)/gen-cpucaps.awk $(src)/cpucaps FORCE
+ $(call if_changed,gen_cpucaps)
diff --git a/arch/riscv/tools/cpucaps b/arch/riscv/tools/cpucaps
new file mode 100644
index 000000000000..cb1ff2747859
--- /dev/null
+++ b/arch/riscv/tools/cpucaps
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Internal CPU capabilities constants, keep this list sorted
+
+HAS_NO_FPU
diff --git a/arch/riscv/tools/gen-cpucaps.awk b/arch/riscv/tools/gen-cpucaps.awk
new file mode 100755
index 000000000000..52a1e1b064ad
--- /dev/null
+++ b/arch/riscv/tools/gen-cpucaps.awk
@@ -0,0 +1,40 @@
+#!/bin/awk -f
+# SPDX-License-Identifier: GPL-2.0
+# gen-cpucaps.awk: riscv cpucaps header generator
+#
+# Usage: awk -f gen-cpucaps.awk cpucaps.txt
+
+# Log an error and terminate
+function fatal(msg) {
+ print "Error at line " NR ": " msg > "/dev/stderr"
+ exit 1
+}
+
+# skip blank lines and comment lines
+/^$/ { next }
+/^#/ { next }
+
+BEGIN {
+ print "#ifndef __ASM_CPUCAPS_H"
+ print "#define __ASM_CPUCAPS_H"
+ print ""
+ print "/* Generated file - do not edit */"
+ cap_num = 0
+ print ""
+}
+
+/^[vA-Z0-9_]+$/ {
+ printf("#define RISCV_%-30s\t%d\n", $0, cap_num++)
+ next
+}
+
+END {
+ printf("#define RISCV_NCAPS\t\t\t\t%d\n", cap_num)
+ print ""
+ print "#endif /* __ASM_CPUCAPS_H */"
+}
+
+# Any lines not handled by previous rules are unexpected
+{
+ fatal("unhandled statement")
+}
--
2.34.1
This is to use the unified cpus_have_{final|const}_cap() instead of
putting static key related here and there.
Signed-off-by: Jisheng Zhang <[email protected]>
---
arch/riscv/include/asm/cpufeature.h | 5 +++++
arch/riscv/include/asm/switch_to.h | 9 ++-------
arch/riscv/kernel/cpufeature.c | 8 ++------
arch/riscv/kernel/process.c | 2 +-
arch/riscv/kernel/signal.c | 4 ++--
5 files changed, 12 insertions(+), 16 deletions(-)
diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h
index d80ddd2f3b49..634a653c7fa2 100644
--- a/arch/riscv/include/asm/cpufeature.h
+++ b/arch/riscv/include/asm/cpufeature.h
@@ -91,4 +91,9 @@ static inline void cpus_set_cap(unsigned int num)
}
}
+static inline bool system_supports_fpu(void)
+{
+ return IS_ENABLED(CONFIG_FPU) && !cpus_have_final_cap(RISCV_HAS_NO_FPU);
+}
+
#endif
diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h
index 0a3f4f95c555..362cb18d12d5 100644
--- a/arch/riscv/include/asm/switch_to.h
+++ b/arch/riscv/include/asm/switch_to.h
@@ -8,6 +8,7 @@
#include <linux/jump_label.h>
#include <linux/sched/task_stack.h>
+#include <asm/cpufeature.h>
#include <asm/processor.h>
#include <asm/ptrace.h>
#include <asm/csr.h>
@@ -56,13 +57,7 @@ static inline void __switch_to_aux(struct task_struct *prev,
fstate_restore(next, task_pt_regs(next));
}
-extern struct static_key_false cpu_hwcap_fpu;
-static __always_inline bool has_fpu(void)
-{
- return static_branch_likely(&cpu_hwcap_fpu);
-}
#else
-static __always_inline bool has_fpu(void) { return false; }
#define fstate_save(task, regs) do { } while (0)
#define fstate_restore(task, regs) do { } while (0)
#define __switch_to_aux(__prev, __next) do { } while (0)
@@ -75,7 +70,7 @@ extern struct task_struct *__switch_to(struct task_struct *,
do { \
struct task_struct *__prev = (prev); \
struct task_struct *__next = (next); \
- if (has_fpu()) \
+ if (system_supports_fpu()) \
__switch_to_aux(__prev, __next); \
((last) = __switch_to(__prev, __next)); \
} while (0)
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index 09331abfa70c..da272b399af6 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -19,10 +19,6 @@ unsigned long elf_hwcap __read_mostly;
/* Host ISA bitmap */
static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly;
-#ifdef CONFIG_FPU
-__ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_fpu);
-#endif
-
DECLARE_BITMAP(cpu_hwcaps, RISCV_NCAPS);
EXPORT_SYMBOL(cpu_hwcaps);
@@ -166,8 +162,8 @@ void __init riscv_fill_hwcap(void)
pr_info("riscv: ELF capabilities %s\n", print_str);
#ifdef CONFIG_FPU
- if (elf_hwcap & (COMPAT_HWCAP_ISA_F | COMPAT_HWCAP_ISA_D))
- static_branch_enable(&cpu_hwcap_fpu);
+ if (!(elf_hwcap & (COMPAT_HWCAP_ISA_F | COMPAT_HWCAP_ISA_D)))
+ cpus_set_cap(RISCV_HAS_NO_FPU);
#endif
enable_cpu_capabilities();
static_branch_enable(&riscv_const_caps_ready);
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index 03ac3aa611f5..ece62392b79f 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -87,7 +87,7 @@ void start_thread(struct pt_regs *regs, unsigned long pc,
unsigned long sp)
{
regs->status = SR_PIE;
- if (has_fpu()) {
+ if (system_supports_fpu()) {
regs->status |= SR_FS_INITIAL;
/*
* Restore the initial value to the FP register
diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c
index c2d5ecbe5526..c236eb777fbc 100644
--- a/arch/riscv/kernel/signal.c
+++ b/arch/riscv/kernel/signal.c
@@ -90,7 +90,7 @@ static long restore_sigcontext(struct pt_regs *regs,
/* sc_regs is structured the same as the start of pt_regs */
err = __copy_from_user(regs, &sc->sc_regs, sizeof(sc->sc_regs));
/* Restore the floating-point state. */
- if (has_fpu())
+ if (system_supports_fpu())
err |= restore_fp_state(regs, &sc->sc_fpregs);
return err;
}
@@ -143,7 +143,7 @@ static long setup_sigcontext(struct rt_sigframe __user *frame,
/* sc_regs is structured the same as the start of pt_regs */
err = __copy_to_user(&sc->sc_regs, regs, sizeof(sc->sc_regs));
/* Save the floating-point state. */
- if (has_fpu())
+ if (system_supports_fpu())
err |= save_fp_state(regs, &sc->sc_fpregs);
return err;
}
--
2.34.1
On a specific HW platform, pgtable_l4_enabled won't change after
boot, and the check sits at hot code path, this characteristic make it
suitable for optimization with static key.
Signed-off-by: Jisheng Zhang <[email protected]>
---
arch/riscv/include/asm/cpufeature.h | 6 ++++++
arch/riscv/include/asm/pgalloc.h | 8 ++++----
arch/riscv/include/asm/pgtable-64.h | 21 ++++++++++-----------
arch/riscv/include/asm/pgtable.h | 3 +--
arch/riscv/kernel/cpu.c | 2 +-
arch/riscv/mm/init.c | 23 ++++++++++-------------
arch/riscv/mm/kasan_init.c | 6 +++---
arch/riscv/tools/cpucaps | 1 +
8 files changed, 36 insertions(+), 34 deletions(-)
diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h
index 634a653c7fa2..10af83d6fb2a 100644
--- a/arch/riscv/include/asm/cpufeature.h
+++ b/arch/riscv/include/asm/cpufeature.h
@@ -96,4 +96,10 @@ static inline bool system_supports_fpu(void)
return IS_ENABLED(CONFIG_FPU) && !cpus_have_final_cap(RISCV_HAS_NO_FPU);
}
+static inline bool system_supports_sv48(void)
+{
+ return IS_ENABLED(CONFIG_64BIT) && !IS_ENABLED(CONFIG_XIP_KERNEL) &&
+ !cpus_have_const_cap(RISCV_HAS_NO_SV48);
+}
+
#endif
diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h
index 11823004b87a..cd37f3777ff1 100644
--- a/arch/riscv/include/asm/pgalloc.h
+++ b/arch/riscv/include/asm/pgalloc.h
@@ -41,7 +41,7 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
static inline void p4d_populate(struct mm_struct *mm, p4d_t *p4d, pud_t *pud)
{
- if (pgtable_l4_enabled) {
+ if (system_supports_sv48()) {
unsigned long pfn = virt_to_pfn(pud);
set_p4d(p4d, __p4d((pfn << _PAGE_PFN_SHIFT) | _PAGE_TABLE));
@@ -51,7 +51,7 @@ static inline void p4d_populate(struct mm_struct *mm, p4d_t *p4d, pud_t *pud)
static inline void p4d_populate_safe(struct mm_struct *mm, p4d_t *p4d,
pud_t *pud)
{
- if (pgtable_l4_enabled) {
+ if (system_supports_sv48()) {
unsigned long pfn = virt_to_pfn(pud);
set_p4d_safe(p4d,
@@ -62,7 +62,7 @@ static inline void p4d_populate_safe(struct mm_struct *mm, p4d_t *p4d,
#define pud_alloc_one pud_alloc_one
static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
return __pud_alloc_one(mm, addr);
return NULL;
@@ -71,7 +71,7 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
#define pud_free pud_free
static inline void pud_free(struct mm_struct *mm, pud_t *pud)
{
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
__pud_free(mm, pud);
}
diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index bbbdd66e5e2f..5ad4311f9c6e 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -7,14 +7,13 @@
#define _ASM_RISCV_PGTABLE_64_H
#include <linux/const.h>
-
-extern bool pgtable_l4_enabled;
+#include <asm/cpufeature.h>
#define PGDIR_SHIFT_L3 30
#define PGDIR_SHIFT_L4 39
#define PGDIR_SIZE_L3 (_AC(1, UL) << PGDIR_SHIFT_L3)
-#define PGDIR_SHIFT (pgtable_l4_enabled ? PGDIR_SHIFT_L4 : PGDIR_SHIFT_L3)
+#define PGDIR_SHIFT (system_supports_sv48() ? PGDIR_SHIFT_L4 : PGDIR_SHIFT_L3)
/* Size of region mapped by a page global directory */
#define PGDIR_SIZE (_AC(1, UL) << PGDIR_SHIFT)
#define PGDIR_MASK (~(PGDIR_SIZE - 1))
@@ -102,7 +101,7 @@ static inline struct page *pud_page(pud_t pud)
#define mm_pud_folded mm_pud_folded
static inline bool mm_pud_folded(struct mm_struct *mm)
{
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
return false;
return true;
@@ -130,7 +129,7 @@ static inline unsigned long _pmd_pfn(pmd_t pmd)
static inline void set_p4d(p4d_t *p4dp, p4d_t p4d)
{
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
*p4dp = p4d;
else
set_pud((pud_t *)p4dp, (pud_t){ p4d_val(p4d) });
@@ -138,7 +137,7 @@ static inline void set_p4d(p4d_t *p4dp, p4d_t p4d)
static inline int p4d_none(p4d_t p4d)
{
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
return (p4d_val(p4d) == 0);
return 0;
@@ -146,7 +145,7 @@ static inline int p4d_none(p4d_t p4d)
static inline int p4d_present(p4d_t p4d)
{
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
return (p4d_val(p4d) & _PAGE_PRESENT);
return 1;
@@ -154,7 +153,7 @@ static inline int p4d_present(p4d_t p4d)
static inline int p4d_bad(p4d_t p4d)
{
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
return !p4d_present(p4d);
return 0;
@@ -162,13 +161,13 @@ static inline int p4d_bad(p4d_t p4d)
static inline void p4d_clear(p4d_t *p4d)
{
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
set_p4d(p4d, __p4d(0));
}
static inline pud_t *p4d_pgtable(p4d_t p4d)
{
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
return (pud_t *)pfn_to_virt(p4d_val(p4d) >> _PAGE_PFN_SHIFT);
return (pud_t *)pud_pgtable((pud_t) { p4d_val(p4d) });
@@ -184,7 +183,7 @@ static inline struct page *p4d_page(p4d_t p4d)
#define pud_offset pud_offset
static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address)
{
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
return p4d_pgtable(*p4d) + pud_index(address);
return (pud_t *)p4d;
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 7e949f25c933..40d999950e5b 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -62,7 +62,7 @@
* position vmemmap directly below the VMALLOC region.
*/
#ifdef CONFIG_64BIT
-#define VA_BITS (pgtable_l4_enabled ? 48 : 39)
+#define VA_BITS (system_supports_sv48() ? 48 : 39)
#else
#define VA_BITS 32
#endif
@@ -735,7 +735,6 @@ extern uintptr_t _dtb_early_pa;
#define dtb_early_pa _dtb_early_pa
#endif /* CONFIG_XIP_KERNEL */
extern u64 satp_mode;
-extern bool pgtable_l4_enabled;
void paging_init(void);
void misc_mem_init(void);
diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
index ad0a7e9f828b..ce38319232ec 100644
--- a/arch/riscv/kernel/cpu.c
+++ b/arch/riscv/kernel/cpu.c
@@ -79,7 +79,7 @@ static void print_mmu(struct seq_file *f)
#if defined(CONFIG_32BIT)
strncpy(sv_type, "sv32", 5);
#elif defined(CONFIG_64BIT)
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
strncpy(sv_type, "sv48", 5);
else
strncpy(sv_type, "sv39", 5);
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index 35586688a1b6..8a84606f99f0 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -44,9 +44,6 @@ u64 satp_mode __ro_after_init = SATP_MODE_32;
#endif
EXPORT_SYMBOL(satp_mode);
-bool pgtable_l4_enabled = IS_ENABLED(CONFIG_64BIT) && !IS_ENABLED(CONFIG_XIP_KERNEL);
-EXPORT_SYMBOL(pgtable_l4_enabled);
-
phys_addr_t phys_ram_base __ro_after_init;
EXPORT_SYMBOL(phys_ram_base);
@@ -459,19 +456,19 @@ static void __init create_pud_mapping(pud_t *pudp,
}
#define pgd_next_t pud_t
-#define alloc_pgd_next(__va) (pgtable_l4_enabled ? \
+#define alloc_pgd_next(__va) (system_supports_sv48() ? \
pt_ops.alloc_pud(__va) : pt_ops.alloc_pmd(__va))
-#define get_pgd_next_virt(__pa) (pgtable_l4_enabled ? \
+#define get_pgd_next_virt(__pa) (system_supports_sv48() ? \
pt_ops.get_pud_virt(__pa) : (pgd_next_t *)pt_ops.get_pmd_virt(__pa))
#define create_pgd_next_mapping(__nextp, __va, __pa, __sz, __prot) \
- (pgtable_l4_enabled ? \
+ (system_supports_sv48() ? \
create_pud_mapping(__nextp, __va, __pa, __sz, __prot) : \
create_pmd_mapping((pmd_t *)__nextp, __va, __pa, __sz, __prot))
-#define fixmap_pgd_next (pgtable_l4_enabled ? \
+#define fixmap_pgd_next (system_supports_sv48() ? \
(uintptr_t)fixmap_pud : (uintptr_t)fixmap_pmd)
-#define trampoline_pgd_next (pgtable_l4_enabled ? \
+#define trampoline_pgd_next (system_supports_sv48() ? \
(uintptr_t)trampoline_pud : (uintptr_t)trampoline_pmd)
-#define early_dtb_pgd_next (pgtable_l4_enabled ? \
+#define early_dtb_pgd_next (system_supports_sv48() ? \
(uintptr_t)early_dtb_pud : (uintptr_t)early_dtb_pmd)
#else
#define pgd_next_t pte_t
@@ -575,7 +572,7 @@ static __init pgprot_t pgprot_from_va(uintptr_t va)
#ifdef CONFIG_64BIT
static void __init disable_pgtable_l4(void)
{
- pgtable_l4_enabled = false;
+ cpus_set_cap(RISCV_HAS_NO_SV48);
kernel_map.page_offset = PAGE_OFFSET_L3;
satp_mode = SATP_MODE_39;
}
@@ -691,7 +688,7 @@ static void __init create_fdt_early_page_table(pgd_t *pgdir, uintptr_t dtb_pa)
PGDIR_SIZE,
IS_ENABLED(CONFIG_64BIT) ? PAGE_TABLE : PAGE_KERNEL);
- if (pgtable_l4_enabled) {
+ if (system_supports_sv48()) {
create_pud_mapping(early_dtb_pud, DTB_EARLY_BASE_VA,
(uintptr_t)early_dtb_pmd, PUD_SIZE, PAGE_TABLE);
}
@@ -819,7 +816,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa)
#ifndef __PAGETABLE_PMD_FOLDED
/* Setup fixmap PUD and PMD */
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
create_pud_mapping(fixmap_pud, FIXADDR_START,
(uintptr_t)fixmap_pmd, PUD_SIZE, PAGE_TABLE);
create_pmd_mapping(fixmap_pmd, FIXADDR_START,
@@ -827,7 +824,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa)
/* Setup trampoline PGD and PMD */
create_pgd_mapping(trampoline_pg_dir, kernel_map.virt_addr,
trampoline_pgd_next, PGDIR_SIZE, PAGE_TABLE);
- if (pgtable_l4_enabled)
+ if (system_supports_sv48())
create_pud_mapping(trampoline_pud, kernel_map.virt_addr,
(uintptr_t)trampoline_pmd, PUD_SIZE, PAGE_TABLE);
#ifdef CONFIG_XIP_KERNEL
diff --git a/arch/riscv/mm/kasan_init.c b/arch/riscv/mm/kasan_init.c
index f61f7ca6fe0f..3d456c5b55c8 100644
--- a/arch/riscv/mm/kasan_init.c
+++ b/arch/riscv/mm/kasan_init.c
@@ -149,11 +149,11 @@ static void __init kasan_populate_pud(pgd_t *pgd,
set_pgd(pgd, pfn_pgd(PFN_DOWN(__pa(base_pud)), PAGE_TABLE));
}
-#define kasan_early_shadow_pgd_next (pgtable_l4_enabled ? \
+#define kasan_early_shadow_pgd_next (system_supports_sv48() ? \
(uintptr_t)kasan_early_shadow_pud : \
(uintptr_t)kasan_early_shadow_pmd)
#define kasan_populate_pgd_next(pgdp, vaddr, next, early) \
- (pgtable_l4_enabled ? \
+ (system_supports_sv48() ? \
kasan_populate_pud(pgdp, vaddr, next, early) : \
kasan_populate_pmd((pud_t *)pgdp, vaddr, next))
@@ -211,7 +211,7 @@ asmlinkage void __init kasan_early_init(void)
(__pa((uintptr_t)kasan_early_shadow_pte)),
PAGE_TABLE));
- if (pgtable_l4_enabled) {
+ if (system_supports_sv48()) {
for (i = 0; i < PTRS_PER_PUD; ++i)
set_pud(kasan_early_shadow_pud + i,
pfn_pud(PFN_DOWN
diff --git a/arch/riscv/tools/cpucaps b/arch/riscv/tools/cpucaps
index cb1ff2747859..1aea959f225d 100644
--- a/arch/riscv/tools/cpucaps
+++ b/arch/riscv/tools/cpucaps
@@ -3,3 +3,4 @@
# Internal CPU capabilities constants, keep this list sorted
HAS_NO_FPU
+HAS_NO_SV48
--
2.34.1
Hi Jisheng,
I love your patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v5.17-rc1 next-20220125]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Jisheng-Zhang/unified-way-to-use-static-key-and-optimize-pgtable_l4_enabled/20220126-010230
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git a08b41ab9e2e468647f78eb17c28e29b93006394
config: riscv-randconfig-r012-20220124 (https://download.01.org/0day-ci/archive/20220126/[email protected]/config)
compiler: clang version 14.0.0 (https://github.com/llvm/llvm-project 997e128e2a78f5a5434fc75997441ae1ee76f8a4)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install riscv cross compiling tool for clang build
# apt-get install binutils-riscv64-linux-gnu
# https://github.com/0day-ci/linux/commit/6822380e5bcac6d3edfa5d0723d829db8ec28405
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Jisheng-Zhang/unified-way-to-use-static-key-and-optimize-pgtable_l4_enabled/20220126-010230
git checkout 6822380e5bcac6d3edfa5d0723d829db8ec28405
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=riscv SHELL=/bin/bash arch/riscv/mm/
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <[email protected]>
All errors (new ones prefixed by >>):
>> arch/riscv/mm/init.c:691:6: error: implicit declaration of function 'system_supports_sv48' [-Werror,-Wimplicit-function-declaration]
if (system_supports_sv48()) {
^
arch/riscv/mm/init.c:721:13: warning: no previous prototype for function 'pt_ops_set_early' [-Wmissing-prototypes]
void __init pt_ops_set_early(void)
^
arch/riscv/mm/init.c:721:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
void __init pt_ops_set_early(void)
^
static
arch/riscv/mm/init.c:741:13: warning: no previous prototype for function 'pt_ops_set_fixmap' [-Wmissing-prototypes]
void __init pt_ops_set_fixmap(void)
^
arch/riscv/mm/init.c:741:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
void __init pt_ops_set_fixmap(void)
^
static
arch/riscv/mm/init.c:757:13: warning: no previous prototype for function 'pt_ops_set_late' [-Wmissing-prototypes]
void __init pt_ops_set_late(void)
^
arch/riscv/mm/init.c:757:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
void __init pt_ops_set_late(void)
^
static
3 warnings and 1 error generated.
vim +/system_supports_sv48 +691 arch/riscv/mm/init.c
675
676 /*
677 * Setup a 4MB mapping that encompasses the device tree: for 64-bit kernel,
678 * this means 2 PMD entries whereas for 32-bit kernel, this is only 1 PGDIR
679 * entry.
680 */
681 static void __init create_fdt_early_page_table(pgd_t *pgdir, uintptr_t dtb_pa)
682 {
683 #ifndef CONFIG_BUILTIN_DTB
684 uintptr_t pa = dtb_pa & ~(PMD_SIZE - 1);
685
686 create_pgd_mapping(early_pg_dir, DTB_EARLY_BASE_VA,
687 IS_ENABLED(CONFIG_64BIT) ? early_dtb_pgd_next : pa,
688 PGDIR_SIZE,
689 IS_ENABLED(CONFIG_64BIT) ? PAGE_TABLE : PAGE_KERNEL);
690
> 691 if (system_supports_sv48()) {
692 create_pud_mapping(early_dtb_pud, DTB_EARLY_BASE_VA,
693 (uintptr_t)early_dtb_pmd, PUD_SIZE, PAGE_TABLE);
694 }
695
696 if (IS_ENABLED(CONFIG_64BIT)) {
697 create_pmd_mapping(early_dtb_pmd, DTB_EARLY_BASE_VA,
698 pa, PMD_SIZE, PAGE_KERNEL);
699 create_pmd_mapping(early_dtb_pmd, DTB_EARLY_BASE_VA + PMD_SIZE,
700 pa + PMD_SIZE, PMD_SIZE, PAGE_KERNEL);
701 }
702
703 dtb_early_va = (void *)DTB_EARLY_BASE_VA + (dtb_pa & (PMD_SIZE - 1));
704 #else
705 /*
706 * For 64-bit kernel, __va can't be used since it would return a linear
707 * mapping address whereas dtb_early_va will be used before
708 * setup_vm_final installs the linear mapping. For 32-bit kernel, as the
709 * kernel is mapped in the linear mapping, that makes no difference.
710 */
711 dtb_early_va = kernel_mapping_pa_to_va(XIP_FIXUP(dtb_pa));
712 #endif
713
714 dtb_early_pa = dtb_pa;
715 }
716
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/[email protected]
Hi Jisheng,
I love your patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v5.17-rc1 next-20220125]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Jisheng-Zhang/unified-way-to-use-static-key-and-optimize-pgtable_l4_enabled/20220126-010230
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git a08b41ab9e2e468647f78eb17c28e29b93006394
config: riscv-randconfig-p001-20220124 (https://download.01.org/0day-ci/archive/20220126/[email protected]/config)
compiler: riscv32-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/6822380e5bcac6d3edfa5d0723d829db8ec28405
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Jisheng-Zhang/unified-way-to-use-static-key-and-optimize-pgtable_l4_enabled/20220126-010230
git checkout 6822380e5bcac6d3edfa5d0723d829db8ec28405
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=riscv SHELL=/bin/bash
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <[email protected]>
All errors (new ones prefixed by >>):
arch/riscv/mm/init.c: In function 'create_fdt_early_page_table':
>> arch/riscv/mm/init.c:691:13: error: implicit declaration of function 'system_supports_sv48' [-Werror=implicit-function-declaration]
691 | if (system_supports_sv48()) {
| ^~~~~~~~~~~~~~~~~~~~
arch/riscv/mm/init.c: At top level:
arch/riscv/mm/init.c:721:13: warning: no previous prototype for 'pt_ops_set_early' [-Wmissing-prototypes]
721 | void __init pt_ops_set_early(void)
| ^~~~~~~~~~~~~~~~
arch/riscv/mm/init.c:741:13: warning: no previous prototype for 'pt_ops_set_fixmap' [-Wmissing-prototypes]
741 | void __init pt_ops_set_fixmap(void)
| ^~~~~~~~~~~~~~~~~
arch/riscv/mm/init.c:757:13: warning: no previous prototype for 'pt_ops_set_late' [-Wmissing-prototypes]
757 | void __init pt_ops_set_late(void)
| ^~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
vim +/system_supports_sv48 +691 arch/riscv/mm/init.c
675
676 /*
677 * Setup a 4MB mapping that encompasses the device tree: for 64-bit kernel,
678 * this means 2 PMD entries whereas for 32-bit kernel, this is only 1 PGDIR
679 * entry.
680 */
681 static void __init create_fdt_early_page_table(pgd_t *pgdir, uintptr_t dtb_pa)
682 {
683 #ifndef CONFIG_BUILTIN_DTB
684 uintptr_t pa = dtb_pa & ~(PMD_SIZE - 1);
685
686 create_pgd_mapping(early_pg_dir, DTB_EARLY_BASE_VA,
687 IS_ENABLED(CONFIG_64BIT) ? early_dtb_pgd_next : pa,
688 PGDIR_SIZE,
689 IS_ENABLED(CONFIG_64BIT) ? PAGE_TABLE : PAGE_KERNEL);
690
> 691 if (system_supports_sv48()) {
692 create_pud_mapping(early_dtb_pud, DTB_EARLY_BASE_VA,
693 (uintptr_t)early_dtb_pmd, PUD_SIZE, PAGE_TABLE);
694 }
695
696 if (IS_ENABLED(CONFIG_64BIT)) {
697 create_pmd_mapping(early_dtb_pmd, DTB_EARLY_BASE_VA,
698 pa, PMD_SIZE, PAGE_KERNEL);
699 create_pmd_mapping(early_dtb_pmd, DTB_EARLY_BASE_VA + PMD_SIZE,
700 pa + PMD_SIZE, PMD_SIZE, PAGE_KERNEL);
701 }
702
703 dtb_early_va = (void *)DTB_EARLY_BASE_VA + (dtb_pa & (PMD_SIZE - 1));
704 #else
705 /*
706 * For 64-bit kernel, __va can't be used since it would return a linear
707 * mapping address whereas dtb_early_va will be used before
708 * setup_vm_final installs the linear mapping. For 32-bit kernel, as the
709 * kernel is mapped in the linear mapping, that makes no difference.
710 */
711 dtb_early_va = kernel_mapping_pa_to_va(XIP_FIXUP(dtb_pa));
712 #endif
713
714 dtb_early_pa = dtb_pa;
715 }
716
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/[email protected]
On Tue, 25 Jan 2022 08:50:33 PST (-0800), [email protected] wrote:
> Currently, riscv has several features why may not be supported on all
> riscv platforms, for example, FPU, SV48 and so on. To support unified
> kernel Image style, we need to check whether the feature is suportted
> or not. If the check sits at hot code path, then performance will be
> impacted a lot. static key can be used to solve the issue. In the
> past FPU support has been converted to use static key mechanism. I
> believe we will have similar cases in the future. For example, the
> SV48 support can take advantage of static key[1].
>
> patch1 introduces an unified mechanism to use static key for riscv cpu
> features.
> patch2 converts has_cpu() to use the mechanism.
> patch3 uses the mechanism to optimize pgtable_l4_enabled.
>
> [1] http://lists.infradead.org/pipermail/linux-riscv/2021-December/011164.html
>
> Jisheng Zhang (3):
> riscv: introduce unified static key mechanism for CPU features
> riscv: replace has_fpu() with system_supports_fpu()
> riscv: convert pgtable_l4_enabled to static key
I see some build failures from LKP, but I don't see a v2. LMK if I
missed it.
>
> arch/riscv/Makefile | 3 +
> arch/riscv/include/asm/cpufeature.h | 105 ++++++++++++++++++++++++++++
> arch/riscv/include/asm/pgalloc.h | 8 +--
> arch/riscv/include/asm/pgtable-64.h | 21 +++---
> arch/riscv/include/asm/pgtable.h | 3 +-
> arch/riscv/include/asm/switch_to.h | 9 +--
> arch/riscv/kernel/cpu.c | 2 +-
> arch/riscv/kernel/cpufeature.c | 29 ++++++--
> arch/riscv/kernel/process.c | 2 +-
> arch/riscv/kernel/signal.c | 4 +-
> arch/riscv/mm/init.c | 23 +++---
> arch/riscv/mm/kasan_init.c | 6 +-
> arch/riscv/tools/Makefile | 22 ++++++
> arch/riscv/tools/cpucaps | 6 ++
> arch/riscv/tools/gen-cpucaps.awk | 40 +++++++++++
> 15 files changed, 234 insertions(+), 49 deletions(-)
> create mode 100644 arch/riscv/include/asm/cpufeature.h
> create mode 100644 arch/riscv/tools/Makefile
> create mode 100644 arch/riscv/tools/cpucaps
> create mode 100755 arch/riscv/tools/gen-cpucaps.awk
On Mon, Feb 14, 2022 at 03:52:44PM -0800, Palmer Dabbelt wrote:
> On Tue, 25 Jan 2022 08:50:33 PST (-0800), [email protected] wrote:
> > Currently, riscv has several features why may not be supported on all
> > riscv platforms, for example, FPU, SV48 and so on. To support unified
> > kernel Image style, we need to check whether the feature is suportted
> > or not. If the check sits at hot code path, then performance will be
> > impacted a lot. static key can be used to solve the issue. In the
> > past FPU support has been converted to use static key mechanism. I
> > believe we will have similar cases in the future. For example, the
> > SV48 support can take advantage of static key[1].
> >
> > patch1 introduces an unified mechanism to use static key for riscv cpu
> > features.
> > patch2 converts has_cpu() to use the mechanism.
> > patch3 uses the mechanism to optimize pgtable_l4_enabled.
> >
> > [1] http://lists.infradead.org/pipermail/linux-riscv/2021-December/011164.html
> >
> > Jisheng Zhang (3):
> > riscv: introduce unified static key mechanism for CPU features
> > riscv: replace has_fpu() with system_supports_fpu()
> > riscv: convert pgtable_l4_enabled to static key
>
> I see some build failures from LKP, but I don't see a v2. LMK if I missed
> it.
Hi Palmer,
I also saw the build failure due to RV32, fixing it is easy but I have
some thoughts/questions here after reading Atish's "fraemework for RISC-V
ISA extensions" series.
Hi All,
I'm considering how to support cpu features or new ISA extensions.
IMHO, we will need some static keys for ISA extenstions as well as
the cpu features. for example:
if (static_branch_likely(&has_isa_ext_foo))
do_something;
So I have a big question here about CPU features VS. ISA extensions:
1. Is CPU feature == ISA extension?
If yes, then it seems we'd better rebase this series on the ISA
extension series. If no, which is the super set? If CPU feature
is the super set, then if one kind of future cpu feature is implemented
via. the ISA extension, do we need to combine the cpu feature bitmap
with the ISA extension bitmap? how?
2. Is SV48 considered as riscv ISA extension?
I will also comment in Atish's series.
Thanks
>
> >
> > arch/riscv/Makefile | 3 +
> > arch/riscv/include/asm/cpufeature.h | 105 ++++++++++++++++++++++++++++
> > arch/riscv/include/asm/pgalloc.h | 8 +--
> > arch/riscv/include/asm/pgtable-64.h | 21 +++---
> > arch/riscv/include/asm/pgtable.h | 3 +-
> > arch/riscv/include/asm/switch_to.h | 9 +--
> > arch/riscv/kernel/cpu.c | 2 +-
> > arch/riscv/kernel/cpufeature.c | 29 ++++++--
> > arch/riscv/kernel/process.c | 2 +-
> > arch/riscv/kernel/signal.c | 4 +-
> > arch/riscv/mm/init.c | 23 +++---
> > arch/riscv/mm/kasan_init.c | 6 +-
> > arch/riscv/tools/Makefile | 22 ++++++
> > arch/riscv/tools/cpucaps | 6 ++
> > arch/riscv/tools/gen-cpucaps.awk | 40 +++++++++++
> > 15 files changed, 234 insertions(+), 49 deletions(-)
> > create mode 100644 arch/riscv/include/asm/cpufeature.h
> > create mode 100644 arch/riscv/tools/Makefile
> > create mode 100644 arch/riscv/tools/cpucaps
> > create mode 100755 arch/riscv/tools/gen-cpucaps.awk