These patches introduce support for the Activity Monitors Unit (AMU)
CPU extension, an optional extension in ARMv8.4 CPUs. This provides
performance counters intended for system management use. Two of these
counters are then used to compute the frequency scale correction
factor needed to achieve frequency invariance.
With the CONFIG_ARM64_AMU_EXTN and amu kernel parameter enabled the
kernel is able to safely run a mix of CPUs with and without support
for the AMU extension. The AMU capability is unconditionally enabled
in the kernel as to allow any late CPU to use the feature: the
cpu_enable function will be called for all CPUs that match the
criteria, including secondary and hotplugged CPUs, marking this
feature as present on that respective CPU.
To be noted that firmware must implement AMU support when running on
CPUs that present the activity monitors extension: allow access to
the registers from lower exception levels, enable the counters,
implement save and restore functionality. More details can be found
in the documentation.
Given that the activity counters inform on activity on the CPUs, and
that not all CPUs might implement the extension, for functional and
security reasons, it's best to disable access to the AMU registers
from userspace (EL0) and KVM guests.
In the last patch of the series, two of the AMU counters are used to
compute the frequency scale factor needed to achieve frequency
invariance of signals in the scheduler, based on an interface added
to support counter-based frequency invariance - arch_scale_freq_tick.
The interface and update point for the counter-based frequency scale
factor is based on the similar approach in the patch that introduces
frequency invariance for x86 [1].
The current series is based on tip/sched/core.
Testing:
- Build tested for multiple architectures and defconfigs.
- AMU feature detection, EL0 and KVM guest access to AMU registers,
feature support in firmware (version 1.5 and later of the ARM
Trusted Firmware) was tested on an Armv8-A Base Platform FVP:
Architecture Envelope Model [2] (supports version 8.0 to 8.5),
with the following configurations:
cluster0.has_arm_v8-4=1
cluster1.has_arm_v8-4=1
cluster0.has_amu=1
cluster1.has_amu=1
v4 -> v5:
- v4 can be found at [6]
- [1/7] rebased on top of latest tip/sched/core and fixed conflicts;
applied Reviewed-by from Valentin;
- [5/7] applied Reviewed-by from Valentin
- [6/7] applied Valentin and Pavan's suggested fixes; reworked
arch_cpu_freq_counters (now arch_freq_counters_available) and
confined it to the arch topology driver as recommended by Valentin
and Lukasz;
v3 -> v4:
- v3 can be found at [5]
- [1/7] renamed and changed format for disable_amu - now amu=<val> as
Suzuki and Vladimir recommended; removed dynamic allocation for
amu_cpus as Suzuki recommended;
- [2-4/7] collected Reviewed-by
- [5/7] modified changelog and collected Acked-by
- [6/7] removed cpu_get_max_freq as Lukasz recommended; improved debug
messages, warnings, and comments, added use of static key, replaced
validation for possible cpus with filterning of present CPUs - as per
Valentin's comments.
- [7/7] modified changelog to avoid confusion related to impact on
activity monitors use and collected Acked-by
v2 -> v3:
- v2 can be found at [4]
- [1/7] used cpumask instead of per-cpu variable to flag AMU presence
as; introduced disable_amu kernel parameter; removed ftr_id_pfr0 AMU
bits - recommended by Suzuki.
- [2/7] replaced obscure label as recommended by Valentin.
- [3/7] clarified activate_traps_vhe comment
- [4/7] dropped changes in arm64/cpu-feature-registers.txt; removed
use of variable names - recommended by Suzuki
- previous [5/6] - dropped as [1] as added to tip/sched/core
- [5/7] new patch introduced to cleanly obtain maximum hardware
frequency from cpufreq
- [6/7] (previously [6/6]):
- Removed use of workqueues by limiting the validation work done on
each cpu to the setting of the reference per-cpu counter variables.
This is now called directly from cpu_enable (cpufeature.c). Also,
further CPU, policy and system validation is done in a
late_initcall_sync function - waits for deferred probe work to
finish as well to ensure the maximum frequency is set by either
cpufreq drivers or platform drivers - recommended by Lukasz.
- Improved AMU use detection for CPUs in arch_set_freq_scale -
recommended by Lukasz.
- Properly validated arch_max_freq_scale and added detailed
documentation for how arch_max_freq_scale and freq_scale are
obtained based on counters - recommended by Valentin.
- Overall - limited tight coupling between AMU use and cpufreq
(use of maximum frequency information and policy validation).
- [7/7] introduced patch to warn if arch_timer_rate is too low
- functionality provided by Valentin.
v1 -> v2:
- v1 can be found at [3]
- Added patches that use the counters for the scheduler's frequency
invariance engine
- In patch arm64: add support for the AMU extension v1 -
- Defined an accessor function cpu_has_amu_feat to allow a read
of amu_feat only from the current CPU, to ensure the safe use
of the per-cpu variable for the current user (arm64 topology
driver) and future users.
- Modified type of amu_feat from bool to u8 to satisfy sparse
checker's warning 'expression using sizeof _Bool [sparse]',
as the size of bool is compiler dependent.
[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://developer.arm.com/tools-and-software/simulation-models/fixed-virtual-platforms
[3] https://lore.kernel.org/lkml/[email protected]/
[4] https://lore.kernel.org/lkml/[email protected]/
[5] https://lore.kernel.org/lkml/[email protected]/
[6] https://lore.kernel.org/lkml/[email protected]/
Ionela Voinescu (7):
arm64: add support for the AMU extension v1
arm64: trap to EL1 accesses to AMU counters from EL0
arm64/kvm: disable access to AMU registers from kvm guests
Documentation: arm64: document support for the AMU extension
cpufreq: add function to get the hardware max frequency
arm64: use activity monitors for frequency invariance
clocksource/drivers/arm_arch_timer: validate arch_timer_rate
.../admin-guide/kernel-parameters.txt | 9 +
Documentation/arm64/amu.rst | 114 +++++++++++
Documentation/arm64/booting.rst | 14 ++
Documentation/arm64/index.rst | 1 +
arch/arm64/Kconfig | 30 +++
arch/arm64/include/asm/assembler.h | 10 +
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/include/asm/cpufeature.h | 5 +
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/sysreg.h | 38 ++++
arch/arm64/include/asm/topology.h | 9 +
arch/arm64/kernel/cpufeature.c | 95 +++++++++
arch/arm64/kernel/topology.c | 180 ++++++++++++++++++
arch/arm64/kvm/hyp/switch.c | 14 +-
arch/arm64/kvm/sys_regs.c | 93 ++++++++-
arch/arm64/mm/proc.S | 3 +
drivers/base/arch_topology.c | 12 ++
drivers/clocksource/arm_arch_timer.c | 18 +-
drivers/cpufreq/cpufreq.c | 20 ++
include/linux/arch_topology.h | 2 +
include/linux/cpufreq.h | 5 +
21 files changed, 669 insertions(+), 7 deletions(-)
create mode 100644 Documentation/arm64/amu.rst
base-commit: a0f03b617c3b2644d3d47bf7d9e60aed01bd5b10
--
2.17.1
The activity monitors extension is an optional extension introduced
by the ARMv8.4 CPU architecture. This implements basic support for
version 1 of the activity monitors architecture, AMUv1.
This support includes:
- Extension detection on each CPU (boot, secondary, hotplugged)
- Register interface for AMU aarch64 registers
- amu=[0/1/on/off/y/n] kernel parameter to disable detection and
counter access at runtime
Signed-off-by: Ionela Voinescu <[email protected]>
Reviewed-by: Valentin Schneider <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: Suzuki K Poulose <[email protected]>
Cc: Marc Zyngier <[email protected]>
Cc: Mark Rutland <[email protected]>
---
.../admin-guide/kernel-parameters.txt | 9 ++
arch/arm64/Kconfig | 30 ++++++
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/include/asm/cpufeature.h | 5 +
arch/arm64/include/asm/sysreg.h | 38 ++++++++
arch/arm64/kernel/cpufeature.c | 91 +++++++++++++++++++
6 files changed, 175 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index dbc22d684627..49f0c436928f 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -318,6 +318,15 @@
Format: <a>,<b>
See also Documentation/input/joydev/joystick.rst
+ amu= [ARM64]
+ Enables or disables detection, enablement and access to
+ counter registers of the Activity Monitors Unit (AMU).
+ Format: amu=[0/1/on/off/y/n]
+ amu=[0/off/n] ensures access to AMU's counter registers
+ is not attempted.
+ amu=[1/on/y] (default) enables detection and access to
+ AMU's counter registers.
+
analog.map= [HW,JOY] Analog joystick and gamepad support
Specifies type or capabilities of an analog joystick
connected to one of 16 gameports
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 0b30e884e088..3404ca7004c3 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1517,6 +1517,36 @@ config ARM64_PTR_AUTH
endmenu
+menu "ARMv8.4 architectural features"
+
+config ARM64_AMU_EXTN
+ bool "Enable support for the Activity Monitors Unit CPU extension"
+ default y
+ help
+ The activity monitors extension is an optional extension introduced
+ by the ARMv8.4 CPU architecture. This enables support for version 1
+ of the activity monitors architecture, AMUv1.
+
+ To enable the use of this extension on CPUs that implement it, say Y.
+
+ Note that for architectural reasons, firmware _must_ implement AMU
+ support when running on CPUs that present the activity monitors
+ extension. The required support is present in:
+ * Version 1.5 and later of the ARM Trusted Firmware
+
+ For kernels that have this configuration enabled but boot with broken
+ firmware, you may need to say N here until the firmware is fixed.
+ Otherwise you may experience firmware panics or lockups when
+ accessing the counter registers. Even if you are not observing these
+ symptoms, the values returned by the register reads might not
+ correctly reflect reality. Most commonly, the value read will be 0,
+ indicating that the counter is not enabled.
+
+ Alternatively, a kernel parameter is provided to disable detection,
+ enablement and access to the counter registers of the Activity
+ Monitors Unit at runtime.
+endmenu
+
menu "ARMv8.5 architectural features"
config ARM64_E0PD
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index 865e0253fc1e..185e44aa2713 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -58,7 +58,8 @@
#define ARM64_WORKAROUND_SPECULATIVE_AT_NVHE 48
#define ARM64_HAS_E0PD 49
#define ARM64_HAS_RNG 50
+#define ARM64_HAS_AMU_EXTN 51
-#define ARM64_NCAPS 51
+#define ARM64_NCAPS 52
#endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 92ef9539874a..485e069d8768 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -678,6 +678,11 @@ static inline bool cpu_has_hw_af(void)
ID_AA64MMFR1_HADBS_SHIFT);
}
+#ifdef CONFIG_ARM64_AMU_EXTN
+/* Check whether the cpu supports the Activity Monitors Unit (AMU) */
+extern bool cpu_has_amu_feat(int cpu);
+#endif
+
#endif /* __ASSEMBLY__ */
#endif
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index b91570ff9db1..a6ad64e324a4 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -386,6 +386,42 @@
#define SYS_TPIDR_EL0 sys_reg(3, 3, 13, 0, 2)
#define SYS_TPIDRRO_EL0 sys_reg(3, 3, 13, 0, 3)
+/* Definitions for system register interface to AMU for ARMv8.4 onwards */
+#define SYS_AM_EL0(crm, op2) sys_reg(3, 3, 13, (crm), (op2))
+#define SYS_AMCR_EL0 SYS_AM_EL0(2, 0)
+#define SYS_AMCFGR_EL0 SYS_AM_EL0(2, 1)
+#define SYS_AMCGCR_EL0 SYS_AM_EL0(2, 2)
+#define SYS_AMUSERENR_EL0 SYS_AM_EL0(2, 3)
+#define SYS_AMCNTENCLR0_EL0 SYS_AM_EL0(2, 4)
+#define SYS_AMCNTENSET0_EL0 SYS_AM_EL0(2, 5)
+#define SYS_AMCNTENCLR1_EL0 SYS_AM_EL0(3, 0)
+#define SYS_AMCNTENSET1_EL0 SYS_AM_EL0(3, 1)
+
+/*
+ * Group 0 of activity monitors (architected):
+ * op0 op1 CRn CRm op2
+ * Counter: 11 011 1101 010:n<3> n<2:0>
+ * Type: 11 011 1101 011:n<3> n<2:0>
+ * n: 0-15
+ *
+ * Group 1 of activity monitors (auxiliary):
+ * op0 op1 CRn CRm op2
+ * Counter: 11 011 1101 110:n<3> n<2:0>
+ * Type: 11 011 1101 111:n<3> n<2:0>
+ * n: 0-15
+ */
+
+#define SYS_AMEVCNTR0_EL0(n) SYS_AM_EL0(4 + ((n) >> 3), (n) & 7)
+#define SYS_AMEVTYPE0_EL0(n) SYS_AM_EL0(6 + ((n) >> 3), (n) & 7)
+#define SYS_AMEVCNTR1_EL0(n) SYS_AM_EL0(12 + ((n) >> 3), (n) & 7)
+#define SYS_AMEVTYPE1_EL0(n) SYS_AM_EL0(14 + ((n) >> 3), (n) & 7)
+
+/* AMU v1: Fixed (architecturally defined) activity monitors */
+#define SYS_AMEVCNTR0_CORE_EL0 SYS_AMEVCNTR0_EL0(0)
+#define SYS_AMEVCNTR0_CONST_EL0 SYS_AMEVCNTR0_EL0(1)
+#define SYS_AMEVCNTR0_INST_RET_EL0 SYS_AMEVCNTR0_EL0(2)
+#define SYS_AMEVCNTR0_MEM_STALL SYS_AMEVCNTR0_EL0(3)
+
#define SYS_CNTFRQ_EL0 sys_reg(3, 3, 14, 0, 0)
#define SYS_CNTP_TVAL_EL0 sys_reg(3, 3, 14, 2, 0)
@@ -598,6 +634,7 @@
#define ID_AA64PFR0_CSV3_SHIFT 60
#define ID_AA64PFR0_CSV2_SHIFT 56
#define ID_AA64PFR0_DIT_SHIFT 48
+#define ID_AA64PFR0_AMU_SHIFT 44
#define ID_AA64PFR0_SVE_SHIFT 32
#define ID_AA64PFR0_RAS_SHIFT 28
#define ID_AA64PFR0_GIC_SHIFT 24
@@ -608,6 +645,7 @@
#define ID_AA64PFR0_EL1_SHIFT 4
#define ID_AA64PFR0_EL0_SHIFT 0
+#define ID_AA64PFR0_AMU 0x1
#define ID_AA64PFR0_SVE 0x1
#define ID_AA64PFR0_RAS_V1 0x1
#define ID_AA64PFR0_FP_NI 0xf
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 0b6715625cf6..60cebc071603 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -163,6 +163,7 @@ static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV2_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_DIT_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_AMU_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE),
FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SVE_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_RAS_SHIFT, 4, 0),
@@ -1222,6 +1223,78 @@ static bool has_hw_dbm(const struct arm64_cpu_capabilities *cap,
#endif
+#ifdef CONFIG_ARM64_AMU_EXTN
+
+/*
+ * The "amu_cpus" cpumask only signals that the CPU implementation for the
+ * flagged CPUs supports the Activity Monitors Unit (AMU) but does not provide
+ * information regarding all the events that it supports. When a CPU bit is
+ * set in the cpumask, the user of this feature can only rely on the presence
+ * of the 4 fixed counters for that CPU. But this does not guarantee that the
+ * counters are enabled or access to these counters is enabled by code
+ * executed at higher exception levels (firmware).
+ */
+static struct cpumask amu_cpus __read_mostly;
+
+bool cpu_has_amu_feat(int cpu)
+{
+ return cpumask_test_cpu(cpu, &amu_cpus);
+}
+
+static void cpu_amu_enable(struct arm64_cpu_capabilities const *cap)
+{
+ if (has_cpuid_feature(cap, SCOPE_LOCAL_CPU)) {
+ pr_info("detected CPU%d: Activity Monitors Unit (AMU)\n",
+ smp_processor_id());
+ cpumask_set_cpu(smp_processor_id(), &amu_cpus);
+ }
+}
+
+/*
+ * For known broken firmware, a kernel parameter ("amu=[0/off/n]") is provided
+ * to ensure access to AMU counter registers is not attempted. By default, the
+ * feature is enabled, but the kernel parameter can both be used to disable or
+ * enable the capability at runtime in case the default changes in the future.
+ *
+ * To be noted that for security considerations, this does not bypass the
+ * setting of AMUSERENR_EL0 to trap accesses from EL0 (userspace) to EL1
+ * (kernel). Therefore, firmware should still ensure accesses to AMU registers
+ * are not trapped in EL2/EL3.
+ */
+static bool enable_amu = true;
+
+static int __init set_enable_amu(char *str)
+{
+ bool value;
+ int ret = strtobool(str, &value);
+
+ if (!ret)
+ enable_amu = value;
+
+ return ret;
+}
+early_param("amu", set_enable_amu);
+
+static bool has_amu(const struct arm64_cpu_capabilities *cap,
+ int __unused)
+{
+ /*
+ * The AMU extension is a non-conflicting feature: the kernel can
+ * safely run a mix of CPUs with and without support for the
+ * activity monitors extension. Therefore, if not disabled through
+ * the kernel command line early parameter, enable the capability
+ * to allow any late CPU to use the feature.
+ *
+ * With this feature enabled, the cpu_enable function will be called
+ * for all CPUs that match the criteria, including secondary and
+ * hotplugged, marking this feature as present on that respective CPU.
+ * The enable function will also print a detection message.
+ */
+
+ return enable_amu;
+}
+#endif
+
#ifdef CONFIG_ARM64_VHE
static bool runs_at_el2(const struct arm64_cpu_capabilities *entry, int __unused)
{
@@ -1499,6 +1572,24 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.cpu_enable = cpu_clear_disr,
},
#endif /* CONFIG_ARM64_RAS_EXTN */
+#ifdef CONFIG_ARM64_AMU_EXTN
+ {
+ /*
+ * The feature is enabled by default if CONFIG_ARM64_AMU_EXTN=y.
+ * Therefore, don't provide .desc as we don't want the detection
+ * message to be shown until at least one CPU is detected to
+ * support the feature.
+ */
+ .capability = ARM64_HAS_AMU_EXTN,
+ .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
+ .matches = has_amu,
+ .sys_reg = SYS_ID_AA64PFR0_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64PFR0_AMU_SHIFT,
+ .min_field_value = ID_AA64PFR0_AMU,
+ .cpu_enable = cpu_amu_enable,
+ },
+#endif /* CONFIG_ARM64_AMU_EXTN */
{
.desc = "Data cache clean to the PoU not required for I/D coherence",
.capability = ARM64_HAS_CACHE_IDC,
--
2.17.1
The activity monitors extension is an optional extension introduced
by the ARMv8.4 CPU architecture. In order to access the activity
monitors counters safely, if desired, the kernel should detect the
presence of the extension through the feature register, and mediate
the access.
Therefore, disable direct accesses to activity monitors counters
from EL0 (userspace) and trap them to EL1 (kernel).
To be noted that the ARM64_AMU_EXTN kernel config and the disable_amu
kernel parameter do not have an effect on this code. Given that the
amuserenr_el0 resets to an UNKNOWN value, setting the trap of EL0
accesses to EL1 is always attempted for safety and security
considerations. Therefore firmware should still ensure accesses to
AMU registers are not trapped in EL2/EL3 as this code cannot be
bypassed if the CPU implements the Activity Monitors Unit.
Signed-off-by: Ionela Voinescu <[email protected]>
Reviewed-by: Suzuki K Poulose <[email protected]>
Reviewed-by: Valentin Schneider <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: Mark Rutland <[email protected]>
Cc: Steve Capper <[email protected]>
---
arch/arm64/include/asm/assembler.h | 10 ++++++++++
arch/arm64/mm/proc.S | 3 +++
2 files changed, 13 insertions(+)
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index aca337d79d12..c5487806273f 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -430,6 +430,16 @@ USER(\label, ic ivau, \tmp2) // invalidate I line PoU
9000:
.endm
+/*
+ * reset_amuserenr_el0 - reset AMUSERENR_EL0 if AMUv1 present
+ */
+ .macro reset_amuserenr_el0, tmpreg
+ mrs \tmpreg, id_aa64pfr0_el1 // Check ID_AA64PFR0_EL1
+ ubfx \tmpreg, \tmpreg, #ID_AA64PFR0_AMU_SHIFT, #4
+ cbz \tmpreg, .Lskip_\@ // Skip if no AMU present
+ msr_s SYS_AMUSERENR_EL0, xzr // Disable AMU access from EL0
+.Lskip_\@:
+ .endm
/*
* copy_page - copy src to dest using temp registers t1-t8
*/
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index aafed6902411..7103027b4e64 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -131,6 +131,7 @@ alternative_endif
ubfx x11, x11, #1, #1
msr oslar_el1, x11
reset_pmuserenr_el0 x0 // Disable PMU access from EL0
+ reset_amuserenr_el0 x0 // Disable AMU access from EL0
alternative_if ARM64_HAS_RAS_EXTN
msr_s SYS_DISR_EL1, xzr
@@ -423,6 +424,8 @@ SYM_FUNC_START(__cpu_setup)
isb // Unmask debug exceptions now,
enable_dbg // since this is per-cpu
reset_pmuserenr_el0 x0 // Disable PMU access from EL0
+ reset_amuserenr_el0 x0 // Disable AMU access from EL0
+
/*
* Memory region attributes
*/
--
2.17.1
Access to the AMU counters should be disabled by default in kvm guests,
as information from the counters might reveal activity in other guests
or activity on the host.
Therefore, disable access to AMU registers from EL0 and EL1 in kvm
guests by:
- Hiding the presence of the extension in the feature register
(SYS_ID_AA64PFR0_EL1) on the VCPU.
- Disabling access to the AMU registers before switching to the guest.
- Trapping accesses and injecting an undefined instruction into the
guest.
Signed-off-by: Ionela Voinescu <[email protected]>
Reviewed-by: Valentin Schneider <[email protected]>
Reviewed-by: Suzuki K Poulose <[email protected]>
Cc: Marc Zyngier <[email protected]>
Cc: James Morse <[email protected]>
Cc: Julien Thierry <[email protected]>
Cc: Suzuki K Poulose <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Will Deacon <[email protected]>
---
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/kvm/hyp/switch.c | 14 ++++-
arch/arm64/kvm/sys_regs.c | 93 +++++++++++++++++++++++++++++++-
3 files changed, 105 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 6e5d839f42b5..51c1d9918999 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -267,6 +267,7 @@
/* Hyp Coprocessor Trap Register */
#define CPTR_EL2_TCPAC (1 << 31)
+#define CPTR_EL2_TAM (1 << 30)
#define CPTR_EL2_TTA (1 << 20)
#define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
#define CPTR_EL2_TZ (1 << 8)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index dfe8dd172512..46292a370781 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -98,6 +98,18 @@ static void activate_traps_vhe(struct kvm_vcpu *vcpu)
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
val &= ~CPACR_EL1_ZEN;
+
+ /*
+ * With VHE (HCR.E2H == 1), accesses to CPACR_EL1 are routed to
+ * CPTR_EL2. In general, CPACR_EL1 has the same layout as CPTR_EL2,
+ * except for some missing controls, such as TAM.
+ * In this case, CPTR_EL2.TAM has the same position with or without
+ * VHE (HCR.E2H == 1) which allows us to use here the CPTR_EL2.TAM
+ * shift value for trapping the AMU accesses.
+ */
+
+ val |= CPTR_EL2_TAM;
+
if (update_fp_enabled(vcpu)) {
if (vcpu_has_sve(vcpu))
val |= CPACR_EL1_ZEN;
@@ -119,7 +131,7 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
__activate_traps_common(vcpu);
val = CPTR_EL2_DEFAULT;
- val |= CPTR_EL2_TTA | CPTR_EL2_TZ;
+ val |= CPTR_EL2_TTA | CPTR_EL2_TZ | CPTR_EL2_TAM;
if (!update_fp_enabled(vcpu)) {
val |= CPTR_EL2_TFP;
__activate_traps_fpsimd32(vcpu);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 3e909b117f0c..44354c812783 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1003,6 +1003,20 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
{ SYS_DESC(SYS_PMEVTYPERn_EL0(n)), \
access_pmu_evtyper, reset_unknown, (PMEVTYPER0_EL0 + n), }
+static bool access_amu(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ kvm_inject_undefined(vcpu);
+
+ return false;
+}
+
+/* Macro to expand the AMU counter and type registers*/
+#define AMU_AMEVCNTR0_EL0(n) { SYS_DESC(SYS_AMEVCNTR0_EL0(n)), access_amu }
+#define AMU_AMEVTYPE0_EL0(n) { SYS_DESC(SYS_AMEVTYPE0_EL0(n)), access_amu }
+#define AMU_AMEVCNTR1_EL0(n) { SYS_DESC(SYS_AMEVCNTR1_EL0(n)), access_amu }
+#define AMU_AMEVTYPE1_EL0(n) { SYS_DESC(SYS_AMEVTYPE1_EL0(n)), access_amu }
+
static bool trap_ptrauth(struct kvm_vcpu *vcpu,
struct sys_reg_params *p,
const struct sys_reg_desc *rd)
@@ -1078,8 +1092,10 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
(u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
u64 val = raz ? 0 : read_sanitised_ftr_reg(id);
- if (id == SYS_ID_AA64PFR0_EL1 && !vcpu_has_sve(vcpu)) {
- val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
+ if (id == SYS_ID_AA64PFR0_EL1) {
+ if (!vcpu_has_sve(vcpu))
+ val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
+ val &= ~(0xfUL << ID_AA64PFR0_AMU_SHIFT);
} else if (id == SYS_ID_AA64ISAR1_EL1 && !vcpu_has_ptrauth(vcpu)) {
val &= ~((0xfUL << ID_AA64ISAR1_APA_SHIFT) |
(0xfUL << ID_AA64ISAR1_API_SHIFT) |
@@ -1565,6 +1581,79 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_TPIDR_EL0), NULL, reset_unknown, TPIDR_EL0 },
{ SYS_DESC(SYS_TPIDRRO_EL0), NULL, reset_unknown, TPIDRRO_EL0 },
+ { SYS_DESC(SYS_AMCR_EL0), access_amu },
+ { SYS_DESC(SYS_AMCFGR_EL0), access_amu },
+ { SYS_DESC(SYS_AMCGCR_EL0), access_amu },
+ { SYS_DESC(SYS_AMUSERENR_EL0), access_amu },
+ { SYS_DESC(SYS_AMCNTENCLR0_EL0), access_amu },
+ { SYS_DESC(SYS_AMCNTENSET0_EL0), access_amu },
+ { SYS_DESC(SYS_AMCNTENCLR1_EL0), access_amu },
+ { SYS_DESC(SYS_AMCNTENSET1_EL0), access_amu },
+ AMU_AMEVCNTR0_EL0(0),
+ AMU_AMEVCNTR0_EL0(1),
+ AMU_AMEVCNTR0_EL0(2),
+ AMU_AMEVCNTR0_EL0(3),
+ AMU_AMEVCNTR0_EL0(4),
+ AMU_AMEVCNTR0_EL0(5),
+ AMU_AMEVCNTR0_EL0(6),
+ AMU_AMEVCNTR0_EL0(7),
+ AMU_AMEVCNTR0_EL0(8),
+ AMU_AMEVCNTR0_EL0(9),
+ AMU_AMEVCNTR0_EL0(10),
+ AMU_AMEVCNTR0_EL0(11),
+ AMU_AMEVCNTR0_EL0(12),
+ AMU_AMEVCNTR0_EL0(13),
+ AMU_AMEVCNTR0_EL0(14),
+ AMU_AMEVCNTR0_EL0(15),
+ AMU_AMEVTYPE0_EL0(0),
+ AMU_AMEVTYPE0_EL0(1),
+ AMU_AMEVTYPE0_EL0(2),
+ AMU_AMEVTYPE0_EL0(3),
+ AMU_AMEVTYPE0_EL0(4),
+ AMU_AMEVTYPE0_EL0(5),
+ AMU_AMEVTYPE0_EL0(6),
+ AMU_AMEVTYPE0_EL0(7),
+ AMU_AMEVTYPE0_EL0(8),
+ AMU_AMEVTYPE0_EL0(9),
+ AMU_AMEVTYPE0_EL0(10),
+ AMU_AMEVTYPE0_EL0(11),
+ AMU_AMEVTYPE0_EL0(12),
+ AMU_AMEVTYPE0_EL0(13),
+ AMU_AMEVTYPE0_EL0(14),
+ AMU_AMEVTYPE0_EL0(15),
+ AMU_AMEVCNTR1_EL0(0),
+ AMU_AMEVCNTR1_EL0(1),
+ AMU_AMEVCNTR1_EL0(2),
+ AMU_AMEVCNTR1_EL0(3),
+ AMU_AMEVCNTR1_EL0(4),
+ AMU_AMEVCNTR1_EL0(5),
+ AMU_AMEVCNTR1_EL0(6),
+ AMU_AMEVCNTR1_EL0(7),
+ AMU_AMEVCNTR1_EL0(8),
+ AMU_AMEVCNTR1_EL0(9),
+ AMU_AMEVCNTR1_EL0(10),
+ AMU_AMEVCNTR1_EL0(11),
+ AMU_AMEVCNTR1_EL0(12),
+ AMU_AMEVCNTR1_EL0(13),
+ AMU_AMEVCNTR1_EL0(14),
+ AMU_AMEVCNTR1_EL0(15),
+ AMU_AMEVTYPE1_EL0(0),
+ AMU_AMEVTYPE1_EL0(1),
+ AMU_AMEVTYPE1_EL0(2),
+ AMU_AMEVTYPE1_EL0(3),
+ AMU_AMEVTYPE1_EL0(4),
+ AMU_AMEVTYPE1_EL0(5),
+ AMU_AMEVTYPE1_EL0(6),
+ AMU_AMEVTYPE1_EL0(7),
+ AMU_AMEVTYPE1_EL0(8),
+ AMU_AMEVTYPE1_EL0(9),
+ AMU_AMEVTYPE1_EL0(10),
+ AMU_AMEVTYPE1_EL0(11),
+ AMU_AMEVTYPE1_EL0(12),
+ AMU_AMEVTYPE1_EL0(13),
+ AMU_AMEVTYPE1_EL0(14),
+ AMU_AMEVTYPE1_EL0(15),
+
{ SYS_DESC(SYS_CNTP_TVAL_EL0), access_arch_timer },
{ SYS_DESC(SYS_CNTP_CTL_EL0), access_arch_timer },
{ SYS_DESC(SYS_CNTP_CVAL_EL0), access_arch_timer },
--
2.17.1
The activity monitors extension is an optional extension introduced
by the ARMv8.4 CPU architecture.
Add initial documentation for the AMUv1 extension:
- arm64/amu.txt: AMUv1 documentation
- arm64/booting.txt: system registers initialisation
Signed-off-by: Ionela Voinescu <[email protected]>
Reviewed-by: Valentin Schneider <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: Jonathan Corbet <[email protected]>
---
Documentation/arm64/amu.rst | 114 ++++++++++++++++++++++++++++++++
Documentation/arm64/booting.rst | 14 ++++
Documentation/arm64/index.rst | 1 +
3 files changed, 129 insertions(+)
create mode 100644 Documentation/arm64/amu.rst
diff --git a/Documentation/arm64/amu.rst b/Documentation/arm64/amu.rst
new file mode 100644
index 000000000000..ad609ada2d8e
--- /dev/null
+++ b/Documentation/arm64/amu.rst
@@ -0,0 +1,114 @@
+=======================================================
+Activity Monitors Unit (AMU) extension in AArch64 Linux
+=======================================================
+
+Author: Ionela Voinescu <[email protected]>
+
+Date: 2019-09-10
+
+This document briefly describes the provision of Activity Monitors Unit
+support in AArch64 Linux.
+
+
+Architecture overview
+---------------------
+
+The activity monitors extension is an optional extension introduced by the
+ARMv8.4 CPU architecture.
+
+The activity monitors unit, implemented in each CPU, provides performance
+counters intended for system management use. The AMU extension provides a
+system register interface to the counter registers and also supports an
+optional external memory-mapped interface.
+
+Version 1 of the Activity Monitors architecture implements a counter group
+of four fixed and architecturally defined 64-bit event counters.
+ - CPU cycle counter: increments at the frequency of the CPU.
+ - Constant counter: increments at the fixed frequency of the system
+ clock.
+ - Instructions retired: increments with every architecturally executed
+ instruction.
+ - Memory stall cycles: counts instruction dispatch stall cycles caused by
+ misses in the last level cache within the clock domain.
+
+When in WFI or WFE these counters do not increment.
+
+The Activity Monitors architecture provides space for up to 16 architected
+event counters. Future versions of the architecture may use this space to
+implement additional architected event counters.
+
+Additionally, version 1 implements a counter group of up to 16 auxiliary
+64-bit event counters.
+
+On cold reset all counters reset to 0.
+
+
+Basic support
+-------------
+
+The kernel can safely run a mix of CPUs with and without support for the
+activity monitors extension. Therefore, if the capability is not disabled
+at system level (either through CONFIG_ARM64_AMU_EXTN or kernel parameter)
+we unconditionally enable the capability to allow any late CPU (secondary
+or hotplugged) to detect and use the feature.
+
+When the feature is detected on a CPU, we flag the availability of the
+feature but this does not guarantee the correct functionality of the
+counters, only the presence of the extension.
+
+Firmware (code running at higher exception levels, e.g. arm-tf) support is
+needed to:
+ - Enable access for lower exception levels (EL2 and EL1) to the AMU
+ registers.
+ - Enable the counters. If not enabled these will read as 0.
+ - Save/restore the counters before/after the CPU is being put/brought up
+ from the 'off' power state.
+
+When using kernels that have this feature enabled but boot with broken
+firmware the user may experience panics or lockups when accessing the
+counter registers. Even if these symptoms are not observed, the values
+returned by the register reads might not correctly reflect reality. Most
+commonly, the counters will read as 0, indicating that they are not
+enabled.
+
+If proper support is not provided in firmware it's best to disable
+CONFIG_ARM64_AMU_EXTN or disable the capability at runtime through the
+corresponding kernel parameter. To be noted that for security reasons,
+this does not bypass the setting of AMUSERENR_EL0 to trap accesses from
+EL0 (userspace) to EL1 (kernel). Therefore, firmware should still ensure
+accesses to AMU registers are not trapped in EL2/EL3.
+
+The fixed counters of AMUv1 are accessible though the following system
+register definitions:
+ - SYS_AMEVCNTR0_CORE_EL0
+ - SYS_AMEVCNTR0_CONST_EL0
+ - SYS_AMEVCNTR0_INST_RET_EL0
+ - SYS_AMEVCNTR0_MEM_STALL_EL0
+
+Auxiliary platform specific counters can be accessed using
+SYS_AMEVCNTR1_EL0(n), where n is a value between 0 and 15.
+
+Details can be found in: arch/arm64/include/asm/sysreg.h.
+
+
+Userspace access
+----------------
+
+Currently, access from userspace to the AMU registers is disabled due to:
+ - Security reasons: they might expose information about code executed in
+ secure mode.
+ - Purpose: AMU counters are intended for system management use.
+
+Also, the presence of the feature is not visible to userspace.
+
+
+Virtualization
+--------------
+
+Currently, access from userspace (EL0) and kernelspace (EL1) on the KVM
+guest side is disabled due to:
+ - Security reasons: they might expose information about code executed
+ by other guests or the host.
+
+Any attempt to access the AMU registers will result in an UNDEFINED
+exception being injected into the guest.
diff --git a/Documentation/arm64/booting.rst b/Documentation/arm64/booting.rst
index 5d78a6f5b0ae..a3f1a47b6f1c 100644
--- a/Documentation/arm64/booting.rst
+++ b/Documentation/arm64/booting.rst
@@ -248,6 +248,20 @@ Before jumping into the kernel, the following conditions must be met:
- HCR_EL2.APK (bit 40) must be initialised to 0b1
- HCR_EL2.API (bit 41) must be initialised to 0b1
+ For CPUs with Activity Monitors Unit v1 (AMUv1) extension present:
+ - If EL3 is present:
+ CPTR_EL3.TAM (bit 30) must be initialised to 0b0
+ CPTR_EL2.TAM (bit 30) must be initialised to 0b0
+ AMCNTENSET0_EL0 must be initialised to 0b1111
+ AMCNTENSET1_EL0 must be initialised to a platform specific value
+ having 0b1 set for the corresponding bit for each of the auxiliary
+ counters present.
+ - If the kernel is entered at EL1:
+ AMCNTENSET0_EL0 must be initialised to 0b1111
+ AMCNTENSET1_EL0 must be initialised to a platform specific value
+ having 0b1 set for the corresponding bit for each of the auxiliary
+ counters present.
+
The requirements described above for CPU mode, caches, MMUs, architected
timers, coherency and system registers apply to all CPUs. All CPUs must
enter the kernel in the same exception level.
diff --git a/Documentation/arm64/index.rst b/Documentation/arm64/index.rst
index 5c0c69dc58aa..09cbb4ed2237 100644
--- a/Documentation/arm64/index.rst
+++ b/Documentation/arm64/index.rst
@@ -6,6 +6,7 @@ ARM64 Architecture
:maxdepth: 1
acpi_object_usage
+ amu
arm-acpi
booting
cpu-feature-registers
--
2.17.1
The Frequency Invariance Engine (FIE) is providing a frequency
scaling correction factor that helps achieve more accurate
load-tracking.
So far, for arm and arm64 platforms, this scale factor has been
obtained based on the ratio between the current frequency and the
maximum supported frequency recorded by the cpufreq policy. The
setting of this scale factor is triggered from cpufreq drivers by
calling arch_set_freq_scale. The current frequency used in computation
is the frequency requested by a governor, but it may not be the
frequency that was implemented by the platform.
This correction factor can also be obtained using a core counter and a
constant counter to get information on the performance (frequency based
only) obtained in a period of time. This will more accurately reflect
the actual current frequency of the CPU, compared with the alternative
implementation that reflects the request of a performance level from
the OS.
Therefore, implement arch_scale_freq_tick to use activity monitors, if
present, for the computation of the frequency scale factor.
The use of AMU counters depends on:
- CONFIG_ARM64_AMU_EXTN - depents on the AMU extension being present
- CONFIG_CPU_FREQ - the current frequency obtained using counter
information is divided by the maximum frequency obtained from the
cpufreq policy.
While it is possible to have a combination of CPUs in the system with
and without support for activity monitors, the use of counters for
frequency invariance is only enabled for a CPU if all related CPUs
(CPUs in the same frequency domain) support and have enabled the core
and constant activity monitor counters. In this way, there is a clear
separation between the policies for which arch_set_freq_scale (cpufreq
based FIE) is used, and the policies for which arch_scale_freq_tick
(counter based FIE) is used to set the frequency scale factor. For
this purpose, a late_initcall_sync is registered to trigger validation
work for policies that will enable or disable the use of AMU counters
for frequency invariance. If CONFIG_CPU_FREQ is not defined, the use
of counters is enabled on all CPUs only if all possible CPUs correctly
support the necessary counters.
Signed-off-by: Ionela Voinescu <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: Sudeep Holla <[email protected]>
---
arch/arm64/include/asm/topology.h | 9 ++
arch/arm64/kernel/cpufeature.c | 4 +
arch/arm64/kernel/topology.c | 180 ++++++++++++++++++++++++++++++
drivers/base/arch_topology.c | 12 ++
include/linux/arch_topology.h | 2 +
5 files changed, 207 insertions(+)
diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
index a4d945db95a2..21d4d40d6243 100644
--- a/arch/arm64/include/asm/topology.h
+++ b/arch/arm64/include/asm/topology.h
@@ -16,6 +16,15 @@ int pcibus_to_node(struct pci_bus *bus);
#include <linux/arch_topology.h>
+#ifdef CONFIG_ARM64_AMU_EXTN
+/*
+ * Replace task scheduler's default counter-based
+ * frequency-invariance scale factor setting.
+ */
+void topology_scale_freq_tick(void);
+#define arch_scale_freq_tick topology_scale_freq_tick
+#endif /* CONFIG_ARM64_AMU_EXTN */
+
/* Replace task scheduler's default frequency-invariant accounting */
#define arch_scale_freq_capacity topology_get_freq_scale
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 60cebc071603..b8ec6c544d32 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1241,12 +1241,16 @@ bool cpu_has_amu_feat(int cpu)
return cpumask_test_cpu(cpu, &amu_cpus);
}
+/* Initialize the use of AMU counters for frequency invariance */
+extern void init_cpu_freq_invariance_counters(void);
+
static void cpu_amu_enable(struct arm64_cpu_capabilities const *cap)
{
if (has_cpuid_feature(cap, SCOPE_LOCAL_CPU)) {
pr_info("detected CPU%d: Activity Monitors Unit (AMU)\n",
smp_processor_id());
cpumask_set_cpu(smp_processor_id(), &amu_cpus);
+ init_cpu_freq_invariance_counters();
}
}
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index fa9528dfd0ce..0801a0f3c156 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -14,6 +14,7 @@
#include <linux/acpi.h>
#include <linux/arch_topology.h>
#include <linux/cacheinfo.h>
+#include <linux/cpufreq.h>
#include <linux/init.h>
#include <linux/percpu.h>
@@ -120,4 +121,183 @@ int __init parse_acpi_topology(void)
}
#endif
+#ifdef CONFIG_ARM64_AMU_EXTN
+#undef pr_fmt
+#define pr_fmt(fmt) "AMU: " fmt
+
+static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale);
+static DEFINE_PER_CPU(u64, arch_const_cycles_prev);
+static DEFINE_PER_CPU(u64, arch_core_cycles_prev);
+static cpumask_var_t amu_fie_cpus;
+
+/* Initialize counter reference per-cpu variables for the current CPU */
+void init_cpu_freq_invariance_counters(void)
+{
+ this_cpu_write(arch_core_cycles_prev,
+ read_sysreg_s(SYS_AMEVCNTR0_CORE_EL0));
+ this_cpu_write(arch_const_cycles_prev,
+ read_sysreg_s(SYS_AMEVCNTR0_CONST_EL0));
+}
+
+static int validate_cpu_freq_invariance_counters(int cpu)
+{
+ u64 max_freq_hz, ratio;
+
+ if (!cpu_has_amu_feat(cpu)) {
+ pr_debug("CPU%d: counters are not supported.\n", cpu);
+ return -EINVAL;
+ }
+
+ if (unlikely(!per_cpu(arch_const_cycles_prev, cpu) ||
+ !per_cpu(arch_core_cycles_prev, cpu))) {
+ pr_debug("CPU%d: cycle counters are not enabled.\n", cpu);
+ return -EINVAL;
+ }
+
+ /* Convert maximum frequency from KHz to Hz and validate */
+ max_freq_hz = cpufreq_get_hw_max_freq(cpu) * 1000;
+ if (unlikely(!max_freq_hz)) {
+ pr_debug("CPU%d: invalid maximum frequency.\n", cpu);
+ return -EINVAL;
+ }
+
+ /*
+ * Pre-compute the fixed ratio between the frequency of the constant
+ * counter and the maximum frequency of the CPU.
+ *
+ * const_freq
+ * arch_max_freq_scale = ---------------- * SCHED_CAPACITY_SCALE²
+ * cpuinfo_max_freq
+ *
+ * We use a factor of 2 * SCHED_CAPACITY_SHIFT -> SCHED_CAPACITY_SCALE²
+ * in order to ensure a good resolution for arch_max_freq_scale for
+ * very low arch timer frequencies (down to the KHz range which should
+ * be unlikely).
+ */
+ ratio = (u64)arch_timer_get_rate() << (2 * SCHED_CAPACITY_SHIFT);
+ ratio = div64_u64(ratio, max_freq_hz);
+ if (!ratio) {
+ WARN_ONCE(1, "System timer frequency too low.\n");
+ return -EINVAL;
+ }
+
+ per_cpu(arch_max_freq_scale, cpu) = (unsigned long)ratio;
+
+ return 0;
+}
+
+static inline bool
+enable_policy_freq_counters(int cpu, cpumask_var_t valid_cpus)
+{
+ struct cpufreq_policy *policy = cpufreq_cpu_get(cpu);
+
+ if (!policy) {
+ pr_debug("CPU%d: No cpufreq policy found.\n", cpu);
+ return false;
+ }
+
+ if (cpumask_subset(policy->related_cpus, valid_cpus))
+ cpumask_or(amu_fie_cpus, policy->related_cpus,
+ amu_fie_cpus);
+
+ cpufreq_cpu_put(policy);
+
+ return true;
+}
+
+static DEFINE_STATIC_KEY_FALSE(amu_fie_key);
+#define amu_freq_invariant() static_branch_unlikely(&amu_fie_key)
+
+static int __init init_amu_fie(void)
+{
+ cpumask_var_t valid_cpus;
+ bool have_policy = false;
+ int ret = 0;
+ int cpu;
+
+ if (!zalloc_cpumask_var(&valid_cpus, GFP_KERNEL))
+ return -ENOMEM;
+
+ if (!zalloc_cpumask_var(&amu_fie_cpus, GFP_KERNEL)) {
+ ret = -ENOMEM;
+ goto free_valid_mask;
+ }
+
+ for_each_present_cpu(cpu) {
+ if (validate_cpu_freq_invariance_counters(cpu))
+ continue;
+ cpumask_set_cpu(cpu, valid_cpus);
+ have_policy |= enable_policy_freq_counters(cpu, valid_cpus);
+ }
+
+ /*
+ * If we are not restricted by cpufreq policies, we only enable
+ * the use of the AMU feature for FIE if all CPUs support AMU.
+ * Otherwise, enable_policy_freq_counters has already enabled
+ * policy cpus.
+ */
+ if (!have_policy && cpumask_equal(valid_cpus, cpu_present_mask))
+ cpumask_or(amu_fie_cpus, amu_fie_cpus, valid_cpus);
+
+ if (!cpumask_empty(amu_fie_cpus)) {
+ pr_info("CPUs[%*pbl]: counters will be used for FIE.",
+ cpumask_pr_args(amu_fie_cpus));
+ static_branch_enable(&amu_fie_key);
+ }
+
+free_valid_mask:
+ free_cpumask_var(valid_cpus);
+
+ return ret;
+}
+late_initcall_sync(init_amu_fie);
+
+bool arch_freq_counters_available(struct cpumask *cpus)
+{
+ return amu_freq_invariant() &&
+ cpumask_subset(cpus, amu_fie_cpus);
+}
+
+void topology_scale_freq_tick(void)
+{
+ u64 prev_core_cnt, prev_const_cnt;
+ u64 core_cnt, const_cnt, scale;
+ int cpu = smp_processor_id();
+
+ if (!amu_freq_invariant())
+ return;
+
+ if (!cpumask_test_cpu(cpu, amu_fie_cpus))
+ return;
+
+ const_cnt = read_sysreg_s(SYS_AMEVCNTR0_CONST_EL0);
+ core_cnt = read_sysreg_s(SYS_AMEVCNTR0_CORE_EL0);
+ prev_const_cnt = this_cpu_read(arch_const_cycles_prev);
+ prev_core_cnt = this_cpu_read(arch_core_cycles_prev);
+
+ if (unlikely(core_cnt <= prev_core_cnt ||
+ const_cnt <= prev_const_cnt))
+ goto store_and_exit;
+
+ /*
+ * /\core arch_max_freq_scale
+ * scale = ------- * --------------------
+ * /\const SCHED_CAPACITY_SCALE
+ *
+ * See validate_cpu_freq_invariance_counters() for details on
+ * arch_max_freq_scale and the use of SCHED_CAPACITY_SHIFT.
+ */
+ scale = core_cnt - prev_core_cnt;
+ scale *= this_cpu_read(arch_max_freq_scale);
+ scale = div64_u64(scale >> SCHED_CAPACITY_SHIFT,
+ const_cnt - prev_const_cnt);
+
+ scale = min_t(unsigned long, scale, SCHED_CAPACITY_SCALE);
+ this_cpu_write(freq_scale, (unsigned long)scale);
+
+store_and_exit:
+ this_cpu_write(arch_core_cycles_prev, core_cnt);
+ this_cpu_write(arch_const_cycles_prev, const_cnt);
+}
+#endif /* CONFIG_ARM64_AMU_EXTN */
diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index 6119e11a9f95..8d63673c1689 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -21,6 +21,10 @@
#include <linux/sched.h>
#include <linux/smp.h>
+__weak bool arch_freq_counters_available(struct cpumask *cpus)
+{
+ return false;
+}
DEFINE_PER_CPU(unsigned long, freq_scale) = SCHED_CAPACITY_SCALE;
void arch_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq,
@@ -29,6 +33,14 @@ void arch_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq,
unsigned long scale;
int i;
+ /*
+ * If the use of counters for FIE is enabled, just return as we don't
+ * want to update the scale factor with information from CPUFREQ.
+ * Instead the scale factor will be updated from arch_scale_freq_tick.
+ */
+ if (arch_freq_counters_available(cpus))
+ return;
+
scale = (cur_freq << SCHED_CAPACITY_SHIFT) / max_freq;
for_each_cpu(i, cpus)
diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h
index 3015ecbb90b1..1ccdddb541a7 100644
--- a/include/linux/arch_topology.h
+++ b/include/linux/arch_topology.h
@@ -33,6 +33,8 @@ unsigned long topology_get_freq_scale(int cpu)
return per_cpu(freq_scale, cpu);
}
+bool arch_freq_counters_available(struct cpumask *cpus);
+
struct cpu_topology {
int thread_id;
int core_id;
--
2.17.1
Using an arch timer with a frequency of less than 1MHz can potentially
result in incorrect functionality in systems that assume a reasonable
rate of the arch timer of 1 to 50MHz, described as typical in the
architecture specification.
Therefore, warn if the arch timer rate is below 1MHz, which is
considered atypical and worth emphasizing.
Signed-off-by: Ionela Voinescu <[email protected]>
Suggested-by: Valentin Schneider <[email protected]>
Acked-by: Marc Zyngier <[email protected]>
Cc: Mark Rutland <[email protected]>
Cc: Marc Zyngier <[email protected]>
---
drivers/clocksource/arm_arch_timer.c | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 9a5464c625b4..4faa930eabf8 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -885,6 +885,17 @@ static int arch_timer_starting_cpu(unsigned int cpu)
return 0;
}
+static int validate_timer_rate(void)
+{
+ if (!arch_timer_rate)
+ return -EINVAL;
+
+ /* Arch timer frequency < 1MHz can cause trouble */
+ WARN_ON(arch_timer_rate < 1000000);
+
+ return 0;
+}
+
/*
* For historical reasons, when probing with DT we use whichever (non-zero)
* rate was probed first, and don't verify that others match. If the first node
@@ -900,7 +911,7 @@ static void arch_timer_of_configure_rate(u32 rate, struct device_node *np)
arch_timer_rate = rate;
/* Check the timer frequency. */
- if (arch_timer_rate == 0)
+ if (validate_timer_rate())
pr_warn("frequency not available\n");
}
@@ -1594,9 +1605,10 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table)
* CNTFRQ value. This *must* be correct.
*/
arch_timer_rate = arch_timer_get_cntfrq();
- if (!arch_timer_rate) {
+ ret = validate_timer_rate();
+ if (ret) {
pr_err(FW_BUG "frequency not available.\n");
- return -EINVAL;
+ return ret;
}
arch_timer_uses_ppi = arch_timer_select_ppi();
--
2.17.1
Add weak function to return the hardware maximum frequency of a CPU,
with the default implementation returning cpuinfo.max_freq, which is
the best information we can generically get from the cpufreq framework.
The default can be overwritten by a strong function in platforms
that want to provide an alternative implementation, with more accurate
information, obtained either from hardware or firmware.
Signed-off-by: Ionela Voinescu <[email protected]>
Acked-by: Viresh Kumar <[email protected]>
Reviewed-by: Valentin Schneider <[email protected]>
Cc: Rafael J. Wysocki <[email protected]>
Cc: Viresh Kumar <[email protected]>
---
drivers/cpufreq/cpufreq.c | 20 ++++++++++++++++++++
include/linux/cpufreq.h | 5 +++++
2 files changed, 25 insertions(+)
diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index cbe6c94bf158..985228aee46f 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -1725,6 +1725,26 @@ unsigned int cpufreq_quick_get_max(unsigned int cpu)
}
EXPORT_SYMBOL(cpufreq_quick_get_max);
+/**
+ * cpufreq_get_hw_max_freq - get the max hardware frequency of the CPU
+ * @cpu: CPU number
+ *
+ * The default return value is the max_freq field of cpuinfo.
+ */
+__weak unsigned int cpufreq_get_hw_max_freq(unsigned int cpu)
+{
+ struct cpufreq_policy *policy = cpufreq_cpu_get(cpu);
+ unsigned int ret_freq = 0;
+
+ if (policy) {
+ ret_freq = policy->cpuinfo.max_freq;
+ cpufreq_cpu_put(policy);
+ }
+
+ return ret_freq;
+}
+EXPORT_SYMBOL(cpufreq_get_hw_max_freq);
+
static unsigned int __cpufreq_get(struct cpufreq_policy *policy)
{
if (unlikely(policy_is_inactive(policy)))
diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h
index 0fb561d1b524..f7240251a949 100644
--- a/include/linux/cpufreq.h
+++ b/include/linux/cpufreq.h
@@ -205,6 +205,7 @@ static inline bool policy_is_shared(struct cpufreq_policy *policy)
unsigned int cpufreq_get(unsigned int cpu);
unsigned int cpufreq_quick_get(unsigned int cpu);
unsigned int cpufreq_quick_get_max(unsigned int cpu);
+unsigned int cpufreq_get_hw_max_freq(unsigned int cpu);
void disable_cpufreq(void);
u64 get_cpu_idle_time(unsigned int cpu, u64 *wall, int io_busy);
@@ -232,6 +233,10 @@ static inline unsigned int cpufreq_quick_get_max(unsigned int cpu)
{
return 0;
}
+static inline unsigned int cpufreq_get_hw_max_freq(unsigned int cpu)
+{
+ return 0;
+}
static inline void disable_cpufreq(void) { }
#endif
--
2.17.1
Hi Ionela,
On Wed, Feb 26 2020, Ionela Voinescu wrote:
> v4 -> v5:
> - v4 can be found at [6]
> - [1/7] rebased on top of latest tip/sched/core and fixed conflicts;
> applied Reviewed-by from Valentin;
> - [5/7] applied Reviewed-by from Valentin
> - [6/7] applied Valentin and Pavan's suggested fixes; reworked
> arch_cpu_freq_counters (now arch_freq_counters_available) and
> confined it to the arch topology driver as recommended by Valentin
> and Lukasz;
>
I don't really have any more comments for v5. AFAICT what we are missing
now is Sudeep's ack for the arch_topology bits, and if Catalin / Will
are okay with it I think this can all go through the arm64 tree.
On 2/26/20 1:29 PM, Ionela Voinescu wrote:
> The Frequency Invariance Engine (FIE) is providing a frequency
> scaling correction factor that helps achieve more accurate
> load-tracking.
>
> So far, for arm and arm64 platforms, this scale factor has been
> obtained based on the ratio between the current frequency and the
> maximum supported frequency recorded by the cpufreq policy. The
> setting of this scale factor is triggered from cpufreq drivers by
> calling arch_set_freq_scale. The current frequency used in computation
> is the frequency requested by a governor, but it may not be the
> frequency that was implemented by the platform.
>
> This correction factor can also be obtained using a core counter and a
> constant counter to get information on the performance (frequency based
> only) obtained in a period of time. This will more accurately reflect
> the actual current frequency of the CPU, compared with the alternative
> implementation that reflects the request of a performance level from
> the OS.
>
> Therefore, implement arch_scale_freq_tick to use activity monitors, if
> present, for the computation of the frequency scale factor.
>
> The use of AMU counters depends on:
> - CONFIG_ARM64_AMU_EXTN - depents on the AMU extension being present
> - CONFIG_CPU_FREQ - the current frequency obtained using counter
> information is divided by the maximum frequency obtained from the
> cpufreq policy.
>
> While it is possible to have a combination of CPUs in the system with
> and without support for activity monitors, the use of counters for
> frequency invariance is only enabled for a CPU if all related CPUs
> (CPUs in the same frequency domain) support and have enabled the core
> and constant activity monitor counters. In this way, there is a clear
> separation between the policies for which arch_set_freq_scale (cpufreq
> based FIE) is used, and the policies for which arch_scale_freq_tick
> (counter based FIE) is used to set the frequency scale factor. For
> this purpose, a late_initcall_sync is registered to trigger validation
> work for policies that will enable or disable the use of AMU counters
> for frequency invariance. If CONFIG_CPU_FREQ is not defined, the use
> of counters is enabled on all CPUs only if all possible CPUs correctly
> support the necessary counters.
>
> Signed-off-by: Ionela Voinescu <[email protected]>
> Cc: Catalin Marinas <[email protected]>
> Cc: Will Deacon <[email protected]>
> Cc: Sudeep Holla <[email protected]>
> ---
> arch/arm64/include/asm/topology.h | 9 ++
> arch/arm64/kernel/cpufeature.c | 4 +
> arch/arm64/kernel/topology.c | 180 ++++++++++++++++++++++++++++++
> drivers/base/arch_topology.c | 12 ++
> include/linux/arch_topology.h | 2 +
> 5 files changed, 207 insertions(+)
>
> diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
> index a4d945db95a2..21d4d40d6243 100644
> --- a/arch/arm64/include/asm/topology.h
> +++ b/arch/arm64/include/asm/topology.h
> @@ -16,6 +16,15 @@ int pcibus_to_node(struct pci_bus *bus);
>
> #include <linux/arch_topology.h>
>
> +#ifdef CONFIG_ARM64_AMU_EXTN
> +/*
> + * Replace task scheduler's default counter-based
> + * frequency-invariance scale factor setting.
> + */
> +void topology_scale_freq_tick(void);
> +#define arch_scale_freq_tick topology_scale_freq_tick
> +#endif /* CONFIG_ARM64_AMU_EXTN */
> +
> /* Replace task scheduler's default frequency-invariant accounting */
> #define arch_scale_freq_capacity topology_get_freq_scale
>
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 60cebc071603..b8ec6c544d32 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -1241,12 +1241,16 @@ bool cpu_has_amu_feat(int cpu)
> return cpumask_test_cpu(cpu, &amu_cpus);
> }
>
> +/* Initialize the use of AMU counters for frequency invariance */
> +extern void init_cpu_freq_invariance_counters(void);
> +
> static void cpu_amu_enable(struct arm64_cpu_capabilities const *cap)
> {
> if (has_cpuid_feature(cap, SCOPE_LOCAL_CPU)) {
> pr_info("detected CPU%d: Activity Monitors Unit (AMU)\n",
> smp_processor_id());
> cpumask_set_cpu(smp_processor_id(), &amu_cpus);
> + init_cpu_freq_invariance_counters();
> }
> }
>
> diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
> index fa9528dfd0ce..0801a0f3c156 100644
> --- a/arch/arm64/kernel/topology.c
> +++ b/arch/arm64/kernel/topology.c
> @@ -14,6 +14,7 @@
> #include <linux/acpi.h>
> #include <linux/arch_topology.h>
> #include <linux/cacheinfo.h>
> +#include <linux/cpufreq.h>
> #include <linux/init.h>
> #include <linux/percpu.h>
>
> @@ -120,4 +121,183 @@ int __init parse_acpi_topology(void)
> }
> #endif
>
> +#ifdef CONFIG_ARM64_AMU_EXTN
>
> +#undef pr_fmt
> +#define pr_fmt(fmt) "AMU: " fmt
> +
> +static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale);
> +static DEFINE_PER_CPU(u64, arch_const_cycles_prev);
> +static DEFINE_PER_CPU(u64, arch_core_cycles_prev);
> +static cpumask_var_t amu_fie_cpus;
> +
> +/* Initialize counter reference per-cpu variables for the current CPU */
> +void init_cpu_freq_invariance_counters(void)
> +{
> + this_cpu_write(arch_core_cycles_prev,
> + read_sysreg_s(SYS_AMEVCNTR0_CORE_EL0));
> + this_cpu_write(arch_const_cycles_prev,
> + read_sysreg_s(SYS_AMEVCNTR0_CONST_EL0));
> +}
> +
> +static int validate_cpu_freq_invariance_counters(int cpu)
> +{
> + u64 max_freq_hz, ratio;
> +
> + if (!cpu_has_amu_feat(cpu)) {
> + pr_debug("CPU%d: counters are not supported.\n", cpu);
> + return -EINVAL;
> + }
> +
> + if (unlikely(!per_cpu(arch_const_cycles_prev, cpu) ||
> + !per_cpu(arch_core_cycles_prev, cpu))) {
> + pr_debug("CPU%d: cycle counters are not enabled.\n", cpu);
> + return -EINVAL;
> + }
> +
> + /* Convert maximum frequency from KHz to Hz and validate */
> + max_freq_hz = cpufreq_get_hw_max_freq(cpu) * 1000;
> + if (unlikely(!max_freq_hz)) {
> + pr_debug("CPU%d: invalid maximum frequency.\n", cpu);
> + return -EINVAL;
> + }
> +
> + /*
> + * Pre-compute the fixed ratio between the frequency of the constant
> + * counter and the maximum frequency of the CPU.
> + *
> + * const_freq
> + * arch_max_freq_scale = ---------------- * SCHED_CAPACITY_SCALE²
> + * cpuinfo_max_freq
> + *
> + * We use a factor of 2 * SCHED_CAPACITY_SHIFT -> SCHED_CAPACITY_SCALE²
> + * in order to ensure a good resolution for arch_max_freq_scale for
> + * very low arch timer frequencies (down to the KHz range which should
> + * be unlikely).
> + */
> + ratio = (u64)arch_timer_get_rate() << (2 * SCHED_CAPACITY_SHIFT);
> + ratio = div64_u64(ratio, max_freq_hz);
> + if (!ratio) {
> + WARN_ONCE(1, "System timer frequency too low.\n");
> + return -EINVAL;
> + }
> +
> + per_cpu(arch_max_freq_scale, cpu) = (unsigned long)ratio;
> +
> + return 0;
> +}
> +
> +static inline bool
> +enable_policy_freq_counters(int cpu, cpumask_var_t valid_cpus)
> +{
> + struct cpufreq_policy *policy = cpufreq_cpu_get(cpu);
> +
> + if (!policy) {
> + pr_debug("CPU%d: No cpufreq policy found.\n", cpu);
> + return false;
> + }
> +
> + if (cpumask_subset(policy->related_cpus, valid_cpus))
> + cpumask_or(amu_fie_cpus, policy->related_cpus,
> + amu_fie_cpus);
> +
> + cpufreq_cpu_put(policy);
> +
> + return true;
> +}
> +
> +static DEFINE_STATIC_KEY_FALSE(amu_fie_key);
> +#define amu_freq_invariant() static_branch_unlikely(&amu_fie_key)
> +
> +static int __init init_amu_fie(void)
> +{
> + cpumask_var_t valid_cpus;
> + bool have_policy = false;
> + int ret = 0;
> + int cpu;
> +
> + if (!zalloc_cpumask_var(&valid_cpus, GFP_KERNEL))
> + return -ENOMEM;
> +
> + if (!zalloc_cpumask_var(&amu_fie_cpus, GFP_KERNEL)) {
> + ret = -ENOMEM;
> + goto free_valid_mask;
> + }
> +
> + for_each_present_cpu(cpu) {
> + if (validate_cpu_freq_invariance_counters(cpu))
> + continue;
> + cpumask_set_cpu(cpu, valid_cpus);
> + have_policy |= enable_policy_freq_counters(cpu, valid_cpus);
> + }
> +
> + /*
> + * If we are not restricted by cpufreq policies, we only enable
> + * the use of the AMU feature for FIE if all CPUs support AMU.
> + * Otherwise, enable_policy_freq_counters has already enabled
> + * policy cpus.
> + */
> + if (!have_policy && cpumask_equal(valid_cpus, cpu_present_mask))
> + cpumask_or(amu_fie_cpus, amu_fie_cpus, valid_cpus);
> +
> + if (!cpumask_empty(amu_fie_cpus)) {
> + pr_info("CPUs[%*pbl]: counters will be used for FIE.",
> + cpumask_pr_args(amu_fie_cpus));
> + static_branch_enable(&amu_fie_key);
> + }
> +
> +free_valid_mask:
> + free_cpumask_var(valid_cpus);
> +
> + return ret;
> +}
> +late_initcall_sync(init_amu_fie);
> +
> +bool arch_freq_counters_available(struct cpumask *cpus)
> +{
> + return amu_freq_invariant() &&
> + cpumask_subset(cpus, amu_fie_cpus);
> +}
> +
> +void topology_scale_freq_tick(void)
> +{
> + u64 prev_core_cnt, prev_const_cnt;
> + u64 core_cnt, const_cnt, scale;
> + int cpu = smp_processor_id();
> +
> + if (!amu_freq_invariant())
> + return;
> +
> + if (!cpumask_test_cpu(cpu, amu_fie_cpus))
> + return;
> +
> + const_cnt = read_sysreg_s(SYS_AMEVCNTR0_CONST_EL0);
> + core_cnt = read_sysreg_s(SYS_AMEVCNTR0_CORE_EL0);
> + prev_const_cnt = this_cpu_read(arch_const_cycles_prev);
> + prev_core_cnt = this_cpu_read(arch_core_cycles_prev);
> +
> + if (unlikely(core_cnt <= prev_core_cnt ||
> + const_cnt <= prev_const_cnt))
> + goto store_and_exit;
> +
> + /*
> + * /\core arch_max_freq_scale
> + * scale = ------- * --------------------
> + * /\const SCHED_CAPACITY_SCALE
> + *
> + * See validate_cpu_freq_invariance_counters() for details on
> + * arch_max_freq_scale and the use of SCHED_CAPACITY_SHIFT.
> + */
> + scale = core_cnt - prev_core_cnt;
> + scale *= this_cpu_read(arch_max_freq_scale);
> + scale = div64_u64(scale >> SCHED_CAPACITY_SHIFT,
> + const_cnt - prev_const_cnt);
> +
> + scale = min_t(unsigned long, scale, SCHED_CAPACITY_SCALE);
> + this_cpu_write(freq_scale, (unsigned long)scale);
> +
> +store_and_exit:
> + this_cpu_write(arch_core_cycles_prev, core_cnt);
> + this_cpu_write(arch_const_cycles_prev, const_cnt);
> +}
> +#endif /* CONFIG_ARM64_AMU_EXTN */
> diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
> index 6119e11a9f95..8d63673c1689 100644
> --- a/drivers/base/arch_topology.c
> +++ b/drivers/base/arch_topology.c
> @@ -21,6 +21,10 @@
> #include <linux/sched.h>
> #include <linux/smp.h>
>
> +__weak bool arch_freq_counters_available(struct cpumask *cpus)
> +{
> + return false;
> +}
> DEFINE_PER_CPU(unsigned long, freq_scale) = SCHED_CAPACITY_SCALE;
>
> void arch_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq,
> @@ -29,6 +33,14 @@ void arch_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq,
> unsigned long scale;
> int i;
>
> + /*
> + * If the use of counters for FIE is enabled, just return as we don't
> + * want to update the scale factor with information from CPUFREQ.
> + * Instead the scale factor will be updated from arch_scale_freq_tick.
> + */
> + if (arch_freq_counters_available(cpus))
> + return;
> +
> scale = (cur_freq << SCHED_CAPACITY_SHIFT) / max_freq;
>
> for_each_cpu(i, cpus)
> diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h
> index 3015ecbb90b1..1ccdddb541a7 100644
> --- a/include/linux/arch_topology.h
> +++ b/include/linux/arch_topology.h
> @@ -33,6 +33,8 @@ unsigned long topology_get_freq_scale(int cpu)
> return per_cpu(freq_scale, cpu);
> }
>
> +bool arch_freq_counters_available(struct cpumask *cpus);
> +
> struct cpu_topology {
> int thread_id;
> int core_id;
>
Looks good
Reviewed-by: Lukasz Luba <[email protected]>
Regards,
Lukasz
Hi Ionela,
On 2020-02-26 13:29, Ionela Voinescu wrote:
> Access to the AMU counters should be disabled by default in kvm guests,
> as information from the counters might reveal activity in other guests
> or activity on the host.
>
> Therefore, disable access to AMU registers from EL0 and EL1 in kvm
> guests by:
> - Hiding the presence of the extension in the feature register
> (SYS_ID_AA64PFR0_EL1) on the VCPU.
> - Disabling access to the AMU registers before switching to the guest.
> - Trapping accesses and injecting an undefined instruction into the
> guest.
>
> Signed-off-by: Ionela Voinescu <[email protected]>
> Reviewed-by: Valentin Schneider <[email protected]>
> Reviewed-by: Suzuki K Poulose <[email protected]>
> Cc: Marc Zyngier <[email protected]>
> Cc: James Morse <[email protected]>
> Cc: Julien Thierry <[email protected]>
> Cc: Suzuki K Poulose <[email protected]>
> Cc: Catalin Marinas <[email protected]>
> Cc: Will Deacon <[email protected]>
Acked-by: Marc Zyngier <[email protected]>
A small comment below:
[...]
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 3e909b117f0c..44354c812783 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -1003,6 +1003,20 @@ static bool access_pmuserenr(struct kvm_vcpu
> *vcpu, struct sys_reg_params *p,
> { SYS_DESC(SYS_PMEVTYPERn_EL0(n)), \
> access_pmu_evtyper, reset_unknown, (PMEVTYPER0_EL0 + n), }
>
> +static bool access_amu(struct kvm_vcpu *vcpu, struct sys_reg_params
> *p,
> + const struct sys_reg_desc *r)
> +{
> + kvm_inject_undefined(vcpu);
> +
> + return false;
> +}
> +
> +/* Macro to expand the AMU counter and type registers*/
> +#define AMU_AMEVCNTR0_EL0(n) { SYS_DESC(SYS_AMEVCNTR0_EL0(n)),
> access_amu }
> +#define AMU_AMEVTYPE0_EL0(n) { SYS_DESC(SYS_AMEVTYPE0_EL0(n)),
> access_amu }
> +#define AMU_AMEVCNTR1_EL0(n) { SYS_DESC(SYS_AMEVCNTR1_EL0(n)),
> access_amu }
> +#define AMU_AMEVTYPE1_EL0(n) { SYS_DESC(SYS_AMEVTYPE1_EL0(n)),
> access_amu }
> +
> static bool trap_ptrauth(struct kvm_vcpu *vcpu,
> struct sys_reg_params *p,
> const struct sys_reg_desc *rd)
> @@ -1078,8 +1092,10 @@ static u64 read_id_reg(const struct kvm_vcpu
> *vcpu,
> (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
> u64 val = raz ? 0 : read_sanitised_ftr_reg(id);
>
> - if (id == SYS_ID_AA64PFR0_EL1 && !vcpu_has_sve(vcpu)) {
> - val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
> + if (id == SYS_ID_AA64PFR0_EL1) {
> + if (!vcpu_has_sve(vcpu))
> + val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
> + val &= ~(0xfUL << ID_AA64PFR0_AMU_SHIFT);
This will definitely conflict with some of the ongoing rework I have[1].
I'm happy to provide this as a stable branch for you to rebase on top,
or use an arm64 provided branch to rebase my stoff on top.
Just let me know how you want to proceed.
Thanks,
M.
[1]
https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/commit/?h=kvm-arm64/debug-fixes-5.6&id=454fb7398d3626328f7f771c07d21e894e4e1a3b
--
Jazz is not dead. It just smells funny...
Hi Ionela,
On Wed, Feb 26, 2020 at 01:29:41PM +0000, Ionela Voinescu wrote:
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index dbc22d684627..49f0c436928f 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -318,6 +318,15 @@
> Format: <a>,<b>
> See also Documentation/input/joydev/joystick.rst
>
> + amu= [ARM64]
> + Enables or disables detection, enablement and access to
> + counter registers of the Activity Monitors Unit (AMU).
> + Format: amu=[0/1/on/off/y/n]
> + amu=[0/off/n] ensures access to AMU's counter registers
> + is not attempted.
> + amu=[1/on/y] (default) enables detection and access to
> + AMU's counter registers.
Is the only reason for this parameter to be able to disable the feature
if the firmware doesn't support it? According to the Kconfig entry, you
may see weird behaviour, firmware lock-up. Is the user supposed to try
again with amu=0?
I'm not particularly fond of adding kernel parameters to work around
broken firmware. We have other architecture features (e.g. PtrAuth) that
need enabling at EL3 but we don't have such parameters. If it's likely
that we hit this issue in practice, I'd rather have the firmware
describing the presence of AMU via some DT entry. But I'd rather not
bother at all, just get the vendors to update their firmware.
If we drop this parameter, patch 1 would need to change. Otherwise the
patches look fine.
Thanks.
--
Catalin
Hi Ionela,
On 26/02/2020 13:29, Ionela Voinescu wrote:
> The activity monitors extension is an optional extension introduced
> by the ARMv8.4 CPU architecture. In order to access the activity
> monitors counters safely, if desired, the kernel should detect the
> presence of the extension through the feature register, and mediate
> the access.
>
> Therefore, disable direct accesses to activity monitors counters
> from EL0 (userspace) and trap them to EL1 (kernel).
>
> To be noted that the ARM64_AMU_EXTN kernel config and the disable_amu
> kernel parameter do not have an effect on this code. Given that the
> amuserenr_el0 resets to an UNKNOWN value, setting the trap of EL0
> accesses to EL1 is always attempted for safety and security
> considerations.
> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> index aafed6902411..7103027b4e64 100644
> --- a/arch/arm64/mm/proc.S
> +++ b/arch/arm64/mm/proc.S
> @@ -131,6 +131,7 @@ alternative_endif
> ubfx x11, x11, #1, #1
> msr oslar_el1, x11
> reset_pmuserenr_el0 x0 // Disable PMU access from EL0
> + reset_amuserenr_el0 x0 // Disable AMU access from EL0
>
> alternative_if ARM64_HAS_RAS_EXTN
> msr_s SYS_DISR_EL1, xzr
(This above hunk is in: cpu_do_resume, and this next one is __cpu_setup,)
> @@ -423,6 +424,8 @@ SYM_FUNC_START(__cpu_setup)
> isb // Unmask debug exceptions now,
> enable_dbg // since this is per-cpu
> reset_pmuserenr_el0 x0 // Disable PMU access from EL0
> + reset_amuserenr_el0 x0 // Disable AMU access from EL0
I think you only need this in __cpu_setup. The entry-point from cpu-idle calls:
| cpu_resume
| ->__cpu_setup
| -->reset_amuserenr_el0
| ->_cpu_resume
| -->cpu_do_resume
| --->reset_amuserenr_el0
(Which means the PMU reset call is redundant too).
Its harmless, and needs cleaning up already, so regardless:
Reviewed-by: James Morse <[email protected]>
Thanks,
James
Hi Catalin,
On Friday 28 Feb 2020 at 10:32:34 (+0000), Catalin Marinas wrote:
> Hi Ionela,
>
> On Wed, Feb 26, 2020 at 01:29:41PM +0000, Ionela Voinescu wrote:
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index dbc22d684627..49f0c436928f 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -318,6 +318,15 @@
> > Format: <a>,<b>
> > See also Documentation/input/joydev/joystick.rst
> >
> > + amu= [ARM64]
> > + Enables or disables detection, enablement and access to
> > + counter registers of the Activity Monitors Unit (AMU).
> > + Format: amu=[0/1/on/off/y/n]
> > + amu=[0/off/n] ensures access to AMU's counter registers
> > + is not attempted.
> > + amu=[1/on/y] (default) enables detection and access to
> > + AMU's counter registers.
>
> Is the only reason for this parameter to be able to disable the feature
> if the firmware doesn't support it? According to the Kconfig entry, you
> may see weird behaviour, firmware lock-up. Is the user supposed to try
> again with amu=0?
>
> I'm not particularly fond of adding kernel parameters to work around
> broken firmware. We have other architecture features (e.g. PtrAuth) that
> need enabling at EL3 but we don't have such parameters. If it's likely
> that we hit this issue in practice, I'd rather have the firmware
> describing the presence of AMU via some DT entry. But I'd rather not
> bother at all, just get the vendors to update their firmware.
>
The firmware is supposed to do three actions for the kernel to be able
to use the counters: enable access to EL2/EL1, enable the counters and
save/restore the counters before/after core-off.
Improper firmware support can trigger different issues: kernel/firmware
lockup/panic, invalid counter values (0, non-monotonic). Some of them
might be less likely (firmware lockups), and some might just be due to
present but improper support(save/restore) and therefore more likely.
The users of the counters, for example frequency invariance [6/7], does
some validation for this, but unfortunately not all conditions can be
fully mitigated - validate and bail out if some condition is not
accomplished - for example the save and restore functionality. This
might result in improper scale values after idle.
Therefore, the amu kernel parameter is not only there if the firmware
does not support AMU, but it's also there if the firmware support is
broken/improper. The kernel parameter was added at Suzuki's
recommendation to be able to bypass its use in single kernels that are
meant to run on multiple platforms. I also believe this is nice to have
even for platforms that properly support AMU, but they might not want
the use of the feature in the kernel.
> If we drop this parameter, patch 1 would need to change. Otherwise the
> patches look fine.
>
This being said, I agree this was added as a 'just in case' and not as
support for a likely scenario, therefore, I don't fully disagree to drop
it for now.
Let me know what you think. If you'd still rather I drop it, I can do that
and rebase on top of Marc's changes and push v6.
Thanks,
Ionela.
Hi Marc,
On Thursday 27 Feb 2020 at 19:58:32 (+0000), Marc Zyngier wrote:
[..]
> > static bool trap_ptrauth(struct kvm_vcpu *vcpu,
> > struct sys_reg_params *p,
> > const struct sys_reg_desc *rd)
> > @@ -1078,8 +1092,10 @@ static u64 read_id_reg(const struct kvm_vcpu
> > *vcpu,
> > (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
> > u64 val = raz ? 0 : read_sanitised_ftr_reg(id);
> >
> > - if (id == SYS_ID_AA64PFR0_EL1 && !vcpu_has_sve(vcpu)) {
> > - val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
> > + if (id == SYS_ID_AA64PFR0_EL1) {
> > + if (!vcpu_has_sve(vcpu))
> > + val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
> > + val &= ~(0xfUL << ID_AA64PFR0_AMU_SHIFT);
>
> This will definitely conflict with some of the ongoing rework I have[1].
> I'm happy to provide this as a stable branch for you to rebase on top,
> or use an arm64 provided branch to rebase my stoff on top.
>
> Just let me know how you want to proceed.
>
Sure, a stable branch with this would be great. I'll wait for a reply
from Catalin for [1/7] and I'll rebase on top of your provided branch
when it's clear whether other changes are needed to this set.
Much appreciated,
Ionela.
> Thanks,
>
> M.
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/commit/?h=kvm-arm64/debug-fixes-5.6&id=454fb7398d3626328f7f771c07d21e894e4e1a3b
> --
> Jazz is not dead. It just smells funny...
On Mon, Mar 02, 2020 at 02:23:26PM +0000, Ionela Voinescu wrote:
> On Friday 28 Feb 2020 at 10:32:34 (+0000), Catalin Marinas wrote:
> > On Wed, Feb 26, 2020 at 01:29:41PM +0000, Ionela Voinescu wrote:
> > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > index dbc22d684627..49f0c436928f 100644
> > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > @@ -318,6 +318,15 @@
> > > Format: <a>,<b>
> > > See also Documentation/input/joydev/joystick.rst
> > >
> > > + amu= [ARM64]
> > > + Enables or disables detection, enablement and access to
> > > + counter registers of the Activity Monitors Unit (AMU).
> > > + Format: amu=[0/1/on/off/y/n]
> > > + amu=[0/off/n] ensures access to AMU's counter registers
> > > + is not attempted.
> > > + amu=[1/on/y] (default) enables detection and access to
> > > + AMU's counter registers.
> >
> > Is the only reason for this parameter to be able to disable the feature
> > if the firmware doesn't support it? According to the Kconfig entry, you
> > may see weird behaviour, firmware lock-up. Is the user supposed to try
> > again with amu=0?
> >
> > I'm not particularly fond of adding kernel parameters to work around
> > broken firmware. We have other architecture features (e.g. PtrAuth) that
> > need enabling at EL3 but we don't have such parameters. If it's likely
> > that we hit this issue in practice, I'd rather have the firmware
> > describing the presence of AMU via some DT entry. But I'd rather not
> > bother at all, just get the vendors to update their firmware.
>
> The firmware is supposed to do three actions for the kernel to be able
> to use the counters: enable access to EL2/EL1, enable the counters and
> save/restore the counters before/after core-off.
[...]
> Therefore, the amu kernel parameter is not only there if the firmware
> does not support AMU, but it's also there if the firmware support is
> broken/improper. The kernel parameter was added at Suzuki's
> recommendation to be able to bypass its use in single kernels that are
> meant to run on multiple platforms.
Single kernel images are supposed to run on multiple platforms while
using the same command line arguments.
There are many other ways firmware can screw up but I'm not keen on
working on such assumption and preemptively adding options to ignore CPU
features.
> I also believe this is nice to have even for platforms that properly
> support AMU, but they might not want the use of the feature in the
> kernel.
Are there any downsides to this feature? If you want it for testing
purposes, i.e. different scheduler behaviour, fine by me but I'd change
the text in the Kconfig to not even imply that firmware is allowed to be
broken as we have a workaround in the kernel.
> > If we drop this parameter, patch 1 would need to change. Otherwise the
> > patches look fine.
>
> This being said, I agree this was added as a 'just in case' and not as
> support for a likely scenario, therefore, I don't fully disagree to drop
> it for now.
If you need it for testing different scheduler behaviours, maybe
big.LITTLE where AMU is only supported on some CPUs, than keep it. If
it's only on the assumption that the firmware may be broken, please
remove it.
Thanks.
--
Catalin
Hi Catalin,
On Tuesday 03 Mar 2020 at 16:58:45 (+0000), Catalin Marinas wrote:
> On Mon, Mar 02, 2020 at 02:23:26PM +0000, Ionela Voinescu wrote:
> > On Friday 28 Feb 2020 at 10:32:34 (+0000), Catalin Marinas wrote:
> > > On Wed, Feb 26, 2020 at 01:29:41PM +0000, Ionela Voinescu wrote:
> > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > > index dbc22d684627..49f0c436928f 100644
> > > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > > @@ -318,6 +318,15 @@
> > > > Format: <a>,<b>
> > > > See also Documentation/input/joydev/joystick.rst
> > > >
> > > > + amu= [ARM64]
> > > > + Enables or disables detection, enablement and access to
> > > > + counter registers of the Activity Monitors Unit (AMU).
> > > > + Format: amu=[0/1/on/off/y/n]
> > > > + amu=[0/off/n] ensures access to AMU's counter registers
> > > > + is not attempted.
> > > > + amu=[1/on/y] (default) enables detection and access to
> > > > + AMU's counter registers.
> > >
> > > Is the only reason for this parameter to be able to disable the feature
> > > if the firmware doesn't support it? According to the Kconfig entry, you
> > > may see weird behaviour, firmware lock-up. Is the user supposed to try
> > > again with amu=0?
> > >
> > > I'm not particularly fond of adding kernel parameters to work around
> > > broken firmware. We have other architecture features (e.g. PtrAuth) that
> > > need enabling at EL3 but we don't have such parameters. If it's likely
> > > that we hit this issue in practice, I'd rather have the firmware
> > > describing the presence of AMU via some DT entry. But I'd rather not
> > > bother at all, just get the vendors to update their firmware.
> >
> > The firmware is supposed to do three actions for the kernel to be able
> > to use the counters: enable access to EL2/EL1, enable the counters and
> > save/restore the counters before/after core-off.
> [...]
> > Therefore, the amu kernel parameter is not only there if the firmware
> > does not support AMU, but it's also there if the firmware support is
> > broken/improper. The kernel parameter was added at Suzuki's
> > recommendation to be able to bypass its use in single kernels that are
> > meant to run on multiple platforms.
>
> Single kernel images are supposed to run on multiple platforms while
> using the same command line arguments.
>
> There are many other ways firmware can screw up but I'm not keen on
> working on such assumption and preemptively adding options to ignore CPU
> features.
>
> > I also believe this is nice to have even for platforms that properly
> > support AMU, but they might not want the use of the feature in the
> > kernel.
>
> Are there any downsides to this feature? If you want it for testing
> purposes, i.e. different scheduler behaviour, fine by me but I'd change
> the text in the Kconfig to not even imply that firmware is allowed to be
> broken as we have a workaround in the kernel.
>
This solution would not be appropriate to select different scheduler
behaviours. That would be the end result of "amu=0", but it would not be
a clean way to select the source of information for frequency
invariance. The scheduler and the frequency invariance engine is only
one of the potential users of activity monitors, while this kernel
parameter would disable detection for the full feature so it will affect
frequency invariance behaviour and other future users of the counters.
In no way I want to send the message that firmware is allowed to be
broken or that this is a good way to tune scheduler behaviour.
My flawed logic above was to suggest that a few small reasons (potential
broken firmware, potential interest to turn off the feature at runtime)
would make a big one to justify the parameter, but none of these
fully stand on their own.
> > > If we drop this parameter, patch 1 would need to change. Otherwise the
> > > patches look fine.
> >
> > This being said, I agree this was added as a 'just in case' and not as
> > support for a likely scenario, therefore, I don't fully disagree to drop
> > it for now.
>
> If you need it for testing different scheduler behaviours, maybe
> big.LITTLE where AMU is only supported on some CPUs, than keep it. If
> it's only on the assumption that the firmware may be broken, please
> remove it.
>
In regards to frequency invariance, there is no downside of using the
feature and, if available, it's preferred to use the counters independent
on whether all CPUs support them on not. But this would be a bad way to
switch between sources of information (cpufreq or counters) for
frequency invariance in any case, so I'll remove the parameter.
Thanks,
Ionela.
> Thanks.
>
> --
> Catalin
>
> _______________________________________________
> linux-arm-kernel mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Hi James,
On Friday 28 Feb 2020 at 16:44:53 (+0000), James Morse wrote:
> Hi Ionela,
>
> On 26/02/2020 13:29, Ionela Voinescu wrote:
> > The activity monitors extension is an optional extension introduced
> > by the ARMv8.4 CPU architecture. In order to access the activity
> > monitors counters safely, if desired, the kernel should detect the
> > presence of the extension through the feature register, and mediate
> > the access.
> >
> > Therefore, disable direct accesses to activity monitors counters
> > from EL0 (userspace) and trap them to EL1 (kernel).
> >
> > To be noted that the ARM64_AMU_EXTN kernel config and the disable_amu
> > kernel parameter do not have an effect on this code. Given that the
> > amuserenr_el0 resets to an UNKNOWN value, setting the trap of EL0
> > accesses to EL1 is always attempted for safety and security
> > considerations.
>
> > diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> > index aafed6902411..7103027b4e64 100644
> > --- a/arch/arm64/mm/proc.S
> > +++ b/arch/arm64/mm/proc.S
>
> > @@ -131,6 +131,7 @@ alternative_endif
> > ubfx x11, x11, #1, #1
> > msr oslar_el1, x11
> > reset_pmuserenr_el0 x0 // Disable PMU access from EL0
> > + reset_amuserenr_el0 x0 // Disable AMU access from EL0
> >
> > alternative_if ARM64_HAS_RAS_EXTN
> > msr_s SYS_DISR_EL1, xzr
>
> (This above hunk is in: cpu_do_resume, and this next one is __cpu_setup,)
>
> > @@ -423,6 +424,8 @@ SYM_FUNC_START(__cpu_setup)
> > isb // Unmask debug exceptions now,
> > enable_dbg // since this is per-cpu
> > reset_pmuserenr_el0 x0 // Disable PMU access from EL0
> > + reset_amuserenr_el0 x0 // Disable AMU access from EL0
>
> I think you only need this in __cpu_setup. The entry-point from cpu-idle calls:
> | cpu_resume
> | ->__cpu_setup
> | -->reset_amuserenr_el0
> | ->_cpu_resume
> | -->cpu_do_resume
> | --->reset_amuserenr_el0
>
> (Which means the PMU reset call is redundant too).
>
Thanks, that seems to be so. I'll submit a separate fix for both amu
and pmu, if you don't mind, after this set, so it will be a specific
cleanup patch.
> Its harmless, and needs cleaning up already, so regardless:
> Reviewed-by: James Morse <[email protected]>
>
Thank you very much for the review,
Ionela.
>
>
> Thanks,
>
> James
Hi Marc,
On Thursday 27 Feb 2020 at 19:58:32 (+0000), Marc Zyngier wrote:
[..]
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index 3e909b117f0c..44354c812783 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -1003,6 +1003,20 @@ static bool access_pmuserenr(struct kvm_vcpu
> > *vcpu, struct sys_reg_params *p,
> > { SYS_DESC(SYS_PMEVTYPERn_EL0(n)), \
> > access_pmu_evtyper, reset_unknown, (PMEVTYPER0_EL0 + n), }
> >
> > +static bool access_amu(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> > + const struct sys_reg_desc *r)
> > +{
> > + kvm_inject_undefined(vcpu);
> > +
> > + return false;
> > +}
> > +
> > +/* Macro to expand the AMU counter and type registers*/
> > +#define AMU_AMEVCNTR0_EL0(n) { SYS_DESC(SYS_AMEVCNTR0_EL0(n)),
> > access_amu }
> > +#define AMU_AMEVTYPE0_EL0(n) { SYS_DESC(SYS_AMEVTYPE0_EL0(n)),
> > access_amu }
> > +#define AMU_AMEVCNTR1_EL0(n) { SYS_DESC(SYS_AMEVCNTR1_EL0(n)),
> > access_amu }
> > +#define AMU_AMEVTYPE1_EL0(n) { SYS_DESC(SYS_AMEVTYPE1_EL0(n)),
> > access_amu }
> > +
> > static bool trap_ptrauth(struct kvm_vcpu *vcpu,
> > struct sys_reg_params *p,
> > const struct sys_reg_desc *rd)
> > @@ -1078,8 +1092,10 @@ static u64 read_id_reg(const struct kvm_vcpu
> > *vcpu,
> > (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
> > u64 val = raz ? 0 : read_sanitised_ftr_reg(id);
> >
> > - if (id == SYS_ID_AA64PFR0_EL1 && !vcpu_has_sve(vcpu)) {
> > - val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
> > + if (id == SYS_ID_AA64PFR0_EL1) {
> > + if (!vcpu_has_sve(vcpu))
> > + val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
> > + val &= ~(0xfUL << ID_AA64PFR0_AMU_SHIFT);
>
> This will definitely conflict with some of the ongoing rework I have[1].
> I'm happy to provide this as a stable branch for you to rebase on top,
> or use an arm64 provided branch to rebase my stoff on top.
>
> Just let me know how you want to proceed.
>
Catalin added the AMU patches on top of 5.6-rc3 at [1].
Is this okay as a base branch for your patches?
Thanks,
Ionela.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/log/?h=for-next/amu
> Thanks,
>
> M.
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/commit/?h=kvm-arm64/debug-fixes-5.6&id=454fb7398d3626328f7f771c07d21e894e4e1a3b
> --
> Jazz is not dead. It just smells funny...
On 2020-03-09 14:25, Ionela Voinescu wrote:
Hi Ionela,
[now with everyone on cc...]
>> This will definitely conflict with some of the ongoing rework I
>> have[1].
>> I'm happy to provide this as a stable branch for you to rebase on top,
>> or use an arm64 provided branch to rebase my stoff on top.
>>
>> Just let me know how you want to proceed.
>>
>
> Catalin added the AMU patches on top of 5.6-rc3 at [1].
> Is this okay as a base branch for your patches?
Sure, no problem. I still need to respin those...
Thanks,
M.
--
Jazz is not dead. It just smells funny...