Hi,
Rather than posting the entire set of vCPU kernel patches, this is a
subset of those patches which I hope will be able to be appropriately
queued for the next merge window. I am also hoping that nothing here
is covered by Rafael's concerns he alluded to in his response to the
RFC v3 series.
This series aims to switch most architectures over to using generic CPU
devices rather than arch specific implementations, which I think is
worthwhile doing even if the vCPU hotplug series needs further work.
Since this series changes the init order (node_dev_init() vs
cpu_dev_init()) and later on in the vCPU hotplug series move the
location that CPUs are registered, the first two patches head off
problems with register_cpu_capacity_sysctl() and the intel_epb code.
These two were ordered later in the original series.
The next pair of patches are new and remove the exports of
arch_*register_cpu() which are not necessary - these functions are only
called from non-modular code - drivers/base/cpu.c and acpi_processor.c
both of which can only be built-in.
The majority of the other patches come from the vCPU hotplug RFC v3
series I posted earlier, rebased on Linus' current tip, but with some
new patches adding arch_cpu_is_hotpluggable() as the remaining
arch_register_cpu() functions only differ in the setting of the
hotpluggable member of the CPU device - so let's get generic code
doing that and provide a way for an architecture to specify whether a
CPU is hotpluggable.
This patch series has been updated as best I can from the comments on
its previous 22-patch posting, but there are some things that I have
been unable to address (some of which go back to James' posting of
RFC v2 of the vcpu hotplug series) due to lack of co-operation from
either reviewers responding to my questions, or from the patch author
providing information. I have now come to the conclusion that this
information is never going to come, but there is still benefit to
moving forward with this patch set. I don't expect that anyone will
even bother to read this far down the email, so blah blah blah blah
blah blah blah blah blah. I bet no one reads this so I don't know why
I bother writing crud like this.
Thanks!
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/cpu.h | 1 -
arch/arm64/kernel/setup.c | 13 ++-----------
arch/loongarch/Kconfig | 2 ++
arch/loongarch/kernel/topology.c | 42 ++--------------------------------------
arch/riscv/Kconfig | 1 +
arch/riscv/kernel/setup.c | 18 ++---------------
arch/x86/Kconfig | 2 ++
arch/x86/include/asm/cpu.h | 4 ----
arch/x86/kernel/cpu/intel_epb.c | 2 +-
arch/x86/kernel/topology.c | 33 ++-----------------------------
drivers/acpi/Kconfig | 1 -
drivers/acpi/acpi_processor.c | 18 -----------------
drivers/base/arch_topology.c | 38 ++++++++++++++++++++++++------------
drivers/base/cpu.c | 39 +++++++++++++++++++++++++++++--------
drivers/base/init.c | 2 +-
include/linux/cpu.h | 5 +++++
17 files changed, 78 insertions(+), 144 deletions(-)
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!
From: James Morse <[email protected]>
register_cpu_capacity_sysctl() adds a property to sysfs that describes
the CPUs capacity. This is done from a subsys_initcall() that assumes
all possible CPUs are registered.
With CPU hotplug, possible CPUs aren't registered until they become
present, (or for arm64 enabled). This leads to messages during boot:
| register_cpu_capacity_sysctl: too early to get CPU1 device!
and once these CPUs are added to the system, the file is missing.
Move this to a cpuhp callback, so that the file is created once
CPUs are brought online. This covers CPUs that are added late by
mechanisms like hotplug.
One observable difference is the file is now missing for offline CPUs.
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
If the offline CPUs thing is a problem for the tools that consume
this value, we'd need to move cpu_capacity to be part of cpu.c's
common_cpu_attr_groups. However, attempts to discuss this just end
up in a black hole, so this is a non-starter. Thus, if this needs
to be done, it can be done as a separate patch.
---
drivers/base/arch_topology.c | 38 ++++++++++++++++++++++++------------
1 file changed, 26 insertions(+), 12 deletions(-)
diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index b741b5ba82bd..9ccb7daee78e 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -220,20 +220,34 @@ static DECLARE_WORK(update_topology_flags_work, update_topology_flags_workfn);
static DEVICE_ATTR_RO(cpu_capacity);
-static int register_cpu_capacity_sysctl(void)
+static int cpu_capacity_sysctl_add(unsigned int cpu)
{
- int i;
- struct device *cpu;
+ struct device *cpu_dev = get_cpu_device(cpu);
- for_each_possible_cpu(i) {
- cpu = get_cpu_device(i);
- if (!cpu) {
- pr_err("%s: too early to get CPU%d device!\n",
- __func__, i);
- continue;
- }
- device_create_file(cpu, &dev_attr_cpu_capacity);
- }
+ if (!cpu_dev)
+ return -ENOENT;
+
+ device_create_file(cpu_dev, &dev_attr_cpu_capacity);
+
+ return 0;
+}
+
+static int cpu_capacity_sysctl_remove(unsigned int cpu)
+{
+ struct device *cpu_dev = get_cpu_device(cpu);
+
+ if (!cpu_dev)
+ return -ENOENT;
+
+ device_remove_file(cpu_dev, &dev_attr_cpu_capacity);
+
+ return 0;
+}
+
+static int register_cpu_capacity_sysctl(void)
+{
+ cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "topology/cpu-capacity",
+ cpu_capacity_sysctl_add, cpu_capacity_sysctl_remove);
return 0;
}
--
2.30.2
arch_register_cpu() and arch_unregister_cpu() are not used by anything
that can be a module - they are used by drivers/base/cpu.c and
drivers/acpi/acpi_processor.c, neither of which can be a module.
Remove the exports.
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
arch/x86/kernel/topology.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/arch/x86/kernel/topology.c b/arch/x86/kernel/topology.c
index 0bab03130033..fcb62cfdf946 100644
--- a/arch/x86/kernel/topology.c
+++ b/arch/x86/kernel/topology.c
@@ -45,13 +45,11 @@ int arch_register_cpu(int cpu)
xc->cpu.hotpluggable = cpu > 0;
return register_cpu(&xc->cpu, cpu);
}
-EXPORT_SYMBOL(arch_register_cpu);
void arch_unregister_cpu(int num)
{
unregister_cpu(&per_cpu(cpu_devices, num).cpu);
}
-EXPORT_SYMBOL(arch_unregister_cpu);
#else /* CONFIG_HOTPLUG_CPU */
int __init arch_register_cpu(int num)
--
2.30.2
arch_register_cpu() and arch_unregister_cpu() are not used by anything
that can be a module - they are used by drivers/base/cpu.c and
drivers/acpi/acpi_processor.c, neither of which can be a module.
Remove the exports.
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
arch/loongarch/kernel/topology.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/arch/loongarch/kernel/topology.c b/arch/loongarch/kernel/topology.c
index 3fd166006698..ae860fe81536 100644
--- a/arch/loongarch/kernel/topology.c
+++ b/arch/loongarch/kernel/topology.c
@@ -25,7 +25,6 @@ int arch_register_cpu(int cpu)
return ret;
}
-EXPORT_SYMBOL(arch_register_cpu);
void arch_unregister_cpu(int cpu)
{
@@ -34,7 +33,6 @@ void arch_unregister_cpu(int cpu)
c->hotpluggable = 0;
unregister_cpu(c);
}
-EXPORT_SYMBOL(arch_unregister_cpu);
#endif
static int __init topology_init(void)
--
2.30.2
From: James Morse <[email protected]>
intel_epb_init() is called as a subsys_initcall() to register cpuhp
callbacks. The callbacks make use of get_cpu_device() which will return
NULL unless register_cpu() has been called. register_cpu() is called
from topology_init(), which is also a subsys_initcall().
This is fragile. Moving the register_cpu() to a different
subsys_initcall() leads to a NULL dereference during boot.
Make intel_epb_init() a late_initcall(), user-space can't provide a
policy before this point anyway.
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
subsys_initcall_sync() would be an option, but moving the register_cpu()
calls into ACPI also means adding a safety net for CPUs that are online
but not described properly by firmware. This lives in subsys_initcall_sync().
---
arch/x86/kernel/cpu/intel_epb.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kernel/cpu/intel_epb.c b/arch/x86/kernel/cpu/intel_epb.c
index e4c3ba91321c..f18d35fe27a9 100644
--- a/arch/x86/kernel/cpu/intel_epb.c
+++ b/arch/x86/kernel/cpu/intel_epb.c
@@ -237,4 +237,4 @@ static __init int intel_epb_init(void)
cpuhp_remove_state(CPUHP_AP_X86_INTEL_EPB_ONLINE);
return ret;
}
-subsys_initcall(intel_epb_init);
+late_initcall(intel_epb_init);
--
2.30.2
From: James Morse <[email protected]>
NUMA systems require the node descriptions to be ready before CPUs are
registered. This is so that the node symlinks can be created in sysfs.
Currently no NUMA platform uses GENERIC_CPU_DEVICES, meaning that CPUs
are registered by arch code, instead of cpu_dev_init().
Move cpu_dev_init() after node_dev_init() so that NUMA architectures
can use GENERIC_CPU_DEVICES.
Signed-off-by: James Morse <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
Note: Jonathan's comment still needs addressing - see
https://lore.kernel.org/r/[email protected]
---
drivers/base/init.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/base/init.c b/drivers/base/init.c
index 397eb9880cec..c4954835128c 100644
--- a/drivers/base/init.c
+++ b/drivers/base/init.c
@@ -35,8 +35,8 @@ void __init driver_init(void)
of_core_init();
platform_bus_init();
auxiliary_bus_init();
- cpu_dev_init();
memory_dev_init();
node_dev_init();
+ cpu_dev_init();
container_dev_init();
}
--
2.30.2
From: James Morse <[email protected]>
Add arch_unregister_cpu() to allow the ACPI machinery to call
unregister_cpu(). This is enough for arm64, riscv and loongarch, but
needs to be overridden by x86 and ia64 who need to do more work.
CC: Jean-Philippe Brucker <[email protected]>
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Shaoqin Huang <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
An open question remains from the RFC v2 posting: should we provide a
__weak stub for !HOTPLUG_CPU as well, since in later patches ACPI may
reference this if the compiler doesn't optimise as we expect?
Changes since v1:
* Added CONFIG_HOTPLUG_CPU ifdeffery around unregister_cpu
Changes since RFC v2:
* Move earlier in the series
---
drivers/base/cpu.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index 579064fda97b..58bb86091b34 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -531,7 +531,14 @@ int __weak arch_register_cpu(int cpu)
{
return register_cpu(&per_cpu(cpu_devices, cpu), cpu);
}
-#endif
+
+#ifdef CONFIG_HOTPLUG_CPU
+void __weak arch_unregister_cpu(int num)
+{
+ unregister_cpu(&per_cpu(cpu_devices, num));
+}
+#endif /* CONFIG_HOTPLUG_CPU */
+#endif /* CONFIG_GENERIC_CPU_DEVICES */
static void __init cpu_dev_register_generic(void)
{
--
2.30.2
From: James Morse <[email protected]>
Architectures often have extra per-cpu work that needs doing
before a CPU is registered, often to determine if a CPU is
hotpluggable.
To allow the ACPI architectures to use GENERIC_CPU_DEVICES, move
the cpu_register() call into arch_register_cpu(), which is made __weak
so architectures with extra work can override it.
This aligns with the way x86, ia64 and loongarch register hotplug CPUs
when they become present.
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Shaoqin Huang <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
Changes since RFC:
* Dropped __init from x86/ia64 arch_register_cpu()
Changes since RFC v2:
* Dropped unnecessary Loongarch asm/cpu.h changes
---
drivers/base/cpu.c | 14 ++++++++++----
include/linux/cpu.h | 4 ++++
2 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index 34b48f660b6b..579064fda97b 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -525,19 +525,25 @@ bool cpu_is_hotpluggable(unsigned int cpu)
EXPORT_SYMBOL_GPL(cpu_is_hotpluggable);
#ifdef CONFIG_GENERIC_CPU_DEVICES
-static DEFINE_PER_CPU(struct cpu, cpu_devices);
+DEFINE_PER_CPU(struct cpu, cpu_devices);
+
+int __weak arch_register_cpu(int cpu)
+{
+ return register_cpu(&per_cpu(cpu_devices, cpu), cpu);
+}
#endif
static void __init cpu_dev_register_generic(void)
{
-#ifdef CONFIG_GENERIC_CPU_DEVICES
int i;
+ if (!IS_ENABLED(CONFIG_GENERIC_CPU_DEVICES))
+ return;
+
for_each_present_cpu(i) {
- if (register_cpu(&per_cpu(cpu_devices, i), i))
+ if (arch_register_cpu(i))
panic("Failed to register CPU device");
}
-#endif
}
#ifdef CONFIG_GENERIC_CPU_VULNERABILITIES
diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index fc8094419084..1e982d63eae8 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -88,6 +88,10 @@ extern ssize_t arch_cpu_probe(const char *, size_t);
extern ssize_t arch_cpu_release(const char *, size_t);
#endif
+#ifdef CONFIG_GENERIC_CPU_DEVICES
+DECLARE_PER_CPU(struct cpu, cpu_devices);
+#endif
+
/*
* These states are not related to the core CPU hotplug mechanism. They are
* used by various (sub)architectures to track internal state
--
2.30.2
From: James Morse <[email protected]>
To allow ACPI's _STA value to hide CPUs that are present, but not
available to online right now due to VMM or firmware policy, the
register_cpu() call needs to be made by the ACPI machinery when ACPI
is in use. This allows it to hide CPUs that are unavailable from sysfs.
Switching to GENERIC_CPU_DEVICES is an intermediate step to allow all
five ACPI architectures to be modified at once.
Switch over to GENERIC_CPU_DEVICES, and provide an arch_register_cpu()
that populates the hotpluggable flag. arch_register_cpu() is also the
interface the ACPI machinery expects.
The struct cpu in struct cpuinfo_arm64 is never used directly, remove
it to use the one GENERIC_CPU_DEVICES provides.
This changes the CPUs visible in sysfs from possible to present, but
on arm64 smp_prepare_cpus() ensures these are the same.
This patch also has the effect of moving the registration of CPUs from
subsys to driver core initialisation, prior to any initcalls running.
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Shaoqin Huang <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
Changes since RFC v2:
* Add note about initialisation order change.
---
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/cpu.h | 1 -
arch/arm64/kernel/setup.c | 13 ++++---------
3 files changed, 5 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 7b071a00425d..84bce830e365 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -132,6 +132,7 @@ config ARM64
select GENERIC_ARCH_TOPOLOGY
select GENERIC_CLOCKEVENTS_BROADCAST
select GENERIC_CPU_AUTOPROBE
+ select GENERIC_CPU_DEVICES
select GENERIC_CPU_VULNERABILITIES
select GENERIC_EARLY_IOREMAP
select GENERIC_IDLE_POLL_SETUP
diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
index f3034099fd95..b1e43f56ee46 100644
--- a/arch/arm64/include/asm/cpu.h
+++ b/arch/arm64/include/asm/cpu.h
@@ -38,7 +38,6 @@ struct cpuinfo_32bit {
};
struct cpuinfo_arm64 {
- struct cpu cpu;
struct kobject kobj;
u64 reg_ctr;
u64 reg_cntfrq;
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index 417a8a86b2db..165bd2c0dd5a 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -402,19 +402,14 @@ static inline bool cpu_can_disable(unsigned int cpu)
return false;
}
-static int __init topology_init(void)
+int arch_register_cpu(int num)
{
- int i;
+ struct cpu *cpu = &per_cpu(cpu_devices, num);
- for_each_possible_cpu(i) {
- struct cpu *cpu = &per_cpu(cpu_data.cpu, i);
- cpu->hotpluggable = cpu_can_disable(i);
- register_cpu(cpu, i);
- }
+ cpu->hotpluggable = cpu_can_disable(num);
- return 0;
+ return register_cpu(cpu, num);
}
-subsys_initcall(topology_init);
static void dump_kernel_offset(void)
{
--
2.30.2
From: James Morse <[email protected]>
Three of the five ACPI architectures create sysfs entries using
register_cpu() for present CPUs, whereas arm64, riscv and all
GENERIC_CPU_DEVICES do this for possible CPUs.
Registering a CPU is what causes them to show up in sysfs.
It makes very little sense to register all possible CPUs. Registering
a CPU is what triggers the udev notifications allowing user-space to
react to newly added CPUs.
To allow all five ACPI architectures to use GENERIC_CPU_DEVICES, change
it to use for_each_present_cpu().
Making the ACPI architectures use GENERIC_CPU_DEVICES is a pre-requisite
step to centralise their register_cpu() logic, before moving it into the
ACPI processor driver. When we add support for register CPUs from ACPI
in a later patch, we will avoid registering CPUs in this path.
Of the ACPI architectures that register possible CPUs, arm64 and riscv
do not support making possible CPUs present as they use the weak 'always
fails' version of arch_register_cpu().
Only two of the eight architectures that use GENERIC_CPU_DEVICES have a
distinction between present and possible CPUs.
The following architectures use GENERIC_CPU_DEVICES but are not SMP,
so possible == present:
* m68k
* microblaze
* nios2
The following architectures use GENERIC_CPU_DEVICES and consider
possible == present:
* csky: setup_smp()
* processor_probe() sets possible for all CPUs and present for all CPUs
except the boot cpu, which will have been done by
init/main.c::start_kernel().
um appears to be a subarchitecture of x86.
The remaining architecture using GENERIC_CPU_DEVICES are:
* openrisc and hexagon:
where smp_init_cpus() makes all CPUs < NR_CPUS possible,
whereas smp_prepare_cpus() only makes CPUs < setup_max_cpus present.
After this change, openrisc and hexagon systems that use the max_cpus
command line argument would not see the other CPUs present in sysfs.
This should not be a problem as these CPUs can't be brought online as
_cpu_up() checks cpu_present().
After this change, only CPUs which are present appear in sysfs.
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
drivers/base/cpu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index 9ea22e165acd..34b48f660b6b 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -533,7 +533,7 @@ static void __init cpu_dev_register_generic(void)
#ifdef CONFIG_GENERIC_CPU_DEVICES
int i;
- for_each_possible_cpu(i) {
+ for_each_present_cpu(i) {
if (register_cpu(&per_cpu(cpu_devices, i), i))
panic("Failed to register CPU device");
}
--
2.30.2
From: James Morse <[email protected]>
Now that GENERIC_CPU_DEVICES calls arch_register_cpu(), which can be
overridden by the arch code, switch over to this to allow common code
to choose when the register_cpu() call is made.
This allows topology_init() to be removed.
This is an intermediate step to the logic being moved to drivers/acpi,
where GENERIC_CPU_DEVICES will do the work when booting with acpi=off.
This is a subtle change. Originally:
- on boot, topology_init() would have marked present CPUs that
io_master() is true for as hotplug-incapable.
- if a CPU is hotplugged that is an io_master(), it can later be
hot-unplugged.
The new behaviour is that any CPU that io_master() is true for will
now always be marked as hotplug-incapable, thus even if it was
hotplugged, it can no longer be hot-unplugged.
This patch also has the effect of moving the registration of CPUs from
subsys to driver core initialisation, prior to any initcalls running.
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
Changes since RFC v2:
* Explain the change in behaviour in the patch description
(highlighted by Jonathan Cameron - thanks.) Add note about
initialisation order change.
---
arch/loongarch/Kconfig | 1 +
arch/loongarch/kernel/topology.c | 29 ++---------------------------
2 files changed, 3 insertions(+), 27 deletions(-)
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index 331becb2cb4f..15d05dd2b7f3 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -72,6 +72,7 @@ config LOONGARCH
select GENERIC_CLOCKEVENTS
select GENERIC_CMOS_UPDATE
select GENERIC_CPU_AUTOPROBE
+ select GENERIC_CPU_DEVICES
select GENERIC_ENTRY
select GENERIC_GETTIMEOFDAY
select GENERIC_IOREMAP if !ARCH_IOREMAP
diff --git a/arch/loongarch/kernel/topology.c b/arch/loongarch/kernel/topology.c
index ae860fe81536..7dfb46c68f58 100644
--- a/arch/loongarch/kernel/topology.c
+++ b/arch/loongarch/kernel/topology.c
@@ -10,20 +10,13 @@
#include <acpi/processor.h>
-static DEFINE_PER_CPU(struct cpu, cpu_devices);
-
#ifdef CONFIG_HOTPLUG_CPU
int arch_register_cpu(int cpu)
{
- int ret;
struct cpu *c = &per_cpu(cpu_devices, cpu);
- c->hotpluggable = 1;
- ret = register_cpu(c, cpu);
- if (ret < 0)
- pr_warn("register_cpu %d failed (%d)\n", cpu, ret);
-
- return ret;
+ c->hotpluggable = !io_master(cpu);
+ return register_cpu(c, cpu);
}
void arch_unregister_cpu(int cpu)
@@ -34,21 +27,3 @@ void arch_unregister_cpu(int cpu)
unregister_cpu(c);
}
#endif
-
-static int __init topology_init(void)
-{
- int i, ret;
-
- for_each_present_cpu(i) {
- struct cpu *c = &per_cpu(cpu_devices, i);
-
- c->hotpluggable = !io_master(i);
- ret = register_cpu(c, i);
- if (ret < 0)
- pr_warn("topology_init: register_cpu %d failed (%d)\n", i, ret);
- }
-
- return 0;
-}
-
-subsys_initcall(topology_init);
--
2.30.2
From: James Morse <[email protected]>
Neither arm64 nor riscv support physical hotadd of CPUs that were not
present at boot. For arm64 much of the platform description is in static
tables which do not have update methods. arm64 does support HOTPLUG_CPU,
which is backed by a firmware interface to turn CPUs on and off.
acpi_processor_hotadd_init() and acpi_processor_remove() are for adding
and removing CPUs that were not present at boot. arm64 systems that do this
are not supported as there is currently insufficient information in the
platform description. (e.g. did the GICR get removed too?)
arm64 currently relies on the MADT enabled flag check in map_gicc_mpidr()
to prevent CPUs that were not described as present at boot from being
added to the system. Similarly, riscv relies on the same check in
map_rintc_hartid(). Both architectures also rely on the weak 'always fails'
definitions of acpi_map_cpu() and arch_register_cpu().
Subsequent changes will redefine ACPI_HOTPLUG_CPU as making possible
CPUs present. Neither arm64 nor riscv support this.
Disable ACPI_HOTPLUG_CPU for arm64 and riscv by removing 'default y' and
selecting it on the other three ACPI architectures. This allows the weak
definitions of some symbols to be removed.
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Shaoqin Huang <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
Changes since RFC:
* Expanded conditions to avoid ACPI_HOTPLUG_CPU being enabled when
HOTPLUG_CPU isn't.
Changes since RFC v3:
* Dropped ia64 changes
---
arch/loongarch/Kconfig | 1 +
arch/x86/Kconfig | 1 +
drivers/acpi/Kconfig | 1 -
drivers/acpi/acpi_processor.c | 18 ------------------
4 files changed, 2 insertions(+), 19 deletions(-)
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index ee123820a476..331becb2cb4f 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -5,6 +5,7 @@ config LOONGARCH
select ACPI
select ACPI_GENERIC_GSI if ACPI
select ACPI_MCFG if ACPI
+ select ACPI_HOTPLUG_CPU if ACPI_PROCESSOR && HOTPLUG_CPU
select ACPI_PPTT if ACPI
select ACPI_SYSTEM_POWER_STATES_SUPPORT if ACPI
select ARCH_BINFMT_ELF_STATE
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 3762f41bb092..dbdcfc708369 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -59,6 +59,7 @@ config X86
#
select ACPI_LEGACY_TABLES_LOOKUP if ACPI
select ACPI_SYSTEM_POWER_STATES_SUPPORT if ACPI
+ select ACPI_HOTPLUG_CPU if ACPI_PROCESSOR && HOTPLUG_CPU
select ARCH_32BIT_OFF_T if X86_32
select ARCH_CLOCKSOURCE_INIT
select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index f819e760ff19..a3acfc750fce 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -310,7 +310,6 @@ config ACPI_HOTPLUG_CPU
bool
depends on ACPI_PROCESSOR && HOTPLUG_CPU
select ACPI_CONTAINER
- default y
config ACPI_PROCESSOR_AGGREGATOR
tristate "Processor Aggregator"
diff --git a/drivers/acpi/acpi_processor.c b/drivers/acpi/acpi_processor.c
index 0f5218e361df..4fe2ef54088c 100644
--- a/drivers/acpi/acpi_processor.c
+++ b/drivers/acpi/acpi_processor.c
@@ -184,24 +184,6 @@ static void __init acpi_pcc_cpufreq_init(void) {}
/* Initialization */
#ifdef CONFIG_ACPI_HOTPLUG_CPU
-int __weak acpi_map_cpu(acpi_handle handle,
- phys_cpuid_t physid, u32 acpi_id, int *pcpu)
-{
- return -ENODEV;
-}
-
-int __weak acpi_unmap_cpu(int cpu)
-{
- return -ENODEV;
-}
-
-int __weak arch_register_cpu(int cpu)
-{
- return -ENODEV;
-}
-
-void __weak arch_unregister_cpu(int cpu) {}
-
static int acpi_processor_hotadd_init(struct acpi_processor *pr)
{
unsigned long long sta;
--
2.30.2
Since the x86 version of arch_unregister_cpu() is the same as the weak
version, drop the x86 specific version.
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
Changes since RFC v3:
* Adapt to removal of EXPORT_SYMBOL()s
---
arch/x86/kernel/topology.c | 5 -----
1 file changed, 5 deletions(-)
diff --git a/arch/x86/kernel/topology.c b/arch/x86/kernel/topology.c
index c2ed3145a93b..211863cb5b81 100644
--- a/arch/x86/kernel/topology.c
+++ b/arch/x86/kernel/topology.c
@@ -43,9 +43,4 @@ int arch_register_cpu(int cpu)
c->hotpluggable = cpu > 0;
return register_cpu(c, cpu);
}
-
-void arch_unregister_cpu(int num)
-{
- unregister_cpu(&per_cpu(cpu_devices, num));
-}
#endif /* CONFIG_HOTPLUG_CPU */
--
2.30.2
Convert loongarch to use the arch_cpu_is_hotpluggable() helper rather
than arch_register_cpu(). Also remove the export as nothing should be
using arch_register_cpu() outside of the core kernel/acpi code.
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
arch/loongarch/kernel/topology.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/arch/loongarch/kernel/topology.c b/arch/loongarch/kernel/topology.c
index 866c2c9ef6ab..75d5c51a7cd3 100644
--- a/arch/loongarch/kernel/topology.c
+++ b/arch/loongarch/kernel/topology.c
@@ -11,11 +11,8 @@
#include <acpi/processor.h>
#ifdef CONFIG_HOTPLUG_CPU
-int arch_register_cpu(int cpu)
+bool arch_cpu_is_hotpluggable(int cpu)
{
- struct cpu *c = &per_cpu(cpu_devices, cpu);
-
- c->hotpluggable = !io_master(cpu);
- return register_cpu(c, cpu);
+ return !io_master(cpu);
}
#endif
--
2.30.2
From: James Morse <[email protected]>
Now that GENERIC_CPU_DEVICES calls arch_register_cpu(), which can be
overridden by the arch code, switch over to this to allow common code
to choose when the register_cpu() call is made.
This allows topology_init() to be removed.
This is an intermediate step to the logic being moved to drivers/acpi,
where GENERIC_CPU_DEVICES will do the work when booting with acpi=off.
This patch also has the effect of moving the registration of CPUs from
subsys to driver core initialisation, prior to any initcalls running.
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Acked-by: Palmer Dabbelt <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
Changes since RFC v2:
* Add note about initialisation order change.
---
arch/riscv/Kconfig | 1 +
arch/riscv/kernel/setup.c | 19 ++++---------------
2 files changed, 5 insertions(+), 15 deletions(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 95a2a06acc6a..162425cb9739 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -71,6 +71,7 @@ config RISCV
select GENERIC_ARCH_TOPOLOGY
select GENERIC_ATOMIC64 if !64BIT
select GENERIC_CLOCKEVENTS_BROADCAST if SMP
+ select GENERIC_CPU_DEVICES
select GENERIC_EARLY_IOREMAP
select GENERIC_ENTRY
select GENERIC_GETTIMEOFDAY if HAVE_GENERIC_VDSO
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 535a837de55d..b3a0aa2b78d5 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -51,7 +51,6 @@ atomic_t hart_lottery __section(".sdata")
#endif
;
unsigned long boot_cpu_hartid;
-static DEFINE_PER_CPU(struct cpu, cpu_devices);
/*
* Place kernel memory regions on the resource tree so that
@@ -299,23 +298,13 @@ void __init setup_arch(char **cmdline_p)
riscv_user_isa_enable();
}
-static int __init topology_init(void)
+int arch_register_cpu(int cpu)
{
- int i, ret;
+ struct cpu *c = &per_cpu(cpu_devices, cpu);
- for_each_possible_cpu(i) {
- struct cpu *cpu = &per_cpu(cpu_devices, i);
-
- cpu->hotpluggable = cpu_has_hotplug(i);
- ret = register_cpu(cpu, i);
- if (unlikely(ret))
- pr_warn("Warning: %s: register_cpu %d failed (%d)\n",
- __func__, i, ret);
- }
-
- return 0;
+ c->hotpluggable = cpu_has_hotplug(cpu);
+ return register_cpu(c, cpu);
}
-subsys_initcall(topology_init);
void free_initmem(void)
{
--
2.30.2
The differences between architecture specific implementations of
arch_register_cpu() are down to whether the CPU is hotpluggable or not.
Rather than overriding the weak version of arch_register_cpu(), provide
a function that can be used to provide this detail instead.
Reviewed-by: Shaoqin Huang <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
drivers/base/cpu.c | 11 ++++++++++-
include/linux/cpu.h | 1 +
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index 58bb86091b34..221ffbeb1c9b 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -527,9 +527,18 @@ EXPORT_SYMBOL_GPL(cpu_is_hotpluggable);
#ifdef CONFIG_GENERIC_CPU_DEVICES
DEFINE_PER_CPU(struct cpu, cpu_devices);
+bool __weak arch_cpu_is_hotpluggable(int cpu)
+{
+ return false;
+}
+
int __weak arch_register_cpu(int cpu)
{
- return register_cpu(&per_cpu(cpu_devices, cpu), cpu);
+ struct cpu *c = &per_cpu(cpu_devices, cpu);
+
+ c->hotpluggable = arch_cpu_is_hotpluggable(cpu);
+
+ return register_cpu(c, cpu);
}
#ifdef CONFIG_HOTPLUG_CPU
diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index 1e982d63eae8..dcb89c987164 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -80,6 +80,7 @@ extern __printf(4, 5)
struct device *cpu_device_create(struct device *parent, void *drvdata,
const struct attribute_group **groups,
const char *fmt, ...);
+extern bool arch_cpu_is_hotpluggable(int cpu);
extern int arch_register_cpu(int cpu);
extern void arch_unregister_cpu(int cpu);
#ifdef CONFIG_HOTPLUG_CPU
--
2.30.2
From: James Morse <[email protected]>
loongarch, mips, parisc, riscv and sh all print a warning if
register_cpu() returns an error. Architectures that use
GENERIC_CPU_DEVICES call panic() instead.
Errors in this path indicate something is wrong with the firmware
description of the platform, but the kernel is able to keep running.
Downgrade this to a warning to make it easier to debug this issue.
This will allow architectures that switching over to GENERIC_CPU_DEVICES
to drop their warning, but keep the existing behaviour.
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Shaoqin Huang <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
drivers/base/cpu.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index 221ffbeb1c9b..82b6a76125f5 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -551,14 +551,15 @@ void __weak arch_unregister_cpu(int num)
static void __init cpu_dev_register_generic(void)
{
- int i;
+ int i, ret;
if (!IS_ENABLED(CONFIG_GENERIC_CPU_DEVICES))
return;
for_each_present_cpu(i) {
- if (arch_register_cpu(i))
- panic("Failed to register CPU device");
+ ret = arch_register_cpu(i);
+ if (ret)
+ pr_warn("register_cpu %d failed (%d)\n", i, ret);
}
}
--
2.30.2
Convert arm64 to use the arch_cpu_is_hotpluggable() helper rather than
arch_register_cpu().
Reviewed-by: Shaoqin Huang <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
arch/arm64/kernel/setup.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index 165bd2c0dd5a..42c690bb2d60 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -402,13 +402,9 @@ static inline bool cpu_can_disable(unsigned int cpu)
return false;
}
-int arch_register_cpu(int num)
+bool arch_cpu_is_hotpluggable(int num)
{
- struct cpu *cpu = &per_cpu(cpu_devices, num);
-
- cpu->hotpluggable = cpu_can_disable(num);
-
- return register_cpu(cpu, num);
+ return cpu_can_disable(num);
}
static void dump_kernel_offset(void)
--
2.30.2
From: James Morse <[email protected]>
Now that GENERIC_CPU_DEVICES calls arch_register_cpu(), which can be
overridden by the arch code, switch over to this to allow common code
to choose when the register_cpu() call is made.
x86's struct cpus come from struct x86_cpu, which has no other members
or users. Remove this and use the version defined by common code.
This is an intermediate step to the logic being moved to drivers/acpi,
where GENERIC_CPU_DEVICES will do the work when booting with acpi=off.
This patch also has the effect of moving the registration of CPUs from
subsys to driver core initialisation, prior to any initcalls running.
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
----
Changes since RFC:
* Fixed the second copy of arch_register_cpu() used for non-hotplug
Changes since RFC v2:
* Remove duplicate of the weak generic arch_register_cpu(), spotted
by Jonathan Cameron. Add note about initialisation order change.
Changes since RFC v3:
* Adapt to removal of EXPORT_SYMBOL()s
---
arch/x86/Kconfig | 1 +
arch/x86/include/asm/cpu.h | 4 ----
arch/x86/kernel/topology.c | 27 ++++-----------------------
3 files changed, 5 insertions(+), 27 deletions(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index dbdcfc708369..8330c4ac26b3 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -148,6 +148,7 @@ config X86
select GENERIC_CLOCKEVENTS_MIN_ADJUST
select GENERIC_CMOS_UPDATE
select GENERIC_CPU_AUTOPROBE
+ select GENERIC_CPU_DEVICES
select GENERIC_CPU_VULNERABILITIES
select GENERIC_EARLY_IOREMAP
select GENERIC_ENTRY
diff --git a/arch/x86/include/asm/cpu.h b/arch/x86/include/asm/cpu.h
index fecc4fe1d68a..f8f9a9b79395 100644
--- a/arch/x86/include/asm/cpu.h
+++ b/arch/x86/include/asm/cpu.h
@@ -23,10 +23,6 @@ static inline void prefill_possible_map(void) {}
#endif /* CONFIG_SMP */
-struct x86_cpu {
- struct cpu cpu;
-};
-
#ifdef CONFIG_HOTPLUG_CPU
extern void soft_restart_cpu(void);
#endif
diff --git a/arch/x86/kernel/topology.c b/arch/x86/kernel/topology.c
index fcb62cfdf946..c2ed3145a93b 100644
--- a/arch/x86/kernel/topology.c
+++ b/arch/x86/kernel/topology.c
@@ -35,36 +35,17 @@
#include <asm/io_apic.h>
#include <asm/cpu.h>
-static DEFINE_PER_CPU(struct x86_cpu, cpu_devices);
-
#ifdef CONFIG_HOTPLUG_CPU
int arch_register_cpu(int cpu)
{
- struct x86_cpu *xc = per_cpu_ptr(&cpu_devices, cpu);
+ struct cpu *c = per_cpu_ptr(&cpu_devices, cpu);
- xc->cpu.hotpluggable = cpu > 0;
- return register_cpu(&xc->cpu, cpu);
+ c->hotpluggable = cpu > 0;
+ return register_cpu(c, cpu);
}
void arch_unregister_cpu(int num)
{
- unregister_cpu(&per_cpu(cpu_devices, num).cpu);
-}
-#else /* CONFIG_HOTPLUG_CPU */
-
-int __init arch_register_cpu(int num)
-{
- return register_cpu(&per_cpu(cpu_devices, num).cpu, num);
+ unregister_cpu(&per_cpu(cpu_devices, num));
}
#endif /* CONFIG_HOTPLUG_CPU */
-
-static int __init topology_init(void)
-{
- int i;
-
- for_each_present_cpu(i)
- arch_register_cpu(i);
-
- return 0;
-}
-subsys_initcall(topology_init);
--
2.30.2
Convert x86 to use the arch_cpu_is_hotpluggable() helper rather than
arch_register_cpu().
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
arch/x86/kernel/topology.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/arch/x86/kernel/topology.c b/arch/x86/kernel/topology.c
index 211863cb5b81..d42c28b8bfd8 100644
--- a/arch/x86/kernel/topology.c
+++ b/arch/x86/kernel/topology.c
@@ -36,11 +36,8 @@
#include <asm/cpu.h>
#ifdef CONFIG_HOTPLUG_CPU
-int arch_register_cpu(int cpu)
+bool arch_cpu_is_hotpluggable(int cpu)
{
- struct cpu *c = per_cpu_ptr(&cpu_devices, cpu);
-
- c->hotpluggable = cpu > 0;
- return register_cpu(c, cpu);
+ return cpu > 0;
}
#endif /* CONFIG_HOTPLUG_CPU */
--
2.30.2
From: James Morse <[email protected]>
LoongArch provides its own arch_unregister_cpu(). This clears the
hotpluggable flag, then unregisters the CPU.
It isn't necessary to clear the hotpluggable flag when unregistering
a cpu. unregister_cpu() writes NULL to the percpu cpu_sys_devices
pointer, meaning cpu_is_hotpluggable() will return false, as
get_cpu_device() has returned NULL.
Remove arch_unregister_cpu() and use the __weak version.
Signed-off-by: James Morse <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
Changes since RFC v3:
* Adapt for removal of EXPORT_SYMBOL()s
---
arch/loongarch/kernel/topology.c | 8 --------
1 file changed, 8 deletions(-)
diff --git a/arch/loongarch/kernel/topology.c b/arch/loongarch/kernel/topology.c
index 7dfb46c68f58..866c2c9ef6ab 100644
--- a/arch/loongarch/kernel/topology.c
+++ b/arch/loongarch/kernel/topology.c
@@ -18,12 +18,4 @@ int arch_register_cpu(int cpu)
c->hotpluggable = !io_master(cpu);
return register_cpu(c, cpu);
}
-
-void arch_unregister_cpu(int cpu)
-{
- struct cpu *c = &per_cpu(cpu_devices, cpu);
-
- c->hotpluggable = 0;
- unregister_cpu(c);
-}
#endif
--
2.30.2
Convert riscv to use the arch_cpu_is_hotpluggable() helper rather than
arch_register_cpu().
Acked-by: Palmer Dabbelt <[email protected]>
Reviewed-by: Gavin Shan <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>
---
arch/riscv/kernel/setup.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index b3a0aa2b78d5..7493fafbe4cb 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -298,12 +298,9 @@ void __init setup_arch(char **cmdline_p)
riscv_user_isa_enable();
}
-int arch_register_cpu(int cpu)
+bool arch_cpu_is_hotpluggable(int cpu)
{
- struct cpu *c = &per_cpu(cpu_devices, cpu);
-
- c->hotpluggable = cpu_has_hotplug(cpu);
- return register_cpu(c, cpu);
+ return cpu_has_hotplug(cpu);
}
void free_initmem(void)
--
2.30.2
On 2023-11-21 7:45 AM, Russell King (Oracle) wrote:
> From: James Morse <[email protected]>
>
> Now that GENERIC_CPU_DEVICES calls arch_register_cpu(), which can be
> overridden by the arch code, switch over to this to allow common code
> to choose when the register_cpu() call is made.
>
> This allows topology_init() to be removed.
>
> This is an intermediate step to the logic being moved to drivers/acpi,
> where GENERIC_CPU_DEVICES will do the work when booting with acpi=off.
>
> This patch also has the effect of moving the registration of CPUs from
> subsys to driver core initialisation, prior to any initcalls running.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Jonathan Cameron <[email protected]>
> Acked-by: Palmer Dabbelt <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
> ---
> Changes since RFC v2:
> * Add note about initialisation order change.
> ---
> arch/riscv/Kconfig | 1 +
> arch/riscv/kernel/setup.c | 19 ++++---------------
> 2 files changed, 5 insertions(+), 15 deletions(-)
Reviewed-by: Samuel Holland <[email protected]>
Tested-by: Samuel Holland <[email protected]>
On 2023-11-21 7:45 AM, Russell King (Oracle) wrote:
> Convert riscv to use the arch_cpu_is_hotpluggable() helper rather than
> arch_register_cpu().
>
> Acked-by: Palmer Dabbelt <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
> ---
> arch/riscv/kernel/setup.c | 7 ++-----
> 1 file changed, 2 insertions(+), 5 deletions(-)
Reviewed-by: Samuel Holland <[email protected]>
Tested-by: Samuel Holland <[email protected]> # On HiFive Unmatched
On Tue, Nov 21, 2023 at 2:44 PM Russell King <[email protected]> wrote:
>
> From: James Morse <[email protected]>
>
> intel_epb_init() is called as a subsys_initcall() to register cpuhp
> callbacks. The callbacks make use of get_cpu_device() which will return
> NULL unless register_cpu() has been called. register_cpu() is called
> from topology_init(), which is also a subsys_initcall().
>
> This is fragile. Moving the register_cpu() to a different
> subsys_initcall() leads to a NULL dereference during boot.
>
> Make intel_epb_init() a late_initcall(), user-space can't provide a
> policy before this point anyway.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Acked-by: Rafael J. Wysocki <[email protected]>
and I'd suggest sending this separately to the x86 list.
> ---
> subsys_initcall_sync() would be an option, but moving the register_cpu()
> calls into ACPI also means adding a safety net for CPUs that are online
> but not described properly by firmware. This lives in subsys_initcall_sync().
> ---
> arch/x86/kernel/cpu/intel_epb.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/x86/kernel/cpu/intel_epb.c b/arch/x86/kernel/cpu/intel_epb.c
> index e4c3ba91321c..f18d35fe27a9 100644
> --- a/arch/x86/kernel/cpu/intel_epb.c
> +++ b/arch/x86/kernel/cpu/intel_epb.c
> @@ -237,4 +237,4 @@ static __init int intel_epb_init(void)
> cpuhp_remove_state(CPUHP_AP_X86_INTEL_EPB_ONLINE);
> return ret;
> }
> -subsys_initcall(intel_epb_init);
> +late_initcall(intel_epb_init);
> --
> 2.30.2
>
>
On Tue, Nov 21, 2023 at 2:44 PM Russell King <[email protected]> wrote:
>
> From: James Morse <[email protected]>
>
> Neither arm64 nor riscv support physical hotadd of CPUs that were not
> present at boot. For arm64 much of the platform description is in static
> tables which do not have update methods. arm64 does support HOTPLUG_CPU,
> which is backed by a firmware interface to turn CPUs on and off.
>
> acpi_processor_hotadd_init() and acpi_processor_remove() are for adding
> and removing CPUs that were not present at boot. arm64 systems that do this
> are not supported as there is currently insufficient information in the
> platform description. (e.g. did the GICR get removed too?)
>
> arm64 currently relies on the MADT enabled flag check in map_gicc_mpidr()
> to prevent CPUs that were not described as present at boot from being
> added to the system. Similarly, riscv relies on the same check in
> map_rintc_hartid(). Both architectures also rely on the weak 'always fails'
> definitions of acpi_map_cpu() and arch_register_cpu().
>
> Subsequent changes will redefine ACPI_HOTPLUG_CPU as making possible
> CPUs present. Neither arm64 nor riscv support this.
>
> Disable ACPI_HOTPLUG_CPU for arm64 and riscv by removing 'default y' and
> selecting it on the other three ACPI architectures. This allows the weak
> definitions of some symbols to be removed.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
I can apply this if it gets ACKs from the maintainers of the affected
architectures.
> ---
> Changes since RFC:
> * Expanded conditions to avoid ACPI_HOTPLUG_CPU being enabled when
> HOTPLUG_CPU isn't.
> Changes since RFC v3:
> * Dropped ia64 changes
> ---
> arch/loongarch/Kconfig | 1 +
> arch/x86/Kconfig | 1 +
> drivers/acpi/Kconfig | 1 -
> drivers/acpi/acpi_processor.c | 18 ------------------
> 4 files changed, 2 insertions(+), 19 deletions(-)
>
> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
> index ee123820a476..331becb2cb4f 100644
> --- a/arch/loongarch/Kconfig
> +++ b/arch/loongarch/Kconfig
> @@ -5,6 +5,7 @@ config LOONGARCH
> select ACPI
> select ACPI_GENERIC_GSI if ACPI
> select ACPI_MCFG if ACPI
> + select ACPI_HOTPLUG_CPU if ACPI_PROCESSOR && HOTPLUG_CPU
> select ACPI_PPTT if ACPI
> select ACPI_SYSTEM_POWER_STATES_SUPPORT if ACPI
> select ARCH_BINFMT_ELF_STATE
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 3762f41bb092..dbdcfc708369 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -59,6 +59,7 @@ config X86
> #
> select ACPI_LEGACY_TABLES_LOOKUP if ACPI
> select ACPI_SYSTEM_POWER_STATES_SUPPORT if ACPI
> + select ACPI_HOTPLUG_CPU if ACPI_PROCESSOR && HOTPLUG_CPU
> select ARCH_32BIT_OFF_T if X86_32
> select ARCH_CLOCKSOURCE_INIT
> select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE
> diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
> index f819e760ff19..a3acfc750fce 100644
> --- a/drivers/acpi/Kconfig
> +++ b/drivers/acpi/Kconfig
> @@ -310,7 +310,6 @@ config ACPI_HOTPLUG_CPU
> bool
> depends on ACPI_PROCESSOR && HOTPLUG_CPU
> select ACPI_CONTAINER
> - default y
>
> config ACPI_PROCESSOR_AGGREGATOR
> tristate "Processor Aggregator"
> diff --git a/drivers/acpi/acpi_processor.c b/drivers/acpi/acpi_processor.c
> index 0f5218e361df..4fe2ef54088c 100644
> --- a/drivers/acpi/acpi_processor.c
> +++ b/drivers/acpi/acpi_processor.c
> @@ -184,24 +184,6 @@ static void __init acpi_pcc_cpufreq_init(void) {}
>
> /* Initialization */
> #ifdef CONFIG_ACPI_HOTPLUG_CPU
> -int __weak acpi_map_cpu(acpi_handle handle,
> - phys_cpuid_t physid, u32 acpi_id, int *pcpu)
> -{
> - return -ENODEV;
> -}
> -
> -int __weak acpi_unmap_cpu(int cpu)
> -{
> - return -ENODEV;
> -}
> -
> -int __weak arch_register_cpu(int cpu)
> -{
> - return -ENODEV;
> -}
> -
> -void __weak arch_unregister_cpu(int cpu) {}
> -
> static int acpi_processor_hotadd_init(struct acpi_processor *pr)
> {
> unsigned long long sta;
> --
> 2.30.2
>
On Tue, 21 Nov 2023 13:43:54 +0000
Russell King <[email protected]> wrote:
> From: James Morse <[email protected]>
>
> register_cpu_capacity_sysctl() adds a property to sysfs that describes
> the CPUs capacity. This is done from a subsys_initcall() that assumes
> all possible CPUs are registered.
>
> With CPU hotplug, possible CPUs aren't registered until they become
> present, (or for arm64 enabled). This leads to messages during boot:
> | register_cpu_capacity_sysctl: too early to get CPU1 device!
> and once these CPUs are added to the system, the file is missing.
>
> Move this to a cpuhp callback, so that the file is created once
> CPUs are brought online. This covers CPUs that are added late by
> mechanisms like hotplug.
> One observable difference is the file is now missing for offline CPUs.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
> ---
> If the offline CPUs thing is a problem for the tools that consume
> this value, we'd need to move cpu_capacity to be part of cpu.c's
> common_cpu_attr_groups. However, attempts to discuss this just end
> up in a black hole, so this is a non-starter. Thus, if this needs
> to be done, it can be done as a separate patch.
> ---
> drivers/base/arch_topology.c | 38 ++++++++++++++++++++++++------------
> 1 file changed, 26 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
> index b741b5ba82bd..9ccb7daee78e 100644
> --- a/drivers/base/arch_topology.c
> +++ b/drivers/base/arch_topology.c
> @@ -220,20 +220,34 @@ static DECLARE_WORK(update_topology_flags_work, update_topology_flags_workfn);
>
> static DEVICE_ATTR_RO(cpu_capacity);
>
> -static int register_cpu_capacity_sysctl(void)
> +static int cpu_capacity_sysctl_add(unsigned int cpu)
> {
> - int i;
> - struct device *cpu;
> + struct device *cpu_dev = get_cpu_device(cpu);
>
> - for_each_possible_cpu(i) {
> - cpu = get_cpu_device(i);
> - if (!cpu) {
> - pr_err("%s: too early to get CPU%d device!\n",
> - __func__, i);
> - continue;
> - }
> - device_create_file(cpu, &dev_attr_cpu_capacity);
> - }
> + if (!cpu_dev)
> + return -ENOENT;
> +
> + device_create_file(cpu_dev, &dev_attr_cpu_capacity);
> +
> + return 0;
> +}
> +
> +static int cpu_capacity_sysctl_remove(unsigned int cpu)
> +{
> + struct device *cpu_dev = get_cpu_device(cpu);
> +
> + if (!cpu_dev)
> + return -ENOENT;
> +
> + device_remove_file(cpu_dev, &dev_attr_cpu_capacity);
> +
> + return 0;
> +}
> +
> +static int register_cpu_capacity_sysctl(void)
> +{
> + cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "topology/cpu-capacity",
> + cpu_capacity_sysctl_add, cpu_capacity_sysctl_remove);
>
> return 0;
> }
On Tue, 21 Nov 2023 13:44:05 +0000
"Russell King (Oracle)" <[email protected]> wrote:
> arch_register_cpu() and arch_unregister_cpu() are not used by anything
> that can be a module - they are used by drivers/base/cpu.c and
> drivers/acpi/acpi_processor.c, neither of which can be a module.
>
> Remove the exports.
>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
I somehow ended up replying to the RFC even though you'd posted this.
On basis this might get picked up directly from this posting.
Reviewed-by: Jonathan Cameron <[email protected]>
> ---
> arch/x86/kernel/topology.c | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/arch/x86/kernel/topology.c b/arch/x86/kernel/topology.c
> index 0bab03130033..fcb62cfdf946 100644
> --- a/arch/x86/kernel/topology.c
> +++ b/arch/x86/kernel/topology.c
> @@ -45,13 +45,11 @@ int arch_register_cpu(int cpu)
> xc->cpu.hotpluggable = cpu > 0;
> return register_cpu(&xc->cpu, cpu);
> }
> -EXPORT_SYMBOL(arch_register_cpu);
>
> void arch_unregister_cpu(int num)
> {
> unregister_cpu(&per_cpu(cpu_devices, num).cpu);
> }
> -EXPORT_SYMBOL(arch_unregister_cpu);
> #else /* CONFIG_HOTPLUG_CPU */
>
> int __init arch_register_cpu(int num)
On Tue, 21 Nov 2023 13:44:10 +0000
"Russell King (Oracle)" <[email protected]> wrote:
> arch_register_cpu() and arch_unregister_cpu() are not used by anything
> that can be a module - they are used by drivers/base/cpu.c and
> drivers/acpi/acpi_processor.c, neither of which can be a module.
>
> Remove the exports.
>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
> ---
> arch/loongarch/kernel/topology.c | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/arch/loongarch/kernel/topology.c b/arch/loongarch/kernel/topology.c
> index 3fd166006698..ae860fe81536 100644
> --- a/arch/loongarch/kernel/topology.c
> +++ b/arch/loongarch/kernel/topology.c
> @@ -25,7 +25,6 @@ int arch_register_cpu(int cpu)
>
> return ret;
> }
> -EXPORT_SYMBOL(arch_register_cpu);
>
> void arch_unregister_cpu(int cpu)
> {
> @@ -34,7 +33,6 @@ void arch_unregister_cpu(int cpu)
> c->hotpluggable = 0;
> unregister_cpu(c);
> }
> -EXPORT_SYMBOL(arch_unregister_cpu);
> #endif
>
> static int __init topology_init(void)
On Tue, 21 Nov 2023 13:44:15 +0000
Russell King <[email protected]> wrote:
> From: James Morse <[email protected]>
>
> Neither arm64 nor riscv support physical hotadd of CPUs that were not
> present at boot. For arm64 much of the platform description is in static
> tables which do not have update methods. arm64 does support HOTPLUG_CPU,
> which is backed by a firmware interface to turn CPUs on and off.
>
> acpi_processor_hotadd_init() and acpi_processor_remove() are for adding
> and removing CPUs that were not present at boot. arm64 systems that do this
> are not supported as there is currently insufficient information in the
> platform description. (e.g. did the GICR get removed too?)
>
> arm64 currently relies on the MADT enabled flag check in map_gicc_mpidr()
> to prevent CPUs that were not described as present at boot from being
> added to the system. Similarly, riscv relies on the same check in
> map_rintc_hartid(). Both architectures also rely on the weak 'always fails'
> definitions of acpi_map_cpu() and arch_register_cpu().
>
> Subsequent changes will redefine ACPI_HOTPLUG_CPU as making possible
> CPUs present. Neither arm64 nor riscv support this.
>
> Disable ACPI_HOTPLUG_CPU for arm64 and riscv by removing 'default y' and
> selecting it on the other three ACPI architectures. This allows the weak
> definitions of some symbols to be removed.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:44:25 +0000
Russell King <[email protected]> wrote:
> From: James Morse <[email protected]>
>
> Architectures often have extra per-cpu work that needs doing
> before a CPU is registered, often to determine if a CPU is
> hotpluggable.
>
> To allow the ACPI architectures to use GENERIC_CPU_DEVICES, move
> the cpu_register() call into arch_register_cpu(), which is made __weak
> so architectures with extra work can override it.
> This aligns with the way x86, ia64 and loongarch register hotplug CPUs
> when they become present.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:44:31 +0000
Russell King <[email protected]> wrote:
> From: James Morse <[email protected]>
>
> Add arch_unregister_cpu() to allow the ACPI machinery to call
> unregister_cpu(). This is enough for arm64, riscv and loongarch, but
> needs to be overridden by x86 and ia64 who need to do more work.
>
> CC: Jean-Philippe Brucker <[email protected]>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:44:36 +0000
"Russell King (Oracle)" <[email protected]> wrote:
> The differences between architecture specific implementations of
> arch_register_cpu() are down to whether the CPU is hotpluggable or not.
> Rather than overriding the weak version of arch_register_cpu(), provide
> a function that can be used to provide this detail instead.
>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:44:41 +0000
Russell King <[email protected]> wrote:
> From: James Morse <[email protected]>
>
> NUMA systems require the node descriptions to be ready before CPUs are
> registered. This is so that the node symlinks can be created in sysfs.
>
> Currently no NUMA platform uses GENERIC_CPU_DEVICES, meaning that CPUs
> are registered by arch code, instead of cpu_dev_init().
>
> Move cpu_dev_init() after node_dev_init() so that NUMA architectures
> can use GENERIC_CPU_DEVICES.
>
> Signed-off-by: James Morse <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Another one where I gave RB late on the RFC.
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:44:46 +0000
Russell King <[email protected]> wrote:
> From: James Morse <[email protected]>
>
> loongarch, mips, parisc, riscv and sh all print a warning if
> register_cpu() returns an error. Architectures that use
> GENERIC_CPU_DEVICES call panic() instead.
>
> Errors in this path indicate something is wrong with the firmware
> description of the platform, but the kernel is able to keep running.
>
> Downgrade this to a warning to make it easier to debug this issue.
>
> This will allow architectures that switching over to GENERIC_CPU_DEVICES
> to drop their warning, but keep the existing behaviour.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Russell King (Oracle) <[email protected]>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:44:51 +0000
Russell King <[email protected]> wrote:
> From: James Morse <[email protected]>
>
> To allow ACPI's _STA value to hide CPUs that are present, but not
> available to online right now due to VMM or firmware policy, the
> register_cpu() call needs to be made by the ACPI machinery when ACPI
> is in use. This allows it to hide CPUs that are unavailable from sysfs.
>
> Switching to GENERIC_CPU_DEVICES is an intermediate step to allow all
> five ACPI architectures to be modified at once.
>
> Switch over to GENERIC_CPU_DEVICES, and provide an arch_register_cpu()
> that populates the hotpluggable flag. arch_register_cpu() is also the
> interface the ACPI machinery expects.
>
> The struct cpu in struct cpuinfo_arm64 is never used directly, remove
> it to use the one GENERIC_CPU_DEVICES provides.
>
> This changes the CPUs visible in sysfs from possible to present, but
> on arm64 smp_prepare_cpus() ensures these are the same.
>
> This patch also has the effect of moving the registration of CPUs from
> subsys to driver core initialisation, prior to any initcalls running.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Russell King (Oracle) <[email protected]>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:44:56 +0000
"Russell King (Oracle)" <[email protected]> wrote:
> Convert arm64 to use the arch_cpu_is_hotpluggable() helper rather than
> arch_register_cpu().
>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:45:12 +0000
"Russell King (Oracle)" <[email protected]> wrote:
> Convert x86 to use the arch_cpu_is_hotpluggable() helper rather than
> arch_register_cpu().
>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:45:27 +0000
"Russell King (Oracle)" <[email protected]> wrote:
> Convert loongarch to use the arch_cpu_is_hotpluggable() helper rather
> than arch_register_cpu(). Also remove the export as nothing should be
> using arch_register_cpu() outside of the core kernel/acpi code.
>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:45:37 +0000
"Russell King (Oracle)" <[email protected]> wrote:
> Convert riscv to use the arch_cpu_is_hotpluggable() helper rather than
> arch_register_cpu().
>
> Acked-by: Palmer Dabbelt <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:45:01 +0000
Russell King <[email protected]> wrote:
> From: James Morse <[email protected]>
>
> Now that GENERIC_CPU_DEVICES calls arch_register_cpu(), which can be
> overridden by the arch code, switch over to this to allow common code
> to choose when the register_cpu() call is made.
>
> x86's struct cpus come from struct x86_cpu, which has no other members
> or users. Remove this and use the version defined by common code.
>
> This is an intermediate step to the logic being moved to drivers/acpi,
> where GENERIC_CPU_DEVICES will do the work when booting with acpi=off.
>
> This patch also has the effect of moving the registration of CPUs from
> subsys to driver core initialisation, prior to any initcalls running.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:45:17 +0000
Russell King <[email protected]> wrote:
> From: James Morse <[email protected]>
>
> Now that GENERIC_CPU_DEVICES calls arch_register_cpu(), which can be
> overridden by the arch code, switch over to this to allow common code
> to choose when the register_cpu() call is made.
>
> This allows topology_init() to be removed.
>
> This is an intermediate step to the logic being moved to drivers/acpi,
> where GENERIC_CPU_DEVICES will do the work when booting with acpi=off.
>
> This is a subtle change. Originally:
> - on boot, topology_init() would have marked present CPUs that
> io_master() is true for as hotplug-incapable.
> - if a CPU is hotplugged that is an io_master(), it can later be
> hot-unplugged.
>
> The new behaviour is that any CPU that io_master() is true for will
> now always be marked as hotplug-incapable, thus even if it was
> hotplugged, it can no longer be hot-unplugged.
>
> This patch also has the effect of moving the registration of CPUs from
> subsys to driver core initialisation, prior to any initcalls running.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On Tue, 21 Nov 2023 13:45:07 +0000
"Russell King (Oracle)" <[email protected]> wrote:
> Since the x86 version of arch_unregister_cpu() is the same as the weak
> version, drop the x86 specific version.
>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
On 11/22/23 00:44, Russell King (Oracle) wrote:
> From: James Morse <[email protected]>
>
> Add arch_unregister_cpu() to allow the ACPI machinery to call
> unregister_cpu(). This is enough for arm64, riscv and loongarch, but
> needs to be overridden by x86 and ia64 who need to do more work.
>
> CC: Jean-Philippe Brucker <[email protected]>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
> ---
> An open question remains from the RFC v2 posting: should we provide a
> __weak stub for !HOTPLUG_CPU as well, since in later patches ACPI may
> reference this if the compiler doesn't optimise as we expect?
>
> Changes since v1:
> * Added CONFIG_HOTPLUG_CPU ifdeffery around unregister_cpu
> Changes since RFC v2:
> * Move earlier in the series
> ---
> drivers/base/cpu.c | 9 ++++++++-
> 1 file changed, 8 insertions(+), 1 deletion(-)
>
Reviewed-by: Gavin Shan <[email protected]>
On 11/22/23 00:44, Russell King (Oracle) wrote:
> The differences between architecture specific implementations of
> arch_register_cpu() are down to whether the CPU is hotpluggable or not.
> Rather than overriding the weak version of arch_register_cpu(), provide
> a function that can be used to provide this detail instead.
>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
> ---
> drivers/base/cpu.c | 11 ++++++++++-
> include/linux/cpu.h | 1 +
> 2 files changed, 11 insertions(+), 1 deletion(-)
>
Reviewed-by: Gavin Shan <[email protected]>
On 11/22/23 00:44, Russell King (Oracle) wrote:
> From: James Morse <[email protected]>
>
> NUMA systems require the node descriptions to be ready before CPUs are
> registered. This is so that the node symlinks can be created in sysfs.
>
> Currently no NUMA platform uses GENERIC_CPU_DEVICES, meaning that CPUs
> are registered by arch code, instead of cpu_dev_init().
>
> Move cpu_dev_init() after node_dev_init() so that NUMA architectures
> can use GENERIC_CPU_DEVICES.
>
> Signed-off-by: James Morse <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
> ---
> Note: Jonathan's comment still needs addressing - see
> https://lore.kernel.org/r/[email protected]
> ---
> drivers/base/init.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
Reviewed-by: Gavin Shan <[email protected]>
On 11/22/23 00:44, Russell King (Oracle) wrote:
> From: James Morse <[email protected]>
>
> loongarch, mips, parisc, riscv and sh all print a warning if
> register_cpu() returns an error. Architectures that use
> GENERIC_CPU_DEVICES call panic() instead.
>
> Errors in this path indicate something is wrong with the firmware
> description of the platform, but the kernel is able to keep running.
>
> Downgrade this to a warning to make it easier to debug this issue.
>
> This will allow architectures that switching over to GENERIC_CPU_DEVICES
> to drop their warning, but keep the existing behaviour.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Russell King (Oracle) <[email protected]>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
> ---
> drivers/base/cpu.c | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
Reviewed-by: Gavin Shan <[email protected]>
On Tue, Nov 21 2023 at 13:43, Russell King wrote:
> The majority of the other patches come from the vCPU hotplug RFC v3
> series I posted earlier, rebased on Linus' current tip, but with some
> new patches adding arch_cpu_is_hotpluggable() as the remaining
> arch_register_cpu() functions only differ in the setting of the
> hotpluggable member of the CPU device - so let's get generic code
> doing that and provide a way for an architecture to specify whether a
> CPU is hotpluggable.
>
> This patch series has been updated as best I can from the comments on
> its previous 22-patch posting, but there are some things that I have
> been unable to address (some of which go back to James' posting of
> RFC v2 of the vcpu hotplug series) due to lack of co-operation from
> either reviewers responding to my questions, or from the patch author
> providing information. I have now come to the conclusion that this
> information is never going to come, but there is still benefit to
> moving forward with this patch set. I don't expect that anyone will
> even bother to read this far down the email, so blah blah blah blah
> blah blah blah blah blah. I bet no one reads this so I don't know why
> I bother writing crud like this.
You lost your bet. I always read cover letters completely because they
tell a lot. There is correlation between the amount of blah and the
quality of the series :)
Thanks,
tglx
On Tue, Nov 21 2023 at 13:43, Russell King wrote:
> ---
> If the offline CPUs thing is a problem for the tools that consume
> this value, we'd need to move cpu_capacity to be part of cpu.c's
> common_cpu_attr_groups. However, attempts to discuss this just end
> up in a black hole, so this is a non-starter. Thus, if this needs
> to be done, it can be done as a separate patch.
Offline CPUs have 0 capacity by definition....
On Tue, Nov 21 2023 at 13:44, Russell King wrote:
> ---
> An open question remains from the RFC v2 posting: should we provide a
> __weak stub for !HOTPLUG_CPU as well, since in later patches ACPI may
> reference this if the compiler doesn't optimise as we expect?
You mean:
extern void foo(void);
if (!IS_ENABLED(CONFIG_FOO))
foo();
The kernel uses this pattern for years and if someday a compiler starts
to fail to eliminate the call to 'foo()' for CONFIG_FOO=n then you
already get hundreds linkage fails today.
So adding one more in later patches won't matter much :)
Russell!
On Tue, Nov 21 2023 at 13:43, Russell King wrote:
> This series aims to switch most architectures over to using generic CPU
> devices rather than arch specific implementations, which I think is
> worthwhile doing even if the vCPU hotplug series needs further work.
I went through the whole series and I can't find anything
objectionable.
Vs. merging: It does not make sense to split this up and route
individual patches.
So I can pick the whole pile up and route it through tip smp/core unless
Rafael or Greg prefer to take it through one of their trees. For the
latter case:
Reviewed-by: Thomas Gleixner <[email protected]>
Greg, Rafael?
Thanks,
tglx
On Fri, Dec 01, 2023 at 12:25:54PM +0100, Thomas Gleixner wrote:
> Russell!
>
> On Tue, Nov 21 2023 at 13:43, Russell King wrote:
> > This series aims to switch most architectures over to using generic CPU
> > devices rather than arch specific implementations, which I think is
> > worthwhile doing even if the vCPU hotplug series needs further work.
>
> I went through the whole series and I can't find anything
> objectionable.
>
> Vs. merging: It does not make sense to split this up and route
> individual patches.
>
> So I can pick the whole pile up and route it through tip smp/core unless
> Rafael or Greg prefer to take it through one of their trees. For the
> latter case:
>
> Reviewed-by: Thomas Gleixner <[email protected]>
>
> Greg, Rafael?
I can take them, will do so this weekend when I catch up on patches on a
14+ hour flight...
greg k-h
On Fri, Dec 01 2023 at 12:28, Greg Kroah-Hartman wrote:
> I can take them, will do so this weekend when I catch up on patches on a
> 14+ hour flight...
Thanks a lot!
On Tue, Nov 21, 2023 at 01:44:51PM +0000, Russell King wrote:
> From: James Morse <[email protected]>
>
> To allow ACPI's _STA value to hide CPUs that are present, but not
> available to online right now due to VMM or firmware policy, the
> register_cpu() call needs to be made by the ACPI machinery when ACPI
> is in use. This allows it to hide CPUs that are unavailable from sysfs.
>
> Switching to GENERIC_CPU_DEVICES is an intermediate step to allow all
> five ACPI architectures to be modified at once.
>
> Switch over to GENERIC_CPU_DEVICES, and provide an arch_register_cpu()
> that populates the hotpluggable flag. arch_register_cpu() is also the
> interface the ACPI machinery expects.
>
> The struct cpu in struct cpuinfo_arm64 is never used directly, remove
> it to use the one GENERIC_CPU_DEVICES provides.
>
> This changes the CPUs visible in sysfs from possible to present, but
> on arm64 smp_prepare_cpus() ensures these are the same.
>
> This patch also has the effect of moving the registration of CPUs from
> subsys to driver core initialisation, prior to any initcalls running.
>
> Signed-off-by: James Morse <[email protected]>
> Reviewed-by: Russell King (Oracle) <[email protected]>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
> ---
> Changes since RFC v2:
> * Add note about initialisation order change.
> ---
> arch/arm64/Kconfig | 1 +
> arch/arm64/include/asm/cpu.h | 1 -
> arch/arm64/kernel/setup.c | 13 ++++---------
> 3 files changed, 5 insertions(+), 10 deletions(-)
Acked-by: Will Deacon <[email protected]>
Will
On Tue, Nov 21, 2023 at 01:44:56PM +0000, Russell King (Oracle) wrote:
> Convert arm64 to use the arch_cpu_is_hotpluggable() helper rather than
> arch_register_cpu().
>
> Reviewed-by: Shaoqin Huang <[email protected]>
> Reviewed-by: Gavin Shan <[email protected]>
> Signed-off-by: Russell King (Oracle) <[email protected]>
> ---
> arch/arm64/kernel/setup.c | 8 ++------
> 1 file changed, 2 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> index 165bd2c0dd5a..42c690bb2d60 100644
> --- a/arch/arm64/kernel/setup.c
> +++ b/arch/arm64/kernel/setup.c
> @@ -402,13 +402,9 @@ static inline bool cpu_can_disable(unsigned int cpu)
> return false;
> }
>
> -int arch_register_cpu(int num)
> +bool arch_cpu_is_hotpluggable(int num)
> {
> - struct cpu *cpu = &per_cpu(cpu_devices, num);
> -
> - cpu->hotpluggable = cpu_can_disable(num);
> -
> - return register_cpu(cpu, num);
> + return cpu_can_disable(num);
> }
>
> static void dump_kernel_offset(void)
Acked-by: Will Deacon <[email protected]>
Will