[Oops: This is correct version: i send from the wrong directory earlier]
[Sorry about that spam... now i thin i know how to use quilt :-)]
This implements cpu hotplug support for X86_64.
Modified after initial feedback from Andi.
[PS: hopefully subject is now fixed with different subject lines]
Seems to hold up for make -j, with online/offline activity.
The series of patches are split as follows:
1. initcall cleanup
- Left __cpuinit cases as before per Andi
- Added a few new ones, and removed couple that could stay __init
even with cpu hotplug.
2. Core logical online/offline of cpus
- start with maxcpus=1, and then echo 1 to /sys/devices/system/online
- Can also bringup all cpus and then bring up/down all but cpu0.
- Also tested with numa=fake=2
3. Cleanup sibling map for cpu hotplug support.
4. Dont use IPI broadcast in smp_call_function() when using CPU hotplug.
Hopefully this is some reasonable middle ground starting point.
- Dont let a new cpu respond to IPI's.
- Automaticaly selected if CPU hotplus is choosen.
- Can also be turned on cmdline via safe_ipi=1
TBD:
1. Track down CONFIG_SCHED_SMT Oops with both cpu up/down.
2. Test on real NUMA hw.
Cheers,
Ashok Raj
--
On Tue, 2005-05-24 at 01:11 -0700, Ashok Raj wrote:
> TBD:
>
> 1. Track down CONFIG_SCHED_SMT Oops with both cpu up/down.
> 2. Test on real NUMA hw.
>
With below patch, cpu hotplug works with SCHED_SMT enabled in my test.
set_cpu_sibling_map is invoked before cpu is set to online.
Thanks,
Shaohua
--- a/arch/x86_64/kernel/smpboot.c.orig 2005-05-24 16:47:57.000000000 +0800
+++ b/arch/x86_64/kernel/smpboot.c 2005-05-24 16:48:31.000000000 +0800
@@ -443,7 +443,7 @@ set_cpu_sibling_map(int cpu)
int i;
if (smp_num_siblings > 1) {
- for_each_online_cpu(i) {
+ for (i = 0; i < NR_CPUS; i++) {
if (cpu_core_id[cpu] == cpu_core_id[i]) {
cpu_set(i, cpu_sibling_map[cpu]);
cpu_set(cpu, cpu_sibling_map[i]);
@@ -454,7 +454,7 @@ set_cpu_sibling_map(int cpu)
}
if (current_cpu_data.x86_num_cores > 1) {
- for_each_online_cpu(i) {
+ for (i = 0; i < NR_CPUS; i++) {
if (phys_proc_id[cpu] == phys_proc_id[i]) {
cpu_set(i, cpu_core_map[cpu]);
cpu_set(cpu, cpu_core_map[i]);
Shaohua Li wrote:
> On Tue, 2005-05-24 at 01:11 -0700, Ashok Raj wrote:
>
>>TBD:
>>
>>1. Track down CONFIG_SCHED_SMT Oops with both cpu up/down.
>>2. Test on real NUMA hw.
>>
>
> With below patch, cpu hotplug works with SCHED_SMT enabled in my test.
> set_cpu_sibling_map is invoked before cpu is set to online.
>
> Thanks,
> Shaohua
I'm not sure, but you probably want "for_each_cpu(i)" instead of "for (i =
0; i < NR_CPUS; i++)" below.
It should be rare that you really want to explicitly itterate over
[0..NR_CPUS] cpus. for_each_cpu() will itterate over all "possible" cpus,
which may well be [0..NR_CPUS], but it at least is cleaner and more
readable. If your arch actually sets up cpu_possible_map correctly, then
it will likely be quite a bit faster and more efficient to use for_each_cpu().
Again, I haven't looked at the surrounding code, but it's very likely that
for_each_cpu() will do what you want.
-Matt
> --- a/arch/x86_64/kernel/smpboot.c.orig 2005-05-24 16:47:57.000000000 +0800
> +++ b/arch/x86_64/kernel/smpboot.c 2005-05-24 16:48:31.000000000 +0800
> @@ -443,7 +443,7 @@ set_cpu_sibling_map(int cpu)
> int i;
>
> if (smp_num_siblings > 1) {
> - for_each_online_cpu(i) {
> + for (i = 0; i < NR_CPUS; i++) {
> if (cpu_core_id[cpu] == cpu_core_id[i]) {
> cpu_set(i, cpu_sibling_map[cpu]);
> cpu_set(cpu, cpu_sibling_map[i]);
> @@ -454,7 +454,7 @@ set_cpu_sibling_map(int cpu)
> }
>
> if (current_cpu_data.x86_num_cores > 1) {
> - for_each_online_cpu(i) {
> + for (i = 0; i < NR_CPUS; i++) {
> if (phys_proc_id[cpu] == phys_proc_id[i]) {
> cpu_set(i, cpu_core_map[cpu]);
> cpu_set(cpu, cpu_core_map[i]);
On Wed, May 25, 2005 at 03:16:40PM -0700, Matthew Dobson wrote:
> Shaohua Li wrote:
> > On Tue, 2005-05-24 at 01:11 -0700, Ashok Raj wrote:
> >
> > With below patch, cpu hotplug works with SCHED_SMT enabled in my test.
> > set_cpu_sibling_map is invoked before cpu is set to online.
> >
> > Thanks,
> > Shaohua
>
> I'm not sure, but you probably want "for_each_cpu(i)" instead of "for (i =
> 0; i < NR_CPUS; i++)" below.
>
I have a new set of patches just getting ready with final comments from Andi
This version already uses for_each_cpu varient.
Cheers,
ashok