2001-02-05 04:01:03

by Rusty Russell

[permalink] [raw]
Subject: [PATCH] Hot swap CPU support for 2.4.1

Hi all,

I did the infrastructure, Anton did the bugfinding and PPC support,
aka. the hard stuff. Other architectures need to implement
__cpu_disable, __cpu_die and __cpu_up for them to work. Volunteers
appreciated.

This patch allows you to down & up CPUs as follows:
# echo 0 > /proc/sys/cpu/0/online
# echo 1 > /proc/sys/cpu/0/online

The relatively trivial patch works as follows:

1) Implements synchronize_kernel() (thanks Andi Kleen for forwarding
Paul McKenney's quiescent-state ideas) which waits for a schedule
on all CPUs.
2) All CPU numbers are now physical: removes cpu_number_map,
cpu_logical_map and smp_num_cpus.
3) Adds cpu_online(cpu) and cpu_num_online() macros.
4) Adds cpu_down() and cpu_up() calls, which call arch-specific
__cpu_disable(cpu), __cpu_die(cpu) and __cpu_up(cpu).
5) Fixes schedule() to check allowed_cpus even if rescheduling same
task.

Since it's 60k long, mime attached bzip2.

Go hack!
Rusty Russell & Anton Blanchard
--


Attachments:
2.4.1-hotswap-cpu.patch.bz2 (13.57 kB)
hotswap CPU patch

2001-02-05 04:51:53

by Frank Davis

[permalink] [raw]
Subject: Re: [PATCH] Hot swap CPU support for 2.4.1

Hello,
Which archs still need to implement it? I briefly looked over the patch and noticed that it had i386, ppc, mips64, and s390 already there.
Regards,
Frank

>Hi all,
>
>I did the infrastructure, Anton did the bugfinding and PPC >support,
>aka. the hard stuff. Other architectures need to implement
>__cpu_disable, __cpu_die and __cpu_up for them to work. >Volunteers
>appreciated.


2001-02-05 05:52:05

by Lars Marowsky-Bree

[permalink] [raw]
Subject: Re: [PATCH] Hot swap CPU support for 2.4.1

On 2001-02-05T15:00:40,
Rusty Russell <[email protected]> said:

> I did the infrastructure, Anton did the bugfinding and PPC support,
> aka. the hard stuff. Other architectures need to implement
> __cpu_disable, __cpu_die and __cpu_up for them to work. Volunteers
> appreciated.

Rusty, what would be needed to "hot-add" CPUs ?

Sincerely,
Lars Marowsky-Br?e <[email protected]>
SuSE Linux AG at the SAP LinuxLab - [email protected]

--
Perfection is our goal, excellence will be tolerated. -- J. Yahl

2001-02-05 06:09:00

by Anton Blanchard

[permalink] [raw]
Subject: Re: [PATCH] Hot swap CPU support for 2.4.1


> Rusty, what would be needed to "hot-add" CPUs ?

The PPC version at the moment simply locks a cpu in the idle loop
with __cli(); while(1); for cpu down and jumps out of it for cpu up.
Good for testing but not very useful. After talking to paulus we
will use the RTAS cpu stop and cpu start.

In order to bring a new cpu up you will need to duplicate a lot of
the stuff in smp_boot_cpus or else just set up all NR_CPUS of these
structures (eg NR_CPUS idle threads etc) at boot time.

Anton

2001-02-05 06:54:29

by Rusty Russell

[permalink] [raw]
Subject: Re: [PATCH] Hot swap CPU support for 2.4.1

In message <[email protected]> you write
:
> Hello,
> Which archs still need to implement it? I briefly looked over the patch an
d noticed that it had i386, ppc, mips64, and s390 already there.

PPC is there (kinda hackish, but proof of concept). For the rest, I
don't consider:

return -ENOSYS;

an implementation. Someone who understands the interrupt controllers
and vagarities for each platform needs to implement them...

Rusty.
--
Premature optmztion is rt of all evl. --DK

2001-02-05 21:19:48

by Matthew Fredrickson

[permalink] [raw]
Subject: Re: [PATCH] Hot swap CPU support for 2.4.1@


Would any special hardware besides a multi-cpu system be necessarey to
test this out?


Matthew Fredrickson

On Mon, Feb 05, 2001 at 03:00:40PM +1100, Rusty Russell wrote:
> Hi all,
>
> I did the infrastructure, Anton did the bugfinding and PPC support,
> aka. the hard stuff. Other architectures need to implement
> __cpu_disable, __cpu_die and __cpu_up for them to work. Volunteers
> appreciated.
>
> This patch allows you to down & up CPUs as follows:
> # echo 0 > /proc/sys/cpu/0/online
> # echo 1 > /proc/sys/cpu/0/online
>
> The relatively trivial patch works as follows:
>
> 1) Implements synchronize_kernel() (thanks Andi Kleen for forwarding
> Paul McKenney's quiescent-state ideas) which waits for a schedule
> on all CPUs.
> 2) All CPU numbers are now physical: removes cpu_number_map,
> cpu_logical_map and smp_num_cpus.
> 3) Adds cpu_online(cpu) and cpu_num_online() macros.
> 4) Adds cpu_down() and cpu_up() calls, which call arch-specific
> __cpu_disable(cpu), __cpu_die(cpu) and __cpu_up(cpu).
> 5) Fixes schedule() to check allowed_cpus even if rescheduling same
> task.
>
> Since it's 60k long, mime attached bzip2.
>
> Go hack!
> Rusty Russell & Anton Blanchard
> --


2001-02-06 14:19:27

by Anton Blanchard

[permalink] [raw]
Subject: Re: [PATCH] Hot swap CPU support for 2.4.1@


> Would any special hardware besides a multi-cpu system be necessarey to
> test this out?

You should be able to run it on any SMP machine assuming you write the
arch specific code (PPC could be used as an example). Of course it isn't
very interesting if the hardware doesn't support hot swap :)

As soon as I get the SMP ultra booting again (I arrived one morning to
hear the disk was making loud grinding noises) I'll code up sparc (ie E10K)
support. It sounds like S390 support will be trivial, I'd love to get my
hands on one of those :)

Anton

2001-02-11 10:45:53

by Pavel Machek

[permalink] [raw]
Subject: Re: [PATCH] Hot swap CPU support for 2.4.1

Hi!

> I did the infrastructure, Anton did the bugfinding and PPC support,
> aka. the hard stuff. Other architectures need to implement
> __cpu_disable, __cpu_die and __cpu_up for them to work. Volunteers
> appreciated.
>
> This patch allows you to down & up CPUs as follows:
> # echo 0 > /proc/sys/cpu/0/online
> # echo 1 > /proc/sys/cpu/0/online
>
> The relatively trivial patch works as follows:
>
> 1) Implements synchronize_kernel() (thanks Andi Kleen for forwarding
> Paul McKenney's quiescent-state ideas) which waits for a schedule
> on all CPUs.
> 2) All CPU numbers are now physical: removes cpu_number_map,
> cpu_logical_map and smp_num_cpus.
> 3) Adds cpu_online(cpu) and cpu_num_online() macros.
> 4) Adds cpu_down() and cpu_up() calls, which call arch-specific
> __cpu_disable(cpu), __cpu_die(cpu) and __cpu_up(cpu).
> 5) Fixes schedule() to check allowed_cpus even if rescheduling same
> task.

This is not quite right:

@@ -1643,7 +1643,7 @@
printk(KERN_NOTICE "apm: disabled on user
request.\n");
return -ENODEV;
}
- if ((smp_num_cpus > 1) && !power_off) {
+ if ((num_online_cpus() > 1) && !power_off) {
printk(KERN_NOTICE "apm: disabled - APM is not SMP
safe.\n");
return -ENODEV;
}
@@ -1697,7 +1697,7 @@

kernel_thread(apm, NULL, CLONE_FS | CLONE_FILES |
CLONE_SIGHAND | SIGCHLD);

- if (smp_num_cpus > 1) {
+ if (num_online_cpus() > 1) {
printk(KERN_NOTICE
"apm: disabled - APM is not SMP safe (power off
active).\n");
return 0;

I do not think it is safe to call APM when there is just CPU #5
running. smp_num_cpus in this context means "if we ever had more than
boot cpu".
Pavel
--
I'm [email protected]. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at [email protected]

2001-02-11 23:06:17

by Rusty Russell

[permalink] [raw]
Subject: Re: [PATCH] Hot swap CPU support for 2.4.1

In message <[email protected]> you write:
> This is not quite right:
>
> @@ -1643,7 +1643,7 @@
> printk(KERN_NOTICE "apm: disabled on user
> request.\n");
> return -ENODEV;
> }
> - if ((smp_num_cpus > 1) && !power_off) {
> + if ((num_online_cpus() > 1) && !power_off) {
> printk(KERN_NOTICE "apm: disabled - APM is not SMP
>
> I do not think it is safe to call APM when there is just CPU #5
> running. smp_num_cpus in this context means "if we ever had more than
> boot cpu".

Um, it's not safe to call APM in SMP full stop: we try anyway.
However, this code changes nothing since it's only run at boot.

Cheers,
Rusty.
--
Premature optmztion is rt of all evl. --DK