2013-05-28 08:00:17

by Chenhui Zhao

[permalink] [raw]
Subject: [PATCH] powerpc/mpic: fix irq distribution problem when MPIC_SINGLE_DEST_CPU

For the mpic with a flag MPIC_SINGLE_DEST_CPU, only one bit should be
set in interrupt destination registers.

The code is applicable to 64-bit platforms as well as 32-bit.

Signed-off-by: Zhao Chenhui <[email protected]>
---
arch/powerpc/sysdev/mpic.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/sysdev/mpic.c b/arch/powerpc/sysdev/mpic.c
index 0a13ecb..3cc2f91 100644
--- a/arch/powerpc/sysdev/mpic.c
+++ b/arch/powerpc/sysdev/mpic.c
@@ -54,7 +54,7 @@ static DEFINE_RAW_SPINLOCK(mpic_lock);

#ifdef CONFIG_PPC32 /* XXX for now */
#ifdef CONFIG_IRQ_ALL_CPUS
-#define distribute_irqs (!(mpic->flags & MPIC_SINGLE_DEST_CPU))
+#define distribute_irqs (1)
#else
#define distribute_irqs (0)
#endif
@@ -1703,7 +1703,7 @@ void mpic_setup_this_cpu(void)
* it differently, then we should make sure we also change the default
* values of irq_desc[].affinity in irq.c.
*/
- if (distribute_irqs) {
+ if (distribute_irqs && !(mpic->flags & MPIC_SINGLE_DEST_CPU)) {
for (i = 0; i < mpic->num_sources ; i++)
mpic_irq_write(i, MPIC_INFO(IRQ_DESTINATION),
mpic_irq_read(i, MPIC_INFO(IRQ_DESTINATION)) | msk);
--
1.7.3


2013-05-28 08:00:26

by Chenhui Zhao

[permalink] [raw]
Subject: [PATCH] powerpc/sysfs: disable hotplug for the boot cpu

Some features depend on the boot cpu, for instance, hibernate/suspend.
So disable hotplug for the boot cpu.

Signed-off-by: Zhao Chenhui <[email protected]>
---
arch/powerpc/kernel/sysfs.c | 4 +++-
1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c
index e68a845..294b1c4e 100644
--- a/arch/powerpc/kernel/sysfs.c
+++ b/arch/powerpc/kernel/sysfs.c
@@ -655,8 +655,10 @@ static int __init topology_init(void)
* CPU. For instance, the boot cpu might never be valid
* for hotplugging.
*/
- if (ppc_md.cpu_die)
+ if (ppc_md.cpu_die && cpu != boot_cpuid)
c->hotpluggable = 1;
+ else
+ c->hotpluggable = 0;

if (cpu_online(cpu) || c->hotpluggable) {
register_cpu(c, cpu);
--
1.7.3

2013-05-31 21:50:04

by Benjamin Herrenschmidt

[permalink] [raw]
Subject: Re: [PATCH] powerpc/sysfs: disable hotplug for the boot cpu

On Tue, 2013-05-28 at 15:59 +0800, Zhao Chenhui wrote:
> Some features depend on the boot cpu, for instance, hibernate/suspend.
> So disable hotplug for the boot cpu.

Don't we have code to "move" the boot CPU around when that happens ?

Ben.

> Signed-off-by: Zhao Chenhui <[email protected]>
> ---
> arch/powerpc/kernel/sysfs.c | 4 +++-
> 1 files changed, 3 insertions(+), 1 deletions(-)
>
> diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c
> index e68a845..294b1c4e 100644
> --- a/arch/powerpc/kernel/sysfs.c
> +++ b/arch/powerpc/kernel/sysfs.c
> @@ -655,8 +655,10 @@ static int __init topology_init(void)
> * CPU. For instance, the boot cpu might never be valid
> * for hotplugging.
> */
> - if (ppc_md.cpu_die)
> + if (ppc_md.cpu_die && cpu != boot_cpuid)
> c->hotpluggable = 1;
> + else
> + c->hotpluggable = 0;
>
> if (cpu_online(cpu) || c->hotpluggable) {
> register_cpu(c, cpu);

2013-06-03 10:43:31

by Chenhui Zhao

[permalink] [raw]
Subject: Re: [PATCH] powerpc/sysfs: disable hotplug for the boot cpu

On Sat, Jun 01, 2013 at 07:49:44AM +1000, Benjamin Herrenschmidt wrote:
> On Tue, 2013-05-28 at 15:59 +0800, Zhao Chenhui wrote:
> > Some features depend on the boot cpu, for instance, hibernate/suspend.
> > So disable hotplug for the boot cpu.
>
> Don't we have code to "move" the boot CPU around when that happens ?
>
> Ben.
>

Currently, the code in generic_cpu_disable() likes this:

if (cpu == boot_cpuid)
return -EBUSY;

If the dying cpu is the boot cpu, it will return -EBUSY. In the subsequent error handling,
cpu_notify_nofail(CPU_DOWN_FAILED) in _cpu_down() will be called. Unfortunately, some
cpu notifier callbacks handled CPU_DOWN_PREPARE, but not CPU_DOWN_FAILED, such as sched_cpu_inactive().
So it will cause issues.

If we set the hotpluggable for the boot cpu, we can prevent user applications from disabling the boot cpu.

-Chenhui

> > Signed-off-by: Zhao Chenhui <[email protected]>
> > ---
> > arch/powerpc/kernel/sysfs.c | 4 +++-
> > 1 files changed, 3 insertions(+), 1 deletions(-)
> >
> > diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c
> > index e68a845..294b1c4e 100644
> > --- a/arch/powerpc/kernel/sysfs.c
> > +++ b/arch/powerpc/kernel/sysfs.c
> > @@ -655,8 +655,10 @@ static int __init topology_init(void)
> > * CPU. For instance, the boot cpu might never be valid
> > * for hotplugging.
> > */
> > - if (ppc_md.cpu_die)
> > + if (ppc_md.cpu_die && cpu != boot_cpuid)
> > c->hotpluggable = 1;
> > + else
> > + c->hotpluggable = 0;
> >
> > if (cpu_online(cpu) || c->hotpluggable) {
> > register_cpu(c, cpu);
>
>
>

2013-06-12 03:25:33

by Benjamin Herrenschmidt

[permalink] [raw]
Subject: Re: [PATCH] powerpc/sysfs: disable hotplug for the boot cpu

On Mon, 2013-06-03 at 18:43 +0800, Zhao Chenhui wrote:
> On Sat, Jun 01, 2013 at 07:49:44AM +1000, Benjamin Herrenschmidt wrote:
> > On Tue, 2013-05-28 at 15:59 +0800, Zhao Chenhui wrote:
> > > Some features depend on the boot cpu, for instance, hibernate/suspend.
> > > So disable hotplug for the boot cpu.
> >
> > Don't we have code to "move" the boot CPU around when that happens ?
> >
> > Ben.
> >
>
> Currently, the code in generic_cpu_disable() likes this:
>
> if (cpu == boot_cpuid)
> return -EBUSY;

But the code in pseries/hotplug-cpu.c doesn't, we just "move" the boot
CPU around when that happens. Any reason we can't do that generically ?

Cheers,
Ben.

> If the dying cpu is the boot cpu, it will return -EBUSY. In the subsequent error handling,
> cpu_notify_nofail(CPU_DOWN_FAILED) in _cpu_down() will be called. Unfortunately, some
> cpu notifier callbacks handled CPU_DOWN_PREPARE, but not CPU_DOWN_FAILED, such as sched_cpu_inactive().
> So it will cause issues.
>
> If we set the hotpluggable for the boot cpu, we can prevent user applications from disabling the boot cpu.
>
> -Chenhui
>
> > > Signed-off-by: Zhao Chenhui <[email protected]>
> > > ---
> > > arch/powerpc/kernel/sysfs.c | 4 +++-
> > > 1 files changed, 3 insertions(+), 1 deletions(-)
> > >
> > > diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c
> > > index e68a845..294b1c4e 100644
> > > --- a/arch/powerpc/kernel/sysfs.c
> > > +++ b/arch/powerpc/kernel/sysfs.c
> > > @@ -655,8 +655,10 @@ static int __init topology_init(void)
> > > * CPU. For instance, the boot cpu might never be valid
> > > * for hotplugging.
> > > */
> > > - if (ppc_md.cpu_die)
> > > + if (ppc_md.cpu_die && cpu != boot_cpuid)
> > > c->hotpluggable = 1;
> > > + else
> > > + c->hotpluggable = 0;
> > >
> > > if (cpu_online(cpu) || c->hotpluggable) {
> > > register_cpu(c, cpu);
> >
> >
> >

2013-06-13 11:26:16

by Chenhui Zhao

[permalink] [raw]
Subject: Re: [PATCH] powerpc/sysfs: disable hotplug for the boot cpu

On Wed, Jun 12, 2013 at 01:25:22PM +1000, Benjamin Herrenschmidt wrote:
> On Mon, 2013-06-03 at 18:43 +0800, Zhao Chenhui wrote:
> > On Sat, Jun 01, 2013 at 07:49:44AM +1000, Benjamin Herrenschmidt wrote:
> > > On Tue, 2013-05-28 at 15:59 +0800, Zhao Chenhui wrote:
> > > > Some features depend on the boot cpu, for instance, hibernate/suspend.
> > > > So disable hotplug for the boot cpu.
> > >
> > > Don't we have code to "move" the boot CPU around when that happens ?
> > >
> > > Ben.
> > >
> >
> > Currently, the code in generic_cpu_disable() likes this:
> >
> > if (cpu == boot_cpuid)
> > return -EBUSY;
>
> But the code in pseries/hotplug-cpu.c doesn't, we just "move" the boot
> CPU around when that happens. Any reason we can't do that generically ?
>
> Cheers,
> Ben.
>

Some multicore SoCs firstly boot up the cpu0 after warm reset.
In some suspend/resume cases, SoC will do a warm reset when resuming.
In order to ensure that the suspending and resuming is running
on a same cpu, cpu0 should be the last cpu to suspend. Here, cpu0 is
the boot_cpuid.

-Chenhui

> > If the dying cpu is the boot cpu, it will return -EBUSY. In the subsequent error handling,
> > cpu_notify_nofail(CPU_DOWN_FAILED) in _cpu_down() will be called. Unfortunately, some
> > cpu notifier callbacks handled CPU_DOWN_PREPARE, but not CPU_DOWN_FAILED, such as sched_cpu_inactive().
> > So it will cause issues.
> >
> > If we set the hotpluggable for the boot cpu, we can prevent user applications from disabling the boot cpu.
> >
> > -Chenhui
> >

2013-06-14 22:53:50

by Benjamin Herrenschmidt

[permalink] [raw]
Subject: Re: [PATCH] powerpc/sysfs: disable hotplug for the boot cpu

On Thu, 2013-06-13 at 19:25 +0800, Zhao Chenhui wrote:
> Some multicore SoCs firstly boot up the cpu0 after warm reset.
> In some suspend/resume cases, SoC will do a warm reset when resuming.
> In order to ensure that the suspending and resuming is running
> on a same cpu, cpu0 should be the last cpu to suspend. Here, cpu0 is
> the boot_cpuid.

Well, so:

- In any case, your patch will break pseries, so it's not acceptable.

- Why does it have to absolutely resume from the same CPU it suspended
from ? Can't you have a little bit of code on the resuming CPU that
checks if it's not online, poke an online one and goes back to sleep ?

Cheers,
Ben.