2015-02-13 19:39:34

by Paul A. Clarke

[permalink] [raw]
Subject: [PATCH] powerpc: re-enable dynticks

implement arch_irq_work_has_interrupt() for powerpc

Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for
full dynamic ticks to be enabled, by expecting architectures to
implement a suitable arch_irq_work_has_interrupt() routine.

Several arches have implemented this routine, including x86 (3010279f)
and arm (09f6edd4), but powerpc was omitted.

This patch implements this routine for powerpc.

The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU
list>" is displayed:
NO_HZ: Can't run full dynticks because arch doesn't support irq
work self-IPIs

after this patch:
NO_HZ: Full dynticks CPUs: <CPU list>.

Tested against 3.19.

CC: Frederic Weisbecker <[email protected]>
CC: Paul E. McKenney <[email protected]>
Signed-off-by: Paul A. Clarke <[email protected]>

diff --git a/arch/powerpc/include/asm/irq_work.h
b/arch/powerpc/include/asm/irq_work.h
new file mode 100644
index 0000000..18365ec
--- /dev/null
+++ b/arch/powerpc/include/asm/irq_work.h
@@ -0,0 +1,11 @@
+#ifndef _ASM_IRQ_WORK_H
+#define _ASM_IRQ_WORK_H
+
+#include <asm/processor.h>
+
+static inline bool arch_irq_work_has_interrupt(void)
+{
+ return 1;
+}
+
+#endif /* _ASM_IRQ_WORK_H */

--
Regards,
Paul Clarke, IBM


2015-02-13 20:42:40

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] powerpc: re-enable dynticks

On Fri, Feb 13, 2015 at 01:38:36PM -0600, Paul Clarke wrote:
> implement arch_irq_work_has_interrupt() for powerpc
>
> Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for
> full dynamic ticks to be enabled, by expecting architectures to
> implement a suitable arch_irq_work_has_interrupt() routine.
>
> Several arches have implemented this routine, including x86
> (3010279f) and arm (09f6edd4), but powerpc was omitted.
>
> This patch implements this routine for powerpc.
>
> The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU
> list>" is displayed:
> NO_HZ: Can't run full dynticks because arch doesn't support irq
> work self-IPIs
>
> after this patch:
> NO_HZ: Full dynticks CPUs: <CPU list>.
>
> Tested against 3.19.
>
> CC: Frederic Weisbecker <[email protected]>
> CC: Paul E. McKenney <[email protected]>
> Signed-off-by: Paul A. Clarke <[email protected]>

Acked-by: Paul E. McKenney <[email protected]>

> diff --git a/arch/powerpc/include/asm/irq_work.h
> b/arch/powerpc/include/asm/irq_work.h
> new file mode 100644
> index 0000000..18365ec
> --- /dev/null
> +++ b/arch/powerpc/include/asm/irq_work.h
> @@ -0,0 +1,11 @@
> +#ifndef _ASM_IRQ_WORK_H
> +#define _ASM_IRQ_WORK_H
> +
> +#include <asm/processor.h>
> +
> +static inline bool arch_irq_work_has_interrupt(void)
> +{
> + return 1;
> +}
> +
> +#endif /* _ASM_IRQ_WORK_H */
>
> --
> Regards,
> Paul Clarke, IBM
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/

2015-02-16 00:08:42

by Michael Ellerman

[permalink] [raw]
Subject: Re: [PATCH] powerpc: re-enable dynticks

On Fri, 2015-02-13 at 13:38 -0600, Paul Clarke wrote:
> implement arch_irq_work_has_interrupt() for powerpc
>
> Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for
> full dynamic ticks to be enabled, by expecting architectures to
> implement a suitable arch_irq_work_has_interrupt() routine.
>
> Several arches have implemented this routine, including x86 (3010279f)
> and arm (09f6edd4), but powerpc was omitted.
>
> This patch implements this routine for powerpc.
>
> The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU
> list>" is displayed:
> NO_HZ: Can't run full dynticks because arch doesn't support irq
> work self-IPIs
>
> after this patch:
> NO_HZ: Full dynticks CPUs: <CPU list>.
>
> Tested against 3.19.

It makes the message change, but is that correct? ie. do we actually implement
"IRQ work self-IPIs"?

> diff --git a/arch/powerpc/include/asm/irq_work.h
> b/arch/powerpc/include/asm/irq_work.h
> new file mode 100644
> index 0000000..18365ec
> --- /dev/null
> +++ b/arch/powerpc/include/asm/irq_work.h
> @@ -0,0 +1,11 @@
> +#ifndef _ASM_IRQ_WORK_H
> +#define _ASM_IRQ_WORK_H
> +
> +#include <asm/processor.h>
> +
> +static inline bool arch_irq_work_has_interrupt(void)
> +{
> + return 1;

Should be "true";

> +}

cheers

2015-02-16 04:07:06

by Benjamin Herrenschmidt

[permalink] [raw]
Subject: Re: [PATCH] powerpc: re-enable dynticks

On Mon, 2015-02-16 at 11:08 +1100, Michael Ellerman wrote:
> On Fri, 2015-02-13 at 13:38 -0600, Paul Clarke wrote:
> > implement arch_irq_work_has_interrupt() for powerpc
> >
> > Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for
> > full dynamic ticks to be enabled, by expecting architectures to
> > implement a suitable arch_irq_work_has_interrupt() routine.
> >
> > Several arches have implemented this routine, including x86 (3010279f)
> > and arm (09f6edd4), but powerpc was omitted.
> >
> > This patch implements this routine for powerpc.
> >
.../...
>
> It makes the message change, but is that correct? ie. do we actually implement
> "IRQ work self-IPIs"?

I think so... Fred, do you think what we do will work ? We hijack our
decrementer (local timer) by making it shoot almost immediately (1 tick
away) and run the irq work at the beginning of __timer_interrupt().

At that point we are on our irq stack and have done irq_enter but that's
about it.

Cheers,
Ben.

> > diff --git a/arch/powerpc/include/asm/irq_work.h
> > b/arch/powerpc/include/asm/irq_work.h
> > new file mode 100644
> > index 0000000..18365ec
> > --- /dev/null
> > +++ b/arch/powerpc/include/asm/irq_work.h
> > @@ -0,0 +1,11 @@
> > +#ifndef _ASM_IRQ_WORK_H
> > +#define _ASM_IRQ_WORK_H
> > +
> > +#include <asm/processor.h>
> > +
> > +static inline bool arch_irq_work_has_interrupt(void)
> > +{
> > + return 1;
>
> Should be "true";
>
> > +}
>
> cheers
>
>
> _______________________________________________
> Linuxppc-dev mailing list
> [email protected]
> https://lists.ozlabs.org/listinfo/linuxppc-dev

2015-02-20 17:08:47

by Paul A. Clarke

[permalink] [raw]
Subject: [PATCH v2] powerpc: re-enable dynticks

implement arch_irq_work_has_interrupt() for powerpc

Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for
full dynamic ticks to be enabled, by expecting architectures to
implement a suitable arch_irq_work_has_interrupt() routine.

Several arches have implemented this routine, including x86 (3010279f)
and arm (09f6edd4), but powerpc was omitted.

This patch implements this routine for powerpc.

The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU
list>" is displayed:
NO_HZ: Can't run full dynticks because arch doesn't support irq
work self-IPIs

after this patch:
NO_HZ: Full dynticks CPUs: <CPU list>.

Tested against 3.19.

v2: changed "return 1" to "return true", per Michael Ellerman

CC: Frederic Weisbecker <[email protected]>
CC: Paul E. McKenney <[email protected]>
Signed-off-by: Paul A. Clarke <[email protected]>

diff --git a/arch/powerpc/include/asm/irq_work.h
b/arch/powerpc/include/asm/irq_work.h
new file mode 100644
index 0000000..99cc0aa
--- /dev/null
+++ b/arch/powerpc/include/asm/irq_work.h
@@ -0,0 +1,11 @@
+#ifndef _ASM_IRQ_WORK_H
+#define _ASM_IRQ_WORK_H
+
+#include <asm/processor.h>
+
+static inline bool arch_irq_work_has_interrupt(void)
+{
+ return true;
+}
+
+#endif /* _ASM_IRQ_WORK_H */

2015-02-20 17:13:39

by Paul A. Clarke

[permalink] [raw]
Subject: [PATCH v2] powerpc: re-enable dynticks


implement arch_irq_work_has_interrupt() for powerpc

(resending because I messed up the e-mail addresses)

Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for
full dynamic ticks to be enabled, by expecting architectures to
implement a suitable arch_irq_work_has_interrupt() routine.

Several arches have implemented this routine, including x86 (3010279f)
and arm (09f6edd4), but powerpc was omitted.

This patch implements this routine for powerpc.

The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU
list>" is displayed:
NO_HZ: Can't run full dynticks because arch doesn't support irq
work self-IPIs

after this patch:
NO_HZ: Full dynticks CPUs: <CPU list>.

Tested against 3.19.

v2: changed "return 1" to "return true", per Michael Ellerman

CC: Frederic Weisbecker <[email protected]>
CC: Paul E. McKenney <[email protected]>
Signed-off-by: Paul A. Clarke <[email protected]>

diff --git a/arch/powerpc/include/asm/irq_work.h
b/arch/powerpc/include/asm/irq_work.h
new file mode 100644
index 0000000..99cc0aa
--- /dev/null
+++ b/arch/powerpc/include/asm/irq_work.h
@@ -0,0 +1,11 @@
+#ifndef _ASM_IRQ_WORK_H
+#define _ASM_IRQ_WORK_H
+
+#include <asm/processor.h>
+
+static inline bool arch_irq_work_has_interrupt(void)
+{
+ return true;
+}
+
+#endif /* _ASM_IRQ_WORK_H */

2015-02-21 05:18:52

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH v2] powerpc: re-enable dynticks

On Fri, Feb 20, 2015 at 11:13:33AM -0600, Paul Clarke wrote:
>
> implement arch_irq_work_has_interrupt() for powerpc
>
> (resending because I messed up the e-mail addresses)
>
> Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for
> full dynamic ticks to be enabled, by expecting architectures to
> implement a suitable arch_irq_work_has_interrupt() routine.
>
> Several arches have implemented this routine, including x86
> (3010279f) and arm (09f6edd4), but powerpc was omitted.
>
> This patch implements this routine for powerpc.
>
> The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU
> list>" is displayed:
> NO_HZ: Can't run full dynticks because arch doesn't support irq
> work self-IPIs
>
> after this patch:
> NO_HZ: Full dynticks CPUs: <CPU list>.
>
> Tested against 3.19.
>
> v2: changed "return 1" to "return true", per Michael Ellerman
>
> CC: Frederic Weisbecker <[email protected]>
> CC: Paul E. McKenney <[email protected]>
> Signed-off-by: Paul A. Clarke <[email protected]>

Reviewed-by: Paul E. McKenney <[email protected]>

> diff --git a/arch/powerpc/include/asm/irq_work.h
> b/arch/powerpc/include/asm/irq_work.h
> new file mode 100644
> index 0000000..99cc0aa
> --- /dev/null
> +++ b/arch/powerpc/include/asm/irq_work.h
> @@ -0,0 +1,11 @@
> +#ifndef _ASM_IRQ_WORK_H
> +#define _ASM_IRQ_WORK_H
> +
> +#include <asm/processor.h>
> +
> +static inline bool arch_irq_work_has_interrupt(void)
> +{
> + return true;
> +}
> +
> +#endif /* _ASM_IRQ_WORK_H */

2015-02-22 22:14:00

by Frederic Weisbecker

[permalink] [raw]
Subject: Re: [PATCH] powerpc: re-enable dynticks

Hi Ben,

2015-02-16 5:06 GMT+01:00 Benjamin Herrenschmidt <[email protected]>:
> On Mon, 2015-02-16 at 11:08 +1100, Michael Ellerman wrote:
>> On Fri, 2015-02-13 at 13:38 -0600, Paul Clarke wrote:
>> > implement arch_irq_work_has_interrupt() for powerpc
>> >
>> > Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for
>> > full dynamic ticks to be enabled, by expecting architectures to
>> > implement a suitable arch_irq_work_has_interrupt() routine.
>> >
>> > Several arches have implemented this routine, including x86 (3010279f)
>> > and arm (09f6edd4), but powerpc was omitted.
>> >
>> > This patch implements this routine for powerpc.
>> >
> .../...
>>
>> It makes the message change, but is that correct? ie. do we actually implement
>> "IRQ work self-IPIs"?
>
> I think so... Fred, do you think what we do will work ? We hijack our
> decrementer (local timer) by making it shoot almost immediately (1 tick
> away) and run the irq work at the beginning of __timer_interrupt().
>
> At that point we are on our irq stack and have done irq_enter but that's
> about it.

Yes that should work. After all "self-IPI" is an oxymoron. One would
expect an IPI to be triggered by an irq controller but if such
operation isn't supported with the current CPU being both source and
destination, anything triggering the desired callback in an interrupt
context in a reasonable amount of time ahead does the job here.

I thought well that's what powerpc was doing for irq work but I wasn't
sure I understood the code correctly. I should have pinged people
about that, sorry.

Thanks.

>
> Cheers,
> Ben.
>
>> > diff --git a/arch/powerpc/include/asm/irq_work.h
>> > b/arch/powerpc/include/asm/irq_work.h
>> > new file mode 100644
>> > index 0000000..18365ec
>> > --- /dev/null
>> > +++ b/arch/powerpc/include/asm/irq_work.h
>> > @@ -0,0 +1,11 @@
>> > +#ifndef _ASM_IRQ_WORK_H
>> > +#define _ASM_IRQ_WORK_H
>> > +
>> > +#include <asm/processor.h>
>> > +
>> > +static inline bool arch_irq_work_has_interrupt(void)
>> > +{
>> > + return 1;
>>
>> Should be "true";
>>
>> > +}
>>
>> cheers
>>
>>
>> _______________________________________________
>> Linuxppc-dev mailing list
>> [email protected]
>> https://lists.ozlabs.org/listinfo/linuxppc-dev
>
>

2015-02-22 22:30:38

by Benjamin Herrenschmidt

[permalink] [raw]
Subject: Re: [PATCH] powerpc: re-enable dynticks

On Sun, 2015-02-22 at 23:13 +0100, Frederic Weisbecker wrote:
> Yes that should work. After all "self-IPI" is an oxymoron. One would
> expect an IPI to be triggered by an irq controller but if such
> operation isn't supported with the current CPU being both source and
> destination, anything triggering the desired callback in an interrupt
> context in a reasonable amount of time ahead does the job here.

We could do self-IPI on platforms that have an SMP-capable interrupt
controller too but it would probably have higher overhead and would
require verifying that the code for each of our different interrupt
controllers is safe to be called from NMIs (hint: ioremap space isn't
safe to access from NMIs for us on some CPU families...).

We might be able to do better than using the decrementer on some CPUs by
using local doorbells, but for now this will do.

> I thought well that's what powerpc was doing for irq work but I wasn't
> sure I understood the code correctly. I should have pinged people
> about that, sorry.

No worries,

Cheers,
Ben.