2013-07-09 15:37:06

by Stephen Boyd

[permalink] [raw]
Subject: [PATCH] ARM: Fix deadlock scenario with smp_send_stop()

If one process calls sys_reboot and that process then stops other
CPUs while those CPUs are within a spin_lock() region we can
potentially encounter a deadlock scenario like below.

CPU 0 CPU 1
----- -----
spin_lock(my_lock)
smp_send_stop()
<send IPI> handle_IPI()
disable_preemption/irqs
while(1);
<PREEMPT>
spin_lock(my_lock) <--- Waits forever

We shouldn't attempt to run any other tasks after we send a stop
IPI to a CPU so disable preemption so that this task runs to
completion.

Reported-by: Sundarajan Srinivasan <[email protected]>
Signed-off-by: Stephen Boyd <[email protected]>
---

Resending this patch now that the context has changed.

arch/arm/kernel/process.c | 3 +++
1 file changed, 3 insertions(+)

diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c
index 7f1efcd..8bc12d7 100644
--- a/arch/arm/kernel/process.c
+++ b/arch/arm/kernel/process.c
@@ -206,6 +206,7 @@ void machine_shutdown(void)
*/
void machine_halt(void)
{
+ preempt_disable();
smp_send_stop();

local_irq_disable();
@@ -220,6 +221,7 @@ void machine_halt(void)
*/
void machine_power_off(void)
{
+ preempt_disable();
smp_send_stop();

if (pm_power_off)
@@ -239,6 +241,7 @@ void machine_power_off(void)
*/
void machine_restart(char *cmd)
{
+ preempt_disable();
smp_send_stop();

arm_pm_restart(reboot_mode, cmd);
--
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation


2013-07-24 18:56:24

by Stephen Boyd

[permalink] [raw]
Subject: Re: [PATCH] ARM: Fix deadlock scenario with smp_send_stop()

On 07/09, Stephen Boyd wrote:
> If one process calls sys_reboot and that process then stops other
> CPUs while those CPUs are within a spin_lock() region we can
> potentially encounter a deadlock scenario like below.
>
> CPU 0 CPU 1
> ----- -----
> spin_lock(my_lock)
> smp_send_stop()
> <send IPI> handle_IPI()
> disable_preemption/irqs
> while(1);
> <PREEMPT>
> spin_lock(my_lock) <--- Waits forever
>
> We shouldn't attempt to run any other tasks after we send a stop
> IPI to a CPU so disable preemption so that this task runs to
> completion.
>
> Reported-by: Sundarajan Srinivasan <[email protected]>
> Signed-off-by: Stephen Boyd <[email protected]>
> ---
>
> Resending this patch now that the context has changed.

Ping? Shall I put this in the patch tracker?

>
> arch/arm/kernel/process.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c
> index 7f1efcd..8bc12d7 100644
> --- a/arch/arm/kernel/process.c
> +++ b/arch/arm/kernel/process.c
> @@ -206,6 +206,7 @@ void machine_shutdown(void)
> */
> void machine_halt(void)
> {
> + preempt_disable();
> smp_send_stop();
>
> local_irq_disable();
> @@ -220,6 +221,7 @@ void machine_halt(void)
> */
> void machine_power_off(void)
> {
> + preempt_disable();
> smp_send_stop();
>
> if (pm_power_off)
> @@ -239,6 +241,7 @@ void machine_power_off(void)
> */
> void machine_restart(char *cmd)
> {
> + preempt_disable();
> smp_send_stop();
>
> arm_pm_restart(reboot_mode, cmd);

--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

2013-07-24 20:22:00

by Russell King - ARM Linux

[permalink] [raw]
Subject: Re: [PATCH] ARM: Fix deadlock scenario with smp_send_stop()

On Wed, Jul 24, 2013 at 11:56:18AM -0700, Stephen Boyd wrote:
> On 07/09, Stephen Boyd wrote:
> > If one process calls sys_reboot and that process then stops other
> > CPUs while those CPUs are within a spin_lock() region we can
> > potentially encounter a deadlock scenario like below.
> >
> > CPU 0 CPU 1
> > ----- -----
> > spin_lock(my_lock)
> > smp_send_stop()
> > <send IPI> handle_IPI()
> > disable_preemption/irqs
> > while(1);
> > <PREEMPT>
> > spin_lock(my_lock) <--- Waits forever
> >
> > We shouldn't attempt to run any other tasks after we send a stop
> > IPI to a CPU so disable preemption so that this task runs to
> > completion.
> >
> > Reported-by: Sundarajan Srinivasan <[email protected]>
> > Signed-off-by: Stephen Boyd <[email protected]>
> > ---
> >
> > Resending this patch now that the context has changed.
>
> Ping? Shall I put this in the patch tracker?

Well, looking at x86, they use local_irq_disable() before sending the
stop, so I think we should do the same for cross-arch consistency.

2013-07-24 20:29:02

by Stephen Boyd

[permalink] [raw]
Subject: Re: [PATCH] ARM: Fix deadlock scenario with smp_send_stop()

On 07/24/13 13:21, Russell King - ARM Linux wrote:
> On Wed, Jul 24, 2013 at 11:56:18AM -0700, Stephen Boyd wrote:
>> On 07/09, Stephen Boyd wrote:
>>> If one process calls sys_reboot and that process then stops other
>>> CPUs while those CPUs are within a spin_lock() region we can
>>> potentially encounter a deadlock scenario like below.
>>>
>>> CPU 0 CPU 1
>>> ----- -----
>>> spin_lock(my_lock)
>>> smp_send_stop()
>>> <send IPI> handle_IPI()
>>> disable_preemption/irqs
>>> while(1);
>>> <PREEMPT>
>>> spin_lock(my_lock) <--- Waits forever
>>>
>>> We shouldn't attempt to run any other tasks after we send a stop
>>> IPI to a CPU so disable preemption so that this task runs to
>>> completion.
>>>
>>> Reported-by: Sundarajan Srinivasan <[email protected]>
>>> Signed-off-by: Stephen Boyd <[email protected]>
>>> ---
>>>
>>> Resending this patch now that the context has changed.
>> Ping? Shall I put this in the patch tracker?
> Well, looking at x86, they use local_irq_disable() before sending the
> stop, so I think we should do the same for cross-arch consistency.

Fair enough. I'll send v2 with that.

--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

2013-07-24 20:36:15

by Stephen Boyd

[permalink] [raw]
Subject: [PATCH v2] ARM: Fix deadlock scenario with smp_send_stop()

If one process calls sys_reboot and that process then stops other
CPUs while those CPUs are within a spin_lock() region we can
potentially encounter a deadlock scenario like below.

CPU 0 CPU 1
----- -----
spin_lock(my_lock)
smp_send_stop()
<send IPI> handle_IPI()
disable_preemption/irqs
while(1);
<PREEMPT>
spin_lock(my_lock) <--- Waits forever

We shouldn't attempt to run any other tasks after we send a stop
IPI to a CPU so disable preemption so that this task runs to
completion. We use local_irq_disable() here for cross-arch
consistency with x86.

Reported-by: Sundarajan Srinivasan <[email protected]>
Signed-off-by: Stephen Boyd <[email protected]>
---

Changes since v1:
- Use local_irq_disable() instead of preempt_disable()

arch/arm/kernel/process.c | 3 +++
1 file changed, 3 insertions(+)

diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c
index d3ca4f6..08b47eb 100644
--- a/arch/arm/kernel/process.c
+++ b/arch/arm/kernel/process.c
@@ -197,6 +197,7 @@ void machine_shutdown(void)
*/
void machine_halt(void)
{
+ local_irq_disable();
smp_send_stop();

local_irq_disable();
@@ -211,6 +212,7 @@ void machine_halt(void)
*/
void machine_power_off(void)
{
+ local_irq_disable();
smp_send_stop();

if (pm_power_off)
@@ -230,6 +232,7 @@ void machine_power_off(void)
*/
void machine_restart(char *cmd)
{
+ local_irq_disable();
smp_send_stop();

arm_pm_restart(reboot_mode, cmd);
--
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
hosted by The Linux Foundation

2013-07-30 22:12:33

by Stephen Boyd

[permalink] [raw]
Subject: Re: [PATCH v2] ARM: Fix deadlock scenario with smp_send_stop()

On 07/24/13 13:36, Stephen Boyd wrote:
> If one process calls sys_reboot and that process then stops other
> CPUs while those CPUs are within a spin_lock() region we can
> potentially encounter a deadlock scenario like below.
>
> CPU 0 CPU 1
> ----- -----
> spin_lock(my_lock)
> smp_send_stop()
> <send IPI> handle_IPI()
> disable_preemption/irqs
> while(1);
> <PREEMPT>
> spin_lock(my_lock) <--- Waits forever
>
> We shouldn't attempt to run any other tasks after we send a stop
> IPI to a CPU so disable preemption so that this task runs to
> completion. We use local_irq_disable() here for cross-arch
> consistency with x86.
>
> Reported-by: Sundarajan Srinivasan <[email protected]>
> Signed-off-by: Stephen Boyd <[email protected]>
> ---

Ok I threw this into the patch tracker because there were no more comments.

> Changes since v1:
> - Use local_irq_disable() instead of preempt_disable()
>
> arch/arm/kernel/process.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c
> index d3ca4f6..08b47eb 100644
> --- a/arch/arm/kernel/process.c
> +++ b/arch/arm/kernel/process.c
> @@ -197,6 +197,7 @@ void machine_shutdown(void)
> */
> void machine_halt(void)
> {
> + local_irq_disable();
> smp_send_stop();
>
> local_irq_disable();
> @@ -211,6 +212,7 @@ void machine_halt(void)
> */
> void machine_power_off(void)
> {
> + local_irq_disable();
> smp_send_stop();
>
> if (pm_power_off)
> @@ -230,6 +232,7 @@ void machine_power_off(void)
> */
> void machine_restart(char *cmd)
> {
> + local_irq_disable();
> smp_send_stop();
>
> arm_pm_restart(reboot_mode, cmd);


--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation