Message-ID: <558A53C0.5030700@hitachi.com>
Date: Wed, 24 Jun 2015 15:52:48 +0900
From: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Organization: Hitachi, Ltd., Japan
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0
MIME-Version: 1.0
To: dwalker@fifo99.com, Ralf Baechle <ralf@linux-mips.org>,
        linux-mips@linux-mips.org, david.daney@cavium.com
CC: d.hatayama@jp.fujitsu.com, vgoyal@redhat.com,
        hidehiro.kawai.ez@hitachi.com, ebiederm@xmission.com,
        linux-kernel@vger.kernel.org
Subject: Re: kexec crash kernel running with watchdog enabled
References: <20150623140548.GA15591@fifo99.com>
In-Reply-To: <20150623140548.GA15591@fifo99.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2216
Lines: 57

Hi,

On 2015/06/23 23:05, dwalker@fifo99.com wrote:
> 
> Hi,
> 
> There was a commit in kernel/panic.c which altered when the kexec crash kernel is executed,
> 
> commit f06e5153f4ae2e2f3b0300f0e260e40cb7fefd45
> Author: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Date:   Fri Jun 6 14:37:07 2014 -0700
> 
>     kernel/panic.c: add "crash_kexec_post_notifiers" option for kdump after panic_notifers
> 
> 
> This made it possible for smp_send_stop() to be executed prior to calling the kexec crash
> kernel.
> 
> The issue is that smp_send_stop() offlines the cores, and other code depend on the cores being online.
> 
> In my case on Octeon here's an example,
> 
> panic()
>  crash_kexec()
>   machine_crash_shutdown()
>    octeon_generic_shutdown()
> 
> Inside octeon_generic_shutdown() the Octeon watchdog is shutdown for_each_online_cpu(), but since
> most of the cpu's already got offlined in smp_send_stop() it means the watchdog is still alive on
> those cores. This results in a reboot during the crash kernel execution.

Ah, I see.

> Another example seem to be in default_machine_crash_shutdown() where crash_kexec_prepare_cpus() depends
> on an IPI for saving the registers on different cores. However, the cpu's are all offlined with
> interrupts disabled so they won't be running those IPI's in this case.
> 
> I'm looking for any advice on how this should be fixed, or if it's already fixed. I'm not going to be
> submitting a patch so if anyone wants to submit one feel free to do so.

Hmm, IMHO, when the cpu goes to offline in appropriate way(smp_send_stop), it should stop
watchdog timer on the offlined cpu too.
Or, you can also register crash handler which stops all watchdogs, but it's a bit tricky.

Thank you,

-- 
Masami HIRAMATSU
Linux Technology Research Center, System Productivity Research Dept.
Center for Technology Innovation - Systems Engineering
Hitachi, Ltd., Research & Development Group
E-mail: masami.hiramatsu.pt@hitachi.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/