During our testing we noticed that nmi watchdogs in sparc could not be disabled or
enabled dynamically using sysctl/proc interface. Sparc uses its own arch specific
nmi watchdogs. There is a sysctl and proc interface(proc/sys/kernel/nmi_watchdog)
to enable/disable nmi watchdogs. However, that is not working for sparc. There
is no interface to feed this parameter to arch specific nmi watchdogs.
These patches extend the same sysctl/proc interface to enable or disable
these arch specific nmi watchdogs dynamically. Introduced new functions
arch_watchdog_nmi_enable and arch_watchdog_nmi_disable which can be implemented
in arch specific handlers.
If you think there is a better way to do this. Please advice.
Tested on sparc. Compile tested on x86.
v2:
a)Sam Ravnborg's comments about making the definitions visible.
With the new approach we dont need those definitions((NMI_WATCHDOG_ENABLED,
SOFT_WATCHDOG_ENABLED etc..) outside watchdog.c. So no action.
b) Made changes per Don Zickus comments.
Don, I could not use your patches as is. Reason is sparc does not define
CONFIG_HARDLOCKUP_DETECTOR. So, defining default __weak function did not
work for me. However, I have used your idea to define __weak functions
arch_watchdog_nmi_enable and arch_watchdog_nmi_disable when CONFIG_HARDLOCKUP_DETECTOR
is not defined. I feel this should have very less impact on the races you are
concerned about. Please take a look. Feel free to suggest.
Patch2 changes: I had to introduce new variable nmi_init_done to synchronize
watchdog thread and kernel init thread.
v1:
Initial version. Discussion thread here
http://www.mail-archive.com/[email protected]/msg1245427.html
Babu Moger (2):
watchdog: Introduce arch_watchdog_nmi_enable and
arch_watchdog_nmi_disable
sparc: Implement arch_watchdog_nmi_enable and
arch_watchdog_nmi_disable
arch/sparc/kernel/nmi.c | 41 +++++++++++++++++++++++++++++-
kernel/watchdog.c | 65 +++++++++++++++++++++++++++++++---------------
2 files changed, 84 insertions(+), 22 deletions(-)
Currently we do not have a way to enable/disable arch specific
watchdog handlers if it was implemented by any of the architectures.
This patch introduces new functions arch_watchdog_nmi_enable and
arch_watchdog_nmi_disable which can be used to enable/disable architecture
specific NMI watchdog handlers. These functions are defined as weak as
architectures can override their definitions to enable/disable nmi
watchdog behaviour.
Signed-off-by: Babu Moger <[email protected]>
---
kernel/watchdog.c | 65 +++++++++++++++++++++++++++++++++++-----------------
1 files changed, 44 insertions(+), 21 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 9acb29f..d1e84e6 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -46,7 +46,7 @@
static DEFINE_MUTEX(watchdog_proc_mutex);
-#ifdef CONFIG_HARDLOCKUP_DETECTOR
+#if defined(CONFIG_HARDLOCKUP_DETECTOR) || defined(CONFIG_HAVE_NMI_WATCHDOG)
static unsigned long __read_mostly watchdog_enabled = SOFT_WATCHDOG_ENABLED|NMI_WATCHDOG_ENABLED;
#else
static unsigned long __read_mostly watchdog_enabled = SOFT_WATCHDOG_ENABLED;
@@ -585,15 +585,11 @@ static void watchdog(unsigned int cpu)
*/
static unsigned long cpu0_err;
-static int watchdog_nmi_enable(unsigned int cpu)
+static int arch_watchdog_nmi_enable(unsigned int cpu)
{
struct perf_event_attr *wd_attr;
struct perf_event *event = per_cpu(watchdog_ev, cpu);
- /* nothing to do if the hard lockup detector is disabled */
- if (!(watchdog_enabled & NMI_WATCHDOG_ENABLED))
- goto out;
-
/* is it already setup and enabled? */
if (event && event->state > PERF_EVENT_STATE_OFF)
goto out;
@@ -619,18 +615,6 @@ static int watchdog_nmi_enable(unsigned int cpu)
goto out_save;
}
- /*
- * Disable the hard lockup detector if _any_ CPU fails to set up
- * set up the hardware perf event. The watchdog() function checks
- * the NMI_WATCHDOG_ENABLED bit periodically.
- *
- * The barriers are for syncing up watchdog_enabled across all the
- * cpus, as clear_bit() does not use barriers.
- */
- smp_mb__before_atomic();
- clear_bit(NMI_WATCHDOG_ENABLED_BIT, &watchdog_enabled);
- smp_mb__after_atomic();
-
/* skip displaying the same error again */
if (cpu > 0 && (PTR_ERR(event) == cpu0_err))
return PTR_ERR(event);
@@ -658,7 +642,7 @@ out:
return 0;
}
-static void watchdog_nmi_disable(unsigned int cpu)
+static void arch_watchdog_nmi_disable(unsigned int cpu)
{
struct perf_event *event = per_cpu(watchdog_ev, cpu);
@@ -676,8 +660,13 @@ static void watchdog_nmi_disable(unsigned int cpu)
}
#else
-static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
-static void watchdog_nmi_disable(unsigned int cpu) { return; }
+/*
+ * These two functions are mostly architecture specific
+ * defining them as weak here.
+ */
+int __weak arch_watchdog_nmi_enable(unsigned int cpu) { return 0; }
+void __weak arch_watchdog_nmi_disable(unsigned int cpu) { return; }
+
#endif /* CONFIG_HARDLOCKUP_DETECTOR */
static struct smp_hotplug_thread watchdog_threads = {
@@ -781,6 +770,40 @@ void lockup_detector_resume(void)
put_online_cpus();
}
+void watchdog_nmi_disable(unsigned int cpu)
+{
+ arch_watchdog_nmi_disable(cpu);
+}
+
+int watchdog_nmi_enable(unsigned int cpu)
+{
+ int err;
+
+ /* nothing to do if the hard lockup detector is disabled */
+ if (!(watchdog_enabled & NMI_WATCHDOG_ENABLED))
+ return 0;
+
+ err = arch_watchdog_nmi_enable(cpu);
+
+ if (err) {
+ /*
+ * Disable the hard lockup detector if _any_ CPU fails to set up
+ * set up the hardware perf event. The watchdog() function checks
+ * the NMI_WATCHDOG_ENABLED bit periodically.
+ *
+ * The barriers are for syncing up watchdog_enabled across all the
+ * cpus, as clear_bit() does not use barriers.
+ */
+ smp_mb__before_atomic();
+ clear_bit(NMI_WATCHDOG_ENABLED_BIT, &watchdog_enabled);
+ smp_mb__after_atomic();
+
+ return err;
+ }
+
+ return 0;
+}
+
static int update_watchdog_all_cpus(void)
{
int ret;
--
1.7.1
Implement functions arch_watchdog_nmi_enable and arch_watchdog_nmi_disable
to enable/disable nmi watchdog. Sparc uses arch specific nmi watchdog
handler. Currently, we do not have a way to enable/disable nmi watchdog
dynamically. With these patches we can enable or disable arch
specific nmi watchdogs using proc or sysctl interface.
Example commands.
To enable: echo 1 > /proc/sys/kernel/nmi_watchdog
To disable: echo 0 > /proc/sys/kernel/nmi_watchdog
It can also achieved using the sysctl parameter kernel.nmi_watchdog
Signed-off-by: Babu Moger <[email protected]>
---
arch/sparc/kernel/nmi.c | 41 ++++++++++++++++++++++++++++++++++++++++-
1 files changed, 40 insertions(+), 1 deletions(-)
diff --git a/arch/sparc/kernel/nmi.c b/arch/sparc/kernel/nmi.c
index a9973bb..d7e2c01 100644
--- a/arch/sparc/kernel/nmi.c
+++ b/arch/sparc/kernel/nmi.c
@@ -42,7 +42,7 @@ static int panic_on_timeout;
*/
atomic_t nmi_active = ATOMIC_INIT(0); /* oprofile uses this */
EXPORT_SYMBOL(nmi_active);
-
+static int nmi_init_done;
static unsigned int nmi_hz = HZ;
static DEFINE_PER_CPU(short, wd_enabled);
static int endflag __initdata;
@@ -153,6 +153,8 @@ static void report_broken_nmi(int cpu, int *prev_nmi_count)
void stop_nmi_watchdog(void *unused)
{
+ if (!__this_cpu_read(wd_enabled))
+ return;
pcr_ops->write_pcr(0, pcr_ops->pcr_nmi_disable);
__this_cpu_write(wd_enabled, 0);
atomic_dec(&nmi_active);
@@ -207,6 +209,9 @@ error:
void start_nmi_watchdog(void *unused)
{
+ if (__this_cpu_read(wd_enabled))
+ return;
+
__this_cpu_write(wd_enabled, 1);
atomic_inc(&nmi_active);
@@ -259,6 +264,8 @@ int __init nmi_init(void)
}
}
+ nmi_init_done = 1;
+
return err;
}
@@ -270,3 +277,35 @@ static int __init setup_nmi_watchdog(char *str)
return 0;
}
__setup("nmi_watchdog=", setup_nmi_watchdog);
+
+/*
+ * sparc specific NMI watchdog enable function.
+ * Enables watchdog if it is not enabled already.
+ */
+int arch_watchdog_nmi_enable(unsigned int cpu)
+{
+ if (atomic_read(&nmi_active) == -1) {
+ pr_info_once("NMI watchdog cannot be enabled\n");
+ return -1;
+ }
+
+ /*
+ * watchdog thread could start even before nmi_init is called.
+ * Just Return in that case. Let nmi_init finish the init
+ * process first.
+ */
+ if (!nmi_init_done)
+ return 0;
+
+ smp_call_function_single(cpu, start_nmi_watchdog, NULL, 1);
+
+ return 0;
+}
+/*
+ * sparc specific NMI watchdog disable function.
+ * Disables watchdog if it is not disabled already.
+ */
+void arch_watchdog_nmi_disable(unsigned int cpu)
+{
+ smp_call_function_single(cpu, stop_nmi_watchdog, NULL, 1);
+}
--
1.7.1
On Thu, Oct 13, 2016 at 01:38:00PM -0700, Babu Moger wrote:
> During our testing we noticed that nmi watchdogs in sparc could not be disabled or
> enabled dynamically using sysctl/proc interface. Sparc uses its own arch specific
> nmi watchdogs. There is a sysctl and proc interface(proc/sys/kernel/nmi_watchdog)
> to enable/disable nmi watchdogs. However, that is not working for sparc. There
> is no interface to feed this parameter to arch specific nmi watchdogs.
>
> These patches extend the same sysctl/proc interface to enable or disable
> these arch specific nmi watchdogs dynamically. Introduced new functions
> arch_watchdog_nmi_enable and arch_watchdog_nmi_disable which can be implemented
> in arch specific handlers.
> If you think there is a better way to do this. Please advice.
>
> Tested on sparc. Compile tested on x86.
Thanks Babu! Looking through the code it seems appropriate with what I
suggested. I am running it through my testsuite and things are failing.
But it is most likely my system is misconfigured. Give me another day or so
to straighten that out.
Cheers,
Don
>
> v2:
> a)Sam Ravnborg's comments about making the definitions visible.
> With the new approach we dont need those definitions((NMI_WATCHDOG_ENABLED,
> SOFT_WATCHDOG_ENABLED etc..) outside watchdog.c. So no action.
>
> b) Made changes per Don Zickus comments.
> Don, I could not use your patches as is. Reason is sparc does not define
> CONFIG_HARDLOCKUP_DETECTOR. So, defining default __weak function did not
> work for me. However, I have used your idea to define __weak functions
> arch_watchdog_nmi_enable and arch_watchdog_nmi_disable when CONFIG_HARDLOCKUP_DETECTOR
> is not defined. I feel this should have very less impact on the races you are
> concerned about. Please take a look. Feel free to suggest.
>
> Patch2 changes: I had to introduce new variable nmi_init_done to synchronize
> watchdog thread and kernel init thread.
>
> v1:
> Initial version. Discussion thread here
> http://www.mail-archive.com/[email protected]/msg1245427.html
>
> Babu Moger (2):
> watchdog: Introduce arch_watchdog_nmi_enable and
> arch_watchdog_nmi_disable
> sparc: Implement arch_watchdog_nmi_enable and
> arch_watchdog_nmi_disable
>
> arch/sparc/kernel/nmi.c | 41 +++++++++++++++++++++++++++++-
> kernel/watchdog.c | 65 +++++++++++++++++++++++++++++++---------------
> 2 files changed, 84 insertions(+), 22 deletions(-)
>
On Thu, Oct 13, 2016 at 01:38:01PM -0700, Babu Moger wrote:
> Currently we do not have a way to enable/disable arch specific
> watchdog handlers if it was implemented by any of the architectures.
>
> This patch introduces new functions arch_watchdog_nmi_enable and
> arch_watchdog_nmi_disable which can be used to enable/disable architecture
> specific NMI watchdog handlers. These functions are defined as weak as
> architectures can override their definitions to enable/disable nmi
> watchdog behaviour.
Hi Babu,
This patch tested fine on my x86 box and I am ok with the changes.
I do have one small cosmetic request below for a failure path. Other than
that I will give my ack.
Cheers,
Don
>
> Signed-off-by: Babu Moger <[email protected]>
> ---
> kernel/watchdog.c | 65 +++++++++++++++++++++++++++++++++++-----------------
> 1 files changed, 44 insertions(+), 21 deletions(-)
>
> diff --git a/kernel/watchdog.c b/kernel/watchdog.c
> index 9acb29f..d1e84e6 100644
> --- a/kernel/watchdog.c
> +++ b/kernel/watchdog.c
> @@ -46,7 +46,7 @@
>
> static DEFINE_MUTEX(watchdog_proc_mutex);
>
> -#ifdef CONFIG_HARDLOCKUP_DETECTOR
> +#if defined(CONFIG_HARDLOCKUP_DETECTOR) || defined(CONFIG_HAVE_NMI_WATCHDOG)
> static unsigned long __read_mostly watchdog_enabled = SOFT_WATCHDOG_ENABLED|NMI_WATCHDOG_ENABLED;
> #else
> static unsigned long __read_mostly watchdog_enabled = SOFT_WATCHDOG_ENABLED;
> @@ -585,15 +585,11 @@ static void watchdog(unsigned int cpu)
> */
> static unsigned long cpu0_err;
>
> -static int watchdog_nmi_enable(unsigned int cpu)
> +static int arch_watchdog_nmi_enable(unsigned int cpu)
> {
> struct perf_event_attr *wd_attr;
> struct perf_event *event = per_cpu(watchdog_ev, cpu);
>
> - /* nothing to do if the hard lockup detector is disabled */
> - if (!(watchdog_enabled & NMI_WATCHDOG_ENABLED))
> - goto out;
> -
> /* is it already setup and enabled? */
> if (event && event->state > PERF_EVENT_STATE_OFF)
> goto out;
> @@ -619,18 +615,6 @@ static int watchdog_nmi_enable(unsigned int cpu)
> goto out_save;
> }
>
> - /*
> - * Disable the hard lockup detector if _any_ CPU fails to set up
> - * set up the hardware perf event. The watchdog() function checks
> - * the NMI_WATCHDOG_ENABLED bit periodically.
> - *
> - * The barriers are for syncing up watchdog_enabled across all the
> - * cpus, as clear_bit() does not use barriers.
> - */
> - smp_mb__before_atomic();
> - clear_bit(NMI_WATCHDOG_ENABLED_BIT, &watchdog_enabled);
> - smp_mb__after_atomic();
> -
> /* skip displaying the same error again */
> if (cpu > 0 && (PTR_ERR(event) == cpu0_err))
> return PTR_ERR(event);
In the arch_watchdog_nmi_enable code is a pr_info on failure
pr_info("Shutting down hard lockup detector on all cpus\n");
that should be moved to below..
> @@ -658,7 +642,7 @@ out:
> return 0;
> }
>
> -static void watchdog_nmi_disable(unsigned int cpu)
> +static void arch_watchdog_nmi_disable(unsigned int cpu)
> {
> struct perf_event *event = per_cpu(watchdog_ev, cpu);
>
> @@ -676,8 +660,13 @@ static void watchdog_nmi_disable(unsigned int cpu)
> }
>
> #else
> -static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
> -static void watchdog_nmi_disable(unsigned int cpu) { return; }
> +/*
> + * These two functions are mostly architecture specific
> + * defining them as weak here.
> + */
> +int __weak arch_watchdog_nmi_enable(unsigned int cpu) { return 0; }
> +void __weak arch_watchdog_nmi_disable(unsigned int cpu) { return; }
> +
> #endif /* CONFIG_HARDLOCKUP_DETECTOR */
>
> static struct smp_hotplug_thread watchdog_threads = {
> @@ -781,6 +770,40 @@ void lockup_detector_resume(void)
> put_online_cpus();
> }
>
> +void watchdog_nmi_disable(unsigned int cpu)
> +{
> + arch_watchdog_nmi_disable(cpu);
> +}
> +
> +int watchdog_nmi_enable(unsigned int cpu)
> +{
> + int err;
> +
> + /* nothing to do if the hard lockup detector is disabled */
> + if (!(watchdog_enabled & NMI_WATCHDOG_ENABLED))
> + return 0;
> +
> + err = arch_watchdog_nmi_enable(cpu);
> +
> + if (err) {
> + /*
> + * Disable the hard lockup detector if _any_ CPU fails to set up
> + * set up the hardware perf event. The watchdog() function checks
> + * the NMI_WATCHDOG_ENABLED bit periodically.
> + *
> + * The barriers are for syncing up watchdog_enabled across all the
> + * cpus, as clear_bit() does not use barriers.
> + */
> + smp_mb__before_atomic();
> + clear_bit(NMI_WATCHDOG_ENABLED_BIT, &watchdog_enabled);
> + smp_mb__after_atomic();
moved to here:
pr_info("Shutting down hard lockup detector on all cpus\n");
This lets the failure message be displayed on all arches instead of just
x86. Though I guess sparc does not call theirs 'hard lockup detector'.
Hmm...
> +
> + return err;
> + }
> +
> + return 0;
> +}
> +
> static int update_watchdog_all_cpus(void)
> {
> int ret;
> --
> 1.7.1
>
Don,
On 10/17/2016 12:31 PM, Don Zickus wrote:
> On Thu, Oct 13, 2016 at 01:38:01PM -0700, Babu Moger wrote:
>> Currently we do not have a way to enable/disable arch specific
>> watchdog handlers if it was implemented by any of the architectures.
>>
>> This patch introduces new functions arch_watchdog_nmi_enable and
>> arch_watchdog_nmi_disable which can be used to enable/disable architecture
>> specific NMI watchdog handlers. These functions are defined as weak as
>> architectures can override their definitions to enable/disable nmi
>> watchdog behaviour.
> Hi Babu,
>
> This patch tested fine on my x86 box and I am ok with the changes.
>
> I do have one small cosmetic request below for a failure path. Other than
> that I will give my ack.
Yes. I am testing these changes. If everything goes as expected, I will
post v3 version
tomorrow. Thanks Babu
>
> Cheers,
> Don
>
>> Signed-off-by: Babu Moger <[email protected]>
>> ---
>> kernel/watchdog.c | 65 +++++++++++++++++++++++++++++++++++-----------------
>> 1 files changed, 44 insertions(+), 21 deletions(-)
>>
>> diff --git a/kernel/watchdog.c b/kernel/watchdog.c
>> index 9acb29f..d1e84e6 100644
>> --- a/kernel/watchdog.c
>> +++ b/kernel/watchdog.c
>> @@ -46,7 +46,7 @@
>>
>> static DEFINE_MUTEX(watchdog_proc_mutex);
>>
>> -#ifdef CONFIG_HARDLOCKUP_DETECTOR
>> +#if defined(CONFIG_HARDLOCKUP_DETECTOR) || defined(CONFIG_HAVE_NMI_WATCHDOG)
>> static unsigned long __read_mostly watchdog_enabled = SOFT_WATCHDOG_ENABLED|NMI_WATCHDOG_ENABLED;
>> #else
>> static unsigned long __read_mostly watchdog_enabled = SOFT_WATCHDOG_ENABLED;
>> @@ -585,15 +585,11 @@ static void watchdog(unsigned int cpu)
>> */
>> static unsigned long cpu0_err;
>>
>> -static int watchdog_nmi_enable(unsigned int cpu)
>> +static int arch_watchdog_nmi_enable(unsigned int cpu)
>> {
>> struct perf_event_attr *wd_attr;
>> struct perf_event *event = per_cpu(watchdog_ev, cpu);
>>
>> - /* nothing to do if the hard lockup detector is disabled */
>> - if (!(watchdog_enabled & NMI_WATCHDOG_ENABLED))
>> - goto out;
>> -
>> /* is it already setup and enabled? */
>> if (event && event->state > PERF_EVENT_STATE_OFF)
>> goto out;
>> @@ -619,18 +615,6 @@ static int watchdog_nmi_enable(unsigned int cpu)
>> goto out_save;
>> }
>>
>> - /*
>> - * Disable the hard lockup detector if _any_ CPU fails to set up
>> - * set up the hardware perf event. The watchdog() function checks
>> - * the NMI_WATCHDOG_ENABLED bit periodically.
>> - *
>> - * The barriers are for syncing up watchdog_enabled across all the
>> - * cpus, as clear_bit() does not use barriers.
>> - */
>> - smp_mb__before_atomic();
>> - clear_bit(NMI_WATCHDOG_ENABLED_BIT, &watchdog_enabled);
>> - smp_mb__after_atomic();
>> -
>> /* skip displaying the same error again */
>> if (cpu > 0 && (PTR_ERR(event) == cpu0_err))
>> return PTR_ERR(event);
> In the arch_watchdog_nmi_enable code is a pr_info on failure
>
> pr_info("Shutting down hard lockup detector on all cpus\n");
>
> that should be moved to below..
>
>
>> @@ -658,7 +642,7 @@ out:
>> return 0;
>> }
>>
>> -static void watchdog_nmi_disable(unsigned int cpu)
>> +static void arch_watchdog_nmi_disable(unsigned int cpu)
>> {
>> struct perf_event *event = per_cpu(watchdog_ev, cpu);
>>
>> @@ -676,8 +660,13 @@ static void watchdog_nmi_disable(unsigned int cpu)
>> }
>>
>> #else
>> -static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
>> -static void watchdog_nmi_disable(unsigned int cpu) { return; }
>> +/*
>> + * These two functions are mostly architecture specific
>> + * defining them as weak here.
>> + */
>> +int __weak arch_watchdog_nmi_enable(unsigned int cpu) { return 0; }
>> +void __weak arch_watchdog_nmi_disable(unsigned int cpu) { return; }
>> +
>> #endif /* CONFIG_HARDLOCKUP_DETECTOR */
>>
>> static struct smp_hotplug_thread watchdog_threads = {
>> @@ -781,6 +770,40 @@ void lockup_detector_resume(void)
>> put_online_cpus();
>> }
>>
>> +void watchdog_nmi_disable(unsigned int cpu)
>> +{
>> + arch_watchdog_nmi_disable(cpu);
>> +}
>> +
>> +int watchdog_nmi_enable(unsigned int cpu)
>> +{
>> + int err;
>> +
>> + /* nothing to do if the hard lockup detector is disabled */
>> + if (!(watchdog_enabled & NMI_WATCHDOG_ENABLED))
>> + return 0;
>> +
>> + err = arch_watchdog_nmi_enable(cpu);
>> +
>> + if (err) {
>> + /*
>> + * Disable the hard lockup detector if _any_ CPU fails to set up
>> + * set up the hardware perf event. The watchdog() function checks
>> + * the NMI_WATCHDOG_ENABLED bit periodically.
>> + *
>> + * The barriers are for syncing up watchdog_enabled across all the
>> + * cpus, as clear_bit() does not use barriers.
>> + */
>> + smp_mb__before_atomic();
>> + clear_bit(NMI_WATCHDOG_ENABLED_BIT, &watchdog_enabled);
>> + smp_mb__after_atomic();
> moved to here:
>
> pr_info("Shutting down hard lockup detector on all cpus\n");
>
>
> This lets the failure message be displayed on all arches instead of just
> x86. Though I guess sparc does not call theirs 'hard lockup detector'.
> Hmm...
>
>
>> +
>> + return err;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> static int update_watchdog_all_cpus(void)
>> {
>> int ret;
>> --
>> 1.7.1
>>
On Thu, 13 Oct 2016 13:38:01 -0700 Babu Moger <[email protected]> wrote:
> Currently we do not have a way to enable/disable arch specific
> watchdog handlers if it was implemented by any of the architectures.
>
> This patch introduces new functions arch_watchdog_nmi_enable and
> arch_watchdog_nmi_disable which can be used to enable/disable architecture
> specific NMI watchdog handlers. These functions are defined as weak as
> architectures can override their definitions to enable/disable nmi
> watchdog behaviour.
>
> --- a/kernel/watchdog.c
> +++ b/kernel/watchdog.c
> @@ -676,8 +660,13 @@ static void watchdog_nmi_disable(unsigned int cpu)
> }
>
> #else
> -static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
> -static void watchdog_nmi_disable(unsigned int cpu) { return; }
> +/*
> + * These two functions are mostly architecture specific
> + * defining them as weak here.
> + */
> +int __weak arch_watchdog_nmi_enable(unsigned int cpu) { return 0; }
> +void __weak arch_watchdog_nmi_disable(unsigned int cpu) { return; }
> +
> #endif /* CONFIG_HARDLOCKUP_DETECTOR */
This is a strange way of using __weak.
Take a look at (one of many examples) kernel/module.c:module_alloc().
We simply provide a default implementation and some other compilation
unit can override (actually replace) that at link time. No strange
ifdeffing needed.
And I'm not really understanding the interaction with
CONFIG_HARDLOCKUP_DETECTOR here. I haven't really worked out why the
code is all this way but it seems.... odd?
On Wed, Oct 19, 2016 at 05:00:12PM -0700, Andrew Morton wrote:
> On Thu, 13 Oct 2016 13:38:01 -0700 Babu Moger <[email protected]> wrote:
>
> > Currently we do not have a way to enable/disable arch specific
> > watchdog handlers if it was implemented by any of the architectures.
> >
> > This patch introduces new functions arch_watchdog_nmi_enable and
> > arch_watchdog_nmi_disable which can be used to enable/disable architecture
> > specific NMI watchdog handlers. These functions are defined as weak as
> > architectures can override their definitions to enable/disable nmi
> > watchdog behaviour.
> >
> > --- a/kernel/watchdog.c
> > +++ b/kernel/watchdog.c
> > @@ -676,8 +660,13 @@ static void watchdog_nmi_disable(unsigned int cpu)
> > }
> >
> > #else
> > -static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
> > -static void watchdog_nmi_disable(unsigned int cpu) { return; }
> > +/*
> > + * These two functions are mostly architecture specific
> > + * defining them as weak here.
> > + */
> > +int __weak arch_watchdog_nmi_enable(unsigned int cpu) { return 0; }
> > +void __weak arch_watchdog_nmi_disable(unsigned int cpu) { return; }
> > +
> > #endif /* CONFIG_HARDLOCKUP_DETECTOR */
>
> This is a strange way of using __weak.
>
> Take a look at (one of many examples) kernel/module.c:module_alloc().
> We simply provide a default implementation and some other compilation
> unit can override (actually replace) that at link time. No strange
> ifdeffing needed.
Yeah, this is mostly because of how we enable the hardlockup detector.
Some arches use the perf hw and enable CONFIG_HARDLOCKUP_DETECTOR. Other
arches just use their own variant of nmi and set CONFIG_HAVE_NMI_WATCHDOG and
the rest of the arches do not use this.
So the thought was if CONFIG_HARDLOCKUP_DETECTOR use that implementation,
everyone else use the __weak version. Then the arches like sparc can override
the weak version with their own nmi enablement.
I don't know how to represent those 3 states correctly and the above is what
we end up with.
>
> And I'm not really understanding the interaction with
> CONFIG_HARDLOCKUP_DETECTOR here. I haven't really worked out why the
> code is all this way but it seems.... odd?
If the above explaination doesn't help, then can you point to some examples
where things seem odd?
Cheers,
Don
On Thu, 20 Oct 2016 12:14:14 -0400 Don Zickus <[email protected]> wrote:
> > > -static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
> > > -static void watchdog_nmi_disable(unsigned int cpu) { return; }
> > > +/*
> > > + * These two functions are mostly architecture specific
> > > + * defining them as weak here.
> > > + */
> > > +int __weak arch_watchdog_nmi_enable(unsigned int cpu) { return 0; }
> > > +void __weak arch_watchdog_nmi_disable(unsigned int cpu) { return; }
> > > +
> > > #endif /* CONFIG_HARDLOCKUP_DETECTOR */
> >
> > This is a strange way of using __weak.
> >
> > Take a look at (one of many examples) kernel/module.c:module_alloc().
> > We simply provide a default implementation and some other compilation
> > unit can override (actually replace) that at link time. No strange
> > ifdeffing needed.
>
> Yeah, this is mostly because of how we enable the hardlockup detector.
>
> Some arches use the perf hw and enable CONFIG_HARDLOCKUP_DETECTOR. Other
> arches just use their own variant of nmi and set CONFIG_HAVE_NMI_WATCHDOG and
> the rest of the arches do not use this.
>
> So the thought was if CONFIG_HARDLOCKUP_DETECTOR use that implementation,
> everyone else use the __weak version. Then the arches like sparc can override
> the weak version with their own nmi enablement.
>
> I don't know how to represent those 3 states correctly and the above is what
> we end up with.
<head spins>
Is there a suitable site where we could capture these considerations in
a code comment?
On Thu, Oct 20, 2016 at 08:25:27PM -0700, Andrew Morton wrote:
> On Thu, 20 Oct 2016 12:14:14 -0400 Don Zickus <[email protected]> wrote:
>
> > > > -static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
> > > > -static void watchdog_nmi_disable(unsigned int cpu) { return; }
> > > > +/*
> > > > + * These two functions are mostly architecture specific
> > > > + * defining them as weak here.
> > > > + */
> > > > +int __weak arch_watchdog_nmi_enable(unsigned int cpu) { return 0; }
> > > > +void __weak arch_watchdog_nmi_disable(unsigned int cpu) { return; }
> > > > +
> > > > #endif /* CONFIG_HARDLOCKUP_DETECTOR */
> > >
> > > This is a strange way of using __weak.
> > >
> > > Take a look at (one of many examples) kernel/module.c:module_alloc().
> > > We simply provide a default implementation and some other compilation
> > > unit can override (actually replace) that at link time. No strange
> > > ifdeffing needed.
> >
> > Yeah, this is mostly because of how we enable the hardlockup detector.
> >
> > Some arches use the perf hw and enable CONFIG_HARDLOCKUP_DETECTOR. Other
> > arches just use their own variant of nmi and set CONFIG_HAVE_NMI_WATCHDOG and
> > the rest of the arches do not use this.
> >
> > So the thought was if CONFIG_HARDLOCKUP_DETECTOR use that implementation,
> > everyone else use the __weak version. Then the arches like sparc can override
> > the weak version with their own nmi enablement.
> >
> > I don't know how to represent those 3 states correctly and the above is what
> > we end up with.
>
> <head spins>
>
> Is there a suitable site where we could capture these considerations in
> a code comment?
Hi Andrew,
I am not sure I understand your question. When you say 'site', are you
referring to the kernel/watchdog.c file?
The other approach that might help de-clutter this file, is to pull out the
HARDLOCKUP_DETECTOR changes (as they are arch specific) and move it to say
kernel/watchdog_hw_ld.c. Then all the nmi hooks in kernel/watchdog.c can be
__weak and overridden by the kernel_watchdog_hw_ld.c file or the sparc
files.
This would leave kernel/watchdog.c with just a framework and the
arch-agnostic softlockup detector. Probably easier to read and digest.
Thoughts?
Cheers,
Don
On Fri, 21 Oct 2016 11:11:14 -0400 Don Zickus <[email protected]> wrote:
> On Thu, Oct 20, 2016 at 08:25:27PM -0700, Andrew Morton wrote:
> > On Thu, 20 Oct 2016 12:14:14 -0400 Don Zickus <[email protected]> wrote:
> >
> > > > > -static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
> > > > > -static void watchdog_nmi_disable(unsigned int cpu) { return; }
> > > > > +/*
> > > > > + * These two functions are mostly architecture specific
> > > > > + * defining them as weak here.
> > > > > + */
> > > > > +int __weak arch_watchdog_nmi_enable(unsigned int cpu) { return 0; }
> > > > > +void __weak arch_watchdog_nmi_disable(unsigned int cpu) { return; }
> > > > > +
> > > > > #endif /* CONFIG_HARDLOCKUP_DETECTOR */
> > > >
> > > > This is a strange way of using __weak.
> > > >
> > > > Take a look at (one of many examples) kernel/module.c:module_alloc().
> > > > We simply provide a default implementation and some other compilation
> > > > unit can override (actually replace) that at link time. No strange
> > > > ifdeffing needed.
> > >
> > > Yeah, this is mostly because of how we enable the hardlockup detector.
> > >
> > > Some arches use the perf hw and enable CONFIG_HARDLOCKUP_DETECTOR. Other
> > > arches just use their own variant of nmi and set CONFIG_HAVE_NMI_WATCHDOG and
> > > the rest of the arches do not use this.
> > >
> > > So the thought was if CONFIG_HARDLOCKUP_DETECTOR use that implementation,
> > > everyone else use the __weak version. Then the arches like sparc can override
> > > the weak version with their own nmi enablement.
> > >
> > > I don't know how to represent those 3 states correctly and the above is what
> > > we end up with.
> >
> > <head spins>
> >
> > Is there a suitable site where we could capture these considerations in
> > a code comment?
>
> Hi Andrew,
>
> I am not sure I understand your question. When you say 'site', are you
> referring to the kernel/watchdog.c file?
Yes, somewhere in there I guess.
The problem with this sort of thing is that the implementation is
splattered over multiple places in one file or in several files so
there's no clear place to document what's happening. But I think this
situation *should* be documented somewhere. Or maybe that just isn't
worthwhile - feel free to disagree!
> The other approach that might help de-clutter this file, is to pull out the
> HARDLOCKUP_DETECTOR changes (as they are arch specific) and move it to say
> kernel/watchdog_hw_ld.c. Then all the nmi hooks in kernel/watchdog.c can be
> __weak and overridden by the kernel_watchdog_hw_ld.c file or the sparc
> files.
>
> This would leave kernel/watchdog.c with just a framework and the
> arch-agnostic softlockup detector. Probably easier to read and digest.
Well, it depends how the code ends up looking. It's best to separate
functional changes from cleanups. Generally I think it's best to do
"cleanup comes first", because it's then simpler to revert the
functional change if it has problems. Plus people are more
*interested* in the functional change so it's best to have that at
top-of-tree.
Don,
On 10/21/2016 2:19 PM, Andrew Morton wrote:
> On Fri, 21 Oct 2016 11:11:14 -0400 Don Zickus <[email protected]> wrote:
>
>> On Thu, Oct 20, 2016 at 08:25:27PM -0700, Andrew Morton wrote:
>>> On Thu, 20 Oct 2016 12:14:14 -0400 Don Zickus <[email protected]> wrote:
>>>
>>>>>> -static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
>>>>>> -static void watchdog_nmi_disable(unsigned int cpu) { return; }
>>>>>> +/*
>>>>>> + * These two functions are mostly architecture specific
>>>>>> + * defining them as weak here.
>>>>>> + */
>>>>>> +int __weak arch_watchdog_nmi_enable(unsigned int cpu) { return 0; }
>>>>>> +void __weak arch_watchdog_nmi_disable(unsigned int cpu) { return; }
>>>>>> +
>>>>>> #endif /* CONFIG_HARDLOCKUP_DETECTOR */
>>>>> This is a strange way of using __weak.
>>>>>
>>>>> Take a look at (one of many examples) kernel/module.c:module_alloc().
>>>>> We simply provide a default implementation and some other compilation
>>>>> unit can override (actually replace) that at link time. No strange
>>>>> ifdeffing needed.
>>>> Yeah, this is mostly because of how we enable the hardlockup detector.
>>>>
>>>> Some arches use the perf hw and enable CONFIG_HARDLOCKUP_DETECTOR. Other
>>>> arches just use their own variant of nmi and set CONFIG_HAVE_NMI_WATCHDOG and
>>>> the rest of the arches do not use this.
>>>>
>>>> So the thought was if CONFIG_HARDLOCKUP_DETECTOR use that implementation,
>>>> everyone else use the __weak version. Then the arches like sparc can override
>>>> the weak version with their own nmi enablement.
>>>>
>>>> I don't know how to represent those 3 states correctly and the above is what
>>>> we end up with.
>>> <head spins>
>>>
>>> Is there a suitable site where we could capture these considerations in
>>> a code comment?
>> Hi Andrew,
>>
>> I am not sure I understand your question. When you say 'site', are you
>> referring to the kernel/watchdog.c file?
> Yes, somewhere in there I guess.
>
> The problem with this sort of thing is that the implementation is
> splattered over multiple places in one file or in several files so
> there's no clear place to document what's happening. But I think this
> situation *should* be documented somewhere. Or maybe that just isn't
> worthwhile - feel free to disagree!
>
>> The other approach that might help de-clutter this file, is to pull out the
>> HARDLOCKUP_DETECTOR changes (as they are arch specific) and move it to say
>> kernel/watchdog_hw_ld.c. Then all the nmi hooks in kernel/watchdog.c can be
>> __weak and overridden by the kernel_watchdog_hw_ld.c file or the sparc
>> files.
>>
>> This would leave kernel/watchdog.c with just a framework and the
>> arch-agnostic softlockup detector. Probably easier to read and digest.
Don, Yes. I am fine with your idea. Let me know if you need any help
here. If you want I can
start working this cleanup myself. I might take sometime as I need to
spend sometime
understanding the whole watchdog stuff first. If you have already
started working on this
then I will let you continue.
> Well, it depends how the code ends up looking. It's best to separate
> functional changes from cleanups. Generally I think it's best to do
> "cleanup comes first", because it's then simpler to revert the
> functional change if it has problems. Plus people are more
> *interested* in the functional change so it's best to have that at
> top-of-tree.
>
On Fri, Oct 21, 2016 at 04:50:21PM -0500, Babu Moger wrote:
> Don,
>
> On 10/21/2016 2:19 PM, Andrew Morton wrote:
> >On Fri, 21 Oct 2016 11:11:14 -0400 Don Zickus <[email protected]> wrote:
> >
> >>On Thu, Oct 20, 2016 at 08:25:27PM -0700, Andrew Morton wrote:
> >>>On Thu, 20 Oct 2016 12:14:14 -0400 Don Zickus <[email protected]> wrote:
> >>>
> >>>>>>-static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
> >>>>>>-static void watchdog_nmi_disable(unsigned int cpu) { return; }
> >>>>>>+/*
> >>>>>>+ * These two functions are mostly architecture specific
> >>>>>>+ * defining them as weak here.
> >>>>>>+ */
> >>>>>>+int __weak arch_watchdog_nmi_enable(unsigned int cpu) { return 0; }
> >>>>>>+void __weak arch_watchdog_nmi_disable(unsigned int cpu) { return; }
> >>>>>>+
> >>>>>> #endif /* CONFIG_HARDLOCKUP_DETECTOR */
> >>>>>This is a strange way of using __weak.
> >>>>>
> >>>>>Take a look at (one of many examples) kernel/module.c:module_alloc().
> >>>>>We simply provide a default implementation and some other compilation
> >>>>>unit can override (actually replace) that at link time. No strange
> >>>>>ifdeffing needed.
> >>>>Yeah, this is mostly because of how we enable the hardlockup detector.
> >>>>
> >>>>Some arches use the perf hw and enable CONFIG_HARDLOCKUP_DETECTOR. Other
> >>>>arches just use their own variant of nmi and set CONFIG_HAVE_NMI_WATCHDOG and
> >>>>the rest of the arches do not use this.
> >>>>
> >>>>So the thought was if CONFIG_HARDLOCKUP_DETECTOR use that implementation,
> >>>>everyone else use the __weak version. Then the arches like sparc can override
> >>>>the weak version with their own nmi enablement.
> >>>>
> >>>>I don't know how to represent those 3 states correctly and the above is what
> >>>>we end up with.
> >>><head spins>
> >>>
> >>>Is there a suitable site where we could capture these considerations in
> >>>a code comment?
> >>Hi Andrew,
> >>
> >>I am not sure I understand your question. When you say 'site', are you
> >>referring to the kernel/watchdog.c file?
> >Yes, somewhere in there I guess.
> >
> >The problem with this sort of thing is that the implementation is
> >splattered over multiple places in one file or in several files so
> >there's no clear place to document what's happening. But I think this
> >situation *should* be documented somewhere. Or maybe that just isn't
> >worthwhile - feel free to disagree!
> >
> >>The other approach that might help de-clutter this file, is to pull out the
> >>HARDLOCKUP_DETECTOR changes (as they are arch specific) and move it to say
> >>kernel/watchdog_hw_ld.c. Then all the nmi hooks in kernel/watchdog.c can be
> >>__weak and overridden by the kernel_watchdog_hw_ld.c file or the sparc
> >>files.
> >>
> >>This would leave kernel/watchdog.c with just a framework and the
> >>arch-agnostic softlockup detector. Probably easier to read and digest.
>
> Don, Yes. I am fine with your idea. Let me know if you need any help here.
> If you want I can
> start working this cleanup myself. I might take sometime as I need to spend
> sometime
> understanding the whole watchdog stuff first. If you have already started
> working on this
> then I will let you continue.
Hi Babu,
Feel free to start looking at it. I am trying to wrap up a couple of things
here and will only be able to little poke at it the next couple of days.
But for the most part you might be able to rip out anything with
CONFIG_HARDLOCKUP_DETECTOR and put it into another file. Then just clean up
the pieces.
Cheers,
Don
>
> >Well, it depends how the code ends up looking. It's best to separate
> >functional changes from cleanups. Generally I think it's best to do
> >"cleanup comes first", because it's then simpler to revert the
> >functional change if it has problems. Plus people are more
> >*interested* in the functional change so it's best to have that at
> >top-of-tree.
> >
>
On 10/24/2016 10:19 AM, Don Zickus wrote:
> On Fri, Oct 21, 2016 at 04:50:21PM -0500, Babu Moger wrote:
>> Don,
>>
>> On 10/21/2016 2:19 PM, Andrew Morton wrote:
>>> On Fri, 21 Oct 2016 11:11:14 -0400 Don Zickus <[email protected]> wrote:
>>>
>>>> On Thu, Oct 20, 2016 at 08:25:27PM -0700, Andrew Morton wrote:
>>>>> On Thu, 20 Oct 2016 12:14:14 -0400 Don Zickus <[email protected]> wrote:
>>>>>
>>>>>>>> -static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
>>>>>>>> -static void watchdog_nmi_disable(unsigned int cpu) { return; }
>>>>>>>> +/*
>>>>>>>> + * These two functions are mostly architecture specific
>>>>>>>> + * defining them as weak here.
>>>>>>>> + */
>>>>>>>> +int __weak arch_watchdog_nmi_enable(unsigned int cpu) { return 0; }
>>>>>>>> +void __weak arch_watchdog_nmi_disable(unsigned int cpu) { return; }
>>>>>>>> +
>>>>>>>> #endif /* CONFIG_HARDLOCKUP_DETECTOR */
>>>>>>> This is a strange way of using __weak.
>>>>>>>
>>>>>>> Take a look at (one of many examples) kernel/module.c:module_alloc().
>>>>>>> We simply provide a default implementation and some other compilation
>>>>>>> unit can override (actually replace) that at link time. No strange
>>>>>>> ifdeffing needed.
>>>>>> Yeah, this is mostly because of how we enable the hardlockup detector.
>>>>>>
>>>>>> Some arches use the perf hw and enable CONFIG_HARDLOCKUP_DETECTOR. Other
>>>>>> arches just use their own variant of nmi and set CONFIG_HAVE_NMI_WATCHDOG and
>>>>>> the rest of the arches do not use this.
>>>>>>
>>>>>> So the thought was if CONFIG_HARDLOCKUP_DETECTOR use that implementation,
>>>>>> everyone else use the __weak version. Then the arches like sparc can override
>>>>>> the weak version with their own nmi enablement.
>>>>>>
>>>>>> I don't know how to represent those 3 states correctly and the above is what
>>>>>> we end up with.
>>>>> <head spins>
>>>>>
>>>>> Is there a suitable site where we could capture these considerations in
>>>>> a code comment?
>>>> Hi Andrew,
>>>>
>>>> I am not sure I understand your question. When you say 'site', are you
>>>> referring to the kernel/watchdog.c file?
>>> Yes, somewhere in there I guess.
>>>
>>> The problem with this sort of thing is that the implementation is
>>> splattered over multiple places in one file or in several files so
>>> there's no clear place to document what's happening. But I think this
>>> situation *should* be documented somewhere. Or maybe that just isn't
>>> worthwhile - feel free to disagree!
>>>
>>>> The other approach that might help de-clutter this file, is to pull out the
>>>> HARDLOCKUP_DETECTOR changes (as they are arch specific) and move it to say
>>>> kernel/watchdog_hw_ld.c. Then all the nmi hooks in kernel/watchdog.c can be
>>>> __weak and overridden by the kernel_watchdog_hw_ld.c file or the sparc
>>>> files.
>>>>
>>>> This would leave kernel/watchdog.c with just a framework and the
>>>> arch-agnostic softlockup detector. Probably easier to read and digest.
>> Don, Yes. I am fine with your idea. Let me know if you need any help here.
>> If you want I can
>> start working this cleanup myself. I might take sometime as I need to spend
>> sometime
>> understanding the whole watchdog stuff first. If you have already started
>> working on this
>> then I will let you continue.
> Hi Babu,
>
> Feel free to start looking at it. I am trying to wrap up a couple of things
> here and will only be able to little poke at it the next couple of days.
> But for the most part you might be able to rip out anything with
> CONFIG_HARDLOCKUP_DETECTOR and put it into another file. Then just clean up
> the pieces.
Don. Sure. I have started on this. Will send RFC version sometime this week.
>
> Cheers,
> Don
>
>>> Well, it depends how the code ends up looking. It's best to separate
>>> functional changes from cleanups. Generally I think it's best to do
>>> "cleanup comes first", because it's then simpler to revert the
>>> functional change if it has problems. Plus people are more
>>> *interested* in the functional change so it's best to have that at
>>> top-of-tree.
>>>