From: Claudiu Beznea <[email protected]>
Hi,
Series adds watchdog support for Renesas RZ/G3S (R9A08G045) SoC.
Patches do the following:
- patch 1/10 makes the driver depend on ARCH_RZG2L || ARCH_R9A09G011
- patch 2/10 makes the driver depend on PM
- patches 3-7/10 adds fixes and cleanups for the watchdog driver
- patch 8/10 adds suspend to RAM to the watchdog driver (to be used by
RZ/G3S)
- patch 9/10 adapt for power domain support
- patch 10/10 documents the RZ/G3S support
Thank you,
Claudiu Beznea
Changes in v8:
- added patch 9
- collected tags
Changes in v7:
- updated the dependency on patch 2/9
Changes in v6:
- update patch 2/9 description
- fixed the dependency on COMPILE_TEST previously introduced in patch
2/9
Changes in v5:
- updated description of patch 2/9
- simplify the code in patch 2/9 by using on a new line:
depends on PM || COMPILE_TEST
Changes in v4:
- added patch "watchdog: rzg2l_wdt: Restrict the driver to ARCH_RZG2L and
ARCH_R9A09G011"
- collected tags
Changes in v3:
- make driver depend on PM not select it
- drop patches already accepted (patches 1, 10, 11 from v2)
- re-arranged the tags in patch 8/8 as they were messed by b4 am/shazam
Changes in v2:
- added patch "watchdog: rzg2l_wdt: Select PM"
- propagate the return status of rzg2l_wdt_start() to it's callers
in patch "watchdog: rzg2l_wdt: Use pm_runtime_resume_and_get()"
- propagate the return status of rzg2l_wdt_stop() to it's callers
in patch "watchdog: rzg2l_wdt: Check return status of pm_runtime_put()"
- removed pm_ptr() from patch "watchdog: rzg2l_wdt: Add suspend/resume support"
- s/G2UL/G2L in patch "dt-bindings: watchdog: renesas,wdt: Document RZ/G3S support"
- collected tags
Claudiu Beznea (10):
watchdog: rzg2l_wdt: Restrict the driver to ARCH_RZG2L and
ARCH_R9A09G011
watchdog: rzg2l_wdt: Make the driver depend on PM
watchdog: rzg2l_wdt: Use pm_runtime_resume_and_get()
watchdog: rzg2l_wdt: Check return status of pm_runtime_put()
watchdog: rzg2l_wdt: Remove reset de-assert from probe
watchdog: rzg2l_wdt: Remove comparison with zero
watchdog: rzg2l_wdt: Rely on the reset driver for doing proper reset
watchdog: rzg2l_wdt: Add suspend/resume support
watchdog: rzg2l_wdt: Power on the PM domain in rzg2l_wdt_restart()
dt-bindings: watchdog: renesas,wdt: Document RZ/G3S support
.../bindings/watchdog/renesas,wdt.yaml | 1 +
drivers/watchdog/Kconfig | 3 +-
drivers/watchdog/rzg2l_wdt.c | 123 +++++++++++-------
3 files changed, 76 insertions(+), 51 deletions(-)
--
2.39.2
From: Claudiu Beznea <[email protected]>
The RZ/G3S supports deep sleep states where power to most of the IP blocks
is cut off. To ensure proper working of the watchdog when resuming from
such states, the suspend function is stopping the watchdog and the resume
function is starting it. There is no need to configure the watchdog
in case the watchdog was stopped prior to starting suspend.
Signed-off-by: Claudiu Beznea <[email protected]>
---
Changes in v8:
- none
Changes in v7:
- none
Changes in v6:
- none
Changes in v5:
- none
Changes in v4:
- none
Changes in v3:
- none
Changes in v2:
- remove the usage of pm_ptr()
drivers/watchdog/rzg2l_wdt.c | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/drivers/watchdog/rzg2l_wdt.c b/drivers/watchdog/rzg2l_wdt.c
index 42f1d5d6f07e..c8c20cfb97a3 100644
--- a/drivers/watchdog/rzg2l_wdt.c
+++ b/drivers/watchdog/rzg2l_wdt.c
@@ -284,6 +284,7 @@ static int rzg2l_wdt_probe(struct platform_device *pdev)
priv->wdev.timeout = WDT_DEFAULT_TIMEOUT;
watchdog_set_drvdata(&priv->wdev, priv);
+ dev_set_drvdata(dev, priv);
ret = devm_add_action_or_reset(&pdev->dev, rzg2l_wdt_pm_disable, &priv->wdev);
if (ret)
return ret;
@@ -305,10 +306,35 @@ static const struct of_device_id rzg2l_wdt_ids[] = {
};
MODULE_DEVICE_TABLE(of, rzg2l_wdt_ids);
+static int rzg2l_wdt_suspend_late(struct device *dev)
+{
+ struct rzg2l_wdt_priv *priv = dev_get_drvdata(dev);
+
+ if (!watchdog_active(&priv->wdev))
+ return 0;
+
+ return rzg2l_wdt_stop(&priv->wdev);
+}
+
+static int rzg2l_wdt_resume_early(struct device *dev)
+{
+ struct rzg2l_wdt_priv *priv = dev_get_drvdata(dev);
+
+ if (!watchdog_active(&priv->wdev))
+ return 0;
+
+ return rzg2l_wdt_start(&priv->wdev);
+}
+
+static const struct dev_pm_ops rzg2l_wdt_pm_ops = {
+ LATE_SYSTEM_SLEEP_PM_OPS(rzg2l_wdt_suspend_late, rzg2l_wdt_resume_early)
+};
+
static struct platform_driver rzg2l_wdt_driver = {
.driver = {
.name = "rzg2l_wdt",
.of_match_table = rzg2l_wdt_ids,
+ .pm = &rzg2l_wdt_pm_ops,
},
.probe = rzg2l_wdt_probe,
};
--
2.39.2
From: Claudiu Beznea <[email protected]>
There is no need to de-assert the reset signal on probe as the watchdog
is not used prior executing start. Also, the clocks are not enabled in
probe (pm_runtime_enable() doesn't do that), thus this is another indicator
that the watchdog wasn't used previously like this. Instead, keep the
watchdog hardware in its previous state at probe (by default it is in
reset state), enable it when it is started and move it to reset state
when it is stopped. This saves some extra power when the watchdog is
unused.
Signed-off-by: Claudiu Beznea <[email protected]>
---
Changes in v8:
- none
Changes in v7:
- none
Changes in v6:
- none
Changes in v5:
- update patch title
Changes in v4:
- none
Changes in v3:
- none
Changes in v2:
- none
drivers/watchdog/rzg2l_wdt.c | 26 +++++++++++++++-----------
1 file changed, 15 insertions(+), 11 deletions(-)
diff --git a/drivers/watchdog/rzg2l_wdt.c b/drivers/watchdog/rzg2l_wdt.c
index 7bce093316c4..93a49fd0c7aa 100644
--- a/drivers/watchdog/rzg2l_wdt.c
+++ b/drivers/watchdog/rzg2l_wdt.c
@@ -129,6 +129,10 @@ static int rzg2l_wdt_start(struct watchdog_device *wdev)
if (ret)
return ret;
+ ret = reset_control_deassert(priv->rstc);
+ if (ret)
+ return ret;
+
/* Initialize time out */
rzg2l_wdt_init_timeout(wdev);
@@ -146,7 +150,9 @@ static int rzg2l_wdt_stop(struct watchdog_device *wdev)
struct rzg2l_wdt_priv *priv = watchdog_get_drvdata(wdev);
int ret;
- rzg2l_wdt_reset(priv);
+ ret = reset_control_assert(priv->rstc);
+ if (ret)
+ return ret;
ret = pm_runtime_put(wdev->parent);
if (ret < 0)
@@ -186,6 +192,12 @@ static int rzg2l_wdt_restart(struct watchdog_device *wdev,
clk_prepare_enable(priv->osc_clk);
if (priv->devtype == WDT_RZG2L) {
+ int ret;
+
+ ret = reset_control_deassert(priv->rstc);
+ if (ret)
+ return ret;
+
/* Generate Reset (WDTRSTB) Signal on parity error */
rzg2l_wdt_write(priv, 0, PECR);
@@ -236,13 +248,11 @@ static const struct watchdog_ops rzg2l_wdt_ops = {
.restart = rzg2l_wdt_restart,
};
-static void rzg2l_wdt_reset_assert_pm_disable(void *data)
+static void rzg2l_wdt_pm_disable(void *data)
{
struct watchdog_device *wdev = data;
- struct rzg2l_wdt_priv *priv = watchdog_get_drvdata(wdev);
pm_runtime_disable(wdev->parent);
- reset_control_assert(priv->rstc);
}
static int rzg2l_wdt_probe(struct platform_device *pdev)
@@ -285,10 +295,6 @@ static int rzg2l_wdt_probe(struct platform_device *pdev)
return dev_err_probe(&pdev->dev, PTR_ERR(priv->rstc),
"failed to get cpg reset");
- ret = reset_control_deassert(priv->rstc);
- if (ret)
- return dev_err_probe(dev, ret, "failed to deassert");
-
priv->devtype = (uintptr_t)of_device_get_match_data(dev);
if (priv->devtype == WDT_RZV2M) {
@@ -309,9 +315,7 @@ static int rzg2l_wdt_probe(struct platform_device *pdev)
priv->wdev.timeout = WDT_DEFAULT_TIMEOUT;
watchdog_set_drvdata(&priv->wdev, priv);
- ret = devm_add_action_or_reset(&pdev->dev,
- rzg2l_wdt_reset_assert_pm_disable,
- &priv->wdev);
+ ret = devm_add_action_or_reset(&pdev->dev, rzg2l_wdt_pm_disable, &priv->wdev);
if (ret < 0)
return ret;
--
2.39.2
From: Claudiu Beznea <[email protected]>
The rzg2l_wdt driver is used only by ARCH_RZG2L and ARCH_R9A09G011
micro-architectures of Renesas. Thus, limit it's usage only to these.
Suggested-by: Biju Das <[email protected]>
Signed-off-by: Claudiu Beznea <[email protected]>
---
Changes in v8:
- none
Changes in v7:
- none
Changes in v6:
- none
Changes in v5:
- none
Changes in v4:
- none; this patch is introduced in v4
drivers/watchdog/Kconfig | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/watchdog/Kconfig b/drivers/watchdog/Kconfig
index 6bee137cfbe0..e2439967417a 100644
--- a/drivers/watchdog/Kconfig
+++ b/drivers/watchdog/Kconfig
@@ -920,7 +920,7 @@ config RENESAS_RZN1WDT
config RENESAS_RZG2LWDT
tristate "Renesas RZ/G2L WDT Watchdog"
- depends on ARCH_RENESAS || COMPILE_TEST
+ depends on ARCH_RZG2L || ARCH_R9A09G011 || COMPILE_TEST
select WATCHDOG_CORE
help
This driver adds watchdog support for the integrated watchdogs in the
--
2.39.2
From: Claudiu Beznea <[email protected]>
The rzg2l_wdt_restart() is called from atomic context. Calling
pm_runtime_{get_sync, resume_and_get}() or any other runtime PM resume
APIs is not an option as it may lead to issues as described in commit
e4cf89596c1f ("watchdog: rzg2l_wdt: Fix 'BUG: Invalid wait context'")
that removed the pm_runtime_get_sync() and used directly the
clk_prepare_enable() APIs.
Starting with RZ/G3S the watchdog could be part of its own software
controlled power domain (see the initial implementation in Link section).
In case the watchdog is not used the power domain is off and accessing
watchdog registers leads to aborts.
To solve this the patch powers on the power domain using
dev_pm_genpd_resume() API before enabling its clock. This is not
sleeping or taking any other locks as the power domain will not be
registered with GENPD_FLAG_IRQ_SAFE flags.
Link: https://lore.kernel.org/all/[email protected]
Signed-off-by: Claudiu Beznea <[email protected]>
---
Changes in v8:
- none, this patch is new
drivers/watchdog/rzg2l_wdt.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/drivers/watchdog/rzg2l_wdt.c b/drivers/watchdog/rzg2l_wdt.c
index c8c20cfb97a3..98e5e9914a5d 100644
--- a/drivers/watchdog/rzg2l_wdt.c
+++ b/drivers/watchdog/rzg2l_wdt.c
@@ -12,6 +12,7 @@
#include <linux/module.h>
#include <linux/of.h>
#include <linux/platform_device.h>
+#include <linux/pm_domain.h>
#include <linux/pm_runtime.h>
#include <linux/reset.h>
#include <linux/units.h>
@@ -164,6 +165,17 @@ static int rzg2l_wdt_restart(struct watchdog_device *wdev,
struct rzg2l_wdt_priv *priv = watchdog_get_drvdata(wdev);
int ret;
+ /*
+ * The device may be part of a power domain that is currently
+ * powered off. We need to power it up before accessing registers.
+ * We don't undo the dev_pm_genpd_resume() as the device need to
+ * be up for the reboot to happen. Also, as we are in atomic context
+ * here there is no need to increment PM runtime usage counter
+ * (to make sure pm_runtime_active() doesn't return wrong code).
+ */
+ if (!pm_runtime_active(wdev->parent))
+ dev_pm_genpd_resume(wdev->parent);
+
clk_prepare_enable(priv->pclk);
clk_prepare_enable(priv->osc_clk);
--
2.39.2
From: Claudiu Beznea <[email protected]>
pm_runtime_get_sync() may return with error. In case it returns with error
dev->power.usage_count needs to be decremented. pm_runtime_resume_and_get()
takes care of this. Thus use it.
Along with it the rzg2l_wdt_set_timeout() function was updated to
propagate the result of rzg2l_wdt_start() to its caller.
Fixes: 2cbc5cd0b55f ("watchdog: Add Watchdog Timer driver for RZ/G2L")
Signed-off-by: Claudiu Beznea <[email protected]>
---
Changes in v8:
- none
Changes in v7:
- none
Changes in v6:
- none
Changes in v5:
- none
Changes in v4:
- none
Changes in v3:
- none
Changes in v2:
- propagate the return code of rzg2l_wdt_start() to it's callers
drivers/watchdog/rzg2l_wdt.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/drivers/watchdog/rzg2l_wdt.c b/drivers/watchdog/rzg2l_wdt.c
index 1741f98ca67c..d87d4f50180c 100644
--- a/drivers/watchdog/rzg2l_wdt.c
+++ b/drivers/watchdog/rzg2l_wdt.c
@@ -123,8 +123,11 @@ static void rzg2l_wdt_init_timeout(struct watchdog_device *wdev)
static int rzg2l_wdt_start(struct watchdog_device *wdev)
{
struct rzg2l_wdt_priv *priv = watchdog_get_drvdata(wdev);
+ int ret;
- pm_runtime_get_sync(wdev->parent);
+ ret = pm_runtime_resume_and_get(wdev->parent);
+ if (ret)
+ return ret;
/* Initialize time out */
rzg2l_wdt_init_timeout(wdev);
@@ -150,6 +153,8 @@ static int rzg2l_wdt_stop(struct watchdog_device *wdev)
static int rzg2l_wdt_set_timeout(struct watchdog_device *wdev, unsigned int timeout)
{
+ int ret = 0;
+
wdev->timeout = timeout;
/*
@@ -159,10 +164,10 @@ static int rzg2l_wdt_set_timeout(struct watchdog_device *wdev, unsigned int time
*/
if (watchdog_active(wdev)) {
rzg2l_wdt_stop(wdev);
- rzg2l_wdt_start(wdev);
+ ret = rzg2l_wdt_start(wdev);
}
- return 0;
+ return ret;
}
static int rzg2l_wdt_restart(struct watchdog_device *wdev,
--
2.39.2
On Wed, 10 Apr 2024 at 16:19, Claudiu <[email protected]> wrote:
>
> From: Claudiu Beznea <[email protected]>
>
> The rzg2l_wdt_restart() is called from atomic context. Calling
> pm_runtime_{get_sync, resume_and_get}() or any other runtime PM resume
> APIs is not an option as it may lead to issues as described in commit
> e4cf89596c1f ("watchdog: rzg2l_wdt: Fix 'BUG: Invalid wait context'")
> that removed the pm_runtime_get_sync() and used directly the
> clk_prepare_enable() APIs.
>
> Starting with RZ/G3S the watchdog could be part of its own software
> controlled power domain (see the initial implementation in Link section).
> In case the watchdog is not used the power domain is off and accessing
> watchdog registers leads to aborts.
>
> To solve this the patch powers on the power domain using
> dev_pm_genpd_resume() API before enabling its clock. This is not
> sleeping or taking any other locks as the power domain will not be
> registered with GENPD_FLAG_IRQ_SAFE flags.
>
> Link: https://lore.kernel.org/all/[email protected]
> Signed-off-by: Claudiu Beznea <[email protected]>
> ---
>
> Changes in v8:
> - none, this patch is new
>
> drivers/watchdog/rzg2l_wdt.c | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/drivers/watchdog/rzg2l_wdt.c b/drivers/watchdog/rzg2l_wdt.c
> index c8c20cfb97a3..98e5e9914a5d 100644
> --- a/drivers/watchdog/rzg2l_wdt.c
> +++ b/drivers/watchdog/rzg2l_wdt.c
> @@ -12,6 +12,7 @@
> #include <linux/module.h>
> #include <linux/of.h>
> #include <linux/platform_device.h>
> +#include <linux/pm_domain.h>
> #include <linux/pm_runtime.h>
> #include <linux/reset.h>
> #include <linux/units.h>
> @@ -164,6 +165,17 @@ static int rzg2l_wdt_restart(struct watchdog_device *wdev,
> struct rzg2l_wdt_priv *priv = watchdog_get_drvdata(wdev);
> int ret;
>
> + /*
> + * The device may be part of a power domain that is currently
> + * powered off. We need to power it up before accessing registers.
> + * We don't undo the dev_pm_genpd_resume() as the device need to
> + * be up for the reboot to happen. Also, as we are in atomic context
> + * here there is no need to increment PM runtime usage counter
> + * (to make sure pm_runtime_active() doesn't return wrong code).
> + */
> + if (!pm_runtime_active(wdev->parent))
> + dev_pm_genpd_resume(wdev->parent);
> +
I doubt this is the correct solution, but I may be wrong. Unless this
is invoked at the syscore stage?
> clk_prepare_enable(priv->pclk);
> clk_prepare_enable(priv->osc_clk);
>
> --
> 2.39.2
>
>
Can you redirectly me to the complete series, so I can have a better
overview of the problem?
Kind regards
Uffe
On Wed, 10 Apr 2024 at 16:19, Claudiu <[email protected]> wrote:
>
> From: Claudiu Beznea <[email protected]>
>
> The rzg2l_wdt_restart() is called from atomic context. Calling
> pm_runtime_{get_sync, resume_and_get}() or any other runtime PM resume
> APIs is not an option as it may lead to issues as described in commit
> e4cf89596c1f ("watchdog: rzg2l_wdt: Fix 'BUG: Invalid wait context'")
> that removed the pm_runtime_get_sync() and used directly the
> clk_prepare_enable() APIs.
Calling clk_prepare_enable() doesn't work from an atomic context
either as it may sleep on the clk prepare mutex.
As I said in the other reply too, it looks like we need a different
solution. I am not sure what, but I am happy to help discuss it.
>
> Starting with RZ/G3S the watchdog could be part of its own software
> controlled power domain (see the initial implementation in Link section).
> In case the watchdog is not used the power domain is off and accessing
> watchdog registers leads to aborts.
>
> To solve this the patch powers on the power domain using
> dev_pm_genpd_resume() API before enabling its clock. This is not
> sleeping or taking any other locks as the power domain will not be
> registered with GENPD_FLAG_IRQ_SAFE flags.
>
> Link: https://lore.kernel.org/all/[email protected]
> Signed-off-by: Claudiu Beznea <[email protected]>
Kind regards
Uffe
> ---
>
> Changes in v8:
> - none, this patch is new
>
> drivers/watchdog/rzg2l_wdt.c | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/drivers/watchdog/rzg2l_wdt.c b/drivers/watchdog/rzg2l_wdt.c
> index c8c20cfb97a3..98e5e9914a5d 100644
> --- a/drivers/watchdog/rzg2l_wdt.c
> +++ b/drivers/watchdog/rzg2l_wdt.c
> @@ -12,6 +12,7 @@
> #include <linux/module.h>
> #include <linux/of.h>
> #include <linux/platform_device.h>
> +#include <linux/pm_domain.h>
> #include <linux/pm_runtime.h>
> #include <linux/reset.h>
> #include <linux/units.h>
> @@ -164,6 +165,17 @@ static int rzg2l_wdt_restart(struct watchdog_device *wdev,
> struct rzg2l_wdt_priv *priv = watchdog_get_drvdata(wdev);
> int ret;
>
> + /*
> + * The device may be part of a power domain that is currently
> + * powered off. We need to power it up before accessing registers.
> + * We don't undo the dev_pm_genpd_resume() as the device need to
> + * be up for the reboot to happen. Also, as we are in atomic context
> + * here there is no need to increment PM runtime usage counter
> + * (to make sure pm_runtime_active() doesn't return wrong code).
> + */
> + if (!pm_runtime_active(wdev->parent))
> + dev_pm_genpd_resume(wdev->parent);
> +
> clk_prepare_enable(priv->pclk);
> clk_prepare_enable(priv->osc_clk);
>
> --
> 2.39.2
>
>
Hi, Ulf,
On 12.04.2024 14:14, Ulf Hansson wrote:
> On Wed, 10 Apr 2024 at 16:19, Claudiu <[email protected]> wrote:
>>
>> From: Claudiu Beznea <[email protected]>
>>
>> The rzg2l_wdt_restart() is called from atomic context. Calling
>> pm_runtime_{get_sync, resume_and_get}() or any other runtime PM resume
>> APIs is not an option as it may lead to issues as described in commit
>> e4cf89596c1f ("watchdog: rzg2l_wdt: Fix 'BUG: Invalid wait context'")
>> that removed the pm_runtime_get_sync() and used directly the
>> clk_prepare_enable() APIs.
>>
>> Starting with RZ/G3S the watchdog could be part of its own software
>> controlled power domain (see the initial implementation in Link section).
>> In case the watchdog is not used the power domain is off and accessing
>> watchdog registers leads to aborts.
>>
>> To solve this the patch powers on the power domain using
>> dev_pm_genpd_resume() API before enabling its clock. This is not
>> sleeping or taking any other locks as the power domain will not be
>> registered with GENPD_FLAG_IRQ_SAFE flags.
>>
>> Link: https://lore.kernel.org/all/[email protected]
>> Signed-off-by: Claudiu Beznea <[email protected]>
>> ---
>>
>> Changes in v8:
>> - none, this patch is new
>>
>> drivers/watchdog/rzg2l_wdt.c | 12 ++++++++++++
>> 1 file changed, 12 insertions(+)
>>
>> diff --git a/drivers/watchdog/rzg2l_wdt.c b/drivers/watchdog/rzg2l_wdt.c
>> index c8c20cfb97a3..98e5e9914a5d 100644
>> --- a/drivers/watchdog/rzg2l_wdt.c
>> +++ b/drivers/watchdog/rzg2l_wdt.c
>> @@ -12,6 +12,7 @@
>> #include <linux/module.h>
>> #include <linux/of.h>
>> #include <linux/platform_device.h>
>> +#include <linux/pm_domain.h>
>> #include <linux/pm_runtime.h>
>> #include <linux/reset.h>
>> #include <linux/units.h>
>> @@ -164,6 +165,17 @@ static int rzg2l_wdt_restart(struct watchdog_device *wdev,
>> struct rzg2l_wdt_priv *priv = watchdog_get_drvdata(wdev);
>> int ret;
>>
>> + /*
>> + * The device may be part of a power domain that is currently
>> + * powered off. We need to power it up before accessing registers.
>> + * We don't undo the dev_pm_genpd_resume() as the device need to
>> + * be up for the reboot to happen. Also, as we are in atomic context
>> + * here there is no need to increment PM runtime usage counter
>> + * (to make sure pm_runtime_active() doesn't return wrong code).
>> + */
>> + if (!pm_runtime_active(wdev->parent))
>> + dev_pm_genpd_resume(wdev->parent);
>> +
>
> I doubt this is the correct solution, but I may be wrong. Unless this
> is invoked at the syscore stage?
On my case I see it invoked from kernel_restart(). As of my code reading,
at that point only one CPU is active with IRQs disabled (done in
machine_restart()). Below is the stack trace decoded on next-20240410 with
this series
(https://lore.kernel.org/all/[email protected]/)
on top and the one from here (adding power domain support):
https://lore.kernel.org/all/[email protected]/
Hardware name: Renesas SMARC EVK version 2 based on r9a08g045s33 (DT)
Call trace:
dump_backtrace (arch/arm64/kernel/stacktrace.c:319)
show_stack (arch/arm64/kernel/stacktrace.c:326)
dump_stack_lvl (lib/dump_stack.c:117)
dump_stack (lib/dump_stack.c:124)
rzg2l_wdt_restart (drivers/watchdog/rzg2l_wdt.c:180)
watchdog_restart_notifier (drivers/watchdog/watchdog_core.c:188)
atomic_notifier_call_chain (kernel/notifier.c:98 kernel/notifier.c:231)
do_kernel_restart (kernel/reboot.c:236)
machine_restart (arch/arm64/kernel/process.c:145)
kernel_restart (kernel/reboot.c:287)
__do_sys_reboot (kernel/reboot.c:755)
__arm64_sys_reboot (kernel/reboot.c:715)
invoke_syscall (arch/arm64/include/asm/current.h:19
arch/arm64/kernel/syscall.c:53)
el0_svc_common.constprop.0 (include/linux/thread_info.h:127
arch/arm64/kernel/syscall.c:141)
do_el0_svc (arch/arm64/kernel/syscall.c:153)
el0_svc (arch/arm64/include/asm/irqflags.h:56
arch/arm64/include/asm/irqflags.h:77 arch/arm64/kernel/entry-common.c:165
arch/arm64/kernel/entry-common.c:178 arch/arm64/kernel/entry-common.c:713)
el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:731)
el0t_64_sync (arch/arm64/kernel/entry.S:598)
The watchdog restart handler is added in restart_handler_list and this list
is invoked though do_kernel_restart(). As of my code investigation the
restart_handler_list is invoked only though do_kernel_restart() and only
though the stack trace above.
Thank you,
Claudiu Beznea
>
>> clk_prepare_enable(priv->pclk);
>> clk_prepare_enable(priv->osc_clk);
>>
>> --
>> 2.39.2
>>
>>
>
> Can you redirectly me to the complete series, so I can have a better
> overview of the problem?
This is the series that adds power domain support for RZ/G3S SoC:
https://lore.kernel.org/all/[email protected]/
This is the series that adds watchdog support for RZ/G3S SoC:
https://lore.kernel.org/all/[email protected]/
Thank you for your review,
Claudiu Beznea
>
> Kind regards
> Uffe
Hi, Ulf,
On 12.04.2024 17:02, claudiu beznea wrote:
> Hi, Ulf,
>
> On 12.04.2024 14:14, Ulf Hansson wrote:
>> On Wed, 10 Apr 2024 at 16:19, Claudiu <[email protected]> wrote:
>>>
>>> From: Claudiu Beznea <[email protected]>
>>>
>>> The rzg2l_wdt_restart() is called from atomic context. Calling
>>> pm_runtime_{get_sync, resume_and_get}() or any other runtime PM resume
>>> APIs is not an option as it may lead to issues as described in commit
>>> e4cf89596c1f ("watchdog: rzg2l_wdt: Fix 'BUG: Invalid wait context'")
>>> that removed the pm_runtime_get_sync() and used directly the
>>> clk_prepare_enable() APIs.
>>>
>>> Starting with RZ/G3S the watchdog could be part of its own software
>>> controlled power domain (see the initial implementation in Link section).
>>> In case the watchdog is not used the power domain is off and accessing
>>> watchdog registers leads to aborts.
>>>
>>> To solve this the patch powers on the power domain using
>>> dev_pm_genpd_resume() API before enabling its clock. This is not
>>> sleeping or taking any other locks as the power domain will not be
>>> registered with GENPD_FLAG_IRQ_SAFE flags.
>>>
>>> Link: https://lore.kernel.org/all/[email protected]
>>> Signed-off-by: Claudiu Beznea <[email protected]>
>>> ---
>>>
>>> Changes in v8:
>>> - none, this patch is new
>>>
>>> drivers/watchdog/rzg2l_wdt.c | 12 ++++++++++++
>>> 1 file changed, 12 insertions(+)
>>>
>>> diff --git a/drivers/watchdog/rzg2l_wdt.c b/drivers/watchdog/rzg2l_wdt.c
>>> index c8c20cfb97a3..98e5e9914a5d 100644
>>> --- a/drivers/watchdog/rzg2l_wdt.c
>>> +++ b/drivers/watchdog/rzg2l_wdt.c
>>> @@ -12,6 +12,7 @@
>>> #include <linux/module.h>
>>> #include <linux/of.h>
>>> #include <linux/platform_device.h>
>>> +#include <linux/pm_domain.h>
>>> #include <linux/pm_runtime.h>
>>> #include <linux/reset.h>
>>> #include <linux/units.h>
>>> @@ -164,6 +165,17 @@ static int rzg2l_wdt_restart(struct watchdog_device *wdev,
>>> struct rzg2l_wdt_priv *priv = watchdog_get_drvdata(wdev);
>>> int ret;
>>>
>>> + /*
>>> + * The device may be part of a power domain that is currently
>>> + * powered off. We need to power it up before accessing registers.
>>> + * We don't undo the dev_pm_genpd_resume() as the device need to
>>> + * be up for the reboot to happen. Also, as we are in atomic context
>>> + * here there is no need to increment PM runtime usage counter
>>> + * (to make sure pm_runtime_active() doesn't return wrong code).
>>> + */
>>> + if (!pm_runtime_active(wdev->parent))
>>> + dev_pm_genpd_resume(wdev->parent);
>>> +
>>
>> I doubt this is the correct solution, but I may be wrong. Unless this
>> is invoked at the syscore stage?
>
> On my case I see it invoked from kernel_restart(). As of my code reading,
With the above explanations, do you consider calling dev_pm_genpd_resume()
here is still wrong?
Do you have any suggestions I could try?
Thank you,
Claudiu Beznea
> at that point only one CPU is active with IRQs disabled (done in
> machine_restart()). Below is the stack trace decoded on next-20240410 with
> this series
> (https://lore.kernel.org/all/[email protected]/)
> on top and the one from here (adding power domain support):
> https://lore.kernel.org/all/[email protected]/
>
> Hardware name: Renesas SMARC EVK version 2 based on r9a08g045s33 (DT)
> Call trace:
> dump_backtrace (arch/arm64/kernel/stacktrace.c:319)
> show_stack (arch/arm64/kernel/stacktrace.c:326)
> dump_stack_lvl (lib/dump_stack.c:117)
> dump_stack (lib/dump_stack.c:124)
> rzg2l_wdt_restart (drivers/watchdog/rzg2l_wdt.c:180)
> watchdog_restart_notifier (drivers/watchdog/watchdog_core.c:188)
> atomic_notifier_call_chain (kernel/notifier.c:98 kernel/notifier.c:231)
> do_kernel_restart (kernel/reboot.c:236)
> machine_restart (arch/arm64/kernel/process.c:145)
> kernel_restart (kernel/reboot.c:287)
> __do_sys_reboot (kernel/reboot.c:755)
> __arm64_sys_reboot (kernel/reboot.c:715)
> invoke_syscall (arch/arm64/include/asm/current.h:19
> arch/arm64/kernel/syscall.c:53)
> el0_svc_common.constprop.0 (include/linux/thread_info.h:127
> arch/arm64/kernel/syscall.c:141)
> do_el0_svc (arch/arm64/kernel/syscall.c:153)
> el0_svc (arch/arm64/include/asm/irqflags.h:56
> arch/arm64/include/asm/irqflags.h:77 arch/arm64/kernel/entry-common.c:165
> arch/arm64/kernel/entry-common.c:178 arch/arm64/kernel/entry-common.c:713)
> el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:731)
> el0t_64_sync (arch/arm64/kernel/entry.S:598)
>
> The watchdog restart handler is added in restart_handler_list and this list
> is invoked though do_kernel_restart(). As of my code investigation the
> restart_handler_list is invoked only though do_kernel_restart() and only
> though the stack trace above.
>
> Thank you,
> Claudiu Beznea
>
>>
>>> clk_prepare_enable(priv->pclk);
>>> clk_prepare_enable(priv->osc_clk);
>>>
>>> --
>>> 2.39.2
>>>
>>>
>>
>> Can you redirectly me to the complete series, so I can have a better
>> overview of the problem?
>
> This is the series that adds power domain support for RZ/G3S SoC:
> https://lore.kernel.org/all/[email protected]/
>
> This is the series that adds watchdog support for RZ/G3S SoC:
> https://lore.kernel.org/all/[email protected]/
>
> Thank you for your review,
> Claudiu Beznea
>
>>
>> Kind regards
>> Uffe
On Wed, 24 Apr 2024 at 13:14, claudiu beznea <[email protected]> wrote:
>
> Hi, Ulf,
>
> On 12.04.2024 17:02, claudiu beznea wrote:
> > Hi, Ulf,
> >
> > On 12.04.2024 14:14, Ulf Hansson wrote:
> >> On Wed, 10 Apr 2024 at 16:19, Claudiu <[email protected]> wrote:
> >>>
> >>> From: Claudiu Beznea <[email protected]>
> >>>
> >>> The rzg2l_wdt_restart() is called from atomic context. Calling
> >>> pm_runtime_{get_sync, resume_and_get}() or any other runtime PM resume
> >>> APIs is not an option as it may lead to issues as described in commit
> >>> e4cf89596c1f ("watchdog: rzg2l_wdt: Fix 'BUG: Invalid wait context'")
> >>> that removed the pm_runtime_get_sync() and used directly the
> >>> clk_prepare_enable() APIs.
> >>>
> >>> Starting with RZ/G3S the watchdog could be part of its own software
> >>> controlled power domain (see the initial implementation in Link section).
> >>> In case the watchdog is not used the power domain is off and accessing
> >>> watchdog registers leads to aborts.
> >>>
> >>> To solve this the patch powers on the power domain using
> >>> dev_pm_genpd_resume() API before enabling its clock. This is not
> >>> sleeping or taking any other locks as the power domain will not be
> >>> registered with GENPD_FLAG_IRQ_SAFE flags.
> >>>
> >>> Link: https://lore.kernel.org/all/[email protected]
> >>> Signed-off-by: Claudiu Beznea <[email protected]>
> >>> ---
> >>>
> >>> Changes in v8:
> >>> - none, this patch is new
> >>>
> >>> drivers/watchdog/rzg2l_wdt.c | 12 ++++++++++++
> >>> 1 file changed, 12 insertions(+)
> >>>
> >>> diff --git a/drivers/watchdog/rzg2l_wdt.c b/drivers/watchdog/rzg2l_wdt.c
> >>> index c8c20cfb97a3..98e5e9914a5d 100644
> >>> --- a/drivers/watchdog/rzg2l_wdt.c
> >>> +++ b/drivers/watchdog/rzg2l_wdt.c
> >>> @@ -12,6 +12,7 @@
> >>> #include <linux/module.h>
> >>> #include <linux/of.h>
> >>> #include <linux/platform_device.h>
> >>> +#include <linux/pm_domain.h>
> >>> #include <linux/pm_runtime.h>
> >>> #include <linux/reset.h>
> >>> #include <linux/units.h>
> >>> @@ -164,6 +165,17 @@ static int rzg2l_wdt_restart(struct watchdog_device *wdev,
> >>> struct rzg2l_wdt_priv *priv = watchdog_get_drvdata(wdev);
> >>> int ret;
> >>>
> >>> + /*
> >>> + * The device may be part of a power domain that is currently
> >>> + * powered off. We need to power it up before accessing registers.
> >>> + * We don't undo the dev_pm_genpd_resume() as the device need to
> >>> + * be up for the reboot to happen. Also, as we are in atomic context
> >>> + * here there is no need to increment PM runtime usage counter
> >>> + * (to make sure pm_runtime_active() doesn't return wrong code).
> >>> + */
> >>> + if (!pm_runtime_active(wdev->parent))
> >>> + dev_pm_genpd_resume(wdev->parent);
> >>> +
> >>
> >> I doubt this is the correct solution, but I may be wrong. Unless this
> >> is invoked at the syscore stage?
> >
> > On my case I see it invoked from kernel_restart(). As of my code reading,
>
> With the above explanations, do you consider calling dev_pm_genpd_resume()
> here is still wrong?
Yes. At least, those genpd functions were not added to cope for cases like this.
Moreover, you still need to find another solution as
clk_prepare_enable() can't be called in this path.
>
> Do you have any suggestions I could try?
Not at the moment, but I will try to circle back to this topic more
thinking next week, when I have some more time.
>
> Thank you,
> Claudiu Beznea
Kind regards
Uffe
>
> > at that point only one CPU is active with IRQs disabled (done in
> > machine_restart()). Below is the stack trace decoded on next-20240410 with
> > this series
> > (https://lore.kernel.org/all/[email protected]/)
> > on top and the one from here (adding power domain support):
> > https://lore.kernel.org/all/[email protected]/
> >
> > Hardware name: Renesas SMARC EVK version 2 based on r9a08g045s33 (DT)
> > Call trace:
> > dump_backtrace (arch/arm64/kernel/stacktrace.c:319)
> > show_stack (arch/arm64/kernel/stacktrace.c:326)
> > dump_stack_lvl (lib/dump_stack.c:117)
> > dump_stack (lib/dump_stack.c:124)
> > rzg2l_wdt_restart (drivers/watchdog/rzg2l_wdt.c:180)
> > watchdog_restart_notifier (drivers/watchdog/watchdog_core.c:188)
> > atomic_notifier_call_chain (kernel/notifier.c:98 kernel/notifier.c:231)
> > do_kernel_restart (kernel/reboot.c:236)
> > machine_restart (arch/arm64/kernel/process.c:145)
> > kernel_restart (kernel/reboot.c:287)
> > __do_sys_reboot (kernel/reboot.c:755)
> > __arm64_sys_reboot (kernel/reboot.c:715)
> > invoke_syscall (arch/arm64/include/asm/current.h:19
> > arch/arm64/kernel/syscall.c:53)
> > el0_svc_common.constprop.0 (include/linux/thread_info.h:127
> > arch/arm64/kernel/syscall.c:141)
> > do_el0_svc (arch/arm64/kernel/syscall.c:153)
> > el0_svc (arch/arm64/include/asm/irqflags.h:56
> > arch/arm64/include/asm/irqflags.h:77 arch/arm64/kernel/entry-common.c:165
> > arch/arm64/kernel/entry-common.c:178 arch/arm64/kernel/entry-common.c:713)
> > el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:731)
> > el0t_64_sync (arch/arm64/kernel/entry.S:598)
> >
> > The watchdog restart handler is added in restart_handler_list and this list
> > is invoked though do_kernel_restart(). As of my code investigation the
> > restart_handler_list is invoked only though do_kernel_restart() and only
> > though the stack trace above.
> >
> > Thank you,
> > Claudiu Beznea
> >
> >>
> >>> clk_prepare_enable(priv->pclk);
> >>> clk_prepare_enable(priv->osc_clk);
> >>>
> >>> --
> >>> 2.39.2
> >>>
> >>>
> >>
> >> Can you redirectly me to the complete series, so I can have a better
> >> overview of the problem?
> >
> > This is the series that adds power domain support for RZ/G3S SoC:
> > https://lore.kernel.org/all/[email protected]/
> >
> > This is the series that adds watchdog support for RZ/G3S SoC:
> > https://lore.kernel.org/all/[email protected]/
> >
> > Thank you for your review,
> > Claudiu Beznea
> >
> >>
> >> Kind regards
> >> Uffe
Hi, Ulf,
On 29.04.2024 17:19, Ulf Hansson wrote:
> On Wed, 24 Apr 2024 at 13:14, claudiu beznea <[email protected]> wrote:
>>
>> Hi, Ulf,
>>
>> On 12.04.2024 17:02, claudiu beznea wrote:
>>> Hi, Ulf,
>>>
>>> On 12.04.2024 14:14, Ulf Hansson wrote:
>>>> On Wed, 10 Apr 2024 at 16:19, Claudiu <[email protected]> wrote:
>>>>>
>>>>> From: Claudiu Beznea <[email protected]>
>>>>>
>>>>> The rzg2l_wdt_restart() is called from atomic context. Calling
>>>>> pm_runtime_{get_sync, resume_and_get}() or any other runtime PM resume
>>>>> APIs is not an option as it may lead to issues as described in commit
>>>>> e4cf89596c1f ("watchdog: rzg2l_wdt: Fix 'BUG: Invalid wait context'")
>>>>> that removed the pm_runtime_get_sync() and used directly the
>>>>> clk_prepare_enable() APIs.
>>>>>
>>>>> Starting with RZ/G3S the watchdog could be part of its own software
>>>>> controlled power domain (see the initial implementation in Link section).
>>>>> In case the watchdog is not used the power domain is off and accessing
>>>>> watchdog registers leads to aborts.
>>>>>
>>>>> To solve this the patch powers on the power domain using
>>>>> dev_pm_genpd_resume() API before enabling its clock. This is not
>>>>> sleeping or taking any other locks as the power domain will not be
>>>>> registered with GENPD_FLAG_IRQ_SAFE flags.
>>>>>
>>>>> Link: https://lore.kernel.org/all/[email protected]
>>>>> Signed-off-by: Claudiu Beznea <[email protected]>
>>>>> ---
>>>>>
>>>>> Changes in v8:
>>>>> - none, this patch is new
>>>>>
>>>>> drivers/watchdog/rzg2l_wdt.c | 12 ++++++++++++
>>>>> 1 file changed, 12 insertions(+)
>>>>>
>>>>> diff --git a/drivers/watchdog/rzg2l_wdt.c b/drivers/watchdog/rzg2l_wdt.c
>>>>> index c8c20cfb97a3..98e5e9914a5d 100644
>>>>> --- a/drivers/watchdog/rzg2l_wdt.c
>>>>> +++ b/drivers/watchdog/rzg2l_wdt.c
>>>>> @@ -12,6 +12,7 @@
>>>>> #include <linux/module.h>
>>>>> #include <linux/of.h>
>>>>> #include <linux/platform_device.h>
>>>>> +#include <linux/pm_domain.h>
>>>>> #include <linux/pm_runtime.h>
>>>>> #include <linux/reset.h>
>>>>> #include <linux/units.h>
>>>>> @@ -164,6 +165,17 @@ static int rzg2l_wdt_restart(struct watchdog_device *wdev,
>>>>> struct rzg2l_wdt_priv *priv = watchdog_get_drvdata(wdev);
>>>>> int ret;
>>>>>
>>>>> + /*
>>>>> + * The device may be part of a power domain that is currently
>>>>> + * powered off. We need to power it up before accessing registers.
>>>>> + * We don't undo the dev_pm_genpd_resume() as the device need to
>>>>> + * be up for the reboot to happen. Also, as we are in atomic context
>>>>> + * here there is no need to increment PM runtime usage counter
>>>>> + * (to make sure pm_runtime_active() doesn't return wrong code).
>>>>> + */
>>>>> + if (!pm_runtime_active(wdev->parent))
>>>>> + dev_pm_genpd_resume(wdev->parent);
>>>>> +
>>>>
>>>> I doubt this is the correct solution, but I may be wrong. Unless this
>>>> is invoked at the syscore stage?
>>>
>>> On my case I see it invoked from kernel_restart(). As of my code reading,
>>
>> With the above explanations, do you consider calling dev_pm_genpd_resume()
>> here is still wrong?
>
> Yes. At least, those genpd functions were not added to cope for cases like this.
Sorry to bother you, do you have some suggestions on this topic?
On my side I did some investigation to see how else it could be implemented
but I don't have much clue how to go forward.
Would you prefer to have a separate API to deal with domain power on in
this scenario? Maybe one that should run only in the reboot context?
Would you consider only updating the description of dev_pm_genpd_resume()
and genpd_sync_power_on() to specify that it could run in a reboot context?
Would you consider updating the genpd_switch_state() to take into system
reboot state and do locking based on that, too?
>
> Moreover, you still need to find another solution as
> clk_prepare_enable() can't be called in this path.
The clock driver doesn't implement clk_ops::prepare in all
micro-architectures that this watchdog driver is used. This may be the
reason the clk_prepare_enable() was used on this path from the beginning.
Even though, a simple solution I have in mind for this is to keep the clk
prepared all the time.
Thank you,
Claudiu Beznea
>
>>
>> Do you have any suggestions I could try?
>
> Not at the moment, but I will try to circle back to this topic more
> thinking next week, when I have some more time.
>
>>
>> Thank you,
>> Claudiu Beznea
>
> Kind regards
> Uffe
>
>>
>>> at that point only one CPU is active with IRQs disabled (done in
>>> machine_restart()). Below is the stack trace decoded on next-20240410 with
>>> this series
>>> (https://lore.kernel.org/all/[email protected]/)
>>> on top and the one from here (adding power domain support):
>>> https://lore.kernel.org/all/[email protected]/
>>>
>>> Hardware name: Renesas SMARC EVK version 2 based on r9a08g045s33 (DT)
>>> Call trace:
>>> dump_backtrace (arch/arm64/kernel/stacktrace.c:319)
>>> show_stack (arch/arm64/kernel/stacktrace.c:326)
>>> dump_stack_lvl (lib/dump_stack.c:117)
>>> dump_stack (lib/dump_stack.c:124)
>>> rzg2l_wdt_restart (drivers/watchdog/rzg2l_wdt.c:180)
>>> watchdog_restart_notifier (drivers/watchdog/watchdog_core.c:188)
>>> atomic_notifier_call_chain (kernel/notifier.c:98 kernel/notifier.c:231)
>>> do_kernel_restart (kernel/reboot.c:236)
>>> machine_restart (arch/arm64/kernel/process.c:145)
>>> kernel_restart (kernel/reboot.c:287)
>>> __do_sys_reboot (kernel/reboot.c:755)
>>> __arm64_sys_reboot (kernel/reboot.c:715)
>>> invoke_syscall (arch/arm64/include/asm/current.h:19
>>> arch/arm64/kernel/syscall.c:53)
>>> el0_svc_common.constprop.0 (include/linux/thread_info.h:127
>>> arch/arm64/kernel/syscall.c:141)
>>> do_el0_svc (arch/arm64/kernel/syscall.c:153)
>>> el0_svc (arch/arm64/include/asm/irqflags.h:56
>>> arch/arm64/include/asm/irqflags.h:77 arch/arm64/kernel/entry-common.c:165
>>> arch/arm64/kernel/entry-common.c:178 arch/arm64/kernel/entry-common.c:713)
>>> el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:731)
>>> el0t_64_sync (arch/arm64/kernel/entry.S:598)
>>>
>>> The watchdog restart handler is added in restart_handler_list and this list
>>> is invoked though do_kernel_restart(). As of my code investigation the
>>> restart_handler_list is invoked only though do_kernel_restart() and only
>>> though the stack trace above.
>>>
>>> Thank you,
>>> Claudiu Beznea
>>>
>>>>
>>>>> clk_prepare_enable(priv->pclk);
>>>>> clk_prepare_enable(priv->osc_clk);
>>>>>
>>>>> --
>>>>> 2.39.2
>>>>>
>>>>>
>>>>
>>>> Can you redirectly me to the complete series, so I can have a better
>>>> overview of the problem?
>>>
>>> This is the series that adds power domain support for RZ/G3S SoC:
>>> https://lore.kernel.org/all/[email protected]/
>>>
>>> This is the series that adds watchdog support for RZ/G3S SoC:
>>> https://lore.kernel.org/all/[email protected]/
>>>
>>> Thank you for your review,
>>> Claudiu Beznea
>>>
>>>>
>>>> Kind regards
>>>> Uffe