Now the System stall is observed on TI AM437x based board
(am437x-gp-evm) during resuming from System suspend when ARM Global
timer is selected as clocksource device - SysRq are working, but
nothing else. The reason of stall is that ARM Global timer loses its
contexts.
The reason of stall is that ARM Global timer loses its contexts during
System suspend:
GT_CONTROL.TIMER_ENABLE = 0 (unbanked)
GT_COUNTERx = 0
Hence, update ARM Global timer driver to reflect above behaviour
- save GT_CONTROL.TIMER_ENABLE during suspend and restore on resume;
- ensure clocksource and clockevent devices have coresponding flags
(CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set
depending on presence of "always-on" DT property.
CC: Arnd Bergmann <[email protected]>
Cc: John Stultz <[email protected]>
Cc: Felipe Balbi <[email protected]>
Cc: Tony Lindgren <[email protected]>
Signed-off-by: Grygorii Strashko <[email protected]>
---
drivers/clocksource/arm_global_timer.c | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c
index a2cb6fa..1bbaf64 100644
--- a/drivers/clocksource/arm_global_timer.c
+++ b/drivers/clocksource/arm_global_timer.c
@@ -51,6 +51,8 @@ static void __iomem *gt_base;
static unsigned long gt_clk_rate;
static int gt_ppi;
static struct clock_event_device __percpu *gt_evt;
+static bool gt_always_on;
+static u32 gt_control;
/*
* To get the value from the Global Timer Counter register proceed as follows:
@@ -168,6 +170,9 @@ static int gt_clockevents_init(struct clock_event_device *clk)
{
int cpu = smp_processor_id();
+ if (!gt_always_on)
+ clk->features |= CLOCK_EVT_FEAT_C3STOP;
+
clk->name = "arm_global_timer";
clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT |
CLOCK_EVT_FEAT_PERCPU;
@@ -195,12 +200,25 @@ static cycle_t gt_clocksource_read(struct clocksource *cs)
return gt_counter_read();
}
+static void gt_suspend(struct clocksource *cs)
+{
+ gt_control = readl(gt_base + GT_CONTROL);
+}
+
+static void gt_resume(struct clocksource *cs)
+{
+ /* enables timer on all the cores */
+ writel(gt_control & GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
+}
+
static struct clocksource gt_clocksource = {
.name = "arm_global_timer",
.rating = 300,
.read = gt_clocksource_read,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
+ .suspend = gt_suspend,
+ .resume = gt_resume,
};
#ifdef CONFIG_CLKSRC_ARM_GLOBAL_TIMER_SCHED_CLOCK
@@ -218,6 +236,9 @@ static void __init gt_clocksource_init(void)
/* enables timer on all the cores */
writel(GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
+ if (gt_always_on)
+ gt_clocksource.flags |= CLOCK_SOURCE_SUSPEND_NONSTOP;
+
#ifdef CONFIG_CLKSRC_ARM_GLOBAL_TIMER_SCHED_CLOCK
sched_clock_register(gt_sched_clock_read, 64, gt_clk_rate);
#endif
@@ -289,6 +310,8 @@ static void __init global_timer_of_register(struct device_node *np)
goto out_clk;
}
+ gt_always_on = of_property_read_bool(np, "always-on");
+
err = request_percpu_irq(gt_ppi, gt_clockevent_interrupt,
"gt", gt_evt);
if (err) {
--
2.6.3
Hi,
Grygorii Strashko <[email protected]> writes:
> Now the System stall is observed on TI AM437x based board
> (am437x-gp-evm) during resuming from System suspend when ARM Global
> timer is selected as clocksource device - SysRq are working, but
> nothing else. The reason of stall is that ARM Global timer loses its
> contexts.
>
> The reason of stall is that ARM Global timer loses its contexts during
> System suspend:
> GT_CONTROL.TIMER_ENABLE = 0 (unbanked)
> GT_COUNTERx = 0
>
> Hence, update ARM Global timer driver to reflect above behaviour
> - save GT_CONTROL.TIMER_ENABLE during suspend and restore on resume;
> - ensure clocksource and clockevent devices have coresponding flags
> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set
> depending on presence of "always-on" DT property.
>
> CC: Arnd Bergmann <[email protected]>
> Cc: John Stultz <[email protected]>
> Cc: Felipe Balbi <[email protected]>
> Cc: Tony Lindgren <[email protected]>
> Signed-off-by: Grygorii Strashko <[email protected]>
> ---
> drivers/clocksource/arm_global_timer.c | 23 +++++++++++++++++++++++
> 1 file changed, 23 insertions(+)
>
> diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c
> index a2cb6fa..1bbaf64 100644
> --- a/drivers/clocksource/arm_global_timer.c
> +++ b/drivers/clocksource/arm_global_timer.c
> @@ -51,6 +51,8 @@ static void __iomem *gt_base;
> static unsigned long gt_clk_rate;
> static int gt_ppi;
> static struct clock_event_device __percpu *gt_evt;
> +static bool gt_always_on;
> +static u32 gt_control;
>
> /*
> * To get the value from the Global Timer Counter register proceed as follows:
> @@ -168,6 +170,9 @@ static int gt_clockevents_init(struct clock_event_device *clk)
> {
> int cpu = smp_processor_id();
>
> + if (!gt_always_on)
> + clk->features |= CLOCK_EVT_FEAT_C3STOP;
> +
> clk->name = "arm_global_timer";
> clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT |
> CLOCK_EVT_FEAT_PERCPU;
> @@ -195,12 +200,25 @@ static cycle_t gt_clocksource_read(struct clocksource *cs)
> return gt_counter_read();
> }
>
> +static void gt_suspend(struct clocksource *cs)
> +{
> + gt_control = readl(gt_base + GT_CONTROL);
> +}
> +
> +static void gt_resume(struct clocksource *cs)
> +{
> + /* enables timer on all the cores */
> + writel(gt_control & GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
do you really need to save context if all you restore is TIMER_ENABLE
bit ? seems like you could skip gt_suspend altogether. Is there really a
situation where this driver is running and GT isn't enabled ?
--
balbi
On 11/13/2015 06:43 PM, Felipe Balbi wrote:
>
> Hi,
>
> Grygorii Strashko <[email protected]> writes:
>> Now the System stall is observed on TI AM437x based board
>> (am437x-gp-evm) during resuming from System suspend when ARM Global
>> timer is selected as clocksource device - SysRq are working, but
>> nothing else. The reason of stall is that ARM Global timer loses its
>> contexts.
>>
>> The reason of stall is that ARM Global timer loses its contexts during
>> System suspend:
>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked)
>> GT_COUNTERx = 0
>>
>> Hence, update ARM Global timer driver to reflect above behaviour
>> - save GT_CONTROL.TIMER_ENABLE during suspend and restore on resume;
>> - ensure clocksource and clockevent devices have coresponding flags
>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set
>> depending on presence of "always-on" DT property.
>>
>> CC: Arnd Bergmann <[email protected]>
>> Cc: John Stultz <[email protected]>
>> Cc: Felipe Balbi <[email protected]>
>> Cc: Tony Lindgren <[email protected]>
>> Signed-off-by: Grygorii Strashko <[email protected]>
>> ---
>> drivers/clocksource/arm_global_timer.c | 23 +++++++++++++++++++++++
>> 1 file changed, 23 insertions(+)
>>
>> diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c
>> index a2cb6fa..1bbaf64 100644
>> --- a/drivers/clocksource/arm_global_timer.c
>> +++ b/drivers/clocksource/arm_global_timer.c
>> @@ -51,6 +51,8 @@ static void __iomem *gt_base;
>> static unsigned long gt_clk_rate;
>> static int gt_ppi;
>> static struct clock_event_device __percpu *gt_evt;
>> +static bool gt_always_on;
>> +static u32 gt_control;
>>
>> /*
>> * To get the value from the Global Timer Counter register proceed as follows:
>> @@ -168,6 +170,9 @@ static int gt_clockevents_init(struct clock_event_device *clk)
>> {
>> int cpu = smp_processor_id();
>>
>> + if (!gt_always_on)
>> + clk->features |= CLOCK_EVT_FEAT_C3STOP;
>> +
>> clk->name = "arm_global_timer";
>> clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT |
>> CLOCK_EVT_FEAT_PERCPU;
>> @@ -195,12 +200,25 @@ static cycle_t gt_clocksource_read(struct clocksource *cs)
>> return gt_counter_read();
>> }
>>
>> +static void gt_suspend(struct clocksource *cs)
>> +{
>> + gt_control = readl(gt_base + GT_CONTROL);
>> +}
>> +
>> +static void gt_resume(struct clocksource *cs)
>> +{
>> + /* enables timer on all the cores */
>> + writel(gt_control & GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
>
> do you really need to save context if all you restore is TIMER_ENABLE
> bit ? seems like you could skip gt_suspend altogether. Is there really a
> situation where this driver is running and GT isn't enabled ?
Now It's not. It's always enabled. I did it because .suspend() is called for
all registered clock sources regardless of their usage. So, potentially
in the future, at the moment when .suspend() is called it might be disabled
(for example, .enable/disable() callbacks can be added and, if ARM Global timer
will not be registered as sched_clock, it will be possible to keep it disabled
if not used now).
But It's not essentially now - I can update it and drop save restore.
Pls, confirm.
--
regards,
-grygorii
Hi,
Grygorii Strashko <[email protected]> writes:
> On 11/13/2015 06:43 PM, Felipe Balbi wrote:
>>
>> Hi,
>>
>> Grygorii Strashko <[email protected]> writes:
>>> Now the System stall is observed on TI AM437x based board
>>> (am437x-gp-evm) during resuming from System suspend when ARM Global
>>> timer is selected as clocksource device - SysRq are working, but
>>> nothing else. The reason of stall is that ARM Global timer loses its
>>> contexts.
>>>
>>> The reason of stall is that ARM Global timer loses its contexts during
>>> System suspend:
>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked)
>>> GT_COUNTERx = 0
>>>
>>> Hence, update ARM Global timer driver to reflect above behaviour
>>> - save GT_CONTROL.TIMER_ENABLE during suspend and restore on resume;
>>> - ensure clocksource and clockevent devices have coresponding flags
>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set
>>> depending on presence of "always-on" DT property.
>>>
>>> CC: Arnd Bergmann <[email protected]>
>>> Cc: John Stultz <[email protected]>
>>> Cc: Felipe Balbi <[email protected]>
>>> Cc: Tony Lindgren <[email protected]>
>>> Signed-off-by: Grygorii Strashko <[email protected]>
>>> ---
>>> drivers/clocksource/arm_global_timer.c | 23 +++++++++++++++++++++++
>>> 1 file changed, 23 insertions(+)
>>>
>>> diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c
>>> index a2cb6fa..1bbaf64 100644
>>> --- a/drivers/clocksource/arm_global_timer.c
>>> +++ b/drivers/clocksource/arm_global_timer.c
>>> @@ -51,6 +51,8 @@ static void __iomem *gt_base;
>>> static unsigned long gt_clk_rate;
>>> static int gt_ppi;
>>> static struct clock_event_device __percpu *gt_evt;
>>> +static bool gt_always_on;
>>> +static u32 gt_control;
>>>
>>> /*
>>> * To get the value from the Global Timer Counter register proceed as follows:
>>> @@ -168,6 +170,9 @@ static int gt_clockevents_init(struct clock_event_device *clk)
>>> {
>>> int cpu = smp_processor_id();
>>>
>>> + if (!gt_always_on)
>>> + clk->features |= CLOCK_EVT_FEAT_C3STOP;
>>> +
>>> clk->name = "arm_global_timer";
>>> clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT |
>>> CLOCK_EVT_FEAT_PERCPU;
>>> @@ -195,12 +200,25 @@ static cycle_t gt_clocksource_read(struct clocksource *cs)
>>> return gt_counter_read();
>>> }
>>>
>>> +static void gt_suspend(struct clocksource *cs)
>>> +{
>>> + gt_control = readl(gt_base + GT_CONTROL);
>>> +}
>>> +
>>> +static void gt_resume(struct clocksource *cs)
>>> +{
>>> + /* enables timer on all the cores */
>>> + writel(gt_control & GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
>>
>> do you really need to save context if all you restore is TIMER_ENABLE
>> bit ? seems like you could skip gt_suspend altogether. Is there really a
>> situation where this driver is running and GT isn't enabled ?
>
> Now It's not. It's always enabled. I did it because .suspend() is called for
> all registered clock sources regardless of their usage. So, potentially
> in the future, at the moment when .suspend() is called it might be disabled
> (for example, .enable/disable() callbacks can be added and, if ARM Global timer
> will not be registered as sched_clock, it will be possible to keep it disabled
> if not used now).
>
> But It's not essentially now - I can update it and drop save restore.
> Pls, confirm.
I think it's best to skip suspend completely. You're not restoring
anything you saved during suspend, unless you meant | where you used &.
--
balbi
On 11/13/2015 07:40 PM, Felipe Balbi wrote:
>
> Hi,
>
> Grygorii Strashko <[email protected]> writes:
>> On 11/13/2015 06:43 PM, Felipe Balbi wrote:
>>>
>>> Hi,
>>>
>>> Grygorii Strashko <[email protected]> writes:
>>>> Now the System stall is observed on TI AM437x based board
>>>> (am437x-gp-evm) during resuming from System suspend when ARM Global
>>>> timer is selected as clocksource device - SysRq are working, but
>>>> nothing else. The reason of stall is that ARM Global timer loses its
>>>> contexts.
>>>>
>>>> The reason of stall is that ARM Global timer loses its contexts during
>>>> System suspend:
>>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked)
>>>> GT_COUNTERx = 0
>>>>
>>>> Hence, update ARM Global timer driver to reflect above behaviour
>>>> - save GT_CONTROL.TIMER_ENABLE during suspend and restore on resume;
>>>> - ensure clocksource and clockevent devices have coresponding flags
>>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set
>>>> depending on presence of "always-on" DT property.
>>>>
>>>> CC: Arnd Bergmann <[email protected]>
>>>> Cc: John Stultz <[email protected]>
>>>> Cc: Felipe Balbi <[email protected]>
>>>> Cc: Tony Lindgren <[email protected]>
>>>> Signed-off-by: Grygorii Strashko <[email protected]>
>>>> ---
>>>> drivers/clocksource/arm_global_timer.c | 23 +++++++++++++++++++++++
>>>> 1 file changed, 23 insertions(+)
>>>>
>>>> diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c
>>>> index a2cb6fa..1bbaf64 100644
>>>> --- a/drivers/clocksource/arm_global_timer.c
>>>> +++ b/drivers/clocksource/arm_global_timer.c
>>>> @@ -51,6 +51,8 @@ static void __iomem *gt_base;
>>>> static unsigned long gt_clk_rate;
>>>> static int gt_ppi;
>>>> static struct clock_event_device __percpu *gt_evt;
>>>> +static bool gt_always_on;
>>>> +static u32 gt_control;
>>>>
>>>> /*
>>>> * To get the value from the Global Timer Counter register proceed as follows:
>>>> @@ -168,6 +170,9 @@ static int gt_clockevents_init(struct clock_event_device *clk)
>>>> {
>>>> int cpu = smp_processor_id();
>>>>
>>>> + if (!gt_always_on)
>>>> + clk->features |= CLOCK_EVT_FEAT_C3STOP;
>>>> +
>>>> clk->name = "arm_global_timer";
>>>> clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT |
>>>> CLOCK_EVT_FEAT_PERCPU;
>>>> @@ -195,12 +200,25 @@ static cycle_t gt_clocksource_read(struct clocksource *cs)
>>>> return gt_counter_read();
>>>> }
>>>>
>>>> +static void gt_suspend(struct clocksource *cs)
>>>> +{
>>>> + gt_control = readl(gt_base + GT_CONTROL);
>>>> +}
>>>> +
>>>> +static void gt_resume(struct clocksource *cs)
>>>> +{
>>>> + /* enables timer on all the cores */
>>>> + writel(gt_control & GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
>>>
>>> do you really need to save context if all you restore is TIMER_ENABLE
>>> bit ? seems like you could skip gt_suspend altogether. Is there really a
>>> situation where this driver is running and GT isn't enabled ?
>>
>> Now It's not. It's always enabled. I did it because .suspend() is called for
>> all registered clock sources regardless of their usage. So, potentially
>> in the future, at the moment when .suspend() is called it might be disabled
>> (for example, .enable/disable() callbacks can be added and, if ARM Global timer
>> will not be registered as sched_clock, it will be possible to keep it disabled
>> if not used now).
>>
>> But It's not essentially now - I can update it and drop save restore.
>> Pls, confirm.
>
> I think it's best to skip suspend completely. You're not restoring
> anything you saved during suspend, unless you meant | where you used &.
>
I didn't get it - I'm restoring one bit(0) only.
--
regards,
-grygorii
Hi,
Grygorii Strashko <[email protected]> writes:
> On 11/13/2015 07:40 PM, Felipe Balbi wrote:
>>
>> Hi,
>>
>> Grygorii Strashko <[email protected]> writes:
>>> On 11/13/2015 06:43 PM, Felipe Balbi wrote:
>>>>
>>>> Hi,
>>>>
>>>> Grygorii Strashko <[email protected]> writes:
>>>>> Now the System stall is observed on TI AM437x based board
>>>>> (am437x-gp-evm) during resuming from System suspend when ARM Global
>>>>> timer is selected as clocksource device - SysRq are working, but
>>>>> nothing else. The reason of stall is that ARM Global timer loses its
>>>>> contexts.
>>>>>
>>>>> The reason of stall is that ARM Global timer loses its contexts during
>>>>> System suspend:
>>>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked)
>>>>> GT_COUNTERx = 0
>>>>>
>>>>> Hence, update ARM Global timer driver to reflect above behaviour
>>>>> - save GT_CONTROL.TIMER_ENABLE during suspend and restore on resume;
>>>>> - ensure clocksource and clockevent devices have coresponding flags
>>>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set
>>>>> depending on presence of "always-on" DT property.
>>>>>
>>>>> CC: Arnd Bergmann <[email protected]>
>>>>> Cc: John Stultz <[email protected]>
>>>>> Cc: Felipe Balbi <[email protected]>
>>>>> Cc: Tony Lindgren <[email protected]>
>>>>> Signed-off-by: Grygorii Strashko <[email protected]>
>>>>> ---
>>>>> drivers/clocksource/arm_global_timer.c | 23 +++++++++++++++++++++++
>>>>> 1 file changed, 23 insertions(+)
>>>>>
>>>>> diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c
>>>>> index a2cb6fa..1bbaf64 100644
>>>>> --- a/drivers/clocksource/arm_global_timer.c
>>>>> +++ b/drivers/clocksource/arm_global_timer.c
>>>>> @@ -51,6 +51,8 @@ static void __iomem *gt_base;
>>>>> static unsigned long gt_clk_rate;
>>>>> static int gt_ppi;
>>>>> static struct clock_event_device __percpu *gt_evt;
>>>>> +static bool gt_always_on;
>>>>> +static u32 gt_control;
>>>>>
>>>>> /*
>>>>> * To get the value from the Global Timer Counter register proceed as follows:
>>>>> @@ -168,6 +170,9 @@ static int gt_clockevents_init(struct clock_event_device *clk)
>>>>> {
>>>>> int cpu = smp_processor_id();
>>>>>
>>>>> + if (!gt_always_on)
>>>>> + clk->features |= CLOCK_EVT_FEAT_C3STOP;
>>>>> +
>>>>> clk->name = "arm_global_timer";
>>>>> clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT |
>>>>> CLOCK_EVT_FEAT_PERCPU;
>>>>> @@ -195,12 +200,25 @@ static cycle_t gt_clocksource_read(struct clocksource *cs)
>>>>> return gt_counter_read();
>>>>> }
>>>>>
>>>>> +static void gt_suspend(struct clocksource *cs)
>>>>> +{
>>>>> + gt_control = readl(gt_base + GT_CONTROL);
>>>>> +}
>>>>> +
>>>>> +static void gt_resume(struct clocksource *cs)
>>>>> +{
>>>>> + /* enables timer on all the cores */
>>>>> + writel(gt_control & GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
>>>>
>>>> do you really need to save context if all you restore is TIMER_ENABLE
>>>> bit ? seems like you could skip gt_suspend altogether. Is there really a
>>>> situation where this driver is running and GT isn't enabled ?
>>>
>>> Now It's not. It's always enabled. I did it because .suspend() is called for
>>> all registered clock sources regardless of their usage. So, potentially
>>> in the future, at the moment when .suspend() is called it might be disabled
>>> (for example, .enable/disable() callbacks can be added and, if ARM Global timer
>>> will not be registered as sched_clock, it will be possible to keep it disabled
>>> if not used now).
>>>
>>> But It's not essentially now - I can update it and drop save restore.
>>> Pls, confirm.
>>
>> I think it's best to skip suspend completely. You're not restoring
>> anything you saved during suspend, unless you meant | where you used &.
>>
>
> I didn't get it - I'm restoring one bit(0) only.
that's the point, if you know you're restoring only that bit. Why save
anything at all ?
--
balbi
On 11/13/2015 08:15 PM, Felipe Balbi wrote:
>
> Hi,
>
> Grygorii Strashko <[email protected]> writes:
>> On 11/13/2015 07:40 PM, Felipe Balbi wrote:
>>>
>>> Hi,
>>>
>>> Grygorii Strashko <[email protected]> writes:
>>>> On 11/13/2015 06:43 PM, Felipe Balbi wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Grygorii Strashko <[email protected]> writes:
>>>>>> Now the System stall is observed on TI AM437x based board
>>>>>> (am437x-gp-evm) during resuming from System suspend when ARM Global
>>>>>> timer is selected as clocksource device - SysRq are working, but
>>>>>> nothing else. The reason of stall is that ARM Global timer loses its
>>>>>> contexts.
>>>>>>
>>>>>> The reason of stall is that ARM Global timer loses its contexts during
>>>>>> System suspend:
>>>>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked)
>>>>>> GT_COUNTERx = 0
>>>>>>
>>>>>> Hence, update ARM Global timer driver to reflect above behaviour
>>>>>> - save GT_CONTROL.TIMER_ENABLE during suspend and restore on resume;
>>>>>> - ensure clocksource and clockevent devices have coresponding flags
>>>>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set
>>>>>> depending on presence of "always-on" DT property.
>>>>>>
>>>>>> CC: Arnd Bergmann <[email protected]>
>>>>>> Cc: John Stultz <[email protected]>
>>>>>> Cc: Felipe Balbi <[email protected]>
>>>>>> Cc: Tony Lindgren <[email protected]>
>>>>>> Signed-off-by: Grygorii Strashko <[email protected]>
>>>>>> ---
>>>>>> drivers/clocksource/arm_global_timer.c | 23 +++++++++++++++++++++++
>>>>>> 1 file changed, 23 insertions(+)
>>>>>>
>>>>>> diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c
>>>>>> index a2cb6fa..1bbaf64 100644
>>>>>> --- a/drivers/clocksource/arm_global_timer.c
>>>>>> +++ b/drivers/clocksource/arm_global_timer.c
>>>>>> @@ -51,6 +51,8 @@ static void __iomem *gt_base;
>>>>>> static unsigned long gt_clk_rate;
>>>>>> static int gt_ppi;
>>>>>> static struct clock_event_device __percpu *gt_evt;
>>>>>> +static bool gt_always_on;
>>>>>> +static u32 gt_control;
>>>>>>
>>>>>> /*
>>>>>> * To get the value from the Global Timer Counter register proceed as follows:
>>>>>> @@ -168,6 +170,9 @@ static int gt_clockevents_init(struct clock_event_device *clk)
>>>>>> {
>>>>>> int cpu = smp_processor_id();
>>>>>>
>>>>>> + if (!gt_always_on)
>>>>>> + clk->features |= CLOCK_EVT_FEAT_C3STOP;
>>>>>> +
>>>>>> clk->name = "arm_global_timer";
>>>>>> clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT |
>>>>>> CLOCK_EVT_FEAT_PERCPU;
>>>>>> @@ -195,12 +200,25 @@ static cycle_t gt_clocksource_read(struct clocksource *cs)
>>>>>> return gt_counter_read();
>>>>>> }
>>>>>>
>>>>>> +static void gt_suspend(struct clocksource *cs)
>>>>>> +{
>>>>>> + gt_control = readl(gt_base + GT_CONTROL);
>>>>>> +}
>>>>>> +
>>>>>> +static void gt_resume(struct clocksource *cs)
>>>>>> +{
>>>>>> + /* enables timer on all the cores */
>>>>>> + writel(gt_control & GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
>>>>>
>>>>> do you really need to save context if all you restore is TIMER_ENABLE
>>>>> bit ? seems like you could skip gt_suspend altogether. Is there really a
>>>>> situation where this driver is running and GT isn't enabled ?
>>>>
>>>> Now It's not. It's always enabled. I did it because .suspend() is called for
>>>> all registered clock sources regardless of their usage. So, potentially
>>>> in the future, at the moment when .suspend() is called it might be disabled
>>>> (for example, .enable/disable() callbacks can be added and, if ARM Global timer
>>>> will not be registered as sched_clock, it will be possible to keep it disabled
>>>> if not used now).
>>>>
>>>> But It's not essentially now - I can update it and drop save restore.
>>>> Pls, confirm.
>>>
>>> I think it's best to skip suspend completely. You're not restoring
>>> anything you saved during suspend, unless you meant | where you used &.
>>>
>>
>> I didn't get it - I'm restoring one bit(0) only.
>
> that's the point, if you know you're restoring only that bit. Why save
> anything at all ?
>
i think there are difference between "restoring" and "re-enabling".
"restoring" - assume saving smth.. then restore saving value.
I'm saving & restoring one bit here.
But I can do just "re-enabling" -
writel(GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
and then I don't need to save anything. It will work with current
code.
--
regards,
-grygorii
Hi,
Grygorii Strashko <[email protected]> writes:
> On 11/13/2015 08:15 PM, Felipe Balbi wrote:
>>
>> Hi,
>>
>> Grygorii Strashko <[email protected]> writes:
>>> On 11/13/2015 07:40 PM, Felipe Balbi wrote:
>>>>
>>>> Hi,
>>>>
>>>> Grygorii Strashko <[email protected]> writes:
>>>>> On 11/13/2015 06:43 PM, Felipe Balbi wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Grygorii Strashko <[email protected]> writes:
>>>>>>> Now the System stall is observed on TI AM437x based board
>>>>>>> (am437x-gp-evm) during resuming from System suspend when ARM Global
>>>>>>> timer is selected as clocksource device - SysRq are working, but
>>>>>>> nothing else. The reason of stall is that ARM Global timer loses its
>>>>>>> contexts.
>>>>>>>
>>>>>>> The reason of stall is that ARM Global timer loses its contexts during
>>>>>>> System suspend:
>>>>>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked)
>>>>>>> GT_COUNTERx = 0
>>>>>>>
>>>>>>> Hence, update ARM Global timer driver to reflect above behaviour
>>>>>>> - save GT_CONTROL.TIMER_ENABLE during suspend and restore on resume;
>>>>>>> - ensure clocksource and clockevent devices have coresponding flags
>>>>>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set
>>>>>>> depending on presence of "always-on" DT property.
>>>>>>>
>>>>>>> CC: Arnd Bergmann <[email protected]>
>>>>>>> Cc: John Stultz <[email protected]>
>>>>>>> Cc: Felipe Balbi <[email protected]>
>>>>>>> Cc: Tony Lindgren <[email protected]>
>>>>>>> Signed-off-by: Grygorii Strashko <[email protected]>
>>>>>>> ---
>>>>>>> drivers/clocksource/arm_global_timer.c | 23 +++++++++++++++++++++++
>>>>>>> 1 file changed, 23 insertions(+)
>>>>>>>
>>>>>>> diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c
>>>>>>> index a2cb6fa..1bbaf64 100644
>>>>>>> --- a/drivers/clocksource/arm_global_timer.c
>>>>>>> +++ b/drivers/clocksource/arm_global_timer.c
>>>>>>> @@ -51,6 +51,8 @@ static void __iomem *gt_base;
>>>>>>> static unsigned long gt_clk_rate;
>>>>>>> static int gt_ppi;
>>>>>>> static struct clock_event_device __percpu *gt_evt;
>>>>>>> +static bool gt_always_on;
>>>>>>> +static u32 gt_control;
>>>>>>>
>>>>>>> /*
>>>>>>> * To get the value from the Global Timer Counter register proceed as follows:
>>>>>>> @@ -168,6 +170,9 @@ static int gt_clockevents_init(struct clock_event_device *clk)
>>>>>>> {
>>>>>>> int cpu = smp_processor_id();
>>>>>>>
>>>>>>> + if (!gt_always_on)
>>>>>>> + clk->features |= CLOCK_EVT_FEAT_C3STOP;
>>>>>>> +
>>>>>>> clk->name = "arm_global_timer";
>>>>>>> clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT |
>>>>>>> CLOCK_EVT_FEAT_PERCPU;
>>>>>>> @@ -195,12 +200,25 @@ static cycle_t gt_clocksource_read(struct clocksource *cs)
>>>>>>> return gt_counter_read();
>>>>>>> }
>>>>>>>
>>>>>>> +static void gt_suspend(struct clocksource *cs)
>>>>>>> +{
>>>>>>> + gt_control = readl(gt_base + GT_CONTROL);
>>>>>>> +}
>>>>>>> +
>>>>>>> +static void gt_resume(struct clocksource *cs)
>>>>>>> +{
>>>>>>> + /* enables timer on all the cores */
>>>>>>> + writel(gt_control & GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
>>>>>>
>>>>>> do you really need to save context if all you restore is TIMER_ENABLE
>>>>>> bit ? seems like you could skip gt_suspend altogether. Is there really a
>>>>>> situation where this driver is running and GT isn't enabled ?
>>>>>
>>>>> Now It's not. It's always enabled. I did it because .suspend() is called for
>>>>> all registered clock sources regardless of their usage. So, potentially
>>>>> in the future, at the moment when .suspend() is called it might be disabled
>>>>> (for example, .enable/disable() callbacks can be added and, if ARM Global timer
>>>>> will not be registered as sched_clock, it will be possible to keep it disabled
>>>>> if not used now).
>>>>>
>>>>> But It's not essentially now - I can update it and drop save restore.
>>>>> Pls, confirm.
>>>>
>>>> I think it's best to skip suspend completely. You're not restoring
>>>> anything you saved during suspend, unless you meant | where you used &.
>>>>
>>>
>>> I didn't get it - I'm restoring one bit(0) only.
>>
>> that's the point, if you know you're restoring only that bit. Why save
>> anything at all ?
>>
>
> i think there are difference between "restoring" and "re-enabling".
> "restoring" - assume saving smth.. then restore saving value.
> I'm saving & restoring one bit here.
with your current suspend/resume, they are the same thing. You save
GT_CONTROL contents, timer goes off and looses context, you set ENABLE
bit. No difference what so ever.
--
balbi
On 11/13/2015 08:32 PM, Felipe Balbi wrote:
>
> Hi,
>
> Grygorii Strashko <[email protected]> writes:
>> On 11/13/2015 08:15 PM, Felipe Balbi wrote:
>>>
>>> Hi,
>>>
>>> Grygorii Strashko <[email protected]> writes:
>>>> On 11/13/2015 07:40 PM, Felipe Balbi wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Grygorii Strashko <[email protected]> writes:
>>>>>> On 11/13/2015 06:43 PM, Felipe Balbi wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Grygorii Strashko <[email protected]> writes:
>>>>>>>> Now the System stall is observed on TI AM437x based board
>>>>>>>> (am437x-gp-evm) during resuming from System suspend when ARM Global
>>>>>>>> timer is selected as clocksource device - SysRq are working, but
>>>>>>>> nothing else. The reason of stall is that ARM Global timer loses its
>>>>>>>> contexts.
>>>>>>>>
>>>>>>>> The reason of stall is that ARM Global timer loses its contexts during
>>>>>>>> System suspend:
>>>>>>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked)
>>>>>>>> GT_COUNTERx = 0
>>>>>>>>
>>>>>>>> Hence, update ARM Global timer driver to reflect above behaviour
>>>>>>>> - save GT_CONTROL.TIMER_ENABLE during suspend and restore on resume;
>>>>>>>> - ensure clocksource and clockevent devices have coresponding flags
>>>>>>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set
>>>>>>>> depending on presence of "always-on" DT property.
>>>>>>>>
>>>>>>>> CC: Arnd Bergmann <[email protected]>
>>>>>>>> Cc: John Stultz <[email protected]>
>>>>>>>> Cc: Felipe Balbi <[email protected]>
>>>>>>>> Cc: Tony Lindgren <[email protected]>
>>>>>>>> Signed-off-by: Grygorii Strashko <[email protected]>
>>>>>>>> ---
>>>>>>>> drivers/clocksource/arm_global_timer.c | 23 +++++++++++++++++++++++
>>>>>>>> 1 file changed, 23 insertions(+)
>>>>>>>>
>>>>>>>> diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c
>>>>>>>> index a2cb6fa..1bbaf64 100644
>>>>>>>> --- a/drivers/clocksource/arm_global_timer.c
>>>>>>>> +++ b/drivers/clocksource/arm_global_timer.c
>>>>>>>> @@ -51,6 +51,8 @@ static void __iomem *gt_base;
>>>>>>>> static unsigned long gt_clk_rate;
>>>>>>>> static int gt_ppi;
>>>>>>>> static struct clock_event_device __percpu *gt_evt;
>>>>>>>> +static bool gt_always_on;
>>>>>>>> +static u32 gt_control;
>>>>>>>>
>>>>>>>> /*
>>>>>>>> * To get the value from the Global Timer Counter register proceed as follows:
>>>>>>>> @@ -168,6 +170,9 @@ static int gt_clockevents_init(struct clock_event_device *clk)
>>>>>>>> {
>>>>>>>> int cpu = smp_processor_id();
>>>>>>>>
>>>>>>>> + if (!gt_always_on)
>>>>>>>> + clk->features |= CLOCK_EVT_FEAT_C3STOP;
>>>>>>>> +
>>>>>>>> clk->name = "arm_global_timer";
>>>>>>>> clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT |
>>>>>>>> CLOCK_EVT_FEAT_PERCPU;
>>>>>>>> @@ -195,12 +200,25 @@ static cycle_t gt_clocksource_read(struct clocksource *cs)
>>>>>>>> return gt_counter_read();
>>>>>>>> }
>>>>>>>>
>>>>>>>> +static void gt_suspend(struct clocksource *cs)
>>>>>>>> +{
>>>>>>>> + gt_control = readl(gt_base + GT_CONTROL);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> +static void gt_resume(struct clocksource *cs)
>>>>>>>> +{
>>>>>>>> + /* enables timer on all the cores */
>>>>>>>> + writel(gt_control & GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
>>>>>>>
>>>>>>> do you really need to save context if all you restore is TIMER_ENABLE
>>>>>>> bit ? seems like you could skip gt_suspend altogether. Is there really a
>>>>>>> situation where this driver is running and GT isn't enabled ?
>>>>>>
>>>>>> Now It's not. It's always enabled. I did it because .suspend() is called for
>>>>>> all registered clock sources regardless of their usage. So, potentially
>>>>>> in the future, at the moment when .suspend() is called it might be disabled
>>>>>> (for example, .enable/disable() callbacks can be added and, if ARM Global timer
>>>>>> will not be registered as sched_clock, it will be possible to keep it disabled
>>>>>> if not used now).
>>>>>>
>>>>>> But It's not essentially now - I can update it and drop save restore.
>>>>>> Pls, confirm.
>>>>>
>>>>> I think it's best to skip suspend completely. You're not restoring
>>>>> anything you saved during suspend, unless you meant | where you used &.
>>>>>
>>>>
>>>> I didn't get it - I'm restoring one bit(0) only.
>>>
>>> that's the point, if you know you're restoring only that bit. Why save
>>> anything at all ?
>>>
>>
>> i think there are difference between "restoring" and "re-enabling".
>> "restoring" - assume saving smth.. then restore saving value.
>> I'm saving & restoring one bit here.
>
> with your current suspend/resume, they are the same thing. You save
> GT_CONTROL contents, timer goes off and looses context, you set ENABLE
> bit. No difference what so ever.
>
I'm writing:
gt_control & GT_CONTROL_TIMER_ENABLE, where gt_control is old value of GT_CONTROL
before entering suspend, so on resume:
- if gt_control == 0x0000 0001 ==> gt_control & GT_CONTROL_TIMER_ENABLE(0x1) == 0x1
- if gt_control == 0x0000 0000 ==> gt_control & GT_CONTROL_TIMER_ENABLE(0x1) == 0x0
What am I missed?
--
regards,
-grygorii
Hi Felipe,
On 11/13/2015 08:59 PM, Grygorii Strashko wrote:
> On 11/13/2015 08:32 PM, Felipe Balbi wrote:
>> Grygorii Strashko <[email protected]> writes:
>>> On 11/13/2015 08:15 PM, Felipe Balbi wrote:
>>>> Grygorii Strashko <[email protected]> writes:
>>>>> On 11/13/2015 07:40 PM, Felipe Balbi wrote:
>>>>>> Grygorii Strashko <[email protected]> writes:
>>>>>>> On 11/13/2015 06:43 PM, Felipe Balbi wrote:
>>>>>>>> GrygoCONFIG_CLKSRC_ARM_GLOBAL_TIMER_SCHED_CLOCKrii Strashko <[email protected]> writes:
>>>>>>>>> Now the System stall is observed on TI AM437x based board
>>>>>>>>> (am437x-gp-evm) during resuming from System suspend when ARM Global
>>>>>>>>> timer is selected as clocksource device - SysRq are working, but
>>>>>>>>> nothing else. The reason of stall is that ARM Global timer loses its
>>>>>>>>> contexts.
>>>>>>>>>
>>>>>>>>> The reason of stall is that ARM Global timer loses its contexts during
>>>>>>>>> System suspend:
>>>>>>>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked)
>>>>>>>>> GT_COUNTERx = 0
>>>>>>>>>
>>>>>>>>> Hence, update ARM Global timer driver to reflect above behaviour
>>>>>>>>> - save GT_CONTROL.TIMER_ENABLE during suspend and restore on resume;
>>>>>>>>> - ensure clocksource and clockevent devices have coresponding flags
>>>>>>>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set
>>>>>>>>> depending on presence of "always-on" DT property.
>>>>>>>>>
>>>>>>>>> CC: Arnd Bergmann <[email protected]>
>>>>>>>>> Cc: John Stultz <[email protected]>
>>>>>>>>> Cc: Felipe Balbi <[email protected]>
>>>>>>>>> Cc: Tony Lindgren <[email protected]>
>>>>>>>>> Signed-off-by: Grygorii Strashko <[email protected]>
>>>>>>>>> ---
>>>>>>>>> drivers/clocksource/arm_global_timer.c | 23 +++++++++++++++++++++++
>>>>>>>>> 1 file changed, 23 insertions(+)
>>>>>>>>>
>>>>>>>>> diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c
>>>>>>>>> index a2cb6fa..1bbaf64 100644
>>>>>>>>> --- a/drivers/clocksource/arm_global_timer.c
>>>>>>>>> +++ b/drivers/clocksource/arm_global_timer.c
>>>>>>>>> @@ -51,6 +51,8 @@ static void __iomem *gt_base;
>>>>>>>>> static unsigned long gt_clk_rate;
>>>>>>>>> static int gt_ppi;
>>>>>>>>> static struct clock_event_device __percpu *gt_evt;
>>>>>>>>> +static bool gt_always_on;
>>>>>>>>> +static u32 gt_control;
>>>>>>>>>
>>>>>>>>> /*
>>>>>>>>> * To get the value from the Global Timer Counter register proceed as follows:
>>>>>>>>> @@ -168,6 +170,9 @@ static int gt_clockevents_init(struct clock_event_device *clk)
>>>>>>>>> {
>>>>>>>>> int cpu = smp_processor_id();
>>>>>>>>>
>>>>>>>>> + if (!gt_always_on)
>>>>>>>>> + clk->features |= CLOCK_EVT_FEAT_C3STOP;
>>>>>>>>> +
>>>>>>>>> clk->name = "arm_global_timer";
>>>>>>>>> clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT |
>>>>>>>>> CLOCK_EVT_FEAT_PERCPU;
>>>>>>>>> @@ -195,12 +200,25 @@ static cycle_t gt_clocksource_read(struct clocksource *cs)
>>>>>>>>> return gt_counter_read();
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> +static void gt_suspend(struct clocksource *cs)
>>>>>>>>> +{
>>>>>>>>> + gt_control = readl(gt_base + GT_CONTROL);
>>>>>>>>> +}
>>>>>>>>> +
>>>>>>>>> +static void gt_resume(struct clocksource *cs)
>>>>>>>>> +{
>>>>>>>>> + /* enables timer on all the cores */
>>>>>>>>> + writel(gt_control & GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL);
>>>>>>>>
>>>>>>>> do you really need to save context if all you restore is TIMER_ENABLE
>>>>>>>> bit ? seems like you could skip gt_suspend altogether. Is there really a
>>>>>>>> situation where this driver is running and GT isn't enabled ?
>>>>>>>
>>>>>>> Now It's not. It's always enabled. I did it because .suspend() is called for
>>>>>>> all registered clock sources regardless of their usage. So, potentially
>>>>>>> in the future, at the moment when .suspend() is called it might be disabled
>>>>>>> (for example, .enable/disable() callbacks can be added and, if ARM Global timer
>>>>>>> will not be registered as sched_clock, it will be possible to keep it disabled
>>>>>>> if not used now).
>>>>>>>
>>>>>>> But It's not essentially now - I can update it and drop save restore.
>>>>>>> Pls, confirm.
>>>>>>
>>>>>> I think it's best to skip suspend completely. You're not restoring
>>>>>> anything you saved during suspend, unless you meant | where you used &.
>>>>>>
>>>>>
>>>>> I didn't get it - I'm restoring one bit(0) only.
>>>>
>>>> that's the point, if you know you're restoring only that bit. Why save
>>>> anything at all ?
>>>>
>>>
>>> i think there are difference between "restoring" and "re-enabling".
>>> "restoring" - assume saving smth.. then restore saving value.
>>> I'm saving & restoring one bit here.
>>
>> with your current suspend/resume, they are the same thing. You save
>> GT_CONTROL contents, timer goes off and looses context, you set ENABLE
>> bit. No difference what so ever.
>>
>
> I'm writing:
> gt_control & GT_CONTROL_TIMER_ENABLE, where gt_control is old value of GT_CONTROL
> before entering suspend, so on resume:
> - if gt_control == 0x0000 0001 ==> gt_control & GT_CONTROL_TIMER_ENABLE(0x1) == 0x1
> - if gt_control == 0x0000 0000 ==> gt_control & GT_CONTROL_TIMER_ENABLE(0x1) == 0x0
>
> What am I missed?
>
Any way, I'll wait for other comments a bit, replace save/restore with
just pure re-enabling and re-send.
Thanks.
--
regards,
-grygorii
On Fri, 13 Nov 2015, Grygorii Strashko wrote:
<Delete 100 lines of useless information>
> Any way, I'll wait for other comments a bit, replace save/restore with
> just pure re-enabling and re-send.
Please folks, trim your emails! I'm really tired of going through
hundreds of quoted (quoting level here was 10!!!) useless lines to
find a single line of useful information.
Thanks,
tglx