2018-07-06 13:20:42

by Mukesh Ojha

[permalink] [raw]
Subject: [PATCH v3] time: Fix incorrect sleeptime injection when suspend fails

Currently, there exists a corner case assuming when there is
only one clocksource e.g RTC, and system failed to go to
suspend mode. While resume rtc_resume() injects the sleeptime
as timekeeping_rtc_skipresume() returned 'false' (default value
of sleeptime_injected) due to which we can see mismatch in
timestamps.

This issue can also come in a system where more than one
clocksource are present and very first suspend fails.

Fix this by handling `sleeptime_injected` flag properly.

Success case:
------------
{sleeptime_injected=false}
rtc_suspend() => timekeeping_suspend() => timekeeping_resume() =>

(sleeptime injected)
rtc_resume()

Failure case:
------------
{failure in sleep path} {sleeptime_injected=false}
rtc_suspend() => rtc_resume()

sleeptime injected again which was not required as the suspend failed)

Originally-by: Thomas Gleixner <[email protected]>
Signed-off-by: Mukesh Ojha <[email protected]>
---
Changes in v3:
* Updated commit subject and description.
* Updated the patch as per the fix given by Thomas Gleixner.

Changes in v2:
* Updated the commit text.
* Removed extra variable and used the earlier static
variable 'sleeptime_injected'.

kernel/time/timekeeping.c | 21 ++++++++++++++++++---
1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 4786df9..32ae9ae 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -1510,8 +1510,20 @@ void __weak read_boot_clock64(struct timespec64 *ts)
ts->tv_nsec = 0;
}

-/* Flag for if timekeeping_resume() has injected sleeptime */
-static bool sleeptime_injected;
+/*
+ * Flag reflecting whether timekeeping_resume() has injected sleeptime.
+ *
+ * The flag starts of true and is only cleared when a suspend reaches
+ * timekeeping_suspend(), timekeeping_resume() sets it when the timekeeper
+ * clocksource is not stopping across suspend and has been used to update
+ * sleep time. If the timekeeper clocksource has stopped then the flag
+ * stays false and is used by the RTC resume code to decide whether sleep
+ * time must be injected and if so the flag gets set then.
+ *
+ * If a suspend fails before reaching timekeeping_resume() then the flag
+ * stays true and prevents erroneous sleeptime injection.
+ */
+static bool sleeptime_injected = true;

/* Flag for if there is a persistent clock on this platform */
static bool persistent_clock_exists;
@@ -1646,6 +1658,8 @@ void timekeeping_inject_sleeptime64(struct timespec64 *delta)
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&tk_core.seq);

+ sleeptime_injected = true;
+
timekeeping_forward_now(tk);

__timekeeping_inject_sleeptime(tk, delta);
@@ -1671,7 +1685,6 @@ void timekeeping_resume(void)
struct timespec64 ts_new, ts_delta;
u64 cycle_now;

- sleeptime_injected = false;
read_persistent_clock64(&ts_new);

clockevents_resume();
@@ -1743,6 +1756,8 @@ int timekeeping_suspend(void)
if (timekeeping_suspend_time.tv_sec || timekeeping_suspend_time.tv_nsec)
persistent_clock_exists = true;

+ sleeptime_injected = false;
+
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
--
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center,
Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project



2018-07-10 20:16:05

by John Stultz

[permalink] [raw]
Subject: Re: [PATCH v3] time: Fix incorrect sleeptime injection when suspend fails

On Fri, Jul 6, 2018 at 6:17 AM, Mukesh Ojha <[email protected]> wrote:
> Currently, there exists a corner case assuming when there is
> only one clocksource e.g RTC, and system failed to go to
> suspend mode. While resume rtc_resume() injects the sleeptime
> as timekeeping_rtc_skipresume() returned 'false' (default value
> of sleeptime_injected) due to which we can see mismatch in
> timestamps.
>
> This issue can also come in a system where more than one
> clocksource are present and very first suspend fails.
>
> Fix this by handling `sleeptime_injected` flag properly.
>
> Success case:
> ------------
> {sleeptime_injected=false}
> rtc_suspend() => timekeeping_suspend() => timekeeping_resume() =>
>
> (sleeptime injected)
> rtc_resume()
>
> Failure case:
> ------------
> {failure in sleep path} {sleeptime_injected=false}
> rtc_suspend() => rtc_resume()
>
> sleeptime injected again which was not required as the suspend failed)
>
> Originally-by: Thomas Gleixner <[email protected]>
> Signed-off-by: Mukesh Ojha <[email protected]>
> ---
> Changes in v3:
> * Updated commit subject and description.
> * Updated the patch as per the fix given by Thomas Gleixner.
>
> Changes in v2:
> * Updated the commit text.
> * Removed extra variable and used the earlier static
> variable 'sleeptime_injected'.
>
> kernel/time/timekeeping.c | 21 ++++++++++++++++++---
> 1 file changed, 18 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
> index 4786df9..32ae9ae 100644
> --- a/kernel/time/timekeeping.c
> +++ b/kernel/time/timekeeping.c
> @@ -1510,8 +1510,20 @@ void __weak read_boot_clock64(struct timespec64 *ts)
> ts->tv_nsec = 0;
> }
>
> -/* Flag for if timekeeping_resume() has injected sleeptime */
> -static bool sleeptime_injected;
> +/*
> + * Flag reflecting whether timekeeping_resume() has injected sleeptime.
> + *
> + * The flag starts of true and is only cleared when a suspend reaches
> + * timekeeping_suspend(), timekeeping_resume() sets it when the timekeeper
> + * clocksource is not stopping across suspend and has been used to update
> + * sleep time. If the timekeeper clocksource has stopped then the flag
> + * stays false and is used by the RTC resume code to decide whether sleep
> + * time must be injected and if so the flag gets set then.
> + *
> + * If a suspend fails before reaching timekeeping_resume() then the flag
> + * stays true and prevents erroneous sleeptime injection.
> + */
> +static bool sleeptime_injected = true;

I worry this upside-down logic is too subtle to be easily reasoned
about, and will just lead to future mistakes.

Can we instead call this "suspend_timing_needed" and only set it to
true when we don't inject any sleep time on resume?

I think that will help make things a bit more clear, no?

thanks
-john

2018-07-13 07:14:58

by Mukesh Ojha

[permalink] [raw]
Subject: Re: [PATCH v3] time: Fix incorrect sleeptime injection when suspend fails

Hi John,

Thanks for your response
Please find my comments inline.

On 7/11/2018 1:43 AM, John Stultz wrote:
> On Fri, Jul 6, 2018 at 6:17 AM, Mukesh Ojha <[email protected]> wrote:
>> Currently, there exists a corner case assuming when there is
>> only one clocksource e.g RTC, and system failed to go to
>> suspend mode. While resume rtc_resume() injects the sleeptime
>> as timekeeping_rtc_skipresume() returned 'false' (default value
>> of sleeptime_injected) due to which we can see mismatch in
>> timestamps.
>>
>> This issue can also come in a system where more than one
>> clocksource are present and very first suspend fails.
>>
>> Fix this by handling `sleeptime_injected` flag properly.
>>
>> Success case:
>> ------------
>> {sleeptime_injected=false}
>> rtc_suspend() => timekeeping_suspend() => timekeeping_resume() =>
>>
>> (sleeptime injected)
>> rtc_resume()
>>
>> Failure case:
>> ------------
>> {failure in sleep path} {sleeptime_injected=false}
>> rtc_suspend() => rtc_resume()
>>
>> sleeptime injected again which was not required as the suspend failed)
>>
>> Originally-by: Thomas Gleixner <[email protected]>
>> Signed-off-by: Mukesh Ojha <[email protected]>
>> ---
>> Changes in v3:
>> * Updated commit subject and description.
>> * Updated the patch as per the fix given by Thomas Gleixner.
>>
>> Changes in v2:
>> * Updated the commit text.
>> * Removed extra variable and used the earlier static
>> variable 'sleeptime_injected'.
>>
>> kernel/time/timekeeping.c | 21 ++++++++++++++++++---
>> 1 file changed, 18 insertions(+), 3 deletions(-)
>>
>> diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
>> index 4786df9..32ae9ae 100644
>> --- a/kernel/time/timekeeping.c
>> +++ b/kernel/time/timekeeping.c
>> @@ -1510,8 +1510,20 @@ void __weak read_boot_clock64(struct timespec64 *ts)
>> ts->tv_nsec = 0;
>> }
>>
>> -/* Flag for if timekeeping_resume() has injected sleeptime */
>> -static bool sleeptime_injected;
>> +/*
>> + * Flag reflecting whether timekeeping_resume() has injected sleeptime.
>> + *
>> + * The flag starts of true and is only cleared when a suspend reaches
>> + * timekeeping_suspend(), timekeeping_resume() sets it when the timekeeper
>> + * clocksource is not stopping across suspend and has been used to update
>> + * sleep time. If the timekeeper clocksource has stopped then the flag
>> + * stays false and is used by the RTC resume code to decide whether sleep
>> + * time must be injected and if so the flag gets set then.
>> + *
>> + * If a suspend fails before reaching timekeeping_resume() then the flag
>> + * stays true and prevents erroneous sleeptime injection.
>> + */
>> +static bool sleeptime_injected = true;
> I worry this upside-down logic is too subtle to be easily reasoned
> about, and will just lead to future mistakes.
>
> Can we instead call this "suspend_timing_needed" and only set it to
> true when we don't inject any sleep time on resume?

I did not get your point "only set it to true when we don't inject any
sleep time on resume? "
How do we know  this ?
This question itself depends on the "sleeptime_injected" if it is true
means no need to inject else need to inject.

Also, we need to make this variable back and forth true, false; suspends
path ensures it to make it false.

Just to add here there are already two path where `sleeptime_injected`
set to true one from
NON-stop clocksource and other from persistant clock and the RTC one was
missing, so we are adding
with this patch.

Cheers,
-Mukesh



2018-07-13 17:21:30

by John Stultz

[permalink] [raw]
Subject: Re: [PATCH v3] time: Fix incorrect sleeptime injection when suspend fails

On Fri, Jul 13, 2018 at 12:13 AM, Mukesh Ojha <[email protected]> wrote:
> Hi John,
>
> Thanks for your response
> Please find my comments inline.
>
>
> On 7/11/2018 1:43 AM, John Stultz wrote:
>>
>> On Fri, Jul 6, 2018 at 6:17 AM, Mukesh Ojha <[email protected]> wrote:
>>>
>>> Currently, there exists a corner case assuming when there is
>>> only one clocksource e.g RTC, and system failed to go to
>>> suspend mode. While resume rtc_resume() injects the sleeptime
>>> as timekeeping_rtc_skipresume() returned 'false' (default value
>>> of sleeptime_injected) due to which we can see mismatch in
>>> timestamps.
>>>
>>> This issue can also come in a system where more than one
>>> clocksource are present and very first suspend fails.
>>>
>>> Fix this by handling `sleeptime_injected` flag properly.
>>>
>>> Success case:
>>> ------------
>>> {sleeptime_injected=false}
>>> rtc_suspend() => timekeeping_suspend() => timekeeping_resume() =>
>>>
>>> (sleeptime injected)
>>> rtc_resume()
>>>
>>> Failure case:
>>> ------------
>>> {failure in sleep path} {sleeptime_injected=false}
>>> rtc_suspend() => rtc_resume()
>>>
>>> sleeptime injected again which was not required as the suspend failed)
>>>
>>> Originally-by: Thomas Gleixner <[email protected]>
>>> Signed-off-by: Mukesh Ojha <[email protected]>
>>> ---
>>> Changes in v3:
>>> * Updated commit subject and description.
>>> * Updated the patch as per the fix given by Thomas Gleixner.
>>>
>>> Changes in v2:
>>> * Updated the commit text.
>>> * Removed extra variable and used the earlier static
>>> variable 'sleeptime_injected'.
>>>
>>> kernel/time/timekeeping.c | 21 ++++++++++++++++++---
>>> 1 file changed, 18 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
>>> index 4786df9..32ae9ae 100644
>>> --- a/kernel/time/timekeeping.c
>>> +++ b/kernel/time/timekeeping.c
>>> @@ -1510,8 +1510,20 @@ void __weak read_boot_clock64(struct timespec64
>>> *ts)
>>> ts->tv_nsec = 0;
>>> }
>>>
>>> -/* Flag for if timekeeping_resume() has injected sleeptime */
>>> -static bool sleeptime_injected;
>>> +/*
>>> + * Flag reflecting whether timekeeping_resume() has injected sleeptime.
>>> + *
>>> + * The flag starts of true and is only cleared when a suspend reaches
>>> + * timekeeping_suspend(), timekeeping_resume() sets it when the
>>> timekeeper
>>> + * clocksource is not stopping across suspend and has been used to
>>> update
>>> + * sleep time. If the timekeeper clocksource has stopped then the flag
>>> + * stays false and is used by the RTC resume code to decide whether
>>> sleep
>>> + * time must be injected and if so the flag gets set then.
>>> + *
>>> + * If a suspend fails before reaching timekeeping_resume() then the flag
>>> + * stays true and prevents erroneous sleeptime injection.
>>> + */
>>> +static bool sleeptime_injected = true;
>>
>> I worry this upside-down logic is too subtle to be easily reasoned
>> about, and will just lead to future mistakes.
>>
>> Can we instead call this "suspend_timing_needed" and only set it to
>> true when we don't inject any sleep time on resume?
>
>
> I did not get your point "only set it to true when we don't inject any sleep
> time on resume? "
> How do we know this ?
> This question itself depends on the "sleeptime_injected" if it is true means
> no need to inject else need to inject.
>
> Also, we need to make this variable back and forth true, false; suspends
> path ensures it to make it false.

So yea, I'm not saying logically the code is really any different,
this is more of a naming nit. So instead of having a variable that is
always on that we occasionally turn off, lets invert the naming and
have it be a flag that we occasionally turn on.

Just the name sleeptime_injected is read a statement, which if we say
is defaults to true, becomes confusing to think about when the
timekeeping_suspend/resume code hasn't yet run (which is the case
where your error cropped up) - and no sleeptime has actually been
injected.

So instead if we call it suspend_timing_needed and only set it on in
timekeeping_resume() after the timekeeping code has not injected any
sleep-time, then I think the code will make more sense to read. (And
yes, we still need to set suspend_timing_needed false on
timekeeping_suspend and in the inject_sleeptime call path - the logic
doesn't change, just the naming and boolean state).

thanks
-john

2018-07-16 16:18:54

by Mukesh Ojha

[permalink] [raw]
Subject: Re: [PATCH v3] time: Fix incorrect sleeptime injection when suspend fails


On 7/13/2018 10:50 PM, John Stultz wrote:
> On Fri, Jul 13, 2018 at 12:13 AM, Mukesh Ojha <[email protected]> wrote:
>> Hi John,
>>
>> Thanks for your response
>> Please find my comments inline.
>>
>>
>> On 7/11/2018 1:43 AM, John Stultz wrote:
>>> On Fri, Jul 6, 2018 at 6:17 AM, Mukesh Ojha <[email protected]> wrote:
>>>> Currently, there exists a corner case assuming when there is
>>>> only one clocksource e.g RTC, and system failed to go to
>>>> suspend mode. While resume rtc_resume() injects the sleeptime
>>>> as timekeeping_rtc_skipresume() returned 'false' (default value
>>>> of sleeptime_injected) due to which we can see mismatch in
>>>> timestamps.
>>>>
>>>> This issue can also come in a system where more than one
>>>> clocksource are present and very first suspend fails.
>>>>
>>>> Fix this by handling `sleeptime_injected` flag properly.
>>>>
>>>> Success case:
>>>> ------------
>>>> {sleeptime_injected=false}
>>>> rtc_suspend() => timekeeping_suspend() => timekeeping_resume() =>
>>>>
>>>> (sleeptime injected)
>>>> rtc_resume()
>>>>
>>>> Failure case:
>>>> ------------
>>>> {failure in sleep path} {sleeptime_injected=false}
>>>> rtc_suspend() => rtc_resume()
>>>>
>>>> sleeptime injected again which was not required as the suspend failed)
>>>>
>>>> Originally-by: Thomas Gleixner <[email protected]>
>>>> Signed-off-by: Mukesh Ojha <[email protected]>
>>>> ---
>>>> Changes in v3:
>>>> * Updated commit subject and description.
>>>> * Updated the patch as per the fix given by Thomas Gleixner.
>>>>
>>>> Changes in v2:
>>>> * Updated the commit text.
>>>> * Removed extra variable and used the earlier static
>>>> variable 'sleeptime_injected'.
>>>>
>>>> kernel/time/timekeeping.c | 21 ++++++++++++++++++---
>>>> 1 file changed, 18 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
>>>> index 4786df9..32ae9ae 100644
>>>> --- a/kernel/time/timekeeping.c
>>>> +++ b/kernel/time/timekeeping.c
>>>> @@ -1510,8 +1510,20 @@ void __weak read_boot_clock64(struct timespec64
>>>> *ts)
>>>> ts->tv_nsec = 0;
>>>> }
>>>>
>>>> -/* Flag for if timekeeping_resume() has injected sleeptime */
>>>> -static bool sleeptime_injected;
>>>> +/*
>>>> + * Flag reflecting whether timekeeping_resume() has injected sleeptime.
>>>> + *
>>>> + * The flag starts of true and is only cleared when a suspend reaches
>>>> + * timekeeping_suspend(), timekeeping_resume() sets it when the
>>>> timekeeper
>>>> + * clocksource is not stopping across suspend and has been used to
>>>> update
>>>> + * sleep time. If the timekeeper clocksource has stopped then the flag
>>>> + * stays false and is used by the RTC resume code to decide whether
>>>> sleep
>>>> + * time must be injected and if so the flag gets set then.
>>>> + *
>>>> + * If a suspend fails before reaching timekeeping_resume() then the flag
>>>> + * stays true and prevents erroneous sleeptime injection.
>>>> + */
>>>> +static bool sleeptime_injected = true;
>>> I worry this upside-down logic is too subtle to be easily reasoned
>>> about, and will just lead to future mistakes.
>>>
>>> Can we instead call this "suspend_timing_needed" and only set it to
>>> true when we don't inject any sleep time on resume?
>>
>> I did not get your point "only set it to true when we don't inject any sleep
>> time on resume? "
>> How do we know this ?
>> This question itself depends on the "sleeptime_injected" if it is true means
>> no need to inject else need to inject.
>>
>> Also, we need to make this variable back and forth true, false; suspends
>> path ensures it to make it false.
> So yea, I'm not saying logically the code is really any different,
> this is more of a naming nit. So instead of having a variable that is
> always on that we occasionally turn off, lets invert the naming and
> have it be a flag that we occasionally turn on.

I understand your concern about the name of the variable will be misleading.
But the changing Boolean state would not solve the actual issue.

If i understand you correctly you meant below code

diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 32ae9ae..becc5bd 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -1523,7 +1523,7 @@ void __weak read_boot_clock64(struct timespec64 *ts)
  * If a suspend fails before reaching timekeeping_resume() then the flag
  * stays true and prevents erroneous sleeptime injection.
  */
-static bool sleeptime_injected = true;
+static bool suspend_timing_needed;

 /* Flag for if there is a persistent clock on this platform */
 static bool persistent_clock_exists;
@@ -1658,7 +1658,7 @@ void timekeeping_inject_sleeptime64(struct
timespec64 *delta)
        raw_spin_lock_irqsave(&timekeeper_lock, flags);
        write_seqcount_begin(&tk_core.seq);

-       sleeptime_injected = true;
+       suspend_timing_needed = false;

        timekeeping_forward_now(tk);

@@ -1714,10 +1714,10 @@ void timekeeping_resume(void)
                                              tk->tkr_mono.mask);
                nsec = mul_u64_u32_shr(cyc_delta, clock->mult,
clock->shift);
                ts_delta = ns_to_timespec64(nsec);
-               sleeptime_injected = true;
+               suspend_timing_needed = true;
        } else if (timespec64_compare(&ts_new,
&timekeeping_suspend_time) > 0) {
                ts_delta = timespec64_sub(ts_new,
timekeeping_suspend_time);
-               sleeptime_injected = true;
+               suspend_timing_needed = true;
        }

        if (sleeptime_injected)
@@ -1756,7 +1756,7 @@ int timekeeping_suspend(void)
        if (timekeeping_suspend_time.tv_sec ||
timekeeping_suspend_time.tv_nsec)
                persistent_clock_exists = true;

-       sleeptime_injected = false;
+       suspend_timing_needed = false;

        raw_spin_lock_irqsave(&timekeeper_lock, flags);


This has a problem..


>
> Just the name sleeptime_injected is read a statement, which if we say
> is defaults to true, becomes confusing to think about when the
> timekeeping_suspend/resume code hasn't yet run (which is the case
> where your error cropped up) - and no sleeptime has actually been
> injected.

Yes, when very first suspend fails and timekeeping_suspend/resume did
not run ; That is the exact issue.
So, exact solution is no need to inject any sleeptime here.

 If we set the default value to false then we will see
timekeeping_resume will inject sleeptime by below code which was not
intended.

static int rtc_resume(struct device *dev)
{
        struct rtc_device       *rtc = to_rtc_device(dev);
        struct rtc_time         tm;
        struct timespec64       new_system, new_rtc;
        struct timespec64       sleep_time;
        int err;

        if (timekeeping_rtc_skipresume())  // it will return the value
false as sleep failed and timekeeping_resume() did not get called.
                return 0;

  <sleeptime injection happens here>
....
..


>
> So instead if we call it suspend_timing_needed and only set it on in
> timekeeping_resume() after the timekeeping code has not injected any
> sleep-time, then I think the code will make more sense to read. (And
> yes, we still need to set suspend_timing_needed false on
> timekeeping_suspend and in the inject_sleeptime call path - the logic
> doesn't change, just the naming and boolean state).

Thanks for your time and patience.

-Mukesh

> thanks
> -john


2018-07-16 16:32:29

by John Stultz

[permalink] [raw]
Subject: Re: [PATCH v3] time: Fix incorrect sleeptime injection when suspend fails

On Mon, Jul 16, 2018 at 9:17 AM, Mukesh Ojha <[email protected]> wrote:
> On 7/13/2018 10:50 PM, John Stultz wrote:
>> On Fri, Jul 13, 2018 at 12:13 AM, Mukesh Ojha <[email protected]>
>>> On 7/11/2018 1:43 AM, John Stultz wrote:
>>>> I worry this upside-down logic is too subtle to be easily reasoned
>>>> about, and will just lead to future mistakes.
>>>>
>>>> Can we instead call this "suspend_timing_needed" and only set it to
>>>> true when we don't inject any sleep time on resume?
>>>
>>>
>>> I did not get your point "only set it to true when we don't inject any
>>> sleep
>>> time on resume? "
>>> How do we know this ?
>>> This question itself depends on the "sleeptime_injected" if it is true
>>> means
>>> no need to inject else need to inject.
>>>
>>> Also, we need to make this variable back and forth true, false; suspends
>>> path ensures it to make it false.
>>
>> So yea, I'm not saying logically the code is really any different,
>> this is more of a naming nit. So instead of having a variable that is
>> always on that we occasionally turn off, lets invert the naming and
>> have it be a flag that we occasionally turn on.
>
>
> I understand your concern about the name of the variable will be misleading.
> But the changing Boolean state would not solve the actual issue.
>
> If i understand you correctly you meant below code
>
> diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
> index 32ae9ae..becc5bd 100644
> --- a/kernel/time/timekeeping.c
> +++ b/kernel/time/timekeeping.c
> @@ -1523,7 +1523,7 @@ void __weak read_boot_clock64(struct timespec64 *ts)
> * If a suspend fails before reaching timekeeping_resume() then the flag
> * stays true and prevents erroneous sleeptime injection.
> */
> -static bool sleeptime_injected = true;
> +static bool suspend_timing_needed;
>
> /* Flag for if there is a persistent clock on this platform */
> static bool persistent_clock_exists;
> @@ -1658,7 +1658,7 @@ void timekeeping_inject_sleeptime64(struct timespec64
> *delta)
> raw_spin_lock_irqsave(&timekeeper_lock, flags);
> write_seqcount_begin(&tk_core.seq);
>
> - sleeptime_injected = true;
> + suspend_timing_needed = false;
>
> timekeeping_forward_now(tk);
>
> @@ -1714,10 +1714,10 @@ void timekeeping_resume(void)
> tk->tkr_mono.mask);
> nsec = mul_u64_u32_shr(cyc_delta, clock->mult,
> clock->shift);
> ts_delta = ns_to_timespec64(nsec);
> - sleeptime_injected = true;
> + suspend_timing_needed = true;
> } else if (timespec64_compare(&ts_new, &timekeeping_suspend_time) >
> 0) {
> ts_delta = timespec64_sub(ts_new, timekeeping_suspend_time);
> - sleeptime_injected = true;
> + suspend_timing_needed = true;
> }

No no... This part is wrong. We only set suspend_timing_needed if we
*didn't* calculate the suspend time in timekeeping_resume.

You have to invert all the boolean logic for it to be equivalent.

> if (sleeptime_injected)
> @@ -1756,7 +1756,7 @@ int timekeeping_suspend(void)
> if (timekeeping_suspend_time.tv_sec ||
> timekeeping_suspend_time.tv_nsec)
> persistent_clock_exists = true;
>
> - sleeptime_injected = false;
> + suspend_timing_needed = false;
>
> raw_spin_lock_irqsave(&timekeeper_lock, flags);
>
>
> This has a problem..
>
>
>>
>> Just the name sleeptime_injected is read a statement, which if we say
>> is defaults to true, becomes confusing to think about when the
>> timekeeping_suspend/resume code hasn't yet run (which is the case
>> where your error cropped up) - and no sleeptime has actually been
>> injected.
>
>
> Yes, when very first suspend fails and timekeeping_suspend/resume did not
> run ; That is the exact issue.
> So, exact solution is no need to inject any sleeptime here.
>
> If we set the default value to false then we will see timekeeping_resume
> will inject sleeptime by below code which was not intended.
>
> static int rtc_resume(struct device *dev)
> {
> struct rtc_device *rtc = to_rtc_device(dev);
> struct rtc_time tm;
> struct timespec64 new_system, new_rtc;
> struct timespec64 sleep_time;
> int err;
>
> if (timekeeping_rtc_skipresume()) // it will return the value false
> as sleep failed and timekeeping_resume() did not get called.
> return 0;
>
> <sleeptime injection happens here>


So, I think with the logic bug above it will work out properly, but
let me know if I'm still missing something.

thanks
-john

2018-07-16 17:15:51

by John Stultz

[permalink] [raw]
Subject: Re: [PATCH v3] time: Fix incorrect sleeptime injection when suspend fails

On Mon, Jul 16, 2018 at 9:30 AM, John Stultz <[email protected]> wrote:
> On Mon, Jul 16, 2018 at 9:17 AM, Mukesh Ojha <[email protected]> wrote:
>> On 7/13/2018 10:50 PM, John Stultz wrote:
>>> On Fri, Jul 13, 2018 at 12:13 AM, Mukesh Ojha <[email protected]>
>>>> On 7/11/2018 1:43 AM, John Stultz wrote:
>>>>> I worry this upside-down logic is too subtle to be easily reasoned
>>>>> about, and will just lead to future mistakes.
>>>>>
>>>>> Can we instead call this "suspend_timing_needed" and only set it to
>>>>> true when we don't inject any sleep time on resume?
>>>>
>>>>
>>>> I did not get your point "only set it to true when we don't inject any
>>>> sleep
>>>> time on resume? "
>>>> How do we know this ?
>>>> This question itself depends on the "sleeptime_injected" if it is true
>>>> means
>>>> no need to inject else need to inject.
>>>>
>>>> Also, we need to make this variable back and forth true, false; suspends
>>>> path ensures it to make it false.
>>>
>>> So yea, I'm not saying logically the code is really any different,
>>> this is more of a naming nit. So instead of having a variable that is
>>> always on that we occasionally turn off, lets invert the naming and
>>> have it be a flag that we occasionally turn on.
>>
>>
>> I understand your concern about the name of the variable will be misleading.
>> But the changing Boolean state would not solve the actual issue.
>>
>> If i understand you correctly you meant below code
>>
>> diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
>> index 32ae9ae..becc5bd 100644
>> --- a/kernel/time/timekeeping.c
>> +++ b/kernel/time/timekeeping.c
>> @@ -1523,7 +1523,7 @@ void __weak read_boot_clock64(struct timespec64 *ts)
>> * If a suspend fails before reaching timekeeping_resume() then the flag
>> * stays true and prevents erroneous sleeptime injection.
>> */
>> -static bool sleeptime_injected = true;
>> +static bool suspend_timing_needed;
>>
>> /* Flag for if there is a persistent clock on this platform */
>> static bool persistent_clock_exists;
>> @@ -1658,7 +1658,7 @@ void timekeeping_inject_sleeptime64(struct timespec64
>> *delta)
>> raw_spin_lock_irqsave(&timekeeper_lock, flags);
>> write_seqcount_begin(&tk_core.seq);
>>
>> - sleeptime_injected = true;
>> + suspend_timing_needed = false;
>>
>> timekeeping_forward_now(tk);
>>
>> @@ -1714,10 +1714,10 @@ void timekeeping_resume(void)
>> tk->tkr_mono.mask);
>> nsec = mul_u64_u32_shr(cyc_delta, clock->mult,
>> clock->shift);
>> ts_delta = ns_to_timespec64(nsec);
>> - sleeptime_injected = true;
>> + suspend_timing_needed = true;
>> } else if (timespec64_compare(&ts_new, &timekeeping_suspend_time) >
>> 0) {
>> ts_delta = timespec64_sub(ts_new, timekeeping_suspend_time);
>> - sleeptime_injected = true;
>> + suspend_timing_needed = true;
>> }
>
> No no... This part is wrong. We only set suspend_timing_needed if we
> *didn't* calculate the suspend time in timekeeping_resume.
>
> You have to invert all the boolean logic for it to be equivalent.
>
...
>> <sleeptime injection happens here>
>
>
> So, I think with the logic bug above it will work out properly, but
> let me know if I'm still missing something.

Sorry, I meant "with the logic bug above fixed it will work out".

thanks
-john

2018-07-16 18:31:28

by Mukesh Ojha

[permalink] [raw]
Subject: Re: [PATCH v3] time: Fix incorrect sleeptime injection when suspend fails



On 7/16/2018 10:44 PM, John Stultz wrote:
> On Mon, Jul 16, 2018 at 9:30 AM, John Stultz <[email protected]> wrote:
>> On Mon, Jul 16, 2018 at 9:17 AM, Mukesh Ojha <[email protected]> wrote:
>>> On 7/13/2018 10:50 PM, John Stultz wrote:
>>>> On Fri, Jul 13, 2018 at 12:13 AM, Mukesh Ojha <[email protected]>
>>>>> On 7/11/2018 1:43 AM, John Stultz wrote:
>>>>>> I worry this upside-down logic is too subtle to be easily reasoned
>>>>>> about, and will just lead to future mistakes.
>>>>>>
>>>>>> Can we instead call this "suspend_timing_needed" and only set it to
>>>>>> true when we don't inject any sleep time on resume?
>>>>>
>>>>> I did not get your point "only set it to true when we don't inject any
>>>>> sleep
>>>>> time on resume? "
>>>>> How do we know this ?
>>>>> This question itself depends on the "sleeptime_injected" if it is true
>>>>> means
>>>>> no need to inject else need to inject.
>>>>>
>>>>> Also, we need to make this variable back and forth true, false; suspends
>>>>> path ensures it to make it false.
>>>> So yea, I'm not saying logically the code is really any different,
>>>> this is more of a naming nit. So instead of having a variable that is
>>>> always on that we occasionally turn off, lets invert the naming and
>>>> have it be a flag that we occasionally turn on.
>>>
>>> I understand your concern about the name of the variable will be misleading.
>>> But the changing Boolean state would not solve the actual issue.
>>>
>>> If i understand you correctly you meant below code
>>>
>>> diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
>>> index 32ae9ae..becc5bd 100644
>>> --- a/kernel/time/timekeeping.c
>>> +++ b/kernel/time/timekeeping.c
>>> @@ -1523,7 +1523,7 @@ void __weak read_boot_clock64(struct timespec64 *ts)
>>> * If a suspend fails before reaching timekeeping_resume() then the flag
>>> * stays true and prevents erroneous sleeptime injection.
>>> */
>>> -static bool sleeptime_injected = true;
>>> +static bool suspend_timing_needed;
>>>
>>> /* Flag for if there is a persistent clock on this platform */
>>> static bool persistent_clock_exists;
>>> @@ -1658,7 +1658,7 @@ void timekeeping_inject_sleeptime64(struct timespec64
>>> *delta)
>>> raw_spin_lock_irqsave(&timekeeper_lock, flags);
>>> write_seqcount_begin(&tk_core.seq);
>>>
>>> - sleeptime_injected = true;
>>> + suspend_timing_needed = false;
>>>
>>> timekeeping_forward_now(tk);
>>>
>>> @@ -1714,10 +1714,10 @@ void timekeeping_resume(void)
>>> tk->tkr_mono.mask);
>>> nsec = mul_u64_u32_shr(cyc_delta, clock->mult,
>>> clock->shift);
>>> ts_delta = ns_to_timespec64(nsec);
>>> - sleeptime_injected = true;
>>> + suspend_timing_needed = true;
>>> } else if (timespec64_compare(&ts_new, &timekeeping_suspend_time) >
>>> 0) {
>>> ts_delta = timespec64_sub(ts_new, timekeeping_suspend_time);
>>> - sleeptime_injected = true;
>>> + suspend_timing_needed = true;
>>> }
>> No no... This part is wrong. We only set suspend_timing_needed if we
>> *didn't* calculate the suspend time in timekeeping_resume.
>>
>> You have to invert all the boolean logic for it to be equivalent.
>>
> ...
>>> <sleeptime injection happens here>
>>
>> So, I think with the logic bug above it will work out properly, but
>> let me know if I'm still missing something.

Please give it thought to a case where very first suspend fails with
your logic.
If i am not able to get your thought, please write a patch.

-Mukesh

> Sorry, I meant "with the logic bug above fixed it will work out".
>
> thanks
> -john


2018-07-16 18:55:37

by John Stultz

[permalink] [raw]
Subject: Re: [PATCH v3] time: Fix incorrect sleeptime injection when suspend fails

On Mon, Jul 16, 2018 at 11:30 AM, Mukesh Ojha <[email protected]> wrote:
>
>
> On 7/16/2018 10:44 PM, John Stultz wrote:
>>
>> On Mon, Jul 16, 2018 at 9:30 AM, John Stultz <[email protected]>
>> wrote:
>>>
>>> On Mon, Jul 16, 2018 at 9:17 AM, Mukesh Ojha <[email protected]>
>>> wrote:
>>>>
>>>> On 7/13/2018 10:50 PM, John Stultz wrote:
>>>>>
>>>>> On Fri, Jul 13, 2018 at 12:13 AM, Mukesh Ojha <[email protected]>
>>>>>>
>>>>>> On 7/11/2018 1:43 AM, John Stultz wrote:
>>>>>>>
>>>>>>> I worry this upside-down logic is too subtle to be easily reasoned
>>>>>>> about, and will just lead to future mistakes.
>>>>>>>
>>>>>>> Can we instead call this "suspend_timing_needed" and only set it to
>>>>>>> true when we don't inject any sleep time on resume?
>>>>>>
>>>>>>
>>>>>> I did not get your point "only set it to true when we don't inject any
>>>>>> sleep
>>>>>> time on resume? "
>>>>>> How do we know this ?
>>>>>> This question itself depends on the "sleeptime_injected" if it is true
>>>>>> means
>>>>>> no need to inject else need to inject.
>>>>>>
>>>>>> Also, we need to make this variable back and forth true, false;
>>>>>> suspends
>>>>>> path ensures it to make it false.
>>>>>
>>>>> So yea, I'm not saying logically the code is really any different,
>>>>> this is more of a naming nit. So instead of having a variable that is
>>>>> always on that we occasionally turn off, lets invert the naming and
>>>>> have it be a flag that we occasionally turn on.
>>>>
>>>>
>>>> I understand your concern about the name of the variable will be
>>>> misleading.
>>>> But the changing Boolean state would not solve the actual issue.
>>>>
>>>> If i understand you correctly you meant below code
>>>>
>>>> diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
>>>> index 32ae9ae..becc5bd 100644
>>>> --- a/kernel/time/timekeeping.c
>>>> +++ b/kernel/time/timekeeping.c
>>>> @@ -1523,7 +1523,7 @@ void __weak read_boot_clock64(struct timespec64
>>>> *ts)
>>>> * If a suspend fails before reaching timekeeping_resume() then the
>>>> flag
>>>> * stays true and prevents erroneous sleeptime injection.
>>>> */
>>>> -static bool sleeptime_injected = true;
>>>> +static bool suspend_timing_needed;
>>>>
>>>> /* Flag for if there is a persistent clock on this platform */
>>>> static bool persistent_clock_exists;
>>>> @@ -1658,7 +1658,7 @@ void timekeeping_inject_sleeptime64(struct
>>>> timespec64
>>>> *delta)
>>>> raw_spin_lock_irqsave(&timekeeper_lock, flags);
>>>> write_seqcount_begin(&tk_core.seq);
>>>>
>>>> - sleeptime_injected = true;
>>>> + suspend_timing_needed = false;
>>>>
>>>> timekeeping_forward_now(tk);
>>>>
>>>> @@ -1714,10 +1714,10 @@ void timekeeping_resume(void)
>>>> tk->tkr_mono.mask);
>>>> nsec = mul_u64_u32_shr(cyc_delta, clock->mult,
>>>> clock->shift);
>>>> ts_delta = ns_to_timespec64(nsec);
>>>> - sleeptime_injected = true;
>>>> + suspend_timing_needed = true;
>>>> } else if (timespec64_compare(&ts_new,
>>>> &timekeeping_suspend_time) >
>>>> 0) {
>>>> ts_delta = timespec64_sub(ts_new,
>>>> timekeeping_suspend_time);
>>>> - sleeptime_injected = true;
>>>> + suspend_timing_needed = true;
>>>> }
>>>
>>> No no... This part is wrong. We only set suspend_timing_needed if we
>>> *didn't* calculate the suspend time in timekeeping_resume.
>>>
>>> You have to invert all the boolean logic for it to be equivalent.
>>>
>> ...
>>>>
>>>> <sleeptime injection happens here>
>>>
>>>
>>> So, I think with the logic bug above it will work out properly, but
>>> let me know if I'm still missing something.
>
>
> Please give it thought to a case where very first suspend fails with your
> logic.

I believe I did. If the first suspend fails, we never reach
timekeeping_resume, so we never set "suspend_time_needed = true", so
then timekeeping_rtc_skipresume can then return true, and we don't
inject the time in the RTC code.

> If i am not able to get your thought, please write a patch.

I probably will, but I'd like to encourage you to follow through on
this one. You reported the issue, and submitted a few patches, so I
think it would be good for you to also get the patch credit here. I
don't believe its a complex request I've made, and I think you can
figure it out.

So, please, take one more real stab at this and I'll rework it if it
seems necessary.

thanks
-john

2018-07-16 19:19:16

by Thomas Gleixner

[permalink] [raw]
Subject: Re: [PATCH v3] time: Fix incorrect sleeptime injection when suspend fails

On Tue, 17 Jul 2018, Mukesh Ojha wrote:
> On 7/16/2018 10:44 PM, John Stultz wrote:
> > > So, I think with the logic bug above it will work out properly, but
> > > let me know if I'm still missing something.
>
> Please give it thought to a case where very first suspend fails with your
> logic.
> If i am not able to get your thought, please write a patch.

John wants you to invert the logic. i.e.

true -> false
false -> true
if (var) -> if (!var)
if (!var) -> if (var)

It's not that hard, right?

Thanks,

tglx