Jakub Drnec reported:
Setting the realtime clock can sometimes make the monotonic clock go
back by over a hundred years. Decreasing the realtime clock across
the y2k38 threshold is one reliable way to reproduce. Allegedly this
can also happen just by running ntpd, I have not managed to
reproduce that other than booting with rtc at >2038 and then running
ntp. When this happens, anything with timers (e.g. openjdk) breaks
rather badly.
And included a test case (slightly edited for brevity):
#define _POSIX_C_SOURCE 199309L
#include <stdio.h>
#include <time.h>
#include <stdlib.h>
#include <unistd.h>
long get_time(void) {
struct timespec tp;
clock_gettime(CLOCK_MONOTONIC, &tp);
return tp.tv_sec + tp.tv_nsec / 1000000000;
}
int main(void) {
long last = get_time();
while(1) {
long now = get_time();
if (now < last) {
printf("clock went backwards by %ld seconds!\n", last - now);
}
last = now;
sleep(1);
}
return 0;
}
Which when run concurrently with:
# date -s 2040-1-1
# date -s 2037-1-1
Will detect the clock going backward.
The root cause is that wtom_clock_sec in struct vdso_data is only a
32-bit signed value, even though we set its value to be equal to
tk->wall_to_monotonic.tv_sec which is 64-bits.
Because the monotonic clock starts at zero when the system boots the
wall_to_montonic.tv_sec offset is negative for current and future
dates. Currently on a freshly booted system the offset will be in the
vicinity of negative 1.5 billion seconds.
However if the wall clock is set past the Y2038 boundary, the offset
from wall to monotonic becomes less than negative 2^31, and no longer
fits in 32-bits. When that value is assigned to wtom_clock_sec it is
truncated and becomes positive, causing the VDSO assembly code to
calculate CLOCK_MONOTONIC incorrectly.
That causes CLOCK_MONOTONIC to jump ahead by ~4 billion seconds which
it is not meant to do. Worse, if the time is then set back before the
Y2038 boundary CLOCK_MONOTONIC will jump backward.
We can fix it simply by storing the full 64-bit offset in the
vdso_data, and using that in the VDSO assembly code. We also shuffle
some of the fields in vdso_data to avoid creating a hole.
The original commit that added the CLOCK_MONOTONIC support to the VDSO
did actually use a 64-bit value for wtom_clock_sec, see commit
a7f290dad32e ("[PATCH] powerpc: Merge vdso's and add vdso support to
32 bits kernel") (Nov 2005). However just 3 days later it was
converted to 32-bits in commit 0c37ec2aa88b ("[PATCH] powerpc: vdso
fixes (take #2)"), and the bug has existed since then AFAICS.
Fixes: 0c37ec2aa88b ("[PATCH] powerpc: vdso fixes (take #2)")
Cc: [email protected] # v2.6.15+
Link: http://lkml.kernel.org/r/[email protected]
Reported-by: Jakub Drnec <[email protected]>
Signed-off-by: Michael Ellerman <[email protected]>
---
arch/powerpc/include/asm/vdso_datapage.h | 8 ++++----
arch/powerpc/kernel/vdso64/gettimeofday.S | 4 ++--
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/include/asm/vdso_datapage.h b/arch/powerpc/include/asm/vdso_datapage.h
index 1afe90ade595..bbc06bd72b1f 100644
--- a/arch/powerpc/include/asm/vdso_datapage.h
+++ b/arch/powerpc/include/asm/vdso_datapage.h
@@ -82,10 +82,10 @@ struct vdso_data {
__u32 icache_block_size; /* L1 i-cache block size */
__u32 dcache_log_block_size; /* L1 d-cache log block size */
__u32 icache_log_block_size; /* L1 i-cache log block size */
- __s32 wtom_clock_sec; /* Wall to monotonic clock */
- __s32 wtom_clock_nsec;
- struct timespec stamp_xtime; /* xtime as at tb_orig_stamp */
- __u32 stamp_sec_fraction; /* fractional seconds of stamp_xtime */
+ __u32 stamp_sec_fraction; /* fractional seconds of stamp_xtime */
+ __s32 wtom_clock_nsec; /* Wall to monotonic clock nsec */
+ __s64 wtom_clock_sec; /* Wall to monotonic clock sec */
+ struct timespec stamp_xtime; /* xtime as at tb_orig_stamp */
__u32 syscall_map_64[SYSCALL_MAP_SIZE]; /* map of syscalls */
__u32 syscall_map_32[SYSCALL_MAP_SIZE]; /* map of syscalls */
};
diff --git a/arch/powerpc/kernel/vdso64/gettimeofday.S b/arch/powerpc/kernel/vdso64/gettimeofday.S
index a4ed9edfd5f0..1f324c28705b 100644
--- a/arch/powerpc/kernel/vdso64/gettimeofday.S
+++ b/arch/powerpc/kernel/vdso64/gettimeofday.S
@@ -92,7 +92,7 @@ V_FUNCTION_BEGIN(__kernel_clock_gettime)
* At this point, r4,r5 contain our sec/nsec values.
*/
- lwa r6,WTOM_CLOCK_SEC(r3)
+ ld r6,WTOM_CLOCK_SEC(r3)
lwa r9,WTOM_CLOCK_NSEC(r3)
/* We now have our result in r6,r9. We create a fake dependency
@@ -125,7 +125,7 @@ V_FUNCTION_BEGIN(__kernel_clock_gettime)
bne cr6,75f
/* CLOCK_MONOTONIC_COARSE */
- lwa r6,WTOM_CLOCK_SEC(r3)
+ ld r6,WTOM_CLOCK_SEC(r3)
lwa r9,WTOM_CLOCK_NSEC(r3)
/* check if counter has updated */
--
2.20.1
On Wed, Mar 13, 2019 at 2:14 PM Michael Ellerman <[email protected]> wrote:
> That causes CLOCK_MONOTONIC to jump ahead by ~4 billion seconds which
> it is not meant to do. Worse, if the time is then set back before the
> Y2038 boundary CLOCK_MONOTONIC will jump backward.
>
> We can fix it simply by storing the full 64-bit offset in the
> vdso_data, and using that in the VDSO assembly code. We also shuffle
> some of the fields in vdso_data to avoid creating a hole.
I see nothing wrong with your patch, but I would point out that there is
a patch series [1] from Vincenzo Frascino to unify the vdso implementation
across architectures that I hope can make it into linux-5.2, and that will
resolve this issue, as well as allow 32-bit architectures to provide
a working interface with 64-bit time_t.
Arnd
[1] https://lore.kernel.org/linux-arm-kernel/[email protected]/
Arnd Bergmann <[email protected]> writes:
> On Wed, Mar 13, 2019 at 2:14 PM Michael Ellerman <[email protected]> wrote:
>
>> That causes CLOCK_MONOTONIC to jump ahead by ~4 billion seconds which
>> it is not meant to do. Worse, if the time is then set back before the
>> Y2038 boundary CLOCK_MONOTONIC will jump backward.
>>
>> We can fix it simply by storing the full 64-bit offset in the
>> vdso_data, and using that in the VDSO assembly code. We also shuffle
>> some of the fields in vdso_data to avoid creating a hole.
>
> I see nothing wrong with your patch,
Thanks.
> but I would point out that there is a patch series [1] from Vincenzo
> Frascino to unify the vdso implementation across architectures that I
> hope can make it into linux-5.2, and that will resolve this issue, as
> well as allow 32-bit architectures to provide a working interface with
> 64-bit time_t.
Yeah I did see that series. I will try and keep an eye on it, though I'm
not sure I'll have time to convert powerpc to use it for 5.2.
I'm also not sure how easy it's going to be to convert to the C
versions, because our syscall ABI is not a simple function call (result
code is returned in CR0.SO).
cheers
On Wed, 2019-03-13 at 13:14:38 UTC, Michael Ellerman wrote:
> Jakub Drnec reported:
> Setting the realtime clock can sometimes make the monotonic clock go
> back by over a hundred years. Decreasing the realtime clock across
> the y2k38 threshold is one reliable way to reproduce. Allegedly this
> can also happen just by running ntpd, I have not managed to
> reproduce that other than booting with rtc at >2038 and then running
> ntp. When this happens, anything with timers (e.g. openjdk) breaks
> rather badly.
>
> And included a test case (slightly edited for brevity):
> #define _POSIX_C_SOURCE 199309L
> #include <stdio.h>
> #include <time.h>
> #include <stdlib.h>
> #include <unistd.h>
>
> long get_time(void) {
> struct timespec tp;
> clock_gettime(CLOCK_MONOTONIC, &tp);
> return tp.tv_sec + tp.tv_nsec / 1000000000;
> }
>
> int main(void) {
> long last = get_time();
> while(1) {
> long now = get_time();
> if (now < last) {
> printf("clock went backwards by %ld seconds!\n", last - now);
> }
> last = now;
> sleep(1);
> }
> return 0;
> }
>
> Which when run concurrently with:
> # date -s 2040-1-1
> # date -s 2037-1-1
>
> Will detect the clock going backward.
>
> The root cause is that wtom_clock_sec in struct vdso_data is only a
> 32-bit signed value, even though we set its value to be equal to
> tk->wall_to_monotonic.tv_sec which is 64-bits.
>
> Because the monotonic clock starts at zero when the system boots the
> wall_to_montonic.tv_sec offset is negative for current and future
> dates. Currently on a freshly booted system the offset will be in the
> vicinity of negative 1.5 billion seconds.
>
> However if the wall clock is set past the Y2038 boundary, the offset
> from wall to monotonic becomes less than negative 2^31, and no longer
> fits in 32-bits. When that value is assigned to wtom_clock_sec it is
> truncated and becomes positive, causing the VDSO assembly code to
> calculate CLOCK_MONOTONIC incorrectly.
>
> That causes CLOCK_MONOTONIC to jump ahead by ~4 billion seconds which
> it is not meant to do. Worse, if the time is then set back before the
> Y2038 boundary CLOCK_MONOTONIC will jump backward.
>
> We can fix it simply by storing the full 64-bit offset in the
> vdso_data, and using that in the VDSO assembly code. We also shuffle
> some of the fields in vdso_data to avoid creating a hole.
>
> The original commit that added the CLOCK_MONOTONIC support to the VDSO
> did actually use a 64-bit value for wtom_clock_sec, see commit
> a7f290dad32e ("[PATCH] powerpc: Merge vdso's and add vdso support to
> 32 bits kernel") (Nov 2005). However just 3 days later it was
> converted to 32-bits in commit 0c37ec2aa88b ("[PATCH] powerpc: vdso
> fixes (take #2)"), and the bug has existed since then AFAICS.
>
> Fixes: 0c37ec2aa88b ("[PATCH] powerpc: vdso fixes (take #2)")
> Cc: [email protected] # v2.6.15+
> Link: http://lkml.kernel.org/r/[email protected]
> Reported-by: Jakub Drnec <[email protected]>
> Signed-off-by: Michael Ellerman <[email protected]>
Applied to powerpc fixes.
https://git.kernel.org/powerpc/c/b5b4453e7912f056da1ca7572574cada
cheers