Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755127AbdC3VNO (ORCPT ); Thu, 30 Mar 2017 17:13:14 -0400 Received: from mail-it0-f51.google.com ([209.85.214.51]:36147 "EHLO mail-it0-f51.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754586AbdC3VNM (ORCPT ); Thu, 30 Mar 2017 17:13:12 -0400 MIME-Version: 1.0 In-Reply-To: <23902.1490904827@warthog.procyon.org.uk> References: <22214.1490895007@warthog.procyon.org.uk> <23410.1490902528@warthog.procyon.org.uk> <23902.1490904827@warthog.procyon.org.uk> From: Linus Torvalds Date: Thu, 30 Mar 2017 14:13:10 -0700 X-Google-Sender-Auth: A0apOQPr4TdylTkJQJyh168lkpA Message-ID: Subject: Re: Apparent backward time travel in timestamps on file creation To: David Howells Cc: Thomas Gleixner , John Stultz , Linux Kernel Mailing List , linux-fsdevel Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3248 Lines: 69 On Thu, Mar 30, 2017 at 1:13 PM, David Howells wrote: > Linus Torvalds wrote: > >> The error bar can be huge, for the simple reason that the filesystem >> you are testing may not be sharing a clock with the CPU at _all_. >> >> IOW, think network filesystems. > > Can't I just not do the tests when the filesystem is a network fs? I don't > think it should be a problem for disk filesystems on network-attached storage. So I actually think that the whole "check timestamps" would be interesting as a test across a lot of filesystems - including very much network filesystems - but I think such a test should be largely informational rather than about correctness. Now, there definitely are correctness issues too wrt file timestamps (ie the whole "writes should update mtime" kind of testing), and I think many of those cound be extended to check relative timestamps on the same filesystem. For example, if you write to file A first, and to file B second, it would certainly be odd and interesting if file B now has a modification time that is before file A. I say "odd and interesting" because I could see it happening (writeback ordering), and there are certainly very real issues wrt things like open/close consistency ordering. For example, if you write to A, then _close_ A, and then open B and write to B, if the timestamps are still in the wrong order that really could cause major real problems for simple things like "make". So there's a lot of valid and interesting timestamp data that could be tested for. The "is it in sync with gettimeofday()" is interesting too, even if the answer is that you don't expect it to be _perfectly_ in sync. A test that just reports maximum slop might be an interesting test, and could show real problems (maybe bad network time synchronization, but maybe actual bugs in our internal xtime handling even for local filesystems!). There are just plain possible *oddities* with file time stamps too. For example, we've definitely had situations where when you wrote to a file, the mtime was updated to be precise to within a millisecond - but then 30 seconds later, when the inode was written back, the mtime might be truncated to the on-disk format that might be just seconds (or hundreds of seconds), so now the mtime ON THE SAME FILE might be going backwards! So I do think that testing for odd cases like this can be interesting, but I think it should be primarily be seen as a "reporting" issue than a "correctness" issue. Then a human might react to the report and say "that looks _really_ odd and is _way_ out of line of the expected clock skew" and elevate it to a correctness issue. So maybe you shouldn't do any real "fudge factor", and just report the raw numbers. Things like just your comment "As can be seen, the three file timestamps are -0.000193078 sec from the prior CPU timestamp" is just interesting, and as long as it's in that fairly small range, it's not a bug but it's a good reminder that filesystem timestamps aren't the same as gettimeofday(). And then if your tool starts reporting times that are off by seconds or minutes, people might say "Hey, that's not right.." and find something. Linus