Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934036AbbELVv4 (ORCPT ); Tue, 12 May 2015 17:51:56 -0400 Received: from ipmail06.adl6.internode.on.net ([150.101.137.145]:48030 "EHLO ipmail06.adl6.internode.on.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933918AbbELVvu (ORCPT ); Tue, 12 May 2015 17:51:50 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2C6CgDpdFJVPPDOLHlcgw+BMoZMrFwBAQEBAQEGmVYCAgEBAoFATQEBAQEBAQcBAQEBQT+EIQEBBDocIxAIAxgJDBkPBQ0YAwcaE4gXAxHFJQ2EfAEBAQEBBQEBAQEeGIV+hSOCTYFMbAcKgw2BFgWZOoIqgVSBJYtOgi5Zgx+DVYEEgQUeBRyBZCwxgkYBAQE Date: Wed, 13 May 2015 07:51:45 +1000 From: Dave Chinner To: Austin S Hemmelgarn Cc: "J. Bruce Fields" , John Stoffel , Kevin Easton , "Theodore Ts'o" , Sage Weil , Trond Myklebust , Zach Brown , Alexander Viro , Linux FS-devel Mailing List , Linux Kernel Mailing List , Linux API Mailing List Subject: Re: [PATCH RFC] vfs: add a O_NOMTIME flag Message-ID: <20150512215145.GA4316@dastard> References: <20150511144719.GA14088@thunk.org> <20150511231021.GC14088@thunk.org> <20150512050821.GA9404@chicago.guarana.org> <5551E7EB.8040301@gmail.com> <21842.1555.38099.868100@quad.stoffel.home> <20150512143637.GA6370@fieldses.org> <555213E9.90701@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <555213E9.90701@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4311 Lines: 87 On Tue, May 12, 2015 at 10:53:29AM -0400, Austin S Hemmelgarn wrote: > On 2015-05-12 10:36, J. Bruce Fields wrote: > >On Tue, May 12, 2015 at 09:54:27AM -0400, John Stoffel wrote: > >>>>>>>"Austin" == Austin S Hemmelgarn writes: > >> > >>Austin> On 2015-05-12 01:08, Kevin Easton wrote: > >>>>On Mon, May 11, 2015 at 07:10:21PM -0400, Theodore Ts'o wrote: > >>>>>On Mon, May 11, 2015 at 09:24:09AM -0700, Sage Weil wrote: > >>>>>>>Let me re-ask the question that I asked last week (and was apparently > >>>>>>>ignored). Why not trying to use the lazytime feature instead of > >>>>>>>pointing a head straight at the application's --- and system > >>>>>>>administrators' --- heads? > >>>>>> > >>>>>>Sorry Ted, I thought I responded already. > >>>>>> > >>>>>>The goal is to avoid inode writeout entirely when we can, and > >>>>>>as I understand it lazytime will still force writeout before the inode > >>>>>>is dropped from the cache. In systems like Ceph in particular, the > >>>>>>IOs can be spread across lots of files, so simply deferring writeout > >>>>>>doesn't always help. > >>>>> > >>>>>Sure, but it would reduce the writeout by orders of magnitude. I can > >>>>>understand if you want to reduce it further, but it might be good > >>>>>enough for your purposes. > >>>>> > >>>>>I considered doing the equivalent of O_NOMTIME for our purposes at > >>>>>$WORK, and our use case is actually not that different from Ceph's > >>>>>(i.e., using a local disk file system to support a cluster file > >>>>>system), and lazytime was (a) something I figured was something I > >>>>>could upstream in good conscience, and (b) was more than good enough > >>>>>for us. > >>>> > >>>>A safer alternative might be a chattr file attribute that if set, the > >>>>mtime is not updated on writes, and stat() on the file always shows the > >>>>mtime as "right now". At least that way, the file won't accidentally > >>>>get left out of backups that rely on the mtime. > >>>> > >>>>(If the file attribute is unset, you immediately update the mtime then > >>>>too, and from then on the file is back to normal). > >>>> > >> > >>Austin> I like this even better than the flag suggestion, it provides > >>Austin> better control, means that you don't need to update > >>Austin> applications to get the benefits, and prevents backup software > >>Austin> from breaking (although backups would be bigger). > >> > >>Me too, it fails in a safer mode, where you do more work on backups > >>than strictly needed. I'm still against this as a mount option > >>though, way way way too many bullets in the foot gun. And as someone > >>else said, once you mount with O_NOMTIME, then unmount, then mount > >>again without O_NOMTIME, you've lost information. Not good. > > > >That was me. Zach also pointed out to me that'd mean figuring out where > >to store that information on-disk for every filesystem you care about. > >I like the idea of something persistent, but maybe it's more trouble > >than it's worth--I honestly don't know. > > > But if we do it as a flag controlled by the API used by chattr, it > becomes the responsibility of the filesystems to deal with where to > store the information, assuming they choose to support it; > personally, I would be really surprised if XFS and BTRFS didn't add > support for this relatively soon after the API getting merged > upstream, and ext4 would likely follow soon afterwards. It's an on-disk format change, which means that there are all sorts of compatibility issues to take into account, as well as all the work needed to teach the filesystem userspace tools about the new flag. e.g. xfs_repair, xfs_db, xfsdump/restore, xfs_io, test code in xfstests, etc. Keep in mind that the moment we make something persistent, the amount of work to implement and verify the new functionality filesystem to implement it goes up by an order of magnitude *for each filesystem*. IOWs, support of new features that require persistence don't just magically appear overnight... Cheers, Dave. -- Dave Chinner david@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/