Received-SPF: pass (google.com: domain of linux-kernel+bounces-75146-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Wed, 21 Feb 2024 08:56:15 -0800
From: "Darrick J. Wong" <djwong@kernel.org>
To: Christoph Hellwig <hch@lst.de>
Cc: John Garry <john.g.garry@oracle.com>, viro@zeniv.linux.org.uk,
	brauner@kernel.org, dchinner@redhat.com, jack@suse.cz,
	chandan.babu@oracle.com, martin.petersen@oracle.com,
	linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org,
	linux-fsdevel@vger.kernel.org, tytso@mit.edu, jbongio@google.com,
	ojaswin@linux.ibm.com
Subject: Re: [PATCH 0/6] block atomic writes for XFS
Message-ID: <20240221165615.GH6184@frogsfrogsfrogs>
References: <20240124142645.9334-1-john.g.garry@oracle.com>
 <20240213072237.GA24218@lst.de>
 <20240213175549.GU616564@frogsfrogsfrogs>
 <20240214074559.GB10006@lst.de>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240214074559.GB10006@lst.de>

On Wed, Feb 14, 2024 at 08:45:59AM +0100, Christoph Hellwig wrote:
> On Tue, Feb 13, 2024 at 09:55:49AM -0800, Darrick J. Wong wrote:
> > On Tue, Feb 13, 2024 at 08:22:37AM +0100, Christoph Hellwig wrote:
> > > From reading the series and the discussions with Darrick and Dave
> > > I'm coming more and more back to my initial position that tying this
> > > user visible feature to hardware limits is wrong and will just keep
> > > on creating ever more painpoints in the future.
> > > 
> > > Based on that I suspect that doing proper software only atomic writes
> > > using the swapext log item and selective always COW mode
> > 
> > Er, what are you thinking w.r.t. swapext and sometimescow?
> 
> What do you mean with sometimescow?  Just normal reflinked inodes?
> 
> > swapext
> > doesn't currently handle COW forks at all, and it can only exchange
> > between two of the same type of fork (e.g. both data forks or both attr
> > forks, no mixing).
> > 
> > Or will that be your next suggestion whenever I get back to fiddling
> > with the online fsck patches? ;)
> 
> Let's take a step back.  If we want atomic write semantics without
> hardware offload, what we need is to allocate new blocks and atomically
> swap them into the data fork.  Basicall an atomic version of
> xfs_reflink_end_cow.  But yes, the details of the current swapext
> item might not be an exact fit, maybe it's just shared infrastructure
> and concepts.

Hmm.  For rt reflink (whenever I get back to that, ha) I've been
starting to think that yes, we actually /do/ want to have a log item
that tracks the progress of remap and cow operations.  That would solve
the problem of someone wanting to reflink a semi-written rtx.

That said, it might complicate the reflink code quite a bit since right
now it writes zeroes to the unwritten parts of an rt file's rtx so that
there's only one mapping record for the whole rtx, and then it remaps
them.  That's most of why I haven't bothered to implement that solution.

> I'm not planning to make you do it, because such a log item would
> generally be pretty useful for always COW mode.

One other thing -- while I was refactoring the swapext code into
exch{range,maps}, it occurred to me that doing an exchange between the
cow and data forks isn't possible because log recovery won't be able to
do anything.  There's no ondisk metadata to map a cow staging extent
back to the file it came from, which means we can't generally resume an
exchange operation.

However for a small write I guess you could simply queue all the log
intent items for all the changes needed and commit that.

> > > and making that
> > > work should be the first step.  We can then avoid that overhead for
> > > properly aligned writs if the hardware supports it.  For your Oracle
> > > DB loads you'll set the alignment hints and maybe even check with
> > > fiemap that everything is fine and will get the offload, but we also
> > > provide a nice and useful API for less performance critical applications
> > > that don't have to care about all these details.
> > 
> > I suspect they might want to fail-fast (back to standard WAL mode or
> > whatever) if the hardware support isn't available.
> 
> Maybe for your particular DB use case.  But there's plenty of
> applications that just want atomic writes without building their
> own infrastruture, including some that want pretty large chunks.
> 
> Also if a file system supports logging data (which I have an
> XFS early prototype for that I plan to finish), we can even do
> the small double writes more efficiently than the application,
> all through the same interface.

Heh.  Ted's been trying to kill data=journal.  Now we've found a use for
it after all. :)

--D