Received-SPF: pass (google.com: domain of linux-kernel+bounces-197790-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Sat, 1 Jun 2024 11:33:25 +0200
From: "Theodore Ts'o" <tytso@mit.edu>
To: John Garry <john.g.garry@oracle.com>
Cc: Luis Chamberlain <mcgrof@kernel.org>, David Bueso <dave@stgolabs.net>,
        lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org,
        linux-mm <linux-mm@kvack.org>,
        "Martin K. Petersen" <martin.petersen@oracle.com>,
        Matthew Wilcox <willy@infradead.org>,
        Dave Chinner <david@fromorbit.com>, linux-kernel@vger.kernel.org,
        catherine.hoang@oracle.com
Subject: Re: [LSF/MM/BPF TOPIC] untorn buffered writes
Message-ID: <20240601093325.GC247052@mit.edu>
References: <20240228061257.GA106651@mit.edu>
 <9e230104-4fb8-44f1-ae5a-a940f69b8d45@oracle.com>
 <Zk5qKUJUOjGXEWus@bombadil.infradead.org>
 <bf638db9-c4d3-44bd-a92c-d36e3d95adb6@oracle.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <bf638db9-c4d3-44bd-a92c-d36e3d95adb6@oracle.com>

On Thu, May 23, 2024 at 12:59:57PM +0100, John Garry wrote:
> 
> That's my point really. There were some positive discussion. I put across
> the idea of implementing buffered atomic writes, and now I want to ensure
> that everyone is satisfied with that going forward. I think that a LWN
> report is now being written.

I checked in with some PostgreSQL developers after LSF/MM, and
unfortunately, the idea of immediately sending atomic buffered I/O
directly to the storage device is going to be problematic for them.
The problem is that they depend on the database to coalesce writes for
them.  So if they are doing a large database commit that involves
touching hundreds or thousands of 16k database pages, they today issue
a separate buffered write request for each database page.  So if we
turn each one into an immediate SCSI/NVMe write request, that would be
disastrous for performance.  Yes, when they migrate to using Direct
I/O, the database is going to have to figure out how to coalesce write
requests; but this is why it's going to take at least 3 years to make
this migration (and some will call this hopelessly optimistic), and
then users will probably wait another 3 to 5 years before they trust
that the database rewrite to use Direct I/O will get it right and
trust their enterprise workloads to it....

So I think this goes back to either (a) trying to track which writes
we've promised atomic write semantics, or (b) using a completely
different API that only promises "untorn writes with a specified
granulatity" approach for the untorn buffered writes I/O interface,
instead in addition to, or instead of, the current "atomic write"
interface which we are currently trying to promulate for Direct I/O.

Personally, I'd advocate for two separate interfaces; one for "atomic"
I/O's, and a different one for "untorn writes with a specified
guaranteed granularity".  And if XFS folks want to turn the atomic I/O
interface into something where you can do a multi-megabyte atomic
write into something that requires allocating new blocks and
atomically mutating the file system metadata to do this kind of
atomicity --- even though the Database folks Don't Care --- God bless.

But let's have something which *just* promises the guarantee requested
by the primary requesteres of this interface, at least for the
buffered I/O case.

Cheers,

						- Ted