Received-SPF: pass (google.com: domain of linux-kernel+bounces-208039-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
From: John Garry <john.g.garry@oracle.com>
To: axboe@kernel.dk, kbusch@kernel.org, hch@lst.de, sagi@grimberg.me,
        jejb@linux.ibm.com, martin.petersen@oracle.com,
        viro@zeniv.linux.org.uk, brauner@kernel.org, dchinner@redhat.com,
        jack@suse.cz
Cc: djwong@kernel.org, linux-block@vger.kernel.org,
        linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org,
        linux-fsdevel@vger.kernel.org, tytso@mit.edu, jbongio@google.com,
        linux-scsi@vger.kernel.org, ojaswin@linux.ibm.com, linux-aio@kvack.org,
        linux-btrfs@vger.kernel.org, io-uring@vger.kernel.org,
        nilay@linux.ibm.com, ritesh.list@gmail.com, willy@infradead.org,
        agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com,
        dm-devel@lists.linux.dev, hare@suse.de,
        John Garry <john.g.garry@oracle.com>
Subject: [PATCH v8 07/10] block: Add fops atomic write support
Date: Mon, 10 Jun 2024 10:43:26 +0000
Message-Id: <20240610104329.3555488-8-john.g.garry@oracle.com>
In-Reply-To: <20240610104329.3555488-1-john.g.garry@oracle.com>
References: <20240610104329.3555488-1-john.g.garry@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?tSDJrwjxJIK198lrQWsivXhpsy7/ap2DJCjQ/R0rbR55x0k2sdhyFVuEFKyQ?=
 =?us-ascii?Q?v04KJZ2rMGA/w0oT1OdfDWNXbMJRzTVtP63PIgO4Xktvrr+J9hNhTOE3cO7+?=
 =?us-ascii?Q?cUXsZ5ZU/Soc4vB10ek/7eiW0Irk8Q9fbaTzM6RV1dw6fcUztP8FvoPD3igy?=
 =?us-ascii?Q?7UJmX4rgY1DM9r9OshwFZVBHubPXmdfEL5aPi7m1Q8FPn31FTU2s1rHqLK2u?=
 =?us-ascii?Q?biZUqjTCma2Fzt606/eBqg2QTrIpMbou1raTO8Qima7Tnt2xUJ7VY/+FZMwd?=
 =?us-ascii?Q?zXqnrT8YedN8Z/TpFro7RwuL60teLshwUV79GaD3iufEPgGtM52d4tsODb7S?=
 =?us-ascii?Q?3vbj/TYnil3HJafEKAdTnExQ0g3MQp57AVzo9eFvdFM7tWf6K09W3raqbcj1?=
 =?us-ascii?Q?wc9hmLZEPUHPDqXEFtHBwShWmYsa7sNA2uqtvPKjKohbEKsVQ2Ho+CqA87IF?=
 =?us-ascii?Q?YTIkk6awg1zJHumehUZ/+PMCounwwb/mPAwrZ3myuq1xFGxZ+OE2NIpP79sZ?=
 =?us-ascii?Q?cKMj3wPYPqPkZa6ZwQ+zSGHf/PSkznyEIthNJsjYrNCexHhUWu9iz0bsEPoX?=
 =?us-ascii?Q?SWTeTtg+Et2ajFxKLamnnIA8Ytq+wo/Iu8Z706eLjslgR2F8ngGbG7QrqHXe?=
 =?us-ascii?Q?JkxqrqMJJG3CEwc1CQNpARKwlNx4E2olo4b0efAcibrM4tujaaIlD+5mMVDW?=
 =?us-ascii?Q?/LY2CWuPQHehR0hl5Moh0rjIk7JKdjJ4uLS6utfVBKZ5aEbehytiLyyJh5hI?=
 =?us-ascii?Q?Xgu9Z+HqSmQT7s/etsOEiV7bbrPsiZPPad4V9kb8pVemc6TkAOYWhHUU7MSo?=
 =?us-ascii?Q?xWPdgmj3SBnacok3jJDugGMZS6wLNobs9s0/OBVxgjrpuSVcAeZxxEOcSwhC?=
 =?us-ascii?Q?nQFqjtM7zsmMU2eb3kcQ+62rtol46ZQa/05glwpfs7TLI1dFPHRacop29sWj?=
 =?us-ascii?Q?fkzdF+7vgJB2k9Wq81cPIBCOkCA2WvO1NnChPwd0mFKrIoJawJ+HZ/JV/1si?=
 =?us-ascii?Q?tvEcwVWMlcM1vcOHEqPFgy1rY+P8VtDTd0JaHJUZqfMFR+AiicIY+8i7anw7?=
 =?us-ascii?Q?qJeVBk+HdQXwN3K8lODzTez0vZiicE16qsYcWmzR8BjiWHrbOogNjQh1k9Uf?=
 =?us-ascii?Q?FaYZ69Vc3LGyHnKEHKL6MDuhenql6k87sFR4cW4OIc4bRYqeGDhKfuvkkQLt?=
 =?us-ascii?Q?8HrqbA2x23pJhNpigNuffQvjtrgFD6ODqn9cIvpHbs8kzzBO0czc/7bn/eJo?=
 =?us-ascii?Q?DhUnrlEI5Reo3yJdjLBC+ddgpmi3Y3m03ipFZXRAaZdRWVw54i3l3XUf2Qzf?=
 =?us-ascii?Q?2GjMl88TFVau3nZUIAQqcTLiUFJwpLnrYbn6Zo5PSufoJPxKdz0n6UynwRSD?=
 =?us-ascii?Q?fA3e7DUGexYDbUnf/j6nR7a3uxVhWJCkBowBxrR3zsn+aKIzrGUSX4Uat0XQ?=
 =?us-ascii?Q?W9WC2PG6k2WSSXuMlLFaXqJgPQL0TLDCekN78hZqxoNG9EMb0cy+t68+3aeL?=
 =?us-ascii?Q?kRXmtuv6HgeiSVJAXerqSBBn+2W4OJxmtwH0QueArG/Sq73orQyNaBJXd9an?=
 =?us-ascii?Q?qSqkQTVB1hyTpHeApWh0KMWD6FpKFmm16ErJhRbhLXNjxYon5pUrTgTAR4Ly?=
 =?us-ascii?Q?+w=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	Txntz8pvY0R6flYTSzvQ2rbNvh24hGus+0KST6DUhNYpvxH9khH9vdnzkgogMoZgheAdtLp6PI2MF9ptCcrb6+IW/0jVYrzrwscigTx6pBmbz6DUcAT3UWJY9tDfwTmhPp6jyFHg5fdN6KuA1hbrhHihMO7lxt01r7z4z3AiSLXO8+A7eXHp4Lq6f7JkHgSny3ANXagYWYCBzyImNzlQKfGLfA3XMsf0RzmG7d3mP+FzMzcD89xX+BCQAvND2OSgEvizPl/F615E9rp+igREx0A9CSf+Uj1m7p/MwTv1gO1GhcgN2rgNXH8MbPbFT4Ww9T2/Bqx77cwz3ucMlYKp/lAfL8NBh2ZQDOVGVXR7QeLIp/BTlCqTBOleHhnV917iI12DCbHWN3EQDfy6dzLeJZCBbrXkUHiWkMk0Kzt68W5N4Mn5hORisiBVtQeFpj5HaUhHVEODWIOpp+Y0Kn+QM97218CTh0raETAQQvgrPBfKdiGzfMi2FrYlyOCg8US5dRGucud+0kry4kD5de3h6UZmjJAiGBiF5M7aM2ERMsiCTUO7Qgx7DkMYAcWr0WGc0hBgiiw1aM78J0HYky7Dp9K4y0Xyvlov0g7rWaJcXhA=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0d4a17a0-b4fb-4d79-c65f-08dc893a3d53
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jun 2024 10:44:01.0844
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Mffuc21RzOF/Nh1ZWw8Xk+IIsw+cwxO6lwuww+oRnI06KW81IvIlQi0cqGU+PPovwOR5sB6fzrIb0E1A6pfaFQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB5613
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16
 definitions=2024-06-10_02,2024-06-10_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999
 bulkscore=0 malwarescore=0 spamscore=0 phishscore=0 mlxscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2405010000 definitions=main-2406100081
X-Proofpoint-GUID: s2pGwygE7nT4orZVtJQCZ2E4fi9B_y7m
X-Proofpoint-ORIG-GUID: s2pGwygE7nT4orZVtJQCZ2E4fi9B_y7m

Support atomic writes by submitting a single BIO with the REQ_ATOMIC set.

It must be ensured that the atomic write adheres to its rules, like
naturally aligned offset, so call blkdev_dio_invalid() ->
blkdev_atomic_write_valid() [with renaming blkdev_dio_unaligned() to
blkdev_dio_invalid()] for this purpose. The BIO submission path currently
checks for atomic writes which are too large, so no need to check here.

In blkdev_direct_IO(), if the nr_pages exceeds BIO_MAX_VECS, then we cannot
produce a single BIO, so error in this case.

Finally set FMODE_CAN_ATOMIC_WRITE when the bdev can support atomic writes
and the associated file flag is for O_DIRECT.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 block/fops.c | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/block/fops.c b/block/fops.c
index 376265935714..be36c9fbd500 100644
--- a/block/fops.c
+++ b/block/fops.c
@@ -34,9 +34,12 @@ static blk_opf_t dio_bio_write_op(struct kiocb *iocb)
 	return opf;
 }
 
-static bool blkdev_dio_unaligned(struct block_device *bdev, loff_t pos,
-			      struct iov_iter *iter)
+static bool blkdev_dio_invalid(struct block_device *bdev, loff_t pos,
+				struct iov_iter *iter, bool is_atomic)
 {
+	if (is_atomic && !generic_atomic_write_valid(iter, pos))
+		return true;
+
 	return pos & (bdev_logical_block_size(bdev) - 1) ||
 		!bdev_iter_is_aligned(bdev, iter);
 }
@@ -72,6 +75,8 @@ static ssize_t __blkdev_direct_IO_simple(struct kiocb *iocb,
 	bio.bi_iter.bi_sector = pos >> SECTOR_SHIFT;
 	bio.bi_write_hint = file_inode(iocb->ki_filp)->i_write_hint;
 	bio.bi_ioprio = iocb->ki_ioprio;
+	if (iocb->ki_flags & IOCB_ATOMIC)
+		bio.bi_opf |= REQ_ATOMIC;
 
 	ret = bio_iov_iter_get_pages(&bio, iter);
 	if (unlikely(ret))
@@ -343,6 +348,9 @@ static ssize_t __blkdev_direct_IO_async(struct kiocb *iocb,
 		task_io_account_write(bio->bi_iter.bi_size);
 	}
 
+	if (iocb->ki_flags & IOCB_ATOMIC)
+		bio->bi_opf |= REQ_ATOMIC;
+
 	if (iocb->ki_flags & IOCB_NOWAIT)
 		bio->bi_opf |= REQ_NOWAIT;
 
@@ -359,12 +367,13 @@ static ssize_t __blkdev_direct_IO_async(struct kiocb *iocb,
 static ssize_t blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 {
 	struct block_device *bdev = I_BDEV(iocb->ki_filp->f_mapping->host);
+	bool is_atomic = iocb->ki_flags & IOCB_ATOMIC;
 	unsigned int nr_pages;
 
 	if (!iov_iter_count(iter))
 		return 0;
 
-	if (blkdev_dio_unaligned(bdev, iocb->ki_pos, iter))
+	if (blkdev_dio_invalid(bdev, iocb->ki_pos, iter, is_atomic))
 		return -EINVAL;
 
 	nr_pages = bio_iov_vecs_to_alloc(iter, BIO_MAX_VECS + 1);
@@ -373,6 +382,8 @@ static ssize_t blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 			return __blkdev_direct_IO_simple(iocb, iter, bdev,
 							nr_pages);
 		return __blkdev_direct_IO_async(iocb, iter, bdev, nr_pages);
+	} else if (is_atomic) {
+		return -EINVAL;
 	}
 	return __blkdev_direct_IO(iocb, iter, bdev, bio_max_segs(nr_pages));
 }
@@ -612,6 +623,9 @@ static int blkdev_open(struct inode *inode, struct file *filp)
 	if (!bdev)
 		return -ENXIO;
 
+	if (bdev_can_atomic_write(bdev) && filp->f_flags & O_DIRECT)
+		filp->f_mode |= FMODE_CAN_ATOMIC_WRITE;
+
 	ret = bdev_open(bdev, mode, filp->private_data, NULL, filp);
 	if (ret)
 		blkdev_put_no_open(bdev);
-- 
2.31.1