Received-SPF: pass (google.com: domain of linux-kernel+bounces-82082-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
From: John Garry <john.g.garry@oracle.com>
To: axboe@kernel.dk, kbusch@kernel.org, hch@lst.de, sagi@grimberg.me,
        jejb@linux.ibm.com, martin.petersen@oracle.com, djwong@kernel.org,
        viro@zeniv.linux.org.uk, brauner@kernel.org, dchinner@redhat.com,
        jack@suse.cz
Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org,
        tytso@mit.edu, jbongio@google.com, linux-scsi@vger.kernel.org,
        ojaswin@linux.ibm.com, linux-aio@kvack.org,
        linux-btrfs@vger.kernel.org, io-uring@vger.kernel.org,
        nilay@linux.ibm.com, ritesh.list@gmail.com,
        John Garry <john.g.garry@oracle.com>
Subject: [PATCH v5 07/10] block: Add fops atomic write support
Date: Mon, 26 Feb 2024 17:36:09 +0000
Message-Id: <20240226173612.1478858-8-john.g.garry@oracle.com>
In-Reply-To: <20240226173612.1478858-1-john.g.garry@oracle.com>
References: <20240226173612.1478858-1-john.g.garry@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?G6Bckomjl1VSue9K58ds8UDuGDUGerJu0Cm7KsEhtTZfakXSoUU0ljymPyAo?=
 =?us-ascii?Q?4y0yaphY2yfN+a9inTdgbdsFQqhnR0EknkoLO02LWt4C3yVA2vF9dXeCq19l?=
 =?us-ascii?Q?KCDQ6Ucr0iHWawLxIuyx+gnz5eaGVzs9YGuhyRKMEOW/8hzd253URDGvRpFN?=
 =?us-ascii?Q?oFYTG+fOuNavlJDj/YRab6baCY2VBbWBIEOb1AP81jkm16tRxN5sopRSEez2?=
 =?us-ascii?Q?63b6ihWLIsZbWgpK0RFLT+ARQvdCcWM1z+qC9RsuCJQFlRe/iRyFHuOpP9Xd?=
 =?us-ascii?Q?v7rIacq5EpQvUK3ocX14Tire/6uRWhpXzuMcHKPCa0SG6Gej0hb4XKGMPuLq?=
 =?us-ascii?Q?/E5bJ/ejP11CWoK1G3/yEtaBUB7IhIu3CHgIUp25df+PzbAlJAko7P12OQ7Z?=
 =?us-ascii?Q?sbyFAChnGbyoyAZR1o0dmVTIdkJsaUUfHaJo29ojZ5/jXoGEYS+gSObum6If?=
 =?us-ascii?Q?bWEom9qVajFtE+m06jvHIUj35+ixS7dcQlVp14RFCv0zhxMR9Un2IsKXmO11?=
 =?us-ascii?Q?moOBPfBNo7ab/2llM2OnFo5fWmULNmRcfhP1yZeabiwel5xozCAoYGdy3iM4?=
 =?us-ascii?Q?XKufLKyftRBOj4Vz9Me1Ffb+KtnmnljEnEoFTLreNPTr3vKpIHz4vq4HA/bD?=
 =?us-ascii?Q?+Y/fSD/PINMbvkhYQC8Kir5K3PF0S+AcamfNfosL3FoF79O3D1J7qLArNQuX?=
 =?us-ascii?Q?UMkiO37C+9K49XeLdklJdrYhEHuokYsHDzq0LYV2e2/axN1bSCroEVKKCfgG?=
 =?us-ascii?Q?8qGlXrebmzhXn0Hl3LEZTIXmq1sNnrXLd1WVmOSzg1VdNvjVCmUshF7tBl5G?=
 =?us-ascii?Q?4CGvuIkhwD15BsoqxPJrcd4vCm/7Ss4ELcBIzBgxrnAvYoEYt7+ak7Dbej0w?=
 =?us-ascii?Q?yWIisDpDVepfAHgJOGw6Bgg+2Yun4Cdl9bJYII3mblQX4yGjtrBXMY09BLYM?=
 =?us-ascii?Q?eYwMq8LEgnBSbnPYbG6kvaltTQe34sIy5aH4SAvyM70Mt/14nNl+0WUthVe5?=
 =?us-ascii?Q?owKPuxpid8s+/vvir+DwV9F8tavYej1X0hlvu6n3FjOR54WaUUGGwyYQTUFK?=
 =?us-ascii?Q?zlJetdwqALmJb8BBNxR3PJlthLxFnqWNFUfP8r65ELKC8Foe6Jo00AUk88ib?=
 =?us-ascii?Q?Hez8hX851m+kidSFuqFoCYkhxrOGdz5ddNYvLDaPe9mNzU7eOhTqaXilmQzk?=
 =?us-ascii?Q?HlZboehM3Dl/wFSauukyWmJdpJT7bwP3qe6TZOtXJbFzArYmOnBJjVWmZBMP?=
 =?us-ascii?Q?rnLcBdDAbtt9zq51Z1k85oHyAXsjfdd20h4/OR5ww/n3MAXpnkZApY9HaZHN?=
 =?us-ascii?Q?+r1buLOOXaffSan9xSopRA6X/cUzAUKzXn61dnz6593YQhmM/v9gr7IiD1Vj?=
 =?us-ascii?Q?wvBlsEPSdT4emDjeY/nD7MBWuU8AbcY5pgEqmmiCS1zto6+gpjvnI+gl6PDE?=
 =?us-ascii?Q?N23D2x12dOMoaUe4/gisLBWBmib3yA495i5RdQZTDHR5hwY7Lszj20gD8ZJd?=
 =?us-ascii?Q?PqNxFGMzLyn8OYhONsTvgVWtL5W8X+dMYwLGirsc1BdijNPwRWF8jsLBXDCD?=
 =?us-ascii?Q?BQdluq/nkC5KfdPfQP45rVM4C/BUg/MgoRt+3zn7jP5mmQpvbovQTn3uD05s?=
 =?us-ascii?Q?9w=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	5QMyxsdah2A8w2XSw4K5BOjIVi9nckHxGC0hv+C6++DucAJUYB9Kn9GOiAf6UYDOzKQ7I8N+SiP4yG8/6DNEIP11LE7J4gaGZvCXL7d6eVWwSCsQFReyg6pk5dieI8JHZb26czBH7GiiKc8ltS1xmGBSWKcOPNBzN/C0KIHUTYt84ESvwZISp9hfLUYgBShYz8WuCEFyJVQiy8XOSohsgP8jBwvDJewM4vTS0RmyGN4VMjwhXlKn11cWDHPqDHyzTVnQahfSn0dWfFePCeIHFGZ6YaPYlNAq+ZGur+DgCUPWQeKLCVIv77koDZXAbmkMw0EPX7DQmoiyFdBCrBs0BIs2rJnQ0L9AyV3fBLdXFlmemfkrIkpnwbtYMrUDgdg6PWwaywqo5tzw0j6qooXUVi5Swr+gk3YBe8lyJS1fh1CX9brwGLvgHBi5G+cit7aXCEJvN3lPeIyPoJ5Z1ZdUlHc+CrhWnkoYfb3nsz5V45e3RucTaaEQ9WVTioGYXfQL2A6SzzxZXIgpsju6utjyylfFqQ9uJ2fkn8lY1onCHUEvAXp6V2pB+oihcdQ0MpkaiMdJfZZS6QG18w1ut8jf/SyBeO0VBGl9yQjze9gu6AY=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b386c6c3-3a08-4d2e-f100-08dc36f18599
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 17:36:53.7584
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: cRk86lElwN+8uUtT/tmWJQsEw7SnAzMEaYcs3BCtkDdtEE5V6zf7QRsnR3+PjENKbe6OgbbPhZiBguobJOR6ug==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR10MB6298
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-26_11,2024-02-26_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 bulkscore=0
 spamscore=0 mlxlogscore=999 phishscore=0 suspectscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000
 definitions=main-2402260134
X-Proofpoint-GUID: IPQHUXO9G5UNkW2KRXn3jgxV-Jw6EgWG
X-Proofpoint-ORIG-GUID: IPQHUXO9G5UNkW2KRXn3jgxV-Jw6EgWG

Support atomic writes by submitting a single BIO with the REQ_ATOMIC set.

It must be ensured that the atomic write adheres to its rules, like
naturally aligned offset, so call blkdev_dio_invalid() ->
blkdev_atomic_write_valid() [with renaming blkdev_dio_unaligned() to
blkdev_dio_invalid()] for this purpose.

In blkdev_direct_IO(), if the nr_pages exceeds BIO_MAX_VECS, then we cannot
produce a single BIO, so error in this case.

Finally set FMODE_CAN_ATOMIC_WRITE when the bdev can support atomic writes
and the associated file flag is for O_DIRECT.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 block/fops.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/block/fops.c b/block/fops.c
index 42955b6a1f5e..3f61b00994d3 100644
--- a/block/fops.c
+++ b/block/fops.c
@@ -34,9 +34,22 @@ static blk_opf_t dio_bio_write_op(struct kiocb *iocb)
 	return opf;
 }
 
-static bool blkdev_dio_unaligned(struct block_device *bdev, loff_t pos,
-			      struct iov_iter *iter)
+static bool blkdev_atomic_write_valid(struct block_device *bdev, loff_t pos,
+				      struct iov_iter *iter)
 {
+	struct request_queue *q = bdev_get_queue(bdev);
+	unsigned int min_bytes = queue_atomic_write_unit_min_bytes(q);
+	unsigned int max_bytes = queue_atomic_write_unit_max_bytes(q);
+
+	return generic_atomic_write_valid(pos, iter, min_bytes, max_bytes);
+}
+
+static bool blkdev_dio_invalid(struct block_device *bdev, loff_t pos,
+				struct iov_iter *iter, bool is_atomic)
+{
+	if (is_atomic && !blkdev_atomic_write_valid(bdev, pos, iter))
+		return true;
+
 	return pos & (bdev_logical_block_size(bdev) - 1) ||
 		!bdev_iter_is_aligned(bdev, iter);
 }
@@ -71,6 +84,8 @@ static ssize_t __blkdev_direct_IO_simple(struct kiocb *iocb,
 	}
 	bio.bi_iter.bi_sector = pos >> SECTOR_SHIFT;
 	bio.bi_ioprio = iocb->ki_ioprio;
+	if (iocb->ki_flags & IOCB_ATOMIC)
+		bio.bi_opf |= REQ_ATOMIC;
 
 	ret = bio_iov_iter_get_pages(&bio, iter);
 	if (unlikely(ret))
@@ -340,6 +355,9 @@ static ssize_t __blkdev_direct_IO_async(struct kiocb *iocb,
 		task_io_account_write(bio->bi_iter.bi_size);
 	}
 
+	if (iocb->ki_flags & IOCB_ATOMIC)
+		bio->bi_opf |= REQ_ATOMIC;
+
 	if (iocb->ki_flags & IOCB_NOWAIT)
 		bio->bi_opf |= REQ_NOWAIT;
 
@@ -356,12 +374,13 @@ static ssize_t __blkdev_direct_IO_async(struct kiocb *iocb,
 static ssize_t blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 {
 	struct block_device *bdev = I_BDEV(iocb->ki_filp->f_mapping->host);
+	bool is_atomic = iocb->ki_flags & IOCB_ATOMIC;
 	unsigned int nr_pages;
 
 	if (!iov_iter_count(iter))
 		return 0;
 
-	if (blkdev_dio_unaligned(bdev, iocb->ki_pos, iter))
+	if (blkdev_dio_invalid(bdev, iocb->ki_pos, iter, is_atomic))
 		return -EINVAL;
 
 	nr_pages = bio_iov_vecs_to_alloc(iter, BIO_MAX_VECS + 1);
@@ -370,6 +389,8 @@ static ssize_t blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 			return __blkdev_direct_IO_simple(iocb, iter, bdev,
 							nr_pages);
 		return __blkdev_direct_IO_async(iocb, iter, bdev, nr_pages);
+	} else if (is_atomic) {
+		return -EINVAL;
 	}
 	return __blkdev_direct_IO(iocb, iter, bdev, bio_max_segs(nr_pages));
 }
@@ -615,6 +636,9 @@ static int blkdev_open(struct inode *inode, struct file *filp)
 	if (bdev_nowait(handle->bdev))
 		filp->f_mode |= FMODE_NOWAIT;
 
+	if (bdev_can_atomic_write(handle->bdev) && filp->f_flags & O_DIRECT)
+		filp->f_mode |= FMODE_CAN_ATOMIC_WRITE;
+
 	filp->f_mapping = handle->bdev->bd_inode->i_mapping;
 	filp->f_wb_err = filemap_sample_wb_err(filp->f_mapping);
 	filp->private_data = handle;
-- 
2.31.1