Received-SPF: pass (google.com: domain of linux-kernel+bounces-36900-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
From: John Garry <john.g.garry@oracle.com>
To: axboe@kernel.dk, kbusch@kernel.org, hch@lst.de, sagi@grimberg.me,
        jejb@linux.ibm.com, martin.petersen@oracle.com, djwong@kernel.org,
        viro@zeniv.linux.org.uk, brauner@kernel.org, dchinner@redhat.com,
        jack@suse.cz
Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-nvme@lists.infradead.org, linux-xfs@vger.kernel.org,
        linux-fsdevel@vger.kernel.org, tytso@mit.edu, jbongio@google.com,
        linux-scsi@vger.kernel.org, ming.lei@redhat.com, ojaswin@linux.ibm.com,
        bvanassche@acm.org, Alan Adamson <alan.adamson@oracle.com>,
        John Garry <john.g.garry@oracle.com>
Subject: [PATCH v3 15/15] nvme: Ensure atomic writes will be executed atomically
Date: Wed, 24 Jan 2024 11:38:41 +0000
Message-Id: <20240124113841.31824-16-john.g.garry@oracle.com>
In-Reply-To: <20240124113841.31824-1-john.g.garry@oracle.com>
References: <20240124113841.31824-1-john.g.garry@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?xlVzclSmEd/I9JJVQpehK/TPLXPRe6AhqBPfrkGp7ELdP7AI+LssrU3p+sAY?=
 =?us-ascii?Q?6xz5/OfD2UXsETPcWg3XHEysXKcrOzI3BUh+opC/fxssvbQAsgVZbBSxTPKt?=
 =?us-ascii?Q?cYIc+dKRt1N7Ob6n4MDvVmDvW602k7K6QK48XGg/jA90h+u+bAUjqljK3K1M?=
 =?us-ascii?Q?hk3BJ1SlFX0Kj9WVYxPMpoaykOmGsodvQM780b+59kWVx656GYXQ05LEAaGV?=
 =?us-ascii?Q?F74RWrRVIHHM8TXBq4XI3canKpJBEXpnXMG/VV1BJ5BPvnRfKcCRF/Bj16gW?=
 =?us-ascii?Q?VhXoViJpzGeWINmtn/EEpDRd6niF2NkbNrsLaqSMnmc0SzZVLUmOBFiaUvNw?=
 =?us-ascii?Q?u+RK131+UK7oCyc0CI91+ZdAuV9oyXQbVbGRFfeAR1ft/ELLXPNRk8TqVpjU?=
 =?us-ascii?Q?gwN83vepzP9G2o7GZtVBJQfZ7MBavSMOOyatNcFjXNuPZnjQ/oa3HTcrM94b?=
 =?us-ascii?Q?PIKpYiNBFK3hA/irltfmYsOgWCZsrIK7I0CNPClvtAbgaDs7sqpywy/rmsZb?=
 =?us-ascii?Q?rdKMEjGA0GakE9qirvx+FujZpBk7+Lcas3uT+eECqcDm14KS0TL+7aEziJIB?=
 =?us-ascii?Q?58N+spuQFGh/MgCGsHAj2PvP3vPMrZh7DpKVxYOT+stmxYSf/km+VvHphq5M?=
 =?us-ascii?Q?eH7fDfkZDQT2Yem8gmUY7HCJSqgV1dsLasMx7pOMqmYcBULNQVIFBoGRiAnB?=
 =?us-ascii?Q?UwIHLZw9vqPHDLnRjRN4QoeOMEJ2GaMGGm6oZ6zarMup7zrVZtDSSXGcfso8?=
 =?us-ascii?Q?RGE68awEgHlKV6JeZytXkeS71F7eTQIuHHx+6maLetub9M83SIbkjYiQfL16?=
 =?us-ascii?Q?kfehpxObtlDRwUISklJtAjJzZaHpqnBT56TgrVLdhUteNShbAmz/KQijyPiC?=
 =?us-ascii?Q?zeJqMPFzUzKipruo5EG0w8i51hzQY/KP7bGcrnTRoXHLCk36EXRnEnDNihWc?=
 =?us-ascii?Q?EsAJhNHpld8BKl94wuO9ACX4fcyZGtBPOzXvp493ezzJEwgEuJY6eCnhmzL7?=
 =?us-ascii?Q?AdVcfXEqgu1G3Gz/No08W7ypyChjf4zUWReM3BSdnkoomO5+NsvBMx29jdAQ?=
 =?us-ascii?Q?BTMjHLjhx3bWoTlXp0adZyPop6ouCQ2jXJUx5w5RZmALzyHTImiQbdpls7Rv?=
 =?us-ascii?Q?gmjkr8kk2Trj24EA7+YhJm62c+6OVFKLrL8EvH1J0R81xrj/d32AymrjIwif?=
 =?us-ascii?Q?AeoAjz6AVt2wPSmQ3VsoLzDeCjPsJRt3jTqC04o0h6V64k0fKnJYp0cwpiPe?=
 =?us-ascii?Q?4nH5hA3K6sbgJ83s6t4h2uE3YE6v5rSjJVFxnB0Lq8IDEzmQzPRojuFhkz9K?=
 =?us-ascii?Q?6nZ3BxaERASzZ2din7fX+p2VVLaJBoarQaQh0MCmCe+WujMXvvA2mgwPmRrD?=
 =?us-ascii?Q?I9PDPqcak2zbX82H1RxK8exMCJ8UJh+yce9OkNnLtX94/HruQXy+HfiJKyGQ?=
 =?us-ascii?Q?mJJ8vd3oVdwreDJOOSg93vIU5nHktSpfml44zW7C6rJRp61D2qn7SAp6TpZa?=
 =?us-ascii?Q?LZtIV/OF6j58FSEIqfL1wr1Qky6VghDc8ZseZfSlYpcKcAG7QkZKP0j2iZAw?=
 =?us-ascii?Q?IIJNoXGp+xwUq/0v53372BsAhTmAqp3EB1uk6OSkvDC+bVJwSiwf+Z9oRaDK?=
 =?us-ascii?Q?Lw=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	r7yZag3rEHUmOtbqfkxkHLm9kNJXIZEgdgzGTjBeFoPxqyUQddQ6pm1+GUTdEL5HA+yXBp7qibameQCQo2mWj6vqAFy/B85/cIkCQe2tNLCzRE0FRElxb88YP7x9GVTwu3gOQcv42KtN3AEWbqaxIbUszUGCS4ZMnegcHkAYGWpgz0GdPraOzvXjSh82REgZiPyUVdXjGcWvld0f4eJ0DrRuo7XpK30YqLMWRBWf7zQUqvaORdWWMasFk7Ba/3hJcRIvQgnzDIWYChQp5w+lpssVpD7fWn0ElrrooKk7gRM0/eToPd4Vvhal44l44VimbE1RQpjlKpBYUH5MbqGvqQNjgQH9A2cmc5vm9bWhCOsoBPU1ZmsuFgfIrSEWGVgFe55c9Si7pj48NZU5xLJH2WWGkltKhNJ7ag7QL6P3Dz8NEP/2TtvPSavZyNmvQMDka4taK/6jtTn6U2SzRkBVsweCD0VMUaxyJ1HLUcYdLiy7tvXBtsU1Y9CtAsX5AqFCTBpCN2nfsQbb6hTMolCo1AaMAKbdfnl6/cUKCk/jWhBpm5sB+JB/x9YycVIci0Fyqk8DqCRvq0Xz3iN5xki6Pvzmz1qCpSzkaHhiZaE5aoQ=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 89741e9d-331c-4fcf-7a86-08dc1cd11dc0
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2024 11:39:25.3772
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jw9e2LBsM2sv3NOisXuB6vyxVlQ3NIZtRqBDZKVIshQQxtl2h9brjqZezMdh28OkWc7jm/Rkk0228ADq+HrivA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR10MB6719
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-01-24_06,2024-01-24_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 suspectscore=0
 phishscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 spamscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311290000 definitions=main-2401240083
X-Proofpoint-ORIG-GUID: vU08l0Vlv9LhDEsWmP3MUho7CIcTXHLM
X-Proofpoint-GUID: vU08l0Vlv9LhDEsWmP3MUho7CIcTXHLM

From: Alan Adamson <alan.adamson@oracle.com>

There is no dedicated NVMe atomic write command (which may error for a
command which exceeds the controller atomic write limits).

As an insurance policy against the block layer sending requests which
cannot be executed atomically, add a check in the queue path.

Signed-off-by: Alan Adamson <alan.adamson@oracle.com>
Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/nvme/host/core.c | 35 +++++++++++++++++++++++++++++++++--
 drivers/nvme/host/nvme.h |  2 ++
 2 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 5045c84f2516..6a34a5d92088 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -911,6 +911,32 @@ static inline blk_status_t nvme_setup_rw(struct nvme_ns *ns,
 	if (req->cmd_flags & REQ_RAHEAD)
 		dsmgmt |= NVME_RW_DSM_FREQ_PREFETCH;
 
+	/*
+	 * Ensure that nothing has been sent which cannot be executed
+	 * atomically.
+	 */
+	if (req->cmd_flags & REQ_ATOMIC) {
+		struct nvme_ns_head *head = ns->head;
+		u32 boundary_bytes = head->atomic_boundary;
+
+		if (blk_rq_bytes(req) > ns->head->atomic_max)
+			return BLK_STS_IOERR;
+
+		if (boundary_bytes) {
+			u32 mask = boundary_bytes - 1, imask = ~mask;
+			u32 start = blk_rq_pos(req) << SECTOR_SHIFT;
+			u32 end = start + blk_rq_bytes(req);
+
+			if (blk_rq_bytes(req) > boundary_bytes)
+				return BLK_STS_IOERR;
+
+			if (((start & imask) != (end & imask)) &&
+			    (end & mask)) {
+				return BLK_STS_IOERR;
+			}
+		}
+	}
+
 	cmnd->rw.opcode = op;
 	cmnd->rw.flags = 0;
 	cmnd->rw.nsid = cpu_to_le32(ns->head->ns_id);
@@ -1912,7 +1938,8 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
 }
 
 static void nvme_update_atomic_write_disk_info(struct gendisk *disk,
-		struct nvme_ctrl *ctrl, struct nvme_id_ns *id, u32 bs, u32 atomic_bs)
+		struct nvme_ctrl *ctrl, struct nvme_ns_head *head,
+		struct nvme_id_ns *id, u32 bs, u32 atomic_bs)
 {
 	unsigned int unit_min = 0, unit_max = 0, boundary = 0, max_bytes = 0;
 	struct request_queue *q = disk->queue;
@@ -1943,6 +1970,9 @@ static void nvme_update_atomic_write_disk_info(struct gendisk *disk,
 		unit_max = rounddown_pow_of_two(atomic_bs);
 	}
 
+	head->atomic_max = max_bytes;
+	head->atomic_boundary = boundary;
+
 	blk_queue_atomic_write_max_bytes(q, max_bytes);
 	blk_queue_atomic_write_unit_min_sectors(q, unit_min >> SECTOR_SHIFT);
 	blk_queue_atomic_write_unit_max_sectors(q, unit_max >> SECTOR_SHIFT);
@@ -1980,7 +2010,8 @@ static void nvme_update_disk_info(struct nvme_ctrl *ctrl, struct gendisk *disk,
 		else
 			atomic_bs = (1 + ctrl->subsys->awupf) * bs;
 
-		nvme_update_atomic_write_disk_info(disk, ctrl, id, bs, atomic_bs);
+		nvme_update_atomic_write_disk_info(disk, ctrl, head, id,
+						   bs, atomic_bs);
 	}
 
 	if (id->nsfeat & NVME_NS_FEAT_IO_OPT) {
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 030c80818240..938060c85e6f 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -476,6 +476,8 @@ struct nvme_ns_head {
 	struct device		cdev_device;
 
 	struct gendisk		*disk;
+	u32 atomic_max;
+	u32 atomic_boundary;
 #ifdef CONFIG_NVME_MULTIPATH
 	struct bio_list		requeue_list;
 	spinlock_t		requeue_lock;
-- 
2.31.1