Received-SPF: pass (google.com: domain of linux-kernel+bounces-36899-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
From: John Garry <john.g.garry@oracle.com>
To: axboe@kernel.dk, kbusch@kernel.org, hch@lst.de, sagi@grimberg.me,
        jejb@linux.ibm.com, martin.petersen@oracle.com, djwong@kernel.org,
        viro@zeniv.linux.org.uk, brauner@kernel.org, dchinner@redhat.com,
        jack@suse.cz
Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-nvme@lists.infradead.org, linux-xfs@vger.kernel.org,
        linux-fsdevel@vger.kernel.org, tytso@mit.edu, jbongio@google.com,
        linux-scsi@vger.kernel.org, ming.lei@redhat.com, ojaswin@linux.ibm.com,
        bvanassche@acm.org, Alan Adamson <alan.adamson@oracle.com>,
        John Garry <john.g.garry@oracle.com>
Subject: [PATCH v3 14/15] nvme: Support atomic writes
Date: Wed, 24 Jan 2024 11:38:40 +0000
Message-Id: <20240124113841.31824-15-john.g.garry@oracle.com>
In-Reply-To: <20240124113841.31824-1-john.g.garry@oracle.com>
References: <20240124113841.31824-1-john.g.garry@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?wGRijSKK82H/jaqh3sLfp5JQxNIKUpXmZdCXXj0Dk0T7uCSxMw9W/LtNF9WM?=
 =?us-ascii?Q?KPuMYxMio5b0RJ8Mlb/pW53OVpXWYE3YXlrDOBoe+EwbGY7ffOUUTQjOQBMd?=
 =?us-ascii?Q?uZ52lZ1MC887Nr7TuEzV8gsTzuoWml9Omxsvt5CyIkJbPTwavWOlotXB+cyp?=
 =?us-ascii?Q?U478VQOLF/InqWUZL0q3QkhaeWa+/6fQN8I23VWeR56u4cKZLmimx8JNeEsK?=
 =?us-ascii?Q?oL3Hujd7I5DzVStEdNp7DRb/bX+B6DBLJ8ob1hQoV6S5mmu2Om2NJ9BfGbW0?=
 =?us-ascii?Q?MbWyhOuFraBxuC76sQYKbT+xXuzWshY9mHtAeIfpSzkUhVrLQ51FCp6vjgct?=
 =?us-ascii?Q?qoym3mPp3/csQi/X6cj1nPqMZjb1jXzSGN6h6y9qzx6BY0Ih64i+MSW6GJMB?=
 =?us-ascii?Q?QbpKirY9fo9+EdpIKQDLS2W7Om0BgI6enay8u6g/HgqbFjp+J+L2K17Gb+8c?=
 =?us-ascii?Q?ApIRGCsK9TA0jyMh9vUapYcM2pDz5P8vHuoxiC56cURL4J2GB/sbBviGMFNB?=
 =?us-ascii?Q?2wVwtAtMCt3sAyra3qMcrYifvWG4LcHgvR3ifOoPzPkZLAQWvMSoizR++8be?=
 =?us-ascii?Q?O1yFnKQLp18KuNEsYXxPex2OYCiCa8kFKaWGNKlilK7gyK/oDhF3S0dVjrxs?=
 =?us-ascii?Q?IAChRvz7RWY/yOC6cGoN7uH7Q7Mv1KhEEuuA9vJK9uiYAGNz1gJ1cvx/D5KU?=
 =?us-ascii?Q?AYD+fbwoDNpkIyb1cCsoDpyOOxm2Jq0kv2bnjLq88kNYhu+GdmW1OGpjESx3?=
 =?us-ascii?Q?Xaal4PMQnEB8dcwWb2eSw6ndLnatRRcZQ1hqFKq0W5Mq0Ob4V5P0b215zmJg?=
 =?us-ascii?Q?pt6JjMmLGOWtT1EvuW55bl9bdRtodpNNkr8f6aUsvCTatso4h8P0SZeY9yPQ?=
 =?us-ascii?Q?pqgBx8nsxRVLjaakA91H1T4vYxfrXHYDH99iC7DbqnYrEJc6QdxrDWz9wI1G?=
 =?us-ascii?Q?4MSE7sTFuIfLYzMhoGIFtvXPhKwd3WZh1j26vIbWykSG3QRHD17R8oL+7SB1?=
 =?us-ascii?Q?476KB2qQTCR5Rz8VYtBN7Xltf6gtJbbRLYhBJ2V09OlmX8kCEaNXcJ8JJvBE?=
 =?us-ascii?Q?UpKPaOM1f11drYHWYwihqz+Uo2li6+3OaiylB38s1Hey+00z5J2qUOdQjyiC?=
 =?us-ascii?Q?F5nhsIpi8XxJ5dCLPYc+1QEgvpIo0abwkVUMn1t1X1yebnBqDyExHpYwmUb9?=
 =?us-ascii?Q?o6hMluxAfjpKUJZQwWer4tIzUET+LNXks2uvd9cTdX1EzWqY7u6PnTma8Zzs?=
 =?us-ascii?Q?+iaqjYLTrRtiyrc3Kv9lD3/d65AN8whwCAGlcTy5FeCjyhNVVFvvR7dcvBJc?=
 =?us-ascii?Q?25n32qtsZHbcIEOkIrvha+NfTxiU/Do9yJQwOju4pV4AKptQbGS54HfyWj+w?=
 =?us-ascii?Q?TCB1jyPEO81KEyAAbj2Oc9pzJvYi494ft/xezzbaz9UWVSugl4Vk5RNfpN6U?=
 =?us-ascii?Q?wGuiN/23/lKsC9Dfrn66giOXEGIqoi5z8koyGv6rm6PCS+TeaRL8yXhmIogU?=
 =?us-ascii?Q?MOGEz7RQKANldpU4GuSwd47q1HDqUchUM1t7ta3Mq8bMFUylyGIVzGt0cCwy?=
 =?us-ascii?Q?PwQB07Jq8GlnwkprTW8YHsCs2ta/wsj8Kh8D18G6xksDHCQxNZ0zhKbi1RKJ?=
 =?us-ascii?Q?mQ=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	yQKTsQYE7OYh3bAbKBAxsQYptkiQL5TkL6+H2alcMY9oyzxVI02ZJ4S+arGZHTxbmvHNVhIFO0OJF83zcOyTfR/odi+y8rdRZ5787nqgNwXW474N/zRMoPhJ+GsG6ufg5wybOkArB8hHHkkYpit0EfOOKOPy1J59azrfrqCweItu+848ZYA6sSe5ePlJSuQlI3XEIFR+5+Zmdm73jww00S86HZHYkicMbGFY8YAc4kwajpQYcLoHyAzbl5FSBJrKyClGNibppDz6caMj08OtkFlYH7CqSmR4GGS12ijbTLuQNtWwKKoI+azVW1/SiCVSjslE0Hus+sCGJEDjSACAAdzp0u/Gwn3MoAW51Cq6eyUHgt+yPA74FaGM74279ogY0AqAaf7o9/QAM2NDCKecoqfsV1z3upGxUsJpCvhNOtxgNKZZ2uHpF0Pk/u37tdt0iWYZ9IxK/czMuJkHALamYQ9BYsU7OPlarhVJe1M8aOcuQjHHFgCy6ySQXV/i2RwIIqR8b6DsMqUBCSo9jXK5DYH2pvfcQpxvM4G44Hy6aDtBK2oO/rVI28kZz2YZYjYRkMkc3+exW7Pj3UMVs7hq33Dl31+P1KrvwA/7Utbiakk=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 468739b1-6f45-4f62-76b9-08dc1cd11d0c
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2024 11:39:24.1887
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NF3zqRoFschKiVEqXAijQzDHHdb7k8Tt5txIOnzarqj0qQOBB3AGf3vv4E0yY3YK4TzRafTSf66QAbk2XO4kQQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB5815
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-01-24_06,2024-01-24_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999
 spamscore=0 mlxscore=0 adultscore=0 bulkscore=0 phishscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311290000 definitions=main-2401240083
X-Proofpoint-GUID: ubca-UJOmyGm77r0jJ8vO84Sbo6DBAjt
X-Proofpoint-ORIG-GUID: ubca-UJOmyGm77r0jJ8vO84Sbo6DBAjt

From: Alan Adamson <alan.adamson@oracle.com>

Support reading atomic write registers to fill in request_queue
properties.

Use following method to calculate limits:
atomic_write_max_bytes = flp2(NAWUPF ?: AWUPF)
atomic_write_unit_min = logical_block_size
atomic_write_unit_max = flp2(NAWUPF ?: AWUPF)
atomic_write_boundary = NABSPF

Signed-off-by: Alan Adamson <alan.adamson@oracle.com>
#jpg: some rewrite
Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/nvme/host/core.c | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 85ab0fcf9e88..5045c84f2516 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1911,6 +1911,44 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
 	blk_queue_write_cache(q, vwc, vwc);
 }
 
+static void nvme_update_atomic_write_disk_info(struct gendisk *disk,
+		struct nvme_ctrl *ctrl, struct nvme_id_ns *id, u32 bs, u32 atomic_bs)
+{
+	unsigned int unit_min = 0, unit_max = 0, boundary = 0, max_bytes = 0;
+	struct request_queue *q = disk->queue;
+
+	if (id->nsfeat & NVME_NS_FEAT_ATOMICS && id->nawupf) {
+		if (le16_to_cpu(id->nabspf))
+			boundary = (le16_to_cpu(id->nabspf) + 1) * bs;
+
+		/*
+		 * The boundary size just needs to be a multiple of unit_max
+		 * (and not necessarily a power-of-2), so this could be relaxed
+		 * in the block layer in future.
+		 * Furthermore, if needed, unit_max could be reduced so that the
+		 * boundary size was compliant - but don't support yet.
+		 */
+		if (!boundary || is_power_of_2(boundary)) {
+			max_bytes = atomic_bs;
+			unit_min = bs;
+			unit_max = rounddown_pow_of_two(atomic_bs);
+		} else {
+			dev_notice(ctrl->device, "Unsupported atomic write boundary (%d)\n",
+				boundary);
+			boundary = 0;
+		}
+	} else if (ctrl->subsys->awupf) {
+		max_bytes = atomic_bs;
+		unit_min = bs;
+		unit_max = rounddown_pow_of_two(atomic_bs);
+	}
+
+	blk_queue_atomic_write_max_bytes(q, max_bytes);
+	blk_queue_atomic_write_unit_min_sectors(q, unit_min >> SECTOR_SHIFT);
+	blk_queue_atomic_write_unit_max_sectors(q, unit_max >> SECTOR_SHIFT);
+	blk_queue_atomic_write_boundary_bytes(q, boundary);
+}
+
 static void nvme_update_disk_info(struct nvme_ctrl *ctrl, struct gendisk *disk,
 		struct nvme_ns_head *head, struct nvme_id_ns *id)
 {
@@ -1941,6 +1979,8 @@ static void nvme_update_disk_info(struct nvme_ctrl *ctrl, struct gendisk *disk,
 			atomic_bs = (1 + le16_to_cpu(id->nawupf)) * bs;
 		else
 			atomic_bs = (1 + ctrl->subsys->awupf) * bs;
+
+		nvme_update_atomic_write_disk_info(disk, ctrl, id, bs, atomic_bs);
 	}
 
 	if (id->nsfeat & NVME_NS_FEAT_IO_OPT) {
-- 
2.31.1