Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32;
From:   John Garry <john.g.garry@oracle.com>
To:     axboe@kernel.dk, kbusch@kernel.org, hch@lst.de, sagi@grimberg.me,
        jejb@linux.ibm.com, martin.petersen@oracle.com, djwong@kernel.org,
        viro@zeniv.linux.org.uk, brauner@kernel.org,
        chandan.babu@oracle.com, dchinner@redhat.com
Cc:     linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-nvme@lists.infradead.org, linux-xfs@vger.kernel.org,
        linux-fsdevel@vger.kernel.org, tytso@mit.edu, jbongio@google.com,
        linux-api@vger.kernel.org, Alan Adamson <alan.adamson@oracle.com>,
        John Garry <john.g.garry@oracle.com>
Subject: [PATCH 21/21] nvme: Support atomic writes
Date:   Fri, 29 Sep 2023 10:27:26 +0000
Message-Id: <20230929102726.2985188-22-john.g.garry@oracle.com>
In-Reply-To: <20230929102726.2985188-1-john.g.garry@oracle.com>
References: <20230929102726.2985188-1-john.g.garry@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?gLShOUaokrHyb6kSex/57BjOP6j5XHeHfRsmuu0WxZJeoRYYVMymYOkPLnRl?=
 =?us-ascii?Q?ZmHR550EBKG8ce1REiVRHClHQaR77YbY6/YRjD3WSusIRFAxSZTVx3mZBa7d?=
 =?us-ascii?Q?Ukjnw2bWYhPcWe2PQ53tRS68a7y90YQ9Gby8ug1nG8kAMxMnHKM6xbiWnm41?=
 =?us-ascii?Q?tfaHzjBq3dsFIFLdvhgfgHzBO645TvUIoRjXEB2QaNYiBkAhB1yZLj0SwLc/?=
 =?us-ascii?Q?YBV4XOnWNb6Fm983JXZid9FBUS1mIeNadnNVluLnUraIk6VCApiV97wRdrcW?=
 =?us-ascii?Q?Bgf7AqaRZISNuwCuLUuhrkNXeWBVbHevngTOV/WF8vssM79mNSkye/HF/sMF?=
 =?us-ascii?Q?Fsrci5BDjUCANKTypX2b1m/UuN/q1nhWl58YTNKcYJaUeJsK1/GYEP1pQTNt?=
 =?us-ascii?Q?FwtaHkYaSyQbF50yBckLsfVsut9Irh2rhLFu0PauwzuRQ+cFyPc4r9ZGQzdO?=
 =?us-ascii?Q?2osPJ4InG4Sky7ekj7i66SgLN3nOpXH6tbu3tfmTPi1IkTvx//0rWrhy7Vx6?=
 =?us-ascii?Q?z+ymLWeJZXQKIMvL/ylW2uCLD1vJ0ETcNCNbRMi0QJ4wA9dRD+x1RaBTkEUi?=
 =?us-ascii?Q?Q6I8KTxa0Ieu9SCRg8qaoC3vDYIeEujayM+35PIsZTBHbZGLIchWlECYkTTI?=
 =?us-ascii?Q?EMQfsQKrKxGExLmI9Xt2u9T/mWyEsdvvYsWlXj4JMGQ107rfXLMC1C/oZtud?=
 =?us-ascii?Q?wYYN6vDwwOUP5lGPo8zmfsja9T4v0MMCtMqLg+j47R7TenSKusnr/NoswW8n?=
 =?us-ascii?Q?mKaqzdXlxaUdtjVXAXUAlt17ITeLd85IDS/2JinsWbKo2M99tmWLcWmcaqmK?=
 =?us-ascii?Q?guiFsRNoR+cXn4s5xtbce+AjhAIPeqw6wjzmqOs6RQoDz0TynTqfH13zZZyT?=
 =?us-ascii?Q?GWUxXowrQvBomTyinUYo5QLCILRGs3XQuVRumLG3Vswn9zI8lDLtdO8gmpLR?=
 =?us-ascii?Q?wT1b+DeNAE7No7lphwgXczpH6nmkLNGS3bhGafHWFyOW1pNoz6klVQsdTJCB?=
 =?us-ascii?Q?izYuKEVhFZL1UVnrf6DgQ6cEOsoU0p3YA6vj5cqVgBUkt+mt3S6glVz8OdVt?=
 =?us-ascii?Q?9smpeIvuFlcBrhULVFjnAzglcVVs13/v9bGQ4Bj845sifxo+/QMUVXvu+C8v?=
 =?us-ascii?Q?Hyz9yNnhQiORE1A2RZhNv+jwcHliaNQB1uWGOWoEkRzOYxNyM7Y5P76oXl7g?=
 =?us-ascii?Q?aT/J0MyUfLSP4+6VrbwBQSw6au52J3SunsRBa+SJGSv3Ksh64tiPYYh0d9RA?=
 =?us-ascii?Q?7TrNNAkMcf7tfcA5gidR43C8EDbBZlTvizsqEJy+p0RzGbqzJmBzT5uTIXYk?=
 =?us-ascii?Q?dfti2ODq77X0BG9ewmJoHEYlnCSQztFA0I97z1vVKLTw/AiC9rjbeEx394sJ?=
 =?us-ascii?Q?N0Q/GpNM/n8CA2UjQZqudPJoUtikHmQssHgx/7LOkxpSHs5RFFUQYsPyE3eD?=
 =?us-ascii?Q?AEitaY31Mskdfa8C/ZAwZotCL6sX6Ww0oARWfCGTUL4Cgpks1Tg2BmETGtht?=
 =?us-ascii?Q?y623wc4MFpQgRu9OXarisT9jxGYaFllWUpVQ6KOq5nlmkd87y4wmPZ5mYhz7?=
 =?us-ascii?Q?+9LR2GxM1e0vYYNpJ4Qv/rwxPbQ8jJZ8fR85DIQwBs2YBRSzvoia/4YPu7Eo?=
 =?us-ascii?Q?5A=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?us-ascii?Q?3LrikrejUN52TV7nnoH7dfcSWvNxiKA2WZuX6bQhmEC5EIE6IQId6AbBTMY/?=
 =?us-ascii?Q?dVkowE/xoXhBqyF/+/E4R0/YwZLeyukr6Vn/CNQNHUcm5XDgQvqGKekSAbwa?=
 =?us-ascii?Q?4q54NrlId1sm6MYUqib4c+TxwmfM4QTS7ui6LkBWFHATYHSHlDnqKT7kwyrw?=
 =?us-ascii?Q?MNApGSSuR3zZsF7XSO77fJUsxJIyN7QX2sKPJOnf+RR+ZPYAlVFlYhn0qU6N?=
 =?us-ascii?Q?nfB6mAv0HUFby7UPcnEDiy3Ga9fvLGseGUfg0lv+XJay84Bu2dnIHZB/8JWz?=
 =?us-ascii?Q?NxVi4i+IwuYw9goDVT2dCFk+JklVMyJcpLm2V/rZ0wtFUoRnRqYs8Xiu0zQB?=
 =?us-ascii?Q?o6qRAxtw0agxTwKd6RVrIebBR8y5KhJ8ggGXl+iQPfltrOjgPreEM6hMhVG4?=
 =?us-ascii?Q?pZTCY+z/DUHvh63jVRqSGyRtnRRe+MdTVohXvRdy+HB+RPn+sweZ5qbAS/3l?=
 =?us-ascii?Q?y50VbauF9krcOUWnDKYdrU21VKZh4JPQxLHyCZqS3chodCgHw63tqwk36I+x?=
 =?us-ascii?Q?aEjke/Pt5OWywbM+s04NUwMGST8iO6aQjUXXM4+htZ6ayDyF0WalEH+ZUBrv?=
 =?us-ascii?Q?hfR8ewhXiBm4ZmfSY/JtxzjZ2mov5JYXyMxT4eEI5PdUYetZrLVs5iMsoQug?=
 =?us-ascii?Q?zu87mosUDnM+F4IkNBmbqucmd7wn5oQkZFteQARKDOpBKYjNGzyu0Q5a6Wky?=
 =?us-ascii?Q?5l4gXyFr9wcWFmdATuNVPzdVUTgX8Wk2pft02yu2ctiyrfy94dXA0WVGf+uj?=
 =?us-ascii?Q?ke3yMNIQKuSKAXy9Ixnx1DqPu+oglsjsWzL8cuS6csI+y5A6o7Ay//9ZvqpJ?=
 =?us-ascii?Q?oI9rhC6PFOeCGQHXD1wPxs45KkkdxzwE3AjSk0VAVwpZwqgWgsQgIYRPOnP3?=
 =?us-ascii?Q?QDDkg1bnJPUOOLUdMnZLvHC6hLCcqdw0vva/BfiRof1IYKIdJK2UXIWD085W?=
 =?us-ascii?Q?BqdsTuu3lmEn2AcG91dhFpR0iVI1BiMednpLKaPbZx5eYvm8pxPiDIAEBTbF?=
 =?us-ascii?Q?0pit3/QIF/qapqvwn8uJorExfYysNqcF4lzQnW+XAkCp8ideOf7rgmdv5pRh?=
 =?us-ascii?Q?J8qLVMFk/5GVsZT9XgdniNn3b7/3EA=3D=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 76345577-4b6a-418a-a0b6-08dbc0d6d8fa
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2023 10:28:40.0181
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bLkhpuaDIZ9uacDVITn7BPz+d8Zvy7f4s560+aFK6nhSAEKR76IplmCmd/+Q8rLnaBlm0jErXMt1wP1uhMkGZw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR10MB4680
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2023-09-29_08,2023-09-28_03,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 mlxscore=0
 mlxlogscore=999 suspectscore=0 phishscore=0 adultscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2309180000
 definitions=main-2309290090
X-Proofpoint-ORIG-GUID: A2RTXmWZQqdEnpftcOxP7cHO2BnvgRXu
X-Proofpoint-GUID: A2RTXmWZQqdEnpftcOxP7cHO2BnvgRXu
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Fri, 29 Sep 2023 03:34:13 -0700 (PDT)

From: Alan Adamson <alan.adamson@oracle.com>

Support reading atomic write registers to fill in request_queue
properties.

Use following method to calculate limits:
atomic_write_max_bytes = flp2(NAWUPF ?: AWUPF)
atomic_write_unit_min = logical_block_size
atomic_write_unit_max = flp2(NAWUPF ?: AWUPF)
atomic_write_boundary = NABSPF

Signed-off-by: Alan Adamson <alan.adamson@oracle.com>
Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 drivers/nvme/host/core.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 21783aa2ee8e..aa0daacf4d7c 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1926,6 +1926,35 @@ static void nvme_update_disk_info(struct gendisk *disk,
 	blk_queue_io_min(disk->queue, phys_bs);
 	blk_queue_io_opt(disk->queue, io_opt);
 
+	atomic_bs = rounddown_pow_of_two(atomic_bs);
+	if (id->nsfeat & NVME_NS_FEAT_ATOMICS && id->nawupf) {
+		if (id->nabo) {
+			dev_err(ns->ctrl->device, "Support atomic NABO=%x\n",
+				id->nabo);
+		} else {
+			u32 boundary = 0;
+
+			if (le16_to_cpu(id->nabspf))
+				boundary = (le16_to_cpu(id->nabspf) + 1) * bs;
+
+			if (is_power_of_2(boundary) || !boundary) {
+				blk_queue_atomic_write_max_bytes(disk->queue, atomic_bs);
+				blk_queue_atomic_write_unit_min_sectors(disk->queue, 1);
+				blk_queue_atomic_write_unit_max_sectors(disk->queue,
+									atomic_bs / bs);
+				blk_queue_atomic_write_boundary_bytes(disk->queue, boundary);
+			} else {
+				dev_err(ns->ctrl->device, "Unsupported atomic boundary=0x%x\n",
+					boundary);
+			}
+		}
+	} else if (ns->ctrl->subsys->awupf) {
+		blk_queue_atomic_write_max_bytes(disk->queue, atomic_bs);
+		blk_queue_atomic_write_unit_min_sectors(disk->queue, 1);
+		blk_queue_atomic_write_unit_max_sectors(disk->queue, atomic_bs / bs);
+		blk_queue_atomic_write_boundary_bytes(disk->queue, 0);
+	}
+
 	/*
 	 * Register a metadata profile for PI, or the plain non-integrity NVMe
 	 * metadata masquerading as Type 0 if supported, otherwise reject block
-- 
2.31.1