Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1;
From:   John Garry <john.g.garry@oracle.com>
To:     linux-kernel@vger.kernel.org, linux-api@vger.kernel.org
Cc:     martin.petersen@oracle.com, djwong@kernel.org, david@fromorbit.com,
        himanshu.madhani@oracle.com, John Garry <john.g.garry@oracle.com>
Subject: [PATCH 3/4] man2/open.2: Document RWF_ATOMIC
Date:   Fri, 29 Sep 2023 09:37:16 +0000
Message-Id: <20230929093717.2972367-4-john.g.garry@oracle.com>
In-Reply-To: <20230929093717.2972367-1-john.g.garry@oracle.com>
References: <20230929093717.2972367-1-john.g.garry@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?GjrftV7ruvrqXEa91ICkNF42cGKIZzDK8j0DiiBNRGfHchXZ87l92wlpS8Hc?=
 =?us-ascii?Q?ILSDjavcyIuischCDmca9qMAxS1dWuvW5khyWNTKHtd9Wzm9mnLPFBdlCaQS?=
 =?us-ascii?Q?5Mu3Sg5WeCnyxQl3QcHmcItBLAqORstFKTUZvtcukH8RCa2dJlLkBgN7OPJ1?=
 =?us-ascii?Q?Rcvt1qUUBlbLEIyDlqu5oKdJh2lsPuZfydwR11tQ9UheddRKyIsr9rvPcBVm?=
 =?us-ascii?Q?aoTYFsLur4tbbHEh5iLUhPIAJTs9J0E63+VNLndt65CrVJHGewe0hdV6x4sj?=
 =?us-ascii?Q?lYGdHUF+gmD3urdW1uZYWpXeu+ZvJ1Im+712hWNTg7RZAMzirRUfJwpQdUKa?=
 =?us-ascii?Q?N0ZIiJU1p5MPOgF7Yh0MGowJcrIFShnjUgUvEI0kRjcRyvkxNZoNUuWPSpa5?=
 =?us-ascii?Q?OJyBgx3VFc+ztj7nEAAAtEzwYaodjYyZ8q0izb5/M5t6rLI4vocyxhB58M3f?=
 =?us-ascii?Q?B+sZwUSE/I9dBlrlbRKl48aU08vC6793jKMSuDq+K4xC2opShGur1RM+XAkt?=
 =?us-ascii?Q?bVVFpYmQO2cINzyp6zM5TX7r+Tlkms7kGF54cZtmw77Wun02lrcEFwlSuqbn?=
 =?us-ascii?Q?YEX92iRpvaOF0YeuVK4SxF/JILWjJdupBTLBl4/ht488syq7q0bTsjJMecJ/?=
 =?us-ascii?Q?Wx15JC2+2UEneX+eywHwoO0DxKrenUonHYGtlu/Fj6mjelZBmTe55vjV+m3H?=
 =?us-ascii?Q?wRpEgP8e1sAAm/SYtWmWVQdx+UZZ2r37mJgdWTnNtFSdwkReqMMv0FEbOjpw?=
 =?us-ascii?Q?3nnvZsnP77YDyJxhezoMNV9wPPBwW+iM9iauCMlUD3dHjEik44H1vL6wInb8?=
 =?us-ascii?Q?T/RmyH8OBoyL0YOrsy0fQxojW0pKROCjKDcneLyWfN58/dn4GjWAi6gqcylj?=
 =?us-ascii?Q?nqhXv/zw1AKNQy88pe9gajzX+cNnJepRodzqlXMH88uY6POgdT2ePQGdvISG?=
 =?us-ascii?Q?9NcWNj7TMrhV0QigGHEroz4NbDgZRSlb9SMwPQCeG92Dlg9WPZU+8rbZ4bag?=
 =?us-ascii?Q?HTY6/XHuGTtqZ5I5jw/Od+iVK8GSPaeF5sKhR68UsxsI+VE1rbHixqdwfy4O?=
 =?us-ascii?Q?pwxCciOKAgVLdJxgJtcIQCR+iw87ebax4ugNJ5zZK9gUB28Kvwb1bqyLxzL0?=
 =?us-ascii?Q?7KCsh5OTE3FGu9RvGq3cjmwDZ/8K+UyfRcqhtsiXNY04bQhhi64KpuP0m+Sj?=
 =?us-ascii?Q?BcNtSHlWIiS7sbB2jVZq6iMlBj1nVMx/vBGp7RCEI2FESrwKwFBdHlm5O2Is?=
 =?us-ascii?Q?vpa0niPVNU7kgy9hHWnLGxMWJPQONSaFMsn2Lc2v5xy1H1S+imSOhqPbB/Am?=
 =?us-ascii?Q?7MMZ5Pwpl6+6tbYP6yhtPx5kYNw93z+E1MkXkulm5S+9parzFKuaSfI1OO3B?=
 =?us-ascii?Q?fRmCqZvpK+e8PcxfHd7qxQJpKECsUs0OpEovFPP+PFFG/KTPjBFF4QzwHk3p?=
 =?us-ascii?Q?Dmhyd7KCAX2ytKFOVahTBLBNbQzecZORnidkc/f3gdAM/7se3Mm5DBlL/LJn?=
 =?us-ascii?Q?jcVBhOtOLtKbPe1bHt5KU+QJsOGvzE+Y4owhGfgcbm7xgT1+gfOYQXXx3uhw?=
 =?us-ascii?Q?HvxGeCp7AAJpYQ3nlb027wWqcYDAhCktRYSX1r34?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: USomWYX4Z2MzJ1xo5MYNSSdyJAqlON7bU3Zbh6NX5LWxWjHPTHEp+i2eI9wN1pmOJIL3VezA0CDvhciwKuUzit0KpYFGfr801i1m3sqnaxkIBzU45EK0bHyIAOc6vZa6H3+9oFOvKhfEZgZpa4BFDtfbjmm+nR/ev+liTYboExk2LJ//y+5knK81GDsGnjKCu3ZfAl8qo1U3qCMr5F5fYMqODny2m4IC9Ztmwxu19+JYNmQwWIbURBGTFMHxMZaMbNhsQadY2WwWl/4mbXB1AgAvP5Rlux+cWuqPGgCak/9DJ/Kfa//H2H14riFBkS7kg2zvsj8tNbbN9FmpNs7rZxTpPZY7xVA8MMhG9FHguLF0+OiYYARzs8U2AtfoUJfb9kIuoz0QAW9fUiTRMjreXYn1YzG/RbN/u6WndQzU/6VxLVVnDwunu3E6yP/BCM88Glb0Jw60iXNUjR7Dy+KEaYD9gz8Ge0/bEJdpLXwnhdifTSuG6aLwVWMR6BjrtfgKs/usJlH+OO+1YGyl3shrndu0p0h+y9W6jY5pK0i57S18K37KqNqw8JjVbAXSiEg/nnQz8TPhd5syAwiQqpjmbOF+KFNcEu25VtG6t4uvDs32A6wne0VDNdsGTDqeE1G5nrigyhRLinNNhWHHQ2aa6ciFXV+p9VQ289HNW0H4YmFkYdLfW/iU31hvB1OBziLVO7OJ+QFlK3Hmb2kfFVv+VK4oDIpGoKO4HkxDV8RJiN5QYDwG96t71W/r+zjtIZA9rOgCctPFO8OjbU9PiUbH7qMrGTFsPETRXTJ3IXAhaQ+C5GHZTqP3czxSJdPCRi/j5WNREGt45Bl5id3VvhSL7A==
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 50f18aa7-1dd5-4a3a-94a8-08dbc0cfc0ec
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2023 09:37:53.1903
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OK5iHTHm8863P3fbfVVBw6hofHBenhcwpywhCQk2ESzpe9QVPwwcqvuE5L6HKYb4+qOmnNWNJ/JHiIANh53QJg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR10MB7197
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2023-09-29_07,2023-09-28_03,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 mlxscore=0
 mlxlogscore=999 suspectscore=0 phishscore=0 adultscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2309180000
 definitions=main-2309290083
X-Proofpoint-GUID: 8HL7VOI_6h7NURaMGP6Zc1pcMxu_k5N8
X-Proofpoint-ORIG-GUID: 8HL7VOI_6h7NURaMGP6Zc1pcMxu_k5N8
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Fri, 29 Sep 2023 02:38:46 -0700 (PDT)

Document special rule when using RWF_ATOMIC flag in conjunction with
O_DIRECT (which is the only time which it can be used).

Some block devices have a virtual boundary, which is a boundary at which
write iovec vectors start and end addresses need to align to - see
virt_boundary_mask in linux kernel sysfs-block Doc.

To avoid splitting writes with torn-write protection in the kernel for
when vectors cross this boundary, special iovec boundary rules are added
such that we don't cross this boundary.

Even though a virt_boundary_mask may be larger than the CPU page size,
we just impose that as a reasonable limit, which matches what the linux
kernel NVMe driver uses today - NVMe is of special interest for supporting
RWF_ATOMIC.

Signed-off-by: John Garry <john.g.garry@oracle.com>
---
 man2/open.2 | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/man2/open.2 b/man2/open.2
index 4c921723c95c..e5b9d107859a 100644
--- a/man2/open.2
+++ b/man2/open.2
@@ -1782,6 +1782,19 @@ blockdev \-\-getss
 .EE
 .in
 .PP
+When
+.B O_DIRECT
+is used in conjunction with
+.BR pwritev2 ()
+and
+.BR RWF_ATOMIC
+flag, in addition to any alignment rules already imposed by the filesystem and
+underlying block device, it must be ensured that
+.I iovec
+vectors have no gaps such that the end alignment of a vector must have the same
+start alignment of any subsequent vector and that alignment must be least
+at a page boundary.
+.PP
 .B O_DIRECT
 I/Os should never be run concurrently with the
 .BR fork (2)
-- 
2.31.1