Received-SPF: pass (google.com: domain of linux-kernel+bounces-90600-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
From: John Garry <john.g.garry@oracle.com>
To: djwong@kernel.org, hch@lst.de, viro@zeniv.linux.org.uk, brauner@kernel.org,
        jack@suse.cz, chandan.babu@oracle.com, david@fromorbit.com,
        axboe@kernel.dk
Cc: martin.petersen@oracle.com, linux-kernel@vger.kernel.org,
        linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org,
        tytso@mit.edu, jbongio@google.com, ojaswin@linux.ibm.com,
        ritesh.list@gmail.com, linux-block@vger.kernel.org,
        John Garry <john.g.garry@oracle.com>
Subject: [PATCH v2 00/14] block atomic writes for XFS
Date: Mon,  4 Mar 2024 13:04:14 +0000
Message-Id: <20240304130428.13026-1-john.g.garry@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?dCMdisYN0FpoqBFHTFoIsmm4s+zfT1laN1bT5MNC2X+0WLUxkwDUy/0tW8/P?=
 =?us-ascii?Q?ukzLGd16K8tePxODSsGGxE5h+rZiiI75ix7m/onDaZepeN/rgWJnxVmhgKjn?=
 =?us-ascii?Q?Glp2BO7faJCY+rOMBJ2PxVBykTDWpokFSidTWSFnQEz/y9F79efnLekttpEV?=
 =?us-ascii?Q?uH0pOM+Sa4ZnYifxwaIgNY+WdaMXLMWnp+7oR/WMvu5quqQyTWD41JDLwgUE?=
 =?us-ascii?Q?T0d9O0TNPnR8WXhwL4LNd6C/BW5EP3lJnUGgeh8y/Pn9YPVBLR26ebVtdfpD?=
 =?us-ascii?Q?WzjIQ8OMnnNw738kncR+Id5yqwgGilWy369G6vPUrli3K0vZoH39em+fOBKt?=
 =?us-ascii?Q?3YjG4dPv38MZEWDdeTQ8l1yV5TdD8Iqxi+EQf0g+anjJfez2OnomB1e6rT0f?=
 =?us-ascii?Q?gAsFtcbZgKw0drVGBblmg2hzlNYK0Mhu+CiLIKJSM/PRdD7Oqxrnxw5jp/pm?=
 =?us-ascii?Q?iywsEpKq5H46CJ/nggB1IOPLA8ZThd4WRHmtwTrgJady492NWuiDvSfV0xTr?=
 =?us-ascii?Q?IJK2K9E8d1RjnHulIwsc6QykkTzUkSvU2ec+4m2rCp05ZUEv47HvObmu5Va4?=
 =?us-ascii?Q?+eLXh9AdzJiBCHteqXnsHQLK0mB8dWXklJGyW35CWV3aerdFvMRAnPicfEBN?=
 =?us-ascii?Q?mhTooomTAa36Hdkg+WBe60d+BhYGgk15H6RVZ4frurqGS51fApyuasRwoEd5?=
 =?us-ascii?Q?KtjvJQwVR5TTRwwmKq1ui9ELsCR3ExO3/VouJ5ibEwzh9hqKey8cdYm0mdGY?=
 =?us-ascii?Q?D8YeV18nhyaSrEYpJ/Oz22x3mNPoA7pi1Rxn/2GTbJGTsEfTLOu39eOq0c/H?=
 =?us-ascii?Q?M11Ugs7pqAyMhM7jqmZNX/rpaLud+3kE4Xep0NtUkdAsOsOzfoEAtxZzJhht?=
 =?us-ascii?Q?dz+D43JBaR2eCjPVNGV3BsHrthQzkWG3Ib/q/rckgOk1pO+OFbeOUmSkJcH/?=
 =?us-ascii?Q?a9Ol/maoJtTxUDC3k8IlJr8vAIwsi4NiTW6TYQALSAcCo5HjAoMkBbQQZIpb?=
 =?us-ascii?Q?jgDoYN0mGai2L9sIKKcrRnRVJDlpZ0+Petxp4EIud0kEwu40G82Uws0OEwMM?=
 =?us-ascii?Q?JoECvlzivF6egcWnoyfOnQZTHrk0C2JGesersBfGWERBNwspy3VUvlD5P5FB?=
 =?us-ascii?Q?hXz1sd0BxsO7iJvZwMxY52qg3xMzeuYiW3olsQF9n5cIX03uZDNT3466MBlC?=
 =?us-ascii?Q?ZmhuB+i21VO4ctlo2R9/PFQ5wAIxICtOObusTRCXWeInmGLOIOB2e+l73uUA?=
 =?us-ascii?Q?vSxjBEDnFwyedSzzXecLT4uwQA5BwX2E9j+lpVa8G/0/Eh6zSMDGWPc0Zfco?=
 =?us-ascii?Q?GyvpdjqJZLcVuscWqTn3icDosKJKuTaGHOd78L4RRMTIBc8V69PO2bAQcpp+?=
 =?us-ascii?Q?HbsKQblHxUsHC0O7N6I1Er9CbvQ8rynvJh7YjA/qZGHGDASd7eaH2i+BWDY6?=
 =?us-ascii?Q?AHTtweQTpLV/wpFRXebfZPN3F7U/QrRhEghsD/tQ90hRAmM0Ecf7WZlrRHiy?=
 =?us-ascii?Q?8u/friRKvE8IpOGO/YZO0QU+2NBFctgrS65yFkoitGtIwpfTt5dMPz3BEUIy?=
 =?us-ascii?Q?qpLlTqL7tRlc4reg2+9yvcamFbFw0fSP21U3bm+slfduLfH+WcOwiWqXdfSe?=
 =?us-ascii?Q?/g=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	Mq2dgdH9F4BLz+ZuO2L93W0lg8BRjdgPW/7GuMC3f7u4dQKGGS98h3q6ApoYl+nVJLegaMjNR/Ps2PcrNDcciVjp/tnzXfMIlBn7bKoh5t6I5ZWHkjEOVo0pNTgOs5qXCOhP+YvFcjmexX+VNwCjXzmBG0T38H7kdK1eesUvxVMEa+G21LUdA1oRcupa44IoPJNHMWzJ5cM+NIOPsvg5llOWhHTCyq+XNi+pVR3cRWuiLsT1Wpdwb2ja8Zl4r/AT/UnG9goQl0ZNDINv+VqCVakFMvMBfgnXBVPbJsz8wZdiq4+traec+eyIGRSQH3RJlu+7e+y1HZEmur63Nkmuc1YEcjdEjRgTqYcIPeRuyMDm03NdO9N4gKcyN7C0ZS+VYnZdSCGrjbXrQP99FNM2/tWuCRYOX4YZBPIjlUmAelF0mqUoKYv8NGv1syT0jGcJ+ELx2KPvnIry7Rx2aD1IHuyCKnUPR4cdXb4aglzmTKkNtd3TOmro+V9FVM1OAQeTDnthjQ83owTswmpOiW1/DdN0ArnAmhRaq0zmBDAbAPQwYOodqi1+o1lE+RGuyNm96QOweVEoKz369fUlkC8/cKPcXiLwdnHaWXqKkJ3l8Wg=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c04f2a3a-749e-41b2-163c-08dc3c4bb0be
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Mar 2024 13:04:56.6570
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Thsj/prLtW9a5arg2fa7AmMXImnPiEKPN/JSKNxms7LlNxh/q+3bAT3QwxUF40E9EVlJIFm/2CCrsODRsNwnPg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR10MB7805
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-03-04_09,2024-03-04_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 suspectscore=0
 adultscore=0 bulkscore=0 spamscore=0 mlxlogscore=999 malwarescore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311290000 definitions=main-2403040098
X-Proofpoint-ORIG-GUID: 02gk-A0QnAP5xNYuwVpNU2eoKtXhbQJT
X-Proofpoint-GUID: 02gk-A0QnAP5xNYuwVpNU2eoKtXhbQJT


This series expands atomic write support to filesystems, specifically
XFS. Extent alignment is based on new feature forcealign (again), and we
do not rely on XFS rtvol extent alignment this time.

Flag FS_XFLAG_ATOMICWRITES is added as an enabling flag for atomic writes.

A FS can be formatted for atomic writes as follows:
mkfs.xfs -i forcealign=1 -d extsize=16384 -d atomic-writes=1  /dev/sda

atomic-writes=1 just enables atomic writes in the SB, but does not auto-
enable atomic writes for each file. There are no mkfs checks yet whether
the underlying HW actually supports atomic writes for at least 16K, but
this will be added.

Support can be enabled through xfs_io command:
$xfs_io -c "lsattr -v" filename
[extsize, force-align]
$xfs_io -c "extsize" filename
[16384] filename
$xfs_io -c "chattr +W" filename
$xfs_io -c "lsattr -v" filename
[extsize, force-align, atomic-writes] filename
$xfs_io -c statx filename
..
stat.stx_atomic_write_unit_min = 4096
stat.stx_atomic_write_unit_max = 16384
stat.stx_atomic_write_segments_max = 1
..

Note that, apart from "Do not free EOF blocks for forcealign" change, XFS
forcealign support has not been updated or comments addressed since
originally sent in:
https://lore.kernel.org/linux-xfs/20230929102726.2985188-1-john.g.garry@oracle.com/

I was waiting until this series was progressed for updating forcealign:
https://lore.kernel.org/linux-xfs/20231004001943.349265-1-david@fromorbit.com/
I don't know the status of that series.

Baseline is following series (which is based on v6.8-rc5):
https://lore.kernel.org/linux-block/20240226173612.1478858-1-john.g.garry@oracle.com/

Basic xfsprogs support at:
https://github.com/johnpgarry/xfsprogs-dev/tree/forcealign_and_atomicwrites_for_v2_xfs_block_atomic_writes

Patches for this series can be found at:
https://github.com/johnpgarry/linux/commits/atomic-writes-v6.8-v5-fs-v2/

Changes since v1:
https://lore.kernel.org/linux-xfs/20240124142645.9334-1-john.g.garry@oracle.com/
- Add blk_validate_atomic_write_op_size() (Darrick suggested idea)
- Swap forcealign for rtvol support (Dave requested forcealign)
- Sub-extent DIO zeroing (Dave wanted rid of XFS_BMAPI_ZERO usage)
- Improve coding for XFS statx support (Darrick, Ojaswin)
- Improve conditions for setting FMODE_CAN_ATOMIC_WRITE (Darrick)
- Improve commit message for FS_XFLAG_ATOMICWRITES flag (Darrick)
- Validate atomic writes in xfs_file_dio_write()
- Drop IOMAP_ATOMIC

Darrick J. Wong (3):
  fs: xfs: Introduce FORCEALIGN inode flag
  fs: xfs: Make file data allocations observe the 'forcealign' flag
  fs: xfs: Enable file data forcealign feature

John Garry (11):
  block: Add blk_validate_atomic_write_op_size()
  fs: xfs: Don't use low-space allocator for alignment > 1
  fs: xfs: Do not free EOF blocks for forcealign
  fs: iomap: Sub-extent zeroing
  fs: xfs: iomap: Sub-extent zeroing
  fs: Add FS_XFLAG_ATOMICWRITES flag
  fs: iomap: Atomic write support
  fs: xfs: Support FS_XFLAG_ATOMICWRITES for forcealign
  fs: xfs: Support atomic write for statx
  fs: xfs: Validate atomic writes
  fs: xfs: Support setting FMODE_CAN_ATOMIC_WRITE

 block/blk-core.c              | 17 ++++++++++
 fs/iomap/direct-io.c          | 24 ++++++++++----
 fs/xfs/libxfs/xfs_bmap.c      | 26 ++++++++++++---
 fs/xfs/libxfs/xfs_format.h    | 16 ++++++++--
 fs/xfs/libxfs/xfs_inode_buf.c | 40 +++++++++++++++++++++++
 fs/xfs/libxfs/xfs_inode_buf.h |  3 ++
 fs/xfs/libxfs/xfs_sb.c        |  4 +++
 fs/xfs/xfs_bmap_util.c        |  7 +++-
 fs/xfs/xfs_file.c             | 60 +++++++++++++++++++++++++++--------
 fs/xfs/xfs_inode.c            | 28 ++++++++++++++++
 fs/xfs/xfs_inode.h            | 11 +++++++
 fs/xfs/xfs_ioctl.c            | 49 ++++++++++++++++++++++++++--
 fs/xfs/xfs_iomap.c            | 19 +++++++++--
 fs/xfs/xfs_iops.c             | 38 ++++++++++++++++++++++
 fs/xfs/xfs_mount.h            |  4 +++
 fs/xfs/xfs_super.c            |  8 +++++
 include/linux/iomap.h         |  1 +
 include/uapi/linux/fs.h       |  3 ++
 18 files changed, 324 insertions(+), 34 deletions(-)

-- 
2.31.1