Received-SPF: pass (google.com: domain of linux-kernel+bounces-20714-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <73d03703-6c57-424a-80ea-965e636c34d6@oracle.com>
Date: Tue, 9 Jan 2024 09:55:24 +0000
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 00/16] block atomic writes
Content-Language: en-US
To: Christoph Hellwig <hch@lst.de>
Cc: "Darrick J. Wong" <djwong@kernel.org>, axboe@kernel.dk, kbusch@kernel.org,
        sagi@grimberg.me, jejb@linux.ibm.com, martin.petersen@oracle.com,
        viro@zeniv.linux.org.uk, brauner@kernel.org, dchinner@redhat.com,
        jack@suse.cz, linux-block@vger.kernel.org,
        linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org,
        linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org,
        tytso@mit.edu, jbongio@google.com, linux-scsi@vger.kernel.org,
        ming.lei@redhat.com, bvanassche@acm.org, ojaswin@linux.ibm.com
References: <20231212110844.19698-1-john.g.garry@oracle.com>
 <20231212163246.GA24594@lst.de>
 <b8b0a9d7-88d2-45a9-877a-ecc5e0f1e645@oracle.com>
 <20231213154409.GA7724@lst.de>
 <c729b03c-b1d1-4458-9983-113f8cd752cd@oracle.com>
 <20231219051456.GB3964019@frogsfrogsfrogs> <20231219052121.GA338@lst.de>
 <76c85021-dd9e-49e3-80e3-25a17c7ca455@oracle.com>
 <20231219151759.GA4468@lst.de>
 <fff50006-ccd2-4944-ba32-84cbb2dbd1f4@oracle.com>
 <20231221065031.GA25778@lst.de>
From: John Garry <john.g.garry@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <20231221065031.GA25778@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?WWs3T3lyaDNqbHRTeHVzMXNMdHIwanBPT3pkTEwrRExtc2xNTU1Lc0NSSlRy?=
 =?utf-8?B?RGdtNEQzZ25iYXJtNWx3alRDa2tKTEVTbXcrUytaRWdYMVAxZUx2emNSVVZU?=
 =?utf-8?B?bm1ucFhGRFhHZ01rV2ZGTXdvNDBwdTRxajRXcmtQdjNTZlk2SFJQMjZmUjNu?=
 =?utf-8?B?aWgyUDZOUlAwSE8yK0pGK1JzQXYwdHNBdzNiREQrWWF0SzNDMUh6T1JNTDly?=
 =?utf-8?B?Rm9FbjEvMWkxK3NubkVpRnlTQ2txc3ZaSEVCeXF6blk0U1JtY1lpZ2lYV1po?=
 =?utf-8?B?ZCtwSFluV1lFaE40VHo2YWRCNTgwajVkSzBzTlYxNFdiNTRRZFZMNVJXd091?=
 =?utf-8?B?eEhIRzJ1Z3VqVmsyeXVNNHo3dHVaNUlBTjd0NXY2MW45TXpiMk5Sa2hQdVN1?=
 =?utf-8?B?RHd5b292VUs0QVpwdHY2YUtPQUZPWG5EZy9GTzhOVjdZUzFNeGdUeERWUHZ6?=
 =?utf-8?B?TzlvendPZEVDUWpmdlQ4NU40clZpWlN5OTd5YiszZk42VTFkdWZhVGVPQVZJ?=
 =?utf-8?B?c2FObUlFazNCYXVTd1lVY0xRYVVLOWtKT3pkdXE3Rk5WZFpWUGVZam9kV3J3?=
 =?utf-8?B?ZHdPZjVBOWsxMVBqMGgrdHNGcDg0cTNPVGtvSmlPeVVxL1g0MkU0VVJRMTNq?=
 =?utf-8?B?dHpCMHJTM3hxVlVFdWt5MkxQemZUUDRUYVc3MjRhS2RRdUJ5TjhGbUcxTXpk?=
 =?utf-8?B?ei9hTGxWNVh2TmNSb2szVnVmQlM5TWpzWUJoOGVuNE5MM2hpdmhKZm81Wndm?=
 =?utf-8?B?aGFVdlZFUGxLVE9WSU4xSmxpV1A2K1I1T0lZaU5xTGJEbGlTZ2ZURFZUdkpm?=
 =?utf-8?B?MEFjOGgrZUlxOFNlQW84YVoxdU5UaXl1emlXS1lNcXhNSmVtbDJVNGpIOXBy?=
 =?utf-8?B?Y0dhdTNyLy9OTTN1U3RvYWFkSE8rYy8vOWVRNE54WmdBNStTVExHM3A1Sy9K?=
 =?utf-8?B?VmpKdVdWbVR1NlQwcnhCbDJEODA2Z05yUDNQZnFQaXVGK25ZblNaRTZZWXVS?=
 =?utf-8?B?bFVPd1k0SWlHZjdjVS9oTFE3RDlUNjhBRmxMQ2pVUUNaMDVTU3NOS0dVRUQx?=
 =?utf-8?B?dHFGMkRGakljNDlOb3NrSGtXTUJXQUdNSm5LRlV6MTN6WCs1MlJ3YW4xWVAz?=
 =?utf-8?B?TnE2KzVrWUw1d3VsZiszME1YL2g4eWc0YlQ2dlNENjFrYzQySUhGbjFNamRn?=
 =?utf-8?B?NTVwc3YraW9pQm1Fa1dKRHl4SDJCeGlaa29rU3ZOZHNncnhyWnFlNG40eXUr?=
 =?utf-8?B?WlJQRjg0QmZtL0tVazRoaEk5YW9PUTdNYWNBMlNNcU5nZ3pFQXByTjVOK2tJ?=
 =?utf-8?B?dGtUaEFLajU4YnprT1A1MUxqYkp4cnppSldhb1VLRGlDRmJiWjgzTGIwRmVa?=
 =?utf-8?B?SWl0ZmxkWnBmKzBIeUFCT2hwbmRuUUJxVXk5akZuc0d6bEo5dXlUVHFUcEdY?=
 =?utf-8?B?Wkc5Uk5TNWk0eC9wWFdmVitjRWZTbjFGVDBENkRYUTd3a04rQmF0ajVzR0sw?=
 =?utf-8?B?eG1GQTZhU3BmVEpSQmVzeGhNcjlzcU82ZzZvNlQxaVRTVUQ2MzJoZ3NtMlo2?=
 =?utf-8?B?YTFLaW5tRUFLRWZpY1VUY1BhWURXaHA2Y0JEbUZPSDd6SlNhYktTQVZrTlZU?=
 =?utf-8?B?T0kwSHV1RXkxZ3c2UmlQTERiQ0w4aGxTUWZhUENhaXNUOFdocGlraGtsbjBQ?=
 =?utf-8?B?WGYvcVVVaEJNOEcyb3dWVE15TFJSNzJLdHB4T25QNVg3dFJwWHVjaTdadVg0?=
 =?utf-8?B?OE4yU0pkMHY2UGxtQVcyMVY4Wnp5WFB0Sy9FdFFQSVZKMFRFWitvYUVMczdO?=
 =?utf-8?B?bGZXcG1kczBwT1lqVi9lL3pZak95MldsdkJMajdQU0JNSXgvNnVGNVY2Q0N2?=
 =?utf-8?B?d0xaNmR0cEZ6L1ZkRHhvZ3JYeHBVVlN6aEtCU2xINWd3TTVaLy93bHVlTVAx?=
 =?utf-8?B?WHhnZEVrdTBBSWVmM203SDVEa01DVjd6bDNtVDdRQU1pcld3QUpDV1Jydk9T?=
 =?utf-8?B?K2dvUEFnMUk2V3R1UFZBcFNUQ1oxM3Fxemh5eDR1M0ZrZW1GVVFlQWIzV211?=
 =?utf-8?B?ZzBFQlJkdkN0VnowVldJYncyY1R4WTlHOXJTSzcwTEcyQkdOVHV0eDJPR01R?=
 =?utf-8?B?Mm9mTVJkeldLZXRoV1RjZ2pEVnVHRk5NK0xlYlVodFVRL1F2UEV0TkMwK0lq?=
 =?utf-8?B?OHc9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	msMzTbYbQzDATVzVQPLOSmXWF+vQGqD73GW3+s0+pmWV/nUB6mnU823Qlu+9uNJjjV13VHxyUeQ/swqJ7a1o3XXdRtJreNSJvb41B3UlzzIogtmRol5cE5d+9cRhpyqW7Y6R6xwi+WuV68EWvB5X9hgYI7g/BD4CTsImlea/fR2bw/JHDLpy+FzzJnPIhasolH+q7MF3oGqaEKpXUox6LYxdGPAm2zNJzQ1DlpuiZ+duq/N/vmIiptAutxr1dd8P1TvhaqX+V/pGc74GimdHxLPL6wDsDKisKrYAlJa6+7mlwxwrFJ/NsDYSU+vtPoRGnbdK0aSRk+Ug6/hvJlr2K6xjBhIMvRFZk6Dw1rfCG+GWJT3Z4plZsHOAq0MMU3zmEnbJWQDQUFh9WqokODT21BCdYs1+bikHOJa93c4LRDrFDqzU3BDp5ixtGXxdsRz20zxkPxAND6oyRYLr489DbXdC+0w+qMTuqxTmi547ALN2v2VgFcvdH97Zr+v3vixHt+o3T/hHwQwqqaZbo8d+pmVQJGURSNXQxOYUT5f9Tt9mYB3eJoKJLasVUO+JAxnh7IoIxN35K00+pnazAl19ZVtD8YeHVAMsz/Uhce8cSeY=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 666b903f-0607-4f32-7bff-08dc10f91cce
X-MS-Exchange-CrossTenant-AuthSource: CH2PR10MB4312.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jan 2024 09:55:29.8694
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DbIcRviZ1xGU7YTxPpo0fRL2Merc7++B7shqejkm/I5RwdCZy/B4huKleINr9PUJ4zD0yuZmBt4zzgml0S/sBw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR10MB4146
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-01-09_03,2024-01-08_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 phishscore=0 mlxscore=0
 spamscore=0 adultscore=0 malwarescore=0 suspectscore=0 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000
 definitions=main-2401090079
X-Proofpoint-GUID: n3ZwaKhzHYOh3rvr1Ta0Sq5_JgpSXDfe
X-Proofpoint-ORIG-GUID: n3ZwaKhzHYOh3rvr1Ta0Sq5_JgpSXDfe

On 21/12/2023 06:50, Christoph Hellwig wrote:
> On Tue, Dec 19, 2023 at 04:53:27PM +0000, John Garry wrote:
>> On 19/12/2023 15:17, Christoph Hellwig wrote:
>>> On Tue, Dec 19, 2023 at 12:41:37PM +0000, John Garry wrote:
>>>> How about something based on fcntl, like below? We will prob also require
>>>> some per-FS flag for enabling atomic writes without HW support. That flag
>>>> might be also useful for XFS for differentiating forcealign for atomic
>>>> writes with just forcealign.
>>> I would have just exposed it through a user visible flag instead of
>>> adding yet another ioctl/fcntl opcode and yet another method.
>>>
>> Any specific type of flag?
>>
>> I would suggest a file attribute which we can set via chattr, but that is
>> still using an ioctl and would require a new inode flag; but at least there
>> is standard userspace support.
> I'd be fine with that, but we're kinda running out of flag there.
> That's why I suggested the FS_XFLAG_ instead, which basically works
> the same.

Hi Christoph,

Coming back to this topic... how about this FS_XFLAG_ and fsxattr update:

diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
index da43810b7485..9ef15fced20c 100644
--- a/include/uapi/linux/fs.h
+++ b/include/uapi/linux/fs.h
@@ -118,7 +118,8 @@ struct fsxattr {
        __u32           fsx_nextents;   /* nextents field value (get)   */
        __u32           fsx_projid;     /* project identifier (get/set) */
        __u32           fsx_cowextsize; /* CoW extsize field value 
(get/set)*/
-       unsigned char   fsx_pad[8];
+       __u32           fsx_atomicwrites_size; /* unit max */
+       unsigned char   fsx_pad[4];
};

/*
@@ -140,6 +141,7 @@ struct fsxattr {
#define FS_XFLAG_FILESTREAM    0x00004000      /* use filestream 
allocator */
#define FS_XFLAG_DAX           0x00008000      /* use DAX for IO */
#define FS_XFLAG_COWEXTSIZE    0x00010000      /* CoW extent size
allocator hint */
+#define FS_XFLAG_ATOMICWRITES  0x00020000
#define FS_XFLAG_HASATTR       0x80000000      /* no DIFLAG for this   */

/* the read-only stuff doesn't really belong here, but any other place is
lines 1-22/22 (END)

Having FS_XFLAG_ATOMICWRITES set will lead to FMODE_CAN_ATOMIC_WRITE 
being set.

So a user can issue:

 >xfs_io -c "atomic-writes 64K" mnt/file
 >xfs_io -c "atomic-writes" mnt/file
[65536] mnt/file

and then:

/xfs_io -c "lsattr" mnt/file
------------------W mnt/file

(W is new flag for atomic writes obvs)

The user will still have to issue statx to get the actual atomic write 
limit for a file, as 'xfs_io -c "atomic-writes"' does not take into 
account any HW/linux block layer atomic write limits.

FS_XFLAG_ATOMICWRITES will force XFS extent size and alignment to 
fsx_atomicwrites_size when we have HW support, so effectively same as 
forcealign.  For no HW support, we still specify a size. In case of 
possible XFS CoW solution for no atomic write HW support, I suppose that 
there would be no size limit in reality, so the specifying the size 
would only be just for userspace experience consistency.

Is this the sort of userspace API which you would like to see?

John