Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36;
Message-ID: <e4fb6875-e552-45aa-b193-58f15d9a786c@oracle.com>
Date:   Tue, 28 Nov 2023 17:42:10 +0000
User-Agent: Mozilla Thunderbird
From:   John Garry <john.g.garry@oracle.com>
Subject: Re: [PATCH 17/21] fs: xfs: iomap atomic write support
To:     Christoph Hellwig <hch@lst.de>
Cc:     axboe@kernel.dk, kbusch@kernel.org, sagi@grimberg.me,
        jejb@linux.ibm.com, martin.petersen@oracle.com, djwong@kernel.org,
        viro@zeniv.linux.org.uk, brauner@kernel.org,
        chandan.babu@oracle.com, dchinner@redhat.com,
        linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-nvme@lists.infradead.org, linux-xfs@vger.kernel.org,
        linux-fsdevel@vger.kernel.org, tytso@mit.edu, jbongio@google.com,
        linux-api@vger.kernel.org
References: <20230929102726.2985188-1-john.g.garry@oracle.com>
 <20230929102726.2985188-18-john.g.garry@oracle.com>
 <20231109152615.GB1521@lst.de>
 <a50a16ca-d4b9-a4d8-4230-833d82752bd2@oracle.com>
 <c78bcca7-8f09-41c7-adf0-03b42cde70d6@oracle.com>
 <20231128135619.GA12202@lst.de>
Content-Language: en-US
Organization: Oracle Corporation
In-Reply-To: <20231128135619.GA12202@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SjRPWmRDRWtmWElsU0M2MVVPQ3lZM2E5cTZKcjFsTEpWVUdoWmp1UmV3U3JF?=
 =?utf-8?B?Q28waTR1TkR2c2s1VHJITkpNQmppNExONXh0amh5T1hrN29XejVzazN5b1lw?=
 =?utf-8?B?Y0RjYU15dzh0UEtUcjhYYzkxV2UzbGV6OFlMdDJqSGFucGdBOHBJVU9iTW5k?=
 =?utf-8?B?ZkdlQ1NFMGNIcmdEWk5iREk0aTF6T1BwQS9nWDlwcXlZcHgrQW1IelhyUFBw?=
 =?utf-8?B?K0J1MzgyT1lLU08rcUg4RjNrdSszV1pPWkQxVm8ybStBUkxhWTVYdGZBaW5s?=
 =?utf-8?B?Vy9hcVRGSkVwdU1peVNQWmY5am9CZjZlTzQ5NC9mT04zdVNCS3plV0FNVGQ0?=
 =?utf-8?B?OUVIeXhYZzN3WHhzN09lT09iTFkzcDlwMkdmYkoyZzR4eVVsOW01NWVZTXIx?=
 =?utf-8?B?Tzl0ZElRNXZKWTdZNUY0cklVMnpDL1ZCVVJ0OUJiR09HaXptVlhMK0JLL3ZZ?=
 =?utf-8?B?MzZNaHI2NXB6WGlDM1YyM3l1M0dKdVhVUE5DMU8yM3gvNDRLMnFwWGpBdTgr?=
 =?utf-8?B?UTAvWCt0aEpETUZHRjlyY0RTOG9TeE81SkQxSkUxZGRVNGd1MW5kTjk3d2tB?=
 =?utf-8?B?bk1XOGQwb3RlZVBvRTJiN010dW02aitScmlUdUNUbGRIUUU5blhXVmx4c2wy?=
 =?utf-8?B?RmlmODIxcXVlTjRBS0tGS0JlWWhHUGdMKyt6WWp5ei9HTUhOeFJtWGhIZEhJ?=
 =?utf-8?B?WTFPdXdBRDhDR3hTR1hpTlJrSW5hai9HaFVBWE5JZ0FFRnZ3WDFWZmVEc0FJ?=
 =?utf-8?B?Tlo4SjRjZTV4VUFtQUFLQkdqcGhPc2RtYXNyUmdzUGJXaURlSHQwZWpQbXQ4?=
 =?utf-8?B?cFRSQzJTNmluTTcwTElhVG1EanFyK2JGcTdZWktMV2FZVGFydzlIWWZiRjNz?=
 =?utf-8?B?SlBRZHQ1RTF2VVN5aWZvejR6ZlR5MGhhT0F4cmhhc1AxUXBMOHRXODBtdy9R?=
 =?utf-8?B?WVNyK0x5VHhEQkgxVjR2Y3ZFSWk4bDVrcDlaWE5sbEp6TC9XTC81Q3labURj?=
 =?utf-8?B?RlllSkZEeXhjQjFNMjNhUHRVcldzTzRPRVE1NytRSndlYThaYURuT0djNFl2?=
 =?utf-8?B?RGhUV09pRERIZmdraTVoeVlCTFNOQkxDbUpzRjRFOW5GYjdKRnBRczlhL0hW?=
 =?utf-8?B?SUZBa0gvYmlvdzNJU202NEN2SU1zY2tDRk0wclBlQ01nK1AzYVJvUUpuNkdT?=
 =?utf-8?B?aStzTDZQVmEzVEF1YTl2Si9ONlp0bjdwS2duMDZpWDgxSW5PdjB5RkFwdlFO?=
 =?utf-8?B?UVRPYmI5Q3BZaVphTFdqQVBBcmVNU0tKQ0xBRk14RmtxcGxIUXFRbkRoUzJQ?=
 =?utf-8?B?WURBRS9ObFpmVWI4dGJUWUQ0RFQ2TWxOcW5MRUtud0xuTkkvUUdtYXNCNG9X?=
 =?utf-8?B?bHJ1TzJWcjlDaGpObDlZSHFnejBxVmFCbURWcFVNdzJ6THF6NExDR05TQklQ?=
 =?utf-8?B?eE51ZXFGZndEMGw5NTJoZytENjZXeGlRS1JqeTVpY1N4M0xSelM3UzJBajlu?=
 =?utf-8?B?QkZ4QUJmWG5qblVjTHR5TUhsdE4zWEVBcTd0YzV1Vnk2bzkzS3dZeXdrR1E3?=
 =?utf-8?B?eTB4Y3NjeEo0eGE4VDBxc2pNZjVBZjV0UkczQWF3MnFGVWZaakEzNmV3bGhn?=
 =?utf-8?B?VjFseUhtdDUwditPQjBzdlNJUUZ4dXIvb1ZublA1TVVKU0JmNm9NS3dlancw?=
 =?utf-8?B?SVgvUUZ4MTBzVVpUeDNkcVRoaDdKaWJ4dUFPQ2x6QktiMXRZdDBEaHBkWE5S?=
 =?utf-8?B?NkFhaWdCUERlM2YyUXpqRjVYYnR1UEN4em85ekMrVy9LZ3ZxakhwVDdEbVQr?=
 =?utf-8?B?RmkyVXNPVVc4WDQ3ZXhMK29ZSGtsTFhvVkY5UWhqdG51MndpaUFHa0cxK29q?=
 =?utf-8?B?K0l2d0hydmRPeXpYR2JsZWxhN3E3UDRKM0dTeHpvRjk2VFJCQlpobko4bUxz?=
 =?utf-8?B?UEdjSzRzMWcrbDZaZDJjRWVnOS9Vb08rMDBOZDdyRjJkOFZya3JyTUxMMktj?=
 =?utf-8?B?cWlURUJZaW1rS0VvZk42NUtjTGdMcVUxdjNUVXAvL3hvQnBuWktTRzZyNVYv?=
 =?utf-8?B?V3dEbkhHNTFTQk1qQ0JLOVZUVTd5c3Znc2ZrbmsyL0hqNCtIWmRWUElLRzN3?=
 =?utf-8?Q?ekBr/i3K36Z+G23ad61bgkHeZ?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?THVQTHErcjJyK0d0enUvQkdMWGxZNXk3L2x6T05yV3ZWeUh0K3JLMEdYT3Fq?=
 =?utf-8?B?SEd4ZnZiV1hPbGFEM21BWmRQbnlUM21WdFRiRSszeXpqeGE4M0hQRlhzcXU3?=
 =?utf-8?B?anh2bjcxaE1wcndyakFWMmRVRUtnU29vOERSRmh6ei9jNmZtQXVsNlRHNFhM?=
 =?utf-8?B?bTBtNDR6M2MxZVhBMFdVaEZsNUJVaUo1ZXk0aEN4K3Z5S2FKOVJrQmJTcGZq?=
 =?utf-8?B?aXkxT0RwRERQRllBK1JHM2RrQ1pnWG9OQ3c0cFhWMFBoVUxHT3YvSWRrOWxO?=
 =?utf-8?B?K0tuemdFTm1UbzZQNUk5WlBsZWRzTXFCZzdnZXY3a0hheTdFVEhwMTd4SzVZ?=
 =?utf-8?B?alNONWZaaVh0Z0hVb1JIajIrTnN2SU5vTTUrOWtNT3JrQmRiQjdjQ3FMUERT?=
 =?utf-8?B?U090Uzcwb3dVOG5xOGNWSWJwOXZzN3ZrWm9Ydkw3R2VtSWhFU1QreU9wUVJS?=
 =?utf-8?B?N0NQT2c5cHhMSmdpdmJOY0l4SGRNd2FrV1EvSmFabGxHY0E2Q3k0WXlzUldT?=
 =?utf-8?B?QmFqZG8yL01qaitQZERzbk5vTS9YYmpIdnFZTmNSUVp4Q0dLZlBkWDFGbkRm?=
 =?utf-8?B?UVhzZHVCY3c1L3hjdWM5QUIxNVp6MmQ4amJHaU5obGkxWmQ1K241TXpyUG1U?=
 =?utf-8?B?TmJTZWU5Y0YvNVZjVnBQclJxRDdyZFkweENFZkkvd25YRG5uaTN4cDd3eUxO?=
 =?utf-8?B?TndocnNqcStUTythdml6REZ3V3FtU1d1ZzlHSVo2Z3JIY3R2Q0xsRTdXeGly?=
 =?utf-8?B?YTdRS0lsVHlXak5TYnJHNlNMTjV2eUZpMXlMZnZydGwrUktDenZXZng5UUVT?=
 =?utf-8?B?WUphMWZ0SC92cmlHaVFWZmxtcmdiTjNuWVJJbWFPcFJHeTFDWDlaRHV2S3lS?=
 =?utf-8?B?VUpJOTc1V1N1MCtNVnExSXBOU2FzbVR0RVJoeFk0RGJMMDNzVElIMHFlSU8z?=
 =?utf-8?B?SldmSUcwRExERksyeGZBQy9xNjBkL1VrNXplSE5DdXdJQTNhL1pTaGQxOEMy?=
 =?utf-8?B?TjFYR01xalhoRldHYzE1SmNUelUyUngzanlzU0tlaXJyQXpzZ0FzTTkzZ2pO?=
 =?utf-8?B?Wk9HR08yRC85ZUxwMzZUdVRnK00zdlQ1enNPcVhMeDBsL2t5Y0F3eDVBb252?=
 =?utf-8?B?R1pGaTdQNlVlUEJlVTY3UXdtQnhhR2xRcy8vZldTSG9FY0IwenMxTFVNRXpi?=
 =?utf-8?B?cGdxKzVySU5Mb0dpRTR6RFRZUFJYaStCQ1dzZUlrMXFRWkRUa2VmVitEU2tE?=
 =?utf-8?B?eFZBT1VVdENNL1diVEkrQy9YWG1tQ1VJZGdrNzgxWGpSUjNyK1JuWVRiRS9L?=
 =?utf-8?B?T0lYcTZmdEhUdWR4VXhIT0tMQmJFbERDbzhxRURPTTZsR3R4cHZCbWNNSmw2?=
 =?utf-8?B?U29JVlJTbmZVdm1VWDI1QzdkQ1hKN0thV3BhblJlMUl1WFZqb0tqeWFQMmNi?=
 =?utf-8?B?dzZ5SU05dHpvTjUxaVVubUVTMGFHcFJIZ3JILzNnPT0=?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5af35aec-0725-4cc2-6e1b-08dbf0395cc6
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Nov 2023 17:42:16.4488
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wq7TTgt3aq3zLkniK8Vl6L7IC3msGDHROKRwttaETMUKLyL/ua3dQtLfXu/2u1mFG64zyEV20MSrlnHzrXMTvA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB7375
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2023-11-28_19,2023-11-27_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 phishscore=0 mlxscore=0
 malwarescore=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=999
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000
 definitions=main-2311280141
X-Proofpoint-GUID: dARGLpdcmOR5R_laSSN-Gc-yZ1PhhKgC
X-Proofpoint-ORIG-GUID: dARGLpdcmOR5R_laSSN-Gc-yZ1PhhKgC
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 28 Nov 2023 09:42:47 -0800 (PST)

On 28/11/2023 13:56, Christoph Hellwig wrote:
> On Tue, Nov 28, 2023 at 08:56:37AM +0000, John Garry wrote:
>> Are you suggesting some sort of hybrid between the atomic write series you
>> had a few years ago and this solution?
> Very roughly, yes.
> 
>> To me that would be continuing with the following:
>> - per-IO RWF_ATOMIC (and not O_ATOMIC semantics of nothing is written until
>> some data sync)
> Yes.
> 
>> - writes must be a power-of-two and at a naturally-aligned offset
> Where offset is offset in the file? 

ok, fine, it would not be required for XFS with CoW. Some concerns still:
a. device atomic write boundary, if any
b. other FSes which do not have CoW support. ext4 is already being used 
for "atomic writes" in the field - see dubious amazon torn-write prevention.

About b., we could add the pow-of-2 and file offset alignment 
requirement for other FSes, but then need to add some method to 
advertise that restriction.

> It would not require it.  You
> probably want to do it for optimal performance, but requiring it
> feeels rather limited.
> 
>> - relying on atomic write HW support always
> And I think that's where we have different opinions.

I'm just trying to understand your idea and that is not necessarily my 
final opinion.

>  I think the hw
> offload is a nice optimization and we should use it wherever we can.

Sure, but to me it is a concern that we have 2x paths to make robust a. 
offload via hw, which may involve CoW b. no HW support, i.e. CoW always

And for no HW support, if we don't follow the O_ATOMIC model of 
committing nothing until a SYNC is issued, would we allocate, write, and 
later free a new extent for each write, right?

> But building the entire userspace API around it feels like a mistake.
> 

ok, but FWIW it works for the usecases which we know.

>> BTW, we also have rtvol support which does not use forcealign as it already
>> can guarantee alignment, but still does rely on the same principle of
>> requiring alignment - would you want CoW support there also?
> Upstream doesn't have out of place write support for the RT subvolume
> yet.  But Darrick has a series for it and we're actively working on
> upstreaming it.
Yeah, I thought that I heard this.

Thanks,
John