Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32;
Message-ID: <a87d48a7-f2a8-40ae-8d9b-e4534ccc29b1@oracle.com>
Date:   Mon, 4 Dec 2023 15:19:15 +0000
User-Agent: Mozilla Thunderbird
From:   John Garry <john.g.garry@oracle.com>
Subject: Re: [PATCH 17/21] fs: xfs: iomap atomic write support
To:     Christoph Hellwig <hch@lst.de>
Cc:     axboe@kernel.dk, kbusch@kernel.org, sagi@grimberg.me,
        jejb@linux.ibm.com, martin.petersen@oracle.com, djwong@kernel.org,
        viro@zeniv.linux.org.uk, brauner@kernel.org,
        chandan.babu@oracle.com, dchinner@redhat.com,
        linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-nvme@lists.infradead.org, linux-xfs@vger.kernel.org,
        linux-fsdevel@vger.kernel.org, tytso@mit.edu, jbongio@google.com,
        linux-api@vger.kernel.org
References: <20230929102726.2985188-1-john.g.garry@oracle.com>
 <20230929102726.2985188-18-john.g.garry@oracle.com>
 <20231109152615.GB1521@lst.de>
 <a50a16ca-d4b9-a4d8-4230-833d82752bd2@oracle.com>
 <c78bcca7-8f09-41c7-adf0-03b42cde70d6@oracle.com>
 <20231128135619.GA12202@lst.de>
 <e4fb6875-e552-45aa-b193-58f15d9a786c@oracle.com>
 <20231204134509.GA25834@lst.de>
Content-Language: en-US
Organization: Oracle Corporation
In-Reply-To: <20231204134509.GA25834@lst.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?c0I5NTdFSzJqUmZqd3F0RDZQVzNVVUVFcEd3MzdOTnBiQzk1QjVuOFJPQzdO?=
 =?utf-8?B?a0RwKzByRkt2Nms1elVHY2J0R2FMdy8ybFVLeUJ6ZW9kdUs0akJXVlRsM3Rz?=
 =?utf-8?B?ZDdWOUE0QXZpVXpYVDl3Y0cwdUhEQUxBa0xtUG9vVEdnS0VPci9ObGdSWjdD?=
 =?utf-8?B?UCsrUWpkUTRpTk5QNE52bzNTUkNZQ2MyWk5xOVd0MEt0VG9rWTZwRzYwUS84?=
 =?utf-8?B?dlFLazAyQ3RkYnkxQlJhc2VrTnN6QVBhb05mSHhlMU1XM0ZnWHN3WTc3dG1J?=
 =?utf-8?B?S1paTmxyWkJXa0JtS1ZEemEwb3pUZWFwTVFDUHhOcmVkSGpuMXp3Vk1UU0hp?=
 =?utf-8?B?L3FzdnhWSHRlb2c4a0szUlV2Umh1dnZ4dWw2aGpLdW96N2NtZVlyd1ZjbEZD?=
 =?utf-8?B?VUQyYVlpc1o5cDRKQ3NUUmg2OFlXRkpQLzFaRFF2eXNkbmZmZko3SldxUVdZ?=
 =?utf-8?B?blJESEtYL1ZhVkQvU1ovMFVRMXJSbGE3b1JjYUlYbUhVTWo4S0FBVXJrTnoz?=
 =?utf-8?B?WXloN3czL1pzMWJCVWtQaWxBMEFkd1FkNzI0Vi9IYXRlK0t5N1B3RTl3ZHFQ?=
 =?utf-8?B?bWFNRlAzYytvdDRNTEN6eWM0ZkNDT25nSFR5RVc0N1NMY3dkTGRVRmFWMlVQ?=
 =?utf-8?B?MUNoNDR2cG9lSEVRbTljVE5JeGNnak9yaFcxODVkQ01IZWJCUDJIQnJyTjh1?=
 =?utf-8?B?Z0pON043V252ejl3czVlSUIwQ1J2ZFA0UUVydU1TV2tBa2lqdmJ5bGRVY2U1?=
 =?utf-8?B?VGxadmZJVXJpcnZMZ2YyVVgwcmhFaG9PZXR4UUZoVmx0SmFCamNkc2pYenJU?=
 =?utf-8?B?Tk9GTTFmaFErMFNPT2hKRmpoVVVMbjZMYzA1U2NCMnpqejE0OUhLTnZtbzJ6?=
 =?utf-8?B?TEhwS2hUWTZNVkcyZjdkYis2TGpVMUxtS3lzRDZHeVNSWmZQWnYwblk0cE5D?=
 =?utf-8?B?VzE4VXV5dzBaNGZmaGYyVEJwRWtpajJmdjJWY2dCcHFsQkI2QXNaMS9mdllI?=
 =?utf-8?B?QlUrMy9IVkdWeWY5bmVYM1dIcHM1QXV0NlFaOTRpY1YzcHFncGdNRXFWdzVu?=
 =?utf-8?B?WENRRVNySUM0aGJHRlFwSnd2YnRaeFBFMU13a1E4bUJOZUVSNU4yYzNGL1NV?=
 =?utf-8?B?WjlmREJGR2pNQThUayt2ZU43MXFRQ1ludGwycHhZemFlaXlLN21Ja0piVWxT?=
 =?utf-8?B?VDZNSFNsYzA5K0Q5aWhIVHFDZnVMNmcycnpTYUdHNjJFTnhXUXpaRmZmWTlq?=
 =?utf-8?B?R2w3SHdKeGRtZmN5U0YzZXFsRWp6NStEUXpJY1hyaEYzUGZGS3lDOXVLMlE3?=
 =?utf-8?B?Nm51YXNpQitNVVh5Z0dQRlo3SDE2eThPNVZNNkExV1N2WlFKYmNzejVxaTdi?=
 =?utf-8?B?M3VhU0VrdVl2Z3FQdDc2R2FEdEV5U0hoc2lIZG1ocDBzcldqTkUwQk5CRStU?=
 =?utf-8?B?Mm5nbitYWG1oMlVLN21XUjUrcHFpRDNOdERNNHFXallPOFE4UTRPdTkvc2ll?=
 =?utf-8?B?c2Vwb00rZXVsclV3Z3B5Zkk4RWUwRVBWMCtZanMyUDg4QlRTSXZxL3F5RG9H?=
 =?utf-8?B?UCtRcEcySzN5WUliNmx4WUlQVVV6Q2RZZlQwVE4ycEpyaHllVmNIeFRXcnJq?=
 =?utf-8?B?bVo3dGdHd1FFeDhMeDFiRHJPZ1k1enZCWmdOMkNKWG5GWGRSNFNLcHkwN3A2?=
 =?utf-8?B?NXIzYnEzZXJ6bjBmMHBoeGJVWmJreEpBTDhhejN4ZkQ1U2E3K2IrQkcxL25u?=
 =?utf-8?B?WmR5bTFEcUI5NTArenhGQ3hqaUNGc1RCU2RJdE1vamRtZkxYSFBiSHhhQm9j?=
 =?utf-8?B?VENpbXdFaiszZFN1elRFUEdYNWFlTklBKzM1eFc2SmxLTzh1QUhZYm9XUWp2?=
 =?utf-8?B?d3RXUkh2cG5aVWpWd3BNN2hTdlFielQ5MUtYbkVyU3l3ajhHb3pDSVNiakdy?=
 =?utf-8?B?c1pGb0MyU1FwaHV0NmdhTXAvSHJxRWpnT0Q1Sm9TUy9UUnYzQk9PQTI0Szc1?=
 =?utf-8?B?L1R6bXVsb0ZNVllhZnNvNEM3Z3hWUTl0VC9QY3RLVUlWVjJGTTdCNzV3ZU5w?=
 =?utf-8?B?bTd2Zyt4Slc1d2pFdW43blcvM1l5VkJpYm15S2IwMkRaNnQwRFpmNE5kUjk0?=
 =?utf-8?Q?LscVRwJ2BpvNCNwBk1aM/tNd2?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?aVFkRTlSOW9WQlk5bEtNNmZCN0R6RStnMzJWVkxsVVJGS1NuZE5La2dVVk1x?=
 =?utf-8?B?NzFiMUNyZ255WXRmYXVhUkJ2OG1OaW5rNFEzbVpaeHlVNk83bWdCeXBObzJj?=
 =?utf-8?B?RjFhSWNCNHZ4Um9YUDloY0tYc1RMRUVXZEpZTWNhTE5qWFFWQnZpQllTSVZo?=
 =?utf-8?B?YUtMMkNiVkRQMXVCUE9HMncxTVZNaEg1RnU4WVNSTlM4bUxrMGIwMkMvVkpr?=
 =?utf-8?B?QzBHRitpekdmTlJlSlJQTWFkeE84UG9GOFMvQzNwVmxPRFpPbWtkR29SWmdB?=
 =?utf-8?B?bTZiNmpQVmtnNW4vNTF1WGdXRkJaeHRTeG1KdjhGNWhuTmRMYmxkRWVTb3Ax?=
 =?utf-8?B?bkM2N1dha0xlN3dxMG1aTklOSjhaM0cycVdJMTdVMUlVSDdFL2hhK0NGaDQv?=
 =?utf-8?B?TVlOd2hrdjNaeXpxNGs0UGUyY1JDMzNXcmJqdHA0N2JOdWVCZzZwbVBMSlp3?=
 =?utf-8?B?VHpzdFBZOXhvd2tGWmFqRURRSDFhdWRzdDFXb25ReWpRbGdEMElWYVJoVTNW?=
 =?utf-8?B?c0IxdTlxNW5ZMU5GK01XNHRZSGZ6NnUzYi9DZCs5OG5OVXlvUm1oTmpMQ25Z?=
 =?utf-8?B?byt5dFN1SnNLVlY1U0ZMSlZOdGVPWU5ZbTEvL3ZYT3BTcDRxWmVRRjFQRnRI?=
 =?utf-8?B?RTB5U1dOajFBcUpnckMwTk4xZWZubSs3K0Q2WDZxdHRVaWZSTEdUeHpOSFBG?=
 =?utf-8?B?OUlnWmZhaE9lTzdSanZNVlV2SHNXS0NPMGQ1REcxU05kVUxCTHY1V1pFSzVF?=
 =?utf-8?B?VW1BQ0xBQ1EwZkNDWE5JeWJ1cXY3ODBIS2hYL3FZYTlzMnh6WUVTeWZSSXVp?=
 =?utf-8?B?YUJITG1pejA2UzNJWWswNW1MY0lvcHAyUnZWSHRpdmxwdFZuOHArOVhuN0JE?=
 =?utf-8?B?UnU3SFJxUFcwYjdtVHVLbGlvZnc2QzAraTBOT2JZL1RsSEd0MzFNcHZsU2pO?=
 =?utf-8?B?UmlQVlF4eUhtTXN5Q1J1TW1HRDJXTWRrMzFIb3ZHVjJKVGUvZ0xOekpsNDZr?=
 =?utf-8?B?Y212MmVEZkJXRExBaS9lTmtNcUEwU3JrdWtlV0ZxRWU0Tnc2SGNMaDI4OU1J?=
 =?utf-8?B?cno2UklSYU1PUkJwblQ2SlY3Y2RDVFkwSld0QkZndS9yYm1GVXRGU3R6MENu?=
 =?utf-8?B?QXVqNGphSHdZOEdQR2JQZGEyMGdyNitGb1pmMGw4aDlBMU5mVjNmQVdpVkpZ?=
 =?utf-8?B?WU1hb2lqeW5vazVBUGNjR0dWSEE0WERab1kxQWI0OEhqOENxeVphcHZ5R0dz?=
 =?utf-8?B?YlRoaUQ1NXF5TmpPbUlnMk5CQ295REcvenlidUxZSXFPbFhaMGRZK1ptdnVm?=
 =?utf-8?B?OHdUZi9GOEordFNhbGFFcjBxWDhGTUZrb1Mvd281N0ZwMnBlMjBYUVNsUlpj?=
 =?utf-8?B?enE4YUc3c0IvV0VzVjMxMGZnTWpoSnd6UnJkdU9nK3h3Kzcwb2pxdkUzMENO?=
 =?utf-8?B?bTRJVjBLbmNHaHlxbW51NndQUXNBK2c3bWRIV3JBPT0=?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1bb691e6-7834-42fe-356b-08dbf4dc630d
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Dec 2023 15:19:19.7071
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: aiCW0Ldj54Ug3pcZPASH1Oyjv34cSGJB8p/lfAIxytxjMusRnOHYQIAT3daFkIsC9ObvVMzvBqHxV9TpGctA9A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB5928
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2023-12-04_14,2023-12-04_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxlogscore=999
 suspectscore=0 malwarescore=0 adultscore=0 phishscore=0 bulkscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311060000 definitions=main-2312040115
X-Proofpoint-GUID: SlkNmeT4xmHaCe-VR9s6IAvIiOtMc-iS
X-Proofpoint-ORIG-GUID: SlkNmeT4xmHaCe-VR9s6IAvIiOtMc-iS
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 04 Dec 2023 07:20:04 -0800 (PST)

On 04/12/2023 13:45, Christoph Hellwig wrote:
> On Tue, Nov 28, 2023 at 05:42:10PM +0000, John Garry wrote:
>> ok, fine, it would not be required for XFS with CoW. Some concerns still:
>> a. device atomic write boundary, if any
>> b. other FSes which do not have CoW support. ext4 is already being used for
>> "atomic writes" in the field - see dubious amazon torn-write prevention.
> 
> What is the 'dubious amazon torn-write prevention'?

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/storage-twp.html

AFAICS, this is without any kernel changes, so no guarantee of unwanted 
splitting or merging of bios.

Anyway, there will still be !CoW FSes which people want to support.

> 
>> About b., we could add the pow-of-2 and file offset alignment requirement
>> for other FSes, but then need to add some method to advertise that
>> restriction.
> 
> We really need a better way to communicate I/O limitations anyway.
> Something like XFS_IOC_DIOINFO on steroids.
> 
>> Sure, but to me it is a concern that we have 2x paths to make robust a.
>> offload via hw, which may involve CoW b. no HW support, i.e. CoW always
> 
> Relying just on the hardware seems very limited, especially as there is
> plenty of hardware that won't guarantee anything larger than 4k, and
> plenty of NVMe hardware without has some other small limit like 32k
> because it doesn't support multiple atomicy mode.

So what would you propose as the next step? Would it to be first achieve 
atomic write support for XFS with HW support + CoW to ensure contiguous 
extents (and without XFS forcealign)?

> 
>> And for no HW support, if we don't follow the O_ATOMIC model of committing
>> nothing until a SYNC is issued, would we allocate, write, and later free a
>> new extent for each write, right?
> 
> Yes. Then again if you do data journalling you do that anyway, and as
> one little project I'm doing right now shows that data journling is
> often the fastest thing we can do for very small writes.

Ignoring FSes, then how is this supposed to work for block devices? We 
just always need HW support, right?

Thanks,
John