Received: by 2002:ab2:1347:0:b0:1f4:ac9d:b246 with SMTP id g7csp127327lqg; Wed, 10 Apr 2024 18:49:28 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCUoilZf+Od/+2cpXuRWZAUetw0wpQOvWh2ZS2Cnr+sSxPifbubzTFs9Bp33JCi1Tuh1IDvlGvVxtRSb8qIsy7aoi0UD1GsUo8+V7GiUHA== X-Google-Smtp-Source: AGHT+IFQGKcc/WjwppNWODtMHs+JJ+Fe//qL7CgvzGrXiB+a/ayhUALJUqDBYm0lQy03VzuDOdKl X-Received: by 2002:a05:6870:ecaa:b0:22e:e0fa:4c36 with SMTP id eo42-20020a056870ecaa00b0022ee0fa4c36mr5035087oab.55.1712800167926; Wed, 10 Apr 2024 18:49:27 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1712800167; cv=pass; d=google.com; s=arc-20160816; b=NA3Xb6phUX3+CwWa6aPUxvMvUMF7Q2DFqYZJGszQ0uHc8nY4d4bSDmie9CrfHlRSF1 mDJvVGrtsRbz5fEuHzXVrOrUDu6l20L8wmkjdUhqCgRo9oYXbw1GTYc29Ssj5N+y5YJU BOpKgj4PD1jAlvWbm4b58PgZaMjt6CALJ6SvT5UISb3eLuWI9ioqvVxTDV6lpoy99pbN d1n54zr0GYOpp/rZtaJwSYTLmeEcYpITvnAheEPZaDCNd8tr9r/x46qUJd1pw8rrCd67 a3a10Wvf2uwzA0TtSAiU7Zt+BFahNRC5P5GctuGfDiBzyVm0nlh4qTXwXEF4H+3a4UNd 4tFw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:in-reply-to:content-disposition:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:references :message-id:subject:cc:to:from:date:dkim-signature; bh=0y/StUI641o9BanPyuvxNVnM75yOu8dqMCwLb08ZeaU=; fh=OQDVtGOtl8Rc71Vy8uxnVXzS2j6EneKc33cVKDU/tsM=; b=FL/vL16chyi9fb5dr9gTQje5/q38XS1JSmoBcfwjF0gLCR/uVQx5JxOWu2myv2EWTI fIyGPQZwZctvY2oBaQjS3fxE8v4p/kkDzurvWlNjmryei93AVaWym/gAjIVg7S9Pe6VP J2WPn9z8ebdOCByc2LgNayTcBNbcwkKyxwHPYFhDYGApexw4R0mJNMKYZ7vwYcWHx8fm ohd5MS1VTJXU3cYxhETGm4XsPGJgl48BeSSceCKXLxdpKzPQCfX7cZ8UZBkTK9aZWPH+ PMI6xgdn8MSoQD+ZZVAP3utng5+4YZ+pfQq5mSEclwiJXYJD89K7QcJlCIT3Zb5MH9m9 MRZQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@infradead.org header.s=bombadil.20210309 header.b=jzSUeYbF; arc=pass (i=1 dkim=pass dkdomain=infradead.org); spf=pass (google.com: domain of linux-kernel+bounces-139590-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-139590-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id y15-20020a056a00190f00b006eaae364749si467744pfi.182.2024.04.10.18.49.27 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 18:49:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-139590-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=bombadil.20210309 header.b=jzSUeYbF; arc=pass (i=1 dkim=pass dkdomain=infradead.org); spf=pass (google.com: domain of linux-kernel+bounces-139590-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-139590-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 8AE93B24740 for ; Thu, 11 Apr 2024 00:39:11 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C24D3D51D; Thu, 11 Apr 2024 00:38:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="jzSUeYbF" Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58D4D946F; Thu, 11 Apr 2024 00:38:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712795925; cv=none; b=Z0gp6J/sxA47O1Ixvev/36tZjnsb5xY0uLFtATuDYmFGUeqWYLzpaRwuUnVkVGrrlUUyZ9ONWgczWsBiubotEcSKaghhntXSDOOddhglih4VoYCtHrSdxleTJwnGpco+Sg5x41ziAMGXSm04xywyTfd9iyUlG9CPNLtEuMXjurU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712795925; c=relaxed/simple; bh=W0yvgk4HYg+/L0IQ8HK3ESN9oPLBuDLxptzYSWkPpc8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=IrWKqjaq2qoDVREvY93Zoa/KkDdv7Fc2qpic7931mPs9eQHbimFyMQ4KixJif6/j80KG2kfA0jxdiYFJN4jGTl+749LJTS74vMKy7EbA/d27DgNzDwfIT7+hRXmaoGFaLD7SxJ2Y+K8EwJJwtg7Qs9hgmFZzM1GMQf2PApLiASc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=jzSUeYbF; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=0y/StUI641o9BanPyuvxNVnM75yOu8dqMCwLb08ZeaU=; b=jzSUeYbF76eftu2Uxlpd+g8UyC bCYxXfoUcQkccIgXePFUpxdaD64xV2V+85xXNe01q/k7YMQD0kftP0eIxahavEwTuP7YHE1Y/Jzrd w0qPY5t6CA/16uxsQkLrROrSuORCKI0bg+FCj8ZVmKFW5DWXssbk+H5extAUhDLqB5l7K3IobopA3 Hr9Ji3H4tUwQNQ9znK8p6AAsahJ5SyJjBk5Dysa38GRbhIB3BhjzAFV909GHjzTaCO27imevzm8EZ OKbQpsokz0960Fa5N6uzK9kYA4pE9mDE+79A2/SpAxH2wWTvncTA2jSrVUsjXAeNZL8E/2qD1MV8r jrPWnUmg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1ruiSa-00000009h9y-0wFj; Thu, 11 Apr 2024 00:38:36 +0000 Date: Wed, 10 Apr 2024 17:38:36 -0700 From: Luis Chamberlain To: Hannes Reinecke Cc: Matthew Wilcox , John Garry , Pankaj Raghav , Daniel Gomez , Javier =?iso-8859-1?Q?Gonz=E1lez?= , axboe@kernel.dk, kbusch@kernel.org, hch@lst.de, sagi@grimberg.me, jejb@linux.ibm.com, martin.petersen@oracle.com, djwong@kernel.org, viro@zeniv.linux.org.uk, brauner@kernel.org, dchinner@redhat.com, jack@suse.cz, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, tytso@mit.edu, jbongio@google.com, linux-scsi@vger.kernel.org, ojaswin@linux.ibm.com, linux-aio@kvack.org, linux-btrfs@vger.kernel.org, io-uring@vger.kernel.org, nilay@linux.ibm.com, ritesh.list@gmail.com Subject: Re: [PATCH v6 00/10] block atomic writes Message-ID: References: <20240326133813.3224593-1-john.g.garry@oracle.com> <94d6d88b-b0e7-491d-94e8-dc9e5fba5620@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <94d6d88b-b0e7-491d-94e8-dc9e5fba5620@suse.de> Sender: Luis Chamberlain On Wed, Apr 10, 2024 at 08:20:37AM +0200, Hannes Reinecke wrote: > On 4/10/24 06:05, Matthew Wilcox wrote: > > On Mon, Apr 08, 2024 at 10:50:47AM -0700, Luis Chamberlain wrote: > > > On Fri, Apr 05, 2024 at 11:06:00AM +0100, John Garry wrote: > > > > On 04/04/2024 17:48, Matthew Wilcox wrote: > > > > > > > The thing is that there's no requirement for an interface as complex as > > > > > > > the one you're proposing here. I've talked to a few database people > > > > > > > and all they want is to increase the untorn write boundary from "one > > > > > > > disc block" to one database block, typically 8kB or 16kB. > > > > > > > > > > > > > > So they would be quite happy with a much simpler interface where they > > > > > > > set the inode block size at inode creation time, > > > > > > We want to support untorn writes for bdev file operations - how can we set > > > > > > the inode block size there? Currently it is based on logical block size. > > > > > ioctl(BLKBSZSET), I guess? That currently limits to PAGE_SIZE, but I > > > > > think we can remove that limitation with the bs>PS patches. > > > > > > I can say a bit more on this, as I explored that. Essentially Matthew, > > > yes, I got that to work but it requires a set of different patches. We have > > > what we tried and then based on feedback from Chinner we have a > > > direction on what to try next. The last effort on that front was having the > > > iomap aops for bdev be used and lifting the PAGE_SIZE limit up to the > > > page cache limits. The crux on that front was that we end requiring > > > disabling BUFFER_HEAD and that is pretty limitting, so my old > > > implementation had dynamic aops so to let us use the buffer-head aops > > > only when using filesystems which require it and use iomap aops > > > otherwise. But as Chinner noted we learned through the DAX experience > > > that's not a route we want to again try, so the real solution is to > > > extend iomap bdev aops code with buffer-head compatibility. > > > > Have you tried just using the buffer_head code? I think you heard bad > > advice at last LSFMM. Since then I've landed a bunch of patches which > > remove PAGE_SIZE assumptions throughout the buffer_head code, and while > > I haven't tried it, it might work. And it might be easier to make work > > than adding more BH hacks to the iomap code. > > > > A quick audit for problems ... > > > > __getblk_slow: > > if (unlikely(size & (bdev_logical_block_size(bdev)-1) || > > (size < 512 || size > PAGE_SIZE))) { > > > > cont_expand_zero (not used by bdev code) > > cont_write_begin (ditto) > > > > That's all I spot from a quick grep for PAGE, offset_in_page() and kmap. > > > > You can't do a lot of buffer_heads per folio, because you'll overrun > > struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE]; > > in block_read_full_folio(), but you can certainly do _one_ buffer_head > > per folio, and that's all you need for bs>PS. > > > Indeed; I got a patch here to just restart the submission loop if one > reaches the end of the array. But maybe submitting one bh at a time and > using plugging should achieve that same thing. Let's see. That's great to hear, what about a target filesystem? Without a buffer-head filesystem to test I'm not sure we'd get enough test coverage. The block device cache isn't exaclty a great filesystem target to test correctness. > > > I suspect this is a use case where perhaps the max folio order could be > > > set for the bdev in the future, the logical block size the min order, > > > and max order the large atomic. > > > > No, that's not what we want to do at all! Minimum writeback size needs > > to be the atomic size, otherwise we have to keep track of which writes > > are atomic and which ones aren't. So, just set the logical block size > > to the atomic size, and we're done. > > > +1. My thoughts all along. Oh, hrm yes, but let's test it out then... Luis