From: Ric Wheeler Subject: Re: [PATCH v6 0/4] ext4: Coordinate data-only flush requests sent by fsync Date: Mon, 29 Nov 2010 19:48:52 -0500 Message-ID: <4CF449F4.3010303@redhat.com> References: <20101129220536.12401.16581.stgit@elm3b57.beaverton.ibm.com> <20101130113906.176ffcad@notabene.brown> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "Darrick J. Wong" , Jens Axboe , "Theodore Ts'o" , Andreas Dilger , Alasdair G Kergon , Jan Kara , Mike Snitzer , linux-kernel , linux-raid@vger.kernel.org, Keith Mannthey , dm-devel@redhat.com, Mingming Cao , Tejun Heo , linux-ext4@vger.kernel.org, Christoph Hellwig , Josef Bacik To: Neil Brown Return-path: Received: from mx1.redhat.com ([209.132.183.28]:49158 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755904Ab0K3ArP (ORCPT ); Mon, 29 Nov 2010 19:47:15 -0500 In-Reply-To: <20101130113906.176ffcad@notabene.brown> Sender: linux-ext4-owner@vger.kernel.org List-ID: On 11/29/2010 07:39 PM, Neil Brown wrote: > On Mon, 29 Nov 2010 14:05:36 -0800 "Darrick J. Wong" > wrote: > >> On certain types of hardware, issuing a write cache flush takes a considerable >> amount of time. Typically, these are simple storage systems with write cache >> enabled and no battery to save that cache after a power failure. When we >> encounter a system with many I/O threads that write data and then call fsync >> after more transactions accumulate, ext4_sync_file performs a data-only flush, >> the performance of which is suboptimal because each of those threads issues its >> own flush command to the drive instead of trying to coordinate the flush, >> thereby wasting execution time. >> >> Instead of each fsync call initiating its own flush, there's now a flag to >> indicate if (0) no flushes are ongoing, (1) we're delaying a short time to >> collect other fsync threads, or (2) we're actually in-progress on a flush. >> >> So, if someone calls ext4_sync_file and no flushes are in progress, the flag >> shifts from 0->1 and the thread delays for a short time to see if there are any >> other threads that are close behind in ext4_sync_file. After that wait, the >> state transitions to 2 and the flush is issued. Once that's done, the state >> goes back to 0 and a completion is signalled. > I haven't seen any of the preceding discussion do I might be missing > something important, but this seems needlessly complex and intrusive. > In particular, I don't like adding code to md to propagate these timings up > to the fs, and I don't the arbitrary '2ms' number. > > Would it not be sufficient to simply gather flushes while a flush is pending. > i.e > - if no flush is pending, set the 'flush pending' flag, submit a flush, > then clear the flag. > - if a flush is pending, then wait for it to complete, and then submit a > single flush on behalf of all pending flushes. > > That way when flush is fast, you do a flush every time, and when it is slow > you gather multiple flushes together. > I think it would issues a few more flushes than your scheme, but it would be > a much neater solution. Have you tried that and found it to be insufficient? > > Thanks, > NeilBrown > The problem with that is that you can introduce a wait for the next flush longer than it would take to complete the flush. Having the wait adjust itself according to the speed of the device is much better I think.... Ric