Received-SPF: pass (google.com: domain of linux-ext4+bounces-2582-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
From: Harshad Shirwadkar <harshadshirwadkar@gmail.com>
To: linux-ext4@vger.kernel.org
Cc: tytso@mit.edu,
	saukad@google.com,
	harshads@google.com,
	Harshad Shirwadkar <harshadshirwadkar@gmail.com>
Subject: [PATCH 00/10] Ext4 fast commit performance patch series
Date: Mon, 20 May 2024 05:51:43 +0000
Message-ID: <20240520055153.136091-1-harshadshirwadkar@gmail.com>
Precedence: bulk
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This is the V5 of the patch series. This patch series supersedes the patch
series "ext4: remove journal barrier during fast commit" sent in July 2022[1].
Except following three patches all the other patches are newly introduced in
this series.

* ext4: convert i_fc_lock to spinlock
* ext4: for committing inode, make ext4_fc_track_inode wait
* ext4: drop i_fc_updates from inode fc info

This patchset does not introduce any on-disk format and / or user visible API
change. This patchset reworks fast commit commit path improve overall
performance of the commit path. Following optimizations have been added in this
series:

* Avoid having to lock the journal throughout the fast commit.
* Remove tracking of open handles per inode.
* Avoid issuing cache flushes when fast commits are contained within a single
  FUA writes and there is no data that needs flushing.
* Temporarily lift committing thread's priority to match that of the journal
  thread to reduce scheduling delays.

With the changes introduced in this patch series, now the commit path for fast
commits is as follows:

1. Lock the journal by calling jbd2_journal_lock_updates_no_rsv(). This ensures
   that all the exsiting handles finish and no new handles can start.
2. Mark all the fast commit eligible inodes as undergoing fast commit
   by setting "EXT4_STATE_FC_COMMITTING" state.
3. Unlock the journal by calling jbd2_journal_unlock_updates. This allows
   starting of new handles. If new handles try to start an update on any of the
   inodes that are being committed, ext4_fc_track_inode() will block until
   those inodes have finished the fast commit.
4. Submit data buffers of all the committing inodes.
5. Wait for [4] to complete.
6. Commit all the directory entry updates in the fast commit space.
7. Commit all the changed inodes in the fast commit space and clear
   "EXT4_STATE_FC_COMMITTING" for all the inodes.
8. Write tail tag to ensure atomicity of commits.

(The above flow has been documented in the code as well)

I verified that the patch series introduces no regressions in "log" groups when
"fast_commit" feature is enabled.

Also, we have a paper on fast commits in USENIX ATC 2024 this year which should
become available on the website[2] in a few months.

[1] https://lwn.net/Articles/902022/
[2] https://www.usenix.org/conference/atc24

Harshad Shirwadkar (10):
  ext4: convert i_fc_lock to spinlock
  ext4: for committing inode, make ext4_fc_track_inode wait
  ext4: mark inode dirty before grabbing i_data_sem in ext4_setattr
  ext4: rework fast commit commit path
  ext4: drop i_fc_updates from inode fc info
  ext4: update code documentation
  ext4: add nolock mode to ext4_map_blocks()
  ext4: introduce selective flushing in fast commit
  ext4: temporarily elevate commit thread priority
  ext4: make fast commit ineligible on ext4_reserve_inode_write failure

 fs/ext4/ext4.h              |  29 ++--
 fs/ext4/ext4_jbd2.h         |  20 +--
 fs/ext4/fast_commit.c       | 279 ++++++++++++++++++++----------------
 fs/ext4/fast_commit.h       |   1 +
 fs/ext4/inline.c            |   3 +
 fs/ext4/inode.c             |  53 ++++---
 fs/ext4/super.c             |   7 +-
 fs/jbd2/journal.c           |   2 -
 fs/jbd2/transaction.c       |  41 ++++--
 include/linux/jbd2.h        |   1 +
 include/trace/events/ext4.h |   7 +-
 11 files changed, 265 insertions(+), 178 deletions(-)

-- 
2.45.0.rc1.225.g2a3ae87e7f-goog