Received-SPF: pass (google.com: domain of linux-kernel+bounces-175701-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
From: Christian Brauner <brauner@kernel.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Christian Brauner <brauner@kernel.org>,
	linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: [GIT PULL] vfs netfs
Date: Fri, 10 May 2024 13:47:02 +0200
Message-ID: <20240510-vfs-netfs-f805bdd4c8ad@brauner>
Precedence: bulk
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Hey Linus,

/* Summary */
This reworks the netfslib writeback implementation so that pages read
from the cache are written to the cache through ->writepages(), thereby
allowing the fscache page flag to be retired.

The reworking also:

 (1) builds on top of the new writeback_iter() infrastructure;

 (2) makes it possible to use vectored write RPCs as discontiguous streams
     of pages can be accommodated;

 (3) makes it easier to do simultaneous content crypto and stream division.

 (4) provides support for retrying writes and re-dividing a stream;

 (5) replaces the ->launder_folio() op, so that ->writepages() is used
     instead;

 (6) uses mempools to allocate the netfs_io_request and netfs_io_subrequest
     structs to avoid allocation failure in the writeback path.

Some code that uses the fscache page flag is retained for compatibility
purposes with nfs and ceph. The code is switched to using the synonymous
private_2 label instead and marked with deprecation comments.

The merge commit contains additional details on the new algorithm that
I've left out of here as it would probably be excessively detailed.

On top of the netfslib infrastructure this contains the work to convert
cifs over to netfslib.

/* Testing */
clang: Debian clang version 16.0.6 (26)
gcc: (Debian 13.2.0-24)

All patches are based on v6.9-rc6 and have been sitting in linux-next.
No build failures or warnings were observed.

/* Conflicts */

No known conflicts.

The following changes since commit e67572cd2204894179d89bd7b984072f19313b03:

  Linux 6.9-rc6 (2024-04-28 13:47:24 -0700)

are available in the Git repository at:

  git@gitolite.kernel.org:pub/scm/linux/kernel/git/vfs/vfs tags/vfs-6.10.netfs

for you to fetch changes up to e2bc9f6cfbd62c72a93a70068daab8886bec32ce:

  Merge branch 'cifs-netfs' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs (2024-05-02 11:23:31 +0200)

Please consider pulling these changes from the signed vfs-6.10.netfs tag.

Thanks!
Christian

----------------------------------------------------------------
vfs-6.10.netfs

----------------------------------------------------------------
Christian Brauner (2):
      Merge branch 'netfs-writeback' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs
      Merge branch 'cifs-netfs' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs

David Howells (38):
      netfs: Update i_blocks when write committed to pagecache
      netfs: Replace PG_fscache by setting folio->private and marking dirty
      mm: Remove the PG_fscache alias for PG_private_2
      netfs: Remove deprecated use of PG_private_2 as a second writeback flag
      netfs: Make netfs_io_request::subreq_counter an atomic_t
      netfs: Use subreq_counter to allocate subreq debug_index values
      mm: Provide a means of invalidation without using launder_folio
      9p: Use alternative invalidation to using launder_folio
      afs: Use alternative invalidation to using launder_folio
      netfs: Remove ->launder_folio() support
      netfs: Use mempools for allocating requests and subrequests
      mm: Export writeback_iter()
      netfs: Switch to using unsigned long long rather than loff_t
      netfs: New writeback implementation
      netfs: Add some write-side stats and clean up some stat names
      netfs, afs: Implement helpers for new write code
      netfs, 9p: Implement helpers for new write code
      netfs, cachefiles: Implement helpers for new write code
      netfs: Cut over to using new writeback code
      netfs: Remove the old writeback code
      netfs: Miscellaneous tidy ups
      netfs, afs: Use writeback retry to deal with alternate keys
      cifs: Use alternative invalidation to using launder_folio
      cifs: Replace cifs_readdata with a wrapper around netfs_io_subrequest
      cifs: Replace cifs_writedata with a wrapper around netfs_io_subrequest
      cifs: Use more fields from netfs_io_subrequest
      cifs: Make wait_mtu_credits take size_t args
      cifs: Replace the writedata replay bool with a netfs sreq flag
      cifs: Move cifs_loose_read_iter() and cifs_file_write_iter() to file.c
      cifs: Set zero_point in the copy_file_range() and remap_file_range()
      cifs: Add mempools for cifs_io_request and cifs_io_subrequest structs
      cifs: Make add_credits_and_wake_if() clear deducted credits
      cifs: Implement netfslib hooks
      cifs: Cut over to using netfslib
      cifs: Remove some code that's no longer used, part 1
      cifs: Remove some code that's no longer used, part 2
      cifs: Remove some code that's no longer used, part 3
      cifs: Enable large folio support

 fs/9p/vfs_addr.c             |   60 +-
 fs/afs/file.c                |    8 +-
 fs/afs/internal.h            |    6 +-
 fs/afs/validation.c          |    4 +-
 fs/afs/write.c               |  189 +--
 fs/cachefiles/io.c           |   76 +-
 fs/ceph/addr.c               |   24 +-
 fs/ceph/inode.c              |    2 +
 fs/netfs/Makefile            |    3 +-
 fs/netfs/buffered_read.c     |   40 +-
 fs/netfs/buffered_write.c    |  829 ++-----------
 fs/netfs/direct_write.c      |   56 +-
 fs/netfs/fscache_io.c        |   14 +-
 fs/netfs/internal.h          |   55 +-
 fs/netfs/io.c                |  162 +--
 fs/netfs/main.c              |   55 +-
 fs/netfs/misc.c              |   10 +-
 fs/netfs/objects.c           |   81 +-
 fs/netfs/output.c            |  478 --------
 fs/netfs/stats.c             |   17 +-
 fs/netfs/write_collect.c     |  808 +++++++++++++
 fs/netfs/write_issue.c       |  684 +++++++++++
 fs/nfs/file.c                |    8 +-
 fs/nfs/fscache.h             |    6 +-
 fs/nfs/write.c               |    4 +-
 fs/smb/client/Kconfig        |    1 +
 fs/smb/client/cifsfs.c       |  124 +-
 fs/smb/client/cifsfs.h       |   11 +-
 fs/smb/client/cifsglob.h     |   65 +-
 fs/smb/client/cifsproto.h    |   12 +-
 fs/smb/client/cifssmb.c      |  120 +-
 fs/smb/client/file.c         | 2720 ++++++------------------------------------
 fs/smb/client/fscache.c      |  109 --
 fs/smb/client/fscache.h      |   54 -
 fs/smb/client/inode.c        |   45 +-
 fs/smb/client/smb2ops.c      |   10 +-
 fs/smb/client/smb2pdu.c      |  186 +--
 fs/smb/client/smb2proto.h    |    5 +-
 fs/smb/client/trace.h        |  144 ++-
 fs/smb/client/transport.c    |   17 +-
 include/linux/fscache.h      |   22 +-
 include/linux/netfs.h        |  197 +--
 include/linux/pagemap.h      |    2 +
 include/net/9p/client.h      |    2 +
 include/trace/events/netfs.h |  250 +++-
 mm/filemap.c                 |   60 +-
 mm/page-writeback.c          |    1 +
 net/9p/Kconfig               |    1 +
 net/9p/client.c              |   49 +
 49 files changed, 3298 insertions(+), 4588 deletions(-)
 delete mode 100644 fs/netfs/output.c
 create mode 100644 fs/netfs/write_collect.c
 create mode 100644 fs/netfs/write_issue.c