Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley
        Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United
        Kingdom.
        Registered in England and Wales under Company Registration No. 3798903
From:   David Howells <dhowells@redhat.com>
In-Reply-To: <da9200f1bded9b8b078a7aef227fd6b92eb028fb.camel@hammerspace.com>
References: <da9200f1bded9b8b078a7aef227fd6b92eb028fb.camel@hammerspace.com> <20220824093501.384755-1-dwysocha@redhat.com> <20220824093501.384755-3-dwysocha@redhat.com> <429ecc819fcffe63d60dbb2b72f9022d2a21ddd8.camel@hammerspace.com> <CALF+zOknvMZyufSUD-g9Z9Y5RfwE-vUFT+CF0kxqbcpR=yJPJw@mail.gmail.com>
To:     Trond Myklebust <trondmy@hammerspace.com>
Cc:     dhowells@redhat.com, willy@infradead.org,
        "dwysocha@redhat.com" <dwysocha@redhat.com>,
        "linux-cachefs@redhat.com" <linux-cachefs@redhat.com>,
        "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>,
        "daire.byrne@gmail.com" <daire.byrne@gmail.com>,
        "anna.schumaker@netapp.com" <anna.schumaker@netapp.com>,
        "benmaynard@google.com" <benmaynard@google.com>
Subject: Re: [RFC PATCH 2/3] NFS: Add support for netfs in struct nfs_inode and Kconfig
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <216679.1661350326.1@warthog.procyon.org.uk>
Date:   Wed, 24 Aug 2022 15:12:06 +0100
Message-ID: <216681.1661350326@warthog.procyon.org.uk>
Precedence: bulk

Trond Myklebust <trondmy@hammerspace.com> wrote:

> As long as it is an opt-in feature, I'm OK. I don't want to have to
> compile it in by default.
> A cachefs should never become a mandatory feature of networked
> filesystems.

netfslib is intended to be used even if fsache is not enabled.  It is intended
to make the underlying network filesystem maintainer's life easier by:

 - Moving the implementation of all the VM ops from the network filesystems as
   much as possible into one place.  The network filesystem then just has to
   provide a read op and a write op.

 - Making it such that the filesystem doesn't have to deal with the difference
   between DIO and buffered I/O

 - Handling VM features on behalf of all filesystems.  This gives the VM folk
   one place to change instead of 5+.  mpage and iomap are similar things but
   for blockdev filesystems.

 - Providing features to those filesystems that can support them, eg.:

   - fscrypt
   - compression
   - bounce buffering
   - local caching
   - disconnected operation


Currently nfs interacts with fscache on a page-by-page basis, but this needs
to change:

 (1) Multipage folios are now a thing.  You need to roll folios out into nfs
     if you're going to take advantage of this.  Also, you may have noticed
     that all the VM interfaces are being recast in terms of folios.

 (2) I need to fix the cache so that it no longer uses the backing
     filesystem's metadata to track content.  To make this use less diskspace,
     I want to increase the cache block size to, say, 256K or 2M.

     This means that the cache needs to have a say in how big a read the
     network filesystem does - and also that a single cache request might need
     to be stitched together from multiple read ops.

 (3) More pagecache changes are lurking in the future, possibly including
     getting rid of the concept of pages entirely from the pagecache.

There are users of nfs + fscache and we'd like to continue to support them as
best as possible but the current code noticably degrades performance here.

Unfortunately, I'm also going to need to drop the fallback interface which nfs
currently uses in the next couple versions, we have to at least get the
fscache enabled conversion done.

I've been dealing with the VM, 9p, ceph and cifs people over the direction
that netfslib might need to go in, but for nfs, it's typically been a flat
"no".  I would like to work out how to make netfslib work for nfs also, if
you're willing to discuss it.

I would be open to having a look at importing nfs page handling into netfslib
and working from that - but it still needs to deal with (1) and (2) above, and
I would like to make it pass iterators down to the lower layers as buffer
descriptions.  It's also very complicated stuff.

Also:

 - I've noted the nfs_page structs that nfs uses and I'm looking at a way of
   having something similar, but held separately so that one struct can span
   and store information about multiple folios.

 - I'm looking at punting write-to-the-cache to writepages() or something like
   that so that the VM folks can reclaim the PG_private_2 flag bit, so that
   won't be available to nfs either in the future.

 - aops->write_begin() and ->write_end() are going to go away.  In netfslib
   what I'm trying to do is make a "netfs_perform_write" as a parallel to
   generic_perform_write().

David