Date: Fri, 1 Sep 2017 15:42:44 -0400
From: "J. Bruce Fields" <bfields@redhat.com>
To: Olga Kornievskaia <kolga@netapp.com>
Cc: linux-nfs@vger.kernel.org
Subject: Re: [RFC v1 00/17] NFSD support for inter+async COPY
Message-ID: <20170901194244.GB27922@parsley.fieldses.org>
References: <20170302160142.30413-1-kolga@netapp.com>
 <20170901194130.GA27922@parsley.fieldses.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
In-Reply-To: <20170901194130.GA27922@parsley.fieldses.org>
Sender: linux-nfs-owner@vger.kernel.org

On Fri, Sep 01, 2017 at 03:41:30PM -0400, J. Bruce Fields wrote:
> Apologies, remind me:
> 
> 	- does wireshark have support for all of this?  (Or are there
> 	  patches?)
> 	- How are you testing?
> 	- What's the status of the client side?
> 	- Do we have any user documentation?  Does the client or server
> 	  administrator need to do any special setup, or is it just a
> 	  matter of exporting and mounting over 4.1 and calling

(Sorry, 4.2!)

--b.

> 	  copy_file_range()?
> 	- what currently happens if you try to copy across krb5 mounts?
> 
> --b.
> 
> On Thu, Mar 02, 2017 at 11:01:24AM -0500, Olga Kornievskaia wrote:
> > This is server-side support for NFSv4.2 inter and async COPY which is
> > on top of existing intra sync COPY. It also depends on the NFS client
> > piece for NFSv4.2 to do client side of the destination server piece
> > in the inter SSC.
> > 
> > NFSD determines if COPY is intra or inter and if sync or async. For
> > inter, NSFD uses NFSv4.1 protocol and creates an internal mount point
> > (superblock). It will destroy the mount point when copy is done.
> > 
> > To do asynchronous copies, NFSD creates a single threaded workqueue
> > and does not tie up an NFSD thread to complete the copy. Upon receiving
> > the COPY, it generates a unique copy stateid (stores a global list
> > for keeping track of state for OFFLOAD_STATUS to be queried by),
> > queues up a workqueue for the copy, and replies back to the client.
> > nfsd4_copy arguments that are allocated on the stack are copied for
> > the work item.
> > 
> > In the async copy handler, it calls into VFS copy_file_range() with
> > 4MB chunks and loops until it completes the requested copy size. If
> > error is encountered it's saved but also we save the amount of data
> > copied so far. Once done, the results are queued for the callback
> > workqueue and sent via CB_OFFLOAD. Also currently, choosing to clean
> > up the copy state information stored in the global list when cope is
> > done and not doing it when callback's release function (it could be
> > done there alternatively if needed it?).
> > 
> > On the source server, upon receiving a COPY_NOTIFY, it generate a
> > unique stateid that's kept in the global list. Upon receiving a READ
> > with a stateid, the code checks the normal list of open stateid and
> > now additionally, it'll check the copy state list as well before
> > deciding to either fail with BAD_STATEID or find one that matches.
> > The stored stateid is only valid to be used for the first time
> > with a choosen lease period (90s currently). When the source server
> > received an OFFLOAD_CANCEL, it will remove the stateid from the
> > global list. Otherwise, the copy stateid is removed upon the removal
> > of its "parent" stateid (open/lock/delegation stateid).
> > 
> > 
> > Andy Adamson (7):
> >   NFSD add ca_source_server<> to COPY
> >   NFSD generalize nfsd4_compound_state flag names
> >   NFSD: allow inter server COPY to have a STALE source server fh
> >   NFSD return nfs4_stid in nfs4_preprocess_stateid_op
> >   NFSD add COPY_NOTIFY operation
> >   NFSD add nfs4 inter ssc to nfsd4_copy
> >   NFSD Unique stateid_t for inter server to server COPY authentication
> > 
> > Olga Kornievskaia (10):
> >   NFSD CB_OFFLOAD xdr
> >   NFSD OFFLOAD_STATUS xdr
> >   NFSD OFFLOAD_CANCEL xdr
> >   NFSD xdr callback stateid in async COPY reply
> >   NFSD first draft of async copy
> >   NFSD handle OFFLOAD_CANCEL op
> >   NFSD stop queued async copies on client shutdown
> >   NFSD create new stateid for async copy
> >   NFSD define EBADF in nfserrno
> >   NFSD support OFFLOAD_STATUS
> > 
> >  fs/nfsd/Kconfig        |  10 +
> >  fs/nfsd/netns.h        |   8 +
> >  fs/nfsd/nfs4callback.c |  95 +++++++
> >  fs/nfsd/nfs4proc.c     | 704 ++++++++++++++++++++++++++++++++++++++++++++++---
> >  fs/nfsd/nfs4state.c    | 142 +++++++++-
> >  fs/nfsd/nfs4xdr.c      | 266 ++++++++++++++++++-
> >  fs/nfsd/nfsctl.c       |   2 +
> >  fs/nfsd/nfsd.h         |   2 +
> >  fs/nfsd/nfsproc.c      |   1 +
> >  fs/nfsd/state.h        |  32 ++-
> >  fs/nfsd/xdr4.h         |  53 +++-
> >  fs/nfsd/xdr4cb.h       |  10 +
> >  include/linux/nfs4.h   |   1 +
> >  13 files changed, 1273 insertions(+), 53 deletions(-)
> > 
> > -- 
> > 1.8.3.1
> >