Received-SPF: pass (google.com: domain of linux-nfs+bounces-3185-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Precedence: bulk
MIME-Version: 1.0
References: <20240506210408.4760-1-cel@kernel.org>
In-Reply-To: <20240506210408.4760-1-cel@kernel.org>
From: Rick Macklem <rick.macklem@gmail.com>
Date: Mon, 6 May 2024 15:30:18 -0700
Message-ID: <CAM5tNy4nceCa7k+E1sKEAi5GYJvMwYDuApspFJvSdVdXBuHjmw@mail.gmail.com>
Subject: Re: [RFC PATCH] NFSD: Force all NFSv4.2 COPY requests to be synchronous
To: cel@kernel.org
Cc: linux-nfs@vger.kernel.org, Chuck Lever <chuck.lever@oracle.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, May 6, 2024 at 2:04=E2=80=AFPM <cel@kernel.org> wrote:
>
> From: Chuck Lever <chuck.lever@oracle.com>
>
> We've discovered that delivering a CB_OFFLOAD operation can be
> unreliable in some pretty unremarkable situations, and the Linux
> NFS client does not yet support sending an OFFLOAD_STATUS
> operation to probe whether an asynchronous COPY operation has
> finished. On Linux NFS clients, COPY can hang until manually
> interrupted.
>
> I've tried a couple of remedies, but so far the side-effects are
> worse than the disease. For now, force COPY operations to be
> synchronous so that the use of CB_OFFLOAD is avoided entirely.
Just a yellow warning light from my experience with the FreeBSD server
(which always does synchronous Copy's).
A large synchronous Copy can take a long time, resulting in a slow RPC
response time. A user reported 25sec.
The 25sec case turned out to be a ZFS specific issue that could be avoided
via a ZFS tunable.

I do not have a good generic solution, but I do have a tunable that can
be used to clip the Copy len, which is a rather blunt and ugly workaround.
(The generic copy routine internal to FreeBSD can accept a flag that indica=
tes
"return after 1sec with a partial copy done", but that is not implemented f=
or
file systems like ZFS.)

Although there is no hard limit in the RFCs for RPC response times,
I've typically
assumed a max of 1-2sec is desirable.

rick

>
> I have some patches that add an OFFLOAD_STATUS implementation to the
> Linux NFS client, but that is not likely to fix older clients.
>
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  fs/nfsd/nfs4proc.c | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> index ea3cc3e870a7..12722c709cc6 100644
> --- a/fs/nfsd/nfs4proc.c
> +++ b/fs/nfsd/nfs4proc.c
> @@ -1807,6 +1807,13 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_co=
mpound_state *cstate,
>         __be32 status;
>         struct nfsd4_copy *async_copy =3D NULL;
>
> +       /*
> +        * Currently, async COPY is not reliable. Force all COPY
> +        * requests to be synchronous to avoid client application
> +        * hangs waiting for completion.
> +        */
> +       nfsd4_copy_set_sync(copy, true);
> +
>         copy->cp_clp =3D cstate->clp;
>         if (nfsd4_ssc_is_inter(copy)) {
>                 trace_nfsd_copy_inter(copy);
>
> base-commit: 939cb14d51a150e3c12ef7a8ce0ba04ce6131bd2
> --
> 2.44.0
>
>