Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Content-Type: text/plain;
        charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\))
Subject: Re: [PATCH 00/20] NFSD support for multiple RPC/RDMA chunks
From:   Chuck Lever <chuck.lever@oracle.com>
In-Reply-To: <20201027172545.GB1644@fieldses.org>
Date:   Tue, 27 Oct 2020 13:29:59 -0400
Cc:     Leon Romanovsky <leon@kernel.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        linux-rdma@vger.kernel.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <D00EBDE2-D748-4977-9E2D-186BFAD56ED5@oracle.com>
References: <160373843299.1886.12604782813896379719.stgit@klimt.1015granger.net>
 <20201027060823.GF4821@unreal>
 <DAC657D8-D254-452C-9B21-3053F70C8C73@oracle.com>
 <20201027172545.GB1644@fieldses.org>
To:     Bruce Fields <bfields@fieldses.org>
Precedence: bulk


> On Oct 27, 2020, at 1:25 PM, bfields@fieldses.org wrote:
>=20
> On Tue, Oct 27, 2020 at 09:24:54AM -0400, Chuck Lever wrote:
>> Hi Leon-
>>=20
>>> On Oct 27, 2020, at 2:08 AM, Leon Romanovsky <leon@kernel.org> =
wrote:
>>>=20
>>> On Mon, Oct 26, 2020 at 02:53:53PM -0400, Chuck Lever wrote:
>>>> This series implements support for multiple RPC/RDMA chunks per RPC
>>>> transaction. This is one of the few remaining generalities that the
>>>> Linux NFS/RDMA server implementation lacks.
>>>>=20
>>>> There is currently one known NFS/RDMA client implementation that =
can
>>>> send multiple chunks per RPC, and that is Solaris. Multiple chunks
>>>> are rare enough that the Linux NFS/RDMA implementation has been
>>>> successful without this support for many years.
>>>=20
>>> So why do we need it? Solaris is dead, and like you wrote Linux =
systems
>>> work without this feature just fine, what are the benefits? Who will =
use it?
>>=20
>> The Linux NFS implementation is living. We can add the ability
>> to provision multiple chunks per RPC to the Linux NFS client at
>> any time.
>>=20
>> Likewise any actively developed NFS/RDMA implementation can add
>> this feature. The RPC/RDMA version 1 protocol does not have the
>> ability to communicate the maximum number of chunks the server
>> will accept per RPC.
>>=20
>> Other server implementations do support multiple chunks per RPC.
>> The Linux NFS/RDMA server implementation has always been incomplete
>> in this regard.
>=20
> Can the client can detect the server's lack of support and fall back, =
or
> does the server's incompleteness violate the RFC in some way that can
> actually cause a failure to interoperate?

The latter. Currently the client has no way to detect that our server
does not comply with RFC 8166, which places no arbitrary limits on
the number of chunks per RPC.

If a client attempts to send more than one chunk the RPC fails (or
worse). RPC/RDMA version 1 does not provide a way to indicate that
the failure was because the client sent too many chunks, so the
client has to terminate the RPC transaction with an error.


>> And the Linux NFS server implementation (the non-transport specific
>> part) already supports multiple data payloads per NFSv4 COMPOUND.
>>=20
>>=20
>> Restoring a little more of the cover letter:
>>=20
>>>> Along with multiple chunk support, this series adds the following
>>>> benefits:
>>>>=20
>>>> - More robust input sanitization of RPC/RDMA headers
>>>> - An internal representation of chunks that is agnostic to their
>>>> wire format
>>=20
>> The Linux NFS/RDMA server implementation does need to have better
>> input sanitization.
>>=20
>> And there is a version 2 of RPC/RDMA under active development:
>>=20
>> =
https://datatracker.ietf.org/doc/draft-ietf-nfsv4-rpcrdma-version-two/
>>=20
>> Having some protocol version agnosticism in our transport might
>> be necessary eventually.
>>=20
>> --
>> Chuck Lever

--
Chuck Lever