Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Nagendra Tomar <Nagendra.Tomar@microsoft.com>
To:     "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
CC:     Trond Myklebust <trond.myklebust@hammerspace.com>,
        "anna.schumaker@netapp.com" <anna.schumaker@netapp.com>
Subject: [PATCH 0/5] nfs: Add mount option for forcing RPC requests for one
 file over one connection
Thread-Topic: [PATCH 0/5] nfs: Add mount option for forcing RPC requests for
 one file over one connection
Thread-Index: Adcfp3hcmhRvn3IHQv25MQlIHJPMUA==
Date:   Tue, 23 Mar 2021 05:46:13 +0000
Message-ID: <SG2P153MB0361E2D69F8AD8AFBF9E7CD89E649@SG2P153MB0361.APCP153.PROD.OUTLOOK.COM>
Accept-Language: en-IN, en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 225f3c89-7e3e-4eea-3878-08d8edbef853
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Mar 2021 05:46:13.6238
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ytqpXSBdG7JrZOC0vJoDmEUhdW3tktNHWkOA/xwBV58EyrA7XeeOeg3AVN82IRUFMxpL76du8UdCSvsiMOthRg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2P15301MB0064
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org

From: Nagendra S Tomar <natomar@microsoft.com>

If a clustered NFS server is behind an L4 loadbalancer the default
nconnect roundrobin policy may cause RPC requests to a file to be
sent to different cluster nodes. This is because the source port
would be different for all the nconnect connections.
While this should functionally work (since the cluster will usually
have a consistent view irrespective of which node is serving the
request), it may not be desirable from performance pov. As an
example we have an NFSv3 frontend to our Object store, where every
NFSv3 file is an object. Now if writes to the same file are sent
roundrobin to different cluster nodes, the writes become very
inefficient due to the consistency requirement for object update
being done from different nodes.
Similarly each node may maintain some kind of cache to serve the file
data/metadata requests faster and even in that case it helps to have
a xprt affinity for a file/dir.
In general we have seen such scheme to scale very well.

This patch introduces a new rpc_xprt_iter_ops for using an additional
u32 (filehandle hash) to affine RPCs to the same file to one xprt.
It adds a new mount option "ncpolicy=3Droundrobin|hash" which can be
used to select the nconnect multipath policy for a given mount and
pass the selected policy to the RPC client.
It adds a new rpc_procinfo member p_fhhash, which can be supplied
by the specific RPC programs to return a u32 hash of the file/dir the
RPC is targetting, and lastly it provides p_fhhash implementation
for various NFS v3/v4/v41/v42 RPCs to generate the hash correctly.

Thoughts?

Thanks,
Tomar

Nagendra S Tomar (5):
  SUNRPC: Add a new multipath xprt policy for xprt selection based
    on target filehandle hash
  SUNRPC/NFSv3/NFSv4: Introduce "enum ncpolicy" to represent the nconnect
    policy and pass it down from mount option to rpc layer
  SUNRPC/NFSv4: Rename RPC_TASK_NO_ROUND_ROBIN -> RPC_TASK_USE_MAIN_XPRT
  NFSv3: Add hash computation methods for NFSv3 RPCs
  NFSv4: Add hash computation methods for NFSv4/NFSv42 RPCs

 fs/nfs/client.c                      |   3 +
 fs/nfs/fs_context.c                  |  26 ++
 fs/nfs/internal.h                    |   2 +
 fs/nfs/nfs3client.c                  |   4 +-
 fs/nfs/nfs3xdr.c                     | 154 +++++++++++
 fs/nfs/nfs42xdr.c                    | 112 ++++++++
 fs/nfs/nfs4client.c                  |  14 +-
 fs/nfs/nfs4proc.c                    |  18 +-
 fs/nfs/nfs4xdr.c                     | 516 ++++++++++++++++++++++++++++++-=
----
 fs/nfs/super.c                       |   7 +-
 include/linux/nfs_fs_sb.h            |   1 +
 include/linux/sunrpc/clnt.h          |  15 +
 include/linux/sunrpc/sched.h         |   2 +-
 include/linux/sunrpc/xprtmultipath.h |   9 +-
 include/trace/events/sunrpc.h        |   4 +-
 net/sunrpc/clnt.c                    |  38 ++-
 net/sunrpc/xprtmultipath.c           |  91 +++++-
 17 files changed, 913 insertions(+), 103 deletions(-)