Received-SPF: pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain;
        charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: [PATCH 7/7] NFS: Add a mount option for READ_PLUS
From:   Chuck Lever <chuck.lever@oracle.com>
In-Reply-To: <20200110223455.528471-8-Anna.Schumaker@Netapp.com>
Date:   Fri, 10 Jan 2020 17:41:24 -0500
Cc:     Trond.Myklebust@hammerspace.com,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        Anna Schumaker <Anna.Schumaker@Netapp.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <41391995-F61E-4CA4-80C5-542A8B2A3947@oracle.com>
References: <20200110223455.528471-1-Anna.Schumaker@Netapp.com>
 <20200110223455.528471-8-Anna.Schumaker@Netapp.com>
To:     Anna Schumaker <schumaker.anna@gmail.com>
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk

Hi Anna-

> On Jan 10, 2020, at 5:34 PM, schumaker.anna@gmail.com wrote:
>=20
> From: Anna Schumaker <Anna.Schumaker@Netapp.com>
>=20
> There are some workloads where READ_PLUS might end up hurting
> performance,

Can you say more about this? Have you seen workloads that are
hurt by READ_PLUS? Nothing jumps out at me from the tables in
the cover letter.


> so let's be nice to users and provide a way to disable this
> operation similar to how READDIR_PLUS can be disabled.

Does it make sense to hold off on a mount option until there
is evidence that there is no other way to work around such a
performance regression?

- Attempt to address the regression directly
- Improve the heuristics about when READ_PLUS is used
- Document that dropping back to vers=3D4.1 will disable it

Any experience with NFS/RDMA?


> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
> ---
> fs/nfs/fs_context.c       | 14 ++++++++++++++
> fs/nfs/nfs4client.c       |  3 +++
> include/linux/nfs_fs_sb.h |  1 +
> 3 files changed, 18 insertions(+)
>=20
> diff --git a/fs/nfs/fs_context.c b/fs/nfs/fs_context.c
> index 0247dcb7b316..82ba07c7c1ce 100644
> --- a/fs/nfs/fs_context.c
> +++ b/fs/nfs/fs_context.c
> @@ -64,6 +64,7 @@ enum nfs_param {
> 	Opt_proto,
> 	Opt_rdirplus,
> 	Opt_rdma,
> +	Opt_readplus,
> 	Opt_resvport,
> 	Opt_retrans,
> 	Opt_retry,
> @@ -120,6 +121,7 @@ static const struct fs_parameter_spec =
nfs_param_specs[] =3D {
> 	fsparam_string("proto",		Opt_proto),
> 	fsparam_flag_no("rdirplus",	Opt_rdirplus),
> 	fsparam_flag  ("rdma",		Opt_rdma),
> +	fsparam_flag_no("readplus",	Opt_readplus),
> 	fsparam_flag_no("resvport",	Opt_resvport),
> 	fsparam_u32   ("retrans",	Opt_retrans),
> 	fsparam_string("retry",		Opt_retry),
> @@ -555,6 +557,12 @@ static int nfs_fs_context_parse_param(struct =
fs_context *fc,
> 		else
> 			ctx->options |=3D NFS_OPTION_MIGRATION;
> 		break;
> +	case Opt_readplus:
> +		if (result.negated)
> +			ctx->options |=3D NFS_OPTION_NO_READ_PLUS;
> +		else
> +			ctx->options &=3D ~NFS_OPTION_NO_READ_PLUS;
> +		break;
>=20
> 		/*
> 		 * options that take numeric values
> @@ -1176,6 +1184,10 @@ static int nfs_fs_context_validate(struct =
fs_context *fc)
> 	    (ctx->version !=3D 4 || ctx->minorversion !=3D 0))
> 		goto out_migration_misuse;
>=20
> +	if (ctx->options & NFS_OPTION_NO_READ_PLUS &&
> +	    (ctx->version !=3D 4 || ctx->minorversion < 2))
> +		goto out_noreadplus_misuse;
> +
> 	/* Verify that any proto=3D/mountproto=3D options match the =
address
> 	 * families in the addr=3D/mountaddr=3D options.
> 	 */
> @@ -1254,6 +1266,8 @@ static int nfs_fs_context_validate(struct =
fs_context *fc)
> 			  ctx->version, ctx->minorversion);
> out_migration_misuse:
> 	return nfs_invalf(fc, "NFS: 'Migration' not supported for this =
NFS version");
> +out_noreadplus_misuse:
> +	return nfs_invalf(fc, "NFS: 'noreadplus' not supported for this =
NFS version\n");
> out_version_unavailable:
> 	nfs_errorf(fc, "NFS: Version unavailable");
> 	return ret;
> diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
> index 0cd767e5c977..868dc3c36ba1 100644
> --- a/fs/nfs/nfs4client.c
> +++ b/fs/nfs/nfs4client.c
> @@ -1016,6 +1016,9 @@ static int nfs4_server_common_setup(struct =
nfs_server *server,
> 	server->caps |=3D server->nfs_client->cl_mvops->init_caps;
> 	if (server->flags & NFS_MOUNT_NORDIRPLUS)
> 			server->caps &=3D ~NFS_CAP_READDIRPLUS;
> +	if (server->options & NFS_OPTION_NO_READ_PLUS)
> +		server->caps &=3D ~NFS_CAP_READ_PLUS;
> +
> 	/*
> 	 * Don't use NFS uid/gid mapping if we're using AUTH_SYS or =
lower
> 	 * authentication.
> diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
> index 11248c5a7b24..360e70c7bbb6 100644
> --- a/include/linux/nfs_fs_sb.h
> +++ b/include/linux/nfs_fs_sb.h
> @@ -172,6 +172,7 @@ struct nfs_server {
> 	unsigned int		clone_blksize;	/* granularity of a =
CLONE operation */
> #define NFS_OPTION_FSCACHE	0x00000001	/* - local caching =
enabled */
> #define NFS_OPTION_MIGRATION	0x00000002	/* - NFSv4 migration =
enabled */
> +#define NFS_OPTION_NO_READ_PLUS	0x00000004	/* - NFSv4.2 =
READ_PLUS enabled */
>=20
> 	struct nfs_fsid		fsid;
> 	__u64			maxfilesize;	/* maximum file size */
> --=20
> 2.24.1
>=20

--
Chuck Lever