Received-SPF: softfail (google.com: domain of transitioning linux-nfs-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Neil Brown <neilb@suse.de>
CC:     Bruce Fields <bfields@fieldses.org>,
        Steve Dickson <SteveD@RedHat.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v2] nfsd: report client confirmation status in "info" file
Thread-Topic: [PATCH v2] nfsd: report client confirmation status in "info"
 file
Thread-Index: AQHXHRCg4I/NVojQCUuAkyEihGCrLa0nUdcA
Date:   Wed, 18 May 2022 14:45:59 +0000
Message-ID: <41FCC628-9C02-4A8D-9E82-292AAF489E9B@oracle.com>
References: <161456493684.22801.323431390819102360.stgit@noble>
 <20210301185037.GB14881@fieldses.org>
 <874khui7hr.fsf@notabene.neil.brown.name>
 <20210302032733.GC16303@fieldses.org>
 <87y2ejerwn.fsf@notabene.neil.brown.name>
 <87v99neruh.fsf@notabene.neil.brown.name>
 <87h7l6epmb.fsf@notabene.neil.brown.name>
In-Reply-To: <87h7l6epmb.fsf@notabene.neil.brown.name>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?/ZkWkS21LCZAflEWovav2qx5nsXbwQU5Tlu/IG/zSC7HP4kAm6gW3p1Do7Xt?=
 =?us-ascii?Q?FjAImidL1Ripxxbx3u3OcToZNv+Q3lAj5eQ5xLSs7hLjqu1oHv8OurOYlZg9?=
 =?us-ascii?Q?Jr9as2wGAsg6nZ2g2zFfIa3NWX0pVXfPTqhYZ/qhWx5l3oF3/fi+M43RK9jB?=
 =?us-ascii?Q?dzK2vfn9WP+kiWBh9sr9Uc+iyaiFyjKysQWp9yQJbJHeOKqrzvcg+llWLAZj?=
 =?us-ascii?Q?C0/hbpGO4EDwsyupMVb3Bbs2wQ9YBlin7aoXSbGJy9nL6K1Gnw2RixsklIIl?=
 =?us-ascii?Q?sx0S/aoe+hGdsxWKZMir4UCV3vfp4ur2eObZNyJPP8ivg1cN3KVd2jnPicwp?=
 =?us-ascii?Q?Jw1psAU4yVd7e/B4b7QatQCq5Q8lWJe01uNgt9UJZFVA+3D58NDOFU2pAY/5?=
 =?us-ascii?Q?wMsElKiSx+CyTsl2pi3oxfwi+AiMFaPFDF3twy6t67Q1TXKMi+aIRPmFc8Bp?=
 =?us-ascii?Q?3HXOUCoeE7Ga5kMF3vCpj8K1Rru75TbpBF3Z+CDeNxoMre1ffW1LKCThGVr8?=
 =?us-ascii?Q?HNuTF85kpBR/71i1Fyuf5PjEH0uFdgufX4yHIzM5xRABQAq9K3WeaJW/nt4u?=
 =?us-ascii?Q?ByznYXRRpQ7/nJbiXf1qNlk9vcsRNOCUzUrP9YSChTffzUSg6x83IqiBVF9U?=
 =?us-ascii?Q?nH32ccu72PvhtlHp0DP7IfTE7E5zuFZwu7oBFugyzgrXxFwFi77VkJRZb2pk?=
 =?us-ascii?Q?Av/XgBaRl2f0JdZorhBngr4dRSUWq6cDcNraddZ1KhhC2I5v843c+JfGLDAv?=
 =?us-ascii?Q?95aDj1cQB8IYmqstmk6g9NMZ8XDvdsqpHVD7+nEvREmsyOwcsTg6DgOlDZko?=
 =?us-ascii?Q?hDuuQIEw6kevruSQm2vz7khQ7n/P0rCfF3sS0cew+lU+6/8hGc1Dcs2F1F2G?=
 =?us-ascii?Q?SqDWFvKhiSkrSgHbDKBI3aqj2KlOtBHA0SuXzNquYN+7IyWuwQRLmasEfgVe?=
 =?us-ascii?Q?zMVr6LBia1uoWyBiydFrvBsu/ujTmRBBGN74WWv1CPHAP3UMv2YKKNJ9FFFq?=
 =?us-ascii?Q?bJ3kM0wlhA03okqWY7op4s/3caRclxrx4RlAICwI5aMw6vHpxet+fQuenlk2?=
 =?us-ascii?Q?qGY5XrbeSDM9aarJXmBcb3fLXMT7LRcRJLcqN1l6XrQzS8LlQLTXNKtoBl5p?=
 =?us-ascii?Q?9a8J6MEbGijw3p2ek2IKvKxxgDxq3QNTY0Jb+buUUs+GsG/li3qZ/gmgT4FT?=
 =?us-ascii?Q?jN7wS6Q+RmxNiJwOeVTr6R9RybF3aQub65HE52gd6BhlPVPvPLSecbqaF1ta?=
 =?us-ascii?Q?7wzTe2yBT5QVPg3PFA+/E86DjPLM5gV0hYRbGAjU5WC42VRv/Fy7TKZHjOj0?=
 =?us-ascii?Q?MCcKETGRNq+gqtGhPKpDtlz8Zrnwa3921mdvuBAU1ZCrtUAJGEWm3a3+MWZp?=
 =?us-ascii?Q?gLO8Ty2Zd+rE5TvmE+xUvp+1MxgFhhnxCFdLFADMcNMCnQtbRSoY+2DYqMwP?=
 =?us-ascii?Q?QKQD3s3xJixryYWOlJwVfRey24/P04ojrXkO13Zzrc1D7R25VHlj3g0D1CS5?=
 =?us-ascii?Q?LbG641w5wXIWiSiQiNayN/OflUehiRCPw/QWmf/BNgYoMWEIBRseBhZ0HK5A?=
 =?us-ascii?Q?QciQ/Ra6+Bc5Y4svn6vL3QokineQxj7ZAOg5R7pcR6KlQwoogoXWfsruVTnD?=
 =?us-ascii?Q?XnRiRWRWbxrFdOaVJafYPfBYbi7+qL6fEMa2jPMDwISTiUAbb339bCf760uY?=
 =?us-ascii?Q?G+hvK0bkeO2I7kRRuRUSZVIvUqo+ALzJ1DsuVAvLM+Qf3tWl3HOzfIC0VOkB?=
 =?us-ascii?Q?gfM3+3nBL6i8D0yuOE25NDgYng1mPdQ=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <F6E2656F09E12E4CB8ED526D1EE8BA21@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 931c4e5a-03f8-4e8e-ef26-08da38dd1fc2
X-MS-Exchange-CrossTenant-originalarrivaltime: 18 May 2022 14:45:59.7987
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ws4w2GmRnA7X2HsrBoajZ1nwknAeIEdQMroKDYFQTelMAyVplU1ZAKeiG+CfZOW+q0rJMexXOo2v0/xIWHLlQw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR10MB4362
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.874
 definitions=2022-05-18_05:2022-05-17,2022-05-18 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxscore=0
 phishscore=0 suspectscore=0 mlxlogscore=999 adultscore=0 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000
 definitions=main-2205180088
X-Proofpoint-ORIG-GUID: LF3-UMLOyvOVzxc9rK9EAFQAalXyyerU
X-Proofpoint-GUID: LF3-UMLOyvOVzxc9rK9EAFQAalXyyerU
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org

Hi Neil-


> On Mar 19, 2021, at 6:38 PM, NeilBrown <neilb@suse.de> wrote:
>=20
>=20
> mountd can now monitor clients appearing and disappearing in
> /proc/fs/nfsd/clients, and will log these events, in liu of the logging
> of mount/unmount events for NFSv3.
>=20
> Currently it cannot distinguish between unconfirmed clients (which might
> be transient and totally uninteresting) and confirmed clients.
>=20
> So add a "status: " line which reports either "confirmed" or
> "unconfirmed", and use fsnotify to report that the info file
> has been modified.
>=20
> This requires a bit of infrastructure to keep the dentry for the "info"
> file.  There is no need to take a counted reference as the dentry must
> remain around until the client is removed.
>=20
> Signed-off-by: NeilBrown <neilb@suse.de>
> ---
> fs/nfsd/nfs4state.c | 19 +++++++++++++++----
> fs/nfsd/nfsctl.c    | 14 ++++++++------
> fs/nfsd/nfsd.h      |  4 +++-
> fs/nfsd/state.h     |  4 ++++
> 4 files changed, 30 insertions(+), 11 deletions(-)
>=20
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index 97447a64bad0..ec1b2dd8fd45 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -43,6 +43,7 @@
> #include <linux/sunrpc/addr.h>
> #include <linux/jhash.h>
> #include <linux/string_helpers.h>
> +#include <linux/fsnotify.h>
> #include "xdr4.h"
> #include "xdr4cb.h"
> #include "vfs.h"
> @@ -2352,6 +2353,10 @@ static int client_info_show(struct seq_file *m, vo=
id *v)
> 	memcpy(&clid, &clp->cl_clientid, sizeof(clid));
> 	seq_printf(m, "clientid: 0x%llx\n", clid);
> 	seq_printf(m, "address: \"%pISpc\"\n", (struct sockaddr *)&clp->cl_addr)=
;
> +	if (test_bit(NFSD4_CLIENT_CONFIRMED, &clp->cl_flags))
> +		seq_puts(m, "status: confirmed\n");
> +	else
> +		seq_puts(m, "status: unconfirmed\n");
> 	seq_printf(m, "name: ");
> 	seq_quote_mem(m, clp->cl_name.data, clp->cl_name.len);
> 	seq_printf(m, "\nminor version: %d\n", clp->cl_minorversion);
> @@ -2702,6 +2707,7 @@ static struct nfs4_client *create_client(struct xdr=
_netobj name,
> 	int ret;
> 	struct net *net =3D SVC_NET(rqstp);
> 	struct nfsd_net *nn =3D net_generic(net, nfsd_net_id);
> +	struct dentry *dentries[ARRAY_SIZE(client_files)];
>=20
> 	clp =3D alloc_client(name);
> 	if (clp =3D=3D NULL)
> @@ -2721,9 +2727,11 @@ static struct nfs4_client *create_client(struct xd=
r_netobj name,
> 	memcpy(&clp->cl_addr, sa, sizeof(struct sockaddr_storage));
> 	clp->cl_cb_session =3D NULL;
> 	clp->net =3D net;
> -	clp->cl_nfsd_dentry =3D nfsd_client_mkdir(nn, &clp->cl_nfsdfs,
> -			clp->cl_clientid.cl_id - nn->clientid_base,
> -			client_files);
> +	clp->cl_nfsd_dentry =3D nfsd_client_mkdir(
> +		nn, &clp->cl_nfsdfs,
> +		clp->cl_clientid.cl_id - nn->clientid_base,
> +		client_files, dentries);
> +	clp->cl_nfsd_info_dentry =3D dentries[0];
> 	if (!clp->cl_nfsd_dentry) {
> 		free_client(clp);
> 		return NULL;
> @@ -2798,7 +2806,10 @@ move_to_confirmed(struct nfs4_client *clp)
> 	list_move(&clp->cl_idhash, &nn->conf_id_hashtbl[idhashval]);
> 	rb_erase(&clp->cl_namenode, &nn->unconf_name_tree);
> 	add_clp_to_name_tree(clp, &nn->conf_name_tree);
> -	set_bit(NFSD4_CLIENT_CONFIRMED, &clp->cl_flags);
> +	if (!test_and_set_bit(NFSD4_CLIENT_CONFIRMED, &clp->cl_flags) &&
> +	    clp->cl_nfsd_dentry &&
> +	    clp->cl_nfsd_info_dentry)
> +		fsnotify_dentry(clp->cl_nfsd_info_dentry, FS_MODIFY);

I hit a "sleep while spin-locked" splat and bisected it to this
commit. fsnotify() can allocate memory with GFP_KERNEL, so it
can't be called while &nn->client_lock is held, unfortunately.


> 	renew_client_locked(clp);
> }
>=20
> diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
> index ef86ed23af82..94ce1eabd9d1 100644
> --- a/fs/nfsd/nfsctl.c
> +++ b/fs/nfsd/nfsctl.c
> @@ -1266,7 +1266,8 @@ static void nfsdfs_remove_files(struct dentry *root=
)
> /* XXX: cut'n'paste from simple_fill_super; figure out if we could share
>  * code instead. */
> static  int nfsdfs_create_files(struct dentry *root,
> -					const struct tree_descr *files)
> +				const struct tree_descr *files,
> +				struct dentry **fdentries)
> {
> 	struct inode *dir =3D d_inode(root);
> 	struct inode *inode;
> @@ -1275,8 +1276,6 @@ static  int nfsdfs_create_files(struct dentry *root=
,
>=20
> 	inode_lock(dir);
> 	for (i =3D 0; files->name && files->name[0]; i++, files++) {
> -		if (!files->name)
> -			continue;
> 		dentry =3D d_alloc_name(root, files->name);
> 		if (!dentry)
> 			goto out;
> @@ -1290,6 +1289,8 @@ static  int nfsdfs_create_files(struct dentry *root=
,
> 		inode->i_private =3D __get_nfsdfs_client(dir);
> 		d_add(dentry, inode);
> 		fsnotify_create(dir, dentry);
> +		if (fdentries)
> +			fdentries[i] =3D dentry;
> 	}
> 	inode_unlock(dir);
> 	return 0;
> @@ -1301,8 +1302,9 @@ static  int nfsdfs_create_files(struct dentry *root=
,
>=20
> /* on success, returns positive number unique to that client. */
> struct dentry *nfsd_client_mkdir(struct nfsd_net *nn,
> -		struct nfsdfs_client *ncl, u32 id,
> -		const struct tree_descr *files)
> +				 struct nfsdfs_client *ncl, u32 id,
> +				 const struct tree_descr *files,
> +				 struct dentry **fdentries)
> {
> 	struct dentry *dentry;
> 	char name[11];
> @@ -1313,7 +1315,7 @@ struct dentry *nfsd_client_mkdir(struct nfsd_net *n=
n,
> 	dentry =3D nfsd_mkdir(nn->nfsd_client_dir, ncl, name);
> 	if (IS_ERR(dentry)) /* XXX: tossing errors? */
> 		return NULL;
> -	ret =3D nfsdfs_create_files(dentry, files);
> +	ret =3D nfsdfs_create_files(dentry, files, fdentries);
> 	if (ret) {
> 		nfsd_client_rmdir(dentry);
> 		return NULL;
> diff --git a/fs/nfsd/nfsd.h b/fs/nfsd/nfsd.h
> index 8bdc37aa2c2e..9aee72f65330 100644
> --- a/fs/nfsd/nfsd.h
> +++ b/fs/nfsd/nfsd.h
> @@ -107,7 +107,9 @@ struct nfsdfs_client {
>=20
> struct nfsdfs_client *get_nfsdfs_client(struct inode *);
> struct dentry *nfsd_client_mkdir(struct nfsd_net *nn,
> -		struct nfsdfs_client *ncl, u32 id, const struct tree_descr *);
> +				 struct nfsdfs_client *ncl, u32 id,
> +				 const struct tree_descr *,
> +				 struct dentry **fdentries);
> void nfsd_client_rmdir(struct dentry *dentry);
>=20
>=20
> diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
> index 73deea353169..54cab651ac1d 100644
> --- a/fs/nfsd/state.h
> +++ b/fs/nfsd/state.h
> @@ -371,6 +371,10 @@ struct nfs4_client {
>=20
> 	/* debugging info directory under nfsd/clients/ : */
> 	struct dentry		*cl_nfsd_dentry;
> +	/* 'info' file within that directory. Ref is not counted,
> +	 * but will remain valid iff cl_nfsd_dentry !=3D NULL
> +	 */
> +	struct dentry		*cl_nfsd_info_dentry;
>=20
> 	/* for nfs41 callbacks */
> 	/* We currently support a single back channel with a single slot */
> --=20
> 2.30.1
>=20

--
Chuck Lever