Received-SPF: pass (google.com: domain of linux-kernel+bounces-133556-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <2ebbe58df9575136583f65cd19d133bdb61d5c20.camel@redhat.com>
Subject: Re: [PATCH v2] nfsd: hold a lighter-weight client reference over
 CB_RECALL_ANY
From: vbenes@redhat.com
To: Chuck Lever <chuck.lever@oracle.com>, Jeff Layton <jlayton@kernel.org>
Cc: Neil Brown <neilb@suse.de>, Olga Kornievskaia <kolga@netapp.com>, Dai
 Ngo <Dai.Ngo@oracle.com>, Tom Talpey <tom@talpey.com>,
 linux-nfs@vger.kernel.org,  linux-kernel@vger.kernel.org
Date: Fri, 05 Apr 2024 22:11:45 +0200
In-Reply-To: <ZhA93BQSxkJqmqaw@tissot.1015granger.net>
References: <20240405-rhel-31513-v2-1-b0f6c10be929@kernel.org>
	 <ZhA93BQSxkJqmqaw@tissot.1015granger.net>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) 
Precedence: bulk
MIME-Version: 1.0

On Fri, 2024-04-05 at 14:07 -0400, Chuck Lever wrote:
> On Fri, Apr 05, 2024 at 01:56:18PM -0400, Jeff Layton wrote:
> > Currently the CB_RECALL_ANY job takes a cl_rpc_users reference to
> > the
> > client. While a callback job is technically an RPC that counter is
> > really more for client-driven RPCs, and this has the effect of
> > preventing the client from being unhashed until the callback
> > completes.
> >=20
> > If nfsd decides to send a CB_RECALL_ANY just as the client reboots,
> > we
> > can end up in a situation where the callback can't complete on the
> > (now
> > dead) callback channel, but the new client can't connect because
> > the old
> > client can't be unhashed. This usually manifests as a NFS4ERR_DELAY
> > return on the CREATE_SESSION operation.
> >=20
> > The job is only holding a reference to the client so it can clear a
> > flag
> > in the after the RPC completes. Fix this by having CB_RECALL_ANY
> > instead
> > hold a reference to the cl_nfsdfs.cl_ref. Typically we only take
> > that
> > sort of reference when dealing with the nfsdfs info files, but it
> > should
> > work appropriately here to ensure that the nfs4_client doesn't
> > disappear.
> >=20
> > Fixes: 44df6f439a17 ("NFSD: add delegation reaper to react to low
> > memory condition")
> > Reported-by: Vladimir Benes <vbenes@redhat.com>
> > Signed-off-by: Jeff Layton <jlayton@kernel.org>
>=20
> Applied to nfsd-fixes while waiting for review and testing. Thanks!
>=20
>=20
> > ---
> > Changes in v2:
> > - Clean up the changelog
> > - Add Fixes: tag
> > - Use kref_get instead of kref_get_unless_zero
> > ---
> > =C2=A0fs/nfsd/nfs4state.c | 7 ++-----
> > =C2=A01 file changed, 2 insertions(+), 5 deletions(-)
> >=20
> > diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> > index 5fcd93f7cb8c..3cef81e196c6 100644
> > --- a/fs/nfsd/nfs4state.c
> > +++ b/fs/nfsd/nfs4state.c
> > @@ -3042,12 +3042,9 @@ static void
> > =C2=A0nfsd4_cb_recall_any_release(struct nfsd4_callback *cb)
> > =C2=A0{
> > =C2=A0	struct nfs4_client *clp =3D cb->cb_clp;
> > -	struct nfsd_net *nn =3D net_generic(clp->net, nfsd_net_id);
> > =C2=A0
> > -	spin_lock(&nn->client_lock);
> > =C2=A0	clear_bit(NFSD4_CLIENT_CB_RECALL_ANY, &clp->cl_flags);
> > -	put_client_renew_locked(clp);
> > -	spin_unlock(&nn->client_lock);
> > +	drop_client(clp);
> > =C2=A0}
> > =C2=A0
> > =C2=A0static int
> > @@ -6616,7 +6613,7 @@ deleg_reaper(struct nfsd_net *nn)
> > =C2=A0		list_add(&clp->cl_ra_cblist, &cblist);
> > =C2=A0
> > =C2=A0		/* release in nfsd4_cb_recall_any_release */
> > -		atomic_inc(&clp->cl_rpc_users);
> > +		kref_get(&clp->cl_nfsdfs.cl_ref);
> > =C2=A0		set_bit(NFSD4_CLIENT_CB_RECALL_ANY, &clp-
> > >cl_flags);
> > =C2=A0		clp->cl_ra_time =3D ktime_get_boottime_seconds();
> > =C2=A0	}
> >=20
> > ---
> > base-commit: 05258a0a69b3c5d2c003f818702c0a52b6fea861
> > change-id: 20240405-rhel-31513-028ab6f14252
> >=20
> > Best regards,
> > --=20
> > Jeff Layton <jlayton@kernel.org>
> >=20
> >=20
>=20
Hi,=20
I've just finished the testing of the new patch on the same HW
configuration and the dracut test suite is stable again.

Thank you for your patches!
Vladimir Benes

Tested-by: Vladimir Benes <vbenes@redhat.com>