Received-SPF: pass (google.com: domain of linux-nfs+bounces-2924-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
MIME-Version: 1.0
From: "NeilBrown" <neilb@suse.de>
To: "Chuck Lever" <chuck.lever@oracle.com>
Cc: "Jeff Layton" <jlayton@kernel.org>, "Olga Kornievskaia" <kolga@netapp.com>,
 "Dai Ngo" <Dai.Ngo@oracle.com>, "Tom Talpey" <tom@talpey.com>,
 "Petr Vorel" <pvorel@suse.cz>, linux-nfs@vger.kernel.org
Subject:
 Re: [PATCH] nfsd: don't fail OP_SETCLIENTID when there are lots of clients.
In-reply-to: <ZiZnbV+htcvGuGQl@tissot.1015granger.net>
References: <171375175915.7600.6526208866216039031@noble.neil.brown.name>,
 <ZiZnbV+htcvGuGQl@tissot.1015granger.net>
Date: Tue, 23 Apr 2024 09:33:46 +1000
Message-id: <171382882695.7600.8486072164212452863@noble.neil.brown.name>

On Mon, 22 Apr 2024, Chuck Lever wrote:
> On Mon, Apr 22, 2024 at 12:09:19PM +1000, NeilBrown wrote:
> > The calculation of how many clients the nfs server can manage is only an
> > heuristic.  Triggering the laundromat to clean up old clients when we
> > have more than the heuristic limit is valid, but refusing to create new
> > clients is not.  Client creation should only fail if there really isn't
> > enough memory available.
> >=20
> > This is not known to have caused a problem is production use, but
> > testing of lots of clients reports an error and it is not clear that
> > this error is justified.
>=20
> It is justified, see 4271c2c08875 ("NFSD: limit the number of v4
> clients to 1024 per 1GB of system memory"). In cases like these,
> the recourse is to add more memory to the test system.

Does each client really need 1MB?
Obviously we don't want all memory to be used by client state....

>=20
> However, that commit claims that the client is told to retry; I
> don't expect client creation to fail outright. Can you describe the
> failure mode you see?

The failure mode is repeated client retries that never succeed.  I think
an outright failure would be preferable - it would be more clear than
memory must be added.

The server has N active clients and M courtesy clients.
Triggering reclaim when N+M exceeds a limit and M>0 makes sense.
A hard failure (NFS4ERR_RESOURCE) when N exceeds a limit makes sense.
A soft failure (NFS4ERR_DELAY) while reclaim is running makes sense.

I don't think a retry while N exceeds the limit makes sense.

Do we have a count of active vs courtesy clients?

>=20
> Meanwhile, we need to have broader and more regular testing of NFSD
> on memory-starved systems. That's a long-term project.

:-)

Thanks,
NeilBrown


>=20
>=20
> > Signed-off-by: NeilBrown <neilb@suse.de>
> > ---
> >  fs/nfsd/nfs4state.c | 5 ++---
> >  1 file changed, 2 insertions(+), 3 deletions(-)
> >=20
> > diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> > index daf83823ba48..8a40bb6a4a67 100644
> > --- a/fs/nfsd/nfs4state.c
> > +++ b/fs/nfsd/nfs4state.c
> > @@ -2223,10 +2223,9 @@ static struct nfs4_client *alloc_client(struct xdr=
_netobj name,
> >  	struct nfs4_client *clp;
> >  	int i;
> > =20
> > -	if (atomic_read(&nn->nfs4_client_count) >=3D nn->nfs4_max_clients) {
> > +	if (atomic_read(&nn->nfs4_client_count) >=3D nn->nfs4_max_clients)
> >  		mod_delayed_work(laundry_wq, &nn->laundromat_work, 0);
> > -		return NULL;
> > -	}
> > +
> >  	clp =3D kmem_cache_zalloc(client_slab, GFP_KERNEL);
> >  	if (clp =3D=3D NULL)
> >  		return NULL;
> > --=20
> > 2.44.0
> >=20
> >=20
>=20
> --=20
> Chuck Lever
>=20