Received-SPF: pass (google.com: domain of linux-nfs+bounces-1199-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Precedence: bulk
MIME-Version: 1.0
References: <CAAvCNcDtTNDRvUVjUy4BE7eBCgmkb6hfkq3P0jaGDC=OXg0=6g@mail.gmail.com>
 <CAKAoaQmmEv+HRjmBMrSMGZn9RQr8C=2W4yeX4vNnohXFJPCV5A@mail.gmail.com>
 <65a29ca8.6b0a0220.ad415.d6d8.GMR@mx.google.com> <CAKAoaQkZ+b7NfrVi=gu1vCJBvv10=k85bG_kZV9G3jE45OOquw@mail.gmail.com>
 <0cd8fbfc707f86784dc7d88653b05cd355f89aad.camel@kernel.org>
 <24ACA376-5239-4941-BE53-70BF5E5E4683@oracle.com> <CAKAoaQny6G=JcKpJTYeLmNBEMgNkkc--T0Uvs1YbEX+JUD-PoA@mail.gmail.com>
In-Reply-To: <CAKAoaQny6G=JcKpJTYeLmNBEMgNkkc--T0Uvs1YbEX+JUD-PoA@mail.gmail.com>
From: Martin Wege <martin.l.wege@gmail.com>
Date: Thu, 18 Jan 2024 10:44:40 +0100
Message-ID: <CANH4o6NcMbcNKxARcqhthXWkKk6_r31iKGjnS-RhFBB_AJFaJg@mail.gmail.com>
Subject: Re: RFE: Linux nfsd's |ca_maxoperations| should be at *least* |64|
 ... / was: Re: kernel.org list issues... / was: Fwd: Turn NFSD_MAX_* into
 tuneables ? / was: Re: Increasing NFSD_MAX_OPS_PER_COMPOUND to 96
To: Roland Mainz <roland.mainz@nrubsig.org>
Cc: Chuck Lever III <chuck.lever@oracle.com>, 
	Linux NFS Mailing List <linux-nfs@vger.kernel.org>, Jeff Layton <jlayton@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Jan 18, 2024 at 2:57=E2=80=AFAM Roland Mainz <roland.mainz@nrubsig.=
org> wrote:
>
> On Sat, Jan 13, 2024 at 5:10=E2=80=AFPM Chuck Lever III <chuck.lever@orac=
le.com> wrote:
> > > On Jan 13, 2024, at 10:09=E2=80=AFAM, Jeff Layton <jlayton@kernel.org=
> wrote:
> > > On Sat, 2024-01-13 at 15:47 +0100, Roland Mainz wrote:
> > >> On Sat, Jan 13, 2024 at 1:19=E2=80=AFAM Dan Shelton <dan.f.shelton@g=
mail.com> wrote:
> [snip]
> > >> Is this the windows client?
> > > No, the ms-nfs41-client (see
> > > https://github.com/kofemann/ms-nfs41-client) uses a limit of |16|, bu=
t
> > > it is on our ToDo list to bump that to |128| (but honoring the limit
> > > set by the NFSv4.1 server during session negotiation) since it now
> > > supports very long paths ([1]) and this issue is a known performance
> > > bottleneck.
> >
> > A better way to optimize this case is to walk the path once
> > and cache the terminal component's file handle. This is what
> > Linux does, and it sounds like Dan's directory walker
> > optimizations do effectively the same thing.
>
> That assumes that no process does random access into deep subdirs. In
> that case the performance is absolutely terrible, unless you devote
> lots of memory to a giant cache (which is not feasible due to cache
> expiration limits, unless someone (please!) finally implements
> directory delegations).
>
> This also ignores the use case of WAN (wide-area-networks) and WLAN
> with the typical high latency and even higher amounts of network
> package loss&&retransmit, where the splitting of the requests comes
> with a HUGE latency penalty (you can reproduce this with network
> tools, just export a large tmpfs on the server, add a package delay of
> 400ms between client and server, use a path like
> "a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/0/1/2/3/4/5/6/7/8/9"=
,
> and compile gcc).
>
> And in the real world the Linux nfsd |ca_maxoperations| default of
> |16| is absolutely CRIPPELING.
> For example in the mfs-nfs41-client we need 4 compounds for initial
> setup for a file lookup, and then 3 per path component. That means
> that a defaut of 16 just fits (16-4)/3=3D4 path elements.
> Unfortunately the statistical average is not 4 - it's 11 (measured
> over five weeks with 81 clients in our company).
> Technically, in this scenario, a default of at least 11*3+4=3D37 would
> be MUCH better.
>
> That's why I think nfsd's |ca_maxoperations| should be at *least* |64|.
>

+1

I consider the default value of 16 even a bug, given the circumstances.

Thanks,
Martin