Received-SPF: pass (google.com: domain of linux-nfs+bounces-1196-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Precedence: bulk
MIME-Version: 1.0
References: <CAAvCNcDtTNDRvUVjUy4BE7eBCgmkb6hfkq3P0jaGDC=OXg0=6g@mail.gmail.com>
 <CAKAoaQmmEv+HRjmBMrSMGZn9RQr8C=2W4yeX4vNnohXFJPCV5A@mail.gmail.com>
 <65a29ca8.6b0a0220.ad415.d6d8.GMR@mx.google.com> <CAKAoaQkZ+b7NfrVi=gu1vCJBvv10=k85bG_kZV9G3jE45OOquw@mail.gmail.com>
 <0cd8fbfc707f86784dc7d88653b05cd355f89aad.camel@kernel.org> <24ACA376-5239-4941-BE53-70BF5E5E4683@oracle.com>
In-Reply-To: <24ACA376-5239-4941-BE53-70BF5E5E4683@oracle.com>
From: Roland Mainz <roland.mainz@nrubsig.org>
Date: Thu, 18 Jan 2024 02:56:12 +0100
Message-ID: <CAKAoaQny6G=JcKpJTYeLmNBEMgNkkc--T0Uvs1YbEX+JUD-PoA@mail.gmail.com>
Subject: RFE: Linux nfsd's |ca_maxoperations| should be at *least* |64| ... /
 was: Re: kernel.org list issues... / was: Fwd: Turn NFSD_MAX_* into tuneables
 ? / was: Re: Increasing NFSD_MAX_OPS_PER_COMPOUND to 96
To: Chuck Lever III <chuck.lever@oracle.com>, 
	Linux NFS Mailing List <linux-nfs@vger.kernel.org>, Jeff Layton <jlayton@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sat, Jan 13, 2024 at 5:10=E2=80=AFPM Chuck Lever III <chuck.lever@oracle=
.com> wrote:
> > On Jan 13, 2024, at 10:09=E2=80=AFAM, Jeff Layton <jlayton@kernel.org> =
wrote:
> > On Sat, 2024-01-13 at 15:47 +0100, Roland Mainz wrote:
> >> On Sat, Jan 13, 2024 at 1:19=E2=80=AFAM Dan Shelton <dan.f.shelton@gma=
il.com> wrote:
[snip]
> >> Is this the windows client?
> > No, the ms-nfs41-client (see
> > https://github.com/kofemann/ms-nfs41-client) uses a limit of |16|, but
> > it is on our ToDo list to bump that to |128| (but honoring the limit
> > set by the NFSv4.1 server during session negotiation) since it now
> > supports very long paths ([1]) and this issue is a known performance
> > bottleneck.
>
> A better way to optimize this case is to walk the path once
> and cache the terminal component's file handle. This is what
> Linux does, and it sounds like Dan's directory walker
> optimizations do effectively the same thing.

That assumes that no process does random access into deep subdirs. In
that case the performance is absolutely terrible, unless you devote
lots of memory to a giant cache (which is not feasible due to cache
expiration limits, unless someone (please!) finally implements
directory delegations).

This also ignores the use case of WAN (wide-area-networks) and WLAN
with the typical high latency and even higher amounts of network
package loss&&retransmit, where the splitting of the requests comes
with a HUGE latency penalty (you can reproduce this with network
tools, just export a large tmpfs on the server, add a package delay of
400ms between client and server, use a path like
"a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/0/1/2/3/4/5/6/7/8/9",
and compile gcc).

And in the real world the Linux nfsd |ca_maxoperations| default of
|16| is absolutely CRIPPELING.
For example in the mfs-nfs41-client we need 4 compounds for initial
setup for a file lookup, and then 3 per path component. That means
that a defaut of 16 just fits (16-4)/3=3D4 path elements.
Unfortunately the statistical average is not 4 - it's 11 (measured
over five weeks with 81 clients in our company).
Technically, in this scenario, a default of at least 11*3+4=3D37 would
be MUCH better.

That's why I think nfsd's |ca_maxoperations| should be at *least* |64|.

----

Bye,
Roland
--=20
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /=3D=3D\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)