Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
From:   "NeilBrown" <neilb@suse.de>
To:     "Chuck Lever III" <chuck.lever@oracle.com>
Cc:     "Tom Talpey" <tom@talpey.com>, "Daire Byrne" <daire@dneg.com>,
        "Linux NFS Mailing List" <linux-nfs@vger.kernel.org>
Subject: Re: NFSv4 versus NFSv3 parallel client op/s
In-reply-to: <66383037-8263-4D7B-B96C-C9CED24042FC@oracle.com>
References: <CAPt2mGMZh9=Vwcqjh0J4XoTu3stOnKwswdzApL4wCA_usOFV_g@mail.gmail.com>,
 <6b528d29-1a9c-d16e-f649-5d994d6222b8@talpey.com>,
 <CAPt2mGOnbN=N5TqCWzVtX7CYoptpknCbnSXGfoX8X87DsvhoKw@mail.gmail.com>,
 <3849f322-94f7-fe73-4e08-1660be516384@talpey.com>,
 <66383037-8263-4D7B-B96C-C9CED24042FC@oracle.com>
Date:   Sat, 19 Feb 2022 11:43:20 +1100
Message-id: <164523140095.10228.17507004698722847604@noble.neil.brown.name>
Precedence: bulk

On Sat, 19 Feb 2022, Chuck Lever III wrote:
> 
> > On Feb 18, 2022, at 4:26 PM, Tom Talpey <tom@talpey.com> wrote:
> > 
> > 
> > On 2/18/2022 2:04 PM, Daire Byrne wrote:
> >>
> >> 2) Why is the default linux client slot count 64 and the server's is
> >> 32? You can tune the linux client down and not up (if using a Linux
> >> server).
> > 
> > That's for Trond and Chuck I guess.
> 
> For the Linux NFS server, there is an enhancement request open
> in this area:
> 
> https://bugzilla.linux-nfs.org/show_bug.cgi?id=375
> 
> If there are any relevant design notes or performance results,
> that would be the place to put them.

I wonder if I have a login there..

> 
> IIRC the only downside to a large default slot count on the
> server is that it can waste memory, and it is difficult to handle
> the corner cases when the server is running on a small physical
> host (or in a small container).

I would have a small default slot count (one page of slots??), which
automatically grew when it reached some level - say 70% - providing the
required kmalloc succeeded (with GFP_NORETRY or similar so that it
doesn't try to hard).  It would register a "shrinker" so that it could
respond to memory pressure and scale back the slot count when memory is
tight.

Freeing slot memory would be not be quick as you might need to wait for
the client to stop using it, so allocating new memory should be
correspondingly sluggish.

Shouldn't be too hard....  Definitely don't want a tunable for this.

NeilBrown