Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 6 Sep 2022 09:00:11 -0400
To:     dai.ngo@oracle.com
Cc:     Chuck Lever III <chuck.lever@oracle.com>,
        Jeff Layton <jlayton@kernel.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        steved@redhat.com
Subject: Re: [PATCH v4 2/2] NFSD: add shrinker to reap courtesy clients on
 low memory condition
Message-ID: <20220906130011.GB25323@fieldses.org>
References: <FA83E721-C874-4A47-87BA-54B13E0B12A3@oracle.com>
 <2df6f1fe-c8eb-d5a1-0a11-2fd965555a33@oracle.com>
 <7041D47D-ECB3-497E-9174-96E9E36FFBDE@oracle.com>
 <eb197dde-8758-7ef4-8a7b-989273e09abc@oracle.com>
 <3CD64E7F-8E81-4B37-AAF3-499B47B25D19@oracle.com>
 <fc5a3aa7-af8e-656d-a16e-c07c201ec62a@oracle.com>
 <44A716C4-3904-424D-A5D6-CE46FC9145F0@oracle.com>
 <2b9549f8-58ec-be8a-1b15-3a6d7751a04f@oracle.com>
 <5B4E5FAA-5D46-4E43-962D-64AFDC035C41@oracle.com>
 <1f395e2b-695f-836f-2038-6a672c651d53@oracle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1f395e2b-695f-836f-2038-6a672c651d53@oracle.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
From:   bfields@fieldses.org (J. Bruce Fields)
Precedence: bulk

On Sat, Sep 03, 2022 at 10:59:17AM -0700, dai.ngo@oracle.com wrote:
> On 9/3/22 10:29 AM, Chuck Lever III wrote:
> >What I was suggesting was a longer term strategy for improving the
> >laundromat. In order to scale well in the number of clients, it
> >needs to schedule client expiry and deletion without serializing.
> >
> >(ie, the laundromat itself can identify a set of clients to clean,
> >but then it should pass that list to other workers so it can run
> >again as soon as it needs to -- and that also means it can use more
> >than just one CPU at a time to do its work).
> 
> I see. Currently on my lowly 1-CPU VM it takes about ~35 secs to
> destroy 128 clients, each with only few states (generated by pynfs's
> CID5 test). We can improve on this.

Careful--it's not the CPU that's the issue, it's waiting for disk.

If you're on a hard drive, for example, it's going to take at least one
seek (probably at least 10ms) to expire a single client, so you're never
going to destroy more than 100 per second.  That's what you need to
parallelize.  See item 3 from

	https://lore.kernel.org/linux-nfs/20220523154026.GD24163@fieldses.org/

Also, looks like current nfs-utils is still doing 3 commits per expiry.

Steve, for some reason I think "nfsdcld: use WAL journal for faster
commits" never got applied:

	https://lore.kernel.org/linux-nfs/20220104222445.GF12040@fieldses.org/

--b.