Received-SPF: softfail (google.com: domain of transitioning linux-nfs-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
From:   "Benjamin Coddington" <bcodding@redhat.com>
To:     "Trond Myklebust" <trondmy@hammerspace.com>
Cc:     linux-nfs@vger.kernel.org
Subject: Re: [PATCH v9 14/27] NFS: Improve heuristic for readdirplus
Date:   Fri, 11 Mar 2022 06:28:10 -0500
Message-ID: <A1B01312-F429-47AB-8EA3-4264C3FEF84A@redhat.com>
In-Reply-To: <28d6a094ddca6e4e6c15e055ec3ef6b10d57cbd3.camel@hammerspace.com>
References: <20220227231227.9038-1-trondmy@kernel.org>
 <20220227231227.9038-2-trondmy@kernel.org>
 <20220227231227.9038-3-trondmy@kernel.org>
 <20220227231227.9038-4-trondmy@kernel.org>
 <20220227231227.9038-5-trondmy@kernel.org>
 <20220227231227.9038-6-trondmy@kernel.org>
 <20220227231227.9038-7-trondmy@kernel.org>
 <20220227231227.9038-8-trondmy@kernel.org>
 <20220227231227.9038-9-trondmy@kernel.org>
 <20220227231227.9038-10-trondmy@kernel.org>
 <20220227231227.9038-11-trondmy@kernel.org>
 <20220227231227.9038-12-trondmy@kernel.org>
 <20220227231227.9038-13-trondmy@kernel.org>
 <20220227231227.9038-14-trondmy@kernel.org>
 <20220227231227.9038-15-trondmy@kernel.org>
 <A7BBBBF2-768E-487C-A890-7E5AF1D40027@redhat.com>
 <28d6a094ddca6e4e6c15e055ec3ef6b10d57cbd3.camel@hammerspace.com>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

On 10 Mar 2022, at 15:15, Trond Myklebust wrote:
>
> The problem is really that 'ls' (or possibly glibc) is passing in a
> pretty huge buffer to the getdents() system call.
>
> On my setup, that buffer appears to be 80K in size. So what happens is
> that we get that first getdents() call, and so we fill the 80K buffer
> with as many files as will fit. That can quickly run into several
> thousand entries, if the filenames are relatively short.
>
> Then 'ls' goes through the contents and does a stat() (or a statx()) on
> each entry, and so we record the statistics. However that means those
> first several thousand entries are indeed going to use cached data, or
> force GETATTR to go on the wire. We only start using forced readdirplus
> on the second pass.
>
> Yes, I suppose we could limit getdents() to ignore the buffer size, and
> just return fewer entries, however what's the "right" size in that
> case?

We can return fewer entries on the first call, so for the first pass the
right size is NFS_READDIR_CACHE_MISS_THRESHOLD + 1.  I sent a patch.

> More to the point, how much pain are we going to accept before we give
> up trying these assorted heuristics, and just define a readdirplus()
> system call modelled on statx()?

We cursed ourselves by creating the heuristic, and now we've had to maintain
it and try to make everyone happy.  The pain for us is when the behavior
keeps changing after sites have come to rely on previous performance.

I hope you can take a look at the patch.

Ben