Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <CAPt2mGOnsA9pcmZQkr2q40d7A+NLj7=xr+dzFh7XwJPdGYW6Hw@mail.gmail.com>
 <a4abb5fcf94d706cc3f47d6b629763d5b1831c21.camel@hammerspace.com>
 <CAPt2mGMOSHssr_J6bcf8A8dnU_oHNf_UuHZsDk1WxVi=TUheWA@mail.gmail.com> <561ef18af88ecda0f7b8abf55c1dfb2b66cf5dea.camel@hammerspace.com>
In-Reply-To: <561ef18af88ecda0f7b8abf55c1dfb2b66cf5dea.camel@hammerspace.com>
From:   Daire Byrne <daire@dneg.com>
Date:   Thu, 1 Sep 2022 16:49:04 +0100
Message-ID: <CAPt2mGNm11o3-b+W66eUUj=bvW-XV9wuiU+_uG+zigFPTQ6TwA@mail.gmail.com>
Subject: Re: directory caching & negative file lookups?
To:     Trond Myklebust <trondmy@hammerspace.com>
Cc:     "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

Yea, got it now. That all makes sense. Thanks!

Apologies for the noise. Now I just have to go and fix a bunch of our
user's code so I can forget about negative lookups again...

Daire

On Thu, 1 Sept 2022 at 16:43, Trond Myklebust <trondmy@hammerspace.com> wrote:
>
> On Thu, 2022-09-01 at 16:27 +0100, Daire Byrne wrote:
> > On Thu, 1 Sept 2022 at 14:55, Trond Myklebust
> > <trondmy@hammerspace.com> wrote:
> > > man 5 nfs
> > >
> > > Look for the section on the 'lookupcache=mode' mount option.
> >
> > So I get that the client caches negative lookups once we've made them
> > (the default lookupcache=all), but what I'm wondering is if we have
> > already cached the entire directory contents before the (negative)
> > lookup, can we not reply that it doesn't exist using that information
> > without having to go across the wire the at all (even the first
> > time)?
> >
> > Or is there no concept of "cached directory contents"? I thought that
> > maybe readdir/readdirplus knew about the "full contents" of a
> > directory?
> >
> > My thinking was that if we did a readdir/readirplus first, we could
> > then do lookups for any random non-existent filename without having
> > to
> > send anything across the wire. Like I said, a newbie question with
> > limited understanding of the actual internals :)
> >
> > Daire
>
> There is no concept of a 'fully cached directory'. The VFS and the
> memory management code are free to kick out any unused cached entries
> from the dcache at any time and for any reason. So the absence of an
> entry is not the same as a negative entry.
>
> Furthermore, certain features like case insensitive filesystems on
> servers makes it hard for the NFS client to know whether or not a
> specific name will or won't match an entry returned by readdir. In
> those circumstances, even if you think you have cached the entire
> directory, you are not guaranteed to know whether the lookup will fail
> or succeed.
>
> --
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace
> trond.myklebust@hammerspace.com
>
>