Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20200608211945.GB30639@fieldses.org> <OSBPR01MB2949040AA49BC9B5F104DA1FEF9B0@OSBPR01MB2949.jpnprd01.prod.outlook.com>
 <22b841f7a8979f19009c96f31a7be88dd177a47a.camel@hammerspace.com>
 <20200618200905.GA10313@fieldses.org> <20200622135222.GA6075@fieldses.org> <20201001214749.GK1496@fieldses.org>
In-Reply-To: <20201001214749.GK1496@fieldses.org>
From:   Matt Benjamin <mbenjami@redhat.com>
Date:   Thu, 1 Oct 2020 18:26:25 -0400
Message-ID: <CAKOnarndL1-u5jGG2VAENz2bEc9wsERH6rGTbZeYZy+WyAUk-w@mail.gmail.com>
Subject: Re: client caching and locks
To:     "bfields@fieldses.org" <bfields@fieldses.org>
Cc:     Trond Myklebust <trondmy@hammerspace.com>,
        "inoguchi.yuki@fujitsu.com" <inoguchi.yuki@fujitsu.com>,
        "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

Hi Bruce,

I'm not sure.  My understanding has been that, NFSv4 does not mandate
a mechanism to update clients of changes outside of any locked range.
In AFS (and I think DCE DFS?) this role is played by DataVersion.  If
I recall correctly, David Noveck provided an errata that addresses
this, that servers could use in a similar manner to DV, but I don't
recall the details.

Matt

On Thu, Oct 1, 2020 at 5:48 PM bfields@fieldses.org
<bfields@fieldses.org> wrote:
>
> On Mon, Jun 22, 2020 at 09:52:22AM -0400, bfields@fieldses.org wrote:
> > On Thu, Jun 18, 2020 at 04:09:05PM -0400, bfields@fieldses.org wrote:
> > > I probably don't understand the algorithm (in particular, how it
> > > revalidates caches after a write).
> > >
> > > How does it avoid a race like this?:
> > >
> > > Start with a file whose data is all 0's and change attribute x:
> > >
> > >         client 0                        client 1
> > >         --------                        --------
> > >         take write lock on byte 0
> > >                                         take write lock on byte 1
> > >         write 1 to offset 0
> > >           change attribute now x+1
> > >                                         write 1 to offset 1
> > >                                           change attribute now x+2
> > >         getattr returns x+2
> > >                                         getattr returns x+2
> > >         unlock
> > >                                         unlock
> > >
> > >         take readlock on byte 1
> > >
> > > At this point a getattr will return change attribute x+2, the same as
> > > was returned after client 0's write.  Does that mean client 0 assumes
> > > the file data is unchanged since its last write?
> >
> > Basically: write-locking less than the whole range doesn't prevent
> > concurrent writes outside that range.  And the change attribute gives us
> > no way to identify whether concurrent writes have happened.  (At least,
> > not without NFS4_CHANGE_TYPE_IS_VERSION_COUNTER.)
> >
> > So as far as I can tell, a client implementation has no reliable way to
> > revalidate its cache outside the write-locked area--instead it needs to
> > just throw out that part of the cache.
>
> Does my description of that race make sense?
>
> --b.
>


-- 

Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-821-5101
fax.  734-769-8938
cel.  734-216-5309