Date:   Tue, 4 Jan 2022 10:32:05 -0500
From:   "bfields@fieldses.org" <bfields@fieldses.org>
To:     Trond Myklebust <trondmy@hammerspace.com>
Cc:     "inoguchi.yuki@fujitsu.com" <inoguchi.yuki@fujitsu.com>,
        "neilb@suse.de" <neilb@suse.de>,
        "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>,
        "mbenjami@redhat.com" <mbenjami@redhat.com>
Subject: Re: client caching and locks
Message-ID: <20220104153205.GA7815@fieldses.org>
References: <22b841f7a8979f19009c96f31a7be88dd177a47a.camel@hammerspace.com>
 <20200618200905.GA10313@fieldses.org>
 <20200622135222.GA6075@fieldses.org>
 <20201001214749.GK1496@fieldses.org>
 <CAKOnarndL1-u5jGG2VAENz2bEc9wsERH6rGTbZeYZy+WyAUk-w@mail.gmail.com>
 <20201006172607.GA32640@fieldses.org>
 <164066831190.25899.16641224253864656420@noble.neil.brown.name>
 <20220103162041.GC21514@fieldses.org>
 <OSZPR01MB7050F9737016E8E3F0FD5255EF4A9@OSZPR01MB7050.jpnprd01.prod.outlook.com>
 <03e4cc01e9e66e523474c10846ee22147b78addf.camel@hammerspace.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <03e4cc01e9e66e523474c10846ee22147b78addf.camel@hammerspace.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Precedence: bulk

On Tue, Jan 04, 2022 at 12:36:14PM +0000, Trond Myklebust wrote:
> On Tue, 2022-01-04 at 09:24 +0000, inoguchi.yuki@fujitsu.com wrote:
> > Hi Neil and Bruce,
> > 
> > Thank you for your comments.
> > Now I have understood that this behavior is by design.
> > 
> > > > With NFSv4 there is no atomicity guarantees relating to writes
> > > > and
> > > > changeid.
> > > > There is provision for atomicity around directory operations, but
> > > > not
> > > > around data operations.
> > 
> > So I feel like clients cannot always trust changeid in NFSv4. 
> > Should it be described in the spec?
> > 
> > For example, the following section refers about the usage of
> > changeid:
> > https://datatracker.ietf.org/doc/html/draft-dnoveck-nfsv4-rfc5661bis#section-14.3.1
> > 
> > It says clients use change attribute " to ensure that the data for
> > the OPENed file is still 
> > correctly reflected in the client's cache." But in fact, it could be
> > wrong.
> 
> The existence of buggy servers is not a problem for the client to deal
> with. It's a problem for the server vendors to fix.

I agree that, in the absence of bugs, there's no problem with using the
change attribute to provide close-to-open cache consistency.

The interesting question to me is how you use locks to provide cache
consistency.

Language like that in
https://datatracker.ietf.org/doc/html/rfc7530#section-10.3.2 implies
that you can get something like local cache consistency by write-locking
the ranges you write, read-locking the ranges you read, flushing before
write unlocks, and checking change attributes before read locks.

In fact, that doesn't guarantee that readers see previous writes.

--b.