Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Mon, 1 Mar 2021 20:06:29 -0500
From:   "J. Bruce Fields" <bfields@fieldses.org>
To:     Timo Rothenpieler <timo@rothenpieler.org>
Cc:     Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: NFSD Regression: client observing a file while other client
 writes to it leads to stale local cache
Message-ID: <20210302010629.GB16303@fieldses.org>
References: <c3efd7e8-dc08-ac1f-9bee-7a7b0d8ac5fb@rothenpieler.org>
 <20210301184633.GA14881@fieldses.org>
 <86ef3b71-bcb4-767c-40a0-461d082f5d44@rothenpieler.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <86ef3b71-bcb4-767c-40a0-461d082f5d44@rothenpieler.org>
User-Agent: Mutt/1.5.21 (2010-09-15)
Precedence: bulk

On Tue, Mar 02, 2021 at 02:03:58AM +0100, Timo Rothenpieler wrote:
> On 01.03.2021 19:46, J. Bruce Fields wrote:
> >Thanks for the bisecting this and reporting the results.
> >
> >The behavior you describe is probably not a bug: NFS close-to-open
> >caching semantics don't guarantee you'll see updates on the reading
> >client in this case.
> >
> >Nevertheless, I don't understand why the behavior changed with that
> >commit, and I'd like to.
> >
> >The only change in behavior I'd expect would be that the writing client
> >would be granted a read delegation.  But I wouldn't expect that to
> >change how it writes back data, or how the reading client checks for
> 
> The writing side of things does not seem to be affected.
> At any point during testing, the file on the servers filesystem is
> in the state I'd expect it to be, growing by one line per second.
> 
> What's also a bit odd about this is that after both clients have
> closed the file, the reading client still sees an older version of
> the file, the one state it was in when it was first opened on the
> client.
> The file size in stat()/ls -l mismatches between the reading client
> and the server.
> It stays in that state for some seemingly arbitrary amount of time.
> Observing the file in any way(cat/tail, or even just ls) appears to
> extend the wait time.
> After leaving the file alone for long enough on the reading client,
> it snaps back to reality.

Hm, I wonder if we're (incorrectly) giving out a delegation to the
reading client.  Is this 100% reproduceable?

--b.

> This change in behavior, while clearly an edge case, is quite
> devastating for our use case. And also somewhat counter-intuitive.
> 
> It's the log output of cluster jobs sent off to compute nodes by the users.
> They then observe what the job is doing via tail -f on the log file
> on the login node.
>