Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH] NFS: Retry the CLOSE if the embedded GETATTR is rejected with
 ERR_STALE
Date:   Wed, 18 Nov 2020 21:29:35 +0000
From:   Anchal Agarwal <anchalag@amazon.com>
To:     Trond Myklebust <trondmy@hammerspace.com>
CC:     "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>,
        "anna.schumaker@netapp.com" <anna.schumaker@netapp.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Message-ID: <20201118212935.GA12762@dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com>
References: <20201118002431.GA6941@dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com>
 <a0babbb7d58f02769a6ce40d029768e7221acf24.camel@hammerspace.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <a0babbb7d58f02769a6ce40d029768e7221acf24.camel@hammerspace.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Precedence: bulk

On Wed, Nov 18, 2020 at 03:17:20AM +0000, Trond Myklebust wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> 
> 
> 
> On Wed, 2020-11-18 at 00:24 +0000, Anchal Agarwal wrote:
> > If our CLOSE RPC call is rejected with an ERR_STALE error, then we
> > should remove the GETATTR call from the compound RPC and retry.
> > This could happen in a scenario where two clients tries to access
> > the same file. One client opens the file and the other client removes
> > the file while it's opened by first client. When the first client
> > attempts to close the file the server returns ESTALE and the file
> > ends
> > up being leaked on the server. This depends on how nfs server is
> > configured and is not reproducible if running against nfsd.
> 
> That would be a seriously broken server. If you return NFS4ERR_STALE to
> the client, you cannot expect any further interaction with that file
> from the client. It won't try to send CLOSE or DELEGRETURN or any other
> stateful operation.
>
In this scenario, the setup we have at EFS is more of a distributed fashion. Multiple
clients are connected to multiple servers with a common filesystem. So the above
scenario leads to leaked open file handles on the client that tries to close deleted
file. So I was of the view, in that case client could retry close without getattr
in the close sequence without anything to do on server side.

Thanks,
Anchal Agarwal
> --
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace
> trond.myklebust@hammerspace.com
> 
>