Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
Date:   Tue, 28 Sep 2021 07:11:41 +0200
From:   Salvatore Bonaccorso <bonaccos@ee.ethz.ch>
To:     "J. Bruce Fields" <bfields@fieldses.org>
Cc:     linux-nfs@vger.kernel.org, Chuck Lever <chuck.lever@oracle.com>
Subject: Re: nfsd4_process_open2 failed to open newly-created file!
 status=10008 ; warning at fs/nfsd/nfs4proc.c for nfsd4_open
In-Reply-To: <20210927155338.GA30593@fieldses.org>
References: <20210927061025.GA20892@varda.ee.ethz.ch>
 <20210927155338.GA30593@fieldses.org>
User-Agent: Roundcube Webmail/1.4.11
Message-ID: <cd38d56c6824ac6776df838dfd66bccd@ee.ethz.ch>
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi Bruce,

On 27.09.2021 17:53, J. Bruce Fields wrote:
> On Mon, Sep 27, 2021 at 08:10:31AM +0200, Salvatore Bonaccorso wrote:
>> We recently got the following traces on a NFS server, but I'm not sure
>> how to further debug this, any hints?
> 
> The server creates and opens a file in two steps, though it should
> really be a single atomic operation.
> 
> That means there's a small possibility somebody could intervene and do
> something like change the permissions:
> 
>> 
>> [5746893.904448] ------------[ cut here ]------------
>> [5746893.910050] nfsd4_process_open2 failed to open newly-created 
>> file! status=10008
> 
> 10008 is NFS4ERR_DELAY, so maybe somebody managed to get a delegation
> before we finished opening?
> 
> We should be able to prevent that....
> 
> In your setup are there processes quickly opening new files created by
> others?

This is very possible. The NFS server is used as a "scratch" place 
accessible from
compute cluster where people can have multiple jobs simultaneously 
running through
Slurm and accessing the data. So it is possible that user create new 
files from
one running instance and accessing it quickly from the other nodes.

I'm so far was unable to arificially trigger the issue but is there 
anything I
can try out to get more information useful for you?

Regards,
Salvatore