DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B989204FD
In-Reply-To: <6416528545ec801c93e9baadc46216ed989ae464.camel@hammerspace.com>
To:     Trond Myklebust <trondmy@hammerspace.com>
Cc:     "bcodding@redhat.com" <bcodding@redhat.com>,
        "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>,
        "linux-nfs-owner@vger.kernel.org" <linux-nfs-owner@vger.kernel.org>,
        "malahal@gmail.com" <malahal@gmail.com>,
        "mbenjami@redhat.com" <mbenjami@redhat.com>
Subject: Re: "(deleted)" directories
From:   "Marc Eshel" <eshel@us.ibm.com>
Date:   Fri, 2 Nov 2018 19:37:47 -0800
References: <CAGD+tkyTyOb-iiykQLYMoa1h5cmizbe_0M0_HS=9cQC1NDznSw@mail.gmail.com>
         <24BEBD2F-BCC2-4AE0-81D7-185D6CAB8CD7@redhat.com>
         <CAGD+tkzyDoq6tdseFQu4ymJCc-+-2i6XEUmBGKe4mu4_ngv7QA@mail.gmail.com>
         <435a5a0fcdbefc30201c91b0a36b6159f6df32eb.camel@hammerspace.com>
         <CAKOnarkG4tpA7F306hgpk14XNELEFY1gaY7=QY4h6Dr-zSQzRw@mail.gmail.com>
         <a45a428ad45e5834fa7030088a75275584b13ab2.camel@hammerspace.com>
         <OF63D29519.B9156C69-ON8825833A.000DEE9B-8825833A.000E8123@notes.na.collabserv.com>
 <6416528545ec801c93e9baadc46216ed989ae464.camel@hammerspace.com>
MIME-Version: 1.0
Message-Id: <OF66B8FFB2.3625C228-ON8825833A.001382C6-8825833A.0013F02E@notes.na.collabserv.com>
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 8BIT
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk

You asked for a reason and I gave you one, the client shouldn't care for 
the reason
since the FH is opaque to the client. It is the server decision what to 
encode in the FH.
The client should just follow the spec.
Marc.

linux-nfs-owner@vger.kernel.org wrote on 11/02/2018 08:27:10 PM:

> From: Trond Myklebust <trondmy@hammerspace.com>
> To: "eshel@us.ibm.com" <eshel@us.ibm.com>
> Cc: "bcodding@redhat.com" <bcodding@redhat.com>, 
> "mbenjami@redhat.com" <mbenjami@redhat.com>, "linux-nfs-
> owner@vger.kernel.org" <linux-nfs-owner@vger.kernel.org>, "linux-
> nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>, 
> "malahal@gmail.com" <malahal@gmail.com>
> Date: 11/02/2018 08:27 PM
> Subject: Re: "(deleted)" directories
> Sent by: linux-nfs-owner@vger.kernel.org
> 
> On Fri, 2018-11-02 at 18:38 -0800, Marc Eshel wrote:
> > One reason to have different FHs for the same file is that a file can
> > be 
> > linked from multiple directories.
> > Adding the parent inode to the FH help finding the the name of the
> > file by 
> > looking for the file inode in
> > the parent directoy.
> > 
> 
> No... We've been there and done that. Encoding parent directories in
> the filehandle breaks rename, unlink, and makes it a pain to recover
> state.
> There is no way in hell we're going to commit to support that model.
> 
> > Marc.
> > 
> > linux-nfs-owner@vger.kernel.org wrote on 11/02/2018 05:15:42 PM:
> > 
> > > From: Trond Myklebust <trondmy@hammerspace.com>
> > > To: "mbenjami@redhat.com" <mbenjami@redhat.com>
> > > Cc: "bcodding@redhat.com" <bcodding@redhat.com>, "malahal@gmail.com
> > > "
> > > <malahal@gmail.com>, "linux-nfs@vger.kernel.org" 
> > <linux-nfs@vger.kernel.org>
> > > Date: 11/02/2018 05:15 PM
> > > Subject: Re: "(deleted)" directories
> > > Sent by: linux-nfs-owner@vger.kernel.org
> > > 
> > > On Fri, 2018-11-02 at 18:07 -0400, Matt Benjamin wrote:
> > > > It sounds like a pretty good one, that goes to the heart of what
> > > > a
> > > > specification is
> > > > 
> > > 
> > > While admittedly it is (still) Dia de los Muertos today, I would
> > > think
> > > that someone who resurrected a part of the NFSv3 spec that has been
> > > unused for the full 23 years of its existence might have some
> > > explanation for why they did so?
> > > 
> > > IOW: not being of a particularly religious persuasion, I usually
> > > want
> > > to understand why features are needed rather than having blind
> > > faith in
> > > the person who wrote the spec.
> > > 
> > > > Matt
> > > > 
> > > > On Fri, Nov 2, 2018 at 4:26 PM, Trond Myklebust <
> > > > trondmy@hammerspace.com> wrote:
> > > > > On Fri, 2018-11-02 at 21:24 +0530, Malahal Naineni wrote:
> > > > > > Ben, NFSv3 RFC1813.txt states: "If two file handles from the
> > > > > > same
> > > > > > server are equal, they must refer to the same file, but if
> > > > > > they  are
> > > > > > not equal, no conclusions can be drawn." Ganesha does return
> > > > > > same
> > > > > > fileid here (inode).
> > > > > > 
> > > > > > In NFSv4, they have introduced "unique_handles" attribute. I
> > > > > > don't
> > > > > > see
> > > > > > Linux NFS client using this at all though.
> > > > > 
> > > > > Why does your server need to have multiple filehandles refer to
> > > > > the
> > > > > same file, and why do you expect clients to support this?
> > > > > 
> > > > > Yes, the spec allows it, but that's not a sufficient reason.
> > > > > 
> > > > > > Regards, Malahal.
> > > > > > On Fri, Nov 2, 2018 at 4:35 PM Benjamin Coddington <
> > > > > > bcodding@redhat.com> wrote:
> > > > > > > On 2 Nov 2018, at 1:26, Malahal Naineni wrote:
> > > > > > > 
> > > > > > > > Hi All, we are using NFS-Ganesha with Linux NFS clients.
> > > > > > > > The
> > > > > > > > client's
> > > > > > > > shell reports the following. Based on lsof, the directory
> > > > > > > > is
> > > > > > > > marked
> > > > > > > > deleted. "cd to ROOT and cd to the same home directory
> > > > > > > > fixes
> > > > > > > > the
> > > > > > > > issue. The client behaves as though the directory is
> > > > > > > > deleted
> > > > > > > > and
> > > > > > > > recreated! Our NFS-Ganesha server implementation uses
> > > > > > > > multiple
> > > > > > > > file
> > > > > > > > handles that point to the same object. NFS spec says this
> > > > > > > > should
> > > > > > > > be
> > > > > > > > fine, but Linux NFS seems to be broken in this regard.
> > > > > > > > tcpdump
> > > > > > > > does
> > > > > > > > indicate file handle change (note that all file handles
> > > > > > > > are
> > > > > > > > permanent,
> > > > > > > > meaning they are valid at the server any time) around
> > > > > > > > this
> > > > > > > > issue
> > > > > > > > time.
> > > > > > > > 
> > > > > > > > "shell-init: error retrieving current directory: getcwd:
> > > > > > > > cannot
> > > > > > > > access
> > > > > > > > parent directories: No such file or directory"
> > > > > > > > sh        112544            malahal  cwd       DIR
> > > > > > > > 0,67
> > > > > > > >      65536   45605209 /home/malahal (deleted)
> > > > > > > > (10.120.154.42:/nfs/malahal-export/)
> > > > > > > > 
> > > > > > > > Function nfs_prime_dcache() seems to invalidate the
> > > > > > > > dcache
> > > > > > > > entry
> > > > > > > > if
> > > > > > > > nfs_same_file() returns false. nfs_same_file() does seem
> > > > > > > > to
> > > > > > > > return
> > > > > > > > false with the following change, if I read it correctly,
> > > > > > > > if
> > > > > > > > there
> > > > > > > > is a
> > > > > > > > file handle change. Can this be the source of my issue?
> > > > > > > > It
> > > > > > > > seems
> > > > > > > > that
> > > > > > > > the client should do this only if the file handle is NOT
> > > > > > > > valid
> > > > > > > > (e.g.
> > > > > > > > if it gets ESTALE), right?
> > > > > > > > 
> > > > > > > > The following commit seems to assume that the objects are
> > > > > > > > different if
> > > > > > > > they have different file handles!
> > > > > > > > commit 7dc72d5f7a0ec97a53e126c46e2cbd2560757955
> > > > > > > > Author: Trond Myklebust <trond.myklebust@primarydata.com>
> > > > > > > > Date:   Thu Sep 22 13:38:52 2016 -0400
> > > > > > > > 
> > > > > > > >     NFS: Fix inode corruption in nfs_prime_dcache()
> > > > > > > 
> > > > > > > My understanding is that for NFSv3 we have to assume that
> > > > > > > distinct
> > > > > > > filehandles are distinct objects, but maybe I'm wrong about
> > > > > > > this.
> > > > > > > 
> > > > > > > For NFSv4.x, we can follow the guidance in RFCs 5661 or
> > > > > > > 7530
> > > > > > > section 10.3.4
> > > > > > > to determine if the differing filehandles are the same
> > > > > > > object,
> > > > > > > specifically
> > > > > > > the fileid recommended attribute needs to be
> > > > > > > implemented.  Is
> > > > > > > Ganesha
> > > > > > > returning the same fileid for both filehandles?
> > > > > > > 
> > > > > > > Ben
> > > > > --
> > > > > Trond Myklebust
> > > > > CTO, Hammerspace Inc
> > > > > 4300 El Camino Real, Suite 105
> > > > > Los Altos, CA 94022
> > > > > www.hammer.space
> > > > > 
> > > > > 
> > > -- 
> > > Trond Myklebust
> > > CTO, Hammerspace Inc
> > > 4300 El Camino Real, Suite 105
> > > Los Altos, CA 94022
> > > www.hammer.space
> > > 
> > > 
> > 
> > 
> -- 
> Trond Myklebust
> CTO, Hammerspace Inc
> 4300 El Camino Real, Suite 105
> Los Altos, CA 94022
> www.hammer.space
> 
>