Received-SPF: pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <1566406146-7887-1-git-send-email-alex@zadara.com>
 <CAOcd+r0bXefi79dnwrwsDN1OecScfTjc8DYS5_9A8D5XKrh7QQ@mail.gmail.com>
 <20190826133951.GC22759@fieldses.org> <CAOcd+r059fh7J8T=6MdjPSCP39K5fpOZTsXZDUKq5TrPv_RcVQ@mail.gmail.com>
 <20190827205158.GB13198@fieldses.org> <CAOcd+r0Ybfr1WszjYc1K19Cf7JmKowy=Go6nc8Fexf5KxNyf=A@mail.gmail.com>
 <20190828165429.GC26284@fieldses.org> <CAOcd+r3e52q_ds3zjya98whYarqoXf5C2umNEX-AGp4-R6=Cuw@mail.gmail.com>
 <20190830190804.GB5053@fieldses.org>
In-Reply-To: <20190830190804.GB5053@fieldses.org>
From:   Alex Lyakas <alex@zadara.com>
Date:   Tue, 3 Sep 2019 16:47:47 +0300
Message-ID: <CAOcd+r2+uC_T6mhb-7s2p_xd762Ki+KNA=GjNUXtUYAZJknQ-w@mail.gmail.com>
Subject: Re: [RFC-PATCH] nfsd: when unhashing openowners, increment
 openowner's refcount
To:     "J. Bruce Fields" <bfields@fieldses.org>
Cc:     chuck.lever@oracle.com, linux-nfs@vger.kernel.org,
        Shyam Kaushik <shyam@zadara.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk

Hi Bruce,

On Fri, Aug 30, 2019 at 10:08 PM J. Bruce Fields <bfields@fieldses.org> wrote:
>
> On Thu, Aug 29, 2019 at 09:12:49PM +0300, Alex Lyakas wrote:
> > We evaluated the network namespaces approach. But, unfortunately, it
> > doesn't fit easily into how our system is currently structured. We
> > would have to create and configure interfaces for every namespace, and
> > have a separate IP address (presumably) for every namespace.
>
> Yes.
>
> > All this
> > seems a bit of an overkill, to just have several local filesystems
> > exported to the same client (which is when we hit the issue). I would
> > assume that some other users would argue as well that creating a
> > separate network namespace for every local filesystem is not the way
> > to go from the administration point of view.
>
> OK, makes sense.
>
> And I take it you don't want to go around to each client and shut things
> down cleanly.  And you're fine with the client applications erroring out
> when you yank their filesystem out from underneath them.
It's not that we don't want to unmount at each client. The problem is
that we don't control the client machines, as they are owned by
customers. We definitely recommend customers to unmount, before
un-exporting. But in some cases it still doesn't happen, most notably
in automated environments. Since the un-export operation is initiated
by customer, I presume she understands that the nfs client might error
out, if not unmounted properly beforehand.

>
> (I wonder what happens these days when that happens on a linux client
> when there are dirty pages.  I think you may just end up with a useless
> mount that you can't get rid of till you power down the client.)
Right, in some cases, the IO process gets stuck forever.

>
> > Regarding the failure injection code, we did not actually enable and
> > use it. We instead wrote some custom code that is highly modeled after
> > the failure injection code.
>
> Sounds interesting....  I'll try to understand it and give some comments
> later.
> ...
> > Currently this code is invoked from a custom procfs entry, by
> > user-space application, before unmounting the local file system.
> >
> > Would moving this code into the "unlock_filesystem" infrastructure be
> > acceptable?
>
> Yes.  I'd be interested in patches.
>
> > Since the "share_id" approach is very custom for our
> > usage, what criteria would you suggest for selecting the openowners to
> > be "forgotten"?
>
> The share_id shouldn't be necessary.  I'll think about it.
>
> --b.