Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <CAN-5tyE4n2WgQhtuX_1JZsHnyXXZgwGJbRYjw5jrA+bfVcC3uw@mail.gmail.com>
 <3136254d57e8e17537ea0d21a195293f162e42b8.camel@hammerspace.com>
In-Reply-To: <3136254d57e8e17537ea0d21a195293f162e42b8.camel@hammerspace.com>
From:   Olga Kornievskaia <aglo@umich.edu>
Date:   Mon, 28 Feb 2022 15:37:52 -0500
Message-ID: <CAN-5tyHyC8w90GWmCzvfqR_KYVx6S5EO2Qoaon9-UN5-UO_i=Q@mail.gmail.com>
Subject: Re: managing trunking
To:     Trond Myklebust <trondmy@hammerspace.com>
Cc:     "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, Feb 28, 2022 at 3:03 PM Trond Myklebust <trondmy@hammerspace.com> wrote:
>
> On Mon, 2022-02-28 at 13:58 -0500, Olga Kornievskaia wrote:
> > Hello folks,
> >
> > I would like to ask for what would be an acceptable solution to deal
> > with created trunking connections when there is a change in trunking
> > membership, specifically when a connection (ie., its endpoint) no
> > longer is part of the same group. An inaction on the client's part
> > would lead to unusable client.
> >
> > Would a proposal to destroy trunking connections in as part of
> > DESTROY_SESSION be acceptable? The logic behind this solution is that
> > trunking membership was established as a part of a session, each
> > connection was tested to belong to trunkable server and added to that
> > session. Once the session is destroy and new created there is no
> > guarantee that the connections are to the same server that the new
> > session is created for. Trunking membership can be re-established at
> > a
> > later time. I have some code that implements this solution but still
> > needs some testing.
> >
> > Alternatively, if we keep connections past DESTROY_SESSION, then we
> > need a way to test that the same connections belong to the new
> > session
> > that has been created, meaning that a probe for each connection on
> > create_session to see if they still belong to the same server as the
> > new session is created. Is that preferred over simply destroying
> > connections? I'm going to work on implementing this too and posting
> > as
> > an alternative.
> >
> > It has been expressed several times that the ultimate goal is to do
> > transport management in userspace so does it mean the solution to
> > this
> > is also in userspace? Should there be upcalls to user land on
> > DESTROY_SESSION and CREATE_SESSION to destroy/create trunking
> > respectively but triggered via user land. But in this approach, while
> > this happens at user land speed,  will we be allowing the client to
> > get into a state where it's unusable (because its connections are
> > talking to servers that don't belong to the same trunking group)? Or
> > to prevent this, will we be allowing the userland to pause activities
> > in the kernel until the transports are squared away? I just don't see
> > how out-sourcing trunking membership changes to the user land is
> > better than handling it in the kernel when no operations can proceed
> > until trunking membership is corrected.
> >
> > Any feedback on the approaches or its alternatives would be much
> > appreciated.
> >
> > Thank you for the feedback.
> >
> > Thank you.
>
> Right now, we only call DESTROY_SESSION on the final unmount of the
> volumes on a given server, just before calling DESTROY_CLIENTID to
> destroy the lease. So the point is really moot within the current
> framework.
>
> Is it therefore your intention to change when we call DESTROY_SESSION?

No I do not propose to change the when we call destroy_session.
Destroy_session is triggered when the client receives a BAD_SESSION
error.

I'm not a fan of this approach. I think the 2nd approach that tests
the connections is a better one.


>
> --
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace
> trond.myklebust@hammerspace.com
>
>