Received-SPF: pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <155917564898.3988.6096672032831115016.stgit@noble.brown>
 <1df23ebc-ffe5-1a57-c40a-d5e9a45c8498@talpey.com> <CAN-5tyHUah=fAsiSLb=n__q5HxRy3qeS83evf-vB59r4cpYgsw@mail.gmail.com>
 <9b64b9d9-b7cf-c818-28e2-58b3a821d39d@talpey.com>
In-Reply-To: <9b64b9d9-b7cf-c818-28e2-58b3a821d39d@talpey.com>
From:   Olga Kornievskaia <aglo@umich.edu>
Date:   Thu, 30 May 2019 14:41:44 -0400
Message-ID: <CAN-5tyEBb3_TLvuN49FBB6qz2kXSZLPxHYZfQQX0jU4qFt4Z4A@mail.gmail.com>
Subject: Re: [PATCH 0/9] Multiple network connections for a single NFS mount.
To:     Tom Talpey <tom@talpey.com>
Cc:     NeilBrown <neilb@suse.com>, Chuck Lever <chuck.lever@oracle.com>,
        Schumaker Anna <Anna.Schumaker@netapp.com>,
        Trond Myklebust <trondmy@hammerspace.com>,
        linux-nfs <linux-nfs@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk

On Thu, May 30, 2019 at 1:41 PM Tom Talpey <tom@talpey.com> wrote:
>
> On 5/30/2019 1:20 PM, Olga Kornievskaia wrote:
> > On Thu, May 30, 2019 at 1:05 PM Tom Talpey <tom@talpey.com> wrote:
> >>
> >> On 5/29/2019 8:41 PM, NeilBrown wrote:
> >>> I've also re-arrange the patches a bit, merged two, and remove the
> >>> restriction to TCP and NFSV4.x,x>=1.  Discussions seemed to suggest
> >>> these restrictions were not needed, I can see no need.
> >>
> >> I believe the need is for the correctness of retries. Because NFSv2,
> >> NFSv3 and NFSv4.0 have no exactly-once semantics of their own, server
> >> duplicate request caches are important (although often imperfect).
> >> These caches use client XID's, source ports and addresses, sometimes
> >> in addition to other methods, to detect retry. Existing clients are
> >> careful to reconnect with the same source port, to ensure this. And
> >> existing servers won't change.
> >
> > Retries are already bound to the same connection so there shouldn't be
> > an issue of a retransmission coming from a different source port.
>
> So, there's no path redundancy? If any connection is lost and can't
> be reestablished, the requests on that connection will time out?

For v3 and v4.0 in the current code base with a single connection,
when it goes down, you are out of luck. When we have multiple
connections and would like the benefit of using them but not
sacrifices replay cache correctness, it's a small price to restrict
the re-transmissions and suffer the consequence of not being able to
do an operation during network issues.

> I think a common configuration will be two NICs and two network paths,

Are you talking about session trunking here?

Why do you think two NICs would be a common configuration. I have
performance numbers that demonstrate performance improvement for a
single NIC case. I would say a single NIC with a high speed networks
(25/40G) would be a common configuration.

> a so-called shotgun. Admins will be quite frustrated to discover it
> gives no additional robustness, and perhaps even less.
>
> Why not simply restrict this to the fully-correct, fully-functional
> NFSv4.1+ scenario, and not try to paper over the shortcomings?

I think mainly because customers are still using v3 but want to
improve performance. I'd love for everybody to switch to 4.1 but
that's not happening.

>
> Tom.
>
> >
> >> Multiple connections will result in multiple source ports, and possibly
> >> multiple source addresses, meaning retried client requests may be
> >> accepted as new, rather than having any chance of being recognized as
> >> retries.
> >>
> >> NFSv4.1+ don't have this issue, but removing the restrictions would
> >> seem to break the downlevel mounts.
> >>
> >> Tom.
> >>
> >
> >