Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   "Benjamin Coddington" <bcodding@redhat.com>
To:     "Patrick Goetz" <pgoetz@math.utexas.edu>
Cc:     "J. Bruce Fields" <bfields@fieldses.org>, linux-nfs@vger.kernel.org
Subject: Re: unsharing tcp connections from different NFS mounts
Date:   Wed, 07 Oct 2020 12:30:33 -0400
Message-ID: <A9A6D668-D40B-48F9-A578-E1BE1DCF25C4@redhat.com>
In-Reply-To: <df7b7b26-c6a7-9e6f-edf6-e3c858623462@math.utexas.edu>
References: <20201006151335.GB28306@fieldses.org>
 <df7b7b26-c6a7-9e6f-edf6-e3c858623462@math.utexas.edu>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk


On 7 Oct 2020, at 9:56, Patrick Goetz wrote:

> On 10/6/20 10:13 AM, J. Bruce Fields wrote:
>> NFSv4.1+ differs from earlier versions in that it always performs
>> trunking discovery that results in mounts to the same server sharing a
>> TCP connection.
>>
>> It turns out this results in performance regressions for some users;
>> apparently the workload on one mount interferes with performance of
>> another mount, and they were previously able to work around the problem
>> by using different server IP addresses for the different mounts.
>>
>> Am I overlooking some hack that would reenable the previous behavior?
>> Or would people be averse to an "-o noshareconn" option?
>>
>> --b.
>>
>
>
> I don't see how sharing a TCP connection can result in a performance
> regression (the performance degradation of *not* sharing a TCP connection
> is why HTTP 1.x is being replaced), or how using different IP addresses on
> the same interface resolves anything.  Does anyone have an explanation?

Well, I think the report we're getting may be using two different network
interfaces on the server-side.  The user was previously doing one mount each
to each ip address on each interface.

Even if you don't have this arrangement, it may still be possible/desirable
to have separate TCP connections if you want to prioritizes some NFS
traffic.  Multi-CPU systems with modern NICs have a number of different ways
to "steer" the traffic they receive to certain CPUs which may have a benefit
or detrimental effect on performance.  You can prioritize wake-ups from the
NIC based on throughput or latency, for example.

I don't know for sure which of these specific details are coming into play,
if any, though.

Ben