Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Message-ID: <f7c7ba0e-06f6-4ee6-b8f1-edff497418d1@redhat.com>
Date:   Fri, 29 Oct 2021 11:24:15 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.2.0
Subject: Re: [PATCH 0/1] Enable inter server to server copies on a export
Content-Language: en-US
To:     "J. Bruce Fields" <bfields@fieldses.org>
Cc:     Linux NFS Mailing list <linux-nfs@vger.kernel.org>
References: <20211028144851.644018-1-steved@redhat.com>
 <20211029134534.GA19967@fieldses.org> <20211029142639.GC19967@fieldses.org>
From:   Steve Dickson <steved@redhat.com>
In-Reply-To: <20211029142639.GC19967@fieldses.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 10/29/21 10:26, J. Bruce Fields wrote:
> On Fri, Oct 29, 2021 at 09:45:34AM -0400, bfields wrote:
>> On Thu, Oct 28, 2021 at 10:48:50AM -0400, Steve Dickson wrote:
>>> This kernel patch and an upcoming nfs-utils patch
>>> adds a new export option 's2sc' which will allow
>>> inter server to server copies.
>>
>> They're already allowed by a module option.  Why is an export option
>> better?  And why should it be set on the destination server and not the
>> source server?
>>
>> Really, first I think we should try to identify what the problem is that
>> we're trying to solve.
> 
> I guess we're thinking: we expect server-to-server copy to be a win in
> some cases, but not others.
Right!

> 
> What would those cases look like?
> 
> Say you've got a client "C" and two servers, "S" and "T", and C is
> copying a file from S to T.
> 
> I'd expect bandwidth of the traditional read-write-loop copy to be the
> minimum of the network bandwidth between S and C, and the network
> bandwidth between C and T.  Are there common cases were the S-to-T
> bandwidth would be significantly less than both of those?
I was thinking the connection between S-to-T would be much
faster (come type of cluster interconnect) that the connection
between C-S or C-T.

> 
> My guess would've been that that's relatively unusual.  So as a first
> pass, just turning on COPY unconditionally doesn't seem so bad.
> 
> I know you're thinking about in cases where S and T are connected by a
> high-performance network not necessarily available to C.
Exactly... well said.

> 
> In that case, we know we want to use server-to-server copy for copies
> between S and T.  But is it necessarily a problem to also use it for
> copies between some other server and T?  Also, does knowing the export
> containing the destination file on T really tell you whether or not the
> copy will be coming from S and not from some other server?
No... All the export is doing is allowing the COPY request
to succeed. I don't think T has any idea where the
COPY is coming from... but you would know better that I :-)

> 
> I'd think the bigger issue in that case is figuring out how to configure
> S so that it returns the right IP address in the cnr_source_server field
> of the COPY_NOTIFY reply.  Currently we return address that the client
> sent the COPY_NOTIFY, and I don't know if that's correct for that case.
I didn't look at this part of the transaction since it was
not controlled by the module param... But How would S know its the
"right IP address" since it is C that did the mount to T and
is giving S the address?

Keeping in context of what I'm trying to do... All I'm
trying to do is enable the feature, and with your points,
I think you are making my case there should be an
alternate of enabling these copies.

steved.