Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH v2] xprtrdma: Provide a buffer to pad Write chunks of
 unaligned length
To:     Chuck Lever III <chuck.lever@oracle.com>
Cc:     Olga Kornievskaia <kolga@netapp.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
References: <163198229142.4159.3151965308454175921.stgit@morisot.1015granger.net>
 <d69234ec-4688-e5d1-17c3-247841d47d16@talpey.com>
 <BAFA239A-2F60-4163-AB20-046AFE6B6F7D@oracle.com>
 <04ec7294-f456-65cb-c401-5a56f61c8dc7@talpey.com>
 <5CE77E8E-97D1-4FFE-89AF-7536C3ABC6F0@oracle.com>
From:   Tom Talpey <tom@talpey.com>
Message-ID: <2e0f81a6-fb56-791f-4574-382596605135@talpey.com>
Date:   Wed, 22 Sep 2021 13:28:07 -0400
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <5CE77E8E-97D1-4FFE-89AF-7536C3ABC6F0@oracle.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 9/22/2021 1:18 PM, Chuck Lever III wrote:
> 
>> On Sep 22, 2021, at 1:15 PM, Tom Talpey <tom@talpey.com> wrote:
>>
>> On 9/21/2021 1:21 PM, Chuck Lever III wrote:
>>>> On Sep 21, 2021, at 12:58 PM, Tom Talpey <tom@talpey.com> wrote:
>>>>
>>>> On 9/18/2021 12:26 PM, Chuck Lever wrote:
>>>>> This is a buffer to be left persistently registered while a
>>>>> connection is up. Connection tear-down will automatically DMA-unmap,
>>>>> invalidate, and dereg the MR. A persistently registered buffer has
>>>>> no-cost to provide, and it can never be elided into the RDMA segment
>>>>> that backs the data payload.
>>>>
>>>> I'm confused by this last sentence. Why is "no-cost" important?
>>> Today, the client turns off this XDR pad when it can because it
>>> is registered and invalidated for every Write chunk with non-
>>> aligned length. That adds a cost to providing it.
>>> No-cost means we don't need to worry about optimizing it away;
>>> it can be provided all the time, if we like, because there's no
>>> additional per-RPC registration/invalidation cost.
>>
>> Sure, I get that, but it's not actually zero. The MR is allocated and
>> therefore consumes a handle. It's a way-better approach than registering a few bytes for each rpc of course. I guess I'm simply suggesting to delete the sentence. The other sentences cover the issue without the possible confusion.
> 
> There is a bug now where SG_GAPS support is coalescing the Write
> pad MR into the data payload, and for various reasons, that's
> not safe to do all the time.
> 
> So this patch addresses that, and this sentence calls out that
> specific part of the fix.

Wait, what? The sentence, as I read it, is talking about the possible
advantage of a pre-registered buffer, not a safety or s/g thing. If
this is fixing a bug, shouldn't that be described?

Tom.