Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Message-ID: <1f13f001-e4d7-fdcd-6575-caa1be1526e1@linux.ibm.com>
Date:   Tue, 25 Jan 2022 10:42:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.3.0
Subject: Re: [RFC PATCH net-next] net/smc: Introduce receive queue flow
 control support
Content-Language: en-US
To:     Guangguan Wang <guangguan.wang@linux.alibaba.com>,
        kgraul@linux.ibm.com, davem@davemloft.net, kuba@kernel.org
Cc:     linux-s390@vger.kernel.org, netdev@vger.kernel.org,
        linux-kernel@vger.kernel.org
References: <20220120065140.5385-1-guangguan.wang@linux.alibaba.com>
From:   Stefan Raspl <raspl@linux.ibm.com>
In-Reply-To: <20220120065140.5385-1-guangguan.wang@linux.alibaba.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 1/20/22 07:51, Guangguan Wang wrote:
> This implement rq flow control in smc-r link layer. QPs
> communicating without rq flow control, in the previous
> version, may result in RNR (reveive not ready) error, which
> means when sq sends a message to the remote qp, but the
> remote qp's rq has no valid rq entities to receive the message.
> In RNR condition, the rdma transport layer may retransmit
> the messages again and again until the rq has any entities,
> which may lower the performance, especially in heavy traffic.
> Using credits to do rq flow control can avoid the occurrence
> of RNR.

That's some truly substantial improvements!
But we need to be careful with protocol-level changes: There are other operating 
systems like z/OS and AIX which have compatible implementations of SMC, too. 
Changes like a reduction of connections per link group or usage of reserved 
fields would need to be coordinated, and likely would have unwanted side-effects 
even when used with older Linux kernel versions.
Changing the protocol is "expensive" insofar as it requires time to thoroughly 
discuss the changes, perform compatibility tests, and so on.
So I would like to urge you to investigate alternative ways that do not require 
protocol-level changes to address this scenario, e.g. by modifying the number of 
completion queue elements, to see if this could yield similar results.

Thx!