Received-SPF: pass (google.com: domain of linux-kernel+bounces-68741-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <350f1cb8-b205-47be-a296-c610b9afe5fc@linux.ibm.com>
Date: Fri, 16 Feb 2024 15:25:04 +0100
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Wenjia Zhang <wenjia@linux.ibm.com>
Subject: Re: [PATCH net-next 13/15] net/smc: introduce loopback-ism DMB type
 control
To: Wen Gu <guwen@linux.alibaba.com>, wintera@linux.ibm.com, hca@linux.ibm.com,
        gor@linux.ibm.com, agordeev@linux.ibm.com, davem@davemloft.net,
        edumazet@google.com, kuba@kernel.org, pabeni@redhat.com,
        jaka@linux.ibm.com, Gerd Bayer <gbayer@linux.ibm.com>
Cc: borntraeger@linux.ibm.com, svens@linux.ibm.com, alibuda@linux.alibaba.com,
        tonylu@linux.alibaba.com, linux-s390@vger.kernel.org,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org
References: <20240111120036.109903-1-guwen@linux.alibaba.com>
 <20240111120036.109903-14-guwen@linux.alibaba.com>
Content-Language: en-GB
In-Reply-To: <20240111120036.109903-14-guwen@linux.alibaba.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit


On 11.01.24 13:00, Wen Gu wrote:
> This provides a way to {get|set} type of DMB offered by loopback-ism,
> whether it is physically or virtually contiguous memory.
> 
> echo 0 > /sys/devices/virtual/smc/loopback-ism/dmb_type # physically
> echo 1 > /sys/devices/virtual/smc/loopback-ism/dmb_type # virtually
> 
> The settings take effect after re-activating loopback-ism by:
> 
> echo 0 > /sys/devices/virtual/smc/loopback-ism/active
> echo 1 > /sys/devices/virtual/smc/loopback-ism/active
> 
> After this, the link group and DMBs related to loopback-ism will be
> flushed and subsequent DMBs created will be of the desired type.
> 
> The motivation of this control is that physically contiguous DMB has
> best performance but is usually expensive, while the virtually
> contiguous DMB is cheap and perform well in most scenarios, but if
> sndbuf and DMB are merged, virtual DMB will be accessed concurrently
> in Tx and Rx and there will be a bottleneck caused by lock contention
> of find_vmap_area when there are many CPUs and CONFIG_HARDENED_USERCOPY
> is set (see link below). So an option is provided.
> 
I'm courious about why you say that physically contiguous DMB has best 
performance. Because we saw even a bit better perfomance with the 
virtual one than the performance with the physical one.