Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2;
Date:   Tue, 5 Dec 2023 15:21:55 +0100
From:   Stefano Garzarella <sgarzare@redhat.com>
To:     Arseniy Krasnov <avkrasnov@salutedevices.com>
Cc:     Stefan Hajnoczi <stefanha@redhat.com>,
        "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        "Michael S. Tsirkin" <mst@redhat.com>,
        Jason Wang <jasowang@redhat.com>,
        Bobby Eshleman <bobby.eshleman@bytedance.com>,
        kvm@vger.kernel.org, virtualization@lists.linux-foundation.org,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        kernel@sberdevices.ru, oxffffaa@gmail.com
Subject: Re: [PATCH net-next v6 3/4] virtio/vsock: fix logic which reduces
 credit update messages
Message-ID: <gqrfreguavurkb7betm2utzdfnefrxgxyoilyveowvmspbwpes@45s6jshyelui>
References: <20231205064806.2851305-1-avkrasnov@salutedevices.com>
 <20231205064806.2851305-4-avkrasnov@salutedevices.com>
 <v335g4fjrn5f6tsw4nysztaklze2obnjwpezps3jgb2xickpge@ea5woxob52nc>
 <809a8962-0082-6443-4e59-549eb28b9a82@salutedevices.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <809a8962-0082-6443-4e59-549eb28b9a82@salutedevices.com>
Precedence: bulk

On Tue, Dec 05, 2023 at 03:07:47PM +0300, Arseniy Krasnov wrote:
>
>
>On 05.12.2023 13:54, Stefano Garzarella wrote:
>> On Tue, Dec 05, 2023 at 09:48:05AM +0300, Arseniy Krasnov wrote:
>>> Add one more condition for sending credit update during dequeue from
>>> stream socket: when number of bytes in the rx queue is smaller than
>>> SO_RCVLOWAT value of the socket. This is actual for non-default value
>>> of SO_RCVLOWAT (e.g. not 1) - idea is to "kick" peer to continue data
>>> transmission, because we need at least SO_RCVLOWAT bytes in our rx
>>> queue to wake up user for reading data (in corner case it is also
>>> possible to stuck both tx and rx sides, this is why 'Fixes' is used).
>>>
>>> Fixes: b89d882dc9fc ("vsock/virtio: reduce credit update messages")
>>> Signed-off-by: Arseniy Krasnov <avkrasnov@salutedevices.com>
>>> ---
>>> net/vmw_vsock/virtio_transport_common.c | 9 +++++++--
>>> 1 file changed, 7 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
>>> index e137d740804e..461c89882142 100644
>>> --- a/net/vmw_vsock/virtio_transport_common.c
>>> +++ b/net/vmw_vsock/virtio_transport_common.c
>>> @@ -558,6 +558,7 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk,
>>> ????struct virtio_vsock_sock *vvs = vsk->trans;
>>> ????size_t bytes, total = 0;
>>> ????struct sk_buff *skb;
>>> +??? bool low_rx_bytes;
>>> ????int err = -EFAULT;
>>> ????u32 free_space;
>>>
>>> @@ -602,6 +603,8 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk,
>>> ????}
>>>
>>> ????free_space = vvs->buf_alloc - (vvs->fwd_cnt - vvs->last_fwd_cnt);
>>> +??? low_rx_bytes = (vvs->rx_bytes <
>>> +??????????? sock_rcvlowat(sk_vsock(vsk), 0, INT_MAX));
>>
>> As in the previous patch, should we avoid the update it if `fwd_cnt` and `last_fwd_cnt` are the same?
>>
>> Now I'm thinking if it is better to add that check directly in virtio_transport_send_credit_update().
>
>Good point, but I think, that it is better to keep this check here, because access to 'fwd_cnt' and 'last_fwd_cnt'
>requires taking rx_lock - so I guess it is better to avoid taking this lock every time in 'virtio_transport_send_credit_update()'.

Yeah, I agree.

>So may be we can do something like:
>
>
>fwd_cnt_delta = vvs->fwd_cnt - vvs->last_fwd_cnt;
>free_space = vvs->buf_alloc - fwd_cnt_delta;

Pre-existing issue, but should we handle the wrap (e.g. fwd_cnt wrapped, 
but last_fwd_cnt not yet?). Maybe in that case we can foce the status
update.

>
>and then, after lock is released:
>
>if (fwd_cnt_delta && (free_space < VIRTIO_VSOCK_MAX_PKT_BUF_SIZE ||
>    low_rx_bytes))
>        virtio_transport_send_credit_update(vsk);
>
>WDYT?

Yep, I agree.

>
>Also, I guess that next idea to update this optimization(in next patchset), is to make
>threshold depends on vvs->buf_alloc. Because if someone changes minimum buffer size to
>for example 32KB, and then sets buffer size to 32KB, then free_space will be always
>non-zero, thus optimization is off now and credit update is sent on 
>every read.

But does it make sense to allow a buffer smaller than 
VIRTIO_VSOCK_MAX_PKT_BUF_SIZE?

Maybe we should fail in virtio_transport_notify_buffer_size() or use it 
as minimum.

Stefano