Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <50eb0df0-89fe-a5df-f89f-07bf69bd00ae@redhat.com>
Date:   Wed, 4 Jan 2023 12:23:07 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.6.1
Subject: Re: [PATCH net V2] virtio-net: correctly enable callback during
 start_xmit
From:   Jason Wang <jasowang@redhat.com>
To:     "Michael S. Tsirkin" <mst@redhat.com>
Cc:     davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
        pabeni@redhat.com, virtualization@lists.linux-foundation.org,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        xuanzhuo@linux.alibaba.com
References: <20221215032719.72294-1-jasowang@redhat.com>
 <20221215034740-mutt-send-email-mst@kernel.org>
 <CACGkMEsLeCRDqyuyGzWw+kjYrTVDjUjOw6+xHESPT2D1p03=sQ@mail.gmail.com>
 <20221215042918-mutt-send-email-mst@kernel.org>
 <CACGkMEsbvTQrEp5dmQRHp58Mu=E7f433Xrvsbs4nZMA5R3B6mQ@mail.gmail.com>
 <CACGkMEsu_OFFs15d2dzNbfSjzAZfYXLn9CNcO3ELPbDqZsndzg@mail.gmail.com>
Content-Language: en-US
In-Reply-To: <CACGkMEsu_OFFs15d2dzNbfSjzAZfYXLn9CNcO3ELPbDqZsndzg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk


在 2022/12/23 14:29, Jason Wang 写道:
> On Fri, Dec 16, 2022 at 11:43 AM Jason Wang <jasowang@redhat.com> wrote:
>> On Thu, Dec 15, 2022 at 5:35 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>>> On Thu, Dec 15, 2022 at 05:15:43PM +0800, Jason Wang wrote:
>>>> On Thu, Dec 15, 2022 at 5:02 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>>>>> On Thu, Dec 15, 2022 at 11:27:19AM +0800, Jason Wang wrote:
>>>>>> Commit a7766ef18b33("virtio_net: disable cb aggressively") enables
>>>>>> virtqueue callback via the following statement:
>>>>>>
>>>>>>          do {
>>>>>>             ......
>>>>>>        } while (use_napi && kick &&
>>>>>>                 unlikely(!virtqueue_enable_cb_delayed(sq->vq)));
>>>>>>
>>>>>> When NAPI is used and kick is false, the callback won't be enabled
>>>>>> here. And when the virtqueue is about to be full, the tx will be
>>>>>> disabled, but we still don't enable tx interrupt which will cause a TX
>>>>>> hang. This could be observed when using pktgen with burst enabled.
>>>>>>
>>>>>> Fixing this by trying to enable tx interrupt after we disable TX when
>>>>>> we're not using napi or kick is false.
>>>>>>
>>>>>> Fixes: a7766ef18b33 ("virtio_net: disable cb aggressively")
>>>>>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>>>>>> ---
>>>>>> The patch is needed for -stable.
>>>>>> Changes since V1:
>>>>>> - enable tx interrupt after we disable tx
>>>>>> ---
>>>>>>   drivers/net/virtio_net.c | 2 +-
>>>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
>>>>>> index 86e52454b5b5..dcf3a536d78a 100644
>>>>>> --- a/drivers/net/virtio_net.c
>>>>>> +++ b/drivers/net/virtio_net.c
>>>>>> @@ -1873,7 +1873,7 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
>>>>>>         */
>>>>>>        if (sq->vq->num_free < 2+MAX_SKB_FRAGS) {
>>>>>>                netif_stop_subqueue(dev, qnum);
>>>>>> -             if (!use_napi &&
>>>>>> +             if ((!use_napi || !kick) &&
>>>>>>                    unlikely(!virtqueue_enable_cb_delayed(sq->vq))) {
>>>>>>                        /* More just got used, free them then recheck. */
>>>>>>                        free_old_xmit_skbs(sq, false);
>>>>> This will work but the following lines are:
>>>>>
>>>>>                         if (sq->vq->num_free >= 2+MAX_SKB_FRAGS) {
>>>>>                                  netif_start_subqueue(dev, qnum);
>>>>>                                  virtqueue_disable_cb(sq->vq);
>>>>>                          }
>>>>>
>>>>>
>>>>> and I thought we are supposed to keep callbacks enabled with napi?
>>>> This seems to be the opposite logic of commit a7766ef18b33 that
>>>> disables callbacks for NAPI.
>>>>
>>>> It said:
>>>>
>>>>      There are currently two cases where we poll TX vq not in response to a
>>>>      callback: start xmit and rx napi.  We currently do this with callbacks
>>>>      enabled which can cause extra interrupts from the card.  Used not to be
>>>>      a big issue as we run with interrupts disabled but that is no longer the
>>>>      case, and in some cases the rate of spurious interrupts is so high
>>>>      linux detects this and actually kills the interrupt.
>>>>
>>>> My undersatnding is that it tries to disable callbacks on TX.
>>> I think we want to disable callbacks while polling, yes. here we are not
>>> polling, and I think we want a callback because otherwise nothing will
>>> orphan skbs and a socket can be blocked, not transmitting anything - a
>>> deadlock.
>> I'm not sure how I got here, did you mean a partial revert of
>> a7766ef18b33 (the part that disables TX callbacks on start_xmit)?
> Michael, any idea on this?
>
> Thanks


Michael, any comment?

Thanks