Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH] vdpa/mlx5: Clear vq ready indication upon device reset
To:     Eli Cohen <elic@nvidia.com>
Cc:     mst@redhat.com, virtualization@lists.linux-foundation.org,
        linux-kernel@vger.kernel.org
References: <20210602085924.62777-1-elic@nvidia.com>
 <6e4f9e1b-2c67-fae6-6edd-1982d0f48e22@redhat.com>
 <782562f2-6903-68cb-d753-ac90aea854e4@redhat.com>
 <20210603073046.GA58414@mtl-vdi-166.wap.labs.mlnx>
From:   Jason Wang <jasowang@redhat.com>
Message-ID: <af2561a4-7b46-3c0c-2956-f5dd37577b8d@redhat.com>
Date:   Thu, 3 Jun 2021 15:37:23 +0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.10.2
MIME-Version: 1.0
In-Reply-To: <20210603073046.GA58414@mtl-vdi-166.wap.labs.mlnx>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Precedence: bulk


在 2021/6/3 下午3:30, Eli Cohen 写道:
> On Thu, Jun 03, 2021 at 03:06:31PM +0800, Jason Wang wrote:
>> 在 2021/6/3 下午3:00, Jason Wang 写道:
>>> 在 2021/6/2 下午4:59, Eli Cohen 写道:
>>>> After device reset, the virtqueues are not ready so clear the ready
>>>> field.
>>>>
>>>> Failing to do so can result in virtio_vdpa failing to load if the device
>>>> was previously used by vhost_vdpa and the old values are ready.
>>>> virtio_vdpa expects to find VQs in "not ready" state.
>>>>
>>>> Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5
>>>> devices")
>>>> Signed-off-by: Eli Cohen <elic@nvidia.com>
>>>
>>> Acked-by: Jason Wang <jasowang@redhat.com>
>>
>> A second thought.
>>
>> destroy_virtqueue() could be called many places.
>>
>> One of them is the mlx5_vdpa_change_map(), if this is case, this looks
>> wrong.
> Right, although most likely VQs become ready only after all map changes
> occur becuase I did not encounter any issue while testing.


Yes, but it's not guaranteed that the map won't be changed. Userspace 
can update the mapping when new memory is plugged into the guest for 
example.


>> It looks to me it's simpler to do this in clear_virtqueues() which can only
>> be called during reset.
> There is no clear_virtqueues() function. You probably mean to insert a
> call in mlx5_vdpa_set_status() in case it performs reset. This function
> will go over all virtqueues and clear their ready flag.


Right.


>
> Alternatively we can add boolean argument to teardown_driver() that
> signifies if we are in reset flow and in this case we clear ready.


Yes, but doing in set_status() seems easier.

Thanks


>
>> Thanks
>>
>>
>>>
>>>> ---
>>>>    drivers/vdpa/mlx5/net/mlx5_vnet.c | 1 +
>>>>    1 file changed, 1 insertion(+)
>>>>
>>>> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>>> b/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>>> index 02a05492204c..e8bc0842b44c 100644
>>>> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>>> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>>> @@ -862,6 +862,7 @@ static void destroy_virtqueue(struct
>>>> mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtq
>>>>            return;
>>>>        }
>>>>        umems_destroy(ndev, mvq);
>>>> +    mvq->ready = false;
>>>>    }
>>>>      static u32 get_rqpn(struct mlx5_vdpa_virtqueue *mvq, bool fw)