Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH V2 3/4] vp_vdpa: allow set vq state to initial state after
 reset
To:     Eli Cohen <elic@nvidia.com>
Cc:     mst@redhat.com, virtualization@lists.linux-foundation.org,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        netdev@vger.kernel.org, eli@mellanox.com
References: <20210602021043.39201-1-jasowang@redhat.com>
 <20210602021043.39201-4-jasowang@redhat.com>
 <20210602061324.GA8662@mtl-vdi-166.wap.labs.mlnx>
From:   Jason Wang <jasowang@redhat.com>
Message-ID: <091dc6d0-8754-7b2a-64ec-985ef9db6329@redhat.com>
Date:   Wed, 2 Jun 2021 15:05:35 +0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.10.2
MIME-Version: 1.0
In-Reply-To: <20210602061324.GA8662@mtl-vdi-166.wap.labs.mlnx>
Content-Type: text/plain; charset=gbk; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Precedence: bulk


?? 2021/6/2 ????2:13, Eli Cohen д??:
> On Wed, Jun 02, 2021 at 10:10:42AM +0800, Jason Wang wrote:
>> We used to fail the set_vq_state() since it was not supported yet by
>> the virtio spec. But if the bus tries to set the state which is equal
>> to the device initial state after reset, we can let it go.
>>
>> This is a must for virtio_vdpa() to set vq state during probe which is
>> required for some vDPA parents.
>>
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> ---
>>   drivers/vdpa/virtio_pci/vp_vdpa.c | 42 ++++++++++++++++++++++++++++---
>>   1 file changed, 39 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/vdpa/virtio_pci/vp_vdpa.c b/drivers/vdpa/virtio_pci/vp_vdpa.c
>> index c76ebb531212..18bf4a422772 100644
>> --- a/drivers/vdpa/virtio_pci/vp_vdpa.c
>> +++ b/drivers/vdpa/virtio_pci/vp_vdpa.c
>> @@ -210,13 +210,49 @@ static int vp_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 qid,
>>   	return -EOPNOTSUPP;
>>   }
>>   
>> +static int vp_vdpa_set_vq_state_split(struct vdpa_device *vdpa,
>> +				      const struct vdpa_vq_state *state)
>> +{
>> +	const struct vdpa_vq_state_split *split = &state->split;
>> +
>> +	if (split->avail_index == 0)
>> +		return 0;
>> +
>> +	return -EOPNOTSUPP;
>> +}
>> +
>> +static int vp_vdpa_set_vq_state_packed(struct vdpa_device *vdpa,
>> +				       const struct vdpa_vq_state *state)
>> +{
>> +	const struct vdpa_vq_state_packed *packed = &state->packed;
>> +
>> +	if (packed->last_avail_counter == 1 &&
> Can you elaborate on the requirement on last_avail_counter and
> last_used_counter?


This is required by the virtio spec:

"
2.7.1 Driver and Device Ring Wrap Counters
Each of the driver and the device are expected to maintain, internally, 
a single-bit ring wrap counter initialized to 1.
"

For virtio-pci device, since there's no way to assign the value of those 
counters, the counters will be reset to 1 after reset, otherwise the 
driver can't work.

Thanks


>
>> +	    packed->last_avail_idx == 0 &&
>> +	    packed->last_used_counter == 1 &&
>> +	    packed->last_used_idx == 0)
>> +		return 0;
>> +
>> +	return -EOPNOTSUPP;
>> +}
>> +
>>   static int vp_vdpa_set_vq_state(struct vdpa_device *vdpa, u16 qid,
>>   				const struct vdpa_vq_state *state)
>>   {
>> -	/* Note that this is not supported by virtio specification, so
>> -	 * we return -ENOPOTSUPP here. This means we can't support live
>> -	 * migration, vhost device start/stop.
>> +	struct virtio_pci_modern_device *mdev = vdpa_to_mdev(vdpa);
>> +
>> +	/* Note that this is not supported by virtio specification.
>> +	 * But if the state is by chance equal to the device initial
>> +	 * state, we can let it go.
>>   	 */
>> +	if ((vp_modern_get_status(mdev) & VIRTIO_CONFIG_S_FEATURES_OK) &&
>> +	    !vp_modern_get_queue_enable(mdev, qid)) {
>> +		if (vp_modern_get_driver_features(mdev) &
>> +		    BIT_ULL(VIRTIO_F_RING_PACKED))
>> +			return vp_vdpa_set_vq_state_packed(vdpa, state);
>> +		else
>> +			return vp_vdpa_set_vq_state_split(vdpa,	state);
>> +	}
>> +
>>   	return -EOPNOTSUPP;
>>   }
>>   
>> -- 
>> 2.25.1
>>