Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH 1/2] vDPA/ifcvf: record virtio notify base
To:     "Zhu, Lingshan" <lingshan.zhu@intel.com>, mst@redhat.com
Cc:     virtualization@lists.linux-foundation.org, netdev@vger.kernel.org,
        kvm@vger.kernel.org, linux-kernel@vger.kernel.org
References: <20210428082133.6766-1-lingshan.zhu@intel.com>
 <20210428082133.6766-2-lingshan.zhu@intel.com>
 <55217869-b456-f3bc-0b5a-6beaf34c19f8@redhat.com>
 <3243eeef-2891-5b79-29cb-bc969802c5dc@intel.com>
From:   Jason Wang <jasowang@redhat.com>
Message-ID: <4cee04f1-a3fc-eaf0-747a-004ca09b06c0@redhat.com>
Date:   Wed, 28 Apr 2021 18:09:11 +0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.10.0
MIME-Version: 1.0
In-Reply-To: <3243eeef-2891-5b79-29cb-bc969802c5dc@intel.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Precedence: bulk


在 2021/4/28 下午6:00, Zhu, Lingshan 写道:
>
>
> On 4/28/2021 4:39 PM, Jason Wang wrote:
>>
>> 在 2021/4/28 下午4:21, Zhu Lingshan 写道:
>>> This commit records virtio notify base addr to implemente
>>> doorbell mapping feature
>>>
>>> Signed-off-by: Zhu Lingshan <lingshan.zhu@intel.com>
>>> ---
>>>   drivers/vdpa/ifcvf/ifcvf_base.c | 1 +
>>>   drivers/vdpa/ifcvf/ifcvf_base.h | 1 +
>>>   2 files changed, 2 insertions(+)
>>>
>>> diff --git a/drivers/vdpa/ifcvf/ifcvf_base.c 
>>> b/drivers/vdpa/ifcvf/ifcvf_base.c
>>> index 1a661ab45af5..cc61a5bfc5b1 100644
>>> --- a/drivers/vdpa/ifcvf/ifcvf_base.c
>>> +++ b/drivers/vdpa/ifcvf/ifcvf_base.c
>>> @@ -133,6 +133,7 @@ int ifcvf_init_hw(struct ifcvf_hw *hw, struct 
>>> pci_dev *pdev)
>>> &hw->notify_off_multiplier);
>>>               hw->notify_bar = cap.bar;
>>>               hw->notify_base = get_cap_addr(hw, &cap);
>>> +            hw->notify_pa = pci_resource_start(pdev, cap.bar) + 
>>> cap.offset;
>>
>>
>> To be more generic and avoid future changes, let's use the math 
>> defined in the virtio spec.
>>
>> You may refer how it is implemented in virtio_pci vdpa driver[1].
> Are you suggesting every vq keep its own notify_pa? In this case, we 
> still need to record notify_pa in hw when init_hw, then initialize 
> vq->notify_pa accrediting to hw->notify_pa.


I meant you need to follow how virtio spec did to calculate the doorbell 
address per vq:

         cap.offset + queue_notify_off * notify_off_multiplier

Obviously, you ignore queue_notify_off and notify_off_multiplier here. 
This may bring troubles for the existing device IFCVF and future devices.

If I understand correctly, this device can be probed by virtio-pci 
driver which use the above math. There's no reason for using ad-hoc hack.

Thanks


>
> Thanks
> Zhu Lingshan
>>
>> Thanks
>>
>> [1] 
>> https://lore.kernel.org/virtualization/20210415073147.19331-5-jasowang@redhat.com/T/
>>
>>
>>> IFCVF_DBG(pdev, "hw->notify_base = %p\n",
>>>                     hw->notify_base);
>>>               break;
>>> diff --git a/drivers/vdpa/ifcvf/ifcvf_base.h 
>>> b/drivers/vdpa/ifcvf/ifcvf_base.h
>>> index 0111bfdeb342..bcca7c1669dd 100644
>>> --- a/drivers/vdpa/ifcvf/ifcvf_base.h
>>> +++ b/drivers/vdpa/ifcvf/ifcvf_base.h
>>> @@ -98,6 +98,7 @@ struct ifcvf_hw {
>>>       char config_msix_name[256];
>>>       struct vdpa_callback config_cb;
>>>       unsigned int config_irq;
>>> +    phys_addr_t  notify_pa;
>>>   };
>>>     struct ifcvf_adapter {
>>
>