Message-ID: <22c1f3f1-beab-83cb-c7ce-d61a970d0f0e@linux.alibaba.com>
Date:   Thu, 9 Dec 2021 16:26:47 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.2.1
Subject: Re: [RFC PATCH] virtio: make sure legacy pci device gain 32bit-pfn vq
Content-Language: en-US
To:     "Michael S. Tsirkin" <mst@redhat.com>
Cc:     Jason Wang <jasowang@redhat.com>,
        "open list:VIRTIO CORE AND NET DRIVERS" 
        <virtualization@lists.linux-foundation.org>,
        open list <linux-kernel@vger.kernel.org>
References: <b50fff4d-9f05-76b3-eba7-91241c351751@linux.alibaba.com>
 <20211207031217-mutt-send-email-mst@kernel.org>
 <8bbfd029-d969-4632-cb8e-482481d65a2f@linux.alibaba.com>
 <20211208021947-mutt-send-email-mst@kernel.org>
 <dfb712d7-1186-1496-9fcc-a72e23c3409b@linux.alibaba.com>
 <20211208060115-mutt-send-email-mst@kernel.org>
 <a5b62e3c-5c24-dc8f-f7a3-027a67ef3bb2@linux.alibaba.com>
 <20211209013833-mutt-send-email-mst@kernel.org>
From:   =?UTF-8?B?546L6LSH?= <yun.wang@linux.alibaba.com>
In-Reply-To: <20211209013833-mutt-send-email-mst@kernel.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk


在 2021/12/9 下午2:40, Michael S. Tsirkin 写道:
[snip]
>>>> Besides, I've checked that patch but it can't address our issue, we actually
>>>> have this legacy pci device on arm platform, and the memory layout is
>>>> unfriendly since allocation rarely providing page-address below 44bit, we
>>>> understand the virtio-iommu case should not do force dma, while we don't
>>>> have that so it's just working fine.
>>>>
>>>> Regards,
>>>> Michael Wang
>>>
>>> Hmm wait a sec is it a physical device or a hypervisor?
>>> If a physical one then doesn't it need VIRTIO_F_ORDER_PLATFORM
>>> on ARM?
>>
>> The PCI device is virtual, I can't see how VIRTIO_F_ORDER_PLATFORM help
>> address this issue, legacy pci config is 32bit but it's 36, seems like will
>> never be included?
>>
>> Regards,
>> Michael Wang
> 
> Oh, if the device is virtual then I think you should just update it please.
> virtio 0.X is architecturally limited to small VMs,
> if your hypervisor supports more it should emulate virtio 1.0.

I see, nice to confirm the proper approach, although we don't have that 
option on the desk :-P

So as long as we don't have any iommu enabled, the force dma approach 
could be safe, is this correct?

Regards,
Michael Wang

> 
> 
> 
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>>> ---
>>>>>>>>      drivers/virtio/virtio_pci_legacy.c | 10 ++++++++++
>>>>>>>>      drivers/virtio/virtio_ring.c       |  3 +++
>>>>>>>>      include/linux/virtio.h             |  1 +
>>>>>>>>      3 files changed, 14 insertions(+)
>>>>>>>>
>>>>>>>> diff --git a/drivers/virtio/virtio_pci_legacy.c
>>>>>>>> b/drivers/virtio/virtio_pci_legacy.c
>>>>>>>> index d62e983..11f2ebf 100644
>>>>>>>> --- a/drivers/virtio/virtio_pci_legacy.c
>>>>>>>> +++ b/drivers/virtio/virtio_pci_legacy.c
>>>>>>>> @@ -263,6 +263,16 @@ int virtio_pci_legacy_probe(struct virtio_pci_device
>>>>>>>> *vp_dev)
>>>>>>>>      	vp_dev->setup_vq = setup_vq;
>>>>>>>>      	vp_dev->del_vq = del_vq;
>>>>>>>>
>>>>>>>> +	/*
>>>>>>>> +	 * The legacy pci device requre 32bit-pfn vq,
>>>>>>>> +	 * or setup_vq() will failed.
>>>>>>>> +	 *
>>>>>>>> +	 * Thus we make sure vring_use_dma_api() will
>>>>>>>> +	 * return true during the allocation by marking
>>>>>>>> +	 * force_dma here.
>>>>>>>> +	 */
>>>>>>>> +	vp_dev->vdev.force_dma = true;
>>>>>>>> +
>>>>>>>>      	return 0;
>>>>>>>>
>>>>>>>>      err_iomap:
>>>>>>>> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
>>>>>>>> index 3035bb6..6562e01 100644
>>>>>>>> --- a/drivers/virtio/virtio_ring.c
>>>>>>>> +++ b/drivers/virtio/virtio_ring.c
>>>>>>>> @@ -245,6 +245,9 @@ static inline bool virtqueue_use_indirect(struct
>>>>>>>> virtqueue *_vq,
>>>>>>>>
>>>>>>>>      static bool vring_use_dma_api(struct virtio_device *vdev)
>>>>>>>>      {
>>>>>>>> +	if (vdev->force_dma)
>>>>>>>> +		return true;
>>>>>>>> +
>>>>>>>>      	if (!virtio_has_dma_quirk(vdev))
>>>>>>>>      		return true;
>>>>>>>>
>>>>>>>> diff --git a/include/linux/virtio.h b/include/linux/virtio.h
>>>>>>>> index 41edbc0..a4eb29d 100644
>>>>>>>> --- a/include/linux/virtio.h
>>>>>>>> +++ b/include/linux/virtio.h
>>>>>>>> @@ -109,6 +109,7 @@ struct virtio_device {
>>>>>>>>      	bool failed;
>>>>>>>>      	bool config_enabled;
>>>>>>>>      	bool config_change_pending;
>>>>>>>> +	bool force_dma;
>>>>>>>>      	spinlock_t config_lock;
>>>>>>>>      	spinlock_t vqs_list_lock; /* Protects VQs list access */
>>>>>>>>      	struct device dev;
>>>>>>>> -- 
>>>>>>>> 1.8.3.1