If the allocation of multiple MSI vectors for multi-MSI fails in the core
PCI framework, the framework will retry the allocation as a single MSI
vector, assuming that meets the min_vecs specified by the requesting
driver.
Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR
domain to implement that for x86. The VECTOR domain does not support
multi-MSI, so the alloc will always fail and fallback to a single MSI
allocation.
In short, Hyper-V advertises a capability it does not implement.
Hyper-V can support multi-MSI because it coordinates with the hypervisor
to map the MSIs in the IOMMU's interrupt remapper, which is something the
VECTOR domain does not have. Therefore the fix is simple - copy what the
x86 IOMMU drivers (AMD/Intel-IR) do by removing
X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's
pci_msi_prepare().
Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
Signed-off-by: Jeffrey Hugo <[email protected]>
Reviewed-by: Dexuan Cui <[email protected]>
---
v2:
-Fix grammatical mistake in added comment
drivers/pci/controller/pci-hyperv.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index d270a204..1cbe24b 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -614,7 +614,16 @@ static void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry,
static int hv_msi_prepare(struct irq_domain *domain, struct device *dev,
int nvec, msi_alloc_info_t *info)
{
- return pci_msi_prepare(domain, dev, nvec, info);
+ int ret = pci_msi_prepare(domain, dev, nvec, info);
+
+ /*
+ * By using the interrupt remapper in the hypervisor IOMMU, contiguous
+ * CPU vectors is not needed for multi-MSI
+ */
+ if (info->type == X86_IRQ_ALLOC_TYPE_PCI_MSI)
+ info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS;
+
+ return ret;
}
/**
--
2.7.4
On 4/13/2022 7:36 AM, Jeffrey Hugo wrote:
> If the allocation of multiple MSI vectors for multi-MSI fails in the core
> PCI framework, the framework will retry the allocation as a single MSI
> vector, assuming that meets the min_vecs specified by the requesting
> driver.
>
> Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR
> domain to implement that for x86. The VECTOR domain does not support
> multi-MSI, so the alloc will always fail and fallback to a single MSI
> allocation.
>
> In short, Hyper-V advertises a capability it does not implement.
>
> Hyper-V can support multi-MSI because it coordinates with the hypervisor
> to map the MSIs in the IOMMU's interrupt remapper, which is something the
> VECTOR domain does not have. Therefore the fix is simple - copy what the
> x86 IOMMU drivers (AMD/Intel-IR) do by removing
> X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's
> pci_msi_prepare().
>
> Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
> Signed-off-by: Jeffrey Hugo <[email protected]>
> Reviewed-by: Dexuan Cui <[email protected]>
> ---
Ping?
I don't see this in -next, nor have I seen any replies. It is possible
I have missed some kind of update, but currently I'm wondering if this
change is progressing or not. If there is some kind of process used in
this area, I'm not familiar with it, so I would appreciate an introduction.
Thanks
-Jeff
On Wed, Apr 20, 2022 at 08:13:22AM -0600, Jeffrey Hugo wrote:
> On 4/13/2022 7:36 AM, Jeffrey Hugo wrote:
> > If the allocation of multiple MSI vectors for multi-MSI fails in the core
> > PCI framework, the framework will retry the allocation as a single MSI
> > vector, assuming that meets the min_vecs specified by the requesting
> > driver.
> >
> > Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR
> > domain to implement that for x86. The VECTOR domain does not support
> > multi-MSI, so the alloc will always fail and fallback to a single MSI
> > allocation.
> >
> > In short, Hyper-V advertises a capability it does not implement.
> >
> > Hyper-V can support multi-MSI because it coordinates with the hypervisor
> > to map the MSIs in the IOMMU's interrupt remapper, which is something the
> > VECTOR domain does not have. Therefore the fix is simple - copy what the
> > x86 IOMMU drivers (AMD/Intel-IR) do by removing
> > X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's
> > pci_msi_prepare().
> >
> > Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
> > Signed-off-by: Jeffrey Hugo <[email protected]>
> > Reviewed-by: Dexuan Cui <[email protected]>
> > ---
>
> Ping?
>
> I don't see this in -next, nor have I seen any replies. It is possible I
> have missed some kind of update, but currently I'm wondering if this change
> is progressing or not. If there is some kind of process used in this area,
> I'm not familiar with it, so I would appreciate an introduction.
I expect the PCI maintainers to pick this up. If I don't see this picked
up in this week I will apply it to hyperv-next.
Thanks,
Wei.
>
> Thanks
>
> -Jeff
On Mon, Apr 25, 2022 at 03:33:44PM +0000, Wei Liu wrote:
> On Wed, Apr 20, 2022 at 08:13:22AM -0600, Jeffrey Hugo wrote:
> > On 4/13/2022 7:36 AM, Jeffrey Hugo wrote:
> > > If the allocation of multiple MSI vectors for multi-MSI fails in the core
> > > PCI framework, the framework will retry the allocation as a single MSI
> > > vector, assuming that meets the min_vecs specified by the requesting
> > > driver.
> > >
> > > Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR
> > > domain to implement that for x86. The VECTOR domain does not support
> > > multi-MSI, so the alloc will always fail and fallback to a single MSI
> > > allocation.
> > >
> > > In short, Hyper-V advertises a capability it does not implement.
> > >
> > > Hyper-V can support multi-MSI because it coordinates with the hypervisor
> > > to map the MSIs in the IOMMU's interrupt remapper, which is something the
> > > VECTOR domain does not have. Therefore the fix is simple - copy what the
> > > x86 IOMMU drivers (AMD/Intel-IR) do by removing
> > > X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's
> > > pci_msi_prepare().
> > >
> > > Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
> > > Signed-off-by: Jeffrey Hugo <[email protected]>
> > > Reviewed-by: Dexuan Cui <[email protected]>
> > > ---
> >
> > Ping?
> >
> > I don't see this in -next, nor have I seen any replies. It is possible I
> > have missed some kind of update, but currently I'm wondering if this change
> > is progressing or not. If there is some kind of process used in this area,
> > I'm not familiar with it, so I would appreciate an introduction.
>
> I expect the PCI maintainers to pick this up. If I don't see this picked
> up in this week I will apply it to hyperv-next.
Actually I will pick this up via hyperv-next, because there is another
series which will also touch this driver but at the some time depend on
vmbus changes. I can fix up any potential conflicts easily.
Thanks,
Wei.
On 4/25/2022 9:49 AM, Wei Liu wrote:
> On Mon, Apr 25, 2022 at 03:33:44PM +0000, Wei Liu wrote:
>> On Wed, Apr 20, 2022 at 08:13:22AM -0600, Jeffrey Hugo wrote:
>>> On 4/13/2022 7:36 AM, Jeffrey Hugo wrote:
>>>> If the allocation of multiple MSI vectors for multi-MSI fails in the core
>>>> PCI framework, the framework will retry the allocation as a single MSI
>>>> vector, assuming that meets the min_vecs specified by the requesting
>>>> driver.
>>>>
>>>> Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR
>>>> domain to implement that for x86. The VECTOR domain does not support
>>>> multi-MSI, so the alloc will always fail and fallback to a single MSI
>>>> allocation.
>>>>
>>>> In short, Hyper-V advertises a capability it does not implement.
>>>>
>>>> Hyper-V can support multi-MSI because it coordinates with the hypervisor
>>>> to map the MSIs in the IOMMU's interrupt remapper, which is something the
>>>> VECTOR domain does not have. Therefore the fix is simple - copy what the
>>>> x86 IOMMU drivers (AMD/Intel-IR) do by removing
>>>> X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's
>>>> pci_msi_prepare().
>>>>
>>>> Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
>>>> Signed-off-by: Jeffrey Hugo <[email protected]>
>>>> Reviewed-by: Dexuan Cui <[email protected]>
>>>> ---
>>>
>>> Ping?
>>>
>>> I don't see this in -next, nor have I seen any replies. It is possible I
>>> have missed some kind of update, but currently I'm wondering if this change
>>> is progressing or not. If there is some kind of process used in this area,
>>> I'm not familiar with it, so I would appreciate an introduction.
>>
>> I expect the PCI maintainers to pick this up. If I don't see this picked
>> up in this week I will apply it to hyperv-next.
>
> Actually I will pick this up via hyperv-next, because there is another
> series which will also touch this driver but at the some time depend on
> vmbus changes. I can fix up any potential conflicts easily.
Sounds good to me. Let me know if you do run into conflicts, and I can
help.
-Jeff