If the allocation of multiple MSI vectors for multi-MSI fails in the core
PCI framework, the framework will retry the allocation as a single MSI
vector, assuming that meets the min_vecs specified by the requesting
driver.
Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR
domain to implement that for x86. The VECTOR domain does not support
multi-MSI, so the alloc will always fail and fallback to a single MSI
allocation.
In short, Hyper-V advertises a capability it does not implement.
Hyper-V can support multi-MSI because it coordinates with the hypervisor
to map the MSIs in the IOMMU's interrupt remapper, which is something the
VECTOR domain does not have. Therefore the fix is simple - copy what the
x86 IOMMU drivers (AMD/Intel-IR) do by removing
X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's
pci_msi_prepare().
Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
Signed-off-by: Jeffrey Hugo <[email protected]>
---
drivers/pci/controller/pci-hyperv.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index d270a204..41be63e 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -614,7 +614,16 @@ static void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry,
static int hv_msi_prepare(struct irq_domain *domain, struct device *dev,
int nvec, msi_alloc_info_t *info)
{
- return pci_msi_prepare(domain, dev, nvec, info);
+ int ret = pci_msi_prepare(domain, dev, nvec, info);
+
+ /*
+ * By using the interrupt remapper in the hypervisor IOMMU, contiguous
+ * CPU vectors in not needed for multi-MSI
+ */
+ if (info->type == X86_IRQ_ALLOC_TYPE_PCI_MSI)
+ info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS;
+
+ return ret;
}
/**
--
2.7.4
> From: Jeffrey Hugo <[email protected]>
> Sent: Tuesday, April 12, 2022 7:17 AM
> ...
> If the allocation of multiple MSI vectors for multi-MSI fails in the core
> PCI framework, the framework will retry the allocation as a single MSI
> vector, assuming that meets the min_vecs specified by the requesting
> driver.
>
> Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR
> domain to implement that for x86. The VECTOR domain does not support
> multi-MSI, so the alloc will always fail and fallback to a single MSI
> allocation.
>
> In short, Hyper-V advertises a capability it does not implement.
>
> Hyper-V can support multi-MSI because it coordinates with the hypervisor
> to map the MSIs in the IOMMU's interrupt remapper, which is something the
> VECTOR domain does not have. Therefore the fix is simple - copy what the
> x86 IOMMU drivers (AMD/Intel-IR) do by removing
> X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's
> pci_msi_prepare().
>
> Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft
> Hyper-V VMs")
> Signed-off-by: Jeffrey Hugo <[email protected]>
Thanks for the fix! This looks good to me.
Reviewed-by: Dexuan Cui <[email protected]>
On 4/12/2022 8:16 AM, Jeffrey Hugo wrote:
> If the allocation of multiple MSI vectors for multi-MSI fails in the core
> PCI framework, the framework will retry the allocation as a single MSI
> vector, assuming that meets the min_vecs specified by the requesting
> driver.
>
> Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR
> domain to implement that for x86. The VECTOR domain does not support
> multi-MSI, so the alloc will always fail and fallback to a single MSI
> allocation.
>
> In short, Hyper-V advertises a capability it does not implement.
>
> Hyper-V can support multi-MSI because it coordinates with the hypervisor
> to map the MSIs in the IOMMU's interrupt remapper, which is something the
> VECTOR domain does not have. Therefore the fix is simple - copy what the
> x86 IOMMU drivers (AMD/Intel-IR) do by removing
> X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's
> pci_msi_prepare().
>
> Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
> Signed-off-by: Jeffrey Hugo <[email protected]>
> ---
> drivers/pci/controller/pci-hyperv.c | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index d270a204..41be63e 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -614,7 +614,16 @@ static void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry,
> static int hv_msi_prepare(struct irq_domain *domain, struct device *dev,
> int nvec, msi_alloc_info_t *info)
> {
> - return pci_msi_prepare(domain, dev, nvec, info);
> + int ret = pci_msi_prepare(domain, dev, nvec, info);
> +
> + /*
> + * By using the interrupt remapper in the hypervisor IOMMU, contiguous
> + * CPU vectors in not needed for multi-MSI
I just noticed that "in" should be "is".
> + */
> + if (info->type == X86_IRQ_ALLOC_TYPE_PCI_MSI)
> + info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS;
> +
> + return ret;
> }
>
> /**