With virtio multiqueue, normally each queue IRQ is mapped to a CPU.
But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ affinity")
this is broken on pseries.
The affinity is correctly computed in msi_desc but this is not applied
to the system IRQs.
It appears the affinity is correctly passed to rtas_setup_msi_irqs() but
lost at this point and never passed to irq_domain_alloc_descs()
(see commit 06ee6d571f0e ("genirq: Add affinity hint to irq allocation"))
because irq_create_mapping() doesn't take an affinity parameter.
As the previous patch has added the affinity parameter to
irq_create_mapping() we can forward the affinity from rtas_setup_msi_irqs()
to irq_domain_alloc_descs().
With this change, the virtqueues are correctly dispatched between the CPUs
on pseries.
This problem cannot be shown on x86_64 for two reasons:
- the call path traverses arch_setup_msi_irqs() that is arch specific:
virtscsi_probe()
virtscsi_init()
vp_modern_find_vqs()
vp_find_vqs()
vp_find_vqs_msix()
pci_alloc_irq_vectors_affinity()
__pci_enable_msix_range()
pci_msi_setup_msi_irqs()
arch_setup_msi_irqs()
rtas_setup_msi_irqs()
irq_create_mapping()
irq_domain_alloc_descs()
__irq_alloc_descs()
- and x86_64 has CONFIG_PCI_MSI_IRQ_DOMAIN that uses another path:
virtscsi_probe()
virtscsi_init()
vp_modern_find_vqs()
vp_find_vqs()
vp_find_vqs_msix()
pci_alloc_irq_vectors_affinity()
__pci_enable_msix_range()
__msi_domain_alloc_irqs()
__irq_domain_alloc_irqs()
__irq_alloc_descs()
Signed-off-by: Laurent Vivier <[email protected]>
---
arch/powerpc/platforms/pseries/msi.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c
index 42ba08eaea91..58197f92c6a2 100644
--- a/arch/powerpc/platforms/pseries/msi.c
+++ b/arch/powerpc/platforms/pseries/msi.c
@@ -458,7 +458,7 @@ static int rtas_setup_msi_irqs(struct pci_dev *pdev, int nvec_in, int type)
return hwirq;
}
- virq = irq_create_mapping(NULL, hwirq, NULL);
+ virq = irq_create_mapping(NULL, hwirq, entry->affinity);
if (!virq) {
pr_debug("rtas_msi: Failed mapping hwirq %d\n", hwirq);
--
2.28.0
On Tue, Nov 24 2020 at 21:03, Laurent Vivier wrote:
> With virtio multiqueue, normally each queue IRQ is mapped to a CPU.
>
> This problem cannot be shown on x86_64 for two reasons:
There is only _ONE_ reason why this is not a problem on x86. x86 uses
the generic PCI/MSI domain which supports this out of the box.
> - the call path traverses arch_setup_msi_irqs() that is arch specific:
>
> virtscsi_probe()
> virtscsi_init()
> vp_modern_find_vqs()
> vp_find_vqs()
> vp_find_vqs_msix()
> pci_alloc_irq_vectors_affinity()
> __pci_enable_msix_range()
> pci_msi_setup_msi_irqs()
> arch_setup_msi_irqs()
> rtas_setup_msi_irqs()
This is a problem on _all_ variants of PPC MSI providers, not only for
pseries. It's not restricted to virtscsi devices either, that's just the
device which made you discover this.
Thanks,
tglx