2022-11-11 15:19:38

by Thomas Gleixner

[permalink] [raw]
Subject: [patch 35/39] PCI/MSI: Reject MSI-X early

Similar to PCI multi-MSI reject MSI-X enablement when a irq domain is
attached to the device which does not support MSI-X.

Signed-off-by: Thomas Gleixner <[email protected]>
---
drivers/pci/msi/msi.c | 4 ++++
1 file changed, 4 insertions(+)

--- a/drivers/pci/msi/msi.c
+++ b/drivers/pci/msi/msi.c
@@ -760,6 +760,10 @@ int __pci_enable_msix_range(struct pci_d
if (WARN_ON_ONCE(dev->msix_enabled))
return -EINVAL;

+ /* Check MSI-X early on irq domain enabled architectures */
+ if (!pci_msi_domain_supports(dev, MSI_FLAG_PCI_MSIX, ALLOW_LEGACY))
+ return -ENOTSUPP;
+
if (!pci_msi_supported(dev, nvec) || dev->current_state != PCI_D0)
return -EINVAL;




2022-11-16 16:40:31

by Bjorn Helgaas

[permalink] [raw]
Subject: Re: [patch 35/39] PCI/MSI: Reject MSI-X early

On Fri, Nov 11, 2022 at 02:55:11PM +0100, Thomas Gleixner wrote:
> Similar to PCI multi-MSI reject MSI-X enablement when a irq domain is
> attached to the device which does not support MSI-X.
>
> Signed-off-by: Thomas Gleixner <[email protected]>

Acked-by: Bjorn Helgaas <[email protected]>

> ---
> drivers/pci/msi/msi.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> --- a/drivers/pci/msi/msi.c
> +++ b/drivers/pci/msi/msi.c
> @@ -760,6 +760,10 @@ int __pci_enable_msix_range(struct pci_d
> if (WARN_ON_ONCE(dev->msix_enabled))
> return -EINVAL;
>
> + /* Check MSI-X early on irq domain enabled architectures */
> + if (!pci_msi_domain_supports(dev, MSI_FLAG_PCI_MSIX, ALLOW_LEGACY))
> + return -ENOTSUPP;
> +
> if (!pci_msi_supported(dev, nvec) || dev->current_state != PCI_D0)
> return -EINVAL;
>
>

2022-11-16 19:13:53

by Jason Gunthorpe

[permalink] [raw]
Subject: Re: [patch 35/39] PCI/MSI: Reject MSI-X early

On Fri, Nov 11, 2022 at 02:55:11PM +0100, Thomas Gleixner wrote:
> Similar to PCI multi-MSI reject MSI-X enablement when a irq domain is
> attached to the device which does not support MSI-X.
>
> Signed-off-by: Thomas Gleixner <[email protected]>
> ---
> drivers/pci/msi/msi.c | 4 ++++
> 1 file changed, 4 insertions(+)

Reviewed-by: Jason Gunthorpe <[email protected]>

Jason

Subject: [tip: irq/core] PCI/MSI: Reject MSI-X early

The following commit has been merged into the irq/core branch of tip:

Commit-ID: 99f3d279765725920aa5924fa445537a20129a6f
Gitweb: https://git.kernel.org/tip/99f3d279765725920aa5924fa445537a20129a6f
Author: Thomas Gleixner <[email protected]>
AuthorDate: Fri, 11 Nov 2022 14:55:11 +01:00
Committer: Thomas Gleixner <[email protected]>
CommitterDate: Thu, 17 Nov 2022 15:15:22 +01:00

PCI/MSI: Reject MSI-X early

Similar to PCI multi-MSI reject MSI-X enablement when a irq domain is
attached to the device which does not support MSI-X.

Signed-off-by: Thomas Gleixner <[email protected]>
Reviewed-by: Jason Gunthorpe <[email protected]>
Acked-by: Bjorn Helgaas <[email protected]>
Link: https://lore.kernel.org/r/[email protected]

---
drivers/pci/msi/msi.c | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/drivers/pci/msi/msi.c b/drivers/pci/msi/msi.c
index bc84647..0740acd 100644
--- a/drivers/pci/msi/msi.c
+++ b/drivers/pci/msi/msi.c
@@ -762,6 +762,10 @@ int __pci_enable_msix_range(struct pci_dev *dev, struct msix_entry *entries, int
if (WARN_ON_ONCE(dev->msix_enabled))
return -EINVAL;

+ /* Check MSI-X early on irq domain enabled architectures */
+ if (!pci_msi_domain_supports(dev, MSI_FLAG_PCI_MSIX, ALLOW_LEGACY))
+ return -ENOTSUPP;
+
if (!pci_msi_supported(dev, nvec) || dev->current_state != PCI_D0)
return -EINVAL;


2023-01-15 22:34:02

by David Woodhouse

[permalink] [raw]
Subject: [PATCH] x86/xen: Set MSI_FLAG_PCI_MSIX support in Xen MSI domain

The Xen MSI → PIRQ magic does support MSI-X, so advertise it.

(In fact it's better off with MSI-X than MSI, because it's actually
broken by design for 32-bit MSI, since it puts the high bits of the
PIRQ# into the high 32 bits of the MSI message address, instead of the
Extended Destination ID field which is in bits 4-11.

Strictly speaking, this really fixes a much older commit 2e4386eba0c0
("x86/xen: Wrap XEN MSI management into irqdomain") which failed to set
the flag. But that never really mattered until __pci_enable_msix_range()
started to check and bail out early. So in 6.2-rc we see failures e.g.
to bring up networking on an Amazon EC2 m4.16xlarge instance:

[ 41.498694] ena 0000:00:03.0 (unnamed net_device) (uninitialized): Failed to enable MSI-X. irq_cnt -524
[ 41.498705] ena 0000:00:03.0: Can not reserve msix vectors
[ 41.498712] ena 0000:00:03.0: Failed to enable and set the admin interrupts

Side note: This is the first bug found, and first patch tested, by running
Xen guests under QEMU/KVM instead of running under actual Xen.

Fixes: 99f3d2797657 ("PCI/MSI: Reject MSI-X early")
Signed-off-by: David Woodhouse <[email protected]>
---
arch/x86/pci/xen.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index b94f727251b6..790550479831 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -433,6 +433,7 @@ static struct msi_domain_ops xen_pci_msi_domain_ops = {
};

static struct msi_domain_info xen_pci_msi_domain_info = {
+ .flags = MSI_FLAG_PCI_MSIX,
.ops = &xen_pci_msi_domain_ops,
};

--
2.34.1



Attachments:
smime.p7s (5.83 kB)
Subject: [tip: x86/urgent] x86/pci/xen: Set MSI_FLAG_PCI_MSIX support in Xen MSI domain

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID: 0a3a58de319552525507a3497da86df039a5e4e0
Gitweb: https://git.kernel.org/tip/0a3a58de319552525507a3497da86df039a5e4e0
Author: David Woodhouse <[email protected]>
AuthorDate: Sun, 15 Jan 2023 22:14:19
Committer: Thomas Gleixner <[email protected]>
CommitterDate: Mon, 16 Jan 2023 20:40:44 +01:00

x86/pci/xen: Set MSI_FLAG_PCI_MSIX support in Xen MSI domain

The Xen MSI → PIRQ magic does support MSI-X, so advertise it.

(In fact it's better off with MSI-X than MSI, because it's actually
broken by design for 32-bit MSI, since it puts the high bits of the
PIRQ# into the high 32 bits of the MSI message address, instead of the
Extended Destination ID field which is in bits 4-11.

Strictly speaking, this really fixes a much older commit 2e4386eba0c0
("x86/xen: Wrap XEN MSI management into irqdomain") which failed to set
the flag. But that never really mattered until __pci_enable_msix_range()
started to check and bail out early. So in 6.2-rc we see failures e.g.
to bring up networking on an Amazon EC2 m4.16xlarge instance:

[ 41.498694] ena 0000:00:03.0 (unnamed net_device) (uninitialized): Failed to enable MSI-X. irq_cnt -524
[ 41.498705] ena 0000:00:03.0: Can not reserve msix vectors
[ 41.498712] ena 0000:00:03.0: Failed to enable and set the admin interrupts

Side note: This is the first bug found, and first patch tested, by running
Xen guests under QEMU/KVM instead of running under actual Xen.

Fixes: 99f3d2797657 ("PCI/MSI: Reject MSI-X early")
Signed-off-by: David Woodhouse <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Link: https://lore.kernel.org/r/[email protected]

---
arch/x86/pci/xen.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index b94f727..7905504 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -433,6 +433,7 @@ static struct msi_domain_ops xen_pci_msi_domain_ops = {
};

static struct msi_domain_info xen_pci_msi_domain_info = {
+ .flags = MSI_FLAG_PCI_MSIX,
.ops = &xen_pci_msi_domain_ops,
};