On many systems that have an AMD IOMMU the following sequence of
warnings is observed during bootup.
```
pci 0000:00:00.2 can't derive routing for PCI INT A
pci 0000:00:00.2: PCI INT A: not connected
```
This series of events happens because of the IOMMU initialization
sequence order and the lack of _PRT entries for the IOMMU.
During initialization the IOMMU driver first enables the PCI device
using pci_enable_device(). This will call acpi_pci_irq_enable()
which will check if the interrupt is declared in a PCI routing table
(_PRT) entry. According to the PCI spec [1] these routing entries
are only required under PCI root bridges:
The _PRT object is required under all PCI root bridges
The IOMMU is directly connected to the root complex, so there is no
parent bridge to look for a _PRT entry. The first warning is emitted
since no entry could be found in the hierarchy. The second warning is
then emitted because the interrupt hasn't yet been configured to any
value. The pin was configured in pci_read_irq() but the byte in
PCI_INTERRUPT_LINE return 0xff which means "Unknown".
After that sequence of events pci_enable_msi() is called and this
will allocate an interrupt.
That is both of these warnings are totally harmless because the IOMMU
uses MSI for interrupts. To avoid even trying to probe for a _PRT
entry mark the IOMMU as IRQ managed. This avoids both warnings.
Link: https://uefi.org/htmlspecs/ACPI_Spec_6_4_html/06_Device_Configuration/Device_Configuration.html?highlight=_prt#prt-pci-routing-table [1]
Signed-off-by: Mario Limonciello <[email protected]>
---
drivers/iommu/amd/init.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index c83bd0c2a1c9..40979b0f5250 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -2068,6 +2068,9 @@ static int __init iommu_init_pci(struct amd_iommu *iommu)
/* Prevent binding other PCI device drivers to IOMMU devices */
iommu->dev->match_driver = false;
+ /* ACPI _PRT won't have an IRQ for IOMMU */
+ iommu->dev->irq_managed = 1;
+
pci_read_config_dword(iommu->dev, cap_ptr + MMIO_CAP_HDR_OFFSET,
&iommu->cap);
base-commit: 75f74f85a42eb294b657f847c33e1bb7921dbec9
--
2.34.1
On 1/22/2024 17:34, Mario Limonciello wrote:
> On many systems that have an AMD IOMMU the following sequence of
> warnings is observed during bootup.
>
> ```
> pci 0000:00:00.2 can't derive routing for PCI INT A
> pci 0000:00:00.2: PCI INT A: not connected
> ```
>
> This series of events happens because of the IOMMU initialization
> sequence order and the lack of _PRT entries for the IOMMU.
>
> During initialization the IOMMU driver first enables the PCI device
> using pci_enable_device(). This will call acpi_pci_irq_enable()
> which will check if the interrupt is declared in a PCI routing table
> (_PRT) entry. According to the PCI spec [1] these routing entries
> are only required under PCI root bridges:
> The _PRT object is required under all PCI root bridges
>
> The IOMMU is directly connected to the root complex, so there is no
> parent bridge to look for a _PRT entry. The first warning is emitted
> since no entry could be found in the hierarchy. The second warning is
> then emitted because the interrupt hasn't yet been configured to any
> value. The pin was configured in pci_read_irq() but the byte in
> PCI_INTERRUPT_LINE return 0xff which means "Unknown".
>
> After that sequence of events pci_enable_msi() is called and this
> will allocate an interrupt.
>
> That is both of these warnings are totally harmless because the IOMMU
> uses MSI for interrupts. To avoid even trying to probe for a _PRT
> entry mark the IOMMU as IRQ managed. This avoids both warnings.
>
> Link: https://uefi.org/htmlspecs/ACPI_Spec_6_4_html/06_Device_Configuration/Device_Configuration.html?highlight=_prt#prt-pci-routing-table [1]
> Signed-off-by: Mario Limonciello <[email protected]>
Hi,
Friendly ping on reviewing this patch.
Thanks,
> ---
> drivers/iommu/amd/init.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
> index c83bd0c2a1c9..40979b0f5250 100644
> --- a/drivers/iommu/amd/init.c
> +++ b/drivers/iommu/amd/init.c
> @@ -2068,6 +2068,9 @@ static int __init iommu_init_pci(struct amd_iommu *iommu)
> /* Prevent binding other PCI device drivers to IOMMU devices */
> iommu->dev->match_driver = false;
>
> + /* ACPI _PRT won't have an IRQ for IOMMU */
> + iommu->dev->irq_managed = 1;
> +
> pci_read_config_dword(iommu->dev, cap_ptr + MMIO_CAP_HDR_OFFSET,
> &iommu->cap);
>
>
> base-commit: 75f74f85a42eb294b657f847c33e1bb7921dbec9
Hi Mario,
On Mon, Feb 05, 2024 at 10:31:47AM -0600, Mario Limonciello wrote:
> Friendly ping on reviewing this patch.
Thanks for the patch. Since this is a fix the patch should contain a
Fixes: tag. Also a review from Suravee and/or Vasant would be great.
Thanks,
Joerg
On 2/9/2024 02:24, Joerg Roedel wrote:
> Hi Mario,
>
> On Mon, Feb 05, 2024 at 10:31:47AM -0600, Mario Limonciello wrote:
>> Friendly ping on reviewing this patch.
>
> Thanks for the patch. Since this is a fix the patch should contain a
> Fixes: tag. Also a review from Suravee and/or Vasant would be great.
>
> Thanks,
>
> Joerg
>
Thanks! This problem should go all the way to when MSI support was
introduced (2.6.28):
a80dc3e0e0dc ("AMD IOMMU: add MSI interrupt support")
But I don't think that's a correct fixes tag because
irq_managed wasn't introduced until 3.19:
cffe0a2b5a34 ("x86, irq: Keep balance of IOAPIC pin reference count")
But then I tried to go off the original introduction of the error
message, but that's quite ancient.
1da177e4c3f4 ("Linux-2.6.12-rc2")
So which tag do you think is appropriate to use? I'm tempted to say
this should just be without a fixes tag just a:
Cc: [email protected] # 3.19+
On Fri, Feb 09, 2024 at 09:48:57AM -0600, Mario Limonciello wrote:
> Thanks! This problem should go all the way to when MSI support was
> introduced (2.6.28):
>
> a80dc3e0e0dc ("AMD IOMMU: add MSI interrupt support")
That makes most sense to me, also if it only uncovers with ...
>
> But I don't think that's a correct fixes tag because
> irq_managed wasn't introduced until 3.19:
>
> cffe0a2b5a34 ("x86, irq: Keep balance of IOAPIC pin reference count")
This one.
> But then I tried to go off the original introduction of the error message,
> but that's quite ancient.
>
> 1da177e4c3f4 ("Linux-2.6.12-rc2")
That is not relevant, as the first AMD IOMMU driver was only merged for
2.6.27 :)
> Cc: [email protected] # 3.19+
That makes sense too.
Regards,
Joerg
On 1/23/2024 5:04 AM, Mario Limonciello wrote:
> On many systems that have an AMD IOMMU the following sequence of
> warnings is observed during bootup.
>
> ```
> pci 0000:00:00.2 can't derive routing for PCI INT A
> pci 0000:00:00.2: PCI INT A: not connected
> ```
>
> This series of events happens because of the IOMMU initialization
> sequence order and the lack of _PRT entries for the IOMMU.
>
> During initialization the IOMMU driver first enables the PCI device
> using pci_enable_device(). This will call acpi_pci_irq_enable()
> which will check if the interrupt is declared in a PCI routing table
> (_PRT) entry. According to the PCI spec [1] these routing entries
> are only required under PCI root bridges:
> The _PRT object is required under all PCI root bridges
>
> The IOMMU is directly connected to the root complex, so there is no
> parent bridge to look for a _PRT entry. The first warning is emitted
> since no entry could be found in the hierarchy. The second warning is
> then emitted because the interrupt hasn't yet been configured to any
> value. The pin was configured in pci_read_irq() but the byte in
> PCI_INTERRUPT_LINE return 0xff which means "Unknown".
>
> After that sequence of events pci_enable_msi() is called and this
> will allocate an interrupt.
>
> That is both of these warnings are totally harmless because the IOMMU
> uses MSI for interrupts. To avoid even trying to probe for a _PRT
> entry mark the IOMMU as IRQ managed. This avoids both warnings.
>
> Link: https://uefi.org/htmlspecs/ACPI_Spec_6_4_html/06_Device_Configuration/Device_Configuration.html?highlight=_prt#prt-pci-routing-table [1]
> Signed-off-by: Mario Limonciello <[email protected]>
Patch looks good to me.
Reviewed-by: Vasant Hegde <[email protected]>
-Vasant
> ---
> drivers/iommu/amd/init.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
> index c83bd0c2a1c9..40979b0f5250 100644
> --- a/drivers/iommu/amd/init.c
> +++ b/drivers/iommu/amd/init.c
> @@ -2068,6 +2068,9 @@ static int __init iommu_init_pci(struct amd_iommu *iommu)
> /* Prevent binding other PCI device drivers to IOMMU devices */
> iommu->dev->match_driver = false;
>
> + /* ACPI _PRT won't have an IRQ for IOMMU */
> + iommu->dev->irq_managed = 1;
> +
> pci_read_config_dword(iommu->dev, cap_ptr + MMIO_CAP_HDR_OFFSET,
> &iommu->cap);
>
>
> base-commit: 75f74f85a42eb294b657f847c33e1bb7921dbec9
On Mon, Jan 22, 2024 at 05:34:00PM -0600, Mario Limonciello wrote:
> Link: https://uefi.org/htmlspecs/ACPI_Spec_6_4_html/06_Device_Configuration/Device_Configuration.html?highlight=_prt#prt-pci-routing-table [1]
> Signed-off-by: Mario Limonciello <[email protected]>
> ---
> drivers/iommu/amd/init.c | 3 +++
> 1 file changed, 3 insertions(+)
Applied, thanks.