Greetings,
I'm running into an issue regarding ath9k interrupts on imx8mm SoC's.
I'm using the latest kernel source (master) but have recreated this
all the way back to 5.15.
I find on boards with an IMX8MM (ie imx8mm-venice-gw320x-0x
device-tree) ath9k will not trigger any legacy interrupts.
# dmesg | grep ath
[ 2.468657] ath9k 0000:04:00.0: enabling device (0000 -> 0002)
[ 2.474702] ath: phy0: WB335 2-ANT card detected
[ 2.479341] ath: phy0: Set BT/WLAN RX diversity capability
[ 2.493720] ath: phy0: Enable LNA combining
[ 2.499099] ath: EEPROM regdomain: 0x6a
[ 2.499106] ath: EEPROM indicates we should expect a direct regpair map
[ 2.499114] ath: Country alpha2 being used: 00
[ 2.499119] ath: Regpair used: 0x6a
[ 8.167876] ath9k 0000:04:00.0 wlp4s0: renamed from wlan0
# wpa_supplicant -i wlp4s0 -c wpa_supplicant.conf -B
Successfully initialized wpa_supplicant
# [ 143.368897] wlp4s0: authenticate with 04:f0:21:59:3c:f8
[ 143.374230] wlp4s0: 80 MHz not supported, disabling VHT
[ 143.391982] wlp4s0: send auth to 04:f0:21:59:3c:f8 (try 1/3)
[ 144.831653] wlp4s0: send auth to 04:f0:21:59:3c:f8 (try 2/3)
[ 145.823657] wlp4s0: send auth to 04:f0:21:59:3c:f8 (try 3/3)
[ 146.815655] wlp4s0: authentication with 04:f0:21:59:3c:f8 timed out
[ 147.896946] wlp4s0: authenticate with 04:f0:21:59:3c:f8
[ 147.902290] wlp4s0: 80 MHz not supported, disabling VHT
....
# grep ath /proc/interrupts
205: 0 0 0 0 GICv3 157 Level ath9k
^^^ no interrupts
I've read that some SoC miniPCIe controllers don't support legacy
interrupts so I try again adding 'ath9k.use_msi=1' to bootargs:
root@jammy-venice:~# dmesg | grep ath
[ 4.314829] systemd[1]: Reached target Path Units.
[ 7.707747] usbcore: registered new interface driver ath3k
[ 7.792173] ath9k 0000:03:00.0: enabling device (0000 -> 0002)
[ 7.792471] ath9k 0000:03:00.0: Using MSI
[ 7.796661] ath: phy0: WB335 2-ANT card detected
[ 7.796674] ath: phy0: Set BT/WLAN RX diversity capability
[ 7.804722] ath: phy0: Enable LNA combining
[ 7.805995] ath: EEPROM regdomain: 0x6a
[ 7.806008] ath: EEPROM indicates we should expect a direct regpair map
[ 7.806015] ath: Country alpha2 being used: 00
[ 7.806019] ath: Regpair used: 0x6a
[ 8.493786] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
[ 8.501691] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
[ 8.515139] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
[ 8.632850] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
[ 8.646438] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
[ 8.716020] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 8.787447] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
...
root@jammy-venice:~# grep ath /proc/interrupts
239: 0 0 0 0 PCI-MSI 1572864
Edge ath9k
^^^ now the driver is having trouble with MSI interrupts and the
device doesn't work
Strangely when I use the same radio and kernel on an IMX6Q board (ie
imx6q-gw54xx) (which has the same PCI MAC but different PHY than the
IMX8MM) I find that I 'must' use legacy interrupts and trying to use
MSI interrupts results in the above behavior.
Note that the boards I'm using have a PCI switch and I'm wondering if
that is causing the issue.
Additionally I can say that ath10k cards work fine on the same imx8mm
board with MSI interrupts.
It seems that Daniel may have been running into something similar back
when commit 7368160f0ab0 ("ath9k: add MSI support") was merged
[1][2][3]
Does anyone have any advice here about what can be going on?
Best Regards,
Tim
[1] https://patchwork.kernel.org/project/linux-wireless/patch/[email protected]/
[2] https://marc.info/?l=linux-pci&m=150238260826803&w=2
[3] https://marc.info/?l=linux-pci&m=150831581725596&w=2
On Sat, Mar 18, 2023 at 8:17 AM Tim Harvey <[email protected]> wrote:
> It seems that Daniel may have been running into something similar back
> when commit 7368160f0ab0 ("ath9k: add MSI support") was merged
> [1][2][3]
>
> Does anyone have any advice here about what can be going on?
The issue we found earlier was that ath9k MSI is broken unless the MSI
IRQ number coincidentally is aligned to 4.
Worked around with:
https://github.com/endlessm/linux/commit/0a2d86a2b9da8f936d873addf0a97e341177fe8a
https://github.com/endlessm/linux/commit/970d2b4b59465cd926e5213e0feb633f6b623447
IIRC the PCI patch wasn't accepted upstream at the time and I never
got any response from Atheros/Qualcomm about this device
hardware/firmware bug.
Daniel
On Mon, Mar 20, 2023 at 1:29 PM Daniel Drake <[email protected]> wrote:
>
> On Sat, Mar 18, 2023 at 8:17 AM Tim Harvey <[email protected]> wrote:
> > It seems that Daniel may have been running into something similar back
> > when commit 7368160f0ab0 ("ath9k: add MSI support") was merged
> > [1][2][3]
> >
> > Does anyone have any advice here about what can be going on?
>
> The issue we found earlier was that ath9k MSI is broken unless the MSI
> IRQ number coincidentally is aligned to 4.
>
> Worked around with:
> https://github.com/endlessm/linux/commit/0a2d86a2b9da8f936d873addf0a97e341177fe8a
> https://github.com/endlessm/linux/commit/970d2b4b59465cd926e5213e0feb633f6b623447
>
> IIRC the PCI patch wasn't accepted upstream at the time
Briefly looking into it, it won't be accepted in the future either (in
the current form).
The solution there as far as I (possibly mistakenly) can see is that
it's only for x86 cases. For ARM or others there is no explanation on
what would be the effect of the MSI enabling. Hence, it's not widely
tested, nor root caused: May it be the certain x86 PCI bridge / switch
issue? May it be the APIC related issue on that x86 platform? Is it a
bug in Atheros FW/HW?
> and I never
> got any response from Atheros/Qualcomm about this device
> hardware/firmware bug.
--
With Best Regards,
Andy Shevchenko
(+linux-pci, Fabio, Richard)
Daniel,
I may have two different issues going on here:
1) issues with ath9k MSI interrupts
2) issues with ath9k legacy interrupts on IMX8MM
To rule out the PCIe switch being an issue for me I moved to an
imx8mm-venice-gw71xx-0x which has an IMX8MM and no PCIe switch.
I've found that with MSI enabled globally, but ath9k not using MSI I
fail to get any interrupts from the radio with nothing abnormal from
dmesg:
# uname -r
6.2.0-09243-ge911eb7b8ec1
# cat /proc/cmdline
# lspci -n
00:00.0 0604: 16c3:abcd (rev 01)
01:00.0 0280: 168c:0036 (rev 01)
# dmesg | grep pci
[ 1.524560] imx6q-pcie 33800000.pcie: host bridge
/soc@0/pcie@33800000 ranges:
[ 1.531870] imx6q-pcie 33800000.pcie: IO
0x001ff80000..0x001ff8ffff -> 0x0000000000
[ 1.544109] imx6q-pcie 33800000.pcie: MEM
0x0018000000..0x001fefffff -> 0x0018000000
[ 1.767610] imx6q-pcie 33800000.pcie: iATU: unroll T, 4 ob, 4 ib,
align 64K, limit 4G
[ 1.875571] imx6q-pcie 33800000.pcie: PCIe Gen.1 x1 link up
[ 1.881241] imx6q-pcie 33800000.pcie: PCIe Gen.1 x1 link up
[ 1.886843] imx6q-pcie 33800000.pcie: Link up, Gen1
[ 1.891746] imx6q-pcie 33800000.pcie: PCIe Gen.1 x1 link up
[ 1.897674] imx6q-pcie 33800000.pcie: PCI host bridge to bus 0000:00
[ 1.904078] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 1.909585] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 1.915784] pci_bus 0000:00: root bus resource [mem 0x18000000-0x1fefffff]
[ 1.922706] pci 0000:00:00.0: [16c3:abcd] type 01 class 0x060400
[ 1.928746] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[ 1.935043] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[ 1.941820] pci 0000:00:00.0: supports D1
[ 1.945849] pci 0000:00:00.0: PME# supported from D0 D1 D3hot D3cold
[ 1.954254] pci 0000:01:00.0: [168c:0036] type 00 class 0x028000
[ 1.960394] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0007ffff 64bit]
[ 1.967342] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[ 1.974389] pci 0000:01:00.0: supports D1 D2
[ 1.978681] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 1.995483] pci 0000:00:00.0: BAR 0: assigned [mem 0x18000000-0x180fffff]
[ 2.002334] pci 0000:00:00.0: BAR 14: assigned [mem 0x18100000-0x181fffff]
[ 2.009240] pci 0000:00:00.0: BAR 15: assigned [mem
0x18200000-0x182fffff pref]
[ 2.016576] pci 0000:00:00.0: BAR 6: assigned [mem
0x18300000-0x1830ffff pref]
[ 2.023833] pci 0000:01:00.0: BAR 0: assigned [mem
0x18100000-0x1817ffff 64bit]
[ 2.031214] pci 0000:01:00.0: BAR 6: assigned [mem
0x18200000-0x1820ffff pref]
[ 2.038462] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[ 2.043711] pci 0000:00:00.0: bridge window [mem 0x18100000-0x181fffff]
[ 2.050518] pci 0000:00:00.0: bridge window [mem
0x18200000-0x182fffff pref]
[ 2.058205] pcieport 0000:00:00.0: PME: Signaling with IRQ 205
# dmesg | grep ath
[ 2.064379] ath9k 0000:01:00.0: enabling device (0000 -> 0002)
[ 2.070445] ath: phy0: WB335 2-ANT card detected
[ 2.075092] ath: phy0: Set BT/WLAN RX diversity capability
[ 2.088891] ath: phy0: Enable LNA combining
[ 2.094278] ath: EEPROM regdomain: 0x6a
[ 2.094285] ath: EEPROM indicates we should expect a direct regpair map
[ 2.094293] ath: Country alpha2 being used: 00
[ 2.094297] ath: Regpair used: 0x6a
[ 14.228284] ath9k 0000:01:00.0 wlp1s0: renamed from wlan0
# wpa_supplicant -i wlp1s0 -c wpa_supplicant.conf
Successfully initialized wpa_supplicant
<snip authentication details>
# grep ath /proc/interrupts
204: 0 0 0 0 GICv3 157 Level ath9k
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
11: 2196 1753 1883 3509 GICv3 30 Level
arch_timer
14: 0 0 0 0 GICv3 79 Level
timer@306a0000
15: 0 0 0 0 GICv3 23 Level arm-pmu
16: 0 0 0 0 GICv3 135 Level
302c0000.dma-controller
17: 0 0 0 0 GICv3 66 Level
302b0000.dma-controller
18: 0 0 0 0 GICv3 34 Level
30bd0000.dma-controller
19: 0 0 0 0 GICv3 58 Level
30860000.serial
20: 0 0 0 0 GICv3 60 Level
30880000.serial
21: 503 0 0 0 GICv3 59 Level
30890000.serial
22: 0 0 0 0 GICv3 110 Level
30280000.watchdog
23: 418 0 0 0 GICv3 56 Level mmc2
41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
66: 0 0 0 0 gpio-mxc 6 Level
gateworks-gsc
194: 750 0 0 0 GICv3 67 Level
30a20000.i2c
195: 0 0 0 0 gateworks-gsc 4
Edge 0-0023
196: 25 0 0 0 GICv3 68 Level
30a30000.i2c
197: 0 0 0 0 GICv3 69 Level
30a40000.i2c
198: 0 0 0 0 GICv3 64 Level
30830000.spi
199: 0 0 0 0 GICv3 150 Level
30be0000.ethernet
200: 0 0 0 0 GICv3 151 Level
30be0000.ethernet
201: 0 0 0 0 GICv3 152 Level
30be0000.ethernet
202: 0 0 0 0 GICv3 153 Level
30be0000.ethernet
204: 0 0 0 0 GICv3 157 Level ath9k
205: 0 0 0 0 PCI-MSI 0 Edge
PCIe PME
206: 0 0 0 0 GICv3 72 Level
32e40000.usb
207: 18 0 0 0 GICv3 73 Level
32e50000.usb
208: 0 0 0 0 gateworks-gsc 0
Edge user_pb1x
209: 0 0 0 0 gateworks-gsc 1
Edge key_erased
210: 0 0 0 0 gateworks-gsc 2
Edge eeprom_wp
211: 0 0 0 0 gateworks-gsc 5
Edge tamper
212: 0 0 0 0 gateworks-gsc 7
Edge switch_hold
213: 0 0 0 0 0-0023 2 Edge user_pb
IPI0: 620 751 665 109 Rescheduling interrupts
IPI1: 361 1387 1285 509 Function call interrupts
IPI2: 0 0 0 0 CPU stop interrupts
IPI3: 0 0 0 0 CPU stop (for
crash dump) interrupts
IPI4: 0 0 0 0 Timer broadcast
interrupts
IPI5: 5 9 10 2 IRQ work interrupts
IPI6: 0 0 0 0 CPU wake-up interrupts
Err: 0
^^^ no interrupts from ath9k radio
^^^ is MSI enabled for the controller but ath9k electing to use legacy
INT's a valid configuration?
With MSI disabled globally via pci=nomsi ath9k gets interrupts and works fine:
# cat /proc/cmdline
pci=nomsi
# dmesg | grep ath
[ 2.052527] ath9k 0000:01:00.0: enabling device (0000 -> 0002)
[ 2.058556] ath: phy0: WB335 2-ANT card detected
[ 2.063199] ath: phy0: Set BT/WLAN RX diversity capability
[ 2.077381] ath: phy0: Enable LNA combining
[ 2.082799] ath: EEPROM regdomain: 0x6a
[ 2.082805] ath: EEPROM indicates we should expect a direct regpair map
[ 2.082813] ath: Country alpha2 being used: 00
[ 2.082818] ath: Regpair used: 0x6a
[ 11.620093] ath9k 0000:01:00.0 wlp1s0: renamed from wlan0
# wpa_supplicant -i wlp1s0 -c wpa_supplicant.conf
Successfully initialized wpa_supplicant
<snip authentication details>
#
# grep ath /proc/interrupts
203: 379 0 0 0 GICv3 157 Level
PCIe PME, ath9k
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
11: 1622 2152 3288 2005 GICv3 30 Level
arch_timer
14: 0 0 0 0 GICv3 79 Level
timer@306a0000
15: 0 0 0 0 GICv3 23 Level arm-pmu
16: 0 0 0 0 GICv3 135 Level
302c0000.dma-controller
17: 0 0 0 0 GICv3 66 Level
302b0000.dma-controller
18: 0 0 0 0 GICv3 34 Level
30bd0000.dma-controller
19: 0 0 0 0 GICv3 58 Level
30860000.serial
20: 0 0 0 0 GICv3 60 Level
30880000.serial
21: 597 0 0 0 GICv3 59 Level
30890000.serial
22: 0 0 0 0 GICv3 110 Level
30280000.watchdog
23: 414 0 0 0 GICv3 56 Level mmc2
41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
66: 0 0 0 0 gpio-mxc 6 Level
gateworks-gsc
194: 978 0 0 0 GICv3 67 Level
30a20000.i2c
195: 0 0 0 0 gateworks-gsc 4
Edge 0-0023
196: 25 0 0 0 GICv3 68 Level
30a30000.i2c
197: 0 0 0 0 GICv3 69 Level
30a40000.i2c
198: 0 0 0 0 GICv3 64 Level
30830000.spi
199: 0 0 0 0 GICv3 150 Level
30be0000.ethernet
200: 0 0 0 0 GICv3 151 Level
30be0000.ethernet
201: 0 0 0 0 GICv3 152 Level
30be0000.ethernet
202: 0 0 0 0 GICv3 153 Level
30be0000.ethernet
203: 379 0 0 0 GICv3 157 Level
PCIe PME, ath9k
204: 0 0 0 0 GICv3 72 Level
32e40000.usb
205: 18 0 0 0 GICv3 73 Level
32e50000.usb
206: 0 0 0 0 gateworks-gsc 0
Edge user_pb1x
207: 0 0 0 0 gateworks-gsc 1
Edge key_erased
208: 0 0 0 0 gateworks-gsc 2
Edge eeprom_wp
209: 0 0 0 0 gateworks-gsc 5
Edge tamper
210: 0 0 0 0 gateworks-gsc 7
Edge switch_hold
211: 0 0 0 0 0-0023 2 Edge user_pb
IPI0: 501 506 73 759 Rescheduling interrupts
IPI1: 878 846 718 1169 Function call interrupts
IPI2: 0 0 0 0 CPU stop interrupts
IPI3: 0 0 0 0 CPU stop (for
crash dump) interrupts
IPI4: 0 0 0 0 Timer broadcast
interrupts
IPI5: 5 4 9 20 IRQ work interrupts
IPI6: 0 0 0 0 CPU wake-up interrupts
Err: 0
^^^ legacy interrupts working as long as I disable MSI globally for IMX8MM
And with MSI enabled globally as well as ath9k (ath9k.use_msi=1) ath9k
fails to get interrupts and shows the interrupt error I ran into
previously:
# cat /proc/cmdline
ath9k.use_msi=1
# dmesg | grep ath
[ 0.000000] Kernel command line: ath9k.use_msi=1
[ 2.056381] ath9k 0000:01:00.0: enabling device (0000 -> 0002)
[ 2.062571] ath9k 0000:01:00.0: Using MSI
[ 2.066651] ath: phy0: WB335 2-ANT card detected
[ 2.071288] ath: phy0: Set BT/WLAN RX diversity capability
[ 2.085083] ath: phy0: Enable LNA combining
[ 2.090470] ath: EEPROM regdomain: 0x6a
[ 2.090476] ath: EEPROM indicates we should expect a direct regpair map
[ 2.090484] ath: Country alpha2 being used: 00
[ 2.090489] ath: Regpair used: 0x6a
[ 13.999159] ath9k 0000:01:00.0 wlp1s0: renamed from wlan0
# wpa_supplicant -i wlp1s0 -c wpa_supplicant.conf
Successfully initialized wpa_supplicant
[ 66.790178] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
[ 66.798000] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
[ 66.811551] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
[ 66.937204] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
[ 66.950679] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
[ 67.020994] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 67.092917] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 67.164905] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 67.236957] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 67.308928] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 67.380902] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 67.452954] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 67.524881] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 67.596917] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 67.668907] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 67.740908] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
[ 67.860948] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
...
<snip authentication details>
# grep ath /proc/interrupts
206: 0 0 0 0 PCI-MSI 524288 Edge
ath9k
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
11: 1421 3067 1999 2767 GICv3 30 Level
arch_timer
14: 0 0 0 0 GICv3 79 Level
timer@306a0000
15: 0 0 0 0 GICv3 23 Level arm-pmu
16: 0 0 0 0 GICv3 135 Level
302c0000.dma-controller
17: 0 0 0 0 GICv3 66 Level
302b0000.dma-controller
18: 0 0 0 0 GICv3 34 Level
30bd0000.dma-controller
19: 0 0 0 0 GICv3 58 Level
30860000.serial
20: 0 0 0 0 GICv3 60 Level
30880000.serial
21: 289 0 0 0 GICv3 59 Level
30890000.serial
22: 0 0 0 0 GICv3 110 Level
30280000.watchdog
23: 390 0 0 0 GICv3 56 Level mmc2
41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
66: 0 0 0 0 gpio-mxc 6 Level
gateworks-gsc
194: 1054 0 0 0 GICv3 67 Level
30a20000.i2c
195: 0 0 0 0 gateworks-gsc 4
Edge 0-0023
196: 25 0 0 0 GICv3 68 Level
30a30000.i2c
197: 0 0 0 0 GICv3 69 Level
30a40000.i2c
198: 0 0 0 0 GICv3 64 Level
30830000.spi
199: 0 0 0 0 GICv3 150 Level
30be0000.ethernet
200: 0 0 0 0 GICv3 151 Level
30be0000.ethernet
201: 0 0 0 0 GICv3 152 Level
30be0000.ethernet
202: 0 0 0 0 GICv3 153 Level
30be0000.ethernet
205: 6891 0 0 0 PCI-MSI 0 Edge
PCIe PME
206: 0 0 0 0 PCI-MSI 524288 Edge
ath9k
207: 0 0 0 0 GICv3 72 Level
32e40000.usb
208: 18 0 0 0 GICv3 73 Level
32e50000.usb
209: 0 0 0 0 gateworks-gsc 0
Edge user_pb1x
210: 0 0 0 0 gateworks-gsc 1
Edge key_erased
211: 0 0 0 0 gateworks-gsc 2
Edge eeprom_wp
212: 0 0 0 0 gateworks-gsc 5
Edge tamper
213: 0 0 0 0 gateworks-gsc 7
Edge switch_hold
214: 0 0 0 0 0-0023 2 Edge user_pb
IPI0: 646 64 692 808 Rescheduling interrupts
IPI1: 1211 893 503 1097 Function call interrupts
IPI2: 0 0 0 0 CPU stop interrupts
IPI3: 0 0 0 0 CPU stop (for
crash dump) interrupts
IPI4: 0 0 0 0 Timer broadcast
interrupts
IPI5: 9 13 12 8 IRQ work interrupts
IPI6: 0 0 0 0 CPU wake-up interrupts
Err: 0
^^^ It looks to me like when using MSI above the interrupts are going
to the PME driver MSI 0 and the ath9k MSI interrupt shows as 524288
which aligns to 4 but seems strange that its so large.
Does this jive with what you encountered?
Note that ath10k which require MSI interrupts are working fine on this
platform and the MSI interrupt is also 524288:
205: 0 0 0 0 PCI-MSI 0 Edge
PCIe PME
206: 632 0 0 0 PCI-MSI 524288 Edge
ath10k_pci
Note that ath11k which requires MSI interrupts is also working fine on
this board:
205: 0 0 0 0 PCI-MSI 0 Edge
PCIe PME
206: 4 0 0 0 PCI-MSI 524288 Edge bhi
207: 4 0 0 0 PCI-MSI 524289 Edge mhi
208: 39 0 0 0 PCI-MSI 524290 Edge mhi
209: 3 0 0 0 PCI-MSI 524291 Edge ce0
210: 26 0 0 1 PCI-MSI 524292 Edge ce1
211: 267 0 0 1 PCI-MSI 524293 Edge ce2
212: 88 0 0 0 PCI-MSI 524294 Edge ce3
213: 0 0 0 1 PCI-MSI 524295 Edge ce5
214: 2 0 0 0 PCI-MSI 524296 Edge
DP_EXT_IRQ
215: 0 0 0 0 PCI-MSI 524297 Edge
DP_EXT_IRQ
216: 0 0 0 0 PCI-MSI 524298 Edge
DP_EXT_IRQ
217: 15 0 0 0 PCI-MSI 524299 Edge
DP_EXT_IRQ
218: 0 0 0 0 PCI-MSI 524300 Edge
DP_EXT_IRQ
219: 2 0 0 0 PCI-MSI 524301 Edge
DP_EXT_IRQ
220: 0 0 0 0 PCI-MSI 524302 Edge
DP_EXT_IRQ
221: 0 0 0 0 PCI-MSI 524303 Edge
DP_EXT_IRQ
Now, if I move back to an imx8mm-venice-gw73xx-0x which has an IMX8MM
and a PCI bridge I can't even get the legacy IRQ configuration to work
(no interrupts):
# cat /proc/cmdline
pci=nomsi
# lspci
00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01)
01:00.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
02:01.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
02:02.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
02:03.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
04:00.0 Network controller: Qualcomm Atheros QCA9565 / AR9565 Wireless
Network Adapter (rev 01)
05:00.0 Ethernet controller: Microchip Technology / SMSC Device 7430 (rev 11)
# dmesg | grep ath
[ 2.475134] ath9k 0000:04:00.0: enabling device (0000 -> 0002)
[ 2.481157] ath: phy0: WB335 2-ANT card detected
[ 2.485797] ath: phy0: Set BT/WLAN RX diversity capability
[ 2.499443] ath: phy0: Enable LNA combining
[ 2.504837] ath: EEPROM regdomain: 0x6a
[ 2.504845] ath: EEPROM indicates we should expect a direct regpair map
[ 2.504853] ath: Country alpha2 being used: 00
[ 2.504858] ath: Regpair used: 0x6a
[ 10.237297] ath9k 0000:04:00.0 wlp4s0: renamed from wlan0
# wpa_supplicant -i wlp4s0 -c wpa_supplicant.conf
<snip failed auth>
# grep -i pci /proc/interrupts
204: 0 0 0 0 GICv3 157 Level
PCIe PME, ath9k
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
11: 1550 3047 2199 4301 GICv3 30 Level
arch_timer
14: 0 0 0 0 GICv3 79 Level
timer@306a0000
15: 0 0 0 0 GICv3 23 Level arm-pmu
16: 0 0 0 0 GICv3 135 Level
302c0000.dma-controller
17: 0 0 0 0 GICv3 66 Level
302b0000.dma-controller
18: 332 0 0 0 GICv3 34 Level
30bd0000.dma-controller
19: 0 0 0 0 GICv3 58 Level
30860000.serial
20: 773 0 0 0 GICv3 59 Level
30890000.serial
21: 0 0 0 0 GICv3 61 Level
30a60000.serial
22: 0 0 0 0 GICv3 110 Level
30280000.watchdog
23: 433 0 0 0 GICv3 56 Level mmc2
41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
66: 0 0 0 0 gpio-mxc 6 Level
gateworks-gsc
72: 0 0 0 0 gpio-mxc 12 Edge
30b50000.mmc cd
170: 4 0 0 0 gpio-mxc 8 Edge
30880000.serial
194: 1282 0 0 0 GICv3 67 Level
30a20000.i2c
195: 0 0 0 0 gateworks-gsc 4
Edge 0-0023
196: 25 0 0 0 GICv3 68 Level
30a30000.i2c
197: 0 0 0 0 GICv3 69 Level
30a40000.i2c
198: 0 0 0 0 GICv3 60 Level
30880000.serial
199: 0 0 0 0 GICv3 64 Level
30830000.spi
200: 0 0 0 0 GICv3 150 Level
30be0000.ethernet
201: 0 0 0 0 GICv3 151 Level
30be0000.ethernet
202: 0 0 0 0 GICv3 152 Level
30be0000.ethernet
203: 0 0 0 0 GICv3 153 Level
30be0000.ethernet
204: 0 0 0 0 GICv3 157 Level
PCIe PME, ath9k
205: 0 0 0 0 GICv3 72 Level
32e40000.usb
206: 105 0 0 0 GICv3 73 Level
32e50000.usb
207: 6244 0 0 0 GICv3 54 Level mmc0
208: 0 0 0 0 GICv3 55 Level mmc1
209: 0 0 0 0 gateworks-gsc 0
Edge user_pb1x
210: 0 0 0 0 gateworks-gsc 1
Edge key_erased
211: 0 0 0 0 gateworks-gsc 2
Edge eeprom_wp
212: 0 0 0 0 gateworks-gsc 5
Edge tamper
213: 0 0 0 0 gateworks-gsc 7
Edge switch_hold
214: 0 0 0 0 0-0023 2 Edge user_pb
IPI0: 367 486 444 286 Rescheduling interrupts
IPI1: 972 1283 1202 1630 Function call interrupts
IPI2: 0 0 0 0 CPU stop interrupts
IPI3: 0 0 0 0 CPU stop (for
crash dump) interrupts
IPI4: 0 0 0 0 Timer broadcast
interrupts
IPI5: 5 6 7 36 IRQ work interrupts
IPI6: 0 0 0 0 CPU wake-up interrupts
Err: 0
^^^ why no interrupts on this board when disabling MSI globally and
using ath9k legacy interrupts.... something to do with the bridge?
Best Regards,
Tim
On Mon, Mar 20, 2023 at 4:29 AM Daniel Drake <[email protected]> wrote:
>
> On Sat, Mar 18, 2023 at 8:17 AM Tim Harvey <[email protected]> wrote:
> > It seems that Daniel may have been running into something similar back
> > when commit 7368160f0ab0 ("ath9k: add MSI support") was merged
> > [1][2][3]
> >
> > Does anyone have any advice here about what can be going on?
>
> The issue we found earlier was that ath9k MSI is broken unless the MSI
> IRQ number coincidentally is aligned to 4.
>
> Worked around with:
> https://github.com/endlessm/linux/commit/0a2d86a2b9da8f936d873addf0a97e341177fe8a
> https://github.com/endlessm/linux/commit/970d2b4b59465cd926e5213e0feb633f6b623447
>
> IIRC the PCI patch wasn't accepted upstream at the time and I never
> got any response from Atheros/Qualcomm about this device
> hardware/firmware bug.
>
> Daniel
On Mon, Mar 20, 2023 at 1:46 PM Tim Harvey <[email protected]> wrote:
>
> (+linux-pci, Fabio, Richard)
>
> Daniel,
>
> I may have two different issues going on here:
> 1) issues with ath9k MSI interrupts
> 2) issues with ath9k legacy interrupts on IMX8MM
Lucas,
I've found an old thread [1] where you state that Designware PCIe core
(not just IMX6) has an issue where as soon as any MSI interrupt is
enabled the core does not forward legacy IRQs anymore so if any card
in your system needs legacy interrupts you must disable MSI
completely.
I assume this is still the understanding and also applies then to
IMX8MM/IMX8MP? This would explain why I'm unable to get legacy
interrupts working without pci=nomsi as the PME driver requests an MSI
interrupt.
Do you know any reason why I can get legacy IRQ's to work with
pci=nomsi on IMX6Q/DL with and without a bridge but on IMX8MM I can
only get it to work without a bridge?
Best Regards,
Tim
[1] https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/#22345435
>
> To rule out the PCIe switch being an issue for me I moved to an
> imx8mm-venice-gw71xx-0x which has an IMX8MM and no PCIe switch.
>
> I've found that with MSI enabled globally, but ath9k not using MSI I
> fail to get any interrupts from the radio with nothing abnormal from
> dmesg:
> # uname -r
> 6.2.0-09243-ge911eb7b8ec1
> # cat /proc/cmdline
>
> # lspci -n
> 00:00.0 0604: 16c3:abcd (rev 01)
> 01:00.0 0280: 168c:0036 (rev 01)
> # dmesg | grep pci
> [ 1.524560] imx6q-pcie 33800000.pcie: host bridge
> /soc@0/pcie@33800000 ranges:
> [ 1.531870] imx6q-pcie 33800000.pcie: IO
> 0x001ff80000..0x001ff8ffff -> 0x0000000000
> [ 1.544109] imx6q-pcie 33800000.pcie: MEM
> 0x0018000000..0x001fefffff -> 0x0018000000
> [ 1.767610] imx6q-pcie 33800000.pcie: iATU: unroll T, 4 ob, 4 ib,
> align 64K, limit 4G
> [ 1.875571] imx6q-pcie 33800000.pcie: PCIe Gen.1 x1 link up
> [ 1.881241] imx6q-pcie 33800000.pcie: PCIe Gen.1 x1 link up
> [ 1.886843] imx6q-pcie 33800000.pcie: Link up, Gen1
> [ 1.891746] imx6q-pcie 33800000.pcie: PCIe Gen.1 x1 link up
> [ 1.897674] imx6q-pcie 33800000.pcie: PCI host bridge to bus 0000:00
> [ 1.904078] pci_bus 0000:00: root bus resource [bus 00-ff]
> [ 1.909585] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
> [ 1.915784] pci_bus 0000:00: root bus resource [mem 0x18000000-0x1fefffff]
> [ 1.922706] pci 0000:00:00.0: [16c3:abcd] type 01 class 0x060400
> [ 1.928746] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
> [ 1.935043] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
> [ 1.941820] pci 0000:00:00.0: supports D1
> [ 1.945849] pci 0000:00:00.0: PME# supported from D0 D1 D3hot D3cold
> [ 1.954254] pci 0000:01:00.0: [168c:0036] type 00 class 0x028000
> [ 1.960394] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0007ffff 64bit]
> [ 1.967342] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
> [ 1.974389] pci 0000:01:00.0: supports D1 D2
> [ 1.978681] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
> [ 1.995483] pci 0000:00:00.0: BAR 0: assigned [mem 0x18000000-0x180fffff]
> [ 2.002334] pci 0000:00:00.0: BAR 14: assigned [mem 0x18100000-0x181fffff]
> [ 2.009240] pci 0000:00:00.0: BAR 15: assigned [mem
> 0x18200000-0x182fffff pref]
> [ 2.016576] pci 0000:00:00.0: BAR 6: assigned [mem
> 0x18300000-0x1830ffff pref]
> [ 2.023833] pci 0000:01:00.0: BAR 0: assigned [mem
> 0x18100000-0x1817ffff 64bit]
> [ 2.031214] pci 0000:01:00.0: BAR 6: assigned [mem
> 0x18200000-0x1820ffff pref]
> [ 2.038462] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> [ 2.043711] pci 0000:00:00.0: bridge window [mem 0x18100000-0x181fffff]
> [ 2.050518] pci 0000:00:00.0: bridge window [mem
> 0x18200000-0x182fffff pref]
> [ 2.058205] pcieport 0000:00:00.0: PME: Signaling with IRQ 205
> # dmesg | grep ath
> [ 2.064379] ath9k 0000:01:00.0: enabling device (0000 -> 0002)
> [ 2.070445] ath: phy0: WB335 2-ANT card detected
> [ 2.075092] ath: phy0: Set BT/WLAN RX diversity capability
> [ 2.088891] ath: phy0: Enable LNA combining
> [ 2.094278] ath: EEPROM regdomain: 0x6a
> [ 2.094285] ath: EEPROM indicates we should expect a direct regpair map
> [ 2.094293] ath: Country alpha2 being used: 00
> [ 2.094297] ath: Regpair used: 0x6a
> [ 14.228284] ath9k 0000:01:00.0 wlp1s0: renamed from wlan0
> # wpa_supplicant -i wlp1s0 -c wpa_supplicant.conf
> Successfully initialized wpa_supplicant
> <snip authentication details>
> # grep ath /proc/interrupts
> 204: 0 0 0 0 GICv3 157 Level ath9k
> # cat /proc/interrupts
> CPU0 CPU1 CPU2 CPU3
> 11: 2196 1753 1883 3509 GICv3 30 Level
> arch_timer
> 14: 0 0 0 0 GICv3 79 Level
> timer@306a0000
> 15: 0 0 0 0 GICv3 23 Level arm-pmu
> 16: 0 0 0 0 GICv3 135 Level
> 302c0000.dma-controller
> 17: 0 0 0 0 GICv3 66 Level
> 302b0000.dma-controller
> 18: 0 0 0 0 GICv3 34 Level
> 30bd0000.dma-controller
> 19: 0 0 0 0 GICv3 58 Level
> 30860000.serial
> 20: 0 0 0 0 GICv3 60 Level
> 30880000.serial
> 21: 503 0 0 0 GICv3 59 Level
> 30890000.serial
> 22: 0 0 0 0 GICv3 110 Level
> 30280000.watchdog
> 23: 418 0 0 0 GICv3 56 Level mmc2
> 41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
> 66: 0 0 0 0 gpio-mxc 6 Level
> gateworks-gsc
> 194: 750 0 0 0 GICv3 67 Level
> 30a20000.i2c
> 195: 0 0 0 0 gateworks-gsc 4
> Edge 0-0023
> 196: 25 0 0 0 GICv3 68 Level
> 30a30000.i2c
> 197: 0 0 0 0 GICv3 69 Level
> 30a40000.i2c
> 198: 0 0 0 0 GICv3 64 Level
> 30830000.spi
> 199: 0 0 0 0 GICv3 150 Level
> 30be0000.ethernet
> 200: 0 0 0 0 GICv3 151 Level
> 30be0000.ethernet
> 201: 0 0 0 0 GICv3 152 Level
> 30be0000.ethernet
> 202: 0 0 0 0 GICv3 153 Level
> 30be0000.ethernet
> 204: 0 0 0 0 GICv3 157 Level ath9k
> 205: 0 0 0 0 PCI-MSI 0 Edge
> PCIe PME
> 206: 0 0 0 0 GICv3 72 Level
> 32e40000.usb
> 207: 18 0 0 0 GICv3 73 Level
> 32e50000.usb
> 208: 0 0 0 0 gateworks-gsc 0
> Edge user_pb1x
> 209: 0 0 0 0 gateworks-gsc 1
> Edge key_erased
> 210: 0 0 0 0 gateworks-gsc 2
> Edge eeprom_wp
> 211: 0 0 0 0 gateworks-gsc 5
> Edge tamper
> 212: 0 0 0 0 gateworks-gsc 7
> Edge switch_hold
> 213: 0 0 0 0 0-0023 2 Edge user_pb
> IPI0: 620 751 665 109 Rescheduling interrupts
> IPI1: 361 1387 1285 509 Function call interrupts
> IPI2: 0 0 0 0 CPU stop interrupts
> IPI3: 0 0 0 0 CPU stop (for
> crash dump) interrupts
> IPI4: 0 0 0 0 Timer broadcast
> interrupts
> IPI5: 5 9 10 2 IRQ work interrupts
> IPI6: 0 0 0 0 CPU wake-up interrupts
> Err: 0
> ^^^ no interrupts from ath9k radio
> ^^^ is MSI enabled for the controller but ath9k electing to use legacy
> INT's a valid configuration?
>
> With MSI disabled globally via pci=nomsi ath9k gets interrupts and works fine:
> # cat /proc/cmdline
> pci=nomsi
> # dmesg | grep ath
> [ 2.052527] ath9k 0000:01:00.0: enabling device (0000 -> 0002)
> [ 2.058556] ath: phy0: WB335 2-ANT card detected
> [ 2.063199] ath: phy0: Set BT/WLAN RX diversity capability
> [ 2.077381] ath: phy0: Enable LNA combining
> [ 2.082799] ath: EEPROM regdomain: 0x6a
> [ 2.082805] ath: EEPROM indicates we should expect a direct regpair map
> [ 2.082813] ath: Country alpha2 being used: 00
> [ 2.082818] ath: Regpair used: 0x6a
> [ 11.620093] ath9k 0000:01:00.0 wlp1s0: renamed from wlan0
> # wpa_supplicant -i wlp1s0 -c wpa_supplicant.conf
> Successfully initialized wpa_supplicant
> <snip authentication details>
> #
> # grep ath /proc/interrupts
> 203: 379 0 0 0 GICv3 157 Level
> PCIe PME, ath9k
> # cat /proc/interrupts
> CPU0 CPU1 CPU2 CPU3
> 11: 1622 2152 3288 2005 GICv3 30 Level
> arch_timer
> 14: 0 0 0 0 GICv3 79 Level
> timer@306a0000
> 15: 0 0 0 0 GICv3 23 Level arm-pmu
> 16: 0 0 0 0 GICv3 135 Level
> 302c0000.dma-controller
> 17: 0 0 0 0 GICv3 66 Level
> 302b0000.dma-controller
> 18: 0 0 0 0 GICv3 34 Level
> 30bd0000.dma-controller
> 19: 0 0 0 0 GICv3 58 Level
> 30860000.serial
> 20: 0 0 0 0 GICv3 60 Level
> 30880000.serial
> 21: 597 0 0 0 GICv3 59 Level
> 30890000.serial
> 22: 0 0 0 0 GICv3 110 Level
> 30280000.watchdog
> 23: 414 0 0 0 GICv3 56 Level mmc2
> 41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
> 66: 0 0 0 0 gpio-mxc 6 Level
> gateworks-gsc
> 194: 978 0 0 0 GICv3 67 Level
> 30a20000.i2c
> 195: 0 0 0 0 gateworks-gsc 4
> Edge 0-0023
> 196: 25 0 0 0 GICv3 68 Level
> 30a30000.i2c
> 197: 0 0 0 0 GICv3 69 Level
> 30a40000.i2c
> 198: 0 0 0 0 GICv3 64 Level
> 30830000.spi
> 199: 0 0 0 0 GICv3 150 Level
> 30be0000.ethernet
> 200: 0 0 0 0 GICv3 151 Level
> 30be0000.ethernet
> 201: 0 0 0 0 GICv3 152 Level
> 30be0000.ethernet
> 202: 0 0 0 0 GICv3 153 Level
> 30be0000.ethernet
> 203: 379 0 0 0 GICv3 157 Level
> PCIe PME, ath9k
> 204: 0 0 0 0 GICv3 72 Level
> 32e40000.usb
> 205: 18 0 0 0 GICv3 73 Level
> 32e50000.usb
> 206: 0 0 0 0 gateworks-gsc 0
> Edge user_pb1x
> 207: 0 0 0 0 gateworks-gsc 1
> Edge key_erased
> 208: 0 0 0 0 gateworks-gsc 2
> Edge eeprom_wp
> 209: 0 0 0 0 gateworks-gsc 5
> Edge tamper
> 210: 0 0 0 0 gateworks-gsc 7
> Edge switch_hold
> 211: 0 0 0 0 0-0023 2 Edge user_pb
> IPI0: 501 506 73 759 Rescheduling interrupts
> IPI1: 878 846 718 1169 Function call interrupts
> IPI2: 0 0 0 0 CPU stop interrupts
> IPI3: 0 0 0 0 CPU stop (for
> crash dump) interrupts
> IPI4: 0 0 0 0 Timer broadcast
> interrupts
> IPI5: 5 4 9 20 IRQ work interrupts
> IPI6: 0 0 0 0 CPU wake-up interrupts
> Err: 0
> ^^^ legacy interrupts working as long as I disable MSI globally for IMX8MM
>
> And with MSI enabled globally as well as ath9k (ath9k.use_msi=1) ath9k
> fails to get interrupts and shows the interrupt error I ran into
> previously:
> # cat /proc/cmdline
> ath9k.use_msi=1
> # dmesg | grep ath
> [ 0.000000] Kernel command line: ath9k.use_msi=1
> [ 2.056381] ath9k 0000:01:00.0: enabling device (0000 -> 0002)
> [ 2.062571] ath9k 0000:01:00.0: Using MSI
> [ 2.066651] ath: phy0: WB335 2-ANT card detected
> [ 2.071288] ath: phy0: Set BT/WLAN RX diversity capability
> [ 2.085083] ath: phy0: Enable LNA combining
> [ 2.090470] ath: EEPROM regdomain: 0x6a
> [ 2.090476] ath: EEPROM indicates we should expect a direct regpair map
> [ 2.090484] ath: Country alpha2 being used: 00
> [ 2.090489] ath: Regpair used: 0x6a
> [ 13.999159] ath9k 0000:01:00.0 wlp1s0: renamed from wlan0
> # wpa_supplicant -i wlp1s0 -c wpa_supplicant.conf
> Successfully initialized wpa_supplicant
> [ 66.790178] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
> [ 66.798000] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
> [ 66.811551] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
> [ 66.937204] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
> [ 66.950679] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
> [ 67.020994] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> [ 67.092917] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> [ 67.164905] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> [ 67.236957] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> [ 67.308928] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> [ 67.380902] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> [ 67.452954] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> [ 67.524881] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> [ 67.596917] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> [ 67.668907] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> [ 67.740908] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> [ 67.860948] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> ...
> <snip authentication details>
> # grep ath /proc/interrupts
> 206: 0 0 0 0 PCI-MSI 524288 Edge
> ath9k
> # cat /proc/interrupts
> CPU0 CPU1 CPU2 CPU3
> 11: 1421 3067 1999 2767 GICv3 30 Level
> arch_timer
> 14: 0 0 0 0 GICv3 79 Level
> timer@306a0000
> 15: 0 0 0 0 GICv3 23 Level arm-pmu
> 16: 0 0 0 0 GICv3 135 Level
> 302c0000.dma-controller
> 17: 0 0 0 0 GICv3 66 Level
> 302b0000.dma-controller
> 18: 0 0 0 0 GICv3 34 Level
> 30bd0000.dma-controller
> 19: 0 0 0 0 GICv3 58 Level
> 30860000.serial
> 20: 0 0 0 0 GICv3 60 Level
> 30880000.serial
> 21: 289 0 0 0 GICv3 59 Level
> 30890000.serial
> 22: 0 0 0 0 GICv3 110 Level
> 30280000.watchdog
> 23: 390 0 0 0 GICv3 56 Level mmc2
> 41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
> 66: 0 0 0 0 gpio-mxc 6 Level
> gateworks-gsc
> 194: 1054 0 0 0 GICv3 67 Level
> 30a20000.i2c
> 195: 0 0 0 0 gateworks-gsc 4
> Edge 0-0023
> 196: 25 0 0 0 GICv3 68 Level
> 30a30000.i2c
> 197: 0 0 0 0 GICv3 69 Level
> 30a40000.i2c
> 198: 0 0 0 0 GICv3 64 Level
> 30830000.spi
> 199: 0 0 0 0 GICv3 150 Level
> 30be0000.ethernet
> 200: 0 0 0 0 GICv3 151 Level
> 30be0000.ethernet
> 201: 0 0 0 0 GICv3 152 Level
> 30be0000.ethernet
> 202: 0 0 0 0 GICv3 153 Level
> 30be0000.ethernet
> 205: 6891 0 0 0 PCI-MSI 0 Edge
> PCIe PME
> 206: 0 0 0 0 PCI-MSI 524288 Edge
> ath9k
> 207: 0 0 0 0 GICv3 72 Level
> 32e40000.usb
> 208: 18 0 0 0 GICv3 73 Level
> 32e50000.usb
> 209: 0 0 0 0 gateworks-gsc 0
> Edge user_pb1x
> 210: 0 0 0 0 gateworks-gsc 1
> Edge key_erased
> 211: 0 0 0 0 gateworks-gsc 2
> Edge eeprom_wp
> 212: 0 0 0 0 gateworks-gsc 5
> Edge tamper
> 213: 0 0 0 0 gateworks-gsc 7
> Edge switch_hold
> 214: 0 0 0 0 0-0023 2 Edge user_pb
> IPI0: 646 64 692 808 Rescheduling interrupts
> IPI1: 1211 893 503 1097 Function call interrupts
> IPI2: 0 0 0 0 CPU stop interrupts
> IPI3: 0 0 0 0 CPU stop (for
> crash dump) interrupts
> IPI4: 0 0 0 0 Timer broadcast
> interrupts
> IPI5: 9 13 12 8 IRQ work interrupts
> IPI6: 0 0 0 0 CPU wake-up interrupts
> Err: 0
> ^^^ It looks to me like when using MSI above the interrupts are going
> to the PME driver MSI 0 and the ath9k MSI interrupt shows as 524288
> which aligns to 4 but seems strange that its so large.
>
> Does this jive with what you encountered?
>
> Note that ath10k which require MSI interrupts are working fine on this
> platform and the MSI interrupt is also 524288:
> 205: 0 0 0 0 PCI-MSI 0 Edge
> PCIe PME
> 206: 632 0 0 0 PCI-MSI 524288 Edge
> ath10k_pci
>
> Note that ath11k which requires MSI interrupts is also working fine on
> this board:
> 205: 0 0 0 0 PCI-MSI 0 Edge
> PCIe PME
> 206: 4 0 0 0 PCI-MSI 524288 Edge bhi
> 207: 4 0 0 0 PCI-MSI 524289 Edge mhi
> 208: 39 0 0 0 PCI-MSI 524290 Edge mhi
> 209: 3 0 0 0 PCI-MSI 524291 Edge ce0
> 210: 26 0 0 1 PCI-MSI 524292 Edge ce1
> 211: 267 0 0 1 PCI-MSI 524293 Edge ce2
> 212: 88 0 0 0 PCI-MSI 524294 Edge ce3
> 213: 0 0 0 1 PCI-MSI 524295 Edge ce5
> 214: 2 0 0 0 PCI-MSI 524296 Edge
> DP_EXT_IRQ
> 215: 0 0 0 0 PCI-MSI 524297 Edge
> DP_EXT_IRQ
> 216: 0 0 0 0 PCI-MSI 524298 Edge
> DP_EXT_IRQ
> 217: 15 0 0 0 PCI-MSI 524299 Edge
> DP_EXT_IRQ
> 218: 0 0 0 0 PCI-MSI 524300 Edge
> DP_EXT_IRQ
> 219: 2 0 0 0 PCI-MSI 524301 Edge
> DP_EXT_IRQ
> 220: 0 0 0 0 PCI-MSI 524302 Edge
> DP_EXT_IRQ
> 221: 0 0 0 0 PCI-MSI 524303 Edge
> DP_EXT_IRQ
>
> Now, if I move back to an imx8mm-venice-gw73xx-0x which has an IMX8MM
> and a PCI bridge I can't even get the legacy IRQ configuration to work
> (no interrupts):
> # cat /proc/cmdline
> pci=nomsi
> # lspci
> 00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01)
> 01:00.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
> 02:01.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
> 02:02.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
> 02:03.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
> 04:00.0 Network controller: Qualcomm Atheros QCA9565 / AR9565 Wireless
> Network Adapter (rev 01)
> 05:00.0 Ethernet controller: Microchip Technology / SMSC Device 7430 (rev 11)
> # dmesg | grep ath
> [ 2.475134] ath9k 0000:04:00.0: enabling device (0000 -> 0002)
> [ 2.481157] ath: phy0: WB335 2-ANT card detected
> [ 2.485797] ath: phy0: Set BT/WLAN RX diversity capability
> [ 2.499443] ath: phy0: Enable LNA combining
> [ 2.504837] ath: EEPROM regdomain: 0x6a
> [ 2.504845] ath: EEPROM indicates we should expect a direct regpair map
> [ 2.504853] ath: Country alpha2 being used: 00
> [ 2.504858] ath: Regpair used: 0x6a
> [ 10.237297] ath9k 0000:04:00.0 wlp4s0: renamed from wlan0
> # wpa_supplicant -i wlp4s0 -c wpa_supplicant.conf
> <snip failed auth>
> # grep -i pci /proc/interrupts
> 204: 0 0 0 0 GICv3 157 Level
> PCIe PME, ath9k
> # cat /proc/interrupts
> CPU0 CPU1 CPU2 CPU3
> 11: 1550 3047 2199 4301 GICv3 30 Level
> arch_timer
> 14: 0 0 0 0 GICv3 79 Level
> timer@306a0000
> 15: 0 0 0 0 GICv3 23 Level arm-pmu
> 16: 0 0 0 0 GICv3 135 Level
> 302c0000.dma-controller
> 17: 0 0 0 0 GICv3 66 Level
> 302b0000.dma-controller
> 18: 332 0 0 0 GICv3 34 Level
> 30bd0000.dma-controller
> 19: 0 0 0 0 GICv3 58 Level
> 30860000.serial
> 20: 773 0 0 0 GICv3 59 Level
> 30890000.serial
> 21: 0 0 0 0 GICv3 61 Level
> 30a60000.serial
> 22: 0 0 0 0 GICv3 110 Level
> 30280000.watchdog
> 23: 433 0 0 0 GICv3 56 Level mmc2
> 41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
> 66: 0 0 0 0 gpio-mxc 6 Level
> gateworks-gsc
> 72: 0 0 0 0 gpio-mxc 12 Edge
> 30b50000.mmc cd
> 170: 4 0 0 0 gpio-mxc 8 Edge
> 30880000.serial
> 194: 1282 0 0 0 GICv3 67 Level
> 30a20000.i2c
> 195: 0 0 0 0 gateworks-gsc 4
> Edge 0-0023
> 196: 25 0 0 0 GICv3 68 Level
> 30a30000.i2c
> 197: 0 0 0 0 GICv3 69 Level
> 30a40000.i2c
> 198: 0 0 0 0 GICv3 60 Level
> 30880000.serial
> 199: 0 0 0 0 GICv3 64 Level
> 30830000.spi
> 200: 0 0 0 0 GICv3 150 Level
> 30be0000.ethernet
> 201: 0 0 0 0 GICv3 151 Level
> 30be0000.ethernet
> 202: 0 0 0 0 GICv3 152 Level
> 30be0000.ethernet
> 203: 0 0 0 0 GICv3 153 Level
> 30be0000.ethernet
> 204: 0 0 0 0 GICv3 157 Level
> PCIe PME, ath9k
> 205: 0 0 0 0 GICv3 72 Level
> 32e40000.usb
> 206: 105 0 0 0 GICv3 73 Level
> 32e50000.usb
> 207: 6244 0 0 0 GICv3 54 Level mmc0
> 208: 0 0 0 0 GICv3 55 Level mmc1
> 209: 0 0 0 0 gateworks-gsc 0
> Edge user_pb1x
> 210: 0 0 0 0 gateworks-gsc 1
> Edge key_erased
> 211: 0 0 0 0 gateworks-gsc 2
> Edge eeprom_wp
> 212: 0 0 0 0 gateworks-gsc 5
> Edge tamper
> 213: 0 0 0 0 gateworks-gsc 7
> Edge switch_hold
> 214: 0 0 0 0 0-0023 2 Edge user_pb
> IPI0: 367 486 444 286 Rescheduling interrupts
> IPI1: 972 1283 1202 1630 Function call interrupts
> IPI2: 0 0 0 0 CPU stop interrupts
> IPI3: 0 0 0 0 CPU stop (for
> crash dump) interrupts
> IPI4: 0 0 0 0 Timer broadcast
> interrupts
> IPI5: 5 6 7 36 IRQ work interrupts
> IPI6: 0 0 0 0 CPU wake-up interrupts
> Err: 0
> ^^^ why no interrupts on this board when disabling MSI globally and
> using ath9k legacy interrupts.... something to do with the bridge?
>
> Best Regards,
>
> Tim
>
>
>
> On Mon, Mar 20, 2023 at 4:29 AM Daniel Drake <[email protected]> wrote:
> >
> > On Sat, Mar 18, 2023 at 8:17 AM Tim Harvey <[email protected]> wrote:
> > > It seems that Daniel may have been running into something similar back
> > > when commit 7368160f0ab0 ("ath9k: add MSI support") was merged
> > > [1][2][3]
> > >
> > > Does anyone have any advice here about what can be going on?
> >
> > The issue we found earlier was that ath9k MSI is broken unless the MSI
> > IRQ number coincidentally is aligned to 4.
> >
> > Worked around with:
> > https://github.com/endlessm/linux/commit/0a2d86a2b9da8f936d873addf0a97e341177fe8a
> > https://github.com/endlessm/linux/commit/970d2b4b59465cd926e5213e0feb633f6b623447
> >
> > IIRC the PCI patch wasn't accepted upstream at the time and I never
> > got any response from Atheros/Qualcomm about this device
> > hardware/firmware bug.
> >
> > Daniel
On Tue, Mar 21, 2023 at 10:55 AM Tim Harvey <[email protected]> wrote:
>
> On Mon, Mar 20, 2023 at 1:46 PM Tim Harvey <[email protected]> wrote:
> >
> > (+linux-pci, Fabio, Richard)
> >
> > Daniel,
> >
> > I may have two different issues going on here:
> > 1) issues with ath9k MSI interrupts
> > 2) issues with ath9k legacy interrupts on IMX8MM
>
> Lucas,
>
> I've found an old thread [1] where you state that Designware PCIe core
> (not just IMX6) has an issue where as soon as any MSI interrupt is
> enabled the core does not forward legacy IRQs anymore so if any card
> in your system needs legacy interrupts you must disable MSI
> completely.
>
> I assume this is still the understanding and also applies then to
> IMX8MM/IMX8MP? This would explain why I'm unable to get legacy
> interrupts working without pci=nomsi as the PME driver requests an MSI
> interrupt.
>
> Do you know any reason why I can get legacy IRQ's to work with
> pci=nomsi on IMX6Q/DL with and without a bridge but on IMX8MM I can
> only get it to work without a bridge?
>
> Best Regards,
>
> Tim
> [1] https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/#22345435
>
Lucas,
Do you have any ideas regarding why imx8mm/imx8mp legacy interrupts
with pci=nomsi won't work through a bridge?
Best Regards,
Tim
> >
> > To rule out the PCIe switch being an issue for me I moved to an
> > imx8mm-venice-gw71xx-0x which has an IMX8MM and no PCIe switch.
> >
> > I've found that with MSI enabled globally, but ath9k not using MSI I
> > fail to get any interrupts from the radio with nothing abnormal from
> > dmesg:
> > # uname -r
> > 6.2.0-09243-ge911eb7b8ec1
> > # cat /proc/cmdline
> >
> > # lspci -n
> > 00:00.0 0604: 16c3:abcd (rev 01)
> > 01:00.0 0280: 168c:0036 (rev 01)
> > # dmesg | grep pci
> > [ 1.524560] imx6q-pcie 33800000.pcie: host bridge
> > /soc@0/pcie@33800000 ranges:
> > [ 1.531870] imx6q-pcie 33800000.pcie: IO
> > 0x001ff80000..0x001ff8ffff -> 0x0000000000
> > [ 1.544109] imx6q-pcie 33800000.pcie: MEM
> > 0x0018000000..0x001fefffff -> 0x0018000000
> > [ 1.767610] imx6q-pcie 33800000.pcie: iATU: unroll T, 4 ob, 4 ib,
> > align 64K, limit 4G
> > [ 1.875571] imx6q-pcie 33800000.pcie: PCIe Gen.1 x1 link up
> > [ 1.881241] imx6q-pcie 33800000.pcie: PCIe Gen.1 x1 link up
> > [ 1.886843] imx6q-pcie 33800000.pcie: Link up, Gen1
> > [ 1.891746] imx6q-pcie 33800000.pcie: PCIe Gen.1 x1 link up
> > [ 1.897674] imx6q-pcie 33800000.pcie: PCI host bridge to bus 0000:00
> > [ 1.904078] pci_bus 0000:00: root bus resource [bus 00-ff]
> > [ 1.909585] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
> > [ 1.915784] pci_bus 0000:00: root bus resource [mem 0x18000000-0x1fefffff]
> > [ 1.922706] pci 0000:00:00.0: [16c3:abcd] type 01 class 0x060400
> > [ 1.928746] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
> > [ 1.935043] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
> > [ 1.941820] pci 0000:00:00.0: supports D1
> > [ 1.945849] pci 0000:00:00.0: PME# supported from D0 D1 D3hot D3cold
> > [ 1.954254] pci 0000:01:00.0: [168c:0036] type 00 class 0x028000
> > [ 1.960394] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0007ffff 64bit]
> > [ 1.967342] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
> > [ 1.974389] pci 0000:01:00.0: supports D1 D2
> > [ 1.978681] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
> > [ 1.995483] pci 0000:00:00.0: BAR 0: assigned [mem 0x18000000-0x180fffff]
> > [ 2.002334] pci 0000:00:00.0: BAR 14: assigned [mem 0x18100000-0x181fffff]
> > [ 2.009240] pci 0000:00:00.0: BAR 15: assigned [mem
> > 0x18200000-0x182fffff pref]
> > [ 2.016576] pci 0000:00:00.0: BAR 6: assigned [mem
> > 0x18300000-0x1830ffff pref]
> > [ 2.023833] pci 0000:01:00.0: BAR 0: assigned [mem
> > 0x18100000-0x1817ffff 64bit]
> > [ 2.031214] pci 0000:01:00.0: BAR 6: assigned [mem
> > 0x18200000-0x1820ffff pref]
> > [ 2.038462] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> > [ 2.043711] pci 0000:00:00.0: bridge window [mem 0x18100000-0x181fffff]
> > [ 2.050518] pci 0000:00:00.0: bridge window [mem
> > 0x18200000-0x182fffff pref]
> > [ 2.058205] pcieport 0000:00:00.0: PME: Signaling with IRQ 205
> > # dmesg | grep ath
> > [ 2.064379] ath9k 0000:01:00.0: enabling device (0000 -> 0002)
> > [ 2.070445] ath: phy0: WB335 2-ANT card detected
> > [ 2.075092] ath: phy0: Set BT/WLAN RX diversity capability
> > [ 2.088891] ath: phy0: Enable LNA combining
> > [ 2.094278] ath: EEPROM regdomain: 0x6a
> > [ 2.094285] ath: EEPROM indicates we should expect a direct regpair map
> > [ 2.094293] ath: Country alpha2 being used: 00
> > [ 2.094297] ath: Regpair used: 0x6a
> > [ 14.228284] ath9k 0000:01:00.0 wlp1s0: renamed from wlan0
> > # wpa_supplicant -i wlp1s0 -c wpa_supplicant.conf
> > Successfully initialized wpa_supplicant
> > <snip authentication details>
> > # grep ath /proc/interrupts
> > 204: 0 0 0 0 GICv3 157 Level ath9k
> > # cat /proc/interrupts
> > CPU0 CPU1 CPU2 CPU3
> > 11: 2196 1753 1883 3509 GICv3 30 Level
> > arch_timer
> > 14: 0 0 0 0 GICv3 79 Level
> > timer@306a0000
> > 15: 0 0 0 0 GICv3 23 Level arm-pmu
> > 16: 0 0 0 0 GICv3 135 Level
> > 302c0000.dma-controller
> > 17: 0 0 0 0 GICv3 66 Level
> > 302b0000.dma-controller
> > 18: 0 0 0 0 GICv3 34 Level
> > 30bd0000.dma-controller
> > 19: 0 0 0 0 GICv3 58 Level
> > 30860000.serial
> > 20: 0 0 0 0 GICv3 60 Level
> > 30880000.serial
> > 21: 503 0 0 0 GICv3 59 Level
> > 30890000.serial
> > 22: 0 0 0 0 GICv3 110 Level
> > 30280000.watchdog
> > 23: 418 0 0 0 GICv3 56 Level mmc2
> > 41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
> > 66: 0 0 0 0 gpio-mxc 6 Level
> > gateworks-gsc
> > 194: 750 0 0 0 GICv3 67 Level
> > 30a20000.i2c
> > 195: 0 0 0 0 gateworks-gsc 4
> > Edge 0-0023
> > 196: 25 0 0 0 GICv3 68 Level
> > 30a30000.i2c
> > 197: 0 0 0 0 GICv3 69 Level
> > 30a40000.i2c
> > 198: 0 0 0 0 GICv3 64 Level
> > 30830000.spi
> > 199: 0 0 0 0 GICv3 150 Level
> > 30be0000.ethernet
> > 200: 0 0 0 0 GICv3 151 Level
> > 30be0000.ethernet
> > 201: 0 0 0 0 GICv3 152 Level
> > 30be0000.ethernet
> > 202: 0 0 0 0 GICv3 153 Level
> > 30be0000.ethernet
> > 204: 0 0 0 0 GICv3 157 Level ath9k
> > 205: 0 0 0 0 PCI-MSI 0 Edge
> > PCIe PME
> > 206: 0 0 0 0 GICv3 72 Level
> > 32e40000.usb
> > 207: 18 0 0 0 GICv3 73 Level
> > 32e50000.usb
> > 208: 0 0 0 0 gateworks-gsc 0
> > Edge user_pb1x
> > 209: 0 0 0 0 gateworks-gsc 1
> > Edge key_erased
> > 210: 0 0 0 0 gateworks-gsc 2
> > Edge eeprom_wp
> > 211: 0 0 0 0 gateworks-gsc 5
> > Edge tamper
> > 212: 0 0 0 0 gateworks-gsc 7
> > Edge switch_hold
> > 213: 0 0 0 0 0-0023 2 Edge user_pb
> > IPI0: 620 751 665 109 Rescheduling interrupts
> > IPI1: 361 1387 1285 509 Function call interrupts
> > IPI2: 0 0 0 0 CPU stop interrupts
> > IPI3: 0 0 0 0 CPU stop (for
> > crash dump) interrupts
> > IPI4: 0 0 0 0 Timer broadcast
> > interrupts
> > IPI5: 5 9 10 2 IRQ work interrupts
> > IPI6: 0 0 0 0 CPU wake-up interrupts
> > Err: 0
> > ^^^ no interrupts from ath9k radio
> > ^^^ is MSI enabled for the controller but ath9k electing to use legacy
> > INT's a valid configuration?
> >
> > With MSI disabled globally via pci=nomsi ath9k gets interrupts and works fine:
> > # cat /proc/cmdline
> > pci=nomsi
> > # dmesg | grep ath
> > [ 2.052527] ath9k 0000:01:00.0: enabling device (0000 -> 0002)
> > [ 2.058556] ath: phy0: WB335 2-ANT card detected
> > [ 2.063199] ath: phy0: Set BT/WLAN RX diversity capability
> > [ 2.077381] ath: phy0: Enable LNA combining
> > [ 2.082799] ath: EEPROM regdomain: 0x6a
> > [ 2.082805] ath: EEPROM indicates we should expect a direct regpair map
> > [ 2.082813] ath: Country alpha2 being used: 00
> > [ 2.082818] ath: Regpair used: 0x6a
> > [ 11.620093] ath9k 0000:01:00.0 wlp1s0: renamed from wlan0
> > # wpa_supplicant -i wlp1s0 -c wpa_supplicant.conf
> > Successfully initialized wpa_supplicant
> > <snip authentication details>
> > #
> > # grep ath /proc/interrupts
> > 203: 379 0 0 0 GICv3 157 Level
> > PCIe PME, ath9k
> > # cat /proc/interrupts
> > CPU0 CPU1 CPU2 CPU3
> > 11: 1622 2152 3288 2005 GICv3 30 Level
> > arch_timer
> > 14: 0 0 0 0 GICv3 79 Level
> > timer@306a0000
> > 15: 0 0 0 0 GICv3 23 Level arm-pmu
> > 16: 0 0 0 0 GICv3 135 Level
> > 302c0000.dma-controller
> > 17: 0 0 0 0 GICv3 66 Level
> > 302b0000.dma-controller
> > 18: 0 0 0 0 GICv3 34 Level
> > 30bd0000.dma-controller
> > 19: 0 0 0 0 GICv3 58 Level
> > 30860000.serial
> > 20: 0 0 0 0 GICv3 60 Level
> > 30880000.serial
> > 21: 597 0 0 0 GICv3 59 Level
> > 30890000.serial
> > 22: 0 0 0 0 GICv3 110 Level
> > 30280000.watchdog
> > 23: 414 0 0 0 GICv3 56 Level mmc2
> > 41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
> > 66: 0 0 0 0 gpio-mxc 6 Level
> > gateworks-gsc
> > 194: 978 0 0 0 GICv3 67 Level
> > 30a20000.i2c
> > 195: 0 0 0 0 gateworks-gsc 4
> > Edge 0-0023
> > 196: 25 0 0 0 GICv3 68 Level
> > 30a30000.i2c
> > 197: 0 0 0 0 GICv3 69 Level
> > 30a40000.i2c
> > 198: 0 0 0 0 GICv3 64 Level
> > 30830000.spi
> > 199: 0 0 0 0 GICv3 150 Level
> > 30be0000.ethernet
> > 200: 0 0 0 0 GICv3 151 Level
> > 30be0000.ethernet
> > 201: 0 0 0 0 GICv3 152 Level
> > 30be0000.ethernet
> > 202: 0 0 0 0 GICv3 153 Level
> > 30be0000.ethernet
> > 203: 379 0 0 0 GICv3 157 Level
> > PCIe PME, ath9k
> > 204: 0 0 0 0 GICv3 72 Level
> > 32e40000.usb
> > 205: 18 0 0 0 GICv3 73 Level
> > 32e50000.usb
> > 206: 0 0 0 0 gateworks-gsc 0
> > Edge user_pb1x
> > 207: 0 0 0 0 gateworks-gsc 1
> > Edge key_erased
> > 208: 0 0 0 0 gateworks-gsc 2
> > Edge eeprom_wp
> > 209: 0 0 0 0 gateworks-gsc 5
> > Edge tamper
> > 210: 0 0 0 0 gateworks-gsc 7
> > Edge switch_hold
> > 211: 0 0 0 0 0-0023 2 Edge user_pb
> > IPI0: 501 506 73 759 Rescheduling interrupts
> > IPI1: 878 846 718 1169 Function call interrupts
> > IPI2: 0 0 0 0 CPU stop interrupts
> > IPI3: 0 0 0 0 CPU stop (for
> > crash dump) interrupts
> > IPI4: 0 0 0 0 Timer broadcast
> > interrupts
> > IPI5: 5 4 9 20 IRQ work interrupts
> > IPI6: 0 0 0 0 CPU wake-up interrupts
> > Err: 0
> > ^^^ legacy interrupts working as long as I disable MSI globally for IMX8MM
> >
> > And with MSI enabled globally as well as ath9k (ath9k.use_msi=1) ath9k
> > fails to get interrupts and shows the interrupt error I ran into
> > previously:
> > # cat /proc/cmdline
> > ath9k.use_msi=1
> > # dmesg | grep ath
> > [ 0.000000] Kernel command line: ath9k.use_msi=1
> > [ 2.056381] ath9k 0000:01:00.0: enabling device (0000 -> 0002)
> > [ 2.062571] ath9k 0000:01:00.0: Using MSI
> > [ 2.066651] ath: phy0: WB335 2-ANT card detected
> > [ 2.071288] ath: phy0: Set BT/WLAN RX diversity capability
> > [ 2.085083] ath: phy0: Enable LNA combining
> > [ 2.090470] ath: EEPROM regdomain: 0x6a
> > [ 2.090476] ath: EEPROM indicates we should expect a direct regpair map
> > [ 2.090484] ath: Country alpha2 being used: 00
> > [ 2.090489] ath: Regpair used: 0x6a
> > [ 13.999159] ath9k 0000:01:00.0 wlp1s0: renamed from wlan0
> > # wpa_supplicant -i wlp1s0 -c wpa_supplicant.conf
> > Successfully initialized wpa_supplicant
> > [ 66.790178] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
> > [ 66.798000] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
> > [ 66.811551] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
> > [ 66.937204] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
> > [ 66.950679] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0x6A0C9C8
> > [ 67.020994] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > [ 67.092917] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > [ 67.164905] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > [ 67.236957] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > [ 67.308928] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > [ 67.380902] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > [ 67.452954] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > [ 67.524881] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > [ 67.596917] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > [ 67.668907] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > [ 67.740908] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > [ 67.860948] ath: phy0: __ath9k_hw_enable_interrupts: _msi_reg = 0xEA0C9C8
> > ...
> > <snip authentication details>
> > # grep ath /proc/interrupts
> > 206: 0 0 0 0 PCI-MSI 524288 Edge
> > ath9k
> > # cat /proc/interrupts
> > CPU0 CPU1 CPU2 CPU3
> > 11: 1421 3067 1999 2767 GICv3 30 Level
> > arch_timer
> > 14: 0 0 0 0 GICv3 79 Level
> > timer@306a0000
> > 15: 0 0 0 0 GICv3 23 Level arm-pmu
> > 16: 0 0 0 0 GICv3 135 Level
> > 302c0000.dma-controller
> > 17: 0 0 0 0 GICv3 66 Level
> > 302b0000.dma-controller
> > 18: 0 0 0 0 GICv3 34 Level
> > 30bd0000.dma-controller
> > 19: 0 0 0 0 GICv3 58 Level
> > 30860000.serial
> > 20: 0 0 0 0 GICv3 60 Level
> > 30880000.serial
> > 21: 289 0 0 0 GICv3 59 Level
> > 30890000.serial
> > 22: 0 0 0 0 GICv3 110 Level
> > 30280000.watchdog
> > 23: 390 0 0 0 GICv3 56 Level mmc2
> > 41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
> > 66: 0 0 0 0 gpio-mxc 6 Level
> > gateworks-gsc
> > 194: 1054 0 0 0 GICv3 67 Level
> > 30a20000.i2c
> > 195: 0 0 0 0 gateworks-gsc 4
> > Edge 0-0023
> > 196: 25 0 0 0 GICv3 68 Level
> > 30a30000.i2c
> > 197: 0 0 0 0 GICv3 69 Level
> > 30a40000.i2c
> > 198: 0 0 0 0 GICv3 64 Level
> > 30830000.spi
> > 199: 0 0 0 0 GICv3 150 Level
> > 30be0000.ethernet
> > 200: 0 0 0 0 GICv3 151 Level
> > 30be0000.ethernet
> > 201: 0 0 0 0 GICv3 152 Level
> > 30be0000.ethernet
> > 202: 0 0 0 0 GICv3 153 Level
> > 30be0000.ethernet
> > 205: 6891 0 0 0 PCI-MSI 0 Edge
> > PCIe PME
> > 206: 0 0 0 0 PCI-MSI 524288 Edge
> > ath9k
> > 207: 0 0 0 0 GICv3 72 Level
> > 32e40000.usb
> > 208: 18 0 0 0 GICv3 73 Level
> > 32e50000.usb
> > 209: 0 0 0 0 gateworks-gsc 0
> > Edge user_pb1x
> > 210: 0 0 0 0 gateworks-gsc 1
> > Edge key_erased
> > 211: 0 0 0 0 gateworks-gsc 2
> > Edge eeprom_wp
> > 212: 0 0 0 0 gateworks-gsc 5
> > Edge tamper
> > 213: 0 0 0 0 gateworks-gsc 7
> > Edge switch_hold
> > 214: 0 0 0 0 0-0023 2 Edge user_pb
> > IPI0: 646 64 692 808 Rescheduling interrupts
> > IPI1: 1211 893 503 1097 Function call interrupts
> > IPI2: 0 0 0 0 CPU stop interrupts
> > IPI3: 0 0 0 0 CPU stop (for
> > crash dump) interrupts
> > IPI4: 0 0 0 0 Timer broadcast
> > interrupts
> > IPI5: 9 13 12 8 IRQ work interrupts
> > IPI6: 0 0 0 0 CPU wake-up interrupts
> > Err: 0
> > ^^^ It looks to me like when using MSI above the interrupts are going
> > to the PME driver MSI 0 and the ath9k MSI interrupt shows as 524288
> > which aligns to 4 but seems strange that its so large.
> >
> > Does this jive with what you encountered?
> >
> > Note that ath10k which require MSI interrupts are working fine on this
> > platform and the MSI interrupt is also 524288:
> > 205: 0 0 0 0 PCI-MSI 0 Edge
> > PCIe PME
> > 206: 632 0 0 0 PCI-MSI 524288 Edge
> > ath10k_pci
> >
> > Note that ath11k which requires MSI interrupts is also working fine on
> > this board:
> > 205: 0 0 0 0 PCI-MSI 0 Edge
> > PCIe PME
> > 206: 4 0 0 0 PCI-MSI 524288 Edge bhi
> > 207: 4 0 0 0 PCI-MSI 524289 Edge mhi
> > 208: 39 0 0 0 PCI-MSI 524290 Edge mhi
> > 209: 3 0 0 0 PCI-MSI 524291 Edge ce0
> > 210: 26 0 0 1 PCI-MSI 524292 Edge ce1
> > 211: 267 0 0 1 PCI-MSI 524293 Edge ce2
> > 212: 88 0 0 0 PCI-MSI 524294 Edge ce3
> > 213: 0 0 0 1 PCI-MSI 524295 Edge ce5
> > 214: 2 0 0 0 PCI-MSI 524296 Edge
> > DP_EXT_IRQ
> > 215: 0 0 0 0 PCI-MSI 524297 Edge
> > DP_EXT_IRQ
> > 216: 0 0 0 0 PCI-MSI 524298 Edge
> > DP_EXT_IRQ
> > 217: 15 0 0 0 PCI-MSI 524299 Edge
> > DP_EXT_IRQ
> > 218: 0 0 0 0 PCI-MSI 524300 Edge
> > DP_EXT_IRQ
> > 219: 2 0 0 0 PCI-MSI 524301 Edge
> > DP_EXT_IRQ
> > 220: 0 0 0 0 PCI-MSI 524302 Edge
> > DP_EXT_IRQ
> > 221: 0 0 0 0 PCI-MSI 524303 Edge
> > DP_EXT_IRQ
> >
> > Now, if I move back to an imx8mm-venice-gw73xx-0x which has an IMX8MM
> > and a PCI bridge I can't even get the legacy IRQ configuration to work
> > (no interrupts):
> > # cat /proc/cmdline
> > pci=nomsi
> > # lspci
> > 00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01)
> > 01:00.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
> > 02:01.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
> > 02:02.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
> > 02:03.0 PCI bridge: Pericom Semiconductor Device b404 (rev 01)
> > 04:00.0 Network controller: Qualcomm Atheros QCA9565 / AR9565 Wireless
> > Network Adapter (rev 01)
> > 05:00.0 Ethernet controller: Microchip Technology / SMSC Device 7430 (rev 11)
> > # dmesg | grep ath
> > [ 2.475134] ath9k 0000:04:00.0: enabling device (0000 -> 0002)
> > [ 2.481157] ath: phy0: WB335 2-ANT card detected
> > [ 2.485797] ath: phy0: Set BT/WLAN RX diversity capability
> > [ 2.499443] ath: phy0: Enable LNA combining
> > [ 2.504837] ath: EEPROM regdomain: 0x6a
> > [ 2.504845] ath: EEPROM indicates we should expect a direct regpair map
> > [ 2.504853] ath: Country alpha2 being used: 00
> > [ 2.504858] ath: Regpair used: 0x6a
> > [ 10.237297] ath9k 0000:04:00.0 wlp4s0: renamed from wlan0
> > # wpa_supplicant -i wlp4s0 -c wpa_supplicant.conf
> > <snip failed auth>
> > # grep -i pci /proc/interrupts
> > 204: 0 0 0 0 GICv3 157 Level
> > PCIe PME, ath9k
> > # cat /proc/interrupts
> > CPU0 CPU1 CPU2 CPU3
> > 11: 1550 3047 2199 4301 GICv3 30 Level
> > arch_timer
> > 14: 0 0 0 0 GICv3 79 Level
> > timer@306a0000
> > 15: 0 0 0 0 GICv3 23 Level arm-pmu
> > 16: 0 0 0 0 GICv3 135 Level
> > 302c0000.dma-controller
> > 17: 0 0 0 0 GICv3 66 Level
> > 302b0000.dma-controller
> > 18: 332 0 0 0 GICv3 34 Level
> > 30bd0000.dma-controller
> > 19: 0 0 0 0 GICv3 58 Level
> > 30860000.serial
> > 20: 773 0 0 0 GICv3 59 Level
> > 30890000.serial
> > 21: 0 0 0 0 GICv3 61 Level
> > 30a60000.serial
> > 22: 0 0 0 0 GICv3 110 Level
> > 30280000.watchdog
> > 23: 433 0 0 0 GICv3 56 Level mmc2
> > 41: 0 0 0 0 gpio-mxc 15 Edge pps.-1
> > 66: 0 0 0 0 gpio-mxc 6 Level
> > gateworks-gsc
> > 72: 0 0 0 0 gpio-mxc 12 Edge
> > 30b50000.mmc cd
> > 170: 4 0 0 0 gpio-mxc 8 Edge
> > 30880000.serial
> > 194: 1282 0 0 0 GICv3 67 Level
> > 30a20000.i2c
> > 195: 0 0 0 0 gateworks-gsc 4
> > Edge 0-0023
> > 196: 25 0 0 0 GICv3 68 Level
> > 30a30000.i2c
> > 197: 0 0 0 0 GICv3 69 Level
> > 30a40000.i2c
> > 198: 0 0 0 0 GICv3 60 Level
> > 30880000.serial
> > 199: 0 0 0 0 GICv3 64 Level
> > 30830000.spi
> > 200: 0 0 0 0 GICv3 150 Level
> > 30be0000.ethernet
> > 201: 0 0 0 0 GICv3 151 Level
> > 30be0000.ethernet
> > 202: 0 0 0 0 GICv3 152 Level
> > 30be0000.ethernet
> > 203: 0 0 0 0 GICv3 153 Level
> > 30be0000.ethernet
> > 204: 0 0 0 0 GICv3 157 Level
> > PCIe PME, ath9k
> > 205: 0 0 0 0 GICv3 72 Level
> > 32e40000.usb
> > 206: 105 0 0 0 GICv3 73 Level
> > 32e50000.usb
> > 207: 6244 0 0 0 GICv3 54 Level mmc0
> > 208: 0 0 0 0 GICv3 55 Level mmc1
> > 209: 0 0 0 0 gateworks-gsc 0
> > Edge user_pb1x
> > 210: 0 0 0 0 gateworks-gsc 1
> > Edge key_erased
> > 211: 0 0 0 0 gateworks-gsc 2
> > Edge eeprom_wp
> > 212: 0 0 0 0 gateworks-gsc 5
> > Edge tamper
> > 213: 0 0 0 0 gateworks-gsc 7
> > Edge switch_hold
> > 214: 0 0 0 0 0-0023 2 Edge user_pb
> > IPI0: 367 486 444 286 Rescheduling interrupts
> > IPI1: 972 1283 1202 1630 Function call interrupts
> > IPI2: 0 0 0 0 CPU stop interrupts
> > IPI3: 0 0 0 0 CPU stop (for
> > crash dump) interrupts
> > IPI4: 0 0 0 0 Timer broadcast
> > interrupts
> > IPI5: 5 6 7 36 IRQ work interrupts
> > IPI6: 0 0 0 0 CPU wake-up interrupts
> > Err: 0
> > ^^^ why no interrupts on this board when disabling MSI globally and
> > using ath9k legacy interrupts.... something to do with the bridge?
> >
> > Best Regards,
> >
> > Tim
> >
> >
> >
> > On Mon, Mar 20, 2023 at 4:29 AM Daniel Drake <[email protected]> wrote:
> > >
> > > On Sat, Mar 18, 2023 at 8:17 AM Tim Harvey <[email protected]> wrote:
> > > > It seems that Daniel may have been running into something similar back
> > > > when commit 7368160f0ab0 ("ath9k: add MSI support") was merged
> > > > [1][2][3]
> > > >
> > > > Does anyone have any advice here about what can be going on?
> > >
> > > The issue we found earlier was that ath9k MSI is broken unless the MSI
> > > IRQ number coincidentally is aligned to 4.
> > >
> > > Worked around with:
> > > https://github.com/endlessm/linux/commit/0a2d86a2b9da8f936d873addf0a97e341177fe8a
> > > https://github.com/endlessm/linux/commit/970d2b4b59465cd926e5213e0feb633f6b623447
> > >
> > > IIRC the PCI patch wasn't accepted upstream at the time and I never
> > > got any response from Atheros/Qualcomm about this device
> > > hardware/firmware bug.
> > >
> > > Daniel