Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port.
Each pci line I/O space require 0x100000 and the current limit in the
specific case of ipq8064 cause the malfunction of the second and the
third line that are commonly used for wifi cards.
Current IO space is set to support only 0x100000 space while for ipq806x
if all 3 pcie lines are connected, it's required 0x300000 of space.
Update the IO_SPACE_LIMIT to permit this larger space and update the
documentation for the Mapping PCI I/O space memory end to the new value
of ff0fffff.
Signed-off-by: Ansuel Smith <[email protected]>
---
Documentation/arm/memory.rst | 2 +-
arch/arm/include/asm/io.h | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/Documentation/arm/memory.rst b/Documentation/arm/memory.rst
index 0cb1e2938823..938fe74c9020 100644
--- a/Documentation/arm/memory.rst
+++ b/Documentation/arm/memory.rst
@@ -53,7 +53,7 @@ ffc00000 ffc7ffff Guard region
ff800000 ffbfffff Permanent, fixed read-only mapping of the
firmware provided DT blob
-fee00000 feffffff Mapping of PCI I/O space. This is a static
+fee00000 ff0fffff Mapping of PCI I/O space. This is a static
mapping within the vmalloc space.
VMALLOC_START VMALLOC_END-1 vmalloc() / ioremap() space.
diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h
index fc748122f1e0..76f1c668df5e 100644
--- a/arch/arm/include/asm/io.h
+++ b/arch/arm/include/asm/io.h
@@ -197,7 +197,7 @@ void __iomem *pci_remap_cfgspace(resource_size_t res_cookie, size_t size);
#ifdef CONFIG_NEED_MACH_IO_H
#include <mach/io.h>
#elif defined(CONFIG_PCI)
-#define IO_SPACE_LIMIT ((resource_size_t)0xfffff)
+#define IO_SPACE_LIMIT ((resource_size_t)0x2fffff)
#define __io(a) __typesafe_io(PCI_IO_VIRT_BASE + ((a) & IO_SPACE_LIMIT))
#else
#define __io(a) __typesafe_io((a) & IO_SPACE_LIMIT)
--
2.30.2
On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote:
> Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port.
Do you really? I mean, yes, theoretically, I understand it, the
hardware supports 64kB of I/O port space per root port. But I/O
port space is rather deprecated these days. My laptop has precisely
two devices with I/O ports, one with 64 bytes and the other with 32
bytes. Would you really suffer by allocating 16kB of I/O port
space to each root port?
On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote:
> On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote:
> > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port.
>
> Do you really? I mean, yes, theoretically, I understand it, the
> hardware supports 64kB of I/O port space per root port. But I/O
> port space is rather deprecated these days. My laptop has precisely
> two devices with I/O ports, one with 64 bytes and the other with 32
> bytes. Would you really suffer by allocating 16kB of I/O port
> space to each root port?
We were talking about this in the other wrong patch. I also think this
much space looks wrong. The current ipq806x dts have this space so it's
actually broken from a long time. The only reason pci worked before was
because the pci driver didn't actually check if the settings were right.
New kernel introduced more checks and this problem showed up. (to be
more precise, the pci port are commonly used by the ath10k wifi and the
second ath10k wifi fails to init because of this problem)
If you can give me any hint on how to check if the space can be reduced
I would be very happy to investigate it.
In the driver I notice that the max buffer is set to 2k, could be this a
hint?
On Tue, 11 May 2021 at 04:32, Ansuel Smith <[email protected]> wrote:
>
> On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote:
> > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote:
> > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port.
> >
> > Do you really? I mean, yes, theoretically, I understand it, the
> > hardware supports 64kB of I/O port space per root port. But I/O
> > port space is rather deprecated these days. My laptop has precisely
> > two devices with I/O ports, one with 64 bytes and the other with 32
> > bytes. Would you really suffer by allocating 16kB of I/O port
> > space to each root port?
>
> We were talking about this in the other wrong patch. I also think this
> much space looks wrong. The current ipq806x dts have this space so it's
> actually broken from a long time. The only reason pci worked before was
> because the pci driver didn't actually check if the settings were right.
> New kernel introduced more checks and this problem showed up. (to be
> more precise, the pci port are commonly used by the ath10k wifi and the
> second ath10k wifi fails to init because of this problem)
> If you can give me any hint on how to check if the space can be reduced
> I would be very happy to investigate it.
> In the driver I notice that the max buffer is set to 2k, could be this a
> hint?
>
Could you share the output of lspci -vv from such a system?
I agree with Matthew that fiddling with the size of the I/O space
range probably papers over another problem, and with the odd
exception, no PCIe card used on ARM systems actually uses their I/O
BARs, even when they have them. (I used to carry a PCIe serial port
card to UEFI plugfests because that was the only thing that would stop
working if a system configured its I/O resource window incorrectly)
On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote:
> On Tue, 11 May 2021 at 04:32, Ansuel Smith <[email protected]> wrote:
> >
> > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote:
> > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote:
> > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port.
> > >
> > > Do you really? I mean, yes, theoretically, I understand it, the
> > > hardware supports 64kB of I/O port space per root port. But I/O
> > > port space is rather deprecated these days. My laptop has precisely
> > > two devices with I/O ports, one with 64 bytes and the other with 32
> > > bytes. Would you really suffer by allocating 16kB of I/O port
> > > space to each root port?
> >
> > We were talking about this in the other wrong patch. I also think this
> > much space looks wrong. The current ipq806x dts have this space so it's
> > actually broken from a long time. The only reason pci worked before was
> > because the pci driver didn't actually check if the settings were right.
> > New kernel introduced more checks and this problem showed up. (to be
> > more precise, the pci port are commonly used by the ath10k wifi and the
> > second ath10k wifi fails to init because of this problem)
> > If you can give me any hint on how to check if the space can be reduced
> > I would be very happy to investigate it.
> > In the driver I notice that the max buffer is set to 2k, could be this a
> > hint?
> >
>
> Could you share the output of lspci -vv from such a system?
>
> I agree with Matthew that fiddling with the size of the I/O space
> range probably papers over another problem, and with the odd
> exception, no PCIe card used on ARM systems actually uses their I/O
> BARs, even when they have them. (I used to carry a PCIe serial port
> card to UEFI plugfests because that was the only thing that would stop
> working if a system configured its I/O resource window incorrectly)
Here is the output of lspci -vv
0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode])
Device tree node: /sys/firmware/devicetree/base/soc/pci@1b500000/bridge@0,0
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 44
Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
I/O behind bridge: [disabled]
Memory behind bridge: 08000000-081fffff [size=2M]
Prefetchable memory behind bridge: [disabled]
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 000000004361bc88 Data: 0000
Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0
ExtTag- RBE+
DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s (ok), Width x1 (downgraded)
TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt+
RootCap: CRSVisible-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd-
AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd-
AtomicOpsCtl: ReqEn- EgressBlck-
LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
RootCmd: CERptEn+ NFERptEn+ FERptEn+
RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd-
FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0
ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000
Kernel driver in use: pcieport
lspci: Unable to load libkmod resources: error -12
0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter
Subsystem: Qualcomm Atheros Device cafe
Device tree node: /sys/firmware/devicetree/base/soc/pci@1b500000/bridge@0,0/wifi@1,0
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 53
Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+
Address: 000000004361bc88 Data: 0001
Masking: fffffffe Pending: 00000000
Capabilities: [70] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00
Capabilities: [158 v1] Latency Tolerance Reporting
Max snoop latency: 0ns
Max no snoop latency: 0ns
Capabilities: [160 v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates-
L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
L1SubCtl2:
Kernel driver in use: ath10k_pci
0001:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode])
Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 46
Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
I/O behind bridge: [disabled]
Memory behind bridge: 2e000000-2e1fffff [size=2M]
Prefetchable memory behind bridge: [disabled]
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000435e8688 Data: 0000
Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0
ExtTag- RBE+
DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded)
TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
RootCap: CRSVisible-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd-
AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd-
AtomicOpsCtl: ReqEn- EgressBlck-
LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
RootCmd: CERptEn+ NFERptEn+ FERptEn+
RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd-
FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0
ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000
Kernel driver in use: pcieport
0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter
Subsystem: Qualcomm Atheros Device cafe
Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 54
Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+
Address: 00000000435e8688 Data: 0001
Masking: fffffffe Pending: 00000000
Capabilities: [70] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (downgraded), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00
Capabilities: [158 v1] Latency Tolerance Reporting
Max snoop latency: 0ns
Max no snoop latency: 0ns
Capabilities: [160 v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates-
L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
L1SubCtl2:
Kernel driver in use: ath10k_pci
On Tue, 11 May 2021 at 14:15, Ansuel Smith <[email protected]> wrote:
>
> On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote:
> > On Tue, 11 May 2021 at 04:32, Ansuel Smith <[email protected]> wrote:
> > >
> > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote:
> > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote:
> > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port.
> > > >
> > > > Do you really? I mean, yes, theoretically, I understand it, the
> > > > hardware supports 64kB of I/O port space per root port. But I/O
> > > > port space is rather deprecated these days. My laptop has precisely
> > > > two devices with I/O ports, one with 64 bytes and the other with 32
> > > > bytes. Would you really suffer by allocating 16kB of I/O port
> > > > space to each root port?
> > >
> > > We were talking about this in the other wrong patch. I also think this
> > > much space looks wrong. The current ipq806x dts have this space so it's
> > > actually broken from a long time. The only reason pci worked before was
> > > because the pci driver didn't actually check if the settings were right.
> > > New kernel introduced more checks and this problem showed up. (to be
> > > more precise, the pci port are commonly used by the ath10k wifi and the
> > > second ath10k wifi fails to init because of this problem)
> > > If you can give me any hint on how to check if the space can be reduced
> > > I would be very happy to investigate it.
> > > In the driver I notice that the max buffer is set to 2k, could be this a
> > > hint?
> > >
> >
> > Could you share the output of lspci -vv from such a system?
> >
> > I agree with Matthew that fiddling with the size of the I/O space
> > range probably papers over another problem, and with the odd
> > exception, no PCIe card used on ARM systems actually uses their I/O
> > BARs, even when they have them. (I used to carry a PCIe serial port
> > card to UEFI plugfests because that was the only thing that would stop
> > working if a system configured its I/O resource window incorrectly)
>
> Here is the output of lspci -vv
>
> 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode])
> Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
> I/O behind bridge: [disabled]
> Memory behind bridge: 08000000-081fffff [size=2M]
> Prefetchable memory behind bridge: [disabled]
So this a MMIO window to the endpoint
...
>
> 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter
> Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M]
... and the endpoint has a single *MMIO* BAR of size 2 MiB.
This has *nothing* to do with port I/O, which is what you are
modifying with your patch.
Did you check that the problem exists without the patch, and that the
patch makes it go away?
> Capabilities: [40] Power Management version 3
> Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
> Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+
> Address: 000000004361bc88 Data: 0001
> Masking: fffffffe Pending: 00000000
> Capabilities: [70] Express (v2) Endpoint, MSI 00
> DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
> ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
> DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
> RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
> MaxPayload 128 bytes, MaxReadReq 128 bytes
> DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
> LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported
> ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
> LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
> ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> LnkSta: Speed 5GT/s (ok), Width x1 (ok)
> TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
> 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
> EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
> FRS- TPHComp- ExtTPHComp-
> AtomicOpsCap: 32bit- 64bit- 128bitCAS-
> DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
> AtomicOpsCtl: ReqEn-
> LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
> LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
> Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
> Compliance De-emphasis: -6dB
> LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
> EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
> Retimer- 2Retimers- CrosslinkRes: unsupported
> Capabilities: [100 v2] Advanced Error Reporting
> UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
> CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
> CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
> AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
> MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
> HeaderLog: 00000000 00000000 00000000 00000000
> Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00
> Capabilities: [158 v1] Latency Tolerance Reporting
> Max snoop latency: 0ns
> Max no snoop latency: 0ns
> Capabilities: [160 v1] L1 PM Substates
> L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates-
> L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
> L1SubCtl2:
> Kernel driver in use: ath10k_pci
>
> 0001:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode])
> Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0
> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> Latency: 0, Cache Line Size: 64 bytes
> Interrupt: pin A routed to IRQ 46
> Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
> I/O behind bridge: [disabled]
> Memory behind bridge: 2e000000-2e1fffff [size=2M]
> Prefetchable memory behind bridge: [disabled]
> Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
> PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> Capabilities: [40] Power Management version 3
> Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
> Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
> Address: 00000000435e8688 Data: 0000
> Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00
> DevCap: MaxPayload 128 bytes, PhantFunc 0
> ExtTag- RBE+
> DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
> RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
> MaxPayload 128 bytes, MaxReadReq 128 bytes
> DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
> LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
> ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
> LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
> ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded)
> TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
> RootCap: CRSVisible-
> RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
> RootSta: PME ReqID 0000, PMEStatus- PMEPending-
> DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR-
> 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
> EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
> FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd-
> AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS-
> DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd-
> AtomicOpsCtl: ReqEn- EgressBlck-
> LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
> LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
> Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
> Compliance De-emphasis: -6dB
> LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
> EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
> Retimer- 2Retimers- CrosslinkRes: unsupported
> Capabilities: [100 v1] Advanced Error Reporting
> UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
> CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
> CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
> AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
> MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
> HeaderLog: 00000000 00000000 00000000 00000000
> RootCmd: CERptEn+ NFERptEn+ FERptEn+
> RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd-
> FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0
> ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000
> Kernel driver in use: pcieport
>
> 0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter
> Subsystem: Qualcomm Atheros Device cafe
> Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0
> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> Latency: 0, Cache Line Size: 64 bytes
> Interrupt: pin A routed to IRQ 54
> Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M]
> Capabilities: [40] Power Management version 3
> Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
> Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+
> Address: 00000000435e8688 Data: 0001
> Masking: fffffffe Pending: 00000000
> Capabilities: [70] Express (v2) Endpoint, MSI 00
> DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
> ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
> DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
> RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
> MaxPayload 128 bytes, MaxReadReq 128 bytes
> DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
> LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported
> ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
> LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
> ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> LnkSta: Speed 2.5GT/s (downgraded), Width x1 (ok)
> TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
> 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
> EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
> FRS- TPHComp- ExtTPHComp-
> AtomicOpsCap: 32bit- 64bit- 128bitCAS-
> DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
> AtomicOpsCtl: ReqEn-
> LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
> LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
> Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
> Compliance De-emphasis: -6dB
> LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
> EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
> Retimer- 2Retimers- CrosslinkRes: unsupported
> Capabilities: [100 v2] Advanced Error Reporting
> UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
> CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
> CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
> AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
> MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
> HeaderLog: 00000000 00000000 00000000 00000000
> Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00
> Capabilities: [158 v1] Latency Tolerance Reporting
> Max snoop latency: 0ns
> Max no snoop latency: 0ns
> Capabilities: [160 v1] L1 PM Substates
> L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates-
> L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
> L1SubCtl2:
> Kernel driver in use: ath10k_pci
On Tue, May 11, 2021 at 02:30:36PM +0200, Ard Biesheuvel wrote:
> On Tue, 11 May 2021 at 14:15, Ansuel Smith <[email protected]> wrote:
> >
> > On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote:
> > > On Tue, 11 May 2021 at 04:32, Ansuel Smith <[email protected]> wrote:
> > > >
> > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote:
> > > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote:
> > > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port.
> > > > >
> > > > > Do you really? I mean, yes, theoretically, I understand it, the
> > > > > hardware supports 64kB of I/O port space per root port. But I/O
> > > > > port space is rather deprecated these days. My laptop has precisely
> > > > > two devices with I/O ports, one with 64 bytes and the other with 32
> > > > > bytes. Would you really suffer by allocating 16kB of I/O port
> > > > > space to each root port?
> > > >
> > > > We were talking about this in the other wrong patch. I also think this
> > > > much space looks wrong. The current ipq806x dts have this space so it's
> > > > actually broken from a long time. The only reason pci worked before was
> > > > because the pci driver didn't actually check if the settings were right.
> > > > New kernel introduced more checks and this problem showed up. (to be
> > > > more precise, the pci port are commonly used by the ath10k wifi and the
> > > > second ath10k wifi fails to init because of this problem)
> > > > If you can give me any hint on how to check if the space can be reduced
> > > > I would be very happy to investigate it.
> > > > In the driver I notice that the max buffer is set to 2k, could be this a
> > > > hint?
> > > >
> > >
> > > Could you share the output of lspci -vv from such a system?
> > >
> > > I agree with Matthew that fiddling with the size of the I/O space
> > > range probably papers over another problem, and with the odd
> > > exception, no PCIe card used on ARM systems actually uses their I/O
> > > BARs, even when they have them. (I used to carry a PCIe serial port
> > > card to UEFI plugfests because that was the only thing that would stop
> > > working if a system configured its I/O resource window incorrectly)
> >
> > Here is the output of lspci -vv
> >
> > 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode])
> > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
> > I/O behind bridge: [disabled]
> > Memory behind bridge: 08000000-081fffff [size=2M]
> > Prefetchable memory behind bridge: [disabled]
>
> So this a MMIO window to the endpoint
>
> ...
>
> >
> > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter
> > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M]
>
> ... and the endpoint has a single *MMIO* BAR of size 2 MiB.
>
> This has *nothing* to do with port I/O, which is what you are
> modifying with your patch.
>
> Did you check that the problem exists without the patch, and that the
> patch makes it go away?
>
>
Yes without the change to IO_SPACE_LIMIT, the ath10k driver fails to
init as it can't access the reg. Only the first pci wifi works but the
second one fails to init. By increasing the limit all comes back to
normal. What I really can't understand is if the big IO space set in the
ipq8064 dtsi was wrong from the start and the ath10k fails to init just
because is missconfigured. Any idea how to find the appropriate max IO space
for the pci?
>
> > Capabilities: [40] Power Management version 3
> > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
> > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> > Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+
> > Address: 000000004361bc88 Data: 0001
> > Masking: fffffffe Pending: 00000000
> > Capabilities: [70] Express (v2) Endpoint, MSI 00
> > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
> > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
> > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
> > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
> > MaxPayload 128 bytes, MaxReadReq 128 bytes
> > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
> > LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported
> > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
> > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
> > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> > LnkSta: Speed 5GT/s (ok), Width x1 (ok)
> > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
> > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
> > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
> > FRS- TPHComp- ExtTPHComp-
> > AtomicOpsCap: 32bit- 64bit- 128bitCAS-
> > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
> > AtomicOpsCtl: ReqEn-
> > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
> > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
> > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
> > Compliance De-emphasis: -6dB
> > LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
> > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
> > Retimer- 2Retimers- CrosslinkRes: unsupported
> > Capabilities: [100 v2] Advanced Error Reporting
> > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
> > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
> > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
> > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
> > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
> > HeaderLog: 00000000 00000000 00000000 00000000
> > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00
> > Capabilities: [158 v1] Latency Tolerance Reporting
> > Max snoop latency: 0ns
> > Max no snoop latency: 0ns
> > Capabilities: [160 v1] L1 PM Substates
> > L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates-
> > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
> > L1SubCtl2:
> > Kernel driver in use: ath10k_pci
> >
> > 0001:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode])
> > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0
> > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
> > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> > Latency: 0, Cache Line Size: 64 bytes
> > Interrupt: pin A routed to IRQ 46
> > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
> > I/O behind bridge: [disabled]
> > Memory behind bridge: 2e000000-2e1fffff [size=2M]
> > Prefetchable memory behind bridge: [disabled]
> > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> > BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
> > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> > Capabilities: [40] Power Management version 3
> > Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
> > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
> > Address: 00000000435e8688 Data: 0000
> > Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00
> > DevCap: MaxPayload 128 bytes, PhantFunc 0
> > ExtTag- RBE+
> > DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
> > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
> > MaxPayload 128 bytes, MaxReadReq 128 bytes
> > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
> > LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
> > ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
> > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
> > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> > LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded)
> > TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
> > RootCap: CRSVisible-
> > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
> > RootSta: PME ReqID 0000, PMEStatus- PMEPending-
> > DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR-
> > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
> > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
> > FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd-
> > AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS-
> > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd-
> > AtomicOpsCtl: ReqEn- EgressBlck-
> > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
> > LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
> > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
> > Compliance De-emphasis: -6dB
> > LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
> > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
> > Retimer- 2Retimers- CrosslinkRes: unsupported
> > Capabilities: [100 v1] Advanced Error Reporting
> > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
> > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
> > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
> > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
> > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
> > HeaderLog: 00000000 00000000 00000000 00000000
> > RootCmd: CERptEn+ NFERptEn+ FERptEn+
> > RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd-
> > FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0
> > ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000
> > Kernel driver in use: pcieport
> >
> > 0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter
> > Subsystem: Qualcomm Atheros Device cafe
> > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0
> > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
> > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> > Latency: 0, Cache Line Size: 64 bytes
> > Interrupt: pin A routed to IRQ 54
> > Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M]
> > Capabilities: [40] Power Management version 3
> > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
> > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> > Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+
> > Address: 00000000435e8688 Data: 0001
> > Masking: fffffffe Pending: 00000000
> > Capabilities: [70] Express (v2) Endpoint, MSI 00
> > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
> > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
> > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
> > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
> > MaxPayload 128 bytes, MaxReadReq 128 bytes
> > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
> > LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported
> > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
> > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
> > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> > LnkSta: Speed 2.5GT/s (downgraded), Width x1 (ok)
> > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
> > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
> > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
> > FRS- TPHComp- ExtTPHComp-
> > AtomicOpsCap: 32bit- 64bit- 128bitCAS-
> > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
> > AtomicOpsCtl: ReqEn-
> > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
> > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
> > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
> > Compliance De-emphasis: -6dB
> > LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
> > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
> > Retimer- 2Retimers- CrosslinkRes: unsupported
> > Capabilities: [100 v2] Advanced Error Reporting
> > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
> > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
> > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
> > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
> > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
> > HeaderLog: 00000000 00000000 00000000 00000000
> > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00
> > Capabilities: [158 v1] Latency Tolerance Reporting
> > Max snoop latency: 0ns
> > Max no snoop latency: 0ns
> > Capabilities: [160 v1] L1 PM Substates
> > L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates-
> > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
> > L1SubCtl2:
> > Kernel driver in use: ath10k_pci
On Tue, 11 May 2021 at 14:37, Ansuel Smith <[email protected]> wrote:
>
> On Tue, May 11, 2021 at 02:30:36PM +0200, Ard Biesheuvel wrote:
> > On Tue, 11 May 2021 at 14:15, Ansuel Smith <[email protected]> wrote:
> > >
> > > On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote:
> > > > On Tue, 11 May 2021 at 04:32, Ansuel Smith <[email protected]> wrote:
> > > > >
> > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote:
> > > > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote:
> > > > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port.
> > > > > >
> > > > > > Do you really? I mean, yes, theoretically, I understand it, the
> > > > > > hardware supports 64kB of I/O port space per root port. But I/O
> > > > > > port space is rather deprecated these days. My laptop has precisely
> > > > > > two devices with I/O ports, one with 64 bytes and the other with 32
> > > > > > bytes. Would you really suffer by allocating 16kB of I/O port
> > > > > > space to each root port?
> > > > >
> > > > > We were talking about this in the other wrong patch. I also think this
> > > > > much space looks wrong. The current ipq806x dts have this space so it's
> > > > > actually broken from a long time. The only reason pci worked before was
> > > > > because the pci driver didn't actually check if the settings were right.
> > > > > New kernel introduced more checks and this problem showed up. (to be
> > > > > more precise, the pci port are commonly used by the ath10k wifi and the
> > > > > second ath10k wifi fails to init because of this problem)
> > > > > If you can give me any hint on how to check if the space can be reduced
> > > > > I would be very happy to investigate it.
> > > > > In the driver I notice that the max buffer is set to 2k, could be this a
> > > > > hint?
> > > > >
> > > >
> > > > Could you share the output of lspci -vv from such a system?
> > > >
> > > > I agree with Matthew that fiddling with the size of the I/O space
> > > > range probably papers over another problem, and with the odd
> > > > exception, no PCIe card used on ARM systems actually uses their I/O
> > > > BARs, even when they have them. (I used to carry a PCIe serial port
> > > > card to UEFI plugfests because that was the only thing that would stop
> > > > working if a system configured its I/O resource window incorrectly)
> > >
> > > Here is the output of lspci -vv
> > >
> > > 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode])
> > > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
> > > I/O behind bridge: [disabled]
> > > Memory behind bridge: 08000000-081fffff [size=2M]
> > > Prefetchable memory behind bridge: [disabled]
> >
> > So this a MMIO window to the endpoint
> >
> > ...
> >
> > >
> > > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter
> > > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M]
> >
> > ... and the endpoint has a single *MMIO* BAR of size 2 MiB.
> >
> > This has *nothing* to do with port I/O, which is what you are
> > modifying with your patch.
> >
> > Did you check that the problem exists without the patch, and that the
> > patch makes it go away?
> >
> >
>
> Yes without the change to IO_SPACE_LIMIT, the ath10k driver fails to
> init as it can't access the reg. Only the first pci wifi works but the
> second one fails to init. By increasing the limit all comes back to
> normal. What I really can't understand is if the big IO space set in the
> ipq8064 dtsi was wrong from the start and the ath10k fails to init just
> because is missconfigured. Any idea how to find the appropriate max IO space
> for the pci?
>
OK, so the entire second host bridge fails to probe, because there is
no virtual address space left for the I/O window.
Just change the 'downstream I/O' window size in the DT to 64k
(0x10000) - I assume the current size (0x100000) is a typo anyway.
On Tue, May 11, 2021 at 02:46:49PM +0200, Ard Biesheuvel wrote:
> On Tue, 11 May 2021 at 14:37, Ansuel Smith <[email protected]> wrote:
> >
> > On Tue, May 11, 2021 at 02:30:36PM +0200, Ard Biesheuvel wrote:
> > > On Tue, 11 May 2021 at 14:15, Ansuel Smith <[email protected]> wrote:
> > > >
> > > > On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote:
> > > > > On Tue, 11 May 2021 at 04:32, Ansuel Smith <[email protected]> wrote:
> > > > > >
> > > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote:
> > > > > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote:
> > > > > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port.
> > > > > > >
> > > > > > > Do you really? I mean, yes, theoretically, I understand it, the
> > > > > > > hardware supports 64kB of I/O port space per root port. But I/O
> > > > > > > port space is rather deprecated these days. My laptop has precisely
> > > > > > > two devices with I/O ports, one with 64 bytes and the other with 32
> > > > > > > bytes. Would you really suffer by allocating 16kB of I/O port
> > > > > > > space to each root port?
> > > > > >
> > > > > > We were talking about this in the other wrong patch. I also think this
> > > > > > much space looks wrong. The current ipq806x dts have this space so it's
> > > > > > actually broken from a long time. The only reason pci worked before was
> > > > > > because the pci driver didn't actually check if the settings were right.
> > > > > > New kernel introduced more checks and this problem showed up. (to be
> > > > > > more precise, the pci port are commonly used by the ath10k wifi and the
> > > > > > second ath10k wifi fails to init because of this problem)
> > > > > > If you can give me any hint on how to check if the space can be reduced
> > > > > > I would be very happy to investigate it.
> > > > > > In the driver I notice that the max buffer is set to 2k, could be this a
> > > > > > hint?
> > > > > >
> > > > >
> > > > > Could you share the output of lspci -vv from such a system?
> > > > >
> > > > > I agree with Matthew that fiddling with the size of the I/O space
> > > > > range probably papers over another problem, and with the odd
> > > > > exception, no PCIe card used on ARM systems actually uses their I/O
> > > > > BARs, even when they have them. (I used to carry a PCIe serial port
> > > > > card to UEFI plugfests because that was the only thing that would stop
> > > > > working if a system configured its I/O resource window incorrectly)
> > > >
> > > > Here is the output of lspci -vv
> > > >
> > > > 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode])
> > > > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0
> > > > I/O behind bridge: [disabled]
> > > > Memory behind bridge: 08000000-081fffff [size=2M]
> > > > Prefetchable memory behind bridge: [disabled]
> > >
> > > So this a MMIO window to the endpoint
> > >
> > > ...
> > >
> > > >
> > > > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter
> > > > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M]
> > >
> > > ... and the endpoint has a single *MMIO* BAR of size 2 MiB.
> > >
> > > This has *nothing* to do with port I/O, which is what you are
> > > modifying with your patch.
> > >
> > > Did you check that the problem exists without the patch, and that the
> > > patch makes it go away?
> > >
> > >
> >
> > Yes without the change to IO_SPACE_LIMIT, the ath10k driver fails to
> > init as it can't access the reg. Only the first pci wifi works but the
> > second one fails to init. By increasing the limit all comes back to
> > normal. What I really can't understand is if the big IO space set in the
> > ipq8064 dtsi was wrong from the start and the ath10k fails to init just
> > because is missconfigured. Any idea how to find the appropriate max IO space
> > for the pci?
> >
>
> OK, so the entire second host bridge fails to probe, because there is
> no virtual address space left for the I/O window.
>
> Just change the 'downstream I/O' window size in the DT to 64k
> (0x10000) - I assume the current size (0x100000) is a typo anyway.
Ok, so my fear were right... someone just typo the IO when the dtsi was
pushed and was wrong from all that times. Much easier and cleaner
solution.
On Tue, May 11, 2021 at 02:15:21PM +0200, Ansuel Smith wrote:
> 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter
> Subsystem: Qualcomm Atheros Device cafe
> Device tree node: /sys/firmware/devicetree/base/soc/pci@1b500000/bridge@0,0/wifi@1,0
> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> Latency: 0, Cache Line Size: 64 bytes
> Interrupt: pin A routed to IRQ 53
> Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M]
It seems this Atheros device does not make any use of I/O, so this
device should be fine.
> 0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter
> Subsystem: Qualcomm Atheros Device cafe
> Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0
> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> Latency: 0, Cache Line Size: 64 bytes
> Interrupt: pin A routed to IRQ 54
> Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M]
This device also seems to not make use of any I/O.
I don't see any reason probing of these two drivers should fail if
there is no or reduced I/O space.
Please can you share the kernel messages so we can see exactly what is
failing and how?
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
On Tue, May 11, 2021 at 02:51:15PM +0200, Ansuel Smith wrote:
> On Tue, May 11, 2021 at 02:46:49PM +0200, Ard Biesheuvel wrote:
> > OK, so the entire second host bridge fails to probe, because there is
> > no virtual address space left for the I/O window.
> >
> > Just change the 'downstream I/O' window size in the DT to 64k
> > (0x10000) - I assume the current size (0x100000) is a typo anyway.
>
> Ok, so my fear were right... someone just typo the IO when the dtsi was
> pushed and was wrong from all that times. Much easier and cleaner
> solution.
Great news.
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
On Tue, May 11, 2021 at 6:26 AM Ard Biesheuvel <[email protected]> wrote:
> I used to carry a PCIe serial port
> card to UEFI plugfests because that was the only thing that would stop
> working if a system configured its I/O resource window incorrectly
I've been looking for a thing like that for testing, I was actually
thinking that it would perhaps be a good idea to add a I/O-resource
requiring device to QEMU just to be able to test this kind of
thing for completeness.
Yours,
Linus Walleij