Hi Bjorn,
The SD card reader in my Acer Aspire S5 doesn't work with 4.20-rc.
Here's what lspci -v says about it (in a bad kernel):
02:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5209 PCI Express Card Reader
(rev 01)
Subsystem: Acer Incorporated [ALI] Device 0704
Flags: bus master, fast devsel, latency 0, IRQ 35
Memory at d9001000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [70] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Device Serial Number 00-00-00-01-00-4c-e0-00
Kernel driver in use: rtsx_pci
Kernel modules: rtsx_pci
When it doesn't work, it doesn't generate any interrupts on device insertion
and removal and this seems to be reproducible 100% of the time.
Bisection turned up
commit de468b755464426c276df2daf1e54bcd64186020
Merge: b1801bf05964 d6112f8def51
Author: Bjorn Helgaas <[email protected]>
Date: Sat Oct 20 11:45:28 2018 -0500
Merge branch 'pci/enumeration'
- Remove x86 and arm64 node-local allocation for host bridge structures
(Punit Agrawal)
- Pay attention to device-specific _PXM node values (Jonathan Cameron)
- Support new Immediate Readiness bit (Felipe Balbi)
* pci/enumeration:
PCI: Add support for Immediate Readiness
ACPI/PCI: Pay attention to device-specific _PXM node values
x86/PCI: Remove node-local allocation when initialising host controller
arm64: PCI: Remove node-local allocations when initialising host controller
as the first bad commit, but the "PCI: Add support for Immediate Readiness"
one was tested as "good" (full bisect log is attached).
I wonder if you have any ideas on what to check?
Cheers,
Rafael
On Monday, November 26, 2018 7:03:58 PM CET Rafael J. Wysocki wrote:
> Hi Bjorn,
>
> The SD card reader in my Acer Aspire S5 doesn't work with 4.20-rc.
>
> Here's what lspci -v says about it (in a bad kernel):
>
> 02:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5209 PCI Express Card Reader
> (rev 01)
> Subsystem: Acer Incorporated [ALI] Device 0704
> Flags: bus master, fast devsel, latency 0, IRQ 35
> Memory at d9001000 (32-bit, non-prefetchable) [size=4K]
> Capabilities: [40] Power Management version 3
> Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
> Capabilities: [70] Express Endpoint, MSI 00
> Capabilities: [100] Advanced Error Reporting
> Capabilities: [140] Device Serial Number 00-00-00-01-00-4c-e0-00
> Kernel driver in use: rtsx_pci
> Kernel modules: rtsx_pci
>
> When it doesn't work, it doesn't generate any interrupts on device insertion
> and removal and this seems to be reproducible 100% of the time.
>
> Bisection turned up
>
> commit de468b755464426c276df2daf1e54bcd64186020
> Merge: b1801bf05964 d6112f8def51
> Author: Bjorn Helgaas <[email protected]>
> Date: Sat Oct 20 11:45:28 2018 -0500
>
> Merge branch 'pci/enumeration'
>
> - Remove x86 and arm64 node-local allocation for host bridge structures
> (Punit Agrawal)
>
> - Pay attention to device-specific _PXM node values (Jonathan Cameron)
>
> - Support new Immediate Readiness bit (Felipe Balbi)
>
> * pci/enumeration:
> PCI: Add support for Immediate Readiness
> ACPI/PCI: Pay attention to device-specific _PXM node values
> x86/PCI: Remove node-local allocation when initialising host controller
> arm64: PCI: Remove node-local allocations when initialising host controller
>
> as the first bad commit, but the "PCI: Add support for Immediate Readiness"
> one was tested as "good" (full bisect log is attached).
>
> I wonder if you have any ideas on what to check?
So reverting 17c91487364f (PCI/ASPM: Do not initialize link state when
aspm_disabled is set) on top of 4.20-rc4 makes the problem go away.
I guess that the device in question needs pcie_aspm_cap_init() to be
called for it even though the FADT says "NO_ASPM".
Thanks,
Rafael
On Mon, Nov 26, 2018 at 11:37:20PM +0100, Rafael J. Wysocki wrote:
> On Monday, November 26, 2018 7:03:58 PM CET Rafael J. Wysocki wrote:
> > Hi Bjorn,
> >
> > The SD card reader in my Acer Aspire S5 doesn't work with 4.20-rc.
> >
> > Here's what lspci -v says about it (in a bad kernel):
> >
> > 02:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5209 PCI Express Card Reader
> > (rev 01)
> > Subsystem: Acer Incorporated [ALI] Device 0704
> > Flags: bus master, fast devsel, latency 0, IRQ 35
> > Memory at d9001000 (32-bit, non-prefetchable) [size=4K]
> > Capabilities: [40] Power Management version 3
> > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
> > Capabilities: [70] Express Endpoint, MSI 00
> > Capabilities: [100] Advanced Error Reporting
> > Capabilities: [140] Device Serial Number 00-00-00-01-00-4c-e0-00
> > Kernel driver in use: rtsx_pci
> > Kernel modules: rtsx_pci
Thanks a lot for bisecting this!
With a good kernel (v4.19 or v4.20-rc with 17c91487364f reverted),
would you mind collecting "lspci -vv" output, the dmesg log with
"pci=earlydump", and the FADT dump?
I'm interested in the initial state of the device at handoff from
BIOS, and what Linux changes even when aspm_disabled is set.
If we can't figure out a way to fix both this issue and the one fixed
by 17c91487364f, I guess the fallback will be to revert 17c91487364f
since it's better to allow a system that was previously broken to
remain broken than it is to break a system that previously worked.
But obviously I hope we can figure out a solution that fixes both
cases.
> > When it doesn't work, it doesn't generate any interrupts on device insertion
> > and removal and this seems to be reproducible 100% of the time.
>
> So reverting 17c91487364f (PCI/ASPM: Do not initialize link state when
> aspm_disabled is set) on top of 4.20-rc4 makes the problem go away.
>
> I guess that the device in question needs pcie_aspm_cap_init() to be
> called for it even though the FADT says "NO_ASPM".
On Tuesday, November 27, 2018 9:25:14 PM CET Bjorn Helgaas wrote:
> On Mon, Nov 26, 2018 at 11:37:20PM +0100, Rafael J. Wysocki wrote:
> > On Monday, November 26, 2018 7:03:58 PM CET Rafael J. Wysocki wrote:
> > > Hi Bjorn,
> > >
> > > The SD card reader in my Acer Aspire S5 doesn't work with 4.20-rc.
> > >
> > > Here's what lspci -v says about it (in a bad kernel):
> > >
> > > 02:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5209 PCI Express Card Reader
> > > (rev 01)
> > > Subsystem: Acer Incorporated [ALI] Device 0704
> > > Flags: bus master, fast devsel, latency 0, IRQ 35
> > > Memory at d9001000 (32-bit, non-prefetchable) [size=4K]
> > > Capabilities: [40] Power Management version 3
> > > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
> > > Capabilities: [70] Express Endpoint, MSI 00
> > > Capabilities: [100] Advanced Error Reporting
> > > Capabilities: [140] Device Serial Number 00-00-00-01-00-4c-e0-00
> > > Kernel driver in use: rtsx_pci
> > > Kernel modules: rtsx_pci
>
> Thanks a lot for bisecting this!
>
> With a good kernel (v4.19 or v4.20-rc with 17c91487364f reverted),
> would you mind collecting "lspci -vv" output, the dmesg log with
> "pci=earlydump", and the FADT dump?
I'll do that tomorrow.
> I'm interested in the initial state of the device at handoff from
> BIOS, and what Linux changes even when aspm_disabled is set.
OK
> If we can't figure out a way to fix both this issue and the one fixed
> by 17c91487364f, I guess the fallback will be to revert 17c91487364f
> since it's better to allow a system that was previously broken to
> remain broken than it is to break a system that previously worked.
Well, depending on how many systems are affected by the issues fixed by
17c91487364f IMO.
Arguably, if the FADT says "NO_ASPM", then the platform should not need the
OS to initialize ASPM to work. I guess a manual workaround (like an extra
kernel command line option or similar) should suffice in this particular
case.
> But obviously I hope we can figure out a solution that fixes both
> cases.
Of course.
On Tuesday, November 27, 2018 9:25:14 PM CET Bjorn Helgaas wrote:
> On Mon, Nov 26, 2018 at 11:37:20PM +0100, Rafael J. Wysocki wrote:
> > On Monday, November 26, 2018 7:03:58 PM CET Rafael J. Wysocki wrote:
> > > Hi Bjorn,
> > >
> > > The SD card reader in my Acer Aspire S5 doesn't work with 4.20-rc.
> > >
> > > Here's what lspci -v says about it (in a bad kernel):
> > >
> > > 02:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5209 PCI Express Card Reader
> > > (rev 01)
> > > Subsystem: Acer Incorporated [ALI] Device 0704
> > > Flags: bus master, fast devsel, latency 0, IRQ 35
> > > Memory at d9001000 (32-bit, non-prefetchable) [size=4K]
> > > Capabilities: [40] Power Management version 3
> > > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
> > > Capabilities: [70] Express Endpoint, MSI 00
> > > Capabilities: [100] Advanced Error Reporting
> > > Capabilities: [140] Device Serial Number 00-00-00-01-00-4c-e0-00
> > > Kernel driver in use: rtsx_pci
> > > Kernel modules: rtsx_pci
>
> Thanks a lot for bisecting this!
>
> With a good kernel (v4.19 or v4.20-rc with 17c91487364f reverted),
> would you mind collecting "lspci -vv" output, the dmesg log with
> "pci=earlydump", and the FADT dump?
All of the information is attached to the BZ entry at
https://bugzilla.kernel.org/show_bug.cgi?id=201801
I have created for the tracking of this issue. Let's continue in the BZ.
Cheers,
Rafael
On Wed, Nov 28, 2018 at 6:13 AM Rafael J. Wysocki <[email protected]> wrote:
>
> On Tuesday, November 27, 2018 9:25:14 PM CET Bjorn Helgaas wrote:
> > On Mon, Nov 26, 2018 at 11:37:20PM +0100, Rafael J. Wysocki wrote:
> > > On Monday, November 26, 2018 7:03:58 PM CET Rafael J. Wysocki wrote:
> > > > Hi Bjorn,
> > > >
> > > > The SD card reader in my Acer Aspire S5 doesn't work with 4.20-rc.
> > > >
> > > > Here's what lspci -v says about it (in a bad kernel):
> > > >
> > > > 02:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5209 PCI Express Card Reader
> > > > (rev 01)
> > > > Subsystem: Acer Incorporated [ALI] Device 0704
> > > > Flags: bus master, fast devsel, latency 0, IRQ 35
> > > > Memory at d9001000 (32-bit, non-prefetchable) [size=4K]
> > > > Capabilities: [40] Power Management version 3
> > > > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
> > > > Capabilities: [70] Express Endpoint, MSI 00
> > > > Capabilities: [100] Advanced Error Reporting
> > > > Capabilities: [140] Device Serial Number 00-00-00-01-00-4c-e0-00
> > > > Kernel driver in use: rtsx_pci
> > > > Kernel modules: rtsx_pci
> >
> > Thanks a lot for bisecting this!
> >
> > With a good kernel (v4.19 or v4.20-rc with 17c91487364f reverted),
> > would you mind collecting "lspci -vv" output, the dmesg log with
> > "pci=earlydump", and the FADT dump?
>
> All of the information is attached to the BZ entry at
>
> https://bugzilla.kernel.org/show_bug.cgi?id=201801
Thanks! I hope Patrick has a chance to look at this. Per the
bugzilla mentioned in 17c91487364f, it fixes a problem with a custom
proprietary PCIe device, and there's a lot of good detailed analysis
there, so hopefully we can figure out a way to address both
situations.
On Wed, Nov 28, 2018 at 02:05:21PM -0600, Bjorn Helgaas wrote:
> On Wed, Nov 28, 2018 at 6:13 AM Rafael J. Wysocki <[email protected]> wrote:
> > On Tuesday, November 27, 2018 9:25:14 PM CET Bjorn Helgaas wrote:
> > > On Mon, Nov 26, 2018 at 11:37:20PM +0100, Rafael J. Wysocki wrote:
> > > > On Monday, November 26, 2018 7:03:58 PM CET Rafael J. Wysocki wrote:
> > > > > Hi Bjorn,
> > > > >
> > > > > The SD card reader in my Acer Aspire S5 doesn't work with 4.20-rc.
> > > > >
> > > > > Here's what lspci -v says about it (in a bad kernel):
> > > > >
> > > > > 02:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5209 PCI Express Card Reader
> > > > > (rev 01)
> > > > > Subsystem: Acer Incorporated [ALI] Device 0704
> > > > > Flags: bus master, fast devsel, latency 0, IRQ 35
> > > > > Memory at d9001000 (32-bit, non-prefetchable) [size=4K]
> > > > > Capabilities: [40] Power Management version 3
> > > > > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
> > > > > Capabilities: [70] Express Endpoint, MSI 00
> > > > > Capabilities: [100] Advanced Error Reporting
> > > > > Capabilities: [140] Device Serial Number 00-00-00-01-00-4c-e0-00
> > > > > Kernel driver in use: rtsx_pci
> > > > > Kernel modules: rtsx_pci
> > >
> > > Thanks a lot for bisecting this!
> > >
> > > With a good kernel (v4.19 or v4.20-rc with 17c91487364f reverted),
> > > would you mind collecting "lspci -vv" output, the dmesg log with
> > > "pci=earlydump", and the FADT dump?
> >
> > All of the information is attached to the BZ entry at
> >
> > https://bugzilla.kernel.org/show_bug.cgi?id=201801
>
> Thanks! I hope Patrick has a chance to look at this. Per the
> bugzilla mentioned in 17c91487364f, it fixes a problem with a custom
> proprietary PCIe device, and there's a lot of good detailed analysis
> there, so hopefully we can figure out a way to address both
> situations.
I queued up a revert on for-linus, since we haven't made any progress on
this yet. I'll be on vacation much of this week, but I want to get
the revert (or better, a fix if we can find one) in before -rc6 comes
out next Sunday.
If we figure out a fix before then, I'll drop the revert and use the
fix instead.
Bjorn
On Tuesday, December 4, 2018 1:10:09 AM CET Bjorn Helgaas wrote:
> On Wed, Nov 28, 2018 at 02:05:21PM -0600, Bjorn Helgaas wrote:
> > On Wed, Nov 28, 2018 at 6:13 AM Rafael J. Wysocki <[email protected]> wrote:
> > > On Tuesday, November 27, 2018 9:25:14 PM CET Bjorn Helgaas wrote:
> > > > On Mon, Nov 26, 2018 at 11:37:20PM +0100, Rafael J. Wysocki wrote:
> > > > > On Monday, November 26, 2018 7:03:58 PM CET Rafael J. Wysocki wrote:
> > > > > > Hi Bjorn,
> > > > > >
> > > > > > The SD card reader in my Acer Aspire S5 doesn't work with 4.20-rc.
> > > > > >
> > > > > > Here's what lspci -v says about it (in a bad kernel):
> > > > > >
> > > > > > 02:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5209 PCI Express Card Reader
> > > > > > (rev 01)
> > > > > > Subsystem: Acer Incorporated [ALI] Device 0704
> > > > > > Flags: bus master, fast devsel, latency 0, IRQ 35
> > > > > > Memory at d9001000 (32-bit, non-prefetchable) [size=4K]
> > > > > > Capabilities: [40] Power Management version 3
> > > > > > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
> > > > > > Capabilities: [70] Express Endpoint, MSI 00
> > > > > > Capabilities: [100] Advanced Error Reporting
> > > > > > Capabilities: [140] Device Serial Number 00-00-00-01-00-4c-e0-00
> > > > > > Kernel driver in use: rtsx_pci
> > > > > > Kernel modules: rtsx_pci
> > > >
> > > > Thanks a lot for bisecting this!
> > > >
> > > > With a good kernel (v4.19 or v4.20-rc with 17c91487364f reverted),
> > > > would you mind collecting "lspci -vv" output, the dmesg log with
> > > > "pci=earlydump", and the FADT dump?
> > >
> > > All of the information is attached to the BZ entry at
> > >
> > > https://bugzilla.kernel.org/show_bug.cgi?id=201801
> >
> > Thanks! I hope Patrick has a chance to look at this. Per the
> > bugzilla mentioned in 17c91487364f, it fixes a problem with a custom
> > proprietary PCIe device, and there's a lot of good detailed analysis
> > there, so hopefully we can figure out a way to address both
> > situations.
>
> I queued up a revert on for-linus, since we haven't made any progress on
> this yet. I'll be on vacation much of this week, but I want to get
> the revert (or better, a fix if we can find one) in before -rc6 comes
> out next Sunday.
Thanks!
> If we figure out a fix before then, I'll drop the revert and use the
> fix instead.