Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From:   Mario Limonciello <mario.limonciello@amd.com>
To:     Bjorn Helgaas <bhelgaas@google.com>,
        Mika Westerberg <mika.westerberg@linux.intel.com>
CC:     <linux-pci@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        "S-k Shyam-sundar" <Shyam-sundar.S-k@amd.com>,
        Natikar Basavaraj <Basavaraj.Natikar@amd.com>,
        Deucher Alexander <Alexander.Deucher@amd.com>,
        Mario Limonciello <mario.limonciello@amd.com>,
        Iain Lane <iain@orangesquash.org.uk>
Subject: [PATCH] PCI: Only put >= 2015 root ports into D3 on Intel
Date:   Mon, 15 May 2023 18:15:15 -0500
Message-ID: <20230515231515.1440-1-mario.limonciello@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2023 23:15:34.7822
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0ed3a954-b627-4a0a-c16b-08db559a497b
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT111.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7941
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Using an XHCI device to wakeup the system from s2idle fails when
that XHCI device is connected to a USB-C port for an AMD USB4
router.

Due to commit 9d26d3a8f1b0 ("PCI: Put PCIe ports into D3 during
suspend") all root port go into D3 during s2idle.
When the root ports are in D3 over s2idle it's not possible for the
platform firmware to properly identify the wakeup source.

As a user presses a key on a keyboard the APU will exit
hardware sleep, but no wake source will be active so the kernel will
let the APU enter back into a hardware sleep state.

Here is an example of that sequence of events.  The USB keyboard was
pressed after 11.9 seconds, and then a GPIO was triggered after
another 12 seconds.
```
PM: suspend-to-idle
ACPI: EC: ACPI EC GPE status set
ACPI: PM: Rearming ACPI SCI for wakeup
amd_pmc AMDI0007:00: SMU idlemask s0i3: 0x8fff9eb5
Timekeeping suspended for 11.985 seconds
PM: Triggering wakeup from IRQ 9
ACPI: EC: ACPI EC GPE status set
ACPI: EC: ACPI EC GPE dispatched
ACPI: EC: ACPI EC work flushed
ACPI: EC: ACPI EC work flushed
ACPI: PM: Rearming ACPI SCI for wakeup
amd_pmc AMDI0007:00: SMU idlemask s0i3: 0x8fff9eb5
PM: Triggering wakeup from IRQ 9
ACPI: EC: ACPI EC GPE status set
ACPI: PM: Rearming ACPI SCI for wakeup
amd_pmc AMDI0007:00: SMU idlemask s0i3: 0x8fff9eb5
Timekeeping suspended for 12.916 seconds
PM: Triggering wakeup from IRQ 9
PM: Triggering wakeup from IRQ 7
ACPI: EC: ACPI EC GPE status set
ACPI: EC: ACPI EC GPE dispatched
ACPI: EC: ACPI EC work flushed
ACPI: PM: Wakeup after ACPI Notify sync
PM: resume from suspend-to-idle
```

If the root ports are in D0 during s2idle, then the wake source is
properly identified and an IRQ is active for the root port, waking
the system up.

Here is the same sequence with root ports in D0.  The USB keyboard
was pressed after 11 seconds.
```
PM: suspend-to-idle
ACPI: EC: ACPI EC GPE status set
ACPI: PM: Rearming ACPI SCI for wakeup
amd_pmc AMDI0007:00: SMU idlemask s0i3: 0x8fff9eb5
Timekeeping suspended for 11.138 seconds
PM: Triggering wakeup from IRQ 9
ACPI: PM: ACPI non-EC GPE wakeup
PM: resume from suspend-to-idle
PM: Triggering wakeup from IRQ 40
```

Comparing registers between Linux and Windows 11 this behavior to put root
ports into D3 at suspend is unique to Linux. Windows does not put the
root ports into D3 over Modern Standby.

As this policy change to put root ports into D3 if they're manufactured
after 2015 was originally introduced for Intel systems narrow it down to
only apply there.

Reported-by: Iain Lane <iain@orangesquash.org.uk>
Closes: https://forums.lenovo.com/t5/Ubuntu/Z13-can-t-resume-from-suspend-with-external-USB-keyboard/m-p/5217121
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
---
 drivers/pci/pci.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 5ede93222bc1..7d1b078b8d40 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3010,12 +3010,17 @@ bool pci_bridge_d3_possible(struct pci_dev *bridge)
 		if (dmi_check_system(bridge_d3_blacklist))
 			return false;
 
+#ifdef CONFIG_X86
 		/*
-		 * It should be safe to put PCIe ports from 2015 or newer
-		 * to D3.
+		 * It should be safe to put PCIe ports from Intel systems
+		 * from 2015 or newer to D3.
+		 * Windows 11 does not do this over Modern Standby and this is
+		 * known to cause problems with s2idle on some AMD systems.
 		 */
-		if (dmi_get_bios_year() >= 2015)
+		if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL &&
+		    dmi_get_bios_year() >= 2015)
 			return true;
+#endif
 		break;
 	}
 
-- 
2.34.1