Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com;
From:   Mario Limonciello <mario.limonciello@amd.com>
To:     <mario.limonciello@amd.com>
CC:     "Rafael J. Wysocki" <rjw@rjwysocki.net>,
        Len Brown <lenb@kernel.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Robert Moore <robert.moore@intel.com>,
        Erik Kaneda <erik.kaneda@intel.com>,
        "open list:ACPI" <linux-acpi@vger.kernel.org>,
        open list <linux-kernel@vger.kernel.org>,
        "open list:PCI SUBSYSTEM" <linux-pci@vger.kernel.org>,
        "open list:ACPI COMPONENT ARCHITECTURE (ACPICA)" <devel@acpica.org>
Subject: [PATCH] PCI: Put power resources not tied to a physical node in D3cold
Date:   Thu, 7 Oct 2021 15:51:26 -0500
Message-ID: <20211007205126.11769-1-mario.limonciello@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2021 20:51:43.0126
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 19d9b98a-d8f8-45a1-5928-08d989d444e8
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT068.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1506
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

I found a case that a system that two physical SATA controllers share
the same ACPI Power Resource.  When a drive is connected to one of
the controllers then it will bind with PCI devices with the ahci driver
and form a relationship with the firmware node and physical node.  During
s2idle I see that the constraints are met for this device as it is
transitioned into the appropriate state. However the second ACPI node
doesn't have any relationship with a physical node and stays in "D0":

```
ACPI: \_SB_.PCI0.GP18.SATA: ACPI: PM: Power state change: D0 -> D3cold
ACPI: PM: Power resource [P0SA] still in use
acpi device:2a: Power state changed to D3cold
```

Due to the refcounting used on the shared power resource putting the
device with a physical node into D3 doesn't result in the _OFF method
being called.

To help with this type of problem, make a new helper function that can
be used to check all the children of an ACPI device and put any firmware
nodes that don't have physical devices into D3cold to allow shared
resources to transition. Call this helper function after PCI devices have
been scanned and ACPI companions have had a chance to associate.

After making this change, here is what the flow looks like:
```
<snip:bootup>
ACPI: \_SB_.PCI0.GP18.SAT1: ACPI: PM: Power state change: D0 -> D3cold
ACPI: PM: Power resource [P0SA] still in use
acpi device:2c: Power state changed to D3cold
<snip:suspend>
ACPI: \_SB_.PCI0.GP18.SATA: ACPI: PM: Power state change: D0 -> D3cold
ACPI: PM: Power resource [P0SA] turned off
acpi device:2a: Power state changed to D3cold
```

Link: https://lore.kernel.org/linux-acpi/0571292a-286b-18f2-70ad-12b125a61469@amd.com/T/#m042055c5ca1e49c2829655511f04b0311c142559
BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=214091
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
---
 drivers/acpi/device_pm.c | 34 ++++++++++++++++++++++++++++++++++
 drivers/pci/probe.c      |  5 +++++
 include/acpi/acpi_bus.h  |  1 +
 3 files changed, 40 insertions(+)

diff --git a/drivers/acpi/device_pm.c b/drivers/acpi/device_pm.c
index 0028b6b51c87..0fb0bbeeae9e 100644
--- a/drivers/acpi/device_pm.c
+++ b/drivers/acpi/device_pm.c
@@ -149,6 +149,40 @@ static int acpi_dev_pm_explicit_set(struct acpi_device *adev, int state)
 	return 0;
 }
 
+/**
+ * acpi_device_turn_off_absent_children - Turn off power resources for
+ *					  children not physically present.
+ * @parent: ACPI bridge device
+ */
+int acpi_device_turn_off_absent_children(struct acpi_device *parent)
+{
+	struct acpi_device *adev;
+	int ret = 0;
+
+	if (!parent)
+		return -EINVAL;
+
+	list_for_each_entry(adev, &parent->children, node) {
+		int state;
+
+		if (!adev->flags.power_manageable ||
+		    !adev->power.flags.power_resources)
+			continue;
+		if (acpi_get_first_physical_node(adev))
+			continue;
+		ret = acpi_device_get_power(adev, &state);
+		if (ret)
+			return ret;
+		if (state == ACPI_STATE_D3_COLD)
+			continue;
+		ret = acpi_device_set_power(adev, ACPI_STATE_D3_COLD);
+		if (ret)
+			return ret;
+	}
+	return ret;
+}
+EXPORT_SYMBOL_GPL(acpi_device_turn_off_absent_children);
+
 /**
  * acpi_device_set_power - Set power state of an ACPI device.
  * @device: Device to set the power state of.
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 79177ac37880..1a45182394d1 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2939,6 +2939,11 @@ static unsigned int pci_scan_child_bus_extend(struct pci_bus *bus,
 		}
 	}
 
+	/* check for and turn off dangling power resources */
+	for_each_pci_bridge(dev, bus) {
+		acpi_device_turn_off_absent_children(ACPI_COMPANION(&dev->dev));
+	}
+
 	/*
 	 * We've scanned the bus and so we know all about what's on
 	 * the other side of any bridges that may be on this bus plus
diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h
index 13d93371790e..0eba08b60e13 100644
--- a/include/acpi/acpi_bus.h
+++ b/include/acpi/acpi_bus.h
@@ -510,6 +510,7 @@ int acpi_bus_get_status(struct acpi_device *device);
 
 int acpi_bus_set_power(acpi_handle handle, int state);
 const char *acpi_power_state_string(int state);
+int acpi_device_turn_off_absent_children(struct acpi_device *parent);
 int acpi_device_set_power(struct acpi_device *device, int state);
 int acpi_bus_init_power(struct acpi_device *device);
 int acpi_device_fix_up_power(struct acpi_device *device);
-- 
2.25.1