Received-SPF: pass (google.com: domain of linux-kernel+bounces-94843-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <be760502-446b-f4cc-776d-8751bdb75ff7@quicinc.com>
Date: Thu, 7 Mar 2024 07:28:54 +0530
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.13.1
Subject: Re: [PATCH v2] PCI: dwc: Enable runtime pm of the host bridge
Content-Language: en-US
To: Bjorn Helgaas <helgaas@kernel.org>
CC: Jingoo Han <jingoohan1@gmail.com>,
        Gustavo Pimentel
	<gustavo.pimentel@synopsys.com>,
        Manivannan Sadhasivam
	<manivannan.sadhasivam@linaro.org>,
        Lorenzo Pieralisi
	<lpieralisi@kernel.org>,
        =?UTF-8?Q?Krzysztof_Wilczy=c5=84ski?=
	<kw@linux.com>,
        Rob Herring <robh@kernel.org>, Bjorn Helgaas
	<bhelgaas@google.com>,
        <linux-pci@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        <quic_vbadigan@quicinc.com>, <quic_ramkri@quicinc.com>,
        <quic_nitegupt@quicinc.com>, <quic_skananth@quicinc.com>,
        <quic_parass@quicinc.com>
References: <20240305195722.GA541937@bhelgaas>
From: Krishna Chaitanya Chundru <quic_krichai@quicinc.com>
In-Reply-To: <20240305195722.GA541937@bhelgaas>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit


On 3/6/2024 1:27 AM, Bjorn Helgaas wrote:
> On Tue, Mar 05, 2024 at 03:19:01PM +0530, Krishna chaitanya chundru wrote:
>> The Controller driver is the parent device of the PCIe host bridge,
>> PCI-PCI bridge and PCIe endpoint as shown below.
> 
> Nit: add blank line here.
> 
Ack.
>> 	PCIe controller(Top level parent & parent of host bridge)
>> 			|
>> 			v
>> 	PCIe Host bridge(Parent of PCI-PCI bridge)
>> 			|
>> 			v
>> 	PCI-PCI bridge(Parent of endpoint driver)
>> 			|
>> 			v
>> 		PCIe endpoint driver
> 
> Nit: use spaces instead of tabs to ensure this still looks good when
> "git log" indents this.  In this case it doesn't seem to matter,
> 
Ack.
>> Since runtime PM is disabled for host bridge, the state of the child
>> devices under the host bridge is not taken into account by PM framework
>> for the top level parent, PCIe controller. So PM framework, allows
>> the controller driver to enter runtime PM irrespective of the state
>> of the devices under the host bridge.
> 
> IIUC this says that we runtime suspend the controller even though
> runtime PM is disabled for the host bridge?  I have a hard time
> parsing this; can you cite a function that does this or some relevant
> documentation about how this part of runtime PM works?
> 
Generally controller should go to runtime suspend when endpoint client
drivers and pci-pci host bridge drivers goes to runtime suspend as the
controller driver is the parent, but we are observing controller driver
goes to runtime suspend even when client drivers and PCI-PCI bridge are
in active state.

Unfortunately I don't have any reference for any documentation about how
runtime PM works.
>> And this causes the topology breakage and also possible PM issues.
> 
> Not sure what this refers to, since you didn't mention topology
> breakage earlier.  And "possible PM" issues is too vague to be useful.
> 
>> So enable pm runtime for the host bridge, so that controller driver
>> goes to suspend only when all child devices goes to runtime suspend.
> 
> s/pm runtime/runtime PM/ so all the references match (unless you mean
> something different here) (also in subject line)
> 
>> Signed-off-by: Krishna chaitanya chundru <quic_krichai@quicinc.com>
>> ---
>> Changes in v2:
>> - Updated commit message as suggested by mani.
>> - Link to v1: https://lore.kernel.org/r/20240219-runtime_pm_enable-v1-1-d39660310504@quicinc.com
>> ---
>>   drivers/pci/controller/dwc/pcie-designware-host.c | 4 ++++
>>   1 file changed, 4 insertions(+)
>>
>> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
>> index d5fc31f8345f..57756a73df30 100644
>> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
>> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
>> @@ -16,6 +16,7 @@
>>   #include <linux/of_pci.h>
>>   #include <linux/pci_regs.h>
>>   #include <linux/platform_device.h>
>> +#include <linux/pm_runtime.h>
>>   
>>   #include "../../pci.h"
>>   #include "pcie-designware.h"
>> @@ -505,6 +506,9 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
>>   	if (pp->ops->post_init)
>>   		pp->ops->post_init(pp);
>>   
>> +	pm_runtime_set_active(&bridge->dev);
> 
> There are currently no callers of pm_runtime_set_active() in
> drivers/pci/controller/.  This adds it to dw_pcie_host_init(), but it
> doesn't seem to be a DWC-specific issue, so I assume other drivers and
> driver cores like cadence and mobiveil should have this, too?
> 
I think cadence and mobiveil also should have this.
>> +	pm_runtime_enable(&bridge->dev);
> 
> There are several existing calls of pci_runtime_enable(), including
> from several DWC drivers.  Are they now redundant?
> 
No These are not redundant, Those drivers are calling this
pm_runtime_enable() for their controller dev node. Here we are calling
runtime enable for the bridge dev which is allocated and used by the
PCIe framework.

- Krishna Chaitanya.
> In addition, [1] suggests that pm_runtime_enable() should be called
> *after* pm_runtime_set_active(), but these existing calls
> (dra7xx_pcie_probe(), ks_pcie_probe(), qcom_pcie_probe(),
> rcar_gen4_pcie_prepare(), tegra_pcie_config_rp()) happen *before*
> dw_pcie_host_init() calls pm_runtime_set_active().
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/power/runtime_pm.rst?id=v6.7#n582
>  >>   	return 0;
>>   
>>   err_stop_link:
>>
>> ---
>> base-commit: 6613476e225e090cc9aad49be7fa504e290dd33d
>> change-id: 20240219-runtime_pm_enable-bdc17914bd50
>>
>> Best regards,
>> -- 
>> Krishna chaitanya chundru <quic_krichai@quicinc.com>
>>