Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <5fb1e6d9-00c0-33f2-753e-2338010da352@quicinc.com>
Date:   Sun, 25 Sep 2022 07:23:02 +0530
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.9.1
Subject: Re: [PATCH v7 1/5] PCI: qcom: Add system suspend and resume support
Content-Language: en-US
To:     Bjorn Helgaas <helgaas@kernel.org>
CC:     <linux-pci@vger.kernel.org>, <linux-arm-msm@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>, <mka@chromium.org>,
        <quic_vbadigan@quicinc.com>, <quic_hemantk@quicinc.com>,
        <quic_nitegupt@quicinc.com>, <quic_skananth@quicinc.com>,
        <quic_ramkri@quicinc.com>, <manivannan.sadhasivam@linaro.org>,
        <swboyd@chromium.org>, <dmitry.baryshkov@linaro.org>,
        <svarbanov@mm-sol.com>, <agross@kernel.org>,
        <andersson@kernel.org>, <konrad.dybcio@somainline.org>,
        <lpieralisi@kernel.org>, <robh@kernel.org>, <kw@linux.com>,
        <bhelgaas@google.com>, <linux-phy@lists.infradead.org>,
        <vkoul@kernel.org>, <kishon@ti.com>, <mturquette@baylibre.com>,
        <linux-clk@vger.kernel.org>,
        Bjorn Andersson <bjorn.andersson@linaro.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, <linux-pm@vger.kernel.org>
References: <20220923142611.GA1389970@bhelgaas>
From:   Krishna Chaitanya Chundru <quic_krichai@quicinc.com>
In-Reply-To: <20220923142611.GA1389970@bhelgaas>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 9/23/2022 7:56 PM, Bjorn Helgaas wrote:
> On Fri, Sep 23, 2022 at 07:29:31AM +0530, Krishna Chaitanya Chundru wrote:
>> On 9/23/2022 12:12 AM, Bjorn Helgaas wrote:
>>> On Thu, Sep 22, 2022 at 09:09:28PM +0530, Krishna Chaitanya Chundru wrote:
>>>> On 9/21/2022 10:26 PM, Bjorn Helgaas wrote:
>>>>> On Wed, Sep 21, 2022 at 03:23:35PM +0530, Krishna Chaitanya Chundru wrote:
>>>>>> On 9/20/2022 11:46 PM, Bjorn Helgaas wrote:
>>>>>>> On Tue, Sep 20, 2022 at 03:52:23PM +0530, Krishna chaitanya chundru wrote:
>>>>>>>> In qcom platform PCIe resources( clocks, phy etc..) can
>>>>>>>> released when the link is in L1ss to reduce the power
>>>>>>>> consumption. So if the link is in L1ss, release the PCIe
>>>>>>>> resources. And when the system resumes, enable the PCIe
>>>>>>>> resources if they released in the suspend path.
>>>>>>> What's the connection with L1.x?  Links enter L1.x based on
>>>>>>> activity and timing.  That doesn't seem like a reliable
>>>>>>> indicator to turn PHYs off and disable clocks.
>>>>>> This is a Qcom PHY-specific feature (retaining the link state in
>>>>>> L1.x with clocks turned off).  It is possible only with the link
>>>>>> being in l1.x. PHY can't retain the link state in L0 with the
>>>>>> clocks turned off and we need to re-train the link if it's in L2
>>>>>> or L3. So we can support this feature only with L1.x.  That is
>>>>>> the reason we are taking l1.x as the trigger to turn off clocks
>>>>>> (in only suspend path).
>>>>> This doesn't address my question.  L1.x is an ASPM feature, which
>>>>> means hardware may enter or leave L1.x autonomously at any time
>>>>> without software intervention.  Therefore, I don't think reading the
>>>>> current state is a reliable way to decide anything.
>>>> After the link enters the L1.x it will come out only if there is
>>>> some activity on the link.  AS system is suspended and NVMe driver
>>>> is also suspended( queues will  freeze in suspend) who else can
>>>> initiate any data.
>>> I don't think we can assume that nothing will happen to cause exit
>>> from L1.x.  For instance, PCIe Messages for INTx signaling, LTR, OBFF,
>>> PTM, etc., may be sent even though we think the device is idle and
>>> there should be no link activity.
>> I don't think after the link enters into L1.x there will some
>> activity on the link as you mentioned, except for PCIe messages like
>> INTx/MSI/MSIX. These messages also will not come because the client
>> drivers like NVMe will keep their device in the lowest power mode.
>>
>> The link will come out of L1.x only when there is config or memory
>> access or some messages to trigger the interrupts from the devices.
>> We are already making sure this access will not be there in S3.  If
>> the link is in L0 or L0s what you said is expected but not in L1.x
> Forgive me for being skeptical, but we just spent a few months
> untangling the fact that some switches send PTM request messages even
> when they're in a non-D0 state.  We expected that devices in D3hot
> would not send such messages because "why would they?"  But it turns
> out the spec allows that, and they actually *do*.
>
> I don't think it's robust interoperable design for a PCI controller
> driver like qcom to assume anything about PCI devices unless it's
> required by the spec.
>
> Bjorn
We will check the spec once and will come back to you,