Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Matthias Kaehlcke <mka@chromium.org>
Date:   Tue, 30 May 2023 21:06:19 +0000
To:     =?utf-8?B?5qWK5a6X57+w?= <ecs.taipeikernel@gmail.com>
Cc:     Bjorn Helgaas <helgaas@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Bob Moragues <moragues@google.com>,
        Abner Yen <abner.yen@ecs.com.tw>,
        Doug Anderson <dianders@chromium.org>,
        Stephen Boyd <swboyd@chromium.org>, Harvey <hunge@google.com>,
        Gavin Lee <gavin.lee@ecs.com.tw>,
        Bjorn Helgaas <bhelgaas@google.com>, linux-pci@vger.kernel.org
Subject: Re: [PATCH v1] drivers: pci: quirks: Add suspend fixup for SSD on
 sc7280
Message-ID: <ZHZlS48EKigmNihh@google.com>
References: <20230525163448.v1.1.Id388e4e2aa48fc56f9cd2d413aabd461ff81d615@changeid>
 <ZG/c1+/mCp/PfFSO@bhelgaas>
 <CAPao8GJNbXnh1R2-9rueMygyYyy-r3kqvQ55xdN61E7m6_dkdw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAPao8GJNbXnh1R2-9rueMygyYyy-r3kqvQ55xdN61E7m6_dkdw@mail.gmail.com>
Precedence: bulk

On Mon, May 29, 2023 at 02:24:53PM +0800, 楊宗翰 wrote:
> Hi Bjorn,
> 
> Thanks for your kindÂ directions.
> 
> Â  - Subject line in style of the file (use "git log --oneline
> Â  Â  drivers/pci/quirks.c").
> Done, and I resend in topic "[PATCH v1] PCI: Add suspend fixup for SSD
> Â on sc7280", please review it.
> 
> Â  - Format commit log correctly (fill 75 columns, no leading spaces).
> Done.
> 
> Â  - Description of incorrect behavior.Â  What does the user see?Â  If
> Â  Â  there's a bug report, include a link to it.
> This issue seems to be discovered in ChromeOS only. SSD will randomlyÂ 
> crashed at 100~250+ suspend/resume cycle. Phison and QualcommÂ 
> found that its due to NVMe entering D3cold instead of L1ss.

It should be noted that D3cold (or whatever condition that causes the
issue) is not always entered, but only in the failure case (at least
that was the case for the Kioxia NVMe, which has a similar issue).

> Â  - Multi-line code comments in style of the file (look at existing
> Â  Â  comments in the file).
> Done.
> 
> Â  - Details of "the correct ASPM state".Â  ASPM may be enabled or
> Â  Â  disabled by the user, so you can't assume any particular ASPM
> Â  Â  configuration.
> According to Qualcomm. This issue has been found last year and they have
> attempt to submit some patches to fix the pci suspend behavior.Â 
> (ref:https://patchwork.kernel.org/project/linux-arm-msm/list/?
> series=665060&state=%2A&archive=both).Â 
> But somehow these patches were rejected because of its complexity. AndÂ 
> we've got advise from Google that it will be more efficient that we implementÂ 
> a quirks to fix this issue.

IIRC the primary goal of this series was to be able to turn off the
PCI clocks during suspend, to allow the SoC to enter a lower power
state. This fixing element for NVMe with the issue described above
is the the retry loop of "PCI: qcom: Add retry logic for link to be
stable in L1ss" [1].

It is currently unclear why *some* NVMe *sometimes* need a longer
time to enter the L1 sub-state. That's something Qualcomm and the
vendors of impacted NVMes should figure out.

[1] https://patchwork.kernel.org/project/linux-arm-msm/patch/1659526134-22978-4-git-send-email-quic_krichai@quicinc.com/

> Â  - Details on the Qualcomm sc7280 connection.Â  This quirk would
> Â  Â  affect Phison SSDs on *all* platforms, not just sc7280.Â  I don't
> Â  Â  want to slow down suspend on all platforms just for a sc7280
> Â  Â  issue.

As of now the issue has only been observed on QC SC7280, I don't
know if ECS has tried this part on other platforms. The issue could
be QC/SC7280-specific or not.

> The DECLARE_PCI_FIXUP_SUSPEND function has already specify the PCI deviceÂ 
> ID. And this SSD will only be used at our Chromebook device only.

It could be used in devices that are produced by other manufacturers.

A dedicated Kconfig option for the Phison NVMe could be an option.
Or a QC specific #ifdef (ugh ...) with a comment explaining that the
issue has been only observed on QC SC7280 *so far*.