Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220706123244.18056-1-kai.heng.feng@canonical.com>
In-Reply-To: <20220706123244.18056-1-kai.heng.feng@canonical.com>
From:   Kai-Heng Feng <kai.heng.feng@canonical.com>
Date:   Mon, 15 Aug 2022 15:55:36 +0800
Message-ID: <CAAd53p4Kg=mu8boPY-jGefsqxSBLdo6WPYHv+=eD5ZYz3_1AXw@mail.gmail.com>
Subject: Re: [PATCH] PCI: Disable upstream port PTM during suspend
To:     bhelgaas@google.com
Cc:     mika.westerberg@linux.intel.com, koba.ko@canonical.com,
        "David E . Box" <david.e.box@linux.intel.com>,
        Sathyanarayanan Kuppuswamy 
        <sathyanarayanan.kuppuswamy@linux.intel.com>,
        linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, Jul 6, 2022 at 8:33 PM Kai-Heng Feng
<kai.heng.feng@canonical.com> wrote:
>
> On Intel Alder Lake platforms, Thunderbolt entering D3cold can cause
> some errors reported by AER:
> pcieport 0000:00:1d.0: AER: Uncorrected (Non-Fatal) error received: 0000:00:1d.0
> pcieport 0000:00:1d.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
> pcieport 0000:00:1d.0:   device [8086:7ab0] error status/mask=00100000/00004000
> pcieport 0000:00:1d.0:    [20] UnsupReq               (First)
> pcieport 0000:00:1d.0: AER:   TLP Header: 34000000 08000052 00000000 00000000
> thunderbolt 0000:0a:00.0: AER: can't recover (no error_detected callback)
> xhci_hcd 0000:3e:00.0: AER: can't recover (no error_detected callback)
> pcieport 0000:00:1d.0: AER: device recovery failed
>
> In addition to that, it can also block system from suspending when
> a Thunderbolt dock is attached to the same system.
>
> The original approach [1] is to disable AER and DPC when link is in
> L2/L3 Ready, L2 and L3, but Bjorn identified the root cause is the Unsupported
> Request:
>   - 08:00.0 sent a PTM Request Message (a Posted Request)
>   - 00:1d.0 received the PTM Request Message
>   - The link transitioned to DL_Down
>   - Per sec 2.9.1, 00:1d.0 discarded the Request and reported an
>     Unsupported Request
>   - Or, per sec 6.21.3, if 00:1d.0 received a PTM Request when its
>     own PTM Enable was clear, it would also be treated as an
>     Unsupported Request
>
> And further: 'David did something like this [1], but just for Root Ports. That
> looks wrong to me because sec 6.21.3 says we should not have PTM enabled in an
> Upstream Port (i.e., in a downstream device like 08:00.0) unless it is already
> enabled in the Downstream Port (i.e., in the Root Port 00:1d.0).'
>
> So also disable upstream port PTM to make the PCI driver conform to the spec
> and solve the issue.
>
> [1] https://lore.kernel.org/all/20220408153159.106741-1-kai.heng.feng@canonical.com/
> [2] https://lore.kernel.org/all/20220422222433.GA1464120@bhelgaas/
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215453
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=216210
> Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
> Cc: David E. Box <david.e.box@linux.intel.com>
> Cc: Sathyanarayanan Kuppuswamy <sathyanarayanan.kuppuswamy@linux.intel.com>
>
> Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com>

A gentle ping...

> ---
>  drivers/pci/pci.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index cfaf40a540a82..8ba8a0e12946e 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -2717,7 +2717,8 @@ int pci_prepare_to_sleep(struct pci_dev *dev)
>          * port to enter a lower-power PM state and the SoC to reach a
>          * lower-power idle state as a whole.
>          */
> -       if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT)
> +       if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT ||
> +           pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM)
>                 pci_disable_ptm(dev);
>
>         pci_enable_wake(dev, target_state, wakeup);
> @@ -2775,7 +2776,8 @@ int pci_finish_runtime_suspend(struct pci_dev *dev)
>          * port to enter a lower-power PM state and the SoC to reach a
>          * lower-power idle state as a whole.
>          */
> -       if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT)
> +       if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT ||
> +           pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM)
>                 pci_disable_ptm(dev);
>
>         __pci_enable_wake(dev, target_state, pci_dev_run_wake(dev));
> --
> 2.36.1
>