Received-SPF: pass (google.com: domain of linux-kernel+bounces-56413-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240207084452.9597-1-drake@endlessos.org> <CAPpJ_eeV45QErAYRADUAHJUsGP0q5DqQJBQsgDdGZL0H0Q1isA@mail.gmail.com>
In-Reply-To: <CAPpJ_eeV45QErAYRADUAHJUsGP0q5DqQJBQsgDdGZL0H0Q1isA@mail.gmail.com>
From: Jian-Hong Pan <jhp@endlessos.org>
Date: Wed, 7 Feb 2024 19:43:51 +0800
Message-ID: <CAPpJ_eeAOjdERye=Br6r0LgzEGf_OnpD+osTGHahjTQ6PdioWw@mail.gmail.com>
Subject: Re: [PATCH v2 1/2] PCI: Disable D3cold on Asus B1400 PCI-NVMe bridge
To: Daniel Drake <drake@endlessos.org>
Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, 
	dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, 
	linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, bhelgaas@google.com, 
	david.e.box@linux.intel.com, mario.limonciello@amd.com, rafael@kernel.org, 
	lenb@kernel.org, linux-acpi@vger.kernel.org, linux@endlessos.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Jian-Hong Pan <jhp@endlessos.org> =E6=96=BC 2024=E5=B9=B42=E6=9C=887=E6=97=
=A5 =E9=80=B1=E4=B8=89 =E4=B8=8B=E5=8D=885:06=E5=AF=AB=E9=81=93=EF=BC=9A
>
> Daniel Drake <drake@endlessos.org> =E6=96=BC 2024=E5=B9=B42=E6=9C=887=E6=
=97=A5 =E9=80=B1=E4=B8=89 =E4=B8=8B=E5=8D=884:44=E5=AF=AB=E9=81=93=EF=BC=9A
> >
> > The Asus B1400 with original shipped firmware versions and VMD disabled
> > cannot resume from suspend: the NVMe device becomes unresponsive and
> > inaccessible.
> >
> > This is because the NVMe device and parent PCI bridge get put into D3co=
ld
> > during suspend, and this PCI bridge cannot be recovered from D3cold mod=
e:
> >
> >   echo "0000:01:00.0" > /sys/bus/pci/drivers/nvme/unbind
> >   echo "0000:00:06.0" > /sys/bus/pci/drivers/pcieport/unbind
> >   setpci -s 00:06.0 CAP_PM+4.b=3D03 # D3hot
> >   acpidbg -b "execute \_SB.PC00.PEG0.PXP._OFF"
> >   acpidbg -b "execute \_SB.PC00.PEG0.PXP._ON"
> >   setpci -s 00:06.0 CAP_PM+4.b=3D0 # D0
> >   echo "0000:00:06.0" > /sys/bus/pci/drivers/pcieport/bind
> >   echo "0000:01:00.0" > /sys/bus/pci/drivers/nvme/bind
> >   # NVMe probe fails here with -ENODEV
> >
> > This appears to be an untested D3cold transition by the vendor; Intel
> > socwatch shows that Windows leaves the NVMe device and parent bridge in=
 D0
> > during suspend, even though these firmware versions have StorageD3Enabl=
e=3D1.
> >
> > Experimenting with the DSDT, the _OFF method calls DL23() which sets a =
L23E
> > bit at offset 0xe2 into the PCI configuration space for this root port.
> > This is the specific write that the _ON routine is unable to recover fr=
om.
> > This register is not documented in the public chipset datasheet.
> >
> > Disallow D3cold on the PCI bridge to enable successful suspend/resume.
> >
> > Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D215742
> > Signed-off-by: Daniel Drake <drake@endlessos.org>
>
> Signed-off-by: Jian-Hong Pan <jhp@endlessos.org>

Change to Acked-by: Jian-Hong Pan <jhp@endlessos.org>

> > ---
> >  arch/x86/pci/fixup.c | 45 ++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 45 insertions(+)
> >
> > v2:
> > Match only specific BIOS versions where this quirk is required.
> > Add subsequent patch to this series to revert the original S3 workaroun=
d
> > now that s2idle is usable again.
> >
> > diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
> > index f347c20247d30..6b0b341178e4f 100644
> > --- a/arch/x86/pci/fixup.c
> > +++ b/arch/x86/pci/fixup.c
> > @@ -907,6 +907,51 @@ static void chromeos_fixup_apl_pci_l1ss_capability=
(struct pci_dev *dev)
> >  DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x5ad6, chromeos_save_apl=
_pci_l1ss_capability);
> >  DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, 0x5ad6, chromeos_fixup_a=
pl_pci_l1ss_capability);
> >
> > +/*
> > + * Disable D3cold on Asus B1400 PCIe bridge at 00:06.0.
> > + *
> > + * On this platform with VMD off, the NVMe's parent PCI bridge cannot
> > + * successfully power back on from D3cold, resulting in unresponsive N=
VMe on
> > + * resume. This appears to be an untested transition by the vendor: Wi=
ndows
> > + * leaves the NVMe and parent bridge in D0 during suspend.
> > + * This is only needed on BIOS versions before 308; the newer versions=
 flip
> > + * StorageD3Enable from 1 to 0.
> > + */
> > +static const struct dmi_system_id asus_nvme_broken_d3cold_table[] =3D =
{
> > +       {
> > +               .matches =3D {
> > +                               DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMP=
UTER INC."),
> > +                               DMI_MATCH(DMI_BIOS_VERSION, "B1400CEAE.=
304"),
> > +               },
> > +       },
> > +       {
> > +               .matches =3D {
> > +                               DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMP=
UTER INC."),
> > +                               DMI_MATCH(DMI_BIOS_VERSION, "B1400CEAE.=
305"),
> > +               },
> > +       },
> > +       {
> > +               .matches =3D {
> > +                               DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMP=
UTER INC."),
> > +                               DMI_MATCH(DMI_BIOS_VERSION, "B1400CEAE.=
306"),
> > +               },
> > +       },
> > +       {
> > +               .matches =3D {
> > +                               DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMP=
UTER INC."),
> > +                               DMI_MATCH(DMI_BIOS_VERSION, "B1400CEAE.=
307"),
> > +               },
> > +       },
> > +       {}
> > +};
> > +
> > +static void asus_disable_nvme_d3cold(struct pci_dev *pdev)
> > +{
> > +       if (dmi_check_system(asus_nvme_broken_d3cold_table) > 0)
> > +               pci_d3cold_disable(pdev);
> > +}
> > +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x9a09, asus_disable_nvme=
_d3cold);
> > +
> >  #ifdef CONFIG_SUSPEND
> >  /*
> >   * Root Ports on some AMD SoCs advertise PME_Support for D3hot and D3c=
old, but
> > --
> > 2.43.0
> >