Received-SPF: pass (google.com: domain of linux-kernel+bounces-59719-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <ecb40fbc2dc50cc2ebe7cc5393c2b0d6da58e4f3.camel@linux.intel.com>
Subject: Re: [PATCH v2 1/2] PCI: Disable D3cold on Asus B1400 PCI-NVMe bridge
From: "David E. Box" <david.e.box@linux.intel.com>
Reply-To: david.e.box@linux.intel.com
To: Daniel Drake <drake@endlessos.org>
Cc: Bjorn Helgaas <helgaas@kernel.org>, tglx@linutronix.de,
 mingo@redhat.com,  bp@alien8.de, dave.hansen@linux.intel.com,
 x86@kernel.org, hpa@zytor.com,  linux-pci@vger.kernel.org,
 linux-kernel@vger.kernel.org, bhelgaas@google.com, 
 mario.limonciello@amd.com, rafael@kernel.org, lenb@kernel.org, 
 linux-acpi@vger.kernel.org, linux@endlessos.org
Date: Fri, 09 Feb 2024 09:19:19 -0800
In-Reply-To: <CAD8Lp44tO_pz_HZmPOKUQ-LEQT=c856eH52xWL9nBtAtJwjL1g@mail.gmail.com>
References: <20240207084452.9597-1-drake@endlessos.org>
	 <20240207200538.GA912749@bhelgaas>
	 <CAD8Lp47DjuAAxqwt+yKD22UNMyvqE00x0u+JeM74KO2OC+Otrg@mail.gmail.com>
	 <CAD8Lp44-8WhPyOrd2dCWyG3rRuCqzJ-aZCH6b1r0kyhfcXJ8xg@mail.gmail.com>
	 <9654146ac849bb00faf2fe963d3da94ad65003b8.camel@linux.intel.com>
	 <CAD8Lp44tO_pz_HZmPOKUQ-LEQT=c856eH52xWL9nBtAtJwjL1g@mail.gmail.com>
Organization: David E. Box
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4-0ubuntu2 
Precedence: bulk
MIME-Version: 1.0

On Fri, 2024-02-09 at 09:36 +0100, Daniel Drake wrote:
> On Thu, Feb 8, 2024 at 5:57=E2=80=AFPM David E. Box <david.e.box@linux.in=
tel.com>
> wrote:
> > This does look like a firmware bug. We've had reports of D3cold support
> > missing
> > when running in non-VMD mode on systems that were designed with VMD for
> > Windows.
> > These issues have been caught and addressed by OEMs during enabling of =
Linux
> > systems. Does D3cold work in VMD mode?
>=20
> On Windows for the VMD=3Don case, we only tested this on a BIOS with
> StorageD3Enable=3D0. The NVMe device and parent bridge stayed in D0 over
> suspend, but that's exactly what the BIOS asked for, so it doesn't
> really answer your question.
>=20
> On Linux with VMD=3Don and StorageD3Enable=3D1, the NVMe storage device
> and the VMD parent bridge are staying in D0 over suspend. I don't know
> why this is, I would have expected at least D3hot.

Yeah something is missing here. When StorageD3Enable is set, the nvme drive=
r
prints the following message during boot:

	"platform quirk: setting simple suspend"

If you don't see this, then the driver never saw StorageD3Enable=3D1. Possi=
ble
reasons are:

	- The property doesn't exist
	- The property isn't set under the ACPI companion device
	- There is no associated ACPI companion device
	- The "nvme=3Dnoacpi" parameter was passed on the kernel cmdline
	- The nvme driver was quirked to not use D3 with
	   NVME_QUIRK_FORCE_NO_SIMPLE_SUSPEND.

How was the D-state status confirmed? You can use the following to see the =
D
state of PCI devices during suspend in the kernel log:

echo -n "file pci-driver.c +p" > /sys/kernel/debug/dynamic_debug/control

David

> =C2=A0 However, given
> that the NVMe device has no firmware_node under the VMD=3Don setup, I
> believe there is no way it would enter D3cold because there's no
> linkage to an ACPI device, so no available _PS3 or _PR0 or whatever is
> the precise definition of D3cold.
>=20
> I also realise I may have made a bad assumption in my previous mail
> when looking at the Dell device: I was assuming that a parent PCI
> bridge cannot go into D3cold if its child devices only got as far as
> D3hot, but I now realise I'm not sure if that constraint actually
> exists.
>=20
> Not sure if these questions are relevant for the consideration of this
> patch, but I'll try to find some time to answer them next week.
>=20
> Daniel