Received-SPF: pass (google.com: domain of linux-kernel+bounces-67987-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240207111854.576402-2-jhp@endlessos.org> <0e7944d410664153b506ea584d92cd6bb0a93f6a.camel@linux.intel.com>
In-Reply-To: <0e7944d410664153b506ea584d92cd6bb0a93f6a.camel@linux.intel.com>
From: Jian-Hong Pan <jhp@endlessos.org>
Date: Fri, 16 Feb 2024 11:13:54 +0800
Message-ID: <CAPpJ_edMhGY+4ULvhV0r5TLZ_QyLCNRewTVHdQcjCv--Qa4X+w@mail.gmail.com>
Subject: Re: [PATCH v3 3/3] PCI/ASPM: Fix L1SS parameters & only enable
 supported features when enable link state
To: david.e.box@linux.intel.com
Cc: Bjorn Helgaas <helgaas@kernel.org>, Johan Hovold <johan@kernel.org>, 
	=?UTF-8?Q?Ilpo_J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>, 
	Mika Westerberg <mika.westerberg@linux.intel.com>, 
	Nirmal Patel <nirmal.patel@linux.intel.com>, 
	Jonathan Derrick <jonathan.derrick@linux.dev>, 
	Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>, linux-pci@vger.kernel.org, 
	linux-kernel@vger.kernel.org, linux@endlessos.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

David E. Box <david.e.box@linux.intel.com> =E6=96=BC 2024=E5=B9=B42=E6=9C=
=888=E6=97=A5 =E9=80=B1=E5=9B=9B =E4=B8=8A=E5=8D=8812:02=E5=AF=AB=E9=81=93=
=EF=BC=9A
>
> On Wed, 2024-02-07 at 19:18 +0800, Jian-Hong Pan wrote:
> > The original __pci_enable_link_state() configs the links directly witho=
ut:
> > * Check the L1 substates features which are supported, or not
> > * Calculate & program related parameters for L1.2, such as T_POWER_ON,
> >   Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD
> >
> > This leads some supported L1 PM substates of the link between VMD remap=
ped
> > PCIe Root Port and NVMe get wrong configs when a caller tries to enable=
d
> > them.
> >
> > Here is a failed example on ASUS B1400CEAE with enabled VMD:
> >
> > Capabilities: [900 v1] L1 PM Substates
> >         L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1-
> > L1_PM_Substates+
> >                   PortCommonModeRestoreTime=3D32us PortTPowerOnTime=3D1=
0us
> >         L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1-
> >                    T_CommonMode=3D0us LTR1.2_Threshold=3D0ns
> >         L1SubCtl2: T_PwrOn=3D10us
> >
> > This patch initializes the link's L1 PM substates to get the supported
> > features and programs relating paramters, if some of them are going to =
be
> > enabled in __pci_enable_link_state(). Then, enables the L1 PM substates=
 if
> > the caller intends to enable them and they are supported.
> >
> > Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D218394
> > Signed-off-by: Jian-Hong Pan <jhp@endlessos.org>
> > ---
> > v2:
> > - Prepare the PCIe LTR parameters before enable L1 Substates
> >
> > v3:
> > - Only enable supported features for the L1 Substates part
> >
> >  drivers/pci/pcie/aspm.c | 12 +++++++-----
> >  1 file changed, 7 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> > index a39d2ee744cb..c866971cae70 100644
> > --- a/drivers/pci/pcie/aspm.c
> > +++ b/drivers/pci/pcie/aspm.c
> > @@ -1389,14 +1389,16 @@ static int __pci_enable_link_state(struct pci_d=
ev
> > *pdev, int state, bool locked)
> >                 link->aspm_default |=3D ASPM_STATE_L0S;
> >         if (state & PCIE_LINK_STATE_L1)
> >                 link->aspm_default |=3D ASPM_STATE_L1;
> > -       /* L1 PM substates require L1 */
> > -       if (state & PCIE_LINK_STATE_L1_1)
> > +       if (state & ASPM_STATE_L1_2_MASK)
> > +               aspm_l1ss_init(link);
>
> This mixes ASPM_STATE flags with PCIE_LINK_STATE register mapping. This m=
ay work
> but I don't know if it's intended to. Rather do,
>
>     if (link->default & ASPM_STATE_L1_2_MASK)
>
> after collecting all of the states to be enabled.
>
> I understand that you are calling aspm_l1ss_init() to do the L1.2 calcula=
tions
> but it does more than this that you don't need. Maybe it would be more
> appropriate to call aspm_calc_l12_info() directly through an additional f=
unction
> that finds the parent and determines both ends of the link support L1SS.

After think & check twice, focusing on fixing the L1.2 parameters makes sen=
se.

I am preparing a new patch.  Thanks!

Jian-Hong Pan

> > +       /* L1 PM substates require L1 and should be in supported list *=
/
> > +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_1)
> >                 link->aspm_default |=3D ASPM_STATE_L1_1 | ASPM_STATE_L1=
;
> > -       if (state & PCIE_LINK_STATE_L1_2)
> > +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_2)
> >                 link->aspm_default |=3D ASPM_STATE_L1_2 | ASPM_STATE_L1=
;
> > -       if (state & PCIE_LINK_STATE_L1_1_PCIPM)
> > +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_1_PCIPM)
> >                 link->aspm_default |=3D ASPM_STATE_L1_1_PCIPM | ASPM_ST=
ATE_L1;
> > -       if (state & PCIE_LINK_STATE_L1_2_PCIPM)
> > +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_2_PCIPM)
> >                 link->aspm_default |=3D ASPM_STATE_L1_2_PCIPM | ASPM_ST=
ATE_L1;
> >         pcie_config_aspm_link(link, policy_to_aspm_state(link));
> >
>
> I don't think these changes are necessary. pcie_config_aspm_link() alread=
y
> checks link->aspm_capable which was initialized from link->aspm_support.
>
> David