Received-SPF: pass (google.com: domain of linux-kernel+bounces-76378-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
From: "Shradha Todi" <shradha.t@samsung.com>
To: "'Manivannan Sadhasivam'" <manivannan.sadhasivam@linaro.org>,
	<bp@alien8.de>, <tony.luck@intel.com>, <james.morse@arm.com>,
	<mchehab@kernel.org>, <rric@kernel.org>
Cc: <lpieralisi@kernel.org>, <kw@linux.com>, <robh@kernel.org>,
	<bhelgaas@google.com>, <jingoohan1@gmail.com>,
	<gustavo.pimentel@synopsys.com>, <josh@joshtriplett.org>,
	<lukas.bulwahn@gmail.com>, <hongxing.zhu@nxp.com>,
	<pankaj.dubey@samsung.com>, <linux-kernel@vger.kernel.org>,
	<linux-pci@vger.kernel.org>, <vidyas@nvidia.com>, <gost.dev@samsung.com>,
	<alim.akhtar@samsung.com>
In-Reply-To: <20240216134921.GH2559@thinkpad>
Subject: RE: [PATCH v2 0/3] Add support for RAS DES feature in PCIe DW
 controller
Date: Thu, 22 Feb 2024 16:30:47 +0530
Message-ID: <120d01da657e$66b9d3b0$342d7b10$@samsung.com>
Precedence: bulk
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Thread-Index: AQG/Q0WxGB9EWFeblm7N70OL8blxhQGR1/IBAmU4G1oB/2O7JgJFQ8VeAXfQWL0B/DJ/LbDu3RAw
Content-Language: en-in
Content-Type: text/plain; charset="utf-8"
CMS-TYPE: 105P
DLP-Filter: Pass
References: <CGME20231130115055epcas5p4e29befa80877be45dbee308846edc0ba@epcas5p4.samsung.com>
	<20231130115044.53512-1-shradha.t@samsung.com>
	<20231130165514.GW3043@thinkpad>
	<000601da3e07$c39e5e00$4adb1a00$@samsung.com>
	<20240104055030.GA3031@thinkpad>
	<0df701da5ff0$df1165a0$9d3430e0$@samsung.com>
	<20240216134921.GH2559@thinkpad>

+ Borislav, Tony, James, Mauro, Robert

Hi All,

Synopsys DesignWare PCIe controllers have a vendor specific capability (whi=
ch
means that this set of registers are only present in DesignWare controllers=
)
to perform debug operations called =22RASDES=22.
The functionalities provided by this extended capability are:

1. Debug: This has some debug related diagnostic features like holding LTSS=
M
in certain states, reading the status of lane detection, checking if any PC=
Ie
lanes are broken (RX Valid) and so on. It's a debug only feature used for d=
iagnostic
use-cases.

2. Error Injection: This is a way to inject certain errors in PCIe like LCR=
C, ECRC,
Bad TLPs and so on. Again, this is a debug feature and generally not used i=
n
functional use-case.

3. Statistical counters: This has 3 parts
 - Error counters
 - Non error counters (covered as part of perf =5B1=5D)
 - Time based analysis counters (covered as part of perf =5B1=5D)

Selective features of  the above functionality has been implemented
by vendor specific PCIe controller drivers (pcie-tegra194.c) that use
Synopsys DesignWare PCIe controllers.
In order to make it useful to all vendors using DWC controller, we had
proposed a common implementation in DWC PCIe controller directory
(drivers/pci/controller/dwc/) and our original idea was based on debugfs
filesystem. v1 and v2 are mentioned in =5B2=5D and =5B3=5D.

We got a suggestion to implement this as part of EDAC framework =5B3=5D and
we looked into the same. But as far as I understood, what I am trying to
implement is a very specific feature (only valid for Synopsys DWC PCIe cont=
rollers).
This doesn't seem to fit in very well with the EDAC framework and we can=20
hardly use any of the EDAC framework APIs. We tried implementing a
=22pci_driver=22 but since a function driver will already be running on the=
 EP and
portdrv on the root-complex, we will not be able to bind 2 drivers to a sin=
gle
PCI device (root-complex or endpoint). Ultimately, what I will be doing is
writing a platform driver with debugfs entries which will be present in EDA=
C
directory instead of DWC directory.

Can  you please help us out by going through this thread =5B3=5D and lettin=
g us
know if our understanding is wrong at any point. If you think it is a bette=
r
idea to integrate this in the EDAC framework, can you guide me as
to how I can utilize the framework better?
Please let me know if you need any other information to conclude.

=5B1=5D https://lore.kernel.org/linux-pci/20231121013400.18367-1-xueshuai=
=40linux.alibaba.com/
=5B2=5D https://lore.kernel.org/all/20210518174618.42089-1-shradha.t=40sams=
ung.com/T/
=5B3=5D https://lore.kernel.org/all/20231130115044.53512-1-shradha.t=40sams=
ung.com/

Thanks,
Shradha

> -----Original Message-----
> From: 'Manivannan Sadhasivam' <manivannan.sadhasivam=40linaro.org>
> Sent: 16 February 2024 19:19
> To: Shradha Todi <shradha.t=40samsung.com>
> Cc: lpieralisi=40kernel.org; kw=40linux.com; robh=40kernel.org;
> bhelgaas=40google.com; jingoohan1=40gmail.com;
> gustavo.pimentel=40synopsys.com; josh=40joshtriplett.org;
> lukas.bulwahn=40gmail.com; hongxing.zhu=40nxp.com;
> pankaj.dubey=40samsung.com; linux-kernel=40vger.kernel.org; linux-
> pci=40vger.kernel.org; vidyas=40nvidia.com; gost.dev=40samsung.com
> Subject: Re: =5BPATCH v2 0/3=5D Add support for RAS DES feature in PCIe D=
W
> controller
>=20
> On Thu, Feb 15, 2024 at 02:55:06PM +0530, Shradha Todi wrote:
> >
> >
>=20
> =5B...=5D
>=20
> > > For the error injection and counters, we already have the EDAC
> > > framework. So adding them in the DWC driver doesn't make sense to me.
> > >
> >
> > Sorry for late response, was going through the EDAC framework to unders=
tand
> better how we can fit RAS DES support in it. Below are some technical cha=
llenges
> found so far:
> > 1: This debugfs framework proposed =5B1=5D can run on both side of the =
link i.e. RC
> and EP as it will be a part of the link controller platform driver. Here =
for the EP
> side the assumption is that it has Linux running, which is primarily a us=
e case for
> chip-to-chip communication.  After your suggestion to migrate to EDAC
> framework we studied and here are the findings:
> > - If we move to EDAC framework, we need to have RAS DES as a
> > pci_driver which will be binded based on vendor_id and device_id. Our
> > observation is that on EP side system we are unable to bind two
> > function driver (pci_driver), as pci_endpoint_test function driver or
> > some other chip-to-chip function driver will already be bound. On the
> > other hand, on RC side we observed that if we have portdrv enabled in
> > Linux running on RC system, it gets bound to RC controller and then it
> > does not allow EDAC pci_driver to bind. So basically we see a problem
> > here, that we can't have two pci_driver binding to same PCI device
> > 2: Another point is even though we use EDAC driver framework, we may no=
t be
> able to use any of EDAC framework APIs as they are mostly suitable for me=
mory
> controller devices sitting on PCI BUS. We will end up using debugfs entri=
es just via
> a pci_driver placed inside EDAC framework.
>=20
> Please wrap your replies to 80 characters.
>=20
> There is no need to bind the edac driver to VID:PID of the device. The ed=
ac driver
> can be a platform driver and you can instantiate the platform device from=
 the
> DWC driver. This way, the PCI device can be assocaited with whatever driv=
er, but
> still there can be a separate edac driver for handling errors.
>=20
> Regarding API limitation, you should ask the maintainer about the possibi=
lity of
> extending them.
>=20
> >
> > Please let me know if my understanding is wrong.
> >
> > > But first check with the perf driver author if they have any plans
> > > on adding the proposed functionality. If they do not have any plan
> > > or not working on it, then look into EDAC.
> > >
> > > - Mani
> > >
> >
> > Since we already worked and posted patches =5B1=5D, =5B2=5D, we will co=
ntinue to work
> on this and based on consent from community we will adopt to most suitabl=
e
> framework.
> > We see many subsystems like ethernet, usb, gpu, cxl having debugfs file=
s that
> give information about the current status of the running system and as of=
 now
> based on our findings, we still feel there is no harm in having debugfs e=
ntry based
> support in DesignWare controller driver itself.
>=20
> There is no issue in exposing the debug information through debugfs, that=
's the
> sole purpose of the interface. But here, you are trying to add support fo=
r DWC
> RAS feature for which a dedicated framework already exists.
>=20
> And there will be more similar requests coming for vendor specific error =
protocols
> as well. So your investigation could benefit everyone.
>=20
> From your above investigation, looks like there are some shortcomings of =
the
> EDAC framework. So let's get that clarified by writing to the EDAC mainta=
iners
> (keep us in CC). If the EDAC maintainer suggests you to add support for t=
his
> feature in DWC driver itself citing some reasons, then no issues with me.
>=20
> - Mani
>=20
> --
> =E0=AE=AE=E0=AE=A3=E0=AE=BF=E0=AE=B5=E0=AE=A3=E0=AF=8D=E0=AE=A3=E0=AE=A9=
=E0=AF=8D=20=E0=AE=9A=E0=AE=A4=E0=AE=BE=E0=AE=9A=E0=AE=BF=E0=AE=B5=E0=AE=AE=
=E0=AF=8D=0D=0A=0D=0A