Received-SPF: pass (google.com: domain of linux-kernel+bounces-67554-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <77b41973-e021-471c-89ad-f8cb8752e278@nvidia.com>
Date: Fri, 16 Feb 2024 00:39:23 +0530
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V16 13/13] PCI: tegra: Add Tegra194 PCIe support
Content-Language: en-US
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>, bhelgaas@google.com,
 robh+dt@kernel.org, mark.rutland@arm.com, thierry.reding@gmail.com,
 jonathanh@nvidia.com, Kishon Vijay Abraham I <kishon@kernel.org>,
 catalin.marinas@arm.com, Will Deacon <will@kernel.org>,
 jingoohan1@gmail.com, gustavo.pimentel@synopsys.com, digetx@gmail.com,
 mperttunen@nvidia.com, linux-pci@vger.kernel.org,
 devicetree@vger.kernel.org, linux-tegra@vger.kernel.org,
 linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
 kthota@nvidia.com, mmaddireddy@nvidia.com, sagar.tv@gmail.com
References: <20240215160157.GA1291755@bhelgaas>
From: Vidya Sagar <vidyas@nvidia.com>
In-Reply-To: <20240215160157.GA1291755@bhelgaas>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VFpiNmFmTSsyNXNxOVYrUXJuWjVqQ0xpL1JGMkFZS3F1T0FXc2paUk80Yjdk?=
 =?utf-8?B?K1gwdEZtVS9aeUoyRXJYL20xalE0UUZyQzZWdGpLUE9CVTdoWXQzR2Fwb1Rq?=
 =?utf-8?B?dGtTR2c5UlhBcVVjeTF2Rm4zR2tNVTJxUkZjK1ZnSWUyQkpNTFBkaWhacXlS?=
 =?utf-8?B?SzE2aU5OSkhhblhQbTlUV1RQQUp2dHBTT3htSFlaeitGY2ZHVmFDcVVrQ3Yz?=
 =?utf-8?B?eEFodC95N2pKSG9pRlNwSVhnRGxtek84cVJxSzVZOCs2RmNYdWlnb3NaN2pQ?=
 =?utf-8?B?WTNFbXFJR01Ec015V3dIdE55QTd1OXZEUlg3dFZTeGJ3TExKQ3Q3WG5VZ3Zr?=
 =?utf-8?B?MTdSbEtSeXhDZlEwRDNwZUwwcTZWc1ZHQjBhYjZrVnZ0QUlsNHgxK0dhQmxK?=
 =?utf-8?B?Q3pxaC9sTnNwZktLRHcvQzhkbktLdnhPRFYydGN4RDhpYnYxSVpZcWsrZ2pV?=
 =?utf-8?B?bkxKYi8vMG9NcWNsV0k2am1BVXFOQnE4S1VVMkFHelpSbFRJZG9LdEtpRU92?=
 =?utf-8?B?VlhxNnlaRS9zTzQ2cnZBVC9WUk1ZOFZMQTBZYU1jUWUvOTN2QjJ0VEpZV0J4?=
 =?utf-8?B?aUNTOEViM3hBZ0NjeFZqalNsZDErYktzdkRnRkhSQVV0cDdxZG1ucGRIM0Ix?=
 =?utf-8?B?a3JBVDRvMDFML1dVeVkxVWpCMWhYYUtNcnVLeUszTTJWb0NiNmx4N0VyK3R1?=
 =?utf-8?B?RHVFT3RPZUk5ZkxwbVQ5ZDJsNm12UjBOZ1Y4bGVNdStUMVJqekgwSVU3N2ht?=
 =?utf-8?B?QUtMbFhCUVRoOGVLQlloRkxMdmU2NEwwR3pLaVdxUDl6d1BVT290allXRHZT?=
 =?utf-8?B?NG5BVkRyY3Q5Nzd5VXZMamVjcDMvU210VTJvakZXUldYZEtBdXJ6Q2lhTGp0?=
 =?utf-8?B?K2xNQ1B6UXQ4T1JiSEJwaGNwajBER1RMdGRUZDlZcGVKclNINk1VRWoyZEM3?=
 =?utf-8?B?dENOdk1zZjJTVnBBdCtsWUZwZWxMTDFUY1k3ZmVWc2J1UEx5dnpVN1BQUm5u?=
 =?utf-8?B?UW5weTBrbXMwZEhxRUZtdW9sSllsb2UvSG1kM1YvQmJ2cnJaaTRNUHNqWTYz?=
 =?utf-8?B?SzE0UE1zc245UGRYNkRrS3dkY0pFSlRFT3NQdDRObTdrdVJ1ZzF6MnNZWDhs?=
 =?utf-8?B?aTNmbExOYVJDYjVqUkplcmJtL1lNQVFHMjlOL21PakFzOGd1aExCOGltd3Y5?=
 =?utf-8?B?NjNVczNKQktSY3p3bWtGcGFSaFplVEtmSnNyWWhrRlhXNGlvcjBXTFBQVmtF?=
 =?utf-8?B?Q0Y5NlpBNnoyaUYvUVBHN09IMkF2Tm0wRTlob0Yzclh6YXkyR0FQVlBrR0k1?=
 =?utf-8?B?bkNSMnUwem9aaU1yeTVRVHRlTDkzRjF5bSsrVXA3Z0ZsVGxrNHBIckwvQVln?=
 =?utf-8?B?b002NVVVZkYvVStTd2E1TlFiN1FOY1hUT1Nra2lUL1A0azJ1UWlyclU4OXN5?=
 =?utf-8?B?U25nUXZ0VHFsNGxPZTYveVgrVkdFZ0pYQWNYWWl5dVQ0QUhMbERDSHlQTkVD?=
 =?utf-8?B?em94cVBCeVZzd28yNjRuRkJMaTFnZUwrMTNPWithWm9OdUtrQ3VrdU9WNWt3?=
 =?utf-8?B?YlZIcjVpT01BMXR5OEZxbWNCN08rcVVaNllEMGlnamh6cXJyRVR1RXVYMER0?=
 =?utf-8?B?TDk1M2R4THNoK3ZWVTcwMUNySEtrbUpCZ21NWk5aZ3k3MUpNRi92eUtRVXQv?=
 =?utf-8?B?N0doalBkMVNBcDZHOHBOeHRIWGFXSnJHVmUxM0swQ1BnWkNQNy81NnY5MEds?=
 =?utf-8?B?WTE4cW14cU8yMk1sRDRjWDVHNFNoQXd6cGZUdkVRT0JwbzYrUFlvQ0NzbHhu?=
 =?utf-8?B?Rng0azNlVmJZQmRXTDlzT1pLeWEvL3g5dUNOMTVrbFVrZEszQjh4TndET3Aw?=
 =?utf-8?B?VDNYRSsxOXBkUEp3YlhrUm1oRGM3c1p6QUJtWmJDYlFqNEtYMVBHTDBUdGlK?=
 =?utf-8?B?Rm5RdjkrZUVRQjczZ0NEdkdlQy9lVUlXN1d4bUgvd2wwemhDRnRyMHZGZ3hU?=
 =?utf-8?B?MDY0UFd0UjIrdXk4aUhYM3RjRm54M0ZLc3h3RzNsR0hHYkVEOURCSHlmQ2FG?=
 =?utf-8?B?R203eHdzcmtXSVg5dzlQbko1MitobS90cERLd2U1YmJJUnhZUHZlL0J0ZzUz?=
 =?utf-8?Q?eXsyM8U39Wn8N5VXNFUEymspH?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bf71be81-adf7-4b10-c0ea-08dc2e59a594
X-MS-Exchange-CrossTenant-AuthSource: PH8PR12MB6674.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2024 19:09:35.0251
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /2HUjzGuoVu9sCshRNP/nHOO0d1oX6xuec/IjGFGpQLQngB6KEphSuagLdbbFLEGXvx0reOCIVYhTbO1cOajJA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4259


On 15-02-2024 21:31, Bjorn Helgaas wrote:
> External email: Use caution opening links or attachments
>
>
> On Thu, Feb 15, 2024 at 04:55:47PM +0530, Vidya Sagar wrote:
>> On 15-02-2024 00:42, Bjorn Helgaas wrote:
>>> Hi Vidya, question about ancient history:
>>>
>>> On Tue, Aug 13, 2019 at 05:06:27PM +0530, Vidya Sagar wrote:
>>>> Add support for Synopsys DesignWare core IP based PCIe host controller
>>>> present in Tegra194 SoC.
>>>> ...
>>>> +static int tegra_pcie_dw_host_init(struct pcie_port *pp)
>>>> +{
>>>> +     struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
>>>> +     struct tegra_pcie_dw *pcie = to_tegra_pcie(pci);
>>>> +     u32 val, tmp, offset, speed;
>>>> +
>>>> +     tegra_pcie_prepare_host(pp);
>>>> +
>>>> +     if (dw_pcie_wait_for_link(pci)) {
>>>> +             /*
>>>> +              * There are some endpoints which can't get the link up if
>>>> +              * root port has Data Link Feature (DLF) enabled.
>>>> +              * Refer Spec rev 4.0 ver 1.0 sec 3.4.2 & 7.7.4 for more info
>>>> +              * on Scaled Flow Control and DLF.
>>>> +              * So, need to confirm that is indeed the case here and attempt
>>>> +              * link up once again with DLF disabled.
>>> This comment suggests that there's an issue with *Endpoints*, not an
>>> issue with the Root Port.  If so, it seems like this problem could
>>> occur with all Root Ports, not just Tegra194.  Do you remember any
>>> details about this?
>>>
>>> I don't remember hearing about any similar issues, and this driver is
>>> the only place PCI_EXT_CAP_ID_DLF is referenced, so maybe it is
>>> actually something related to Tegra194?
>> We noticed PCIe link-up issues with some endpoints. link-up at the physical
>> layer level but NOT at the Data link layer level precisely. We further
>> figured out that it is the DLFE DLLPs that the root port sends during the
>> link up process which are causing the endpoints get confused and preventing
>> them from sending the InitFC DLLPs leading to the link not being up at
>> Data Link Layer level.
> Do you happen to remember any of the endpoints that have issues?
We observed this issue with an ASMedia USB controller card. Unfortunately,
I didn't keep a record of the Vendor-ID and Device-ID of that card.
>
> Could save some painful debugging if we trip over this issue on other
> systems.  We have seen a few cases where links wouldn't train at full
> speed unless they trained at a lower speed first, e.g.,
> imx6_pcie_start_link(), fu740_pcie_start_link().  I guess there are
> probably lots of edge cases that can cause link failures.
>
> Bjorn