Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <815d102d-be0c-8e5d-ac12-1500d90628da@nvidia.com>
Date:   Tue, 25 Jul 2023 13:49:35 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Subject: Re: [PATCH V1] Revert "PCI: tegra194: Enable support for 256 Byte
 payload"
To:     Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Cc:     lpieralisi@kernel.org, kw@linux.com, robh@kernel.org,
        bhelgaas@google.com, thierry.reding@gmail.com,
        jonathanh@nvidia.com, Sergey.Semin@baikalelectronics.ru,
        linux-pci@vger.kernel.org, linux-tegra@vger.kernel.org,
        linux-kernel@vger.kernel.org, kthota@nvidia.com,
        mmaddireddy@nvidia.com, sagar.tv@gmail.com
References: <20230608093652.1409485-1-vidyas@nvidia.com>
 <20230725075159.GB22139@thinkpad>
Content-Language: en-US
From:   Vidya Sagar <vidyas@nvidia.com>
In-Reply-To: <20230725075159.GB22139@thinkpad>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?anFOeWZOSCsrM2N3NlpBeW9kSXVXVnZOQVNUSGRLTFN2Vzlvd0cwRWJkWTdY?=
 =?utf-8?B?UjdNMXgwRXhrWElQU2NTUXNmTEhtdzRkQ1lNSytFVnZOWjJOY3A5d3RTZGs5?=
 =?utf-8?B?bTVCOUFmM2d0MnZlS1dydStwaEJCT0RqUTBWeGI4dGwyRnUrUlZxZ2hrdGhQ?=
 =?utf-8?B?WTM3QlVOMGJpb0lOTXlnaFBhVDZFUHhhQjFBa1BPbnlZeW5NdjlGK0h3VVh5?=
 =?utf-8?B?ZHVsUWRGM3RiUzZJZ1BhUk9POHJBUkZBV2Z2ZWlqRHZGcDFVRXFkOFU2NVpN?=
 =?utf-8?B?eUNHS2h1TzdQVkpZVVlCK0cybDJaYk9HZVNoeUt3VktsSVUyMWkrdUJlZ0Rm?=
 =?utf-8?B?b1JFQ0ZrRTF2YnZ4ekM0RXl0bUNnOWxvWm94bUZvVGhORTgxVU94dXlLVmNI?=
 =?utf-8?B?SEhKTTEzMlg5aEVzKzcyOUNDWG03VXo4Vjk5RGI4OEJtMHRJU0F6eWxyQngv?=
 =?utf-8?B?c1FJUkVSVUlIaGpzQUpnWkp2WmxWelhrQXY1b1MvbmxaYmNOQmhYQjFtQkwy?=
 =?utf-8?B?bEZpNzl4L0VMWWRoR1E0emVqYjVvMlYrRDJlY1JWSFBGQWR0Mnh1c2crdHFM?=
 =?utf-8?B?NC9VTkpURE94VEZWZm9jMEc2Um41RWhSRlN6ZkZvQXRIaFRablVrMW1KMFNt?=
 =?utf-8?B?cEJDMFNoelZkcktKRFBCbFE2d3N5ZURWYUt0VUQ5R0dKMmVJOTVidlRLUDU2?=
 =?utf-8?B?Q0VNd1dMc3Q2QXo0TTU0cHcyZlVhS1FkU0xtaDRhR0orRUNJWDNwMTh5aUhu?=
 =?utf-8?B?MFdsM2pNTitjM3prYWRma0szTVN3L2dLMERQcHJHM054VjRRb1VlckIvcTVR?=
 =?utf-8?B?aDJpbWh3eXMxVEJac0haaDdTdFhaVjBnV1ErU0pwaVBlV0lJS0ZIcXBwVmg4?=
 =?utf-8?B?Rmd6UStFME9MOGFmcytRKzQwQ2g4c1Z5Wm1WSnZReWQ4L3NpanNUamd2d09z?=
 =?utf-8?B?MHA3MFdDY3ZJZjhtSmJ5ZGtFbUxQcFZuZk9JajRPYTRBYmFHdlJIRy9mOTY4?=
 =?utf-8?B?dFcrazNja25uelNBNWZUUXljVmViQ2NmYXlDUnhWeFZSclJDeVpYWnJIaXNF?=
 =?utf-8?B?L3hzL3ZDL1RMeXhaV0ZZOTA2bzJpcmgzeUwrdGc0SUJCRmRUYzRPdEY1ZWZT?=
 =?utf-8?B?WGV4MnRQNHVDZTlrTG9aNUdFZTFPbG13UkJyQnRRa2NTVUNzdEhNekxYTW50?=
 =?utf-8?B?NThpdEhaK3dGSktZNjhDNG0zM0RHWUtycXJhdGlkcnAzek4rOVYxb0NxUTdJ?=
 =?utf-8?B?ZnhvWmE3dG5pMHN4MFdzODVLdmp5ak9ST3ZoRXlsbEJrTHU4WTN4OVhLZ0dG?=
 =?utf-8?B?K2haVFBJSGx1eXJwNHNEVWluVmJIbnA1eWVqOEF3bzNPS0VrNHRheG84Rjlk?=
 =?utf-8?B?ZFBTdXNSbkU4eS96Um9TRkFvTjFaWTdIRDhjQ1dxT1RwY1Irc1pseXdDTHJr?=
 =?utf-8?B?dXB2Lys4UjY3cVJkR3dud2lRd3RBemM3U1g4N3JxcEhwWlZhTmhDWXFJSGRD?=
 =?utf-8?B?WXZLaWFvSkliL0RaK3pEK21ZUEkzOXpnd1BiU21LdHQrcks3ZDQ2YzNmY1NO?=
 =?utf-8?B?OEhzTE9wdldZcTcvTzBPUWt2a2tFdWVMY1d3OW9NRUJJWVFoa3EydDJ5VGJo?=
 =?utf-8?B?ODRVWWw0SjlibXhqZjR3KzBiL29pMjA2ZHVKMm1SQlFlK0NndTI5Y3VPZnBt?=
 =?utf-8?B?VkMrTGxvbGJmenJMZWlRUytHTVRQMnZWZVIvNklJb01UWkdzQnVqS1RqL1lS?=
 =?utf-8?B?T3NVMGpKbEZNVk1jc3BoZGlXWlhhTUlBUUdqTlNZc3JzYkVCeXM5Sit5OTZ0?=
 =?utf-8?B?eXA0MWVLTGxuRVpVM1JobDBRQ0JnYXFha20ybzl5SXQ1R2oybEgvcTNvcExH?=
 =?utf-8?B?WHFQVGpaQ1NDQjV1SzZpd2dMeXRyc3BmZWJranNMdWdvUk5LeGJjUHFvTHd6?=
 =?utf-8?B?RHJML3I0NmFaVjQ1aWlrcDc0Z3l2RVdqWUI0UWFUcks2aW5vRXpmZG52YTly?=
 =?utf-8?B?S2hpdkl2TFJPQVJVY0g3K3JlSDZ4UWp1QVlsUmljOWIvaThiZUVBZjJBSGlG?=
 =?utf-8?B?VFBtMU1uc09xWXBvb1NqUk9KV1g3RldEV0ZSbURFQWN3ZUFDR1BrTXFpQlRP?=
 =?utf-8?Q?YZeX3H7uSheS8s0IIRlZfYfM2?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 88ec7da0-599e-43bf-8c88-08db8ce7ea0f
X-MS-Exchange-CrossTenant-AuthSource: BN8PR12MB2900.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jul 2023 08:19:50.0327
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KeioAlDxrCBSHTG6n0IbAc+v1eByCx97RWJR7/iDubi96rEMxDPbVKpaL4RI7QLpDwalFjxv7um6mEoxWmAq8A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB8793
X-Spam-Status: No, score=-1.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        NICE_REPLY_A,SPF_HELO_PASS,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 7/25/2023 1:21 PM, Manivannan Sadhasivam wrote:
> External email: Use caution opening links or attachments
> 
> 
> On Thu, Jun 08, 2023 at 03:06:52PM +0530, Vidya Sagar wrote:
>> This reverts commit 4fb8e46c1bc4 ("PCI: tegra194: Enable
>> support for 256 Byte payload")
>>
>> Consider a PCIe hierarchy with a PCIe switch and a device connected
>> downstream of the switch that has support for MPS which is the minimum
>> in the hierarchy, and root port programmed with an MPS in its DevCtl
>> register that is greater than the minimum. In this scenario, the default
>> bus configuration of the kernel i.e. "PCIE_BUS_DEFAULT" doesn't
>> configure the MPS settings in the hierarchy correctly resulting in the
>> device with support for minimum MPS in the hierarchy receiving the TLPs
>> of size more than that. Although this can be addresed by appending
>> "pci=pcie_bus_safe" to the kernel command line, it doesn't seem to be a
>> good idea to always have this commandline argument even for the basic
>> functionality to work.
>> Reverting commit 4fb8e46c1bc4 ("PCI: tegra194: Enable support for 256
>> Byte payload") avoids this requirement and ensures that the basic
>> functionality of the devices irrespective of the hierarchy and the MPS of
>> the devices in the hierarchy.
>> To reap the benefits of having support for higher MPS, optionally, one can
>> always append the kernel command line with "pci=pcie_bus_perf".
>>
>> Fixes: 4fb8e46c1bc4 ("PCI: tegra194: Enable support for 256 Byte payload")
>> Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
> 
> I know that this patch is merged. But I happen to test a similar change on Qcom
> platform during a patch review and found that the PCI core changes MPS to 128
> when a 128byte supported device is found:
> 
> [    3.174290] pci 0000:01:00.0: Upstream bridge's Max Payload Size set to 128 (was 256, max 128)
> [    3.186538] pci 0000:01:00.0: Max Payload Size set to 128 (was 128, max 128)
> 
> This was just randomly tested on a platform whose Root port DEVCAP was 128, but
> it shouldn't matter. And I didn't change the default bus configuration.
> 
> Wondering how you ended up facing issues with it.

If the endpiont device that has support only for 128byte MPS is 
connected directly to the root port, then, I agree that the PCIe 
sub-system takes care of changing the MPS to the common lowest MPS, but 
if the endpoint device is connected behind a PCIe switch, then the PCIe 
subsystem doesn't configure the MPS properly.

-Vidya Sagar

> 
> - Mani
> 
>> ---
>>   drivers/pci/controller/dwc/pcie-tegra194.c | 13 -------------
>>   1 file changed, 13 deletions(-)
>>
>> diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
>> index 4fdadc7b045f..877d81b13334 100644
>> --- a/drivers/pci/controller/dwc/pcie-tegra194.c
>> +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
>> @@ -892,7 +892,6 @@ static int tegra_pcie_dw_host_init(struct dw_pcie_rp *pp)
>>        struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
>>        struct tegra_pcie_dw *pcie = to_tegra_pcie(pci);
>>        u32 val;
>> -     u16 val_16;
>>
>>        pp->bridge->ops = &tegra_pci_ops;
>>
>> @@ -900,11 +899,6 @@ static int tegra_pcie_dw_host_init(struct dw_pcie_rp *pp)
>>                pcie->pcie_cap_base = dw_pcie_find_capability(&pcie->pci,
>>                                                              PCI_CAP_ID_EXP);
>>
>> -     val_16 = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_DEVCTL);
>> -     val_16 &= ~PCI_EXP_DEVCTL_PAYLOAD;
>> -     val_16 |= PCI_EXP_DEVCTL_PAYLOAD_256B;
>> -     dw_pcie_writew_dbi(pci, pcie->pcie_cap_base + PCI_EXP_DEVCTL, val_16);
>> -
>>        val = dw_pcie_readl_dbi(pci, PCI_IO_BASE);
>>        val &= ~(IO_BASE_IO_DECODE | IO_BASE_IO_DECODE_BIT8);
>>        dw_pcie_writel_dbi(pci, PCI_IO_BASE, val);
>> @@ -1756,7 +1750,6 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
>>        struct device *dev = pcie->dev;
>>        u32 val;
>>        int ret;
>> -     u16 val_16;
>>
>>        if (pcie->ep_state == EP_STATE_ENABLED)
>>                return;
>> @@ -1887,11 +1880,6 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
>>        pcie->pcie_cap_base = dw_pcie_find_capability(&pcie->pci,
>>                                                      PCI_CAP_ID_EXP);
>>
>> -     val_16 = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_DEVCTL);
>> -     val_16 &= ~PCI_EXP_DEVCTL_PAYLOAD;
>> -     val_16 |= PCI_EXP_DEVCTL_PAYLOAD_256B;
>> -     dw_pcie_writew_dbi(pci, pcie->pcie_cap_base + PCI_EXP_DEVCTL, val_16);
>> -
>>        /* Clear Slot Clock Configuration bit if SRNS configuration */
>>        if (pcie->enable_srns) {
>>                val_16 = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base +
>> @@ -1900,7 +1888,6 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
>>                dw_pcie_writew_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA,
>>                                   val_16);
>>        }
>> -
>>        clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ);
>>
>>        val = (ep->msi_mem_phys & MSIX_ADDR_MATCH_LOW_OFF_MASK);
>> --
>> 2.25.1
>>
> 
> --
> மணிவண்ணன் சதாசிவம்