Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32;
Message-ID: <70b35a0e-5ccd-4e19-a8ac-4cf095007a69@amd.com>
Date:   Wed, 15 Nov 2023 11:08:43 -0600
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 5/7] PCI: ACPI: Detect PCIe root ports that are used
 for tunneling
Content-Language: en-US
To:     Mika Westerberg <mika.westerberg@linux.intel.com>
Cc:     Karol Herbst <kherbst@redhat.com>, Lyude Paul <lyude@redhat.com>,
        Alex Deucher <alexander.deucher@amd.com>,
        =?UTF-8?Q?Christian_K=C3=B6nig?= <christian.koenig@amd.com>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Lukas Wunner <lukas@wunner.de>,
        Danilo Krummrich <dakr@redhat.com>,
        David Airlie <airlied@gmail.com>,
        Daniel Vetter <daniel@ffwll.ch>,
        Xinhui Pan <Xinhui.Pan@amd.com>,
        "Rafael J . Wysocki" <rafael@kernel.org>,
        =?UTF-8?Q?Ilpo_J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>,
        =?UTF-8?Q?Pali_Roh=C3=A1r?= <pali@kernel.org>,
        =?UTF-8?Q?Marek_Beh=C3=BAn?= <kabel@kernel.org>,
        "Maciej W . Rozycki" <macro@orcam.me.uk>,
        Manivannan Sadhasivam <mani@kernel.org>,
        "open list:DRM DRIVER FOR NVIDIA GEFORCE/QUADRO GPUS" 
        <dri-devel@lists.freedesktop.org>,
        "open list:DRM DRIVER FOR NVIDIA GEFORCE/QUADRO GPUS" 
        <nouveau@lists.freedesktop.org>,
        open list <linux-kernel@vger.kernel.org>,
        "open list:RADEON and AMDGPU DRM DRIVERS" 
        <amd-gfx@lists.freedesktop.org>,
        "open list:PCI SUBSYSTEM" <linux-pci@vger.kernel.org>,
        "open list:ACPI" <linux-acpi@vger.kernel.org>
References: <20231114200755.14911-1-mario.limonciello@amd.com>
 <20231114200755.14911-6-mario.limonciello@amd.com>
 <20231115104019.GY17433@black.fi.intel.com>
From:   Mario Limonciello <mario.limonciello@amd.com>
In-Reply-To: <20231115104019.GY17433@black.fi.intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eVJlNVJJTmd4SlIxWmNQM0xFRVFWYTdZd25SRHF0RzFGbEhYTk1Zb0ZrUHlG?=
 =?utf-8?B?U2VhMlVtZWVZT1pCekNHbWNiZ2NmcnIxNVZzOWRRK1ozYnJsOVl3WFFjcHlQ?=
 =?utf-8?B?TDJ4VnByQk9wNStybHBkRUUybkMrNzdwUG8xRVkxQ2tqaXk3eEoyMnF3eVFU?=
 =?utf-8?B?SjZqVS80NFVjT1EyS3JQU2h2eVUzblNPVlFJK3RHRG1xZGYyZmh1UEFmTkhk?=
 =?utf-8?B?UGJxank5clZ4REJsRVpQbGdFWmhxNTR4R1V0bDV6cngvSnN5VkF5WC80akIr?=
 =?utf-8?B?bURBc2IyR01zMW13ZTlGSSsvZTNkYStzSkgxbWh3SjFESzdtem1DQjdKdmRV?=
 =?utf-8?B?VHYya2ZXREhDUGV5NURKZUl6UG0zUm8xNHhkOHFBNHc4QXdkZ0g3ZXFkYk9o?=
 =?utf-8?B?T05FRi9GY1I5UExaL1dzcDJsK2NVRjFaTzRyZ0wxeFVnN2xkVThxL2V6dGhB?=
 =?utf-8?B?My8rdkIrRzBLMWcxNXJ5WWQ1eHZJWmFLM2RGSDl6YmI1TWpwUlVKWkU1MStS?=
 =?utf-8?B?YVpNYXdOc0laeTVjR1BDZXZ6QWcxeGxjZVJVMGFxZnR1YVhyNEo5emZCQzBi?=
 =?utf-8?B?a3VEdUxIL1M1OHV1ZGNNRmNKZXZHRVNiTDhuNSs2ZVNCY01kcjBZN0FVRmgw?=
 =?utf-8?B?aXdZK2k1Z1RlS0E1amw3MmE3eGwrWjVBamIrMzlFU3hTcGo3R3Z1cUZtOThM?=
 =?utf-8?B?aGNHakZxeWFHSVF4elZDL0p4U01aZTNuTUxGU3RwVlk5WW5nbG05eStwbUtT?=
 =?utf-8?B?cGgwN1kzb0hnc0hsWUNtNy9TdFg1ODlUSGN2SEhyLzh6VkMzUk9GVzJnVlNt?=
 =?utf-8?B?SzFSYlNzS2IxMENpckNWUVM0amZpUExHdW5tbWoyUFR1ZERVM1h5SWVTWUty?=
 =?utf-8?B?Y3dTeVNrTmdtYm96T3lqNFFjM2dwN2NONWhqUWZjMjJZN3hWSnJ4RXROVlh5?=
 =?utf-8?B?TlBYNElRRFdQTE84ZEpqdGpxRy9vNlRHMWpQNm5vVENsVHlzRXYyd1lvOEFu?=
 =?utf-8?B?OWtiZk1LT2NEcGEzUFdFYkJvMmJZOVZvZjlJeDBPMGM0Unl2ZldPYkcxTDBU?=
 =?utf-8?B?aG9Ra0F4eXU0TTJHR1FFYVVEV21SQjBuU28vRmJPeUVCajlLajJMUjhGZ0Iy?=
 =?utf-8?B?cU9waUZzM2lxbkRCa1RyV1djL3VXM2ZOY1Q5ckswOHp4bmNHQWRzU3lBT2Mz?=
 =?utf-8?B?cnpoVFhIV01wQS9iUUwzZ1dYa0Z3ZlQzUmFlYXU4TVR5eUZWN3prSWtkRlNJ?=
 =?utf-8?B?ODFtbCtIZXJ1VTFSZVE1ZndXcVVPcXZ6Uzk0T29aV2Z1OFI5aEVKd0RUaW1s?=
 =?utf-8?B?aHVpV0hFdllrUllvRlkra0NXOHd4SXJwcXMwSk1pWFRhTkdtQmRTNXY5TGNI?=
 =?utf-8?B?a01ERHhtT0cycm12YXVjMXBNS0tNc2lYQ1pWYm5WZDBtbzFPNkMwaTZEVzZH?=
 =?utf-8?B?VHltOXQzd0ZMemtQNk1MRkpheXdXZEY5WU9tTGVHMnRrT0R4Tmd2NE1DMmxo?=
 =?utf-8?B?dnM2aEhUOVU3dUcvdWpZeTd4TXNwWGV2ZDRJb2pROFJZSTh3dkNlOWpYbmpG?=
 =?utf-8?B?MWE3NW9jejVvME5sUTMvNTFmVzl6SGRUOE12ZzZWN0RsTnh5U2ZoNWEreEJP?=
 =?utf-8?B?bEY4eUFObm9IZzRPQjJYU0pUMG8yZjJpV1FIZlAxVE1KQlJialIvK1k5MTAw?=
 =?utf-8?B?S1hWdWpYQi9nRmNsTS9HcElTRmpXWEo1YVRIbERmdDBheXBXZ2NNdTUrSEwx?=
 =?utf-8?B?NWMvNnlKeFZIZjhrWGJWcU85Rml5a3JnQjlKNzdRdXA1NGQ3eGF3Q2l1aGc5?=
 =?utf-8?B?VmZQMFVDL1YzVHFYc08wdnFEM2lreTVEN3N5TFFWa1lNV3lKYnFSd290dGYz?=
 =?utf-8?B?Y2N6SmlBQjMxbE9LbGRuWUpOdlFHSVExR0JaeHJJMHFoYTBuYUg1bEszRGdT?=
 =?utf-8?B?Z1U1OHFqZmdmNGU4S2VQR2RBSlcrTzBpOFMwSjBDeCtzTm9SOVVXMmtsSFU3?=
 =?utf-8?B?RGRTWkI2NXNDNW1KOEpqT3MwQ0xlZmFrOFArWi9aZDFvQ1pyZk9QUFVHNDFy?=
 =?utf-8?B?akU3NVBZV0k5N25XNmtLd29KRjg2MmI3MG84RlhMckJVaGU2cU5zSi9sTlNJ?=
 =?utf-8?Q?vZJ6uymqoNBms8G/FSMZL2PxI?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bc14c823-7c88-4fed-0819-08dbe5fd8742
X-MS-Exchange-CrossTenant-AuthSource: MN0PR12MB6101.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Nov 2023 17:08:46.3441
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: IEiiFuwE+NvGO/VwH7j9JFZSw4N8Y/OeFrGEhY0DUXJIglWvnMb4C2rOGF9oDvsZdePyh7qMbMEVBSgMN6dI5g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7948
X-Spam-Status: No, score=-1.0 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 15 Nov 2023 09:09:15 -0800 (PST)

On 11/15/2023 04:40, Mika Westerberg wrote:
> Hi Mario,
> 
> On Tue, Nov 14, 2023 at 02:07:53PM -0600, Mario Limonciello wrote:
>> USB4 routers support a feature called "PCIe tunneling". This
>> allows PCIe traffic to be transmitted over USB4 fabric.
>>
>> PCIe root ports that are used in this fashion can be discovered
>> by device specific data that specifies the USB4 router they are
>> connected to. For the PCI core, the specific connection information
>> doesn't matter, but it's interesting to know that this root port is
>> used for tunneling traffic. This will allow other decisions to be
>> made based upon it.
>>
>> Detect the `usb4-host-interface` _DSD and if it's found save it
>> into a new `is_virtual_link` bit in `struct pci_device`.
> 
> While this is fine for the "first" tunneled link, this does not take
> into account possible other "virtual" links that lead to the endpoint in
> question. Typically for eGPU it only makes sense to plug it directly to
> the host but say there is a USB4 hub (with PCIe tunneling capabilities)
> in the middle. Now the link from the hub to the eGPU that is also
> "virtual" is not marked as such and the bandwidth calculations may not
> get what is expected.

Right; you mentioned the DVSEC available for hubs in this case.  As I 
don't have one of these to validate it works properly I was thinking 
that should be a follow up.

If you think it should be part of the same series I'll add it, but I'd 
ask if you can please check I did it right on one that reports the DVSEC?

> 
> It should be possible to map the PCIe ports that go over USB4 links
> through router port operation "Get PCIe Downstream Entry Mapping" and
> for the Thunderbolt 3 there is the DROM entries (I believe Lukas has
> patches for this part already) but I guess it is outside of the scope of
> this series. 

Yeah I'd prefer to avoid the kitchen sink for the first pass and then we 
an add more cases to is_virtual_link later.