Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 29 Mar 2022 20:42:19 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Shlomo Pongratz <shlomopongratz@gmail.com>
Cc:     linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
        andrew.maier@eideticom.com, logang@deltatee.com,
        bhelgaas@google.com, Shlomo Pongratz <shlomop@pliops.com>
Subject: Re: [PATCH V3 1/1] Intel Sky Lake-E host root ports check.
Message-ID: <20220329234219.GD1716663@nvidia.com>
References: <20220329104321.4712-1-shlomop@pliops.com>
 <20220329104321.4712-2-shlomop@pliops.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220329104321.4712-2-shlomop@pliops.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?SVHGDBZdTELXf9ujfHkbnFRUiu6nNsJOWJjS4g2+/aEpD9Z/lCnUx3oB2zmS?=
 =?us-ascii?Q?TVJiWTtMnAVHGe8UHXPyuKqE/A2nhKop5hiV3STDVLwQkkPbQfGxH42/L3v5?=
 =?us-ascii?Q?Bb/kLKL3vpkNVqsy4iFOezt378jbtLJCECE9u4h0ogTHloPLjZB1zRNqgUZK?=
 =?us-ascii?Q?PaF2r8iv82CRDllCLEc19Wv3MqCc+VIO7bximqFK2EnNSBxqufZX6u7qm6+x?=
 =?us-ascii?Q?DK1v8G4qAa68cTXWGsrnm9ycPHGcZWL6A2RyTCZ1nQnqAmOOwzZu7uRk+RkC?=
 =?us-ascii?Q?x20SSfXTU0+0W9TUxIlHMDvTZgCfIx+nszn8Iv9L0ldTDiD6EW7J1ChfQHhZ?=
 =?us-ascii?Q?HUe5oG6+IV4L9QG/5i6Ai7qmzBWhPmTunmRlHFmRrMel+fqkGMdEuypM0fb/?=
 =?us-ascii?Q?A7djHluvvTyYLlkSd8JzhwcaqtEos3QMkV4fy9WX1cpZ/eW3YMLAxcyRJpfN?=
 =?us-ascii?Q?8Rk6fASu18Je8ZW/YLjSlRcVkhH3t/A5pJWnEzCwEh5YTYFNAqtLTyr9DopU?=
 =?us-ascii?Q?lkpw2q2PIMmn7l2TLkPHQnMjDw0GBEbUQesGp8hDiPjk+NSj535iPVZcqdmD?=
 =?us-ascii?Q?HE4uWYV1h4ddPcYUcF9ipRChWvasDZ5OpVAgxOD1y7VvW0mGWvHNNZMFAlhW?=
 =?us-ascii?Q?iju75sgyRrnVlsUSH2y0waO8HdO1hbSL2q8IcX5B+e4lMhsSyDgt8/YbBhjn?=
 =?us-ascii?Q?0vRQQe8bypVVhLzYd5/H7tUYKAsTNSBQwBHpBA1BRfbjPBYpSLiFvG2gihNQ?=
 =?us-ascii?Q?S8q0EwH18PF7IjZTuQnu1HS34zxrWSBpA5AqZvI46DrHraxy8++xPhWSqUQF?=
 =?us-ascii?Q?tPGwCaQ7P1Vcwui2Jhw35zu29ZWIPbfSHXSZ/LZSOOq0JhlkRjShA6FAyy4E?=
 =?us-ascii?Q?OKoHMjRYlMql42aiP6UrWGjBPsZXmE46BR65JrOblv+jNZmYgjadMlsbRtdm?=
 =?us-ascii?Q?lKlzk2yAecFFUMOSngjpa/74v/JjWzaZ4ccFaQvc3hw/DGq9U+sVLARTQeIN?=
 =?us-ascii?Q?ca/GsYOdVckKe/ugrnd6JRkbhjSnTCzjytbFa3KaVmJgrKxTgAZOOsiL6w7B?=
 =?us-ascii?Q?WUTgGvRr3e7Wz361+khNS700mUR5FDnqvPNtxVMpbacotAdyWenbk6P0eU9H?=
 =?us-ascii?Q?RfMxv2vCE+TgH1M44eO+ujjJ/NIbyvR0k6aoR8rv24GWItJOISDuTtjQWkCR?=
 =?us-ascii?Q?wzTFDQeEd2eqvgUpz/1T+1QkrnN+QyG77TokQLt1jAXE3hbBFl7HvYuQCfyh?=
 =?us-ascii?Q?bWQa1Q+8vndl1Cv3tSJDzn8EoB8V22X6HSWDjQCHnWdXrPmrbrBSPZ1nzngi?=
 =?us-ascii?Q?RF1h+GHe+ZaxFkmqN6ma4Hs2XX60GsaIR3XKycnVUiteTV3vHrDq1n568EWa?=
 =?us-ascii?Q?XNBjnEoXk7u9ZOhK9Tt/cP5MKt0I4MT46m1lJeKk9UP0StzwRreIfdte8nCK?=
 =?us-ascii?Q?QQNfWTM2HH/dNCjeTnRL7i9Ehc4C3EzNY0i27jqxCEDHMfBe9MvONVBKp+5L?=
 =?us-ascii?Q?aYhONtxx61jlf193d4bi2FCBZMkBDDv+zQycSKSw7xZEj0XonIQrsj328hfU?=
 =?us-ascii?Q?IO/+k+cnRugcewJKcUom0pULLIEhFSsIt9H7r0fQ3BJCLJFsrCt/LHlSJxJo?=
 =?us-ascii?Q?u6zNDhJ/hGnbB1wu4sgRoMU6vf5P7aLF9Y5e83TZkO/IO490ShNy40k2SeUA?=
 =?us-ascii?Q?DwzD4f8672lEqJZ8zS7ttFE8Wg109zksFaqxfqNYly6O8M5NTlXlVSOObq2E?=
 =?us-ascii?Q?BczTfwZsmw=3D=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 20bca188-e4c5-41ee-6eb5-08da11ddc48b
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Mar 2022 23:42:21.1450
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ldIeTXrIJ5sOGFP9sYh1mEaOpt5YXTCrqHFRp3BpG8tGxdy5FVWvmYt0YgSonJHt
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4996
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Mar 29, 2022 at 01:43:21PM +0300, Shlomo Pongratz wrote:
> On commit 7b94b53db34f ("PCI/P2PDMA: Add Intel Sky Lake-E Root Ports B, C, D to the whitelist")

'in commit' and it is expected to word wrap the description

> Andrew Maier added the Sky Lake-E additional devices
> 2031, 2032 and 2033 root ports to the already existing 2030 device.
> 
> The Intel devices 2030, 2031, 2032 and 2033 which are ports A, B, C and D,
> and if all exist they will occupy slots 0 till 3 in that order.
> 
> Now if for example device 2030 is missing then there will no device on slot 0, but
> other devices can reside on other slots according to there port.
> For this reason the test that insisted that the bridge should be on slot 0 was modified
> to support bridges that are not on slot 0.

This needs updating to the new test

> 
> Signed-off-by: Shlomo Pongratz <shlomop@pliops.com>
>  drivers/pci/p2pdma.c | 42 +++++++++++++++++++++++++++++++++++++-----
>  1 file changed, 37 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
> index 30b1df3c9d2f..c088d4ab64f4 100644
> +++ b/drivers/pci/p2pdma.c
> @@ -307,6 +307,7 @@ static const struct pci_p2pdma_whitelist_entry {
>  	unsigned short device;
>  	enum {
>  		REQ_SAME_HOST_BRIDGE	= 1 << 0,
> +		IS_ROOT_PORT		= 1 << 1,
>  	} flags;
>  } pci_p2pdma_whitelist[] = {
>  	/* Intel Xeon E5/Core i7 */
> @@ -316,15 +317,38 @@ static const struct pci_p2pdma_whitelist_entry {
>  	{PCI_VENDOR_ID_INTEL,	0x2f00, REQ_SAME_HOST_BRIDGE},
>  	{PCI_VENDOR_ID_INTEL,	0x2f01, REQ_SAME_HOST_BRIDGE},
>  	/* Intel SkyLake-E */
> -	{PCI_VENDOR_ID_INTEL,	0x2030, 0},
> -	{PCI_VENDOR_ID_INTEL,	0x2031, 0},
> -	{PCI_VENDOR_ID_INTEL,	0x2032, 0},
> -	{PCI_VENDOR_ID_INTEL,	0x2033, 0},
> +	{PCI_VENDOR_ID_INTEL,	0x2030, IS_ROOT_PORT},
> +	{PCI_VENDOR_ID_INTEL,	0x2031, IS_ROOT_PORT},
> +	{PCI_VENDOR_ID_INTEL,	0x2032, IS_ROOT_PORT},
> +	{PCI_VENDOR_ID_INTEL,	0x2033, IS_ROOT_PORT},
>  	{PCI_VENDOR_ID_INTEL,	0x2020, 0},
>  	{PCI_VENDOR_ID_INTEL,	0x09a2, 0},
>  	{}
>  };
>  
> +/*
> + * The functionality of thisunction can be integrated into
> + * __host_bridge_whitelist function but this will make the code
> + * less readable

This comment is not useful, explain what the function does or delete
it

> + */
> +static bool pci_is_root_port(struct pci_dev *root)
> +{
> +	const struct pci_p2pdma_whitelist_entry *entry;
> +	unsigned short vendor, device;
> +
> +	vendor = root->vendor;
> +	device = root->device;
> +
> +	for (entry = pci_p2pdma_whitelist; entry->vendor; entry++) {
> +		if (vendor != entry->vendor || device != entry->device)
> +			continue;
> +
> +		if (entry->flags & IS_ROOT_PORT)
> +			return true;
> +	}
> +	return false;
> +}
> +
>  /*
>   * This lookup function tries to find the PCI device corresponding to a given
>   * host bridge.
> @@ -333,6 +357,11 @@ static const struct pci_p2pdma_whitelist_entry {
>   * bus->devices list and that the devfn is 00.0. These assumptions should hold
>   * for all the devices in the whitelist above.
>   *
> + * The method above will work in most cases but not for all.
> + * Note that the Intel devices 2030, 2031, 2032 and 2033 are ports A, B, C and D.
> + * Consider on a bus X only port C has devices connected to it so in the PCI scan only
> + * device 8086:2032 on 0000:X:02.0 will be found as bridges with no children are ignored
> + *
>   * This function is equivalent to pci_get_slot(host->bus, 0), however it does
>   * not take the pci_bus_sem lock seeing __host_bridge_whitelist() must not
>   * sleep.
> @@ -350,7 +379,9 @@ static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host)
>  
>  	if (!root)
>  		return NULL;
> -	if (root->devfn != PCI_DEVFN(0, 0))
> +
> +	/* Intel Sky Lake-E host root ports can be on no zero slot */
> +	if (root->devfn != PCI_DEVFN(0, 0) && !pci_is_root_port(root))
>  		return NULL;

The name pci_host_bridge_dev needs some attention if it is now
returning a root port.

> @@ -372,6 +403,7 @@ static bool __host_bridge_whitelist(struct pci_host_bridge *host,
>  	for (entry = pci_p2pdma_whitelist; entry->vendor; entry++) {
>  		if (vendor != entry->vendor || device != entry->device)
>  			continue;
> +
>  		if (entry->flags & REQ_SAME_HOST_BRIDGE && !same_host_bridge)
>  			return false;

Unnecessary hunk

Jason