Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 14 Nov 2022 16:07:58 -0800
From:   Dan Williams <dan.j.williams@intel.com>
To:     Robert Richter <rrichter@amd.com>,
        Alison Schofield <alison.schofield@intel.com>,
        Vishal Verma <vishal.l.verma@intel.com>,
        "Ira Weiny" <ira.weiny@intel.com>,
        Ben Widawsky <bwidawsk@kernel.org>,
        Dan Williams <dan.j.williams@intel.com>
CC:     <linux-cxl@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        "Rafael J. Wysocki" <rafael@kernel.org>,
        Len Brown <lenb@kernel.org>,
        Jonathan Cameron <Jonathan.Cameron@huawei.com>,
        "Davidlohr Bueso" <dave@stgolabs.net>,
        Dave Jiang <dave.jiang@intel.com>,
        Robert Richter <rrichter@amd.com>
Subject: RE: [PATCH v3 4/9] cxl/mem: Skip intermediate port enumeration of
 restricted endpoints (RCDs)
Message-ID: <6372d85ed793f_12cdff294f@dwillia2-xfh.jf.intel.com.notmuch>
References: <20221109104059.766720-1-rrichter@amd.com>
 <20221109104059.766720-5-rrichter@amd.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20221109104059.766720-5-rrichter@amd.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?MRd5Cv0SBhxg1IeHx2NIo7fKfLfQXv5ZjRzxRZvL9ayr7zxY3lMWXR+vLlBI?=
 =?us-ascii?Q?9bu41VcRuP9TFHi12ZXrFFnYvBaJ0HRZMACXOs7B4WYJrxFuJMVnSRRvXhXj?=
 =?us-ascii?Q?BDeEWrOleqRx6mXwqNXL+jPRhFO9y6b5RdIp70hOwf7wje3V2PpYQ7pWkBnQ?=
 =?us-ascii?Q?NAofe9MPoOPxYAPjG3MurHO38ek1Tw/v4fwuNdu6a4PaBiDlBo2nYywd+n9B?=
 =?us-ascii?Q?WMmYWercD25udpeAQrC58UcnWUIOAAUAbfW40o0GwKq42KqcFxn7kkN3DBWY?=
 =?us-ascii?Q?0lbVTotPllF4cS6XGzqmMncJN0qN3hZvYW9gRra/k5EhoZ4caAABN+JQ9tFE?=
 =?us-ascii?Q?wue1JBivoCM3Z/4E6KW4eBKGWJBYruQmOXIEIQrADfWcobmNPOye6mBoO3vw?=
 =?us-ascii?Q?CstE0z+sNWuodYJaX0a6ILdEqwB5OaNi34RoJO6CiGIAwZ/lkHQJ/O8r6s37?=
 =?us-ascii?Q?UNH9+t3tg4tqz2eSN+dd8z/3+ZG7U0Nhjq+pave+fP9anWzYOLrpqOsYzWYA?=
 =?us-ascii?Q?CRleRUpIDd83C24cpmHJNweqny8TZCsdX2/0diwe3drSjdNp7oYA5Hhsmw1q?=
 =?us-ascii?Q?kyTbVQ+T5UEmM5CkL9dP6qkR7zFWJDOorNMi3Je0a4YhDs6zzFMd3i0w7Y6p?=
 =?us-ascii?Q?aX8wbJW6+6/6LZ9ss6vi2Bcj6OVyfV+5HUZfje53yAMfd8zvdsytmCBEHncq?=
 =?us-ascii?Q?yu/wIpSHJkGkgb58QM4al1JdtPeBOJBGkGolIeVingrjNnvUNSA705zGOwM+?=
 =?us-ascii?Q?XxuDpuh40eUt0+EcKt8f86hAArhwcIoS37tsjKHrlCs2HR9hhbiQbWRHU7OF?=
 =?us-ascii?Q?DscLxmO8TCZHw/bRjC8PAJKqU0kqFGodaliismQfZdju/8PN0Kd7XVWbXagg?=
 =?us-ascii?Q?QzWSfo1rz2zBHexCsN5+Z9uTxpL7fMFR3XXfeQqdM7dCecJJVWrbHhB95n2U?=
 =?us-ascii?Q?E0AqBUpUEfdPjQJA2N1y7EmFv2I/+QXU//Vzxun0AryERVMufoGXIEmefHwJ?=
 =?us-ascii?Q?15qS2sX/JH3KkvTVzOL7nEnKZVUFLrG7h001wj94bpGJd4FW6wx9sUFy+F42?=
 =?us-ascii?Q?6Qa+IZXr9wCS6QRY64VcMJt//F0zojdpFwiotx6sUCFb5hP1PamuoXOb74Iz?=
 =?us-ascii?Q?M8ek7yutlDPJXyxiPJAtfytB+bAhMo2/FMARvhd3MQX+ssNsU6ObIab5fYJ7?=
 =?us-ascii?Q?Nt9fhsk7CY90fcdgv5CKJQVIfZo4kvhGazjPLNtijkkRRkXKnTchr7HjIVTG?=
 =?us-ascii?Q?fYdSdw8cP2xM0ia9IHnNRdo8g/C5684R/tCGH34KjulWT5GkgsWECaxxITl8?=
 =?us-ascii?Q?GPUdrPuzCz4nExzotOv20sO86yMoToLYTVZ/zk5S/aWe1tVygu0x4VcTcJHr?=
 =?us-ascii?Q?oMbCsZ2KwfAPwv4B0BZtG2vQ9YEBRALfwZ3jIoFgfJGILkJpCcs4724/mH/c?=
 =?us-ascii?Q?skuTI84PUzGscA8TZeIMG3eLX56bir6nzEyKEBdANchkZ2Iyvhhb2DwkyNPG?=
 =?us-ascii?Q?ynU+OzIoeYW5JFu89lmAvtvO6yGRhtcBq7UiiFvKjW9qVtJw006E+Eib1vok?=
 =?us-ascii?Q?wGJwaXOXgOqnaOM8ZUqUKMXmtY7Xg75qma0/0eW96dabSrAprar4WHHd8cNI?=
 =?us-ascii?Q?Zg=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 8c100c57-43c0-4895-06c4-08dac69d760e
X-MS-Exchange-CrossTenant-AuthSource: MWHPR1101MB2126.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Nov 2022 00:08:02.0860
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RMnFLjsFpGzbx23kPaTkHdzgidtXIC0LZxeFI8Ryv1O/f3Ey2fUXx9MJCpfHA7qaEF0KugSVgzV0KG3rawRdsb1Ufa0FqmPkBfXFiUp7BYY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR11MB4687
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Robert Richter wrote:
> When an endpoint is found, all ports in beetween the endpoint and the
> CXL host bridge need to be created. In the RCH case there are no ports
> in between a host bridge and the endpoint. Skip the enumeration of
> intermediate ports.
> 
> The port enumeration does not only create all ports, it also
> initializes the endpoint chain by adding the endpoint to every
> downstream port up to the root bridge. This must be done also in RCD
> mode, but is much more simple as the endpoint only needs to be added
> to the host bridge's dport.
> 
> Note: For endpoint removal the cxl_detach_ep() is not needed as it is
> released in cxl_port_release().
> 
> Signed-off-by: Robert Richter <rrichter@amd.com>
> ---
>  drivers/cxl/core/port.c | 18 ++++++++++++++++--
>  1 file changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
> index d10c3580719b..e21a9c3fe4da 100644
> --- a/drivers/cxl/core/port.c
> +++ b/drivers/cxl/core/port.c
> @@ -1366,8 +1366,24 @@ int devm_cxl_enumerate_ports(struct cxl_memdev *cxlmd)
>  {
>  	struct device *dev = &cxlmd->dev;
>  	struct device *iter;
> +	struct cxl_dport *dport;
> +	struct cxl_port *port;
>  	int rc;
>  
> +	/*
> +	 * Skip intermediate port enumeration in the RCH case, there
> +	 * are no ports in between a host bridge and an endpoint. Only
> +	 * initialize the EP chain.
> +	 */
> +	if (is_cxl_restricted(cxlmd)) {

I changed this to:

	if (cxlmd->cxlds->rcd) {

...where the cxl_pci driver sets that rcd flag. Aside from keeping some
PCI specifics out of this helper it also allows RCH/RCD configurations
to be mocked with cxl_test.

> +		port = cxl_mem_find_port(cxlmd, &dport);
> +		if (!port)
> +			return -ENXIO;
> +		rc = cxl_add_ep(dport, &cxlmd->dev);

Ah, good catch, I had missed this detail previously.

> +		put_device(&port->dev);
> +		return rc;
> +	}
> +
>  	rc = devm_add_action_or_reset(&cxlmd->dev, cxl_detach_ep, cxlmd);
>  	if (rc)
>  		return rc;
> @@ -1381,8 +1397,6 @@ int devm_cxl_enumerate_ports(struct cxl_memdev *cxlmd)
>  	for (iter = dev; iter; iter = grandparent(iter)) {
>  		struct device *dport_dev = grandparent(iter);
>  		struct device *uport_dev;
> -		struct cxl_dport *dport;
> -		struct cxl_port *port;
>  
>  		if (!dport_dev)
>  			return 0;
> -- 
> 2.30.2
>