Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <ea00deef-fcbb-a2d0-f806-cfe73ac0de42@intel.com>
Date:   Thu, 22 Jun 2023 15:38:22 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Betterbird/102.11.1
Subject: Re: [PATCH v6 03/27] cxl: Updates for CXL Test to work with RCH
To:     Terry Bowman <terry.bowman@amd.com>, <alison.schofield@intel.com>,
        <vishal.l.verma@intel.com>, <ira.weiny@intel.com>,
        <bwidawsk@kernel.org>, <dan.j.williams@intel.com>,
        <Jonathan.Cameron@huawei.com>, <linux-cxl@vger.kernel.org>
CC:     <rrichter@amd.com>, <linux-kernel@vger.kernel.org>,
        <bhelgaas@google.com>
References: <20230622035126.4130151-1-terry.bowman@amd.com>
 <20230622035126.4130151-4-terry.bowman@amd.com>
Content-Language: en-US
From:   Dave Jiang <dave.jiang@intel.com>
In-Reply-To: <20230622035126.4130151-4-terry.bowman@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?REVLZGdzS1lTTHJlRlRaOXBRVVdIWmovZ2QwM2N5Q1hac2dxTGpHWTZEeUFr?=
 =?utf-8?B?MTNUUmFuYXFTVEZrdk03WjJtMUpURVJSZk4wd1NQa1dlTDlJNDcvZVpLWE9U?=
 =?utf-8?B?aFp5VktXMDZySG5od3pOTEhFZW4zZ0Q5ZW5PeFN1VnN2NlEzMWFHY05oMjZo?=
 =?utf-8?B?WUI5UWZDMGtyMTNHK3U5NDNERnlCSGxqYk9jS01HcFdjUk1UbWN2SWtZVWhu?=
 =?utf-8?B?UXFoYnJUUTQwdWFCQVdYaVAyaU1iRld1dTRuU2pmNnFKVGFtbFg2R3ZXSTc5?=
 =?utf-8?B?b2lXWUpra1hYL00wNjJ4cDNMaDFTUVJnUisyN0F0dVBNeTFGNkx6dDVFNFNB?=
 =?utf-8?B?WVJXTTZNNnNPcVYyTUFndEplUXJjSnhLeHhrQmdjQTJPZGZYU21JY0d1dmN6?=
 =?utf-8?B?WWp1VnRzRVNuRTdxUk91WWdFcVNxK2Evb2hIeHpnNkdqR0VOU21OMU9sSm03?=
 =?utf-8?B?WS9xL2YwRkRUdUZVQWN4NUIvb0ZWNHRHVmp4Q1RyMEVZVUpXaDF2RG1sNUEy?=
 =?utf-8?B?TWdiaEc5ei9Qa0lkMktpb3Jpek1KMVZCdFpldEtFZE9tTndJSEpCakpLYmtS?=
 =?utf-8?B?L0g4TW90Yk1Vb2c2RnlYZHNVdVp3bEdyNFRObTNLbGhtMXIwR0FFY2dhaWhI?=
 =?utf-8?B?RVBRLyttMXVWMjd5UDZVR3llMUdnRWxwakM1ZlFRc1V1SGFYT050TUx5NDNx?=
 =?utf-8?B?VlZ0MDRJcjk1LzNWZDAwZXd4d0g3aHB4QzNCNVZPQ24wdER4SHZzeU42cHlq?=
 =?utf-8?B?eGgvMkdoNGloaVFsWVFUcmNWSHFRUUMwMmczSDRLNVloVWhCR3l3Z0JHYVBj?=
 =?utf-8?B?K0FHTU8wRE5oK25wS1Q1TDFDM2IzV2daVE0xZGNVcGJOWmkzcmlmQ3Z0VWhp?=
 =?utf-8?B?MmlmODlWWFJHUHc4VXZ0aGZub0hCMVpuZDFTRzJJNjA5VThnRGxJSnErU0pT?=
 =?utf-8?B?YzEyT1RMbytyeUltbTgzeFMvUGs3QXZwU2dCakc3TU9EUitlZkNDZ2VjSEtO?=
 =?utf-8?B?WStNSHN6WFBJeE1LZGlxOWdKQXdZT3pKc1podHJVekhvbW9ObEdyREZpQmtp?=
 =?utf-8?B?K1dxa1lJQzJoQ2FVOUl5NDZUZUFTd1JlUmxVSmY5Mk82OUZtRFcrZk9uQmZL?=
 =?utf-8?B?QTJqWEVRWGV0UExvM3d4WXorcjRnUVFYUUJZeUlsSkJ2YytybGI2TEw0Mitj?=
 =?utf-8?B?TjJsZHdlQ2JLYTdhY2thT0lITWQvbHlSaXdQVXY3SjJLR01LdlpiSlJUL0JH?=
 =?utf-8?B?alJKZDVCMjhjT1hDSHJUN3hqbmFJVzJobFVBZCs3d3VDalMwa1hUWXU3T3N4?=
 =?utf-8?B?RXZ0SC8rNUNCNFlySW9wYUppRjc5a0s0QjRVZ21oMEZ3dGlrTkE1YnY1U3ph?=
 =?utf-8?B?Nkc1akpBbTlSMU12VVB3ckRMcDVWcDQxd0tlZXlNSlZWek5MekhudUtOY1lo?=
 =?utf-8?B?dUxBMVgyL2lJUERHdUFoR3owTGhFK3dKWDl3VjAwcTNLVlVrZGdNYUF5OS9x?=
 =?utf-8?B?NGU3VXRuQWEzRHVCY2V0RnFuMDJvWUJSVlhEVXRqb041ZUUzK0pDVFV1THVX?=
 =?utf-8?B?ZnJZQXVDanQ5UTN5b3RhQTJCQVpjdUo4VkZESEhGZnBNeHhpRE1GbXNkSXNB?=
 =?utf-8?B?NTNpd2o1MW04enprd29jQnNSN3phNGFLQlU0cFdXczdhMjl6S2t6QThWSEVu?=
 =?utf-8?B?eXBhbXpKa3lqeUd5T096dC9hWWdySllqZVZzT1dNdnN0MTM1T2hwbE9vbTdx?=
 =?utf-8?B?Q2tKMmo3dkovSER3NEhJZHErQ0ROdFR2ZUpjRXFPVU9xaTYyV01tY05aTTFw?=
 =?utf-8?B?RW16ZklzNjcyMzZBTTRvUDE3UkM0WllQamovVkxqbktQS0VqV00vT2duSno2?=
 =?utf-8?B?d1ZkZjQ0MEoyY1hZUy8rNXh4Mkhaa2xzbGpxMDkva3N2VlcyNWxTdkFKeW9B?=
 =?utf-8?B?RmVNbk9pMUpaRVJpOFFpeW9WTUFZeTZISUJZQU9MZHRoMGp2WHltVlJuVU9P?=
 =?utf-8?B?SkxQSXZYTDBwS3ZFVDFSRDFDOUVZekc2bk52bGlGSmlkVFA1YXJVSEZYNmxs?=
 =?utf-8?B?N09VTUdNbkNJb1RuRklrdWQvaUNJb25ob1k4R25seGhkMFVwM2V3YjVaYWtG?=
 =?utf-8?Q?lDObRCjkvvNsYMF7X72vSrSPC?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0f137619-16c4-441b-9b6d-08db7371648a
X-MS-Exchange-CrossTenant-AuthSource: PH7PR11MB5984.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2023 22:38:26.1381
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2GjQkmhl04xq/BhyrNklcnLJ7Ke8G3KQznso24csp8Vga6Oqn36PwhF5gp+Vk1cLPKoscxQDi3MPV+OiDJsuxg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB8549
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 6/21/23 20:51, Terry Bowman wrote:
> From: Dan Williams <dan.j.williams@intel.com>
> 
> CXL test needs to be updated to work with latest RCH changes.
> Add CXL test support for devm_cxl_add_rch_dport and
> cxl_rcd_component_reg_phys functions.
> 
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Terry Bowman <terry.bowman@amd.com>
> Signed-off-by: Robert Richter <rrichter@amd.com>

Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> ---
>   drivers/cxl/core/core.h       |  8 ++++++++
>   drivers/cxl/core/port.c       |  4 ++--
>   drivers/cxl/core/regs.c       | 15 +++++++++++----
>   drivers/cxl/cxl.h             | 11 +++--------
>   drivers/cxl/mem.c             |  4 ++--
>   tools/testing/cxl/Kbuild      |  3 ++-
>   tools/testing/cxl/test/cxl.c  | 10 ----------
>   tools/testing/cxl/test/mock.c | 34 +++++++++++++++++++++++++++-------
>   tools/testing/cxl/test/mock.h |  3 ---
>   9 files changed, 55 insertions(+), 37 deletions(-)
> 
> diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
> index 27f0968449de..bd0a5788c696 100644
> --- a/drivers/cxl/core/core.h
> +++ b/drivers/cxl/core/core.h
> @@ -63,6 +63,14 @@ int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, unsigned long long size);
>   int cxl_dpa_free(struct cxl_endpoint_decoder *cxled);
>   resource_size_t cxl_dpa_size(struct cxl_endpoint_decoder *cxled);
>   resource_size_t cxl_dpa_resource_start(struct cxl_endpoint_decoder *cxled);
> +
> +enum cxl_rcrb {
> +	CXL_RCRB_DOWNSTREAM,
> +	CXL_RCRB_UPSTREAM,
> +};
> +resource_size_t __rcrb_to_component(struct device *dev, resource_size_t rcrb,
> +				    enum cxl_rcrb which);
> +
>   extern struct rw_semaphore cxl_dpa_rwsem;
>   
>   int cxl_memdev_init(void);
> diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
> index 1a3f8729a616..45f5299af7a6 100644
> --- a/drivers/cxl/core/port.c
> +++ b/drivers/cxl/core/port.c
> @@ -939,8 +939,8 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev,
>   		return ERR_PTR(-ENOMEM);
>   
>   	if (rcrb != CXL_RESOURCE_NONE) {
> -		component_reg_phys = cxl_rcrb_to_component(dport_dev,
> -						rcrb, CXL_RCRB_DOWNSTREAM);
> +		component_reg_phys = __rcrb_to_component(dport_dev, rcrb,
> +							 CXL_RCRB_DOWNSTREAM);
>   		if (component_reg_phys == CXL_RESOURCE_NONE) {
>   			dev_warn(dport_dev, "Invalid Component Registers in RCRB");
>   			return ERR_PTR(-ENXIO);
> diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c
> index 1476a0299c9b..564dd430258a 100644
> --- a/drivers/cxl/core/regs.c
> +++ b/drivers/cxl/core/regs.c
> @@ -332,9 +332,8 @@ int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type,
>   }
>   EXPORT_SYMBOL_NS_GPL(cxl_find_regblock, CXL);
>   
> -resource_size_t cxl_rcrb_to_component(struct device *dev,
> -				      resource_size_t rcrb,
> -				      enum cxl_rcrb which)
> +resource_size_t __rcrb_to_component(struct device *dev, resource_size_t rcrb,
> +				    enum cxl_rcrb which)
>   {
>   	resource_size_t component_reg_phys;
>   	void __iomem *addr;
> @@ -395,4 +394,12 @@ resource_size_t cxl_rcrb_to_component(struct device *dev,
>   
>   	return component_reg_phys;
>   }
> -EXPORT_SYMBOL_NS_GPL(cxl_rcrb_to_component, CXL);
> +
> +resource_size_t cxl_rcd_component_reg_phys(struct device *dev,
> +					   struct cxl_dport *dport)
> +{
> +	if (!dport->rch)
> +		return CXL_RESOURCE_NONE;
> +	return __rcrb_to_component(dev, dport->rcrb, CXL_RCRB_UPSTREAM);
> +}
> +EXPORT_SYMBOL_NS_GPL(cxl_rcd_component_reg_phys, CXL);
> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> index a5cd661face2..28888bb0c088 100644
> --- a/drivers/cxl/cxl.h
> +++ b/drivers/cxl/cxl.h
> @@ -262,14 +262,9 @@ int cxl_map_device_regs(struct device *dev, struct cxl_device_regs *regs,
>   enum cxl_regloc_type;
>   int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type,
>   		      struct cxl_register_map *map);
> -
> -enum cxl_rcrb {
> -	CXL_RCRB_DOWNSTREAM,
> -	CXL_RCRB_UPSTREAM,
> -};
> -resource_size_t cxl_rcrb_to_component(struct device *dev,
> -				      resource_size_t rcrb,
> -				      enum cxl_rcrb which);
> +struct cxl_dport;
> +resource_size_t cxl_rcd_component_reg_phys(struct device *dev,
> +					   struct cxl_dport *dport);
>   
>   #define CXL_RESOURCE_NONE ((resource_size_t) -1)
>   #define CXL_TARGET_STRLEN 20
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index 519edd0eb196..45d4c32d78b0 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -72,8 +72,8 @@ static int devm_cxl_add_endpoint(struct device *host, struct cxl_memdev *cxlmd,
>   	 * typical register locator mechanism.
>   	 */
>   	if (parent_dport->rch && cxlds->component_reg_phys == CXL_RESOURCE_NONE)
> -		component_reg_phys = cxl_rcrb_to_component(
> -			&cxlmd->dev, parent_dport->rcrb, CXL_RCRB_UPSTREAM);
> +		component_reg_phys =
> +			cxl_rcd_component_reg_phys(&cxlmd->dev, parent_dport);
>   	else
>   		component_reg_phys = cxlds->component_reg_phys;
>   	endpoint = devm_cxl_add_port(host, &cxlmd->dev, component_reg_phys,
> diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild
> index 6f9347ade82c..8a87d7d5f7f8 100644
> --- a/tools/testing/cxl/Kbuild
> +++ b/tools/testing/cxl/Kbuild
> @@ -12,7 +12,8 @@ ldflags-y += --wrap=devm_cxl_enumerate_decoders
>   ldflags-y += --wrap=cxl_await_media_ready
>   ldflags-y += --wrap=cxl_hdm_decode_init
>   ldflags-y += --wrap=cxl_dvsec_rr_decode
> -ldflags-y += --wrap=cxl_rcrb_to_component
> +ldflags-y += --wrap=devm_cxl_add_rch_dport
> +ldflags-y += --wrap=cxl_rcd_component_reg_phys
>   
>   DRIVERS := ../../../drivers
>   CXL_SRC := $(DRIVERS)/cxl
> diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c
> index bf00dc52fe96..f5c04787bcc8 100644
> --- a/tools/testing/cxl/test/cxl.c
> +++ b/tools/testing/cxl/test/cxl.c
> @@ -971,15 +971,6 @@ static int mock_cxl_port_enumerate_dports(struct cxl_port *port)
>   	return 0;
>   }
>   
> -resource_size_t mock_cxl_rcrb_to_component(struct device *dev,
> -					   resource_size_t rcrb,
> -					   enum cxl_rcrb which)
> -{
> -	dev_dbg(dev, "rcrb: %pa which: %d\n", &rcrb, which);
> -
> -	return (resource_size_t) which + 1;
> -}
> -
>   static struct cxl_mock_ops cxl_mock_ops = {
>   	.is_mock_adev = is_mock_adev,
>   	.is_mock_bridge = is_mock_bridge,
> @@ -988,7 +979,6 @@ static struct cxl_mock_ops cxl_mock_ops = {
>   	.is_mock_dev = is_mock_dev,
>   	.acpi_table_parse_cedt = mock_acpi_table_parse_cedt,
>   	.acpi_evaluate_integer = mock_acpi_evaluate_integer,
> -	.cxl_rcrb_to_component = mock_cxl_rcrb_to_component,
>   	.acpi_pci_find_root = mock_acpi_pci_find_root,
>   	.devm_cxl_port_enumerate_dports = mock_cxl_port_enumerate_dports,
>   	.devm_cxl_setup_hdm = mock_cxl_setup_hdm,
> diff --git a/tools/testing/cxl/test/mock.c b/tools/testing/cxl/test/mock.c
> index 284416527644..30119a16ae85 100644
> --- a/tools/testing/cxl/test/mock.c
> +++ b/tools/testing/cxl/test/mock.c
> @@ -259,24 +259,44 @@ int __wrap_cxl_dvsec_rr_decode(struct device *dev, int dvsec,
>   }
>   EXPORT_SYMBOL_NS_GPL(__wrap_cxl_dvsec_rr_decode, CXL);
>   
> -resource_size_t __wrap_cxl_rcrb_to_component(struct device *dev,
> -					     resource_size_t rcrb,
> -					     enum cxl_rcrb which)
> +struct cxl_dport *__wrap_devm_cxl_add_rch_dport(struct cxl_port *port,
> +						struct device *dport_dev,
> +						int port_id,
> +						resource_size_t rcrb)
> +{
> +	int index;
> +	struct cxl_dport *dport;
> +	struct cxl_mock_ops *ops = get_cxl_mock_ops(&index);
> +
> +	if (ops && ops->is_mock_port(dport_dev)) {
> +		dport = devm_cxl_add_dport(port, dport_dev, port_id,
> +					   CXL_RESOURCE_NONE);
> +		if (!IS_ERR(dport))
> +			dport->rch = true;
> +	} else
> +		dport = devm_cxl_add_rch_dport(port, dport_dev, port_id, rcrb);
> +	put_cxl_mock_ops(index);
> +
> +	return dport;
> +}
> +EXPORT_SYMBOL_NS_GPL(__wrap_devm_cxl_add_rch_dport, CXL);
> +
> +resource_size_t __wrap_cxl_rcd_component_reg_phys(struct device *dev,
> +						  struct cxl_dport *dport)
>   {
>   	int index;
>   	resource_size_t component_reg_phys;
>   	struct cxl_mock_ops *ops = get_cxl_mock_ops(&index);
>   
>   	if (ops && ops->is_mock_port(dev))
> -		component_reg_phys =
> -			ops->cxl_rcrb_to_component(dev, rcrb, which);
> +		component_reg_phys = CXL_RESOURCE_NONE;
>   	else
> -		component_reg_phys = cxl_rcrb_to_component(dev, rcrb, which);
> +		component_reg_phys = cxl_rcd_component_reg_phys(dev, dport);
>   	put_cxl_mock_ops(index);
>   
>   	return component_reg_phys;
>   }
> -EXPORT_SYMBOL_NS_GPL(__wrap_cxl_rcrb_to_component, CXL);
> +EXPORT_SYMBOL_NS_GPL(__wrap_cxl_rcd_component_reg_phys, CXL);
>   
>   MODULE_LICENSE("GPL v2");
>   MODULE_IMPORT_NS(ACPI);
> diff --git a/tools/testing/cxl/test/mock.h b/tools/testing/cxl/test/mock.h
> index bef8817b01f2..a94223750346 100644
> --- a/tools/testing/cxl/test/mock.h
> +++ b/tools/testing/cxl/test/mock.h
> @@ -15,9 +15,6 @@ struct cxl_mock_ops {
>   					     acpi_string pathname,
>   					     struct acpi_object_list *arguments,
>   					     unsigned long long *data);
> -	resource_size_t (*cxl_rcrb_to_component)(struct device *dev,
> -						 resource_size_t rcrb,
> -						 enum cxl_rcrb which);
>   	struct acpi_pci_root *(*acpi_pci_find_root)(acpi_handle handle);
>   	bool (*is_mock_bus)(struct pci_bus *bus);
>   	bool (*is_mock_port)(struct device *dev);