Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 9 Jun 2023 18:44:41 -0700
From:   Dan Williams <dan.j.williams@intel.com>
To:     Terry Bowman <terry.bowman@amd.com>, <alison.schofield@intel.com>,
        <vishal.l.verma@intel.com>, <ira.weiny@intel.com>,
        <bwidawsk@kernel.org>, <dan.j.williams@intel.com>,
        <dave.jiang@intel.com>, <Jonathan.Cameron@huawei.com>,
        <linux-cxl@vger.kernel.org>
CC:     <terry.bowman@amd.com>, <rrichter@amd.com>,
        <linux-kernel@vger.kernel.org>, <bhelgaas@google.com>
Subject: RE: [PATCH v5 11/26] cxl/pci: Early setup RCH dport component
 registers from RCRB
Message-ID: <6483d589153b0_e067a2943e@dwillia2-xfh.jf.intel.com.notmuch>
References: <20230607221651.2454764-1-terry.bowman@amd.com>
 <20230607221651.2454764-12-terry.bowman@amd.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230607221651.2454764-12-terry.bowman@amd.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?09eDR0JtePB+l/f5+QqiECe6AYczZvXOTD7dgCH7p6sZqYew1dL2TDCSuWyW?=
 =?us-ascii?Q?9l141oTRNCRRtB5aIoXqpZMwgixxLVkNVdoiVeev/ZUXCPcUgMcHxXmeEa8l?=
 =?us-ascii?Q?Hi6BTV14gf3vFNJI7GZfmZuiCzO6FP2aA9p27dhn2jMeJkvDBVZdx1S9sOzE?=
 =?us-ascii?Q?mNiO6zFIPKtbw/1Vj3ggiF4jE8Wt0xJ4LdxFSuCHUFFUqZyOJopVMSYNxa22?=
 =?us-ascii?Q?muUDU059OL/IHaVy+k2qA8S0CQn31ErqJw5RXNyhRSuOPdX1VoQXrkc12pkX?=
 =?us-ascii?Q?vZN0CbQQ3bFEkF4tvpmBJ4rFhY/iLaGiN3UPV3PZuzHDQs6KQyVk2QYT6ZxD?=
 =?us-ascii?Q?UhqUF1urmbEu23p8N9oPxcmpwYzWKWV+5MTKVzolB0Ri7tTkF0tPXDW4n+tp?=
 =?us-ascii?Q?CpCPqruTBHZsW6rrLsVEQJLlf6cyUavWd2T0FT98oIJQqVJTXA5DFrzDof/H?=
 =?us-ascii?Q?itUS26VKBRhuouGgGSCXfqzlQX99PV1heg9aU6B3UmeH7E+9zlaKfG3iLep8?=
 =?us-ascii?Q?IkAk6OgtofQR8a/AdlKLeSKiwl5xx98ebrhNtMstaItK6ZSkZdN+uHlZewD9?=
 =?us-ascii?Q?Tk0IaaJTtoTc17gBRdK66O3YaeDKqCcUpf4xmUQ7/491SlauEE6IwgJJh8Wp?=
 =?us-ascii?Q?06CQXPp3GebGLvIPWZVLqeIkWjVpCxgKulAjvvQNx8qPZ5HVV1KW78rR7b0y?=
 =?us-ascii?Q?l07EbkZ2WRLW4yoCoO41DdqXgHaOBVBtItGagSdhui+vmhJMcMx1yPBO8lMj?=
 =?us-ascii?Q?gOYqvtgk4D3rqDurFifoJlQRVJ2RxVAT8aUKfwjwQDH/P/1HwD8RiGN5ACX3?=
 =?us-ascii?Q?cXIp4mBkAmQ8Hq1RriXiKfaVRb50apV8XgtTo2i65g+7fIpAyfCm4KDdTGa1?=
 =?us-ascii?Q?Aeb3YUpCIcPZnuctl/zJKue8PVX38KGYc7NTNavhGAscO9CC1CuhLvzSeWgn?=
 =?us-ascii?Q?egrffYaIztkxy5dA7BIuapCQxE5usNRcdl3ClPq2fsbY6orwuZ/4d9xxakMW?=
 =?us-ascii?Q?Xs9YHpG2Tgu2CLhak734o3mZiRsabYqFF1Mv8wAv+QJYSNEz5+oKDZnGBcah?=
 =?us-ascii?Q?CiW11ATuYbaV/6pENRg1mObnJZTij+vqIn2/dRE5WcWN+aBdWnA/bwiOWwdQ?=
 =?us-ascii?Q?5U7g80rmPKykJ2M9HI2lfmyWd56MGirgGf70GBzlJ6CPvPs5NXeuFyhtQ3yh?=
 =?us-ascii?Q?qmFeaNn2F5yU/Wo2kHpCGdqbVHDFZ4CZFDhwq7pcaqxnaSE+NiIB33HZwnYo?=
 =?us-ascii?Q?pjmR5yK15Y524Blx73zTe+4SiYuEZwT9nmzH3QXUOOdxcsiRUo+6mayjyEKc?=
 =?us-ascii?Q?3lEIzYqB4C1ZQ/GiIqEscvdsEM5pdnXaGRXaihdKLHgUgrILwRlStQ4M+jo6?=
 =?us-ascii?Q?cn2ydE0Fe3F2TEVzYzue2nWVK0pv8YNCgdVHUY+I2i9LYtJDiBpvqHR0gD6b?=
 =?us-ascii?Q?y+sOiiwl6ECt0IgBVO8HGMYs+IsPwWpw0wfP3ILM3DdFj8TxS28HPdacWCsR?=
 =?us-ascii?Q?BcVkYryWqT4uXKqS+GFy/zmPsHJbXVrr4l1Yn7nlM5sj0Y15dsrwZi7ppu76?=
 =?us-ascii?Q?3VaXpaU+c1ZyV2m0U4INaa7+IyVhNFG3nAoWrAAKYryLst3Jlas9gYd9jBQ6?=
 =?us-ascii?Q?lg=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: a7808c56-1926-4555-4208-08db6954449b
X-MS-Exchange-CrossTenant-AuthSource: PH8PR11MB8107.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jun 2023 01:44:45.5024
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7GsruUwuLs6Cls96eguTjiSIWMUzxYq32sPjhfr+hSdrmkEv9HSVZJ86cWQqGQg0gpdQbT4bcnE/BUJByWM7wiZiALn620K9VpqUmcjaujo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR11MB6963
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Terry Bowman wrote:
> From: Robert Richter <rrichter@amd.com>
> 
> CXL RAS capabilities must be enabled and accessible as soon as the CXL
> endpoint is detected in the PCI hierarchy and bound to the cxl_pci
> driver. This needs to be independent of other modules such as cxl_port
> or cxl_mem.
> 
> CXL RAS capabilities reside in the Component Registers. For an RCH
> this is determined by probing RCRB which is implemented very late once
> the CXL Memory Device is created.
> 
> Change this by moving the RCRB probe to the cxl_pci driver. Do this by
> using a new introduced function cxl_pci_find_port() similar to
> cxl_mem_find_port() to determine the involved dport by the endpoint's
> PCI handle. Plug this into the existing cxl_pci_setup_regs() function
> to setup Component Registers. Probe the RCRB in case the Component
> Registers cannot be located through the CXL Register Locator
> capability.
> 
> This unifies code and early sets up the Component Registers at the
> same time for both, VH and RCH mode. Only the cxl_pci driver is
> involved for this. This allows an early mapping of the CXL RAS
> capability registers.
> 
> Signed-off-by: Robert Richter <rrichter@amd.com>
> Signed-off-by: Terry Bowman <terry.bowman@amd.com>
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> ---
>  drivers/cxl/core/port.c |  7 +++++++
>  drivers/cxl/cxl.h       |  2 ++
>  drivers/cxl/mem.c       | 10 ----------
>  drivers/cxl/pci.c       | 37 ++++++++++++++++++++++++++++++++++++-
>  4 files changed, 45 insertions(+), 11 deletions(-)
> 
[..]
> diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c
> index 945ca0304d68..2975b232fcd1 100644
> --- a/drivers/cxl/pci.c
> +++ b/drivers/cxl/pci.c
> @@ -274,13 +274,48 @@ static int cxl_pci_setup_mailbox(struct cxl_dev_state *cxlds)
>  	return 0;
>  }
>  
> +/* Extract RCRB, use same function interface as cxl_find_regblock(). */
> +static int cxl_rcrb_get_comp_regs(struct pci_dev *pdev,
> +				  enum cxl_regloc_type type,
> +				  struct cxl_register_map *map)
> +{
> +	struct cxl_dport *dport;
> +	resource_size_t component_reg_phys;
> +
> +	memset(map, 0, sizeof(*map));
> +	map->dev = &pdev->dev;
> +	map->resource = CXL_RESOURCE_NONE;
> +
> +	if (type != CXL_REGLOC_RBI_COMPONENT)
> +		return -ENODEV;
> +
> +	if (!cxl_pci_find_port(pdev, &dport) || !dport->rch)
> +		return -ENXIO;
> +
> +	component_reg_phys = cxl_probe_rcrb(&pdev->dev, dport->rcrb.base,
> +					    NULL, CXL_RCRB_UPSTREAM);
> +	if (component_reg_phys == CXL_RESOURCE_NONE)
> +		return -ENXIO;
> +
> +	map->resource = component_reg_phys;
> +	map->reg_type = type;
> +	map->max_size = CXL_COMPONENT_REG_BLOCK_SIZE;

One more note, I would prefer a designated initializer for this.