Received-SPF: pass (google.com: domain of linux-kernel+bounces-103971-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Thu, 14 Mar 2024 18:45:38 -0700
From: Dan Williams <dan.j.williams@intel.com>
To: Li Ming <ming4.li@intel.com>, <dan.j.williams@intel.com>,
	<rrichter@amd.com>, <terry.bowman@amd.com>
CC: <linux-cxl@vger.kernel.org>, <linux-kernel@vger.kernel.org>, Li Ming
	<ming4.li@intel.com>
Subject: Re: [RFC PATCH 0/6] Add support for root port RAS error handling
Message-ID: <65f3a842988d6_a9b4294f7@dwillia2-mobl3.amr.corp.intel.com.notmuch>
References: <20240313083602.239201-1-ming4.li@intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240313083602.239201-1-ming4.li@intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?n+BEDVCs/+wXvlr3SRQ1vlSrQPpywd8juvBJizhhsMCoy/IXyqXMjua+7F/o?=
 =?us-ascii?Q?Z1D8a6XVO8OIXRW/eLJvG7Kf3o7wWcFns6uH4JlyQCkLepMaCBbkd0bFZtZy?=
 =?us-ascii?Q?0TbX+WcjUqCGZ05ffdDHazeX4EhwfMa8U6ESKYg+5uXC0HH9GeJr+N7P8v5s?=
 =?us-ascii?Q?2hL7QKPGHJ7ryMgiNKmdsx1bBUsFoFDQ5MXRhibkTNFF/9it8qK51r/G3N10?=
 =?us-ascii?Q?mwcDvXmMKmscjLAeqgdelyZyXP2kSRuL8apc5KbKalC/OZY0eEGRz6QqHCHy?=
 =?us-ascii?Q?/B+zgXjxlEE/+uDAimLUnqAsr9ThI/lLYQFQk0roJQFDSAx0kbXOcp/qjivD?=
 =?us-ascii?Q?BFsUhqU3EmTDyFdba19d6JcCpsVN/3JNhqtT6Y7KdJZbXtW6WJmSyJFLYTr+?=
 =?us-ascii?Q?a5esHO1VwICgYvrEygJ8Yd4S7Rp1C1QiVPtHvG4ddh7KZERlSViJC4sq0OvE?=
 =?us-ascii?Q?OGI/g4HjnbfynlF0Ij5hVq7/FR+oPmcTqhUzfGSiiz3JDnDLKtMVPSSCP83m?=
 =?us-ascii?Q?IDF1wtmHOb2iz7m9bK8gDXpRFHNZGaVeMBBdVxI41YFydT2mPcAs5hE+3zMw?=
 =?us-ascii?Q?FDR5gpMp7yfxcwwHQI0Cpya6DHB4nZ9Jn+z4Ka6iVkFCenzBKJ7wkR3vKUj+?=
 =?us-ascii?Q?PHuicyf+U/lwbqi1pq1nIfmAuDfDlJ9Jn7DHJoVtYvwchR3L4coVGo+mUAfI?=
 =?us-ascii?Q?o8fPKW09fzr/CkoakDUJJ4OZd7IWGAnVOI9q2qEq7ILE+80afHtbvAkaCzkC?=
 =?us-ascii?Q?wECDyKZINS7cFuOCzZkS8Gn42OIK/MS1E2RAeSr/PghwQQNERjZiv9avGwMp?=
 =?us-ascii?Q?GlCBoNNekwyLJ0D2IUvQUeIgsdEvWYop7SBpFz3SzOHgTEkb0s48dxcm8DXW?=
 =?us-ascii?Q?tWmIR9dDjHba7QgZygsTkpSYGWqjj4IMoVkuHrDt683d3ZuQ+87ne7JWNZjF?=
 =?us-ascii?Q?kKgqJ8/BcsPl8/0zi+nku8gKM1RUMpAttZX+q04sUO8nYzXDbbhJJ3zOsvzx?=
 =?us-ascii?Q?EGE88pP0jHF/7EbXPXm0U//0+ZghVeoKvoy+pUpaKLDTYske3+GJ/5RjU64Z?=
 =?us-ascii?Q?UajvwXxCh0pCviwabr1Jpj4aBtJYdHj+vLk3ROaascTlmXepqUnyHZlGB/B8?=
 =?us-ascii?Q?XAtzhqawmqfGodxn256DcO6fOxHCwgDvbE9ewTtsoygwChC2moiaZj8h0KXs?=
 =?us-ascii?Q?ldBMIon5UaOIg8rArRHM1LKzEKEZWBy2lVA9EpVYFqq7zIMVqx61o4wNYwZB?=
 =?us-ascii?Q?saD7ZbXXZ9LmOJI+jd8IwPPWnOY17VygGbkQVfVU3pi/MjxRxcX6mVRPdLk8?=
 =?us-ascii?Q?Nu5VP/OFAcx5GQXaEeHlfK3oJWyd9zdpRY9O35Jlp34w6aW3/ayGsm8qH3Gh?=
 =?us-ascii?Q?ECtndWa8mx9F0MXOY/JSS0zRUjQG/kkLkSzeEvK1w3EkM2wWzlISf7k3a4pf?=
 =?us-ascii?Q?GMB+3Ul0wuR3RY8WOor71qdxZczg0GtsOA98JCMh3mcKSj6is3kUEnFkKiW+?=
 =?us-ascii?Q?HvCQtrOakoRYOL4I/mU8pBe9K5m+ZSMobLyywdJ/j4LsHK372pAbC2CmEtlR?=
 =?us-ascii?Q?vv3CEhMpbI920hXm3knLr0D1bw0EpQUC2naPmKRZvM6nqUYyn2BRPsBT9Fe4?=
 =?us-ascii?Q?MQ=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 01d580d9-c36f-47ae-c3fd-08dc44919ee8
X-MS-Exchange-CrossTenant-AuthSource: PH8PR11MB8107.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Mar 2024 01:45:40.9136
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 06Cl3Sxs3uTBdqM0DWrFFl7VJ/d2PAePpXIxrFwKDGgl3cmE0Ks9AtJ7z75WjDrv0BR3oz3AvNUhEOyST9T2TpAYShxvNsDVWnAPXaybsNM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB4579
X-OriginatorOrg: intel.com

Li Ming wrote:
> Protocol errors signaled to a CXL root port may be captured by a Root
> Complex Event Collector(RCEC). If those errors are not cleared and
> reported the system owner loses forensic information for system failure
> analysis.
> 
> Per CXL r3.1 section 9.18.1.5, the recommendation for this case from CXL
> specification is the 'Else' statement in 'IMPLEMENTATION NODE' under
> 'Table 9-24 RDPAS Structure':
> 
> 	"Probe all CXL Downstream Ports and determine whether they have logged an
> 	error in the CXL.io or CXL.cachemem status registers."
> 
> The CXL subsystem already supports RCH RAS Error handling that has a
> dependency on the RCEC. Reuse and extend that RCH topoogy support to
> handle reported errors in the VH topology case. The implementation is
> composed of:
> * Provide a new interface from RCEC side to support walk all devices
>   under RCEC and RCEC associated bus range. PCIe AER core uses this
>   interface to walk all CXL endpoints and all CXL root ports under the
>   bus ranges.
> * Update the PCIe AER core to enable Uncorrectable Internal Errors and
>   Correctable Internal Errors report for root ports.

Thanks for the above background.

> * Invoke the cxl_pci error handler for RCEC reported errors.

So what do you expect happens when a switch is involved? In the RCH case
it knows that the only thing that can fire RCEC is a root complex
integrated endpoint implementation driven by cxl_pci. In the VH case it
could be a switch.

> * Handle root-port errors in the cxl_pci handler when the device is
>   direct attached.

I do expect direct-attach to be a predominant use case, but I want to
make sure that the implementation at least does not make the switch port
error handling case more difficult to implement.