Received-SPF: pass (google.com: domain of linux-kernel+bounces-104033-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <f2dd9f36-5e69-4c6e-9640-7140a694086c@intel.com>
Date: Fri, 15 Mar 2024 11:43:01 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH 4/6] PCI/AER: Extend RCH RAS error handling to support
 VH topology case
To: Dan Williams <dan.j.williams@intel.com>, <rrichter@amd.com>,
	<terry.bowman@amd.com>
CC: <linux-cxl@vger.kernel.org>, <linux-kernel@vger.kernel.org>
References: <20240313083602.239201-1-ming4.li@intel.com>
 <20240313083602.239201-5-ming4.li@intel.com>
 <65f3b2b29cfe9_aa22294e8@dwillia2-mobl3.amr.corp.intel.com.notmuch>
Content-Language: en-US
From: "Li, Ming" <ming4.li@intel.com>
Organization: Intel
In-Reply-To: <65f3b2b29cfe9_aa22294e8@dwillia2-mobl3.amr.corp.intel.com.notmuch>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?V3dhaTdpakVhUDU5UnZJSVRuYnloTmpaY2hUY0Z4VWhuelRyNVBGT1VoVll0?=
 =?utf-8?B?aUNtSVZTbmRYejFWWFh0am1qWVBJZlEvS0dGNFp2Y1g1T0JhaHJaQnF4Q0Zs?=
 =?utf-8?B?S1I1c2NwUVlLdURLdndEdFprTElkclpqaWZBZGorc1ZjSTJxNHE0cXIvZjVG?=
 =?utf-8?B?TWhVWFp6UnRMdmQ2L1I0dWdCT2pxeEE5cEFPS0EzZEVFU091T1VtMUpmTGVx?=
 =?utf-8?B?cFJEYTQrMWxrZXBzS0VLQmY4dk14YVpScnRRYm5rYUZURmlPYlM0YlY1UWla?=
 =?utf-8?B?dXVtdUR5bUhwdmFhcjJOcGVPVzlQbFZsaFdYRElxbDEvYVFIdTRzeEZXZDNF?=
 =?utf-8?B?aTNsb3hhT2lrVlZSbFNtZEh5N3NGMmF5QVV3N1V5RElsYUhOMVM3aG9WeHN3?=
 =?utf-8?B?blZxU2JLOHhFdXlvaHVCN05kMktncVR3YTJFSjhueWdXNDkvenBZNG42QUd5?=
 =?utf-8?B?WlZkRS8veXowaGMrUDNQSExNQlRjRTc4RlBXSFJLZGQrY0lNcmxvZHE1M0NQ?=
 =?utf-8?B?cmE1cHQreTRsQzk2UE1JWitQakxVZS8xOHQzMXZ1UFlDTmI3N3pDS0sxT2VR?=
 =?utf-8?B?SzNkR2x6c01oNGkyMGIxSmltM2pBMURsWll6YkRvZzlzVGgwWFRaWmI3R3l1?=
 =?utf-8?B?RE01Qis2UUQybmtPMldTR0t3empMejRtcjBGaXFBbllOWFptdk9YcjVQNWZV?=
 =?utf-8?B?SlJ2MkVTNW5aaC84SVRKcVZ1MzBXc1Y0QTZVL1NidC82RVZRTmgzcTFCRy9n?=
 =?utf-8?B?YTJkbTBFVHJuaEZ4QlIrTzhZb1FXcEhYZTBjNVA2UVdSWXJCTXhwVjdudWlL?=
 =?utf-8?B?cFB2U1hRMjNBLytaYnVCNHd0b3c3Q3d3dkRDVjZ5M25ab2JrNTNOeGFKNUsy?=
 =?utf-8?B?RDl4WGpZamg5SVV6R0lkWHZOS3NjQTZwS3dDRDNkdUJGSFNJbVRYV0UwVzdl?=
 =?utf-8?B?TjgxN3FIeEFUVUVzNnBYaHJwNVlOMjJSVE5xOEtobnRqN3o1a1QwVFJqbm4v?=
 =?utf-8?B?OW5wSEZIdFBzeGQ2SHlpN2l6OU5TZVVXWG9ONVBZQWd1b0IvSllPQWEzVDJq?=
 =?utf-8?B?dk1JUGc2QW51ZnZDVzUyRmc5UkFTVEFWaThLa05qUktrdFZYMmgrTXk0eHhm?=
 =?utf-8?B?cnpJdDdBeEJHZHJyc3dmaC9vdG5sWlRvNDBoMHVMcjJPa0wwcEU4TldBVExj?=
 =?utf-8?B?UStxVXhvVmJWdERkWkZ6bE8yRml5M08vdnZOVTJzVC9EY3dTMTIxUlA3bHZn?=
 =?utf-8?B?OC90bE5VYVA0UWh2a0RBT1dPemR6OEFtUE9PUS80V1pFVHlCZVpwVU14SHNa?=
 =?utf-8?B?a2h3em5qZThrQkN2WDBnTGdhL1JROEE3M3hMVkEvUzY0UkFNVGZSc2YvWXZa?=
 =?utf-8?B?ai9jQStrN0xadXZWVTNvQ3AyNkJlRFNTVjNtWmxVclRLVjlGOGhnemxKZm1r?=
 =?utf-8?B?YVk5VWJ1QWhpY1lYelNoVUxqdzFBMGt3ZXQzbCtSL1RvaEpMVmF5YUdRK2dW?=
 =?utf-8?B?MUhUVDRQMnlqSGRTV2VGRE1Gc1VnNnEwclRjQnpMZ2lnT2tXaEIyR1gvdzVW?=
 =?utf-8?B?VTk3cmFURjV5WHRCbWJraHhQQ2Q0YzJZZ0IwSW05bTRRZm4vUjNQK0JicG51?=
 =?utf-8?B?L05TTG4wakFBNlNjMUsra3NaOWpHS1ZMb21kQjU5VXdVYWFmTzgzZTlLSDAy?=
 =?utf-8?B?ZkY5UkU1RHNuNEpTNmF5bVJDVXZ4clVRUTd4SHVoZVBpV1VKejllNHJVS1ZP?=
 =?utf-8?B?V1l3SUNuS1I2OVVTb0dDYUpPcHlZWUdPdjU4bDFCbTgzanMrNEI4ZVRnQTVa?=
 =?utf-8?B?MHBtaStJL2taZW1GM2dReEZGc1BxalRhSDBKMEZtOG5oZFcyanVkbWp0S29s?=
 =?utf-8?B?SG9RYUhvNitOY2I5Q0JyaGZZZXRSck9vc2N4OGpkV3BtZS9QRmxjUjMwdjY1?=
 =?utf-8?B?eVM0Z2xIQWNDV09WdzlTRmM0L3l5TjQySkdGVSs0UnNzWnpkbjRnMzhpc1RC?=
 =?utf-8?B?VnJWVkRuRWJBR0ZuNXR4a05WeVlVUUZoeitJUkdsMjZadnVLNzNsSE16NVph?=
 =?utf-8?B?dGlkb05vcVZWVnhad08wL3VNZ1Fya2JlUE1jREl3c09XUkhQdnRwN3EyVkYr?=
 =?utf-8?Q?f0R1tOLk6zZz9NNFQZnWrPe9I?=
X-MS-Exchange-CrossTenant-Network-Message-Id: a39af957-3bf8-4a1b-e8a1-08dc44a207e2
X-MS-Exchange-CrossTenant-AuthSource: IA1PR11MB7200.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Mar 2024 03:43:09.0625
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: j1DXsWZOkaOxh8J0gH9gn9Bv1AzHnjUZpqZd4q83Q7XOORSIppZ+DPG5vrxGGCuzr2GF4K+BJ4IW/TLA9ycHmA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR11MB7778
X-OriginatorOrg: intel.com

On 3/15/2024 10:30 AM, Dan Williams wrote:
> Li Ming wrote:
>> When RCEC captures CXL.cachemem protocol errors detected by CXL root
>> port, the recommendation from CXL r3.1 9.18.1.5 is :
>>
>> 	"Probe all CXL Downstream Ports and determine whether they have logged an
>> 	error in the CXL.io or CXL.cachemem status registers."
>>
>> The flow is similar with RCH RAS error handling, so reuse it to support
>> above case.
>>
>> Signed-off-by: Li Ming <ming4.li@intel.com>
>> ---
>>  drivers/pci/pcie/aer.c | 20 ++++++++++++--------
>>  1 file changed, 12 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c
>> index 364c74e47273..79bfa5fb78f4 100644
>> --- a/drivers/pci/pcie/aer.c
>> +++ b/drivers/pci/pcie/aer.c
>> @@ -996,11 +996,15 @@ static bool is_internal_error(struct aer_err_info *info)
>>  	return info->status & PCI_ERR_UNC_INTN;
>>  }
>>  
>> -static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data)
>> +static int cxl_handle_error_iter(struct pci_dev *dev, void *data)
>>  {
>>  	struct aer_err_info *info = (struct aer_err_info *)data;
>>  	const struct pci_error_handlers *err_handler;
>>  
>> +	/* Skip the RCiEP devices not associating with RCEC */
>> +	if ((pci_pcie_type(dev) == PCI_EXP_TYPE_RC_END) &&
>> +	    !dev->rcec)
>> +		return 0;
>>  	if (!is_cxl_mem_dev(dev) || !cxl_error_is_native(dev))
>>  		return 0;
> 
> is_cxl_mem_dev(dev) will always be false in the VH case, so how does
> this change help the VH case?

Hi Dan,

I think it won't be false if the CXL memory device is an endpoint.
pcie_walk_rcec_all() will walk all pci_dev in RCEC assocaited bus ranges. So these two checkings can help us to filter:
1. CXL memory device is an RCiEP associated with RCEC in the RCH case
2. CXL memory device is not an RCiEP, so it should be an endpoint in the VH case.