Received-SPF: pass (google.com: domain of linux-kernel+bounces-104780-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Fri, 15 Mar 2024 11:21:42 -0700
From: Dan Williams <dan.j.williams@intel.com>
To: "Li, Ming" <ming4.li@intel.com>, Dan Williams <dan.j.williams@intel.com>,
	<rrichter@amd.com>, <terry.bowman@amd.com>
CC: <linux-cxl@vger.kernel.org>, <linux-kernel@vger.kernel.org>
Subject: Re: [RFC PATCH 0/6] Add support for root port RAS error handling
Message-ID: <65f491b64f15a_aa2229497@dwillia2-mobl3.amr.corp.intel.com.notmuch>
References: <20240313083602.239201-1-ming4.li@intel.com>
 <65f3a842988d6_a9b4294f7@dwillia2-mobl3.amr.corp.intel.com.notmuch>
 <16ab732d-a009-45ee-a438-3faf048c7acd@intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <16ab732d-a009-45ee-a438-3faf048c7acd@intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?SMmxtxadEkI0f3IUlfF8NvV6ok6tFLHC/LUKA87CwzR+4wezzjbq8Q1jnqdS?=
 =?us-ascii?Q?c80Jphv8aE1EMu3SptuX3Qbtk2duPTKJXRhJTN/4ifrPQstCyhBdOSLFOjqC?=
 =?us-ascii?Q?zthyNTzb0VWxWg+UP3v+kRW6n3tSBeu79c5cSFJhRFu8PGVBAxaqMEnHI/lK?=
 =?us-ascii?Q?mQBympOC5WLFdV/2Zl6qLn1dyGSPBioUI2d/zucfeSEjxlaihhJmf/PyvXvH?=
 =?us-ascii?Q?DBt2sDFmGcyvepIVM2HM5hU6Jd50BwJZ3kh0gByBA7S2bJzSnmiWSrXRZR5l?=
 =?us-ascii?Q?i08yCKX+nC0V2mqHs7WmuJT/MCGcqN1PnqBt3sR8WxD/DScLEfkL3I//7usx?=
 =?us-ascii?Q?EGFORDr8KfkGMNVzv9I6BXE4NP21mJVntwn1CTc03V4MYWDN4oYyv/yOsoq8?=
 =?us-ascii?Q?YDwfgSbIY0UVpiNb6TMvP/NsM3nU70xp6CXom26Pv2JQEWmftJqKiaySipBZ?=
 =?us-ascii?Q?3lBAI3kjuMrFv+PQZtjN9ge3Tyj2dbqRYP0RxGeQ+M6G75BBxRLvvPkL6Wqw?=
 =?us-ascii?Q?p4igD9ifSG4Btmza8EplcvWve+qTCK3DuJ4twXKLqtgq9QewL1syozNgsSZh?=
 =?us-ascii?Q?krkQfYab0oxn+RLqkVdJSnMYc6DSbTc7cQUtWqW4Laa7g7Gs9B1O0fm62rx7?=
 =?us-ascii?Q?bb/Xe3uLt/UkTJbIHTNoenCrZMIjTlORN9Qqjeou3IjXuMx2umVZZzR4b/iv?=
 =?us-ascii?Q?3gG0ubuaccQvjlRtuINK+lrHdmjWkt3QS+VhM55xcxQovYRIwCvyQafNM2JO?=
 =?us-ascii?Q?5Jn+Pwum5RVChm84lUKvvYx0GIawpthWHSR86BFAS8znyNy6mRAcyx2mRI4g?=
 =?us-ascii?Q?16skq1qasVyl4ZV2Gzb/I+nlmrg0njSCDJZdmKsjRprs6VINgMsjfqMmoq9L?=
 =?us-ascii?Q?jvauQemJATgZ5F/RkgKOoHj4O31K8YqyyWjnRtCzeTIfzDkNy58+ibjeVvFH?=
 =?us-ascii?Q?0K2FdTM/wadD3vPyDqC5G1sk/1hoDBZhk3ZWmY9B2kFa2NyANk6uIv909YGO?=
 =?us-ascii?Q?r8svT9MNj0Umce3ometVSitzRW7rGRkmom2W6kRFQC6OBVkdQy7fTZDq4Nyr?=
 =?us-ascii?Q?5+ZYOAZ1gUuEsyINPs8xSSfv2n/ZJgNssfJEmFEsZCtEQDZ7zamevjHV83kv?=
 =?us-ascii?Q?YeDRYYuYSCQIyruVhxZ6xAECYxgkheWSUxmcZJGvVTkp3DUOzseX4TKP7WSL?=
 =?us-ascii?Q?kjb8HxpnYmxYboohT26H4jupeIZ2iZnAn1M5mCkVOCuIyQUM5ItZJwvRUSZi?=
 =?us-ascii?Q?IJi8lF0JCTZwhot26q2z02SqL17eEJgLqP9tGwBsRtLWd506gkaNQOb/RmD5?=
 =?us-ascii?Q?DjWo6grOitdbbSuiT4A672nGgtIcHF0s7tPQpIXu0L/Mo204eXtnojjJKKZ9?=
 =?us-ascii?Q?N5BJVygPABfbDDudlxTksGGudvKin8zliCzeZ0qRMR+L4zoclA7yJhE945VT?=
 =?us-ascii?Q?+grH2koSaxVsKYw1aoNuxIqJ6Igp9+yviRq4saIOAu2NxTDg7f30LKT6eFzy?=
 =?us-ascii?Q?JNGwNowi7xVXPzBP7eEJqBWiiENGYbCpGNqdjnjDL0MmG7ZvFg3SISgQ6YV+?=
 =?us-ascii?Q?2GWwt+P/KEBUoP2SNpc2yjUaacYQOuwO/jp4Bcu2/WmZut19zhABMmQXciai?=
 =?us-ascii?Q?yA=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4c6dfee4-69bb-4bd0-f924-08dc451cc4e2
X-MS-Exchange-CrossTenant-AuthSource: PH8PR11MB8107.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Mar 2024 18:21:44.5805
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HGhSG9C8ElOGuZFxr9a/RuKJ7AFDlCXtxVztmEtUj4KkOZx1IsCdnT4+AOiFLLwGCcLkZaZVet7V2w4Az++aPJl3JIXGUrOHNFqqMWKWw6Q=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR11MB8062
X-OriginatorOrg: intel.com

Li, Ming wrote:
[..]
> > I do expect direct-attach to be a predominant use case, but I want to
> > make sure that the implementation at least does not make the switch port
> > error handling case more difficult to implement.
> 
> Hi Dan,
> 
> Currently, A rough idea I have is that:
> If a CXL switch connected to the CXL RP, there should be two cases,
> 1. no CXL memory device connected to the switch, in this case, I'm not
> sure whether CXL.cachemem protocol errors is still possibly happened
> between RP and switch without CXL memory device. If not, maybe we
> don't need to consider such case?

Protocol errors can happen between any 2 ports, just like PCI protocol
errors.

> 2. a CXL memory device connected to the switch. I think cxl_pci error
> handler could also help to handle CXL.cachemem protocol errors
> happened in switch USP/DSP.

No, for 2 reasons:

* The cxl_pci driver is only for general CXL type-3 memory
  expanders. Even though no CXL.cache devices have upstream drivers they
  do exist and they would experience protocol errors that the PCI core
  needs to consider.

* When a switch is present it is possible to have a protocol error
  between the switch upstream port and the root port, and not between
  the switch downstream port and the endpoint.

The more I think about it I do not think it is appropriate for cxl_pci
to be involved in clearing root port errors in the VH case, it only
works the RCH case because of the way the device and the root-port get
combined.