Date:   Mon, 13 Mar 2023 15:51:31 -0700
From:   Ira Weiny <ira.weiny@intel.com>
To:     <alison.schofield@intel.com>,
        Dan Williams <dan.j.williams@intel.com>,
        Ira Weiny <ira.weiny@intel.com>,
        Vishal Verma <vishal.l.verma@intel.com>,
        Dave Jiang <dave.jiang@intel.com>,
        Ben Widawsky <bwidawsk@kernel.org>,
        Steven Rostedt <rostedt@goodmis.org>
CC:     Alison Schofield <alison.schofield@intel.com>,
        <linux-cxl@vger.kernel.org>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v8 4/6] cxl/region: Provide region info to the cxl_poison
 trace event
Message-ID: <640fa8f32013d_22c07f294e8@iweiny-mobl.notmuch>
References: <cover.1678468593.git.alison.schofield@intel.com>
 <b5ba1187b04a18f49be9ebb91b699cfa8573eef2.1678468593.git.alison.schofield@intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <b5ba1187b04a18f49be9ebb91b699cfa8573eef2.1678468593.git.alison.schofield@intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/QLWQzZh9CEnvYUXRs5Gd75MBl6s+qmuj9YpU0LjU/5HFnEQ2TU5JRiDdB/8?=
 =?us-ascii?Q?oFI2JcyTIaqa515/JaMz0C/EsaBUxizZkn7+XW91ROccPZnjRaRDAK7yq5Qg?=
 =?us-ascii?Q?NVRTzxPYEZ504FJOy/wpzyASBuQ+cIfmmGaMUJ8j3/qI8GKuZx2+qGBp19Li?=
 =?us-ascii?Q?TsOItETf0cfA7lpJ8Wx22oe0TtJ5cwza8wGMIMmAUHV7ExSMqDZdCeL/KsUM?=
 =?us-ascii?Q?7E04UKXS9brNtRj8Vwe/QwzsZQfEBuFW92iT/LIIuz0EfWvK1XqyOFM2rfFo?=
 =?us-ascii?Q?6Es0aG49IFhSjZ6iKJDP14626r0quA4UuAk8YYc8ZRDO/18/AlDvrxlMJdiF?=
 =?us-ascii?Q?nHiOMV0NHSMaglZSJXKLzU+HCRVWlnRtyb/GA9AszCuNlU5BMZMj8HCxehI0?=
 =?us-ascii?Q?o+Mq5oUwQo7KlRDbFxao80I9BE6WGwCEo6vvvB2PmYP4rz997GfrjtwnLEvy?=
 =?us-ascii?Q?o/T8l0dM55LFNQCRXmwe6s+nA6kGp1qDglJhwpmVzPA12YrnTD0VCM4LYkfy?=
 =?us-ascii?Q?D3rpO2PGomMQiyDqWDlSQbvuY9oQvjB/nBCAiN13Dx5cH0gUuy3J16nRmlR1?=
 =?us-ascii?Q?TZBF3q0yURPNZSBayLH4sHhLZ46RCemVENnMEvSuqPTpI9zJP7BDGsv6PASj?=
 =?us-ascii?Q?C/HZU/MMHMq884rkz62WNCVX/BEBHKourMK6xV8n/mAt/FX8WPUn4HTiJAFM?=
 =?us-ascii?Q?hOVV7l5ykRmAncwikhf/ENMykVqhioMKt+xWNR6YUX4sapcATOviihfLZVBx?=
 =?us-ascii?Q?ClCMH7K7HVQuQzvveG/BHdXy05/xlmziCbMWkpvTKyGzbdWUtwkH0ewIgkKz?=
 =?us-ascii?Q?maPcrW4nU4rddcQlJ/pgXGN4pvuFuQ3d5CNOFB54ggIvX93/kdmSHZj/MBT0?=
 =?us-ascii?Q?6RrzPhHfhLn30R+MmSpLGOJlbNNqnUag2mJr00E5pMH6/pfJacmsUe2qcbi8?=
 =?us-ascii?Q?JAYohHKsOPLtbjq7DieIxh5OZgnb5z4cXNyvwafmNCtb/7M4rqZuD1B7H7tc?=
 =?us-ascii?Q?sphkbeIQ8gVujb1Z90qSIwq8ewoB7i400z8/uEcYRyvJgqBU9IBzqaXUh/Br?=
 =?us-ascii?Q?V2fPliZtap2Cv6mXyejFvOhzTN+P0fZLRFCHEnKXvOHTvSz+4SPuGP7UKSQm?=
 =?us-ascii?Q?L3VDvAAScVGdJBW9UEImdxmp3tfZGBF7D9jTTWSgC1f3Z7AQDYqPPxK62hX7?=
 =?us-ascii?Q?inMBf/JqD/z+GCDOSDWsE0U89q+mdy1+g2oOrnyU4FbAxpPCUYTC27xQsd5W?=
 =?us-ascii?Q?Z4nzEzEkG0vYLjHWC7kmalNZBFRVFTeUrwgwoYGa/BuuI7kBYxCVhLNVzcAo?=
 =?us-ascii?Q?sHdhxy8u+eG7zOHa8b+PywRT6logd395rWLDaP+NVXCNFRLiWFA1wOlH7VuY?=
 =?us-ascii?Q?IAiNi+DJu+OwZtPJYkc5Bgh9Ty9hKoUTj7VQnEUisMf4vyuJ3c3PoeHyHkwE?=
 =?us-ascii?Q?VeK2N0duv5bdMCOLltyK8oTa+6ctEOTgjKPwNc5sI6FQKYXKjeZacJpFxsul?=
 =?us-ascii?Q?8SygshU02St77If7xyZqYknsn4GoBLtZPZm8OQ4GfcF29EjUIyVryLUmOR1q?=
 =?us-ascii?Q?Mb3XBFeu5Ri3TnMkSVdXXuH1gVq5yh4uqUdF93A2?=
X-MS-Exchange-CrossTenant-Network-Message-Id: ef9795df-ea59-4412-6a1f-08db24157ea1
X-MS-Exchange-CrossTenant-AuthSource: SA1PR11MB6733.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Mar 2023 22:51:34.2365
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Pyl5WeWUhXBZ2xS5TTt3mP8/CgG/4yR9byGeR64pWGBSD7XYFohZCfNtvbIMeT7b6qveSuBwgh6HEUV6+UAd+g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR11MB4745
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

alison.schofield@ wrote:
> From: Alison Schofield <alison.schofield@intel.com>
> 
> User space may need to know which region, if any, maps the poison
> address(es) logged in a cxl_poison trace event. Since the mapping
> of DPAs (device physical addresses) to a region can change, the
> kernel must provide this information at the time the poison list
> is read. The event informs user space that at event <timestamp>
> this <region> mapped to this <DPA>, which is poisoned.
> 
> The cxl_poison trace event is already wired up to log the region
> name and uuid if it receives param 'struct cxl_region'.
> 
> In order to provide that cxl_region, add another method for gathering
> poison - by committed endpoint decoder mappings. This method is only
> available with CONFIG_CXL_REGION and is only used if a region actually
> maps the memdev where poison is being read. After the poison list is
> read for all the mapped resources, poison is read for the unmapped
> resources, and those events are logged without the region info.
> 
> Mixed mode decoders are not currently supported in Linux. Add a debug
> message to the poison request path. That will serve as an alert that
> poison list retrieval needs to add support for mixed mode.
> 
> The default method remains: read the poison by memdev resource.
> 
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>

[...]

> diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c
> index ea996057815e..5e65818d2171 100644
> --- a/drivers/cxl/core/memdev.c
> +++ b/drivers/cxl/core/memdev.c
> @@ -139,14 +139,29 @@ static ssize_t trigger_poison_list_store(struct device *dev,
>  					 const char *buf, size_t len)
>  {
>  	struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
> +	struct cxl_port *port;
>  	bool trigger;
>  	int rc;
>  
>  	if (kstrtobool(buf, &trigger) || !trigger)
>  		return -EINVAL;
>  
> +	port = dev_get_drvdata(&cxlmd->dev);
> +	if (!port || !is_cxl_endpoint(port))
> +		return -EINVAL;
> +
>  	down_read(&cxl_dpa_rwsem);
> -	rc = cxl_get_poison_by_memdev(cxlmd);
> +	if (port->commit_end == -1) {
> +		/* No regions mapped to this memdev */
> +		rc = cxl_get_poison_by_memdev(cxlmd);
> +	} else {
> +		/* Regions mapped, collect poison by endpoint */
> +		rc = device_for_each_child(&port->dev, port,
> +					   cxl_get_poison_by_endpoint);
> +		if (rc == 1)
> +			rc = 0;
> +	}
> +
>  	up_read(&cxl_dpa_rwsem);
>  
>  	return rc ? rc : len;
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index f29028148806..1a558adfe32d 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -2213,6 +2213,102 @@ struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev)
>  }
>  EXPORT_SYMBOL_NS_GPL(to_cxl_pmem_region, CXL);
>  
> +int cxl_get_poison_by_endpoint(struct device *dev, void *data)
> +{
> +	struct cxl_endpoint_decoder *cxled;
> +	struct cxl_port *port = data;
> +	struct cxl_dev_state *cxlds;
> +	struct cxl_memdev *cxlmd;
> +	u64 offset, length;
> +	int rc = 0;
> +
> +	down_read(&cxl_dpa_rwsem);

Oh just realized aren't we already under the read lock when this is called?

Replacing this with a lockdep_assert() could make the logic here easier.

Ira

[...]