Received-SPF: pass (google.com: domain of linux-kernel+bounces-133572-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Fri, 5 Apr 2024 13:56:08 -0700
From: Ira Weiny <ira.weiny@intel.com>
To: Jonathan Cameron <Jonathan.Cameron@huawei.com>, <ira.weiny@intel.com>
CC: Dave Jiang <dave.jiang@intel.com>, Fan Ni <fan.ni@samsung.com>, "Navneet
 Singh" <navneet.singh@intel.com>, Dan Williams <dan.j.williams@intel.com>,
	Davidlohr Bueso <dave@stgolabs.net>, Alison Schofield
	<alison.schofield@intel.com>, Vishal Verma <vishal.l.verma@intel.com>,
	<linux-btrfs@vger.kernel.org>, <linux-cxl@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 06/26] cxl/port: Add Dynamic Capacity mode support to
 endpoint decoders
Message-ID: <661065683552f_e9f9f2946@iweiny-mobl.notmuch>
References: <20240324-dcd-type2-upstream-v1-0-b7b00d623625@intel.com>
 <20240324-dcd-type2-upstream-v1-6-b7b00d623625@intel.com>
 <20240404093201.00004f33@Huawei.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240404093201.00004f33@Huawei.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?4CsX6b7SOwBRoFfw9N+I10Vx+GzJCyXKA6wiJupHbc3/UrfUN3MFDsVzLQpY?=
 =?us-ascii?Q?N4my7e+dIzCMV/4m9e4DToLICQKOZm0Demenu510lTWcEainOrG+iPg9VOtM?=
 =?us-ascii?Q?LPXEms/m/EEo8T2dL8yd50yAkU0DEIuHPm5vKKDeWqPplLBENY+hL276xCgI?=
 =?us-ascii?Q?1ayA8z42dzIbn4hRREbctz2UcpaW7COmsO6gbZwJhf+3KV5cUAGjDdgwZtYs?=
 =?us-ascii?Q?CLbJoz+dgHGpgNd9k2faELR9/ic2PWZqn9VsyoCSvYRyfmzUUn7y1FHtOBg/?=
 =?us-ascii?Q?8pcPhhsKkAkDD/08h5YJeH2/HKxwy+vLt5wpVTp2Mb9UotrSTdtfIK6Ysh9/?=
 =?us-ascii?Q?1Jc7M/cWYlTCvOXRjEz7JuxEz7ZOMS+NejRCA2T7JIG9L4NL4FYuWt6A6iCT?=
 =?us-ascii?Q?xm65Cq57g53B5xRp1VVHCvjTWgfhiLZM3jdDDSrHdNziy9K59NtMAYgLjqPf?=
 =?us-ascii?Q?ZSXSdfj45G+Dg9sBNJKaSNY0eyjw4QRLJl26Gym/BghFXLdXHbDNoJ1uvUDC?=
 =?us-ascii?Q?obPREtk89Jfltvs49Lm1utIswUVxrTu5YCHTw9a1LuemYSwAcV5SCuXmPE6u?=
 =?us-ascii?Q?KYu3S7g1tI4IeWklZw4zrHQ/50B0IpAROQI2ebZ8DMI50o90RE6CmGPjp7Gq?=
 =?us-ascii?Q?k/TuBnz0MYnvyVo4AQ5fOnFTztMiUDQhxRKVihZ/lgSNWNSxYxbeCOXBLM42?=
 =?us-ascii?Q?jotoMW6W/TX0EFdAXuGaHCrJiXWOO3sNahyxLG2TQratxB/rkmb/6d62+U5J?=
 =?us-ascii?Q?fJ397pcie46yIe85KEFNLhfUVo8xgywbqr66bo89EzQkpR2kUTko+nJNKSBI?=
 =?us-ascii?Q?nVGHXn200xwZ7Jdpec0WOmdDrjkuJtLEFIWbGtBMNN1V8o3oeC/6xbWNtFAp?=
 =?us-ascii?Q?gwK0h2+oM91ZF3b6ca/NKY+oNsHw5tVijz3WwPN+zD68uo7UOL9QB0j2HWxd?=
 =?us-ascii?Q?bjgHbRokL/2slBNnxJVbmiItQGyAJBmmjbwYE2JZF5ClFvMcp47I3sqvoOS9?=
 =?us-ascii?Q?TtERTpE+O4smXjSSq5j3kWAXmQXkYd4ymknpJUaTwGjTMKxTKt0tRMez1cWW?=
 =?us-ascii?Q?yaEW/K89ZXQyGq5MstZsUZnWlfjPFzoHHaog7/0k5PkXFxQC4x1DL6drDLJm?=
 =?us-ascii?Q?63vxz9Gd1hyTHXrw2F24DClqdf0523tCjCCBAHIsKuN3HiqXCBlDafvq6T5u?=
 =?us-ascii?Q?NlFSEUlC6yah/hsL6pfoiVLz2+JiLRc+GlTKrqhbgNJTEWByKV/+WgzHozbk?=
 =?us-ascii?Q?IKhZdbRHb7k360gMyfzSBLGiqHlOKNe5dha1roslClbZgX4+tXhxEAXJnnlq?=
 =?us-ascii?Q?STbSona0wApUZwBU/QWM1lXvNInfyDGng9uMsjF8rgVosmx6tl+4PRJyqzuX?=
 =?us-ascii?Q?imdcZAaW1wXbXVvEGzY6FW+GgUA0pieT1OLSmwskTJTEZ61TUUSofDNgFrGb?=
 =?us-ascii?Q?tK7vFaHvSWzGLM/BqayqudrxdvQUE6xACPqNEhYW2uv7s97rW6ivhFHFfjv1?=
 =?us-ascii?Q?eXODLmlLUhL8Q2cl00ytzRgwzUxgwLm4ZXog1NscMuVJDCRnYjnNpcXkaPts?=
 =?us-ascii?Q?2bvJ1CswyxdjeD/C+BNdPAFwEfE6je6ykl0r2EZn?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7e65bebe-e840-4a46-ccb8-08dc55b2d316
X-MS-Exchange-CrossTenant-AuthSource: SA1PR11MB6733.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Apr 2024 20:56:12.1566
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0SXlIBISxlYC61oAKSX02EwhqVEgbIRAtbmCkSFAxDU54nSiIEn0jtc9MAjCO9p76GWWaxBSQDQGQg0lk3WYeQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR11MB6128
X-OriginatorOrg: intel.com

Jonathan Cameron wrote:
> On Sun, 24 Mar 2024 16:18:09 -0700
> ira.weiny@intel.com wrote:
> 
> > From: Navneet Singh <navneet.singh@intel.com>
> > 

[snip]

> > diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl
> > index fff2581b8033..8b3efaf6563c 100644
> > --- a/Documentation/ABI/testing/sysfs-bus-cxl
> > +++ b/Documentation/ABI/testing/sysfs-bus-cxl
> > @@ -316,23 +316,24 @@ Description:
> >  
> >  
> >  What:		/sys/bus/cxl/devices/decoderX.Y/mode
> > -Date:		May, 2022
> > -KernelVersion:	v6.0
> > +Date:		May, 2022, June 2024
> > +KernelVersion:	v6.0, v6.10 (dcY)
> >  Contact:	linux-cxl@vger.kernel.org
> >  Description:
> >  		(RW) When a CXL decoder is of devtype "cxl_decoder_endpoint" it
> >  		translates from a host physical address range, to a device local
> >  		address range. Device-local address ranges are further split
> > -		into a 'ram' (volatile memory) range and 'pmem' (persistent
> > -		memory) range. The 'mode' attribute emits one of 'ram', 'pmem',
> > -		'mixed', or 'none'. The 'mixed' indication is for error cases
> > -		when a decoder straddles the volatile/persistent partition
> > -		boundary, and 'none' indicates the decoder is not actively
> > -		decoding, or no DPA allocation policy has been set.
> > +		into a 'ram' (volatile memory) range, 'pmem' (persistent
> > +		memory) range, or Dynamic Capacity (DC) range. The 'mode'
> > +		attribute emits one of 'ram', 'pmem', 'dcY', 'mixed', or
> > +		'none'. The 'mixed' indication is for error cases when a
> > +		decoder straddles the volatile/persistent partition boundary,
> 
> I love corners.  What happen if no persistent and decoder straddles
> volatile / dc0?  Would only happen if the bios was having fun I think..

Yep same issue with a wonky bios...  I did not change this text.  Only got reformatted.  But it is worth changing to:


.. The 'mixed' indication is for error cases when a decoder
    straddles partition boundaries, ...

> 
> > +		and 'none' indicates the decoder is not actively decoding, or
> > +		no DPA allocation policy has been set.
> >  
> >  		'mode' can be written, when the decoder is in the 'disabled'
> > -		state, with either 'ram' or 'pmem' to set the boundaries for the
> > -		next allocation.
> > +		state, with 'ram', 'pmem', or 'dcY' to set the boundaries for
> > +		the next allocation.
> >  
> >  
> >  What:		/sys/bus/cxl/devices/decoderX.Y/dpa_resource
> > diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
> > index 66b8419fd0c3..e22b6f4f7145 100644
> > --- a/drivers/cxl/core/hdm.c
> > +++ b/drivers/cxl/core/hdm.c
> > @@ -255,6 +255,14 @@ static void devm_cxl_dpa_release(struct cxl_endpoint_decoder *cxled)
> >  	__cxl_dpa_release(cxled);
> >  }
> >  
> > +static int dc_mode_to_region_index(enum cxl_decoder_mode mode)
> > +{
> > +	if (mode < CXL_DECODER_DC0 || CXL_DECODER_DC7 < mode)
> > +		return -EINVAL;
> > +
> > +	return mode - CXL_DECODER_DC0;
> > +}
> > +
> >  static int __cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled,
> >  			     resource_size_t base, resource_size_t len,
> >  			     resource_size_t skipped)
> > @@ -411,6 +419,7 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
> >  	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> >  	struct cxl_dev_state *cxlds = cxlmd->cxlds;
> >  	struct device *dev = &cxled->cxld.dev;
> > +	int rc;
> >  
> >  	guard(rwsem_write)(&cxl_dpa_rwsem);
> >  	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE)
> > @@ -433,6 +442,16 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled,
> >  			return -ENXIO;
> >  		}
> >  		break;
> > +	case CXL_DECODER_DC0 ... CXL_DECODER_DC7:
> > +		rc = dc_mode_to_region_index(mode);
> > +		if (rc < 0)
> > +			return rc;
> 
> Can't fail, so you could not bother checking..  Seems very unlikely
> that function will gain other error cases in the future.

Sure, done.

> 
> > +
> > +		if (resource_size(&cxlds->dc_res[rc]) == 0) {
> > +			dev_dbg(dev, "no available dynamic capacity\n");
> > +			return -ENXIO;
> > +		}
> > +		break;
> >  	default:
> >  		dev_dbg(dev, "unsupported mode: %d\n", mode);
> >  		return -EINVAL;
> > diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
> > index e59d9d37aa65..80c0651794eb 100644
> > --- a/drivers/cxl/core/port.c
> > +++ b/drivers/cxl/core/port.c
> > @@ -208,6 +208,22 @@ static ssize_t mode_store(struct device *dev, struct device_attribute *attr,
> >  		mode = CXL_DECODER_PMEM;
> >  	else if (sysfs_streq(buf, "ram"))
> >  		mode = CXL_DECODER_RAM;
> > +	else if (sysfs_streq(buf, "dc0"))
> > +		mode = CXL_DECODER_DC0;
> > +	else if (sysfs_streq(buf, "dc1"))
> > +		mode = CXL_DECODER_DC1;
> > +	else if (sysfs_streq(buf, "dc2"))
> > +		mode = CXL_DECODER_DC2;
> > +	else if (sysfs_streq(buf, "dc3"))
> > +		mode = CXL_DECODER_DC3;
> > +	else if (sysfs_streq(buf, "dc4"))
> > +		mode = CXL_DECODER_DC4;
> > +	else if (sysfs_streq(buf, "dc5"))
> > +		mode = CXL_DECODER_DC5;
> > +	else if (sysfs_streq(buf, "dc6"))
> > +		mode = CXL_DECODER_DC6;
> > +	else if (sysfs_streq(buf, "dc7"))
> > +		mode = CXL_DECODER_DC7;
> 
> Fully agree with the comment that a string + enum table and search
> is probably appropriate here.
> 

Ok, done.

Ira