Received-SPF: pass (google.com: domain of linux-kernel+bounces-17115-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Date: Thu, 4 Jan 2024 10:59:16 -0800
From: Dan Williams <dan.j.williams@intel.com>
To: Bjorn Helgaas <helgaas@kernel.org>, Dan Williams
	<dan.j.williams@intel.com>
CC: Bjorn Helgaas <bhelgaas@google.com>, Ira Weiny <ira.weiny@intel.com>,
	Jonathan Cameron <jonathan.cameron@huawei.com>, Smita Koralahalli
	<Smita.KoralahalliChannabasappa@amd.com>, Shiju Jose <shiju.jose@huawei.com>,
	Yazen Ghannam <yazen.ghannam@amd.com>, Davidlohr Bueso <dave@stgolabs.net>,
	Dave Jiang <dave.jiang@intel.com>, Alison Schofield
	<alison.schofield@intel.com>, Vishal Verma <vishal.l.verma@intel.com>, "Ard
 Biesheuvel" <ardb@kernel.org>, <linux-efi@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <linux-cxl@vger.kernel.org>,
	<linux-pci@vger.kernel.org>
Subject: Re: [PATCH v5 8/9] PCI: Define scoped based management functions
Message-ID: <65970003ee9ef_8dc6829412@dwillia2-xfh.jf.intel.com.notmuch>
References: <6595f9eec5986_be70229443@dwillia2-xfh.jf.intel.com.notmuch>
 <20240104183218.GA1820872@bhelgaas>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240104183218.GA1820872@bhelgaas>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?qyw/CCrSROKclACNcSddqsY0sQnSQ18QfxNvW1A/t5jXWZW50W0r0EePX50D?=
 =?us-ascii?Q?He6mDVvPlALe8nm6xcfpoklqo3J6kKZpeadXPwFsFfXQc4DzCcBNbRdGuIEn?=
 =?us-ascii?Q?k5b+YmdpwRzk/oUZGW0MaSgJfcxExqJOdf5yS+YkE6/pA4JOFz3OWCduWwAg?=
 =?us-ascii?Q?6z34M/zrA6BY9iG5RrHEmp1l/UofT6/KdXWlEwb/r9tSv7cNcsmRRZkYfWMl?=
 =?us-ascii?Q?WmQRwCVUbENebXsd1h+hOjw6m0w0KWuoQjGCyhHfOzceFIOotagP2lTaFtVS?=
 =?us-ascii?Q?/fotBs5n0I0554ikl/O6hilDnue4ounor7gVhT8lWxefTPF42vjjM3Exzy/W?=
 =?us-ascii?Q?ljrbYk1eKJjul1NQW3jVV2Zm1aEV5VpegOKphX8ocuq9TmaA4RYCSDVBBI4G?=
 =?us-ascii?Q?jeFkmpm//3AU/ye2flZ5G4X95gGopSFayhr3FrqEL7ej9Q/MKH/yY7UiJeXp?=
 =?us-ascii?Q?7UZh9yIErtFra9Oixu+h1UgH+xH5fuhTLd+xno1xxGDD3mdFuFo+PH3oWtuv?=
 =?us-ascii?Q?QAb5567vx2EOUaKsAfgU/SoP1MPq8awdKrujqeOAFA5k7iR1kawCMtgtCk58?=
 =?us-ascii?Q?3sGSKTp/CZTve1Lgsbq0X8yHw7EuBgzYgnGISS3mvh3I7UJS7DmSA36Ld1n9?=
 =?us-ascii?Q?isIqvQ8S64YaPePWzPzpxB9f+4UhcbvSlS7qYySFjKAT3cG/NwKTlsQs22AH?=
 =?us-ascii?Q?UKf+iaqFjL8qdDsMnmc9Mq6XQVtTpmYVsNh4DGtsWUNsQo1heRG9GM+HtR8D?=
 =?us-ascii?Q?eu+lwfK1RFgzquNScKAoHZu+6qhYhNFWj5LOLAEi5RrdkCw+RrJNFiL6Idgs?=
 =?us-ascii?Q?aVy6xJRba73B9mD0iSbiE7klJ9p2TA5GtiCsjHHmDmB+zblRCRY1+cWeat6V?=
 =?us-ascii?Q?Bnn47yi9RWElP8s1DmiPJy0x7I+KTc1oLqrmApz8tOL0zO+sM5ahKQLZ6Dkk?=
 =?us-ascii?Q?lmZm9O/QQzW99664zu9JM/FzcC6MTKDMbzPBExL06SrV4Vp9tDbfrNepwibP?=
 =?us-ascii?Q?KjiRhJhrGt4wSoTohiKitHQVn0ZSLNv1k+p+AsFahqYSPK8E2ip0dJ5LWKxq?=
 =?us-ascii?Q?PV2avvwQGuNvw2MeOCrNpnGLA6ziabiUovPyE1Hb8tvDx3087hlrwv2LG0Vd?=
 =?us-ascii?Q?ryshYxv/hhuHi6ImjJ7kJohF9LuHrA0BAHSaw9rvHICAUI92ZoX6vTqm8JSe?=
 =?us-ascii?Q?QKh+jIYq6V8j3/MSSm/+wGXhA4+Zv5NOqeqDgOGems5v90K0DpyPV4P4j86Q?=
 =?us-ascii?Q?6wkcaeEE5J+p+1WuBBDkVJ6j2tuVcY/JaDbzIQwgbFj1acOK3XyxnMgjA5k9?=
 =?us-ascii?Q?QsOQ1dzGrm1lEJhsmyvi9+fhAiwsIjb5RXPiytM7w391vRPuXBLROAgiLcSu?=
 =?us-ascii?Q?SP4tRj3eCKBFVeYDWDHCb7WWueDLShRBEWotcqG5nm6Bbhe1sqqpQFGU2mqB?=
 =?us-ascii?Q?U4L+mOIlMGM9TStyKxAkpHnmdITyuPUoFyXrEAtQWwsjON98cMIoZ6KCHPPu?=
 =?us-ascii?Q?doPdBwpyy4EsMjpGtl69FiShfsdCB8VPJck+rdv5s1lcORuqEnqi1iajnAYC?=
 =?us-ascii?Q?WGEwyBXgrRPYV+9FBI2eaAc7eKsjUhpYYgOoBNtWowNWjg/FD18hT59fxTDN?=
 =?us-ascii?Q?0Q=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 9e1ec9bd-0592-41e4-e17a-08dc0d57417a
X-MS-Exchange-CrossTenant-AuthSource: PH8PR11MB8107.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jan 2024 18:59:19.3214
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RU90qYfWcT36or7Wo6n0Tqo6nwEQcKVIf/IfGgyHS+fAw4Cub3PYTVo3WDXsBlZlr0VeleBB10p4QnROlMvRUAB4UEBUL5HogLJGsIpPvHs=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR11MB6536
X-OriginatorOrg: intel.com

Bjorn Helgaas wrote:
[..]
> > ---
> > PCI: Introduce cleanup helpers for device reference counts and locks
> > 
> > The "goto error" pattern is notorious for introducing subtle resource
> > leaks. Use the new cleanup.h helpers for PCI device reference counts and
> > locks.
> > 
> > Similar to the new put_device() and device_lock() cleanup helpers,
> > __free(put_device) and guard(device), define the same for PCI devices,
> > __free(pci_dev_put) and guard(pci_dev).  These helpers eliminate the
> > need for "goto free;" and "goto unlock;" patterns. For example, A
> > 'struct pci_dev *' instance declared as:
> > 
> > 	struct pci_dev *pdev __free(pci_dev_put) = NULL;
> 
> I see several similar __free() uses with NULL initializations in gpio,
> but I think this idiom would be slightly improved if the __free()
> function were more closely associated with the actual pci_dev_get():
> 
>   struct pci_dev *pdev __free(pci_dev_put) = pci_get_device(...);
> 
> Not always possible, I know, but easier to analyze when it is.

I tend to agree, but it does lead to some long lines, for example:

diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c
index 4fd1f207c84e..549ba4b8294e 100644
--- a/drivers/cxl/pci.c
+++ b/drivers/cxl/pci.c
@@ -975,15 +975,14 @@ static void cxl_cper_event_call(enum cxl_event_type ev_type,
 				struct cxl_cper_event_rec *rec)
 {
 	struct cper_cxl_event_devid *device_id = &rec->hdr.device_id;
-	struct pci_dev *pdev __free(pci_dev_put) = NULL;
 	enum cxl_event_log_type log_type;
 	struct cxl_dev_state *cxlds;
 	unsigned int devfn;
 	u32 hdr_flags;
 
 	devfn = PCI_DEVFN(device_id->device_num, device_id->func_num);
-	pdev = pci_get_domain_bus_and_slot(device_id->segment_num,
-					   device_id->bus_num, devfn);
+	struct pci_dev *pdev __free(pci_dev_put) = pci_get_domain_bus_and_slot(
+		device_id->segment_num, device_id->bus_num, devfn);
 	if (!pdev)
 		return;
 
...so I think people are choosing the "... __free(x) = NULL;" style for
code density readability.

> 
> > ...will automatically call pci_dev_put() if @pdev is non-NULL when @pdev
> > goes out of scope (automatic variable scope). If a function wants to
> > invoke pci_dev_put() on error, but return @pdev on success, it can do:
> > 
> > 	return no_free_ptr(pdev);
> > 
> > ...or:
> > 
> > 	return_ptr(pdev);
> > 
> > For potential cleanup opportunity there are 587 open-coded calls to
> > pci_dev_put() in the kernel with 65 instances within 10 lines of a goto
> > statement with the CXL driver threatening to add another one.
> > 
> > The guard() helper holds the associated lock for the remainder of the
> > current scope in which it was invoked. So, for example:
> > 
> > 	func(...)
> > 	{
> > 		if (...) {
> > 			...
> > 			guard(pci_dev); /* pci_dev_lock() invoked here */
> > 			...
> > 		} /* <- implied pci_dev_unlock() triggered here */
> > 	}
> 
> Thanks for this!  I had skimmed cleanup.h previously, but it makes a
> lot more sense after your description here.  
> 
> I think a little introduction along these lines would be even more
> useful in cleanup.h since the concept is general and not PCI-specific.

Ok, let me ponder an update here.

> E.g., the motivation (avoid resource leaks with "goto error" pattern),
> a definition of "__free() based cleanup function" (IIUC, a function to
> be run when a variable goes out of scope), maybe something about
> ordering (it's important in the "goto error" pattern that the cleanups
> are done in a specific order; how does that translate to __free()?)

The __free() callbacks are invoked in reverse declaration (FILO) order.
However, as I say that another reviewer recommendation falls out. Be
careful about the variable declaration order diverging from the init
order. 

I.e. save the reader from needing to wonder if there are intra variable
init order dependencies by making it clear that init order ==
declaration order.

> 
> But the commit log above is fine with me.  (It does contain tabs,
> which get slightly mangled when "git log" indents it.)
> 
> > There are 15 invocations of pci_dev_unlock() in the kernel with 5
> > instances within 10 lines of a goto statement. Again, the CXL driver is
> > threatening to add another.
> > 
> > Introduce these helpers to preclude the addition of new more error prone
> > goto put; / goto unlock; sequences. For now, these helpers are used in
> > drivers/cxl/pci.c to allow ACPI error reports to be fed back into the
> > CXL driver associated with the PCI device identified in the report.
> 
> This part is also fine but doesn't seem strictly necessary to me.  I
> think the part about error reports being fed back needs a lot more
> background to understand the connection, and probably only makes sense
> in the context of that patch.

Sure I can trim that out and just say that the CXL driver is one such
occasion where a new goto for pci_dev_put() and pci_dev_unlock() was
about to be introduced.

> > As for reviewing conversions that use these new helpers, one of the
> > gotchas I have found is that it is easy to make a mistake if a goto
> > still exists in the function after the conversion. So unless and until
> > all of the resources a function acquires, that currently need a goto to
> > unwind them on error, can be converted to cleanup.h based helpers it is
> > best not to mix styles.
> > 
> > I think the function documentation in include/linux/cleanup.h does a
> > decent job of explaining how to use the helpers. However, I am happy to
> > suggest some updates if you think it would help.
> 
> Thanks, Dan!
> 
> Acked-by: Bjorn Helgaas <bhelgaas@google.com>

Thanks, Bjorn!