Received-SPF: pass (google.com: domain of linux-kernel+bounces-16140-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Date: Wed, 3 Jan 2024 16:21:02 -0800
From: Dan Williams <dan.j.williams@intel.com>
To: Bjorn Helgaas <helgaas@kernel.org>, Dan Williams
	<dan.j.williams@intel.com>
CC: Bjorn Helgaas <bhelgaas@google.com>, Ira Weiny <ira.weiny@intel.com>,
	Jonathan Cameron <jonathan.cameron@huawei.com>, Smita Koralahalli
	<Smita.KoralahalliChannabasappa@amd.com>, Shiju Jose <shiju.jose@huawei.com>,
	Yazen Ghannam <yazen.ghannam@amd.com>, Davidlohr Bueso <dave@stgolabs.net>,
	Dave Jiang <dave.jiang@intel.com>, Alison Schofield
	<alison.schofield@intel.com>, Vishal Verma <vishal.l.verma@intel.com>, "Ard
 Biesheuvel" <ardb@kernel.org>, <linux-efi@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <linux-cxl@vger.kernel.org>,
	<linux-pci@vger.kernel.org>
Subject: Re: [PATCH v5 8/9] PCI: Define scoped based management functions
Message-ID: <6595f9eec5986_be70229443@dwillia2-xfh.jf.intel.com.notmuch>
References: <6595e201beb4_be7022944d@dwillia2-xfh.jf.intel.com.notmuch>
 <20240103230147.GA1800245@bhelgaas>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240103230147.GA1800245@bhelgaas>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?O8aburBUOLzBiA+FO5aVrBguM3WSpdwm0HXEDA9dvuCa70Ld1tW1HlB7My+i?=
 =?us-ascii?Q?Qny/5I162jdOeT2K/S1dmkqDKzJelPKOV/QxPpmMvy1+7ELXnsP45wph3yeT?=
 =?us-ascii?Q?jt+Y+ahGggtdYHu5xbxMlxNbWT0yQumj/Z8DH0fsFYh/WLtBUkuNCKckMyjB?=
 =?us-ascii?Q?wGjEn7AyYvNytzJKsXOQ3Bp8gdFLd27+7sTR5/eQSzyDtcRj264iwEYbbHrs?=
 =?us-ascii?Q?DjO46NAO457dKyPwiF9mq6dYxbwlRYfFVr52+63Skb8O/IzDz1LjDQ9GxTcF?=
 =?us-ascii?Q?U8CFA+3yNe1rmvmJFEI8jUqfJ/AbBi6HwvsCkfW050YOKT+PcslyemQNsz3p?=
 =?us-ascii?Q?pDpZkXAx+JFyueRd3bpyZkBDeP45/1xXUnUu0vKplMPTF2cKOEOouzu9Prbs?=
 =?us-ascii?Q?jc6oTCGpEZcbmzXhHQLF3D5vgWU3zXv/O3Mg1qsGybFGBZSZvOf5WApuktd8?=
 =?us-ascii?Q?6bbtCRGG57W0lTf6j+p6q7p2Z4UgyPih5VTYO6GAgpfwJtEbASHAxdLv7VTu?=
 =?us-ascii?Q?1QuiVaAYEheI4eTmlx9igKWEfSdVVQ5KpaQipvTi6gsCBtJZlmMUwQL9whfa?=
 =?us-ascii?Q?NqpqSc6iQjx2Vjvc72GUNk/EQiuNc9FVGA++mmbw/x7iFI7hN1Uw7LSj5Ke6?=
 =?us-ascii?Q?joyqezYklMDIUJTrfT6O90Jri37OpnCcLcFXFAapkeC6g3OP9KGytpk+hccI?=
 =?us-ascii?Q?vTm3l4lCB9k7TCJUVrtwqdkudBUBsymTD9Fg0Y9wH9srBxUJoCN2Xvu3Qh7r?=
 =?us-ascii?Q?/9F5x1qM16YeuTJrsYBANJp8csgLfbeFMo6JHyw0FaXgHdoSS3iLhlowy0tY?=
 =?us-ascii?Q?nLoVE2yK7oho39Z3BXVDl8xfj53xslLlPLMTwjc0UbVSUiLIVZJSkRZdgtb3?=
 =?us-ascii?Q?w/zG/ovAUcBbdNk8lbdImyRL+w7UlkBh9vml8CM9KeCctNG499KeXcJLLdD3?=
 =?us-ascii?Q?WTaavt1uvmz2wmfXyZJHABj9NcmyEjzgf6CGFRqB0MMGSE+pURpwxKeLxd7+?=
 =?us-ascii?Q?8MZ/9FDFZRgEymRHLB8YIhZAfl3GjfknXF2KXNo4E1dNfSnq47b1YZriwVmQ?=
 =?us-ascii?Q?GciEBzXPyL/WvAyzr97UF+Vaz8Pqbz5CMJygtyFRkZvs4qyELo0+biiSSkmM?=
 =?us-ascii?Q?somA/YBnx5Sbwy3UMWPzQfbS72n/M9fo5AEAhuUsS4GqIFJUkpIvFXC+Zu67?=
 =?us-ascii?Q?9tbIZvs5usS6VNmoy41nvjKKuk/CRt412LTo2YW40jH/1ge+E30xIUdemJKm?=
 =?us-ascii?Q?UW1yc5UTrbjKmr/IE6bK/OcO6khOjZZGMkhO8nYYZI4mchgrW2/MifXOTegr?=
 =?us-ascii?Q?GtS4/ddRMBPIp4EAZQL8V4f09jnZSFTT/NZK6oQarWs/W7L3jkPBbycMfexs?=
 =?us-ascii?Q?4TJFyELO9OZyoBvX+RYwuCB8vRXw+drfAoBoZxvt+2fPomxnEhI+eFxm5oKJ?=
 =?us-ascii?Q?F0lJf7O/HDVcRbt9CsUSA3JjYCSEfzb0nQsiR3VAexZzXMAvxJxgRGwXHoVA?=
 =?us-ascii?Q?JzzpSthw1fi5JbMgV1rKlyhSyJjR4FZRCZWlgv9Gt6FOILHXcacwJEaxifDb?=
 =?us-ascii?Q?MKMQK1r+0sqg8GzC6SE/mrl13hSNmlxwf8h9nV6+1WD+eE2rW6PkOVaJIZNb?=
 =?us-ascii?Q?Jg=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d70e80a6-57df-4324-08c6-08dc0cbb0b6c
X-MS-Exchange-CrossTenant-AuthSource: PH8PR11MB8107.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jan 2024 00:21:07.1117
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4l1DgrOjlWk+A4DHobHnzNgfXV+GNFToGpcPF7bBbN0YY8zboNzZyhDFC3Pdv9W5xKSzmJ/XKqlGd5xgV/X/mgyiDy5h5ubsvG9HmcuiL0w=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR11MB5436
X-OriginatorOrg: intel.com

Bjorn Helgaas wrote:
> On Wed, Jan 03, 2024 at 02:38:57PM -0800, Dan Williams wrote:
> > Ira Weiny wrote:
> > > Users of pci_dev_get() can benefit from a scoped based put.  Also,
> > > locking a PCI device is often done within a single scope.
> > > 
> > > Define a pci_dev_put() free function and a PCI device lock guard.  These
> > > will initially be used in new CXL event processing code but is defined
> > > in a separate patch for others to pickup and use/backport easier.
> > 
> > Any heartburn if I take this through cxl.git with the rest in this
> > series? Patch 9 has a dependency on this one.
> 
> No real heartburn.  I was trying to figure out what this does
> since I'm not familiar with the "scoped based put" idea and
> 'git grep -i "scope.*base"' wasn't much help.
> 
> I would kind of like the commit log to say a little more about what
> the "scoped based put" (does that have too many past tenses in it?)
> means and how users of pci_dev_get() will benefit.

That is completely fair, and I should have checked to make sure that the
changelog clarified the impact of the change.

> I don't know what "locking a PCI device is often done within a single
> scope" is trying to tell me either.  What if it's *not* done within a
> single scope?
> 
> Does this change anything for callers of pci_dev_get() and
> pci_dev_put()?
> 
> Does this avoid a common programming error?  I just don't know what
> the benefit of this is yet.
> 
> I'm sure this is really cool stuff, but there's little documentation,
> few existing users, and I don't know what I need to look for when
> reviewing things.

Here a is a re-write of the changelog:

---
PCI: Introduce cleanup helpers for device reference counts and locks

The "goto error" pattern is notorious for introducing subtle resource
leaks. Use the new cleanup.h helpers for PCI device reference counts and
locks.

Similar to the new put_device() and device_lock() cleanup helpers,
__free(put_device) and guard(device), define the same for PCI devices,
__free(pci_dev_put) and guard(pci_dev).  These helpers eliminate the
need for "goto free;" and "goto unlock;" patterns. For example, A
'struct pci_dev *' instance declared as:

	struct pci_dev *pdev __free(pci_dev_put) = NULL;

...will automatically call pci_dev_put() if @pdev is non-NULL when @pdev
goes out of scope (automatic variable scope). If a function wants to
invoke pci_dev_put() on error, but return @pdev on success, it can do:

	return no_free_ptr(pdev);

...or:

	return_ptr(pdev);

For potential cleanup opportunity there are 587 open-coded calls to
pci_dev_put() in the kernel with 65 instances within 10 lines of a goto
statement with the CXL driver threatening to add another one.

The guard() helper holds the associated lock for the remainder of the
current scope in which it was invoked. So, for example:

	func(...)
	{
		if (...) {
			...
			guard(pci_dev); /* pci_dev_lock() invoked here */
			...
		} /* <- implied pci_dev_unlock() triggered here */
	}

There are 15 invocations of pci_dev_unlock() in the kernel with 5
instances within 10 lines of a goto statement. Again, the CXL driver is
threatening to add another.

Introduce these helpers to preclude the addition of new more error prone
goto put; / goto unlock; sequences. For now, these helpers are used in
drivers/cxl/pci.c to allow ACPI error reports to be fed back into the
CXL driver associated with the PCI device identified in the report.

---

As for reviewing conversions that use these new helpers, one of the
gotchas I have found is that it is easy to make a mistake if a goto
still exists in the function after the conversion. So unless and until
all of the resources a function acquires, that currently need a goto to
unwind them on error, can be converted to cleanup.h based helpers it is
best not to mix styles.

I think the function documentation in include/linux/cleanup.h does a
decent job of explaining how to use the helpers. However, I am happy to
suggest some updates if you think it would help.