Received-SPF: pass (google.com: domain of linux-kernel+bounces-86613-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <9cc3d11bc3e1bb89a1c725f865d0c8d1494111c5.camel@gmail.com>
Subject: Re: [PATCH v3 1/2] driver core: Introduce device_link_wait_removal()
From: Nuno =?ISO-8859-1?Q?S=E1?= <noname.nuno@gmail.com>
To: Herve Codina <herve.codina@bootlin.com>, Greg Kroah-Hartman
	 <gregkh@linuxfoundation.org>, "Rafael J. Wysocki" <rafael@kernel.org>, Rob
	Herring <robh+dt@kernel.org>, Frank Rowand <frowand.list@gmail.com>
Cc: Lizhi Hou <lizhi.hou@amd.com>, Max Zhen <max.zhen@amd.com>, Sonal Santan
 <sonal.santan@amd.com>, Stefano Stabellini <stefano.stabellini@xilinx.com>,
  Jonathan Cameron <Jonathan.Cameron@Huawei.com>,
 linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, Allan Nielsen
 <allan.nielsen@microchip.com>, Horatiu Vultur
 <horatiu.vultur@microchip.com>, Steen Hegelund
 <steen.hegelund@microchip.com>, Luca Ceresoli <luca.ceresoli@bootlin.com>,
 Nuno Sa <nuno.sa@analog.com>, Thomas Petazzoni
 <thomas.petazzoni@bootlin.com>,  stable@vger.kernel.org
Date: Thu, 29 Feb 2024 12:16:44 +0100
In-Reply-To: <20240229105204.720717-2-herve.codina@bootlin.com>
References: <20240229105204.720717-1-herve.codina@bootlin.com>
	 <20240229105204.720717-2-herve.codina@bootlin.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.4 
Precedence: bulk
MIME-Version: 1.0

Hi,

Just copy pasting my previous comments :)

On Thu, 2024-02-29 at 11:52 +0100, Herve Codina wrote:
> The commit 80dd33cf72d1 ("drivers: base: Fix device link removal")
> introduces a workqueue to release the consumer and supplier devices used
> in the devlink.
> In the job queued, devices are release and in turn, when all the
> references to these devices are dropped, the release function of the
> device itself is called.
>=20
> Nothing is present to provide some synchronisation with this workqueue
> in order to ensure that all ongoing releasing operations are done and
> so, some other operations can be started safely.
>=20
> For instance, in the following sequence:
> =C2=A0 1) of_platform_depopulate()
> =C2=A0 2) of_overlay_remove()
>=20
> During the step 1, devices are released and related devlinks are removed
> (jobs pushed in the workqueue).
> During the step 2, OF nodes are destroyed but, without any
> synchronisation with devlink removal jobs, of_overlay_remove() can raise
> warnings related to missing of_node_put():
> =C2=A0 ERROR: memory leak, expected refcount 1 instead of 2
>=20
> Indeed, the missing of_node_put() call is going to be done, too late,
> from the workqueue job execution.
>=20
> Introduce device_link_wait_removal() to offer a way to synchronize
> operations waiting for the end of devlink removals (i.e. end of
> workqueue jobs).
> Also, as a flushing operation is done on the workqueue, the workqueue
> used is moved from a system-wide workqueue to a local one.
>=20
> Fixes: 80dd33cf72d1 ("drivers: base: Fix device link removal")
> Cc: stable@vger.kernel.org
> Signed-off-by: Herve Codina <herve.codina@bootlin.com>
> ---
> =C2=A0drivers/base/core.c=C2=A0=C2=A0=C2=A0 | 26 +++++++++++++++++++++++-=
--
> =C2=A0include/linux/device.h |=C2=A0 1 +
> =C2=A02 files changed, 24 insertions(+), 3 deletions(-)
>=20
> diff --git a/drivers/base/core.c b/drivers/base/core.c
> index d5f4e4aac09b..80d9430856a8 100644
> --- a/drivers/base/core.c
> +++ b/drivers/base/core.c
> @@ -44,6 +44,7 @@ static bool fw_devlink_is_permissive(void);
> =C2=A0static void __fw_devlink_link_to_consumers(struct device *dev);
> =C2=A0static bool fw_devlink_drv_reg_done;
> =C2=A0static bool fw_devlink_best_effort;
> +static struct workqueue_struct *device_link_wq;
> =C2=A0
> =C2=A0/**
> =C2=A0 * __fwnode_link_add - Create a link between two fwnode_handles.
> @@ -532,12 +533,26 @@ static void devlink_dev_release(struct device *dev)
> =C2=A0	/*
> =C2=A0	 * It may take a while to complete this work because of the SRCU
> =C2=A0	 * synchronization in device_link_release_fn() and if the consumer=
 or
> -	 * supplier devices get deleted when it runs, so put it into the
> "long"
> -	 * workqueue.
> +	 * supplier devices get deleted when it runs, so put it into the
> +	 * dedicated workqueue.
> =C2=A0	 */
> -	queue_work(system_long_wq, &link->rm_work);
> +	queue_work(device_link_wq, &link->rm_work);
> =C2=A0}
> =C2=A0
> +/**
> + * device_link_wait_removal - Wait for ongoing devlink removal jobs to
> terminate
> + */
> +void device_link_wait_removal(void)
> +{
> +	/*
> +	 * devlink removal jobs are queued in the dedicated work queue.
> +	 * To be sure that all removal jobs are terminated, ensure that any
> +	 * scheduled work has run to completion.
> +	 */
> +	drain_workqueue(device_link_wq);
> +}

I'm still not convinced we can have a recursive call into devlinks removal =
so I
do think flush_workqueue() is enough. I will defer to Saravana though...

> +EXPORT_SYMBOL_GPL(device_link_wait_removal);
> +
> =C2=A0static struct class devlink_class =3D {
> =C2=A0	.name =3D "devlink",
> =C2=A0	.dev_groups =3D devlink_groups,
> @@ -4099,9 +4114,14 @@ int __init devices_init(void)
> =C2=A0	sysfs_dev_char_kobj =3D kobject_create_and_add("char", dev_kobj);
> =C2=A0	if (!sysfs_dev_char_kobj)
> =C2=A0		goto char_kobj_err;
> +	device_link_wq =3D alloc_workqueue("device_link_wq", 0, 0);
> +	if (!device_link_wq)
> +		goto wq_err;
>=20

I still think this makes more sense in devlink_class_init() as this really
device link specific. Moreover, as I said to Saravana, we need to "convince=
"
Rafael about this as he (in my series) did not agreed with erroring out in =
case
we fail to allocate the queue.

Rafael?

- Nuno S=C3=A1