Received-SPF: pass (google.com: domain of linux-kernel+bounces-95535-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <c0f7ea40a2b1abc22242a892e162e4511a7c99f1.camel@gmail.com>
Subject: Re: [PATCH v5 1/2] driver core: Introduce device_link_wait_removal()
From: Nuno =?ISO-8859-1?Q?S=E1?= <noname.nuno@gmail.com>
To: Herve Codina <herve.codina@bootlin.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, "Rafael J. Wysocki"
 <rafael@kernel.org>, Rob Herring <robh+dt@kernel.org>, Frank Rowand
 <frowand.list@gmail.com>, Saravana Kannan <saravanak@google.com>, Lizhi Hou
 <lizhi.hou@amd.com>, Max Zhen <max.zhen@amd.com>, Sonal Santan
 <sonal.santan@amd.com>, Stefano Stabellini <stefano.stabellini@xilinx.com>,
  Jonathan Cameron <Jonathan.Cameron@Huawei.com>,
 linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, Allan Nielsen
 <allan.nielsen@microchip.com>, Horatiu Vultur
 <horatiu.vultur@microchip.com>, Steen Hegelund
 <steen.hegelund@microchip.com>, Luca Ceresoli <luca.ceresoli@bootlin.com>,
 Nuno Sa <nuno.sa@analog.com>, Thomas Petazzoni
 <thomas.petazzoni@bootlin.com>,  stable@vger.kernel.org
Date: Thu, 07 Mar 2024 13:29:19 +0100
In-Reply-To: <20240307131623.467e1def@bootlin.com>
References: <20240307111036.225007-1-herve.codina@bootlin.com>
	 <20240307111036.225007-2-herve.codina@bootlin.com>
	 <94997e8720bc0a68afa85be3ef521c8844d0f0a0.camel@gmail.com>
	 <20240307131623.467e1def@bootlin.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) 
Precedence: bulk
MIME-Version: 1.0

On Thu, 2024-03-07 at 13:16 +0100, Herve Codina wrote:
> Hi Nuno,
>=20
> On Thu, 07 Mar 2024 12:50:52 +0100
> Nuno S=C3=A1 <noname.nuno@gmail.com> wrote:
>=20
> > Hi Herve,
> >=20
> >=20
> > On Thu, 2024-03-07 at 12:10 +0100, Herve Codina wrote:
> > > The commit 80dd33cf72d1 ("drivers: base: Fix device link removal")
> > > introduces a workqueue to release the consumer and supplier devices u=
sed
> > > in the devlink.
> > > In the job queued, devices are release and in turn, when all the
> > > references to these devices are dropped, the release function of the
> > > device itself is called.
> > >=20
> > > Nothing is present to provide some synchronisation with this workqueu=
e
> > > in order to ensure that all ongoing releasing operations are done and
> > > so, some other operations can be started safely.
> > >=20
> > > For instance, in the following sequence:
> > > =C2=A0 1) of_platform_depopulate()
> > > =C2=A0 2) of_overlay_remove()
> > >=20
> > > During the step 1, devices are released and related devlinks are remo=
ved
> > > (jobs pushed in the workqueue).
> > > During the step 2, OF nodes are destroyed but, without any
> > > synchronisation with devlink removal jobs, of_overlay_remove() can ra=
ise
> > > warnings related to missing of_node_put():
> > > =C2=A0 ERROR: memory leak, expected refcount 1 instead of 2
> > >=20
> > > Indeed, the missing of_node_put() call is going to be done, too late,
> > > from the workqueue job execution.
> > >=20
> > > Introduce device_link_wait_removal() to offer a way to synchronize
> > > operations waiting for the end of devlink removals (i.e. end of
> > > workqueue jobs).
> > > Also, as a flushing operation is done on the workqueue, the workqueue
> > > used is moved from a system-wide workqueue to a local one.
> > >=20
> > > Cc: stable@vger.kernel.org
> > > Signed-off-by: Herve Codina <herve.codina@bootlin.com>
> > > Tested-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
> > > Reviewed-by: Nuno Sa <nuno.sa@analog.com>
> > > ---
> > > =C2=A0drivers/base/core.c=C2=A0=C2=A0=C2=A0 | 26 ++++++++++++++++++++=
+++---
> > > =C2=A0include/linux/device.h |=C2=A0 1 +
> > > =C2=A02 files changed, 24 insertions(+), 3 deletions(-)
> > >=20
> > > diff --git a/drivers/base/core.c b/drivers/base/core.c
> > > index d5f4e4aac09b..48b28c59c592 100644
> > > --- a/drivers/base/core.c
> > > +++ b/drivers/base/core.c
> > > @@ -44,6 +44,7 @@ static bool fw_devlink_is_permissive(void);
> > > =C2=A0static void __fw_devlink_link_to_consumers(struct device *dev);
> > > =C2=A0static bool fw_devlink_drv_reg_done;
> > > =C2=A0static bool fw_devlink_best_effort;
> > > +static struct workqueue_struct *device_link_wq;
> > > =C2=A0
> > > =C2=A0/**
> > > =C2=A0 * __fwnode_link_add - Create a link between two fwnode_handles=
.
> > > @@ -532,12 +533,26 @@ static void devlink_dev_release(struct device *=
dev)
> > > =C2=A0	/*
> > > =C2=A0	 * It may take a while to complete this work because of the SR=
CU
> > > =C2=A0	 * synchronization in device_link_release_fn() and if the cons=
umer or
> > > -	 * supplier devices get deleted when it runs, so put it into the
> > > "long"
> > > -	 * workqueue.
> > > +	 * supplier devices get deleted when it runs, so put it into the
> > > +	 * dedicated workqueue.
> > > =C2=A0	 */
> > > -	queue_work(system_long_wq, &link->rm_work);
> > > +	queue_work(device_link_wq, &link->rm_work);
> > > =C2=A0}
> > > =C2=A0
> > > +/**
> > > + * device_link_wait_removal - Wait for ongoing devlink removal jobs =
to
> > > terminate
> > > + */
> > > +void device_link_wait_removal(void)
> > > +{
> > > +	/*
> > > +	 * devlink removal jobs are queued in the dedicated work queue.
> > > +	 * To be sure that all removal jobs are terminated, ensure that any
> > > +	 * scheduled work has run to completion.
> > > +	 */
> > > +	flush_workqueue(device_link_wq);
> > > +}
> > > +EXPORT_SYMBOL_GPL(device_link_wait_removal);
> > > +
> > > =C2=A0static struct class devlink_class =3D {
> > > =C2=A0	.name =3D "devlink",
> > > =C2=A0	.dev_groups =3D devlink_groups,
> > > @@ -4099,9 +4114,14 @@ int __init devices_init(void)
> > > =C2=A0	sysfs_dev_char_kobj =3D kobject_create_and_add("char", dev_kob=
j);
> > > =C2=A0	if (!sysfs_dev_char_kobj)
> > > =C2=A0		goto char_kobj_err;
> > > +	device_link_wq =3D alloc_workqueue("device_link_wq", 0, 0);=C2=A0=
=20
> >=20
> > My rb tag was with the assumption this is moved into devlink_class_init=
(). IIUC,
> > Saravana also agreed with that [1]. But it looks like he missed that we=
 are
> > allocating the queue in devices_init() and not in devlink_class_init().
> >=20
> > I'm also not sure if this is in line with what Rafael wanted for ccing =
stable.
> > How do
> > we know the next patch depends on this one?
> >=20
> > [1]:
> > https://lore.kernel.org/lkml/CAGETcx_gNWOTsSZMaZu+XU1-5Z60WEcMhw08t4Sn_=
-YgkCCUmA@mail.gmail.com/
> >=20
>=20
> We discussed that point and I understood that you were ok to do that on y=
our
> side:
> =C2=A0
> https://lore.kernel.org/linux-kernel/f42ceee61ddb8b50c347589649d4131476ab=
5d81.camel@gmail.com/
>=20
> Sorry if I misunderstood.

Oh, yeah, I can do that. But given Saravana reply I thought the expectation=
 is to
have the queue already allocated in devlink_class_init().

>=20
> I am going to wait for other comments on this current series before re-se=
nding
> with our 'Reviewed-by' removed if needed. Let me know.
>=20

Anyways, if your expectation was for me to do it later, fine. No need to re=
move the
tag. Sorry for the noise.

- Nuno S=C3=A1
>=20