2020-11-04 23:30:18

by Saravana Kannan

[permalink] [raw]
Subject: [PATCH v1 00/18] Refactor fw_devlink to significantly improve boot time

The current implementation of fw_devlink is very inefficient because it
tries to get away without creating fwnode links in the name of saving
memory usage. Past attempts to optimize runtime at the cost of memory
usage were blocked with request for data showing that the optimization
made significant improvement for real world scenarios.

We have those scenarios now. There have been several reports of boot
time increase in the order of seconds in this thread [1]. Several OEMs
and SoC manufacturers have also privately reported significant
(350-400ms) increase in boot time due to all the parsing done by
fw_devlink.

So this patch series refactors fw_devlink to be more efficient. The key
difference now is the addition of support for fwnode links -- just a few
simple APIs. This also allows most of the code to be moved out of
firmware specific (DT mostly) code into driver core.

This brings the following benefits:
- Instead of parsing the device tree multiple times (complexity was
close to O(N^3) where N in the number of properties) during bootup,
fw_devlink parses each fwnode node/property only once and creates
fwnode links. The rest of the fw_devlink code then just looks at these
fwnode links to do rest of the work.

- Makes it much easier to debug probe issue due to fw_devlink in the
future. fw_devlink=on blocks the probing of devices if they depend on
a device that hasn't been added yet. With this refactor, it'll be very
easy to tell what that device is because we now have a reference to
the fwnode of the device.

- Much easier to add fw_devlink support to ACPI and other firmware
types. A refactor to move the common bits from DT specific code to
driver core was in my TODO list as a prerequisite to adding ACPI
support to fw_devlink. This series gets that done.

Tomi/Laurent/Grygorii,

If you can test this series, that'd be great!

Thanks,
Saravana

[1] - https://lore.kernel.org/linux-pm/CAGETcx-aiW251dhEXT1GNb9bi6YcX8W=jLBrro5CnPuEjGL09g@mail.gmail.com/

Saravana Kannan (18):
Revert "driver core: Avoid deferred probe due to fw_devlink_pause/resume()"
Revert "driver core: Rename dev_links_info.defer_sync to defer_hook"
Revert "driver core: Don't do deferred probe in parallel with kernel_init thread"
Revert "driver core: Remove check in driver_deferred_probe_force_trigger()"
Revert "of: platform: Batch fwnode parsing when adding all top level devices"
Revert "driver core: fw_devlink: Add support for batching fwnode parsing"
driver core: Add fwnode_init()
driver core: Add fwnode link support
driver core: Allow only unprobed consumers for SYNC_STATE_ONLY device links
device property: Add fwnode_is_ancestor_of()
driver core: Redefine the meaning of fwnode_operations.add_links()
driver core: Add fw_devlink_parse_fwtree()
driver core: Add fwnode_get_next_parent_dev() helper function
driver core: Use device's fwnode to check if it is waiting for suppliers
of: property: Update implementation of add_links() to create fwnode links
efi: Update implementation of add_links() to create fwnode links
driver core: Add helper functions to convert fwnode links to device links
driver core: Refactor fw_devlink feature

drivers/acpi/property.c | 2 +-
drivers/acpi/scan.c | 2 +-
drivers/base/core.c | 584 +++++++++++++++++++++-----------
drivers/base/property.c | 27 ++
drivers/base/swnode.c | 2 +-
drivers/firmware/efi/efi-init.c | 31 +-
drivers/of/dynamic.c | 1 +
drivers/of/platform.c | 2 -
drivers/of/property.c | 150 +++-----
include/linux/device.h | 10 +-
include/linux/fwnode.h | 66 ++--
include/linux/of.h | 2 +-
include/linux/property.h | 2 +
kernel/irq/irqdomain.c | 2 +-
14 files changed, 490 insertions(+), 393 deletions(-)

--
2.29.1.341.ge80a0c044ae-goog


2020-11-04 23:30:24

by Saravana Kannan

[permalink] [raw]
Subject: [PATCH v1 05/18] Revert "of: platform: Batch fwnode parsing when adding all top level devices"

This reverts commit 93d2e4322aa74c1ad1e8c2160608eb9a960d69ff.

Signed-off-by: Saravana Kannan <[email protected]>
---
drivers/of/platform.c | 2 --
1 file changed, 2 deletions(-)

diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index b557a0fcd4ba..79bd5f5a1bf1 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -538,9 +538,7 @@ static int __init of_platform_default_populate_init(void)
}

/* Populate everything else. */
- fw_devlink_pause();
of_platform_default_populate(NULL, NULL, NULL);
- fw_devlink_resume();

return 0;
}
--
2.29.1.341.ge80a0c044ae-goog

2020-11-04 23:30:26

by Saravana Kannan

[permalink] [raw]
Subject: Re: [PATCH v1 00/18] Refactor fw_devlink to significantly improve boot time

On Wed, Nov 4, 2020 at 3:23 PM Saravana Kannan <[email protected]> wrote:
>
> The current implementation of fw_devlink is very inefficient because it
> tries to get away without creating fwnode links in the name of saving
> memory usage. Past attempts to optimize runtime at the cost of memory
> usage were blocked with request for data showing that the optimization
> made significant improvement for real world scenarios.
>
> We have those scenarios now. There have been several reports of boot
> time increase in the order of seconds in this thread [1]. Several OEMs
> and SoC manufacturers have also privately reported significant
> (350-400ms) increase in boot time due to all the parsing done by
> fw_devlink.
>
> So this patch series refactors fw_devlink to be more efficient. The key
> difference now is the addition of support for fwnode links -- just a few
> simple APIs. This also allows most of the code to be moved out of
> firmware specific (DT mostly) code into driver core.
>
> This brings the following benefits:
> - Instead of parsing the device tree multiple times (complexity was
> close to O(N^3) where N in the number of properties) during bootup,
> fw_devlink parses each fwnode node/property only once and creates
> fwnode links. The rest of the fw_devlink code then just looks at these
> fwnode links to do rest of the work.
>
> - Makes it much easier to debug probe issue due to fw_devlink in the
> future. fw_devlink=on blocks the probing of devices if they depend on
> a device that hasn't been added yet. With this refactor, it'll be very
> easy to tell what that device is because we now have a reference to
> the fwnode of the device.
>
> - Much easier to add fw_devlink support to ACPI and other firmware
> types. A refactor to move the common bits from DT specific code to
> driver core was in my TODO list as a prerequisite to adding ACPI
> support to fw_devlink. This series gets that done.
>
> Tomi/Laurent/Grygorii,
>
> If you can test this series, that'd be great!
>
> Thanks,
> Saravana
>
> [1] - https://lore.kernel.org/linux-pm/CAGETcx-aiW251dhEXT1GNb9bi6YcX8W=jLBrro5CnPuEjGL09g@mail.gmail.com/

Forgot the update this link in the cover letter. Here's a much better
link to the thread that discusses boot time regression:
https://lore.kernel.org/linux-omap/[email protected]/


-Saravana

2020-11-05 02:43:17

by Saravana Kannan

[permalink] [raw]
Subject: [PATCH v1 10/18] device property: Add fwnode_is_ancestor_of()

Add a helper function to check if a fwnode is an ancestor of another
fwnode. This will be useful for fw_devlink feature.

Signed-off-by: Saravana Kannan <[email protected]>
---
drivers/base/property.c | 27 +++++++++++++++++++++++++++
include/linux/property.h | 2 ++
2 files changed, 29 insertions(+)

diff --git a/drivers/base/property.c b/drivers/base/property.c
index 4c43d30145c6..2569ebd52e6b 100644
--- a/drivers/base/property.c
+++ b/drivers/base/property.c
@@ -660,6 +660,33 @@ struct fwnode_handle *fwnode_get_nth_parent(struct fwnode_handle *fwnode,
}
EXPORT_SYMBOL_GPL(fwnode_get_nth_parent);

+/**
+ * fwnode_is_ancestor_of - Test if @test_ancestor is ancestor of @test_child
+ * @test_ancestor: Firmware which is tested for being an ancestor
+ * @test_child: Firmware which is tested for being the child
+ *
+ * A node is considered an ancestor of itself too.
+ *
+ * Returns true if @test_ancestor is an ancestor of @test_child.
+ * Otherwise, returns false.
+ */
+bool fwnode_is_ancestor_of(struct fwnode_handle *test_ancestor,
+ struct fwnode_handle *test_child)
+{
+ if (!test_ancestor)
+ return false;
+
+ fwnode_handle_get(test_child);
+ while (test_child) {
+ if (test_child == test_ancestor) {
+ fwnode_handle_put(test_child);
+ return true;
+ }
+ test_child = fwnode_get_next_parent(test_child);
+ }
+ return false;
+}
+
/**
* fwnode_get_next_child_node - Return the next child node handle for a node
* @fwnode: Firmware node to find the next child node for.
diff --git a/include/linux/property.h b/include/linux/property.h
index 2d4542629d80..2e5eed3ddf1c 100644
--- a/include/linux/property.h
+++ b/include/linux/property.h
@@ -88,6 +88,8 @@ struct fwnode_handle *fwnode_get_next_parent(
unsigned int fwnode_count_parents(const struct fwnode_handle *fwn);
struct fwnode_handle *fwnode_get_nth_parent(struct fwnode_handle *fwn,
unsigned int depth);
+bool fwnode_is_ancestor_of(struct fwnode_handle *test_ancestor,
+ struct fwnode_handle *test_child);
struct fwnode_handle *fwnode_get_next_child_node(
const struct fwnode_handle *fwnode, struct fwnode_handle *child);
struct fwnode_handle *fwnode_get_next_available_child_node(
--
2.29.1.341.ge80a0c044ae-goog

2020-11-05 02:57:53

by Saravana Kannan

[permalink] [raw]
Subject: [PATCH v1 12/18] driver core: Add fw_devlink_parse_fwtree()

This function is a wrapper around fwnode_operations.add_links().

This function parses each node in a fwnode tree and create fwnode links
for each of those nodes. The information for creating the fwnode links
(the supplier and consumer fwnode) is obtained by parsing the properties
in each of the fwnodes.

This function also ensures that no fwnode is parsed more than once by
marking the fwnodes as parsed.

Signed-off-by: Saravana Kannan <[email protected]>
---
drivers/base/core.c | 19 +++++++++++++++++++
include/linux/fwnode.h | 3 +++
2 files changed, 22 insertions(+)

diff --git a/drivers/base/core.c b/drivers/base/core.c
index 4a0907574646..ee28d8c7ee85 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -1543,6 +1543,25 @@ static bool fw_devlink_is_permissive(void)
return fw_devlink_flags == DL_FLAG_SYNC_STATE_ONLY;
}

+static void fw_devlink_parse_fwnode(struct fwnode_handle *fwnode)
+{
+ if (fwnode->flags & FWNODE_FLAG_LINKS_ADDED)
+ return;
+
+ fwnode_call_int_op(fwnode, add_links, NULL);
+ fwnode->flags |= FWNODE_FLAG_LINKS_ADDED;
+}
+
+static void fw_devlink_parse_fwtree(struct fwnode_handle *fwnode)
+{
+ struct fwnode_handle *child = NULL;
+
+ fw_devlink_parse_fwnode(fwnode);
+
+ while ((child = fwnode_get_next_available_child_node(fwnode, child)))
+ fw_devlink_parse_fwtree(child);
+}
+
static void fw_devlink_link_device(struct device *dev)
{
int fw_ret;
diff --git a/include/linux/fwnode.h b/include/linux/fwnode.h
index ec02e1e939cc..9aaf9e4f3994 100644
--- a/include/linux/fwnode.h
+++ b/include/linux/fwnode.h
@@ -15,12 +15,15 @@
struct fwnode_operations;
struct device;

+#define FWNODE_FLAG_LINKS_ADDED BIT(0)
+
struct fwnode_handle {
struct fwnode_handle *secondary;
const struct fwnode_operations *ops;
struct device *dev;
struct list_head suppliers;
struct list_head consumers;
+ u32 flags;
};

struct fwnode_link {
--
2.29.1.341.ge80a0c044ae-goog

2020-11-05 02:57:53

by Saravana Kannan

[permalink] [raw]
Subject: [PATCH v1 16/18] efi: Update implementation of add_links() to create fwnode links

The semantics of add_links() has changed from creating device link
between devices to creating fwnode links between fwnodes. So, update the
implementation of add_links() to match the new semantics.

Signed-off-by: Saravana Kannan <[email protected]>
---
drivers/firmware/efi/efi-init.c | 23 ++---------------------
1 file changed, 2 insertions(+), 21 deletions(-)

diff --git a/drivers/firmware/efi/efi-init.c b/drivers/firmware/efi/efi-init.c
index b148f1459fb3..c0c3d4c3837a 100644
--- a/drivers/firmware/efi/efi-init.c
+++ b/drivers/firmware/efi/efi-init.c
@@ -316,11 +316,10 @@ static struct device_node *find_pci_overlap_node(void)
* resource reservation conflict on the memory window that the efifb
* framebuffer steals from the PCIe host bridge.
*/
-static int efifb_add_links(const struct fwnode_handle *fwnode,
+static int efifb_add_links(struct fwnode_handle *fwnode,
struct device *dev)
{
struct device_node *sup_np;
- struct device *sup_dev;

sup_np = find_pci_overlap_node();

@@ -331,27 +330,9 @@ static int efifb_add_links(const struct fwnode_handle *fwnode,
if (!sup_np)
return 0;

- sup_dev = get_dev_from_fwnode(&sup_np->fwnode);
+ fwnode_link_add(fwnode, of_fwnode_handle(sup_np));
of_node_put(sup_np);

- /*
- * Return -ENODEV if the PCI graphics controller device hasn't been
- * registered yet. This ensures that efifb isn't allowed to probe
- * and this function is retried again when new devices are
- * registered.
- */
- if (!sup_dev)
- return -ENODEV;
-
- /*
- * If this fails, retrying this function at a later point won't
- * change anything. So, don't return an error after this.
- */
- if (!device_link_add(dev, sup_dev, fw_devlink_get_flags()))
- dev_warn(dev, "device_link_add() failed\n");
-
- put_device(sup_dev);
-
return 0;
}

--
2.29.1.341.ge80a0c044ae-goog

2020-11-05 02:57:53

by Saravana Kannan

[permalink] [raw]
Subject: [PATCH v1 14/18] driver core: Use device's fwnode to check if it is waiting for suppliers

To check if a device is still waiting for its supplier devices to be
added, we used to check if the devices is in a global
waiting_for_suppliers list. Since the global list will be deleted in
subsequent patches, this patch stops using this check.

Instead, this patch uses a more device specific check. It checks if the
device's fwnode has any fwnode links that haven't been converted to
device links yet.

Signed-off-by: Saravana Kannan <[email protected]>
---
drivers/base/core.c | 18 ++++++++----------
1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/drivers/base/core.c b/drivers/base/core.c
index 4ae5f2885ac5..d51dd564add1 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -51,6 +51,7 @@ static DEFINE_MUTEX(wfs_lock);
static LIST_HEAD(deferred_sync);
static unsigned int defer_sync_state_count = 1;
static DEFINE_MUTEX(fwnode_link_lock);
+static bool fw_devlink_is_permissive(void);

/**
* fwnode_link_add - Create a link between two fwnode_handles.
@@ -994,13 +995,13 @@ int device_links_check_suppliers(struct device *dev)
* Device waiting for supplier to become available is not allowed to
* probe.
*/
- mutex_lock(&wfs_lock);
- if (!list_empty(&dev->links.needs_suppliers) &&
- dev->links.need_for_probe) {
- mutex_unlock(&wfs_lock);
+ mutex_lock(&fwnode_link_lock);
+ if (dev->fwnode && !list_empty(&dev->fwnode->suppliers) &&
+ !fw_devlink_is_permissive()) {
+ mutex_unlock(&fwnode_link_lock);
return -EPROBE_DEFER;
}
- mutex_unlock(&wfs_lock);
+ mutex_unlock(&fwnode_link_lock);

device_links_write_lock();

@@ -1166,10 +1167,7 @@ static ssize_t waiting_for_supplier_show(struct device *dev,
bool val;

device_lock(dev);
- mutex_lock(&wfs_lock);
- val = !list_empty(&dev->links.needs_suppliers)
- && dev->links.need_for_probe;
- mutex_unlock(&wfs_lock);
+ val = !list_empty(&dev->fwnode->suppliers);
device_unlock(dev);
return sysfs_emit(buf, "%u\n", val);
}
@@ -2226,7 +2224,7 @@ static int device_add_attrs(struct device *dev)
goto err_remove_dev_groups;
}

- if (fw_devlink_flags && !fw_devlink_is_permissive()) {
+ if (fw_devlink_flags && !fw_devlink_is_permissive() && dev->fwnode) {
error = device_create_file(dev, &dev_attr_waiting_for_supplier);
if (error)
goto err_remove_dev_online;
--
2.29.1.341.ge80a0c044ae-goog

2020-11-05 03:02:51

by Saravana Kannan

[permalink] [raw]
Subject: [PATCH v1 11/18] driver core: Redefine the meaning of fwnode_operations.add_links()

Change the meaning of fwnode_operations.add_links() to just create
fwnode links by parsing the properties of a given fwnode.

This patch doesn't actually make any code changes. To keeps things more
digestable, the actual functional changes come in later patches in this
series.

Signed-off-by: Saravana Kannan <[email protected]>
---
include/linux/fwnode.h | 42 +++---------------------------------------
1 file changed, 3 insertions(+), 39 deletions(-)

diff --git a/include/linux/fwnode.h b/include/linux/fwnode.h
index afde643f37a2..ec02e1e939cc 100644
--- a/include/linux/fwnode.h
+++ b/include/linux/fwnode.h
@@ -78,44 +78,8 @@ struct fwnode_reference_args {
* endpoint node.
* @graph_get_port_parent: Return the parent node of a port node.
* @graph_parse_endpoint: Parse endpoint for port and endpoint id.
- * @add_links: Called after the device corresponding to the fwnode is added
- * using device_add(). The function is expected to create device
- * links to all the suppliers of the device that are available at
- * the time this function is called. The function must NOT stop
- * at the first failed device link if other unlinked supplier
- * devices are present in the system. This is necessary for the
- * driver/bus sync_state() callbacks to work correctly.
- *
- * For example, say Device-C depends on suppliers Device-S1 and
- * Device-S2 and the dependency is listed in that order in the
- * firmware. Say, S1 gets populated from the firmware after
- * late_initcall_sync(). Say S2 is populated and probed way
- * before that in device_initcall(). When C is populated, if this
- * add_links() function doesn't continue past a "failed linking to
- * S1" and continue linking C to S2, then S2 will get a
- * sync_state() callback before C is probed. This is because from
- * the perspective of S2, C was never a consumer when its
- * sync_state() evaluation is done. To avoid this, the add_links()
- * function has to go through all available suppliers of the
- * device (that corresponds to this fwnode) and link to them
- * before returning.
- *
- * If some suppliers are not yet available (indicated by an error
- * return value), this function will be called again when other
- * devices are added to allow creating device links to any newly
- * available suppliers.
- *
- * Return 0 if device links have been successfully created to all
- * the known suppliers of this device or if the supplier
- * information is not known.
- *
- * Return -ENODEV if the suppliers needed for probing this device
- * have not been registered yet (because device links can only be
- * created to devices registered with the driver core).
- *
- * Return -EAGAIN if some of the suppliers of this device have not
- * been registered yet, but none of those suppliers are necessary
- * for probing the device.
+ * @add_links: Create fwnode links to all the suppliers of the fwnode. Return
+ * zero on success, a negative error code otherwise.
*/
struct fwnode_operations {
struct fwnode_handle *(*get)(struct fwnode_handle *fwnode);
@@ -155,7 +119,7 @@ struct fwnode_operations {
(*graph_get_port_parent)(struct fwnode_handle *fwnode);
int (*graph_parse_endpoint)(const struct fwnode_handle *fwnode,
struct fwnode_endpoint *endpoint);
- int (*add_links)(const struct fwnode_handle *fwnode,
+ int (*add_links)(struct fwnode_handle *fwnode,
struct device *dev);
};

--
2.29.1.341.ge80a0c044ae-goog

2020-11-05 09:44:28

by Greg Kroah-Hartman

[permalink] [raw]
Subject: Re: [PATCH v1 16/18] efi: Update implementation of add_links() to create fwnode links

On Wed, Nov 04, 2020 at 03:23:53PM -0800, Saravana Kannan wrote:
> The semantics of add_links() has changed from creating device link
> between devices to creating fwnode links between fwnodes. So, update the
> implementation of add_links() to match the new semantics.
>
> Signed-off-by: Saravana Kannan <[email protected]>
> ---
> drivers/firmware/efi/efi-init.c | 23 ++---------------------
> 1 file changed, 2 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/firmware/efi/efi-init.c b/drivers/firmware/efi/efi-init.c
> index b148f1459fb3..c0c3d4c3837a 100644
> --- a/drivers/firmware/efi/efi-init.c
> +++ b/drivers/firmware/efi/efi-init.c
> @@ -316,11 +316,10 @@ static struct device_node *find_pci_overlap_node(void)
> * resource reservation conflict on the memory window that the efifb
> * framebuffer steals from the PCIe host bridge.
> */
> -static int efifb_add_links(const struct fwnode_handle *fwnode,
> +static int efifb_add_links(struct fwnode_handle *fwnode,
> struct device *dev)

So you are fixing the build warning you added a few patches ago here?
Please fix up the function signatures when you made that change, not
here later on.

thanks,

greg k-h

2020-11-05 23:30:23

by Saravana Kannan

[permalink] [raw]
Subject: Re: [PATCH v1 16/18] efi: Update implementation of add_links() to create fwnode links

On Thu, Nov 5, 2020 at 1:42 AM Greg Kroah-Hartman
<[email protected]> wrote:
>
> On Wed, Nov 04, 2020 at 03:23:53PM -0800, Saravana Kannan wrote:
> > The semantics of add_links() has changed from creating device link
> > between devices to creating fwnode links between fwnodes. So, update the
> > implementation of add_links() to match the new semantics.
> >
> > Signed-off-by: Saravana Kannan <[email protected]>
> > ---
> > drivers/firmware/efi/efi-init.c | 23 ++---------------------
> > 1 file changed, 2 insertions(+), 21 deletions(-)
> >
> > diff --git a/drivers/firmware/efi/efi-init.c b/drivers/firmware/efi/efi-init.c
> > index b148f1459fb3..c0c3d4c3837a 100644
> > --- a/drivers/firmware/efi/efi-init.c
> > +++ b/drivers/firmware/efi/efi-init.c
> > @@ -316,11 +316,10 @@ static struct device_node *find_pci_overlap_node(void)
> > * resource reservation conflict on the memory window that the efifb
> > * framebuffer steals from the PCIe host bridge.
> > */
> > -static int efifb_add_links(const struct fwnode_handle *fwnode,
> > +static int efifb_add_links(struct fwnode_handle *fwnode,
> > struct device *dev)
>
> So you are fixing the build warning you added a few patches ago here?
> Please fix up the function signatures when you made that change, not
> here later on.

I'm trying not to have a mega patcht that changes a lot of code.

I guess I can drop this "const" diff from this patch and then merge it
with the earlier patch that removes the const. But still leave the
rest of the changes in this patch as is. Does that work for you?

-Saravana

2020-11-06 05:14:48

by Laurent Pinchart

[permalink] [raw]
Subject: Re: [PATCH v1 00/18] Refactor fw_devlink to significantly improve boot time

Hi Saravana,

Thank you for working on this !

On Wed, Nov 04, 2020 at 03:23:37PM -0800, Saravana Kannan wrote:
> The current implementation of fw_devlink is very inefficient because it
> tries to get away without creating fwnode links in the name of saving
> memory usage. Past attempts to optimize runtime at the cost of memory
> usage were blocked with request for data showing that the optimization
> made significant improvement for real world scenarios.
>
> We have those scenarios now. There have been several reports of boot
> time increase in the order of seconds in this thread [1]. Several OEMs
> and SoC manufacturers have also privately reported significant
> (350-400ms) increase in boot time due to all the parsing done by
> fw_devlink.
>
> So this patch series refactors fw_devlink to be more efficient. The key
> difference now is the addition of support for fwnode links -- just a few
> simple APIs. This also allows most of the code to be moved out of
> firmware specific (DT mostly) code into driver core.
>
> This brings the following benefits:
> - Instead of parsing the device tree multiple times (complexity was
> close to O(N^3) where N in the number of properties) during bootup,
> fw_devlink parses each fwnode node/property only once and creates
> fwnode links. The rest of the fw_devlink code then just looks at these
> fwnode links to do rest of the work.
>
> - Makes it much easier to debug probe issue due to fw_devlink in the
> future. fw_devlink=on blocks the probing of devices if they depend on
> a device that hasn't been added yet. With this refactor, it'll be very
> easy to tell what that device is because we now have a reference to
> the fwnode of the device.
>
> - Much easier to add fw_devlink support to ACPI and other firmware
> types. A refactor to move the common bits from DT specific code to
> driver core was in my TODO list as a prerequisite to adding ACPI
> support to fw_devlink. This series gets that done.
>
> Tomi/Laurent/Grygorii,
>
> If you can test this series, that'd be great!

I gave it a try, rebasing my branch from v5.9 to v5.10-rc2 first. On
v5.10-rc2 the kernel dies when booting due to a deadlock (reported by
lockdep, so hopefully not too hard to debug). *sigh*. Fortunately, it
dies after the fw_devlink initialization, so I can still report results.

Before your series:

[ 0.743065] cpuidle: using governor menu
[ 13.350259] No ATAGs?

With your series applied:

[ 0.722670] cpuidle: using governor menu
[ 1.135859] No ATAGs?

That's a very clear improvement :-)

Tested-by: Laurent Pinchart <[email protected]>

> [1] - https://lore.kernel.org/linux-pm/CAGETcx-aiW251dhEXT1GNb9bi6YcX8W=jLBrro5CnPuEjGL09g@mail.gmail.com/
>
> Saravana Kannan (18):
> Revert "driver core: Avoid deferred probe due to fw_devlink_pause/resume()"
> Revert "driver core: Rename dev_links_info.defer_sync to defer_hook"
> Revert "driver core: Don't do deferred probe in parallel with kernel_init thread"
> Revert "driver core: Remove check in driver_deferred_probe_force_trigger()"
> Revert "of: platform: Batch fwnode parsing when adding all top level devices"
> Revert "driver core: fw_devlink: Add support for batching fwnode parsing"
> driver core: Add fwnode_init()
> driver core: Add fwnode link support
> driver core: Allow only unprobed consumers for SYNC_STATE_ONLY device links
> device property: Add fwnode_is_ancestor_of()
> driver core: Redefine the meaning of fwnode_operations.add_links()
> driver core: Add fw_devlink_parse_fwtree()
> driver core: Add fwnode_get_next_parent_dev() helper function
> driver core: Use device's fwnode to check if it is waiting for suppliers
> of: property: Update implementation of add_links() to create fwnode links
> efi: Update implementation of add_links() to create fwnode links
> driver core: Add helper functions to convert fwnode links to device links
> driver core: Refactor fw_devlink feature
>
> drivers/acpi/property.c | 2 +-
> drivers/acpi/scan.c | 2 +-
> drivers/base/core.c | 584 +++++++++++++++++++++-----------
> drivers/base/property.c | 27 ++
> drivers/base/swnode.c | 2 +-
> drivers/firmware/efi/efi-init.c | 31 +-
> drivers/of/dynamic.c | 1 +
> drivers/of/platform.c | 2 -
> drivers/of/property.c | 150 +++-----
> include/linux/device.h | 10 +-
> include/linux/fwnode.h | 66 ++--
> include/linux/of.h | 2 +-
> include/linux/property.h | 2 +
> kernel/irq/irqdomain.c | 2 +-
> 14 files changed, 490 insertions(+), 393 deletions(-)

--
Regards,

Laurent Pinchart

2020-11-06 06:49:38

by Greg Kroah-Hartman

[permalink] [raw]
Subject: Re: [PATCH v1 16/18] efi: Update implementation of add_links() to create fwnode links

On Thu, Nov 05, 2020 at 03:27:52PM -0800, Saravana Kannan wrote:
> On Thu, Nov 5, 2020 at 1:42 AM Greg Kroah-Hartman
> <[email protected]> wrote:
> >
> > On Wed, Nov 04, 2020 at 03:23:53PM -0800, Saravana Kannan wrote:
> > > The semantics of add_links() has changed from creating device link
> > > between devices to creating fwnode links between fwnodes. So, update the
> > > implementation of add_links() to match the new semantics.
> > >
> > > Signed-off-by: Saravana Kannan <[email protected]>
> > > ---
> > > drivers/firmware/efi/efi-init.c | 23 ++---------------------
> > > 1 file changed, 2 insertions(+), 21 deletions(-)
> > >
> > > diff --git a/drivers/firmware/efi/efi-init.c b/drivers/firmware/efi/efi-init.c
> > > index b148f1459fb3..c0c3d4c3837a 100644
> > > --- a/drivers/firmware/efi/efi-init.c
> > > +++ b/drivers/firmware/efi/efi-init.c
> > > @@ -316,11 +316,10 @@ static struct device_node *find_pci_overlap_node(void)
> > > * resource reservation conflict on the memory window that the efifb
> > > * framebuffer steals from the PCIe host bridge.
> > > */
> > > -static int efifb_add_links(const struct fwnode_handle *fwnode,
> > > +static int efifb_add_links(struct fwnode_handle *fwnode,
> > > struct device *dev)
> >
> > So you are fixing the build warning you added a few patches ago here?
> > Please fix up the function signatures when you made that change, not
> > here later on.
>
> I'm trying not to have a mega patcht that changes a lot of code.
>
> I guess I can drop this "const" diff from this patch and then merge it
> with the earlier patch that removes the const. But still leave the
> rest of the changes in this patch as is. Does that work for you?

Yes, that's fine, you just can't add build warnings along the way.

thanks,

greg k-h

2020-11-06 08:41:03

by Saravana Kannan

[permalink] [raw]
Subject: Re: [PATCH v1 00/18] Refactor fw_devlink to significantly improve boot time

On Thu, Nov 5, 2020 at 9:09 PM Laurent Pinchart
<[email protected]> wrote:
>
> Hi Saravana,
>
> Thank you for working on this !
>
> On Wed, Nov 04, 2020 at 03:23:37PM -0800, Saravana Kannan wrote:
> > The current implementation of fw_devlink is very inefficient because it
> > tries to get away without creating fwnode links in the name of saving
> > memory usage. Past attempts to optimize runtime at the cost of memory
> > usage were blocked with request for data showing that the optimization
> > made significant improvement for real world scenarios.
> >
> > We have those scenarios now. There have been several reports of boot
> > time increase in the order of seconds in this thread [1]. Several OEMs
> > and SoC manufacturers have also privately reported significant
> > (350-400ms) increase in boot time due to all the parsing done by
> > fw_devlink.
> >
> > So this patch series refactors fw_devlink to be more efficient. The key
> > difference now is the addition of support for fwnode links -- just a few
> > simple APIs. This also allows most of the code to be moved out of
> > firmware specific (DT mostly) code into driver core.
> >
> > This brings the following benefits:
> > - Instead of parsing the device tree multiple times (complexity was
> > close to O(N^3) where N in the number of properties) during bootup,
> > fw_devlink parses each fwnode node/property only once and creates
> > fwnode links. The rest of the fw_devlink code then just looks at these
> > fwnode links to do rest of the work.
> >
> > - Makes it much easier to debug probe issue due to fw_devlink in the
> > future. fw_devlink=on blocks the probing of devices if they depend on
> > a device that hasn't been added yet. With this refactor, it'll be very
> > easy to tell what that device is because we now have a reference to
> > the fwnode of the device.
> >
> > - Much easier to add fw_devlink support to ACPI and other firmware
> > types. A refactor to move the common bits from DT specific code to
> > driver core was in my TODO list as a prerequisite to adding ACPI
> > support to fw_devlink. This series gets that done.
> >
> > Tomi/Laurent/Grygorii,
> >
> > If you can test this series, that'd be great!
>
> I gave it a try, rebasing my branch from v5.9 to v5.10-rc2 first. On
> v5.10-rc2 the kernel dies when booting due to a deadlock (reported by
> lockdep, so hopefully not too hard to debug). *sigh*. Fortunately, it
> dies after the fw_devlink initialization, so I can still report results.

Phew! For a sec I thought you said fw_devlink was causing a deadlock.

>
> Before your series:
>
> [ 0.743065] cpuidle: using governor menu
> [ 13.350259] No ATAGs?
>
> With your series applied:
>
> [ 0.722670] cpuidle: using governor menu
> [ 1.135859] No ATAGs?
>
> That's a very clear improvement :-)

Thanks for testing. Great to hear it's helping!

> Tested-by: Laurent Pinchart <[email protected]>

I'll add it to my v2 series.

-Saravana

2020-11-06 12:48:17

by Grygorii Strashko

[permalink] [raw]
Subject: Re: [PATCH v1 00/18] Refactor fw_devlink to significantly improve boot time



On 06/11/2020 10:36, Saravana Kannan wrote:
> On Thu, Nov 5, 2020 at 9:09 PM Laurent Pinchart
> <[email protected]> wrote:
>>
>> Hi Saravana,
>>
>> Thank you for working on this !
>>
>> On Wed, Nov 04, 2020 at 03:23:37PM -0800, Saravana Kannan wrote:
>>> The current implementation of fw_devlink is very inefficient because it
>>> tries to get away without creating fwnode links in the name of saving
>>> memory usage. Past attempts to optimize runtime at the cost of memory
>>> usage were blocked with request for data showing that the optimization
>>> made significant improvement for real world scenarios.
>>>
>>> We have those scenarios now. There have been several reports of boot
>>> time increase in the order of seconds in this thread [1]. Several OEMs
>>> and SoC manufacturers have also privately reported significant
>>> (350-400ms) increase in boot time due to all the parsing done by
>>> fw_devlink.
>>>
>>> So this patch series refactors fw_devlink to be more efficient. The key
>>> difference now is the addition of support for fwnode links -- just a few
>>> simple APIs. This also allows most of the code to be moved out of
>>> firmware specific (DT mostly) code into driver core.
>>>
>>> This brings the following benefits:
>>> - Instead of parsing the device tree multiple times (complexity was
>>> close to O(N^3) where N in the number of properties) during bootup,
>>> fw_devlink parses each fwnode node/property only once and creates
>>> fwnode links. The rest of the fw_devlink code then just looks at these
>>> fwnode links to do rest of the work.
>>>
>>> - Makes it much easier to debug probe issue due to fw_devlink in the
>>> future. fw_devlink=on blocks the probing of devices if they depend on
>>> a device that hasn't been added yet. With this refactor, it'll be very
>>> easy to tell what that device is because we now have a reference to
>>> the fwnode of the device.
>>>
>>> - Much easier to add fw_devlink support to ACPI and other firmware
>>> types. A refactor to move the common bits from DT specific code to
>>> driver core was in my TODO list as a prerequisite to adding ACPI
>>> support to fw_devlink. This series gets that done.
>>>
>>> Tomi/Laurent/Grygorii,
>>>
>>> If you can test this series, that'd be great!
>>
>> I gave it a try, rebasing my branch from v5.9 to v5.10-rc2 first. On
>> v5.10-rc2 the kernel dies when booting due to a deadlock (reported by
>> lockdep, so hopefully not too hard to debug). *sigh*. Fortunately, it
>> dies after the fw_devlink initialization, so I can still report results.
>
> Phew! For a sec I thought you said fw_devlink was causing a deadlock.
>
>>
>> Before your series:
>>
>> [ 0.743065] cpuidle: using governor menu
>> [ 13.350259] No ATAGs?
>>
>> With your series applied:
>>
>> [ 0.722670] cpuidle: using governor menu
>> [ 1.135859] No ATAGs?
>>
>> That's a very clear improvement :-)
>
> Thanks for testing. Great to hear it's helping!
>
>> Tested-by: Laurent Pinchart <[email protected]>
>
> I'll add it to my v2 series.

I've tried your series on top of
521b619acdc8 Merge tag 'linux-kselftest-kunit-fixes-5.10-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
on am571x-idk

Before:
[ 0.049395] cpuidle: using governor menu
[ 1.654766] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[ 2.315266] No ATAGs?
[ 2.315317] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
[ 2.315327] hw-breakpoint: maximum watchpoint size is 8 bytes.
...
[ 6.549595] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
[ 6.557794] VFS: Mounted root (ext4 filesystem) on device 179:26.
[ 6.574103] devtmpfs: mounted
[ 6.577749] Freeing unused kernel memory: 1024K
[ 6.582433] Run /sbin/init as init process


after:
[ 0.049223] cpuidle: using governor menu
[ 0.095893] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[ 0.102958] No ATAGs?
[ 0.103010] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
[ 0.103020] hw-breakpoint: maximum watchpoint size is 8 bytes.
...
[ 3.518623] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
[ 3.526822] VFS: Mounted root (ext4 filesystem) on device 179:26.
[ 3.543128] devtmpfs: mounted
[ 3.546781] Freeing unused kernel memory: 1024K
[ 3.551463] Run /sbin/init as init process

So, it's much better. Thank you.
Tested-by: Grygorii Strashko <[email protected]>

--
Best regards,
grygorii

2020-11-16 16:18:11

by Rafael J. Wysocki

[permalink] [raw]
Subject: Re: [PATCH v1 11/18] driver core: Redefine the meaning of fwnode_operations.add_links()

On Thu, Nov 5, 2020 at 12:24 AM Saravana Kannan <[email protected]> wrote:
>
> Change the meaning of fwnode_operations.add_links() to just create
> fwnode links by parsing the properties of a given fwnode.
>
> This patch doesn't actually make any code changes. To keeps things more
> digestable, the actual functional changes come in later patches in this
> series.
>
> Signed-off-by: Saravana Kannan <[email protected]>
> ---
> include/linux/fwnode.h | 42 +++---------------------------------------
> 1 file changed, 3 insertions(+), 39 deletions(-)
>
> diff --git a/include/linux/fwnode.h b/include/linux/fwnode.h
> index afde643f37a2..ec02e1e939cc 100644
> --- a/include/linux/fwnode.h
> +++ b/include/linux/fwnode.h
> @@ -78,44 +78,8 @@ struct fwnode_reference_args {
> * endpoint node.
> * @graph_get_port_parent: Return the parent node of a port node.
> * @graph_parse_endpoint: Parse endpoint for port and endpoint id.
> - * @add_links: Called after the device corresponding to the fwnode is added
> - * using device_add(). The function is expected to create device
> - * links to all the suppliers of the device that are available at
> - * the time this function is called. The function must NOT stop
> - * at the first failed device link if other unlinked supplier
> - * devices are present in the system. This is necessary for the
> - * driver/bus sync_state() callbacks to work correctly.
> - *
> - * For example, say Device-C depends on suppliers Device-S1 and
> - * Device-S2 and the dependency is listed in that order in the
> - * firmware. Say, S1 gets populated from the firmware after
> - * late_initcall_sync(). Say S2 is populated and probed way
> - * before that in device_initcall(). When C is populated, if this
> - * add_links() function doesn't continue past a "failed linking to
> - * S1" and continue linking C to S2, then S2 will get a
> - * sync_state() callback before C is probed. This is because from
> - * the perspective of S2, C was never a consumer when its
> - * sync_state() evaluation is done. To avoid this, the add_links()
> - * function has to go through all available suppliers of the
> - * device (that corresponds to this fwnode) and link to them
> - * before returning.
> - *
> - * If some suppliers are not yet available (indicated by an error
> - * return value), this function will be called again when other
> - * devices are added to allow creating device links to any newly
> - * available suppliers.
> - *
> - * Return 0 if device links have been successfully created to all
> - * the known suppliers of this device or if the supplier
> - * information is not known.
> - *
> - * Return -ENODEV if the suppliers needed for probing this device
> - * have not been registered yet (because device links can only be
> - * created to devices registered with the driver core).
> - *
> - * Return -EAGAIN if some of the suppliers of this device have not
> - * been registered yet, but none of those suppliers are necessary
> - * for probing the device.
> + * @add_links: Create fwnode links to all the suppliers of the fwnode. Return
> + * zero on success, a negative error code otherwise.

I'd say something like "Create fwnode links to all nodes that
represent devices supplying resources to the device represented by the
current fwnode. Return ..., or a negative ... on failure."

> */
> struct fwnode_operations {
> struct fwnode_handle *(*get)(struct fwnode_handle *fwnode);
> @@ -155,7 +119,7 @@ struct fwnode_operations {
> (*graph_get_port_parent)(struct fwnode_handle *fwnode);
> int (*graph_parse_endpoint)(const struct fwnode_handle *fwnode,
> struct fwnode_endpoint *endpoint);
> - int (*add_links)(const struct fwnode_handle *fwnode,
> + int (*add_links)(struct fwnode_handle *fwnode,
> struct device *dev);
> };
>
> --
> 2.29.1.341.ge80a0c044ae-goog
>

2020-11-16 16:30:39

by Rafael J. Wysocki

[permalink] [raw]
Subject: Re: [PATCH v1 12/18] driver core: Add fw_devlink_parse_fwtree()

On Thu, Nov 5, 2020 at 12:24 AM Saravana Kannan <[email protected]> wrote:
>
> This function is a wrapper around fwnode_operations.add_links().
>
> This function parses each node in a fwnode tree and create fwnode links
> for each of those nodes. The information for creating the fwnode links
> (the supplier and consumer fwnode) is obtained by parsing the properties
> in each of the fwnodes.
>
> This function also ensures that no fwnode is parsed more than once by
> marking the fwnodes as parsed.
>
> Signed-off-by: Saravana Kannan <[email protected]>
> ---
> drivers/base/core.c | 19 +++++++++++++++++++
> include/linux/fwnode.h | 3 +++
> 2 files changed, 22 insertions(+)
>
> diff --git a/drivers/base/core.c b/drivers/base/core.c
> index 4a0907574646..ee28d8c7ee85 100644
> --- a/drivers/base/core.c
> +++ b/drivers/base/core.c
> @@ -1543,6 +1543,25 @@ static bool fw_devlink_is_permissive(void)
> return fw_devlink_flags == DL_FLAG_SYNC_STATE_ONLY;
> }
>
> +static void fw_devlink_parse_fwnode(struct fwnode_handle *fwnode)
> +{
> + if (fwnode->flags & FWNODE_FLAG_LINKS_ADDED)
> + return;

Why is the flag needed?

Duplicate links won't be created anyway and it doesn't cause the tree
walk to be terminated.

> +
> + fwnode_call_int_op(fwnode, add_links, NULL);
> + fwnode->flags |= FWNODE_FLAG_LINKS_ADDED;
> +}
> +
> +static void fw_devlink_parse_fwtree(struct fwnode_handle *fwnode)
> +{
> + struct fwnode_handle *child = NULL;
> +
> + fw_devlink_parse_fwnode(fwnode);
> +
> + while ((child = fwnode_get_next_available_child_node(fwnode, child)))

I'd prefer

for (child = NULL; child; child =
fwnode_get_next_available_child_node(fwnode, child))

> + fw_devlink_parse_fwtree(child);
> +}
> +
> static void fw_devlink_link_device(struct device *dev)
> {
> int fw_ret;
> diff --git a/include/linux/fwnode.h b/include/linux/fwnode.h
> index ec02e1e939cc..9aaf9e4f3994 100644
> --- a/include/linux/fwnode.h
> +++ b/include/linux/fwnode.h
> @@ -15,12 +15,15 @@
> struct fwnode_operations;
> struct device;
>

Description here, please.

> +#define FWNODE_FLAG_LINKS_ADDED BIT(0)
> +
> struct fwnode_handle {
> struct fwnode_handle *secondary;
> const struct fwnode_operations *ops;
> struct device *dev;
> struct list_head suppliers;
> struct list_head consumers;
> + u32 flags;

That's a bit wasteful. Maybe u8 would suffice for the time being?

> };
>
> struct fwnode_link {
> --
> 2.29.1.341.ge80a0c044ae-goog
>

2020-11-16 16:38:18

by Rafael J. Wysocki

[permalink] [raw]
Subject: Re: [PATCH v1 14/18] driver core: Use device's fwnode to check if it is waiting for suppliers

On Thu, Nov 5, 2020 at 12:24 AM Saravana Kannan <[email protected]> wrote:
>
> To check if a device is still waiting for its supplier devices to be
> added, we used to check if the devices is in a global
> waiting_for_suppliers list. Since the global list will be deleted in
> subsequent patches, this patch stops using this check.

My kind of educated guess is that you want to drop
waiting_for_suppliers and that's why you want to use supplier links
here.

>
> Instead, this patch uses a more device specific check. It checks if the
> device's fwnode has any fwnode links that haven't been converted to
> device links yet.
>
> Signed-off-by: Saravana Kannan <[email protected]>
> ---
> drivers/base/core.c | 18 ++++++++----------
> 1 file changed, 8 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/base/core.c b/drivers/base/core.c
> index 4ae5f2885ac5..d51dd564add1 100644
> --- a/drivers/base/core.c
> +++ b/drivers/base/core.c
> @@ -51,6 +51,7 @@ static DEFINE_MUTEX(wfs_lock);
> static LIST_HEAD(deferred_sync);
> static unsigned int defer_sync_state_count = 1;
> static DEFINE_MUTEX(fwnode_link_lock);
> +static bool fw_devlink_is_permissive(void);
>
> /**
> * fwnode_link_add - Create a link between two fwnode_handles.
> @@ -994,13 +995,13 @@ int device_links_check_suppliers(struct device *dev)
> * Device waiting for supplier to become available is not allowed to
> * probe.
> */
> - mutex_lock(&wfs_lock);
> - if (!list_empty(&dev->links.needs_suppliers) &&
> - dev->links.need_for_probe) {
> - mutex_unlock(&wfs_lock);
> + mutex_lock(&fwnode_link_lock);
> + if (dev->fwnode && !list_empty(&dev->fwnode->suppliers) &&
> + !fw_devlink_is_permissive()) {
> + mutex_unlock(&fwnode_link_lock);
> return -EPROBE_DEFER;
> }
> - mutex_unlock(&wfs_lock);
> + mutex_unlock(&fwnode_link_lock);
>
> device_links_write_lock();
>
> @@ -1166,10 +1167,7 @@ static ssize_t waiting_for_supplier_show(struct device *dev,
> bool val;
>
> device_lock(dev);
> - mutex_lock(&wfs_lock);
> - val = !list_empty(&dev->links.needs_suppliers)
> - && dev->links.need_for_probe;
> - mutex_unlock(&wfs_lock);

Why isn't the lock needed any more?

Or maybe it wasn't needed previously too?

> + val = !list_empty(&dev->fwnode->suppliers);
> device_unlock(dev);
> return sysfs_emit(buf, "%u\n", val);
> }
> @@ -2226,7 +2224,7 @@ static int device_add_attrs(struct device *dev)
> goto err_remove_dev_groups;
> }
>
> - if (fw_devlink_flags && !fw_devlink_is_permissive()) {
> + if (fw_devlink_flags && !fw_devlink_is_permissive() && dev->fwnode) {

And why is this change needed?

> error = device_create_file(dev, &dev_attr_waiting_for_supplier);
> if (error)
> goto err_remove_dev_online;
> --
> 2.29.1.341.ge80a0c044ae-goog
>

2020-11-21 02:03:33

by Saravana Kannan

[permalink] [raw]
Subject: Re: [PATCH v1 11/18] driver core: Redefine the meaning of fwnode_operations.add_links()

On Mon, Nov 16, 2020 at 8:16 AM Rafael J. Wysocki <[email protected]> wrote:
>
> On Thu, Nov 5, 2020 at 12:24 AM Saravana Kannan <[email protected]> wrote:
> >
> > Change the meaning of fwnode_operations.add_links() to just create
> > fwnode links by parsing the properties of a given fwnode.
> >
> > This patch doesn't actually make any code changes. To keeps things more
> > digestable, the actual functional changes come in later patches in this
> > series.
> >
> > Signed-off-by: Saravana Kannan <[email protected]>
> > ---
> > include/linux/fwnode.h | 42 +++---------------------------------------
> > 1 file changed, 3 insertions(+), 39 deletions(-)
> >
> > diff --git a/include/linux/fwnode.h b/include/linux/fwnode.h
> > index afde643f37a2..ec02e1e939cc 100644
> > --- a/include/linux/fwnode.h
> > +++ b/include/linux/fwnode.h
> > @@ -78,44 +78,8 @@ struct fwnode_reference_args {
> > * endpoint node.
> > * @graph_get_port_parent: Return the parent node of a port node.
> > * @graph_parse_endpoint: Parse endpoint for port and endpoint id.
> > - * @add_links: Called after the device corresponding to the fwnode is added
> > - * using device_add(). The function is expected to create device
> > - * links to all the suppliers of the device that are available at
> > - * the time this function is called. The function must NOT stop
> > - * at the first failed device link if other unlinked supplier
> > - * devices are present in the system. This is necessary for the
> > - * driver/bus sync_state() callbacks to work correctly.
> > - *
> > - * For example, say Device-C depends on suppliers Device-S1 and
> > - * Device-S2 and the dependency is listed in that order in the
> > - * firmware. Say, S1 gets populated from the firmware after
> > - * late_initcall_sync(). Say S2 is populated and probed way
> > - * before that in device_initcall(). When C is populated, if this
> > - * add_links() function doesn't continue past a "failed linking to
> > - * S1" and continue linking C to S2, then S2 will get a
> > - * sync_state() callback before C is probed. This is because from
> > - * the perspective of S2, C was never a consumer when its
> > - * sync_state() evaluation is done. To avoid this, the add_links()
> > - * function has to go through all available suppliers of the
> > - * device (that corresponds to this fwnode) and link to them
> > - * before returning.
> > - *
> > - * If some suppliers are not yet available (indicated by an error
> > - * return value), this function will be called again when other
> > - * devices are added to allow creating device links to any newly
> > - * available suppliers.
> > - *
> > - * Return 0 if device links have been successfully created to all
> > - * the known suppliers of this device or if the supplier
> > - * information is not known.
> > - *
> > - * Return -ENODEV if the suppliers needed for probing this device
> > - * have not been registered yet (because device links can only be
> > - * created to devices registered with the driver core).
> > - *
> > - * Return -EAGAIN if some of the suppliers of this device have not
> > - * been registered yet, but none of those suppliers are necessary
> > - * for probing the device.
> > + * @add_links: Create fwnode links to all the suppliers of the fwnode. Return
> > + * zero on success, a negative error code otherwise.
>
> I'd say something like "Create fwnode links to all nodes that
> represent devices supplying resources to the device represented by the
> current fwnode. Return ..., or a negative ... on failure."

I don't have a strong opinion about this, but want to clarify that I'm
intentionally choosing not to say "device" because not all fwnodes
will have devices created for them. Do you still want me to make this
change?


>
> > */
> > struct fwnode_operations {
> > struct fwnode_handle *(*get)(struct fwnode_handle *fwnode);
> > @@ -155,7 +119,7 @@ struct fwnode_operations {
> > (*graph_get_port_parent)(struct fwnode_handle *fwnode);
> > int (*graph_parse_endpoint)(const struct fwnode_handle *fwnode,
> > struct fwnode_endpoint *endpoint);
> > - int (*add_links)(const struct fwnode_handle *fwnode,
> > + int (*add_links)(struct fwnode_handle *fwnode,
> > struct device *dev);
> > };
> >
> > --
> > 2.29.1.341.ge80a0c044ae-goog
> >

2020-11-21 02:03:57

by Saravana Kannan

[permalink] [raw]
Subject: Re: [PATCH v1 12/18] driver core: Add fw_devlink_parse_fwtree()

On Mon, Nov 16, 2020 at 8:25 AM Rafael J. Wysocki <[email protected]> wrote:
>
> On Thu, Nov 5, 2020 at 12:24 AM Saravana Kannan <[email protected]> wrote:
> >
> > This function is a wrapper around fwnode_operations.add_links().
> >
> > This function parses each node in a fwnode tree and create fwnode links
> > for each of those nodes. The information for creating the fwnode links
> > (the supplier and consumer fwnode) is obtained by parsing the properties
> > in each of the fwnodes.
> >
> > This function also ensures that no fwnode is parsed more than once by
> > marking the fwnodes as parsed.
> >
> > Signed-off-by: Saravana Kannan <[email protected]>
> > ---
> > drivers/base/core.c | 19 +++++++++++++++++++
> > include/linux/fwnode.h | 3 +++
> > 2 files changed, 22 insertions(+)
> >
> > diff --git a/drivers/base/core.c b/drivers/base/core.c
> > index 4a0907574646..ee28d8c7ee85 100644
> > --- a/drivers/base/core.c
> > +++ b/drivers/base/core.c
> > @@ -1543,6 +1543,25 @@ static bool fw_devlink_is_permissive(void)
> > return fw_devlink_flags == DL_FLAG_SYNC_STATE_ONLY;
> > }
> >
> > +static void fw_devlink_parse_fwnode(struct fwnode_handle *fwnode)
> > +{
> > + if (fwnode->flags & FWNODE_FLAG_LINKS_ADDED)
> > + return;
>
> Why is the flag needed?
>
> Duplicate links won't be created anyway and it doesn't cause the tree
> walk to be terminated.

To avoid parsing a fwnode more than once. The cumulative impact of the
repeated parsing is actually quite high.

And I intentionally didn't do this check at the tree walk level
because DT overlay can add/remove/change individual fwnodes and I want
to reparse those when they are added while avoiding parsing other
nodes that have already been parsed and not changed by DT overlay.

>
> > +
> > + fwnode_call_int_op(fwnode, add_links, NULL);
> > + fwnode->flags |= FWNODE_FLAG_LINKS_ADDED;
> > +}
> > +
> > +static void fw_devlink_parse_fwtree(struct fwnode_handle *fwnode)
> > +{
> > + struct fwnode_handle *child = NULL;
> > +
> > + fw_devlink_parse_fwnode(fwnode);
> > +
> > + while ((child = fwnode_get_next_available_child_node(fwnode, child)))
>
> I'd prefer
>
> for (child = NULL; child; child =
> fwnode_get_next_available_child_node(fwnode, child))

I was about to change to this and then realized it won't work. It
would have to be

for (child = fwnode_get_next_available_child_node(fwnode, NULL));
child;
child = fwnode_get_next_available_child_node(fwnode, child))

Is that really better? The while() seems a lot more readable to me. I
don't have a strong opinion, so I'll go with whatever you say after
reading this.

>
> > + fw_devlink_parse_fwtree(child);
> > +}
> > +
> > static void fw_devlink_link_device(struct device *dev)
> > {
> > int fw_ret;
> > diff --git a/include/linux/fwnode.h b/include/linux/fwnode.h
> > index ec02e1e939cc..9aaf9e4f3994 100644
> > --- a/include/linux/fwnode.h
> > +++ b/include/linux/fwnode.h
> > @@ -15,12 +15,15 @@
> > struct fwnode_operations;
> > struct device;
> >
>
> Description here, please.

Ack

>
> > +#define FWNODE_FLAG_LINKS_ADDED BIT(0)
> > +
> > struct fwnode_handle {
> > struct fwnode_handle *secondary;
> > const struct fwnode_operations *ops;
> > struct device *dev;
> > struct list_head suppliers;
> > struct list_head consumers;
> > + u32 flags;
>
> That's a bit wasteful. Maybe u8 would suffice for the time being?

Ack.


-Saravana

2020-11-21 02:04:10

by Saravana Kannan

[permalink] [raw]
Subject: Re: [PATCH v1 14/18] driver core: Use device's fwnode to check if it is waiting for suppliers

On Mon, Nov 16, 2020 at 8:34 AM Rafael J. Wysocki <[email protected]> wrote:
>
> On Thu, Nov 5, 2020 at 12:24 AM Saravana Kannan <[email protected]> wrote:
> >
> > To check if a device is still waiting for its supplier devices to be
> > added, we used to check if the devices is in a global
> > waiting_for_suppliers list. Since the global list will be deleted in
> > subsequent patches, this patch stops using this check.
>
> My kind of educated guess is that you want to drop
> waiting_for_suppliers and that's why you want to use supplier links
> here.

Yes, and a device would never be added waiting_for_suppliers list.

> >
> > Instead, this patch uses a more device specific check. It checks if the
> > device's fwnode has any fwnode links that haven't been converted to
> > device links yet.
> >
> > Signed-off-by: Saravana Kannan <[email protected]>
> > ---
> > drivers/base/core.c | 18 ++++++++----------
> > 1 file changed, 8 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/base/core.c b/drivers/base/core.c
> > index 4ae5f2885ac5..d51dd564add1 100644
> > --- a/drivers/base/core.c
> > +++ b/drivers/base/core.c
> > @@ -51,6 +51,7 @@ static DEFINE_MUTEX(wfs_lock);
> > static LIST_HEAD(deferred_sync);
> > static unsigned int defer_sync_state_count = 1;
> > static DEFINE_MUTEX(fwnode_link_lock);
> > +static bool fw_devlink_is_permissive(void);
> >
> > /**
> > * fwnode_link_add - Create a link between two fwnode_handles.
> > @@ -994,13 +995,13 @@ int device_links_check_suppliers(struct device *dev)
> > * Device waiting for supplier to become available is not allowed to
> > * probe.
> > */
> > - mutex_lock(&wfs_lock);
> > - if (!list_empty(&dev->links.needs_suppliers) &&
> > - dev->links.need_for_probe) {
> > - mutex_unlock(&wfs_lock);
> > + mutex_lock(&fwnode_link_lock);
> > + if (dev->fwnode && !list_empty(&dev->fwnode->suppliers) &&
> > + !fw_devlink_is_permissive()) {
> > + mutex_unlock(&fwnode_link_lock);
> > return -EPROBE_DEFER;
> > }
> > - mutex_unlock(&wfs_lock);
> > + mutex_unlock(&fwnode_link_lock);
> >
> > device_links_write_lock();
> >
> > @@ -1166,10 +1167,7 @@ static ssize_t waiting_for_supplier_show(struct device *dev,
> > bool val;
> >
> > device_lock(dev);
> > - mutex_lock(&wfs_lock);
> > - val = !list_empty(&dev->links.needs_suppliers)
> > - && dev->links.need_for_probe;
> > - mutex_unlock(&wfs_lock);
>
> Why isn't the lock needed any more?
>
> Or maybe it wasn't needed previously too?

Yeah, I sent a separate patch for dropping this lock [1]. But I didn't
want to wait for that to land to write this series. The lock wasn't
needed in the first place and it was causing a lockdep warning.

>
> > + val = !list_empty(&dev->fwnode->suppliers);
> > device_unlock(dev);
> > return sysfs_emit(buf, "%u\n", val);
> > }
> > @@ -2226,7 +2224,7 @@ static int device_add_attrs(struct device *dev)
> > goto err_remove_dev_groups;
> > }
> >
> > - if (fw_devlink_flags && !fw_devlink_is_permissive()) {
> > + if (fw_devlink_flags && !fw_devlink_is_permissive() && dev->fwnode) {
>
> And why is this change needed?

Because if a device doesn't have a fwnode, it can't ever be waiting on
a supplier. Also, the "show" function dereferences
dev->fwnode->suppliers.

-Saravana

[1] - https://lore.kernel.org/lkml/[email protected]/
Ignore the 1/2 thing. There's only 1 relevant patch.