2017-06-05 13:49:29

by Stefani Seibold

[permalink] [raw]
Subject: [PATCH] external references for device tree overlays

From: Stefani Seibold <[email protected]>

This patch enables external references for symbols which are not
exported by the current device tree. For example

// RASPI example (only for testing)
/dts-v1/;
/plugin/;

/ {
compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";

fragment@0 {
target-path = "/soc/i2s@7e203000";
__overlay__ {
#address-cells = <0x00000001>;
#size-cells = <0x00000001>;
test = "test";
timer = <&timer>;
};
};

__external_symbols__ {
timer = "/soc/timer@7e003000";
};
};

The "timer" symbol is not exported by the RASPI device tree, because it is
missing in the __symbols__ section of the device tree.

In case of the RASPI device tree this could be simple fixed by modifing
the device tree source, but when the device tree is provided by a closed
source BIOS this kind of missing symbol could not be fixed.

An additional benefit is to override a (possible broken) symbol exported
by the currect live device tree.

The patch is based and tested on linux 4.12-rc3.

Signed-off-by: Stefani Seibold <[email protected]>
Signed-off-by: Stefani Seibold <[email protected]>
---
drivers/of/overlay.c | 19 +++++++++++++++++++
drivers/of/resolver.c | 27 ++++++++++++++++++++++-----
2 files changed, 41 insertions(+), 5 deletions(-)

diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
index 7827786718d8..de6516ea0fcd 100644
--- a/drivers/of/overlay.c
+++ b/drivers/of/overlay.c
@@ -50,6 +50,7 @@ struct of_overlay {
int id;
struct list_head node;
int count;
+ struct device_node *tree;
struct of_overlay_info *ovinfo_tab;
struct of_changeset cset;
};
@@ -422,6 +423,8 @@ int of_overlay_create(struct device_node *tree)
/* add to the tail of the overlay list */
list_add_tail(&ov->node, &ov_list);

+ ov->tree = tree;
+
of_overlay_notify(ov, OF_OVERLAY_POST_APPLY);

mutex_unlock(&of_mutex);
@@ -524,6 +527,7 @@ int of_overlay_destroy(int id)
{
struct of_overlay *ov;
int err;
+ phandle phandle;

mutex_lock(&of_mutex);

@@ -540,6 +544,8 @@ int of_overlay_destroy(int id)
goto out;
}

+ phandle = ov->tree->phandle;
+
of_overlay_notify(ov, OF_OVERLAY_PRE_REMOVE);
list_del(&ov->node);
__of_changeset_revert(&ov->cset);
@@ -549,6 +555,19 @@ int of_overlay_destroy(int id)
of_changeset_destroy(&ov->cset);
kfree(ov);

+ if (phandle) {
+ struct device_node *node;
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&devtree_lock, flags);
+ for_each_of_allnodes(node) {
+ if (node->phandle >= phandle)
+ node->phandle = 0;
+ }
+ raw_spin_unlock_irqrestore(&devtree_lock, flags);
+ }
+
+
err = 0;

out:
diff --git a/drivers/of/resolver.c b/drivers/of/resolver.c
index 771f4844c781..31b5f32c9b27 100644
--- a/drivers/of/resolver.c
+++ b/drivers/of/resolver.c
@@ -286,13 +286,14 @@ static int adjust_local_phandle_references(struct device_node *local_fixups,
int of_resolve_phandles(struct device_node *overlay)
{
struct device_node *child, *local_fixups, *refnode;
- struct device_node *tree_symbols, *overlay_fixups;
+ struct device_node *tree_symbols, *ext_symbols, *overlay_fixups;
struct property *prop;
const char *refpath;
phandle phandle, phandle_delta;
int err;

tree_symbols = NULL;
+ ext_symbols = NULL;

if (!overlay) {
pr_err("null overlay\n");
@@ -321,6 +322,9 @@ int of_resolve_phandles(struct device_node *overlay)
for_each_child_of_node(overlay, child) {
if (!of_node_cmp(child->name, "__fixups__"))
overlay_fixups = child;
+ else
+ if (!of_node_cmp(child->name, "__external_symbols__"))
+ ext_symbols = child;
}

if (!overlay_fixups) {
@@ -329,20 +333,30 @@ int of_resolve_phandles(struct device_node *overlay)
}

tree_symbols = of_find_node_by_path("/__symbols__");
- if (!tree_symbols) {
- pr_err("no symbols in root of device tree.\n");
+ if (!tree_symbols && !ext_symbols) {
+ pr_err("no symbols for resolve in device tree.\n");
err = -EINVAL;
goto out;
}

+ phandle_delta = live_tree_max_phandle() + 1;
+
for_each_property_of_node(overlay_fixups, prop) {

/* skip properties added automatically */
if (!of_prop_cmp(prop->name, "name"))
continue;

- err = of_property_read_string(tree_symbols,
- prop->name, &refpath);
+ err = -1;
+
+ if (ext_symbols)
+ err = of_property_read_string(ext_symbols,
+ prop->name, &refpath);
+
+ if (err && tree_symbols)
+ err = of_property_read_string(tree_symbols,
+ prop->name, &refpath);
+
if (err)
goto out;

@@ -352,6 +366,9 @@ int of_resolve_phandles(struct device_node *overlay)
goto out;
}

+ if (!refnode->phandle)
+ refnode->phandle = ++phandle_delta;
+
phandle = refnode->phandle;
of_node_put(refnode);

--
2.13.0


2017-06-05 18:44:36

by Pantelis Antoniou

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

Hi Stefani,

On Mon, 2017-06-05 at 14:59 +0200, Stefani Seibold wrote:
> From: Stefani Seibold <[email protected]>
>
> This patch enables external references for symbols which are not
> exported by the current device tree. For example
>
> // RASPI example (only for testing)
> /dts-v1/;
> /plugin/;
>
> / {
> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
>
> fragment@0 {
> target-path = "/soc/i2s@7e203000";
> __overlay__ {
> #address-cells = <0x00000001>;
> #size-cells = <0x00000001>;
> test = "test";
> timer = <&timer>;
> };
> };
>
> __external_symbols__ {
> timer = "/soc/timer@7e003000";
> };
> };
>

I understand the problem. I am just not fond of the __external_symbols__
solution.

There's a facility in the DT source language that allows to declare
pathspec labels.

The 'timer = <&timer>;' statement could be rewritten as
'timer = <&{/soc/timer@7e0030000}>;'

Internally you can 'catch' that this refers to a symbol in the base tree
and then do the same symbol insertion as the patch you've submitted.

The benefit to the above is that you don't introduce manually edited
special nodes.

Regards

-- Pantelis

> The "timer" symbol is not exported by the RASPI device tree, because it is
> missing in the __symbols__ section of the device tree.
>
> In case of the RASPI device tree this could be simple fixed by modifing
> the device tree source, but when the device tree is provided by a closed
> source BIOS this kind of missing symbol could not be fixed.
>
> An additional benefit is to override a (possible broken) symbol exported
> by the currect live device tree.
>
> The patch is based and tested on linux 4.12-rc3.
>
> Signed-off-by: Stefani Seibold <[email protected]>
> Signed-off-by: Stefani Seibold <[email protected]>
> ---
> drivers/of/overlay.c | 19 +++++++++++++++++++
> drivers/of/resolver.c | 27 ++++++++++++++++++++++-----
> 2 files changed, 41 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
> index 7827786718d8..de6516ea0fcd 100644
> --- a/drivers/of/overlay.c
> +++ b/drivers/of/overlay.c
> @@ -50,6 +50,7 @@ struct of_overlay {
> int id;
> struct list_head node;
> int count;
> + struct device_node *tree;
> struct of_overlay_info *ovinfo_tab;
> struct of_changeset cset;
> };
> @@ -422,6 +423,8 @@ int of_overlay_create(struct device_node *tree)
> /* add to the tail of the overlay list */
> list_add_tail(&ov->node, &ov_list);
>
> + ov->tree = tree;
> +
> of_overlay_notify(ov, OF_OVERLAY_POST_APPLY);
>
> mutex_unlock(&of_mutex);
> @@ -524,6 +527,7 @@ int of_overlay_destroy(int id)
> {
> struct of_overlay *ov;
> int err;
> + phandle phandle;
>
> mutex_lock(&of_mutex);
>
> @@ -540,6 +544,8 @@ int of_overlay_destroy(int id)
> goto out;
> }
>
> + phandle = ov->tree->phandle;
> +
> of_overlay_notify(ov, OF_OVERLAY_PRE_REMOVE);
> list_del(&ov->node);
> __of_changeset_revert(&ov->cset);
> @@ -549,6 +555,19 @@ int of_overlay_destroy(int id)
> of_changeset_destroy(&ov->cset);
> kfree(ov);
>
> + if (phandle) {
> + struct device_node *node;
> + unsigned long flags;
> +
> + raw_spin_lock_irqsave(&devtree_lock, flags);
> + for_each_of_allnodes(node) {
> + if (node->phandle >= phandle)
> + node->phandle = 0;
> + }
> + raw_spin_unlock_irqrestore(&devtree_lock, flags);
> + }
> +
> +
> err = 0;
>
> out:
> diff --git a/drivers/of/resolver.c b/drivers/of/resolver.c
> index 771f4844c781..31b5f32c9b27 100644
> --- a/drivers/of/resolver.c
> +++ b/drivers/of/resolver.c
> @@ -286,13 +286,14 @@ static int adjust_local_phandle_references(struct device_node *local_fixups,
> int of_resolve_phandles(struct device_node *overlay)
> {
> struct device_node *child, *local_fixups, *refnode;
> - struct device_node *tree_symbols, *overlay_fixups;
> + struct device_node *tree_symbols, *ext_symbols, *overlay_fixups;
> struct property *prop;
> const char *refpath;
> phandle phandle, phandle_delta;
> int err;
>
> tree_symbols = NULL;
> + ext_symbols = NULL;
>
> if (!overlay) {
> pr_err("null overlay\n");
> @@ -321,6 +322,9 @@ int of_resolve_phandles(struct device_node *overlay)
> for_each_child_of_node(overlay, child) {
> if (!of_node_cmp(child->name, "__fixups__"))
> overlay_fixups = child;
> + else
> + if (!of_node_cmp(child->name, "__external_symbols__"))
> + ext_symbols = child;
> }
>
> if (!overlay_fixups) {
> @@ -329,20 +333,30 @@ int of_resolve_phandles(struct device_node *overlay)
> }
>
> tree_symbols = of_find_node_by_path("/__symbols__");
> - if (!tree_symbols) {
> - pr_err("no symbols in root of device tree.\n");
> + if (!tree_symbols && !ext_symbols) {
> + pr_err("no symbols for resolve in device tree.\n");
> err = -EINVAL;
> goto out;
> }
>
> + phandle_delta = live_tree_max_phandle() + 1;
> +
> for_each_property_of_node(overlay_fixups, prop) {
>
> /* skip properties added automatically */
> if (!of_prop_cmp(prop->name, "name"))
> continue;
>
> - err = of_property_read_string(tree_symbols,
> - prop->name, &refpath);
> + err = -1;
> +
> + if (ext_symbols)
> + err = of_property_read_string(ext_symbols,
> + prop->name, &refpath);
> +
> + if (err && tree_symbols)
> + err = of_property_read_string(tree_symbols,
> + prop->name, &refpath);
> +
> if (err)
> goto out;
>
> @@ -352,6 +366,9 @@ int of_resolve_phandles(struct device_node *overlay)
> goto out;
> }
>
> + if (!refnode->phandle)
> + refnode->phandle = ++phandle_delta;
> +
> phandle = refnode->phandle;
> of_node_put(refnode);
>


2017-06-06 07:20:26

by Frank Rowand

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

On 06/05/17 05:59, Stefani Seibold wrote:
> From: Stefani Seibold <[email protected]>
>
> This patch enables external references for symbols which are not
> exported by the current device tree. For example
>
> // RASPI example (only for testing)
> /dts-v1/;
> /plugin/;
>
> / {
> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
>
> fragment@0 {
> target-path = "/soc/i2s@7e203000";
> __overlay__ {
> #address-cells = <0x00000001>;
> #size-cells = <0x00000001>;
> test = "test";
> timer = <&timer>;
> };
> };
>
> __external_symbols__ {
> timer = "/soc/timer@7e003000";
> };
> };

My hope is that the dtc compiler will stop supporting specification of the
__symbols__ node in dts source, and only generate it automatically in the dtb.
That change to dtc would not allow any node name specified in a dts to begin
with an underscore. Thus node __external_symbols__ would not be allowed.


>
> The "timer" symbol is not exported by the RASPI device tree, because it is
> missing in the __symbols__ section of the device tree.
>
> In case of the RASPI device tree this could be simple fixed by modifing
> the device tree source, but when the device tree is provided by a closed
> source BIOS this kind of missing symbol could not be fixed.

Is there a real example of this issue, or is this a theoretical concern?
If this is a real example, we should be discouraging such behavior.

The suggestion by Pantelis should work, but that is just a hack to get
you out of a bad situation, not a good practice.

>
> An additional benefit is to override a (possible broken) symbol exported
> by the currect live device tree.
>
> The patch is based and tested on linux 4.12-rc3.
>
> Signed-off-by: Stefani Seibold <[email protected]>
> Signed-off-by: Stefani Seibold <[email protected]>
> ---
> drivers/of/overlay.c | 19 +++++++++++++++++++
> drivers/of/resolver.c | 27 ++++++++++++++++++++++-----
> 2 files changed, 41 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
> index 7827786718d8..de6516ea0fcd 100644
> --- a/drivers/of/overlay.c
> +++ b/drivers/of/overlay.c

< snip >

-Frank

2017-06-06 16:12:28

by Frank Rowand

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

On 06/06/17 00:20, Frank Rowand wrote:
> On 06/05/17 05:59, Stefani Seibold wrote:
>> From: Stefani Seibold <[email protected]>
>>
>> This patch enables external references for symbols which are not
>> exported by the current device tree. For example
>>
>> // RASPI example (only for testing)
>> /dts-v1/;
>> /plugin/;
>>
>> / {
>> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
>>
>> fragment@0 {
>> target-path = "/soc/i2s@7e203000";
>> __overlay__ {
>> #address-cells = <0x00000001>;
>> #size-cells = <0x00000001>;
>> test = "test";
>> timer = <&timer>;
>> };
>> };
>>
>> __external_symbols__ {
>> timer = "/soc/timer@7e003000";
>> };
>> };
>
> My hope is that the dtc compiler will stop supporting specification of the
> __symbols__ node in dts source, and only generate it automatically in the dtb.
> That change to dtc would not allow any node name specified in a dts to begin
> with an underscore. Thus node __external_symbols__ would not be allowed.
>
>
>>
>> The "timer" symbol is not exported by the RASPI device tree, because it is
>> missing in the __symbols__ section of the device tree.
>>
>> In case of the RASPI device tree this could be simple fixed by modifing
>> the device tree source, but when the device tree is provided by a closed
>> source BIOS this kind of missing symbol could not be fixed.
>
> Is there a real example of this issue, or is this a theoretical concern?
> If this is a real example, we should be discouraging such behavior.
>
> The suggestion by Pantelis should work, but that is just a hack to get
> you out of a bad situation, not a good practice.

Digging a little bit deeper into the suggestion by Pantelis, I don't like
that approach either. It is still rather hacky.

>
>>
>> An additional benefit is to override a (possible broken) symbol exported
>> by the currect live device tree.
>>
>> The patch is based and tested on linux 4.12-rc3.
>>
>> Signed-off-by: Stefani Seibold <[email protected]>
>> Signed-off-by: Stefani Seibold <[email protected]>
>> ---
>> drivers/of/overlay.c | 19 +++++++++++++++++++
>> drivers/of/resolver.c | 27 ++++++++++++++++++++++-----
>> 2 files changed, 41 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
>> index 7827786718d8..de6516ea0fcd 100644
>> --- a/drivers/of/overlay.c
>> +++ b/drivers/of/overlay.c
>
> < snip >
>
> -Frank
>

2017-06-06 19:24:04

by Stefani Seibold

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

Hi Frank,

On 06.06.2017, 00:20 -0700 Frank Rowand wrote::
> On 06/05/17 05:59, Stefani Seibold wrote:
> > From: Stefani Seibold <[email protected]>
> >
> > This patch enables external references for symbols which are not
> > exported by the current device tree. For example
> >
> > // RASPI example (only for testing)
> > /dts-v1/;
> > /plugin/;
> >
> > / {
> > compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
> >
> > fragment@0 {
> > target-path = "/soc/i2s@7e203000";
> > __overlay__ {
> > #address-cells = <0x00000001>;
> > #size-cells = <0x00000001>;
> > test = "test";
> > timer = <&timer>;
> > };
> > };
> >
> > __external_symbols__ {
> > timer = "/soc/timer@7e003000";
> > };
> > };
>
> My hope is that the dtc compiler will stop supporting specification
> of the
> __symbols__ node in dts source, and only generate it automatically in
> the dtb.
> That change to dtc would not allow any node name specified in a dts
> to begin
> with an underscore. Thus node __external_symbols__ would not be
> allowed.
>

The name is not so important to me, only the solution.

> > In case of the RASPI device tree this could be simple fixed by
> > modifing
> > the device tree source, but when the device tree is provided by a
> > closed
> > source BIOS this kind of missing symbol could not be fixed.
>
> Is there a real example of this issue, or is this a theoretical
> concern?
> If this is a real example, we should be discouraging such behavior.
>

Yes, I have a BIOS on some ARM64 servers which provides broken device tree. It also lacks some devices in this tree which needs references to other devices which lacks a phandle.


> The suggestion by Pantelis should work, but that is just a hack to
> get
> you out of a bad situation, not a good practice.
>

I tried it, but it doesn't work. Look at my post to Pantelis.

- Stefani

2017-06-06 19:35:58

by Stefani Seibold

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

Hi Pantelis,

thanks for the suggestion. This feature is not very well documented. I
tried this on my rasp1 running 4.12.0-rc3 and it doesn't work. My
source is:

// rapsi example
/dts-v1/;
/plugin/;

/ {
    compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";

    fragment@0 {
        target-path = "/soc/i2s@7e203000";
        __overlay__ {
            #address-cells = <0x00000001>;
            #size-cells = <0x00000001>;
            test = "test";
            timer = <&{/soc/timer@7e0030000}>;
        };
    };
};


The resulting overlay is (decompiled with fdtdump):

/dts-v1/;
// magic: 0xd00dfeed
// totalsize: 0x19a (410)
// off_dt_struct: 0x38
// off_dt_strings: 0x148
// off_mem_rsvmap: 0x28
// version: 17
// last_comp_version: 16
// boot_cpuid_phys: 0x0
// size_dt_strings: 0x52
// size_dt_struct: 0x110

/ {
    compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
    fragment@0 {
        target-path = "/soc/i2s@7e203000";
        __overlay__ {
            #address-cells = <0x00000001>;
            #size-cells = <0x00000001>;
            test = "test";
            timer = <0xdeadbeef>;
        };
    };
    __fixups__ {
        /soc/timer@7e0030000 = "/fragment@0/__overlay__:timer:0";
    };
};

But this will not apply:

OF: resolver: overlay phandle fixup failed: -22
create_overlay: Failed to resolve tree


Anyway, the reason for my patch is that i can reference to nodes which
lacks a phandle. The phandle will be created on the fly and also
destroyed when the overlay is unloaded.

I have a real use case for this patch:

I have a BIOS on some ARM64 servers which provides broken device tree.
It also lacks some devices in this tree which needs references to other
devices which lacks a phandle.

Since the BIOSes are closed source i need a way to work arround this
problem without patching all the drivers involved to this devices.

Hope this helps to understand the reason for this patch.

- Stefani

Am Montag, den 05.06.2017, 21:43 +0300 schrieb Pantelis Antoniou:
> Hi Stefani,
>
> On Mon, 2017-06-05 at 14:59 +0200, Stefani Seibold wrote:
> > From: Stefani Seibold <[email protected]>
> >
> > This patch enables external references for symbols which are not
> > exported by the current device tree. For example
> >
> > // RASPI example (only for testing)
> > /dts-v1/;
> > /plugin/;
> >
> > / {
> >     compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
> >
> >     fragment@0 {
> >         target-path = "/soc/i2s@7e203000";
> >         __overlay__ {
> >             #address-cells = <0x00000001>;
> >             #size-cells = <0x00000001>;
> >             test = "test";
> >             timer = <&timer>;
> >         };
> >     };
> >
> >     __external_symbols__ {
> >         timer = "/soc/timer@7e003000";
> >     };
> > };
> >
>
> I understand the problem. I am just not fond of the
> __external_symbols__
> solution.
>
> There's a facility in the DT source language that allows to declare
> pathspec labels.
>
> The 'timer = <&timer>;' statement could be rewritten as 
> 'timer = <&{/soc/timer@7e0030000}>;'
>
> Internally you can 'catch' that this refers to a symbol in the base
> tree
> and then do the same symbol insertion as the patch you've submitted.
>
> The benefit to the above is that you don't introduce manually edited
> special nodes.
>
> Regards
>
> -- Pantelis
>
> > The "timer" symbol is not exported by the RASPI device tree,
> > because it is
> > missing in the __symbols__ section of the device tree.
> >
> > In case of the RASPI device tree this could be simple fixed by
> > modifing
> > the device tree source, but when the device tree is provided by a
> > closed
> > source BIOS this kind of missing symbol could not be fixed.
> >
> > An additional benefit is to override a (possible broken) symbol
> > exported
> > by the currect live device tree.
> >
> > The patch is based and tested on linux 4.12-rc3.
> >
> > Signed-off-by: Stefani Seibold <[email protected]>
> > Signed-off-by: Stefani Seibold <[email protected]>
> > ---
> >  drivers/of/overlay.c  | 19 +++++++++++++++++++
> >  drivers/of/resolver.c | 27 ++++++++++++++++++++++-----
> >  2 files changed, 41 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
> > index 7827786718d8..de6516ea0fcd 100644
> > --- a/drivers/of/overlay.c
> > +++ b/drivers/of/overlay.c
> > @@ -50,6 +50,7 @@ struct of_overlay {
> >   int id;
> >   struct list_head node;
> >   int count;
> > + struct device_node *tree;
> >   struct of_overlay_info *ovinfo_tab;
> >   struct of_changeset cset;
> >  };
> > @@ -422,6 +423,8 @@ int of_overlay_create(struct device_node *tree)
> >   /* add to the tail of the overlay list */
> >   list_add_tail(&ov->node, &ov_list);
> >  
> > + ov->tree = tree;
> > +
> >   of_overlay_notify(ov, OF_OVERLAY_POST_APPLY);
> >  
> >   mutex_unlock(&of_mutex);
> > @@ -524,6 +527,7 @@ int of_overlay_destroy(int id)
> >  {
> >   struct of_overlay *ov;
> >   int err;
> > + phandle phandle;
> >  
> >   mutex_lock(&of_mutex);
> >  
> > @@ -540,6 +544,8 @@ int of_overlay_destroy(int id)
> >   goto out;
> >   }
> >  
> > + phandle = ov->tree->phandle;
> > +
> >   of_overlay_notify(ov, OF_OVERLAY_PRE_REMOVE);
> >   list_del(&ov->node);
> >   __of_changeset_revert(&ov->cset);
> > @@ -549,6 +555,19 @@ int of_overlay_destroy(int id)
> >   of_changeset_destroy(&ov->cset);
> >   kfree(ov);
> >  
> > + if (phandle) {
> > + struct device_node *node;
> > + unsigned long flags;
> > +
> > + raw_spin_lock_irqsave(&devtree_lock, flags);
> > + for_each_of_allnodes(node) {
> > + if (node->phandle >= phandle)
> > + node->phandle = 0;
> > + }
> > + raw_spin_unlock_irqrestore(&devtree_lock, flags);
> > + }
> > +
> > +
> >   err = 0;
> >  
> >  out:
> > diff --git a/drivers/of/resolver.c b/drivers/of/resolver.c
> > index 771f4844c781..31b5f32c9b27 100644
> > --- a/drivers/of/resolver.c
> > +++ b/drivers/of/resolver.c
> > @@ -286,13 +286,14 @@ static int
> > adjust_local_phandle_references(struct device_node *local_fixups,
> >  int of_resolve_phandles(struct device_node *overlay)
> >  {
> >   struct device_node *child, *local_fixups, *refnode;
> > - struct device_node *tree_symbols, *overlay_fixups;
> > + struct device_node *tree_symbols, *ext_symbols,
> > *overlay_fixups;
> >   struct property *prop;
> >   const char *refpath;
> >   phandle phandle, phandle_delta;
> >   int err;
> >  
> >   tree_symbols = NULL;
> > + ext_symbols = NULL;
> >  
> >   if (!overlay) {
> >   pr_err("null overlay\n");
> > @@ -321,6 +322,9 @@ int of_resolve_phandles(struct device_node
> > *overlay)
> >   for_each_child_of_node(overlay, child) {
> >   if (!of_node_cmp(child->name, "__fixups__"))
> >   overlay_fixups = child;
> > + else
> > + if (!of_node_cmp(child->name,
> > "__external_symbols__"))
> > + ext_symbols = child;
> >   }
> >  
> >   if (!overlay_fixups) {
> > @@ -329,20 +333,30 @@ int of_resolve_phandles(struct device_node
> > *overlay)
> >   }
> >  
> >   tree_symbols = of_find_node_by_path("/__symbols__");
> > - if (!tree_symbols) {
> > - pr_err("no symbols in root of device tree.\n");
> > + if (!tree_symbols && !ext_symbols) {
> > + pr_err("no symbols for resolve in device
> > tree.\n");
> >   err = -EINVAL;
> >   goto out;
> >   }
> >  
> > + phandle_delta = live_tree_max_phandle() + 1;
> > +
> >   for_each_property_of_node(overlay_fixups, prop) {
> >  
> >   /* skip properties added automatically */
> >   if (!of_prop_cmp(prop->name, "name"))
> >   continue;
> >  
> > - err = of_property_read_string(tree_symbols,
> > - prop->name, &refpath);
> > + err = -1;
> > +
> > + if (ext_symbols)
> > + err = of_property_read_string(ext_symbols,
> > + prop->name, &refpath);
> > +
> > + if (err && tree_symbols)
> > + err =
> > of_property_read_string(tree_symbols,
> > + prop->name, &refpath);
> > +
> >   if (err)
> >   goto out;
> >  
> > @@ -352,6 +366,9 @@ int of_resolve_phandles(struct device_node
> > *overlay)
> >   goto out;
> >   }
> >  
> > + if (!refnode->phandle)
> > + refnode->phandle = ++phandle_delta;
> > +
> >   phandle = refnode->phandle;
> >   of_node_put(refnode);
> >  
>
>

2017-06-06 22:05:42

by Rob Herring

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

On Tue, Jun 6, 2017 at 2:17 PM, Stefani Seibold <[email protected]> wrote:
> Hi Pantelis,
>
> thanks for the suggestion. This feature is not very well documented. I
> tried this on my rasp1 running 4.12.0-rc3 and it doesn't work. My
> source is:
>
> // rapsi example
> /dts-v1/;
> /plugin/;
>
> / {
> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
>
> fragment@0 {
> target-path = "/soc/i2s@7e203000";
> __overlay__ {
> #address-cells = <0x00000001>;
> #size-cells = <0x00000001>;
> test = "test";
> timer = <&{/soc/timer@7e0030000}>;
> };
> };
> };
>
>
> The resulting overlay is (decompiled with fdtdump):
>
> /dts-v1/;
> // magic: 0xd00dfeed
> // totalsize: 0x19a (410)
> // off_dt_struct: 0x38
> // off_dt_strings: 0x148
> // off_mem_rsvmap: 0x28
> // version: 17
> // last_comp_version: 16
> // boot_cpuid_phys: 0x0
> // size_dt_strings: 0x52
> // size_dt_struct: 0x110
>
> / {
> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
> fragment@0 {
> target-path = "/soc/i2s@7e203000";
> __overlay__ {
> #address-cells = <0x00000001>;
> #size-cells = <0x00000001>;
> test = "test";
> timer = <0xdeadbeef>;
> };
> };
> __fixups__ {
> /soc/timer@7e0030000 = "/fragment@0/__overlay__:timer:0";

Looks like you (Pantelis had the typo) have an extra 0 in 7e0030000
compared to your original example.

Rob

2017-06-07 00:46:39

by Frank Rowand

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

On 06/06/17 12:22, Stefani Seibold wrote:
> Hi Frank,
>
> On 06.06.2017, 00:20 -0700 Frank Rowand wrote::
>> On 06/05/17 05:59, Stefani Seibold wrote:
>>> From: Stefani Seibold <[email protected]>
>>>
>>> This patch enables external references for symbols which are not
>>> exported by the current device tree. For example
>>>
>>> // RASPI example (only for testing)
>>> /dts-v1/;
>>> /plugin/;
>>>
>>> / {
>>> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
>>>
>>> fragment@0 {
>>> target-path = "/soc/i2s@7e203000";
>>> __overlay__ {
>>> #address-cells = <0x00000001>;
>>> #size-cells = <0x00000001>;
>>> test = "test";
>>> timer = <&timer>;
>>> };
>>> };
>>>
>>> __external_symbols__ {
>>> timer = "/soc/timer@7e003000";
>>> };
>>> };
>>
>> My hope is that the dtc compiler will stop supporting specification
>> of the
>> __symbols__ node in dts source, and only generate it automatically in
>> the dtb.
>> That change to dtc would not allow any node name specified in a dts
>> to begin
>> with an underscore. Thus node __external_symbols__ would not be
>> allowed.
>>
>
> The name is not so important to me, only the solution.
>
>>> In case of the RASPI device tree this could be simple fixed by
>>> modifing
>>> the device tree source, but when the device tree is provided by a
>>> closed
>>> source BIOS this kind of missing symbol could not be fixed.
>>
>> Is there a real example of this issue, or is this a theoretical
>> concern?
>> If this is a real example, we should be discouraging such behavior.
>>
>
> Yes, I have a BIOS on some ARM64 servers which provides broken device
> tree. It also lacks some devices in this tree which needs references
> to other devices which lacks a phandle.

Jon Masters is pushing a message that if the firmware on your arm64 server
is broken, then insist that the vendor fix it. I think he was talking
about ACPI, but the same message should also apply to device tree.

If you are having trouble getting your vendor to fix it, ask Jon if he
is willing to help apply pressure.


>> The suggestion by Pantelis should work, but that is just a hack to
>> get
>> you out of a bad situation, not a good practice.
>>
>
> I tried it, but it doesn't work. Look at my post to Pantelis.

Yes, I realized that the method Pantelis gave would also require
a code change. I don't like that code change either.


>
> - Stefani
> .
>

2017-06-07 08:13:05

by Pantelis Antoniou

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

Hi Stefani,

On Tue, 2017-06-06 at 21:17 +0200, Stefani Seibold wrote:
> Hi Pantelis,
>
> thanks for the suggestion. This feature is not very well documented. I
> tried this on my rasp1 running 4.12.0-rc3 and it doesn't work. My
> source is:
>
> // rapsi example
> /dts-v1/;
> /plugin/;
>
> / {
> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
>
> fragment@0 {
> target-path = "/soc/i2s@7e203000";
> __overlay__ {
> #address-cells = <0x00000001>;
> #size-cells = <0x00000001>;
> test = "test";
> timer = <&{/soc/timer@7e0030000}>;
> };
> };
> };
>
>
> The resulting overlay is (decompiled with fdtdump):
>
> /dts-v1/;
> // magic: 0xd00dfeed
> // totalsize: 0x19a (410)
> // off_dt_struct: 0x38
> // off_dt_strings: 0x148
> // off_mem_rsvmap: 0x28
> // version: 17
> // last_comp_version: 16
> // boot_cpuid_phys: 0x0
> // size_dt_strings: 0x52
> // size_dt_struct: 0x110
>
> / {
> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
> fragment@0 {
> target-path = "/soc/i2s@7e203000";
> __overlay__ {
> #address-cells = <0x00000001>;
> #size-cells = <0x00000001>;
> test = "test";
> timer = <0xdeadbeef>;
> };
> };
> __fixups__ {
> /soc/timer@7e0030000 = "/fragment@0/__overlay__:timer:0";
> };
> };
>
> But this will not apply:
>
> OF: resolver: overlay phandle fixup failed: -22
> create_overlay: Failed to resolve tree
>
>

Yes, it will not work as it is; my point is that you don't need the
magic __*__ node.

You will need to modify the overlay application code to live insert a
phandle (if it doesn't exist) when it encounters a /path fixup.

> Anyway, the reason for my patch is that i can reference to nodes which
> lacks a phandle. The phandle will be created on the fly and also
> destroyed when the overlay is unloaded.
>
> I have a real use case for this patch:
>
> I have a BIOS on some ARM64 servers which provides broken device tree.
> It also lacks some devices in this tree which needs references to other
> devices which lacks a phandle.
>
> Since the BIOSes are closed source i need a way to work arround this
> problem without patching all the drivers involved to this devices.
>
> Hope this helps to understand the reason for this patch.
>

FWIW your problem seems like something that would happen on the field.
We can berate the vendor of not providing the correct device tree, but
in the end workarounds for broken vendor things are common in the
kernel.

Regards

-- Pantelis

> - Stefani
>
> Am Montag, den 05.06.2017, 21:43 +0300 schrieb Pantelis Antoniou:
> > Hi Stefani,
> >
> > On Mon, 2017-06-05 at 14:59 +0200, Stefani Seibold wrote:
> > > From: Stefani Seibold <[email protected]>
> > >
> > > This patch enables external references for symbols which are not
> > > exported by the current device tree. For example
> > >
> > > // RASPI example (only for testing)
> > > /dts-v1/;
> > > /plugin/;
> > >
> > > / {
> > > compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
> > >
> > > fragment@0 {
> > > target-path = "/soc/i2s@7e203000";
> > > __overlay__ {
> > > #address-cells = <0x00000001>;
> > > #size-cells = <0x00000001>;
> > > test = "test";
> > > timer = <&timer>;
> > > };
> > > };
> > >
> > > __external_symbols__ {
> > > timer = "/soc/timer@7e003000";
> > > };
> > > };
> > >
> >
> > I understand the problem. I am just not fond of the
> > __external_symbols__
> > solution.
> >
> > There's a facility in the DT source language that allows to declare
> > pathspec labels.
> >
> > The 'timer = <&timer>;' statement could be rewritten as
> > 'timer = <&{/soc/timer@7e0030000}>;'
> >
> > Internally you can 'catch' that this refers to a symbol in the base
> > tree
> > and then do the same symbol insertion as the patch you've submitted.
> >
> > The benefit to the above is that you don't introduce manually edited
> > special nodes.
> >
> > Regards
> >
> > -- Pantelis
> >
> > > The "timer" symbol is not exported by the RASPI device tree,
> > > because it is
> > > missing in the __symbols__ section of the device tree.
> > >
> > > In case of the RASPI device tree this could be simple fixed by
> > > modifing
> > > the device tree source, but when the device tree is provided by a
> > > closed
> > > source BIOS this kind of missing symbol could not be fixed.
> > >
> > > An additional benefit is to override a (possible broken) symbol
> > > exported
> > > by the currect live device tree.
> > >
> > > The patch is based and tested on linux 4.12-rc3.
> > >
> > > Signed-off-by: Stefani Seibold <[email protected]>
> > > Signed-off-by: Stefani Seibold <[email protected]>
> > > ---
> > > drivers/of/overlay.c | 19 +++++++++++++++++++
> > > drivers/of/resolver.c | 27 ++++++++++++++++++++++-----
> > > 2 files changed, 41 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
> > > index 7827786718d8..de6516ea0fcd 100644
> > > --- a/drivers/of/overlay.c
> > > +++ b/drivers/of/overlay.c
> > > @@ -50,6 +50,7 @@ struct of_overlay {
> > > int id;
> > > struct list_head node;
> > > int count;
> > > + struct device_node *tree;
> > > struct of_overlay_info *ovinfo_tab;
> > > struct of_changeset cset;
> > > };
> > > @@ -422,6 +423,8 @@ int of_overlay_create(struct device_node *tree)
> > > /* add to the tail of the overlay list */
> > > list_add_tail(&ov->node, &ov_list);
> > >
> > > + ov->tree = tree;
> > > +
> > > of_overlay_notify(ov, OF_OVERLAY_POST_APPLY);
> > >
> > > mutex_unlock(&of_mutex);
> > > @@ -524,6 +527,7 @@ int of_overlay_destroy(int id)
> > > {
> > > struct of_overlay *ov;
> > > int err;
> > > + phandle phandle;
> > >
> > > mutex_lock(&of_mutex);
> > >
> > > @@ -540,6 +544,8 @@ int of_overlay_destroy(int id)
> > > goto out;
> > > }
> > >
> > > + phandle = ov->tree->phandle;
> > > +
> > > of_overlay_notify(ov, OF_OVERLAY_PRE_REMOVE);
> > > list_del(&ov->node);
> > > __of_changeset_revert(&ov->cset);
> > > @@ -549,6 +555,19 @@ int of_overlay_destroy(int id)
> > > of_changeset_destroy(&ov->cset);
> > > kfree(ov);
> > >
> > > + if (phandle) {
> > > + struct device_node *node;
> > > + unsigned long flags;
> > > +
> > > + raw_spin_lock_irqsave(&devtree_lock, flags);
> > > + for_each_of_allnodes(node) {
> > > + if (node->phandle >= phandle)
> > > + node->phandle = 0;
> > > + }
> > > + raw_spin_unlock_irqrestore(&devtree_lock, flags);
> > > + }
> > > +
> > > +
> > > err = 0;
> > >
> > > out:
> > > diff --git a/drivers/of/resolver.c b/drivers/of/resolver.c
> > > index 771f4844c781..31b5f32c9b27 100644
> > > --- a/drivers/of/resolver.c
> > > +++ b/drivers/of/resolver.c
> > > @@ -286,13 +286,14 @@ static int
> > > adjust_local_phandle_references(struct device_node *local_fixups,
> > > int of_resolve_phandles(struct device_node *overlay)
> > > {
> > > struct device_node *child, *local_fixups, *refnode;
> > > - struct device_node *tree_symbols, *overlay_fixups;
> > > + struct device_node *tree_symbols, *ext_symbols,
> > > *overlay_fixups;
> > > struct property *prop;
> > > const char *refpath;
> > > phandle phandle, phandle_delta;
> > > int err;
> > >
> > > tree_symbols = NULL;
> > > + ext_symbols = NULL;
> > >
> > > if (!overlay) {
> > > pr_err("null overlay\n");
> > > @@ -321,6 +322,9 @@ int of_resolve_phandles(struct device_node
> > > *overlay)
> > > for_each_child_of_node(overlay, child) {
> > > if (!of_node_cmp(child->name, "__fixups__"))
> > > overlay_fixups = child;
> > > + else
> > > + if (!of_node_cmp(child->name,
> > > "__external_symbols__"))
> > > + ext_symbols = child;
> > > }
> > >
> > > if (!overlay_fixups) {
> > > @@ -329,20 +333,30 @@ int of_resolve_phandles(struct device_node
> > > *overlay)
> > > }
> > >
> > > tree_symbols = of_find_node_by_path("/__symbols__");
> > > - if (!tree_symbols) {
> > > - pr_err("no symbols in root of device tree.\n");
> > > + if (!tree_symbols && !ext_symbols) {
> > > + pr_err("no symbols for resolve in device
> > > tree.\n");
> > > err = -EINVAL;
> > > goto out;
> > > }
> > >
> > > + phandle_delta = live_tree_max_phandle() + 1;
> > > +
> > > for_each_property_of_node(overlay_fixups, prop) {
> > >
> > > /* skip properties added automatically */
> > > if (!of_prop_cmp(prop->name, "name"))
> > > continue;
> > >
> > > - err = of_property_read_string(tree_symbols,
> > > - prop->name, &refpath);
> > > + err = -1;
> > > +
> > > + if (ext_symbols)
> > > + err = of_property_read_string(ext_symbols,
> > > + prop->name, &refpath);
> > > +
> > > + if (err && tree_symbols)
> > > + err =
> > > of_property_read_string(tree_symbols,
> > > + prop->name, &refpath);
> > > +
> > > if (err)
> > > goto out;
> > >
> > > @@ -352,6 +366,9 @@ int of_resolve_phandles(struct device_node
> > > *overlay)
> > > goto out;
> > > }
> > >
> > > + if (!refnode->phandle)
> > > + refnode->phandle = ++phandle_delta;
> > > +
> > > phandle = refnode->phandle;
> > > of_node_put(refnode);
> > >
> >
> >


2017-06-07 22:19:57

by Rob Herring

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

On Wed, Jun 7, 2017 at 3:11 AM, Pantelis Antoniou
<[email protected]> wrote:
> Hi Stefani,
>
> On Tue, 2017-06-06 at 21:17 +0200, Stefani Seibold wrote:
>> Hi Pantelis,
>>
>> thanks for the suggestion. This feature is not very well documented. I
>> tried this on my rasp1 running 4.12.0-rc3 and it doesn't work. My
>> source is:
>>
>> // rapsi example
>> /dts-v1/;
>> /plugin/;
>>
>> / {
>> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
>>
>> fragment@0 {
>> target-path = "/soc/i2s@7e203000";
>> __overlay__ {
>> #address-cells = <0x00000001>;
>> #size-cells = <0x00000001>;
>> test = "test";
>> timer = <&{/soc/timer@7e0030000}>;
>> };
>> };
>> };
>>
>>
>> The resulting overlay is (decompiled with fdtdump):
>>
>> /dts-v1/;
>> // magic: 0xd00dfeed
>> // totalsize: 0x19a (410)
>> // off_dt_struct: 0x38
>> // off_dt_strings: 0x148
>> // off_mem_rsvmap: 0x28
>> // version: 17
>> // last_comp_version: 16
>> // boot_cpuid_phys: 0x0
>> // size_dt_strings: 0x52
>> // size_dt_struct: 0x110
>>
>> / {
>> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
>> fragment@0 {
>> target-path = "/soc/i2s@7e203000";
>> __overlay__ {
>> #address-cells = <0x00000001>;
>> #size-cells = <0x00000001>;
>> test = "test";
>> timer = <0xdeadbeef>;
>> };
>> };
>> __fixups__ {
>> /soc/timer@7e0030000 = "/fragment@0/__overlay__:timer:0";
>> };
>> };
>>
>> But this will not apply:
>>
>> OF: resolver: overlay phandle fixup failed: -22
>> create_overlay: Failed to resolve tree
>>
>>
>
> Yes, it will not work as it is; my point is that you don't need the
> magic __*__ node.
>
> You will need to modify the overlay application code to live insert a
> phandle (if it doesn't exist) when it encounters a /path fixup.

phandles only exist if something in the base tree refers to that node.
Adding them when they don't exist should definitely be something we
support for overlays. But don't call that a broken DT. That would be a
separate issue.

Rob

2017-06-08 06:48:37

by Stefani Seibold

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

Hi Pantelis,

On Wed, 2017-06-07 at 11:11 +0300, Pantelis Antoniou wrote:
> Hi Stefani,
>
> On Tue, 2017-06-06 at 21:17 +0200, Stefani Seibold wrote:
> > Hi Pantelis,
> >
> > thanks for the suggestion. This feature is not very well
> > documented. I
> > tried this on my rasp1 running 4.12.0-rc3 and it doesn't work. My
> > source is:
> >
> > // rapsi example
> > /dts-v1/;
> > /plugin/;
> >
> > / {
> >     compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
> >
> >     fragment@0 {
> >         target-path = "/soc/i2s@7e203000";
> >         __overlay__ {
> >             #address-cells = <0x00000001>;
> >             #size-cells = <0x00000001>;
> >             test = "test";
> >             timer = <&{/soc/timer@7e0030000}>;
> >         };
> >     };
> > };
> >
> >
> > The resulting overlay is (decompiled with fdtdump):
> >
> > /dts-v1/;
> > // magic: 0xd00dfeed
> > // totalsize: 0x19a (410)
> > // off_dt_struct: 0x38
> > // off_dt_strings: 0x148
> > // off_mem_rsvmap: 0x28
> > // version: 17
> > // last_comp_version: 16
> > // boot_cpuid_phys: 0x0
> > // size_dt_strings: 0x52
> > // size_dt_struct: 0x110
> >
> > / {
> >     compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
> >     fragment@0 {
> >         target-path = "/soc/i2s@7e203000";
> >         __overlay__ {
> >             #address-cells = <0x00000001>;
> >             #size-cells = <0x00000001>;
> >             test = "test";
> >             timer = <0xdeadbeef>;
> >         };
> >     };
> >     __fixups__ {
> >         /soc/timer@7e0030000 = "/fragment@0/__overlay__:timer:0";
> >     };
> > };
> >
> > But this will not apply:
> >
> > OF: resolver: overlay phandle fixup failed: -22
> > create_overlay: Failed to resolve tree
> >
> >
>
> Yes, it will not work as it is; my point is that you don't need the
> magic __*__ node.
>

The magic __fixups__ node was inserted by the device tree compiler. I
use the dtc from https://github.com/pantoniou/dtc at commit
d990b8013889b816ec054c7e07a77db59c56c400.

> You will need to modify the overlay application code to live insert a
> phandle (if it doesn't exist) when it encounters a /path fixup.
>

That is part of my patch!

> > Anyway, the reason for my patch is that i can reference to nodes
> > which
> > lacks a phandle. The phandle will be created on the fly and also
> > destroyed when the overlay is unloaded.
> >
> > I have a real use case for this patch:
> >
> > I have a BIOS on some ARM64 servers which provides broken device
> > tree.
> > It also lacks some devices in this tree which needs references to
> > other
> > devices which lacks a phandle.
> >
> > Since the BIOSes are closed source i need a way to work arround
> > this
> > problem without patching all the drivers involved to this devices.
> >
> > Hope this helps to understand the reason for this patch.
> >
>
> FWIW your problem seems like something that would happen on the
> field.
> We can berate the vendor of not providing the correct device tree,
> but
> in the end workarounds for broken vendor things are common in the
> kernel.
>

Yes, that is the way how linux do the things. Linux has a long history
to bypassing bugs of BIOSes, ACPI or broken devices.

Greetings,
Stefani

2017-06-08 06:51:20

by Stefani Seibold

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

On Wed, 2017-06-07 at 17:19 -0500, Rob Herring wrote:
> On Wed, Jun 7, 2017 at 3:11 AM, Pantelis Antoniou
> <[email protected]> wrote:
> > Hi Stefani,
> >
> > On Tue, 2017-06-06 at 21:17 +0200, Stefani Seibold wrote:
> > > Hi Pantelis,
> > >
> > > thanks for the suggestion. This feature is not very well
> > > documented. I
> > > tried this on my rasp1 running 4.12.0-rc3 and it doesn't work. My
> > > source is:
> > >
> > > // rapsi example
> > > /dts-v1/;
> > > /plugin/;
> > >
> > > / {
> > >     compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
> > >
> > >     fragment@0 {
> > >         target-path = "/soc/i2s@7e203000";
> > >         __overlay__ {
> > >             #address-cells = <0x00000001>;
> > >             #size-cells = <0x00000001>;
> > >             test = "test";
> > >             timer = <&{/soc/timer@7e0030000}>;
> > >         };
> > >     };
> > > };
> > >
> > >
> > > The resulting overlay is (decompiled with fdtdump):
> > >
> > > /dts-v1/;
> > > // magic:             0xd00dfeed
> > > // totalsize:         0x19a (410)
> > > // off_dt_struct:     0x38
> > > // off_dt_strings:    0x148
> > > // off_mem_rsvmap:    0x28
> > > // version:           17
> > > // last_comp_version: 16
> > > // boot_cpuid_phys:   0x0
> > > // size_dt_strings:   0x52
> > > // size_dt_struct:    0x110
> > >
> > > / {
> > >     compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
> > >     fragment@0 {
> > >         target-path = "/soc/i2s@7e203000";
> > >         __overlay__ {
> > >             #address-cells = <0x00000001>;
> > >             #size-cells = <0x00000001>;
> > >             test = "test";
> > >             timer = <0xdeadbeef>;
> > >         };
> > >     };
> > >     __fixups__ {
> > >         /soc/timer@7e0030000 = "/fragment@0/__overlay__:timer:0";
> > >     };
> > > };
> > >
> > > But this will not apply:
> > >
> > > OF: resolver: overlay phandle fixup failed: -22
> > > create_overlay: Failed to resolve tree
> > >
> > >
> >
> > Yes, it will not work as it is; my point is that you don't need the
> > magic __*__ node.
> >
> > You will need to modify the overlay application code to live insert
> > a
> > phandle (if it doesn't exist) when it encounters a /path fixup.
>
> phandles only exist if something in the base tree refers to that
> node.
> Adding them when they don't exist should definitely be something we
> support for overlays. But don't call that a broken DT. That would be
> a
> separate issue.
>

Believe me it is broken. Due a NDA i am not able to give you more
details about the vendor. But there forgot do provide an device node
which must refer to the attached network and interrupt controller.

- Stefani

2017-06-11 23:05:37

by Frank Rowand

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

On 06/07/17 23:48, Stefani Seibold wrote:
> Hi Pantelis,
>
> On Wed, 2017-06-07 at 11:11 +0300, Pantelis Antoniou wrote:
>> Hi Stefani,
>>
>> On Tue, 2017-06-06 at 21:17 +0200, Stefani Seibold wrote:
>>> Hi Pantelis,
>>>
>>> thanks for the suggestion. This feature is not very well
>>> documented. I
>>> tried this on my rasp1 running 4.12.0-rc3 and it doesn't work. My
>>> source is:
>>>
>>> // rapsi example
>>> /dts-v1/;
>>> /plugin/;
>>>
>>> / {
>>> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
>>>
>>> fragment@0 {
>>> target-path = "/soc/i2s@7e203000";
>>> __overlay__ {
>>> #address-cells = <0x00000001>;
>>> #size-cells = <0x00000001>;
>>> test = "test";
>>> timer = <&{/soc/timer@7e0030000}>;
>>> };
>>> };
>>> };
>>>
>>>
>>> The resulting overlay is (decompiled with fdtdump):
>>>
>>> /dts-v1/;
>>> // magic: 0xd00dfeed
>>> // totalsize: 0x19a (410)
>>> // off_dt_struct: 0x38
>>> // off_dt_strings: 0x148
>>> // off_mem_rsvmap: 0x28
>>> // version: 17
>>> // last_comp_version: 16
>>> // boot_cpuid_phys: 0x0
>>> // size_dt_strings: 0x52
>>> // size_dt_struct: 0x110
>>>
>>> / {
>>> compatible = "brcm,bcm2835", "brcm,bcm2708", "brcm,bcm2709";
>>> fragment@0 {
>>> target-path = "/soc/i2s@7e203000";
>>> __overlay__ {
>>> #address-cells = <0x00000001>;
>>> #size-cells = <0x00000001>;
>>> test = "test";
>>> timer = <0xdeadbeef>;
>>> };
>>> };
>>> __fixups__ {
>>> /soc/timer@7e0030000 = "/fragment@0/__overlay__:timer:0";
>>> };
>>> };
>>>
>>> But this will not apply:
>>>
>>> OF: resolver: overlay phandle fixup failed: -22
>>> create_overlay: Failed to resolve tree
>>>
>>>
>>
>> Yes, it will not work as it is; my point is that you don't need the
>> magic __*__ node.
>>
>
> The magic __fixups__ node was inserted by the device tree compiler. I
> use the dtc from https://github.com/pantoniou/dtc at commit
> d990b8013889b816ec054c7e07a77db59c56c400.
>
>> You will need to modify the overlay application code to live insert a
>> phandle (if it doesn't exist) when it encounters a /path fixup.
>>
>
> That is part of my patch!
>
>>> Anyway, the reason for my patch is that i can reference to nodes
>>> which
>>> lacks a phandle. The phandle will be created on the fly and also
>>> destroyed when the overlay is unloaded.
>>>
>>> I have a real use case for this patch:
>>>
>>> I have a BIOS on some ARM64 servers which provides broken device
>>> tree.
>>> It also lacks some devices in this tree which needs references to
>>> other
>>> devices which lacks a phandle.
>>>
>>> Since the BIOSes are closed source i need a way to work arround
>>> this
>>> problem without patching all the drivers involved to this devices.
>>>
>>> Hope this helps to understand the reason for this patch.
>>>
>>
>> FWIW your problem seems like something that would happen on the
>> field.
>> We can berate the vendor of not providing the correct device tree,
>> but
>> in the end workarounds for broken vendor things are common in the
>> kernel.
>>
>
> Yes, that is the way how linux do the things. Linux has a long history
> to bypassing bugs of BIOSes, ACPI or broken devices.

ARM device tree in Linux is not like BIOSes or ACPI.

ARM device tree in Linux is GPL v2 licensed (yes, it may also be dual
licensed for other uses) and thus "free software".

One of the key points of "free software" is that you have access to the
source and the ability to modify it.

Instead of bypassing device tree bugs, you have the ability to fix
device tree bugs. This is a fundamental difference.

-Frank

>
> Greetings,
> Stefani
>
>

2017-06-12 18:47:34

by Frank Rowand

[permalink] [raw]
Subject: Re: [PATCH] external references for device tree overlays

Adding back the Cc: list

On 06/11/17 23:13, Stefani Seibold wrote:
> Hi Frank,i
>
> i am in vaction for the next two weeks. I will rewrite the patch when i
> am back.
>
> Regards,
> Stefani

Hi Stefani,

Please let us know how you intend to solve the problems you are facing
before you invest a lot of time developing the patch.

< snip >

-Frank