by Miquel Raynal

[permalink] [raw]

Subject: Re: [PATCH v6 1/3] dt-bindings: mtd: partitions: Add binman compatible

Hi Simon,

[email protected] wrote on Wed, 13 Mar 2024 11:25:42 +1300:

> Hi Miquel,
>
> On Fri, 8 Mar 2024 at 20:42, Miquel Raynal <[email protected]> wrote:
> >
> > Hi Simon,
> >
> > [email protected] wrote on Fri, 8 Mar 2024 15:44:25 +1300:
> >
> > > Hi Miquel,
> > >
> > > On Tue, 6 Feb 2024 at 01:17, Miquel Raynal <[email protected]> wrote:
> > > >
> > > > Hi Simon,
> > > >
> > > > > > > > > > > > > > > > +description: |
> > > > > > > > > > > > > > > > + The binman node provides a layout for firmware, used when packaging firmware
> > > > > > > > > > > > > > > > + from multiple projects. It is based on fixed-partitions, with some
> > > > > > > > > > > > > > > > + extensions, but uses 'compatible' to indicate the contents of the node, to
> > > > > > > > > > > > > > > > + avoid perturbing or confusing existing installations which use 'label' for a
> > > > > > > > > > > > > > > > + particular purpose.
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + Binman supports properties used as inputs to the firmware-packaging process,
> > > > > > > > > > > > > > > > + such as those which control alignment of partitions. This binding addresses
> > > > > > > > > > > > > > > > + these 'input' properties. For example, it is common for the 'reg' property
> > > > > > > > > > > > > > > > + (an 'output' property) to be set by Binman, based on the alignment requested
> > > > > > > > > > > > > > > > + in the input.
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + Once processing is complete, input properties have mostly served their
> > > > > > > > > > > > > > > > + purpose, at least until the firmware is repacked later, e.g. due to a
> > > > > > > > > > > > > > > > + firmware update. The 'fixed-partitions' binding should provide enough
> > > > > > > > > > > > > > > > + information to read the firmware at runtime, including decompression if
> > > > > > > > > > > > > > > > + needed.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > How is this going to work exactly? binman reads these nodes and then
> > > > > > > > > > > > > > > writes out 'fixed-partitions' nodes. But then you've lost the binman
> > > > > > > > > > > > > > > specifc parts needed for repacking.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > No, they are the same node. I do want the extra information to stick
> > > > > > > > > > > > > > around. So long as it is compatible with fixed-partition as well, this
> > > > > > > > > > > > > > should work OK.
> > > > > > > > > > > > >
> > > > > > > > > > > > > How can it be both? The partitions node compatible can be either
> > > > > > > > > > > > > 'fixed-partitions' or 'binman'.
> > > > > > > > > > > >
> > > > > > > > > > > > Can we not allow it to be both? I have tried to adjust things in
> > > > > > > > > > > > response to feedback but perhaps the feedback was leading me down the
> > > > > > > > > > > > wrong path?
> > > > > > > > > > >
> > > > > > > > > > > Sure, but then the schema has to and that means extending
> > > > > > > > > > > fixed-partitions.
> > > > > > > > > >
> > > > > > > > > > Can we cross that bridge later? There might be resistance to it. I'm
> > > > > > > > > > not sure. For now, perhaps just a binman compatible works well enough
> > > > > > > > > > to make progress.
> > > > > > > > >
> > > > > > > > > Is there any way to make progress on this? I would like to have
> > > > > > > > > software which doesn't understand the binman compatible to at least be
> > > > > > > > > able to understand the fixed-partition compatible. Is that acceptable?
> > > > > > > >
> > > > > > > > There's only 2 ways that it can work. Either binman writes out
> > > > > > > > fixed-partition nodes dropping/replacing anything only defined for
> > > > > > > > binman or fixed-partition is extended to include what binman needs.
> > > > > > >
> > > > > > > OK, then I suppose the best way is to add a new binman compatible, as
> > > > > > > is done with this v6 series. People then need to choose it instead of
> > > > > > > fixed-partition.
> > > > > >
> > > > > > I'm sorry this is not at all what Rob suggested, or did I totally
> > > > > > misunderstand his answer?
> > > > > >
> > > > > > In both cases the solution is to generate a "fixed-partition" node. Now
> > > > > > up to you to decide whether binman should adapt the output to the
> > > > > > current schema, or if the current schema should be extended to
> > > > > > understand all binman's output.
> > > > > >
> > > > > > At least that is my understanding and also what I kind of agree with.
> > > > >
> > > > > I do want to binman schema to include all the features of Binman.
> > > > >
> > > > > So are you saying that there should not be a 'binman' schema, but I
> > > > > should just add all the binman properties to the fixed-partition
> > > > > schema?
> > > >
> > > > This is my current understanding, yes. But acknowledgment from Rob is
> > > > also welcome.
> > >
> > > I am trying again to wade through all the confusion here.
> > >
> > > There is not actually a 'fixed-partition' node. So are you saying I
> > > should add one? There is already a 'partitions' node. Won't they
> > > conflict?
> >
> > Sorry for the confusion, there is a 'partitions' node indeed. This
> > node shall declare it's "programming model" (let's say), ie. how it
> > should be parsed. What defines this programming model today is the
> > 'fixed-partitions' compatible. I think we (Rob and myself, but again,
> > Rob, please confirm) agree on the fact that we don't want to duplicate
> > the fixed-partitions compatible/logic and thus the binman compatible
> > was rejected.
> >
> > Hence, in order to move forward, I would definitely appreciate an
> > update of the fixed-partitions binding in order to support what binman
> > can generate.
>
> OK, so I think my confusion is that I thought you were referring to a
> 'partitions' compatible. But you are just referring to the name of the
> node being 'partitions', with the compatible string being
> 'fixed-partitions'.

Yes.

>
> I believe I can make this work by adding a new 'binman.yaml' with the
> compatibles that I want to introduce. I cannot change partition.yaml
> since it does not itself specify a compatible.

What about fixed-partitions.ymal? The yaml file name should match the
compatible.

> > We are here talking about the output of binman, not its input. TBH I
> > haven't understood the point in having binman's input parsed by the
> > generic yaml binding. I would advise to focus on binman's output first
> > because it feels more relevant, at a first glance.
>
> Yes that is fine.
>
> >
> > > Would it be possible for you to look at my patches and suggest
> > > something? I think at this point, after so many hours of trying
> > > different things and trying to understand what is needed, I could
> > > really use a little help.
> >
> > I hope the above details will help.
>
> I think so, thank you. I will send another version.
>
> Regards,
> Simon

Thanks,
Miquèl

2024-03-14 02:15:52

by Simon Glass

[permalink] [raw]

Subject: Re: [PATCH v6 1/3] dt-bindings: mtd: partitions: Add binman compatible

Hi Miquel,

On Wed, 13 Mar 2024 at 20:35, Miquel Raynal <[email protected]> wrote:
>
> Hi Simon,
>
> [email protected] wrote on Wed, 13 Mar 2024 11:25:42 +1300:
>
> > Hi Miquel,
> >
> > On Fri, 8 Mar 2024 at 20:42, Miquel Raynal <[email protected]> wrote:
> > >
> > > Hi Simon,
> > >
> > > [email protected] wrote on Fri, 8 Mar 2024 15:44:25 +1300:
> > >
> > > > Hi Miquel,
> > > >
> > > > On Tue, 6 Feb 2024 at 01:17, Miquel Raynal <[email protected]> wrote:
> > > > >
> > > > > Hi Simon,
> > > > >
> > > > > > > > > > > > > > > > > +description: |
> > > > > > > > > > > > > > > > > + The binman node provides a layout for firmware, used when packaging firmware
> > > > > > > > > > > > > > > > > + from multiple projects. It is based on fixed-partitions, with some
> > > > > > > > > > > > > > > > > + extensions, but uses 'compatible' to indicate the contents of the node, to
> > > > > > > > > > > > > > > > > + avoid perturbing or confusing existing installations which use 'label' for a
> > > > > > > > > > > > > > > > > + particular purpose.
> > > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > > + Binman supports properties used as inputs to the firmware-packaging process,
> > > > > > > > > > > > > > > > > + such as those which control alignment of partitions. This binding addresses
> > > > > > > > > > > > > > > > > + these 'input' properties. For example, it is common for the 'reg' property
> > > > > > > > > > > > > > > > > + (an 'output' property) to be set by Binman, based on the alignment requested
> > > > > > > > > > > > > > > > > + in the input.
> > > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > > + Once processing is complete, input properties have mostly served their
> > > > > > > > > > > > > > > > > + purpose, at least until the firmware is repacked later, e.g. due to a
> > > > > > > > > > > > > > > > > + firmware update. The 'fixed-partitions' binding should provide enough
> > > > > > > > > > > > > > > > > + information to read the firmware at runtime, including decompression if
> > > > > > > > > > > > > > > > > + needed.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > How is this going to work exactly? binman reads these nodes and then
> > > > > > > > > > > > > > > > writes out 'fixed-partitions' nodes. But then you've lost the binman
> > > > > > > > > > > > > > > > specifc parts needed for repacking.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > No, they are the same node. I do want the extra information to stick
> > > > > > > > > > > > > > > around. So long as it is compatible with fixed-partition as well, this
> > > > > > > > > > > > > > > should work OK.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > How can it be both? The partitions node compatible can be either
> > > > > > > > > > > > > > 'fixed-partitions' or 'binman'.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Can we not allow it to be both? I have tried to adjust things in
> > > > > > > > > > > > > response to feedback but perhaps the feedback was leading me down the
> > > > > > > > > > > > > wrong path?
> > > > > > > > > > > >
> > > > > > > > > > > > Sure, but then the schema has to and that means extending
> > > > > > > > > > > > fixed-partitions.
> > > > > > > > > > >
> > > > > > > > > > > Can we cross that bridge later? There might be resistance to it. I'm
> > > > > > > > > > > not sure. For now, perhaps just a binman compatible works well enough
> > > > > > > > > > > to make progress.
> > > > > > > > > >
> > > > > > > > > > Is there any way to make progress on this? I would like to have
> > > > > > > > > > software which doesn't understand the binman compatible to at least be
> > > > > > > > > > able to understand the fixed-partition compatible. Is that acceptable?
> > > > > > > > >
> > > > > > > > > There's only 2 ways that it can work. Either binman writes out
> > > > > > > > > fixed-partition nodes dropping/replacing anything only defined for
> > > > > > > > > binman or fixed-partition is extended to include what binman needs.
> > > > > > > >
> > > > > > > > OK, then I suppose the best way is to add a new binman compatible, as
> > > > > > > > is done with this v6 series. People then need to choose it instead of
> > > > > > > > fixed-partition.
> > > > > > >
> > > > > > > I'm sorry this is not at all what Rob suggested, or did I totally
> > > > > > > misunderstand his answer?
> > > > > > >
> > > > > > > In both cases the solution is to generate a "fixed-partition" node. Now
> > > > > > > up to you to decide whether binman should adapt the output to the
> > > > > > > current schema, or if the current schema should be extended to
> > > > > > > understand all binman's output.
> > > > > > >
> > > > > > > At least that is my understanding and also what I kind of agree with.
> > > > > >
> > > > > > I do want to binman schema to include all the features of Binman.
> > > > > >
> > > > > > So are you saying that there should not be a 'binman' schema, but I
> > > > > > should just add all the binman properties to the fixed-partition
> > > > > > schema?
> > > > >
> > > > > This is my current understanding, yes. But acknowledgment from Rob is
> > > > > also welcome.
> > > >
> > > > I am trying again to wade through all the confusion here.
> > > >
> > > > There is not actually a 'fixed-partition' node. So are you saying I
> > > > should add one? There is already a 'partitions' node. Won't they
> > > > conflict?
> > >
> > > Sorry for the confusion, there is a 'partitions' node indeed. This
> > > node shall declare it's "programming model" (let's say), ie. how it
> > > should be parsed. What defines this programming model today is the
> > > 'fixed-partitions' compatible. I think we (Rob and myself, but again,
> > > Rob, please confirm) agree on the fact that we don't want to duplicate
> > > the fixed-partitions compatible/logic and thus the binman compatible
> > > was rejected.
> > >
> > > Hence, in order to move forward, I would definitely appreciate an
> > > update of the fixed-partitions binding in order to support what binman
> > > can generate.
> >
> > OK, so I think my confusion is that I thought you were referring to a
> > 'partitions' compatible. But you are just referring to the name of the
> > node being 'partitions', with the compatible string being
> > 'fixed-partitions'.
>
> Yes.
>
> >
> > I believe I can make this work by adding a new 'binman.yaml' with the
> > compatibles that I want to introduce. I cannot change partition.yaml
> > since it does not itself specify a compatible.
>
> What about fixed-partitions.ymal? The yaml file name should match the
> compatible.

But we already decided we cannot add a new 'binman' compatible but
want to use the existing fixed-partitions.yaml

I cannot add compatible strings into that, since these are defined by
the default files which make use of partition.yaml

Anyway, I will see v7 so you can see what I mean.

>
> > > We are here talking about the output of binman, not its input. TBH I
> > > haven't understood the point in having binman's input parsed by the
> > > generic yaml binding. I would advise to focus on binman's output first
> > > because it feels more relevant, at a first glance.
> >
> > Yes that is fine.
> >
> > >
> > > > Would it be possible for you to look at my patches and suggest
> > > > something? I think at this point, after so many hours of trying
> > > > different things and trying to understand what is needed, I could
> > > > really use a little help.
> > >
> > > I hope the above details will help.
> >
> > I think so, thank you. I will send another version.
> >
> > Regards,
> > Simon

Regards,
Simon