LinuxLists.cc - [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Tabi Timur-B04825 <[email protected]> wrote:

>On Mon, Nov 5, 2012 at 2:40 PM, Grant Likely
><[email protected]> wrote:
>
>> Jane is building custom BeagleBone expansion boards called 'capes'.
>She
>> can boot the system with a stock BeagleBoard device tree, but
>additional
>> data is needed before a cape can be used. She could replace the FDT
>file
>> used by U-Boot with one that contains the extra data, but she uses
>the
>> same Linux system image regardless of the cape, and it is
>inconvenient
>> to have to select a different device tree at boot time depending on
>the
>> cape.
>
>What's wrong with having the boot loader detect the presence of the
>Cape and update the device tree accordingly? We do this all the time
>in U-Boot. Doing stuff like reading EEPROMs and testing for the
>presence of hardware is easier in U-Boot than in Linux.
>
>For configurations that can be determined by the boot loader, I'm not
>sure overlays are practical.

>From the discussion in the previous thread, I'm sufficiently convinced that it is an important use case. I certainly disagree with the assertion that it is always easier to do it in U-Boot. Sometimes the kernel is the better place.

>
>> Nigel is building a real-time video processing system around a MIPS
>SoC
>> and a Virtex FPGA. Video data is streamed through the FPGA for post
>> processing operations like motion tracking or compression. The FPGA
>is
>> configured via the SPI bus, and is also connected to GPIO lines and
>the
>> memory mapped peripheral bus. Nigel has designed several FPGA
>> configurations for different video processing tasks. The user will
>> choose which configuration to load which can even be reprogrammed at
>> runtime to switch tasks.
>
>Now this, on the other hand, makes more sense. If the hardware
>configuration is literally user-configurable, then okay. However, I'm
>not sure I see the need to update the device tree. The device tree is
>generally for hardware that cannot be discovered/probed by the device
>driver. If we're loading a configuration from user space, doesn't the
>driver already know what the hardware's capabilities are, since it's
>the one doing the uploading of a new FPGA code?

Not if the driver is only responsible for loading the bitstream. There is already a xilinx driver that does things this way.

> Why not skip the
>device tree update and just tell the driver what the new capabilities
>are?

How? What format will that data be in if not device tree?

g.

--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

2012-11-06 10:30:11

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Grant,

On Nov 5, 2012, at 9:40 PM, Grant Likely wrote:

> Hey folks,
>
> As promised, here is my early draft to try and capture what device
> tree overlays need to do and how to get there. Comments and
> suggestions greatly appreciated.
>
> Device Tree Overlay Feature
>
> Purpose
> =======
> Sometimes it is not convenient to describe an entire system with a
> single FDT. For example, processor modules that are plugged into one or
> more modules (a la the BeagleBone), or systems with an FPGA peripheral
> that is programmed after the system is booted.
>
> For these cases it is proposed to implement an overlay feature for the
> so that the initial device tree data can be modified by userspace at
> runtime by loading additional overlay FDTs that amend the original data.
>
> User Stories
> ============
> Note - These are potential use cases, but just because it is listed here
> doesn't mean it is important. I just want to thoroughly think through the
> implications before making design decisions.
>
>
> Jane is building custom BeagleBone expansion boards called 'capes'. She
> can boot the system with a stock BeagleBoard device tree, but additional
> data is needed before a cape can be used. She could replace the FDT file
> used by U-Boot with one that contains the extra data, but she uses the
> same Linux system image regardless of the cape, and it is inconvenient
> to have to select a different device tree at boot time depending on the
> cape.
>
> Jane solves this problem by storing an FDT overlay for each cape in the
> root filesystem. When the kernel detects that a cape is installed it
> reads the cape's eeprom to identify it and uses request_firmware() to
> obtain the appropriate overlay. Userspace passes the overlay to the
> kernel in the normal way. If the cape doesn't have an eeprom, then the
> kernel will still use firmware_request(), but userspace needs to already
> know which cape is installed.

Jane is a really productive hardware engineer - she manages to fix a
number of problems with her cape design by spinning different revisions
of the cape. Using the flexibility that the DT provides, documents and
defines the hardware changes of the cape revisions in the FDT overlay.
The loader matches the revision of the cape with the proper FDT overlay
so that the drivers are relieved of having to do revision management.

>
>
> Nigel is building a real-time video processing system around a MIPS SoC
> and a Virtex FPGA. Video data is streamed through the FPGA for post
> processing operations like motion tracking or compression. The FPGA is
> configured via the SPI bus, and is also connected to GPIO lines and the
> memory mapped peripheral bus. Nigel has designed several FPGA
> configurations for different video processing tasks. The user will
> choose which configuration to load which can even be reprogrammed at
> runtime to switch tasks.
>
> Each FPGA has a different interface to the processor, so the kernel
> needs additional data before it can use each device. Nigel is passing
> that data to the kernel using an FDT overlay. When Linux loads a new
> FPGA configuration, it uses request_firmare() to obtain the overlay for
> that FPGA. When the FPGA gets reprogrammed, the kernel throws away the
> previous overlay data and uses request_firmware() to get the overlay for
> the new design.
>
>
> Mandy has a Raspberry Pi which she has wired by hand up to sensors and
> motor controllers in her prototype autonomous robot project. She is doing
> self-hosted driver development on the Raspberry Pi itself. However, she
> needs a method to tell the kernel about the attached devices.
>
> By installing dtc on the Pi, Mandy compiles the overlay for her
> prototype hardware. However, she doesn't have a copy of the Pi's
> original FDT source, so instead she uses the dtc 'fs' input format to
> compile the overlay file against the live DT data in /proc.
>
>

Jane (the cape designer) can use this too. Developing the cape, she really
appreciates that she doesn't have to reboot every time she makes a change
in the cape hardware. By removing the FDT overlay, compiling with the dtc
on the board, and re-inserting the overlay, she can be more productive by
waiting less.

Johnny, Jane's little son, doesn't know anything about device trees, linux
kernel trees, or hard-core s/w engineering. He is a bright kid, and due to
the board having a node.js based educational electronic design kit, he
can use the web-based simplified development environment, that allows
him graphically to connect the parts in his kit. He can save the design
and the IDE creates on the fly the DT overlay for later use.

> Amit is writing kernel drivers for Jane's BeagleBone capes, but he
> finds loading new DT files into the root filesystem inconvenient.
> Instead, he includes the FDT overlay file in the initramfs that is built
> and linked in at kernel compile time so that the kernel can find and
> load overlays automatically.
>
>
> Joanne has purchased one of Jane's capes and packaged it into a rugged
> case for data logging. As far as Joanne is concerned, the BeagleBone and
> cape together are a single unit and she'd prefer a single monolithic FDT
> instead of using an FDT overlay.
> Option A: Using dtc, she uses the BeagleBone and cape .dts source files
> to generate a single .dtb for the entire system which is
> loaded by U-Boot. -or-

Unlikely.
> Option B: Joanne uses a tool to merge the BeagleBone and cape .dtb files
> (instead of .dts files), -or-
Possible but low probability.

> Option C: U-Boot loads both the base and overlay FDT files, merges them,
> and passes the resolved tree to the kernel.
>

Could be made to work. Only really required if Joanne wants the
cape interface to work for u-boot too. For example if the cape has some
kind of network interface that u-boot will use to boot from.

> ....
>
> Summary points:
> - Create an FDT overlay data format and usage model
> - SHALL reliable resolve or validate of phandles between base and
> overlay trees
> - SHOULD reliably handle changes between different underlying overlays
> (ie. what happens to existing .dtb overly files if the structure of
> the dtb it is layered over changes. If not possible, then SHALL
> detect when the base tree doesn't match and refuse to apply the
> overlay.
> - dts syntax needs to be extended for overlay .dtb files
> - DTC tool needs to be modified to support overlay .dtb generation
> - Overlays SHOULD be able to be applied either by firmware or the kernel
> - libfdt SHALL be extended to parse and apply overlays

- ftdump should be fixed and work for the overlay syntax too.

> - Linux SHALL be extended to parse and apply overlays from userspace
> - Linux SHALL be extended to notify drivers of changes to device tree
> - Linux MAY use request_firmware() to load overlays via sysfs
> - (mechanism for triggering overlay load TBD)
> - Linux MAY support removal of overlays (harder, but some use-cases want
> this)
> - Linux MAY be extended to remove devices associated with removed overlays
>
> Technical details to resolve:
> - How does an overlay get attached to the correct base tree nodes?
> - How are phandles fixed up and/or verified?
> - What is the model for overlays?
> - Can an overlay modify existing properties?
> - Can an overlay add new properties to existing nodes?
> - Can an overlay delete existing nodes/properties?
> - Does FDT need a schema? - Schema should be tightly associated with binding
> documentation. Verifying schema at runtime should be simple. Runtime
> checking is harder, but could be used as part of phandle fixup.
> - Similarly, does FDT need data typing? Other than reading the binding
> there is no mechanism to determine if a single cell is a phandle, an
> integer (possibly enum), a flags field, or part of a string.
> - Data typing would require additional per-property data; probably by
> adding companion properties containing the data typing. ie. an
> optional '.reg,format' property could contain the format of the
> 'reg' property. (The naming scheme for the new property can be
> debated, it just needs to be something that won't conflict with
> regular names).
> - Data type data could be used as part of phandle fixups.
> - How do bus drivers get notified when FDT data changes?
> - node addition/removal
> - property changes on existing nodes
> - A notifier chain may work here, or maybe a generic "firmware data
> changed" device driver callback. I could see this being generically
> useful for other driver data like ACPI
>

This is much grander in vision that I had in mind :)

It can handle our use cases, but I'm worried if we're bitting more
that what we can chew. Perhaps a staged approach? I.e. target the
low hanging fruit first, get it work, and then work on the hardest
parts?

> Project plan
> ============
>
> 1) Create syntax for overlays
> -----------------------------
> DTC already uses an overlay model internally which is often used in
> conjunction with .dtsi files. It should be a natural extension to
> generate FDT overlay files from the existing syntax (maybe minor
> modifications)
>
> Doing this will require a method to differentiate between overlay and
> include directives. Maybe they aren't really different at all and it
> will depend on dtc options to determine whether or not to produce an
> overlay. Suggestions are welcome here.
>
> 2) Create a data format for overlays
> ------------------------------------
> The overlay data format could be a direct extension from the existing
> dtb data format. One option is to construct the overlay tree by creating
> empty nodes for each node that it either adds children to or references
> the phandle of in the base tree. Empty nodes could even be tagged as a
> prereq by adding a '.readonly' property to those nodes. When an overlay
> parser encounters those nodes, it would know that it needs to already
> exist in the tree.
>
> There is a problem though with phandles. If the phandle values in the
> base tree don't match the ones in the overlay, then the overlay won't be
> able to correctly reference base nodes. At a minimum, the parser must
> verify that the phandles match before applying the overlay. Ideally,
> phandles in the overlay should be fixed up at application time, but this
> isn't easy. It would probably require first adding property data type
> information to the device tree.
>
> It may be sufficient to solve it by making the phandle values less
> volatile. Right now dtc generates phandles linearly. Generated phandles
> could be overridden with explicit phandle properties, but it isn't a
> fantastic solution. Perhaps generating the phandle from a hash of the
> node name would be sufficient.
>

I doubt the hash method will work reliably. We only have 32 bits to work with,
nothing like the SHA hashes of git.

> Example: an overlay source file might look something like this:
>
> /include/ "base-file.dts" /* 'include' may not be the best syntax here */
> &i2c0 { /* i2c0 resolved by label */
> touchpad@10 {
> compatible = "acme,touchpad";
> reg = <0x10>;
> interrupt-parent = <&intc>;
> interrupts = <100>;
> };
> };
>
> And the generated overlay dtb may look like this:
>
> / {
> .readonly;
> interrupt-controller@0x10005000 {
> .readonly;
> phandle = <0x1234>;
> };
> peripheral-bus {
> .readonly;
> i2c@20001000 {
> touchpad@10 {
> compatible = "acme,touchpad";
> reg = <0x10>;
> interrupt-parent = <0x1234>;
> interrupts = <100>;
> };
> };
> };
> };
>
> Which is obviously missing a bunch of information for the rest of the
> system, but contains enough data to ensure that the path to the touchpad
> node is valid and that the phandle has not changed.
>
> This handles many of the use cases, but it assumes that an overlay is
> board specific. If it ever is required to support multiple base boards
> with a single overlay file then there is a problem. The .dtb overlays
> generated in this manor cannot handle different phandles or nodes that
> are in a different place. On the other hand, the overlay source files
> should have no problem being compiled for multiple targets.

It will work for our case. The board-file dependency problem is not a big
concern right now.

>
> 3) Modify DTC and libfdt to process overlays
> --------------------------------------------
> Direct follow on from above. This will require DTC to be modified to
> generate the overlay output format and new test cases to verify it
> works.
>
> In the process libfdt will get modified to add overlay support which
> will immediately be available to U-Boot and other bootloaders.
>
> 4) Teach Linux to request FDT overlays
> --------------------------------------
> Add request_firmware() hook for adding overlays. If the platform
> supports detecting the attached expansion board, then this may
> manipulate the filename for a specific file.
>
> The kernel will graft the overlay into the existing live tree
>

5) Have a method to attach FDT overlay to a kernel module.

For some drivers it might be better if the kernel module and the
DT overlay is packaged in the same file. You be in a part of
the module binary as a special section that request_firmware can
pick up automatically.

> Workitems:
> Document device tree node lifecycle
> Add tests to enforce device tree node lifecycle
> Add interface to firmware_request() an FDT overlay
> Add support to merge overlay FDT nodes into base tree
> Add support to track FDT changes per-overlay
> Add runtime fixups of overlay phandles
> - Without this overlay phandles must exactly match the base tree
> Add support to remove an overlay (question: is this important?)
>

For hot-plugging, you need it. Whether kernel code can deal with
large parts of the DT going away... How about we use the dead
properties method and move/tag the removed modes as such, and not
really remove them.

> 5) Teach Linux driver model to respond to device tree changes
> -------------------------------------------------------------
> There are several parts to this. The most obvious are adding hooks for
> platform_device, i2c_device and spi_device creation. In all three cases,
> the users should already be providing enough information to support
> dynamic addition (and possibly removal) of dt nodes.
> of_platform_populate() is called for platform devices,
> of_register_spi_devices() for spi and of_i2c_register_devices(). Right
> now all three functions register all the child devices and then
> immediately return. However, if they could set up a notifier for node
> addition, then it would be easy to trigger additional device creation
> when new nodes are added.
>
> Other busses would be similar, but i2c, spi and platform are the most
> common use cases currently.
>
> Workitems:
> Modify of_platform_populate() to get new node notifications
> Modify of_register_spi_devices to get new node notifications
> Modify of_register_i2c_devices to get new node notifications
>

w1 is the same. Possibly more.

Another can of worms is the pinctrl nodes.

> 6) Other work
> -------------
> The device node user space interface could use some work. Here are some
> random work items that are peripherally related to the overlay feature.
>
> Other Workitems:
> Define FDT schema syntax
> Add FDT schema support to FDT (basically lint-style testing)
> Investigate runtime schema validation
> Make device_nodes first-class kobjects and remove the procfs interface
> - it can be emulated with a symlink
> Add symlinks from devices to devicetree nodes in sysfs

That's going to take a while :)

Regards

-- Pantelis

2012-11-06 10:31:47

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Timur,

On Nov 5, 2012, at 10:40 PM, Tabi Timur-B04825 wrote:

> On Mon, Nov 5, 2012 at 2:40 PM, Grant Likely <[email protected]> wrote:
>
>> Jane is building custom BeagleBone expansion boards called 'capes'. She
>> can boot the system with a stock BeagleBoard device tree, but additional
>> data is needed before a cape can be used. She could replace the FDT file
>> used by U-Boot with one that contains the extra data, but she uses the
>> same Linux system image regardless of the cape, and it is inconvenient
>> to have to select a different device tree at boot time depending on the
>> cape.
>
> What's wrong with having the boot loader detect the presence of the
> Cape and update the device tree accordingly? We do this all the time
> in U-Boot. Doing stuff like reading EEPROMs and testing for the
> presence of hardware is easier in U-Boot than in Linux.
>
> For configurations that can be determined by the boot loader, I'm not
> sure overlays are practical.
>

Each use case is different. For our use-cases boot loader DT modifications
just won't work.

What's nice about the stuff we're talking about is that if you don't use
the new functionality, nothing changes for you. Go on and use DT the old
way if you'd like.

>> Nigel is building a real-time video processing system around a MIPS SoC
>> and a Virtex FPGA. Video data is streamed through the FPGA for post
>> processing operations like motion tracking or compression. The FPGA is
>> configured via the SPI bus, and is also connected to GPIO lines and the
>> memory mapped peripheral bus. Nigel has designed several FPGA
>> configurations for different video processing tasks. The user will
>> choose which configuration to load which can even be reprogrammed at
>> runtime to switch tasks.
>
> Now this, on the other hand, makes more sense. If the hardware
> configuration is literally user-configurable, then okay. However, I'm
> not sure I see the need to update the device tree. The device tree is
> generally for hardware that cannot be discovered/probed by the device
> driver. If we're loading a configuration from user space, doesn't the
> driver already know what the hardware's capabilities are, since it's
> the one doing the uploading of a new FPGA code? Why not skip the
> device tree update and just tell the driver what the new capabilities
> are?
>
> --
> Timur Tabi
> Linux kernel developer at Freescale

Regards

-- Pantelis

2012-11-06 11:15:05

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Tue, Nov 6, 2012 at 10:30 AM, Pantelis Antoniou
<[email protected]> wrote:
> Hi Grant,
>
> On Nov 5, 2012, at 9:40 PM, Grant Likely wrote:
>
>> Hey folks,
>>
>> As promised, here is my early draft to try and capture what device
>> tree overlays need to do and how to get there. Comments and
>> suggestions greatly appreciated.
>>
>> Device Tree Overlay Feature
>>
>> Purpose
>> =======
>> Sometimes it is not convenient to describe an entire system with a
>> single FDT. For example, processor modules that are plugged into one or
>> more modules (a la the BeagleBone), or systems with an FPGA peripheral
>> that is programmed after the system is booted.
>>
>> For these cases it is proposed to implement an overlay feature for the
>> so that the initial device tree data can be modified by userspace at
>> runtime by loading additional overlay FDTs that amend the original data.
>>
>> User Stories
>> ============
>> Note - These are potential use cases, but just because it is listed here
>> doesn't mean it is important. I just want to thoroughly think through the
>> implications before making design decisions.
>>
>>
>> Jane is building custom BeagleBone expansion boards called 'capes'. She
>> can boot the system with a stock BeagleBoard device tree, but additional
>> data is needed before a cape can be used. She could replace the FDT file
>> used by U-Boot with one that contains the extra data, but she uses the
>> same Linux system image regardless of the cape, and it is inconvenient
>> to have to select a different device tree at boot time depending on the
>> cape.
>>
>> Jane solves this problem by storing an FDT overlay for each cape in the
>> root filesystem. When the kernel detects that a cape is installed it
>> reads the cape's eeprom to identify it and uses request_firmware() to
>> obtain the appropriate overlay. Userspace passes the overlay to the
>> kernel in the normal way. If the cape doesn't have an eeprom, then the
>> kernel will still use firmware_request(), but userspace needs to already
>> know which cape is installed.
>
> Jane is a really productive hardware engineer - she manages to fix a
> number of problems with her cape design by spinning different revisions
> of the cape. Using the flexibility that the DT provides, documents and
> defines the hardware changes of the cape revisions in the FDT overlay.
> The loader matches the revision of the cape with the proper FDT overlay
> so that the drivers are relieved of having to do revision management.

Okay

>> By installing dtc on the Pi, Mandy compiles the overlay for her
>> prototype hardware. However, she doesn't have a copy of the Pi's
>> original FDT source, so instead she uses the dtc 'fs' input format to
>> compile the overlay file against the live DT data in /proc.
>
> Jane (the cape designer) can use this too. Developing the cape, she really
> appreciates that she doesn't have to reboot every time she makes a change
> in the cape hardware. By removing the FDT overlay, compiling with the dtc
> on the board, and re-inserting the overlay, she can be more productive by
> waiting less.

Yes, but I'll leave this paragraph out of the spec. It isn't
significantly different from what is already there.

> Johnny, Jane's little son, doesn't know anything about device trees, linux
> kernel trees, or hard-core s/w engineering. He is a bright kid, and due to
> the board having a node.js based educational electronic design kit, he
> can use the web-based simplified development environment, that allows
> him graphically to connect the parts in his kit. He can save the design
> and the IDE creates on the fly the DT overlay for later use.

Yes.

>> Joanne has purchased one of Jane's capes and packaged it into a rugged
>> case for data logging. As far as Joanne is concerned, the BeagleBone and
>> cape together are a single unit and she'd prefer a single monolithic FDT
>> instead of using an FDT overlay.
>> Option A: Using dtc, she uses the BeagleBone and cape .dts source files
>> to generate a single .dtb for the entire system which is
>> loaded by U-Boot. -or-
> Unlikely.
>> Option B: Joanne uses a tool to merge the BeagleBone and cape .dtb files
>> (instead of .dts files), -or-
> Possible but low probability.
>> Option C: U-Boot loads both the base and overlay FDT files, merges them,
>> and passes the resolved tree to the kernel.
> Could be made to work. Only really required if Joanne wants the
> cape interface to work for u-boot too. For example if the cape has some
> kind of network interface that u-boot will use to boot from.

Unlikely for your focus perhaps, but I'm trying to capture all the
relevant permutations, and I can guarantee that some people really
will want this. If not on the bone, then on some other platform.

>> Summary points:
>> - Create an FDT overlay data format and usage model
>> - SHALL reliable resolve or validate of phandles between base and
>> overlay trees
>> - SHOULD reliably handle changes between different underlying overlays
>> (ie. what happens to existing .dtb overly files if the structure of
>> the dtb it is layered over changes. If not possible, then SHALL
>> detect when the base tree doesn't match and refuse to apply the
>> overlay.
>> - dts syntax needs to be extended for overlay .dtb files
>> - DTC tool needs to be modified to support overlay .dtb generation
>> - Overlays SHOULD be able to be applied either by firmware or the kernel
>> - libfdt SHALL be extended to parse and apply overlays
>
> - ftdump should be fixed and work for the overlay syntax too.

Okay

> This is much grander in vision that I had in mind :)
>
> It can handle our use cases, but I'm worried if we're bitting more
> that what we can chew. Perhaps a staged approach? I.e. target the
> low hanging fruit first, get it work, and then work on the hardest
> parts?

Actually, I'm not to scared about the work and yes I think that it
*must* be a staged approach. To start focus on adding overlays without
phandle resolution (phandles must match) or unloading support.
Unloading and phandle resolution can be separate follow-on features.
Unloading and phandle resolution are the hard bits anyway.

>> It may be sufficient to solve it by making the phandle values less
>> volatile. Right now dtc generates phandles linearly. Generated phandles
>> could be overridden with explicit phandle properties, but it isn't a
>> fantastic solution. Perhaps generating the phandle from a hash of the
>> node name would be sufficient.
>>
>
> I doubt the hash method will work reliably. We only have 32 bits to work with,
> nothing like the SHA hashes of git.
>

I think the biggest trees have on the order of 100 nodes and a 32 bit
numberspace is 4Gi. Surely collisions can be avoided. :-)

It is also possible to explicitly specify the phandle when the hash
method breaks down, or if the node full path needs to change, but I'd
like to avoid that approach as much as possible.

> 5) Have a method to attach FDT overlay to a kernel module.
>
> For some drivers it might be better if the kernel module and the
> DT overlay is packaged in the same file. You be in a part of
> the module binary as a special section that request_firmware can
> pick up automatically.

It used to be that firmware blobs could be linked into the kernel and
request_firmware() would find them. I'd like to investigate if that is
still possible.

>> Add support to remove an overlay (question: is this important?)
>>
>
> For hot-plugging, you need it. Whether kernel code can deal with
> large parts of the DT going away... How about we use the dead
> properties method and move/tag the removed modes as such, and not
> really remove them.

Nodes already use krefs, and I'm thinking about making them kobjects
so that they appear in sysfs and we'll have some tools to figure out
when reference counts don't get decremented properly.

>> Workitems:
>> Modify of_platform_populate() to get new node notifications
>> Modify of_register_spi_devices to get new node notifications
>> Modify of_register_i2c_devices to get new node notifications
>>
>
> w1 is the same. Possibly more.

w1?

>
> Another can of worms is the pinctrl nodes.

Yes... new pinctrl data would need to trigger adding new data to
pinctrl. I don't know if the pinctrl api supports that.

>
>> 6) Other work
>> -------------
>> The device node user space interface could use some work. Here are some
>> random work items that are peripherally related to the overlay feature.
>>
>> Other Workitems:
>> Define FDT schema syntax
>> Add FDT schema support to FDT (basically lint-style testing)
>> Investigate runtime schema validation
>> Make device_nodes first-class kobjects and remove the procfs interface
>> - it can be emulated with a symlink
>> Add symlinks from devices to devicetree nodes in sysfs
>
> That's going to take a while :)

:-) But as you've already pointed out this should be taken in a staged
approach. Simple overlay support is still useful and shouldn't be too
complex to implement.

g.

2012-11-06 18:35:48

by Tony Lindgren

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

* Grant Likely <[email protected]> [121106 03:16]:
> On Tue, Nov 6, 2012 at 10:30 AM, Pantelis Antoniou
> <[email protected]> wrote:
> >
> > Another can of worms is the pinctrl nodes.
>
> Yes... new pinctrl data would need to trigger adding new data to
> pinctrl. I don't know if the pinctrl api supports that.

The actual pins stay the same, just their configuration
changes. AFAIK all that is already supported using the
pinctrl framework.

For example, considering hotplugging capes on the beaglebone:

1. You need to map all the sensible modes for the pins exposed
to the capes in the board specific .dts file. This will
add roughly 4 x nr_capbus_pins named modes in the .dts file
so not too bad.

2. Claim all the capebus pins during the capbus driver probe
and set them to some safe mode.

3. Try to detect the connected cape(s) over i2c.

4. Use pinctr_select_state to set the desired modes for
the pins used by the cape(s).

5. Enable capebus regulators and clocks etc.

6. Load the driver modules for whatever omap internal
devices the cape supports.

You could also claim the pin for the omap internal
devices instead of claiming them in the capebus, but then
things can get messy with binding and unbinding the
drivers. So just claiming all the pins in the capebus
probably keeps things simpler.

Regards,

Tony

2012-11-06 19:30:03

by Russ Dill

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Tue, Nov 6, 2012 at 10:35 AM, Tony Lindgren <[email protected]> wrote:
> * Grant Likely <[email protected]> [121106 03:16]:
>> On Tue, Nov 6, 2012 at 10:30 AM, Pantelis Antoniou
>> <[email protected]> wrote:
>> >
>> > Another can of worms is the pinctrl nodes.
>>
>> Yes... new pinctrl data would need to trigger adding new data to
>> pinctrl. I don't know if the pinctrl api supports that.
>
> The actual pins stay the same, just their configuration
> changes. AFAIK all that is already supported using the
> pinctrl framework.
>
> For example, considering hotplugging capes on the beaglebone:
>
> 1. You need to map all the sensible modes for the pins exposed
> to the capes in the board specific .dts file. This will
> add roughly 4 x nr_capbus_pins named modes in the .dts file
> so not too bad.
>
> 2. Claim all the capebus pins during the capbus driver probe
> and set them to some safe mode.
>
> 3. Try to detect the connected cape(s) over i2c.
>
> 4. Use pinctr_select_state to set the desired modes for
> the pins used by the cape(s).
>
> 5. Enable capebus regulators and clocks etc.
>
> 6. Load the driver modules for whatever omap internal
> devices the cape supports.
>
> You could also claim the pin for the omap internal
> devices instead of claiming them in the capebus, but then
> things can get messy with binding and unbinding the
> drivers. So just claiming all the pins in the capebus
> probably keeps things simpler.

That assumes that for a particular external bus, certain pins aren't
already shared with functions already on the board, for instance if an
I²C bus brought out to the external bus already has a chip connected
to it.

2012-11-06 19:34:59

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Grant,

On Nov 6, 2012, at 12:14 PM, Grant Likely wrote:

> On Tue, Nov 6, 2012 at 10:30 AM, Pantelis Antoniou
> <[email protected]> wrote:
>> Hi Grant,
>>
>> On Nov 5, 2012, at 9:40 PM, Grant Likely wrote:
>>
>>> Hey folks,
>>>
>>> As promised, here is my early draft to try and capture what device
>>> tree overlays need to do and how to get there. Comments and
>>> suggestions greatly appreciated.
>>>
>>> Device Tree Overlay Feature
>>>
>>> Purpose
>>> =======
>>> Sometimes it is not convenient to describe an entire system with a
>>> single FDT. For example, processor modules that are plugged into one or
>>> more modules (a la the BeagleBone), or systems with an FPGA peripheral
>>> that is programmed after the system is booted.
>>>
>>> For these cases it is proposed to implement an overlay feature for the
>>> so that the initial device tree data can be modified by userspace at
>>> runtime by loading additional overlay FDTs that amend the original data.
>>>
>>> User Stories
>>> ============
>>> Note - These are potential use cases, but just because it is listed here
>>> doesn't mean it is important. I just want to thoroughly think through the
>>> implications before making design decisions.
>>>
>>>
>>> Jane is building custom BeagleBone expansion boards called 'capes'. She
>>> can boot the system with a stock BeagleBoard device tree, but additional
>>> data is needed before a cape can be used. She could replace the FDT file
>>> used by U-Boot with one that contains the extra data, but she uses the
>>> same Linux system image regardless of the cape, and it is inconvenient
>>> to have to select a different device tree at boot time depending on the
>>> cape.
>>>
>>> Jane solves this problem by storing an FDT overlay for each cape in the
>>> root filesystem. When the kernel detects that a cape is installed it
>>> reads the cape's eeprom to identify it and uses request_firmware() to
>>> obtain the appropriate overlay. Userspace passes the overlay to the
>>> kernel in the normal way. If the cape doesn't have an eeprom, then the
>>> kernel will still use firmware_request(), but userspace needs to already
>>> know which cape is installed.
>>
>> Jane is a really productive hardware engineer - she manages to fix a
>> number of problems with her cape design by spinning different revisions
>> of the cape. Using the flexibility that the DT provides, documents and
>> defines the hardware changes of the cape revisions in the FDT overlay.
>> The loader matches the revision of the cape with the proper FDT overlay
>> so that the drivers are relieved of having to do revision management.
>
> Okay
>
>>> By installing dtc on the Pi, Mandy compiles the overlay for her
>>> prototype hardware. However, she doesn't have a copy of the Pi's
>>> original FDT source, so instead she uses the dtc 'fs' input format to
>>> compile the overlay file against the live DT data in /proc.
>>
>> Jane (the cape designer) can use this too. Developing the cape, she really
>> appreciates that she doesn't have to reboot every time she makes a change
>> in the cape hardware. By removing the FDT overlay, compiling with the dtc
>> on the board, and re-inserting the overlay, she can be more productive by
>> waiting less.
>
> Yes, but I'll leave this paragraph out of the spec. It isn't
> significantly different from what is already there.
>

No problem.

>> Johnny, Jane's little son, doesn't know anything about device trees, linux
>> kernel trees, or hard-core s/w engineering. He is a bright kid, and due to
>> the board having a node.js based educational electronic design kit, he
>> can use the web-based simplified development environment, that allows
>> him graphically to connect the parts in his kit. He can save the design
>> and the IDE creates on the fly the DT overlay for later use.
>
> Yes.
>
>>> Joanne has purchased one of Jane's capes and packaged it into a rugged
>>> case for data logging. As far as Joanne is concerned, the BeagleBone and
>>> cape together are a single unit and she'd prefer a single monolithic FDT
>>> instead of using an FDT overlay.
>>> Option A: Using dtc, she uses the BeagleBone and cape .dts source files
>>> to generate a single .dtb for the entire system which is
>>> loaded by U-Boot. -or-
>> Unlikely.
>>> Option B: Joanne uses a tool to merge the BeagleBone and cape .dtb files
>>> (instead of .dts files), -or-
>> Possible but low probability.
>>> Option C: U-Boot loads both the base and overlay FDT files, merges them,
>>> and passes the resolved tree to the kernel.
>> Could be made to work. Only really required if Joanne wants the
>> cape interface to work for u-boot too. For example if the cape has some
>> kind of network interface that u-boot will use to boot from.
>
> Unlikely for your focus perhaps, but I'm trying to capture all the
> relevant permutations, and I can guarantee that some people really
> will want this. If not on the bone, then on some other platform.
>

No problem there. Certainly they are valid scenarios.

>>> Summary points:
>>> - Create an FDT overlay data format and usage model
>>> - SHALL reliable resolve or validate of phandles between base and
>>> overlay trees
>>> - SHOULD reliably handle changes between different underlying overlays
>>> (ie. what happens to existing .dtb overly files if the structure of
>>> the dtb it is layered over changes. If not possible, then SHALL
>>> detect when the base tree doesn't match and refuse to apply the
>>> overlay.
>>> - dts syntax needs to be extended for overlay .dtb files
>>> - DTC tool needs to be modified to support overlay .dtb generation
>>> - Overlays SHOULD be able to be applied either by firmware or the kernel
>>> - libfdt SHALL be extended to parse and apply overlays
>>
>> - ftdump should be fixed and work for the overlay syntax too.
>
> Okay
>
>> This is much grander in vision that I had in mind :)
>>
>> It can handle our use cases, but I'm worried if we're bitting more
>> that what we can chew. Perhaps a staged approach? I.e. target the
>> low hanging fruit first, get it work, and then work on the hardest
>> parts?
>
> Actually, I'm not to scared about the work and yes I think that it
> *must* be a staged approach. To start focus on adding overlays without
> phandle resolution (phandles must match) or unloading support.
> Unloading and phandle resolution can be separate follow-on features.
> Unloading and phandle resolution are the hard bits anyway.
>

Okay.

>>> It may be sufficient to solve it by making the phandle values less
>>> volatile. Right now dtc generates phandles linearly. Generated phandles
>>> could be overridden with explicit phandle properties, but it isn't a
>>> fantastic solution. Perhaps generating the phandle from a hash of the
>>> node name would be sufficient.
>>>
>>
>> I doubt the hash method will work reliably. We only have 32 bits to work with,
>> nothing like the SHA hashes of git.
>>
>
> I think the biggest trees have on the order of 100 nodes and a 32 bit
> numberspace is 4Gi. Surely collisions can be avoided. :-)
>
> It is also possible to explicitly specify the phandle when the hash
> method breaks down, or if the node full path needs to change, but I'd
> like to avoid that approach as much as possible.
>

Something like

foo = <&bar>;

unresolved-phandle-paths {
bar = "/soc/bar@deadbeef";

}
?

It is not very nice looking admittedly.

Could you explain your hashing method a bit? How will you deal with the
possible conflicts?

>> 5) Have a method to attach FDT overlay to a kernel module.
>>
>> For some drivers it might be better if the kernel module and the
>> DT overlay is packaged in the same file. You be in a part of
>> the module binary as a special section that request_firmware can
>> pick up automatically.
>
> It used to be that firmware blobs could be linked into the kernel and
> request_firmware() would find them. I'd like to investigate if that is
> still possible.
>

It should work. Modules should work too.

>>> Add support to remove an overlay (question: is this important?)
>>>
>>
>> For hot-plugging, you need it. Whether kernel code can deal with
>> large parts of the DT going away... How about we use the dead
>> properties method and move/tag the removed modes as such, and not
>> really remove them.
>
> Nodes already use krefs, and I'm thinking about making them kobjects
> so that they appear in sysfs and we'll have some tools to figure out
> when reference counts don't get decremented properly.
>

>From the little I've looked in the of code, and the drivers, it's going
to be pretty bad. I don't think all users take references properly, and
we have a big global lock for accessing the DT.

Adding and removing nodes at runtime as part of the normal operation of
the system (and not as something that happens once in a blue moon under
controlled conditions) will uncover lots of bugs.

So let's think about locking too.

>>> Workitems:
>>> Modify of_platform_populate() to get new node notifications
>>> Modify of_register_spi_devices to get new node notifications
>>> Modify of_register_i2c_devices to get new node notifications
>>>
>>
>> w1 is the same. Possibly more.
>
> w1?

One-Wire. Very simple sensor bus using one wire.
>
>>
>> Another can of worms is the pinctrl nodes.
>
> Yes... new pinctrl data would need to trigger adding new data to
> pinctrl. I don't know if the pinctrl api supports that.
>

I would be happy for now just being able to move the pinctrl definitions
in the DT fragment. I have been told this has been a topic of discussion and
people decided that their place being all together was better.
I'm hoping we can address this.

>>
>>> 6) Other work
>>> -------------
>>> The device node user space interface could use some work. Here are some
>>> random work items that are peripherally related to the overlay feature.
>>>
>>> Other Workitems:
>>> Define FDT schema syntax
>>> Add FDT schema support to FDT (basically lint-style testing)
>>> Investigate runtime schema validation
>>> Make device_nodes first-class kobjects and remove the procfs interface
>>> - it can be emulated with a symlink
>>> Add symlinks from devices to devicetree nodes in sysfs
>>
>> That's going to take a while :)
>
> :-) But as you've already pointed out this should be taken in a staged
> approach. Simple overlay support is still useful and shouldn't be too
> complex to implement.
>

Famous last words... :)

> g.

One final thing. Some people have expressed concern that DT processing tends
to take some time; time that badly affects booting speed. Perhaps if large
parts of the DT are disabled, or if the system can operate in some manner
using a lazy parsing method, we could look at that too.

Regards

-- Pantelis

2012-11-06 19:41:11

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Russ,

On Nov 6, 2012, at 8:29 PM, Russ Dill wrote:

> On Tue, Nov 6, 2012 at 10:35 AM, Tony Lindgren <[email protected]> wrote:
>> * Grant Likely <[email protected]> [121106 03:16]:
>>> On Tue, Nov 6, 2012 at 10:30 AM, Pantelis Antoniou
>>> <[email protected]> wrote:
>>>>
>>>> Another can of worms is the pinctrl nodes.
>>>
>>> Yes... new pinctrl data would need to trigger adding new data to
>>> pinctrl. I don't know if the pinctrl api supports that.
>>
>> The actual pins stay the same, just their configuration
>> changes. AFAIK all that is already supported using the
>> pinctrl framework.
>>
>> For example, considering hotplugging capes on the beaglebone:
>>
>> 1. You need to map all the sensible modes for the pins exposed
>> to the capes in the board specific .dts file. This will
>> add roughly 4 x nr_capbus_pins named modes in the .dts file
>> so not too bad.
>>
>> 2. Claim all the capebus pins during the capbus driver probe
>> and set them to some safe mode.
>>
>> 3. Try to detect the connected cape(s) over i2c.
>>
>> 4. Use pinctr_select_state to set the desired modes for
>> the pins used by the cape(s).
>>
>> 5. Enable capebus regulators and clocks etc.
>>
>> 6. Load the driver modules for whatever omap internal
>> devices the cape supports.
>>
>> You could also claim the pin for the omap internal
>> devices instead of claiming them in the capebus, but then
>> things can get messy with binding and unbinding the
>> drivers. So just claiming all the pins in the capebus
>> probably keeps things simpler.
>
> That assumes that for a particular external bus, certain pins aren't
> already shared with functions already on the board, for instance if an
> I?C bus brought out to the external bus already has a chip connected
> to it.

This is our case on the bone. We don't enable the peripheral until
a cape that references it is enabled.

I don't think that very big changes are going to be needed TBH.

So now we use:

am3358_pinmux: pinmux@44e10800 {
....
bone_dvi_cape_led_pins: pinmux_bone_dvi_cape_led_pins {
pinctrl-single,pins = <
0x48 0x07 /* gpmc_a2.gpio1_18, OUTPUT | MODE7 */
0x4c 0x07 /* gpmc_a3.gpio1_19, OUTPUT | MODE7 */
>;
};
....
};

And in the cape definition:

pinctrl-0 = <&bone_geiger_cape_pins>;

Ideally if we could do this in the cape definition:

cape_pinmux {
parent = <&am3358_pinmux>;

bone_dvi_cape_led_pins: pinmux_bone_dvi_cape_led_pins {
pinctrl-single,pins = <
0x48 0x07 /* gpmc_a2.gpio1_18, OUTPUT | MODE7 */
0x4c 0x07 /* gpmc_a3.gpio1_19, OUTPUT | MODE7 */
>;
};

pinctrl-0 = <&bone_geiger_cape_pins>;

It would be just fine.

Regards

-- Pantelis

2012-11-06 20:45:31

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Tue, Nov 6, 2012 at 7:34 PM, Pantelis Antoniou
<[email protected]> wrote:
> On Nov 6, 2012, at 12:14 PM, Grant Likely wrote:
>> On Tue, Nov 6, 2012 at 10:30 AM, Pantelis Antoniou
>> <[email protected]> wrote:
>>> For hot-plugging, you need it. Whether kernel code can deal with
>>> large parts of the DT going away... How about we use the dead
>>> properties method and move/tag the removed modes as such, and not
>>> really remove them.
>>
>> Nodes already use krefs, and I'm thinking about making them kobjects
>> so that they appear in sysfs and we'll have some tools to figure out
>> when reference counts don't get decremented properly.
>>
>
> From the little I've looked in the of code, and the drivers, it's going
> to be pretty bad. I don't think all users take references properly, and
> we have a big global lock for accessing the DT.

I'm a lot more optimistic on this front... I wrote a patch today to
make the change and took some measurements:

On the versatile express qemu model I measured the free memory with
/proc/device-tree, with /sys/device-tree, and with both. Here's what I
found:

/proc/device-tree only: 114776kB free
/sys/device-tree only: 114792kB free
both enabled: 114716kB free

The back of a napkin calculation indicates that on this platform
/proc/devicetree costs 76kB and /sys/device-tree costs 60kb. I'm happy
to see that using /sys instead of /proc appears to be slightly cheaper
which makes it easier to justify the change. The diffstat makes me
even happier:

arch/arm/plat-omap/Kconfig | 1 -
arch/powerpc/platforms/pseries/dlpar.c | 23 -----------
arch/powerpc/platforms/pseries/reconfig.c | 40 ------------------
drivers/of/Kconfig | 8 ----
drivers/of/base.c | 116
++++++++++++++++++++++++++++------------------------
drivers/of/fdt.c | 5 ++-
fs/proc/Makefile | 1 -
fs/proc/proc_devtree.c | 13 +-----
fs/proc/root.c | 4 +-
include/linux/of.h | 35 ++++++++++++----
include/linux/proc_fs.h | 16 --------
include/linux/string.h | 11 +++++
12 files changed, 107 insertions(+), 166 deletions(-)

There are still a few odds and ends that need to be tidied up, but
I'll get it out for review shortly. I've not touched the sparc code
yet, and I need to take another look over the existing OF_DYNAMIC
code. I think that CONFIG_OF_DYNAMIC will probably go away and the add
node/property functions will get used by fdt.c and pdt.c for initial
construction of the device tree.

> Adding and removing nodes at runtime as part of the normal operation of
> the system (and not as something that happens once in a blue moon under
> controlled conditions) will uncover lots of bugs.

I'm hoping so! Its time to clean that mess up. :-) Fortunately adding
nodes is not where we're going to have problems. The problems will be
on node removal. Addition-only at least means we can have something
useful before hunting down and squashing all the bugs.

> So let's think about locking too

Yes, the locking does need to be sorted out.

g.

2012-11-06 20:50:40

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Tue, Nov 6, 2012 at 8:45 PM, Grant Likely <[email protected]> wrote:
> The back of a napkin calculation indicates that on this platform
> /proc/devicetree costs 76kB and /sys/device-tree costs 60kb. I'm happy
> to see that using /sys instead of /proc appears to be slightly cheaper
> which makes it easier to justify the change. The diffstat makes me
> even happier:
>
> arch/arm/plat-omap/Kconfig | 1 -
> arch/powerpc/platforms/pseries/dlpar.c | 23 -----------
> arch/powerpc/platforms/pseries/reconfig.c | 40 ------------------
> drivers/of/Kconfig | 8 ----
> drivers/of/base.c | 116
> ++++++++++++++++++++++++++++------------------------
> drivers/of/fdt.c | 5 ++-
> fs/proc/Makefile | 1 -
> fs/proc/proc_devtree.c | 13 +-----
> fs/proc/root.c | 4 +-
> include/linux/of.h | 35 ++++++++++++----
> include/linux/proc_fs.h | 16 --------
> include/linux/string.h | 11 +++++
> 12 files changed, 107 insertions(+), 166 deletions(-)

Make that 96 insertions. I got an extra patch caught up in that diffstat.

g.

2012-11-06 22:18:05

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/06/2012 12:41 PM, Pantelis Antoniou wrote:
> Hi Russ,
>
> On Nov 6, 2012, at 8:29 PM, Russ Dill wrote:
>
>> On Tue, Nov 6, 2012 at 10:35 AM, Tony Lindgren <[email protected]> wrote:
>>> * Grant Likely <[email protected]> [121106 03:16]:
>>>> On Tue, Nov 6, 2012 at 10:30 AM, Pantelis Antoniou
>>>> <[email protected]> wrote:
>>>>>
>>>>> Another can of worms is the pinctrl nodes.
>>>>
>>>> Yes... new pinctrl data would need to trigger adding new data to
>>>> pinctrl. I don't know if the pinctrl api supports that.
>>>
>>> The actual pins stay the same, just their configuration
>>> changes. AFAIK all that is already supported using the
>>> pinctrl framework.
>>>
>>> For example, considering hotplugging capes on the beaglebone:
...
>> That assumes that for a particular external bus, certain pins aren't
>> already shared with functions already on the board, for instance if an
>> I?C bus brought out to the external bus already has a chip connected
>> to it.
>
> This is our case on the bone. We don't enable the peripheral until
> a cape that references it is enabled.
>
> I don't think that very big changes are going to be needed TBH.
...
> Ideally if we could do this in the cape definition:
>
> cape_pinmux {
> parent = <&am3358_pinmux>;

I think the cape overlay would simply add nodes to the existing pin
controller node, so I'd presume you would replace the two lines
immediately above with:

am3358_pinmux: pinmux {

>
> bone_dvi_cape_led_pins: pinmux_bone_dvi_cape_led_pins {
> pinctrl-single,pins = <
> 0x48 0x07 /* gpmc_a2.gpio1_18, OUTPUT | MODE7 */
> 0x4c 0x07 /* gpmc_a3.gpio1_19, OUTPUT | MODE7 */
> >;
> };
>
> pinctrl-0 = <&bone_geiger_cape_pins>;

2012-11-06 22:38:04

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/05/2012 01:40 PM, Grant Likely wrote:
> Hey folks,
>
> As promised, here is my early draft to try and capture what device
> tree overlays need to do and how to get there. Comments and
> suggestions greatly appreciated.

Interesting. This just came up internally at NVIDIA within the last
couple weeks, and was discussed on the U-Boot mailing list very recently
too:

http://lists.denx.de/pipermail/u-boot/2012-October/thread.html#138227
(it spills into the November archive too)

> For these cases it is proposed to implement an overlay feature for the
> so that the initial device tree data can be modified by userspace at

I don't know if you're maintaining this as a document and taking patches
to it, but if so:

"for the so" split across those two lines.

> Jane solves this problem by storing an FDT overlay for each cape in the
> root filesystem. When the kernel detects that a cape is installed it
> reads the cape's eeprom to identify it and uses request_firmware() to
> obtain the appropriate overlay. Userspace passes the overlay to the
> kernel in the normal way. If the cape doesn't have an eeprom, then the
> kernel will still use firmware_request(), but userspace needs to already
> know which cape is installed.

As mentioned by Pantelis, multiple versions of a board is also very
common. We already have the following .dts files in the kernel where
this applies, for the main board even:

arch/arm/boot/dts/tegra30-cardhu.dtsi
arch/arm/boot/dts/tegra30-cardhu-a02.dts
arch/arm/boot/dts/tegra30-cardhu-a04.dts

> Summary points:

> - SHOULD reliably handle changes between different underlying overlays
> (ie. what happens to existing .dtb overly files if the structure of
> the dtb it is layered over changes. If not possible, then SHALL
> detect when the base tree doesn't match and refuse to apply the
> overlay.

Perhaps use (versioned) DT bindings to represent the interface between
the two .dts files? See the links to the U-Boot mailing list discussions
below?

> - What is the model for overlays?
> - Can an overlay modify existing properties?
> - Can an overlay add new properties to existing nodes?
> - Can an overlay delete existing nodes/properties?

This proposal is very oriented at an overlay-based approach. I'm not
totally convinced that a pure overlay approach (as in how dtc does
overlayed DT nodes) will be flexible enough, but would love to be
persuaded. Again see below.

> It may be sufficient to solve it by making the phandle values less
> volatile. Right now dtc generates phandles linearly. Generated phandles
> could be overridden with explicit phandle properties, but it isn't a
> fantastic solution. Perhaps generating the phandle from a hash of the
> node name would be sufficient.

Node names don't have to be unique though right; perhaps hash the
path-name instead of the node-name? But then, why not just reference by
path name; similar to <{&/path/to/node}> rather than <&label>?

> This handles many of the use cases, but it assumes that an overlay is
> board specific. If it ever is required to support multiple base boards
> with a single overlay file then there is a problem. The .dtb overlays
> generated in this manor cannot handle different phandles or nodes that
> are in a different place. On the other hand, the overlay source files
> should have no problem being compiled for multiple targets.

s/manor/manner/

I do rather suspect this use-case is quite common. NVIDIA certainly has
a bunch of development boards with pluggable
PMIC/audio/WiFi/display/..., and I believe there's some ability to
re-use the pluggable components with a variety of base-boards.

Given people within NVIDIA started talking about this recently, I asked
them to enumerate all the boards we have that support pluggable
components, and how common it is that some boards support being plugged
into different main boards. I don't know when that enumeration will
complete (or even start) but hopefully I can provide some feedback on
how common the use-case is for us once it's done.

My earlier thoughts on how to support this included explicit
inter-board/-component connector objects in the .dts files that allow
"renaming" of GPIOs, I2C buses, regulators, etc.:

http://lists.denx.de/pipermail/u-boot/2012-October/138476.html
http://lists.denx.de/pipermail/u-boot/2012-November/138925.html

2012-11-07 00:54:57

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/6/2012 12:37 PM, Stephen Warren wrote:
> On 11/05/2012 01:40 PM, Grant Likely wrote:
>> Hey folks,
>>
>> As promised, here is my early draft to try and capture what device
>> tree overlays need to do and how to get there. Comments and
>> suggestions greatly appreciated.
>
> Interesting. This just came up internally at NVIDIA within the last
> couple weeks, and was discussed on the U-Boot mailing list very recently
> too:
>
> http://lists.denx.de/pipermail/u-boot/2012-October/thread.html#138227
> (it spills into the November archive too)
>
>> For these cases it is proposed to implement an overlay feature for the
>> so that the initial device tree data can be modified by userspace at
>
> I don't know if you're maintaining this as a document and taking patches
> to it, but if so:
>
> "for the so" split across those two lines.
>
>> Jane solves this problem by storing an FDT overlay for each cape in the
>> root filesystem. When the kernel detects that a cape is installed it
>> reads the cape's eeprom to identify it and uses request_firmware() to
>> obtain the appropriate overlay. Userspace passes the overlay to the
>> kernel in the normal way. If the cape doesn't have an eeprom, then the
>> kernel will still use firmware_request(), but userspace needs to already
>> know which cape is installed.
>
> As mentioned by Pantelis, multiple versions of a board is also very
> common. We already have the following .dts files in the kernel where
> this applies, for the main board even:
>
> arch/arm/boot/dts/tegra30-cardhu.dtsi
> arch/arm/boot/dts/tegra30-cardhu-a02.dts
> arch/arm/boot/dts/tegra30-cardhu-a04.dts
>
>> Summary points:
>
>> - SHOULD reliably handle changes between different underlying overlays
>> (ie. what happens to existing .dtb overly files if the structure of
>> the dtb it is layered over changes. If not possible, then SHALL
>> detect when the base tree doesn't match and refuse to apply the
>> overlay.
>
> Perhaps use (versioned) DT bindings to represent the interface between
> the two .dts files? See the links to the U-Boot mailing list discussions
> below?
>
>> - What is the model for overlays?
>> - Can an overlay modify existing properties?
>> - Can an overlay add new properties to existing nodes?
>> - Can an overlay delete existing nodes/properties?
>
> This proposal is very oriented at an overlay-based approach. I'm not
> totally convinced that a pure overlay approach (as in how dtc does
> overlayed DT nodes) will be flexible enough, but would love to be
> persuaded. Again see below.

An overlay approach will not be powerful enough to solve the sorts of
problems that occur when a product goes into full production, becomes a
family, and starts to evolve. Issues like second-source parts that
aren't quite compatible and need to be detected and reported,
board-stuff options for different customer profiles, speed grades of
parts that aren't properly probeable but instead need to be identified
by some subterfuge, the list of tedious issues goes on and on.

It's nice to pretend that the world fits into a few coherent simple
use cases, but 30 years of experience shipping computer product families
proves otherwise. You need a programming language to solve the full
set of problems - which I why the device tree is designed so it can
be generated and modified by a program.

>
>> It may be sufficient to solve it by making the phandle values less
>> volatile. Right now dtc generates phandles linearly. Generated phandles
>> could be overridden with explicit phandle properties, but it isn't a
>> fantastic solution. Perhaps generating the phandle from a hash of the
>> node name would be sufficient.
>
> Node names don't have to be unique though right; perhaps hash the
> path-name instead of the node-name? But then, why not just reference by
> path name; similar to <{&/path/to/node}> rather than <&label>?
>
>> This handles many of the use cases, but it assumes that an overlay is
>> board specific. If it ever is required to support multiple base boards
>> with a single overlay file then there is a problem. The .dtb overlays
>> generated in this manor cannot handle different phandles or nodes that
>> are in a different place. On the other hand, the overlay source files
>> should have no problem being compiled for multiple targets.
>
> s/manor/manner/
>
> I do rather suspect this use-case is quite common. NVIDIA certainly has
> a bunch of development boards with pluggable
> PMIC/audio/WiFi/display/..., and I believe there's some ability to
> re-use the pluggable components with a variety of base-boards.
>
> Given people within NVIDIA started talking about this recently, I asked
> them to enumerate all the boards we have that support pluggable
> components, and how common it is that some boards support being plugged
> into different main boards. I don't know when that enumeration will
> complete (or even start) but hopefully I can provide some feedback on
> how common the use-case is for us once it's done.
>
> My earlier thoughts on how to support this included explicit
> inter-board/-component connector objects in the .dts files that allow
> "renaming" of GPIOs, I2C buses, regulators, etc.:
>
> http://lists.denx.de/pipermail/u-boot/2012-October/138476.html
> http://lists.denx.de/pipermail/u-boot/2012-November/138925.html
> _______________________________________________
> devicetree-discuss mailing list
> [email protected]
> https://lists.ozlabs.org/listinfo/devicetree-discuss
>

2012-11-07 08:07:31

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Grant,

On Nov 6, 2012, at 9:45 PM, Grant Likely wrote:

> On Tue, Nov 6, 2012 at 7:34 PM, Pantelis Antoniou
> <[email protected]> wrote:
>> On Nov 6, 2012, at 12:14 PM, Grant Likely wrote:
>>> On Tue, Nov 6, 2012 at 10:30 AM, Pantelis Antoniou
>>> <[email protected]> wrote:
>>>> For hot-plugging, you need it. Whether kernel code can deal with
>>>> large parts of the DT going away... How about we use the dead
>>>> properties method and move/tag the removed modes as such, and not
>>>> really remove them.
>>>
>>> Nodes already use krefs, and I'm thinking about making them kobjects
>>> so that they appear in sysfs and we'll have some tools to figure out
>>> when reference counts don't get decremented properly.
>>>
>>
>> From the little I've looked in the of code, and the drivers, it's going
>> to be pretty bad. I don't think all users take references properly, and
>> we have a big global lock for accessing the DT.
>
> I'm a lot more optimistic on this front... I wrote a patch today to
> make the change and took some measurements:
>
> On the versatile express qemu model I measured the free memory with
> /proc/device-tree, with /sys/device-tree, and with both. Here's what I
> found:
>
> /proc/device-tree only: 114776kB free
> /sys/device-tree only: 114792kB free
> both enabled: 114716kB free
>
> The back of a napkin calculation indicates that on this platform
> /proc/devicetree costs 76kB and /sys/device-tree costs 60kb. I'm happy
> to see that using /sys instead of /proc appears to be slightly cheaper
> which makes it easier to justify the change. The diffstat makes me
> even happier:
>
> arch/arm/plat-omap/Kconfig | 1 -
> arch/powerpc/platforms/pseries/dlpar.c | 23 -----------
> arch/powerpc/platforms/pseries/reconfig.c | 40 ------------------
> drivers/of/Kconfig | 8 ----
> drivers/of/base.c | 116
> ++++++++++++++++++++++++++++------------------------
> drivers/of/fdt.c | 5 ++-
> fs/proc/Makefile | 1 -
> fs/proc/proc_devtree.c | 13 +-----
> fs/proc/root.c | 4 +-
> include/linux/of.h | 35 ++++++++++++----
> include/linux/proc_fs.h | 16 --------
> include/linux/string.h | 11 +++++
> 12 files changed, 107 insertions(+), 166 deletions(-)
>

Interesting. Not so bad then.

> There are still a few odds and ends that need to be tidied up, but
> I'll get it out for review shortly. I've not touched the sparc code
> yet, and I need to take another look over the existing OF_DYNAMIC
> code. I think that CONFIG_OF_DYNAMIC will probably go away and the add
> node/property functions will get used by fdt.c and pdt.c for initial
> construction of the device tree.

CONFIG_OF_DYNAMIC never made sense to me. Glad to see the config option
gone. I'm not totally up to date with the -next dt stuff, but if we're
there can we rename all the prom_ functions to something saner?

>
>> Adding and removing nodes at runtime as part of the normal operation of
>> the system (and not as something that happens once in a blue moon under
>> controlled conditions) will uncover lots of bugs.
>
> I'm hoping so! Its time to clean that mess up. :-) Fortunately adding
> nodes is not where we're going to have problems. The problems will be
> on node removal. Addition-only at least means we can have something
> useful before hunting down and squashing all the bugs.

I'll admit that removing nodes is going to be quite rare at least for
me use cases. I did come across a couple of people that do hot-plugging
(using something completely different) that could use it for sure.

>
>> So let's think about locking too
>
> Yes, the locking does need to be sorted out.
>

Perhaps come up with a dt-stress test tool/boot time validator?

> g.

Regards

-- Pantelis

P.S. Lots of teeth grinding in the ELCE about the lack of a DT preprocessor.
The pinctrl arguments have been mentioned more than once.

2012-11-07 08:13:54

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Grant

On Nov 6, 2012, at 9:45 PM, Grant Likely wrote:

> On Tue, Nov 6, 2012 at 7:34 PM, Pantelis Antoniou
> <[email protected]> wrote:

[ snip ]
>
> g.

Since we've started talking about longer term goals, and the versioning
provision seems to stand, I hope we address how much the fragment versioning
thing is similar to the way board revisions progress.

If a versioning syntax is available then one could create a single DT
file for a bunch of 'almost' similar board and board revisions.

Using a single DTB in the same manner you have a single uImage would
make some people quite happy, since you won't have to do any bootloader
magic to make sure you pass the correct DTB for the specific revision.

Regards

-- Pantelis

2012-11-07 08:48:24

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Stephen,

On Nov 6, 2012, at 11:37 PM, Stephen Warren wrote:

> On 11/05/2012 01:40 PM, Grant Likely wrote:
>> Hey folks,
>>
>> As promised, here is my early draft to try and capture what device
>> tree overlays need to do and how to get there. Comments and
>> suggestions greatly appreciated.
>
> Interesting. This just came up internally at NVIDIA within the last
> couple weeks, and was discussed on the U-Boot mailing list very recently
> too:
>
> http://lists.denx.de/pipermail/u-boot/2012-October/thread.html#138227
> (it spills into the November archive too)

I am aware of this discussion. For our use case u-boot DT manipulation was
tried, but then abandoned. Asking our user base to modify anything in u-boot
was ruled out.

>
>> For these cases it is proposed to implement an overlay feature for the
>> so that the initial device tree data can be modified by userspace at
>
> I don't know if you're maintaining this as a document and taking patches
> to it, but if so:
>
> "for the so" split across those two lines.
>
>> Jane solves this problem by storing an FDT overlay for each cape in the
>> root filesystem. When the kernel detects that a cape is installed it
>> reads the cape's eeprom to identify it and uses request_firmware() to
>> obtain the appropriate overlay. Userspace passes the overlay to the
>> kernel in the normal way. If the cape doesn't have an eeprom, then the
>> kernel will still use firmware_request(), but userspace needs to already
>> know which cape is installed.
>
> As mentioned by Pantelis, multiple versions of a board is also very
> common. We already have the following .dts files in the kernel where
> this applies, for the main board even:
>
> arch/arm/boot/dts/tegra30-cardhu.dtsi
> arch/arm/boot/dts/tegra30-cardhu-a02.dts
> arch/arm/boot/dts/tegra30-cardhu-a04.dts
>

Exactly. I've made this point in another email, but IMHO board-revision
management is exactly the same with cape revision management.

Ideally you'd like to get rid of those three, and replace it with only
one that's properly versioned.

>> Summary points:
>
>> - SHOULD reliably handle changes between different underlying overlays
>> (ie. what happens to existing .dtb overly files if the structure of
>> the dtb it is layered over changes. If not possible, then SHALL
>> detect when the base tree doesn't match and refuse to apply the
>> overlay.
>
> Perhaps use (versioned) DT bindings to represent the interface between
> the two .dts files? See the links to the U-Boot mailing list discussions
> below?
>
>> - What is the model for overlays?
>> - Can an overlay modify existing properties?
>> - Can an overlay add new properties to existing nodes?
>> - Can an overlay delete existing nodes/properties?
>
> This proposal is very oriented at an overlay-based approach. I'm not
> totally convinced that a pure overlay approach (as in how dtc does
> overlayed DT nodes) will be flexible enough, but would love to be
> persuaded. Again see below.
>
>> It may be sufficient to solve it by making the phandle values less
>> volatile. Right now dtc generates phandles linearly. Generated phandles
>> could be overridden with explicit phandle properties, but it isn't a
>> fantastic solution. Perhaps generating the phandle from a hash of the
>> node name would be sufficient.
>
> Node names don't have to be unique though right; perhaps hash the
> path-name instead of the node-name? But then, why not just reference by
> path name; similar to <{&/path/to/node}> rather than <&label>?
>

It would work for references to the known base DTS. If you have a
cape that's cross-device compatible that can simply fail.

I like this for it's simplicity though.

>> This handles many of the use cases, but it assumes that an overlay is
>> board specific. If it ever is required to support multiple base boards
>> with a single overlay file then there is a problem. The .dtb overlays
>> generated in this manor cannot handle different phandles or nodes that
>> are in a different place. On the other hand, the overlay source files
>> should have no problem being compiled for multiple targets.
>
> s/manor/manner/
>
> I do rather suspect this use-case is quite common. NVIDIA certainly has
> a bunch of development boards with pluggable
> PMIC/audio/WiFi/display/..., and I believe there's some ability to
> re-use the pluggable components with a variety of base-boards.
>
> Given people within NVIDIA started talking about this recently, I asked
> them to enumerate all the boards we have that support pluggable
> components, and how common it is that some boards support being plugged
> into different main boards. I don't know when that enumeration will
> complete (or even start) but hopefully I can provide some feedback on
> how common the use-case is for us once it's done.
>
> My earlier thoughts on how to support this included explicit
> inter-board/-component connector objects in the .dts files that allow
> "renaming" of GPIOs, I2C buses, regulators, etc.:
>
> http://lists.denx.de/pipermail/u-boot/2012-October/138476.html
> http://lists.denx.de/pipermail/u-boot/2012-November/138925.html

Sounds very similar to our case.

Very interesting to say the least ;)

Regards

-- Pantelis

2012-11-07 10:19:26

by Benoit Cousson

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Panto,

On 11/07/2012 09:13 AM, Pantelis Antoniou wrote:
> Hi Grant
>
> On Nov 6, 2012, at 9:45 PM, Grant Likely wrote:
>
>> On Tue, Nov 6, 2012 at 7:34 PM, Pantelis Antoniou
>> <[email protected]> wrote:
>
> [ snip ]
>>
>> g.
>
> Since we've started talking about longer term goals, and the versioning
> provision seems to stand, I hope we address how much the fragment versioning
> thing is similar to the way board revisions progress.
>
> If a versioning syntax is available then one could create a single DT
> file for a bunch of 'almost' similar board and board revisions.

I even think that the version issue is probably much more important for the short term than the overlay aspect. Well at least as important. We start having as well a bunch a panda board version with different HW setup.

Having a single board-XXX.dts that will support all these versions is probably the best approach to avoid choosing that from the bootloader.

We need to figure out a format + mechanism compatible with the current non-versioned format to allow filtering the nodes at runtime to keep only the relevant one.

Something that can find the driver that will provide the proper board version or subsystem version or whatever like that:

compatible-version = "ti,panda-version", "panda";

Then at runtime we should create only the node with the correct match between the driver version and the string version.

/* regular panda audio routing */
sound: sound {
compatible = "ti,abe-twl6040";
ti,model = "PandaBoard";
compatible-version = "ti,panda-version", "panda";

/* Audio routing */
ti,audio-routing =
"Headset Stereophone", "HSOL",
"Headset Stereophone", "HSOR",
"Ext Spk", "HFL",
"Ext Spk", "HFR",
"Line Out", "AUXL",
"Line Out", "AUXR",
"HSMIC", "Headset Mic",
"Headset Mic", "Headset Mic Bias",
"AFML", "Line In",
"AFMR", "Line In";
};

/* Audio routing is different between PandaBoard4430 and PandaBoardES */
&sound {
ti,model = "PandaBoardES";
compatible-version = "ti,panda-version", "panda-es";

/* Audio routing */
ti,audio-routing =
"Headset Stereophone", "HSOL",
"Headset Stereophone", "HSOR",
"Ext Spk", "HFL",
"Ext Spk", "HFR",
"Line Out", "AUXL",
"Line Out", "AUXR",
"AFML", "Line In",
"AFMR", "Line In";
};

Maybe some extra version match table can just be passed during the board machine_init

of_platform_populate(NULL, omap_dt_match_table, NULL, NULL, panda_version_match_table);

Regards,
Benoit

2012-11-07 11:02:40

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Benoit,

On Nov 7, 2012, at 11:19 AM, Benoit Cousson wrote:

> Hi Panto,
>
> On 11/07/2012 09:13 AM, Pantelis Antoniou wrote:
>> Hi Grant
>>
>> On Nov 6, 2012, at 9:45 PM, Grant Likely wrote:
>>
>>> On Tue, Nov 6, 2012 at 7:34 PM, Pantelis Antoniou
>>> <[email protected]> wrote:
>>
>> [ snip ]
>>>
>>> g.
>>
>> Since we've started talking about longer term goals, and the versioning
>> provision seems to stand, I hope we address how much the fragment versioning
>> thing is similar to the way board revisions progress.
>>
>> If a versioning syntax is available then one could create a single DT
>> file for a bunch of 'almost' similar board and board revisions.
>
> I even think that the version issue is probably much more important for the short term than the overlay aspect. Well at least as important. We start having as well a bunch a panda board version with different HW setup.
>
> Having a single board-XXX.dts that will support all these versions is probably the best approach to avoid choosing that from the bootloader.
>
> We need to figure out a format + mechanism compatible with the current non-versioned format to allow filtering the nodes at runtime to keep only the relevant one.
>
> Something that can find the driver that will provide the proper board version or subsystem version or whatever like that:
>
> compatible-version = "ti,panda-version", "panda";
>
> Then at runtime we should create only the node with the correct match between the driver version and the string version.
>
>

This is exactly what we need. FWIW the capebus syntax is a little bit different.

> /* regular panda audio routing */
> sound: sound {
> compatible = "ti,abe-twl6040";
> ti,model = "PandaBoard";
> compatible-version = "ti,panda-version", "panda";
>
> /* Audio routing */
> ti,audio-routing =
> "Headset Stereophone", "HSOL",
> "Headset Stereophone", "HSOR",
> "Ext Spk", "HFL",
> "Ext Spk", "HFR",
> "Line Out", "AUXL",
> "Line Out", "AUXR",
> "HSMIC", "Headset Mic",
> "Headset Mic", "Headset Mic Bias",
> "AFML", "Line In",
> "AFMR", "Line In";
> };
>
>
> /* Audio routing is different between PandaBoard4430 and PandaBoardES */
> &sound {
> ti,model = "PandaBoardES";
> compatible-version = "ti,panda-version", "panda-es";
>
> /* Audio routing */
> ti,audio-routing =
> "Headset Stereophone", "HSOL",
> "Headset Stereophone", "HSOR",
> "Ext Spk", "HFL",
> "Ext Spk", "HFR",
> "Line Out", "AUXL",
> "Line Out", "AUXR",
> "AFML", "Line In",
> "AFMR", "Line In";
> };
>

We use this syntax for capebus (totally non-standard of-course),

sound: sound {
compatible = "ti,abe-twl6040";

model@0 {
ti,model = "PandaBoard";
ti,audio-routing =
"Headset Stereophone", "HSOL",
"Headset Stereophone", "HSOR",
"Ext Spk", "HFL",
"Ext Spk", "HFR",
"Line Out", "AUXL",
"Line Out", "AUXR",
"HSMIC", "Headset Mic",
"Headset Mic", "Headset Mic Bias",
"AFML", "Line In",
"AFMR", "Line In";
};

model@1 {
ti,model = "PandaBoardES";
ti,audio-routing =
"Headset Stereophone", "HSOL",
"Headset Stereophone", "HSOR",
"Ext Spk", "HFL",
"Ext Spk", "HFR",
"Line Out", "AUXL",
"Line Out", "AUXR",
"AFML", "Line In",
"AFMR", "Line In";
};

/* common properties for either model */
};

The difference is that you don't get to repeat the common properties.

I don't know if this breaks any conventions but seems to work fine for our case.

>
> Maybe some extra version match table can just be passed during the board machine_init
>
> of_platform_populate(NULL, omap_dt_match_table, NULL, NULL, panda_version_match_table);
>

Would we need explicit of_platform_populate calls if we have node modification notifiers?
In that case the notifier would pick it up automatically, and can do the per
version matching internally.

>
> Regards,
> Benoit

Regards

-- Pantelis

2012-11-07 11:13:17

by Benoit Cousson

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/07/2012 12:02 PM, Pantelis Antoniou wrote:
> Hi Benoit,
>
> On Nov 7, 2012, at 11:19 AM, Benoit Cousson wrote:
>
>> Hi Panto,
>>
>> On 11/07/2012 09:13 AM, Pantelis Antoniou wrote:
>>> Hi Grant
>>>
>>> On Nov 6, 2012, at 9:45 PM, Grant Likely wrote:
>>>
>>>> On Tue, Nov 6, 2012 at 7:34 PM, Pantelis Antoniou
>>>> <[email protected]> wrote:
>>>
>>> [ snip ]
>>>>
>>>> g.
>>>
>>> Since we've started talking about longer term goals, and the versioning
>>> provision seems to stand, I hope we address how much the fragment versioning
>>> thing is similar to the way board revisions progress.
>>>
>>> If a versioning syntax is available then one could create a single DT
>>> file for a bunch of 'almost' similar board and board revisions.
>>
>> I even think that the version issue is probably much more important for the short term than the overlay aspect. Well at least as important. We start having as well a bunch a panda board version with different HW setup.
>>
>> Having a single board-XXX.dts that will support all these versions is probably the best approach to avoid choosing that from the bootloader.
>>
>> We need to figure out a format + mechanism compatible with the current non-versioned format to allow filtering the nodes at runtime to keep only the relevant one.
>>
>> Something that can find the driver that will provide the proper board version or subsystem version or whatever like that:
>>
>> compatible-version = "ti,panda-version", "panda";
>>
>> Then at runtime we should create only the node with the correct match between the driver version and the string version.
>>
>>
>
> This is exactly what we need. FWIW the capebus syntax is a little bit different.
>
>> /* regular panda audio routing */
>> sound: sound {
>> compatible = "ti,abe-twl6040";
>> ti,model = "PandaBoard";
>> compatible-version = "ti,panda-version", "panda";
>>
>> /* Audio routing */
>> ti,audio-routing =
>> "Headset Stereophone", "HSOL",
>> "Headset Stereophone", "HSOR",
>> "Ext Spk", "HFL",
>> "Ext Spk", "HFR",
>> "Line Out", "AUXL",
>> "Line Out", "AUXR",
>> "HSMIC", "Headset Mic",
>> "Headset Mic", "Headset Mic Bias",
>> "AFML", "Line In",
>> "AFMR", "Line In";
>> };
>>
>>
>> /* Audio routing is different between PandaBoard4430 and PandaBoardES */
>> &sound {
>> ti,model = "PandaBoardES";
>> compatible-version = "ti,panda-version", "panda-es";
>>
>> /* Audio routing */
>> ti,audio-routing =
>> "Headset Stereophone", "HSOL",
>> "Headset Stereophone", "HSOR",
>> "Ext Spk", "HFL",
>> "Ext Spk", "HFR",
>> "Line Out", "AUXL",
>> "Line Out", "AUXR",
>> "AFML", "Line In",
>> "AFMR", "Line In";
>> };
>>
>
> We use this syntax for capebus (totally non-standard of-course),
>
> sound: sound {
> compatible = "ti,abe-twl6040";
>
>
> model@0 {
> ti,model = "PandaBoard";
> ti,audio-routing =
> "Headset Stereophone", "HSOL",
> "Headset Stereophone", "HSOR",
> "Ext Spk", "HFL",
> "Ext Spk", "HFR",
> "Line Out", "AUXL",
> "Line Out", "AUXR",
> "HSMIC", "Headset Mic",
> "Headset Mic", "Headset Mic Bias",
> "AFML", "Line In",
> "AFMR", "Line In";
> };
>
> model@1 {
> ti,model = "PandaBoardES";
> ti,audio-routing =
> "Headset Stereophone", "HSOL",
> "Headset Stereophone", "HSOR",
> "Ext Spk", "HFL",
> "Ext Spk", "HFR",
> "Line Out", "AUXL",
> "Line Out", "AUXR",
> "AFML", "Line In",
> "AFMR", "Line In";
> };
>
> /* common properties for either model */
> };
>
> The difference is that you don't get to repeat the common properties.
>
> I don't know if this breaks any conventions but seems to work fine for our case.

Yeah, my main concern with that approach is that you change the
structure of the tree by adding an extra node/hierarchy that will not be
there in case of non-versioned tree.
That's why I think we should have something lighter that will not change
the structure.
Ideally we should be able to add extra versioned node to the original
dts without changing it at all.

>> Maybe some extra version match table can just be passed during the board machine_init
>>
>> of_platform_populate(NULL, omap_dt_match_table, NULL, NULL, panda_version_match_table);
>>
>
> Would we need explicit of_platform_populate calls if we have node modification notifiers?
> In that case the notifier would pick it up automatically, and can do the per
> version matching internally.

Yes indeed, but here I was thinking about an intermediate step, i.e.
now, without any dynamic node insertion mechanism.
Thanks to this simple approach, when can already fix the board
versionning problem.

Regards,
Benoit

2012-11-07 11:24:03

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Benoit,

On Nov 7, 2012, at 12:12 PM, Benoit Cousson wrote:

> On 11/07/2012 12:02 PM, Pantelis Antoniou wrote:
>> Hi Benoit,
>>

[snip]

>> I don't know if this breaks any conventions but seems to work fine for our case.
>
> Yeah, my main concern with that approach is that you change the
> structure of the tree by adding an extra node/hierarchy that will not be
> there in case of non-versioned tree.
> That's why I think we should have something lighter that will not change
> the structure.
> Ideally we should be able to add extra versioned node to the original
> dts without changing it at all.
>

You will still need the versioned nodes to be injected to the non-versioned
ones. FWIW the driver will use the standard of_property_read_* interface.

You can patch of_property_read to hide the version node matching, and it will
work.

I'll leave Grant answer what approach is better, I don't claim to have the insight
to handle all cases.

>>> Maybe some extra version match table can just be passed during the board machine_init
>>>
>>> of_platform_populate(NULL, omap_dt_match_table, NULL, NULL, panda_version_match_table);
>>>
>>
>> Would we need explicit of_platform_populate calls if we have node modification notifiers?
>> In that case the notifier would pick it up automatically, and can do the per
>> version matching internally.
>
> Yes indeed, but here I was thinking about an intermediate step, i.e.
> now, without any dynamic node insertion mechanism.
> Thanks to this simple approach, when can already fix the board
> versionning problem.
>

As I pointed, with a kind of injection mechanism. the versioned node contents end up in
the proper place in the device tree.
Your method will work in a much more simpler way.

> Regards,
> Benoit
>

Regards

-- Pantelis

2012-11-07 15:36:07

by Alan Tull

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Wed, 2012-11-07 at 09:06 +0100, Pantelis Antoniou wrote:
> Hi Grant,
>
> On Nov 6, 2012, at 9:45 PM, Grant Likely wrote:
>
> > On Tue, Nov 6, 2012 at 7:34 PM, Pantelis Antoniou
> > <[email protected]> wrote:
> >> On Nov 6, 2012, at 12:14 PM, Grant Likely wrote:
> >>> On Tue, Nov 6, 2012 at 10:30 AM, Pantelis Antoniou
> >>> <[email protected]> wrote:
> >>>> For hot-plugging, you need it. Whether kernel code can deal with
> >>>> large parts of the DT going away... How about we use the dead
> >>>> properties method and move/tag the removed modes as such, and not
> >>>> really remove them.
> >>>
> >>> Nodes already use krefs, and I'm thinking about making them kobjects
> >>> so that they appear in sysfs and we'll have some tools to figure out
> >>> when reference counts don't get decremented properly.
> >>>
> >>
> >> From the little I've looked in the of code, and the drivers, it's going
> >> to be pretty bad. I don't think all users take references properly, and
> >> we have a big global lock for accessing the DT.
> >
> > I'm a lot more optimistic on this front... I wrote a patch today to
> > make the change and took some measurements:
> >
> > On the versatile express qemu model I measured the free memory with
> > /proc/device-tree, with /sys/device-tree, and with both. Here's what I
> > found:
> >
> > /proc/device-tree only: 114776kB free
> > /sys/device-tree only: 114792kB free
> > both enabled: 114716kB free
> >
> > The back of a napkin calculation indicates that on this platform
> > /proc/devicetree costs 76kB and /sys/device-tree costs 60kb. I'm happy
> > to see that using /sys instead of /proc appears to be slightly cheaper
> > which makes it easier to justify the change. The diffstat makes me
> > even happier:
> >
> > arch/arm/plat-omap/Kconfig | 1 -
> > arch/powerpc/platforms/pseries/dlpar.c | 23 -----------
> > arch/powerpc/platforms/pseries/reconfig.c | 40 ------------------
> > drivers/of/Kconfig | 8 ----
> > drivers/of/base.c | 116
> > ++++++++++++++++++++++++++++------------------------
> > drivers/of/fdt.c | 5 ++-
> > fs/proc/Makefile | 1 -
> > fs/proc/proc_devtree.c | 13 +-----
> > fs/proc/root.c | 4 +-
> > include/linux/of.h | 35 ++++++++++++----
> > include/linux/proc_fs.h | 16 --------
> > include/linux/string.h | 11 +++++
> > 12 files changed, 107 insertions(+), 166 deletions(-)
> >
>
> Interesting. Not so bad then.
>
> > There are still a few odds and ends that need to be tidied up, but
> > I'll get it out for review shortly. I've not touched the sparc code
> > yet, and I need to take another look over the existing OF_DYNAMIC
> > code. I think that CONFIG_OF_DYNAMIC will probably go away and the add
> > node/property functions will get used by fdt.c and pdt.c for initial
> > construction of the device tree.
>
> CONFIG_OF_DYNAMIC never made sense to me. Glad to see the config option
> gone. I'm not totally up to date with the -next dt stuff, but if we're
> there can we rename all the prom_ functions to something saner?
>
I'd be glad to see the dynamic stuff be integrated better than it
currently is. Currently unflatten_dt_node assumes that the flattened
tree will stay around and not be freed. So using it to unflatten dynamic
nodes after boot is problematic if the nodes may be later removed.

> >
> >> Adding and removing nodes at runtime as part of the normal operation of
> >> the system (and not as something that happens once in a blue moon under
> >> controlled conditions) will uncover lots of bugs.
> >
> > I'm hoping so! Its time to clean that mess up. :-) Fortunately adding
> > nodes is not where we're going to have problems. The problems will be
> > on node removal. Addition-only at least means we can have something
> > useful before hunting down and squashing all the bugs.
>
> I'll admit that removing nodes is going to be quite rare at least for
> me use cases. I did come across a couple of people that do hot-plugging
> (using something completely different) that could use it for sure.
>

We have been looking at an approach that would allow adding/deleting
whole nodes under userspace direction. Such as a variation of the real-time
video processing system example where the FPGA has some IP blocks that are
dsp filters. The user could rearrange them from
some userspace GUI without having to reboot. For that to work, device nodes that
are removed really have to be freed or there's a memory leak.

> >
> >> So let's think about locking too
> >
> > Yes, the locking does need to be sorted out.
> >
>
> Perhaps come up with a dt-stress test tool/boot time validator?
>
> > g.
>
> Regards
>
> -- Pantelis
>
> P.S. Lots of teeth grinding in the ELCE about the lack of a DT preprocessor.
> The pinctrl arguments have been mentioned more than once.
>
>
> _______________________________________________
> devicetree-discuss mailing list
> [email protected]
> https://lists.ozlabs.org/listinfo/devicetree-discuss
>

2012-11-07 17:18:15

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/07/2012 01:47 AM, Pantelis Antoniou wrote:
> Hi Stephen,
>
> On Nov 6, 2012, at 11:37 PM, Stephen Warren wrote:
>
>> On 11/05/2012 01:40 PM, Grant Likely wrote:
>>> Hey folks,
>>>
>>> As promised, here is my early draft to try and capture what device
>>> tree overlays need to do and how to get there. Comments and
>>> suggestions greatly appreciated.
>>
>> Interesting. This just came up internally at NVIDIA within the last
>> couple weeks, and was discussed on the U-Boot mailing list very recently
>> too:
>>
>> http://lists.denx.de/pipermail/u-boot/2012-October/thread.html#138227
>> (it spills into the November archive too)
>
> I am aware of this discussion. For our use case u-boot DT manipulation was
> tried, but then abandoned. Asking our user base to modify anything in u-boot
> was ruled out.
>
>>
>>> For these cases it is proposed to implement an overlay feature for the
>>> so that the initial device tree data can be modified by userspace at
>>
>> I don't know if you're maintaining this as a document and taking patches
>> to it, but if so:
>>
>> "for the so" split across those two lines.
>>
>>> Jane solves this problem by storing an FDT overlay for each cape in the
>>> root filesystem. When the kernel detects that a cape is installed it
>>> reads the cape's eeprom to identify it and uses request_firmware() to
>>> obtain the appropriate overlay. Userspace passes the overlay to the
>>> kernel in the normal way. If the cape doesn't have an eeprom, then the
>>> kernel will still use firmware_request(), but userspace needs to already
>>> know which cape is installed.
>>
>> As mentioned by Pantelis, multiple versions of a board is also very
>> common. We already have the following .dts files in the kernel where
>> this applies, for the main board even:
>>
>> arch/arm/boot/dts/tegra30-cardhu.dtsi
>> arch/arm/boot/dts/tegra30-cardhu-a02.dts
>> arch/arm/boot/dts/tegra30-cardhu-a04.dts
>
> Exactly. I've made this point in another email, but IMHO board-revision
> management is exactly the same with cape revision management.
>
> Ideally you'd like to get rid of those three, and replace it with only
> one that's properly versioned.

I don't expect we would ever get rid of some of those .dts files; there
is after all a common subset that applies to all boards, and an
incremental difference that applies to only A02/3, and another for
A04/5/... Representing those as separate source files seems appropriate
to me. If we try and dump all the multiple versions into a single file
with some markup indicating which version of the board some sub-sections
of the .dts apply to, I think we'll end up with rather complex .dts
files. In this case, the simple overlay model works extremely well.

2012-11-07 17:25:29

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/07/2012 03:19 AM, Benoit Cousson wrote:
> Hi Panto,
>
> On 11/07/2012 09:13 AM, Pantelis Antoniou wrote:
>> Hi Grant
>>
>> On Nov 6, 2012, at 9:45 PM, Grant Likely wrote:
>>
>>> On Tue, Nov 6, 2012 at 7:34 PM, Pantelis Antoniou
>>> <[email protected]> wrote:
>>
>> [ snip ]
>>>
>>> g.
>>
>> Since we've started talking about longer term goals, and the versioning
>> provision seems to stand, I hope we address how much the fragment versioning
>> thing is similar to the way board revisions progress.
>>
>> If a versioning syntax is available then one could create a single DT
>> file for a bunch of 'almost' similar board and board revisions.
>
> I even think that the version issue is probably much more important for the short term than the overlay aspect. Well at least as important. We start having as well a bunch a panda board version with different HW setup.
>
> Having a single board-XXX.dts that will support all these versions is probably the best approach to avoid choosing that from the bootloader.
>
> We need to figure out a format + mechanism compatible with the current non-versioned format to allow filtering the nodes at runtime to keep only the relevant one.
>
> Something that can find the driver that will provide the proper board version or subsystem version or whatever like that:
>
> compatible-version = "ti,panda-version", "panda";
>
> Then at runtime we should create only the node with the correct match between the driver version and the string version.
>
>
> /* regular panda audio routing */
> sound: sound {
> compatible = "ti,abe-twl6040";
> ti,model = "PandaBoard";
> compatible-version = "ti,panda-version", "panda";
>
> /* Audio routing */
> ti,audio-routing =
> "Headset Stereophone", "HSOL",
> "Headset Stereophone", "HSOR",
> "Ext Spk", "HFL",
> "Ext Spk", "HFR",
> "Line Out", "AUXL",
> "Line Out", "AUXR",
> "HSMIC", "Headset Mic",
> "Headset Mic", "Headset Mic Bias",
> "AFML", "Line In",
> "AFMR", "Line In";
> };
>
>
> /* Audio routing is different between PandaBoard4430 and PandaBoardES */
> &sound {
> ti,model = "PandaBoardES";
> compatible-version = "ti,panda-version", "panda-es";
>
> /* Audio routing */
> ti,audio-routing =
> "Headset Stereophone", "HSOL",
> "Headset Stereophone", "HSOR",
> "Ext Spk", "HFL",
> "Ext Spk", "HFR",
> "Line Out", "AUXL",
> "Line Out", "AUXR",
> "AFML", "Line In",
> "AFMR", "Line In";
> };
>
>
> Maybe some extra version match table can just be passed during the board machine_init
>
> of_platform_populate(NULL, omap_dt_match_table, NULL, NULL, panda_version_match_table);

Is the only difference here the content of the ti,audio-routing
property? If so, then I don't think there's any need for infra-structure
for this; the driver code already reads that property and adjusts its
behaviour based upon it.

I do see that "Headset Mic" exists only in one of the tables above.
Perhaps the driver could scan the routing table, and only create the
ASoC object for the headset if it's mentioned in the routing table?

If there are additional differences, then you can always use the .data
field in of_device_id to automatically associate some configuration data
with different compatible values.

Even better might be to extend the binding so that all HW differences
are represented explicitly as properties; that way you wouldn't even
need different compatible values.

2012-11-07 22:08:21

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Stephen,

On Nov 7, 2012, at 6:18 PM, Stephen Warren wrote:

> On 11/07/2012 01:47 AM, Pantelis Antoniou wrote:
>> Hi Stephen,
>>
>> On Nov 6, 2012, at 11:37 PM, Stephen Warren wrote:
>>
>>> On 11/05/2012 01:40 PM, Grant Likely wrote:
>>>> Hey folks,
>>>>
>>>> As promised, here is my early draft to try and capture what device
>>>> tree overlays need to do and how to get there. Comments and
>>>> suggestions greatly appreciated.
>>>
>>> Interesting. This just came up internally at NVIDIA within the last
>>> couple weeks, and was discussed on the U-Boot mailing list very recently
>>> too:
>>>
>>> http://lists.denx.de/pipermail/u-boot/2012-October/thread.html#138227
>>> (it spills into the November archive too)
>>
>> I am aware of this discussion. For our use case u-boot DT manipulation was
>> tried, but then abandoned. Asking our user base to modify anything in u-boot
>> was ruled out.
>>
>>>
>>>> For these cases it is proposed to implement an overlay feature for the
>>>> so that the initial device tree data can be modified by userspace at
>>>
>>> I don't know if you're maintaining this as a document and taking patches
>>> to it, but if so:
>>>
>>> "for the so" split across those two lines.
>>>
>>>> Jane solves this problem by storing an FDT overlay for each cape in the
>>>> root filesystem. When the kernel detects that a cape is installed it
>>>> reads the cape's eeprom to identify it and uses request_firmware() to
>>>> obtain the appropriate overlay. Userspace passes the overlay to the
>>>> kernel in the normal way. If the cape doesn't have an eeprom, then the
>>>> kernel will still use firmware_request(), but userspace needs to already
>>>> know which cape is installed.
>>>
>>> As mentioned by Pantelis, multiple versions of a board is also very
>>> common. We already have the following .dts files in the kernel where
>>> this applies, for the main board even:
>>>
>>> arch/arm/boot/dts/tegra30-cardhu.dtsi
>>> arch/arm/boot/dts/tegra30-cardhu-a02.dts
>>> arch/arm/boot/dts/tegra30-cardhu-a04.dts
>>
>> Exactly. I've made this point in another email, but IMHO board-revision
>> management is exactly the same with cape revision management.
>>
>> Ideally you'd like to get rid of those three, and replace it with only
>> one that's properly versioned.
>
> I don't expect we would ever get rid of some of those .dts files; there
> is after all a common subset that applies to all boards, and an
> incremental difference that applies to only A02/3, and another for
> A04/5/... Representing those as separate source files seems appropriate
> to me. If we try and dump all the multiple versions into a single file
> with some markup indicating which version of the board some sub-sections
> of the .dts apply to, I think we'll end up with rather complex .dts
> files. In this case, the simple overlay model works extremely well.

If that's your use case, fine. We're not really trying to impose any
special format for the DT files. We think that for our use cases
a single DT works better, not so much from the developer point of view
but from the users. A single DT is much easier to support, especially
for users that aren't so great at boot-loader option juggling.

Regards

-- Pantelis

2012-11-07 22:10:40

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Stephen,

On Nov 7, 2012, at 6:25 PM, Stephen Warren wrote:

> On 11/07/2012 03:19 AM, Benoit Cousson wrote:
>> Hi Panto,
>>
>> On 11/07/2012 09:13 AM, Pantelis Antoniou wrote:
>>> Hi Grant
>>>
>>> On Nov 6, 2012, at 9:45 PM, Grant Likely wrote:
>>>
>>>> On Tue, Nov 6, 2012 at 7:34 PM, Pantelis Antoniou
>>>> <[email protected]> wrote:
>>>
>>> [ snip ]
>>>>
>>>> g.
>>>
>>> Since we've started talking about longer term goals, and the versioning
>>> provision seems to stand, I hope we address how much the fragment versioning
>>> thing is similar to the way board revisions progress.
>>>
>>> If a versioning syntax is available then one could create a single DT
>>> file for a bunch of 'almost' similar board and board revisions.
>>
>> I even think that the version issue is probably much more important for the short term than the overlay aspect. Well at least as important. We start having as well a bunch a panda board version with different HW setup.
>>
>> Having a single board-XXX.dts that will support all these versions is probably the best approach to avoid choosing that from the bootloader.
>>
>> We need to figure out a format + mechanism compatible with the current non-versioned format to allow filtering the nodes at runtime to keep only the relevant one.
>>
>> Something that can find the driver that will provide the proper board version or subsystem version or whatever like that:
>>
>> compatible-version = "ti,panda-version", "panda";
>>
>> Then at runtime we should create only the node with the correct match between the driver version and the string version.
>>
>>
>> /* regular panda audio routing */
>> sound: sound {
>> compatible = "ti,abe-twl6040";
>> ti,model = "PandaBoard";
>> compatible-version = "ti,panda-version", "panda";
>>
>> /* Audio routing */
>> ti,audio-routing =
>> "Headset Stereophone", "HSOL",
>> "Headset Stereophone", "HSOR",
>> "Ext Spk", "HFL",
>> "Ext Spk", "HFR",
>> "Line Out", "AUXL",
>> "Line Out", "AUXR",
>> "HSMIC", "Headset Mic",
>> "Headset Mic", "Headset Mic Bias",
>> "AFML", "Line In",
>> "AFMR", "Line In";
>> };
>>
>>
>> /* Audio routing is different between PandaBoard4430 and PandaBoardES */
>> &sound {
>> ti,model = "PandaBoardES";
>> compatible-version = "ti,panda-version", "panda-es";
>>
>> /* Audio routing */
>> ti,audio-routing =
>> "Headset Stereophone", "HSOL",
>> "Headset Stereophone", "HSOR",
>> "Ext Spk", "HFL",
>> "Ext Spk", "HFR",
>> "Line Out", "AUXL",
>> "Line Out", "AUXR",
>> "AFML", "Line In",
>> "AFMR", "Line In";
>> };
>>
>>
>> Maybe some extra version match table can just be passed during the board machine_init
>>
>> of_platform_populate(NULL, omap_dt_match_table, NULL, NULL, panda_version_match_table);
>
> Is the only difference here the content of the ti,audio-routing
> property? If so, then I don't think there's any need for infra-structure
> for this; the driver code already reads that property and adjusts its
> behaviour based upon it.
>
> I do see that "Headset Mic" exists only in one of the tables above.
> Perhaps the driver could scan the routing table, and only create the
> ASoC object for the headset if it's mentioned in the routing table?
>
> If there are additional differences, then you can always use the .data
> field in of_device_id to automatically associate some configuration data
> with different compatible values.
>
> Even better might be to extend the binding so that all HW differences
> are represented explicitly as properties; that way you wouldn't even
> need different compatible values.

I think that perhaps the choice of the SoC specific driver was unfortunate.

There are cases where a standard driver should be configured differently
depending on the board revision/model. In that case per-revision
driver changes are out of the question.

Regards

-- Pantelis

2012-11-07 22:36:01

by Ryan Mallon

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 06/11/12 08:40, Tabi Timur-B04825 wrote:
> On Mon, Nov 5, 2012 at 2:40 PM, Grant Likely <[email protected]> wrote:
>
>> Jane is building custom BeagleBone expansion boards called 'capes'. She
>> can boot the system with a stock BeagleBoard device tree, but additional
>> data is needed before a cape can be used. She could replace the FDT file
>> used by U-Boot with one that contains the extra data, but she uses the
>> same Linux system image regardless of the cape, and it is inconvenient
>> to have to select a different device tree at boot time depending on the
>> cape.
>
> What's wrong with having the boot loader detect the presence of the
> Cape and update the device tree accordingly? We do this all the time
> in U-Boot. Doing stuff like reading EEPROMs and testing for the
> presence of hardware is easier in U-Boot than in Linux.

This is probably okay for some hardware, but doesn't work in the general
case. Not all hardware is detectable, for example a cape which just adds
a set of LEDs for GPIO pins. Also, some hardware might not easily be
detectable without adding additional complexity to the boot loader.

~Ryan

2012-11-08 10:37:07

by Benoit Cousson

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

+ Peter

Hi Stephen,

On 11/7/2012 6:25 PM, Stephen Warren wrote:
> On 11/07/2012 03:19 AM, Benoit Cousson wrote:
>> Hi Panto,
>>
>> On 11/07/2012 09:13 AM, Pantelis Antoniou wrote:
>>> Hi Grant
>>>
>>> On Nov 6, 2012, at 9:45 PM, Grant Likely wrote:
>>>
>>>> On Tue, Nov 6, 2012 at 7:34 PM, Pantelis Antoniou
>>>> <[email protected]> wrote:
>>>
>>> [ snip ]
>>>>
>>>> g.
>>>
>>> Since we've started talking about longer term goals, and the versioning
>>> provision seems to stand, I hope we address how much the fragment versioning
>>> thing is similar to the way board revisions progress.
>>>
>>> If a versioning syntax is available then one could create a single DT
>>> file for a bunch of 'almost' similar board and board revisions.
>>
>> I even think that the version issue is probably much more important for the short term than the overlay aspect. Well at least as important. We start having as well a bunch a panda board version with different HW setup.
>>
>> Having a single board-XXX.dts that will support all these versions is probably the best approach to avoid choosing that from the bootloader.
>>
>> We need to figure out a format + mechanism compatible with the current non-versioned format to allow filtering the nodes at runtime to keep only the relevant one.
>>
>> Something that can find the driver that will provide the proper board version or subsystem version or whatever like that:
>>
>> compatible-version = "ti,panda-version", "panda";
>>
>> Then at runtime we should create only the node with the correct match between the driver version and the string version.
>>
>>
>> /* regular panda audio routing */
>> sound: sound {
>> compatible = "ti,abe-twl6040";
>> ti,model = "PandaBoard";
>> compatible-version = "ti,panda-version", "panda";
>>
>> /* Audio routing */
>> ti,audio-routing =
>> "Headset Stereophone", "HSOL",
>> "Headset Stereophone", "HSOR",
>> "Ext Spk", "HFL",
>> "Ext Spk", "HFR",
>> "Line Out", "AUXL",
>> "Line Out", "AUXR",
>> "HSMIC", "Headset Mic",
>> "Headset Mic", "Headset Mic Bias",
>> "AFML", "Line In",
>> "AFMR", "Line In";
>> };
>>
>>
>> /* Audio routing is different between PandaBoard4430 and PandaBoardES */
>> &sound {
>> ti,model = "PandaBoardES";
>> compatible-version = "ti,panda-version", "panda-es";
>>
>> /* Audio routing */
>> ti,audio-routing =
>> "Headset Stereophone", "HSOL",
>> "Headset Stereophone", "HSOR",
>> "Ext Spk", "HFL",
>> "Ext Spk", "HFR",
>> "Line Out", "AUXL",
>> "Line Out", "AUXR",
>> "AFML", "Line In",
>> "AFMR", "Line In";
>> };
>>
>>
>> Maybe some extra version match table can just be passed during the board machine_init
>>
>> of_platform_populate(NULL, omap_dt_match_table, NULL, NULL, panda_version_match_table);
>
> Is the only difference here the content of the ti,audio-routing
> property? If so, then I don't think there's any need for infra-structure
> for this; the driver code already reads that property and adjusts its
> behaviour based upon it.

That was just an example, and maybe not the best one. It could be any
kind of HW differences, like a different GPIO line, a different I2C
peripheral, an extra DCDC...
The point was just that you might have several version of the same node
with different attributes depending of the board revision.

Regards,
Benoit

2012-11-08 13:28:45

by Koen Kooi

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Op 7 nov. 2012, om 23:35 heeft Ryan Mallon <[email protected]> het volgende geschreven:

> On 06/11/12 08:40, Tabi Timur-B04825 wrote:
>> On Mon, Nov 5, 2012 at 2:40 PM, Grant Likely <[email protected]> wrote:
>>
>>> Jane is building custom BeagleBone expansion boards called 'capes'. She
>>> can boot the system with a stock BeagleBoard device tree, but additional
>>> data is needed before a cape can be used. She could replace the FDT file
>>> used by U-Boot with one that contains the extra data, but she uses the
>>> same Linux system image regardless of the cape, and it is inconvenient
>>> to have to select a different device tree at boot time depending on the
>>> cape.
>>
>> What's wrong with having the boot loader detect the presence of the
>> Cape and update the device tree accordingly? We do this all the time
>> in U-Boot. Doing stuff like reading EEPROMs and testing for the
>> presence of hardware is easier in U-Boot than in Linux.
>
> This is probably okay for some hardware, but doesn't work in the general
> case. Not all hardware is detectable, for example a cape which just adds
> a set of LEDs for GPIO pins. Also, some hardware might not easily be
> detectable without adding additional complexity to the boot loader.

And as Pantelis mentioned before, I really don't want my users to change the bootloader whenever they add a new LED. Touching the bootloader is just too accident prone, we had a ton of RMA requests for older versions of the beagleboard from people trying to upgrade u-boot.

Apart from the above I'd like to have fewer points of failure. Right now I need to keep uImage and foo.dtb in sync and I hate to add u-boot to that equasion as well.

regards,

Koen-

2012-11-08 14:09:33

by Timur Tabi

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Koen Kooi wrote:
> And as Pantelis mentioned before, I really don't want my users to change the bootloader whenever they add a new LED.

Well, U-Boot does allow you to manipulate the device tree from the
command-line, but I understand that this feature doesn't work that well.

--
Timur Tabi
Linux kernel developer at Freescale

2012-11-08 17:01:48

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/8/2012 3:28 AM, Koen Kooi wrote:
>
> Op 7 nov. 2012, om 23:35 heeft Ryan Mallon <[email protected]> het volgende geschreven:
>
>> On 06/11/12 08:40, Tabi Timur-B04825 wrote:
>>> On Mon, Nov 5, 2012 at 2:40 PM, Grant Likely <[email protected]> wrote:
>>>
>>>> Jane is building custom BeagleBone expansion boards called 'capes'. She
>>>> can boot the system with a stock BeagleBoard device tree, but additional
>>>> data is needed before a cape can be used. She could replace the FDT file
>>>> used by U-Boot with one that contains the extra data, but she uses the
>>>> same Linux system image regardless of the cape, and it is inconvenient
>>>> to have to select a different device tree at boot time depending on the
>>>> cape.
>>>
>>> What's wrong with having the boot loader detect the presence of the
>>> Cape and update the device tree accordingly? We do this all the time
>>> in U-Boot. Doing stuff like reading EEPROMs and testing for the
>>> presence of hardware is easier in U-Boot than in Linux.
>>
>> This is probably okay for some hardware, but doesn't work in the general
>> case. Not all hardware is detectable, for example a cape which just adds
>> a set of LEDs for GPIO pins. Also, some hardware might not easily be
>> detectable without adding additional complexity to the boot loader.
>
> And as Pantelis mentioned before, I really don't want my users to change the bootloader whenever they add a new LED. Touching the bootloader is just too accident prone, we had a ton of RMA requests for older versions of the beagleboard from people trying to upgrade u-boot.

One possibility for dynamic device tree mods would be to run Open
Firmware from u-boot and have it generate the device tree and possibly
modify it either interactively or from a script loaded from a file or
the network. OFW could then either load Linux directly or return to
u-boot, which would proceed with loading.

>
> Apart from the above I'd like to have fewer points of failure. Right now I need to keep uImage and foo.dtb in sync and I hate to add u-boot to that equasion as well.
>
> regards,
>
> Koen
> _______________________________________________
> devicetree-discuss mailing list
> [email protected]
> https://lists.ozlabs.org/listinfo/devicetree-discuss
>

2012-11-09 02:24:51

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Mon, Nov 05, 2012 at 08:40:30PM +0000, Grant Likely wrote:
> Hey folks,
>
> As promised, here is my early draft to try and capture what device
> tree overlays need to do and how to get there. Comments and
> suggestions greatly appreciated.
>
> Device Tree Overlay Feature

Hrm. So, you may yet convince me otherwise, but I'm really getting a
feeling of overengineering from this proposal so far.

> Purpose
> =======
> Sometimes it is not convenient to describe an entire system with a
> single FDT. For example, processor modules that are plugged into one or
> more modules (a la the BeagleBone), or systems with an FPGA peripheral
> that is programmed after the system is booted.
>
> For these cases it is proposed to implement an overlay feature for the
> so that the initial device tree data can be modified by userspace at
> runtime by loading additional overlay FDTs that amend the original data.
>
> User Stories
> ============

[snip]

The user stories mostly sound reasonable to me, but I don't know
enough about the hardware in question to know what they'll mean in
terms of what needs to be added to the base device tree.

> Summary points:
> - Create an FDT overlay data format and usage model
> - SHALL reliable resolve or validate of phandles between base and
> overlay trees

So, I'm not at all clear on what this proposed phandle validation
would involve. I'm also not convinced it's as necessary as you
think, more on that below.

[snip - lots of technical stuff]

So, let me take a stab at this from a more bottom-up approach, and see
if we meet in the middle somewhere. As I discussed in the other
thread with Daniel Mack, I can see two different operationso on the
fdt that might be useful in this context. I think of them as "graft"
- which takes one fdt and adds it as a new subtree to an existing fdt
- and "overlay" where a new fdt adds or overrides arbitrary properties
in an existing tree. Overlay is more or less what we do at the source
level in dtc already.

Overlay is obviously more general - you can add, change and possibly
delete any existing node or property.

Graft can only add new nodes and properties, not modify existing ones.
But that restriction comes with some advantages: reversing the
operation is just a matter of deleting the subtree with no extra
tracking info required. It's simple to see how to have rules or
permissions about where subtrees can be grafted, and if the graft
point is identified by a label or id, rather than full path, it
automatically adapts to at least some changes in the base tree
structure.

I think graft is basically a safer operation, particular if we're
doing this at runtime with userspace passing in these fdt fragments.
In fact I'd go so far as to say if you really need the full overlay
functionality, then you really ought to be working at the bootloader
or early kernel load level to assemble the correct full device tree.
And as Mitch says, an existing programming language (C, OFW Forth or
whatever as you please) will serve you better for this sort of general
manipulation than a limited template system.

I also think graft will handle most of your use cases, although as I
said I don't fully understand the implications of some of them, so I
could be wrong. So, the actual insertion of the subtree is pretty
trivial to implement. phandles are the obvious other matter to be
dealt with. I haven't found the right thread where what you've
envisaged so far is discussed, so here are things I can see:

1) Avoiding phandle collisions between main tree and subtree (or
between subtrees).

I'm hopeful that this can be resolved just by establishing some
conventions about the ranges of phandles to be used for various
components. I'd certainly be happy to add a directive to dtc which
enforces allocation of phandles within a specified range.

2) Resolving phandle references within a subtree

If we can handle (1) by convention, we don't need anything here, the
references are fine as is.

(3) Resolving phandle references from the subtree to the main tree.

So, I think this can actually be avoided, at least in cases where what
physical connections are available to the expansion module is well
defined. The main causes to have external references are interrupts
and gpios. Interrupts we handle by defining an interrupt specifier
space for the interrupts available on the expansion
socket/connector/bus/whatever. In the master tree we then have
something like:

...
expansion-socket@XXXX {
expansion-id = "SlotA";
interrupt-map = < /* map expansion irq specs to
board interrupt controllers */ >;
interrupt-map-mask = < ... >;
ranges = < /* map expansion local addresses to global
mmio */ >;
};

The subtree for the expansion module gets attached as a subnode of
this one. It doesn't use explicit interrupt-parents but instead just
uses the expansion local irq specifiers, letting the parent be the
default which will bubble up to this socket node where the
interrupt-map will send them to the right places.

I don't recall the gpio bindings off hand, but as I recall we based
them off the irq tree bindings so we ought to be able to do the same
thing for them.

Likewise, if there are several interchangeable expansion sockets that
have some address bits hard wired to distinguish them, we can just use
socket local mmio addresses within the subtree and the ranges property
here will sort that out.

--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson

2012-11-09 05:32:13

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Pantelis,

I hope I'm not too late to reply as I'm traveling.

On Nov 6, 2012, at 5:30 AM, Pantelis Antoniou
<[email protected]> wrote:

>
>>
>> Joanne has purchased one of Jane's capes and packaged it into a rugged
>> case for data logging. As far as Joanne is concerned, the BeagleBone and
>> cape together are a single unit and she'd prefer a single monolithic FDT
>> instead of using an FDT overlay.
>> Option A: Using dtc, she uses the BeagleBone and cape .dts source files
>> to generate a single .dtb for the entire system which is
>> loaded by U-Boot. -or-
>
> Unlikely.
>> Option B: Joanne uses a tool to merge the BeagleBone and cape .dtb files
>> (instead of .dts files), -or-
> Possible but low probability.
>
>> Option C: U-Boot loads both the base and overlay FDT files, merges them,
>> and passes the resolved tree to the kernel.
>>
>
> Could be made to work. Only really required if Joanne wants the
> cape interface to work for u-boot too. For example if the cape has some
> kind of network interface that u-boot will use to boot from.
>

I love Grant's hashing idea a lot keeping the phandle problem for
compile time and not requiring fixups.

IMO it is still a cleaner approach if u-boot does the simple tree merging for
all cases, and not the kernel reasons mentioned below.

(1)
>From a development standpoint, very little or nothing will
have to be changed in kernel (except for scripts/dtc) considering we
are moving forward with hashing.

(2)
Also this discussed a while back but at some point is going to brought
up again- loading of dt fragment directly from EEPROM and merging at
run time. If we were to implement this in kernel, we would have to add
cape specific EEPROM reading code, merge the tree before it is
unflattened and parse. I think doing tree merging in kernel is messy
and we should do it in uboot considering we might have to read EEPROM for
this use case. Ideally reading the fragment from the EEPROM for all capes
and merging without worrying about version detection, Doing the merge and
passing the merged blob to the kernel which (kernel) works the same way
it does today.

>> It may be sufficient to solve it by making the phandle values less
>> volatile. Right now dtc generates phandles linearly. Generated phandles
>> could be overridden with explicit phandle properties, but it isn't a
>> fantastic solution. Perhaps generating the phandle from a hash of the
>> node name would be sufficient.
>>
>
> I doubt the hash method will work reliably. We only have 32 bits to work with,
> nothing like the SHA hashes of git.
>

I was wondering I have worked with kernel's crypto code in the past to
generate 32 bit md5sums of 1000s of dataitems, from what I've seen,
collisions are rare and since we are talking about just a few nodes
that are being referenced in the base dt. I think the probability is
even less (ofcourse such an analysis strongly depends on dataset).
this method also takes away a lot of complexity with having it to do
runtime fixups and will help us get off the ground quickly.

We can also put in a collision handling mechanism if needed.
I think it is worthy doing a sample hash of all nodes in all dts we
have in a script and see for once if we have collisions and what it
looks like.

Alternatively to hashing, reading David Gibson's paper I followed,
phandle is supposed to 'uniquely' identity node. I wonder why the node
name itself is not sufficient to uniquely identify. The code that does
the tree walking can then just strcmp the node name while it walks the
tree instead of having to find a node with a phandle number. I guess
the reason is phandles are small to store as data values. Another
approach can be to arrange the string block in alphabetical order
(unless it already is), and store phandle as index of the node name
referenced relative to the starting of the strong block. This will not
affect nodes in dtb being moved around since they will still have the
same index value. the problem being adding or removing nodes Changes
the index of all other nodes in the string block as well.. Hmm.

Regards,
Joel

2012-11-09 12:06:25

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Mon, Nov 5, 2012 at 11:22 PM, Tony Lindgren <[email protected]> wrote:
> Hi,
>
> * Tabi Timur-B04825 <[email protected]> [121105 13:42]:
>> On Mon, Nov 5, 2012 at 2:40 PM, Grant Likely <[email protected]> wrote:
>>
>> > Jane is building custom BeagleBone expansion boards called 'capes'. She
>> > can boot the system with a stock BeagleBoard device tree, but additional
>> > data is needed before a cape can be used. She could replace the FDT file
>> > used by U-Boot with one that contains the extra data, but she uses the
>> > same Linux system image regardless of the cape, and it is inconvenient
>> > to have to select a different device tree at boot time depending on the
>> > cape.
>>
>> What's wrong with having the boot loader detect the presence of the
>> Cape and update the device tree accordingly? We do this all the time
>> in U-Boot. Doing stuff like reading EEPROMs and testing for the
>> presence of hardware is easier in U-Boot than in Linux.
>>
>> For configurations that can be determined by the boot loader, I'm not
>> sure overlays are practical.
>
> I guess the beaglebone capes could be stackable and hotpluggable if
> handled carefully enough.

And even if it can't on the beaglebone, it will happen somewhere else.
I don't want to exclude that use-case just because nobody thought
about it.

g.

2012-11-09 14:27:38

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 09, 2012 at 12:32:09AM -0500, Joel A Fernandes wrote:
> Hi Pantelis,
>
> I hope I'm not too late to reply as I'm traveling.
>
> On Nov 6, 2012, at 5:30 AM, Pantelis Antoniou
> <[email protected]> wrote:
>
> >> Joanne has purchased one of Jane's capes and packaged it into a rugged
> >> case for data logging. As far as Joanne is concerned, the BeagleBone and
> >> cape together are a single unit and she'd prefer a single monolithic FDT
> >> instead of using an FDT overlay.
> >> Option A: Using dtc, she uses the BeagleBone and cape .dts source files
> >> to generate a single .dtb for the entire system which is
> >> loaded by U-Boot. -or-
> >
> > Unlikely.
> >> Option B: Joanne uses a tool to merge the BeagleBone and cape .dtb files
> >> (instead of .dts files), -or-
> > Possible but low probability.
> >
> >> Option C: U-Boot loads both the base and overlay FDT files, merges them,
> >> and passes the resolved tree to the kernel.
> >>
> >
> > Could be made to work. Only really required if Joanne wants the
> > cape interface to work for u-boot too. For example if the cape has some
> > kind of network interface that u-boot will use to boot from.
> >
>
> I love Grant's hashing idea a lot keeping the phandle problem for
> compile time and not requiring fixups.

Well, using a hash only moves the problem of fixed phandles to a
problem of fixed node paths. The details of node paths are, if
anything, more mutable than phandles.

[snip]
> Alternatively to hashing, reading David Gibson's paper I followed,
> phandle is supposed to 'uniquely' identity node. I wonder why the node
> name itself is not sufficient to uniquely identify.

Node names are not unique, not even close. If you have two similar
NICs in slot 0 of two different PCI domains, they'll almost certainly
both be called 'ethernet@0,0'. Similar examples abound on other
buses. Node paths are unique, but they are long.

The other big reason for phandles in OF history is that they would be
more stable than paths. The device tree could be manipulated during
OF runtime, but phandles would generally be internal pointers in OF
and so remain a consistent handle even if the node moved in the tree.
That's not really relevant for flat trees, but we need to work with
the same structures.

> The code that does
> the tree walking can then just strcmp the node name while it walks the
> tree instead of having to find a node with a phandle number. I guess
> the reason is phandles are small to store as data values. Another
> approach can be to arrange the string block in alphabetical order
> (unless it already is),

They're not, and doing so would be a painful change to maintain
compatibility across. And in any case only property names use the
strings block, not node names.

> and store phandle as index of the node name
> referenced relative to the starting of the strong block. This will not
> affect nodes in dtb being moved around since they will still have the
> same index value. the problem being adding or removing nodes Changes
> the index of all other nodes in the string block as well.. Hmm.

--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson

2012-11-09 15:41:32

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi David,

On Nov 9, 2012, at 3:26 AM, David Gibson wrote:

> On Mon, Nov 05, 2012 at 08:40:30PM +0000, Grant Likely wrote:
>> Hey folks,
>>
>> As promised, here is my early draft to try and capture what device
>> tree overlays need to do and how to get there. Comments and
>> suggestions greatly appreciated.
>>
>> Device Tree Overlay Feature
>
> Hrm. So, you may yet convince me otherwise, but I'm really getting a
> feeling of overengineering from this proposal so far.
>
>> Purpose
>> =======
>> Sometimes it is not convenient to describe an entire system with a
>> single FDT. For example, processor modules that are plugged into one or
>> more modules (a la the BeagleBone), or systems with an FPGA peripheral
>> that is programmed after the system is booted.
>>
>> For these cases it is proposed to implement an overlay feature for the
>> so that the initial device tree data can be modified by userspace at
>> runtime by loading additional overlay FDTs that amend the original data.
>>
>> User Stories
>> ============
>
> [snip]
>
> The user stories mostly sound reasonable to me, but I don't know
> enough about the hardware in question to know what they'll mean in
> terms of what needs to be added to the base device tree.
>
>> Summary points:
>> - Create an FDT overlay data format and usage model
>> - SHALL reliable resolve or validate of phandles between base and
>> overlay trees
>
> So, I'm not at all clear on what this proposed phandle validation
> would involve. I'm also not convinced it's as necessary as you
> think, more on that below.
>
> [snip - lots of technical stuff]
>
> So, let me take a stab at this from a more bottom-up approach, and see
> if we meet in the middle somewhere. As I discussed in the other
> thread with Daniel Mack, I can see two different operationso on the
> fdt that might be useful in this context. I think of them as "graft"
> - which takes one fdt and adds it as a new subtree to an existing fdt
> - and "overlay" where a new fdt adds or overrides arbitrary properties
> in an existing tree. Overlay is more or less what we do at the source
> level in dtc already.
>
> Overlay is obviously more general - you can add, change and possibly
> delete any existing node or property.
>
> Graft can only add new nodes and properties, not modify existing ones.
> But that restriction comes with some advantages: reversing the
> operation is just a matter of deleting the subtree with no extra
> tracking info required. It's simple to see how to have rules or
> permissions about where subtrees can be grafted, and if the graft
> point is identified by a label or id, rather than full path, it
> automatically adapts to at least some changes in the base tree
> structure.
>
> I think graft is basically a safer operation, particular if we're
> doing this at runtime with userspace passing in these fdt fragments.
> In fact I'd go so far as to say if you really need the full overlay
> functionality, then you really ought to be working at the bootloader
> or early kernel load level to assemble the correct full device tree.
> And as Mitch says, an existing programming language (C, OFW Forth or
> whatever as you please) will serve you better for this sort of general
> manipulation than a limited template system.
>
> I also think graft will handle most of your use cases, although as I
> said I don't fully understand the implications of some of them, so I
> could be wrong. So, the actual insertion of the subtree is pretty
> trivial to implement. phandles are the obvious other matter to be
> dealt with. I haven't found the right thread where what you've
> envisaged so far is discussed, so here are things I can see:
>

An overlay is more generic and can handle more complex cases.
For our use case, a graft should work - with a few caveats.

Due to the insertion/removal of the DT fragments other node's state
change from 'disabled' <-> 'okay' and platform devices created or
removed. This can be handled either via overlays, or via special
casing it.

> 1) Avoiding phandle collisions between main tree and subtree (or
> between subtrees).
>
> I'm hopeful that this can be resolved just by establishing some
> conventions about the ranges of phandles to be used for various
> components. I'd certainly be happy to add a directive to dtc which
> enforces allocation of phandles within a specified range.
>

Really doubtful IMHO. That will impose yet more structure on the DT
syntax, and it will make it even more difficult for users.

We're talking about users that do understand the hardware, but can't
really grok linux kernel development.

DT is used a structured h/w definition and seems to work just
fine for these kind of users.

> 2) Resolving phandle references within a subtree
>
> If we can handle (1) by convention, we don't need anything here, the
> references are fine as is.
>
> (3) Resolving phandle references from the subtree to the main tree.
>
> So, I think this can actually be avoided, at least in cases where what
> physical connections are available to the expansion module is well
> defined. The main causes to have external references are interrupts
> and gpios. Interrupts we handle by defining an interrupt specifier
> space for the interrupts available on the expansion
> socket/connector/bus/whatever. In the master tree we then have
> something like:
>
> ...
> expansion-socket@XXXX {
> expansion-id = "SlotA";
> interrupt-map = < /* map expansion irq specs to
> board interrupt controllers */ >;
> interrupt-map-mask = < ... >;
> ranges = < /* map expansion local addresses to global
> mmio */ >;
> };
>
> The subtree for the expansion module gets attached as a subnode of
> this one. It doesn't use explicit interrupt-parents but instead just
> uses the expansion local irq specifiers, letting the parent be the
> default which will bubble up to this socket node where the
> interrupt-map will send them to the right places.
>
> I don't recall the gpio bindings off hand, but as I recall we based
> them off the irq tree bindings so we ought to be able to do the same
> thing for them.
>

Please, don't try to do it this way. It is very debatable that we'll
identify all the types that will need special handling.
For example for our cape use, we already have references that do not
follow this example.

A generic way to resolve references is what we need.

> Likewise, if there are several interchangeable expansion sockets that
> have some address bits hard wired to distinguish them, we can just use
> socket local mmio addresses within the subtree and the ranges property
> here will sort that out.
>
> --
> David Gibson | I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
> | _way_ _around_!
> http://www.ozlabs.org/~dgibson

2012-11-09 16:29:24

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
> On 11/05/2012 01:40 PM, Grant Likely wrote:
>> Hey folks,
>>
>> As promised, here is my early draft to try and capture what device
>> tree overlays need to do and how to get there. Comments and
>> suggestions greatly appreciated.
>
> Interesting. This just came up internally at NVIDIA within the last
> couple weeks, and was discussed on the U-Boot mailing list very recently
> too:
>
> http://lists.denx.de/pipermail/u-boot/2012-October/thread.html#138227
> (it spills into the November archive too)
>
>> For these cases it is proposed to implement an overlay feature for the
>> so that the initial device tree data can be modified by userspace at
>
> I don't know if you're maintaining this as a document and taking patches
> to it, but if so:

Sure, why not...

http://git.secretlab.ca/?p=devicetree-overlays.git;a=summary

>
> "for the so" split across those two lines.

fixed

>> - SHOULD reliably handle changes between different underlying overlays
>> (ie. what happens to existing .dtb overly files if the structure of
>> the dtb it is layered over changes. If not possible, then SHALL
>> detect when the base tree doesn't match and refuse to apply the
>> overlay.
>
> Perhaps use (versioned) DT bindings to represent the interface between
> the two .dts files? See the links to the U-Boot mailing list discussions
> below?

Implementing versioning is conceptually a lot more complex than plain
overlays since it means either the kernel or u-boot needs to start
filtering the data that it's given. This can get really complex in a
hurry. Mitch makes a valid point later in this thread that when it
comes to manipulating the data depending on the board then the data
overlay model alone doesn't handle it well.

I'm not actually opposed to it, but it needs to be done in an elegant
way. The DT data model already imposes more of a conceptual learning
curve than I wish it did and I don't want to make that worse with a
versioning model that is difficult to get ones head around.

Suggestions and proposals are definitely welcome here.

>
>> - What is the model for overlays?
>> - Can an overlay modify existing properties?
>> - Can an overlay add new properties to existing nodes?
>> - Can an overlay delete existing nodes/properties?
>
> This proposal is very oriented at an overlay-based approach. I'm not
> totally convinced that a pure overlay approach (as in how dtc does
> overlayed DT nodes) will be flexible enough, but would love to be
> persuaded. Again see below.

The way I'm currently thinking about the overlay approach is as a
discrete set of changes that all can be applied at once.* That
certainly doesn't preclude the change being generated with a script or
other tool in either firmware or userspace. For most changes it works
really well. Of the scenarios that don't work well, I can think of
two. The first is it moving or renaming existing nodes, and the second
is if the structure of the base tree changes (say due to a firmware
update). Although the second limitation is specifically with binary
.dtb overlays. Recompiling the dts overlay against the new tree would
work fine.**

*with the caveat that not all types of changes are a good idea and we
may disallow. For example, is changing properties in existing nodes a
good idea?
** all this is based on my current ideas about the .dtb overlay format
which would be trivial to implement, but I'm not committed to that
approach just yet. The above problems could be solved.

>> It may be sufficient to solve it by making the phandle values less
>> volatile. Right now dtc generates phandles linearly. Generated phandles
>> could be overridden with explicit phandle properties, but it isn't a
>> fantastic solution. Perhaps generating the phandle from a hash of the
>> node name would be sufficient.
>
> Node names don't have to be unique though right; perhaps hash the
> path-name instead of the node-name? But then, why not just reference by
> path name; similar to <{&/path/to/node}> rather than <&label>?

I was thinking about using the full_name for generating phandles. On
the odd chance there is a collision, one of the nodes will get
something like the hash value +1. Or in that condition we can require
one of the nodes to have the phandle value explicitly set in the
source file.

Note; this doesn't actually affect anything outside of dtc. Right now
dtc starts at 1 and assigns phandles incrementally. I'm suggesting
using a hash of the full_name to assign the phandle for a node so that
it will not change unless the full_path changes.

>> This handles many of the use cases, but it assumes that an overlay is
>> board specific. If it ever is required to support multiple base boards
>> with a single overlay file then there is a problem. The .dtb overlays
>> generated in this manor cannot handle different phandles or nodes that
>> are in a different place. On the other hand, the overlay source files
>> should have no problem being compiled for multiple targets.
>
> s/manor/manner/

Fixed

>
> I do rather suspect this use-case is quite common. NVIDIA certainly has
> a bunch of development boards with pluggable
> PMIC/audio/WiFi/display/..., and I believe there's some ability to
> re-use the pluggable components with a variety of base-boards.
>
> Given people within NVIDIA started talking about this recently, I asked
> them to enumerate all the boards we have that support pluggable
> components, and how common it is that some boards support being plugged
> into different main boards. I don't know when that enumeration will
> complete (or even start) but hopefully I can provide some feedback on
> how common the use-case is for us once it's done.

>From your perspective, is it important to use the exact same .dtb
overlays for those add-on boards, or is it okay to have a separate
build of the overlay for each base tree?

g.

2012-11-09 17:02:29

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Wed, Nov 7, 2012 at 12:54 AM, Mitch Bradley <[email protected]> wrote:
> On 11/6/2012 12:37 PM, Stephen Warren wrote:
>> This proposal is very oriented at an overlay-based approach. I'm not
>> totally convinced that a pure overlay approach (as in how dtc does
>> overlayed DT nodes) will be flexible enough, but would love to be
>> persuaded. Again see below.
>
>
> An overlay approach will not be powerful enough to solve the sorts of
> problems that occur when a product goes into full production, becomes a
> family, and starts to evolve. Issues like second-source parts that
> aren't quite compatible and need to be detected and reported,
> board-stuff options for different customer profiles, speed grades of
> parts that aren't properly probeable but instead need to be identified
> by some subterfuge, the list of tedious issues goes on and on.
>
> It's nice to pretend that the world fits into a few coherent simple
> use cases, but 30 years of experience shipping computer product families
> proves otherwise. You need a programming language to solve the full
> set of problems - which I why the device tree is designed so it can
> be generated and modified by a program.

I'm not going to argue with that. There is nothing saying that the
overlay wouldn't be generated or applied by a script. However, I do
strongly think that the data model needs to be independent of any tool
or execution environment used to manipulate it. I certainly am not
interested in encoding scripts or bytecode into the tree - the
opposite of the approach used by ACPI which must execute AML to even
retrieve the device tree. I like the overlay approach because it is
conceptually straightforward and provides a working model of how
changes could be made from userspace while still being usable by
firmware.

An alternate approach from userspace would be to use a live virtual
filesystem to manipulate the device tree, though that approach only
applies to Linux userspace. Firmware would still need its own
approach.

g.

2012-11-09 17:03:25

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Wed, Nov 7, 2012 at 8:06 AM, Pantelis Antoniou
<[email protected]> wrote:
> Hi Grant,
>
> On Nov 6, 2012, at 9:45 PM, Grant Likely wrote:
>> Yes, the locking does need to be sorted out.
>>
>
> Perhaps come up with a dt-stress test tool/boot time validator?

I would like that. I've started adding DT test cases to the kernel source tree.

g.

2012-11-09 20:34:10

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Wed, Nov 7, 2012 at 11:02 AM, Pantelis Antoniou
<[email protected]> wrote:
> On Nov 7, 2012, at 11:19 AM, Benoit Cousson wrote:
>> Maybe some extra version match table can just be passed during the board machine_init
>>
>> of_platform_populate(NULL, omap_dt_match_table, NULL, NULL, panda_version_match_table);
>>
>
> Would we need explicit of_platform_populate calls if we have node modification notifiers?
> In that case the notifier would pick it up automatically, and can do the per
> version matching internally.

There still needs to be something to register "everything below this
node is interesting" which is exactly what of_platform_populate() does
now. I see the notifiers being used by the of_platform_populate
backend to know when nodes have been created (or destroyed).

g.

2012-11-09 21:08:37

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 9, 2012 at 2:26 AM, David Gibson
<[email protected]> wrote:
>> Summary points:
>> - Create an FDT overlay data format and usage model
>> - SHALL reliable resolve or validate of phandles between base and
>> overlay trees
>
> So, I'm not at all clear on what this proposed phandle validation
> would involve. I'm also not convinced it's as necessary as you
> think, more on that below.

Simply this: I'm taking this example from the omap3-beagle-xm.dts. It
has the following stanza which is currently rolled into the resulting
.dtb when compiled.

&i2c1 {
clock-frequency = <2600000>;

twl: twl@48 {
reg = <0x48>;
interrupts = <7>; /* SYS_NIRQ cascaded to intc */
interrupt-parent = <&intc>;

vsim: regulator-vsim {
compatible = "ti,twl4030-vsim";
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <3000000>;
};

twl_audio: audio {
compatible = "ti,twl4030-audio";
codec {
};
};
};
};

However, if it were compiled into a separate dtb overlay it might look
like this:

/ {
.readonly;
ocp {
.readonly;
interrupt-controller@48200000 {
phandle = <0x1234>; /* EXPECTED PHANDLE */
.readonly;
};
i2c@48070000 {
.must-exist;
clock-frequency = <2600000>;

twl@48 {
reg = <0x48>;
interrupts = <7>;
interrupt-parent = <0x1234>; /* RESOLVED PHANDLE */

vsim: regulator-vsim {
compatible = "ti,twl4030-vsim";
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <3000000>;
};

twl_audio: audio {
compatible = "ti,twl4030-audio";
codec {
};
};
};
};
};
};

Notice I've included the intc node and it's phandle. By phandle
validation I merely mean that when applying an overly the firmware or
kernel must verify that the phandles in the overlay match the phandle
in the base tree. If they don't match, then refuse to apply the
overhead. This approach avoids the need to find and fixup phandles in
the overlay. And if the phandle is generated from a hash of the
full_name, then the resulting phandle will only change if the node
moves.

Similarly, at application time it should be verified that the nodes
with a .readonly or .must-exist property could be verified to actually
exist before attempting to apply the overlay. I used two different
properties with the idea that only certain nodes would need to be
modified... exactly what the policies should be is yet to be
determined.

g.

2012-11-09 21:22:58

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 9, 2012 at 5:32 AM, Joel A Fernandes <[email protected]> wrote:
> Hi Pantelis,
>
> I hope I'm not too late to reply as I'm traveling.
>
> On Nov 6, 2012, at 5:30 AM, Pantelis Antoniou
> <[email protected]> wrote:
>
>>
>>>
>>> Joanne has purchased one of Jane's capes and packaged it into a rugged
>>> case for data logging. As far as Joanne is concerned, the BeagleBone and
>>> cape together are a single unit and she'd prefer a single monolithic FDT
>>> instead of using an FDT overlay.
>>> Option A: Using dtc, she uses the BeagleBone and cape .dts source files
>>> to generate a single .dtb for the entire system which is
>>> loaded by U-Boot. -or-
>>
>> Unlikely.
>>> Option B: Joanne uses a tool to merge the BeagleBone and cape .dtb files
>>> (instead of .dts files), -or-
>> Possible but low probability.
>>
>>> Option C: U-Boot loads both the base and overlay FDT files, merges them,
>>> and passes the resolved tree to the kernel.
>>>
>>
>> Could be made to work. Only really required if Joanne wants the
>> cape interface to work for u-boot too. For example if the cape has some
>> kind of network interface that u-boot will use to boot from.
>>
>
> I love Grant's hashing idea a lot keeping the phandle problem for
> compile time and not requiring fixups.
>
> IMO it is still a cleaner approach if u-boot does the simple tree merging for
> all cases, and not the kernel reasons mentioned below.

I'm more than sufficiently convinced that making changes at runtime
from userspace is pretty much required.

Also consider: it is order of magnitudes easier to modify the kernel
than it is to change the firmware for end users.

> (1)
> From a development standpoint, very little or nothing will
> have to be changed in kernel (except for scripts/dtc) considering we
> are moving forward with hashing.
>
> (2)
> Also this discussed a while back but at some point is going to brought
> up again- loading of dt fragment directly from EEPROM and merging at
> run time. If we were to implement this in kernel, we would have to add
> cape specific EEPROM reading code, merge the tree before it is
> unflattened and parse.

Unless it is required for boot to userspace I'm not considering
merging before userspace starts. That's well after the tree is
unflattened into the live form. If it is require to boot then I agree
that is should be done in firmware. I see zero problem with having a
beaglebone specific cape driver that knows to read the eeprom and
request a specific configuration file. Heck, the kernel doesn't even
need to parse the eeprom data. It can be read from userspace and
userspace decides which overlay to provide. There's nothing stopping
userspace from reading the eeprom, looking up the correct dts for the
board, downloading the file from the Internet, compiling it with dtc
and installing it.... and yes that is getting a little extreme.

> I think doing tree merging in kernel is messy
> and we should do it in uboot considering we might have to read EEPROM for
> this use case. Ideally reading the fragment from the EEPROM for all capes
> and merging without worrying about version detection, Doing the merge and
> passing the merged blob to the kernel which (kernel) works the same way
> it does today.
>
>>> It may be sufficient to solve it by making the phandle values less
>>> volatile. Right now dtc generates phandles linearly. Generated phandles
>>> could be overridden with explicit phandle properties, but it isn't a
>>> fantastic solution. Perhaps generating the phandle from a hash of the
>>> node name would be sufficient.
>>>
>>
>> I doubt the hash method will work reliably. We only have 32 bits to work with,
>> nothing like the SHA hashes of git.
>>
>
> I was wondering I have worked with kernel's crypto code in the past to
> generate 32 bit md5sums of 1000s of dataitems, from what I've seen,
> collisions are rare and since we are talking about just a few nodes
> that are being referenced in the base dt. I think the probability is
> even less (ofcourse such an analysis strongly depends on dataset).
> this method also takes away a lot of complexity with having it to do
> runtime fixups and will help us get off the ground quickly.

It wouldn't be hard to put together a test and run it on all the .dts
files in the kernel; generating md5 sums for all the full_name paths
and seeing if we've got any collisions yet.

> We can also put in a collision handling mechanism if needed.
> I think it is worthy doing a sample hash of all nodes in all dts we
> have in a script and see for once if we have collisions and what it
> looks like.

Collision handling is a must, but again this is all internal to dtc. I
don't foresee any problems with it.

> Alternatively to hashing, reading David Gibson's paper I followed,
> phandle is supposed to 'uniquely' identity node. I wonder why the node
> name itself is not sufficient to uniquely identify.

Simply because the FDT draws upon the existing OFW bindings which use
a 32bit value to reference other nodes.

> The code that does
> the tree walking can then just strcmp the node name while it walks the
> tree instead of having to find a node with a phandle number. I guess
> the reason is phandles are small to store as data values. Another
> approach can be to arrange the string block in alphabetical order
> (unless it already is), and store phandle as index of the node name
> referenced relative to the starting of the strong block. This will not
> affect nodes in dtb being moved around since they will still have the
> same index value. the problem being adding or removing nodes Changes
> the index of all other nodes in the string block as well.. Hmm.

And that still doesn't help find all the phandle locations in the tree
for doing fixups. It would need to be a table of
nodes+properties+offsets that contain phandles for fixup to work, but
I shy away from that approach because I think it will be fragile.

g.

2012-11-09 21:43:01

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 9, 2012 at 2:26 AM, David Gibson
<[email protected]> wrote:
> (3) Resolving phandle references from the subtree to the main tree.
>
> So, I think this can actually be avoided, at least in cases where what
> physical connections are available to the expansion module is well
> defined. The main causes to have external references are interrupts
> and gpios. Interrupts we handle by defining an interrupt specifier
> space for the interrupts available on the expansion
> socket/connector/bus/whatever. In the master tree we then have
> something like:
>
> ...
> expansion-socket@XXXX {
> expansion-id = "SlotA";
> interrupt-map = < /* map expansion irq specs to
> board interrupt controllers */ >;
> interrupt-map-mask = < ... >;
> ranges = < /* map expansion local addresses to global
> mmio */ >;
> };
>
> The subtree for the expansion module gets attached as a subnode of
> this one. It doesn't use explicit interrupt-parents but instead just
> uses the expansion local irq specifiers, letting the parent be the
> default which will bubble up to this socket node where the
> interrupt-map will send them to the right places.
>
> I don't recall the gpio bindings off hand, but as I recall we based
> them off the irq tree bindings so we ought to be able to do the same
> thing for them.
>
> Likewise, if there are several interchangeable expansion sockets that
> have some address bits hard wired to distinguish them, we can just use
> socket local mmio addresses within the subtree and the ranges property
> here will sort that out.

If I'm reading correctly, the suggestion is that everything be grafted
below a single node and all connections route through that node using
mapping. Correct?

For interrupts that works today
For gpios, it isn't currently supported, but we could do it.
For SPI it would mean that the new spi devices would not appear below
the actual spi bus they are attached to
For I2C, MDIO, and one wire, same problem.
For memory mapped devices, the expansion node would need to a ranges
for all the windows that map through it, and it assumes only one
memory mapped bus (or at least it prefers only one memory mapped bus.
If there were more than one then the expansion node placement wouldn't
have a natural place to sit)

The problem is that this is not like a PCI bus where there is only one
kind of interface. It is a whole bunch of interfaces that happen to be
grouped together loosely (as an expansion connector in the beaglebone
case, but expansion isn't the only problem that I'm hearing about).

So, with a group of i2c, spi, memory mapped and other devices than are
all plugged in together, how does that look? They really should not
sit on the same level. An spi device cannot be a peer of an i2c device
for instance, the address mapping is entirely different. The kernel
really wants i2c devices to be a child of the actual i2c bus which may
already have an i2c device or too on the main board. Does the
expansion node need to have some kind of redirect node for each of the
busses where the children of it need to create devices as children of
the master bus?

To me that seems to get really complex in a hurry. More complex than
the overlay approach.

g.

2012-11-09 22:57:44

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/08/2012 07:26 PM, David Gibson wrote:
...
> I also think graft will handle most of your use cases, although as I
> said I don't fully understand the implications of some of them, so I
> could be wrong. So, the actual insertion of the subtree is pretty
> trivial to implement. phandles are the obvious other matter to be
> dealt with. I haven't found the right thread where what you've
> envisaged so far is discussed, so here are things I can see:
>
> 1) Avoiding phandle collisions between main tree and subtree (or
> between subtrees).
>
> I'm hopeful that this can be resolved just by establishing some
> conventions about the ranges of phandles to be used for various
> components. I'd certainly be happy to add a directive to dtc which
> enforces allocation of phandles within a specified range.

One case I wonder about is:

Base board A that's split into two .dts files e.g. due to there being
two versions of the base board which are 90% identical, yet there being
a few differences between them.

Base board B that's just a single .dts file.

We might allocate phandle range 0..999 for the base board.

Child board X plugs into either, so the two base boards need to "export"
the same set of phandle IDs to the child board to allow it to reference
them.

If base board A version 2 comes along after the phandle IDs that the
child board uses are defined, then how do we handle assigning a specific
phandle ID range to each of base board A's two version-specific
overlays, when the version-specific changes might affect arbitrary
phandle IDs within the range, and require some to be moved into the base
board version-specific overlays and some to stay in the common base
board .dts?

> 2) Resolving phandle references within a subtree
>
> If we can handle (1) by convention, we don't need anything here, the
> references are fine as is.
>
> (3) Resolving phandle references from the subtree to the main tree.
>
> So, I think this can actually be avoided, at least in cases where what
> physical connections are available to the expansion module is well
> defined. The main causes to have external references are interrupts
> and gpios. Interrupts we handle by defining an interrupt specifier
> space for the interrupts available on the expansion
> socket/connector/bus/whatever. In the master tree we then have
> something like:
>
> ...
> expansion-socket@XXXX {
> expansion-id = "SlotA";
> interrupt-map = < /* map expansion irq specs to
> board interrupt controllers */ >;
> interrupt-map-mask = < ... >;
> ranges = < /* map expansion local addresses to global
> mmio */ >;
> };

We would end up needing an interrupt-map or ranges type of property for
basically any resource type.

For example, what about an I2C bus that's routed to the daughter board,
and the daughter board contains an I2C bus mux, whose control path isn't
through I2C but rather GPIOs? In this case, the I2C bus mux isn't a
child of the I2C bus, but a separate entity that indicates its parent
I2C bus using a phandle. I presume a similar argument applies to almost
any kind of resource. This is probably do-able, but certainly something
to consider with the socket approach. I do like the socket approach though.

2012-11-09 22:59:44

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/08/2012 10:32 PM, Joel A Fernandes wrote:
...
> Alternatively to hashing, reading David Gibson's paper I followed,
> phandle is supposed to 'uniquely' identity node. I wonder why the node
> name itself is not sufficient to uniquely identify. The code that does
> the tree walking can then just strcmp the node name while it walks the
> tree instead of having to find a node with a phandle number.

Node names or node paths?

If you're talking node names, consider an SoC with two different I2C
controllers each attached to an I2C EEPROM with the same I2C address.
The nodes representing those EEPROMs might both get the name "eeprom@2c"
for example. The node paths should be unique though.

2012-11-09 23:06:22

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/05/2012 01:40 PM, Grant Likely wrote:
> As promised, here is my early draft to try and capture what device
> tree overlays need to do and how to get there. Comments and
> suggestions greatly appreciated.

Here's one other requirement I'd like that I don't think I saw
explicitly mentioned in your document:

Assuming a base file board.dts and a child board file child.dts, the
compiled file child.dtb should be usable with a modified board.dtb
assuming it exports the same set of attachment-points (hashed phandles,
socket objects, whatever). This allows bug-fixes etc. to board.dts
without forcing every child.dts to be recompiled.

If the attachment points is hashed node names or node content from
board.dts, I'm not sure how this is possible?

I suppose this is implicit if child.dtb can be used with either
board-a.dtb and board-b.dtb though.

2012-11-09 23:14:33

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/08/2012 07:26 PM, David Gibson wrote:
...
> So, let me take a stab at this from a more bottom-up approach, and see
> if we meet in the middle somewhere. As I discussed in the other
> thread with Daniel Mack, I can see two different operationso on the
> fdt that might be useful in this context. I think of them as "graft"
> - which takes one fdt and adds it as a new subtree to an existing fdt
> - and "overlay" where a new fdt adds or overrides arbitrary properties
> in an existing tree. Overlay is more or less what we do at the source
> level in dtc already.

One more thought on the differences between overlay and grafting:

With overlay, it's possible to make your overlay a complete DT tree
rooted at /. In some cases, you might find a lower-level node which all
overlaid elements share, and root the overlay process there.

With grafting, you're basically guaranteed to want the child/graft file
to have different parts of itself grafted into different points in the
parent/underlying tree, e.g. to add nodes to an I2C bus, an SPI bus, and
perhaps some bus-less devices at the root (e.g. a new gpio-leds device).
This will require that a child/graft file to consist of multiple chunks
of DT all to be grafted as separate operations, whereas with overlay you
may be able to get away with a single chunk to be overlaid (although
with some of the use-cases discussed, even the overlay approach might
require multiple chunks to be applied).

2012-11-09 23:24:00

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/09/2012 09:28 AM, Grant Likely wrote:
> On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
...
>> I do rather suspect this use-case is quite common. NVIDIA certainly has
>> a bunch of development boards with pluggable
>> PMIC/audio/WiFi/display/..., and I believe there's some ability to
>> re-use the pluggable components with a variety of base-boards.
>>
>> Given people within NVIDIA started talking about this recently, I asked
>> them to enumerate all the boards we have that support pluggable
>> components, and how common it is that some boards support being plugged
>> into different main boards. I don't know when that enumeration will
>> complete (or even start) but hopefully I can provide some feedback on
>> how common the use-case is for us once it's done.
>
> From your perspective, is it important to use the exact same .dtb
> overlays for those add-on boards, or is it okay to have a separate
> build of the overlay for each base tree?

I certainly think it'd be extremely beneficial to use the exact same
child board .dtb with arbitrary base boards.

Consider something like the Arduino shield connector format, which I
/believe/ has been re-used across a wide variety of Arduino boards and
other compatible or imitation boards. Now consider a vendor of an
Arduino shield. The shield vendor probably wants to publish a single
.dtb file that works for users irrespective of which board they're using
it with.

(Well, I'm not sure that Arduino can run Linux; perhaps that's why you
picked BeagleBone capes for your document!)

I suppose it would be acceptable for the shield vendor to ship the .dts
file rather than the .dtb, and hence need to build the shield .dtb for a
specific base board.

However, I think the process for an end-user needs to be as simple as
"drop this .dts/.dtb file into some standard directory", and I imagine
it'll be much easier for distros/... to make that process work if
they're dealing with a .dtb that they can just blast into the kernel's
firmware loader interface, rather than having to also locate the
base-board .dts/.dtb file, and run dtc and/or other tools on both .dts
files together.

2012-11-09 23:27:25

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 9, 2012 at 10:57 PM, Stephen Warren <[email protected]> wrote:
> On 11/08/2012 07:26 PM, David Gibson wrote:
> ...
>> I also think graft will handle most of your use cases, although as I
>> said I don't fully understand the implications of some of them, so I
>> could be wrong. So, the actual insertion of the subtree is pretty
>> trivial to implement. phandles are the obvious other matter to be
>> dealt with. I haven't found the right thread where what you've
>> envisaged so far is discussed, so here are things I can see:
>>
>> 1) Avoiding phandle collisions between main tree and subtree (or
>> between subtrees).
>>
>> I'm hopeful that this can be resolved just by establishing some
>> conventions about the ranges of phandles to be used for various
>> components. I'd certainly be happy to add a directive to dtc which
>> enforces allocation of phandles within a specified range.
>
> One case I wonder about is:
>
> Base board A that's split into two .dts files e.g. due to there being
> two versions of the base board which are 90% identical, yet there being
> a few differences between them.
>
> Base board B that's just a single .dts file.
>
> We might allocate phandle range 0..999 for the base board.
>
> Child board X plugs into either, so the two base boards need to "export"
> the same set of phandle IDs to the child board to allow it to reference
> them.

It's more than just that. The boards really need to be equivalent
since the irq specifiers and gpio specifiers need to match the gpio
and irq controllers provided by the board. So a simple overlay design
won't cover the case of a single file that will work generically on
any board.

> If base board A version 2 comes along after the phandle IDs that the
> child board uses are defined, then how do we handle assigning a specific
> phandle ID range to each of base board A's two version-specific
> overlays, when the version-specific changes might affect arbitrary
> phandle IDs within the range, and require some to be moved into the base
> board version-specific overlays and some to stay in the common base
> board .dts?

With the design I'm currently considering, at the dts level the
overlay could be compiled against any base boards if the specifiers
are equivalent and the labels match up as indicated above. The
compiled dtb could also be somewhat portable, but only if care is
taken to make sure the phandles match too; possibly by explicitly
specifying them instead of letting them be generated by a hash.

g.

2012-11-09 23:32:34

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 9, 2012 at 11:06 PM, Stephen Warren <[email protected]> wrote:
> On 11/05/2012 01:40 PM, Grant Likely wrote:
>> As promised, here is my early draft to try and capture what device
>> tree overlays need to do and how to get there. Comments and
>> suggestions greatly appreciated.
>
> Here's one other requirement I'd like that I don't think I saw
> explicitly mentioned in your document:
>
> Assuming a base file board.dts and a child board file child.dts, the
> compiled file child.dtb should be usable with a modified board.dtb
> assuming it exports the same set of attachment-points (hashed phandles,
> socket objects, whatever). This allows bug-fixes etc. to board.dts
> without forcing every child.dts to be recompiled.

No, I hadn't explicitly captured that one, but yes it is the intent.

> If the attachment points is hashed node names or node content from
> board.dts, I'm not sure how this is possible?

Ummm, I think there is misunderstanding about the hashed phandles. My
thought is merely that using a hash to generate a phandle is 'better'
than starting at 1 and counting upwards when dtc compiles the tree.
That way, if the path to the node has not changed, then the phandle
will not have changed.

However, phandles can still be explicitly specified if slightly
different trees need to have the same target point.

That said, if portability of dtb files is a strong requirement, then
it will be difficult to do with simple overlays. Even if the phandles
line up, the irq/gpio specifiers probably need to be different. That
makes things harder.

g.

2012-11-09 23:40:31

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 9, 2012 at 11:23 PM, Stephen Warren <[email protected]> wrote:
> On 11/09/2012 09:28 AM, Grant Likely wrote:
>> On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
> ...
>>> I do rather suspect this use-case is quite common. NVIDIA certainly has
>>> a bunch of development boards with pluggable
>>> PMIC/audio/WiFi/display/..., and I believe there's some ability to
>>> re-use the pluggable components with a variety of base-boards.
>>>
>>> Given people within NVIDIA started talking about this recently, I asked
>>> them to enumerate all the boards we have that support pluggable
>>> components, and how common it is that some boards support being plugged
>>> into different main boards. I don't know when that enumeration will
>>> complete (or even start) but hopefully I can provide some feedback on
>>> how common the use-case is for us once it's done.
>>
>> From your perspective, is it important to use the exact same .dtb
>> overlays for those add-on boards, or is it okay to have a separate
>> build of the overlay for each base tree?
>
> I certainly think it'd be extremely beneficial to use the exact same
> child board .dtb with arbitrary base boards.
>
> Consider something like the Arduino shield connector format, which I
> /believe/ has been re-used across a wide variety of Arduino boards and
> other compatible or imitation boards. Now consider a vendor of an
> Arduino shield. The shield vendor probably wants to publish a single
> .dtb file that works for users irrespective of which board they're using
> it with.
>
> (Well, I'm not sure that Arduino can run Linux; perhaps that's why you
> picked BeagleBone capes for your document!)

Correct, the Arduino is only an AVR with a tiny amount of ram. No Linux there.

However, Arduino shields are a good example of a use case. I think
there are even some Arduino shield compatible Linux boards out there.

> I suppose it would be acceptable for the shield vendor to ship the .dts
> file rather than the .dtb, and hence need to build the shield .dtb for a
> specific base board.

That would be better I think than relying on a binary. However, some
though needs to go into how to handle base boards that /aren't/ mostly
equivalent. Such as if they have a different GPIO controller. It may
be that for gpios and irqs, the solution really is to use
interrupt-map and create a gpio-map. i2c, spi and others still would
need to become children of the correct bus.

> However, I think the process for an end-user needs to be as simple as
> "drop this .dts/.dtb file into some standard directory", and I imagine
> it'll be much easier for distros/... to make that process work if
> they're dealing with a .dtb that they can just blast into the kernel's
> firmware loader interface, rather than having to also locate the
> base-board .dts/.dtb file, and run dtc and/or other tools on both .dts
> files together.

The base-board .dts is unnecessary. dtc is fully capable of using
/proc/device-tree as the source material. :-)

g.

--
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

2012-11-10 03:15:27

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 9, 2012 at 8:29 AM, David Gibson
<[email protected]> wrote:
> On Fri, Nov 09, 2012 at 12:32:09AM -0500, Joel A Fernandes wrote:
>> Hi Pantelis,
>>
>> I hope I'm not too late to reply as I'm traveling.
>>
>> On Nov 6, 2012, at 5:30 AM, Pantelis Antoniou
>> <[email protected]> wrote:
>>
>> >> Joanne has purchased one of Jane's capes and packaged it into a rugged
>> >> case for data logging. As far as Joanne is concerned, the BeagleBone and
>> >> cape together are a single unit and she'd prefer a single monolithic FDT
>> >> instead of using an FDT overlay.
>> >> Option A: Using dtc, she uses the BeagleBone and cape .dts source files
>> >> to generate a single .dtb for the entire system which is
>> >> loaded by U-Boot. -or-
>> >
>> > Unlikely.
>> >> Option B: Joanne uses a tool to merge the BeagleBone and cape .dtb files
>> >> (instead of .dts files), -or-
>> > Possible but low probability.
>> >
>> >> Option C: U-Boot loads both the base and overlay FDT files, merges them,
>> >> and passes the resolved tree to the kernel.
>> >>
>> >
>> > Could be made to work. Only really required if Joanne wants the
>> > cape interface to work for u-boot too. For example if the cape has some
>> > kind of network interface that u-boot will use to boot from.
>> >
>>
>> I love Grant's hashing idea a lot keeping the phandle problem for
>> compile time and not requiring fixups.
>
> Well, using a hash only moves the problem of fixed phandles to a
> problem of fixed node paths. The details of node paths are, if
> anything, more mutable than phandles.

So what you're saying is there's no way to make a phandle a signature
of a (property of a node) since no one property is unique. If I
follow, even node path can't be used because hash value changes if
node is moved around in the tree. This shoots down the hashing idea
then, which I guess is looked down upon anyway due to dynamic changes
to the base DT as discussed in the usecases.

> [snip]
>> Alternatively to hashing, reading David Gibson's paper I followed,
>> phandle is supposed to 'uniquely' identity node. I wonder why the node
>> name itself is not sufficient to uniquely identify.
>
> Node names are not unique, not even close. If you have two similar
> NICs in slot 0 of two different PCI domains, they'll almost certainly
> both be called 'ethernet@0,0'. Similar examples abound on other
> buses. Node paths are unique, but they are long.
>
> The other big reason for phandles in OF history is that they would be
> more stable than paths. The device tree could be manipulated during
> OF runtime, but phandles would generally be internal pointers in OF
> and so remain a consistent handle even if the node moved in the tree.
> That's not really relevant for flat trees, but we need to work with
> the same structures.

Thanks.

>> The code that does
>> the tree walking can then just strcmp the node name while it walks the
>> tree instead of having to find a node with a phandle number. I guess
>> the reason is phandles are small to store as data values. Another
>> approach can be to arrange the string block in alphabetical order
>> (unless it already is),
>
> They're not, and doing so would be a painful change to maintain
> compatibility across. And in any case only property names use the
> strings block, not node names.

Understood, thanks.

Regards,
Joel

2012-11-10 03:36:29

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Pantelis,

On Fri, Nov 9, 2012 at 2:13 AM, Pantelis Antoniou
<[email protected]> wrote:

>>>> Option C: U-Boot loads both the base and overlay FDT files, merges them,
>>>> and passes the resolved tree to the kernel.
>>>>
>>>
>>> Could be made to work. Only really required if Joanne wants the
>>> cape interface to work for u-boot too. For example if the cape has some
>>> kind of network interface that u-boot will use to boot from.
>>>
>>
>> I love Grant's hashing idea a lot keeping the phandle problem for
>> compile time and not requiring fixups.
>>
>> IMO it is still a cleaner approach if u-boot does the tree merging for
>> all cases, and not the kernel.
>>
>> That way from a development standpoint, very little or nothing will
>> have to be changed in kernel (except for scripts/dtc) considering we
>> are moving forward with hashing.
>>
>> Also this discussed a while back but at some point is going to brought
>> up again- loading of dt fragment directly from EEPROM and merging at
>> run time. If we were to implement this in kernel, we would have to add
>> cape specific EEPROM reading code, merge the tree before it is
>> unflattened and parse. I think doing tree merging in kernel is messy
>> and we should do it in uboot. Ideally reading the fragment from the
>> EEPROM for all capes and merging without worrying about version
>> detection, Doing the merge and passing the merged blob to the kernel
>> which (kernel) works the same way it does today.
>
> Not going to work, for a lot of cases. Doing it in the kernel seems to be
> the cleaner option. There are valid use cases for doing in u-boot too.

True, if dynamic runtime stuff from userspace is what we're talking
about, then yeah I see the important need for kernel to do the merge.

>> Alternatively to hashing, reading david gibsons paper I followed,
>> phandle is supposed to 'uniquely' identity node. I wonder why the node
>> name itself is not sufficient to unquiely identify. The code that does
>> the tree walking can then just strcmp the node name while it walks the
>> tree instead of having to find a node with a phandle number. I guess
>> the reason is phandles are small to store as data values. Another
>> approach can be to arrange the string block in alphabetical order
>> (unless it already is), and store phandle as index of the node name
>> referenced relative to the starting of the strong block. This will not
>> affect nodes in dtb being moved around since they will still have the
>> same index value. the problem being adding or removing nodes Changes
>> the offsets of all other nodes in the string block as well.. Hmm.
>>
>
> This is pretty radical change to the DT format, no?

Yes, true and the only way hypothetically to replace the phandle
tree-walking mechanism is to store node paths instead of phandle which
David pointed is too long to store, so I guess this wont work after
all. Anyway this was an interesting exercise, thanks.

Regards,
Joel

2012-11-11 20:47:20

by Rob Landley

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/09/2012 10:28:59 AM, Grant Likely wrote:
> On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren
> <[email protected]> wrote:
> > On 11/05/2012 01:40 PM, Grant Likely wrote:
> I'm not actually opposed to it, but it needs to be done in an elegant
> way. The DT data model already imposes more of a conceptual learning
> curve than I wish it did and I don't want to make that worse with a
> versioning model that is difficult to get ones head around.

Speaking of which...

I want to poke at board emulation in qemu, from scratch. Specifically,
I want to start with an unpopulated board (just the processor), add a
block of physical memory and a serial device, and boot an initramfs in
there with stdin/stdout. Then I want to incrementally add an RTC,
network card, and three block devices.

I'd like to define this board by giving qemu and the kernel the same
device tree they can parse, and I'd like to _build_ this device tree so
I understand what's in it. I'd like to repeat this exercize for arm,
mips, ppc, x86, x86-64, sparc, sh4, and maybe other boards.

And I'd like to write up an article on doing it as a learning exercise.

Last time I checked, doing this wasn't possible. (qemu couldn't define
a board by parsing a device tree, the kernel used the device tree as a
guideline but it only really read data the drivers you linked in were
expecting, the only documentation about what any of the nodes were was
was read the other device trees as examples or read the source code of
the drivers looking for data in the tree...)

Is it a more realistic project now? If so, where would I start? (Once
upon a time I read the booting-without-of document, back when it lived
in the ppc directory. It didn't really say what should go in any of the
nodes.)

Rob-

2012-11-12 10:53:55

by Koen Kooi

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Op 10 nov. 2012, om 00:40 heeft Grant Likely <[email protected]> het volgende geschreven:

> On Fri, Nov 9, 2012 at 11:23 PM, Stephen Warren <[email protected]> wrote:
>> On 11/09/2012 09:28 AM, Grant Likely wrote:
>>> On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
>> ...
>>>> I do rather suspect this use-case is quite common. NVIDIA certainly has
>>>> a bunch of development boards with pluggable
>>>> PMIC/audio/WiFi/display/..., and I believe there's some ability to
>>>> re-use the pluggable components with a variety of base-boards.
>>>>
>>>> Given people within NVIDIA started talking about this recently, I asked
>>>> them to enumerate all the boards we have that support pluggable
>>>> components, and how common it is that some boards support being plugged
>>>> into different main boards. I don't know when that enumeration will
>>>> complete (or even start) but hopefully I can provide some feedback on
>>>> how common the use-case is for us once it's done.
>>>
>>> From your perspective, is it important to use the exact same .dtb
>>> overlays for those add-on boards, or is it okay to have a separate
>>> build of the overlay for each base tree?
>>
>> I certainly think it'd be extremely beneficial to use the exact same
>> child board .dtb with arbitrary base boards.
>>
>> Consider something like the Arduino shield connector format, which I
>> /believe/ has been re-used across a wide variety of Arduino boards and
>> other compatible or imitation boards. Now consider a vendor of an
>> Arduino shield. The shield vendor probably wants to publish a single
>> .dtb file that works for users irrespective of which board they're using
>> it with.
>>
>> (Well, I'm not sure that Arduino can run Linux; perhaps that's why you
>> picked BeagleBone capes for your document!)
>
> Correct, the Arduino is only an AVR with a tiny amount of ram. No Linux there.
>
> However, Arduino shields are a good example of a use case. I think
> there are even some Arduino shield compatible Linux boards out there.

A good example of those would be the Rascal Micro: http://rascalmicro.com/

regards,

Koen-

2012-11-12 11:03:32

by Koen Kooi

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Op 5 nov. 2012, om 21:40 heeft Grant Likely <[email protected]> het volgende geschreven:

> Hey folks,
>
> As promised, here is my early draft to try and capture what device
> tree overlays need to do and how to get there. Comments and
> suggestions greatly appreciated.
>
> Device Tree Overlay Feature
>
> Purpose
> =======
> Sometimes it is not convenient to describe an entire system with a
> single FDT. For example, processor modules that are plugged into one or
> more modules (a la the BeagleBone), or systems with an FPGA peripheral
> that is programmed after the system is booted.
>
> For these cases it is proposed to implement an overlay feature for the
> so that the initial device tree data can be modified by userspace at
> runtime by loading additional overlay FDTs that amend the original data.
>
> User Stories
> ============
> Note - These are potential use cases, but just because it is listed here
> doesn't mean it is important. I just want to thoroughly think through the
> implications before making design decisions.

I think the beaglebone use cases cover it as well, but it deserves a seperate mention: SOMs. Gumstix is a good example of those, their website has a list of the different expansionboards they sell so we can see if we missed a use case somewhere.
Their expansionboards have an EEPROM to ID them, just like the beagleboard classic/xM expansionboards, but I don't know if all 3rd party vendors honour that standard. I know the Ettus USRP E-100 on my desk has it.

regards,

Koen-

2012-11-12 11:23:57

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Grant,

Sorry for the late comments, travelling...

On Nov 9, 2012, at 6:28 PM, Grant Likely wrote:

> On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
>> On 11/05/2012 01:40 PM, Grant Likely wrote:
>>> Hey folks,
>>>
>>> As promised, here is my early draft to try and capture what device
>>> tree overlays need to do and how to get there. Comments and
>>> suggestions greatly appreciated.
>>
>> Interesting. This just came up internally at NVIDIA within the last
>> couple weeks, and was discussed on the U-Boot mailing list very recently
>> too:
>>
>> http://lists.denx.de/pipermail/u-boot/2012-October/thread.html#138227
>> (it spills into the November archive too)
>>
>>> For these cases it is proposed to implement an overlay feature for the
>>> so that the initial device tree data can be modified by userspace at
>>
>> I don't know if you're maintaining this as a document and taking patches
>> to it, but if so:
>
> Sure, why not...
>
> http://git.secretlab.ca/?p=devicetree-overlays.git;a=summary
>
>>
>> "for the so" split across those two lines.
>
> fixed
>
>>> - SHOULD reliably handle changes between different underlying overlays
>>> (ie. what happens to existing .dtb overly files if the structure of
>>> the dtb it is layered over changes. If not possible, then SHALL
>>> detect when the base tree doesn't match and refuse to apply the
>>> overlay.
>>
>> Perhaps use (versioned) DT bindings to represent the interface between
>> the two .dts files? See the links to the U-Boot mailing list discussions
>> below?
>
> Implementing versioning is conceptually a lot more complex than plain
> overlays since it means either the kernel or u-boot needs to start
> filtering the data that it's given. This can get really complex in a
> hurry. Mitch makes a valid point later in this thread that when it
> comes to manipulating the data depending on the board then the data
> overlay model alone doesn't handle it well.
>
> I'm not actually opposed to it, but it needs to be done in an elegant
> way. The DT data model already imposes more of a conceptual learning
> curve than I wish it did and I don't want to make that worse with a
> versioning model that is difficult to get ones head around.
>
> Suggestions and proposals are definitely welcome here.
>

I didn't find versioning all that difficult TBH.

Making the property access functions work sanely with versioning
should cover most (if not all) kernel users.
Keep non versioned property access function available to possibly
users with a prefix for those that need it.

>>
>>> - What is the model for overlays?
>>> - Can an overlay modify existing properties?
>>> - Can an overlay add new properties to existing nodes?
>>> - Can an overlay delete existing nodes/properties?
>>
>> This proposal is very oriented at an overlay-based approach. I'm not
>> totally convinced that a pure overlay approach (as in how dtc does
>> overlayed DT nodes) will be flexible enough, but would love to be
>> persuaded. Again see below.
>
> The way I'm currently thinking about the overlay approach is as a
> discrete set of changes that all can be applied at once.* That
> certainly doesn't preclude the change being generated with a script or
> other tool in either firmware or userspace. For most changes it works
> really well. Of the scenarios that don't work well, I can think of
> two. The first is it moving or renaming existing nodes, and the second
> is if the structure of the base tree changes (say due to a firmware
> update). Although the second limitation is specifically with binary
> .dtb overlays. Recompiling the dts overlay against the new tree would
> work fine.**
>

Atomicity should be handled correctly. I can't imagine how hard would
be to back out a semi-applied overlay without some kind of core DT
tracking support. Leaving it to the driver/user is too difficult to get
right.

About moving and renaming nodes, I can't think of a user-case today that
needs it. Perhaps it can be handled by removal & re-insertion?

> *with the caveat that not all types of changes are a good idea and we
> may disallow. For example, is changing properties in existing nodes a
> good idea?

Yes, changing properties is something that we need. One such change is
the change of the bus controller 'status' properties to enabled upon
insertion of a child device node, and change back to 'on-demand' when
all the child device nodes are gone.

> ** all this is based on my current ideas about the .dtb overlay format
> which would be trivial to implement, but I'm not committed to that
> approach just yet. The above problems could be solved.
>
>>> It may be sufficient to solve it by making the phandle values less
>>> volatile. Right now dtc generates phandles linearly. Generated phandles
>>> could be overridden with explicit phandle properties, but it isn't a
>>> fantastic solution. Perhaps generating the phandle from a hash of the
>>> node name would be sufficient.
>>
>> Node names don't have to be unique though right; perhaps hash the
>> path-name instead of the node-name? But then, why not just reference by
>> path name; similar to <{&/path/to/node}> rather than <&label>?
>
> I was thinking about using the full_name for generating phandles. On
> the odd chance there is a collision, one of the nodes will get
> something like the hash value +1. Or in that condition we can require
> one of the nodes to have the phandle value explicitly set in the
> source file.
>
> Note; this doesn't actually affect anything outside of dtc. Right now
> dtc starts at 1 and assigns phandles incrementally. I'm suggesting
> using a hash of the full_name to assign the phandle for a node so that
> it will not change unless the full_path changes.
>
>>> This handles many of the use cases, but it assumes that an overlay is
>>> board specific. If it ever is required to support multiple base boards
>>> with a single overlay file then there is a problem. The .dtb overlays
>>> generated in this manor cannot handle different phandles or nodes that
>>> are in a different place. On the other hand, the overlay source files
>>> should have no problem being compiled for multiple targets.
>>
>> s/manor/manner/
>
> Fixed
>
>>
>> I do rather suspect this use-case is quite common. NVIDIA certainly has
>> a bunch of development boards with pluggable
>> PMIC/audio/WiFi/display/..., and I believe there's some ability to
>> re-use the pluggable components with a variety of base-boards.
>>
>> Given people within NVIDIA started talking about this recently, I asked
>> them to enumerate all the boards we have that support pluggable
>> components, and how common it is that some boards support being plugged
>> into different main boards. I don't know when that enumeration will
>> complete (or even start) but hopefully I can provide some feedback on
>> how common the use-case is for us once it's done.
>
> From your perspective, is it important to use the exact same .dtb
> overlays for those add-on boards, or is it okay to have a separate
> build of the overlay for each base tree?
>
> g.

Regards

-- Pantelis

2012-11-12 11:29:37

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Grant,

On Nov 9, 2012, at 7:02 PM, Grant Likely wrote:

> On Wed, Nov 7, 2012 at 12:54 AM, Mitch Bradley <[email protected]> wrote:
>> On 11/6/2012 12:37 PM, Stephen Warren wrote:
>>> This proposal is very oriented at an overlay-based approach. I'm not
>>> totally convinced that a pure overlay approach (as in how dtc does
>>> overlayed DT nodes) will be flexible enough, but would love to be
>>> persuaded. Again see below.
>>
>>
>> An overlay approach will not be powerful enough to solve the sorts of
>> problems that occur when a product goes into full production, becomes a
>> family, and starts to evolve. Issues like second-source parts that
>> aren't quite compatible and need to be detected and reported,
>> board-stuff options for different customer profiles, speed grades of
>> parts that aren't properly probeable but instead need to be identified
>> by some subterfuge, the list of tedious issues goes on and on.
>>
>> It's nice to pretend that the world fits into a few coherent simple
>> use cases, but 30 years of experience shipping computer product families
>> proves otherwise. You need a programming language to solve the full
>> set of problems - which I why the device tree is designed so it can
>> be generated and modified by a program.
>
> I'm not going to argue with that. There is nothing saying that the
> overlay wouldn't be generated or applied by a script. However, I do
> strongly think that the data model needs to be independent of any tool
> or execution environment used to manipulate it. I certainly am not
> interested in encoding scripts or bytecode into the tree - the
> opposite of the approach used by ACPI which must execute AML to even
> retrieve the device tree. I like the overlay approach because it is
> conceptually straightforward and provides a working model of how
> changes could be made from userspace while still being usable by
> firmware.
>
> An alternate approach from userspace would be to use a live virtual
> filesystem to manipulate the device tree, though that approach only
> applies to Linux userspace. Firmware would still need its own
> approach.
>
> g.

I completely agree here.

I certainly wouldn't want to introduce any kind of bytecode or a
programming model for manipulating DT. I know OF has a full blown
forth interpreter, but that's not what modern DT should be (IMHO).

Having DT provide such a programming model will preclude a number
of users of accessing it, and on top of that, complexity will
increase considerably.

When faced with a new problem, vendors will just code a workaround,
never bothering fixing it properly.

In a nutshell, let's not turn DT into ACPI, please.

Regards

-- Pantelis

2012-11-12 11:34:22

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Grant,

On Nov 9, 2012, at 10:33 PM, Grant Likely wrote:

> On Wed, Nov 7, 2012 at 11:02 AM, Pantelis Antoniou
> <[email protected]> wrote:
>> On Nov 7, 2012, at 11:19 AM, Benoit Cousson wrote:
>>> Maybe some extra version match table can just be passed during the board machine_init
>>>
>>> of_platform_populate(NULL, omap_dt_match_table, NULL, NULL, panda_version_match_table);
>>>
>>
>> Would we need explicit of_platform_populate calls if we have node modification notifiers?
>> In that case the notifier would pick it up automatically, and can do the per
>> version matching internally.
>
> There still needs to be something to register "everything below this
> node is interesting" which is exactly what of_platform_populate() does
> now. I see the notifiers being used by the of_platform_populate
> backend to know when nodes have been created (or destroyed).
>
> g.

I see. So of_platform_populate could just register the notifier and
not do the tree walk itself. Perhaps the name is a bit misleading then?

Regards

-- Pantelis

2012-11-12 11:48:09

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Grant,

On Nov 9, 2012, at 11:22 PM, Grant Likely wrote:

> On Fri, Nov 9, 2012 at 5:32 AM, Joel A Fernandes <[email protected]> wrote:
>> Hi Pantelis,
>>
>> I hope I'm not too late to reply as I'm traveling.
>>
>> On Nov 6, 2012, at 5:30 AM, Pantelis Antoniou
>> <[email protected]> wrote:
>>
>>>
>>>>
>>>> Joanne has purchased one of Jane's capes and packaged it into a rugged
>>>> case for data logging. As far as Joanne is concerned, the BeagleBone and
>>>> cape together are a single unit and she'd prefer a single monolithic FDT
>>>> instead of using an FDT overlay.
>>>> Option A: Using dtc, she uses the BeagleBone and cape .dts source files
>>>> to generate a single .dtb for the entire system which is
>>>> loaded by U-Boot. -or-
>>>
>>> Unlikely.
>>>> Option B: Joanne uses a tool to merge the BeagleBone and cape .dtb files
>>>> (instead of .dts files), -or-
>>> Possible but low probability.
>>>
>>>> Option C: U-Boot loads both the base and overlay FDT files, merges them,
>>>> and passes the resolved tree to the kernel.
>>>>
>>>
>>> Could be made to work. Only really required if Joanne wants the
>>> cape interface to work for u-boot too. For example if the cape has some
>>> kind of network interface that u-boot will use to boot from.
>>>
>>
>> I love Grant's hashing idea a lot keeping the phandle problem for
>> compile time and not requiring fixups.
>>
>> IMO it is still a cleaner approach if u-boot does the simple tree merging for
>> all cases, and not the kernel reasons mentioned below.
>
> I'm more than sufficiently convinced that making changes at runtime
> from userspace is pretty much required.
>
> Also consider: it is order of magnitudes easier to modify the kernel
> than it is to change the firmware for end users.
>

Complete agreement here.

>> (1)
>> From a development standpoint, very little or nothing will
>> have to be changed in kernel (except for scripts/dtc) considering we
>> are moving forward with hashing.
>>
>> (2)
>> Also this discussed a while back but at some point is going to brought
>> up again- loading of dt fragment directly from EEPROM and merging at
>> run time. If we were to implement this in kernel, we would have to add
>> cape specific EEPROM reading code, merge the tree before it is
>> unflattened and parse.
>
> Unless it is required for boot to userspace I'm not considering
> merging before userspace starts. That's well after the tree is
> unflattened into the live form. If it is require to boot then I agree
> that is should be done in firmware. I see zero problem with having a
> beaglebone specific cape driver that knows to read the eeprom and
> request a specific configuration file. Heck, the kernel doesn't even
> need to parse the eeprom data. It can be read from userspace and
> userspace decides which overlay to provide. There's nothing stopping
> userspace from reading the eeprom, looking up the correct dts for the
> board, downloading the file from the Internet, compiling it with dtc
> and installing it.... and yes that is getting a little extreme.
>

We're trying to come up with the method that will work best for us.

>From an ease of use perspective, having a kernel driver doing the
probing and performing the DT fragment insertion looks the best.
It's especially nice for the manufacturer, since he can make sure
that when he ships a board a single kernel image will contain everything
with no possibility of RMAs.

For h/w prototyping, where the user is tinkering around with his
own design, the user space approach would be best.

The downloading over the internet DTS file, is a bit extreme for now :)

>> I think doing tree merging in kernel is messy
>> and we should do it in uboot considering we might have to read EEPROM for
>> this use case. Ideally reading the fragment from the EEPROM for all capes
>> and merging without worrying about version detection, Doing the merge and
>> passing the merged blob to the kernel which (kernel) works the same way
>> it does today.
>>
>>>> It may be sufficient to solve it by making the phandle values less
>>>> volatile. Right now dtc generates phandles linearly. Generated phandles
>>>> could be overridden with explicit phandle properties, but it isn't a
>>>> fantastic solution. Perhaps generating the phandle from a hash of the
>>>> node name would be sufficient.
>>>>
>>>
>>> I doubt the hash method will work reliably. We only have 32 bits to work with,
>>> nothing like the SHA hashes of git.
>>>
>>
>> I was wondering I have worked with kernel's crypto code in the past to
>> generate 32 bit md5sums of 1000s of dataitems, from what I've seen,
>> collisions are rare and since we are talking about just a few nodes
>> that are being referenced in the base dt. I think the probability is
>> even less (ofcourse such an analysis strongly depends on dataset).
>> this method also takes away a lot of complexity with having it to do
>> runtime fixups and will help us get off the ground quickly.
>
> It wouldn't be hard to put together a test and run it on all the .dts
> files in the kernel; generating md5 sums for all the full_name paths
> and seeing if we've got any collisions yet.
>
>> We can also put in a collision handling mechanism if needed.
>> I think it is worthy doing a sample hash of all nodes in all dts we
>> have in a script and see for once if we have collisions and what it
>> looks like.
>
> Collision handling is a must, but again this is all internal to dtc. I
> don't foresee any problems with it.
>
>> Alternatively to hashing, reading David Gibson's paper I followed,
>> phandle is supposed to 'uniquely' identity node. I wonder why the node
>> name itself is not sufficient to uniquely identify.
>
> Simply because the FDT draws upon the existing OFW bindings which use
> a 32bit value to reference other nodes.
>
>> The code that does
>> the tree walking can then just strcmp the node name while it walks the
>> tree instead of having to find a node with a phandle number. I guess
>> the reason is phandles are small to store as data values. Another
>> approach can be to arrange the string block in alphabetical order
>> (unless it already is), and store phandle as index of the node name
>> referenced relative to the starting of the strong block. This will not
>> affect nodes in dtb being moved around since they will still have the
>> same index value. the problem being adding or removing nodes Changes
>> the index of all other nodes in the string block as well.. Hmm.
>
> And that still doesn't help find all the phandle locations in the tree
> for doing fixups. It would need to be a table of
> nodes+properties+offsets that contain phandles for fixup to work, but
> I shy away from that approach because I think it will be fragile.
>
> g.

Maybe, but IMHO it will work. I don't know, we're at the point where
we have to start coding and see what come out.

Regards

-- Pantelis

2012-11-12 12:05:57

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Stephen,

On Nov 10, 2012, at 12:57 AM, Stephen Warren wrote:

> On 11/08/2012 07:26 PM, David Gibson wrote:
> ...
>> I also think graft will handle most of your use cases, although as I
>> said I don't fully understand the implications of some of them, so I
>> could be wrong. So, the actual insertion of the subtree is pretty
>> trivial to implement. phandles are the obvious other matter to be
>> dealt with. I haven't found the right thread where what you've
>> envisaged so far is discussed, so here are things I can see:
>>
>> 1) Avoiding phandle collisions between main tree and subtree (or
>> between subtrees).
>>
>> I'm hopeful that this can be resolved just by establishing some
>> conventions about the ranges of phandles to be used for various
>> components. I'd certainly be happy to add a directive to dtc which
>> enforces allocation of phandles within a specified range.
>
> One case I wonder about is:
>
> Base board A that's split into two .dts files e.g. due to there being
> two versions of the base board which are 90% identical, yet there being
> a few differences between them.
>
> Base board B that's just a single .dts file.
>
> We might allocate phandle range 0..999 for the base board.
>
> Child board X plugs into either, so the two base boards need to "export"
> the same set of phandle IDs to the child board to allow it to reference
> them.
>
> If base board A version 2 comes along after the phandle IDs that the
> child board uses are defined, then how do we handle assigning a specific
> phandle ID range to each of base board A's two version-specific
> overlays, when the version-specific changes might affect arbitrary
> phandle IDs within the range, and require some to be moved into the base
> board version-specific overlays and some to stay in the common base
> board .dts?
>

Static phandle allocation, could work, but not without considerable trouble.

Maybe we're over-engineering things. Perhaps having the kernel use the
phandle values generated by dtc is not required.

What about the following simple scheme:

1) Have dtc create local phandle values the same way, as it is today.
Generate the flat tree normally, but keep in a table the locations
of all phandle references. Keep track of non resolvable phandle references
in a different table. One could use the fixup mechanism I described in
a previous email, if you don't care about keeping a big table.
2) Upon loading the base DT or the overlay, the kernel makes sure the phandles
don't overlap; simply add a per DT fragment constant offset.
3) Resolve phandle references that were unresolved at compile time.

>> 2) Resolving phandle references within a subtree
>>
>> If we can handle (1) by convention, we don't need anything here, the
>> references are fine as is.
>>
>> (3) Resolving phandle references from the subtree to the main tree.
>>
>> So, I think this can actually be avoided, at least in cases where what
>> physical connections are available to the expansion module is well
>> defined. The main causes to have external references are interrupts
>> and gpios. Interrupts we handle by defining an interrupt specifier
>> space for the interrupts available on the expansion
>> socket/connector/bus/whatever. In the master tree we then have
>> something like:
>>
>> ...
>> expansion-socket@XXXX {
>> expansion-id = "SlotA";
>> interrupt-map = < /* map expansion irq specs to
>> board interrupt controllers */ >;
>> interrupt-map-mask = < ... >;
>> ranges = < /* map expansion local addresses to global
>> mmio */ >;
>> };
>
> We would end up needing an interrupt-map or ranges type of property for
> basically any resource type.
>
> For example, what about an I2C bus that's routed to the daughter board,
> and the daughter board contains an I2C bus mux, whose control path isn't
> through I2C but rather GPIOs? In this case, the I2C bus mux isn't a
> child of the I2C bus, but a separate entity that indicates its parent
> I2C bus using a phandle. I presume a similar argument applies to almost
> any kind of resource. This is probably do-able, but certainly something
> to consider with the socket approach. I do like the socket approach though.

Capebus has taken this approach.

You see, perhaps from the standpoint of the standard platform devices that
a cape provides, a bus is not a very fitting construct.

But from a hardware engineer/user perspective, a cape is an expansion board,
and virtual slots are used. This is similar to what you're saying
with socket approach, right?

Regards

-- Pantelis

2012-11-12 12:10:24

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Stephen,

On Nov 10, 2012, at 1:23 AM, Stephen Warren wrote:

> On 11/09/2012 09:28 AM, Grant Likely wrote:
>> On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
> ...
>>> I do rather suspect this use-case is quite common. NVIDIA certainly has
>>> a bunch of development boards with pluggable
>>> PMIC/audio/WiFi/display/..., and I believe there's some ability to
>>> re-use the pluggable components with a variety of base-boards.
>>>
>>> Given people within NVIDIA started talking about this recently, I asked
>>> them to enumerate all the boards we have that support pluggable
>>> components, and how common it is that some boards support being plugged
>>> into different main boards. I don't know when that enumeration will
>>> complete (or even start) but hopefully I can provide some feedback on
>>> how common the use-case is for us once it's done.
>>
>> From your perspective, is it important to use the exact same .dtb
>> overlays for those add-on boards, or is it okay to have a separate
>> build of the overlay for each base tree?
>
> I certainly think it'd be extremely beneficial to use the exact same
> child board .dtb with arbitrary base boards.
>

Oh yes. In fact if one was to use a single kernel image for beagleboard
and beaglebone, for the cape to work for both, it is required for it's
dtb to be compatible.

We're not there yet, but people would like to have an upgrade path.

beaglebone -> beagleboard -> pandaboard -> future-boards

It is not possible yet, but perhaps newer boards could have that.

> Consider something like the Arduino shield connector format, which I
> /believe/ has been re-used across a wide variety of Arduino boards and
> other compatible or imitation boards. Now consider a vendor of an
> Arduino shield. The shield vendor probably wants to publish a single
> .dtb file that works for users irrespective of which board they're using
> it with.
>
> (Well, I'm not sure that Arduino can run Linux; perhaps that's why you
> picked BeagleBone capes for your document!)
>

Arduino can't possible do it. Using arduino shields with an adapter could
be possible though.

> I suppose it would be acceptable for the shield vendor to ship the .dts
> file rather than the .dtb, and hence need to build the shield .dtb for a
> specific base board.
>
> However, I think the process for an end-user needs to be as simple as
> "drop this .dts/.dtb file into some standard directory", and I imagine
> it'll be much easier for distros/... to make that process work if
> they're dealing with a .dtb that they can just blast into the kernel's
> firmware loader interface, rather than having to also locate the
> base-board .dts/.dtb file, and run dtc and/or other tools on both .dts
> files together.

Yes. A single supported dtb is the way to go ideally.

Regards

-- Pantelis

2012-11-12 12:48:43

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Joel,

Again, sorry for the late reply due to travel.

On Nov 10, 2012, at 5:36 AM, Joel A Fernandes wrote:

> Hi Pantelis,
>
> On Fri, Nov 9, 2012 at 2:13 AM, Pantelis Antoniou
> <[email protected]> wrote:
>
>>>>> Option C: U-Boot loads both the base and overlay FDT files, merges them,
>>>>> and passes the resolved tree to the kernel.
>>>>>
>>>>
>>>> Could be made to work. Only really required if Joanne wants the
>>>> cape interface to work for u-boot too. For example if the cape has some
>>>> kind of network interface that u-boot will use to boot from.
>>>>
>>>
>>> I love Grant's hashing idea a lot keeping the phandle problem for
>>> compile time and not requiring fixups.
>>>
>>> IMO it is still a cleaner approach if u-boot does the tree merging for
>>> all cases, and not the kernel.
>>>
>>> That way from a development standpoint, very little or nothing will
>>> have to be changed in kernel (except for scripts/dtc) considering we
>>> are moving forward with hashing.
>>>
>>> Also this discussed a while back but at some point is going to brought
>>> up again- loading of dt fragment directly from EEPROM and merging at
>>> run time. If we were to implement this in kernel, we would have to add
>>> cape specific EEPROM reading code, merge the tree before it is
>>> unflattened and parse. I think doing tree merging in kernel is messy
>>> and we should do it in uboot. Ideally reading the fragment from the
>>> EEPROM for all capes and merging without worrying about version
>>> detection, Doing the merge and passing the merged blob to the kernel
>>> which (kernel) works the same way it does today.
>>
>> Not going to work, for a lot of cases. Doing it in the kernel seems to be
>> the cleaner option. There are valid use cases for doing in u-boot too.
>
> True, if dynamic runtime stuff from userspace is what we're talking
> about, then yeah I see the important need for kernel to do the merge.
>

Kernel doing the merge is our use case, and I feel it's the use case of
about 90% of users.

u-boot doing the merge is the rest.

>>> Alternatively to hashing, reading david gibsons paper I followed,
>>> phandle is supposed to 'uniquely' identity node. I wonder why the node
>>> name itself is not sufficient to unquiely identify. The code that does
>>> the tree walking can then just strcmp the node name while it walks the
>>> tree instead of having to find a node with a phandle number. I guess
>>> the reason is phandles are small to store as data values. Another
>>> approach can be to arrange the string block in alphabetical order
>>> (unless it already is), and store phandle as index of the node name
>>> referenced relative to the starting of the strong block. This will not
>>> affect nodes in dtb being moved around since they will still have the
>>> same index value. the problem being adding or removing nodes Changes
>>> the offsets of all other nodes in the string block as well.. Hmm.
>>>
>>
>> This is pretty radical change to the DT format, no?
>
> Yes, true and the only way hypothetically to replace the phandle
> tree-walking mechanism is to store node paths instead of phandle which
> David pointed is too long to store, so I guess this wont work after
> all. Anyway this was an interesting exercise, thanks.
>

It is always nice to have a fresh perspective. Thank you.

> Regards,
> Joel

Regards

-- Pantelis

2012-11-12 12:50:54

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Rob.

On Nov 11, 2012, at 10:47 PM, Rob Landley wrote:

> On 11/09/2012 10:28:59 AM, Grant Likely wrote:
>> On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
>> > On 11/05/2012 01:40 PM, Grant Likely wrote:
>> I'm not actually opposed to it, but it needs to be done in an elegant
>> way. The DT data model already imposes more of a conceptual learning
>> curve than I wish it did and I don't want to make that worse with a
>> versioning model that is difficult to get ones head around.
>
> Speaking of which...
>
> I want to poke at board emulation in qemu, from scratch. Specifically, I want to start with an unpopulated board (just the processor), add a block of physical memory and a serial device, and boot an initramfs in there with stdin/stdout. Then I want to incrementally add an RTC, network card, and three block devices.
>
> I'd like to define this board by giving qemu and the kernel the same device tree they can parse, and I'd like to _build_ this device tree so I understand what's in it. I'd like to repeat this exercize for arm, mips, ppc, x86, x86-64, sparc, sh4, and maybe other boards.
>
> And I'd like to write up an article on doing it as a learning exercise.
>
> Last time I checked, doing this wasn't possible. (qemu couldn't define a board by parsing a device tree, the kernel used the device tree as a guideline but it only really read data the drivers you linked in were expecting, the only documentation about what any of the nodes were was was read the other device trees as examples or read the source code of the drivers looking for data in the tree...)
>
> Is it a more realistic project now? If so, where would I start? (Once upon a time I read the booting-without-of document, back when it lived in the ppc directory. It didn't really say what should go in any of the nodes.)
>
> Rob

It should be possible when the stuff we're talking about is ready.

I don't know what you'll find is broken during the exercise, but I guess
your article is going to be entertaining at least :)

Regards

-- Pantelis

2012-11-12 13:01:49

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Mon, Nov 12, 2012 at 11:34 AM, Pantelis Antoniou
<[email protected]> wrote:
> Hi Grant,
>
> On Nov 9, 2012, at 10:33 PM, Grant Likely wrote:
>
>> On Wed, Nov 7, 2012 at 11:02 AM, Pantelis Antoniou
>> <[email protected]> wrote:
>>> On Nov 7, 2012, at 11:19 AM, Benoit Cousson wrote:
>>>> Maybe some extra version match table can just be passed during the board machine_init
>>>>
>>>> of_platform_populate(NULL, omap_dt_match_table, NULL, NULL, panda_version_match_table);
>>>>
>>>
>>> Would we need explicit of_platform_populate calls if we have node modification notifiers?
>>> In that case the notifier would pick it up automatically, and can do the per
>>> version matching internally.
>>
>> There still needs to be something to register "everything below this
>> node is interesting" which is exactly what of_platform_populate() does
>> now. I see the notifiers being used by the of_platform_populate
>> backend to know when nodes have been created (or destroyed).
>>
>> g.
>
> I see. So of_platform_populate could just register the notifier and
> not do the tree walk itself. Perhaps the name is a bit misleading then?

Kind of, yes. of_platform_populate() would still have the same effect
that it does now except that it would also pay attention to additions
and removals from the DT nodes it is interested in. This would work
cleanly enough for node additions/removals, but it wouldn't process
changes to properties on existing nodes.

g.

2012-11-12 16:45:51

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/09/2012 09:28 AM, Grant Likely wrote:
> On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
>> On 11/05/2012 01:40 PM, Grant Likely wrote:
>>> Hey folks,
>>>
>>> As promised, here is my early draft to try and capture what device
>>> tree overlays need to do and how to get there. Comments and
>>> suggestions greatly appreciated.
>>
>> Interesting. This just came up internally at NVIDIA within the last
>> couple weeks, and was discussed on the U-Boot mailing list very recently
>> too:
>>
>> http://lists.denx.de/pipermail/u-boot/2012-October/thread.html#138227
>> (it spills into the November archive too)
>>
>>> For these cases it is proposed to implement an overlay feature for the
>>> so that the initial device tree data can be modified by userspace at
>>
>> I don't know if you're maintaining this as a document and taking patches
>> to it, but if so:
>
> Sure, why not...
>
> http://git.secretlab.ca/?p=devicetree-overlays.git;a=summary
>
>>
>> "for the so" split across those two lines.
>
> fixed
>
>>> - SHOULD reliably handle changes between different underlying overlays
>>> (ie. what happens to existing .dtb overly files if the structure of
>>> the dtb it is layered over changes. If not possible, then SHALL
>>> detect when the base tree doesn't match and refuse to apply the
>>> overlay.
>>
>> Perhaps use (versioned) DT bindings to represent the interface between
>> the two .dts files? See the links to the U-Boot mailing list discussions
>> below?
>
> Implementing versioning is conceptually a lot more complex than plain
> overlays since it means either the kernel or u-boot needs to start
> filtering the data that it's given. This can get really complex in a
> hurry. Mitch makes a valid point later in this thread that when it
> comes to manipulating the data depending on the board then the data
> overlay model alone doesn't handle it well.

What I was thinking of sounds a lot simpler than what I think you're
responding to.

All I was thinking about was that we define some kind of explicit
interface between .dts files, e.g. some kind of direct representation of
the connector between the boards.

The versioning aspect is then:

Then, when the connector design changes, we change the naming of that
interface representation. This would happen in just the same way that
we'd name/represent the connector design of a BeagleBone differently
from that of an Arduino.

So, perhaps we start out with:

ti,beaglebone-cape-connector
arduino,shield-connector

and later end up with:

ti,beaglebone-cape-connector-v2

as far as any code that handled/references the v1/v2 interface
definitions, they'd probably be completely unrelated types in the
general case, just named very similarly.

2012-11-12 16:49:57

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/12/2012 04:23 AM, Pantelis Antoniou wrote:
> Hi Grant,
>
> Sorry for the late comments, travelling...
>
> On Nov 9, 2012, at 6:28 PM, Grant Likely wrote:
...
>> *with the caveat that not all types of changes are a good idea and we
>> may disallow. For example, is changing properties in existing nodes a
>> good idea?
>
> Yes, changing properties is something that we need. One such change is
> the change of the bus controller 'status' properties to enabled upon
> insertion of a child device node, and change back to 'on-demand' when
> all the child device nodes are gone.

Do we actually need to do that?

>From the base-board perspective, consider an SoC's I2C channel that is
routed to the child board connector. The routing to the connector is
always present on the base board. Only the presence (or lack thereof) of
devices on that I2C bus is influenced by whether a child board is
connected or not, and the design of the child board. Therefore, wouldn't
it make sense for the base board to always enable the I2C controller?

That would make it easier for someone to hook up a couple wires to the
I2C pins and use utilities such as i2cget/set to communicate with it,
without going through the whole process of creating a DT to represent
the device. This could speed up simple hacking/prototyping and allow it
to proceed in a less structured way.

2012-11-12 16:52:38

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/12/2012 05:10 AM, Pantelis Antoniou wrote:
> Hi Stephen,
>
> On Nov 10, 2012, at 1:23 AM, Stephen Warren wrote:
>
>> On 11/09/2012 09:28 AM, Grant Likely wrote:
>>> On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
>> ...
>>>> I do rather suspect this use-case is quite common. NVIDIA certainly has
>>>> a bunch of development boards with pluggable
>>>> PMIC/audio/WiFi/display/..., and I believe there's some ability to
>>>> re-use the pluggable components with a variety of base-boards.
>>>>
>>>> Given people within NVIDIA started talking about this recently, I asked
>>>> them to enumerate all the boards we have that support pluggable
>>>> components, and how common it is that some boards support being plugged
>>>> into different main boards. I don't know when that enumeration will
>>>> complete (or even start) but hopefully I can provide some feedback on
>>>> how common the use-case is for us once it's done.
>>>
>>> From your perspective, is it important to use the exact same .dtb
>>> overlays for those add-on boards, or is it okay to have a separate
>>> build of the overlay for each base tree?
>>
>> I certainly think it'd be extremely beneficial to use the exact same
>> child board .dtb with arbitrary base boards.
>>
>
> Oh yes. In fact if one was to use a single kernel image for beagleboard
> and beaglebone, for the cape to work for both, it is required for it's
> dtb to be compatible.

Well, as Grant pointed out, it's not actually strictly necessary for the
.dtb to be compatible; only the .dts /need/ be compatible, and the .dtb
can be generated at run-time using dtc for example.

Of course, relying on .dts compatibility rather than .dtb compatibility
might negatively impact the complexity of an initrd environment if we
end up loading overlays from there...

> We're not there yet, but people would like to have an upgrade path.
>
> beaglebone -> beagleboard -> pandaboard -> future-boards
>
> It is not possible yet, but perhaps newer boards could have that.

2012-11-12 16:54:08

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/12/2012 05:50 AM, Pantelis Antoniou wrote:
> Hi Rob.
>
> On Nov 11, 2012, at 10:47 PM, Rob Landley wrote:
>
>> On 11/09/2012 10:28:59 AM, Grant Likely wrote:
>>> On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
>>>> On 11/05/2012 01:40 PM, Grant Likely wrote:
>>> I'm not actually opposed to it, but it needs to be done in an elegant
>>> way. The DT data model already imposes more of a conceptual learning
>>> curve than I wish it did and I don't want to make that worse with a
>>> versioning model that is difficult to get ones head around.
>>
>> Speaking of which...
>>
>> I want to poke at board emulation in qemu, from scratch. Specifically, I want to start with an unpopulated board (just the processor), add a block of physical memory and a serial device, and boot an initramfs in there with stdin/stdout. Then I want to incrementally add an RTC, network card, and three block devices.
>>
>> I'd like to define this board by giving qemu and the kernel the same device tree they can parse, and I'd like to _build_ this device tree so I understand what's in it. I'd like to repeat this exercize for arm, mips, ppc, x86, x86-64, sparc, sh4, and maybe other boards.
>>
>> And I'd like to write up an article on doing it as a learning exercise.
>>
>> Last time I checked, doing this wasn't possible. (qemu couldn't define a board by parsing a device tree, the kernel used the device tree as a guideline but it only really read data the drivers you linked in were expecting, the only documentation about what any of the nodes were was was read the other device trees as examples or read the source code of the drivers looking for data in the tree...)
>>
>> Is it a more realistic project now? If so, where would I start? (Once upon a time I read the booting-without-of document, back when it lived in the ppc directory. It didn't really say what should go in any of the nodes.)
>>
>> Rob
>
> It should be possible when the stuff we're talking about is ready.
>
> I don't know what you'll find is broken during the exercise, but I guess
> your article is going to be entertaining at least :)

To be honest, I think Rob's proposal should be possible irrespective of
this conversation; if he wants to simply define the HW structure once
before running qemu, then none of this overlay discussion is relevant.

Perhaps the missing piece is the Documentation/devicetree/bindings/
directory?

2012-11-12 17:00:24

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Stephen,

On Nov 12, 2012, at 6:49 PM, Stephen Warren wrote:

> On 11/12/2012 04:23 AM, Pantelis Antoniou wrote:
>> Hi Grant,
>>
>> Sorry for the late comments, travelling...
>>
>> On Nov 9, 2012, at 6:28 PM, Grant Likely wrote:
> ...
>>> *with the caveat that not all types of changes are a good idea and we
>>> may disallow. For example, is changing properties in existing nodes a
>>> good idea?
>>
>> Yes, changing properties is something that we need. One such change is
>> the change of the bus controller 'status' properties to enabled upon
>> insertion of a child device node, and change back to 'on-demand' when
>> all the child device nodes are gone.
>
> Do we actually need to do that?
>
> From the base-board perspective, consider an SoC's I2C channel that is
> routed to the child board connector. The routing to the connector is
> always present on the base board. Only the presence (or lack thereof) of
> devices on that I2C bus is influenced by whether a child board is
> connected or not, and the design of the child board. Therefore, wouldn't
> it make sense for the base board to always enable the I2C controller?
>
> That would make it easier for someone to hook up a couple wires to the
> I2C pins and use utilities such as i2cget/set to communicate with it,
> without going through the whole process of creating a DT to represent
> the device. This could speed up simple hacking/prototyping and allow it
> to proceed in a less structured way.

Unfortunately that doesn't work for the beaglebone (am355x) and a large
number of general purpose SoCs.

What is different between general purpose SoCs and vertical market SoCs
(i.e. OMAP) is that the design of the the latter is for devices where
pretty much all the interfaces are expected to be active at the same time.

General purpose SoCs put more peripherals in the SoC, but due to packaging
considerations (and price), the peripheral pins are highly muxed.

So the i2c bus pins are shared with the LCD controller are shared with the
analog input and so on.

It is impossible (and on top of that really dangerous) to enable peripheral
blocks without knowing what's connected on the other side.

In a nutshell, for the bone and similar devices, probing with the devices
enabled doesn't work.

Regards

-- Pantelis

2012-11-12 17:10:59

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/12/2012 10:00 AM, Pantelis Antoniou wrote:
> Hi Stephen,
>
> On Nov 12, 2012, at 6:49 PM, Stephen Warren wrote:
>
>> On 11/12/2012 04:23 AM, Pantelis Antoniou wrote:
>>> Hi Grant,
>>>
>>> Sorry for the late comments, travelling...
>>>
>>> On Nov 9, 2012, at 6:28 PM, Grant Likely wrote:
>> ...
>>>> *with the caveat that not all types of changes are a good idea and we
>>>> may disallow. For example, is changing properties in existing nodes a
>>>> good idea?
>>>
>>> Yes, changing properties is something that we need. One such change is
>>> the change of the bus controller 'status' properties to enabled upon
>>> insertion of a child device node, and change back to 'on-demand' when
>>> all the child device nodes are gone.
>>
>> Do we actually need to do that?
>>
>> From the base-board perspective, consider an SoC's I2C channel that is
>> routed to the child board connector. The routing to the connector is
>> always present on the base board. Only the presence (or lack thereof) of
>> devices on that I2C bus is influenced by whether a child board is
>> connected or not, and the design of the child board. Therefore, wouldn't
>> it make sense for the base board to always enable the I2C controller?
>>
>> That would make it easier for someone to hook up a couple wires to the
>> I2C pins and use utilities such as i2cget/set to communicate with it,
>> without going through the whole process of creating a DT to represent
>> the device. This could speed up simple hacking/prototyping and allow it
>> to proceed in a less structured way.
>
> Unfortunately that doesn't work for the beaglebone (am355x) and a large
> number of general purpose SoCs.
>
> What is different between general purpose SoCs and vertical market SoCs
> (i.e. OMAP) is that the design of the the latter is for devices where
> pretty much all the interfaces are expected to be active at the same time.
>
> General purpose SoCs put more peripherals in the SoC, but due to packaging
> considerations (and price), the peripheral pins are highly muxed.
>
> So the i2c bus pins are shared with the LCD controller are shared with the
> analog input and so on.
>
> It is impossible (and on top of that really dangerous) to enable peripheral
> blocks without knowing what's connected on the other side.
>
> In a nutshell, for the bone and similar devices, probing with the devices
> enabled doesn't work.

I think this can be solved by deferring the pinmux to the .dts for the
child board. Presumably the I2C controller node can be enabled just
fine, but the I2C signals not actually routed out to any pins on the SoC
until the specific pinmux configuration is loaded.

But your explanation still feels a little odd; is it really true on the
BeagleBone, you can either use LCD for graphics /or/ you can use the I2C
controller to communicate with a cape? I certainly fully understand that
from a SoC's perspective (and soc.dtsi perspective) you can't just
enable everything due to pinmux constraints, but it seems that once you
get to the level of a particular board design, the designer has made
those trade-offs and so you know which specific combination is to be
selected.

2012-11-12 17:19:16

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Stephen,

On Nov 12, 2012, at 7:10 PM, Stephen Warren wrote:

> On 11/12/2012 10:00 AM, Pantelis Antoniou wrote:
>> Hi Stephen,
>>
>> On Nov 12, 2012, at 6:49 PM, Stephen Warren wrote:
>>
>>> On 11/12/2012 04:23 AM, Pantelis Antoniou wrote:
>>>> Hi Grant,
>>>>
>>>> Sorry for the late comments, travelling...
>>>>
>>>> On Nov 9, 2012, at 6:28 PM, Grant Likely wrote:
>>> ...
>>>>> *with the caveat that not all types of changes are a good idea and we
>>>>> may disallow. For example, is changing properties in existing nodes a
>>>>> good idea?
>>>>
>>>> Yes, changing properties is something that we need. One such change is
>>>> the change of the bus controller 'status' properties to enabled upon
>>>> insertion of a child device node, and change back to 'on-demand' when
>>>> all the child device nodes are gone.
>>>
>>> Do we actually need to do that?
>>>
>>> From the base-board perspective, consider an SoC's I2C channel that is
>>> routed to the child board connector. The routing to the connector is
>>> always present on the base board. Only the presence (or lack thereof) of
>>> devices on that I2C bus is influenced by whether a child board is
>>> connected or not, and the design of the child board. Therefore, wouldn't
>>> it make sense for the base board to always enable the I2C controller?
>>>
>>> That would make it easier for someone to hook up a couple wires to the
>>> I2C pins and use utilities such as i2cget/set to communicate with it,
>>> without going through the whole process of creating a DT to represent
>>> the device. This could speed up simple hacking/prototyping and allow it
>>> to proceed in a less structured way.
>>
>> Unfortunately that doesn't work for the beaglebone (am355x) and a large
>> number of general purpose SoCs.
>>
>> What is different between general purpose SoCs and vertical market SoCs
>> (i.e. OMAP) is that the design of the the latter is for devices where
>> pretty much all the interfaces are expected to be active at the same time.
>>
>> General purpose SoCs put more peripherals in the SoC, but due to packaging
>> considerations (and price), the peripheral pins are highly muxed.
>>
>> So the i2c bus pins are shared with the LCD controller are shared with the
>> analog input and so on.
>>
>> It is impossible (and on top of that really dangerous) to enable peripheral
>> blocks without knowing what's connected on the other side.
>>
>> In a nutshell, for the bone and similar devices, probing with the devices
>> enabled doesn't work.
>
> I think this can be solved by deferring the pinmux to the .dts for the
> child board. Presumably the I2C controller node can be enabled just
> fine, but the I2C signals not actually routed out to any pins on the SoC
> until the specific pinmux configuration is loaded.
>

Well, I2C is just an example; same thing happens with SPI, LCD, audio, etc.

So you'll end up hacking the device drivers to handle a weird state that
basically means 'don't activate until I tell you to go'. It is simpler
just not to create the device.

> But your explanation still feels a little odd; is it really true on the
> BeagleBone, you can either use LCD for graphics /or/ you can use the I2C
> controller to communicate with a cape? I certainly fully understand that
> from a SoC's perspective (and soc.dtsi perspective) you can't just
> enable everything due to pinmux constraints, but it seems that once you
> get to the level of a particular board design, the designer has made
> those trade-offs and so you know which specific combination is to be
> selected.

There are multiple I2C busses, multiple SPI buses, multiple options to
connect something that produces audio, and so on. It is not odd at the least
for the kind of market these chips are for.

Even TI doesn't know exactly the kind of applications these devices will be
used for. The chip is there to provide connection option at a (cheap) price
point.

Think of a swiss-army knife with some tools that can be used at the
same time, and some that can't.

Regards

-- Pantelis

2012-11-12 17:29:08

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/12/2012 10:19 AM, Pantelis Antoniou wrote:
> Hi Stephen,
>
> On Nov 12, 2012, at 7:10 PM, Stephen Warren wrote:
>
>> On 11/12/2012 10:00 AM, Pantelis Antoniou wrote:
>>> Hi Stephen,
>>>
>>> On Nov 12, 2012, at 6:49 PM, Stephen Warren wrote:
>>>
>>>> On 11/12/2012 04:23 AM, Pantelis Antoniou wrote:
>>>>> Hi Grant,
>>>>>
>>>>> Sorry for the late comments, travelling...
>>>>>
>>>>> On Nov 9, 2012, at 6:28 PM, Grant Likely wrote:
>>>> ...
>>>>>> *with the caveat that not all types of changes are a good idea and we
>>>>>> may disallow. For example, is changing properties in existing nodes a
>>>>>> good idea?
>>>>>
>>>>> Yes, changing properties is something that we need. One such change is
>>>>> the change of the bus controller 'status' properties to enabled upon
>>>>> insertion of a child device node, and change back to 'on-demand' when
>>>>> all the child device nodes are gone.
>>>>
>>>> Do we actually need to do that?
>>>>
>>>> From the base-board perspective, consider an SoC's I2C channel that is
>>>> routed to the child board connector. The routing to the connector is
>>>> always present on the base board. Only the presence (or lack thereof) of
>>>> devices on that I2C bus is influenced by whether a child board is
>>>> connected or not, and the design of the child board. Therefore, wouldn't
>>>> it make sense for the base board to always enable the I2C controller?
>>>>
>>>> That would make it easier for someone to hook up a couple wires to the
>>>> I2C pins and use utilities such as i2cget/set to communicate with it,
>>>> without going through the whole process of creating a DT to represent
>>>> the device. This could speed up simple hacking/prototyping and allow it
>>>> to proceed in a less structured way.
>>>
>>> Unfortunately that doesn't work for the beaglebone (am355x) and a large
>>> number of general purpose SoCs.
>>>
>>> What is different between general purpose SoCs and vertical market SoCs
>>> (i.e. OMAP) is that the design of the the latter is for devices where
>>> pretty much all the interfaces are expected to be active at the same time.
>>>
>>> General purpose SoCs put more peripherals in the SoC, but due to packaging
>>> considerations (and price), the peripheral pins are highly muxed.
>>>
>>> So the i2c bus pins are shared with the LCD controller are shared with the
>>> analog input and so on.
>>>
>>> It is impossible (and on top of that really dangerous) to enable peripheral
>>> blocks without knowing what's connected on the other side.
>>>
>>> In a nutshell, for the bone and similar devices, probing with the devices
>>> enabled doesn't work.
>>
>> I think this can be solved by deferring the pinmux to the .dts for the
>> child board. Presumably the I2C controller node can be enabled just
>> fine, but the I2C signals not actually routed out to any pins on the SoC
>> until the specific pinmux configuration is loaded.
>>
>
> Well, I2C is just an example; same thing happens with SPI, LCD, audio, etc.
>
> So you'll end up hacking the device drivers to handle a weird state that
> basically means 'don't activate until I tell you to go'. It is simpler
> just not to create the device.
>
>> But your explanation still feels a little odd; is it really true on the
>> BeagleBone, you can either use LCD for graphics /or/ you can use the I2C
>> controller to communicate with a cape? I certainly fully understand that
>> from a SoC's perspective (and soc.dtsi perspective) you can't just
>> enable everything due to pinmux constraints, but it seems that once you
>> get to the level of a particular board design, the designer has made
>> those trade-offs and so you know which specific combination is to be
>> selected.
>
> There are multiple I2C busses, multiple SPI buses, multiple options to
> connect something that produces audio, and so on. It is not odd at the least
> for the kind of market these chips are for.
>
> Even TI doesn't know exactly the kind of applications these devices will be
> used for. The chip is there to provide connection option at a (cheap) price
> point.
>
> Think of a swiss-army knife with some tools that can be used at the
> same time, and some that can't.

Yes, I fully understand the flexibility of the SoC; the NVIDIA Tegra SoC
I maintain has exactly the same kind of flexibility.

However, I'm still thining that presumably the pins that could be used
for hypothetical example as any of (a) LCD, (b) I2C controller 2, (c)
UART 3 have been routed to the cape-bus connector specifically for the
purpose of e.g. I2C controller 2, and hence their usage has been locked in.

Is that not the case? Is it instead true that some SoC pins are routed
to both an LCD connector /and/ the cape-bus connector so the board
itself has a usage conflict, or is it true that the cape-bus connector
is in fact just a set of 100 "random" pins each with no specific
pre-defined purpose, and it's up to the cape designer to pick the SoC's
pinmux configuration rather than the base board designer? That would
make the cape-bus pinout specific to the BeagleBone or the specific SoC
on the board, which if IIUC is something very different to the Arduino
shield pinout design, where each pin on the connector is designated for
a specific purpose, and hence arbitrary different SoCs could (and do)
host the same connector design without needing the same pinmux HW
design, which would obviously be impossible.

2012-11-12 17:38:37

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Stephen,

On Nov 12, 2012, at 7:29 PM, Stephen Warren wrote:

> On 11/12/2012 10:19 AM, Pantelis Antoniou wrote:
>> Hi Stephen,
>>
>> On Nov 12, 2012, at 7:10 PM, Stephen Warren wrote:
>>
>>> On 11/12/2012 10:00 AM, Pantelis Antoniou wrote:
>>>> Hi Stephen,
>>>>
>>>> On Nov 12, 2012, at 6:49 PM, Stephen Warren wrote:
>>>>
>>>>> On 11/12/2012 04:23 AM, Pantelis Antoniou wrote:
>>>>>> Hi Grant,
>>>>>>
>>>>>> Sorry for the late comments, travelling...
>>>>>>
>>>>>> On Nov 9, 2012, at 6:28 PM, Grant Likely wrote:
>>>>> ...
>>>>>>> *with the caveat that not all types of changes are a good idea and we
>>>>>>> may disallow. For example, is changing properties in existing nodes a
>>>>>>> good idea?
>>>>>>
>>>>>> Yes, changing properties is something that we need. One such change is
>>>>>> the change of the bus controller 'status' properties to enabled upon
>>>>>> insertion of a child device node, and change back to 'on-demand' when
>>>>>> all the child device nodes are gone.
>>>>>
>>>>> Do we actually need to do that?
>>>>>
>>>>> From the base-board perspective, consider an SoC's I2C channel that is
>>>>> routed to the child board connector. The routing to the connector is
>>>>> always present on the base board. Only the presence (or lack thereof) of
>>>>> devices on that I2C bus is influenced by whether a child board is
>>>>> connected or not, and the design of the child board. Therefore, wouldn't
>>>>> it make sense for the base board to always enable the I2C controller?
>>>>>
>>>>> That would make it easier for someone to hook up a couple wires to the
>>>>> I2C pins and use utilities such as i2cget/set to communicate with it,
>>>>> without going through the whole process of creating a DT to represent
>>>>> the device. This could speed up simple hacking/prototyping and allow it
>>>>> to proceed in a less structured way.
>>>>
>>>> Unfortunately that doesn't work for the beaglebone (am355x) and a large
>>>> number of general purpose SoCs.
>>>>
>>>> What is different between general purpose SoCs and vertical market SoCs
>>>> (i.e. OMAP) is that the design of the the latter is for devices where
>>>> pretty much all the interfaces are expected to be active at the same time.
>>>>
>>>> General purpose SoCs put more peripherals in the SoC, but due to packaging
>>>> considerations (and price), the peripheral pins are highly muxed.
>>>>
>>>> So the i2c bus pins are shared with the LCD controller are shared with the
>>>> analog input and so on.
>>>>
>>>> It is impossible (and on top of that really dangerous) to enable peripheral
>>>> blocks without knowing what's connected on the other side.
>>>>
>>>> In a nutshell, for the bone and similar devices, probing with the devices
>>>> enabled doesn't work.
>>>
>>> I think this can be solved by deferring the pinmux to the .dts for the
>>> child board. Presumably the I2C controller node can be enabled just
>>> fine, but the I2C signals not actually routed out to any pins on the SoC
>>> until the specific pinmux configuration is loaded.
>>>
>>
>> Well, I2C is just an example; same thing happens with SPI, LCD, audio, etc.
>>
>> So you'll end up hacking the device drivers to handle a weird state that
>> basically means 'don't activate until I tell you to go'. It is simpler
>> just not to create the device.
>>
>>> But your explanation still feels a little odd; is it really true on the
>>> BeagleBone, you can either use LCD for graphics /or/ you can use the I2C
>>> controller to communicate with a cape? I certainly fully understand that
>>> from a SoC's perspective (and soc.dtsi perspective) you can't just
>>> enable everything due to pinmux constraints, but it seems that once you
>>> get to the level of a particular board design, the designer has made
>>> those trade-offs and so you know which specific combination is to be
>>> selected.
>>
>> There are multiple I2C busses, multiple SPI buses, multiple options to
>> connect something that produces audio, and so on. It is not odd at the least
>> for the kind of market these chips are for.
>>
>> Even TI doesn't know exactly the kind of applications these devices will be
>> used for. The chip is there to provide connection option at a (cheap) price
>> point.
>>
>> Think of a swiss-army knife with some tools that can be used at the
>> same time, and some that can't.
>
> Yes, I fully understand the flexibility of the SoC; the NVIDIA Tegra SoC
> I maintain has exactly the same kind of flexibility.
>
> However, I'm still thining that presumably the pins that could be used
> for hypothetical example as any of (a) LCD, (b) I2C controller 2, (c)
> UART 3 have been routed to the cape-bus connector specifically for the
> purpose of e.g. I2C controller 2, and hence their usage has been locked in.
>
> Is that not the case? Is it instead true that some SoC pins are routed
> to both an LCD connector /and/ the cape-bus connector so the board
> itself has a usage conflict, or is it true that the cape-bus connector
> is in fact just a set of 100 "random" pins each with no specific
> pre-defined purpose, and it's up to the cape designer to pick the SoC's
> pinmux configuration rather than the base board designer? That would
> make the cape-bus pinout specific to the BeagleBone or the specific SoC
> on the board, which if IIUC is something very different to the Arduino
> shield pinout design, where each pin on the connector is designated for
> a specific purpose, and hence arbitrary different SoCs could (and do)
> host the same connector design without needing the same pinmux HW
> design, which would obviously be impossible.

There is no LCD specific connector, and no preset pin configuration.
The only set of pins that are defined to have a set configuration are
the pins of the i2c2 bus, on which the EEPROM identifying the cape
is located.

And of the pins that do connect a standard peripheral, all of them
can be reconfigured for basic GPIO for starters.

The beaglebone is infinitely more flexible and capable than an arduino.

Everything is up to the cape designer. All that the kernel can do
is handle conflicts and not enable capes that conflict with each other.

Regards

-- Pantelis

2012-11-12 20:16:27

by Russ Dill

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Mon, Nov 12, 2012 at 3:23 AM, Pantelis Antoniou
<[email protected]> wrote:
> Hi Grant,
>
> Sorry for the late comments, travelling...
>
> On Nov 9, 2012, at 6:28 PM, Grant Likely wrote:
>
>> On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
>>> On 11/05/2012 01:40 PM, Grant Likely wrote:
>>>> Hey folks,
>>>>
>>>> As promised, here is my early draft to try and capture what device
>>>> tree overlays need to do and how to get there. Comments and
>>>> suggestions greatly appreciated.
>>>
>>> Interesting. This just came up internally at NVIDIA within the last
>>> couple weeks, and was discussed on the U-Boot mailing list very recently
>>> too:
>>>
>>> http://lists.denx.de/pipermail/u-boot/2012-October/thread.html#138227
>>> (it spills into the November archive too)
>>>
>>>> For these cases it is proposed to implement an overlay feature for the
>>>> so that the initial device tree data can be modified by userspace at
>>>
>>> I don't know if you're maintaining this as a document and taking patches
>>> to it, but if so:
>>
>> Sure, why not...
>>
>> http://git.secretlab.ca/?p=devicetree-overlays.git;a=summary
>>
>>>
>>> "for the so" split across those two lines.
>>
>> fixed
>>
>>>> - SHOULD reliably handle changes between different underlying overlays
>>>> (ie. what happens to existing .dtb overly files if the structure of
>>>> the dtb it is layered over changes. If not possible, then SHALL
>>>> detect when the base tree doesn't match and refuse to apply the
>>>> overlay.
>>>
>>> Perhaps use (versioned) DT bindings to represent the interface between
>>> the two .dts files? See the links to the U-Boot mailing list discussions
>>> below?
>>
>> Implementing versioning is conceptually a lot more complex than plain
>> overlays since it means either the kernel or u-boot needs to start
>> filtering the data that it's given. This can get really complex in a
>> hurry. Mitch makes a valid point later in this thread that when it
>> comes to manipulating the data depending on the board then the data
>> overlay model alone doesn't handle it well.
>>
>> I'm not actually opposed to it, but it needs to be done in an elegant
>> way. The DT data model already imposes more of a conceptual learning
>> curve than I wish it did and I don't want to make that worse with a
>> versioning model that is difficult to get ones head around.
>>
>> Suggestions and proposals are definitely welcome here.
>>
>
> I didn't find versioning all that difficult TBH.
>
> Making the property access functions work sanely with versioning
> should cover most (if not all) kernel users.
> Keep non versioned property access function available to possibly
> users with a prefix for those that need it.
>
>>>
>>>> - What is the model for overlays?
>>>> - Can an overlay modify existing properties?
>>>> - Can an overlay add new properties to existing nodes?
>>>> - Can an overlay delete existing nodes/properties?
>>>
>>> This proposal is very oriented at an overlay-based approach. I'm not
>>> totally convinced that a pure overlay approach (as in how dtc does
>>> overlayed DT nodes) will be flexible enough, but would love to be
>>> persuaded. Again see below.
>>
>> The way I'm currently thinking about the overlay approach is as a
>> discrete set of changes that all can be applied at once.* That
>> certainly doesn't preclude the change being generated with a script or
>> other tool in either firmware or userspace. For most changes it works
>> really well. Of the scenarios that don't work well, I can think of
>> two. The first is it moving or renaming existing nodes, and the second
>> is if the structure of the base tree changes (say due to a firmware
>> update). Although the second limitation is specifically with binary
>> .dtb overlays. Recompiling the dts overlay against the new tree would
>> work fine.**
>>
>
> Atomicity should be handled correctly. I can't imagine how hard would
> be to back out a semi-applied overlay without some kind of core DT
> tracking support. Leaving it to the driver/user is too difficult to get
> right.
>
> About moving and renaming nodes, I can't think of a user-case today that
> needs it. Perhaps it can be handled by removal & re-insertion?
>
>> *with the caveat that not all types of changes are a good idea and we
>> may disallow. For example, is changing properties in existing nodes a
>> good idea?
>
> Yes, changing properties is something that we need. One such change is
> the change of the bus controller 'status' properties to enabled upon
> insertion of a child device node, and change back to 'on-demand' when
> all the child device nodes are gone.

Some devices will probably have to support on the fly changes that
they may not have otherwise supported. An example of a driver that
needs to modify it's device based on DT would be an I²C bus speed
change.

2012-11-13 00:30:52

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 09, 2012 at 09:08:14PM +0000, Grant Likely wrote:
> On Fri, Nov 9, 2012 at 2:26 AM, David Gibson
> <[email protected]> wrote:
> >> Summary points:
> >> - Create an FDT overlay data format and usage model
> >> - SHALL reliable resolve or validate of phandles between base and
> >> overlay trees
> >
> > So, I'm not at all clear on what this proposed phandle validation
> > would involve. I'm also not convinced it's as necessary as you
> > think, more on that below.
>
> Simply this: I'm taking this example from the omap3-beagle-xm.dts. It
> has the following stanza which is currently rolled into the resulting
> .dtb when compiled.
>
> &i2c1 {
> clock-frequency = <2600000>;
>
> twl: twl@48 {
> reg = <0x48>;
> interrupts = <7>; /* SYS_NIRQ cascaded to intc */
> interrupt-parent = <&intc>;
>
> vsim: regulator-vsim {
> compatible = "ti,twl4030-vsim";
> regulator-min-microvolt = <1800000>;
> regulator-max-microvolt = <3000000>;
> };
>
> twl_audio: audio {
> compatible = "ti,twl4030-audio";
> codec {
> };
> };
> };
> };
>
> However, if it were compiled into a separate dtb overlay it might look
> like this:
>
> / {
> .readonly;
> ocp {
> .readonly;
> interrupt-controller@48200000 {
> phandle = <0x1234>; /* EXPECTED PHANDLE */
> .readonly;
> };
> i2c@48070000 {
> .must-exist;
> clock-frequency = <2600000>;
>
> twl@48 {
> reg = <0x48>;
> interrupts = <7>;
> interrupt-parent = <0x1234>; /* RESOLVED PHANDLE */
>
> vsim: regulator-vsim {
> compatible = "ti,twl4030-vsim";
> regulator-min-microvolt = <1800000>;
> regulator-max-microvolt = <3000000>;
> };
>
> twl_audio: audio {
> compatible = "ti,twl4030-audio";
> codec {
> };
> };
> };
> };
> };
> };
>
> Notice I've included the intc node and it's phandle. By phandle
> validation I merely mean that when applying an overly the firmware or
> kernel must verify that the phandles in the overlay match the phandle
> in the base tree. If they don't match, then refuse to apply the
> overhead. This approach avoids the need to find and fixup phandles in
> the overlay. And if the phandle is generated from a hash of the
> full_name, then the resulting phandle will only change if the node
> moves.
>
> Similarly, at application time it should be verified that the nodes
> with a .readonly or .must-exist property could be verified to actually
> exist before attempting to apply the overlay. I used two different
> properties with the idea that only certain nodes would need to be
> modified... exactly what the policies should be is yet to be
> determined.

Ok, I see. I really don't like it much, but I understand.

--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson

2012-11-13 00:30:51

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 09, 2012 at 04:40:15PM +0100, Pantelis Antoniou wrote:
> Hi David,

[snip]
> > I think graft is basically a safer operation, particular if we're
> > doing this at runtime with userspace passing in these fdt fragments.
> > In fact I'd go so far as to say if you really need the full overlay
> > functionality, then you really ought to be working at the bootloader
> > or early kernel load level to assemble the correct full device tree.
> > And as Mitch says, an existing programming language (C, OFW Forth or
> > whatever as you please) will serve you better for this sort of general
> > manipulation than a limited template system.
> >
> > I also think graft will handle most of your use cases, although as I
> > said I don't fully understand the implications of some of them, so I
> > could be wrong. So, the actual insertion of the subtree is pretty
> > trivial to implement. phandles are the obvious other matter to be
> > dealt with. I haven't found the right thread where what you've
> > envisaged so far is discussed, so here are things I can see:
> >
>
> An overlay is more generic and can handle more complex cases.
> For our use case, a graft should work - with a few caveats.

Obviously an overlay is more generic. But the ability to arbitrarily
modify existing tree nodes, without any obvious way to enforce rules
about what can be altered and what can't frankly gives me the
heebie-jeebies.

> Due to the insertion/removal of the DT fragments other node's state
> change from 'disabled' <-> 'okay' and platform devices created or
> removed. This can be handled either via overlays, or via special
> casing it.

Ah, yes, the status transition is a good point.

[snip]
> > 1) Avoiding phandle collisions between main tree and subtree (or
> > between subtrees).
> >
> > I'm hopeful that this can be resolved just by establishing some
> > conventions about the ranges of phandles to be used for various
> > components. I'd certainly be happy to add a directive to dtc which
> > enforces allocation of phandles within a specified range.
> >
>
> Really doubtful IMHO. That will impose yet more structure on the DT
> syntax, and it will make it even more difficult for users.

Um.. I really don't see what's so hard about adding an incantation
like:
/phandle-range/ 1-0xffff;
at the top of your base dts. Then
/phandle-range/ 0x10000-0x1ffff;
to the plugin dts.

Especially if we made the most common base range the default.

> We're talking about users that do understand the hardware, but can't
> really grok linux kernel development.
>
> DT is used a structured h/w definition and seems to work just
> fine for these kind of users.
>
> > 2) Resolving phandle references within a subtree
> >
> > If we can handle (1) by convention, we don't need anything here, the
> > references are fine as is.
> >
> > (3) Resolving phandle references from the subtree to the main tree.
> >
> > So, I think this can actually be avoided, at least in cases where what
> > physical connections are available to the expansion module is well
> > defined. The main causes to have external references are interrupts
> > and gpios. Interrupts we handle by defining an interrupt specifier
> > space for the interrupts available on the expansion
> > socket/connector/bus/whatever. In the master tree we then have
> > something like:
> >
> > ...
> > expansion-socket@XXXX {
> > expansion-id = "SlotA";
> > interrupt-map = < /* map expansion irq specs to
> > board interrupt controllers */ >;
> > interrupt-map-mask = < ... >;
> > ranges = < /* map expansion local addresses to global
> > mmio */ >;
> > };
> >
> > The subtree for the expansion module gets attached as a subnode of
> > this one. It doesn't use explicit interrupt-parents but instead just
> > uses the expansion local irq specifiers, letting the parent be the
> > default which will bubble up to this socket node where the
> > interrupt-map will send them to the right places.
> >
> > I don't recall the gpio bindings off hand, but as I recall we based
> > them off the irq tree bindings so we ought to be able to do the same
> > thing for them.
>
> Please, don't try to do it this way. It is very debatable that we'll
> identify all the types that will need special handling.
> For example for our cape use, we already have references that do not
> follow this example.
>
> A generic way to resolve references is what we need.

Hrm... *mutters dubiously*.

--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson

2012-11-13 01:06:14

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 09, 2012 at 09:42:37PM +0000, Grant Likely wrote:
> On Fri, Nov 9, 2012 at 2:26 AM, David Gibson
> <[email protected]> wrote:
> > (3) Resolving phandle references from the subtree to the main tree.
> >
> > So, I think this can actually be avoided, at least in cases where what
> > physical connections are available to the expansion module is well
> > defined. The main causes to have external references are interrupts
> > and gpios. Interrupts we handle by defining an interrupt specifier
> > space for the interrupts available on the expansion
> > socket/connector/bus/whatever. In the master tree we then have
> > something like:
> >
> > ...
> > expansion-socket@XXXX {
> > expansion-id = "SlotA";
> > interrupt-map = < /* map expansion irq specs to
> > board interrupt controllers */ >;
> > interrupt-map-mask = < ... >;
> > ranges = < /* map expansion local addresses to global
> > mmio */ >;
> > };
> >
> > The subtree for the expansion module gets attached as a subnode of
> > this one. It doesn't use explicit interrupt-parents but instead just
> > uses the expansion local irq specifiers, letting the parent be the
> > default which will bubble up to this socket node where the
> > interrupt-map will send them to the right places.
> >
> > I don't recall the gpio bindings off hand, but as I recall we based
> > them off the irq tree bindings so we ought to be able to do the same
> > thing for them.
> >
> > Likewise, if there are several interchangeable expansion sockets that
> > have some address bits hard wired to distinguish them, we can just use
> > socket local mmio addresses within the subtree and the ranges property
> > here will sort that out.
>
> If I'm reading correctly, the suggestion is that everything be grafted
> below a single node and all connections route through that node using
> mapping. Correct?
>
> For interrupts that works today
> For gpios, it isn't currently supported, but we could do it.
> For SPI it would mean that the new spi devices would not appear below
> the actual spi bus they are attached to
> For I2C, MDIO, and one wire, same problem.
> For memory mapped devices, the expansion node would need to a ranges
> for all the windows that map through it, and it assumes only one
> memory mapped bus (or at least it prefers only one memory mapped bus.
> If there were more than one then the expansion node placement wouldn't
> have a natural place to sit)
>
> The problem is that this is not like a PCI bus where there is only one
> kind of interface. It is a whole bunch of interfaces that happen to be
> grouped together loosely (as an expansion connector in the beaglebone
> case, but expansion isn't the only problem that I'm hearing about).
>
> So, with a group of i2c, spi, memory mapped and other devices than are
> all plugged in together, how does that look? They really should not
> sit on the same level. An spi device cannot be a peer of an i2c device
> for instance, the address mapping is entirely different. The kernel
> really wants i2c devices to be a child of the actual i2c bus which may
> already have an i2c device or too on the main board. Does the
> expansion node need to have some kind of redirect node for each of the
> busses where the children of it need to create devices as children of
> the master bus?
>
> To me that seems to get really complex in a hurry. More complex than
> the overlay approach.

Ah, yes, I see. Yeah, that's a genuine showstopper for my original
proposal. Ok, let me offer a couple of counter-proposals:

1) bus-ranges

The notion of bus-reg and bus-ranges properties is something I've
toyed with before for other reasons, as a way of augmenting the
"normal" DT tree structure with interrupt-tree like DAG sections.
bus-reg and bus-ranges would act like the normal reg and ranges
properties except that each entry includes a phandle explicitly giving
the parent bus. In that case a suitable 'bus-ranges' in the socket
node would resurrect my graft proposal.

I'm not particularly sold on this idea, but I mention it because it
has applications other than this one. In particular it would mean we
could avoid having two different nodes for a device which is mostly
accessed via MMIO but also has a few sideband registers controlled by
i2c (or whatever).

2) graft bundle

The base tree has something like this:

...
i2c@XXX {
...
cape-socket {
compatible = "vendor,cape-socket";
id = "Socket-A";
piece-id = "i2c";
ranges = < ... >;
};
};
...
spi@YYY {
...
cape-socket {
compatible = "vendor,cape-socket";
id = "Socket-A";
piece-id = "spi";
ranges = < ... >;
};
};
...
cape-socket {
compatible = "vendor,cape-socket";
id = "Socket-A";
piece-id = "misc";
interrupt-map = < ... >;
interrupt-map-mask = < ... >;
gpio-map = < ... >;
gpio-map-mask = < ... >;
};

Then instead of grafting a single subtree for the socket, we install a
"bundle" of subtrees, one each for each of the pieces within the
socket. That bundle could either be an actual set of multiple fdts,
or they could be placed into one fdt with a dummy root node, something like:
/ {
plugin-bundle;
compatible = "vendor,cape-plugin";
version = ...;
i2c-piece = {
piece-id = "i2c";
...
};
misc-piece = {
piece-id = "misc";
...
};
};
That does have the additional advantage of letting us easily expand
the metadata in the future by adding more properties to the dummy root
node.

It's a bit more complex than the overlay proposal, but I think it's
safer - because the base tree clearly defines where it can be
extended. Plus it has the big advantage that it's no longer dependent
on the exact node paths in the base tree, the matching for each
component is done by the piece-id or other metadata.

--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson

2012-11-13 02:36:30

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Fri, Nov 09, 2012 at 09:36:26PM -0600, Joel A Fernandes wrote:
> Hi Pantelis,
>
> On Fri, Nov 9, 2012 at 2:13 AM, Pantelis Antoniou
> <[email protected]> wrote:
>
> >>>> Option C: U-Boot loads both the base and overlay FDT files, merges them,
> >>>> and passes the resolved tree to the kernel.
> >>>>
> >>>
> >>> Could be made to work. Only really required if Joanne wants the
> >>> cape interface to work for u-boot too. For example if the cape has some
> >>> kind of network interface that u-boot will use to boot from.
> >>>
> >>
> >> I love Grant's hashing idea a lot keeping the phandle problem for
> >> compile time and not requiring fixups.
> >>
> >> IMO it is still a cleaner approach if u-boot does the tree merging for
> >> all cases, and not the kernel.
> >>
> >> That way from a development standpoint, very little or nothing will
> >> have to be changed in kernel (except for scripts/dtc) considering we
> >> are moving forward with hashing.
> >>
> >> Also this discussed a while back but at some point is going to brought
> >> up again- loading of dt fragment directly from EEPROM and merging at
> >> run time. If we were to implement this in kernel, we would have to add
> >> cape specific EEPROM reading code, merge the tree before it is
> >> unflattened and parse. I think doing tree merging in kernel is messy
> >> and we should do it in uboot. Ideally reading the fragment from the
> >> EEPROM for all capes and merging without worrying about version
> >> detection, Doing the merge and passing the merged blob to the kernel
> >> which (kernel) works the same way it does today.
> >
> > Not going to work, for a lot of cases. Doing it in the kernel seems to be
> > the cleaner option. There are valid use cases for doing in u-boot too.
>
> True, if dynamic runtime stuff from userspace is what we're talking
> about, then yeah I see the important need for kernel to do the merge.
>
> >> Alternatively to hashing, reading david gibsons paper I followed,
> >> phandle is supposed to 'uniquely' identity node. I wonder why the node
> >> name itself is not sufficient to unquiely identify. The code that does
> >> the tree walking can then just strcmp the node name while it walks the
> >> tree instead of having to find a node with a phandle number. I guess
> >> the reason is phandles are small to store as data values. Another
> >> approach can be to arrange the string block in alphabetical order
> >> (unless it already is), and store phandle as index of the node name
> >> referenced relative to the starting of the strong block. This will not
> >> affect nodes in dtb being moved around since they will still have the
> >> same index value. the problem being adding or removing nodes Changes
> >> the offsets of all other nodes in the string block as well.. Hmm.
> >>
> >
> > This is pretty radical change to the DT format, no?
>
> Yes, true and the only way hypothetically to replace the phandle
> tree-walking mechanism is to store node paths instead of phandle which
> David pointed is too long to store, so I guess this wont work after
> all. Anyway this was an interesting exercise, thanks.

They're not too long to store, but changing to paths would break years
of existing OF practice.

--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson

2012-11-13 03:59:16

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Grant,

On Fri, Nov 9, 2012 at 3:22 PM, Grant Likely <[email protected]> wrote:
>> (2)
>> Also this discussed a while back but at some point is going to brought
>> up again- loading of dt fragment directly from EEPROM and merging at
>> run time. If we were to implement this in kernel, we would have to add
>> cape specific EEPROM reading code, merge the tree before it is
>> unflattened and parse.
>
> Unless it is required for boot to userspace I'm not considering
> merging before userspace starts. That's well after the tree is
> unflattened into the live form. If it is require to boot then I agree
> that is should be done in firmware. I see zero problem with having a
> beaglebone specific cape driver that knows to read the eeprom and
> request a specific configuration file. Heck, the kernel doesn't even
> need to parse the eeprom data. It can be read from userspace and
> userspace decides which overlay to provide. There's nothing stopping
> userspace from reading the eeprom, looking up the correct dts for the
> board, downloading the file from the Internet, compiling it with dtc
> and installing it.... and yes that is getting a little extreme.

Actually, I was speaking of the case where today we read EEPROM anyway
for capes before userspace, so it would convenient do the overlay this
way. Further, userspace doesn't have to do any parsing as
kernel/u-boot could blindly load the dtb overlay directly from EEPROM
regardless of which cape. So we don't have do any revision detection
and dtb search, the dtb itself being in the EEPROM.
That way no matter what the userspace is, the cape hardware is already
configured and ready to use by userspace. This might be a corner-case
just for beaglebone though, and if requesting overlay from userspace
is a more general solution for all usecases, then probably we should
go with that approach. OTOH, I guess there's no harm in doing some
overlays from kernel for some cases, and userspace for others.

>> The code that does
>> the tree walking can then just strcmp the node name while it walks the
>> tree instead of having to find a node with a phandle number. I guess
>> the reason is phandles are small to store as data values. Another
>> approach can be to arrange the string block in alphabetical order
>> (unless it already is), and store phandle as index of the node name
>> referenced relative to the starting of the strong block. This will not
>> affect nodes in dtb being moved around since they will still have the
>> same index value. the problem being adding or removing nodes Changes
>> the index of all other nodes in the string block as well.. Hmm.
>
> And that still doesn't help find all the phandle locations in the tree
> for doing fixups. It would need to be a table of
> nodes+properties+offsets that contain phandles for fixup to work, but
> I shy away from that approach because I think it will be fragile.

I guess this approach (which wouldn't work for other reasons) was
intended to serve the same purpose as Hashing, having a phandle that
doesn't easily change hence not requiring overlay fixups.

Actually I'm a bit confused if how maintaining a list of
name/node-paths for fix up will even work, because node names are not
unique as we discussed. Also node paths can change, so what will the
overlay dtb store in its fixup table (say there is one)? A table of
node,properties etc as you mentioned might not be unique as there can
be 2 nodes with same name and properties. Also if we use node path,
instead of node name, in this table, then moving nodes around wont be
possible as the fixup table will be pointing to node path references
that have now changed- how will the kernel now what's the node's new
actual path, say, if there are 2 nodes with the same name? Only thing
possible is, to do a _mandatory_ recompile of overlay dtb if a node's
path changes.

Would be you be in agreement that- When a node's path changes in base
dtb, Overlays have to be recompiled and a new fixup table has to be
created, or a new hash has to be generated out of the new node path
regardless. So moving nodes around in base dtb can't be dynamically
accommodated without recompiling overlay.

Considering the above is ok, the only 2 approaches I can see for
phandle resolutions are:
(1) hashing of the full node path and avoiding runtime fixups.
(2) Using today's linear phandle increment approach, maintaining a
table in the overlay with the key as 'node path' and value as 'phandle
fixup offset', and then doing runtime fixups.

I believe (2) is considered fragile and it does the same thing as (1)
except that (2) can even handle collisions but at the cost of storing
the full node path in the overlay's fixup table. So (1) looks like our
only possible and meaningful approach. Do you agree with this
analysis? Or, maybe I am missing something.

Regards,
Joel

2012-11-13 05:22:11

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/12/2012 06:05 PM, David Gibson wrote:
> On Fri, Nov 09, 2012 at 09:42:37PM +0000, Grant Likely wrote:
...
> 2) graft bundle
>
> The base tree has something like this:
>
> ...
> i2c@XXX {
> ...
> cape-socket {
> compatible = "vendor,cape-socket";
> id = "Socket-A";
> piece-id = "i2c";
> ranges = < ... >;
> };
> };
> ...
> spi@YYY {
> ...
> cape-socket {
> compatible = "vendor,cape-socket";
> id = "Socket-A";
> piece-id = "spi";
> ranges = < ... >;
> };
> };
> ...
> cape-socket {
> compatible = "vendor,cape-socket";
> id = "Socket-A";
> piece-id = "misc";
> interrupt-map = < ... >;
> interrupt-map-mask = < ... >;
> gpio-map = < ... >;
> gpio-map-mask = < ... >;
> };
>
> Then instead of grafting a single subtree for the socket, we install a
> "bundle" of subtrees, one each for each of the pieces within the
> socket. That bundle could either be an actual set of multiple fdts,
> or they could be placed into one fdt with a dummy root node, something like:
>
> / {
> plugin-bundle;
> compatible = "vendor,cape-plugin";
> version = ...;
> i2c-piece = {
> piece-id = "i2c";
> ...
> };
> misc-piece = {
> piece-id = "misc";
> ...
> };
> };

I do like this approach; it's the kind of thing I proposed at:

> http://www.mail-archive.com/[email protected]/msg20414.html

One question though: Perhaps the base board has two instances of the
same type of connector vendor,cape-socket, allowing 2 independent capes
to be plugged in. When overlaying/grafting the child board's .dts, we'd
need some way to specify the socket ID that was being plugged into. Is
that the intent of the "id" property in your base board example above?

2012-11-13 07:23:56

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Mon, Nov 12, 2012 at 09:52:32AM -0700, Stephen Warren wrote:
> On 11/12/2012 05:10 AM, Pantelis Antoniou wrote:
[snip]
> > Oh yes. In fact if one was to use a single kernel image for beagleboard
> > and beaglebone, for the cape to work for both, it is required for it's
> > dtb to be compatible.
>
> Well, as Grant pointed out, it's not actually strictly necessary for the
> .dtb to be compatible; only the .dts /need/ be compatible, and the .dtb
> can be generated at run-time using dtc for example.

So, actually, I think a whole bunch of problems with phandle
resolution disappear if we don't try to define an overlay .dtb format,
or at least treat it only as a very shortlived object. A more precise
proposal below. Note that this works more or less equally well with
either the original overlay approach or the graft/graft-bundle
proposal I made elsewhere.

1) We annotate the base tree with some extra label information for
nodes which overlays are likely to want to reference by phandle. e.g.

beaglebone_pic: interrupt-controller@XXXXX {
...
phandle,symbolic-name = "beaglebone_pic";
};

We could extend dtc to (optionally?) auto-generate those properties
from its existing label syntax. Not sure if that's a good idea or
not yet. In any case, we compile this augmented base tree to .dtb as
normal and boot our kernel with it.

2) The information for the capes/modules/whatever is
distributed/packaged as .dts, never .dtb. When userspace detects the
new module (or the user explicitly tells it, if it's not probeable) it
picks up the correct dts and runs it through dtc in a special mode.
In this mode dtc takes the existing base tree (from /proc/device-tree,
say) as well as the new dts. In this mode, dtc allocates phandles for
the new tree fragment so as not to collide with anything from the
supplied base tree (as well as avoiding internal conflicts,
obviously). It also allows node references to the base tree by using
those label annotations from (1) to match symbolic names to the
phandle values in the base tree.

3) The resulting partial .dtb for the module is highly specific to the
base tree (which if the base tree was generated at runtime by firmware
could even be specific to a particular boot). But that's ok, because
we just spit it into the kernel, absolute phandle values and all, then
throw it away. Next time we need the module info, we recompile it
again.

> Of course, relying on .dts compatibility rather than .dtb compatibility
> might negatively impact the complexity of an initrd environment if we
> end up loading overlays from there...

Well, it does mean we'd need dtc in the initrd. But dtc has no
library dependencies except libc, so that really shouldn't be too
bad. In return we entirely avoid inventing a new phandle resolution
protocol.

--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson

2012-11-13 07:23:55

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Mon, Nov 12, 2012 at 10:22:07PM -0700, Stephen Warren wrote:
> On 11/12/2012 06:05 PM, David Gibson wrote:
> > On Fri, Nov 09, 2012 at 09:42:37PM +0000, Grant Likely wrote:
> ...
> > 2) graft bundle
> >
> > The base tree has something like this:
> >
> > ...
> > i2c@XXX {
> > ...
> > cape-socket {
> > compatible = "vendor,cape-socket";
> > id = "Socket-A";
> > piece-id = "i2c";
> > ranges = < ... >;
> > };
> > };
> > ...
> > spi@YYY {
> > ...
> > cape-socket {
> > compatible = "vendor,cape-socket";
> > id = "Socket-A";
> > piece-id = "spi";
> > ranges = < ... >;
> > };
> > };
> > ...
> > cape-socket {
> > compatible = "vendor,cape-socket";
> > id = "Socket-A";
> > piece-id = "misc";
> > interrupt-map = < ... >;
> > interrupt-map-mask = < ... >;
> > gpio-map = < ... >;
> > gpio-map-mask = < ... >;
> > };
> >
> > Then instead of grafting a single subtree for the socket, we install a
> > "bundle" of subtrees, one each for each of the pieces within the
> > socket. That bundle could either be an actual set of multiple fdts,
> > or they could be placed into one fdt with a dummy root node, something like:
> >
> > / {
> > plugin-bundle;
> > compatible = "vendor,cape-plugin";
> > version = ...;
> > i2c-piece = {
> > piece-id = "i2c";
> > ...
> > };
> > misc-piece = {
> > piece-id = "misc";
> > ...
> > };
> > };
>
> I do like this approach; it's the kind of thing I proposed at:
>
> > http://www.mail-archive.com/[email protected]/msg20414.html

Roughly, yes, though a little streamlined from the syntax suggested
there.

> One question though: Perhaps the base board has two instances of the
> same type of connector vendor,cape-socket, allowing 2 independent capes
> to be plugged in. When overlaying/grafting the child board's .dts, we'd
> need some way to specify the socket ID that was being plugged into. Is
> that the intent of the "id" property in your base board example above?

Yes, that's exactly what I had in mind for the "id" property.
Property names and other details entirely negotiable at this stage,
of course.

By the by, I think having multiple interchangable sockets could break
the convention based approach for avoiding collisions between phandles
I suggested, but another mail with some better thoughts on that
shortly to be posted.

--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson

2012-11-13 08:09:37

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi David,

On Nov 13, 2012, at 9:25 AM, David Gibson wrote:

> On Mon, Nov 12, 2012 at 09:52:32AM -0700, Stephen Warren wrote:
>> On 11/12/2012 05:10 AM, Pantelis Antoniou wrote:
> [snip]
>>> Oh yes. In fact if one was to use a single kernel image for beagleboard
>>> and beaglebone, for the cape to work for both, it is required for it's
>>> dtb to be compatible.
>>
>> Well, as Grant pointed out, it's not actually strictly necessary for the
>> .dtb to be compatible; only the .dts /need/ be compatible, and the .dtb
>> can be generated at run-time using dtc for example.
>
> So, actually, I think a whole bunch of problems with phandle
> resolution disappear if we don't try to define an overlay .dtb format,
> or at least treat it only as a very shortlived object. A more precise
> proposal below. Note that this works more or less equally well with
> either the original overlay approach or the graft/graft-bundle
> proposal I made elsewhere.
>
> 1) We annotate the base tree with some extra label information for
> nodes which overlays are likely to want to reference by phandle. e.g.
>
> beaglebone_pic: interrupt-controller@XXXXX {
> ...
> phandle,symbolic-name = "beaglebone_pic";
> };
>
> We could extend dtc to (optionally?) auto-generate those properties
> from its existing label syntax. Not sure if that's a good idea or
> not yet. In any case, we compile this augmented base tree to .dtb as
> normal and boot our kernel with it.
>

I'm fine with that. You can auto-generate when there's a label to a node.
The cape dt fragment will use the label name to reference a node.
More details below...

> 2) The information for the capes/modules/whatever is
> distributed/packaged as .dts, never .dtb. When userspace detects the
> new module (or the user explicitly tells it, if it's not probeable) it
> picks up the correct dts and runs it through dtc in a special mode.
> In this mode dtc takes the existing base tree (from /proc/device-tree,
> say) as well as the new dts. In this mode, dtc allocates phandles for
> the new tree fragment so as not to collide with anything from the
> supplied base tree (as well as avoiding internal conflicts,
> obviously). It also allows node references to the base tree by using
> those label annotations from (1) to match symbolic names to the
> phandle values in the base tree.
>

Not good to rely on userspace kicking off dtc and compiling from source.
Some capes/expansion boards might have your root fs device, for example
there is an eMMC cape coming up, while networking capes are common too.

However I have a compromise.

I agree that compiling from source can be an option for a runtime definable
cape, but for built-in capes we could do a bit better.

In particular, I don't see any particular need to have a DT fragment
reference anything that dependent of the runtime device tree. It should
be possible to compile the DT fragment in kernel, against the generated
flattened device tree, producing a flattened DT fragment with the phandles
already resolved.

So the sequence could be something like this:

$ dtc -O dtb -o am335x-bone.dtb -b 0 am335x-bone.dts -@ ${LAST_PHANDLE_FILE}
$ dtc -O dtbf -R am335x-bone.dtb -o weather-cape.dtb -b 0 weather-cape.dts -@ ${LAST_PHANDLE_FILE}
$ dtc -O dtbf -R am335x-bone.dtb -o geiger-cape.dtb -b 0 geiger-cape.dts -@ ${LAST_PHANDLE_FILE}

The ${LAST_PHANDLE_FILE} can be updated with the last phandle value generated.

Alternatively we could have a way to statically assign a phandle range
for well known capes. All the others will have to use the runtime compile
mechanism.
$ dtc -O dtb -o am335x-bone.dtb -b 0 am335x-bone.dts
$ dtc -O dtbf -R am335x-bone.dtb -o weather-cape.dtb -b 0 weather-cape.dts
$ dtc -O dtbf -R am335x-bone.dtb -o geiger-cape.dtb -b 0 geiger-cape.dts

With the cape dtses having a /phandle-range/ statement at the top.

This can work because the cape dts do not cross-reference each other, and
neither the boot dts references the capes.

That way we can use request_firmware() pretty early in the boot sequence
and get the DT fragment we need even before user-space starts and root fs
has mounted. request_firmware() can locate the fragments in the kernel
image before rootfs.

I don't know if this will cover all the cases Grant has in mind though.

So just to make sure I got it right, this could work for our case.

i2c2: i2c@4819c000 {
compatible = "ti,omap4-i2c";
#address-cells = <1>;
#size-cells = <0>;
ti,hwmods = "i2c3";
reg = <0x4819c000 0x1000>;
interrupt-parent = <&intc>;
interrupts = <30>;
status = "disabled";
};

And in the cape definition (when compiled with the special mode I describe
below)

/ {
plugin-bundle;
compatible = "cco,weather-cape";
version = <00A0>;

i2c2-graft = {
compatible = <dt,graft>;
graft-point = <&i2c2>;

#address-cells = <1>;
#size-cells = <0>;

/* Ambient light sensor */
tsl2550@39 {
compatible = "tsl,tsl2550";
reg = <0x39>;
};
};
};

DTC when compiling in the special fragment mode will pick up that
&i2c2 can not be resolved and lookup the phandle on the main dtb.
That way, even 'phandle,symbolic-name = "i2c2";' is redundant.

> 3) The resulting partial .dtb for the module is highly specific to the
> base tree (which if the base tree was generated at runtime by firmware
> could even be specific to a particular boot). But that's ok, because
> we just spit it into the kernel, absolute phandle values and all, then
> throw it away. Next time we need the module info, we recompile it
> again.
>
>> Of course, relying on .dts compatibility rather than .dtb compatibility
>> might negatively impact the complexity of an initrd environment if we
>> end up loading overlays from there...
>
> Well, it does mean we'd need dtc in the initrd. But dtc has no
> library dependencies except libc, so that really shouldn't be too
> bad. In return we entirely avoid inventing a new phandle resolution
> protocol.
>

Not every board boots with initrd; most embedded boards don't use it
at all. This way we make initrd a hard requirement.

It's best if we avoid it.

> --
> David Gibson | I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
> | _way_ _around_!
> http://www.ozlabs.org/~dgibson

Regards

-- Pantelis

2012-11-13 12:25:04

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Tue, Nov 13, 2012 at 8:09 AM, Pantelis Antoniou
<[email protected]> wrote:
> On Nov 13, 2012, at 9:25 AM, David Gibson wrote:
> Not good to rely on userspace kicking off dtc and compiling from source.
> Some capes/expansion boards might have your root fs device, for example
> there is an eMMC cape coming up, while networking capes are common too.
>
> However I have a compromise.
>
> I agree that compiling from source can be an option for a runtime definable
> cape, but for built-in capes we could do a bit better.
>
> In particular, I don't see any particular need to have a DT fragment
> reference anything that dependent of the runtime device tree. It should
> be possible to compile the DT fragment in kernel, against the generated
> flattened device tree, producing a flattened DT fragment with the phandles
> already resolved.

Do you mean linking dtc into the kernel?

> So the sequence could be something like this:
>
> $ dtc -O dtb -o am335x-bone.dtb -b 0 am335x-bone.dts -@ ${LAST_PHANDLE_FILE}
> $ dtc -O dtbf -R am335x-bone.dtb -o weather-cape.dtb -b 0 weather-cape.dts -@ ${LAST_PHANDLE_FILE}
> $ dtc -O dtbf -R am335x-bone.dtb -o geiger-cape.dtb -b 0 geiger-cape.dts -@ ${LAST_PHANDLE_FILE}
>
> The ${LAST_PHANDLE_FILE} can be updated with the last phandle value generated.
>
> Alternatively we could have a way to statically assign a phandle range
> for well known capes. All the others will have to use the runtime compile
> mechanism.
> $ dtc -O dtb -o am335x-bone.dtb -b 0 am335x-bone.dts
> $ dtc -O dtbf -R am335x-bone.dtb -o weather-cape.dtb -b 0 weather-cape.dts
> $ dtc -O dtbf -R am335x-bone.dtb -o geiger-cape.dtb -b 0 geiger-cape.dts
>
> With the cape dtses having a /phandle-range/ statement at the top.

I really think the whole phandle allocation thing is a non problem.
Generating phandles is the easy part, and using a good hash function
pretty much solves it entirely. I know people are worried about
collisions, but there are only two scenarios where collisions may
happen:

1) base and overlay try to define same phandle
- Easy to detect - kernel can complain and refuse to load if it happens
- Will never happen when overlay is compiled against the base used
to boot (dtc can resolve the conflict)
- Hash phandle generation makes this exceptionally rare
2) two overlays try to define same phandle
- Also easy to detect on loading the second overlay - kernel will
- Hash phandle generation still makes this exceptionally rare

In both cases a collision is extremely rare and if it does happen it
will not fail silently. Besides, in the odd scenario where it does
happen, a node can be manually assigned a phandle. It is far better to
do the manual override maybe once or twice (actually, I'd be surprised
if it is ever needed) than to get into any kind of numberspace
management for phandles. That's just a maintenance nightmare.

The hard bit to solve has always been when the overlay expects
different phandles than the base is using, and that problem only
occurs if the overlay is compiled against a different base dtb than
was used to boot the system. Hashed phandle generation even makes
phandles very stable when the base dtb is recompiled, and if they do
change, then the kernel can detect it and report an error. Again, no
silent failures. Phandle fixup does make the problem disappears
entirely but I'm concerned that it is still incomplete (see below) and
would be conceptually expensive.

Once of the requests is to support one overlay .dtb with many
different baseboards, but now that I've thought about it more I
realize that it just won't work for anything beyond the most trivial
case. Phandle fixup isn's sufficient. The format of GPIO, Interrupt,
clock, etc specifiers may change if the base board nodes change. The
specifiers are not portable.

So, I no longer think that plain dtb overlays alone will work against
any base board. Something more is needed. It may be the mechanism for
loading new data into the kernel, but something really generic does
require either being compiled at runtime (as David suggests) or each
expansion interface (ie. the BeagleBone expansion or the RPi
expansion) to really specify what kinds of references are allowed and
run all specifiers through translation nodes. Similar to how
interrupt-map works.

> i2c2: i2c@4819c000 {
> compatible = "ti,omap4-i2c";
> #address-cells = <1>;
> #size-cells = <0>;
> ti,hwmods = "i2c3";
> reg = <0x4819c000 0x1000>;
> interrupt-parent = <&intc>;
> interrupts = <30>;
> status = "disabled";
> };
>
> And in the cape definition (when compiled with the special mode I describe
> below)
>
> / {
> plugin-bundle;
> compatible = "cco,weather-cape";
> version = <00A0>;
>
> i2c2-graft = {
> compatible = <dt,graft>;
> graft-point = <&i2c2>;

Since overlays are different, we can interpret them slightly
differently. The node name itself could be the graft point, and the
name could be either a full path "/foo/bar/i2c@10002000" or an alias
reference "i2c2". I think that is a bit better than a new graft-point
property.

g.

2012-11-13 13:38:35

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Grant,

On Nov 13, 2012, at 2:24 PM, Grant Likely wrote:

> On Tue, Nov 13, 2012 at 8:09 AM, Pantelis Antoniou
> <[email protected]> wrote:
>> On Nov 13, 2012, at 9:25 AM, David Gibson wrote:
>> Not good to rely on userspace kicking off dtc and compiling from source.
>> Some capes/expansion boards might have your root fs device, for example
>> there is an eMMC cape coming up, while networking capes are common too.
>>
>> However I have a compromise.
>>
>> I agree that compiling from source can be an option for a runtime definable
>> cape, but for built-in capes we could do a bit better.
>>
>> In particular, I don't see any particular need to have a DT fragment
>> reference anything that dependent of the runtime device tree. It should
>> be possible to compile the DT fragment in kernel, against the generated
>> flattened device tree, producing a flattened DT fragment with the phandles
>> already resolved.
>
> Do you mean linking dtc into the kernel?

No, no :)

Typo there, s/in kernel/outside of the kernel/

>
>> So the sequence could be something like this:
>>
>> $ dtc -O dtb -o am335x-bone.dtb -b 0 am335x-bone.dts -@ ${LAST_PHANDLE_FILE}
>> $ dtc -O dtbf -R am335x-bone.dtb -o weather-cape.dtb -b 0 weather-cape.dts -@ ${LAST_PHANDLE_FILE}
>> $ dtc -O dtbf -R am335x-bone.dtb -o geiger-cape.dtb -b 0 geiger-cape.dts -@ ${LAST_PHANDLE_FILE}
>>
>> The ${LAST_PHANDLE_FILE} can be updated with the last phandle value generated.
>>
>> Alternatively we could have a way to statically assign a phandle range
>> for well known capes. All the others will have to use the runtime compile
>> mechanism.
>> $ dtc -O dtb -o am335x-bone.dtb -b 0 am335x-bone.dts
>> $ dtc -O dtbf -R am335x-bone.dtb -o weather-cape.dtb -b 0 weather-cape.dts
>> $ dtc -O dtbf -R am335x-bone.dtb -o geiger-cape.dtb -b 0 geiger-cape.dts
>>
>> With the cape dtses having a /phandle-range/ statement at the top.
>
> I really think the whole phandle allocation thing is a non problem.
> Generating phandles is the easy part, and using a good hash function
> pretty much solves it entirely. I know people are worried about
> collisions, but there are only two scenarios where collisions may
> happen:
>
> 1) base and overlay try to define same phandle
> - Easy to detect - kernel can complain and refuse to load if it happens
> - Will never happen when overlay is compiled against the base used
> to boot (dtc can resolve the conflict)
> - Hash phandle generation makes this exceptionally rare
> 2) two overlays try to define same phandle
> - Also easy to detect on loading the second overlay - kernel will
> - Hash phandle generation still makes this exceptionally rare
>
> In both cases a collision is extremely rare and if it does happen it
> will not fail silently. Besides, in the odd scenario where it does
> happen, a node can be manually assigned a phandle. It is far better to
> do the manual override maybe once or twice (actually, I'd be surprised
> if it is ever needed) than to get into any kind of numberspace
> management for phandles. That's just a maintenance nightmare.
>

The reason people (or at least me) is wary of collisions is that it
throws the user completely out of the normal workflow.

i.e. normal workflow is like this:

a) Edit cape DTS
b) Feed the DTB to the running kernel
c) It works.

When a collision happens c turns into

c.1) Fails with a message that 'there's a phandle collision'
c.2) Google about the error message, land on a page describing solution.
c.3) Modify the cape DTS to use an explicit phandle value.
c.4) It finally works.

Let's just say I don't expect users to deal with this easily.

Anyway, it's all academic at this point. I mostly care about making
the in-kernel dtc compile to handle the collisions automatically.

I think we agree that compiling a fragment that doesn't export any
phandles against the base dts, should produce a dtb that would load
without any collisions.

> The hard bit to solve has always been when the overlay expects
> different phandles than the base is using, and that problem only
> occurs if the overlay is compiled against a different base dtb than
> was used to boot the system. Hashed phandle generation even makes
> phandles very stable when the base dtb is recompiled, and if they do
> change, then the kernel can detect it and report an error. Again, no
> silent failures. Phandle fixup does make the problem disappears
> entirely but I'm concerned that it is still incomplete (see below) and
> would be conceptually expensive.
>
> Once of the requests is to support one overlay .dtb with many
> different baseboards, but now that I've thought about it more I
> realize that it just won't work for anything beyond the most trivial
> case. Phandle fixup isn's sufficient. The format of GPIO, Interrupt,
> clock, etc specifiers may change if the base board nodes change. The
> specifiers are not portable.
>

My intention wasn't never to make overlays overly portable. My intention
was to make them in a way that portability can be introduced if the boards
are 'close' enough, but not for arbitrary boards.

There have to be compatible interfaces both on the base, and the overlay
dtbs.

> So, I no longer think that plain dtb overlays alone will work against
> any base board. Something more is needed. It may be the mechanism for
> loading new data into the kernel, but something really generic does
> require either being compiled at runtime (as David suggests) or each
> expansion interface (ie. the BeagleBone expansion or the RPi
> expansion) to really specify what kinds of references are allowed and
> run all specifiers through translation nodes. Similar to how
> interrupt-map works.
>

All things point to the latter.

>> i2c2: i2c@4819c000 {
>> compatible = "ti,omap4-i2c";
>> #address-cells = <1>;
>> #size-cells = <0>;
>> ti,hwmods = "i2c3";
>> reg = <0x4819c000 0x1000>;
>> interrupt-parent = <&intc>;
>> interrupts = <30>;
>> status = "disabled";
>> };
>>
>> And in the cape definition (when compiled with the special mode I describe
>> below)
>>
>> / {
>> plugin-bundle;
>> compatible = "cco,weather-cape";
>> version = <00A0>;
>>
>> i2c2-graft = {
>> compatible = <dt,graft>;
>> graft-point = <&i2c2>;
>
> Since overlays are different, we can interpret them slightly
> differently. The node name itself could be the graft point, and the
> name could be either a full path "/foo/bar/i2c@10002000" or an alias
> reference "i2c2". I think that is a bit better than a new graft-point
> property.
>

Can you expand a bit on the exact syntax?

> g.

Regards

-- Pantelis

2012-11-13 16:57:28

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/13/2012 12:25 AM, David Gibson wrote:
> On Mon, Nov 12, 2012 at 09:52:32AM -0700, Stephen Warren wrote:
>> On 11/12/2012 05:10 AM, Pantelis Antoniou wrote:
> [snip]
>>> Oh yes. In fact if one was to use a single kernel image for beagleboard
>>> and beaglebone, for the cape to work for both, it is required for it's
>>> dtb to be compatible.
>>
>> Well, as Grant pointed out, it's not actually strictly necessary for the
>> .dtb to be compatible; only the .dts /need/ be compatible, and the .dtb
>> can be generated at run-time using dtc for example.
>
> So, actually, I think a whole bunch of problems with phandle
> resolution disappear if we don't try to define an overlay .dtb format,
> or at least treat it only as a very shortlived object. A more precise
> proposal below. Note that this works more or less equally well with
> either the original overlay approach or the graft/graft-bundle
> proposal I made elsewhere.
>
> 1) We annotate the base tree with some extra label information for
> nodes which overlays are likely to want to reference by phandle. e.g.
>
> beaglebone_pic: interrupt-controller@XXXXX {
> ...
> phandle,symbolic-name = "beaglebone_pic";
> };
>
> We could extend dtc to (optionally?) auto-generate those properties
> from its existing label syntax. Not sure if that's a good idea or
> not yet. In any case, we compile this augmented base tree to .dtb as
> normal and boot our kernel with it.

Yes, I think a name-based approach is preferable over using
opaque/arbitrary phandle IDs/ranges/...

> 2) The information for the capes/modules/whatever is
> distributed/packaged as .dts, never .dtb. When userspace detects the
> new module (or the user explicitly tells it, if it's not probeable) it
> picks up the correct dts and runs it through dtc in a special mode.
> In this mode dtc takes the existing base tree (from /proc/device-tree,
> say) as well as the new dts. In this mode, dtc allocates phandles for
> the new tree fragment so as not to collide with anything from the
> supplied base tree (as well as avoiding internal conflicts,
> obviously). It also allows node references to the base tree by using
> those label annotations from (1) to match symbolic names to the
> phandle values in the base tree.
>
> 3) The resulting partial .dtb for the module is highly specific to the
> base tree (which if the base tree was generated at runtime by firmware
> could even be specific to a particular boot). But that's ok, because
> we just spit it into the kernel, absolute phandle values and all, then
> throw it away. Next time we need the module info, we recompile it
> again.

Once you've booted with a base tree, and loaded a partial .dtb for one
child board, and are then loading a .dtb for another child board (or you
unloaded the original child board and are loading a replacement), then
presumably the current in-kernel device tree also depends on all the
runtime history too.

So then going back to your point (2), that means we need to have
user-space serialize the dtc execution so that we don't compile two new
partial .dtbs in parallel, and end up with each not conflicting with the
current in-kernel device tree, but still conflicting with each-other. I
imagine that's easily solvable though.

2012-11-13 17:10:20

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/13/2012 01:09 AM, Pantelis Antoniou wrote:
> On Nov 13, 2012, at 9:25 AM, David Gibson wrote:
...
>> 1) We annotate the base tree with some extra label information for
>> nodes which overlays are likely to want to reference by phandle. e.g.
>>
>> beaglebone_pic: interrupt-controller@XXXXX {
>> ...
>> phandle,symbolic-name = "beaglebone_pic";
>> };
>>
>> We could extend dtc to (optionally?) auto-generate those properties
>> from its existing label syntax. Not sure if that's a good idea or
>> not yet. In any case, we compile this augmented base tree to .dtb as
>> normal and boot our kernel with it.
>
> I'm fine with that. You can auto-generate when there's a label to a node.
> The cape dt fragment will use the label name to reference a node.
> More details below...
>
>> 2) The information for the capes/modules/whatever is
>> distributed/packaged as .dts, never .dtb. When userspace detects the
>> new module (or the user explicitly tells it, if it's not probeable) it
>> picks up the correct dts and runs it through dtc in a special mode.
>> In this mode dtc takes the existing base tree (from /proc/device-tree,
>> say) as well as the new dts. In this mode, dtc allocates phandles for
>> the new tree fragment so as not to collide with anything from the
>> supplied base tree (as well as avoiding internal conflicts,
>> obviously). It also allows node references to the base tree by using
>> those label annotations from (1) to match symbolic names to the
>> phandle values in the base tree.
>
> Not good to rely on userspace kicking off dtc and compiling from source.
> Some capes/expansion boards might have your root fs device, for example
> there is an eMMC cape coming up, while networking capes are common too.
>
> However I have a compromise.
>
> I agree that compiling from source can be an option for a runtime definable
> cape, but for built-in capes we could do a bit better.
>
> In particular, I don't see any particular need to have a DT fragment
> reference anything that dependent of the runtime device tree. It should
> be possible to compile the DT fragment in kernel, against the generated
> flattened device tree, producing a flattened DT fragment with the phandles
> already resolved.
>
> So the sequence could be something like this:
>
> $ dtc -O dtb -o am335x-bone.dtb -b 0 am335x-bone.dts -@ ${LAST_PHANDLE_FILE}
> $ dtc -O dtbf -R am335x-bone.dtb -o weather-cape.dtb -b 0 weather-cape.dts -@ ${LAST_PHANDLE_FILE}
> $ dtc -O dtbf -R am335x-bone.dtb -o geiger-cape.dtb -b 0 geiger-cape.dts -@ ${LAST_PHANDLE_FILE}
>
> The ${LAST_PHANDLE_FILE} can be updated with the last phandle value generated.

I'm not sure that will avoid phandle collisions though.

If you build everything at runtime, you'll always be compiling each new
partial .dtb based on the phandles actually in-use in the kernel's
current complete device tree. You can always easily avoid collisions
that way.

If you instead pre-compile both the based .dtb /and/ some partial .dtbs,
then you're essentially statically defining some phandle values, but
splitting those static assignments across 3 (in your example) different
.dtbs. However, what guarantees that those 3 .dtbs are already loaded
into the system when some fourth partial .dtb is loaded, such that the
static phandle assignments influence that fourth partial .dtb's runtime
compilation?

Put another way, what happens when you:

1) Boot the kernel using am335x-bone.dtb.
2) Incrementally load weather-cape.dtb. Don't incrementally load
geiger-cape.dtb for whatever reason.
3) Load runtime-compiled-cape.dtb. This avoids conflicts with the
already-loaded am335x-bone.dtb and weather-cape.dtb since
runtime-compiled-cape.dtb is compiled at runtime based on the currently
loaded .dtbs. However, it may well end up conflicting with
geiger-cape.dtb since that isn't loaded.
4) Now attempt to load geiger-cape.dtb - possible conflict.

Given all of that, I'd assert that if you want to statically assign any
phandles, all static assignments need to be done in the base .dtb that
the kernel is loaded with, and not in any pre-compiled partial .dtbs.

So, if your root filesystem is on an eMMC cape, then you need to create
a board .dts file that /include/s the base board .dts and the cape .dts
files and boot the kernel with that.

Or, you get U-Boot/... to merge the relevant pre-compiled .dtbs and boot
the kernel with that.

Or I suppose you could have a well-known range of phandles for
pre-compiled .dtbs that the runtime dtc mode always avoided, preferably
without the need for explicit phandle-range statements in the .dts
files, e.g. driven by a dtc static-assignment-vs-dynamic-assignment
command-line option?

2012-11-13 18:10:32

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

It seems to me that this capebus discussion is missing an important
point. The name capebus suggests that it is a bus, so there should be a
parent node to represent that bus. It should have a driver whose API
implements all of the system-interface functions a cape needs.

If you look at the way that interrupt specifiers work, the default case
is that a child device implicitly delegates the mapping to its parent.
The use of phandles to break out of the tree structure was intended for
use within the "hardwired motherboard domain", not for plug-in devices.

The "new" phandle-based GPIO and clock mechanisms don't have that
parent-delegation feature, but they should, because hierarchical
hardware is a good thing when it exists. One fix would be to designate
a reserved phandle value - for example 0 or -1 - to mean "my parent". t
The parent node would contain some translator to resolve the actual
target node, similarly to interrupts and addresses.

If done correctly, capebus "overlays" would then just be proper child
nodes of the capebus bus node and there would no need to refer to
"global" information like non-parent phandles.

If something about the design of capebus makes that impossible, I
respectfully suggest that its design should be reviewed, taking into
account the many years of industry experience about modularity.

Mitch

2012-11-13 18:35:16

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/13/2012 11:10 AM, Mitch Bradley wrote:
> It seems to me that this capebus discussion is missing an important
> point. The name capebus suggests that it is a bus, so there should be a
> parent node to represent that bus. It should have a driver whose API
> implements all of the system-interface functions a cape needs.

It was discussed earlier that capebus isn't actually a bus. It's simply
a collection of a bunch of pins from the SoC hooked up to connectors.
I'd agree that it's mis-named.

2012-11-13 19:09:27

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 11/13/2012 8:29 AM, Stephen Warren wrote:
> On 11/13/2012 11:10 AM, Mitch Bradley wrote:
>> It seems to me that this capebus discussion is missing an important
>> point. The name capebus suggests that it is a bus, so there should be a
>> parent node to represent that bus. It should have a driver whose API
>> implements all of the system-interface functions a cape needs.
>
> It was discussed earlier that capebus isn't actually a bus. It's simply
> a collection of a bunch of pins from the SoC hooked up to connectors.
> I'd agree that it's mis-named.
>

Nevertheless, to the extent that the set of pins is finite and
well-defined, it should be possible to define a set of software
interfaces to support the functionality represented by those pins.

It might depend on the underlying SoC, but even so, it would still be
best to encapsulate the interface set. I hear all these use cases that
presuppose a wide variety of user skill sets. If one really wants to
support such users well, it's important to define a coherent single
point of interface.

2012-11-13 19:11:46

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi Mitch,

On Nov 13, 2012, at 9:09 PM, Mitch Bradley wrote:

> On 11/13/2012 8:29 AM, Stephen Warren wrote:
>> On 11/13/2012 11:10 AM, Mitch Bradley wrote:
>>> It seems to me that this capebus discussion is missing an important
>>> point. The name capebus suggests that it is a bus, so there should be a
>>> parent node to represent that bus. It should have a driver whose API
>>> implements all of the system-interface functions a cape needs.
>>
>> It was discussed earlier that capebus isn't actually a bus. It's simply
>> a collection of a bunch of pins from the SoC hooked up to connectors.
>> I'd agree that it's mis-named.
>>
>
> Nevertheless, to the extent that the set of pins is finite and
> well-defined, it should be possible to define a set of software
> interfaces to support the functionality represented by those pins.
>
> It might depend on the underlying SoC, but even so, it would still be
> best to encapsulate the interface set. I hear all these use cases that
> presuppose a wide variety of user skill sets. If one really wants to
> support such users well, it's important to define a coherent single
> point of interface.
>
>

That's what capebus is. Too bad there's such a fuss about the name.
Check out the thread from the start for the sordid details.

Regards

-- Pantelis

2012-11-13 23:29:18

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Tue, Nov 13, 2012 at 10:09:28AM +0200, Pantelis Antoniou wrote:
> Hi David,
>
> On Nov 13, 2012, at 9:25 AM, David Gibson wrote:
>
> > On Mon, Nov 12, 2012 at 09:52:32AM -0700, Stephen Warren wrote:
> >> On 11/12/2012 05:10 AM, Pantelis Antoniou wrote:
> > [snip]
> >>> Oh yes. In fact if one was to use a single kernel image for beagleboard
> >>> and beaglebone, for the cape to work for both, it is required for it's
> >>> dtb to be compatible.
> >>
> >> Well, as Grant pointed out, it's not actually strictly necessary for the
> >> .dtb to be compatible; only the .dts /need/ be compatible, and the .dtb
> >> can be generated at run-time using dtc for example.
> >
> > So, actually, I think a whole bunch of problems with phandle
> > resolution disappear if we don't try to define an overlay .dtb format,
> > or at least treat it only as a very shortlived object. A more precise
> > proposal below. Note that this works more or less equally well with
> > either the original overlay approach or the graft/graft-bundle
> > proposal I made elsewhere.
> >
> > 1) We annotate the base tree with some extra label information for
> > nodes which overlays are likely to want to reference by phandle. e.g.
> >
> > beaglebone_pic: interrupt-controller@XXXXX {
> > ...
> > phandle,symbolic-name = "beaglebone_pic";
> > };
> >
> > We could extend dtc to (optionally?) auto-generate those properties
> > from its existing label syntax. Not sure if that's a good idea or
> > not yet. In any case, we compile this augmented base tree to .dtb as
> > normal and boot our kernel with it.
> >
>
> I'm fine with that. You can auto-generate when there's a label to a node.
> The cape dt fragment will use the label name to reference a node.
> More details below...
>
> > 2) The information for the capes/modules/whatever is
> > distributed/packaged as .dts, never .dtb. When userspace detects the
> > new module (or the user explicitly tells it, if it's not probeable) it
> > picks up the correct dts and runs it through dtc in a special mode.
> > In this mode dtc takes the existing base tree (from /proc/device-tree,
> > say) as well as the new dts. In this mode, dtc allocates phandles for
> > the new tree fragment so as not to collide with anything from the
> > supplied base tree (as well as avoiding internal conflicts,
> > obviously). It also allows node references to the base tree by using
> > those label annotations from (1) to match symbolic names to the
> > phandle values in the base tree.
> >
>
> Not good to rely on userspace kicking off dtc and compiling from source.
> Some capes/expansion boards might have your root fs device, for example
> there is an eMMC cape coming up, while networking capes are common too.

So? dtc can go in an initramfs, just like mdadm or whatever other
tools are there.

> However I have a compromise.
>
> I agree that compiling from source can be an option for a runtime definable
> cape, but for built-in capes we could do a bit better.
>
> In particular, I don't see any particular need to have a DT fragment
> reference anything that dependent of the runtime device tree. It should
> be possible to compile the DT fragment in kernel, against the generated
> flattened device tree, producing a flattened DT fragment with the phandles
> already resolved.

Um..? Sorry, I can't parse that paragraph.

> So the sequence could be something like this:
>
> $ dtc -O dtb -o am335x-bone.dtb -b 0 am335x-bone.dts -@ ${LAST_PHANDLE_FILE}
> $ dtc -O dtbf -R am335x-bone.dtb -o weather-cape.dtb -b 0 weather-cape.dts -@ ${LAST_PHANDLE_FILE}
> $ dtc -O dtbf -R am335x-bone.dtb -o geiger-cape.dtb -b 0 geiger-cape.dts -@ ${LAST_PHANDLE_FILE}
>
> The ${LAST_PHANDLE_FILE} can be updated with the last phandle value generated.
>
> Alternatively we could have a way to statically assign a phandle range
> for well known capes. All the others will have to use the runtime compile
> mechanism.
> $ dtc -O dtb -o am335x-bone.dtb -b 0 am335x-bone.dts
> $ dtc -O dtbf -R am335x-bone.dtb -o weather-cape.dtb -b 0 weather-cape.dts
> $ dtc -O dtbf -R am335x-bone.dtb -o geiger-cape.dtb -b 0 geiger-cape.dts
>
> With the cape dtses having a /phandle-range/ statement at the top.
>
> This can work because the cape dts do not cross-reference each other, and
> neither the boot dts references the capes.
>
> That way we can use request_firmware() pretty early in the boot sequence
> and get the DT fragment we need even before user-space starts and root fs
> has mounted. request_firmware() can locate the fragments in the kernel
> image before rootfs.
>
> I don't know if this will cover all the cases Grant has in mind though.
>
> So just to make sure I got it right, this could work for our case.
>
> i2c2: i2c@4819c000 {
> compatible = "ti,omap4-i2c";
> #address-cells = <1>;
> #size-cells = <0>;
> ti,hwmods = "i2c3";
> reg = <0x4819c000 0x1000>;
> interrupt-parent = <&intc>;
> interrupts = <30>;
> status = "disabled";
> };
>
> And in the cape definition (when compiled with the special mode I describe
> below)
>
> / {
> plugin-bundle;
> compatible = "cco,weather-cape";
> version = <00A0>;
>
> i2c2-graft = {
> compatible = <dt,graft>;
> graft-point = <&i2c2>;
>
> #address-cells = <1>;
> #size-cells = <0>;
>
> /* Ambient light sensor */
> tsl2550@39 {
> compatible = "tsl,tsl2550";
> reg = <0x39>;
> };
> };
> };
>
> DTC when compiling in the special fragment mode will pick up that
> &i2c2 can not be resolved and lookup the phandle on the main dtb.
> That way, even 'phandle,symbolic-name = "i2c2";' is redundant.

Well, no, because I'm assuming dtc fragment mode only has access to
the base dtb, not the base dts, so labels will be gone from it, unless
we add properties to preserve them especially. That's what the
symbolic-name thing is about.

> > 3) The resulting partial .dtb for the module is highly specific to the
> > base tree (which if the base tree was generated at runtime by firmware
> > could even be specific to a particular boot). But that's ok, because
> > we just spit it into the kernel, absolute phandle values and all, then
> > throw it away. Next time we need the module info, we recompile it
> > again.
> >
> >> Of course, relying on .dts compatibility rather than .dtb compatibility
> >> might negatively impact the complexity of an initrd environment if we
> >> end up loading overlays from there...
> >
> > Well, it does mean we'd need dtc in the initrd. But dtc has no
> > library dependencies except libc, so that really shouldn't be too
> > bad. In return we entirely avoid inventing a new phandle resolution
> > protocol.
>
> Not every board boots with initrd; most embedded boards don't use it
> at all. This way we make initrd a hard requirement.

Well, not really. You can use initramfs, or you can assemble all the
necessary dt fragents for boot into a complete dtb included with the
kernel. Doing that involves pretty much the same sorts of things you
need to do to statically configure a kernel for boot without
initramfs.

--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson

2012-11-14 00:00:57

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

Hi David,

Since this is getting to be too long-winded I went ahead a coded
something quick and dirty.

On Nov 14, 2012, at 1:30 AM, David Gibson wrote:

> On Tue, Nov 13, 2012 at 10:09:28AM +0200, Pantelis Antoniou wrote:
>> Hi David,
>>
>> On Nov 13, 2012, at 9:25 AM, David Gibson wrote:
>>
>>> On Mon, Nov 12, 2012 at 09:52:32AM -0700, Stephen Warren wrote:
>>>> On 11/12/2012 05:10 AM, Pantelis Antoniou wrote:
>>> [snip]
>>>>> Oh yes. In fact if one was to use a single kernel image for beagleboard
>>>>> and beaglebone, for the cape to work for both, it is required for it's
>>>>> dtb to be compatible.
>>>>
>>>> Well, as Grant pointed out, it's not actually strictly necessary for the
>>>> .dtb to be compatible; only the .dts /need/ be compatible, and the .dtb
>>>> can be generated at run-time using dtc for example.
>>>
>>> So, actually, I think a whole bunch of problems with phandle
>>> resolution disappear if we don't try to define an overlay .dtb format,
>>> or at least treat it only as a very shortlived object. A more precise
>>> proposal below. Note that this works more or less equally well with
>>> either the original overlay approach or the graft/graft-bundle
>>> proposal I made elsewhere.
>>>
>>> 1) We annotate the base tree with some extra label information for
>>> nodes which overlays are likely to want to reference by phandle. e.g.
>>>
>>> beaglebone_pic: interrupt-controller@XXXXX {
>>> ...
>>> phandle,symbolic-name = "beaglebone_pic";
>>> };
>>>
>>> We could extend dtc to (optionally?) auto-generate those properties
>>> from its existing label syntax. Not sure if that's a good idea or
>>> not yet. In any case, we compile this augmented base tree to .dtb as
>>> normal and boot our kernel with it.
>>>
>>
>> I'm fine with that. You can auto-generate when there's a label to a node.
>> The cape dt fragment will use the label name to reference a node.
>> More details below...
>>
>>> 2) The information for the capes/modules/whatever is
>>> distributed/packaged as .dts, never .dtb. When userspace detects the
>>> new module (or the user explicitly tells it, if it's not probeable) it
>>> picks up the correct dts and runs it through dtc in a special mode.
>>> In this mode dtc takes the existing base tree (from /proc/device-tree,
>>> say) as well as the new dts. In this mode, dtc allocates phandles for
>>> the new tree fragment so as not to collide with anything from the
>>> supplied base tree (as well as avoiding internal conflicts,
>>> obviously). It also allows node references to the base tree by using
>>> those label annotations from (1) to match symbolic names to the
>>> phandle values in the base tree.
>>>
>>
>> Not good to rely on userspace kicking off dtc and compiling from source.
>> Some capes/expansion boards might have your root fs device, for example
>> there is an eMMC cape coming up, while networking capes are common too.
>
> So? dtc can go in an initramfs, just like mdadm or whatever other
> tools are there.
>

Embedded systems aren't servers. Having an initramfs is one more thing that
can break, or require people to modify their build systems.

>> However I have a compromise.
>>
>> I agree that compiling from source can be an option for a runtime definable
>> cape, but for built-in capes we could do a bit better.
>>
>> In particular, I don't see any particular need to have a DT fragment
>> reference anything that dependent of the runtime device tree. It should
>> be possible to compile the DT fragment in kernel, against the generated
>> flattened device tree, producing a flattened DT fragment with the phandles
>> already resolved.
>
> Um..? Sorry, I can't parse that paragraph.
>

No need for runtime resolution for common use cases.

>> So the sequence could be something like this:
>>
>> $ dtc -O dtb -o am335x-bone.dtb -b 0 am335x-bone.dts -@ ${LAST_PHANDLE_FILE}
>> $ dtc -O dtbf -R am335x-bone.dtb -o weather-cape.dtb -b 0 weather-cape.dts -@ ${LAST_PHANDLE_FILE}
>> $ dtc -O dtbf -R am335x-bone.dtb -o geiger-cape.dtb -b 0 geiger-cape.dts -@ ${LAST_PHANDLE_FILE}
>>
>> The ${LAST_PHANDLE_FILE} can be updated with the last phandle value generated.
>>
>> Alternatively we could have a way to statically assign a phandle range
>> for well known capes. All the others will have to use the runtime compile
>> mechanism.
>> $ dtc -O dtb -o am335x-bone.dtb -b 0 am335x-bone.dts
>> $ dtc -O dtbf -R am335x-bone.dtb -o weather-cape.dtb -b 0 weather-cape.dts
>> $ dtc -O dtbf -R am335x-bone.dtb -o geiger-cape.dtb -b 0 geiger-cape.dts
>>
>> With the cape dtses having a /phandle-range/ statement at the top.
>>
>> This can work because the cape dts do not cross-reference each other, and
>> neither the boot dts references the capes.
>>
>> That way we can use request_firmware() pretty early in the boot sequence
>> and get the DT fragment we need even before user-space starts and root fs
>> has mounted. request_firmware() can locate the fragments in the kernel
>> image before rootfs.
>>
>> I don't know if this will cover all the cases Grant has in mind though.
>>
>> So just to make sure I got it right, this could work for our case.
>>
>> i2c2: i2c@4819c000 {
>> compatible = "ti,omap4-i2c";
>> #address-cells = <1>;
>> #size-cells = <0>;
>> ti,hwmods = "i2c3";
>> reg = <0x4819c000 0x1000>;
>> interrupt-parent = <&intc>;
>> interrupts = <30>;
>> status = "disabled";
>> };
>>
>> And in the cape definition (when compiled with the special mode I describe
>> below)
>>
>> / {
>> plugin-bundle;
>> compatible = "cco,weather-cape";
>> version = <00A0>;
>>
>> i2c2-graft = {
>> compatible = <dt,graft>;
>> graft-point = <&i2c2>;
>>
>> #address-cells = <1>;
>> #size-cells = <0>;
>>
>> /* Ambient light sensor */
>> tsl2550@39 {
>> compatible = "tsl,tsl2550";
>> reg = <0x39>;
>> };
>> };
>> };
>>
>> DTC when compiling in the special fragment mode will pick up that
>> &i2c2 can not be resolved and lookup the phandle on the main dtb.
>> That way, even 'phandle,symbolic-name = "i2c2";' is redundant.
>
> Well, no, because I'm assuming dtc fragment mode only has access to
> the base dtb, not the base dts, so labels will be gone from it, unless
> we add properties to preserve them especially. That's what the
> symbolic-name thing is about.
>

They can be easily included, as the patch shows.

>>> 3) The resulting partial .dtb for the module is highly specific to the
>>> base tree (which if the base tree was generated at runtime by firmware
>>> could even be specific to a particular boot). But that's ok, because
>>> we just spit it into the kernel, absolute phandle values and all, then
>>> throw it away. Next time we need the module info, we recompile it
>>> again.
>>>
>>>> Of course, relying on .dts compatibility rather than .dtb compatibility
>>>> might negatively impact the complexity of an initrd environment if we
>>>> end up loading overlays from there...
>>>
>>> Well, it does mean we'd need dtc in the initrd. But dtc has no
>>> library dependencies except libc, so that really shouldn't be too
>>> bad. In return we entirely avoid inventing a new phandle resolution
>>> protocol.
>>
>> Not every board boots with initrd; most embedded boards don't use it
>> at all. This way we make initrd a hard requirement.
>
> Well, not really. You can use initramfs, or you can assemble all the
> necessary dt fragents for boot into a complete dtb included with the
> kernel. Doing that involves pretty much the same sorts of things you
> need to do to statically configure a kernel for boot without
> initramfs.
>

You can't assemble the dtb without runtime probing of what's out there.

> --
> David Gibson | I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
> | _way_ _around_!
> http://www.ozlabs.org/~dgibson

Anyway, here's a small patch that shows that it is possible to do both fixups
and symbol tracking using relatively unmodified DT syntax. The only thing
new is the /plugin/; statement.

Also included two dumps of generated dtbs as well as the plugin dts.

Should be relatively simple to come up with a kernel loader and linker using
something similar like this.

Regards

-- Pantelis

Attachments:

dtc-symbols-fixups-test.patch (8.55 kB)
geiger-cape.fdtdump (1.87 kB)
geiger-cape.dts (1.31 kB)
am335x-bone.fdtdump (44.62 kB)
Download all attachments

2012-11-15 05:26:26

by Jean-Christophe PLAGNIOL-VILLARD

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On Tue, Nov 13, 2012 at 03:38:18PM +0200, Pantelis Antoniou wrote:
> Hi Grant,
>
> On Nov 13, 2012, at 2:24 PM, Grant Likely wrote:

> > On Tue, Nov 13, 2012 at 8:09 AM, Pantelis Antoniou
[snip]
> My intention wasn't never to make overlays overly portable. My intention
> was to make them in a way that portability can be introduced if the boards
> are 'close' enough, but not for arbitrary boards.
>
> There have to be compatible interfaces both on the base, and the overlay
> dtbs.

Right. And this is why I'm arguing that those interfaces should be
described explicitly - using existing OF mechanisms like interrupt-map
where possible, rather than having a very general, but very low-level
interface to make arbitrary changes to the DT.

--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson

2012-11-17 22:36:58

[permalink] [raw]

Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)

On 16:23 Fri 09 Nov , Stephen Warren wrote:
> On 11/09/2012 09:28 AM, Grant Likely wrote:
> > On Tue, Nov 6, 2012 at 10:37 PM, Stephen Warren <[email protected]> wrote:
> ...
> >> I do rather suspect this use-case is quite common. NVIDIA certainly has
> >> a bunch of development boards with pluggable
> >> PMIC/audio/WiFi/display/..., and I believe there's some ability to
> >> re-use the pluggable components with a variety of base-boards.
> >>
> >> Given people within NVIDIA started talking about this recently, I asked
> >> them to enumerate all the boards we have that support pluggable
> >> components, and how common it is that some boards support being plugged
> >> into different main boards. I don't know when that enumeration will
> >> complete (or even start) but hopefully I can provide some feedback on
> >> how common the use-case is for us once it's done.
> >
> > From your perspective, is it important to use the exact same .dtb
> > overlays for those add-on boards, or is it okay to have a separate
> > build of the overlay for each base tree?
>
> I certainly think it'd be extremely beneficial to use the exact same
> child board .dtb with arbitrary base boards.
>
> Consider something like the Arduino shield connector format, which I
> /believe/ has been re-used across a wide variety of Arduino boards and
> other compatible or imitation boards. Now consider a vendor of an
> Arduino shield. The shield vendor probably wants to publish a single
> .dtb file that works for users irrespective of which board they're using
> it with.
>
> (Well, I'm not sure that Arduino can run Linux; perhaps that's why you
> picked BeagleBone capes for your document!)
>
> I suppose it would be acceptable for the shield vendor to ship the .dts
> file rather than the .dtb, and hence need to build the shield .dtb for a
> specific base board.
>
> However, I think the process for an end-user needs to be as simple as
> "drop this .dts/.dtb file into some standard directory", and I imagine
> it'll be much easier for distros/... to make that process work if
> they're dealing with a .dtb that they can just blast into the kernel's
> firmware loader interface, rather than having to also locate the
> base-board .dts/.dtb file, and run dtc and/or other tools on both .dts
> files together.
I've exactly the same issue on Calao or the new atmel boards

We have lego boards

with different cpu-modues running on differetn mother boards with
diferrentdaugther boards

on atmel we are lucky enough we can identity via 1-wire all of them but
on Calao no

On Somfy platform we can detect hardware version and need different pinctrl

So personally I'll prefer to be able to request dtb from the kernel or push
them from the userspace as it will depends where you will detect the hardware
present

The main concern will which part of the kenel will now handle hw detection?

Best Regards,
J.

2012-11-20 17:09:48