Hi,
First off apologies for the large CC list -- I think this catches the
arch list for all the arches with device tree source in the tree.
Various folks have expressed an interest in eventually splitting the
device tree bindings out of the Linux git repository into a separate
tree. This should help reduce the cross talk between the code and the
bindings and to make it difficult to accidentally "co-evolve" the
bindings and the code (i.e. break compatibility) etc. There are also
other projects (such as Xen) which would also like to use device-tree as
an OS agnostic description of the hardware.
I was talking to Grant about this at this Spring's LinaroConnect in Hong
Kong and as a first step he was interested in a device-tree.git
repository which is automatically kept insync with the main Linux tree.
Somehow I found myself volunteering to set up that tree.
An RFC repository can be found at:
http://xenbits.xen.org/gitweb/?p=people/ianc/device-tree-rebasing.git
This is created using git filter-branch and retains the full history for
the device tree source files up to v3.9-rc8.
The master branch contains everything including the required build
infrastructure while upstream/master and upstream/dts contain the most
recently converted upstream master branch and the pristine converted
version respectively. Each upstream tag T is paired with a tag T-dts
which is the converted version of that tag.
Note that the tree will be potentially rebasing (hence the name) for the
time being while I'm still smoothing out the conversion process.
The paths to include in the conversion are described in
scripts/rewrite-paths.sed. The generic cases are:
arch/ARCH/boot/dts/*.dts and *.dts? (for dtsi and dtsp etc)
arch/ARCH/boot/*.dts and *.dts?
arch/ARCH/include/dts/* (currently unused?)
which become src/ARCH/*.dts and *.dts? plus src/ARCH/include/*
There are also some special cases for some arches which don't follow
this pattern and for older versions of the kernel which were less
consistent. The paths were gleaned from git ls-tree + grep on every tag
in the tree, so if a file was added and moved between two rcs then the
original path may not be covered (so the move will look like it just
adds the files).
In principal this supports the new .dtsp files and includes the required
include paths in the conversion but none of them seem to be in mainline
yet, so we'll have to see!
The initial conversion took in excess of 40 hours (running out of a
ramdisk) so even if the result is stable in terms of commit ids etc a
fresh conversion every time isn't an option for a ~daily sync so I had
to create a slightly hacked around git-filter-branch (found in
scripts/git-filter-branch) to support incremental filtering, which I
intend to send to the git folks soon.
Please let me know what you think.
Ian.
[0] real 2533m32.142s
user 2393m35.039s
sys 343m44.385s
Hi Ian,
On 04/24/2013 12:48 PM, Ian Campbell wrote:
> Hi,
>
> First off apologies for the large CC list -- I think this catches the
> arch list for all the arches with device tree source in the tree.
>
> Various folks have expressed an interest in eventually splitting the
> device tree bindings out of the Linux git repository into a separate
> tree. This should help reduce the cross talk between the code and the
> bindings and to make it difficult to accidentally "co-evolve" the
> bindings and the code (i.e. break compatibility) etc. There are also
> other projects (such as Xen) which would also like to use device-tree as
> an OS agnostic description of the hardware.
>
> I was talking to Grant about this at this Spring's LinaroConnect in Hong
> Kong and as a first step he was interested in a device-tree.git
> repository which is automatically kept insync with the main Linux tree.
> Somehow I found myself volunteering to set up that tree.
>
> An RFC repository can be found at:
> http://xenbits.xen.org/gitweb/?p=people/ianc/device-tree-rebasing.git
>
> This is created using git filter-branch and retains the full history for
> the device tree source files up to v3.9-rc8.
>
> The master branch contains everything including the required build
> infrastructure while upstream/master and upstream/dts contain the most
> recently converted upstream master branch and the pristine converted
> version respectively. Each upstream tag T is paired with a tag T-dts
> which is the converted version of that tag.
>
> Note that the tree will be potentially rebasing (hence the name) for the
> time being while I'm still smoothing out the conversion process.
>
> The paths to include in the conversion are described in
> scripts/rewrite-paths.sed. The generic cases are:
> arch/ARCH/boot/dts/*.dts and *.dts? (for dtsi and dtsp etc)
> arch/ARCH/boot/*.dts and *.dts?
> arch/ARCH/include/dts/* (currently unused?)
> which become src/ARCH/*.dts and *.dts? plus src/ARCH/include/*
>
> There are also some special cases for some arches which don't follow
> this pattern and for older versions of the kernel which were less
> consistent. The paths were gleaned from git ls-tree + grep on every tag
> in the tree, so if a file was added and moved between two rcs then the
> original path may not be covered (so the move will look like it just
> adds the files).
>
> In principal this supports the new .dtsp files and includes the required
> include paths in the conversion but none of them seem to be in mainline
> yet, so we'll have to see!
>
> The initial conversion took in excess of 40 hours (running out of a
> ramdisk) so even if the result is stable in terms of commit ids etc a
> fresh conversion every time isn't an option for a ~daily sync so I had
> to create a slightly hacked around git-filter-branch (found in
> scripts/git-filter-branch) to support incremental filtering, which I
> intend to send to the git folks soon.
>
> Please let me know what you think.
I have seen that discussion on youtube about this and connection
to arm dtses in the kernel.
Let me describe Microblaze case because probably Microblaze was the first
architecture in the Linux kernel which was DT only from the beginning.
Just small overview it is a Xilinx soft core cpu where you can even setup
some parameters for core itself - multiplier, divider, BS, fpu, cache sizes, etc.
You have to also compose the whole system and every platform/configuration is different
because you can setup addresses, IP on the bus, IRQs, etc.
Based on this configuration we have created tcl script which is able to generate
DTS directly from Xilinx design tool and it is working quite well for several years
and everybody just use it without any problem.
As you see in your repo there is only one microblaze DTS which is for one of mine
ancient configuration which none used.
It means from microblaze point of view we can simple remove it from mainline kernel
because it is useless.
I also care about arm zynq platform where situation is partially different because
zynq is fixed block but you can add others thing to programmable logic.
It means for zynq case we are almost in the same situation where every zynq based
platform is using different configuration and that's why fpgas are so great.
It means for zynq case everybody will need different DTS but will be just good
to describe or show binding.
Currently we have just one dts for zc702 xilinx reference board.
Let's move to my point.
Based on our experience all xilinx boards don't depend on any dts in the linux kernel
and our users just understand the reason why they should use our tcl script for
DTS generation.
Back to your point about moving DTSes out of the kernel. For microblaze - no problem
just do it. For arm zynq this is more problematic because there is weird binding
for ARM. For example PMU which is out of bus and should be probably in cpu node.
Also scu devices, scutimers, watchdog which lie on the bus for our case and we
need to use PPI interrupt cpu mask. Different clock binding, maybe pinmux binding, etc.
It means from my point of view if binding is correct, no problem to move it
out of the kernel. If a kernel patch change binding, it is worth for me to change
dts in the kernel too to reflect this change and track this change too.
My proposal is, let's clean all DTSes in the arm kernel that all platform use
the same binding where all platforms are just correctly described.
The reaching this point I would suggest that for arm, arm-soc maintainers should
keep eyes on any dts binding change and all these changes require ACK from Rob or Grant
(like device-tree maintainers).
Thanks,
Michal
--
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91
w: http://www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel - Microblaze cpu - http://www.monstr.eu/fdt/
Maintainer of Linux kernel - Xilinx Zynq ARM architecture
Microblaze U-BOOT custodian and responsible for u-boot arm zynq platform
On Mon, 2013-05-13 at 08:02 +0100, Michal Simek wrote:
> Just small overview it is a Xilinx soft core cpu where you can even setup
> some parameters for core itself - multiplier, divider, BS, fpu, cache sizes, etc.
> You have to also compose the whole system and every platform/configuration is different
> because you can setup addresses, IP on the bus, IRQs, etc.
> Based on this configuration we have created tcl script which is able to generate
> DTS directly from Xilinx design tool and it is working quite well for several years
> and everybody just use it without any problem.
That sounds very neat!
Does this tcl script live in the kernel tree? If so would you think it
would make sense for it to also migrate to device-tree.git? I'm not at
all sure if that makes sense but if you think it does please let me know
which paths need top be carried over.
> As you see in your repo there is only one microblaze DTS which is for one of mine
> ancient configuration which none used.
> It means from microblaze point of view we can simple remove it from mainline kernel
> because it is useless.
That will then naturally get propagated over to device-tree.git.
> I also care about arm zynq platform where situation is partially different because
> zynq is fixed block but you can add others thing to programmable logic.
> It means for zynq case we are almost in the same situation where every zynq based
> platform is using different configuration and that's why fpgas are so great.
>
> It means for zynq case everybody will need different DTS but will be just good
> to describe or show binding.
> Currently we have just one dts for zc702 xilinx reference board.
>
> Let's move to my point.
> Based on our experience all xilinx boards don't depend on any dts in the linux kernel
> and our users just understand the reason why they should use our tcl script for
> DTS generation.
>
> Back to your point about moving DTSes out of the kernel.
I suppose you are now commenting on the Phase II bit where maintenance
of the DTS moves out of linux.git into device-tree.git, rather than
Phase I work, which is creating a split repo which is automatically
synchronised from linux.git but maintenance remains in linux.git, i.e.
what I'm doing here.
> For microblaze - no problem
> just do it. For arm zynq this is more problematic because there is weird binding
> for ARM. For example PMU which is out of bus and should be probably in cpu node.
> Also scu devices, scutimers, watchdog which lie on the bus for our case and we
> need to use PPI interrupt cpu mask. Different clock binding, maybe pinmux binding, etc.
>
> It means from my point of view if binding is correct, no problem to move it
> out of the kernel. If a kernel patch change binding, it is worth for me to change
> dts in the kernel too to reflect this change and track this change too.
> My proposal is, let's clean all DTSes in the arm kernel that all platform use
> the same binding where all platforms are just correctly described.
AIUI this split/move isn't intended to change the existing policy, which
is already that DTS files are supposed to remain compatible across
kernel versions and that "flag days" are to be avoided. The split is
supposed to make it harder (if not impossible) to accidentally break
that policy.
On the other hand I suppose there is an argument to be made for clearing
up the cruft *before* making the split.
Ultimately I think this will be up to Grant & co.
> The reaching this point I would suggest that for arm, arm-soc maintainers should
> keep eyes on any dts binding change and all these changes require ACK from Rob or Grant
> (like device-tree maintainers).
Yes, once we move onto Phase II I don't expect it will end up being me
that is the DTS maintainer -- I expect the maintenance will remain with
those who take care of it in linux.git today.
My involvement in Phase I is really just to help out with the transition
(ulterior motive: the Xen project would also like to use these DTS
files...) not to perform a "land grab" or take over maintenance etc. I
certainly don't think I am the right person to become the long term
maintainer of device-tree.git!
Ian.
On 05/13/2013 01:59 PM, Ian Campbell wrote:
> On Mon, 2013-05-13 at 08:02 +0100, Michal Simek wrote:
>> Just small overview it is a Xilinx soft core cpu where you can even setup
>> some parameters for core itself - multiplier, divider, BS, fpu, cache sizes, etc.
>> You have to also compose the whole system and every platform/configuration is different
>> because you can setup addresses, IP on the bus, IRQs, etc.
>> Based on this configuration we have created tcl script which is able to generate
>> DTS directly from Xilinx design tool and it is working quite well for several years
>> and everybody just use it without any problem.
>
> That sounds very neat!
>
> Does this tcl script live in the kernel tree? If so would you think it
> would make sense for it to also migrate to device-tree.git? I'm not at
> all sure if that makes sense but if you think it does please let me know
> which paths need top be carried over.
No. This script is here.
https://github.com/Xilinx/device-tree/tree/master-next
It is tightly connected to Xilinx design tool that's why I don't think it is useful
to add it to any other tree.
>> As you see in your repo there is only one microblaze DTS which is for one of mine
>> ancient configuration which none used.
>> It means from microblaze point of view we can simple remove it from mainline kernel
>> because it is useless.
>
> That will then naturally get propagated over to device-tree.git.
I will think about it for a while and probably just remove it through my microblaze tree.
>> I also care about arm zynq platform where situation is partially different because
>> zynq is fixed block but you can add others thing to programmable logic.
>> It means for zynq case we are almost in the same situation where every zynq based
>> platform is using different configuration and that's why fpgas are so great.
>>
>> It means for zynq case everybody will need different DTS but will be just good
>> to describe or show binding.
>> Currently we have just one dts for zc702 xilinx reference board.
>>
>> Let's move to my point.
>> Based on our experience all xilinx boards don't depend on any dts in the linux kernel
>> and our users just understand the reason why they should use our tcl script for
>> DTS generation.
>>
>> Back to your point about moving DTSes out of the kernel.
>
> I suppose you are now commenting on the Phase II bit where maintenance
> of the DTS moves out of linux.git into device-tree.git, rather than
> Phase I work, which is creating a split repo which is automatically
> synchronised from linux.git but maintenance remains in linux.git, i.e.
> what I'm doing here.
ok.
>> For microblaze - no problem
>> just do it. For arm zynq this is more problematic because there is weird binding
>> for ARM. For example PMU which is out of bus and should be probably in cpu node.
>> Also scu devices, scutimers, watchdog which lie on the bus for our case and we
>> need to use PPI interrupt cpu mask. Different clock binding, maybe pinmux binding, etc.
>>
>> It means from my point of view if binding is correct, no problem to move it
>> out of the kernel. If a kernel patch change binding, it is worth for me to change
>> dts in the kernel too to reflect this change and track this change too.
>> My proposal is, let's clean all DTSes in the arm kernel that all platform use
>> the same binding where all platforms are just correctly described.
>
> AIUI this split/move isn't intended to change the existing policy, which
> is already that DTS files are supposed to remain compatible across
> kernel versions and that "flag days" are to be avoided. The split is
> supposed to make it harder (if not impossible) to accidentally break
> that policy.
>
> On the other hand I suppose there is an argument to be made for clearing
> up the cruft *before* making the split.
>
> Ultimately I think this will be up to Grant & co.
ok.
>> The reaching this point I would suggest that for arm, arm-soc maintainers should
>> keep eyes on any dts binding change and all these changes require ACK from Rob or Grant
>> (like device-tree maintainers).
>
> Yes, once we move onto Phase II I don't expect it will end up being me
> that is the DTS maintainer -- I expect the maintenance will remain with
> those who take care of it in linux.git today.
>
> My involvement in Phase I is really just to help out with the transition
> (ulterior motive: the Xen project would also like to use these DTS
> files...) not to perform a "land grab" or take over maintenance etc. I
> certainly don't think I am the right person to become the long term
> maintainer of device-tree.git!
Ok. I see you point right now in connection to different project where
your Phase I make more sense.
Our flow, because of a lot of flexibility in fpga word, is more based on DTS which
we don't have in hand and everyone has to maintain it.
Starting to using not correct DTSes by another project will be problematic.
It is good step but it suggests that they can start to use it but one change
in the kernel binging will caused that it breaks another project.
From my point of view this Phase I is good to do for DTSes which are correct.
Then another project can be sure that they won't change a lot.
I have the same experience with our DTS driven Qemu that it works when description
is stable but till that time it is just pain because you need to be sure
that all binding changes are also done in Qemu.
Thanks,
Michal
--
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91
w: http://www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel - Microblaze cpu - http://www.monstr.eu/fdt/
Maintainer of Linux kernel - Xilinx Zynq ARM architecture
Microblaze U-BOOT custodian and responsible for u-boot arm zynq platform