Subject: Re: devicetree live reconfigureable hardware (eoma68)
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\))
Content-Type: text/plain; charset=utf-8
From: Pantelis Antoniou <pantelis.antoniou@konsulko.com>
In-Reply-To: <CAPweEDwnvcL0PkbyGU2p__TyNV3OCMUhDAsizS_MiXP1vdZ1VQ@mail.gmail.com>
Date: Tue, 10 Nov 2015 22:30:22 +0200
Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Grant Likely <grant.likely@linaro.org>
Content-Transfer-Encoding: 8BIT
Message-Id: <B18AC6D0-BFAB-4CDB-89BA-64449894E903@konsulko.com>
References: <CAPweEDwnvcL0PkbyGU2p__TyNV3OCMUhDAsizS_MiXP1vdZ1VQ@mail.gmail.com>
To: Luke Kenneth Casson Leighton <lkcl@lkcl.net>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 10749
Lines: 258


> On Nov 7, 2015, at 19:36 , Luke Kenneth Casson Leighton <lkcl@lkcl.net> wrote:
> 
> hi folks,
> 

Hi Luke,

This is a quite lengthy treatise you wrote here.

I was not very familiar with EOMA68 so I hit the web to read about
it first. At first glance it seems quite reasonable.
 
> i raised the issue of dynamic reconfigureable hardware back in 2012,
> and thank you (once again) to alan for kindly answering:
> https://lkml.org/lkml/2012/3/10/71
> 
> since then, yaay!  beaglebone with "capes" has driven some
> modifications to the linux kernel:
> http://events.linuxfoundation.org/sites/events/files/slides/dynamic-dt-elce14.pdf
> 
> the hardware differences between beaglebone capes and EOMA68 hardware
> is that EOMA68 CPU Cards may remain powered up whilst being removed
> from a "chassis" (whether it be a laptop, a tablet, an engineering
> board, a desktop - whatever) and plugged into a new one [i'm sure that
> it's theoretically possible for beaglebone capes to be carefully
> removed from a live-running beagleboard but i certainly wouldn't like
> to try it].

There is no support for hotplug (yet) for beaglebone capes.
There is no real software reason, it’s just that the hardware is not designed
for it.
> 
> the difference between the beaglebone cape on-board EEPROM and the
> original idea that i came up with for EOMA68 is that the on-board
> EEPROM in EOMA68 would contain actual device-tree configurations
> describing the plugin-hardware, whereas from what i can gather
> beaglebone cape EEPROMs merely contain an "identifier”.

I would caution against putting the device tree configuration in the
storage of the expansion board. While appealing at first IMHO we are
not at a point where device tree blobs are perfectly forward compatible.

For some of the messy details about why, please take a look at the excellent
presentation that Thomas Petazzoni made:

http://free-electrons.com/pub/conferences/2015/elc/petazzoni-dt-as-stable-abi-fairy-tale/petazzoni-dt-as-stable-abi-fairy-tale.pdf

The identifier can be easily mapped to a device tree blob object from
the filesystem which we can update in lockstep with the kernel. 

> 

> so, first thing that a CPU Card does is: read the EEPROM data, get the
> various device-trees in it, and start totally reconfiguring the 68
> pins and associated hardware.  not a problem, right? weelll... :)
> 
> here's the problem that i see, perhaps someone could tell me an
> appropriate place to discuss this (if LKML isn't), and if the issue i
> describe has already been catered for and is easily (and conveniently)
> solvable.
> 
> first, EOMA68 is 68 pins, many of which share a GPIO function.  so,
> for example, the UART TX and RX pins may also be multiplexed to plain
> GPIO.  likewise all of the pins of SPI as well as SD/MMC, these may be
> GPIO - just like with the beaglebone capes.
> 
> so, first complication: it is necessary to select "SD/MMC" interface
> instead of the GPIO functions.... but then, 5 minutes later, whoops,
> we have to select GPIO pins.  but, remember, there are at present
> *three* totally different CPU Cards (based on the allwinner a20,
> another on the ingenic jz4775, and a 64-bit ARM card is being
> designed).
> 
> to clarify that: beaglebone is {1}-Board to {M}-Many-Capes, whereas
> EOMA68 is {N}-CPUCards to {M}-Chassis.
> 
> so whereas with the beaglebone it's perfectly fine to just hard-code
> "oh yeah, pin 50 is GPIO but it's also multiplexed with SD CLK line",
> with EOMA68 it is simply flat-out impossible to do that.  you simply
> cannot know, in the future, what GPIO will be connected on a CPU Card.
> the *only* information that you have is, "EOMA68 pin 16 is GPIO0 or SD
> Data line 3".  you have *NO IDEA* what *actual* GPIO that goes to on a
> particular SoC, especially ones that will be available over the next
> decade.
> 
> so that's complication (1).
> 

We are already (me and gcl) trying to design something that will address
this problem. And no, your EOMA68 boards are not the first ones to hit it.

The 96board family has the same problem, all the boards with arduino compatible
headers have the same problem, etc..

> however, there is a secondary complication in that a hardware designer
> may choose *not* to use *all* of the pins associated with a particular
> hardware function.  so for example, with SPI, it is possible to use
> the 4-wire variant which has DDR capability (100mbytes/sec which is
> pretty awesome, thanks to intel's work i believe).  or, you could go
> down to a 2-wire data width bus.  or, you could go back to the
> standard 4-wire configuration (NSS, CLK, MOSI, MISO), *or* you could
> do the 3-wire SPI variant which is uniplexed single-wire data for both
> transmit and send (NSS, CLK, MOSI shared).  and, not only that, but if
> someone decides they don't want to use the standard NSS wire, that's
> fine - they can use another GPIO pin for NSS!
> 
> another example: SD/MMC.  it's either SPI-compatibility mode (3 or 4
> wire), or it's a 1-wire SD/MMC data bus, or a 2-wire, or a 4-wire bus.
> oh, and there's a "Card Detect" GPIO... which is also optional, and
> you use polling instead of EINT IRQ on that wire!
> 
> how in god's green earth are you supposed to describe this in
> reconfigureable device-tree blocks??? :)
> 
> should we write:
> 
> * one device-tree file to describe the SPI 4-wire DDR mode
> * one for the SPI 2-wire DDR mode
> * one for the SPI 4-wire standard variant
> * one for the SPI 3-wire standard variant
> * one for... o fuk i forgot that you need to double those up because
> of the NSS possibly being a different GPIO pin...
> 
> argh now we have *EIGHT* possible device-tree files to dynamically
> choose from... and that's just the SPI interface!
> 
> now we have to do the same thing for SD/MMC... that's what... eight
> possible device-tree files to choose from there, as well??
> 
> and we haven't started on the RGB/TTL interface yet!  there's even the
> possibility, with the RGB/TTL interface, that it could be connected to
> a converter IC, where a standard monitor (DVI, VGA) with EDID
> capability could be connected to it.  oh and don't forget that it
> could be 18-bit output or 15-bit output.
> 
> 
> possible solution, here, which solves both complication (1) as well as
> complication (2): a level of abstraction is needed.  the abstraction
> is needed because, as mentioned in complication (1) the only
> consistent naming is in the EOMA68 interface, *NOT* in the SoCs
> *BEHIND* the EOMA68 interface.
> 

No, that way is madness.

The idea I’m toying with requires the introduction of something called
a ‘connector’ which will deal with all mapping of resources from the
platform agnostic connector format to the internal SoC/CPU card format.
That includes an extension to the notion of pinmuxing to something that
is more abstracted.

So the device tree blobs would be the same for each board that supports
the connector and the resources they define will be mapped either statically
or dynamically using the connector ‘object’ as a translating proxy.

Sorry, I have been quite busy lately so I haven’t been able to post
new patches.

> the only possible way at the moment to do that is, i believe, to use
> device tree "includes".  there would be files, on each SoC's kernel,
> such as this:
> 
> * eoma68_pin_1_when_configured_as_gpio.dtsi
> * eoma68_pin_35_when_configured_as_gpio.dtsi
> * .... gpio ...
> * etc.
> 
> and then:
> 
> * eoma68_sdmmc_in_4wire_data_mode.dtsi
> * eoma68_sdmmc_in_2wire_data_mode.dtsi
> * eoma68_sdmmc_in_1wire_data_mode.dtsi
> * eoma68_sdmmc_in_spi_mode.dtsi
> 
> and so on and so forth.
> 
> then, having those available as well-known library files, which are
> guaranteed to cover all and every possibility, a "Chassis" may then
> have a single devicetree file which comprises:
> 
> /include/ eoma68_pin_1_when_configured_as_gpio.dtsi
> ....
> ....
> 
> and pretty much nothing else.
> 
> absolutely insane and guaranteed to make even the most sensible linux
> kernel programmer spew and hurl their lunch, projectile-vomit-fashion,
> over a deep and wide area.
> 
> so.
> 

No. Let’s not do that ^^^

> first question.
> 
> is there *anything* in devicetree that has a level of abstraction
> where all these capabilities may be compressed into a single file?
> naming, classes, anything?   the thought of splitting things up into
> almost 100 separate dtsi files is genuinely making me cringe.
> 
> i have an even more horrible idea of overloading phandles, like
> setting a variable "eoma68_pin_1_as_gpio = <0xe0680001>" or something
> like that - using the phandle 32-bit values as a predefined convention
> that, when it comes to loading the devicetree file off the chassis
> EEPROM, those predefined 32-bit conventions are the necessary
> abstraction.
> 
> ... but honestly, it would be much much better to have strings, not
> 32-bit values.
> 
> any other ideas out there?
> 

Nope, don’t do that either :)

> now, for anyone considering that this discussion might be irrelevant,
> consider this: in the near future (if it hasn't happened already) some
> bright spark is going to realise that the beaglebone "capes" is a
> ready-made market for creating a far superior "compatible" form-factor
> engineering board - perhaps something with a dual-core or quad-core
> processor... just like has *already* happened with arduinos.
> 
> in other words, the current beaglebone black hardware expansion
> interface becomes a de-facto industry standard, just like the arduino
> shield interface already is [the only reason we haven't had this come
> up in the linux kernel community with arduino shields and boards
> already is because arduino boards aren't powerful enough to run the
> linux kernel].
> 
> the implications there are that in a relatively short time, beaglebone
> "capes" will likely turn from a 1-to-many scenario into the exact same
> many-to-many scenario as faced by EOMA68 hardware right now.  if not
> beaglebone, then something else.
> 
> so this is *not* a theoretical problem, isolated to a single hardware
> design, it's a generic problem that's going to present itself and
> smack people in the teeth fairly shortly.  yaay! :)
> 
> l.

It’s not a theoretical problem, it’s already hitting people in the face.
You can’t get smacked in the face by a theory.

I’m working on it, hope I’ll have something to present before the Christmas vacations,
but, unfortunately time is limited.

Regards

— Pantelis

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/