From: Bartosz Golaszewski <[email protected]>
When discussing the recent user-space changes with Kent and while working
on dbus API for libgpiod I noticed that we really don't have any way of
keeping the line info synchronized between the kernel and user-space
processes. We can of course periodically re-read the line information or
even do it every time we want to read a property but this isn't optimal.
This series adds a new ioctl() that allows user-space to retrieve a
file-descriptor which can then be polled for events emitted by the kernel
when the line is requested, released or its status changed. This of course
doesn't require the line to be requested. Multiple user-space processes
can watch the same lines.
The first couple patches just fix some issues I noticed when implementing
the new interface. Patch 7/8 provides the actual ioctl() implementation
while patch 8/8 adds a simple user-space program to tools that can be used
to watch the line info changes.
Bartosz Golaszewski (8):
gpiolib: use 'unsigned int' instead of 'unsigned' in gpio_set_config()
gpiolib: have a single place of calling set_config()
gpiolib: convert the type of hwnum to unsigned int in
gpiochip_get_desc()
gpiolib: use gpiochip_get_desc() in linehandle_create()
gpiolib: use gpiochip_get_desc() in lineevent_create()
gpiolib: actually protect the line event kfifo with mutex
gpiolib: add new ioctl() for monitoring changes in line info
tools: gpio: implement gpio-watch
drivers/gpio/gpiolib.c | 267 +++++++++++++++++++++++++++++++++---
drivers/gpio/gpiolib.h | 4 +-
include/linux/gpio/driver.h | 3 +-
include/uapi/linux/gpio.h | 36 +++++
tools/gpio/.gitignore | 1 +
tools/gpio/Build | 1 +
tools/gpio/Makefile | 11 +-
tools/gpio/gpio-watch.c | 114 +++++++++++++++
8 files changed, 415 insertions(+), 22 deletions(-)
create mode 100644 tools/gpio/gpio-watch.c
--
2.23.0
On Wed, Nov 27, 2019 at 2:35 PM Bartosz Golaszewski <[email protected]> wrote:
> This series adds a new ioctl() that allows user-space to retrieve a
> file-descriptor which can then be polled for events emitted by the kernel
> when the line is requested, released or its status changed. This of course
> doesn't require the line to be requested. Multiple user-space processes
> can watch the same lines.
So if I understand correctly all the series do is expose metadata about
all GPIO lines to userspace?
I think up until now the use case assumptions have been:
- The kernel will pick off some GPIO lines, mostly during boot but
occasionally at runtime (by users such as kernel modules or
hotlugged devices).
- Userspace will pick some lines from those that are available,
after the kernel picked those it wants. If it tries to pick one of
those that the kernel already picked, the request will be denied.
The assumption (at least in my head) was that the GPIOs the
kernel picks will not be a very dynamic business.
So this appears to be dealing with this very dynamic business.
Is the *main* use case different userspace processes trying
to use the same pins and getting denied? Because in that
case we might be putting a bit too much userspace plumbing
into the kernel and we need to think about that for a while.
(Binder and kdbus etc comes to mind.)
So there is some feature growth happening here and I want
to be aware of the whole picture.
On a side track:
There is a bit about policy that needs to happen here I suppose,
like for example what if the kernel actually wants one of the
lines that userspace has picked? Should userspace be kicked
out and kernel get what it wants? (Arguably yes.)
Yours,
Linus Walleij
pt., 29 lis 2019 o 11:04 Linus Walleij <[email protected]> napisał(a):
>
> On Wed, Nov 27, 2019 at 2:35 PM Bartosz Golaszewski <[email protected]> wrote:
>
> > This series adds a new ioctl() that allows user-space to retrieve a
> > file-descriptor which can then be polled for events emitted by the kernel
> > when the line is requested, released or its status changed. This of course
> > doesn't require the line to be requested. Multiple user-space processes
> > can watch the same lines.
>
> So if I understand correctly all the series do is expose metadata about
> all GPIO lines to userspace?
>
I'd say it's about adding a way to dynamically watch changes in this metadata.
> I think up until now the use case assumptions have been:
>
> - The kernel will pick off some GPIO lines, mostly during boot but
> occasionally at runtime (by users such as kernel modules or
> hotlugged devices).
>
Indeed.
> - Userspace will pick some lines from those that are available,
> after the kernel picked those it wants. If it tries to pick one of
> those that the kernel already picked, the request will be denied.
>
> The assumption (at least in my head) was that the GPIOs the
> kernel picks will not be a very dynamic business.
>
> So this appears to be dealing with this very dynamic business.
>
> Is the *main* use case different userspace processes trying
> to use the same pins and getting denied? Because in that
> case we might be putting a bit too much userspace plumbing
> into the kernel and we need to think about that for a while.
> (Binder and kdbus etc comes to mind.)
>
No, it really is just about keeping the line information in user-space
synchronized with the one in the kernel without re-reading the line
info periodically. Whether it's the kernel that requests the line or
other user-space process doesn't matter. We just want to stay
up-to-date with the information we already do have access to.
> So there is some feature growth happening here and I want
> to be aware of the whole picture.
>
It may seem like a feature-creep because this is the third new feature
being added to the character device in a short span of time. But at
the same time: user-space wants to use GPIOs and they're mostly doing
it over sysfs. If you want people to switch over to the character
device - we must make it at least as useful as sysfs.
These new features are not unjustified: I receive a significant amount
of mail with feature-requests for libgpiod (also from people who are
not well aware that I can only support features exposed by mainline
kernel).
It turns out that RPi people really wanted the BIAS settings because
the downstream RPi GPIO interface supports it. Having added this may
now make them switch to libgpiod.
Old sysfs interface allows to change the direction of lines or their
active-low setting at run-time and it turned out this too is a
functionality people want to see in libgpiod. Thanks to Kent's effort
we now have it.
Last thing that users often complain about is the fact that with the
character device, the state of GPIO lines used by user-space needs to
be kept by the process using them. This unfortunately often comes from
the lack of understanding of how a character device functions, but it
really seems users want to have a centralized agent that takes control
of the lines, provides standardized interface to interact with them
and exports their metadata. Recognizing this fact, I implemented a
proof-of-concept dbus daemon, but one thing that it could really
benefit from is dynamic, event-based synchronization and this series
tries to add just that (BTW please take a look at the discussion under
patch 7/8 - the code in this series will probably change a lot).
I believe this may be the last missing piece and after that we can
really consider this ABI feature-complete.
> On a side track:
>
> There is a bit about policy that needs to happen here I suppose,
> like for example what if the kernel actually wants one of the
> lines that userspace has picked? Should userspace be kicked
> out and kernel get what it wants? (Arguably yes.)
>
We should probably start a new thread on this. I'll play the
user-space's advocate and say: has anyone ever needed this? Do other
kernel sub-systems do this?
> Yours,
> Linus Walleij
Best regards,
Bartosz
On Fri, Nov 29, 2019 at 11:58 AM Bartosz Golaszewski <[email protected]> wrote:
> pt., 29 lis 2019 o 11:04 Linus Walleij <[email protected]> napisał(a):
> > Is the *main* use case different userspace processes trying
> > to use the same pins and getting denied? Because in that
> > case we might be putting a bit too much userspace plumbing
> > into the kernel and we need to think about that for a while.
> > (Binder and kdbus etc comes to mind.)
>
> No, it really is just about keeping the line information in user-space
> synchronized with the one in the kernel without re-reading the line
> info periodically. Whether it's the kernel that requests the line or
> other user-space process doesn't matter. We just want to stay
> up-to-date with the information we already do have access to.
I was more after whether the expected conflict we resolve
will be mostly between kernel and userspace or between
userspace and userspace, statistically speaking. Like: what
is happening to actual systems people are trying to deploy.
> > So there is some feature growth happening here and I want
> > to be aware of the whole picture.
>
> It may seem like a feature-creep because this is the third new feature
> being added to the character device in a short span of time. But at
> the same time: user-space wants to use GPIOs and they're mostly doing
> it over sysfs. If you want people to switch over to the character
> device - we must make it at least as useful as sysfs.
So a part of the design criteria is to provide the same functions
as e.g. putting inotify's on /sys/class/gpio/* and watching as
files come and go or something like this?
(That sounds reasonable.)
> These new features are not unjustified: I receive a significant amount
> of mail with feature-requests for libgpiod (also from people who are
> not well aware that I can only support features exposed by mainline
> kernel).
OK
> It turns out that RPi people really wanted the BIAS settings because
> the downstream RPi GPIO interface supports it. Having added this may
> now make them switch to libgpiod.
That's good news, I think Drew said the same thing about
Beagle.
> Old sysfs interface allows to change the direction of lines or their
> active-low setting at run-time and it turned out this too is a
> functionality people want to see in libgpiod. Thanks to Kent's effort
> we now have it.
Yeah that's a win.
> Last thing that users often complain about is the fact that with the
> character device, the state of GPIO lines used by user-space needs to
> be kept by the process using them. This unfortunately often comes from
> the lack of understanding of how a character device functions, but it
> really seems users want to have a centralized agent that takes control
> of the lines, provides standardized interface to interact with them
> and exports their metadata. Recognizing this fact, I implemented a
> proof-of-concept dbus daemon, but one thing that it could really
> benefit from is dynamic, event-based synchronization and this series
> tries to add just that (BTW please take a look at the discussion under
> patch 7/8 - the code in this series will probably change a lot).
OK
> I believe this may be the last missing piece and after that we can
> really consider this ABI feature-complete.
OK that is a good point.
My own pet peeve is the industrial automation and control use
case: here we have the design space where people today use
either PLC:s or RaspberryPi's or Beagle boards, or even some
custom computers.
For me personally that is a design space we should cover and
if this helps the RaspberryPi to do that better I'm all for it.
An especially interesting case is multiple GPIO expanders
plugged in on pluggable busses such as PCI or USB. I think
that kind of discoverability and dynamically expandable GPIO
blocks is something people in the industry are quite keen to
get.
What we need to do is to make it dirt simple to use GPIOs for
custom hacks and construction of factories and doorstops
and what not, while at the same time strongly discouraging
it to be used to manage hardware such as laptops, tablets
or phones from userspace. That's maybe hard, and we might
be victims of our own success ...
However the eradication of the sysfs ABI seems to be well
on track!
> > On a side track:
> >
> > There is a bit about policy that needs to happen here I suppose,
> > like for example what if the kernel actually wants one of the
> > lines that userspace has picked? Should userspace be kicked
> > out and kernel get what it wants? (Arguably yes.)
> >
>
> We should probably start a new thread on this. I'll play the
> user-space's advocate and say: has anyone ever needed this? Do other
> kernel sub-systems do this?
So far it was designed under the assumption that this kind of
collisions are handled on a first-come-first serve basis, and
that is indeed how it works today.
I'm just not sure that this assumption is going to hold in the
future so wanted to make a little check-up.
Linus
On 29.11.19 11:58, Bartosz Golaszewski wrote:
Hi folks,
missed much of this thread, so please excuse me if I'm going to say
something stupid ;-)
> No, it really is just about keeping the line information in user-space
> synchronized with the one in the kernel without re-reading the line
> info periodically. Whether it's the kernel that requests the line or
> other user-space process doesn't matter. We just want to stay
> up-to-date with the information we already do have access to.
In that case, why not just using inotify+friends or some blocking read
for that ?
Personally, I'm a really big friend of synthentic filesystems instead of
ioctl()s, because they're so simple to use (don't wanna repeat the whole
Plan9 literature here ;-)), so I'd do it in this way:
* have a file in /sys/class/gpio/... holding the current gpio
information, in some easily parsable format. (either one file
globally or one per chip)
* a) this file can be monitored via inotify (just one event when
something had changed, or maybe have mtime reflect the time of
last change)
* b) allow read after EOF, which is blocking, until something changes
That way, userland implementation would be trivial enough to do it
in a simple shell script.
> It may seem like a feature-creep because this is the third new feature
> being added to the character device in a short span of time.
Personally, I'm not a fan of such chardevs at all, but this raises
another interesting pretty general topic: sidechannel/out-of-band data
of chardevs.
Originally, Unix chardevs have been designed as something operates on
streams of bytes, maybe with some extra operations (eg. setting baud
rate, etc). ioctl()s have been implemented as some escape route for such
extra functions. Over the many years, there's been a massive ioctl()
inflation, even hardware specific ones. IMHO, this is a huge mess, which
requires to lots of extra work for userland applications, which should
instead get generic interfaces, even leading to ridiculous things like
HAL, etc.
Seriously, we should lean back and think of better ways. One idea would
be making devices more directly-like, where things like oob-streams
or config attributes can be enumerated and addressed by names.
(actually, if I could change history, I'd make them actual directories)
> But at
> the same time: user-space wants to use GPIOs and they're mostly doing
> it over sysfs. If you want people to switch over to the character
> device - we must make it at least as useful as sysfs.
Personally, I don't see any practical reason, why I should use the
chardev at all - sysfs is much simpler. The only reason I see is when
needing to set several lines at once - but I haven't seen practical a
usecase where there isn't some specific device behind them, that
deservers it's own driver, attaching to some suitable subsystem.
(actually, I rarely talk to gpios directly from userland - except for
some lowlevel debugging purposes).
> These new features are not unjustified: I receive a significant amount
> of mail with feature-requests for libgpiod (also from people who are
> not well aware that I can only support features exposed by mainline
> kernel).
Do you have more detailed information on what these folks are really
up to, what their actual usecases are ?
> Last thing that users often complain about is the fact that with the
> character device, the state of GPIO lines used by user-space needs to
> be kept by the process using them. This unfortunately often comes from
> the lack of understanding of how a character device functions, but it
> really seems users want to have a centralized agent that takes control
> of the lines, provides standardized interface to interact with them
> and exports their metadata.
Yeah, I also sometimes have those kinds of clients. So far, in all cases
turned out that they actually needed a more high level device driver
(or extend an existing one) instead of touching gpios directly from
within the application. (usually things like relais or extra uart
signalling lines, power control, etc, via gpio, etc).
> Recognizing this fact, I implemented a proof-of-concept dbus daemon,
> but one thing that it could really
> benefit from is dynamic, event-based synchronization and this series
> tries to add just that (BTW please take a look at the discussion under
> patch 7/8 - the code in this series will probably change a lot).
Oh, no, I've been afraid that this would be coming ... desktop bus in
embedded system is a real nightmare. (yes, some people are even insane
enough to put that thing on an public IP interface and so make critical
machinery like cards easily remote controllable by average school kids)
Seriously, I've seen a lot of similar constructs in the field, all of
them have quickly turned out as complete crap - usually coming from
folks who need >50kLoc ontop of trivial cansocket with silly excuses
like formal certification :o). Please, don't give these kids a machine
gun, they will just hurt innocent people with it :P
> We should probably start a new thread on this. I'll play the
> user-space's advocate and say: has anyone ever needed this? Do other
> kernel sub-systems do this?
Hotplug could lead to such situations.
Playing the kernel advocate here: better don't even consider opening
this apocalypse box and focus on clean kernel drivers instead :p
--mtx
---
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287
On 29.11.19 13:57, Linus Walleij wrote:
Hi,
> My own pet peeve is the industrial automation and control use
> case: here we have the design space where people today use
> either PLC:s or RaspberryPi's or Beagle boards, or even some
> custom computers.
>
> For me personally that is a design space we should cover and
> if this helps the RaspberryPi to do that better I'm all for it.
Yeah, I also have clients in that field. The main problem here,
IMHO is their way of thinking: no device abstraction at all, but all
pretty much hardwired for specific installations (also the reason,
why they'll fail miserable on the new IOT field :p). These folks are
still used to the ancient pure PLC way, where there isn't even anything
like an operating system.
IMHO, it's an educational problem: people need to understand that
there're device abstractions for good reasons, and they should respect
and use them. Basically, they have to understand the concept of
modularization and abstractions. IEC 1131-3 obviously isn't made for
that.
For example, take simple heating installation. Pumps won't be controlled
by raw inverter configuration anymore, but rpm or m²/sec. Temp or flow
sensors don't give raw numbers, but degress Kelvin or m^/sec, etc.
In that case, the corresponding subsystem would be IIO - no need to ever
care about gpios, pwms, etc, directly.
Following this modular approach, everything suddenly gets much easier,
eg. replacing a pump by a different model just requires a minor
reconfiguration instead of rewriting huge parts of the plc code.
I've managed to teach this to an Siemens field technician in one evening
with a few beers, so it can't be that hard to understand.
> An especially interesting case is multiple GPIO expanders
> plugged in on pluggable busses such as PCI or USB. I think
> that kind of discoverability and dynamically expandable GPIO
> blocks is something people in the industry are quite keen to
> get.
Smells like a case for oftree overlays ...
> What we need to do is to make it dirt simple to use GPIOs for
> custom hacks and construction of factories and doorstops
> and what not, while at the same time strongly discouraging
> it to be used to manage hardware such as laptops, tablets
> or phones from userspace. That's maybe hard, and we might
> be victims of our own success ...
I contradict. We should encourage industrial/construction folks to
do decently structured, professional engineering - IOW: use
modularization and highlevel drivers, instead of tinkering with
raw gpios directly like a school kid.
--mtx
--
Dringender Hinweis: aufgrund existenzieller Bedrohung durch "Emotet"
sollten Sie *niemals* MS-Office-Dokumente via E-Mail annehmen/öffenen,
selbst wenn diese von vermeintlich vertrauenswürdigen Absendern zu
stammen scheinen. Andernfalls droht Totalschaden.
---
Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
GPG/PGP-Schlüssel zu.
---
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287
śr., 4 gru 2019 o 13:06 Enrico Weigelt, metux IT consult
<[email protected]> napisał(a):
>
> On 29.11.19 11:58, Bartosz Golaszewski wrote:
>
> Hi folks,
>
>
> missed much of this thread, so please excuse me if I'm going to say
> something stupid ;-)
>
> > No, it really is just about keeping the line information in user-space
> > synchronized with the one in the kernel without re-reading the line
> > info periodically. Whether it's the kernel that requests the line or
> > other user-space process doesn't matter. We just want to stay
> > up-to-date with the information we already do have access to.
>
> In that case, why not just using inotify+friends or some blocking read
> for that ?
>
> Personally, I'm a really big friend of synthentic filesystems instead of
> ioctl()s, because they're so simple to use (don't wanna repeat the whole
> Plan9 literature here ;-)), so I'd do it in this way:
>
> * have a file in /sys/class/gpio/... holding the current gpio
> information, in some easily parsable format. (either one file
> globally or one per chip)
> * a) this file can be monitored via inotify (just one event when
> something had changed, or maybe have mtime reflect the time of
> last change)
> * b) allow read after EOF, which is blocking, until something changes
>
> That way, userland implementation would be trivial enough to do it
> in a simple shell script.
>
> > It may seem like a feature-creep because this is the third new feature
> > being added to the character device in a short span of time.
>
> Personally, I'm not a fan of such chardevs at all, but this raises
> another interesting pretty general topic: sidechannel/out-of-band data
> of chardevs.
>
> Originally, Unix chardevs have been designed as something operates on
> streams of bytes, maybe with some extra operations (eg. setting baud
> rate, etc). ioctl()s have been implemented as some escape route for such
> extra functions. Over the many years, there's been a massive ioctl()
> inflation, even hardware specific ones. IMHO, this is a huge mess, which
> requires to lots of extra work for userland applications, which should
> instead get generic interfaces, even leading to ridiculous things like
> HAL, etc.
>
> Seriously, we should lean back and think of better ways. One idea would
> be making devices more directly-like, where things like oob-streams
> or config attributes can be enumerated and addressed by names.
> (actually, if I could change history, I'd make them actual directories)
>
> > But at
> > the same time: user-space wants to use GPIOs and they're mostly doing
> > it over sysfs. If you want people to switch over to the character
> > device - we must make it at least as useful as sysfs.
>
> Personally, I don't see any practical reason, why I should use the
> chardev at all - sysfs is much simpler. The only reason I see is when
> needing to set several lines at once - but I haven't seen practical a
> usecase where there isn't some specific device behind them, that
> deservers it's own driver, attaching to some suitable subsystem.
> (actually, I rarely talk to gpios directly from userland - except for
> some lowlevel debugging purposes).
>
> > These new features are not unjustified: I receive a significant amount
> > of mail with feature-requests for libgpiod (also from people who are
> > not well aware that I can only support features exposed by mainline
> > kernel).
>
> Do you have more detailed information on what these folks are really
> up to, what their actual usecases are ?
>
> > Last thing that users often complain about is the fact that with the
> > character device, the state of GPIO lines used by user-space needs to
> > be kept by the process using them. This unfortunately often comes from
> > the lack of understanding of how a character device functions, but it
> > really seems users want to have a centralized agent that takes control
> > of the lines, provides standardized interface to interact with them
> > and exports their metadata.
>
> Yeah, I also sometimes have those kinds of clients. So far, in all cases
> turned out that they actually needed a more high level device driver
> (or extend an existing one) instead of touching gpios directly from
> within the application. (usually things like relais or extra uart
> signalling lines, power control, etc, via gpio, etc).
>
> > Recognizing this fact, I implemented a proof-of-concept dbus daemon,
> > but one thing that it could really
> > benefit from is dynamic, event-based synchronization and this series
> > tries to add just that (BTW please take a look at the discussion under
> > patch 7/8 - the code in this series will probably change a lot).
>
> Oh, no, I've been afraid that this would be coming ... desktop bus in
> embedded system is a real nightmare. (yes, some people are even insane
> enough to put that thing on an public IP interface and so make critical
> machinery like cards easily remote controllable by average school kids)
>
Nah, you're overreacting, nobody is putting anything on the network.
Most of my work isn't in upstream kernel development - it's actually
launching CE products. And it's been a long time since I've worked
with a client that could be convinced not to use systemd and dbus on
an embedded system. This just makes things so easy for people writing
high-level application code that it's become de-facto standard -
whether we like it or not.
> Seriously, I've seen a lot of similar constructs in the field, all of
> them have quickly turned out as complete crap - usually coming from
> folks who need >50kLoc ontop of trivial cansocket with silly excuses
> like formal certification :o). Please, don't give these kids a machine
> gun, they will just hurt innocent people with it :P
>
This is just your personal anecdote. I could bring up several examples
of good implementations of event-based, dbus-oriented systems all
right. Bad programmers don't make tools they're using bad.
As for the sysfs vs chardev: this has been discussed a lot. We won't
be adding any new features to the sysfs interface. I want to add this
new feature to allow for creating a central entity controlling GPIO
lines in the user-space. In case of sysfs - IT IS such a central
entity except that it lives in the kernel. It doesn't need to be
notified about changes in line properties.
Bartosz
> > We should probably start a new thread on this. I'll play the
> > user-space's advocate and say: has anyone ever needed this? Do other
> > kernel sub-systems do this?
>
> Hotplug could lead to such situations.
>
> Playing the kernel advocate here: better don't even consider opening
> this apocalypse box and focus on clean kernel drivers instead :p
>
>
> --mtx
>
> ---
> Enrico Weigelt, metux IT consult
> Free software and Linux embedded engineering
> [email protected] -- +49-151-27565287
Hi,
>> Oh, no, I've been afraid that this would be coming ... desktop bus in
>> embedded system is a real nightmare. (yes, some people are even insane
>> enough to put that thing on an public IP interface and so make critical
>> machinery like cards easily remote controllable by average school kids)
>
> Nah, you're overreacting, nobody is putting anything on the network.
Maybe you've just missed that (maybe better for mental health ;-)), but
people actually do put desktop bus on open networks. Chrysler/Harmann
are an famous example (rendering the cars easily remote-controllable)
I've had lots of similar cases w/ my clients. Whats going on in critical
industries like automotive or medical often is a real nightmare. One of
the reasons why I only buy old cars and do a careful inspection of
physicians before allowing them to treat me or my family.
IMHO, we should be careful about not encouraging folks out there to do
silly things, just because they look so easy. (embedded development
isn't entirely easy)
> Most of my work isn't in upstream kernel development - it's actually
> launching CE products. And it's been a long time since I've worked
> with a client that could be convinced not to use systemd and dbus on
> an embedded system.
Yeah, same problem here. But actually, it's easier than one might think:
let them run into a scenario where that malware really does horrible
things and let the client drive against the wall. Some will learn,
others won't - just let evolution optimize that :p
> This just makes things so easy for people writing
> high-level application code that it's become de-facto standard -
> whether we like it or not.
Yeah, "hilevel" code, that completely ignores vital aspects of the
machine it then runs on, eg. RT or memory constraints. Seen that so
many times. Emperical experience tells: sooner or later it will go
nuts and things go horribly wrong. The unplesant question just is: how
many people have get hurt or killed in the process.
Lennartware isn't the only problem here, but also ancient and completely
unmaintained vendor kernels, patched-to-deah by rookies (eg. ublox).
OTOH, the positive side, eg. if I really need to get a *new* car, I'd
pick one where I can disable surveillance stuff like "ecall" easily :p
In general, what those industries completely ignore is maintainability
and stability. There, this only exists on the paper (cubic meters of
papers that formally look nice, but don't really tell anything about
the *actual* important facts) - formally proven bullshit.
I've made the personal decision that I don't wanna become yet another
Edward Teller, neither do I wanna allow some BSG-type scenario. That's
the reason I'm spending so much time in this discussion.
> This is just your personal anecdote. I could bring up several examples
> of good implementations of event-based, dbus-oriented systems all
> right. Bad programmers don't make tools they're using bad.
Problem is: good programmers who can really deal with those concepts
are hard to find. And those systems usually are hard to test. Desktop
bus is magnitudes more complex than isolated FSMs.
Within the last decades, I never had a single case where desktop bus had
been the superior choice (actually, only few cases where purely event-
driven architecture was a good idea).
> I want to add this
> new feature to allow for creating a central entity controlling GPIO
> lines in the user-space. In case of sysfs - IT IS such a central
> entity except that it lives in the kernel. It doesn't need to be
> notified about changes in line properties.
I'm still wondering why exactly one would want such an central entity
in userspace and then letting it talk to others via desktop bus. At
least in critical embedded systems, not talking about self-knittet
hobby projects. Seems those ideas originate from people who don't know
what the OS/Kernel is for :p
Actually, I had a case where somebody attemted to do so (folks who tried
linux embedded development from a windows machine :o). Fortunately,
before they could write their planned "GPIO HAL" (several 10kLoc for
just doing some trivial sysfs ops), I've configured the lines to the
proper drivers in DT (basically, just some LEDs, inputs, rs485 signal
lines, etc). Later on they started to write some "HAL" for LEDs,
inputs, rs485 ... the manager of neighboring department on the other
end of the room then ask me: "don't they know that they've got an
operating system" ;-)
Seriously, we shouldn't do everything that some strange folks are asking
for, just to make them happy. We don't wanna desktop bus in the kernel,
do we ? :p
By the way: we *do* need some improvements for PLC-like applications:
Highlevel provisioning of platform devices, eg. configure which devices
are attached to certain interfaces in an specific installation, so the
applications don't ever need any knowledge about that, but just talk
to specific devices.
Most of what's needed (on kernel side) is already there: DT overlays.
Just yet have to get this working on non-DT (eg. ACPI platform).
In fact that's already in my pipeline, also for other purposes, eg.
replace simple board or mfd drivers by just a DT snippet. Yet lacked the
time to work out a suitable way for mixing DT and ACPI and allowing DT
to override ACPI. But that's a topic for a completely different thread.
--mtx--
---
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287
On Wed, Dec 4, 2019 at 1:06 PM Enrico Weigelt, metux IT consult
<[email protected]> wrote:
> Personally, I'm a really big friend of synthentic filesystems instead of
> ioctl()s, because they're so simple to use (don't wanna repeat the whole
> Plan9 literature here ;-)), so I'd do it in this way:
>
> * have a file in /sys/class/gpio/... holding the current gpio
> information, in some easily parsable format. (either one file
> globally or one per chip)
The topology for the current solution is in /sys/bus/gpio actually,
but I get what you mean.
> * a) this file can be monitored via inotify (just one event when
> something had changed, or maybe have mtime reflect the time of
> last change)
> * b) allow read after EOF, which is blocking, until something changes
>
> That way, userland implementation would be trivial enough to do it
> in a simple shell script.
The current (deprecated) sysfs pretty much does this.
The main issue sysfs in its current form had to die was that it relied
on global GPIO numbers. An alternative to the character device
would be to use e.g. subdirs for each GPIO chip and export
local offset numbers from there, but well we
reached a fork in the road with the chardev I'd say.
The main problem solved with the chardev was that scripts that
died/crashed left the sysfs nodes explicitly exported and
populated and everything just in the general mess it was at the time
the application crashed.
Of course it is easy to pose things like that the application should
register crash handlers or whatnot, but it turns out people weren't
doing that and with a character device, then it cleans up automatically
if the application dies or get terminated by a signal for example,
and the same application can just be relaunched without problems.
> > But at
> > the same time: user-space wants to use GPIOs and they're mostly doing
> > it over sysfs. If you want people to switch over to the character
> > device - we must make it at least as useful as sysfs.
>
> Personally, I don't see any practical reason, why I should use the
> chardev at all - sysfs is much simpler.
I see your stance, but it also makes it much easier to shoot
yourself in the foot.
> (actually, I rarely talk to gpios directly from userland - except for
> some lowlevel debugging purposes).
Nobody should. The users of userspace GPIO are factory lines,
industrial control and automation, maker communities and odd
prototypes. Not deployed products like phones or computers.
> Do you have more detailed information on what these folks are really
> up to, what their actual usecases are ?
The typical cases involves rigging a few relays and sensors
up in a lab to perform some automation, not dissimilar to e.g.
PLC (programmable logic controllers) and such. The world is
full of these one-offs, some in more expensive and intimidating
environments than others. Some are the lab bench of a few
select makers. Makers are not important to big capital and
big business (who are not talking to us) but they are important
to the community exactly because they are talking to us.
> > Recognizing this fact, I implemented a proof-of-concept dbus daemon,
> > but one thing that it could really
> > benefit from is dynamic, event-based synchronization and this series
> > tries to add just that (BTW please take a look at the discussion under
> > patch 7/8 - the code in this series will probably change a lot).
>
> Oh, no, I've been afraid that this would be coming ... desktop bus in
> embedded system is a real nightmare. (yes, some people are even insane
> enough to put that thing on an public IP interface and so make critical
> machinery like cards easily remote controllable by average school kids)
>
> Seriously, I've seen a lot of similar constructs in the field, all of
> them have quickly turned out as complete crap - usually coming from
> folks who need >50kLoc ontop of trivial cansocket with silly excuses
> like formal certification :o). Please, don't give these kids a machine
> gun, they will just hurt innocent people with it :P
I don't think that argument by likeness (with something you don't
like) is a very good one.
What we are discussing is what is nowadays referred to as
the "service layer" including D-Bus etc and systemd, whether
it uses D-Bus or something else is irrelevant.
OpenWrt has a service layer, it is called ubus, and it has a
system layer daemon as well (similar to systemd). This is deployed
in millions of routers as router manufacturers almost exclusively
build on top of OpenWrt these days. It is not a lot of code.
https://openwrt.org/docs/techref/ubus
Yours,
Linus Walleij
pt., 6 gru 2019 o 13:53 Enrico Weigelt, metux IT consult
<[email protected]> napisał(a):
>
> Hi,
>
> >> Oh, no, I've been afraid that this would be coming ... desktop bus in
> >> embedded system is a real nightmare. (yes, some people are even insane
> >> enough to put that thing on an public IP interface and so make critical
> >> machinery like cards easily remote controllable by average school kids)
> >
> > Nah, you're overreacting, nobody is putting anything on the network.
>
> Maybe you've just missed that (maybe better for mental health ;-)), but
> people actually do put desktop bus on open networks. Chrysler/Harmann
> are an famous example (rendering the cars easily remote-controllable)
> I've had lots of similar cases w/ my clients. Whats going on in critical
> industries like automotive or medical often is a real nightmare. One of
> the reasons why I only buy old cars and do a careful inspection of
> physicians before allowing them to treat me or my family.
>
I admit - I mostly work in consumer electronics. The two times I
contracted for automotive companies I was actually surprised by how
seriously security was approached. Nothing like your stories. Maybe I
was just lucky though.
> IMHO, we should be careful about not encouraging folks out there to do
> silly things, just because they look so easy. (embedded development
> isn't entirely easy)
>
> > Most of my work isn't in upstream kernel development - it's actually
> > launching CE products. And it's been a long time since I've worked
> > with a client that could be convinced not to use systemd and dbus on
> > an embedded system.
>
> Yeah, same problem here. But actually, it's easier than one might think:
> let them run into a scenario where that malware really does horrible
> things and let the client drive against the wall. Some will learn,
> others won't - just let evolution optimize that :p
>
> > This just makes things so easy for people writing
> > high-level application code that it's become de-facto standard -
> > whether we like it or not.
>
> Yeah, "hilevel" code, that completely ignores vital aspects of the
> machine it then runs on, eg. RT or memory constraints. Seen that so
> many times. Emperical experience tells: sooner or later it will go
> nuts and things go horribly wrong. The unplesant question just is: how
> many people have get hurt or killed in the process.
>
> Lennartware isn't the only problem here, but also ancient and completely
> unmaintained vendor kernels, patched-to-deah by rookies (eg. ublox).
> OTOH, the positive side, eg. if I really need to get a *new* car, I'd
> pick one where I can disable surveillance stuff like "ecall" easily :p
>
> In general, what those industries completely ignore is maintainability
> and stability. There, this only exists on the paper (cubic meters of
> papers that formally look nice, but don't really tell anything about
> the *actual* important facts) - formally proven bullshit.
>
> I've made the personal decision that I don't wanna become yet another
> Edward Teller, neither do I wanna allow some BSG-type scenario. That's
> the reason I'm spending so much time in this discussion.
>
So I'm going through this e-mail for the third time now and I'm under
the impression it's mostly just an unrelated rant. I'm not even sure
how to respond.
> > This is just your personal anecdote. I could bring up several examples
> > of good implementations of event-based, dbus-oriented systems all
> > right. Bad programmers don't make tools they're using bad.
>
> Problem is: good programmers who can really deal with those concepts
> are hard to find. And those systems usually are hard to test. Desktop
> bus is magnitudes more complex than isolated FSMs.
>
> Within the last decades, I never had a single case where desktop bus had
> been the superior choice (actually, only few cases where purely event-
> driven architecture was a good idea).
>
It depends on what you're hired to do. Most of the time our clients
will have the entire applicative layer done by the time they reach out
to us. What they need is the BSP, OTA updates, provisioning and
factory tests. Good luck signing a deal if you start your pitch by
saying "your architecture is bad and you should feel bad". :)
> > I want to add this
> > new feature to allow for creating a central entity controlling GPIO
> > lines in the user-space. In case of sysfs - IT IS such a central
> > entity except that it lives in the kernel. It doesn't need to be
> > notified about changes in line properties.
>
> I'm still wondering why exactly one would want such an central entity
> in userspace and then letting it talk to others via desktop bus. At
> least in critical embedded systems, not talking about self-knittet
> hobby projects. Seems those ideas originate from people who don't know
> what the OS/Kernel is for :p
>
> Actually, I had a case where somebody attemted to do so (folks who tried
> linux embedded development from a windows machine :o). Fortunately,
> before they could write their planned "GPIO HAL" (several 10kLoc for
> just doing some trivial sysfs ops), I've configured the lines to the
> proper drivers in DT (basically, just some LEDs, inputs, rs485 signal
> lines, etc). Later on they started to write some "HAL" for LEDs,
> inputs, rs485 ... the manager of neighboring department on the other
> end of the room then ask me: "don't they know that they've got an
> operating system" ;-)
>
> Seriously, we shouldn't do everything that some strange folks are asking
> for, just to make them happy. We don't wanna desktop bus in the kernel,
> do we ? :p
>
> By the way: we *do* need some improvements for PLC-like applications:
> Highlevel provisioning of platform devices, eg. configure which devices
> are attached to certain interfaces in an specific installation, so the
> applications don't ever need any knowledge about that, but just talk
> to specific devices.
>
> Most of what's needed (on kernel side) is already there: DT overlays.
> Just yet have to get this working on non-DT (eg. ACPI platform).
>
Sure, but this has been discussed a lot already on linux-gpio. Linus
has just pointed out where GPIOs are used from user-space.
Bart
> In fact that's already in my pipeline, also for other purposes, eg.
> replace simple board or mfd drivers by just a DT snippet. Yet lacked the
> time to work out a suitable way for mixing DT and ACPI and allowing DT
> to override ACPI. But that's a topic for a completely different thread.
>
>
> --mtx--
>
> ---
> Enrico Weigelt, metux IT consult
> Free software and Linux embedded engineering
> [email protected] -- +49-151-27565287
On 06.12.19 16:44, Linus Walleij wrote:
> The main issue sysfs in its current form had to die was that it relied
> on global GPIO numbers. An alternative to the character device
> would be to use e.g. subdirs for each GPIO chip and export
> local offset numbers from there,
Yes, shouldn't be such a big deal. Maybe I'll find some time to fix
this. Another nice feature would be supporting gpio names for export
and unexport, as well as a file that lists all available gpios.
> but well we reached a fork in the road with the chardev I'd say.
Why can't we support both ? These two interfaces have different use
cases, neither one can replace the other one completely. The cool
thing w/ Linux is the configurability (once I've got some spare time,
I'll create some patches to make the chardev optional :p).
> The main problem solved with the chardev was that scripts that
> died/crashed left the sysfs nodes explicitly exported and
> populated and everything just in the general mess it was at the time
> the application crashed.
Yes, but this cleanup-on-close also could have been done in the old
interface. OTOH, there are use cases where you do not want this,
where sysfs interface is fine (eg. set up certain things in a boot
script, etc)
> Of course it is easy to pose things like that the application should
> register crash handlers or whatnot, but it turns out people weren't
> doing that and with a character device, then it cleans up automatically
> if the application dies or get terminated by a signal for example,
I don't question this usecase. Such automatic cleanup is indeed an
important feature. (even though I never had a practical usecase myself,
where I did go via raw gpios instead of binding some driver).
My bad feelings about chardev comes from entirely different angles, eg:
* not portable, not network transparent (needs special ioctl())
* needs a lot of more code in application side (yes: I also count in
libraries :p)
* no access control for individual lines
* can't use filesystem (eg. symlinks) to assign application /
installation specific (path)names, which individual programs can
act on.
> I see your stance, but it also makes it much easier to shoot
> yourself in the foot.
Yes, you have to care about state tracking (even in error cases) on your
own. Typical issue for such lowlevel things. The optimal case would be
having higher level drivers (eg. led, keys, ...) to bind to, but we're
talking about use cases where that isn't an option.
> Nobody should. The users of userspace GPIO are factory lines,
> industrial control and automation, maker communities and odd
> prototypes. Not deployed products like phones or computers.
Unfortunately, the real-world is different :(
I've had many cases where folks we using raw gpios (with insanely huge
userland code around that) even for trivial things like leds, keys,
serial control lines, etc.
Okay, this isn't a technical, but an educational problem. Don't know
a good solution, except for either trying educate folks or just
ignoring them :o
> The typical cases involves rigging a few relays and sensors
> up in a lab to perform some automation, not dissimilar to e.g.
> PLC (programmable logic controllers) and such. The world is
> full of these one-offs, some in more expensive and intimidating
> environments than others. Some are the lab bench of a few
> select makers. Makers are not important to big capital and
> big business (who are not talking to us) but they are important
> to the community exactly because they are talking to us.
Yeah, expected you'll be saying that ;-)
That bad side here is, this stuff often is going into the field
this way. Seen that many times.
I believe the real problem is on a completely different layer: it's
still too hard (for non-kernel-hackers) to configure things like
platform devices. For DT-based systems we've got DT-overlays (already
solves the kernel side here) - but for non-DT (eg. ACPI) world, it's
ugly. And even worse: if things like gpios come in on probe'able
bus'es like PCI or USB, it gets even more weird.
So, IMHO; the actual topic we should concentrate on is dynamic
multi-level device provisioning / binding. Yes, it's a big topic, but
something I definitively have on my 2do list.
--mtx
---
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287