This patchset adds another D-Bus API for attaching applications, called
AttachFD.
When application uses that API, it receives one end of a datagram socket
pair, which it's supposed to recv() from in order to receive mesh
packets, instead of wating for *MessageReceived() calls over D-Bus.
This significantly reduces system load for high traffic environment
(e.g. an application that subscribes to a large number of publications
in a big network).
Message delivery is one way only: application is still supposed to call
*Send methods via D-Bus, although the socket pair is bidirectional, so
it would be possible to add sending as well.
Michał Lowas-Rzechonek (1):
mesh: Implement AttachFD method
Przemysław Fierek (1):
mesh: Add documentation for AttachFD
doc/mesh-api.txt | 40 +++++++++++++++
mesh/mesh.c | 12 ++++-
mesh/model.c | 126 ++++++++++++++++++++++++++++++++++++++++++++++-
mesh/node.c | 83 ++++++++++++++++++++++++++++++-
mesh/node.h | 4 +-
5 files changed, 259 insertions(+), 6 deletions(-)
--
2.20.1
Hi Michał,
On Tue, 2020-06-16 at 14:27 +0200, Michał Lowas-Rzechonek wrote:
> This patchset adds another D-Bus API for attaching applications, called
> AttachFD.
We have talked a little about this, but I am personally opposed to
switching the daemon--> App message delivery system to socket based.
This has the feeling of something that has been developed as the
result of a stress test to see how much data can be pushed through
the system as fast as possible, which should not be a common mesh use
case.
I also worry about the increase in system socket resorces...
Currently the daemon consumes 1 socket for App <--> daemon
communication (the one to the dbus daemon), and each App currently
being supported uses one more (again, to dbus daemon) And all
MUXing, marshaling and unmarshaling of messages is handled by
DBUS... the very reason for it's existance. While creating a
new socket between App and daemon would make the messages flow
faster, it comes at the cost of re-inventing MUXing, marshaling,
unmarshaling plus all the additional sockets. A larger code
footprint, and a *much* larger resource footprint.
This just feels to me like the kind of customization that is fine
for a vendor to do for a specialized high-flow mesh, but not
something that we want to impose on everyone who uses Mesh in the
open source community.
> When application uses that API, it receives one end of a datagram socket
> pair, which it's supposed to recv() from in order to receive mesh
> packets, instead of wating for *MessageReceived() calls over D-Bus.
>
> This significantly reduces system load for high traffic environment
> (e.g. an application that subscribes to a large number of publications
> in a big network).
>
> Message delivery is one way only: application is still supposed to call
> *Send methods via D-Bus, although the socket pair is bidirectional, so
> it would be possible to add sending as well.
>
> Michał Lowas-Rzechonek (1):
> mesh: Implement AttachFD method
>
> Przemysław Fierek (1):
> mesh: Add documentation for AttachFD
>
> doc/mesh-api.txt | 40 +++++++++++++++
> mesh/mesh.c | 12 ++++-
> mesh/model.c | 126 ++++++++++++++++++++++++++++++++++++++++++++++-
> mesh/node.c | 83 ++++++++++++++++++++++++++++++-
> mesh/node.h | 4 +-
> 5 files changed, 259 insertions(+), 6 deletions(-)
>
Brian,
On 06/16, Gix, Brian wrote:
> This has the feeling of something that has been developed as the
> result of a stress test to see how much data can be pushed through
> the system as fast as possible, which should not be a common mesh use
> case.
Well, not really. We came up with this mechanism because of performance
issues with *real world* networks on embedded platforms. And by "real
world" I mean a single network of 200+ nodes.
We need a way to monitor the nodes in such networks, and potentially
much bigger ones too - mesh addess space is 32k+ nodes, and we already
have runnin installations with ~1k nodes, all of which are publishing
sensor data.
Without this, bluetooth-meshd is *unusable*.
> I also worry about the increase in system socket resorces...
> Currently the daemon consumes 1 socket for App <--> daemon
> communication (the one to the dbus daemon), and each App currently
> being supported uses one more (again, to dbus daemon) And all
> MUXing, marshaling and unmarshaling of messages is handled by
> DBUS... the very reason for it's existance. While creating a
> new socket between App and daemon would make the messages flow
> faster, it comes at the cost of re-inventing MUXing, marshaling,
> unmarshaling plus all the additional sockets. A larger code
> footprint, and a *much* larger resource footprint.
As you can see in the patch, serialization format is very simple (*much*
simpler than D-Bus marshalling).
As for MUXing, I don't really understand what you mean? A node can have
only one application attached to it, so the daemon knows exactly which
socket to use. Same for the application.
About file descriptors: this patch targets primarily an embedded
platform. While I've seen issues with number of open files on busy
network servers, I don't think an IoT device is going to face such
problems.
As for the "resource footprint", it's a tradeoff between open files vs.
CPU. By using D-Bus calls to deliver network packets, you need to:
- copy the packet from the mesh daemon to dbus daemon
- have the daemon look up the application by D-Bus address
- have the daemon deserialize the message header and check permissions
- copy the packet from dbus daemon to application
This requires an additional copy and an additional context switch, and a
potentially costly permission check (I've mentioned AppArmor in the
previous mail).
So this delivery method in fact consumes *more* resources.
> This just feels to me like the kind of customization that is fine
> for a vendor to do for a specialized high-flow mesh, but not
> something that we want to impose on everyone who uses Mesh in the
> open source community.
I don't want to "impose" anything... the API is entirely optional and
the "regular" Attach() works as it used to, and might be perfectly fine
for simpler applications.
Note that GATT API does the same thing wrt. AquireNotify() call.
--
Michał Lowas-Rzechonek <[email protected]>
Silvair http://silvair.com
Jasnogórska 44, 31-358 Krakow, POLAND