Received-SPF: pass (google.com: domain of linux-bluetooth-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 17 Jun 2020 10:24:30 +0200
From:   "michal.lowas-rzechonek@silvair.com" 
        <michal.lowas-rzechonek@silvair.com>
To:     "Gix, Brian" <brian.gix@intel.com>
Cc:     "linux-bluetooth@vger.kernel.org" <linux-bluetooth@vger.kernel.org>,
        "Stotland, Inga" <inga.stotland@intel.com>,
        "przemyslaw.fierek@silvair.com" <przemyslaw.fierek@silvair.com>
Subject: Re: [RFC BlueZ 0/2] mesh: Deliver mesh packets over datagram socket
Message-ID: <20200617082430.hezshwckqtxq73bc@mlowasrzechonek2133>
Mail-Followup-To: "Gix, Brian" <brian.gix@intel.com>,
        "linux-bluetooth@vger.kernel.org" <linux-bluetooth@vger.kernel.org>,
        "Stotland, Inga" <inga.stotland@intel.com>,
        "przemyslaw.fierek@silvair.com" <przemyslaw.fierek@silvair.com>
References: <20200616122745.25056-1-michal.lowas-rzechonek@silvair.com>
 <cf1b99049f6b843affbdc7355038a97331993d14.camel@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <cf1b99049f6b843affbdc7355038a97331993d14.camel@intel.com>
User-Agent: NeoMutt/20180716
Sender: linux-bluetooth-owner@vger.kernel.org
Precedence: bulk

Brian,

On 06/16, Gix, Brian wrote:
> This has the feeling of something that has been developed as the
> result of a stress test to see how much data can be pushed through
> the system as fast as possible, which should not be a common mesh use
> case.

Well, not really. We came up with this mechanism because of performance
issues with *real world* networks on embedded platforms. And by "real
world" I mean a single network of 200+ nodes.

We need a way to monitor the nodes in such networks, and potentially
much bigger ones too - mesh addess space is 32k+ nodes, and we already
have runnin installations with ~1k nodes, all of which are publishing
sensor data.

Without this, bluetooth-meshd is *unusable*.

> I also worry about the increase in system socket resorces...
> Currently the daemon consumes 1 socket for App <--> daemon
> communication (the one to the dbus daemon), and each App currently
> being supported uses one more (again, to dbus daemon) And all
> MUXing, marshaling and unmarshaling of messages is handled by
> DBUS...   the very reason for it's existance. While creating a
> new socket between App and daemon would make the messages flow
> faster, it comes at the cost of re-inventing MUXing, marshaling,
> unmarshaling plus all the additional sockets. A larger code
> footprint, and a *much* larger resource footprint.

As you can see in the patch, serialization format is very simple (*much*
simpler than D-Bus marshalling).

As for MUXing, I don't really understand what you mean? A node can have
only one application attached to it, so the daemon knows exactly which
socket to use. Same for the application.

About file descriptors: this patch targets primarily an embedded
platform. While I've seen issues with number of open files on busy
network servers, I don't think an IoT device is going to face such
problems.

As for the "resource footprint", it's a tradeoff between open files vs.
CPU. By using D-Bus calls to deliver network packets, you need to:

 - copy the packet from the mesh daemon to dbus daemon
 - have the daemon look up the application by D-Bus address
 - have the daemon deserialize the message header and check permissions
 - copy the packet from dbus daemon to application

This requires an additional copy and an additional context switch, and a
potentially costly permission check (I've mentioned AppArmor in the
previous mail).

So this delivery method in fact consumes *more* resources.

> This just feels to me like the kind of customization that is fine
> for a vendor to do for a specialized high-flow mesh, but not
> something that we want to impose on everyone who uses Mesh in the
> open source community.

I don't want to "impose" anything... the API is entirely optional and
the "regular" Attach() works as it used to, and might be perfectly fine
for simpler applications.

Note that GATT API does the same thing wrt. AquireNotify() call.

-- 
Michał Lowas-Rzechonek <michal.lowas-rzechonek@silvair.com>
Silvair http://silvair.com
Jasnogórska 44, 31-358 Krakow, POLAND