+Wim and George from PipeWire
On Tue, 3 Dec 2019, at 12:15 AM, Pali Rohár wrote:
> On Monday 02 December 2019 19:01:11 Tanu Kaskinen wrote:
> > On Sun, 2019-12-01 at 19:57 +0100, Pali Rohár wrote:
> > > Hello!
> > >
> > > I'm sending this email to relevant mailing lists and other people who
> > > could be interested in it. (I'm not subscribed to all of ML, so please
> > > CC me when replying).
> > >
> > >
> > > I would like to open a discussion about a completely new way of handling
> > > Bluetooth HSP and HFP profiles on Linux. These two profiles are the only
> > > standard way how to access microphone data from Bluetooth Headsets.
> > >
> > >
> > > Previously in bluez4, HFP profile was implemented by bluez daemon and
> > > telephony HFP functionality was provided by either dummy modem, ofono
> > > modem or by Nokia's CSD Maemo modem.
> > >
> > > In bluez5 version was modem code together with implementation of HFP
> > > profile removed. And let implementation of HSP and HFP profiles to
> > > external application.
> > >
> > > Currently HSP profile is implemented in pulseaudio daemon to handle
> > > microphone and Bluetooth speakers. HFP profile is not implemented yet.
> > >
> > >
> > > HSP and HFP profiles use AT modem commands, so its implementation needs
> > > to parse and generates AT commands, plus implement needed state machine
> > > for it.
> > >
> > > And now problem is that last version of HFP profile specification is too
> > > complicated, plus Bluetooth headsets vendors started to inventing and
> > > using of own custom extensions to HFP profile and AT commands.
> > >
> > > Main problem of this "external" implementation outside of bluez is that
> > > only one application can communicate with remote Bluetooth device. It
> > > is application which received needed socket from bluez.
> > >
> > > So in this design if audio daemon (pulseaudio) implements HFP profile
> > > for processing audio, and e.g. power supply application wants to
> > > retrieve battery level from Bluetooth device, it means that audio daemon
> > > needs to implement also battery related functionality.
> > >
> > > It does not make sense to force power supply daemon (upower) to
> > > implement audio routing/encoding/decoding or audio daemon (power supply)
> > > to force implementing battery related operations.
> > >
> > >
> > > For handle this problem I would like to propose a new way how to use and
> > > implement HSP and HFP profiles on Linux.
> > >
> > > Implement a new HSP/HFP daemon (I called it hsphfpd) which register HSP
> > > and HFP profiles in bluez and then exports functionality for all other
> > > specific applications via DBus API (API for audio, power supply, input
> > > layer, telephony functions, vendor extensions, etc...). So it would acts
> > > as proxy daemon between bluez and target applications (pulseaudio,
> > > upower, ofono, ...)
> > >
> > > This would simplify whole HFP usage as applications would not need to
> > > re-implement parsing and processing of AT commands and it would allow
> > > more applications to use HFP profile at one time. And also it means that
> > > audio software does not have to implement telephony stack or power
> > > supply operations.
> > >
> > >
> > > I wrote a document how such DBus API could look like, see here:
> > >
> > > https://github.com/pali/hsphfpd-prototype/raw/prototype/hsphfpd.txt
> > >
> > >
> > > And also I implemented "prototype" implementation to verify that
> > > designed API make sense and can be really implemented. Prototype fully
> > > supports HSP profile in both HS and AG role, plus HFP profile in HF
> > > role. This prototype implementation is available here:
> > >
> > > https://github.com/pali/hsphfpd-prototype
> > >
> > > Some other details are written in README:
> > >
> > > https://github.com/pali/hsphfpd-prototype/raw/prototype/README
> > >
> > >
> > > What do you think about it? Does it make sense to have such design?
> > > Would you accept usage of such hsphfpd daemon, implemented according to
> > > specification, on Linux desktop?
> > >
> > > I would like to hear your opinion if I should continue with this hsphfpd
> > > design, or not.
> > >
> > >
> > > With this design and implementation of hsphfpd is possible to easily fix
> > > pulseaudio issue about power supply properties:
> > >
> > > https://gitlab.freedesktop.org/pulseaudio/pulseaudio/issues/722
> >
> > I quite like this proposal. Avoiding the need to implement the button
> > press, battery level etc. handling separately in PulseAudio, oFono and
> > PipeWire seems like a pretty good justification to me. I assume you're
> > volunteering to maintain this new daemon?
>
> Yes, I can maintain this new daemon.
>
> > It's not clear to me how this would affect the PulseAudio <-> oFono
> > interaction, if at all.
>
> Only one application can register to bluez and "own" bluez socket for
> communication with bluetooth headset. Currently it can be either
> pulseaudio (with native backend in pulseaudio for HSP profile) or ofono
> (with ofono backend in pulseaudio for HFP profile).
>
> So if hsphfpd would be in use, then ofono would not be able to get
> socket from bluez. And it would be needed to add a new "plugin" into
> ofono to use hsphpfd for telephony operations. See hsphfpd.txt file
> (there is diagram of usage).
>
> > When oFono is used, would PulseAudio get the
> > audio socket from oFono or hsphfpd?
>
> Always from hsphfpd.
>
> > What about volume commands, would they go through oFono or would
> > PulseAudio interact with hsphfpd directly?
>
> Internally they are handled by hsphpfd directly and appropriate API is
> exported via DBus for audio application daemon (pulseaudio). For details
> see hsphfpd.txt file, interface org.hsphfpd.AudioTransport.
>
> > I think hsphfpd should be part of bluetoothd, but if that's not
> > possible, then that's not possible.
>
> I do not know if bluez developers are interested in having this code as
> part of bluez project, specially when in bluez4 HFP profile was there
> and in bluez5 was HFP code completely removed.
>
> But it should not matter where hsphpfd is implemented.
>
> > (I don't want to get into a lengthy discussion about programming
> > languages, but I'll just note here that I don't like Perl.)
>
> It is just a prototype. I used Perl just because it is faster for me to
> implement parsing of AT commands needed for prototype purposes.
Broadly, I think this is a good thing to do. My main concern is having this be adequately maintained over a longer period of time.
The other thing to note is that in PipeWire, it is possible for external entities to implement the equivalent of a "sink" or "source" (it's just a node in PipeWire terminology), so in the long run, it might be good to have a single BT audio daemon that manages A2DP and HSP/HFP..
This would also have the benefit of moving codec dependencies out of PulseAudio (which I'll help mitigate in other ways within PulseAudio anyway).
Cheers,
Arun
On Wednesday 04 December 2019 20:45:23 Arun Raghavan wrote:
> Broadly, I think this is a good thing to do. My main concern is having this be adequately maintained over a longer period of time.
I already prepared prototype implementation, so I know how hard is to
write or maintain this new daemon.
Are there any interested people in either implementing, maintaining or
in other way helping with this project?
The main problem in maintaining this HSP/HFP codebase I see that I do
not have enough testing devices. As we know lot of manufactures produce
devices which are not compliant to standards and this needs to be
handled in HSP/HFP codebase.
> The other thing to note is that in PipeWire, it is possible for external entities to implement the equivalent of a "sink" or "source" (it's just a node in PipeWire terminology), so in the long run, it might be good to have a single BT audio daemon that manages A2DP and HSP/HFP..
PipeWire or any other audio server (jack or bluez-alsa) can benefit from
this hsphfpd daemon that it does not have to implement (again) parsing
and interpreting AT commands (as needed for HFP standard).
For PipeWire developers: Do you have any special requirements for this
hsphfpd daemon which are needed for processing of audio?
See proposed DBus interface, specially "Audio Agent hierarchy":
https://github.com/pali/hsphfpd-prototype/raw/prototype/hsphfpd.txt
> This would also have the benefit of moving codec dependencies out of PulseAudio (which I'll help mitigate in other ways within PulseAudio anyway).
hsphfpd just prepares SCO audio socket and then via DBus pass it to
target application (pulseaudio, pipewire, jacks, ...). And it is up to
the audio application to do codec processing. hsphfpd is not going to do
anything with audio samples or codecs. So this does not help directly...
... But I'm planing to try to separate A2DP code from pulseaudio into
some external library which would not depend on pulseaudio. So this
library can help all audio server projects to not re-implement again and
again A2DP (and HSP/HFP) audio processing. This library should do all
needed codec stuff (encoding, decoding, setup of A2DP, ...).
--
Pali Rohár
[email protected]