Hello,
>From a Linux distribution point of view, what part of the stack is
expected (or would you recommend) to trigger the user-space btattach
command, to have Bluetooth enabled automatically at boot?
On an older computer, hot-plugging a Bluetooth USB adapter starts
Bluetooth automatically. On a ThinkPad 8 tablet, a manual btattach
command is currently needed to attach the hardware and load the
firmware for its Broadcom chip.
What do distro maintainers need to adapt to achieve the equivalent of
the USB auto-loading case for btattach-based chipsets, now that
btattach is starting to be included in some Linux distributions (such
as in Debian following this report:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=3D816865 even if only
in experimental right now)?
Is it an already planned evolution on BlueZ side? Reading through the
mailing list, it doesn't seem to be the case but I may have missed a
discussion.
If not, any hints would be really appreciated, and especially your
view on which SW component would be a good candidate to take care of
this initialization task. Thanks a lot.
Regards,
J=C3=A9r=C3=B4me
Hi Arend, Hi Hante,
Do you think you could get some feedback on Broadcom side to the following
question: when is it *not* a good timing to load the Bluetooth firmware via
btattach during the Linux boot on combo Wi-Fi & BT chipsets? Are there some
conditions that a Broadcom combo chip needs to respect to avoid getting in a
stalled Bluetooth state?
I am asking this question because with a BCM43241 rev B5 chip (on a Lenovo
ThinkPad 8 tablet), enabling Bluetooth automatically via scripts during boot
seems quite heavily timing dependent to succeed. The working / non-working
cases are 100% reproducible with Linux 4.6 up to 4.8(rc5) on my setup.
Could you please forward this question internally, if you think some
Broadcom colleagues may have the right knowledge on that topic?
Please find quite a few details below already and earlier in the thread too.
If other inputs would be useful, I would be glad to try to provide them.
> > I've shared my results so far here on StackExchange:
> >
> > http://unix.stackexchange.com/questions/309219/how-to-enable-bluetooth-
> > automatically-at-boot-for-recent-intel-and-broadcom-ch/309263#309263
> > To sum up in a few words:
> > 1/ launching btattach from a udev rule enables Bluetooth but it gets
> > killed.
> > 2/ launching btattach via systemd works, only when adding a few sec
> > delay.
> > 3/ launching the systemd service from 2/ directly via udev, using the
> > exact same trigger conditions as in 1/ ends up with Bluetooth not
> > working...
> >
> > What leaves me really puzzled is the difference between 1/ and 3/, as
> > they
> > launch exactly the same command at almost the same time during the boot
> > process, but they give a different result in a 100% reproducible way.
>
> I am attaching 5 different dmesg outputs to give a sense of the boot
> timing in the different scenarios:
>
> - dmesg.simple-udev-rule corresponds to 1/ -> OK
>
> - dmesg.systemd-standalone-rule-with-1s-delay corresponds to 2/ -> OK
> - dmesg.systemd-standalone-rule-no-delay : 2/ without the 1s delay -> KO
>
> - dmesg.systemd-triggered-by-udev-rule-no-delay corresponds to 3/ -> KO
> - dmesg.systemd-tigggered-by-udev-rule-with-1s-delay -> OK
>
> with the line:
> "bluetooth hci0: firmware: direct-loading firmware brcm/BCM.hcd"
> giving a good indication of when btattach is launched during the sequence.
>
> Maybe these files will help reveal some clues?
An idea I had at first was that perhaps the firmware loading with btattach
was triggered "too early" during the boot sequence somehow.
However, scenario 1 with a pure udev rule (not systemd at all) is working
fine consistently and is the fastest among all the cases I've tried.
The scenarios using systemd work only when adding a delay (1s being enough
in my own experiences) and they trigger the firmware loading later than when
using the pure udev rule.
These are the scenarios in between time-wise that lead to the stalled
situations (reproduced using systemd but without adding the 1s delay hack).
btattach is still running in the list of processes, dmesg is showing the BT
firmware loading properly with no errors, but Bluetooth is simply not
working at the end of the boot. bluetoothctl blocks when launched and the
Gnome BT settings page remain grayed out until I kill / relaunch btattach.
I'm making the assumption that something is going wrong when the BT firmware
is loading at the same time than other on-going events on the combo chipet
(on the Wi-Fi side?). Or maybe the "deadlock" is linked to other parts of
the Bluetooth stack still initializing though.
I am out of ideas to deep dive more just by myself, so any hints would be
really appreciated to help solve this weird but annoying issue. Thanks a
lot.
Regards,
Jérôme
Hello again,
> To sum up in a few words:
> 1/ launching btattach from a udev rule enables Bluetooth but it gets
> killed.
> 2/ launching btattach via systemd works, only when adding a few sec delay.
> 3/ launching the systemd service from 2/ directly via udev, using the
> exact same trigger conditions as in 1/ ends up with Bluetooth not
> working...
>
> What leaves me really puzzled is the difference between 1/ and 3/, as they
> launch exactly the same command at almost the same time during the boot
> process, but they give a different result in a 100% reproducible way.
I am attaching 5 different dmesg outputs to give a sense of the boot timing
in the different scenarios:
- dmesg.simple-udev-rule corresponds to 1/ -> OK
- dmesg.systemd-standalone-rule-with-1s-delay corresponds to 2/ -> OK
- dmesg.systemd-standalone-rule-no-delay : 2/ without the 1s delay -> KO
- dmesg.systemd-triggered-by-udev-rule-no-delay corresponds to 3/ -> KO
- dmesg.systemd-tigggered-by-udev-rule-with-1s-delay -> OK
with the line:
"bluetooth hci0: firmware: direct-loading firmware brcm/BCM.hcd"
giving a good indication of when btattach is launched during the sequence.
Maybe these files will help reveal some clues?
Regards,
Jérôme
Hi Marcel,
> > [...] For that part, I will continue my investigations off the list as
> > I've found blogs / articles explaining several solutions and how to do
> > it using systemd for instance.
> >
I've shared my results so far here on StackExchange:
http://unix.stackexchange.com/questions/309219/how-to-enable-bluetooth-autom
atically-at-boot-for-recent-intel-and-broadcom-ch/309263#309263
To sum up in a few words:
1/ launching btattach from a udev rule enables Bluetooth but it gets killed.
2/ launching btattach via systemd works, only when adding a few sec delay.
3/ launching the systemd service from 2/ directly via udev, using the exact
same trigger conditions as in 1/ ends up with Bluetooth not working...
What leaves me really puzzled is the difference between 1/ and 3/, as they
launch exactly the same command at almost the same time during the boot
process, but they give a different result in a 100% reproducible way.
In both cases I can see btattach launched (when checking with $ ps auxw) and
the firmware properly loaded (from $ dmesg) but in scenario 3/ Bluetooth is
simply stuck. I need to kill the existing btattach and trigger a new one to
have Bluetooth working.
That's what gave me the idea to try and add a delay leading to option 2/. I
wasn't really expecting this to work as I had the feeling that launching
btattach from systemd wasn't working for other reasons but that wasn't the
case. I am using this setup 2/ currently and it is doing the job quiet well.
I just don't like this ugly hack and would like to better understand what's
going on, to replace the random delay with the real needed pre-condition to
load btattach "at the right time". So far, my udev rule is simply looking
for the "BCM2E55:00" device event. I've looked at the btattach source code
quickly but I didn't find any clear reference saying what's needed for
btattach to work during the boot sequence.
Do you have any clues or suggestions that I could try on my system?
> we could add a --daemon mode into btattach that can be just started at
> boot and then monitors udev to auto-attach devices.
Coming back to your proposition, if I interpret it correctly, I don't think
that Linux distributions would like the idea of launching btattach as a
daemon (when and in which case?) to then auto-attach devices via udev later
on if needed (on which condition? The above one? With the same timing
issue?). That would add one more dependency. It seems backwards to me, as
udev or systemd monitoring are designed to be the trigger and that's what
I've assessed and got working based on your earlier suggestion.
Once the above boot sequence order / timing dependency is solved, I have the
feeling that option 3/ will be the way to go.
btattach doesn't seem to need such a modification as it is working perfectly
fine as-is. It just needs to be properly integrated within the existing
distro stack. And with serial bus enumeration in-kernel in the future, all
of this will disappear of course :-)
Regards,
Jérôme
Hi Jerome,
>>>> Until then, you need an userspace part that triggers btattach with the
>>>> right hardware id on the right /dev/ttySx device node as soon as it
>>>> becomes available.
>>>
>>> Maybe my mistake is that I've interpreted your sentence: "as soon as
>>> *it* becomes available" to assume that "it" was referring to the
>>> /dev/ttyS1 device node. I'll try to see if I can create another udev
>>> rule that would trigger based on the actual BT chipset becoming
>>> available, and not based on /dev/ttyS1 , but I'm still looking for how
>>> to do it with dev.
>>
>> maybe the RFKILL switch part gets discovered later than the TTY. I don't
>> have a platform to verify, but just at few printk into the probe routine
>> and see if its finds the according GPIOs for power control.
>
> I've continued my investigation first by trying to create another udev rule
> detecting the actual BCM4324(1) rev B5 Bluetooth chipset this time.
>
> And I'm happy to share that it works finally!! as I have Bluetooth working
> at boot.
>
> This is not over though as I've just discovered this via "man udev" :-(
>
> Starting daemons or other long-running processes is not appropriate
> for udev; the forked processes, detached or not, will be
> unconditionally *killed* after the event handling has finished.
>
> and indeed btattach gets killed later on. For that part, I will continue my
> investigations off the list as I've found blogs / articles explaining
> several solutions and how to do it using systemd for instance.
>
> Once I'm done, would you be interested to see such a rule integrated into
> the BlueZ project directly? Or do you consider this "last mile" integration
> work to be more the responsibility of the various Linux distributions?
we could add a --daemon mode into btattach that can be just started at boot and then monitors udev to auto-attach devices.
Regards
Marcel
Hi Marcel, hi everyone,
> > > Until then, you need an userspace part that triggers btattach with the
> > > right hardware id on the right /dev/ttySx device node as soon as it
> > > becomes available.
> >
> > Maybe my mistake is that I've interpreted your sentence: "as soon as
> > *it* becomes available" to assume that "it" was referring to the
> > /dev/ttyS1 device node. I'll try to see if I can create another udev
> > rule that would trigger based on the actual BT chipset becoming
> > available, and not based on /dev/ttyS1 , but I'm still looking for how
> > to do it with dev.
>
> maybe the RFKILL switch part gets discovered later than the TTY. I don't
> have a platform to verify, but just at few printk into the probe routine
> and see if its finds the according GPIOs for power control.
I've continued my investigation first by trying to create another udev rule
detecting the actual BCM4324(1) rev B5 Bluetooth chipset this time.
And I'm happy to share that it works finally!! as I have Bluetooth working
at boot.
This is not over though as I've just discovered this via "man udev" :-(
Starting daemons or other long-running processes is not appropriate
for udev; the forked processes, detached or not, will be
unconditionally *killed* after the event handling has finished.
and indeed btattach gets killed later on. For that part, I will continue my
investigations off the list as I've found blogs / articles explaining
several solutions and how to do it using systemd for instance.
Once I'm done, would you be interested to see such a rule integrated into
the BlueZ project directly? Or do you consider this "last mile" integration
work to be more the responsibility of the various Linux distributions?
Thanks again for all your help and your great work on the Bluetooth stack.
Regards,
Jérôme
P.S. For a ThinkPad 8 tablet, a hardcoded version of the basic working rule
looks like this, in case that can be useful for someone else:
$ cat /etc/udev/rules.d/98-bluetooth-attach-broadcom.rule
KERNEL=="BCM2E55:00", RUN+="/usr/bin/btattach --bredr /dev/ttyS1 -P bcm"
Hi Jerome,
>> So that means udev rules. [...] If you are unlucky you need a DT overlay
>> or hardcode it.
>
> So I've followed your suggestion and I've created a udev rule (on a Debian
> system). Here is what it looks like for the moment, choosing first the
> simpler hardcoding option:
>
> jdb@thinkpad8:~$ cat /etc/udev/rules.d/99-bluetooth-attach-broadcom.rules
> SUBSYSTEM=="tty", KERNEL=="ttyS1", KERNELS=="80860F0A:00",
> RUN+="/usr/bin/btattach --bredr /dev/ttyS1 -P bcm"
>
> Rebooting the tablet, good news, the rule seems to properly execute as I can
> find btattach in the list of running processes:
>
> jdb@thinkpad8:~$ ps auxw | grep btattach
> root 1155 0.0 0.0 6372 728 ? S 22:04 0:00
> /usr/bin/btattach --bredr /dev/ttyS1 -P bcm
>
> and I can see the firmware loading happening quite early in the boot process
>
> jdb@thinkpad8:~$ dmesg | grep "bluetooth.* firmware"
> [ 5.528293] bluetooth hci0: firmware: direct-loading firmware
> brcm/BCM.hcd
>
> However to my surprise Bluetooth is not enabled properly or at all, the
> Bluetooth panel in the Gnome settings remains grayed out and bluetoothctl
> doesn't detect a Controller. At some later points, I even see the btattach
> process disappear by itself.
>
> That's just an uneducated assumption but it seems that somehow I am hitting
> a timing condition issue and maybe it ends up trying to attach too early
> (compared to what precisely? I have no idea). I'm saying this because if I
> launch the btattach command manually at any later timee, it works fine.
>
> Do you have any guess of what could be the issue I'm hitting here? Any
> logs/commands in particular that would be useful?
>
>
>> Until then, you need an userspace part that triggers btattach with the
>> right hardware id on the right /dev/ttySx device node as soon as it
>> becomes available.
>
> Maybe my mistake is that I've interpreted your sentence: "as soon as *it*
> becomes available" to assume that "it" was referring to the /dev/ttyS1
> device node. I'll try to see if I can create another udev rule that would
> trigger based on the actual BT chipset becoming available, and not based on
> /dev/ttyS1 , but I'm still looking for how to do it with dev.
maybe the RFKILL switch part gets discovered later than the TTY. I don't have a platform to verify, but just at few printk into the probe routine and see if its finds the according GPIOs for power control.
Regards
Marcel
Hi Marcel,
> So that means udev rules. [...] If you are unlucky you need a DT overlay
> or hardcode it.
So I've followed your suggestion and I've created a udev rule (on a Debian
system). Here is what it looks like for the moment, choosing first the
simpler hardcoding option:
jdb@thinkpad8:~$ cat /etc/udev/rules.d/99-bluetooth-attach-broadcom.rules
SUBSYSTEM=="tty", KERNEL=="ttyS1", KERNELS=="80860F0A:00",
RUN+="/usr/bin/btattach --bredr /dev/ttyS1 -P bcm"
Rebooting the tablet, good news, the rule seems to properly execute as I can
find btattach in the list of running processes:
jdb@thinkpad8:~$ ps auxw | grep btattach
root 1155 0.0 0.0 6372 728 ? S 22:04 0:00
/usr/bin/btattach --bredr /dev/ttyS1 -P bcm
and I can see the firmware loading happening quite early in the boot process
jdb@thinkpad8:~$ dmesg | grep "bluetooth.* firmware"
[ 5.528293] bluetooth hci0: firmware: direct-loading firmware
brcm/BCM.hcd
However to my surprise Bluetooth is not enabled properly or at all, the
Bluetooth panel in the Gnome settings remains grayed out and bluetoothctl
doesn't detect a Controller. At some later points, I even see the btattach
process disappear by itself.
That's just an uneducated assumption but it seems that somehow I am hitting
a timing condition issue and maybe it ends up trying to attach too early
(compared to what precisely? I have no idea). I'm saying this because if I
launch the btattach command manually at any later timee, it works fine.
Do you have any guess of what could be the issue I'm hitting here? Any
logs/commands in particular that would be useful?
> Until then, you need an userspace part that triggers btattach with the
> right hardware id on the right /dev/ttySx device node as soon as it
> becomes available.
Maybe my mistake is that I've interpreted your sentence: "as soon as *it*
becomes available" to assume that "it" was referring to the /dev/ttyS1
device node. I'll try to see if I can create another udev rule that would
trigger based on the actual BT chipset becoming available, and not based on
/dev/ttyS1 , but I'm still looking for how to do it with udev.
Regards,
Jérôme
P.S. Here is the output of the udevadm command I've used to help me create
the ttyS1-based rule in the first place:
jdb@thinkpad8:~$ udevadm info -a -n /dev/ttyS1
Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.
looking at device '/devices/platform/80860F0A:00/tty/ttyS1':
KERNEL=="ttyS1"
SUBSYSTEM=="tty"
DRIVER==""
looking at parent device '/devices/platform/80860F0A:00':
KERNELS=="80860F0A:00"
SUBSYSTEMS=="platform"
DRIVERS=="dw-apb-uart"
ATTRS{driver_override}=="(null)"
looking at parent device '/devices/platform':
KERNELS=="platform"
SUBSYSTEMS==""
DRIVERS==""
Hi Marcel,
>>> the real evolution would be that we get a serial bus (as discussed a
>>> few weeks ago) which then the UART kernel drivers can enumerate its
>>> devices on.
>>
>> If I understood correctly, you would like to stop exposing /dev/ttyX
>> for Bluetooth UART drivers in the long run. Would it mean that the
>> user-space btattach will then be deprecated and the firmware loading
>> will be moved into the kernel itself? Or another approach instead?
>
> firmware loading is already in the kernel. At least for Broadcom and
> Intel based devices.
I wanted to mean: moving the firmware loading *triggering* all in kernel,
with the future UART device enumeration. Or do you still see a benefit to
keep the "attach" step in user space, to cover some specific use cases
maybe?
I'll keep following the serial bus discussions, this is a every interesting
evolution.
Thanks again,
Jérôme
Hi Marcel,
> >> the real evolution would be that we get a serial bus (as discussed a
> >> few weeks ago) which then the UART kernel drivers can enumerate its
> >> devices on.
> >
> > If I understood correctly, you would like to stop exposing /dev/ttyX
> > for Bluetooth UART drivers in the long run. Would it mean that the
> > user-space btattach will then be deprecated and the firmware loading
> > will be moved into the kernel itself? Or another approach instead?
>
> firmware loading is already in the kernel. At least for Broadcom and
> Intel based devices.
I wanted to mean: moving the firmware loading *triggering* all in kernel,
with the future UART device enumeration. Or do you still see a benefit to
keep the "attach" step in user space, to cover some specific use cases
maybe?
I'll keep following the serial bus discussions, this is a every interesting
evolution.
Thanks again,
Jérôme
Hi Jerome,
>> the real evolution would be that we get a serial bus (as discussed a
>> few weeks ago) which then the UART kernel drivers can enumerate its
>> devices on.
>
> I've read this discussion, thanks for the pointer, and I've seen the
> latest patch set proposal from Rob Herring reusing the existing serio
> bus.
>
> If I understood correctly, you would like to stop exposing /dev/ttyX
> for Bluetooth UART drivers in the long run. Would it mean that the
> user-space btattach will then be deprecated and the firmware loading
> will be moved into the kernel itself? Or another approach instead?
firmware loading is already in the kernel. At least for Broadcom and Intel based devices.
>> Until then, you need an userspace part that triggers btattach with
>> the right hardware id on the right /dev/ttySx device node as soon as
>> it becomes available. So that means udev rules.
>
> Great, I will investigate the udev rules direction.
>
>> However the problem is and always has been to figure out what
>> hardware is behind what /dev/ttySx. If you are lucky it is part of
>> ACPI tables or DT. If you are unlucky you need a DT overlay or
>> hardcode it.
>
> I faced that precise limitation when I was trying to find out what was
> the chipset ACPI ID on the ThinkPad 8 tablet, to guess which btattach
> parameter to use. And I don't remember finding it in the ACPI tables :(
>
> I fear I will have to hardcode it to get started, and then investigate
> the usage of a DT overlay in a second step.
I do not have the ACPI tables available, but the kernel drivers will map the ACPI ID. So if you look into hci_bcm.c or hci_intel.c you should find the ones. Each chip should have a new ACPI ID on Intel platforms.
Regards
Marcel
Hi Marcel,
> the real evolution would be that we get a serial bus (as discussed a
> few weeks ago) which then the UART kernel drivers can enumerate its
> devices on.
I've read this discussion, thanks for the pointer, and I've seen the
latest patch set proposal from Rob Herring reusing the existing serio
bus.
If I understood correctly, you would like to stop exposing /dev/ttyX
for Bluetooth UART drivers in the long run. Would it mean that the
user-space btattach will then be deprecated and the firmware loading
will be moved into the kernel itself? Or another approach instead?
> Until then, you need an userspace part that triggers btattach with
> the right hardware id on the right /dev/ttySx device node as soon as
> it becomes available. So that means udev rules.
Great, I will investigate the udev rules direction.
> However the problem is and always has been to figure out what
> hardware is behind what /dev/ttySx. If you are lucky it is part of
> ACPI tables or DT. If you are unlucky you need a DT overlay or
> hardcode it.
I faced that precise limitation when I was trying to find out what was
the chipset ACPI ID on the ThinkPad 8 tablet, to guess which btattach
parameter to use. And I don't remember finding it in the ACPI tables :(
I fear I will have to hardcode it to get started, and then investigate
the usage of a DT overlay in a second step.
Regards,
Jérôme
Hi Jerome,
> From a Linux distribution point of view, what part of the stack is
> expected (or would you recommend) to trigger the user-space btattach
> command, to have Bluetooth enabled automatically at boot?
>
> On an older computer, hot-plugging a Bluetooth USB adapter starts
> Bluetooth automatically. On a ThinkPad 8 tablet, a manual btattach
> command is currently needed to attach the hardware and load the
> firmware for its Broadcom chip.
>
> What do distro maintainers need to adapt to achieve the equivalent of
> the USB auto-loading case for btattach-based chipsets, now that
> btattach is starting to be included in some Linux distributions (such
> as in Debian following this report:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=816865 even if only
> in experimental right now)?
>
> Is it an already planned evolution on BlueZ side? Reading through the
> mailing list, it doesn't seem to be the case but I may have missed a
> discussion.
>
> If not, any hints would be really appreciated, and especially your
> view on which SW component would be a good candidate to take care of
> this initialization task. Thanks a lot.
the real evolution would be that we get a serial bus (as discussed a few weeks ago) which then the UART kernel drivers can enumerate its devices on.
Until then, you need an userspace part that triggers btattach with the right hardware id on the right /dev/ttySx device node as soon as it becomes available. So that means udev rules. However the problem is and always has been to figure out what hardware is behind what /dev/ttySx. If you are lucky it is part of ACPI tables or DT. If you are unlucky you need a DT overlay or hardcode it.
The pain point is really that the kernel has all the information we need. Except people kept insisting on creating /dev/ttySx devices and let someone else deal with the fact that it is an actual Bluetooth chip behind it.
That said, it has gotten better when we moved power control into the Intel and Broadcom drivers. Which means you no longer need to fiddle with RFKILL to get your GPIOs set up. That was a huge mess as well when people though we just expose the power control of the TTY as RFKILL switches and leave userspace totally in the dark and jump through 10 hoops before getting things working.
Regards
Marcel