Since kernel 5.17, some users are seeing problems connecting to
bluetooth devices, the problem doesn't happen with 5.16 series
kernels.
So far two different devices are affected, although it's possible it's
not even device related, but could be the interface between kernel and
user space.
Case 1:
Bus 001 Device 003: ID 0cf3:e301 Qualcomm Atheros Communications
Not seeing anything that stands out in dmesg, bluetoothd -d shows
various curious messages like
bluetoothd[5889]: src/service.c:change_state() 0x7f33becdebc0: device
94:65:2D:DC:F4:A7 profile avrcp-controller state changed: disconnected
-> unavailable (0)
and
bluetoothd[5889]: src/service.c:change_state() 0x7f33becd91e0: device
94:65:2D:DC:F4:A7 profile Hands-Free Voice gateway state changed:
unavailable -> disconnected (0)
Case 2:
Bus 001 Device 005: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
Jefferson Peak (JfP)
[ 5.923865] kernel: Bluetooth: hci0: Found device firmware:
intel/ibt-17-16-1.sfi
[ 5.930536] kernel: Bluetooth: hci0: Boot Address: 0x40800
[ 5.930539] kernel: Bluetooth: hci0: Firmware Version: 60-44.21
...
[16102.640651] Bluetooth: hci0: unexpected event 0xff length: 5 > 0
[16102.640802] Bluetooth: hci0: Waiting for device to boot
[16102.654712] Bluetooth: hci0: Device booted in 13698 usecs
[16102.654753] Bluetooth: hci0: unexpected event 0xff length: 7 > 0
[16102.655907] Bluetooth: hci0: Found Intel DDC parameters:
intel/ibt-17-16-1.ddc
[16102.657821] Bluetooth: hci0: Applying Intel DDC parameters completed
[16102.658829] Bluetooth: hci0: Firmware revision 0.1 build 60 week 44 2021
[17422.210558] Bluetooth: hci0: command 0x0c03 tx timeout
[17430.338412] Bluetooth: hci0: HCI reset during shutdown failed
[15975.194153] bluetoothd[42895]:
src/adapter.c:device_found_callback() hci0 addr 75:0B:19:0C:47:F6,
rssi -82 flags 0x0000 eir_len 17
[15975.194412] bluetoothd[42895]:
src/adv_monitor.c:btd_adv_monitor_offload_supported() Manager is NULL,
get offload support failed
[15975.194510] bluetoothd[42895]: src/device.c:device_set_legacy() legacy 0
[15975.194599] bluetoothd[42895]:
src/device.c:device_set_rssi_with_delta() rssi -82 delta 1
[15975.194661] bluetoothd[42895]: src/device.c:device_set_flags() flags 26
There are many many of the "Manager is NULL, get offload support
failed" messages in this case. I'm not entirely sure the root causes
are the same for the two cases.
Downstream bug has attachments (dmesg, btmon, lsbsb, journalctl
snippet with bluetoothd in debug, for both)
https://bugzilla.redhat.com/show_bug.cgi?id=2053283
Thanks,
--
Chris Murphy
On Tue, Feb 15, 2022 at 8:29 AM Chris Murphy <[email protected]> wrote:
>
> On Thu, Feb 10, 2022 at 6:44 PM Chris Murphy <[email protected]> wrote:
>
> > Case 2:
> >
> > Bus 001 Device 005: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> > Jefferson Peak (JfP)
>
> Comparing 5.16.9 (working) and 5.17.0-rc7 (non-working) on this
> Thinkpad X1 Carbon, I'm seeing two notable differences:
>
> Multiple messages like this:
> [ 15.731405] kernel: Bluetooth: hci0: unexpected event 0xff length: 5 > 0
This matches with a complaint I found here
https://lore.kernel.org/linux-bluetooth/[email protected]/
about this patch
https://lore.kernel.org/linux-bluetooth/[email protected]/
--
Chris Murphy
On Thu, Feb 10, 2022 at 6:44 PM Chris Murphy <[email protected]> wrote:
> Case 2:
>
> Bus 001 Device 005: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> Jefferson Peak (JfP)
Comparing 5.16.9 (working) and 5.17.0-rc7 (non-working) on this
Thinkpad X1 Carbon, I'm seeing two notable differences:
Multiple messages like this:
[ 15.731405] kernel: Bluetooth: hci0: unexpected event 0xff length: 5 > 0
And a lockdep warning:
[ 26.658252] kernel: Chain exists of:
sk_lock-AF_BLUETOOTH-BTPROTO_RFCOMM -->
rfcomm_mutex --> &d->lock
5.17.0-0.rc4.96.fc36.x86_64+debug
[ 11.210423] systemd[1]: unit_file_build_name_map: alias:
/etc/systemd/system/dbus-org.bluez.service → bluetooth.service
[ 11.222138] systemd[1]: unit_file_build_name_map: normal unit file:
/usr/lib/systemd/system/bluetooth.service
[ 11.222142] systemd[1]: unit_file_build_name_map: normal unit file:
/usr/lib/systemd/system/bluetooth.target
[ 13.403381] kernel: thinkpad_acpi: rfkill switch
tpacpi_bluetooth_sw: radio is unblocked
[ 13.723178] kernel: Bluetooth: Core ver 2.22
[ 13.726055] kernel: NET: Registered PF_BLUETOOTH protocol family
[ 13.728463] kernel: Bluetooth: HCI device and connection manager initialized
[ 13.730914] kernel: Bluetooth: HCI socket layer initialized
[ 13.734079] kernel: Bluetooth: L2CAP socket layer initialized
[ 13.736746] kernel: Bluetooth: SCO socket layer initialized
[ 14.017250] kernel: Bluetooth: hci0: Bootloader revision 0.1 build
42 week 52 2015
[ 14.023991] kernel: Bluetooth: hci0: Device revision is 2
[ 14.026848] kernel: Bluetooth: hci0: Secure boot is enabled
[ 14.029674] kernel: Bluetooth: hci0: OTP lock is enabled
[ 14.032315] kernel: Bluetooth: hci0: API lock is enabled
[ 14.035066] kernel: Bluetooth: hci0: Debug lock is disabled
[ 14.037744] kernel: Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[ 14.091618] kernel: Bluetooth: hci0: Found device firmware:
intel/ibt-17-16-1.sfi
[ 14.093532] kernel: Bluetooth: hci0: Boot Address: 0x40800
[ 14.095001] kernel: Bluetooth: hci0: Firmware Version: 86-46.21
[ 14.538285] kernel: Modules linked in: acpi_cpufreq(-)
snd_hda_codec_hdmi intel_tcc_cooling x86_pkg_temp_thermal
intel_powerclamp snd_hda_codec_realtek coretemp snd_hda_codec_generic
kvm_intel snd_sof_pci_intel_cnl snd_sof_intel_hda_common kvm
soundwire_intel soundwire_generic_allocation soundwire_cadence
snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof soundwire_bus
irqbypass rapl snd_soc_skl intel_cstate snd_soc_hdac_hda intel_uncore
snd_hda_ext_core snd_soc_sst_ipc snd_soc_sst_dsp
snd_soc_acpi_intel_match snd_soc_acpi snd_soc_core snd_compress
ac97_bus iwlmvm snd_pcm_dmaengine snd_hda_intel snd_intel_dspcfg
snd_intel_sdw_acpi snd_hda_codec mac80211 snd_hda_core snd_hwdep
snd_seq snd_seq_device btusb think_lmi(+) snd_pcm
firmware_attributes_class libarc4 wmi_bmof intel_wmi_thunderbolt btrtl
i2c_i801 snd_timer uvcvideo i2c_smbus btbcm btintel videobuf2_vmalloc
videobuf2_memops iwlwifi btmtk videobuf2_v4l2 videobuf2_common
thunderbolt bluetooth videodev joydev cfg80211
[ 15.730572] kernel: Bluetooth: hci0: Waiting for firmware download
to complete
[ 15.731394] kernel: Bluetooth: hci0: Firmware loaded in 1599609 usecs
[ 15.731405] kernel: Bluetooth: hci0: unexpected event 0xff length: 5 > 0
[ 15.732383] kernel: Bluetooth: hci0: Waiting for device to boot
[ 15.745966] kernel: Bluetooth: hci0: unexpected event 0xff length: 7 > 0
[ 15.745981] kernel: Bluetooth: hci0: Device booted in 13808 usecs
[ 15.754072] kernel: Bluetooth: hci0: Found Intel DDC parameters:
intel/ibt-17-16-1.ddc
[ 15.761008] kernel: Bluetooth: hci0: Applying Intel DDC parameters completed
[ 15.761991] kernel: Bluetooth: hci0: Firmware revision 0.1 build 86
week 46 2021
[ 16.172456] kernel: Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[ 16.172460] kernel: Bluetooth: BNEP filters: protocol multicast
[ 16.172471] kernel: Bluetooth: BNEP socket layer initialized
[ 21.136887] kernel: Bluetooth: RFCOMM TTY layer initialized
[ 21.136911] kernel: Bluetooth: RFCOMM socket layer initialized
[ 21.137263] kernel: Bluetooth: RFCOMM ver 1.11
[ 26.492364] kernel: Bluetooth: hci0: unexpected event 0xff length: 4 > 0
[ 26.657785] kernel:
[ 26.657800] kernel: ======================================================
[ 26.657806] kernel: WARNING: possible circular locking dependency detected
[ 26.657812] kernel: 5.17.0-0.rc4.96.fc36.x86_64+debug #1 Tainted: G
W --------- ---
[ 26.657820] kernel: ------------------------------------------------------
[ 26.657824] kernel: krfcommd/1391 is trying to acquire lock:
[ 26.657831] kernel: ffff8d697a534940
(sk_lock-AF_BLUETOOTH-BTPROTO_RFCOMM){+.+.}-{0:0}, at:
rfcomm_sk_state_change+0x4d/0x110 [rfcomm]
[ 26.657879] kernel:
but task is already holding lock:
[ 26.657884] kernel: ffff8d695be69540 (&d->lock){+.+.}-{3:3}, at:
__rfcomm_dlc_close+0x8b/0x230 [rfcomm]
[ 26.657920] kernel:
which lock already depends on the new lock.
[ 26.657924] kernel:
the existing dependency chain (in reverse order) is:
[ 26.657929] kernel:
-> #2 (&d->lock){+.+.}-{3:3}:
[ 26.657945] kernel: __mutex_lock+0x93/0x7d0
[ 26.657958] kernel: __rfcomm_dlc_close+0x8b/0x230 [rfcomm]
[ 26.657985] kernel: rfcomm_session_close+0x42/0x90 [rfcomm]
[ 26.658012] kernel: rfcomm_run+0x61a/0x1860 [rfcomm]
[ 26.658038] kernel: kthread+0xf2/0x120
[ 26.658049] kernel: ret_from_fork+0x1f/0x30
[ 26.658059] kernel:
-> #1 (rfcomm_mutex){+.+.}-{3:3}:
[ 26.658069] kernel: __mutex_lock+0x93/0x7d0
[ 26.658075] kernel: rfcomm_dlc_open+0x30/0x340 [rfcomm]
[ 26.658090] kernel: rfcomm_sock_connect+0xd4/0x130 [rfcomm]
[ 26.658106] kernel: __sys_connect+0x8c/0xb0
[ 26.658116] kernel: __x64_sys_connect+0x14/0x20
[ 26.658125] kernel: do_syscall_64+0x37/0x80
[ 26.658134] kernel: entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 26.658143] kernel:
-> #0 (sk_lock-AF_BLUETOOTH-BTPROTO_RFCOMM){+.+.}-{0:0}:
[ 26.658152] kernel: __lock_acquire+0x12b5/0x1ec0
[ 26.658161] kernel: lock_acquire+0xd0/0x2d0
[ 26.658167] kernel: lock_sock_nested+0x2e/0x80
[ 26.658174] kernel: rfcomm_sk_state_change+0x4d/0x110 [rfcomm]
[ 26.658189] kernel: __rfcomm_dlc_close+0xa4/0x230 [rfcomm]
[ 26.658204] kernel: rfcomm_session_close+0x42/0x90 [rfcomm]
[ 26.658218] kernel: rfcomm_run+0x61a/0x1860 [rfcomm]
[ 26.658233] kernel: kthread+0xf2/0x120
[ 26.658239] kernel: ret_from_fork+0x1f/0x30
[ 26.658248] kernel:
other info that might help us debug this:
[ 26.658252] kernel: Chain exists of:
sk_lock-AF_BLUETOOTH-BTPROTO_RFCOMM -->
rfcomm_mutex --> &d->lock
[ 26.658264] kernel: Possible unsafe locking scenario:
[ 26.658267] kernel: CPU0 CPU1
[ 26.658270] kernel: ---- ----
[ 26.658272] kernel: lock(&d->lock);
[ 26.658278] kernel: lock(rfcomm_mutex);
[ 26.658283] kernel: lock(&d->lock);
[ 26.658289] kernel: lock(sk_lock-AF_BLUETOOTH-BTPROTO_RFCOMM);
[ 26.658294] kernel:
*** DEADLOCK ***
[ 26.658297] kernel: 2 locks held by krfcommd/1391:
[ 26.658302] kernel: #0: ffffffffc162b130
(rfcomm_mutex){+.+.}-{3:3}, at: rfcomm_run+0x135/0x1860 [rfcomm]
[ 26.658327] kernel: #1: ffff8d695be69540 (&d->lock){+.+.}-{3:3},
at: __rfcomm_dlc_close+0x8b/0x230 [rfcomm]
[ 26.658350] kernel:
stack backtrace:
[ 26.658354] kernel: CPU: 2 PID: 1391 Comm: krfcommd Tainted: G
W --------- --- 5.17.0-0.rc4.96.fc36.x86_64+debug #1
[ 26.658363] kernel: Hardware name: LENOVO 20QDS3E200/20QDS3E200,
BIOS N2HET66W (1.49 ) 11/10/2021
[ 26.658368] kernel: Call Trace:
[ 26.658373] kernel: <TASK>
[ 26.658380] kernel: dump_stack_lvl+0x5e/0x77
[ 26.658392] kernel: check_noncircular+0xdc/0x100
[ 26.658400] kernel: ? asm_sysvec_apic_timer_interrupt+0x12/0x20
[ 26.658416] kernel: __lock_acquire+0x12b5/0x1ec0
[ 26.658429] kernel: lock_acquire+0xd0/0x2d0
[ 26.658438] kernel: ? rfcomm_sk_state_change+0x4d/0x110 [rfcomm]
[ 26.658455] kernel: ? __mutex_lock+0xe7/0x7d0
[ 26.658463] kernel: ? __rfcomm_dlc_close+0x8b/0x230 [rfcomm]
[ 26.658480] kernel: ? rfcomm_check_accept+0xa0/0xa0 [rfcomm]
[ 26.658495] kernel: lock_sock_nested+0x2e/0x80
[ 26.658503] kernel: ? rfcomm_sk_state_change+0x4d/0x110 [rfcomm]
[ 26.658518] kernel: rfcomm_sk_state_change+0x4d/0x110 [rfcomm]
[ 26.658535] kernel: __rfcomm_dlc_close+0xa4/0x230 [rfcomm]
[ 26.658552] kernel: rfcomm_session_close+0x42/0x90 [rfcomm]
[ 26.658569] kernel: rfcomm_run+0x61a/0x1860 [rfcomm]
[ 26.658584] kernel: ? sched_clock_cpu+0xb/0xc0
[ 26.658598] kernel: ? __init_waitqueue_head+0x60/0x60
[ 26.658614] kernel: ? _raw_spin_unlock_irqrestore+0x41/0x60
[ 26.658625] kernel: ? rfcomm_check_accept+0xa0/0xa0 [rfcomm]
[ 26.658640] kernel: kthread+0xf2/0x120
[ 26.658647] kernel: ? kthread_complete_and_exit+0x20/0x20
[ 26.658657] kernel: ret_from_fork+0x1f/0x30
[ 26.658675] kernel: </TASK>
[ 167.235571] kernel: Bluetooth: hci0: unexpected event 0xff length: 4 > 0
[ 190.233772] kernel: Bluetooth: hci0: unexpected event 0xff length: 4 > 0
[ 190.433812] kernel: Bluetooth: hci0: Opcode 0x 401 failed: -16
[ 194.234441] kernel: Bluetooth: hci0: unexpected event 0xff length: 4 > 0
[ 199.234214] kernel: Bluetooth: hci0: unexpected event 0xff length: 4 > 0
--
Chris Murphy
On Tue, Feb 15, 2022 at 8:38 AM Chris Murphy <[email protected]> wrote:
>
> On Tue, Feb 15, 2022 at 8:29 AM Chris Murphy <[email protected]> wrote:
> >
> > On Thu, Feb 10, 2022 at 6:44 PM Chris Murphy <[email protected]> wrote:
> >
> > > Case 2:
> > >
> > > Bus 001 Device 005: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> > > Jefferson Peak (JfP)
> >
> > Comparing 5.16.9 (working) and 5.17.0-rc7 (non-working) on this
> > Thinkpad X1 Carbon, I'm seeing two notable differences:
> >
> > Multiple messages like this:
> > [ 15.731405] kernel: Bluetooth: hci0: unexpected event 0xff length: 5 > 0
btmon is here:
https://bugzilla.redhat.com/attachment.cgi?id=1860485
>
> This matches with a complaint I found here
> https://lore.kernel.org/linux-bluetooth/[email protected]/
> about this patch
> https://lore.kernel.org/linux-bluetooth/[email protected]/
>
>
>
> --
> Chris Murphy
--
Chris Murphy
On Wed, Feb 16, 2022 at 4:49 PM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Chris,
>
> On Tue, Feb 15, 2022 at 10:40 PM Chris Murphy <[email protected]> wrote:
> >
> > On Tue, Feb 15, 2022 at 8:38 AM Chris Murphy <[email protected]> wrote:
> > >
> > > On Tue, Feb 15, 2022 at 8:29 AM Chris Murphy <[email protected]> wrote:
> > > >
> > > > On Thu, Feb 10, 2022 at 6:44 PM Chris Murphy <[email protected]> wrote:
> > > >
> > > > > Case 2:
> > > > >
> > > > > Bus 001 Device 005: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> > > > > Jefferson Peak (JfP)
> > > >
> > > > Comparing 5.16.9 (working) and 5.17.0-rc7 (non-working) on this
> > > > Thinkpad X1 Carbon, I'm seeing two notable differences:
> > > >
> > > > Multiple messages like this:
> > > > [ 15.731405] kernel: Bluetooth: hci0: unexpected event 0xff length: 5 > 0
> >
> >
> > btmon is here:
> > https://bugzilla.redhat.com/attachment.cgi?id=1860485
>
> LE or Classic? Perhaps this is related to:
I don't understand the question, how can I find out? All I know is
it's a Intel Corp. Bluetooth 9460/9560 Jefferson Peak (JfP)
> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
>
> Btw, in the logs it doesn't really show any connection attempt just
> advertisements reports so perhaps you want to collect the logs while
> attempting to connect or perhaps you are just waiting for the
> auto-connect to kick in? Does your device appear to be advertising?
I was collecting logs while toggling the "connect" switch in GNOME
settings->bluetooth-> for my bluetooth headset that had already been
paired; and then I chose to "forget" the device in the list; turned
the device off then back on in pairing mode, but it just hangs when I
try to set it up. All while running btmon. Seems like it can see
devices, but won't pair or connect.
When I keep user space the same and revert to kernel 5.16, the headset
immediately connects within seconds of being turned on.
Thanks!
--
Chris Murphy
Hi Chris,
On Tue, Feb 15, 2022 at 10:40 PM Chris Murphy <[email protected]> wrote:
>
> On Tue, Feb 15, 2022 at 8:38 AM Chris Murphy <[email protected]> wrote:
> >
> > On Tue, Feb 15, 2022 at 8:29 AM Chris Murphy <[email protected]> wrote:
> > >
> > > On Thu, Feb 10, 2022 at 6:44 PM Chris Murphy <[email protected]> wrote:
> > >
> > > > Case 2:
> > > >
> > > > Bus 001 Device 005: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> > > > Jefferson Peak (JfP)
> > >
> > > Comparing 5.16.9 (working) and 5.17.0-rc7 (non-working) on this
> > > Thinkpad X1 Carbon, I'm seeing two notable differences:
> > >
> > > Multiple messages like this:
> > > [ 15.731405] kernel: Bluetooth: hci0: unexpected event 0xff length: 5 > 0
>
>
> btmon is here:
> https://bugzilla.redhat.com/attachment.cgi?id=1860485
LE or Classic? Perhaps this is related to:
https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
Btw, in the logs it doesn't really show any connection attempt just
advertisements reports so perhaps you want to collect the logs while
attempting to connect or perhaps you are just waiting for the
auto-connect to kick in? Does your device appear to be advertising?
>
>
> >
> > This matches with a complaint I found here
> > https://lore.kernel.org/linux-bluetooth/[email protected]/
> > about this patch
> > https://lore.kernel.org/linux-bluetooth/[email protected]/
> >
> >
> >
> > --
> > Chris Murphy
>
>
>
> --
> Chris Murphy
--
Luiz Augusto von Dentz
OK I started over, and for now keeping the reporting constrained to
the hardware I personally have on hand.
Hardware:
Lenovo Thinkpad X1 Carbon Gen 7
Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
Jefferson Peak (JfP)
Sony 1000XM3 headset
bluez-5.63-3.fc36.x86_64
kernel 5.17.0-rc4
* remove the paired headset with bluetoothctl
* reset the headset so it's not longer paired either
* put the headset in pairing mode
* GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
* click on Not Setup and nothing happens
The above activity is recorded with 'btmon -w btmon-w-headsetpair.txt'
and i get both termin output and a binary file
terminal output
https://drive.google.com/file/d/1fFbyRm32Lf0yzrHhE1vEE0ztnSeoe7VB/view?usp=sharing
binary output
https://drive.google.com/file/d/1BmnO-utOooMWgcEHTuoz13Ndd5hGuqe_/view?usp=sharing
kernel 5.16.9
* bluetooth panel says it's connected to LE_WH-1000XM3, as if the
previous pairing attempt above worked
* bluetooth panel name for the device flips between LE_WH-1000XM3 and WH-1000XM3
* headset is still in pairing mode from when i was trying to pair with
5.17.0 (flashing blue lights), it eventually powers off so as far as
it's concern pairing didn't succeed
* again remove the headset with bluetoothctl
* put the headset back into pairing mode, click on LE_WH-1000XM3 in
the list, "Not Setup" flips to "Connected" in the GNOME bluetooth
panel, and the headset reports pairing is successful
this is what bluetoothd is reporting during the name flipping
https://drive.google.com/file/d/1qjAlkODkjJzR33WpHD2Sxze8zq4t1UhS/view?usp=sharing
Definitely seems kernel related.
--
Chris Murphy
Hi Chris,
On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
>
> OK I started over, and for now keeping the reporting constrained to
> the hardware I personally have on hand.
>
> Hardware:
> Lenovo Thinkpad X1 Carbon Gen 7
> Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> Jefferson Peak (JfP)
> Sony 1000XM3 headset
> bluez-5.63-3.fc36.x86_64
>
> kernel 5.17.0-rc4
> * remove the paired headset with bluetoothctl
> * reset the headset so it's not longer paired either
> * put the headset in pairing mode
> * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
> * click on Not Setup and nothing happens
Well from the logs it doesn't seem the GNOME Setting is trying to do
anything, have you tried bluetoothctl> connect <address>
> The above activity is recorded with 'btmon -w btmon-w-headsetpair.txt'
> and i get both termin output and a binary file
>
> terminal output
> https://drive.google.com/file/d/1fFbyRm32Lf0yzrHhE1vEE0ztnSeoe7VB/view?usp=sharing
>
> binary output
> https://drive.google.com/file/d/1BmnO-utOooMWgcEHTuoz13Ndd5hGuqe_/view?usp=sharing
>
> kernel 5.16.9
> * bluetooth panel says it's connected to LE_WH-1000XM3, as if the
> previous pairing attempt above worked
> * bluetooth panel name for the device flips between LE_WH-1000XM3 and WH-1000XM3
> * headset is still in pairing mode from when i was trying to pair with
> 5.17.0 (flashing blue lights), it eventually powers off so as far as
> it's concern pairing didn't succeed
> * again remove the headset with bluetoothctl
> * put the headset back into pairing mode, click on LE_WH-1000XM3 in
> the list, "Not Setup" flips to "Connected" in the GNOME bluetooth
> panel, and the headset reports pairing is successful
> this is what bluetoothd is reporting during the name flipping
> https://drive.google.com/file/d/1qjAlkODkjJzR33WpHD2Sxze8zq4t1UhS/view?usp=sharing
>
> Definitely seems kernel related.
>
>
> --
> Chris Murphy
--
Luiz Augusto von Dentz
On Thu, Feb 17, 2022 at 5:15 PM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Chris,
>
> On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
> >
> > OK I started over, and for now keeping the reporting constrained to
> > the hardware I personally have on hand.
> >
> > Hardware:
> > Lenovo Thinkpad X1 Carbon Gen 7
> > Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> > Jefferson Peak (JfP)
> > Sony 1000XM3 headset
> > bluez-5.63-3.fc36.x86_64
> >
> > kernel 5.17.0-rc4
> > * remove the paired headset with bluetoothctl
> > * reset the headset so it's not longer paired either
> > * put the headset in pairing mode
> > * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
> > * click on Not Setup and nothing happens
>
> Well from the logs it doesn't seem the GNOME Setting is trying to do
> anything, have you tried bluetoothctl> connect <address>
`bluetoothctl scan on` does see the device
$ bluetoothctl pair 38:18:4C:24:2D:1D
Device 38:18:4C:24:2D:1D not available
$ bluetoothctl connect 38:18:4C:24:2D:1D
Device 38:18:4C:24:2D:1D not available
$ journalctl -b -o short-monotonic --no-hostname | grep -i blue
https://drive.google.com/file/d/1x9EDvDx6XUowyRy2056n6uW-4PLx5KRb/view?usp=sharing
--
Chris Murphy
Sorry folks, clicked Send instead of Save Draft in my earlier message.
Anyway...
On 18/02/2022 03:49, Chris Murphy wrote:
> On Thu, Feb 17, 2022 at 5:15 PM Luiz Augusto von Dentz
> <[email protected]> wrote:
>>
>> Hi Chris,
>>
>> On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
>>>
>>> OK I started over, and for now keeping the reporting constrained to
>>> the hardware I personally have on hand.
>>>
>>> Hardware:
>>> Lenovo Thinkpad X1 Carbon Gen 7
>>> Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
>>> Jefferson Peak (JfP)
>>> Sony 1000XM3 headset
>>> bluez-5.63-3.fc36.x86_64
>>>
>>> kernel 5.17.0-rc4
>>> * remove the paired headset with bluetoothctl
>>> * reset the headset so it's not longer paired either
>>> * put the headset in pairing mode
>>> * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
>>> * click on Not Setup and nothing happens
>>
>> Well from the logs it doesn't seem the GNOME Setting is trying to do
>> anything, have you tried bluetoothctl> connect <address>
>
> `bluetoothctl scan on` does see the device
> $ bluetoothctl pair 38:18:4C:24:2D:1D
> Device 38:18:4C:24:2D:1D not available
> $ bluetoothctl connect 38:18:4C:24:2D:1D
> Device 38:18:4C:24:2D:1D not available
>
> $ journalctl -b -o short-monotonic --no-hostname | grep -i blue
> https://drive.google.com/file/d/1x9EDvDx6XUowyRy2056n6uW-4PLx5KRb/view?usp=sharing
>
>
I too am experiencing the problem that already-paired devices fail to connect to my laptop when running a 5.17 kernel.
Extract from dmesg shows:
[ 3.825684] Bluetooth: hci0: Waiting for firmware download to complete
[ 3.825910] Bluetooth: hci0: Firmware loaded in 1551910 usecs
[ 3.825910] Bluetooth: hci0: unexpected event 0xff length: 5 > 0
[ 3.825936] Bluetooth: hci0: Waiting for device to boot
[ 3.839948] Bluetooth: hci0: unexpected event 0xff length: 7 > 0
[ 3.839973] Bluetooth: hci0: Device booted in 13715 usecs
[ 3.840205] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-19-0-4.ddc
[ 3.843002] Bluetooth: hci0: Applying Intel DDC parameters completed
[ 3.843926] Bluetooth: hci0: Firmware revision 0.4 build 125 week 46 2021
Extract from lshw shows:
description: Bluetooth wireless interface
product: AX201 Bluetooth
vendor: Intel Corp.
physical id: e
bus info: usb@1:e
version: 0.02
capabilities: bluetooth usb-2.01
configuration: driver=btusb maxpower=100mA speed=12Mbit/s
I don't know whether this will help, but I've found that the problem only occurs when boot from cold (i.e power on the
laptop. If I then do a warm reboot, my bluetooh devices connect successfully. The significant difference may be that on
a cold start, the firmware needs to loaded whereas on a warm reboot I see:
[ 2.000989] Bluetooth: hci0: Firmware already loaded
Hope this helps. I am happy to test any fixes or provide additional diagnostics, but I'm not subscribed so please cc me.
Chris
>
> --
> Chris Murphy
>
Hi.
On Mon, Feb 21, 2022 at 9:12 AM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Chris,
>
> On Thu, Feb 17, 2022 at 7:49 PM Chris Murphy <[email protected]> wrote:
> >
> > On Thu, Feb 17, 2022 at 5:15 PM Luiz Augusto von Dentz
> > <[email protected]> wrote:
> > >
> > > Hi Chris,
> > >
> > > On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
> > > >
> > > > OK I started over, and for now keeping the reporting constrained to
> > > > the hardware I personally have on hand.
> > > >
> > > > Hardware:
> > > > Lenovo Thinkpad X1 Carbon Gen 7
> > > > Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> > > > Jefferson Peak (JfP)
> > > > Sony 1000XM3 headset
> > > > bluez-5.63-3.fc36.x86_64
> > > >
> > > > kernel 5.17.0-rc4
> > > > * remove the paired headset with bluetoothctl
> > > > * reset the headset so it's not longer paired either
> > > > * put the headset in pairing mode
> > > > * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
> > > > * click on Not Setup and nothing happens
> > >
> > > Well from the logs it doesn't seem the GNOME Setting is trying to do
> > > anything, have you tried bluetoothctl> connect <address>
> >
> > `bluetoothctl scan on` does see the device
> > $ bluetoothctl pair 38:18:4C:24:2D:1D
> > Device 38:18:4C:24:2D:1D not available
> > $ bluetoothctl connect 38:18:4C:24:2D:1D
> > Device 38:18:4C:24:2D:1D not available
>
> Well you are unable to scan the device you won't be able to connect to
> it, are you sure the device is discoverable?
>
> > $ journalctl -b -o short-monotonic --no-hostname | grep -i blue
> > https://drive.google.com/file/d/1x9EDvDx6XUowyRy2056n6uW-4PLx5KRb/view?usp=sharing
So looking at the logs it does seem the device is found and it is
created so I wonder why you cannot pair it or perhaps it got removed
when you stopped scanning?
> >
> >
> >
> > --
> > Chris Murphy
>
>
>
> --
> Luiz Augusto von Dentz
--
Luiz Augusto von Dentz
Thanks Luiz.
On 21/02/2022 21:27, Luiz Augusto von Dentz wrote:
> Hi Chris,
>
> On Mon, Feb 21, 2022 at 12:23 PM Chris Clayton <[email protected]> wrote:
>>
>> Hi Luiz,
>>
>> On 21/02/2022 17:11, Luiz Augusto von Dentz wrote:
>>> Hi Chris,
>>>
>>> On Mon, Feb 21, 2022 at 5:22 AM Chris Clayton <[email protected]> wrote:
>>>>
>>>> Sorry folks, clicked Send instead of Save Draft in my earlier message.
>>>>
>>>> Anyway...
>>>>
>>>> On 18/02/2022 03:49, Chris Murphy wrote:
>>>>> On Thu, Feb 17, 2022 at 5:15 PM Luiz Augusto von Dentz
>>>>> <[email protected]> wrote:
>>>>>>
>>>>>> Hi Chris,
>>>>>>
>>>>>> On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
>>>>>>>
>>>>>>> OK I started over, and for now keeping the reporting constrained to
>>>>>>> the hardware I personally have on hand.
>>>>>>>
>>>>>>> Hardware:
>>>>>>> Lenovo Thinkpad X1 Carbon Gen 7
>>>>>>> Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
>>>>>>> Jefferson Peak (JfP)
>>>>>>> Sony 1000XM3 headset
>>>>>>> bluez-5.63-3.fc36.x86_64
>>>>>>>
>>>>>>> kernel 5.17.0-rc4
>>>>>>> * remove the paired headset with bluetoothctl
>>>>>>> * reset the headset so it's not longer paired either
>>>>>>> * put the headset in pairing mode
>>>>>>> * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
>>>>>>> * click on Not Setup and nothing happens
>>>>>>
>>>>>> Well from the logs it doesn't seem the GNOME Setting is trying to do
>>>>>> anything, have you tried bluetoothctl> connect <address>
>>>>>
>>>>> `bluetoothctl scan on` does see the device
>>>>> $ bluetoothctl pair 38:18:4C:24:2D:1D
>>>>> Device 38:18:4C:24:2D:1D not available
>>>>> $ bluetoothctl connect 38:18:4C:24:2D:1D
>>>>> Device 38:18:4C:24:2D:1D not available
>>>>>
>>>>> $ journalctl -b -o short-monotonic --no-hostname | grep -i blue
>>>>> https://drive.google.com/file/d/1x9EDvDx6XUowyRy2056n6uW-4PLx5KRb/view?usp=sharing
>>>>>
>>>>>
>>>>
>>>> I too am experiencing the problem that already-paired devices fail to connect to my laptop when running a 5.17 kernel.
>>>>
>>>> Extract from dmesg shows:
>>>> [ 3.825684] Bluetooth: hci0: Waiting for firmware download to complete
>>>> [ 3.825910] Bluetooth: hci0: Firmware loaded in 1551910 usecs
>>>> [ 3.825910] Bluetooth: hci0: unexpected event 0xff length: 5 > 0
>>>> [ 3.825936] Bluetooth: hci0: Waiting for device to boot
>>>> [ 3.839948] Bluetooth: hci0: unexpected event 0xff length: 7 > 0
>>>> [ 3.839973] Bluetooth: hci0: Device booted in 13715 usecs
>>>> [ 3.840205] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-19-0-4.ddc
>>>> [ 3.843002] Bluetooth: hci0: Applying Intel DDC parameters completed
>>>> [ 3.843926] Bluetooth: hci0: Firmware revision 0.4 build 125 week 46 2021
>>>>
>>>> Extract from lshw shows:
>>>> description: Bluetooth wireless interface
>>>> product: AX201 Bluetooth
>>>> vendor: Intel Corp.
>>>> physical id: e
>>>> bus info: usb@1:e
>>>> version: 0.02
>>>> capabilities: bluetooth usb-2.01
>>>> configuration: driver=btusb maxpower=100mA speed=12Mbit/s
>>>>
>>>> I don't know whether this will help, but I've found that the problem only occurs when boot from cold (i.e power on the
>>>> laptop. If I then do a warm reboot, my bluetooh devices connect successfully. The significant difference may be that on
>>>> a cold start, the firmware needs to loaded whereas on a warm reboot I see:
>>>>
>>>> [ 2.000989] Bluetooth: hci0: Firmware already loaded
>>>>
>>>> Hope this helps. I am happy to test any fixes or provide additional diagnostics, but I'm not subscribed so please cc me.
>>>
>>> What exactly doesn't work? Can't you power up the controller, etc?
>>
>> I have two bluetooth audio devices. One is a set of headphones and the other is a speaker. Both are paired with my
>> laptop and, normally, both automatically connect to the laptop when I power them on. I've had the speaker for three
>> years or more is has worked fine with all kernels that I have used up to and including the latest stable series -
>> 5.16.10. The headphones were acquired a year or so ago and to date have worked with all kernels I have had installed
>> since then. Consequently, this problem is a 5,17 regression.
>>
>> After a cold (power-on) boot with a 5.17 kernel, they do no connect automatically when switched on. Furthermore, if I
>> use the blueman application to attempt to connect, that attempt fails. The only way that I have found to connect
>> successfully is to do a reboot, after which the devices can connect automatically when I switch them on.
>>
>> I'm sorry, I have no idea what you mean by "Can't you power up the controller, etc?"
>
> Use btmon to capture the trace when you attempt to connect, it also
> would be a good idea to use bluetoothctl when attempting to connect.
It's getting late now, so I'll do a btmon trace tomorrow. From it's name, I assume bluetoothctl is part of the systemd
suite. I don't have systemd on the laptop but use sysvinit to start userspace including, of course, bluetoothd.
Chris
>
Hi Chris,
On Mon, Feb 21, 2022 at 12:23 PM Chris Clayton <[email protected]> wrote:
>
> Hi Luiz,
>
> On 21/02/2022 17:11, Luiz Augusto von Dentz wrote:
> > Hi Chris,
> >
> > On Mon, Feb 21, 2022 at 5:22 AM Chris Clayton <[email protected]> wrote:
> >>
> >> Sorry folks, clicked Send instead of Save Draft in my earlier message.
> >>
> >> Anyway...
> >>
> >> On 18/02/2022 03:49, Chris Murphy wrote:
> >>> On Thu, Feb 17, 2022 at 5:15 PM Luiz Augusto von Dentz
> >>> <[email protected]> wrote:
> >>>>
> >>>> Hi Chris,
> >>>>
> >>>> On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
> >>>>>
> >>>>> OK I started over, and for now keeping the reporting constrained to
> >>>>> the hardware I personally have on hand.
> >>>>>
> >>>>> Hardware:
> >>>>> Lenovo Thinkpad X1 Carbon Gen 7
> >>>>> Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> >>>>> Jefferson Peak (JfP)
> >>>>> Sony 1000XM3 headset
> >>>>> bluez-5.63-3.fc36.x86_64
> >>>>>
> >>>>> kernel 5.17.0-rc4
> >>>>> * remove the paired headset with bluetoothctl
> >>>>> * reset the headset so it's not longer paired either
> >>>>> * put the headset in pairing mode
> >>>>> * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
> >>>>> * click on Not Setup and nothing happens
> >>>>
> >>>> Well from the logs it doesn't seem the GNOME Setting is trying to do
> >>>> anything, have you tried bluetoothctl> connect <address>
> >>>
> >>> `bluetoothctl scan on` does see the device
> >>> $ bluetoothctl pair 38:18:4C:24:2D:1D
> >>> Device 38:18:4C:24:2D:1D not available
> >>> $ bluetoothctl connect 38:18:4C:24:2D:1D
> >>> Device 38:18:4C:24:2D:1D not available
> >>>
> >>> $ journalctl -b -o short-monotonic --no-hostname | grep -i blue
> >>> https://drive.google.com/file/d/1x9EDvDx6XUowyRy2056n6uW-4PLx5KRb/view?usp=sharing
> >>>
> >>>
> >>
> >> I too am experiencing the problem that already-paired devices fail to connect to my laptop when running a 5.17 kernel.
> >>
> >> Extract from dmesg shows:
> >> [ 3.825684] Bluetooth: hci0: Waiting for firmware download to complete
> >> [ 3.825910] Bluetooth: hci0: Firmware loaded in 1551910 usecs
> >> [ 3.825910] Bluetooth: hci0: unexpected event 0xff length: 5 > 0
> >> [ 3.825936] Bluetooth: hci0: Waiting for device to boot
> >> [ 3.839948] Bluetooth: hci0: unexpected event 0xff length: 7 > 0
> >> [ 3.839973] Bluetooth: hci0: Device booted in 13715 usecs
> >> [ 3.840205] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-19-0-4.ddc
> >> [ 3.843002] Bluetooth: hci0: Applying Intel DDC parameters completed
> >> [ 3.843926] Bluetooth: hci0: Firmware revision 0.4 build 125 week 46 2021
> >>
> >> Extract from lshw shows:
> >> description: Bluetooth wireless interface
> >> product: AX201 Bluetooth
> >> vendor: Intel Corp.
> >> physical id: e
> >> bus info: usb@1:e
> >> version: 0.02
> >> capabilities: bluetooth usb-2.01
> >> configuration: driver=btusb maxpower=100mA speed=12Mbit/s
> >>
> >> I don't know whether this will help, but I've found that the problem only occurs when boot from cold (i.e power on the
> >> laptop. If I then do a warm reboot, my bluetooh devices connect successfully. The significant difference may be that on
> >> a cold start, the firmware needs to loaded whereas on a warm reboot I see:
> >>
> >> [ 2.000989] Bluetooth: hci0: Firmware already loaded
> >>
> >> Hope this helps. I am happy to test any fixes or provide additional diagnostics, but I'm not subscribed so please cc me.
> >
> > What exactly doesn't work? Can't you power up the controller, etc?
>
> I have two bluetooth audio devices. One is a set of headphones and the other is a speaker. Both are paired with my
> laptop and, normally, both automatically connect to the laptop when I power them on. I've had the speaker for three
> years or more is has worked fine with all kernels that I have used up to and including the latest stable series -
> 5.16.10. The headphones were acquired a year or so ago and to date have worked with all kernels I have had installed
> since then. Consequently, this problem is a 5,17 regression.
>
> After a cold (power-on) boot with a 5.17 kernel, they do no connect automatically when switched on. Furthermore, if I
> use the blueman application to attempt to connect, that attempt fails. The only way that I have found to connect
> successfully is to do a reboot, after which the devices can connect automatically when I switch them on.
>
> I'm sorry, I have no idea what you mean by "Can't you power up the controller, etc?"
Use btmon to capture the trace when you attempt to connect, it also
would be a good idea to use bluetoothctl when attempting to connect.
--
Luiz Augusto von Dentz
Hi Luiz,
On 21/02/2022 17:11, Luiz Augusto von Dentz wrote:
> Hi Chris,
>
> On Mon, Feb 21, 2022 at 5:22 AM Chris Clayton <[email protected]> wrote:
>>
>> Sorry folks, clicked Send instead of Save Draft in my earlier message.
>>
>> Anyway...
>>
>> On 18/02/2022 03:49, Chris Murphy wrote:
>>> On Thu, Feb 17, 2022 at 5:15 PM Luiz Augusto von Dentz
>>> <[email protected]> wrote:
>>>>
>>>> Hi Chris,
>>>>
>>>> On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
>>>>>
>>>>> OK I started over, and for now keeping the reporting constrained to
>>>>> the hardware I personally have on hand.
>>>>>
>>>>> Hardware:
>>>>> Lenovo Thinkpad X1 Carbon Gen 7
>>>>> Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
>>>>> Jefferson Peak (JfP)
>>>>> Sony 1000XM3 headset
>>>>> bluez-5.63-3.fc36.x86_64
>>>>>
>>>>> kernel 5.17.0-rc4
>>>>> * remove the paired headset with bluetoothctl
>>>>> * reset the headset so it's not longer paired either
>>>>> * put the headset in pairing mode
>>>>> * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
>>>>> * click on Not Setup and nothing happens
>>>>
>>>> Well from the logs it doesn't seem the GNOME Setting is trying to do
>>>> anything, have you tried bluetoothctl> connect <address>
>>>
>>> `bluetoothctl scan on` does see the device
>>> $ bluetoothctl pair 38:18:4C:24:2D:1D
>>> Device 38:18:4C:24:2D:1D not available
>>> $ bluetoothctl connect 38:18:4C:24:2D:1D
>>> Device 38:18:4C:24:2D:1D not available
>>>
>>> $ journalctl -b -o short-monotonic --no-hostname | grep -i blue
>>> https://drive.google.com/file/d/1x9EDvDx6XUowyRy2056n6uW-4PLx5KRb/view?usp=sharing
>>>
>>>
>>
>> I too am experiencing the problem that already-paired devices fail to connect to my laptop when running a 5.17 kernel.
>>
>> Extract from dmesg shows:
>> [ 3.825684] Bluetooth: hci0: Waiting for firmware download to complete
>> [ 3.825910] Bluetooth: hci0: Firmware loaded in 1551910 usecs
>> [ 3.825910] Bluetooth: hci0: unexpected event 0xff length: 5 > 0
>> [ 3.825936] Bluetooth: hci0: Waiting for device to boot
>> [ 3.839948] Bluetooth: hci0: unexpected event 0xff length: 7 > 0
>> [ 3.839973] Bluetooth: hci0: Device booted in 13715 usecs
>> [ 3.840205] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-19-0-4.ddc
>> [ 3.843002] Bluetooth: hci0: Applying Intel DDC parameters completed
>> [ 3.843926] Bluetooth: hci0: Firmware revision 0.4 build 125 week 46 2021
>>
>> Extract from lshw shows:
>> description: Bluetooth wireless interface
>> product: AX201 Bluetooth
>> vendor: Intel Corp.
>> physical id: e
>> bus info: usb@1:e
>> version: 0.02
>> capabilities: bluetooth usb-2.01
>> configuration: driver=btusb maxpower=100mA speed=12Mbit/s
>>
>> I don't know whether this will help, but I've found that the problem only occurs when boot from cold (i.e power on the
>> laptop. If I then do a warm reboot, my bluetooh devices connect successfully. The significant difference may be that on
>> a cold start, the firmware needs to loaded whereas on a warm reboot I see:
>>
>> [ 2.000989] Bluetooth: hci0: Firmware already loaded
>>
>> Hope this helps. I am happy to test any fixes or provide additional diagnostics, but I'm not subscribed so please cc me.
>
> What exactly doesn't work? Can't you power up the controller, etc?
I have two bluetooth audio devices. One is a set of headphones and the other is a speaker. Both are paired with my
laptop and, normally, both automatically connect to the laptop when I power them on. I've had the speaker for three
years or more is has worked fine with all kernels that I have used up to and including the latest stable series -
5.16.10. The headphones were acquired a year or so ago and to date have worked with all kernels I have had installed
since then. Consequently, this problem is a 5,17 regression.
After a cold (power-on) boot with a 5.17 kernel, they do no connect automatically when switched on. Furthermore, if I
use the blueman application to attempt to connect, that attempt fails. The only way that I have found to connect
successfully is to do a reboot, after which the devices can connect automatically when I switch them on.
I'm sorry, I have no idea what you mean by "Can't you power up the controller, etc?"
Chris
Hi Chris,
On Mon, Feb 21, 2022 at 5:22 AM Chris Clayton <[email protected]> wrote:
>
> Sorry folks, clicked Send instead of Save Draft in my earlier message.
>
> Anyway...
>
> On 18/02/2022 03:49, Chris Murphy wrote:
> > On Thu, Feb 17, 2022 at 5:15 PM Luiz Augusto von Dentz
> > <[email protected]> wrote:
> >>
> >> Hi Chris,
> >>
> >> On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
> >>>
> >>> OK I started over, and for now keeping the reporting constrained to
> >>> the hardware I personally have on hand.
> >>>
> >>> Hardware:
> >>> Lenovo Thinkpad X1 Carbon Gen 7
> >>> Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> >>> Jefferson Peak (JfP)
> >>> Sony 1000XM3 headset
> >>> bluez-5.63-3.fc36.x86_64
> >>>
> >>> kernel 5.17.0-rc4
> >>> * remove the paired headset with bluetoothctl
> >>> * reset the headset so it's not longer paired either
> >>> * put the headset in pairing mode
> >>> * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
> >>> * click on Not Setup and nothing happens
> >>
> >> Well from the logs it doesn't seem the GNOME Setting is trying to do
> >> anything, have you tried bluetoothctl> connect <address>
> >
> > `bluetoothctl scan on` does see the device
> > $ bluetoothctl pair 38:18:4C:24:2D:1D
> > Device 38:18:4C:24:2D:1D not available
> > $ bluetoothctl connect 38:18:4C:24:2D:1D
> > Device 38:18:4C:24:2D:1D not available
> >
> > $ journalctl -b -o short-monotonic --no-hostname | grep -i blue
> > https://drive.google.com/file/d/1x9EDvDx6XUowyRy2056n6uW-4PLx5KRb/view?usp=sharing
> >
> >
>
> I too am experiencing the problem that already-paired devices fail to connect to my laptop when running a 5.17 kernel.
>
> Extract from dmesg shows:
> [ 3.825684] Bluetooth: hci0: Waiting for firmware download to complete
> [ 3.825910] Bluetooth: hci0: Firmware loaded in 1551910 usecs
> [ 3.825910] Bluetooth: hci0: unexpected event 0xff length: 5 > 0
> [ 3.825936] Bluetooth: hci0: Waiting for device to boot
> [ 3.839948] Bluetooth: hci0: unexpected event 0xff length: 7 > 0
> [ 3.839973] Bluetooth: hci0: Device booted in 13715 usecs
> [ 3.840205] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-19-0-4.ddc
> [ 3.843002] Bluetooth: hci0: Applying Intel DDC parameters completed
> [ 3.843926] Bluetooth: hci0: Firmware revision 0.4 build 125 week 46 2021
>
> Extract from lshw shows:
> description: Bluetooth wireless interface
> product: AX201 Bluetooth
> vendor: Intel Corp.
> physical id: e
> bus info: usb@1:e
> version: 0.02
> capabilities: bluetooth usb-2.01
> configuration: driver=btusb maxpower=100mA speed=12Mbit/s
>
> I don't know whether this will help, but I've found that the problem only occurs when boot from cold (i.e power on the
> laptop. If I then do a warm reboot, my bluetooh devices connect successfully. The significant difference may be that on
> a cold start, the firmware needs to loaded whereas on a warm reboot I see:
>
> [ 2.000989] Bluetooth: hci0: Firmware already loaded
>
> Hope this helps. I am happy to test any fixes or provide additional diagnostics, but I'm not subscribed so please cc me.
What exactly doesn't work? Can't you power up the controller, etc?
--
Luiz Augusto von Dentz
Hi Chris,
On Thu, Feb 17, 2022 at 7:49 PM Chris Murphy <[email protected]> wrote:
>
> On Thu, Feb 17, 2022 at 5:15 PM Luiz Augusto von Dentz
> <[email protected]> wrote:
> >
> > Hi Chris,
> >
> > On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
> > >
> > > OK I started over, and for now keeping the reporting constrained to
> > > the hardware I personally have on hand.
> > >
> > > Hardware:
> > > Lenovo Thinkpad X1 Carbon Gen 7
> > > Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> > > Jefferson Peak (JfP)
> > > Sony 1000XM3 headset
> > > bluez-5.63-3.fc36.x86_64
> > >
> > > kernel 5.17.0-rc4
> > > * remove the paired headset with bluetoothctl
> > > * reset the headset so it's not longer paired either
> > > * put the headset in pairing mode
> > > * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
> > > * click on Not Setup and nothing happens
> >
> > Well from the logs it doesn't seem the GNOME Setting is trying to do
> > anything, have you tried bluetoothctl> connect <address>
>
> `bluetoothctl scan on` does see the device
> $ bluetoothctl pair 38:18:4C:24:2D:1D
> Device 38:18:4C:24:2D:1D not available
> $ bluetoothctl connect 38:18:4C:24:2D:1D
> Device 38:18:4C:24:2D:1D not available
Well you are unable to scan the device you won't be able to connect to
it, are you sure the device is discoverable?
> $ journalctl -b -o short-monotonic --no-hostname | grep -i blue
> https://drive.google.com/file/d/1x9EDvDx6XUowyRy2056n6uW-4PLx5KRb/view?usp=sharing
>
>
>
> --
> Chris Murphy
--
Luiz Augusto von Dentz
Hi folks.
On 18/02/2022 03:49, Chris Murphy wrote:
> On Thu, Feb 17, 2022 at 5:15 PM Luiz Augusto von Dentz
> <[email protected]> wrote:
>>
>> Hi Chris,
>>
>> On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
>>>
>>> OK I started over, and for now keeping the reporting constrained to
>>> the hardware I personally have on hand.
>>>
>>> Hardware:
>>> Lenovo Thinkpad X1 Carbon Gen 7
>>> Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
>>> Jefferson Peak (JfP)
>>> Sony 1000XM3 headset
>>> bluez-5.63-3.fc36.x86_64
>>>
>>> kernel 5.17.0-rc4
>>> * remove the paired headset with bluetoothctl
>>> * reset the headset so it's not longer paired either
>>> * put the headset in pairing mode
>>> * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
>>> * click on Not Setup and nothing happens
>>
>> Well from the logs it doesn't seem the GNOME Setting is trying to do
>> anything, have you tried bluetoothctl> connect <address>
>
> `bluetoothctl scan on` does see the device
> $ bluetoothctl pair 38:18:4C:24:2D:1D
> Device 38:18:4C:24:2D:1D not available
> $ bluetoothctl connect 38:18:4C:24:2D:1D
> Device 38:18:4C:24:2D:1D not available
>
> $ journalctl -b -o short-monotonic --no-hostname | grep -i blue
> https://drive.google.com/file/d/1x9EDvDx6XUowyRy2056n6uW-4PLx5KRb/view?usp=sharing
>
>
I too am finding that my (already paired) bluetooth devices can't be connected to using 5.17 - 5.17.0-rc5 to be precise.
However, I find that if add a return
>
> --
> Chris Murphy
>
Hi Luiz.
On 21/02/2022 22:33, Chris Clayton wrote:
> Thanks Luiz.
>
> On 21/02/2022 21:27, Luiz Augusto von Dentz wrote:
>> Hi Chris,
>>
>> On Mon, Feb 21, 2022 at 12:23 PM Chris Clayton <[email protected]> wrote:
>>>
>>> Hi Luiz,
>>>
>>> On 21/02/2022 17:11, Luiz Augusto von Dentz wrote:
>>>> Hi Chris,
>>>>
>>>> On Mon, Feb 21, 2022 at 5:22 AM Chris Clayton <[email protected]> wrote:
>>>>>
>>>>> Sorry folks, clicked Send instead of Save Draft in my earlier message.
>>>>>
>>>>> Anyway...
>>>>>
>>>>> On 18/02/2022 03:49, Chris Murphy wrote:
>>>>>> On Thu, Feb 17, 2022 at 5:15 PM Luiz Augusto von Dentz
>>>>>> <[email protected]> wrote:
>>>>>>>
>>>>>>> Hi Chris,
>>>>>>>
>>>>>>> On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
>>>>>>>>
>>>>>>>> OK I started over, and for now keeping the reporting constrained to
>>>>>>>> the hardware I personally have on hand.
>>>>>>>>
>>>>>>>> Hardware:
>>>>>>>> Lenovo Thinkpad X1 Carbon Gen 7
>>>>>>>> Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
>>>>>>>> Jefferson Peak (JfP)
>>>>>>>> Sony 1000XM3 headset
>>>>>>>> bluez-5.63-3.fc36.x86_64
>>>>>>>>
>>>>>>>> kernel 5.17.0-rc4
>>>>>>>> * remove the paired headset with bluetoothctl
>>>>>>>> * reset the headset so it's not longer paired either
>>>>>>>> * put the headset in pairing mode
>>>>>>>> * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
>>>>>>>> * click on Not Setup and nothing happens
>>>>>>>
>>>>>>> Well from the logs it doesn't seem the GNOME Setting is trying to do
>>>>>>> anything, have you tried bluetoothctl> connect <address>
>>>>>>
>>>>>> `bluetoothctl scan on` does see the device
>>>>>> $ bluetoothctl pair 38:18:4C:24:2D:1D
>>>>>> Device 38:18:4C:24:2D:1D not available
>>>>>> $ bluetoothctl connect 38:18:4C:24:2D:1D
>>>>>> Device 38:18:4C:24:2D:1D not available
>>>>>>
>>>>>> $ journalctl -b -o short-monotonic --no-hostname | grep -i blue
>>>>>> https://drive.google.com/file/d/1x9EDvDx6XUowyRy2056n6uW-4PLx5KRb/view?usp=sharing
>>>>>>
>>>>>>
>>>>>
>>>>> I too am experiencing the problem that already-paired devices fail to connect to my laptop when running a 5.17 kernel.
>>>>>
>>>>> Extract from dmesg shows:
>>>>> [ 3.825684] Bluetooth: hci0: Waiting for firmware download to complete
>>>>> [ 3.825910] Bluetooth: hci0: Firmware loaded in 1551910 usecs
>>>>> [ 3.825910] Bluetooth: hci0: unexpected event 0xff length: 5 > 0
>>>>> [ 3.825936] Bluetooth: hci0: Waiting for device to boot
>>>>> [ 3.839948] Bluetooth: hci0: unexpected event 0xff length: 7 > 0
>>>>> [ 3.839973] Bluetooth: hci0: Device booted in 13715 usecs
>>>>> [ 3.840205] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-19-0-4.ddc
>>>>> [ 3.843002] Bluetooth: hci0: Applying Intel DDC parameters completed
>>>>> [ 3.843926] Bluetooth: hci0: Firmware revision 0.4 build 125 week 46 2021
>>>>>
>>>>> Extract from lshw shows:
>>>>> description: Bluetooth wireless interface
>>>>> product: AX201 Bluetooth
>>>>> vendor: Intel Corp.
>>>>> physical id: e
>>>>> bus info: usb@1:e
>>>>> version: 0.02
>>>>> capabilities: bluetooth usb-2.01
>>>>> configuration: driver=btusb maxpower=100mA speed=12Mbit/s
>>>>>
>>>>> I don't know whether this will help, but I've found that the problem only occurs when boot from cold (i.e power on the
>>>>> laptop. If I then do a warm reboot, my bluetooh devices connect successfully. The significant difference may be that on
>>>>> a cold start, the firmware needs to loaded whereas on a warm reboot I see:
>>>>>
>>>>> [ 2.000989] Bluetooth: hci0: Firmware already loaded
>>>>>
>>>>> Hope this helps. I am happy to test any fixes or provide additional diagnostics, but I'm not subscribed so please cc me.
>>>>
>>>> What exactly doesn't work? Can't you power up the controller, etc?
>>>
>>> I have two bluetooth audio devices. One is a set of headphones and the other is a speaker. Both are paired with my
>>> laptop and, normally, both automatically connect to the laptop when I power them on. I've had the speaker for three
>>> years or more is has worked fine with all kernels that I have used up to and including the latest stable series -
>>> 5.16.10. The headphones were acquired a year or so ago and to date have worked with all kernels I have had installed
>>> since then. Consequently, this problem is a 5,17 regression.
>>>
>>> After a cold (power-on) boot with a 5.17 kernel, they do no connect automatically when switched on. Furthermore, if I
>>> use the blueman application to attempt to connect, that attempt fails. The only way that I have found to connect
>>> successfully is to do a reboot, after which the devices can connect automatically when I switch them on.
>>>
>>> I'm sorry, I have no idea what you mean by "Can't you power up the controller, etc?"
>>
>> Use btmon to capture the trace when you attempt to connect, it also
>> would be a good idea to use bluetoothctl when attempting to connect.
>
> It's getting late now, so I'll do a btmon trace tomorrow. From it's name, I assume bluetoothctl is part of the systemd
> suite. I don't have systemd on the laptop but use sysvinit to start userspace including, of course, bluetoothd.
>
I've trying to get a btmon trace but the problem of devices failing to connect has become intermittent. I pulled the
latest changes from Linus' tree this morning and built and installed the related kernel. When connection fails I see the
header it always spits out, but nothing else. When connection succeeds, there is plenty of output.
Tomorrow, I'll turn on debug in bluetoothd and see what the difference between a successful and a failed connection is.
Chris
> Chris
>
>
>>
Hi.
<snip>
>
> We are starting to suspect this is not a new issue, it just become
> easier to reproduce with newer kernels since the mgmt commands are now
> handled by a different work/thread which probably takes longer to
> respond hitting problems such as:
>
> https://github.com/bluez/bluez/issues/275#issuecomment-1020608282
>
> This has been fixed by:
>
> https://github.com/bluez/bluez/commit/faad125c5505b941f06c54525616e192a6369208
> https://github.com/bluez/bluez/commit/5f378404bff6bbfea3f20e36535c494efa5066a5
>
I cloned bluez, but that FTBFS, so I applied the two patches by hand.
After the first boot, my bluetooth devices connected fine. But after a poweroff and boot, they didn't. Nor did they on
the third and fourth boots, so the patches don't seem to be the answer. (They couldn't really be anyway because changes
to the kernel have broken user-space which I understand is a big no no unless there is a really compelling reason.)
I've gathered some diagnostics today and they are attached. They consist of 6 files containing the output from btmon and
dmesg and the log file for the system daemons, which, of course, includes bluetoothd. There are 2 sets of these files -
one from a boot that resulted in a system where my devices would not connect and another from a boot where they could
not connect. You'll note that the btmon log is empty for a failed connection.
I also tried a bisection with v5.16 as good and v5.17-rc1 as bad. Unfortunately, I found several steps resulted in a
kernel where bluetooth seemed to be substantially borked - to the extent that blueman was non-functional and clicking on
the tray icon did not start up the blueman-manager application.
I also booted into a 5.16.10 kernel and connecting bluetooth devices worked flawlessly. (This was with the unpatched
bluez daemon)
Chris
> So the timer doesn't start until the request is sent. but obvoiusly
> older versions of userspace don't have that fix so they end up
> cancelling the loading of LTKs, this would explain why reloading the
> daemon would make it work again.
>
Hi,
On Tue, Feb 22, 2022 at 12:03 PM Chris Clayton <[email protected]> wrote:
>
> Hi Luiz.
>
> On 21/02/2022 22:33, Chris Clayton wrote:
> > Thanks Luiz.
> >
> > On 21/02/2022 21:27, Luiz Augusto von Dentz wrote:
> >> Hi Chris,
> >>
> >> On Mon, Feb 21, 2022 at 12:23 PM Chris Clayton <[email protected]> wrote:
> >>>
> >>> Hi Luiz,
> >>>
> >>> On 21/02/2022 17:11, Luiz Augusto von Dentz wrote:
> >>>> Hi Chris,
> >>>>
> >>>> On Mon, Feb 21, 2022 at 5:22 AM Chris Clayton <[email protected]> wrote:
> >>>>>
> >>>>> Sorry folks, clicked Send instead of Save Draft in my earlier message.
> >>>>>
> >>>>> Anyway...
> >>>>>
> >>>>> On 18/02/2022 03:49, Chris Murphy wrote:
> >>>>>> On Thu, Feb 17, 2022 at 5:15 PM Luiz Augusto von Dentz
> >>>>>> <[email protected]> wrote:
> >>>>>>>
> >>>>>>> Hi Chris,
> >>>>>>>
> >>>>>>> On Thu, Feb 17, 2022 at 3:36 PM Chris Murphy <[email protected]> wrote:
> >>>>>>>>
> >>>>>>>> OK I started over, and for now keeping the reporting constrained to
> >>>>>>>> the hardware I personally have on hand.
> >>>>>>>>
> >>>>>>>> Hardware:
> >>>>>>>> Lenovo Thinkpad X1 Carbon Gen 7
> >>>>>>>> Bus 001 Device 004: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> >>>>>>>> Jefferson Peak (JfP)
> >>>>>>>> Sony 1000XM3 headset
> >>>>>>>> bluez-5.63-3.fc36.x86_64
> >>>>>>>>
> >>>>>>>> kernel 5.17.0-rc4
> >>>>>>>> * remove the paired headset with bluetoothctl
> >>>>>>>> * reset the headset so it's not longer paired either
> >>>>>>>> * put the headset in pairing mode
> >>>>>>>> * GNOME Settings Bluetooth panel sees -> LE_WH-1000XM3, Not Setup
> >>>>>>>> * click on Not Setup and nothing happens
> >>>>>>>
> >>>>>>> Well from the logs it doesn't seem the GNOME Setting is trying to do
> >>>>>>> anything, have you tried bluetoothctl> connect <address>
> >>>>>>
> >>>>>> `bluetoothctl scan on` does see the device
> >>>>>> $ bluetoothctl pair 38:18:4C:24:2D:1D
> >>>>>> Device 38:18:4C:24:2D:1D not available
> >>>>>> $ bluetoothctl connect 38:18:4C:24:2D:1D
> >>>>>> Device 38:18:4C:24:2D:1D not available
> >>>>>>
> >>>>>> $ journalctl -b -o short-monotonic --no-hostname | grep -i blue
> >>>>>> https://drive.google.com/file/d/1x9EDvDx6XUowyRy2056n6uW-4PLx5KRb/view?usp=sharing
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> I too am experiencing the problem that already-paired devices fail to connect to my laptop when running a 5.17 kernel.
> >>>>>
> >>>>> Extract from dmesg shows:
> >>>>> [ 3.825684] Bluetooth: hci0: Waiting for firmware download to complete
> >>>>> [ 3.825910] Bluetooth: hci0: Firmware loaded in 1551910 usecs
> >>>>> [ 3.825910] Bluetooth: hci0: unexpected event 0xff length: 5 > 0
> >>>>> [ 3.825936] Bluetooth: hci0: Waiting for device to boot
> >>>>> [ 3.839948] Bluetooth: hci0: unexpected event 0xff length: 7 > 0
> >>>>> [ 3.839973] Bluetooth: hci0: Device booted in 13715 usecs
> >>>>> [ 3.840205] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-19-0-4.ddc
> >>>>> [ 3.843002] Bluetooth: hci0: Applying Intel DDC parameters completed
> >>>>> [ 3.843926] Bluetooth: hci0: Firmware revision 0.4 build 125 week 46 2021
> >>>>>
> >>>>> Extract from lshw shows:
> >>>>> description: Bluetooth wireless interface
> >>>>> product: AX201 Bluetooth
> >>>>> vendor: Intel Corp.
> >>>>> physical id: e
> >>>>> bus info: usb@1:e
> >>>>> version: 0.02
> >>>>> capabilities: bluetooth usb-2.01
> >>>>> configuration: driver=btusb maxpower=100mA speed=12Mbit/s
> >>>>>
> >>>>> I don't know whether this will help, but I've found that the problem only occurs when boot from cold (i.e power on the
> >>>>> laptop. If I then do a warm reboot, my bluetooh devices connect successfully. The significant difference may be that on
> >>>>> a cold start, the firmware needs to loaded whereas on a warm reboot I see:
> >>>>>
> >>>>> [ 2.000989] Bluetooth: hci0: Firmware already loaded
> >>>>>
> >>>>> Hope this helps. I am happy to test any fixes or provide additional diagnostics, but I'm not subscribed so please cc me.
> >>>>
> >>>> What exactly doesn't work? Can't you power up the controller, etc?
> >>>
> >>> I have two bluetooth audio devices. One is a set of headphones and the other is a speaker. Both are paired with my
> >>> laptop and, normally, both automatically connect to the laptop when I power them on. I've had the speaker for three
> >>> years or more is has worked fine with all kernels that I have used up to and including the latest stable series -
> >>> 5.16.10. The headphones were acquired a year or so ago and to date have worked with all kernels I have had installed
> >>> since then. Consequently, this problem is a 5,17 regression.
> >>>
> >>> After a cold (power-on) boot with a 5.17 kernel, they do no connect automatically when switched on. Furthermore, if I
> >>> use the blueman application to attempt to connect, that attempt fails. The only way that I have found to connect
> >>> successfully is to do a reboot, after which the devices can connect automatically when I switch them on.
> >>>
> >>> I'm sorry, I have no idea what you mean by "Can't you power up the controller, etc?"
> >>
> >> Use btmon to capture the trace when you attempt to connect, it also
> >> would be a good idea to use bluetoothctl when attempting to connect.
> >
> > It's getting late now, so I'll do a btmon trace tomorrow. From it's name, I assume bluetoothctl is part of the systemd
> > suite. I don't have systemd on the laptop but use sysvinit to start userspace including, of course, bluetoothd.
> >
>
> I've trying to get a btmon trace but the problem of devices failing to connect has become intermittent. I pulled the
> latest changes from Linus' tree this morning and built and installed the related kernel. When connection fails I see the
> header it always spits out, but nothing else. When connection succeeds, there is plenty of output.
>
> Tomorrow, I'll turn on debug in bluetoothd and see what the difference between a successful and a failed connection is.
We are starting to suspect this is not a new issue, it just become
easier to reproduce with newer kernels since the mgmt commands are now
handled by a different work/thread which probably takes longer to
respond hitting problems such as:
https://github.com/bluez/bluez/issues/275#issuecomment-1020608282
This has been fixed by:
https://github.com/bluez/bluez/commit/faad125c5505b941f06c54525616e192a6369208
https://github.com/bluez/bluez/commit/5f378404bff6bbfea3f20e36535c494efa5066a5
So the timer doesn't start until the request is sent. but obviously
older versions of userspace don't have that fix so they end up
cancelling the loading of LTKs, this would explain why reloading the
daemon would make it work again.
--
Luiz Augusto von Dentz
On 23/02/2022 22:42, Chris Clayton wrote:
> Hi.
>
> <snip>
>
>>
>> We are starting to suspect this is not a new issue, it just become
>> easier to reproduce with newer kernels since the mgmt commands are now
>> handled by a different work/thread which probably takes longer to
>> respond hitting problems such as:
>>
>> https://github.com/bluez/bluez/issues/275#issuecomment-1020608282
>>
>> This has been fixed by:
>>
>> https://github.com/bluez/bluez/commit/faad125c5505b941f06c54525616e192a6369208
>> https://github.com/bluez/bluez/commit/5f378404bff6bbfea3f20e36535c494efa5066a5
>>
>
> I cloned bluez, but that FTBFS, so I applied the two patches by hand.
>
> After the first boot, my bluetooth devices connected fine. But after a poweroff and boot, they didn't. Nor did they on
> the third and fourth boots, so the patches don't seem to be the answer. (They couldn't really be anyway because changes
> to the kernel have broken user-space which I understand is a big no no unless there is a really compelling reason.)
>
> I've gathered some diagnostics today and they are attached. They consist of 6 files containing the output from btmon and
> dmesg and the log file for the system daemons, which, of course, includes bluetoothd. There are 2 sets of these files -
> one from a boot that resulted in a system where my devices would not connect and another from a boot where they could
s/would not connect and another/would connect and another/
> not connect. You'll note that the btmon log is empty for a failed connection.
>
> I also tried a bisection with v5.16 as good and v5.17-rc1 as bad. Unfortunately, I found several steps resulted in a
> kernel where bluetooth seemed to be substantially borked - to the extent that blueman was non-functional and clicking on
> the tray icon did not start up the blueman-manager application.
>
> I also booted into a 5.16.10 kernel and connecting bluetooth devices worked flawlessly. (This was with the unpatched
> bluez daemon)
>
> Chris
>> So the timer doesn't start until the request is sent. but obvoiusly
>> older versions of userspace don't have that fix so they end up
>> cancelling the loading of LTKs, this would explain why reloading the
>> daemon would make it work again.
Hi Chris,
On Wed, Feb 23, 2022 at 9:59 PM Chris Clayton <[email protected]> wrote:
>
>
>
> On 23/02/2022 22:42, Chris Clayton wrote:
> > Hi.
> >
> > <snip>
> >
> >>
> >> We are starting to suspect this is not a new issue, it just become
> >> easier to reproduce with newer kernels since the mgmt commands are now
> >> handled by a different work/thread which probably takes longer to
> >> respond hitting problems such as:
> >>
> >> https://github.com/bluez/bluez/issues/275#issuecomment-1020608282
> >>
> >> This has been fixed by:
> >>
> >> https://github.com/bluez/bluez/commit/faad125c5505b941f06c54525616e192a6369208
> >> https://github.com/bluez/bluez/commit/5f378404bff6bbfea3f20e36535c494efa5066a5
> >>
> >
> > I cloned bluez, but that FTBFS, so I applied the two patches by hand.
> >
> > After the first boot, my bluetooth devices connected fine. But after a poweroff and boot, they didn't. Nor did they on
> > the third and fourth boots, so the patches don't seem to be the answer. (They couldn't really be anyway because changes
> > to the kernel have broken user-space which I understand is a big no no unless there is a really compelling reason.)
> >
> > I've gathered some diagnostics today and they are attached. They consist of 6 files containing the output from btmon and
> > dmesg and the log file for the system daemons, which, of course, includes bluetoothd. There are 2 sets of these files -
> > one from a boot that resulted in a system where my devices would not connect and another from a boot where they could
>
> s/would not connect and another/would connect and another/
>
> > not connect. You'll note that the btmon log is empty for a failed connection.
> >
> > I also tried a bisection with v5.16 as good and v5.17-rc1 as bad. Unfortunately, I found several steps resulted in a
> > kernel where bluetooth seemed to be substantially borked - to the extent that blueman was non-functional and clicking on
> > the tray icon did not start up the blueman-manager application.
Running with 5.17.0-0.rc5.102.fc37.x86_64 there doesn't seem to be any
problems, well apart from LE device with Privacy/RPA:
https://gist.github.com/Vudentz/5792f4989198c7f2994af2e31eb973af
Ive tried suspend/resume a couple of times just to see if there is
something odd going own, anyway I'm running with the latest userspace
so perhaps I really need some old userspace to reproduce this. There
might be something with name resolving though, it appears gnome don't
show devices if they don't have a proper name (that is not their
address) so perhaps that gives the impression there is nothing going
on when in fact that all normal, well apart from the fact that names
takes way too long to be resolved.
> > I also booted into a 5.16.10 kernel and connecting bluetooth devices worked flawlessly. (This was with the unpatched
> > bluez daemon)
> >
> > Chris
> >> So the timer doesn't start until the request is sent. but obvoiusly
> >> older versions of userspace don't have that fix so they end up
> >> cancelling the loading of LTKs, this would explain why reloading the
> >> daemon would make it work again.
--
Luiz Augusto von Dentz
Hello,
On 24/02/2022 05:59, Chris Clayton wrote:
>
>
> On 23/02/2022 22:42, Chris Clayton wrote:
>> Hi.
>>
>> <snip>
>>
>>>
>>> We are starting to suspect this is not a new issue, it just become
>>> easier to reproduce with newer kernels since the mgmt commands are now
>>> handled by a different work/thread which probably takes longer to
>>> respond hitting problems such as:
>>>
>>> https://github.com/bluez/bluez/issues/275#issuecomment-1020608282
>>>
>>> This has been fixed by:
>>>
>>> https://github.com/bluez/bluez/commit/faad125c5505b941f06c54525616e192a6369208
>>> https://github.com/bluez/bluez/commit/5f378404bff6bbfea3f20e36535c494efa5066a5
>>>
>>
>> I cloned bluez, but that FTBFS, so I applied the two patches by hand.
>>
Sorry, the FTBFS was my error - I didn't do the autotools magic properly. More haste, less speed!
So I've now built and tested 5.17-rc5+ (v5.17-rc5-21-g23d04328444a) with bluetoothd built from a fresh clone of the
bluez git tree taken this morning (HEAD is d89af9a). I'm sorry to say that I still get the intermittent failures when
trying to connect my bluetooth devices. By intermittent I mean that if after starting the system from cold, if the first
attempt to connect fails, then subsequent attempts also fail. If, however, the first attempt succeeds, I can disconnect
and connect again as many times as I want. With a 5.16.10 kernel, connecting and disconnecting works reliably like it
did with vanilla bluez-5.63.
Chris
>> After the first boot, my bluetooth devices connected fine. But after a poweroff and boot, they didn't. Nor did they on
>> the third and fourth boots, so the patches don't seem to be the answer. (They couldn't really be anyway because changes
>> to the kernel have broken user-space which I understand is a big no no unless there is a really compelling reason.)
>>
>> I've gathered some diagnostics today and they are attached. They consist of 6 files containing the output from btmon and
>> dmesg and the log file for the system daemons, which, of course, includes bluetoothd. There are 2 sets of these files -
>> one from a boot that resulted in a system where my devices would not connect and another from a boot where they could
>
> s/would not connect and another/would connect and another/
>
>> not connect. You'll note that the btmon log is empty for a failed connection.
>>
>> I also tried a bisection with v5.16 as good and v5.17-rc1 as bad. Unfortunately, I found several steps resulted in a
>> kernel where bluetooth seemed to be substantially borked - to the extent that blueman was non-functional and clicking on
>> the tray icon did not start up the blueman-manager application.
>>
>> I also booted into a 5.16.10 kernel and connecting bluetooth devices worked flawlessly. (This was with the unpatched
>> bluez daemon)
>>
>> Chris
>>> So the timer doesn't start until the request is sent. but obvoiusly
>>> older versions of userspace don't have that fix so they end up
>>> cancelling the loading of LTKs, this would explain why reloading the
>>> daemon would make it work again.
Hi Luiz,
On 24/02/2022 08:16, Luiz Augusto von Dentz wrote:
> Hi Chris,
>
> On Wed, Feb 23, 2022 at 9:59 PM Chris Clayton <[email protected]> wrote:
>>
>>
>>
>> On 23/02/2022 22:42, Chris Clayton wrote:
>>> Hi.
>>>
>>> <snip>
>>>
>>>>
>>>> We are starting to suspect this is not a new issue, it just become
>>>> easier to reproduce with newer kernels since the mgmt commands are now
>>>> handled by a different work/thread which probably takes longer to
>>>> respond hitting problems such as:
>>>>
>>>> https://github.com/bluez/bluez/issues/275#issuecomment-1020608282
>>>>
>>>> This has been fixed by:
>>>>
>>>> https://github.com/bluez/bluez/commit/faad125c5505b941f06c54525616e192a6369208
>>>> https://github.com/bluez/bluez/commit/5f378404bff6bbfea3f20e36535c494efa5066a5
>>>>
>>>
>>> I cloned bluez, but that FTBFS, so I applied the two patches by hand.
>>>
>>> After the first boot, my bluetooth devices connected fine. But after a poweroff and boot, they didn't. Nor did they on
>>> the third and fourth boots, so the patches don't seem to be the answer. (They couldn't really be anyway because changes
>>> to the kernel have broken user-space which I understand is a big no no unless there is a really compelling reason.)
>>>
>>> I've gathered some diagnostics today and they are attached. They consist of 6 files containing the output from btmon and
>>> dmesg and the log file for the system daemons, which, of course, includes bluetoothd. There are 2 sets of these files -
>>> one from a boot that resulted in a system where my devices would not connect and another from a boot where they could
>>
>> s/would not connect and another/would connect and another/
>>
>>> not connect. You'll note that the btmon log is empty for a failed connection.
>>>
>>> I also tried a bisection with v5.16 as good and v5.17-rc1 as bad. Unfortunately, I found several steps resulted in a
>>> kernel where bluetooth seemed to be substantially borked - to the extent that blueman was non-functional and clicking on
>>> the tray icon did not start up the blueman-manager application.
>
> Running with 5.17.0-0.rc5.102.fc37.x86_64 there doesn't seem to be any
> problems, well apart from LE device with Privacy/RPA:
>
> https://gist.github.com/Vudentz/5792f4989198c7f2994af2e31eb973af
>
> Ive tried suspend/resume a couple of times just to see if there is
> something odd going own, anyway I'm running with the latest userspace
> so perhaps I really need some old userspace to reproduce this. There
I don't have an old userspace. I built my system from source about four years ago using the methods from Linux From
Scratch and Beyond Linux From Scratch. I have been updating the packages since then as and when they are released. In a
way, you could equate my system to one of the rolling releases. It is very up-to-date but at the same time very stable.
This morning I updated to the latest releases of the 5.15 and 5.16 stable series. I've also built and installed the
latest in the 5.4 and 5.10 stable series. (All four had new releases yesterday.) Bluetooth works as expected on all 4 of
those kernels. It has also worked on every kernel released by Linus and every stables series kernel in the last four
years. I usually start trying out a development kernel when rc3, 4 or 5 is released, depending on time available. If I
find any problems I report them and provide any diagnostics required and test patches.
Bluetooth is unreliable on my system when I boot 5.17-rc kernels. There is little doubt in my mind that something that
has changed in 5.17 is at the root of this. Whether it is actually a bug in userspace doesn't change the fact that there
is a regression because of one or more changes in the kernel. As I said yesterday, my understanding is that such
regressions are frowned upon.
I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
In the meantime I've copied this email to [email protected], so they can track this.
Chris
> might be something with name resolving though, it appears gnome don't
> show devices if they don't have a proper name (that is not their
> address) so perhaps that gives the impression there is nothing going
> on when in fact that all normal, well apart from the fact that names
> takes way too long to be resolved.
>
>>> I also booted into a 5.16.10 kernel and connecting bluetooth devices worked flawlessly. (This was with the unpatched
>>> bluez daemon)
>>>
>>> Chris
>>>> So the timer doesn't start until the request is sent. but obvoiusly
>>>> older versions of userspace don't have that fix so they end up
>>>> cancelling the loading of LTKs, this would explain why reloading the
>>>> daemon would make it work again.
>
>
>
[TLDR: I'm adding the regression report below to regzbot, the Linux
kernel regression tracking bot; all text you find below is compiled from
a few templates paragraphs you might have encountered already already
from similar mails.]
Hi, this is your Linux kernel regression tracker speaking.
Thanks for the report and CCing the regression list in a downstream reply.
To be sure this issue doesn't fall through the cracks unnoticed, I'm
adding it to regzbot, my Linux kernel regression tracking bot:
#regzbot ^introduced v5.16..
#regzbot title net: bluetooth: qualcom and intel adapters, unable to
reliably connect to bluetooth devices
#regzbot ignore-activity
Reminder for developers: when fixing the issue, please add a 'Link:'
tags pointing to the report (the mail quoted above) using
lore.kernel.org/r/, as explained in
'Documentation/process/submitting-patches.rst' and
'Documentation/process/5.Posting.rst'. This allows the bot to connect
the report with any patches posted or committed to fix the issue; this
again allows the bot to show the current status of regressions and
automatically resolve the issue when the fix hits the right tree.
I'm sending this to everyone that got the initial report, to make them
aware of the tracking. I also hope that messages like this motivate
people to directly get at least the regression mailing list and ideally
even regzbot involved when dealing with regressions, as messages like
this wouldn't be needed then.
Don't worry, I'll send further messages wrt to this regression just to
the lists (with a tag in the subject so people can filter them away), if
they are relevant just for regzbot. With a bit of luck no such messages
will be needed anyway.
Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
P.S.: As the Linux kernel's regression tracker I'm getting a lot of
reports on my table. I can only look briefly into most of them and lack
knowledge about most of the areas they concern. I thus unfortunately
will sometimes get things wrong or miss something important. I hope
that's not the case here; if you think it is, don't hesitate to tell me
in a public reply, it's in everyone's interest to set the public record
straight.
On 11.02.22 02:44, Chris Murphy wrote:
> Since kernel 5.17, some users are seeing problems connecting to
> bluetooth devices, the problem doesn't happen with 5.16 series
> kernels.
>
> So far two different devices are affected, although it's possible it's
> not even device related, but could be the interface between kernel and
> user space.
>
> Case 1:
>
> Bus 001 Device 003: ID 0cf3:e301 Qualcomm Atheros Communications
>
> Not seeing anything that stands out in dmesg, bluetoothd -d shows
> various curious messages like
>
> bluetoothd[5889]: src/service.c:change_state() 0x7f33becdebc0: device
> 94:65:2D:DC:F4:A7 profile avrcp-controller state changed: disconnected
> -> unavailable (0)
> and
> bluetoothd[5889]: src/service.c:change_state() 0x7f33becd91e0: device
> 94:65:2D:DC:F4:A7 profile Hands-Free Voice gateway state changed:
> unavailable -> disconnected (0)
>
>
> Case 2:
>
> Bus 001 Device 005: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560
> Jefferson Peak (JfP)
> [ 5.923865] kernel: Bluetooth: hci0: Found device firmware:
> intel/ibt-17-16-1.sfi
> [ 5.930536] kernel: Bluetooth: hci0: Boot Address: 0x40800
> [ 5.930539] kernel: Bluetooth: hci0: Firmware Version: 60-44.21
> ...
> [16102.640651] Bluetooth: hci0: unexpected event 0xff length: 5 > 0
> [16102.640802] Bluetooth: hci0: Waiting for device to boot
> [16102.654712] Bluetooth: hci0: Device booted in 13698 usecs
> [16102.654753] Bluetooth: hci0: unexpected event 0xff length: 7 > 0
> [16102.655907] Bluetooth: hci0: Found Intel DDC parameters:
> intel/ibt-17-16-1.ddc
> [16102.657821] Bluetooth: hci0: Applying Intel DDC parameters completed
> [16102.658829] Bluetooth: hci0: Firmware revision 0.1 build 60 week 44 2021
> [17422.210558] Bluetooth: hci0: command 0x0c03 tx timeout
> [17430.338412] Bluetooth: hci0: HCI reset during shutdown failed
>
> [15975.194153] bluetoothd[42895]:
> src/adapter.c:device_found_callback() hci0 addr 75:0B:19:0C:47:F6,
> rssi -82 flags 0x0000 eir_len 17
> [15975.194412] bluetoothd[42895]:
> src/adv_monitor.c:btd_adv_monitor_offload_supported() Manager is NULL,
> get offload support failed
> [15975.194510] bluetoothd[42895]: src/device.c:device_set_legacy() legacy 0
> [15975.194599] bluetoothd[42895]:
> src/device.c:device_set_rssi_with_delta() rssi -82 delta 1
> [15975.194661] bluetoothd[42895]: src/device.c:device_set_flags() flags 26
>
> There are many many of the "Manager is NULL, get offload support
> failed" messages in this case. I'm not entirely sure the root causes
> are the same for the two cases.
>
> Downstream bug has attachments (dmesg, btmon, lsbsb, journalctl
> snippet with bluetoothd in debug, for both)
> https://bugzilla.redhat.com/show_bug.cgi?id=2053283
>
> Thanks,
>
--
Additional information about regzbot:
If you want to know more about regzbot, check out its web-interface, the
getting start guide, and the references documentation:
https://linux-regtracking.leemhuis.info/regzbot/
https://gitlab.com/knurd42/regzbot/-/blob/main/docs/getting_started.md
https://gitlab.com/knurd42/regzbot/-/blob/main/docs/reference.md
The last two documents will explain how you can interact with regzbot
yourself if your want to.
Hint for reporters: when reporting a regression it's in your interest to
CC the regression list and tell regzbot about the issue, as that ensures
the regression makes it onto the radar of the Linux kernel's regression
tracker -- that's in your interest, as it ensures your report won't fall
through the cracks unnoticed.
Hint for developers: you normally don't need to care about regzbot once
it's involved. Fix the issue as you normally would, just remember to
include 'Link:' tag in the patch descriptions pointing to all reports
about the issue. This has been expected from developers even before
regzbot showed up for reasons explained in
'Documentation/process/submitting-patches.rst' and
'Documentation/process/5.Posting.rst'.
Hi Chris,
On Thu, Feb 24, 2022 at 2:08 AM Chris Clayton <[email protected]> wrote:
>
> Hi Luiz,
>
> On 24/02/2022 08:16, Luiz Augusto von Dentz wrote:
> > Hi Chris,
> >
> > On Wed, Feb 23, 2022 at 9:59 PM Chris Clayton <[email protected]> wrote:
> >>
> >>
> >>
> >> On 23/02/2022 22:42, Chris Clayton wrote:
> >>> Hi.
> >>>
> >>> <snip>
> >>>
> >>>>
> >>>> We are starting to suspect this is not a new issue, it just become
> >>>> easier to reproduce with newer kernels since the mgmt commands are now
> >>>> handled by a different work/thread which probably takes longer to
> >>>> respond hitting problems such as:
> >>>>
> >>>> https://github.com/bluez/bluez/issues/275#issuecomment-1020608282
> >>>>
> >>>> This has been fixed by:
> >>>>
> >>>> https://github.com/bluez/bluez/commit/faad125c5505b941f06c54525616e192a6369208
> >>>> https://github.com/bluez/bluez/commit/5f378404bff6bbfea3f20e36535c494efa5066a5
> >>>>
> >>>
> >>> I cloned bluez, but that FTBFS, so I applied the two patches by hand.
> >>>
> >>> After the first boot, my bluetooth devices connected fine. But after a poweroff and boot, they didn't. Nor did they on
> >>> the third and fourth boots, so the patches don't seem to be the answer. (They couldn't really be anyway because changes
> >>> to the kernel have broken user-space which I understand is a big no no unless there is a really compelling reason.)
> >>>
> >>> I've gathered some diagnostics today and they are attached. They consist of 6 files containing the output from btmon and
> >>> dmesg and the log file for the system daemons, which, of course, includes bluetoothd. There are 2 sets of these files -
> >>> one from a boot that resulted in a system where my devices would not connect and another from a boot where they could
> >>
> >> s/would not connect and another/would connect and another/
> >>
> >>> not connect. You'll note that the btmon log is empty for a failed connection.
> >>>
> >>> I also tried a bisection with v5.16 as good and v5.17-rc1 as bad. Unfortunately, I found several steps resulted in a
> >>> kernel where bluetooth seemed to be substantially borked - to the extent that blueman was non-functional and clicking on
> >>> the tray icon did not start up the blueman-manager application.
> >
> > Running with 5.17.0-0.rc5.102.fc37.x86_64 there doesn't seem to be any
> > problems, well apart from LE device with Privacy/RPA:
> >
> > https://gist.github.com/Vudentz/5792f4989198c7f2994af2e31eb973af
> >
> > Ive tried suspend/resume a couple of times just to see if there is
> > something odd going own, anyway I'm running with the latest userspace
> > so perhaps I really need some old userspace to reproduce this. There
>
> I don't have an old userspace. I built my system from source about four years ago using the methods from Linux From
> Scratch and Beyond Linux From Scratch. I have been updating the packages since then as and when they are released. In a
> way, you could equate my system to one of the rolling releases. It is very up-to-date but at the same time very stable.
>
> This morning I updated to the latest releases of the 5.15 and 5.16 stable series. I've also built and installed the
> latest in the 5.4 and 5.10 stable series. (All four had new releases yesterday.) Bluetooth works as expected on all 4 of
> those kernels. It has also worked on every kernel released by Linus and every stables series kernel in the last four
> years. I usually start trying out a development kernel when rc3, 4 or 5 is released, depending on time available. If I
> find any problems I report them and provide any diagnostics required and test patches.
>
> Bluetooth is unreliable on my system when I boot 5.17-rc kernels. There is little doubt in my mind that something that
> has changed in 5.17 is at the root of this. Whether it is actually a bug in userspace doesn't change the fact that there
> is a regression because of one or more changes in the kernel. As I said yesterday, my understanding is that such
> regressions are frowned upon.
>
> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
Please record the HCI with btmon, it must be producing something since
it records even the mgmt commands.
> In the meantime I've copied this email to [email protected], so they can track this.
Hmm, is that really necessary? 5.17 has not been release tagged yet
and you are already considering it a regression?
> Chris
>
> > might be something with name resolving though, it appears gnome don't
> > show devices if they don't have a proper name (that is not their
> > address) so perhaps that gives the impression there is nothing going
> > on when in fact that all normal, well apart from the fact that names
> > takes way too long to be resolved.
> >
> >>> I also booted into a 5.16.10 kernel and connecting bluetooth devices worked flawlessly. (This was with the unpatched
> >>> bluez daemon)
> >>>
> >>> Chris
> >>>> So the timer doesn't start until the request is sent. but obvoiusly
> >>>> older versions of userspace don't have that fix so they end up
> >>>> cancelling the loading of LTKs, this would explain why reloading the
> >>>> daemon would make it work again.
> >
> >
> >
--
Luiz Augusto von Dentz
On 24.02.22 16:16, Luiz Augusto von Dentz wrote:
> On Thu, Feb 24, 2022 at 2:08 AM Chris Clayton <[email protected]> wrote:
>> On 24/02/2022 08:16, Luiz Augusto von Dentz wrote:
>>> On Wed, Feb 23, 2022 at 9:59 PM Chris Clayton <[email protected]> wrote:
>>>> On 23/02/2022 22:42, Chris Clayton wrote:
> [...]
>> In the meantime I've copied this email to [email protected], so they can track this.
Many thx for this, much appreciated.
> Hmm, is that really necessary? 5.17 has not been release tagged yet
> and you are already considering it a regression?
Sure it's a regression, just one that didn't make it into a release yet.
But tracking regressions in mainline is just as important as tracking
them in stable trees, as it allows Linus to take a look at the known
regressions when he needs to decide between "release another rc" and
"call this finished and release the final". That's why mainline in the
past often was the main focus of regression tracking or the only tree
for which is was performed!
> [...]
Ciao, Thorsten
Hi,
On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
> Please record the HCI with btmon, it must be producing something since
> it records even the mgmt commands.
>
Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
to get the hci trace that Luiz has asked for.
Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
however, I can provide any other diagnostics, please let me know.
Chris
Hi Chris,
On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
>
> Hi,
>
> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
> >> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
>
> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
>
> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
>
> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
>
> > Please record the HCI with btmon, it must be producing something since
> > it records even the mgmt commands.
> >
>
> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
> to get the hci trace that Luiz has asked for.
>
> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
>
> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
>
> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
> however, I can provide any other diagnostics, please let me know.
>
> Chris
Can you try with the following patch:
https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
--
Luiz Augusto von Dentz
Hi Luiz,
On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
> Hi Chris,
>
> On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
>>
>> Hi,
>>
>> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
>>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
>>
>> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
>> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
>>
>> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
>> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
>>
>> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
>> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
>> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
>> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
>>
>>> Please record the HCI with btmon, it must be producing something since
>>> it records even the mgmt commands.
>>>
>>
>> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
>> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
>> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
>> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
>> to get the hci trace that Luiz has asked for.
>>
>> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
>> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
>> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
>> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
>> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
>>
>> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
>> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
>> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
>> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
>> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
>> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
>>
>> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
>> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
>> however, I can provide any other diagnostics, please let me know.
>>
>> Chris
>
> Can you try with the following patch:
>
> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
>
>
Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
reboot they would not connect without an unload and reload of the btusb module.
Chris
Hi Chris,
On Mon, Feb 28, 2022 at 1:02 PM Chris Clayton <[email protected]> wrote:
>
> Hi Luiz,
>
> On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
> > Hi Chris,
> >
> > On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
> >>
> >> Hi,
> >>
> >> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
> >>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
> >>
> >> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
> >> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
> >>
> >> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
> >> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
> >>
> >> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
> >> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
> >> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
> >> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
> >>
> >>> Please record the HCI with btmon, it must be producing something since
> >>> it records even the mgmt commands.
> >>>
> >>
> >> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
> >> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
> >> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
> >> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
> >> to get the hci trace that Luiz has asked for.
> >>
> >> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
> >> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
> >> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
> >> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
> >> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
> >>
> >> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
> >> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
> >> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
> >> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
> >> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
> >> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
> >>
> >> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
> >> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
> >> however, I can provide any other diagnostics, please let me know.
> >>
> >> Chris
> >
> > Can you try with the following patch:
> >
> > https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
> >
> >
> Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
> reboot they would not connect without an unload and reload of the btusb module.
Can you tell us exactly what steps you are using? Are you applying on
top of what, rc6?
--
Luiz Augusto von Dentz
Hi Luiz,
I guess you are hoping for PEBKAC :-)
On 28/02/2022 21:20, Luiz Augusto von Dentz wrote:
> Hi Chris,
>
> On Mon, Feb 28, 2022 at 1:02 PM Chris Clayton <[email protected]> wrote:
>>
>> Hi Luiz,
>>
>> On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
>>> Hi Chris,
>>>
>>> On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
>>>>
>>>> Hi,
>>>>
>>>> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
>>>>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
>>>>
>>>> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
>>>> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
>>>>
>>>> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
>>>> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
>>>>
>>>> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
>>>> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
>>>> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
>>>> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
>>>>
>>>>> Please record the HCI with btmon, it must be producing something since
>>>>> it records even the mgmt commands.
>>>>>
>>>>
>>>> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
>>>> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
>>>> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
>>>> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
>>>> to get the hci trace that Luiz has asked for.
>>>>
>>>> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
>>>> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
>>>> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
>>>> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
>>>> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
>>>>
>>>> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
>>>> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
>>>> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
>>>> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
>>>> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
>>>> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
>>>>
>>>> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
>>>> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
>>>> however, I can provide any other diagnostics, please let me know.
>>>>
>>>> Chris
>>>
>>> Can you try with the following patch:
>>>
>>> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
>>>
>>>
>> Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
>> reboot they would not connect without an unload and reload of the btusb module.
>
> Can you tell us exactly what steps you are using? Are you applying on
> top of what, rc6?
>
Until I got your patch yesterday, I was using a clone of
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git which I update frequently and have been doing so for as
long as I can remember. Just in case there was a hidden flaw in that tree, I took a new clone yesterday, (so yes, the
patch was tested on top of rc6) copied over the .config file and applied the patch. Then I built and installed the
kernel, updated grub, powered off the laptop and booted into the new kernel. Once the laptop had booted and logged in to
my LXQt desktop, I powered on my headphones and a connection was establisehed almost straight away. I powered the
headphones off and the disconnection worked fine.
Knowing that the problem crops up intermittently, I then rebooted the laptop. When the boot was complete, I then powered
on my bluetooth headphones an waited for them to connect to the laptop. After about 20 seconds, a connection had not
been established. I powered off the headphones, used modprobe to unload and then reload the btusb module. When I powered
on the headphones, a connectiin was established within 2 or 3 seconds.
I've booted this laptop countless times over the last few days. Doing the bisect, I didn't mark a commit as good until I
had done five boots and been able to connect my headphones on each boot. What I can say from that work is that two
consecutive boots into a working kernel are very rare. I can't remeber an occasion when it took more than two boots to
establish that a kernel was bad.
Chris
Hi
On 01/03/2022 18:57, Luiz Augusto von Dentz wrote:
> Hi Chris,
>
> On Tue, Mar 1, 2022 at 10:34 AM Luiz Augusto von Dentz
> <[email protected]> wrote:
>>
>> Hi Chris,
>>
>> On Tue, Mar 1, 2022 at 1:26 AM Chris Clayton <[email protected]> wrote:
>>>
>>> Hi Luiz,
>>>
>>> I guess you are hoping for PEBKAC :-)
>>>
>>> On 28/02/2022 21:20, Luiz Augusto von Dentz wrote:
>>>> Hi Chris,
>>>>
>>>> On Mon, Feb 28, 2022 at 1:02 PM Chris Clayton <[email protected]> wrote:
>>>>>
>>>>> Hi Luiz,
>>>>>
>>>>> On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
>>>>>> Hi Chris,
>>>>>>
>>>>>> On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
>>>>>>>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
>>>>>>>
>>>>>>> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
>>>>>>> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
>>>>>>>
>>>>>>> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
>>>>>>> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
>>>>>>>
>>>>>>> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
>>>>>>> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
>>>>>>> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
>>>>>>> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
>>>>>>>
>>>>>>>> Please record the HCI with btmon, it must be producing something since
>>>>>>>> it records even the mgmt commands.
>>>>>>>>
>>>>>>>
>>>>>>> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
>>>>>>> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
>>>>>>> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
>>>>>>> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
>>>>>>> to get the hci trace that Luiz has asked for.
>>>>>>>
>>>>>>> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
>>>>>>> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
>>>>>>> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
>>>>>>> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
>>>>>>> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
>>>>>>>
>>>>>>> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
>>>>>>> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
>>>>>>> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
>>>>>>> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
>>>>>>> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
>>>>>>> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
>>>>>>>
>>>>>>> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
>>>>>>> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
>>>>>>> however, I can provide any other diagnostics, please let me know.
>>>>>>>
>>>>>>> Chris
>>>>>>
>>>>>> Can you try with the following patch:
>>>>>>
>>>>>> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
>>>>>>
>>>>>>
>>>>> Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
>>>>> reboot they would not connect without an unload and reload of the btusb module.
>>>>
>>>> Can you tell us exactly what steps you are using? Are you applying on
>>>> top of what, rc6?
>>>>
>>>
>>> Until I got your patch yesterday, I was using a clone of
>>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git which I update frequently and have been doing so for as
>>> long as I can remember. Just in case there was a hidden flaw in that tree, I took a new clone yesterday, (so yes, the
>>> patch was tested on top of rc6) copied over the .config file and applied the patch. Then I built and installed the
>>> kernel, updated grub, powered off the laptop and booted into the new kernel. Once the laptop had booted and logged in to
>>> my LXQt desktop, I powered on my headphones and a connection was establisehed almost straight away. I powered the
>>> headphones off and the disconnection worked fine.
>>>
>>> Knowing that the problem crops up intermittently, I then rebooted the laptop. When the boot was complete, I then powered
>>> on my bluetooth headphones an waited for them to connect to the laptop. After about 20 seconds, a connection had not
>>> been established. I powered off the headphones, used modprobe to unload and then reload the btusb module. When I powered
>>> on the headphones, a connectiin was established within 2 or 3 seconds.
>
> Ive attempted 5 restart with 5.17.0-0.rc6.109.fc37.x86_64, my headset
> was able to reconnect every single time without any problem. The only
For your five tests, did it connect on the first boot? As I've said, sometimes it fails to connect on the first boot,
but if it succeeds, it has always failed after a power-off and restart. Looking back at the notes I took during the
bisect, I've didn't have a single bisection step where I had to boot more more than twice to ascertain that it was a bad
kernel. As I said, I didn't mark a kernel as good until I'd had five successful boots.
> normally, once from gdm and then another time when gnome is loading,
> but I assume it is normal nowadays since it appears when switching
> session pipewire unregisters its audio endpoints.
>
I don't use pipewire. Prior to 5.17, bluetooth has worked more or less trouble free for at least 4 years. I've read
about pipewire in Linux Magazine but don't see what it would bring to my party except complication.
ends on how bluetoothd is being
> started or something.
>
Did you see the warnings that read "Bluetooth: hci0: unexpected event 0xff length: 5 > 0"? That seems to indicate that
something is sending events that are unexpected. What effect will that have? As I said, according to lshw, my system's
bluetooth hardware is Intel AX201. Is that what you are testing on?
>>> I've booted this laptop countless times over the last few days. Doing the bisect, I didn't mark a commit as good until I
>>> had done five boots and been able to connect my headphones on each boot. What I can say from that work is that two
>>> consecutive boots into a working kernel are very rare. I can't remeber an occasion when it took more than two boots to
>>> establish that a kernel was bad.
>>
>> Do commands such as bluetoothctl power on or scan on works? Try
>> running bluetoothd -dn from a shell (disable bluetooth.service), also
>> are there any settings changed in main.conf?
>>
Sorry, I forgot to answer this question earlier. I haven't changed main.conf. Besides, my bluetooth devices connect
successfully every time with 5.16.11 and 5.15.25 kernels. As I've said before, that strongly suggests that there is a
code regression in 5.17.
>>
>> --
>> Luiz Augusto von Dentz
>
>
>
Hi
On 01/03/2022 18:34, Luiz Augusto von Dentz wrote:
> Hi Chris,
>
> On Tue, Mar 1, 2022 at 1:26 AM Chris Clayton <[email protected]> wrote:
>>
>> Hi Luiz,
>>
>> I guess you are hoping for PEBKAC :-)
>>
>> On 28/02/2022 21:20, Luiz Augusto von Dentz wrote:
>>> Hi Chris,
>>>
>>> On Mon, Feb 28, 2022 at 1:02 PM Chris Clayton <[email protected]> wrote:
>>>>
>>>> Hi Luiz,
>>>>
>>>> On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
>>>>> Hi Chris,
>>>>>
>>>>> On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
>>>>>>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
>>>>>>
>>>>>> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
>>>>>> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
>>>>>>
>>>>>> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
>>>>>> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
>>>>>>
>>>>>> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
>>>>>> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
>>>>>> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
>>>>>> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
>>>>>>
>>>>>>> Please record the HCI with btmon, it must be producing something since
>>>>>>> it records even the mgmt commands.
>>>>>>>
>>>>>>
>>>>>> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
>>>>>> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
>>>>>> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
>>>>>> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
>>>>>> to get the hci trace that Luiz has asked for.
>>>>>>
>>>>>> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
>>>>>> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
>>>>>> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
>>>>>> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
>>>>>> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
>>>>>>
>>>>>> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
>>>>>> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
>>>>>> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
>>>>>> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
>>>>>> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
>>>>>> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
>>>>>>
>>>>>> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
>>>>>> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
>>>>>> however, I can provide any other diagnostics, please let me know.
>>>>>>
>>>>>> Chris
>>>>>
>>>>> Can you try with the following patch:
>>>>>
>>>>> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
>>>>>
>>>>>
>>>> Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
>>>> reboot they would not connect without an unload and reload of the btusb module.
>>>
>>> Can you tell us exactly what steps you are using? Are you applying on
>>> top of what, rc6?
>>>
>>
>> Until I got your patch yesterday, I was using a clone of
>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git which I update frequently and have been doing so for as
>> long as I can remember. Just in case there was a hidden flaw in that tree, I took a new clone yesterday, (so yes, the
>> patch was tested on top of rc6) copied over the .config file and applied the patch. Then I built and installed the
>> kernel, updated grub, powered off the laptop and booted into the new kernel. Once the laptop had booted and logged in to
>> my LXQt desktop, I powered on my headphones and a connection was establisehed almost straight away. I powered the
>> headphones off and the disconnection worked fine.
>>
>> Knowing that the problem crops up intermittently, I then rebooted the laptop. When the boot was complete, I then powered
>> on my bluetooth headphones an waited for them to connect to the laptop. After about 20 seconds, a connection had not
>> been established. I powered off the headphones, used modprobe to unload and then reload the btusb module. When I powered
>> on the headphones, a connectiin was established within 2 or 3 seconds.
>>
>> I've booted this laptop countless times over the last few days. Doing the bisect, I didn't mark a commit as good until I
>> had done five boots and been able to connect my headphones on each boot. What I can say from that work is that two
>> consecutive boots into a working kernel are very rare. I can't remeber an occasion when it took more than two boots to
>> establish that a kernel was bad.
>
> Do commands such as bluetoothctl power on or scan on works?
You're taking about systemd things here but as I said last week, I don't use systemd. I use sysvinit and an associated
set of boot scripts.
Try
> running bluetoothd -dn from a shell (disable bluetooth.service), also
> are there any settings changed in main.conf?
>
OK, I'll try that later, It might be tomorrow because I'm busy with other stuff at the moment.
Chris
>
Hi Chris,
On Tue, Mar 1, 2022 at 2:47 PM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Chris,
>
> On Tue, Mar 1, 2022 at 2:40 PM Luiz Augusto von Dentz
> <[email protected]> wrote:
> >
> > Hi Chris,
> >
> > On Tue, Mar 1, 2022 at 11:40 AM Chris Clayton <[email protected]> wrote:
> > >
> > > Hi
> > >
> > > On 01/03/2022 18:57, Luiz Augusto von Dentz wrote:
> > > > Hi Chris,
> > > >
> > > > On Tue, Mar 1, 2022 at 10:34 AM Luiz Augusto von Dentz
> > > > <[email protected]> wrote:
> > > >>
> > > >> Hi Chris,
> > > >>
> > > >> On Tue, Mar 1, 2022 at 1:26 AM Chris Clayton <[email protected]> wrote:
> > > >>>
> > > >>> Hi Luiz,
> > > >>>
> > > >>> I guess you are hoping for PEBKAC :-)
> > > >>>
> > > >>> On 28/02/2022 21:20, Luiz Augusto von Dentz wrote:
> > > >>>> Hi Chris,
> > > >>>>
> > > >>>> On Mon, Feb 28, 2022 at 1:02 PM Chris Clayton <[email protected]> wrote:
> > > >>>>>
> > > >>>>> Hi Luiz,
> > > >>>>>
> > > >>>>> On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
> > > >>>>>> Hi Chris,
> > > >>>>>>
> > > >>>>>> On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
> > > >>>>>>>
> > > >>>>>>> Hi,
> > > >>>>>>>
> > > >>>>>>> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
> > > >>>>>>>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
> > > >>>>>>>
> > > >>>>>>> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
> > > >>>>>>> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
> > > >>>>>>>
> > > >>>>>>> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
> > > >>>>>>> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
> > > >>>>>>>
> > > >>>>>>> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
> > > >>>>>>> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
> > > >>>>>>> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
> > > >>>>>>> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
> > > >>>>>>>
> > > >>>>>>>> Please record the HCI with btmon, it must be producing something since
> > > >>>>>>>> it records even the mgmt commands.
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
> > > >>>>>>> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
> > > >>>>>>> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
> > > >>>>>>> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
> > > >>>>>>> to get the hci trace that Luiz has asked for.
> > > >>>>>>>
> > > >>>>>>> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
> > > >>>>>>> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
> > > >>>>>>> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
> > > >>>>>>> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
> > > >>>>>>> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
> > > >>>>>>>
> > > >>>>>>> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
> > > >>>>>>> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
> > > >>>>>>> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
> > > >>>>>>> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
> > > >>>>>>> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
> > > >>>>>>> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
> > > >>>>>>>
> > > >>>>>>> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
> > > >>>>>>> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
> > > >>>>>>> however, I can provide any other diagnostics, please let me know.
> > > >>>>>>>
> > > >>>>>>> Chris
> > > >>>>>>
> > > >>>>>> Can you try with the following patch:
> > > >>>>>>
> > > >>>>>> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
> > > >>>>>>
> > > >>>>>>
> > > >>>>> Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
> > > >>>>> reboot they would not connect without an unload and reload of the btusb module.
> > > >>>>
> > > >>>> Can you tell us exactly what steps you are using? Are you applying on
> > > >>>> top of what, rc6?
> > > >>>>
> > > >>>
> > > >>> Until I got your patch yesterday, I was using a clone of
> > > >>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git which I update frequently and have been doing so for as
> > > >>> long as I can remember. Just in case there was a hidden flaw in that tree, I took a new clone yesterday, (so yes, the
> > > >>> patch was tested on top of rc6) copied over the .config file and applied the patch. Then I built and installed the
> > > >>> kernel, updated grub, powered off the laptop and booted into the new kernel. Once the laptop had booted and logged in to
> > > >>> my LXQt desktop, I powered on my headphones and a connection was establisehed almost straight away. I powered the
> > > >>> headphones off and the disconnection worked fine.
> > > >>>
> > > >>> Knowing that the problem crops up intermittently, I then rebooted the laptop. When the boot was complete, I then powered
> > > >>> on my bluetooth headphones an waited for them to connect to the laptop. After about 20 seconds, a connection had not
> > > >>> been established. I powered off the headphones, used modprobe to unload and then reload the btusb module. When I powered
> > > >>> on the headphones, a connectiin was established within 2 or 3 seconds.
> > > >
> > > > Ive attempted 5 restart with 5.17.0-0.rc6.109.fc37.x86_64, my headset
> > > > was able to reconnect every single time without any problem. The only
> > >
> > > For your five tests, did it connect on the first boot? As I've said, sometimes it fails to connect on the first boot,
> > > but if it succeeds, it has always failed after a power-off and restart. Looking back at the notes I took during the
> > > bisect, I've didn't have a single bisection step where I had to boot more more than twice to ascertain that it was a bad
> > > kernel. As I said, I didn't mark a kernel as good until I'd had five successful boots.
> >
> > It did connect every single time.
> >
> > > > normally, once from gdm and then another time when gnome is loading,
> > > > but I assume it is normal nowadays since it appears when switching
> > > > session pipewire unregisters its audio endpoints.
> > > >
> > >
> > > I don't use pipewire. Prior to 5.17, bluetooth has worked more or less trouble free for at least 4 years. I've read
> > > about pipewire in Linux Magazine but don't see what it would bring to my party except complication.
> > >
> > > ends on how bluetoothd is being
> > > > started or something.
> > > >
> > >
> > > Did you see the warnings that read "Bluetooth: hci0: unexpected event 0xff length: 5 > 0"? That seems to indicate that
> > > something is sending events that are unexpected. What effect will that have? As I said, according to lshw, my system's
> > > bluetooth hardware is Intel AX201. Is that what you are testing on?
> >
> > I have an AX200 on my system, AX201 is very similar so Id be surprised
> > if that is the problem, btw Ive also got some unexpected events but
> > that didn't stop the headset to reconnect.
> >
> > > >>> I've booted this laptop countless times over the last few days. Doing the bisect, I didn't mark a commit as good until I
> > > >>> had done five boots and been able to connect my headphones on each boot. What I can say from that work is that two
> > > >>> consecutive boots into a working kernel are very rare. I can't remeber an occasion when it took more than two boots to
> > > >>> establish that a kernel was bad.
> > > >>
> > > >> Do commands such as bluetoothctl power on or scan on works? Try
> > > >> running bluetoothd -dn from a shell (disable bluetooth.service), also
> > > >> are there any settings changed in main.conf?
> > > >>
> > >
> > > Sorry, I forgot to answer this question earlier. I haven't changed main.conf. Besides, my bluetooth devices connect
> > > successfully every time with 5.16.11 and 5.15.25 kernels. As I've said before, that strongly suggests that there is a
> > > code regression in 5.17.
> >
> > Not saying there isn't something wrong, we have sent a couple of fixes
> > that doesn't seem to be merged yet, and we are working on another one
> > for fixing the scan:
> >
> > https://lore.kernel.org/linux-bluetooth/[email protected]/T/
>
> Btw, are you by any chance doing something like hciconfig hci0 up on
> your init scripts?
Looks like I was able to reproduce but I still don't know the cause,
anyway the symptom is the following:
[23412.856410] remove_uuid:2273: hci0: sock 0000000049dcd70a
[23412.856467] mgmt_class_complete:2174: hci0: err 0
[23412.856470] mgmt_cmd_complete:176: sock 00000000d63e046a
It looks like the cmd->sk is wrong/corrupted, what is even more
strange is that the socket pointer does seem to match previously
connected MGMT clients and after a few attempts with btmgmt> power on
it does come back to life, very bizarre...
Can you try to enable some kernel debugs before you start bluetoothd:
echo "file net/bluetooth/mgmt.c +pfl" > /sys/kernel/debug/dynamic_debug/control
echo "file net/bluetooth/mgmt_util.c +pfl" >
/sys/kernel/debug/dynamic_debug/control
Maybe that should give us a clue what triggers it.
> > > >>
> > > >> --
> > > >> Luiz Augusto von Dentz
> > > >
> > > >
> > > >
> >
> >
> >
> > --
> > Luiz Augusto von Dentz
>
>
>
> --
> Luiz Augusto von Dentz
--
Luiz Augusto von Dentz
Hi Chris,
On Tue, Mar 1, 2022 at 2:40 PM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Chris,
>
> On Tue, Mar 1, 2022 at 11:40 AM Chris Clayton <[email protected]> wrote:
> >
> > Hi
> >
> > On 01/03/2022 18:57, Luiz Augusto von Dentz wrote:
> > > Hi Chris,
> > >
> > > On Tue, Mar 1, 2022 at 10:34 AM Luiz Augusto von Dentz
> > > <[email protected]> wrote:
> > >>
> > >> Hi Chris,
> > >>
> > >> On Tue, Mar 1, 2022 at 1:26 AM Chris Clayton <[email protected]> wrote:
> > >>>
> > >>> Hi Luiz,
> > >>>
> > >>> I guess you are hoping for PEBKAC :-)
> > >>>
> > >>> On 28/02/2022 21:20, Luiz Augusto von Dentz wrote:
> > >>>> Hi Chris,
> > >>>>
> > >>>> On Mon, Feb 28, 2022 at 1:02 PM Chris Clayton <[email protected]> wrote:
> > >>>>>
> > >>>>> Hi Luiz,
> > >>>>>
> > >>>>> On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
> > >>>>>> Hi Chris,
> > >>>>>>
> > >>>>>> On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
> > >>>>>>>
> > >>>>>>> Hi,
> > >>>>>>>
> > >>>>>>> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
> > >>>>>>>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
> > >>>>>>>
> > >>>>>>> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
> > >>>>>>> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
> > >>>>>>>
> > >>>>>>> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
> > >>>>>>> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
> > >>>>>>>
> > >>>>>>> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
> > >>>>>>> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
> > >>>>>>> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
> > >>>>>>> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
> > >>>>>>>
> > >>>>>>>> Please record the HCI with btmon, it must be producing something since
> > >>>>>>>> it records even the mgmt commands.
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
> > >>>>>>> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
> > >>>>>>> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
> > >>>>>>> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
> > >>>>>>> to get the hci trace that Luiz has asked for.
> > >>>>>>>
> > >>>>>>> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
> > >>>>>>> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
> > >>>>>>> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
> > >>>>>>> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
> > >>>>>>> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
> > >>>>>>>
> > >>>>>>> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
> > >>>>>>> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
> > >>>>>>> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
> > >>>>>>> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
> > >>>>>>> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
> > >>>>>>> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
> > >>>>>>>
> > >>>>>>> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
> > >>>>>>> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
> > >>>>>>> however, I can provide any other diagnostics, please let me know.
> > >>>>>>>
> > >>>>>>> Chris
> > >>>>>>
> > >>>>>> Can you try with the following patch:
> > >>>>>>
> > >>>>>> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
> > >>>>>>
> > >>>>>>
> > >>>>> Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
> > >>>>> reboot they would not connect without an unload and reload of the btusb module.
> > >>>>
> > >>>> Can you tell us exactly what steps you are using? Are you applying on
> > >>>> top of what, rc6?
> > >>>>
> > >>>
> > >>> Until I got your patch yesterday, I was using a clone of
> > >>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git which I update frequently and have been doing so for as
> > >>> long as I can remember. Just in case there was a hidden flaw in that tree, I took a new clone yesterday, (so yes, the
> > >>> patch was tested on top of rc6) copied over the .config file and applied the patch. Then I built and installed the
> > >>> kernel, updated grub, powered off the laptop and booted into the new kernel. Once the laptop had booted and logged in to
> > >>> my LXQt desktop, I powered on my headphones and a connection was establisehed almost straight away. I powered the
> > >>> headphones off and the disconnection worked fine.
> > >>>
> > >>> Knowing that the problem crops up intermittently, I then rebooted the laptop. When the boot was complete, I then powered
> > >>> on my bluetooth headphones an waited for them to connect to the laptop. After about 20 seconds, a connection had not
> > >>> been established. I powered off the headphones, used modprobe to unload and then reload the btusb module. When I powered
> > >>> on the headphones, a connectiin was established within 2 or 3 seconds.
> > >
> > > Ive attempted 5 restart with 5.17.0-0.rc6.109.fc37.x86_64, my headset
> > > was able to reconnect every single time without any problem. The only
> >
> > For your five tests, did it connect on the first boot? As I've said, sometimes it fails to connect on the first boot,
> > but if it succeeds, it has always failed after a power-off and restart. Looking back at the notes I took during the
> > bisect, I've didn't have a single bisection step where I had to boot more more than twice to ascertain that it was a bad
> > kernel. As I said, I didn't mark a kernel as good until I'd had five successful boots.
>
> It did connect every single time.
>
> > > normally, once from gdm and then another time when gnome is loading,
> > > but I assume it is normal nowadays since it appears when switching
> > > session pipewire unregisters its audio endpoints.
> > >
> >
> > I don't use pipewire. Prior to 5.17, bluetooth has worked more or less trouble free for at least 4 years. I've read
> > about pipewire in Linux Magazine but don't see what it would bring to my party except complication.
> >
> > ends on how bluetoothd is being
> > > started or something.
> > >
> >
> > Did you see the warnings that read "Bluetooth: hci0: unexpected event 0xff length: 5 > 0"? That seems to indicate that
> > something is sending events that are unexpected. What effect will that have? As I said, according to lshw, my system's
> > bluetooth hardware is Intel AX201. Is that what you are testing on?
>
> I have an AX200 on my system, AX201 is very similar so Id be surprised
> if that is the problem, btw Ive also got some unexpected events but
> that didn't stop the headset to reconnect.
>
> > >>> I've booted this laptop countless times over the last few days. Doing the bisect, I didn't mark a commit as good until I
> > >>> had done five boots and been able to connect my headphones on each boot. What I can say from that work is that two
> > >>> consecutive boots into a working kernel are very rare. I can't remeber an occasion when it took more than two boots to
> > >>> establish that a kernel was bad.
> > >>
> > >> Do commands such as bluetoothctl power on or scan on works? Try
> > >> running bluetoothd -dn from a shell (disable bluetooth.service), also
> > >> are there any settings changed in main.conf?
> > >>
> >
> > Sorry, I forgot to answer this question earlier. I haven't changed main.conf. Besides, my bluetooth devices connect
> > successfully every time with 5.16.11 and 5.15.25 kernels. As I've said before, that strongly suggests that there is a
> > code regression in 5.17.
>
> Not saying there isn't something wrong, we have sent a couple of fixes
> that doesn't seem to be merged yet, and we are working on another one
> for fixing the scan:
>
> https://lore.kernel.org/linux-bluetooth/[email protected]/T/
Btw, are you by any chance doing something like hciconfig hci0 up on
your init scripts?
> > >>
> > >> --
> > >> Luiz Augusto von Dentz
> > >
> > >
> > >
>
>
>
> --
> Luiz Augusto von Dentz
--
Luiz Augusto von Dentz
On 01/03/2022 22:47, Luiz Augusto von Dentz wrote:
> Hi Chris,
>
> On Tue, Mar 1, 2022 at 2:40 PM Luiz Augusto von Dentz
> <[email protected]> wrote:
>>
>> Hi Chris,
>>
>> On Tue, Mar 1, 2022 at 11:40 AM Chris Clayton <[email protected]> wrote:
>>>
>>> Hi
>>>
>>> On 01/03/2022 18:57, Luiz Augusto von Dentz wrote:
>>>> Hi Chris,
>>>>
>>>> On Tue, Mar 1, 2022 at 10:34 AM Luiz Augusto von Dentz
>>>> <[email protected]> wrote:
>>>>>
>>>>> Hi Chris,
>>>>>
>>>>> On Tue, Mar 1, 2022 at 1:26 AM Chris Clayton <[email protected]> wrote:
>>>>>>
>>>>>> Hi Luiz,
>>>>>>
>>>>>> I guess you are hoping for PEBKAC :-)
>>>>>>
>>>>>> On 28/02/2022 21:20, Luiz Augusto von Dentz wrote:
>>>>>>> Hi Chris,
>>>>>>>
>>>>>>> On Mon, Feb 28, 2022 at 1:02 PM Chris Clayton <[email protected]> wrote:
>>>>>>>>
>>>>>>>> Hi Luiz,
>>>>>>>>
>>>>>>>> On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
>>>>>>>>> Hi Chris,
>>>>>>>>>
>>>>>>>>> On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
>>>>>>>>>>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
>>>>>>>>>>
>>>>>>>>>> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
>>>>>>>>>> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
>>>>>>>>>>
>>>>>>>>>> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
>>>>>>>>>> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
>>>>>>>>>>
>>>>>>>>>> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
>>>>>>>>>> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
>>>>>>>>>> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
>>>>>>>>>> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
>>>>>>>>>>
>>>>>>>>>>> Please record the HCI with btmon, it must be producing something since
>>>>>>>>>>> it records even the mgmt commands.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
>>>>>>>>>> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
>>>>>>>>>> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
>>>>>>>>>> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
>>>>>>>>>> to get the hci trace that Luiz has asked for.
>>>>>>>>>>
>>>>>>>>>> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
>>>>>>>>>> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
>>>>>>>>>> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
>>>>>>>>>> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
>>>>>>>>>> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
>>>>>>>>>>
>>>>>>>>>> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
>>>>>>>>>> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
>>>>>>>>>> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
>>>>>>>>>> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
>>>>>>>>>> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
>>>>>>>>>> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
>>>>>>>>>>
>>>>>>>>>> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
>>>>>>>>>> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
>>>>>>>>>> however, I can provide any other diagnostics, please let me know.
>>>>>>>>>>
>>>>>>>>>> Chris
>>>>>>>>>
>>>>>>>>> Can you try with the following patch:
>>>>>>>>>
>>>>>>>>> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
>>>>>>>> reboot they would not connect without an unload and reload of the btusb module.
>>>>>>>
>>>>>>> Can you tell us exactly what steps you are using? Are you applying on
>>>>>>> top of what, rc6?
>>>>>>>
>>>>>>
>>>>>> Until I got your patch yesterday, I was using a clone of
>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git which I update frequently and have been doing so for as
>>>>>> long as I can remember. Just in case there was a hidden flaw in that tree, I took a new clone yesterday, (so yes, the
>>>>>> patch was tested on top of rc6) copied over the .config file and applied the patch. Then I built and installed the
>>>>>> kernel, updated grub, powered off the laptop and booted into the new kernel. Once the laptop had booted and logged in to
>>>>>> my LXQt desktop, I powered on my headphones and a connection was establisehed almost straight away. I powered the
>>>>>> headphones off and the disconnection worked fine.
>>>>>>
>>>>>> Knowing that the problem crops up intermittently, I then rebooted the laptop. When the boot was complete, I then powered
>>>>>> on my bluetooth headphones an waited for them to connect to the laptop. After about 20 seconds, a connection had not
>>>>>> been established. I powered off the headphones, used modprobe to unload and then reload the btusb module. When I powered
>>>>>> on the headphones, a connectiin was established within 2 or 3 seconds.
>>>>
>>>> Ive attempted 5 restart with 5.17.0-0.rc6.109.fc37.x86_64, my headset
>>>> was able to reconnect every single time without any problem. The only
>>>
>>> For your five tests, did it connect on the first boot? As I've said, sometimes it fails to connect on the first boot,
>>> but if it succeeds, it has always failed after a power-off and restart. Looking back at the notes I took during the
>>> bisect, I've didn't have a single bisection step where I had to boot more more than twice to ascertain that it was a bad
>>> kernel. As I said, I didn't mark a kernel as good until I'd had five successful boots.
>>
>> It did connect every single time.
>>
>>>> normally, once from gdm and then another time when gnome is loading,
>>>> but I assume it is normal nowadays since it appears when switching
>>>> session pipewire unregisters its audio endpoints.
>>>>
>>>
>>> I don't use pipewire. Prior to 5.17, bluetooth has worked more or less trouble free for at least 4 years. I've read
>>> about pipewire in Linux Magazine but don't see what it would bring to my party except complication.
>>>
>>> ends on how bluetoothd is being
>>>> started or something.
>>>>
>>>
>>> Did you see the warnings that read "Bluetooth: hci0: unexpected event 0xff length: 5 > 0"? That seems to indicate that
>>> something is sending events that are unexpected. What effect will that have? As I said, according to lshw, my system's
>>> bluetooth hardware is Intel AX201. Is that what you are testing on?
>>
>> I have an AX200 on my system, AX201 is very similar so Id be surprised
>> if that is the problem, btw Ive also got some unexpected events but
>> that didn't stop the headset to reconnect.
>>
>>>>>> I've booted this laptop countless times over the last few days. Doing the bisect, I didn't mark a commit as good until I
>>>>>> had done five boots and been able to connect my headphones on each boot. What I can say from that work is that two
>>>>>> consecutive boots into a working kernel are very rare. I can't remeber an occasion when it took more than two boots to
>>>>>> establish that a kernel was bad.
>>>>>
>>>>> Do commands such as bluetoothctl power on or scan on works? Try
>>>>> running bluetoothd -dn from a shell (disable bluetooth.service), also
>>>>> are there any settings changed in main.conf?
>>>>>
>>>
>>> Sorry, I forgot to answer this question earlier. I haven't changed main.conf. Besides, my bluetooth devices connect
>>> successfully every time with 5.16.11 and 5.15.25 kernels. As I've said before, that strongly suggests that there is a
>>> code regression in 5.17.
>>
>> Not saying there isn't something wrong, we have sent a couple of fixes
>> that doesn't seem to be merged yet, and we are working on another one
>> for fixing the scan:
>>
>> https://lore.kernel.org/linux-bluetooth/[email protected]/T/
>
> Btw, are you by any chance doing something like hciconfig hci0 up on
> your init scripts?
>
Yes , that is included in /etc/init.d/bluetooth.
>>>>>
>>>>> --
>>>>> Luiz Augusto von Dentz
>>>>
>>>>
>>>>
>>
>>
>>
>> --
>> Luiz Augusto von Dentz
>
>
>
Hi Chris,
On Tue, Mar 1, 2022 at 1:26 AM Chris Clayton <[email protected]> wrote:
>
> Hi Luiz,
>
> I guess you are hoping for PEBKAC :-)
>
> On 28/02/2022 21:20, Luiz Augusto von Dentz wrote:
> > Hi Chris,
> >
> > On Mon, Feb 28, 2022 at 1:02 PM Chris Clayton <[email protected]> wrote:
> >>
> >> Hi Luiz,
> >>
> >> On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
> >>> Hi Chris,
> >>>
> >>> On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
> >>>>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
> >>>>
> >>>> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
> >>>> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
> >>>>
> >>>> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
> >>>> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
> >>>>
> >>>> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
> >>>> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
> >>>> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
> >>>> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
> >>>>
> >>>>> Please record the HCI with btmon, it must be producing something since
> >>>>> it records even the mgmt commands.
> >>>>>
> >>>>
> >>>> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
> >>>> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
> >>>> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
> >>>> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
> >>>> to get the hci trace that Luiz has asked for.
> >>>>
> >>>> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
> >>>> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
> >>>> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
> >>>> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
> >>>> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
> >>>>
> >>>> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
> >>>> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
> >>>> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
> >>>> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
> >>>> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
> >>>> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
> >>>>
> >>>> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
> >>>> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
> >>>> however, I can provide any other diagnostics, please let me know.
> >>>>
> >>>> Chris
> >>>
> >>> Can you try with the following patch:
> >>>
> >>> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
> >>>
> >>>
> >> Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
> >> reboot they would not connect without an unload and reload of the btusb module.
> >
> > Can you tell us exactly what steps you are using? Are you applying on
> > top of what, rc6?
> >
>
> Until I got your patch yesterday, I was using a clone of
> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git which I update frequently and have been doing so for as
> long as I can remember. Just in case there was a hidden flaw in that tree, I took a new clone yesterday, (so yes, the
> patch was tested on top of rc6) copied over the .config file and applied the patch. Then I built and installed the
> kernel, updated grub, powered off the laptop and booted into the new kernel. Once the laptop had booted and logged in to
> my LXQt desktop, I powered on my headphones and a connection was establisehed almost straight away. I powered the
> headphones off and the disconnection worked fine.
>
> Knowing that the problem crops up intermittently, I then rebooted the laptop. When the boot was complete, I then powered
> on my bluetooth headphones an waited for them to connect to the laptop. After about 20 seconds, a connection had not
> been established. I powered off the headphones, used modprobe to unload and then reload the btusb module. When I powered
> on the headphones, a connectiin was established within 2 or 3 seconds.
>
> I've booted this laptop countless times over the last few days. Doing the bisect, I didn't mark a commit as good until I
> had done five boots and been able to connect my headphones on each boot. What I can say from that work is that two
> consecutive boots into a working kernel are very rare. I can't remeber an occasion when it took more than two boots to
> establish that a kernel was bad.
Do commands such as bluetoothctl power on or scan on works? Try
running bluetoothd -dn from a shell (disable bluetooth.service), also
are there any settings changed in main.conf?
--
Luiz Augusto von Dentz
Hi Chris,
On Tue, Mar 1, 2022 at 10:34 AM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Chris,
>
> On Tue, Mar 1, 2022 at 1:26 AM Chris Clayton <[email protected]> wrote:
> >
> > Hi Luiz,
> >
> > I guess you are hoping for PEBKAC :-)
> >
> > On 28/02/2022 21:20, Luiz Augusto von Dentz wrote:
> > > Hi Chris,
> > >
> > > On Mon, Feb 28, 2022 at 1:02 PM Chris Clayton <[email protected]> wrote:
> > >>
> > >> Hi Luiz,
> > >>
> > >> On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
> > >>> Hi Chris,
> > >>>
> > >>> On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
> > >>>>
> > >>>> Hi,
> > >>>>
> > >>>> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
> > >>>>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
> > >>>>
> > >>>> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
> > >>>> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
> > >>>>
> > >>>> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
> > >>>> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
> > >>>>
> > >>>> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
> > >>>> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
> > >>>> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
> > >>>> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
> > >>>>
> > >>>>> Please record the HCI with btmon, it must be producing something since
> > >>>>> it records even the mgmt commands.
> > >>>>>
> > >>>>
> > >>>> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
> > >>>> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
> > >>>> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
> > >>>> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
> > >>>> to get the hci trace that Luiz has asked for.
> > >>>>
> > >>>> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
> > >>>> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
> > >>>> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
> > >>>> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
> > >>>> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
> > >>>>
> > >>>> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
> > >>>> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
> > >>>> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
> > >>>> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
> > >>>> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
> > >>>> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
> > >>>>
> > >>>> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
> > >>>> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
> > >>>> however, I can provide any other diagnostics, please let me know.
> > >>>>
> > >>>> Chris
> > >>>
> > >>> Can you try with the following patch:
> > >>>
> > >>> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
> > >>>
> > >>>
> > >> Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
> > >> reboot they would not connect without an unload and reload of the btusb module.
> > >
> > > Can you tell us exactly what steps you are using? Are you applying on
> > > top of what, rc6?
> > >
> >
> > Until I got your patch yesterday, I was using a clone of
> > git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git which I update frequently and have been doing so for as
> > long as I can remember. Just in case there was a hidden flaw in that tree, I took a new clone yesterday, (so yes, the
> > patch was tested on top of rc6) copied over the .config file and applied the patch. Then I built and installed the
> > kernel, updated grub, powered off the laptop and booted into the new kernel. Once the laptop had booted and logged in to
> > my LXQt desktop, I powered on my headphones and a connection was establisehed almost straight away. I powered the
> > headphones off and the disconnection worked fine.
> >
> > Knowing that the problem crops up intermittently, I then rebooted the laptop. When the boot was complete, I then powered
> > on my bluetooth headphones an waited for them to connect to the laptop. After about 20 seconds, a connection had not
> > been established. I powered off the headphones, used modprobe to unload and then reload the btusb module. When I powered
> > on the headphones, a connectiin was established within 2 or 3 seconds.
Ive attempted 5 restart with 5.17.0-0.rc6.109.fc37.x86_64, my headset
was able to reconnect every single time without any problem. The only
think that was a little bit strange was that it reconnects two
normally, once from gdm and then another time when gnome is loading,
but I assume it is normal nowadays since it appears when switching
session pipewire unregisters its audio endpoints.
Perhaps it is some race that depends on how bluetoothd is being
started or something.
> > I've booted this laptop countless times over the last few days. Doing the bisect, I didn't mark a commit as good until I
> > had done five boots and been able to connect my headphones on each boot. What I can say from that work is that two
> > consecutive boots into a working kernel are very rare. I can't remeber an occasion when it took more than two boots to
> > establish that a kernel was bad.
>
> Do commands such as bluetoothctl power on or scan on works? Try
> running bluetoothd -dn from a shell (disable bluetooth.service), also
> are there any settings changed in main.conf?
>
>
> --
> Luiz Augusto von Dentz
--
Luiz Augusto von Dentz
Hi Chris,
On Tue, Mar 1, 2022 at 11:40 AM Chris Clayton <[email protected]> wrote:
>
> Hi
>
> On 01/03/2022 18:57, Luiz Augusto von Dentz wrote:
> > Hi Chris,
> >
> > On Tue, Mar 1, 2022 at 10:34 AM Luiz Augusto von Dentz
> > <[email protected]> wrote:
> >>
> >> Hi Chris,
> >>
> >> On Tue, Mar 1, 2022 at 1:26 AM Chris Clayton <[email protected]> wrote:
> >>>
> >>> Hi Luiz,
> >>>
> >>> I guess you are hoping for PEBKAC :-)
> >>>
> >>> On 28/02/2022 21:20, Luiz Augusto von Dentz wrote:
> >>>> Hi Chris,
> >>>>
> >>>> On Mon, Feb 28, 2022 at 1:02 PM Chris Clayton <[email protected]> wrote:
> >>>>>
> >>>>> Hi Luiz,
> >>>>>
> >>>>> On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
> >>>>>> Hi Chris,
> >>>>>>
> >>>>>> On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
> >>>>>>>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
> >>>>>>>
> >>>>>>> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
> >>>>>>> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
> >>>>>>>
> >>>>>>> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
> >>>>>>> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
> >>>>>>>
> >>>>>>> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
> >>>>>>> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
> >>>>>>> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
> >>>>>>> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
> >>>>>>>
> >>>>>>>> Please record the HCI with btmon, it must be producing something since
> >>>>>>>> it records even the mgmt commands.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
> >>>>>>> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
> >>>>>>> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
> >>>>>>> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
> >>>>>>> to get the hci trace that Luiz has asked for.
> >>>>>>>
> >>>>>>> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
> >>>>>>> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
> >>>>>>> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
> >>>>>>> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
> >>>>>>> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
> >>>>>>>
> >>>>>>> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
> >>>>>>> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
> >>>>>>> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
> >>>>>>> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
> >>>>>>> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
> >>>>>>> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
> >>>>>>>
> >>>>>>> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
> >>>>>>> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
> >>>>>>> however, I can provide any other diagnostics, please let me know.
> >>>>>>>
> >>>>>>> Chris
> >>>>>>
> >>>>>> Can you try with the following patch:
> >>>>>>
> >>>>>> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
> >>>>>>
> >>>>>>
> >>>>> Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
> >>>>> reboot they would not connect without an unload and reload of the btusb module.
> >>>>
> >>>> Can you tell us exactly what steps you are using? Are you applying on
> >>>> top of what, rc6?
> >>>>
> >>>
> >>> Until I got your patch yesterday, I was using a clone of
> >>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git which I update frequently and have been doing so for as
> >>> long as I can remember. Just in case there was a hidden flaw in that tree, I took a new clone yesterday, (so yes, the
> >>> patch was tested on top of rc6) copied over the .config file and applied the patch. Then I built and installed the
> >>> kernel, updated grub, powered off the laptop and booted into the new kernel. Once the laptop had booted and logged in to
> >>> my LXQt desktop, I powered on my headphones and a connection was establisehed almost straight away. I powered the
> >>> headphones off and the disconnection worked fine.
> >>>
> >>> Knowing that the problem crops up intermittently, I then rebooted the laptop. When the boot was complete, I then powered
> >>> on my bluetooth headphones an waited for them to connect to the laptop. After about 20 seconds, a connection had not
> >>> been established. I powered off the headphones, used modprobe to unload and then reload the btusb module. When I powered
> >>> on the headphones, a connectiin was established within 2 or 3 seconds.
> >
> > Ive attempted 5 restart with 5.17.0-0.rc6.109.fc37.x86_64, my headset
> > was able to reconnect every single time without any problem. The only
>
> For your five tests, did it connect on the first boot? As I've said, sometimes it fails to connect on the first boot,
> but if it succeeds, it has always failed after a power-off and restart. Looking back at the notes I took during the
> bisect, I've didn't have a single bisection step where I had to boot more more than twice to ascertain that it was a bad
> kernel. As I said, I didn't mark a kernel as good until I'd had five successful boots.
It did connect every single time.
> > normally, once from gdm and then another time when gnome is loading,
> > but I assume it is normal nowadays since it appears when switching
> > session pipewire unregisters its audio endpoints.
> >
>
> I don't use pipewire. Prior to 5.17, bluetooth has worked more or less trouble free for at least 4 years. I've read
> about pipewire in Linux Magazine but don't see what it would bring to my party except complication.
>
> ends on how bluetoothd is being
> > started or something.
> >
>
> Did you see the warnings that read "Bluetooth: hci0: unexpected event 0xff length: 5 > 0"? That seems to indicate that
> something is sending events that are unexpected. What effect will that have? As I said, according to lshw, my system's
> bluetooth hardware is Intel AX201. Is that what you are testing on?
I have an AX200 on my system, AX201 is very similar so Id be surprised
if that is the problem, btw Ive also got some unexpected events but
that didn't stop the headset to reconnect.
> >>> I've booted this laptop countless times over the last few days. Doing the bisect, I didn't mark a commit as good until I
> >>> had done five boots and been able to connect my headphones on each boot. What I can say from that work is that two
> >>> consecutive boots into a working kernel are very rare. I can't remeber an occasion when it took more than two boots to
> >>> establish that a kernel was bad.
> >>
> >> Do commands such as bluetoothctl power on or scan on works? Try
> >> running bluetoothd -dn from a shell (disable bluetooth.service), also
> >> are there any settings changed in main.conf?
> >>
>
> Sorry, I forgot to answer this question earlier. I haven't changed main.conf. Besides, my bluetooth devices connect
> successfully every time with 5.16.11 and 5.15.25 kernels. As I've said before, that strongly suggests that there is a
> code regression in 5.17.
Not saying there isn't something wrong, we have sent a couple of fixes
that doesn't seem to be merged yet, and we are working on another one
for fixing the scan:
https://lore.kernel.org/linux-bluetooth/[email protected]/T/
> >>
> >> --
> >> Luiz Augusto von Dentz
> >
> >
> >
--
Luiz Augusto von Dentz
Hi Chris,
On Tue, Mar 1, 2022 at 6:31 PM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Chris,
>
> On Tue, Mar 1, 2022 at 2:47 PM Luiz Augusto von Dentz
> <[email protected]> wrote:
> >
> > Hi Chris,
> >
> > On Tue, Mar 1, 2022 at 2:40 PM Luiz Augusto von Dentz
> > <[email protected]> wrote:
> > >
> > > Hi Chris,
> > >
> > > On Tue, Mar 1, 2022 at 11:40 AM Chris Clayton <[email protected]> wrote:
> > > >
> > > > Hi
> > > >
> > > > On 01/03/2022 18:57, Luiz Augusto von Dentz wrote:
> > > > > Hi Chris,
> > > > >
> > > > > On Tue, Mar 1, 2022 at 10:34 AM Luiz Augusto von Dentz
> > > > > <[email protected]> wrote:
> > > > >>
> > > > >> Hi Chris,
> > > > >>
> > > > >> On Tue, Mar 1, 2022 at 1:26 AM Chris Clayton <[email protected]> wrote:
> > > > >>>
> > > > >>> Hi Luiz,
> > > > >>>
> > > > >>> I guess you are hoping for PEBKAC :-)
> > > > >>>
> > > > >>> On 28/02/2022 21:20, Luiz Augusto von Dentz wrote:
> > > > >>>> Hi Chris,
> > > > >>>>
> > > > >>>> On Mon, Feb 28, 2022 at 1:02 PM Chris Clayton <[email protected]> wrote:
> > > > >>>>>
> > > > >>>>> Hi Luiz,
> > > > >>>>>
> > > > >>>>> On 28/02/2022 19:34, Luiz Augusto von Dentz wrote:
> > > > >>>>>> Hi Chris,
> > > > >>>>>>
> > > > >>>>>> On Sat, Feb 26, 2022 at 12:04 AM Chris Clayton <[email protected]> wrote:
> > > > >>>>>>>
> > > > >>>>>>> Hi,
> > > > >>>>>>>
> > > > >>>>>>> On 24/02/2022 15:16, Luiz Augusto von Dentz wrote:
> > > > >>>>>>>>> I'll try another bisection today, but limit its range to changes made in the net/bluetooth directory.
> > > > >>>>>>>
> > > > >>>>>>> That bisection has proved very difficult because the bluetooth "service" in kernels at some steps of the bisection were
> > > > >>>>>>> completely borked to the extent that blueman's device-manager application wouldn't start and emitted the messages:
> > > > >>>>>>>
> > > > >>>>>>> blueman-manager 12.00.37 ERROR Manager:137 on_dbus_name_appeared: Default adapter not found, trying first available.
> > > > >>>>>>> blueman-manager 12.00.37 ERROR Manager:141 on_dbus_name_appeared: No adapter(s) found, exiting
> > > > >>>>>>>
> > > > >>>>>>> Obviously, I don't know whether the problem I am trying to pinpoint is hiding behind this more fundamental problem with
> > > > >>>>>>> the bluetooth "service", so being unable to say whether that kernel was good or bad, I had to skip. There seems to be a
> > > > >>>>>>> batch of commits that mean that, whilst the kernel builds okay, hunting down a bluetooth-related problem is not
> > > > >>>>>>> possible. Eventually and I cursed and gave up. Whatever was causing this breakage has obviously been fixed.
> > > > >>>>>>>
> > > > >>>>>>>> Please record the HCI with btmon, it must be producing something since
> > > > >>>>>>>> it records even the mgmt commands.
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> Refreshed by a good night's sleep, I started another bisection (between 5.16 and 5.17-rc1) yesterday morning but this
> > > > >>>>>>> time did not limit it to net/bluetooth. That was going okay until I ran into what I assume is the same batch of borked
> > > > >>>>>>> kernels. I've been more persistent this time but have just had a run of 16 steps in which the bluetooth support in the
> > > > >>>>>>> kernel is broken so badly that testing bluetooth is not possible. I will push on today, but I've suspended that activity
> > > > >>>>>>> to get the hci trace that Luiz has asked for.
> > > > >>>>>>>
> > > > >>>>>>> Using information from the bisect, I built a kernel that had tested as bad (but not borked). The commit is
> > > > >>>>>>> f2b551fad8d8f2ac5e1f810ad595298381e0b0c5. As I've mentioned before, the problem with devices not connecting is
> > > > >>>>>>> intermittent - for a given kernel, sometimes a connection works and other times it doesn't. On the first boot of this
> > > > >>>>>>> kernel, my bluetooth devices could connect, Attached are 4 files related to this - the output from btmon, and the
> > > > >>>>>>> related portions of daemon.log, kern.log and sys.log from /var/log/. Each of the these files is suffixed with ".good".
> > > > >>>>>>>
> > > > >>>>>>> I then powered down the laptop and booted into the same kernel. This time the bluetooth devices could not connect. Four
> > > > >>>>>>> more files are attached for this boot and are suffixed with ".bad". I said in an earlier email that when connection
> > > > >>>>>>> fails, there is no output from btmon, so that log is empty. That's still the case, but I guess that fact itself is a
> > > > >>>>>>> clue to what the problem might be. What I can add, however, is that if, in that same bad kernel, I unload and then
> > > > >>>>>>> reload the btusb module, connections start to work. Maybe that too is a clue. The same unload/load process revives
> > > > >>>>>>> bluetooth on a kernel built after a pull of Linus' latest and greatest this morning.
> > > > >>>>>>>
> > > > >>>>>>> Since I now have a workround, I'm going stop the current bisection that I was doing. I've done another couple of steps
> > > > >>>>>>> this morning and both produced kernels on which I could not test bluetooth and had to tell git bisect to skip. If
> > > > >>>>>>> however, I can provide any other diagnostics, please let me know.
> > > > >>>>>>>
> > > > >>>>>>> Chris
> > > > >>>>>>
> > > > >>>>>> Can you try with the following patch:
> > > > >>>>>>
> > > > >>>>>> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>> Sorry, that patch has made no difference. After the first boot my headphones connected okay, but after a power-down and
> > > > >>>>> reboot they would not connect without an unload and reload of the btusb module.
> > > > >>>>
> > > > >>>> Can you tell us exactly what steps you are using? Are you applying on
> > > > >>>> top of what, rc6?
> > > > >>>>
> > > > >>>
> > > > >>> Until I got your patch yesterday, I was using a clone of
> > > > >>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git which I update frequently and have been doing so for as
> > > > >>> long as I can remember. Just in case there was a hidden flaw in that tree, I took a new clone yesterday, (so yes, the
> > > > >>> patch was tested on top of rc6) copied over the .config file and applied the patch. Then I built and installed the
> > > > >>> kernel, updated grub, powered off the laptop and booted into the new kernel. Once the laptop had booted and logged in to
> > > > >>> my LXQt desktop, I powered on my headphones and a connection was establisehed almost straight away. I powered the
> > > > >>> headphones off and the disconnection worked fine.
> > > > >>>
> > > > >>> Knowing that the problem crops up intermittently, I then rebooted the laptop. When the boot was complete, I then powered
> > > > >>> on my bluetooth headphones an waited for them to connect to the laptop. After about 20 seconds, a connection had not
> > > > >>> been established. I powered off the headphones, used modprobe to unload and then reload the btusb module. When I powered
> > > > >>> on the headphones, a connectiin was established within 2 or 3 seconds.
> > > > >
> > > > > Ive attempted 5 restart with 5.17.0-0.rc6.109.fc37.x86_64, my headset
> > > > > was able to reconnect every single time without any problem. The only
> > > >
> > > > For your five tests, did it connect on the first boot? As I've said, sometimes it fails to connect on the first boot,
> > > > but if it succeeds, it has always failed after a power-off and restart. Looking back at the notes I took during the
> > > > bisect, I've didn't have a single bisection step where I had to boot more more than twice to ascertain that it was a bad
> > > > kernel. As I said, I didn't mark a kernel as good until I'd had five successful boots.
> > >
> > > It did connect every single time.
> > >
> > > > > normally, once from gdm and then another time when gnome is loading,
> > > > > but I assume it is normal nowadays since it appears when switching
> > > > > session pipewire unregisters its audio endpoints.
> > > > >
> > > >
> > > > I don't use pipewire. Prior to 5.17, bluetooth has worked more or less trouble free for at least 4 years. I've read
> > > > about pipewire in Linux Magazine but don't see what it would bring to my party except complication.
> > > >
> > > > ends on how bluetoothd is being
> > > > > started or something.
> > > > >
> > > >
> > > > Did you see the warnings that read "Bluetooth: hci0: unexpected event 0xff length: 5 > 0"? That seems to indicate that
> > > > something is sending events that are unexpected. What effect will that have? As I said, according to lshw, my system's
> > > > bluetooth hardware is Intel AX201. Is that what you are testing on?
> > >
> > > I have an AX200 on my system, AX201 is very similar so Id be surprised
> > > if that is the problem, btw Ive also got some unexpected events but
> > > that didn't stop the headset to reconnect.
> > >
> > > > >>> I've booted this laptop countless times over the last few days. Doing the bisect, I didn't mark a commit as good until I
> > > > >>> had done five boots and been able to connect my headphones on each boot. What I can say from that work is that two
> > > > >>> consecutive boots into a working kernel are very rare. I can't remeber an occasion when it took more than two boots to
> > > > >>> establish that a kernel was bad.
> > > > >>
> > > > >> Do commands such as bluetoothctl power on or scan on works? Try
> > > > >> running bluetoothd -dn from a shell (disable bluetooth.service), also
> > > > >> are there any settings changed in main.conf?
> > > > >>
> > > >
> > > > Sorry, I forgot to answer this question earlier. I haven't changed main.conf. Besides, my bluetooth devices connect
> > > > successfully every time with 5.16.11 and 5.15.25 kernels. As I've said before, that strongly suggests that there is a
> > > > code regression in 5.17.
> > >
> > > Not saying there isn't something wrong, we have sent a couple of fixes
> > > that doesn't seem to be merged yet, and we are working on another one
> > > for fixing the scan:
> > >
> > > https://lore.kernel.org/linux-bluetooth/[email protected]/T/
> >
> > Btw, are you by any chance doing something like hciconfig hci0 up on
> > your init scripts?
>
> Looks like I was able to reproduce but I still don't know the cause,
> anyway the symptom is the following:
>
> [23412.856410] remove_uuid:2273: hci0: sock 0000000049dcd70a
> [23412.856467] mgmt_class_complete:2174: hci0: err 0
> [23412.856470] mgmt_cmd_complete:176: sock 00000000d63e046a
>
> It looks like the cmd->sk is wrong/corrupted, what is even more
> strange is that the socket pointer does seem to match previously
> connected MGMT clients and after a few attempts with btmgmt> power on
> it does come back to life, very bizarre...
>
> Can you try to enable some kernel debugs before you start bluetoothd:
>
> echo "file net/bluetooth/mgmt.c +pfl" > /sys/kernel/debug/dynamic_debug/control
> echo "file net/bluetooth/mgmt_util.c +pfl" >
> /sys/kernel/debug/dynamic_debug/control
>
> Maybe that should give us a clue what triggers it.
Here is an attempt to fix the problem:
https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
Also it probably make more sense to test with the following tree since
we are using it to push regression fixes:
https://git.kernel.org/pub/scm/linux/kernel/git/bluetooth/bluetooth.git/
> > > > >>
> > > > >> --
> > > > >> Luiz Augusto von Dentz
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > Luiz Augusto von Dentz
> >
> >
> >
> > --
> > Luiz Augusto von Dentz
>
>
>
> --
> Luiz Augusto von Dentz
--
Luiz Augusto von Dentz
Hi Luiz,
Good news...
On 02/03/2022 06:55, Luiz Augusto von Dentz wrote:
<snip>
>>
>> Looks like I was able to reproduce but I still don't know the cause,
>> anyway the symptom is the following:
>>
>> [23412.856410] remove_uuid:2273: hci0: sock 0000000049dcd70a
>> [23412.856467] mgmt_class_complete:2174: hci0: err 0
>> [23412.856470] mgmt_cmd_complete:176: sock 00000000d63e046a
>>
>> It looks like the cmd->sk is wrong/corrupted, what is even more
>> strange is that the socket pointer does seem to match previously
>> connected MGMT clients and after a few attempts with btmgmt> power on
>> it does come back to life, very bizarre...
>>
>> Can you try to enable some kernel debugs before you start bluetoothd:
>>
>> echo "file net/bluetooth/mgmt.c +pfl" > /sys/kernel/debug/dynamic_debug/control
>> echo "file net/bluetooth/mgmt_util.c +pfl" >
>> /sys/kernel/debug/dynamic_debug/control
>>
>> Maybe that should give us a clue what triggers it.
>
> Here is an attempt to fix the problem:
>
> https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/
>
> Also it probably make more sense to test with the following tree since
> we are using it to push regression fixes:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/bluetooth/bluetooth.git/
>
I've cloned bluetooth.git, applied the patch and installed the resultant kernel. I've booted into the kernel eight times
and on each occasion , my bluetooth devices connected successfully. Additionally, I've the latest updates from Linus'
tree into my local clone and applied your patch to that. That too has resulted in a working bluetooth service.
Thanks and well done to you and your colleagues.
For the patch above:
Tested-by: Chris Clayton <[email protected]>
>>>>>>>
>>>>>>> --
>>>>>>> Luiz Augusto von Dentz
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Luiz Augusto von Dentz
>>>
>>>
>>>
>>> --
>>> Luiz Augusto von Dentz
>>
>>
>>
>> --
>> Luiz Augusto von Dentz
>
>
>