LinuxLists.cc - [PATCH v2 1/2] Bluetooth: Add new HCI_QUIRK_NO_SUSPEND

2021-01-28 16:40:43

Subject: [PATCH v2 1/2] Bluetooth: Add new HCI_QUIRK_NO_SUSPEND_NOTIFIER quirk

Some devices, e.g. the RTL8723BS bluetooth part, some USB attached devices,
completely drop from the bus on a system-suspend. These devices will
have their driver unbound and rebound on resume (when the dropping of
the bus gets detected) and will show up as a new HCI after resume.

These devices do not benefit from the suspend / resume handling work done
by the hci_suspend_notifier. At best this unnecessarily adds some time to
the suspend/resume time. But this may also actually cause problems, if the
code doing the driver unbinding runs after the pm-notifier then the
hci_suspend_notifier code will try to talk to a device which is now in
an uninitialized state.

This commit adds a new HCI_QUIRK_NO_SUSPEND_NOTIFIER quirk which allows
drivers to opt-out of the hci_suspend_notifier when they know beforehand
that their device will be fully re-initialized / reprobed on resume.

Cc: Abhishek Pandit-Subedi <[email protected]>
Signed-off-by: Hans de Goede <[email protected]>
---
Changes in v2:
- This is a new patch in v2 of this patch-set
---
include/net/bluetooth/hci.h | 8 ++++++++
net/bluetooth/hci_core.c | 18 +++++++++++-------
2 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/include/net/bluetooth/hci.h b/include/net/bluetooth/hci.h
index c1504aa3d9cf..ba2f439bc04d 100644
--- a/include/net/bluetooth/hci.h
+++ b/include/net/bluetooth/hci.h
@@ -238,6 +238,14 @@ enum {
* during the hdev->setup vendor callback.
*/
HCI_QUIRK_BROKEN_ERR_DATA_REPORTING,
+
+ /*
+ * When this quirk is set, then the hci_suspend_notifier is not
+ * registered. This is intended for devices which drop completely
+ * from the bus on system-suspend and which will show up as a new
+ * HCI after resume.
+ */
+ HCI_QUIRK_NO_SUSPEND_NOTIFIER,
};

/* HCI device flags */
diff --git a/net/bluetooth/hci_core.c b/net/bluetooth/hci_core.c
index 9d2c9a1c552f..9d8dbb38585c 100644
--- a/net/bluetooth/hci_core.c
+++ b/net/bluetooth/hci_core.c
@@ -3827,10 +3827,12 @@ int hci_register_dev(struct hci_dev *hdev)
hci_sock_dev_event(hdev, HCI_DEV_REG);
hci_dev_hold(hdev);

- hdev->suspend_notifier.notifier_call = hci_suspend_notifier;
- error = register_pm_notifier(&hdev->suspend_notifier);
- if (error)
- goto err_wqueue;
+ if (!test_bit(HCI_QUIRK_NO_SUSPEND_NOTIFIER, &hdev->quirks)) {
+ hdev->suspend_notifier.notifier_call = hci_suspend_notifier;
+ error = register_pm_notifier(&hdev->suspend_notifier);
+ if (error)
+ goto err_wqueue;
+ }

queue_work(hdev->req_workqueue, &hdev->power_on);

@@ -3865,9 +3867,11 @@ void hci_unregister_dev(struct hci_dev *hdev)

cancel_work_sync(&hdev->power_on);

- hci_suspend_clear_tasks(hdev);
- unregister_pm_notifier(&hdev->suspend_notifier);
- cancel_work_sync(&hdev->suspend_prepare);
+ if (!test_bit(HCI_QUIRK_NO_SUSPEND_NOTIFIER, &hdev->quirks)) {
+ hci_suspend_clear_tasks(hdev);
+ unregister_pm_notifier(&hdev->suspend_notifier);
+ cancel_work_sync(&hdev->suspend_prepare);
+ }

hci_dev_do_close(hdev);

--
2.29.2

2021-01-28 16:40:43

by Hans de Goede

[permalink] [raw]

Subject: [PATCH v2 2/2] Bluetooth: hci_h5: Disable the hci_suspend_notifier for btrtl devices

The hci_suspend_notifier which was introduced last year, is causing
problems for uart attached btrtl devices. These devices may loose their
firmware and their baudrate setting over a suspend/resume.

Since we don't even know the baudrate after a suspend/resume recovering
from this is tricky. The driver solves this by treating these devices
the same as USB BT HCIs which drop of the bus during suspend.

Specifically the driver:
1. Simply unconditionally turns the device fully off during
system-suspend to save maximum power.
2. Calls device_reprobe() from a workqueue to fully re-init the device
from scratch on system-resume (unregistering the old HCI and
registering a new HCI).

This means that these devices do not benefit from the suspend / resume
handling work done by the hci_suspend_notifier. At best this unnecessarily
adds some time to the suspend/resume time.

But in practice this is actually causing problems:

1. These btrtl devices seem to not like the HCI_OP_WRITE_SCAN_ENABLE(
SCAN_DISABLED) request being send to them when entering the
BT_SUSPEND_CONFIGURE_WAKE state. The same request send on
BT_SUSPEND_DISCONNECT works fine, but the second one send (unnecessarily?)
from the BT_SUSPEND_CONFIGURE_WAKE transition causes the device to hang:

[ 573.497754] PM: suspend entry (s2idle)
[ 573.554615] Filesystems sync: 0.056 seconds
[ 575.837753] Bluetooth: hci0: Timed out waiting for suspend events
[ 575.837801] Bluetooth: hci0: Suspend timeout bit: 4
[ 575.837925] Bluetooth: hci0: Suspend notifier action (3) failed: -110

2. The PM_POST_SUSPEND / BT_RUNNING transition races with the
driver-unbinding done by the device_reprobe() work.
If the hci_suspend_notifier wins the race it is talking to a dead
device leading to the following errors being logged:

[ 598.686060] Bluetooth: hci0: Timed out waiting for suspend events
[ 598.686124] Bluetooth: hci0: Suspend timeout bit: 5
[ 598.686237] Bluetooth: hci0: Suspend notifier action (4) failed: -110

In both cases things still work, but the suspend-notifier is causing
these ugly errors getting logged and ut increase both the suspend- and
the resume-time by 2 seconds.

This commit avoids these problems by disabling the hci_suspend_notifier.

Cc: Luiz Augusto von Dentz <[email protected]>
Cc: Vasily Khoruzhick <[email protected]>
Cc: Abhishek Pandit-Subedi <[email protected]>
Signed-off-by: Hans de Goede <[email protected]>
---
Changes in v2:
- Use the new HCI_QUIRK_NO_SUSPEND_NOTIFIER quirk, instead of directly
unregistering the notifier from hci_h5.c
---
drivers/bluetooth/hci_h5.c | 7 +++++++
drivers/bluetooth/hci_serdev.c | 3 +++
drivers/bluetooth/hci_uart.h | 13 +++++++------
3 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/drivers/bluetooth/hci_h5.c b/drivers/bluetooth/hci_h5.c
index 7be16a7f653b..e8f3afab6587 100644
--- a/drivers/bluetooth/hci_h5.c
+++ b/drivers/bluetooth/hci_h5.c
@@ -914,6 +914,13 @@ static int h5_btrtl_setup(struct h5 *h5)

static void h5_btrtl_open(struct h5 *h5)
{
+ /*
+ * Since h5_btrtl_resume() does a device_reprobe() the suspend handling
+ * done by the hci_suspend_notifier is not necessary; it actually causes
+ * delays and a bunch of errors to get logged, so disable it.
+ */
+ set_bit(HCI_UART_NO_SUSPEND_NOTIFIER, &h5->hu->hdev_flags);
+
/* Devices always start with these fixed parameters */
serdev_device_set_flow_control(h5->hu->serdev, false);
serdev_device_set_parity(h5->hu->serdev, SERDEV_PARITY_EVEN);
diff --git a/drivers/bluetooth/hci_serdev.c b/drivers/bluetooth/hci_serdev.c
index ef96ad06fa54..dbc14b8ac477 100644
--- a/drivers/bluetooth/hci_serdev.c
+++ b/drivers/bluetooth/hci_serdev.c
@@ -349,6 +349,9 @@ int hci_uart_register_device(struct hci_uart *hu,
if (test_bit(HCI_UART_EXT_CONFIG, &hu->hdev_flags))
set_bit(HCI_QUIRK_EXTERNAL_CONFIG, &hdev->quirks);

+ if (test_bit(HCI_UART_NO_SUSPEND_NOTIFIER, &hu->hdev_flags))
+ set_bit(HCI_QUIRK_NO_SUSPEND_NOTIFIER, &hdev->quirks);
+
if (test_bit(HCI_UART_CREATE_AMP, &hu->hdev_flags))
hdev->dev_type = HCI_AMP;
else
diff --git a/drivers/bluetooth/hci_uart.h b/drivers/bluetooth/hci_uart.h
index 4e039d7a16f8..4df2330ac103 100644
--- a/drivers/bluetooth/hci_uart.h
+++ b/drivers/bluetooth/hci_uart.h
@@ -35,12 +35,13 @@
#define HCI_UART_NOKIA 10
#define HCI_UART_MRVL 11

-#define HCI_UART_RAW_DEVICE 0
-#define HCI_UART_RESET_ON_INIT 1
-#define HCI_UART_CREATE_AMP 2
-#define HCI_UART_INIT_PENDING 3
-#define HCI_UART_EXT_CONFIG 4
-#define HCI_UART_VND_DETECT 5
+#define HCI_UART_RAW_DEVICE 0
+#define HCI_UART_RESET_ON_INIT 1
+#define HCI_UART_CREATE_AMP 2
+#define HCI_UART_INIT_PENDING 3
+#define HCI_UART_EXT_CONFIG 4
+#define HCI_UART_VND_DETECT 5
+#define HCI_UART_NO_SUSPEND_NOTIFIER 6

struct hci_uart;
struct serdev_device;
--
2.29.2

2021-01-28 17:30:07

by Abhishek Pandit-Subedi

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] Bluetooth: Add new HCI_QUIRK_NO_SUSPEND_NOTIFIER quirk

Hi,

On Thu, Jan 28, 2021 at 8:34 AM Hans de Goede <[email protected]> wrote:
>
> Some devices, e.g. the RTL8723BS bluetooth part, some USB attached devices,
> completely drop from the bus on a system-suspend. These devices will
> have their driver unbound and rebound on resume (when the dropping of
> the bus gets detected) and will show up as a new HCI after resume.
>
> These devices do not benefit from the suspend / resume handling work done
> by the hci_suspend_notifier. At best this unnecessarily adds some time to
> the suspend/resume time. But this may also actually cause problems, if the
> code doing the driver unbinding runs after the pm-notifier then the
> hci_suspend_notifier code will try to talk to a device which is now in
> an uninitialized state.
>
> This commit adds a new HCI_QUIRK_NO_SUSPEND_NOTIFIER quirk which allows
> drivers to opt-out of the hci_suspend_notifier when they know beforehand
> that their device will be fully re-initialized / reprobed on resume.
>
> Cc: Abhishek Pandit-Subedi <[email protected]>
> Signed-off-by: Hans de Goede <[email protected]>
> ---

This looks good to me.

Reviewed-by: Abhishek Pandit-Subedi <[email protected]>

2021-01-28 17:40:51

by Abhishek Pandit-Subedi

[permalink] [raw]

Subject: Re: [PATCH v2 2/2] Bluetooth: hci_h5: Disable the hci_suspend_notifier for btrtl devices

Hi,

On Thu, Jan 28, 2021 at 8:34 AM Hans de Goede <[email protected]> wrote:
>
> The hci_suspend_notifier which was introduced last year, is causing
> problems for uart attached btrtl devices. These devices may loose their
> firmware and their baudrate setting over a suspend/resume.
>
> Since we don't even know the baudrate after a suspend/resume recovering
> from this is tricky. The driver solves this by treating these devices
> the same as USB BT HCIs which drop of the bus during suspend.
>
> Specifically the driver:
> 1. Simply unconditionally turns the device fully off during
> system-suspend to save maximum power.
> 2. Calls device_reprobe() from a workqueue to fully re-init the device
> from scratch on system-resume (unregistering the old HCI and
> registering a new HCI).
>
> This means that these devices do not benefit from the suspend / resume
> handling work done by the hci_suspend_notifier. At best this unnecessarily
> adds some time to the suspend/resume time.
>
> But in practice this is actually causing problems:
>
> 1. These btrtl devices seem to not like the HCI_OP_WRITE_SCAN_ENABLE(
> SCAN_DISABLED) request being send to them when entering the
> BT_SUSPEND_CONFIGURE_WAKE state. The same request send on
> BT_SUSPEND_DISCONNECT works fine, but the second one send (unnecessarily?)
> from the BT_SUSPEND_CONFIGURE_WAKE transition causes the device to hang:
>
> [ 573.497754] PM: suspend entry (s2idle)
> [ 573.554615] Filesystems sync: 0.056 seconds
> [ 575.837753] Bluetooth: hci0: Timed out waiting for suspend events
> [ 575.837801] Bluetooth: hci0: Suspend timeout bit: 4
> [ 575.837925] Bluetooth: hci0: Suspend notifier action (3) failed: -110
>
> 2. The PM_POST_SUSPEND / BT_RUNNING transition races with the
> driver-unbinding done by the device_reprobe() work.
> If the hci_suspend_notifier wins the race it is talking to a dead
> device leading to the following errors being logged:
>
> [ 598.686060] Bluetooth: hci0: Timed out waiting for suspend events
> [ 598.686124] Bluetooth: hci0: Suspend timeout bit: 5
> [ 598.686237] Bluetooth: hci0: Suspend notifier action (4) failed: -110
>
> In both cases things still work, but the suspend-notifier is causing
> these ugly errors getting logged and ut increase both the suspend- and
> the resume-time by 2 seconds.
>
> This commit avoids these problems by disabling the hci_suspend_notifier.
>
> Cc: Luiz Augusto von Dentz <[email protected]>
> Cc: Vasily Khoruzhick <[email protected]>
> Cc: Abhishek Pandit-Subedi <[email protected]>
> Signed-off-by: Hans de Goede <[email protected]>
> ---
> Changes in v2:
> - Use the new HCI_QUIRK_NO_SUSPEND_NOTIFIER quirk, instead of directly
> unregistering the notifier from hci_h5.c
> ---
> drivers/bluetooth/hci_h5.c | 7 +++++++
> drivers/bluetooth/hci_serdev.c | 3 +++
> drivers/bluetooth/hci_uart.h | 13 +++++++------
> 3 files changed, 17 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/bluetooth/hci_h5.c b/drivers/bluetooth/hci_h5.c
> index 7be16a7f653b..e8f3afab6587 100644
> --- a/drivers/bluetooth/hci_h5.c
> +++ b/drivers/bluetooth/hci_h5.c
> @@ -914,6 +914,13 @@ static int h5_btrtl_setup(struct h5 *h5)
>
> static void h5_btrtl_open(struct h5 *h5)
> {
> + /*
> + * Since h5_btrtl_resume() does a device_reprobe() the suspend handling
> + * done by the hci_suspend_notifier is not necessary; it actually causes
> + * delays and a bunch of errors to get logged, so disable it.
> + */
> + set_bit(HCI_UART_NO_SUSPEND_NOTIFIER, &h5->hu->hdev_flags);

I'm not sure we should enable this for all RTL devices rather than
based on the specific project. RTL8822C will also be using hci_h5 but
intends to support wake on bt (meaning it shouldn't be losing firmware
around suspend).

+ Max Chou: You are proposing a change to add project id to btrtl.
Should we use that instead to set this quirk for 8723 devices (and
others which lose fw around suspend)?
(https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/)

> +
> /* Devices always start with these fixed parameters */
> serdev_device_set_flow_control(h5->hu->serdev, false);
> serdev_device_set_parity(h5->hu->serdev, SERDEV_PARITY_EVEN);
> diff --git a/drivers/bluetooth/hci_serdev.c b/drivers/bluetooth/hci_serdev.c
> index ef96ad06fa54..dbc14b8ac477 100644
> --- a/drivers/bluetooth/hci_serdev.c
> +++ b/drivers/bluetooth/hci_serdev.c
> @@ -349,6 +349,9 @@ int hci_uart_register_device(struct hci_uart *hu,
> if (test_bit(HCI_UART_EXT_CONFIG, &hu->hdev_flags))
> set_bit(HCI_QUIRK_EXTERNAL_CONFIG, &hdev->quirks);
>
> + if (test_bit(HCI_UART_NO_SUSPEND_NOTIFIER, &hu->hdev_flags))
> + set_bit(HCI_QUIRK_NO_SUSPEND_NOTIFIER, &hdev->quirks);
> +
> if (test_bit(HCI_UART_CREATE_AMP, &hu->hdev_flags))
> hdev->dev_type = HCI_AMP;
> else
> diff --git a/drivers/bluetooth/hci_uart.h b/drivers/bluetooth/hci_uart.h
> index 4e039d7a16f8..4df2330ac103 100644
> --- a/drivers/bluetooth/hci_uart.h
> +++ b/drivers/bluetooth/hci_uart.h
> @@ -35,12 +35,13 @@
> #define HCI_UART_NOKIA 10
> #define HCI_UART_MRVL 11
>
> -#define HCI_UART_RAW_DEVICE 0
> -#define HCI_UART_RESET_ON_INIT 1
> -#define HCI_UART_CREATE_AMP 2
> -#define HCI_UART_INIT_PENDING 3
> -#define HCI_UART_EXT_CONFIG 4
> -#define HCI_UART_VND_DETECT 5
> +#define HCI_UART_RAW_DEVICE 0
> +#define HCI_UART_RESET_ON_INIT 1
> +#define HCI_UART_CREATE_AMP 2
> +#define HCI_UART_INIT_PENDING 3
> +#define HCI_UART_EXT_CONFIG 4
> +#define HCI_UART_VND_DETECT 5
> +#define HCI_UART_NO_SUSPEND_NOTIFIER 6
>
> struct hci_uart;
> struct serdev_device;
> --
> 2.29.2
>

2021-01-28 17:55:16

by bluez.test.bot

[permalink] [raw]

Subject: RE: [v2,1/2] Bluetooth: Add new HCI_QUIRK_NO_SUSPEND_NOTIFIER quirk

This is automated email and please do not reply to this email!

Dear submitter,

Thank you for submitting the patches to the linux bluetooth mailing list.
This is a CI test results with your patch series:
PW Link:https://patchwork.kernel.org/project/bluetooth/list/?series=423915

---Test result---

##############################
Test: CheckPatch - PASS

##############################
Test: CheckGitLint - PASS

##############################
Test: CheckBuildK - PASS

##############################
Test: CheckTestRunner: Setup - PASS

##############################
Test: CheckTestRunner: l2cap-tester - PASS
Total: 40, Passed: 34 (85.0%), Failed: 0, Not Run: 6

##############################
Test: CheckTestRunner: bnep-tester - PASS
Total: 1, Passed: 1 (100.0%), Failed: 0, Not Run: 0

##############################
Test: CheckTestRunner: mgmt-tester - FAIL
Total: 416, Passed: 401 (96.4%), Failed: 1, Not Run: 14

##############################
Test: CheckTestRunner: rfcomm-tester - PASS
Total: 9, Passed: 9 (100.0%), Failed: 0, Not Run: 0

##############################
Test: CheckTestRunner: sco-tester - PASS
Total: 8, Passed: 8 (100.0%), Failed: 0, Not Run: 0

##############################
Test: CheckTestRunner: smp-tester - PASS
Total: 8, Passed: 8 (100.0%), Failed: 0, Not Run: 0

##############################
Test: CheckTestRunner: userchan-tester - PASS
Total: 3, Passed: 3 (100.0%), Failed: 0, Not Run: 0

---
Regards,
Linux Bluetooth

Attachments:

l2cap-tester.log (42.33 kB)
bnep-tester.log (3.45 kB)
mgmt-tester.log (533.80 kB)
rfcomm-tester.log (11.38 kB)
sco-tester.log (9.65 kB)
smp-tester.log (11.52 kB)
userchan-tester.log (5.30 kB)
Download all attachments

2021-01-29 17:49:01

by Abhishek Pandit-Subedi

[permalink] [raw]

Subject: Re: [PATCH v2 2/2] Bluetooth: hci_h5: Disable the hci_suspend_notifier for btrtl devices

Hi,

On Fri, Jan 29, 2021 at 6:28 AM Hans de Goede <[email protected]> wrote:
>
> Hi,
>
> On 1/29/21 9:25 AM, Max Chou wrote:
> > Hi,
> >
> >> I'm not sure we should enable this for all RTL devices rather than based on the specific project. RTL8822C will also be using hci_h5 but intends to support wake on bt (meaning it shouldn't be losing firmware around suspend).
> >
> >> + Max Chou: You are proposing a change to add project id to btrtl.
> >> Should we use that instead to set this quirk for 8723 devices (and others which lose fw around suspend)?
> >> (https://patchwork.kernel.org/project/bluetooth/patch/[email protected]/)
> >
> > Agree. Recommend to use the same way as Abhishek mentions that limiting the quirk only for RTL8723B devices if this patch is necessary.
> > Therefore, some of the projects use RTL8822C devices with BT UART interface would apply BT wakes Host.
> > So far, I've not heard the issue as this topic.
>
> So I just checked because I was not aware that the hci_h5 code was also being used for the RTL8822C.
> I mainly focus on x86/ACPI use and there is no 8822 ACPI device-id in the h5_acpi_match table, but
> there is indeed a match for this in the rtl_bluetooth_of_match table.
>
> But ATM the code is treating the 8822 exactly the same as the 8723, including doing a device_reprobe
> on resume. So to me it makes sense to set the HCI_UART_NO_SUSPEND_NOTIFIER unconditionally to, as
> it is used because of the device_reprobe being done.
>
> Now it might be a good idea to opt out of the device_reprobe for 8822 devices, and/or maybe even for
> all devicetree enumerated cases (the device being completely shutoff is an ACPI thing, with dt/of we
> should have more control).
>
> To me it seems that since for now the device_reprobe() is unconditional that the matching setting
> of the HCI_UART_NO_SUSPEND_NOTIFIER flag should be unconditional too; and then when the device_reprobe()
> stuff is made unconditional, then we can make the setting of the flag unconditional using the same check.

I didn't realize this was currently unconditional in code. In that
case, I think it's fine for you to add the flag unconditionally in
hci_h5. When we add support for wake on bt to the h5 driver, we should
make this flag conditional based on whether the driver reprobes on
resume.

I will go back and +1 the original patch.

Thanks,
Abhishek

>
> With that said if people really want it I'm happy to respin this to only apply to the 8723 case,
> but that seems weird given that the device_reprobe ATM is being done on the 8822 too.
>
> > +Clair: Have you met this issue during suspend/resume when BT controller is RTL8822C with BT UART interface?
> > Please see the patchwork. https://patchwork.kernel.org/project/bluetooth/list/?series=423915
>
> Regards,
>
> Hans
>

2021-01-29 17:49:22

by Abhishek Pandit-Subedi

[permalink] [raw]