2021-04-13 09:03:48

by Jamie Mccrae

[permalink] [raw]
Subject: RE: Disabled bluetooth cache. But the app still getting wrong data?

Hi Kenny,
Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.
Thanks,
Jamie

-----Original Message-----
From: Kenny Bian <[email protected]>
Sent: 13 April 2021 06:59
To: [email protected]
Subject: Disabled bluetooth cache. But the app still getting wrong data?

EXTERNAL EMAIL: Be careful with attachments and links.

Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
In order to keep the pairing information, the "/var/lib/bluetooth"
folder is copied over to the new build's partition. We realized that there is no "service changed indication". The app can't handle the changed services. So we disabled the bluetooth cache by set this in
"/etc/bluetooth/main.conf":
[GATT]
Cache = no

But recently, we saw the problem again even if the bluetooth cache is
disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.

The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.

I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.

Questions:
1. How is it possible that this still happens even if the bluetooth cache is disabled?
2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
3. Is there anything else we should look into?

We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.

Thanks!
THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.


2021-04-14 03:18:08

by Kenny Bian

[permalink] [raw]
Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?

Hi Jamie,

Thank you for your email. Do you know how to add "Service Changed
Indication"? We would like to support that. But we don't know how to
do it.

In fact, I posted the question before:
https://www.spinics.net/lists/linux-bluetooth/msg89959.html. It seems
"If you are registering the services with Bluetoothd then it should
generate the service change". Our GATT server is based on
https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/test/example-gatt-server.
By saying "registering the services", does it mean our code should
call "RegisterApplication()"? We did call RegisterApplication(). But
we never saw the "[ServiceChanged]"
section(https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/settings-storage.txt#n321).
Please let us know if you know how to add "Service Changed
Indication".

Thanks!
Kenny

Kenny

On Mon, Apr 12, 2021 at 11:39 PM Jamie Mccrae
<[email protected]> wrote:
>
> Hi Kenny,
> Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.
> Thanks,
> Jamie
>
> -----Original Message-----
> From: Kenny Bian <[email protected]>
> Sent: 13 April 2021 06:59
> To: [email protected]
> Subject: Disabled bluetooth cache. But the app still getting wrong data?
>
> EXTERNAL EMAIL: Be careful with attachments and links.
>
> Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
> By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
> In order to keep the pairing information, the "/var/lib/bluetooth"
> folder is copied over to the new build's partition. We realized that there is no "service changed indication". The app can't handle the changed services. So we disabled the bluetooth cache by set this in
> "/etc/bluetooth/main.conf":
> [GATT]
> Cache = no
>
> But recently, we saw the problem again even if the bluetooth cache is
> disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.
>
> The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.
>
> I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.
>
> Questions:
> 1. How is it possible that this still happens even if the bluetooth cache is disabled?
> 2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
> 3. Is there anything else we should look into?
>
> We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.
>
> Thanks!
> THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.

2021-04-14 08:42:00

by Luiz Augusto von Dentz

[permalink] [raw]
Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?

Hi Jamie, Brian,

On Tue, Apr 13, 2021 at 2:03 AM Jamie Mccrae
<[email protected]> wrote:
>
> Hi Kenny,
> Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.

Yep, and while at it implement the so called Robust Caching feature so
we can detect if anything has changed by reading the DB Hash.

> Thanks,
> Jamie
>
> -----Original Message-----
> From: Kenny Bian <[email protected]>
> Sent: 13 April 2021 06:59
> To: [email protected]
> Subject: Disabled bluetooth cache. But the app still getting wrong data?
>
> EXTERNAL EMAIL: Be careful with attachments and links.
>
> Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
> By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
> In order to keep the pairing information, the "/var/lib/bluetooth"
> folder is copied over to the new build's partition. We realized that there is no "service changed indication". The app can't handle the changed services. So we disabled the bluetooth cache by set this in
> "/etc/bluetooth/main.conf":
> [GATT]
> Cache = no

When you say the app can't handle changed service do you mean BlueZ
doesn't emit changes to the attributes (via Service Changed) or is it
really the application not being able to handle the changes?

> But recently, we saw the problem again even if the bluetooth cache is
> disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.

So BlueZ is acting as the server, right? The Cache only applies to the
client portion, there is no such thing as disabling the remote cache.
I don't see any incoming Read By Group Request from the remote so it
is very likely that it has cached the values, there is no Read By Type
for the DB Hash either which is quite surprising to me since that is
required for stacks supporting Robust Caching which I believe is the
case of iOS.

Anyway, I would check that the following lines are being triggered:

When starting:

https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3798

On connect:

https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3741

If those lines are not being triggered it is likely a client problem
which for some reason had not subscribed to received service changes
for some reason, if it doesn't subscribe to service changes then it
shall not cache any attribute and attempt to rediscover on every
connection.

> The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.
>
> I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.
>
> Questions:
> 1. How is it possible that this still happens even if the bluetooth cache is disabled?
> 2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
> 3. Is there anything else we should look into?
>
> We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.
>
> Thanks!
> THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.



--
Luiz Augusto von Dentz

2021-04-14 12:32:13

by Kenny Bian

[permalink] [raw]
Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?

Hi Jamie, Luiz,

Thanks for your info.

Luiz, to answer your questions:
BlueZ doesn't seem to emit "Service Changed Indication". Or it does
emit it, but we don't know how to verify it.
Yes, BlueZ acts as a GATT server which runs on Linux.

So my questions:
How can I tell if "Service Changed Indication" is working? I need to
show it to our app developers that the GATT server does send the
"Service Changed Indication" when there is a service change.
By disabling the GATT's cache in "/etc/bluetooth/main.conf", the
"Service Changed Indication" is disabled?
Can "Service Changed Indication" be enabled in a conf file? Or should
it be done in the Python code? Could you please help find out that
info?
Our GATT server is based on the Python code
examples(https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/test).
I don't know how to enable "Service Changed Indication" or "Robust
Caching" in Python code on the GATT server side. There is no document
or code samples about that. Please let me know if there is any
document or Python code example.

Thanks!
Kenny

On Tue, Apr 13, 2021 at 2:57 PM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Jamie, Brian,
>
> On Tue, Apr 13, 2021 at 2:03 AM Jamie Mccrae
> <[email protected]> wrote:
> >
> > Hi Kenny,
> > Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.
>
> Yep, and while at it implement the so called Robust Caching feature so
> we can detect if anything has changed by reading the DB Hash.
>
> > Thanks,
> > Jamie
> >
> > -----Original Message-----
> > From: Kenny Bian <[email protected]>
> > Sent: 13 April 2021 06:59
> > To: [email protected]
> > Subject: Disabled bluetooth cache. But the app still getting wrong data?
> >
> > EXTERNAL EMAIL: Be careful with attachments and links.
> >
> > Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
> > By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
> > In order to keep the pairing information, the "/var/lib/bluetooth"
> > folder is copied over to the new build's partition. We realized that there is no "service changed indication". The app can't handle the changed services. So we disabled the bluetooth cache by set this in
> > "/etc/bluetooth/main.conf":
> > [GATT]
> > Cache = no
>
> When you say the app can't handle changed service do you mean BlueZ
> doesn't emit changes to the attributes (via Service Changed) or is it
> really the application not being able to handle the changes?
>
> > But recently, we saw the problem again even if the bluetooth cache is
> > disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.
>
> So BlueZ is acting as the server, right? The Cache only applies to the
> client portion, there is no such thing as disabling the remote cache.
> I don't see any incoming Read By Group Request from the remote so it
> is very likely that it has cached the values, there is no Read By Type
> for the DB Hash either which is quite surprising to me since that is
> required for stacks supporting Robust Caching which I believe is the
> case of iOS.
>
> Anyway, I would check that the following lines are being triggered:
>
> When starting:
>
> https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3798
>
> On connect:
>
> https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3741
>
> If those lines are not being triggered it is likely a client problem
> which for some reason had not subscribed to received service changes
> for some reason, if it doesn't subscribe to service changes then it
> shall not cache any attribute and attempt to rediscover on every
> connection.
>
> > The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.
> >
> > I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.
> >
> > Questions:
> > 1. How is it possible that this still happens even if the bluetooth cache is disabled?
> > 2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
> > 3. Is there anything else we should look into?
> >
> > We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.
> >
> > Thanks!
> > THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.
>
>
>
> --
> Luiz Augusto von Dentz

2021-04-15 05:45:14

by Kenny Bian

[permalink] [raw]
Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?

Hi Luiz,

There is a UI example of nRF Connect which shows the "Service Changed
Indication" is under "Generic
Attribute"(http://forum.espruino.com/conversations/357737/). We're
using BlueZ 5.48. I tried to add it by modifying example-gatt-server:


class ServiceChangedChrc(Characteristic):
SCI_UUID = '00002a05-0000-1000-8000-00805f9b34fb'

def __init__(self, bus, index, service):
Characteristic.__init__(
self, bus, index,
self.SCI_UUID,
['indicate'],
service)

def IndicateValue(self, options):
# This function doesn't exist. Just list it here.
pass


class ServiceChangedService(Service):
MY_UUID = '00001801-0000-1000-8000-00805f9b34fb'

def __init__(self, bus, index):
Service.__init__(self, bus, index, self.MY_UUID, True)
self.add_characteristic(ServiceChangedChrc(bus, 0, self))


class Application(dbus.service.Object):
"""
org.bluez.GattApplication1 interface implementation
"""
def __init__(self, bus):
self.path = '/'
self.services = []
dbus.service.Object.__init__(self, bus, self.path)
self.add_service(HeartRateService(bus, 0))
self.add_service(BatteryService(bus, 1))
self.add_service(TestService(bus, 2))
self.add_service(ServiceChangedService(bus, 3))


When I run it, there is "Failed to register application:
org.bluez.Error.Failed: Failed to create entry in database". That is
in the function client_ready_cb() in gatt-database.c in BlueZ code. Do
you know how to fix it?

Thanks!
Kenny

On Tue, Apr 13, 2021 at 9:46 PM Kenny Bian <[email protected]> wrote:
>
> Hi Jamie, Luiz,
>
> Thanks for your info.
>
> Luiz, to answer your questions:
> BlueZ doesn't seem to emit "Service Changed Indication". Or it does
> emit it, but we don't know how to verify it.
> Yes, BlueZ acts as a GATT server which runs on Linux.
>
> So my questions:
> How can I tell if "Service Changed Indication" is working? I need to
> show it to our app developers that the GATT server does send the
> "Service Changed Indication" when there is a service change.
> By disabling the GATT's cache in "/etc/bluetooth/main.conf", the
> "Service Changed Indication" is disabled?
> Can "Service Changed Indication" be enabled in a conf file? Or should
> it be done in the Python code? Could you please help find out that
> info?
> Our GATT server is based on the Python code
> examples(https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/test).
> I don't know how to enable "Service Changed Indication" or "Robust
> Caching" in Python code on the GATT server side. There is no document
> or code samples about that. Please let me know if there is any
> document or Python code example.
>
> Thanks!
> Kenny
>
> On Tue, Apr 13, 2021 at 2:57 PM Luiz Augusto von Dentz
> <[email protected]> wrote:
> >
> > Hi Jamie, Brian,
> >
> > On Tue, Apr 13, 2021 at 2:03 AM Jamie Mccrae
> > <[email protected]> wrote:
> > >
> > > Hi Kenny,
> > > Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.
> >
> > Yep, and while at it implement the so called Robust Caching feature so
> > we can detect if anything has changed by reading the DB Hash.
> >
> > > Thanks,
> > > Jamie
> > >
> > > -----Original Message-----
> > > From: Kenny Bian <[email protected]>
> > > Sent: 13 April 2021 06:59
> > > To: [email protected]
> > > Subject: Disabled bluetooth cache. But the app still getting wrong data?
> > >
> > > EXTERNAL EMAIL: Be careful with attachments and links.
> > >
> > > Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
> > > By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
> > > In order to keep the pairing information, the "/var/lib/bluetooth"
> > > folder is copied over to the new build's partition. We realized that there is no "service changed indication". The app can't handle the changed services. So we disabled the bluetooth cache by set this in
> > > "/etc/bluetooth/main.conf":
> > > [GATT]
> > > Cache = no
> >
> > When you say the app can't handle changed service do you mean BlueZ
> > doesn't emit changes to the attributes (via Service Changed) or is it
> > really the application not being able to handle the changes?
> >
> > > But recently, we saw the problem again even if the bluetooth cache is
> > > disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.
> >
> > So BlueZ is acting as the server, right? The Cache only applies to the
> > client portion, there is no such thing as disabling the remote cache.
> > I don't see any incoming Read By Group Request from the remote so it
> > is very likely that it has cached the values, there is no Read By Type
> > for the DB Hash either which is quite surprising to me since that is
> > required for stacks supporting Robust Caching which I believe is the
> > case of iOS.
> >
> > Anyway, I would check that the following lines are being triggered:
> >
> > When starting:
> >
> > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3798
> >
> > On connect:
> >
> > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3741
> >
> > If those lines are not being triggered it is likely a client problem
> > which for some reason had not subscribed to received service changes
> > for some reason, if it doesn't subscribe to service changes then it
> > shall not cache any attribute and attempt to rediscover on every
> > connection.
> >
> > > The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.
> > >
> > > I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.
> > >
> > > Questions:
> > > 1. How is it possible that this still happens even if the bluetooth cache is disabled?
> > > 2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
> > > 3. Is there anything else we should look into?
> > >
> > > We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.
> > >
> > > Thanks!
> > > THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.
> >
> >
> >
> > --
> > Luiz Augusto von Dentz

2021-04-15 22:16:15

by Luiz Augusto von Dentz

[permalink] [raw]
Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?

Hi Kenny,

On Wed, Apr 14, 2021 at 10:44 PM Kenny Bian <[email protected]> wrote:
>
> Hi Luiz,
>
> There is a UI example of nRF Connect which shows the "Service Changed
> Indication" is under "Generic
> Attribute"(http://forum.espruino.com/conversations/357737/). We're
> using BlueZ 5.48. I tried to add it by modifying example-gatt-server:


You should have mentioned that you are using 5.48, you need the following patch:

commit a0b886e26c83ad4dfbf0b2b2e024c8ce9ead9a7d
Author: Szymon Janc <[email protected]>
Date: Wed Mar 28 12:10:55 2018 +0200

gatt: Add support for storing Service Changed CCC value

This adds support for storing CCC value of Service Changed
characteristic. Once bluetoothd is restart stored values are read
and any device subscribed to indications will receive Service Changed
indication with 0x00010-0xffff value. This is to invalidate any
non-core services since there is no way to verify if applications
will register their services in same order (or at all).

This fix accessing invalid handles by stacks that rely only on Service
Changed indication for rediscovery ie. Apple iOS.

Or upgrade to a BlueZ release that has the above patch, which should be 5.50:

+ver 5.50:
+ Fix issue with GATT and reading long values.
+ Fix issue with GATT and reading multiple includes.
+ Fix issue with GATT and service changes when offline.
+ Fix issue with handling secondary service discovery.
+ Fix issue with handling persistency of CCC values.

>
> class ServiceChangedChrc(Characteristic):
> SCI_UUID = '00002a05-0000-1000-8000-00805f9b34fb'
>
> def __init__(self, bus, index, service):
> Characteristic.__init__(
> self, bus, index,
> self.SCI_UUID,
> ['indicate'],
> service)
>
> def IndicateValue(self, options):
> # This function doesn't exist. Just list it here.
> pass
>
>
> class ServiceChangedService(Service):
> MY_UUID = '00001801-0000-1000-8000-00805f9b34fb'
>
> def __init__(self, bus, index):
> Service.__init__(self, bus, index, self.MY_UUID, True)
> self.add_characteristic(ServiceChangedChrc(bus, 0, self))
>
>
> class Application(dbus.service.Object):
> """
> org.bluez.GattApplication1 interface implementation
> """
> def __init__(self, bus):
> self.path = '/'
> self.services = []
> dbus.service.Object.__init__(self, bus, self.path)
> self.add_service(HeartRateService(bus, 0))
> self.add_service(BatteryService(bus, 1))
> self.add_service(TestService(bus, 2))
> self.add_service(ServiceChangedService(bus, 3))
>
>
> When I run it, there is "Failed to register application:
> org.bluez.Error.Failed: Failed to create entry in database". That is
> in the function client_ready_cb() in gatt-database.c in BlueZ code. Do
> you know how to fix it?

You don't need to register Service Changed, the daemon already takes
care of registering it and there could only be one Service Changed in
the database.

> Thanks!
> Kenny
>
> On Tue, Apr 13, 2021 at 9:46 PM Kenny Bian <[email protected]> wrote:
> >
> > Hi Jamie, Luiz,
> >
> > Thanks for your info.
> >
> > Luiz, to answer your questions:
> > BlueZ doesn't seem to emit "Service Changed Indication". Or it does
> > emit it, but we don't know how to verify it.
> > Yes, BlueZ acts as a GATT server which runs on Linux.
> >
> > So my questions:
> > How can I tell if "Service Changed Indication" is working? I need to
> > show it to our app developers that the GATT server does send the
> > "Service Changed Indication" when there is a service change.
> > By disabling the GATT's cache in "/etc/bluetooth/main.conf", the
> > "Service Changed Indication" is disabled?
> > Can "Service Changed Indication" be enabled in a conf file? Or should
> > it be done in the Python code? Could you please help find out that
> > info?
> > Our GATT server is based on the Python code
> > examples(https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/test).
> > I don't know how to enable "Service Changed Indication" or "Robust
> > Caching" in Python code on the GATT server side. There is no document
> > or code samples about that. Please let me know if there is any
> > document or Python code example.
> >
> > Thanks!
> > Kenny
> >
> > On Tue, Apr 13, 2021 at 2:57 PM Luiz Augusto von Dentz
> > <[email protected]> wrote:
> > >
> > > Hi Jamie, Brian,
> > >
> > > On Tue, Apr 13, 2021 at 2:03 AM Jamie Mccrae
> > > <[email protected]> wrote:
> > > >
> > > > Hi Kenny,
> > > > Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.
> > >
> > > Yep, and while at it implement the so called Robust Caching feature so
> > > we can detect if anything has changed by reading the DB Hash.
> > >
> > > > Thanks,
> > > > Jamie
> > > >
> > > > -----Original Message-----
> > > > From: Kenny Bian <[email protected]>
> > > > Sent: 13 April 2021 06:59
> > > > To: [email protected]
> > > > Subject: Disabled bluetooth cache. But the app still getting wrong data?
> > > >
> > > > EXTERNAL EMAIL: Be careful with attachments and links.
> > > >
> > > > Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
> > > > By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
> > > > In order to keep the pairing information, the "/var/lib/bluetooth"
> > > > folder is copied over to the new build's partition. We realized that there is no "service changed indication". The app can't handle the changed services. So we disabled the bluetooth cache by set this in
> > > > "/etc/bluetooth/main.conf":
> > > > [GATT]
> > > > Cache = no
> > >
> > > When you say the app can't handle changed service do you mean BlueZ
> > > doesn't emit changes to the attributes (via Service Changed) or is it
> > > really the application not being able to handle the changes?
> > >
> > > > But recently, we saw the problem again even if the bluetooth cache is
> > > > disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.
> > >
> > > So BlueZ is acting as the server, right? The Cache only applies to the
> > > client portion, there is no such thing as disabling the remote cache.
> > > I don't see any incoming Read By Group Request from the remote so it
> > > is very likely that it has cached the values, there is no Read By Type
> > > for the DB Hash either which is quite surprising to me since that is
> > > required for stacks supporting Robust Caching which I believe is the
> > > case of iOS.
> > >
> > > Anyway, I would check that the following lines are being triggered:
> > >
> > > When starting:
> > >
> > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3798
> > >
> > > On connect:
> > >
> > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3741
> > >
> > > If those lines are not being triggered it is likely a client problem
> > > which for some reason had not subscribed to received service changes
> > > for some reason, if it doesn't subscribe to service changes then it
> > > shall not cache any attribute and attempt to rediscover on every
> > > connection.
> > >
> > > > The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.
> > > >
> > > > I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.
> > > >
> > > > Questions:
> > > > 1. How is it possible that this still happens even if the bluetooth cache is disabled?
> > > > 2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
> > > > 3. Is there anything else we should look into?
> > > >
> > > > We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.
> > > >
> > > > Thanks!
> > > > THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.
> > >
> > >
> > >
> > > --
> > > Luiz Augusto von Dentz



--
Luiz Augusto von Dentz

2021-04-16 02:11:14

by Kenny Bian

[permalink] [raw]
Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?

Hi Luiz,

Thank you so much. I appreciate it.

That was my bad. I just checked. Our system is Ubuntu 18.04. The BlueZ
version is "5.48-0ubuntu3.4" which is different from the versions in
http://www.bluez.org/. I'm trying to figure out if this version
includes the version 5.50. Do you know the corresponding BlueZ version
for "5.48-0ubuntu3.4"?

Thanks!
Kenny

On Thu, Apr 15, 2021 at 1:31 PM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Kenny,
>
> On Wed, Apr 14, 2021 at 10:44 PM Kenny Bian <[email protected]> wrote:
> >
> > Hi Luiz,
> >
> > There is a UI example of nRF Connect which shows the "Service Changed
> > Indication" is under "Generic
> > Attribute"(http://forum.espruino.com/conversations/357737/). We're
> > using BlueZ 5.48. I tried to add it by modifying example-gatt-server:
>
>
> You should have mentioned that you are using 5.48, you need the following patch:
>
> commit a0b886e26c83ad4dfbf0b2b2e024c8ce9ead9a7d
> Author: Szymon Janc <[email protected]>
> Date: Wed Mar 28 12:10:55 2018 +0200
>
> gatt: Add support for storing Service Changed CCC value
>
> This adds support for storing CCC value of Service Changed
> characteristic. Once bluetoothd is restart stored values are read
> and any device subscribed to indications will receive Service Changed
> indication with 0x00010-0xffff value. This is to invalidate any
> non-core services since there is no way to verify if applications
> will register their services in same order (or at all).
>
> This fix accessing invalid handles by stacks that rely only on Service
> Changed indication for rediscovery ie. Apple iOS.
>
> Or upgrade to a BlueZ release that has the above patch, which should be 5.50:
>
> +ver 5.50:
> + Fix issue with GATT and reading long values.
> + Fix issue with GATT and reading multiple includes.
> + Fix issue with GATT and service changes when offline.
> + Fix issue with handling secondary service discovery.
> + Fix issue with handling persistency of CCC values.
>
> >
> > class ServiceChangedChrc(Characteristic):
> > SCI_UUID = '00002a05-0000-1000-8000-00805f9b34fb'
> >
> > def __init__(self, bus, index, service):
> > Characteristic.__init__(
> > self, bus, index,
> > self.SCI_UUID,
> > ['indicate'],
> > service)
> >
> > def IndicateValue(self, options):
> > # This function doesn't exist. Just list it here.
> > pass
> >
> >
> > class ServiceChangedService(Service):
> > MY_UUID = '00001801-0000-1000-8000-00805f9b34fb'
> >
> > def __init__(self, bus, index):
> > Service.__init__(self, bus, index, self.MY_UUID, True)
> > self.add_characteristic(ServiceChangedChrc(bus, 0, self))
> >
> >
> > class Application(dbus.service.Object):
> > """
> > org.bluez.GattApplication1 interface implementation
> > """
> > def __init__(self, bus):
> > self.path = '/'
> > self.services = []
> > dbus.service.Object.__init__(self, bus, self.path)
> > self.add_service(HeartRateService(bus, 0))
> > self.add_service(BatteryService(bus, 1))
> > self.add_service(TestService(bus, 2))
> > self.add_service(ServiceChangedService(bus, 3))
> >
> >
> > When I run it, there is "Failed to register application:
> > org.bluez.Error.Failed: Failed to create entry in database". That is
> > in the function client_ready_cb() in gatt-database.c in BlueZ code. Do
> > you know how to fix it?
>
> You don't need to register Service Changed, the daemon already takes
> care of registering it and there could only be one Service Changed in
> the database.
>
> > Thanks!
> > Kenny
> >
> > On Tue, Apr 13, 2021 at 9:46 PM Kenny Bian <[email protected]> wrote:
> > >
> > > Hi Jamie, Luiz,
> > >
> > > Thanks for your info.
> > >
> > > Luiz, to answer your questions:
> > > BlueZ doesn't seem to emit "Service Changed Indication". Or it does
> > > emit it, but we don't know how to verify it.
> > > Yes, BlueZ acts as a GATT server which runs on Linux.
> > >
> > > So my questions:
> > > How can I tell if "Service Changed Indication" is working? I need to
> > > show it to our app developers that the GATT server does send the
> > > "Service Changed Indication" when there is a service change.
> > > By disabling the GATT's cache in "/etc/bluetooth/main.conf", the
> > > "Service Changed Indication" is disabled?
> > > Can "Service Changed Indication" be enabled in a conf file? Or should
> > > it be done in the Python code? Could you please help find out that
> > > info?
> > > Our GATT server is based on the Python code
> > > examples(https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/test).
> > > I don't know how to enable "Service Changed Indication" or "Robust
> > > Caching" in Python code on the GATT server side. There is no document
> > > or code samples about that. Please let me know if there is any
> > > document or Python code example.
> > >
> > > Thanks!
> > > Kenny
> > >
> > > On Tue, Apr 13, 2021 at 2:57 PM Luiz Augusto von Dentz
> > > <[email protected]> wrote:
> > > >
> > > > Hi Jamie, Brian,
> > > >
> > > > On Tue, Apr 13, 2021 at 2:03 AM Jamie Mccrae
> > > > <[email protected]> wrote:
> > > > >
> > > > > Hi Kenny,
> > > > > Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.
> > > >
> > > > Yep, and while at it implement the so called Robust Caching feature so
> > > > we can detect if anything has changed by reading the DB Hash.
> > > >
> > > > > Thanks,
> > > > > Jamie
> > > > >
> > > > > -----Original Message-----
> > > > > From: Kenny Bian <[email protected]>
> > > > > Sent: 13 April 2021 06:59
> > > > > To: [email protected]
> > > > > Subject: Disabled bluetooth cache. But the app still getting wrong data?
> > > > >
> > > > > EXTERNAL EMAIL: Be careful with attachments and links.
> > > > >
> > > > > Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
> > > > > By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
> > > > > In order to keep the pairing information, the "/var/lib/bluetooth"
> > > > > folder is copied over to the new build's partition. We realized that there is no "service changed indication". The app can't handle the changed services. So we disabled the bluetooth cache by set this in
> > > > > "/etc/bluetooth/main.conf":
> > > > > [GATT]
> > > > > Cache = no
> > > >
> > > > When you say the app can't handle changed service do you mean BlueZ
> > > > doesn't emit changes to the attributes (via Service Changed) or is it
> > > > really the application not being able to handle the changes?
> > > >
> > > > > But recently, we saw the problem again even if the bluetooth cache is
> > > > > disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.
> > > >
> > > > So BlueZ is acting as the server, right? The Cache only applies to the
> > > > client portion, there is no such thing as disabling the remote cache.
> > > > I don't see any incoming Read By Group Request from the remote so it
> > > > is very likely that it has cached the values, there is no Read By Type
> > > > for the DB Hash either which is quite surprising to me since that is
> > > > required for stacks supporting Robust Caching which I believe is the
> > > > case of iOS.
> > > >
> > > > Anyway, I would check that the following lines are being triggered:
> > > >
> > > > When starting:
> > > >
> > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3798
> > > >
> > > > On connect:
> > > >
> > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3741
> > > >
> > > > If those lines are not being triggered it is likely a client problem
> > > > which for some reason had not subscribed to received service changes
> > > > for some reason, if it doesn't subscribe to service changes then it
> > > > shall not cache any attribute and attempt to rediscover on every
> > > > connection.
> > > >
> > > > > The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.
> > > > >
> > > > > I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.
> > > > >
> > > > > Questions:
> > > > > 1. How is it possible that this still happens even if the bluetooth cache is disabled?
> > > > > 2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
> > > > > 3. Is there anything else we should look into?
> > > > >
> > > > > We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.
> > > > >
> > > > > Thanks!
> > > > > THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.
> > > >
> > > >
> > > >
> > > > --
> > > > Luiz Augusto von Dentz
>
>
>
> --
> Luiz Augusto von Dentz

2021-04-16 02:30:55

by Luiz Augusto von Dentz

[permalink] [raw]
Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?

Hi Kenny,

On Thu, Apr 15, 2021 at 5:21 PM Kenny Bian <[email protected]> wrote:
>
> Hi Luiz,
>
> Thank you so much. I appreciate it.
>
> That was my bad. I just checked. Our system is Ubuntu 18.04. The BlueZ
> version is "5.48-0ubuntu3.4" which is different from the versions in
> http://www.bluez.org/. I'm trying to figure out if this version
> includes the version 5.50. Do you know the corresponding BlueZ version
> for "5.48-0ubuntu3.4"?

5.48, which was the version available in 2018, more than 3 years/10
releases ago, a lot of things have been fixed over this time so Id
really recommend to upgrade to the latest if you want to have some
support.

> Thanks!
> Kenny
>
> On Thu, Apr 15, 2021 at 1:31 PM Luiz Augusto von Dentz
> <[email protected]> wrote:
> >
> > Hi Kenny,
> >
> > On Wed, Apr 14, 2021 at 10:44 PM Kenny Bian <[email protected]> wrote:
> > >
> > > Hi Luiz,
> > >
> > > There is a UI example of nRF Connect which shows the "Service Changed
> > > Indication" is under "Generic
> > > Attribute"(http://forum.espruino.com/conversations/357737/). We're
> > > using BlueZ 5.48. I tried to add it by modifying example-gatt-server:
> >
> >
> > You should have mentioned that you are using 5.48, you need the following patch:
> >
> > commit a0b886e26c83ad4dfbf0b2b2e024c8ce9ead9a7d
> > Author: Szymon Janc <[email protected]>
> > Date: Wed Mar 28 12:10:55 2018 +0200
> >
> > gatt: Add support for storing Service Changed CCC value
> >
> > This adds support for storing CCC value of Service Changed
> > characteristic. Once bluetoothd is restart stored values are read
> > and any device subscribed to indications will receive Service Changed
> > indication with 0x00010-0xffff value. This is to invalidate any
> > non-core services since there is no way to verify if applications
> > will register their services in same order (or at all).
> >
> > This fix accessing invalid handles by stacks that rely only on Service
> > Changed indication for rediscovery ie. Apple iOS.
> >
> > Or upgrade to a BlueZ release that has the above patch, which should be 5.50:
> >
> > +ver 5.50:
> > + Fix issue with GATT and reading long values.
> > + Fix issue with GATT and reading multiple includes.
> > + Fix issue with GATT and service changes when offline.
> > + Fix issue with handling secondary service discovery.
> > + Fix issue with handling persistency of CCC values.
> >
> > >
> > > class ServiceChangedChrc(Characteristic):
> > > SCI_UUID = '00002a05-0000-1000-8000-00805f9b34fb'
> > >
> > > def __init__(self, bus, index, service):
> > > Characteristic.__init__(
> > > self, bus, index,
> > > self.SCI_UUID,
> > > ['indicate'],
> > > service)
> > >
> > > def IndicateValue(self, options):
> > > # This function doesn't exist. Just list it here.
> > > pass
> > >
> > >
> > > class ServiceChangedService(Service):
> > > MY_UUID = '00001801-0000-1000-8000-00805f9b34fb'
> > >
> > > def __init__(self, bus, index):
> > > Service.__init__(self, bus, index, self.MY_UUID, True)
> > > self.add_characteristic(ServiceChangedChrc(bus, 0, self))
> > >
> > >
> > > class Application(dbus.service.Object):
> > > """
> > > org.bluez.GattApplication1 interface implementation
> > > """
> > > def __init__(self, bus):
> > > self.path = '/'
> > > self.services = []
> > > dbus.service.Object.__init__(self, bus, self.path)
> > > self.add_service(HeartRateService(bus, 0))
> > > self.add_service(BatteryService(bus, 1))
> > > self.add_service(TestService(bus, 2))
> > > self.add_service(ServiceChangedService(bus, 3))
> > >
> > >
> > > When I run it, there is "Failed to register application:
> > > org.bluez.Error.Failed: Failed to create entry in database". That is
> > > in the function client_ready_cb() in gatt-database.c in BlueZ code. Do
> > > you know how to fix it?
> >
> > You don't need to register Service Changed, the daemon already takes
> > care of registering it and there could only be one Service Changed in
> > the database.
> >
> > > Thanks!
> > > Kenny
> > >
> > > On Tue, Apr 13, 2021 at 9:46 PM Kenny Bian <[email protected]> wrote:
> > > >
> > > > Hi Jamie, Luiz,
> > > >
> > > > Thanks for your info.
> > > >
> > > > Luiz, to answer your questions:
> > > > BlueZ doesn't seem to emit "Service Changed Indication". Or it does
> > > > emit it, but we don't know how to verify it.
> > > > Yes, BlueZ acts as a GATT server which runs on Linux.
> > > >
> > > > So my questions:
> > > > How can I tell if "Service Changed Indication" is working? I need to
> > > > show it to our app developers that the GATT server does send the
> > > > "Service Changed Indication" when there is a service change.
> > > > By disabling the GATT's cache in "/etc/bluetooth/main.conf", the
> > > > "Service Changed Indication" is disabled?
> > > > Can "Service Changed Indication" be enabled in a conf file? Or should
> > > > it be done in the Python code? Could you please help find out that
> > > > info?
> > > > Our GATT server is based on the Python code
> > > > examples(https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/test).
> > > > I don't know how to enable "Service Changed Indication" or "Robust
> > > > Caching" in Python code on the GATT server side. There is no document
> > > > or code samples about that. Please let me know if there is any
> > > > document or Python code example.
> > > >
> > > > Thanks!
> > > > Kenny
> > > >
> > > > On Tue, Apr 13, 2021 at 2:57 PM Luiz Augusto von Dentz
> > > > <[email protected]> wrote:
> > > > >
> > > > > Hi Jamie, Brian,
> > > > >
> > > > > On Tue, Apr 13, 2021 at 2:03 AM Jamie Mccrae
> > > > > <[email protected]> wrote:
> > > > > >
> > > > > > Hi Kenny,
> > > > > > Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.
> > > > >
> > > > > Yep, and while at it implement the so called Robust Caching feature so
> > > > > we can detect if anything has changed by reading the DB Hash.
> > > > >
> > > > > > Thanks,
> > > > > > Jamie
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Kenny Bian <[email protected]>
> > > > > > Sent: 13 April 2021 06:59
> > > > > > To: [email protected]
> > > > > > Subject: Disabled bluetooth cache. But the app still getting wrong data?
> > > > > >
> > > > > > EXTERNAL EMAIL: Be careful with attachments and links.
> > > > > >
> > > > > > Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
> > > > > > By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
> > > > > > In order to keep the pairing information, the "/var/lib/bluetooth"
> > > > > > folder is copied over to the new build's partition. We realized that there is no "service changed indication". The app can't handle the changed services. So we disabled the bluetooth cache by set this in
> > > > > > "/etc/bluetooth/main.conf":
> > > > > > [GATT]
> > > > > > Cache = no
> > > > >
> > > > > When you say the app can't handle changed service do you mean BlueZ
> > > > > doesn't emit changes to the attributes (via Service Changed) or is it
> > > > > really the application not being able to handle the changes?
> > > > >
> > > > > > But recently, we saw the problem again even if the bluetooth cache is
> > > > > > disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.
> > > > >
> > > > > So BlueZ is acting as the server, right? The Cache only applies to the
> > > > > client portion, there is no such thing as disabling the remote cache.
> > > > > I don't see any incoming Read By Group Request from the remote so it
> > > > > is very likely that it has cached the values, there is no Read By Type
> > > > > for the DB Hash either which is quite surprising to me since that is
> > > > > required for stacks supporting Robust Caching which I believe is the
> > > > > case of iOS.
> > > > >
> > > > > Anyway, I would check that the following lines are being triggered:
> > > > >
> > > > > When starting:
> > > > >
> > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3798
> > > > >
> > > > > On connect:
> > > > >
> > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3741
> > > > >
> > > > > If those lines are not being triggered it is likely a client problem
> > > > > which for some reason had not subscribed to received service changes
> > > > > for some reason, if it doesn't subscribe to service changes then it
> > > > > shall not cache any attribute and attempt to rediscover on every
> > > > > connection.
> > > > >
> > > > > > The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.
> > > > > >
> > > > > > I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.
> > > > > >
> > > > > > Questions:
> > > > > > 1. How is it possible that this still happens even if the bluetooth cache is disabled?
> > > > > > 2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
> > > > > > 3. Is there anything else we should look into?
> > > > > >
> > > > > > We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.
> > > > > >
> > > > > > Thanks!
> > > > > > THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Luiz Augusto von Dentz
> >
> >
> >
> > --
> > Luiz Augusto von Dentz



--
Luiz Augusto von Dentz

2021-04-16 03:10:18

by Kenny Bian

[permalink] [raw]
Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?

Hi Luiz,

Thank you so much! You're absolutely right.

I'll talk to our team to upgrade our Ubuntu to 20.04 LTS. Then I'll
test the BlueZ 5.53 which is the latest version on Ubuntu 20.04 LTS.

Thanks again!
Kenny

On Thu, Apr 15, 2021 at 5:49 PM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Kenny,
>
> On Thu, Apr 15, 2021 at 5:21 PM Kenny Bian <[email protected]> wrote:
> >
> > Hi Luiz,
> >
> > Thank you so much. I appreciate it.
> >
> > That was my bad. I just checked. Our system is Ubuntu 18.04. The BlueZ
> > version is "5.48-0ubuntu3.4" which is different from the versions in
> > http://www.bluez.org/. I'm trying to figure out if this version
> > includes the version 5.50. Do you know the corresponding BlueZ version
> > for "5.48-0ubuntu3.4"?
>
> 5.48, which was the version available in 2018, more than 3 years/10
> releases ago, a lot of things have been fixed over this time so Id
> really recommend to upgrade to the latest if you want to have some
> support.
>
> > Thanks!
> > Kenny
> >
> > On Thu, Apr 15, 2021 at 1:31 PM Luiz Augusto von Dentz
> > <[email protected]> wrote:
> > >
> > > Hi Kenny,
> > >
> > > On Wed, Apr 14, 2021 at 10:44 PM Kenny Bian <[email protected]> wrote:
> > > >
> > > > Hi Luiz,
> > > >
> > > > There is a UI example of nRF Connect which shows the "Service Changed
> > > > Indication" is under "Generic
> > > > Attribute"(http://forum.espruino.com/conversations/357737/). We're
> > > > using BlueZ 5.48. I tried to add it by modifying example-gatt-server:
> > >
> > >
> > > You should have mentioned that you are using 5.48, you need the following patch:
> > >
> > > commit a0b886e26c83ad4dfbf0b2b2e024c8ce9ead9a7d
> > > Author: Szymon Janc <[email protected]>
> > > Date: Wed Mar 28 12:10:55 2018 +0200
> > >
> > > gatt: Add support for storing Service Changed CCC value
> > >
> > > This adds support for storing CCC value of Service Changed
> > > characteristic. Once bluetoothd is restart stored values are read
> > > and any device subscribed to indications will receive Service Changed
> > > indication with 0x00010-0xffff value. This is to invalidate any
> > > non-core services since there is no way to verify if applications
> > > will register their services in same order (or at all).
> > >
> > > This fix accessing invalid handles by stacks that rely only on Service
> > > Changed indication for rediscovery ie. Apple iOS.
> > >
> > > Or upgrade to a BlueZ release that has the above patch, which should be 5.50:
> > >
> > > +ver 5.50:
> > > + Fix issue with GATT and reading long values.
> > > + Fix issue with GATT and reading multiple includes.
> > > + Fix issue with GATT and service changes when offline.
> > > + Fix issue with handling secondary service discovery.
> > > + Fix issue with handling persistency of CCC values.
> > >
> > > >
> > > > class ServiceChangedChrc(Characteristic):
> > > > SCI_UUID = '00002a05-0000-1000-8000-00805f9b34fb'
> > > >
> > > > def __init__(self, bus, index, service):
> > > > Characteristic.__init__(
> > > > self, bus, index,
> > > > self.SCI_UUID,
> > > > ['indicate'],
> > > > service)
> > > >
> > > > def IndicateValue(self, options):
> > > > # This function doesn't exist. Just list it here.
> > > > pass
> > > >
> > > >
> > > > class ServiceChangedService(Service):
> > > > MY_UUID = '00001801-0000-1000-8000-00805f9b34fb'
> > > >
> > > > def __init__(self, bus, index):
> > > > Service.__init__(self, bus, index, self.MY_UUID, True)
> > > > self.add_characteristic(ServiceChangedChrc(bus, 0, self))
> > > >
> > > >
> > > > class Application(dbus.service.Object):
> > > > """
> > > > org.bluez.GattApplication1 interface implementation
> > > > """
> > > > def __init__(self, bus):
> > > > self.path = '/'
> > > > self.services = []
> > > > dbus.service.Object.__init__(self, bus, self.path)
> > > > self.add_service(HeartRateService(bus, 0))
> > > > self.add_service(BatteryService(bus, 1))
> > > > self.add_service(TestService(bus, 2))
> > > > self.add_service(ServiceChangedService(bus, 3))
> > > >
> > > >
> > > > When I run it, there is "Failed to register application:
> > > > org.bluez.Error.Failed: Failed to create entry in database". That is
> > > > in the function client_ready_cb() in gatt-database.c in BlueZ code. Do
> > > > you know how to fix it?
> > >
> > > You don't need to register Service Changed, the daemon already takes
> > > care of registering it and there could only be one Service Changed in
> > > the database.
> > >
> > > > Thanks!
> > > > Kenny
> > > >
> > > > On Tue, Apr 13, 2021 at 9:46 PM Kenny Bian <[email protected]> wrote:
> > > > >
> > > > > Hi Jamie, Luiz,
> > > > >
> > > > > Thanks for your info.
> > > > >
> > > > > Luiz, to answer your questions:
> > > > > BlueZ doesn't seem to emit "Service Changed Indication". Or it does
> > > > > emit it, but we don't know how to verify it.
> > > > > Yes, BlueZ acts as a GATT server which runs on Linux.
> > > > >
> > > > > So my questions:
> > > > > How can I tell if "Service Changed Indication" is working? I need to
> > > > > show it to our app developers that the GATT server does send the
> > > > > "Service Changed Indication" when there is a service change.
> > > > > By disabling the GATT's cache in "/etc/bluetooth/main.conf", the
> > > > > "Service Changed Indication" is disabled?
> > > > > Can "Service Changed Indication" be enabled in a conf file? Or should
> > > > > it be done in the Python code? Could you please help find out that
> > > > > info?
> > > > > Our GATT server is based on the Python code
> > > > > examples(https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/test).
> > > > > I don't know how to enable "Service Changed Indication" or "Robust
> > > > > Caching" in Python code on the GATT server side. There is no document
> > > > > or code samples about that. Please let me know if there is any
> > > > > document or Python code example.
> > > > >
> > > > > Thanks!
> > > > > Kenny
> > > > >
> > > > > On Tue, Apr 13, 2021 at 2:57 PM Luiz Augusto von Dentz
> > > > > <[email protected]> wrote:
> > > > > >
> > > > > > Hi Jamie, Brian,
> > > > > >
> > > > > > On Tue, Apr 13, 2021 at 2:03 AM Jamie Mccrae
> > > > > > <[email protected]> wrote:
> > > > > > >
> > > > > > > Hi Kenny,
> > > > > > > Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.
> > > > > >
> > > > > > Yep, and while at it implement the so called Robust Caching feature so
> > > > > > we can detect if anything has changed by reading the DB Hash.
> > > > > >
> > > > > > > Thanks,
> > > > > > > Jamie
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Kenny Bian <[email protected]>
> > > > > > > Sent: 13 April 2021 06:59
> > > > > > > To: [email protected]
> > > > > > > Subject: Disabled bluetooth cache. But the app still getting wrong data?
> > > > > > >
> > > > > > > EXTERNAL EMAIL: Be careful with attachments and links.
> > > > > > >
> > > > > > > Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
> > > > > > > By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
> > > > > > > In order to keep the pairing information, the "/var/lib/bluetooth"
> > > > > > > folder is copied over to the new build's partition. We realized that there is no "service changed indication". The app can't handle the changed services. So we disabled the bluetooth cache by set this in
> > > > > > > "/etc/bluetooth/main.conf":
> > > > > > > [GATT]
> > > > > > > Cache = no
> > > > > >
> > > > > > When you say the app can't handle changed service do you mean BlueZ
> > > > > > doesn't emit changes to the attributes (via Service Changed) or is it
> > > > > > really the application not being able to handle the changes?
> > > > > >
> > > > > > > But recently, we saw the problem again even if the bluetooth cache is
> > > > > > > disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.
> > > > > >
> > > > > > So BlueZ is acting as the server, right? The Cache only applies to the
> > > > > > client portion, there is no such thing as disabling the remote cache.
> > > > > > I don't see any incoming Read By Group Request from the remote so it
> > > > > > is very likely that it has cached the values, there is no Read By Type
> > > > > > for the DB Hash either which is quite surprising to me since that is
> > > > > > required for stacks supporting Robust Caching which I believe is the
> > > > > > case of iOS.
> > > > > >
> > > > > > Anyway, I would check that the following lines are being triggered:
> > > > > >
> > > > > > When starting:
> > > > > >
> > > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3798
> > > > > >
> > > > > > On connect:
> > > > > >
> > > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3741
> > > > > >
> > > > > > If those lines are not being triggered it is likely a client problem
> > > > > > which for some reason had not subscribed to received service changes
> > > > > > for some reason, if it doesn't subscribe to service changes then it
> > > > > > shall not cache any attribute and attempt to rediscover on every
> > > > > > connection.
> > > > > >
> > > > > > > The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.
> > > > > > >
> > > > > > > I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.
> > > > > > >
> > > > > > > Questions:
> > > > > > > 1. How is it possible that this still happens even if the bluetooth cache is disabled?
> > > > > > > 2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
> > > > > > > 3. Is there anything else we should look into?
> > > > > > >
> > > > > > > We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.
> > > > > > >
> > > > > > > Thanks!
> > > > > > > THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Luiz Augusto von Dentz
> > >
> > >
> > >
> > > --
> > > Luiz Augusto von Dentz
>
>
>
> --
> Luiz Augusto von Dentz

2021-04-16 04:39:02

by Kenny Bian

[permalink] [raw]
Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?

Hi Luiz,

One more question. Once we upgrade our device to Ubuntu 20.04 LTS with
BlueZ 5.53, we'll test "Service Changed Indication". Is GATT cache
related to "Service Changed Indication"? If they are related, then I
guess we should turn on the GATT cache in /etc/bluetooth/main.conf?
Right now, the GATT cache is turned off in our device.

Thanks,
Kenny

On Thu, Apr 15, 2021 at 8:06 PM Kenny Bian <[email protected]> wrote:
>
> Hi Luiz,
>
> Thank you so much! You're absolutely right.
>
> I'll talk to our team to upgrade our Ubuntu to 20.04 LTS. Then I'll
> test the BlueZ 5.53 which is the latest version on Ubuntu 20.04 LTS.
>
> Thanks again!
> Kenny
>
> On Thu, Apr 15, 2021 at 5:49 PM Luiz Augusto von Dentz
> <[email protected]> wrote:
> >
> > Hi Kenny,
> >
> > On Thu, Apr 15, 2021 at 5:21 PM Kenny Bian <[email protected]> wrote:
> > >
> > > Hi Luiz,
> > >
> > > Thank you so much. I appreciate it.
> > >
> > > That was my bad. I just checked. Our system is Ubuntu 18.04. The BlueZ
> > > version is "5.48-0ubuntu3.4" which is different from the versions in
> > > http://www.bluez.org/. I'm trying to figure out if this version
> > > includes the version 5.50. Do you know the corresponding BlueZ version
> > > for "5.48-0ubuntu3.4"?
> >
> > 5.48, which was the version available in 2018, more than 3 years/10
> > releases ago, a lot of things have been fixed over this time so Id
> > really recommend to upgrade to the latest if you want to have some
> > support.
> >
> > > Thanks!
> > > Kenny
> > >
> > > On Thu, Apr 15, 2021 at 1:31 PM Luiz Augusto von Dentz
> > > <[email protected]> wrote:
> > > >
> > > > Hi Kenny,
> > > >
> > > > On Wed, Apr 14, 2021 at 10:44 PM Kenny Bian <[email protected]> wrote:
> > > > >
> > > > > Hi Luiz,
> > > > >
> > > > > There is a UI example of nRF Connect which shows the "Service Changed
> > > > > Indication" is under "Generic
> > > > > Attribute"(http://forum.espruino.com/conversations/357737/). We're
> > > > > using BlueZ 5.48. I tried to add it by modifying example-gatt-server:
> > > >
> > > >
> > > > You should have mentioned that you are using 5.48, you need the following patch:
> > > >
> > > > commit a0b886e26c83ad4dfbf0b2b2e024c8ce9ead9a7d
> > > > Author: Szymon Janc <[email protected]>
> > > > Date: Wed Mar 28 12:10:55 2018 +0200
> > > >
> > > > gatt: Add support for storing Service Changed CCC value
> > > >
> > > > This adds support for storing CCC value of Service Changed
> > > > characteristic. Once bluetoothd is restart stored values are read
> > > > and any device subscribed to indications will receive Service Changed
> > > > indication with 0x00010-0xffff value. This is to invalidate any
> > > > non-core services since there is no way to verify if applications
> > > > will register their services in same order (or at all).
> > > >
> > > > This fix accessing invalid handles by stacks that rely only on Service
> > > > Changed indication for rediscovery ie. Apple iOS.
> > > >
> > > > Or upgrade to a BlueZ release that has the above patch, which should be 5.50:
> > > >
> > > > +ver 5.50:
> > > > + Fix issue with GATT and reading long values.
> > > > + Fix issue with GATT and reading multiple includes.
> > > > + Fix issue with GATT and service changes when offline.
> > > > + Fix issue with handling secondary service discovery.
> > > > + Fix issue with handling persistency of CCC values.
> > > >
> > > > >
> > > > > class ServiceChangedChrc(Characteristic):
> > > > > SCI_UUID = '00002a05-0000-1000-8000-00805f9b34fb'
> > > > >
> > > > > def __init__(self, bus, index, service):
> > > > > Characteristic.__init__(
> > > > > self, bus, index,
> > > > > self.SCI_UUID,
> > > > > ['indicate'],
> > > > > service)
> > > > >
> > > > > def IndicateValue(self, options):
> > > > > # This function doesn't exist. Just list it here.
> > > > > pass
> > > > >
> > > > >
> > > > > class ServiceChangedService(Service):
> > > > > MY_UUID = '00001801-0000-1000-8000-00805f9b34fb'
> > > > >
> > > > > def __init__(self, bus, index):
> > > > > Service.__init__(self, bus, index, self.MY_UUID, True)
> > > > > self.add_characteristic(ServiceChangedChrc(bus, 0, self))
> > > > >
> > > > >
> > > > > class Application(dbus.service.Object):
> > > > > """
> > > > > org.bluez.GattApplication1 interface implementation
> > > > > """
> > > > > def __init__(self, bus):
> > > > > self.path = '/'
> > > > > self.services = []
> > > > > dbus.service.Object.__init__(self, bus, self.path)
> > > > > self.add_service(HeartRateService(bus, 0))
> > > > > self.add_service(BatteryService(bus, 1))
> > > > > self.add_service(TestService(bus, 2))
> > > > > self.add_service(ServiceChangedService(bus, 3))
> > > > >
> > > > >
> > > > > When I run it, there is "Failed to register application:
> > > > > org.bluez.Error.Failed: Failed to create entry in database". That is
> > > > > in the function client_ready_cb() in gatt-database.c in BlueZ code. Do
> > > > > you know how to fix it?
> > > >
> > > > You don't need to register Service Changed, the daemon already takes
> > > > care of registering it and there could only be one Service Changed in
> > > > the database.
> > > >
> > > > > Thanks!
> > > > > Kenny
> > > > >
> > > > > On Tue, Apr 13, 2021 at 9:46 PM Kenny Bian <[email protected]> wrote:
> > > > > >
> > > > > > Hi Jamie, Luiz,
> > > > > >
> > > > > > Thanks for your info.
> > > > > >
> > > > > > Luiz, to answer your questions:
> > > > > > BlueZ doesn't seem to emit "Service Changed Indication". Or it does
> > > > > > emit it, but we don't know how to verify it.
> > > > > > Yes, BlueZ acts as a GATT server which runs on Linux.
> > > > > >
> > > > > > So my questions:
> > > > > > How can I tell if "Service Changed Indication" is working? I need to
> > > > > > show it to our app developers that the GATT server does send the
> > > > > > "Service Changed Indication" when there is a service change.
> > > > > > By disabling the GATT's cache in "/etc/bluetooth/main.conf", the
> > > > > > "Service Changed Indication" is disabled?
> > > > > > Can "Service Changed Indication" be enabled in a conf file? Or should
> > > > > > it be done in the Python code? Could you please help find out that
> > > > > > info?
> > > > > > Our GATT server is based on the Python code
> > > > > > examples(https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/test).
> > > > > > I don't know how to enable "Service Changed Indication" or "Robust
> > > > > > Caching" in Python code on the GATT server side. There is no document
> > > > > > or code samples about that. Please let me know if there is any
> > > > > > document or Python code example.
> > > > > >
> > > > > > Thanks!
> > > > > > Kenny
> > > > > >
> > > > > > On Tue, Apr 13, 2021 at 2:57 PM Luiz Augusto von Dentz
> > > > > > <[email protected]> wrote:
> > > > > > >
> > > > > > > Hi Jamie, Brian,
> > > > > > >
> > > > > > > On Tue, Apr 13, 2021 at 2:03 AM Jamie Mccrae
> > > > > > > <[email protected]> wrote:
> > > > > > > >
> > > > > > > > Hi Kenny,
> > > > > > > > Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.
> > > > > > >
> > > > > > > Yep, and while at it implement the so called Robust Caching feature so
> > > > > > > we can detect if anything has changed by reading the DB Hash.
> > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Jamie
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Kenny Bian <[email protected]>
> > > > > > > > Sent: 13 April 2021 06:59
> > > > > > > > To: [email protected]
> > > > > > > > Subject: Disabled bluetooth cache. But the app still getting wrong data?
> > > > > > > >
> > > > > > > > EXTERNAL EMAIL: Be careful with attachments and links.
> > > > > > > >
> > > > > > > > Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
> > > > > > > > By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
> > > > > > > > In order to keep the pairing information, the "/var/lib/bluetooth"
> > > > > > > > folder is copied over to the new build's partition. We realized that there is no "service changed indication". The app can't handle the changed services. So we disabled the bluetooth cache by set this in
> > > > > > > > "/etc/bluetooth/main.conf":
> > > > > > > > [GATT]
> > > > > > > > Cache = no
> > > > > > >
> > > > > > > When you say the app can't handle changed service do you mean BlueZ
> > > > > > > doesn't emit changes to the attributes (via Service Changed) or is it
> > > > > > > really the application not being able to handle the changes?
> > > > > > >
> > > > > > > > But recently, we saw the problem again even if the bluetooth cache is
> > > > > > > > disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.
> > > > > > >
> > > > > > > So BlueZ is acting as the server, right? The Cache only applies to the
> > > > > > > client portion, there is no such thing as disabling the remote cache.
> > > > > > > I don't see any incoming Read By Group Request from the remote so it
> > > > > > > is very likely that it has cached the values, there is no Read By Type
> > > > > > > for the DB Hash either which is quite surprising to me since that is
> > > > > > > required for stacks supporting Robust Caching which I believe is the
> > > > > > > case of iOS.
> > > > > > >
> > > > > > > Anyway, I would check that the following lines are being triggered:
> > > > > > >
> > > > > > > When starting:
> > > > > > >
> > > > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3798
> > > > > > >
> > > > > > > On connect:
> > > > > > >
> > > > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/gatt-database.c#n3741
> > > > > > >
> > > > > > > If those lines are not being triggered it is likely a client problem
> > > > > > > which for some reason had not subscribed to received service changes
> > > > > > > for some reason, if it doesn't subscribe to service changes then it
> > > > > > > shall not cache any attribute and attempt to rediscover on every
> > > > > > > connection.
> > > > > > >
> > > > > > > > The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.
> > > > > > > >
> > > > > > > > I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.
> > > > > > > >
> > > > > > > > Questions:
> > > > > > > > 1. How is it possible that this still happens even if the bluetooth cache is disabled?
> > > > > > > > 2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
> > > > > > > > 3. Is there anything else we should look into?
> > > > > > > >
> > > > > > > > We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > > THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Luiz Augusto von Dentz
> > > >
> > > >
> > > >
> > > > --
> > > > Luiz Augusto von Dentz
> >
> >
> >
> > --
> > Luiz Augusto von Dentz

2021-04-16 08:01:56

by Jamie Mccrae

[permalink] [raw]
Subject: RE: Disabled bluetooth cache. But the app still getting wrong data?

Hi Kenny,
The way it works is when a device connects (it might just be a normal connection, it might be only if it pairs, it's vender specific) it will generally cache the GATT table of the remote device, this is so that future connections are faster as it won't need to send multiple read commands to get it. The easy way for that device to know if the GATT table has changed is it uses a specific GATT attribute with a GATT table hash and reads that, if it's changed then it needs to clear the cache and re-read the table. That only works when a device connects and reads it, if the GATT table is updated during a connection, then an indication can be sent to that device telling it the GATT table hash has changed and it can re-read it and clear it during a connection.
If you are connecting a phone to your embedded device with the GATT table being on the embedded device then the cache is stored on the phone, there doesn't need to be a GATT table hash on the embedded device because the phone doesn't have a GATT table and iOS/android will create the cache themselves, you cannot disable that from the embedded device side, whatever cache you enable/disable on the embedded system is for that system only, it does not propagate that configuration value to other devices.
Thanks,
Jamie

-----Original Message-----
From: Kenny Bian <[email protected]>
Sent: 16 April 2021 05:03
To: Luiz Augusto von Dentz <[email protected]>
Cc: Jamie Mccrae <[email protected]>; [email protected]
Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?

EXTERNAL EMAIL: Be careful with attachments and links.

Hi Luiz,

One more question. Once we upgrade our device to Ubuntu 20.04 LTS with BlueZ 5.53, we'll test "Service Changed Indication". Is GATT cache related to "Service Changed Indication"? If they are related, then I guess we should turn on the GATT cache in /etc/bluetooth/main.conf?
Right now, the GATT cache is turned off in our device.

Thanks,
Kenny

On Thu, Apr 15, 2021 at 8:06 PM Kenny Bian <[email protected]> wrote:
>
> Hi Luiz,
>
> Thank you so much! You're absolutely right.
>
> I'll talk to our team to upgrade our Ubuntu to 20.04 LTS. Then I'll
> test the BlueZ 5.53 which is the latest version on Ubuntu 20.04 LTS.
>
> Thanks again!
> Kenny
>
> On Thu, Apr 15, 2021 at 5:49 PM Luiz Augusto von Dentz
> <[email protected]> wrote:
> >
> > Hi Kenny,
> >
> > On Thu, Apr 15, 2021 at 5:21 PM Kenny Bian <[email protected]> wrote:
> > >
> > > Hi Luiz,
> > >
> > > Thank you so much. I appreciate it.
> > >
> > > That was my bad. I just checked. Our system is Ubuntu 18.04. The
> > > BlueZ version is "5.48-0ubuntu3.4" which is different from the
> > > versions in http://www.bluez.org/. I'm trying to figure out if
> > > this version includes the version 5.50. Do you know the
> > > corresponding BlueZ version for "5.48-0ubuntu3.4"?
> >
> > 5.48, which was the version available in 2018, more than 3 years/10
> > releases ago, a lot of things have been fixed over this time so Id
> > really recommend to upgrade to the latest if you want to have some
> > support.
> >
> > > Thanks!
> > > Kenny
> > >
> > > On Thu, Apr 15, 2021 at 1:31 PM Luiz Augusto von Dentz
> > > <[email protected]> wrote:
> > > >
> > > > Hi Kenny,
> > > >
> > > > On Wed, Apr 14, 2021 at 10:44 PM Kenny Bian <[email protected]> wrote:
> > > > >
> > > > > Hi Luiz,
> > > > >
> > > > > There is a UI example of nRF Connect which shows the "Service
> > > > > Changed Indication" is under "Generic
> > > > > Attribute"(http://forum.espruino.com/conversations/357737/).
> > > > > We're using BlueZ 5.48. I tried to add it by modifying example-gatt-server:
> > > >
> > > >
> > > > You should have mentioned that you are using 5.48, you need the following patch:
> > > >
> > > > commit a0b886e26c83ad4dfbf0b2b2e024c8ce9ead9a7d
> > > > Author: Szymon Janc <[email protected]>
> > > > Date: Wed Mar 28 12:10:55 2018 +0200
> > > >
> > > > gatt: Add support for storing Service Changed CCC value
> > > >
> > > > This adds support for storing CCC value of Service Changed
> > > > characteristic. Once bluetoothd is restart stored values are read
> > > > and any device subscribed to indications will receive Service Changed
> > > > indication with 0x00010-0xffff value. This is to invalidate any
> > > > non-core services since there is no way to verify if applications
> > > > will register their services in same order (or at all).
> > > >
> > > > This fix accessing invalid handles by stacks that rely only on Service
> > > > Changed indication for rediscovery ie. Apple iOS.
> > > >
> > > > Or upgrade to a BlueZ release that has the above patch, which should be 5.50:
> > > >
> > > > +ver 5.50:
> > > > + Fix issue with GATT and reading long values.
> > > > + Fix issue with GATT and reading multiple includes.
> > > > + Fix issue with GATT and service changes when offline.
> > > > + Fix issue with handling secondary service discovery.
> > > > + Fix issue with handling persistency of CCC values.
> > > >
> > > > >
> > > > > class ServiceChangedChrc(Characteristic):
> > > > > SCI_UUID = '00002a05-0000-1000-8000-00805f9b34fb'
> > > > >
> > > > > def __init__(self, bus, index, service):
> > > > > Characteristic.__init__(
> > > > > self, bus, index,
> > > > > self.SCI_UUID,
> > > > > ['indicate'],
> > > > > service)
> > > > >
> > > > > def IndicateValue(self, options):
> > > > > # This function doesn't exist. Just list it here.
> > > > > pass
> > > > >
> > > > >
> > > > > class ServiceChangedService(Service):
> > > > > MY_UUID = '00001801-0000-1000-8000-00805f9b34fb'
> > > > >
> > > > > def __init__(self, bus, index):
> > > > > Service.__init__(self, bus, index, self.MY_UUID, True)
> > > > > self.add_characteristic(ServiceChangedChrc(bus, 0,
> > > > > self))
> > > > >
> > > > >
> > > > > class Application(dbus.service.Object):
> > > > > """
> > > > > org.bluez.GattApplication1 interface implementation
> > > > > """
> > > > > def __init__(self, bus):
> > > > > self.path = '/'
> > > > > self.services = []
> > > > > dbus.service.Object.__init__(self, bus, self.path)
> > > > > self.add_service(HeartRateService(bus, 0))
> > > > > self.add_service(BatteryService(bus, 1))
> > > > > self.add_service(TestService(bus, 2))
> > > > > self.add_service(ServiceChangedService(bus, 3))
> > > > >
> > > > >
> > > > > When I run it, there is "Failed to register application:
> > > > > org.bluez.Error.Failed: Failed to create entry in database".
> > > > > That is in the function client_ready_cb() in gatt-database.c
> > > > > in BlueZ code. Do you know how to fix it?
> > > >
> > > > You don't need to register Service Changed, the daemon already
> > > > takes care of registering it and there could only be one Service
> > > > Changed in the database.
> > > >
> > > > > Thanks!
> > > > > Kenny
> > > > >
> > > > > On Tue, Apr 13, 2021 at 9:46 PM Kenny Bian <[email protected]> wrote:
> > > > > >
> > > > > > Hi Jamie, Luiz,
> > > > > >
> > > > > > Thanks for your info.
> > > > > >
> > > > > > Luiz, to answer your questions:
> > > > > > BlueZ doesn't seem to emit "Service Changed Indication". Or
> > > > > > it does emit it, but we don't know how to verify it.
> > > > > > Yes, BlueZ acts as a GATT server which runs on Linux.
> > > > > >
> > > > > > So my questions:
> > > > > > How can I tell if "Service Changed Indication" is working? I
> > > > > > need to show it to our app developers that the GATT server
> > > > > > does send the "Service Changed Indication" when there is a service change.
> > > > > > By disabling the GATT's cache in "/etc/bluetooth/main.conf",
> > > > > > the "Service Changed Indication" is disabled?
> > > > > > Can "Service Changed Indication" be enabled in a conf file?
> > > > > > Or should it be done in the Python code? Could you please
> > > > > > help find out that info?
> > > > > > Our GATT server is based on the Python code
> > > > > > examples(https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/test).
> > > > > > I don't know how to enable "Service Changed Indication" or
> > > > > > "Robust Caching" in Python code on the GATT server side.
> > > > > > There is no document or code samples about that. Please let
> > > > > > me know if there is any document or Python code example.
> > > > > >
> > > > > > Thanks!
> > > > > > Kenny
> > > > > >
> > > > > > On Tue, Apr 13, 2021 at 2:57 PM Luiz Augusto von Dentz
> > > > > > <[email protected]> wrote:
> > > > > > >
> > > > > > > Hi Jamie, Brian,
> > > > > > >
> > > > > > > On Tue, Apr 13, 2021 at 2:03 AM Jamie Mccrae
> > > > > > > <[email protected]> wrote:
> > > > > > > >
> > > > > > > > Hi Kenny,
> > > > > > > > Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.
> > > > > > >
> > > > > > > Yep, and while at it implement the so called Robust
> > > > > > > Caching feature so we can detect if anything has changed by reading the DB Hash.
> > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Jamie
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Kenny Bian <[email protected]>
> > > > > > > > Sent: 13 April 2021 06:59
> > > > > > > > To: [email protected]
> > > > > > > > Subject: Disabled bluetooth cache. But the app still getting wrong data?
> > > > > > > >
> > > > > > > > EXTERNAL EMAIL: Be careful with attachments and links.
> > > > > > > >
> > > > > > > > Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
> > > > > > > > By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
> > > > > > > > In order to keep the pairing information, the "/var/lib/bluetooth"
> > > > > > > > folder is copied over to the new build's partition. We
> > > > > > > > realized that there is no "service changed indication".
> > > > > > > > The app can't handle the changed services. So we
> > > > > > > > disabled the bluetooth cache by set this in
> > > > > > > > "/etc/bluetooth/main.conf":
> > > > > > > > [GATT]
> > > > > > > > Cache = no
> > > > > > >
> > > > > > > When you say the app can't handle changed service do you
> > > > > > > mean BlueZ doesn't emit changes to the attributes (via
> > > > > > > Service Changed) or is it really the application not being able to handle the changes?
> > > > > > >
> > > > > > > > But recently, we saw the problem again even if the
> > > > > > > > bluetooth cache is
> > > > > > > > disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.
> > > > > > >
> > > > > > > So BlueZ is acting as the server, right? The Cache only
> > > > > > > applies to the client portion, there is no such thing as disabling the remote cache.
> > > > > > > I don't see any incoming Read By Group Request from the
> > > > > > > remote so it is very likely that it has cached the values,
> > > > > > > there is no Read By Type for the DB Hash either which is
> > > > > > > quite surprising to me since that is required for stacks
> > > > > > > supporting Robust Caching which I believe is the case of iOS.
> > > > > > >
> > > > > > > Anyway, I would check that the following lines are being triggered:
> > > > > > >
> > > > > > > When starting:
> > > > > > >
> > > > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/sr
> > > > > > > c/gatt-database.c#n3798
> > > > > > >
> > > > > > > On connect:
> > > > > > >
> > > > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/sr
> > > > > > > c/gatt-database.c#n3741
> > > > > > >
> > > > > > > If those lines are not being triggered it is likely a
> > > > > > > client problem which for some reason had not subscribed to
> > > > > > > received service changes for some reason, if it doesn't
> > > > > > > subscribe to service changes then it shall not cache any
> > > > > > > attribute and attempt to rediscover on every connection.
> > > > > > >
> > > > > > > > The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.
> > > > > > > >
> > > > > > > > I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.
> > > > > > > >
> > > > > > > > Questions:
> > > > > > > > 1. How is it possible that this still happens even if the bluetooth cache is disabled?
> > > > > > > > 2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
> > > > > > > > 3. Is there anything else we should look into?
> > > > > > > >
> > > > > > > > We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > > THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Luiz Augusto von Dentz
> > > >
> > > >
> > > >
> > > > --
> > > > Luiz Augusto von Dentz
> >
> >
> >
> > --
> > Luiz Augusto von Dentz

2021-04-20 21:37:23

by Kenny Bian

[permalink] [raw]
Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?

Thank you so much, Jamie. That well explained it.

On Thu, Apr 15, 2021 at 11:34 PM Jamie Mccrae
<[email protected]> wrote:
>
> Hi Kenny,
> The way it works is when a device connects (it might just be a normal connection, it might be only if it pairs, it's vender specific) it will generally cache the GATT table of the remote device, this is so that future connections are faster as it won't need to send multiple read commands to get it. The easy way for that device to know if the GATT table has changed is it uses a specific GATT attribute with a GATT table hash and reads that, if it's changed then it needs to clear the cache and re-read the table. That only works when a device connects and reads it, if the GATT table is updated during a connection, then an indication can be sent to that device telling it the GATT table hash has changed and it can re-read it and clear it during a connection.
> If you are connecting a phone to your embedded device with the GATT table being on the embedded device then the cache is stored on the phone, there doesn't need to be a GATT table hash on the embedded device because the phone doesn't have a GATT table and iOS/android will create the cache themselves, you cannot disable that from the embedded device side, whatever cache you enable/disable on the embedded system is for that system only, it does not propagate that configuration value to other devices.
> Thanks,
> Jamie
>
> -----Original Message-----
> From: Kenny Bian <[email protected]>
> Sent: 16 April 2021 05:03
> To: Luiz Augusto von Dentz <[email protected]>
> Cc: Jamie Mccrae <[email protected]>; [email protected]
> Subject: Re: Disabled bluetooth cache. But the app still getting wrong data?
>
> EXTERNAL EMAIL: Be careful with attachments and links.
>
> Hi Luiz,
>
> One more question. Once we upgrade our device to Ubuntu 20.04 LTS with BlueZ 5.53, we'll test "Service Changed Indication". Is GATT cache related to "Service Changed Indication"? If they are related, then I guess we should turn on the GATT cache in /etc/bluetooth/main.conf?
> Right now, the GATT cache is turned off in our device.
>
> Thanks,
> Kenny
>
> On Thu, Apr 15, 2021 at 8:06 PM Kenny Bian <[email protected]> wrote:
> >
> > Hi Luiz,
> >
> > Thank you so much! You're absolutely right.
> >
> > I'll talk to our team to upgrade our Ubuntu to 20.04 LTS. Then I'll
> > test the BlueZ 5.53 which is the latest version on Ubuntu 20.04 LTS.
> >
> > Thanks again!
> > Kenny
> >
> > On Thu, Apr 15, 2021 at 5:49 PM Luiz Augusto von Dentz
> > <[email protected]> wrote:
> > >
> > > Hi Kenny,
> > >
> > > On Thu, Apr 15, 2021 at 5:21 PM Kenny Bian <[email protected]> wrote:
> > > >
> > > > Hi Luiz,
> > > >
> > > > Thank you so much. I appreciate it.
> > > >
> > > > That was my bad. I just checked. Our system is Ubuntu 18.04. The
> > > > BlueZ version is "5.48-0ubuntu3.4" which is different from the
> > > > versions in http://www.bluez.org/. I'm trying to figure out if
> > > > this version includes the version 5.50. Do you know the
> > > > corresponding BlueZ version for "5.48-0ubuntu3.4"?
> > >
> > > 5.48, which was the version available in 2018, more than 3 years/10
> > > releases ago, a lot of things have been fixed over this time so Id
> > > really recommend to upgrade to the latest if you want to have some
> > > support.
> > >
> > > > Thanks!
> > > > Kenny
> > > >
> > > > On Thu, Apr 15, 2021 at 1:31 PM Luiz Augusto von Dentz
> > > > <[email protected]> wrote:
> > > > >
> > > > > Hi Kenny,
> > > > >
> > > > > On Wed, Apr 14, 2021 at 10:44 PM Kenny Bian <[email protected]> wrote:
> > > > > >
> > > > > > Hi Luiz,
> > > > > >
> > > > > > There is a UI example of nRF Connect which shows the "Service
> > > > > > Changed Indication" is under "Generic
> > > > > > Attribute"(http://forum.espruino.com/conversations/357737/).
> > > > > > We're using BlueZ 5.48. I tried to add it by modifying example-gatt-server:
> > > > >
> > > > >
> > > > > You should have mentioned that you are using 5.48, you need the following patch:
> > > > >
> > > > > commit a0b886e26c83ad4dfbf0b2b2e024c8ce9ead9a7d
> > > > > Author: Szymon Janc <[email protected]>
> > > > > Date: Wed Mar 28 12:10:55 2018 +0200
> > > > >
> > > > > gatt: Add support for storing Service Changed CCC value
> > > > >
> > > > > This adds support for storing CCC value of Service Changed
> > > > > characteristic. Once bluetoothd is restart stored values are read
> > > > > and any device subscribed to indications will receive Service Changed
> > > > > indication with 0x00010-0xffff value. This is to invalidate any
> > > > > non-core services since there is no way to verify if applications
> > > > > will register their services in same order (or at all).
> > > > >
> > > > > This fix accessing invalid handles by stacks that rely only on Service
> > > > > Changed indication for rediscovery ie. Apple iOS.
> > > > >
> > > > > Or upgrade to a BlueZ release that has the above patch, which should be 5.50:
> > > > >
> > > > > +ver 5.50:
> > > > > + Fix issue with GATT and reading long values.
> > > > > + Fix issue with GATT and reading multiple includes.
> > > > > + Fix issue with GATT and service changes when offline.
> > > > > + Fix issue with handling secondary service discovery.
> > > > > + Fix issue with handling persistency of CCC values.
> > > > >
> > > > > >
> > > > > > class ServiceChangedChrc(Characteristic):
> > > > > > SCI_UUID = '00002a05-0000-1000-8000-00805f9b34fb'
> > > > > >
> > > > > > def __init__(self, bus, index, service):
> > > > > > Characteristic.__init__(
> > > > > > self, bus, index,
> > > > > > self.SCI_UUID,
> > > > > > ['indicate'],
> > > > > > service)
> > > > > >
> > > > > > def IndicateValue(self, options):
> > > > > > # This function doesn't exist. Just list it here.
> > > > > > pass
> > > > > >
> > > > > >
> > > > > > class ServiceChangedService(Service):
> > > > > > MY_UUID = '00001801-0000-1000-8000-00805f9b34fb'
> > > > > >
> > > > > > def __init__(self, bus, index):
> > > > > > Service.__init__(self, bus, index, self.MY_UUID, True)
> > > > > > self.add_characteristic(ServiceChangedChrc(bus, 0,
> > > > > > self))
> > > > > >
> > > > > >
> > > > > > class Application(dbus.service.Object):
> > > > > > """
> > > > > > org.bluez.GattApplication1 interface implementation
> > > > > > """
> > > > > > def __init__(self, bus):
> > > > > > self.path = '/'
> > > > > > self.services = []
> > > > > > dbus.service.Object.__init__(self, bus, self.path)
> > > > > > self.add_service(HeartRateService(bus, 0))
> > > > > > self.add_service(BatteryService(bus, 1))
> > > > > > self.add_service(TestService(bus, 2))
> > > > > > self.add_service(ServiceChangedService(bus, 3))
> > > > > >
> > > > > >
> > > > > > When I run it, there is "Failed to register application:
> > > > > > org.bluez.Error.Failed: Failed to create entry in database".
> > > > > > That is in the function client_ready_cb() in gatt-database.c
> > > > > > in BlueZ code. Do you know how to fix it?
> > > > >
> > > > > You don't need to register Service Changed, the daemon already
> > > > > takes care of registering it and there could only be one Service
> > > > > Changed in the database.
> > > > >
> > > > > > Thanks!
> > > > > > Kenny
> > > > > >
> > > > > > On Tue, Apr 13, 2021 at 9:46 PM Kenny Bian <[email protected]> wrote:
> > > > > > >
> > > > > > > Hi Jamie, Luiz,
> > > > > > >
> > > > > > > Thanks for your info.
> > > > > > >
> > > > > > > Luiz, to answer your questions:
> > > > > > > BlueZ doesn't seem to emit "Service Changed Indication". Or
> > > > > > > it does emit it, but we don't know how to verify it.
> > > > > > > Yes, BlueZ acts as a GATT server which runs on Linux.
> > > > > > >
> > > > > > > So my questions:
> > > > > > > How can I tell if "Service Changed Indication" is working? I
> > > > > > > need to show it to our app developers that the GATT server
> > > > > > > does send the "Service Changed Indication" when there is a service change.
> > > > > > > By disabling the GATT's cache in "/etc/bluetooth/main.conf",
> > > > > > > the "Service Changed Indication" is disabled?
> > > > > > > Can "Service Changed Indication" be enabled in a conf file?
> > > > > > > Or should it be done in the Python code? Could you please
> > > > > > > help find out that info?
> > > > > > > Our GATT server is based on the Python code
> > > > > > > examples(https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/test).
> > > > > > > I don't know how to enable "Service Changed Indication" or
> > > > > > > "Robust Caching" in Python code on the GATT server side.
> > > > > > > There is no document or code samples about that. Please let
> > > > > > > me know if there is any document or Python code example.
> > > > > > >
> > > > > > > Thanks!
> > > > > > > Kenny
> > > > > > >
> > > > > > > On Tue, Apr 13, 2021 at 2:57 PM Luiz Augusto von Dentz
> > > > > > > <[email protected]> wrote:
> > > > > > > >
> > > > > > > > Hi Jamie, Brian,
> > > > > > > >
> > > > > > > > On Tue, Apr 13, 2021 at 2:03 AM Jamie Mccrae
> > > > > > > > <[email protected]> wrote:
> > > > > > > > >
> > > > > > > > > Hi Kenny,
> > > > > > > > > Why not just add the service changed indication as you refer to below? It was purposely designed for this specific purpose, you're trying to work around an issue created because you don't want to use the feature that prevents this issue. Any workaround is just that, a workaround, and might not work as intended.
> > > > > > > >
> > > > > > > > Yep, and while at it implement the so called Robust
> > > > > > > > Caching feature so we can detect if anything has changed by reading the DB Hash.
> > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Jamie
> > > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: Kenny Bian <[email protected]>
> > > > > > > > > Sent: 13 April 2021 06:59
> > > > > > > > > To: [email protected]
> > > > > > > > > Subject: Disabled bluetooth cache. But the app still getting wrong data?
> > > > > > > > >
> > > > > > > > > EXTERNAL EMAIL: Be careful with attachments and links.
> > > > > > > > >
> > > > > > > > > Previously we had an issue: if there is a change of characteristics in the new build of our firmware, then the app will get the wrong data.
> > > > > > > > > By saying changed characteristics, it can be an added or removed characteristic, or adding notification to an existing characteristic.
> > > > > > > > > In order to keep the pairing information, the "/var/lib/bluetooth"
> > > > > > > > > folder is copied over to the new build's partition. We
> > > > > > > > > realized that there is no "service changed indication".
> > > > > > > > > The app can't handle the changed services. So we
> > > > > > > > > disabled the bluetooth cache by set this in
> > > > > > > > > "/etc/bluetooth/main.conf":
> > > > > > > > > [GATT]
> > > > > > > > > Cache = no
> > > > > > > >
> > > > > > > > When you say the app can't handle changed service do you
> > > > > > > > mean BlueZ doesn't emit changes to the attributes (via
> > > > > > > > Service Changed) or is it really the application not being able to handle the changes?
> > > > > > > >
> > > > > > > > > But recently, we saw the problem again even if the
> > > > > > > > > bluetooth cache is
> > > > > > > > > disabled: in the build number 101, a characteristic is removed. But when we upgrade the build from 100 to 101, the app gets the wrong data. We looked at the log. When the app tries to read temperature by using the temperature UUID, somehow the bluetooth service we created received the request to read the "device name"(device name UUID). So the "device name" is returned to the app as the temperature. This looks like the same behavior as the bluetooth cache is not disabled. I looked at the "/var/lib/bluetooth/[BT_MAC]/cache" folder. There is no "[Attributes]" section in the files in the folder. That means the disabled cache seems working.
> > > > > > > >
> > > > > > > > So BlueZ is acting as the server, right? The Cache only
> > > > > > > > applies to the client portion, there is no such thing as disabling the remote cache.
> > > > > > > > I don't see any incoming Read By Group Request from the
> > > > > > > > remote so it is very likely that it has cached the values,
> > > > > > > > there is no Read By Type for the DB Hash either which is
> > > > > > > > quite surprising to me since that is required for stacks
> > > > > > > > supporting Robust Caching which I believe is the case of iOS.
> > > > > > > >
> > > > > > > > Anyway, I would check that the following lines are being triggered:
> > > > > > > >
> > > > > > > > When starting:
> > > > > > > >
> > > > > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/sr
> > > > > > > > c/gatt-database.c#n3798
> > > > > > > >
> > > > > > > > On connect:
> > > > > > > >
> > > > > > > > https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/sr
> > > > > > > > c/gatt-database.c#n3741
> > > > > > > >
> > > > > > > > If those lines are not being triggered it is likely a
> > > > > > > > client problem which for some reason had not subscribed to
> > > > > > > > received service changes for some reason, if it doesn't
> > > > > > > > subscribe to service changes then it shall not cache any
> > > > > > > > attribute and attempt to rediscover on every connection.
> > > > > > > >
> > > > > > > > > The only way to fix this issue is to force exit the mobile app on the phone and "Forget This Device" in iOS or "Unpair" in Android.
> > > > > > > > >
> > > > > > > > > I looked at the btmon(see attached). For the working btmon log, there is "Attribute group list: XX entries" under "ACL Data TX". But there is no "Attribute group list: XX entries" under "ACL Data TX" in the attached problematic btmon log.
> > > > > > > > >
> > > > > > > > > Questions:
> > > > > > > > > 1. How is it possible that this still happens even if the bluetooth cache is disabled?
> > > > > > > > > 2. Is this the problem on the Linux side which runs the GATT server or on the mobile side?
> > > > > > > > > 3. Is there anything else we should look into?
> > > > > > > > >
> > > > > > > > > We're going to release our product soon. This is a critical issue for us. Please help if you have any suggestions.
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > > THIS MESSAGE, ANY ATTACHMENT(S), AND THE INFORMATION CONTAINED HEREIN MAY BE PROPRIETARY TO LAIRD CONNECTIVITY, INC. AND/OR ANOTHER PARTY, AND MAY FURTHER BE INTENDED TO BE KEPT CONFIDENTIAL. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE DELETE THE EMAIL AND ANY ATTACHMENTS, AND IMMEDIATELY NOTIFY THE SENDER BY RETURN EMAIL. THIS MESSAGE AND ITS CONTENTS ARE THE PROPERTY OF LAIRD CONNECTIVITY, INC. AND MAY NOT BE REPRODUCED OR USED WITHOUT THE EXPRESS WRITTEN CONSENT OF LAIRD CONNECTIVITY, INC.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Luiz Augusto von Dentz
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Luiz Augusto von Dentz
> > >
> > >
> > >
> > > --
> > > Luiz Augusto von Dentz