2021-03-17 07:40:52

by Kenny Bian

[permalink] [raw]
Subject: Have to delete "/var/lib/bluetooth" folder in order to get all service characteristics available

In order to keep the pairing information, we carry over the
"/var/lib/bluetooth" folder from an older build to the new build in
the Linux system when doing firmware upgrade. This way, our end user
doesn't have to choose "Forget This Device" each time after the
firmware upgrade. Otherwise, our end user has to "Forget This Device"
and re-pair because the pairing info can't be found in the Linux
device. We also set the GATT cache as "no" in
"/etc/bluetooth/main.conf". Then the mobile app won't crash when there
is any "service changed" such as adding notification to some existing
characteristics.

We organize our characteristics under the services. There are 10
services on the Linux device. Each service has 5 to 10
characteristics. However, we found a new problem recently. The problem
is: the first 8 services are fine. But the last one or two services
only have the service UUID listed in nRF or our mobile app. There are
no characteristics showing up under these one or two services. This
caused our mobile app can't read the characteristics data under these
problematic services. I checked the nRF. It's the same thing. nRF only
shows the last one or two services with the service UUIDs. There are
no characteristics listed under those 2 services.

In order to fix this issue, we have to do these(let's call it "The Steps"):
- Remove the "/var/lib/bluetooth" folder from the Linux system
- Restart our BlueZ based service in Python in the Linux system
- Choose "Forget This Device" on the iPhone. After this operation, the
"/var/lib/bluetooth" is generated in the Linux device.
- Connect to the device from our mobile app or from nRF. We can see
all of the 10 services in the list. Each characteristic under the
service can be seen.
- We are able to read the data by using our mobile app or by using nRF

I compared the files in the "/var/lib/bluetooth" folder. As we
disabled the GATT cache, there is no "cache" folder. There are only 2
files in the folder. Here is the folder structure:
/var/lib/bluetooth:
|
+----00:11:22:33:44:55/
|
+----6C:92:11:87:11:25/
|
+----attributes
+----info

I kept one copy of the files after the firmware upgrade. I also kept a
copy of the files after the folder was re-generated. I compared the
file "attributes" and "info". For the attributes, they are the same.
For the "info" file, there are only 3 differences:
[General]
Name=PhoneX
. . .

[SlaveLongTermKey]
Key=252256E19E26F3ED9FD0B99B7B5B0193
. . .

[LinkKey]
Key=1B593296571E2C48CF6952E60395E304
. . .


I don't think these differences could cause the issue. But the fact
is, if I delete the "/var/lib/bluetooth" folder and go thru "The
Steps" above, the problem will be gone. All of the services will have
their characteristics showing up in nRF. The data can be read from the
characteristics. This means the folder "/var/lib/bluetooth" is related
to the issue. But I don't know why.

Here is also another thing. I subscribed to the "PropertiesChanged"
signal. In the callback function "properties_changed", I print out the
"changed". Here is the code to subscribe the "PropertiesChanged":

bus.add_signal_receiver(
properties_changed,
dbus_interface="org.freedesktop.DBus.Properties",
signal_name="PropertiesChanged",
arg0="org.bluez.Device1",
path_keyword="path",
)

If the folder "/var/lib/bluetooth" is removed and go thru "The Steps",
I saw the log like this:
dbus.Dictionary({dbus.String('UUIDs'):
dbus.Array([dbus.String('00000000-deca-fade-deca-deafdecacafe'),
dbus.String('00001000-0000-1000-8000-00805f9b34fb'),
dbus.String('0000110a-0000-1000-8000-00805f9b34fb'),
dbus.String('0000110c-0000-1000-8000-00805f9b34fb'),
dbus.String('0000110e-0000-1000-8000-00805f9b34fb'),
dbus.String('00001116-0000-1000-8000-00805f9b34fb'),
dbus.String('0000111f-0000-1000-8000-00805f9b34fb'),
dbus.String('0000112f-0000-1000-8000-00805f9b34fb'),
dbus.String('00001132-0000-1000-8000-00805f9b34fb'),
dbus.String('00001200-0000-1000-8000-00805f9b34fb'),
dbus.String('00001800-0000-1000-8000-00805f9b34fb'),
dbus.String('00001801-0000-1000-8000-00805f9b34fb'),
dbus.String('00001805-0000-1000-8000-00805f9b34fb'),
dbus.String('0000180a-0000-1000-8000-00805f9b34fb'),
dbus.String('0000180f-0000-1000-8000-00805f9b34fb'),
dbus.String('02030302-1d19-415f-86f2-22a2106a0a77'),
dbus.String('7905f431-b5ce-4e99-a40f-4b1e122d00d0'),
dbus.String('89d3502b-0f36-433a-8ef4-c502ad55f8dc'),
dbus.String('9fa480e0-4967-4542-9390-d343dc5d04ae'),
dbus.String('d0611e78-bbb4-4591-a5f8-487910ae4366')],
signature=dbus.Signature('s'), variant_level=1),
dbus.String('ServicesResolved'): dbus.Boolean(True, variant_level=1)},
signature=dbus.Signature('sv'))

If I check the log after the firmware is upgraded without deleting the
"/var/lib/bluetooth" folder, the log looks like this:
dbus.Dictionary({dbus.String('ServicesResolved'): dbus.Boolean(True,
variant_level=1)}, signature=dbus.Signature('sv'))

As we can see, they are different. Is it something to do with D-Bus?

We understand that if we don't carry over the "/var/lib/bluetooth"
folder, then there is no this issue. But our end user will have to
choose "Forget This Device" each time after the firmware is upgraded.
Otherwise, the end user can't connect to the Linux device because the
pairing info can't be found. Is there a way to fix this issue? Namely,
is there a way to keep the pairing info and also avoid this issue?

Thanks in advance!


2021-03-17 21:43:28

by Luiz Augusto von Dentz

[permalink] [raw]
Subject: Re: Have to delete "/var/lib/bluetooth" folder in order to get all service characteristics available

Hi Kenny,

On Wed, Mar 17, 2021 at 12:40 AM Kenny Bian <[email protected]> wrote:
>
> In order to keep the pairing information, we carry over the
> "/var/lib/bluetooth" folder from an older build to the new build in
> the Linux system when doing firmware upgrade. This way, our end user
> doesn't have to choose "Forget This Device" each time after the
> firmware upgrade. Otherwise, our end user has to "Forget This Device"
> and re-pair because the pairing info can't be found in the Linux
> device. We also set the GATT cache as "no" in
> "/etc/bluetooth/main.conf". Then the mobile app won't crash when there
> is any "service changed" such as adding notification to some existing
> characteristics.

But the cache only applies to client, service changed is for the
server not sure why your mobile would be crashing with that, or is it
disabling ServiceChanged indications with that? If there is no
indication that would explain why the services are not updated since
the mobile OS won't notice any changes on the database.

> We organize our characteristics under the services. There are 10
> services on the Linux device. Each service has 5 to 10
> characteristics. However, we found a new problem recently. The problem
> is: the first 8 services are fine. But the last one or two services
> only have the service UUID listed in nRF or our mobile app. There are
> no characteristics showing up under these one or two services. This
> caused our mobile app can't read the characteristics data under these
> problematic services. I checked the nRF. It's the same thing. nRF only
> shows the last one or two services with the service UUIDs. There are
> no characteristics listed under those 2 services.
>
> In order to fix this issue, we have to do these(let's call it "The Steps"):
> - Remove the "/var/lib/bluetooth" folder from the Linux system
> - Restart our BlueZ based service in Python in the Linux system
> - Choose "Forget This Device" on the iPhone. After this operation, the
> "/var/lib/bluetooth" is generated in the Linux device.
> - Connect to the device from our mobile app or from nRF. We can see
> all of the 10 services in the list. Each characteristic under the
> service can be seen.
> - We are able to read the data by using our mobile app or by using nRF
>
> I compared the files in the "/var/lib/bluetooth" folder. As we
> disabled the GATT cache, there is no "cache" folder. There are only 2
> files in the folder. Here is the folder structure:
> /var/lib/bluetooth:
> |
> +----00:11:22:33:44:55/
> |
> +----6C:92:11:87:11:25/
> |
> +----attributes
> +----info
>
> I kept one copy of the files after the firmware upgrade. I also kept a
> copy of the files after the folder was re-generated. I compared the
> file "attributes" and "info". For the attributes, they are the same.
> For the "info" file, there are only 3 differences:
> [General]
> Name=PhoneX
> . . .
>
> [SlaveLongTermKey]
> Key=252256E19E26F3ED9FD0B99B7B5B0193
> . . .
>
> [LinkKey]
> Key=1B593296571E2C48CF6952E60395E304
> . . .
>
>
> I don't think these differences could cause the issue. But the fact
> is, if I delete the "/var/lib/bluetooth" folder and go thru "The
> Steps" above, the problem will be gone. All of the services will have
> their characteristics showing up in nRF. The data can be read from the
> characteristics. This means the folder "/var/lib/bluetooth" is related
> to the issue. But I don't know why.
>
> Here is also another thing. I subscribed to the "PropertiesChanged"
> signal. In the callback function "properties_changed", I print out the
> "changed". Here is the code to subscribe the "PropertiesChanged":
>
> bus.add_signal_receiver(
> properties_changed,
> dbus_interface="org.freedesktop.DBus.Properties",
> signal_name="PropertiesChanged",
> arg0="org.bluez.Device1",
> path_keyword="path",
> )
>
> If the folder "/var/lib/bluetooth" is removed and go thru "The Steps",
> I saw the log like this:
> dbus.Dictionary({dbus.String('UUIDs'):
> dbus.Array([dbus.String('00000000-deca-fade-deca-deafdecacafe'),
> dbus.String('00001000-0000-1000-8000-00805f9b34fb'),
> dbus.String('0000110a-0000-1000-8000-00805f9b34fb'),
> dbus.String('0000110c-0000-1000-8000-00805f9b34fb'),
> dbus.String('0000110e-0000-1000-8000-00805f9b34fb'),
> dbus.String('00001116-0000-1000-8000-00805f9b34fb'),
> dbus.String('0000111f-0000-1000-8000-00805f9b34fb'),
> dbus.String('0000112f-0000-1000-8000-00805f9b34fb'),
> dbus.String('00001132-0000-1000-8000-00805f9b34fb'),
> dbus.String('00001200-0000-1000-8000-00805f9b34fb'),
> dbus.String('00001800-0000-1000-8000-00805f9b34fb'),
> dbus.String('00001801-0000-1000-8000-00805f9b34fb'),
> dbus.String('00001805-0000-1000-8000-00805f9b34fb'),
> dbus.String('0000180a-0000-1000-8000-00805f9b34fb'),
> dbus.String('0000180f-0000-1000-8000-00805f9b34fb'),
> dbus.String('02030302-1d19-415f-86f2-22a2106a0a77'),
> dbus.String('7905f431-b5ce-4e99-a40f-4b1e122d00d0'),
> dbus.String('89d3502b-0f36-433a-8ef4-c502ad55f8dc'),
> dbus.String('9fa480e0-4967-4542-9390-d343dc5d04ae'),
> dbus.String('d0611e78-bbb4-4591-a5f8-487910ae4366')],
> signature=dbus.Signature('s'), variant_level=1),
> dbus.String('ServicesResolved'): dbus.Boolean(True, variant_level=1)},
> signature=dbus.Signature('sv'))
>
> If I check the log after the firmware is upgraded without deleting the
> "/var/lib/bluetooth" folder, the log looks like this:
> dbus.Dictionary({dbus.String('ServicesResolved'): dbus.Boolean(True,
> variant_level=1)}, signature=dbus.Signature('sv'))
>
> As we can see, they are different. Is it something to do with D-Bus?
>
> We understand that if we don't carry over the "/var/lib/bluetooth"
> folder, then there is no this issue. But our end user will have to
> choose "Forget This Device" each time after the firmware is upgraded.
> Otherwise, the end user can't connect to the Linux device because the
> pairing info can't be found. Is there a way to fix this issue? Namely,
> is there a way to keep the pairing info and also avoid this issue?

This sounds like ServiceChanged is not being properly propagated when
you do the firmware update, do you have ServiceChanged entry on the
storage:

https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/settings-storage.txt#n321

Is there any indication on btmon when you first connect after doing
the firmware update?

--
Luiz Augusto von Dentz

2021-03-18 05:20:22

by Kenny Bian

[permalink] [raw]
Subject: Re: Have to delete "/var/lib/bluetooth" folder in order to get all service characteristics available

Thank you so much for your reply, Luiz.

We have a small device which runs a Linux system in it. In this
embedded Linux system, there is GATT server runs BLE service which
provides the services with the characteristics for reading and writing
from the mobile device such as iPhone. The folder "/var/lib/bluetooth"
is in the embedded Linux system. We call this embedded Linux as BLE
Server, and the iPhone as the BLE Client. During the testing a few
weeks ago, not only our mobile app crashed, but the nRF also crashed
as the service was changed. Since we have no way to tell whether the
ServiceChanged indication works or not. We disabled the bluetooth
cache. Then we have this new issue.

I never saw the "[ServiceChanged]" section from the "info" file. I
tried to change the attributes of some characteristics. For example,
adding notification in some characteristics. Then restart the BlE
service from the Linux system. I did check the "info" file after that,
but I didn't see the "[ServiceChanged]" section. Is there another way
to have the "[ServiceChanged]" showing up?

Today, I ran the testing again by doing the firmware upgrade. Right
after the firmware upgrade, I used the btmon to capture the
characteristics reading by using our mobile app. Please see the
attached log. As you can see, there are a lot of errors because there
are 5 services which only have the service itself listed(I checked it
by using nRF). For about 30 characteristics under those services can't
be seen. That's why there are errors when the mobile app tries to read
the data. I double checked, the BLE python files between these two
versions of the firmware are exactly the same. So there are no
ServiceChanged. As the reading didn't work, I had to remove the
"/var/lib/bluetooth" folder. Then this folder was recreated again. I
compared the file "attributes" and "info" before and after removal of
the "/var/lib/bluetooth" folder. The "attributes" files are exactly
the same. The "info" files are different. But it makes sense because
only the "[SlaveLongTermKey]" and "[LinkKey]" are different which is
caused by re-pairing from the mobile side, I guess.

So in this case, the BLE service python code is exactly the same. The
"attributes" and "info" files are also the same or slightly different
because of the re-pairing. How come some characteristics cannot be
seen from nRF? As those characteristics are not listed in nRF, neither
nRF or our mobile app can read the data.

Honestly, this situation is pretty much the same as there is no
firmware upgrade because the service file itself and those files under
"/var/lib/bluetooth" are almost the same. How come the normal reading
from the mobile app works fine? But after the firmware upgrade, some
characteristics can't be seen? Is there anything wrong about
advertisement? In fact, the firmware upgrade only copy over the
"/var/lib/bluetooth" in order to keep the pairing information. I have
no idea something went wrong. Are there anything else to look into?

Thanks in advance!

On Wed, Mar 17, 2021 at 11:15 AM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Kenny,
>
> On Wed, Mar 17, 2021 at 12:40 AM Kenny Bian <[email protected]> wrote:
> >
> > In order to keep the pairing information, we carry over the
> > "/var/lib/bluetooth" folder from an older build to the new build in
> > the Linux system when doing firmware upgrade. This way, our end user
> > doesn't have to choose "Forget This Device" each time after the
> > firmware upgrade. Otherwise, our end user has to "Forget This Device"
> > and re-pair because the pairing info can't be found in the Linux
> > device. We also set the GATT cache as "no" in
> > "/etc/bluetooth/main.conf". Then the mobile app won't crash when there
> > is any "service changed" such as adding notification to some existing
> > characteristics.
>
> But the cache only applies to client, service changed is for the
> server not sure why your mobile would be crashing with that, or is it
> disabling ServiceChanged indications with that? If there is no
> indication that would explain why the services are not updated since
> the mobile OS won't notice any changes on the database.
>
> > We organize our characteristics under the services. There are 10
> > services on the Linux device. Each service has 5 to 10
> > characteristics. However, we found a new problem recently. The problem
> > is: the first 8 services are fine. But the last one or two services
> > only have the service UUID listed in nRF or our mobile app. There are
> > no characteristics showing up under these one or two services. This
> > caused our mobile app can't read the characteristics data under these
> > problematic services. I checked the nRF. It's the same thing. nRF only
> > shows the last one or two services with the service UUIDs. There are
> > no characteristics listed under those 2 services.
> >
> > In order to fix this issue, we have to do these(let's call it "The Steps"):
> > - Remove the "/var/lib/bluetooth" folder from the Linux system
> > - Restart our BlueZ based service in Python in the Linux system
> > - Choose "Forget This Device" on the iPhone. After this operation, the
> > "/var/lib/bluetooth" is generated in the Linux device.
> > - Connect to the device from our mobile app or from nRF. We can see
> > all of the 10 services in the list. Each characteristic under the
> > service can be seen.
> > - We are able to read the data by using our mobile app or by using nRF
> >
> > I compared the files in the "/var/lib/bluetooth" folder. As we
> > disabled the GATT cache, there is no "cache" folder. There are only 2
> > files in the folder. Here is the folder structure:
> > /var/lib/bluetooth:
> > |
> > +----00:11:22:33:44:55/
> > |
> > +----6C:92:11:87:11:25/
> > |
> > +----attributes
> > +----info
> >
> > I kept one copy of the files after the firmware upgrade. I also kept a
> > copy of the files after the folder was re-generated. I compared the
> > file "attributes" and "info". For the attributes, they are the same.
> > For the "info" file, there are only 3 differences:
> > [General]
> > Name=PhoneX
> > . . .
> >
> > [SlaveLongTermKey]
> > Key=252256E19E26F3ED9FD0B99B7B5B0193
> > . . .
> >
> > [LinkKey]
> > Key=1B593296571E2C48CF6952E60395E304
> > . . .
> >
> >
> > I don't think these differences could cause the issue. But the fact
> > is, if I delete the "/var/lib/bluetooth" folder and go thru "The
> > Steps" above, the problem will be gone. All of the services will have
> > their characteristics showing up in nRF. The data can be read from the
> > characteristics. This means the folder "/var/lib/bluetooth" is related
> > to the issue. But I don't know why.
> >
> > Here is also another thing. I subscribed to the "PropertiesChanged"
> > signal. In the callback function "properties_changed", I print out the
> > "changed". Here is the code to subscribe the "PropertiesChanged":
> >
> > bus.add_signal_receiver(
> > properties_changed,
> > dbus_interface="org.freedesktop.DBus.Properties",
> > signal_name="PropertiesChanged",
> > arg0="org.bluez.Device1",
> > path_keyword="path",
> > )
> >
> > If the folder "/var/lib/bluetooth" is removed and go thru "The Steps",
> > I saw the log like this:
> > dbus.Dictionary({dbus.String('UUIDs'):
> > dbus.Array([dbus.String('00000000-deca-fade-deca-deafdecacafe'),
> > dbus.String('00001000-0000-1000-8000-00805f9b34fb'),
> > dbus.String('0000110a-0000-1000-8000-00805f9b34fb'),
> > dbus.String('0000110c-0000-1000-8000-00805f9b34fb'),
> > dbus.String('0000110e-0000-1000-8000-00805f9b34fb'),
> > dbus.String('00001116-0000-1000-8000-00805f9b34fb'),
> > dbus.String('0000111f-0000-1000-8000-00805f9b34fb'),
> > dbus.String('0000112f-0000-1000-8000-00805f9b34fb'),
> > dbus.String('00001132-0000-1000-8000-00805f9b34fb'),
> > dbus.String('00001200-0000-1000-8000-00805f9b34fb'),
> > dbus.String('00001800-0000-1000-8000-00805f9b34fb'),
> > dbus.String('00001801-0000-1000-8000-00805f9b34fb'),
> > dbus.String('00001805-0000-1000-8000-00805f9b34fb'),
> > dbus.String('0000180a-0000-1000-8000-00805f9b34fb'),
> > dbus.String('0000180f-0000-1000-8000-00805f9b34fb'),
> > dbus.String('02030302-1d19-415f-86f2-22a2106a0a77'),
> > dbus.String('7905f431-b5ce-4e99-a40f-4b1e122d00d0'),
> > dbus.String('89d3502b-0f36-433a-8ef4-c502ad55f8dc'),
> > dbus.String('9fa480e0-4967-4542-9390-d343dc5d04ae'),
> > dbus.String('d0611e78-bbb4-4591-a5f8-487910ae4366')],
> > signature=dbus.Signature('s'), variant_level=1),
> > dbus.String('ServicesResolved'): dbus.Boolean(True, variant_level=1)},
> > signature=dbus.Signature('sv'))
> >
> > If I check the log after the firmware is upgraded without deleting the
> > "/var/lib/bluetooth" folder, the log looks like this:
> > dbus.Dictionary({dbus.String('ServicesResolved'): dbus.Boolean(True,
> > variant_level=1)}, signature=dbus.Signature('sv'))
> >
> > As we can see, they are different. Is it something to do with D-Bus?
> >
> > We understand that if we don't carry over the "/var/lib/bluetooth"
> > folder, then there is no this issue. But our end user will have to
> > choose "Forget This Device" each time after the firmware is upgraded.
> > Otherwise, the end user can't connect to the Linux device because the
> > pairing info can't be found. Is there a way to fix this issue? Namely,
> > is there a way to keep the pairing info and also avoid this issue?
>
> This sounds like ServiceChanged is not being properly propagated when
> you do the firmware update, do you have ServiceChanged entry on the
> storage:
>
> https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/settings-storage.txt#n321
>
> Is there any indication on btmon when you first connect after doing
> the firmware update?
>
> --
> Luiz Augusto von Dentz


Attachments:
btmon_unavailable_chars.log (184.00 kB)

2021-03-18 16:33:47

by Luiz Augusto von Dentz

[permalink] [raw]
Subject: Re: Have to delete "/var/lib/bluetooth" folder in order to get all service characteristics available

Hi Kenny,

On Wed, Mar 17, 2021 at 10:17 PM Kenny Bian <[email protected]> wrote:
>
> Thank you so much for your reply, Luiz.
>
> We have a small device which runs a Linux system in it. In this
> embedded Linux system, there is GATT server runs BLE service which
> provides the services with the characteristics for reading and writing
> from the mobile device such as iPhone. The folder "/var/lib/bluetooth"
> is in the embedded Linux system. We call this embedded Linux as BLE
> Server, and the iPhone as the BLE Client. During the testing a few
> weeks ago, not only our mobile app crashed, but the nRF also crashed
> as the service was changed. Since we have no way to tell whether the
> ServiceChanged indication works or not. We disabled the bluetooth
> cache. Then we have this new issue.
>
> I never saw the "[ServiceChanged]" section from the "info" file. I
> tried to change the attributes of some characteristics. For example,
> adding notification in some characteristics. Then restart the BlE
> service from the Linux system. I did check the "info" file after that,
> but I didn't see the "[ServiceChanged]" section. Is there another way
> to have the "[ServiceChanged]" showing up?
>
> Today, I ran the testing again by doing the firmware upgrade. Right
> after the firmware upgrade, I used the btmon to capture the
> characteristics reading by using our mobile app. Please see the
> attached log. As you can see, there are a lot of errors because there
> are 5 services which only have the service itself listed(I checked it
> by using nRF). For about 30 characteristics under those services can't
> be seen. That's why there are errors when the mobile app tries to read
> the data. I double checked, the BLE python files between these two
> versions of the firmware are exactly the same. So there are no
> ServiceChanged. As the reading didn't work, I had to remove the
> "/var/lib/bluetooth" folder. Then this folder was recreated again. I
> compared the file "attributes" and "info" before and after removal of
> the "/var/lib/bluetooth" folder. The "attributes" files are exactly
> the same. The "info" files are different. But it makes sense because
> only the "[SlaveLongTermKey]" and "[LinkKey]" are different which is
> caused by re-pairing from the mobile side, I guess.
>
> So in this case, the BLE service python code is exactly the same. The
> "attributes" and "info" files are also the same or slightly different
> because of the re-pairing. How come some characteristics cannot be
> seen from nRF? As those characteristics are not listed in nRF, neither
> nRF or our mobile app can read the data.
>
> Honestly, this situation is pretty much the same as there is no
> firmware upgrade because the service file itself and those files under
> "/var/lib/bluetooth" are almost the same. How come the normal reading
> from the mobile app works fine? But after the firmware upgrade, some
> characteristics can't be seen? Is there anything wrong about
> advertisement? In fact, the firmware upgrade only copy over the
> "/var/lib/bluetooth" in order to keep the pairing information. I have
> no idea something went wrong. Are there anything else to look into?

< ACL Data TX: Handle 64 flags 0x00 dlen 186
ATT: Read By Group Type Response (0x11) len 181
Attribute data length: 20
Attribute group list: 9 entries
Handle range: 0x000a-0x0013
UUID: Vendor specific (efb39360-a7e3-438f-a20d-e9f00e0e22b1)
Handle range: 0x0014-0x004c
UUID: Vendor specific (f57793c9-9544-46dc-bfa0-5fd149953c86)
Handle range: 0x004d-0x0061
UUID: Vendor specific (5040556b-340f-4c6f-b411-448089694628)
Handle range: 0x0062-0x00ac
UUID: Vendor specific (e05ad2ac-9a01-45f5-a56d-9c3c889d4dc6)
Handle range: 0x00ad-0x00ca
UUID: Vendor specific (51e16ff1-20d3-45ec-915c-f18290a893c5)
Handle range: 0x00cb-0x00fc
UUID: Vendor specific (e03d645c-3f2b-4693-a2fb-99840ee2581d)
Handle range: 0x00fd-0x0122
UUID: Vendor specific (5526a99e-7975-42ff-a27b-94d5a1ad9986)
Handle range: 0x0123-0x0144
UUID: Vendor specific (d72ce428-ba8c-4061-b6de-6f682736fe08)
Handle range: 0x0145-0x0151
UUID: Vendor specific (76a136d4-29fc-4217-b358-9bff4d6601ce

So if I got you right you are saying that characteristics under the
last 2 services are not listed? But they appeared on the logs:

> ACL Data RX: Handle 64 flags 0x02 dlen 11
ATT: Read By Type Request (0x08) len 6
Handle range: 0x0123-0x0144
Attribute type: Characteristic (0x2803)
< ACL Data TX: Handle 64 flags 0x00 dlen 174
ATT: Read By Type Response (0x09) len 169
Attribute data length: 21
Attribute data list: 8 entries
Handle: 0x0124
Value: 8225012a7609433923a6925d42d5d4bf9307c4
Handle: 0x0127
Value: 82280181c6d99e78e5409a26411b1570c9c420
Handle: 0x012a
Value: 822b01d56926acbb8915bcad433323d6d170f4
Handle: 0x012d
Value: 822e01ba148cb7ef078a8406420e179ac55080
Handle: 0x0130
Value: 8231017715d4d9cd34078f004286be53568f03
Handle: 0x0133
Value: 8234012d4ead967eb781bb25442973c6247972
Handle: 0x0136
Value: 8237019997e76edb16c4b0b347be4445ee259c
Handle: 0x0139
Value: 823a01cdef05a932746f85b44aa6b0b1f07b6
> ACL Data RX: Handle 64 flags 0x02 dlen 11
ATT: Read By Type Request (0x08) len 6
Handle range: 0x013b-0x0144
Attribute type: Characteristic (0x2803)
< ACL Data TX: Handle 64 flags 0x00 dlen 69
ATT: Read By Type Response (0x09) len 64
Attribute data length: 21
Attribute data list: 3 entries
Handle: 0x013c
Value: 823d01f1406c52df184a929e44b967b5d5bba3
Handle: 0x013f
Value: 884001e85f5356a5e92a9a794cb5c0c68a90bd
Handle: 0x0142
Value: 8a430168db5fe9dcd35c8dec46efcb334ffa8

So this informed the remote that there are a total of 11 (8+3)
characteristics in the range of 0x0123-0x0144.

> ACL Data RX: Handle 64 flags 0x02 dlen 11
ATT: Read By Type Request (0x08) len 6
Handle range: 0x0145-0x0151
Attribute type: Characteristic (0x2803)
< ACL Data TX: Handle 64 flags 0x00 dlen 90
ATT: Read By Type Response (0x09) len 85
Attribute data length: 21
Attribute data list: 4 entries
Handle: 0x0146
Value: 884701f09f9394856720a26047025b9ef90752
Handle: 0x0149
Value: 884a013fd59d9e09e280b95d4ef082b6251241
Handle: 0x014c
Value: 8a4d01f2b9f5d7d417f1bfb14fe66a4c9a476c
Handle: 0x014f
Value: 885001da299c6a9df1309f43421ece432e9233

And there are 4 characteristics in the range of 0x0145-0x0151. In fact
all the services seem to contain characteristics so if the remote
cannot list them then the problem is on their side, also there are
writes on the handles 0x04xx where there are no services whatsoever,
so this tells me that the remote has a invalid cache even after
performing a discovery and found out there are no services in the
range 0x0152-0xffff:

> ACL Data RX: Handle 64 flags 0x02 dlen 11
ATT: Read By Group Type Request (0x10) len 6
Handle range: 0x0152-0xffff
Attribute group type: Primary Service (0x2800)
< ACL Data TX: Handle 64 flags 0x00 dlen 9
ATT: Error Response (0x01) len 4
Read By Group Type Request (0x10)
Handle: 0x0152
Error: Attribute Not Found (0x0a)

2021-03-19 06:10:16

by Kenny Bian

[permalink] [raw]
Subject: Re: Have to delete "/var/lib/bluetooth" folder in order to get all service characteristics available

Thank you so much, Luiz. I appreciate it.

Yes, you're right. I counted the characteristics from the log. There
are exactly 9 services which include 92 characteristics. Do you think
it is "too many" by having 92 characteristics? Do you know if there is
any limit in BlueZ? It seems someone mentioned the number of
characteristics in this link:
https://devzone.nordicsemi.com/f/nordic-q-a/16233/nrf-connect-fails-when-using-many-characteristics

For the write request for the non-exist handle, I've asked our mobile
developer to check it on the mobile side.

Thanks again!

On Thu, Mar 18, 2021 at 9:30 AM Luiz Augusto von Dentz
<[email protected]> wrote:
>
> Hi Kenny,
>
> On Wed, Mar 17, 2021 at 10:17 PM Kenny Bian <[email protected]> wrote:
> >
> > Thank you so much for your reply, Luiz.
> >
> > We have a small device which runs a Linux system in it. In this
> > embedded Linux system, there is GATT server runs BLE service which
> > provides the services with the characteristics for reading and writing
> > from the mobile device such as iPhone. The folder "/var/lib/bluetooth"
> > is in the embedded Linux system. We call this embedded Linux as BLE
> > Server, and the iPhone as the BLE Client. During the testing a few
> > weeks ago, not only our mobile app crashed, but the nRF also crashed
> > as the service was changed. Since we have no way to tell whether the
> > ServiceChanged indication works or not. We disabled the bluetooth
> > cache. Then we have this new issue.
> >
> > I never saw the "[ServiceChanged]" section from the "info" file. I
> > tried to change the attributes of some characteristics. For example,
> > adding notification in some characteristics. Then restart the BlE
> > service from the Linux system. I did check the "info" file after that,
> > but I didn't see the "[ServiceChanged]" section. Is there another way
> > to have the "[ServiceChanged]" showing up?
> >
> > Today, I ran the testing again by doing the firmware upgrade. Right
> > after the firmware upgrade, I used the btmon to capture the
> > characteristics reading by using our mobile app. Please see the
> > attached log. As you can see, there are a lot of errors because there
> > are 5 services which only have the service itself listed(I checked it
> > by using nRF). For about 30 characteristics under those services can't
> > be seen. That's why there are errors when the mobile app tries to read
> > the data. I double checked, the BLE python files between these two
> > versions of the firmware are exactly the same. So there are no
> > ServiceChanged. As the reading didn't work, I had to remove the
> > "/var/lib/bluetooth" folder. Then this folder was recreated again. I
> > compared the file "attributes" and "info" before and after removal of
> > the "/var/lib/bluetooth" folder. The "attributes" files are exactly
> > the same. The "info" files are different. But it makes sense because
> > only the "[SlaveLongTermKey]" and "[LinkKey]" are different which is
> > caused by re-pairing from the mobile side, I guess.
> >
> > So in this case, the BLE service python code is exactly the same. The
> > "attributes" and "info" files are also the same or slightly different
> > because of the re-pairing. How come some characteristics cannot be
> > seen from nRF? As those characteristics are not listed in nRF, neither
> > nRF or our mobile app can read the data.
> >
> > Honestly, this situation is pretty much the same as there is no
> > firmware upgrade because the service file itself and those files under
> > "/var/lib/bluetooth" are almost the same. How come the normal reading
> > from the mobile app works fine? But after the firmware upgrade, some
> > characteristics can't be seen? Is there anything wrong about
> > advertisement? In fact, the firmware upgrade only copy over the
> > "/var/lib/bluetooth" in order to keep the pairing information. I have
> > no idea something went wrong. Are there anything else to look into?
>
> < ACL Data TX: Handle 64 flags 0x00 dlen 186
> ATT: Read By Group Type Response (0x11) len 181
> Attribute data length: 20
> Attribute group list: 9 entries
> Handle range: 0x000a-0x0013
> UUID: Vendor specific (efb39360-a7e3-438f-a20d-e9f00e0e22b1)
> Handle range: 0x0014-0x004c
> UUID: Vendor specific (f57793c9-9544-46dc-bfa0-5fd149953c86)
> Handle range: 0x004d-0x0061
> UUID: Vendor specific (5040556b-340f-4c6f-b411-448089694628)
> Handle range: 0x0062-0x00ac
> UUID: Vendor specific (e05ad2ac-9a01-45f5-a56d-9c3c889d4dc6)
> Handle range: 0x00ad-0x00ca
> UUID: Vendor specific (51e16ff1-20d3-45ec-915c-f18290a893c5)
> Handle range: 0x00cb-0x00fc
> UUID: Vendor specific (e03d645c-3f2b-4693-a2fb-99840ee2581d)
> Handle range: 0x00fd-0x0122
> UUID: Vendor specific (5526a99e-7975-42ff-a27b-94d5a1ad9986)
> Handle range: 0x0123-0x0144
> UUID: Vendor specific (d72ce428-ba8c-4061-b6de-6f682736fe08)
> Handle range: 0x0145-0x0151
> UUID: Vendor specific (76a136d4-29fc-4217-b358-9bff4d6601ce
>
> So if I got you right you are saying that characteristics under the
> last 2 services are not listed? But they appeared on the logs:
>
> > ACL Data RX: Handle 64 flags 0x02 dlen 11
> ATT: Read By Type Request (0x08) len 6
> Handle range: 0x0123-0x0144
> Attribute type: Characteristic (0x2803)
> < ACL Data TX: Handle 64 flags 0x00 dlen 174
> ATT: Read By Type Response (0x09) len 169
> Attribute data length: 21
> Attribute data list: 8 entries
> Handle: 0x0124
> Value: 8225012a7609433923a6925d42d5d4bf9307c4
> Handle: 0x0127
> Value: 82280181c6d99e78e5409a26411b1570c9c420
> Handle: 0x012a
> Value: 822b01d56926acbb8915bcad433323d6d170f4
> Handle: 0x012d
> Value: 822e01ba148cb7ef078a8406420e179ac55080
> Handle: 0x0130
> Value: 8231017715d4d9cd34078f004286be53568f03
> Handle: 0x0133
> Value: 8234012d4ead967eb781bb25442973c6247972
> Handle: 0x0136
> Value: 8237019997e76edb16c4b0b347be4445ee259c
> Handle: 0x0139
> Value: 823a01cdef05a932746f85b44aa6b0b1f07b6
> > ACL Data RX: Handle 64 flags 0x02 dlen 11
> ATT: Read By Type Request (0x08) len 6
> Handle range: 0x013b-0x0144
> Attribute type: Characteristic (0x2803)
> < ACL Data TX: Handle 64 flags 0x00 dlen 69
> ATT: Read By Type Response (0x09) len 64
> Attribute data length: 21
> Attribute data list: 3 entries
> Handle: 0x013c
> Value: 823d01f1406c52df184a929e44b967b5d5bba3
> Handle: 0x013f
> Value: 884001e85f5356a5e92a9a794cb5c0c68a90bd
> Handle: 0x0142
> Value: 8a430168db5fe9dcd35c8dec46efcb334ffa8
>
> So this informed the remote that there are a total of 11 (8+3)
> characteristics in the range of 0x0123-0x0144.
>
> > ACL Data RX: Handle 64 flags 0x02 dlen 11
> ATT: Read By Type Request (0x08) len 6
> Handle range: 0x0145-0x0151
> Attribute type: Characteristic (0x2803)
> < ACL Data TX: Handle 64 flags 0x00 dlen 90
> ATT: Read By Type Response (0x09) len 85
> Attribute data length: 21
> Attribute data list: 4 entries
> Handle: 0x0146
> Value: 884701f09f9394856720a26047025b9ef90752
> Handle: 0x0149
> Value: 884a013fd59d9e09e280b95d4ef082b6251241
> Handle: 0x014c
> Value: 8a4d01f2b9f5d7d417f1bfb14fe66a4c9a476c
> Handle: 0x014f
> Value: 885001da299c6a9df1309f43421ece432e9233
>
> And there are 4 characteristics in the range of 0x0145-0x0151. In fact
> all the services seem to contain characteristics so if the remote
> cannot list them then the problem is on their side, also there are
> writes on the handles 0x04xx where there are no services whatsoever,
> so this tells me that the remote has a invalid cache even after
> performing a discovery and found out there are no services in the
> range 0x0152-0xffff:
>
> > ACL Data RX: Handle 64 flags 0x02 dlen 11
> ATT: Read By Group Type Request (0x10) len 6
> Handle range: 0x0152-0xffff
> Attribute group type: Primary Service (0x2800)
> < ACL Data TX: Handle 64 flags 0x00 dlen 9
> ATT: Error Response (0x01) len 4
> Read By Group Type Request (0x10)
> Handle: 0x0152
> Error: Attribute Not Found (0x0a)