2016-10-18 12:57:29

by Amitkumar Karwar

[permalink] [raw]
Subject: [PATCH] btusb: fix zero BD address problem during stress test

From: Ganapathi Bhat <[email protected]>

We came across a corner case issue during reboot stress test
in which hciconfig shows BD address is all zero. Reason is we
don't get response for HCI RESET command during initialization

The issue is tracked to a race where USB subsystem calls
btusb_intr_complete() to deliver a data(NOOP frame) received
on interrupt endpoint. HCI_RUNNING flag is not yet set by
bluetooth subsystem. So we ignore that frame and return.

As we missed to resubmit the buffer to interrupt endpoint in
this case, we don't get response for BT reset command downloaded
after this.

This patch handles the corner case to resolve zero BD address
problem.

Signed-off-by: Ganapathi Bhat <[email protected]>
Signed-off-by: Amitkumar Karwar <[email protected]>
---
drivers/bluetooth/btusb.c | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
index 811f9b9..b5596ac 100644
--- a/drivers/bluetooth/btusb.c
+++ b/drivers/bluetooth/btusb.c
@@ -607,10 +607,7 @@ static void btusb_intr_complete(struct urb *urb)
BT_DBG("%s urb %p status %d count %d", hdev->name, urb, urb->status,
urb->actual_length);

- if (!test_bit(HCI_RUNNING, &hdev->flags))
- return;
-
- if (urb->status == 0) {
+ if (urb->status == 0 && test_bit(HCI_RUNNING, &hdev->flags)) {
hdev->stat.byte_rx += urb->actual_length;

if (btusb_recv_intr(data, urb->transfer_buffer,
--
1.9.1


2016-11-25 11:21:19

by Amitkumar Karwar

[permalink] [raw]
Subject: RE: [PATCH] btusb: fix zero BD address problem during stress test

SGkgTWFyY2VsLA0KDQo+IEZyb206IEFtaXRrdW1hciBLYXJ3YXINCj4gU2VudDogV2VkbmVzZGF5
LCBOb3ZlbWJlciAyMywgMjAxNiAzOjE0IFBNDQo+IFRvOiAnTWFyY2VsIEhvbHRtYW5uJw0KPiBD
YzogbGludXgtYmx1ZXRvb3RoQHZnZXIua2VybmVsLm9yZzsgbGludXgta2VybmVsQHZnZXIua2Vy
bmVsLm9yZzsNCj4gQ2F0aHkgTHVvOyBOaXNoYW50IFNhcm11a2FkYW07IEdhbmFwYXRoaSBCaGF0
DQo+IFN1YmplY3Q6IFJFOiBbUEFUQ0hdIGJ0dXNiOiBmaXggemVybyBCRCBhZGRyZXNzIHByb2Js
ZW0gZHVyaW5nIHN0cmVzcw0KPiB0ZXN0DQo+IA0KPiBIaSBNYXJjZWwsDQo+IA0KPiA+IEZyb206
IE1hcmNlbCBIb2x0bWFubiBbbWFpbHRvOm1hcmNlbEBob2x0bWFubi5vcmddDQo+ID4gU2VudDog
V2VkbmVzZGF5LCBOb3ZlbWJlciAyMywgMjAxNiAxOjQ2IFBNDQo+ID4gVG86IEFtaXRrdW1hciBL
YXJ3YXINCj4gPiBDYzogbGludXgtYmx1ZXRvb3RoQHZnZXIua2VybmVsLm9yZzsgbGludXgta2Vy
bmVsQHZnZXIua2VybmVsLm9yZzsNCj4gPiBDYXRoeSBMdW87IE5pc2hhbnQgU2FybXVrYWRhbTsg
R2FuYXBhdGhpIEJoYXQNCj4gPiBTdWJqZWN0OiBSZTogW1BBVENIXSBidHVzYjogZml4IHplcm8g
QkQgYWRkcmVzcyBwcm9ibGVtIGR1cmluZyBzdHJlc3MNCj4gPiB0ZXN0DQo+ID4NCj4gPiBIaSBB
bWl0a3VtYXIsDQo+ID4NCj4gPiA+Pj4+IEZyb206IEFtaXRrdW1hciBLYXJ3YXIgW21haWx0bzph
a2Fyd2FyQG1hcnZlbGwuY29tXQ0KPiA+ID4+Pj4gU2VudDogVHVlc2RheSwgT2N0b2JlciAxOCwg
MjAxNiA2OjI3IFBNDQo+ID4gPj4+PiBUbzogbGludXgtYmx1ZXRvb3RoQHZnZXIua2VybmVsLm9y
Zw0KPiA+ID4+Pj4gQ2M6IG1hcmNlbEBob2x0bWFubi5vcmc7IGxpbnV4LWtlcm5lbEB2Z2VyLmtl
cm5lbC5vcmc7IENhdGh5DQo+IEx1bzsNCj4gPiA+Pj4+IE5pc2hhbnQgU2FybXVrYWRhbTsgR2Fu
YXBhdGhpIEJoYXQ7IEFtaXRrdW1hciBLYXJ3YXINCj4gPiA+Pj4+IFN1YmplY3Q6IFtQQVRDSF0g
YnR1c2I6IGZpeCB6ZXJvIEJEIGFkZHJlc3MgcHJvYmxlbSBkdXJpbmcNCj4gc3RyZXNzDQo+ID4g
Pj4+PiB0ZXN0DQo+ID4gPj4+Pg0KPiA+ID4+Pj4gRnJvbTogR2FuYXBhdGhpIEJoYXQgPGdiaGF0
QG1hcnZlbGwuY29tPg0KPiA+ID4+Pj4NCj4gPiA+Pj4+IFdlIGNhbWUgYWNyb3NzIGEgY29ybmVy
IGNhc2UgaXNzdWUgZHVyaW5nIHJlYm9vdCBzdHJlc3MgdGVzdCBpbg0KPiA+ID4+IHdoaWNoDQo+
ID4gPj4+PiBoY2ljb25maWcgc2hvd3MgQkQgYWRkcmVzcyBpcyBhbGwgemVyby4gUmVhc29uIGlz
IHdlIGRvbid0IGdldA0KPiA+ID4+Pj4gcmVzcG9uc2UgZm9yIEhDSSBSRVNFVCBjb21tYW5kIGR1
cmluZyBpbml0aWFsaXphdGlvbg0KPiA+ID4+Pj4NCj4gPiA+Pj4+IFRoZSBpc3N1ZSBpcyB0cmFj
a2VkIHRvIGEgcmFjZSB3aGVyZSBVU0Igc3Vic3lzdGVtIGNhbGxzDQo+ID4gPj4+PiBidHVzYl9p
bnRyX2NvbXBsZXRlKCkgdG8gZGVsaXZlciBhIGRhdGEoTk9PUCBmcmFtZSkgcmVjZWl2ZWQgb24N
Cj4gPiA+Pj4+IGludGVycnVwdCBlbmRwb2ludC4gSENJX1JVTk5JTkcgZmxhZyBpcyBub3QgeWV0
IHNldCBieSBibHVldG9vdGgNCj4gPiA+Pj4+IHN1YnN5c3RlbS4gU28gd2UgaWdub3JlIHRoYXQg
ZnJhbWUgYW5kIHJldHVybi4NCj4gPiA+Pj4+DQo+ID4gPj4+PiBBcyB3ZSBtaXNzZWQgdG8gcmVz
dWJtaXQgdGhlIGJ1ZmZlciB0byBpbnRlcnJ1cHQgZW5kcG9pbnQgaW4NCj4gdGhpcw0KPiA+ID4+
Pj4gY2FzZSwgd2UgZG9uJ3QgZ2V0IHJlc3BvbnNlIGZvciBCVCByZXNldCBjb21tYW5kIGRvd25s
b2FkZWQNCj4gYWZ0ZXINCj4gPiA+PiB0aGlzLg0KPiA+ID4+Pj4NCj4gPiA+Pj4+IFRoaXMgcGF0
Y2ggaGFuZGxlcyB0aGUgY29ybmVyIGNhc2UgdG8gcmVzb2x2ZSB6ZXJvIEJEIGFkZHJlc3MNCj4g
PiA+PiBwcm9ibGVtLg0KPiA+ID4+Pj4NCj4gPiA+Pj4+IFNpZ25lZC1vZmYtYnk6IEdhbmFwYXRo
aSBCaGF0IDxnYmhhdEBtYXJ2ZWxsLmNvbT4NCj4gPiA+Pj4+IFNpZ25lZC1vZmYtYnk6IEFtaXRr
dW1hciBLYXJ3YXIgPGFrYXJ3YXJAbWFydmVsbC5jb20+DQo+ID4gPj4+PiAtLS0NCj4gPiA+Pj4+
IGRyaXZlcnMvYmx1ZXRvb3RoL2J0dXNiLmMgfCA1ICstLS0tDQo+ID4gPj4+PiAxIGZpbGUgY2hh
bmdlZCwgMSBpbnNlcnRpb24oKyksIDQgZGVsZXRpb25zKC0pDQo+ID4gPj4+Pg0KPiA+ID4+Pj4g
ZGlmZiAtLWdpdCBhL2RyaXZlcnMvYmx1ZXRvb3RoL2J0dXNiLmMNCj4gPiA+Pj4+IGIvZHJpdmVy
cy9ibHVldG9vdGgvYnR1c2IuYyBpbmRleCA4MTFmOWI5Li5iNTU5NmFjIDEwMDY0NA0KPiA+ID4+
Pj4gLS0tIGEvZHJpdmVycy9ibHVldG9vdGgvYnR1c2IuYw0KPiA+ID4+Pj4gKysrIGIvZHJpdmVy
cy9ibHVldG9vdGgvYnR1c2IuYw0KPiA+ID4+Pj4gQEAgLTYwNywxMCArNjA3LDcgQEAgc3RhdGlj
IHZvaWQgYnR1c2JfaW50cl9jb21wbGV0ZShzdHJ1Y3QgdXJiDQo+ID4gPj4gKnVyYikNCj4gPiA+
Pj4+IAlCVF9EQkcoIiVzIHVyYiAlcCBzdGF0dXMgJWQgY291bnQgJWQiLCBoZGV2LT5uYW1lLCB1
cmIsDQo+IHVyYi0NCj4gPiA+Pj4+PiBzdGF0dXMsDQo+ID4gPj4+PiAJICAgICAgIHVyYi0+YWN0
dWFsX2xlbmd0aCk7DQo+ID4gPj4+Pg0KPiA+ID4+Pj4gLQlpZiAoIXRlc3RfYml0KEhDSV9SVU5O
SU5HLCAmaGRldi0+ZmxhZ3MpKQ0KPiA+ID4+Pj4gLQkJcmV0dXJuOw0KPiA+ID4+Pj4gLQ0KPiA+
ID4+Pj4gLQlpZiAodXJiLT5zdGF0dXMgPT0gMCkgew0KPiA+ID4+Pj4gKwlpZiAodXJiLT5zdGF0
dXMgPT0gMCAmJiB0ZXN0X2JpdChIQ0lfUlVOTklORywgJmhkZXYtDQo+ID4gPmZsYWdzKSkgew0K
PiA+ID4+Pj4gCQloZGV2LT5zdGF0LmJ5dGVfcnggKz0gdXJiLT5hY3R1YWxfbGVuZ3RoOw0KPiA+
ID4+Pj4NCj4gPiA+Pj4+IAkJaWYgKGJ0dXNiX3JlY3ZfaW50cihkYXRhLCB1cmItPnRyYW5zZmVy
X2J1ZmZlciwNCj4gPiA+Pj4NCj4gPiA+Pj4gRGlkIHlvdSBnZXQgYSBjaGFuY2UgdG8gY2hlY2sg
dGhpcz8NCj4gPiA+Pj4gUGxlYXNlIGxldCB1cyBrbm93IGlmIHlvdSBoYXZlIGFueSByZXZpZXcg
Y29tbWVudHMuDQo+ID4gPj4NCj4gPiA+PiBjYW4geW91IGV4cGxhaW4gaG93IHRoaXMgaXMgY29y
cmVjdCBhbmQgc2hvdyBtZSB0aGUgSENJIHRyYWNlcyBmb3INCj4gPiA+PiB0aGlzLg0KPiA+ID4+
DQo+ID4gPg0KPiA+ID4gSSBzdXBwb3NlIEhDSSB0cmFjZSBtZWFucyBoY2lkdW1wIGxvZ3MgaGVy
ZS4gQXMgZGV2aWNlIGhhc24ndCB5ZXQNCj4gPiBpbml0aWFsaXplZCwgaGNpZHVtcCB3b24ndCBz
aG93IGFueXRoaW5nLg0KPiA+ID4gV2UgaGFkIGFkZGVkIGRlYnVnIGluZm8gaW4gYnR1c2IgZHJp
dmVyIHRvIHRyYWNlIHRoZSBkYXRhIHJlY2VpdmVkDQo+ID4gPiBvbg0KPiA+IGFsbCBVU0IgZW5k
cG9pbnRzIGFuZCBhbHNvIGNoZWNrZWQgdXNibW9uIGxvZ3MuDQo+ID4NCj4gPiB1c2UgYnRtb24g
YW5kIGl0IHdpbGwgc2hvdyBpdC4NCj4gPg0KPiA+ID4gSGVyZSBpcyB0aGUgc2VxdWVuY2Ugb2Yg
ZXZlbnRzIHdlIG9ic2VydmVkIGluIGEgY29ybmVyIGNhc2Ugd2hpbGUNCj4gPiBydW5uaW5nIHN0
cmVzcyB0ZXN0Lg0KPiA+ID4gMSkgSW5zaWRlIGJ0dXNiX29wZW4oKSBjYWxsIC0tLS0tLSBUaHJl
YWQgMQ0KPiA+ID4gMikgYnR1c2Jfc3VibWl0X2ludHJfdXJiKCkgc3VibWl0cyB0aGUgVVJCIGZv
ciByZWNlaXZpbmcgZGF0YSBvbg0KPiA+ID4gaW50ZXJydXB0IGVuZHBvaW50IC0tLS0gVGhyZWFk
IDENCj4gPiA+IDMpIGJ0dXNiX2ludHJfY29tcGxldGUoKSBnZXRzIGNhbGxlZCB0byBkZWxpdmVy
IE5PUCBmcmFtZSBmcm9tIEhDSQ0KPiA+ID4gY29udHJvbGxlciAtLS0tIFRocmVhZCAyDQo+ID4g
PiA0KSBIQ0lfUlVOTklORyBpc24ndCBzZXQgeWV0LiBTbyB3ZSByZXR1cm4gZnJvbQ0KPiA+ID4g
YnR1c2JfaW50cl9jb21wbGV0ZSgpIHdpdGhvdXQgcmVzdWJtaXR0aW5nIHRoZSBidWZmZXIgLS0t
IFRocmVhZCAyDQo+ID4gPiA1KSBFeGl0IGJ0dXNiX29wZW4oKSAtLS0tIFRocmVhZCAxDQo+ID4g
PiA2KSAic2V0X2JpdChIQ0lfUlVOTklORywgJmhkZXYtPmZsYWdzKSIgZG9uZSBieSBibHVldG9v
dGggY29yZSAtLS0tDQo+ID4gPiBUaHJlYWQgMQ0KPiA+ID4NCj4gPiA+IExhdGVyIEhDSV9SRVNF
VCBjb21tYW5kIGdldHMgdGltZWRvdXQsIGFzIHdlIGhhdmVuJ3QgcmUtc3VibWl0dGVkDQo+ID4g
YnVmZmVyIGZvciBpbnRlcnJ1cHQgZW5kcG9pbnQgaW4gc3RlcCAoNCkgYWJvdmUuDQo+ID4gPg0K
PiA+ID4gUGxlYXNlIGZpbmQgYXR0YWNoZWQgbG9ncy4gdXNibW9uIGxvZyBzaG93cyBmaXJzdCBm
cmFtZSByZWNlaXZlZCBvbg0KPiA+IGludGVycnVwdCBlbmRwb2ludCBpcyBOT1AoU2VhcmNoIGZv
ciBNYXJ2ZWxsIGluIGxvZykuDQo+ID4gPg0KPiA+ID4gSGVyZSBpcyB3aGF0IGJsdWV0b290aCBz
cGVjIHNheXMgYWJvdXQgTk9QIGZyYW1lLg0KPiA+ID4NCj4gPiA+ICJUbyBpbmRpY2F0ZSB0byB0
aGUgSG9zdCB0aGF0IHRoZSBDb250cm9sbGVyIGlzIHJlYWR5IHRvIHJlY2VpdmUNCj4gSENJDQo+
ID4gPiBjb21tYW5kIHBhY2tldHMsIHRoZSBDb250cm9sbGVyIGdlbmVyYXRlcyBhIENvbW1hbmQg
U3RhdHVzIGV2ZW50DQo+ID4gPiB3aXRoIFN0YXR1cyAweDAwIGFuZCBDb21tYW5kX09wY29kZSAw
eDAwMDAsIGFuZCB0aGUNCj4gPiBOdW1fSENJX0NvbW1hbmRfUGFja2V0cw0KPiA+ID4gZXZlbnQg
cGFyYW1ldGVyIGlzIHNldCB0byAxIG9yIG1vcmUuIENvbW1hbmRfT3Bjb2RlLCAweDAwMDAgaXMg
YQ0KPiBOT1ANCj4gPiAoTm8gT3BlcmF0aW9uKeKAnQ0KPiA+DQo+ID4gU28gSSB3b25kZXIgaWYg
d2UgbmVlZCB0byByZW1vdmUgdGhlIEhDSV9SVU5OSU5HIGxvZ2ljIGZyb20gdGhlDQo+ID4gZHJp
dmVycy4gSXQgaXMgb25seSBsZWZ0IGluIGEgZmV3IFVTQiBkcml2ZXJzIGFuZCBJIHJlbW92ZWQg
dGhlIHJlc3QNCj4gPiBhbmQgbW92ZWQgaXQgaW50byB0aGUgY29yZS4gSSBhbSBub3QgaW4gZmF2
b3VyIG9mIHBhcGVyaW5nIG92ZXIgdGhpcw0KPiA+IGlzc3VlLiBJIG5lZWQgdG8gdW5kZXJzdGFu
ZCB3aGF0IGlzIHdyb25nLiBBbmQgYWN0dWFsbHkgSENJX1JVTk5JTkcNCj4gPiBuZWVkcyB0byBi
ZSB0YWtlbiBhd2F5IGZyb20gdGhlIGRyaXZlcnMuIFNvIHRoZSBxdWVzdGlvbiBpcyBpZg0KPiBi
dHVzYi5jDQo+ID4gc3RpbGwgbmVlZHMgaXQgb3IgaWYgdGhhdCBpcyBqdXN0IGEgbGVmdG92ZXIu
IE1lYW5pbmcgaXMgaXQNCj4gcHJvdGVjdGluZw0KPiA+IGFueXRoaW5nPw0KPiA+DQo+IA0KPiBJ
IGNhbiBzZWUgSENJX1JVTk5JTkcgY2hlY2sgYmVpbmcgdXNlZCBhdCBiZWxvdyBwbGFjZXMgaW4g
YnR1c2IuYw0KPiANCj4gMSkgYnR1c2JfaW50cl9jb21wbGV0ZQ0KPiAyKSBidHVzYl9idWxrX2Nv
bXBsZXRlDQo+IDMpIGJ0dXNiX2lzb2NfY29tcGxldGUNCj4gNCkgYnR1c2JfdHhfY29tcGxldGUN
Cj4gNSkgYnR1c2JfaXNvY190eF9jb21wbGV0ZQ0KPiA2KSBidHVzYl9iY21fc2V0X2RpYWcNCj4g
NykgYnR1c2JfcmVzdW1lDQo+IA0KPiBJIHRoaW5rLCBpdCBjYW4gYmUgcmVtb3ZlZCBmcm9tICgx
KSwgKDIpIGFuZCAoMyksIGFzIHdlIGFscmVhZHkgaGF2ZQ0KPiBiZWxvdyBjaGVjayBpbnNpZGUg
aGNpX3JlY3ZfZnJhbWUoKQ0KPiANCj4gICAgICAgICBpZiAoIWhkZXYgfHwgKCF0ZXN0X2JpdChI
Q0lfVVAsICZoZGV2LT5mbGFncykNCj4gICAgICAgICAgICAgICAgICAgICAgICYmICF0ZXN0X2Jp
dChIQ0lfSU5JVCwgJmhkZXYtPmZsYWdzKSkpIHsNCj4gICAgICAgICAgICAgICAgIGtmcmVlX3Nr
Yihza2IpOw0KPiAgICAgICAgICAgICAgICAgcmV0dXJuIC1FTlhJTzsNCj4gICAgICAgICB9DQo+
IA0KPiBGb3IgKDQpIGFuZCAoNSksIGl0J3MganVzdCB1c2VkIHRvIGRlY2lkZSBpZiB3ZSBzaG91
bGQgdXBkYXRlICJoZGV2LQ0KPiA+c3RhdCIgb3Igbm90Lg0KPiANCg0KUGxlYXNlIGd1aWRlIG9u
IGhvdyBzaG91bGQgd2UgZml4IHRoaXMgaXNzdWUuIFdlIG9ic2VydmUgdGltZW91dCBmb3IgSENJ
X1JFU0VUIGNvbW1hbmQgc29tZXRpbWVzIHdoZW4gc3lzdGVtIGlzIGJvb3RlZC4gVGhpcyBoYXBw
ZW5zIGR1ZSB0byBtaXNzaW5nIHVyYl9zdWJtaXR0KCkgd2hlbiBmaXJzdCBmcmFtZShOT1ApIHJl
Y2VpdmVkIG9uIElOVCBlbmRwb2ludCBpcyBpZ25vcmVkLiBCZWxvdyBjaGFuZ2Ugd291bGQgYWxz
byBoZWxwLg0KDQotLS0gYS9uZXQvYmx1ZXRvb3RoL2hjaV9jb3JlLmMNCisrKyBiL25ldC9ibHVl
dG9vdGgvaGNpX2NvcmUuYw0KQEAgLTEzMTgsMTIgKzEzMTgsMTMgQEAgc3RhdGljIGludCBoY2lf
ZGV2X2RvX29wZW4oc3RydWN0IGhjaV9kZXYgKmhkZXYpDQogICAgICAgICAgICAgICAgZ290byBk
b25lOw0KICAgICAgICB9DQoNCisgICAgICAgc2V0X2JpdChIQ0lfUlVOTklORywgJmhkZXYtPmZs
YWdzKTsNCiAgICAgICAgaWYgKGhkZXYtPm9wZW4oaGRldikpIHsNCisgICAgICAgICAgICAgICBj
bGVhcl9iaXQoSENJX1JVTk5JTkcsICZoZGV2LT5mbGFncyk7DQogICAgICAgICAgICAgICAgcmV0
ID0gLUVJTzsNCiAgICAgICAgICAgICAgICBnb3RvIGRvbmU7DQogICAgICAgIH0NCg0KLSAgICAg
ICBzZXRfYml0KEhDSV9SVU5OSU5HLCAmaGRldi0+ZmxhZ3MpOw0KICAgICAgICBoY2lfc29ja19k
ZXZfZXZlbnQoaGRldiwgSENJX0RFVl9PUEVOKTsNCg0KUmVnYXJkcywNCkFtaXRrdW1hcg0K

2016-11-23 09:44:15

by Amitkumar Karwar

[permalink] [raw]
Subject: RE: [PATCH] btusb: fix zero BD address problem during stress test

SGkgTWFyY2VsLA0KDQo+IEZyb206IE1hcmNlbCBIb2x0bWFubiBbbWFpbHRvOm1hcmNlbEBob2x0
bWFubi5vcmddDQo+IFNlbnQ6IFdlZG5lc2RheSwgTm92ZW1iZXIgMjMsIDIwMTYgMTo0NiBQTQ0K
PiBUbzogQW1pdGt1bWFyIEthcndhcg0KPiBDYzogbGludXgtYmx1ZXRvb3RoQHZnZXIua2VybmVs
Lm9yZzsgbGludXgta2VybmVsQHZnZXIua2VybmVsLm9yZzsNCj4gQ2F0aHkgTHVvOyBOaXNoYW50
IFNhcm11a2FkYW07IEdhbmFwYXRoaSBCaGF0DQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0hdIGJ0dXNi
OiBmaXggemVybyBCRCBhZGRyZXNzIHByb2JsZW0gZHVyaW5nIHN0cmVzcw0KPiB0ZXN0DQo+IA0K
PiBIaSBBbWl0a3VtYXIsDQo+IA0KPiA+Pj4+IEZyb206IEFtaXRrdW1hciBLYXJ3YXIgW21haWx0
bzpha2Fyd2FyQG1hcnZlbGwuY29tXQ0KPiA+Pj4+IFNlbnQ6IFR1ZXNkYXksIE9jdG9iZXIgMTgs
IDIwMTYgNjoyNyBQTQ0KPiA+Pj4+IFRvOiBsaW51eC1ibHVldG9vdGhAdmdlci5rZXJuZWwub3Jn
DQo+ID4+Pj4gQ2M6IG1hcmNlbEBob2x0bWFubi5vcmc7IGxpbnV4LWtlcm5lbEB2Z2VyLmtlcm5l
bC5vcmc7IENhdGh5IEx1bzsNCj4gPj4+PiBOaXNoYW50IFNhcm11a2FkYW07IEdhbmFwYXRoaSBC
aGF0OyBBbWl0a3VtYXIgS2Fyd2FyDQo+ID4+Pj4gU3ViamVjdDogW1BBVENIXSBidHVzYjogZml4
IHplcm8gQkQgYWRkcmVzcyBwcm9ibGVtIGR1cmluZyBzdHJlc3MNCj4gPj4+PiB0ZXN0DQo+ID4+
Pj4NCj4gPj4+PiBGcm9tOiBHYW5hcGF0aGkgQmhhdCA8Z2JoYXRAbWFydmVsbC5jb20+DQo+ID4+
Pj4NCj4gPj4+PiBXZSBjYW1lIGFjcm9zcyBhIGNvcm5lciBjYXNlIGlzc3VlIGR1cmluZyByZWJv
b3Qgc3RyZXNzIHRlc3QgaW4NCj4gPj4gd2hpY2gNCj4gPj4+PiBoY2ljb25maWcgc2hvd3MgQkQg
YWRkcmVzcyBpcyBhbGwgemVyby4gUmVhc29uIGlzIHdlIGRvbid0IGdldA0KPiA+Pj4+IHJlc3Bv
bnNlIGZvciBIQ0kgUkVTRVQgY29tbWFuZCBkdXJpbmcgaW5pdGlhbGl6YXRpb24NCj4gPj4+Pg0K
PiA+Pj4+IFRoZSBpc3N1ZSBpcyB0cmFja2VkIHRvIGEgcmFjZSB3aGVyZSBVU0Igc3Vic3lzdGVt
IGNhbGxzDQo+ID4+Pj4gYnR1c2JfaW50cl9jb21wbGV0ZSgpIHRvIGRlbGl2ZXIgYSBkYXRhKE5P
T1AgZnJhbWUpIHJlY2VpdmVkIG9uDQo+ID4+Pj4gaW50ZXJydXB0IGVuZHBvaW50LiBIQ0lfUlVO
TklORyBmbGFnIGlzIG5vdCB5ZXQgc2V0IGJ5IGJsdWV0b290aA0KPiA+Pj4+IHN1YnN5c3RlbS4g
U28gd2UgaWdub3JlIHRoYXQgZnJhbWUgYW5kIHJldHVybi4NCj4gPj4+Pg0KPiA+Pj4+IEFzIHdl
IG1pc3NlZCB0byByZXN1Ym1pdCB0aGUgYnVmZmVyIHRvIGludGVycnVwdCBlbmRwb2ludCBpbiB0
aGlzDQo+ID4+Pj4gY2FzZSwgd2UgZG9uJ3QgZ2V0IHJlc3BvbnNlIGZvciBCVCByZXNldCBjb21t
YW5kIGRvd25sb2FkZWQgYWZ0ZXINCj4gPj4gdGhpcy4NCj4gPj4+Pg0KPiA+Pj4+IFRoaXMgcGF0
Y2ggaGFuZGxlcyB0aGUgY29ybmVyIGNhc2UgdG8gcmVzb2x2ZSB6ZXJvIEJEIGFkZHJlc3MNCj4g
Pj4gcHJvYmxlbS4NCj4gPj4+Pg0KPiA+Pj4+IFNpZ25lZC1vZmYtYnk6IEdhbmFwYXRoaSBCaGF0
IDxnYmhhdEBtYXJ2ZWxsLmNvbT4NCj4gPj4+PiBTaWduZWQtb2ZmLWJ5OiBBbWl0a3VtYXIgS2Fy
d2FyIDxha2Fyd2FyQG1hcnZlbGwuY29tPg0KPiA+Pj4+IC0tLQ0KPiA+Pj4+IGRyaXZlcnMvYmx1
ZXRvb3RoL2J0dXNiLmMgfCA1ICstLS0tDQo+ID4+Pj4gMSBmaWxlIGNoYW5nZWQsIDEgaW5zZXJ0
aW9uKCspLCA0IGRlbGV0aW9ucygtKQ0KPiA+Pj4+DQo+ID4+Pj4gZGlmZiAtLWdpdCBhL2RyaXZl
cnMvYmx1ZXRvb3RoL2J0dXNiLmMgYi9kcml2ZXJzL2JsdWV0b290aC9idHVzYi5jDQo+ID4+Pj4g
aW5kZXggODExZjliOS4uYjU1OTZhYyAxMDA2NDQNCj4gPj4+PiAtLS0gYS9kcml2ZXJzL2JsdWV0
b290aC9idHVzYi5jDQo+ID4+Pj4gKysrIGIvZHJpdmVycy9ibHVldG9vdGgvYnR1c2IuYw0KPiA+
Pj4+IEBAIC02MDcsMTAgKzYwNyw3IEBAIHN0YXRpYyB2b2lkIGJ0dXNiX2ludHJfY29tcGxldGUo
c3RydWN0IHVyYg0KPiA+PiAqdXJiKQ0KPiA+Pj4+IAlCVF9EQkcoIiVzIHVyYiAlcCBzdGF0dXMg
JWQgY291bnQgJWQiLCBoZGV2LT5uYW1lLCB1cmIsIHVyYi0NCj4gPj4+Pj4gc3RhdHVzLA0KPiA+
Pj4+IAkgICAgICAgdXJiLT5hY3R1YWxfbGVuZ3RoKTsNCj4gPj4+Pg0KPiA+Pj4+IC0JaWYgKCF0
ZXN0X2JpdChIQ0lfUlVOTklORywgJmhkZXYtPmZsYWdzKSkNCj4gPj4+PiAtCQlyZXR1cm47DQo+
ID4+Pj4gLQ0KPiA+Pj4+IC0JaWYgKHVyYi0+c3RhdHVzID09IDApIHsNCj4gPj4+PiArCWlmICh1
cmItPnN0YXR1cyA9PSAwICYmIHRlc3RfYml0KEhDSV9SVU5OSU5HLCAmaGRldi0NCj4gPmZsYWdz
KSkgew0KPiA+Pj4+IAkJaGRldi0+c3RhdC5ieXRlX3J4ICs9IHVyYi0+YWN0dWFsX2xlbmd0aDsN
Cj4gPj4+Pg0KPiA+Pj4+IAkJaWYgKGJ0dXNiX3JlY3ZfaW50cihkYXRhLCB1cmItPnRyYW5zZmVy
X2J1ZmZlciwNCj4gPj4+DQo+ID4+PiBEaWQgeW91IGdldCBhIGNoYW5jZSB0byBjaGVjayB0aGlz
Pw0KPiA+Pj4gUGxlYXNlIGxldCB1cyBrbm93IGlmIHlvdSBoYXZlIGFueSByZXZpZXcgY29tbWVu
dHMuDQo+ID4+DQo+ID4+IGNhbiB5b3UgZXhwbGFpbiBob3cgdGhpcyBpcyBjb3JyZWN0IGFuZCBz
aG93IG1lIHRoZSBIQ0kgdHJhY2VzIGZvcg0KPiA+PiB0aGlzLg0KPiA+Pg0KPiA+DQo+ID4gSSBz
dXBwb3NlIEhDSSB0cmFjZSBtZWFucyBoY2lkdW1wIGxvZ3MgaGVyZS4gQXMgZGV2aWNlIGhhc24n
dCB5ZXQNCj4gaW5pdGlhbGl6ZWQsIGhjaWR1bXAgd29uJ3Qgc2hvdyBhbnl0aGluZy4NCj4gPiBX
ZSBoYWQgYWRkZWQgZGVidWcgaW5mbyBpbiBidHVzYiBkcml2ZXIgdG8gdHJhY2UgdGhlIGRhdGEg
cmVjZWl2ZWQgb24NCj4gYWxsIFVTQiBlbmRwb2ludHMgYW5kIGFsc28gY2hlY2tlZCB1c2Jtb24g
bG9ncy4NCj4gDQo+IHVzZSBidG1vbiBhbmQgaXQgd2lsbCBzaG93IGl0Lg0KPiANCj4gPiBIZXJl
IGlzIHRoZSBzZXF1ZW5jZSBvZiBldmVudHMgd2Ugb2JzZXJ2ZWQgaW4gYSBjb3JuZXIgY2FzZSB3
aGlsZQ0KPiBydW5uaW5nIHN0cmVzcyB0ZXN0Lg0KPiA+IDEpIEluc2lkZSBidHVzYl9vcGVuKCkg
Y2FsbCAtLS0tLS0gVGhyZWFkIDENCj4gPiAyKSBidHVzYl9zdWJtaXRfaW50cl91cmIoKSBzdWJt
aXRzIHRoZSBVUkIgZm9yIHJlY2VpdmluZyBkYXRhIG9uDQo+ID4gaW50ZXJydXB0IGVuZHBvaW50
IC0tLS0gVGhyZWFkIDENCj4gPiAzKSBidHVzYl9pbnRyX2NvbXBsZXRlKCkgZ2V0cyBjYWxsZWQg
dG8gZGVsaXZlciBOT1AgZnJhbWUgZnJvbSBIQ0kNCj4gPiBjb250cm9sbGVyIC0tLS0gVGhyZWFk
IDINCj4gPiA0KSBIQ0lfUlVOTklORyBpc24ndCBzZXQgeWV0LiBTbyB3ZSByZXR1cm4gZnJvbSBi
dHVzYl9pbnRyX2NvbXBsZXRlKCkNCj4gPiB3aXRob3V0IHJlc3VibWl0dGluZyB0aGUgYnVmZmVy
IC0tLSBUaHJlYWQgMg0KPiA+IDUpIEV4aXQgYnR1c2Jfb3BlbigpIC0tLS0gVGhyZWFkIDENCj4g
PiA2KSAic2V0X2JpdChIQ0lfUlVOTklORywgJmhkZXYtPmZsYWdzKSIgZG9uZSBieSBibHVldG9v
dGggY29yZSAtLS0tDQo+ID4gVGhyZWFkIDENCj4gPg0KPiA+IExhdGVyIEhDSV9SRVNFVCBjb21t
YW5kIGdldHMgdGltZWRvdXQsIGFzIHdlIGhhdmVuJ3QgcmUtc3VibWl0dGVkDQo+IGJ1ZmZlciBm
b3IgaW50ZXJydXB0IGVuZHBvaW50IGluIHN0ZXAgKDQpIGFib3ZlLg0KPiA+DQo+ID4gUGxlYXNl
IGZpbmQgYXR0YWNoZWQgbG9ncy4gdXNibW9uIGxvZyBzaG93cyBmaXJzdCBmcmFtZSByZWNlaXZl
ZCBvbg0KPiBpbnRlcnJ1cHQgZW5kcG9pbnQgaXMgTk9QKFNlYXJjaCBmb3IgTWFydmVsbCBpbiBs
b2cpLg0KPiA+DQo+ID4gSGVyZSBpcyB3aGF0IGJsdWV0b290aCBzcGVjIHNheXMgYWJvdXQgTk9Q
IGZyYW1lLg0KPiA+DQo+ID4gIlRvIGluZGljYXRlIHRvIHRoZSBIb3N0IHRoYXQgdGhlIENvbnRy
b2xsZXIgaXMgcmVhZHkgdG8gcmVjZWl2ZSBIQ0kNCj4gPiBjb21tYW5kIHBhY2tldHMsIHRoZSBD
b250cm9sbGVyIGdlbmVyYXRlcyBhIENvbW1hbmQgU3RhdHVzIGV2ZW50IHdpdGgNCj4gPiBTdGF0
dXMgMHgwMCBhbmQgQ29tbWFuZF9PcGNvZGUgMHgwMDAwLCBhbmQgdGhlDQo+IE51bV9IQ0lfQ29t
bWFuZF9QYWNrZXRzDQo+ID4gZXZlbnQgcGFyYW1ldGVyIGlzIHNldCB0byAxIG9yIG1vcmUuIENv
bW1hbmRfT3Bjb2RlLCAweDAwMDAgaXMgYSBOT1ANCj4gKE5vIE9wZXJhdGlvbinigJ0NCj4gDQo+
IFNvIEkgd29uZGVyIGlmIHdlIG5lZWQgdG8gcmVtb3ZlIHRoZSBIQ0lfUlVOTklORyBsb2dpYyBm
cm9tIHRoZQ0KPiBkcml2ZXJzLiBJdCBpcyBvbmx5IGxlZnQgaW4gYSBmZXcgVVNCIGRyaXZlcnMg
YW5kIEkgcmVtb3ZlZCB0aGUgcmVzdA0KPiBhbmQgbW92ZWQgaXQgaW50byB0aGUgY29yZS4gSSBh
bSBub3QgaW4gZmF2b3VyIG9mIHBhcGVyaW5nIG92ZXIgdGhpcw0KPiBpc3N1ZS4gSSBuZWVkIHRv
IHVuZGVyc3RhbmQgd2hhdCBpcyB3cm9uZy4gQW5kIGFjdHVhbGx5IEhDSV9SVU5OSU5HDQo+IG5l
ZWRzIHRvIGJlIHRha2VuIGF3YXkgZnJvbSB0aGUgZHJpdmVycy4gU28gdGhlIHF1ZXN0aW9uIGlz
IGlmIGJ0dXNiLmMNCj4gc3RpbGwgbmVlZHMgaXQgb3IgaWYgdGhhdCBpcyBqdXN0IGEgbGVmdG92
ZXIuIE1lYW5pbmcgaXMgaXQgcHJvdGVjdGluZw0KPiBhbnl0aGluZz8NCj4gDQoNCkkgY2FuIHNl
ZSBIQ0lfUlVOTklORyBjaGVjayBiZWluZyB1c2VkIGF0IGJlbG93IHBsYWNlcyBpbiBidHVzYi5j
DQoNCjEpIGJ0dXNiX2ludHJfY29tcGxldGUNCjIpIGJ0dXNiX2J1bGtfY29tcGxldGUNCjMpIGJ0
dXNiX2lzb2NfY29tcGxldGUNCjQpIGJ0dXNiX3R4X2NvbXBsZXRlDQo1KSBidHVzYl9pc29jX3R4
X2NvbXBsZXRlDQo2KSBidHVzYl9iY21fc2V0X2RpYWcNCjcpIGJ0dXNiX3Jlc3VtZQ0KDQpJIHRo
aW5rLCBpdCBjYW4gYmUgcmVtb3ZlZCBmcm9tICgxKSwgKDIpIGFuZCAoMyksIGFzIHdlIGFscmVh
ZHkgaGF2ZSBiZWxvdyBjaGVjayBpbnNpZGUgaGNpX3JlY3ZfZnJhbWUoKSANCg0KICAgICAgICBp
ZiAoIWhkZXYgfHwgKCF0ZXN0X2JpdChIQ0lfVVAsICZoZGV2LT5mbGFncykNCiAgICAgICAgICAg
ICAgICAgICAgICAmJiAhdGVzdF9iaXQoSENJX0lOSVQsICZoZGV2LT5mbGFncykpKSB7DQogICAg
ICAgICAgICAgICAga2ZyZWVfc2tiKHNrYik7DQogICAgICAgICAgICAgICAgcmV0dXJuIC1FTlhJ
TzsNCiAgICAgICAgfQ0KDQpGb3IgKDQpIGFuZCAoNSksIGl0J3MganVzdCB1c2VkIHRvIGRlY2lk
ZSBpZiB3ZSBzaG91bGQgdXBkYXRlICJoZGV2LT5zdGF0IiBvciBub3QuDQoNClJlZ2FyZHMsDQpB
bWl0a3VtYXINCg==

2016-11-23 08:16:04

by Marcel Holtmann

[permalink] [raw]
Subject: Re: [PATCH] btusb: fix zero BD address problem during stress test

Hi Amitkumar,

>>>> From: Amitkumar Karwar [mailto:[email protected]]
>>>> Sent: Tuesday, October 18, 2016 6:27 PM
>>>> To: [email protected]
>>>> Cc: [email protected]; [email protected]; Cathy Luo;
>>>> Nishant Sarmukadam; Ganapathi Bhat; Amitkumar Karwar
>>>> Subject: [PATCH] btusb: fix zero BD address problem during stress
>>>> test
>>>>
>>>> From: Ganapathi Bhat <[email protected]>
>>>>
>>>> We came across a corner case issue during reboot stress test in
>> which
>>>> hciconfig shows BD address is all zero. Reason is we don't get
>>>> response for HCI RESET command during initialization
>>>>
>>>> The issue is tracked to a race where USB subsystem calls
>>>> btusb_intr_complete() to deliver a data(NOOP frame) received on
>>>> interrupt endpoint. HCI_RUNNING flag is not yet set by bluetooth
>>>> subsystem. So we ignore that frame and return.
>>>>
>>>> As we missed to resubmit the buffer to interrupt endpoint in this
>>>> case, we don't get response for BT reset command downloaded after
>> this.
>>>>
>>>> This patch handles the corner case to resolve zero BD address
>> problem.
>>>>
>>>> Signed-off-by: Ganapathi Bhat <[email protected]>
>>>> Signed-off-by: Amitkumar Karwar <[email protected]>
>>>> ---
>>>> drivers/bluetooth/btusb.c | 5 +----
>>>> 1 file changed, 1 insertion(+), 4 deletions(-)
>>>>
>>>> diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
>>>> index 811f9b9..b5596ac 100644
>>>> --- a/drivers/bluetooth/btusb.c
>>>> +++ b/drivers/bluetooth/btusb.c
>>>> @@ -607,10 +607,7 @@ static void btusb_intr_complete(struct urb
>> *urb)
>>>> BT_DBG("%s urb %p status %d count %d", hdev->name, urb, urb-
>>>>> status,
>>>> urb->actual_length);
>>>>
>>>> - if (!test_bit(HCI_RUNNING, &hdev->flags))
>>>> - return;
>>>> -
>>>> - if (urb->status == 0) {
>>>> + if (urb->status == 0 && test_bit(HCI_RUNNING, &hdev->flags)) {
>>>> hdev->stat.byte_rx += urb->actual_length;
>>>>
>>>> if (btusb_recv_intr(data, urb->transfer_buffer,
>>>
>>> Did you get a chance to check this?
>>> Please let us know if you have any review comments.
>>
>> can you explain how this is correct and show me the HCI traces for
>> this.
>>
>
> I suppose HCI trace means hcidump logs here. As device hasn't yet initialized, hcidump won't show anything.
> We had added debug info in btusb driver to trace the data received on all USB endpoints and also checked usbmon logs.

use btmon and it will show it.

> Here is the sequence of events we observed in a corner case while running stress test.
> 1) Inside btusb_open() call ------ Thread 1
> 2) btusb_submit_intr_urb() submits the URB for receiving data on interrupt endpoint ---- Thread 1
> 3) btusb_intr_complete() gets called to deliver NOP frame from HCI controller ---- Thread 2
> 4) HCI_RUNNING isn't set yet. So we return from btusb_intr_complete() without resubmitting the buffer --- Thread 2
> 5) Exit btusb_open() ---- Thread 1
> 6) "set_bit(HCI_RUNNING, &hdev->flags)" done by bluetooth core ---- Thread 1
>
> Later HCI_RESET command gets timedout, as we haven't re-submitted buffer for interrupt endpoint in step (4) above.
>
> Please find attached logs. usbmon log shows first frame received on interrupt endpoint is NOP(Search for Marvell in log).
>
> Here is what bluetooth spec says about NOP frame.
>
> "To indicate to the Host that the Controller is ready to receive HCI command packets,
> the Controller generates a Command Status event with Status 0x00 and
> Command_Opcode 0x0000, and the Num_HCI_Command_Packets event
> parameter is set to 1 or more. Command_Opcode, 0x0000 is a NOP (No Operation)”

So I wonder if we need to remove the HCI_RUNNING logic from the drivers. It is only left in a few USB drivers and I removed the rest and moved it into the core. I am not in favour of papering over this issue. I need to understand what is wrong. And actually HCI_RUNNING needs to be taken away from the drivers. So the question is if btusb.c still needs it or if that is just a leftover. Meaning is it protecting anything?

Regards

Marcel

2016-11-23 07:50:41

by Amitkumar Karwar

[permalink] [raw]
Subject: RE: [PATCH] btusb: fix zero BD address problem during stress test

Hi Marcel,

> From: Marcel Holtmann [mailto:[email protected]]
> Sent: Wednesday, November 23, 2016 3:25 AM
> To: Amitkumar Karwar
> Cc: [email protected]; [email protected];
> Cathy Luo; Nishant Sarmukadam; Ganapathi Bhat
> Subject: Re: [PATCH] btusb: fix zero BD address problem during stress
> test
>
> Hi Amitkumar,
>
> >> From: Amitkumar Karwar [mailto:[email protected]]
> >> Sent: Tuesday, October 18, 2016 6:27 PM
> >> To: [email protected]
> >> Cc: [email protected]; [email protected]; Cathy Luo;
> >> Nishant Sarmukadam; Ganapathi Bhat; Amitkumar Karwar
> >> Subject: [PATCH] btusb: fix zero BD address problem during stress
> >> test
> >>
> >> From: Ganapathi Bhat <[email protected]>
> >>
> >> We came across a corner case issue during reboot stress test in
> which
> >> hciconfig shows BD address is all zero. Reason is we don't get
> >> response for HCI RESET command during initialization
> >>
> >> The issue is tracked to a race where USB subsystem calls
> >> btusb_intr_complete() to deliver a data(NOOP frame) received on
> >> interrupt endpoint. HCI_RUNNING flag is not yet set by bluetooth
> >> subsystem. So we ignore that frame and return.
> >>
> >> As we missed to resubmit the buffer to interrupt endpoint in this
> >> case, we don't get response for BT reset command downloaded after
> this.
> >>
> >> This patch handles the corner case to resolve zero BD address
> problem.
> >>
> >> Signed-off-by: Ganapathi Bhat <[email protected]>
> >> Signed-off-by: Amitkumar Karwar <[email protected]>
> >> ---
> >> drivers/bluetooth/btusb.c | 5 +----
> >> 1 file changed, 1 insertion(+), 4 deletions(-)
> >>
> >> diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
> >> index 811f9b9..b5596ac 100644
> >> --- a/drivers/bluetooth/btusb.c
> >> +++ b/drivers/bluetooth/btusb.c
> >> @@ -607,10 +607,7 @@ static void btusb_intr_complete(struct urb
> *urb)
> >> BT_DBG("%s urb %p status %d count %d", hdev->name, urb, urb-
> >>> status,
> >> urb->actual_length);
> >>
> >> - if (!test_bit(HCI_RUNNING, &hdev->flags))
> >> - return;
> >> -
> >> - if (urb->status == 0) {
> >> + if (urb->status == 0 && test_bit(HCI_RUNNING, &hdev->flags)) {
> >> hdev->stat.byte_rx += urb->actual_length;
> >>
> >> if (btusb_recv_intr(data, urb->transfer_buffer,
> >
> > Did you get a chance to check this?
> > Please let us know if you have any review comments.
>
> can you explain how this is correct and show me the HCI traces for
> this.
>

I suppose HCI trace means hcidump logs here. As device hasn't yet initialized, hcidump won't show anything.
We had added debug info in btusb driver to trace the data received on all USB endpoints and also checked usbmon logs.

Here is the sequence of events we observed in a corner case while running stress test.
1) Inside btusb_open() call ------ Thread 1
2) btusb_submit_intr_urb() submits the URB for receiving data on interrupt endpoint ---- Thread 1
3) btusb_intr_complete() gets called to deliver NOP frame from HCI controller ---- Thread 2
4) HCI_RUNNING isn't set yet. So we return from btusb_intr_complete() without resubmitting the buffer --- Thread 2
5) Exit btusb_open() ---- Thread 1
6) "set_bit(HCI_RUNNING, &hdev->flags)" done by bluetooth core ---- Thread 1

Later HCI_RESET command gets timedout, as we haven't re-submitted buffer for interrupt endpoint in step (4) above.

Please find attached logs. usbmon log shows first frame received on interrupt endpoint is NOP(Search for Marvell in log).

Here is what bluetooth spec says about NOP frame.

"To indicate to the Host that the Controller is ready to receive HCI command packets,
the Controller generates a Command Status event with Status 0x00 and
Command_Opcode 0x0000, and the Num_HCI_Command_Packets event
parameter is set to 1 or more. Command_Opcode, 0x0000 is a NOP (No Operation)"

Regards,
Amitkumar


Attachments:
bd_addr_000000_case.log (1.55 kB)
bd_addr_000000_case.log
bd_addr_normal_case.log (70.78 kB)
bd_addr_normal_case.log
usbmon.log (102.24 kB)
usbmon.log
Download all attachments

2016-11-22 21:55:10

by Marcel Holtmann

[permalink] [raw]
Subject: Re: [PATCH] btusb: fix zero BD address problem during stress test

Hi Amitkumar,

>> From: Amitkumar Karwar [mailto:[email protected]]
>> Sent: Tuesday, October 18, 2016 6:27 PM
>> To: [email protected]
>> Cc: [email protected]; [email protected]; Cathy Luo;
>> Nishant Sarmukadam; Ganapathi Bhat; Amitkumar Karwar
>> Subject: [PATCH] btusb: fix zero BD address problem during stress test
>>
>> From: Ganapathi Bhat <[email protected]>
>>
>> We came across a corner case issue during reboot stress test in which
>> hciconfig shows BD address is all zero. Reason is we don't get response
>> for HCI RESET command during initialization
>>
>> The issue is tracked to a race where USB subsystem calls
>> btusb_intr_complete() to deliver a data(NOOP frame) received on
>> interrupt endpoint. HCI_RUNNING flag is not yet set by bluetooth
>> subsystem. So we ignore that frame and return.
>>
>> As we missed to resubmit the buffer to interrupt endpoint in this case,
>> we don't get response for BT reset command downloaded after this.
>>
>> This patch handles the corner case to resolve zero BD address problem.
>>
>> Signed-off-by: Ganapathi Bhat <[email protected]>
>> Signed-off-by: Amitkumar Karwar <[email protected]>
>> ---
>> drivers/bluetooth/btusb.c | 5 +----
>> 1 file changed, 1 insertion(+), 4 deletions(-)
>>
>> diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
>> index 811f9b9..b5596ac 100644
>> --- a/drivers/bluetooth/btusb.c
>> +++ b/drivers/bluetooth/btusb.c
>> @@ -607,10 +607,7 @@ static void btusb_intr_complete(struct urb *urb)
>> BT_DBG("%s urb %p status %d count %d", hdev->name, urb, urb-
>>> status,
>> urb->actual_length);
>>
>> - if (!test_bit(HCI_RUNNING, &hdev->flags))
>> - return;
>> -
>> - if (urb->status == 0) {
>> + if (urb->status == 0 && test_bit(HCI_RUNNING, &hdev->flags)) {
>> hdev->stat.byte_rx += urb->actual_length;
>>
>> if (btusb_recv_intr(data, urb->transfer_buffer,
>
> Did you get a chance to check this?
> Please let us know if you have any review comments.

can you explain how this is correct and show me the HCI traces for this.

Regards

Marcel

2016-11-16 06:10:40

by Amitkumar Karwar

[permalink] [raw]
Subject: RE: [PATCH] btusb: fix zero BD address problem during stress test

Hi Marcel,

> From: Amitkumar Karwar [mailto:[email protected]]
> Sent: Tuesday, October 18, 2016 6:27 PM
> To: [email protected]
> Cc: [email protected]; [email protected]; Cathy Luo;
> Nishant Sarmukadam; Ganapathi Bhat; Amitkumar Karwar
> Subject: [PATCH] btusb: fix zero BD address problem during stress test
>=20
> From: Ganapathi Bhat <[email protected]>
>=20
> We came across a corner case issue during reboot stress test in which
> hciconfig shows BD address is all zero. Reason is we don't get response
> for HCI RESET command during initialization
>=20
> The issue is tracked to a race where USB subsystem calls
> btusb_intr_complete() to deliver a data(NOOP frame) received on
> interrupt endpoint. HCI_RUNNING flag is not yet set by bluetooth
> subsystem. So we ignore that frame and return.
>=20
> As we missed to resubmit the buffer to interrupt endpoint in this case,
> we don't get response for BT reset command downloaded after this.
>=20
> This patch handles the corner case to resolve zero BD address problem.
>=20
> Signed-off-by: Ganapathi Bhat <[email protected]>
> Signed-off-by: Amitkumar Karwar <[email protected]>
> ---
> drivers/bluetooth/btusb.c | 5 +----
> 1 file changed, 1 insertion(+), 4 deletions(-)
>=20
> diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
> index 811f9b9..b5596ac 100644
> --- a/drivers/bluetooth/btusb.c
> +++ b/drivers/bluetooth/btusb.c
> @@ -607,10 +607,7 @@ static void btusb_intr_complete(struct urb *urb)
> BT_DBG("%s urb %p status %d count %d", hdev->name, urb, urb-
> >status,
> urb->actual_length);
>=20
> - if (!test_bit(HCI_RUNNING, &hdev->flags))
> - return;
> -
> - if (urb->status =3D=3D 0) {
> + if (urb->status =3D=3D 0 && test_bit(HCI_RUNNING, &hdev->flags)) {
> hdev->stat.byte_rx +=3D urb->actual_length;
>=20
> if (btusb_recv_intr(data, urb->transfer_buffer,

Did you get a chance to check this?
Please let us know if you have any review comments.

Regards,
Amitkumar

2016-12-07 09:07:41

by Marcel Holtmann

[permalink] [raw]
Subject: Re: [PATCH] btusb: fix zero BD address problem during stress test

Hi Amitkumar,

>>>>>>>> From: Amitkumar Karwar [mailto:[email protected]]
>>>>>>>> Sent: Tuesday, October 18, 2016 6:27 PM
>>>>>>>> To: [email protected]
>>>>>>>> Cc: [email protected]; [email protected]; Cathy
>>> Luo;
>>>>>>>> Nishant Sarmukadam; Ganapathi Bhat; Amitkumar Karwar
>>>>>>>> Subject: [PATCH] btusb: fix zero BD address problem during
>>> stress
>>>>>>>> test
>>>>>>>>
>>>>>>>> From: Ganapathi Bhat <[email protected]>
>>>>>>>>
>>>>>>>> We came across a corner case issue during reboot stress test
>> in
>>>>>> which
>>>>>>>> hciconfig shows BD address is all zero. Reason is we don't get
>>>>>>>> response for HCI RESET command during initialization
>>>>>>>>
>>>>>>>> The issue is tracked to a race where USB subsystem calls
>>>>>>>> btusb_intr_complete() to deliver a data(NOOP frame) received
>> on
>>>>>>>> interrupt endpoint. HCI_RUNNING flag is not yet set by
>>>>>>>> bluetooth subsystem. So we ignore that frame and return.
>>>>>>>>
>>>>>>>> As we missed to resubmit the buffer to interrupt endpoint in
>>> this
>>>>>>>> case, we don't get response for BT reset command downloaded
>>> after
>>>>>> this.
>>>>>>>>
>>>>>>>> This patch handles the corner case to resolve zero BD address
>>>>>> problem.
>>>>>>>>
>>>>>>>> Signed-off-by: Ganapathi Bhat <[email protected]>
>>>>>>>> Signed-off-by: Amitkumar Karwar <[email protected]>
>>>>>>>> ---
>>>>>>>> drivers/bluetooth/btusb.c | 5 +----
>>>>>>>> 1 file changed, 1 insertion(+), 4 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/drivers/bluetooth/btusb.c
>>>>>>>> b/drivers/bluetooth/btusb.c index 811f9b9..b5596ac 100644
>>>>>>>> --- a/drivers/bluetooth/btusb.c
>>>>>>>> +++ b/drivers/bluetooth/btusb.c
>>>>>>>> @@ -607,10 +607,7 @@ static void btusb_intr_complete(struct
>> urb
>>>>>> *urb)
>>>>>>>> BT_DBG("%s urb %p status %d count %d", hdev->name, urb,
>>> urb-
>>>>>>>>> status,
>>>>>>>> urb->actual_length);
>>>>>>>>
>>>>>>>> - if (!test_bit(HCI_RUNNING, &hdev->flags))
>>>>>>>> - return;
>>>>>>>> -
>>>>>>>> - if (urb->status == 0) {
>>>>>>>> + if (urb->status == 0 && test_bit(HCI_RUNNING, &hdev-
>>>>> flags)) {
>>>>>>>> hdev->stat.byte_rx += urb->actual_length;
>>>>>>>>
>>>>>>>> if (btusb_recv_intr(data, urb->transfer_buffer,
>>>>>>>
>>>>>>> Did you get a chance to check this?
>>>>>>> Please let us know if you have any review comments.
>>>>>>
>>>>>> can you explain how this is correct and show me the HCI traces
>>>>>> for this.
>>>>>>
>>>>>
>>>>> I suppose HCI trace means hcidump logs here. As device hasn't yet
>>>> initialized, hcidump won't show anything.
>>>>> We had added debug info in btusb driver to trace the data
>> received
>>>>> on
>>>> all USB endpoints and also checked usbmon logs.
>>>>
>>>> use btmon and it will show it.
>>>>
>>>>> Here is the sequence of events we observed in a corner case while
>>>> running stress test.
>>>>> 1) Inside btusb_open() call ------ Thread 1
>>>>> 2) btusb_submit_intr_urb() submits the URB for receiving data on
>>>>> interrupt endpoint ---- Thread 1
>>>>> 3) btusb_intr_complete() gets called to deliver NOP frame from
>> HCI
>>>>> controller ---- Thread 2
>>>>> 4) HCI_RUNNING isn't set yet. So we return from
>>>>> btusb_intr_complete() without resubmitting the buffer --- Thread
>> 2
>>>>> 5) Exit btusb_open() ---- Thread 1
>>>>> 6) "set_bit(HCI_RUNNING, &hdev->flags)" done by bluetooth core
>>>>> ---- Thread 1
>>>>>
>>>>> Later HCI_RESET command gets timedout, as we haven't re-submitted
>>>> buffer for interrupt endpoint in step (4) above.
>>>>>
>>>>> Please find attached logs. usbmon log shows first frame received
>>>>> on
>>>> interrupt endpoint is NOP(Search for Marvell in log).
>>>>>
>>>>> Here is what bluetooth spec says about NOP frame.
>>>>>
>>>>> "To indicate to the Host that the Controller is ready to receive
>>> HCI
>>>>> command packets, the Controller generates a Command Status event
>>>>> with Status 0x00 and Command_Opcode 0x0000, and the
>>>> Num_HCI_Command_Packets
>>>>> event parameter is set to 1 or more. Command_Opcode, 0x0000 is a
>>> NOP
>>>> (No Operation)”
>>>>
>>>> So I wonder if we need to remove the HCI_RUNNING logic from the
>>>> drivers. It is only left in a few USB drivers and I removed the
>> rest
>>>> and moved it into the core. I am not in favour of papering over
>> this
>>>> issue. I need to understand what is wrong. And actually HCI_RUNNING
>>>> needs to be taken away from the drivers. So the question is if
>>> btusb.c
>>>> still needs it or if that is just a leftover. Meaning is it
>>> protecting
>>>> anything?
>>>>
>>>
>>> I can see HCI_RUNNING check being used at below places in btusb.c
>>>
>>> 1) btusb_intr_complete
>>> 2) btusb_bulk_complete
>>> 3) btusb_isoc_complete
>>> 4) btusb_tx_complete
>>> 5) btusb_isoc_tx_complete
>>> 6) btusb_bcm_set_diag
>>> 7) btusb_resume
>>>
>>> I think, it can be removed from (1), (2) and (3), as we already have
>>> below check inside hci_recv_frame()
>>>
>>> if (!hdev || (!test_bit(HCI_UP, &hdev->flags)
>>> && !test_bit(HCI_INIT, &hdev->flags))) {
>>> kfree_skb(skb);
>>> return -ENXIO;
>>> }
>>>
>>> For (4) and (5), it's just used to decide if we should update "hdev-
>>>> stat" or not.
>>>
>>
>> Please guide on how should we fix this issue. We observe timeout for
>> HCI_RESET command sometimes when system is booted. This happens due to
>> missing urb_submitt() when first frame(NOP) received on INT endpoint is
>> ignored. Below change would also help.
>>
>> --- a/net/bluetooth/hci_core.c
>> +++ b/net/bluetooth/hci_core.c
>> @@ -1318,12 +1318,13 @@ static int hci_dev_do_open(struct hci_dev
>> *hdev)
>> goto done;
>> }
>>
>> + set_bit(HCI_RUNNING, &hdev->flags);
>> if (hdev->open(hdev)) {
>> + clear_bit(HCI_RUNNING, &hdev->flags);
>> ret = -EIO;
>> goto done;
>> }
>>
>> - set_bit(HCI_RUNNING, &hdev->flags);
>> hci_sock_dev_event(hdev, HCI_DEV_OPEN);
>
> How can we proceed on this? I'm not much familiar with btusb driver, so not sure if HCI_RUNNING logic can be completely removed.
> Meanwhile could you accept proposed fix in this patch for the potential issue we spotted during stress tests.
>
> The change doesn't have any side-effects. During teardown, race between btusb_close() and btusb_intr_complete() is appropriately taken care of with the help of BTUSB_INTR_RUNNING flag.

so what is the downside of just replacing all HCI_RUNNING with the appropriate BTUSB_INTR_RUNNING, BTUSB_BULK_RUNNING etc.?

Regards

Marcel

2016-12-07 07:44:34

by Amitkumar Karwar

[permalink] [raw]
Subject: RE: [PATCH] btusb: fix zero BD address problem during stress test

SGkgTWFyY2VsLA0KDQo+IEZyb206IEFtaXRrdW1hciBLYXJ3YXINCj4gU2VudDogRnJpZGF5LCBO
b3ZlbWJlciAyNSwgMjAxNiA0OjUxIFBNDQo+IFRvOiAnTWFyY2VsIEhvbHRtYW5uJw0KPiBDYzog
J2xpbnV4LWJsdWV0b290aEB2Z2VyLmtlcm5lbC5vcmcnOyAnbGludXgta2VybmVsQHZnZXIua2Vy
bmVsLm9yZyc7DQo+IENhdGh5IEx1bzsgTmlzaGFudCBTYXJtdWthZGFtOyBHYW5hcGF0aGkgQmhh
dA0KPiBTdWJqZWN0OiBSRTogW1BBVENIXSBidHVzYjogZml4IHplcm8gQkQgYWRkcmVzcyBwcm9i
bGVtIGR1cmluZyBzdHJlc3MNCj4gdGVzdA0KPiANCj4gSGkgTWFyY2VsLA0KPiANCj4gPiBGcm9t
OiBBbWl0a3VtYXIgS2Fyd2FyDQo+ID4gU2VudDogV2VkbmVzZGF5LCBOb3ZlbWJlciAyMywgMjAx
NiAzOjE0IFBNDQo+ID4gVG86ICdNYXJjZWwgSG9sdG1hbm4nDQo+ID4gQ2M6IGxpbnV4LWJsdWV0
b290aEB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWtlcm5lbEB2Z2VyLmtlcm5lbC5vcmc7DQo+ID4g
Q2F0aHkgTHVvOyBOaXNoYW50IFNhcm11a2FkYW07IEdhbmFwYXRoaSBCaGF0DQo+ID4gU3ViamVj
dDogUkU6IFtQQVRDSF0gYnR1c2I6IGZpeCB6ZXJvIEJEIGFkZHJlc3MgcHJvYmxlbSBkdXJpbmcg
c3RyZXNzDQo+ID4gdGVzdA0KPiA+DQo+ID4gSGkgTWFyY2VsLA0KPiA+DQo+ID4gPiBGcm9tOiBN
YXJjZWwgSG9sdG1hbm4gW21haWx0bzptYXJjZWxAaG9sdG1hbm4ub3JnXQ0KPiA+ID4gU2VudDog
V2VkbmVzZGF5LCBOb3ZlbWJlciAyMywgMjAxNiAxOjQ2IFBNDQo+ID4gPiBUbzogQW1pdGt1bWFy
IEthcndhcg0KPiA+ID4gQ2M6IGxpbnV4LWJsdWV0b290aEB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4
LWtlcm5lbEB2Z2VyLmtlcm5lbC5vcmc7DQo+ID4gPiBDYXRoeSBMdW87IE5pc2hhbnQgU2FybXVr
YWRhbTsgR2FuYXBhdGhpIEJoYXQNCj4gPiA+IFN1YmplY3Q6IFJlOiBbUEFUQ0hdIGJ0dXNiOiBm
aXggemVybyBCRCBhZGRyZXNzIHByb2JsZW0gZHVyaW5nDQo+ID4gPiBzdHJlc3MgdGVzdA0KPiA+
ID4NCj4gPiA+IEhpIEFtaXRrdW1hciwNCj4gPiA+DQo+ID4gPiA+Pj4+IEZyb206IEFtaXRrdW1h
ciBLYXJ3YXIgW21haWx0bzpha2Fyd2FyQG1hcnZlbGwuY29tXQ0KPiA+ID4gPj4+PiBTZW50OiBU
dWVzZGF5LCBPY3RvYmVyIDE4LCAyMDE2IDY6MjcgUE0NCj4gPiA+ID4+Pj4gVG86IGxpbnV4LWJs
dWV0b290aEB2Z2VyLmtlcm5lbC5vcmcNCj4gPiA+ID4+Pj4gQ2M6IG1hcmNlbEBob2x0bWFubi5v
cmc7IGxpbnV4LWtlcm5lbEB2Z2VyLmtlcm5lbC5vcmc7IENhdGh5DQo+ID4gTHVvOw0KPiA+ID4g
Pj4+PiBOaXNoYW50IFNhcm11a2FkYW07IEdhbmFwYXRoaSBCaGF0OyBBbWl0a3VtYXIgS2Fyd2Fy
DQo+ID4gPiA+Pj4+IFN1YmplY3Q6IFtQQVRDSF0gYnR1c2I6IGZpeCB6ZXJvIEJEIGFkZHJlc3Mg
cHJvYmxlbSBkdXJpbmcNCj4gPiBzdHJlc3MNCj4gPiA+ID4+Pj4gdGVzdA0KPiA+ID4gPj4+Pg0K
PiA+ID4gPj4+PiBGcm9tOiBHYW5hcGF0aGkgQmhhdCA8Z2JoYXRAbWFydmVsbC5jb20+DQo+ID4g
PiA+Pj4+DQo+ID4gPiA+Pj4+IFdlIGNhbWUgYWNyb3NzIGEgY29ybmVyIGNhc2UgaXNzdWUgZHVy
aW5nIHJlYm9vdCBzdHJlc3MgdGVzdA0KPiBpbg0KPiA+ID4gPj4gd2hpY2gNCj4gPiA+ID4+Pj4g
aGNpY29uZmlnIHNob3dzIEJEIGFkZHJlc3MgaXMgYWxsIHplcm8uIFJlYXNvbiBpcyB3ZSBkb24n
dCBnZXQNCj4gPiA+ID4+Pj4gcmVzcG9uc2UgZm9yIEhDSSBSRVNFVCBjb21tYW5kIGR1cmluZyBp
bml0aWFsaXphdGlvbg0KPiA+ID4gPj4+Pg0KPiA+ID4gPj4+PiBUaGUgaXNzdWUgaXMgdHJhY2tl
ZCB0byBhIHJhY2Ugd2hlcmUgVVNCIHN1YnN5c3RlbSBjYWxscw0KPiA+ID4gPj4+PiBidHVzYl9p
bnRyX2NvbXBsZXRlKCkgdG8gZGVsaXZlciBhIGRhdGEoTk9PUCBmcmFtZSkgcmVjZWl2ZWQNCj4g
b24NCj4gPiA+ID4+Pj4gaW50ZXJydXB0IGVuZHBvaW50LiBIQ0lfUlVOTklORyBmbGFnIGlzIG5v
dCB5ZXQgc2V0IGJ5DQo+ID4gPiA+Pj4+IGJsdWV0b290aCBzdWJzeXN0ZW0uIFNvIHdlIGlnbm9y
ZSB0aGF0IGZyYW1lIGFuZCByZXR1cm4uDQo+ID4gPiA+Pj4+DQo+ID4gPiA+Pj4+IEFzIHdlIG1p
c3NlZCB0byByZXN1Ym1pdCB0aGUgYnVmZmVyIHRvIGludGVycnVwdCBlbmRwb2ludCBpbg0KPiA+
IHRoaXMNCj4gPiA+ID4+Pj4gY2FzZSwgd2UgZG9uJ3QgZ2V0IHJlc3BvbnNlIGZvciBCVCByZXNl
dCBjb21tYW5kIGRvd25sb2FkZWQNCj4gPiBhZnRlcg0KPiA+ID4gPj4gdGhpcy4NCj4gPiA+ID4+
Pj4NCj4gPiA+ID4+Pj4gVGhpcyBwYXRjaCBoYW5kbGVzIHRoZSBjb3JuZXIgY2FzZSB0byByZXNv
bHZlIHplcm8gQkQgYWRkcmVzcw0KPiA+ID4gPj4gcHJvYmxlbS4NCj4gPiA+ID4+Pj4NCj4gPiA+
ID4+Pj4gU2lnbmVkLW9mZi1ieTogR2FuYXBhdGhpIEJoYXQgPGdiaGF0QG1hcnZlbGwuY29tPg0K
PiA+ID4gPj4+PiBTaWduZWQtb2ZmLWJ5OiBBbWl0a3VtYXIgS2Fyd2FyIDxha2Fyd2FyQG1hcnZl
bGwuY29tPg0KPiA+ID4gPj4+PiAtLS0NCj4gPiA+ID4+Pj4gZHJpdmVycy9ibHVldG9vdGgvYnR1
c2IuYyB8IDUgKy0tLS0NCj4gPiA+ID4+Pj4gMSBmaWxlIGNoYW5nZWQsIDEgaW5zZXJ0aW9uKCsp
LCA0IGRlbGV0aW9ucygtKQ0KPiA+ID4gPj4+Pg0KPiA+ID4gPj4+PiBkaWZmIC0tZ2l0IGEvZHJp
dmVycy9ibHVldG9vdGgvYnR1c2IuYw0KPiA+ID4gPj4+PiBiL2RyaXZlcnMvYmx1ZXRvb3RoL2J0
dXNiLmMgaW5kZXggODExZjliOS4uYjU1OTZhYyAxMDA2NDQNCj4gPiA+ID4+Pj4gLS0tIGEvZHJp
dmVycy9ibHVldG9vdGgvYnR1c2IuYw0KPiA+ID4gPj4+PiArKysgYi9kcml2ZXJzL2JsdWV0b290
aC9idHVzYi5jDQo+ID4gPiA+Pj4+IEBAIC02MDcsMTAgKzYwNyw3IEBAIHN0YXRpYyB2b2lkIGJ0
dXNiX2ludHJfY29tcGxldGUoc3RydWN0DQo+IHVyYg0KPiA+ID4gPj4gKnVyYikNCj4gPiA+ID4+
Pj4gCUJUX0RCRygiJXMgdXJiICVwIHN0YXR1cyAlZCBjb3VudCAlZCIsIGhkZXYtPm5hbWUsIHVy
YiwNCj4gPiB1cmItDQo+ID4gPiA+Pj4+PiBzdGF0dXMsDQo+ID4gPiA+Pj4+IAkgICAgICAgdXJi
LT5hY3R1YWxfbGVuZ3RoKTsNCj4gPiA+ID4+Pj4NCj4gPiA+ID4+Pj4gLQlpZiAoIXRlc3RfYml0
KEhDSV9SVU5OSU5HLCAmaGRldi0+ZmxhZ3MpKQ0KPiA+ID4gPj4+PiAtCQlyZXR1cm47DQo+ID4g
PiA+Pj4+IC0NCj4gPiA+ID4+Pj4gLQlpZiAodXJiLT5zdGF0dXMgPT0gMCkgew0KPiA+ID4gPj4+
PiArCWlmICh1cmItPnN0YXR1cyA9PSAwICYmIHRlc3RfYml0KEhDSV9SVU5OSU5HLCAmaGRldi0N
Cj4gPiA+ID5mbGFncykpIHsNCj4gPiA+ID4+Pj4gCQloZGV2LT5zdGF0LmJ5dGVfcnggKz0gdXJi
LT5hY3R1YWxfbGVuZ3RoOw0KPiA+ID4gPj4+Pg0KPiA+ID4gPj4+PiAJCWlmIChidHVzYl9yZWN2
X2ludHIoZGF0YSwgdXJiLT50cmFuc2Zlcl9idWZmZXIsDQo+ID4gPiA+Pj4NCj4gPiA+ID4+PiBE
aWQgeW91IGdldCBhIGNoYW5jZSB0byBjaGVjayB0aGlzPw0KPiA+ID4gPj4+IFBsZWFzZSBsZXQg
dXMga25vdyBpZiB5b3UgaGF2ZSBhbnkgcmV2aWV3IGNvbW1lbnRzLg0KPiA+ID4gPj4NCj4gPiA+
ID4+IGNhbiB5b3UgZXhwbGFpbiBob3cgdGhpcyBpcyBjb3JyZWN0IGFuZCBzaG93IG1lIHRoZSBI
Q0kgdHJhY2VzDQo+ID4gPiA+PiBmb3IgdGhpcy4NCj4gPiA+ID4+DQo+ID4gPiA+DQo+ID4gPiA+
IEkgc3VwcG9zZSBIQ0kgdHJhY2UgbWVhbnMgaGNpZHVtcCBsb2dzIGhlcmUuIEFzIGRldmljZSBo
YXNuJ3QgeWV0DQo+ID4gPiBpbml0aWFsaXplZCwgaGNpZHVtcCB3b24ndCBzaG93IGFueXRoaW5n
Lg0KPiA+ID4gPiBXZSBoYWQgYWRkZWQgZGVidWcgaW5mbyBpbiBidHVzYiBkcml2ZXIgdG8gdHJh
Y2UgdGhlIGRhdGENCj4gcmVjZWl2ZWQNCj4gPiA+ID4gb24NCj4gPiA+IGFsbCBVU0IgZW5kcG9p
bnRzIGFuZCBhbHNvIGNoZWNrZWQgdXNibW9uIGxvZ3MuDQo+ID4gPg0KPiA+ID4gdXNlIGJ0bW9u
IGFuZCBpdCB3aWxsIHNob3cgaXQuDQo+ID4gPg0KPiA+ID4gPiBIZXJlIGlzIHRoZSBzZXF1ZW5j
ZSBvZiBldmVudHMgd2Ugb2JzZXJ2ZWQgaW4gYSBjb3JuZXIgY2FzZSB3aGlsZQ0KPiA+ID4gcnVu
bmluZyBzdHJlc3MgdGVzdC4NCj4gPiA+ID4gMSkgSW5zaWRlIGJ0dXNiX29wZW4oKSBjYWxsIC0t
LS0tLSBUaHJlYWQgMQ0KPiA+ID4gPiAyKSBidHVzYl9zdWJtaXRfaW50cl91cmIoKSBzdWJtaXRz
IHRoZSBVUkIgZm9yIHJlY2VpdmluZyBkYXRhIG9uDQo+ID4gPiA+IGludGVycnVwdCBlbmRwb2lu
dCAtLS0tIFRocmVhZCAxDQo+ID4gPiA+IDMpIGJ0dXNiX2ludHJfY29tcGxldGUoKSBnZXRzIGNh
bGxlZCB0byBkZWxpdmVyIE5PUCBmcmFtZSBmcm9tDQo+IEhDSQ0KPiA+ID4gPiBjb250cm9sbGVy
IC0tLS0gVGhyZWFkIDINCj4gPiA+ID4gNCkgSENJX1JVTk5JTkcgaXNuJ3Qgc2V0IHlldC4gU28g
d2UgcmV0dXJuIGZyb20NCj4gPiA+ID4gYnR1c2JfaW50cl9jb21wbGV0ZSgpIHdpdGhvdXQgcmVz
dWJtaXR0aW5nIHRoZSBidWZmZXIgLS0tIFRocmVhZA0KPiAyDQo+ID4gPiA+IDUpIEV4aXQgYnR1
c2Jfb3BlbigpIC0tLS0gVGhyZWFkIDENCj4gPiA+ID4gNikgInNldF9iaXQoSENJX1JVTk5JTkcs
ICZoZGV2LT5mbGFncykiIGRvbmUgYnkgYmx1ZXRvb3RoIGNvcmUNCj4gPiA+ID4gLS0tLSBUaHJl
YWQgMQ0KPiA+ID4gPg0KPiA+ID4gPiBMYXRlciBIQ0lfUkVTRVQgY29tbWFuZCBnZXRzIHRpbWVk
b3V0LCBhcyB3ZSBoYXZlbid0IHJlLXN1Ym1pdHRlZA0KPiA+ID4gYnVmZmVyIGZvciBpbnRlcnJ1
cHQgZW5kcG9pbnQgaW4gc3RlcCAoNCkgYWJvdmUuDQo+ID4gPiA+DQo+ID4gPiA+IFBsZWFzZSBm
aW5kIGF0dGFjaGVkIGxvZ3MuIHVzYm1vbiBsb2cgc2hvd3MgZmlyc3QgZnJhbWUgcmVjZWl2ZWQN
Cj4gPiA+ID4gb24NCj4gPiA+IGludGVycnVwdCBlbmRwb2ludCBpcyBOT1AoU2VhcmNoIGZvciBN
YXJ2ZWxsIGluIGxvZykuDQo+ID4gPiA+DQo+ID4gPiA+IEhlcmUgaXMgd2hhdCBibHVldG9vdGgg
c3BlYyBzYXlzIGFib3V0IE5PUCBmcmFtZS4NCj4gPiA+ID4NCj4gPiA+ID4gIlRvIGluZGljYXRl
IHRvIHRoZSBIb3N0IHRoYXQgdGhlIENvbnRyb2xsZXIgaXMgcmVhZHkgdG8gcmVjZWl2ZQ0KPiA+
IEhDSQ0KPiA+ID4gPiBjb21tYW5kIHBhY2tldHMsIHRoZSBDb250cm9sbGVyIGdlbmVyYXRlcyBh
IENvbW1hbmQgU3RhdHVzIGV2ZW50DQo+ID4gPiA+IHdpdGggU3RhdHVzIDB4MDAgYW5kIENvbW1h
bmRfT3Bjb2RlIDB4MDAwMCwgYW5kIHRoZQ0KPiA+ID4gTnVtX0hDSV9Db21tYW5kX1BhY2tldHMN
Cj4gPiA+ID4gZXZlbnQgcGFyYW1ldGVyIGlzIHNldCB0byAxIG9yIG1vcmUuIENvbW1hbmRfT3Bj
b2RlLCAweDAwMDAgaXMgYQ0KPiA+IE5PUA0KPiA+ID4gKE5vIE9wZXJhdGlvbinigJ0NCj4gPiA+
DQo+ID4gPiBTbyBJIHdvbmRlciBpZiB3ZSBuZWVkIHRvIHJlbW92ZSB0aGUgSENJX1JVTk5JTkcg
bG9naWMgZnJvbSB0aGUNCj4gPiA+IGRyaXZlcnMuIEl0IGlzIG9ubHkgbGVmdCBpbiBhIGZldyBV
U0IgZHJpdmVycyBhbmQgSSByZW1vdmVkIHRoZQ0KPiByZXN0DQo+ID4gPiBhbmQgbW92ZWQgaXQg
aW50byB0aGUgY29yZS4gSSBhbSBub3QgaW4gZmF2b3VyIG9mIHBhcGVyaW5nIG92ZXINCj4gdGhp
cw0KPiA+ID4gaXNzdWUuIEkgbmVlZCB0byB1bmRlcnN0YW5kIHdoYXQgaXMgd3JvbmcuIEFuZCBh
Y3R1YWxseSBIQ0lfUlVOTklORw0KPiA+ID4gbmVlZHMgdG8gYmUgdGFrZW4gYXdheSBmcm9tIHRo
ZSBkcml2ZXJzLiBTbyB0aGUgcXVlc3Rpb24gaXMgaWYNCj4gPiBidHVzYi5jDQo+ID4gPiBzdGls
bCBuZWVkcyBpdCBvciBpZiB0aGF0IGlzIGp1c3QgYSBsZWZ0b3Zlci4gTWVhbmluZyBpcyBpdA0K
PiA+IHByb3RlY3RpbmcNCj4gPiA+IGFueXRoaW5nPw0KPiA+ID4NCj4gPg0KPiA+IEkgY2FuIHNl
ZSBIQ0lfUlVOTklORyBjaGVjayBiZWluZyB1c2VkIGF0IGJlbG93IHBsYWNlcyBpbiBidHVzYi5j
DQo+ID4NCj4gPiAxKSBidHVzYl9pbnRyX2NvbXBsZXRlDQo+ID4gMikgYnR1c2JfYnVsa19jb21w
bGV0ZQ0KPiA+IDMpIGJ0dXNiX2lzb2NfY29tcGxldGUNCj4gPiA0KSBidHVzYl90eF9jb21wbGV0
ZQ0KPiA+IDUpIGJ0dXNiX2lzb2NfdHhfY29tcGxldGUNCj4gPiA2KSBidHVzYl9iY21fc2V0X2Rp
YWcNCj4gPiA3KSBidHVzYl9yZXN1bWUNCj4gPg0KPiA+IEkgdGhpbmssIGl0IGNhbiBiZSByZW1v
dmVkIGZyb20gKDEpLCAoMikgYW5kICgzKSwgYXMgd2UgYWxyZWFkeSBoYXZlDQo+ID4gYmVsb3cg
Y2hlY2sgaW5zaWRlIGhjaV9yZWN2X2ZyYW1lKCkNCj4gPg0KPiA+ICAgICAgICAgaWYgKCFoZGV2
IHx8ICghdGVzdF9iaXQoSENJX1VQLCAmaGRldi0+ZmxhZ3MpDQo+ID4gICAgICAgICAgICAgICAg
ICAgICAgICYmICF0ZXN0X2JpdChIQ0lfSU5JVCwgJmhkZXYtPmZsYWdzKSkpIHsNCj4gPiAgICAg
ICAgICAgICAgICAga2ZyZWVfc2tiKHNrYik7DQo+ID4gICAgICAgICAgICAgICAgIHJldHVybiAt
RU5YSU87DQo+ID4gICAgICAgICB9DQo+ID4NCj4gPiBGb3IgKDQpIGFuZCAoNSksIGl0J3MganVz
dCB1c2VkIHRvIGRlY2lkZSBpZiB3ZSBzaG91bGQgdXBkYXRlICJoZGV2LQ0KPiA+ID5zdGF0IiBv
ciBub3QuDQo+ID4NCj4gDQo+IFBsZWFzZSBndWlkZSBvbiBob3cgc2hvdWxkIHdlIGZpeCB0aGlz
IGlzc3VlLiBXZSBvYnNlcnZlIHRpbWVvdXQgZm9yDQo+IEhDSV9SRVNFVCBjb21tYW5kIHNvbWV0
aW1lcyB3aGVuIHN5c3RlbSBpcyBib290ZWQuIFRoaXMgaGFwcGVucyBkdWUgdG8NCj4gbWlzc2lu
ZyB1cmJfc3VibWl0dCgpIHdoZW4gZmlyc3QgZnJhbWUoTk9QKSByZWNlaXZlZCBvbiBJTlQgZW5k
cG9pbnQgaXMNCj4gaWdub3JlZC4gQmVsb3cgY2hhbmdlIHdvdWxkIGFsc28gaGVscC4NCj4gDQo+
IC0tLSBhL25ldC9ibHVldG9vdGgvaGNpX2NvcmUuYw0KPiArKysgYi9uZXQvYmx1ZXRvb3RoL2hj
aV9jb3JlLmMNCj4gQEAgLTEzMTgsMTIgKzEzMTgsMTMgQEAgc3RhdGljIGludCBoY2lfZGV2X2Rv
X29wZW4oc3RydWN0IGhjaV9kZXYNCj4gKmhkZXYpDQo+ICAgICAgICAgICAgICAgICBnb3RvIGRv
bmU7DQo+ICAgICAgICAgfQ0KPiANCj4gKyAgICAgICBzZXRfYml0KEhDSV9SVU5OSU5HLCAmaGRl
di0+ZmxhZ3MpOw0KPiAgICAgICAgIGlmIChoZGV2LT5vcGVuKGhkZXYpKSB7DQo+ICsgICAgICAg
ICAgICAgICBjbGVhcl9iaXQoSENJX1JVTk5JTkcsICZoZGV2LT5mbGFncyk7DQo+ICAgICAgICAg
ICAgICAgICByZXQgPSAtRUlPOw0KPiAgICAgICAgICAgICAgICAgZ290byBkb25lOw0KPiAgICAg
ICAgIH0NCj4gDQo+IC0gICAgICAgc2V0X2JpdChIQ0lfUlVOTklORywgJmhkZXYtPmZsYWdzKTsN
Cj4gICAgICAgICBoY2lfc29ja19kZXZfZXZlbnQoaGRldiwgSENJX0RFVl9PUEVOKTsNCg0KSG93
IGNhbiB3ZSBwcm9jZWVkIG9uIHRoaXM/IEknbSBub3QgbXVjaCBmYW1pbGlhciB3aXRoIGJ0dXNi
IGRyaXZlciwgc28gbm90IHN1cmUgaWYgSENJX1JVTk5JTkcgbG9naWMgY2FuIGJlIGNvbXBsZXRl
bHkgcmVtb3ZlZC4NCk1lYW53aGlsZSBjb3VsZCB5b3UgYWNjZXB0IHByb3Bvc2VkIGZpeCBpbiB0
aGlzIHBhdGNoIGZvciB0aGUgcG90ZW50aWFsIGlzc3VlIHdlIHNwb3R0ZWQgZHVyaW5nIHN0cmVz
cyB0ZXN0cy4NCg0KVGhlIGNoYW5nZSBkb2Vzbid0IGhhdmUgYW55IHNpZGUtZWZmZWN0cy4gRHVy
aW5nIHRlYXJkb3duLCByYWNlIGJldHdlZW4gYnR1c2JfY2xvc2UoKSBhbmQgYnR1c2JfaW50cl9j
b21wbGV0ZSgpIGlzIGFwcHJvcHJpYXRlbHkgdGFrZW4gY2FyZSBvZiB3aXRoIHRoZSBoZWxwIG9m
IEJUVVNCX0lOVFJfUlVOTklORyBmbGFnLg0KDQpSZWdhcmRzLA0KQW1pdGt1bWFyIEthcndhcg0K