2017-11-19 15:16:03

by Ellie Reeves

[permalink] [raw]
Subject: Marvell Mwifiex wireless driver: mwifiex_pcie broken on Marvell ESPRESSOBin

Hello,
I noticed the Marvell Mwifiex wireless driver ended up broken since
kernel 4.13.0, and it is still the case with 4.14.0 on the Marvell
ESPRESSOBin, soled by Globalscale Technology. I bought the official
wireless card from them at the same time as the board. However since
kernel 4.13.0, wifi as provided via mwifiex_pcie module stopped working.
I don't have much info on the userspace side, as all that happens is the
wireless interface is never brought up and stays in 'NO-CARRIER'  state
according to 'ip a'. However on kernel side, I hope I have some info
that are more useful ! :) This was logged via systemd-journald with
debug turned on in mwifiex driver. I am willing to try anything to get
this kernel working, as it's a lts kernel contrary to 4.12.12 on which
I'm forced to run for now.

https://gist.githubusercontent.com/mk-fg/27c2505fd3bffcf0af87777aaafb71fe/raw/fda70ff9f8eaab248a104e03922323cf2f4754bf/tmp.txt

I hope these info will be useful in debugging. I'm new to all these
kernel stuff, so I may have posted to the wrong place or in a wrong way,
if so, please let me know.

Thanks in advance

--
Ellie Reeves


2017-11-22 10:13:31

by Ellie Reeves

[permalink] [raw]
Subject: Re: [EXT] Marvell Mwifiex wireless driver: mwifiex_pcie broken on Marvell ESPRESSOBin

Hello,
is the recent patch for mwifiex the one Ganapathi was talking about, to
fix mwifiex on espressobin ? If so, I tried it (in v1 I admit), without
no real change. The interface is no longer shown as 'no-carrier' but
there is still the timeout, and all the messages about fw being in bad
state. :/ Sorry if I confused the patch with something unrelated to the
espressobin.

--
Ellie Reeves

2017-11-30 10:57:50

by Ellie Reeves

[permalink] [raw]
Subject: Re: [EXT] Marvell Mwifiex wireless driver: mwifiex_pcie broken on Marvell ESPRESSOBin

Hi again,
I tried the few patch that made it to this list without any real
success... I suppose the issue isn't that easy to reproduce on your side
so... Shall I try a git bisect, building only that module and see if I
can narrow it down a bit ? I'm no expert with git or git bisect for that
matter, but I'm guessing building the module will be less time consuming
than building the entire kernel... Unless you want me to do something
else to help in debugging. I want to help as much as I can to solve
this, heh, even despite me annoying my father because his wifi access
point goes poof. :)
Thanks,

--
Ellie Reeves

2017-11-20 13:53:10

by Ganapathi Bhat

[permalink] [raw]
Subject: RE: [EXT] Marvell Mwifiex wireless driver: mwifiex_pcie broken on Marvell ESPRESSOBin

SGkgRWxpZSwNCg0KPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+IEhlbGxvLA0KPiBJIG5vdGljZWQgdGhlIE1h
cnZlbGwgTXdpZmlleCB3aXJlbGVzcyBkcml2ZXIgZW5kZWQgdXAgYnJva2VuIHNpbmNlIGtlcm5l
bA0KPiA0LjEzLjAsIGFuZCBpdCBpcyBzdGlsbCB0aGUgY2FzZSB3aXRoIDQuMTQuMCBvbiB0aGUg
TWFydmVsbCBFU1BSRVNTT0Jpbiwgc29sZWQgYnkNCj4gR2xvYmFsc2NhbGUgVGVjaG5vbG9neS4g
SSBib3VnaHQgdGhlIG9mZmljaWFsIHdpcmVsZXNzIGNhcmQgZnJvbSB0aGVtIGF0IHRoZQ0KPiBz
YW1lIHRpbWUgYXMgdGhlIGJvYXJkLiBIb3dldmVyIHNpbmNlIGtlcm5lbCA0LjEzLjAsIHdpZmkg
YXMgcHJvdmlkZWQgdmlhDQo+IG13aWZpZXhfcGNpZSBtb2R1bGUgc3RvcHBlZCB3b3JraW5nLg0K
PiBJIGRvbid0IGhhdmUgbXVjaCBpbmZvIG9uIHRoZSB1c2Vyc3BhY2Ugc2lkZSwgYXMgYWxsIHRo
YXQgaGFwcGVucyBpcyB0aGUNCj4gd2lyZWxlc3MgaW50ZXJmYWNlIGlzIG5ldmVyIGJyb3VnaHQg
dXAgYW5kIHN0YXlzIGluICdOTy1DQVJSSUVSJz8gc3RhdGUNCj4gYWNjb3JkaW5nIHRvICdpcCBh
Jy4gSG93ZXZlciBvbiBrZXJuZWwgc2lkZSwgSSBob3BlIEkgaGF2ZSBzb21lIGluZm8gdGhhdCBh
cmUNCj4gbW9yZSB1c2VmdWwgISA6KSBUaGlzIHdhcyBsb2dnZWQgdmlhIHN5c3RlbWQtam91cm5h
bGQgd2l0aCBkZWJ1ZyB0dXJuZWQgb24NCj4gaW4gbXdpZmlleCBkcml2ZXIuIEkgYW0gd2lsbGlu
ZyB0byB0cnkgYW55dGhpbmcgdG8gZ2V0IHRoaXMga2VybmVsIHdvcmtpbmcsIGFzIGl0J3MNCj4g
YSBsdHMga2VybmVsIGNvbnRyYXJ5IHRvIDQuMTIuMTIgb24gd2hpY2ggSSdtIGZvcmNlZCB0byBy
dW4gZm9yIG5vdy4NCj4NCj4gaHR0cHM6Ly9naXN0LmdpdGh1YnVzZXJjb250ZW50LmNvbS9tay0N
Cj4gZmcvMjdjMjUwNWZkM2JmZmNmMGFmODc3NzdhYWFmYjcxZmUvcmF3L2ZkYTcwZmY5ZjhlYWFi
MjQ4YTEwNGUwMzkyMjMyDQo+IDNjZjJmNDc1NGJmL3RtcC50eHQNCj4NCj4gSSBob3BlIHRoZXNl
IGluZm8gd2lsbCBiZSB1c2VmdWwgaW4gZGVidWdnaW5nLiBJJ20gbmV3IHRvIGFsbCB0aGVzZSBr
ZXJuZWwNCj4gc3R1ZmYsIHNvIEkgbWF5IGhhdmUgcG9zdGVkIHRvIHRoZSB3cm9uZyBwbGFjZSBv
ciBpbiBhIHdyb25nIHdheSwgaWYgc28sDQo+IHBsZWFzZSBsZXQgbWUga25vdy4NCj4NCj4gVGhh
bmtzIGluIGFkdmFuY2UNCj4NCg0KVGhhbmtzIGZvciB0aGUgbm90aWZpY2F0aW9uIGFuZCBsb2dz
LiAgV2UgaGF2ZSBhIGZpeCBmb3IgdGhpcyBpc3N1ZSwgd2hpY2ggd2lsbCBiZSBzZW5kIHVwc3Ry
ZWFtIHNvb24uDQoNCj4gLS0NCj4gRWxsaWUgUmVldmVzDQoNCg0KUmVnYXJkcywNCkdhbmFwYXRo
aQ0K

2017-11-22 13:14:26

by Ellie Reeves

[permalink] [raw]
Subject: Re: [EXT] Marvell Mwifiex wireless driver: mwifiex_pcie broken on Marvell ESPRESSOBin

Hello,
thanks for the quick answer. To answer Ganapathi's question, all I have
to do in order to trigger the issue is to:

1. Upgrade kernel as usual.
2. Reboot and see the wifi going insane.

If I want to get it working properly I have to:

1. Downgrade kernel to 4.12.12.
2. Make sure initramfs and the uImage for u-boot have been generated for
the 4.12.12 kernel again (my distrubution has a script that takes care
of this).
3. Perform a "shutdown now". A simple soft reboot, and even pressing the
reset button on the ESPRESSObin isn't enough to completely reset the
state, if you reboot this way you will get the "fw is already running !"
message and it will not work. You have to unplug and then replug the
board. After that kernel 4.12.12 will boot and wifi will work.

Here is the dmesg output:

https://paste.xogium.me/view/4dc3bebd
--

Ellie Reeves

2017-11-20 20:51:27

by Ellie Reeves

[permalink] [raw]
Subject: Re: [EXT] Marvell Mwifiex wireless driver: mwifiex_pcie broken on Marvell ESPRESSOBin

Hi Ganapathi,
thank you a lot for the quick answer :). Would it be possible for you to
send me the patch to fix this, so I could patch the kernel myself ? I'd
really appreciate that, and also would like to know a bit more about the
issue, yes, I'm curious. :) I like learning things.

--
Ellie Reeves

2017-11-22 12:34:06

by Ganapathi Bhat

[permalink] [raw]
Subject: RE: [EXT] Marvell Mwifiex wireless driver: mwifiex_pcie broken on Marvell ESPRESSOBin

SGkgRWxsaWUsDQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogRWxsaWUg
UmVldmVzIFttYWlsdG86ZWxsaWVyZXZ2ZXNAZ21haWwuY29tXQ0KPiBTZW50OiBXZWRuZXNkYXks
IE5vdmVtYmVyIDIyLCAyMDE3IDM6NDMgUE0NCj4gVG86IEdhbmFwYXRoaSBCaGF0DQo+IENjOiBs
aW51eC13aXJlbGVzc0B2Z2VyLmtlcm5lbC5vcmc7IGFtaXRrYXJ3YXJAZ21haWwuY29tOyBOaXNo
YW50DQo+IFNhcm11a2FkYW07IFhpbm1pbmcgSHUNCj4gU3ViamVjdDogUmU6IFtFWFRdIE1hcnZl
bGwgTXdpZmlleCB3aXJlbGVzcyBkcml2ZXI6IG13aWZpZXhfcGNpZSBicm9rZW4gb24NCj4gTWFy
dmVsbCBFU1BSRVNTT0Jpbg0KPg0KPiBIZWxsbywNCj4gaXMgdGhlIHJlY2VudCBwYXRjaCBmb3Ig
bXdpZmlleCB0aGUgb25lIEdhbmFwYXRoaSB3YXMgdGFsa2luZyBhYm91dCwgdG8gZml4DQo+IG13
aWZpZXggb24gZXNwcmVzc29iaW4gPw0KDQpZZXMNCg0KPiBJZiBzbywgSSB0cmllZCBpdCAoaW4g
djEgSSBhZG1pdCksIHdpdGhvdXQgbm8gcmVhbA0KPiBjaGFuZ2UuDQpDYW4geW91IGp1c3Qgc2hh
cmUgdGhlIGRtc2cgbG9ncz8gQWxzbyBsZXQgdXMga25vdyBpZiB0aGVyZSBhcmUgYW55IHNwZWNp
ZmljIHN0ZXBzIHlvdSBmb2xsb3cuDQoNCj4gVGhlIGludGVyZmFjZSBpcyBubyBsb25nZXIgc2hv
d24gYXMgJ25vLWNhcnJpZXInIGJ1dCB0aGVyZSBpcyBzdGlsbCB0aGUNCj4gdGltZW91dCwgYW5k
IGFsbCB0aGUgbWVzc2FnZXMgYWJvdXQgZncgYmVpbmcgaW4gYmFkIHN0YXRlLiA6LyBTb3JyeSBp
ZiBJDQo+IGNvbmZ1c2VkIHRoZSBwYXRjaCB3aXRoIHNvbWV0aGluZyB1bnJlbGF0ZWQgdG8gdGhl
IGVzcHJlc3NvYmluLg0KPg0KPiAtLQ0KPiBFbGxpZSBSZWV2ZXMNCg0KVGhhbmtzLA0KR2FuYXBh
dGhpDQo=

2017-12-13 10:34:58

by Ganapathi Bhat

[permalink] [raw]
Subject: RE: [EXT] Marvell Mwifiex wireless driver: mwifiex_pcie broken on Marvell ESPRESSOBin

SGkgRWxsaWUNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBFbGxpZSBS
ZWV2ZXMgW21haWx0bzplbGxpZXJldnZlc0BnbWFpbC5jb21dDQo+IFNlbnQ6IEZyaWRheSwgRGVj
ZW1iZXIgMDEsIDIwMTcgOTowNyBQTQ0KPiBUbzogR2FuYXBhdGhpIEJoYXQNCj4gQ2M6IGxpbnV4
LXdpcmVsZXNzQHZnZXIua2VybmVsLm9yZzsgYW1pdGthcndhckBnbWFpbC5jb207IE5pc2hhbnQN
Cj4gU2FybXVrYWRhbTsgWGlubWluZyBIdQ0KPiBTdWJqZWN0OiBSZTogW0VYVF0gTWFydmVsbCBN
d2lmaWV4IHdpcmVsZXNzIGRyaXZlcjogbXdpZmlleF9wY2llIGJyb2tlbiBvbg0KPiBNYXJ2ZWxs
IEVTUFJFU1NPQmluDQo+DQo+IEhpLA0KPiBqdXN0IGEgc2hvcnQgbWVzc2FnZSB0byBtZW50aW9u
IEkgYm9vdGVkIHRoZSA0LjE0IGtlcm5lbCBhbmQgZ3JhYmJlZCB3aGF0DQo+IHdhcyBpbiAvc3lz
L2NsYXNzL2RldmNvcmVkdW1wL2RldmNkMS9kYXRhIGludG8gYSBmaWxlLiBJJ20gbm90IHN1cmUg
aWYgaXQNCj4gY291bGQgYmUgdXNlZnVsLCBidXQgZGlkIGl0IGFueXdheXMuLi4gTGV0IG1lIGtu
b3cgaWYgaXQncyB1c2VmdWwgYW5kIEknbGwgcG9zdA0KPiB0aGUgYmluYXJ5IHNvbWV3aGVyZS4N
Cj4gVGhhbmtzDQpZb3UgY2FuIHNoYXJlIHRoYXQuICBBZGRpdGlvbmFsbHksIGNhbiB5b3Ugc2hh
cmUgdGhlIGRtZXNnIGxvZ3Mgd2l0aCBleHRlbnNpdmUgbG9nIGxldmVsLg0KWW91IGNhbiB1c2Ug
YmVsb3cgY29tbWFuZCB0byBzZXQgdGhlIHNhbWU6DQplY2hvICAgIDB4ZmZmZmZmZmYgICAgICA+
ICAgICAgIC9zeXMva2VybmVsL2RlYnVnL213aWZpZXgvbWxhbjAvZGVidWdfbWFzaw0KPg0KPiAt
LS0tLS0tLSBNZXNzYWdlIG9yaWdpbmFsIC0tLS0tLS0tDQo+ID4gSGkgYWdhaW4sDQo+ID4gSSB0
cmllZCB0aGUgZmV3IHBhdGNoIHRoYXQgbWFkZSBpdCB0byB0aGlzIGxpc3Qgd2l0aG91dCBhbnkg
cmVhbA0KPiA+IHN1Y2Nlc3MuLi4gSSBzdXBwb3NlIHRoZSBpc3N1ZSBpc24ndCB0aGF0IGVhc3kg
dG8gcmVwcm9kdWNlIG9uIHlvdXINCj4gPiBzaWRlIHNvLi4uIFNoYWxsIEkgdHJ5IGEgZ2l0IGJp
c2VjdCwgYnVpbGRpbmcgb25seSB0aGF0IG1vZHVsZSBhbmQgc2VlDQo+ID4gaWYgSSBjYW4gbmFy
cm93IGl0IGRvd24gYSBiaXQgPyBJJ20gbm8gZXhwZXJ0IHdpdGggZ2l0IG9yIGdpdCBiaXNlY3QN
Cj4gPiBmb3IgdGhhdCBtYXR0ZXIsIGJ1dCBJJ20gZ3Vlc3NpbmcgYnVpbGRpbmcgdGhlIG1vZHVs
ZSB3aWxsIGJlIGxlc3MNCj4gPiB0aW1lIGNvbnN1bWluZyB0aGFuIGJ1aWxkaW5nIHRoZSBlbnRp
cmUga2VybmVsLi4uIFVubGVzcyB5b3Ugd2FudCBtZQ0KPiA+IHRvIGRvIHNvbWV0aGluZyBlbHNl
IHRvIGhlbHAgaW4gZGVidWdnaW5nLiBJIHdhbnQgdG8gaGVscCBhcyBtdWNoIGFzIEkNCj4gPiBj
YW4gdG8gc29sdmUgdGhpcywgaGVoLCBldmVuIGRlc3BpdGUgbWUgYW5ub3lpbmcgbXkgZmF0aGVy
IGJlY2F1c2UgaGlzDQo+ID4gd2lmaSBhY2Nlc3MgcG9pbnQgZ29lcyBwb29mLiA6KSBUaGFua3Ms
DQo+ID4NCj4NCj4gLS0NCj4gRWxsaWUgUmVldmVzDQoNCg==

2017-12-13 10:46:25

by Ellie Reeves

[permalink] [raw]
Subject: Re: [EXT] Marvell Mwifiex wireless driver: mwifiex_pcie broken on Marvell ESPRESSOBin

Hi all,
here is the firmware dump... It looks more like a binary blob to me, but
I guess there are some tools to inspect it:
https://cloud.xogium.me/s/wDf9bOWZU3cngw8/download

(dirrect link). As for the dmesg output, the first logs I posted were
exactly this, all kernel messages filtered from journald, and nothing
serious seems to have changed. I will redo it as soon as I have a chance
and post it. I cannot use dmesg because there are so much info generated
the buffer is incomplete, hence the use of journalctl -b -k. Doesn't
really help that the poor board is having a load average of 5 while
doing this, but I will try to capture a new debug log.

Hope this helps

-------- Message original --------
> Hi Ellie
>
>> -----Original Message-----
>> From: Ellie Reeves [mailto:[email protected]]
>> Sent: Friday, December 01, 2017 9:07 PM
>> To: Ganapathi Bhat
>> Cc: [email protected]; [email protected]; Nishant
>> Sarmukadam; Xinming Hu
>> Subject: Re: [EXT] Marvell Mwifiex wireless driver: mwifiex_pcie broken on
>> Marvell ESPRESSOBin
>>
>> Hi,
>> just a short message to mention I booted the 4.14 kernel and grabbed what
>> was in /sys/class/devcoredump/devcd1/data into a file. I'm not sure if it
>> could be useful, but did it anyways... Let me know if it's useful and I'll post
>> the binary somewhere.
>> Thanks
> You can share that. Additionally, can you share the dmesg logs with extensive log level.
> You can use below command to set the same:
> echo 0xffffffff > /sys/kernel/debug/mwifiex/mlan0/debug_mask
>> -------- Message original --------
>>> Hi again,
>>> I tried the few patch that made it to this list without any real
>>> success... I suppose the issue isn't that easy to reproduce on your
>>> side so... Shall I try a git bisect, building only that module and see
>>> if I can narrow it down a bit ? I'm no expert with git or git bisect
>>> for that matter, but I'm guessing building the module will be less
>>> time consuming than building the entire kernel... Unless you want me
>>> to do something else to help in debugging. I want to help as much as I
>>> can to solve this, heh, even despite me annoying my father because his
>>> wifi access point goes poof. :) Thanks,
>>>
>> --
>> Ellie Reeves

--
Ellie Reeves

2017-12-01 15:37:29

by Ellie Reeves

[permalink] [raw]
Subject: Re: [EXT] Marvell Mwifiex wireless driver: mwifiex_pcie broken on Marvell ESPRESSOBin

Hi,
just a short message to mention I booted the 4.14 kernel and grabbed
what was in /sys/class/devcoredump/devcd1/data into a file. I'm not sure
if it could be useful, but did it anyways... Let me know if it's useful
and I'll post the binary somewhere.
Thanks

-------- Message original --------
> Hi again,
> I tried the few patch that made it to this list without any real
> success... I suppose the issue isn't that easy to reproduce on your
> side so... Shall I try a git bisect, building only that module and see
> if I can narrow it down a bit ? I'm no expert with git or git bisect
> for that matter, but I'm guessing building the module will be less
> time consuming than building the entire kernel... Unless you want me
> to do something else to help in debugging. I want to help as much as I
> can to solve this, heh, even despite me annoying my father because his
> wifi access point goes poof. :)
> Thanks,
>

--
Ellie Reeves