2021-07-03 22:17:21

by Rafał Miłecki

[permalink] [raw]
Subject: Unstable WiFi with mt76 on MT7628AN

Hi,

I decided to start my adventure with mt76. I bought the cheapest router
I could find: Xiaomi Mi Router 4C and installed OpenWrt 21.02-rc3 on it.

OpenWrt 21.02-rc3 contains mt76 from May:
https://git.openwrt.org/?p=openwrt/openwrt.git;a=commitdiff;h=43e4ba586336383b34698a00d4588a17b853922f

It *does not* include some later changes from June:
https://git.openwrt.org/?p=openwrt/openwrt.git;a=commitdiff;h=20f66649dda3094320e515006cd5ddc9c096ac30
(or the very latest changes).

My problem is unstable WiFi (as vague as this description is). I often
experience stalls (I can't even see beacons) and sometimes some slow
downs. I started using this device about 6 hours ago and experienced 3
or 4 traffic stops with no beacons.

WiFi can work flawlessly for an hour, e.g. this iperf2 session:
[ 3] 0.0-3339.0 sec 25.3 GBytes 65.1 Mbits/sec

And another time it can suddently stop:
[ 3] 54.0-55.0 sec 5.38 MBytes 45.1 Mbits/sec
[ 3] 55.0-56.0 sec 5.88 MBytes 49.3 Mbits/sec
[ 3] 56.0-57.0 sec 8.12 MBytes 68.2 Mbits/sec
[ 3] 57.0-58.0 sec 5.12 MBytes 43.0 Mbits/sec
[ 3] 58.0-59.0 sec 6.88 MBytes 57.7 Mbits/sec
[ 3] 59.0-60.0 sec 7.88 MBytes 66.1 Mbits/sec
[ 3] 60.0-61.0 sec 7.88 MBytes 66.1 Mbits/sec
[ 3] 61.0-62.0 sec 5.04 MBytes 42.3 Mbits/sec
[ 3] 62.0-63.0 sec 318 KBytes 2.61 Mbits/sec
[ 3] 63.0-64.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 64.0-65.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 65.0-66.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 66.0-67.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 67.0-68.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 68.0-69.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 69.0-70.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 70.0-71.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 71.0-72.0 sec 0.00 Bytes 0.00 bits/sec
(I didn't move my AP, didn't move my STA, didn't get new STAs).

After stall as above I can't even reconnect. Most attempts fail as I
can't even see a single beacon. Sometimes I catch a single beacon and
even manage to connect but not even ping can reach my AP.

Sometimes traffic just stops for a few seconds:
[ 3] 1120.0-1121.0 sec 2.88 MBytes 24.1 Mbits/sec
[ 3] 1121.0-1122.0 sec 3.62 MBytes 30.4 Mbits/sec
[ 3] 1122.0-1123.0 sec 1.20 MBytes 10.1 Mbits/sec
[ 3] 1123.0-1124.0 sec 2.00 MBytes 16.8 Mbits/sec
[ 3] 1124.0-1125.0 sec 3.12 MBytes 26.2 Mbits/sec
[ 3] 1125.0-1126.0 sec 5.12 MBytes 43.0 Mbits/sec
[ 3] 1126.0-1127.0 sec 4.38 MBytes 36.7 Mbits/sec
[ 3] 1127.0-1128.0 sec 5.25 MBytes 44.0 Mbits/sec
[ 3] 1128.0-1129.0 sec 3.49 MBytes 29.2 Mbits/sec
[ 3] 1129.0-1130.0 sec 509 KBytes 4.17 Mbits/sec
[ 3] 1130.0-1131.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1131.0-1132.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1132.0-1133.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1133.0-1134.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1134.0-1135.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1135.0-1136.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1136.0-1137.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1137.0-1138.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1138.0-1139.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1139.0-1140.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1140.0-1141.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1141.0-1142.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1142.0-1143.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1143.0-1144.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 1144.0-1145.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 1145.0-1146.0 sec 3.62 MBytes 30.4 Mbits/sec
[ 3] 1146.0-1147.0 sec 4.38 MBytes 36.7 Mbits/sec
[ 3] 1147.0-1148.0 sec 3.88 MBytes 32.5 Mbits/sec
[ 3] 1148.0-1149.0 sec 4.50 MBytes 37.7 Mbits/sec
[ 3] 1149.0-1150.0 sec 3.75 MBytes 31.5 Mbits/sec

Or slows down:
[ 3] 1316.0-1317.0 sec 3.88 MBytes 32.5 Mbits/sec
[ 3] 1317.0-1318.0 sec 3.00 MBytes 25.2 Mbits/sec
[ 3] 1318.0-1319.0 sec 4.75 MBytes 39.8 Mbits/sec
[ 3] 1319.0-1320.0 sec 3.00 MBytes 25.2 Mbits/sec
[ 3] 1320.0-1321.0 sec 3.88 MBytes 32.5 Mbits/sec
[ 3] 1321.0-1322.0 sec 2.95 MBytes 24.7 Mbits/sec
[ 3] 1322.0-1323.0 sec 1018 KBytes 8.34 Mbits/sec
[ 3] 1323.0-1324.0 sec 954 KBytes 7.82 Mbits/sec
[ 3] 1324.0-1325.0 sec 827 KBytes 6.78 Mbits/sec
[ 3] 1325.0-1326.0 sec 891 KBytes 7.30 Mbits/sec
[ 3] 1326.0-1327.0 sec 827 KBytes 6.78 Mbits/sec
[ 3] 1327.0-1328.0 sec 764 KBytes 6.26 Mbits/sec
[ 3] 1328.0-1329.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 1329.0-1330.0 sec 1.23 MBytes 10.4 Mbits/sec
[ 3] 1330.0-1331.0 sec 1.18 MBytes 9.93 Mbits/sec
[ 3] 1331.0-1332.0 sec 3.00 MBytes 25.2 Mbits/sec
[ 3] 1332.0-1333.0 sec 2.25 MBytes 18.9 Mbits/sec
[ 3] 1333.0-1334.0 sec 1.34 MBytes 11.2 Mbits/sec
[ 3] 1334.0-1335.0 sec 1.24 MBytes 10.4 Mbits/sec
[ 3] 1335.0-1336.0 sec 1.24 MBytes 10.4 Mbits/sec
[ 3] 1336.0-1337.0 sec 1.75 MBytes 14.7 Mbits/sec
[ 3] 1337.0-1338.0 sec 5.25 MBytes 44.0 Mbits/sec
[ 3] 1338.0-1339.0 sec 4.38 MBytes 36.7 Mbits/sec
[ 3] 1339.0-1340.0 sec 3.75 MBytes 31.5 Mbits/sec
[ 3] 1340.0-1341.0 sec 3.88 MBytes 32.5 Mbits/sec

Whenever any issue occurs I don't see anything in my dmesg. Every time
it happens it affects all of my 3 STAs (notebook + 2 smartphones).

Waiting for few minutes brings things back to normal. I don't need any
hostapd restart / device reboot.

Stability problem was reported for this router multiple times by
different users. Reports go back to the October 2020 when OpenWrt
received that router support.

https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/42
https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/46
https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/48
https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/52
https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/65
https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/97
https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/112
https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/113

Is there anything I can do to help debugging / fixing this?


2021-07-03 22:19:34

by Rafał Miłecki

[permalink] [raw]
Subject: Re: Unstable WiFi with mt76 on MT7628AN

On 04.07.2021 00:16, Rafał Miłecki wrote:
> My problem is unstable WiFi (as vague as this description is). I often
> experience stalls (I can't even see beacons) and sometimes some slow
> downs. I started using this device about 6 hours ago and experienced 3
> or 4 traffic stops with no beacons.

It may be worth mentioning I found 1 WARNING in my dmesg. I can't tell
if that has happened during any slow down / stall or not. Unforunately I
don't have symbols in my backtrace.

[ 3245.170684] ------------[ cut here ]------------
[ 3245.175434] WARNING: CPU: 0 PID: 627 at target-mipsel_24kc_musl/linux-ramips_mt76x8/mt76-2021-05-15-9d736545/mt7603/mac.c:206 0x835e44ac [mt7603e@bb8c9e4c+0x9520]
[ 3245.190168] Modules linked in: pppoe ppp_async iptable_nat xt_state xt_nat xt_conntrack xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD xt_CT pppox ppp_generic nf_nat nf_flow_table_hw nf_flow_table nf_conntrack mt7603e mt76 mac80211 ipt_REJECT cfg80211 xt_time xt_tcpudp xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG slhc nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_filter ip_tables crc_ccitt compat nf_log_ipv6 nf_log_common ip6table_mangle
ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 leds_gpio gpio_button_hotplug
[ 3245.241986] CPU: 0 PID: 627 Comm: kworker/u3:1 Not tainted 5.4.124 #0
[ 3245.248532] Workqueue: napi_workq 0x80353380
[ 3245.252871] Stack : 805a0000 834ff700 80505c80 835e44ac 00000000 00000000 00000000 00000000
[ 3245.261358] 00000000 00000000 00000000 00000000 00000000 00000001 83629bb8 985d1d7f
[ 3245.269844] 83629c50 00000000 00000000 00000000 00000038 804eb7e4 616e203a 775f6970
[ 3245.278337] 00000000 000000a1 00000000 0003cad4 00000000 83629b98 00000000 835e44ac
[ 3245.286827] 00000009 00000002 00000002 835f0000 00000003 802b8bdc 00000000 806f0000
[ 3245.295314] ...
[ 3245.297802] Call Trace:
[ 3245.297833] [<835e44ac>] 0x835e44ac [mt7603e@bb8c9e4c+0x9520]
[ 3245.306164] [<804eb7e4>] 0x804eb7e4
[ 3245.309725] [<835e44ac>] 0x835e44ac [mt7603e@bb8c9e4c+0x9520]
[ 3245.315571] [<802b8bdc>] 0x802b8bdc
[ 3245.319131] [<80009790>] 0x80009790
[ 3245.322680] [<80009798>] 0x80009798
[ 3245.326227] [<800236d0>] 0x800236d0
[ 3245.329783] [<835e44ac>] 0x835e44ac [mt7603e@bb8c9e4c+0x9520]
[ 3245.335623] [<80023778>] 0x80023778
[ 3245.339181] [<835e44ac>] 0x835e44ac [mt7603e@bb8c9e4c+0x9520]
[ 3245.345018] [<80351c4c>] 0x80351c4c
[ 3245.348594] [<835e4644>] 0x835e4644 [mt7603e@bb8c9e4c+0x9520]
[ 3245.354440] [<835d598c>] 0x835d598c [mt76@879d366c+0x93a0]
[ 3245.360021] [<835d3538>] 0x835d3538 [mt76@879d366c+0x93a0]
[ 3245.365614] [<835e1a1c>] 0x835e1a1c [mt7603e@bb8c9e4c+0x9520]
[ 3245.371470] [<835d47d0>] 0x835d47d0 [mt76@879d366c+0x93a0]
[ 3245.377089] [<835d153c>] 0x835d153c [mt76@879d366c+0x93a0]
[ 3245.382669] [<804f0cc8>] 0x804f0cc8
[ 3245.386216] [<803533a4>] 0x803533a4
[ 3245.389786] [<803532b0>] 0x803532b0
[ 3245.393342] [<803533a4>] 0x803533a4
[ 3245.396899] [<803533c0>] 0x803533c0
[ 3245.400447] [<804ec9c0>] 0x804ec9c0
[ 3245.403988] [<8003ac64>] 0x8003ac64
[ 3245.407547] [<8003b02c>] 0x8003b02c
[ 3245.411095] [<8003aec4>] 0x8003aec4
[ 3245.414642] [<804ecbf8>] 0x804ecbf8
[ 3245.418202] [<8003aec4>] 0x8003aec4
[ 3245.421750] [<8003fd68>] 0x8003fd68
[ 3245.425298] [<8003fc30>] 0x8003fc30
[ 3245.428852] [<8003fc30>] 0x8003fc30
[ 3245.432407] [<800050b8>] 0x800050b8
[ 3245.435945]
[ 3245.437468] ---[ end trace 9c6fbf57c9d56e22 ]---

2021-07-04 07:38:47

by Rafał Miłecki

[permalink] [raw]
Subject: Re: Unstable WiFi with mt76 on MT7628AN

A small update (no good news).

On 04.07.2021 00:16, Rafał Miłecki wrote:
> I decided to start my adventure with mt76. I bought the cheapest router
> I could find: Xiaomi Mi Router 4C and installed OpenWrt 21.02-rc3 on it.
>
> OpenWrt 21.02-rc3 contains mt76 from May:
> https://git.openwrt.org/?p=openwrt/openwrt.git;a=commitdiff;h=43e4ba586336383b34698a00d4588a17b853922f
>
> It *does not* include some later changes from June:
> https://git.openwrt.org/?p=openwrt/openwrt.git;a=commitdiff;h=20f66649dda3094320e515006cd5ddc9c096ac30
> (or the very latest changes).
>
> My problem is unstable WiFi (as vague as this description is). I often
> experience stalls (I can't even see beacons) and sometimes some slow
> downs. I started using this device about 6 hours ago and experienced 3
> or 4 traffic stops with no beacons.

I updated my OpenWrt distro to the latest 21.02 snapshot which includes:
https://git.openwrt.org/?p=openwrt/openwrt.git;a=commitdiff;h=20f66649dda3094320e515006cd5ddc9c096ac30

Problem persists. WiFi is unusable for any daily use.

I'm attaching:
1. iperf & ping output from my notebook (STA)
2. system log from router (AP)


> WiFi can work flawlessly for an hour, e.g. this iperf2 session:
> [  3]  0.0-3339.0 sec  25.3 GBytes  65.1 Mbits/sec
>
> And another time it can suddently stop:
> [  3] 54.0-55.0 sec  5.38 MBytes  45.1 Mbits/sec
> [  3] 55.0-56.0 sec  5.88 MBytes  49.3 Mbits/sec
> [  3] 56.0-57.0 sec  8.12 MBytes  68.2 Mbits/sec
> [  3] 57.0-58.0 sec  5.12 MBytes  43.0 Mbits/sec
> [  3] 58.0-59.0 sec  6.88 MBytes  57.7 Mbits/sec
> [  3] 59.0-60.0 sec  7.88 MBytes  66.1 Mbits/sec
> [  3] 60.0-61.0 sec  7.88 MBytes  66.1 Mbits/sec
> [  3] 61.0-62.0 sec  5.04 MBytes  42.3 Mbits/sec
> [  3] 62.0-63.0 sec   318 KBytes  2.61 Mbits/sec
> [  3] 63.0-64.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 64.0-65.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 65.0-66.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 66.0-67.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 67.0-68.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 68.0-69.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 69.0-70.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 70.0-71.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 71.0-72.0 sec  0.00 Bytes  0.00 bits/sec
> (I didn't move my AP, didn't move my STA, didn't get new STAs).
>
> After stall as above I can't even reconnect. Most attempts fail as I
> can't even see a single beacon. Sometimes I catch a single beacon and
> even manage to connect but not even ping can reach my AP.
>
> Sometimes traffic just stops for a few seconds:
> [  3] 1120.0-1121.0 sec  2.88 MBytes  24.1 Mbits/sec
> [  3] 1121.0-1122.0 sec  3.62 MBytes  30.4 Mbits/sec
> [  3] 1122.0-1123.0 sec  1.20 MBytes  10.1 Mbits/sec
> [  3] 1123.0-1124.0 sec  2.00 MBytes  16.8 Mbits/sec
> [  3] 1124.0-1125.0 sec  3.12 MBytes  26.2 Mbits/sec
> [  3] 1125.0-1126.0 sec  5.12 MBytes  43.0 Mbits/sec
> [  3] 1126.0-1127.0 sec  4.38 MBytes  36.7 Mbits/sec
> [  3] 1127.0-1128.0 sec  5.25 MBytes  44.0 Mbits/sec
> [  3] 1128.0-1129.0 sec  3.49 MBytes  29.2 Mbits/sec
> [  3] 1129.0-1130.0 sec   509 KBytes  4.17 Mbits/sec
> [  3] 1130.0-1131.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1131.0-1132.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1132.0-1133.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1133.0-1134.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1134.0-1135.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1135.0-1136.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1136.0-1137.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1137.0-1138.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1138.0-1139.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1139.0-1140.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1140.0-1141.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1141.0-1142.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1142.0-1143.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1143.0-1144.0 sec  0.00 Bytes  0.00 bits/sec
> [  3] 1144.0-1145.0 sec   128 KBytes  1.05 Mbits/sec
> [  3] 1145.0-1146.0 sec  3.62 MBytes  30.4 Mbits/sec
> [  3] 1146.0-1147.0 sec  4.38 MBytes  36.7 Mbits/sec
> [  3] 1147.0-1148.0 sec  3.88 MBytes  32.5 Mbits/sec
> [  3] 1148.0-1149.0 sec  4.50 MBytes  37.7 Mbits/sec
> [  3] 1149.0-1150.0 sec  3.75 MBytes  31.5 Mbits/sec
>
> Or slows down:
> [  3] 1316.0-1317.0 sec  3.88 MBytes  32.5 Mbits/sec
> [  3] 1317.0-1318.0 sec  3.00 MBytes  25.2 Mbits/sec
> [  3] 1318.0-1319.0 sec  4.75 MBytes  39.8 Mbits/sec
> [  3] 1319.0-1320.0 sec  3.00 MBytes  25.2 Mbits/sec
> [  3] 1320.0-1321.0 sec  3.88 MBytes  32.5 Mbits/sec
> [  3] 1321.0-1322.0 sec  2.95 MBytes  24.7 Mbits/sec
> [  3] 1322.0-1323.0 sec  1018 KBytes  8.34 Mbits/sec
> [  3] 1323.0-1324.0 sec   954 KBytes  7.82 Mbits/sec
> [  3] 1324.0-1325.0 sec   827 KBytes  6.78 Mbits/sec
> [  3] 1325.0-1326.0 sec   891 KBytes  7.30 Mbits/sec
> [  3] 1326.0-1327.0 sec   827 KBytes  6.78 Mbits/sec
> [  3] 1327.0-1328.0 sec   764 KBytes  6.26 Mbits/sec
> [  3] 1328.0-1329.0 sec  1.50 MBytes  12.6 Mbits/sec
> [  3] 1329.0-1330.0 sec  1.23 MBytes  10.4 Mbits/sec
> [  3] 1330.0-1331.0 sec  1.18 MBytes  9.93 Mbits/sec
> [  3] 1331.0-1332.0 sec  3.00 MBytes  25.2 Mbits/sec
> [  3] 1332.0-1333.0 sec  2.25 MBytes  18.9 Mbits/sec
> [  3] 1333.0-1334.0 sec  1.34 MBytes  11.2 Mbits/sec
> [  3] 1334.0-1335.0 sec  1.24 MBytes  10.4 Mbits/sec
> [  3] 1335.0-1336.0 sec  1.24 MBytes  10.4 Mbits/sec
> [  3] 1336.0-1337.0 sec  1.75 MBytes  14.7 Mbits/sec
> [  3] 1337.0-1338.0 sec  5.25 MBytes  44.0 Mbits/sec
> [  3] 1338.0-1339.0 sec  4.38 MBytes  36.7 Mbits/sec
> [  3] 1339.0-1340.0 sec  3.75 MBytes  31.5 Mbits/sec
> [  3] 1340.0-1341.0 sec  3.88 MBytes  32.5 Mbits/sec
>
> Whenever any issue occurs I don't see anything in my dmesg. Every time
> it happens it affects all of my 3 STAs (notebook + 2 smartphones).
>
> Waiting for few minutes brings things back to normal. I don't need any
> hostapd restart / device reboot.
>
> Stability problem was reported for this router multiple times by
> different users. Reports go back to the October 2020 when OpenWrt
> received that router support.
>
> https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/42
> https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/46
> https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/48
> https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/52
> https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/65
> https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/97
> https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/112
> https://forum.openwrt.org/t/openwrt-for-xiaomi-mi-router-4c/72175/113
>
> Is there anything I can do to help debugging / fixing this


Attachments:
iperf.txt (89.53 kB)
ping.txt (74.14 kB)
system.log.txt (7.58 kB)
Download all attachments

2021-07-06 22:35:07

by Rafał Miłecki

[permalink] [raw]
Subject: Re: Unstable WiFi with mt76 on MT7628AN

On 04.07.2021 09:36, Rafał Miłecki wrote:
> A small update (no good news).

Felix suggested that EEPROM access may be not setup correctly on my
device.


First, let me provide a dump of the "factory" MTD partition:

# hexdump -n 4096 -C /dev/mtdblock2
00000000 28 76 00 02 9c 9d 7e 58 10 56 00 00 00 00 00 00 |(v....~X.V......|
00000010 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
00000020 00 00 00 00 20 00 00 00 9c 9d 7e 58 10 55 00 0c |.... .....~X.U..|
00000030 43 e1 76 2a 22 34 00 20 ff ff 00 01 00 00 00 00 |C.v*"4. ........|
00000040 00 00 22 00 00 00 00 00 30 00 00 00 00 00 00 00 |..".....0.......|
00000050 80 00 00 94 40 ba c0 ca 16 84 82 82 40 ca 16 83 |....@.......@...|
00000060 83 83 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
000000a0 c0 c0 c0 c0 c0 c0 c0 c0 c0 c0 c0 c0 c0 c0 00 00 |................|
000000b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000c0 00 00 00 00 00 00 00 00 00 1a 22 2a 31 35 01 35 |.........."*15.5|
000000d0 39 40 46 4d 7f 7f 7f 00 00 00 00 00 00 00 00 00 |9@FM............|
000000e0 11 1d 11 1d 1c 35 1c 35 1e 35 1e 35 17 19 17 19 |.....5.5.5.5....|
000000f0 02 00 00 00 c3 00 c0 00 0a 00 00 00 00 00 00 00 |................|
00000100 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 77 00 |..............w.|
00000130 11 1d 11 1d 15 7f 15 7f 17 7f 17 7f 10 3b 10 3b |.............;.;|
00000140 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00001000


I modified mt7603_eeprom_load to dump EEPROM beginning make sure mt76
access exactly that data. It does:

[ 11.188447] [mt7603_eeprom_load] 28 76 00 02 9c 9d 7e 58 10 56 00 00 00 00 00 00
[ 11.195950] [mt7603_eeprom_load] ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff


Finally I dumped all known EEPROM entries to let you check them. I think
they don't look all random and may make some sense:

[ 11.204332] MT_EE_CHIP_ID: 28 76
[ 11.207598] MT_EE_VERSION: 00 02
[ 11.210875] MT_EE_MAC_ADDR: 9c 9d 7e 58 10 56
[ 11.215284] MT_EE_NIC_CONF_0: 22 34
[ 11.218823] MT_EE_NIC_CONF_1: 00 20
[ 11.222352] MT_EE_NIC_CONF_2: 22 00
[ 11.225881] MT_EE_XTAL_TRIM_1: 00
[ 11.229243] MT_EE_RSSI_OFFSET_2G: 00
[ 11.232859] MT_EE_WIFI_RF_SETTING: 30
[ 11.236564] MT_EE_RSSI_OFFSET_5G: 00
[ 11.240191] MT_EE_TX_POWER_DELTA_BW40: 80
[ 11.244249] MT_EE_TX_POWER_DELTA_BW80: 00
[ 11.248306] MT_EE_TX_POWER_EXT_PA_5G: 40
[ 11.252283] MT_EE_TEMP_SENSOR_CAL: ba
[ 11.255988] MT_EE_TX_POWER_0_START_2G: c0 ca 16 84 82 82
[ 11.261375] MT_EE_TX_POWER_1_START_2G: 40 ca 16 83 83 83
[ 11.266753] MT_EE_TX_POWER_0_START_5G: 00
[ 11.270820] MT_EE_TX_POWER_0_GRP3_TX_POWER_DELTA: 00
[ 11.275845] MT_EE_TX_POWER_0_GRP4_TSSI_SLOPE: 00
[ 11.280528] MT_EE_TX_POWER_1_START_5G: 00
[ 11.284584] MT_EE_TX_POWER_CCK: c0 c0
[ 11.288290] MT_EE_TX_POWER_OFDM_2G_6M: c0 c0
[ 11.292620] MT_EE_TX_POWER_OFDM_2G_24M: c0 c0
[ 11.297030] MT_EE_TX_POWER_OFDM_2G_54M: c0 c0
[ 11.301448] MT_EE_TX_POWER_HT_BPSK_QPSK: c0 c0
[ 11.305946] MT_EE_TX_POWER_HT_16_64_QAM: c0 c0
[ 11.310451] MT_EE_TX_POWER_HT_64_QAM: c0 c0
[ 11.314685] MT_EE_ELAN_RX_MODE_GAIN: 00
[ 11.318575] MT_EE_ELAN_RX_MODE_NF: 00
[ 11.322281] MT_EE_ELAN_RX_MODE_P1DB: 00
[ 11.326162] MT_EE_ELAN_BYPASS_MODE_GAIN: 00
[ 11.330404] MT_EE_ELAN_BYPASS_MODE_NF: 00
[ 11.334462] MT_EE_ELAN_BYPASS_MODE_P1DB: 00
[ 11.338705] MT_EE_STEP_NUM_NEG_6_7: 00 00
[ 11.342762] MT_EE_STEP_NUM_NEG_4_5: 00 1a
[ 11.346819] MT_EE_STEP_NUM_NEG_2_3: 22 2a
[ 11.350885] MT_EE_STEP_NUM_NEG_0_1: 31 35
[ 11.354942] MT_EE_REF_STEP_24G: 01 35
[ 11.358657] MT_EE_STEP_NUM_PLUS_1_2: 39 40
[ 11.362802] MT_EE_STEP_NUM_PLUS_3_4: 46 4d
[ 11.366947] MT_EE_STEP_NUM_PLUS_5_6: 7f 7f
[ 11.371101] MT_EE_STEP_NUM_PLUS_7: 7f
[ 11.374807] MT_EE_CP_FT_VERSION: 02
[ 11.378336] MT_EE_TX_POWER_TSSI_OFF: 00
[ 11.382231] MT_EE_XTAL_FREQ_OFFSET: c3
[ 11.386025] MT_EE_XTAL_TRIM_2_COMP: 00
[ 11.389828] MT_EE_XTAL_TRIM_3_COMP: c0
[ 11.393620] MT_EE_XTAL_WF_RFCAL: 00


My device has OTP but DTS file doesn't include
mediatek,eeprom-merge-otp;

If I decided to add that property it'd result in following changes:
[ 11.397156] [mt7603_apply_cal_free_data:121] [055] ba -> ba
[ 11.402810] [mt7603_apply_cal_free_data:121] [0f0] 02 -> 02
[ 11.408464] [mt7603_apply_cal_free_data:121] [0f4] c3 -> ba
[ 11.414109] [mt7603_apply_cal_free_data:121] [0f7] 00 -> 88

So it seems OTP contains different values for MT_EE_XTAL_FREQ_OFFSET and
MT_EE_XTAL_WF_RFCAL.


Any other ideas what may cause this unstable WiFi, how to debug it?


diff --git a/mt7603/eeprom.c b/mt7603/eeprom.c
index d951cb8..cc68085 100644
--- a/mt7603/eeprom.c
+++ b/mt7603/eeprom.c
@@ -118,6 +118,7 @@ mt7603_apply_cal_free_data(struct mt7603_dev *dev, u8 *efuse)
for (i = 0; i < n; i++) {
int offset = cal_free_bytes[i];

+ pr_info("[%s:%d] [%03x] %02x -> %02x\n", __func__, __LINE__, offset, eeprom[offset], efuse[offset]);
eeprom[offset] = efuse[offset];
}
}
@@ -131,6 +132,9 @@ mt7603_eeprom_load(struct mt7603_dev *dev)
if (ret < 0)
return ret;

+ pr_info("[%s] %*ph\n", __func__, 16, dev->mt76.eeprom.data);
+ pr_info("[%s] %*ph\n", __func__, 16, dev->mt76.eeprom.data + 16);
+
return mt7603_efuse_init(dev);
}

@@ -162,6 +166,56 @@ int mt7603_eeprom_init(struct mt7603_dev *dev)
if (ret < 0)
return ret;

+ eeprom = (u8 *)dev->mt76.eeprom.data;
+ pr_info("MT_EE_CHIP_ID:\t%*ph\n", 2, &eeprom[MT_EE_CHIP_ID]);
+ pr_info("MT_EE_VERSION:\t%*ph\n", 2, &eeprom[MT_EE_VERSION]);
+ pr_info("MT_EE_MAC_ADDR:\t%*ph\n", 6, &eeprom[MT_EE_MAC_ADDR]);
+ pr_info("MT_EE_NIC_CONF_0:\t%*ph\n", 2, &eeprom[MT_EE_NIC_CONF_0]);
+ pr_info("MT_EE_NIC_CONF_1:\t%*ph\n", 2, &eeprom[MT_EE_NIC_CONF_1]);
+ pr_info("MT_EE_NIC_CONF_2:\t%*ph\n", 2, &eeprom[MT_EE_NIC_CONF_2]);
+ pr_info("MT_EE_XTAL_TRIM_1:\t%*ph\n", 1, &eeprom[MT_EE_XTAL_TRIM_1]);
+ pr_info("MT_EE_RSSI_OFFSET_2G:\t%*ph\n", 1, &eeprom[MT_EE_RSSI_OFFSET_2G]);
+ pr_info("MT_EE_WIFI_RF_SETTING:\t%*ph\n", 1, &eeprom[MT_EE_WIFI_RF_SETTING]);
+ pr_info("MT_EE_RSSI_OFFSET_5G:\t%*ph\n", 1, &eeprom[MT_EE_RSSI_OFFSET_5G]);
+ pr_info("MT_EE_TX_POWER_DELTA_BW40:\t%*ph\n", 1, &eeprom[MT_EE_TX_POWER_DELTA_BW40]);
+ pr_info("MT_EE_TX_POWER_DELTA_BW80:\t%*ph\n", 1, &eeprom[MT_EE_TX_POWER_DELTA_BW80]);
+ pr_info("MT_EE_TX_POWER_EXT_PA_5G:\t%*ph\n", 1, &eeprom[MT_EE_TX_POWER_EXT_PA_5G]);
+ pr_info("MT_EE_TEMP_SENSOR_CAL:\t%*ph\n", 1, &eeprom[MT_EE_TEMP_SENSOR_CAL]);
+ pr_info("MT_EE_TX_POWER_0_START_2G:\t%*ph\n", 6, &eeprom[MT_EE_TX_POWER_0_START_2G]);
+ pr_info("MT_EE_TX_POWER_1_START_2G:\t%*ph\n", 6, &eeprom[MT_EE_TX_POWER_1_START_2G]);
+ pr_info("MT_EE_TX_POWER_0_START_5G:\t%*ph\n", 1, &eeprom[MT_EE_TX_POWER_0_START_5G]);
+ pr_info("MT_EE_TX_POWER_0_GRP3_TX_POWER_DELTA:\t%*ph\n", 1, &eeprom[MT_EE_TX_POWER_0_GRP3_TX_POWER_DELTA]);
+ pr_info("MT_EE_TX_POWER_0_GRP4_TSSI_SLOPE:\t%*ph\n", 1, &eeprom[MT_EE_TX_POWER_0_GRP4_TSSI_SLOPE]);
+ pr_info("MT_EE_TX_POWER_1_START_5G:\t%*ph\n", 1, &eeprom[MT_EE_TX_POWER_1_START_5G]);
+ pr_info("MT_EE_TX_POWER_CCK:\t%*ph\n", 2, &eeprom[MT_EE_TX_POWER_CCK]);
+ pr_info("MT_EE_TX_POWER_OFDM_2G_6M:\t%*ph\n", 2, &eeprom[MT_EE_TX_POWER_OFDM_2G_6M]);
+ pr_info("MT_EE_TX_POWER_OFDM_2G_24M:\t%*ph\n", 2, &eeprom[MT_EE_TX_POWER_OFDM_2G_24M]);
+ pr_info("MT_EE_TX_POWER_OFDM_2G_54M:\t%*ph\n", 2, &eeprom[MT_EE_TX_POWER_OFDM_2G_54M]);
+ pr_info("MT_EE_TX_POWER_HT_BPSK_QPSK:\t%*ph\n", 2, &eeprom[MT_EE_TX_POWER_HT_BPSK_QPSK]);
+ pr_info("MT_EE_TX_POWER_HT_16_64_QAM:\t%*ph\n", 2, &eeprom[MT_EE_TX_POWER_HT_16_64_QAM]);
+ pr_info("MT_EE_TX_POWER_HT_64_QAM:\t%*ph\n", 2, &eeprom[MT_EE_TX_POWER_HT_64_QAM]);
+ pr_info("MT_EE_ELAN_RX_MODE_GAIN:\t%*ph\n", 1, &eeprom[MT_EE_ELAN_RX_MODE_GAIN]);
+ pr_info("MT_EE_ELAN_RX_MODE_NF:\t%*ph\n", 1, &eeprom[MT_EE_ELAN_RX_MODE_NF]);
+ pr_info("MT_EE_ELAN_RX_MODE_P1DB:\t%*ph\n", 1, &eeprom[MT_EE_ELAN_RX_MODE_P1DB]);
+ pr_info("MT_EE_ELAN_BYPASS_MODE_GAIN:\t%*ph\n", 1, &eeprom[MT_EE_ELAN_BYPASS_MODE_GAIN]);
+ pr_info("MT_EE_ELAN_BYPASS_MODE_NF:\t%*ph\n", 1, &eeprom[MT_EE_ELAN_BYPASS_MODE_NF]);
+ pr_info("MT_EE_ELAN_BYPASS_MODE_P1DB:\t%*ph\n", 1, &eeprom[MT_EE_ELAN_BYPASS_MODE_P1DB]);
+ pr_info("MT_EE_STEP_NUM_NEG_6_7:\t%*ph\n", 2, &eeprom[MT_EE_STEP_NUM_NEG_6_7]);
+ pr_info("MT_EE_STEP_NUM_NEG_4_5:\t%*ph\n", 2, &eeprom[MT_EE_STEP_NUM_NEG_4_5]);
+ pr_info("MT_EE_STEP_NUM_NEG_2_3:\t%*ph\n", 2, &eeprom[MT_EE_STEP_NUM_NEG_2_3]);
+ pr_info("MT_EE_STEP_NUM_NEG_0_1:\t%*ph\n", 2, &eeprom[MT_EE_STEP_NUM_NEG_0_1]);
+ pr_info("MT_EE_REF_STEP_24G:\t%*ph\n", 2, &eeprom[MT_EE_REF_STEP_24G]);
+ pr_info("MT_EE_STEP_NUM_PLUS_1_2:\t%*ph\n", 2, &eeprom[MT_EE_STEP_NUM_PLUS_1_2]);
+ pr_info("MT_EE_STEP_NUM_PLUS_3_4:\t%*ph\n", 2, &eeprom[MT_EE_STEP_NUM_PLUS_3_4]);
+ pr_info("MT_EE_STEP_NUM_PLUS_5_6:\t%*ph\n", 2, &eeprom[MT_EE_STEP_NUM_PLUS_5_6]);
+ pr_info("MT_EE_STEP_NUM_PLUS_7:\t%*ph\n", 1, &eeprom[MT_EE_STEP_NUM_PLUS_7]);
+ pr_info("MT_EE_CP_FT_VERSION:\t%*ph\n", 1, &eeprom[MT_EE_CP_FT_VERSION]);
+ pr_info("MT_EE_TX_POWER_TSSI_OFF:\t%*ph\n", 1, &eeprom[MT_EE_TX_POWER_TSSI_OFF]);
+ pr_info("MT_EE_XTAL_FREQ_OFFSET:\t%*ph\n", 1, &eeprom[MT_EE_XTAL_FREQ_OFFSET]);
+ pr_info("MT_EE_XTAL_TRIM_2_COMP:\t%*ph\n", 1, &eeprom[MT_EE_XTAL_TRIM_2_COMP]);
+ pr_info("MT_EE_XTAL_TRIM_3_COMP:\t%*ph\n", 1, &eeprom[MT_EE_XTAL_TRIM_3_COMP]);
+ pr_info("MT_EE_XTAL_WF_RFCAL:\t%*ph\n", 1, &eeprom[MT_EE_XTAL_WF_RFCAL]);
+
if (dev->mt76.otp.data) {
if (mt7603_check_eeprom(&dev->mt76) == 0)
mt7603_apply_cal_free_data(dev, dev->mt76.otp.data);

2021-07-07 21:52:37

by Rafał Miłecki

[permalink] [raw]
Subject: Re: Unstable WiFi with mt76 on MT7628AN

On 04.07.2021 00:19, Rafał Miłecki wrote:
> It may be worth mentioning I found 1 WARNING in my dmesg. I can't tell
> if that has happened during any slow down / stall or not. Unforunately I
> don't have symbols in my backtrace.
>
> [ 3245.170684] ------------[ cut here ]------------
> [ 3245.175434] WARNING: CPU: 0 PID: 627 at target-mipsel_24kc_musl/linux-ramips_mt76x8/mt76-2021-05-15-9d736545/mt7603/mac.c:206 0x835e44ac [mt7603e@bb8c9e4c+0x9520]

I've installed OpenWrt snapshot with debug symbols included. I
reproduced the WARNING. It doesn't seem to be related to slow downs /
stalls / no beacons.

[ 3090.333926] ------------[ cut here ]------------
[ 3090.338714] WARNING: CPU: 0 PID: 118 at target-mipsel_24kc_musl/linux-ramips_mt76x8/mt76-2021-06-06-22b69033/mt7603/mac.c:206 mt7603_filter_tx+0x16c/0x174 [mt7603e]
[ 3090.353623] Modules linked in: pppoe ppp_async iptable_nat xt_state xt_nat xt_conntrack xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD xt_CT pppox ppp_generic nf_nat nf_flow_table_hw nf_flow_table nf_conntrack mt7603e mt76 mac80211 ipt_REJECT cfg80211 xt_time xt_tcpudp xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG slhc nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_filter ip_tables crc_ccitt compat nf_log_ipv6 nf_log_common ip6table_mangle
ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 sha256_generic libsha256 seqiv jitterentropy_rng drbg hmac ghash_generic gf128mul gcm ctr cmac ccm leds_gpio gpio_button_hotplug aead cryptomgr crypto_null crypto_hash
[ 3090.417635] CPU: 0 PID: 118 Comm: kworker/u3:0 Not tainted 5.4.128 #0
[ 3090.424186] Workqueue: napi_workq napi_workfn
[ 3090.428616] Stack : 80589abc 835744ac 00000009 00000000 00000000 8005b100 83d64cc0 00000000
[ 3090.437106] 00000017 83c98dbc 8057a6b8 83de5bec 80620000 00000001 83de5bc0 601c1e37
[ 3090.445583] 00000000 00000000 806393c0 00000000 716b726f 000000a0 6f775f69 6e666b72
[ 3090.454070] 00000000 00000099 00000000 000678fa 00000000 00000009 00000000 835744ac
[ 3090.462557] 00000009 00000000 00000000 83580000 00000000 802bbac8 00000000 80780000
[ 3090.471046] ...
[ 3090.473522] Call Trace:
[ 3090.476017] [<80009c90>] show_stack+0x30/0x100
[ 3090.480542] [<80023ba0>] __warn+0xc0/0x10c
[ 3090.484693] [<80023c48>] warn_slowpath_fmt+0x5c/0xac
[ 3090.489754] [<835744ac>] mt7603_filter_tx+0x16c/0x174 [mt7603e]
[ 3090.495767] [<83574644>] mt7603_wtbl_set_ps+0x12c/0x134 [mt7603e]
[ 3090.501973] [<83571a1c>] mt7603_sta_ps+0x38/0x428 [mt7603e]
[ 3090.507669] [<835547f4>] mt76_rx_poll_complete+0x2e0/0x5b0 [mt76]
[ 3090.513860] [<8355153c>] mt76_dma_rx_poll+0x3d8/0xefc [mt76]
[ 3090.519634] [<8035b3dc>] __napi_poll+0x3c/0x10c
[ 3090.524227] [<8035b4ec>] napi_workfn+0x40/0x90
[ 3090.528751] [<8003b0e8>] process_one_work+0x220/0x480
[ 3090.533878] [<8003b4b0>] worker_thread+0x168/0x5c8
[ 3090.538759] [<800401ec>] kthread+0x138/0x140
[ 3090.543088] [<80005178>] ret_from_kernel_thread+0x14/0x1c
[ 3090.548577] ---[ end trace 068a759ed1654b52 ]---

2021-07-10 00:17:22

by Rafał Miłecki

[permalink] [raw]
Subject: Re: Unstable WiFi with mt76 on MT7628AN

On 04.07.2021 00:16, Rafał Miłecki wrote:
> I decided to start my adventure with mt76. I bought the cheapest router
> I could find: Xiaomi Mi Router 4C and installed OpenWrt 21.02-rc3 on it.

I found & bought another cheap router with mt76 supported WiFi: Netgear
R6220 with MT7621ST (SoC) + MT7603EN (WiFi) + MT7612EN (WiFi). I even
found some possitive comments on mt76 support for MT7603 but it appears
it's the same story. I decided to focus on 2 GHz (MT7603) for now.

***

First I run vendor's firmware and iperf test for 2 hours. Speeds were
verying between 35 Mb/s and 60 Mb/s but it didn't fail even once:

[ 3] 7195.0-7196.0 sec 4.12 MBytes 34.6 Mbits/sec
[ 3] 7196.0-7197.0 sec 2.54 MBytes 21.3 Mbits/sec
[ 3] 7197.0-7198.0 sec 5.50 MBytes 46.1 Mbits/sec
[ 3] 7198.0-7199.0 sec 5.62 MBytes 47.2 Mbits/sec
[ 3] 7199.0-7200.0 sec 5.50 MBytes 46.1 Mbits/sec
[ 3] 0.0-7200.3 sec 37.2 GBytes 44.4 Mbits/sec

***

Then I installed OpenWrt 19.07.7 with mt76 from February 2021. It failed
me almost immediately:

[ 3] 0.0- 1.0 sec 6.50 MBytes 54.5 Mbits/sec
[ 3] 1.0- 2.0 sec 5.38 MBytes 45.1 Mbits/sec
[ 3] 2.0- 3.0 sec 5.88 MBytes 49.3 Mbits/sec
[ 3] 3.0- 4.0 sec 5.12 MBytes 43.0 Mbits/sec
[ 3] 4.0- 5.0 sec 6.38 MBytes 53.5 Mbits/sec
[ 3] 5.0- 6.0 sec 5.88 MBytes 49.3 Mbits/sec
[ 3] 6.0- 7.0 sec 6.50 MBytes 54.5 Mbits/sec
[ 3] 7.0- 8.0 sec 5.88 MBytes 49.3 Mbits/sec
[ 3] 8.0- 9.0 sec 6.88 MBytes 57.7 Mbits/sec
[ 3] 9.0-10.0 sec 6.12 MBytes 51.4 Mbits/sec
[ 3] 10.0-11.0 sec 6.50 MBytes 54.5 Mbits/sec
[ 3] 11.0-12.0 sec 5.75 MBytes 48.2 Mbits/sec
[ 3] 12.0-13.0 sec 4.38 MBytes 36.7 Mbits/sec
[ 3] 13.0-14.0 sec 1.98 MBytes 16.6 Mbits/sec
[ 3] 14.0-15.0 sec 1.80 MBytes 15.1 Mbits/sec
[ 3] 15.0-16.0 sec 1.74 MBytes 14.6 Mbits/sec
[ 3] 16.0-17.0 sec 509 KBytes 4.17 Mbits/sec
[ 3] 17.0-18.0 sec 509 KBytes 4.17 Mbits/sec
[ 3] 18.0-19.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 19.0-20.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 20.0-21.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 21.0-22.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 22.0-23.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 23.0-24.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 24.0-25.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 25.0-26.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 26.0-27.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 27.0-28.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 28.0-29.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 29.0-30.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 30.0-31.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 31.0-32.0 sec 2.38 MBytes 19.9 Mbits/sec
[ 3] 32.0-33.0 sec 5.62 MBytes 47.2 Mbits/sec
[ 3] 33.0-34.0 sec 6.75 MBytes 56.6 Mbits/sec
[ 3] 34.0-35.0 sec 5.88 MBytes 49.3 Mbits/sec

The most recent OpenWrt with mt76 from June 2021 was no better.

***

Then I focused on iw & station info from MT7603 perspective. That is
where I noticed something interesting. With idle wireless (no real
traffic at all) I could see "rx bitrate" reported by MT7603 for my STA
to drop down to 6 Mbit/s regularly.

> while [ 1 ]; do date; iw dev wlan0 station dump | grep bitrate; sleep 1; done
Fri Jul 9 16:40:42 UTC 2021
tx bitrate: 72.2 MBit/s MCS 7 short GI
rx bitrate: 72.2 MBit/s MCS 7 short GI
Fri Jul 9 16:40:43 UTC 2021
tx bitrate: 72.2 MBit/s MCS 7 short GI
rx bitrate: 72.2 MBit/s MCS 7 short GI
Fri Jul 9 16:40:44 UTC 2021
tx bitrate: 72.2 MBit/s MCS 7 short GI
rx bitrate: 72.2 MBit/s MCS 7 short GI
Fri Jul 9 16:40:45 UTC 2021
tx bitrate: 72.2 MBit/s MCS 7 short GI
rx bitrate: 6.0 MBit/s
Fri Jul 9 16:40:46 UTC 2021
tx bitrate: 39.0 MBit/s MCS 4
rx bitrate: 6.0 MBit/s
Fri Jul 9 16:40:47 UTC 2021
tx bitrate: 26.0 MBit/s MCS 3
rx bitrate: 6.0 MBit/s
Fri Jul 9 16:40:48 UTC 2021
tx bitrate: 72.2 MBit/s MCS 7 short GI
rx bitrate: 72.2 MBit/s MCS 7 short GI
Fri Jul 9 16:40:49 UTC 2021
tx bitrate: 72.2 MBit/s MCS 7 short GI
rx bitrate: 72.2 MBit/s MCS 7 short GI
Fri Jul 9 16:40:50 UTC 2021
tx bitrate: 72.2 MBit/s MCS 7 short GI
rx bitrate: 72.2 MBit/s MCS 7 short GI

Then I started iperf and noticed that iperf slows down exactly when "iw"
reports lower (6 Mbit/s) "rx rate".

I decided to run iperf with "-b 20M" and got something like:

[ 3] 465.0-466.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 466.0-467.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 467.0-468.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 468.0-469.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 469.0-470.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 470.0-471.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 471.0-472.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 472.0-473.0 sec 1.08 MBytes 9.05 Mbits/sec ← "iw" on MT7603 reported lower "rx bitrate" at that moment
[ 3] 473.0-474.0 sec 1.55 MBytes 13.0 Mbits/sec
[ 3] 474.0-475.0 sec 1.37 MBytes 11.5 Mbits/sec
[ 3] 475.0-476.0 sec 1.18 MBytes 9.90 Mbits/sec
[ 3] 476.0-477.0 sec 954 KBytes 7.82 Mbits/sec
[ 3] 477.0-478.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 478.0-479.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 479.0-480.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 480.0-481.0 sec 0.00 Bytes 0.00 bits/sec
[ 3] 481.0-482.0 sec 1.06 MBytes 8.86 Mbits/sec
[ 3] 482.0-483.0 sec 255 KBytes 2.09 Mbits/sec
[ 3] 483.0-484.0 sec 954 KBytes 7.82 Mbits/sec
[ 3] 484.0-485.0 sec 1.93 MBytes 16.2 Mbits/sec
[ 3] 485.0-486.0 sec 3.36 MBytes 28.2 Mbits/sec
[ 3] 486.0-487.0 sec 6.96 MBytes 58.4 Mbits/sec
[ 3] 487.0-488.0 sec 6.90 MBytes 57.9 Mbits/sec
[ 3] 488.0-489.0 sec 6.96 MBytes 58.4 Mbits/sec
[ 3] 489.0-490.0 sec 5.78 MBytes 48.5 Mbits/sec
[ 3] 490.0-491.0 sec 5.07 MBytes 42.5 Mbits/sec
[ 3] 491.0-492.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 492.0-493.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 493.0-494.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 494.0-495.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 495.0-496.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 496.0-497.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 497.0-498.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 498.0-499.0 sec 2.38 MBytes 20.0 Mbits/sec
[ 3] 499.0-500.0 sec 2.38 MBytes 20.0 Mbits/sec

For few times iperf speed recovered after slow down (as in above log)
but after few successes it has failed and slow down ended up in a
totally stalled traffic.

Then I decided to run iperf with "-b 10M" and got a kind of success:

[ 3] 16590.0-16591.0 sec 1.19 MBytes 10.0 Mbits/sec
[ 3] 16591.0-16592.0 sec 1.19 MBytes 10.0 Mbits/sec
[ 3] 16592.0-16593.0 sec 1.19 MBytes 10.0 Mbits/sec
[ 3] 16593.0-16594.0 sec 1.19 MBytes 10.0 Mbits/sec
[ 3] 16594.0-16595.0 sec 1.19 MBytes 10.0 Mbits/sec
[ 3] 16595.0-16596.0 sec 1.19 MBytes 10.0 Mbits/sec
[ 3] 16596.0-16597.0 sec 1.19 MBytes 10.0 Mbits/sec
[ 3] 16597.0-16598.0 sec 1.19 MBytes 10.0 Mbits/sec
[ 3] 16598.0-16599.0 sec 1.19 MBytes 10.0 Mbits/sec
[ 3] 16599.0-16600.0 sec 1.19 MBytes 10.0 Mbits/sec
[ 3] 16600.0-16601.0 sec 1.19 MBytes 10.0 Mbits/sec
[ 3] 0.0-16601.3 sec 19.3 GBytes 10.0 Mbits/sec

I had iperf running for 4,5 hours with only 1 slow down but that was
very stable after all and I reached a record of stable WiFi time!

***

So it seems there are two problems I managed to narrown down:

1. On MT7603 "rx bitrate" of STA drops 72.2 → 6 MBit/s regularly
2. When catching up with traffic (after slow down) WiFi occasionally stalls

2021-07-10 16:55:20

by Rafał Miłecki

[permalink] [raw]
Subject: Re: Unstable WiFi with mt76 on MT7628AN

On 10.07.2021 02:16, Rafał Miłecki wrote:
> So it seems there are two problems I managed to narrown down:
>
> 1. On MT7603 "rx bitrate" of STA drops 72.2 → 6 MBit/s regularly
> 2. When catching up with traffic (after slow down) WiFi occasionally stalls

Today I decided to use my Netgear R6220's 5 GHz (MT7612EN).

On that chipset "rx bitrate" reported for my STA stays between 292 and
390 for most of them time BUT is also regularly drops to 6 Mbps:

> while [ 1 ]; do date; iw dev wlan1 station dump | grep bitrate; sleep 1; done

Sat Jul 10 15:00:27 UTC 2021
tx bitrate: 292.5 MBit/s VHT-MCS 7 80MHz VHT-NSS 1
rx bitrate: 390.0 MBit/s VHT-MCS 9 80MHz VHT-NSS 1
Sat Jul 10 15:00:28 UTC 2021
tx bitrate: 433.3 MBit/s VHT-MCS 9 80MHz short GI VHT-NSS 1
rx bitrate: 390.0 MBit/s VHT-MCS 8 80MHz short GI VHT-NSS 1
Sat Jul 10 15:00:29 UTC 2021
tx bitrate: 433.3 MBit/s VHT-MCS 9 80MHz short GI VHT-NSS 1
rx bitrate: 325.0 MBit/s VHT-MCS 7 80MHz short GI VHT-NSS 1
Sat Jul 10 15:00:30 UTC 2021
tx bitrate: 390.0 MBit/s VHT-MCS 9 80MHz VHT-NSS 1
rx bitrate: 6.0 MBit/s
Sat Jul 10 15:00:32 UTC 2021
tx bitrate: 292.5 MBit/s VHT-MCS 7 80MHz VHT-NSS 1
rx bitrate: 6.0 MBit/s
Sat Jul 10 15:00:33 UTC 2021
tx bitrate: 390.0 MBit/s VHT-MCS 8 80MHz short GI VHT-NSS 1
rx bitrate: 325.0 MBit/s VHT-MCS 7 80MHz short GI VHT-NSS 1
Sat Jul 10 15:00:34 UTC 2021
tx bitrate: 292.5 MBit/s VHT-MCS 7 80MHz VHT-NSS 1
rx bitrate: 325.0 MBit/s VHT-MCS 7 80MHz short GI VHT-NSS 1
Sat Jul 10 15:00:35 UTC 2021
tx bitrate: 390.0 MBit/s VHT-MCS 8 80MHz short GI VHT-NSS 1
rx bitrate: 325.0 MBit/s VHT-MCS 7 80MHz short GI VHT-NSS 1
Sat Jul 10 15:00:36 UTC 2021
tx bitrate: 390.0 MBit/s VHT-MCS 9 80MHz VHT-NSS 1
rx bitrate: 325.0 MBit/s VHT-MCS 7 80MHz short GI VHT-NSS 1

Whenever it drops to 6 MBit/s I can see iperf slowing down from 150 -
180 Mb/s down to 100 - 130 Mb/s;

[ 3] 19148.0-19149.0 sec 19.1 MBytes 160 Mbits/sec
[ 3] 19149.0-19150.0 sec 21.9 MBytes 184 Mbits/sec
[ 3] 19150.0-19151.0 sec 18.6 MBytes 156 Mbits/sec
[ 3] 19151.0-19152.0 sec 21.2 MBytes 178 Mbits/sec
[ 3] 19152.0-19153.0 sec 15.6 MBytes 131 Mbits/sec ← "iw" on MT7612EN reported 6 Mb/s "rx bitrate" at that moment
[ 3] 19153.0-19154.0 sec 18.8 MBytes 157 Mbits/sec
[ 3] 19154.0-19155.0 sec 20.0 MBytes 168 Mbits/sec
[ 3] 19155.0-19156.0 sec 20.4 MBytes 171 Mbits/sec
[ 3] 19156.0-19157.0 sec 19.6 MBytes 165 Mbits/sec
[ 3] 19157.0-19158.0 sec 20.2 MBytes 170 Mbits/sec
[ 3] 19158.0-19159.0 sec 21.6 MBytes 181 Mbits/sec

(...)

[ 3] 19207.0-19208.0 sec 18.4 MBytes 154 Mbits/sec
[ 3] 19208.0-19209.0 sec 18.1 MBytes 152 Mbits/sec
[ 3] 19209.0-19210.0 sec 17.6 MBytes 148 Mbits/sec
[ 3] 19210.0-19211.0 sec 18.6 MBytes 156 Mbits/sec
[ 3] 19211.0-19212.0 sec 22.2 MBytes 187 Mbits/sec
[ 3] 19212.0-19213.0 sec 15.1 MBytes 127 Mbits/sec ← "iw" on MT7612EN reported 6 Mb/s "rx bitrate" at that moment
[ 3] 19213.0-19214.0 sec 16.9 MBytes 142 Mbits/sec
[ 3] 19214.0-19215.0 sec 17.1 MBytes 144 Mbits/sec
[ 3] 19215.0-19216.0 sec 20.2 MBytes 170 Mbits/sec
[ 3] 19216.0-19217.0 sec 21.1 MBytes 177 Mbits/sec
[ 3] 19217.0-19218.0 sec 21.6 MBytes 181 Mbits/sec

(...)

[ 3] 19267.0-19268.0 sec 15.9 MBytes 133 Mbits/sec
[ 3] 19268.0-19269.0 sec 18.2 MBytes 153 Mbits/sec
[ 3] 19269.0-19270.0 sec 17.2 MBytes 145 Mbits/sec
[ 3] 19270.0-19271.0 sec 17.0 MBytes 143 Mbits/sec
[ 3] 19271.0-19272.0 sec 18.2 MBytes 153 Mbits/sec
[ 3] 19272.0-19273.0 sec 15.2 MBytes 128 Mbits/sec ← "iw" on MT7612EN reported 6 Mb/s "rx bitrate" at that moment
[ 3] 19273.0-19274.0 sec 14.0 MBytes 117 Mbits/sec ← "iw" on MT7612EN reported 6 Mb/s "rx bitrate" at that moment
[ 3] 19274.0-19275.0 sec 18.9 MBytes 158 Mbits/sec
[ 3] 19275.0-19276.0 sec 16.9 MBytes 142 Mbits/sec
[ 3] 19276.0-19277.0 sec 18.6 MBytes 156 Mbits/sec
[ 3] 19277.0-19278.0 sec 16.2 MBytes 136 Mbits/sec
[ 3] 19278.0-19279.0 sec 23.9 MBytes 200 Mbits/sec
[ 3] 19279.0-19280.0 sec 21.1 MBytes 177 Mbits/sec

(...)

[ 3] 19297.0-19298.0 sec 21.6 MBytes 181 Mbits/sec
[ 3] 19298.0-19299.0 sec 19.9 MBytes 167 Mbits/sec
[ 3] 19299.0-19300.0 sec 22.9 MBytes 192 Mbits/sec
[ 3] 19300.0-19301.0 sec 21.4 MBytes 179 Mbits/sec
[ 3] 19301.0-19302.0 sec 19.6 MBytes 165 Mbits/sec

[ 3] 19302.0-19303.0 sec 15.0 MBytes 126 Mbits/sec ← "iw" on MT7612EN reported 6 Mb/s "rx bitrate" at that moment
[ 3] 19303.0-19304.0 sec 11.0 MBytes 92.3 Mbits/sec
[ 3] 19304.0-19305.0 sec 11.2 MBytes 94.4 Mbits/sec

[ 3] 19305.0-19306.0 sec 20.1 MBytes 169 Mbits/sec
[ 3] 19306.0-19307.0 sec 18.4 MBytes 154 Mbits/sec
[ 3] 19307.0-19308.0 sec 21.1 MBytes 177 Mbits/sec
[ 3] 19308.0-19309.0 sec 22.6 MBytes 190 Mbits/sec
[ 3] 19309.0-19310.0 sec 21.5 MBytes 180 Mbits/sec
[ 3] 19310.0-19311.0 sec 20.0 MBytes 168 Mbits/sec

In 5 GHz case however mt76 successfully recovers from all those slow
downs. Traffic didn't stop for good not even once. I had it running
for over 5 hours:

[ 3] 0.0-19330.1 sec 352 GBytes 157 Mbits/sec