2022-02-28 10:49:47

by Wen Gong

[permalink] [raw]
Subject: [PATCH v8 0/4] ath11k: add feature for device recovery

v8: change "fail_cont_count = atomic_inc_return(&ab->fail_cont_count)" to "atomic_inc(&ab->fail_cont_count)" in
"ath11k: add support for device recovery for QCA6390/WCN6855"

v7:
1. rebased to ath.git ath-202202211030
2. add patch "ath11k: fix the warning of dev_wake in mhi_pm_disable_transition()"
3. remove below 3 patch which has upstream.
ath11k: add ath11k_qmi_free_resource() for recovery
ath11k: configure RDDM size to mhi for recovery by firmware
ath11k: add support for device recovery for QCA6390/WCN6855

v6: add patch "ath11k: Add hw-restart option to simulate_fw_crash"

v5:
1. rebased to ath.git ath-202202030905
2. change a few commit message
3. fix count set sequence of ath11k_core_reset in "ath11k: add synchronization operation between reconfigure of mac80211 and ath11k_base"
4. add patch "ath11k: configure RDDM size to mhi for recovery by firmware" to support RDDM
5. move destroy_workqueue(ab->workqueue_aux) from ath11k_pci_remove() to ath11k_core_free()

v4: add patch "ath11k: fix invalid m3 buffer address"
recovery will fail when download firmware without this patch

v3: remove time_left set but not used in
"ath11k: add synchronization operation between reconfigure of mac80211 and ath11k_base"

v2: s/initilized/initialized in commit log of patch
"ath11k: add synchronization operation between reconfigure of mac80211 and ath11k_base"

Add support for device recovery.

Wen Gong (4):
ath11k: add support for device recovery for QCA6390/WCN6855
ath11k: add synchronization operation between reconfigure of mac80211
and ath11k_base
ath11k: Add hw-restart option to simulate_fw_crash
ath11k: fix the warning of dev_wake in mhi_pm_disable_transition()

drivers/net/wireless/ath/ath11k/core.c | 121 ++++++++++++++++++++--
drivers/net/wireless/ath/ath11k/core.h | 18 ++++
drivers/net/wireless/ath/ath11k/debugfs.c | 4 +
drivers/net/wireless/ath/ath11k/mac.c | 40 +++++++
drivers/net/wireless/ath/ath11k/mhi.c | 33 ++++++
drivers/net/wireless/ath/ath11k/pci.c | 12 ++-
6 files changed, 216 insertions(+), 12 deletions(-)


base-commit: 193dc0446985372a9b06b7572e1b98e1963c23a9
--
2.31.1


2022-02-28 10:51:30

by Wen Gong

[permalink] [raw]
Subject: [PATCH v8 3/4] ath11k: Add hw-restart option to simulate_fw_crash

Add hw-restart to directly restart wlan. Like UTF mode start it will
restart hardware and download firmware again.

Usage:
1. Run command:
echo hw-restart > /sys/kernel/debug/ath11k/qca6390\ hw2.0/simulate_fw_crash
echo hw-restart > /sys/kernel/debug/ath11k/wcn6855\ hw2.0/simulate_fw_crash
2. wlan will be restart and do recovery process and success.

Tested-on: WCN6855 hw2.0 PCI WLAN.HSP.1.1-03003-QCAHSPSWPL_V1_V2_SILICONZ_LITE-2

Signed-off-by: Wen Gong <[email protected]>
---
drivers/net/wireless/ath/ath11k/debugfs.c | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/drivers/net/wireless/ath/ath11k/debugfs.c b/drivers/net/wireless/ath/ath11k/debugfs.c
index c0ebdb1262f0..7375defb0f23 100644
--- a/drivers/net/wireless/ath/ath11k/debugfs.c
+++ b/drivers/net/wireless/ath/ath11k/debugfs.c
@@ -557,6 +557,10 @@ static ssize_t ath11k_write_simulate_fw_crash(struct file *file,
ret = ath11k_wmi_force_fw_hang_cmd(ar,
ATH11K_WMI_FW_HANG_ASSERT_TYPE,
ATH11K_WMI_FW_HANG_DELAY);
+ } else if (!strcmp(buf, "hw-restart")) {
+ ath11k_info(ab, "user requested hw restart\n");
+ queue_work(ab->workqueue_aux, &ab->reset_work);
+ ret = 0;
} else {
ret = -EINVAL;
goto exit;
--
2.31.1

2022-02-28 12:06:54

by Wen Gong

[permalink] [raw]
Subject: [PATCH v8 4/4] ath11k: fix the warning of dev_wake in mhi_pm_disable_transition()

When test device recovery with below command, it has warning in message
as below.
echo assert > /sys/kernel/debug/ath11k/wcn6855\ hw2.0/simulate_fw_crash
echo assert > /sys/kernel/debug/ath11k/qca6390\ hw2.0/simulate_fw_crash

warning message:
[ 1965.642121] ath11k_pci 0000:06:00.0: simulating firmware assert crash
[ 1968.471364] ieee80211 phy0: Hardware restart was requested
[ 1968.511305] ------------[ cut here ]------------
[ 1968.511368] WARNING: CPU: 3 PID: 1546 at drivers/bus/mhi/core/pm.c:505 mhi_pm_disable_transition+0xb37/0xda0 [mhi]
[ 1968.511443] Modules linked in: ath11k_pci ath11k mac80211 libarc4 cfg80211 qmi_helpers qrtr_mhi mhi qrtr nvme nvme_core
[ 1968.511563] CPU: 3 PID: 1546 Comm: kworker/u17:0 Kdump: loaded Tainted: G W 5.17.0-rc3-wt-ath+ #579
[ 1968.511629] Hardware name: Intel(R) Client Systems NUC8i7HVK/NUC8i7HVB, BIOS HNKBLi70.86A.0067.2021.0528.1339 05/28/2021
[ 1968.511704] Workqueue: mhi_hiprio_wq mhi_pm_st_worker [mhi]
[ 1968.511787] RIP: 0010:mhi_pm_disable_transition+0xb37/0xda0 [mhi]
[ 1968.511870] Code: a9 fe ff ff 4c 89 ff 44 89 04 24 e8 03 46 f6 e5 44 8b 04 24 41 83 f8 01 0f 84 21 fe ff ff e9 4c fd ff ff 0f 0b e9 af f8 ff ff <0f> 0b e9 5c f8 ff ff 48 89 df e8 da 9e ee e3 e9 12 fd ff ff 4c 89
[ 1968.511923] RSP: 0018:ffffc900024efbf0 EFLAGS: 00010286
[ 1968.511969] RAX: 00000000ffffffff RBX: ffff88811d241250 RCX: ffffffffc0176922
[ 1968.512014] RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffff888118a90a24
[ 1968.512059] RBP: ffff888118a90800 R08: 0000000000000000 R09: ffff888118a90a27
[ 1968.512102] R10: ffffed1023152144 R11: 0000000000000001 R12: ffff888118a908ac
[ 1968.512229] R13: ffff888118a90928 R14: dffffc0000000000 R15: ffff888118a90a24
[ 1968.512310] FS: 0000000000000000(0000) GS:ffff888234200000(0000) knlGS:0000000000000000
[ 1968.512405] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1968.512493] CR2: 00007f5538f443a8 CR3: 000000016dc28001 CR4: 00000000003706e0
[ 1968.512587] Call Trace:
[ 1968.512672] <TASK>
[ 1968.512751] ? _raw_spin_unlock_irq+0x1f/0x40
[ 1968.512859] mhi_pm_st_worker+0x3ac/0x790 [mhi]
[ 1968.512959] ? mhi_pm_mission_mode_transition.isra.0+0x7d0/0x7d0 [mhi]
[ 1968.513063] process_one_work+0x86a/0x1400
[ 1968.513184] ? pwq_dec_nr_in_flight+0x230/0x230
[ 1968.513312] ? move_linked_works+0x125/0x290
[ 1968.513416] worker_thread+0x6db/0xf60
[ 1968.513536] ? process_one_work+0x1400/0x1400
[ 1968.513627] kthread+0x241/0x2d0
[ 1968.513733] ? kthread_complete_and_exit+0x20/0x20
[ 1968.513821] ret_from_fork+0x22/0x30
[ 1968.513924] </TASK>

Reason is mhi_deassert_dev_wake() from mhi_device_put() is called
but mhi_assert_dev_wake() from __mhi_device_get_sync() is not called
in progress of recovery. Commit 8e0559921f9a ("bus: mhi: core:
Skip device wake in error or shutdown state") add check for the
pm_state of mhi in __mhi_device_get_sync(), and the pm_state is not
the normal state untill recovery is completed, so it leads the
dev_wake is not 0 and above warning print in mhi_pm_disable_transition()
while checking mhi_cntrl->dev_wake.

Add check in ath11k_pci_write32()/ath11k_pci_read32() to skip call
mhi_device_put() if mhi_device_get_sync() does not really do wake,
then the warning gone.

Tested-on: WCN6855 hw2.0 PCI WLAN.HSP.1.1-03003-QCAHSPSWPL_V1_V2_SILICONZ_LITE-2

Signed-off-by: Wen Gong <[email protected]>
---
drivers/net/wireless/ath/ath11k/pci.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/net/wireless/ath/ath11k/pci.c b/drivers/net/wireless/ath/ath11k/pci.c
index 903758751c99..8a3ff12057e8 100644
--- a/drivers/net/wireless/ath/ath11k/pci.c
+++ b/drivers/net/wireless/ath/ath11k/pci.c
@@ -191,6 +191,7 @@ void ath11k_pci_write32(struct ath11k_base *ab, u32 offset, u32 value)
{
struct ath11k_pci *ab_pci = ath11k_pci_priv(ab);
u32 window_start;
+ int ret = 0;

/* for offset beyond BAR + 4K - 32, may
* need to wakeup MHI to access.
@@ -198,7 +199,7 @@ void ath11k_pci_write32(struct ath11k_base *ab, u32 offset, u32 value)
if (ab->hw_params.wakeup_mhi &&
test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
offset >= ACCESS_ALWAYS_OFF)
- mhi_device_get_sync(ab_pci->mhi_ctrl->mhi_dev);
+ ret = mhi_device_get_sync(ab_pci->mhi_ctrl->mhi_dev);

if (offset < WINDOW_START) {
iowrite32(value, ab->mem + offset);
@@ -222,7 +223,8 @@ void ath11k_pci_write32(struct ath11k_base *ab, u32 offset, u32 value)

if (ab->hw_params.wakeup_mhi &&
test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
- offset >= ACCESS_ALWAYS_OFF)
+ offset >= ACCESS_ALWAYS_OFF &&
+ !ret)
mhi_device_put(ab_pci->mhi_ctrl->mhi_dev);
}

@@ -230,6 +232,7 @@ u32 ath11k_pci_read32(struct ath11k_base *ab, u32 offset)
{
struct ath11k_pci *ab_pci = ath11k_pci_priv(ab);
u32 val, window_start;
+ int ret = 0;

/* for offset beyond BAR + 4K - 32, may
* need to wakeup MHI to access.
@@ -237,7 +240,7 @@ u32 ath11k_pci_read32(struct ath11k_base *ab, u32 offset)
if (ab->hw_params.wakeup_mhi &&
test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
offset >= ACCESS_ALWAYS_OFF)
- mhi_device_get_sync(ab_pci->mhi_ctrl->mhi_dev);
+ ret = mhi_device_get_sync(ab_pci->mhi_ctrl->mhi_dev);

if (offset < WINDOW_START) {
val = ioread32(ab->mem + offset);
@@ -261,7 +264,8 @@ u32 ath11k_pci_read32(struct ath11k_base *ab, u32 offset)

if (ab->hw_params.wakeup_mhi &&
test_bit(ATH11K_PCI_FLAG_INIT_DONE, &ab_pci->flags) &&
- offset >= ACCESS_ALWAYS_OFF)
+ offset >= ACCESS_ALWAYS_OFF &&
+ !ret)
mhi_device_put(ab_pci->mhi_ctrl->mhi_dev);

return val;
--
2.31.1