Hi,
i discovered a random oops during probe of brcmfmac on Raspberry Pi 3 in yesterdays kernelci run for net-next [1]. I need to point out there is no DT entry for the wifi chip on Raspberry Pi 3 in the lack of a driver for the necessary GPIO expander. So the "HT Avail timeout" is expected.
I was also able to trigger this oops by calling "modprobe brcmfmac" on my Raspberry Pi 3 with latest linux-next.
Any help to fix this is appreciated.
[1] - https://storage.kernelci.org/net-next/master/v4.14-rc8-2221-ga8a6f1e4ea78/arm64/defconfig+kselftest/lab-baylibre/boot-bcm2837-rpi-3-b.txt
[ 94.922915] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[ 94.933262] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[ 94.940503] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[ 94.950222] cfg80211: failed to load regulatory.db
[ 94.977028] brcmfmac: brcmf_fw_map_chip_to_name: using brcm/brcmfmac43430-sdio.bin for chip 0x00a9a6(43430) rev 0x000001
[ 94.990977] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43430-sdio.txt failed with error -2
[ 96.021578] brcmfmac: brcmf_sdio_htclk: HT Avail timeout (1000000): clkctl 0x50
[ 97.034210] brcmfmac: brcmf_sdio_htclk: HT Avail timeout (1000000): clkctl 0x50
[ 97.071241] Unable to handle kernel paging request at virtual address 726f6674616cd8
[ 97.079464] Mem abort info:
[ 97.082437] ESR = 0x96000004
[ 97.085744] Exception class = DABT (current EL), IL = 32 bits
[ 97.092140] SET = 0, FnV = 0
[ 97.095438] EA = 0, S1PTW = 0
[ 97.098775] Data abort info:
[ 97.101851] ISV = 0, ISS = 0x00000004
[ 97.105942] CM = 0, WnR = 0
[ 97.118824] [00726f6674616cd8] address between user and kernel address ranges
[ 97.135901] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[ 97.151348] Modules linked in: brcmfmac cfg80211 brcmutil vc4 cec drm_kms_helper drm crc32_ce smsc95xx rfkill bcm2835_rng usbnet rng_core pwm_bcm2835 bcm2835_dma i2c_bcm2835 ip_tables x_tables ipv6 [last unloaded: cfg80211]
[ 97.182218] CPU: 3 PID: 92 Comm: kworker/3:1 Tainted: G W 4.14.0-rc8-next-20171110 #1
[ 97.202052] Hardware name: Raspberry Pi 3 Model B Rev 1.2 (DT)
[ 97.218562] Workqueue: events request_firmware_work_func
[ 97.234663] task: ffff8000372dd400 task.stack: ffff000009420000
[ 97.251451] pstate: 60000005 (nZCv daif -PAN -UAO)
[ 97.267105] pc : mutex_lock+0x30/0x60
[ 97.281628] lr : mutex_lock+0x24/0x60
[ 97.296124] sp : ffff000009423bd0
[ 97.309945] x29: ffff000009423bd0 x28: 0000000000000000
[ 97.325824] x27: ffff00000932bd20 x26: ffff000008fafbf0
[ 97.341362] x25: ffff8000372dd400 x24: ffff800036d8c100
[ 97.356869] x23: 0000000000000000 x22: ffff800038fd3d00
[ 97.372367] x21: 0000000000000000 x20: 6d726f6674616cd8
[ 97.387876] x19: 6d726f6674616cd8 x18: 0000000000000000
[ 97.403458] x17: 0000ffffacf22930 x16: ffff00000825b470
[ 97.419094] x15: 0000000000000000 x14: 0000000000000400
[ 97.434747] x13: 0000000000000400 x12: 0000000000000000
[ 97.450344] x11: 0000000000000000 x10: 0000000000000000
[ 97.465868] x9 : 0000000000000000 x8 : ffff8000312aa9a8
[ 97.481377] x7 : ffff8000312a1458 x6 : 0000000000000000
[ 97.496942] x5 : 0000000000000000 x4 : ffff8000372dd400
[ 97.512552] x3 : 0000000000000000 x2 : 0000000004208060
[ 97.528144] x1 : 0000000000000000 x0 : ffff8000372dd400
[ 97.543741] Process kworker/3:1 (pid: 92, stack limit = 0xffff000009420000)
[ 97.561203] Call trace:
[ 97.574047] mutex_lock+0x30/0x60
[ 97.587808] device_release_driver_internal+0x38/0x1f0
[ 97.603563] device_release_driver+0x14/0x20
[ 97.618589] brcmf_sdio_firmware_callback+0x160/0x578 [brcmfmac]
[ 97.635534] brcmf_fw_request_nvram_done+0x258/0x5e8 [brcmfmac]
[ 97.652336] request_firmware_work_func+0x34/0x60
[ 97.667928] process_one_work+0x1ac/0x328
[ 97.682825] worker_thread+0x50/0x460
[ 97.697361] kthread+0x104/0x130
[ 97.711466] ret_from_fork+0x10/0x18
[ 97.725960] Code: 97dc9c70 d2800001 d5384100 f9800271 (c85ffe62)
[ 97.743203] ---[ end trace a27498e726fb3a35 ]---
Hi Arend,
> Arend van Spriel <[email protected]> hat am 24. November 2017 um 13:52 geschrieben:
>
>
> ...
>
> I just realized the remove of sdio func #1 removes everything so we must
> first remove sdio func #2. Can you try the patch below.
>
> Regards,
> Arend
> ---
> diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
> b/drivers/n
> index b2256aa..58fa438 100644
> --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
> +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
> @@ -4121,8 +4121,8 @@ static void brcmf_sdio_firmware_callback(struct
> device *de
> sdio_release_host(sdiodev->func[1]);
> fail:
> brcmf_dbg(TRACE, "failed: dev=%s, err=%d\n", dev_name(dev), err);
> - device_release_driver(dev);
> device_release_driver(&sdiodev->func[2]->dev);
> + device_release_driver(dev);
> }
>
> struct brcmf_sdio *brcmf_sdio_probe(struct brcmf_sdio_dev *sdiodev)
>
>
that's it. The issue disappeared.
Thanks
Stefan
Hi Arend,
> Arend van Spriel <[email protected]> hat am 22. November 2017 um 19:23 geschrieben:
>
>
> On 22-11-17 17:20, Stefan Wahren wrote:
> > Hi,
> >
> >> Stefan Wahren <[email protected]> hat am 12. November 2017 um 17:50 geschrieben:
> >>
> >>
> >> Hi,
> >> i discovered a random oops during probe of brcmfmac on Raspberry Pi 3 in yesterdays kernelci run for net-next [1]. I need to point out there is no DT entry for the wifi chip on Raspberry Pi 3 in the lack of a driver for the necessary GPIO expander. So the "HT Avail timeout" is expected.
> >>
> >> I was also able to trigger this oops by calling "modprobe brcmfmac" on my Raspberry Pi 3 with latest linux-next.
> >>
> >> Any help to fix this is appreciated.
> >>
> >> [1] - https://storage.kernelci.org/net-next/master/v4.14-rc8-2221-ga8a6f1e4ea78/arm64/defconfig+kselftest/lab-baylibre/boot-bcm2837-rpi-3-b.txt
> >>
> >
> > the issue still persists in linux-next-20171122:
> >
> ...
>
> seems like a use-after-free. We do a device_release_driver() twice. Once
> for sdio func #1 and for sdio func #2. This was introduced by:
>
> commit 7a51461fc2da82a6c565a3ee65c41c197f28225d
> Author: Arend Van Spriel <[email protected]>
> Date: Mon Jun 12 12:47:34 2017 +0100
>
> brcmfmac: unbind all devices upon failure in firmware callback
>
> What we do is:
>
> device_release_driver(dev);
> device_release_driver(&sdiodev->func[2]->dev);
>
> with the assumption that dev == &sdiodev->func[1]->dev. I wonder if that
> is always true. The error print did not make it in your log. Maybe we
> should make it explicit here (see below).
i tried it, but the issue still occurs.
Maybe this is helpful but this issue doesn't happend everytime. Sometimes i need 2 attemps via unloading/loading.
>
> Regards,
> Arend
> ---
>
> diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
> b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
> index 613caca..0fda9a4 100644
> --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
> +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
> @@ -4096,7 +4096,7 @@ static void brcmf_sdio_firmware_callback(struct
> device *dev, int err,
> sdio_release_host(sdiodev->func[1]);
> fail:
> brcmf_dbg(TRACE, "failed: dev=%s, err=%d\n", dev_name(dev), err);
> - device_release_driver(dev);
> + device_release_driver(&sdiodev->func[1]->dev);
> device_release_driver(&sdiodev->func[2]->dev);
> }
>
On 11/22/2017 8:17 PM, Stefan Wahren wrote:
> Hi Arend,
>
>> Arend van Spriel <[email protected]> hat am 22. November 2017 um 19:23 geschrieben:
>>
>>
>> On 22-11-17 17:20, Stefan Wahren wrote:
>>> Hi,
>>>
>>>> Stefan Wahren <[email protected]> hat am 12. November 2017 um 17:50 geschrieben:
>>>>
>>>>
>>>> Hi,
>>>> i discovered a random oops during probe of brcmfmac on Raspberry Pi 3 in yesterdays kernelci run for net-next [1]. I need to point out there is no DT entry for the wifi chip on Raspberry Pi 3 in the lack of a driver for the necessary GPIO expander. So the "HT Avail timeout" is expected.
>>>>
>>>> I was also able to trigger this oops by calling "modprobe brcmfmac" on my Raspberry Pi 3 with latest linux-next.
>>>>
>>>> Any help to fix this is appreciated.
>>>>
>>>> [1] - https://storage.kernelci.org/net-next/master/v4.14-rc8-2221-ga8a6f1e4ea78/arm64/defconfig+kselftest/lab-baylibre/boot-bcm2837-rpi-3-b.txt
>>>>
>>>
>>> the issue still persists in linux-next-20171122:
>>>
>> ...
>>
>> seems like a use-after-free. We do a device_release_driver() twice. Once
>> for sdio func #1 and for sdio func #2. This was introduced by:
>>
>> commit 7a51461fc2da82a6c565a3ee65c41c197f28225d
>> Author: Arend Van Spriel <[email protected]>
>> Date: Mon Jun 12 12:47:34 2017 +0100
>>
>> brcmfmac: unbind all devices upon failure in firmware callback
>>
>> What we do is:
>>
>> device_release_driver(dev);
>> device_release_driver(&sdiodev->func[2]->dev);
>>
>> with the assumption that dev == &sdiodev->func[1]->dev. I wonder if that
>> is always true. The error print did not make it in your log. Maybe we
>> should make it explicit here (see below).
>
> i tried it, but the issue still occurs.
>
> Maybe this is helpful but this issue doesn't happend everytime. Sometimes i need 2 attemps via unloading/loading.
>
I just realized the remove of sdio func #1 removes everything so we must
first remove sdio func #2. Can you try the patch below.
Regards,
Arend
---
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
b/drivers/n
index b2256aa..58fa438 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
@@ -4121,8 +4121,8 @@ static void brcmf_sdio_firmware_callback(struct
device *de
sdio_release_host(sdiodev->func[1]);
fail:
brcmf_dbg(TRACE, "failed: dev=%s, err=%d\n", dev_name(dev), err);
- device_release_driver(dev);
device_release_driver(&sdiodev->func[2]->dev);
+ device_release_driver(dev);
}
struct brcmf_sdio *brcmf_sdio_probe(struct brcmf_sdio_dev *sdiodev)
On 22-11-17 17:20, Stefan Wahren wrote:
> Hi,
>
>> Stefan Wahren <[email protected]> hat am 12. November 2017 um 17:50 geschrieben:
>>
>>
>> Hi,
>> i discovered a random oops during probe of brcmfmac on Raspberry Pi 3 in yesterdays kernelci run for net-next [1]. I need to point out there is no DT entry for the wifi chip on Raspberry Pi 3 in the lack of a driver for the necessary GPIO expander. So the "HT Avail timeout" is expected.
>>
>> I was also able to trigger this oops by calling "modprobe brcmfmac" on my Raspberry Pi 3 with latest linux-next.
>>
>> Any help to fix this is appreciated.
>>
>> [1] - https://storage.kernelci.org/net-next/master/v4.14-rc8-2221-ga8a6f1e4ea78/arm64/defconfig+kselftest/lab-baylibre/boot-bcm2837-rpi-3-b.txt
>>
>
> the issue still persists in linux-next-20171122:
>
> [ 817.659632] brcmfmac: brcmf_fw_map_chip_to_name: using brcm/brcmfmac43430-sdio.bin for chip 0x00a9a6(43430) rev 0x000001
> [ 817.672739] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43430-sdio.txt failed with error -2
> [ 818.708435] brcmfmac: brcmf_sdio_htclk: HT Avail timeout (1000000): clkctl 0x50
> [ 819.722103] brcmfmac: brcmf_sdio_htclk: HT Avail timeout (1000000): clkctl 0x50
> [ 819.755321] Unable to handle kernel paging request at virtual address 726f6674616cd8
> [ 819.763255] Mem abort info:
> [ 819.766131] ESR = 0x96000004
> [ 819.769240] Exception class = DABT (current EL), IL = 32 bits
> [ 819.775290] SET = 0, FnV = 0
> [ 819.778416] EA = 0, S1PTW = 0
> [ 819.781605] Data abort info:
> [ 819.784552] ISV = 0, ISS = 0x00000004
> [ 819.788495] CM = 0, WnR = 0
> [ 819.791532] [00726f6674616cd8] address between user and kernel address ranges
> [ 819.798805] Internal error: Oops: 96000004 [#1] PREEMPT SMP
> [ 819.804469] Modules linked in: brcmfmac cfg80211 brcmutil vc4 cec drm_kms_helper drm bcm2835_rng rfkill rng_core smsc95xx usbnet bcm2835_dma crc32_ce pwm_bcm2835 i2c_bcm2835 ip_tables x_tables ipv6 [last unloaded: cfg80211]
> [ 819.824687] CPU: 3 PID: 92 Comm: kworker/3:1 Not tainted 4.14.0-next-20171122 #1
> [ 819.832207] Hardware name: Raspberry Pi 3 Model B Rev 1.2 (DT)
> [ 819.838152] Workqueue: events request_firmware_work_func
> [ 819.843561] task: ffff80003833aa00 task.stack: ffff000009498000
> [ 819.849583] pstate: 60000005 (nZCv daif -PAN -UAO)
> [ 819.854460] pc : mutex_lock+0xc/0x40
> [ 819.858101] lr : device_release_driver_internal+0x38/0x1f0
> [ 819.863676] sp : ffff00000949bbf0
> [ 819.867045] x29: ffff00000949bbf0 x28: 0000000000000000
> [ 819.872451] x27: ffff00000931bd20 x26: ffff000008f9fbb0
> [ 819.877856] x25: ffff80003833aa00 x24: ffff800036e20f80
> [ 819.883261] x23: 0000000000000000 x22: ffff800039fd3d00
> [ 819.888665] x21: 0000000000000000 x20: 6d726f6674616cd8
> [ 819.894068] x19: 6d726f6674616c78 x18: 0000000000000000
> [ 819.899473] x17: 0000ffffa52ba930 x16: ffff000008256b30
> [ 819.904877] x15: 0000000000000000 x14: 0000000000000400
> [ 819.910282] x13: 0000000000000400 x12: 0000000000000000
> [ 819.915685] x11: 0000000000000000 x10: 0000000000000001
> [ 819.921087] x9 : 0000000000000001 x8 : ffff800032fdec08
> [ 819.926493] x7 : ffff800032d7c158 x6 : 0000000000000000
> [ 819.931895] x5 : ffff800032d7c288 x4 : 0000000000000000
> [ 819.937299] x3 : ffff80003833aa00 x2 : 0000000000000000
> [ 819.942705] x1 : ffff80003833aa00 x0 : 6d726f6674616cd8
> [ 819.948112] Process kworker/3:1 (pid: 92, stack limit = 0xffff000009498000)
> [ 819.955188] Call trace:
> [ 819.957681] mutex_lock+0xc/0x40
> [ 819.960967] device_release_driver+0x14/0x20
> [ 819.965450] brcmf_sdio_firmware_callback+0x160/0x578 [brcmfmac]
> [ 819.971689] brcmf_fw_request_nvram_done+0x258/0x5e8 [brcmfmac]
> [ 819.977717] request_firmware_work_func+0x34/0x60
> [ 819.982510] process_one_work+0x1cc/0x328
> [ 819.986592] worker_thread+0x50/0x450
> [ 819.990320] kthread+0xf8/0x128
> [ 819.993516] ret_from_fork+0x10/0x18
> [ 819.997154] Code: d65f03c0 d5384101 d2800002 f9800011 (c85ffc03)
> [ 820.003351] ---[ end trace f4bff35241327a18 ]---
>
seems like a use-after-free. We do a device_release_driver() twice. Once
for sdio func #1 and for sdio func #2. This was introduced by:
commit 7a51461fc2da82a6c565a3ee65c41c197f28225d
Author: Arend Van Spriel <[email protected]>
Date: Mon Jun 12 12:47:34 2017 +0100
brcmfmac: unbind all devices upon failure in firmware callback
What we do is:
device_release_driver(dev);
device_release_driver(&sdiodev->func[2]->dev);
with the assumption that dev == &sdiodev->func[1]->dev. I wonder if that
is always true. The error print did not make it in your log. Maybe we
should make it explicit here (see below).
Regards,
Arend
---
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
index 613caca..0fda9a4 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
@@ -4096,7 +4096,7 @@ static void brcmf_sdio_firmware_callback(struct
device *dev, int err,
sdio_release_host(sdiodev->func[1]);
fail:
brcmf_dbg(TRACE, "failed: dev=%s, err=%d\n", dev_name(dev), err);
- device_release_driver(dev);
+ device_release_driver(&sdiodev->func[1]->dev);
device_release_driver(&sdiodev->func[2]->dev);
}
Hi,
> Stefan Wahren <[email protected]> hat am 12. November 2017 um 17:50 geschrieben:
>
>
> Hi,
> i discovered a random oops during probe of brcmfmac on Raspberry Pi 3 in yesterdays kernelci run for net-next [1]. I need to point out there is no DT entry for the wifi chip on Raspberry Pi 3 in the lack of a driver for the necessary GPIO expander. So the "HT Avail timeout" is expected.
>
> I was also able to trigger this oops by calling "modprobe brcmfmac" on my Raspberry Pi 3 with latest linux-next.
>
> Any help to fix this is appreciated.
>
> [1] - https://storage.kernelci.org/net-next/master/v4.14-rc8-2221-ga8a6f1e4ea78/arm64/defconfig+kselftest/lab-baylibre/boot-bcm2837-rpi-3-b.txt
>
the issue still persists in linux-next-20171122:
[ 817.659632] brcmfmac: brcmf_fw_map_chip_to_name: using brcm/brcmfmac43430-sdio.bin for chip 0x00a9a6(43430) rev 0x000001
[ 817.672739] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43430-sdio.txt failed with error -2
[ 818.708435] brcmfmac: brcmf_sdio_htclk: HT Avail timeout (1000000): clkctl 0x50
[ 819.722103] brcmfmac: brcmf_sdio_htclk: HT Avail timeout (1000000): clkctl 0x50
[ 819.755321] Unable to handle kernel paging request at virtual address 726f6674616cd8
[ 819.763255] Mem abort info:
[ 819.766131] ESR = 0x96000004
[ 819.769240] Exception class = DABT (current EL), IL = 32 bits
[ 819.775290] SET = 0, FnV = 0
[ 819.778416] EA = 0, S1PTW = 0
[ 819.781605] Data abort info:
[ 819.784552] ISV = 0, ISS = 0x00000004
[ 819.788495] CM = 0, WnR = 0
[ 819.791532] [00726f6674616cd8] address between user and kernel address ranges
[ 819.798805] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[ 819.804469] Modules linked in: brcmfmac cfg80211 brcmutil vc4 cec drm_kms_helper drm bcm2835_rng rfkill rng_core smsc95xx usbnet bcm2835_dma crc32_ce pwm_bcm2835 i2c_bcm2835 ip_tables x_tables ipv6 [last unloaded: cfg80211]
[ 819.824687] CPU: 3 PID: 92 Comm: kworker/3:1 Not tainted 4.14.0-next-20171122 #1
[ 819.832207] Hardware name: Raspberry Pi 3 Model B Rev 1.2 (DT)
[ 819.838152] Workqueue: events request_firmware_work_func
[ 819.843561] task: ffff80003833aa00 task.stack: ffff000009498000
[ 819.849583] pstate: 60000005 (nZCv daif -PAN -UAO)
[ 819.854460] pc : mutex_lock+0xc/0x40
[ 819.858101] lr : device_release_driver_internal+0x38/0x1f0
[ 819.863676] sp : ffff00000949bbf0
[ 819.867045] x29: ffff00000949bbf0 x28: 0000000000000000
[ 819.872451] x27: ffff00000931bd20 x26: ffff000008f9fbb0
[ 819.877856] x25: ffff80003833aa00 x24: ffff800036e20f80
[ 819.883261] x23: 0000000000000000 x22: ffff800039fd3d00
[ 819.888665] x21: 0000000000000000 x20: 6d726f6674616cd8
[ 819.894068] x19: 6d726f6674616c78 x18: 0000000000000000
[ 819.899473] x17: 0000ffffa52ba930 x16: ffff000008256b30
[ 819.904877] x15: 0000000000000000 x14: 0000000000000400
[ 819.910282] x13: 0000000000000400 x12: 0000000000000000
[ 819.915685] x11: 0000000000000000 x10: 0000000000000001
[ 819.921087] x9 : 0000000000000001 x8 : ffff800032fdec08
[ 819.926493] x7 : ffff800032d7c158 x6 : 0000000000000000
[ 819.931895] x5 : ffff800032d7c288 x4 : 0000000000000000
[ 819.937299] x3 : ffff80003833aa00 x2 : 0000000000000000
[ 819.942705] x1 : ffff80003833aa00 x0 : 6d726f6674616cd8
[ 819.948112] Process kworker/3:1 (pid: 92, stack limit = 0xffff000009498000)
[ 819.955188] Call trace:
[ 819.957681] mutex_lock+0xc/0x40
[ 819.960967] device_release_driver+0x14/0x20
[ 819.965450] brcmf_sdio_firmware_callback+0x160/0x578 [brcmfmac]
[ 819.971689] brcmf_fw_request_nvram_done+0x258/0x5e8 [brcmfmac]
[ 819.977717] request_firmware_work_func+0x34/0x60
[ 819.982510] process_one_work+0x1cc/0x328
[ 819.986592] worker_thread+0x50/0x450
[ 819.990320] kthread+0xf8/0x128
[ 819.993516] ret_from_fork+0x10/0x18
[ 819.997154] Code: d65f03c0 d5384101 d2800002 f9800011 (c85ffc03)
[ 820.003351] ---[ end trace f4bff35241327a18 ]---