Hello,
the problem is described in the following patch.
Some remarks:
I'm not quite sure if the function proposed in the patch should be placed
somewhere else in the code... The patch now calls it during device
deinitialization after urbs deallocation, and I suppose the interrupt
callback ath9k_hif_usb_rx_stream() not to be executed during/after that
period (in this case the leak would still remain, but I couldn't reproduce
this particular case). Maybe the function freeing remain_skbs should be
placed right before calling 'kfree(hif_dev)'?
The patch is made on top of [1] master branch.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/
---
Regards,
Fedor
hif_dev->remain_skb is allocated and used exclusively in
ath9k_hif_usb_rx_stream(). It is implied that an allocated remain_skb is
processed and subsequently freed (in error paths) only during the next
call of ath9k_hif_usb_rx_stream().
So, if the device is deinitialized between those two calls or if the skb
contents are incorrect, it is possible that ath9k_hif_usb_rx_stream() is
not called next time and the allocated remain_skb is leaked. Our local
Syzkaller instance was able to trigger that.
Fix the leak by introducing a function to explicitly free remain_skb (if
it is not NULL) when the device is being deinitialized. remain_skb is NULL
when it has not been allocated at all (hif_dev struct is kzalloced) or
when it has been proccesed in next call to ath9k_hif_usb_rx_stream().
Proper spinlocks are held to prevent possible concurrent access to
remain_skb from the interrupt context ath9k_hif_usb_rx_stream(). These
accesses should not happen as rx_urbs have been deallocated before but
it prevents a dangerous race condition in these cases.
Found by Linux Verification Center (linuxtesting.org) with Syzkaller.
Fixes: fb9987d0f748 ("ath9k_htc: Support for AR9271 chipset.")
Signed-off-by: Fedor Pchelkin <[email protected]>
Signed-off-by: Alexey Khoroshilov <[email protected]>
---
drivers/net/wireless/ath/ath9k/hif_usb.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/drivers/net/wireless/ath/ath9k/hif_usb.c b/drivers/net/wireless/ath/ath9k/hif_usb.c
index f521dfa2f194..e03ab972edf7 100644
--- a/drivers/net/wireless/ath/ath9k/hif_usb.c
+++ b/drivers/net/wireless/ath/ath9k/hif_usb.c
@@ -534,6 +534,23 @@ static struct ath9k_htc_hif hif_usb = {
.send = hif_usb_send,
};
+/* Need to free remain_skb allocated in ath9k_hif_usb_rx_stream
+ * in case ath9k_hif_usb_rx_stream wasn't called next time to
+ * process the buffer and subsequently free it.
+ */
+static void ath9k_hif_usb_free_rx_remain_skb(struct hif_device_usb *hif_dev)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&hif_dev->rx_lock, flags);
+ if (hif_dev->remain_skb) {
+ dev_kfree_skb_any(hif_dev->remain_skb);
+ hif_dev->remain_skb = NULL;
+ hif_dev->rx_remain_len = 0;
+ }
+ spin_unlock_irqrestore(&hif_dev->rx_lock, flags);
+}
+
static void ath9k_hif_usb_rx_stream(struct hif_device_usb *hif_dev,
struct sk_buff *skb)
{
@@ -1129,6 +1146,7 @@ static int ath9k_hif_usb_dev_init(struct hif_device_usb *hif_dev)
static void ath9k_hif_usb_dev_deinit(struct hif_device_usb *hif_dev)
{
ath9k_hif_usb_dealloc_urbs(hif_dev);
+ ath9k_hif_usb_free_rx_remain_skb(hif_dev);
}
/*
--
2.34.1
Fedor Pchelkin <[email protected]> writes:
> hif_dev->remain_skb is allocated and used exclusively in
> ath9k_hif_usb_rx_stream(). It is implied that an allocated remain_skb is
> processed and subsequently freed (in error paths) only during the next
> call of ath9k_hif_usb_rx_stream().
>
> So, if the device is deinitialized between those two calls or if the skb
> contents are incorrect, it is possible that ath9k_hif_usb_rx_stream() is
> not called next time and the allocated remain_skb is leaked. Our local
> Syzkaller instance was able to trigger that.
>
> Fix the leak by introducing a function to explicitly free remain_skb (if
> it is not NULL) when the device is being deinitialized. remain_skb is NULL
> when it has not been allocated at all (hif_dev struct is kzalloced) or
> when it has been proccesed in next call to ath9k_hif_usb_rx_stream().
>
> Proper spinlocks are held to prevent possible concurrent access to
> remain_skb from the interrupt context ath9k_hif_usb_rx_stream(). These
> accesses should not happen as rx_urbs have been deallocated before but
> it prevents a dangerous race condition in these cases.
>
> Found by Linux Verification Center (linuxtesting.org) with Syzkaller.
>
> Fixes: fb9987d0f748 ("ath9k_htc: Support for AR9271 chipset.")
> Signed-off-by: Fedor Pchelkin <[email protected]>
> Signed-off-by: Alexey Khoroshilov <[email protected]>
> ---
> drivers/net/wireless/ath/ath9k/hif_usb.c | 18 ++++++++++++++++++
> 1 file changed, 18 insertions(+)
>
> diff --git a/drivers/net/wireless/ath/ath9k/hif_usb.c b/drivers/net/wireless/ath/ath9k/hif_usb.c
> index f521dfa2f194..e03ab972edf7 100644
> --- a/drivers/net/wireless/ath/ath9k/hif_usb.c
> +++ b/drivers/net/wireless/ath/ath9k/hif_usb.c
> @@ -534,6 +534,23 @@ static struct ath9k_htc_hif hif_usb = {
> .send = hif_usb_send,
> };
>
> +/* Need to free remain_skb allocated in ath9k_hif_usb_rx_stream
> + * in case ath9k_hif_usb_rx_stream wasn't called next time to
> + * process the buffer and subsequently free it.
> + */
> +static void ath9k_hif_usb_free_rx_remain_skb(struct hif_device_usb *hif_dev)
> +{
> + unsigned long flags;
> +
> + spin_lock_irqsave(&hif_dev->rx_lock, flags);
> + if (hif_dev->remain_skb) {
> + dev_kfree_skb_any(hif_dev->remain_skb);
> + hif_dev->remain_skb = NULL;
> + hif_dev->rx_remain_len = 0;
> + }
> + spin_unlock_irqrestore(&hif_dev->rx_lock, flags);
> +}
> +
> static void ath9k_hif_usb_rx_stream(struct hif_device_usb *hif_dev,
> struct sk_buff *skb)
> {
> @@ -1129,6 +1146,7 @@ static int ath9k_hif_usb_dev_init(struct hif_device_usb *hif_dev)
> static void ath9k_hif_usb_dev_deinit(struct hif_device_usb *hif_dev)
> {
> ath9k_hif_usb_dealloc_urbs(hif_dev);
> + ath9k_hif_usb_free_rx_remain_skb(hif_dev);
> }
Erm, does this actually fix the leak? AFAICT, ath9k_hif_usb_dev_deinit()
is only called on the error path of ath9k_hif_usb_firmware_cb(), not
when the device is subsequently torn down in
ath9k_htc_disconnect_device()?
I think the right place to put this is probably inside
ath9k_hif_usb_dealloc_urbs()? That gets called on USB suspend as well,
but it seems to me that if we're suspending the device to an extent that
we're deallocating the urbs, we should be clearing out the cached skb in
remain_skb anyway?
-Toke
On 16.02.2023 19:15, Toke Høiland-Jørgensen wrote:
> Erm, does this actually fix the leak? AFAICT, ath9k_hif_usb_dev_deinit()
> is only called on the error path of ath9k_hif_usb_firmware_cb(), not
> when the device is subsequently torn down in
> ath9k_htc_disconnect_device()?
ath9k_hif_usb_dev_deinit() is also called inside
ath9k_hif_usb_disconnect(). I see it to be the only place wherehif_dev is
freed (apart from an early error path), so the current patchimplementation
actually fixes the leak. However, as you have noticed, itis not probably
the best place to put the deallocation: we need to clearthe cached skb
not only when freeing the device but in urbs deallocationcase, too - in
order to avoid its irrelevant processing later.
> I think the right place to put this is probably inside
> ath9k_hif_usb_dealloc_urbs()? That gets called on USB suspend as well,
> but it seems to me that if we're suspending the device to an extent that
> we're deallocating the urbs, we should be clearing out the cached skb in
> remain_skb anyway?
>
> -Toke
Thank you for the advice! As I can see, remain_skb makes sense when
receiving two consecutive urbs which are logically linked together, i.e.
a specific data field from the first skb indicates a cached skb to be
allocated, memcpy'd with some data and subsequently processed in the
next call to rx callback (see 6ce708f54cc8 ("ath9k: Fix out-of-bound
memcpy in ath9k_hif_usb_rx_stream")). Urbs deallocation, I suppose,
makes that link irrelevant.
So I agree with you that remain_skb freeing should be done when
deallocating the urbs. I would just place that specifically into
ath9k_hif_usb_dealloc_rx_urbs() as remain_skb is associated with rx
urbs.
RX_STAT_INC(hif_dev, skb_dropped), I think, should be also called when
freeing afilled remain_skb?
---
Regards,
Fedor
Fedor Pchelkin <[email protected]> writes:
> On 16.02.2023 19:15, Toke Høiland-Jørgensen wrote:
> > Erm, does this actually fix the leak? AFAICT, ath9k_hif_usb_dev_deinit()
> > is only called on the error path of ath9k_hif_usb_firmware_cb(), not
> > when the device is subsequently torn down in
> > ath9k_htc_disconnect_device()?
>
> ath9k_hif_usb_dev_deinit() is also called inside
> ath9k_hif_usb_disconnect().
No it's not, as of:
f099c5c9e2ba ("wifi: ath9k: Fix use-after-free in ath9k_hif_usb_disconnect()")
I guess you're looking at an older tree? Please base your patches on an
up-to-date ath-next tree.
> I see it to be the only place wherehif_dev is freed (apart from an
> early error path), so the current patchimplementation actually fixes
> the leak. However, as you have noticed, itis not probably the best
> place to put the deallocation: we need to clearthe cached skb not only
> when freeing the device but in urbs deallocationcase, too - in order
> to avoid its irrelevant processing later.
>
> > I think the right place to put this is probably inside
> > ath9k_hif_usb_dealloc_urbs()? That gets called on USB suspend as well,
> > but it seems to me that if we're suspending the device to an extent that
> > we're deallocating the urbs, we should be clearing out the cached skb in
> > remain_skb anyway?
> >
> > -Toke
>
> Thank you for the advice! As I can see, remain_skb makes sense when
> receiving two consecutive urbs which are logically linked together, i.e.
> a specific data field from the first skb indicates a cached skb to be
> allocated, memcpy'd with some data and subsequently processed in the
> next call to rx callback (see 6ce708f54cc8 ("ath9k: Fix out-of-bound
> memcpy in ath9k_hif_usb_rx_stream")). Urbs deallocation, I suppose,
> makes that link irrelevant.
>
> So I agree with you that remain_skb freeing should be done when
> deallocating the urbs. I would just place that specifically into
> ath9k_hif_usb_dealloc_rx_urbs() as remain_skb is associated with rx
> urbs.
SGTM.
> RX_STAT_INC(hif_dev, skb_dropped), I think, should be also called when
> freeing afilled remain_skb?
Well, if this is mostly something that happens if the device is going
away I'm not sure that anyone will actually see that; but I suppose if
it happens on suspend, the stat increase may be useful, and it shouldn't
hurt otherwise, so sure, let's add that :)
-Toke
On 16.02.2023 21:05, Toke Høiland-Jørgensen wrote:
> Fedor Pchelkin <[email protected]> writes:
>
>> On 16.02.2023 19:15, Toke Høiland-Jørgensen wrote:
>> > Erm, does this actually fix the leak? AFAICT, ath9k_hif_usb_dev_deinit()
>> > is only called on the error path of ath9k_hif_usb_firmware_cb(), not
>> > when the device is subsequently torn down in
>> > ath9k_htc_disconnect_device()?
>>
>> ath9k_hif_usb_dev_deinit() is also called inside
>> ath9k_hif_usb_disconnect().
> No it's not, as of:
>
> f099c5c9e2ba ("wifi: ath9k: Fix use-after-free in ath9k_hif_usb_disconnect()")
>
> I guess you're looking at an older tree? Please base your patches on an
> up-to-date ath-next tree.
>
Oops, that's my fault, I indeed patched the wrong tree.
Thanks for clarifying!
hif_dev->remain_skb is allocated and used exclusively in
ath9k_hif_usb_rx_stream(). It is implied that an allocated remain_skb is
processed and subsequently freed (in error paths) only during the next
call of ath9k_hif_usb_rx_stream().
So, if the urbs are deallocated between those two calls due to the device
deinitialization or suspend, it is possible that ath9k_hif_usb_rx_stream()
is not called next time and the allocated remain_skb is leaked. Our local
Syzkaller instance was able to trigger that.
remain_skb makes sense when receiving two consecutive urbs which are
logically linked together, i.e. a specific data field from the first skb
indicates a cached skb to be allocated, memcpy'd with some data and
subsequently processed in the next call to ath9k_hif_usb_rx_stream(). Urbs
deallocation supposedly makes that link irrelevant so we need to free the
cached skb in those cases.
Fix the leak by introducing a function to explicitly free remain_skb (if
it is not NULL) when the rx urbs have been deallocated. remain_skb is NULL
when it has not been allocated at all (hif_dev struct is kzalloced) or
when it has been processed in next call to ath9k_hif_usb_rx_stream().
Found by Linux Verification Center (linuxtesting.org) with Syzkaller.
Fixes: fb9987d0f748 ("ath9k_htc: Support for AR9271 chipset.")
Signed-off-by: Fedor Pchelkin <[email protected]>
Signed-off-by: Alexey Khoroshilov <[email protected]>
---
v1->v2: move ath9k_hif_usb_free_rx_remain_skb call into urbs dealloc
function as advised by Toke; add a stat macro
drivers/net/wireless/ath/ath9k/hif_usb.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/drivers/net/wireless/ath/ath9k/hif_usb.c b/drivers/net/wireless/ath/ath9k/hif_usb.c
index f521dfa2f194..e0130beb304d 100644
--- a/drivers/net/wireless/ath/ath9k/hif_usb.c
+++ b/drivers/net/wireless/ath/ath9k/hif_usb.c
@@ -534,6 +534,24 @@ static struct ath9k_htc_hif hif_usb = {
.send = hif_usb_send,
};
+/* Need to free remain_skb allocated in ath9k_hif_usb_rx_stream
+ * in case ath9k_hif_usb_rx_stream wasn't called next time to
+ * process the buffer and subsequently free it.
+ */
+static void ath9k_hif_usb_free_rx_remain_skb(struct hif_device_usb *hif_dev)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&hif_dev->rx_lock, flags);
+ if (hif_dev->remain_skb) {
+ dev_kfree_skb_any(hif_dev->remain_skb);
+ hif_dev->remain_skb = NULL;
+ hif_dev->rx_remain_len = 0;
+ RX_STAT_INC(hif_dev, skb_dropped);
+ }
+ spin_unlock_irqrestore(&hif_dev->rx_lock, flags);
+}
+
static void ath9k_hif_usb_rx_stream(struct hif_device_usb *hif_dev,
struct sk_buff *skb)
{
@@ -868,6 +886,7 @@ static int ath9k_hif_usb_alloc_tx_urbs(struct hif_device_usb *hif_dev)
static void ath9k_hif_usb_dealloc_rx_urbs(struct hif_device_usb *hif_dev)
{
usb_kill_anchored_urbs(&hif_dev->rx_submitted);
+ ath9k_hif_usb_free_rx_remain_skb(hif_dev);
}
static int ath9k_hif_usb_alloc_rx_urbs(struct hif_device_usb *hif_dev)
--
2.34.1
Fedor Pchelkin <[email protected]> writes:
> hif_dev->remain_skb is allocated and used exclusively in
> ath9k_hif_usb_rx_stream(). It is implied that an allocated remain_skb is
> processed and subsequently freed (in error paths) only during the next
> call of ath9k_hif_usb_rx_stream().
>
> So, if the urbs are deallocated between those two calls due to the device
> deinitialization or suspend, it is possible that ath9k_hif_usb_rx_stream()
> is not called next time and the allocated remain_skb is leaked. Our local
> Syzkaller instance was able to trigger that.
>
> remain_skb makes sense when receiving two consecutive urbs which are
> logically linked together, i.e. a specific data field from the first skb
> indicates a cached skb to be allocated, memcpy'd with some data and
> subsequently processed in the next call to ath9k_hif_usb_rx_stream(). Urbs
> deallocation supposedly makes that link irrelevant so we need to free the
> cached skb in those cases.
>
> Fix the leak by introducing a function to explicitly free remain_skb (if
> it is not NULL) when the rx urbs have been deallocated. remain_skb is NULL
> when it has not been allocated at all (hif_dev struct is kzalloced) or
> when it has been processed in next call to ath9k_hif_usb_rx_stream().
>
> Found by Linux Verification Center (linuxtesting.org) with Syzkaller.
>
> Fixes: fb9987d0f748 ("ath9k_htc: Support for AR9271 chipset.")
> Signed-off-by: Fedor Pchelkin <[email protected]>
> Signed-off-by: Alexey Khoroshilov <[email protected]>
Thank you for the fix!
Acked-by: Toke Høiland-Jørgensen <[email protected]>
Fedor Pchelkin <[email protected]> wrote:
> hif_dev->remain_skb is allocated and used exclusively in
> ath9k_hif_usb_rx_stream(). It is implied that an allocated remain_skb is
> processed and subsequently freed (in error paths) only during the next
> call of ath9k_hif_usb_rx_stream().
>
> So, if the urbs are deallocated between those two calls due to the device
> deinitialization or suspend, it is possible that ath9k_hif_usb_rx_stream()
> is not called next time and the allocated remain_skb is leaked. Our local
> Syzkaller instance was able to trigger that.
>
> remain_skb makes sense when receiving two consecutive urbs which are
> logically linked together, i.e. a specific data field from the first skb
> indicates a cached skb to be allocated, memcpy'd with some data and
> subsequently processed in the next call to ath9k_hif_usb_rx_stream(). Urbs
> deallocation supposedly makes that link irrelevant so we need to free the
> cached skb in those cases.
>
> Fix the leak by introducing a function to explicitly free remain_skb (if
> it is not NULL) when the rx urbs have been deallocated. remain_skb is NULL
> when it has not been allocated at all (hif_dev struct is kzalloced) or
> when it has been processed in next call to ath9k_hif_usb_rx_stream().
>
> Found by Linux Verification Center (linuxtesting.org) with Syzkaller.
>
> Fixes: fb9987d0f748 ("ath9k_htc: Support for AR9271 chipset.")
> Signed-off-by: Fedor Pchelkin <[email protected]>
> Signed-off-by: Alexey Khoroshilov <[email protected]>
> Acked-by: Toke Høiland-Jørgensen <[email protected]>
> Signed-off-by: Kalle Valo <[email protected]>
Patch applied to ath-next branch of ath.git, thanks.
7654cc03eb69 wifi: ath9k: hif_usb: fix memory leak of remain_skbs
--
https://patchwork.kernel.org/project/linux-wireless/patch/[email protected]/
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches