2023-05-26 16:26:44

by Vladislav Efanov

[permalink] [raw]
Subject: [PATCH] batman-adv: Broken sync while rescheduling delayed work

Syzkaller got a lot of crashes like:
KASAN: use-after-free Write in *_timers*

All of these crashes point to the same memory area:

The buggy address belongs to the object at ffff88801f870000
which belongs to the cache kmalloc-8k of size 8192
The buggy address is located 5320 bytes inside of
8192-byte region [ffff88801f870000, ffff88801f872000)

This area belongs to :
batadv_priv->batadv_priv_dat->delayed_work->timer_list

The reason for these issues is the lack of synchronization. Delayed
work (batadv_dat_purge) schedules new timer/work while the device
is being deleted. As the result new timer/delayed work is set after
cancel_delayed_work_sync() was called. So after the device is freed
the timer list contains pointer to already freed memory.

Found by Linux Verification Center (linuxtesting.org) with syzkaller.

Fixes: 2f1dfbe18507 ("batman-adv: Distributed ARP Table - implement local storage")
Signed-off-by: Vladislav Efanov <[email protected]>
---
net/batman-adv/distributed-arp-table.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c
index 6968e55eb971..28a939d56090 100644
--- a/net/batman-adv/distributed-arp-table.c
+++ b/net/batman-adv/distributed-arp-table.c
@@ -101,7 +101,6 @@ static void batadv_dat_purge(struct work_struct *work);
*/
static void batadv_dat_start_timer(struct batadv_priv *bat_priv)
{
- INIT_DELAYED_WORK(&bat_priv->dat.work, batadv_dat_purge);
queue_delayed_work(batadv_event_workqueue, &bat_priv->dat.work,
msecs_to_jiffies(10000));
}
@@ -819,6 +818,7 @@ int batadv_dat_init(struct batadv_priv *bat_priv)
if (!bat_priv->dat.hash)
return -ENOMEM;

+ INIT_DELAYED_WORK(&bat_priv->dat.work, batadv_dat_purge);
batadv_dat_start_timer(bat_priv);

batadv_tvlv_handler_register(bat_priv, batadv_dat_tvlv_ogm_handler_v1,
--
2.34.1



2023-05-26 17:11:14

by Sven Eckelmann

[permalink] [raw]
Subject: Re: [PATCH] batman-adv: Broken sync while rescheduling delayed work

On Friday, 26 May 2023 18:16:32 CEST Vladislav Efanov wrote:
> The reason for these issues is the lack of synchronization. Delayed
> work (batadv_dat_purge) schedules new timer/work while the device
> is being deleted. As the result new timer/delayed work is set after
> cancel_delayed_work_sync() was called. So after the device is freed
> the timer list contains pointer to already freed memory.

You are most likely right but could you please point out what in the worker is
checked by the workqueue code that prevents it from being scheduled again?
(and which seems to be overwritten as your patch seems to suggest)

I think __cancel_work_timer marked the work as canceling but
batadv_dat_start_timer reinitialized the worked (thus removing this important
state). Would be nice if you could either correct me or confirm what I think to
remember.

Kind regards,
Sven


Attachments:
signature.asc (849.00 B)
This is a digitally signed message part.

2023-05-26 18:10:59

by Vladislav Efanov

[permalink] [raw]
Subject: Re: [PATCH] batman-adv: Broken sync while rescheduling delayed work

Sven,


cancel_delayed_work_sync() and queue_delayed_work()

use WORK_STRUCT_PENDING_BIT in work->data to synchronize.

INIT_DELAYED_WORK() clears this bit.


The situation is :  __cancel_work_timer() sets WORK_STRUCT_PENDING_BIT

but INIT_DELAYED_WORK() in batadv_dat_start_timer() clears it

and queue_delayed_work() schedules new work.


Best regards,

Vlad.

On 26.05.2023 19:49, Sven Eckelmann wrote:
> On Friday, 26 May 2023 18:16:32 CEST Vladislav Efanov wrote:
>> The reason for these issues is the lack of synchronization. Delayed
>> work (batadv_dat_purge) schedules new timer/work while the device
>> is being deleted. As the result new timer/delayed work is set after
>> cancel_delayed_work_sync() was called. So after the device is freed
>> the timer list contains pointer to already freed memory.
> You are most likely right but could you please point out what in the worker is
> checked by the workqueue code that prevents it from being scheduled again?
> (and which seems to be overwritten as your patch seems to suggest)
>
> I think __cancel_work_timer marked the work as canceling but
> batadv_dat_start_timer reinitialized the worked (thus removing this important
> state). Would be nice if you could either correct me or confirm what I think to
> remember.
>
> Kind regards,
> Sven

2023-05-26 21:01:37

by Sven Eckelmann

[permalink] [raw]
Subject: Re: [PATCH] batman-adv: Broken sync while rescheduling delayed work

On Friday, 26 May 2023 18:16:32 CEST Vladislav Efanov wrote:
> Syzkaller got a lot of crashes like:
> KASAN: use-after-free Write in *_timers*
>
> All of these crashes point to the same memory area:
>
> The buggy address belongs to the object at ffff88801f870000
> which belongs to the cache kmalloc-8k of size 8192
> The buggy address is located 5320 bytes inside of
> 8192-byte region [ffff88801f870000, ffff88801f872000)
>
> This area belongs to :
> batadv_priv->batadv_priv_dat->delayed_work->timer_list
>
> The reason for these issues is the lack of synchronization. Delayed
> work (batadv_dat_purge) schedules new timer/work while the device
> is being deleted. As the result new timer/delayed work is set after
> cancel_delayed_work_sync() was called. So after the device is freed
> the timer list contains pointer to already freed memory.
>
> Found by Linux Verification Center (linuxtesting.org) with syzkaller.
>
> Fixes: 2f1dfbe18507 ("batman-adv: Distributed ARP Table - implement local storage")
> Signed-off-by: Vladislav Efanov <[email protected]>
> ---


Acked-by: Sven Eckelmann <[email protected]>

Thanks,
Sven


Attachments:
signature.asc (849.00 B)
This is a digitally signed message part.