2023-10-14 03:17:13

by Mirsad Todorovac

[permalink] [raw]
Subject: [PATCH v1 3/3] r8169: fix the KCSAN reported data race in rtl_rx while reading desc->opts1

KCSAN reported the following data-race bug:

==================================================================
BUG: KCSAN: data-race in rtl8169_poll (drivers/net/ethernet/realtek/r8169_main.c:4430 drivers/net/ethernet/realtek/r8169_main.c:4583) r8169

race at unknown origin, with read to 0xffff888117e43510 of 4 bytes by interrupt on cpu 21:
rtl8169_poll (drivers/net/ethernet/realtek/r8169_main.c:4430 drivers/net/ethernet/realtek/r8169_main.c:4583) r8169
__napi_poll (net/core/dev.c:6527)
net_rx_action (net/core/dev.c:6596 net/core/dev.c:6727)
__do_softirq (kernel/softirq.c:553)
__irq_exit_rcu (kernel/softirq.c:427 kernel/softirq.c:632)
irq_exit_rcu (kernel/softirq.c:647)
sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1074 (discriminator 14))
asm_sysvec_apic_timer_interrupt (./arch/x86/include/asm/idtentry.h:645)
cpuidle_enter_state (drivers/cpuidle/cpuidle.c:291)
cpuidle_enter (drivers/cpuidle/cpuidle.c:390)
call_cpuidle (kernel/sched/idle.c:135)
do_idle (kernel/sched/idle.c:219 kernel/sched/idle.c:282)
cpu_startup_entry (kernel/sched/idle.c:378 (discriminator 1))
start_secondary (arch/x86/kernel/smpboot.c:210 arch/x86/kernel/smpboot.c:294)
secondary_startup_64_no_verify (arch/x86/kernel/head_64.S:433)

value changed: 0x80003fff -> 0x3402805f

Reported by Kernel Concurrency Sanitizer on:
CPU: 21 PID: 0 Comm: swapper/21 Tainted: G L 6.6.0-rc2-kcsan-00143-gb5cbe7c00aa0 #41
Hardware name: ASRock X670E PG Lightning/X670E PG Lightning, BIOS 1.21 04/26/2023
==================================================================

drivers/net/ethernet/realtek/r8169_main.c:
==========================================
4429
→ 4430 status = le32_to_cpu(desc->opts1);
4431 if (status & DescOwn)
4432 break;
4433
4434 /* This barrier is needed to keep us from reading
4435 * any other fields out of the Rx descriptor until
4436 * we know the status of DescOwn
4437 */
4438 dma_rmb();
4439
4440 if (unlikely(status & RxRES)) {
4441 if (net_ratelimit())
4442 netdev_warn(dev, "Rx ERROR. status = %08x\n",

Marco Elver explained that dma_rmb() doesn't prevent the compiler to tear up the access to
desc->opts1 which can be written to concurrently. READ_ONCE() should prevent that from
happening:

4429
→ 4430 status = le32_to_cpu(READ_ONCE(desc->opts1));
4431 if (status & DescOwn)
4432 break;
4433

As the consequence of this fix, this KCSAN warning was eliminated.

Fixes: 6202806e7c03a ("r8169: drop member opts1_mask from struct rtl8169_private")
Suggested-by: Marco Elver <[email protected]>
Cc: Heiner Kallweit <[email protected]>
Cc: [email protected]
Cc: "David S. Miller" <[email protected]>
Cc: Eric Dumazet <[email protected]>
Cc: Jakub Kicinski <[email protected]>
Cc: Paolo Abeni <[email protected]>
Cc: [email protected]
Link: https://lore.kernel.org/lkml/[email protected]/
Signed-off-by: Mirsad Goran Todorovac <[email protected]>

Signed-off-by: Mirsad Goran Todorovac <[email protected]>
---
drivers/net/ethernet/realtek/r8169_main.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c
index 281aaa851847..81be6085a480 100644
--- a/drivers/net/ethernet/realtek/r8169_main.c
+++ b/drivers/net/ethernet/realtek/r8169_main.c
@@ -4427,7 +4427,7 @@ static int rtl_rx(struct net_device *dev, struct rtl8169_private *tp, int budget
dma_addr_t addr;
u32 status;

- status = le32_to_cpu(desc->opts1);
+ status = le32_to_cpu(READ_ONCE(desc->opts1));
if (status & DescOwn)
break;

--
2.34.1


2023-10-16 07:36:50

by Marco Elver

[permalink] [raw]
Subject: Re: [PATCH v1 3/3] r8169: fix the KCSAN reported data race in rtl_rx while reading desc->opts1

On Sat, 14 Oct 2023 at 05:16, Mirsad Goran Todorovac
<[email protected]> wrote:
>
> KCSAN reported the following data-race bug:
>
> ==================================================================
> BUG: KCSAN: data-race in rtl8169_poll (drivers/net/ethernet/realtek/r8169_main.c:4430 drivers/net/ethernet/realtek/r8169_main.c:4583) r8169
>
> race at unknown origin, with read to 0xffff888117e43510 of 4 bytes by interrupt on cpu 21:
> rtl8169_poll (drivers/net/ethernet/realtek/r8169_main.c:4430 drivers/net/ethernet/realtek/r8169_main.c:4583) r8169
> __napi_poll (net/core/dev.c:6527)
> net_rx_action (net/core/dev.c:6596 net/core/dev.c:6727)
> __do_softirq (kernel/softirq.c:553)
> __irq_exit_rcu (kernel/softirq.c:427 kernel/softirq.c:632)
> irq_exit_rcu (kernel/softirq.c:647)
> sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1074 (discriminator 14))
> asm_sysvec_apic_timer_interrupt (./arch/x86/include/asm/idtentry.h:645)
> cpuidle_enter_state (drivers/cpuidle/cpuidle.c:291)
> cpuidle_enter (drivers/cpuidle/cpuidle.c:390)
> call_cpuidle (kernel/sched/idle.c:135)
> do_idle (kernel/sched/idle.c:219 kernel/sched/idle.c:282)
> cpu_startup_entry (kernel/sched/idle.c:378 (discriminator 1))
> start_secondary (arch/x86/kernel/smpboot.c:210 arch/x86/kernel/smpboot.c:294)
> secondary_startup_64_no_verify (arch/x86/kernel/head_64.S:433)
>
> value changed: 0x80003fff -> 0x3402805f
>
> Reported by Kernel Concurrency Sanitizer on:
> CPU: 21 PID: 0 Comm: swapper/21 Tainted: G L 6.6.0-rc2-kcsan-00143-gb5cbe7c00aa0 #41
> Hardware name: ASRock X670E PG Lightning/X670E PG Lightning, BIOS 1.21 04/26/2023
> ==================================================================
>
> drivers/net/ethernet/realtek/r8169_main.c:
> ==========================================
> 4429
> → 4430 status = le32_to_cpu(desc->opts1);
> 4431 if (status & DescOwn)
> 4432 break;
> 4433
> 4434 /* This barrier is needed to keep us from reading
> 4435 * any other fields out of the Rx descriptor until
> 4436 * we know the status of DescOwn
> 4437 */
> 4438 dma_rmb();
> 4439
> 4440 if (unlikely(status & RxRES)) {
> 4441 if (net_ratelimit())
> 4442 netdev_warn(dev, "Rx ERROR. status = %08x\n",
>
> Marco Elver explained that dma_rmb() doesn't prevent the compiler to tear up the access to
> desc->opts1 which can be written to concurrently. READ_ONCE() should prevent that from
> happening:
>
> 4429
> → 4430 status = le32_to_cpu(READ_ONCE(desc->opts1));
> 4431 if (status & DescOwn)
> 4432 break;
> 4433
>
> As the consequence of this fix, this KCSAN warning was eliminated.
>
> Fixes: 6202806e7c03a ("r8169: drop member opts1_mask from struct rtl8169_private")
> Suggested-by: Marco Elver <[email protected]>
> Cc: Heiner Kallweit <[email protected]>
> Cc: [email protected]
> Cc: "David S. Miller" <[email protected]>
> Cc: Eric Dumazet <[email protected]>
> Cc: Jakub Kicinski <[email protected]>
> Cc: Paolo Abeni <[email protected]>
> Cc: [email protected]
> Link: https://lore.kernel.org/lkml/[email protected]/
> Signed-off-by: Mirsad Goran Todorovac <[email protected]>
>
> Signed-off-by: Mirsad Goran Todorovac <[email protected]>

Double sign-off? Otherwise,

Acked-by: Marco Elver <[email protected]>

> ---
> drivers/net/ethernet/realtek/r8169_main.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c
> index 281aaa851847..81be6085a480 100644
> --- a/drivers/net/ethernet/realtek/r8169_main.c
> +++ b/drivers/net/ethernet/realtek/r8169_main.c
> @@ -4427,7 +4427,7 @@ static int rtl_rx(struct net_device *dev, struct rtl8169_private *tp, int budget
> dma_addr_t addr;
> u32 status;
>
> - status = le32_to_cpu(desc->opts1);
> + status = le32_to_cpu(READ_ONCE(desc->opts1));
> if (status & DescOwn)
> break;
>
> --
> 2.34.1
>

2023-10-16 21:53:01

by Mirsad Todorovac

[permalink] [raw]
Subject: Re: [PATCH v1 3/3] r8169: fix the KCSAN reported data race in rtl_rx while reading desc->opts1



On 10/16/23 09:35, Marco Elver wrote:
> On Sat, 14 Oct 2023 at 05:16, Mirsad Goran Todorovac
> <[email protected]> wrote:
>>
>> KCSAN reported the following data-race bug:
>>
>> ==================================================================
>> BUG: KCSAN: data-race in rtl8169_poll (drivers/net/ethernet/realtek/r8169_main.c:4430 drivers/net/ethernet/realtek/r8169_main.c:4583) r8169
>>
>> race at unknown origin, with read to 0xffff888117e43510 of 4 bytes by interrupt on cpu 21:
>> rtl8169_poll (drivers/net/ethernet/realtek/r8169_main.c:4430 drivers/net/ethernet/realtek/r8169_main.c:4583) r8169
>> __napi_poll (net/core/dev.c:6527)
>> net_rx_action (net/core/dev.c:6596 net/core/dev.c:6727)
>> __do_softirq (kernel/softirq.c:553)
>> __irq_exit_rcu (kernel/softirq.c:427 kernel/softirq.c:632)
>> irq_exit_rcu (kernel/softirq.c:647)
>> sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1074 (discriminator 14))
>> asm_sysvec_apic_timer_interrupt (./arch/x86/include/asm/idtentry.h:645)
>> cpuidle_enter_state (drivers/cpuidle/cpuidle.c:291)
>> cpuidle_enter (drivers/cpuidle/cpuidle.c:390)
>> call_cpuidle (kernel/sched/idle.c:135)
>> do_idle (kernel/sched/idle.c:219 kernel/sched/idle.c:282)
>> cpu_startup_entry (kernel/sched/idle.c:378 (discriminator 1))
>> start_secondary (arch/x86/kernel/smpboot.c:210 arch/x86/kernel/smpboot.c:294)
>> secondary_startup_64_no_verify (arch/x86/kernel/head_64.S:433)
>>
>> value changed: 0x80003fff -> 0x3402805f
>>
>> Reported by Kernel Concurrency Sanitizer on:
>> CPU: 21 PID: 0 Comm: swapper/21 Tainted: G L 6.6.0-rc2-kcsan-00143-gb5cbe7c00aa0 #41
>> Hardware name: ASRock X670E PG Lightning/X670E PG Lightning, BIOS 1.21 04/26/2023
>> ==================================================================
>>
>> drivers/net/ethernet/realtek/r8169_main.c:
>> ==========================================
>> 4429
>> → 4430 status = le32_to_cpu(desc->opts1);
>> 4431 if (status & DescOwn)
>> 4432 break;
>> 4433
>> 4434 /* This barrier is needed to keep us from reading
>> 4435 * any other fields out of the Rx descriptor until
>> 4436 * we know the status of DescOwn
>> 4437 */
>> 4438 dma_rmb();
>> 4439
>> 4440 if (unlikely(status & RxRES)) {
>> 4441 if (net_ratelimit())
>> 4442 netdev_warn(dev, "Rx ERROR. status = %08x\n",
>>
>> Marco Elver explained that dma_rmb() doesn't prevent the compiler to tear up the access to
>> desc->opts1 which can be written to concurrently. READ_ONCE() should prevent that from
>> happening:
>>
>> 4429
>> → 4430 status = le32_to_cpu(READ_ONCE(desc->opts1));
>> 4431 if (status & DescOwn)
>> 4432 break;
>> 4433
>>
>> As the consequence of this fix, this KCSAN warning was eliminated.
>>
>> Fixes: 6202806e7c03a ("r8169: drop member opts1_mask from struct rtl8169_private")
>> Suggested-by: Marco Elver <[email protected]>
>> Cc: Heiner Kallweit <[email protected]>
>> Cc: [email protected]
>> Cc: "David S. Miller" <[email protected]>
>> Cc: Eric Dumazet <[email protected]>
>> Cc: Jakub Kicinski <[email protected]>
>> Cc: Paolo Abeni <[email protected]>
>> Cc: [email protected]
>> Link: https://lore.kernel.org/lkml/[email protected]/
>> Signed-off-by: Mirsad Goran Todorovac <[email protected]>
>>
>> Signed-off-by: Mirsad Goran Todorovac <[email protected]>
>
> Double sign-off? Otherwise,
>
> Acked-by: Marco Elver <[email protected]>

Missed that one, sorry.

Fixed it in v2 of the patch.

Best regards,
Mirsad Todorovac

>> ---
>> drivers/net/ethernet/realtek/r8169_main.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c
>> index 281aaa851847..81be6085a480 100644
>> --- a/drivers/net/ethernet/realtek/r8169_main.c
>> +++ b/drivers/net/ethernet/realtek/r8169_main.c
>> @@ -4427,7 +4427,7 @@ static int rtl_rx(struct net_device *dev, struct rtl8169_private *tp, int budget
>> dma_addr_t addr;
>> u32 status;
>>
>> - status = le32_to_cpu(desc->opts1);
>> + status = le32_to_cpu(READ_ONCE(desc->opts1));
>> if (status & DescOwn)
>> break;
>>
>> --
>> 2.34.1
>>