Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220416081355.2155050-1-jcmvbkbc@gmail.com> <CANpmjNNW0kLf2Ou6i_dNeRLO=Qrru4bOEfJ=be=Dfig4wnQ67g@mail.gmail.com>
In-Reply-To: <CANpmjNNW0kLf2Ou6i_dNeRLO=Qrru4bOEfJ=be=Dfig4wnQ67g@mail.gmail.com>
From:   Max Filippov <jcmvbkbc@gmail.com>
Date:   Tue, 19 Apr 2022 19:59:00 -0700
Message-ID: <CAMo8BfJM0JHqh8Nz3LuK7Ccu7WB1Cup0mX+RYvO1yft_K4hyLQ@mail.gmail.com>
Subject: Re: [PATCH] xtensa: enable KCSAN
To:     Marco Elver <elver@google.com>
Cc:     "open list:TENSILICA XTENSA PORT (xtensa)" 
        <linux-xtensa@linux-xtensa.org>, Chris Zankel <chris@zankel.net>,
        LKML <linux-kernel@vger.kernel.org>,
        Dmitry Vyukov <dvyukov@google.com>, kasan-dev@googlegroups.com
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

Hi Marco,

On Tue, Apr 19, 2022 at 3:16 AM Marco Elver <elver@google.com> wrote:
>
> Nice to see this happen!
>
> On Sat, 16 Apr 2022 at 10:14, Max Filippov <jcmvbkbc@gmail.com> wrote:
> > Provide stubs for 64-bit atomics when building with KCSAN.
>
> The stubs are the only thing I don't understand. More elaboration on
> why this is required would be useful (maybe there's another way to
> solve?).

It doesn't build without it, because the compiler left function calls
in the code:

xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic32_compare_exchange_val':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_load_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic64_load':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_load_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.c:1262: undefined reference
to `__atomic_store_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic64_store':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_store_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.c:1262: undefined reference
to `__atomic_exchange_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic64_exchange':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_exchange_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.c:1262: undefined reference
to `__atomic_fetch_add_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic64_fetch_add':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_fetch_add_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.c:1262: undefined reference
to `__atomic_fetch_sub_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic64_fetch_sub':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_fetch_sub_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.c:1262: undefined reference
to `__atomic_fetch_and_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic64_fetch_and':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_fetch_and_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.c:1262: undefined reference
to `__atomic_fetch_or_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic64_fetch_or':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_fetch_or_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.c:1262: undefined reference
to `__atomic_fetch_xor_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic64_fetch_xor':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_fetch_xor_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.c:1262: undefined reference
to `__atomic_fetch_nand_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic64_fetch_nand':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_fetch_nand_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.c:1262: undefined reference
to `__atomic_compare_exchange_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic64_compare_exchange_strong':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_compare_exchange_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.c:1262: undefined reference
to `__atomic_compare_exchange_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.o: in function
`__tsan_atomic64_compare_exchange_weak':
kernel/kcsan/core.c:1262: undefined reference to `__atomic_compare_exchange_8'
xtensa-de233_fpu-elf-ld: kernel/kcsan/core.c:1262: undefined reference
to `__atomic_compare_exchange_8'

None of these functions are called because xtensa doesn't have
64-bit atomic ops.

I guess that another way to fix it would be making
DEFINE_TSAN_ATOMIC_OPS(64);
conditional and not enabling it when building for xtensa.

> > Disable KCSAN instrumentation in arch/xtensa/boot.
>
> Given you went for barrier instrumentation, I assume you tested with a
> CONFIG_KCSAN_STRICT=y config?

Yes.

> Did the kcsan_test pass?

current results are the following on QEMU:

     # test_missing_barrier: EXPECTATION FAILED at
kernel/kcsan/kcsan_test.c:1313
     Expected match_expect to be true, but is false
     # test_atomic_builtins_missing_barrier: EXPECTATION FAILED at
kernel/kcsan/kcsan_test.c:1356
     Expected match_expect to be true, but is false
 # kcsan: pass:27 fail:2 skip:0 total:29
 # Totals: pass:193 fail:4 skip:0 total:197

and the following on the real hardware:

    # test_concurrent_races: EXPECTATION FAILED at kernel/kcsan/kcsan_test.c:762
    Expected match_expect to be true, but is false
    # test_write_write_struct_part: EXPECTATION FAILED at
kernel/kcsan/kcsan_test.c:910
    Expected match_expect to be true, but is false
    # test_assert_exclusive_access_writer: EXPECTATION FAILED at
kernel/kcsan/kcsan_test.c:1077
    Expected match_expect_access_writer to be true, but is false
    # test_assert_exclusive_bits_change: EXPECTATION FAILED at
kernel/kcsan/kcsan_test.c:1098
    Expected match_expect to be true, but is false
    # test_assert_exclusive_writer_scoped: EXPECTATION FAILED at
kernel/kcsan/kcsan_test.c:1136
    Expected match_expect_start to be true, but is false
    # test_missing_barrier: EXPECTATION FAILED at kernel/kcsan/kcsan_test.c:1313
    Expected match_expect to be true, but is false
    # test_atomic_builtins_missing_barrier: EXPECTATION FAILED at
kernel/kcsan/kcsan_test.c:1356
    Expected match_expect to be true, but is false
# kcsan: pass:22 fail:7 skip:0 total:29
# Totals: pass:177 fail:20 skip:0 total:197

-- 
Thanks.
-- Max