Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <2a0dcd75-5461-5266-2749-808f638f4c50@riseup.net>
Date:   Tue, 2 Aug 2022 15:43:32 -0300
MIME-Version: 1.0
Subject: Re: [PATCH 0/3] Introduce KUNIT_EXPECT_ARREQ and KUNIT_EXPECT_ARRNEQ
 macros
Content-Language: en-US
To:     Daniel Latypov <dlatypov@google.com>,
        =?UTF-8?Q?Ma=c3=adra_Canal?= <mairacanal@riseup.net>
Cc:     Brendan Higgins <brendanhiggins@google.com>, davidgow@google.com,
        airlied@linux.ie, daniel@ffwll.ch, davem@davemloft.net,
        kuba@kernel.org, jose.exposito89@gmail.com, javierm@redhat.com,
        andrealmeid@riseup.net, melissa.srw@gmail.com,
        siqueirajordao@riseup.net, Isabella Basso <isabbasso@riseup.net>,
        magalilemes00@gmail.com, tales.aparecida@gmail.com,
        linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com,
        linux-kernel@vger.kernel.org
References: <20220802161206.228707-1-mairacanal@riseup.net>
 <CAGS_qxoAc934AwB7SZ34PpoVxvF3Eua-g+aO77kjJf6d1m0+Qw@mail.gmail.com>
From:   =?UTF-8?Q?Ma=c3=adra_Canal?= <mairacanal@riseup.net>
In-Reply-To: <CAGS_qxoAc934AwB7SZ34PpoVxvF3Eua-g+aO77kjJf6d1m0+Qw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 8/2/22 13:59, 'Daniel Latypov' via KUnit Development wrote:
> On Tue, Aug 2, 2022 at 9:12 AM Maíra Canal <mairacanal@riseup.net> wrote:
>>
>> Currently, in order to compare arrays in KUnit, the KUNIT_EXPECT_EQ or
>> KUNIT_EXPECT_FALSE macros are used in conjunction with the memcmp function,
>> such as:
>>   KUNIT_EXPECT_EQ(test, memcmp(foo, bar, size), 0);
>>
>> Although this usage produces correct results for the test cases, if the
>> expectation fails the error message is not very helpful, indicating only the
>> return of the memcmp function.
>>
>> Therefore, create a new set of macros KUNIT_EXPECT_ARREQ and
>> KUNIT_EXPECT_ARRNEQ that compare memory blocks until a determined size. In
>> case of expectation failure, those macros print the hex dump of the memory
>> blocks, making it easier to debug test failures for arrays.
> 
> I totally agree with this.
> 
> The only reason I hadn't sent an RFC out for this so far is
> * we didn't have enough use cases quite yet (now resolved)
> * I wasn't sure how we'd want to format the failure message.
> 
> For the latter, right now this series produces
> dst ==
> 00000000: 33 0a 60 12 00 a8 00 00 00 00 8e 6b 33 0a 60 12
> 00000010: 00 00 00 00 00 a8 8e 6b 33 0a 00 00 00 00
> result->expected ==
> 00000000: 31 0a 60 12 00 a8 00 00 00 00 81 6b 33 0a 60 12
> 00000010: 00 00 00 00 01 a8 8e 6b 33 0a 00 00 00 00
> 
> I was thinking something like what KASAN produces would be nice, e.g.
> from https://www.kernel.org/doc/html/v5.19/dev-tools/kasan.html#error-reports
> (I'll paste the bit here, but my email client doesn't support
> monospaced fonts, so it won't look nice on my end)
> 
> Memory state around the buggy address:
>  ffff8801f44ec200: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
>  ffff8801f44ec280: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
>> ffff8801f44ec300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03
>                                                                 ^
> I just wasn't quite sure how to do it for a diff, since this only
> really works well when showing one bad byte.
> If we blindly followed that approach, we get
> 
> dst ==
>> 00000000: 33 0a 60 12 00 a8 00 00 00 00 8e 6b 33 0a 60 12
>                                           ^
>> 00000010: 00 00 00 00 00 a8 8e 6b 33 0a 00 00 00 00
>                         ^
> result->expected ==
>> 00000000: 31 0a 60 12 00 a8 00 00 00 00 81 6b 33 0a 60 12
>                                           ^
>> 00000010: 00 00 00 00 01 a8 8e 6b 33 0a 00 00 00 00
>                         ^
> 
> But perhaps we could instead highlight the bad bytes with something like
> dst ==
> 00000000: 33 0a 60 12 00 a8 00 00 00 00 <8e> 6b 33 0a 60 12
> 00000010: 00 00 00 00 <00> a8 8e 6b 33 0a 00 00 00 00
> result->expected ==
> 00000000: 31 0a 60 12 00 a8 00 00 00 00 <81> 6b 33 0a 60 12
> 00000010: 00 00 00 00 <01> a8 8e 6b 33 0a 00 00 00 00

My problem with this approach is that the bytes get slightly misaligned
when adding the <>. Maybe if we aligned as:

dst:
00000000: <33> 0a 60 12  00  a8 00 00 00 00 <8e> 6b 33 0a 60 12
00000010:  00  00 00 00 <00> a8 8e 6b 33 0a  00  00 00 00
result->expected:
00000000: <31> 0a 60 12  00  a8 00 00 00 00 <81> 6b 33 0a 60 12
00000010:  00  00 00 00 <01> a8 8e 6b 33 0a  00  00 00 00

Although I don't know exactly how we can produce this output. I was
using hex_dump_to_buffer to produce the hexdump, so maybe I need to
change the strategy to generate the hexdump.

I guess the KASAN approach could be easier to implement. But I guess it
can turn out to be a little polluted if many bytes differ. For example:

dst:
00000000: 33 31 31 31 31 31 31 31 31 31 8e 31 33 0a 60 12
           ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^
00000010: 00 00 00 00 00 a8 8e 6b 33 0a 00 00 00 00
                       ^
result->expected:
00000000: 31 0a 60 12 00 a8 00 00 00 00 81 6b 33 0a 60 12
           ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^
00000010: 00 00 00 00 01 a8 8e 6b 33 0a 00 00 00 00
                       ^

I don't know exactly with option I lean.


Thank you for your inputs, Daniel!

- Maíra Canal

> 
> Thoughts, suggestions?
>