Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <443632b6-c589-ef62-2385-3e8406680343@gmail.com>
Date:   Thu, 18 Aug 2022 15:08:01 -0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.12.0
Subject: Re: Running kunit tests on platform devices
Content-Language: en-US
To:     Ramon Fried <rfried.dev@gmail.com>, David Gow <davidgow@google.com>
Cc:     Brendan Higgins <brendanhiggins@google.com>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@vger.kernel.org>,
        KUnit Development <kunit-dev@googlegroups.com>,
        LKML <linux-kernel@vger.kernel.org>
References: <CAGi-RUKv-8zyku-3pjXdjxbyMP_X_YV5rd6y2wyaCOJJ9UOQAw@mail.gmail.com>
 <CABVgOSnxa=H5A-0U0QRCAv+OWCGb0zN_f7gP5FmhGjRy7NCUSw@mail.gmail.com>
 <CAGi-RUL46gA_0ah_TTJVpc9RRS8nvd7yoqt=OLXxvUjL6TAvyQ@mail.gmail.com>
Reply-To: CAGi-RUL46gA_0ah_TTJVpc9RRS8nvd7yoqt=OLXxvUjL6TAvyQ@mail.gmail.com
From:   Tales Lelo da Aparecida <tales.aparecida@gmail.com>
In-Reply-To: <CAGi-RUL46gA_0ah_TTJVpc9RRS8nvd7yoqt=OLXxvUjL6TAvyQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 17/08/2022 09:19, Ramon Fried wrote:
> On Wed, Aug 17, 2022 at 7:43 AM David Gow <davidgow@google.com> wrote:
>>
>> On Mon, Aug 15, 2022 at 5:16 PM Ramon Fried <rfried.dev@gmail.com> wrote:
>>>
>>> Hi.
>>> I implemented a test suite that checks a platform driver, that's the
>>> only way I can test interrupts behavior in the code.
>>> Because it's a platform, I can't use kunit_test_suite(), so I call
>>> __kunit_test_suites_init() as part of the platform driver probe
>>> function.
>>>
>>> This works fine but has the following problems.
>>> "TAP version 14" string is not printed and it's impossible to parse
>>> the results using the script.
>>> In addition, the suite is not displayed in /sys/kernel/debug/kunit.
>>>
>>> It would be my pleasure to provide a patch that fixes this, I just
>>> wanted to make sure that my testing strategy makes sense.
>>>
>>> Thanks,
>>> Ramon
>>>
>> Hi Ramon,
>>
>> Thanks for reaching out. Calling __kunit_test_suites_init() directly
>> is not something we'd recommend (and are trying desperately to remove
>> existing uses), so several of the issues re: the "TAP version 14"
>> string et al are side effects of calling what is supposed to be an
>> internal KUnit function manually.
>>
>> That being said, we definitely do want to make testing platform
>> drivers as convenient as possible. I'm not sure why kunit_test_suite()
>> doesn't work for you for platform drivers: are you just missing some
>> way of instantiating the device from within a test context?
>>
>> I know Brendan has experimented with some hardware faking code, for
>> which there's some example use here:
>> https://kunit-review.googlesource.com/c/linux/+/5275
>> (Note that you'll need to look at the other patches in the 'relation
>> chain' for dependencies.)
>>
>> Equally, I've experimented a bit with testing old soundcard drivers
>> (via a platform device) here, which may be an easier way to look
>> through:
>> https://github.com/sulix/linux/commit/4e1620c86553b9edde7f032318cf417dc13e4d26
>>
>> Note that neither of those examples are anything other than
>> experiments, so may not work as-is, or be ideal.
>>
>> Otherwise, we're always happy to accept patches, though again, if
>> there's any way of getting your tests working without a direct call to
>> __kunit_test_suites_init() --- even if that would require patches to
>> work --- that'd be preferable on our end.
>>
>> Cheers,
>> -- David
> Hi David,
> Thanks for replying.
> I looked at the examples you shared, and they all fake the actual device.
> My intention is to actually interact with the real device. - get a
> real interrupt, etc.
> Is it wrong, was the intention that the platform device be mocked ?
> Thanks,
> Ramon.
> 

Hi, Ramon,

I particularly don't condemn writing tests that require hardware, but 
they're best avoided, mostly to facilitate running those tests.

Can you share your code?
I would be happy to take a look if its not a problem for you!

Kind regards,
Tales