Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 5 Sep 2022 15:11:25 +0200
From:   =?utf-8?Q?Micha=C5=82?= Winiarski <michal.winiarski@intel.com>
To:     Maxime Ripard <maxime@cerno.tech>
CC:     Jani Nikula <jani.nikula@linux.intel.com>,
        =?utf-8?B?TWHDrXJh?= Canal <mairacanal@riseup.net>,
        Isabella Basso <isabbasso@riseup.net>,
        <magalilemes00@gmail.com>, <tales.aparecida@gmail.com>,
        <mwen@igalia.com>, <andrealmeid@riseup.net>,
        <siqueirajordao@riseup.net>, Trevor Woerner <twoerner@gmail.com>,
        Daniel Vetter <daniel@ffwll.ch>,
        David Airlie <airlied@linux.ie>,
        Javier Martinez Canillas <javierm@redhat.com>,
        David Gow <davidgow@google.com>, <brendanhiggins@google.com>,
        Arthur Grillo <arthur.grillo@usp.br>,
        =?utf-8?B?Sm9zw6kgRXhww7NzaXRv?= <jose.exposito89@gmail.com>,
        <linux-kernel@vger.kernel.org>, <dri-devel@lists.freedesktop.org>,
        <kunit-dev@googlegroups.com>
Subject: Re: [PATCH v2 2/2] drm/tests: Change "igt_" prefix to "test_drm_"
Message-ID: <20220905131125.bbqxihhipc37oshq@nostramo>
References: <20220901124210.591994-1-mairacanal@riseup.net>
 <20220901124210.591994-2-mairacanal@riseup.net>
 <20220901125530.b56s4zisnkfuigvc@houat>
 <04aeba53-793c-3196-3137-915f0640dc2a@riseup.net>
 <87h71qfbi9.fsf@intel.com>
 <20220902123400.5ljgc7z6zw34d64m@houat>
 <87mtbidj3b.fsf@intel.com>
 <20220902133828.ufwp6bgzd37yu6bv@nostramo.hardline.pl>
 <20220905121000.m2xppgjlfjlmppfr@houat>
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20220905121000.m2xppgjlfjlmppfr@houat>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Q2FjbGNSdllxTWFCZkNGK3ByUmMrd1pkUzU4aXhPclZxT2txWUpRQVM3VmNi?=
 =?utf-8?B?V2ZmZ0RyS3hkMzV5dzZNYzdSMmNHNDF5NE9TZE1qSXNHd2dHYVBmMDNXZGwy?=
 =?utf-8?B?T1ZwUVpvR3RCSFB1UTVlU1hEdU9qeUU1Tm96RWlHTGY5Q2NUNVlWTUFveWtT?=
 =?utf-8?B?TW55M3FYRCt6aWM1UG12cnBWaHlrSDNxSXhIUGZ6QXhwcHg1Qm43QWVoZGh6?=
 =?utf-8?B?SGg3dnBreTIzNUphckEvdU1JN0pSM0xwNkJoV0xTa201ZndBMVRteE1VUXNX?=
 =?utf-8?B?MW02WEhSRG9rTVBraGtYVy9tVWN6eWxHSmIrUmplOCt1b0FlUVF2bDFjanVn?=
 =?utf-8?B?Nm1iUU15cHFwQVNHMWZCZ0luWklnUTB2SERLdzJDNHptcnhpU2t5VmtxTGph?=
 =?utf-8?B?TWxuUGlza3FSSE54ZVp6T2pIU1FWRCtGL0FjNkZNNk9iT2o3Yk9QYmNCU25n?=
 =?utf-8?B?b2g4TENKUXZ3L2NoUkdwQ3Fpc2M1WHdVLzNiTlZKU3NsdDRHa1RPRFU4MFkr?=
 =?utf-8?B?aHB1Wnh3MVFXMHpSN0ptUXZGMzZjbmtRQW04YkU2eS9vVFgvcVZRL2Q1bVVh?=
 =?utf-8?B?MDFJOThuZ0lneDFySkZuUXhvbWRZQkNZODVDZ2JsdEVmU0RCUGhGdWpsMEli?=
 =?utf-8?B?MUlhUHJjODgva0lTME5BemorYXdnY2ZNQVdGdU8yOVZ1dUVIRXZFS29NeEpM?=
 =?utf-8?B?RkdZZ0ljRjY5Y1dhTjlsWlJjdVo5bEdpb3h5ZVZYVlFKVTgvemFBalp1cXBV?=
 =?utf-8?B?eFBsdWxOcG05OVpHSjRGN00vRWI5S1BiSjBWZmJHcFJTa0h5S1NZRHNzdG51?=
 =?utf-8?B?dkJrUVdMNk1rUXNYNUV6RmJ2cHBnVjhXdTV3UFpuSDZiSjRaYVd4OWdYOGNw?=
 =?utf-8?B?UmJ6clIyd09ySmRqYVJtd3RZY0ZSYTg3TE9BY0Y2UlU4dFFqRjBKck9hZUhK?=
 =?utf-8?B?VGIzS3U0Wm9kbFkzampQT3RKOVVrOTNORkJGYWQzNk85QitXZmkzN0lGWTN4?=
 =?utf-8?B?VnorQ2p3QmlkNGxUT3J5ZGxoWXAvRG1nWCtqRkFPNzlwcEljQlVBRENWQVo1?=
 =?utf-8?B?Mlc1T0FMVWRQLzhPamQ0enpDelEvRFlJbXJ3OEdSaVd2SElFdWVRV0VOM0Vo?=
 =?utf-8?B?UjBFbFQ1eHJxWlpHZXJhUWxiM2ovWFBibG9kRm05RlFnWmhlUzIxdldWS1Ez?=
 =?utf-8?B?TUJiVXRxSnB4b01wTjB4d01PNDNVcDBvYU9UcGtzY1prd3VtYk9vekZ3cklq?=
 =?utf-8?B?TXFjZm02VFJhZkpSZ1pIbjNJcFNRTmI0T25aSk16TnNnTC8wR3dBL2Fvc2NE?=
 =?utf-8?B?ZVJ1TUhvbE9ad2t0MlJlSnRxSXc3L0FjbGViMU1rNnE4WldrMmdodkI5Q3Zu?=
 =?utf-8?B?VmthTnVVdW1Ta2syU3ZEWGxXU24vejJGL0gvd0tRY1pVT004SC9vZ0VvUEd4?=
 =?utf-8?B?MHlJNGVIWVBvaHNmQVhyaTYyYXlITmpSM3B3Tmh3cFp5MjZ0NWpydE1yeXNh?=
 =?utf-8?B?ZDB2SlVjMll2dHgrclFhOHdSNWJoTVFyNTZPMEgyckxEbWxmZHRhNVF5Rklx?=
 =?utf-8?B?bERYbjFGekRtS1UwaHdBODdBRVhXWC9zTUVLSXc2bCtuTDUrbXFVaUR1bnY2?=
 =?utf-8?B?Yk1Zei9ieU1tRHRmZUtSQzgzM3JadlhKSFpraXdKcjFLN3VzMXpyRjlqQUJ4?=
 =?utf-8?B?T0o4L1NrUmFWWGdBTHpjR2pTbEZhakJSMUZNdHlPNXQyVzZDWjFYanl5aytS?=
 =?utf-8?B?UzZQNVdGQlZFK2Z2dytocVhidVR4Zmt2V1Jmd1pzL1hJOFZESEtiYndidUYy?=
 =?utf-8?B?UTFBU2QzQkNGdlhQM2lkVXVjcEVId090clhXSHF6WTNLWWlweGhFVVlWVTNm?=
 =?utf-8?B?SkpiTU56QjZIN1oxSlVKM0l6Z2hodVNmUHhsZUYyb0RrYkN5UHBUR1pSWGNZ?=
 =?utf-8?B?ME9Mek5zMllza0VSckFiM01VcTYyM1dRR1J1UGk3YVZiM252bTlPaHBPNEU2?=
 =?utf-8?B?V0FRb3JNRDRlL3NnMTZXRnNnTllHcWhPSFpBeGNOdytCS1Nwb0llVmZZMHBv?=
 =?utf-8?B?UTdjRjRKM1RBVmtib0g2Z0RxUnJtRzNHUTdqazVWcWVpVXBWS3o1b1NpZmlj?=
 =?utf-8?B?QzdHMW8zSlN6QU56QUwrakZVWVNtcmdMSVhJajRJb1Vydm1OaXJ3aXZaK2dR?=
 =?utf-8?B?alE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 6a5a8e1f-0869-4c57-af4b-08da8f40278b
X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5373.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Sep 2022 13:11:33.2365
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: n8XSXX3JOAlgIFHyYg0DkCjhPqFQAWpTS1102AXkI+K2nbV4coDYxIgnjpbzoHJmdom4GBToMc9dafNTP4k/DdDnnvfx5OaJ1nle8PrZkfE=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB7102
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, Sep 05, 2022 at 02:10:00PM +0200, Maxime Ripard wrote:
> On Fri, Sep 02, 2022 at 03:38:28PM +0200, Michał Winiarski wrote:
> > On Fri, Sep 02, 2022 at 04:03:20PM +0300, Jani Nikula wrote:
> > > On Fri, 02 Sep 2022, Maxime Ripard <maxime@cerno.tech> wrote:
> > > > On Fri, Sep 02, 2022 at 11:04:14AM +0300, Jani Nikula wrote:
> > > >> On Thu, 01 Sep 2022, Maíra Canal <mairacanal@riseup.net> wrote:
> > > >> > Hi Maxime,
> > > >> >
> > > >> > On 9/1/22 09:55, Maxime Ripard wrote:
> > > >> >> Hi,
> > > >> >> 
> > > >> >> On Thu, Sep 01, 2022 at 09:42:10AM -0300, Maíra Canal wrote:
> > > >> >>> With the introduction of KUnit, IGT is no longer the only option to run
> > > >> >>> the DRM unit tests, as the tests can be run through kunit-tool or on
> > > >> >>> real hardware with CONFIG_KUNIT.
> > > >> >>>
> > > >> >>> Therefore, remove the "igt_" prefix from the tests and replace it with
> > > >> >>> the "test_drm_" prefix, making the tests' names independent from the tool
> > > >> >>> used.
> > > >> >>>
> > > >> >>> Signed-off-by: Maíra Canal <mairacanal@riseup.net>
> > > >> >>>
> > > >> >>> ---
> > > >> >>> v1 -> v2: https://lore.kernel.org/dri-devel/20220830211603.191734-1-mairacanal@riseup.net/
> > > >> >>> - Change "drm_" prefix to "test_drm_", as "drm_" can be a bit confusing (Jani Nikula).
> > > >> >> 
> > > >> >> I appreciate it's a bit of a bikeshed but I disagree with this. The
> > > >> >> majority of the kunit tests already out there start with the framework
> > > >> >> name, including *all* the examples in the kunit doc. Plus, it's fairly
> > > >> >> obvious that it's a test, kunit is only about running tests in the first
> > > >> >> place.
> > > >> >
> > > >> > Would it be better to keep it as "drm_"?
> > > >> 
> > > >> That's not "keeping". That's renaming igt to drm.
> > > >
> > > > Well, there's like half the tests that are prefixed with drm, the other
> > > > with igt, so it's both really
> > > >
> > > >> > Currently, I don't think it is appropriate to hold the "igt_" prefix, as
> > > >> > the tests are not IGT exclusive, but I don't have a strong opinion on
> > > >> > using the "drm_" or the "test_drm" prefixes.
> > > >> 
> > > >> I repeat my stance that "drm_" alone is confusing.
> > > >
> > > > What are you confusing it with?
> > > >
> > > >> For the reason alone that it pollutes the code tagging tools, mixing
> > > >> actual drm_ types and functions with unit test functions.
> > > >
> > > > I don't get it, I'm sorry. All these functions are static and not part
> > > > of any API, so I can't see how it would pollute a code tagging tool. Or
> > > > at least, not more than any driver does.
> > > >
> > > > And we're part of a larger project here, it's about consistency with the
> > > > rest of the ecosystem.
> > > 
> > > Okay, so I'm just going to explain what I mean, but say "whatever" right
> > > after and move on.
> > > 
> > > For example, drm_buddy_test.c includes drm_buddy.h so with the igt_ ->
> > > drm_ rename none of the test functions may clash with the drm_buddy_
> > > prefixed existing functions. Ditto for all tests similarly.
> > > 
> > > For example drm_buddy_alloc_range() as a name sounds like something that
> > > allocs a range, not something that tests range allocation. On the other
> > > hand, you have drm_buddy_alloc_blocks() which is actually a real
> > > drm_buddy function, not a test. What would you call a test that tests
> > > that? Here, we end up with names that are all prefixed drm_buddy and you
> > > won't know what's the actual function and what's the test unless you
> > > look it up.
> > > 
> > > I use code tagging that I can use for finding and completing
> > > e.g. functions. Currently I have the following completions, for
> > > igt_buddy_ and drm_buddy_, respectively:
> > > 
> > > Possible completions are:
> > > igt_buddy_alloc_limit 	igt_buddy_alloc_optimistic 	igt_buddy_alloc_pathological
> > > igt_buddy_alloc_pessimistic 	igt_buddy_alloc_range 	igt_buddy_alloc_smoke
> > > 
> > > Possible completions are:
> > > drm_buddy_alloc_blocks 	drm_buddy_block 	drm_buddy_block_is_allocated 	drm_buddy_block_is_free
> > > drm_buddy_block_is_split 	drm_buddy_block_offset 	drm_buddy_block_order 	drm_buddy_block_print
> > > drm_buddy_block_size 	drm_buddy_block_state 	drm_buddy_block_trim 	drm_buddy_fini
> > > drm_buddy_free_block 	drm_buddy_free_list 	drm_buddy_init 	drm_buddy_init_test
> > > drm_buddy_module_exit 	drm_buddy_module_init 	drm_buddy_print
> > > 
> > > With the patch at hand, they'll all be lumped under drm_buddy_
> > > completions, and some of them will be actual drm buddy functions and
> > > some not.
> > > 
> > > I just find it a very odd convention to name the tests in a way that's
> > > indistinguishable from the real things. Even *within* drm_buddy_test.c
> > > where you read the test code. Because currently you do have calls to
> > > igt_buddy_ prefixed functions from other igt_buddy_ prefixed functions,
> > > along with the drm_buddy_ prefixed calls. I think it's just going to be
> > > a mess.
> > > 
> > > /rant
> > > 
> > > Whatever. Moving on.
> > 
> > KUnit docs [1] state:
> > https://docs.kernel.org/dev-tools/kunit/style.html#test-cases
> > "As tests are themselves functions, their names cannot conflict with other
> > C identifiers in the kernel. This may require some creative naming."
> > And give examples of names. But this should be local to individual test suite -
> > as long as the test is readable, and the name describes what it is testing, we
> > should be fine. We don't even need to pass drm_* prefix, as this convention is
> > expected for test suites, not test cases [2].
> > 
> > Having said that - I do believe that igt_* prefix don't belong here (which is
> > why I'm progressively trying to get rid of in the patches that refactor some of
> > the tests).
> > I agree with Jani - can we take it on a case-by-case basis?
> > If the test name is too similar to the function that it is testing, we could go
> > with one of the following (taking igt_buddy_alloc_limit as example):
> > drm_buddy_test_alloc_limit
> > test_drm_buddy_alloc_limit
> > buddy_test_alloc_limit
> > test_buddy_alloc_limit
> 
> We also have drm_test_buddy_alloc_limit, or drm_buddy_test_alloc_limit
> 
> Both would be fine for me, with a small preference for the former, which
> should also address Jani's concerns?
> 
> > And either of those is fine in my opinion (I'd personally go with
> > test_buddy_alloc_limit in this case).
> > We don't really need a DRM-wide (or worse, kernel wide) convention for test case
> > names (it's only really needed for test suites).
> 
> Sure we do. kunit.py can take some filters too. Being able to only run
> DRM tests with a single filter is super convenient, and if we fail to
> provide a consistent naming we're pretty much sure people running the
> tests are going to miss some.

We have "test suite" - and we do have a naming convention for that.
Filter glob takes test_suite.test_name.

Let's use specific examples.

Starting with igt_buddy_alloc_limit, "full" test name is
"drm_buddy.igt_buddy_alloc_limit" - the way to run this single test would be:
./kunit.py run 'drm_buddy.igt_buddy_alloc_limit'

Say we want to run all drm_buddy tests, we can do it with:
./kunit.py run 'drm_buddy.*'

We can also run all DRM tests:
./kunit.py run 'drm*'

What you're arguing, is that we need to rename all test cases to have drm_test
prefix, so the full testname for igt_buddy_alloc_limit becomes
drm_buddy.drm_test_buddy_alloc_limit, because we want to be able to use this
specific glob for matching by test case names (not test suites):
./kunit.py run '*.drm*'

Why? We can already do that with 'drm*', that's what the test suites are for,
and the glob is simpler and more intuitive IMO.

-Michał

> 
> Maxime