Received: by 2002:a25:f815:0:0:0:0:0 with SMTP id u21csp1308558ybd; Wed, 26 Jun 2019 15:18:31 -0700 (PDT) X-Google-Smtp-Source: APXvYqxwRVjpE/RA7EWCeCCFM3P7H32bYzxsdoWri422k3qKJF92nF2aHQzb90IcU3iaot0KrpSJ X-Received: by 2002:a65:64d6:: with SMTP id t22mr276971pgv.406.1561587511228; Wed, 26 Jun 2019 15:18:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561587511; cv=none; d=google.com; s=arc-20160816; b=J1LRJIP3lJz5RBifqejYW6fNMiL9yiLrqN0q2AuLmzW23iNfL6/3c3EIcjaQ46/SYf ILy+aR0Th8cDuF68PR/kf8H9zG0rhf/OrQ7xd3NHNa8qAY0zqaNbbfebon4p4if0Srol I6kJrARaxQika3OMi/Lo/7baDwpvW/rN/D0arF5R+jVaRMIcfjibVI3vWtVOeJZGNXmI wxD90ngtZuvPc7TqYim2bQ+ZL1CDM1VEpgGZ9q+pS/4PTWsI/NO2cz+i5vl1LvRKZ9g0 iOejDZJe/Y7Ge19cFmOCobGOCxgAhvUxXvJUTy8hjTZnNrvBw4f1sLOIbRE5XPF1xhU8 zTWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=GeRJM7RinPoMIQ6Cxu4JfD/FJhf/Q7830l4heE4Z/70=; b=I1d+U2HLKTHP/0oCA7wKgCJereqS2GGjmPxQny7ewhuV+8u3RwkdRX91LaGDylLzpj BfGR1r0jTKopfNCQO9AKLig1p95QLITsBVCHaC0FQzvYrwIqAELUTvGHrspEO+Yba22m zKfhF6TmS6pOMRAZ0vyyNSmNjyWyegm6mbnRggGOJhfMNLD5RZR93CEt1ckvX6//s2N0 cLHZ29LImh5abNi+IfrJlw9P+gsCoesOwZ9C7YEILfmF+l0X7dYtkd5wAlda35wnQqAk w1U2A9V2RUpoDEJ1EIthKa3L9lv4hQKegrsjEBk4ci+egiSUiMHrQ5qHG0R+29YnjM5a x7Ew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=S6f2tU7M; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q3si3629544pjb.1.2019.06.26.15.18.14; Wed, 26 Jun 2019 15:18:31 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=S6f2tU7M; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726489AbfFZWQu (ORCPT + 99 others); Wed, 26 Jun 2019 18:16:50 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:46264 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726470AbfFZWQu (ORCPT ); Wed, 26 Jun 2019 18:16:50 -0400 Received: by mail-pf1-f196.google.com with SMTP id 81so127936pfy.13 for ; Wed, 26 Jun 2019 15:16:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=GeRJM7RinPoMIQ6Cxu4JfD/FJhf/Q7830l4heE4Z/70=; b=S6f2tU7Miw4rgiEW47807L33bYsjxaGq9kSV5FlqDefi2WxZ9lTn6WNXPehPhg/T1u QlukqMtHkRexD1aKOxiS1bTcFztoPMq6vRmfl07xvAlF8qkmKEYcRYv6yewHpyrNh/C2 b54UNQQh2z4Ml5QHakDNUwkkVSa3wpee9qOwZX656U2rr12uOzmYMsnmgvSxd7nO2lJu uPRgbm1UzaYtxmtry6zEXpmUHhjd6DhSWjC+3Gn8V/HQxryzbZ7pGyLo06Eq0gUDpp5U DjOJXTOOdEhiALmttRxHwycmxPuBaL40MgKhCWjDKs1lUcvOyyLeVeXFZ4cT8o8pd53u IzAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=GeRJM7RinPoMIQ6Cxu4JfD/FJhf/Q7830l4heE4Z/70=; b=miOEqwFU1M645Pfib+roKcv4+unJw4Fg8b0pDG+M/bWg4iGkbqNQm7refktULrq0L4 zm19r6D1BCLSaCiZC8sHX9Au2NJZotBw3jeYBoqs2u7Wt6UtyUMp79dU/H87QTUicYJn kKvvDuNWr9yqkb8Wbv3gUdHGrAtt+AyTJhSuKX/yRbxvwqEVKscB37YjfoXnISaosUlx nhlgCHFOT4DI3VIxOyve/U6u6pwOZiMBpNvLl4rZRGsJnVE/LwUNzS5/md29k/AgV5t2 GhdLQYDPePE0Zb7U7JLX/34qNcfi7DZOmwHZwYOtTVWuFLMXWtpmRaE92Gukn/qY/8d1 Lg2Q== X-Gm-Message-State: APjAAAVGrzYSaBQzxu1yqoshMmC7Z+IwA2GZ21nyPlpGKuKVLyJXVjR1 9LeiJcMFO9/TuLrqFWP+dQY+0e/Rf42ZW6XiTN8Tag== X-Received: by 2002:a17:90b:f0e:: with SMTP id br14mr1630712pjb.117.1561587408779; Wed, 26 Jun 2019 15:16:48 -0700 (PDT) MIME-Version: 1.0 References: <20190617082613.109131-1-brendanhiggins@google.com> <20190617082613.109131-2-brendanhiggins@google.com> <20190625223312.GP19023@42.do-not-panic.com> <20190626033643.GX19023@42.do-not-panic.com> In-Reply-To: <20190626033643.GX19023@42.do-not-panic.com> From: Brendan Higgins Date: Wed, 26 Jun 2019 15:16:36 -0700 Message-ID: Subject: Re: [PATCH v5 01/18] kunit: test: add KUnit test runner core To: Luis Chamberlain Cc: Frank Rowand , Greg KH , Josh Poimboeuf , Kees Cook , Kieran Bingham , Peter Zijlstra , Rob Herring , Stephen Boyd , shuah , "Theodore Ts'o" , Masahiro Yamada , devicetree , dri-devel , kunit-dev@googlegroups.com, "open list:DOCUMENTATION" , linux-fsdevel@vger.kernel.org, linux-kbuild , Linux Kernel Mailing List , "open list:KERNEL SELFTEST FRAMEWORK" , linux-nvdimm , linux-um@lists.infradead.org, Sasha Levin , "Bird, Timothy" , Amir Goldstein , Dan Carpenter , Daniel Vetter , Jeff Dike , Joel Stanley , Julia Lawall , Kevin Hilman , Knut Omang , Logan Gunthorpe , Michael Ellerman , Petr Mladek , Randy Dunlap , Richard Weinberger , David Rientjes , Steven Rostedt , wfg@linux.intel.com Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jun 25, 2019 at 8:36 PM Luis Chamberlain wrote: > > On Tue, Jun 25, 2019 at 05:07:32PM -0700, Brendan Higgins wrote: > > On Tue, Jun 25, 2019 at 3:33 PM Luis Chamberlain wrote: > > > > > > On Mon, Jun 17, 2019 at 01:25:56AM -0700, Brendan Higgins wrote: > > > > +/** > > > > + * module_test() - used to register a &struct kunit_module with KUnit. > > > > + * @module: a statically allocated &struct kunit_module. > > > > + * > > > > + * Registers @module with the test framework. See &struct kunit_module for more > > > > + * information. > > > > + */ > > > > +#define module_test(module) \ > > > > + static int module_kunit_init##module(void) \ > > > > + { \ > > > > + return kunit_run_tests(&module); \ > > > > + } \ > > > > + late_initcall(module_kunit_init##module) > > > > > > Becuase late_initcall() is used, if these modules are built-in, this > > > would preclude the ability to test things prior to this part of the > > > kernel under UML or whatever architecture runs the tests. So, this > > > limits the scope of testing. Small detail but the scope whould be > > > documented. > > > > You aren't the first person to complain about this (and I am not sure > > it is the first time you have complained about it). Anyway, I have > > some follow on patches that will improve the late_initcall thing, and > > people seemed okay with discussing the follow on patches as part of a > > subsequent patchset after this gets merged. > > > > I will nevertheless document the restriction until then. > > To be clear, I am not complaining about it. I just find it simply > critical to document its limitations, so folks don't try to invest > time and energy on kunit right away for an early init test, if it > cannot support it. > > If support for that requires some work, it may be worth mentioning > as well. Makes sense. And in anycase, it is something I do want to do, just not right now. I will put a TODO here in the next revision. > > > > +static void kunit_print_tap_version(void) > > > > +{ > > > > + if (!kunit_has_printed_tap_version) { > > > > + kunit_printk_emit(LOGLEVEL_INFO, "TAP version 14\n"); > > > > > > What is this TAP thing? Why should we care what version it is on? > > > Why are we printing this? > > > > It's part of the TAP specification[1]. Greg and Frank asked me to make > > the intermediate format conform to TAP. Seems like something else I > > should probable document... > > Yes thanks! > > > > > + kunit_has_printed_tap_version = true; > > > > + } > > > > +} > > > > + > > > > +static size_t kunit_test_cases_len(struct kunit_case *test_cases) > > > > +{ > > > > + struct kunit_case *test_case; > > > > + size_t len = 0; > > > > + > > > > + for (test_case = test_cases; test_case->run_case; test_case++) > > > > > > If we make the last test case NULL, we'd just check for test_case here, > > > and save ourselves an extra few bytes per test module. Any reason why > > > the last test case cannot be NULL? > > > > Is there anyway to make that work with a statically defined array? > > No you're right. > > > Basically, I want to be able to do something like: > > > > static struct kunit_case example_test_cases[] = { > > KUNIT_CASE(example_simple_test), > > KUNIT_CASE(example_mock_test), > > {} > > }; > > > > FYI, > > #define KUNIT_CASE(test_name) { .run_case = test_name, .name = #test_name } > > > > > In order to do what you are proposing, I think I need an array of > > pointers to test cases, which is not ideal. > > Yeah, you're right. The only other alternative is to have a: > > struct kunit_module { > const char name[256]; > int (*init)(struct kunit *test); > void (*exit)(struct kunit *test); > struct kunit_case *test_cases; > + unsigned int num_cases; > }; > > And then something like: > > #define KUNIT_MODULE(name, init, exit, cases) { \ > .name = name, \ > .init = init, \ > .exit = exit, \ > .test_cases = cases, > num_cases = ARRAY_SIZE(cases), \ > } > > Let's evaluate which is better: one extra test case per all test cases, or > an extra unsigned int for each kunit module. I am in favor of the current method since init and exit are optional arguments. I could see myself (actually I am planning on) adding more optional things to the kunit_module, so having optional arguments will make my life a lot easier since I won't have to go through big refactorings around the kernel to support new features that tie in here.