Hi,
Continuing the documentation refactoring started by Harinder Singh[1],
removes kunit-tool.rst, which had its information rearranged into run_wrapper,
and employs further work in the index and the getting-started guide.
This series was written on top of another[2] that haven't got applied yet,
but the only dependency it has is the "kunit-on-qemu" anchor used in start.rst.
The patches working with the start.rst might be squashable, feel free to suggest so, if
you concur. Still about this file, I realize the note about "mrproper" was removed in
the recent refactoring, but having worked in the last months with people new to kunit,
it showed itself as a common occurrence, so I'm suggesting here to restore it.
Regarding the last two patches, I wasn't sure about either renaming run_wrapper to
kunit-tool to keep old references working or do as I did, updating the references I
could find.
Thanks in advance for your feedbacks,
Tales
[1] https://lore.kernel.org/r/[email protected]/
[2] https://lore.kernel.org/r/[email protected]/
Tales Aparecida (8):
Documentation: KUnit: remove duplicated docs for kunit_tool
Documentation: KUnit: avoid repeating "kunit.py run" in start.rst
Documentation: KUnit: restore note about mrproper in start.rst
Documentation: KUnit: Reword start guide for selecting tests
Documentation: KUnit: add intro to the getting-started page
Documentation: KUnit: update links in the index page
lib: overflow: update reference to kunit-tool
lib: stackinit: update reference to kunit-tool
Documentation/dev-tools/kunit/index.rst | 16 +-
Documentation/dev-tools/kunit/kunit-tool.rst | 232 ------------------
Documentation/dev-tools/kunit/run_wrapper.rst | 4 +-
Documentation/dev-tools/kunit/start.rst | 137 +++++++----
lib/overflow_kunit.c | 2 +-
lib/stackinit_kunit.c | 2 +-
6 files changed, 103 insertions(+), 290 deletions(-)
delete mode 100644 Documentation/dev-tools/kunit/kunit-tool.rst
base-commit: 568035b01cfb107af8d2e4bd2fb9aea22cf5b868
prerequisite-patch-id: b794218cd939a6644aaf5fb2a73997c56a624c80
prerequisite-patch-id: ccd24491ae99152ebdc6dcb8ddb9499d3456a4a0
prerequisite-patch-id: cc17b80d42fd5f5049e144da5c04e922036a33eb
prerequisite-patch-id: ba7edd270c6f285352e0e17bfe65ff6119192113
--
2.37.1
Describe the objective of the Getting Started page, which should be a
brief and beginner-friendly walkthrough for running and writing tests,
showing the reader where to find detailed instructions in other pages.
Signed-off-by: Tales Aparecida <[email protected]>
---
Documentation/dev-tools/kunit/start.rst | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
index f0ec64207bd3..3855402a5b3e 100644
--- a/Documentation/dev-tools/kunit/start.rst
+++ b/Documentation/dev-tools/kunit/start.rst
@@ -4,6 +4,11 @@
Getting Started
===============
+This page contains an overview about the kunit_tool and Kunit framework,
+teaching how to run existent tests and then how to write a simple test-case,
+and covering common problems users face when using Kunit for the first time.
+It is recommended that the reader had compiled the Kernel at least once before.
+
Installing Dependencies
=======================
KUnit has the same dependencies as the Linux kernel. As long as you can
--
2.37.1
Reword "Creating a ``.kunitconfig``" into "Selecting which tests to run"
covering the current alternatives for editing configs and glob-filtering
Signed-off-by: Tales Aparecida <[email protected]>
---
Documentation/dev-tools/kunit/start.rst | 90 +++++++++++++++++--------
1 file changed, 63 insertions(+), 27 deletions(-)
diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
index e4b73adde6d0..f0ec64207bd3 100644
--- a/Documentation/dev-tools/kunit/start.rst
+++ b/Documentation/dev-tools/kunit/start.rst
@@ -52,27 +52,20 @@ The tests will pass or fail.
For detailed information on this wrapper, see:
Documentation/dev-tools/kunit/run_wrapper.rst.
-Creating a ``.kunitconfig``
----------------------------
-
-By default, kunit_tool runs a selection of tests. However, you can specify which
-unit tests to run by creating a ``.kunitconfig`` file with kernel config options
-that enable only a specific set of tests and their dependencies.
-The ``.kunitconfig`` file contains a list of kconfig options which are required
-to run the desired targets. The ``.kunitconfig`` also contains any other test
-specific config options, such as test dependencies. For example: the
-``FAT_FS`` tests - ``FAT_KUNIT_TEST``, depends on
-``FAT_FS``. ``FAT_FS`` can be enabled by selecting either ``MSDOS_FS``
-or ``VFAT_FS``. To run ``FAT_KUNIT_TEST``, the ``.kunitconfig`` has:
+Selecting which tests to run
+----------------------------
-.. code-block:: none
+By default, kunit_tool runs all tests reachable with minimal configuration,
+that is, using default values for most of the kconfig options. However,
+you can select which tests to run by:
- CONFIG_KUNIT=y
- CONFIG_MSDOS_FS=y
- CONFIG_FAT_KUNIT_TEST=y
+- `Customizing Kconfig`_ used to compile the kernel, or
+- `Filtering tests by name`_ to select specifically which compiled tests to run.
-1. A good starting point for the ``.kunitconfig`` is the KUnit default config.
- You can generate it by running:
+Customizing Kconfig
+~~~~~~~~~~~~~~~~~~~
+A good starting point for the ``.kunitconfig`` is the KUnit default config.
+If you didn't run ``kunit.py run`` yet, you can generate it by running:
.. code-block:: bash
@@ -84,27 +77,70 @@ or ``VFAT_FS``. To run ``FAT_KUNIT_TEST``, the ``.kunitconfig`` has:
``.kunitconfig`` lives in the ``--build_dir`` used by kunit.py, which is
``.kunit`` by default.
-.. note ::
+Before running the tests, kunit_tool ensures that all config options
+set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn
+you if you have not included dependencies for the options used.
+
+There are many ways to customize the configurations:
+
+a. Edit ``.kunit/.kunitconfig``. The file should contain the list of kconfig
+ options required to run the desired tests, including their dependencies.
You may want to remove CONFIG_KUNIT_ALL_TESTS from the ``.kunitconfig`` as
it will enable a number of additional tests that you may not want.
+ If you need to run on an architecture other than UML see :ref:`kunit-on-qemu`.
-2. You can then add any other Kconfig options, for example:
+b. Enable additional kconfig options on top of ``.kunit/.kunitconfig``.
+ For example, to include the kernel's linked-list test you can run::
-.. code-block:: none
+ ./tools/testing/kunit/kunit.py run \
+ --kconfig_add CONFIG_LIST_KUNIT_TEST=y
- CONFIG_LIST_KUNIT_TEST=y
+c. Provide the path of one or more .kunitconfig files from the tree.
+ For example, to run only ``FAT_FS`` and ``EXT4`` tests you can run::
-Before running the tests, kunit_tool ensures that all config options
-set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn
-you if you have not included dependencies for the options used.
+ ./tools/testing/kunit/kunit.py run \
+ --kunitconfig ./fs/fat/.kunitconfig \
+ --kunitconfig ./fs/ext4/.kunitconfig
-.. note ::
- If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the
+d. If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the
``.config`` file. But you can edit the ``.config`` file directly or with
tools like ``make menuconfig O=.kunit``. As long as its a superset of
``.kunitconfig``, kunit.py won't overwrite your changes.
+.. note ::
+
+ To save a .kunitconfig after finding a satisfactory configuration::
+
+ make savedefconfig O=.kunit
+ cp .kunit/defconfig .kunit/.kunitconfig
+
+Filtering tests by name
+~~~~~~~~~~~~~~~~~~~~~~~
+If you want to be more specific than Kconfig can provide, it is also possible
+to select which tests to execute at boot-time by passing a glob filter
+(read instructions regarding the pattern in the manpage :manpage:`glob(7)`).
+If there is a ``"."`` (period) in the filter, it will be interpreted as a
+separator between the name of the test-suite and the test-case,
+otherwise, it will be interpreted as the name of the test suite.
+For example, let's assume we are using the default config:
+
+a. inform the name of a test suite, like ``"kunit_executor_test"``,
+ to run every test case it contains::
+
+ ./tools/testing/kunit/kunit.py run "kunit_executor_test"
+
+b. inform the name of a test case prefixed by its test suite,
+ like ``"example.example_simple_test"``, to run specifically that test case::
+
+ ./tools/testing/kunit/kunit.py run "example.example_simple_test"
+
+c. use wildcard characters (``*?[``) to run any test-case that match the pattern,
+ like ``"*.*64*"`` to run test-cases containing ``"64"`` in the name inside
+ any test-suite::
+
+ ./tools/testing/kunit/kunit.py run "*.*64*"
+
Running Tests without the KUnit Wrapper
=======================================
If you do not want to use the KUnit Wrapper (for example: you want code
--
2.37.1
Delete "kunit-tool.rst" to remove repeated info from KUnit docs.
"What is kunit_tool?" was integrated into index.rst, the remaining
sections were moved into run_wrapper.rst and renamed as follows:
"What is a .kunitconfig?" -> "Create a ``.kunitconfig`` File"
"Getting Started with kunit_tool" -> "Run Tests with kunit_tool"
"Configuring, Building, and Running Tests" ->
"Configure, Build, and Run Tests"
"Running Tests on QEMU" -> "Run Tests on QEMU"
"Parsing Test Results" -> "Parse Test Results"
"Filtering Tests" -> "Run Selected Test Suites"
"Other Useful Options" -> "Command-Line Arguments"
Signed-off-by: Tales Aparecida <[email protected]>
---
Documentation/dev-tools/kunit/index.rst | 3 -
Documentation/dev-tools/kunit/kunit-tool.rst | 232 ------------------
Documentation/dev-tools/kunit/run_wrapper.rst | 4 +-
Documentation/dev-tools/kunit/start.rst | 2 -
4 files changed, 2 insertions(+), 239 deletions(-)
delete mode 100644 Documentation/dev-tools/kunit/kunit-tool.rst
diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst
index bc91ad7b8961..d7187282ba28 100644
--- a/Documentation/dev-tools/kunit/index.rst
+++ b/Documentation/dev-tools/kunit/index.rst
@@ -13,7 +13,6 @@ KUnit - Linux Kernel Unit Testing
run_wrapper
run_manual
usage
- kunit-tool
api/index
style
faq
@@ -109,7 +108,5 @@ How do I use it?
examples.
* Documentation/dev-tools/kunit/api/index.rst - KUnit APIs
used for testing.
-* Documentation/dev-tools/kunit/kunit-tool.rst - kunit_tool helper
- script.
* Documentation/dev-tools/kunit/faq.rst - KUnit common questions and
answers.
diff --git a/Documentation/dev-tools/kunit/kunit-tool.rst b/Documentation/dev-tools/kunit/kunit-tool.rst
deleted file mode 100644
index ae52e0f489f9..000000000000
--- a/Documentation/dev-tools/kunit/kunit-tool.rst
+++ /dev/null
@@ -1,232 +0,0 @@
-.. SPDX-License-Identifier: GPL-2.0
-
-=================
-kunit_tool How-To
-=================
-
-What is kunit_tool?
-===================
-
-kunit_tool is a script (``tools/testing/kunit/kunit.py``) that aids in building
-the Linux kernel as UML (`User Mode Linux
-<http://user-mode-linux.sourceforge.net/>`_), running KUnit tests, parsing
-the test results and displaying them in a user friendly manner.
-
-kunit_tool addresses the problem of being able to run tests without needing a
-virtual machine or actual hardware with User Mode Linux. User Mode Linux is a
-Linux architecture, like ARM or x86; however, unlike other architectures it
-compiles the kernel as a standalone Linux executable that can be run like any
-other program directly inside of a host operating system. To be clear, it does
-not require any virtualization support: it is just a regular program.
-
-What is a .kunitconfig?
-=======================
-
-It's just a defconfig that kunit_tool looks for in the build directory
-(``.kunit`` by default). kunit_tool uses it to generate a .config as you might
-expect. In addition, it verifies that the generated .config contains the CONFIG
-options in the .kunitconfig; the reason it does this is so that it is easy to
-be sure that a CONFIG that enables a test actually ends up in the .config.
-
-It's also possible to pass a separate .kunitconfig fragment to kunit_tool,
-which is useful if you have several different groups of tests you wish
-to run independently, or if you want to use pre-defined test configs for
-certain subsystems.
-
-Getting Started with kunit_tool
-===============================
-
-If a kunitconfig is present at the root directory, all you have to do is:
-
-.. code-block:: bash
-
- ./tools/testing/kunit/kunit.py run
-
-However, you most likely want to use it with the following options:
-
-.. code-block:: bash
-
- ./tools/testing/kunit/kunit.py run --timeout=30 --jobs=`nproc --all`
-
-- ``--timeout`` sets a maximum amount of time to allow tests to run.
-- ``--jobs`` sets the number of threads to use to build the kernel.
-
-.. note::
- This command will work even without a .kunitconfig file: if no
- .kunitconfig is present, a default one will be used instead.
-
-If you wish to use a different .kunitconfig file (such as one provided for
-testing a particular subsystem), you can pass it as an option.
-
-.. code-block:: bash
-
- ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig
-
-For a list of all the flags supported by kunit_tool, you can run:
-
-.. code-block:: bash
-
- ./tools/testing/kunit/kunit.py run --help
-
-Configuring, Building, and Running Tests
-========================================
-
-It's also possible to run just parts of the KUnit build process independently,
-which is useful if you want to make manual changes to part of the process.
-
-A .config can be generated from a .kunitconfig by using the ``config`` argument
-when running kunit_tool:
-
-.. code-block:: bash
-
- ./tools/testing/kunit/kunit.py config
-
-Similarly, if you just want to build a KUnit kernel from the current .config,
-you can use the ``build`` argument:
-
-.. code-block:: bash
-
- ./tools/testing/kunit/kunit.py build
-
-And, if you already have a built UML kernel with built-in KUnit tests, you can
-run the kernel and display the test results with the ``exec`` argument:
-
-.. code-block:: bash
-
- ./tools/testing/kunit/kunit.py exec
-
-The ``run`` command which is discussed above is equivalent to running all three
-of these in sequence.
-
-All of these commands accept a number of optional command-line arguments. The
-``--help`` flag will give a complete list of these, or keep reading this page
-for a guide to some of the more useful ones.
-
-Parsing Test Results
-====================
-
-KUnit tests output their results in TAP (Test Anything Protocol) format.
-kunit_tool will, when running tests, parse this output and print a summary
-which is much more pleasant to read. If you wish to look at the raw test
-results in TAP format, you can pass the ``--raw_output`` argument.
-
-.. code-block:: bash
-
- ./tools/testing/kunit/kunit.py run --raw_output
-
-The raw output from test runs may contain other, non-KUnit kernel log
-lines. You can see just KUnit output with ``--raw_output=kunit``:
-
-.. code-block:: bash
-
- ./tools/testing/kunit/kunit.py run --raw_output=kunit
-
-If you have KUnit results in their raw TAP format, you can parse them and print
-the human-readable summary with the ``parse`` command for kunit_tool. This
-accepts a filename for an argument, or will read from standard input.
-
-.. code-block:: bash
-
- # Reading from a file
- ./tools/testing/kunit/kunit.py parse /var/log/dmesg
- # Reading from stdin
- dmesg | ./tools/testing/kunit/kunit.py parse
-
-This is very useful if you wish to run tests in a configuration not supported
-by kunit_tool (such as on real hardware, or an unsupported architecture).
-
-Filtering Tests
-===============
-
-It's possible to run only a subset of the tests built into a kernel by passing
-a filter to the ``exec`` or ``run`` commands. For example, if you only wanted
-to run KUnit resource tests, you could use:
-
-.. code-block:: bash
-
- ./tools/testing/kunit/kunit.py run 'kunit-resource*'
-
-This uses the standard glob format for wildcards.
-
-Running Tests on QEMU
-=====================
-
-kunit_tool supports running tests on QEMU as well as via UML (as mentioned
-elsewhere). The default way of running tests on QEMU requires two flags:
-
-``--arch``
- Selects a collection of configs (Kconfig as well as QEMU configs
- options, etc) that allow KUnit tests to be run on the specified
- architecture in a minimal way; this is usually not much slower than
- using UML. The architecture argument is the same as the name of the
- option passed to the ``ARCH`` variable used by Kbuild. Not all
- architectures are currently supported by this flag, but can be handled
- by the ``--qemu_config`` discussed later. If ``um`` is passed (or this
- this flag is ignored) the tests will run via UML. Non-UML architectures,
- e.g. i386, x86_64, arm, um, etc. Non-UML run on QEMU.
-
-``--cross_compile``
- Specifies the use of a toolchain by Kbuild. The argument passed here is
- the same passed to the ``CROSS_COMPILE`` variable used by Kbuild. As a
- reminder this will be the prefix for the toolchain binaries such as gcc
- for example ``sparc64-linux-gnu-`` if you have the sparc toolchain
- installed on your system, or
- ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux-``
- if you have downloaded the microblaze toolchain from the 0-day website
- to a directory in your home directory called ``toolchains``.
-
-In many cases it is likely that you may want to run an architecture which is
-not supported by the ``--arch`` flag, or you may want to just run KUnit tests
-on QEMU using a non-default configuration. For this use case, you can write
-your own QemuConfig. These QemuConfigs are written in Python. They must have an
-import line ``from ..qemu_config import QemuArchParams`` at the top of the file
-and the file must contain a variable called ``QEMU_ARCH`` that has an instance
-of ``QemuArchParams`` assigned to it. An example can be seen in
-``tools/testing/kunit/qemu_configs/x86_64.py``.
-
-Once you have a QemuConfig you can pass it into kunit_tool using the
-``--qemu_config`` flag; when used this flag replaces the ``--arch`` flag. If we
-were to do this with the ``x86_64.py`` example from above, the invocation would
-look something like this:
-
-.. code-block:: bash
-
- ./tools/testing/kunit/kunit.py run \
- --timeout=60 \
- --jobs=12 \
- --qemu_config=./tools/testing/kunit/qemu_configs/x86_64.py
-
-Other Useful Options
-====================
-
-kunit_tool has a number of other command-line arguments which can be useful
-when adapting it to fit your environment or needs.
-
-Some of the more useful ones are:
-
-``--help``
- Lists all of the available options. Note that different commands
- (``config``, ``build``, ``run``, etc) will have different supported
- options. Place ``--help`` before the command to list common options,
- and after the command for options specific to that command.
-
-``--build_dir``
- Specifies the build directory that kunit_tool will use. This is where
- the .kunitconfig file is located, as well as where the .config and
- compiled kernel will be placed. Defaults to ``.kunit``.
-
-``--make_options``
- Specifies additional options to pass to ``make`` when compiling a
- kernel (with the ``build`` or ``run`` commands). For example, to enable
- compiler warnings, you can pass ``--make_options W=1``.
-
-``--alltests``
- Builds a UML kernel with all config options enabled using ``make
- allyesconfig``. This allows you to run as many tests as is possible,
- but is very slow and prone to breakage as new options are added or
- modified. In most cases, enabling all tests which have satisfied
- dependencies by adding ``CONFIG_KUNIT_ALL_TESTS=1`` to your
- .kunitconfig is preferable.
-
-There are several other options (and new ones are often added), so do check
-``--help`` if you're looking for something not mentioned here.
diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
index a1070def284f..24373db26f9d 100644
--- a/Documentation/dev-tools/kunit/run_wrapper.rst
+++ b/Documentation/dev-tools/kunit/run_wrapper.rst
@@ -58,7 +58,7 @@ To view kunit_tool flags (optional command-line arguments), run:
./tools/testing/kunit/kunit.py run --help
-Create a ``.kunitconfig`` File
+Create a ``.kunitconfig`` File
===============================
If we want to run a specific set of tests (rather than those listed
@@ -167,7 +167,7 @@ This uses the standard glob format with wildcard characters.
.. _kunit-on-qemu:
-Run Tests on qemu
+Run Tests on QEMU
=================
kunit_tool supports running tests on qemu as well as
diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
index 867a4bba6bf6..e730df1f468e 100644
--- a/Documentation/dev-tools/kunit/start.rst
+++ b/Documentation/dev-tools/kunit/start.rst
@@ -254,7 +254,5 @@ Next Steps
examples.
* Documentation/dev-tools/kunit/api/index.rst - KUnit APIs
used for testing.
-* Documentation/dev-tools/kunit/kunit-tool.rst - kunit_tool helper
- script.
* Documentation/dev-tools/kunit/faq.rst - KUnit common questions and
answers.
--
2.37.1
Hi Tales
On 8/19/22 02:32, Tales Aparecida wrote:
> Describe the objective of the Getting Started page, which should be a
> brief and beginner-friendly walkthrough for running and writing tests,
> showing the reader where to find detailed instructions in other pages.
>
> Signed-off-by: Tales Aparecida <[email protected]>
> ---
> Documentation/dev-tools/kunit/start.rst | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
> index f0ec64207bd3..3855402a5b3e 100644
> --- a/Documentation/dev-tools/kunit/start.rst
> +++ b/Documentation/dev-tools/kunit/start.rst
> @@ -4,6 +4,11 @@
> Getting Started
> ===============
>
> +This page contains an overview about the kunit_tool and Kunit framework,
> +teaching how to run existent tests and then how to write a simple test-case,
> +and covering common problems users face when using Kunit for the first time.
> +It is recommended that the reader had compiled the Kernel at least once before.
Some grammar nits:
- s/an overview about/an overview of
- s/Kunit/KUnit for consistency
- s/existent/existing
- s/covering/covers
Other than this small nits,
Reviewed-by: Maíra Canal <[email protected]>
Best Regards,
- Maíra Canal
> +
> Installing Dependencies
> =======================
> KUnit has the same dependencies as the Linux kernel. As long as you can
Hi Tales
On 8/19/22 02:32, Tales Aparecida wrote:
> Reword "Creating a ``.kunitconfig``" into "Selecting which tests to run"
> covering the current alternatives for editing configs and glob-filtering
>
> Signed-off-by: Tales Aparecida <[email protected]>
> ---
> Documentation/dev-tools/kunit/start.rst | 90 +++++++++++++++++--------
> 1 file changed, 63 insertions(+), 27 deletions(-)
>
> diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
> index e4b73adde6d0..f0ec64207bd3 100644
> --- a/Documentation/dev-tools/kunit/start.rst
> +++ b/Documentation/dev-tools/kunit/start.rst
> @@ -52,27 +52,20 @@ The tests will pass or fail.
> For detailed information on this wrapper, see:
> Documentation/dev-tools/kunit/run_wrapper.rst.
>
> -Creating a ``.kunitconfig``
> ----------------------------
> -
> -By default, kunit_tool runs a selection of tests. However, you can specify which
> -unit tests to run by creating a ``.kunitconfig`` file with kernel config options
> -that enable only a specific set of tests and their dependencies.
> -The ``.kunitconfig`` file contains a list of kconfig options which are required
> -to run the desired targets. The ``.kunitconfig`` also contains any other test
> -specific config options, such as test dependencies. For example: the
> -``FAT_FS`` tests - ``FAT_KUNIT_TEST``, depends on
> -``FAT_FS``. ``FAT_FS`` can be enabled by selecting either ``MSDOS_FS``
> -or ``VFAT_FS``. To run ``FAT_KUNIT_TEST``, the ``.kunitconfig`` has:
> +Selecting which tests to run
> +----------------------------
>
> -.. code-block:: none
> +By default, kunit_tool runs all tests reachable with minimal configuration,
> +that is, using default values for most of the kconfig options. However,
> +you can select which tests to run by:
>
> - CONFIG_KUNIT=y
> - CONFIG_MSDOS_FS=y
> - CONFIG_FAT_KUNIT_TEST=y
> +- `Customizing Kconfig`_ used to compile the kernel, or
> +- `Filtering tests by name`_ to select specifically which compiled tests to run.
>
> -1. A good starting point for the ``.kunitconfig`` is the KUnit default config.
> - You can generate it by running:
> +Customizing Kconfig
> +~~~~~~~~~~~~~~~~~~~
> +A good starting point for the ``.kunitconfig`` is the KUnit default config.
> +If you didn't run ``kunit.py run`` yet, you can generate it by running:
>
> .. code-block:: bash
>
> @@ -84,27 +77,70 @@ or ``VFAT_FS``. To run ``FAT_KUNIT_TEST``, the ``.kunitconfig`` has:
> ``.kunitconfig`` lives in the ``--build_dir`` used by kunit.py, which is
> ``.kunit`` by default.
>
> -.. note ::
> +Before running the tests, kunit_tool ensures that all config options
> +set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn
> +you if you have not included dependencies for the options used.
> +
> +There are many ways to customize the configurations:
> +
> +a. Edit ``.kunit/.kunitconfig``. The file should contain the list of kconfig
> + options required to run the desired tests, including their dependencies.
> You may want to remove CONFIG_KUNIT_ALL_TESTS from the ``.kunitconfig`` as
> it will enable a number of additional tests that you may not want.
> + If you need to run on an architecture other than UML see :ref:`kunit-on-qemu`.
>
> -2. You can then add any other Kconfig options, for example:
> +b. Enable additional kconfig options on top of ``.kunit/.kunitconfig``.
> + For example, to include the kernel's linked-list test you can run::
>
> -.. code-block:: none
> + ./tools/testing/kunit/kunit.py run \
> + --kconfig_add CONFIG_LIST_KUNIT_TEST=y
>
> - CONFIG_LIST_KUNIT_TEST=y
> +c. Provide the path of one or more .kunitconfig files from the tree.
> + For example, to run only ``FAT_FS`` and ``EXT4`` tests you can run::
>
> -Before running the tests, kunit_tool ensures that all config options
> -set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn
> -you if you have not included dependencies for the options used.
> + ./tools/testing/kunit/kunit.py run \
> + --kunitconfig ./fs/fat/.kunitconfig \
> + --kunitconfig ./fs/ext4/.kunitconfig
>
> -.. note ::
> - If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the
> +d. If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the
> ``.config`` file. But you can edit the ``.config`` file directly or with
> tools like ``make menuconfig O=.kunit``. As long as its a superset of
> ``.kunitconfig``, kunit.py won't overwrite your changes.
>
>
> +.. note ::
> +
> + To save a .kunitconfig after finding a satisfactory configuration::
> +
> + make savedefconfig O=.kunit
> + cp .kunit/defconfig .kunit/.kunitconfig
> +
> +Filtering tests by name
> +~~~~~~~~~~~~~~~~~~~~~~~
> +If you want to be more specific than Kconfig can provide, it is also possible
> +to select which tests to execute at boot-time by passing a glob filter
> +(read instructions regarding the pattern in the manpage :manpage:`glob(7)`).
> +If there is a ``"."`` (period) in the filter, it will be interpreted as a
> +separator between the name of the test-suite and the test-case,
> +otherwise, it will be interpreted as the name of the test suite.
> +For example, let's assume we are using the default config:
Another small nit: you could keep the consistency on naming test-suite
and test-cases. Currently, you are using "test-suite" and "test suite"
(same for test-cases). I guess any of the two options are fine, but it
would be nice to keep it consistent.
> +
> +a. inform the name of a test suite, like ``"kunit_executor_test"``,
> + to run every test case it contains::
> +
> + ./tools/testing/kunit/kunit.py run "kunit_executor_test"
> +
> +b. inform the name of a test case prefixed by its test suite,
> + like ``"example.example_simple_test"``, to run specifically that test case::
> +
> + ./tools/testing/kunit/kunit.py run "example.example_simple_test"
> +
> +c. use wildcard characters (``*?[``) to run any test-case that match the pattern,
I guess it would be "matches" instead of "match" here.
Other than this small nits, this is a great improvement on the docs!
Reviewed-by: Maíra Canal <[email protected]>
Best Regards,
- Maíra Canal
> + like ``"*.*64*"`` to run test-cases containing ``"64"`` in the name inside
> + any test-suite::
> +
> + ./tools/testing/kunit/kunit.py run "*.*64*"
> +
> Running Tests without the KUnit Wrapper
> =======================================
> If you do not want to use the KUnit Wrapper (for example: you want code
On Fri, Aug 19, 2022 at 11:02 AM Tales Aparecida
<[email protected]> wrote:
>
> Delete "kunit-tool.rst" to remove repeated info from KUnit docs.
> "What is kunit_tool?" was integrated into index.rst, the remaining
> sections were moved into run_wrapper.rst and renamed as follows:
>
> "What is a .kunitconfig?" -> "Create a ``.kunitconfig`` File"
> "Getting Started with kunit_tool" -> "Run Tests with kunit_tool"
> "Configuring, Building, and Running Tests" ->
> "Configure, Build, and Run Tests"
> "Running Tests on QEMU" -> "Run Tests on QEMU"
> "Parsing Test Results" -> "Parse Test Results"
> "Filtering Tests" -> "Run Selected Test Suites"
> "Other Useful Options" -> "Command-Line Arguments"
>
> Signed-off-by: Tales Aparecida <[email protected]>
> ---
Hi Tales,
Thank you for removing repeated content from the docs. It definitely
helps. You could change the headings as follows:
"What is a .kunitconfig?" -> "Create a ``.kunitconfig`` File". Keep it
as "Creating a ``.kunitconfig`` file".
"Getting Started with kunit_tool" -> "Run Tests with kunit_tool". Keep
it as "Running tests with kunit_tool".
"Configuring, Building, and Running Tests" ->
"Configure, Build, and Run Tests". Keep it as "Configuring, building,
and running tests"
"Running Tests on QEMU" -> "Run Tests on QEMU". Keep it as "Running
tests on QEMU"
"Parsing Test Results" -> "Parse Test Results". Keep it as "Parsing
test results"
"Filtering Tests" -> "Run Selected Test Suites". Keep it as "Filtering tests"
"Other Useful Options" -> "Command-Line Arguments". Keep it as
"Running command-line arguments". This would help to display the right
results when any user is searching with the keyword command-line.
Makes the document more discoverable.
Reviewed-by: Sadiya Kazi <[email protected]>
Regards,
Sadiya Kazi
> Documentation/dev-tools/kunit/index.rst | 3 -
> Documentation/dev-tools/kunit/kunit-tool.rst | 232 ------------------
> Documentation/dev-tools/kunit/run_wrapper.rst | 4 +-
> Documentation/dev-tools/kunit/start.rst | 2 -
> 4 files changed, 2 insertions(+), 239 deletions(-)
> delete mode 100644 Documentation/dev-tools/kunit/kunit-tool.rst
>
> diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst
> index bc91ad7b8961..d7187282ba28 100644
> --- a/Documentation/dev-tools/kunit/index.rst
> +++ b/Documentation/dev-tools/kunit/index.rst
> @@ -13,7 +13,6 @@ KUnit - Linux Kernel Unit Testing
> run_wrapper
> run_manual
> usage
> - kunit-tool
> api/index
> style
> faq
> @@ -109,7 +108,5 @@ How do I use it?
> examples.
> * Documentation/dev-tools/kunit/api/index.rst - KUnit APIs
> used for testing.
> -* Documentation/dev-tools/kunit/kunit-tool.rst - kunit_tool helper
> - script.
> * Documentation/dev-tools/kunit/faq.rst - KUnit common questions and
> answers.
> diff --git a/Documentation/dev-tools/kunit/kunit-tool.rst b/Documentation/dev-tools/kunit/kunit-tool.rst
> deleted file mode 100644
> index ae52e0f489f9..000000000000
> --- a/Documentation/dev-tools/kunit/kunit-tool.rst
> +++ /dev/null
> @@ -1,232 +0,0 @@
> -.. SPDX-License-Identifier: GPL-2.0
> -
> -=================
> -kunit_tool How-To
> -=================
> -
> -What is kunit_tool?
> -===================
> -
> -kunit_tool is a script (``tools/testing/kunit/kunit.py``) that aids in building
> -the Linux kernel as UML (`User Mode Linux
> -<http://user-mode-linux.sourceforge.net/>`_), running KUnit tests, parsing
> -the test results and displaying them in a user friendly manner.
> -
> -kunit_tool addresses the problem of being able to run tests without needing a
> -virtual machine or actual hardware with User Mode Linux. User Mode Linux is a
> -Linux architecture, like ARM or x86; however, unlike other architectures it
> -compiles the kernel as a standalone Linux executable that can be run like any
> -other program directly inside of a host operating system. To be clear, it does
> -not require any virtualization support: it is just a regular program.
> -
> -What is a .kunitconfig?
> -=======================
> -
> -It's just a defconfig that kunit_tool looks for in the build directory
> -(``.kunit`` by default). kunit_tool uses it to generate a .config as you might
> -expect. In addition, it verifies that the generated .config contains the CONFIG
> -options in the .kunitconfig; the reason it does this is so that it is easy to
> -be sure that a CONFIG that enables a test actually ends up in the .config.
> -
> -It's also possible to pass a separate .kunitconfig fragment to kunit_tool,
> -which is useful if you have several different groups of tests you wish
> -to run independently, or if you want to use pre-defined test configs for
> -certain subsystems.
> -
> -Getting Started with kunit_tool
> -===============================
> -
> -If a kunitconfig is present at the root directory, all you have to do is:
> -
> -.. code-block:: bash
> -
> - ./tools/testing/kunit/kunit.py run
> -
> -However, you most likely want to use it with the following options:
> -
> -.. code-block:: bash
> -
> - ./tools/testing/kunit/kunit.py run --timeout=30 --jobs=`nproc --all`
> -
> -- ``--timeout`` sets a maximum amount of time to allow tests to run.
> -- ``--jobs`` sets the number of threads to use to build the kernel.
> -
> -.. note::
> - This command will work even without a .kunitconfig file: if no
> - .kunitconfig is present, a default one will be used instead.
> -
> -If you wish to use a different .kunitconfig file (such as one provided for
> -testing a particular subsystem), you can pass it as an option.
> -
> -.. code-block:: bash
> -
> - ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig
> -
> -For a list of all the flags supported by kunit_tool, you can run:
> -
> -.. code-block:: bash
> -
> - ./tools/testing/kunit/kunit.py run --help
> -
> -Configuring, Building, and Running Tests
> -========================================
> -
> -It's also possible to run just parts of the KUnit build process independently,
> -which is useful if you want to make manual changes to part of the process.
> -
> -A .config can be generated from a .kunitconfig by using the ``config`` argument
> -when running kunit_tool:
> -
> -.. code-block:: bash
> -
> - ./tools/testing/kunit/kunit.py config
> -
> -Similarly, if you just want to build a KUnit kernel from the current .config,
> -you can use the ``build`` argument:
> -
> -.. code-block:: bash
> -
> - ./tools/testing/kunit/kunit.py build
> -
> -And, if you already have a built UML kernel with built-in KUnit tests, you can
> -run the kernel and display the test results with the ``exec`` argument:
> -
> -.. code-block:: bash
> -
> - ./tools/testing/kunit/kunit.py exec
> -
> -The ``run`` command which is discussed above is equivalent to running all three
> -of these in sequence.
> -
> -All of these commands accept a number of optional command-line arguments. The
> -``--help`` flag will give a complete list of these, or keep reading this page
> -for a guide to some of the more useful ones.
> -
> -Parsing Test Results
> -====================
> -
> -KUnit tests output their results in TAP (Test Anything Protocol) format.
> -kunit_tool will, when running tests, parse this output and print a summary
> -which is much more pleasant to read. If you wish to look at the raw test
> -results in TAP format, you can pass the ``--raw_output`` argument.
> -
> -.. code-block:: bash
> -
> - ./tools/testing/kunit/kunit.py run --raw_output
> -
> -The raw output from test runs may contain other, non-KUnit kernel log
> -lines. You can see just KUnit output with ``--raw_output=kunit``:
> -
> -.. code-block:: bash
> -
> - ./tools/testing/kunit/kunit.py run --raw_output=kunit
> -
> -If you have KUnit results in their raw TAP format, you can parse them and print
> -the human-readable summary with the ``parse`` command for kunit_tool. This
> -accepts a filename for an argument, or will read from standard input.
> -
> -.. code-block:: bash
> -
> - # Reading from a file
> - ./tools/testing/kunit/kunit.py parse /var/log/dmesg
> - # Reading from stdin
> - dmesg | ./tools/testing/kunit/kunit.py parse
> -
> -This is very useful if you wish to run tests in a configuration not supported
> -by kunit_tool (such as on real hardware, or an unsupported architecture).
> -
> -Filtering Tests
> -===============
> -
> -It's possible to run only a subset of the tests built into a kernel by passing
> -a filter to the ``exec`` or ``run`` commands. For example, if you only wanted
> -to run KUnit resource tests, you could use:
> -
> -.. code-block:: bash
> -
> - ./tools/testing/kunit/kunit.py run 'kunit-resource*'
> -
> -This uses the standard glob format for wildcards.
> -
> -Running Tests on QEMU
> -=====================
> -
> -kunit_tool supports running tests on QEMU as well as via UML (as mentioned
> -elsewhere). The default way of running tests on QEMU requires two flags:
> -
> -``--arch``
> - Selects a collection of configs (Kconfig as well as QEMU configs
> - options, etc) that allow KUnit tests to be run on the specified
> - architecture in a minimal way; this is usually not much slower than
> - using UML. The architecture argument is the same as the name of the
> - option passed to the ``ARCH`` variable used by Kbuild. Not all
> - architectures are currently supported by this flag, but can be handled
> - by the ``--qemu_config`` discussed later. If ``um`` is passed (or this
> - this flag is ignored) the tests will run via UML. Non-UML architectures,
> - e.g. i386, x86_64, arm, um, etc. Non-UML run on QEMU.
> -
> -``--cross_compile``
> - Specifies the use of a toolchain by Kbuild. The argument passed here is
> - the same passed to the ``CROSS_COMPILE`` variable used by Kbuild. As a
> - reminder this will be the prefix for the toolchain binaries such as gcc
> - for example ``sparc64-linux-gnu-`` if you have the sparc toolchain
> - installed on your system, or
> - ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux-``
> - if you have downloaded the microblaze toolchain from the 0-day website
> - to a directory in your home directory called ``toolchains``.
> -
> -In many cases it is likely that you may want to run an architecture which is
> -not supported by the ``--arch`` flag, or you may want to just run KUnit tests
> -on QEMU using a non-default configuration. For this use case, you can write
> -your own QemuConfig. These QemuConfigs are written in Python. They must have an
> -import line ``from ..qemu_config import QemuArchParams`` at the top of the file
> -and the file must contain a variable called ``QEMU_ARCH`` that has an instance
> -of ``QemuArchParams`` assigned to it. An example can be seen in
> -``tools/testing/kunit/qemu_configs/x86_64.py``.
> -
> -Once you have a QemuConfig you can pass it into kunit_tool using the
> -``--qemu_config`` flag; when used this flag replaces the ``--arch`` flag. If we
> -were to do this with the ``x86_64.py`` example from above, the invocation would
> -look something like this:
> -
> -.. code-block:: bash
> -
> - ./tools/testing/kunit/kunit.py run \
> - --timeout=60 \
> - --jobs=12 \
> - --qemu_config=./tools/testing/kunit/qemu_configs/x86_64.py
> -
> -Other Useful Options
> -====================
> -
> -kunit_tool has a number of other command-line arguments which can be useful
> -when adapting it to fit your environment or needs.
> -
> -Some of the more useful ones are:
> -
> -``--help``
> - Lists all of the available options. Note that different commands
> - (``config``, ``build``, ``run``, etc) will have different supported
> - options. Place ``--help`` before the command to list common options,
> - and after the command for options specific to that command.
> -
> -``--build_dir``
> - Specifies the build directory that kunit_tool will use. This is where
> - the .kunitconfig file is located, as well as where the .config and
> - compiled kernel will be placed. Defaults to ``.kunit``.
> -
> -``--make_options``
> - Specifies additional options to pass to ``make`` when compiling a
> - kernel (with the ``build`` or ``run`` commands). For example, to enable
> - compiler warnings, you can pass ``--make_options W=1``.
> -
> -``--alltests``
> - Builds a UML kernel with all config options enabled using ``make
> - allyesconfig``. This allows you to run as many tests as is possible,
> - but is very slow and prone to breakage as new options are added or
> - modified. In most cases, enabling all tests which have satisfied
> - dependencies by adding ``CONFIG_KUNIT_ALL_TESTS=1`` to your
> - .kunitconfig is preferable.
> -
> -There are several other options (and new ones are often added), so do check
> -``--help`` if you're looking for something not mentioned here.
> diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
> index a1070def284f..24373db26f9d 100644
> --- a/Documentation/dev-tools/kunit/run_wrapper.rst
> +++ b/Documentation/dev-tools/kunit/run_wrapper.rst
> @@ -58,7 +58,7 @@ To view kunit_tool flags (optional command-line arguments), run:
>
> ./tools/testing/kunit/kunit.py run --help
>
> -Create a ``.kunitconfig`` File
> +Create a ``.kunitconfig`` File
> ===============================
>
> If we want to run a specific set of tests (rather than those listed
> @@ -167,7 +167,7 @@ This uses the standard glob format with wildcard characters.
>
> .. _kunit-on-qemu:
>
> -Run Tests on qemu
> +Run Tests on QEMU
> =================
>
> kunit_tool supports running tests on qemu as well as
> diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
> index 867a4bba6bf6..e730df1f468e 100644
> --- a/Documentation/dev-tools/kunit/start.rst
> +++ b/Documentation/dev-tools/kunit/start.rst
> @@ -254,7 +254,5 @@ Next Steps
> examples.
> * Documentation/dev-tools/kunit/api/index.rst - KUnit APIs
> used for testing.
> -* Documentation/dev-tools/kunit/kunit-tool.rst - kunit_tool helper
> - script.
> * Documentation/dev-tools/kunit/faq.rst - KUnit common questions and
> answers.
> --
> 2.37.1
>
> -----Original Message-----
> From: Tales Aparecida <[email protected]>
>
> Describe the objective of the Getting Started page, which should be a
> brief and beginner-friendly walkthrough for running and writing tests,
> showing the reader where to find detailed instructions in other pages.
>
> Signed-off-by: Tales Aparecida <[email protected]>
> ---
> Documentation/dev-tools/kunit/start.rst | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
> index f0ec64207bd3..3855402a5b3e 100644
> --- a/Documentation/dev-tools/kunit/start.rst
> +++ b/Documentation/dev-tools/kunit/start.rst
> @@ -4,6 +4,11 @@
> Getting Started
> ===============
>
> +This page contains an overview about the kunit_tool and Kunit framework,
> +teaching how to run existent tests and then how to write a simple test-case,
> +and covering common problems users face when using Kunit for the first time.
> +It is recommended that the reader had compiled the Kernel at least once before.
had -> has
before. -> before (what?? - reading these instructions?, running kunit?)
> +
> Installing Dependencies
> =======================
> KUnit has the same dependencies as the Linux kernel. As long as you can
> --
> 2.37.1