Hi,
Changes since v2 [1]:
* Added a new patch (sent separately earlier) at the end, to error out
if "make headers" has not yet been run.
* Reworked and simplified the uffd movement patch. Now it only moves
some uffd*() routines, not all, and doesn't have to touch the Makefile
at all. This lighter touch also allowed me to drop the "move psize(),
pshift() into vm_utils.c" entirely. I expect Peter Xu will be a little
happier with this new approach.
* Fixed the commit description for the MADV_COLLAPSE patch.
* Added more Reviewed-by tags from David Hildenbrand and Peter Xu.
[1] https://lore.kernel.org/all/[email protected]/
John Hubbard (11):
selftests/mm: fix uffd-stress unused function warning
selftests/mm: fix unused variable warnings in hugetlb-madvise.c,
migration.c
selftests/mm: fix "warning: expression which evaluates to zero..." in
mlock2-tests.c
selftests/mm: fix invocation of tests that are run via shell scripts
selftests/mm: .gitignore: add mkdirty, va_high_addr_switch
selftests/mm: fix two -Wformat-security warnings in uffd builds
selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h
selftests/mm: fix build failures due to missing MADV_COLLAPSE
selftests/mm: move certain uffd*() routines from vm_util.c to
uffd-common.c
Documentation: kselftest: "make headers" is a prerequisite
selftests: error out if kernel header files are not yet built
Documentation/dev-tools/kselftest.rst | 1 +
tools/testing/selftests/lib.mk | 36 +++++++++++-
tools/testing/selftests/mm/.gitignore | 2 +
tools/testing/selftests/mm/cow.c | 7 ---
tools/testing/selftests/mm/hugetlb-madvise.c | 8 ++-
tools/testing/selftests/mm/khugepaged.c | 10 ----
tools/testing/selftests/mm/migration.c | 5 +-
tools/testing/selftests/mm/mlock2-tests.c | 1 -
tools/testing/selftests/mm/pkey-x86.h | 2 +-
tools/testing/selftests/mm/run_vmtests.sh | 6 +-
tools/testing/selftests/mm/uffd-common.c | 59 ++++++++++++++++++++
tools/testing/selftests/mm/uffd-common.h | 5 ++
tools/testing/selftests/mm/uffd-stress.c | 10 ----
tools/testing/selftests/mm/uffd-unit-tests.c | 16 ++----
tools/testing/selftests/mm/vm_util.c | 59 --------------------
tools/testing/selftests/mm/vm_util.h | 14 +++--
16 files changed, 130 insertions(+), 111 deletions(-)
base-commit: f8dba31b0a826e691949cd4fdfa5c30defaac8c5
--
2.40.1
MADV_PAGEOUT, MADV_POPULATE_READ, MADV_COLLAPSE are conditionally
defined as necessary. However, that was being done in .c files, and a
new build failure came up that would have been automatically avoided had
these been in a common header file.
So consolidate and move them all to vm_util.h, which fixes the build
failure.
An alternative approach from Muhammad Usama Anjum was: rely on "make
headers" being required, and include asm-generic/mman-common.h. This
works in the sense that it builds, but it still generates warnings about
duplicate MADV_* symbols, and the goal here is to get a fully clean (no
warnings) build here.
Reviewed-by: David Hildenbrand <[email protected]>
Cc: Peter Xu <[email protected]>
Cc: Muhammad Usama Anjum <[email protected]>
Signed-off-by: John Hubbard <[email protected]>
---
tools/testing/selftests/mm/cow.c | 7 -------
tools/testing/selftests/mm/khugepaged.c | 10 ----------
tools/testing/selftests/mm/vm_util.h | 10 ++++++++++
3 files changed, 10 insertions(+), 17 deletions(-)
diff --git a/tools/testing/selftests/mm/cow.c b/tools/testing/selftests/mm/cow.c
index dc9d6fe86028..8882b05ec9c8 100644
--- a/tools/testing/selftests/mm/cow.c
+++ b/tools/testing/selftests/mm/cow.c
@@ -30,13 +30,6 @@
#include "../kselftest.h"
#include "vm_util.h"
-#ifndef MADV_PAGEOUT
-#define MADV_PAGEOUT 21
-#endif
-#ifndef MADV_COLLAPSE
-#define MADV_COLLAPSE 25
-#endif
-
static size_t pagesize;
static int pagemap_fd;
static size_t thpsize;
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
index 97adc0f34f9c..e88ee039d0eb 100644
--- a/tools/testing/selftests/mm/khugepaged.c
+++ b/tools/testing/selftests/mm/khugepaged.c
@@ -22,16 +22,6 @@
#include "vm_util.h"
-#ifndef MADV_PAGEOUT
-#define MADV_PAGEOUT 21
-#endif
-#ifndef MADV_POPULATE_READ
-#define MADV_POPULATE_READ 22
-#endif
-#ifndef MADV_COLLAPSE
-#define MADV_COLLAPSE 25
-#endif
-
#define BASE_ADDR ((void *)(1UL << 30))
static unsigned long hpage_pmd_size;
static unsigned long page_size;
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index b950bd16083a..07f39ed2efba 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -63,3 +63,13 @@ int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
#define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
#define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
+
+#ifndef MADV_PAGEOUT
+#define MADV_PAGEOUT 21
+#endif
+#ifndef MADV_POPULATE_READ
+#define MADV_POPULATE_READ 22
+#endif
+#ifndef MADV_COLLAPSE
+#define MADV_COLLAPSE 25
+#endif
--
2.40.1
As per a discussion with Muhammad Usama Anjum [1], the following is how
one is supposed to build selftests:
make headers && make -C tools/testing/selftests/mm
Change the selftest build system's lib.mk to fail out with a helpful
message if that prerequisite "make headers" has not been done yet.
[1] https://lore.kernel.org/all/[email protected]/
Cc: David Hildenbrand <[email protected]>
Cc: Peter Xu <[email protected]>
Cc: Muhammad Usama Anjum <[email protected]>
Cc: Jonathan Corbet <[email protected]>
Cc: [email protected]
Signed-off-by: John Hubbard <[email protected]>
---
tools/testing/selftests/lib.mk | 36 +++++++++++++++++++++++++++++++---
1 file changed, 33 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index 05400462c779..b8ea03b9a015 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -44,10 +44,22 @@ endif
selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
top_srcdir = $(selfdir)/../../..
-ifeq ($(KHDR_INCLUDES),)
-KHDR_INCLUDES := -isystem $(top_srcdir)/usr/include
+ifneq ($(KBUILD_OUTPUT),)
+ # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
+ # expand a shell special character '~'. We use a somewhat tedious way here.
+ abs_objtree := $(shell cd $(top_srcdir) && mkdir -p $(KBUILD_OUTPUT) && cd $(KBUILD_OUTPUT) && pwd)
+ $(if $(abs_objtree),, \
+ $(error failed to create output directory "$(KBUILD_OUTPUT)"))
+ # $(realpath ...) resolves symlinks
+ abs_objtree := $(realpath $(abs_objtree))
+ KHDR_DIR := ${abs_objtree}/usr/include
+else
+ abs_srctree := $(shell cd $(top_srcdir) && pwd)
+ KHDR_DIR := ${abs_srctree}/usr/include
endif
+KHDR_INCLUDES := -isystem $(KHDR_DIR)
+
# The following are built by lib.mk common compile rules.
# TEST_CUSTOM_PROGS should be used by tests that require
# custom build rule and prevent common build rule use.
@@ -58,7 +70,25 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS))
TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED))
TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES))
-all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
+all: kernel_header_files $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) \
+ $(TEST_GEN_FILES)
+
+kernel_header_files:
+ @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
+ if [ $$? -ne 0 ]; then \
+ RED='\033[1;31m'; \
+ NOCOLOR='\033[0m'; \
+ echo; \
+ echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
+ echo "Please run this and try again:"; \
+ echo; \
+ echo " cd $(top_srcdir)"; \
+ echo " make headers"; \
+ echo; \
+ exit 1; \
+ fi
+
+.PHONY: kernel_header_files
define RUN_TESTS
BASE_DIR="$(selfdir)"; \
--
2.40.1
The uffd tests generate two compile time warnings from clang's
-Wformat-security setting. These trigger at the call sites for
uffd_test_start() and uffd_test_skip().
1) Fix the uffd_test_start() issue by removing the intermediate
test_name variable (thanks to David Hildenbrand for showing how to do
this).
2) Fix the uffd_test_skip() issue by observing that there is no need for
a macro and a variable args approach, because all callers of
uffd_test_skip() pass in a simple char* string, without any format
specifiers. So just change uffd_test_skip() into a regular C function.
Reviewed-by: David Hildenbrand <[email protected]>
Reviewed-by: Peter Xu <[email protected]>
Signed-off-by: John Hubbard <[email protected]>
---
tools/testing/selftests/mm/uffd-unit-tests.c | 16 ++++++----------
1 file changed, 6 insertions(+), 10 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 269c86768a02..04d91f144d1c 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -109,12 +109,11 @@ static void uffd_test_pass(void)
ksft_inc_fail_cnt(); \
} while (0)
-#define uffd_test_skip(...) do { \
- printf("skipped [reason: "); \
- printf(__VA_ARGS__); \
- printf("]\n"); \
- ksft_inc_xskip_cnt(); \
- } while (0)
+static void uffd_test_skip(const char *message)
+{
+ printf("skipped [reason: %s]\n", message);
+ ksft_inc_xskip_cnt();
+}
/*
* Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll
@@ -1149,7 +1148,6 @@ int main(int argc, char *argv[])
uffd_test_case_t *test;
mem_type_t *mem_type;
uffd_test_args_t args;
- char test_name[128];
const char *errmsg;
int has_uffd, opt;
int i, j;
@@ -1192,10 +1190,8 @@ int main(int argc, char *argv[])
mem_type = &mem_types[j];
if (!(test->mem_targets & mem_type->mem_flag))
continue;
- snprintf(test_name, sizeof(test_name),
- "%s on %s", test->name, mem_type->name);
- uffd_test_start(test_name);
+ uffd_test_start("%s on %s", test->name, mem_type->name);
if (!uffd_feature_supported(test)) {
uffd_test_skip("feature missing");
continue;
--
2.40.1
As per a discussion with Muhammad Usama Anjum [1], the following is how
one is supposed to build selftests:
make headers && make -C tools/testing/selftests/mm
However, that's not yet documented anywhere. So add it to
Documentation/dev-tools/kselftest.rst .
[1] https://lore.kernel.org/all/[email protected]/
Cc: Peter Xu <[email protected]>
Cc: Muhammad Usama Anjum <[email protected]>
Cc: Jonathan Corbet <[email protected]>
Cc: [email protected]
Reviewed-by: David Hildenbrand <[email protected]>
Signed-off-by: John Hubbard <[email protected]>
---
Documentation/dev-tools/kselftest.rst | 1 +
1 file changed, 1 insertion(+)
diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst
index 12b575b76b20..6e35d042199c 100644
--- a/Documentation/dev-tools/kselftest.rst
+++ b/Documentation/dev-tools/kselftest.rst
@@ -36,6 +36,7 @@ Running the selftests (hotplug tests are run in limited mode)
To build the tests::
+ $ make headers
$ make -C tools/testing/selftests
To run the tests::
--
2.40.1
There are only three uffd*() routines that are used outside of the uffd
selftests. Leave these in vm_util.c, where they are available to any mm
selftest program:
uffd_register()
uffd_unregister()
uffd_register_with_ioctls().
A few other uffd*() routines, however, are only used by the uffd-focused
tests found in uffd-stress.c and uffd-unit-tests.c. Move those routines
into uffd-common.c.
Cc: Peter Xu <[email protected]>
Acked-by: David Hildenbrand <[email protected]>
Signed-off-by: John Hubbard <[email protected]>
---
tools/testing/selftests/mm/uffd-common.c | 59 ++++++++++++++++++++++++
tools/testing/selftests/mm/uffd-common.h | 5 ++
tools/testing/selftests/mm/vm_util.c | 59 ------------------------
tools/testing/selftests/mm/vm_util.h | 4 --
4 files changed, 64 insertions(+), 63 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index 61c6250adf93..ba20d7504022 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -616,3 +616,62 @@ int copy_page(int ufd, unsigned long offset, bool wp)
{
return __copy_page(ufd, offset, false, wp);
}
+
+int uffd_open_dev(unsigned int flags)
+{
+ int fd, uffd;
+
+ fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
+ if (fd < 0)
+ return fd;
+ uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
+ close(fd);
+
+ return uffd;
+}
+
+int uffd_open_sys(unsigned int flags)
+{
+#ifdef __NR_userfaultfd
+ return syscall(__NR_userfaultfd, flags);
+#else
+ return -1;
+#endif
+}
+
+int uffd_open(unsigned int flags)
+{
+ int uffd = uffd_open_sys(flags);
+
+ if (uffd < 0)
+ uffd = uffd_open_dev(flags);
+
+ return uffd;
+}
+
+int uffd_get_features(uint64_t *features)
+{
+ struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
+ /*
+ * This should by default work in most kernels; the feature list
+ * will be the same no matter what we pass in here.
+ */
+ int fd = uffd_open(UFFD_USER_MODE_ONLY);
+
+ if (fd < 0)
+ /* Maybe the kernel is older than user-only mode? */
+ fd = uffd_open(0);
+
+ if (fd < 0)
+ return fd;
+
+ if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
+ close(fd);
+ return -errno;
+ }
+
+ *features = uffdio_api.features;
+ close(fd);
+
+ return 0;
+}
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 6068f2346b86..197f5262fe0d 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -110,6 +110,11 @@ int __copy_page(int ufd, unsigned long offset, bool retry, bool wp);
int copy_page(int ufd, unsigned long offset, bool wp);
void *uffd_poll_thread(void *arg);
+int uffd_open_dev(unsigned int flags);
+int uffd_open_sys(unsigned int flags);
+int uffd_open(unsigned int flags);
+int uffd_get_features(uint64_t *features);
+
#define TEST_ANON 1
#define TEST_HUGETLB 2
#define TEST_SHMEM 3
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 9b06a5034808..681277615839 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -242,62 +242,3 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)
return ret;
}
-
-int uffd_open_dev(unsigned int flags)
-{
- int fd, uffd;
-
- fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
- if (fd < 0)
- return fd;
- uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
- close(fd);
-
- return uffd;
-}
-
-int uffd_open_sys(unsigned int flags)
-{
-#ifdef __NR_userfaultfd
- return syscall(__NR_userfaultfd, flags);
-#else
- return -1;
-#endif
-}
-
-int uffd_open(unsigned int flags)
-{
- int uffd = uffd_open_sys(flags);
-
- if (uffd < 0)
- uffd = uffd_open_dev(flags);
-
- return uffd;
-}
-
-int uffd_get_features(uint64_t *features)
-{
- struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
- /*
- * This should by default work in most kernels; the feature list
- * will be the same no matter what we pass in here.
- */
- int fd = uffd_open(UFFD_USER_MODE_ONLY);
-
- if (fd < 0)
- /* Maybe the kernel is older than user-only mode? */
- fd = uffd_open(0);
-
- if (fd < 0)
- return fd;
-
- if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
- close(fd);
- return -errno;
- }
-
- *features = uffdio_api.features;
- close(fd);
-
- return 0;
-}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index 07f39ed2efba..c2d4ff798b91 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -48,10 +48,6 @@ unsigned long default_huge_page_size(void);
int uffd_register(int uffd, void *addr, uint64_t len,
bool miss, bool wp, bool minor);
int uffd_unregister(int uffd, void *addr, uint64_t len);
-int uffd_open_dev(unsigned int flags);
-int uffd_open_sys(unsigned int flags);
-int uffd_open(unsigned int flags);
-int uffd_get_features(uint64_t *features);
int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
bool miss, bool wp, bool minor, uint64_t *ioctls);
--
2.40.1
The stop variable is a char*, and the code was assigning a char value to
it. This was generating a warning when compiling with clang.
However, as both David and Peter pointed out, stop is not even used
after the problematic assignment to a char type. So just delete that
line entirely.
Reviewed-by: David Hildenbrand <[email protected]>
Reviewed-by: Peter Xu <[email protected]>
Signed-off-by: John Hubbard <[email protected]>
---
tools/testing/selftests/mm/mlock2-tests.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/tools/testing/selftests/mm/mlock2-tests.c b/tools/testing/selftests/mm/mlock2-tests.c
index 11b2301f3aa3..80cddc0de206 100644
--- a/tools/testing/selftests/mm/mlock2-tests.c
+++ b/tools/testing/selftests/mm/mlock2-tests.c
@@ -50,7 +50,6 @@ static int get_vm_area(unsigned long addr, struct vm_boundaries *area)
printf("cannot parse /proc/self/maps\n");
goto out;
}
- stop = '\0';
sscanf(line, "%lx", &start);
sscanf(end_addr, "%lx", &end);
--
2.40.1
This fixes a real bug, too, because xstate_size() was assuming that
the stack variable xstate_size was initialized to zero. That's not
guaranteed nor even especially likely.
Reviewed-by: David Hildenbrand <[email protected]>
Cc: Peter Xu <[email protected]>
Signed-off-by: John Hubbard <[email protected]>
---
tools/testing/selftests/mm/pkey-x86.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/mm/pkey-x86.h b/tools/testing/selftests/mm/pkey-x86.h
index 72c14cd3ddc7..e32ae8a1cd99 100644
--- a/tools/testing/selftests/mm/pkey-x86.h
+++ b/tools/testing/selftests/mm/pkey-x86.h
@@ -132,7 +132,7 @@ int pkey_reg_xstate_offset(void)
unsigned int ecx;
unsigned int edx;
int xstate_offset;
- int xstate_size;
+ int xstate_size = 0;
unsigned long XSTATE_CPUID = 0xd;
int leaf;
--
2.40.1
We cannot depend upon git to reliably retain the executable bit on shell
scripts, or so I was told several years ago while working on this same
run_vmtests.sh script. And sure enough, things such as test_hmm.sh are
lately failing to run, due to lacking execute permissions.
Fix this by explicitly adding "bash" to each of the shell script
invocations. Leave fixing the overall approach to another day.
Acked-by: David Hildenbrand <[email protected]>
Cc: Peter Xu <[email protected]>
Signed-off-by: John Hubbard <[email protected]>
---
tools/testing/selftests/mm/run_vmtests.sh | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index 4893eb60d96d..8f81432e4bac 100644
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -242,18 +242,18 @@ if [ $VADDR64 -ne 0 ]; then
if [ "$ARCH" == "$ARCH_ARM64" ]; then
echo 6 > /proc/sys/vm/nr_hugepages
fi
- CATEGORY="hugevm" run_test ./va_high_addr_switch.sh
+ CATEGORY="hugevm" run_test bash ./va_high_addr_switch.sh
if [ "$ARCH" == "$ARCH_ARM64" ]; then
echo $prev_nr_hugepages > /proc/sys/vm/nr_hugepages
fi
fi # VADDR64
# vmalloc stability smoke test
-CATEGORY="vmalloc" run_test ./test_vmalloc.sh smoke
+CATEGORY="vmalloc" run_test bash ./test_vmalloc.sh smoke
CATEGORY="mremap" run_test ./mremap_dontunmap
-CATEGORY="hmm" run_test ./test_hmm.sh smoke
+CATEGORY="hmm" run_test bash ./test_hmm.sh smoke
# MADV_POPULATE_READ and MADV_POPULATE_WRITE tests
CATEGORY="madv_populate" run_test ./madv_populate
--
2.40.1
On 6/6/23 12:16 PM, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> Change the selftest build system's lib.mk to fail out with a helpful
> message if that prerequisite "make headers" has not been done yet.
>
> [1] https://lore.kernel.org/all/[email protected]/
>
> Cc: David Hildenbrand <[email protected]>
> Cc: Peter Xu <[email protected]>
> Cc: Muhammad Usama Anjum <[email protected]>
> Cc: Jonathan Corbet <[email protected]>
> Cc: [email protected]
> Signed-off-by: John Hubbard <[email protected]>
> ---
> tools/testing/selftests/lib.mk | 36 +++++++++++++++++++++++++++++++---
> 1 file changed, 33 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
> index 05400462c779..b8ea03b9a015 100644
> --- a/tools/testing/selftests/lib.mk
> +++ b/tools/testing/selftests/lib.mk
> @@ -44,10 +44,22 @@ endif
> selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
> top_srcdir = $(selfdir)/../../..
>
> -ifeq ($(KHDR_INCLUDES),)
> -KHDR_INCLUDES := -isystem $(top_srcdir)/usr/include
> +ifneq ($(KBUILD_OUTPUT),)
> + # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
> + # expand a shell special character '~'. We use a somewhat tedious way here.
> + abs_objtree := $(shell cd $(top_srcdir) && mkdir -p $(KBUILD_OUTPUT) && cd $(KBUILD_OUTPUT) && pwd)
> + $(if $(abs_objtree),, \
> + $(error failed to create output directory "$(KBUILD_OUTPUT)"))
> + # $(realpath ...) resolves symlinks
> + abs_objtree := $(realpath $(abs_objtree))
> + KHDR_DIR := ${abs_objtree}/usr/include
> +else
> + abs_srctree := $(shell cd $(top_srcdir) && pwd)
> + KHDR_DIR := ${abs_srctree}/usr/include
> endif
>
> +KHDR_INCLUDES := -isystem $(KHDR_DIR)
> +
> # The following are built by lib.mk common compile rules.
> # TEST_CUSTOM_PROGS should be used by tests that require
> # custom build rule and prevent common build rule use.
> @@ -58,7 +70,25 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS))
> TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED))
> TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES))
>
> -all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
> +all: kernel_header_files $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) \
> + $(TEST_GEN_FILES)
> +
> +kernel_header_files:
> + @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
> + if [ $$? -ne 0 ]; then \
> + RED='\033[1;31m'; \
> + NOCOLOR='\033[0m'; \
> + echo; \
> + echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
> + echo "Please run this and try again:"; \
> + echo; \
> + echo " cd $(top_srcdir)"; \
> + echo " make headers"; \
> + echo; \
> + exit 1; \
> + fi
Thank you for adding this. This is outputting error for every selftest
directory. We should try to make it even better by just aborting the
Make-ing process the first time headers aren't detected. We can do this now
or later, fine by me.
make[1]: Entering directory
'/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/futex'
-e error: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: *** [../lib.mk:77: kernel_header_files] Error 1
make[1]: Leaving directory
'/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/futex'
make[1]: Entering directory
'/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/gpio'
-e error: missing kernel header files.
Please run this and try again:
cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../..
make headers
make[1]: *** [../lib.mk:77: kernel_header_files] Error 1
make[1]: Leaving directory
'/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/gpio'
m
Complete error log file is attached.
> +
> +.PHONY: kernel_header_files
>
> define RUN_TESTS
> BASE_DIR="$(selfdir)"; \
--
BR,
Muhammad Usama Anjum
On 6/6/23 12:16 PM, John Hubbard wrote:
> We cannot depend upon git to reliably retain the executable bit on shell
> scripts, or so I was told several years ago while working on this same
> run_vmtests.sh script. And sure enough, things such as test_hmm.sh are
> lately failing to run, due to lacking execute permissions.
>
> Fix this by explicitly adding "bash" to each of the shell script
> invocations. Leave fixing the overall approach to another day.
>
> Acked-by: David Hildenbrand <[email protected]>
> Cc: Peter Xu <[email protected]>
> Signed-off-by: John Hubbard <[email protected]>
Tested-by: Muhammad Usama Anjum <[email protected]>
> ---
> tools/testing/selftests/mm/run_vmtests.sh | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index 4893eb60d96d..8f81432e4bac 100644
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -242,18 +242,18 @@ if [ $VADDR64 -ne 0 ]; then
> if [ "$ARCH" == "$ARCH_ARM64" ]; then
> echo 6 > /proc/sys/vm/nr_hugepages
> fi
> - CATEGORY="hugevm" run_test ./va_high_addr_switch.sh
> + CATEGORY="hugevm" run_test bash ./va_high_addr_switch.sh
> if [ "$ARCH" == "$ARCH_ARM64" ]; then
> echo $prev_nr_hugepages > /proc/sys/vm/nr_hugepages
> fi
> fi # VADDR64
>
> # vmalloc stability smoke test
> -CATEGORY="vmalloc" run_test ./test_vmalloc.sh smoke
> +CATEGORY="vmalloc" run_test bash ./test_vmalloc.sh smoke
>
> CATEGORY="mremap" run_test ./mremap_dontunmap
>
> -CATEGORY="hmm" run_test ./test_hmm.sh smoke
> +CATEGORY="hmm" run_test bash ./test_hmm.sh smoke
>
> # MADV_POPULATE_READ and MADV_POPULATE_WRITE tests
> CATEGORY="madv_populate" run_test ./madv_populate
--
BR,
Muhammad Usama Anjum
On 6/6/23 12:16 PM, John Hubbard wrote:
> MADV_PAGEOUT, MADV_POPULATE_READ, MADV_COLLAPSE are conditionally
> defined as necessary. However, that was being done in .c files, and a
> new build failure came up that would have been automatically avoided had
> these been in a common header file.
>
> So consolidate and move them all to vm_util.h, which fixes the build
> failure.
>
> An alternative approach from Muhammad Usama Anjum was: rely on "make
> headers" being required, and include asm-generic/mman-common.h. This
> works in the sense that it builds, but it still generates warnings about
> duplicate MADV_* symbols, and the goal here is to get a fully clean (no
> warnings) build here.
>
> Reviewed-by: David Hildenbrand <[email protected]>
> Cc: Peter Xu <[email protected]>
> Cc: Muhammad Usama Anjum <[email protected]>
> Signed-off-by: John Hubbard <[email protected]>
Tested-by: Muhammad Usama Anjum <[email protected]>
> ---
> tools/testing/selftests/mm/cow.c | 7 -------
> tools/testing/selftests/mm/khugepaged.c | 10 ----------
> tools/testing/selftests/mm/vm_util.h | 10 ++++++++++
> 3 files changed, 10 insertions(+), 17 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/cow.c b/tools/testing/selftests/mm/cow.c
> index dc9d6fe86028..8882b05ec9c8 100644
> --- a/tools/testing/selftests/mm/cow.c
> +++ b/tools/testing/selftests/mm/cow.c
> @@ -30,13 +30,6 @@
> #include "../kselftest.h"
> #include "vm_util.h"
>
> -#ifndef MADV_PAGEOUT
> -#define MADV_PAGEOUT 21
> -#endif
> -#ifndef MADV_COLLAPSE
> -#define MADV_COLLAPSE 25
> -#endif
> -
> static size_t pagesize;
> static int pagemap_fd;
> static size_t thpsize;
> diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
> index 97adc0f34f9c..e88ee039d0eb 100644
> --- a/tools/testing/selftests/mm/khugepaged.c
> +++ b/tools/testing/selftests/mm/khugepaged.c
> @@ -22,16 +22,6 @@
>
> #include "vm_util.h"
>
> -#ifndef MADV_PAGEOUT
> -#define MADV_PAGEOUT 21
> -#endif
> -#ifndef MADV_POPULATE_READ
> -#define MADV_POPULATE_READ 22
> -#endif
> -#ifndef MADV_COLLAPSE
> -#define MADV_COLLAPSE 25
> -#endif
> -
> #define BASE_ADDR ((void *)(1UL << 30))
> static unsigned long hpage_pmd_size;
> static unsigned long page_size;
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index b950bd16083a..07f39ed2efba 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -63,3 +63,13 @@ int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
>
> #define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
> #define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
> +
> +#ifndef MADV_PAGEOUT
> +#define MADV_PAGEOUT 21
> +#endif
> +#ifndef MADV_POPULATE_READ
> +#define MADV_POPULATE_READ 22
> +#endif
> +#ifndef MADV_COLLAPSE
> +#define MADV_COLLAPSE 25
> +#endif
--
BR,
Muhammad Usama Anjum
On 6/6/23 12:16 PM, John Hubbard wrote:
> The stop variable is a char*, and the code was assigning a char value to
> it. This was generating a warning when compiling with clang.
>
> However, as both David and Peter pointed out, stop is not even used
> after the problematic assignment to a char type. So just delete that
> line entirely.
>
> Reviewed-by: David Hildenbrand <[email protected]>
> Reviewed-by: Peter Xu <[email protected]>
> Signed-off-by: John Hubbard <[email protected]>
Tested-by: Muhammad Usama Anjum <[email protected]>
> ---
> tools/testing/selftests/mm/mlock2-tests.c | 1 -
> 1 file changed, 1 deletion(-)
>
> diff --git a/tools/testing/selftests/mm/mlock2-tests.c b/tools/testing/selftests/mm/mlock2-tests.c
> index 11b2301f3aa3..80cddc0de206 100644
> --- a/tools/testing/selftests/mm/mlock2-tests.c
> +++ b/tools/testing/selftests/mm/mlock2-tests.c
> @@ -50,7 +50,6 @@ static int get_vm_area(unsigned long addr, struct vm_boundaries *area)
> printf("cannot parse /proc/self/maps\n");
> goto out;
> }
> - stop = '\0';
>
> sscanf(line, "%lx", &start);
> sscanf(end_addr, "%lx", &end);
--
BR,
Muhammad Usama Anjum
On 6/6/23 12:16 PM, John Hubbard wrote:
> This fixes a real bug, too, because xstate_size() was assuming that
> the stack variable xstate_size was initialized to zero. That's not
> guaranteed nor even especially likely.
>
> Reviewed-by: David Hildenbrand <[email protected]>
> Cc: Peter Xu <[email protected]>
> Signed-off-by: John Hubbard <[email protected]>
Tested-by: Muhammad Usama Anjum <[email protected]>
> ---
> tools/testing/selftests/mm/pkey-x86.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/mm/pkey-x86.h b/tools/testing/selftests/mm/pkey-x86.h
> index 72c14cd3ddc7..e32ae8a1cd99 100644
> --- a/tools/testing/selftests/mm/pkey-x86.h
> +++ b/tools/testing/selftests/mm/pkey-x86.h
> @@ -132,7 +132,7 @@ int pkey_reg_xstate_offset(void)
> unsigned int ecx;
> unsigned int edx;
> int xstate_offset;
> - int xstate_size;
> + int xstate_size = 0;
> unsigned long XSTATE_CPUID = 0xd;
> int leaf;
>
--
BR,
Muhammad Usama Anjum
On 6/6/23 12:16 PM, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> However, that's not yet documented anywhere. So add it to
> Documentation/dev-tools/kselftest.rst .
>
> [1] https://lore.kernel.org/all/[email protected]/
>
> Cc: Peter Xu <[email protected]>
> Cc: Muhammad Usama Anjum <[email protected]>
> Cc: Jonathan Corbet <[email protected]>
> Cc: [email protected]
> Reviewed-by: David Hildenbrand <[email protected]>
> Signed-off-by: John Hubbard <[email protected]>
Reviewed-by: Muhammad Usama Anjum <[email protected]>
> ---
> Documentation/dev-tools/kselftest.rst | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst
> index 12b575b76b20..6e35d042199c 100644
> --- a/Documentation/dev-tools/kselftest.rst
> +++ b/Documentation/dev-tools/kselftest.rst
> @@ -36,6 +36,7 @@ Running the selftests (hotplug tests are run in limited mode)
>
> To build the tests::
>
> + $ make headers
> $ make -C tools/testing/selftests
>
> To run the tests::
--
BR,
Muhammad Usama Anjum
On 6/6/23 12:16 PM, John Hubbard wrote:
> There are only three uffd*() routines that are used outside of the uffd
> selftests. Leave these in vm_util.c, where they are available to any mm
> selftest program:
>
> uffd_register()
> uffd_unregister()
> uffd_register_with_ioctls().
>
> A few other uffd*() routines, however, are only used by the uffd-focused
> tests found in uffd-stress.c and uffd-unit-tests.c. Move those routines
> into uffd-common.c.
>
> Cc: Peter Xu <[email protected]>
> Acked-by: David Hildenbrand <[email protected]>
> Signed-off-by: John Hubbard <[email protected]>
Tested-by: Muhammad Usama Anjum <[email protected]>
> ---
> tools/testing/selftests/mm/uffd-common.c | 59 ++++++++++++++++++++++++
> tools/testing/selftests/mm/uffd-common.h | 5 ++
> tools/testing/selftests/mm/vm_util.c | 59 ------------------------
> tools/testing/selftests/mm/vm_util.h | 4 --
> 4 files changed, 64 insertions(+), 63 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> index 61c6250adf93..ba20d7504022 100644
> --- a/tools/testing/selftests/mm/uffd-common.c
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -616,3 +616,62 @@ int copy_page(int ufd, unsigned long offset, bool wp)
> {
> return __copy_page(ufd, offset, false, wp);
> }
> +
> +int uffd_open_dev(unsigned int flags)
> +{
> + int fd, uffd;
> +
> + fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
> + if (fd < 0)
> + return fd;
> + uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
> + close(fd);
> +
> + return uffd;
> +}
> +
> +int uffd_open_sys(unsigned int flags)
> +{
> +#ifdef __NR_userfaultfd
> + return syscall(__NR_userfaultfd, flags);
> +#else
> + return -1;
> +#endif
> +}
> +
> +int uffd_open(unsigned int flags)
> +{
> + int uffd = uffd_open_sys(flags);
> +
> + if (uffd < 0)
> + uffd = uffd_open_dev(flags);
> +
> + return uffd;
> +}
> +
> +int uffd_get_features(uint64_t *features)
> +{
> + struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
> + /*
> + * This should by default work in most kernels; the feature list
> + * will be the same no matter what we pass in here.
> + */
> + int fd = uffd_open(UFFD_USER_MODE_ONLY);
> +
> + if (fd < 0)
> + /* Maybe the kernel is older than user-only mode? */
> + fd = uffd_open(0);
> +
> + if (fd < 0)
> + return fd;
> +
> + if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
> + close(fd);
> + return -errno;
> + }
> +
> + *features = uffdio_api.features;
> + close(fd);
> +
> + return 0;
> +}
> diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
> index 6068f2346b86..197f5262fe0d 100644
> --- a/tools/testing/selftests/mm/uffd-common.h
> +++ b/tools/testing/selftests/mm/uffd-common.h
> @@ -110,6 +110,11 @@ int __copy_page(int ufd, unsigned long offset, bool retry, bool wp);
> int copy_page(int ufd, unsigned long offset, bool wp);
> void *uffd_poll_thread(void *arg);
>
> +int uffd_open_dev(unsigned int flags);
> +int uffd_open_sys(unsigned int flags);
> +int uffd_open(unsigned int flags);
> +int uffd_get_features(uint64_t *features);
> +
> #define TEST_ANON 1
> #define TEST_HUGETLB 2
> #define TEST_SHMEM 3
> diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
> index 9b06a5034808..681277615839 100644
> --- a/tools/testing/selftests/mm/vm_util.c
> +++ b/tools/testing/selftests/mm/vm_util.c
> @@ -242,62 +242,3 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)
>
> return ret;
> }
> -
> -int uffd_open_dev(unsigned int flags)
> -{
> - int fd, uffd;
> -
> - fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
> - if (fd < 0)
> - return fd;
> - uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
> - close(fd);
> -
> - return uffd;
> -}
> -
> -int uffd_open_sys(unsigned int flags)
> -{
> -#ifdef __NR_userfaultfd
> - return syscall(__NR_userfaultfd, flags);
> -#else
> - return -1;
> -#endif
> -}
> -
> -int uffd_open(unsigned int flags)
> -{
> - int uffd = uffd_open_sys(flags);
> -
> - if (uffd < 0)
> - uffd = uffd_open_dev(flags);
> -
> - return uffd;
> -}
> -
> -int uffd_get_features(uint64_t *features)
> -{
> - struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
> - /*
> - * This should by default work in most kernels; the feature list
> - * will be the same no matter what we pass in here.
> - */
> - int fd = uffd_open(UFFD_USER_MODE_ONLY);
> -
> - if (fd < 0)
> - /* Maybe the kernel is older than user-only mode? */
> - fd = uffd_open(0);
> -
> - if (fd < 0)
> - return fd;
> -
> - if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
> - close(fd);
> - return -errno;
> - }
> -
> - *features = uffdio_api.features;
> - close(fd);
> -
> - return 0;
> -}
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index 07f39ed2efba..c2d4ff798b91 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -48,10 +48,6 @@ unsigned long default_huge_page_size(void);
> int uffd_register(int uffd, void *addr, uint64_t len,
> bool miss, bool wp, bool minor);
> int uffd_unregister(int uffd, void *addr, uint64_t len);
> -int uffd_open_dev(unsigned int flags);
> -int uffd_open_sys(unsigned int flags);
> -int uffd_open(unsigned int flags);
> -int uffd_get_features(uint64_t *features);
> int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
> bool miss, bool wp, bool minor, uint64_t *ioctls);
>
--
BR,
Muhammad Usama Anjum
On 6/6/23 12:16 PM, John Hubbard wrote:
> The uffd tests generate two compile time warnings from clang's
> -Wformat-security setting. These trigger at the call sites for
> uffd_test_start() and uffd_test_skip().
>
> 1) Fix the uffd_test_start() issue by removing the intermediate
> test_name variable (thanks to David Hildenbrand for showing how to do
> this).
>
> 2) Fix the uffd_test_skip() issue by observing that there is no need for
> a macro and a variable args approach, because all callers of
> uffd_test_skip() pass in a simple char* string, without any format
> specifiers. So just change uffd_test_skip() into a regular C function.
>
> Reviewed-by: David Hildenbrand <[email protected]>
> Reviewed-by: Peter Xu <[email protected]>
> Signed-off-by: John Hubbard <[email protected]>
Tested-by: Muhammad Usama Anjum <[email protected]>
> ---
> tools/testing/selftests/mm/uffd-unit-tests.c | 16 ++++++----------
> 1 file changed, 6 insertions(+), 10 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
> index 269c86768a02..04d91f144d1c 100644
> --- a/tools/testing/selftests/mm/uffd-unit-tests.c
> +++ b/tools/testing/selftests/mm/uffd-unit-tests.c
> @@ -109,12 +109,11 @@ static void uffd_test_pass(void)
> ksft_inc_fail_cnt(); \
> } while (0)
>
> -#define uffd_test_skip(...) do { \
> - printf("skipped [reason: "); \
> - printf(__VA_ARGS__); \
> - printf("]\n"); \
> - ksft_inc_xskip_cnt(); \
> - } while (0)
> +static void uffd_test_skip(const char *message)
> +{
> + printf("skipped [reason: %s]\n", message);
> + ksft_inc_xskip_cnt();
> +}
>
> /*
> * Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll
> @@ -1149,7 +1148,6 @@ int main(int argc, char *argv[])
> uffd_test_case_t *test;
> mem_type_t *mem_type;
> uffd_test_args_t args;
> - char test_name[128];
> const char *errmsg;
> int has_uffd, opt;
> int i, j;
> @@ -1192,10 +1190,8 @@ int main(int argc, char *argv[])
> mem_type = &mem_types[j];
> if (!(test->mem_targets & mem_type->mem_flag))
> continue;
> - snprintf(test_name, sizeof(test_name),
> - "%s on %s", test->name, mem_type->name);
>
> - uffd_test_start(test_name);
> + uffd_test_start("%s on %s", test->name, mem_type->name);
> if (!uffd_feature_supported(test)) {
> uffd_test_skip("feature missing");
> continue;
--
BR,
Muhammad Usama Anjum
On 6/6/23 12:16 PM, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> Change the selftest build system's lib.mk to fail out with a helpful
> message if that prerequisite "make headers" has not been done yet.
>
> [1] https://lore.kernel.org/all/[email protected]/
>
> Cc: David Hildenbrand <[email protected]>
> Cc: Peter Xu <[email protected]>
> Cc: Muhammad Usama Anjum <[email protected]>
> Cc: Jonathan Corbet <[email protected]>
> Cc: [email protected]
> Signed-off-by: John Hubbard <[email protected]>
Reviewed-by: Muhammad Usama Anjum <[email protected]>
> ---
> tools/testing/selftests/lib.mk | 36 +++++++++++++++++++++++++++++++---
> 1 file changed, 33 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
> index 05400462c779..b8ea03b9a015 100644
> --- a/tools/testing/selftests/lib.mk
> +++ b/tools/testing/selftests/lib.mk
> @@ -44,10 +44,22 @@ endif
> selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
> top_srcdir = $(selfdir)/../../..
>
> -ifeq ($(KHDR_INCLUDES),)
> -KHDR_INCLUDES := -isystem $(top_srcdir)/usr/include
> +ifneq ($(KBUILD_OUTPUT),)
> + # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
> + # expand a shell special character '~'. We use a somewhat tedious way here.
> + abs_objtree := $(shell cd $(top_srcdir) && mkdir -p $(KBUILD_OUTPUT) && cd $(KBUILD_OUTPUT) && pwd)
> + $(if $(abs_objtree),, \
> + $(error failed to create output directory "$(KBUILD_OUTPUT)"))
> + # $(realpath ...) resolves symlinks
> + abs_objtree := $(realpath $(abs_objtree))
> + KHDR_DIR := ${abs_objtree}/usr/include
> +else
> + abs_srctree := $(shell cd $(top_srcdir) && pwd)
> + KHDR_DIR := ${abs_srctree}/usr/include
> endif
>
> +KHDR_INCLUDES := -isystem $(KHDR_DIR)
> +
> # The following are built by lib.mk common compile rules.
> # TEST_CUSTOM_PROGS should be used by tests that require
> # custom build rule and prevent common build rule use.
> @@ -58,7 +70,25 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS))
> TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED))
> TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES))
>
> -all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
> +all: kernel_header_files $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) \
> + $(TEST_GEN_FILES)
> +
> +kernel_header_files:
> + @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
> + if [ $$? -ne 0 ]; then \
> + RED='\033[1;31m'; \
> + NOCOLOR='\033[0m'; \
> + echo; \
> + echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
> + echo "Please run this and try again:"; \
> + echo; \
> + echo " cd $(top_srcdir)"; \
> + echo " make headers"; \
> + echo; \
> + exit 1; \
> + fi
> +
> +.PHONY: kernel_header_files
>
> define RUN_TESTS
> BASE_DIR="$(selfdir)"; \
--
BR,
Muhammad Usama Anjum
On 6/6/23 00:38, Muhammad Usama Anjum wrote:
...
>> +kernel_header_files:
>> + @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
>> + if [ $$? -ne 0 ]; then \
>> + RED='\033[1;31m'; \
>> + NOCOLOR='\033[0m'; \
>> + echo; \
>> + echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
>> + echo "Please run this and try again:"; \
>> + echo; \
>> + echo " cd $(top_srcdir)"; \
>> + echo " make headers"; \
>> + echo; \
>> + exit 1; \
>> + fi
> Thank you for adding this. This is outputting error for every selftest
> directory. We should try to make it even better by just aborting the
> Make-ing process the first time headers aren't detected. We can do this now
> or later, fine by me.
>
OK, I see. Yes, this can be improved by adding the same mechanism to the
selftests/Makefile, that is in selftests/mm/Makefile.
I'd like to keep both, because as I mentioned earlier, mm folks like to
run just that one Makefile, sometimes, and selftests/mm/Makefile is not
invoking the top level Makefile. Rather, it includes lib.mk--which the
top level Makefile does *not* include.
Arguably, using includes instead of recursive Make, would improve this
framework: reduce duplication such as the above. But that's a larger
project and just food for thought at this point.
Anyway, this works nicely on my system, and I'll attach it as a patch
also in case you want to try it out. What do you think of this:
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 90a62cf75008..bdca160063d8 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -144,10 +144,12 @@ ifneq ($(KBUILD_OUTPUT),)
abs_objtree := $(realpath $(abs_objtree))
BUILD := $(abs_objtree)/kselftest
KHDR_INCLUDES := -isystem ${abs_objtree}/usr/include
+ KHDR_DIR := ${abs_objtree}/usr/include
else
BUILD := $(CURDIR)
abs_srctree := $(shell cd $(top_srcdir) && pwd)
KHDR_INCLUDES := -isystem ${abs_srctree}/usr/include
+ KHDR_DIR := ${abs_srctree}/usr/include
DEFAULT_INSTALL_HDR_PATH := 1
endif
@@ -161,7 +163,7 @@ export KHDR_INCLUDES
# all isn't the first target in the file.
.DEFAULT_GOAL := all
-all:
+all: kernel_header_files
@ret=1; \
for TARGET in $(TARGETS); do \
BUILD_TARGET=$$BUILD/$$TARGET; \
@@ -172,6 +174,23 @@ all:
ret=$$((ret * $$?)); \
done; exit $$ret;
+kernel_header_files:
+ @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
+ if [ $$? -ne 0 ]; then \
+ RED='\033[1;31m'; \
+ NOCOLOR='\033[0m'; \
+ echo; \
+ echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
+ echo "Please run this and try again:"; \
+ echo; \
+ echo " cd $(top_srcdir)"; \
+ echo " make headers"; \
+ echo; \
+ exit 1; \
+ fi
+
+.PHONY: kernel_header_files
+
run_tests: all
@for TARGET in $(TARGETS); do \
BUILD_TARGET=$$BUILD/$$TARGET; \
thanks,
--
John Hubbard
NVIDIA
On 6/7/23 1:10 AM, John Hubbard wrote:
> On 6/6/23 00:38, Muhammad Usama Anjum wrote:
> ...
>>> +kernel_header_files:
>>> + @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
>>> + if [ $$? -ne 0 ]; then \
>>> + RED='\033[1;31m'; \
>>> + NOCOLOR='\033[0m'; \
>>> + echo; \
>>> + echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
>>> + echo "Please run this and try again:"; \
>>> + echo; \
>>> + echo " cd $(top_srcdir)"; \
>>> + echo " make headers"; \
>>> + echo; \
>>> + exit 1; \
>>> + fi
>> Thank you for adding this. This is outputting error for every selftest
>> directory. We should try to make it even better by just aborting the
>> Make-ing process the first time headers aren't detected. We can do this now
>> or later, fine by me.
>>
> OK, I see. Yes, this can be improved by adding the same mechanism to the
> selftests/Makefile, that is in selftests/mm/Makefile.
>
> I'd like to keep both, because as I mentioned earlier, mm folks like to
> run just that one Makefile, sometimes, and selftests/mm/Makefile is not
> invoking the top level Makefile. Rather, it includes lib.mk--which the
> top level Makefile does *not* include.
>
> Arguably, using includes instead of recursive Make, would improve this
> framework: reduce duplication such as the above. But that's a larger
> project and just food for thought at this point.
>
> Anyway, this works nicely on my system, and I'll attach it as a patch
> also in case you want to try it out. What do you think of this:
Nice patch. Thanks. Lets add this patch as well. Please add the tag for
this new patch:
Tested-by: Muhammad Usama Anjum <[email protected]>
>
> diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
> index 90a62cf75008..bdca160063d8 100644
> --- a/tools/testing/selftests/Makefile
> +++ b/tools/testing/selftests/Makefile
> @@ -144,10 +144,12 @@ ifneq ($(KBUILD_OUTPUT),)
> abs_objtree := $(realpath $(abs_objtree))
> BUILD := $(abs_objtree)/kselftest
> KHDR_INCLUDES := -isystem ${abs_objtree}/usr/include
> + KHDR_DIR := ${abs_objtree}/usr/include
> else
> BUILD := $(CURDIR)
> abs_srctree := $(shell cd $(top_srcdir) && pwd)
> KHDR_INCLUDES := -isystem ${abs_srctree}/usr/include
> + KHDR_DIR := ${abs_srctree}/usr/include
> DEFAULT_INSTALL_HDR_PATH := 1
> endif
>
> @@ -161,7 +163,7 @@ export KHDR_INCLUDES
> # all isn't the first target in the file.
> .DEFAULT_GOAL := all
>
> -all:
> +all: kernel_header_files
> @ret=1; \
> for TARGET in $(TARGETS); do \
> BUILD_TARGET=$$BUILD/$$TARGET; \
> @@ -172,6 +174,23 @@ all:
> ret=$$((ret * $$?)); \
> done; exit $$ret;
>
> +kernel_header_files:
> + @ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null; \
> + if [ $$? -ne 0 ]; then \
> + RED='\033[1;31m'; \
> + NOCOLOR='\033[0m'; \
> + echo; \
> + echo -e "$${RED}error$${NOCOLOR}: missing kernel header files."; \
> + echo "Please run this and try again:"; \
> + echo; \
> + echo " cd $(top_srcdir)"; \
> + echo " make headers"; \
> + echo; \
> + exit 1; \
> + fi
> +
> +.PHONY: kernel_header_files
> +
> run_tests: all
> @for TARGET in $(TARGETS); do \
> BUILD_TARGET=$$BUILD/$$TARGET; \
>
>
>
> thanks,
--
BR,
Muhammad Usama Anjum
On Tue, Jun 06, 2023 at 12:16:36AM -0700, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> However, that's not yet documented anywhere. So add it to
> Documentation/dev-tools/kselftest.rst .
This is breaking the arm64 selftests, I've sent a revert:
https://lore.kernel.org/linux-kselftest/[email protected]/T/#u
(logs included in the above patch.)
On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> Change the selftest build system's lib.mk to fail out with a helpful
> message if that prerequisite "make headers" has not been done yet.
>
NAK NAK NAK
This now means I can no longer run selftests, I thank you very much! :-/
root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
***
*** The source tree is not clean, please run 'make mrproper'
*** in /usr/src/linux-2.6
I've always done:
cd tools/testing/selftests/x86; make
and that has always worked
Now I can't bloody well build *any* selftest or risk not being able to
do builds.
On 03.11.23 13:16, Peter Zijlstra wrote:
> On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
>> As per a discussion with Muhammad Usama Anjum [1], the following is how
>> one is supposed to build selftests:
>>
>> make headers && make -C tools/testing/selftests/mm
>>
>> Change the selftest build system's lib.mk to fail out with a helpful
>> message if that prerequisite "make headers" has not been done yet.
>>
>
> NAK NAK NAK
>
> This now means I can no longer run selftests, I thank you very much! :-/
>
> root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
> make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
> ***
> *** The source tree is not clean, please run 'make mrproper'
> *** in /usr/src/linux-2.6
>
>
> I've always done:
>
> cd tools/testing/selftests/x86; make
>
> and that has always worked
>
> Now I can't bloody well build *any* selftest or risk not being able to
> do builds.
This change landed in 6.5, no? And 6.6 was just released. Just curious
why you notice that now.
--
Cheers,
David / dhildenb
On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
> On 03.11.23 13:16, Peter Zijlstra wrote:
> > On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
> > > As per a discussion with Muhammad Usama Anjum [1], the following is how
> > > one is supposed to build selftests:
> > >
> > > make headers && make -C tools/testing/selftests/mm
> > >
> > > Change the selftest build system's lib.mk to fail out with a helpful
> > > message if that prerequisite "make headers" has not been done yet.
> > >
> >
> > NAK NAK NAK
> >
> > This now means I can no longer run selftests, I thank you very much! :-/
> >
> > root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
> > make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
> > ***
> > *** The source tree is not clean, please run 'make mrproper'
> > *** in /usr/src/linux-2.6
> >
> >
> > I've always done:
> >
> > cd tools/testing/selftests/x86; make
> >
> > and that has always worked
> >
> > Now I can't bloody well build *any* selftest or risk not being able to
> > do builds.
>
> This change landed in 6.5, no? And 6.6 was just released. Just curious why
> you notice that now.
Dunno, last time I edited the selftests and needed to recompile was a
few weeks ago.
On 03.11.23 13:46, Peter Zijlstra wrote:
> On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
>> On 03.11.23 13:16, Peter Zijlstra wrote:
>>> On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
>>>> As per a discussion with Muhammad Usama Anjum [1], the following is how
>>>> one is supposed to build selftests:
>>>>
>>>> make headers && make -C tools/testing/selftests/mm
>>>>
>>>> Change the selftest build system's lib.mk to fail out with a helpful
>>>> message if that prerequisite "make headers" has not been done yet.
>>>>
>>>
>>> NAK NAK NAK
>>>
>>> This now means I can no longer run selftests, I thank you very much! :-/
>>>
>>> root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
>>> make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
>>> ***
>>> *** The source tree is not clean, please run 'make mrproper'
>>> *** in /usr/src/linux-2.6
>>>
>>>
>>> I've always done:
>>>
>>> cd tools/testing/selftests/x86; make
>>>
>>> and that has always worked
>>>
>>> Now I can't bloody well build *any* selftest or risk not being able to
>>> do builds.
>>
>> This change landed in 6.5, no? And 6.6 was just released. Just curious why
>> you notice that now.
>
> Dunno, last time I edited the selftests and needed to recompile was a
> few weeks ago.
Okay. the question is if your workflow can be easily adjusted, or if we
can improve that header handling as a whole.
The problem I had with this recently: just because we did a "make
headers" once in a git tree doesn't mean that it is still up-to-date.
So once some selftest changes showed up that require newer headers,
building the selftests again fails without a hint that another round of
"make headers" would be required.
--
Cheers,
David / dhildenb
On 03.11.23 13:59, David Hildenbrand wrote:
> On 03.11.23 13:46, Peter Zijlstra wrote:
>> On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
>>> On 03.11.23 13:16, Peter Zijlstra wrote:
>>>> On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
>>>>> As per a discussion with Muhammad Usama Anjum [1], the following is how
>>>>> one is supposed to build selftests:
>>>>>
>>>>> make headers && make -C tools/testing/selftests/mm
>>>>>
>>>>> Change the selftest build system's lib.mk to fail out with a helpful
>>>>> message if that prerequisite "make headers" has not been done yet.
>>>>>
>>>>
>>>> NAK NAK NAK
>>>>
>>>> This now means I can no longer run selftests, I thank you very much! :-/
>>>>
>>>> root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
>>>> make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
>>>> ***
>>>> *** The source tree is not clean, please run 'make mrproper'
>>>> *** in /usr/src/linux-2.6
>>>>
>>>>
>>>> I've always done:
>>>>
>>>> cd tools/testing/selftests/x86; make
>>>>
>>>> and that has always worked
>>>>
>>>> Now I can't bloody well build *any* selftest or risk not being able to
>>>> do builds.
>>>
>>> This change landed in 6.5, no? And 6.6 was just released. Just curious why
>>> you notice that now.
>>
>> Dunno, last time I edited the selftests and needed to recompile was a
>> few weeks ago.
>
> Okay. the question is if your workflow can be easily adjusted, or if we
> can improve that header handling as a whole.
>
> The problem I had with this recently: just because we did a "make
> headers" once in a git tree doesn't mean that it is still up-to-date.
>
> So once some selftest changes showed up that require newer headers,
> building the selftests again fails without a hint that another round of
> "make headers" would be required.
To clarify: maybe some kind of a warning would be better, ideally that
the headers might be outdated and that another "make headers" would be
required in case there are any build errors.
--
Cheers,
David / dhildenb
On Fri, Nov 03, 2023 at 01:59:28PM +0100, David Hildenbrand wrote:
> Okay. the question is if your workflow can be easily adjusted, or if we can
> improve that header handling as a whole.
So on IRC the following was suggested:
make O=defconfig-build headers ; make O=defconfig-build -C tools/testing/selftests/x86
But that makes absolutely no sense to me; because the headers and
selftests are not .config dependent. Furthermore I don't want them in a
kernel build dir.
> The problem I had with this recently: just because we did a "make headers"
> once in a git tree doesn't mean that it is still up-to-date.
>
> So once some selftest changes showed up that require newer headers, building
> the selftests again fails without a hint that another round of "make
> headers" would be required.
Yeah, so I've been adding #ifndef guards all over the place for decades
and that just works. You need it in normal userspace too.
This super reliance on the very latestesetst headers is just a total
pain.
Hi John, Muhammad,
On Tue, 6 Jun 2023, John Hubbard wrote:
> As per a discussion with Muhammad Usama Anjum [1], the following is how
> one is supposed to build selftests:
>
> make headers && make -C tools/testing/selftests/mm
>
> Change the selftest build system's lib.mk to fail out with a helpful
> message if that prerequisite "make headers" has not been done yet.
>
> [1] https://lore.kernel.org/all/[email protected]/
could you, please, elaborate more on that one is supposed to build
selftests with 'make headers'? Yes, Documentation/dev-tools/kselftest.rst
mentions that because you might need headers but...
The common way how we test the kernel is to build the kernel, install it
somewhere and run selftests on top. The sequence basically being "make
rpm-pkg; rpm -ivh; cd tools/testing/selftest/livepatch/ in source tree;
sudo make run_tests" (or a similar variation of the procedure). The point
is that we want to test the running kernel with its respective environment
installed in /lib/modules/`uname -r`/ (if needed). This way we can run
newer selftests from the current mainline tree on older kernels among
others.
The commit breaks the use case which worked for a long long time.
It also breaks what Marcos proposed for livepatch selftests in
https://lore.kernel.org/all/[email protected]/
I guess we can always work around it by letting subsystem selftests to
override KHDR_DIR but I am not comfortable with the behaviour that your
commit introduced in the first place to be honest.
Thank you,
Miroslav
On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
> On 03.11.23 13:16, Peter Zijlstra wrote:
> > On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
> > > As per a discussion with Muhammad Usama Anjum [1], the following is how
> > > one is supposed to build selftests:
> > >
> > > make headers && make -C tools/testing/selftests/mm
> > >
> > > Change the selftest build system's lib.mk to fail out with a helpful
> > > message if that prerequisite "make headers" has not been done yet.
> > >
> >
> > NAK NAK NAK
> >
> > This now means I can no longer run selftests, I thank you very much! :-/
> >
> > root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
> > make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
> > ***
> > *** The source tree is not clean, please run 'make mrproper'
> > *** in /usr/src/linux-2.6
> >
> >
> > I've always done:
> >
> > cd tools/testing/selftests/x86; make
> >
> > and that has always worked
> >
> > Now I can't bloody well build *any* selftest or risk not being able to
> > do builds.
>
> This change landed in 6.5, no? And 6.6 was just released. Just curious why
> you notice that now.
And I hit it again (different box etc..)
Can we please get this garbage fixed already?
On 08.12.23 16:14, Peter Zijlstra wrote:
> On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
>> On 03.11.23 13:16, Peter Zijlstra wrote:
>>> On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
>>>> As per a discussion with Muhammad Usama Anjum [1], the following is how
>>>> one is supposed to build selftests:
>>>>
>>>> make headers && make -C tools/testing/selftests/mm
>>>>
>>>> Change the selftest build system's lib.mk to fail out with a helpful
>>>> message if that prerequisite "make headers" has not been done yet.
>>>>
>>>
>>> NAK NAK NAK
>>>
>>> This now means I can no longer run selftests, I thank you very much! :-/
>>>
>>> root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
>>> make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
>>> ***
>>> *** The source tree is not clean, please run 'make mrproper'
>>> *** in /usr/src/linux-2.6
>>>
>>>
>>> I've always done:
>>>
>>> cd tools/testing/selftests/x86; make
>>>
>>> and that has always worked
>>>
>>> Now I can't bloody well build *any* selftest or risk not being able to
>>> do builds.
>>
>> This change landed in 6.5, no? And 6.6 was just released. Just curious why
>> you notice that now.
>
> And I hit it again (different box etc..)
>
> Can we please get this garbage fixed already?
I'd suggest to either revert or turn into a warning.
@John?
--
Cheers,
David / dhildenb
On 12/8/23 07:21, David Hildenbrand wrote:
> On 08.12.23 16:14, Peter Zijlstra wrote:
>> On Fri, Nov 03, 2023 at 01:22:54PM +0100, David Hildenbrand wrote:
>>> On 03.11.23 13:16, Peter Zijlstra wrote:
>>>> On Tue, Jun 06, 2023 at 12:16:37AM -0700, John Hubbard wrote:
>>>>> As per a discussion with Muhammad Usama Anjum [1], the following is
>>>>> how
>>>>> one is supposed to build selftests:
>>>>>
>>>>> make headers && make -C tools/testing/selftests/mm
>>>>>
>>>>> Change the selftest build system's lib.mk to fail out with a helpful
>>>>> message if that prerequisite "make headers" has not been done yet.
>>>>>
>>>>
>>>> NAK NAK NAK
>>>>
>>>> This now means I can no longer run selftests, I thank you very much!
>>>> :-/
>>>>
>>>> root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
>>>> make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
>>>> ***
>>>> *** The source tree is not clean, please run 'make mrproper'
>>>> *** in /usr/src/linux-2.6
>>>>
>>>>
>>>> I've always done:
>>>>
>>>> cd tools/testing/selftests/x86; make
>>>>
>>>> and that has always worked
>>>>
>>>> Now I can't bloody well build *any* selftest or risk not being able to
>>>> do builds.
>>>
>>> This change landed in 6.5, no? And 6.6 was just released. Just
>>> curious why
>>> you notice that now.
>>
>> And I hit it again (different box etc..)
>>
>> Can we please get this garbage fixed already?
>
> I'd suggest to either revert or turn into a warning.
That would put us back into a half-broken sort of situation, though...
see below.
>
> @John?
>
I don't have a strong opinion about how this should be done, and in
fact I believed at the time that I was bringing the system into
compliance with what everyone wanted here. :)
There seem to be two conflicting visions:
a) The way it was (much) earlier: use ifdefs and defines to get by
without the latest kernel headers, or
b) Requiring recent kernel headers to build the various selftests.
Shuah, Peter, others: can we choose a direction please? Either
way will work, and I personally don't care which one we choose.
thanks,
--
John Hubbard
NVIDIA
On Fri, Dec 08, 2023 at 12:29:37PM -0800, John Hubbard wrote:
> I don't have a strong opinion about how this should be done, and in
> fact I believed at the time that I was bringing the system into
> compliance with what everyone wanted here. :)
>
> There seem to be two conflicting visions:
>
> a) The way it was (much) earlier: use ifdefs and defines to get by
> without the latest kernel headers, or
>
> b) Requiring recent kernel headers to build the various selftests.
>
> Shuah, Peter, others: can we choose a direction please? Either
> way will work, and I personally don't care which one we choose.
So as David already argued, the current thing does not in fact help with
b. You just have to install once and the error goes away, then carry
that tree for a year and you're running old crap again.
My biggest beef with the whole thing is that I simply do not want to use
'make headers', it doesn't work for me.
I have a ton of output directories and I don't care to build tools into
the output dirs, in fact some of them flat out refuse to work that way
(bpf comes to mind).
On 12/8/23 14:10, Peter Zijlstra wrote:
> So as David already argued, the current thing does not in fact help with
> b. You just have to install once and the error goes away, then carry
> that tree for a year and you're running old crap again.
>
> My biggest beef with the whole thing is that I simply do not want to use
> 'make headers', it doesn't work for me.
>
> I have a ton of output directories and I don't care to build tools into
> the output dirs, in fact some of them flat out refuse to work that way
> (bpf comes to mind).
Going with that, then, I believe it is best to simply revert commit
9fc96c7c19df ("selftests: error out if kernel header files are not
yet built"). And then follow up with a series of (many) changes to
wean the various selftests off of the kernel headers.
I'll post the revert shortly.
thanks,
--
John Hubbard
NVIDIA